2023-09-30T22:07:22.922 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-30T22:07:22.922 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_dparmar@teuthology --name dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi --archive /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042 --description fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}} -- /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/orig.config.yaml 2023-09-30T22:07:22.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T22:07:23.059 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042 branch: fix-nfs-apply-err-reporting description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}} email: null first_in_suite: false job_id: '7408042' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: fix-nfs-apply-err-reporting ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62953 sha1: 6973b24e3d99fd494b753449d2ea12077599cbd6 owner: scheduled_dparmar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 seed: 2683 sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 sleep_before_teardown: 0 suite: fs:upgrade suite_branch: wip-62953 suite_path: /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa suite_relpath: qa suite_repo: https://github.com/dparmar18/ceph suite_sha1: 6973b24e3d99fd494b753449d2ea12077599cbd6 targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGWsMKYuwsdGXjp6h34gZzBBh7PMayUXXTdEL6+J6Wa8ik43W5zzZGraVOj6EMsemGb/p2oTMkpaEPWxgLu8C6s= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWwuWg1cgtHrdcSkqy8HMifSeAJRJaas2j9YYMDpI0uJcMS/XGWzDLabQwiCS8c2GHvWxSIgIcSkmi4OXt3cFU= tasks: - install: branch: pacific exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs - ceph-volume extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs extra_system_packages: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - print: '**** done installing pacific' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client pacific - print: '**** done ceph' - ceph-fuse: null - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: reef - print: '**** done install.upgrade the host' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph osd require-osd-release reef - ceph.healthy: null - print: '**** done ceph.restart' - ceph-fuse: client.0: mounted: false - print: '**** done unmount client.0' - install.upgrade: client.0: null - print: '**** done install.upgrade on client.0' - ceph.healthy: null - ceph-fuse: client.0: null - print: '**** done remount client' - cephfs_test_runner: modules: - tasks.cephfs.test_newops teuthology: branch: pacific fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end - if not is_fuse() then reject() end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-30_21:53:45 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-30T22:07:23.060 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa; will attempt to use it 2023-09-30T22:07:23.061 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks 2023-09-30T22:07:23.061 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-30T22:07:23.062 INFO:teuthology.task.internal:Checking packages... 2023-09-30T22:07:23.084 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d6eb91756d99e822eeed7c6b783104a6ad86054' 2023-09-30T22:07:23.084 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:07:23.084 INFO:teuthology.packaging:ref: None 2023-09-30T22:07:23.085 INFO:teuthology.packaging:tag: None 2023-09-30T22:07:23.085 INFO:teuthology.packaging:branch: fix-nfs-apply-err-reporting 2023-09-30T22:07:23.085 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:07:23.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=fix-nfs-apply-err-reporting 2023-09-30T22:07:23.302 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5795.g3d6eb917 2023-09-30T22:07:23.302 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-30T22:07:23.308 INFO:teuthology.task.internal:no buildpackages task found 2023-09-30T22:07:23.308 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-30T22:07:23.318 INFO:teuthology.task.internal:Saving configuration 2023-09-30T22:07:23.339 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-30T22:07:23.345 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:07:23.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:02:23.993557', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGWsMKYuwsdGXjp6h34gZzBBh7PMayUXXTdEL6+J6Wa8ik43W5zzZGraVOj6EMsemGb/p2oTMkpaEPWxgLu8C6s='} 2023-09-30T22:07:23.394 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:02:23.991688', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWwuWg1cgtHrdcSkqy8HMifSeAJRJaas2j9YYMDpI0uJcMS/XGWzDLabQwiCS8c2GHvWxSIgIcSkmi4OXt3cFU='} 2023-09-30T22:07:23.395 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-30T22:07:23.400 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2023-09-30T22:07:23.400 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['client.0'] 2023-09-30T22:07:23.400 INFO:teuthology.run_tasks:Running task console_log... 2023-09-30T22:07:23.466 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4271409f70>, signals=[15]) 2023-09-30T22:07:23.467 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-30T22:07:23.474 INFO:teuthology.task.internal:Opening connections... 2023-09-30T22:07:23.474 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:07:23.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:07:23.547 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2023-09-30T22:07:23.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:07:23.615 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-30T22:07:23.622 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-09-30T22:07:23.645 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-09-30T22:07:23.645 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-09-30T22:07:23.707 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T22:07:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T22:07:23.709 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-30T22:07:23.709 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-09-30T22:07:23.794 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-09-30T22:07:23.816 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-09-30T22:07:23.817 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-09-30T22:07:23.879 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2023-09-30T22:07:23.879 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8" 2023-09-30T22:07:23.879 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2023-09-30T22:07:23.879 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2023-09-30T22:07:23.879 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T22:07:23.880 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-30T22:07:23.881 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-09-30T22:07:23.906 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-30T22:07:23.914 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-30T22:07:23.970 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-30T22:07:23.971 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-09-30T22:07:23.973 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2023-09-30T22:07:23.994 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-30T22:07:24.000 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-30T22:07:24.000 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-09-30T22:07:24.034 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2023-09-30T22:07:24.075 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-30T22:07:24.129 INFO:teuthology.run_tasks:Running task kernel... 2023-09-30T22:07:24.144 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-30T22:07:24.144 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-30T22:07:24.145 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T22:07:24.145 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T22:07:24.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:07:24.166 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-09-30T22:07:24.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:07:24.168 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-09-30T22:07:24.228 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-489.el8.x86_64 2023-09-30T22:07:24.228 INFO:teuthology.task.kernel:Running kernel on smithi121: 4.18.0-489.el8.x86_64 2023-09-30T22:07:24.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2023-09-30T22:07:24.230 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-489.el8.x86_64 2023-09-30T22:07:24.230 INFO:teuthology.task.kernel:Running kernel on smithi032: 4.18.0-489.el8.x86_64 2023-09-30T22:07:24.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2023-09-30T22:07:28.678 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - AppStream 18 MB/s | 33 MB 00:01 2023-09-30T22:07:28.880 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - AppStream 17 MB/s | 33 MB 00:01 2023-09-30T22:07:36.452 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - BaseOS 27 MB/s | 48 MB 00:01 2023-09-30T22:07:37.864 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - BaseOS 19 MB/s | 48 MB 00:02 2023-09-30T22:07:43.610 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras 59 kB/s | 18 kB 00:00 2023-09-30T22:07:43.994 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras common packages 33 kB/s | 6.9 kB 00:00 2023-09-30T22:07:44.295 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-09-30T22:07:45.480 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras 58 kB/s | 18 kB 00:00 2023-09-30T22:07:46.131 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras common packages 15 kB/s | 6.9 kB 00:00 2023-09-30T22:07:46.441 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-09-30T22:07:46.692 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 16 MB 00:02 2023-09-30T22:07:47.583 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-09-30T22:07:50.555 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 317 kB/s | 24 kB 00:00 2023-09-30T22:07:51.633 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 495 kB/s | 24 kB 00:00 2023-09-30T22:07:54.194 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-30T22:07:54.194 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-30T22:07:54.470 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-09-30T22:07:54.471 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-09-30T22:07:54.471 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2023-09-30T22:07:54.471 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-09-30T22:07:54.471 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-09-30T22:07:54.471 INFO:teuthology.orchestra.run.smithi121.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:Install 3 Packages 2023-09-30T22:07:54.472 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:07:54.473 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 88 M 2023-09-30T22:07:54.473 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 96 M 2023-09-30T22:07:54.473 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-09-30T22:07:55.288 INFO:teuthology.orchestra.run.smithi121.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 16 MB/s | 10 MB 00:00 2023-09-30T22:07:55.391 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-30T22:07:55.392 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-30T22:07:55.645 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T22:07:55.645 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2023-09-30T22:07:55.646 INFO:teuthology.orchestra.run.smithi032.stdout:Install 3 Packages 2023-09-30T22:07:55.647 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:07:55.647 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 88 M 2023-09-30T22:07:55.647 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 96 M 2023-09-30T22:07:55.647 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-09-30T22:07:56.080 INFO:teuthology.orchestra.run.smithi121.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 25 MB/s | 35 MB 00:01 2023-09-30T22:07:56.402 INFO:teuthology.orchestra.run.smithi032.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 15 MB/s | 10 MB 00:00 2023-09-30T22:07:56.769 INFO:teuthology.orchestra.run.smithi121.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 20 MB/s | 43 MB 00:02 2023-09-30T22:07:56.769 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:07:56.769 INFO:teuthology.orchestra.run.smithi121.stdout:Total 38 MB/s | 88 MB 00:02 2023-09-30T22:07:57.442 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-09-30T22:07:57.660 INFO:teuthology.orchestra.run.smithi032.stdout:(2/3): kernel-core-4.18.0-514.el8.x86_64.rpm 22 MB/s | 43 MB 00:01 2023-09-30T22:07:57.893 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-09-30T22:07:57.893 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-09-30T22:07:57.989 INFO:teuthology.orchestra.run.smithi032.stdout:(3/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 16 MB/s | 35 MB 00:02 2023-09-30T22:07:57.989 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:07:57.989 INFO:teuthology.orchestra.run.smithi032.stdout:Total 38 MB/s | 88 MB 00:02 2023-09-30T22:07:58.689 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-09-30T22:07:59.128 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-09-30T22:07:59.150 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-09-30T22:07:59.151 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-09-30T22:07:59.153 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-09-30T22:08:00.476 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-09-30T22:08:00.502 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-09-30T22:08:01.950 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-09-30T22:08:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:08:03.434 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-09-30T22:08:03.872 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:08:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:08:06.422 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:08:07.706 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:08:08.205 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:08:09.800 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:08:10.104 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:11.829 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:08:12.141 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:31.612 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:31.636 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:31.975 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:31.975 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:08:31.975 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:08:32.671 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:32.672 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:08:32.672 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-09-30T22:08:32.672 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-09-30T22:08:32.672 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-09-30T22:08:32.672 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:08:32.672 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-09-30T22:08:32.825 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:08:33.038 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:08:33.038 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:08:33.038 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-09-30T22:08:33.038 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T22:08:33.038 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T22:08:33.039 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-09-30T22:08:33.039 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi121.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T22:08:33.039 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:08:33.246 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:08:33.246 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:08:33.246 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -qi grub2-tools 2023-09-30T22:08:33.279 INFO:teuthology.orchestra.run.smithi121.stdout:Name : grub2-tools 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Epoch : 1 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Version : 2.02 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Release : 129.el8 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Architecture: x86_64 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Group : System Environment/Base 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:Size : 9265586 2023-09-30T22:08:33.280 INFO:teuthology.orchestra.run.smithi121.stdout:License : GPLv3+ 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Relocations : (not relocatable) 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Packager : CentOS BuildSystem 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:Vendor : CentOS 2023-09-30T22:08:33.281 INFO:teuthology.orchestra.run.smithi121.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:Summary : Support tools for GRUB. 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:Description : 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:hardware devices. 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:08:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:This subpackage provides tools for support of all platforms. 2023-09-30T22:08:33.283 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-30T22:08:33.284 INFO:teuthology.task.kernel:Updating grub on smithi121 to boot 4.18.0-514.el8.x86_64 2023-09-30T22:08:33.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-30T22:08:35.102 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2023-09-30T22:08:35.354 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:35.379 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:35.746 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:08:35.747 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:08:36.539 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T22:08:36.716 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:08:36.990 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:08:36.991 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:08:36.991 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-09-30T22:08:36.991 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T22:08:36.991 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T22:08:36.991 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-30T22:08:36.991 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T22:08:36.991 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:08:37.143 INFO:teuthology.orchestra.run.smithi121.stderr:done 2023-09-30T22:08:37.145 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2023-09-30T22:08:37.173 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.wdSOtOBI8E 2023-09-30T22:08:37.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wdSOtOBI8E 2023-09-30T22:08:37.210 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:08:37.210 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:08:37.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2023-09-30T22:08:37.244 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.02 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 129.el8 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Group : System Environment/Base 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 9265586 2023-09-30T22:08:37.245 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Relocations : (not relocatable) 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : CentOS BuildSystem 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : CentOS 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2023-09-30T22:08:37.246 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:08:37.247 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-30T22:08:37.247 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-30T22:08:37.247 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-30T22:08:37.247 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2023-09-30T22:08:37.247 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:08:37.247 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2023-09-30T22:08:37.248 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-30T22:08:37.248 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 4.18.0-514.el8.x86_64 2023-09-30T22:08:37.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-30T22:08:37.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.wdSOtOBI8E 2023-09-30T22:08:37.364 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.wdSOtOBI8E is 6KB 2023-09-30T22:08:37.412 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.wdSOtOBI8E 2023-09-30T22:08:37.430 DEBUG:teuthology.orchestra.run.smithi121:> sudo /bin/ls /boot/loader/entries || true 2023-09-30T22:08:37.499 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-30T22:08:37.499 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-30T22:08:37.499 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-30T22:08:37.499 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-09-30T22:08:37.500 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-09-30T22:08:37.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo shutdown -r now 2023-09-30T22:08:38.523 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-09-30T22:08:40.222 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-09-30T22:08:40.224 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-09-30T22:08:40.261 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.Ixt2uVqVMH 2023-09-30T22:08:40.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ixt2uVqVMH 2023-09-30T22:08:40.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.Ixt2uVqVMH 2023-09-30T22:08:40.475 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.Ixt2uVqVMH is 6KB 2023-09-30T22:08:40.524 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.Ixt2uVqVMH 2023-09-30T22:08:40.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2023-09-30T22:08:40.610 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-30T22:08:40.610 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-30T22:08:40.610 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-30T22:08:40.610 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-09-30T22:08:40.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-09-30T22:08:40.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2023-09-30T22:09:07.718 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-30T22:09:07.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:09:07.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:09:10.826 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-30T22:09:10.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T22:09:10.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:09:26.138 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-09-30T22:09:29.210 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-09-30T22:09:35.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:09:35.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:09:38.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-09-30T22:09:38.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T22:09:38.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:09:50.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:09:50.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:09:56.858 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-09-30T22:10:08.870 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T22:10:08.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:10:08.894 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-09-30T22:10:09.148 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-09-30T22:10:09.413 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T22:10:09.413 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-09-30T22:10:09.413 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-09-30T22:10:09.433 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-514.el8.x86_64 2023-09-30T22:10:09.434 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-09-30T22:10:09.434 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T22:10:09.434 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-30T22:10:10.435 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-30T22:10:10.436 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T22:10:10.507 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2023-09-30T22:10:10.533 DEBUG:teuthology.parallel:result is None 2023-09-30T22:10:23.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:10:23.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:10:24.720 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-09-30T22:10:25.363 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:10:25.364 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-514.el8.x86_64"... 2023-09-30T22:10:25.364 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-09-30T22:10:25.381 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-514.el8.x86_64 2023-09-30T22:10:25.381 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-09-30T22:10:25.382 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T22:10:25.382 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-30T22:10:26.382 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2023-09-30T22:10:26.383 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T22:10:26.448 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2023-09-30T22:10:26.466 DEBUG:teuthology.parallel:result is None 2023-09-30T22:10:26.466 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-30T22:10:26.474 INFO:teuthology.task.internal:Creating test directory... 2023-09-30T22:10:26.474 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T22:10:26.478 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T22:10:26.500 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-30T22:10:26.507 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-30T22:10:26.520 INFO:teuthology.task.internal:Creating archive directory... 2023-09-30T22:10:26.521 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T22:10:26.543 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T22:10:26.582 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-30T22:10:26.589 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-30T22:10:26.590 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T22:10:26.614 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T22:10:26.647 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:10:26.657 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:10:26.662 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:10:26.675 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:10:26.676 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-30T22:10:26.683 INFO:teuthology.task.internal:Configuring sudo... 2023-09-30T22:10:26.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T22:10:26.686 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T22:10:26.726 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-30T22:10:26.737 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-30T22:10:26.737 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:10:26.760 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:10:26.785 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:10:26.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:10:26.929 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:10:27.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:10:27.071 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:10:27.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T22:10:27.146 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:10:27.192 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:10:27.249 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:10:27.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:10:27.375 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:10:27.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T22:10:27.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-09-30T22:10:27.445 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2023-09-30T22:10:27.516 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:10:27.540 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:10:27.975 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-30T22:10:27.983 INFO:teuthology.task.internal:Starting timer... 2023-09-30T22:10:27.983 INFO:teuthology.run_tasks:Running task pcp... 2023-09-30T22:10:28.007 INFO:teuthology.run_tasks:Running task selinux... 2023-09-30T22:10:28.017 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-30T22:10:28.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2023-09-30T22:10:28.118 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2023-09-30T22:10:28.120 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2023-09-30T22:10:28.204 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2023-09-30T22:10:28.206 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-30T22:10:28.206 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2023-09-30T22:10:28.241 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2023-09-30T22:10:28.242 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2023-09-30T22:10:28.262 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2023-09-30T22:10:28.262 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2023-09-30T22:10:28.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:10:28.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:10:28.317 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:10:28.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:10:28.345 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-30T22:10:28.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2023-09-30T22:10:28.392 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2023-09-30T22:10:28.418 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-30T22:10:28.428 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-30T22:10:28.433 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-30T22:10:28.601 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-30T22:10:28.616 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-30T22:10:28.617 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-30T22:18:34.024 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2023-09-30T22:18:34.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T22:18:34.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:18:34.098 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-09-30T22:18:34.173 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-09-30T22:18:34.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:18:34.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:18:34.244 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-09-30T22:18:34.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-09-30T22:18:34.314 INFO:teuthology.run_tasks:Running task clock... 2023-09-30T22:18:34.325 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-30T22:18:34.325 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T22:18:34.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:18:34.328 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T22:18:34.328 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:18:34.364 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T22:18:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T22:18:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T22:18:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T22:18:34.457 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2023-09-30T22:18:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2023-09-30T22:18:34.471 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2023-09-30T22:18:34.476 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2023-09-30T22:18:34.488 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T22:18:34.489 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T22:18:34.502 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T22:18:34.504 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T22:18:34.578 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2023-09-30T22:18:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:18:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2023-09-30T22:18:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-09-30T22:18:34.589 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:18:34.589 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-09-30T22:18:34.589 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.589 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.589 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.590 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:18:34.590 INFO:teuthology.run_tasks:Running task install... 2023-09-30T22:18:34.606 DEBUG:teuthology.task.install:project ceph 2023-09-30T22:18:34.606 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}} 2023-09-30T22:18:34.607 DEBUG:teuthology.task.install:config {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-09-30T22:18:34.607 INFO:teuthology.task.install:Using flavor: default 2023-09-30T22:18:34.613 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T22:18:34.613 INFO:teuthology.task.install:extra packages: ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2023-09-30T22:18:34.613 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-09-30T22:18:34.613 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:18:34.613 INFO:teuthology.packaging:ref: None 2023-09-30T22:18:34.614 INFO:teuthology.packaging:tag: None 2023-09-30T22:18:34.614 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:18:34.614 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:18:34.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:18:34.616 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-09-30T22:18:34.616 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:18:34.616 INFO:teuthology.packaging:ref: None 2023-09-30T22:18:34.616 INFO:teuthology.packaging:tag: None 2023-09-30T22:18:34.616 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:18:34.616 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:18:34.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:18:34.823 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-09-30T22:18:34.823 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-09-30T22:18:34.988 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:18:34.988 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:18:34.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:18:35.024 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-09-30T22:18:35.024 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:18:35.025 INFO:teuthology.packaging:ref: None 2023-09-30T22:18:35.025 INFO:teuthology.packaging:tag: None 2023-09-30T22:18:35.025 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:18:35.025 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:18:35.025 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:18:35.028 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-09-30T22:18:35.028 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-09-30T22:18:35.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:18:35.189 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:18:35.207 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:18:35.208 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:18:35.208 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:18:35.222 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2023-09-30T22:18:35.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2023-09-30T22:18:35.237 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-09-30T22:18:35.238 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:18:35.238 INFO:teuthology.packaging:ref: None 2023-09-30T22:18:35.238 INFO:teuthology.packaging:tag: None 2023-09-30T22:18:35.238 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:18:35.238 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:18:35.238 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:18:35.307 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:18:35.385 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:18:35.453 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2023-09-30T22:18:35.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2023-09-30T22:18:35.538 INFO:teuthology.orchestra.run.smithi032.stdout:76 files removed 2023-09-30T22:18:35.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-09-30T22:18:35.716 INFO:teuthology.orchestra.run.smithi121.stdout:76 files removed 2023-09-30T22:18:35.745 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-09-30T22:18:36.599 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 169 kB/s | 76 kB 00:00 2023-09-30T22:18:36.694 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 117 kB/s | 76 kB 00:00 2023-09-30T22:18:36.975 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 45 kB/s | 15 kB 00:00 2023-09-30T22:18:37.370 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 24 kB/s | 15 kB 00:00 2023-09-30T22:18:37.634 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 2.3 kB/s | 1.4 kB 00:00 2023-09-30T22:18:37.870 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 3.1 kB/s | 1.4 kB 00:00 2023-09-30T22:18:39.826 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - AppStream 15 MB/s | 33 MB 00:02 2023-09-30T22:18:39.946 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - AppStream 16 MB/s | 33 MB 00:02 2023-09-30T22:18:49.535 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 48 MB 00:03 2023-09-30T22:18:50.521 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 48 MB 00:03 2023-09-30T22:18:56.751 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras 58 kB/s | 18 kB 00:00 2023-09-30T22:18:57.318 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras common packages 18 kB/s | 6.9 kB 00:00 2023-09-30T22:18:58.389 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras 113 kB/s | 18 kB 00:00 2023-09-30T22:18:58.594 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - PowerTools 5.8 MB/s | 6.5 MB 00:01 2023-09-30T22:18:59.123 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 6.9 kB 00:00 2023-09-30T22:18:59.996 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-09-30T22:19:00.424 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - PowerTools 5.7 MB/s | 6.5 MB 00:01 2023-09-30T22:19:01.898 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-09-30T22:19:02.025 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-09-30T22:19:03.210 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-09-30T22:19:05.738 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 359 kB/s | 24 kB 00:00 2023-09-30T22:19:07.718 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 135 kB/s | 24 kB 00:00 2023-09-30T22:19:09.991 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:19:09.992 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-09-30T22:19:10.018 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-09-30T22:19:10.020 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-09-30T22:19:10.229 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-09-30T22:19:10.240 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2023-09-30T22:19:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-09-30T22:19:10.240 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-09-30T22:19:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-09-30T22:19:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:19:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-09-30T22:19:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-09-30T22:19:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-09-30T22:19:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-09-30T22:19:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T22:19:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-30T22:19:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T22:19:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-30T22:19:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout:Install 108 Packages 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 14 Packages 2023-09-30T22:19:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:19:10.253 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 200 M 2023-09-30T22:19:10.253 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-09-30T22:19:10.923 INFO:teuthology.orchestra.run.smithi121.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 48 kB/s | 6.4 kB 00:00 2023-09-30T22:19:11.173 INFO:teuthology.orchestra.run.smithi121.stdout:(2/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 3.6 MB/s | 910 kB 00:00 2023-09-30T22:19:11.282 INFO:teuthology.orchestra.run.smithi121.stdout:(3/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 12 MB/s | 6.1 MB 00:00 2023-09-30T22:19:11.398 INFO:teuthology.orchestra.run.smithi121.stdout:(4/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 14 MB/s | 1.6 MB 00:00 2023-09-30T22:19:11.474 INFO:teuthology.orchestra.run.smithi121.stdout:(5/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 8.1 MB/s | 2.4 MB 00:00 2023-09-30T22:19:11.707 INFO:teuthology.orchestra.run.smithi121.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 16 MB/s | 5.0 MB 00:00 2023-09-30T22:19:12.374 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:19:12.375 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:19:12.402 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-09-30T22:19:12.403 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-09-30T22:19:12.403 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-09-30T22:19:12.405 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-09-30T22:19:12.483 INFO:teuthology.orchestra.run.smithi121.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 15 MB/s | 12 MB 00:00 2023-09-30T22:19:12.566 INFO:teuthology.orchestra.run.smithi121.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 308 kB/s | 26 kB 00:00 2023-09-30T22:19:12.637 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T22:19:12.650 INFO:teuthology.orchestra.run.smithi032.stdout:==================================================================================================== 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout:==================================================================================================== 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-09-30T22:19:12.651 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-09-30T22:19:12.652 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-09-30T22:19:12.653 INFO:teuthology.orchestra.run.smithi032.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-09-30T22:19:12.654 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-09-30T22:19:12.655 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T22:19:12.656 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T22:19:12.657 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T22:19:12.658 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-30T22:19:12.659 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-30T22:19:12.660 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-30T22:19:12.661 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:==================================================================================================== 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:Install 108 Packages 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 14 Packages 2023-09-30T22:19:12.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:19:12.663 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 200 M 2023-09-30T22:19:12.663 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-09-30T22:19:13.228 INFO:teuthology.orchestra.run.smithi032.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 20 kB/s | 6.4 kB 00:00 2023-09-30T22:19:13.408 INFO:teuthology.orchestra.run.smithi121.stdout:(9/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86_ 9.5 MB/s | 18 MB 00:01 2023-09-30T22:19:13.458 INFO:teuthology.orchestra.run.smithi121.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 511 kB/s | 25 kB 00:00 2023-09-30T22:19:13.536 INFO:teuthology.orchestra.run.smithi032.stdout:(2/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 2.9 MB/s | 910 kB 00:00 2023-09-30T22:19:13.850 INFO:teuthology.orchestra.run.smithi121.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 2.1 MB/s | 824 kB 00:00 2023-09-30T22:19:13.951 INFO:teuthology.orchestra.run.smithi121.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 1.7 MB/s | 174 kB 00:00 2023-09-30T22:19:14.042 INFO:teuthology.orchestra.run.smithi121.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 1.4 MB/s | 131 kB 00:00 2023-09-30T22:19:14.218 INFO:teuthology.orchestra.run.smithi121.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 2.8 MB/s | 493 kB 00:00 2023-09-30T22:19:14.503 INFO:teuthology.orchestra.run.smithi032.stdout:(3/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 2.5 MB/s | 2.4 MB 00:00 2023-09-30T22:19:14.827 INFO:teuthology.orchestra.run.smithi121.stdout:(15/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 6.2 MB/s | 3.8 MB 00:00 2023-09-30T22:19:14.910 INFO:teuthology.orchestra.run.smithi121.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 540 kB/s | 45 kB 00:00 2023-09-30T22:19:14.994 INFO:teuthology.orchestra.run.smithi121.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 1.2 MB/s | 99 kB 00:00 2023-09-30T22:19:15.077 INFO:teuthology.orchestra.run.smithi121.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 2.5 MB/s | 215 kB 00:00 2023-09-30T22:19:15.078 INFO:teuthology.orchestra.run.smithi032.stdout:(4/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 2.8 MB/s | 6.1 MB 00:02 2023-09-30T22:19:15.129 INFO:teuthology.orchestra.run.smithi032.stdout:(5/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 2.6 MB/s | 1.6 MB 00:00 2023-09-30T22:19:15.192 INFO:teuthology.orchestra.run.smithi121.stdout:(19/122): python3-rados-16.2.14-44.geeb13ac9.el 3.3 MB/s | 390 kB 00:00 2023-09-30T22:19:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:(20/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 3.5 MB/s | 369 kB 00:00 2023-09-30T22:19:15.386 INFO:teuthology.orchestra.run.smithi121.stdout:(21/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.2 MB/s | 114 kB 00:00 2023-09-30T22:19:15.478 INFO:teuthology.orchestra.run.smithi121.stdout:(22/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 964 kB/s | 88 kB 00:00 2023-09-30T22:19:16.170 INFO:teuthology.orchestra.run.smithi121.stdout:(23/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 5.3 MB/s | 3.7 MB 00:00 2023-09-30T22:19:16.254 INFO:teuthology.orchestra.run.smithi121.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 2.0 MB/s | 173 kB 00:00 2023-09-30T22:19:16.354 INFO:teuthology.orchestra.run.smithi121.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 244 kB/s | 24 kB 00:00 2023-09-30T22:19:16.779 INFO:teuthology.orchestra.run.smithi032.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 3.0 MB/s | 5.0 MB 00:01 2023-09-30T22:19:17.138 INFO:teuthology.orchestra.run.smithi121.stdout:(26/122): ceph-test-16.2.14-44.geeb13ac9.el8.x8 11 MB/s | 52 MB 00:04 2023-09-30T22:19:17.205 INFO:teuthology.orchestra.run.smithi121.stdout:(27/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 3.7 MB/s | 3.1 MB 00:00 2023-09-30T22:19:17.439 INFO:teuthology.orchestra.run.smithi121.stdout:(28/122): ceph-common-16.2.14-44.geeb13ac9.el8. 3.6 MB/s | 24 MB 00:06 2023-09-30T22:19:17.464 INFO:teuthology.orchestra.run.smithi121.stdout:(29/122): ceph-mgr-modules-core-16.2.14-44.geeb 946 kB/s | 244 kB 00:00 2023-09-30T22:19:17.514 INFO:teuthology.orchestra.run.smithi121.stdout:(30/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 365 kB/s | 27 kB 00:00 2023-09-30T22:19:17.531 INFO:teuthology.orchestra.run.smithi121.stdout:(31/122): ceph-prometheus-alerts-16.2.14-44.gee 222 kB/s | 15 kB 00:00 2023-09-30T22:19:17.923 INFO:teuthology.orchestra.run.smithi121.stdout:(32/122): ceph-mgr-diskprediction-local-16.2.14 11 MB/s | 8.6 MB 00:00 2023-09-30T22:19:18.073 INFO:teuthology.orchestra.run.smithi121.stdout:(33/122): flex-2.6.1-9.el8.x86_64.rpm 590 kB/s | 320 kB 00:00 2023-09-30T22:19:18.182 INFO:teuthology.orchestra.run.smithi121.stdout:(34/122): bison-3.0.4-10.el8.x86_64.rpm 1.0 MB/s | 688 kB 00:00 2023-09-30T22:19:18.274 INFO:teuthology.orchestra.run.smithi121.stdout:(35/122): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2023-09-30T22:19:18.308 INFO:teuthology.orchestra.run.smithi121.stdout:(36/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-09-30T22:19:18.383 INFO:teuthology.orchestra.run.smithi121.stdout:(37/122): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-09-30T22:19:18.409 INFO:teuthology.orchestra.run.smithi121.stdout:(38/122): jq-1.6-7.el8.x86_64.rpm 418 kB/s | 202 kB 00:00 2023-09-30T22:19:18.471 INFO:teuthology.orchestra.run.smithi032.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 6.8 MB/s | 12 MB 00:01 2023-09-30T22:19:18.513 INFO:teuthology.orchestra.run.smithi032.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 620 kB/s | 26 kB 00:00 2023-09-30T22:19:18.526 INFO:teuthology.orchestra.run.smithi121.stdout:(39/122): python3-jsonpatch-1.21-2.el8.noarch.r 228 kB/s | 27 kB 00:00 2023-09-30T22:19:18.635 INFO:teuthology.orchestra.run.smithi121.stdout:(40/122): python3-jsonpointer-1.10-11.el8.noarc 184 kB/s | 20 kB 00:00 2023-09-30T22:19:18.727 INFO:teuthology.orchestra.run.smithi121.stdout:(41/122): openblas-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-09-30T22:19:18.793 INFO:teuthology.orchestra.run.smithi121.stdout:(42/122): openblas-threads-0.3.15-4.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-09-30T22:19:18.818 INFO:teuthology.orchestra.run.smithi121.stdout:(43/122): python3-mako-1.0.6-14.el8.noarch.rpm 853 kB/s | 157 kB 00:00 2023-09-30T22:19:18.845 INFO:teuthology.orchestra.run.smithi121.stdout:(44/122): python3-markupsafe-0.23-19.el8.x86_64 331 kB/s | 39 kB 00:00 2023-09-30T22:19:18.881 INFO:teuthology.orchestra.run.smithi032.stdout:(9/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86_ 4.9 MB/s | 18 MB 00:03 2023-09-30T22:19:18.931 INFO:teuthology.orchestra.run.smithi032.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 515 kB/s | 25 kB 00:00 2023-09-30T22:19:18.978 INFO:teuthology.orchestra.run.smithi121.stdout:(45/122): python3-prettytable-0.7.2-14.el8.noar 330 kB/s | 44 kB 00:00 2023-09-30T22:19:19.003 INFO:teuthology.orchestra.run.smithi121.stdout:(46/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.2 MB/s | 225 kB 00:00 2023-09-30T22:19:19.074 INFO:teuthology.orchestra.run.smithi032.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 5.7 MB/s | 824 kB 00:00 2023-09-30T22:19:19.103 INFO:teuthology.orchestra.run.smithi121.stdout:(47/122): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-09-30T22:19:19.120 INFO:teuthology.orchestra.run.smithi121.stdout:(48/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 727 kB/s | 103 kB 00:00 2023-09-30T22:19:19.123 INFO:teuthology.orchestra.run.smithi032.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 3.4 MB/s | 174 kB 00:00 2023-09-30T22:19:19.146 INFO:teuthology.orchestra.run.smithi121.stdout:(49/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 887 kB/s | 126 kB 00:00 2023-09-30T22:19:19.182 INFO:teuthology.orchestra.run.smithi032.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 2.2 MB/s | 131 kB 00:00 2023-09-30T22:19:19.204 INFO:teuthology.orchestra.run.smithi121.stdout:(50/122): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-09-30T22:19:19.229 INFO:teuthology.orchestra.run.smithi121.stdout:(51/122): python3-pytz-2017.2-11.el8.noarch.rpm 522 kB/s | 57 kB 00:00 2023-09-30T22:19:19.281 INFO:teuthology.orchestra.run.smithi032.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 4.8 MB/s | 493 kB 00:00 2023-09-30T22:19:19.313 INFO:teuthology.orchestra.run.smithi121.stdout:(52/122): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2023-09-30T22:19:19.338 INFO:teuthology.orchestra.run.smithi121.stdout:(53/122): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-09-30T22:19:19.432 INFO:teuthology.orchestra.run.smithi121.stdout:(54/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 592 kB/s | 71 kB 00:00 2023-09-30T22:19:19.473 INFO:teuthology.orchestra.run.smithi121.stdout:(55/122): elfutils-libelf-devel-0.189-3.el8.x86 461 kB/s | 62 kB 00:00 2023-09-30T22:19:19.531 INFO:teuthology.orchestra.run.smithi121.stdout:(56/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 823 kB/s | 48 kB 00:00 2023-09-30T22:19:19.589 INFO:teuthology.orchestra.run.smithi121.stdout:(57/122): fuse-2.9.7-17.el8.x86_64.rpm 531 kB/s | 83 kB 00:00 2023-09-30T22:19:19.639 INFO:teuthology.orchestra.run.smithi121.stdout:(58/122): libcom_err-devel-1.45.6-5.el8.x86_64. 776 kB/s | 39 kB 00:00 2023-09-30T22:19:19.665 INFO:teuthology.orchestra.run.smithi121.stdout:(59/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 4.1 MB/s | 561 kB 00:00 2023-09-30T22:19:19.715 INFO:teuthology.orchestra.run.smithi032.stdout:(15/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 8.7 MB/s | 3.8 MB 00:00 2023-09-30T22:19:19.724 INFO:teuthology.orchestra.run.smithi121.stdout:(60/122): libquadmath-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 172 kB 00:00 2023-09-30T22:19:19.765 INFO:teuthology.orchestra.run.smithi032.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 907 kB/s | 45 kB 00:00 2023-09-30T22:19:19.774 INFO:teuthology.orchestra.run.smithi121.stdout:(61/122): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2023-09-30T22:19:19.791 INFO:teuthology.orchestra.run.smithi121.stdout:(62/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-09-30T22:19:19.841 INFO:teuthology.orchestra.run.smithi121.stdout:(63/122): libselinux-devel-2.9-8.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-09-30T22:19:19.849 INFO:teuthology.orchestra.run.smithi032.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 1.2 MB/s | 99 kB 00:00 2023-09-30T22:19:19.866 INFO:teuthology.orchestra.run.smithi121.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-09-30T22:19:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 4.2 MB/s | 215 kB 00:00 2023-09-30T22:19:19.917 INFO:teuthology.orchestra.run.smithi121.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 240 kB/s | 18 kB 00:00 2023-09-30T22:19:19.942 INFO:teuthology.orchestra.run.smithi121.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 582 kB/s | 44 kB 00:00 2023-09-30T22:19:20.084 INFO:teuthology.orchestra.run.smithi121.stdout:(67/122): python3-scipy-1.0.0-21.module_el8.5.0 15 MB/s | 14 MB 00:00 2023-09-30T22:19:20.118 INFO:teuthology.orchestra.run.smithi121.stdout:(68/122): pcre2-devel-10.32-3.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2023-09-30T22:19:20.141 INFO:teuthology.orchestra.run.smithi032.stdout:(19/122): ceph-common-16.2.14-44.geeb13ac9.el8. 3.3 MB/s | 24 MB 00:07 2023-09-30T22:19:20.175 INFO:teuthology.orchestra.run.smithi032.stdout:(20/122): python3-rados-16.2.14-44.geeb13ac9.el 1.4 MB/s | 390 kB 00:00 2023-09-30T22:19:20.201 INFO:teuthology.orchestra.run.smithi121.stdout:(69/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-09-30T22:19:20.242 INFO:teuthology.orchestra.run.smithi032.stdout:(21/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.7 MB/s | 114 kB 00:00 2023-09-30T22:19:20.276 INFO:teuthology.orchestra.run.smithi121.stdout:(70/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.2 MB/s | 249 kB 00:00 2023-09-30T22:19:20.292 INFO:teuthology.orchestra.run.smithi032.stdout:(22/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 1.7 MB/s | 88 kB 00:00 2023-09-30T22:19:20.302 INFO:teuthology.orchestra.run.smithi121.stdout:(71/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2023-09-30T22:19:20.317 INFO:teuthology.orchestra.run.smithi032.stdout:(23/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 2.1 MB/s | 369 kB 00:00 2023-09-30T22:19:20.372 INFO:teuthology.orchestra.run.smithi121.stdout:(72/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 7.0 MB/s | 3.2 MB 00:00 2023-09-30T22:19:20.384 INFO:teuthology.orchestra.run.smithi032.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 2.5 MB/s | 173 kB 00:00 2023-09-30T22:19:20.402 INFO:teuthology.orchestra.run.smithi121.stdout:(73/122): python3-jwt-1.6.1-2.el8.noarch.rpm 424 kB/s | 43 kB 00:00 2023-09-30T22:19:20.427 INFO:teuthology.orchestra.run.smithi121.stdout:(74/122): python3-oauthlib-2.1.0-1.el8.noarch.r 2.7 MB/s | 155 kB 00:00 2023-09-30T22:19:20.451 INFO:teuthology.orchestra.run.smithi032.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 365 kB/s | 24 kB 00:00 2023-09-30T22:19:20.453 INFO:teuthology.orchestra.run.smithi121.stdout:(75/122): python3-cryptography-3.2.1-6.el8.x86_ 3.6 MB/s | 641 kB 00:00 2023-09-30T22:19:20.478 INFO:teuthology.orchestra.run.smithi121.stdout:(76/122): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-30T22:19:20.503 INFO:teuthology.orchestra.run.smithi121.stdout:(77/122): python3-requests-oauthlib-1.0.0-1.el8 563 kB/s | 43 kB 00:00 2023-09-30T22:19:20.529 INFO:teuthology.orchestra.run.smithi121.stdout:(78/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 767 kB/s | 58 kB 00:00 2023-09-30T22:19:20.736 INFO:teuthology.orchestra.run.smithi032.stdout:(26/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 8.3 MB/s | 3.7 MB 00:00 2023-09-30T22:19:20.870 INFO:teuthology.orchestra.run.smithi121.stdout:(79/122): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-09-30T22:19:20.979 INFO:teuthology.orchestra.run.smithi121.stdout:(80/122): leveldb-1.22-1.el8.x86_64.rpm 381 kB/s | 181 kB 00:00 2023-09-30T22:19:21.013 INFO:teuthology.orchestra.run.smithi121.stdout:(81/122): gperftools-libs-2.7-9.el8.x86_64.rpm 573 kB/s | 306 kB 00:00 2023-09-30T22:19:21.038 INFO:teuthology.orchestra.run.smithi121.stdout:(82/122): libunwind-1.3.1-3.el8.x86_64.rpm 449 kB/s | 75 kB 00:00 2023-09-30T22:19:21.069 INFO:teuthology.orchestra.run.smithi032.stdout:(27/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 5.1 MB/s | 3.1 MB 00:00 2023-09-30T22:19:21.088 INFO:teuthology.orchestra.run.smithi121.stdout:(83/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 401 kB/s | 44 kB 00:00 2023-09-30T22:19:21.139 INFO:teuthology.orchestra.run.smithi121.stdout:(84/122): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-09-30T22:19:21.164 INFO:teuthology.orchestra.run.smithi121.stdout:(85/122): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2023-09-30T22:19:21.177 INFO:teuthology.orchestra.run.smithi032.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 2.2 MB/s | 244 kB 00:00 2023-09-30T22:19:21.189 INFO:teuthology.orchestra.run.smithi121.stdout:(86/122): python3-certifi-2018.10.15-7.el8.noar 162 kB/s | 16 kB 00:00 2023-09-30T22:19:21.244 INFO:teuthology.orchestra.run.smithi032.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 412 kB/s | 27 kB 00:00 2023-09-30T22:19:21.264 INFO:teuthology.orchestra.run.smithi121.stdout:(87/122): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-09-30T22:19:21.311 INFO:teuthology.orchestra.run.smithi032.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 223 kB/s | 15 kB 00:00 2023-09-30T22:19:21.323 INFO:teuthology.orchestra.run.smithi121.stdout:(88/122): python3-google-auth-1.1.1-10.el8.noar 608 kB/s | 81 kB 00:00 2023-09-30T22:19:21.382 INFO:teuthology.orchestra.run.smithi121.stdout:(89/122): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-09-30T22:19:21.407 INFO:teuthology.orchestra.run.smithi121.stdout:(90/122): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-09-30T22:19:21.432 INFO:teuthology.orchestra.run.smithi121.stdout:(91/122): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-09-30T22:19:21.533 INFO:teuthology.orchestra.run.smithi121.stdout:(92/122): python3-logutils-0.3.5-11.el8.noarch. 387 kB/s | 49 kB 00:00 2023-09-30T22:19:21.558 INFO:teuthology.orchestra.run.smithi121.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 467 kB/s | 59 kB 00:00 2023-09-30T22:19:21.667 INFO:teuthology.orchestra.run.smithi121.stdout:(94/122): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-09-30T22:19:21.678 INFO:teuthology.orchestra.run.smithi032.stdout:(31/122): ceph-mgr-diskprediction-local-16.2.14 9.1 MB/s | 8.6 MB 00:00 2023-09-30T22:19:21.700 INFO:teuthology.orchestra.run.smithi121.stdout:(95/122): python3-kubernetes-11.0.0-6.el8.noarc 3.7 MB/s | 1.2 MB 00:00 2023-09-30T22:19:21.725 INFO:teuthology.orchestra.run.smithi121.stdout:(96/122): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-09-30T22:19:21.784 INFO:teuthology.orchestra.run.smithi121.stdout:(97/122): python3-repoze-lru-0.7-6.el8.noarch.r 287 kB/s | 34 kB 00:00 2023-09-30T22:19:21.826 INFO:teuthology.orchestra.run.smithi121.stdout:(98/122): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-09-30T22:19:21.851 INFO:teuthology.orchestra.run.smithi121.stdout:(99/122): python3-rsa-4.9-2.el8.noarch.rpm 511 kB/s | 64 kB 00:00 2023-09-30T22:19:21.876 INFO:teuthology.orchestra.run.smithi121.stdout:(100/122): python3-simplegeneric-0.8.1-17.el8.n 203 kB/s | 19 kB 00:00 2023-09-30T22:19:21.927 INFO:teuthology.orchestra.run.smithi121.stdout:(101/122): python3-singledispatch-3.4.0.3-18.el 238 kB/s | 24 kB 00:00 2023-09-30T22:19:21.952 INFO:teuthology.orchestra.run.smithi121.stdout:(102/122): python3-tempora-1.14.1-5.el8.noarch. 291 kB/s | 29 kB 00:00 2023-09-30T22:19:21.962 INFO:teuthology.orchestra.run.smithi032.stdout:(32/122): bison-3.0.4-10.el8.x86_64.rpm 1.0 MB/s | 688 kB 00:00 2023-09-30T22:19:21.969 INFO:teuthology.orchestra.run.smithi121.stdout:(103/122): python3-trustme-0.6.0-4.el8.noarch.r 298 kB/s | 27 kB 00:00 2023-09-30T22:19:22.036 INFO:teuthology.orchestra.run.smithi121.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 2.3 MB/s | 255 kB 00:00 2023-09-30T22:19:22.078 INFO:teuthology.orchestra.run.smithi121.stdout:(105/122): python3-webob-1.8.5-1.el8.1.noarch.r 2.0 MB/s | 251 kB 00:00 2023-09-30T22:19:22.087 INFO:teuthology.orchestra.run.smithi032.stdout:(33/122): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-09-30T22:19:22.095 INFO:teuthology.orchestra.run.smithi121.stdout:(106/122): python3-websocket-client-0.56.0-5.el 488 kB/s | 61 kB 00:00 2023-09-30T22:19:22.137 INFO:teuthology.orchestra.run.smithi121.stdout:(107/122): python3-webtest-2.0.33-1.el8.noarch. 848 kB/s | 85 kB 00:00 2023-09-30T22:19:22.187 INFO:teuthology.orchestra.run.smithi121.stdout:(108/122): python3-zc-lockfile-2.0-2.el8.noarch 211 kB/s | 23 kB 00:00 2023-09-30T22:19:22.237 INFO:teuthology.orchestra.run.smithi032.stdout:(34/122): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2023-09-30T22:19:22.271 INFO:teuthology.orchestra.run.smithi032.stdout:(35/122): flex-2.6.1-9.el8.x86_64.rpm 540 kB/s | 320 kB 00:00 2023-09-30T22:19:22.338 INFO:teuthology.orchestra.run.smithi121.stdout:(109/122): NetworkManager-initscripts-updown-1. 957 kB/s | 143 kB 00:00 2023-09-30T22:19:22.363 INFO:teuthology.orchestra.run.smithi032.stdout:(36/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-09-30T22:19:22.388 INFO:teuthology.orchestra.run.smithi121.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 1.5 MB/s | 76 kB 00:00 2023-09-30T22:19:22.422 INFO:teuthology.orchestra.run.smithi032.stdout:(37/122): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-09-30T22:19:22.463 INFO:teuthology.orchestra.run.smithi121.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 3.0 MB/s | 232 kB 00:00 2023-09-30T22:19:22.530 INFO:teuthology.orchestra.run.smithi121.stdout:(112/122): elfutils-libs-0.189-3.el8.x86_64.rpm 4.4 MB/s | 303 kB 00:00 2023-09-30T22:19:22.606 INFO:teuthology.orchestra.run.smithi121.stdout:(113/122): librados2-16.2.14-44.geeb13ac9.el8.x 7.3 MB/s | 3.7 MB 00:00 2023-09-30T22:19:22.664 INFO:teuthology.orchestra.run.smithi121.stdout:(114/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 7.9 MB/s | 4.2 MB 00:00 2023-09-30T22:19:22.723 INFO:teuthology.orchestra.run.smithi121.stdout:(115/122): iproute-6.2.0-2.el8.x86_64.rpm 4.5 MB/s | 880 kB 00:00 2023-09-30T22:19:22.772 INFO:teuthology.orchestra.run.smithi032.stdout:(38/122): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-09-30T22:19:22.890 INFO:teuthology.orchestra.run.smithi121.stdout:(116/122): libmount-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 236 kB 00:00 2023-09-30T22:19:22.902 INFO:teuthology.orchestra.run.smithi032.stdout:(39/122): python3-jsonpatch-1.21-2.el8.noarch.r 205 kB/s | 27 kB 00:00 2023-09-30T22:19:22.915 INFO:teuthology.orchestra.run.smithi121.stdout:(117/122): libblkid-2.32.1-43.el8.x86_64.rpm 713 kB/s | 220 kB 00:00 2023-09-30T22:19:22.940 INFO:teuthology.orchestra.run.smithi121.stdout:(118/122): libfdisk-2.32.1-43.el8.x86_64.rpm 920 kB/s | 254 kB 00:00 2023-09-30T22:19:22.965 INFO:teuthology.orchestra.run.smithi121.stdout:(119/122): libsmartcols-2.32.1-43.el8.x86_64.rp 2.3 MB/s | 179 kB 00:00 2023-09-30T22:19:22.989 INFO:teuthology.orchestra.run.smithi032.stdout:(40/122): openblas-threads-0.3.15-4.el8.x86_64. 8.6 MB/s | 4.9 MB 00:00 2023-09-30T22:19:22.990 INFO:teuthology.orchestra.run.smithi121.stdout:(120/122): libuuid-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 99 kB 00:00 2023-09-30T22:19:23.015 INFO:teuthology.orchestra.run.smithi032.stdout:(41/122): python3-jsonpointer-1.10-11.el8.noarc 178 kB/s | 20 kB 00:00 2023-09-30T22:19:23.016 INFO:teuthology.orchestra.run.smithi121.stdout:(121/122): util-linux-user-2.32.1-43.el8.x86_64 2.0 MB/s | 103 kB 00:00 2023-09-30T22:19:23.099 INFO:teuthology.orchestra.run.smithi032.stdout:(42/122): python3-mako-1.0.6-14.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-09-30T22:19:23.124 INFO:teuthology.orchestra.run.smithi032.stdout:(43/122): python3-markupsafe-0.23-19.el8.x86_64 357 kB/s | 39 kB 00:00 2023-09-30T22:19:23.208 INFO:teuthology.orchestra.run.smithi121.stdout:(122/122): util-linux-2.32.1-43.el8.x86_64.rpm 9.3 MB/s | 2.5 MB 00:00 2023-09-30T22:19:23.211 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:19:23.211 INFO:teuthology.orchestra.run.smithi121.stdout:Total 15 MB/s | 200 MB 00:12 2023-09-30T22:19:23.232 INFO:teuthology.orchestra.run.smithi032.stdout:(44/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2023-09-30T22:19:23.341 INFO:teuthology.orchestra.run.smithi032.stdout:(45/122): python3-prettytable-0.7.2-14.el8.noar 405 kB/s | 44 kB 00:00 2023-09-30T22:19:23.416 INFO:teuthology.orchestra.run.smithi032.stdout:(46/122): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-09-30T22:19:23.441 INFO:teuthology.orchestra.run.smithi032.stdout:(47/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-09-30T22:19:23.509 INFO:teuthology.orchestra.run.smithi032.stdout:(48/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.3 MB/s | 126 kB 00:00 2023-09-30T22:19:23.542 INFO:teuthology.orchestra.run.smithi032.stdout:(49/122): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-09-30T22:19:23.951 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-09-30T22:19:24.064 INFO:teuthology.orchestra.run.smithi032.stdout:(50/122): ceph-test-16.2.14-44.geeb13ac9.el8.x8 9.3 MB/s | 52 MB 00:05 2023-09-30T22:19:24.093 INFO:teuthology.orchestra.run.smithi032.stdout:(51/122): python3-pytz-2017.2-11.el8.noarch.rpm 98 kB/s | 57 kB 00:00 2023-09-30T22:19:24.177 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-09-30T22:19:24.181 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-09-30T22:19:24.210 INFO:teuthology.orchestra.run.smithi032.stdout:(52/122): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2023-09-30T22:19:24.310 INFO:teuthology.orchestra.run.smithi032.stdout:(53/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 708 kB/s | 71 kB 00:00 2023-09-30T22:19:24.477 INFO:teuthology.orchestra.run.smithi032.stdout:(54/122): elfutils-libelf-devel-0.189-3.el8.x86 371 kB/s | 62 kB 00:00 2023-09-30T22:19:24.561 INFO:teuthology.orchestra.run.smithi032.stdout:(55/122): fuse-2.9.7-17.el8.x86_64.rpm 994 kB/s | 83 kB 00:00 2023-09-30T22:19:24.611 INFO:teuthology.orchestra.run.smithi032.stdout:(56/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 963 kB/s | 48 kB 00:00 2023-09-30T22:19:24.686 INFO:teuthology.orchestra.run.smithi032.stdout:(57/122): python3-werkzeug-0.12.2-4.el8.noarch. 736 kB/s | 457 kB 00:00 2023-09-30T22:19:24.828 INFO:teuthology.orchestra.run.smithi032.stdout:(58/122): python3-scipy-1.0.0-21.module_el8.5.0 11 MB/s | 14 MB 00:01 2023-09-30T22:19:24.895 INFO:teuthology.orchestra.run.smithi032.stdout:(59/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.9 MB/s | 561 kB 00:00 2023-09-30T22:19:24.920 INFO:teuthology.orchestra.run.smithi032.stdout:(60/122): libcom_err-devel-1.45.6-5.el8.x86_64. 166 kB/s | 39 kB 00:00 2023-09-30T22:19:24.945 INFO:teuthology.orchestra.run.smithi032.stdout:(61/122): libquadmath-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 172 kB 00:00 2023-09-30T22:19:24.971 INFO:teuthology.orchestra.run.smithi032.stdout:(62/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 933 kB/s | 47 kB 00:00 2023-09-30T22:19:24.996 INFO:teuthology.orchestra.run.smithi032.stdout:(63/122): libselinux-devel-2.9-8.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2023-09-30T22:19:25.021 INFO:teuthology.orchestra.run.smithi032.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-30T22:19:25.047 INFO:teuthology.orchestra.run.smithi032.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 359 kB/s | 18 kB 00:00 2023-09-30T22:19:25.072 INFO:teuthology.orchestra.run.smithi032.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 874 kB/s | 44 kB 00:00 2023-09-30T22:19:25.105 INFO:teuthology.orchestra.run.smithi032.stdout:(67/122): libgfortran-8.5.0-20.el8.x86_64.rpm 2.3 MB/s | 645 kB 00:00 2023-09-30T22:19:25.164 INFO:teuthology.orchestra.run.smithi032.stdout:(68/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-09-30T22:19:25.189 INFO:teuthology.orchestra.run.smithi032.stdout:(69/122): pcre2-devel-10.32-3.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-09-30T22:19:25.257 INFO:teuthology.orchestra.run.smithi032.stdout:(70/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-09-30T22:19:25.282 INFO:teuthology.orchestra.run.smithi032.stdout:(71/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.6 MB/s | 249 kB 00:00 2023-09-30T22:19:25.332 INFO:teuthology.orchestra.run.smithi032.stdout:(72/122): python3-jwt-1.6.1-2.el8.noarch.rpm 853 kB/s | 43 kB 00:00 2023-09-30T22:19:25.357 INFO:teuthology.orchestra.run.smithi032.stdout:(73/122): python3-cryptography-3.2.1-6.el8.x86_ 6.2 MB/s | 641 kB 00:00 2023-09-30T22:19:25.408 INFO:teuthology.orchestra.run.smithi032.stdout:(74/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 8.8 MB/s | 3.2 MB 00:00 2023-09-30T22:19:25.433 INFO:teuthology.orchestra.run.smithi032.stdout:(75/122): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-09-30T22:19:25.458 INFO:teuthology.orchestra.run.smithi032.stdout:(76/122): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2023-09-30T22:19:25.483 INFO:teuthology.orchestra.run.smithi032.stdout:(77/122): python3-requests-oauthlib-1.0.0-1.el8 565 kB/s | 43 kB 00:00 2023-09-30T22:19:25.495 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-09-30T22:19:25.499 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-09-30T22:19:25.509 INFO:teuthology.orchestra.run.smithi032.stdout:(78/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 769 kB/s | 58 kB 00:00 2023-09-30T22:19:25.909 INFO:teuthology.orchestra.run.smithi032.stdout:(79/122): liboath-2.6.2-3.el8.x86_64.rpm 147 kB/s | 59 kB 00:00 2023-09-30T22:19:25.959 INFO:teuthology.orchestra.run.smithi032.stdout:(80/122): leveldb-1.22-1.el8.x86_64.rpm 381 kB/s | 181 kB 00:00 2023-09-30T22:19:26.076 INFO:teuthology.orchestra.run.smithi032.stdout:(81/122): gperftools-libs-2.7-9.el8.x86_64.rpm 496 kB/s | 306 kB 00:00 2023-09-30T22:19:26.110 INFO:teuthology.orchestra.run.smithi032.stdout:(82/122): libunwind-1.3.1-3.el8.x86_64.rpm 375 kB/s | 75 kB 00:00 2023-09-30T22:19:26.135 INFO:teuthology.orchestra.run.smithi032.stdout:(83/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 249 kB/s | 44 kB 00:00 2023-09-30T22:19:26.260 INFO:teuthology.orchestra.run.smithi032.stdout:(84/122): python3-cachetools-3.1.1-4.el8.noarch 220 kB/s | 33 kB 00:00 2023-09-30T22:19:26.277 INFO:teuthology.orchestra.run.smithi032.stdout:(85/122): python3-certifi-2018.10.15-7.el8.noar 115 kB/s | 16 kB 00:00 2023-09-30T22:19:26.302 INFO:teuthology.orchestra.run.smithi032.stdout:(86/122): python3-beautifulsoup4-4.6.3-2.el8.1. 819 kB/s | 185 kB 00:00 2023-09-30T22:19:26.569 INFO:teuthology.orchestra.run.smithi032.stdout:(87/122): python3-google-auth-1.1.1-10.el8.noar 305 kB/s | 81 kB 00:00 2023-09-30T22:19:26.595 INFO:teuthology.orchestra.run.smithi032.stdout:(88/122): python3-cheroot-8.5.2-1.el8.noarch.rp 517 kB/s | 173 kB 00:00 2023-09-30T22:19:26.745 INFO:teuthology.orchestra.run.smithi032.stdout:(89/122): python3-jaraco-6.2-6.el8.noarch.rpm 63 kB/s | 11 kB 00:00 2023-09-30T22:19:26.803 INFO:teuthology.orchestra.run.smithi032.stdout:(90/122): python3-cherrypy-18.4.0-1.el8.noarch. 731 kB/s | 384 kB 00:00 2023-09-30T22:19:26.821 INFO:teuthology.orchestra.run.smithi032.stdout:(91/122): python3-jaraco-functools-2.0-4.el8.no 78 kB/s | 18 kB 00:00 2023-09-30T22:19:26.971 INFO:teuthology.orchestra.run.smithi032.stdout:(92/122): python3-logutils-0.3.5-11.el8.noarch. 292 kB/s | 49 kB 00:00 2023-09-30T22:19:27.029 INFO:teuthology.orchestra.run.smithi032.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 282 kB/s | 59 kB 00:00 2023-09-30T22:19:27.105 INFO:teuthology.orchestra.run.smithi032.stdout:(94/122): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-09-30T22:19:27.130 INFO:teuthology.orchestra.run.smithi032.stdout:(95/122): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-09-30T22:19:27.180 INFO:teuthology.orchestra.run.smithi032.stdout:(96/122): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-09-30T22:19:27.205 INFO:teuthology.orchestra.run.smithi032.stdout:(97/122): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2023-09-30T22:19:27.272 INFO:teuthology.orchestra.run.smithi032.stdout:(98/122): python3-rsa-4.9-2.el8.noarch.rpm 700 kB/s | 64 kB 00:00 2023-09-30T22:19:27.306 INFO:teuthology.orchestra.run.smithi032.stdout:(99/122): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-09-30T22:19:27.331 INFO:teuthology.orchestra.run.smithi032.stdout:(100/122): python3-simplegeneric-0.8.1-17.el8.n 149 kB/s | 19 kB 00:00 2023-09-30T22:19:27.432 INFO:teuthology.orchestra.run.smithi032.stdout:(101/122): python3-singledispatch-3.4.0.3-18.el 150 kB/s | 24 kB 00:00 2023-09-30T22:19:27.457 INFO:teuthology.orchestra.run.smithi032.stdout:(102/122): python3-trustme-0.6.0-4.el8.noarch.r 219 kB/s | 27 kB 00:00 2023-09-30T22:19:27.482 INFO:teuthology.orchestra.run.smithi032.stdout:(103/122): python3-tempora-1.14.1-5.el8.noarch. 166 kB/s | 29 kB 00:00 2023-09-30T22:19:27.624 INFO:teuthology.orchestra.run.smithi032.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 1.3 MB/s | 255 kB 00:00 2023-09-30T22:19:27.658 INFO:teuthology.orchestra.run.smithi032.stdout:(105/122): python3-webob-1.8.5-1.el8.1.noarch.r 1.2 MB/s | 251 kB 00:00 2023-09-30T22:19:27.691 INFO:teuthology.orchestra.run.smithi032.stdout:(106/122): python3-websocket-client-0.56.0-5.el 293 kB/s | 61 kB 00:00 2023-09-30T22:19:27.842 INFO:teuthology.orchestra.run.smithi032.stdout:(107/122): python3-webtest-2.0.33-1.el8.noarch. 393 kB/s | 85 kB 00:00 2023-09-30T22:19:27.867 INFO:teuthology.orchestra.run.smithi032.stdout:(108/122): python3-zc-lockfile-2.0-2.el8.noarch 110 kB/s | 23 kB 00:00 2023-09-30T22:19:28.001 INFO:teuthology.orchestra.run.smithi032.stdout:(109/122): NetworkManager-initscripts-updown-1. 1.1 MB/s | 143 kB 00:00 2023-09-30T22:19:28.042 INFO:teuthology.orchestra.run.smithi032.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 1.8 MB/s | 76 kB 00:00 2023-09-30T22:19:28.102 INFO:teuthology.orchestra.run.smithi032.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 3.9 MB/s | 232 kB 00:00 2023-09-30T22:19:28.160 INFO:teuthology.orchestra.run.smithi032.stdout:(112/122): elfutils-libs-0.189-3.el8.x86_64.rpm 5.1 MB/s | 303 kB 00:00 2023-09-30T22:19:28.218 INFO:teuthology.orchestra.run.smithi032.stdout:(113/122): librados2-16.2.14-44.geeb13ac9.el8.x 7.1 MB/s | 3.7 MB 00:00 2023-09-30T22:19:28.343 INFO:teuthology.orchestra.run.smithi032.stdout:(114/122): iproute-6.2.0-2.el8.x86_64.rpm 4.7 MB/s | 880 kB 00:00 2023-09-30T22:19:28.420 INFO:teuthology.orchestra.run.smithi032.stdout:(115/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 7.2 MB/s | 4.2 MB 00:00 2023-09-30T22:19:28.444 INFO:teuthology.orchestra.run.smithi032.stdout:(116/122): libblkid-2.32.1-43.el8.x86_64.rpm 978 kB/s | 220 kB 00:00 2023-09-30T22:19:28.470 INFO:teuthology.orchestra.run.smithi032.stdout:(117/122): libfdisk-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 254 kB 00:00 2023-09-30T22:19:28.495 INFO:teuthology.orchestra.run.smithi032.stdout:(118/122): libsmartcols-2.32.1-43.el8.x86_64.rp 3.5 MB/s | 179 kB 00:00 2023-09-30T22:19:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:(119/122): libuuid-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2023-09-30T22:19:28.570 INFO:teuthology.orchestra.run.smithi032.stdout:(120/122): util-linux-user-2.32.1-43.el8.x86_64 2.0 MB/s | 103 kB 00:00 2023-09-30T22:19:28.596 INFO:teuthology.orchestra.run.smithi032.stdout:(121/122): libmount-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 236 kB 00:00 2023-09-30T22:19:28.763 INFO:teuthology.orchestra.run.smithi032.stdout:(122/122): util-linux-2.32.1-43.el8.x86_64.rpm 9.3 MB/s | 2.5 MB 00:00 2023-09-30T22:19:28.766 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:19:28.766 INFO:teuthology.orchestra.run.smithi032.stdout:Total 12 MB/s | 200 MB 00:16 2023-09-30T22:19:29.572 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-09-30T22:19:29.827 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-09-30T22:19:29.827 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-09-30T22:19:30.423 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-09-30T22:19:30.462 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-09-30T22:19:30.670 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-09-30T22:19:30.874 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-09-30T22:19:31.011 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:19:31.075 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:19:31.253 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:19:31.299 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-09-30T22:19:31.302 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-09-30T22:19:31.333 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:19:31.533 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-09-30T22:19:31.754 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-09-30T22:19:31.936 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:19:32.330 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:19:32.552 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:19:32.994 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:19:33.178 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:19:33.292 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:19:33.602 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:19:33.700 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:19:33.934 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:19:34.127 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:19:34.511 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:19:34.669 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:19:34.727 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:19:34.957 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-09-30T22:19:35.167 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-09-30T22:19:35.294 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:19:35.435 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:19:35.577 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:19:35.683 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:19:35.900 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-09-30T22:19:36.053 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-09-30T22:19:36.177 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:19:36.288 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:19:36.443 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:19:36.458 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-09-30T22:19:36.503 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-09-30T22:19:36.829 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-09-30T22:19:37.027 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:19:37.052 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-09-30T22:19:37.197 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:19:37.252 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:19:37.327 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:19:37.481 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:19:37.584 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:19:37.811 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-09-30T22:19:37.853 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:19:38.010 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-09-30T22:19:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:19:38.180 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:19:38.603 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:19:38.694 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:19:38.797 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:19:39.110 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-09-30T22:19:39.245 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:19:39.346 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-09-30T22:19:39.430 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:19:39.557 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:19:39.582 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-09-30T22:19:39.747 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-09-30T22:19:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:19:39.878 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:19:39.964 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-09-30T22:19:40.125 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:19:40.776 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:19:40.776 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:19:40.793 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:19:40.938 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:19:41.012 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:19:41.279 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-09-30T22:19:41.630 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-09-30T22:19:41.763 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:19:41.863 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:19:41.959 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:19:42.022 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:19:42.153 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:19:42.271 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-09-30T22:19:42.396 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-09-30T22:19:42.484 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-09-30T22:19:42.553 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-09-30T22:19:42.663 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:19:42.696 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-09-30T22:19:42.829 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:19:42.969 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-09-30T22:19:42.971 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:19:43.146 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-09-30T22:19:43.409 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-09-30T22:19:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-09-30T22:19:43.633 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:19:43.821 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:19:43.825 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-09-30T22:19:44.025 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:19:44.091 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:19:44.334 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:19:44.405 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:19:44.521 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:19:44.567 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:19:44.730 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:19:45.075 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:19:45.359 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:19:45.374 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:19:45.520 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:19:45.560 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-09-30T22:19:45.824 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-09-30T22:19:46.022 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-09-30T22:19:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-09-30T22:19:46.300 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-09-30T22:19:46.304 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-09-30T22:19:46.477 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-09-30T22:19:46.503 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-09-30T22:19:46.699 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-09-30T22:19:46.713 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-09-30T22:19:46.917 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-09-30T22:19:46.948 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-09-30T22:19:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-09-30T22:19:47.262 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:19:47.280 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-09-30T22:19:47.458 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-09-30T22:19:47.650 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-09-30T22:19:47.867 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-09-30T22:19:48.131 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-09-30T22:19:48.317 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-09-30T22:19:48.558 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-09-30T22:19:48.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-09-30T22:19:49.032 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-09-30T22:19:49.272 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:19:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-09-30T22:19:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-09-30T22:19:49.673 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-09-30T22:19:49.839 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-09-30T22:19:49.975 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-09-30T22:19:50.200 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-09-30T22:19:50.326 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-09-30T22:19:50.437 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-09-30T22:19:50.562 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-09-30T22:19:50.661 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-09-30T22:19:50.760 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-09-30T22:19:51.045 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-09-30T22:19:51.217 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-09-30T22:19:51.438 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-09-30T22:19:51.645 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:19:51.753 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-09-30T22:19:51.767 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:19:51.937 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:19:52.010 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:19:52.023 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-09-30T22:19:52.228 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:19:52.247 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-09-30T22:19:52.499 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-09-30T22:19:52.734 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-09-30T22:19:52.790 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:19:52.952 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-09-30T22:19:53.096 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:19:53.114 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:19:53.230 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-09-30T22:19:53.336 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-09-30T22:19:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-09-30T22:19:53.584 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-09-30T22:19:53.593 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-09-30T22:19:53.779 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-09-30T22:19:53.845 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-09-30T22:19:54.075 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-09-30T22:19:54.126 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-09-30T22:19:54.250 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-09-30T22:19:54.493 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-09-30T22:19:54.593 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-09-30T22:19:54.726 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-09-30T22:19:54.760 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-09-30T22:19:54.932 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-09-30T22:19:55.006 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-09-30T22:19:55.093 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-09-30T22:19:55.191 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-09-30T22:19:55.296 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-09-30T22:19:55.391 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-09-30T22:19:55.464 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-09-30T22:19:55.620 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-09-30T22:19:55.709 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-09-30T22:19:55.847 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-09-30T22:19:55.950 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-09-30T22:19:56.019 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-09-30T22:19:56.193 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-09-30T22:19:56.212 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-09-30T22:19:56.455 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-09-30T22:19:56.564 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-09-30T22:19:56.642 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-09-30T22:19:56.840 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-09-30T22:19:56.880 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-09-30T22:19:56.998 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-09-30T22:19:57.106 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-09-30T22:19:57.522 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-09-30T22:19:57.841 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-09-30T22:19:58.094 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-09-30T22:19:58.346 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-09-30T22:19:58.597 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-09-30T22:19:59.531 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:19:59.764 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:19:59.785 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-09-30T22:20:00.080 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-09-30T22:20:00.322 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-09-30T22:20:00.465 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:20:00.614 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-09-30T22:20:00.764 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:20:00.871 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:20:00.900 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-09-30T22:20:01.030 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:20:01.092 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-09-30T22:20:01.439 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-09-30T22:20:01.658 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-09-30T22:20:01.797 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-09-30T22:20:01.972 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-09-30T22:20:02.251 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-09-30T22:20:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-09-30T22:20:02.618 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-09-30T22:20:02.894 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-09-30T22:20:03.125 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-09-30T22:20:03.332 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-09-30T22:20:03.612 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-09-30T22:20:03.787 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-09-30T22:20:03.991 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-09-30T22:20:04.167 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-09-30T22:20:04.500 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-09-30T22:20:04.796 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-09-30T22:20:04.961 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-09-30T22:20:07.739 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:20:07.952 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:20:08.744 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:20:09.058 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:20:09.177 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:20:09.360 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:20:19.547 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:20:19.547 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2023-09-30T22:20:19.547 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2023-09-30T22:20:19.547 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-09-30T22:20:19.547 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /run 2023-09-30T22:20:19.548 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2023-09-30T22:20:19.548 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2023-09-30T22:20:19.548 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2023-09-30T22:20:19.548 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2023-09-30T22:20:19.548 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-09-30T22:20:19.548 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:19.974 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:20:20.403 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:20:20.576 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:20:22.059 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:20:22.292 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:20:22.326 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:20:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-09-30T22:20:22.900 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-09-30T22:20:23.215 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-09-30T22:20:23.433 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-09-30T22:20:23.677 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-09-30T22:20:23.841 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:20:23.920 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:20:24.153 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-09-30T22:20:24.273 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-09-30T22:20:25.306 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-09-30T22:20:25.583 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:20:26.625 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:20:26.832 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:20:26.921 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:20:27.300 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-09-30T22:20:27.500 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:20:27.525 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:20:27.724 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:20:27.735 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:20:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:20:30.440 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /run 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2023-09-30T22:20:30.441 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:30.910 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:20:31.389 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:20:31.565 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:20:32.383 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-09-30T22:20:33.202 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:20:33.407 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:20:33.448 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:20:33.646 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:20:33.712 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-09-30T22:20:33.923 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:20:34.060 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-09-30T22:20:34.326 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:20:34.388 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-09-30T22:20:34.506 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:20:34.599 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:20:34.607 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-09-30T22:20:34.862 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-09-30T22:20:35.006 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:20:35.096 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:20:35.301 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-09-30T22:20:35.423 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-09-30T22:20:35.490 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-09-30T22:20:35.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-09-30T22:20:35.990 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-09-30T22:20:36.201 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-09-30T22:20:36.373 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-09-30T22:20:36.570 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-09-30T22:20:36.602 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-09-30T22:20:36.878 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:20:36.891 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-09-30T22:20:37.166 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 119/136 2023-09-30T22:20:37.331 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:20:37.397 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:20:37.630 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:20:37.714 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:20:37.915 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:20:37.927 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:20:38.030 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:20:38.140 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-09-30T22:20:38.214 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:20:38.309 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-09-30T22:20:38.309 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:20:38.317 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:20:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:20:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:20:38.699 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:20:38.700 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:20:38.701 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-09-30T22:20:38.912 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:20:38.941 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:20:38.944 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:20:39.131 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:20:39.146 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:20:39.169 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-09-30T22:20:39.347 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-09-30T22:20:39.598 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:20:39.598 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:20:39.849 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:20:39.849 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:20:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:20:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:20:40.315 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:20:40.316 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:20:40.549 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:20:40.549 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:20:40.750 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:20:40.973 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:20:41.539 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:20:41.539 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-09-30T22:20:41.539 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-09-30T22:20:41.539 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-09-30T22:20:41.539 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-09-30T22:20:41.539 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-09-30T22:20:41.540 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-09-30T22:20:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-09-30T22:20:41.542 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-09-30T22:20:41.542 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-09-30T22:20:41.551 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-09-30T22:20:41.552 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-09-30T22:20:41.553 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-09-30T22:20:41.554 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-09-30T22:20:41.555 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-09-30T22:20:41.556 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-09-30T22:20:41.557 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-09-30T22:20:41.558 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 121/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:20:41.559 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-09-30T22:20:41.560 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:20:41.560 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-09-30T22:20:43.587 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: iproute-6.2.0-2.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: libblkid-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: libmount-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: util-linux-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.588 INFO:teuthology.orchestra.run.smithi121.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.589 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.590 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.7-17.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: jq-1.6-7.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: leveldb-1.22-1.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.591 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T22:20:43.592 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-30T22:20:43.593 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.594 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T22:20:43.595 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T22:20:43.596 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T22:20:43.597 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T22:20:43.598 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T22:20:43.599 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T22:20:43.600 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-09-30T22:20:43.600 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:43.600 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-09-30T22:20:43.797 DEBUG:teuthology.parallel:result is None 2023-09-30T22:20:44.264 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-09-30T22:20:45.601 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:20:45.828 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:20:46.269 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:20:46.497 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:20:46.603 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:20:47.615 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-09-30T22:20:47.894 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-09-30T22:20:48.124 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-09-30T22:20:48.334 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-09-30T22:20:48.489 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-09-30T22:20:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-09-30T22:20:49.009 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-09-30T22:20:49.299 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 119/136 2023-09-30T22:20:49.438 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:20:49.504 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:20:49.754 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:20:49.846 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:20:50.088 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:20:50.103 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:20:50.290 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-09-30T22:20:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-09-30T22:20:50.454 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:20:50.715 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:20:50.722 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:20:50.833 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:20:51.120 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:20:51.298 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-09-30T22:20:51.520 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-09-30T22:20:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:20:51.740 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:20:51.974 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:20:51.974 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:20:52.149 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:20:52.149 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:20:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:20:52.382 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:20:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:20:52.632 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:20:52.816 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:20:53.040 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:20:53.677 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:20:53.678 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-09-30T22:20:53.679 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-09-30T22:20:53.680 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-09-30T22:20:53.681 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-09-30T22:20:53.682 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-09-30T22:20:53.683 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-09-30T22:20:53.684 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-09-30T22:20:53.685 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-09-30T22:20:53.686 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:20:53.687 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 121/136 2023-09-30T22:20:53.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:20:53.689 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-09-30T22:20:53.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:20:53.690 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-09-30T22:20:55.774 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-09-30T22:20:55.774 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: iproute-6.2.0-2.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: libblkid-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: libmount-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.775 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.776 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:20:55.777 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.7-17.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-7.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T22:20:55.778 INFO:teuthology.orchestra.run.smithi032.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: leveldb-1.22-1.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-30T22:20:55.779 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T22:20:55.780 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-09-30T22:20:55.781 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.782 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T22:20:55.783 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T22:20:55.784 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T22:20:55.785 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T22:20:55.786 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T22:20:55.787 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T22:20:55.788 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T22:20:55.789 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-09-30T22:20:55.789 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:55.789 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T22:20:55.975 DEBUG:teuthology.parallel:result is None 2023-09-30T22:20:55.976 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:20:55.976 INFO:teuthology.packaging:ref: None 2023-09-30T22:20:55.976 INFO:teuthology.packaging:tag: None 2023-09-30T22:20:55.976 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:20:55.976 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:20:55.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:20:56.192 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:20:56.223 INFO:teuthology.orchestra.run.smithi032.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:20:56.223 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:20:56.223 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-09-30T22:20:56.223 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:20:56.224 INFO:teuthology.packaging:ref: None 2023-09-30T22:20:56.224 INFO:teuthology.packaging:tag: None 2023-09-30T22:20:56.224 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:20:56.224 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:20:56.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:20:56.442 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:20:56.471 INFO:teuthology.orchestra.run.smithi121.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:20:56.471 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:20:56.471 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-09-30T22:20:56.471 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-30T22:20:56.472 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:56.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T22:20:56.511 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:20:56.512 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T22:20:56.547 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-30T22:20:56.548 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:56.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T22:20:56.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T22:20:56.661 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:20:56.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T22:20:56.692 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T22:20:56.761 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-30T22:20:56.761 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:56.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T22:20:56.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T22:20:56.870 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:20:56.870 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T22:20:56.900 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T22:20:56.969 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-30T22:20:56.970 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:56.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T22:20:57.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T22:20:57.081 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:20:57.082 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T22:20:57.111 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T22:20:57.184 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:20:57.193 INFO:teuthology.task.print:**** done installing pacific 2023-09-30T22:20:57.193 INFO:teuthology.run_tasks:Running task ceph... 2023-09-30T22:20:57.255 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-30T22:20:57.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2023-09-30T22:20:57.257 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2023-09-30T22:20:57.289 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-30T22:20:57.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T22:20:57.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T22:20:57.364 INFO:tasks.ceph:Creating extra log directories... 2023-09-30T22:20:57.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T22:20:57.408 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T22:20:57.454 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-30T22:20:57.454 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-30T22:20:57.455 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042', 'branch': 'fix-nfs-apply-err-reporting', 'description': 'fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}}', 'email': None, 'first_in_suite': False, 'job_id': '7408042', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'fix-nfs-apply-err-reporting'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62953', 'sha1': '6973b24e3d99fd494b753449d2ea12077599cbd6'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0']], 'seed': 2683, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'sleep_before_teardown': 0, 'suite': 'fs:upgrade', 'suite_branch': 'wip-62953', 'suite_path': '/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/dparmar18/ceph', 'suite_sha1': '6973b24e3d99fd494b753449d2ea12077599cbd6', 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGWsMKYuwsdGXjp6h34gZzBBh7PMayUXXTdEL6+J6Wa8ik43W5zzZGraVOj6EMsemGb/p2oTMkpaEPWxgLu8C6s=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWwuWg1cgtHrdcSkqy8HMifSeAJRJaas2j9YYMDpI0uJcMS/XGWzDLabQwiCS8c2GHvWxSIgIcSkmi4OXt3cFU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, {'print': '**** done installing pacific'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client pacific']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'reef'}}}, {'print': '**** done install.upgrade the host'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done', 'ceph osd require-osd-release reef']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'ceph-fuse': {'client.0': {'mounted': False}}}, {'print': '**** done unmount client.0'}, {'install.upgrade': {'client.0': None}}, {'print': '**** done install.upgrade on client.0'}, {'ceph.healthy': None}, {'ceph-fuse': {'client.0': None}}, {'print': '**** done remount client'}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_newops']}}], 'teuthology': {'branch': 'pacific', 'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end", 'if not is_fuse() then reject() end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-30_21:53:45', 'tube': 'smithi', 'user': 'dparmar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-30T22:20:57.455 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T22:20:57.484 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T22:20:57.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T22:20:57.546 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T22:20:57.578 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:57.578 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T22:20:57.641 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T22:20:57.641 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-09-30T22:20:57.704 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T22:20:57.704 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:20:57.704 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 396338 Links: 1 2023-09-30T22:20:57.704 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:20:57.704 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:20:57.704 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 22:20:53.265467500 +0000 2023-09-30T22:20:57.705 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 22:17:52.649213757 +0000 2023-09-30T22:20:57.705 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 22:17:52.649213757 +0000 2023-09-30T22:20:57.705 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T22:20:57.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T22:20:57.778 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T22:20:57.779 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T22:20:57.779 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.00012482 s, 4.1 MB/s 2023-09-30T22:20:57.780 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T22:20:57.845 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-09-30T22:20:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T22:20:57.907 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:20:57.907 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 386525 Links: 1 2023-09-30T22:20:57.907 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:20:57.907 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:20:57.907 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 22:20:53.265467500 +0000 2023-09-30T22:20:57.908 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 22:17:53.208202884 +0000 2023-09-30T22:20:57.908 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 22:17:53.208202884 +0000 2023-09-30T22:20:57.908 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T22:20:57.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T22:20:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T22:20:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T22:20:57.981 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000119771 s, 4.3 MB/s 2023-09-30T22:20:57.982 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T22:20:58.047 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-09-30T22:20:58.110 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 395766 Links: 1 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 22:20:53.265467500 +0000 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 22:17:53.787191623 +0000 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 22:17:53.787191623 +0000 2023-09-30T22:20:58.111 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T22:20:58.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T22:20:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T22:20:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T22:20:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000113304 s, 4.5 MB/s 2023-09-30T22:20:58.185 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T22:20:58.250 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 396541 Links: 1 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-09-30 22:20:53.265467500 +0000 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-09-30 22:17:54.363180420 +0000 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-09-30 22:17:54.363180420 +0000 2023-09-30T22:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-09-30T22:20:58.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T22:20:58.386 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-09-30T22:20:58.386 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-09-30T22:20:58.387 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000104718 s, 4.9 MB/s 2023-09-30T22:20:58.388 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T22:20:58.460 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:20:58.460 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-09-30T22:20:58.460 INFO:tasks.ceph:Generating config... 2023-09-30T22:20:58.461 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-09-30T22:20:58.461 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-09-30T22:20:58.461 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2023-09-30T22:20:58.461 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2023-09-30T22:20:58.461 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2023-09-30T22:20:58.461 INFO:tasks.ceph:[client] client mount timeout = 600 2023-09-30T22:20:58.462 INFO:tasks.ceph:[client] debug client = 20 2023-09-30T22:20:58.462 INFO:tasks.ceph:[client] debug ms = 1 2023-09-30T22:20:58.462 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-09-30T22:20:58.462 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] debug mds = 20 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] debug ms = 1 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] mds debug frag = True 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-09-30T22:20:58.462 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-09-30T22:20:58.463 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2023-09-30T22:20:58.463 INFO:tasks.ceph:[osd] bdev async discard = True 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-30T22:20:58.464 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-30T22:20:58.465 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-30T22:20:58.465 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-30T22:20:58.465 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-09-30T22:20:58.465 INFO:tasks.ceph:Setting up mon.a... 2023-09-30T22:20:58.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-30T22:20:58.737 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2023-09-30T22:20:58.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-30T22:20:58.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:20:58.825 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]'), ('mon.c', '[v2:172.21.15.32:3302,v1:172.21.15.32:6791]')] 2023-09-30T22:20:58.826 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-09-30T22:20:58.827 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:58.827 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T22:20:58.890 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --addv b '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]' --addv c '[v2:172.21.15.32:3302,v1:172.21.15.32:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:20:58.985 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:20:58.985 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid 9e20f926-ec7d-45ec-8e27-e5e820815322 2023-09-30T22:20:58.985 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2023-09-30T22:20:58.985 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 9e20f926-ec7d-45ec-8e27-e5e820815322 2023-09-30T22:20:58.985 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2023-09-30T22:20:58.986062+0000 2023-09-30T22:20:58.985 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-09-30T22:20:58.986062+0000 2023-09-30T22:20:58.986 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 0 (unknown) 2023-09-30T22:20:58.986 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2023-09-30T22:20:58.986 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2023-09-30T22:20:58.986 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] mon.b 2023-09-30T22:20:58.986 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] mon.c 2023-09-30T22:20:58.986 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-30T22:20:58.987 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T22:20:59.008 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9e20f926-ec7d-45ec-8e27-e5e820815322... 2023-09-30T22:20:59.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T22:20:59.051 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T22:20:59.104 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.105 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T22:20:59.106 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2023-09-30T22:20:59.107 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.108 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]" 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on pool no app = False 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: ms bind msgr2 = False 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore warn on legacy statfs = False 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore warn on no per pool omap = False 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn min per osd = 0 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = 9e20f926-ec7d-45ec-8e27-e5e820815322 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.109 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.110 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.111 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore allocator = bitmap 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2023-09-30T22:20:59.112 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 1/20 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 1/20 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 4/10 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2023-09-30T22:20:59.113 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2023-09-30T22:20:59.114 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:20:59.115 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T22:20:59.116 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: mon op complaint time = 120 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = False 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-09-30T22:20:59.117 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: client mount timeout = 600 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2023-09-30T22:20:59.118 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout:[mds] 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds = 20 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds balancer = 20 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug frag = True 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug scatterstat = True 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: mds op complaint time = 180 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: mds verify scatter = True 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2023-09-30T22:20:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2023-09-30T22:20:59.120 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2023-09-30T22:20:59.121 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-09-30T22:20:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2023-09-30T22:20:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T22:20:59.121 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2023-09-30T22:20:59.122 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.123 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T22:20:59.124 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.125 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790],[v2:172.21.15.32:3302,v1:172.21.15.32:6791]" 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on pool no app = False 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: ms bind msgr2 = False 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore warn on legacy statfs = False 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore warn on no per pool omap = False 2023-09-30T22:20:59.126 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn min per osd = 0 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 9e20f926-ec7d-45ec-8e27-e5e820815322 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2023-09-30T22:20:59.127 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.128 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.129 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2023-09-30T22:20:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2023-09-30T22:20:59.131 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2023-09-30T22:20:59.132 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-09-30T22:20:59.133 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.134 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = False 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2023-09-30T22:20:59.135 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2023-09-30T22:20:59.136 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-09-30T22:20:59.137 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-09-30T22:20:59.138 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-30T22:20:59.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-30T22:20:59.228 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-30T22:20:59.229 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:59.229 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-30T22:20:59.250 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:59.250 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-30T22:20:59.312 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:20:59.312 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:59.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T22:20:59.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:20:59.399 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:20:59.399 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:20:59.463 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2023-09-30T22:20:59.463 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:20:59.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T22:20:59.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:20:59.506 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:20:59.507 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:20:59.567 INFO:tasks.ceph:Setting up mon nodes... 2023-09-30T22:20:59.567 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-30T22:20:59.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T22:20:59.630 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T22:20:59.632 INFO:tasks.ceph:Setting up mds nodes... 2023-09-30T22:20:59.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-30T22:20:59.692 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-30T22:20:59.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-30T22:20:59.727 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-30T22:20:59.727 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-30T22:20:59.846 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-30T22:20:59.860 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-30T22:20:59.860 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-09-30T22:20:59.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-30T22:20:59.895 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:20:59.895 INFO:tasks.ceph:role: osd.0 2023-09-30T22:20:59.895 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:20:59.895 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T22:20:59.970 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:20:59.970 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:20:59.971 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:20:59.993 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-09-30T22:20:59.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-09-30T22:20:59.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-30T22:21:00.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-30T22:21:00.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-30T22:21:00.198 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:21:00.198 INFO:tasks.ceph:role: osd.1 2023-09-30T22:21:00.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:21:00.199 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T22:21:00.274 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:21:00.274 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:21:00.275 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:21:00.297 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-09-30T22:21:00.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-09-30T22:21:00.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-30T22:21:00.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-30T22:21:00.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-30T22:21:00.499 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:21:00.499 INFO:tasks.ceph:role: osd.2 2023-09-30T22:21:00.499 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:21:00.499 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T22:21:00.575 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:21:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:21:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:21:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:21:00.575 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:21:00.575 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:21:00.576 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:21:00.576 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:21:00.576 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:21:00.576 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:21:00.598 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-09-30T22:21:00.599 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-09-30T22:21:00.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-30T22:21:00.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-30T22:21:00.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-30T22:21:00.800 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:21:00.800 INFO:tasks.ceph:role: osd.3 2023-09-30T22:21:00.800 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:21:00.801 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:21:00.876 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:21:00.877 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:21:00.877 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:21:00.877 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:21:00.898 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-09-30T22:21:00.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-09-30T22:21:00.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-30T22:21:00.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-30T22:21:01.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:21:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:01.128+0000 7f08477f5200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-30T22:21:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:01.128+0000 7f08477f5200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-30T22:21:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:01.128+0000 7f08477f5200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:21:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:01.128+0000 7f08477f5200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:21:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:01.128+0000 7f08477f5200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:21:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:01.128+0000 7f08477f5200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-30T22:21:03.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-30T22:21:04.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:04.101+0000 7fb5d71ad200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-30T22:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:04.102+0000 7fb5d71ad200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-30T22:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:04.102+0000 7fb5d71ad200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:04.102+0000 7fb5d71ad200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:04.102+0000 7fb5d71ad200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:21:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:04.102+0000 7fb5d71ad200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-30T22:21:06.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-30T22:21:06.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:21:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:07.078+0000 7fc81e55f200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-30T22:21:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:07.078+0000 7fc81e55f200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-30T22:21:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:07.078+0000 7fc81e55f200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:21:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:07.078+0000 7fc81e55f200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:21:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:07.078+0000 7fc81e55f200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:21:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:07.078+0000 7fc81e55f200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-30T22:21:09.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-30T22:21:09.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:21:10.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:10.048+0000 7f13ff82f200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-30T22:21:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:10.049+0000 7f13ff82f200 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-30T22:21:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:10.049+0000 7f13ff82f200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:21:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:10.049+0000 7f13ff82f200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:21:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:10.049+0000 7f13ff82f200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:21:10.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:10.049+0000 7f13ff82f200 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-30T22:21:12.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-30T22:21:12.924 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-30T22:21:12.925 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:12.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-30T22:21:13.000 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:13.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-30T22:21:13.075 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:13.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-30T22:21:13.150 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:13.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-30T22:21:13.224 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:13.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-30T22:21:13.299 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:13.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-30T22:21:13.376 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:21:13.376 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-30T22:21:13.398 INFO:tasks.ceph:Adding keys to all mons... 2023-09-30T22:21:13.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBLnxhl5R2dJRAAMgOXni2OpzCzcG801l6gvA== 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBLnxhlTfZZKRAASjKs3D/dKbqtuSj+q9UnpA== 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBNnxhl8jCrBxAACnwIPJuSeqtTr3w+NVoJ9g== 2023-09-30T22:21:13.450 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBQnxhlpocaBhAAnbTMxwbi6HriUvFh5MdacA== 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBTnxhlORSzBBAAZaiJrngJNFEcCOjozZ1sOA== 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBWnxhl5oHwAhAAVlGfddEqX/oj+7fxJL/MUQ== 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-09-30T22:21:13.451 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBLnxhluHeHMhAAEJN17bMKX2BKrMlC/zZcQw== 2023-09-30T22:21:13.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T22:21:13.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-30T22:21:13.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:21:13.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:21:13.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:21:13.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:21:13.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T22:21:13.894 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-30T22:21:13.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-30T22:21:13.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:21:14.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-30T22:21:14.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-30T22:21:14.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:21:14.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-30T22:21:14.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-30T22:21:14.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:21:14.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-30T22:21:14.514 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:21:14.581 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-30T22:21:14.581 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T22:21:14.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T22:21:14.627 INFO:tasks.ceph.mon.a:Started 2023-09-30T22:21:14.627 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T22:21:14.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T22:21:14.633 INFO:tasks.ceph.mon.b:Started 2023-09-30T22:21:14.633 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T22:21:14.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T22:21:14.640 INFO:tasks.ceph.mon.c:Started 2023-09-30T22:21:14.640 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-30T22:21:14.640 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T22:21:14.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T22:21:14.643 INFO:tasks.ceph.mgr.x:Started 2023-09-30T22:21:14.643 INFO:tasks.ceph:Setting crush tunables to default 2023-09-30T22:21:14.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.807+0000 7fe7f98ff700 1 Processor -- start 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.807+0000 7fe7f98ff700 1 -- start start 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f98ff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f98ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f98ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f4096370 con 0x7fe7f412f4e0 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f98ff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40981d0 con 0x7fe7f4097df0 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f98ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4099ed0 con 0x7fe7f4098c00 2023-09-30T22:21:14.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46026/0 (socket says 172.21.15.32:46026) 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 learned_addr learned my addr 172.21.15.32:0/2468397286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:21:14.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:14.808+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:21:15.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:15.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:15.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:21:15.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.009+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:21:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:15.410+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:21:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:16.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7ea7fc700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:16.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:16.211+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:21:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f4098fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f4098fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7f4098710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7f4098710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:17.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 >> v1:172.21.15.32:6789/0 conn(0x7fe7f412f4e0 legacy=0x7fe7f41318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40981d0 con 0x7fe7d4003550 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:17.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:17.807+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:21:18.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.008+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:18.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.008+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:18.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.008+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:18.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.008+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:18.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.008+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:21:18.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.409+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:18.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.409+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:18.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.409+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:18.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.409+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:18.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:18.409+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:21:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:19.210+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:19.210+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:21:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:19.210+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:21:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:19.210+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:21:19.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:19.210+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:21:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 msgr2=0x7fe7d40059e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4003550 0x7fe7d40059e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7d4008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7d400ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7d400d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40981d0 con 0x7fe7f4097df0 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d400d7e0 con 0x7fe7f4098c00 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f17fa700 1 -- 172.21.15.32:0/2468397286 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4003a50 con 0x7fe7d4006240 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7d4008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7f37fe700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7d400d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.807+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7d400ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f37fe700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 msgr2=0x7fe7d4008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f37fe700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7d4008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f37fe700 1 -- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 msgr2=0x7fe7d400ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f37fe700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7d400ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f37fe700 1 -- 172.21.15.32:0/2468397286 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4094e30 con 0x7fe7f4098c00 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7d4008690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7d400ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f37fe700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7d400d2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fe7f4095e60 tx=0x7fe7ec001640).ready entity=mon.1 client_cookie=dd89c65a0a571965 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f27fc700 1 -- 172.21.15.32:0/2468397286 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec006a20 con 0x7fe7f4098c00 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f27fc700 1 -- 172.21.15.32:0/2468397286 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec006b80 con 0x7fe7f4098c00 2023-09-30T22:21:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.808+0000 7fe7f27fc700 1 -- 172.21.15.32:0/2468397286 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007020 con 0x7fe7f4098c00 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7d400d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7d400d2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fe7f4095e60 tx=0x7fe7ec001640).stop 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2468397286 shutdown_connections 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7d400d2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2468397286 >> v2:172.21.15.32:3300/0 conn(0x7fe7f4097df0 0x7fe7d400ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2468397286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7d4008690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2468397286 >> 172.21.15.32:0/2468397286 conn(0x7fe7f409dcf0 msgr2=0x7fe7f409e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2468397286 shutdown_connections 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.809+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2468397286 wait complete. 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 Processor -- start 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 -- start start 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7f411df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f4097df0 0x7fe7f4119f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:20.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f411a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4099cc0 con 0x7fe7f4097df0 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f412f0c0 con 0x7fe7f4098c00 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.811+0000 7fe7f98ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f411da50 con 0x7fe7d4006240 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7f411df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f411a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7f411df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38270/0 (socket says 172.21.15.32:38270) 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f411a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41854/0 (socket says 172.21.15.32:41854) 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2179483457 learned_addr learned my addr 172.21.15.32:0/2179483457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7ea7fc700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f4097df0 0x7fe7f4119f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 msgr2=0x7fe7f411a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f411a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f4097df0 msgr2=0x7fe7f4119f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f4097df0 0x7fe7f4119f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 -- 172.21.15.32:0/2179483457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e400e040 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f37fe700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f411a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f2ffd700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7f411df50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fe7e4017b20 tx=0x7fe7e401b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4027070 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e401f040 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc00e040 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.812+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e401e030 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.813+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f4096370 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.813+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fe7e402c410 con 0x7fe7d4006240 2023-09-30T22:21:20.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.813+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe7e4023750 con 0x7fe7d4006240 2023-09-30T22:21:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.815+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f4095680 con 0x7fe7d4006240 2023-09-30T22:21:20.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.817+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fe7e402c8a0 con 0x7fe7d4006240 2023-09-30T22:21:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.895+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe7f405a260 con 0x7fe7d4006240 2023-09-30T22:21:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.899+0000 7fe7eb7fe700 1 -- 172.21.15.32:0/2179483457 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe7e402ca40 con 0x7fe7d4006240 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 msgr2=0x7fe7f411df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7f411df50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fe7e4017b20 tx=0x7fe7e401b040).stop 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 shutdown_connections 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f4098c00 0x7fe7f411a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d4006240 0x7fe7f411df50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 --2- 172.21.15.32:0/2179483457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f4097df0 0x7fe7f4119f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 >> 172.21.15.32:0/2179483457 conn(0x7fe7f409dcf0 msgr2=0x7fe7f40938e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 shutdown_connections 2023-09-30T22:21:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:20.900+0000 7fe7f98ff700 1 -- 172.21.15.32:0/2179483457 wait complete. 2023-09-30T22:21:20.902 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2023-09-30T22:21:20.915 INFO:tasks.ceph:check_enable_crimson: False 2023-09-30T22:21:20.915 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-30T22:21:20.916 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:20.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-30T22:21:20.923 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:20.922+0000 7fbaea31a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T22:21:20.946 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:20.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-30T22:21:21.003 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:21.002+0000 7fbaea31a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T22:21:21.017 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:21.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-30T22:21:21.088 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:21:21.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-30T22:21:21.098 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:21.098+0000 7fbaea31a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T22:21:21.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new 762e2f2b-1ff4-4d81-840e-a100cc5ed4e6 0 2023-09-30T22:21:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.294+0000 7fd24c1fb700 1 Processor -- start 2023-09-30T22:21:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24c1fb700 1 -- start start 2023-09-30T22:21:21.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24c1fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd244093210 0x7fd24412ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24c1fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244130460 0x7fd244132850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24c1fb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24412ad50 con 0x7fd24412af10 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24c1fb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd244068290 con 0x7fd244093210 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24c1fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440673a0 con 0x7fd244130460 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd249796700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd244093210 0x7fd24412ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd249f97700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd24412af10 0x7fd24412b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53586/0 (socket says 172.21.15.32:53586) 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd249f97700 1 -- 172.21.15.32:0/3515850575 learned_addr learned my addr 172.21.15.32:0/3515850575 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.295+0000 7fd24a798700 1 --2- 172.21.15.32:0/3515850575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244130460 0x7fd244132850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd24a798700 1 -- 172.21.15.32:0/3515850575 >> v2:172.21.15.32:3300/0 conn(0x7fd244093210 msgr2=0x7fd24412ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd24a798700 1 --2- 172.21.15.32:0/3515850575 >> v2:172.21.15.32:3300/0 conn(0x7fd244093210 0x7fd24412ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd24a798700 1 -- 172.21.15.32:0/3515850575 >> v1:172.21.15.32:6789/0 conn(0x7fd24412af10 legacy=0x7fd24412b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd248f95700 1 -- 172.21.15.32:0/3515850575 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118197468 0 0) 0x7fd24412ad50 con 0x7fd24412af10 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd24a798700 1 -- 172.21.15.32:0/3515850575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440675b0 con 0x7fd244130460 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd249796700 1 --2- 172.21.15.32:0/3515850575 >> v2:172.21.15.32:3300/0 conn(0x7fd244093210 0x7fd24412ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd24a798700 1 --2- 172.21.15.32:0/3515850575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244130460 0x7fd244132850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd23c009e60 tx=0x7fd23c009780).ready entity=mon.1 client_cookie=7a371a79867b993a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd248f95700 1 -- 172.21.15.32:0/3515850575 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00e070 con 0x7fd244130460 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd248f95700 1 -- 172.21.15.32:0/3515850575 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c00bb60 con 0x7fd244130460 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.296+0000 7fd248f95700 1 -- 172.21.15.32:0/3515850575 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c012410 con 0x7fd244130460 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 -- 172.21.15.32:0/3515850575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244130460 msgr2=0x7fd244132850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/3515850575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244130460 0x7fd244132850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd23c009e60 tx=0x7fd23c009780).stop 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 -- 172.21.15.32:0/3515850575 shutdown_connections 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/3515850575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244130460 0x7fd244132850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/3515850575 >> v2:172.21.15.32:3300/0 conn(0x7fd244093210 0x7fd24412ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 -- 172.21.15.32:0/3515850575 >> 172.21.15.32:0/3515850575 conn(0x7fd24409a3d0 msgr2=0x7fd24409a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 -- 172.21.15.32:0/3515850575 shutdown_connections 2023-09-30T22:21:21.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.297+0000 7fd24c1fb700 1 -- 172.21.15.32:0/3515850575 wait complete. 2023-09-30T22:21:21.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 Processor -- start 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 -- start start 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244093210 0x7fd2441188c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 0x7fd244114890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd244130460 0x7fd244114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244130150 con 0x7fd24412af10 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd244066f90 con 0x7fd244093210 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24c1fb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd244117b10 con 0x7fd244130460 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd249796700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 0x7fd244114890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd249f97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244093210 0x7fd2441188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd24a798700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd244130460 0x7fd244114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd249796700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 0x7fd244114890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43864/0 (socket says 172.21.15.32:43864) 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.298+0000 7fd249796700 1 -- 172.21.15.32:0/2053884745 learned_addr learned my addr 172.21.15.32:0/2053884745 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249796700 1 -- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd244130460 msgr2=0x7fd244114d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249796700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd244130460 0x7fd244114d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249796700 1 -- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244093210 msgr2=0x7fd2441188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249796700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244093210 0x7fd2441188c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249796700 1 -- 172.21.15.32:0/2053884745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380092a0 con 0x7fd24412af10 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249f97700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244093210 0x7fd2441188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd249796700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 0x7fd244114890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd23800ea20 tx=0x7fd238009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800ee60 con 0x7fd24412af10 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23800c850 con 0x7fd24412af10 2023-09-30T22:21:21.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c009a20 con 0x7fd24412af10 2023-09-30T22:21:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800cbd0 con 0x7fd24412af10 2023-09-30T22:21:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fd23801b660 con 0x7fd24412af10 2023-09-30T22:21:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.299+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244116530 con 0x7fd24412af10 2023-09-30T22:21:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.300+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd2380123f0 con 0x7fd24412af10 2023-09-30T22:21:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.300+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24405fc50 con 0x7fd24412af10 2023-09-30T22:21:21.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.303+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fd238016020 con 0x7fd24412af10 2023-09-30T22:21:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.439+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "762e2f2b-1ff4-4d81-840e-a100cc5ed4e6", "id": 0} v 0) v1 -- 0x7fd244069bd0 con 0x7fd24412af10 2023-09-30T22:21:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.441+0000 7fd235ffb700 1 -- 172.21.15.32:0/2053884745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "762e2f2b-1ff4-4d81-840e-a100cc5ed4e6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd238012670 con 0x7fd24412af10 2023-09-30T22:21:21.442 INFO:teuthology.orchestra.run.smithi032.stdout:0 2023-09-30T22:21:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 msgr2=0x7fd244114890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 0x7fd244114890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd23800ea20 tx=0x7fd238009060).stop 2023-09-30T22:21:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 shutdown_connections 2023-09-30T22:21:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd244130460 0x7fd244114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd244093210 0x7fd2441188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 --2- 172.21.15.32:0/2053884745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24412af10 0x7fd244114890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 >> 172.21.15.32:0/2053884745 conn(0x7fd24409a3d0 msgr2=0x7fd24412f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 shutdown_connections 2023-09-30T22:21:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.443+0000 7fd24c1fb700 1 -- 172.21.15.32:0/2053884745 wait complete. 2023-09-30T22:21:21.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new 3ff901b4-5d78-49f6-b425-d90361753515 1 2023-09-30T22:21:21.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 Processor -- start 2023-09-30T22:21:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 -- start start 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c066d80 0x7fc21c067160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c12e080 0x7fc21c130470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c069580 con 0x7fc21c068590 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c094430 con 0x7fc21c12e080 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc221bd4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c094050 con 0x7fc21c066d80 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc21affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc21c068590 0x7fc21c067650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53588/0 (socket says 172.21.15.32:53588) 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc21bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c12e080 0x7fc21c130470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc21affd700 1 -- 172.21.15.32:0/869966824 learned_addr learned my addr 172.21.15.32:0/869966824 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.590+0000 7fc21b7fe700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c066d80 0x7fc21c067160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21a7fc700 1 -- 172.21.15.32:0/869966824 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020269913 0 0) 0x7fc21c069580 con 0x7fc21c068590 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21a7fc700 1 -- 172.21.15.32:0/869966824 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2040033f0 con 0x7fc21c068590 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21b7fe700 1 -- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c12e080 msgr2=0x7fc21c130470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21b7fe700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c12e080 0x7fc21c130470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21b7fe700 1 -- 172.21.15.32:0/869966824 >> v1:172.21.15.32:6789/0 conn(0x7fc21c068590 legacy=0x7fc21c067650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21b7fe700 1 -- 172.21.15.32:0/869966824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c069840 con 0x7fc21c066d80 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21bfff700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c12e080 0x7fc21c130470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21b7fe700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c066d80 0x7fc21c067160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc20c009790 tx=0x7fc20c009060).ready entity=mon.2 client_cookie=ea32b4a9251e7637 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21a7fc700 1 -- 172.21.15.32:0/869966824 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0090d0 con 0x7fc21c066d80 2023-09-30T22:21:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21a7fc700 1 -- 172.21.15.32:0/869966824 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00f510 con 0x7fc21c066d80 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.591+0000 7fc21a7fc700 1 -- 172.21.15.32:0/869966824 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00f890 con 0x7fc21c066d80 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 -- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c066d80 msgr2=0x7fc21c067160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c066d80 0x7fc21c067160 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc20c009790 tx=0x7fc20c009060).stop 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 -- 172.21.15.32:0/869966824 shutdown_connections 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c12e080 0x7fc21c130470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 --2- 172.21.15.32:0/869966824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c066d80 0x7fc21c067160 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 -- 172.21.15.32:0/869966824 >> 172.21.15.32:0/869966824 conn(0x7fc21c09a3d0 msgr2=0x7fc21c09a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 -- 172.21.15.32:0/869966824 shutdown_connections 2023-09-30T22:21:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.592+0000 7fc221bd4700 1 -- 172.21.15.32:0/869966824 wait complete. 2023-09-30T22:21:21.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.593+0000 7fc221bd4700 1 Processor -- start 2023-09-30T22:21:21.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.593+0000 7fc221bd4700 1 -- start start 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc221bd4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c066d80 0x7fc21c115670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc221bd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 0x7fc21c111a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc221bd4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 0x7fc21c111ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc221bd4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1149b0 con 0x7fc21c068590 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc221bd4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c12e460 con 0x7fc21c066d80 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc221bd4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0690f0 con 0x7fc21c12e080 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 0x7fc21c111ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 0x7fc21c111a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 0x7fc21c111ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38300/0 (socket says 172.21.15.32:38300) 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 0x7fc21c111a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43874/0 (socket says 172.21.15.32:43874) 2023-09-30T22:21:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21bfff700 1 -- 172.21.15.32:0/1126657060 learned_addr learned my addr 172.21.15.32:0/1126657060 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21b7fe700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c066d80 0x7fc21c115670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 -- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 msgr2=0x7fc21c111ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 0x7fc21c111ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 -- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c066d80 msgr2=0x7fc21c115670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c066d80 0x7fc21c115670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 -- 172.21.15.32:0/1126657060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009a20 con 0x7fc21c068590 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21bfff700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 0x7fc21c111ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.594+0000 7fc21affd700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 0x7fc21c111a00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fc21c067d60 tx=0x7fc210007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100070d0 con 0x7fc21c068590 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210007250 con 0x7fc21c068590 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c0092a0 con 0x7fc21c068590 2023-09-30T22:21:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210007610 con 0x7fc21c068590 2023-09-30T22:21:21.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c1139e0 con 0x7fc21c068590 2023-09-30T22:21:21.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fc210005c10 con 0x7fc21c068590 2023-09-30T22:21:21.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.595+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fc210013bf0 con 0x7fc21c068590 2023-09-30T22:21:21.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.596+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c05a260 con 0x7fc21c068590 2023-09-30T22:21:21.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.598+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fc210018070 con 0x7fc21c068590 2023-09-30T22:21:21.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.682+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ff901b4-5d78-49f6-b425-d90361753515", "id": 1} v 0) v1 -- 0x7fc21c112800 con 0x7fc21c068590 2023-09-30T22:21:21.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.685+0000 7fc2037fe700 1 -- 172.21.15.32:0/1126657060 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ff901b4-5d78-49f6-b425-d90361753515", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc21001d440 con 0x7fc21c068590 2023-09-30T22:21:21.686 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-09-30T22:21:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 msgr2=0x7fc21c111a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 0x7fc21c111a00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fc21c067d60 tx=0x7fc210007bd0).stop 2023-09-30T22:21:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 shutdown_connections 2023-09-30T22:21:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc21c12e080 0x7fc21c111ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc21c066d80 0x7fc21c115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 --2- 172.21.15.32:0/1126657060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc21c068590 0x7fc21c111a00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 >> 172.21.15.32:0/1126657060 conn(0x7fc21c09a3d0 msgr2=0x7fc21c0982d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 shutdown_connections 2023-09-30T22:21:21.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.686+0000 7fc221bd4700 1 -- 172.21.15.32:0/1126657060 wait complete. 2023-09-30T22:21:21.700 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new bfe588b1-e7fe-46ec-a3f2-fef1646f54c1 2 2023-09-30T22:21:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.828+0000 7f4a0b95a700 1 Processor -- start 2023-09-30T22:21:21.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.828+0000 7f4a0b95a700 1 -- start start 2023-09-30T22:21:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a0b95a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a04067e90 0x7f4a04068270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a0b95a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a041282b0 0x7f4a0412fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a0b95a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040690b0 con 0x7f4a04068760 2023-09-30T22:21:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a0b95a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040920e0 con 0x7f4a04067e90 2023-09-30T22:21:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a0b95a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04092330 con 0x7f4a041282b0 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a04067e90 0x7f4a04068270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a09ef7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a041282b0 0x7f4a0412fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a04067e90 0x7f4a04068270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43880/0 (socket says 172.21.15.32:43880) 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a09ef7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a041282b0 0x7f4a0412fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38304/0 (socket says 172.21.15.32:38304) 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 -- 172.21.15.32:0/1601452403 learned_addr learned my addr 172.21.15.32:0/1601452403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 -- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a041282b0 msgr2=0x7f4a0412fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 --2- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a041282b0 0x7f4a0412fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 -- 172.21.15.32:0/1601452403 >> v1:172.21.15.32:6789/0 conn(0x7f4a04068760 legacy=0x7f4a04127ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f49f7fff700 1 -- 172.21.15.32:0/1601452403 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183324477 0 0) 0x7f4a040690b0 con 0x7f4a04068760 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.829+0000 7f4a096f6700 1 -- 172.21.15.32:0/1601452403 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04069b20 con 0x7f4a04067e90 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a096f6700 1 --2- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04067e90 0x7f4a04068270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f49f8006fd0 tx=0x7f49f800b3b0).ready entity=mon.0 client_cookie=b7d36fe1d561c714 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f49f7fff700 1 -- 172.21.15.32:0/1601452403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800da70 con 0x7f4a04067e90 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f49f7fff700 1 -- 172.21.15.32:0/1601452403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f800dbd0 con 0x7f4a04067e90 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f49f7fff700 1 -- 172.21.15.32:0/1601452403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8010470 con 0x7f4a04067e90 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 -- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04067e90 msgr2=0x7f4a04068270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04067e90 0x7f4a04068270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f49f8006fd0 tx=0x7f49f800b3b0).stop 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 -- 172.21.15.32:0/1601452403 shutdown_connections 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a041282b0 0x7f4a0412fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/1601452403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a04067e90 0x7f4a04068270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 -- 172.21.15.32:0/1601452403 >> 172.21.15.32:0/1601452403 conn(0x7f4a0409a3d0 msgr2=0x7f4a0409a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 -- 172.21.15.32:0/1601452403 shutdown_connections 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.830+0000 7f4a0b95a700 1 -- 172.21.15.32:0/1601452403 wait complete. 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.831+0000 7f4a0b95a700 1 Processor -- start 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.831+0000 7f4a0b95a700 1 -- start start 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a0b95a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a04067e90 0x7f4a041159c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a0b95a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a04068760 0x7f4a04115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a0b95a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 0x7f4a04135f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a0b95a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04092e80 con 0x7f4a041282b0 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a0b95a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04099eb0 con 0x7f4a04067e90 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a0b95a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a041186b0 con 0x7f4a04068760 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a09ef7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 0x7f4a04135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a09ef7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 0x7f4a04135f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43882/0 (socket says 172.21.15.32:43882) 2023-09-30T22:21:21.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a096f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a04067e90 0x7f4a041159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a09ef7700 1 -- 172.21.15.32:0/3721482412 learned_addr learned my addr 172.21.15.32:0/3721482412 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.832+0000 7f4a08ef5700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a04068760 0x7f4a04115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a09ef7700 1 -- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a04068760 msgr2=0x7f4a04115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a09ef7700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a04068760 0x7f4a04115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a09ef7700 1 -- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a04067e90 msgr2=0x7f4a041159c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a09ef7700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a04067e90 0x7f4a041159c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a09ef7700 1 -- 172.21.15.32:0/3721482412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f800b610 con 0x7f4a041282b0 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a08ef5700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a04068760 0x7f4a04115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a096f6700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a04067e90 0x7f4a041159c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a09ef7700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 0x7f4a04135f00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f49fc009e90 tx=0x7f49fc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:21.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00cb50 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc00f040 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a041156b0 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00cec0 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a041194f0 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f49fc013870 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.833+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f49fc01b790 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.834+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a041188b0 con 0x7f4a041282b0 2023-09-30T22:21:21.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.836+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f4a041188b0 con 0x7f4a041282b0 2023-09-30T22:21:21.955 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:21.954+0000 7fbaea31a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T22:21:21.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.969+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bfe588b1-e7fe-46ec-a3f2-fef1646f54c1", "id": 2} v 0) v1 -- 0x7f4a0405a260 con 0x7f4a041282b0 2023-09-30T22:21:21.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.972+0000 7f49f57fa700 1 -- 172.21.15.32:0/3721482412 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bfe588b1-e7fe-46ec-a3f2-fef1646f54c1", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4a041194f0 con 0x7f4a041282b0 2023-09-30T22:21:21.973 INFO:teuthology.orchestra.run.smithi032.stdout:2 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 msgr2=0x7f4a04135f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 0x7f4a04135f00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f49fc009e90 tx=0x7f49fc00b040).stop 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 shutdown_connections 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a041282b0 0x7f4a04135f00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a04067e90 0x7f4a041159c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 --2- 172.21.15.32:0/3721482412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a04068760 0x7f4a04115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 >> 172.21.15.32:0/3721482412 conn(0x7f4a0409a3d0 msgr2=0x7f4a04096af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:21.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 shutdown_connections 2023-09-30T22:21:21.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:21.973+0000 7f4a0b95a700 1 -- 172.21.15.32:0/3721482412 wait complete. 2023-09-30T22:21:21.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new 5b3ee2e9-f923-4976-ad87-11ba143ec874 3 2023-09-30T22:21:22.081 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:22.080+0000 7fbaea31a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T22:21:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 Processor -- start 2023-09-30T22:21:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 -- start start 2023-09-30T22:21:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8128840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8128e60 0x7f75f81292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f8097070 con 0x7f75f8093bf0 2023-09-30T22:21:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8096210 con 0x7f75f8096c90 2023-09-30T22:21:22.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75fe00c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8095790 con 0x7f75f8128e60 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75f7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75f8093bf0 0x7f75f80940a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53604/0 (socket says 172.21.15.32:53604) 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75f6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8128e60 0x7f75f81292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75f7fff700 1 -- 172.21.15.32:0/1431961205 learned_addr learned my addr 172.21.15.32:0/1431961205 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.117+0000 7f75f77fe700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8128840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f67fc700 1 -- 172.21.15.32:0/1431961205 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855290893 0 0) 0x7f75f8097070 con 0x7f75f8093bf0 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f67fc700 1 -- 172.21.15.32:0/1431961205 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e40033f0 con 0x7f75f8093bf0 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f77fe700 1 -- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8128e60 msgr2=0x7f75f81292e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f77fe700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8128e60 0x7f75f81292e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f77fe700 1 -- 172.21.15.32:0/1431961205 >> v1:172.21.15.32:6789/0 conn(0x7f75f8093bf0 legacy=0x7f75f80940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f77fe700 1 -- 172.21.15.32:0/1431961205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8059230 con 0x7f75f8096c90 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f6ffd700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8128e60 0x7f75f81292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:22.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f77fe700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8128840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f75e8009e90 tx=0x7f75e8009780).ready entity=mon.1 client_cookie=7edcc76025d9cdc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f67fc700 1 -- 172.21.15.32:0/1431961205 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800e070 con 0x7f75f8096c90 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f67fc700 1 -- 172.21.15.32:0/1431961205 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e800bab0 con 0x7f75f8096c90 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.118+0000 7f75f67fc700 1 -- 172.21.15.32:0/1431961205 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800be30 con 0x7f75f8096c90 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 -- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 msgr2=0x7f75f8128840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8128840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f75e8009e90 tx=0x7f75e8009780).stop 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 -- 172.21.15.32:0/1431961205 shutdown_connections 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8128840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 --2- 172.21.15.32:0/1431961205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8128e60 0x7f75f81292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 -- 172.21.15.32:0/1431961205 >> 172.21.15.32:0/1431961205 conn(0x7f75f80974f0 msgr2=0x7f75f8097900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 -- 172.21.15.32:0/1431961205 shutdown_connections 2023-09-30T22:21:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.119+0000 7f75fe00c700 1 -- 172.21.15.32:0/1431961205 wait complete. 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.120+0000 7f75fe00c700 1 Processor -- start 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.120+0000 7f75fe00c700 1 -- start start 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75fe00c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 0x7f75f8092110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75fe00c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8092600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75fe00c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f8128e60 0x7f75f8092af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75fe00c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f81170e0 con 0x7f75f8128e60 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75fe00c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8127e90 con 0x7f75f8096c90 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75fe00c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8116aa0 con 0x7f75f8093bf0 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8092600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 0x7f75f8092110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8092600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41924/0 (socket says 172.21.15.32:41924) 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 0x7f75f8092110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38316/0 (socket says 172.21.15.32:38316) 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 -- 172.21.15.32:0/303184099 learned_addr learned my addr 172.21.15.32:0/303184099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f7fff700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f8128e60 0x7f75f8092af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 -- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 msgr2=0x7f75f8092110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 0x7f75f8092110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 -- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f8128e60 msgr2=0x7f75f8092af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f8128e60 0x7f75f8092af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 -- 172.21.15.32:0/303184099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00092a0 con 0x7f75f8096c90 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f77fe700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 0x7f75f8092110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f7fff700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f8128e60 0x7f75f8092af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:22.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.121+0000 7f75f6ffd700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8092600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f75e000e9d0 tx=0x7f75e0009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e000c6a0 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e000c800 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e8009a20 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e000cb80 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f81178e0 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f75e001b660 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.122+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f75e0012890 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.123+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f8096410 con 0x7f75f8096c90 2023-09-30T22:21:22.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.125+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f75e003ae50 con 0x7f75f8096c90 2023-09-30T22:21:22.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.210+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5b3ee2e9-f923-4976-ad87-11ba143ec874", "id": 3} v 0) v1 -- 0x7f75f805a260 con 0x7f75f8096c90 2023-09-30T22:21:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.214+0000 7f75ef7fe700 1 -- 172.21.15.32:0/303184099 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5b3ee2e9-f923-4976-ad87-11ba143ec874", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f75e0029030 con 0x7f75f8096c90 2023-09-30T22:21:22.215 INFO:teuthology.orchestra.run.smithi032.stdout:3 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 msgr2=0x7f75f8092600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8092600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f75e000e9d0 tx=0x7f75e0009060).stop 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 shutdown_connections 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f8128e60 0x7f75f8092af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f8093bf0 0x7f75f8092110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 --2- 172.21.15.32:0/303184099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f8096c90 0x7f75f8092600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 >> 172.21.15.32:0/303184099 conn(0x7f75f80974f0 msgr2=0x7f75f812bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 shutdown_connections 2023-09-30T22:21:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.215+0000 7f75fe00c700 1 -- 172.21.15.32:0/303184099 wait complete. 2023-09-30T22:21:22.229 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T22:21:22.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T22:21:22.231 INFO:tasks.ceph.osd.0:Started 2023-09-30T22:21:22.231 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T22:21:22.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T22:21:22.234 INFO:tasks.ceph.osd.1:Started 2023-09-30T22:21:22.234 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T22:21:22.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T22:21:22.235 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:22.235+0000 7fbaea31a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T22:21:22.236 INFO:tasks.ceph.osd.2:Started 2023-09-30T22:21:22.236 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T22:21:22.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T22:21:22.238 INFO:tasks.ceph.osd.3:Started 2023-09-30T22:21:22.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.409+0000 7f677da89700 1 Processor -- start 2023-09-30T22:21:22.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.409+0000 7f677da89700 1 -- start start 2023-09-30T22:21:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f677da89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f6778098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f677da89700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f6778132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f677da89700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6778065ed0 con 0x7f6778098b30 2023-09-30T22:21:22.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f677da89700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6778095880 con 0x7f6778098f70 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f677da89700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6778096120 con 0x7f67780654a0 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f67777fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f6778098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6777fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f6778132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f676ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6778098b30 0x7f677812f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53622/0 (socket says 172.21.15.32:53622) 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f676ffff700 1 -- 172.21.15.32:0/2213364991 learned_addr learned my addr 172.21.15.32:0/2213364991 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6776ffd700 1 -- 172.21.15.32:0/2213364991 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150664972 0 0) 0x7f6778065ed0 con 0x7f6778098b30 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6776ffd700 1 -- 172.21.15.32:0/2213364991 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67640033f0 con 0x7f6778098b30 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6777fff700 1 -- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 msgr2=0x7f6778098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6777fff700 1 --2- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f6778098640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6777fff700 1 -- 172.21.15.32:0/2213364991 >> v1:172.21.15.32:6789/0 conn(0x7f6778098b30 legacy=0x7f677812f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6777fff700 1 -- 172.21.15.32:0/2213364991 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778066280 con 0x7f6778098f70 2023-09-30T22:21:22.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f67777fe700 1 --2- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f6778098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.410+0000 7f6777fff700 1 --2- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f6778132ba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f6770009ec0 tx=0x7f6770009780).ready entity=mon.1 client_cookie=770e597ec1f6f39 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f6776ffd700 1 -- 172.21.15.32:0/2213364991 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677000e070 con 0x7f6778098f70 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f6776ffd700 1 -- 172.21.15.32:0/2213364991 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677000bb40 con 0x7f6778098f70 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f6776ffd700 1 -- 172.21.15.32:0/2213364991 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770012410 con 0x7f6778098f70 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 -- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 msgr2=0x7f6778132ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 --2- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f6778132ba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f6770009ec0 tx=0x7f6770009780).stop 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 -- 172.21.15.32:0/2213364991 shutdown_connections 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 --2- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f6778132ba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 --2- 172.21.15.32:0/2213364991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f6778098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 -- 172.21.15.32:0/2213364991 >> 172.21.15.32:0/2213364991 conn(0x7f677809a740 msgr2=0x7f677809ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.411+0000 7f677da89700 1 -- 172.21.15.32:0/2213364991 shutdown_connections 2023-09-30T22:21:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.412+0000 7f677da89700 1 -- 172.21.15.32:0/2213364991 wait complete. 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.412+0000 7f677da89700 1 Processor -- start 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 -- start start 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f677813a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6778098b30 0x7f677813a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f677813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677811de50 con 0x7f6778098b30 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6778099b00 con 0x7f6778098f70 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f677da89700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67780946c0 con 0x7f67780654a0 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f6777fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f677813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f67777fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f677813a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f676ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6778098b30 0x7f677813a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f67777fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f677813a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38364/0 (socket says 172.21.15.32:38364) 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f6777fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f677813aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41978/0 (socket says 172.21.15.32:41978) 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.413+0000 7f67777fe700 1 -- 172.21.15.32:0/3920622226 learned_addr learned my addr 172.21.15.32:0/3920622226 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f67777fe700 1 -- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 msgr2=0x7f677813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f67777fe700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f677813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f67777fe700 1 -- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6778098b30 msgr2=0x7f677813a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f67777fe700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6778098b30 0x7f677813a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f67777fe700 1 -- 172.21.15.32:0/3920622226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67680092a0 con 0x7f67780654a0 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f676ffff700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6778098b30 0x7f677813a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f6777fff700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f677813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f67777fe700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f677813a0e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f676800ea80 tx=0x7f6768009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800eec0 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676800c850 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6770009a20 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800cbd0 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677811e660 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f676801b660 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.414+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f67680123f0 con 0x7f67780654a0 2023-09-30T22:21:22.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.415+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677805a260 con 0x7f67780654a0 2023-09-30T22:21:22.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.418+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f677811e660 con 0x7f67780654a0 2023-09-30T22:21:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.490+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f677811da40 con 0x7f67780654a0 2023-09-30T22:21:22.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.490+0000 7f676f7fe700 1 -- 172.21.15.32:0/3920622226 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7f6768016020 con 0x7f67780654a0 2023-09-30T22:21:22.492 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:22.492 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:22.213069+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 msgr2=0x7f677813a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f677813a0e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f676800ea80 tx=0x7f6768009060).stop 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 shutdown_connections 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6778098f70 0x7f677813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67780654a0 0x7f677813a0e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 --2- 172.21.15.32:0/3920622226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6778098b30 0x7f677813a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.491+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 >> 172.21.15.32:0/3920622226 conn(0x7f677809a740 msgr2=0x7f677809bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.492+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 shutdown_connections 2023-09-30T22:21:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:22.492+0000 7f677da89700 1 -- 172.21.15.32:0/3920622226 wait complete. 2023-09-30T22:21:22.506 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-30T22:21:22.506 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-30T22:21:22.760 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:22.759+0000 7fbaea31a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T22:21:22.851 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:21:22.850+0000 7f3999133200 -1 Falling back to public interface 2023-09-30T22:21:22.858 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:21:22.857+0000 7ffa6c87c200 -1 Falling back to public interface 2023-09-30T22:21:22.883 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:21:22.882+0000 7f149edc7200 -1 Falling back to public interface 2023-09-30T22:21:22.883 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:21:22.882+0000 7fca8c26b200 -1 Falling back to public interface 2023-09-30T22:21:22.884 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:21:22.883+0000 7f1498f6f700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6808/0: (98) Address already in use 2023-09-30T22:21:22.908 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:22.961 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:22.960+0000 7fbaea31a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:21:23.028 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.028+0000 7f71873de700 1 Processor -- start 2023-09-30T22:21:23.028 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.028+0000 7f71873de700 1 -- start start 2023-09-30T22:21:23.030 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f71873de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f7180095bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:23.030 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f71873de700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7180130910 0x7f7180132d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:23.030 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f71873de700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7180094ee0 con 0x7f71800960a0 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f71873de700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7180096b50 con 0x7f7180130910 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f71873de700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7180092f30 con 0x7f71800957d0 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7180130910 0x7f7180132d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718517a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f7180095bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7180130910 0x7f7180132d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43952/0 (socket says 172.21.15.32:43952) 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718517a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f7180095bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38374/0 (socket says 172.21.15.32:38374) 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 -- 172.21.15.32:0/725221787 learned_addr learned my addr 172.21.15.32:0/725221787 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f7184979700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71800960a0 0x7f718012f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53638/0 (socket says 172.21.15.32:53638) 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 -- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 msgr2=0x7f7180095bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 --2- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f7180095bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:23.031 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 -- 172.21.15.32:0/725221787 >> v1:172.21.15.32:6789/0 conn(0x7f71800960a0 legacy=0x7f718012f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f7177fff700 1 -- 172.21.15.32:0/725221787 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131477240 0 0) 0x7f7180094ee0 con 0x7f71800960a0 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718597b700 1 -- 172.21.15.32:0/725221787 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800632f0 con 0x7f7180130910 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.029+0000 7f718517a700 1 --2- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f7180095bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f718597b700 1 --2- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180130910 0x7f7180132d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f7178009f80 tx=0x7f7178009840).ready entity=mon.0 client_cookie=df774d1824f3484b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f7177fff700 1 -- 172.21.15.32:0/725221787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800e070 con 0x7f7180130910 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f7177fff700 1 -- 172.21.15.32:0/725221787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800ba00 con 0x7f7180130910 2023-09-30T22:21:23.032 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f7177fff700 1 -- 172.21.15.32:0/725221787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800bd80 con 0x7f7180130910 2023-09-30T22:21:23.034 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f71873de700 1 -- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180130910 msgr2=0x7f7180132d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.034 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f71873de700 1 --2- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180130910 0x7f7180132d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f7178009f80 tx=0x7f7178009840).stop 2023-09-30T22:21:23.034 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f71873de700 1 -- 172.21.15.32:0/725221787 shutdown_connections 2023-09-30T22:21:23.034 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f71873de700 1 --2- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7180130910 0x7f7180132d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f71873de700 1 --2- 172.21.15.32:0/725221787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f7180095bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.030+0000 7f71873de700 1 -- 172.21.15.32:0/725221787 >> 172.21.15.32:0/725221787 conn(0x7f718009ab90 msgr2=0x7f718009afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.031+0000 7f71873de700 1 -- 172.21.15.32:0/725221787 shutdown_connections 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.031+0000 7f71873de700 1 -- 172.21.15.32:0/725221787 wait complete. 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.031+0000 7f71873de700 1 Processor -- start 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 -- start start 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f71801173c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71800960a0 0x7f71801178b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7180130910 0x7f7180117da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718012d4e0 con 0x7f71800960a0 2023-09-30T22:21:23.035 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71801199f0 con 0x7f7180130910 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f71873de700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7180096df0 con 0x7f71800957d0 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f718517a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f71801173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f7184979700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71800960a0 0x7f71801178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f718517a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f71801173c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38388/0 (socket says 172.21.15.32:38388) 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.032+0000 7f718517a700 1 -- 172.21.15.32:0/941326657 learned_addr learned my addr 172.21.15.32:0/941326657 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f718517a700 1 -- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7180130910 msgr2=0x7f7180117da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f718517a700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7180130910 0x7f7180117da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f718517a700 1 -- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71800960a0 msgr2=0x7f71801178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f718517a700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71800960a0 0x7f71801178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:23.036 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f718517a700 1 -- 172.21.15.32:0/941326657 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0092a0 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f7184979700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71800960a0 0x7f71801178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f718517a700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f71801173c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f717c00ea20 tx=0x7f717c009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00c850 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00c9b0 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7178009ae0 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00cd30 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718011b540 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f717c012860 con 0x7f71800957d0 2023-09-30T22:21:23.037 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.033+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f717c01b910 con 0x7f71800957d0 2023-09-30T22:21:23.038 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.035+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718005a260 con 0x7f71800957d0 2023-09-30T22:21:23.038 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.037+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f717c03ae50 con 0x7f71800957d0 2023-09-30T22:21:23.038 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:23.038+0000 7fbaea31a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T22:21:23.108 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.107+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7180092bd0 con 0x7f71800957d0 2023-09-30T22:21:23.109 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.108+0000 7f71757fa700 1 -- 172.21.15.32:0/941326657 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7f717c029030 con 0x7f71800957d0 2023-09-30T22:21:23.109 INFO:teuthology.misc.health.smithi032.stdout: 2023-09-30T22:21:23.109 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":6,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:22.213069+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 msgr2=0x7f71801173c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f71801173c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f717c00ea20 tx=0x7f717c009060).stop 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 shutdown_connections 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7180130910 0x7f7180117da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71800957d0 0x7f71801173c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 --2- 172.21.15.32:0/941326657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71800960a0 0x7f71801178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 >> 172.21.15.32:0/941326657 conn(0x7f718009ab90 msgr2=0x7f71801301b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 shutdown_connections 2023-09-30T22:21:23.110 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:23.109+0000 7f71873de700 1 -- 172.21.15.32:0/941326657 wait complete. 2023-09-30T22:21:23.121 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-09-30T22:21:23.127 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:23.126+0000 7fbaea31a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T22:21:23.207 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:23.207+0000 7fbaea31a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T22:21:23.604 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:23.603+0000 7fbaea31a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T22:21:23.700 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:23.700+0000 7fbaea31a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T22:21:24.425 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:24.425+0000 7fbaea31a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T22:21:24.504 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:24.503+0000 7fbaea31a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T22:21:24.592 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:24.591+0000 7fbaea31a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T22:21:24.924 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:24.923+0000 7fbaea31a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T22:21:25.003 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:25.003+0000 7fbaea31a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T22:21:25.135 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:25.135+0000 7fbaea31a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T22:21:25.256 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:25.255+0000 7fbaea31a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:21:25.435 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:25.434+0000 7fbaea31a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T22:21:25.513 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:25.513+0000 7fbaea31a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T22:21:26.350 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:21:26.350+0000 7f3999133200 -1 osd.1 0 log_to_monitors {default=true} 2023-09-30T22:21:26.390 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:21:26.389+0000 7fca8c26b200 -1 osd.2 0 log_to_monitors {default=true} 2023-09-30T22:21:26.416 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:21:26.415+0000 7f149edc7200 -1 osd.3 0 log_to_monitors {default=true} 2023-09-30T22:21:26.535 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:26.535+0000 7fbaa0d57700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T22:21:26.536 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:21:26.535+0000 7fbaa0d57700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T22:21:26.601 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:21:26.601+0000 7ffa6c87c200 -1 osd.0 0 log_to_monitors {default=true} 2023-09-30T22:21:27.787 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:21:27.786+0000 7fca82c0c700 -1 osd.2 0 waiting for initial osdmap 2023-09-30T22:21:27.787 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:21:27.786+0000 7f1495768700 -1 osd.3 0 waiting for initial osdmap 2023-09-30T22:21:27.789 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:21:27.784+0000 7f398fad4700 -1 osd.1 0 waiting for initial osdmap 2023-09-30T22:21:27.789 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:21:27.789+0000 7ffa6321d700 -1 osd.0 0 waiting for initial osdmap 2023-09-30T22:21:27.792 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:21:27.791+0000 7f398c250700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:21:27.792 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:21:27.791+0000 7fca7f388700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:21:27.792 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:21:27.791+0000 7f1491ee4700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:21:27.794 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:21:27.793+0000 7ffa5f999700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:21:29.525 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:29.657 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 Processor -- start 2023-09-30T22:21:29.657 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 -- start start 2023-09-30T22:21:29.657 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 0x7f601c0958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.657 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c0961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 0x7f601c12e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c094e40 con 0x7f601c094a60 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c091b90 con 0x7f601c095dd0 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.655+0000 7f60224bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f601c096d40 con 0x7f601c12c1c0 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c0961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 0x7f601c12e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 0x7f601c0958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 0x7f601c12e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38444/0 (socket says 172.21.15.32:38444) 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 0x7f601c0958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44018/0 (socket says 172.21.15.32:44018) 2023-09-30T22:21:29.658 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c0961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42020/0 (socket says 172.21.15.32:42020) 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 -- 172.21.15.32:0/763448219 learned_addr learned my addr 172.21.15.32:0/763448219 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 -- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 msgr2=0x7f601c0961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 --2- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c0961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 -- 172.21.15.32:0/763448219 >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 msgr2=0x7f601c0958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 --2- 172.21.15.32:0/763448219 >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 0x7f601c0958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 -- 172.21.15.32:0/763448219 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0934f0 con 0x7f601c12c1c0 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601bfff700 1 --2- 172.21.15.32:0/763448219 >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 0x7f601c0958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f6020a5c700 1 --2- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 0x7f601c12e5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f6014009f50 tx=0x7f6014009840).ready entity=mon.2 client_cookie=e276b28c33b0fae6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601affd700 1 -- 172.21.15.32:0/763448219 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601400e070 con 0x7f601c12c1c0 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601affd700 1 -- 172.21.15.32:0/763448219 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601400b910 con 0x7f601c12c1c0 2023-09-30T22:21:29.659 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.656+0000 7f601affd700 1 -- 172.21.15.32:0/763448219 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601400bc90 con 0x7f601c12c1c0 2023-09-30T22:21:29.662 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 -- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 msgr2=0x7f601c12e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.662 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 --2- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 0x7f601c12e5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f6014009f50 tx=0x7f6014009840).stop 2023-09-30T22:21:29.662 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 -- 172.21.15.32:0/763448219 shutdown_connections 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 --2- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c12c1c0 0x7f601c12e5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 --2- 172.21.15.32:0/763448219 >> v2:172.21.15.32:3300/0 conn(0x7f601c094a60 0x7f601c0958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 --2- 172.21.15.32:0/763448219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c0961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 -- 172.21.15.32:0/763448219 >> 172.21.15.32:0/763448219 conn(0x7f601c09ab90 msgr2=0x7f601c09afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 -- 172.21.15.32:0/763448219 shutdown_connections 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.657+0000 7f60224bf700 1 -- 172.21.15.32:0/763448219 wait complete. 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.658+0000 7f60224bf700 1 Processor -- start 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.658+0000 7f60224bf700 1 -- start start 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f60224bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 0x7f601c118040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f60224bf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c118530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.663 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f60224bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c12c1c0 0x7f601c118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f60224bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c118f10 con 0x7f601c12c1c0 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f60224bf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c119bd0 con 0x7f601c095dd0 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f60224bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f601c11b540 con 0x7f601c094a60 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 0x7f601c118040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 0x7f601c118040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38458/0 (socket says 172.21.15.32:38458) 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 -- 172.21.15.32:0/4215698003 learned_addr learned my addr 172.21.15.32:0/4215698003 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f6020a5c700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c12c1c0 0x7f601c118a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601b7fe700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c118530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 -- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 msgr2=0x7f601c118530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c118530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.664 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 -- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c12c1c0 msgr2=0x7f601c118a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c12c1c0 0x7f601c118a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 -- 172.21.15.32:0/4215698003 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040092a0 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f6020a5c700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c12c1c0 0x7f601c118a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f601bfff700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 0x7f601c118040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f600400ea50 tx=0x7f6004009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.659+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400ee90 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.660+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400c850 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.660+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400cbd0 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.660+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6014009ae0 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.660+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c0924d0 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.660+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f600401b400 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.661+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2069+0+0 (secure 0 0 0) 0x7f600403ae90 con 0x7f601c094a60 2023-09-30T22:21:29.665 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.661+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c11ba20 con 0x7f601c094a60 2023-09-30T22:21:29.666 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.666+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f601c11ba20 con 0x7f601c094a60 2023-09-30T22:21:29.846 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.846+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f601c11ba20 con 0x7f601c094a60 2023-09-30T22:21:29.847 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.846+0000 7f6013fff700 1 -- 172.21.15.32:0/4215698003 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6597 (secure 0 0 0) 0x7f601c11ba20 con 0x7f601c094a60 2023-09-30T22:21:29.847 INFO:teuthology.misc.health.smithi032.stdout: 2023-09-30T22:21:29.847 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:29.785382+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:29.850 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.848+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 >> v1:172.21.15.32:6816/118188 conn(0x7f5ffc02bb60 legacy=0x7f5ffc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.850 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.848+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 msgr2=0x7f601c118040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.850 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.848+0000 7f60224bf700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 0x7f601c118040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f600400ea50 tx=0x7f6004009060).stop 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.848+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 shutdown_connections 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.848+0000 7f60224bf700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f601c12c1c0 0x7f601c118a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.848+0000 7f60224bf700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f601c094a60 0x7f601c118040 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.849+0000 7f60224bf700 1 --2- 172.21.15.32:0/4215698003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f601c095dd0 0x7f601c118530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.849+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 >> 172.21.15.32:0/4215698003 conn(0x7f601c09ab90 msgr2=0x7f601c09b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.849+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 shutdown_connections 2023-09-30T22:21:29.851 INFO:teuthology.misc.health.smithi032.stderr:2023-09-30T22:21:29.849+0000 7f60224bf700 1 -- 172.21.15.32:0/4215698003 wait complete. 2023-09-30T22:21:29.862 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-09-30T22:21:29.862 INFO:tasks.ceph:Creating RBD pool 2023-09-30T22:21:29.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-30T22:21:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.991+0000 7f3b8eac6700 1 Processor -- start 2023-09-30T22:21:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.991+0000 7f3b8eac6700 1 -- start start 2023-09-30T22:21:29.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8eac6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b881275c0 0x7f3b8812fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8eac6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b88130220 0x7f3b88132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8eac6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b88091c80 con 0x7f3b880926d0 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8eac6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b881279e0 con 0x7f3b88130220 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8eac6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880999d0 con 0x7f3b881275c0 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b87fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b881275c0 0x7f3b8812fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8d063700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b88130220 0x7f3b88132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8c862700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3b880926d0 0x7f3b88127be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53666/0 (socket says 172.21.15.32:53666) 2023-09-30T22:21:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.992+0000 7f3b8c862700 1 -- 172.21.15.32:0/2154333431 learned_addr learned my addr 172.21.15.32:0/2154333431 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b8d063700 1 -- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b881275c0 msgr2=0x7f3b8812fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b8d063700 1 --2- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b881275c0 0x7f3b8812fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b8d063700 1 -- 172.21.15.32:0/2154333431 >> v1:172.21.15.32:6789/0 conn(0x7f3b880926d0 legacy=0x7f3b88127be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b877fe700 1 -- 172.21.15.32:0/2154333431 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722506245 0 0) 0x7f3b88091c80 con 0x7f3b880926d0 2023-09-30T22:21:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b8d063700 1 -- 172.21.15.32:0/2154333431 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8809a280 con 0x7f3b88130220 2023-09-30T22:21:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b87fff700 1 --2- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b881275c0 0x7f3b8812fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b8d063700 1 --2- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b88130220 0x7f3b88132610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3b80009e60 tx=0x7f3b80009780).ready entity=mon.0 client_cookie=bee46ca691a3ae46 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b877fe700 1 -- 172.21.15.32:0/2154333431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000e070 con 0x7f3b88130220 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b877fe700 1 -- 172.21.15.32:0/2154333431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000bb60 con 0x7f3b88130220 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.993+0000 7f3b877fe700 1 -- 172.21.15.32:0/2154333431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80012410 con 0x7f3b88130220 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 -- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b88130220 msgr2=0x7f3b88132610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b88130220 0x7f3b88132610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f3b80009e60 tx=0x7f3b80009780).stop 2023-09-30T22:21:29.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 -- 172.21.15.32:0/2154333431 shutdown_connections 2023-09-30T22:21:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b88130220 0x7f3b88132610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/2154333431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b881275c0 0x7f3b8812fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 -- 172.21.15.32:0/2154333431 >> 172.21.15.32:0/2154333431 conn(0x7f3b88097480 msgr2=0x7f3b88097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 -- 172.21.15.32:0/2154333431 shutdown_connections 2023-09-30T22:21:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.994+0000 7f3b8eac6700 1 -- 172.21.15.32:0/2154333431 wait complete. 2023-09-30T22:21:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 Processor -- start 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 -- start start 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b880926d0 0x7f3b8812ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b881275c0 0x7f3b8812e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 0x7f3b8812e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8812ec90 con 0x7f3b881275c0 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b881124a0 con 0x7f3b880926d0 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8eac6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88112170 con 0x7f3b88130220 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8c862700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b880926d0 0x7f3b8812ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b8d063700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 0x7f3b8812e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 0x7f3b8812e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38480/0 (socket says 172.21.15.32:38480) 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 -- 172.21.15.32:0/1088863373 learned_addr learned my addr 172.21.15.32:0/1088863373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8c862700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b880926d0 0x7f3b8812ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42034/0 (socket says 172.21.15.32:42034) 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.995+0000 7f3b87fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b881275c0 0x7f3b8812e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 -- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b880926d0 msgr2=0x7f3b8812ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b880926d0 0x7f3b8812ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 -- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b881275c0 msgr2=0x7f3b8812e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b881275c0 0x7f3b8812e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 -- 172.21.15.32:0/1088863373 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b700092a0 con 0x7f3b88130220 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b87fff700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b881275c0 0x7f3b8812e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8d063700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 0x7f3b8812e7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3b80011000 tx=0x7f3b8809a410).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80012410 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8001d630 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b80009a20 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8001d9f0 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.996+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b88099460 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.997+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3b80003040 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.997+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2461+0+0 (secure 0 0 0) 0x7f3b8003c8a0 con 0x7f3b88130220 2023-09-30T22:21:30.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:29.997+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8809a4b0 con 0x7f3b88130220 2023-09-30T22:21:30.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.002+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3b80025bb0 con 0x7f3b88130220 2023-09-30T22:21:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.177+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3b8805a260 con 0x7f3b88130220 2023-09-30T22:21:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.790+0000 7f3b84ff9700 1 -- 172.21.15.32:0/1088863373 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v11) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3b80011e60 con 0x7f3b88130220 2023-09-30T22:21:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 >> v1:172.21.15.32:6816/118188 conn(0x7f3b6802bb60 legacy=0x7f3b6802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 msgr2=0x7f3b8812e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 0x7f3b8812e7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3b80011000 tx=0x7f3b8809a410).stop 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 shutdown_connections 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b88130220 0x7f3b8812e7a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b880926d0 0x7f3b8812ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.794+0000 7f3b8eac6700 1 --2- 172.21.15.32:0/1088863373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b881275c0 0x7f3b8812e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.795+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 >> 172.21.15.32:0/1088863373 conn(0x7f3b88097480 msgr2=0x7f3b88092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.795+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 shutdown_connections 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.795+0000 7f3b8eac6700 1 -- 172.21.15.32:0/1088863373 wait complete. 2023-09-30T22:21:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'rbd' created 2023-09-30T22:21:30.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-30T22:21:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.946+0000 7f685a5c5700 1 Processor -- start 2023-09-30T22:21:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.946+0000 7f685a5c5700 1 -- start start 2023-09-30T22:21:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f685a5c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f685a5c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f68541306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f685a5c5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685406a770 con 0x7f6854096900 2023-09-30T22:21:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f685a5c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854069750 con 0x7f685412e2c0 2023-09-30T22:21:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f685a5c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6854069a30 con 0x7f6854069370 2023-09-30T22:21:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6858b62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f68541306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854096410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40298/0 (socket says 172.21.15.32:40298) 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f68537fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6854096900 0x7f6854096ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50992/0 (socket says 172.21.15.32:50992) 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6858b62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f68541306b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37070/0 (socket says 172.21.15.32:37070) 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f68537fe700 1 -- 172.21.15.32:0/4043103992 learned_addr learned my addr 172.21.15.32:0/4043103992 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 -- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 msgr2=0x7f68541306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 --2- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f68541306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6852ffd700 1 -- 172.21.15.32:0/4043103992 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805017055 0 0) 0x7f685406a770 con 0x7f6854096900 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 -- 172.21.15.32:0/4043103992 >> v1:172.21.15.32:6789/0 conn(0x7f6854096900 legacy=0x7f6854096ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 -- 172.21.15.32:0/4043103992 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854097810 con 0x7f6854069370 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6858b62700 1 --2- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f68541306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.947+0000 7f6853fff700 1 --2- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854096410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f683c009790 tx=0x7f683c009060).ready entity=mon.2 client_cookie=d2cbaabf274280ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f6852ffd700 1 -- 172.21.15.32:0/4043103992 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00f510 con 0x7f6854069370 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f6852ffd700 1 -- 172.21.15.32:0/4043103992 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00f670 con 0x7f6854069370 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f6852ffd700 1 -- 172.21.15.32:0/4043103992 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00f9f0 con 0x7f6854069370 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 -- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 msgr2=0x7f6854096410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 --2- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854096410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f683c009790 tx=0x7f683c009060).stop 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 -- 172.21.15.32:0/4043103992 shutdown_connections 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 --2- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f68541306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 --2- 172.21.15.32:0/4043103992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854096410 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 -- 172.21.15.32:0/4043103992 >> 172.21.15.32:0/4043103992 conn(0x7f6854059cd0 msgr2=0x7f685409bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 -- 172.21.15.32:0/4043103992 shutdown_connections 2023-09-30T22:21:30.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.948+0000 7f685a5c5700 1 -- 172.21.15.32:0/4043103992 wait complete. 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.949+0000 7f685a5c5700 1 Processor -- start 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.949+0000 7f685a5c5700 1 -- start start 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f685a5c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854117f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f685a5c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6854096900 0x7f6854118480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f685a5c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f6854118970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f685a5c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854118e60 con 0x7f6854096900 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f685a5c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854119400 con 0x7f685412e2c0 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f685a5c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f685411b490 con 0x7f6854069370 2023-09-30T22:21:30.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854117f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f68537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6854096900 0x7f6854118480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854117f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40302/0 (socket says 172.21.15.32:40302) 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 -- 172.21.15.32:0/2740981734 learned_addr learned my addr 172.21.15.32:0/2740981734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6858b62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f6854118970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 -- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 msgr2=0x7f6854118970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f6854118970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 -- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6854096900 msgr2=0x7f6854118480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6854096900 0x7f6854118480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 -- 172.21.15.32:0/2740981734 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009a20 con 0x7f6854069370 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.950+0000 7f6853fff700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854117f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6854069bc0 tx=0x7f683c017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.951+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00fe80 con 0x7f6854069370 2023-09-30T22:21:30.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.951+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00f7e0 con 0x7f6854069370 2023-09-30T22:21:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.951+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c0092a0 con 0x7f6854069370 2023-09-30T22:21:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.951+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c01b670 con 0x7f6854069370 2023-09-30T22:21:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.951+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6854067bb0 con 0x7f6854069370 2023-09-30T22:21:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.952+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f683c01ba60 con 0x7f6854069370 2023-09-30T22:21:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.952+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2827+0+0 (secure 0 0 0) 0x7f683c03c2b0 con 0x7f6854069370 2023-09-30T22:21:30.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.952+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68540677e0 con 0x7f6854069370 2023-09-30T22:21:30.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:30.957+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f683c028020 con 0x7f6854069370 2023-09-30T22:21:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.132+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f685405a260 con 0x7f6854069370 2023-09-30T22:21:31.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.790+0000 7f684bfff700 1 -- 172.21.15.32:0/2740981734 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v12) v1 ==== 173+0+0 (secure 0 0 0) 0x7f683c01a3a0 con 0x7f6854069370 2023-09-30T22:21:31.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 >> v1:172.21.15.32:6816/118188 conn(0x7f683402bb60 legacy=0x7f683402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 msgr2=0x7f6854117f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854117f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f6854069bc0 tx=0x7f683c017000).stop 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 shutdown_connections 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f685412e2c0 0x7f6854118970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6854069370 0x7f6854117f90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 --2- 172.21.15.32:0/2740981734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6854096900 0x7f6854118480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.795+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 >> 172.21.15.32:0/2740981734 conn(0x7f6854059cd0 msgr2=0x7f685409a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.796+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 shutdown_connections 2023-09-30T22:21:31.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.797+0000 7f685a5c5700 1 -- 172.21.15.32:0/2740981734 wait complete. 2023-09-30T22:21:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-30T22:21:31.814 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-30T22:21:31.814 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-30T22:21:31.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-30T22:21:31.816 INFO:tasks.ceph.mds.a:Started 2023-09-30T22:21:31.816 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-30T22:21:31.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:31.905 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-09-30T22:21:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.953+0000 7f824bdc6700 1 Processor -- start 2023-09-30T22:21:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.953+0000 7f824bdc6700 1 -- start start 2023-09-30T22:21:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824bdc6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f8244127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824bdc6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8244127390 0x7f8244127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824bdc6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f82440957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824bdc6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8244096940 con 0x7f8244127390 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824bdc6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244059c00 con 0x7f8244095370 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824bdc6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8244091b60 con 0x7f8244059070 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f8249b62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f8244127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f82440957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f8249361700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8244127390 0x7f8244127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f82440957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37088/0 (socket says 172.21.15.32:37088) 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f8249b62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f8244127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40306/0 (socket says 172.21.15.32:40306) 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 -- 172.21.15.32:0/661432838 learned_addr learned my addr 172.21.15.32:0/661432838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 -- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 msgr2=0x7f8244127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 --2- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f8244127890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 -- 172.21.15.32:0/661432838 >> v2:172.21.15.32:3300/0 conn(0x7f8244127390 msgr2=0x7f8244127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 --2- 172.21.15.32:0/661432838 >> v2:172.21.15.32:3300/0 conn(0x7f8244127390 0x7f8244127d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f824a363700 1 -- 172.21.15.32:0/661432838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440929d0 con 0x7f8244095370 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f8249b62700 1 --2- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f8244127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.954+0000 7f8249361700 1 --2- 172.21.15.32:0/661432838 >> v2:172.21.15.32:3300/0 conn(0x7f8244127390 0x7f8244127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824a363700 1 --2- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f82440957f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f823c009f20 tx=0x7f823c009840).ready entity=mon.1 client_cookie=df2f35092bcca6ab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f8248b60700 1 -- 172.21.15.32:0/661432838 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c00e070 con 0x7f8244095370 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f8248b60700 1 -- 172.21.15.32:0/661432838 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c00b970 con 0x7f8244095370 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f8248b60700 1 -- 172.21.15.32:0/661432838 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c00bcf0 con 0x7f8244095370 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 -- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 msgr2=0x7f82440957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 --2- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f82440957f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f823c009f20 tx=0x7f823c009840).stop 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 -- 172.21.15.32:0/661432838 shutdown_connections 2023-09-30T22:21:31.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 --2- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f82440957f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 --2- 172.21.15.32:0/661432838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f8244127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 --2- 172.21.15.32:0/661432838 >> v2:172.21.15.32:3300/0 conn(0x7f8244127390 0x7f8244127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 -- 172.21.15.32:0/661432838 >> 172.21.15.32:0/661432838 conn(0x7f8244097470 msgr2=0x7f8244097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 -- 172.21.15.32:0/661432838 shutdown_connections 2023-09-30T22:21:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.955+0000 7f824bdc6700 1 -- 172.21.15.32:0/661432838 wait complete. 2023-09-30T22:21:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.956+0000 7f824bdc6700 1 Processor -- start 2023-09-30T22:21:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.956+0000 7f824bdc6700 1 -- start start 2023-09-30T22:21:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824bdc6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f82441175b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824bdc6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f824413a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824bdc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244127390 0x7f824413a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:31.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824bdc6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244116b10 con 0x7f8244127390 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824bdc6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244124f20 con 0x7f8244095370 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824bdc6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8244117290 con 0x7f8244059070 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f82441175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f82441175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40318/0 (socket says 172.21.15.32:40318) 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 -- 172.21.15.32:0/287190053 learned_addr learned my addr 172.21.15.32:0/287190053 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249361700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f824413a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f824a363700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244127390 0x7f824413a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 -- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 msgr2=0x7f824413a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f824413a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 -- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244127390 msgr2=0x7f824413a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244127390 0x7f824413a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 -- 172.21.15.32:0/287190053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400092a0 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.957+0000 7f8249b62700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f82441175b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f824000ea20 tx=0x7f8240009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.958+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824000ee60 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.958+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8240009600 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.958+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82340092a0 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.958+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824000ca60 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.958+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c009ae0 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.958+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f824001b400 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.959+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f824003b110 con 0x7f8244059070 2023-09-30T22:21:31.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.959+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824405a260 con 0x7f8244059070 2023-09-30T22:21:31.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:31.964+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8240016020 con 0x7f8244059070 2023-09-30T22:21:32.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.139+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8244046c20 con 0x7f8244059070 2023-09-30T22:21:32.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.139+0000 7f8239ffb700 1 -- 172.21.15.32:0/287190053 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+8137 (secure 0 0 0) 0x7f8240011d20 con 0x7f8244059070 2023-09-30T22:21:32.140 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:32.140 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":12,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:31.787883+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:32.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 >> v1:172.21.15.32:6816/118188 conn(0x7f822802bb60 legacy=0x7f822802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 msgr2=0x7f82441175b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f82441175b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f824000ea20 tx=0x7f8240009060).stop 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 shutdown_connections 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244127390 0x7f824413a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8244059070 0x7f82441175b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 --2- 172.21.15.32:0/287190053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8244095370 0x7f824413a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 >> 172.21.15.32:0/287190053 conn(0x7f8244097470 msgr2=0x7f824412a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 shutdown_connections 2023-09-30T22:21:32.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.141+0000 7f824bdc6700 1 -- 172.21.15.32:0/287190053 wait complete. 2023-09-30T22:21:32.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-30T22:21:32.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:21:32.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.288+0000 7f65e46bc700 1 Processor -- start 2023-09-30T22:21:32.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.288+0000 7f65e46bc700 1 -- start start 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.288+0000 7f65e46bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc098ea0 0x7f65dc099280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e46bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65dc12d160 0x7f65dc12f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e46bc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc092d80 con 0x7f65dc093840 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e46bc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc093c20 con 0x7f65dc12d160 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e46bc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc092b90 con 0x7f65dc098ea0 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2c59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f65dc12d160 0x7f65dc12f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e1c57700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc098ea0 0x7f65dc099280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2458700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65dc093840 0x7f65dc098790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51006/0 (socket says 172.21.15.32:51006) 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2458700 1 -- 172.21.15.32:0/3843414272 learned_addr learned my addr 172.21.15.32:0/3843414272 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2c59700 1 -- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc098ea0 msgr2=0x7f65dc099280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2c59700 1 --2- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc098ea0 0x7f65dc099280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2c59700 1 -- 172.21.15.32:0/3843414272 >> v1:172.21.15.32:6789/0 conn(0x7f65dc093840 legacy=0x7f65dc098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e1456700 1 -- 172.21.15.32:0/3843414272 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488899474 0 0) 0x7f65dc092d80 con 0x7f65dc093840 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2c59700 1 -- 172.21.15.32:0/3843414272 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc099b70 con 0x7f65dc12d160 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e1c57700 1 --2- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc098ea0 0x7f65dc099280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e2c59700 1 --2- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc12f550 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f65d4009e90 tx=0x7f65d4009780).ready entity=mon.0 client_cookie=43c3d0a9ca62599a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e1456700 1 -- 172.21.15.32:0/3843414272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400e070 con 0x7f65dc12d160 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e1456700 1 -- 172.21.15.32:0/3843414272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400bb20 con 0x7f65dc12d160 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.289+0000 7f65e1456700 1 -- 172.21.15.32:0/3843414272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400bea0 con 0x7f65dc12d160 2023-09-30T22:21:32.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 -- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 msgr2=0x7f65dc12f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 --2- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc12f550 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f65d4009e90 tx=0x7f65d4009780).stop 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 -- 172.21.15.32:0/3843414272 shutdown_connections 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 --2- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc12f550 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 --2- 172.21.15.32:0/3843414272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc098ea0 0x7f65dc099280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 -- 172.21.15.32:0/3843414272 >> 172.21.15.32:0/3843414272 conn(0x7f65dc097480 msgr2=0x7f65dc097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 -- 172.21.15.32:0/3843414272 shutdown_connections 2023-09-30T22:21:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.290+0000 7f65e46bc700 1 -- 172.21.15.32:0/3843414272 wait complete. 2023-09-30T22:21:32.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.291+0000 7f65e46bc700 1 Processor -- start 2023-09-30T22:21:32.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.291+0000 7f65e46bc700 1 -- start start 2023-09-30T22:21:32.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 0x7f65dc12cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f65dc098ea0 0x7f65dc115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc095f70 con 0x7f65dc12d160 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc06a520 con 0x7f65dc093840 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc119bb0 con 0x7f65dc098ea0 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 0x7f65dc12cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2c59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e1c57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f65dc098ea0 0x7f65dc115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 0x7f65dc12cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37130/0 (socket says 172.21.15.32:37130) 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2c59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48984/0 (socket says 172.21.15.32:48984) 2023-09-30T22:21:32.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 -- 172.21.15.32:0/2794150642 learned_addr learned my addr 172.21.15.32:0/2794150642 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 -- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f65dc098ea0 msgr2=0x7f65dc115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f65dc098ea0 0x7f65dc115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 -- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 msgr2=0x7f65dc116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 -- 172.21.15.32:0/2794150642 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0092a0 con 0x7f65dc093840 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2c59700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e1c57700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f65dc098ea0 0x7f65dc115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e2458700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 0x7f65dc12cbe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f65d800c4f0 tx=0x7f65d8007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800ce20 con 0x7f65dc093840 2023-09-30T22:21:32.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80072e0 con 0x7f65dc093840 2023-09-30T22:21:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4009a20 con 0x7f65dc093840 2023-09-30T22:21:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.292+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80057a0 con 0x7f65dc093840 2023-09-30T22:21:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.293+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc116fa0 con 0x7f65dc093840 2023-09-30T22:21:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.293+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f65d8005bd0 con 0x7f65dc093840 2023-09-30T22:21:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.293+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f65d803a2c0 con 0x7f65dc093840 2023-09-30T22:21:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.294+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc119210 con 0x7f65dc093840 2023-09-30T22:21:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.298+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f65d801d3f0 con 0x7f65dc093840 2023-09-30T22:21:32.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.472+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f65dc05a260 con 0x7f65dc093840 2023-09-30T22:21:32.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.472+0000 7f65d27fc700 1 -- 172.21.15.32:0/2794150642 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v12) v1 ==== 110+0+10 (secure 0 0 0) 0x7f65d801c350 con 0x7f65dc093840 2023-09-30T22:21:32.473 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:21:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 >> v1:172.21.15.32:6816/118188 conn(0x7f65c002bb60 legacy=0x7f65c002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 msgr2=0x7f65dc12cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 0x7f65dc12cbe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f65d800c4f0 tx=0x7f65d8007c10).stop 2023-09-30T22:21:32.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 shutdown_connections 2023-09-30T22:21:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc12d160 0x7f65dc116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc093840 0x7f65dc12cbe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 --2- 172.21.15.32:0/2794150642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f65dc098ea0 0x7f65dc115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 >> 172.21.15.32:0/2794150642 conn(0x7f65dc097480 msgr2=0x7f65dc0691f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 shutdown_connections 2023-09-30T22:21:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.474+0000 7f65e46bc700 1 -- 172.21.15.32:0/2794150642 wait complete. 2023-09-30T22:21:32.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:21:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 Processor -- start 2023-09-30T22:21:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 -- start start 2023-09-30T22:21:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 0x7fa2e8096f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2e812e240 0x7fa2e8130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e8095e60 con 0x7fa2e80694d0 2023-09-30T22:21:32.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e8068d40 con 0x7fa2e812e240 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2ee49a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8068fb0 con 0x7fa2e8096b20 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2e77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 0x7fa2e8096f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2eca37700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2e812e240 0x7fa2e8130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2e7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2e80694d0 0x7fa2e8096410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51018/0 (socket says 172.21.15.32:51018) 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2eca37700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2e812e240 0x7fa2e8130630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48988/0 (socket says 172.21.15.32:48988) 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2e7fff700 1 -- 172.21.15.32:0/3565873464 learned_addr learned my addr 172.21.15.32:0/3565873464 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.620+0000 7fa2e77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 0x7fa2e8096f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37136/0 (socket says 172.21.15.32:37136) 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2eca37700 1 -- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 msgr2=0x7fa2e8096f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2eca37700 1 --2- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 0x7fa2e8096f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2eca37700 1 -- 172.21.15.32:0/3565873464 >> v1:172.21.15.32:6789/0 conn(0x7fa2e80694d0 legacy=0x7fa2e8096410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2e6ffd700 1 -- 172.21.15.32:0/3565873464 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826186977 0 0) 0x7fa2e8095e60 con 0x7fa2e80694d0 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2eca37700 1 -- 172.21.15.32:0/3565873464 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8068a10 con 0x7fa2e812e240 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2e77fe700 1 --2- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 0x7fa2e8096f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2eca37700 1 --2- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8130630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa2e0009e60 tx=0x7fa2e0009780).ready entity=mon.0 client_cookie=7e7abe4d5ede74f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2e6ffd700 1 -- 172.21.15.32:0/3565873464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000e070 con 0x7fa2e812e240 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2e6ffd700 1 -- 172.21.15.32:0/3565873464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e000bb60 con 0x7fa2e812e240 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.621+0000 7fa2e6ffd700 1 -- 172.21.15.32:0/3565873464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0012410 con 0x7fa2e812e240 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 -- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 msgr2=0x7fa2e8130630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8130630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa2e0009e60 tx=0x7fa2e0009780).stop 2023-09-30T22:21:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 -- 172.21.15.32:0/3565873464 shutdown_connections 2023-09-30T22:21:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8130630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/3565873464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e8096b20 0x7fa2e8096f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 -- 172.21.15.32:0/3565873464 >> 172.21.15.32:0/3565873464 conn(0x7fa2e809ba60 msgr2=0x7fa2e809be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 -- 172.21.15.32:0/3565873464 shutdown_connections 2023-09-30T22:21:32.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.622+0000 7fa2ee49a700 1 -- 172.21.15.32:0/3565873464 wait complete. 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 Processor -- start 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 -- start start 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e80694d0 0x7fa2e811dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2e8096b20 0x7fa2e8064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e812dcc0 con 0x7fa2e812e240 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e811d6c0 con 0x7fa2e80694d0 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2ee49a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8091b10 con 0x7fa2e8096b20 2023-09-30T22:21:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2e7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e80694d0 0x7fa2e811dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2e7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e80694d0 0x7fa2e811dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37152/0 (socket says 172.21.15.32:37152) 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2e7fff700 1 -- 172.21.15.32:0/1108536747 learned_addr learned my addr 172.21.15.32:0/1108536747 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2e77fe700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2e8096b20 0x7fa2e8064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.623+0000 7fa2eca37700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2e77fe700 1 -- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e80694d0 msgr2=0x7fa2e811dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2e77fe700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e80694d0 0x7fa2e811dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2e77fe700 1 -- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 msgr2=0x7fa2e8064a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2e77fe700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8064a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2e77fe700 1 -- 172.21.15.32:0/1108536747 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80092a0 con 0x7fa2e8096b20 2023-09-30T22:21:32.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2eca37700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2e77fe700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2e8096b20 0x7fa2e8064540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa2d800e950 tx=0x7fa2d8009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8009580 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8012600 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e0009a20 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80129a0 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.624+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e8066290 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.625+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fa2d8012d10 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.625+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fa2d803b0f0 con 0x7fa2e8096b20 2023-09-30T22:21:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.625+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e811d500 con 0x7fa2e8096b20 2023-09-30T22:21:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.630+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fa2d8019310 con 0x7fa2e8096b20 2023-09-30T22:21:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.807+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2e805a260 con 0x7fa2e8096b20 2023-09-30T22:21:32.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.808+0000 7fa2dffff700 1 -- 172.21.15.32:0/1108536747 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2d802a0a0 con 0x7fa2e8096b20 2023-09-30T22:21:32.809 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:21:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 >> v1:172.21.15.32:6816/118188 conn(0x7fa2c802bb60 legacy=0x7fa2c802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2e8096b20 msgr2=0x7fa2e8064540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2e8096b20 0x7fa2e8064540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa2d800e950 tx=0x7fa2d8009060).stop 2023-09-30T22:21:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 shutdown_connections 2023-09-30T22:21:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2e812e240 0x7fa2e8064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2e80694d0 0x7fa2e811dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 --2- 172.21.15.32:0/1108536747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2e8096b20 0x7fa2e8064540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 >> 172.21.15.32:0/1108536747 conn(0x7fa2e809ba60 msgr2=0x7fa2e8094a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 shutdown_connections 2023-09-30T22:21:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.810+0000 7fa2ee49a700 1 -- 172.21.15.32:0/1108536747 wait complete. 2023-09-30T22:21:32.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:32.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.954+0000 7fe406e77700 1 Processor -- start 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.954+0000 7fe406e77700 1 -- start start 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe406e77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4000613a0 0x7fe40012ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe406e77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe40012b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe406e77700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40012abd0 con 0x7fe400095df0 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe406e77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe400061780 con 0x7fe4000613a0 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe406e77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe400067530 con 0x7fe40012b2f0 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe40012b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe404c13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4000613a0 0x7fe40012ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe405414700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe400095df0 0x7fe4000962a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51032/0 (socket says 172.21.15.32:51032) 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe40012b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37166/0 (socket says 172.21.15.32:37166) 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe404c13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4000613a0 0x7fe40012ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49010/0 (socket says 172.21.15.32:49010) 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe405414700 1 -- 172.21.15.32:0/1362208643 learned_addr learned my addr 172.21.15.32:0/1362208643 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.955+0000 7fe404c13700 1 -- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 msgr2=0x7fe40012b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe404c13700 1 --2- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe40012b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe404c13700 1 -- 172.21.15.32:0/1362208643 >> v1:172.21.15.32:6789/0 conn(0x7fe400095df0 legacy=0x7fe4000962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe3ff7fe700 1 -- 172.21.15.32:0/1362208643 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129280945 0 0) 0x7fe40012abd0 con 0x7fe400095df0 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe404c13700 1 -- 172.21.15.32:0/1362208643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000683c0 con 0x7fe4000613a0 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe404c13700 1 --2- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4000613a0 0x7fe40012ae00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe3f0009760 tx=0x7fe3f0009060).ready entity=mon.0 client_cookie=749ef5240d660846 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe3ff7fe700 1 -- 172.21.15.32:0/1362208643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000f510 con 0x7fe4000613a0 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe3ff7fe700 1 -- 172.21.15.32:0/1362208643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000f670 con 0x7fe4000613a0 2023-09-30T22:21:32.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe3ff7fe700 1 -- 172.21.15.32:0/1362208643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000f9f0 con 0x7fe4000613a0 2023-09-30T22:21:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe406e77700 1 -- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4000613a0 msgr2=0x7fe40012ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.956+0000 7fe406e77700 1 --2- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4000613a0 0x7fe40012ae00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe3f0009760 tx=0x7fe3f0009060).stop 2023-09-30T22:21:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 -- 172.21.15.32:0/1362208643 shutdown_connections 2023-09-30T22:21:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 --2- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4000613a0 0x7fe40012ae00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 --2- 172.21.15.32:0/1362208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe40012b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 -- 172.21.15.32:0/1362208643 >> 172.21.15.32:0/1362208643 conn(0x7fe40009a220 msgr2=0x7fe40009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 -- 172.21.15.32:0/1362208643 shutdown_connections 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 -- 172.21.15.32:0/1362208643 wait complete. 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.957+0000 7fe406e77700 1 Processor -- start 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 -- start start 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4000613a0 0x7fe40013a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 0x7fe40013aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe400113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001161d0 con 0x7fe400095df0 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40012f320 con 0x7fe40012b2f0 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe406e77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe400098c80 con 0x7fe4000613a0 2023-09-30T22:21:32.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 0x7fe40013aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe404c13700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4000613a0 0x7fe40013a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe405414700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe400113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 0x7fe40013aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49020/0 (socket says 172.21.15.32:49020) 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.958+0000 7fe3fffff700 1 -- 172.21.15.32:0/3941209472 learned_addr learned my addr 172.21.15.32:0/3941209472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fffff700 1 -- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4000613a0 msgr2=0x7fe40013a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fffff700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4000613a0 0x7fe40013a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fffff700 1 -- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 msgr2=0x7fe400113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fffff700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe400113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fffff700 1 -- 172.21.15.32:0/3941209472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f00092a0 con 0x7fe400095df0 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe405414700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe400113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fffff700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 0x7fe40013aa50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe3e800e930 tx=0x7fe3e8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8012600 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e8012760 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe400114f40 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8012ae0 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.959+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe400117ea0 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.960+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe3e801b400 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.960+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fe3e803aef0 con 0x7fe400095df0 2023-09-30T22:21:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.960+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe400117ea0 con 0x7fe400095df0 2023-09-30T22:21:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:32.965+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe400117ea0 con 0x7fe400095df0 2023-09-30T22:21:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.179+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe400117ea0 con 0x7fe400095df0 2023-09-30T22:21:33.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.179+0000 7fe3fcff9700 1 -- 172.21.15.32:0/3941209472 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1196 (secure 0 0 0) 0x7fe400117ea0 con 0x7fe400095df0 2023-09-30T22:21:33.180 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:33.181 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-09-30T22:21:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 >> v1:172.21.15.32:6816/118188 conn(0x7fe3e002bb60 legacy=0x7fe3e002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 msgr2=0x7fe40013aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 0x7fe40013aa50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe3e800e930 tx=0x7fe3e8009060).stop 2023-09-30T22:21:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 shutdown_connections 2023-09-30T22:21:33.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe40012b2f0 0x7fe400113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4000613a0 0x7fe40013a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 --2- 172.21.15.32:0/3941209472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe400095df0 0x7fe40013aa50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 >> 172.21.15.32:0/3941209472 conn(0x7fe40009a220 msgr2=0x7fe400065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 shutdown_connections 2023-09-30T22:21:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.181+0000 7fe406e77700 1 -- 172.21.15.32:0/3941209472 wait complete. 2023-09-30T22:21:33.183 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2023-09-30T22:21:33.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:33.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.324+0000 7fba6eed7700 1 Processor -- start 2023-09-30T22:21:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.324+0000 7fba6eed7700 1 -- start start 2023-09-30T22:21:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6eed7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 0x7fba680969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6eed7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 0x7fba68067de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6eed7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68068740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6eed7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba68068c30 con 0x7fba68096eb0 2023-09-30T22:21:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6eed7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba680933b0 con 0x7fba680682d0 2023-09-30T22:21:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6eed7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba68092190 con 0x7fba680965e0 2023-09-30T22:21:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6d474700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6cc73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 0x7fba680969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba67fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 0x7fba68067de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba67fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 0x7fba68067de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49026/0 (socket says 172.21.15.32:49026) 2023-09-30T22:21:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6d474700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68068740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37182/0 (socket says 172.21.15.32:37182) 2023-09-30T22:21:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6cc73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 0x7fba680969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40346/0 (socket says 172.21.15.32:40346) 2023-09-30T22:21:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba67fff700 1 -- 172.21.15.32:0/1830056738 learned_addr learned my addr 172.21.15.32:0/1830056738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6d474700 1 -- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 msgr2=0x7fba680969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6d474700 1 --2- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 0x7fba680969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.325+0000 7fba6d474700 1 -- 172.21.15.32:0/1830056738 >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 msgr2=0x7fba68067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6d474700 1 --2- 172.21.15.32:0/1830056738 >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 0x7fba68067de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6d474700 1 -- 172.21.15.32:0/1830056738 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68069a20 con 0x7fba680682d0 2023-09-30T22:21:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba67fff700 1 --2- 172.21.15.32:0/1830056738 >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 0x7fba68067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6cc73700 1 --2- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 0x7fba680969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6d474700 1 --2- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68068740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fba60009e60 tx=0x7fba60009780).ready entity=mon.1 client_cookie=3a3766b6ad9414ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba677fe700 1 -- 172.21.15.32:0/1830056738 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000e070 con 0x7fba680682d0 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba677fe700 1 -- 172.21.15.32:0/1830056738 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6000bab0 con 0x7fba680682d0 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba677fe700 1 -- 172.21.15.32:0/1830056738 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000be30 con 0x7fba680682d0 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 -- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 msgr2=0x7fba68068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68068740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fba60009e60 tx=0x7fba60009780).stop 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 -- 172.21.15.32:0/1830056738 shutdown_connections 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68068740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1830056738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba680965e0 0x7fba680969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1830056738 >> v2:172.21.15.32:3300/0 conn(0x7fba68096eb0 0x7fba68067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.326+0000 7fba6eed7700 1 -- 172.21.15.32:0/1830056738 >> 172.21.15.32:0/1830056738 conn(0x7fba6809a220 msgr2=0x7fba6809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.327+0000 7fba6eed7700 1 -- 172.21.15.32:0/1830056738 shutdown_connections 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.327+0000 7fba6eed7700 1 -- 172.21.15.32:0/1830056738 wait complete. 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.327+0000 7fba6eed7700 1 Processor -- start 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 -- start start 2023-09-30T22:21:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68119ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680965e0 0x7fba68119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba68096eb0 0x7fba6811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6811a9b0 con 0x7fba680965e0 2023-09-30T22:21:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba68099b40 con 0x7fba680682d0 2023-09-30T22:21:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6eed7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba68091bf0 con 0x7fba68096eb0 2023-09-30T22:21:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6cc73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68119ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6d474700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba68096eb0 0x7fba6811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6cc73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68119ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37190/0 (socket says 172.21.15.32:37190) 2023-09-30T22:21:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6d474700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba68096eb0 0x7fba6811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40350/0 (socket says 172.21.15.32:40350) 2023-09-30T22:21:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba6cc73700 1 -- 172.21.15.32:0/1613504018 learned_addr learned my addr 172.21.15.32:0/1613504018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.328+0000 7fba67fff700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680965e0 0x7fba68119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6cc73700 1 -- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba68096eb0 msgr2=0x7fba6811a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6cc73700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba68096eb0 0x7fba6811a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6cc73700 1 -- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680965e0 msgr2=0x7fba68119fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6cc73700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680965e0 0x7fba68119fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6cc73700 1 -- 172.21.15.32:0/1613504018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580092a0 con 0x7fba680682d0 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6cc73700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68119ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fba5800ea20 tx=0x7fba58009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5800ee60 con 0x7fba680682d0 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5800c850 con 0x7fba680682d0 2023-09-30T22:21:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5000b610 con 0x7fba680682d0 2023-09-30T22:21:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5800cbd0 con 0x7fba680682d0 2023-09-30T22:21:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.329+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba60009a20 con 0x7fba680682d0 2023-09-30T22:21:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.330+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fba5801b400 con 0x7fba680682d0 2023-09-30T22:21:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.330+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fba5803b230 con 0x7fba680682d0 2023-09-30T22:21:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.330+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6805a260 con 0x7fba680682d0 2023-09-30T22:21:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.335+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fba58019900 con 0x7fba680682d0 2023-09-30T22:21:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.511+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba68069ce0 con 0x7fba680682d0 2023-09-30T22:21:33.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.511+0000 7fba64ff9700 1 -- 172.21.15.32:0/1613504018 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8124 (secure 0 0 0) 0x7fba5802a090 con 0x7fba680682d0 2023-09-30T22:21:33.512 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:33.513 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:32.789139+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 >> v1:172.21.15.32:6816/118188 conn(0x7fba4802bb90 legacy=0x7fba4802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 msgr2=0x7fba68119ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68119ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fba5800ea20 tx=0x7fba58009060).stop 2023-09-30T22:21:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 shutdown_connections 2023-09-30T22:21:33.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba68096eb0 0x7fba6811a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba680682d0 0x7fba68119ae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 --2- 172.21.15.32:0/1613504018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba680965e0 0x7fba68119fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.513+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 >> 172.21.15.32:0/1613504018 conn(0x7fba6809a220 msgr2=0x7fba68095b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.514+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 shutdown_connections 2023-09-30T22:21:33.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.514+0000 7fba6eed7700 1 -- 172.21.15.32:0/1613504018 wait complete. 2023-09-30T22:21:33.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-30T22:21:33.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:21:33.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.660+0000 7f6463c56700 1 Processor -- start 2023-09-30T22:21:33.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.660+0000 7f6463c56700 1 -- start start 2023-09-30T22:21:33.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f6463c56700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c12bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f6463c56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f645c0925d0 0x7f645c12ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f6463c56700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c09a050 con 0x7f645c12f320 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f6463c56700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c06a750 con 0x7f645c0925d0 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f6463c56700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f645c12c1a0 con 0x7f645c12c3c0 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64611f1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f645c0925d0 0x7f645c12ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64619f2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c12bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64621f3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f645c12f320 0x7f645c136130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51034/0 (socket says 172.21.15.32:51034) 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64621f3700 1 -- 172.21.15.32:0/1062015487 learned_addr learned my addr 172.21.15.32:0/1062015487 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64611f1700 1 -- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 msgr2=0x7f645c12bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64611f1700 1 --2- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c12bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64611f1700 1 -- 172.21.15.32:0/1062015487 >> v1:172.21.15.32:6789/0 conn(0x7f645c12f320 legacy=0x7f645c136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64611f1700 1 -- 172.21.15.32:0/1062015487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c091ed0 con 0x7f645c0925d0 2023-09-30T22:21:33.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64609f0700 1 -- 172.21.15.32:0/1062015487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470107111 0 0) 0x7f645c09a050 con 0x7f645c12f320 2023-09-30T22:21:33.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.661+0000 7f64611f1700 1 --2- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c12ee30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f644c009760 tx=0x7f644c009060).ready entity=mon.0 client_cookie=b49da0b6ded0e05b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:33.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f64609f0700 1 -- 172.21.15.32:0/1062015487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c00f510 con 0x7f645c0925d0 2023-09-30T22:21:33.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f64609f0700 1 -- 172.21.15.32:0/1062015487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c00f670 con 0x7f645c0925d0 2023-09-30T22:21:33.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f64609f0700 1 -- 172.21.15.32:0/1062015487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c00f9f0 con 0x7f645c0925d0 2023-09-30T22:21:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 -- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 msgr2=0x7f645c12ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 --2- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c12ee30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f644c009760 tx=0x7f644c009060).stop 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 -- 172.21.15.32:0/1062015487 shutdown_connections 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 --2- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c12bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 --2- 172.21.15.32:0/1062015487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c12ee30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 -- 172.21.15.32:0/1062015487 >> 172.21.15.32:0/1062015487 conn(0x7f645c097480 msgr2=0x7f645c097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 -- 172.21.15.32:0/1062015487 shutdown_connections 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.662+0000 7f6463c56700 1 -- 172.21.15.32:0/1062015487 wait complete. 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.663+0000 7f6463c56700 1 Processor -- start 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.663+0000 7f6463c56700 1 -- start start 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6463c56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c11ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6463c56700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c11a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6463c56700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645c12f320 0x7f645c11abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6463c56700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c09a280 con 0x7f645c0925d0 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6463c56700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c064fb0 con 0x7f645c12f320 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6463c56700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f645c11a440 con 0x7f645c12c3c0 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c11ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c11ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49056/0 (socket says 172.21.15.32:49056) 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64621f3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645c12f320 0x7f645c11abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 -- 172.21.15.32:0/3520129848 learned_addr learned my addr 172.21.15.32:0/3520129848 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64611f1700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c11a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 -- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 msgr2=0x7f645c11a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c11a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 -- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645c12f320 msgr2=0x7f645c11abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645c12f320 0x7f645c11abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 -- 172.21.15.32:0/3520129848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0092a0 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64611f1700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c11a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64621f3700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645c12f320 0x7f645c11abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f64619f2700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c11ded0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f645800e930 tx=0x7f6458009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.664+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458009510 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.665+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6458012600 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.665+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c11d430 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.665+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458012980 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.665+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c11b0c0 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.666+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6458012d30 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.666+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f645803aff0 con 0x7f645c0925d0 2023-09-30T22:21:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.666+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c05a260 con 0x7f645c0925d0 2023-09-30T22:21:33.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.671+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6458011b00 con 0x7f645c0925d0 2023-09-30T22:21:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.846+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f645c12c7a0 con 0x7f645c0925d0 2023-09-30T22:21:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.846+0000 7f6451ffb700 1 -- 172.21.15.32:0/3520129848 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v13) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6458029030 con 0x7f645c0925d0 2023-09-30T22:21:33.847 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:21:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 >> v1:172.21.15.32:6816/118188 conn(0x7f644002bb60 legacy=0x7f644002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 msgr2=0x7f645c11ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c11ded0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f645800e930 tx=0x7f6458009060).stop 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 shutdown_connections 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645c12f320 0x7f645c11abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c0925d0 0x7f645c11ded0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 --2- 172.21.15.32:0/3520129848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645c12c3c0 0x7f645c11a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 >> 172.21.15.32:0/3520129848 conn(0x7f645c097480 msgr2=0x7f645c093340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 shutdown_connections 2023-09-30T22:21:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.848+0000 7f6463c56700 1 -- 172.21.15.32:0/3520129848 wait complete. 2023-09-30T22:21:33.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:21:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.997+0000 7fc6184cb700 1 Processor -- start 2023-09-30T22:21:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.997+0000 7fc6184cb700 1 -- start start 2023-09-30T22:21:34.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc6184cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6100919e0 0x7fc610091dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc6184cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 0x7fc610127990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc6184cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610128330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc6184cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc610128820 con 0x7fc6100919e0 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc6184cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc610092c00 con 0x7fc610127e80 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc6184cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100941a0 con 0x7fc6100922b0 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc616267700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6100919e0 0x7fc610091dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc615a66700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 0x7fc610127990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc616a68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610128330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc615a66700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 0x7fc610127990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40382/0 (socket says 172.21.15.32:40382) 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc616267700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6100919e0 0x7fc610091dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49062/0 (socket says 172.21.15.32:49062) 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.998+0000 7fc615a66700 1 -- 172.21.15.32:0/760506059 learned_addr learned my addr 172.21.15.32:0/760506059 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc616267700 1 -- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 msgr2=0x7fc610127990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc616267700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 0x7fc610127990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc616267700 1 -- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 msgr2=0x7fc610128330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc616267700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610128330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc616267700 1 -- 172.21.15.32:0/760506059 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610094420 con 0x7fc6100919e0 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc615a66700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 0x7fc610127990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc616267700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100919e0 0x7fc610091dc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc60c009790 tx=0x7fc60c009060).ready entity=mon.0 client_cookie=d78e432f9b8427d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc615265700 1 -- 172.21.15.32:0/760506059 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0090d0 con 0x7fc6100919e0 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc615265700 1 -- 172.21.15.32:0/760506059 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c00d4a0 con 0x7fc6100919e0 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc615265700 1 -- 172.21.15.32:0/760506059 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00d820 con 0x7fc6100919e0 2023-09-30T22:21:34.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc6184cb700 1 -- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100919e0 msgr2=0x7fc610091dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:33.999+0000 7fc6184cb700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100919e0 0x7fc610091dc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc60c009790 tx=0x7fc60c009060).stop 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 -- 172.21.15.32:0/760506059 shutdown_connections 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610128330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100919e0 0x7fc610091dc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 --2- 172.21.15.32:0/760506059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100922b0 0x7fc610127990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 -- 172.21.15.32:0/760506059 >> 172.21.15.32:0/760506059 conn(0x7fc61009a1c0 msgr2=0x7fc61009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 -- 172.21.15.32:0/760506059 shutdown_connections 2023-09-30T22:21:34.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.000+0000 7fc6184cb700 1 -- 172.21.15.32:0/760506059 wait complete. 2023-09-30T22:21:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 Processor -- start 2023-09-30T22:21:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 -- start start 2023-09-30T22:21:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 0x7fc610127410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100922b0 0x7fc610119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610119870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61011cf40 con 0x7fc6100922b0 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc610098070 con 0x7fc610127e80 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc6184cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc610093d10 con 0x7fc6100919e0 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc616267700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 0x7fc610127410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc616a68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610119870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc616267700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 0x7fc610127410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40390/0 (socket says 172.21.15.32:40390) 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc616267700 1 -- 172.21.15.32:0/447402483 learned_addr learned my addr 172.21.15.32:0/447402483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.001+0000 7fc615a66700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100922b0 0x7fc610119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616a68700 1 -- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 msgr2=0x7fc610127410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616a68700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 0x7fc610127410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616a68700 1 -- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100922b0 msgr2=0x7fc610119380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616a68700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100922b0 0x7fc610119380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616a68700 1 -- 172.21.15.32:0/447402483 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60000b610 con 0x7fc610127e80 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616267700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 0x7fc610127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc615a66700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100922b0 0x7fc610119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc616a68700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610119870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc608009e80 tx=0x7fc60800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800caa0 con 0x7fc610127e80 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60800f040 con 0x7fc610127e80 2023-09-30T22:21:34.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c0092a0 con 0x7fc610127e80 2023-09-30T22:21:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800ce10 con 0x7fc610127e80 2023-09-30T22:21:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.002+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61011d4e0 con 0x7fc610127e80 2023-09-30T22:21:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.003+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc608013610 con 0x7fc610127e80 2023-09-30T22:21:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.003+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc60803b4a0 con 0x7fc610127e80 2023-09-30T22:21:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.003+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61011ca00 con 0x7fc610127e80 2023-09-30T22:21:34.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.008+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc60801d460 con 0x7fc610127e80 2023-09-30T22:21:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.182+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc61005a260 con 0x7fc610127e80 2023-09-30T22:21:34.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.182+0000 7fc6067fc700 1 -- 172.21.15.32:0/447402483 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6080076e0 con 0x7fc610127e80 2023-09-30T22:21:34.183 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 >> v1:172.21.15.32:6816/118188 conn(0x7fc5f402bb60 legacy=0x7fc5f402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 msgr2=0x7fc610119870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610119870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc608009e80 tx=0x7fc60800b040).stop 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 shutdown_connections 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc610127e80 0x7fc610119870 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6100919e0 0x7fc610127410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 --2- 172.21.15.32:0/447402483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6100922b0 0x7fc610119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 >> 172.21.15.32:0/447402483 conn(0x7fc61009a1c0 msgr2=0x7fc610069050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 shutdown_connections 2023-09-30T22:21:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.184+0000 7fc6184cb700 1 -- 172.21.15.32:0/447402483 wait complete. 2023-09-30T22:21:34.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-30T22:21:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.327+0000 7f9821676700 1 Processor -- start 2023-09-30T22:21:34.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.327+0000 7f9821676700 1 -- start start 2023-09-30T22:21:34.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f9821676700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c097c30 0x7f981c098db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f9821676700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f981c0992a0 0x7f981c132480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f9821676700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c096160 con 0x7f981c0650f0 2023-09-30T22:21:34.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f9821676700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c0654d0 con 0x7f981c0992a0 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f9821676700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f981c065dd0 con 0x7f981c097c30 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f981c0992a0 0x7f981c132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c097c30 0x7f981c098db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f981c0650f0 0x7f981c0986a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51048/0 (socket says 172.21.15.32:51048) 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981affd700 1 -- 172.21.15.32:0/2501124757 learned_addr learned my addr 172.21.15.32:0/2501124757 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c097c30 0x7f981c098db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40394/0 (socket says 172.21.15.32:40394) 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981b7fe700 1 --2- 172.21.15.32:0/2501124757 >> v2:172.21.15.32:3300/0 conn(0x7f981c0992a0 0x7f981c132480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49084/0 (socket says 172.21.15.32:49084) 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981b7fe700 1 -- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c097c30 msgr2=0x7f981c098db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.328+0000 7f981b7fe700 1 --2- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c097c30 0x7f981c098db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f981b7fe700 1 -- 172.21.15.32:0/2501124757 >> v1:172.21.15.32:6789/0 conn(0x7f981c0650f0 legacy=0x7f981c0986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f981b7fe700 1 -- 172.21.15.32:0/2501124757 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c099790 con 0x7f981c0992a0 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9819ffb700 1 -- 172.21.15.32:0/2501124757 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302274631 0 0) 0x7f981c096160 con 0x7f981c0650f0 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f981b7fe700 1 --2- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c132480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9814009760 tx=0x7f9814009060).ready entity=mon.0 client_cookie=559cf2438f57f06d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9819ffb700 1 -- 172.21.15.32:0/2501124757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981400f510 con 0x7f981c0992a0 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9819ffb700 1 -- 172.21.15.32:0/2501124757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981400f670 con 0x7f981c0992a0 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9819ffb700 1 -- 172.21.15.32:0/2501124757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981400f9f0 con 0x7f981c0992a0 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9821676700 1 -- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 msgr2=0x7f981c132480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9821676700 1 --2- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c132480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9814009760 tx=0x7f9814009060).stop 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9821676700 1 -- 172.21.15.32:0/2501124757 shutdown_connections 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9821676700 1 --2- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c132480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9821676700 1 --2- 172.21.15.32:0/2501124757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c097c30 0x7f981c098db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.329+0000 7f9821676700 1 -- 172.21.15.32:0/2501124757 >> 172.21.15.32:0/2501124757 conn(0x7f981c09dc70 msgr2=0x7f981c09e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.330+0000 7f9821676700 1 -- 172.21.15.32:0/2501124757 shutdown_connections 2023-09-30T22:21:34.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.330+0000 7f9821676700 1 -- 172.21.15.32:0/2501124757 wait complete. 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.331+0000 7f9821676700 1 Processor -- start 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.331+0000 7f9821676700 1 -- start start 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f9821676700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 0x7f981c11e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f9821676700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c11a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f9821676700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f981c11a9a0 0x7f981c11ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f9821676700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c11b940 con 0x7f981c0992a0 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f9821676700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c1325f0 con 0x7f981c11a9a0 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f9821676700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f981c11cae0 con 0x7f981c0650f0 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 0x7f981c11e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c11a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 0x7f981c11e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40402/0 (socket says 172.21.15.32:40402) 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c11a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49086/0 (socket says 172.21.15.32:49086) 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 -- 172.21.15.32:0/2317143668 learned_addr learned my addr 172.21.15.32:0/2317143668 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981b7fe700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f981c11a9a0 0x7f981c11ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 -- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f981c11a9a0 msgr2=0x7f981c11ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f981c11a9a0 0x7f981c11ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 -- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 msgr2=0x7f981c11a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c11a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 -- 172.21.15.32:0/2317143668 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140092a0 con 0x7f981c0650f0 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981a7fc700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c11a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.332+0000 7f981affd700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 0x7f981c11e450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f980c00c810 tx=0x7f980c007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.333+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c005d30 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.333+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c005e90 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.333+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c11d9b0 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.333+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c007340 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.333+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c11c040 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.333+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f980c007770 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.334+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f980c03ac50 con 0x7f981c0650f0 2023-09-30T22:21:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.334+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c09c660 con 0x7f981c0650f0 2023-09-30T22:21:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.339+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f980c01c3e0 con 0x7f981c0650f0 2023-09-30T22:21:34.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.514+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f981c09bff0 con 0x7f981c0650f0 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.515+0000 7f9802ffd700 1 -- 172.21.15.32:0/2317143668 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v13) v1 ==== 84+0+143 (secure 0 0 0) 0x7f980c01a300 con 0x7f981c0650f0 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "device_health_metrics" 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:21:34.516 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:21:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 2, 2023-09-30T22:21:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "rbd" 2023-09-30T22:21:34.517 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:21:34.517 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-09-30T22:21:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 >> v1:172.21.15.32:6816/118188 conn(0x7f980802bb60 legacy=0x7f980802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 msgr2=0x7f981c11e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 0x7f981c11e450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f980c00c810 tx=0x7f980c007f90).stop 2023-09-30T22:21:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 shutdown_connections 2023-09-30T22:21:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f981c11a9a0 0x7f981c11ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f981c0650f0 0x7f981c11e450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 --2- 172.21.15.32:0/2317143668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f981c0992a0 0x7f981c11a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 >> 172.21.15.32:0/2317143668 conn(0x7f981c09dc70 msgr2=0x7f981c09a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 shutdown_connections 2023-09-30T22:21:34.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.517+0000 7f9821676700 1 -- 172.21.15.32:0/2317143668 wait complete. 2023-09-30T22:21:34.531 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-30T22:21:34.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-30T22:21:34.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 Processor -- start 2023-09-30T22:21:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 -- start start 2023-09-30T22:21:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f300653f0 0x7f7f30098710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f30099310 0x7f7f30132500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f30065fe0 con 0x7f7f30097ef0 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f300657d0 con 0x7f7f30099310 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f352e3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30095b50 con 0x7f7f300653f0 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f2effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f300653f0 0x7f7f30098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f2f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f30099310 0x7f7f30132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f2e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f30097ef0 0x7f7f30098c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51054/0 (socket says 172.21.15.32:51054) 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.665+0000 7f7f2e7fc700 1 -- 172.21.15.32:0/802042230 learned_addr learned my addr 172.21.15.32:0/802042230 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2dffb700 1 -- 172.21.15.32:0/802042230 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900582843 0 0) 0x7f7f30065fe0 con 0x7f7f30097ef0 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2f7fe700 1 -- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f300653f0 msgr2=0x7f7f30098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:34.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2f7fe700 1 --2- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f300653f0 0x7f7f30098710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2f7fe700 1 -- 172.21.15.32:0/802042230 >> v1:172.21.15.32:6789/0 conn(0x7f7f30097ef0 legacy=0x7f7f30098c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2f7fe700 1 -- 172.21.15.32:0/802042230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f300663b0 con 0x7f7f30099310 2023-09-30T22:21:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2effd700 1 --2- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f300653f0 0x7f7f30098710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2f7fe700 1 --2- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f30099310 0x7f7f30132500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7f28009e90 tx=0x7f7f28009780).ready entity=mon.1 client_cookie=3cbf8c391420b6bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2dffb700 1 -- 172.21.15.32:0/802042230 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800e070 con 0x7f7f30099310 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2dffb700 1 -- 172.21.15.32:0/802042230 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800bb60 con 0x7f7f30099310 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.666+0000 7f7f2dffb700 1 -- 172.21.15.32:0/802042230 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28012410 con 0x7f7f30099310 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 -- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f30099310 msgr2=0x7f7f30132500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 --2- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f30099310 0x7f7f30132500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7f28009e90 tx=0x7f7f28009780).stop 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 -- 172.21.15.32:0/802042230 shutdown_connections 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 --2- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f30099310 0x7f7f30132500 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 --2- 172.21.15.32:0/802042230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f300653f0 0x7f7f30098710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 -- 172.21.15.32:0/802042230 >> 172.21.15.32:0/802042230 conn(0x7f7f3009dcf0 msgr2=0x7f7f3009e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:34.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 -- 172.21.15.32:0/802042230 shutdown_connections 2023-09-30T22:21:34.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.667+0000 7f7f352e3700 1 -- 172.21.15.32:0/802042230 wait complete. 2023-09-30T22:21:34.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 Processor -- start 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 -- start start 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 0x7f7f301321d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f30099310 0x7f7f3011a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f3011a4f0 0x7f7f3011a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3011b4c0 con 0x7f7f300653f0 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3011c6c0 con 0x7f7f3011a4f0 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f352e3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30099da0 con 0x7f7f30099310 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f2effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 0x7f7f301321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f2e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f30099310 0x7f7f3011a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f2effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 0x7f7f301321d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49094/0 (socket says 172.21.15.32:49094) 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f2effd700 1 -- 172.21.15.32:0/2221997980 learned_addr learned my addr 172.21.15.32:0/2221997980 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.668+0000 7f7f2f7fe700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f3011a4f0 0x7f7f3011a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2effd700 1 -- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f30099310 msgr2=0x7f7f3011a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2effd700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f30099310 0x7f7f3011a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2effd700 1 -- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f3011a4f0 msgr2=0x7f7f3011a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2effd700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f3011a4f0 0x7f7f3011a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2effd700 1 -- 172.21.15.32:0/2221997980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f200092a0 con 0x7f7f300653f0 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2e7fc700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f30099310 0x7f7f3011a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2f7fe700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f3011a4f0 0x7f7f3011a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f2effd700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 0x7f7f301321d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7f2000ea20 tx=0x7f7f20009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000ee60 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2000c850 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f28009a20 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000cbd0 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.669+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f301326c0 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.670+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7f2001b400 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.670+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f7f2003b170 con 0x7f7f300653f0 2023-09-30T22:21:34.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.670+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f30132910 con 0x7f7f300653f0 2023-09-30T22:21:34.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.675+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7f20019900 con 0x7f7f300653f0 2023-09-30T22:21:34.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:34.852+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7f3005a260 con 0x7f7f300653f0 2023-09-30T22:21:35.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.777+0000 7f7f16ffd700 1 -- 172.21.15.32:0/2221997980 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v14) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f20019310 con 0x7f7f300653f0 2023-09-30T22:21:35.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.781+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 >> v1:172.21.15.32:6816/118188 conn(0x7f7f1c02bb90 legacy=0x7f7f1c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.781+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 msgr2=0x7f7f301321d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.781+0000 7f7f352e3700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 0x7f7f301321d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7f2000ea20 tx=0x7f7f20009060).stop 2023-09-30T22:21:35.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.782+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 shutdown_connections 2023-09-30T22:21:35.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.782+0000 7f7f352e3700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f3011a4f0 0x7f7f3011a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.782+0000 7f7f352e3700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f300653f0 0x7f7f301321d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.782+0000 7f7f352e3700 1 --2- 172.21.15.32:0/2221997980 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f30099310 0x7f7f3011a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.782+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 >> 172.21.15.32:0/2221997980 conn(0x7f7f3009dcf0 msgr2=0x7f7f3009bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.785+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 shutdown_connections 2023-09-30T22:21:35.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.786+0000 7f7f352e3700 1 -- 172.21.15.32:0/2221997980 wait complete. 2023-09-30T22:21:35.788 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-09-30T22:21:35.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-30T22:21:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.943+0000 7f38f1611700 1 Processor -- start 2023-09-30T22:21:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.943+0000 7f38f1611700 1 -- start start 2023-09-30T22:21:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.943+0000 7f38f1611700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec098710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38f1611700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38ec098c00 0x7f38ec098fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38f1611700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec096370 con 0x7f38ec12f4e0 2023-09-30T22:21:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38f1611700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0981d0 con 0x7f38ec098c00 2023-09-30T22:21:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38f1611700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec099ed0 con 0x7f38ec097df0 2023-09-30T22:21:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38ea7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38ec098c00 0x7f38ec098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec098710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40426/0 (socket says 172.21.15.32:40426) 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 -- 172.21.15.32:0/919807940 learned_addr learned my addr 172.21.15.32:0/919807940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eb7fe700 1 --1- 172.21.15.32:0/919807940 >> v1:172.21.15.32:6789/0 conn(0x7f38ec12f4e0 0x7f38ec1318d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51070/0 (socket says 172.21.15.32:51070) 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 -- 172.21.15.32:0/919807940 >> v2:172.21.15.32:3300/0 conn(0x7f38ec098c00 msgr2=0x7f38ec098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 --2- 172.21.15.32:0/919807940 >> v2:172.21.15.32:3300/0 conn(0x7f38ec098c00 0x7f38ec098fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 -- 172.21.15.32:0/919807940 >> v1:172.21.15.32:6789/0 conn(0x7f38ec12f4e0 legacy=0x7f38ec1318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 -- 172.21.15.32:0/919807940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec094e30 con 0x7f38ec097df0 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38eaffd700 1 --2- 172.21.15.32:0/919807940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec098710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f38dc009830 tx=0x7f38dc0090f0).ready entity=mon.2 client_cookie=3fb7e7a206b1e122 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38e9ffb700 1 -- 172.21.15.32:0/919807940 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00b580 con 0x7f38ec097df0 2023-09-30T22:21:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38e9ffb700 1 -- 172.21.15.32:0/919807940 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00b6e0 con 0x7f38ec097df0 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.944+0000 7f38e9ffb700 1 -- 172.21.15.32:0/919807940 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00ba80 con 0x7f38ec097df0 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 -- 172.21.15.32:0/919807940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 msgr2=0x7f38ec098710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 --2- 172.21.15.32:0/919807940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec098710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f38dc009830 tx=0x7f38dc0090f0).stop 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 -- 172.21.15.32:0/919807940 shutdown_connections 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 --2- 172.21.15.32:0/919807940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec098710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 --2- 172.21.15.32:0/919807940 >> v2:172.21.15.32:3300/0 conn(0x7f38ec098c00 0x7f38ec098fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 -- 172.21.15.32:0/919807940 >> 172.21.15.32:0/919807940 conn(0x7f38ec09dcf0 msgr2=0x7f38ec09e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 -- 172.21.15.32:0/919807940 shutdown_connections 2023-09-30T22:21:35.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.945+0000 7f38f1611700 1 -- 172.21.15.32:0/919807940 wait complete. 2023-09-30T22:21:35.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.946+0000 7f38f1611700 1 Processor -- start 2023-09-30T22:21:35.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.946+0000 7f38f1611700 1 -- start start 2023-09-30T22:21:35.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38f1611700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec13aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:35.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38f1611700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 0x7f38ec13b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:35.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38f1611700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38ec12f4e0 0x7f38ec119660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:35.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38f1611700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec12f960 con 0x7f38ec098c00 2023-09-30T22:21:35.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38f1611700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec09cb30 con 0x7f38ec12f4e0 2023-09-30T22:21:35.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38f1611700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec099cc0 con 0x7f38ec097df0 2023-09-30T22:21:35.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec13aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:35.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 0x7f38ec13b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:35.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38eb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38ec12f4e0 0x7f38ec119660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:35.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 0x7f38ec13b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49108/0 (socket says 172.21.15.32:49108) 2023-09-30T22:21:35.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1180238225 learned_addr learned my addr 172.21.15.32:0/1180238225 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:35.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 msgr2=0x7f38ec13aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec13aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38ec12f4e0 msgr2=0x7f38ec119660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:35.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38ec12f4e0 0x7f38ec119660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:35.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 -- 172.21.15.32:0/1180238225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009370 con 0x7f38ec098c00 2023-09-30T22:21:35.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38eb7fe700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38ec12f4e0 0x7f38ec119660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:35.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38eaffd700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec13aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:35.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38ea7fc700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 0x7f38ec13b090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f38e000e900 tx=0x7f38e0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:35.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.947+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0012600 con 0x7f38ec098c00 2023-09-30T22:21:35.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.948+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e0012760 con 0x7f38ec098c00 2023-09-30T22:21:35.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.948+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec11cbf0 con 0x7f38ec098c00 2023-09-30T22:21:35.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.948+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0012ae0 con 0x7f38ec098c00 2023-09-30T22:21:35.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.948+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec11da30 con 0x7f38ec098c00 2023-09-30T22:21:35.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.948+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f38e001b400 con 0x7f38ec098c00 2023-09-30T22:21:35.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.949+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3208+0+0 (secure 0 0 0) 0x7f38e003b180 con 0x7f38ec098c00 2023-09-30T22:21:35.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.949+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec11da30 con 0x7f38ec098c00 2023-09-30T22:21:35.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:35.954+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f38ec11da30 con 0x7f38ec098c00 2023-09-30T22:21:36.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.129+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f38ec11ce10 con 0x7f38ec098c00 2023-09-30T22:21:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.779+0000 7f38d2ffd700 1 -- 172.21.15.32:0/1180238225 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v15) v1 ==== 170+0+0 (secure 0 0 0) 0x7f38ec11ce10 con 0x7f38ec098c00 2023-09-30T22:21:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.790+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 >> v1:172.21.15.32:6816/118188 conn(0x7f38d802bb60 legacy=0x7f38d802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.790+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 msgr2=0x7f38ec13b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.790+0000 7f38f1611700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 0x7f38ec13b090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f38e000e900 tx=0x7f38e0009060).stop 2023-09-30T22:21:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.791+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 shutdown_connections 2023-09-30T22:21:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.791+0000 7f38f1611700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38ec12f4e0 0x7f38ec119660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.791+0000 7f38f1611700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38ec097df0 0x7f38ec13aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.791+0000 7f38f1611700 1 --2- 172.21.15.32:0/1180238225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec098c00 0x7f38ec13b090 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0x7f38e000e900 tx=0x7f38e0009060).stop 2023-09-30T22:21:36.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.791+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 >> 172.21.15.32:0/1180238225 conn(0x7f38ec09dcf0 msgr2=0x7f38ec12e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:36.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.793+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 shutdown_connections 2023-09-30T22:21:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.794+0000 7f38f1611700 1 -- 172.21.15.32:0/1180238225 wait complete. 2023-09-30T22:21:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-09-30T22:21:36.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-30T22:21:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 Processor -- start 2023-09-30T22:21:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 -- start start 2023-09-30T22:21:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 0x7f6b3c097420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c132160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3c064e00 con 0x7f6b3c12add0 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0664f0 con 0x7f6b3c066110 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b42889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c065d80 con 0x7f6b3c12fd70 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.958+0000 7f6b40e26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c132160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 0x7f6b3c097420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 0x7f6b3c097420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49120/0 (socket says 172.21.15.32:49120) 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b40e26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c132160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37268/0 (socket says 172.21.15.32:37268) 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3bfff700 1 -- 172.21.15.32:0/1720434509 learned_addr learned my addr 172.21.15.32:0/1720434509 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b40e26700 1 -- 172.21.15.32:0/1720434509 >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 msgr2=0x7f6b3c097420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b40e26700 1 --2- 172.21.15.32:0/1720434509 >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 0x7f6b3c097420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b40e26700 1 -- 172.21.15.32:0/1720434509 >> v1:172.21.15.32:6789/0 conn(0x7f6b3c12add0 legacy=0x7f6b3c12b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b40e26700 1 -- 172.21.15.32:0/1720434509 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c065910 con 0x7f6b3c12fd70 2023-09-30T22:21:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3bfff700 1 --2- 172.21.15.32:0/1720434509 >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 0x7f6b3c097420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b40e26700 1 --2- 172.21.15.32:0/1720434509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c132160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6b34009e60 tx=0x7f6b34009780).ready entity=mon.1 client_cookie=fa12f39e7c2859f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3affd700 1 -- 172.21.15.32:0/1720434509 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400e070 con 0x7f6b3c12fd70 2023-09-30T22:21:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3affd700 1 -- 172.21.15.32:0/1720434509 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3400bb60 con 0x7f6b3c12fd70 2023-09-30T22:21:36.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.959+0000 7f6b3affd700 1 -- 172.21.15.32:0/1720434509 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34012410 con 0x7f6b3c12fd70 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 -- 172.21.15.32:0/1720434509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 msgr2=0x7f6b3c132160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 --2- 172.21.15.32:0/1720434509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c132160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6b34009e60 tx=0x7f6b34009780).stop 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 -- 172.21.15.32:0/1720434509 shutdown_connections 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 --2- 172.21.15.32:0/1720434509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c132160 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 --2- 172.21.15.32:0/1720434509 >> v2:172.21.15.32:3300/0 conn(0x7f6b3c066110 0x7f6b3c097420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 -- 172.21.15.32:0/1720434509 >> 172.21.15.32:0/1720434509 conn(0x7f6b3c09ba60 msgr2=0x7f6b3c09be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 -- 172.21.15.32:0/1720434509 shutdown_connections 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.960+0000 7f6b42889700 1 -- 172.21.15.32:0/1720434509 wait complete. 2023-09-30T22:21:36.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.961+0000 7f6b42889700 1 Processor -- start 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.961+0000 7f6b42889700 1 -- start start 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.961+0000 7f6b42889700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b3c066110 0x7f6b3c069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.961+0000 7f6b42889700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c12add0 0x7f6b3c13aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b42889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c13aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b42889700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c12f640 con 0x7f6b3c12add0 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b42889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c12cc60 con 0x7f6b3c12fd70 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b42889700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c129b60 con 0x7f6b3c066110 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b3bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b3c066110 0x7f6b3c069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c13aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b3b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c12add0 0x7f6b3c13aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b3bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b3c066110 0x7f6b3c069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40458/0 (socket says 172.21.15.32:40458) 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b3bfff700 1 -- 172.21.15.32:0/427696145 learned_addr learned my addr 172.21.15.32:0/427696145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 -- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b3c066110 msgr2=0x7f6b3c069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b3c066110 0x7f6b3c069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 -- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c12add0 msgr2=0x7f6b3c13aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c12add0 0x7f6b3c13aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 -- 172.21.15.32:0/427696145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c009370 con 0x7f6b3c12fd70 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b3b7fe700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c12add0 0x7f6b3c13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b40e26700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c13aef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6b34014000 tx=0x7f6b34009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34012410 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34007080 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3401d880 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b34009a20 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.962+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c12d340 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.963+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6b3401dc30 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.963+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3590+0+0 (secure 0 0 0) 0x7f6b3403ce50 con 0x7f6b3c12fd70 2023-09-30T22:21:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.963+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c12f7b0 con 0x7f6b3c12fd70 2023-09-30T22:21:36.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:36.968+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6b3c12f7b0 con 0x7f6b3c12fd70 2023-09-30T22:21:37.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.185+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6b3c12f7b0 con 0x7f6b3c12fd70 2023-09-30T22:21:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.196+0000 7f6b23fff700 1 -- 172.21.15.32:0/427696145 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6b3c12f7b0 con 0x7f6b3c12fd70 2023-09-30T22:21:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.209+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 >> v1:172.21.15.32:6816/118188 conn(0x7f6b2802bb60 legacy=0x7f6b2802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.209+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 msgr2=0x7f6b3c13aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.209+0000 7f6b42889700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c13aef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6b34014000 tx=0x7f6b34009780).stop 2023-09-30T22:21:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.220+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 shutdown_connections 2023-09-30T22:21:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.220+0000 7f6b42889700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b3c12fd70 0x7f6b3c13aef0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.220+0000 7f6b42889700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b3c066110 0x7f6b3c069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.220+0000 7f6b42889700 1 --2- 172.21.15.32:0/427696145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b3c12add0 0x7f6b3c13aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.220+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 >> 172.21.15.32:0/427696145 conn(0x7f6b3c09ba60 msgr2=0x7f6b3c09a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.222+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 shutdown_connections 2023-09-30T22:21:37.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.222+0000 7f6b42889700 1 -- 172.21.15.32:0/427696145 wait complete. 2023-09-30T22:21:37.233 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 3 and data pool 4 2023-09-30T22:21:37.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.375+0000 7f53f262e700 1 Processor -- start 2023-09-30T22:21:37.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.375+0000 7f53f262e700 1 -- start start 2023-09-30T22:21:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f262e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 0x7f53ec12da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f262e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53ec12df30 0x7f53ec130320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f262e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec096120 con 0x7f53ec096bc0 2023-09-30T22:21:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f262e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0956a0 con 0x7f53ec12df30 2023-09-30T22:21:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f262e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec096fa0 con 0x7f53ec125160 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 0x7f53ec12da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f0bcb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53ec12df30 0x7f53ec130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53ebfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53ec096bc0 0x7f53ec125780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51098/0 (socket says 172.21.15.32:51098) 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 0x7f53ec12da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40460/0 (socket says 172.21.15.32:40460) 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53ebfff700 1 -- 172.21.15.32:0/14850657 learned_addr learned my addr 172.21.15.32:0/14850657 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f0bcb700 1 -- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 msgr2=0x7f53ec12da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f0bcb700 1 --2- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 0x7f53ec12da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f0bcb700 1 -- 172.21.15.32:0/14850657 >> v1:172.21.15.32:6789/0 conn(0x7f53ec096bc0 legacy=0x7f53ec125780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f0bcb700 1 -- 172.21.15.32:0/14850657 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0963c0 con 0x7f53ec12df30 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53eaffd700 1 -- 172.21.15.32:0/14850657 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107176343 0 0) 0x7f53ec096120 con 0x7f53ec096bc0 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53eb7fe700 1 --2- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 0x7f53ec12da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.376+0000 7f53f0bcb700 1 --2- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec12df30 0x7f53ec130320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f53e4009e60 tx=0x7f53e4009780).ready entity=mon.0 client_cookie=40f033b42fe5573d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53eaffd700 1 -- 172.21.15.32:0/14850657 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400e070 con 0x7f53ec12df30 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53eaffd700 1 -- 172.21.15.32:0/14850657 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e400bb60 con 0x7f53ec12df30 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53eaffd700 1 -- 172.21.15.32:0/14850657 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4012410 con 0x7f53ec12df30 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 -- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec12df30 msgr2=0x7f53ec130320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 --2- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec12df30 0x7f53ec130320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f53e4009e60 tx=0x7f53e4009780).stop 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 -- 172.21.15.32:0/14850657 shutdown_connections 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 --2- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec12df30 0x7f53ec130320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 --2- 172.21.15.32:0/14850657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec125160 0x7f53ec12da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 -- 172.21.15.32:0/14850657 >> 172.21.15.32:0/14850657 conn(0x7f53ec097470 msgr2=0x7f53ec097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 -- 172.21.15.32:0/14850657 shutdown_connections 2023-09-30T22:21:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.377+0000 7f53f262e700 1 -- 172.21.15.32:0/14850657 wait complete. 2023-09-30T22:21:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.378+0000 7f53f262e700 1 Processor -- start 2023-09-30T22:21:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 -- start start 2023-09-30T22:21:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 0x7f53ec124b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53ec125160 0x7f53ec12b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec12df30 0x7f53ec12ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec112f70 con 0x7f53ec096bc0 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec112b80 con 0x7f53ec125160 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f262e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec112590 con 0x7f53ec12df30 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 0x7f53ec124b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53f0bcb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec12df30 0x7f53ec12ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 0x7f53ec124b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49144/0 (socket says 172.21.15.32:49144) 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53ebfff700 1 -- 172.21.15.32:0/2053171074 learned_addr learned my addr 172.21.15.32:0/2053171074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.379+0000 7f53eb7fe700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53ec125160 0x7f53ec12b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53ebfff700 1 -- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec12df30 msgr2=0x7f53ec12ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53ebfff700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec12df30 0x7f53ec12ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53ebfff700 1 -- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53ec125160 msgr2=0x7f53ec12b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53ebfff700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53ec125160 0x7f53ec12b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53ebfff700 1 -- 172.21.15.32:0/2053171074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40092a0 con 0x7f53ec096bc0 2023-09-30T22:21:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53ebfff700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 0x7f53ec124b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f53dc00c4c0 tx=0x7f53dc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00ce30 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc0072e0 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e4009a20 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0057a0 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.380+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec112040 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.381+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f53dc005bd0 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.381+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f53dc03a740 con 0x7f53ec096bc0 2023-09-30T22:21:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.381+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec1123d0 con 0x7f53ec096bc0 2023-09-30T22:21:37.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.386+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f53dc018030 con 0x7f53ec096bc0 2023-09-30T22:21:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.565+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53ec05a260 con 0x7f53ec096bc0 2023-09-30T22:21:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.565+0000 7f53e3fff700 1 -- 172.21.15.32:0/2053171074 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11354 (secure 0 0 0) 0x7f53dc01cb90 con 0x7f53ec096bc0 2023-09-30T22:21:37.566 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:37.566 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:37.187706+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.567+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 >> v1:172.21.15.32:6816/118188 conn(0x7f53cc02bb60 legacy=0x7f53cc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.567+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 msgr2=0x7f53ec124b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.567+0000 7f53f262e700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 0x7f53ec124b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f53dc00c4c0 tx=0x7f53dc007c10).stop 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 shutdown_connections 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f53ec12df30 0x7f53ec12ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53ec096bc0 0x7f53ec124b70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 --2- 172.21.15.32:0/2053171074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53ec125160 0x7f53ec12b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 >> 172.21.15.32:0/2053171074 conn(0x7f53ec097470 msgr2=0x7f53ec1289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 shutdown_connections 2023-09-30T22:21:37.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.568+0000 7f53f262e700 1 -- 172.21.15.32:0/2053171074 wait complete. 2023-09-30T22:21:37.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.711+0000 7f141b520700 1 Processor -- start 2023-09-30T22:21:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.711+0000 7f141b520700 1 -- start start 2023-09-30T22:21:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f141b520700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 0x7f1414127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f141b520700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1414127d80 0x7f1414095a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f141b520700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1414091b60 con 0x7f1414127390 2023-09-30T22:21:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f141b520700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1414059450 con 0x7f1414127d80 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f141b520700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140968c0 con 0x7f1414059070 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f14192bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 0x7f1414127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f1419abd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1414127d80 0x7f1414095a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f1418abb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1414127390 0x7f1414095370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51108/0 (socket says 172.21.15.32:51108) 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f1418abb700 1 -- 172.21.15.32:0/693707883 learned_addr learned my addr 172.21.15.32:0/693707883 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f14192bc700 1 --2- 172.21.15.32:0/693707883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 0x7f1414127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37290/0 (socket says 172.21.15.32:37290) 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.712+0000 7f14192bc700 1 -- 172.21.15.32:0/693707883 >> v2:172.21.15.32:3300/0 conn(0x7f1414127d80 msgr2=0x7f1414095a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f14192bc700 1 --2- 172.21.15.32:0/693707883 >> v2:172.21.15.32:3300/0 conn(0x7f1414127d80 0x7f1414095a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f14192bc700 1 -- 172.21.15.32:0/693707883 >> v1:172.21.15.32:6789/0 conn(0x7f1414127390 legacy=0x7f1414095370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f14192bc700 1 -- 172.21.15.32:0/693707883 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414092a10 con 0x7f1414059070 2023-09-30T22:21:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f1419abd700 1 --2- 172.21.15.32:0/693707883 >> v2:172.21.15.32:3300/0 conn(0x7f1414127d80 0x7f1414095a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f14192bc700 1 --2- 172.21.15.32:0/693707883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 0x7f1414127890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1410009790 tx=0x7f1410009060).ready entity=mon.1 client_cookie=1a7a7db3d5bbced8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f140bfff700 1 -- 172.21.15.32:0/693707883 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100090d0 con 0x7f1414059070 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f140bfff700 1 -- 172.21.15.32:0/693707883 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000f510 con 0x7f1414059070 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f140bfff700 1 -- 172.21.15.32:0/693707883 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000f890 con 0x7f1414059070 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f141b520700 1 -- 172.21.15.32:0/693707883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 msgr2=0x7f1414127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f141b520700 1 --2- 172.21.15.32:0/693707883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 0x7f1414127890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f1410009790 tx=0x7f1410009060).stop 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f141b520700 1 -- 172.21.15.32:0/693707883 shutdown_connections 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f141b520700 1 --2- 172.21.15.32:0/693707883 >> v2:172.21.15.32:3300/0 conn(0x7f1414127d80 0x7f1414095a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f141b520700 1 --2- 172.21.15.32:0/693707883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414059070 0x7f1414127890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.713+0000 7f141b520700 1 -- 172.21.15.32:0/693707883 >> 172.21.15.32:0/693707883 conn(0x7f1414097470 msgr2=0x7f1414097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.714+0000 7f141b520700 1 -- 172.21.15.32:0/693707883 shutdown_connections 2023-09-30T22:21:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.714+0000 7f141b520700 1 -- 172.21.15.32:0/693707883 wait complete. 2023-09-30T22:21:37.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.714+0000 7f141b520700 1 Processor -- start 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 -- start start 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1414059070 0x7f14141179b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414127390 0x7f1414113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 0x7f1414113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414115e70 con 0x7f1414127390 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1414124ae0 con 0x7f1414127d80 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f141b520700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1414129290 con 0x7f1414059070 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f14192bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1414059070 0x7f14141179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f1419abd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 0x7f1414113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f1418abb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414127390 0x7f1414113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f1419abd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 0x7f1414113f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37300/0 (socket says 172.21.15.32:37300) 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f14192bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1414059070 0x7f14141179b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40476/0 (socket says 172.21.15.32:40476) 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.715+0000 7f1419abd700 1 -- 172.21.15.32:0/4053152471 learned_addr learned my addr 172.21.15.32:0/4053152471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1419abd700 1 -- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1414059070 msgr2=0x7f14141179b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1419abd700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1414059070 0x7f14141179b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1419abd700 1 -- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414127390 msgr2=0x7f1414113a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1419abd700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414127390 0x7f1414113a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1419abd700 1 -- 172.21.15.32:0/4053152471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009a20 con 0x7f1414127d80 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1418abb700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414127390 0x7f1414113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f1419abd700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 0x7f1414113f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f140c009fc0 tx=0x7f140c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c00cbd0 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c00f040 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14100092a0 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c013400 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.716+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1414116f10 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.717+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f140c0137f0 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.717+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f140c03ba00 con 0x7f1414127d80 2023-09-30T22:21:37.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.717+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1414091d80 con 0x7f1414127d80 2023-09-30T22:21:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.722+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f140c013aa0 con 0x7f1414127d80 2023-09-30T22:21:37.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.896+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f141405a260 con 0x7f1414127d80 2023-09-30T22:21:37.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.897+0000 7f14097fa700 1 -- 172.21.15.32:0/4053152471 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11354 (secure 0 0 0) 0x7f140c02c090 con 0x7f1414127d80 2023-09-30T22:21:37.898 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:37.898 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:37.187706+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 >> v1:172.21.15.32:6816/118188 conn(0x7f13f802bb60 legacy=0x7f13f802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 msgr2=0x7f1414113f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 0x7f1414113f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f140c009fc0 tx=0x7f140c00b040).stop 2023-09-30T22:21:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 shutdown_connections 2023-09-30T22:21:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1414127d80 0x7f1414113f00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1414059070 0x7f14141179b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 --2- 172.21.15.32:0/4053152471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1414127390 0x7f1414113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 >> 172.21.15.32:0/4053152471 conn(0x7f1414097470 msgr2=0x7f1414125ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 shutdown_connections 2023-09-30T22:21:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:37.899+0000 7f141b520700 1 -- 172.21.15.32:0/4053152471 wait complete. 2023-09-30T22:21:37.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-30T22:21:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.049+0000 7fb1ffbd7700 1 Processor -- start 2023-09-30T22:21:38.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.049+0000 7fb1ffbd7700 1 -- start start 2023-09-30T22:21:38.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1ffbd7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1f80613a0 0x7fb1f812ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1ffbd7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8132d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1ffbd7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f8098210 con 0x7fb1f812b340 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1ffbd7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f8093380 con 0x7fb1f80613a0 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1ffbd7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8099670 con 0x7fb1f8069430 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1fd973700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1f80613a0 0x7fb1f812ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1fe174700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1fd172700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb1f812b340 0x7fb1f8068d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51122/0 (socket says 172.21.15.32:51122) 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.050+0000 7fb1fd172700 1 -- 172.21.15.32:0/2921588364 learned_addr learned my addr 172.21.15.32:0/2921588364 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fe174700 1 -- 172.21.15.32:0/2921588364 >> v2:172.21.15.32:3300/0 conn(0x7fb1f80613a0 msgr2=0x7fb1f812ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fe174700 1 --2- 172.21.15.32:0/2921588364 >> v2:172.21.15.32:3300/0 conn(0x7fb1f80613a0 0x7fb1f812ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fe174700 1 -- 172.21.15.32:0/2921588364 >> v1:172.21.15.32:6789/0 conn(0x7fb1f812b340 legacy=0x7fb1f8068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fc971700 1 -- 172.21.15.32:0/2921588364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019983302 0 0) 0x7fb1f8098210 con 0x7fb1f812b340 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fe174700 1 -- 172.21.15.32:0/2921588364 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8093710 con 0x7fb1f8069430 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fd973700 1 --2- 172.21.15.32:0/2921588364 >> v2:172.21.15.32:3300/0 conn(0x7fb1f80613a0 0x7fb1f812ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fe174700 1 --2- 172.21.15.32:0/2921588364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8132d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb1f0009e90 tx=0x7fb1f0009780).ready entity=mon.1 client_cookie=e4c25e78168a7379 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fc971700 1 -- 172.21.15.32:0/2921588364 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000e070 con 0x7fb1f8069430 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fc971700 1 -- 172.21.15.32:0/2921588364 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f000bb20 con 0x7fb1f8069430 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1fc971700 1 -- 172.21.15.32:0/2921588364 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000bea0 con 0x7fb1f8069430 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/2921588364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 msgr2=0x7fb1f8132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/2921588364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8132d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb1f0009e90 tx=0x7fb1f0009780).stop 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/2921588364 shutdown_connections 2023-09-30T22:21:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/2921588364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8132d80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/2921588364 >> v2:172.21.15.32:3300/0 conn(0x7fb1f80613a0 0x7fb1f812ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.051+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/2921588364 >> 172.21.15.32:0/2921588364 conn(0x7fb1f809a1c0 msgr2=0x7fb1f809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.052+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/2921588364 shutdown_connections 2023-09-30T22:21:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.052+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/2921588364 wait complete. 2023-09-30T22:21:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.052+0000 7fb1ffbd7700 1 Processor -- start 2023-09-30T22:21:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 -- start start 2023-09-30T22:21:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 0x7fb1f8115990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:38.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8115e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1f812b340 0x7fb1f8116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8116860 con 0x7fb1f80613a0 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8118e90 con 0x7fb1f8069430 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1ffbd7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f81176a0 con 0x7fb1f812b340 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1fd973700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 0x7fb1f8115990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1fd973700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 0x7fb1f8115990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49176/0 (socket says 172.21.15.32:49176) 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1fd973700 1 -- 172.21.15.32:0/3891761707 learned_addr learned my addr 172.21.15.32:0/3891761707 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1fd172700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8115e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.053+0000 7fb1fe174700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1f812b340 0x7fb1f8116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd973700 1 -- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1f812b340 msgr2=0x7fb1f8116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd973700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1f812b340 0x7fb1f8116370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd973700 1 -- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 msgr2=0x7fb1f8115e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd973700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8115e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd973700 1 -- 172.21.15.32:0/3891761707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40092a0 con 0x7fb1f80613a0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fe174700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1f812b340 0x7fb1f8116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd172700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8115e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1fd973700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 0x7fb1f8115990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb1f400ea20 tx=0x7fb1f4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400ee60 con 0x7fb1f80613a0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f4009600 con 0x7fb1f80613a0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f0009a20 con 0x7fb1f80613a0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400ca60 con 0x7fb1f80613a0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.054+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f8093940 con 0x7fb1f80613a0 2023-09-30T22:21:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.055+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb1f401b400 con 0x7fb1f80613a0 2023-09-30T22:21:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.055+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb1f403b4f0 con 0x7fb1f80613a0 2023-09-30T22:21:38.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.055+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f805a260 con 0x7fb1f80613a0 2023-09-30T22:21:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.060+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb1f4016030 con 0x7fb1f80613a0 2023-09-30T22:21:38.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:38.279+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb1f8098800 con 0x7fb1f80613a0 2023-09-30T22:21:39.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.194+0000 7fb1edffb700 1 -- 172.21.15.32:0/3891761707 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb1f4019310 con 0x7fb1f80613a0 2023-09-30T22:21:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 >> v1:172.21.15.32:6816/118188 conn(0x7fb1dc02bb60 legacy=0x7fb1dc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 msgr2=0x7fb1f8115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 0x7fb1f8115990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb1f400ea20 tx=0x7fb1f4009060).stop 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 shutdown_connections 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1f812b340 0x7fb1f8116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1f80613a0 0x7fb1f8115990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 --2- 172.21.15.32:0/3891761707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1f8069430 0x7fb1f8115e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.197+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 >> 172.21.15.32:0/3891761707 conn(0x7fb1f809a1c0 msgr2=0x7fb1f8066b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.198+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 shutdown_connections 2023-09-30T22:21:39.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.198+0000 7fb1ffbd7700 1 -- 172.21.15.32:0/3891761707 wait complete. 2023-09-30T22:21:39.213 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-30T22:21:39.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 Processor -- start 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 -- start start 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde84098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde84098c80 0x7fde84099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde84092d80 con 0x7fde8412c150 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde84093c20 con 0x7fde84098c80 2023-09-30T22:21:39.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde8af2f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde84092b90 con 0x7fde84093840 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde88ccb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde84098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde83fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde84098c80 0x7fde84099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde894cc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fde8412c150 0x7fde8412e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51138/0 (socket says 172.21.15.32:51138) 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde88ccb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde84098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40484/0 (socket says 172.21.15.32:40484) 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.350+0000 7fde894cc700 1 -- 172.21.15.32:0/1563239872 learned_addr learned my addr 172.21.15.32:0/1563239872 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde83fff700 1 -- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 msgr2=0x7fde84098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde83fff700 1 --2- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde84098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde83fff700 1 -- 172.21.15.32:0/1563239872 >> v1:172.21.15.32:6789/0 conn(0x7fde8412c150 legacy=0x7fde8412e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde837fe700 1 -- 172.21.15.32:0/1563239872 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961211382 0 0) 0x7fde84092d80 con 0x7fde8412c150 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde83fff700 1 -- 172.21.15.32:0/1563239872 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84099b70 con 0x7fde84098c80 2023-09-30T22:21:39.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde83fff700 1 --2- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde84099060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fde78009760 tx=0x7fde78009060).ready entity=mon.0 client_cookie=450f742bdeeb2482 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde837fe700 1 -- 172.21.15.32:0/1563239872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800f510 con 0x7fde84098c80 2023-09-30T22:21:39.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde837fe700 1 -- 172.21.15.32:0/1563239872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800f670 con 0x7fde84098c80 2023-09-30T22:21:39.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.351+0000 7fde837fe700 1 -- 172.21.15.32:0/1563239872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800f9f0 con 0x7fde84098c80 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 -- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 msgr2=0x7fde84099060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 --2- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde84099060 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fde78009760 tx=0x7fde78009060).stop 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 -- 172.21.15.32:0/1563239872 shutdown_connections 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 --2- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde84098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 --2- 172.21.15.32:0/1563239872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde84099060 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 -- 172.21.15.32:0/1563239872 >> 172.21.15.32:0/1563239872 conn(0x7fde84097480 msgr2=0x7fde84097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 -- 172.21.15.32:0/1563239872 shutdown_connections 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.352+0000 7fde8af2f700 1 -- 172.21.15.32:0/1563239872 wait complete. 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.353+0000 7fde8af2f700 1 Processor -- start 2023-09-30T22:21:39.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.353+0000 7fde8af2f700 1 -- start start 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.353+0000 7fde8af2f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde8412b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.353+0000 7fde8af2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde8412b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde8af2f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde8412c150 0x7fde84117190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde8af2f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8406a540 con 0x7fde84098c80 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde8af2f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde84064dc0 con 0x7fde8412c150 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde8af2f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde841187b0 con 0x7fde84093840 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde8412b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde8412b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49192/0 (socket says 172.21.15.32:49192) 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 -- 172.21.15.32:0/2995905440 learned_addr learned my addr 172.21.15.32:0/2995905440 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde88ccb700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde8412b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:39.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde894cc700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde8412c150 0x7fde84117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 -- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 msgr2=0x7fde8412b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde8412b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 -- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde8412c150 msgr2=0x7fde84117190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde8412c150 0x7fde84117190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 -- 172.21.15.32:0/2995905440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780092a0 con 0x7fde84098c80 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde83fff700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde8412b520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fde78009e00 tx=0x7fde78017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800f7e0 con 0x7fde84098c80 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800f940 con 0x7fde84098c80 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7801b660 con 0x7fde84098c80 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8411d960 con 0x7fde84098c80 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.354+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8411e830 con 0x7fde84098c80 2023-09-30T22:21:39.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.355+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fde7801ba10 con 0x7fde84098c80 2023-09-30T22:21:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.355+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fde7803c7a0 con 0x7fde84098c80 2023-09-30T22:21:39.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.356+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8411d960 con 0x7fde84098c80 2023-09-30T22:21:39.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.360+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fde8411d960 con 0x7fde84098c80 2023-09-30T22:21:39.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:39.581+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde8411d960 con 0x7fde84098c80 2023-09-30T22:21:40.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.201+0000 7fde80ff9700 1 -- 172.21.15.32:0/2995905440 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde8411d960 con 0x7fde84098c80 2023-09-30T22:21:40.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.203+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 >> v1:172.21.15.32:6816/118188 conn(0x7fde7002bb60 legacy=0x7fde7002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.203+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 msgr2=0x7fde8412b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.203+0000 7fde8af2f700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde8412b520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fde78009e00 tx=0x7fde78017000).stop 2023-09-30T22:21:40.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 shutdown_connections 2023-09-30T22:21:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde8412c150 0x7fde84117190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde84093840 0x7fde8412b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 --2- 172.21.15.32:0/2995905440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde84098c80 0x7fde8412b520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 >> 172.21.15.32:0/2995905440 conn(0x7fde84097480 msgr2=0x7fde84096aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 shutdown_connections 2023-09-30T22:21:40.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.204+0000 7fde8af2f700 1 -- 172.21.15.32:0/2995905440 wait complete. 2023-09-30T22:21:40.220 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-30T22:21:40.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:40.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.353+0000 7f1cc7f55700 1 Processor -- start 2023-09-30T22:21:40.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.354+0000 7f1cc7f55700 1 -- start start 2023-09-30T22:21:40.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.354+0000 7f1cc7f55700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc00654a0 0x7f1cc0098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.354+0000 7f1cc7f55700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc0098f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.354+0000 7f1cc7f55700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc0065ed0 con 0x7f1cc012f470 2023-09-30T22:21:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.354+0000 7f1cc7f55700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0095880 con 0x7f1cc00654a0 2023-09-30T22:21:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.354+0000 7f1cc7f55700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0096120 con 0x7f1cc0098b30 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc5cf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc00654a0 0x7f1cc0098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc54f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc0098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc64f2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1cc012f470 0x7f1cc0131860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51146/0 (socket says 172.21.15.32:51146) 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc64f2700 1 -- 172.21.15.32:0/1351837642 learned_addr learned my addr 172.21.15.32:0/1351837642 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 msgr2=0x7f1cc0098f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc4cef700 1 -- 172.21.15.32:0/1351837642 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015697490 0 0) 0x7f1cc0065ed0 con 0x7f1cc012f470 2023-09-30T22:21:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc5cf1700 1 --2- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc0098f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1351837642 >> v1:172.21.15.32:6789/0 conn(0x7f1cc012f470 legacy=0x7f1cc0131860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1351837642 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0099460 con 0x7f1cc00654a0 2023-09-30T22:21:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc54f0700 1 --2- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc0098f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc5cf1700 1 --2- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc00654a0 0x7f1cc0098640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1cbc0097c0 tx=0x7f1cbc009060).ready entity=mon.1 client_cookie=303f55695b0461c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc4cef700 1 -- 172.21.15.32:0/1351837642 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00f510 con 0x7f1cc00654a0 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc4cef700 1 -- 172.21.15.32:0/1351837642 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00f670 con 0x7f1cc00654a0 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.355+0000 7f1cc4cef700 1 -- 172.21.15.32:0/1351837642 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00f9f0 con 0x7f1cc00654a0 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc00654a0 msgr2=0x7f1cc0098640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc00654a0 0x7f1cc0098640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1cbc0097c0 tx=0x7f1cbc009060).stop 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1351837642 shutdown_connections 2023-09-30T22:21:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc00654a0 0x7f1cc0098640 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1351837642 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc0098f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1351837642 >> 172.21.15.32:0/1351837642 conn(0x7f1cc009a740 msgr2=0x7f1cc009ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1351837642 shutdown_connections 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.356+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1351837642 wait complete. 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.357+0000 7f1cc7f55700 1 Processor -- start 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.357+0000 7f1cc7f55700 1 -- start start 2023-09-30T22:21:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc7f55700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 0x7f1cc013aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc7f55700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc013af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc7f55700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc012f470 0x7f1cc011a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc7f55700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc011dfd0 con 0x7f1cc00654a0 2023-09-30T22:21:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc7f55700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc012f850 con 0x7f1cc012f470 2023-09-30T22:21:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc7f55700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0099b00 con 0x7f1cc0098b30 2023-09-30T22:21:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc5cf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 0x7f1cc013aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc64f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc012f470 0x7f1cc011a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc5cf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 0x7f1cc013aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49200/0 (socket says 172.21.15.32:49200) 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1657788691 learned_addr learned my addr 172.21.15.32:0/1657788691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.358+0000 7f1cc54f0700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc013af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 msgr2=0x7f1cc013af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc5cf1700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc013af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc012f470 msgr2=0x7f1cc011a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc5cf1700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc012f470 0x7f1cc011a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc5cf1700 1 -- 172.21.15.32:0/1657788691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00092a0 con 0x7f1cc00654a0 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc5cf1700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 0x7f1cc013aa90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1cbc009e60 tx=0x7f1cbc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00f7e0 con 0x7f1cc00654a0 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00f940 con 0x7f1cc00654a0 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc0092a0 con 0x7f1cc00654a0 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc01b660 con 0x7f1cc00654a0 2023-09-30T22:21:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.359+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc011c0c0 con 0x7f1cc00654a0 2023-09-30T22:21:40.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.360+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1cbc01ba50 con 0x7f1cc00654a0 2023-09-30T22:21:40.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.360+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1cbc03c8c0 con 0x7f1cc00654a0 2023-09-30T22:21:40.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.360+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc005a260 con 0x7f1cc00654a0 2023-09-30T22:21:40.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.365+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1cbc029080 con 0x7f1cc00654a0 2023-09-30T22:21:40.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.583+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cc00940c0 con 0x7f1cc00654a0 2023-09-30T22:21:40.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.583+0000 7f1cb5ffb700 1 -- 172.21.15.32:0/1657788691 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f1cbc01f030 con 0x7f1cc00654a0 2023-09-30T22:21:40.584 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:40.585 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:40.197617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:21:40.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 >> v1:172.21.15.32:6816/118188 conn(0x7f1ca402bb60 legacy=0x7f1ca402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 msgr2=0x7f1cc013aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 0x7f1cc013aa90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1cbc009e60 tx=0x7f1cbc017000).stop 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 shutdown_connections 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1cc012f470 0x7f1cc011a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cc00654a0 0x7f1cc013aa90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 --2- 172.21.15.32:0/1657788691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1cc0098b30 0x7f1cc013af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 >> 172.21.15.32:0/1657788691 conn(0x7f1cc009a740 msgr2=0x7f1cc01142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 shutdown_connections 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.585+0000 7f1cc7f55700 1 -- 172.21.15.32:0/1657788691 wait complete. 2023-09-30T22:21:40.587 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2023-09-30T22:21:40.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 Processor -- start 2023-09-30T22:21:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 -- start start 2023-09-30T22:21:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b640613a0 0x7f2b6412ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 0x7f2b6412b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64096270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b64061780 con 0x7f2b6412b2f0 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6412abd0 con 0x7f2b640613a0 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6aa8e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64067530 con 0x7f2b64095df0 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b63fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 0x7f2b6412b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6902b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64096270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6882a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b640613a0 0x7f2b6412ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b63fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 0x7f2b6412b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54362/0 (socket says 172.21.15.32:54362) 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b6902b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64096270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35414/0 (socket says 172.21.15.32:35414) 2023-09-30T22:21:40.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.730+0000 7f2b63fff700 1 -- 172.21.15.32:0/1489113305 learned_addr learned my addr 172.21.15.32:0/1489113305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:40.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6902b700 1 -- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b640613a0 msgr2=0x7f2b6412ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:40.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6902b700 1 --2- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b640613a0 0x7f2b6412ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6902b700 1 -- 172.21.15.32:0/1489113305 >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 msgr2=0x7f2b6412b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:40.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6902b700 1 --2- 172.21.15.32:0/1489113305 >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 0x7f2b6412b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6902b700 1 -- 172.21.15.32:0/1489113305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b640683c0 con 0x7f2b64095df0 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b63fff700 1 --2- 172.21.15.32:0/1489113305 >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 0x7f2b6412b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6882a700 1 --2- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b640613a0 0x7f2b6412ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b6902b700 1 --2- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64096270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2b5c009e60 tx=0x7f2b5c009780).ready entity=mon.2 client_cookie=1b86b328629de118 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b637fe700 1 -- 172.21.15.32:0/1489113305 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00e070 con 0x7f2b64095df0 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b637fe700 1 -- 172.21.15.32:0/1489113305 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c00bab0 con 0x7f2b64095df0 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.731+0000 7f2b637fe700 1 -- 172.21.15.32:0/1489113305 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00be30 con 0x7f2b64095df0 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 msgr2=0x7f2b64096270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64096270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2b5c009e60 tx=0x7f2b5c009780).stop 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/1489113305 shutdown_connections 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b640613a0 0x7f2b6412ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/1489113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64096270 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/1489113305 >> v2:172.21.15.32:3300/0 conn(0x7f2b6412b2f0 0x7f2b6412b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/1489113305 >> 172.21.15.32:0/1489113305 conn(0x7f2b6409a220 msgr2=0x7f2b6409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/1489113305 shutdown_connections 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.732+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/1489113305 wait complete. 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 Processor -- start 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 -- start start 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b640613a0 0x7f2b64115a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6412b2f0 0x7f2b641163e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b641168d0 con 0x7f2b640613a0 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64116eb0 con 0x7f2b6412b2f0 2023-09-30T22:21:40.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6aa8e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64118f90 con 0x7f2b64095df0 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.733+0000 7f2b6882a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b640613a0 0x7f2b64115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6412b2f0 0x7f2b641163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6882a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b640613a0 0x7f2b64115a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54372/0 (socket says 172.21.15.32:54372) 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b63fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6882a700 1 -- 172.21.15.32:0/591322403 learned_addr learned my addr 172.21.15.32:0/591322403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 -- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 msgr2=0x7f2b64115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64115ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 -- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b640613a0 msgr2=0x7f2b64115a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b640613a0 0x7f2b64115a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 -- 172.21.15.32:0/591322403 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540092a0 con 0x7f2b6412b2f0 2023-09-30T22:21:40.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b63fff700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:40.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6902b700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6412b2f0 0x7f2b641163e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f2b5c014000 tx=0x7f2b64067ed0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:40.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c013e40 con 0x7f2b6412b2f0 2023-09-30T22:21:40.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c01d630 con 0x7f2b6412b2f0 2023-09-30T22:21:40.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c009370 con 0x7f2b6412b2f0 2023-09-30T22:21:40.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01d9f0 con 0x7f2b6412b2f0 2023-09-30T22:21:40.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.734+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c009a20 con 0x7f2b6412b2f0 2023-09-30T22:21:40.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.735+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2b5c003040 con 0x7f2b6412b2f0 2023-09-30T22:21:40.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.735+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2b5c03cb70 con 0x7f2b6412b2f0 2023-09-30T22:21:40.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.735+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b64067130 con 0x7f2b6412b2f0 2023-09-30T22:21:40.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.740+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2b5c025bb0 con 0x7f2b6412b2f0 2023-09-30T22:21:40.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.916+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b6405a260 con 0x7f2b6412b2f0 2023-09-30T22:21:40.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.917+0000 7f2b60ff9700 1 -- 172.21.15.32:0/591322403 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f2b5c011e60 con 0x7f2b6412b2f0 2023-09-30T22:21:40.918 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:40.918 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:38.189468+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 >> v1:172.21.15.32:6816/118188 conn(0x7f2b4402bb60 legacy=0x7f2b4402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6412b2f0 msgr2=0x7f2b641163e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6412b2f0 0x7f2b641163e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f2b5c014000 tx=0x7f2b64067ed0).stop 2023-09-30T22:21:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 shutdown_connections 2023-09-30T22:21:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6412b2f0 0x7f2b641163e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b640613a0 0x7f2b64115a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 --2- 172.21.15.32:0/591322403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b64095df0 0x7f2b64115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 >> 172.21.15.32:0/591322403 conn(0x7f2b6409a220 msgr2=0x7f2b6412e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 shutdown_connections 2023-09-30T22:21:40.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:40.919+0000 7f2b6aa8e700 1 -- 172.21.15.32:0/591322403 wait complete. 2023-09-30T22:21:40.933 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-30T22:21:40.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:21:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.062+0000 7f21750a7700 1 Processor -- start 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f21750a7700 1 -- start start 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f21750a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2170096380 0x7f2170094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f21750a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f21750a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2170096760 con 0x7f2170091b00 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f21750a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2170091ff0 con 0x7f2170096380 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f21750a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2170092c80 con 0x7f2170094c20 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f216f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f216659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2170091b00 0x7f2170127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54726/0 (socket says 172.21.15.32:54726) 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f216659c700 1 -- 172.21.15.32:0/1364773211 learned_addr learned my addr 172.21.15.32:0/1364773211 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f216f59e700 1 --2- 172.21.15.32:0/1364773211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170127d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40744/0 (socket says 172.21.15.32:40744) 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.063+0000 7f216ed9d700 1 --2- 172.21.15.32:0/1364773211 >> v2:172.21.15.32:3300/0 conn(0x7f2170096380 0x7f2170094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216f59e700 1 -- 172.21.15.32:0/1364773211 >> v2:172.21.15.32:3300/0 conn(0x7f2170096380 msgr2=0x7f2170094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216f59e700 1 --2- 172.21.15.32:0/1364773211 >> v2:172.21.15.32:3300/0 conn(0x7f2170096380 0x7f2170094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216f59e700 1 -- 172.21.15.32:0/1364773211 >> v1:172.21.15.32:6789/0 conn(0x7f2170091b00 legacy=0x7f2170127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216f59e700 1 -- 172.21.15.32:0/1364773211 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170093900 con 0x7f2170094c20 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216ed9d700 1 --2- 172.21.15.32:0/1364773211 >> v2:172.21.15.32:3300/0 conn(0x7f2170096380 0x7f2170094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216f59e700 1 --2- 172.21.15.32:0/1364773211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170127d10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2168009e60 tx=0x7f2168009780).ready entity=mon.1 client_cookie=f8e0777a4e0a0a55 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216e59c700 1 -- 172.21.15.32:0/1364773211 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216800e070 con 0x7f2170094c20 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216e59c700 1 -- 172.21.15.32:0/1364773211 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216800bb60 con 0x7f2170094c20 2023-09-30T22:21:41.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f216e59c700 1 -- 172.21.15.32:0/1364773211 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168012410 con 0x7f2170094c20 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f21750a7700 1 -- 172.21.15.32:0/1364773211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 msgr2=0x7f2170127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f21750a7700 1 --2- 172.21.15.32:0/1364773211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170127d10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f2168009e60 tx=0x7f2168009780).stop 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.064+0000 7f21750a7700 1 -- 172.21.15.32:0/1364773211 shutdown_connections 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.065+0000 7f21750a7700 1 --2- 172.21.15.32:0/1364773211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170127d10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.065+0000 7f21750a7700 1 --2- 172.21.15.32:0/1364773211 >> v2:172.21.15.32:3300/0 conn(0x7f2170096380 0x7f2170094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.065+0000 7f21750a7700 1 -- 172.21.15.32:0/1364773211 >> 172.21.15.32:0/1364773211 conn(0x7f2170097470 msgr2=0x7f2170097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.065+0000 7f21750a7700 1 -- 172.21.15.32:0/1364773211 shutdown_connections 2023-09-30T22:21:41.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.065+0000 7f21750a7700 1 -- 172.21.15.32:0/1364773211 wait complete. 2023-09-30T22:21:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 Processor -- start 2023-09-30T22:21:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 -- start start 2023-09-30T22:21:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2170091b00 0x7f217011af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170125510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 0x7f2170125a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2170127130 con 0x7f2170096380 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217011a4f0 con 0x7f2170094c20 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f21750a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2170098850 con 0x7f2170091b00 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f216659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170125510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f216f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 0x7f2170125a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f216ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2170091b00 0x7f217011af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f216f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 0x7f2170125a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54396/0 (socket says 172.21.15.32:54396) 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.066+0000 7f216f59e700 1 -- 172.21.15.32:0/1406106437 learned_addr learned my addr 172.21.15.32:0/1406106437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216ed9d700 1 -- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 msgr2=0x7f2170125510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216ed9d700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170125510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216ed9d700 1 -- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 msgr2=0x7f2170125a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216ed9d700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 0x7f2170125a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216ed9d700 1 -- 172.21.15.32:0/1406106437 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600092a0 con 0x7f2170091b00 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216659c700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216f59e700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 0x7f2170125a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f216ed9d700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2170091b00 0x7f217011af90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f216000e980 tx=0x7f2160009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160012600 con 0x7f2170091b00 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160012760 con 0x7f2170091b00 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2168009a20 con 0x7f2170091b00 2023-09-30T22:21:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160012ae0 con 0x7f2170091b00 2023-09-30T22:21:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.067+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217011a1d0 con 0x7f2170091b00 2023-09-30T22:21:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.068+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f216001b400 con 0x7f2170091b00 2023-09-30T22:21:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.068+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f216003b480 con 0x7f2170091b00 2023-09-30T22:21:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.068+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217005a260 con 0x7f2170091b00 2023-09-30T22:21:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.073+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2160016070 con 0x7f2170091b00 2023-09-30T22:21:41.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.246+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 --> v1:172.21.15.32:6816/118188 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2170126310 con 0x7f215002bb60 2023-09-30T22:21:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.253+0000 7f21677fe700 1 -- 172.21.15.32:0/1406106437 <== mgr.4101 v1:172.21.15.32:6816/118188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242457 (unknown 2965378022 0 2276109892) 0x7f2170126310 con 0x7f215002bb60 2023-09-30T22:21:41.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 >> v1:172.21.15.32:6816/118188 conn(0x7f215002bb60 legacy=0x7f215002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2170091b00 msgr2=0x7f217011af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2170091b00 0x7f217011af90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f216000e980 tx=0x7f2160009060).stop 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 shutdown_connections 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2170096380 0x7f2170125a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2170091b00 0x7f217011af90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 --2- 172.21.15.32:0/1406106437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2170094c20 0x7f2170125510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 >> 172.21.15.32:0/1406106437 conn(0x7f2170097470 msgr2=0x7f2170059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 shutdown_connections 2023-09-30T22:21:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.256+0000 7f21750a7700 1 -- 172.21.15.32:0/1406106437 wait complete. 2023-09-30T22:21:41.260 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T22:21:41.270 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:21:39.534226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188832,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298528,"statfs":{"total":386547056640,"available":385329692672,"internally_reserved":0,"allocated":458752,"data_stored":120816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.795563+0000","last_change":"2023-09-30T22:21:31.794405+0000","last_active":"2023-09-30T22:21:31.795563+0000","last_peered":"2023-09-30T22:21:31.795563+0000","last_clean":"2023-09-30T22:21:31.795563+0000","last_became_active":"2023-09-30T22:21:31.794052+0000","last_became_peered":"2023-09-30T22:21:31.794052+0000","last_unstale":"2023-09-30T22:21:31.795563+0000","last_undegraded":"2023-09-30T22:21:31.795563+0000","last_fullsized":"2023-09-30T22:21:31.795563+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.800457+0000","last_change":"2023-09-30T22:21:31.798962+0000","last_active":"2023-09-30T22:21:31.800457+0000","last_peered":"2023-09-30T22:21:31.800457+0000","last_clean":"2023-09-30T22:21:31.800457+0000","last_became_active":"2023-09-30T22:21:31.798641+0000","last_became_peered":"2023-09-30T22:21:31.798641+0000","last_unstale":"2023-09-30T22:21:31.800457+0000","last_undegraded":"2023-09-30T22:21:31.800457+0000","last_fullsized":"2023-09-30T22:21:31.800457+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.799877+0000","last_change":"2023-09-30T22:21:31.797737+0000","last_active":"2023-09-30T22:21:31.799877+0000","last_peered":"2023-09-30T22:21:31.799877+0000","last_clean":"2023-09-30T22:21:31.799877+0000","last_became_active":"2023-09-30T22:21:31.797396+0000","last_became_peered":"2023-09-30T22:21:31.797396+0000","last_unstale":"2023-09-30T22:21:31.799877+0000","last_undegraded":"2023-09-30T22:21:31.799877+0000","last_fullsized":"2023-09-30T22:21:31.799877+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.800335+0000","last_change":"2023-09-30T22:21:31.798294+0000","last_active":"2023-09-30T22:21:31.800335+0000","last_peered":"2023-09-30T22:21:31.800335+0000","last_clean":"2023-09-30T22:21:31.800335+0000","last_became_active":"2023-09-30T22:21:31.797999+0000","last_became_peered":"2023-09-30T22:21:31.797999+0000","last_unstale":"2023-09-30T22:21:31.800335+0000","last_undegraded":"2023-09-30T22:21:31.800335+0000","last_fullsized":"2023-09-30T22:21:31.800335+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.791200+0000","last_change":"2023-09-30T22:21:30.796302+0000","last_active":"2023-09-30T22:21:31.791200+0000","last_peered":"2023-09-30T22:21:31.791200+0000","last_clean":"2023-09-30T22:21:31.791200+0000","last_became_active":"2023-09-30T22:21:30.795885+0000","last_became_peered":"2023-09-30T22:21:30.795885+0000","last_unstale":"2023-09-30T22:21:31.791200+0000","last_undegraded":"2023-09-30T22:21:31.791200+0000","last_fullsized":"2023-09-30T22:21:31.791200+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_clean_scrub_stamp":"2023-09-30T22:21:29.785382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.800360+0000","last_change":"2023-09-30T22:21:31.798735+0000","last_active":"2023-09-30T22:21:31.800360+0000","last_peered":"2023-09-30T22:21:31.800360+0000","last_clean":"2023-09-30T22:21:31.800360+0000","last_became_active":"2023-09-30T22:21:31.798405+0000","last_became_peered":"2023-09-30T22:21:31.798405+0000","last_unstale":"2023-09-30T22:21:31.800360+0000","last_undegraded":"2023-09-30T22:21:31.800360+0000","last_fullsized":"2023-09-30T22:21:31.800360+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.800924+0000","last_change":"2023-09-30T22:21:31.800121+0000","last_active":"2023-09-30T22:21:31.800924+0000","last_peered":"2023-09-30T22:21:31.800924+0000","last_clean":"2023-09-30T22:21:31.800924+0000","last_became_active":"2023-09-30T22:21:31.799775+0000","last_became_peered":"2023-09-30T22:21:31.799775+0000","last_unstale":"2023-09-30T22:21:31.800924+0000","last_undegraded":"2023-09-30T22:21:31.800924+0000","last_fullsized":"2023-09-30T22:21:31.800924+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.795594+0000","last_change":"2023-09-30T22:21:31.794560+0000","last_active":"2023-09-30T22:21:31.795594+0000","last_peered":"2023-09-30T22:21:31.795594+0000","last_clean":"2023-09-30T22:21:31.795594+0000","last_became_active":"2023-09-30T22:21:31.794101+0000","last_became_peered":"2023-09-30T22:21:31.794101+0000","last_unstale":"2023-09-30T22:21:31.795594+0000","last_undegraded":"2023-09-30T22:21:31.795594+0000","last_fullsized":"2023-09-30T22:21:31.795594+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-09-30T22:21:31.800904+0000","last_change":"2023-09-30T22:21:31.800079+0000","last_active":"2023-09-30T22:21:31.800904+0000","last_peered":"2023-09-30T22:21:31.800904+0000","last_clean":"2023-09-30T22:21:31.800904+0000","last_became_active":"2023-09-30T22:21:31.799718+0000","last_became_peered":"2023-09-30T22:21:31.799718+0000","last_unstale":"2023-09-30T22:21:31.800904+0000","last_undegraded":"2023-09-30T22:21:31.800904+0000","last_fullsized":"2023-09-30T22:21:31.800904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:35.773378+0000","last_change":"2023-09-30T22:21:35.773378+0000","last_active":"2023-09-30T22:21:35.773378+0000","last_peered":"2023-09-30T22:21:35.773378+0000","last_clean":"2023-09-30T22:21:35.773378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:35.773378+0000","last_undegraded":"2023-09-30T22:21:35.773378+0000","last_fullsized":"2023-09-30T22:21:35.773378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:21:36.775197+0000","last_change":"2023-09-30T22:21:36.775197+0000","last_active":"2023-09-30T22:21:36.775197+0000","last_peered":"2023-09-30T22:21:36.775197+0000","last_clean":"2023-09-30T22:21:36.775197+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:21:36.775197+0000","last_undegraded":"2023-09-30T22:21:36.775197+0000","last_fullsized":"2023-09-30T22:21:36.775197+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]}]},{"osd":1,"up_from":9,"seq":38654705666,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297208,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074632,"statfs":{"total":96636764160,"available":96332423168,"internally_reserved":0,"allocated":114688,"data_stored":30204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:21:41.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.443+0000 7f7e6ac28700 1 Processor -- start 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.443+0000 7f7e6ac28700 1 -- start start 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e6ac28700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e640986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e6ac28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e64097c30 0x7f7e64098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e6ac28700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e64096160 con 0x7f7e64099080 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e6ac28700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e640654d0 con 0x7f7e64097c30 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e6ac28700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64099530 con 0x7f7e640650f0 2023-09-30T22:21:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e640986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e63fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e64097c30 0x7f7e64098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e691c5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e64099080 0x7f7e64131470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54738/0 (socket says 172.21.15.32:54738) 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e640986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35448/0 (socket says 172.21.15.32:35448) 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e691c5700 1 -- 172.21.15.32:0/2998410541 learned_addr learned my addr 172.21.15.32:0/2998410541 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 -- 172.21.15.32:0/2998410541 >> v2:172.21.15.32:3300/0 conn(0x7f7e64097c30 msgr2=0x7f7e64098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 --2- 172.21.15.32:0/2998410541 >> v2:172.21.15.32:3300/0 conn(0x7f7e64097c30 0x7f7e64098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 -- 172.21.15.32:0/2998410541 >> v1:172.21.15.32:6789/0 conn(0x7f7e64099080 legacy=0x7f7e64131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 -- 172.21.15.32:0/2998410541 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64099790 con 0x7f7e640650f0 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e63fff700 1 --2- 172.21.15.32:0/2998410541 >> v2:172.21.15.32:3300/0 conn(0x7f7e64097c30 0x7f7e64098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e689c4700 1 --2- 172.21.15.32:0/2998410541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e640986a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7e580097c0 tx=0x7f7e58009060).ready entity=mon.2 client_cookie=df80afabf51b4177 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e637fe700 1 -- 172.21.15.32:0/2998410541 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800f510 con 0x7f7e640650f0 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e637fe700 1 -- 172.21.15.32:0/2998410541 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5800f670 con 0x7f7e640650f0 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.444+0000 7f7e637fe700 1 -- 172.21.15.32:0/2998410541 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800f9f0 con 0x7f7e640650f0 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 -- 172.21.15.32:0/2998410541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 msgr2=0x7f7e640986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/2998410541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e640986a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f7e580097c0 tx=0x7f7e58009060).stop 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 -- 172.21.15.32:0/2998410541 shutdown_connections 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/2998410541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e640986a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/2998410541 >> v2:172.21.15.32:3300/0 conn(0x7f7e64097c30 0x7f7e64098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 -- 172.21.15.32:0/2998410541 >> 172.21.15.32:0/2998410541 conn(0x7f7e6409dc70 msgr2=0x7f7e6409e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 -- 172.21.15.32:0/2998410541 shutdown_connections 2023-09-30T22:21:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.445+0000 7f7e6ac28700 1 -- 172.21.15.32:0/2998410541 wait complete. 2023-09-30T22:21:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.446+0000 7f7e6ac28700 1 Processor -- start 2023-09-30T22:21:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.446+0000 7f7e6ac28700 1 -- start start 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e6ac28700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e6413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e6ac28700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e64099080 0x7f7e6413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e6413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e6413a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35456/0 (socket says 172.21.15.32:35456) 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e63fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e64099080 0x7f7e6413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e63fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e64099080 0x7f7e6413ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40762/0 (socket says 172.21.15.32:40762) 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e6ac28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e6413b350 0x7f7e6411a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 -- 172.21.15.32:0/4043427378 learned_addr learned my addr 172.21.15.32:0/4043427378 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6412f730 con 0x7f7e6413b350 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6411cc60 con 0x7f7e64099080 2023-09-30T22:21:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6411db30 con 0x7f7e640650f0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e691c5700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e6413b350 0x7f7e6411a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 -- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e64099080 msgr2=0x7f7e6413ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e64099080 0x7f7e6413ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 -- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e6413b350 msgr2=0x7f7e6411a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e6413b350 0x7f7e6411a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e500092a0 con 0x7f7e640650f0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.447+0000 7f7e689c4700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e6413a970 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f7e58009e60 tx=0x7f7e58015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.448+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800f7e0 con 0x7f7e640650f0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.448+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5800f960 con 0x7f7e640650f0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.448+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e580092a0 con 0x7f7e640650f0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.448+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5801b660 con 0x7f7e640650f0 2023-09-30T22:21:41.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.448+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6411e790 con 0x7f7e640650f0 2023-09-30T22:21:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.448+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7e5801ba10 con 0x7f7e640650f0 2023-09-30T22:21:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.449+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7e5803c940 con 0x7f7e640650f0 2023-09-30T22:21:41.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.449+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e64099a80 con 0x7f7e640650f0 2023-09-30T22:21:41.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.453+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7e58010760 con 0x7f7e640650f0 2023-09-30T22:21:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.625+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 --> v1:172.21.15.32:6816/118188 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e6411e980 con 0x7f7e4802bb90 2023-09-30T22:21:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.632+0000 7f7e60ff9700 1 -- 172.21.15.32:0/4043427378 <== mgr.4101 v1:172.21.15.32:6816/118188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250038 (unknown 2965378022 0 2661551545) 0x7f7e6411e980 con 0x7f7e4802bb90 2023-09-30T22:21:41.635 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:41.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 >> v1:172.21.15.32:6816/118188 conn(0x7f7e4802bb90 legacy=0x7f7e4802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 msgr2=0x7f7e6413a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e6413a970 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f7e58009e60 tx=0x7f7e58015000).stop 2023-09-30T22:21:41.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 shutdown_connections 2023-09-30T22:21:41.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e6413b350 0x7f7e6411a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e640650f0 0x7f7e6413a970 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 --2- 172.21.15.32:0/4043427378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e64099080 0x7f7e6413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 >> 172.21.15.32:0/4043427378 conn(0x7f7e6409dc70 msgr2=0x7f7e641142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 shutdown_connections 2023-09-30T22:21:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.635+0000 7f7e6ac28700 1 -- 172.21.15.32:0/4043427378 wait complete. 2023-09-30T22:21:41.638 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T22:21:41.650 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-09-30T22:21:41.534570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221356+0000","last_change":"2023-09-30T22:21:37.213744+0000","last_active":"2023-09-30T22:21:37.221356+0000","last_peered":"2023-09-30T22:21:37.221356+0000","last_clean":"2023-09-30T22:21:37.221356+0000","last_became_active":"2023-09-30T22:21:37.213432+0000","last_became_peered":"2023-09-30T22:21:37.213432+0000","last_unstale":"2023-09-30T22:21:37.221356+0000","last_undegraded":"2023-09-30T22:21:37.221356+0000","last_fullsized":"2023-09-30T22:21:37.221356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221210+0000","last_change":"2023-09-30T22:21:37.209966+0000","last_active":"2023-09-30T22:21:37.221210+0000","last_peered":"2023-09-30T22:21:37.221210+0000","last_clean":"2023-09-30T22:21:37.221210+0000","last_became_active":"2023-09-30T22:21:37.209700+0000","last_became_peered":"2023-09-30T22:21:37.209700+0000","last_unstale":"2023-09-30T22:21:37.221210+0000","last_undegraded":"2023-09-30T22:21:37.221210+0000","last_fullsized":"2023-09-30T22:21:37.221210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233883+0000","last_change":"2023-09-30T22:21:37.227972+0000","last_active":"2023-09-30T22:21:37.233883+0000","last_peered":"2023-09-30T22:21:37.233883+0000","last_clean":"2023-09-30T22:21:37.233883+0000","last_became_active":"2023-09-30T22:21:37.227698+0000","last_became_peered":"2023-09-30T22:21:37.227698+0000","last_unstale":"2023-09-30T22:21:37.233883+0000","last_undegraded":"2023-09-30T22:21:37.233883+0000","last_fullsized":"2023-09-30T22:21:37.233883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.218248+0000","last_change":"2023-09-30T22:21:37.217682+0000","last_active":"2023-09-30T22:21:37.218248+0000","last_peered":"2023-09-30T22:21:37.218248+0000","last_clean":"2023-09-30T22:21:37.218248+0000","last_became_active":"2023-09-30T22:21:37.217395+0000","last_became_peered":"2023-09-30T22:21:37.217395+0000","last_unstale":"2023-09-30T22:21:37.218248+0000","last_undegraded":"2023-09-30T22:21:37.218248+0000","last_fullsized":"2023-09-30T22:21:37.218248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234071+0000","last_change":"2023-09-30T22:21:37.229907+0000","last_active":"2023-09-30T22:21:37.234071+0000","last_peered":"2023-09-30T22:21:37.234071+0000","last_clean":"2023-09-30T22:21:37.234071+0000","last_became_active":"2023-09-30T22:21:37.229717+0000","last_became_peered":"2023-09-30T22:21:37.229717+0000","last_unstale":"2023-09-30T22:21:37.234071+0000","last_undegraded":"2023-09-30T22:21:37.234071+0000","last_fullsized":"2023-09-30T22:21:37.234071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233648+0000","last_change":"2023-09-30T22:21:37.224923+0000","last_active":"2023-09-30T22:21:37.233648+0000","last_peered":"2023-09-30T22:21:37.233648+0000","last_clean":"2023-09-30T22:21:37.233648+0000","last_became_active":"2023-09-30T22:21:37.224651+0000","last_became_peered":"2023-09-30T22:21:37.224651+0000","last_unstale":"2023-09-30T22:21:37.233648+0000","last_undegraded":"2023-09-30T22:21:37.233648+0000","last_fullsized":"2023-09-30T22:21:37.233648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225058+0000","last_change":"2023-09-30T22:21:37.220818+0000","last_active":"2023-09-30T22:21:37.225058+0000","last_peered":"2023-09-30T22:21:37.225058+0000","last_clean":"2023-09-30T22:21:37.225058+0000","last_became_active":"2023-09-30T22:21:37.217825+0000","last_became_peered":"2023-09-30T22:21:37.217825+0000","last_unstale":"2023-09-30T22:21:37.225058+0000","last_undegraded":"2023-09-30T22:21:37.225058+0000","last_fullsized":"2023-09-30T22:21:37.225058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239537+0000","last_change":"2023-09-30T22:21:37.237512+0000","last_active":"2023-09-30T22:21:37.239537+0000","last_peered":"2023-09-30T22:21:37.239537+0000","last_clean":"2023-09-30T22:21:37.239537+0000","last_became_active":"2023-09-30T22:21:37.237217+0000","last_became_peered":"2023-09-30T22:21:37.237217+0000","last_unstale":"2023-09-30T22:21:37.239537+0000","last_undegraded":"2023-09-30T22:21:37.239537+0000","last_fullsized":"2023-09-30T22:21:37.239537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234278+0000","last_change":"2023-09-30T22:21:37.231447+0000","last_active":"2023-09-30T22:21:37.234278+0000","last_peered":"2023-09-30T22:21:37.234278+0000","last_clean":"2023-09-30T22:21:37.234278+0000","last_became_active":"2023-09-30T22:21:37.231106+0000","last_became_peered":"2023-09-30T22:21:37.231106+0000","last_unstale":"2023-09-30T22:21:37.234278+0000","last_undegraded":"2023-09-30T22:21:37.234278+0000","last_fullsized":"2023-09-30T22:21:37.234278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239686+0000","last_change":"2023-09-30T22:21:37.238710+0000","last_active":"2023-09-30T22:21:37.239686+0000","last_peered":"2023-09-30T22:21:37.239686+0000","last_clean":"2023-09-30T22:21:37.239686+0000","last_became_active":"2023-09-30T22:21:37.237230+0000","last_became_peered":"2023-09-30T22:21:37.237230+0000","last_unstale":"2023-09-30T22:21:37.239686+0000","last_undegraded":"2023-09-30T22:21:37.239686+0000","last_fullsized":"2023-09-30T22:21:37.239686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239518+0000","last_change":"2023-09-30T22:21:37.238711+0000","last_active":"2023-09-30T22:21:37.239518+0000","last_peered":"2023-09-30T22:21:37.239518+0000","last_clean":"2023-09-30T22:21:37.239518+0000","last_became_active":"2023-09-30T22:21:37.237399+0000","last_became_peered":"2023-09-30T22:21:37.237399+0000","last_unstale":"2023-09-30T22:21:37.239518+0000","last_undegraded":"2023-09-30T22:21:37.239518+0000","last_fullsized":"2023-09-30T22:21:37.239518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223819+0000","last_change":"2023-09-30T22:21:37.215868+0000","last_active":"2023-09-30T22:21:37.223819+0000","last_peered":"2023-09-30T22:21:37.223819+0000","last_clean":"2023-09-30T22:21:37.223819+0000","last_became_active":"2023-09-30T22:21:37.215601+0000","last_became_peered":"2023-09-30T22:21:37.215601+0000","last_unstale":"2023-09-30T22:21:37.223819+0000","last_undegraded":"2023-09-30T22:21:37.223819+0000","last_fullsized":"2023-09-30T22:21:37.223819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232299+0000","last_change":"2023-09-30T22:21:37.229175+0000","last_active":"2023-09-30T22:21:37.232299+0000","last_peered":"2023-09-30T22:21:37.232299+0000","last_clean":"2023-09-30T22:21:37.232299+0000","last_became_active":"2023-09-30T22:21:37.226712+0000","last_became_peered":"2023-09-30T22:21:37.226712+0000","last_unstale":"2023-09-30T22:21:37.232299+0000","last_undegraded":"2023-09-30T22:21:37.232299+0000","last_fullsized":"2023-09-30T22:21:37.232299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225658+0000","last_change":"2023-09-30T22:21:37.222596+0000","last_active":"2023-09-30T22:21:37.225658+0000","last_peered":"2023-09-30T22:21:37.225658+0000","last_clean":"2023-09-30T22:21:37.225658+0000","last_became_active":"2023-09-30T22:21:37.222295+0000","last_became_peered":"2023-09-30T22:21:37.222295+0000","last_unstale":"2023-09-30T22:21:37.225658+0000","last_undegraded":"2023-09-30T22:21:37.225658+0000","last_fullsized":"2023-09-30T22:21:37.225658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.218011+0000","last_change":"2023-09-30T22:21:37.215505+0000","last_active":"2023-09-30T22:21:37.218011+0000","last_peered":"2023-09-30T22:21:37.218011+0000","last_clean":"2023-09-30T22:21:37.218011+0000","last_became_active":"2023-09-30T22:21:37.215015+0000","last_became_peered":"2023-09-30T22:21:37.215015+0000","last_unstale":"2023-09-30T22:21:37.218011+0000","last_undegraded":"2023-09-30T22:21:37.218011+0000","last_fullsized":"2023-09-30T22:21:37.218011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.224429+0000","last_change":"2023-09-30T22:21:37.220367+0000","last_active":"2023-09-30T22:21:37.224429+0000","last_peered":"2023-09-30T22:21:37.224429+0000","last_clean":"2023-09-30T22:21:37.224429+0000","last_became_active":"2023-09-30T22:21:37.217445+0000","last_became_peered":"2023-09-30T22:21:37.217445+0000","last_unstale":"2023-09-30T22:21:37.224429+0000","last_undegraded":"2023-09-30T22:21:37.224429+0000","last_fullsized":"2023-09-30T22:21:37.224429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234262+0000","last_change":"2023-09-30T22:21:37.231390+0000","last_active":"2023-09-30T22:21:37.234262+0000","last_peered":"2023-09-30T22:21:37.234262+0000","last_clean":"2023-09-30T22:21:37.234262+0000","last_became_active":"2023-09-30T22:21:37.231116+0000","last_became_peered":"2023-09-30T22:21:37.231116+0000","last_unstale":"2023-09-30T22:21:37.234262+0000","last_undegraded":"2023-09-30T22:21:37.234262+0000","last_fullsized":"2023-09-30T22:21:37.234262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234120+0000","last_change":"2023-09-30T22:21:37.228923+0000","last_active":"2023-09-30T22:21:37.234120+0000","last_peered":"2023-09-30T22:21:37.234120+0000","last_clean":"2023-09-30T22:21:37.234120+0000","last_became_active":"2023-09-30T22:21:37.220140+0000","last_became_peered":"2023-09-30T22:21:37.220140+0000","last_unstale":"2023-09-30T22:21:37.234120+0000","last_undegraded":"2023-09-30T22:21:37.234120+0000","last_fullsized":"2023-09-30T22:21:37.234120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.228910+0000","last_change":"2023-09-30T22:21:37.225422+0000","last_active":"2023-09-30T22:21:37.228910+0000","last_peered":"2023-09-30T22:21:37.228910+0000","last_clean":"2023-09-30T22:21:37.228910+0000","last_became_active":"2023-09-30T22:21:37.225118+0000","last_became_peered":"2023-09-30T22:21:37.225118+0000","last_unstale":"2023-09-30T22:21:37.228910+0000","last_undegraded":"2023-09-30T22:21:37.228910+0000","last_fullsized":"2023-09-30T22:21:37.228910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220876+0000","last_change":"2023-09-30T22:21:37.218484+0000","last_active":"2023-09-30T22:21:37.220876+0000","last_peered":"2023-09-30T22:21:37.220876+0000","last_clean":"2023-09-30T22:21:37.220876+0000","last_became_active":"2023-09-30T22:21:37.218141+0000","last_became_peered":"2023-09-30T22:21:37.218141+0000","last_unstale":"2023-09-30T22:21:37.220876+0000","last_undegraded":"2023-09-30T22:21:37.220876+0000","last_fullsized":"2023-09-30T22:21:37.220876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.219680+0000","last_change":"2023-09-30T22:21:37.216211+0000","last_active":"2023-09-30T22:21:37.219680+0000","last_peered":"2023-09-30T22:21:37.219680+0000","last_clean":"2023-09-30T22:21:37.219680+0000","last_became_active":"2023-09-30T22:21:37.215922+0000","last_became_peered":"2023-09-30T22:21:37.215922+0000","last_unstale":"2023-09-30T22:21:37.219680+0000","last_undegraded":"2023-09-30T22:21:37.219680+0000","last_fullsized":"2023-09-30T22:21:37.219680+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233722+0000","last_change":"2023-09-30T22:21:37.227137+0000","last_active":"2023-09-30T22:21:37.233722+0000","last_peered":"2023-09-30T22:21:37.233722+0000","last_clean":"2023-09-30T22:21:37.233722+0000","last_became_active":"2023-09-30T22:21:37.226848+0000","last_became_peered":"2023-09-30T22:21:37.226848+0000","last_unstale":"2023-09-30T22:21:37.233722+0000","last_undegraded":"2023-09-30T22:21:37.233722+0000","last_fullsized":"2023-09-30T22:21:37.233722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225020+0000","last_change":"2023-09-30T22:21:37.220415+0000","last_active":"2023-09-30T22:21:37.225020+0000","last_peered":"2023-09-30T22:21:37.225020+0000","last_clean":"2023-09-30T22:21:37.225020+0000","last_became_active":"2023-09-30T22:21:37.217468+0000","last_became_peered":"2023-09-30T22:21:37.217468+0000","last_unstale":"2023-09-30T22:21:37.225020+0000","last_undegraded":"2023-09-30T22:21:37.225020+0000","last_fullsized":"2023-09-30T22:21:37.225020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221521+0000","last_change":"2023-09-30T22:21:37.220204+0000","last_active":"2023-09-30T22:21:37.221521+0000","last_peered":"2023-09-30T22:21:37.221521+0000","last_clean":"2023-09-30T22:21:37.221521+0000","last_became_active":"2023-09-30T22:21:37.219099+0000","last_became_peered":"2023-09-30T22:21:37.219099+0000","last_unstale":"2023-09-30T22:21:37.221521+0000","last_undegraded":"2023-09-30T22:21:37.221521+0000","last_fullsized":"2023-09-30T22:21:37.221521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220336+0000","last_change":"2023-09-30T22:21:37.217211+0000","last_active":"2023-09-30T22:21:37.220336+0000","last_peered":"2023-09-30T22:21:37.220336+0000","last_clean":"2023-09-30T22:21:37.220336+0000","last_became_active":"2023-09-30T22:21:37.216537+0000","last_became_peered":"2023-09-30T22:21:37.216537+0000","last_unstale":"2023-09-30T22:21:37.220336+0000","last_undegraded":"2023-09-30T22:21:37.220336+0000","last_fullsized":"2023-09-30T22:21:37.220336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.213682+0000","last_change":"2023-09-30T22:21:37.209836+0000","last_active":"2023-09-30T22:21:37.213682+0000","last_peered":"2023-09-30T22:21:37.213682+0000","last_clean":"2023-09-30T22:21:37.213682+0000","last_became_active":"2023-09-30T22:21:37.209561+0000","last_became_peered":"2023-09-30T22:21:37.209561+0000","last_unstale":"2023-09-30T22:21:37.213682+0000","last_undegraded":"2023-09-30T22:21:37.213682+0000","last_fullsized":"2023-09-30T22:21:37.213682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226409+0000","last_change":"2023-09-30T22:21:37.220637+0000","last_active":"2023-09-30T22:21:37.226409+0000","last_peered":"2023-09-30T22:21:37.226409+0000","last_clean":"2023-09-30T22:21:37.226409+0000","last_became_active":"2023-09-30T22:21:37.217741+0000","last_became_peered":"2023-09-30T22:21:37.217741+0000","last_unstale":"2023-09-30T22:21:37.226409+0000","last_undegraded":"2023-09-30T22:21:37.226409+0000","last_fullsized":"2023-09-30T22:21:37.226409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.222723+0000","last_change":"2023-09-30T22:21:36.800514+0000","last_active":"2023-09-30T22:21:37.222723+0000","last_peered":"2023-09-30T22:21:37.222723+0000","last_clean":"2023-09-30T22:21:37.222723+0000","last_became_active":"2023-09-30T22:21:36.799775+0000","last_became_peered":"2023-09-30T22:21:36.799775+0000","last_unstale":"2023-09-30T22:21:37.222723+0000","last_undegraded":"2023-09-30T22:21:37.222723+0000","last_fullsized":"2023-09-30T22:21:37.222723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.224088+0000","last_change":"2023-09-30T22:21:37.211849+0000","last_active":"2023-09-30T22:21:37.224088+0000","last_peered":"2023-09-30T22:21:37.224088+0000","last_clean":"2023-09-30T22:21:37.224088+0000","last_became_active":"2023-09-30T22:21:37.209128+0000","last_became_peered":"2023-09-30T22:21:37.209128+0000","last_unstale":"2023-09-30T22:21:37.224088+0000","last_undegraded":"2023-09-30T22:21:37.224088+0000","last_fullsized":"2023-09-30T22:21:37.224088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198669+0000","last_change":"2023-09-30T22:21:36.803467+0000","last_active":"2023-09-30T22:21:37.198669+0000","last_peered":"2023-09-30T22:21:37.198669+0000","last_clean":"2023-09-30T22:21:37.198669+0000","last_became_active":"2023-09-30T22:21:36.803141+0000","last_became_peered":"2023-09-30T22:21:36.803141+0000","last_unstale":"2023-09-30T22:21:37.198669+0000","last_undegraded":"2023-09-30T22:21:37.198669+0000","last_fullsized":"2023-09-30T22:21:37.198669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233388+0000","last_change":"2023-09-30T22:21:37.224566+0000","last_active":"2023-09-30T22:21:37.233388+0000","last_peered":"2023-09-30T22:21:37.233388+0000","last_clean":"2023-09-30T22:21:37.233388+0000","last_became_active":"2023-09-30T22:21:37.219843+0000","last_became_peered":"2023-09-30T22:21:37.219843+0000","last_unstale":"2023-09-30T22:21:37.233388+0000","last_undegraded":"2023-09-30T22:21:37.233388+0000","last_fullsized":"2023-09-30T22:21:37.233388+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195923+0000","last_change":"2023-09-30T22:21:36.813865+0000","last_active":"2023-09-30T22:21:37.195923+0000","last_peered":"2023-09-30T22:21:37.195923+0000","last_clean":"2023-09-30T22:21:37.195923+0000","last_became_active":"2023-09-30T22:21:36.812871+0000","last_became_peered":"2023-09-30T22:21:36.812871+0000","last_unstale":"2023-09-30T22:21:37.195923+0000","last_undegraded":"2023-09-30T22:21:37.195923+0000","last_fullsized":"2023-09-30T22:21:37.195923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226177+0000","last_change":"2023-09-30T22:21:37.215838+0000","last_active":"2023-09-30T22:21:37.226177+0000","last_peered":"2023-09-30T22:21:37.226177+0000","last_clean":"2023-09-30T22:21:37.226177+0000","last_became_active":"2023-09-30T22:21:37.215314+0000","last_became_peered":"2023-09-30T22:21:37.215314+0000","last_unstale":"2023-09-30T22:21:37.226177+0000","last_undegraded":"2023-09-30T22:21:37.226177+0000","last_fullsized":"2023-09-30T22:21:37.226177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204605+0000","last_change":"2023-09-30T22:21:36.809280+0000","last_active":"2023-09-30T22:21:37.204605+0000","last_peered":"2023-09-30T22:21:37.204605+0000","last_clean":"2023-09-30T22:21:37.204605+0000","last_became_active":"2023-09-30T22:21:36.808975+0000","last_became_peered":"2023-09-30T22:21:36.808975+0000","last_unstale":"2023-09-30T22:21:37.204605+0000","last_undegraded":"2023-09-30T22:21:37.204605+0000","last_fullsized":"2023-09-30T22:21:37.204605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223892+0000","last_change":"2023-09-30T22:21:37.215010+0000","last_active":"2023-09-30T22:21:37.223892+0000","last_peered":"2023-09-30T22:21:37.223892+0000","last_clean":"2023-09-30T22:21:37.223892+0000","last_became_active":"2023-09-30T22:21:37.212913+0000","last_became_peered":"2023-09-30T22:21:37.212913+0000","last_unstale":"2023-09-30T22:21:37.223892+0000","last_undegraded":"2023-09-30T22:21:37.223892+0000","last_fullsized":"2023-09-30T22:21:37.223892+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.201567+0000","last_change":"2023-09-30T22:21:36.800431+0000","last_active":"2023-09-30T22:21:37.201567+0000","last_peered":"2023-09-30T22:21:37.201567+0000","last_clean":"2023-09-30T22:21:37.201567+0000","last_became_active":"2023-09-30T22:21:36.800130+0000","last_became_peered":"2023-09-30T22:21:36.800130+0000","last_unstale":"2023-09-30T22:21:37.201567+0000","last_undegraded":"2023-09-30T22:21:37.201567+0000","last_fullsized":"2023-09-30T22:21:37.201567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225077+0000","last_change":"2023-09-30T22:21:37.220978+0000","last_active":"2023-09-30T22:21:37.225077+0000","last_peered":"2023-09-30T22:21:37.225077+0000","last_clean":"2023-09-30T22:21:37.225077+0000","last_became_active":"2023-09-30T22:21:37.218384+0000","last_became_peered":"2023-09-30T22:21:37.218384+0000","last_unstale":"2023-09-30T22:21:37.225077+0000","last_undegraded":"2023-09-30T22:21:37.225077+0000","last_fullsized":"2023-09-30T22:21:37.225077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.231706+0000","last_change":"2023-09-30T22:21:36.810931+0000","last_active":"2023-09-30T22:21:37.231706+0000","last_peered":"2023-09-30T22:21:37.231706+0000","last_clean":"2023-09-30T22:21:37.231706+0000","last_became_active":"2023-09-30T22:21:36.810625+0000","last_became_peered":"2023-09-30T22:21:36.810625+0000","last_unstale":"2023-09-30T22:21:37.231706+0000","last_undegraded":"2023-09-30T22:21:37.231706+0000","last_fullsized":"2023-09-30T22:21:37.231706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220744+0000","last_change":"2023-09-30T22:21:37.217701+0000","last_active":"2023-09-30T22:21:37.220744+0000","last_peered":"2023-09-30T22:21:37.220744+0000","last_clean":"2023-09-30T22:21:37.220744+0000","last_became_active":"2023-09-30T22:21:37.217076+0000","last_became_peered":"2023-09-30T22:21:37.217076+0000","last_unstale":"2023-09-30T22:21:37.220744+0000","last_undegraded":"2023-09-30T22:21:37.220744+0000","last_fullsized":"2023-09-30T22:21:37.220744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.236574+0000","last_change":"2023-09-30T22:21:36.811297+0000","last_active":"2023-09-30T22:21:37.236574+0000","last_peered":"2023-09-30T22:21:37.236574+0000","last_clean":"2023-09-30T22:21:37.236574+0000","last_became_active":"2023-09-30T22:21:36.810987+0000","last_became_peered":"2023-09-30T22:21:36.810987+0000","last_unstale":"2023-09-30T22:21:37.236574+0000","last_undegraded":"2023-09-30T22:21:37.236574+0000","last_fullsized":"2023-09-30T22:21:37.236574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232281+0000","last_change":"2023-09-30T22:21:37.228789+0000","last_active":"2023-09-30T22:21:37.232281+0000","last_peered":"2023-09-30T22:21:37.232281+0000","last_clean":"2023-09-30T22:21:37.232281+0000","last_became_active":"2023-09-30T22:21:37.225220+0000","last_became_peered":"2023-09-30T22:21:37.225220+0000","last_unstale":"2023-09-30T22:21:37.232281+0000","last_undegraded":"2023-09-30T22:21:37.232281+0000","last_fullsized":"2023-09-30T22:21:37.232281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235370+0000","last_change":"2023-09-30T22:21:36.813567+0000","last_active":"2023-09-30T22:21:37.235370+0000","last_peered":"2023-09-30T22:21:37.235370+0000","last_clean":"2023-09-30T22:21:37.235370+0000","last_became_active":"2023-09-30T22:21:36.813257+0000","last_became_peered":"2023-09-30T22:21:36.813257+0000","last_unstale":"2023-09-30T22:21:37.235370+0000","last_undegraded":"2023-09-30T22:21:37.235370+0000","last_fullsized":"2023-09-30T22:21:37.235370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223168+0000","last_change":"2023-09-30T22:21:37.213340+0000","last_active":"2023-09-30T22:21:37.223168+0000","last_peered":"2023-09-30T22:21:37.223168+0000","last_clean":"2023-09-30T22:21:37.223168+0000","last_became_active":"2023-09-30T22:21:37.213040+0000","last_became_peered":"2023-09-30T22:21:37.213040+0000","last_unstale":"2023-09-30T22:21:37.223168+0000","last_undegraded":"2023-09-30T22:21:37.223168+0000","last_fullsized":"2023-09-30T22:21:37.223168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.205101+0000","last_change":"2023-09-30T22:21:36.806015+0000","last_active":"2023-09-30T22:21:37.205101+0000","last_peered":"2023-09-30T22:21:37.205101+0000","last_clean":"2023-09-30T22:21:37.205101+0000","last_became_active":"2023-09-30T22:21:36.805679+0000","last_became_peered":"2023-09-30T22:21:36.805679+0000","last_unstale":"2023-09-30T22:21:37.205101+0000","last_undegraded":"2023-09-30T22:21:37.205101+0000","last_fullsized":"2023-09-30T22:21:37.205101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226450+0000","last_change":"2023-09-30T22:21:37.221673+0000","last_active":"2023-09-30T22:21:37.226450+0000","last_peered":"2023-09-30T22:21:37.226450+0000","last_clean":"2023-09-30T22:21:37.226450+0000","last_became_active":"2023-09-30T22:21:37.221388+0000","last_became_peered":"2023-09-30T22:21:37.221388+0000","last_unstale":"2023-09-30T22:21:37.226450+0000","last_undegraded":"2023-09-30T22:21:37.226450+0000","last_fullsized":"2023-09-30T22:21:37.226450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199788+0000","last_change":"2023-09-30T22:21:36.812303+0000","last_active":"2023-09-30T22:21:37.199788+0000","last_peered":"2023-09-30T22:21:37.199788+0000","last_clean":"2023-09-30T22:21:37.199788+0000","last_became_active":"2023-09-30T22:21:36.811887+0000","last_became_peered":"2023-09-30T22:21:36.811887+0000","last_unstale":"2023-09-30T22:21:37.199788+0000","last_undegraded":"2023-09-30T22:21:37.199788+0000","last_fullsized":"2023-09-30T22:21:37.199788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234099+0000","last_change":"2023-09-30T22:21:37.225272+0000","last_active":"2023-09-30T22:21:37.234099+0000","last_peered":"2023-09-30T22:21:37.234099+0000","last_clean":"2023-09-30T22:21:37.234099+0000","last_became_active":"2023-09-30T22:21:37.224827+0000","last_became_peered":"2023-09-30T22:21:37.224827+0000","last_unstale":"2023-09-30T22:21:37.234099+0000","last_undegraded":"2023-09-30T22:21:37.234099+0000","last_fullsized":"2023-09-30T22:21:37.234099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239712+0000","last_change":"2023-09-30T22:21:36.804342+0000","last_active":"2023-09-30T22:21:37.239712+0000","last_peered":"2023-09-30T22:21:37.239712+0000","last_clean":"2023-09-30T22:21:37.239712+0000","last_became_active":"2023-09-30T22:21:36.804002+0000","last_became_peered":"2023-09-30T22:21:36.804002+0000","last_unstale":"2023-09-30T22:21:37.239712+0000","last_undegraded":"2023-09-30T22:21:37.239712+0000","last_fullsized":"2023-09-30T22:21:37.239712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226154+0000","last_change":"2023-09-30T22:21:37.215043+0000","last_active":"2023-09-30T22:21:37.226154+0000","last_peered":"2023-09-30T22:21:37.226154+0000","last_clean":"2023-09-30T22:21:37.226154+0000","last_became_active":"2023-09-30T22:21:37.214750+0000","last_became_peered":"2023-09-30T22:21:37.214750+0000","last_unstale":"2023-09-30T22:21:37.226154+0000","last_undegraded":"2023-09-30T22:21:37.226154+0000","last_fullsized":"2023-09-30T22:21:37.226154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239337+0000","last_change":"2023-09-30T22:21:36.796103+0000","last_active":"2023-09-30T22:21:37.239337+0000","last_peered":"2023-09-30T22:21:37.239337+0000","last_clean":"2023-09-30T22:21:37.239337+0000","last_became_active":"2023-09-30T22:21:36.795669+0000","last_became_peered":"2023-09-30T22:21:36.795669+0000","last_unstale":"2023-09-30T22:21:37.239337+0000","last_undegraded":"2023-09-30T22:21:37.239337+0000","last_fullsized":"2023-09-30T22:21:37.239337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226133+0000","last_change":"2023-09-30T22:21:37.223782+0000","last_active":"2023-09-30T22:21:37.226133+0000","last_peered":"2023-09-30T22:21:37.226133+0000","last_clean":"2023-09-30T22:21:37.226133+0000","last_became_active":"2023-09-30T22:21:37.223508+0000","last_became_peered":"2023-09-30T22:21:37.223508+0000","last_unstale":"2023-09-30T22:21:37.226133+0000","last_undegraded":"2023-09-30T22:21:37.226133+0000","last_fullsized":"2023-09-30T22:21:37.226133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.230042+0000","last_change":"2023-09-30T22:21:36.798241+0000","last_active":"2023-09-30T22:21:37.230042+0000","last_peered":"2023-09-30T22:21:37.230042+0000","last_clean":"2023-09-30T22:21:37.230042+0000","last_became_active":"2023-09-30T22:21:36.797923+0000","last_became_peered":"2023-09-30T22:21:36.797923+0000","last_unstale":"2023-09-30T22:21:37.230042+0000","last_undegraded":"2023-09-30T22:21:37.230042+0000","last_fullsized":"2023-09-30T22:21:37.230042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225577+0000","last_change":"2023-09-30T22:21:37.223841+0000","last_active":"2023-09-30T22:21:37.225577+0000","last_peered":"2023-09-30T22:21:37.225577+0000","last_clean":"2023-09-30T22:21:37.225577+0000","last_became_active":"2023-09-30T22:21:37.223485+0000","last_became_peered":"2023-09-30T22:21:37.223485+0000","last_unstale":"2023-09-30T22:21:37.225577+0000","last_undegraded":"2023-09-30T22:21:37.225577+0000","last_fullsized":"2023-09-30T22:21:37.225577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238213+0000","last_change":"2023-09-30T22:21:36.817374+0000","last_active":"2023-09-30T22:21:37.238213+0000","last_peered":"2023-09-30T22:21:37.238213+0000","last_clean":"2023-09-30T22:21:37.238213+0000","last_became_active":"2023-09-30T22:21:36.816913+0000","last_became_peered":"2023-09-30T22:21:36.816913+0000","last_unstale":"2023-09-30T22:21:37.238213+0000","last_undegraded":"2023-09-30T22:21:37.238213+0000","last_fullsized":"2023-09-30T22:21:37.238213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221929+0000","last_change":"2023-09-30T22:21:37.214893+0000","last_active":"2023-09-30T22:21:37.221929+0000","last_peered":"2023-09-30T22:21:37.221929+0000","last_clean":"2023-09-30T22:21:37.221929+0000","last_became_active":"2023-09-30T22:21:37.214616+0000","last_became_peered":"2023-09-30T22:21:37.214616+0000","last_unstale":"2023-09-30T22:21:37.221929+0000","last_undegraded":"2023-09-30T22:21:37.221929+0000","last_fullsized":"2023-09-30T22:21:37.221929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.202352+0000","last_change":"2023-09-30T22:21:36.806594+0000","last_active":"2023-09-30T22:21:37.202352+0000","last_peered":"2023-09-30T22:21:37.202352+0000","last_clean":"2023-09-30T22:21:37.202352+0000","last_became_active":"2023-09-30T22:21:36.806305+0000","last_became_peered":"2023-09-30T22:21:36.806305+0000","last_unstale":"2023-09-30T22:21:37.202352+0000","last_undegraded":"2023-09-30T22:21:37.202352+0000","last_fullsized":"2023-09-30T22:21:37.202352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233473+0000","last_change":"2023-09-30T22:21:37.224756+0000","last_active":"2023-09-30T22:21:37.233473+0000","last_peered":"2023-09-30T22:21:37.233473+0000","last_clean":"2023-09-30T22:21:37.233473+0000","last_became_active":"2023-09-30T22:21:37.224444+0000","last_became_peered":"2023-09-30T22:21:37.224444+0000","last_unstale":"2023-09-30T22:21:37.233473+0000","last_undegraded":"2023-09-30T22:21:37.233473+0000","last_fullsized":"2023-09-30T22:21:37.233473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235343+0000","last_change":"2023-09-30T22:21:36.813533+0000","last_active":"2023-09-30T22:21:37.235343+0000","last_peered":"2023-09-30T22:21:37.235343+0000","last_clean":"2023-09-30T22:21:37.235343+0000","last_became_active":"2023-09-30T22:21:36.813227+0000","last_became_peered":"2023-09-30T22:21:36.813227+0000","last_unstale":"2023-09-30T22:21:37.235343+0000","last_undegraded":"2023-09-30T22:21:37.235343+0000","last_fullsized":"2023-09-30T22:21:37.235343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234223+0000","last_change":"2023-09-30T22:21:37.230482+0000","last_active":"2023-09-30T22:21:37.234223+0000","last_peered":"2023-09-30T22:21:37.234223+0000","last_clean":"2023-09-30T22:21:37.234223+0000","last_became_active":"2023-09-30T22:21:37.229876+0000","last_became_peered":"2023-09-30T22:21:37.229876+0000","last_unstale":"2023-09-30T22:21:37.234223+0000","last_undegraded":"2023-09-30T22:21:37.234223+0000","last_fullsized":"2023-09-30T22:21:37.234223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199874+0000","last_change":"2023-09-30T22:21:36.811138+0000","last_active":"2023-09-30T22:21:37.199874+0000","last_peered":"2023-09-30T22:21:37.199874+0000","last_clean":"2023-09-30T22:21:37.199874+0000","last_became_active":"2023-09-30T22:21:36.810834+0000","last_became_peered":"2023-09-30T22:21:36.810834+0000","last_unstale":"2023-09-30T22:21:37.199874+0000","last_undegraded":"2023-09-30T22:21:37.199874+0000","last_fullsized":"2023-09-30T22:21:37.199874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226390+0000","last_change":"2023-09-30T22:21:37.220428+0000","last_active":"2023-09-30T22:21:37.226390+0000","last_peered":"2023-09-30T22:21:37.226390+0000","last_clean":"2023-09-30T22:21:37.226390+0000","last_became_active":"2023-09-30T22:21:37.217353+0000","last_became_peered":"2023-09-30T22:21:37.217353+0000","last_unstale":"2023-09-30T22:21:37.226390+0000","last_undegraded":"2023-09-30T22:21:37.226390+0000","last_fullsized":"2023-09-30T22:21:37.226390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195804+0000","last_change":"2023-09-30T22:21:36.812156+0000","last_active":"2023-09-30T22:21:37.195804+0000","last_peered":"2023-09-30T22:21:37.195804+0000","last_clean":"2023-09-30T22:21:37.195804+0000","last_became_active":"2023-09-30T22:21:36.811527+0000","last_became_peered":"2023-09-30T22:21:36.811527+0000","last_unstale":"2023-09-30T22:21:37.195804+0000","last_undegraded":"2023-09-30T22:21:37.195804+0000","last_fullsized":"2023-09-30T22:21:37.195804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234586+0000","last_change":"2023-09-30T22:21:37.226310+0000","last_active":"2023-09-30T22:21:37.234586+0000","last_peered":"2023-09-30T22:21:37.234586+0000","last_clean":"2023-09-30T22:21:37.234586+0000","last_became_active":"2023-09-30T22:21:37.225749+0000","last_became_peered":"2023-09-30T22:21:37.225749+0000","last_unstale":"2023-09-30T22:21:37.234586+0000","last_undegraded":"2023-09-30T22:21:37.234586+0000","last_fullsized":"2023-09-30T22:21:37.234586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195360+0000","last_change":"2023-09-30T22:21:36.811756+0000","last_active":"2023-09-30T22:21:37.195360+0000","last_peered":"2023-09-30T22:21:37.195360+0000","last_clean":"2023-09-30T22:21:37.195360+0000","last_became_active":"2023-09-30T22:21:36.811393+0000","last_became_peered":"2023-09-30T22:21:36.811393+0000","last_unstale":"2023-09-30T22:21:37.195360+0000","last_undegraded":"2023-09-30T22:21:37.195360+0000","last_fullsized":"2023-09-30T22:21:37.195360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.219948+0000","last_change":"2023-09-30T22:21:37.217010+0000","last_active":"2023-09-30T22:21:37.219948+0000","last_peered":"2023-09-30T22:21:37.219948+0000","last_clean":"2023-09-30T22:21:37.219948+0000","last_became_active":"2023-09-30T22:21:37.216071+0000","last_became_peered":"2023-09-30T22:21:37.216071+0000","last_unstale":"2023-09-30T22:21:37.219948+0000","last_undegraded":"2023-09-30T22:21:37.219948+0000","last_fullsized":"2023-09-30T22:21:37.219948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199310+0000","last_change":"2023-09-30T22:21:31.794405+0000","last_active":"2023-09-30T22:21:37.199310+0000","last_peered":"2023-09-30T22:21:37.199310+0000","last_clean":"2023-09-30T22:21:37.199310+0000","last_became_active":"2023-09-30T22:21:31.794052+0000","last_became_peered":"2023-09-30T22:21:31.794052+0000","last_unstale":"2023-09-30T22:21:37.199310+0000","last_undegraded":"2023-09-30T22:21:37.199310+0000","last_fullsized":"2023-09-30T22:21:37.199310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197712+0000","last_change":"2023-09-30T22:21:36.799995+0000","last_active":"2023-09-30T22:21:37.197712+0000","last_peered":"2023-09-30T22:21:37.197712+0000","last_clean":"2023-09-30T22:21:37.197712+0000","last_became_active":"2023-09-30T22:21:36.799674+0000","last_became_peered":"2023-09-30T22:21:36.799674+0000","last_unstale":"2023-09-30T22:21:37.197712+0000","last_undegraded":"2023-09-30T22:21:37.197712+0000","last_fullsized":"2023-09-30T22:21:37.197712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220766+0000","last_change":"2023-09-30T22:21:37.217877+0000","last_active":"2023-09-30T22:21:37.220766+0000","last_peered":"2023-09-30T22:21:37.220766+0000","last_clean":"2023-09-30T22:21:37.220766+0000","last_became_active":"2023-09-30T22:21:37.217430+0000","last_became_peered":"2023-09-30T22:21:37.217430+0000","last_unstale":"2023-09-30T22:21:37.220766+0000","last_undegraded":"2023-09-30T22:21:37.220766+0000","last_fullsized":"2023-09-30T22:21:37.220766+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199951+0000","last_change":"2023-09-30T22:21:31.798962+0000","last_active":"2023-09-30T22:21:37.199951+0000","last_peered":"2023-09-30T22:21:37.199951+0000","last_clean":"2023-09-30T22:21:37.199951+0000","last_became_active":"2023-09-30T22:21:31.798641+0000","last_became_peered":"2023-09-30T22:21:31.798641+0000","last_unstale":"2023-09-30T22:21:37.199951+0000","last_undegraded":"2023-09-30T22:21:37.199951+0000","last_fullsized":"2023-09-30T22:21:37.199951+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204066+0000","last_change":"2023-09-30T22:21:36.793285+0000","last_active":"2023-09-30T22:21:37.204066+0000","last_peered":"2023-09-30T22:21:37.204066+0000","last_clean":"2023-09-30T22:21:37.204066+0000","last_became_active":"2023-09-30T22:21:36.793005+0000","last_became_peered":"2023-09-30T22:21:36.793005+0000","last_unstale":"2023-09-30T22:21:37.204066+0000","last_undegraded":"2023-09-30T22:21:37.204066+0000","last_fullsized":"2023-09-30T22:21:37.204066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238787+0000","last_change":"2023-09-30T22:21:37.236712+0000","last_active":"2023-09-30T22:21:37.238787+0000","last_peered":"2023-09-30T22:21:37.238787+0000","last_clean":"2023-09-30T22:21:37.238787+0000","last_became_active":"2023-09-30T22:21:37.226418+0000","last_became_peered":"2023-09-30T22:21:37.226418+0000","last_unstale":"2023-09-30T22:21:37.238787+0000","last_undegraded":"2023-09-30T22:21:37.238787+0000","last_fullsized":"2023-09-30T22:21:37.238787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197851+0000","last_change":"2023-09-30T22:21:31.797737+0000","last_active":"2023-09-30T22:21:37.197851+0000","last_peered":"2023-09-30T22:21:37.197851+0000","last_clean":"2023-09-30T22:21:37.197851+0000","last_became_active":"2023-09-30T22:21:31.797396+0000","last_became_peered":"2023-09-30T22:21:31.797396+0000","last_unstale":"2023-09-30T22:21:37.197851+0000","last_undegraded":"2023-09-30T22:21:37.197851+0000","last_fullsized":"2023-09-30T22:21:37.197851+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198511+0000","last_change":"2023-09-30T22:21:36.799423+0000","last_active":"2023-09-30T22:21:37.198511+0000","last_peered":"2023-09-30T22:21:37.198511+0000","last_clean":"2023-09-30T22:21:37.198511+0000","last_became_active":"2023-09-30T22:21:36.798817+0000","last_became_peered":"2023-09-30T22:21:36.798817+0000","last_unstale":"2023-09-30T22:21:37.198511+0000","last_undegraded":"2023-09-30T22:21:37.198511+0000","last_fullsized":"2023-09-30T22:21:37.198511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239506+0000","last_change":"2023-09-30T22:21:37.237647+0000","last_active":"2023-09-30T22:21:37.239506+0000","last_peered":"2023-09-30T22:21:37.239506+0000","last_clean":"2023-09-30T22:21:37.239506+0000","last_became_active":"2023-09-30T22:21:37.237287+0000","last_became_peered":"2023-09-30T22:21:37.237287+0000","last_unstale":"2023-09-30T22:21:37.239506+0000","last_undegraded":"2023-09-30T22:21:37.239506+0000","last_fullsized":"2023-09-30T22:21:37.239506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.196834+0000","last_change":"2023-09-30T22:21:31.798294+0000","last_active":"2023-09-30T22:21:37.196834+0000","last_peered":"2023-09-30T22:21:37.196834+0000","last_clean":"2023-09-30T22:21:37.196834+0000","last_became_active":"2023-09-30T22:21:31.797999+0000","last_became_peered":"2023-09-30T22:21:31.797999+0000","last_unstale":"2023-09-30T22:21:37.196834+0000","last_undegraded":"2023-09-30T22:21:37.196834+0000","last_fullsized":"2023-09-30T22:21:37.196834+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198428+0000","last_change":"2023-09-30T22:21:30.796302+0000","last_active":"2023-09-30T22:21:37.198428+0000","last_peered":"2023-09-30T22:21:37.198428+0000","last_clean":"2023-09-30T22:21:37.198428+0000","last_became_active":"2023-09-30T22:21:30.795885+0000","last_became_peered":"2023-09-30T22:21:30.795885+0000","last_unstale":"2023-09-30T22:21:37.198428+0000","last_undegraded":"2023-09-30T22:21:37.198428+0000","last_fullsized":"2023-09-30T22:21:37.198428+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_clean_scrub_stamp":"2023-09-30T22:21:29.785382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199764+0000","last_change":"2023-09-30T22:21:36.811325+0000","last_active":"2023-09-30T22:21:37.199764+0000","last_peered":"2023-09-30T22:21:37.199764+0000","last_clean":"2023-09-30T22:21:37.199764+0000","last_became_active":"2023-09-30T22:21:36.811014+0000","last_became_peered":"2023-09-30T22:21:36.811014+0000","last_unstale":"2023-09-30T22:21:37.199764+0000","last_undegraded":"2023-09-30T22:21:37.199764+0000","last_fullsized":"2023-09-30T22:21:37.199764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226368+0000","last_change":"2023-09-30T22:21:37.220230+0000","last_active":"2023-09-30T22:21:37.226368+0000","last_peered":"2023-09-30T22:21:37.226368+0000","last_clean":"2023-09-30T22:21:37.226368+0000","last_became_active":"2023-09-30T22:21:37.215165+0000","last_became_peered":"2023-09-30T22:21:37.215165+0000","last_unstale":"2023-09-30T22:21:37.226368+0000","last_undegraded":"2023-09-30T22:21:37.226368+0000","last_fullsized":"2023-09-30T22:21:37.226368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195816+0000","last_change":"2023-09-30T22:21:31.798735+0000","last_active":"2023-09-30T22:21:37.195816+0000","last_peered":"2023-09-30T22:21:37.195816+0000","last_clean":"2023-09-30T22:21:37.195816+0000","last_became_active":"2023-09-30T22:21:31.798405+0000","last_became_peered":"2023-09-30T22:21:31.798405+0000","last_unstale":"2023-09-30T22:21:37.195816+0000","last_undegraded":"2023-09-30T22:21:37.195816+0000","last_fullsized":"2023-09-30T22:21:37.195816+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204992+0000","last_change":"2023-09-30T22:21:36.793235+0000","last_active":"2023-09-30T22:21:37.204992+0000","last_peered":"2023-09-30T22:21:37.204992+0000","last_clean":"2023-09-30T22:21:37.204992+0000","last_became_active":"2023-09-30T22:21:36.792921+0000","last_became_peered":"2023-09-30T22:21:36.792921+0000","last_unstale":"2023-09-30T22:21:37.204992+0000","last_undegraded":"2023-09-30T22:21:37.204992+0000","last_fullsized":"2023-09-30T22:21:37.204992+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233365+0000","last_change":"2023-09-30T22:21:37.224362+0000","last_active":"2023-09-30T22:21:37.233365+0000","last_peered":"2023-09-30T22:21:37.233365+0000","last_clean":"2023-09-30T22:21:37.233365+0000","last_became_active":"2023-09-30T22:21:37.219539+0000","last_became_peered":"2023-09-30T22:21:37.219539+0000","last_unstale":"2023-09-30T22:21:37.233365+0000","last_undegraded":"2023-09-30T22:21:37.233365+0000","last_fullsized":"2023-09-30T22:21:37.233365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197166+0000","last_change":"2023-09-30T22:21:31.800121+0000","last_active":"2023-09-30T22:21:37.197166+0000","last_peered":"2023-09-30T22:21:37.197166+0000","last_clean":"2023-09-30T22:21:37.197166+0000","last_became_active":"2023-09-30T22:21:31.799775+0000","last_became_peered":"2023-09-30T22:21:31.799775+0000","last_unstale":"2023-09-30T22:21:37.197166+0000","last_undegraded":"2023-09-30T22:21:37.197166+0000","last_fullsized":"2023-09-30T22:21:37.197166+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199098+0000","last_change":"2023-09-30T22:21:36.799552+0000","last_active":"2023-09-30T22:21:37.199098+0000","last_peered":"2023-09-30T22:21:37.199098+0000","last_clean":"2023-09-30T22:21:37.199098+0000","last_became_active":"2023-09-30T22:21:36.798979+0000","last_became_peered":"2023-09-30T22:21:36.798979+0000","last_unstale":"2023-09-30T22:21:37.199098+0000","last_undegraded":"2023-09-30T22:21:37.199098+0000","last_fullsized":"2023-09-30T22:21:37.199098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233846+0000","last_change":"2023-09-30T22:21:37.227699+0000","last_active":"2023-09-30T22:21:37.233846+0000","last_peered":"2023-09-30T22:21:37.233846+0000","last_clean":"2023-09-30T22:21:37.233846+0000","last_became_active":"2023-09-30T22:21:37.227403+0000","last_became_peered":"2023-09-30T22:21:37.227403+0000","last_unstale":"2023-09-30T22:21:37.233846+0000","last_undegraded":"2023-09-30T22:21:37.233846+0000","last_fullsized":"2023-09-30T22:21:37.233846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233095+0000","last_change":"2023-09-30T22:21:36.817294+0000","last_active":"2023-09-30T22:21:37.233095+0000","last_peered":"2023-09-30T22:21:37.233095+0000","last_clean":"2023-09-30T22:21:37.233095+0000","last_became_active":"2023-09-30T22:21:36.816805+0000","last_became_peered":"2023-09-30T22:21:36.816805+0000","last_unstale":"2023-09-30T22:21:37.233095+0000","last_undegraded":"2023-09-30T22:21:37.233095+0000","last_fullsized":"2023-09-30T22:21:37.233095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.200191+0000","last_change":"2023-09-30T22:21:31.794560+0000","last_active":"2023-09-30T22:21:37.200191+0000","last_peered":"2023-09-30T22:21:37.200191+0000","last_clean":"2023-09-30T22:21:37.200191+0000","last_became_active":"2023-09-30T22:21:31.794101+0000","last_became_peered":"2023-09-30T22:21:31.794101+0000","last_unstale":"2023-09-30T22:21:37.200191+0000","last_undegraded":"2023-09-30T22:21:37.200191+0000","last_fullsized":"2023-09-30T22:21:37.200191+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223871+0000","last_change":"2023-09-30T22:21:37.214706+0000","last_active":"2023-09-30T22:21:37.223871+0000","last_peered":"2023-09-30T22:21:37.223871+0000","last_clean":"2023-09-30T22:21:37.223871+0000","last_became_active":"2023-09-30T22:21:37.212393+0000","last_became_peered":"2023-09-30T22:21:37.212393+0000","last_unstale":"2023-09-30T22:21:37.223871+0000","last_undegraded":"2023-09-30T22:21:37.223871+0000","last_fullsized":"2023-09-30T22:21:37.223871+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.196857+0000","last_change":"2023-09-30T22:21:31.800079+0000","last_active":"2023-09-30T22:21:37.196857+0000","last_peered":"2023-09-30T22:21:37.196857+0000","last_clean":"2023-09-30T22:21:37.196857+0000","last_became_active":"2023-09-30T22:21:31.799718+0000","last_became_peered":"2023-09-30T22:21:31.799718+0000","last_unstale":"2023-09-30T22:21:37.196857+0000","last_undegraded":"2023-09-30T22:21:37.196857+0000","last_fullsized":"2023-09-30T22:21:37.196857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.200663+0000","last_change":"2023-09-30T22:21:36.803700+0000","last_active":"2023-09-30T22:21:37.200663+0000","last_peered":"2023-09-30T22:21:37.200663+0000","last_clean":"2023-09-30T22:21:37.200663+0000","last_became_active":"2023-09-30T22:21:36.803365+0000","last_became_peered":"2023-09-30T22:21:36.803365+0000","last_unstale":"2023-09-30T22:21:37.200663+0000","last_undegraded":"2023-09-30T22:21:37.200663+0000","last_fullsized":"2023-09-30T22:21:37.200663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232184+0000","last_change":"2023-09-30T22:21:37.226913+0000","last_active":"2023-09-30T22:21:37.232184+0000","last_peered":"2023-09-30T22:21:37.232184+0000","last_clean":"2023-09-30T22:21:37.232184+0000","last_became_active":"2023-09-30T22:21:37.225725+0000","last_became_peered":"2023-09-30T22:21:37.225725+0000","last_unstale":"2023-09-30T22:21:37.232184+0000","last_undegraded":"2023-09-30T22:21:37.232184+0000","last_fullsized":"2023-09-30T22:21:37.232184+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.230024+0000","last_change":"2023-09-30T22:21:36.808872+0000","last_active":"2023-09-30T22:21:37.230024+0000","last_peered":"2023-09-30T22:21:37.230024+0000","last_clean":"2023-09-30T22:21:37.230024+0000","last_became_active":"2023-09-30T22:21:36.806238+0000","last_became_peered":"2023-09-30T22:21:36.806238+0000","last_unstale":"2023-09-30T22:21:37.230024+0000","last_undegraded":"2023-09-30T22:21:37.230024+0000","last_fullsized":"2023-09-30T22:21:37.230024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223800+0000","last_change":"2023-09-30T22:21:37.208459+0000","last_active":"2023-09-30T22:21:37.223800+0000","last_peered":"2023-09-30T22:21:37.223800+0000","last_clean":"2023-09-30T22:21:37.223800+0000","last_became_active":"2023-09-30T22:21:37.208076+0000","last_became_peered":"2023-09-30T22:21:37.208076+0000","last_unstale":"2023-09-30T22:21:37.223800+0000","last_undegraded":"2023-09-30T22:21:37.223800+0000","last_fullsized":"2023-09-30T22:21:37.223800+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235981+0000","last_change":"2023-09-30T22:21:36.806969+0000","last_active":"2023-09-30T22:21:37.235981+0000","last_peered":"2023-09-30T22:21:37.235981+0000","last_clean":"2023-09-30T22:21:37.235981+0000","last_became_active":"2023-09-30T22:21:36.806649+0000","last_became_peered":"2023-09-30T22:21:36.806649+0000","last_unstale":"2023-09-30T22:21:37.235981+0000","last_undegraded":"2023-09-30T22:21:37.235981+0000","last_fullsized":"2023-09-30T22:21:37.235981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232315+0000","last_change":"2023-09-30T22:21:37.229486+0000","last_active":"2023-09-30T22:21:37.232315+0000","last_peered":"2023-09-30T22:21:37.232315+0000","last_clean":"2023-09-30T22:21:37.232315+0000","last_became_active":"2023-09-30T22:21:37.226308+0000","last_became_peered":"2023-09-30T22:21:37.226308+0000","last_unstale":"2023-09-30T22:21:37.232315+0000","last_undegraded":"2023-09-30T22:21:37.232315+0000","last_fullsized":"2023-09-30T22:21:37.232315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235069+0000","last_change":"2023-09-30T22:21:36.815782+0000","last_active":"2023-09-30T22:21:37.235069+0000","last_peered":"2023-09-30T22:21:37.235069+0000","last_clean":"2023-09-30T22:21:37.235069+0000","last_became_active":"2023-09-30T22:21:36.815431+0000","last_became_peered":"2023-09-30T22:21:36.815431+0000","last_unstale":"2023-09-30T22:21:37.235069+0000","last_undegraded":"2023-09-30T22:21:37.235069+0000","last_fullsized":"2023-09-30T22:21:37.235069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221951+0000","last_change":"2023-09-30T22:21:37.215011+0000","last_active":"2023-09-30T22:21:37.221951+0000","last_peered":"2023-09-30T22:21:37.221951+0000","last_clean":"2023-09-30T22:21:37.221951+0000","last_became_active":"2023-09-30T22:21:37.214739+0000","last_became_peered":"2023-09-30T22:21:37.214739+0000","last_unstale":"2023-09-30T22:21:37.221951+0000","last_undegraded":"2023-09-30T22:21:37.221951+0000","last_fullsized":"2023-09-30T22:21:37.221951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235070+0000","last_change":"2023-09-30T22:21:36.812958+0000","last_active":"2023-09-30T22:21:37.235070+0000","last_peered":"2023-09-30T22:21:37.235070+0000","last_clean":"2023-09-30T22:21:37.235070+0000","last_became_active":"2023-09-30T22:21:36.812662+0000","last_became_peered":"2023-09-30T22:21:36.812662+0000","last_unstale":"2023-09-30T22:21:37.235070+0000","last_undegraded":"2023-09-30T22:21:37.235070+0000","last_fullsized":"2023-09-30T22:21:37.235070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234543+0000","last_change":"2023-09-30T22:21:37.225020+0000","last_active":"2023-09-30T22:21:37.234543+0000","last_peered":"2023-09-30T22:21:37.234543+0000","last_clean":"2023-09-30T22:21:37.234543+0000","last_became_active":"2023-09-30T22:21:37.224728+0000","last_became_peered":"2023-09-30T22:21:37.224728+0000","last_unstale":"2023-09-30T22:21:37.234543+0000","last_undegraded":"2023-09-30T22:21:37.234543+0000","last_fullsized":"2023-09-30T22:21:37.234543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238512+0000","last_change":"2023-09-30T22:21:36.796108+0000","last_active":"2023-09-30T22:21:37.238512+0000","last_peered":"2023-09-30T22:21:37.238512+0000","last_clean":"2023-09-30T22:21:37.238512+0000","last_became_active":"2023-09-30T22:21:36.795665+0000","last_became_peered":"2023-09-30T22:21:36.795665+0000","last_unstale":"2023-09-30T22:21:37.238512+0000","last_undegraded":"2023-09-30T22:21:37.238512+0000","last_fullsized":"2023-09-30T22:21:37.238512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.217296+0000","last_change":"2023-09-30T22:21:37.207178+0000","last_active":"2023-09-30T22:21:37.217296+0000","last_peered":"2023-09-30T22:21:37.217296+0000","last_clean":"2023-09-30T22:21:37.217296+0000","last_became_active":"2023-09-30T22:21:37.206889+0000","last_became_peered":"2023-09-30T22:21:37.206889+0000","last_unstale":"2023-09-30T22:21:37.217296+0000","last_undegraded":"2023-09-30T22:21:37.217296+0000","last_fullsized":"2023-09-30T22:21:37.217296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.210508+0000","last_change":"2023-09-30T22:21:37.206551+0000","last_active":"2023-09-30T22:21:37.210508+0000","last_peered":"2023-09-30T22:21:37.210508+0000","last_clean":"2023-09-30T22:21:37.210508+0000","last_became_active":"2023-09-30T22:21:37.206176+0000","last_became_peered":"2023-09-30T22:21:37.206176+0000","last_unstale":"2023-09-30T22:21:37.210508+0000","last_undegraded":"2023-09-30T22:21:37.210508+0000","last_fullsized":"2023-09-30T22:21:37.210508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225040+0000","last_change":"2023-09-30T22:21:37.220615+0000","last_active":"2023-09-30T22:21:37.225040+0000","last_peered":"2023-09-30T22:21:37.225040+0000","last_clean":"2023-09-30T22:21:37.225040+0000","last_became_active":"2023-09-30T22:21:37.217961+0000","last_became_peered":"2023-09-30T22:21:37.217961+0000","last_unstale":"2023-09-30T22:21:37.225040+0000","last_undegraded":"2023-09-30T22:21:37.225040+0000","last_fullsized":"2023-09-30T22:21:37.225040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234039+0000","last_change":"2023-09-30T22:21:37.229799+0000","last_active":"2023-09-30T22:21:37.234039+0000","last_peered":"2023-09-30T22:21:37.234039+0000","last_clean":"2023-09-30T22:21:37.234039+0000","last_became_active":"2023-09-30T22:21:37.229494+0000","last_became_peered":"2023-09-30T22:21:37.229494+0000","last_unstale":"2023-09-30T22:21:37.234039+0000","last_undegraded":"2023-09-30T22:21:37.234039+0000","last_fullsized":"2023-09-30T22:21:37.234039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223296+0000","last_change":"2023-09-30T22:21:37.215694+0000","last_active":"2023-09-30T22:21:37.223296+0000","last_peered":"2023-09-30T22:21:37.223296+0000","last_clean":"2023-09-30T22:21:37.223296+0000","last_became_active":"2023-09-30T22:21:37.215371+0000","last_became_peered":"2023-09-30T22:21:37.215371+0000","last_unstale":"2023-09-30T22:21:37.223296+0000","last_undegraded":"2023-09-30T22:21:37.223296+0000","last_fullsized":"2023-09-30T22:21:37.223296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.210455+0000","last_change":"2023-09-30T22:21:37.206797+0000","last_active":"2023-09-30T22:21:37.210455+0000","last_peered":"2023-09-30T22:21:37.210455+0000","last_clean":"2023-09-30T22:21:37.210455+0000","last_became_active":"2023-09-30T22:21:37.206384+0000","last_became_peered":"2023-09-30T22:21:37.206384+0000","last_unstale":"2023-09-30T22:21:37.210455+0000","last_undegraded":"2023-09-30T22:21:37.210455+0000","last_fullsized":"2023-09-30T22:21:37.210455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]}]},{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:21:41.657 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-30T22:21:41.657 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:21:41.657 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:21:41.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:21:41.658 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-30T22:21:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 Processor -- start 2023-09-30T22:21:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 -- start start 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 0x7f819412ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 0x7f819412b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 0x7f8194096270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8194061780 con 0x7f81940613a0 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819412abd0 con 0x7f819412b2f0 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819907f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8194067530 con 0x7f8194095df0 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f8192d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 0x7f819412ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819359e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 0x7f8194096270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 0x7f819412b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 0x7f819412b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40776/0 (socket says 172.21.15.32:40776) 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f8192d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 0x7f819412ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54434/0 (socket says 172.21.15.32:54434) 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 -- 172.21.15.32:0/3122900944 learned_addr learned my addr 172.21.15.32:0/3122900944 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.822+0000 7f819359e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 0x7f8194096270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35462/0 (socket says 172.21.15.32:35462) 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 -- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 msgr2=0x7f8194096270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 --2- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 0x7f8194096270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 -- 172.21.15.32:0/3122900944 >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 msgr2=0x7f819412ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 --2- 172.21.15.32:0/3122900944 >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 0x7f819412ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 -- 172.21.15.32:0/3122900944 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81940683c0 con 0x7f819412b2f0 2023-09-30T22:21:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819359e700 1 --2- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 0x7f8194096270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f8192d9d700 1 --2- 172.21.15.32:0/3122900944 >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 0x7f819412ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f819259c700 1 --2- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 0x7f819412b6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8184006fd0 tx=0x7f81840090f0).ready entity=mon.1 client_cookie=2f082790c49b5675 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f8191d9b700 1 -- 172.21.15.32:0/3122900944 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400f510 con 0x7f819412b2f0 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f8191d9b700 1 -- 172.21.15.32:0/3122900944 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400f670 con 0x7f819412b2f0 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.823+0000 7f8191d9b700 1 -- 172.21.15.32:0/3122900944 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400f9f0 con 0x7f819412b2f0 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 -- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 msgr2=0x7f819412b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 --2- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 0x7f819412b6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8184006fd0 tx=0x7f81840090f0).stop 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 -- 172.21.15.32:0/3122900944 shutdown_connections 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 --2- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8194095df0 0x7f8194096270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 --2- 172.21.15.32:0/3122900944 >> v2:172.21.15.32:3300/0 conn(0x7f81940613a0 0x7f819412ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 --2- 172.21.15.32:0/3122900944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f819412b2f0 0x7f819412b6d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 -- 172.21.15.32:0/3122900944 >> 172.21.15.32:0/3122900944 conn(0x7f819409a220 msgr2=0x7f819409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 -- 172.21.15.32:0/3122900944 shutdown_connections 2023-09-30T22:21:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.824+0000 7f819907f700 1 -- 172.21.15.32:0/3122900944 wait complete. 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 Processor -- start 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 -- start start 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81940613a0 0x7f819411b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 0x7f81941171b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 0x7f81941176a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194128310 con 0x7f819412b2f0 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8194098bb0 con 0x7f8194095df0 2023-09-30T22:21:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.825+0000 7f819907f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8194067130 con 0x7f81940613a0 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 0x7f81941171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 0x7f81941176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 0x7f81941171b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40792/0 (socket says 172.21.15.32:40792) 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 0x7f81941176a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54438/0 (socket says 172.21.15.32:54438) 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 -- 172.21.15.32:0/4143464141 learned_addr learned my addr 172.21.15.32:0/4143464141 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f8192d9d700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81940613a0 0x7f819411b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 -- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81940613a0 msgr2=0x7f819411b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81940613a0 0x7f819411b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 -- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 msgr2=0x7f81941176a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 0x7f81941176a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 -- 172.21.15.32:0/4143464141 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c0092a0 con 0x7f8194095df0 2023-09-30T22:21:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f8192d9d700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81940613a0 0x7f819411b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819359e700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 0x7f81941176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819259c700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 0x7f81941171b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8194097790 tx=0x7f8184017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184006a40 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400f7e0 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c009a20 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818401d680 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.826+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8184009370 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.827+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f818401da30 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.827+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f818403d1c0 con 0x7f8194095df0 2023-09-30T22:21:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.827+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819411ab20 con 0x7f8194095df0 2023-09-30T22:21:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:41.832+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8184021070 con 0x7f8194095df0 2023-09-30T22:21:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.053+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f819411a900 con 0x7f8194095df0 2023-09-30T22:21:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.054+0000 7f818affd700 1 -- 172.21.15.32:0/4143464141 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+64129 (secure 0 0 0) 0x7f8184021020 con 0x7f8194095df0 2023-09-30T22:21:42.059 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.056+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 >> v1:172.21.15.32:6816/118188 conn(0x7f817402bb60 legacy=0x7f817402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.056+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 msgr2=0x7f81941171b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.056+0000 7f819907f700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 0x7f81941171b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8194097790 tx=0x7f8184017000).stop 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 shutdown_connections 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f819412b2f0 0x7f81941176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81940613a0 0x7f819411b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 --2- 172.21.15.32:0/4143464141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8194095df0 0x7f81941171b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 >> 172.21.15.32:0/4143464141 conn(0x7f819409a220 msgr2=0x7f8194129270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 shutdown_connections 2023-09-30T22:21:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.057+0000 7f819907f700 1 -- 172.21.15.32:0/4143464141 wait complete. 2023-09-30T22:21:42.070 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":4,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6816","nonce":118188}]},"active_addr":"172.21.15.32:6816/118188","active_change":"2023-09-30T22:21:25.527255+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":708160155}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2862279367}]}]} 2023-09-30T22:21:42.073 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:21:42.073 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:21:42.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.203+0000 7f9e0cbab700 1 Processor -- start 2023-09-30T22:21:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.203+0000 7f9e0cbab700 1 -- start start 2023-09-30T22:21:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0cbab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08096380 0x7f9e08094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0cbab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e08091b00 0x7f9e08127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0cbab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08094c20 0x7f9e08127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0cbab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e08096760 con 0x7f9e08091b00 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0cbab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08091ff0 con 0x7f9e08094c20 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0cbab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08092c80 con 0x7f9e08096380 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e05d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e08091b00 0x7f9e08127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e05d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e08091b00 0x7f9e08127600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54442/0 (socket says 172.21.15.32:54442) 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e05d9b700 1 -- 172.21.15.32:0/3516200793 learned_addr learned my addr 172.21.15.32:0/3516200793 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e06d9d700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08094c20 0x7f9e08127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.204+0000 7f9e0659c700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08096380 0x7f9e08094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e05d9b700 1 -- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08096380 msgr2=0x7f9e08094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e05d9b700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08096380 0x7f9e08094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e05d9b700 1 -- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08094c20 msgr2=0x7f9e08127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e05d9b700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08094c20 0x7f9e08127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e05d9b700 1 -- 172.21.15.32:0/3516200793 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08093900 con 0x7f9e08091b00 2023-09-30T22:21:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e05d9b700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08091b00 0x7f9e08127600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f9df0006fd0 tx=0x7f9df00090f0).ready entity=mon.0 client_cookie=a865bf2b3e3efd5a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.205+0000 7f9e0559a700 1 -- 172.21.15.32:0/3516200793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000f510 con 0x7f9e08091b00 2023-09-30T22:21:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0559a700 1 -- 172.21.15.32:0/3516200793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df000f670 con 0x7f9e08091b00 2023-09-30T22:21:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0559a700 1 -- 172.21.15.32:0/3516200793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000f9f0 con 0x7f9e08091b00 2023-09-30T22:21:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 -- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08091b00 msgr2=0x7f9e08127600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08091b00 0x7f9e08127600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f9df0006fd0 tx=0x7f9df00090f0).stop 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 -- 172.21.15.32:0/3516200793 shutdown_connections 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08094c20 0x7f9e08127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08096380 0x7f9e08094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/3516200793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08091b00 0x7f9e08127600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 -- 172.21.15.32:0/3516200793 >> 172.21.15.32:0/3516200793 conn(0x7f9e08097470 msgr2=0x7f9e08097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 -- 172.21.15.32:0/3516200793 shutdown_connections 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.206+0000 7f9e0cbab700 1 -- 172.21.15.32:0/3516200793 wait complete. 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.207+0000 7f9e0cbab700 1 Processor -- start 2023-09-30T22:21:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.207+0000 7f9e0cbab700 1 -- start start 2023-09-30T22:21:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0cbab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08091b00 0x7f9e08120a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0cbab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 0x7f9e08124990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0cbab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08096380 0x7f9e08124e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0cbab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08127710 con 0x7f9e08096380 2023-09-30T22:21:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0cbab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0806a520 con 0x7f9e08091b00 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0cbab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08122ff0 con 0x7f9e08094c20 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e05d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 0x7f9e08124990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08091b00 0x7f9e08120a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e05d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 0x7f9e08124990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35488/0 (socket says 172.21.15.32:35488) 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e05d9b700 1 -- 172.21.15.32:0/64279491 learned_addr learned my addr 172.21.15.32:0/64279491 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e06d9d700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08096380 0x7f9e08124e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0659c700 1 -- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 msgr2=0x7f9e08124990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0659c700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 0x7f9e08124990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0659c700 1 -- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08096380 msgr2=0x7f9e08124e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0659c700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08096380 0x7f9e08124e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e0659c700 1 -- 172.21.15.32:0/64279491 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0009370 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.208+0000 7f9e05d9b700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 0x7f9e08124990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.209+0000 7f9e0659c700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08091b00 0x7f9e08120a70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9df800e930 tx=0x7f9df8009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.209+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8009450 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.209+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8012600 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.209+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e08121710 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.209+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8012980 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.209+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e08124540 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.210+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9df801b400 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.210+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f9df803b450 con 0x7f9e08091b00 2023-09-30T22:21:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.210+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0811efd0 con 0x7f9e08091b00 2023-09-30T22:21:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.215+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9df8019310 con 0x7f9e08091b00 2023-09-30T22:21:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.393+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e0805a260 con 0x7f9e08091b00 2023-09-30T22:21:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.394+0000 7f9dfe7fc700 1 -- 172.21.15.32:0/64279491 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f9df8023350 con 0x7f9e08091b00 2023-09-30T22:21:42.395 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:42.395 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:38.189468+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 >> v1:172.21.15.32:6816/118188 conn(0x7f9de802bb60 legacy=0x7f9de802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08091b00 msgr2=0x7f9e08120a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08091b00 0x7f9e08120a70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9df800e930 tx=0x7f9df8009060).stop 2023-09-30T22:21:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 shutdown_connections 2023-09-30T22:21:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e08096380 0x7f9e08124e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e08091b00 0x7f9e08120a70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 --2- 172.21.15.32:0/64279491 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e08094c20 0x7f9e08124990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 >> 172.21.15.32:0/64279491 conn(0x7f9e08097470 msgr2=0x7f9e0811e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 shutdown_connections 2023-09-30T22:21:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.396+0000 7f9e0cbab700 1 -- 172.21.15.32:0/64279491 wait complete. 2023-09-30T22:21:42.410 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:21:42.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.539+0000 7f11916c2700 1 Processor -- start 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.539+0000 7f11916c2700 1 -- start start 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f11916c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 0x7f118c096c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f11916c2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f118c067940 0x7f118c0970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f11916c2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 0x7f118c12f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f11916c2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c068df0 con 0x7f118c067940 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f11916c2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c06a4c0 con 0x7f118c0975e0 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f11916c2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f118c098260 con 0x7f118c096820 2023-09-30T22:21:42.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 0x7f118c096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 0x7f118c12f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 0x7f118c096c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35498/0 (socket says 172.21.15.32:35498) 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 0x7f118c12f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40822/0 (socket says 172.21.15.32:40822) 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118affd700 1 -- 172.21.15.32:0/1740481796 learned_addr learned my addr 172.21.15.32:0/1740481796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118a7fc700 1 --2- 172.21.15.32:0/1740481796 >> v2:172.21.15.32:3300/0 conn(0x7f118c067940 0x7f118c0970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118a7fc700 1 -- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 msgr2=0x7f118c096c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118a7fc700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 0x7f118c096c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118a7fc700 1 -- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 msgr2=0x7f118c12f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118a7fc700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 0x7f118c12f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118a7fc700 1 -- 172.21.15.32:0/1740481796 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c095e60 con 0x7f118c067940 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118b7fe700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 0x7f118c12f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.540+0000 7f118affd700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 0x7f118c096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f118a7fc700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c067940 0x7f118c0970f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1180009760 tx=0x7f1180009060).ready entity=mon.0 client_cookie=2c10926837b63a8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f1189ffb700 1 -- 172.21.15.32:0/1740481796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000d4a0 con 0x7f118c067940 2023-09-30T22:21:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f1189ffb700 1 -- 172.21.15.32:0/1740481796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000d600 con 0x7f118c067940 2023-09-30T22:21:42.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f1189ffb700 1 -- 172.21.15.32:0/1740481796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000d980 con 0x7f118c067940 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 -- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c067940 msgr2=0x7f118c0970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c067940 0x7f118c0970f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1180009760 tx=0x7f1180009060).stop 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 -- 172.21.15.32:0/1740481796 shutdown_connections 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c0975e0 0x7f118c12f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c096820 0x7f118c096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 --2- 172.21.15.32:0/1740481796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c067940 0x7f118c0970f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.541+0000 7f11916c2700 1 -- 172.21.15.32:0/1740481796 >> 172.21.15.32:0/1740481796 conn(0x7f118c09ba50 msgr2=0x7f118c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.542+0000 7f11916c2700 1 -- 172.21.15.32:0/1740481796 shutdown_connections 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.542+0000 7f11916c2700 1 -- 172.21.15.32:0/1740481796 wait complete. 2023-09-30T22:21:42.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 Processor -- start 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 -- start start 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 0x7f118c118980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c096820 0x7f118c118e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c0975e0 0x7f118c119360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c091f60 con 0x7f118c0975e0 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c067e30 con 0x7f118c096820 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f11916c2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f118c11a8c0 con 0x7f118c067940 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f118affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 0x7f118c118980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c0975e0 0x7f118c119360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f118affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 0x7f118c118980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35510/0 (socket says 172.21.15.32:35510) 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f118affd700 1 -- 172.21.15.32:0/3618760932 learned_addr learned my addr 172.21.15.32:0/3618760932 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.543+0000 7f118a7fc700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c096820 0x7f118c118e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118b7fe700 1 -- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 msgr2=0x7f118c118980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118b7fe700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 0x7f118c118980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118b7fe700 1 -- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c096820 msgr2=0x7f118c118e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118b7fe700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c096820 0x7f118c118e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118b7fe700 1 -- 172.21.15.32:0/3618760932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009370 con 0x7f118c0975e0 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118a7fc700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c096820 0x7f118c118e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118affd700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 0x7f118c118980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f118b7fe700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c0975e0 0x7f118c119360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1184009f10 tx=0x7f118400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118400cb70 con 0x7f118c0975e0 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118400f040 con 0x7f118c0975e0 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1184009a20 con 0x7f118c0975e0 2023-09-30T22:21:42.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184013400 con 0x7f118c0975e0 2023-09-30T22:21:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.544+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11800092a0 con 0x7f118c0975e0 2023-09-30T22:21:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.545+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f11840137f0 con 0x7f118c0975e0 2023-09-30T22:21:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.545+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f118403b840 con 0x7f118c0975e0 2023-09-30T22:21:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.545+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c11c5f0 con 0x7f118c0975e0 2023-09-30T22:21:42.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.550+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1184013aa0 con 0x7f118c0975e0 2023-09-30T22:21:42.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.726+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f118c05a260 con 0x7f118c0975e0 2023-09-30T22:21:42.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.727+0000 7f1172ffd700 1 -- 172.21.15.32:0/3618760932 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f118401b7e0 con 0x7f118c0975e0 2023-09-30T22:21:42.728 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:42.728 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:38.189468+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 >> v1:172.21.15.32:6816/118188 conn(0x7f117802bb60 legacy=0x7f117802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c0975e0 msgr2=0x7f118c119360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c0975e0 0x7f118c119360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1184009f10 tx=0x7f118400b040).stop 2023-09-30T22:21:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 shutdown_connections 2023-09-30T22:21:42.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f118c0975e0 0x7f118c119360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f118c067940 0x7f118c118980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 --2- 172.21.15.32:0/3618760932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f118c096820 0x7f118c118e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 >> 172.21.15.32:0/3618760932 conn(0x7f118c09ba50 msgr2=0x7f118c099970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 shutdown_connections 2023-09-30T22:21:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.729+0000 7f11916c2700 1 -- 172.21.15.32:0/3618760932 wait complete. 2023-09-30T22:21:42.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:21:42.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:21:42.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:21:42.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:21:42.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 Processor -- start 2023-09-30T22:21:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 -- start start 2023-09-30T22:21:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f6eb43ed700 1 Processor -- start 2023-09-30T22:21:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc12fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97fc127fa0 0x7f97fc132150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc091b60 con 0x7f97fc059070 2023-09-30T22:21:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc059450 con 0x7f97fc127fa0 2023-09-30T22:21:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f9803112700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0968c0 con 0x7f97fc127390 2023-09-30T22:21:42.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f6eb43ed700 1 -- start start 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc12fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.885+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc12fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35512/0 (socket says 172.21.15.32:35512) 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fbfff700 1 -- 172.21.15.32:0/3436644986 learned_addr learned my addr 172.21.15.32:0/3436644986 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fbfff700 1 -- 172.21.15.32:0/3436644986 >> v2:172.21.15.32:3300/0 conn(0x7f97fc127fa0 msgr2=0x7f97fc132150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fbfff700 1 --2- 172.21.15.32:0/3436644986 >> v2:172.21.15.32:3300/0 conn(0x7f97fc127fa0 0x7f97fc132150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fbfff700 1 -- 172.21.15.32:0/3436644986 >> v1:172.21.15.32:6789/0 conn(0x7f97fc059070 legacy=0x7f97fc127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fbfff700 1 -- 172.21.15.32:0/3436644986 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc092a10 con 0x7f97fc127390 2023-09-30T22:21:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fbfff700 1 --2- 172.21.15.32:0/3436644986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc12fc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f97e8009830 tx=0x7f97e80090f0).ready entity=mon.2 client_cookie=e42e6e00b7b9c9d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f6eb43ed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6eac123e70 0x7f6eac067ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f6eb43ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f6eb43ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac068880 con 0x7f6eac123a90 2023-09-30T22:21:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f6eb43ed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eac124250 con 0x7f6eac123e70 2023-09-30T22:21:42.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f6eb43ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac068c90 con 0x7f6eac092340 2023-09-30T22:21:42.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fb7fe700 1 -- 172.21.15.32:0/3436644986 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e800b600 con 0x7f97fc127390 2023-09-30T22:21:42.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fb7fe700 1 -- 172.21.15.32:0/3436644986 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e800b760 con 0x7f97fc127390 2023-09-30T22:21:42.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.886+0000 7f97fb7fe700 1 -- 172.21.15.32:0/3436644986 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e800bb00 con 0x7f97fc127390 2023-09-30T22:21:42.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 -- 172.21.15.32:0/3436644986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 msgr2=0x7f97fc12fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 --2- 172.21.15.32:0/3436644986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc12fc50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f97e8009830 tx=0x7f97e80090f0).stop 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 -- 172.21.15.32:0/3436644986 shutdown_connections 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 --2- 172.21.15.32:0/3436644986 >> v2:172.21.15.32:3300/0 conn(0x7f97fc127fa0 0x7f97fc132150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 --2- 172.21.15.32:0/3436644986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc12fc50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 -- 172.21.15.32:0/3436644986 >> 172.21.15.32:0/3436644986 conn(0x7f97fc097470 msgr2=0x7f97fc097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 -- 172.21.15.32:0/3436644986 shutdown_connections 2023-09-30T22:21:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f9803112700 1 -- 172.21.15.32:0/3436644986 wait complete. 2023-09-30T22:21:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f6eb298a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f6eb298a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac127680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40842/0 (socket says 172.21.15.32:40842) 2023-09-30T22:21:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.887+0000 7f6eb298a700 1 -- 172.21.15.32:0/3989483704 learned_addr learned my addr 172.21.15.32:0/3989483704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1988700 1 --2- 172.21.15.32:0/3989483704 >> v2:172.21.15.32:3300/0 conn(0x7f6eac123e70 0x7f6eac067ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f9803112700 1 Processor -- start 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1187700 1 -- 172.21.15.32:0/3989483704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782141382 0 0) 0x7f6eac068880 con 0x7f6eac123a90 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1187700 1 -- 172.21.15.32:0/3989483704 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e980033f0 con 0x7f6eac123a90 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1988700 1 -- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 msgr2=0x7f6eac127680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1988700 1 --2- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac127680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1988700 1 -- 172.21.15.32:0/3989483704 >> v1:172.21.15.32:6789/0 conn(0x7f6eac123a90 legacy=0x7f6eac126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1988700 1 -- 172.21.15.32:0/3989483704 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac124430 con 0x7f6eac123e70 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.888+0000 7f6eb1988700 1 --2- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123e70 0x7f6eac067ce0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6e9c009790 tx=0x7f6e9c009060).ready entity=mon.0 client_cookie=f9e4ebdb60e1526a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 -- start start 2023-09-30T22:21:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f6eb1187700 1 -- 172.21.15.32:0/3989483704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00d4a0 con 0x7f6eac123e70 2023-09-30T22:21:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f6eb1187700 1 -- 172.21.15.32:0/3989483704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c00d600 con 0x7f6eac123e70 2023-09-30T22:21:42.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f6eb1187700 1 -- 172.21.15.32:0/3989483704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c00d980 con 0x7f6eac123e70 2023-09-30T22:21:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97fc059070 0x7f97fc12ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc130430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f97fc127fa0 0x7f97fc130920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc130e10 con 0x7f97fc059070 2023-09-30T22:21:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc131380 con 0x7f97fc127fa0 2023-09-30T22:21:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9803112700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc131b50 con 0x7f97fc127390 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9800eae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97fc059070 0x7f97fc12ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc130430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.889+0000 7f9800eae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97fc059070 0x7f97fc12ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54502/0 (socket says 172.21.15.32:54502) 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 -- 172.21.15.32:0/1250685080 learned_addr learned my addr 172.21.15.32:0/1250685080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 -- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 msgr2=0x7f97fc130430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 --2- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc130430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 -- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f97fc127fa0 msgr2=0x7f97fc130920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 --2- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f97fc127fa0 0x7f97fc130920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 -- 172.21.15.32:0/1250685080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009370 con 0x7f97fc059070 2023-09-30T22:21:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f97fbfff700 1 --2- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f97fc127390 0x7f97fc130430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 -- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123e70 msgr2=0x7f6eac067ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 --2- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123e70 0x7f6eac067ce0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6e9c009790 tx=0x7f6e9c009060).stop 2023-09-30T22:21:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9800eae700 1 --2- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97fc059070 0x7f97fc12ff40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f97f000b490 tx=0x7f97f0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 -- 172.21.15.32:0/3989483704 shutdown_connections 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 --2- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac127680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 --2- 172.21.15.32:0/3989483704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123e70 0x7f6eac067ce0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 -- 172.21.15.32:0/3989483704 >> 172.21.15.32:0/3989483704 conn(0x7f6eac09a220 msgr2=0x7f6eac09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000be90 con 0x7f97fc059070 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0012410 con 0x7f97fc059070 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00127d0 con 0x7f97fc059070 2023-09-30T22:21:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 -- 172.21.15.32:0/3989483704 shutdown_connections 2023-09-30T22:21:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc096ea0 con 0x7f97fc059070 2023-09-30T22:21:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f6eb43ed700 1 -- 172.21.15.32:0/3989483704 wait complete. 2023-09-30T22:21:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.890+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc113290 con 0x7f97fc059070 2023-09-30T22:21:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.891+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f97f0012bc0 con 0x7f97fc059070 2023-09-30T22:21:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.891+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f97f003ac50 con 0x7f97fc059070 2023-09-30T22:21:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 Processor -- start 2023-09-30T22:21:42.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 --> v1:172.21.15.32:6804/118763 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97fc12ca30 con 0x7f97fc1121f0 2023-09-30T22:21:42.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 -- start start 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123a90 0x7f6eac111e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac123e70 0x7f6eac112300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0697c0 con 0x7f6eac123a90 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.892+0000 7f6eb43ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac127790 con 0x7f6eac092340 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb43ed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac11e620 con 0x7f6eac123e70 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb1988700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123a90 0x7f6eac111e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac111920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40868/0 (socket says 172.21.15.32:40868) 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb1988700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123a90 0x7f6eac111e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54516/0 (socket says 172.21.15.32:54516) 2023-09-30T22:21:42.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 -- 172.21.15.32:0/2617084074 learned_addr learned my addr 172.21.15.32:0/2617084074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb298a700 1 --2- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac123e70 0x7f6eac112300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 -- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac123e70 msgr2=0x7f6eac112300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 --2- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac123e70 0x7f6eac112300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 -- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123a90 msgr2=0x7f6eac111e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 --2- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac123a90 0x7f6eac111e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 -- 172.21.15.32:0/2617084074 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0092a0 con 0x7f6eac092340 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.893+0000 7f6eb2189700 1 --2- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac111920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f6ea800c4f0 tx=0x7f6ea8007c90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.894+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800ce20 con 0x7f6eac092340 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.894+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea800f470 con 0x7f6eac092340 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.894+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac114eb0 con 0x7f6eac092340 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.894+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac115950 con 0x7f6eac092340 2023-09-30T22:21:42.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.894+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800f830 con 0x7f6eac092340 2023-09-30T22:21:42.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.894+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e94000fc0 con 0x7f6eac092340 2023-09-30T22:21:42.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.895+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== osd.0 v1:172.21.15.32:6804/118763 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f97fc12ca30 con 0x7f97fc1121f0 2023-09-30T22:21:42.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.895+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6ea800fbe0 con 0x7f6eac092340 2023-09-30T22:21:42.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.895+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6ea803abe0 con 0x7f6eac092340 2023-09-30T22:21:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.896+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 --> v1:172.21.15.32:6808/118793 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e900384e0 con 0x7f6e900347f0 2023-09-30T22:21:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.896+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea803afe0 con 0x7f6eac092340 2023-09-30T22:21:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.897+0000 7f438785e700 1 Processor -- start 2023-09-30T22:21:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.897+0000 7f242ead9700 1 Processor -- start 2023-09-30T22:21:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.897+0000 7f242ead9700 1 -- start start 2023-09-30T22:21:42.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.897+0000 7f242ead9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2428096380 0x7f2428094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242ead9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242ead9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2428096760 con 0x7f2428091b00 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242ead9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2428091ff0 con 0x7f2428096380 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242ead9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428092c80 con 0x7f2428094c20 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f2427fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2428091b00 0x7f2428127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54778/0 (socket says 172.21.15.32:54778) 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f2427fff700 1 -- 172.21.15.32:0/603037158 learned_addr learned my addr 172.21.15.32:0/603037158 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:42.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== osd.2 v1:172.21.15.32:6808/118793 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f6e900384e0 con 0x7f6e900347f0 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242d076700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242c875700 1 --2- 172.21.15.32:0/603037158 >> v2:172.21.15.32:3300/0 conn(0x7f2428096380 0x7f2428094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.897+0000 7f438785e700 1 -- start start 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f438785e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800654a0 0x7f4380098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f438785e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4380098b30 0x7f4380098f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f438785e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f438012f470 0x7f4380131860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f438785e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4380095880 con 0x7f4380098b30 2023-09-30T22:21:42.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f438785e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4380065ed0 con 0x7f43800654a0 2023-09-30T22:21:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f438785e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4380096120 con 0x7f438012f470 2023-09-30T22:21:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f24277fe700 1 -- 172.21.15.32:0/603037158 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102299368 0 0) 0x7f2428096760 con 0x7f2428091b00 2023-09-30T22:21:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f24277fe700 1 -- 172.21.15.32:0/603037158 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24180033f0 con 0x7f2428091b00 2023-09-30T22:21:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f4384df9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4380098b30 0x7f4380098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f4384df9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4380098b30 0x7f4380098f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54534/0 (socket says 172.21.15.32:54534) 2023-09-30T22:21:42.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f4384df9700 1 -- 172.21.15.32:0/3308381486 learned_addr learned my addr 172.21.15.32:0/3308381486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242c875700 1 -- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 msgr2=0x7f2428127d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242c875700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428127d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242c875700 1 -- 172.21.15.32:0/603037158 >> v1:172.21.15.32:6789/0 conn(0x7f2428091b00 legacy=0x7f2428127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242c875700 1 -- 172.21.15.32:0/603037158 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428093900 con 0x7f2428096380 2023-09-30T22:21:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.898+0000 7f242d076700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:42.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f242c875700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 0x7f2428094730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f241c009760 tx=0x7f241c009060).ready entity=mon.0 client_cookie=bef307648f0ba341 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4384df9700 1 -- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f438012f470 msgr2=0x7f4380131860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4384df9700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f438012f470 0x7f4380131860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4384df9700 1 -- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800654a0 msgr2=0x7f4380098640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:21:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4384df9700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800654a0 0x7f4380098640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4384df9700 1 -- 172.21.15.32:0/3308381486 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4380099460 con 0x7f4380098b30 2023-09-30T22:21:42.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f24277fe700 1 -- 172.21.15.32:0/603037158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00d4a0 con 0x7f2428096380 2023-09-30T22:21:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f24277fe700 1 -- 172.21.15.32:0/603037158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00d600 con 0x7f2428096380 2023-09-30T22:21:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f24277fe700 1 -- 172.21.15.32:0/603037158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00d980 con 0x7f2428096380 2023-09-30T22:21:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4384df9700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4380098b30 0x7f4380098f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4370009830 tx=0x7f43700090f0).ready entity=mon.0 client_cookie=57d447ba84128c19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4377fff700 1 -- 172.21.15.32:0/3308381486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000b600 con 0x7f4380098b30 2023-09-30T22:21:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4377fff700 1 -- 172.21.15.32:0/3308381486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437000b760 con 0x7f4380098b30 2023-09-30T22:21:42.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.899+0000 7f4377fff700 1 -- 172.21.15.32:0/3308381486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000bb00 con 0x7f4380098b30 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 -- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 msgr2=0x7f2428094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 0x7f2428094730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f241c009760 tx=0x7f241c009060).stop 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 -- 172.21.15.32:0/603037158 shutdown_connections 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 --2- 172.21.15.32:0/603037158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 0x7f2428094730 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 -- 172.21.15.32:0/603037158 >> 172.21.15.32:0/603037158 conn(0x7f2428097470 msgr2=0x7f2428097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 -- 172.21.15.32:0/603037158 shutdown_connections 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f242ead9700 1 -- 172.21.15.32:0/603037158 wait complete. 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f438785e700 1 -- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4380098b30 msgr2=0x7f4380098f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.900+0000 7f438785e700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4380098b30 0x7f4380098f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4370009830 tx=0x7f43700090f0).stop 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.901+0000 7f438785e700 1 -- 172.21.15.32:0/3308381486 shutdown_connections 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.901+0000 7f438785e700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800654a0 0x7f4380098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.901+0000 7f438785e700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f438012f470 0x7f4380131860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.901+0000 7f438785e700 1 --2- 172.21.15.32:0/3308381486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4380098b30 0x7f4380098f70 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0x7f4370009830 tx=0x7f43700090f0).stop 2023-09-30T22:21:42.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.901+0000 7f438785e700 1 -- 172.21.15.32:0/3308381486 >> 172.21.15.32:0/3308381486 conn(0x7f438009a740 msgr2=0x7f438009ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.901+0000 7f242ead9700 1 Processor -- start 2023-09-30T22:21:42.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 -- start start 2023-09-30T22:21:42.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f438785e700 1 -- 172.21.15.32:0/3308381486 shutdown_connections 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2428091b00 0x7f24281169e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 0x7f2428112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428124b20 con 0x7f2428096380 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428127e20 con 0x7f2428094c20 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242ead9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f242806a4e0 con 0x7f2428091b00 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242c875700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2428091b00 0x7f24281169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f2427fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242c875700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2428091b00 0x7f24281169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35550/0 (socket says 172.21.15.32:35550) 2023-09-30T22:21:42.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242c875700 1 -- 172.21.15.32:0/1217827678 learned_addr learned my addr 172.21.15.32:0/1217827678 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f438785e700 1 -- 172.21.15.32:0/3308381486 wait complete. 2023-09-30T22:21:42.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.902+0000 7f242d076700 1 --2- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 0x7f2428112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f242c875700 1 -- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 msgr2=0x7f2428112ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f242c875700 1 --2- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2428094c20 0x7f2428112ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f242c875700 1 -- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 msgr2=0x7f2428112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f242c875700 1 --2- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2428096380 0x7f2428112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f242c875700 1 -- 172.21.15.32:0/1217827678 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420009a20 con 0x7f2428091b00 2023-09-30T22:21:42.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f242c875700 1 --2- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2428091b00 0x7f24281169e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f241c009d50 tx=0x7f241c016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.903+0000 7f438785e700 1 Processor -- start 2023-09-30T22:21:42.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.904+0000 7f438785e700 1 -- start start 2023-09-30T22:21:42.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f438785e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f43800654a0 0x7f4380122960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f438785e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f438012f470 0x7f438011f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f438785e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438011ee00 0x7f438011f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f438785e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4380122180 con 0x7f438011ee00 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f438785e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4380121440 con 0x7f438012f470 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f438785e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4380121ec0 con 0x7f43800654a0 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f438012f470 0x7f438011f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f438012f470 0x7f438011f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40892/0 (socket says 172.21.15.32:40892) 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 -- 172.21.15.32:0/1310563400 learned_addr learned my addr 172.21.15.32:0/1310563400 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 -- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f43800654a0 msgr2=0x7f4380122960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 --2- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f43800654a0 0x7f4380122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 -- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438011ee00 msgr2=0x7f438011f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4385dfb700 1 --2- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438011ee00 0x7f438011f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 --2- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438011ee00 0x7f438011f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 -- 172.21.15.32:0/1310563400 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009370 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4385dfb700 1 --2- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f438011ee00 0x7f438011f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.905+0000 7f4384df9700 1 --2- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f438012f470 0x7f438011f2d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4370009130 tx=0x7f4370017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.906+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011ca0 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.906+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370011e00 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.906+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437001a640 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.906+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4380120650 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.906+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438011fe70 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.907+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f437001a9f0 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.907+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4380122370 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.907+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4370040070 con 0x7f438012f470 2023-09-30T22:21:42.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.907+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 --> v1:172.21.15.32:6809/118794 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4364038460 con 0x7f4364034770 2023-09-30T22:21:42.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.907+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f437003f030 con 0x7f438012f470 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.908+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00dd30 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.908+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00de90 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.908+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00d980 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.908+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0092a0 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.908+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428113d00 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.909+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2428116000 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.909+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f241c01d640 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.909+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f241c03d2c0 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.909+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 --> v1:172.21.15.32:6800/118762 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f240c038630 con 0x7f240c034940 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.909+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241c03d5f0 con 0x7f2428091b00 2023-09-30T22:21:42.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.914+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== osd.1 v1:172.21.15.32:6800/118762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f240c038630 con 0x7f240c034940 2023-09-30T22:21:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.914+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== osd.3 v1:172.21.15.32:6809/118794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f4364038460 con 0x7f4364034770 2023-09-30T22:21:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.926+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 --> v1:172.21.15.32:6804/118763 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97fc112830 con 0x7f97fc1121f0 2023-09-30T22:21:42.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.926+0000 7f97f8ff9700 1 -- 172.21.15.32:0/1250685080 <== osd.0 v1:172.21.15.32:6804/118763 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f97f4002f30 con 0x7f97fc1121f0 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 >> v1:172.21.15.32:6804/118763 conn(0x7f97fc1121f0 legacy=0x7f97fc05e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 >> v1:172.21.15.32:6816/118188 conn(0x7f97e002bbe0 legacy=0x7f97e002e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97fc059070 msgr2=0x7f97fc12ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f9803112700 1 --2- 172.21.15.32:0/1250685080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97fc059070 0x7f97fc12ff40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f97f000b490 tx=0x7f97f0009060).stop 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f98016af700 1 -- 172.21.15.32:0/1250685080 reap_dead start 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 shutdown_connections 2023-09-30T22:21:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.927+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 >> 172.21.15.32:0/1250685080 conn(0x7f97fc097470 msgr2=0x7f97fc0952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.928+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 shutdown_connections 2023-09-30T22:21:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.928+0000 7f9803112700 1 -- 172.21.15.32:0/1250685080 wait complete. 2023-09-30T22:21:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.929+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 --> v1:172.21.15.32:6808/118793 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e94002b10 con 0x7f6e900347f0 2023-09-30T22:21:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.929+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 --> v1:172.21.15.32:6800/118762 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f242805a260 con 0x7f240c034940 2023-09-30T22:21:42.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.929+0000 7f6ea27fc700 1 -- 172.21.15.32:0/2617084074 <== osd.2 v1:172.21.15.32:6808/118793 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f6e94002b10 con 0x7f6e900347f0 2023-09-30T22:21:42.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.929+0000 7f2424ff9700 1 -- 172.21.15.32:0/1217827678 <== osd.1 v1:172.21.15.32:6800/118762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f242805a260 con 0x7f240c034940 2023-09-30T22:21:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.929+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 >> v1:172.21.15.32:6808/118793 conn(0x7f6e900347f0 legacy=0x7f6e90036c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 --> v1:172.21.15.32:6809/118794 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f438013edd0 con 0x7f4364034770 2023-09-30T22:21:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 >> v1:172.21.15.32:6816/118188 conn(0x7f6e9002bb60 legacy=0x7f6e9002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 msgr2=0x7f6eac111920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f6eb43ed700 1 --2- 172.21.15.32:0/2617084074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac092340 0x7f6eac111920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f6ea800c4f0 tx=0x7f6ea8007c90).stop 2023-09-30T22:21:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 >> v1:172.21.15.32:6800/118762 conn(0x7f240c034940 legacy=0x7f240c036dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f6eb298a700 1 -- 172.21.15.32:0/2617084074 reap_dead start 2023-09-30T22:21:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 >> v1:172.21.15.32:6816/118188 conn(0x7f240c02bbb0 legacy=0x7f240c02e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2428091b00 msgr2=0x7f24281169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 --2- 172.21.15.32:0/1217827678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2428091b00 0x7f24281169e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f241c009d50 tx=0x7f241c016000).stop 2023-09-30T22:21:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242d076700 1 -- 172.21.15.32:0/1217827678 reap_dead start 2023-09-30T22:21:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 shutdown_connections 2023-09-30T22:21:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 >> 172.21.15.32:0/1217827678 conn(0x7f2428097470 msgr2=0x7f2428098f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 shutdown_connections 2023-09-30T22:21:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f242ead9700 1 -- 172.21.15.32:0/1217827678 wait complete. 2023-09-30T22:21:42.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.930+0000 7f43757fa700 1 -- 172.21.15.32:0/1310563400 <== osd.3 v1:172.21.15.32:6809/118794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f438013edd0 con 0x7f4364034770 2023-09-30T22:21:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 shutdown_connections 2023-09-30T22:21:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 >> 172.21.15.32:0/2617084074 conn(0x7f6eac09a220 msgr2=0x7f6eac099000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 shutdown_connections 2023-09-30T22:21:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f6eb43ed700 1 -- 172.21.15.32:0/2617084074 wait complete. 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 >> v1:172.21.15.32:6809/118794 conn(0x7f4364034770 legacy=0x7f4364036bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 >> v1:172.21.15.32:6816/118188 conn(0x7f436402bb60 legacy=0x7f436402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f438012f470 msgr2=0x7f438011f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 --2- 172.21.15.32:0/1310563400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f438012f470 0x7f438011f2d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4370009130 tx=0x7f4370017000).stop 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f4385dfb700 1 -- 172.21.15.32:0/1310563400 reap_dead start 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 shutdown_connections 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 >> 172.21.15.32:0/1310563400 conn(0x7f438009a740 msgr2=0x7f4380066720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 shutdown_connections 2023-09-30T22:21:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:42.931+0000 7f438785e700 1 -- 172.21.15.32:0/1310563400 wait complete. 2023-09-30T22:21:42.948 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:42.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:42.949 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:42.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:42.952 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:42.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:42.954 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:42.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.093+0000 7fdbb96b9700 1 Processor -- start 2023-09-30T22:21:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.093+0000 7fdbb96b9700 1 -- start start 2023-09-30T22:21:43.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb96b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 0x7fdbb4098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb96b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb4098b30 0x7fdbb4098f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb96b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb4131860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb96b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb4095880 con 0x7fdbb40654a0 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb96b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4065ed0 con 0x7fdbb412f470 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb96b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4096120 con 0x7fdbb4098b30 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 0x7fdbb4098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 0x7fdbb4098640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54550/0 (socket says 172.21.15.32:54550) 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb2ffd700 1 -- 172.21.15.32:0/713839582 learned_addr learned my addr 172.21.15.32:0/713839582 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb27fc700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb4098b30 0x7fdbb4098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.094+0000 7fdbb37fe700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb4131860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb27fc700 1 -- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 msgr2=0x7fdbb4131860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb27fc700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb4131860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb27fc700 1 -- 172.21.15.32:0/713839582 >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 msgr2=0x7fdbb4098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb27fc700 1 --2- 172.21.15.32:0/713839582 >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 0x7fdbb4098640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb27fc700 1 -- 172.21.15.32:0/713839582 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4099460 con 0x7fdbb4098b30 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb2ffd700 1 --2- 172.21.15.32:0/713839582 >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 0x7fdbb4098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:43.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb37fe700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb4131860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb27fc700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb4098b30 0x7fdbb4098f70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdba8009760 tx=0x7fdba8009060).ready entity=mon.2 client_cookie=5d422bc9d18a6ca6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb1ffb700 1 -- 172.21.15.32:0/713839582 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800d4a0 con 0x7fdbb4098b30 2023-09-30T22:21:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb1ffb700 1 -- 172.21.15.32:0/713839582 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800d600 con 0x7fdbb4098b30 2023-09-30T22:21:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.095+0000 7fdbb1ffb700 1 -- 172.21.15.32:0/713839582 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800d980 con 0x7fdbb4098b30 2023-09-30T22:21:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 -- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb4098b30 msgr2=0x7fdbb4098f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb4098b30 0x7fdbb4098f70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fdba8009760 tx=0x7fdba8009060).stop 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 -- 172.21.15.32:0/713839582 shutdown_connections 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb4131860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/713839582 >> v2:172.21.15.32:3300/0 conn(0x7fdbb40654a0 0x7fdbb4098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/713839582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb4098b30 0x7fdbb4098f70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 -- 172.21.15.32:0/713839582 >> 172.21.15.32:0/713839582 conn(0x7fdbb409a740 msgr2=0x7fdbb409ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 -- 172.21.15.32:0/713839582 shutdown_connections 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.096+0000 7fdbb96b9700 1 -- 172.21.15.32:0/713839582 wait complete. 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 Processor -- start 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 -- start start 2023-09-30T22:21:43.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 0x7fdbb413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb413af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbb413b470 0x7fdbb411a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb412f850 con 0x7fdbb413b470 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb411d9d0 con 0x7fdbb412f470 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.098+0000 7fdbb96b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4093c30 con 0x7fdbb40654a0 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 0x7fdbb413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 0x7fdbb413aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35574/0 (socket says 172.21.15.32:35574) 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 -- 172.21.15.32:0/282659434 learned_addr learned my addr 172.21.15.32:0/282659434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb27fc700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb413af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb37fe700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbb413b470 0x7fdbb411a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 -- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 msgr2=0x7fdbb413af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb413af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 -- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbb413b470 msgr2=0x7fdbb411a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbb413b470 0x7fdbb411a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 -- 172.21.15.32:0/282659434 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4009370 con 0x7fdbb40654a0 2023-09-30T22:21:43.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.099+0000 7fdbb2ffd700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 0x7fdbb413aa90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fdba400ea20 tx=0x7fdba40090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.100+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400c850 con 0x7fdbb40654a0 2023-09-30T22:21:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.100+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400c9b0 con 0x7fdbb40654a0 2023-09-30T22:21:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.100+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac009a20 con 0x7fdbb40654a0 2023-09-30T22:21:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.100+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400cd30 con 0x7fdbb40654a0 2023-09-30T22:21:43.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.100+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba80092a0 con 0x7fdbb40654a0 2023-09-30T22:21:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.101+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fdba4012600 con 0x7fdbb40654a0 2023-09-30T22:21:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.101+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fdba403b5d0 con 0x7fdbb40654a0 2023-09-30T22:21:43.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.101+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb94005120 con 0x7fdbb40654a0 2023-09-30T22:21:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 Processor -- start 2023-09-30T22:21:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 -- start start 2023-09-30T22:21:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64cc065450 0x7f64cc095d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098640 0x7f64cc098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098f50 0x7f64cc135c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc095880 con 0x7f64cc065450 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc065eb0 con 0x7f64cc098640 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.102+0000 7f64d0e49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc06a990 con 0x7f64cc098f50 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64ca59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64cc065450 0x7f64cc095d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64ca59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64cc065450 0x7f64cc095d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54568/0 (socket says 172.21.15.32:54568) 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64ca59c700 1 -- 172.21.15.32:0/1618348806 learned_addr learned my addr 172.21.15.32:0/1618348806 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64cad9d700 1 --2- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098f50 0x7f64cc135c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64cad9d700 1 -- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098640 msgr2=0x7f64cc098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64cad9d700 1 --2- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098640 0x7f64cc098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64cad9d700 1 -- 172.21.15.32:0/1618348806 >> v2:172.21.15.32:3300/0 conn(0x7f64cc065450 msgr2=0x7f64cc095d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64cad9d700 1 --2- 172.21.15.32:0/1618348806 >> v2:172.21.15.32:3300/0 conn(0x7f64cc065450 0x7f64cc095d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.103+0000 7f64cad9d700 1 -- 172.21.15.32:0/1618348806 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc097cb0 con 0x7f64cc098f50 2023-09-30T22:21:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.104+0000 7f64cad9d700 1 --2- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098f50 0x7f64cc135c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f64c4009e90 tx=0x7f64c4009780).ready entity=mon.2 client_cookie=aecc3a3d88d90ec4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.107+0000 7f64c959a700 1 -- 172.21.15.32:0/1618348806 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400e070 con 0x7f64cc098f50 2023-09-30T22:21:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.107+0000 7f64c959a700 1 -- 172.21.15.32:0/1618348806 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c400bb20 con 0x7f64cc098f50 2023-09-30T22:21:43.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.107+0000 7f64c959a700 1 -- 172.21.15.32:0/1618348806 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400bea0 con 0x7f64cc098f50 2023-09-30T22:21:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.107+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fdba4019900 con 0x7fdbb40654a0 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.107+0000 7f64d0e49700 1 -- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098f50 msgr2=0x7f64cc135c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.107+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098f50 0x7f64cc135c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f64c4009e90 tx=0x7f64c4009780).stop 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 -- 172.21.15.32:0/1618348806 shutdown_connections 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098f50 0x7f64cc135c50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1618348806 >> v2:172.21.15.32:3300/0 conn(0x7f64cc065450 0x7f64cc095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1618348806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098640 0x7f64cc098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 -- 172.21.15.32:0/1618348806 >> 172.21.15.32:0/1618348806 conn(0x7f64cc09a740 msgr2=0x7f64cc09ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 -- 172.21.15.32:0/1618348806 shutdown_connections 2023-09-30T22:21:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.108+0000 7f64d0e49700 1 -- 172.21.15.32:0/1618348806 wait complete. 2023-09-30T22:21:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.109+0000 7f64d0e49700 1 Processor -- start 2023-09-30T22:21:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.109+0000 7f64d0e49700 1 -- start start 2023-09-30T22:21:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64d0e49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 0x7f64cc11f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64d0e49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098640 0x7f64cc11f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64d0e49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098f50 0x7f64cc11f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64d0e49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc11ee20 con 0x7f64cc065450 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64d0e49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc09cbb0 con 0x7f64cc098f50 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64d0e49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc094e50 con 0x7f64cc098640 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64ca59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 0x7f64cc11f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64ca59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 0x7f64cc11f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54570/0 (socket says 172.21.15.32:54570) 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64ca59c700 1 -- 172.21.15.32:0/1010104758 learned_addr learned my addr 172.21.15.32:0/1010104758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64cad9d700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098f50 0x7f64cc11f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.110+0000 7f64c9d9b700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098640 0x7f64cc11f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ca59c700 1 -- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098640 msgr2=0x7f64cc11f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ca59c700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098640 0x7f64cc11f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ca59c700 1 -- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098f50 msgr2=0x7f64cc11f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ca59c700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098f50 0x7f64cc11f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ca59c700 1 -- 172.21.15.32:0/1010104758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4009a20 con 0x7f64cc065450 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64cad9d700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098f50 0x7f64cc11f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ca59c700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 0x7f64cc11f010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f64bc00e900 tx=0x7f64bc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc012600 con 0x7f64cc065450 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc012760 con 0x7f64cc065450 2023-09-30T22:21:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc012ae0 con 0x7f64cc065450 2023-09-30T22:21:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc120f00 con 0x7f64cc065450 2023-09-30T22:21:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.111+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc120510 con 0x7f64cc065450 2023-09-30T22:21:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.112+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f64bc01b400 con 0x7f64cc065450 2023-09-30T22:21:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.113+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f64bc03b3b0 con 0x7f64cc065450 2023-09-30T22:21:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.113+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc096cd0 con 0x7f64cc065450 2023-09-30T22:21:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.118+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f64bc019310 con 0x7f64cc065450 2023-09-30T22:21:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.136+0000 7f2da29c3700 1 Processor -- start 2023-09-30T22:21:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.136+0000 7f2da29c3700 1 -- start start 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.137+0000 7f2da29c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d9c091b00 0x7f2d9c12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.137+0000 7f2da29c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c12ce60 0x7f2d9c12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.137+0000 7f2da29c3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c096760 con 0x7f2d9c096380 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.137+0000 7f2da29c3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c091ff0 con 0x7f2d9c091b00 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.137+0000 7f2da29c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c092c80 con 0x7f2d9c12ce60 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d9c091b00 0x7f2d9c12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d9c091b00 0x7f2d9c12c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54586/0 (socket says 172.21.15.32:54586) 2023-09-30T22:21:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/4284562404 learned_addr learned my addr 172.21.15.32:0/4284562404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2da0f60700 1 --2- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c12ce60 0x7f2d9c12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c12ce60 msgr2=0x7f2d9c12f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 --2- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c12ce60 0x7f2d9c12f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/4284562404 >> v1:172.21.15.32:6789/0 conn(0x7f2d9c096380 legacy=0x7f2d9c094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/4284562404 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c094e40 con 0x7f2d9c091b00 2023-09-30T22:21:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9b7fe700 1 --2- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 0x7f2d9c12c970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2d90006fd0 tx=0x7f2d900090f0).ready entity=mon.0 client_cookie=39029f7d5260108e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9affd700 1 -- 172.21.15.32:0/4284562404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000b600 con 0x7f2d9c091b00 2023-09-30T22:21:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9affd700 1 -- 172.21.15.32:0/4284562404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9000b760 con 0x7f2d9c091b00 2023-09-30T22:21:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.138+0000 7f2d9affd700 1 -- 172.21.15.32:0/4284562404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000bae0 con 0x7f2d9c091b00 2023-09-30T22:21:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.139+0000 7f2da29c3700 1 -- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 msgr2=0x7f2d9c12c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.139+0000 7f2da29c3700 1 --2- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 0x7f2d9c12c970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2d90006fd0 tx=0x7f2d900090f0).stop 2023-09-30T22:21:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.140+0000 7f2da29c3700 1 -- 172.21.15.32:0/4284562404 shutdown_connections 2023-09-30T22:21:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.140+0000 7f2da29c3700 1 --2- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c12ce60 0x7f2d9c12f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.140+0000 7f2da29c3700 1 --2- 172.21.15.32:0/4284562404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 0x7f2d9c12c970 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.140+0000 7f2da29c3700 1 -- 172.21.15.32:0/4284562404 >> 172.21.15.32:0/4284562404 conn(0x7f2d9c097470 msgr2=0x7f2d9c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.140+0000 7f2da29c3700 1 -- 172.21.15.32:0/4284562404 shutdown_connections 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.140+0000 7f2da29c3700 1 -- 172.21.15.32:0/4284562404 wait complete. 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 Processor -- start 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 -- start start 2023-09-30T22:21:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 0x7f2d9c1294f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 0x7f2d9c12ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d9c12ce60 0x7f2d9c12b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c128ce0 con 0x7f2d9c091b00 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c129bb0 con 0x7f2d9c096380 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.141+0000 7f2da29c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c12c8b0 con 0x7f2d9c12ce60 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 0x7f2d9c12ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2da0f60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d9c12ce60 0x7f2d9c12b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 0x7f2d9c12ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40950/0 (socket says 172.21.15.32:40950) 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/227452122 learned_addr learned my addr 172.21.15.32:0/227452122 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d9c12ce60 msgr2=0x7f2d9c12b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d9c12ce60 0x7f2d9c12b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 msgr2=0x7f2d9c1294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 0x7f2d9c1294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 -- 172.21.15.32:0/227452122 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90009370 con 0x7f2d9c096380 2023-09-30T22:21:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d9b7fe700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 0x7f2d9c12ad30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2d9c092ab0 tx=0x7f2d90017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000b8d0 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c068490 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9000ba30 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.142+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9001c670 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.143+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c129e70 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.143+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2d9001c9e0 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.143+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2d9003d1f0 con 0x7f2d9c096380 2023-09-30T22:21:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.144+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c05a260 con 0x7f2d9c096380 2023-09-30T22:21:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 Processor -- start 2023-09-30T22:21:43.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 -- start start 2023-09-30T22:21:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 0x7f4bf80969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bf80684f0 0x7f4bf812fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80933b0 con 0x7f4bf8096eb0 2023-09-30T22:21:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf8092190 con 0x7f4bf80684f0 2023-09-30T22:21:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bfe283700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8069a20 con 0x7f4bf80965e0 2023-09-30T22:21:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 0x7f4bf80969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 0x7f4bf80969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40962/0 (socket says 172.21.15.32:40962) 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.149+0000 7f4bf7fff700 1 -- 172.21.15.32:0/967340241 learned_addr learned my addr 172.21.15.32:0/967340241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bfc820700 1 --2- 172.21.15.32:0/967340241 >> v2:172.21.15.32:3300/0 conn(0x7f4bf80684f0 0x7f4bf812fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bfc820700 1 -- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 msgr2=0x7f4bf80969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bfc820700 1 --2- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 0x7f4bf80969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bfc820700 1 -- 172.21.15.32:0/967340241 >> v1:172.21.15.32:6789/0 conn(0x7f4bf8096eb0 legacy=0x7f4bf8067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bfc820700 1 -- 172.21.15.32:0/967340241 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8069ca0 con 0x7f4bf80684f0 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bf7fff700 1 --2- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 0x7f4bf80969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bfc820700 1 --2- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80684f0 0x7f4bf812fc50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f4bf0009e60 tx=0x7f4bf0009780).ready entity=mon.0 client_cookie=6e457f97706321a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bf6ffd700 1 -- 172.21.15.32:0/967340241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000e070 con 0x7f4bf80684f0 2023-09-30T22:21:43.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bf6ffd700 1 -- 172.21.15.32:0/967340241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000bb60 con 0x7f4bf80684f0 2023-09-30T22:21:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.150+0000 7f4bf6ffd700 1 -- 172.21.15.32:0/967340241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0012410 con 0x7f4bf80684f0 2023-09-30T22:21:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 -- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80684f0 msgr2=0x7f4bf812fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 --2- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80684f0 0x7f4bf812fc50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f4bf0009e60 tx=0x7f4bf0009780).stop 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 -- 172.21.15.32:0/967340241 shutdown_connections 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 --2- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80684f0 0x7f4bf812fc50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 --2- 172.21.15.32:0/967340241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf80965e0 0x7f4bf80969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 -- 172.21.15.32:0/967340241 >> 172.21.15.32:0/967340241 conn(0x7f4bf809a220 msgr2=0x7f4bf809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 -- 172.21.15.32:0/967340241 shutdown_connections 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.151+0000 7f4bfe283700 1 -- 172.21.15.32:0/967340241 wait complete. 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.152+0000 7f4bfe283700 1 Processor -- start 2023-09-30T22:21:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.152+0000 7f4bfe283700 1 -- start start 2023-09-30T22:21:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2d9002a070 con 0x7f2d9c096380 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfe283700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 0x7f4bf81363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfe283700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80965e0 0x7f4bf81368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfe283700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf8096eb0 0x7f4bf8113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfe283700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8091bf0 con 0x7f4bf80965e0 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfe283700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8117460 con 0x7f4bf8096eb0 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfe283700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf812b170 con 0x7f4bf80684f0 2023-09-30T22:21:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 0x7f4bf81363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 0x7f4bf81363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35620/0 (socket says 172.21.15.32:35620) 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 -- 172.21.15.32:0/4164817251 learned_addr learned my addr 172.21.15.32:0/4164817251 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf77fe700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80965e0 0x7f4bf81368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bfc820700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf8096eb0 0x7f4bf8113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 -- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf8096eb0 msgr2=0x7f4bf8113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf8096eb0 0x7f4bf8113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 -- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80965e0 msgr2=0x7f4bf81368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.153+0000 7f4bf7fff700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80965e0 0x7f4bf81368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bf7fff700 1 -- 172.21.15.32:0/4164817251 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8009370 con 0x7f4bf80684f0 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bf7fff700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 0x7f4bf81363d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4be800ea50 tx=0x7f4be80090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800ee90 con 0x7f4bf80684f0 2023-09-30T22:21:43.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800c850 con 0x7f4bf80684f0 2023-09-30T22:21:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800cbd0 con 0x7f4bf80684f0 2023-09-30T22:21:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf0009a20 con 0x7f4bf80684f0 2023-09-30T22:21:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.154+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf81171a0 con 0x7f4bf80684f0 2023-09-30T22:21:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.155+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4be801b400 con 0x7f4bf80684f0 2023-09-30T22:21:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.155+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4be803b5a0 con 0x7f4bf80684f0 2023-09-30T22:21:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.155+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf805a260 con 0x7f4bf80684f0 2023-09-30T22:21:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.160+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4be8016070 con 0x7f4bf80684f0 2023-09-30T22:21:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.310+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64cc122080 con 0x7f64cc065450 2023-09-30T22:21:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.310+0000 7f64ba7fc700 1 -- 172.21.15.32:0/1010104758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64bc029030 con 0x7f64cc065450 2023-09-30T22:21:43.311 INFO:teuthology.orchestra.run.smithi032.stdout:38654705667 2023-09-30T22:21:43.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.312+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdb940057f0 con 0x7fdbb40654a0 2023-09-30T22:21:43.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.312+0000 7fdb9affd700 1 -- 172.21.15.32:0/282659434 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdba402a090 con 0x7fdbb40654a0 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 >> v1:172.21.15.32:6816/118188 conn(0x7f64ac02bb60 legacy=0x7f64ac02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 msgr2=0x7f64cc11f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 0x7f64cc11f010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f64bc00e900 tx=0x7f64bc0090f0).stop 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 shutdown_connections 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64cc098f50 0x7f64cc11f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64cc065450 0x7f64cc11f010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 --2- 172.21.15.32:0/1010104758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64cc098640 0x7f64cc11f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 >> 172.21.15.32:0/1010104758 conn(0x7f64cc09a740 msgr2=0x7f64cc09bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 shutdown_connections 2023-09-30T22:21:43.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.313+0000 7f64d0e49700 1 -- 172.21.15.32:0/1010104758 wait complete. 2023-09-30T22:21:43.316 INFO:teuthology.orchestra.run.smithi032.stdout:38654705667 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.315+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 >> v1:172.21.15.32:6816/118188 conn(0x7fdba002bb90 legacy=0x7fdba002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.315+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 msgr2=0x7fdbb413aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.315+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 0x7fdbb413aa90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fdba400ea20 tx=0x7fdba40090f0).stop 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 shutdown_connections 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbb413b470 0x7fdbb411a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbb40654a0 0x7fdbb413aa90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 --2- 172.21.15.32:0/282659434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbb412f470 0x7fdbb413af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 >> 172.21.15.32:0/282659434 conn(0x7fdbb409a740 msgr2=0x7fdbb41142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 shutdown_connections 2023-09-30T22:21:43.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.316+0000 7fdbb96b9700 1 -- 172.21.15.32:0/282659434 wait complete. 2023-09-30T22:21:43.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2023-09-30T22:21:43.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.3 2023-09-30T22:21:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.348+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4bf8069210 con 0x7f4bf80684f0 2023-09-30T22:21:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.349+0000 7f4bdffff700 1 -- 172.21.15.32:0/4164817251 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4be8016020 con 0x7f4bf80684f0 2023-09-30T22:21:43.350 INFO:teuthology.orchestra.run.smithi032.stdout:38654705667 2023-09-30T22:21:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 >> v1:172.21.15.32:6816/118188 conn(0x7f4be402bb60 legacy=0x7f4be402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 msgr2=0x7f4bf81363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 0x7f4bf81363d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4be800ea50 tx=0x7f4be80090f0).stop 2023-09-30T22:21:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 shutdown_connections 2023-09-30T22:21:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf8096eb0 0x7f4bf8113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf80684f0 0x7f4bf81363d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 --2- 172.21.15.32:0/4164817251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80965e0 0x7f4bf81368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 >> 172.21.15.32:0/4164817251 conn(0x7f4bf809a220 msgr2=0x7f4bf8099000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 shutdown_connections 2023-09-30T22:21:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.351+0000 7f4bfe283700 1 -- 172.21.15.32:0/4164817251 wait complete. 2023-09-30T22:21:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.359+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2d9c093940 con 0x7f2d9c096380 2023-09-30T22:21:43.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.360+0000 7f2d83fff700 1 -- 172.21.15.32:0/227452122 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d90034090 con 0x7f2d9c096380 2023-09-30T22:21:43.360 INFO:teuthology.orchestra.run.smithi032.stdout:38654705667 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 >> v1:172.21.15.32:6816/118188 conn(0x7f2d8802bb90 legacy=0x7f2d8802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 msgr2=0x7f2d9c12ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 0x7f2d9c12ad30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f2d9c092ab0 tx=0x7f2d90017000).stop 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 shutdown_connections 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d9c12ce60 0x7f2d9c12b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c091b00 0x7f2d9c1294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:21:43.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 --2- 172.21.15.32:0/227452122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d9c096380 0x7f2d9c12ad30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:43.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 >> 172.21.15.32:0/227452122 conn(0x7f2d9c097470 msgr2=0x7f2d9c099100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:43.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 shutdown_connections 2023-09-30T22:21:43.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:43.362+0000 7f2da29c3700 1 -- 172.21.15.32:0/227452122 wait complete. 2023-09-30T22:21:43.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.1 2023-09-30T22:21:43.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2023-09-30T22:21:44.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:21:44.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:21:44.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:21:44.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.483+0000 7f78ee2a7700 1 Processor -- start 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.483+0000 7f78ee2a7700 1 -- start start 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78ee2a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 0x7f78e80969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78ee2a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78e80684f0 0x7f78e812fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78ee2a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80933b0 con 0x7f78e8096eb0 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78ee2a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e8092190 con 0x7f78e80684f0 2023-09-30T22:21:44.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78ee2a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8069a20 con 0x7f78e80965e0 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 0x7f78e80969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 0x7f78e80969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40980/0 (socket says 172.21.15.32:40980) 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78e7fff700 1 -- 172.21.15.32:0/1522994883 learned_addr learned my addr 172.21.15.32:0/1522994883 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.484+0000 7f78ec844700 1 --2- 172.21.15.32:0/1522994883 >> v2:172.21.15.32:3300/0 conn(0x7f78e80684f0 0x7f78e812fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e7fff700 1 -- 172.21.15.32:0/1522994883 >> v2:172.21.15.32:3300/0 conn(0x7f78e80684f0 msgr2=0x7f78e812fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e7fff700 1 --2- 172.21.15.32:0/1522994883 >> v2:172.21.15.32:3300/0 conn(0x7f78e80684f0 0x7f78e812fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e7fff700 1 -- 172.21.15.32:0/1522994883 >> v1:172.21.15.32:6789/0 conn(0x7f78e8096eb0 legacy=0x7f78e8067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e7fff700 1 -- 172.21.15.32:0/1522994883 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8069ca0 con 0x7f78e80965e0 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e7fff700 1 --2- 172.21.15.32:0/1522994883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 0x7f78e80969c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78d8006fd0 tx=0x7f78d800b3b0).ready entity=mon.1 client_cookie=91cb5d29b5778b96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e6ffd700 1 -- 172.21.15.32:0/1522994883 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d800da70 con 0x7f78e80965e0 2023-09-30T22:21:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e6ffd700 1 -- 172.21.15.32:0/1522994883 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d800dbd0 con 0x7f78e80965e0 2023-09-30T22:21:44.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.485+0000 7f78e6ffd700 1 -- 172.21.15.32:0/1522994883 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010470 con 0x7f78e80965e0 2023-09-30T22:21:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 -- 172.21.15.32:0/1522994883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 msgr2=0x7f78e80969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/1522994883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 0x7f78e80969c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78d8006fd0 tx=0x7f78d800b3b0).stop 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 -- 172.21.15.32:0/1522994883 shutdown_connections 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/1522994883 >> v2:172.21.15.32:3300/0 conn(0x7f78e80684f0 0x7f78e812fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/1522994883 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e80965e0 0x7f78e80969c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 -- 172.21.15.32:0/1522994883 >> 172.21.15.32:0/1522994883 conn(0x7f78e809a220 msgr2=0x7f78e809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 -- 172.21.15.32:0/1522994883 shutdown_connections 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.486+0000 7f78ee2a7700 1 -- 172.21.15.32:0/1522994883 wait complete. 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.487+0000 7f78ee2a7700 1 Processor -- start 2023-09-30T22:21:44.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.487+0000 7f78ee2a7700 1 -- start start 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ee2a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 0x7f78e813a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ee2a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80965e0 0x7f78e813ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ee2a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e8096eb0 0x7f78e813b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ee2a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e811e410 con 0x7f78e80965e0 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ee2a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8066f80 con 0x7f78e8096eb0 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ee2a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8091bf0 con 0x7f78e80684f0 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 0x7f78e813a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ec844700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e8096eb0 0x7f78e813b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 0x7f78e813a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35624/0 (socket says 172.21.15.32:35624) 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 -- 172.21.15.32:0/3730114681 learned_addr learned my addr 172.21.15.32:0/3730114681 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e77fe700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80965e0 0x7f78e813ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 -- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e8096eb0 msgr2=0x7f78e813b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e8096eb0 0x7f78e813b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 -- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80965e0 msgr2=0x7f78e813ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80965e0 0x7f78e813ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e7fff700 1 -- 172.21.15.32:0/3730114681 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d800b610 con 0x7f78e80684f0 2023-09-30T22:21:44.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78e77fe700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80965e0 0x7f78e813ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.488+0000 7f78ec844700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e8096eb0 0x7f78e813b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.489+0000 7f78e7fff700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 0x7f78e813a6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f78d800d5d0 tx=0x7f78d800b3b0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.489+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8019450 con 0x7f78e80684f0 2023-09-30T22:21:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.489+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d80195b0 con 0x7f78e80684f0 2023-09-30T22:21:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.489+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8019930 con 0x7f78e80684f0 2023-09-30T22:21:44.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.489+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e811cea0 con 0x7f78e80684f0 2023-09-30T22:21:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.489+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e811ec20 con 0x7f78e80684f0 2023-09-30T22:21:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.490+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f78d8019ca0 con 0x7f78e80684f0 2023-09-30T22:21:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.490+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f78d803c6e0 con 0x7f78e80684f0 2023-09-30T22:21:44.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.490+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e811ec20 con 0x7f78e80684f0 2023-09-30T22:21:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.491+0000 7efdbe9a7700 1 Processor -- start 2023-09-30T22:21:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbe9a7700 1 -- start start 2023-09-30T22:21:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbe9a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8123e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbe9a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbe9a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb8068880 con 0x7efdb8067ce0 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbe9a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80680c0 con 0x7efdb8092340 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbe9a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8068c90 con 0x7efdb8123a90 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbcf44700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbcf44700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8127680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41006/0 (socket says 172.21.15.32:41006) 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.492+0000 7efdbcf44700 1 -- 172.21.15.32:0/2329940286 learned_addr learned my addr 172.21.15.32:0/2329940286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdb7fff700 1 --2- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8123e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdbcf44700 1 -- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 msgr2=0x7efdb8123e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdbcf44700 1 --2- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8123e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdbcf44700 1 -- 172.21.15.32:0/2329940286 >> v1:172.21.15.32:6789/0 conn(0x7efdb8067ce0 legacy=0x7efdb8126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdbcf44700 1 -- 172.21.15.32:0/2329940286 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8124430 con 0x7efdb8092340 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdbcf44700 1 --2- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8127680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7efdb0009f50 tx=0x7efdb0009840).ready entity=mon.1 client_cookie=62f02e27810d3ba6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdb6ffd700 1 -- 172.21.15.32:0/2329940286 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000e070 con 0x7efdb8092340 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdb6ffd700 1 -- 172.21.15.32:0/2329940286 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb000bab0 con 0x7efdb8092340 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.493+0000 7efdb6ffd700 1 -- 172.21.15.32:0/2329940286 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000be30 con 0x7efdb8092340 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 msgr2=0x7efdb8127680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 --2- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8127680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7efdb0009f50 tx=0x7efdb0009840).stop 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2329940286 shutdown_connections 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 --2- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8127680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 --2- 172.21.15.32:0/2329940286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8123e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2329940286 >> 172.21.15.32:0/2329940286 conn(0x7efdb809a220 msgr2=0x7efdb809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2329940286 shutdown_connections 2023-09-30T22:21:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.494+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2329940286 wait complete. 2023-09-30T22:21:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.495+0000 7efdbe9a7700 1 Processor -- start 2023-09-30T22:21:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.495+0000 7efdbe9a7700 1 -- start start 2023-09-30T22:21:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbe9a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb8067ce0 0x7efdb8111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f78e811ec20 con 0x7f78e80684f0 2023-09-30T22:21:44.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbe9a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8111c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbe9a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8112140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbe9a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8112630 con 0x7efdb8067ce0 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbe9a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8112c30 con 0x7efdb8092340 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbe9a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8114ca0 con 0x7efdb8123a90 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbcf44700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbcf44700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8112140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35654/0 (socket says 172.21.15.32:35654) 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdbcf44700 1 -- 172.21.15.32:0/2815213699 learned_addr learned my addr 172.21.15.32:0/2815213699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.496+0000 7efdb7fff700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb8067ce0 0x7efdb8111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdb77fe700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8111c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbcf44700 1 -- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 msgr2=0x7efdb8111c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbcf44700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8111c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbcf44700 1 -- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb8067ce0 msgr2=0x7efdb8111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbcf44700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb8067ce0 0x7efdb8111760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbcf44700 1 -- 172.21.15.32:0/2815213699 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb0009ae0 con 0x7efdb8123a90 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdb77fe700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8111c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbcf44700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8112140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7efdb000b770 tx=0x7efdb8069f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb00092e0 con 0x7efdb8123a90 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb001e410 con 0x7efdb8123a90 2023-09-30T22:21:44.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2815213699 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb8114970 con 0x7efdb8123a90 2023-09-30T22:21:44.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb001e7b0 con 0x7efdb8123a90 2023-09-30T22:21:44.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.497+0000 7efdbe9a7700 1 -- 172.21.15.32:0/2815213699 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb8115920 con 0x7efdb8123a90 2023-09-30T22:21:44.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.499+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb8115180 con 0x7efdb8123a90 2023-09-30T22:21:44.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.499+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7efdb0003040 con 0x7efdb8123a90 2023-09-30T22:21:44.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.500+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7efdb003cdc0 con 0x7efdb8123a90 2023-09-30T22:21:44.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.504+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7efdb0011960 con 0x7efdb8123a90 2023-09-30T22:21:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.526+0000 7f4970e6e700 1 Processor -- start 2023-09-30T22:21:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.526+0000 7f4970e6e700 1 -- start start 2023-09-30T22:21:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4970e6e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c091b00 0x7f496c12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4970e6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f496c12ce60 0x7f496c12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4970e6e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496c096760 con 0x7f496c096380 2023-09-30T22:21:44.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4970e6e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496c091ff0 con 0x7f496c12ce60 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4970e6e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f496c092c80 con 0x7f496c091b00 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f496a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f496c096380 0x7f496c094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54804/0 (socket says 172.21.15.32:54804) 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4969d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c091b00 0x7f496c12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f496a59c700 1 -- 172.21.15.32:0/3053215214 learned_addr learned my addr 172.21.15.32:0/3053215214 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f496ad9d700 1 --2- 172.21.15.32:0/3053215214 >> v2:172.21.15.32:3300/0 conn(0x7f496c12ce60 0x7f496c12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f496959a700 1 -- 172.21.15.32:0/3053215214 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076773623 0 0) 0x7f496c096760 con 0x7f496c096380 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4969d9b700 1 -- 172.21.15.32:0/3053215214 >> v2:172.21.15.32:3300/0 conn(0x7f496c12ce60 msgr2=0x7f496c12f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.527+0000 7f4969d9b700 1 --2- 172.21.15.32:0/3053215214 >> v2:172.21.15.32:3300/0 conn(0x7f496c12ce60 0x7f496c12f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.528+0000 7f4969d9b700 1 -- 172.21.15.32:0/3053215214 >> v1:172.21.15.32:6789/0 conn(0x7f496c096380 legacy=0x7f496c094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.528+0000 7f4969d9b700 1 -- 172.21.15.32:0/3053215214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c094e40 con 0x7f496c091b00 2023-09-30T22:21:44.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.528+0000 7f4969d9b700 1 --2- 172.21.15.32:0/3053215214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c091b00 0x7f496c12c970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4960009760 tx=0x7f4960009060).ready entity=mon.2 client_cookie=8eb0ea9ac7337fb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.528+0000 7f496959a700 1 -- 172.21.15.32:0/3053215214 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000f510 con 0x7f496c091b00 2023-09-30T22:21:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.528+0000 7f496959a700 1 -- 172.21.15.32:0/3053215214 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496000f670 con 0x7f496c091b00 2023-09-30T22:21:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.528+0000 7f496959a700 1 -- 172.21.15.32:0/3053215214 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000f9f0 con 0x7f496c091b00 2023-09-30T22:21:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 -- 172.21.15.32:0/3053215214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c091b00 msgr2=0x7f496c12c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 --2- 172.21.15.32:0/3053215214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c091b00 0x7f496c12c970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4960009760 tx=0x7f4960009060).stop 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 -- 172.21.15.32:0/3053215214 shutdown_connections 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 --2- 172.21.15.32:0/3053215214 >> v2:172.21.15.32:3300/0 conn(0x7f496c12ce60 0x7f496c12f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 --2- 172.21.15.32:0/3053215214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c091b00 0x7f496c12c970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 -- 172.21.15.32:0/3053215214 >> 172.21.15.32:0/3053215214 conn(0x7f496c097470 msgr2=0x7f496c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 -- 172.21.15.32:0/3053215214 shutdown_connections 2023-09-30T22:21:44.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.529+0000 7f4970e6e700 1 -- 172.21.15.32:0/3053215214 wait complete. 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 Processor -- start 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 -- start start 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f496c091b00 0x7f496c138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 0x7f496c138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c12ce60 0x7f496c138d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c114570 con 0x7f496c096380 2023-09-30T22:21:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496c059600 con 0x7f496c091b00 2023-09-30T22:21:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4970e6e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f496c06a520 con 0x7f496c12ce60 2023-09-30T22:21:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4969d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 0x7f496c138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.530+0000 7f4969d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 0x7f496c138850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54662/0 (socket says 172.21.15.32:54662) 2023-09-30T22:21:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 -- 172.21.15.32:0/415728750 learned_addr learned my addr 172.21.15.32:0/415728750 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f496a59c700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f496c091b00 0x7f496c138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f496ad9d700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c12ce60 0x7f496c138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 -- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c12ce60 msgr2=0x7f496c138d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c12ce60 0x7f496c138d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 -- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f496c091b00 msgr2=0x7f496c138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f496c091b00 0x7f496c138360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 -- 172.21.15.32:0/415728750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600092a0 con 0x7f496c096380 2023-09-30T22:21:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f4969d9b700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 0x7f496c138850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4960009e00 tx=0x7f4960017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000f7e0 con 0x7f496c096380 2023-09-30T22:21:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496000f940 con 0x7f496c096380 2023-09-30T22:21:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.531+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496001b640 con 0x7f496c096380 2023-09-30T22:21:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.532+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c12c540 con 0x7f496c096380 2023-09-30T22:21:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.532+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c112460 con 0x7f496c096380 2023-09-30T22:21:44.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.532+0000 7f6db6569700 1 Processor -- start 2023-09-30T22:21:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.532+0000 7f6db6569700 1 -- start start 2023-09-30T22:21:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f496001b9f0 con 0x7f496c096380 2023-09-30T22:21:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496c05a260 con 0x7f496c096380 2023-09-30T22:21:44.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db6569700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6db0066cd0 0x7f6db00670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db6569700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db00675a0 0x7f6db012f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db6569700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 0x7f6db012fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db6569700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db0069d50 con 0x7f6db0066cd0 2023-09-30T22:21:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db6569700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0092260 con 0x7f6db012f6e0 2023-09-30T22:21:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db6569700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0069040 con 0x7f6db00675a0 2023-09-30T22:21:44.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f496003ca10 con 0x7f496c096380 2023-09-30T22:21:44.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db4b06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 0x7f6db012fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.533+0000 7f6db4b06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 0x7f6db012fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41040/0 (socket says 172.21.15.32:41040) 2023-09-30T22:21:44.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 -- 172.21.15.32:0/1590973552 learned_addr learned my addr 172.21.15.32:0/1590973552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6daf7fe700 1 --2- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db00675a0 0x7f6db012f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 -- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db00675a0 msgr2=0x7f6db012f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 --2- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db00675a0 0x7f6db012f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 -- 172.21.15.32:0/1590973552 >> v2:172.21.15.32:3300/0 conn(0x7f6db0066cd0 msgr2=0x7f6db00670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6daffff700 1 --2- 172.21.15.32:0/1590973552 >> v2:172.21.15.32:3300/0 conn(0x7f6db0066cd0 0x7f6db00670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 --2- 172.21.15.32:0/1590973552 >> v2:172.21.15.32:3300/0 conn(0x7f6db0066cd0 0x7f6db00670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 -- 172.21.15.32:0/1590973552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0069a70 con 0x7f6db012f6e0 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6daffff700 1 --2- 172.21.15.32:0/1590973552 >> v2:172.21.15.32:3300/0 conn(0x7f6db0066cd0 0x7f6db00670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6db4b06700 1 --2- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 0x7f6db012fb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6da8009f20 tx=0x7f6da8009840).ready entity=mon.1 client_cookie=6ddd2987ee305240 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6daeffd700 1 -- 172.21.15.32:0/1590973552 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800e070 con 0x7f6db012f6e0 2023-09-30T22:21:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6daeffd700 1 -- 172.21.15.32:0/1590973552 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800b9d0 con 0x7f6db012f6e0 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.534+0000 7f6daeffd700 1 -- 172.21.15.32:0/1590973552 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800bd50 con 0x7f6db012f6e0 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 -- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 msgr2=0x7f6db012fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 --2- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 0x7f6db012fb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6da8009f20 tx=0x7f6da8009840).stop 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 -- 172.21.15.32:0/1590973552 shutdown_connections 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 --2- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db012f6e0 0x7f6db012fb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 --2- 172.21.15.32:0/1590973552 >> v2:172.21.15.32:3300/0 conn(0x7f6db0066cd0 0x7f6db00670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 --2- 172.21.15.32:0/1590973552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db00675a0 0x7f6db012f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 -- 172.21.15.32:0/1590973552 >> 172.21.15.32:0/1590973552 conn(0x7f6db009a220 msgr2=0x7f6db009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 -- 172.21.15.32:0/1590973552 shutdown_connections 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.535+0000 7f6db6569700 1 -- 172.21.15.32:0/1590973552 wait complete. 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.537+0000 7f6db6569700 1 Processor -- start 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.537+0000 7f6db6569700 1 -- start start 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db6569700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 0x7f6db0112630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db6569700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db00675a0 0x7f6db012d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db6569700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db012f6e0 0x7f6db012d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db6569700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db012e8c0 con 0x7f6db012f6e0 2023-09-30T22:21:44.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db6569700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01121a0 con 0x7f6db00675a0 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db6569700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6db0099b30 con 0x7f6db0066cd0 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 0x7f6db0112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 0x7f6db0112630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35684/0 (socket says 172.21.15.32:35684) 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 -- 172.21.15.32:0/157838325 learned_addr learned my addr 172.21.15.32:0/157838325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daf7fe700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db00675a0 0x7f6db012d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6db4b06700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db012f6e0 0x7f6db012d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 -- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db00675a0 msgr2=0x7f6db012d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db00675a0 0x7f6db012d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 -- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db012f6e0 msgr2=0x7f6db012d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db012f6e0 0x7f6db012d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.538+0000 7f6daffff700 1 -- 172.21.15.32:0/157838325 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da8009ae0 con 0x7f6db0066cd0 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.539+0000 7f6daffff700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 0x7f6db0112630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6da000c880 tx=0x7f6da0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.539+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0010580 con 0x7f6db0066cd0 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.539+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db012f0c0 con 0x7f6db0066cd0 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.539+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da00106e0 con 0x7f6db0066cd0 2023-09-30T22:21:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.539+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0010a60 con 0x7f6db0066cd0 2023-09-30T22:21:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.539+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db012dde0 con 0x7f6db0066cd0 2023-09-30T22:21:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.540+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6da001b460 con 0x7f6db0066cd0 2023-09-30T22:21:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.540+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6da003b080 con 0x7f6db0066cd0 2023-09-30T22:21:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.541+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db0111f60 con 0x7f6db0066cd0 2023-09-30T22:21:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.542+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4960022930 con 0x7f496c096380 2023-09-30T22:21:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.548+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6da00188e0 con 0x7f6db0066cd0 2023-09-30T22:21:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.688+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f78e811ec20 con 0x7f78e80684f0 2023-09-30T22:21:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.688+0000 7f78cffff700 1 -- 172.21.15.32:0/3730114681 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78e811ec20 con 0x7f78e80684f0 2023-09-30T22:21:44.689 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 >> v1:172.21.15.32:6816/118188 conn(0x7f78d402bb60 legacy=0x7f78d402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 msgr2=0x7f78e813a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 0x7f78e813a6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f78d800d5d0 tx=0x7f78d800b3b0).stop 2023-09-30T22:21:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 shutdown_connections 2023-09-30T22:21:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78e8096eb0 0x7f78e813b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78e80684f0 0x7f78e813a6a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 --2- 172.21.15.32:0/3730114681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e80965e0 0x7f78e813ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 >> 172.21.15.32:0/3730114681 conn(0x7f78e809a220 msgr2=0x7f78e8094f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 shutdown_connections 2023-09-30T22:21:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.690+0000 7f78ee2a7700 1 -- 172.21.15.32:0/3730114681 wait complete. 2023-09-30T22:21:44.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2023-09-30T22:21:44.705 DEBUG:teuthology.parallel:result is None 2023-09-30T22:21:44.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.710+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efdb8115180 con 0x7efdb8123a90 2023-09-30T22:21:44.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.710+0000 7efd9ffff700 1 -- 172.21.15.32:0/2815213699 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efdb0011370 con 0x7efdb8123a90 2023-09-30T22:21:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 >> v1:172.21.15.32:6816/118188 conn(0x7efda402bbb0 legacy=0x7efda402e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 msgr2=0x7efdb8112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8112140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7efdb000b770 tx=0x7efdb8069f00).stop 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 shutdown_connections 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdb8123a90 0x7efdb8112140 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb8067ce0 0x7efdb8111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 --2- 172.21.15.32:0/2815213699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdb8092340 0x7efdb8111c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 >> 172.21.15.32:0/2815213699 conn(0x7efdb809a220 msgr2=0x7efdb8066b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 shutdown_connections 2023-09-30T22:21:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.712+0000 7efd9dffb700 1 -- 172.21.15.32:0/2815213699 wait complete. 2023-09-30T22:21:44.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.3 2023-09-30T22:21:44.726 DEBUG:teuthology.parallel:result is None 2023-09-30T22:21:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.726+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f496c046c20 con 0x7f496c096380 2023-09-30T22:21:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.727+0000 7f495a7fc700 1 -- 172.21.15.32:0/415728750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4960029020 con 0x7f496c096380 2023-09-30T22:21:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 >> v1:172.21.15.32:6816/118188 conn(0x7f494c02bb90 legacy=0x7f494c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 msgr2=0x7f496c138850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 0x7f496c138850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4960009e00 tx=0x7f4960017000).stop 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 shutdown_connections 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f496c12ce60 0x7f496c138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f496c091b00 0x7f496c138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 --2- 172.21.15.32:0/415728750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f496c096380 0x7f496c138850 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 >> 172.21.15.32:0/415728750 conn(0x7f496c097470 msgr2=0x7f496c099900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 shutdown_connections 2023-09-30T22:21:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.729+0000 7f4970e6e700 1 -- 172.21.15.32:0/415728750 wait complete. 2023-09-30T22:21:44.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.1 2023-09-30T22:21:44.743 DEBUG:teuthology.parallel:result is None 2023-09-30T22:21:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.746+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6db0128750 con 0x7f6db0066cd0 2023-09-30T22:21:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.746+0000 7f6da7fff700 1 -- 172.21.15.32:0/157838325 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6da0018310 con 0x7f6db0066cd0 2023-09-30T22:21:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:38654705668 2023-09-30T22:21:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 >> v1:172.21.15.32:6816/118188 conn(0x7f6d9002bb60 legacy=0x7f6d9002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 msgr2=0x7f6db0112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 0x7f6db0112630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6da000c880 tx=0x7f6da0008040).stop 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 shutdown_connections 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db012f6e0 0x7f6db012d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6db0066cd0 0x7f6db0112630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 --2- 172.21.15.32:0/157838325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6db00675a0 0x7f6db012d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 >> 172.21.15.32:0/157838325 conn(0x7f6db009a220 msgr2=0x7f6db0098150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 shutdown_connections 2023-09-30T22:21:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.748+0000 7f6db6569700 1 -- 172.21.15.32:0/157838325 wait complete. 2023-09-30T22:21:44.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2023-09-30T22:21:44.762 DEBUG:teuthology.parallel:result is None 2023-09-30T22:21:44.762 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:21:44.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:21:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.935+0000 7f7a14f74700 1 Processor -- start 2023-09-30T22:21:44.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.935+0000 7f7a14f74700 1 -- start start 2023-09-30T22:21:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a14f74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10097310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a14f74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a10097800 0x7f7a10130280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a14f74700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a10068df0 con 0x7f7a10096820 2023-09-30T22:21:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a14f74700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1006a4c0 con 0x7f7a10097800 2023-09-30T22:21:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a14f74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10098260 con 0x7f7a10067940 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10097310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a10097800 0x7f7a10130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7a10096820 0x7f7a10096c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54814/0 (socket says 172.21.15.32:54814) 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0e59c700 1 -- 172.21.15.32:0/2317349534 learned_addr learned my addr 172.21.15.32:0/2317349534 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0ed9d700 1 -- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 msgr2=0x7f7a10097310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0ed9d700 1 --2- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10097310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0ed9d700 1 -- 172.21.15.32:0/2317349534 >> v1:172.21.15.32:6789/0 conn(0x7f7a10096820 legacy=0x7f7a10096c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0ed9d700 1 -- 172.21.15.32:0/2317349534 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a10097c70 con 0x7f7a10097800 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0d59a700 1 -- 172.21.15.32:0/2317349534 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115762912 0 0) 0x7f7a10068df0 con 0x7f7a10096820 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0dd9b700 1 --2- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10097310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.936+0000 7f7a0ed9d700 1 --2- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10097800 0x7f7a10130280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7a08009e60 tx=0x7f7a08009780).ready entity=mon.0 client_cookie=c2420c1df34df0c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a0d59a700 1 -- 172.21.15.32:0/2317349534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0800e070 con 0x7f7a10097800 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a0d59a700 1 -- 172.21.15.32:0/2317349534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0800bb60 con 0x7f7a10097800 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a0d59a700 1 -- 172.21.15.32:0/2317349534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08012410 con 0x7f7a10097800 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 -- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10097800 msgr2=0x7f7a10130280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 --2- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10097800 0x7f7a10130280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7a08009e60 tx=0x7f7a08009780).stop 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 -- 172.21.15.32:0/2317349534 shutdown_connections 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 --2- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10097800 0x7f7a10130280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 --2- 172.21.15.32:0/2317349534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10097310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 -- 172.21.15.32:0/2317349534 >> 172.21.15.32:0/2317349534 conn(0x7f7a1009ba50 msgr2=0x7f7a1009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 -- 172.21.15.32:0/2317349534 shutdown_connections 2023-09-30T22:21:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.937+0000 7f7a14f74700 1 -- 172.21.15.32:0/2317349534 wait complete. 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.938+0000 7f7a14f74700 1 Processor -- start 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.938+0000 7f7a14f74700 1 -- start start 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a14f74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10117fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a14f74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 0x7f7a10118490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a14f74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10097800 0x7f7a10118980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a14f74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a101300e0 con 0x7f7a10096820 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a14f74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1011a5d0 con 0x7f7a10097800 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a14f74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10067e60 con 0x7f7a10067940 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10117fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10097800 0x7f7a10118980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 0x7f7a10118490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10097800 0x7f7a10118980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41066/0 (socket says 172.21.15.32:41066) 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 0x7f7a10118490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54694/0 (socket says 172.21.15.32:54694) 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0ed9d700 1 -- 172.21.15.32:0/1643618406 learned_addr learned my addr 172.21.15.32:0/1643618406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10117fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35706/0 (socket says 172.21.15.32:35706) 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 -- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10097800 msgr2=0x7f7a10118980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10097800 0x7f7a10118980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 -- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 msgr2=0x7f7a10118490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 0x7f7a10118490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 -- 172.21.15.32:0/1643618406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80092a0 con 0x7f7a10067940 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0dd9b700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 0x7f7a10118490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a0e59c700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10117fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7a00000c00 tx=0x7f7a00007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:44.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a000070d0 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00007250 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.939+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00007610 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.940+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a08009a20 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.940+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1011c120 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.940+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7a000059b0 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.940+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7a0003a570 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.941+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1005a260 con 0x7f7a10067940 2023-09-30T22:21:44.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:44.945+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7a00013970 con 0x7f7a10067940 2023-09-30T22:21:45.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.115+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 --> v1:172.21.15.32:6816/118188 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a10046c20 con 0x7f79f002bb90 2023-09-30T22:21:45.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.122+0000 7f7a067fc700 1 -- 172.21.15.32:0/1643618406 <== mgr.4101 v1:172.21.15.32:6816/118188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249961 (unknown 2965378022 0 2536888045) 0x7f7a10046c20 con 0x7f79f002bb90 2023-09-30T22:21:45.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 >> v1:172.21.15.32:6816/118188 conn(0x7f79f002bb90 legacy=0x7f79f002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 msgr2=0x7f7a10117fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10117fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7a00000c00 tx=0x7f7a00007b90).stop 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 shutdown_connections 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10097800 0x7f7a10118980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a10067940 0x7f7a10117fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 --2- 172.21.15.32:0/1643618406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10096820 0x7f7a10118490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 >> 172.21.15.32:0/1643618406 conn(0x7f7a1009ba50 msgr2=0x7f7a10099880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 shutdown_connections 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.125+0000 7f7a14f74700 1 -- 172.21.15.32:0/1643618406 wait complete. 2023-09-30T22:21:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T22:21:45.139 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-09-30T22:21:43.535081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000511"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221356+0000","last_change":"2023-09-30T22:21:37.213744+0000","last_active":"2023-09-30T22:21:37.221356+0000","last_peered":"2023-09-30T22:21:37.221356+0000","last_clean":"2023-09-30T22:21:37.221356+0000","last_became_active":"2023-09-30T22:21:37.213432+0000","last_became_peered":"2023-09-30T22:21:37.213432+0000","last_unstale":"2023-09-30T22:21:37.221356+0000","last_undegraded":"2023-09-30T22:21:37.221356+0000","last_fullsized":"2023-09-30T22:21:37.221356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221210+0000","last_change":"2023-09-30T22:21:37.209966+0000","last_active":"2023-09-30T22:21:37.221210+0000","last_peered":"2023-09-30T22:21:37.221210+0000","last_clean":"2023-09-30T22:21:37.221210+0000","last_became_active":"2023-09-30T22:21:37.209700+0000","last_became_peered":"2023-09-30T22:21:37.209700+0000","last_unstale":"2023-09-30T22:21:37.221210+0000","last_undegraded":"2023-09-30T22:21:37.221210+0000","last_fullsized":"2023-09-30T22:21:37.221210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233883+0000","last_change":"2023-09-30T22:21:37.227972+0000","last_active":"2023-09-30T22:21:37.233883+0000","last_peered":"2023-09-30T22:21:37.233883+0000","last_clean":"2023-09-30T22:21:37.233883+0000","last_became_active":"2023-09-30T22:21:37.227698+0000","last_became_peered":"2023-09-30T22:21:37.227698+0000","last_unstale":"2023-09-30T22:21:37.233883+0000","last_undegraded":"2023-09-30T22:21:37.233883+0000","last_fullsized":"2023-09-30T22:21:37.233883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.218248+0000","last_change":"2023-09-30T22:21:37.217682+0000","last_active":"2023-09-30T22:21:37.218248+0000","last_peered":"2023-09-30T22:21:37.218248+0000","last_clean":"2023-09-30T22:21:37.218248+0000","last_became_active":"2023-09-30T22:21:37.217395+0000","last_became_peered":"2023-09-30T22:21:37.217395+0000","last_unstale":"2023-09-30T22:21:37.218248+0000","last_undegraded":"2023-09-30T22:21:37.218248+0000","last_fullsized":"2023-09-30T22:21:37.218248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234071+0000","last_change":"2023-09-30T22:21:37.229907+0000","last_active":"2023-09-30T22:21:37.234071+0000","last_peered":"2023-09-30T22:21:37.234071+0000","last_clean":"2023-09-30T22:21:37.234071+0000","last_became_active":"2023-09-30T22:21:37.229717+0000","last_became_peered":"2023-09-30T22:21:37.229717+0000","last_unstale":"2023-09-30T22:21:37.234071+0000","last_undegraded":"2023-09-30T22:21:37.234071+0000","last_fullsized":"2023-09-30T22:21:37.234071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233648+0000","last_change":"2023-09-30T22:21:37.224923+0000","last_active":"2023-09-30T22:21:37.233648+0000","last_peered":"2023-09-30T22:21:37.233648+0000","last_clean":"2023-09-30T22:21:37.233648+0000","last_became_active":"2023-09-30T22:21:37.224651+0000","last_became_peered":"2023-09-30T22:21:37.224651+0000","last_unstale":"2023-09-30T22:21:37.233648+0000","last_undegraded":"2023-09-30T22:21:37.233648+0000","last_fullsized":"2023-09-30T22:21:37.233648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225058+0000","last_change":"2023-09-30T22:21:37.220818+0000","last_active":"2023-09-30T22:21:37.225058+0000","last_peered":"2023-09-30T22:21:37.225058+0000","last_clean":"2023-09-30T22:21:37.225058+0000","last_became_active":"2023-09-30T22:21:37.217825+0000","last_became_peered":"2023-09-30T22:21:37.217825+0000","last_unstale":"2023-09-30T22:21:37.225058+0000","last_undegraded":"2023-09-30T22:21:37.225058+0000","last_fullsized":"2023-09-30T22:21:37.225058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239537+0000","last_change":"2023-09-30T22:21:37.237512+0000","last_active":"2023-09-30T22:21:37.239537+0000","last_peered":"2023-09-30T22:21:37.239537+0000","last_clean":"2023-09-30T22:21:37.239537+0000","last_became_active":"2023-09-30T22:21:37.237217+0000","last_became_peered":"2023-09-30T22:21:37.237217+0000","last_unstale":"2023-09-30T22:21:37.239537+0000","last_undegraded":"2023-09-30T22:21:37.239537+0000","last_fullsized":"2023-09-30T22:21:37.239537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234278+0000","last_change":"2023-09-30T22:21:37.231447+0000","last_active":"2023-09-30T22:21:37.234278+0000","last_peered":"2023-09-30T22:21:37.234278+0000","last_clean":"2023-09-30T22:21:37.234278+0000","last_became_active":"2023-09-30T22:21:37.231106+0000","last_became_peered":"2023-09-30T22:21:37.231106+0000","last_unstale":"2023-09-30T22:21:37.234278+0000","last_undegraded":"2023-09-30T22:21:37.234278+0000","last_fullsized":"2023-09-30T22:21:37.234278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239686+0000","last_change":"2023-09-30T22:21:37.238710+0000","last_active":"2023-09-30T22:21:37.239686+0000","last_peered":"2023-09-30T22:21:37.239686+0000","last_clean":"2023-09-30T22:21:37.239686+0000","last_became_active":"2023-09-30T22:21:37.237230+0000","last_became_peered":"2023-09-30T22:21:37.237230+0000","last_unstale":"2023-09-30T22:21:37.239686+0000","last_undegraded":"2023-09-30T22:21:37.239686+0000","last_fullsized":"2023-09-30T22:21:37.239686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239518+0000","last_change":"2023-09-30T22:21:37.238711+0000","last_active":"2023-09-30T22:21:37.239518+0000","last_peered":"2023-09-30T22:21:37.239518+0000","last_clean":"2023-09-30T22:21:37.239518+0000","last_became_active":"2023-09-30T22:21:37.237399+0000","last_became_peered":"2023-09-30T22:21:37.237399+0000","last_unstale":"2023-09-30T22:21:37.239518+0000","last_undegraded":"2023-09-30T22:21:37.239518+0000","last_fullsized":"2023-09-30T22:21:37.239518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223819+0000","last_change":"2023-09-30T22:21:37.215868+0000","last_active":"2023-09-30T22:21:37.223819+0000","last_peered":"2023-09-30T22:21:37.223819+0000","last_clean":"2023-09-30T22:21:37.223819+0000","last_became_active":"2023-09-30T22:21:37.215601+0000","last_became_peered":"2023-09-30T22:21:37.215601+0000","last_unstale":"2023-09-30T22:21:37.223819+0000","last_undegraded":"2023-09-30T22:21:37.223819+0000","last_fullsized":"2023-09-30T22:21:37.223819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232299+0000","last_change":"2023-09-30T22:21:37.229175+0000","last_active":"2023-09-30T22:21:37.232299+0000","last_peered":"2023-09-30T22:21:37.232299+0000","last_clean":"2023-09-30T22:21:37.232299+0000","last_became_active":"2023-09-30T22:21:37.226712+0000","last_became_peered":"2023-09-30T22:21:37.226712+0000","last_unstale":"2023-09-30T22:21:37.232299+0000","last_undegraded":"2023-09-30T22:21:37.232299+0000","last_fullsized":"2023-09-30T22:21:37.232299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225658+0000","last_change":"2023-09-30T22:21:37.222596+0000","last_active":"2023-09-30T22:21:37.225658+0000","last_peered":"2023-09-30T22:21:37.225658+0000","last_clean":"2023-09-30T22:21:37.225658+0000","last_became_active":"2023-09-30T22:21:37.222295+0000","last_became_peered":"2023-09-30T22:21:37.222295+0000","last_unstale":"2023-09-30T22:21:37.225658+0000","last_undegraded":"2023-09-30T22:21:37.225658+0000","last_fullsized":"2023-09-30T22:21:37.225658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.218011+0000","last_change":"2023-09-30T22:21:37.215505+0000","last_active":"2023-09-30T22:21:37.218011+0000","last_peered":"2023-09-30T22:21:37.218011+0000","last_clean":"2023-09-30T22:21:37.218011+0000","last_became_active":"2023-09-30T22:21:37.215015+0000","last_became_peered":"2023-09-30T22:21:37.215015+0000","last_unstale":"2023-09-30T22:21:37.218011+0000","last_undegraded":"2023-09-30T22:21:37.218011+0000","last_fullsized":"2023-09-30T22:21:37.218011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.224429+0000","last_change":"2023-09-30T22:21:37.220367+0000","last_active":"2023-09-30T22:21:37.224429+0000","last_peered":"2023-09-30T22:21:37.224429+0000","last_clean":"2023-09-30T22:21:37.224429+0000","last_became_active":"2023-09-30T22:21:37.217445+0000","last_became_peered":"2023-09-30T22:21:37.217445+0000","last_unstale":"2023-09-30T22:21:37.224429+0000","last_undegraded":"2023-09-30T22:21:37.224429+0000","last_fullsized":"2023-09-30T22:21:37.224429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234262+0000","last_change":"2023-09-30T22:21:37.231390+0000","last_active":"2023-09-30T22:21:37.234262+0000","last_peered":"2023-09-30T22:21:37.234262+0000","last_clean":"2023-09-30T22:21:37.234262+0000","last_became_active":"2023-09-30T22:21:37.231116+0000","last_became_peered":"2023-09-30T22:21:37.231116+0000","last_unstale":"2023-09-30T22:21:37.234262+0000","last_undegraded":"2023-09-30T22:21:37.234262+0000","last_fullsized":"2023-09-30T22:21:37.234262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234120+0000","last_change":"2023-09-30T22:21:37.228923+0000","last_active":"2023-09-30T22:21:37.234120+0000","last_peered":"2023-09-30T22:21:37.234120+0000","last_clean":"2023-09-30T22:21:37.234120+0000","last_became_active":"2023-09-30T22:21:37.220140+0000","last_became_peered":"2023-09-30T22:21:37.220140+0000","last_unstale":"2023-09-30T22:21:37.234120+0000","last_undegraded":"2023-09-30T22:21:37.234120+0000","last_fullsized":"2023-09-30T22:21:37.234120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.228910+0000","last_change":"2023-09-30T22:21:37.225422+0000","last_active":"2023-09-30T22:21:37.228910+0000","last_peered":"2023-09-30T22:21:37.228910+0000","last_clean":"2023-09-30T22:21:37.228910+0000","last_became_active":"2023-09-30T22:21:37.225118+0000","last_became_peered":"2023-09-30T22:21:37.225118+0000","last_unstale":"2023-09-30T22:21:37.228910+0000","last_undegraded":"2023-09-30T22:21:37.228910+0000","last_fullsized":"2023-09-30T22:21:37.228910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220876+0000","last_change":"2023-09-30T22:21:37.218484+0000","last_active":"2023-09-30T22:21:37.220876+0000","last_peered":"2023-09-30T22:21:37.220876+0000","last_clean":"2023-09-30T22:21:37.220876+0000","last_became_active":"2023-09-30T22:21:37.218141+0000","last_became_peered":"2023-09-30T22:21:37.218141+0000","last_unstale":"2023-09-30T22:21:37.220876+0000","last_undegraded":"2023-09-30T22:21:37.220876+0000","last_fullsized":"2023-09-30T22:21:37.220876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.219680+0000","last_change":"2023-09-30T22:21:37.216211+0000","last_active":"2023-09-30T22:21:37.219680+0000","last_peered":"2023-09-30T22:21:37.219680+0000","last_clean":"2023-09-30T22:21:37.219680+0000","last_became_active":"2023-09-30T22:21:37.215922+0000","last_became_peered":"2023-09-30T22:21:37.215922+0000","last_unstale":"2023-09-30T22:21:37.219680+0000","last_undegraded":"2023-09-30T22:21:37.219680+0000","last_fullsized":"2023-09-30T22:21:37.219680+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233722+0000","last_change":"2023-09-30T22:21:37.227137+0000","last_active":"2023-09-30T22:21:37.233722+0000","last_peered":"2023-09-30T22:21:37.233722+0000","last_clean":"2023-09-30T22:21:37.233722+0000","last_became_active":"2023-09-30T22:21:37.226848+0000","last_became_peered":"2023-09-30T22:21:37.226848+0000","last_unstale":"2023-09-30T22:21:37.233722+0000","last_undegraded":"2023-09-30T22:21:37.233722+0000","last_fullsized":"2023-09-30T22:21:37.233722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225020+0000","last_change":"2023-09-30T22:21:37.220415+0000","last_active":"2023-09-30T22:21:37.225020+0000","last_peered":"2023-09-30T22:21:37.225020+0000","last_clean":"2023-09-30T22:21:37.225020+0000","last_became_active":"2023-09-30T22:21:37.217468+0000","last_became_peered":"2023-09-30T22:21:37.217468+0000","last_unstale":"2023-09-30T22:21:37.225020+0000","last_undegraded":"2023-09-30T22:21:37.225020+0000","last_fullsized":"2023-09-30T22:21:37.225020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221521+0000","last_change":"2023-09-30T22:21:37.220204+0000","last_active":"2023-09-30T22:21:37.221521+0000","last_peered":"2023-09-30T22:21:37.221521+0000","last_clean":"2023-09-30T22:21:37.221521+0000","last_became_active":"2023-09-30T22:21:37.219099+0000","last_became_peered":"2023-09-30T22:21:37.219099+0000","last_unstale":"2023-09-30T22:21:37.221521+0000","last_undegraded":"2023-09-30T22:21:37.221521+0000","last_fullsized":"2023-09-30T22:21:37.221521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220336+0000","last_change":"2023-09-30T22:21:37.217211+0000","last_active":"2023-09-30T22:21:37.220336+0000","last_peered":"2023-09-30T22:21:37.220336+0000","last_clean":"2023-09-30T22:21:37.220336+0000","last_became_active":"2023-09-30T22:21:37.216537+0000","last_became_peered":"2023-09-30T22:21:37.216537+0000","last_unstale":"2023-09-30T22:21:37.220336+0000","last_undegraded":"2023-09-30T22:21:37.220336+0000","last_fullsized":"2023-09-30T22:21:37.220336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.213682+0000","last_change":"2023-09-30T22:21:37.209836+0000","last_active":"2023-09-30T22:21:37.213682+0000","last_peered":"2023-09-30T22:21:37.213682+0000","last_clean":"2023-09-30T22:21:37.213682+0000","last_became_active":"2023-09-30T22:21:37.209561+0000","last_became_peered":"2023-09-30T22:21:37.209561+0000","last_unstale":"2023-09-30T22:21:37.213682+0000","last_undegraded":"2023-09-30T22:21:37.213682+0000","last_fullsized":"2023-09-30T22:21:37.213682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226409+0000","last_change":"2023-09-30T22:21:37.220637+0000","last_active":"2023-09-30T22:21:37.226409+0000","last_peered":"2023-09-30T22:21:37.226409+0000","last_clean":"2023-09-30T22:21:37.226409+0000","last_became_active":"2023-09-30T22:21:37.217741+0000","last_became_peered":"2023-09-30T22:21:37.217741+0000","last_unstale":"2023-09-30T22:21:37.226409+0000","last_undegraded":"2023-09-30T22:21:37.226409+0000","last_fullsized":"2023-09-30T22:21:37.226409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.222723+0000","last_change":"2023-09-30T22:21:36.800514+0000","last_active":"2023-09-30T22:21:37.222723+0000","last_peered":"2023-09-30T22:21:37.222723+0000","last_clean":"2023-09-30T22:21:37.222723+0000","last_became_active":"2023-09-30T22:21:36.799775+0000","last_became_peered":"2023-09-30T22:21:36.799775+0000","last_unstale":"2023-09-30T22:21:37.222723+0000","last_undegraded":"2023-09-30T22:21:37.222723+0000","last_fullsized":"2023-09-30T22:21:37.222723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.224088+0000","last_change":"2023-09-30T22:21:37.211849+0000","last_active":"2023-09-30T22:21:37.224088+0000","last_peered":"2023-09-30T22:21:37.224088+0000","last_clean":"2023-09-30T22:21:37.224088+0000","last_became_active":"2023-09-30T22:21:37.209128+0000","last_became_peered":"2023-09-30T22:21:37.209128+0000","last_unstale":"2023-09-30T22:21:37.224088+0000","last_undegraded":"2023-09-30T22:21:37.224088+0000","last_fullsized":"2023-09-30T22:21:37.224088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198669+0000","last_change":"2023-09-30T22:21:36.803467+0000","last_active":"2023-09-30T22:21:37.198669+0000","last_peered":"2023-09-30T22:21:37.198669+0000","last_clean":"2023-09-30T22:21:37.198669+0000","last_became_active":"2023-09-30T22:21:36.803141+0000","last_became_peered":"2023-09-30T22:21:36.803141+0000","last_unstale":"2023-09-30T22:21:37.198669+0000","last_undegraded":"2023-09-30T22:21:37.198669+0000","last_fullsized":"2023-09-30T22:21:37.198669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233388+0000","last_change":"2023-09-30T22:21:37.224566+0000","last_active":"2023-09-30T22:21:37.233388+0000","last_peered":"2023-09-30T22:21:37.233388+0000","last_clean":"2023-09-30T22:21:37.233388+0000","last_became_active":"2023-09-30T22:21:37.219843+0000","last_became_peered":"2023-09-30T22:21:37.219843+0000","last_unstale":"2023-09-30T22:21:37.233388+0000","last_undegraded":"2023-09-30T22:21:37.233388+0000","last_fullsized":"2023-09-30T22:21:37.233388+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195923+0000","last_change":"2023-09-30T22:21:36.813865+0000","last_active":"2023-09-30T22:21:37.195923+0000","last_peered":"2023-09-30T22:21:37.195923+0000","last_clean":"2023-09-30T22:21:37.195923+0000","last_became_active":"2023-09-30T22:21:36.812871+0000","last_became_peered":"2023-09-30T22:21:36.812871+0000","last_unstale":"2023-09-30T22:21:37.195923+0000","last_undegraded":"2023-09-30T22:21:37.195923+0000","last_fullsized":"2023-09-30T22:21:37.195923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226177+0000","last_change":"2023-09-30T22:21:37.215838+0000","last_active":"2023-09-30T22:21:37.226177+0000","last_peered":"2023-09-30T22:21:37.226177+0000","last_clean":"2023-09-30T22:21:37.226177+0000","last_became_active":"2023-09-30T22:21:37.215314+0000","last_became_peered":"2023-09-30T22:21:37.215314+0000","last_unstale":"2023-09-30T22:21:37.226177+0000","last_undegraded":"2023-09-30T22:21:37.226177+0000","last_fullsized":"2023-09-30T22:21:37.226177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204605+0000","last_change":"2023-09-30T22:21:36.809280+0000","last_active":"2023-09-30T22:21:37.204605+0000","last_peered":"2023-09-30T22:21:37.204605+0000","last_clean":"2023-09-30T22:21:37.204605+0000","last_became_active":"2023-09-30T22:21:36.808975+0000","last_became_peered":"2023-09-30T22:21:36.808975+0000","last_unstale":"2023-09-30T22:21:37.204605+0000","last_undegraded":"2023-09-30T22:21:37.204605+0000","last_fullsized":"2023-09-30T22:21:37.204605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223892+0000","last_change":"2023-09-30T22:21:37.215010+0000","last_active":"2023-09-30T22:21:37.223892+0000","last_peered":"2023-09-30T22:21:37.223892+0000","last_clean":"2023-09-30T22:21:37.223892+0000","last_became_active":"2023-09-30T22:21:37.212913+0000","last_became_peered":"2023-09-30T22:21:37.212913+0000","last_unstale":"2023-09-30T22:21:37.223892+0000","last_undegraded":"2023-09-30T22:21:37.223892+0000","last_fullsized":"2023-09-30T22:21:37.223892+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.201567+0000","last_change":"2023-09-30T22:21:36.800431+0000","last_active":"2023-09-30T22:21:37.201567+0000","last_peered":"2023-09-30T22:21:37.201567+0000","last_clean":"2023-09-30T22:21:37.201567+0000","last_became_active":"2023-09-30T22:21:36.800130+0000","last_became_peered":"2023-09-30T22:21:36.800130+0000","last_unstale":"2023-09-30T22:21:37.201567+0000","last_undegraded":"2023-09-30T22:21:37.201567+0000","last_fullsized":"2023-09-30T22:21:37.201567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225077+0000","last_change":"2023-09-30T22:21:37.220978+0000","last_active":"2023-09-30T22:21:37.225077+0000","last_peered":"2023-09-30T22:21:37.225077+0000","last_clean":"2023-09-30T22:21:37.225077+0000","last_became_active":"2023-09-30T22:21:37.218384+0000","last_became_peered":"2023-09-30T22:21:37.218384+0000","last_unstale":"2023-09-30T22:21:37.225077+0000","last_undegraded":"2023-09-30T22:21:37.225077+0000","last_fullsized":"2023-09-30T22:21:37.225077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.231706+0000","last_change":"2023-09-30T22:21:36.810931+0000","last_active":"2023-09-30T22:21:37.231706+0000","last_peered":"2023-09-30T22:21:37.231706+0000","last_clean":"2023-09-30T22:21:37.231706+0000","last_became_active":"2023-09-30T22:21:36.810625+0000","last_became_peered":"2023-09-30T22:21:36.810625+0000","last_unstale":"2023-09-30T22:21:37.231706+0000","last_undegraded":"2023-09-30T22:21:37.231706+0000","last_fullsized":"2023-09-30T22:21:37.231706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220744+0000","last_change":"2023-09-30T22:21:37.217701+0000","last_active":"2023-09-30T22:21:37.220744+0000","last_peered":"2023-09-30T22:21:37.220744+0000","last_clean":"2023-09-30T22:21:37.220744+0000","last_became_active":"2023-09-30T22:21:37.217076+0000","last_became_peered":"2023-09-30T22:21:37.217076+0000","last_unstale":"2023-09-30T22:21:37.220744+0000","last_undegraded":"2023-09-30T22:21:37.220744+0000","last_fullsized":"2023-09-30T22:21:37.220744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.236574+0000","last_change":"2023-09-30T22:21:36.811297+0000","last_active":"2023-09-30T22:21:37.236574+0000","last_peered":"2023-09-30T22:21:37.236574+0000","last_clean":"2023-09-30T22:21:37.236574+0000","last_became_active":"2023-09-30T22:21:36.810987+0000","last_became_peered":"2023-09-30T22:21:36.810987+0000","last_unstale":"2023-09-30T22:21:37.236574+0000","last_undegraded":"2023-09-30T22:21:37.236574+0000","last_fullsized":"2023-09-30T22:21:37.236574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232281+0000","last_change":"2023-09-30T22:21:37.228789+0000","last_active":"2023-09-30T22:21:37.232281+0000","last_peered":"2023-09-30T22:21:37.232281+0000","last_clean":"2023-09-30T22:21:37.232281+0000","last_became_active":"2023-09-30T22:21:37.225220+0000","last_became_peered":"2023-09-30T22:21:37.225220+0000","last_unstale":"2023-09-30T22:21:37.232281+0000","last_undegraded":"2023-09-30T22:21:37.232281+0000","last_fullsized":"2023-09-30T22:21:37.232281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235370+0000","last_change":"2023-09-30T22:21:36.813567+0000","last_active":"2023-09-30T22:21:37.235370+0000","last_peered":"2023-09-30T22:21:37.235370+0000","last_clean":"2023-09-30T22:21:37.235370+0000","last_became_active":"2023-09-30T22:21:36.813257+0000","last_became_peered":"2023-09-30T22:21:36.813257+0000","last_unstale":"2023-09-30T22:21:37.235370+0000","last_undegraded":"2023-09-30T22:21:37.235370+0000","last_fullsized":"2023-09-30T22:21:37.235370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223168+0000","last_change":"2023-09-30T22:21:37.213340+0000","last_active":"2023-09-30T22:21:37.223168+0000","last_peered":"2023-09-30T22:21:37.223168+0000","last_clean":"2023-09-30T22:21:37.223168+0000","last_became_active":"2023-09-30T22:21:37.213040+0000","last_became_peered":"2023-09-30T22:21:37.213040+0000","last_unstale":"2023-09-30T22:21:37.223168+0000","last_undegraded":"2023-09-30T22:21:37.223168+0000","last_fullsized":"2023-09-30T22:21:37.223168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.205101+0000","last_change":"2023-09-30T22:21:36.806015+0000","last_active":"2023-09-30T22:21:37.205101+0000","last_peered":"2023-09-30T22:21:37.205101+0000","last_clean":"2023-09-30T22:21:37.205101+0000","last_became_active":"2023-09-30T22:21:36.805679+0000","last_became_peered":"2023-09-30T22:21:36.805679+0000","last_unstale":"2023-09-30T22:21:37.205101+0000","last_undegraded":"2023-09-30T22:21:37.205101+0000","last_fullsized":"2023-09-30T22:21:37.205101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226450+0000","last_change":"2023-09-30T22:21:37.221673+0000","last_active":"2023-09-30T22:21:37.226450+0000","last_peered":"2023-09-30T22:21:37.226450+0000","last_clean":"2023-09-30T22:21:37.226450+0000","last_became_active":"2023-09-30T22:21:37.221388+0000","last_became_peered":"2023-09-30T22:21:37.221388+0000","last_unstale":"2023-09-30T22:21:37.226450+0000","last_undegraded":"2023-09-30T22:21:37.226450+0000","last_fullsized":"2023-09-30T22:21:37.226450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199788+0000","last_change":"2023-09-30T22:21:36.812303+0000","last_active":"2023-09-30T22:21:37.199788+0000","last_peered":"2023-09-30T22:21:37.199788+0000","last_clean":"2023-09-30T22:21:37.199788+0000","last_became_active":"2023-09-30T22:21:36.811887+0000","last_became_peered":"2023-09-30T22:21:36.811887+0000","last_unstale":"2023-09-30T22:21:37.199788+0000","last_undegraded":"2023-09-30T22:21:37.199788+0000","last_fullsized":"2023-09-30T22:21:37.199788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234099+0000","last_change":"2023-09-30T22:21:37.225272+0000","last_active":"2023-09-30T22:21:37.234099+0000","last_peered":"2023-09-30T22:21:37.234099+0000","last_clean":"2023-09-30T22:21:37.234099+0000","last_became_active":"2023-09-30T22:21:37.224827+0000","last_became_peered":"2023-09-30T22:21:37.224827+0000","last_unstale":"2023-09-30T22:21:37.234099+0000","last_undegraded":"2023-09-30T22:21:37.234099+0000","last_fullsized":"2023-09-30T22:21:37.234099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239712+0000","last_change":"2023-09-30T22:21:36.804342+0000","last_active":"2023-09-30T22:21:37.239712+0000","last_peered":"2023-09-30T22:21:37.239712+0000","last_clean":"2023-09-30T22:21:37.239712+0000","last_became_active":"2023-09-30T22:21:36.804002+0000","last_became_peered":"2023-09-30T22:21:36.804002+0000","last_unstale":"2023-09-30T22:21:37.239712+0000","last_undegraded":"2023-09-30T22:21:37.239712+0000","last_fullsized":"2023-09-30T22:21:37.239712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226154+0000","last_change":"2023-09-30T22:21:37.215043+0000","last_active":"2023-09-30T22:21:37.226154+0000","last_peered":"2023-09-30T22:21:37.226154+0000","last_clean":"2023-09-30T22:21:37.226154+0000","last_became_active":"2023-09-30T22:21:37.214750+0000","last_became_peered":"2023-09-30T22:21:37.214750+0000","last_unstale":"2023-09-30T22:21:37.226154+0000","last_undegraded":"2023-09-30T22:21:37.226154+0000","last_fullsized":"2023-09-30T22:21:37.226154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239337+0000","last_change":"2023-09-30T22:21:36.796103+0000","last_active":"2023-09-30T22:21:37.239337+0000","last_peered":"2023-09-30T22:21:37.239337+0000","last_clean":"2023-09-30T22:21:37.239337+0000","last_became_active":"2023-09-30T22:21:36.795669+0000","last_became_peered":"2023-09-30T22:21:36.795669+0000","last_unstale":"2023-09-30T22:21:37.239337+0000","last_undegraded":"2023-09-30T22:21:37.239337+0000","last_fullsized":"2023-09-30T22:21:37.239337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226133+0000","last_change":"2023-09-30T22:21:37.223782+0000","last_active":"2023-09-30T22:21:37.226133+0000","last_peered":"2023-09-30T22:21:37.226133+0000","last_clean":"2023-09-30T22:21:37.226133+0000","last_became_active":"2023-09-30T22:21:37.223508+0000","last_became_peered":"2023-09-30T22:21:37.223508+0000","last_unstale":"2023-09-30T22:21:37.226133+0000","last_undegraded":"2023-09-30T22:21:37.226133+0000","last_fullsized":"2023-09-30T22:21:37.226133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.230042+0000","last_change":"2023-09-30T22:21:36.798241+0000","last_active":"2023-09-30T22:21:37.230042+0000","last_peered":"2023-09-30T22:21:37.230042+0000","last_clean":"2023-09-30T22:21:37.230042+0000","last_became_active":"2023-09-30T22:21:36.797923+0000","last_became_peered":"2023-09-30T22:21:36.797923+0000","last_unstale":"2023-09-30T22:21:37.230042+0000","last_undegraded":"2023-09-30T22:21:37.230042+0000","last_fullsized":"2023-09-30T22:21:37.230042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225577+0000","last_change":"2023-09-30T22:21:37.223841+0000","last_active":"2023-09-30T22:21:37.225577+0000","last_peered":"2023-09-30T22:21:37.225577+0000","last_clean":"2023-09-30T22:21:37.225577+0000","last_became_active":"2023-09-30T22:21:37.223485+0000","last_became_peered":"2023-09-30T22:21:37.223485+0000","last_unstale":"2023-09-30T22:21:37.225577+0000","last_undegraded":"2023-09-30T22:21:37.225577+0000","last_fullsized":"2023-09-30T22:21:37.225577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238213+0000","last_change":"2023-09-30T22:21:36.817374+0000","last_active":"2023-09-30T22:21:37.238213+0000","last_peered":"2023-09-30T22:21:37.238213+0000","last_clean":"2023-09-30T22:21:37.238213+0000","last_became_active":"2023-09-30T22:21:36.816913+0000","last_became_peered":"2023-09-30T22:21:36.816913+0000","last_unstale":"2023-09-30T22:21:37.238213+0000","last_undegraded":"2023-09-30T22:21:37.238213+0000","last_fullsized":"2023-09-30T22:21:37.238213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221929+0000","last_change":"2023-09-30T22:21:37.214893+0000","last_active":"2023-09-30T22:21:37.221929+0000","last_peered":"2023-09-30T22:21:37.221929+0000","last_clean":"2023-09-30T22:21:37.221929+0000","last_became_active":"2023-09-30T22:21:37.214616+0000","last_became_peered":"2023-09-30T22:21:37.214616+0000","last_unstale":"2023-09-30T22:21:37.221929+0000","last_undegraded":"2023-09-30T22:21:37.221929+0000","last_fullsized":"2023-09-30T22:21:37.221929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.202352+0000","last_change":"2023-09-30T22:21:36.806594+0000","last_active":"2023-09-30T22:21:37.202352+0000","last_peered":"2023-09-30T22:21:37.202352+0000","last_clean":"2023-09-30T22:21:37.202352+0000","last_became_active":"2023-09-30T22:21:36.806305+0000","last_became_peered":"2023-09-30T22:21:36.806305+0000","last_unstale":"2023-09-30T22:21:37.202352+0000","last_undegraded":"2023-09-30T22:21:37.202352+0000","last_fullsized":"2023-09-30T22:21:37.202352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233473+0000","last_change":"2023-09-30T22:21:37.224756+0000","last_active":"2023-09-30T22:21:37.233473+0000","last_peered":"2023-09-30T22:21:37.233473+0000","last_clean":"2023-09-30T22:21:37.233473+0000","last_became_active":"2023-09-30T22:21:37.224444+0000","last_became_peered":"2023-09-30T22:21:37.224444+0000","last_unstale":"2023-09-30T22:21:37.233473+0000","last_undegraded":"2023-09-30T22:21:37.233473+0000","last_fullsized":"2023-09-30T22:21:37.233473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235343+0000","last_change":"2023-09-30T22:21:36.813533+0000","last_active":"2023-09-30T22:21:37.235343+0000","last_peered":"2023-09-30T22:21:37.235343+0000","last_clean":"2023-09-30T22:21:37.235343+0000","last_became_active":"2023-09-30T22:21:36.813227+0000","last_became_peered":"2023-09-30T22:21:36.813227+0000","last_unstale":"2023-09-30T22:21:37.235343+0000","last_undegraded":"2023-09-30T22:21:37.235343+0000","last_fullsized":"2023-09-30T22:21:37.235343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234223+0000","last_change":"2023-09-30T22:21:37.230482+0000","last_active":"2023-09-30T22:21:37.234223+0000","last_peered":"2023-09-30T22:21:37.234223+0000","last_clean":"2023-09-30T22:21:37.234223+0000","last_became_active":"2023-09-30T22:21:37.229876+0000","last_became_peered":"2023-09-30T22:21:37.229876+0000","last_unstale":"2023-09-30T22:21:37.234223+0000","last_undegraded":"2023-09-30T22:21:37.234223+0000","last_fullsized":"2023-09-30T22:21:37.234223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199874+0000","last_change":"2023-09-30T22:21:36.811138+0000","last_active":"2023-09-30T22:21:37.199874+0000","last_peered":"2023-09-30T22:21:37.199874+0000","last_clean":"2023-09-30T22:21:37.199874+0000","last_became_active":"2023-09-30T22:21:36.810834+0000","last_became_peered":"2023-09-30T22:21:36.810834+0000","last_unstale":"2023-09-30T22:21:37.199874+0000","last_undegraded":"2023-09-30T22:21:37.199874+0000","last_fullsized":"2023-09-30T22:21:37.199874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226390+0000","last_change":"2023-09-30T22:21:37.220428+0000","last_active":"2023-09-30T22:21:37.226390+0000","last_peered":"2023-09-30T22:21:37.226390+0000","last_clean":"2023-09-30T22:21:37.226390+0000","last_became_active":"2023-09-30T22:21:37.217353+0000","last_became_peered":"2023-09-30T22:21:37.217353+0000","last_unstale":"2023-09-30T22:21:37.226390+0000","last_undegraded":"2023-09-30T22:21:37.226390+0000","last_fullsized":"2023-09-30T22:21:37.226390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195804+0000","last_change":"2023-09-30T22:21:36.812156+0000","last_active":"2023-09-30T22:21:37.195804+0000","last_peered":"2023-09-30T22:21:37.195804+0000","last_clean":"2023-09-30T22:21:37.195804+0000","last_became_active":"2023-09-30T22:21:36.811527+0000","last_became_peered":"2023-09-30T22:21:36.811527+0000","last_unstale":"2023-09-30T22:21:37.195804+0000","last_undegraded":"2023-09-30T22:21:37.195804+0000","last_fullsized":"2023-09-30T22:21:37.195804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234586+0000","last_change":"2023-09-30T22:21:37.226310+0000","last_active":"2023-09-30T22:21:37.234586+0000","last_peered":"2023-09-30T22:21:37.234586+0000","last_clean":"2023-09-30T22:21:37.234586+0000","last_became_active":"2023-09-30T22:21:37.225749+0000","last_became_peered":"2023-09-30T22:21:37.225749+0000","last_unstale":"2023-09-30T22:21:37.234586+0000","last_undegraded":"2023-09-30T22:21:37.234586+0000","last_fullsized":"2023-09-30T22:21:37.234586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195360+0000","last_change":"2023-09-30T22:21:36.811756+0000","last_active":"2023-09-30T22:21:37.195360+0000","last_peered":"2023-09-30T22:21:37.195360+0000","last_clean":"2023-09-30T22:21:37.195360+0000","last_became_active":"2023-09-30T22:21:36.811393+0000","last_became_peered":"2023-09-30T22:21:36.811393+0000","last_unstale":"2023-09-30T22:21:37.195360+0000","last_undegraded":"2023-09-30T22:21:37.195360+0000","last_fullsized":"2023-09-30T22:21:37.195360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.219948+0000","last_change":"2023-09-30T22:21:37.217010+0000","last_active":"2023-09-30T22:21:37.219948+0000","last_peered":"2023-09-30T22:21:37.219948+0000","last_clean":"2023-09-30T22:21:37.219948+0000","last_became_active":"2023-09-30T22:21:37.216071+0000","last_became_peered":"2023-09-30T22:21:37.216071+0000","last_unstale":"2023-09-30T22:21:37.219948+0000","last_undegraded":"2023-09-30T22:21:37.219948+0000","last_fullsized":"2023-09-30T22:21:37.219948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199310+0000","last_change":"2023-09-30T22:21:31.794405+0000","last_active":"2023-09-30T22:21:37.199310+0000","last_peered":"2023-09-30T22:21:37.199310+0000","last_clean":"2023-09-30T22:21:37.199310+0000","last_became_active":"2023-09-30T22:21:31.794052+0000","last_became_peered":"2023-09-30T22:21:31.794052+0000","last_unstale":"2023-09-30T22:21:37.199310+0000","last_undegraded":"2023-09-30T22:21:37.199310+0000","last_fullsized":"2023-09-30T22:21:37.199310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197712+0000","last_change":"2023-09-30T22:21:36.799995+0000","last_active":"2023-09-30T22:21:37.197712+0000","last_peered":"2023-09-30T22:21:37.197712+0000","last_clean":"2023-09-30T22:21:37.197712+0000","last_became_active":"2023-09-30T22:21:36.799674+0000","last_became_peered":"2023-09-30T22:21:36.799674+0000","last_unstale":"2023-09-30T22:21:37.197712+0000","last_undegraded":"2023-09-30T22:21:37.197712+0000","last_fullsized":"2023-09-30T22:21:37.197712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220766+0000","last_change":"2023-09-30T22:21:37.217877+0000","last_active":"2023-09-30T22:21:37.220766+0000","last_peered":"2023-09-30T22:21:37.220766+0000","last_clean":"2023-09-30T22:21:37.220766+0000","last_became_active":"2023-09-30T22:21:37.217430+0000","last_became_peered":"2023-09-30T22:21:37.217430+0000","last_unstale":"2023-09-30T22:21:37.220766+0000","last_undegraded":"2023-09-30T22:21:37.220766+0000","last_fullsized":"2023-09-30T22:21:37.220766+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199951+0000","last_change":"2023-09-30T22:21:31.798962+0000","last_active":"2023-09-30T22:21:37.199951+0000","last_peered":"2023-09-30T22:21:37.199951+0000","last_clean":"2023-09-30T22:21:37.199951+0000","last_became_active":"2023-09-30T22:21:31.798641+0000","last_became_peered":"2023-09-30T22:21:31.798641+0000","last_unstale":"2023-09-30T22:21:37.199951+0000","last_undegraded":"2023-09-30T22:21:37.199951+0000","last_fullsized":"2023-09-30T22:21:37.199951+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204066+0000","last_change":"2023-09-30T22:21:36.793285+0000","last_active":"2023-09-30T22:21:37.204066+0000","last_peered":"2023-09-30T22:21:37.204066+0000","last_clean":"2023-09-30T22:21:37.204066+0000","last_became_active":"2023-09-30T22:21:36.793005+0000","last_became_peered":"2023-09-30T22:21:36.793005+0000","last_unstale":"2023-09-30T22:21:37.204066+0000","last_undegraded":"2023-09-30T22:21:37.204066+0000","last_fullsized":"2023-09-30T22:21:37.204066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238787+0000","last_change":"2023-09-30T22:21:37.236712+0000","last_active":"2023-09-30T22:21:37.238787+0000","last_peered":"2023-09-30T22:21:37.238787+0000","last_clean":"2023-09-30T22:21:37.238787+0000","last_became_active":"2023-09-30T22:21:37.226418+0000","last_became_peered":"2023-09-30T22:21:37.226418+0000","last_unstale":"2023-09-30T22:21:37.238787+0000","last_undegraded":"2023-09-30T22:21:37.238787+0000","last_fullsized":"2023-09-30T22:21:37.238787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197851+0000","last_change":"2023-09-30T22:21:31.797737+0000","last_active":"2023-09-30T22:21:37.197851+0000","last_peered":"2023-09-30T22:21:37.197851+0000","last_clean":"2023-09-30T22:21:37.197851+0000","last_became_active":"2023-09-30T22:21:31.797396+0000","last_became_peered":"2023-09-30T22:21:31.797396+0000","last_unstale":"2023-09-30T22:21:37.197851+0000","last_undegraded":"2023-09-30T22:21:37.197851+0000","last_fullsized":"2023-09-30T22:21:37.197851+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198511+0000","last_change":"2023-09-30T22:21:36.799423+0000","last_active":"2023-09-30T22:21:37.198511+0000","last_peered":"2023-09-30T22:21:37.198511+0000","last_clean":"2023-09-30T22:21:37.198511+0000","last_became_active":"2023-09-30T22:21:36.798817+0000","last_became_peered":"2023-09-30T22:21:36.798817+0000","last_unstale":"2023-09-30T22:21:37.198511+0000","last_undegraded":"2023-09-30T22:21:37.198511+0000","last_fullsized":"2023-09-30T22:21:37.198511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239506+0000","last_change":"2023-09-30T22:21:37.237647+0000","last_active":"2023-09-30T22:21:37.239506+0000","last_peered":"2023-09-30T22:21:37.239506+0000","last_clean":"2023-09-30T22:21:37.239506+0000","last_became_active":"2023-09-30T22:21:37.237287+0000","last_became_peered":"2023-09-30T22:21:37.237287+0000","last_unstale":"2023-09-30T22:21:37.239506+0000","last_undegraded":"2023-09-30T22:21:37.239506+0000","last_fullsized":"2023-09-30T22:21:37.239506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.196834+0000","last_change":"2023-09-30T22:21:31.798294+0000","last_active":"2023-09-30T22:21:37.196834+0000","last_peered":"2023-09-30T22:21:37.196834+0000","last_clean":"2023-09-30T22:21:37.196834+0000","last_became_active":"2023-09-30T22:21:31.797999+0000","last_became_peered":"2023-09-30T22:21:31.797999+0000","last_unstale":"2023-09-30T22:21:37.196834+0000","last_undegraded":"2023-09-30T22:21:37.196834+0000","last_fullsized":"2023-09-30T22:21:37.196834+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198428+0000","last_change":"2023-09-30T22:21:30.796302+0000","last_active":"2023-09-30T22:21:37.198428+0000","last_peered":"2023-09-30T22:21:37.198428+0000","last_clean":"2023-09-30T22:21:37.198428+0000","last_became_active":"2023-09-30T22:21:30.795885+0000","last_became_peered":"2023-09-30T22:21:30.795885+0000","last_unstale":"2023-09-30T22:21:37.198428+0000","last_undegraded":"2023-09-30T22:21:37.198428+0000","last_fullsized":"2023-09-30T22:21:37.198428+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_clean_scrub_stamp":"2023-09-30T22:21:29.785382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199764+0000","last_change":"2023-09-30T22:21:36.811325+0000","last_active":"2023-09-30T22:21:37.199764+0000","last_peered":"2023-09-30T22:21:37.199764+0000","last_clean":"2023-09-30T22:21:37.199764+0000","last_became_active":"2023-09-30T22:21:36.811014+0000","last_became_peered":"2023-09-30T22:21:36.811014+0000","last_unstale":"2023-09-30T22:21:37.199764+0000","last_undegraded":"2023-09-30T22:21:37.199764+0000","last_fullsized":"2023-09-30T22:21:37.199764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226368+0000","last_change":"2023-09-30T22:21:37.220230+0000","last_active":"2023-09-30T22:21:37.226368+0000","last_peered":"2023-09-30T22:21:37.226368+0000","last_clean":"2023-09-30T22:21:37.226368+0000","last_became_active":"2023-09-30T22:21:37.215165+0000","last_became_peered":"2023-09-30T22:21:37.215165+0000","last_unstale":"2023-09-30T22:21:37.226368+0000","last_undegraded":"2023-09-30T22:21:37.226368+0000","last_fullsized":"2023-09-30T22:21:37.226368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195816+0000","last_change":"2023-09-30T22:21:31.798735+0000","last_active":"2023-09-30T22:21:37.195816+0000","last_peered":"2023-09-30T22:21:37.195816+0000","last_clean":"2023-09-30T22:21:37.195816+0000","last_became_active":"2023-09-30T22:21:31.798405+0000","last_became_peered":"2023-09-30T22:21:31.798405+0000","last_unstale":"2023-09-30T22:21:37.195816+0000","last_undegraded":"2023-09-30T22:21:37.195816+0000","last_fullsized":"2023-09-30T22:21:37.195816+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204992+0000","last_change":"2023-09-30T22:21:36.793235+0000","last_active":"2023-09-30T22:21:37.204992+0000","last_peered":"2023-09-30T22:21:37.204992+0000","last_clean":"2023-09-30T22:21:37.204992+0000","last_became_active":"2023-09-30T22:21:36.792921+0000","last_became_peered":"2023-09-30T22:21:36.792921+0000","last_unstale":"2023-09-30T22:21:37.204992+0000","last_undegraded":"2023-09-30T22:21:37.204992+0000","last_fullsized":"2023-09-30T22:21:37.204992+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233365+0000","last_change":"2023-09-30T22:21:37.224362+0000","last_active":"2023-09-30T22:21:37.233365+0000","last_peered":"2023-09-30T22:21:37.233365+0000","last_clean":"2023-09-30T22:21:37.233365+0000","last_became_active":"2023-09-30T22:21:37.219539+0000","last_became_peered":"2023-09-30T22:21:37.219539+0000","last_unstale":"2023-09-30T22:21:37.233365+0000","last_undegraded":"2023-09-30T22:21:37.233365+0000","last_fullsized":"2023-09-30T22:21:37.233365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197166+0000","last_change":"2023-09-30T22:21:31.800121+0000","last_active":"2023-09-30T22:21:37.197166+0000","last_peered":"2023-09-30T22:21:37.197166+0000","last_clean":"2023-09-30T22:21:37.197166+0000","last_became_active":"2023-09-30T22:21:31.799775+0000","last_became_peered":"2023-09-30T22:21:31.799775+0000","last_unstale":"2023-09-30T22:21:37.197166+0000","last_undegraded":"2023-09-30T22:21:37.197166+0000","last_fullsized":"2023-09-30T22:21:37.197166+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199098+0000","last_change":"2023-09-30T22:21:36.799552+0000","last_active":"2023-09-30T22:21:37.199098+0000","last_peered":"2023-09-30T22:21:37.199098+0000","last_clean":"2023-09-30T22:21:37.199098+0000","last_became_active":"2023-09-30T22:21:36.798979+0000","last_became_peered":"2023-09-30T22:21:36.798979+0000","last_unstale":"2023-09-30T22:21:37.199098+0000","last_undegraded":"2023-09-30T22:21:37.199098+0000","last_fullsized":"2023-09-30T22:21:37.199098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233846+0000","last_change":"2023-09-30T22:21:37.227699+0000","last_active":"2023-09-30T22:21:37.233846+0000","last_peered":"2023-09-30T22:21:37.233846+0000","last_clean":"2023-09-30T22:21:37.233846+0000","last_became_active":"2023-09-30T22:21:37.227403+0000","last_became_peered":"2023-09-30T22:21:37.227403+0000","last_unstale":"2023-09-30T22:21:37.233846+0000","last_undegraded":"2023-09-30T22:21:37.233846+0000","last_fullsized":"2023-09-30T22:21:37.233846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233095+0000","last_change":"2023-09-30T22:21:36.817294+0000","last_active":"2023-09-30T22:21:37.233095+0000","last_peered":"2023-09-30T22:21:37.233095+0000","last_clean":"2023-09-30T22:21:37.233095+0000","last_became_active":"2023-09-30T22:21:36.816805+0000","last_became_peered":"2023-09-30T22:21:36.816805+0000","last_unstale":"2023-09-30T22:21:37.233095+0000","last_undegraded":"2023-09-30T22:21:37.233095+0000","last_fullsized":"2023-09-30T22:21:37.233095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.200191+0000","last_change":"2023-09-30T22:21:31.794560+0000","last_active":"2023-09-30T22:21:37.200191+0000","last_peered":"2023-09-30T22:21:37.200191+0000","last_clean":"2023-09-30T22:21:37.200191+0000","last_became_active":"2023-09-30T22:21:31.794101+0000","last_became_peered":"2023-09-30T22:21:31.794101+0000","last_unstale":"2023-09-30T22:21:37.200191+0000","last_undegraded":"2023-09-30T22:21:37.200191+0000","last_fullsized":"2023-09-30T22:21:37.200191+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223871+0000","last_change":"2023-09-30T22:21:37.214706+0000","last_active":"2023-09-30T22:21:37.223871+0000","last_peered":"2023-09-30T22:21:37.223871+0000","last_clean":"2023-09-30T22:21:37.223871+0000","last_became_active":"2023-09-30T22:21:37.212393+0000","last_became_peered":"2023-09-30T22:21:37.212393+0000","last_unstale":"2023-09-30T22:21:37.223871+0000","last_undegraded":"2023-09-30T22:21:37.223871+0000","last_fullsized":"2023-09-30T22:21:37.223871+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.196857+0000","last_change":"2023-09-30T22:21:31.800079+0000","last_active":"2023-09-30T22:21:37.196857+0000","last_peered":"2023-09-30T22:21:37.196857+0000","last_clean":"2023-09-30T22:21:37.196857+0000","last_became_active":"2023-09-30T22:21:31.799718+0000","last_became_peered":"2023-09-30T22:21:31.799718+0000","last_unstale":"2023-09-30T22:21:37.196857+0000","last_undegraded":"2023-09-30T22:21:37.196857+0000","last_fullsized":"2023-09-30T22:21:37.196857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T22:21:38.198008+0000","last_change":"2023-09-30T22:21:36.803700+0000","last_active":"2023-09-30T22:21:38.198008+0000","last_peered":"2023-09-30T22:21:38.198008+0000","last_clean":"2023-09-30T22:21:38.198008+0000","last_became_active":"2023-09-30T22:21:36.803365+0000","last_became_peered":"2023-09-30T22:21:36.803365+0000","last_unstale":"2023-09-30T22:21:38.198008+0000","last_undegraded":"2023-09-30T22:21:38.198008+0000","last_fullsized":"2023-09-30T22:21:38.198008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232184+0000","last_change":"2023-09-30T22:21:37.226913+0000","last_active":"2023-09-30T22:21:37.232184+0000","last_peered":"2023-09-30T22:21:37.232184+0000","last_clean":"2023-09-30T22:21:37.232184+0000","last_became_active":"2023-09-30T22:21:37.225725+0000","last_became_peered":"2023-09-30T22:21:37.225725+0000","last_unstale":"2023-09-30T22:21:37.232184+0000","last_undegraded":"2023-09-30T22:21:37.232184+0000","last_fullsized":"2023-09-30T22:21:37.232184+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.230024+0000","last_change":"2023-09-30T22:21:36.808872+0000","last_active":"2023-09-30T22:21:37.230024+0000","last_peered":"2023-09-30T22:21:37.230024+0000","last_clean":"2023-09-30T22:21:37.230024+0000","last_became_active":"2023-09-30T22:21:36.806238+0000","last_became_peered":"2023-09-30T22:21:36.806238+0000","last_unstale":"2023-09-30T22:21:37.230024+0000","last_undegraded":"2023-09-30T22:21:37.230024+0000","last_fullsized":"2023-09-30T22:21:37.230024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223800+0000","last_change":"2023-09-30T22:21:37.208459+0000","last_active":"2023-09-30T22:21:37.223800+0000","last_peered":"2023-09-30T22:21:37.223800+0000","last_clean":"2023-09-30T22:21:37.223800+0000","last_became_active":"2023-09-30T22:21:37.208076+0000","last_became_peered":"2023-09-30T22:21:37.208076+0000","last_unstale":"2023-09-30T22:21:37.223800+0000","last_undegraded":"2023-09-30T22:21:37.223800+0000","last_fullsized":"2023-09-30T22:21:37.223800+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235981+0000","last_change":"2023-09-30T22:21:36.806969+0000","last_active":"2023-09-30T22:21:37.235981+0000","last_peered":"2023-09-30T22:21:37.235981+0000","last_clean":"2023-09-30T22:21:37.235981+0000","last_became_active":"2023-09-30T22:21:36.806649+0000","last_became_peered":"2023-09-30T22:21:36.806649+0000","last_unstale":"2023-09-30T22:21:37.235981+0000","last_undegraded":"2023-09-30T22:21:37.235981+0000","last_fullsized":"2023-09-30T22:21:37.235981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232315+0000","last_change":"2023-09-30T22:21:37.229486+0000","last_active":"2023-09-30T22:21:37.232315+0000","last_peered":"2023-09-30T22:21:37.232315+0000","last_clean":"2023-09-30T22:21:37.232315+0000","last_became_active":"2023-09-30T22:21:37.226308+0000","last_became_peered":"2023-09-30T22:21:37.226308+0000","last_unstale":"2023-09-30T22:21:37.232315+0000","last_undegraded":"2023-09-30T22:21:37.232315+0000","last_fullsized":"2023-09-30T22:21:37.232315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235069+0000","last_change":"2023-09-30T22:21:36.815782+0000","last_active":"2023-09-30T22:21:37.235069+0000","last_peered":"2023-09-30T22:21:37.235069+0000","last_clean":"2023-09-30T22:21:37.235069+0000","last_became_active":"2023-09-30T22:21:36.815431+0000","last_became_peered":"2023-09-30T22:21:36.815431+0000","last_unstale":"2023-09-30T22:21:37.235069+0000","last_undegraded":"2023-09-30T22:21:37.235069+0000","last_fullsized":"2023-09-30T22:21:37.235069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221951+0000","last_change":"2023-09-30T22:21:37.215011+0000","last_active":"2023-09-30T22:21:37.221951+0000","last_peered":"2023-09-30T22:21:37.221951+0000","last_clean":"2023-09-30T22:21:37.221951+0000","last_became_active":"2023-09-30T22:21:37.214739+0000","last_became_peered":"2023-09-30T22:21:37.214739+0000","last_unstale":"2023-09-30T22:21:37.221951+0000","last_undegraded":"2023-09-30T22:21:37.221951+0000","last_fullsized":"2023-09-30T22:21:37.221951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235070+0000","last_change":"2023-09-30T22:21:36.812958+0000","last_active":"2023-09-30T22:21:37.235070+0000","last_peered":"2023-09-30T22:21:37.235070+0000","last_clean":"2023-09-30T22:21:37.235070+0000","last_became_active":"2023-09-30T22:21:36.812662+0000","last_became_peered":"2023-09-30T22:21:36.812662+0000","last_unstale":"2023-09-30T22:21:37.235070+0000","last_undegraded":"2023-09-30T22:21:37.235070+0000","last_fullsized":"2023-09-30T22:21:37.235070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234543+0000","last_change":"2023-09-30T22:21:37.225020+0000","last_active":"2023-09-30T22:21:37.234543+0000","last_peered":"2023-09-30T22:21:37.234543+0000","last_clean":"2023-09-30T22:21:37.234543+0000","last_became_active":"2023-09-30T22:21:37.224728+0000","last_became_peered":"2023-09-30T22:21:37.224728+0000","last_unstale":"2023-09-30T22:21:37.234543+0000","last_undegraded":"2023-09-30T22:21:37.234543+0000","last_fullsized":"2023-09-30T22:21:37.234543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238512+0000","last_change":"2023-09-30T22:21:36.796108+0000","last_active":"2023-09-30T22:21:37.238512+0000","last_peered":"2023-09-30T22:21:37.238512+0000","last_clean":"2023-09-30T22:21:37.238512+0000","last_became_active":"2023-09-30T22:21:36.795665+0000","last_became_peered":"2023-09-30T22:21:36.795665+0000","last_unstale":"2023-09-30T22:21:37.238512+0000","last_undegraded":"2023-09-30T22:21:37.238512+0000","last_fullsized":"2023-09-30T22:21:37.238512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.217296+0000","last_change":"2023-09-30T22:21:37.207178+0000","last_active":"2023-09-30T22:21:37.217296+0000","last_peered":"2023-09-30T22:21:37.217296+0000","last_clean":"2023-09-30T22:21:37.217296+0000","last_became_active":"2023-09-30T22:21:37.206889+0000","last_became_peered":"2023-09-30T22:21:37.206889+0000","last_unstale":"2023-09-30T22:21:37.217296+0000","last_undegraded":"2023-09-30T22:21:37.217296+0000","last_fullsized":"2023-09-30T22:21:37.217296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.210508+0000","last_change":"2023-09-30T22:21:37.206551+0000","last_active":"2023-09-30T22:21:37.210508+0000","last_peered":"2023-09-30T22:21:37.210508+0000","last_clean":"2023-09-30T22:21:37.210508+0000","last_became_active":"2023-09-30T22:21:37.206176+0000","last_became_peered":"2023-09-30T22:21:37.206176+0000","last_unstale":"2023-09-30T22:21:37.210508+0000","last_undegraded":"2023-09-30T22:21:37.210508+0000","last_fullsized":"2023-09-30T22:21:37.210508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225040+0000","last_change":"2023-09-30T22:21:37.220615+0000","last_active":"2023-09-30T22:21:37.225040+0000","last_peered":"2023-09-30T22:21:37.225040+0000","last_clean":"2023-09-30T22:21:37.225040+0000","last_became_active":"2023-09-30T22:21:37.217961+0000","last_became_peered":"2023-09-30T22:21:37.217961+0000","last_unstale":"2023-09-30T22:21:37.225040+0000","last_undegraded":"2023-09-30T22:21:37.225040+0000","last_fullsized":"2023-09-30T22:21:37.225040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234039+0000","last_change":"2023-09-30T22:21:37.229799+0000","last_active":"2023-09-30T22:21:37.234039+0000","last_peered":"2023-09-30T22:21:37.234039+0000","last_clean":"2023-09-30T22:21:37.234039+0000","last_became_active":"2023-09-30T22:21:37.229494+0000","last_became_peered":"2023-09-30T22:21:37.229494+0000","last_unstale":"2023-09-30T22:21:37.234039+0000","last_undegraded":"2023-09-30T22:21:37.234039+0000","last_fullsized":"2023-09-30T22:21:37.234039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223296+0000","last_change":"2023-09-30T22:21:37.215694+0000","last_active":"2023-09-30T22:21:37.223296+0000","last_peered":"2023-09-30T22:21:37.223296+0000","last_clean":"2023-09-30T22:21:37.223296+0000","last_became_active":"2023-09-30T22:21:37.215371+0000","last_became_peered":"2023-09-30T22:21:37.215371+0000","last_unstale":"2023-09-30T22:21:37.223296+0000","last_undegraded":"2023-09-30T22:21:37.223296+0000","last_fullsized":"2023-09-30T22:21:37.223296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.210455+0000","last_change":"2023-09-30T22:21:37.206797+0000","last_active":"2023-09-30T22:21:37.210455+0000","last_peered":"2023-09-30T22:21:37.210455+0000","last_clean":"2023-09-30T22:21:37.210455+0000","last_became_active":"2023-09-30T22:21:37.206384+0000","last_became_peered":"2023-09-30T22:21:37.206384+0000","last_unstale":"2023-09-30T22:21:37.210455+0000","last_undegraded":"2023-09-30T22:21:37.210455+0000","last_fullsized":"2023-09-30T22:21:37.210455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]}]},{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:21:45.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:21:45.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 Processor -- start 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 -- start start 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 0x7fc89c127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 0x7fc89c127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c095820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c059450 con 0x7fc89c059070 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0968c0 con 0x7fc89c095370 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.316+0000 7fc8a2b10700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c094f30 con 0x7fc89c127390 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 0x7fc89c127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a08ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 0x7fc89c127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c095820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 0x7fc89c127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35716/0 (socket says 172.21.15.32:35716) 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a08ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 0x7fc89c127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54708/0 (socket says 172.21.15.32:54708) 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89bfff700 1 -- 172.21.15.32:0/3077939118 learned_addr learned my addr 172.21.15.32:0/3077939118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 -- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 msgr2=0x7fc89c127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 --2- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 0x7fc89c127d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 -- 172.21.15.32:0/3077939118 >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 msgr2=0x7fc89c127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 --2- 172.21.15.32:0/3077939118 >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 0x7fc89c127890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 -- 172.21.15.32:0/3077939118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c091b60 con 0x7fc89c095370 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89bfff700 1 --2- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 0x7fc89c127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a08ac700 1 --2- 172.21.15.32:0/3077939118 >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 0x7fc89c127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc8a10ad700 1 --2- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c095820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc894009e60 tx=0x7fc894009780).ready entity=mon.1 client_cookie=a8faff41c92f07f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89b7fe700 1 -- 172.21.15.32:0/3077939118 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400e070 con 0x7fc89c095370 2023-09-30T22:21:45.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89b7fe700 1 -- 172.21.15.32:0/3077939118 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400bab0 con 0x7fc89c095370 2023-09-30T22:21:45.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.317+0000 7fc89b7fe700 1 -- 172.21.15.32:0/3077939118 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400be30 con 0x7fc89c095370 2023-09-30T22:21:45.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 -- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 msgr2=0x7fc89c095820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c095820 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc894009e60 tx=0x7fc894009780).stop 2023-09-30T22:21:45.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 -- 172.21.15.32:0/3077939118 shutdown_connections 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c095820 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/3077939118 >> v2:172.21.15.32:3300/0 conn(0x7fc89c059070 0x7fc89c127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/3077939118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c127390 0x7fc89c127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 -- 172.21.15.32:0/3077939118 >> 172.21.15.32:0/3077939118 conn(0x7fc89c097470 msgr2=0x7fc89c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 -- 172.21.15.32:0/3077939118 shutdown_connections 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.318+0000 7fc8a2b10700 1 -- 172.21.15.32:0/3077939118 wait complete. 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.319+0000 7fc8a2b10700 1 Processor -- start 2023-09-30T22:21:45.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.319+0000 7fc8a2b10700 1 -- start start 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 0x7fc89c125930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c1147d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89c127390 0x7fc89c114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c096ea0 con 0x7fc89c127390 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c118300 con 0x7fc89c095370 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c128f40 con 0x7fc89c059070 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 0x7fc89c125930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 0x7fc89c125930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35732/0 (socket says 172.21.15.32:35732) 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc89bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c1147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a10ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89c127390 0x7fc89c114cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 -- 172.21.15.32:0/1329306475 learned_addr learned my addr 172.21.15.32:0/1329306475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:45.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 -- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 msgr2=0x7fc89c1147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c1147d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 -- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89c127390 msgr2=0x7fc89c114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89c127390 0x7fc89c114cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 -- 172.21.15.32:0/1329306475 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8900092a0 con 0x7fc89c059070 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc89bfff700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c1147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a10ad700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89c127390 0x7fc89c114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a08ac700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 0x7fc89c125930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc89000ea20 tx=0x7fc890009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89000ee60 con 0x7fc89c059070 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89000c850 con 0x7fc89c059070 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.320+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc888009370 con 0x7fc89c059070 2023-09-30T22:21:45.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.321+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89000cbd0 con 0x7fc89c059070 2023-09-30T22:21:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.321+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc894009a20 con 0x7fc89c059070 2023-09-30T22:21:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.321+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc89001b400 con 0x7fc89c059070 2023-09-30T22:21:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.321+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc89003b510 con 0x7fc89c059070 2023-09-30T22:21:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.322+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c128890 con 0x7fc89c059070 2023-09-30T22:21:45.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.327+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc890019900 con 0x7fc89c059070 2023-09-30T22:21:45.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.495+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 --> v1:172.21.15.32:6816/118188 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc89c05a260 con 0x7fc88002bb60 2023-09-30T22:21:45.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.502+0000 7fc898ff9700 1 -- 172.21.15.32:0/1329306475 <== mgr.4101 v1:172.21.15.32:6816/118188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249961 (unknown 2965378022 0 2536888045) 0x7fc89c05a260 con 0x7fc88002bb60 2023-09-30T22:21:45.504 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:45.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 >> v1:172.21.15.32:6816/118188 conn(0x7fc88002bb60 legacy=0x7fc88002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 msgr2=0x7fc89c125930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 0x7fc89c125930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc89000ea20 tx=0x7fc890009060).stop 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 shutdown_connections 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc89c127390 0x7fc89c114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc89c059070 0x7fc89c125930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 --2- 172.21.15.32:0/1329306475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc89c095370 0x7fc89c1147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 >> 172.21.15.32:0/1329306475 conn(0x7fc89c097470 msgr2=0x7fc89c099d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 shutdown_connections 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.505+0000 7fc8a2b10700 1 -- 172.21.15.32:0/1329306475 wait complete. 2023-09-30T22:21:45.511 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T22:21:45.519 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-09-30T22:21:43.535081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189024,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298336,"statfs":{"total":386547056640,"available":385329496064,"internally_reserved":0,"allocated":655360,"data_stored":193408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000511"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221356+0000","last_change":"2023-09-30T22:21:37.213744+0000","last_active":"2023-09-30T22:21:37.221356+0000","last_peered":"2023-09-30T22:21:37.221356+0000","last_clean":"2023-09-30T22:21:37.221356+0000","last_became_active":"2023-09-30T22:21:37.213432+0000","last_became_peered":"2023-09-30T22:21:37.213432+0000","last_unstale":"2023-09-30T22:21:37.221356+0000","last_undegraded":"2023-09-30T22:21:37.221356+0000","last_fullsized":"2023-09-30T22:21:37.221356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221210+0000","last_change":"2023-09-30T22:21:37.209966+0000","last_active":"2023-09-30T22:21:37.221210+0000","last_peered":"2023-09-30T22:21:37.221210+0000","last_clean":"2023-09-30T22:21:37.221210+0000","last_became_active":"2023-09-30T22:21:37.209700+0000","last_became_peered":"2023-09-30T22:21:37.209700+0000","last_unstale":"2023-09-30T22:21:37.221210+0000","last_undegraded":"2023-09-30T22:21:37.221210+0000","last_fullsized":"2023-09-30T22:21:37.221210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233883+0000","last_change":"2023-09-30T22:21:37.227972+0000","last_active":"2023-09-30T22:21:37.233883+0000","last_peered":"2023-09-30T22:21:37.233883+0000","last_clean":"2023-09-30T22:21:37.233883+0000","last_became_active":"2023-09-30T22:21:37.227698+0000","last_became_peered":"2023-09-30T22:21:37.227698+0000","last_unstale":"2023-09-30T22:21:37.233883+0000","last_undegraded":"2023-09-30T22:21:37.233883+0000","last_fullsized":"2023-09-30T22:21:37.233883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.218248+0000","last_change":"2023-09-30T22:21:37.217682+0000","last_active":"2023-09-30T22:21:37.218248+0000","last_peered":"2023-09-30T22:21:37.218248+0000","last_clean":"2023-09-30T22:21:37.218248+0000","last_became_active":"2023-09-30T22:21:37.217395+0000","last_became_peered":"2023-09-30T22:21:37.217395+0000","last_unstale":"2023-09-30T22:21:37.218248+0000","last_undegraded":"2023-09-30T22:21:37.218248+0000","last_fullsized":"2023-09-30T22:21:37.218248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234071+0000","last_change":"2023-09-30T22:21:37.229907+0000","last_active":"2023-09-30T22:21:37.234071+0000","last_peered":"2023-09-30T22:21:37.234071+0000","last_clean":"2023-09-30T22:21:37.234071+0000","last_became_active":"2023-09-30T22:21:37.229717+0000","last_became_peered":"2023-09-30T22:21:37.229717+0000","last_unstale":"2023-09-30T22:21:37.234071+0000","last_undegraded":"2023-09-30T22:21:37.234071+0000","last_fullsized":"2023-09-30T22:21:37.234071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233648+0000","last_change":"2023-09-30T22:21:37.224923+0000","last_active":"2023-09-30T22:21:37.233648+0000","last_peered":"2023-09-30T22:21:37.233648+0000","last_clean":"2023-09-30T22:21:37.233648+0000","last_became_active":"2023-09-30T22:21:37.224651+0000","last_became_peered":"2023-09-30T22:21:37.224651+0000","last_unstale":"2023-09-30T22:21:37.233648+0000","last_undegraded":"2023-09-30T22:21:37.233648+0000","last_fullsized":"2023-09-30T22:21:37.233648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225058+0000","last_change":"2023-09-30T22:21:37.220818+0000","last_active":"2023-09-30T22:21:37.225058+0000","last_peered":"2023-09-30T22:21:37.225058+0000","last_clean":"2023-09-30T22:21:37.225058+0000","last_became_active":"2023-09-30T22:21:37.217825+0000","last_became_peered":"2023-09-30T22:21:37.217825+0000","last_unstale":"2023-09-30T22:21:37.225058+0000","last_undegraded":"2023-09-30T22:21:37.225058+0000","last_fullsized":"2023-09-30T22:21:37.225058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239537+0000","last_change":"2023-09-30T22:21:37.237512+0000","last_active":"2023-09-30T22:21:37.239537+0000","last_peered":"2023-09-30T22:21:37.239537+0000","last_clean":"2023-09-30T22:21:37.239537+0000","last_became_active":"2023-09-30T22:21:37.237217+0000","last_became_peered":"2023-09-30T22:21:37.237217+0000","last_unstale":"2023-09-30T22:21:37.239537+0000","last_undegraded":"2023-09-30T22:21:37.239537+0000","last_fullsized":"2023-09-30T22:21:37.239537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234278+0000","last_change":"2023-09-30T22:21:37.231447+0000","last_active":"2023-09-30T22:21:37.234278+0000","last_peered":"2023-09-30T22:21:37.234278+0000","last_clean":"2023-09-30T22:21:37.234278+0000","last_became_active":"2023-09-30T22:21:37.231106+0000","last_became_peered":"2023-09-30T22:21:37.231106+0000","last_unstale":"2023-09-30T22:21:37.234278+0000","last_undegraded":"2023-09-30T22:21:37.234278+0000","last_fullsized":"2023-09-30T22:21:37.234278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239686+0000","last_change":"2023-09-30T22:21:37.238710+0000","last_active":"2023-09-30T22:21:37.239686+0000","last_peered":"2023-09-30T22:21:37.239686+0000","last_clean":"2023-09-30T22:21:37.239686+0000","last_became_active":"2023-09-30T22:21:37.237230+0000","last_became_peered":"2023-09-30T22:21:37.237230+0000","last_unstale":"2023-09-30T22:21:37.239686+0000","last_undegraded":"2023-09-30T22:21:37.239686+0000","last_fullsized":"2023-09-30T22:21:37.239686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239518+0000","last_change":"2023-09-30T22:21:37.238711+0000","last_active":"2023-09-30T22:21:37.239518+0000","last_peered":"2023-09-30T22:21:37.239518+0000","last_clean":"2023-09-30T22:21:37.239518+0000","last_became_active":"2023-09-30T22:21:37.237399+0000","last_became_peered":"2023-09-30T22:21:37.237399+0000","last_unstale":"2023-09-30T22:21:37.239518+0000","last_undegraded":"2023-09-30T22:21:37.239518+0000","last_fullsized":"2023-09-30T22:21:37.239518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223819+0000","last_change":"2023-09-30T22:21:37.215868+0000","last_active":"2023-09-30T22:21:37.223819+0000","last_peered":"2023-09-30T22:21:37.223819+0000","last_clean":"2023-09-30T22:21:37.223819+0000","last_became_active":"2023-09-30T22:21:37.215601+0000","last_became_peered":"2023-09-30T22:21:37.215601+0000","last_unstale":"2023-09-30T22:21:37.223819+0000","last_undegraded":"2023-09-30T22:21:37.223819+0000","last_fullsized":"2023-09-30T22:21:37.223819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232299+0000","last_change":"2023-09-30T22:21:37.229175+0000","last_active":"2023-09-30T22:21:37.232299+0000","last_peered":"2023-09-30T22:21:37.232299+0000","last_clean":"2023-09-30T22:21:37.232299+0000","last_became_active":"2023-09-30T22:21:37.226712+0000","last_became_peered":"2023-09-30T22:21:37.226712+0000","last_unstale":"2023-09-30T22:21:37.232299+0000","last_undegraded":"2023-09-30T22:21:37.232299+0000","last_fullsized":"2023-09-30T22:21:37.232299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225658+0000","last_change":"2023-09-30T22:21:37.222596+0000","last_active":"2023-09-30T22:21:37.225658+0000","last_peered":"2023-09-30T22:21:37.225658+0000","last_clean":"2023-09-30T22:21:37.225658+0000","last_became_active":"2023-09-30T22:21:37.222295+0000","last_became_peered":"2023-09-30T22:21:37.222295+0000","last_unstale":"2023-09-30T22:21:37.225658+0000","last_undegraded":"2023-09-30T22:21:37.225658+0000","last_fullsized":"2023-09-30T22:21:37.225658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.218011+0000","last_change":"2023-09-30T22:21:37.215505+0000","last_active":"2023-09-30T22:21:37.218011+0000","last_peered":"2023-09-30T22:21:37.218011+0000","last_clean":"2023-09-30T22:21:37.218011+0000","last_became_active":"2023-09-30T22:21:37.215015+0000","last_became_peered":"2023-09-30T22:21:37.215015+0000","last_unstale":"2023-09-30T22:21:37.218011+0000","last_undegraded":"2023-09-30T22:21:37.218011+0000","last_fullsized":"2023-09-30T22:21:37.218011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.224429+0000","last_change":"2023-09-30T22:21:37.220367+0000","last_active":"2023-09-30T22:21:37.224429+0000","last_peered":"2023-09-30T22:21:37.224429+0000","last_clean":"2023-09-30T22:21:37.224429+0000","last_became_active":"2023-09-30T22:21:37.217445+0000","last_became_peered":"2023-09-30T22:21:37.217445+0000","last_unstale":"2023-09-30T22:21:37.224429+0000","last_undegraded":"2023-09-30T22:21:37.224429+0000","last_fullsized":"2023-09-30T22:21:37.224429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234262+0000","last_change":"2023-09-30T22:21:37.231390+0000","last_active":"2023-09-30T22:21:37.234262+0000","last_peered":"2023-09-30T22:21:37.234262+0000","last_clean":"2023-09-30T22:21:37.234262+0000","last_became_active":"2023-09-30T22:21:37.231116+0000","last_became_peered":"2023-09-30T22:21:37.231116+0000","last_unstale":"2023-09-30T22:21:37.234262+0000","last_undegraded":"2023-09-30T22:21:37.234262+0000","last_fullsized":"2023-09-30T22:21:37.234262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234120+0000","last_change":"2023-09-30T22:21:37.228923+0000","last_active":"2023-09-30T22:21:37.234120+0000","last_peered":"2023-09-30T22:21:37.234120+0000","last_clean":"2023-09-30T22:21:37.234120+0000","last_became_active":"2023-09-30T22:21:37.220140+0000","last_became_peered":"2023-09-30T22:21:37.220140+0000","last_unstale":"2023-09-30T22:21:37.234120+0000","last_undegraded":"2023-09-30T22:21:37.234120+0000","last_fullsized":"2023-09-30T22:21:37.234120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.228910+0000","last_change":"2023-09-30T22:21:37.225422+0000","last_active":"2023-09-30T22:21:37.228910+0000","last_peered":"2023-09-30T22:21:37.228910+0000","last_clean":"2023-09-30T22:21:37.228910+0000","last_became_active":"2023-09-30T22:21:37.225118+0000","last_became_peered":"2023-09-30T22:21:37.225118+0000","last_unstale":"2023-09-30T22:21:37.228910+0000","last_undegraded":"2023-09-30T22:21:37.228910+0000","last_fullsized":"2023-09-30T22:21:37.228910+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220876+0000","last_change":"2023-09-30T22:21:37.218484+0000","last_active":"2023-09-30T22:21:37.220876+0000","last_peered":"2023-09-30T22:21:37.220876+0000","last_clean":"2023-09-30T22:21:37.220876+0000","last_became_active":"2023-09-30T22:21:37.218141+0000","last_became_peered":"2023-09-30T22:21:37.218141+0000","last_unstale":"2023-09-30T22:21:37.220876+0000","last_undegraded":"2023-09-30T22:21:37.220876+0000","last_fullsized":"2023-09-30T22:21:37.220876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.219680+0000","last_change":"2023-09-30T22:21:37.216211+0000","last_active":"2023-09-30T22:21:37.219680+0000","last_peered":"2023-09-30T22:21:37.219680+0000","last_clean":"2023-09-30T22:21:37.219680+0000","last_became_active":"2023-09-30T22:21:37.215922+0000","last_became_peered":"2023-09-30T22:21:37.215922+0000","last_unstale":"2023-09-30T22:21:37.219680+0000","last_undegraded":"2023-09-30T22:21:37.219680+0000","last_fullsized":"2023-09-30T22:21:37.219680+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233722+0000","last_change":"2023-09-30T22:21:37.227137+0000","last_active":"2023-09-30T22:21:37.233722+0000","last_peered":"2023-09-30T22:21:37.233722+0000","last_clean":"2023-09-30T22:21:37.233722+0000","last_became_active":"2023-09-30T22:21:37.226848+0000","last_became_peered":"2023-09-30T22:21:37.226848+0000","last_unstale":"2023-09-30T22:21:37.233722+0000","last_undegraded":"2023-09-30T22:21:37.233722+0000","last_fullsized":"2023-09-30T22:21:37.233722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225020+0000","last_change":"2023-09-30T22:21:37.220415+0000","last_active":"2023-09-30T22:21:37.225020+0000","last_peered":"2023-09-30T22:21:37.225020+0000","last_clean":"2023-09-30T22:21:37.225020+0000","last_became_active":"2023-09-30T22:21:37.217468+0000","last_became_peered":"2023-09-30T22:21:37.217468+0000","last_unstale":"2023-09-30T22:21:37.225020+0000","last_undegraded":"2023-09-30T22:21:37.225020+0000","last_fullsized":"2023-09-30T22:21:37.225020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221521+0000","last_change":"2023-09-30T22:21:37.220204+0000","last_active":"2023-09-30T22:21:37.221521+0000","last_peered":"2023-09-30T22:21:37.221521+0000","last_clean":"2023-09-30T22:21:37.221521+0000","last_became_active":"2023-09-30T22:21:37.219099+0000","last_became_peered":"2023-09-30T22:21:37.219099+0000","last_unstale":"2023-09-30T22:21:37.221521+0000","last_undegraded":"2023-09-30T22:21:37.221521+0000","last_fullsized":"2023-09-30T22:21:37.221521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220336+0000","last_change":"2023-09-30T22:21:37.217211+0000","last_active":"2023-09-30T22:21:37.220336+0000","last_peered":"2023-09-30T22:21:37.220336+0000","last_clean":"2023-09-30T22:21:37.220336+0000","last_became_active":"2023-09-30T22:21:37.216537+0000","last_became_peered":"2023-09-30T22:21:37.216537+0000","last_unstale":"2023-09-30T22:21:37.220336+0000","last_undegraded":"2023-09-30T22:21:37.220336+0000","last_fullsized":"2023-09-30T22:21:37.220336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.213682+0000","last_change":"2023-09-30T22:21:37.209836+0000","last_active":"2023-09-30T22:21:37.213682+0000","last_peered":"2023-09-30T22:21:37.213682+0000","last_clean":"2023-09-30T22:21:37.213682+0000","last_became_active":"2023-09-30T22:21:37.209561+0000","last_became_peered":"2023-09-30T22:21:37.209561+0000","last_unstale":"2023-09-30T22:21:37.213682+0000","last_undegraded":"2023-09-30T22:21:37.213682+0000","last_fullsized":"2023-09-30T22:21:37.213682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226409+0000","last_change":"2023-09-30T22:21:37.220637+0000","last_active":"2023-09-30T22:21:37.226409+0000","last_peered":"2023-09-30T22:21:37.226409+0000","last_clean":"2023-09-30T22:21:37.226409+0000","last_became_active":"2023-09-30T22:21:37.217741+0000","last_became_peered":"2023-09-30T22:21:37.217741+0000","last_unstale":"2023-09-30T22:21:37.226409+0000","last_undegraded":"2023-09-30T22:21:37.226409+0000","last_fullsized":"2023-09-30T22:21:37.226409+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.222723+0000","last_change":"2023-09-30T22:21:36.800514+0000","last_active":"2023-09-30T22:21:37.222723+0000","last_peered":"2023-09-30T22:21:37.222723+0000","last_clean":"2023-09-30T22:21:37.222723+0000","last_became_active":"2023-09-30T22:21:36.799775+0000","last_became_peered":"2023-09-30T22:21:36.799775+0000","last_unstale":"2023-09-30T22:21:37.222723+0000","last_undegraded":"2023-09-30T22:21:37.222723+0000","last_fullsized":"2023-09-30T22:21:37.222723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.224088+0000","last_change":"2023-09-30T22:21:37.211849+0000","last_active":"2023-09-30T22:21:37.224088+0000","last_peered":"2023-09-30T22:21:37.224088+0000","last_clean":"2023-09-30T22:21:37.224088+0000","last_became_active":"2023-09-30T22:21:37.209128+0000","last_became_peered":"2023-09-30T22:21:37.209128+0000","last_unstale":"2023-09-30T22:21:37.224088+0000","last_undegraded":"2023-09-30T22:21:37.224088+0000","last_fullsized":"2023-09-30T22:21:37.224088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198669+0000","last_change":"2023-09-30T22:21:36.803467+0000","last_active":"2023-09-30T22:21:37.198669+0000","last_peered":"2023-09-30T22:21:37.198669+0000","last_clean":"2023-09-30T22:21:37.198669+0000","last_became_active":"2023-09-30T22:21:36.803141+0000","last_became_peered":"2023-09-30T22:21:36.803141+0000","last_unstale":"2023-09-30T22:21:37.198669+0000","last_undegraded":"2023-09-30T22:21:37.198669+0000","last_fullsized":"2023-09-30T22:21:37.198669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233388+0000","last_change":"2023-09-30T22:21:37.224566+0000","last_active":"2023-09-30T22:21:37.233388+0000","last_peered":"2023-09-30T22:21:37.233388+0000","last_clean":"2023-09-30T22:21:37.233388+0000","last_became_active":"2023-09-30T22:21:37.219843+0000","last_became_peered":"2023-09-30T22:21:37.219843+0000","last_unstale":"2023-09-30T22:21:37.233388+0000","last_undegraded":"2023-09-30T22:21:37.233388+0000","last_fullsized":"2023-09-30T22:21:37.233388+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195923+0000","last_change":"2023-09-30T22:21:36.813865+0000","last_active":"2023-09-30T22:21:37.195923+0000","last_peered":"2023-09-30T22:21:37.195923+0000","last_clean":"2023-09-30T22:21:37.195923+0000","last_became_active":"2023-09-30T22:21:36.812871+0000","last_became_peered":"2023-09-30T22:21:36.812871+0000","last_unstale":"2023-09-30T22:21:37.195923+0000","last_undegraded":"2023-09-30T22:21:37.195923+0000","last_fullsized":"2023-09-30T22:21:37.195923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226177+0000","last_change":"2023-09-30T22:21:37.215838+0000","last_active":"2023-09-30T22:21:37.226177+0000","last_peered":"2023-09-30T22:21:37.226177+0000","last_clean":"2023-09-30T22:21:37.226177+0000","last_became_active":"2023-09-30T22:21:37.215314+0000","last_became_peered":"2023-09-30T22:21:37.215314+0000","last_unstale":"2023-09-30T22:21:37.226177+0000","last_undegraded":"2023-09-30T22:21:37.226177+0000","last_fullsized":"2023-09-30T22:21:37.226177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204605+0000","last_change":"2023-09-30T22:21:36.809280+0000","last_active":"2023-09-30T22:21:37.204605+0000","last_peered":"2023-09-30T22:21:37.204605+0000","last_clean":"2023-09-30T22:21:37.204605+0000","last_became_active":"2023-09-30T22:21:36.808975+0000","last_became_peered":"2023-09-30T22:21:36.808975+0000","last_unstale":"2023-09-30T22:21:37.204605+0000","last_undegraded":"2023-09-30T22:21:37.204605+0000","last_fullsized":"2023-09-30T22:21:37.204605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223892+0000","last_change":"2023-09-30T22:21:37.215010+0000","last_active":"2023-09-30T22:21:37.223892+0000","last_peered":"2023-09-30T22:21:37.223892+0000","last_clean":"2023-09-30T22:21:37.223892+0000","last_became_active":"2023-09-30T22:21:37.212913+0000","last_became_peered":"2023-09-30T22:21:37.212913+0000","last_unstale":"2023-09-30T22:21:37.223892+0000","last_undegraded":"2023-09-30T22:21:37.223892+0000","last_fullsized":"2023-09-30T22:21:37.223892+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.201567+0000","last_change":"2023-09-30T22:21:36.800431+0000","last_active":"2023-09-30T22:21:37.201567+0000","last_peered":"2023-09-30T22:21:37.201567+0000","last_clean":"2023-09-30T22:21:37.201567+0000","last_became_active":"2023-09-30T22:21:36.800130+0000","last_became_peered":"2023-09-30T22:21:36.800130+0000","last_unstale":"2023-09-30T22:21:37.201567+0000","last_undegraded":"2023-09-30T22:21:37.201567+0000","last_fullsized":"2023-09-30T22:21:37.201567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225077+0000","last_change":"2023-09-30T22:21:37.220978+0000","last_active":"2023-09-30T22:21:37.225077+0000","last_peered":"2023-09-30T22:21:37.225077+0000","last_clean":"2023-09-30T22:21:37.225077+0000","last_became_active":"2023-09-30T22:21:37.218384+0000","last_became_peered":"2023-09-30T22:21:37.218384+0000","last_unstale":"2023-09-30T22:21:37.225077+0000","last_undegraded":"2023-09-30T22:21:37.225077+0000","last_fullsized":"2023-09-30T22:21:37.225077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.231706+0000","last_change":"2023-09-30T22:21:36.810931+0000","last_active":"2023-09-30T22:21:37.231706+0000","last_peered":"2023-09-30T22:21:37.231706+0000","last_clean":"2023-09-30T22:21:37.231706+0000","last_became_active":"2023-09-30T22:21:36.810625+0000","last_became_peered":"2023-09-30T22:21:36.810625+0000","last_unstale":"2023-09-30T22:21:37.231706+0000","last_undegraded":"2023-09-30T22:21:37.231706+0000","last_fullsized":"2023-09-30T22:21:37.231706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220744+0000","last_change":"2023-09-30T22:21:37.217701+0000","last_active":"2023-09-30T22:21:37.220744+0000","last_peered":"2023-09-30T22:21:37.220744+0000","last_clean":"2023-09-30T22:21:37.220744+0000","last_became_active":"2023-09-30T22:21:37.217076+0000","last_became_peered":"2023-09-30T22:21:37.217076+0000","last_unstale":"2023-09-30T22:21:37.220744+0000","last_undegraded":"2023-09-30T22:21:37.220744+0000","last_fullsized":"2023-09-30T22:21:37.220744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"16'6","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.236574+0000","last_change":"2023-09-30T22:21:36.811297+0000","last_active":"2023-09-30T22:21:37.236574+0000","last_peered":"2023-09-30T22:21:37.236574+0000","last_clean":"2023-09-30T22:21:37.236574+0000","last_became_active":"2023-09-30T22:21:36.810987+0000","last_became_peered":"2023-09-30T22:21:36.810987+0000","last_unstale":"2023-09-30T22:21:37.236574+0000","last_undegraded":"2023-09-30T22:21:37.236574+0000","last_fullsized":"2023-09-30T22:21:37.236574+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232281+0000","last_change":"2023-09-30T22:21:37.228789+0000","last_active":"2023-09-30T22:21:37.232281+0000","last_peered":"2023-09-30T22:21:37.232281+0000","last_clean":"2023-09-30T22:21:37.232281+0000","last_became_active":"2023-09-30T22:21:37.225220+0000","last_became_peered":"2023-09-30T22:21:37.225220+0000","last_unstale":"2023-09-30T22:21:37.232281+0000","last_undegraded":"2023-09-30T22:21:37.232281+0000","last_fullsized":"2023-09-30T22:21:37.232281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235370+0000","last_change":"2023-09-30T22:21:36.813567+0000","last_active":"2023-09-30T22:21:37.235370+0000","last_peered":"2023-09-30T22:21:37.235370+0000","last_clean":"2023-09-30T22:21:37.235370+0000","last_became_active":"2023-09-30T22:21:36.813257+0000","last_became_peered":"2023-09-30T22:21:36.813257+0000","last_unstale":"2023-09-30T22:21:37.235370+0000","last_undegraded":"2023-09-30T22:21:37.235370+0000","last_fullsized":"2023-09-30T22:21:37.235370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223168+0000","last_change":"2023-09-30T22:21:37.213340+0000","last_active":"2023-09-30T22:21:37.223168+0000","last_peered":"2023-09-30T22:21:37.223168+0000","last_clean":"2023-09-30T22:21:37.223168+0000","last_became_active":"2023-09-30T22:21:37.213040+0000","last_became_peered":"2023-09-30T22:21:37.213040+0000","last_unstale":"2023-09-30T22:21:37.223168+0000","last_undegraded":"2023-09-30T22:21:37.223168+0000","last_fullsized":"2023-09-30T22:21:37.223168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.205101+0000","last_change":"2023-09-30T22:21:36.806015+0000","last_active":"2023-09-30T22:21:37.205101+0000","last_peered":"2023-09-30T22:21:37.205101+0000","last_clean":"2023-09-30T22:21:37.205101+0000","last_became_active":"2023-09-30T22:21:36.805679+0000","last_became_peered":"2023-09-30T22:21:36.805679+0000","last_unstale":"2023-09-30T22:21:37.205101+0000","last_undegraded":"2023-09-30T22:21:37.205101+0000","last_fullsized":"2023-09-30T22:21:37.205101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226450+0000","last_change":"2023-09-30T22:21:37.221673+0000","last_active":"2023-09-30T22:21:37.226450+0000","last_peered":"2023-09-30T22:21:37.226450+0000","last_clean":"2023-09-30T22:21:37.226450+0000","last_became_active":"2023-09-30T22:21:37.221388+0000","last_became_peered":"2023-09-30T22:21:37.221388+0000","last_unstale":"2023-09-30T22:21:37.226450+0000","last_undegraded":"2023-09-30T22:21:37.226450+0000","last_fullsized":"2023-09-30T22:21:37.226450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199788+0000","last_change":"2023-09-30T22:21:36.812303+0000","last_active":"2023-09-30T22:21:37.199788+0000","last_peered":"2023-09-30T22:21:37.199788+0000","last_clean":"2023-09-30T22:21:37.199788+0000","last_became_active":"2023-09-30T22:21:36.811887+0000","last_became_peered":"2023-09-30T22:21:36.811887+0000","last_unstale":"2023-09-30T22:21:37.199788+0000","last_undegraded":"2023-09-30T22:21:37.199788+0000","last_fullsized":"2023-09-30T22:21:37.199788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234099+0000","last_change":"2023-09-30T22:21:37.225272+0000","last_active":"2023-09-30T22:21:37.234099+0000","last_peered":"2023-09-30T22:21:37.234099+0000","last_clean":"2023-09-30T22:21:37.234099+0000","last_became_active":"2023-09-30T22:21:37.224827+0000","last_became_peered":"2023-09-30T22:21:37.224827+0000","last_unstale":"2023-09-30T22:21:37.234099+0000","last_undegraded":"2023-09-30T22:21:37.234099+0000","last_fullsized":"2023-09-30T22:21:37.234099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239712+0000","last_change":"2023-09-30T22:21:36.804342+0000","last_active":"2023-09-30T22:21:37.239712+0000","last_peered":"2023-09-30T22:21:37.239712+0000","last_clean":"2023-09-30T22:21:37.239712+0000","last_became_active":"2023-09-30T22:21:36.804002+0000","last_became_peered":"2023-09-30T22:21:36.804002+0000","last_unstale":"2023-09-30T22:21:37.239712+0000","last_undegraded":"2023-09-30T22:21:37.239712+0000","last_fullsized":"2023-09-30T22:21:37.239712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226154+0000","last_change":"2023-09-30T22:21:37.215043+0000","last_active":"2023-09-30T22:21:37.226154+0000","last_peered":"2023-09-30T22:21:37.226154+0000","last_clean":"2023-09-30T22:21:37.226154+0000","last_became_active":"2023-09-30T22:21:37.214750+0000","last_became_peered":"2023-09-30T22:21:37.214750+0000","last_unstale":"2023-09-30T22:21:37.226154+0000","last_undegraded":"2023-09-30T22:21:37.226154+0000","last_fullsized":"2023-09-30T22:21:37.226154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239337+0000","last_change":"2023-09-30T22:21:36.796103+0000","last_active":"2023-09-30T22:21:37.239337+0000","last_peered":"2023-09-30T22:21:37.239337+0000","last_clean":"2023-09-30T22:21:37.239337+0000","last_became_active":"2023-09-30T22:21:36.795669+0000","last_became_peered":"2023-09-30T22:21:36.795669+0000","last_unstale":"2023-09-30T22:21:37.239337+0000","last_undegraded":"2023-09-30T22:21:37.239337+0000","last_fullsized":"2023-09-30T22:21:37.239337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226133+0000","last_change":"2023-09-30T22:21:37.223782+0000","last_active":"2023-09-30T22:21:37.226133+0000","last_peered":"2023-09-30T22:21:37.226133+0000","last_clean":"2023-09-30T22:21:37.226133+0000","last_became_active":"2023-09-30T22:21:37.223508+0000","last_became_peered":"2023-09-30T22:21:37.223508+0000","last_unstale":"2023-09-30T22:21:37.226133+0000","last_undegraded":"2023-09-30T22:21:37.226133+0000","last_fullsized":"2023-09-30T22:21:37.226133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.230042+0000","last_change":"2023-09-30T22:21:36.798241+0000","last_active":"2023-09-30T22:21:37.230042+0000","last_peered":"2023-09-30T22:21:37.230042+0000","last_clean":"2023-09-30T22:21:37.230042+0000","last_became_active":"2023-09-30T22:21:36.797923+0000","last_became_peered":"2023-09-30T22:21:36.797923+0000","last_unstale":"2023-09-30T22:21:37.230042+0000","last_undegraded":"2023-09-30T22:21:37.230042+0000","last_fullsized":"2023-09-30T22:21:37.230042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225577+0000","last_change":"2023-09-30T22:21:37.223841+0000","last_active":"2023-09-30T22:21:37.225577+0000","last_peered":"2023-09-30T22:21:37.225577+0000","last_clean":"2023-09-30T22:21:37.225577+0000","last_became_active":"2023-09-30T22:21:37.223485+0000","last_became_peered":"2023-09-30T22:21:37.223485+0000","last_unstale":"2023-09-30T22:21:37.225577+0000","last_undegraded":"2023-09-30T22:21:37.225577+0000","last_fullsized":"2023-09-30T22:21:37.225577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238213+0000","last_change":"2023-09-30T22:21:36.817374+0000","last_active":"2023-09-30T22:21:37.238213+0000","last_peered":"2023-09-30T22:21:37.238213+0000","last_clean":"2023-09-30T22:21:37.238213+0000","last_became_active":"2023-09-30T22:21:36.816913+0000","last_became_peered":"2023-09-30T22:21:36.816913+0000","last_unstale":"2023-09-30T22:21:37.238213+0000","last_undegraded":"2023-09-30T22:21:37.238213+0000","last_fullsized":"2023-09-30T22:21:37.238213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221929+0000","last_change":"2023-09-30T22:21:37.214893+0000","last_active":"2023-09-30T22:21:37.221929+0000","last_peered":"2023-09-30T22:21:37.221929+0000","last_clean":"2023-09-30T22:21:37.221929+0000","last_became_active":"2023-09-30T22:21:37.214616+0000","last_became_peered":"2023-09-30T22:21:37.214616+0000","last_unstale":"2023-09-30T22:21:37.221929+0000","last_undegraded":"2023-09-30T22:21:37.221929+0000","last_fullsized":"2023-09-30T22:21:37.221929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.202352+0000","last_change":"2023-09-30T22:21:36.806594+0000","last_active":"2023-09-30T22:21:37.202352+0000","last_peered":"2023-09-30T22:21:37.202352+0000","last_clean":"2023-09-30T22:21:37.202352+0000","last_became_active":"2023-09-30T22:21:36.806305+0000","last_became_peered":"2023-09-30T22:21:36.806305+0000","last_unstale":"2023-09-30T22:21:37.202352+0000","last_undegraded":"2023-09-30T22:21:37.202352+0000","last_fullsized":"2023-09-30T22:21:37.202352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233473+0000","last_change":"2023-09-30T22:21:37.224756+0000","last_active":"2023-09-30T22:21:37.233473+0000","last_peered":"2023-09-30T22:21:37.233473+0000","last_clean":"2023-09-30T22:21:37.233473+0000","last_became_active":"2023-09-30T22:21:37.224444+0000","last_became_peered":"2023-09-30T22:21:37.224444+0000","last_unstale":"2023-09-30T22:21:37.233473+0000","last_undegraded":"2023-09-30T22:21:37.233473+0000","last_fullsized":"2023-09-30T22:21:37.233473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235343+0000","last_change":"2023-09-30T22:21:36.813533+0000","last_active":"2023-09-30T22:21:37.235343+0000","last_peered":"2023-09-30T22:21:37.235343+0000","last_clean":"2023-09-30T22:21:37.235343+0000","last_became_active":"2023-09-30T22:21:36.813227+0000","last_became_peered":"2023-09-30T22:21:36.813227+0000","last_unstale":"2023-09-30T22:21:37.235343+0000","last_undegraded":"2023-09-30T22:21:37.235343+0000","last_fullsized":"2023-09-30T22:21:37.235343+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234223+0000","last_change":"2023-09-30T22:21:37.230482+0000","last_active":"2023-09-30T22:21:37.234223+0000","last_peered":"2023-09-30T22:21:37.234223+0000","last_clean":"2023-09-30T22:21:37.234223+0000","last_became_active":"2023-09-30T22:21:37.229876+0000","last_became_peered":"2023-09-30T22:21:37.229876+0000","last_unstale":"2023-09-30T22:21:37.234223+0000","last_undegraded":"2023-09-30T22:21:37.234223+0000","last_fullsized":"2023-09-30T22:21:37.234223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199874+0000","last_change":"2023-09-30T22:21:36.811138+0000","last_active":"2023-09-30T22:21:37.199874+0000","last_peered":"2023-09-30T22:21:37.199874+0000","last_clean":"2023-09-30T22:21:37.199874+0000","last_became_active":"2023-09-30T22:21:36.810834+0000","last_became_peered":"2023-09-30T22:21:36.810834+0000","last_unstale":"2023-09-30T22:21:37.199874+0000","last_undegraded":"2023-09-30T22:21:37.199874+0000","last_fullsized":"2023-09-30T22:21:37.199874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226390+0000","last_change":"2023-09-30T22:21:37.220428+0000","last_active":"2023-09-30T22:21:37.226390+0000","last_peered":"2023-09-30T22:21:37.226390+0000","last_clean":"2023-09-30T22:21:37.226390+0000","last_became_active":"2023-09-30T22:21:37.217353+0000","last_became_peered":"2023-09-30T22:21:37.217353+0000","last_unstale":"2023-09-30T22:21:37.226390+0000","last_undegraded":"2023-09-30T22:21:37.226390+0000","last_fullsized":"2023-09-30T22:21:37.226390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195804+0000","last_change":"2023-09-30T22:21:36.812156+0000","last_active":"2023-09-30T22:21:37.195804+0000","last_peered":"2023-09-30T22:21:37.195804+0000","last_clean":"2023-09-30T22:21:37.195804+0000","last_became_active":"2023-09-30T22:21:36.811527+0000","last_became_peered":"2023-09-30T22:21:36.811527+0000","last_unstale":"2023-09-30T22:21:37.195804+0000","last_undegraded":"2023-09-30T22:21:37.195804+0000","last_fullsized":"2023-09-30T22:21:37.195804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234586+0000","last_change":"2023-09-30T22:21:37.226310+0000","last_active":"2023-09-30T22:21:37.234586+0000","last_peered":"2023-09-30T22:21:37.234586+0000","last_clean":"2023-09-30T22:21:37.234586+0000","last_became_active":"2023-09-30T22:21:37.225749+0000","last_became_peered":"2023-09-30T22:21:37.225749+0000","last_unstale":"2023-09-30T22:21:37.234586+0000","last_undegraded":"2023-09-30T22:21:37.234586+0000","last_fullsized":"2023-09-30T22:21:37.234586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195360+0000","last_change":"2023-09-30T22:21:36.811756+0000","last_active":"2023-09-30T22:21:37.195360+0000","last_peered":"2023-09-30T22:21:37.195360+0000","last_clean":"2023-09-30T22:21:37.195360+0000","last_became_active":"2023-09-30T22:21:36.811393+0000","last_became_peered":"2023-09-30T22:21:36.811393+0000","last_unstale":"2023-09-30T22:21:37.195360+0000","last_undegraded":"2023-09-30T22:21:37.195360+0000","last_fullsized":"2023-09-30T22:21:37.195360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.219948+0000","last_change":"2023-09-30T22:21:37.217010+0000","last_active":"2023-09-30T22:21:37.219948+0000","last_peered":"2023-09-30T22:21:37.219948+0000","last_clean":"2023-09-30T22:21:37.219948+0000","last_became_active":"2023-09-30T22:21:37.216071+0000","last_became_peered":"2023-09-30T22:21:37.216071+0000","last_unstale":"2023-09-30T22:21:37.219948+0000","last_undegraded":"2023-09-30T22:21:37.219948+0000","last_fullsized":"2023-09-30T22:21:37.219948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199310+0000","last_change":"2023-09-30T22:21:31.794405+0000","last_active":"2023-09-30T22:21:37.199310+0000","last_peered":"2023-09-30T22:21:37.199310+0000","last_clean":"2023-09-30T22:21:37.199310+0000","last_became_active":"2023-09-30T22:21:31.794052+0000","last_became_peered":"2023-09-30T22:21:31.794052+0000","last_unstale":"2023-09-30T22:21:37.199310+0000","last_undegraded":"2023-09-30T22:21:37.199310+0000","last_fullsized":"2023-09-30T22:21:37.199310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197712+0000","last_change":"2023-09-30T22:21:36.799995+0000","last_active":"2023-09-30T22:21:37.197712+0000","last_peered":"2023-09-30T22:21:37.197712+0000","last_clean":"2023-09-30T22:21:37.197712+0000","last_became_active":"2023-09-30T22:21:36.799674+0000","last_became_peered":"2023-09-30T22:21:36.799674+0000","last_unstale":"2023-09-30T22:21:37.197712+0000","last_undegraded":"2023-09-30T22:21:37.197712+0000","last_fullsized":"2023-09-30T22:21:37.197712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.220766+0000","last_change":"2023-09-30T22:21:37.217877+0000","last_active":"2023-09-30T22:21:37.220766+0000","last_peered":"2023-09-30T22:21:37.220766+0000","last_clean":"2023-09-30T22:21:37.220766+0000","last_became_active":"2023-09-30T22:21:37.217430+0000","last_became_peered":"2023-09-30T22:21:37.217430+0000","last_unstale":"2023-09-30T22:21:37.220766+0000","last_undegraded":"2023-09-30T22:21:37.220766+0000","last_fullsized":"2023-09-30T22:21:37.220766+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199951+0000","last_change":"2023-09-30T22:21:31.798962+0000","last_active":"2023-09-30T22:21:37.199951+0000","last_peered":"2023-09-30T22:21:37.199951+0000","last_clean":"2023-09-30T22:21:37.199951+0000","last_became_active":"2023-09-30T22:21:31.798641+0000","last_became_peered":"2023-09-30T22:21:31.798641+0000","last_unstale":"2023-09-30T22:21:37.199951+0000","last_undegraded":"2023-09-30T22:21:37.199951+0000","last_fullsized":"2023-09-30T22:21:37.199951+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204066+0000","last_change":"2023-09-30T22:21:36.793285+0000","last_active":"2023-09-30T22:21:37.204066+0000","last_peered":"2023-09-30T22:21:37.204066+0000","last_clean":"2023-09-30T22:21:37.204066+0000","last_became_active":"2023-09-30T22:21:36.793005+0000","last_became_peered":"2023-09-30T22:21:36.793005+0000","last_unstale":"2023-09-30T22:21:37.204066+0000","last_undegraded":"2023-09-30T22:21:37.204066+0000","last_fullsized":"2023-09-30T22:21:37.204066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238787+0000","last_change":"2023-09-30T22:21:37.236712+0000","last_active":"2023-09-30T22:21:37.238787+0000","last_peered":"2023-09-30T22:21:37.238787+0000","last_clean":"2023-09-30T22:21:37.238787+0000","last_became_active":"2023-09-30T22:21:37.226418+0000","last_became_peered":"2023-09-30T22:21:37.226418+0000","last_unstale":"2023-09-30T22:21:37.238787+0000","last_undegraded":"2023-09-30T22:21:37.238787+0000","last_fullsized":"2023-09-30T22:21:37.238787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197851+0000","last_change":"2023-09-30T22:21:31.797737+0000","last_active":"2023-09-30T22:21:37.197851+0000","last_peered":"2023-09-30T22:21:37.197851+0000","last_clean":"2023-09-30T22:21:37.197851+0000","last_became_active":"2023-09-30T22:21:31.797396+0000","last_became_peered":"2023-09-30T22:21:31.797396+0000","last_unstale":"2023-09-30T22:21:37.197851+0000","last_undegraded":"2023-09-30T22:21:37.197851+0000","last_fullsized":"2023-09-30T22:21:37.197851+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198511+0000","last_change":"2023-09-30T22:21:36.799423+0000","last_active":"2023-09-30T22:21:37.198511+0000","last_peered":"2023-09-30T22:21:37.198511+0000","last_clean":"2023-09-30T22:21:37.198511+0000","last_became_active":"2023-09-30T22:21:36.798817+0000","last_became_peered":"2023-09-30T22:21:36.798817+0000","last_unstale":"2023-09-30T22:21:37.198511+0000","last_undegraded":"2023-09-30T22:21:37.198511+0000","last_fullsized":"2023-09-30T22:21:37.198511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.239506+0000","last_change":"2023-09-30T22:21:37.237647+0000","last_active":"2023-09-30T22:21:37.239506+0000","last_peered":"2023-09-30T22:21:37.239506+0000","last_clean":"2023-09-30T22:21:37.239506+0000","last_became_active":"2023-09-30T22:21:37.237287+0000","last_became_peered":"2023-09-30T22:21:37.237287+0000","last_unstale":"2023-09-30T22:21:37.239506+0000","last_undegraded":"2023-09-30T22:21:37.239506+0000","last_fullsized":"2023-09-30T22:21:37.239506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.196834+0000","last_change":"2023-09-30T22:21:31.798294+0000","last_active":"2023-09-30T22:21:37.196834+0000","last_peered":"2023-09-30T22:21:37.196834+0000","last_clean":"2023-09-30T22:21:37.196834+0000","last_became_active":"2023-09-30T22:21:31.797999+0000","last_became_peered":"2023-09-30T22:21:31.797999+0000","last_unstale":"2023-09-30T22:21:37.196834+0000","last_undegraded":"2023-09-30T22:21:37.196834+0000","last_fullsized":"2023-09-30T22:21:37.196834+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.198428+0000","last_change":"2023-09-30T22:21:30.796302+0000","last_active":"2023-09-30T22:21:37.198428+0000","last_peered":"2023-09-30T22:21:37.198428+0000","last_clean":"2023-09-30T22:21:37.198428+0000","last_became_active":"2023-09-30T22:21:30.795885+0000","last_became_peered":"2023-09-30T22:21:30.795885+0000","last_unstale":"2023-09-30T22:21:37.198428+0000","last_undegraded":"2023-09-30T22:21:37.198428+0000","last_fullsized":"2023-09-30T22:21:37.198428+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_clean_scrub_stamp":"2023-09-30T22:21:29.785382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199764+0000","last_change":"2023-09-30T22:21:36.811325+0000","last_active":"2023-09-30T22:21:37.199764+0000","last_peered":"2023-09-30T22:21:37.199764+0000","last_clean":"2023-09-30T22:21:37.199764+0000","last_became_active":"2023-09-30T22:21:36.811014+0000","last_became_peered":"2023-09-30T22:21:36.811014+0000","last_unstale":"2023-09-30T22:21:37.199764+0000","last_undegraded":"2023-09-30T22:21:37.199764+0000","last_fullsized":"2023-09-30T22:21:37.199764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.226368+0000","last_change":"2023-09-30T22:21:37.220230+0000","last_active":"2023-09-30T22:21:37.226368+0000","last_peered":"2023-09-30T22:21:37.226368+0000","last_clean":"2023-09-30T22:21:37.226368+0000","last_became_active":"2023-09-30T22:21:37.215165+0000","last_became_peered":"2023-09-30T22:21:37.215165+0000","last_unstale":"2023-09-30T22:21:37.226368+0000","last_undegraded":"2023-09-30T22:21:37.226368+0000","last_fullsized":"2023-09-30T22:21:37.226368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.195816+0000","last_change":"2023-09-30T22:21:31.798735+0000","last_active":"2023-09-30T22:21:37.195816+0000","last_peered":"2023-09-30T22:21:37.195816+0000","last_clean":"2023-09-30T22:21:37.195816+0000","last_became_active":"2023-09-30T22:21:31.798405+0000","last_became_peered":"2023-09-30T22:21:31.798405+0000","last_unstale":"2023-09-30T22:21:37.195816+0000","last_undegraded":"2023-09-30T22:21:37.195816+0000","last_fullsized":"2023-09-30T22:21:37.195816+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.204992+0000","last_change":"2023-09-30T22:21:36.793235+0000","last_active":"2023-09-30T22:21:37.204992+0000","last_peered":"2023-09-30T22:21:37.204992+0000","last_clean":"2023-09-30T22:21:37.204992+0000","last_became_active":"2023-09-30T22:21:36.792921+0000","last_became_peered":"2023-09-30T22:21:36.792921+0000","last_unstale":"2023-09-30T22:21:37.204992+0000","last_undegraded":"2023-09-30T22:21:37.204992+0000","last_fullsized":"2023-09-30T22:21:37.204992+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233365+0000","last_change":"2023-09-30T22:21:37.224362+0000","last_active":"2023-09-30T22:21:37.233365+0000","last_peered":"2023-09-30T22:21:37.233365+0000","last_clean":"2023-09-30T22:21:37.233365+0000","last_became_active":"2023-09-30T22:21:37.219539+0000","last_became_peered":"2023-09-30T22:21:37.219539+0000","last_unstale":"2023-09-30T22:21:37.233365+0000","last_undegraded":"2023-09-30T22:21:37.233365+0000","last_fullsized":"2023-09-30T22:21:37.233365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.197166+0000","last_change":"2023-09-30T22:21:31.800121+0000","last_active":"2023-09-30T22:21:37.197166+0000","last_peered":"2023-09-30T22:21:37.197166+0000","last_clean":"2023-09-30T22:21:37.197166+0000","last_became_active":"2023-09-30T22:21:31.799775+0000","last_became_peered":"2023-09-30T22:21:31.799775+0000","last_unstale":"2023-09-30T22:21:37.197166+0000","last_undegraded":"2023-09-30T22:21:37.197166+0000","last_fullsized":"2023-09-30T22:21:37.197166+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.199098+0000","last_change":"2023-09-30T22:21:36.799552+0000","last_active":"2023-09-30T22:21:37.199098+0000","last_peered":"2023-09-30T22:21:37.199098+0000","last_clean":"2023-09-30T22:21:37.199098+0000","last_became_active":"2023-09-30T22:21:36.798979+0000","last_became_peered":"2023-09-30T22:21:36.798979+0000","last_unstale":"2023-09-30T22:21:37.199098+0000","last_undegraded":"2023-09-30T22:21:37.199098+0000","last_fullsized":"2023-09-30T22:21:37.199098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233846+0000","last_change":"2023-09-30T22:21:37.227699+0000","last_active":"2023-09-30T22:21:37.233846+0000","last_peered":"2023-09-30T22:21:37.233846+0000","last_clean":"2023-09-30T22:21:37.233846+0000","last_became_active":"2023-09-30T22:21:37.227403+0000","last_became_peered":"2023-09-30T22:21:37.227403+0000","last_unstale":"2023-09-30T22:21:37.233846+0000","last_undegraded":"2023-09-30T22:21:37.233846+0000","last_fullsized":"2023-09-30T22:21:37.233846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.233095+0000","last_change":"2023-09-30T22:21:36.817294+0000","last_active":"2023-09-30T22:21:37.233095+0000","last_peered":"2023-09-30T22:21:37.233095+0000","last_clean":"2023-09-30T22:21:37.233095+0000","last_became_active":"2023-09-30T22:21:36.816805+0000","last_became_peered":"2023-09-30T22:21:36.816805+0000","last_unstale":"2023-09-30T22:21:37.233095+0000","last_undegraded":"2023-09-30T22:21:37.233095+0000","last_fullsized":"2023-09-30T22:21:37.233095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.200191+0000","last_change":"2023-09-30T22:21:31.794560+0000","last_active":"2023-09-30T22:21:37.200191+0000","last_peered":"2023-09-30T22:21:37.200191+0000","last_clean":"2023-09-30T22:21:37.200191+0000","last_became_active":"2023-09-30T22:21:31.794101+0000","last_became_peered":"2023-09-30T22:21:31.794101+0000","last_unstale":"2023-09-30T22:21:37.200191+0000","last_undegraded":"2023-09-30T22:21:37.200191+0000","last_fullsized":"2023-09-30T22:21:37.200191+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223871+0000","last_change":"2023-09-30T22:21:37.214706+0000","last_active":"2023-09-30T22:21:37.223871+0000","last_peered":"2023-09-30T22:21:37.223871+0000","last_clean":"2023-09-30T22:21:37.223871+0000","last_became_active":"2023-09-30T22:21:37.212393+0000","last_became_peered":"2023-09-30T22:21:37.212393+0000","last_unstale":"2023-09-30T22:21:37.223871+0000","last_undegraded":"2023-09-30T22:21:37.223871+0000","last_fullsized":"2023-09-30T22:21:37.223871+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.196857+0000","last_change":"2023-09-30T22:21:31.800079+0000","last_active":"2023-09-30T22:21:37.196857+0000","last_peered":"2023-09-30T22:21:37.196857+0000","last_clean":"2023-09-30T22:21:37.196857+0000","last_became_active":"2023-09-30T22:21:31.799718+0000","last_became_peered":"2023-09-30T22:21:31.799718+0000","last_unstale":"2023-09-30T22:21:37.196857+0000","last_undegraded":"2023-09-30T22:21:37.196857+0000","last_fullsized":"2023-09-30T22:21:37.196857+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-09-30T22:21:38.198008+0000","last_change":"2023-09-30T22:21:36.803700+0000","last_active":"2023-09-30T22:21:38.198008+0000","last_peered":"2023-09-30T22:21:38.198008+0000","last_clean":"2023-09-30T22:21:38.198008+0000","last_became_active":"2023-09-30T22:21:36.803365+0000","last_became_peered":"2023-09-30T22:21:36.803365+0000","last_unstale":"2023-09-30T22:21:38.198008+0000","last_undegraded":"2023-09-30T22:21:38.198008+0000","last_fullsized":"2023-09-30T22:21:38.198008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232184+0000","last_change":"2023-09-30T22:21:37.226913+0000","last_active":"2023-09-30T22:21:37.232184+0000","last_peered":"2023-09-30T22:21:37.232184+0000","last_clean":"2023-09-30T22:21:37.232184+0000","last_became_active":"2023-09-30T22:21:37.225725+0000","last_became_peered":"2023-09-30T22:21:37.225725+0000","last_unstale":"2023-09-30T22:21:37.232184+0000","last_undegraded":"2023-09-30T22:21:37.232184+0000","last_fullsized":"2023-09-30T22:21:37.232184+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.230024+0000","last_change":"2023-09-30T22:21:36.808872+0000","last_active":"2023-09-30T22:21:37.230024+0000","last_peered":"2023-09-30T22:21:37.230024+0000","last_clean":"2023-09-30T22:21:37.230024+0000","last_became_active":"2023-09-30T22:21:36.806238+0000","last_became_peered":"2023-09-30T22:21:36.806238+0000","last_unstale":"2023-09-30T22:21:37.230024+0000","last_undegraded":"2023-09-30T22:21:37.230024+0000","last_fullsized":"2023-09-30T22:21:37.230024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223800+0000","last_change":"2023-09-30T22:21:37.208459+0000","last_active":"2023-09-30T22:21:37.223800+0000","last_peered":"2023-09-30T22:21:37.223800+0000","last_clean":"2023-09-30T22:21:37.223800+0000","last_became_active":"2023-09-30T22:21:37.208076+0000","last_became_peered":"2023-09-30T22:21:37.208076+0000","last_unstale":"2023-09-30T22:21:37.223800+0000","last_undegraded":"2023-09-30T22:21:37.223800+0000","last_fullsized":"2023-09-30T22:21:37.223800+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235981+0000","last_change":"2023-09-30T22:21:36.806969+0000","last_active":"2023-09-30T22:21:37.235981+0000","last_peered":"2023-09-30T22:21:37.235981+0000","last_clean":"2023-09-30T22:21:37.235981+0000","last_became_active":"2023-09-30T22:21:36.806649+0000","last_became_peered":"2023-09-30T22:21:36.806649+0000","last_unstale":"2023-09-30T22:21:37.235981+0000","last_undegraded":"2023-09-30T22:21:37.235981+0000","last_fullsized":"2023-09-30T22:21:37.235981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.232315+0000","last_change":"2023-09-30T22:21:37.229486+0000","last_active":"2023-09-30T22:21:37.232315+0000","last_peered":"2023-09-30T22:21:37.232315+0000","last_clean":"2023-09-30T22:21:37.232315+0000","last_became_active":"2023-09-30T22:21:37.226308+0000","last_became_peered":"2023-09-30T22:21:37.226308+0000","last_unstale":"2023-09-30T22:21:37.232315+0000","last_undegraded":"2023-09-30T22:21:37.232315+0000","last_fullsized":"2023-09-30T22:21:37.232315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235069+0000","last_change":"2023-09-30T22:21:36.815782+0000","last_active":"2023-09-30T22:21:37.235069+0000","last_peered":"2023-09-30T22:21:37.235069+0000","last_clean":"2023-09-30T22:21:37.235069+0000","last_became_active":"2023-09-30T22:21:36.815431+0000","last_became_peered":"2023-09-30T22:21:36.815431+0000","last_unstale":"2023-09-30T22:21:37.235069+0000","last_undegraded":"2023-09-30T22:21:37.235069+0000","last_fullsized":"2023-09-30T22:21:37.235069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.221951+0000","last_change":"2023-09-30T22:21:37.215011+0000","last_active":"2023-09-30T22:21:37.221951+0000","last_peered":"2023-09-30T22:21:37.221951+0000","last_clean":"2023-09-30T22:21:37.221951+0000","last_became_active":"2023-09-30T22:21:37.214739+0000","last_became_peered":"2023-09-30T22:21:37.214739+0000","last_unstale":"2023-09-30T22:21:37.221951+0000","last_undegraded":"2023-09-30T22:21:37.221951+0000","last_fullsized":"2023-09-30T22:21:37.221951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.235070+0000","last_change":"2023-09-30T22:21:36.812958+0000","last_active":"2023-09-30T22:21:37.235070+0000","last_peered":"2023-09-30T22:21:37.235070+0000","last_clean":"2023-09-30T22:21:37.235070+0000","last_became_active":"2023-09-30T22:21:36.812662+0000","last_became_peered":"2023-09-30T22:21:36.812662+0000","last_unstale":"2023-09-30T22:21:37.235070+0000","last_undegraded":"2023-09-30T22:21:37.235070+0000","last_fullsized":"2023-09-30T22:21:37.235070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234543+0000","last_change":"2023-09-30T22:21:37.225020+0000","last_active":"2023-09-30T22:21:37.234543+0000","last_peered":"2023-09-30T22:21:37.234543+0000","last_clean":"2023-09-30T22:21:37.234543+0000","last_became_active":"2023-09-30T22:21:37.224728+0000","last_became_peered":"2023-09-30T22:21:37.224728+0000","last_unstale":"2023-09-30T22:21:37.234543+0000","last_undegraded":"2023-09-30T22:21:37.234543+0000","last_fullsized":"2023-09-30T22:21:37.234543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.238512+0000","last_change":"2023-09-30T22:21:36.796108+0000","last_active":"2023-09-30T22:21:37.238512+0000","last_peered":"2023-09-30T22:21:37.238512+0000","last_clean":"2023-09-30T22:21:37.238512+0000","last_became_active":"2023-09-30T22:21:36.795665+0000","last_became_peered":"2023-09-30T22:21:36.795665+0000","last_unstale":"2023-09-30T22:21:37.238512+0000","last_undegraded":"2023-09-30T22:21:37.238512+0000","last_fullsized":"2023-09-30T22:21:37.238512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.217296+0000","last_change":"2023-09-30T22:21:37.207178+0000","last_active":"2023-09-30T22:21:37.217296+0000","last_peered":"2023-09-30T22:21:37.217296+0000","last_clean":"2023-09-30T22:21:37.217296+0000","last_became_active":"2023-09-30T22:21:37.206889+0000","last_became_peered":"2023-09-30T22:21:37.206889+0000","last_unstale":"2023-09-30T22:21:37.217296+0000","last_undegraded":"2023-09-30T22:21:37.217296+0000","last_fullsized":"2023-09-30T22:21:37.217296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.210508+0000","last_change":"2023-09-30T22:21:37.206551+0000","last_active":"2023-09-30T22:21:37.210508+0000","last_peered":"2023-09-30T22:21:37.210508+0000","last_clean":"2023-09-30T22:21:37.210508+0000","last_became_active":"2023-09-30T22:21:37.206176+0000","last_became_peered":"2023-09-30T22:21:37.206176+0000","last_unstale":"2023-09-30T22:21:37.210508+0000","last_undegraded":"2023-09-30T22:21:37.210508+0000","last_fullsized":"2023-09-30T22:21:37.210508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.225040+0000","last_change":"2023-09-30T22:21:37.220615+0000","last_active":"2023-09-30T22:21:37.225040+0000","last_peered":"2023-09-30T22:21:37.225040+0000","last_clean":"2023-09-30T22:21:37.225040+0000","last_became_active":"2023-09-30T22:21:37.217961+0000","last_became_peered":"2023-09-30T22:21:37.217961+0000","last_unstale":"2023-09-30T22:21:37.225040+0000","last_undegraded":"2023-09-30T22:21:37.225040+0000","last_fullsized":"2023-09-30T22:21:37.225040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.234039+0000","last_change":"2023-09-30T22:21:37.229799+0000","last_active":"2023-09-30T22:21:37.234039+0000","last_peered":"2023-09-30T22:21:37.234039+0000","last_clean":"2023-09-30T22:21:37.234039+0000","last_became_active":"2023-09-30T22:21:37.229494+0000","last_became_peered":"2023-09-30T22:21:37.229494+0000","last_unstale":"2023-09-30T22:21:37.234039+0000","last_undegraded":"2023-09-30T22:21:37.234039+0000","last_fullsized":"2023-09-30T22:21:37.234039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.223296+0000","last_change":"2023-09-30T22:21:37.215694+0000","last_active":"2023-09-30T22:21:37.223296+0000","last_peered":"2023-09-30T22:21:37.223296+0000","last_clean":"2023-09-30T22:21:37.223296+0000","last_became_active":"2023-09-30T22:21:37.215371+0000","last_became_peered":"2023-09-30T22:21:37.215371+0000","last_unstale":"2023-09-30T22:21:37.223296+0000","last_undegraded":"2023-09-30T22:21:37.223296+0000","last_fullsized":"2023-09-30T22:21:37.223296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:21:37.210455+0000","last_change":"2023-09-30T22:21:37.206797+0000","last_active":"2023-09-30T22:21:37.210455+0000","last_peered":"2023-09-30T22:21:37.210455+0000","last_clean":"2023-09-30T22:21:37.210455+0000","last_became_active":"2023-09-30T22:21:37.206384+0000","last_became_peered":"2023-09-30T22:21:37.206384+0000","last_unstale":"2023-09-30T22:21:37.210455+0000","last_undegraded":"2023-09-30T22:21:37.210455+0000","last_fullsized":"2023-09-30T22:21:37.210455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":47899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]}]},{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297264,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074576,"statfs":{"total":96636764160,"available":96332365824,"internally_reserved":0,"allocated":172032,"data_stored":48929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":47845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297252,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074588,"statfs":{"total":96636764160,"available":96332378112,"internally_reserved":0,"allocated":159744,"data_stored":48735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:21:45.524 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T22:21:45.525 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T22:21:45.525 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T22:21:45.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T22:21:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.694+0000 7f20d0c21700 1 Processor -- start 2023-09-30T22:21:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.694+0000 7f20d0c21700 1 -- start start 2023-09-30T22:21:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20d0c21700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20cc059070 0x7f20cc127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20d0c21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 0x7f20cc127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20d0c21700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc091b60 con 0x7f20cc095370 2023-09-30T22:21:45.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20d0c21700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc096940 con 0x7f20cc059070 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20d0c21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc059c00 con 0x7f20cc127390 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20c9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 0x7f20cc127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20ca59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20cc059070 0x7f20cc127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20c9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 0x7f20cc127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41084/0 (socket says 172.21.15.32:41084) 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20cad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f20cc095370 0x7f20cc095820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54820/0 (socket says 172.21.15.32:54820) 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20c9d9b700 1 -- 172.21.15.32:0/2254356340 learned_addr learned my addr 172.21.15.32:0/2254356340 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20ca59c700 1 -- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 msgr2=0x7f20cc127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.695+0000 7f20ca59c700 1 --2- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 0x7f20cc127d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20ca59c700 1 -- 172.21.15.32:0/2254356340 >> v1:172.21.15.32:6789/0 conn(0x7f20cc095370 legacy=0x7f20cc095820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20c959a700 1 -- 172.21.15.32:0/2254356340 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769026020 0 0) 0x7f20cc091b60 con 0x7f20cc095370 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20ca59c700 1 -- 172.21.15.32:0/2254356340 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0929d0 con 0x7f20cc059070 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20c9d9b700 1 --2- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 0x7f20cc127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20ca59c700 1 --2- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc059070 0x7f20cc127890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f20bc009760 tx=0x7f20bc009060).ready entity=mon.0 client_cookie=2b9ad77ed708f7bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20c959a700 1 -- 172.21.15.32:0/2254356340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0090d0 con 0x7f20cc059070 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20c959a700 1 -- 172.21.15.32:0/2254356340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc00f510 con 0x7f20cc059070 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20c959a700 1 -- 172.21.15.32:0/2254356340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00f890 con 0x7f20cc059070 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20d0c21700 1 -- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc059070 msgr2=0x7f20cc127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20d0c21700 1 --2- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc059070 0x7f20cc127890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f20bc009760 tx=0x7f20bc009060).stop 2023-09-30T22:21:45.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20d0c21700 1 -- 172.21.15.32:0/2254356340 shutdown_connections 2023-09-30T22:21:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20d0c21700 1 --2- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc059070 0x7f20cc127890 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.696+0000 7f20d0c21700 1 --2- 172.21.15.32:0/2254356340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc127390 0x7f20cc127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.697+0000 7f20d0c21700 1 -- 172.21.15.32:0/2254356340 >> 172.21.15.32:0/2254356340 conn(0x7f20cc097470 msgr2=0x7f20cc097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.697+0000 7f20d0c21700 1 -- 172.21.15.32:0/2254356340 shutdown_connections 2023-09-30T22:21:45.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.697+0000 7f20d0c21700 1 -- 172.21.15.32:0/2254356340 wait complete. 2023-09-30T22:21:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.697+0000 7f20d0c21700 1 Processor -- start 2023-09-30T22:21:45.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.697+0000 7f20d0c21700 1 -- start start 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20d0c21700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 0x7f20cc13a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20d0c21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc095370 0x7f20cc13a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20d0c21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 0x7f20cc114710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20d0c21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc115be0 con 0x7f20cc127390 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20d0c21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc137e20 con 0x7f20cc095370 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20d0c21700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc1269a0 con 0x7f20cc059070 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 0x7f20cc114710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20ca59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 0x7f20cc13a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 0x7f20cc114710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54730/0 (socket says 172.21.15.32:54730) 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 -- 172.21.15.32:0/1300719222 learned_addr learned my addr 172.21.15.32:0/1300719222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:45.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20ca59c700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 0x7f20cc13a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35736/0 (socket says 172.21.15.32:35736) 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20c9d9b700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc095370 0x7f20cc13a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 -- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 msgr2=0x7f20cc13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 0x7f20cc13a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 -- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc095370 msgr2=0x7f20cc13a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc095370 0x7f20cc13a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20cad9d700 1 -- 172.21.15.32:0/1300719222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4009370 con 0x7f20cc127390 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20c9d9b700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc095370 0x7f20cc13a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.698+0000 7f20ca59c700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 0x7f20cc13a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.699+0000 7f20cad9d700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 0x7f20cc114710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f20c400c7e0 tx=0x7f20c4007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.699+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4005c70 con 0x7f20cc127390 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.699+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c4005dd0 con 0x7f20cc127390 2023-09-30T22:21:45.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.699+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc0092a0 con 0x7f20cc127390 2023-09-30T22:21:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.699+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007320 con 0x7f20cc127390 2023-09-30T22:21:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.699+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc118810 con 0x7f20cc127390 2023-09-30T22:21:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.700+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f20c4007710 con 0x7f20cc127390 2023-09-30T22:21:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.700+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f20c403af80 con 0x7f20cc127390 2023-09-30T22:21:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.700+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc05a260 con 0x7f20cc127390 2023-09-30T22:21:45.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.705+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f20c401c3e0 con 0x7f20cc127390 2023-09-30T22:21:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.916+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f20cc046c20 con 0x7f20cc127390 2023-09-30T22:21:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.916+0000 7f20c27fc700 1 -- 172.21.15.32:0/1300719222 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f20c401a300 con 0x7f20cc127390 2023-09-30T22:21:45.917 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 >> v1:172.21.15.32:6816/118188 conn(0x7f20ac02bb60 legacy=0x7f20ac02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 msgr2=0x7f20cc114710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 0x7f20cc114710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f20c400c7e0 tx=0x7f20c4007f60).stop 2023-09-30T22:21:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 shutdown_connections 2023-09-30T22:21:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20cc127390 0x7f20cc114710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20cc059070 0x7f20cc13a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 --2- 172.21.15.32:0/1300719222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20cc095370 0x7f20cc13a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 >> 172.21.15.32:0/1300719222 conn(0x7f20cc097470 msgr2=0x7f20cc099a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 shutdown_connections 2023-09-30T22:21:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:45.919+0000 7f20d0c21700 1 -- 172.21.15.32:0/1300719222 wait complete. 2023-09-30T22:21:45.931 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T22:21:45.932 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-30T22:21:45.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:46.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 Processor -- start 2023-09-30T22:21:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 -- start start 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac40613a0 0x7f2ac412ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 0x7f2ac412b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 0x7f2ac4095df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac4061780 con 0x7f2ac412b660 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4067530 con 0x7f2ac40613a0 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.063+0000 7f2ac892d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4066ee0 con 0x7f2ac4097cb0 2023-09-30T22:21:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 0x7f2ac412b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 0x7f2ac4095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac40613a0 0x7f2ac412ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 0x7f2ac4095df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54746/0 (socket says 172.21.15.32:54746) 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 0x7f2ac412b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35740/0 (socket says 172.21.15.32:35740) 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac2d9d700 1 -- 172.21.15.32:0/631340223 learned_addr learned my addr 172.21.15.32:0/631340223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 -- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac40613a0 msgr2=0x7f2ac412ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 --2- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac40613a0 0x7f2ac412ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 -- 172.21.15.32:0/631340223 >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 msgr2=0x7f2ac4095df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 --2- 172.21.15.32:0/631340223 >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 0x7f2ac4095df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 -- 172.21.15.32:0/631340223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40962e0 con 0x7f2ac4097cb0 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac2d9d700 1 --2- 172.21.15.32:0/631340223 >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 0x7f2ac4095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac1d9b700 1 --2- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 0x7f2ac412b170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2ab8009760 tx=0x7f2ab8009060).ready entity=mon.2 client_cookie=e2bde0183492c9a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac159a700 1 -- 172.21.15.32:0/631340223 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800d4a0 con 0x7f2ac4097cb0 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac159a700 1 -- 172.21.15.32:0/631340223 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab800d600 con 0x7f2ac4097cb0 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.064+0000 7f2ac159a700 1 -- 172.21.15.32:0/631340223 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800d980 con 0x7f2ac4097cb0 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 -- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 msgr2=0x7f2ac412b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 --2- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 0x7f2ac412b170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2ab8009760 tx=0x7f2ab8009060).stop 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 -- 172.21.15.32:0/631340223 shutdown_connections 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 --2- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac40613a0 0x7f2ac412ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 --2- 172.21.15.32:0/631340223 >> v2:172.21.15.32:3300/0 conn(0x7f2ac412b660 0x7f2ac4095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 --2- 172.21.15.32:0/631340223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac4097cb0 0x7f2ac412b170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 -- 172.21.15.32:0/631340223 >> 172.21.15.32:0/631340223 conn(0x7f2ac409a220 msgr2=0x7f2ac409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 -- 172.21.15.32:0/631340223 shutdown_connections 2023-09-30T22:21:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.065+0000 7f2ac892d700 1 -- 172.21.15.32:0/631340223 wait complete. 2023-09-30T22:21:46.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 Processor -- start 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 -- start start 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 0x7f2ac413a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac4097cb0 0x7f2ac413aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ac412b660 0x7f2ac4114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac412f3a0 con 0x7f2ac412b660 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4098c70 con 0x7f2ac4097cb0 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac892d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4067110 con 0x7f2ac40613a0 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.066+0000 7f2ac259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 0x7f2ac413a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 0x7f2ac413a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35750/0 (socket says 172.21.15.32:35750) 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ac412b660 0x7f2ac4114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 -- 172.21.15.32:0/650559959 learned_addr learned my addr 172.21.15.32:0/650559959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac1d9b700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac4097cb0 0x7f2ac413aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 -- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac4097cb0 msgr2=0x7f2ac413aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac4097cb0 0x7f2ac413aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 -- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ac412b660 msgr2=0x7f2ac4114ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ac412b660 0x7f2ac4114ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 -- 172.21.15.32:0/650559959 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc009ae0 con 0x7f2ac40613a0 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac2d9d700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ac412b660 0x7f2ac4114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac259c700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 0x7f2ac413a5e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f2ab400e930 tx=0x7f2ab4009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4012600 con 0x7f2ac40613a0 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4012760 con 0x7f2ac40613a0 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab80092a0 con 0x7f2ac40613a0 2023-09-30T22:21:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4012ae0 con 0x7f2ac40613a0 2023-09-30T22:21:46.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.067+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac41190b0 con 0x7f2ac40613a0 2023-09-30T22:21:46.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.068+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2ab401b400 con 0x7f2ac40613a0 2023-09-30T22:21:46.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.068+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2ab403b4e0 con 0x7f2ac40613a0 2023-09-30T22:21:46.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.068+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac405a260 con 0x7f2ac40613a0 2023-09-30T22:21:46.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.073+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2ab4019900 con 0x7f2ac40613a0 2023-09-30T22:21:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.250+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ac4118810 con 0x7f2ac40613a0 2023-09-30T22:21:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.251+0000 7f2ab27fc700 1 -- 172.21.15.32:0/650559959 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f2ab4019310 con 0x7f2ac40613a0 2023-09-30T22:21:46.251 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:46.251 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:38.189468+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 >> v1:172.21.15.32:6816/118188 conn(0x7f2aa402bb60 legacy=0x7f2aa402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 msgr2=0x7f2ac413a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 0x7f2ac413a5e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f2ab400e930 tx=0x7f2ab4009060).stop 2023-09-30T22:21:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 shutdown_connections 2023-09-30T22:21:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ac412b660 0x7f2ac4114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ac40613a0 0x7f2ac413a5e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 --2- 172.21.15.32:0/650559959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ac4097cb0 0x7f2ac413aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 >> 172.21.15.32:0/650559959 conn(0x7f2ac409a220 msgr2=0x7f2ac4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 shutdown_connections 2023-09-30T22:21:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.253+0000 7f2ac892d700 1 -- 172.21.15.32:0/650559959 wait complete. 2023-09-30T22:21:46.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:21:46.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:21:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.400+0000 7f7e399ea700 1 Processor -- start 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.400+0000 7f7e399ea700 1 -- start start 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e399ea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e34098d50 0x7f7e34099130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e399ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 0x7f7e34132830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e399ea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e340663b0 con 0x7f7e340653a0 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e399ea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e34065e80 con 0x7f7e34098d50 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e399ea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e34065780 con 0x7f7e34130440 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 0x7f7e34132830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e327fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e34098d50 0x7f7e34099130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 0x7f7e34132830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41132/0 (socket says 172.21.15.32:41132) 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e32ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e340653a0 0x7f7e34098640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54830/0 (socket says 172.21.15.32:54830) 2023-09-30T22:21:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e337fe700 1 -- 172.21.15.32:0/1968932096 learned_addr learned my addr 172.21.15.32:0/1968932096 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e327fc700 1 -- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 msgr2=0x7f7e34132830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e327fc700 1 --2- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 0x7f7e34132830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e327fc700 1 -- 172.21.15.32:0/1968932096 >> v1:172.21.15.32:6789/0 conn(0x7f7e340653a0 legacy=0x7f7e34098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e327fc700 1 -- 172.21.15.32:0/1968932096 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34096180 con 0x7f7e34098d50 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e31ffb700 1 -- 172.21.15.32:0/1968932096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896585854 0 0) 0x7f7e340663b0 con 0x7f7e340653a0 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e337fe700 1 --2- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 0x7f7e34132830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.401+0000 7f7e327fc700 1 --2- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34098d50 0x7f7e34099130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f7e1c009760 tx=0x7f7e1c009060).ready entity=mon.0 client_cookie=5e05eba7b54e7d9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e31ffb700 1 -- 172.21.15.32:0/1968932096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00f510 con 0x7f7e34098d50 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e31ffb700 1 -- 172.21.15.32:0/1968932096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c00f670 con 0x7f7e34098d50 2023-09-30T22:21:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e31ffb700 1 -- 172.21.15.32:0/1968932096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00f9f0 con 0x7f7e34098d50 2023-09-30T22:21:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 -- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34098d50 msgr2=0x7f7e34099130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 --2- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34098d50 0x7f7e34099130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f7e1c009760 tx=0x7f7e1c009060).stop 2023-09-30T22:21:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 -- 172.21.15.32:0/1968932096 shutdown_connections 2023-09-30T22:21:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 --2- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34130440 0x7f7e34132830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 --2- 172.21.15.32:0/1968932096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34098d50 0x7f7e34099130 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 -- 172.21.15.32:0/1968932096 >> 172.21.15.32:0/1968932096 conn(0x7f7e3409dc20 msgr2=0x7f7e3409e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 -- 172.21.15.32:0/1968932096 shutdown_connections 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.402+0000 7f7e399ea700 1 -- 172.21.15.32:0/1968932096 wait complete. 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.404+0000 7f7e399ea700 1 Processor -- start 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.404+0000 7f7e399ea700 1 -- start start 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.404+0000 7f7e399ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 0x7f7e3411e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.404+0000 7f7e399ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34098d50 0x7f7e3411a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e399ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34130440 0x7f7e3411a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e399ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e34099a90 con 0x7f7e34130440 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e399ea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3411d850 con 0x7f7e34098d50 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e399ea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e34094830 con 0x7f7e340653a0 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34130440 0x7f7e3411a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 0x7f7e3411e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34130440 0x7f7e3411a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54774/0 (socket says 172.21.15.32:54774) 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 0x7f7e3411e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35756/0 (socket says 172.21.15.32:35756) 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e337fe700 1 -- 172.21.15.32:0/953788249 learned_addr learned my addr 172.21.15.32:0/953788249 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e327fc700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34098d50 0x7f7e3411a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 -- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34098d50 msgr2=0x7f7e3411a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34098d50 0x7f7e3411a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 -- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34130440 msgr2=0x7f7e3411a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34130440 0x7f7e3411a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 -- 172.21.15.32:0/953788249 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c009ae0 con 0x7f7e340653a0 2023-09-30T22:21:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e327fc700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34098d50 0x7f7e3411a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:21:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e32ffd700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 0x7f7e3411e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7e24000c00 tx=0x7f7e24007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.405+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240072e0 con 0x7f7e340653a0 2023-09-30T22:21:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.406+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24007440 con 0x7f7e340653a0 2023-09-30T22:21:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.406+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24005930 con 0x7f7e340653a0 2023-09-30T22:21:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.406+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c0092a0 con 0x7f7e340653a0 2023-09-30T22:21:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.406+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3411ddf0 con 0x7f7e340653a0 2023-09-30T22:21:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.406+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7e24005ce0 con 0x7f7e340653a0 2023-09-30T22:21:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.407+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7e2403a6f0 con 0x7f7e340653a0 2023-09-30T22:21:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.407+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3405a260 con 0x7f7e340653a0 2023-09-30T22:21:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.411+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7e2401d3f0 con 0x7f7e340653a0 2023-09-30T22:21:46.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.584+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7e3411b0c0 con 0x7f7e340653a0 2023-09-30T22:21:46.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.585+0000 7f7e2affd700 1 -- 172.21.15.32:0/953788249 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7e2401c350 con 0x7f7e340653a0 2023-09-30T22:21:46.585 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:21:46.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.586+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 >> v1:172.21.15.32:6816/118188 conn(0x7f7e1402bb60 legacy=0x7f7e1402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.586+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 msgr2=0x7f7e3411e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.586+0000 7f7e399ea700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 0x7f7e3411e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7e24000c00 tx=0x7f7e24007bb0).stop 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 shutdown_connections 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e34130440 0x7f7e3411a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e340653a0 0x7f7e3411e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 --2- 172.21.15.32:0/953788249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e34098d50 0x7f7e3411a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 >> 172.21.15.32:0/953788249 conn(0x7f7e3409dc20 msgr2=0x7f7e3409bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 shutdown_connections 2023-09-30T22:21:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.587+0000 7f7e399ea700 1 -- 172.21.15.32:0/953788249 wait complete. 2023-09-30T22:21:46.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:21:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.732+0000 7fe401c25700 1 Processor -- start 2023-09-30T22:21:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe401c25700 1 -- start start 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe401c25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 0x7fe3fc12b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe401c25700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 0x7fe3fc132130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe401c25700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc09a770 con 0x7fe3fc059070 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe401c25700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc059450 con 0x7fe3fc12fd40 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe401c25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc091ed0 con 0x7fe3fc12a9b0 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe3faffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 0x7fe3fc12b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe3fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 0x7fe3fc132130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe3fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 0x7fe3fc132130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54786/0 (socket says 172.21.15.32:54786) 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe3faffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 0x7fe3fc12b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41152/0 (socket says 172.21.15.32:41152) 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe3fb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3fc059070 0x7fe3fc12ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54832/0 (socket says 172.21.15.32:54832) 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.733+0000 7fe3fbfff700 1 -- 172.21.15.32:0/4070920091 learned_addr learned my addr 172.21.15.32:0/4070920091 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3faffd700 1 -- 172.21.15.32:0/4070920091 >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 msgr2=0x7fe3fc132130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3faffd700 1 --2- 172.21.15.32:0/4070920091 >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 0x7fe3fc132130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3faffd700 1 -- 172.21.15.32:0/4070920091 >> v1:172.21.15.32:6789/0 conn(0x7fe3fc059070 legacy=0x7fe3fc12ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3fa7fc700 1 -- 172.21.15.32:0/4070920091 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996756487 0 0) 0x7fe3fc09a770 con 0x7fe3fc059070 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3faffd700 1 -- 172.21.15.32:0/4070920091 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc091c10 con 0x7fe3fc12a9b0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3fbfff700 1 --2- 172.21.15.32:0/4070920091 >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 0x7fe3fc132130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3faffd700 1 --2- 172.21.15.32:0/4070920091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 0x7fe3fc12b550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe3f0009790 tx=0x7fe3f0009060).ready entity=mon.1 client_cookie=a54f63d8e90d5cd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3fa7fc700 1 -- 172.21.15.32:0/4070920091 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00090d0 con 0x7fe3fc12a9b0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3fa7fc700 1 -- 172.21.15.32:0/4070920091 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000f510 con 0x7fe3fc12a9b0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.734+0000 7fe3fa7fc700 1 -- 172.21.15.32:0/4070920091 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000f890 con 0x7fe3fc12a9b0 2023-09-30T22:21:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 -- 172.21.15.32:0/4070920091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 msgr2=0x7fe3fc12b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 --2- 172.21.15.32:0/4070920091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 0x7fe3fc12b550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe3f0009790 tx=0x7fe3f0009060).stop 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 -- 172.21.15.32:0/4070920091 shutdown_connections 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 --2- 172.21.15.32:0/4070920091 >> v2:172.21.15.32:3300/0 conn(0x7fe3fc12fd40 0x7fe3fc132130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 --2- 172.21.15.32:0/4070920091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12a9b0 0x7fe3fc12b550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 -- 172.21.15.32:0/4070920091 >> 172.21.15.32:0/4070920091 conn(0x7fe3fc097480 msgr2=0x7fe3fc097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 -- 172.21.15.32:0/4070920091 shutdown_connections 2023-09-30T22:21:46.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.735+0000 7fe401c25700 1 -- 172.21.15.32:0/4070920091 wait complete. 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 Processor -- start 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 -- start start 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 0x7fe3fc12bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3fc12fd40 0x7fe3fc12c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 0x7fe3fc140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc12d380 con 0x7fe3fc059070 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc12dad0 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe401c25700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc1323c0 con 0x7fe3fc12fd40 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3fbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 0x7fe3fc140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3faffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3fc12fd40 0x7fe3fc12c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 0x7fe3fc12bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 0x7fe3fc12bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54800/0 (socket says 172.21.15.32:54800) 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3fbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 0x7fe3fc140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41166/0 (socket says 172.21.15.32:41166) 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3faffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3fc12fd40 0x7fe3fc12c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35760/0 (socket says 172.21.15.32:35760) 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.736+0000 7fe3fb7fe700 1 -- 172.21.15.32:0/20325547 learned_addr learned my addr 172.21.15.32:0/20325547 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fbfff700 1 -- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3fc12fd40 msgr2=0x7fe3fc12c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fbfff700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3fc12fd40 0x7fe3fc12c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fbfff700 1 -- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 msgr2=0x7fe3fc12bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fbfff700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 0x7fe3fc12bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fbfff700 1 -- 172.21.15.32:0/20325547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f4009ae0 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fb7fe700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 0x7fe3fc12bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3fbfff700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 0x7fe3fc140910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe3f4009ee0 tx=0x7fe3f400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40092e0 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f400f040 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f00092a0 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400cc20 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.737+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc12ef10 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.738+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe3f4014460 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.738+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fe3f403b060 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.738+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc12f130 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.743+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe3f4012360 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.917+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3fc05a260 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.917+0000 7fe3e37fe700 1 -- 172.21.15.32:0/20325547 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3f401cb10 con 0x7fe3fc12c8f0 2023-09-30T22:21:46.917 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:21:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 >> v1:172.21.15.32:6816/118188 conn(0x7fe3e802bb60 legacy=0x7fe3e802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 msgr2=0x7fe3fc140910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 0x7fe3fc140910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe3f4009ee0 tx=0x7fe3f400b040).stop 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 shutdown_connections 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3fc12c8f0 0x7fe3fc140910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3fc059070 0x7fe3fc12bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 --2- 172.21.15.32:0/20325547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3fc12fd40 0x7fe3fc12c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 >> 172.21.15.32:0/20325547 conn(0x7fe3fc097480 msgr2=0x7fe3fc092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 shutdown_connections 2023-09-30T22:21:46.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:46.919+0000 7fe401c25700 1 -- 172.21.15.32:0/20325547 wait complete. 2023-09-30T22:21:46.933 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:21:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.066+0000 7f964fdd3700 1 Processor -- start 2023-09-30T22:21:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.066+0000 7f964fdd3700 1 -- start start 2023-09-30T22:21:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964fdd3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964812c3c0 0x7f964812bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964fdd3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648137140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964fdd3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964809a050 con 0x7f96480925d0 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964fdd3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964806a750 con 0x7f964812c3c0 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964fdd3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964812c1a0 con 0x7f964812f540 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964e370700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964db6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964812c3c0 0x7f964812bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964d36e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96480925d0 0x7f964812ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54840/0 (socket says 172.21.15.32:54840) 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964e370700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648137140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35764/0 (socket says 172.21.15.32:35764) 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964d36e700 1 -- 172.21.15.32:0/3858548716 learned_addr learned my addr 172.21.15.32:0/3858548716 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964db6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f964812c3c0 0x7f964812bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54802/0 (socket says 172.21.15.32:54802) 2023-09-30T22:21:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964db6f700 1 -- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 msgr2=0x7f9648137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964db6f700 1 --2- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648137140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964db6f700 1 -- 172.21.15.32:0/3858548716 >> v1:172.21.15.32:6789/0 conn(0x7f96480925d0 legacy=0x7f964812ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964cb6d700 1 -- 172.21.15.32:0/3858548716 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582638312 0 0) 0x7f964809a050 con 0x7f96480925d0 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964db6f700 1 -- 172.21.15.32:0/3858548716 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648091ed0 con 0x7f964812c3c0 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.067+0000 7f964e370700 1 --2- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964db6f700 1 --2- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964812c3c0 0x7f964812bb80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f9644009760 tx=0x7f9644009060).ready entity=mon.0 client_cookie=f91f1689eb88f390 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964cb6d700 1 -- 172.21.15.32:0/3858548716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400f510 con 0x7f964812c3c0 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964cb6d700 1 -- 172.21.15.32:0/3858548716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964400f670 con 0x7f964812c3c0 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964cb6d700 1 -- 172.21.15.32:0/3858548716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400f9f0 con 0x7f964812c3c0 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 -- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964812c3c0 msgr2=0x7f964812bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964812c3c0 0x7f964812bb80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f9644009760 tx=0x7f9644009060).stop 2023-09-30T22:21:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 -- 172.21.15.32:0/3858548716 shutdown_connections 2023-09-30T22:21:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3858548716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f964812c3c0 0x7f964812bb80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 -- 172.21.15.32:0/3858548716 >> 172.21.15.32:0/3858548716 conn(0x7f9648097480 msgr2=0x7f9648097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 -- 172.21.15.32:0/3858548716 shutdown_connections 2023-09-30T22:21:47.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.068+0000 7f964fdd3700 1 -- 172.21.15.32:0/3858548716 wait complete. 2023-09-30T22:21:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.070+0000 7f964fdd3700 1 Processor -- start 2023-09-30T22:21:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.070+0000 7f964fdd3700 1 -- start start 2023-09-30T22:21:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.070+0000 7f964fdd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 0x7f9648068530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.070+0000 7f964fdd3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f964812c3c0 0x7f9648068a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.070+0000 7f964fdd3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648068f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964fdd3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964812f100 con 0x7f96480925d0 2023-09-30T22:21:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964fdd3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964809a280 con 0x7f964812c3c0 2023-09-30T22:21:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964fdd3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964811a5b0 con 0x7f964812f540 2023-09-30T22:21:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 0x7f9648068530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 0x7f9648068530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54808/0 (socket says 172.21.15.32:54808) 2023-09-30T22:21:47.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 -- 172.21.15.32:0/3679861805 learned_addr learned my addr 172.21.15.32:0/3679861805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964e370700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964d36e700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f964812c3c0 0x7f9648068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 -- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 msgr2=0x7f9648068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 -- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f964812c3c0 msgr2=0x7f9648068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f964812c3c0 0x7f9648068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 -- 172.21.15.32:0/3679861805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640009a20 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964e370700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964db6f700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 0x7f9648068530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9644009e00 tx=0x7f9644017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400f7e0 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964400f940 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96440092a0 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.071+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964401b660 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.072+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964811af70 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.072+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f964401b9d0 con 0x7f96480925d0 2023-09-30T22:21:47.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.072+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f964403c8c0 con 0x7f96480925d0 2023-09-30T22:21:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.073+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9648064cb0 con 0x7f96480925d0 2023-09-30T22:21:47.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.077+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9644029080 con 0x7f96480925d0 2023-09-30T22:21:47.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.252+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f964805a260 con 0x7f96480925d0 2023-09-30T22:21:47.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.252+0000 7f9639ffb700 1 -- 172.21.15.32:0/3679861805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f964401f020 con 0x7f96480925d0 2023-09-30T22:21:47.252 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:21:47.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 >> v1:172.21.15.32:6816/118188 conn(0x7f962c02bb90 legacy=0x7f962c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 msgr2=0x7f9648068530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 0x7f9648068530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f9644009e00 tx=0x7f9644017000).stop 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 shutdown_connections 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f964812f540 0x7f9648068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96480925d0 0x7f9648068530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 --2- 172.21.15.32:0/3679861805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f964812c3c0 0x7f9648068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.254+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 >> 172.21.15.32:0/3679861805 conn(0x7f9648097480 msgr2=0x7f9648132190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.255+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 shutdown_connections 2023-09-30T22:21:47.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.255+0000 7f964fdd3700 1 -- 172.21.15.32:0/3679861805 wait complete. 2023-09-30T22:21:47.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:21:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 Processor -- start 2023-09-30T22:21:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 -- start start 2023-09-30T22:21:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb80653a0 0x7f7bb8098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bb8098b30 0x7f7bb8098f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 0x7f7bb8131820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb8065e80 con 0x7f7bb8098b30 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80663b0 con 0x7f7bb812f430 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.401+0000 7f7bbee2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8065780 con 0x7f7bb80653a0 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bb8098b30 0x7f7bb8098f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bbd3ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 0x7f7bb8131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bbcbc9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb80653a0 0x7f7bb8098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bb8098b30 0x7f7bb8098f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54814/0 (socket says 172.21.15.32:54814) 2023-09-30T22:21:47.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bbd3ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 0x7f7bb8131820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41192/0 (socket says 172.21.15.32:41192) 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 -- 172.21.15.32:0/1602375770 learned_addr learned my addr 172.21.15.32:0/1602375770 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 -- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb80653a0 msgr2=0x7f7bb8098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb80653a0 0x7f7bb8098640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 -- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 msgr2=0x7f7bb8131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 0x7f7bb8131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 -- 172.21.15.32:0/1602375770 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8096180 con 0x7f7bb8098b30 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bbd3ca700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 0x7f7bb8131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb7fff700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8098b30 0x7f7bb8098f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f7ba4006fd0 tx=0x7f7ba40090f0).ready entity=mon.0 client_cookie=2b19fc0ec17ef733 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb77fe700 1 -- 172.21.15.32:0/1602375770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400f510 con 0x7f7bb8098b30 2023-09-30T22:21:47.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb77fe700 1 -- 172.21.15.32:0/1602375770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400f670 con 0x7f7bb8098b30 2023-09-30T22:21:47.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.402+0000 7f7bb77fe700 1 -- 172.21.15.32:0/1602375770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400f9f0 con 0x7f7bb8098b30 2023-09-30T22:21:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 -- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8098b30 msgr2=0x7f7bb8098f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8098b30 0x7f7bb8098f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f7ba4006fd0 tx=0x7f7ba40090f0).stop 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 -- 172.21.15.32:0/1602375770 shutdown_connections 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb812f430 0x7f7bb8131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb80653a0 0x7f7bb8098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/1602375770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb8098b30 0x7f7bb8098f10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 -- 172.21.15.32:0/1602375770 >> 172.21.15.32:0/1602375770 conn(0x7f7bb809dc20 msgr2=0x7f7bb809e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 -- 172.21.15.32:0/1602375770 shutdown_connections 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.403+0000 7f7bbee2d700 1 -- 172.21.15.32:0/1602375770 wait complete. 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.404+0000 7f7bbee2d700 1 Processor -- start 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.404+0000 7f7bbee2d700 1 -- start start 2023-09-30T22:21:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 0x7f7bb811a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb8098b30 0x7f7bb811a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb812f430 0x7f7bb811ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb811b1d0 con 0x7f7bb80653a0 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb811d530 con 0x7f7bb8098b30 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb811b750 con 0x7f7bb812f430 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 0x7f7bb811a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 0x7f7bb811a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54828/0 (socket says 172.21.15.32:54828) 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 -- 172.21.15.32:0/2606106686 learned_addr learned my addr 172.21.15.32:0/2606106686 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bb7fff700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb8098b30 0x7f7bb811a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbd3ca700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb812f430 0x7f7bb811ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 -- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb812f430 msgr2=0x7f7bb811ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb812f430 0x7f7bb811ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 -- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb8098b30 msgr2=0x7f7bb811a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb8098b30 0x7f7bb811a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 -- 172.21.15.32:0/2606106686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0009a20 con 0x7f7bb80653a0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbcbc9700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 0x7f7bb811a300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f7bac00e960 tx=0x7f7bac009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac009510 con 0x7f7bb80653a0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac012600 con 0x7f7bb80653a0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.405+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba4009370 con 0x7f7bb80653a0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.406+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac012980 con 0x7f7bb80653a0 2023-09-30T22:21:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.406+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb8099a90 con 0x7f7bb80653a0 2023-09-30T22:21:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.406+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7bac012d30 con 0x7f7bb80653a0 2023-09-30T22:21:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.406+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7bac03b3e0 con 0x7f7bb80653a0 2023-09-30T22:21:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.407+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb805a260 con 0x7f7bb80653a0 2023-09-30T22:21:47.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.411+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7bac011b00 con 0x7f7bb80653a0 2023-09-30T22:21:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.583+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7bb8097400 con 0x7f7bb80653a0 2023-09-30T22:21:47.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.583+0000 7f7bb4ff9700 1 -- 172.21.15.32:0/2606106686 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7bac011b00 con 0x7f7bb80653a0 2023-09-30T22:21:47.583 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:21:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 >> v1:172.21.15.32:6816/118188 conn(0x7f7b9c02bb60 legacy=0x7f7b9c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 msgr2=0x7f7bb811a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 0x7f7bb811a300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f7bac00e960 tx=0x7f7bac009060).stop 2023-09-30T22:21:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 shutdown_connections 2023-09-30T22:21:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bb812f430 0x7f7bb811ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bb80653a0 0x7f7bb811a300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 --2- 172.21.15.32:0/2606106686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bb8098b30 0x7f7bb811a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 >> 172.21.15.32:0/2606106686 conn(0x7f7bb809dc20 msgr2=0x7f7bb809bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 shutdown_connections 2023-09-30T22:21:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.585+0000 7f7bbee2d700 1 -- 172.21.15.32:0/2606106686 wait complete. 2023-09-30T22:21:47.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.729+0000 7febc7588700 1 Processor -- start 2023-09-30T22:21:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.729+0000 7febc7588700 1 -- start start 2023-09-30T22:21:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc7588700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0123e70 0x7febc0067ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc7588700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0092340 0x7febc0127680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc7588700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc0068880 con 0x7febc0123a90 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc7588700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc0124250 con 0x7febc0092340 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc7588700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc0068c90 con 0x7febc0123e70 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc5324700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7febc0123a90 0x7febc0126f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54856/0 (socket says 172.21.15.32:54856) 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc4b23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0123e70 0x7febc0067ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc5324700 1 -- 172.21.15.32:0/1155497546 learned_addr learned my addr 172.21.15.32:0/1155497546 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc5b25700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0092340 0x7febc0127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febb7fff700 1 -- 172.21.15.32:0/1155497546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205205790 0 0) 0x7febc0068880 con 0x7febc0123a90 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febb7fff700 1 -- 172.21.15.32:0/1155497546 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febac0033f0 con 0x7febc0123a90 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc4b23700 1 -- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0092340 msgr2=0x7febc0127680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.730+0000 7febc4b23700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0092340 0x7febc0127680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc4b23700 1 -- 172.21.15.32:0/1155497546 >> v1:172.21.15.32:6789/0 conn(0x7febc0123a90 legacy=0x7febc0126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc4b23700 1 -- 172.21.15.32:0/1155497546 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0124430 con 0x7febc0123e70 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc5b25700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0092340 0x7febc0127680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc4b23700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0123e70 0x7febc0067ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7febb00097c0 tx=0x7febb0009060).ready entity=mon.2 client_cookie=8fd279ec473da2ef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febb7fff700 1 -- 172.21.15.32:0/1155497546 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000d4a0 con 0x7febc0123e70 2023-09-30T22:21:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febb7fff700 1 -- 172.21.15.32:0/1155497546 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb000d600 con 0x7febc0123e70 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febb7fff700 1 -- 172.21.15.32:0/1155497546 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000d980 con 0x7febc0123e70 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc7588700 1 -- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0123e70 msgr2=0x7febc0067ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc7588700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0123e70 0x7febc0067ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7febb00097c0 tx=0x7febb0009060).stop 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc7588700 1 -- 172.21.15.32:0/1155497546 shutdown_connections 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc7588700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0092340 0x7febc0127680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc7588700 1 --2- 172.21.15.32:0/1155497546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0123e70 0x7febc0067ce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.731+0000 7febc7588700 1 -- 172.21.15.32:0/1155497546 >> 172.21.15.32:0/1155497546 conn(0x7febc009a220 msgr2=0x7febc009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.732+0000 7febc7588700 1 -- 172.21.15.32:0/1155497546 shutdown_connections 2023-09-30T22:21:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.732+0000 7febc7588700 1 -- 172.21.15.32:0/1155497546 wait complete. 2023-09-30T22:21:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.732+0000 7febc7588700 1 Processor -- start 2023-09-30T22:21:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 -- start start 2023-09-30T22:21:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 0x7febc0122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febc0123a90 0x7febc0122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0123e70 0x7febc0115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01193f0 con 0x7febc0123a90 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc00697c0 con 0x7febc0123e70 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc7588700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc01274f0 con 0x7febc0092340 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc5324700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 0x7febc0122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc4b23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febc0123a90 0x7febc0122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc5324700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 0x7febc0122820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35820/0 (socket says 172.21.15.32:35820) 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc5324700 1 -- 172.21.15.32:0/1638258622 learned_addr learned my addr 172.21.15.32:0/1638258622 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.733+0000 7febc5b25700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0123e70 0x7febc0115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc4b23700 1 -- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 msgr2=0x7febc0122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc4b23700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 0x7febc0122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc4b23700 1 -- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0123e70 msgr2=0x7febc0115c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc4b23700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0123e70 0x7febc0115c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc4b23700 1 -- 172.21.15.32:0/1638258622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb800ab30 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc5324700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 0x7febc0122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc4b23700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febc0123a90 0x7febc0122d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7febb0013000 tx=0x7febb0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000dd30 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb000de90 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000d9c0 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb00092a0 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.734+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc0119130 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.735+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7febb001d640 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.735+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7febb003b900 con 0x7febc0123a90 2023-09-30T22:21:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.735+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc0119540 con 0x7febc0123a90 2023-09-30T22:21:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.740+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7febb0022020 con 0x7febc0123a90 2023-09-30T22:21:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.960+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc005a260 con 0x7febc0123a90 2023-09-30T22:21:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.961+0000 7febb57fa700 1 -- 172.21.15.32:0/1638258622 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7febb001d8f0 con 0x7febc0123a90 2023-09-30T22:21:47.961 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:47.961 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:40.197617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 >> v1:172.21.15.32:6816/118188 conn(0x7feba402bb90 legacy=0x7feba402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febc0123a90 msgr2=0x7febc0122d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febc0123a90 0x7febc0122d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7febb0013000 tx=0x7febb0016000).stop 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 shutdown_connections 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febc0123e70 0x7febc0115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febc0092340 0x7febc0122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 --2- 172.21.15.32:0/1638258622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febc0123a90 0x7febc0122d10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 >> 172.21.15.32:0/1638258622 conn(0x7febc009a220 msgr2=0x7febc0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 shutdown_connections 2023-09-30T22:21:47.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:47.963+0000 7febc7588700 1 -- 172.21.15.32:0/1638258622 wait complete. 2023-09-30T22:21:47.964 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2023-09-30T22:21:47.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 Processor -- start 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 -- start start 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd881251c0 0x7efd8812da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd88125dd0 0x7efd8812ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd88095f00 con 0x7efd88059070 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd88059450 con 0x7efd881251c0 2023-09-30T22:21:48.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8e7e2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd88096190 con 0x7efd88125dd0 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd8cd7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd88125dd0 0x7efd8812ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd877fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd881251c0 0x7efd8812da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd87fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efd88059070 0x7efd881256c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54870/0 (socket says 172.21.15.32:54870) 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.106+0000 7efd87fff700 1 -- 172.21.15.32:0/3165296888 learned_addr learned my addr 172.21.15.32:0/3165296888 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd877fe700 1 -- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd88125dd0 msgr2=0x7efd8812ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd877fe700 1 --2- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd88125dd0 0x7efd8812ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd877fe700 1 -- 172.21.15.32:0/3165296888 >> v1:172.21.15.32:6789/0 conn(0x7efd88059070 legacy=0x7efd881256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd877fe700 1 -- 172.21.15.32:0/3165296888 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88059dd0 con 0x7efd881251c0 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd86ffd700 1 -- 172.21.15.32:0/3165296888 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098200527 0 0) 0x7efd88095f00 con 0x7efd88059070 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd8cd7f700 1 --2- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd88125dd0 0x7efd8812ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd877fe700 1 --2- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd881251c0 0x7efd8812da80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7efd7c009760 tx=0x7efd7c009060).ready entity=mon.0 client_cookie=2a967363cc89f43e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd86ffd700 1 -- 172.21.15.32:0/3165296888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00f510 con 0x7efd881251c0 2023-09-30T22:21:48.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd86ffd700 1 -- 172.21.15.32:0/3165296888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c00f670 con 0x7efd881251c0 2023-09-30T22:21:48.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.107+0000 7efd86ffd700 1 -- 172.21.15.32:0/3165296888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00f9f0 con 0x7efd881251c0 2023-09-30T22:21:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd881251c0 msgr2=0x7efd8812da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd881251c0 0x7efd8812da80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7efd7c009760 tx=0x7efd7c009060).stop 2023-09-30T22:21:48.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3165296888 shutdown_connections 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd88125dd0 0x7efd8812ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3165296888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd881251c0 0x7efd8812da80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3165296888 >> 172.21.15.32:0/3165296888 conn(0x7efd88097470 msgr2=0x7efd88097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3165296888 shutdown_connections 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.108+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3165296888 wait complete. 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 Processor -- start 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 -- start start 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88059070 0x7efd8812dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 0x7efd8812e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd88125dd0 0x7efd8812e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88111130 con 0x7efd88059070 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8812fcf0 con 0x7efd881251c0 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8e7e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd880958d0 con 0x7efd88125dd0 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 0x7efd8812e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd87fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88059070 0x7efd8812dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd8cd7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd88125dd0 0x7efd8812e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 0x7efd8812e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41232/0 (socket says 172.21.15.32:41232) 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.109+0000 7efd877fe700 1 -- 172.21.15.32:0/3473743455 learned_addr learned my addr 172.21.15.32:0/3473743455 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd877fe700 1 -- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd88125dd0 msgr2=0x7efd8812e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd877fe700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd88125dd0 0x7efd8812e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd877fe700 1 -- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88059070 msgr2=0x7efd8812dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd877fe700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88059070 0x7efd8812dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd877fe700 1 -- 172.21.15.32:0/3473743455 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80009a20 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd87fff700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88059070 0x7efd8812dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd877fe700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 0x7efd8812e150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7efd7c006db0 tx=0x7efd7c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00f7e0 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c00f940 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c0092a0 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c01b660 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.110+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88093840 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.111+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7efd7c01ba10 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.111+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7efd7c03c8c0 con 0x7efd881251c0 2023-09-30T22:21:48.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.111+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8805a260 con 0x7efd881251c0 2023-09-30T22:21:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.116+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7efd7c029080 con 0x7efd881251c0 2023-09-30T22:21:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.296+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd88093a60 con 0x7efd881251c0 2023-09-30T22:21:48.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.297+0000 7efd6ffff700 1 -- 172.21.15.32:0/3473743455 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7efd7c01f020 con 0x7efd881251c0 2023-09-30T22:21:48.297 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:48.297 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:38.189468+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 >> v1:172.21.15.32:6816/118188 conn(0x7efd7402bb60 legacy=0x7efd7402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 msgr2=0x7efd8812e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 0x7efd8812e150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7efd7c006db0 tx=0x7efd7c017000).stop 2023-09-30T22:21:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 shutdown_connections 2023-09-30T22:21:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd88125dd0 0x7efd8812e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd88059070 0x7efd8812dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 --2- 172.21.15.32:0/3473743455 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd881251c0 0x7efd8812e150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 >> 172.21.15.32:0/3473743455 conn(0x7efd88097470 msgr2=0x7efd8809a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 shutdown_connections 2023-09-30T22:21:48.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.299+0000 7efd8e7e2700 1 -- 172.21.15.32:0/3473743455 wait complete. 2023-09-30T22:21:48.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:21:48.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:21:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.446+0000 7f00ed915700 1 Processor -- start 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.446+0000 7f00ed915700 1 -- start start 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00ed915700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 0x7f00e8097420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00ed915700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 0x7f00e812b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00ed915700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e8064e00 con 0x7f00e81274d0 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00ed915700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e80664f0 con 0x7f00e812add0 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00ed915700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8065d80 con 0x7f00e8066110 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 0x7f00e8097420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 0x7f00e812b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 0x7f00e8097420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35840/0 (socket says 172.21.15.32:35840) 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 0x7f00e812b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54858/0 (socket says 172.21.15.32:54858) 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f00e81274d0 0x7f00e8127980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54872/0 (socket says 172.21.15.32:54872) 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2223094359 learned_addr learned my addr 172.21.15.32:0/2223094359 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2223094359 >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 msgr2=0x7f00e812b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 --2- 172.21.15.32:0/2223094359 >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 0x7f00e812b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2223094359 >> v1:172.21.15.32:6789/0 conn(0x7f00e81274d0 legacy=0x7f00e8127980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2223094359 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8065910 con 0x7f00e8066110 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e67fc700 1 --2- 172.21.15.32:0/2223094359 >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 0x7f00e812b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.447+0000 7f00e6ffd700 1 --2- 172.21.15.32:0/2223094359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 0x7f00e8097420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f00d8006fd0 tx=0x7f00d80090f0).ready entity=mon.2 client_cookie=38ff5b8e25ab2075 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00e5ffb700 1 -- 172.21.15.32:0/2223094359 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800b580 con 0x7f00e8066110 2023-09-30T22:21:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00e5ffb700 1 -- 172.21.15.32:0/2223094359 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800b6e0 con 0x7f00e8066110 2023-09-30T22:21:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00e5ffb700 1 -- 172.21.15.32:0/2223094359 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800ba60 con 0x7f00e8066110 2023-09-30T22:21:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 -- 172.21.15.32:0/2223094359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 msgr2=0x7f00e8097420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 --2- 172.21.15.32:0/2223094359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 0x7f00e8097420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f00d8006fd0 tx=0x7f00d80090f0).stop 2023-09-30T22:21:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 -- 172.21.15.32:0/2223094359 shutdown_connections 2023-09-30T22:21:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 --2- 172.21.15.32:0/2223094359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8066110 0x7f00e8097420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 --2- 172.21.15.32:0/2223094359 >> v2:172.21.15.32:3300/0 conn(0x7f00e812add0 0x7f00e812b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 -- 172.21.15.32:0/2223094359 >> 172.21.15.32:0/2223094359 conn(0x7f00e809ba60 msgr2=0x7f00e809be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 -- 172.21.15.32:0/2223094359 shutdown_connections 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.448+0000 7f00ed915700 1 -- 172.21.15.32:0/2223094359 wait complete. 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.449+0000 7f00ed915700 1 Processor -- start 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.449+0000 7f00ed915700 1 -- start start 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00ed915700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 0x7f00e8114390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00ed915700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e81274d0 0x7f00e8114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00ed915700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e812add0 0x7f00e8114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00ed915700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8115860 con 0x7f00e8066110 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00ed915700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8116880 con 0x7f00e812add0 2023-09-30T22:21:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00ed915700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8118b40 con 0x7f00e81274d0 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 0x7f00e8114390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e81274d0 0x7f00e8114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 0x7f00e8114390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54860/0 (socket says 172.21.15.32:54860) 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2438123877 learned_addr learned my addr 172.21.15.32:0/2438123877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e77fe700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e812add0 0x7f00e8114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e81274d0 msgr2=0x7f00e8114880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e81274d0 0x7f00e8114880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e812add0 msgr2=0x7f00e8114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e812add0 0x7f00e8114d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 -- 172.21.15.32:0/2438123877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc0092a0 con 0x7f00e8066110 2023-09-30T22:21:48.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.450+0000 7f00e6ffd700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 0x7f00e8114390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f00d8009130 tx=0x7f00d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.451+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8000bc0 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.451+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800b850 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.451+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d8009370 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.451+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8000d20 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.451+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8117920 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.451+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f00d801b870 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.452+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f00d803c8a0 con 0x7f00e8066110 2023-09-30T22:21:48.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.452+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e8098d10 con 0x7f00e8066110 2023-09-30T22:21:48.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.457+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f00d8029080 con 0x7f00e8066110 2023-09-30T22:21:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.630+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f00e805a260 con 0x7f00e8066110 2023-09-30T22:21:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.631+0000 7f00ceffd700 1 -- 172.21.15.32:0/2438123877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f00d801f020 con 0x7f00e8066110 2023-09-30T22:21:48.631 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 >> v1:172.21.15.32:6816/118188 conn(0x7f00d402bb60 legacy=0x7f00d402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 msgr2=0x7f00e8114390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 0x7f00e8114390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f00d8009130 tx=0x7f00d8017000).stop 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 shutdown_connections 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e812add0 0x7f00e8114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e8066110 0x7f00e8114390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 --2- 172.21.15.32:0/2438123877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e81274d0 0x7f00e8114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 >> 172.21.15.32:0/2438123877 conn(0x7f00e809ba60 msgr2=0x7f00e81287f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 shutdown_connections 2023-09-30T22:21:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.633+0000 7f00ed915700 1 -- 172.21.15.32:0/2438123877 wait complete. 2023-09-30T22:21:48.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:21:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 Processor -- start 2023-09-30T22:21:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 -- start start 2023-09-30T22:21:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34125f00 0x7f0d34125250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d3412db40 0x7f0d3412ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d34098790 con 0x7f0d34059070 2023-09-30T22:21:48.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d34059450 con 0x7f0d3412db40 2023-09-30T22:21:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d391d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34059da0 con 0x7f0d34125f00 2023-09-30T22:21:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34125f00 0x7f0d34125250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d3359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d3412db40 0x7f0d3412ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d32d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d34059070 0x7f0d341257f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54882/0 (socket says 172.21.15.32:54882) 2023-09-30T22:21:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.781+0000 7f0d32d9d700 1 -- 172.21.15.32:0/4000637279 learned_addr learned my addr 172.21.15.32:0/4000637279 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d3359e700 1 -- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34125f00 msgr2=0x7f0d34125250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d3359e700 1 --2- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34125f00 0x7f0d34125250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d3359e700 1 -- 172.21.15.32:0/4000637279 >> v1:172.21.15.32:6789/0 conn(0x7f0d34059070 legacy=0x7f0d341257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d3359e700 1 -- 172.21.15.32:0/4000637279 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340996f0 con 0x7f0d3412db40 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d31d9b700 1 -- 172.21.15.32:0/4000637279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106455443 0 0) 0x7f0d34098790 con 0x7f0d34059070 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d3259c700 1 --2- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34125f00 0x7f0d34125250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d3359e700 1 --2- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d3412db40 0x7f0d3412ff30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0d2c009e60 tx=0x7f0d2c009780).ready entity=mon.0 client_cookie=dad4d522aa99f613 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d31d9b700 1 -- 172.21.15.32:0/4000637279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00e070 con 0x7f0d3412db40 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d31d9b700 1 -- 172.21.15.32:0/4000637279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c00bb60 con 0x7f0d3412db40 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d31d9b700 1 -- 172.21.15.32:0/4000637279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c012410 con 0x7f0d3412db40 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d391d7700 1 -- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d3412db40 msgr2=0x7f0d3412ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.782+0000 7f0d391d7700 1 --2- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d3412db40 0x7f0d3412ff30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f0d2c009e60 tx=0x7f0d2c009780).stop 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.783+0000 7f0d391d7700 1 -- 172.21.15.32:0/4000637279 shutdown_connections 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.783+0000 7f0d391d7700 1 --2- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d3412db40 0x7f0d3412ff30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.783+0000 7f0d391d7700 1 --2- 172.21.15.32:0/4000637279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34125f00 0x7f0d34125250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.783+0000 7f0d391d7700 1 -- 172.21.15.32:0/4000637279 >> 172.21.15.32:0/4000637279 conn(0x7f0d34097480 msgr2=0x7f0d34097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.783+0000 7f0d391d7700 1 -- 172.21.15.32:0/4000637279 shutdown_connections 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.783+0000 7f0d391d7700 1 -- 172.21.15.32:0/4000637279 wait complete. 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 Processor -- start 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 -- start start 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 0x7f0d34069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 0x7f0d3412bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d3412db40 0x7f0d3412c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3412d9f0 con 0x7f0d34125f00 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34128220 con 0x7f0d34059070 2023-09-30T22:21:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d391d7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34069e40 con 0x7f0d3412db40 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 0x7f0d3412bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 0x7f0d34069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 0x7f0d3412bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54882/0 (socket says 172.21.15.32:54882) 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d3259c700 1 -- 172.21.15.32:0/203089433 learned_addr learned my addr 172.21.15.32:0/203089433 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d32d9d700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 0x7f0d34069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41244/0 (socket says 172.21.15.32:41244) 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.784+0000 7f0d3359e700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d3412db40 0x7f0d3412c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d32d9d700 1 -- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d3412db40 msgr2=0x7f0d3412c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d32d9d700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d3412db40 0x7f0d3412c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d32d9d700 1 -- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 msgr2=0x7f0d3412bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d32d9d700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 0x7f0d3412bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d32d9d700 1 -- 172.21.15.32:0/203089433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0092a0 con 0x7f0d34059070 2023-09-30T22:21:48.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d3359e700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d3412db40 0x7f0d3412c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d3259c700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 0x7f0d3412bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d32d9d700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 0x7f0d34069100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0d24000c00 tx=0x7f0d24007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24013070 con 0x7f0d34059070 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d240070d0 con 0x7f0d34059070 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c009a20 con 0x7f0d34059070 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240072a0 con 0x7f0d34059070 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.785+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3412b6d0 con 0x7f0d34059070 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.786+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0d24005720 con 0x7f0d34059070 2023-09-30T22:21:48.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.786+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0d2403e3c0 con 0x7f0d34059070 2023-09-30T22:21:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.786+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3405a260 con 0x7f0d34059070 2023-09-30T22:21:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.791+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0d2401e030 con 0x7f0d34059070 2023-09-30T22:21:48.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.964+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d3409a3e0 con 0x7f0d34059070 2023-09-30T22:21:48.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.965+0000 7f0d2affd700 1 -- 172.21.15.32:0/203089433 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d2400c7d0 con 0x7f0d34059070 2023-09-30T22:21:48.965 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 >> v1:172.21.15.32:6816/118188 conn(0x7f0d1402bb90 legacy=0x7f0d1402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 msgr2=0x7f0d34069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 0x7f0d34069100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0d24000c00 tx=0x7f0d24007c20).stop 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 shutdown_connections 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d3412db40 0x7f0d3412c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d34059070 0x7f0d34069100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 --2- 172.21.15.32:0/203089433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d34125f00 0x7f0d3412bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:48.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 >> 172.21.15.32:0/203089433 conn(0x7f0d34097480 msgr2=0x7f0d3412f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 shutdown_connections 2023-09-30T22:21:48.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:48.967+0000 7f0d391d7700 1 -- 172.21.15.32:0/203089433 wait complete. 2023-09-30T22:21:48.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:21:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.113+0000 7f7ba653b700 1 Processor -- start 2023-09-30T22:21:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.113+0000 7f7ba653b700 1 -- start start 2023-09-30T22:21:49.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7ba653b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 0x7f7ba0098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7ba653b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ba012d160 0x7f7ba012f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7ba653b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba0092d80 con 0x7f7ba0098c80 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7ba653b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba0093c20 con 0x7f7ba012d160 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7ba653b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0092b90 con 0x7f7ba0093840 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 0x7f7ba0098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7ba4ad8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ba012d160 0x7f7ba012f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7b9f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7ba0098c80 0x7f7ba0099060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54894/0 (socket says 172.21.15.32:54894) 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 0x7f7ba0098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35864/0 (socket says 172.21.15.32:35864) 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.114+0000 7f7b9f7fe700 1 -- 172.21.15.32:0/4270691261 learned_addr learned my addr 172.21.15.32:0/4270691261 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7b9effd700 1 -- 172.21.15.32:0/4270691261 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246241289 0 0) 0x7f7ba0092d80 con 0x7f7ba0098c80 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7b9effd700 1 -- 172.21.15.32:0/4270691261 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b8c0033f0 con 0x7f7ba0098c80 2023-09-30T22:21:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba4ad8700 1 -- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 msgr2=0x7f7ba0098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba4ad8700 1 --2- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 0x7f7ba0098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba4ad8700 1 -- 172.21.15.32:0/4270691261 >> v1:172.21.15.32:6789/0 conn(0x7f7ba0098c80 legacy=0x7f7ba0099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba4ad8700 1 -- 172.21.15.32:0/4270691261 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0099b70 con 0x7f7ba012d160 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7b9ffff700 1 --2- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 0x7f7ba0098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba4ad8700 1 --2- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba012d160 0x7f7ba012f550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7b98009e90 tx=0x7f7b98009780).ready entity=mon.0 client_cookie=124c19043accdd39 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7b9effd700 1 -- 172.21.15.32:0/4270691261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800e070 con 0x7f7ba012d160 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7b9effd700 1 -- 172.21.15.32:0/4270691261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800ba70 con 0x7f7ba012d160 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7b9effd700 1 -- 172.21.15.32:0/4270691261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800bdf0 con 0x7f7ba012d160 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba653b700 1 -- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba012d160 msgr2=0x7f7ba012f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.115+0000 7f7ba653b700 1 --2- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba012d160 0x7f7ba012f550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7b98009e90 tx=0x7f7b98009780).stop 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 -- 172.21.15.32:0/4270691261 shutdown_connections 2023-09-30T22:21:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 --2- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba012d160 0x7f7ba012f550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 --2- 172.21.15.32:0/4270691261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba0093840 0x7f7ba0098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 -- 172.21.15.32:0/4270691261 >> 172.21.15.32:0/4270691261 conn(0x7f7ba0097480 msgr2=0x7f7ba0097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 -- 172.21.15.32:0/4270691261 shutdown_connections 2023-09-30T22:21:49.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 -- 172.21.15.32:0/4270691261 wait complete. 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.116+0000 7f7ba653b700 1 Processor -- start 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 -- start start 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 0x7f7ba00590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba0098c80 0x7f7ba0118e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba012d160 0x7f7ba0119330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba006a520 con 0x7f7ba0093840 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba011af10 con 0x7f7ba0098c80 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba653b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0064ce0 con 0x7f7ba012d160 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 0x7f7ba00590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7b9f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba0098c80 0x7f7ba0118e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7b9ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 0x7f7ba00590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54910/0 (socket says 172.21.15.32:54910) 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7b9ffff700 1 -- 172.21.15.32:0/1224187005 learned_addr learned my addr 172.21.15.32:0/1224187005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.117+0000 7f7ba4ad8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba012d160 0x7f7ba0119330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b9ffff700 1 -- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba012d160 msgr2=0x7f7ba0119330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b9ffff700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba012d160 0x7f7ba0119330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b9ffff700 1 -- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba0098c80 msgr2=0x7f7ba0118e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b9ffff700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba0098c80 0x7f7ba0118e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b9ffff700 1 -- 172.21.15.32:0/1224187005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900092a0 con 0x7f7ba0093840 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b9ffff700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 0x7f7ba00590a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7b9000ea20 tx=0x7f7b90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000ee60 con 0x7f7ba0093840 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000c850 con 0x7f7ba0093840 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000cbd0 con 0x7f7ba0093840 2023-09-30T22:21:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b98009a20 con 0x7f7ba0093840 2023-09-30T22:21:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.118+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba0059c20 con 0x7f7ba0093840 2023-09-30T22:21:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.119+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7b9001b400 con 0x7f7ba0093840 2023-09-30T22:21:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.119+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7b9003b530 con 0x7f7ba0093840 2023-09-30T22:21:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.119+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba0097860 con 0x7f7ba0093840 2023-09-30T22:21:49.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.124+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7b90019900 con 0x7f7ba0093840 2023-09-30T22:21:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.299+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ba005a260 con 0x7f7ba0093840 2023-09-30T22:21:49.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.299+0000 7f7b97fff700 1 -- 172.21.15.32:0/1224187005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b90019310 con 0x7f7ba0093840 2023-09-30T22:21:49.299 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:21:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 >> v1:172.21.15.32:6816/118188 conn(0x7f7b8002bb90 legacy=0x7f7b8002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 msgr2=0x7f7ba00590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 0x7f7ba00590a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f7b9000ea20 tx=0x7f7b90009060).stop 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 shutdown_connections 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba012d160 0x7f7ba0119330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba0093840 0x7f7ba00590a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.301+0000 7f7ba653b700 1 --2- 172.21.15.32:0/1224187005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba0098c80 0x7f7ba0118e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.302+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 >> 172.21.15.32:0/1224187005 conn(0x7f7ba0097480 msgr2=0x7f7ba0096280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.302+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 shutdown_connections 2023-09-30T22:21:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.302+0000 7f7ba653b700 1 -- 172.21.15.32:0/1224187005 wait complete. 2023-09-30T22:21:49.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:21:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.448+0000 7efdfb3a1700 1 Processor -- start 2023-09-30T22:21:49.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.448+0000 7efdfb3a1700 1 -- start start 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdfb3a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdfb3a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf4130220 0x7efdf4132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdfb3a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf4091c80 con 0x7efdf40926d0 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdfb3a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41279e0 con 0x7efdf4130220 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdfb3a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40999d0 con 0x7efdf41275c0 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdf993e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf4130220 0x7efdf4132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdf913d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efdf40926d0 0x7efdf4127be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54910/0 (socket says 172.21.15.32:54910) 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdf913d700 1 -- 172.21.15.32:0/492952231 learned_addr learned my addr 172.21.15.32:0/492952231 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:49.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.449+0000 7efdf893c700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efde7fff700 1 -- 172.21.15.32:0/492952231 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367521995 0 0) 0x7efdf4091c80 con 0x7efdf40926d0 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdf893c700 1 -- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf4130220 msgr2=0x7efdf4132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdf893c700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf4130220 0x7efdf4132610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdf893c700 1 -- 172.21.15.32:0/492952231 >> v1:172.21.15.32:6789/0 conn(0x7efdf40926d0 legacy=0x7efdf4127be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdf893c700 1 -- 172.21.15.32:0/492952231 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf409a280 con 0x7efdf41275c0 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdf993e700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf4130220 0x7efdf4132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdf893c700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412fd30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7efdf00097c0 tx=0x7efdf0009060).ready entity=mon.2 client_cookie=421f6a0a30590f50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efde7fff700 1 -- 172.21.15.32:0/492952231 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000f510 con 0x7efdf41275c0 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efde7fff700 1 -- 172.21.15.32:0/492952231 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000f670 con 0x7efdf41275c0 2023-09-30T22:21:49.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efde7fff700 1 -- 172.21.15.32:0/492952231 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000f9f0 con 0x7efdf41275c0 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdfb3a1700 1 -- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 msgr2=0x7efdf412fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.450+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412fd30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7efdf00097c0 tx=0x7efdf0009060).stop 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.451+0000 7efdfb3a1700 1 -- 172.21.15.32:0/492952231 shutdown_connections 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.451+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf4130220 0x7efdf4132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.451+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/492952231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412fd30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.451+0000 7efdfb3a1700 1 -- 172.21.15.32:0/492952231 >> 172.21.15.32:0/492952231 conn(0x7efdf4097480 msgr2=0x7efdf4097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.451+0000 7efdfb3a1700 1 -- 172.21.15.32:0/492952231 shutdown_connections 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.451+0000 7efdfb3a1700 1 -- 172.21.15.32:0/492952231 wait complete. 2023-09-30T22:21:49.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 Processor -- start 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 -- start start 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 0x7efdf412e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 0x7efdf412ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf412ef60 con 0x7efdf4130220 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf412f500 con 0x7efdf40926d0 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdfb3a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdf409a4b0 con 0x7efdf41275c0 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdf913d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 0x7efdf412e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdf993e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 0x7efdf412ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdf913d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 0x7efdf412e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41258/0 (socket says 172.21.15.32:41258) 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdf913d700 1 -- 172.21.15.32:0/1776600683 learned_addr learned my addr 172.21.15.32:0/1776600683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdf993e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 0x7efdf412ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54926/0 (socket says 172.21.15.32:54926) 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.452+0000 7efdf893c700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf893c700 1 -- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 msgr2=0x7efdf412e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf893c700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 0x7efdf412e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf893c700 1 -- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 msgr2=0x7efdf412ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf893c700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 0x7efdf412ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf893c700 1 -- 172.21.15.32:0/1776600683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009a20 con 0x7efdf41275c0 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf913d700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 0x7efdf412e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf993e700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 0x7efdf412ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdf893c700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412e580 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7efdf0014000 tx=0x7efdf0017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000f7e0 con 0x7efdf41275c0 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000f940 con 0x7efdf41275c0 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf00092a0 con 0x7efdf41275c0 2023-09-30T22:21:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf001b660 con 0x7efdf41275c0 2023-09-30T22:21:49.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.453+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf412fbe0 con 0x7efdf41275c0 2023-09-30T22:21:49.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.454+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7efdf001ba50 con 0x7efdf41275c0 2023-09-30T22:21:49.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.454+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7efdf003c8f0 con 0x7efdf41275c0 2023-09-30T22:21:49.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.454+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4110fc0 con 0x7efdf41275c0 2023-09-30T22:21:49.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.459+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7efdf0029080 con 0x7efdf41275c0 2023-09-30T22:21:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.636+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efdf405a260 con 0x7efdf41275c0 2023-09-30T22:21:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.636+0000 7efde57fa700 1 -- 172.21.15.32:0/1776600683 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7efdf001f030 con 0x7efdf41275c0 2023-09-30T22:21:49.636 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:21:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 >> v1:172.21.15.32:6816/118188 conn(0x7efdd802bb60 legacy=0x7efdd802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 msgr2=0x7efdf412e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412e580 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7efdf0014000 tx=0x7efdf0017000).stop 2023-09-30T22:21:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 shutdown_connections 2023-09-30T22:21:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdf4130220 0x7efdf412ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdf40926d0 0x7efdf412e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 --2- 172.21.15.32:0/1776600683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdf41275c0 0x7efdf412e580 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 >> 172.21.15.32:0/1776600683 conn(0x7efdf4097480 msgr2=0x7efdf40963c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.638+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 shutdown_connections 2023-09-30T22:21:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.639+0000 7efdfb3a1700 1 -- 172.21.15.32:0/1776600683 wait complete. 2023-09-30T22:21:49.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:49.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 Processor -- start 2023-09-30T22:21:49.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 -- start start 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac091b00 0x7f73ac12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac12ce60 0x7f73ac12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac096760 con 0x7f73ac096380 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac091ff0 con 0x7f73ac091b00 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.783+0000 7f73b3d88700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac092c80 con 0x7f73ac12ce60 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b1b24700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f73ac096380 0x7f73ac094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54918/0 (socket says 172.21.15.32:54918) 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b1323700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac091b00 0x7f73ac12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b1b24700 1 -- 172.21.15.32:0/1655324167 learned_addr learned my addr 172.21.15.32:0/1655324167 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:49.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b2325700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac12ce60 0x7f73ac12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b2325700 1 -- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac091b00 msgr2=0x7f73ac12c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b2325700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac091b00 0x7f73ac12c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b0b22700 1 -- 172.21.15.32:0/1655324167 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231211208 0 0) 0x7f73ac096760 con 0x7f73ac096380 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b2325700 1 -- 172.21.15.32:0/1655324167 >> v1:172.21.15.32:6789/0 conn(0x7f73ac096380 legacy=0x7f73ac094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b2325700 1 -- 172.21.15.32:0/1655324167 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac094e40 con 0x7f73ac12ce60 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b1323700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac091b00 0x7f73ac12c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b2325700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac12ce60 0x7f73ac12f250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f73a4009e90 tx=0x7f73a4009780).ready entity=mon.2 client_cookie=841c1d17130580b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b0b22700 1 -- 172.21.15.32:0/1655324167 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400e070 con 0x7f73ac12ce60 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b0b22700 1 -- 172.21.15.32:0/1655324167 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400bb60 con 0x7f73ac12ce60 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.784+0000 7f73b0b22700 1 -- 172.21.15.32:0/1655324167 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4012410 con 0x7f73ac12ce60 2023-09-30T22:21:49.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 -- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac12ce60 msgr2=0x7f73ac12f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac12ce60 0x7f73ac12f250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f73a4009e90 tx=0x7f73a4009780).stop 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 -- 172.21.15.32:0/1655324167 shutdown_connections 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac12ce60 0x7f73ac12f250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 --2- 172.21.15.32:0/1655324167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac091b00 0x7f73ac12c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 -- 172.21.15.32:0/1655324167 >> 172.21.15.32:0/1655324167 conn(0x7f73ac097470 msgr2=0x7f73ac097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 -- 172.21.15.32:0/1655324167 shutdown_connections 2023-09-30T22:21:49.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.785+0000 7f73b3d88700 1 -- 172.21.15.32:0/1655324167 wait complete. 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.786+0000 7f73b3d88700 1 Processor -- start 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.786+0000 7f73b3d88700 1 -- start start 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac091b00 0x7f73ac13a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 0x7f73ac13a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac12ce60 0x7f73ac067990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:49.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac068db0 con 0x7f73ac096380 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac12ffb0 con 0x7f73ac12ce60 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac06a520 con 0x7f73ac091b00 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 0x7f73ac13a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b2325700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac12ce60 0x7f73ac067990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 0x7f73ac13a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54938/0 (socket says 172.21.15.32:54938) 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 -- 172.21.15.32:0/2596694616 learned_addr learned my addr 172.21.15.32:0/2596694616 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1b24700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac091b00 0x7f73ac13a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 -- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac091b00 msgr2=0x7f73ac13a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac091b00 0x7f73ac13a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 -- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac12ce60 msgr2=0x7f73ac067990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac12ce60 0x7f73ac067990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 -- 172.21.15.32:0/2596694616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c0092a0 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b2325700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac12ce60 0x7f73ac067990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b1323700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 0x7f73ac13a6d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f739c00ea50 tx=0x7f739c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00ee90 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c009600 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.787+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a4009a20 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.788+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00ca60 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.788+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac06a070 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.788+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f739c01b400 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.789+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f739c03b430 con 0x7f73ac096380 2023-09-30T22:21:49.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.789+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac05a260 con 0x7f73ac096380 2023-09-30T22:21:49.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:49.794+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f739c011d20 con 0x7f73ac096380 2023-09-30T22:21:50.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.011+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73ac046c20 con 0x7f73ac096380 2023-09-30T22:21:50.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.012+0000 7f73a1ffb700 1 -- 172.21.15.32:0/2596694616 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f739c023350 con 0x7f73ac096380 2023-09-30T22:21:50.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:50.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:40.197617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:21:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 >> v1:172.21.15.32:6816/118188 conn(0x7f739002bb60 legacy=0x7f739002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 msgr2=0x7f73ac13a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 0x7f73ac13a6d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f739c00ea50 tx=0x7f739c009060).stop 2023-09-30T22:21:50.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 shutdown_connections 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f73ac12ce60 0x7f73ac067990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f73ac091b00 0x7f73ac13a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 --2- 172.21.15.32:0/2596694616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73ac096380 0x7f73ac13a6d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 >> 172.21.15.32:0/2596694616 conn(0x7f73ac097470 msgr2=0x7f73ac0952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 shutdown_connections 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.014+0000 7f73b3d88700 1 -- 172.21.15.32:0/2596694616 wait complete. 2023-09-30T22:21:50.015 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2023-09-30T22:21:50.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 Processor -- start 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 -- start start 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f923c0650f0 0x7f923c0986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c097c30 0x7f923c098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c096160 con 0x7f923c099080 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0654d0 con 0x7f923c0650f0 2023-09-30T22:21:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f9241311700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c099530 con 0x7f923c097c30 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f923a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c097c30 0x7f923c098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f923affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f923c0650f0 0x7f923c0986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.158+0000 7f923b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f923c099080 0x7f923c131470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54928/0 (socket says 172.21.15.32:54928) 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923b7fe700 1 -- 172.21.15.32:0/190388397 learned_addr learned my addr 172.21.15.32:0/190388397 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923a7fc700 1 -- 172.21.15.32:0/190388397 >> v2:172.21.15.32:3300/0 conn(0x7f923c0650f0 msgr2=0x7f923c0986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923a7fc700 1 --2- 172.21.15.32:0/190388397 >> v2:172.21.15.32:3300/0 conn(0x7f923c0650f0 0x7f923c0986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923a7fc700 1 -- 172.21.15.32:0/190388397 >> v1:172.21.15.32:6789/0 conn(0x7f923c099080 legacy=0x7f923c131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923a7fc700 1 -- 172.21.15.32:0/190388397 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c099790 con 0x7f923c097c30 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923affd700 1 --2- 172.21.15.32:0/190388397 >> v2:172.21.15.32:3300/0 conn(0x7f923c0650f0 0x7f923c0986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f923a7fc700 1 --2- 172.21.15.32:0/190388397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c097c30 0x7f923c098b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f92300097c0 tx=0x7f9230009060).ready entity=mon.1 client_cookie=41711a8e4b49a0c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f9239ffb700 1 -- 172.21.15.32:0/190388397 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000f510 con 0x7f923c097c30 2023-09-30T22:21:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f9239ffb700 1 -- 172.21.15.32:0/190388397 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000f670 con 0x7f923c097c30 2023-09-30T22:21:50.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.159+0000 7f9239ffb700 1 -- 172.21.15.32:0/190388397 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000f9f0 con 0x7f923c097c30 2023-09-30T22:21:50.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 -- 172.21.15.32:0/190388397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c097c30 msgr2=0x7f923c098b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 --2- 172.21.15.32:0/190388397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c097c30 0x7f923c098b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f92300097c0 tx=0x7f9230009060).stop 2023-09-30T22:21:50.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 -- 172.21.15.32:0/190388397 shutdown_connections 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 --2- 172.21.15.32:0/190388397 >> v2:172.21.15.32:3300/0 conn(0x7f923c0650f0 0x7f923c0986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 --2- 172.21.15.32:0/190388397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c097c30 0x7f923c098b90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 -- 172.21.15.32:0/190388397 >> 172.21.15.32:0/190388397 conn(0x7f923c09dc70 msgr2=0x7f923c09e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 -- 172.21.15.32:0/190388397 shutdown_connections 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.160+0000 7f9241311700 1 -- 172.21.15.32:0/190388397 wait complete. 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 Processor -- start 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 -- start start 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923c0650f0 0x7f923c13a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 0x7f923c13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923c13af30 0x7f923c11acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c099a80 con 0x7f923c0650f0 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c09ce60 con 0x7f923c099080 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f9241311700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f923c13b350 con 0x7f923c13af30 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.161+0000 7f923a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 0x7f923c13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923c13af30 0x7f923c11acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 0x7f923c13aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41308/0 (socket says 172.21.15.32:41308) 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 -- 172.21.15.32:0/1817372477 learned_addr learned my addr 172.21.15.32:0/1817372477 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923affd700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923c0650f0 0x7f923c13a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 -- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923c13af30 msgr2=0x7f923c11acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923c13af30 0x7f923c11acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 -- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923c0650f0 msgr2=0x7f923c13a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923c0650f0 0x7f923c13a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 -- 172.21.15.32:0/1817372477 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0092a0 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923affd700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923c0650f0 0x7f923c13a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f923a7fc700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 0x7f923c13aa40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9230011000 tx=0x7f9230015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000f7e0 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000f960 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92300092a0 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923001b660 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.162+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c11ce00 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.163+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f923001ba10 con 0x7f923c099080 2023-09-30T22:21:50.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.163+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f923003c7d0 con 0x7f923c099080 2023-09-30T22:21:50.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.163+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c05a260 con 0x7f923c099080 2023-09-30T22:21:50.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.168+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9230029080 con 0x7f923c099080 2023-09-30T22:21:50.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.347+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f923c0973a0 con 0x7f923c099080 2023-09-30T22:21:50.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.347+0000 7f9222ffd700 1 -- 172.21.15.32:0/1817372477 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f923001f020 con 0x7f923c099080 2023-09-30T22:21:50.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:50.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:38.189468+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:50.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 >> v1:172.21.15.32:6816/118188 conn(0x7f922802bb60 legacy=0x7f922802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 msgr2=0x7f923c13aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 0x7f923c13aa40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9230011000 tx=0x7f9230015000).stop 2023-09-30T22:21:50.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 shutdown_connections 2023-09-30T22:21:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923c13af30 0x7f923c11acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923c0650f0 0x7f923c13a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 --2- 172.21.15.32:0/1817372477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923c099080 0x7f923c13aa40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 >> 172.21.15.32:0/1817372477 conn(0x7f923c09dc70 msgr2=0x7f923c1142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 shutdown_connections 2023-09-30T22:21:50.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.349+0000 7f9241311700 1 -- 172.21.15.32:0/1817372477 wait complete. 2023-09-30T22:21:50.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.493+0000 7f931441b700 1 Processor -- start 2023-09-30T22:21:50.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.493+0000 7f931441b700 1 -- start start 2023-09-30T22:21:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f931441b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 0x7f930c094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f931441b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c091b00 0x7f930c127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f931441b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f930c094c20 0x7f930c127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f931441b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930c096760 con 0x7f930c094c20 2023-09-30T22:21:50.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f931441b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930c091ff0 con 0x7f930c091b00 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f931441b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f930c092c80 con 0x7f930c096380 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 0x7f930c094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93119b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c091b00 0x7f930c127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93129b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f930c094c20 0x7f930c127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 0x7f930c094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33600/0 (socket says 172.21.15.32:33600) 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 -- 172.21.15.32:0/4085502493 learned_addr learned my addr 172.21.15.32:0/4085502493 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 -- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c091b00 msgr2=0x7f930c127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 --2- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c091b00 0x7f930c127600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 -- 172.21.15.32:0/4085502493 >> v2:172.21.15.32:3300/0 conn(0x7f930c094c20 msgr2=0x7f930c127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 --2- 172.21.15.32:0/4085502493 >> v2:172.21.15.32:3300/0 conn(0x7f930c094c20 0x7f930c127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 -- 172.21.15.32:0/4085502493 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c093900 con 0x7f930c096380 2023-09-30T22:21:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93119b6700 1 --2- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c091b00 0x7f930c127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93129b8700 1 --2- 172.21.15.32:0/4085502493 >> v2:172.21.15.32:3300/0 conn(0x7f930c094c20 0x7f930c127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93121b7700 1 --2- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 0x7f930c094730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9308009830 tx=0x7f93080090f0).ready entity=mon.2 client_cookie=fb54cf6a0cab0694 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.494+0000 7f93111b5700 1 -- 172.21.15.32:0/4085502493 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930800f510 con 0x7f930c096380 2023-09-30T22:21:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f93111b5700 1 -- 172.21.15.32:0/4085502493 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930800f670 con 0x7f930c096380 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f93111b5700 1 -- 172.21.15.32:0/4085502493 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930800f9f0 con 0x7f930c096380 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 -- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 msgr2=0x7f930c094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 --2- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 0x7f930c094730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9308009830 tx=0x7f93080090f0).stop 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 -- 172.21.15.32:0/4085502493 shutdown_connections 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 --2- 172.21.15.32:0/4085502493 >> v2:172.21.15.32:3300/0 conn(0x7f930c094c20 0x7f930c127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 --2- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c096380 0x7f930c094730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 --2- 172.21.15.32:0/4085502493 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c091b00 0x7f930c127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 -- 172.21.15.32:0/4085502493 >> 172.21.15.32:0/4085502493 conn(0x7f930c097470 msgr2=0x7f930c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 -- 172.21.15.32:0/4085502493 shutdown_connections 2023-09-30T22:21:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.495+0000 7f931441b700 1 -- 172.21.15.32:0/4085502493 wait complete. 2023-09-30T22:21:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.496+0000 7f931441b700 1 Processor -- start 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.496+0000 7f931441b700 1 -- start start 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930c091b00 0x7f930c123f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 0x7f930c124420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c096380 0x7f930c113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1150c0 con 0x7f930c091b00 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930c06a520 con 0x7f930c096380 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f930c11a860 con 0x7f930c094c20 2023-09-30T22:21:50.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93119b6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 0x7f930c124420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c096380 0x7f930c113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93121b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930c091b00 0x7f930c123f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93119b6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 0x7f930c124420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33612/0 (socket says 172.21.15.32:33612) 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93119b6700 1 -- 172.21.15.32:0/2138514176 learned_addr learned my addr 172.21.15.32:0/2138514176 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 -- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 msgr2=0x7f930c124420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 0x7f930c124420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 -- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930c091b00 msgr2=0x7f930c123f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930c091b00 0x7f930c123f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 -- 172.21.15.32:0/2138514176 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0092a0 con 0x7f930c096380 2023-09-30T22:21:50.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93119b6700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 0x7f930c124420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93129b8700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c096380 0x7f930c113fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9304009f40 tx=0x7f930400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304009c70 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930400f040 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9304009a20 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.497+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400cd80 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.498+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9308009370 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.498+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9304013610 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.498+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f930403b9a0 con 0x7f930c096380 2023-09-30T22:21:50.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.499+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c11ac00 con 0x7f930c096380 2023-09-30T22:21:50.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.503+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f930401d460 con 0x7f930c096380 2023-09-30T22:21:50.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.725+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f930c05a260 con 0x7f930c096380 2023-09-30T22:21:50.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.725+0000 7f93027fc700 1 -- 172.21.15.32:0/2138514176 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f930401c3c0 con 0x7f930c096380 2023-09-30T22:21:50.726 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:50.726 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:40.197617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:21:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 >> v1:172.21.15.32:6816/118188 conn(0x7f92f002bb60 legacy=0x7f92f002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c096380 msgr2=0x7f930c113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c096380 0x7f930c113fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9304009f40 tx=0x7f930400b040).stop 2023-09-30T22:21:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 shutdown_connections 2023-09-30T22:21:50.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f930c096380 0x7f930c113fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930c091b00 0x7f930c123f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.727+0000 7f931441b700 1 --2- 172.21.15.32:0/2138514176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f930c094c20 0x7f930c124420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.728+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 >> 172.21.15.32:0/2138514176 conn(0x7f930c097470 msgr2=0x7f930c1269b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.728+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 shutdown_connections 2023-09-30T22:21:50.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.728+0000 7f931441b700 1 -- 172.21.15.32:0/2138514176 wait complete. 2023-09-30T22:21:50.730 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2023-09-30T22:21:50.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:21:37.188029+0000', 'modified': '2023-09-30T22:21:40.197617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4177': {'gid': 4177, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6817/3095020018', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 3095020018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-30T22:21:50.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:21:50.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:21:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.979+0000 7fd55793f700 1 Processor -- start 2023-09-30T22:21:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.979+0000 7fd55793f700 1 -- start start 2023-09-30T22:21:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd55793f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd550091b00 0x7fd55012c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd55793f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd55012ce60 0x7fd55012f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd55793f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd550096760 con 0x7fd550096380 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd55793f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd550091ff0 con 0x7fd55012ce60 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd55793f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd550092c80 con 0x7fd550091b00 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd5556db700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd550096380 0x7fd550094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57662/0 (socket says 172.21.15.32:57662) 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd554eda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd550091b00 0x7fd55012c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd5556db700 1 -- 172.21.15.32:0/32941220 learned_addr learned my addr 172.21.15.32:0/32941220 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd555edc700 1 --2- 172.21.15.32:0/32941220 >> v2:172.21.15.32:3300/0 conn(0x7fd55012ce60 0x7fd55012f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd543fff700 1 -- 172.21.15.32:0/32941220 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104031724 0 0) 0x7fd550096760 con 0x7fd550096380 2023-09-30T22:21:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd543fff700 1 -- 172.21.15.32:0/32941220 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd53c0033f0 con 0x7fd550096380 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd554eda700 1 -- 172.21.15.32:0/32941220 >> v2:172.21.15.32:3300/0 conn(0x7fd55012ce60 msgr2=0x7fd55012f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd554eda700 1 --2- 172.21.15.32:0/32941220 >> v2:172.21.15.32:3300/0 conn(0x7fd55012ce60 0x7fd55012f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.980+0000 7fd554eda700 1 -- 172.21.15.32:0/32941220 >> v1:172.21.15.32:6789/0 conn(0x7fd550096380 legacy=0x7fd550094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd554eda700 1 -- 172.21.15.32:0/32941220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550094e40 con 0x7fd550091b00 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd555edc700 1 --2- 172.21.15.32:0/32941220 >> v2:172.21.15.32:3300/0 conn(0x7fd55012ce60 0x7fd55012f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd554eda700 1 --2- 172.21.15.32:0/32941220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd550091b00 0x7fd55012c970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd544009760 tx=0x7fd544009060).ready entity=mon.1 client_cookie=3bd3575670c9432b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd543fff700 1 -- 172.21.15.32:0/32941220 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400d4a0 con 0x7fd550091b00 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd543fff700 1 -- 172.21.15.32:0/32941220 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54400d600 con 0x7fd550091b00 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd543fff700 1 -- 172.21.15.32:0/32941220 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400d980 con 0x7fd550091b00 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd55793f700 1 -- 172.21.15.32:0/32941220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd550091b00 msgr2=0x7fd55012c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd55793f700 1 --2- 172.21.15.32:0/32941220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd550091b00 0x7fd55012c970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd544009760 tx=0x7fd544009060).stop 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd55793f700 1 -- 172.21.15.32:0/32941220 shutdown_connections 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd55793f700 1 --2- 172.21.15.32:0/32941220 >> v2:172.21.15.32:3300/0 conn(0x7fd55012ce60 0x7fd55012f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd55793f700 1 --2- 172.21.15.32:0/32941220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd550091b00 0x7fd55012c970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.981+0000 7fd55793f700 1 -- 172.21.15.32:0/32941220 >> 172.21.15.32:0/32941220 conn(0x7fd550097470 msgr2=0x7fd550097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.982+0000 7fd55793f700 1 -- 172.21.15.32:0/32941220 shutdown_connections 2023-09-30T22:21:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.982+0000 7fd55793f700 1 -- 172.21.15.32:0/32941220 wait complete. 2023-09-30T22:21:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.982+0000 7fd55793f700 1 Processor -- start 2023-09-30T22:21:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 -- start start 2023-09-30T22:21:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 0x7fd55013a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd550096380 0x7fd55013ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55012ce60 0x7fd550129790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd550116400 con 0x7fd550091b00 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55012d350 con 0x7fd55012ce60 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd55793f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd55012c020 con 0x7fd550096380 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 0x7fd55013a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd555edc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55012ce60 0x7fd550129790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 0x7fd55013a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42756/0 (socket says 172.21.15.32:42756) 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd554eda700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd550096380 0x7fd55013ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.983+0000 7fd5556db700 1 -- 172.21.15.32:0/1278921545 learned_addr learned my addr 172.21.15.32:0/1278921545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5556db700 1 -- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd550096380 msgr2=0x7fd55013ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5556db700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd550096380 0x7fd55013ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5556db700 1 -- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55012ce60 msgr2=0x7fd550129790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5556db700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55012ce60 0x7fd550129790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5556db700 1 -- 172.21.15.32:0/1278921545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd548009a20 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5556db700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 0x7fd55013a620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd54c00c590 tx=0x7fd54c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5417fa700 1 -- 172.21.15.32:0/1278921545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0072e0 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5417fa700 1 -- 172.21.15.32:0/1278921545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c007440 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5440092a0 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd5417fa700 1 -- 172.21.15.32:0/1278921545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0057a0 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.984+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5501168a0 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.985+0000 7fd5417fa700 1 -- 172.21.15.32:0/1278921545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd54c005b90 con 0x7fd550091b00 2023-09-30T22:21:50.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.985+0000 7fd5417fa700 1 -- 172.21.15.32:0/1278921545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd54c03a700 con 0x7fd550091b00 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 Processor -- start 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 -- start start 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 0x56382d8dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d9070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d9075b0 0x56382d9099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382d8a7610 con 0x56382d8dd170 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x56382d81ddb0 con 0x56382d8dda40 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x56382d833030 con 0x56382d9075b0 2023-09-30T22:21:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 0x56382d8dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd554eda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d9070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd555edc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d9075b0 0x56382d9099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 0x56382d8dd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42760/0 (socket says 172.21.15.32:42760) 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.988+0000 7fd5556db700 1 -- 172.21.15.32:0/420713734 learned_addr learned my addr 172.21.15.32:0/420713734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd554eda700 1 -- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d9075b0 msgr2=0x56382d9099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd554eda700 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d9075b0 0x56382d9099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd554eda700 1 -- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 msgr2=0x56382d8dd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd554eda700 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 0x56382d8dd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd554eda700 1 -- 172.21.15.32:0/420713734 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56382d6ce710 con 0x56382d8dda40 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd5556db700 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 0x56382d8dd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd554eda700 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d9070c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd544013000 tx=0x7fd544016000).ready entity=mon.1 client_cookie=bb37ef61bdad258f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd53a7fc700 1 -- 172.21.15.32:0/420713734 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400dd30 con 0x56382d8dda40 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd53a7fc700 1 -- 172.21.15.32:0/420713734 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544024070 con 0x56382d8dda40 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.989+0000 7fd53a7fc700 1 -- 172.21.15.32:0/420713734 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400de90 con 0x56382d8dda40 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 -- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 msgr2=0x56382d9070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d9070c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd544013000 tx=0x7fd544016000).stop 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 -- 172.21.15.32:0/420713734 shutdown_connections 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d9075b0 0x56382d9099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dd170 0x56382d8dd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/420713734 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d9070c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 -- 172.21.15.32:0/420713734 >> 172.21.15.32:0/420713734 conn(0x56382d74b940 msgr2=0x56382d8fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 -- 172.21.15.32:0/420713734 shutdown_connections 2023-09-30T22:21:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 -- 172.21.15.32:0/420713734 wait complete. 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.990+0000 7fd5692f2b80 1 -- start start 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 Processor -- start 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d90d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d9075b0 0x56382d902b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382d833170 con 0x56382d9075b0 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x56382d8333f0 con 0x56382d8dda40 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x56382d7fe920 con 0x56382d8dd170 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5692f2b80 10 client.0 ll_register_callbacks cb 0x56382d7d3ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33652/0 (socket says 172.21.15.32:33652) 2023-09-30T22:21:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd554eda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d90d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd5556db700 1 -- 172.21.15.32:0/1520582314 learned_addr learned my addr 172.21.15.32:0/1520582314 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.991+0000 7fd555edc700 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d9075b0 0x56382d902b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5556db700 1 -- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 msgr2=0x56382d90d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5556db700 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d90d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5556db700 1 -- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d9075b0 msgr2=0x56382d902b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5556db700 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d9075b0 0x56382d902b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5556db700 1 -- 172.21.15.32:0/1520582314 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd550094e40 con 0x56382d8dd170 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5556db700 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90cda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd54c038eb0 tx=0x7fd54c015000).ready entity=mon.2 client_cookie=a24e5ece052d8539 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd52f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3302/0 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd52f7fe700 1 -- 172.21.15.32:0/1520582314 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c031070 con 0x56382d8dd170 2023-09-30T22:21:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd52f7fe700 1 -- 172.21.15.32:0/1520582314 <== mon.2 v2:172.21.15.32:3302/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd54c007d10 con 0x56382d8dd170 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56382d8d5f20 con 0x56382d8dd170 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd52f7fe700 1 -- 172.21.15.32:0/1520582314 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd54c0393f0 con 0x56382d8dd170 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 10 client.4445 fetch_fsmap learned FSMap version 6 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56382d805b00 con 0x56382d8dd170 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd52f7fe700 1 -- 172.21.15.32:0/1520582314 <== mon.2 v2:172.21.15.32:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56382d8d5f20 con 0x56382d8dd170 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd52f7fe700 1 -- 172.21.15.32:0/1520582314 <== mon.2 v2:172.21.15.32:3302/0 5 ==== fsmap(e 6) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fd54c02a120 con 0x56382d8dd170 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 10 client.4445 fetch_fsmap finished waiting for FSMap version 6 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 10 client.4445 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 20 client.4445 populate_metadata read hostname 'smithi032' 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 4 client.4445 mds_command: new command op to 4177 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.992+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 --> v1:172.21.15.32:6817/3095020018 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56382d8ab250 con 0x56382d9036c0 2023-09-30T22:21:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.993+0000 7fd52f7fe700 10 client.4445 ms_handle_connect on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.993+0000 7fd52f7fe700 1 -- 172.21.15.32:0/1520582314 <== mds.0 v1:172.21.15.32:6817/3095020018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x56382d8ab250 con 0x56382d9036c0 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.993+0000 7fd52f7fe700 10 client.4445 handle_command_reply: tid=0 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.993+0000 7fd5692f2b80 1 client.4445 shutdown 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 msgr2=0x56382d90cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90cda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd54c038eb0 tx=0x7fd54c015000).stop 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 shutdown_connections 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d9075b0 0x56382d902b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd52f7fe700 0 client.4445 ms_handle_reset on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90cda0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/1520582314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d8dda40 0x56382d90d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 >> 172.21.15.32:0/1520582314 conn(0x56382d74b940 msgr2=0x56382d8fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 shutdown_connections 2023-09-30T22:21:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 1 -- 172.21.15.32:0/1520582314 wait complete. 2023-09-30T22:21:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:50.994+0000 7fd5692f2b80 20 client.4445 trim_cache size 0 max 16384 2023-09-30T22:21:51.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.011+0000 7fd5692f2b80 1 Processor -- start 2023-09-30T22:21:51.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.011+0000 7fd5692f2b80 1 -- start start 2023-09-30T22:21:51.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.011+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d8e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.011+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d8fdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d919010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382d8f8870 con 0x56382d8dda40 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x56382d8a24c0 con 0x56382d9075b0 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x56382d8dd8f0 con 0x56382d8dd170 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d8e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd555edc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d919010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d8e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33664/0 (socket says 172.21.15.32:33664) 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd554eda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d8fdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 -- 172.21.15.32:0/2339215141 learned_addr learned my addr 172.21.15.32:0/2339215141 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 -- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 msgr2=0x56382d919010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d919010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 -- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 msgr2=0x56382d8fdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d8fdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 -- 172.21.15.32:0/2339215141 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56382d82bf70 con 0x56382d8dd170 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd5556db700 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d8e15b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fd54c010470 tx=0x7fd54c031000).ready entity=mon.2 client_cookie=7f4aa32c7d14bb2d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd53affd700 1 -- 172.21.15.32:0/2339215141 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c007d10 con 0x56382d8dd170 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd53affd700 1 -- 172.21.15.32:0/2339215141 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c0104e0 con 0x56382d8dd170 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.012+0000 7fd53affd700 1 -- 172.21.15.32:0/2339215141 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c005980 con 0x56382d8dd170 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 msgr2=0x56382d8e15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d8e15b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fd54c010470 tx=0x7fd54c031000).stop 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2339215141 shutdown_connections 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d919010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d8e15b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2339215141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d8fdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2339215141 >> 172.21.15.32:0/2339215141 conn(0x56382d74b940 msgr2=0x56382d903aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2339215141 shutdown_connections 2023-09-30T22:21:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2339215141 wait complete. 2023-09-30T22:21:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.013+0000 7fd5692f2b80 1 -- start start 2023-09-30T22:21:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 Processor -- start 2023-09-30T22:21:51.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d90c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d90c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56382d946070 con 0x56382d8dda40 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x56382d90ba40 con 0x56382d9075b0 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x56382d945500 con 0x56382d8dd170 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5692f2b80 10 client.0 ll_register_callbacks cb 0x56382d7d3ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5556db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33666/0 (socket says 172.21.15.32:33666) 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd5556db700 1 -- 172.21.15.32:0/2216179752 learned_addr learned my addr 172.21.15.32:0/2216179752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd555edc700 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d90c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.014+0000 7fd554eda700 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d90c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5556db700 1 -- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 msgr2=0x56382d90c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5556db700 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d90c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5556db700 1 -- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 msgr2=0x56382d90c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5556db700 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d90c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5556db700 1 -- 172.21.15.32:0/2216179752 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56382d805b00 con 0x56382d8dd170 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd555edc700 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d90c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5556db700 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90bc70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd54c00c560 tx=0x7fd54c015000).ready entity=mon.2 client_cookie=853f7ab98028ba58 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd52f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3302/0 2023-09-30T22:21:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd52f7fe700 1 -- 172.21.15.32:0/2216179752 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c031040 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd52f7fe700 1 -- 172.21.15.32:0/2216179752 <== mon.2 v2:172.21.15.32:3302/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd54c022020 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56382d8fb970 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd52f7fe700 1 -- 172.21.15.32:0/2216179752 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd54c02a120 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 10 client.4451 fetch_fsmap learned FSMap version 6 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56382d827580 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd52f7fe700 1 -- 172.21.15.32:0/2216179752 <== mon.2 v2:172.21.15.32:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c047030 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd52f7fe700 1 -- 172.21.15.32:0/2216179752 <== mon.2 v2:172.21.15.32:3302/0 5 ==== fsmap(e 6) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fd54c041020 con 0x56382d8dd170 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 10 client.4451 fetch_fsmap finished waiting for FSMap version 6 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 10 client.4451 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 20 client.4451 populate_metadata read hostname 'smithi032' 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 4 client.4451 mds_command: new command op to 4177 tid=0[{"prefix": "status"}] 2023-09-30T22:21:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.015+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 --> v1:172.21.15.32:6817/3095020018 -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd544024070 con 0x56382d961ee0 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd52f7fe700 10 client.4451 ms_handle_connect on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd52f7fe700 1 -- 172.21.15.32:0/2216179752 <== mds.0 v1:172.21.15.32:6817/3095020018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 2737338693) 0x7fd544024070 con 0x56382d961ee0 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd52f7fe700 10 client.4451 handle_command_reply: tid=0 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 client.4451 shutdown 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 msgr2=0x56382d90bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90bc70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd54c00c560 tx=0x7fd54c015000).stop 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 shutdown_connections 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x56382d9075b0 0x56382d90c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x56382d8dd170 0x56382d90bc70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 --2- 172.21.15.32:0/2216179752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56382d8dda40 0x56382d90c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 >> 172.21.15.32:0/2216179752 conn(0x56382d74b940 msgr2=0x56382d8fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd52f7fe700 0 client.4451 ms_handle_reset on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 shutdown_connections 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.016+0000 7fd5692f2b80 1 -- 172.21.15.32:0/2216179752 wait complete. 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd5692f2b80 20 client.4451 trim_cache size 0 max 16384 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9e20f926-ec7d-45ec-8e27-e5e820815322", 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4177, 2023-09-30T22:21:51.024 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 13.827112751, 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 6, 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 17, 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 17, 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 19.110446975999999 2023-09-30T22:21:51.025 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T22:21:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 >> v1:172.21.15.32:6816/118188 conn(0x7fd53402bb60 legacy=0x7fd53402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 msgr2=0x7fd55013a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 0x7fd55013a620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd54c00c590 tx=0x7fd54c007c80).stop 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 shutdown_connections 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55012ce60 0x7fd550129790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd550091b00 0x7fd55013a620 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 --2- 172.21.15.32:0/1278921545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd550096380 0x7fd55013ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 >> 172.21.15.32:0/1278921545 conn(0x7fd550097470 msgr2=0x7fd550059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 shutdown_connections 2023-09-30T22:21:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.017+0000 7fd55793f700 1 -- 172.21.15.32:0/1278921545 wait complete. 2023-09-30T22:21:51.030 INFO:teuthology.run_tasks:Running task exec... 2023-09-30T22:21:51.040 INFO:teuthology.task.exec:Executing custom commands... 2023-09-30T22:21:51.040 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:21:51.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client pacific' 2023-09-30T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.170+0000 7fd1a152b700 1 Processor -- start 2023-09-30T22:21:51.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.170+0000 7fd1a152b700 1 -- start start 2023-09-30T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.170+0000 7fd1a152b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c065500 0x7fd19c098510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd1a152b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 0x7fd19c098a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd1a152b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c066040 con 0x7fd19c098ef0 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd1a152b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0658e0 con 0x7fd19c097da0 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd1a152b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c066530 con 0x7fd19c065500 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 0x7fd19c098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd19c098ef0 0x7fd19c131480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57664/0 (socket says 172.21.15.32:57664) 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 0x7fd19c098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44802/0 (socket says 172.21.15.32:44802) 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c065500 0x7fd19c098510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19b7fe700 1 -- 172.21.15.32:0/783599969 learned_addr learned my addr 172.21.15.32:0/783599969 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 -- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c065500 msgr2=0x7fd19c098510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:51.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 --2- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c065500 0x7fd19c098510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd199ffb700 1 -- 172.21.15.32:0/783599969 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387812141 0 0) 0x7fd19c066040 con 0x7fd19c098ef0 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 -- 172.21.15.32:0/783599969 >> v1:172.21.15.32:6789/0 conn(0x7fd19c098ef0 legacy=0x7fd19c131480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 -- 172.21.15.32:0/783599969 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0952f0 con 0x7fd19c097da0 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19affd700 1 --2- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c065500 0x7fd19c098510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd19a7fc700 1 --2- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 0x7fd19c098a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd1840097c0 tx=0x7fd184009060).ready entity=mon.1 client_cookie=b2ef683c3b8ddbb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd199ffb700 1 -- 172.21.15.32:0/783599969 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1840090d0 con 0x7fd19c097da0 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd199ffb700 1 -- 172.21.15.32:0/783599969 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18400f510 con 0x7fd19c097da0 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.171+0000 7fd199ffb700 1 -- 172.21.15.32:0/783599969 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400f890 con 0x7fd19c097da0 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 -- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 msgr2=0x7fd19c098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 --2- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 0x7fd19c098a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd1840097c0 tx=0x7fd184009060).stop 2023-09-30T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 -- 172.21.15.32:0/783599969 shutdown_connections 2023-09-30T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 --2- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c065500 0x7fd19c098510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 --2- 172.21.15.32:0/783599969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c097da0 0x7fd19c098a00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 -- 172.21.15.32:0/783599969 >> 172.21.15.32:0/783599969 conn(0x7fd19c09a6e0 msgr2=0x7fd19c09aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 -- 172.21.15.32:0/783599969 shutdown_connections 2023-09-30T22:21:51.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.172+0000 7fd1a152b700 1 -- 172.21.15.32:0/783599969 wait complete. 2023-09-30T22:21:51.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.173+0000 7fd1a152b700 1 Processor -- start 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.173+0000 7fd1a152b700 1 -- start start 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd1a152b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 0x7fd19c13a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd1a152b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c098ef0 0x7fd19c13a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd1a152b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c13adc0 0x7fd19c11ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd1a152b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c099bb0 con 0x7fd19c065500 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd1a152b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c09ceb0 con 0x7fd19c13adc0 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd1a152b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c13b1e0 con 0x7fd19c098ef0 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 0x7fd19c13a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 0x7fd19c13a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42794/0 (socket says 172.21.15.32:42794) 2023-09-30T22:21:51.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c13adc0 0x7fd19c11ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 -- 172.21.15.32:0/1057012559 learned_addr learned my addr 172.21.15.32:0/1057012559 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19a7fc700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c098ef0 0x7fd19c13a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 -- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c098ef0 msgr2=0x7fd19c13a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c098ef0 0x7fd19c13a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 -- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c13adc0 msgr2=0x7fd19c11ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c13adc0 0x7fd19c11ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 -- 172.21.15.32:0/1057012559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c009250 con 0x7fd19c065500 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c13adc0 0x7fd19c11ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd19affd700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 0x7fd19c13a3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd18c00ec80 tx=0x7fd18c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.174+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c005880 con 0x7fd19c065500 2023-09-30T22:21:51.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.175+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c0059e0 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.175+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1840092a0 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.175+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c005d60 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.175+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c11e960 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.176+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd18c00ca60 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.176+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd18c03af60 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.176+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c092100 con 0x7fd19c065500 2023-09-30T22:21:51.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.180+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd18c016070 con 0x7fd19c065500 2023-09-30T22:21:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:51.368+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "pacific"} v 0) v1 -- 0x7fd19c11e960 con 0x7fd19c065500 2023-09-30T22:21:52.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.231+0000 7fd192ffd700 1 -- 172.21.15.32:0/1057012559 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "pacific"}]=0 set require_min_compat_client to pacific v18) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd19c11e960 con 0x7fd19c065500 2023-09-30T22:21:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 >> v1:172.21.15.32:6816/118188 conn(0x7fd17c02bb60 legacy=0x7fd17c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 msgr2=0x7fd19c13a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 0x7fd19c13a3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd18c00ec80 tx=0x7fd18c006f40).stop 2023-09-30T22:21:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 shutdown_connections 2023-09-30T22:21:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c13adc0 0x7fd19c11ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c065500 0x7fd19c13a3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 --2- 172.21.15.32:0/1057012559 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c098ef0 0x7fd19c13a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 >> 172.21.15.32:0/1057012559 conn(0x7fd19c09a6e0 msgr2=0x7fd19c1142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.237+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 shutdown_connections 2023-09-30T22:21:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.238+0000 7fd1a152b700 1 -- 172.21.15.32:0/1057012559 wait complete. 2023-09-30T22:21:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:set require_min_compat_client to pacific 2023-09-30T22:21:52.252 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:21:52.262 INFO:teuthology.task.print:**** done ceph 2023-09-30T22:21:52.262 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-30T22:21:52.287 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-30T22:21:52.287 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-09-30T22:21:52.287 INFO:tasks.ceph_fuse:client.0 config is {} 2023-09-30T22:21:52.287 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-30T22:21:52.287 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:52.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:52.288 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-09-30T22:21:52.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:52.313 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-09-30T22:21:52.386 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2023-09-30T22:21:52.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:21:52.388 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-30T22:21:52.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-30T22:21:52.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 Processor -- start 2023-09-30T22:21:52.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 -- start start 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724096380 0x7f3724094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724091b00 0x7f3724127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3724096760 con 0x7f3724094c20 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3724091ff0 con 0x7f3724091b00 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f372b729700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3724092c80 con 0x7f3724096380 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f3728cc4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724091b00 0x7f3724127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f37294c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724096380 0x7f3724094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.528+0000 7f3729cc6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3724094c20 0x7f3724127af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57680/0 (socket says 172.21.15.32:57680) 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f3729cc6700 1 -- 172.21.15.32:0/1308650744 learned_addr learned my addr 172.21.15.32:0/1308650744 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f371bfff700 1 -- 172.21.15.32:0/1308650744 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524717804 0 0) 0x7f3724096760 con 0x7f3724094c20 2023-09-30T22:21:52.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f3728cc4700 1 -- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724096380 msgr2=0x7f3724094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f3728cc4700 1 --2- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724096380 0x7f3724094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f3728cc4700 1 -- 172.21.15.32:0/1308650744 >> v1:172.21.15.32:6789/0 conn(0x7f3724094c20 legacy=0x7f3724127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f3728cc4700 1 -- 172.21.15.32:0/1308650744 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724093900 con 0x7f3724091b00 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f37294c5700 1 --2- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724096380 0x7f3724094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f3728cc4700 1 --2- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724091b00 0x7f3724127600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3714009790 tx=0x7f3714009060).ready entity=mon.1 client_cookie=d13ce1efa33a44d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f371bfff700 1 -- 172.21.15.32:0/1308650744 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371400f510 con 0x7f3724091b00 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f371bfff700 1 -- 172.21.15.32:0/1308650744 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371400f670 con 0x7f3724091b00 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.529+0000 7f371bfff700 1 -- 172.21.15.32:0/1308650744 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371400f9f0 con 0x7f3724091b00 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 -- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724091b00 msgr2=0x7f3724127600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 --2- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724091b00 0x7f3724127600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3714009790 tx=0x7f3714009060).stop 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 -- 172.21.15.32:0/1308650744 shutdown_connections 2023-09-30T22:21:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 --2- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724096380 0x7f3724094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 --2- 172.21.15.32:0/1308650744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724091b00 0x7f3724127600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 -- 172.21.15.32:0/1308650744 >> 172.21.15.32:0/1308650744 conn(0x7f3724097470 msgr2=0x7f3724097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 -- 172.21.15.32:0/1308650744 shutdown_connections 2023-09-30T22:21:52.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.530+0000 7f372b729700 1 -- 172.21.15.32:0/1308650744 wait complete. 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.531+0000 7f372b729700 1 Processor -- start 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.531+0000 7f372b729700 1 -- start start 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3724091b00 0x7f37241165d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 0x7f3724116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724096380 0x7f3724116fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37241174a0 con 0x7f3724091b00 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3724099250 con 0x7f3724096380 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f372406a520 con 0x7f3724094c20 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f37294c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3724091b00 0x7f37241165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:52.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 0x7f3724116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f37294c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3724091b00 0x7f37241165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42804/0 (socket says 172.21.15.32:42804) 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 0x7f3724116ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33692/0 (socket says 172.21.15.32:33692) 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f37294c5700 1 -- 172.21.15.32:0/511145698 learned_addr learned my addr 172.21.15.32:0/511145698 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3729cc6700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724096380 0x7f3724116fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 -- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724096380 msgr2=0x7f3724116fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724096380 0x7f3724116fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 -- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3724091b00 msgr2=0x7f37241165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3724091b00 0x7f37241165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 -- 172.21.15.32:0/511145698 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200092a0 con 0x7f3724094c20 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f3728cc4700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 0x7f3724116ac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3714014000 tx=0x7f3714017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:52.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371400fe80 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371400f7e0 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371401b670 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37140092a0 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.532+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3724119960 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.533+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f371401b9e0 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.533+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f371403c4e0 con 0x7f3724094c20 2023-09-30T22:21:52.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.533+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372409a500 con 0x7f3724094c20 2023-09-30T22:21:52.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.538+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f37140189a0 con 0x7f3724094c20 2023-09-30T22:21:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.754+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3724097850 con 0x7f3724094c20 2023-09-30T22:21:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.754+0000 7f37197fa700 1 -- 172.21.15.32:0/511145698 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3714018350 con 0x7f3724094c20 2023-09-30T22:21:52.754 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-30T22:21:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 >> v1:172.21.15.32:6816/118188 conn(0x7f370802bb60 legacy=0x7f370802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 msgr2=0x7f3724116ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 0x7f3724116ac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3714014000 tx=0x7f3714017000).stop 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 shutdown_connections 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3724096380 0x7f3724116fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3724091b00 0x7f37241165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 --2- 172.21.15.32:0/511145698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3724094c20 0x7f3724116ac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 >> 172.21.15.32:0/511145698 conn(0x7f3724097470 msgr2=0x7f3724124b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 shutdown_connections 2023-09-30T22:21:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:52.756+0000 7f372b729700 1 -- 172.21.15.32:0/511145698 wait complete. 2023-09-30T22:21:52.769 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-30T22:21:52.769 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-30T22:21:52.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2023-09-30T22:21:52.769 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-30T22:21:52.769 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:52.769 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-30T22:21:52.770 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-30T22:21:52.770 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-30T22:21:52.770 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-30T22:21:52.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:52.770 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2023-09-30T22:21:52.792 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2023-09-30T22:21:52.793 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42503sec preferred_lft 42503sec 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2023-09-30T22:21:52.794 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:21:52.795 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-30T22:21:52.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-30T22:21:52.795 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-09-30T22:21:52.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:52.960 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-30T22:21:52.992 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-09-30T22:21:52.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:52.994 DEBUG:teuthology.orchestra.run.smithi121:> route 2023-09-30T22:21:53.076 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2023-09-30T22:21:53.076 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-30T22:21:53.077 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-30T22:21:53.077 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-30T22:21:53.077 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-30T22:21:53.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:53.077 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-09-30T22:21:53.077 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-09-30T22:21:53.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-30T22:21:53.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-30T22:21:53.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-30T22:21:53.078 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-09-30T22:21:53.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:53.363 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-09-30T22:21:53.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:53.382 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2023-09-30T22:21:53.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:53.440 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-09-30T22:21:53.440 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-09-30T22:21:53.440 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:21:53.440 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-30T22:21:53.440 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-09-30T22:21:53.535 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-30T22:21:53.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-30T22:21:53.536 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-09-30T22:21:53.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:53.762 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2023-09-30T22:21:53.762 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2023-09-30T22:21:53.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2023-09-30T22:21:53.763 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2023-09-30T22:21:53.763 DEBUG:teuthology.orchestra.run.smithi121:> ') 2023-09-30T22:21:53.861 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-30T22:21:53.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T22:21:53.861 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:53.878 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-30T22:21:53.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T22:21:53.879 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:53.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2023-09-30T22:21:54.004 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2023-09-30T22:21:54.062 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/systemd 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2023-09-30T22:21:54.063 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/cpuset 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/rdma 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/blkio 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/memory 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/devices 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/freezer 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/hugetlb 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/perf_event 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-30T22:21:54.064 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/pids 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2023-09-30T22:21:54.065 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2023-09-30T22:21:54.066 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2023-09-30T22:21:54.066 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:21:54.066 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:21:54.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:54.067 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2023-09-30T22:21:54.124 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-30T22:21:54.124 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-30T22:21:54.167 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2023-09-30T22:21:54.193 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2023-09-30T22:21:54.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:2023-09-30T22:21:54.252+0000 7f727b54d3c0 -1 init, newargv = 0x558aefcdb600 newargc=15 2023-09-30T22:21:54.252 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[112242]: starting ceph client 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/systemd 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2023-09-30T22:21:54.253 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/cpuset 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/rdma 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/blkio 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/memory 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/devices 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/freezer 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/hugetlb 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/perf_event 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-30T22:21:54.254 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup/pids 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2023-09-30T22:21:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2023-09-30T22:21:54.256 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2023-09-30T22:21:54.256 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:21:54.256 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:21:54.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:54.257 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2023-09-30T22:21:54.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[112242]: starting fuse 2023-09-30T22:21:54.274 INFO:teuthology.orchestra.run.smithi121.stdout:46 2023-09-30T22:21:54.275 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-09-30T22:21:54.275 DEBUG:teuthology.orchestra.run.smithi121:> sudo stdin-killer -- python3 -c ' 2023-09-30T22:21:54.275 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2023-09-30T22:21:54.275 DEBUG:teuthology.orchestra.run.smithi121:> import re 2023-09-30T22:21:54.275 DEBUG:teuthology.orchestra.run.smithi121:> import os 2023-09-30T22:21:54.275 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2023-09-30T22:21:54.276 DEBUG:teuthology.orchestra.run.smithi121:> 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> return f 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.0")) 2023-09-30T22:21:54.277 DEBUG:teuthology.orchestra.run.smithi121:> ' 2023-09-30T22:21:54.402 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.0.112242.asok 2023-09-30T22:21:54.403 INFO:teuthology.orchestra.run.smithi121.stderr:2023-09-30T22:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-30T22:21:54.409 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112242.asok 2023-09-30T22:21:54.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:54.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112242.asok status 2023-09-30T22:21:54.560 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-09-30T22:21:54.561 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2023-09-30T22:21:54.561 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "eeb13ac974775bd0cce96a2c687e8bcc2faca43d", 2023-09-30T22:21:54.561 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 16.2.14-44-geeb13ac9 (eeb13ac974775bd0cce96a2c687e8bcc2faca43d) pacific (stable)", 2023-09-30T22:21:54.561 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "0", 2023-09-30T22:21:54.561 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2023-09-30T22:21:54.561 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-30T22:21:54.562 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "112242", 2023-09-30T22:21:54.562 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2023-09-30T22:21:54.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-09-30T22:21:54.562 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2023-09-30T22:21:54.562 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4469, 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 4469 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2023-09-30T22:21:54.563 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "172.21.15.121:0", 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 1059206754 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "172.21.15.121:0", 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 1059206754 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.4469 172.21.15.121:0/1059206754", 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "172.21.15.121:0/1059206754", 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2023-09-30T22:21:54.564 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 6, 2023-09-30T22:21:54.565 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 18, 2023-09-30T22:21:54.565 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2023-09-30T22:21:54.565 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2023-09-30T22:21:54.565 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2023-09-30T22:21:54.565 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-09-30T22:21:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:54.571 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2023-09-30T22:21:54.571 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:54.591 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2023-09-30T22:21:54.591 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:54.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:21:54.591 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-30T22:21:54.661 INFO:teuthology.run_tasks:Running task mds_pre_upgrade... 2023-09-30T22:21:54.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.811+0000 7fccbee4e700 1 Processor -- start 2023-09-30T22:21:54.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.811+0000 7fccbee4e700 1 -- start start 2023-09-30T22:21:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbee4e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbee4e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbee4e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb8095f00 con 0x7fccb8059070 2023-09-30T22:21:54.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbee4e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8059450 con 0x7fccb81251c0 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbee4e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8096190 con 0x7fccb8125dd0 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccb7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbcbea700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fccb8059070 0x7fccb81256c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57682/0 (socket says 172.21.15.32:57682) 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbd3eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbcbea700 1 -- 172.21.15.32:0/3935766630 learned_addr learned my addr 172.21.15.32:0/3935766630 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccb77fe700 1 -- 172.21.15.32:0/3935766630 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936767383 0 0) 0x7fccb8095f00 con 0x7fccb8059070 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccb77fe700 1 -- 172.21.15.32:0/3935766630 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca80033f0 con 0x7fccb8059070 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbd3eb700 1 -- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 msgr2=0x7fccb812da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbd3eb700 1 --2- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbd3eb700 1 -- 172.21.15.32:0/3935766630 >> v1:172.21.15.32:6789/0 conn(0x7fccb8059070 legacy=0x7fccb81256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbd3eb700 1 -- 172.21.15.32:0/3935766630 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8059dd0 con 0x7fccb8125dd0 2023-09-30T22:21:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccb7fff700 1 --2- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:54.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.812+0000 7fccbd3eb700 1 --2- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812ff80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fccb0009e90 tx=0x7fccb0009780).ready entity=mon.2 client_cookie=cd0f12437cb734d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:54.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccb77fe700 1 -- 172.21.15.32:0/3935766630 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000e070 con 0x7fccb8125dd0 2023-09-30T22:21:54.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccb77fe700 1 -- 172.21.15.32:0/3935766630 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb000bab0 con 0x7fccb8125dd0 2023-09-30T22:21:54.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccb77fe700 1 -- 172.21.15.32:0/3935766630 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000be30 con 0x7fccb8125dd0 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 -- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 msgr2=0x7fccb812ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 --2- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812ff80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fccb0009e90 tx=0x7fccb0009780).stop 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 -- 172.21.15.32:0/3935766630 shutdown_connections 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 --2- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812ff80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 --2- 172.21.15.32:0/3935766630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 -- 172.21.15.32:0/3935766630 >> 172.21.15.32:0/3935766630 conn(0x7fccb8097470 msgr2=0x7fccb8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 -- 172.21.15.32:0/3935766630 shutdown_connections 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.813+0000 7fccbee4e700 1 -- 172.21.15.32:0/3935766630 wait complete. 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 Processor -- start 2023-09-30T22:21:54.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 -- start start 2023-09-30T22:21:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb8059070 0x7fccb812d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb812ce30 con 0x7fccb8059070 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80958d0 con 0x7fccb81251c0 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.815+0000 7fccbee4e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccb812f850 con 0x7fccb8125dd0 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccb7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccb7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44864/0 (socket says 172.21.15.32:44864) 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccb7fff700 1 -- 172.21.15.32:0/2221355985 learned_addr learned my addr 172.21.15.32:0/2221355985 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbcbea700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb8059070 0x7fccb812d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 -- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 msgr2=0x7fccb812a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 -- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb8059070 msgr2=0x7fccb812d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb8059070 0x7fccb812d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 -- 172.21.15.32:0/2221355985 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40092a0 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbd3eb700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812d9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fccb0014000 tx=0x7fccb8095bd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0013e40 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb001d630 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb0009a20 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb001d9f0 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.816+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb812aeb0 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.817+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fccb0003040 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.817+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fccb003cc90 con 0x7fccb8125dd0 2023-09-30T22:21:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.818+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb812b040 con 0x7fccb8125dd0 2023-09-30T22:21:54.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.822+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fccb0025dc0 con 0x7fccb8125dd0 2023-09-30T22:21:54.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.998+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccb805a260 con 0x7fccb8125dd0 2023-09-30T22:21:54.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:54.999+0000 7fccb4ff9700 1 -- 172.21.15.32:0/2221355985 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+11340 (secure 0 0 0) 0x7fccb0025360 con 0x7fccb8125dd0 2023-09-30T22:21:54.999 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:54.999 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:52.226037+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 >> v1:172.21.15.32:6816/118188 conn(0x7fcc9c02bb60 legacy=0x7fcc9c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 msgr2=0x7fccb812d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812d9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fccb0014000 tx=0x7fccb8095bd0).stop 2023-09-30T22:21:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 shutdown_connections 2023-09-30T22:21:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fccb8125dd0 0x7fccb812d9a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccb8059070 0x7fccb812d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 --2- 172.21.15.32:0/2221355985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fccb81251c0 0x7fccb812a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 >> 172.21.15.32:0/2221355985 conn(0x7fccb8097470 msgr2=0x7fccb8099ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 shutdown_connections 2023-09-30T22:21:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.001+0000 7fccbee4e700 1 -- 172.21.15.32:0/2221355985 wait complete. 2023-09-30T22:21:55.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:21:55.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:21:55.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.148+0000 7ff9f30f9700 1 Processor -- start 2023-09-30T22:21:55.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.148+0000 7ff9f30f9700 1 -- start start 2023-09-30T22:21:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f30f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec125f00 0x7ff9ec125250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f30f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f30f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec098790 con 0x7ff9ec059070 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f30f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec059450 con 0x7ff9ec125f00 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f30f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec059da0 con 0x7ff9ec12db40 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9ebfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec125f00 0x7ff9ec125250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f0e95700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9ec059070 0x7ff9ec1257f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57688/0 (socket says 172.21.15.32:57688) 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f0e95700 1 -- 172.21.15.32:0/1546749934 learned_addr learned my addr 172.21.15.32:0/1546749934 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f1696700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9eb7fe700 1 -- 172.21.15.32:0/1546749934 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269609330 0 0) 0x7ff9ec098790 con 0x7ff9ec059070 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9eb7fe700 1 -- 172.21.15.32:0/1546749934 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d80033f0 con 0x7ff9ec059070 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9ebfff700 1 -- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 msgr2=0x7ff9ec12ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9ebfff700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9ebfff700 1 -- 172.21.15.32:0/1546749934 >> v1:172.21.15.32:6789/0 conn(0x7ff9ec059070 legacy=0x7ff9ec1257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9ebfff700 1 -- 172.21.15.32:0/1546749934 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0996f0 con 0x7ff9ec125f00 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9f1696700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.149+0000 7ff9ebfff700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec125f00 0x7ff9ec125250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff9d4009790 tx=0x7ff9d4009060).ready entity=mon.1 client_cookie=622e19d1f88b99a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9eb7fe700 1 -- 172.21.15.32:0/1546749934 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40090d0 con 0x7ff9ec125f00 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9eb7fe700 1 -- 172.21.15.32:0/1546749934 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d400f510 con 0x7ff9ec125f00 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9eb7fe700 1 -- 172.21.15.32:0/1546749934 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400f890 con 0x7ff9ec125f00 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 -- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec125f00 msgr2=0x7ff9ec125250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec125f00 0x7ff9ec125250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff9d4009790 tx=0x7ff9d4009060).stop 2023-09-30T22:21:55.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 -- 172.21.15.32:0/1546749934 shutdown_connections 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/1546749934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec125f00 0x7ff9ec125250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 -- 172.21.15.32:0/1546749934 >> 172.21.15.32:0/1546749934 conn(0x7ff9ec097480 msgr2=0x7ff9ec097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 -- 172.21.15.32:0/1546749934 shutdown_connections 2023-09-30T22:21:55.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.150+0000 7ff9f30f9700 1 -- 172.21.15.32:0/1546749934 wait complete. 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.151+0000 7ff9f30f9700 1 Processor -- start 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.151+0000 7ff9f30f9700 1 -- start start 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f30f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 0x7ff9ec12b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f30f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9ec125f00 0x7ff9ec12bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f30f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f30f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec065040 con 0x7ff9ec125f00 2023-09-30T22:21:55.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f30f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec098ea0 con 0x7ff9ec059070 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f30f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec12a8d0 con 0x7ff9ec12db40 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 0x7ff9ec12b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f1696700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 0x7ff9ec12b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44880/0 (socket says 172.21.15.32:44880) 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 -- 172.21.15.32:0/4245495073 learned_addr learned my addr 172.21.15.32:0/4245495073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9ebfff700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9ec125f00 0x7ff9ec12bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 -- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 msgr2=0x7ff9ec12c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 -- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9ec125f00 msgr2=0x7ff9ec12bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9ec125f00 0x7ff9ec12bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 -- 172.21.15.32:0/4245495073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e4009a20 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f1696700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9ebfff700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9ec125f00 0x7ff9ec12bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.152+0000 7ff9f0e95700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 0x7ff9ec12b820 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff9dc00c5c0 tx=0x7ff9dc007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.153+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0072e0 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.153+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc007440 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.153+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d40092a0 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.153+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0057a0 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.153+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec06a150 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.153+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff9dc005b90 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.154+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff9dc03a650 con 0x7ff9ec059070 2023-09-30T22:21:55.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.154+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec05a260 con 0x7ff9ec059070 2023-09-30T22:21:55.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.159+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff9dc01cb40 con 0x7ff9ec059070 2023-09-30T22:21:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.333+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff9ec09a3e0 con 0x7ff9ec059070 2023-09-30T22:21:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.333+0000 7ff9e8ff9700 1 -- 172.21.15.32:0/4245495073 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff9dc011350 con 0x7ff9ec059070 2023-09-30T22:21:55.333 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:21:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 >> v1:172.21.15.32:6816/118188 conn(0x7ff9cc02bb60 legacy=0x7ff9cc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 msgr2=0x7ff9ec12b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 0x7ff9ec12b820 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff9dc00c5c0 tx=0x7ff9dc007c80).stop 2023-09-30T22:21:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 shutdown_connections 2023-09-30T22:21:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9ec12db40 0x7ff9ec12c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9ec059070 0x7ff9ec12b820 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 --2- 172.21.15.32:0/4245495073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9ec125f00 0x7ff9ec12bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 >> 172.21.15.32:0/4245495073 conn(0x7ff9ec097480 msgr2=0x7ff9ec1286d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 shutdown_connections 2023-09-30T22:21:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.335+0000 7ff9f30f9700 1 -- 172.21.15.32:0/4245495073 wait complete. 2023-09-30T22:21:55.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.482+0000 7f0e6228a700 1 Processor -- start 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e6228a700 1 -- start start 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e6228a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 0x7f0e5c1275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e6228a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e5c128000 0x7f0e5c12b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e6228a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c091c80 con 0x7f0e5c127be0 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e6228a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c099dd0 con 0x7f0e5c128000 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e6228a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0999d0 con 0x7f0e5c0926d0 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 0x7f0e5c1275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e60827700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e5c128000 0x7f0e5c12b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 0x7f0e5c1275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44896/0 (socket says 172.21.15.32:44896) 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e5b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e5c127be0 0x7f0e5c12a9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57696/0 (socket says 172.21.15.32:57696) 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.483+0000 7f0e5bfff700 1 -- 172.21.15.32:0/4143532156 learned_addr learned my addr 172.21.15.32:0/4143532156 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5bfff700 1 -- 172.21.15.32:0/4143532156 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c128000 msgr2=0x7f0e5c12b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5bfff700 1 --2- 172.21.15.32:0/4143532156 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c128000 0x7f0e5c12b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5bfff700 1 -- 172.21.15.32:0/4143532156 >> v1:172.21.15.32:6789/0 conn(0x7f0e5c127be0 legacy=0x7f0e5c12a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5bfff700 1 -- 172.21.15.32:0/4143532156 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c09a280 con 0x7f0e5c0926d0 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5bfff700 1 --2- 172.21.15.32:0/4143532156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 0x7f0e5c1275c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0e4c009830 tx=0x7f0e4c0090f0).ready entity=mon.1 client_cookie=62f758d92df8677c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5affd700 1 -- 172.21.15.32:0/4143532156 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00b580 con 0x7f0e5c0926d0 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5affd700 1 -- 172.21.15.32:0/4143532156 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c00b6e0 con 0x7f0e5c0926d0 2023-09-30T22:21:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e5affd700 1 -- 172.21.15.32:0/4143532156 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00ba60 con 0x7f0e5c0926d0 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e6228a700 1 -- 172.21.15.32:0/4143532156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 msgr2=0x7f0e5c1275c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.484+0000 7f0e6228a700 1 --2- 172.21.15.32:0/4143532156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 0x7f0e5c1275c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0e4c009830 tx=0x7f0e4c0090f0).stop 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 -- 172.21.15.32:0/4143532156 shutdown_connections 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 --2- 172.21.15.32:0/4143532156 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c128000 0x7f0e5c12b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 --2- 172.21.15.32:0/4143532156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0926d0 0x7f0e5c1275c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 -- 172.21.15.32:0/4143532156 >> 172.21.15.32:0/4143532156 conn(0x7f0e5c097480 msgr2=0x7f0e5c097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 -- 172.21.15.32:0/4143532156 shutdown_connections 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 -- 172.21.15.32:0/4143532156 wait complete. 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.485+0000 7f0e6228a700 1 Processor -- start 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 -- start start 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e5c0926d0 0x7f0e5c113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 0x7f0e5c114310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c128000 0x7f0e5c114800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c114cf0 con 0x7f0e5c128000 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c1152f0 con 0x7f0e5c127be0 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e6228a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c116f80 con 0x7f0e5c0926d0 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e5b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 0x7f0e5c114310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e60827700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c128000 0x7f0e5c114800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e5b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 0x7f0e5c114310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44900/0 (socket says 172.21.15.32:44900) 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e5b7fe700 1 -- 172.21.15.32:0/962506773 learned_addr learned my addr 172.21.15.32:0/962506773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.486+0000 7f0e5bfff700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e5c0926d0 0x7f0e5c113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e60827700 1 -- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e5c0926d0 msgr2=0x7f0e5c113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e60827700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e5c0926d0 0x7f0e5c113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e60827700 1 -- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 msgr2=0x7f0e5c114310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e60827700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 0x7f0e5c114310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e60827700 1 -- 172.21.15.32:0/962506773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c009370 con 0x7f0e5c128000 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e5b7fe700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 0x7f0e5c114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e60827700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c128000 0x7f0e5c114800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f0e54009e50 tx=0x7f0e5400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54009bb0 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5400f040 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c117320 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400ccd0 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.487+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c094030 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.488+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0e54013610 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.488+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0e5403b7f0 con 0x7f0e5c128000 2023-09-30T22:21:55.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.488+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c117800 con 0x7f0e5c128000 2023-09-30T22:21:55.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.493+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0e5c117800 con 0x7f0e5c128000 2023-09-30T22:21:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.667+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e5c117800 con 0x7f0e5c128000 2023-09-30T22:21:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.667+0000 7f0e43fff700 1 -- 172.21.15.32:0/962506773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e5c117800 con 0x7f0e5c128000 2023-09-30T22:21:55.667 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:21:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 >> v1:172.21.15.32:6816/118188 conn(0x7f0e4802bb60 legacy=0x7f0e4802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c128000 msgr2=0x7f0e5c114800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c128000 0x7f0e5c114800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f0e54009e50 tx=0x7f0e5400b040).stop 2023-09-30T22:21:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 shutdown_connections 2023-09-30T22:21:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c128000 0x7f0e5c114800 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e5c0926d0 0x7f0e5c113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 --2- 172.21.15.32:0/962506773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c127be0 0x7f0e5c114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 >> 172.21.15.32:0/962506773 conn(0x7f0e5c097480 msgr2=0x7f0e5c12a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 shutdown_connections 2023-09-30T22:21:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.669+0000 7f0e6228a700 1 -- 172.21.15.32:0/962506773 wait complete. 2023-09-30T22:21:55.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:21:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.817+0000 7efcd1185700 1 Processor -- start 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.817+0000 7efcd1185700 1 -- start start 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcd1185700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efccc0694d0 0x7efccc096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcd1185700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc12e240 0x7efccc130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcd1185700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efccc095e60 con 0x7efccc096900 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcd1185700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efccc068d40 con 0x7efccc0694d0 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcd1185700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efccc068fb0 con 0x7efccc12e240 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efccc0694d0 0x7efccc096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccb59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc12e240 0x7efccc130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcca59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efccc096900 0x7efccc096d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57702/0 (socket says 172.21.15.32:57702) 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcca59c700 1 -- 172.21.15.32:0/2256484484 learned_addr learned my addr 172.21.15.32:0/2256484484 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccad9d700 1 -- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc12e240 msgr2=0x7efccc130630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccad9d700 1 --2- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc12e240 0x7efccc130630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccad9d700 1 -- 172.21.15.32:0/2256484484 >> v1:172.21.15.32:6789/0 conn(0x7efccc096900 legacy=0x7efccc096d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccad9d700 1 -- 172.21.15.32:0/2256484484 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc068a10 con 0x7efccc0694d0 2023-09-30T22:21:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efcc9d9b700 1 -- 172.21.15.32:0/2256484484 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094136030 0 0) 0x7efccc095e60 con 0x7efccc096900 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccb59e700 1 --2- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc12e240 0x7efccc130630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.818+0000 7efccad9d700 1 --2- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc096410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7efcbc009760 tx=0x7efcbc009060).ready entity=mon.0 client_cookie=1f5445872df0d175 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcc9d9b700 1 -- 172.21.15.32:0/2256484484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00f510 con 0x7efccc0694d0 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcc9d9b700 1 -- 172.21.15.32:0/2256484484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc00f670 con 0x7efccc0694d0 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcc9d9b700 1 -- 172.21.15.32:0/2256484484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00f9f0 con 0x7efccc0694d0 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 -- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 msgr2=0x7efccc096410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 --2- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc096410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7efcbc009760 tx=0x7efcbc009060).stop 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 -- 172.21.15.32:0/2256484484 shutdown_connections 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 --2- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc12e240 0x7efccc130630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 --2- 172.21.15.32:0/2256484484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc096410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 -- 172.21.15.32:0/2256484484 >> 172.21.15.32:0/2256484484 conn(0x7efccc09ba60 msgr2=0x7efccc09be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 -- 172.21.15.32:0/2256484484 shutdown_connections 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.819+0000 7efcd1185700 1 -- 172.21.15.32:0/2256484484 wait complete. 2023-09-30T22:21:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.820+0000 7efcd1185700 1 Processor -- start 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.820+0000 7efcd1185700 1 -- start start 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcd1185700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcd1185700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 0x7efccc064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcd1185700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efccc12e240 0x7efccc064f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcd1185700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc065a10 con 0x7efccc0694d0 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcd1185700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc11e6b0 con 0x7efccc12e240 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcd1185700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efccc067d80 con 0x7efccc096900 2023-09-30T22:21:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 0x7efccc064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efccb59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efccc12e240 0x7efccc064f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efccad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 0x7efccc064a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33742/0 (socket says 172.21.15.32:33742) 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efccb59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efccc12e240 0x7efccc064f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44914/0 (socket says 172.21.15.32:44914) 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 -- 172.21.15.32:0/2206069941 learned_addr learned my addr 172.21.15.32:0/2206069941 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 -- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efccc12e240 msgr2=0x7efccc064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efccc12e240 0x7efccc064f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 -- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 msgr2=0x7efccc064540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc064540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.821+0000 7efcca59c700 1 -- 172.21.15.32:0/2206069941 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc4009a20 con 0x7efccc096900 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.822+0000 7efcca59c700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 0x7efccc064a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7efcc000c4c0 tx=0x7efcc0007c10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.822+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00072e0 con 0x7efccc096900 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.822+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc0007460 con 0x7efccc096900 2023-09-30T22:21:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.822+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0092a0 con 0x7efccc096900 2023-09-30T22:21:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.822+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00059f0 con 0x7efccc096900 2023-09-30T22:21:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.822+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efccc11d7f0 con 0x7efccc096900 2023-09-30T22:21:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.823+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7efcc0013450 con 0x7efccc096900 2023-09-30T22:21:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.823+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7efcc003a790 con 0x7efccc096900 2023-09-30T22:21:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.823+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efccc099090 con 0x7efccc096900 2023-09-30T22:21:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:55.828+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7efcc0018030 con 0x7efccc096900 2023-09-30T22:21:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.003+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efccc11d260 con 0x7efccc096900 2023-09-30T22:21:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.003+0000 7efcb2ffd700 1 -- 172.21.15.32:0/2206069941 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v18) v1 ==== 104+0+11 (secure 0 0 0) 0x7efcc001de60 con 0x7efccc096900 2023-09-30T22:21:56.003 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:21:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 >> v1:172.21.15.32:6816/118188 conn(0x7efcb802bb60 legacy=0x7efcb802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 msgr2=0x7efccc064a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 0x7efccc064a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7efcc000c4c0 tx=0x7efcc0007c10).stop 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 shutdown_connections 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efccc12e240 0x7efccc064f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efccc0694d0 0x7efccc064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 --2- 172.21.15.32:0/2206069941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efccc096900 0x7efccc064a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 >> 172.21.15.32:0/2206069941 conn(0x7efccc09ba60 msgr2=0x7efccc09a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 shutdown_connections 2023-09-30T22:21:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.005+0000 7efcd1185700 1 -- 172.21.15.32:0/2206069941 wait complete. 2023-09-30T22:21:56.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:21:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.151+0000 7f2e07f6a700 1 Processor -- start 2023-09-30T22:21:56.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.151+0000 7f2e07f6a700 1 -- start start 2023-09-30T22:21:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.151+0000 7f2e07f6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 0x7f2e00098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.151+0000 7f2e07f6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 0x7f2e00099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e07f6a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 0x7f2e0012e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e07f6a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e00093c20 con 0x7f2e0012c150 2023-09-30T22:21:56.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e07f6a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00092b90 con 0x7f2e00098c80 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e07f6a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00092d80 con 0x7f2e00093840 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05505700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 0x7f2e00099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 0x7f2e00098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e06507700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 0x7f2e0012e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 0x7f2e00098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33758/0 (socket says 172.21.15.32:33758) 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05505700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 0x7f2e00099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44918/0 (socket says 172.21.15.32:44918) 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 -- 172.21.15.32:0/3192479242 learned_addr learned my addr 172.21.15.32:0/3192479242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e06507700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 0x7f2e0012e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42848/0 (socket says 172.21.15.32:42848) 2023-09-30T22:21:56.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 -- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 msgr2=0x7f2e00099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 --2- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 0x7f2e00099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 -- 172.21.15.32:0/3192479242 >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 msgr2=0x7f2e0012e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 --2- 172.21.15.32:0/3192479242 >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 0x7f2e0012e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 -- 172.21.15.32:0/3192479242 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e00099b70 con 0x7f2e00093840 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e06507700 1 --2- 172.21.15.32:0/3192479242 >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 0x7f2e0012e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05505700 1 --2- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 0x7f2e00099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e05d06700 1 --2- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 0x7f2e00098790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2dfc006fd0 tx=0x7f2dfc0090f0).ready entity=mon.2 client_cookie=431ef92dc0779b19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e04d04700 1 -- 172.21.15.32:0/3192479242 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00f510 con 0x7f2e00093840 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e04d04700 1 -- 172.21.15.32:0/3192479242 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc00f670 con 0x7f2e00093840 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.152+0000 7f2e04d04700 1 -- 172.21.15.32:0/3192479242 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00f9f0 con 0x7f2e00093840 2023-09-30T22:21:56.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 -- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 msgr2=0x7f2e00098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 0x7f2e00098790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2dfc006fd0 tx=0x7f2dfc0090f0).stop 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 -- 172.21.15.32:0/3192479242 shutdown_connections 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/3192479242 >> v2:172.21.15.32:3300/0 conn(0x7f2e0012c150 0x7f2e0012e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e00093840 0x7f2e00098790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/3192479242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00098c80 0x7f2e00099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 -- 172.21.15.32:0/3192479242 >> 172.21.15.32:0/3192479242 conn(0x7f2e00097480 msgr2=0x7f2e00097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 -- 172.21.15.32:0/3192479242 shutdown_connections 2023-09-30T22:21:56.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.153+0000 7f2e07f6a700 1 -- 172.21.15.32:0/3192479242 wait complete. 2023-09-30T22:21:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.154+0000 7f2e07f6a700 1 Processor -- start 2023-09-30T22:21:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.154+0000 7f2e07f6a700 1 -- start start 2023-09-30T22:21:56.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e07f6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 0x7f2e0012ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e07f6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e00098c80 0x7f2e0012b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e07f6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e0012c150 0x7f2e00116ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e07f6a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00066690 con 0x7f2e00098c80 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e07f6a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0011db00 con 0x7f2e00093840 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e07f6a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0006a520 con 0x7f2e0012c150 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 0x7f2e0012ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 0x7f2e0012ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44930/0 (socket says 172.21.15.32:44930) 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 -- 172.21.15.32:0/4141129431 learned_addr learned my addr 172.21.15.32:0/4141129431 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05505700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e00098c80 0x7f2e0012b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e06507700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e0012c150 0x7f2e00116ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 -- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e0012c150 msgr2=0x7f2e00116ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e0012c150 0x7f2e00116ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 -- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e00098c80 msgr2=0x7f2e0012b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e00098c80 0x7f2e0012b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 -- 172.21.15.32:0/4141129431 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00092a0 con 0x7f2e00093840 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05505700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e00098c80 0x7f2e0012b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2e05d06700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 0x7f2e0012ae20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2dfc009df0 tx=0x7f2dfc017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.155+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00f7e0 con 0x7f2e00093840 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.156+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc00f940 con 0x7f2e00093840 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.156+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df8009a20 con 0x7f2e00093840 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.156+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc01c670 con 0x7f2e00093840 2023-09-30T22:21:56.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.156+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc009370 con 0x7f2e00093840 2023-09-30T22:21:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.157+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2dfc01c9e0 con 0x7f2e00093840 2023-09-30T22:21:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.157+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2dfc03cc30 con 0x7f2e00093840 2023-09-30T22:21:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.157+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0005a260 con 0x7f2e00093840 2023-09-30T22:21:56.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.162+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2dfc020030 con 0x7f2e00093840 2023-09-30T22:21:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.335+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e000993b0 con 0x7f2e00093840 2023-09-30T22:21:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.336+0000 7f2df5ffb700 1 -- 172.21.15.32:0/4141129431 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v18) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2dfc006a40 con 0x7f2e00093840 2023-09-30T22:21:56.336 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 >> v1:172.21.15.32:6816/118188 conn(0x7f2de402bb60 legacy=0x7f2de402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 msgr2=0x7f2e0012ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 0x7f2e0012ae20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2dfc009df0 tx=0x7f2dfc017000).stop 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 shutdown_connections 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e0012c150 0x7f2e00116ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e00093840 0x7f2e0012ae20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 --2- 172.21.15.32:0/4141129431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e00098c80 0x7f2e0012b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 >> 172.21.15.32:0/4141129431 conn(0x7f2e00097480 msgr2=0x7f2e00110fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 shutdown_connections 2023-09-30T22:21:56.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.338+0000 7f2e07f6a700 1 -- 172.21.15.32:0/4141129431 wait complete. 2023-09-30T22:21:56.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:56.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.482+0000 7fb233e68700 1 Processor -- start 2023-09-30T22:21:56.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.482+0000 7fb233e68700 1 -- start start 2023-09-30T22:21:56.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb233e68700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0650f0 0x7fb22c0986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb233e68700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb22c0992a0 0x7fb22c132480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb233e68700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c096160 con 0x7fb22c097c30 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb233e68700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22c0654d0 con 0x7fb22c0992a0 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb233e68700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c065dd0 con 0x7fb22c0650f0 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb22c0992a0 0x7fb22c132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb22c0992a0 0x7fb22c132480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42864/0 (socket says 172.21.15.32:42864) 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 -- 172.21.15.32:0/1629738012 learned_addr learned my addr 172.21.15.32:0/1629738012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb231c04700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0650f0 0x7fb22c0986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb230c02700 1 -- 172.21.15.32:0/1629738012 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366326898 0 0) 0x7fb22c096160 con 0x7fb22c097c30 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb230c02700 1 -- 172.21.15.32:0/1629738012 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2180033f0 con 0x7fb22c097c30 2023-09-30T22:21:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 -- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0650f0 msgr2=0x7fb22c0986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0650f0 0x7fb22c0986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 -- 172.21.15.32:0/1629738012 >> v1:172.21.15.32:6789/0 conn(0x7fb22c097c30 legacy=0x7fb22c098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb232405700 1 -- 172.21.15.32:0/1629738012 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c099790 con 0x7fb22c0992a0 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.483+0000 7fb231c04700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0650f0 0x7fb22c0986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.484+0000 7fb232405700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0992a0 0x7fb22c132480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fb224009f20 tx=0x7fb224009840).ready entity=mon.0 client_cookie=83fd67435319754c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.484+0000 7fb230c02700 1 -- 172.21.15.32:0/1629738012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400e070 con 0x7fb22c0992a0 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.484+0000 7fb230c02700 1 -- 172.21.15.32:0/1629738012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22400b970 con 0x7fb22c0992a0 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.484+0000 7fb230c02700 1 -- 172.21.15.32:0/1629738012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400bcf0 con 0x7fb22c0992a0 2023-09-30T22:21:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 -- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0992a0 msgr2=0x7fb22c132480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0992a0 0x7fb22c132480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fb224009f20 tx=0x7fb224009840).stop 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 -- 172.21.15.32:0/1629738012 shutdown_connections 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0992a0 0x7fb22c132480 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 --2- 172.21.15.32:0/1629738012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0650f0 0x7fb22c0986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 -- 172.21.15.32:0/1629738012 >> 172.21.15.32:0/1629738012 conn(0x7fb22c09dc70 msgr2=0x7fb22c09e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 -- 172.21.15.32:0/1629738012 shutdown_connections 2023-09-30T22:21:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.485+0000 7fb233e68700 1 -- 172.21.15.32:0/1629738012 wait complete. 2023-09-30T22:21:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 Processor -- start 2023-09-30T22:21:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 -- start start 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 0x7fb22c11a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0992a0 0x7fb22c11a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb22c11abb0 0x7fb22c11aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c132310 con 0x7fb22c0650f0 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c11c800 con 0x7fb22c11abb0 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb233e68700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c099a80 con 0x7fb22c0992a0 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb231c04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 0x7fb22c11a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb231403700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0992a0 0x7fb22c11a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb231c04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 0x7fb22c11a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42868/0 (socket says 172.21.15.32:42868) 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb231c04700 1 -- 172.21.15.32:0/66933462 learned_addr learned my addr 172.21.15.32:0/66933462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.486+0000 7fb232405700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb22c11abb0 0x7fb22c11aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb231c04700 1 -- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0992a0 msgr2=0x7fb22c11a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb231c04700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0992a0 0x7fb22c11a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb231c04700 1 -- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb22c11abb0 msgr2=0x7fb22c11aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb231c04700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb22c11abb0 0x7fb22c11aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb231c04700 1 -- 172.21.15.32:0/66933462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280092a0 con 0x7fb22c0650f0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb231c04700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 0x7fb22c11a1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb22800ea20 tx=0x7fb228009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800ee60 con 0x7fb22c0650f0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228009600 con 0x7fb22c0650f0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb224009ae0 con 0x7fb22c0650f0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800ca60 con 0x7fb22c0650f0 2023-09-30T22:21:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.487+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c11b4e0 con 0x7fb22c0650f0 2023-09-30T22:21:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.488+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb22801b400 con 0x7fb22c0650f0 2023-09-30T22:21:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.488+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb22803b480 con 0x7fb22c0650f0 2023-09-30T22:21:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.488+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c05a260 con 0x7fb22c0650f0 2023-09-30T22:21:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.493+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb228016030 con 0x7fb22c0650f0 2023-09-30T22:21:56.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.717+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb22c0973a0 con 0x7fb22c0650f0 2023-09-30T22:21:56.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.717+0000 7fb21dffb700 1 -- 172.21.15.32:0/66933462 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7fb228019310 con 0x7fb22c0650f0 2023-09-30T22:21:56.717 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:56.717 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:40.197617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:21:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 >> v1:172.21.15.32:6816/118188 conn(0x7fb21002bb90 legacy=0x7fb21002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 msgr2=0x7fb22c11a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 0x7fb22c11a1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fb22800ea20 tx=0x7fb228009060).stop 2023-09-30T22:21:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 shutdown_connections 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb22c11abb0 0x7fb22c11aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb22c0650f0 0x7fb22c11a1d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 --2- 172.21.15.32:0/66933462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb22c0992a0 0x7fb22c11a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 >> 172.21.15.32:0/66933462 conn(0x7fb22c09dc70 msgr2=0x7fb22c09a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 shutdown_connections 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.719+0000 7fb233e68700 1 -- 172.21.15.32:0/66933462 wait complete. 2023-09-30T22:21:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2023-09-30T22:21:56.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:21:56.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.862+0000 7fde08868700 1 Processor -- start 2023-09-30T22:21:56.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde08868700 1 -- start start 2023-09-30T22:21:56.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde08868700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 0x7fde04094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde08868700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde04094c20 0x7fde04127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde08868700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde04096760 con 0x7fde04091b00 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde08868700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde04091ff0 con 0x7fde04094c20 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde08868700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde04092c80 con 0x7fde04096380 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde0259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 0x7fde04094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde0259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 0x7fde04094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44946/0 (socket says 172.21.15.32:44946) 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde0259c700 1 -- 172.21.15.32:0/789131790 learned_addr learned my addr 172.21.15.32:0/789131790 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.863+0000 7fde02d9d700 1 --2- 172.21.15.32:0/789131790 >> v2:172.21.15.32:3300/0 conn(0x7fde04094c20 0x7fde04127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde02d9d700 1 -- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 msgr2=0x7fde04094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde02d9d700 1 --2- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 0x7fde04094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde02d9d700 1 -- 172.21.15.32:0/789131790 >> v1:172.21.15.32:6789/0 conn(0x7fde04091b00 legacy=0x7fde04127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde02d9d700 1 -- 172.21.15.32:0/789131790 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde04093900 con 0x7fde04094c20 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde0159a700 1 -- 172.21.15.32:0/789131790 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975926518 0 0) 0x7fde04096760 con 0x7fde04091b00 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde0259c700 1 --2- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 0x7fde04094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde02d9d700 1 --2- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04094c20 0x7fde04127d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fddfc009e90 tx=0x7fddfc009780).ready entity=mon.0 client_cookie=adb7b18436f29b89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde0159a700 1 -- 172.21.15.32:0/789131790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00e070 con 0x7fde04094c20 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde0159a700 1 -- 172.21.15.32:0/789131790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc00bb20 con 0x7fde04094c20 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.864+0000 7fde0159a700 1 -- 172.21.15.32:0/789131790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00bea0 con 0x7fde04094c20 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 -- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04094c20 msgr2=0x7fde04127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 --2- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04094c20 0x7fde04127d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fddfc009e90 tx=0x7fddfc009780).stop 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 -- 172.21.15.32:0/789131790 shutdown_connections 2023-09-30T22:21:56.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 --2- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04094c20 0x7fde04127d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 --2- 172.21.15.32:0/789131790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04096380 0x7fde04094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 -- 172.21.15.32:0/789131790 >> 172.21.15.32:0/789131790 conn(0x7fde04097470 msgr2=0x7fde04097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 -- 172.21.15.32:0/789131790 shutdown_connections 2023-09-30T22:21:56.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.865+0000 7fde08868700 1 -- 172.21.15.32:0/789131790 wait complete. 2023-09-30T22:21:56.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 Processor -- start 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 -- start start 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde04091b00 0x7fde04116840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 0x7fde041128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 0x7fde04112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde04124980 con 0x7fde04096380 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde04098e40 con 0x7fde04094c20 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde08868700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde0406a520 con 0x7fde04091b00 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde01d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 0x7fde041128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde02d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 0x7fde04112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde02d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 0x7fde04112d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42878/0 (socket says 172.21.15.32:42878) 2023-09-30T22:21:56.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde01d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 0x7fde041128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44950/0 (socket says 172.21.15.32:44950) 2023-09-30T22:21:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde02d9d700 1 -- 172.21.15.32:0/3466949394 learned_addr learned my addr 172.21.15.32:0/3466949394 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.866+0000 7fde0259c700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde04091b00 0x7fde04116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde01d9b700 1 -- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde04091b00 msgr2=0x7fde04116840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde01d9b700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde04091b00 0x7fde04116840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde01d9b700 1 -- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 msgr2=0x7fde04112d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:56.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde01d9b700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 0x7fde04112d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde01d9b700 1 -- 172.21.15.32:0/3466949394 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009370 con 0x7fde04094c20 2023-09-30T22:21:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde02d9d700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 0x7fde04112d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde01d9b700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 0x7fde041128a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fddf800c410 tx=0x7fddf8007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8012400 con 0x7fde04094c20 2023-09-30T22:21:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8012580 con 0x7fde04094c20 2023-09-30T22:21:56.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc009a20 con 0x7fde04094c20 2023-09-30T22:21:56.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8012940 con 0x7fde04094c20 2023-09-30T22:21:56.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.867+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde04115da0 con 0x7fde04094c20 2023-09-30T22:21:56.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.868+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fddf8012cf0 con 0x7fde04094c20 2023-09-30T22:21:56.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.868+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fddf803a4b0 con 0x7fde04094c20 2023-09-30T22:21:56.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.868+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde04124fc0 con 0x7fde04094c20 2023-09-30T22:21:56.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:56.873+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fddf8017030 con 0x7fde04094c20 2023-09-30T22:21:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.053+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde0405a260 con 0x7fde04094c20 2023-09-30T22:21:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.054+0000 7fddf27fc700 1 -- 172.21.15.32:0/3466949394 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+11340 (secure 0 0 0) 0x7fddf8024070 con 0x7fde04094c20 2023-09-30T22:21:57.054 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:57.054 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:21:52.226037+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 >> v1:172.21.15.32:6816/118188 conn(0x7fdde402bb60 legacy=0x7fdde402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 msgr2=0x7fde041128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 0x7fde041128a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fddf800c410 tx=0x7fddf8007b60).stop 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 shutdown_connections 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde04096380 0x7fde04112d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde04091b00 0x7fde04116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 --2- 172.21.15.32:0/3466949394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde04094c20 0x7fde041128a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 >> 172.21.15.32:0/3466949394 conn(0x7fde04097470 msgr2=0x7fde04126b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 shutdown_connections 2023-09-30T22:21:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.056+0000 7fde08868700 1 -- 172.21.15.32:0/3466949394 wait complete. 2023-09-30T22:21:57.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-30T22:21:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 Processor -- start 2023-09-30T22:21:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 -- start start 2023-09-30T22:21:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 0x7fe630125260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe630125c70 0x7fe630094300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe630098790 con 0x7fe6301257f0 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe630059450 con 0x7fe630125c70 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.202+0000 7fe636b9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300996f0 con 0x7fe630059070 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe634938700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 0x7fe630125260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe635139700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe630125c70 0x7fe630094300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe634938700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 0x7fe630125260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33782/0 (socket says 172.21.15.32:33782) 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe634938700 1 -- 172.21.15.32:0/2886540382 learned_addr learned my addr 172.21.15.32:0/2886540382 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe62ffff700 1 --1- 172.21.15.32:0/2886540382 >> v1:172.21.15.32:6789/0 conn(0x7fe6301257f0 0x7fe630093bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57722/0 (socket says 172.21.15.32:57722) 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe635139700 1 -- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 msgr2=0x7fe630125260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe635139700 1 --2- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 0x7fe630125260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe635139700 1 -- 172.21.15.32:0/2886540382 >> v1:172.21.15.32:6789/0 conn(0x7fe6301257f0 legacy=0x7fe630093bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe635139700 1 -- 172.21.15.32:0/2886540382 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63009a5a0 con 0x7fe630125c70 2023-09-30T22:21:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe62f7fe700 1 -- 172.21.15.32:0/2886540382 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745558680 0 0) 0x7fe630098790 con 0x7fe6301257f0 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe634938700 1 --2- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 0x7fe630125260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe635139700 1 --2- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe630125c70 0x7fe630094300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fe628009e60 tx=0x7fe628009780).ready entity=mon.0 client_cookie=c3d16397c080a9a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe62f7fe700 1 -- 172.21.15.32:0/2886540382 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800e070 con 0x7fe630125c70 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe62f7fe700 1 -- 172.21.15.32:0/2886540382 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62800bb60 con 0x7fe630125c70 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.203+0000 7fe62f7fe700 1 -- 172.21.15.32:0/2886540382 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628012410 con 0x7fe630125c70 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 -- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe630125c70 msgr2=0x7fe630094300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 --2- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe630125c70 0x7fe630094300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fe628009e60 tx=0x7fe628009780).stop 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 -- 172.21.15.32:0/2886540382 shutdown_connections 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 --2- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe630125c70 0x7fe630094300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 --2- 172.21.15.32:0/2886540382 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630059070 0x7fe630125260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 -- 172.21.15.32:0/2886540382 >> 172.21.15.32:0/2886540382 conn(0x7fe630097480 msgr2=0x7fe630097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 -- 172.21.15.32:0/2886540382 shutdown_connections 2023-09-30T22:21:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.204+0000 7fe636b9c700 1 -- 172.21.15.32:0/2886540382 wait complete. 2023-09-30T22:21:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.205+0000 7fe636b9c700 1 Processor -- start 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.205+0000 7fe636b9c700 1 -- start start 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 0x7fe630137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6301257f0 0x7fe630138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 0x7fe630067710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6301232c0 con 0x7fe6301257f0 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63012cd60 con 0x7fe630059070 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe630098ea0 con 0x7fe630125c70 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 0x7fe630137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe635139700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 0x7fe630067710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe62ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6301257f0 0x7fe630138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 0x7fe630137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44958/0 (socket says 172.21.15.32:44958) 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 -- 172.21.15.32:0/173617699 learned_addr learned my addr 172.21.15.32:0/173617699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe635139700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 0x7fe630067710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33784/0 (socket says 172.21.15.32:33784) 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 -- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 msgr2=0x7fe630067710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 0x7fe630067710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 -- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6301257f0 msgr2=0x7fe630138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6301257f0 0x7fe630138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 -- 172.21.15.32:0/173617699 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009370 con 0x7fe630059070 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe635139700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 0x7fe630067710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe634938700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 0x7fe630137fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe624009cb0 tx=0x7fe6240090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624015070 con 0x7fe630059070 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62400ba00 con 0x7fe630059070 2023-09-30T22:21:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.206+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe628009a20 con 0x7fe630059070 2023-09-30T22:21:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.207+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400bd80 con 0x7fe630059070 2023-09-30T22:21:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.207+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe630069330 con 0x7fe630059070 2023-09-30T22:21:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.207+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fe62401a630 con 0x7fe630059070 2023-09-30T22:21:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.207+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fe62403ee20 con 0x7fe630059070 2023-09-30T22:21:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.208+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe630111c70 con 0x7fe630059070 2023-09-30T22:21:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.212+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fe62401a8e0 con 0x7fe630059070 2023-09-30T22:21:57.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.430+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe63005a260 con 0x7fe630059070 2023-09-30T22:21:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.786+0000 7fe62cff9700 1 -- 172.21.15.32:0/173617699 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe624022340 con 0x7fe630059070 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 >> v1:172.21.15.32:6816/118188 conn(0x7fe61402bb60 legacy=0x7fe61402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 msgr2=0x7fe630137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 0x7fe630137fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe624009cb0 tx=0x7fe6240090f0).stop 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 shutdown_connections 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe630125c70 0x7fe630067710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe630059070 0x7fe630137fa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 --2- 172.21.15.32:0/173617699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6301257f0 0x7fe630138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 >> 172.21.15.32:0/173617699 conn(0x7fe630097480 msgr2=0x7fe630095320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 shutdown_connections 2023-09-30T22:21:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.789+0000 7fe636b9c700 1 -- 172.21.15.32:0/173617699 wait complete. 2023-09-30T22:21:57.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2023-09-30T22:21:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.940+0000 7f1adf560700 1 Processor -- start 2023-09-30T22:21:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 -- start start 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 0x7f1ad81275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad8127be0 0x7f1ad8128000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 0x7f1ad812ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad8099dd0 con 0x7f1ad812a9a0 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8091c80 con 0x7f1ad80926d0 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adf560700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad80999d0 con 0x7f1ad8127be0 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1addafd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 0x7f1ad812ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1add2fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 0x7f1ad81275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.941+0000 7f1adcafb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad8127be0 0x7f1ad8128000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1add2fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 0x7f1ad81275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44964/0 (socket says 172.21.15.32:44964) 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1addafd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 0x7f1ad812ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42900/0 (socket says 172.21.15.32:42900) 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1add2fc700 1 -- 172.21.15.32:0/748931987 learned_addr learned my addr 172.21.15.32:0/748931987 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1adcafb700 1 -- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 msgr2=0x7f1ad81275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1adcafb700 1 --2- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 0x7f1ad81275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1adcafb700 1 -- 172.21.15.32:0/748931987 >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 msgr2=0x7f1ad812ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1adcafb700 1 --2- 172.21.15.32:0/748931987 >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 0x7f1ad812ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1adcafb700 1 -- 172.21.15.32:0/748931987 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad809a280 con 0x7f1ad8127be0 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1add2fc700 1 --2- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 0x7f1ad81275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1addafd700 1 --2- 172.21.15.32:0/748931987 >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 0x7f1ad812ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1adcafb700 1 --2- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad8127be0 0x7f1ad8128000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1ac8009790 tx=0x7f1ac8009060).ready entity=mon.2 client_cookie=4fc7b8f343616952 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1acffff700 1 -- 172.21.15.32:0/748931987 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800d4a0 con 0x7f1ad8127be0 2023-09-30T22:21:57.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1acffff700 1 -- 172.21.15.32:0/748931987 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac800d600 con 0x7f1ad8127be0 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.942+0000 7f1acffff700 1 -- 172.21.15.32:0/748931987 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800d980 con 0x7f1ad8127be0 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 -- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad8127be0 msgr2=0x7f1ad8128000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 --2- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad8127be0 0x7f1ad8128000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1ac8009790 tx=0x7f1ac8009060).stop 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 -- 172.21.15.32:0/748931987 shutdown_connections 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 --2- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad80926d0 0x7f1ad81275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 --2- 172.21.15.32:0/748931987 >> v2:172.21.15.32:3300/0 conn(0x7f1ad812a9a0 0x7f1ad812ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 --2- 172.21.15.32:0/748931987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad8127be0 0x7f1ad8128000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 -- 172.21.15.32:0/748931987 >> 172.21.15.32:0/748931987 conn(0x7f1ad8097480 msgr2=0x7f1ad8097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 -- 172.21.15.32:0/748931987 shutdown_connections 2023-09-30T22:21:57.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.943+0000 7f1adf560700 1 -- 172.21.15.32:0/748931987 wait complete. 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 Processor -- start 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 -- start start 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 0x7f1ad8117d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ad8127be0 0x7f1ad8113d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad812a9a0 0x7f1ad8114220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8093930 con 0x7f1ad8127be0 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad809a4b0 con 0x7f1ad812a9a0 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.944+0000 7f1adf560700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8117540 con 0x7f1ad80926d0 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 0x7f1ad8117d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1addafd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad812a9a0 0x7f1ad8114220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 0x7f1ad8117d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33802/0 (socket says 172.21.15.32:33802) 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 -- 172.21.15.32:0/4141833451 learned_addr learned my addr 172.21.15.32:0/4141833451 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1adcafb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ad8127be0 0x7f1ad8113d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 -- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad812a9a0 msgr2=0x7f1ad8114220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad812a9a0 0x7f1ad8114220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 -- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ad8127be0 msgr2=0x7f1ad8113d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ad8127be0 0x7f1ad8113d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 -- 172.21.15.32:0/4141833451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad4009370 con 0x7f1ad80926d0 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1addafd700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad812a9a0 0x7f1ad8114220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1adcafb700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ad8127be0 0x7f1ad8113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1add2fc700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 0x7f1ad8117d20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1ad400ea20 tx=0x7f1ad40090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400c850 con 0x7f1ad80926d0 2023-09-30T22:21:57.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad400c9b0 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad0009a20 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400cd30 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.945+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac80092a0 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.946+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1ad4012600 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.946+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1ad403b5b0 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.946+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad8094c20 con 0x7f1ad80926d0 2023-09-30T22:21:57.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:57.951+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1ad4016070 con 0x7f1ad80926d0 2023-09-30T22:21:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.170+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f1ad805a260 con 0x7f1ad80926d0 2023-09-30T22:21:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.787+0000 7f1acd7fa700 1 -- 172.21.15.32:0/4141833451 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1ad4016020 con 0x7f1ad80926d0 2023-09-30T22:21:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.790+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 >> v1:172.21.15.32:6816/118188 conn(0x7f1abc02bb60 legacy=0x7f1abc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.790+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 msgr2=0x7f1ad8117d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.790+0000 7f1adf560700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 0x7f1ad8117d20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1ad400ea20 tx=0x7f1ad40090f0).stop 2023-09-30T22:21:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 shutdown_connections 2023-09-30T22:21:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ad812a9a0 0x7f1ad8114220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ad80926d0 0x7f1ad8117d20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 --2- 172.21.15.32:0/4141833451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ad8127be0 0x7f1ad8113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 >> 172.21.15.32:0/4141833451 conn(0x7f1ad8097480 msgr2=0x7f1ad8096150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 shutdown_connections 2023-09-30T22:21:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.791+0000 7f1adf560700 1 -- 172.21.15.32:0/4141833451 wait complete. 2023-09-30T22:21:58.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:21:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 Processor -- start 2023-09-30T22:21:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 -- start start 2023-09-30T22:21:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92b4096380 0x7f92b4094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 0x7f92b4127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 0x7f92b4127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:58.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b4096760 con 0x7f92b4096380 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4091ff0 con 0x7f92b4091b00 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.940+0000 7f92baabc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4092c80 con 0x7f92b4094c20 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 0x7f92b4127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b9059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 0x7f92b4127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b9059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 0x7f92b4127af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33816/0 (socket says 172.21.15.32:33816) 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b9059700 1 -- 172.21.15.32:0/1584621009 learned_addr learned my addr 172.21.15.32:0/1584621009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 0x7f92b4127600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44986/0 (socket says 172.21.15.32:44986) 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b8858700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92b4096380 0x7f92b4094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:58.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 -- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 msgr2=0x7f92b4127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 --2- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 0x7f92b4127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 -- 172.21.15.32:0/1584621009 >> v2:172.21.15.32:3300/0 conn(0x7f92b4096380 msgr2=0x7f92b4094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 --2- 172.21.15.32:0/1584621009 >> v2:172.21.15.32:3300/0 conn(0x7f92b4096380 0x7f92b4094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 -- 172.21.15.32:0/1584621009 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4093900 con 0x7f92b4091b00 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b9059700 1 --2- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 0x7f92b4127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b8858700 1 --2- 172.21.15.32:0/1584621009 >> v2:172.21.15.32:3300/0 conn(0x7f92b4096380 0x7f92b4094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b3fff700 1 --2- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 0x7f92b4127600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f92a0009760 tx=0x7f92a0009060).ready entity=mon.1 client_cookie=dcbe02248a08bef7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b37fe700 1 -- 172.21.15.32:0/1584621009 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a000d4a0 con 0x7f92b4091b00 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b37fe700 1 -- 172.21.15.32:0/1584621009 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a000d600 con 0x7f92b4091b00 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.941+0000 7f92b37fe700 1 -- 172.21.15.32:0/1584621009 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a000d980 con 0x7f92b4091b00 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 -- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 msgr2=0x7f92b4127600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 --2- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 0x7f92b4127600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f92a0009760 tx=0x7f92a0009060).stop 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 -- 172.21.15.32:0/1584621009 shutdown_connections 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 --2- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4094c20 0x7f92b4127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 --2- 172.21.15.32:0/1584621009 >> v2:172.21.15.32:3300/0 conn(0x7f92b4096380 0x7f92b4094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 --2- 172.21.15.32:0/1584621009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4091b00 0x7f92b4127600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 -- 172.21.15.32:0/1584621009 >> 172.21.15.32:0/1584621009 conn(0x7f92b4097470 msgr2=0x7f92b4097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 -- 172.21.15.32:0/1584621009 shutdown_connections 2023-09-30T22:21:58.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.942+0000 7f92baabc700 1 -- 172.21.15.32:0/1584621009 wait complete. 2023-09-30T22:21:58.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.943+0000 7f92baabc700 1 Processor -- start 2023-09-30T22:21:58.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.943+0000 7f92baabc700 1 -- start start 2023-09-30T22:21:58.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.943+0000 7f92baabc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 0x7f92b4123e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.943+0000 7f92baabc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4094c20 0x7f92b4124360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.943+0000 7f92baabc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4096380 0x7f92b41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.943+0000 7f92baabc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4115540 con 0x7f92b4091b00 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92baabc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41175e0 con 0x7f92b4094c20 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92baabc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4118990 con 0x7f92b4096380 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b8858700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 0x7f92b4123e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4096380 0x7f92b41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b8858700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 0x7f92b4123e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42918/0 (socket says 172.21.15.32:42918) 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b8858700 1 -- 172.21.15.32:0/3283141109 learned_addr learned my addr 172.21.15.32:0/3283141109 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b3fff700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4094c20 0x7f92b4124360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 -- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4094c20 msgr2=0x7f92b4124360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4094c20 0x7f92b4124360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 -- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 msgr2=0x7f92b4123e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 0x7f92b4123e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 -- 172.21.15.32:0/3283141109 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac009ae0 con 0x7f92b4096380 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b8858700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 0x7f92b4123e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b9059700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4096380 0x7f92b41140d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f92ac009f10 tx=0x7f92ac00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0092e0 con 0x7f92b4096380 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac00f040 con 0x7f92b4096380 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a80092a0 con 0x7f92b4096380 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00cb70 con 0x7f92b4096380 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.944+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a00092a0 con 0x7f92b4096380 2023-09-30T22:21:58.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.945+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f92ac014460 con 0x7f92b4096380 2023-09-30T22:21:58.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.945+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f92ac03b0a0 con 0x7f92b4096380 2023-09-30T22:21:58.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.946+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b406a4e0 con 0x7f92b4096380 2023-09-30T22:21:58.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:58.950+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f92ac012360 con 0x7f92b4096380 2023-09-30T22:21:59.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.168+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92b405a260 con 0x7f92b4096380 2023-09-30T22:21:59.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.169+0000 7f92b0ff9700 1 -- 172.21.15.32:0/3283141109 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f92ac01cb10 con 0x7f92b4096380 2023-09-30T22:21:59.169 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:21:59.169 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:58.784702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 >> v1:172.21.15.32:6816/118188 conn(0x7f929802bb60 legacy=0x7f929802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4096380 msgr2=0x7f92b41140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4096380 0x7f92b41140d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f92ac009f10 tx=0x7f92ac00b040).stop 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 shutdown_connections 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92b4096380 0x7f92b41140d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b4091b00 0x7f92b4123e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 --2- 172.21.15.32:0/3283141109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92b4094c20 0x7f92b4124360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 >> 172.21.15.32:0/3283141109 conn(0x7f92b4097470 msgr2=0x7f92b4126980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 shutdown_connections 2023-09-30T22:21:59.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.171+0000 7f92baabc700 1 -- 172.21.15.32:0/3283141109 wait complete. 2023-09-30T22:21:59.172 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-09-30T22:21:59.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:21:37.188029+0000', 'modified': '2023-09-30T22:21:58.784702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4177': {'gid': 4177, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6817/3095020018', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 3095020018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-30T22:21:59.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:21:59.285 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:21:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.422+0000 7f261cb7c700 1 Processor -- start 2023-09-30T22:21:59.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.422+0000 7f261cb7c700 1 -- start start 2023-09-30T22:21:59.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261cb7c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2618059070 0x7f2618127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261cb7c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f2618127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261cb7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 0x7f26180957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261cb7c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2618096940 con 0x7f2618059070 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261cb7c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2618059c00 con 0x7f2618127390 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261cb7c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2618091b60 con 0x7f2618095370 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2618059070 0x7f2618127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f2618127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 0x7f26180957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2618059070 0x7f2618127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42930/0 (socket says 172.21.15.32:42930) 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f2618127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45004/0 (socket says 172.21.15.32:45004) 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 0x7f26180957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33844/0 (socket says 172.21.15.32:33844) 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 -- 172.21.15.32:0/1562406924 learned_addr learned my addr 172.21.15.32:0/1562406924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 -- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 msgr2=0x7f26180957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 0x7f26180957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 -- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 msgr2=0x7f2618127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f2618127d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 -- 172.21.15.32:0/1562406924 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26180929d0 con 0x7f2618059070 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f2616d9d700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 0x7f26180957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f2615d9b700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f2618127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.423+0000 7f261659c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618059070 0x7f2618127890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f2608009830 tx=0x7f26080090f0).ready entity=mon.0 client_cookie=5d23ce233ed726e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261559a700 1 -- 172.21.15.32:0/1562406924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800f510 con 0x7f2618059070 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261559a700 1 -- 172.21.15.32:0/1562406924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260800f670 con 0x7f2618059070 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261559a700 1 -- 172.21.15.32:0/1562406924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800fa10 con 0x7f2618059070 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 -- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618059070 msgr2=0x7f2618127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618059070 0x7f2618127890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f2608009830 tx=0x7f26080090f0).stop 2023-09-30T22:21:59.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 -- 172.21.15.32:0/1562406924 shutdown_connections 2023-09-30T22:21:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618095370 0x7f26180957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618059070 0x7f2618127890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 --2- 172.21.15.32:0/1562406924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f2618127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 -- 172.21.15.32:0/1562406924 >> 172.21.15.32:0/1562406924 conn(0x7f2618097470 msgr2=0x7f2618097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 -- 172.21.15.32:0/1562406924 shutdown_connections 2023-09-30T22:21:59.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.424+0000 7f261cb7c700 1 -- 172.21.15.32:0/1562406924 wait complete. 2023-09-30T22:21:59.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.425+0000 7f261cb7c700 1 Processor -- start 2023-09-30T22:21:59.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.425+0000 7f261cb7c700 1 -- start start 2023-09-30T22:21:59.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261cb7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 0x7f2618113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261cb7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 0x7f26181142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261cb7c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f26181147c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261cb7c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618117330 con 0x7f2618095370 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261cb7c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2618096f20 con 0x7f2618127390 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261cb7c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f261809a180 con 0x7f2618059070 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 0x7f2618113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 0x7f26181142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 0x7f2618113de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33860/0 (socket says 172.21.15.32:33860) 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 -- 172.21.15.32:0/4122166884 learned_addr learned my addr 172.21.15.32:0/4122166884 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 0x7f26181142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42940/0 (socket says 172.21.15.32:42940) 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f2616d9d700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f26181147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 -- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 msgr2=0x7f26181147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f26181147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 -- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 msgr2=0x7f26181142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 0x7f26181142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 -- 172.21.15.32:0/4122166884 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610009a20 con 0x7f2618059070 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f2615d9b700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 0x7f26181142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.426+0000 7f261659c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 0x7f2618113de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f2608009e20 tx=0x7f2608015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.427+0000 7f260e7fc700 1 -- 172.21.15.32:0/4122166884 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608006910 con 0x7f2618059070 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.427+0000 7f260e7fc700 1 -- 172.21.15.32:0/4122166884 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608000bc0 con 0x7f2618059070 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.427+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26000092a0 con 0x7f2618059070 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.427+0000 7f260e7fc700 1 -- 172.21.15.32:0/4122166884 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608024470 con 0x7f2618059070 2023-09-30T22:21:59.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.427+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2608009370 con 0x7f2618059070 2023-09-30T22:21:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.427+0000 7f260e7fc700 1 -- 172.21.15.32:0/4122166884 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2608021070 con 0x7f2618059070 2023-09-30T22:21:59.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.428+0000 7f260e7fc700 1 -- 172.21.15.32:0/4122166884 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f260803c820 con 0x7f2618059070 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 Processor -- start 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 -- start start 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x5609437500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x5609437529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609436f0610 con 0x5609437505b0 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x560943666db0 con 0x560943726170 2023-09-30T22:21:59.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x56094367c030 con 0x560943726a40 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x5609437500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x5609437529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x5609437529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42946/0 (socket says 172.21.15.32:42946) 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f2616d9d700 1 -- 172.21.15.32:0/1071998807 learned_addr learned my addr 172.21.15.32:0/1071998807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.431+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x5609437500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33876/0 (socket says 172.21.15.32:33876) 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f261659c700 1 -- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 msgr2=0x5609437500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f261659c700 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x5609437500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f261659c700 1 -- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 msgr2=0x5609437529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f261659c700 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x5609437529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f261659c700 1 -- 172.21.15.32:0/1071998807 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560943517710 con 0x560943726170 2023-09-30T22:21:59.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f2616d9d700 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x5609437529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f261659c700 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f2608010b50 tx=0x7f260801d070).ready entity=mon.1 client_cookie=f9155927eccf0f32 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f25f77fe700 1 -- 172.21.15.32:0/1071998807 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260801a7f0 con 0x560943726170 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f25f77fe700 1 -- 172.21.15.32:0/1071998807 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260801b410 con 0x560943726170 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.432+0000 7f25f77fe700 1 -- 172.21.15.32:0/1071998807 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260801b580 con 0x560943726170 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 -- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 msgr2=0x560943726550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f2608010b50 tx=0x7f260801d070).stop 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 -- 172.21.15.32:0/1071998807 shutdown_connections 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x5609437529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 --2- 172.21.15.32:0/1071998807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x5609437500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 -- 172.21.15.32:0/1071998807 >> 172.21.15.32:0/1071998807 conn(0x560943594940 msgr2=0x560943746cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 -- 172.21.15.32:0/1071998807 shutdown_connections 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 -- 172.21.15.32:0/1071998807 wait complete. 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 -- start start 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.433+0000 7f262e52fb80 1 Processor -- start 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x56094374bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x56094374c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x56094374c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094367c170 con 0x5609437505b0 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x56094367c3f0 con 0x560943726170 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x560943647920 con 0x560943726a40 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f262e52fb80 10 client.0 ll_register_callbacks cb 0x56094361cce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x56094374bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x56094374c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x56094374bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45032/0 (socket says 172.21.15.32:45032) 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f261659c700 1 -- 172.21.15.32:0/3102503125 learned_addr learned my addr 172.21.15.32:0/3102503125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.434+0000 7f2615d9b700 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x56094374c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f261659c700 1 -- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 msgr2=0x56094374c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f261659c700 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x56094374c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f261659c700 1 -- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 msgr2=0x56094374c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f261659c700 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x56094374c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f261659c700 1 -- 172.21.15.32:0/3102503125 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2608009370 con 0x560943726170 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f261659c700 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x56094374bca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f260803b8c0 tx=0x7f2608010890).ready entity=mon.1 client_cookie=ce396cc57f45e59d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f25f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3301/0 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f25f4ff9700 1 -- 172.21.15.32:0/3102503125 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608006d90 con 0x560943726170 2023-09-30T22:21:59.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f25f4ff9700 1 -- 172.21.15.32:0/3102503125 <== mon.1 v2:172.21.15.32:3301/0 2 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f260801b580 con 0x560943726170 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56094371ef20 con 0x560943726170 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f25f4ff9700 1 -- 172.21.15.32:0/3102503125 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f260801a600 con 0x560943726170 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 10 client.4528 fetch_fsmap learned FSMap version 8 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56094364eb00 con 0x560943726170 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f25f4ff9700 1 -- 172.21.15.32:0/3102503125 <== mon.1 v2:172.21.15.32:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56094371ef20 con 0x560943726170 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f25f4ff9700 1 -- 172.21.15.32:0/3102503125 <== mon.1 v2:172.21.15.32:3301/0 5 ==== fsmap(e 8) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f260804c070 con 0x560943726170 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 10 client.4528 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 10 client.4528 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 20 client.4528 populate_metadata read hostname 'smithi032' 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 4 client.4528 mds_command: new command op to 4177 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:21:59.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.435+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 --> v1:172.21.15.32:6817/3095020018 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609436f4250 con 0x560943740540 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.436+0000 7f25f4ff9700 10 client.4528 ms_handle_connect on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.436+0000 7f25f4ff9700 1 -- 172.21.15.32:0/3102503125 <== mds.0 v1:172.21.15.32:6817/3095020018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x5609436f4250 con 0x560943740540 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.436+0000 7f25f4ff9700 10 client.4528 handle_command_reply: tid=0 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.436+0000 7f262e52fb80 1 client.4528 shutdown 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 msgr2=0x56094374bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x56094374bca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f260803b8c0 tx=0x7f2608010890).stop 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 shutdown_connections 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5609437505b0 0x56094374c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f25f4ff9700 0 client.4528 ms_handle_reset on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x56094374bca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 --2- 172.21.15.32:0/3102503125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726a40 0x56094374c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 >> 172.21.15.32:0/3102503125 conn(0x560943594940 msgr2=0x560943746c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 shutdown_connections 2023-09-30T22:21:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 1 -- 172.21.15.32:0/3102503125 wait complete. 2023-09-30T22:21:59.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.437+0000 7f262e52fb80 20 client.4528 trim_cache size 0 max 16384 2023-09-30T22:21:59.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.454+0000 7f262e52fb80 1 Processor -- start 2023-09-30T22:21:59.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.454+0000 7f262e52fb80 1 -- start start 2023-09-30T22:21:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.454+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x56094372a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x5609437505b0 0x560943743380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560943728190 con 0x560943726a40 2023-09-30T22:21:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609436f4250 con 0x560943726170 2023-09-30T22:21:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x5609436f0480 con 0x5609437505b0 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x5609437505b0 0x560943743380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x56094372a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45046/0 (socket says 172.21.15.32:45046) 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 -- 172.21.15.32:0/187873031 learned_addr learned my addr 172.21.15.32:0/187873031 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 -- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x5609437505b0 msgr2=0x560943743380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x5609437505b0 0x560943743380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 -- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 msgr2=0x56094372a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x56094372a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 -- 172.21.15.32:0/187873031 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560943674f70 con 0x560943726170 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f2615d9b700 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x56094372a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:21:59.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f261659c700 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f260800ffd0 tx=0x7f260803e040).ready entity=mon.1 client_cookie=63680a2a79485dff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f25f7fff700 1 -- 172.21.15.32:0/187873031 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260801a600 con 0x560943726170 2023-09-30T22:21:59.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f25f7fff700 1 -- 172.21.15.32:0/187873031 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260801b580 con 0x560943726170 2023-09-30T22:21:59.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.455+0000 7f25f7fff700 1 -- 172.21.15.32:0/187873031 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608006d90 con 0x560943726170 2023-09-30T22:21:59.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 -- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 msgr2=0x560943726550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f260800ffd0 tx=0x7f260803e040).stop 2023-09-30T22:21:59.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 -- 172.21.15.32:0/187873031 shutdown_connections 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x5609437505b0 0x560943743380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x560943726170 0x560943726550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 --2- 172.21.15.32:0/187873031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x56094372a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 -- 172.21.15.32:0/187873031 >> 172.21.15.32:0/187873031 conn(0x560943594940 msgr2=0x560943725980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 -- 172.21.15.32:0/187873031 shutdown_connections 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 -- 172.21.15.32:0/187873031 wait complete. 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 -- start start 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.456+0000 7f262e52fb80 1 Processor -- start 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 0x5609437a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x5609437aa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x5609437505b0 0x5609437aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:21:59.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x560943766060 con 0x560943726a40 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609437aadf0 con 0x5609437505b0 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x56094374b680 con 0x560943726170 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f262e52fb80 10 client.0 ll_register_callbacks cb 0x7f2618061790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 0x5609437a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f261659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 0x5609437a9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33882/0 (socket says 172.21.15.32:33882) 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f261659c700 1 -- 172.21.15.32:0/639184077 learned_addr learned my addr 172.21.15.32:0/639184077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f2616d9d700 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x5609437505b0 0x5609437aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.457+0000 7f2615d9b700 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x5609437aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f261659c700 1 -- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x5609437505b0 msgr2=0x5609437aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f261659c700 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x5609437505b0 0x5609437aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f261659c700 1 -- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 msgr2=0x5609437aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:21:59.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f261659c700 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x5609437aa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f261659c700 1 -- 172.21.15.32:0/639184077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2600013960 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f261659c700 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 0x5609437a9f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f260802d650 tx=0x7f2608027000).ready entity=mon.2 client_cookie=8552745d3b638e2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f25f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3302/0 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f25f4ff9700 1 -- 172.21.15.32:0/639184077 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260801a600 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f25f4ff9700 1 -- 172.21.15.32:0/639184077 <== mon.2 v2:172.21.15.32:3302/0 2 ==== osd_map(18..18 src has 1..18) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f26080245f0 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f260801b410 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f25f4ff9700 1 -- 172.21.15.32:0/639184077 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2608010580 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 10 client.4541 fetch_fsmap learned FSMap version 8 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56094364eb00 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f25f4ff9700 1 -- 172.21.15.32:0/639184077 <== mon.2 v2:172.21.15.32:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260801b410 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f25f4ff9700 1 -- 172.21.15.32:0/639184077 <== mon.2 v2:172.21.15.32:3302/0 5 ==== fsmap(e 8) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f260802dcc0 con 0x560943726170 2023-09-30T22:21:59.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 10 client.4541 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 10 client.4541 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 20 client.4541 populate_metadata read hostname 'smithi032' 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 4 client.4541 mds_command: new command op to 4177 tid=0[{"prefix": "status"}] 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.458+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 --> v1:172.21.15.32:6817/3095020018 -- command(tid 0: {"prefix": "status"}) v1 -- 0x5609434e4ef0 con 0x56094374bef0 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f25f4ff9700 10 client.4541 ms_handle_connect on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f25f4ff9700 1 -- 172.21.15.32:0/639184077 <== mds.0 v1:172.21.15.32:6817/3095020018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 4159600709) 0x5609434e4ef0 con 0x56094374bef0 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f25f4ff9700 10 client.4541 handle_command_reply: tid=0 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 client.4541 shutdown 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 msgr2=0x5609437a9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 0x5609437a9f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f260802d650 tx=0x7f2608027000).stop 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 shutdown_connections 2023-09-30T22:21:59.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x5609437505b0 0x5609437aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x560943726170 0x5609437a9f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 --2- 172.21.15.32:0/639184077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x560943726a40 0x5609437aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f25f4ff9700 0 client.4541 ms_handle_reset on v1:172.21.15.32:6817/3095020018 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 >> 172.21.15.32:0/639184077 conn(0x560943594940 msgr2=0x560943743380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 shutdown_connections 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.459+0000 7f262e52fb80 1 -- 172.21.15.32:0/639184077 wait complete. 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f262e52fb80 20 client.4541 trim_cache size 0 max 16384 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T22:21:59.465 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9e20f926-ec7d-45ec-8e27-e5e820815322", 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4177, 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 22.270120386999999, 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 8, 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 18, 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 18, 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 27.553454447 2023-09-30T22:21:59.466 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T22:21:59.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 >> v1:172.21.15.32:6816/118188 conn(0x7f25f802bb90 legacy=0x7f25f802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 msgr2=0x7f2618113de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:21:59.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 0x7f2618113de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f2608009e20 tx=0x7f2608015000).stop 2023-09-30T22:21:59.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 shutdown_connections 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2618127390 0x7f26181147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2618059070 0x7f2618113de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 --2- 172.21.15.32:0/4122166884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2618095370 0x7f26181142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 >> 172.21.15.32:0/4122166884 conn(0x7f2618097470 msgr2=0x7f2618098780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 shutdown_connections 2023-09-30T22:21:59.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:21:59.460+0000 7f261cb7c700 1 -- 172.21.15.32:0/4122166884 wait complete. 2023-09-30T22:21:59.473 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:21:59.482 INFO:teuthology.task.print:**** done mds pre-upgrade sequence 2023-09-30T22:21:59.482 INFO:teuthology.run_tasks:Running task install.upgrade... 2023-09-30T22:21:59.506 INFO:teuthology.task.install:project ceph config {'mon.a': {'branch': 'reef'}} overrides {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-09-30T22:21:59.506 INFO:teuthology.task.install:config contains sha1|tag|branch, removing those keys from override 2023-09-30T22:21:59.506 INFO:teuthology.task.install:remote ubuntu@smithi032.front.sepia.ceph.com config {'branch': 'reef', 'flavor': 'default'} 2023-09-30T22:21:59.506 INFO:teuthology.task.install:extra packages: [] 2023-09-30T22:21:59.512 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T22:21:59.512 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2023-09-30T22:21:59.512 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:21:59.538 INFO:teuthology.orchestra.run.smithi032.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:21:59.539 INFO:teuthology.packaging:The installed version of ceph-common is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:21:59.539 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:21:59.827 INFO:teuthology.task.install:Ceph rpm upgrade from 16.2.14-44.geeb13ac9.el8 to 18.2.0-501.gfc1e90a3 2023-09-30T22:21:59.828 INFO:teuthology.task.install.rpm:Host smithi032 is: centos 8 x86_64 2023-09-30T22:21:59.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:22:00.113 INFO:teuthology.task.install.rpm:Repo base URL: https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/ 2023-09-30T22:22:00.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/yum.repos.d/ceph.repo 2023-09-30T22:22:00.334 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:22:00.335 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-09-30T22:22:00.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:22:00.370 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:22:00.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:22:00.526 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:22:00.589 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2023-09-30T22:22:00.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2023-09-30T22:22:00.914 INFO:teuthology.orchestra.run.smithi032.stdout:71 files removed 2023-09-30T22:22:00.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2023-09-30T22:22:01.802 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 216 kB/s | 80 kB 00:00 2023-09-30T22:22:02.135 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-09-30T22:22:02.519 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2023-09-30T22:22:05.344 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - AppStream 12 MB/s | 33 MB 00:02 2023-09-30T22:22:17.542 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - BaseOS 8.7 MB/s | 48 MB 00:05 2023-09-30T22:22:25.355 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras 103 kB/s | 18 kB 00:00 2023-09-30T22:22:26.264 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras common packages 9.8 kB/s | 6.9 kB 00:00 2023-09-30T22:22:26.882 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - PowerTools 15 MB/s | 6.5 MB 00:00 2023-09-30T22:22:28.417 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-09-30T22:22:29.935 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-09-30T22:22:34.285 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 395 kB/s | 24 kB 00:00 2023-09-30T22:22:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:Package cephadm available, but not installed. 2023-09-30T22:22:38.942 INFO:teuthology.orchestra.run.smithi032.stdout:No match for argument: cephadm 2023-09-30T22:22:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:Package ceph-immutable-object-cache available, but not installed. 2023-09-30T22:22:38.943 INFO:teuthology.orchestra.run.smithi032.stdout:No match for argument: ceph-immutable-object-cache 2023-09-30T22:22:38.957 INFO:teuthology.orchestra.run.smithi032.stderr:Package ceph-mgr-cephadm available, but not installed. 2023-09-30T22:22:38.957 INFO:teuthology.orchestra.run.smithi032.stdout:No match for argument: ceph-mgr-cephadm 2023-09-30T22:22:38.961 INFO:teuthology.orchestra.run.smithi032.stderr:Package ceph-volume available, but not installed. 2023-09-30T22:22:38.961 INFO:teuthology.orchestra.run.smithi032.stdout:No match for argument: ceph-volume 2023-09-30T22:22:39.269 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout:====================================================================================== 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout:====================================================================================== 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 6.4 k 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 5.4 M 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 22 M 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 847 k 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 23 k 2023-09-30T22:22:39.275 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 2.3 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 1.5 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 1.7 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 7.4 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 241 k 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 47 k 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 4.6 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 18 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 15 k 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 13 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 24 k 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 47 M 2023-09-30T22:22:39.276 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 30 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 769 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 137 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 139 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.7 M 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 474 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.6 M 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 6.5 M 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 45 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 123 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 190 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 371 k 2023-09-30T22:22:39.277 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 364 k 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 107 k 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 94 k 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.4 M 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 179 k 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout:====================================================================================== 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 34 Packages 2023-09-30T22:22:39.278 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:22:39.279 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 145 M 2023-09-30T22:22:39.279 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2023-09-30T22:22:39.612 INFO:teuthology.orchestra.run.smithi032.stdout:(1/35): ceph-18.2.0-501.gfc1e90a3.el8.x86_64.rp 35 kB/s | 6.4 kB 00:00 2023-09-30T22:22:39.945 INFO:teuthology.orchestra.run.smithi032.stdout:(2/35): ceph-base-18.2.0-501.gfc1e90a3.el8.x86_ 10 MB/s | 5.4 MB 00:00 2023-09-30T22:22:40.021 INFO:teuthology.orchestra.run.smithi032.stdout:(3/35): ceph-fuse-18.2.0-501.gfc1e90a3.el8.x86_ 11 MB/s | 847 kB 00:00 2023-09-30T22:22:40.054 INFO:teuthology.orchestra.run.smithi032.stdout:(4/35): thrift-0.13.0-2.el8.x86_64.rpm 2.8 MB/s | 1.7 MB 00:00 2023-09-30T22:22:40.162 INFO:teuthology.orchestra.run.smithi032.stdout:(5/35): ceph-mds-18.2.0-501.gfc1e90a3.el8.x86_6 16 MB/s | 2.3 MB 00:00 2023-09-30T22:22:40.413 INFO:teuthology.orchestra.run.smithi032.stdout:(6/35): ceph-mon-18.2.0-501.gfc1e90a3.el8.x86_6 19 MB/s | 4.6 MB 00:00 2023-09-30T22:22:41.305 INFO:teuthology.orchestra.run.smithi032.stdout:(7/35): ceph-osd-18.2.0-501.gfc1e90a3.el8.x86_6 20 MB/s | 18 MB 00:00 2023-09-30T22:22:41.347 INFO:teuthology.orchestra.run.smithi032.stdout:(8/35): ceph-mgr-18.2.0-501.gfc1e90a3.el8.x86_6 1.2 MB/s | 1.5 MB 00:01 2023-09-30T22:22:41.422 INFO:teuthology.orchestra.run.smithi032.stdout:(9/35): ceph-selinux-18.2.0-501.gfc1e90a3.el8.x 325 kB/s | 24 kB 00:00 2023-09-30T22:22:42.031 INFO:teuthology.orchestra.run.smithi032.stdout:(10/35): ceph-radosgw-18.2.0-501.gfc1e90a3.el8. 18 MB/s | 13 MB 00:00 2023-09-30T22:22:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:(11/35): libcephfs-devel-18.2.0-501.gfc1e90a3.e 614 kB/s | 30 kB 00:00 2023-09-30T22:22:42.173 INFO:teuthology.orchestra.run.smithi032.stdout:(12/35): libcephfs2-18.2.0-501.gfc1e90a3.el8.x8 8.2 MB/s | 769 kB 00:00 2023-09-30T22:22:42.273 INFO:teuthology.orchestra.run.smithi032.stdout:(13/35): libcephsqlite-18.2.0-501.gfc1e90a3.el8 1.3 MB/s | 137 kB 00:00 2023-09-30T22:22:42.323 INFO:teuthology.orchestra.run.smithi032.stdout:(14/35): librados-devel-18.2.0-501.gfc1e90a3.el 2.7 MB/s | 139 kB 00:00 2023-09-30T22:22:42.621 INFO:teuthology.orchestra.run.smithi032.stdout:(15/35): librados2-18.2.0-501.gfc1e90a3.el8.x86 12 MB/s | 3.7 MB 00:00 2023-09-30T22:22:42.699 INFO:teuthology.orchestra.run.smithi032.stdout:(16/35): libradosstriper1-18.2.0-501.gfc1e90a3. 6.0 MB/s | 474 kB 00:00 2023-09-30T22:22:43.017 INFO:teuthology.orchestra.run.smithi032.stdout:(17/35): librbd1-18.2.0-501.gfc1e90a3.el8.x86_6 11 MB/s | 3.6 MB 00:00 2023-09-30T22:22:43.333 INFO:teuthology.orchestra.run.smithi032.stdout:(18/35): ceph-common-18.2.0-501.gfc1e90a3.el8.x 5.8 MB/s | 22 MB 00:03 2023-09-30T22:22:43.417 INFO:teuthology.orchestra.run.smithi032.stdout:(19/35): python3-ceph-argparse-18.2.0-501.gfc1e 537 kB/s | 45 kB 00:00 2023-09-30T22:22:43.500 INFO:teuthology.orchestra.run.smithi032.stdout:(20/35): librgw2-18.2.0-501.gfc1e90a3.el8.x86_6 14 MB/s | 6.5 MB 00:00 2023-09-30T22:22:43.526 INFO:teuthology.orchestra.run.smithi032.stdout:(21/35): python3-ceph-common-18.2.0-501.gfc1e90 1.1 MB/s | 123 kB 00:00 2023-09-30T22:22:43.551 INFO:teuthology.orchestra.run.smithi032.stdout:(22/35): python3-cephfs-18.2.0-501.gfc1e90a3.el 3.7 MB/s | 190 kB 00:00 2023-09-30T22:22:43.609 INFO:teuthology.orchestra.run.smithi032.stdout:(23/35): python3-rbd-18.2.0-501.gfc1e90a3.el8.x 6.1 MB/s | 364 kB 00:00 2023-09-30T22:22:43.635 INFO:teuthology.orchestra.run.smithi032.stdout:(24/35): python3-rados-18.2.0-501.gfc1e90a3.el8 3.3 MB/s | 371 kB 00:00 2023-09-30T22:22:43.660 INFO:teuthology.orchestra.run.smithi032.stdout:(25/35): python3-rgw-18.2.0-501.gfc1e90a3.el8.x 2.1 MB/s | 107 kB 00:00 2023-09-30T22:22:43.686 INFO:teuthology.orchestra.run.smithi032.stdout:(26/35): rbd-fuse-18.2.0-501.gfc1e90a3.el8.x86_ 1.8 MB/s | 94 kB 00:00 2023-09-30T22:22:43.803 INFO:teuthology.orchestra.run.smithi032.stdout:(27/35): rbd-nbd-18.2.0-501.gfc1e90a3.el8.x86_6 1.5 MB/s | 179 kB 00:00 2023-09-30T22:22:43.870 INFO:teuthology.orchestra.run.smithi032.stdout:(28/35): rbd-mirror-18.2.0-501.gfc1e90a3.el8.x8 17 MB/s | 3.4 MB 00:00 2023-09-30T22:22:43.895 INFO:teuthology.orchestra.run.smithi032.stdout:(29/35): ceph-grafana-dashboards-18.2.0-501.gfc 251 kB/s | 23 kB 00:00 2023-09-30T22:22:44.012 INFO:teuthology.orchestra.run.smithi032.stdout:(30/35): ceph-mgr-dashboard-18.2.0-501.gfc1e90a 12 MB/s | 1.7 MB 00:00 2023-09-30T22:22:44.079 INFO:teuthology.orchestra.run.smithi032.stdout:(31/35): ceph-mgr-modules-core-18.2.0-501.gfc1e 3.5 MB/s | 241 kB 00:00 2023-09-30T22:22:44.121 INFO:teuthology.orchestra.run.smithi032.stdout:(32/35): ceph-mgr-rook-18.2.0-501.gfc1e90a3.el8 1.1 MB/s | 47 kB 00:00 2023-09-30T22:22:44.188 INFO:teuthology.orchestra.run.smithi032.stdout:(33/35): ceph-prometheus-alerts-18.2.0-501.gfc1 218 kB/s | 15 kB 00:00 2023-09-30T22:22:44.557 INFO:teuthology.orchestra.run.smithi032.stdout:(34/35): ceph-mgr-diskprediction-local-18.2.0-5 11 MB/s | 7.4 MB 00:00 2023-09-30T22:22:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:(35/35): ceph-test-18.2.0-501.gfc1e90a3.el8.x86 6.1 MB/s | 47 MB 00:07 2023-09-30T22:22:49.091 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:22:49.091 INFO:teuthology.orchestra.run.smithi032.stdout:Total 15 MB/s | 145 MB 00:09 2023-09-30T22:22:49.092 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2023-09-30T22:22:49.172 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2023-09-30T22:22:49.172 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2023-09-30T22:22:49.904 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2023-09-30T22:22:49.907 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2023-09-30T22:22:52.302 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2023-09-30T22:22:52.506 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-09-30T22:22:53.095 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/69 2023-09-30T22:22:53.322 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:22:53.791 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:22:53.981 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 3/69 2023-09-30T22:22:54.116 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 3/69 2023-09-30T22:22:54.339 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:22:55.202 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:22:55.422 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 5/69 2023-09-30T22:22:55.511 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 5/69 2023-09-30T22:22:56.011 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 6/69 2023-09-30T22:22:56.222 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 7/69 2023-09-30T22:22:56.321 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 7/69 2023-09-30T22:22:56.556 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:22:56.632 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:22:56.803 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x 9/69 2023-09-30T22:22:57.071 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 10/69 2023-09-30T22:22:57.294 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 11/69 2023-09-30T22:22:57.555 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 12/69 2023-09-30T22:22:57.766 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 13/69 2023-09-30T22:22:57.877 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8. 14/69 2023-09-30T22:22:58.052 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8 15/69 2023-09-30T22:22:58.322 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86 16/69 2023-09-30T22:23:00.847 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:23:01.106 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:23:01.790 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:23:02.110 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:23:02.181 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:23:02.346 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 19/69 2023-09-30T22:23:22.361 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 19/69 2023-09-30T22:23:22.607 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 20/69 2023-09-30T22:23:23.903 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 20/69 2023-09-30T22:23:24.110 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 21/69 2023-09-30T22:23:24.217 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 21/69 2023-09-30T22:23:24.620 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.n 22/69 2023-09-30T22:23:24.883 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 23/69 2023-09-30T22:23:24.911 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 23/69 2023-09-30T22:23:25.085 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 24/69 2023-09-30T22:23:25.313 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 24/69 2023-09-30T22:23:25.552 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 25/69 2023-09-30T22:23:26.194 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 25/69 2023-09-30T22:23:26.385 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:23:28.391 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:23:28.624 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 27/69 2023-09-30T22:23:28.638 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 27/69 2023-09-30T22:23:30.255 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 28/69 2023-09-30T22:23:30.473 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 29/69 2023-09-30T22:23:30.915 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 29/69 2023-09-30T22:23:31.159 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:23:36.652 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:23:36.948 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 31/69 2023-09-30T22:23:37.102 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 32/69 2023-09-30T22:23:37.344 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 33/69 2023-09-30T22:23:37.642 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 34/69 2023-09-30T22:23:37.869 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 35/69 2023-09-30T22:23:37.980 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 36/69 2023-09-30T22:23:38.210 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-09-30T22:23:38.219 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:23:38.220 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:23:38.524 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:23:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-09-30T22:23:38.652 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-09-30T22:23:38.819 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-09-30T22:23:38.826 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:23:38.827 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:23:39.046 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:23:39.240 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 42/69 2023-09-30T22:23:39.445 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 43/69 2023-09-30T22:23:39.450 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:23:39.451 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:23:39.685 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:23:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-09-30T22:23:39.853 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-09-30T22:23:40.085 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-09-30T22:23:40.251 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-09-30T22:23:40.260 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:23:40.261 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:23:40.672 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:23:40.672 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-09-30T22:23:40.941 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-09-30T22:23:40.948 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:23:40.949 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:23:41.119 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:23:41.126 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:23:41.127 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:23:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:23:41.335 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-09-30T22:23:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-09-30T22:23:41.508 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:23:41.509 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:23:41.828 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:23:41.828 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-09-30T22:23:42.077 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:23:42.277 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:23:42.484 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 56/69 2023-09-30T22:23:42.676 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-09-30T22:23:42.677 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-09-30T22:23:42.918 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-09-30T22:23:43.134 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 59/69 2023-09-30T22:23:43.134 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-09-30T22:23:43.351 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-09-30T22:23:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 61/69 2023-09-30T22:23:43.575 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-09-30T22:23:43.794 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-09-30T22:23:43.992 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 63/69 2023-09-30T22:23:44.102 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 64/69 2023-09-30T22:23:44.317 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 65/69 2023-09-30T22:23:44.425 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 66/69 2023-09-30T22:23:44.528 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 67/69 2023-09-30T22:23:44.528 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-09-30T22:23:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-09-30T22:23:44.702 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-09-30T22:23:45.467 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-09-30T22:23:45.467 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 1/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 3/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 5/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 6/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 7/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 9/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 10/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 11/69 2023-09-30T22:23:45.468 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 12/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 13/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 14/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 15/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 16/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 17/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 19/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 20/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 21/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 22/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 23/69 2023-09-30T22:23:45.469 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 24/69 2023-09-30T22:23:45.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 25/69 2023-09-30T22:23:45.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:23:45.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 27/69 2023-09-30T22:23:45.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 28/69 2023-09-30T22:23:45.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 29/69 2023-09-30T22:23:45.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:23:45.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 31/69 2023-09-30T22:23:45.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 32/69 2023-09-30T22:23:45.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 33/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 34/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 35/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 36/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 38/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x 40/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 41/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86 42/69 2023-09-30T22:23:45.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 43/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 44/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 46/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 48/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 49/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 50/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 52/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 54/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:23:45.475 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 56/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8 58/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 59/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 60/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 61/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 62/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 63/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.n 64/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 65/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 66/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 67/69 2023-09-30T22:23:45.476 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8. 68/69 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 69/69 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.315 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.316 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.317 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:47.318 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2023-09-30T22:23:47.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:23:47.785 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:23:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:18.2.0-501.gfc1e90a3.el8 2023-09-30T22:23:47.816 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-501.gfc1e90a3.el8 2023-09-30T22:23:47.816 INFO:teuthology.task.install:The correct ceph version 18.2.0-501.gfc1e90a3 is installed. 2023-09-30T22:23:47.816 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:23:47.829 INFO:teuthology.task.print:**** done install.upgrade the host 2023-09-30T22:23:47.829 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:23:47.851 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T22:23:47.851 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:23:47.869 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T22:23:47.869 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T22:23:47.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T22:23:47.912 INFO:tasks.ceph.mon.a:Started 2023-09-30T22:23:47.912 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T22:23:47.912 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:23:47.926 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T22:23:47.926 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T22:23:47.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T22:23:47.970 INFO:tasks.ceph.mon.b:Started 2023-09-30T22:23:47.970 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T22:23:47.970 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:23:47.985 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T22:23:47.985 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T22:23:47.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T22:23:48.029 INFO:tasks.ceph.mon.c:Started 2023-09-30T22:23:48.030 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T22:23:48.030 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:23:48.061 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T22:23:48.061 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T22:23:48.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T22:23:48.104 INFO:tasks.ceph.mgr.x:Started 2023-09-30T22:23:48.104 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-09-30T22:23:48.114 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:23:48.114 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:23:48.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.257+0000 7f54d5fa0700 1 Processor -- start 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 -- start start 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d009b6e0 con 0x7f54d009d8d0 2023-09-30T22:23:48.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d009b470 con 0x7f54d009e1e0 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d5fa0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54d009eb80 con 0x7f54d009e610 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54cffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60940/0 (socket says 172.21.15.32:60940) 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 -- 172.21.15.32:0/3304312507 learned_addr learned my addr 172.21.15.32:0/3304312507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d579f700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 -- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 msgr2=0x7f54d009dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54cffff700 1 -- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 msgr2=0x7f54d0130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:23:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 -- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 msgr2=0x7f54d009dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54cffff700 1 -- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 msgr2=0x7f54d0130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 --2- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54cffff700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d579f700 1 -- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 msgr2=0x7f54d01309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d579f700 1 -- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 msgr2=0x7f54d01309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d579f700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d4f9e700 1 --2- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54cffff700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:23:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.258+0000 7f54d579f700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:23:48.422 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:48.422+0000 7f4e8b1c9240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T22:23:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54d4f9e700 1 --2- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54d579f700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 -- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 msgr2=0x7f54d01309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 -- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 msgr2=0x7f54d009dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 --2- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 -- 172.21.15.32:0/3304312507 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d009ae00 con 0x7f54d009e1e0 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54d4f9e700 1 --2- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54d579f700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:48.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.459+0000 7f54cffff700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54b8000ed0 tx=0x7f54b800cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6381e1743bb992 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.460+0000 7f54cf7fe700 1 -- 172.21.15.32:0/3304312507 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8005c20 con 0x7f54d009e1e0 2023-09-30T22:23:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.460+0000 7f54cf7fe700 1 -- 172.21.15.32:0/3304312507 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8005d80 con 0x7f54d009e1e0 2023-09-30T22:23:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.460+0000 7f54cf7fe700 1 -- 172.21.15.32:0/3304312507 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8006290 con 0x7f54d009e1e0 2023-09-30T22:23:48.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 -- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 msgr2=0x7f54d0130460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54b8000ed0 tx=0x7f54b800cbb0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 -- 172.21.15.32:0/3304312507 shutdown_connections 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e610 0x7f54d01309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/3304312507 >> v2:172.21.15.32:3300/0 conn(0x7f54d009d8d0 0x7f54d009dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/3304312507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e1e0 0x7f54d0130460 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 -- 172.21.15.32:0/3304312507 >> 172.21.15.32:0/3304312507 conn(0x7f54d00a2e20 msgr2=0x7f54d00a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 -- 172.21.15.32:0/3304312507 shutdown_connections 2023-09-30T22:23:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.461+0000 7f54d5fa0700 1 -- 172.21.15.32:0/3304312507 wait complete. 2023-09-30T22:23:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.462+0000 7f54d5fa0700 1 Processor -- start 2023-09-30T22:23:48.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.463+0000 7f54d5fa0700 1 -- start start 2023-09-30T22:23:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.463+0000 7f54d5fa0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d009d8d0 0x7f54d0132c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d5fa0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e1e0 0x7f54d01331d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d5fa0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 0x7f54d0133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d5fa0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0134810 con 0x7f54d009d8d0 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d5fa0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0135af0 con 0x7f54d009e610 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d5fa0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54d00a1b50 con 0x7f54d009e1e0 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 0x7f54d0133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d4f9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d009d8d0 0x7f54d0132c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 0x7f54d0133710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59124/0 (socket says 172.21.15.32:59124) 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d4f9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d009d8d0 0x7f54d0132c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60970/0 (socket says 172.21.15.32:60970) 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 -- 172.21.15.32:0/2090795114 learned_addr learned my addr 172.21.15.32:0/2090795114 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54cffff700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e1e0 0x7f54d01331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 -- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e1e0 msgr2=0x7f54d01331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e1e0 0x7f54d01331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 -- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d009d8d0 msgr2=0x7f54d0132c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d009d8d0 0x7f54d0132c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 -- 172.21.15.32:0/2090795114 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00095b0 con 0x7f54d009e610 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54cffff700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e1e0 0x7f54d01331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d579f700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 0x7f54d0133710 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f54c8010550 tx=0x7f54c8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c800b2e0 con 0x7f54d009e610 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c800b440 con 0x7f54d009e610 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.464+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c8009ad0 con 0x7f54d009e610 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.465+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c800b7f0 con 0x7f54d009e610 2023-09-30T22:23:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.465+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b800c4e0 con 0x7f54d009e610 2023-09-30T22:23:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.465+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f54c800b950 con 0x7f54d009e610 2023-09-30T22:23:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.466+0000 7f54d4f9e700 1 -- 172.21.15.32:0/2090795114 >> v1:172.21.15.32:6816/118188 conn(0x7f54b002c1e0 legacy=0x7f54b002e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.466+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f54c803c9e0 con 0x7f54d009e610 2023-09-30T22:23:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.466+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d004f2a0 con 0x7f54d009e610 2023-09-30T22:23:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.474+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f54c8019020 con 0x7f54d009e610 2023-09-30T22:23:48.621 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:48.621+0000 7f4e8b1c9240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T22:23:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.667+0000 7f54d4f9e700 1 -- 172.21.15.32:0/2090795114 >> v1:172.21.15.32:6816/118188 conn(0x7f54b002c1e0 legacy=0x7f54b002e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:48.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.763+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f54d0135070 con 0x7f54d009e610 2023-09-30T22:23:48.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.764+0000 7f54cd7fa700 1 -- 172.21.15.32:0/2090795114 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+64281 (secure 0 0 0) 0x7f54c802c090 con 0x7f54d009e610 2023-09-30T22:23:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 >> v1:172.21.15.32:6816/118188 conn(0x7f54b002c1e0 legacy=0x7f54b002e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 msgr2=0x7f54d0133710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 0x7f54d0133710 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f54c8010550 tx=0x7f54c8010860 comp rx=0 tx=0).stop 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 shutdown_connections 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d009e610 0x7f54d0133710 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d009d8d0 0x7f54d0132c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 --2- 172.21.15.32:0/2090795114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d009e1e0 0x7f54d01331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 >> 172.21.15.32:0/2090795114 conn(0x7f54d00a2e20 msgr2=0x7f54d0070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 shutdown_connections 2023-09-30T22:23:48.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.767+0000 7f54d5fa0700 1 -- 172.21.15.32:0/2090795114 wait complete. 2023-09-30T22:23:48.781 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6816","nonce":118188}]},"active_addr":"172.21.15.32:6816/118188","active_change":"2023-09-30T22:21:25.527255+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":708160155}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2862279367}]}]} 2023-09-30T22:23:48.783 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:23:48.783 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:23:48.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:23:48.907 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:48.908+0000 7f4e8b1c9240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T22:23:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.920+0000 7fe5420cb700 1 Processor -- start 2023-09-30T22:23:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.920+0000 7fe5420cb700 1 -- start start 2023-09-30T22:23:48.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe5420cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe53c09e930 0x7fe53c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe5420cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f240 0x7fe53c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe5420cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f650 0x7fe53c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe5420cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53c070940 con 0x7fe53c09e930 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe5420cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c06fd60 con 0x7fe53c09f240 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe5420cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c06fa80 con 0x7fe53c09f650 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe53c09e930 0x7fe53c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f650 0x7fe53c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f240 0x7fe53c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe53c09e930 0x7fe53c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60984/0 (socket says 172.21.15.32:60984) 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53b7fe700 1 -- 172.21.15.32:0/3827545722 learned_addr learned my addr 172.21.15.32:0/3827545722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53bfff700 1 -- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f240 msgr2=0x7fe53c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53bfff700 1 --2- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f240 0x7fe53c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53bfff700 1 -- 172.21.15.32:0/3827545722 >> v2:172.21.15.32:3300/0 conn(0x7fe53c09e930 msgr2=0x7fe53c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53bfff700 1 --2- 172.21.15.32:0/3827545722 >> v2:172.21.15.32:3300/0 conn(0x7fe53c09e930 0x7fe53c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.921+0000 7fe53bfff700 1 -- 172.21.15.32:0/3827545722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c09fb60 con 0x7fe53c09f650 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe53affd700 1 --2- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f240 0x7fe53c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe53bfff700 1 --2- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f650 0x7fe53c131d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe534009840 tx=0x7fe534009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d990b633f114337 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe53a7fc700 1 -- 172.21.15.32:0/3827545722 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534011830 con 0x7fe53c09f650 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe53a7fc700 1 -- 172.21.15.32:0/3827545722 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe534011990 con 0x7fe53c09f650 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe53a7fc700 1 -- 172.21.15.32:0/3827545722 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534011d20 con 0x7fe53c09f650 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe5420cb700 1 -- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f650 msgr2=0x7fe53c131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.922+0000 7fe5420cb700 1 --2- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f650 0x7fe53c131d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe534009840 tx=0x7fe534009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 -- 172.21.15.32:0/3827545722 shutdown_connections 2023-09-30T22:23:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 --2- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f650 0x7fe53c131d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 --2- 172.21.15.32:0/3827545722 >> v2:172.21.15.32:3300/0 conn(0x7fe53c09e930 0x7fe53c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 --2- 172.21.15.32:0/3827545722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f240 0x7fe53c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 -- 172.21.15.32:0/3827545722 >> 172.21.15.32:0/3827545722 conn(0x7fe53c0a4210 msgr2=0x7fe53c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 -- 172.21.15.32:0/3827545722 shutdown_connections 2023-09-30T22:23:48.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.923+0000 7fe5420cb700 1 -- 172.21.15.32:0/3827545722 wait complete. 2023-09-30T22:23:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.924+0000 7fe5420cb700 1 Processor -- start 2023-09-30T22:23:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.924+0000 7fe5420cb700 1 -- start start 2023-09-30T22:23:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe5420cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 0x7fe53c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe5420cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f240 0x7fe53c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe5420cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f650 0x7fe53c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe5420cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1a8810 con 0x7fe53c09e930 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe5420cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1a9850 con 0x7fe53c09f650 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe5420cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c06cd10 con 0x7fe53c09f240 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 0x7fe53c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe53b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 0x7fe53c06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60992/0 (socket says 172.21.15.32:60992) 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe53b7fe700 1 -- 172.21.15.32:0/2478091432 learned_addr learned my addr 172.21.15.32:0/2478091432 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe53bfff700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f650 0x7fe53c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.925+0000 7fe53affd700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f240 0x7fe53c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe53b7fe700 1 -- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f240 msgr2=0x7fe53c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe53b7fe700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f240 0x7fe53c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe53b7fe700 1 -- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f650 msgr2=0x7fe53c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe53b7fe700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f650 0x7fe53c06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe53b7fe700 1 -- 172.21.15.32:0/2478091432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300094a0 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe53b7fe700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 0x7fe53c06b8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00ef00 tx=0x7fe52c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c010040 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c00b8f0 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00bc60 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5340094a0 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.926+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c0a0190 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.927+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe52c01d410 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.927+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fe52c03b790 con 0x7fe53c09e930 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.927+0000 7fe53affd700 1 -- 172.21.15.32:0/2478091432 >> v1:172.21.15.32:6816/118188 conn(0x7fe52802c1e0 legacy=0x7fe52802e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.927+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53c1a0b30 con 0x7fe53c09e930 2023-09-30T22:23:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:48.932+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fe52c01b900 con 0x7fe53c09e930 2023-09-30T22:23:49.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.117+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe53c074ea0 con 0x7fe53c09e930 2023-09-30T22:23:49.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.119+0000 7fe523fff700 1 -- 172.21.15.32:0/2478091432 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+12689 (secure 0 0 0) 0x7fe52c01b310 con 0x7fe53c09e930 2023-09-30T22:23:49.119 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:49.119 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:22:26.786266+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:23:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 >> v1:172.21.15.32:6816/118188 conn(0x7fe52802c1e0 legacy=0x7fe52802e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 msgr2=0x7fe53c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 0x7fe53c06b8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00ef00 tx=0x7fe52c00b390 comp rx=0 tx=0).stop 2023-09-30T22:23:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 shutdown_connections 2023-09-30T22:23:49.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe53c09f650 0x7fe53c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe53c09e930 0x7fe53c06b8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 --2- 172.21.15.32:0/2478091432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe53c09f240 0x7fe53c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 >> 172.21.15.32:0/2478091432 conn(0x7fe53c0a4210 msgr2=0x7fe53c1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 shutdown_connections 2023-09-30T22:23:49.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.121+0000 7fe5420cb700 1 -- 172.21.15.32:0/2478091432 wait complete. 2023-09-30T22:23:49.135 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:23:49.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:23:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.276+0000 7f6971b63700 1 Processor -- start 2023-09-30T22:23:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f6971b63700 1 -- start start 2023-09-30T22:23:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f6971b63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 0x7f696c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f6971b63700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f696c138d80 0x7f696c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f6971b63700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696c0a2a70 con 0x7f696c09d630 2023-09-30T22:23:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f6971b63700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696c0a01d0 con 0x7f696c138d80 2023-09-30T22:23:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f6971b63700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c09e2b0 con 0x7f696c0a0a30 2023-09-30T22:23:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 0x7f696c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 0x7f696c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59160/0 (socket says 172.21.15.32:59160) 2023-09-30T22:23:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f696b7fe700 1 -- 172.21.15.32:0/2263221933 learned_addr learned my addr 172.21.15.32:0/2263221933 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.277+0000 7f696bfff700 1 --2- 172.21.15.32:0/2263221933 >> v2:172.21.15.32:3300/0 conn(0x7f696c138d80 0x7f696c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696b7fe700 1 -- 172.21.15.32:0/2263221933 >> v2:172.21.15.32:3300/0 conn(0x7f696c138d80 msgr2=0x7f696c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696b7fe700 1 --2- 172.21.15.32:0/2263221933 >> v2:172.21.15.32:3300/0 conn(0x7f696c138d80 0x7f696c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696b7fe700 1 -- 172.21.15.32:0/2263221933 >> v1:172.21.15.32:6789/0 conn(0x7f696c09d630 legacy=0x7f696c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696b7fe700 1 -- 172.21.15.32:0/2263221933 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c09dc50 con 0x7f696c0a0a30 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696b7fe700 1 --2- 172.21.15.32:0/2263221933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 0x7f696c0a0e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f695c009e50 tx=0x7f695c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc5372cfc4003f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696a7fc700 1 -- 172.21.15.32:0/2263221933 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c00e070 con 0x7f696c0a0a30 2023-09-30T22:23:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696a7fc700 1 -- 172.21.15.32:0/2263221933 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c003a20 con 0x7f696c0a0a30 2023-09-30T22:23:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.278+0000 7f696a7fc700 1 -- 172.21.15.32:0/2263221933 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c003dd0 con 0x7f696c0a0a30 2023-09-30T22:23:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.279+0000 7f6971b63700 1 -- 172.21.15.32:0/2263221933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 msgr2=0x7f696c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.279+0000 7f6971b63700 1 --2- 172.21.15.32:0/2263221933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 0x7f696c0a0e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f695c009e50 tx=0x7f695c00b330 comp rx=0 tx=0).stop 2023-09-30T22:23:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.279+0000 7f6971b63700 1 -- 172.21.15.32:0/2263221933 shutdown_connections 2023-09-30T22:23:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.279+0000 7f6971b63700 1 --2- 172.21.15.32:0/2263221933 >> v2:172.21.15.32:3300/0 conn(0x7f696c138d80 0x7f696c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.279+0000 7f6971b63700 1 --2- 172.21.15.32:0/2263221933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c0a0a30 0x7f696c0a0e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.279+0000 7f6971b63700 1 -- 172.21.15.32:0/2263221933 >> 172.21.15.32:0/2263221933 conn(0x7f696c0a3050 msgr2=0x7f696c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.280+0000 7f6971b63700 1 -- 172.21.15.32:0/2263221933 shutdown_connections 2023-09-30T22:23:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.280+0000 7f6971b63700 1 -- 172.21.15.32:0/2263221933 wait complete. 2023-09-30T22:23:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.282+0000 7f6971b63700 1 Processor -- start 2023-09-30T22:23:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.282+0000 7f6971b63700 1 -- start start 2023-09-30T22:23:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f6971b63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 0x7f696c1ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f6971b63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0a0a30 0x7f696c1cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f6971b63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f696c138d80 0x7f696c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f6971b63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c06f760 con 0x7f696c0a0a30 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f6971b63700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696c06f530 con 0x7f696c09d630 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f6971b63700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f696c071540 con 0x7f696c138d80 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 0x7f696c1ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 0x7f696c1ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59170/0 (socket says 172.21.15.32:59170) 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 -- 172.21.15.32:0/3375140692 learned_addr learned my addr 172.21.15.32:0/3375140692 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696bfff700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f696c138d80 0x7f696c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 -- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f696c138d80 msgr2=0x7f696c06da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f696c138d80 0x7f696c06da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 -- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0a0a30 msgr2=0x7f696c1cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0a0a30 0x7f696c1cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.283+0000 7f696b7fe700 1 -- 172.21.15.32:0/3375140692 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c009b30 con 0x7f696c09d630 2023-09-30T22:23:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.284+0000 7f696b7fe700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 0x7f696c1ccbb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f695c00b680 tx=0x7f695c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.284+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c004440 con 0x7f696c09d630 2023-09-30T22:23:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.284+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c1c0650 con 0x7f696c09d630 2023-09-30T22:23:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.284+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0045c0 con 0x7f696c09d630 2023-09-30T22:23:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.284+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c004970 con 0x7f696c09d630 2023-09-30T22:23:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.285+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c0a0410 con 0x7f696c09d630 2023-09-30T22:23:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.285+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c071380 con 0x7f696c09d630 2023-09-30T22:23:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.286+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f695c02e430 con 0x7f696c09d630 2023-09-30T22:23:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.286+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f695c03cdb0 con 0x7f696c09d630 2023-09-30T22:23:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.286+0000 7f696affd700 1 -- 172.21.15.32:0/3375140692 >> v1:172.21.15.32:6816/118188 conn(0x7f695802c1e0 legacy=0x7f695802e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.290+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f695c0243b0 con 0x7f696c09d630 2023-09-30T22:23:49.467 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:49.466+0000 7f4e8b1c9240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T22:23:49.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.474+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f696c071380 con 0x7f696c09d630 2023-09-30T22:23:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.476+0000 7f6953fff700 1 -- 172.21.15.32:0/3375140692 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+12689 (secure 0 0 0) 0x7f696c071380 con 0x7f696c09d630 2023-09-30T22:23:49.477 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:49.477 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:22:26.786266+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:23:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 >> v1:172.21.15.32:6816/118188 conn(0x7f695802c1e0 legacy=0x7f695802e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 msgr2=0x7f696c1ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 0x7f696c1ccbb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f695c00b680 tx=0x7f695c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 shutdown_connections 2023-09-30T22:23:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f696c138d80 0x7f696c06da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f696c09d630 0x7f696c1ccbb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 --2- 172.21.15.32:0/3375140692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f696c0a0a30 0x7f696c1cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 >> 172.21.15.32:0/3375140692 conn(0x7f696c0a3050 msgr2=0x7f696c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 shutdown_connections 2023-09-30T22:23:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.478+0000 7f6971b63700 1 -- 172.21.15.32:0/3375140692 wait complete. 2023-09-30T22:23:49.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:23:49.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:23:49.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:23:49.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:23:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.645+0000 7fea091b3700 1 Processor -- start 2023-09-30T22:23:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.645+0000 7fea091b3700 1 -- start start 2023-09-30T22:23:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea091b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea0412cf60 0x7fea0406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea091b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 0x7fea0412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea091b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea04130260 0x7fea04136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea091b3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0412d330 con 0x7fea0412cf60 2023-09-30T22:23:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea091b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0412cd90 con 0x7fea04130260 2023-09-30T22:23:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea091b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea041306d0 con 0x7fea0412f8b0 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 0x7fea0412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 0x7fea0412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34174/0 (socket says 172.21.15.32:34174) 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 -- 172.21.15.32:0/2158151571 learned_addr learned my addr 172.21.15.32:0/2158151571 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea02d9d700 1 --2- 172.21.15.32:0/2158151571 >> v2:172.21.15.32:3300/0 conn(0x7fea0412cf60 0x7fea0406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 -- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea04130260 msgr2=0x7fea04136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 --2- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea04130260 0x7fea04136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 -- 172.21.15.32:0/2158151571 >> v2:172.21.15.32:3300/0 conn(0x7fea0412cf60 msgr2=0x7fea0406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 --2- 172.21.15.32:0/2158151571 >> v2:172.21.15.32:3300/0 conn(0x7fea0412cf60 0x7fea0406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.646+0000 7fea0259c700 1 -- 172.21.15.32:0/2158151571 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04071780 con 0x7fea0412f8b0 2023-09-30T22:23:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.647+0000 7fea0259c700 1 --2- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 0x7fea0412fd20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800bbc0 tx=0x7fe9f800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0341a5978cc2ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.650+0000 7fea01d9b700 1 -- 172.21.15.32:0/2158151571 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800dc40 con 0x7fea0412f8b0 2023-09-30T22:23:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.650+0000 7fea01d9b700 1 -- 172.21.15.32:0/2158151571 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800dda0 con 0x7fea0412f8b0 2023-09-30T22:23:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.650+0000 7fea01d9b700 1 -- 172.21.15.32:0/2158151571 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8017630 con 0x7fea0412f8b0 2023-09-30T22:23:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.652+0000 7fea091b3700 1 -- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 msgr2=0x7fea0412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.652+0000 7fea091b3700 1 --2- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 0x7fea0412fd20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800bbc0 tx=0x7fe9f800bed0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 -- 172.21.15.32:0/2158151571 shutdown_connections 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 --2- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea04130260 0x7fea04136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 --2- 172.21.15.32:0/2158151571 >> v2:172.21.15.32:3300/0 conn(0x7fea0412cf60 0x7fea0406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 --2- 172.21.15.32:0/2158151571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0412f8b0 0x7fea0412fd20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 -- 172.21.15.32:0/2158151571 >> 172.21.15.32:0/2158151571 conn(0x7fea040a2aa0 msgr2=0x7fea040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 -- 172.21.15.32:0/2158151571 shutdown_connections 2023-09-30T22:23:49.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.653+0000 7fea091b3700 1 -- 172.21.15.32:0/2158151571 wait complete. 2023-09-30T22:23:49.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 Processor -- start 2023-09-30T22:23:49.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fec1423d700 1 Processor -- start 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 -- start start 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0412cf60 0x7fea0409b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0412f8b0 0x7fea0409b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea04130260 0x7fea0409bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0409e5f0 con 0x7fea0412cf60 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0409e3c0 con 0x7fea0412f8b0 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.655+0000 7fea091b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea0409c6b0 con 0x7fea04130260 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0412cf60 0x7fea0409b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0412cf60 0x7fea0409b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32798/0 (socket says 172.21.15.32:32798) 2023-09-30T22:23:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 -- 172.21.15.32:0/3521790474 learned_addr learned my addr 172.21.15.32:0/3521790474 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea0359e700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea04130260 0x7fea0409bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 -- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea04130260 msgr2=0x7fea0409bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea04130260 0x7fea0409bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 -- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0412f8b0 msgr2=0x7fea0409b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea0259c700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0412f8b0 0x7fea0409b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0412f8b0 0x7fea0409b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea02d9d700 1 -- 172.21.15.32:0/3521790474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f800b7f0 con 0x7fea0412cf60 2023-09-30T22:23:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fea0259c700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0412f8b0 0x7fea0409b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:49.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fea02d9d700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0412cf60 0x7fea0409b250 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e840 tx=0x7fe9f400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.656+0000 7fec1423d700 1 -- start start 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec1423d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12fc70 0x7fec0c130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec1423d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec0c12cb40 0x7fec0c137950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec1423d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c09be70 con 0x7fec0c12c060 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec1423d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c130580 con 0x7fec0c12cb40 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec1423d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c09a6e0 con 0x7fec0c12fc70 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec117d8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec0c12c060 0x7fec0c12c430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59614/0 (socket says 172.21.15.32:59614) 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec117d8700 1 -- 172.21.15.32:0/48203974 learned_addr learned my addr 172.21.15.32:0/48203974 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:49.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec11fd9700 1 --2- 172.21.15.32:0/48203974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12fc70 0x7fec0c130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec11fd9700 1 -- 172.21.15.32:0/48203974 >> v2:172.21.15.32:3300/0 conn(0x7fec0c12cb40 msgr2=0x7fec0c137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec11fd9700 1 --2- 172.21.15.32:0/48203974 >> v2:172.21.15.32:3300/0 conn(0x7fec0c12cb40 0x7fec0c137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec11fd9700 1 -- 172.21.15.32:0/48203974 >> v1:172.21.15.32:6789/0 conn(0x7fec0c12c060 legacy=0x7fec0c12c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fec11fd9700 1 -- 172.21.15.32:0/48203974 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c09ac80 con 0x7fec0c12fc70 2023-09-30T22:23:49.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 Processor -- start 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fec11fd9700 1 --2- 172.21.15.32:0/48203974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12fc70 0x7fec0c130040 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec08007360 tx=0x7fec08007670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f663a7dbb7b7e877 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4013610 con 0x7fea0412cf60 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4013770 con 0x7fea0412cf60 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4013b00 con 0x7fea0412cf60 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0409ea90 con 0x7fea0412cf60 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.657+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0409e7d0 con 0x7fea0412cf60 2023-09-30T22:23:49.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea0412c620 con 0x7fea0412cf60 2023-09-30T22:23:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 -- start start 2023-09-30T22:23:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 0x7fe78c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe78c09e000 0x7fe78c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78c09f130 con 0x7fe78c09e410 2023-09-30T22:23:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78c09f520 con 0x7fe78c09e000 2023-09-30T22:23:49.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fe792186700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c09e930 con 0x7fe78c09d6f0 2023-09-30T22:23:49.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fec10fd7700 1 -- 172.21.15.32:0/48203974 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800f070 con 0x7fec0c12fc70 2023-09-30T22:23:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fec10fd7700 1 -- 172.21.15.32:0/48203974 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0800ae70 con 0x7fec0c12fc70 2023-09-30T22:23:49.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.658+0000 7fec10fd7700 1 -- 172.21.15.32:0/48203974 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08008df0 con 0x7fec0c12fc70 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 0x7fe78c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 0x7fe78c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34202/0 (socket says 172.21.15.32:34202) 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 -- 172.21.15.32:0/981379128 learned_addr learned my addr 172.21.15.32:0/981379128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 -- 172.21.15.32:0/981379128 >> v2:172.21.15.32:3300/0 conn(0x7fe78c09e000 msgr2=0x7fe78c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 --2- 172.21.15.32:0/981379128 >> v2:172.21.15.32:3300/0 conn(0x7fe78c09e000 0x7fe78c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 -- 172.21.15.32:0/981379128 >> v1:172.21.15.32:6789/0 conn(0x7fe78c09e410 legacy=0x7fe78c130ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 -- 172.21.15.32:0/981379128 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c09b2d0 con 0x7fe78c09d6f0 2023-09-30T22:23:49.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78b7fe700 1 --2- 172.21.15.32:0/981379128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 0x7fe78c09dac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098f0 tx=0x7fe77c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411b96d6f02aa6e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fec1423d700 1 -- 172.21.15.32:0/48203974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12fc70 msgr2=0x7fec0c130040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fec1423d700 1 --2- 172.21.15.32:0/48203974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12fc70 0x7fec0c130040 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec08007360 tx=0x7fec08007670 comp rx=0 tx=0).stop 2023-09-30T22:23:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78a7fc700 1 -- 172.21.15.32:0/981379128 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c004230 con 0x7fe78c09d6f0 2023-09-30T22:23:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78a7fc700 1 -- 172.21.15.32:0/981379128 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c004390 con 0x7fe78c09d6f0 2023-09-30T22:23:49.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.659+0000 7fe78a7fc700 1 -- 172.21.15.32:0/981379128 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c0036a0 con 0x7fe78c09d6f0 2023-09-30T22:23:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fec1423d700 1 -- 172.21.15.32:0/48203974 shutdown_connections 2023-09-30T22:23:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fec1423d700 1 --2- 172.21.15.32:0/48203974 >> v2:172.21.15.32:3300/0 conn(0x7fec0c12cb40 0x7fec0c137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fec1423d700 1 --2- 172.21.15.32:0/48203974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12fc70 0x7fec0c130040 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fec1423d700 1 -- 172.21.15.32:0/48203974 >> 172.21.15.32:0/48203974 conn(0x7fec0c09e7e0 msgr2=0x7fec0c09ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe9f4013c60 con 0x7fea0412cf60 2023-09-30T22:23:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fea0259c700 1 -- 172.21.15.32:0/3521790474 >> v1:172.21.15.32:6816/118188 conn(0x7fe9f002c1b0 legacy=0x7fe9f002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.660+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fe9f403bc30 con 0x7fea0412cf60 2023-09-30T22:23:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 --> v1:172.21.15.32:6809/118794 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9f0038e10 con 0x7fe9f0035110 2023-09-30T22:23:49.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f403c030 con 0x7fea0412cf60 2023-09-30T22:23:49.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 -- 172.21.15.32:0/981379128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 msgr2=0x7fe78c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 --2- 172.21.15.32:0/981379128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 0x7fe78c09dac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0098f0 tx=0x7fe77c009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:49.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fec1423d700 1 -- 172.21.15.32:0/48203974 shutdown_connections 2023-09-30T22:23:49.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 -- 172.21.15.32:0/981379128 shutdown_connections 2023-09-30T22:23:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 --2- 172.21.15.32:0/981379128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09d6f0 0x7fe78c09dac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 --2- 172.21.15.32:0/981379128 >> v2:172.21.15.32:3300/0 conn(0x7fe78c09e000 0x7fe78c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 -- 172.21.15.32:0/981379128 >> 172.21.15.32:0/981379128 conn(0x7fe78c066f50 msgr2=0x7fe78c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fec1423d700 1 -- 172.21.15.32:0/48203974 wait complete. 2023-09-30T22:23:49.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 -- 172.21.15.32:0/981379128 shutdown_connections 2023-09-30T22:23:49.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.661+0000 7fe792186700 1 -- 172.21.15.32:0/981379128 wait complete. 2023-09-30T22:23:49.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 Processor -- start 2023-09-30T22:23:49.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fe792186700 1 Processor -- start 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 -- start start 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c12c060 0x7fec0c127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12cb40 0x7fec0c127dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c12fc70 0x7fec0c128300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06e380 con 0x7fec0c12c060 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06f520 con 0x7fec0c12fc70 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.662+0000 7fec1423d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06e5c0 con 0x7fec0c12cb40 2023-09-30T22:23:49.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 -- start start 2023-09-30T22:23:49.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12cb40 0x7fec0c127dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12cb40 0x7fec0c127dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34206/0 (socket says 172.21.15.32:34206) 2023-09-30T22:23:49.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 -- 172.21.15.32:0/1180569864 learned_addr learned my addr 172.21.15.32:0/1180569864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec127da700 1 --2- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c12fc70 0x7fec0c128300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 -- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c12fc70 msgr2=0x7fec0c128300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 --2- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c12fc70 0x7fec0c128300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 -- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c12c060 msgr2=0x7fec0c127880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 --2- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c12c060 0x7fec0c127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fec117d8700 1 -- 172.21.15.32:0/1180569864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08007040 con 0x7fec0c12cb40 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe78c09d6f0 0x7fe78c126e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09e000 0x7fe78c127370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe78c09e410 0x7fe78c1cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c129160 con 0x7fe78c09e410 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c129da0 con 0x7fe78c09d6f0 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.663+0000 7fe792186700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c071c60 con 0x7fe78c09e000 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09e000 0x7fe78c127370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09e000 0x7fe78c127370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34212/0 (socket says 172.21.15.32:34212) 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 -- 172.21.15.32:0/1448024218 learned_addr learned my addr 172.21.15.32:0/1448024218 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78b7fe700 1 --2- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe78c09d6f0 0x7fe78c126e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== osd.3 v1:172.21.15.32:6809/118794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fe9f0038e10 con 0x7fe9f0035110 2023-09-30T22:23:49.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 -- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe78c09d6f0 msgr2=0x7fe78c126e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 --2- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe78c09d6f0 0x7fe78c126e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 -- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe78c09e410 msgr2=0x7fe78c1cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 --2- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe78c09e410 0x7fe78c1cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.664+0000 7fe78affd700 1 -- 172.21.15.32:0/1448024218 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009550 con 0x7fe78c09e000 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.665+0000 7fe78affd700 1 --2- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09e000 0x7fe78c127370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b530 tx=0x7fe78000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.667+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780011830 con 0x7fe78c09e000 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.667+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800119b0 con 0x7fe78c09e000 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.667+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780011d60 con 0x7fe78c09e000 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.667+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c129ae0 con 0x7fe78c09e000 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.667+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c12a810 con 0x7fe78c09e000 2023-09-30T22:23:49.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.668+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe78c09c000 con 0x7fe78c09e000 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.668+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe78001a400 con 0x7fe78c09e000 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.668+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fe780044400 con 0x7fe78c09e000 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.668+0000 7fe78b7fe700 1 -- 172.21.15.32:0/1448024218 >> v1:172.21.15.32:6816/118188 conn(0x7fe77802c1b0 legacy=0x7fe77802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.669+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 --> v1:172.21.15.32:6808/118793 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe778038d90 con 0x7fe778035090 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.669+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe78001a6b0 con 0x7fe78c09e000 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.669+0000 7fec117d8700 1 --2- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12cb40 0x7fec0c127dc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7febfc00e790 tx=0x7febfc00eaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.669+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc005d70 con 0x7fec0c12cb40 2023-09-30T22:23:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.669+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc007070 con 0x7fec0c12cb40 2023-09-30T22:23:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.669+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc007400 con 0x7fec0c12cb40 2023-09-30T22:23:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.670+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== osd.2 v1:172.21.15.32:6808/118793 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fe778038d90 con 0x7fe778035090 2023-09-30T22:23:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.671+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c06d850 con 0x7fec0c12cb40 2023-09-30T22:23:49.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.671+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c1293d0 con 0x7fec0c12cb40 2023-09-30T22:23:49.681 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:49.671+0000 7f4e8b1c9240 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T22:23:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.672+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec0c09b9b0 con 0x7fec0c12cb40 2023-09-30T22:23:49.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.673+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7febfc007560 con 0x7fec0c12cb40 2023-09-30T22:23:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.673+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7febfc03df60 con 0x7fec0c12cb40 2023-09-30T22:23:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.673+0000 7fec11fd9700 1 -- 172.21.15.32:0/1180569864 >> v1:172.21.15.32:6816/118188 conn(0x7febf002c200 legacy=0x7febf002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.673+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 --> v1:172.21.15.32:6804/118763 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febf0038ee0 con 0x7febf00351e0 2023-09-30T22:23:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.673+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7febfc03e1d0 con 0x7fec0c12cb40 2023-09-30T22:23:49.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.680+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== osd.0 v1:172.21.15.32:6804/118763 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7febf0038ee0 con 0x7febf00351e0 2023-09-30T22:23:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.683+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 --> v1:172.21.15.32:6809/118794 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea04072220 con 0x7fe9f0035110 2023-09-30T22:23:49.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.683+0000 7fe9eb7fe700 1 -- 172.21.15.32:0/3521790474 <== osd.3 v1:172.21.15.32:6809/118794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1880693312) 0x7fe9fc009aa0 con 0x7fe9f0035110 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 >> v1:172.21.15.32:6809/118794 conn(0x7fe9f0035110 legacy=0x7fe9f00375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 >> v1:172.21.15.32:6816/118188 conn(0x7fe9f002c1b0 legacy=0x7fe9f002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0412cf60 msgr2=0x7fea0409b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea091b3700 1 --2- 172.21.15.32:0/3521790474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0412cf60 0x7fea0409b250 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e840 tx=0x7fe9f400eb50 comp rx=0 tx=0).stop 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea0359e700 1 -- 172.21.15.32:0/3521790474 reap_dead start 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 shutdown_connections 2023-09-30T22:23:49.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.684+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 >> 172.21.15.32:0/3521790474 conn(0x7fea040a2aa0 msgr2=0x7fea041b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.688+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 shutdown_connections 2023-09-30T22:23:49.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.688+0000 7fea091b3700 1 -- 172.21.15.32:0/3521790474 wait complete. 2023-09-30T22:23:49.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.695+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 --> v1:172.21.15.32:6808/118793 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe78c09c450 con 0x7fe778035090 2023-09-30T22:23:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.695+0000 7f573fcbd700 1 Processor -- start 2023-09-30T22:23:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.695+0000 7fe773fff700 1 -- 172.21.15.32:0/1448024218 <== osd.2 v1:172.21.15.32:6808/118793 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1880693312) 0x7fe784001830 con 0x7fe778035090 2023-09-30T22:23:49.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.695+0000 7f573fcbd700 1 -- start start 2023-09-30T22:23:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573fcbd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5738130e20 0x7f5738136760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573fcbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 0x7f5738139080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573fcbd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57380716a0 con 0x7f5738130340 2023-09-30T22:23:49.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573fcbd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573809a390 con 0x7f5738130e20 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573fcbd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5738071290 con 0x7f5738136ca0 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 0x7f5738139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 0x7f5738139080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59224/0 (socket says 172.21.15.32:59224) 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 -- 172.21.15.32:0/2182611070 learned_addr learned my addr 172.21.15.32:0/2182611070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 -- 172.21.15.32:0/2182611070 >> v2:172.21.15.32:3300/0 conn(0x7f5738130e20 msgr2=0x7f5738136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 --2- 172.21.15.32:0/2182611070 >> v2:172.21.15.32:3300/0 conn(0x7f5738130e20 0x7f5738136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 -- 172.21.15.32:0/2182611070 >> v1:172.21.15.32:6789/0 conn(0x7f5738130340 legacy=0x7f5738130710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.696+0000 7f573e25a700 1 -- 172.21.15.32:0/2182611070 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738070b80 con 0x7f5738136ca0 2023-09-30T22:23:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 >> v1:172.21.15.32:6808/118793 conn(0x7fe778035090 legacy=0x7fe778037520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 >> v1:172.21.15.32:6816/118188 conn(0x7fe77802c1b0 legacy=0x7fe77802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09e000 msgr2=0x7fe78c127370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 --2- 172.21.15.32:0/1448024218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe78c09e000 0x7fe78c127370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe78000b530 tx=0x7fe78000b840 comp rx=0 tx=0).stop 2023-09-30T22:23:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe78bfff700 1 -- 172.21.15.32:0/1448024218 reap_dead start 2023-09-30T22:23:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 shutdown_connections 2023-09-30T22:23:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 >> 172.21.15.32:0/1448024218 conn(0x7fe78c066f50 msgr2=0x7fe78c0a1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 shutdown_connections 2023-09-30T22:23:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.697+0000 7fe792186700 1 -- 172.21.15.32:0/1448024218 wait complete. 2023-09-30T22:23:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.699+0000 7f573e25a700 1 --2- 172.21.15.32:0/2182611070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 0x7f5738139080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5730009e50 tx=0x7f573000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56be6e8085c2def server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.702+0000 7f573ca57700 1 -- 172.21.15.32:0/2182611070 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000e070 con 0x7f5738136ca0 2023-09-30T22:23:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.702+0000 7f573ca57700 1 -- 172.21.15.32:0/2182611070 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730003a20 con 0x7f5738136ca0 2023-09-30T22:23:49.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.702+0000 7f573ca57700 1 -- 172.21.15.32:0/2182611070 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730003dd0 con 0x7f5738136ca0 2023-09-30T22:23:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.703+0000 7f573fcbd700 1 -- 172.21.15.32:0/2182611070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 msgr2=0x7f5738139080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.703+0000 7f573fcbd700 1 --2- 172.21.15.32:0/2182611070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 0x7f5738139080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5730009e50 tx=0x7f573000b330 comp rx=0 tx=0).stop 2023-09-30T22:23:49.705 INFO:teuthology.orchestra.run.smithi032.stdout:38654705694 2023-09-30T22:23:49.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.704+0000 7f573fcbd700 1 -- 172.21.15.32:0/2182611070 shutdown_connections 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.704+0000 7f573fcbd700 1 --2- 172.21.15.32:0/2182611070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738136ca0 0x7f5738139080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.704+0000 7f573fcbd700 1 --2- 172.21.15.32:0/2182611070 >> v2:172.21.15.32:3300/0 conn(0x7f5738130e20 0x7f5738136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.704+0000 7f573fcbd700 1 -- 172.21.15.32:0/2182611070 >> 172.21.15.32:0/2182611070 conn(0x7f57380a2aa0 msgr2=0x7f57380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.704+0000 7f573fcbd700 1 -- 172.21.15.32:0/2182611070 shutdown_connections 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.704+0000 7f573fcbd700 1 -- 172.21.15.32:0/2182611070 wait complete. 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 Processor -- start 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 -- start start 2023-09-30T22:23:49.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5738130340 0x7f57381c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738130e20 0x7f57381c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5738136ca0 0x7f57381c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380718e0 con 0x7f5738136ca0 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5738132110 con 0x7f5738130e20 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.705+0000 7f573fcbd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f573809e4d0 con 0x7f5738130340 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5738130340 0x7f57381c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5738130340 0x7f57381c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34218/0 (socket says 172.21.15.32:34218) 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 -- 172.21.15.32:0/2663232529 learned_addr learned my addr 172.21.15.32:0/2663232529 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 -- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738130e20 msgr2=0x7f57381c8670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 --2- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5738130e20 0x7f57381c8670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 -- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5738136ca0 msgr2=0x7f57381c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573e25a700 1 --2- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5738136ca0 0x7f57381c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 --2- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5738136ca0 0x7f57381c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 -- 172.21.15.32:0/2663232529 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5730009b30 con 0x7f5738130340 2023-09-30T22:23:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573e25a700 1 --2- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5738136ca0 0x7f57381c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f573da59700 1 --2- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5738130340 0x7f57381c8130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f573400b470 tx=0x7f573400b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734011830 con 0x7f5738130340 2023-09-30T22:23:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.707+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5738131a30 con 0x7f5738130340 2023-09-30T22:23:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.706+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734011990 con 0x7f5738130340 2023-09-30T22:23:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.707+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734011d40 con 0x7f5738130340 2023-09-30T22:23:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.707+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573812cb30 con 0x7f5738130340 2023-09-30T22:23:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.707+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f573401a400 con 0x7f5738130340 2023-09-30T22:23:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.707+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f573403b870 con 0x7f5738130340 2023-09-30T22:23:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.708+0000 7f573d258700 1 -- 172.21.15.32:0/2663232529 >> v1:172.21.15.32:6816/118188 conn(0x7f571c02c1b0 legacy=0x7f571c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.708+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 --> v1:172.21.15.32:6800/118762 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57380621a0 con 0x7f573812ba70 2023-09-30T22:23:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.710+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== osd.1 v1:172.21.15.32:6800/118762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f57380621a0 con 0x7f573812ba70 2023-09-30T22:23:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.711+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 --> v1:172.21.15.32:6804/118763 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec0c099b00 con 0x7febf00351e0 2023-09-30T22:23:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.711+0000 7fec027fc700 1 -- 172.21.15.32:0/1180569864 <== osd.0 v1:172.21.15.32:6804/118763 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1880693312) 0x7fec04009aa0 con 0x7febf00351e0 2023-09-30T22:23:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.712+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 >> v1:172.21.15.32:6804/118763 conn(0x7febf00351e0 legacy=0x7febf0037670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 >> v1:172.21.15.32:6816/118188 conn(0x7febf002c200 legacy=0x7febf002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12cb40 msgr2=0x7fec0c127dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 --2- 172.21.15.32:0/1180569864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c12cb40 0x7fec0c127dc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7febfc00e790 tx=0x7febfc00eaa0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec127da700 1 -- 172.21.15.32:0/1180569864 reap_dead start 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 shutdown_connections 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 >> 172.21.15.32:0/1180569864 conn(0x7fec0c09e7e0 msgr2=0x7fec0c098670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 shutdown_connections 2023-09-30T22:23:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.714+0000 7fec1423d700 1 -- 172.21.15.32:0/1180569864 wait complete. 2023-09-30T22:23:49.719 INFO:teuthology.orchestra.run.smithi032.stdout:38654705694 2023-09-30T22:23:49.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:49.730 INFO:teuthology.orchestra.run.smithi032.stdout:38654705694 2023-09-30T22:23:49.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.738+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 --> v1:172.21.15.32:6800/118762 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5738070e50 con 0x7f573812ba70 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f572e7fc700 1 -- 172.21.15.32:0/2663232529 <== osd.1 v1:172.21.15.32:6800/118762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1880693312) 0x7f5730003b90 con 0x7f573812ba70 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 >> v1:172.21.15.32:6800/118762 conn(0x7f573812ba70 legacy=0x7f573812be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 >> v1:172.21.15.32:6816/118188 conn(0x7f571c02c1b0 legacy=0x7f571c02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5738130340 msgr2=0x7f57381c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 --2- 172.21.15.32:0/2663232529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5738130340 0x7f57381c8130 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f573400b470 tx=0x7f573400b830 comp rx=0 tx=0).stop 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573e25a700 1 -- 172.21.15.32:0/2663232529 reap_dead start 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 shutdown_connections 2023-09-30T22:23:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 >> 172.21.15.32:0/2663232529 conn(0x7f57380a2aa0 msgr2=0x7f573812d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.739+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 shutdown_connections 2023-09-30T22:23:49.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.740+0000 7f573fcbd700 1 -- 172.21.15.32:0/2663232529 wait complete. 2023-09-30T22:23:49.760 INFO:teuthology.orchestra.run.smithi032.stdout:38654705694 2023-09-30T22:23:49.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:49.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.868+0000 7fcc8763b700 1 Processor -- start 2023-09-30T22:23:49.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.869+0000 7fcc8763b700 1 -- start start 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.869+0000 7fcc8763b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.869+0000 7fcc8763b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.869+0000 7fcc8763b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8009f130 con 0x7fcc8009e410 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.869+0000 7fcc8763b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8009f520 con 0x7fcc8009e000 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.869+0000 7fcc8763b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8009e930 con 0x7fcc8009d6f0 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc84bd6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59238/0 (socket says 172.21.15.32:59238) 2023-09-30T22:23:49.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 -- 172.21.15.32:0/960078955 learned_addr learned my addr 172.21.15.32:0/960078955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 -- 172.21.15.32:0/960078955 >> v2:172.21.15.32:3300/0 conn(0x7fcc8009e000 msgr2=0x7fcc80130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 --2- 172.21.15.32:0/960078955 >> v2:172.21.15.32:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 -- 172.21.15.32:0/960078955 >> v1:172.21.15.32:6789/0 conn(0x7fcc8009e410 legacy=0x7fcc80130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc77fff700 1 -- 172.21.15.32:0/960078955 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313079841 0 0) 0x7fcc8009f130 con 0x7fcc8009e410 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.870+0000 7fcc853d7700 1 -- 172.21.15.32:0/960078955 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8009b2d0 con 0x7fcc8009d6f0 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.871+0000 7fcc853d7700 1 --2- 172.21.15.32:0/960078955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0098f0 tx=0x7fcc7c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fb53e64c959bb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.871+0000 7fcc77fff700 1 -- 172.21.15.32:0/960078955 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c011830 con 0x7fcc8009d6f0 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.871+0000 7fcc77fff700 1 -- 172.21.15.32:0/960078955 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c011990 con 0x7fcc8009d6f0 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.871+0000 7fcc77fff700 1 -- 172.21.15.32:0/960078955 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c011d20 con 0x7fcc8009d6f0 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 -- 172.21.15.32:0/960078955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 msgr2=0x7fcc8009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 --2- 172.21.15.32:0/960078955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0098f0 tx=0x7fcc7c009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 -- 172.21.15.32:0/960078955 shutdown_connections 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 --2- 172.21.15.32:0/960078955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8009dac0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 --2- 172.21.15.32:0/960078955 >> v2:172.21.15.32:3300/0 conn(0x7fcc8009e000 0x7fcc80130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 -- 172.21.15.32:0/960078955 >> 172.21.15.32:0/960078955 conn(0x7fcc80066f50 msgr2=0x7fcc8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 -- 172.21.15.32:0/960078955 shutdown_connections 2023-09-30T22:23:49.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.872+0000 7fcc8763b700 1 -- 172.21.15.32:0/960078955 wait complete. 2023-09-30T22:23:49.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.874+0000 7fcc8763b700 1 Processor -- start 2023-09-30T22:23:49.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.874+0000 7fcc8763b700 1 -- start start 2023-09-30T22:23:49.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc8763b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8011f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc8763b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc8009e000 0x7fcc8011f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc8763b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8009e410 0x7fcc8011fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc8763b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80130420 con 0x7fcc8009e410 2023-09-30T22:23:49.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc8763b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80126920 con 0x7fcc8009d6f0 2023-09-30T22:23:49.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc8763b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80125c60 con 0x7fcc8009e000 2023-09-30T22:23:49.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8011f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8011f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59252/0 (socket says 172.21.15.32:59252) 2023-09-30T22:23:49.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 -- 172.21.15.32:0/1795744193 learned_addr learned my addr 172.21.15.32:0/1795744193 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc85bd8700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8009e410 0x7fcc8011fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc84bd6700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc8009e000 0x7fcc8011f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 -- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc8009e000 msgr2=0x7fcc8011f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc8009e000 0x7fcc8011f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 -- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8009e410 msgr2=0x7fcc8011fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8009e410 0x7fcc8011fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc853d7700 1 -- 172.21.15.32:0/1795744193 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c009550 con 0x7fcc8009d6f0 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.875+0000 7fcc85bd8700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8009e410 0x7fcc8011fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.876+0000 7fcc853d7700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8011f1f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009130 tx=0x7fcc7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.876+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0036a0 con 0x7fcc8009d6f0 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.876+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c01e070 con 0x7fcc8009d6f0 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.876+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8009a0c0 con 0x7fcc8009d6f0 2023-09-30T22:23:49.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.876+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003a20 con 0x7fcc8009d6f0 2023-09-30T22:23:49.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.877+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc80125f40 con 0x7fcc8009d6f0 2023-09-30T22:23:49.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.877+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc80074ea0 con 0x7fcc8009d6f0 2023-09-30T22:23:49.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.878+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fcc7c003b80 con 0x7fcc8009d6f0 2023-09-30T22:23:49.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.878+0000 7fcc84bd6700 1 -- 172.21.15.32:0/1795744193 >> v1:172.21.15.32:6816/118188 conn(0x7fcc6402c200 legacy=0x7fcc6402e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.880+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fcc7c019c40 con 0x7fcc8009d6f0 2023-09-30T22:23:49.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.882+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fcc7c042060 con 0x7fcc8009d6f0 2023-09-30T22:23:49.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 Processor -- start 2023-09-30T22:23:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 -- start start 2023-09-30T22:23:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f169c06f5d0 0x7f169c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 0x7f169c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c06fa40 con 0x7f169c12cf60 2023-09-30T22:23:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169c12cd90 con 0x7f169c06f5d0 2023-09-30T22:23:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.900+0000 7f16a1de7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169c071780 con 0x7f169c135c20 2023-09-30T22:23:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 0x7f169c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 0x7f169c138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34238/0 (socket says 172.21.15.32:34238) 2023-09-30T22:23:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 -- 172.21.15.32:0/2423032296 learned_addr learned my addr 172.21.15.32:0/2423032296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169affd700 1 --2- 172.21.15.32:0/2423032296 >> v2:172.21.15.32:3300/0 conn(0x7f169c06f5d0 0x7f169c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 -- 172.21.15.32:0/2423032296 >> v2:172.21.15.32:3300/0 conn(0x7f169c06f5d0 msgr2=0x7f169c12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 --2- 172.21.15.32:0/2423032296 >> v2:172.21.15.32:3300/0 conn(0x7f169c06f5d0 0x7f169c12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 -- 172.21.15.32:0/2423032296 >> v1:172.21.15.32:6789/0 conn(0x7f169c12cf60 legacy=0x7f169c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 -- 172.21.15.32:0/2423032296 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c071a40 con 0x7f169c135c20 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.901+0000 7f169bfff700 1 --2- 172.21.15.32:0/2423032296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 0x7f169c138000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f169400d150 tx=0x7f169400d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5e7676dfeed9c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.902+0000 7f169a7fc700 1 -- 172.21.15.32:0/2423032296 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694010070 con 0x7f169c135c20 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.902+0000 7f169a7fc700 1 -- 172.21.15.32:0/2423032296 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169400dbb0 con 0x7f169c135c20 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.902+0000 7f169a7fc700 1 -- 172.21.15.32:0/2423032296 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694016400 con 0x7f169c135c20 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 -- 172.21.15.32:0/2423032296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 msgr2=0x7f169c138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2423032296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 0x7f169c138000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f169400d150 tx=0x7f169400d460 comp rx=0 tx=0).stop 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 -- 172.21.15.32:0/2423032296 shutdown_connections 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2423032296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c135c20 0x7f169c138000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2423032296 >> v2:172.21.15.32:3300/0 conn(0x7f169c06f5d0 0x7f169c12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 -- 172.21.15.32:0/2423032296 >> 172.21.15.32:0/2423032296 conn(0x7f169c0a2aa0 msgr2=0x7f169c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 -- 172.21.15.32:0/2423032296 shutdown_connections 2023-09-30T22:23:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.903+0000 7f16a1de7700 1 -- 172.21.15.32:0/2423032296 wait complete. 2023-09-30T22:23:49.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.904+0000 7f16a1de7700 1 Processor -- start 2023-09-30T22:23:49.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.904+0000 7f16a1de7700 1 -- start start 2023-09-30T22:23:49.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.904+0000 7f16a1de7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f169c06f5d0 0x7f169c1c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.904+0000 7f16a1de7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 0x7f169c1c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f16a1de7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169c135c20 0x7f169c1c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f16a1de7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c134830 con 0x7f169c135c20 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f16a1de7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c119040 con 0x7f169c06f5d0 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f16a1de7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169c135760 con 0x7f169c12cf60 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 0x7f169c1c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 0x7f169c1c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34252/0 (socket says 172.21.15.32:34252) 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169affd700 1 -- 172.21.15.32:0/2425747308 learned_addr learned my addr 172.21.15.32:0/2425747308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169c135c20 0x7f169c1c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 -- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 msgr2=0x7f169c1c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 0x7f169c1c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 -- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f169c06f5d0 msgr2=0x7f169c1c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f169c06f5d0 0x7f169c1c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 -- 172.21.15.32:0/2425747308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169400bd40 con 0x7f169c135c20 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169affd700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 0x7f169c1c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.905+0000 7f169bfff700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169c135c20 0x7f169c1c8d20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1694019000 tx=0x7f1694004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.906+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694016400 con 0x7f169c135c20 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.906+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694007160 con 0x7f169c135c20 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.906+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694017ce0 con 0x7f169c135c20 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.906+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c133030 con 0x7f169c135c20 2023-09-30T22:23:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.906+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c09e3d0 con 0x7f169c135c20 2023-09-30T22:23:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.907+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c1355a0 con 0x7f169c135c20 2023-09-30T22:23:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.908+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f1694004040 con 0x7f169c135c20 2023-09-30T22:23:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.908+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f169403ddc0 con 0x7f169c135c20 2023-09-30T22:23:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.908+0000 7f169b7fe700 1 -- 172.21.15.32:0/2425747308 >> v1:172.21.15.32:6816/118188 conn(0x7f168802c1b0 legacy=0x7f168802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.912+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f169402a460 con 0x7f169c135c20 2023-09-30T22:23:49.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.935+0000 7fe693542700 1 Processor -- start 2023-09-30T22:23:49.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.935+0000 7fe693542700 1 -- start start 2023-09-30T22:23:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe693542700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe693542700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe68c09a1c0 0x7fe68c1385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe693542700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c09c280 con 0x7fe68c099680 2023-09-30T22:23:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe693542700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c09aaf0 con 0x7fe68c09a1c0 2023-09-30T22:23:49.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe693542700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c09b960 con 0x7fe68c0954c0 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe6912de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe690add700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe68c099680 0x7fe68c099ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59648/0 (socket says 172.21.15.32:59648) 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe6912de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c095890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59268/0 (socket says 172.21.15.32:59268) 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.936+0000 7fe690add700 1 -- 172.21.15.32:0/42412655 learned_addr learned my addr 172.21.15.32:0/42412655 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe691adf700 1 --2- 172.21.15.32:0/42412655 >> v2:172.21.15.32:3300/0 conn(0x7fe68c09a1c0 0x7fe68c1385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe6912de700 1 -- 172.21.15.32:0/42412655 >> v2:172.21.15.32:3300/0 conn(0x7fe68c09a1c0 msgr2=0x7fe68c1385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe6912de700 1 --2- 172.21.15.32:0/42412655 >> v2:172.21.15.32:3300/0 conn(0x7fe68c09a1c0 0x7fe68c1385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe6912de700 1 -- 172.21.15.32:0/42412655 >> v1:172.21.15.32:6789/0 conn(0x7fe68c099680 legacy=0x7fe68c099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe6912de700 1 -- 172.21.15.32:0/42412655 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c09a7b0 con 0x7fe68c0954c0 2023-09-30T22:23:49.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe6912de700 1 --2- 172.21.15.32:0/42412655 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c095890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe688009840 tx=0x7fe688009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72eb429f9b4eef1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe683fff700 1 -- 172.21.15.32:0/42412655 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688012b80 con 0x7fe68c0954c0 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.937+0000 7fe683fff700 1 -- 172.21.15.32:0/42412655 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688012ce0 con 0x7fe68c0954c0 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.938+0000 7fe683fff700 1 -- 172.21.15.32:0/42412655 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68801b620 con 0x7fe68c0954c0 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 -- 172.21.15.32:0/42412655 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 msgr2=0x7fe68c095890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 --2- 172.21.15.32:0/42412655 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c095890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe688009840 tx=0x7fe688009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 -- 172.21.15.32:0/42412655 shutdown_connections 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 --2- 172.21.15.32:0/42412655 >> v2:172.21.15.32:3300/0 conn(0x7fe68c09a1c0 0x7fe68c1385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 --2- 172.21.15.32:0/42412655 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c095890 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 -- 172.21.15.32:0/42412655 >> 172.21.15.32:0/42412655 conn(0x7fe68c0a2360 msgr2=0x7fe68c0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 -- 172.21.15.32:0/42412655 shutdown_connections 2023-09-30T22:23:49.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.939+0000 7fe693542700 1 -- 172.21.15.32:0/42412655 wait complete. 2023-09-30T22:23:49.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.940+0000 7fe693542700 1 Processor -- start 2023-09-30T22:23:49.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.940+0000 7fe693542700 1 -- start start 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe693542700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c127240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe693542700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe68c099680 0x7fe68c127780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe693542700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c09a1c0 0x7fe68c127cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe693542700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1292c0 con 0x7fe68c09a1c0 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe693542700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0a0c40 con 0x7fe68c0954c0 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe693542700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c09b000 con 0x7fe68c099680 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe6912de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c127240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe6912de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c127240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59274/0 (socket says 172.21.15.32:59274) 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.941+0000 7fe6912de700 1 -- 172.21.15.32:0/623561760 learned_addr learned my addr 172.21.15.32:0/623561760 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe690add700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe68c099680 0x7fe68c127780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe6912de700 1 -- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe68c099680 msgr2=0x7fe68c127780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe691adf700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c09a1c0 0x7fe68c127cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe6912de700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe68c099680 0x7fe68c127780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe6912de700 1 -- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c09a1c0 msgr2=0x7fe68c127cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe6912de700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c09a1c0 0x7fe68c127cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe6912de700 1 -- 172.21.15.32:0/623561760 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880094a0 con 0x7fe68c0954c0 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe690add700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe68c099680 0x7fe68c127780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:49.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe691adf700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c09a1c0 0x7fe68c127cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:23:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe6912de700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c127240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe688009fd0 tx=0x7fe688004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.942+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688020070 con 0x7fe68c0954c0 2023-09-30T22:23:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.943+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6880037e0 con 0x7fe68c0954c0 2023-09-30T22:23:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.943+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688003b70 con 0x7fe68c0954c0 2023-09-30T22:23:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.943+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c128ac0 con 0x7fe68c0954c0 2023-09-30T22:23:49.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.943+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c1294a0 con 0x7fe68c0954c0 2023-09-30T22:23:49.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.944+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c1290f0 con 0x7fe68c0954c0 2023-09-30T22:23:49.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.945+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe688024070 con 0x7fe68c0954c0 2023-09-30T22:23:49.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.945+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fe688040520 con 0x7fe68c0954c0 2023-09-30T22:23:49.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.945+0000 7fe690add700 1 -- 172.21.15.32:0/623561760 >> v1:172.21.15.32:6816/118188 conn(0x7fe67002c1b0 legacy=0x7fe67002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.954+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fe688025db0 con 0x7fe68c0954c0 2023-09-30T22:23:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.958+0000 7fee651b9700 1 Processor -- start 2023-09-30T22:23:49.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.958+0000 7fee651b9700 1 -- start start 2023-09-30T22:23:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.959+0000 7fee651b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.959+0000 7fee651b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009e000 0x7fee60130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.959+0000 7fee651b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 0x7fee60130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.959+0000 7fee651b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6009f520 con 0x7fee6009d6f0 2023-09-30T22:23:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.959+0000 7fee651b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6009f130 con 0x7fee6009e000 2023-09-30T22:23:49.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.959+0000 7fee651b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6009e930 con 0x7fee6009e410 2023-09-30T22:23:49.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 0x7fee60130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 0x7fee60130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34274/0 (socket says 172.21.15.32:34274) 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 -- 172.21.15.32:0/3856191749 learned_addr learned my addr 172.21.15.32:0/3856191749 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5ed9d700 1 --2- 172.21.15.32:0/3856191749 >> v2:172.21.15.32:3300/0 conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 -- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009e000 msgr2=0x7fee60130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 --2- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009e000 0x7fee60130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 -- 172.21.15.32:0/3856191749 >> v2:172.21.15.32:3300/0 conn(0x7fee6009d6f0 msgr2=0x7fee6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 --2- 172.21.15.32:0/3856191749 >> v2:172.21.15.32:3300/0 conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 -- 172.21.15.32:0/3856191749 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6009b2d0 con 0x7fee6009e410 2023-09-30T22:23:49.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.961+0000 7fee5f59e700 1 --2- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 0x7fee60130ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fee5800d180 tx=0x7fee5800d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5937ba3f4ea7771 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.962+0000 7fee5dd9b700 1 -- 172.21.15.32:0/3856191749 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58010070 con 0x7fee6009e410 2023-09-30T22:23:49.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.962+0000 7fee5dd9b700 1 -- 172.21.15.32:0/3856191749 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800dd00 con 0x7fee6009e410 2023-09-30T22:23:49.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.962+0000 7fee5dd9b700 1 -- 172.21.15.32:0/3856191749 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800b310 con 0x7fee6009e410 2023-09-30T22:23:49.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.962+0000 7fee651b9700 1 -- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 msgr2=0x7fee60130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.962+0000 7fee651b9700 1 --2- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 0x7fee60130ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fee5800d180 tx=0x7fee5800d490 comp rx=0 tx=0).stop 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 -- 172.21.15.32:0/3856191749 shutdown_connections 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 --2- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e410 0x7fee60130ba0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 --2- 172.21.15.32:0/3856191749 >> v2:172.21.15.32:3300/0 conn(0x7fee6009d6f0 0x7fee6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 --2- 172.21.15.32:0/3856191749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009e000 0x7fee60130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 -- 172.21.15.32:0/3856191749 >> 172.21.15.32:0/3856191749 conn(0x7fee60066f50 msgr2=0x7fee6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 -- 172.21.15.32:0/3856191749 shutdown_connections 2023-09-30T22:23:49.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.963+0000 7fee651b9700 1 -- 172.21.15.32:0/3856191749 wait complete. 2023-09-30T22:23:49.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.964+0000 7fee651b9700 1 Processor -- start 2023-09-30T22:23:49.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 -- start start 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 0x7fee6011de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e000 0x7fee6011e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6009e410 0x7fee6011e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee60120070 con 0x7fee6009e410 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee60121930 con 0x7fee6009d6f0 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee651b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee60120d00 con 0x7fee6009e000 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee5ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 0x7fee6011de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee5ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 0x7fee6011de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59300/0 (socket says 172.21.15.32:59300) 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.965+0000 7fee5ed9d700 1 -- 172.21.15.32:0/1893457459 learned_addr learned my addr 172.21.15.32:0/1893457459 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:49.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5f59e700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6009e410 0x7fee6011e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5ed9d700 1 -- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e000 msgr2=0x7fee6011e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5ed9d700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e000 0x7fee6011e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5ed9d700 1 -- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6009e410 msgr2=0x7fee6011e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5ed9d700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6009e410 0x7fee6011e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5ed9d700 1 -- 172.21.15.32:0/1893457459 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5800bd40 con 0x7fee6009d6f0 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5f59e700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6009e410 0x7fee6011e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:49.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee5ed9d700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 0x7fee6011de10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee5000e9b0 tx=0x7fee5000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000c850 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000c9b0 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.966+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000cd20 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.967+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6011f250 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.967+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6011fa20 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.968+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fee50013470 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.968+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6011ac20 con 0x7fee6009d6f0 2023-09-30T22:23:49.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.968+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fee5003bab0 con 0x7fee6009d6f0 2023-09-30T22:23:49.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.968+0000 7fee5e59c700 1 -- 172.21.15.32:0/1893457459 >> v1:172.21.15.32:6816/118188 conn(0x7fee4c02c200 legacy=0x7fee4c02e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:49.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:49.973+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fee50017080 con 0x7fee6009d6f0 2023-09-30T22:23:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.079+0000 7fcc84bd6700 1 -- 172.21.15.32:0/1795744193 >> v1:172.21.15.32:6816/118188 conn(0x7fcc6402c200 legacy=0x7fcc6402e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:50.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.109+0000 7f169b7fe700 1 -- 172.21.15.32:0/2425747308 >> v1:172.21.15.32:6816/118188 conn(0x7f168802c1b0 legacy=0x7f168802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:50.122 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.122+0000 7f4e8b1c9240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T22:23:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.146+0000 7fe690add700 1 -- 172.21.15.32:0/623561760 >> v1:172.21.15.32:6816/118188 conn(0x7fe67002c1b0 legacy=0x7fe67002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.154+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe68c128450 con 0x7fe68c0954c0 2023-09-30T22:23:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.155+0000 7fe681ffb700 1 -- 172.21.15.32:0/623561760 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe68802b020 con 0x7fe68c0954c0 2023-09-30T22:23:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:50.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.157+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 >> v1:172.21.15.32:6816/118188 conn(0x7fe67002c1b0 legacy=0x7fe67002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.157+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 msgr2=0x7fe68c127240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.157+0000 7fe693542700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c127240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe688009fd0 tx=0x7fe688004750 comp rx=0 tx=0).stop 2023-09-30T22:23:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 shutdown_connections 2023-09-30T22:23:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe68c09a1c0 0x7fe68c127cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe68c0954c0 0x7fe68c127240 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 --2- 172.21.15.32:0/623561760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe68c099680 0x7fe68c127780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 >> 172.21.15.32:0/623561760 conn(0x7fe68c0a2360 msgr2=0x7fe68c130b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 shutdown_connections 2023-09-30T22:23:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fe693542700 1 -- 172.21.15.32:0/623561760 wait complete. 2023-09-30T22:23:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.158+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fee60074ea0 con 0x7fee6009d6f0 2023-09-30T22:23:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.159+0000 7fee477fe700 1 -- 172.21.15.32:0/1893457459 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee50021030 con 0x7fee6009d6f0 2023-09-30T22:23:50.160 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:50.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.163+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 >> v1:172.21.15.32:6816/118188 conn(0x7fee4c02c200 legacy=0x7fee4c02e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.163+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 msgr2=0x7fee6011de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.163+0000 7fee651b9700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 0x7fee6011de10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee5000e9b0 tx=0x7fee5000ed70 comp rx=0 tx=0).stop 2023-09-30T22:23:50.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 shutdown_connections 2023-09-30T22:23:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6009e410 0x7fee6011e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6009d6f0 0x7fee6011de10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 --2- 172.21.15.32:0/1893457459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6009e000 0x7fee6011e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 >> 172.21.15.32:0/1893457459 conn(0x7fee60066f50 msgr2=0x7fee601309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 shutdown_connections 2023-09-30T22:23:50.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.164+0000 7fee651b9700 1 -- 172.21.15.32:0/1893457459 wait complete. 2023-09-30T22:23:50.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.2 2023-09-30T22:23:50.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:23:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.209+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc80125da0 con 0x7fcc8009d6f0 2023-09-30T22:23:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.209+0000 7fcc75ffb700 1 -- 172.21.15.32:0/1795744193 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcc80125da0 con 0x7fcc8009d6f0 2023-09-30T22:23:50.210 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 >> v1:172.21.15.32:6816/118188 conn(0x7fcc6402c200 legacy=0x7fcc6402e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 msgr2=0x7fcc8011f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8011f1f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009130 tx=0x7fcc7c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 shutdown_connections 2023-09-30T22:23:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8009e410 0x7fcc8011fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc8009d6f0 0x7fcc8011f1f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 --2- 172.21.15.32:0/1795744193 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc8009e000 0x7fcc8011f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 >> 172.21.15.32:0/1795744193 conn(0x7fcc80066f50 msgr2=0x7fcc80070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 shutdown_connections 2023-09-30T22:23:50.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.211+0000 7fcc8763b700 1 -- 172.21.15.32:0/1795744193 wait complete. 2023-09-30T22:23:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.220+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f169c1352d0 con 0x7f169c135c20 2023-09-30T22:23:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.221+0000 7f1683fff700 1 -- 172.21.15.32:0/2425747308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1694020400 con 0x7f169c135c20 2023-09-30T22:23:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 >> v1:172.21.15.32:6816/118188 conn(0x7f168802c1b0 legacy=0x7f168802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169c135c20 msgr2=0x7f169c1c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169c135c20 0x7f169c1c8d20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1694019000 tx=0x7f1694004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 shutdown_connections 2023-09-30T22:23:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f169c135c20 0x7f169c1c8d20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f169c06f5d0 0x7f169c1c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 --2- 172.21.15.32:0/2425747308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f169c12cf60 0x7f169c1c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 >> 172.21.15.32:0/2425747308 conn(0x7f169c0a2aa0 msgr2=0x7f169c12f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 shutdown_connections 2023-09-30T22:23:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:50.224+0000 7f16a1de7700 1 -- 172.21.15.32:0/2425747308 wait complete. 2023-09-30T22:23:50.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:23:50.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.0 2023-09-30T22:23:50.288 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.287+0000 7f4e8b1c9240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T22:23:50.547 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.370+0000 7f4e8b1c9240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T22:23:50.547 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.523+0000 7f4e8b1c9240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T22:23:50.602 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.602+0000 7f4e8b1c9240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T22:23:50.914 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.784+0000 7f4e8b1c9240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T22:23:50.914 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.873+0000 7f4e8b1c9240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T22:23:50.961 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:50.960+0000 7f4e8b1c9240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T22:23:51.066 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:51.062+0000 7f4e8b1c9240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T22:23:51.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:51.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:51.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:51.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:51.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.333+0000 7f6044b48700 1 Processor -- start 2023-09-30T22:23:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.333+0000 7f6044b48700 1 -- start start 2023-09-30T22:23:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f6044b48700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f604006e570 0x7f604006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f6044b48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604006ee80 0x7f6040134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f6044b48700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6040070ee0 con 0x7f604009eca0 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f6044b48700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6040071480 con 0x7f604006e570 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f6044b48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6040071b90 con 0x7f604006ee80 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f603dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f604009eca0 0x7f6040134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50524/0 (socket says 172.21.15.32:50524) 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f603dd9b700 1 -- 172.21.15.32:0/2703965778 learned_addr learned my addr 172.21.15.32:0/2703965778 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f603ed9d700 1 --2- 172.21.15.32:0/2703965778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604006ee80 0x7f6040134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f603e59c700 1 --2- 172.21.15.32:0/2703965778 >> v2:172.21.15.32:3300/0 conn(0x7f604006e570 0x7f604006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f603d59a700 1 -- 172.21.15.32:0/2703965778 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288240837 0 0) 0x7f6040070ee0 con 0x7f604009eca0 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.334+0000 7f603d59a700 1 -- 172.21.15.32:0/2703965778 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60280035b0 con 0x7f604009eca0 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603ed9d700 1 -- 172.21.15.32:0/2703965778 >> v2:172.21.15.32:3300/0 conn(0x7f604006e570 msgr2=0x7f604006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603ed9d700 1 --2- 172.21.15.32:0/2703965778 >> v2:172.21.15.32:3300/0 conn(0x7f604006e570 0x7f604006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603ed9d700 1 -- 172.21.15.32:0/2703965778 >> v1:172.21.15.32:6789/0 conn(0x7f604009eca0 legacy=0x7f6040134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603ed9d700 1 -- 172.21.15.32:0/2703965778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400711a0 con 0x7f604006ee80 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603ed9d700 1 --2- 172.21.15.32:0/2703965778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604006ee80 0x7f6040134d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6038009d90 tx=0x7f603800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a40548901ff515 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603d59a700 1 -- 172.21.15.32:0/2703965778 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603800e070 con 0x7f604006ee80 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603d59a700 1 -- 172.21.15.32:0/2703965778 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603800bb20 con 0x7f604006ee80 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.335+0000 7f603d59a700 1 -- 172.21.15.32:0/2703965778 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038012430 con 0x7f604006ee80 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 -- 172.21.15.32:0/2703965778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604006ee80 msgr2=0x7f6040134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 --2- 172.21.15.32:0/2703965778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604006ee80 0x7f6040134d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6038009d90 tx=0x7f603800b330 comp rx=0 tx=0).stop 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 -- 172.21.15.32:0/2703965778 shutdown_connections 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 --2- 172.21.15.32:0/2703965778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604006ee80 0x7f6040134d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 --2- 172.21.15.32:0/2703965778 >> v2:172.21.15.32:3300/0 conn(0x7f604006e570 0x7f604006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 -- 172.21.15.32:0/2703965778 >> 172.21.15.32:0/2703965778 conn(0x7f60400a2aa0 msgr2=0x7f60400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 -- 172.21.15.32:0/2703965778 shutdown_connections 2023-09-30T22:23:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.336+0000 7f6044b48700 1 -- 172.21.15.32:0/2703965778 wait complete. 2023-09-30T22:23:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.338+0000 7f6044b48700 1 Processor -- start 2023-09-30T22:23:51.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.338+0000 7f6044b48700 1 -- start start 2023-09-30T22:23:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f6044b48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604006e570 0x7f604013beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f6044b48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 0x7f6040125fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f6044b48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604013edf0 0x7f60401264e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f6044b48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604013e710 con 0x7f604006ee80 2023-09-30T22:23:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f6044b48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6040127780 con 0x7f604006e570 2023-09-30T22:23:51.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f6044b48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f604013eb30 con 0x7f604013edf0 2023-09-30T22:23:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f603dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 0x7f6040125fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f603dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 0x7f6040125fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46240/0 (socket says 172.21.15.32:46240) 2023-09-30T22:23:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f603dd9b700 1 -- 172.21.15.32:0/1815034103 learned_addr learned my addr 172.21.15.32:0/1815034103 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f603ed9d700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604013edf0 0x7f60401264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.339+0000 7f603e59c700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604006e570 0x7f604013beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f603dd9b700 1 -- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604013edf0 msgr2=0x7f60401264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f603dd9b700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604013edf0 0x7f60401264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f603dd9b700 1 -- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604006e570 msgr2=0x7f604013beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f603dd9b700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604006e570 0x7f604013beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f603dd9b700 1 -- 172.21.15.32:0/1815034103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038009a70 con 0x7f604006ee80 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f603dd9b700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 0x7f6040125fa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f603400c4f0 tx=0x7f603400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034005d70 con 0x7f604006ee80 2023-09-30T22:23:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60340070d0 con 0x7f604006ee80 2023-09-30T22:23:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6040138680 con 0x7f604006ee80 2023-09-30T22:23:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604013d1d0 con 0x7f604006ee80 2023-09-30T22:23:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.340+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034007440 con 0x7f604006ee80 2023-09-30T22:23:51.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.342+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60240052e0 con 0x7f604006ee80 2023-09-30T22:23:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.342+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f60340075a0 con 0x7f604006ee80 2023-09-30T22:23:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.342+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f603403b160 con 0x7f604006ee80 2023-09-30T22:23:51.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.342+0000 7f603e59c700 1 -- 172.21.15.32:0/1815034103 >> v1:172.21.15.32:6816/118188 conn(0x7f602002c200 legacy=0x7f602002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.347+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f6034019b20 con 0x7f604006ee80 2023-09-30T22:23:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.379+0000 7f55c247f700 1 Processor -- start 2023-09-30T22:23:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.379+0000 7f55c247f700 1 -- start start 2023-09-30T22:23:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.379+0000 7f55c247f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.379+0000 7f55c247f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55bc09eca0 0x7f55bc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55c247f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc070ee0 con 0x7f55bc134610 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55c247f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc071480 con 0x7f55bc09eca0 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55c247f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc06f3c0 con 0x7f55bc06e570 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41592/0 (socket says 172.21.15.32:41592) 2023-09-30T22:23:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 -- 172.21.15.32:0/312571628 learned_addr learned my addr 172.21.15.32:0/312571628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 -- 172.21.15.32:0/312571628 >> v2:172.21.15.32:3300/0 conn(0x7f55bc09eca0 msgr2=0x7f55bc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 --2- 172.21.15.32:0/312571628 >> v2:172.21.15.32:3300/0 conn(0x7f55bc09eca0 0x7f55bc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 -- 172.21.15.32:0/312571628 >> v1:172.21.15.32:6789/0 conn(0x7f55bc134610 legacy=0x7f55bc134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 -- 172.21.15.32:0/312571628 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc071b90 con 0x7f55bc06e570 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.380+0000 7f55bbfff700 1 --2- 172.21.15.32:0/312571628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc06e940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098f0 tx=0x7f55ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ae252b775d4e1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.381+0000 7f55baffd700 1 -- 172.21.15.32:0/312571628 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac004230 con 0x7f55bc06e570 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.381+0000 7f55baffd700 1 -- 172.21.15.32:0/312571628 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac004390 con 0x7f55bc06e570 2023-09-30T22:23:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.381+0000 7f55baffd700 1 -- 172.21.15.32:0/312571628 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0036a0 con 0x7f55bc06e570 2023-09-30T22:23:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.381+0000 7f55c247f700 1 -- 172.21.15.32:0/312571628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 msgr2=0x7f55bc06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.381+0000 7f55c247f700 1 --2- 172.21.15.32:0/312571628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc06e940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0098f0 tx=0x7f55ac009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:51.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.382+0000 7f55c247f700 1 -- 172.21.15.32:0/312571628 shutdown_connections 2023-09-30T22:23:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.382+0000 7f55c247f700 1 --2- 172.21.15.32:0/312571628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc06e940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.382+0000 7f55c247f700 1 --2- 172.21.15.32:0/312571628 >> v2:172.21.15.32:3300/0 conn(0x7f55bc09eca0 0x7f55bc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.382+0000 7f55c247f700 1 -- 172.21.15.32:0/312571628 >> 172.21.15.32:0/312571628 conn(0x7f55bc0a2aa0 msgr2=0x7f55bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.382+0000 7f55c247f700 1 -- 172.21.15.32:0/312571628 shutdown_connections 2023-09-30T22:23:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.382+0000 7f55c247f700 1 -- 172.21.15.32:0/312571628 wait complete. 2023-09-30T22:23:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.383+0000 7f55c247f700 1 Processor -- start 2023-09-30T22:23:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.383+0000 7f55c247f700 1 -- start start 2023-09-30T22:23:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.383+0000 7f55c247f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.383+0000 7f55c247f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc134610 0x7f55bc133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c247f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 0x7f55bc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c247f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1344b0 con 0x7f55bc134610 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c247f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc13e7e0 con 0x7f55bc1335c0 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c247f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1408a0 con 0x7f55bc06e570 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 0x7f55bc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55bb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc134610 0x7f55bc133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 0x7f55bc133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52060/0 (socket says 172.21.15.32:52060) 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 -- 172.21.15.32:0/1836948029 learned_addr learned my addr 172.21.15.32:0/1836948029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55bbfff700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 -- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 msgr2=0x7f55bc140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 -- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc134610 msgr2=0x7f55bc133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc134610 0x7f55bc133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.384+0000 7f55c0a1c700 1 -- 172.21.15.32:0/1836948029 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac009550 con 0x7f55bc1335c0 2023-09-30T22:23:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.385+0000 7f55c0a1c700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 0x7f55bc133a60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c7a0 tx=0x7f55b400cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.386+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4003e10 con 0x7f55bc1335c0 2023-09-30T22:23:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.386+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc13d220 con 0x7f55bc1335c0 2023-09-30T22:23:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.386+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc13cd60 con 0x7f55bc1335c0 2023-09-30T22:23:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.387+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b4005070 con 0x7f55bc1335c0 2023-09-30T22:23:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.387+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4005420 con 0x7f55bc1335c0 2023-09-30T22:23:51.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.388+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc13dab0 con 0x7f55bc1335c0 2023-09-30T22:23:51.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.388+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f55b4005580 con 0x7f55bc1335c0 2023-09-30T22:23:51.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.388+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f55b403af80 con 0x7f55bc1335c0 2023-09-30T22:23:51.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.388+0000 7f55bbfff700 1 -- 172.21.15.32:0/1836948029 >> v1:172.21.15.32:6816/118188 conn(0x7f55a802c1b0 legacy=0x7f55a802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.397+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f55b4019cd0 con 0x7f55bc1335c0 2023-09-30T22:23:51.481 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:51.410+0000 7f4e8b1c9240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T22:23:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.421+0000 7f23029af700 1 Processor -- start 2023-09-30T22:23:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.421+0000 7f23029af700 1 -- start start 2023-09-30T22:23:51.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.422+0000 7f23029af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12cf60 0x7f22fc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.422+0000 7f23029af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 0x7f22fc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.422+0000 7f23029af700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22fc130260 0x7f22fc136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.422+0000 7f23029af700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc12d330 con 0x7f22fc130260 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.422+0000 7f23029af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc12cd90 con 0x7f22fc12cf60 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.422+0000 7f23029af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1306d0 con 0x7f22fc12f8b0 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 0x7f22fc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 0x7f22fc12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41614/0 (socket says 172.21.15.32:41614) 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 -- 172.21.15.32:0/2134135072 learned_addr learned my addr 172.21.15.32:0/2134135072 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f2300f4c700 1 --2- 172.21.15.32:0/2134135072 >> v2:172.21.15.32:3300/0 conn(0x7f22fc130260 0x7f22fc136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 -- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12cf60 msgr2=0x7f22fc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12cf60 0x7f22fc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 -- 172.21.15.32:0/2134135072 >> v2:172.21.15.32:3300/0 conn(0x7f22fc130260 msgr2=0x7f22fc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/2134135072 >> v2:172.21.15.32:3300/0 conn(0x7f22fc130260 0x7f22fc136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 -- 172.21.15.32:0/2134135072 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc071780 con 0x7f22fc12f8b0 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.423+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 0x7f22fc12fd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22f000bb90 tx=0x7f22f000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26a8b34580b8300 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.424+0000 7f22faffd700 1 -- 172.21.15.32:0/2134135072 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f000dc20 con 0x7f22fc12f8b0 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.424+0000 7f22faffd700 1 -- 172.21.15.32:0/2134135072 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f000dd80 con 0x7f22fc12f8b0 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.424+0000 7f22faffd700 1 -- 172.21.15.32:0/2134135072 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0017660 con 0x7f22fc12f8b0 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 -- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 msgr2=0x7f22fc12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 --2- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 0x7f22fc12fd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22f000bb90 tx=0x7f22f000bea0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 -- 172.21.15.32:0/2134135072 shutdown_connections 2023-09-30T22:23:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 --2- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12cf60 0x7f22fc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 --2- 172.21.15.32:0/2134135072 >> v2:172.21.15.32:3300/0 conn(0x7f22fc130260 0x7f22fc136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 --2- 172.21.15.32:0/2134135072 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc12f8b0 0x7f22fc12fd20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 -- 172.21.15.32:0/2134135072 >> 172.21.15.32:0/2134135072 conn(0x7f22fc0a2aa0 msgr2=0x7f22fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 -- 172.21.15.32:0/2134135072 shutdown_connections 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.425+0000 7f23029af700 1 -- 172.21.15.32:0/2134135072 wait complete. 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.426+0000 7f23029af700 1 Processor -- start 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 -- start start 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc12cf60 0x7f22fc1c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12f8b0 0x7f22fc1c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 0x7f22fc1c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc09ead0 con 0x7f22fc12cf60 2023-09-30T22:23:51.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc129fa0 con 0x7f22fc12f8b0 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f23029af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc09cdc0 con 0x7f22fc130260 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f2300f4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 0x7f22fc1c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f2300f4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 0x7f22fc1c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41616/0 (socket says 172.21.15.32:41616) 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f2300f4c700 1 -- 172.21.15.32:0/2951978044 learned_addr learned my addr 172.21.15.32:0/2951978044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.427+0000 7f22fbfff700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc12cf60 0x7f22fc1c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12f8b0 0x7f22fc1c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f2300f4c700 1 -- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12f8b0 msgr2=0x7f22fc1c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f2300f4c700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12f8b0 0x7f22fc1c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f2300f4c700 1 -- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc12cf60 msgr2=0x7f22fc1c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f2300f4c700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc12cf60 0x7f22fc1c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f2300f4c700 1 -- 172.21.15.32:0/2951978044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f000b7f0 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f22fb7fe700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12f8b0 0x7f22fc1c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f2300f4c700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 0x7f22fc1c4aa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009e30 tx=0x7f22f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f22fbfff700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc12cf60 0x7f22fc1c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400ce00 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400f040 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc12f590 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40146c0 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.428+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc09d5b0 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.429+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f22f4014820 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.430+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc09e780 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.430+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f22f4014e00 con 0x7f22fc130260 2023-09-30T22:23:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.431+0000 7f22fbfff700 1 -- 172.21.15.32:0/2951978044 >> v1:172.21.15.32:6816/118188 conn(0x7f22e802c1b0 legacy=0x7f22e802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.434+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f22f4041060 con 0x7f22fc130260 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.488+0000 7fbf6a664700 1 Processor -- start 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.488+0000 7fbf6a664700 1 -- start start 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.489+0000 7fbf6a664700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf6409d6f0 0x7fbf6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.489+0000 7fbf6a664700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6409e410 0x7fbf64130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.489+0000 7fbf6a664700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6409f130 con 0x7fbf6409e000 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.489+0000 7fbf6a664700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6409f520 con 0x7fbf6409d6f0 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.489+0000 7fbf6a664700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6409e930 con 0x7fbf6409e410 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf637fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf6409e000 0x7fbf64130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50532/0 (socket says 172.21.15.32:50532) 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf637fe700 1 -- 172.21.15.32:0/3814853445 learned_addr learned my addr 172.21.15.32:0/3814853445 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf68c01700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6409e410 0x7fbf64130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf63fff700 1 --2- 172.21.15.32:0/3814853445 >> v2:172.21.15.32:3300/0 conn(0x7fbf6409d6f0 0x7fbf6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111765541 0 0) 0x7fbf6409f130 con 0x7fbf6409e000 2023-09-30T22:23:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf4c0035b0 con 0x7fbf6409e000 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31383166 0 0) 0x7fbf4c0035b0 con 0x7fbf6409e000 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6409e410 msgr2=0x7fbf64130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6409e410 0x7fbf64130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 >> v2:172.21.15.32:3300/0 conn(0x7fbf6409d6f0 msgr2=0x7fbf6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 --2- 172.21.15.32:0/3814853445 >> v2:172.21.15.32:3300/0 conn(0x7fbf6409d6f0 0x7fbf6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6409b2d0 con 0x7fbf6409e000 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1846968574 0 0) 0x7fbf58001670 con 0x7fbf6409e000 2023-09-30T22:23:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.490+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 >> v1:172.21.15.32:6789/0 conn(0x7fbf6409e000 legacy=0x7fbf64130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf62ffd700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c003a00 0x7fbf4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf62ffd700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf4c0043e0 0x7fbf4c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf62ffd700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf4c004ff0 0x7fbf4c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6409e930 con 0x7fbf4c003a00 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c00e160 con 0x7fbf4c0043e0 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c00e2a0 con 0x7fbf4c004ff0 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf4c0043e0 0x7fbf4c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 -- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf4c004ff0 msgr2=0x7fbf4c00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf4c004ff0 0x7fbf4c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 -- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c003a00 msgr2=0x7fbf4c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c003a00 0x7fbf4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 -- 172.21.15.32:0/3814853445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0035b0 con 0x7fbf4c0043e0 2023-09-30T22:23:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.491+0000 7fbf637fe700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf4c0043e0 0x7fbf4c0048b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c2f0 tx=0x7fbf4c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b4427a5a23a3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.492+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800ce30 con 0x7fbf4c0043e0 2023-09-30T22:23:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.492+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58005960 con 0x7fbf4c0043e0 2023-09-30T22:23:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.492+0000 7fbf62ffd700 1 -- 172.21.15.32:0/3814853445 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58005d10 con 0x7fbf4c0043e0 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf6a664700 1 -- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf4c0043e0 msgr2=0x7fbf4c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf6a664700 1 --2- 172.21.15.32:0/3814853445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf4c0043e0 0x7fbf4c0048b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c2f0 tx=0x7fbf4c005880 comp rx=0 tx=0).stop 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf68c01700 1 -- 172.21.15.32:0/3814853445 reap_dead start 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf6a664700 1 -- 172.21.15.32:0/3814853445 shutdown_connections 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf6a664700 1 -- 172.21.15.32:0/3814853445 >> 172.21.15.32:0/3814853445 conn(0x7fbf64066f50 msgr2=0x7fbf6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf6a664700 1 -- 172.21.15.32:0/3814853445 shutdown_connections 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.493+0000 7fbf6a664700 1 -- 172.21.15.32:0/3814853445 wait complete. 2023-09-30T22:23:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.495+0000 7fbf6a664700 1 Processor -- start 2023-09-30T22:23:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.495+0000 7fbf6a664700 1 -- start start 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf6a664700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c0043e0 0x7fbf6411e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf6a664700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 0x7fbf6411ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf6a664700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf641c3f00 0x7fbf641c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf6a664700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6411f780 con 0x7fbf4c0043e0 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf6a664700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf64120500 con 0x7fbf641c3f00 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf6a664700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6411f140 con 0x7fbf6411e810 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 0x7fbf6411ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 0x7fbf6411ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41626/0 (socket says 172.21.15.32:41626) 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf637fe700 1 -- 172.21.15.32:0/2987611239 learned_addr learned my addr 172.21.15.32:0/2987611239 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.496+0000 7fbf63fff700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c0043e0 0x7fbf6411e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf637fe700 1 -- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf641c3f00 msgr2=0x7fbf641c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf637fe700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf641c3f00 0x7fbf641c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf637fe700 1 -- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c0043e0 msgr2=0x7fbf6411e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf637fe700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c0043e0 0x7fbf6411e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf637fe700 1 -- 172.21.15.32:0/2987611239 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf64130230 con 0x7fbf6411e810 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf63fff700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c0043e0 0x7fbf6411e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf637fe700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 0x7fbf6411ec00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005800 tx=0x7fbf6409d470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58001660 con 0x7fbf6411e810 2023-09-30T22:23:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58004a80 con 0x7fbf6411e810 2023-09-30T22:23:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004e30 con 0x7fbf6411e810 2023-09-30T22:23:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf64121df0 con 0x7fbf6411e810 2023-09-30T22:23:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.497+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6409c700 con 0x7fbf6411e810 2023-09-30T22:23:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.500+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fbf580070d0 con 0x7fbf6411e810 2023-09-30T22:23:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.500+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fbf5803ce90 con 0x7fbf6411e810 2023-09-30T22:23:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.500+0000 7fbf63fff700 1 -- 172.21.15.32:0/2987611239 >> v1:172.21.15.32:6816/118188 conn(0x7fbf500305c0 legacy=0x7fbf50032a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.500+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf64074ea0 con 0x7fbf6411e810 2023-09-30T22:23:51.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.509+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fbf58063fe0 con 0x7fbf6411e810 2023-09-30T22:23:51.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.543+0000 7f603e59c700 1 -- 172.21.15.32:0/1815034103 >> v1:172.21.15.32:6816/118188 conn(0x7f602002c200 legacy=0x7f602002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.544 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:51.544+0000 7f4e8b1c9240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T22:23:51.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.584+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6024005860 con 0x7f604006ee80 2023-09-30T22:23:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.584+0000 7f602effd700 1 -- 172.21.15.32:0/1815034103 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6034019730 con 0x7f604006ee80 2023-09-30T22:23:51.585 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.587+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 >> v1:172.21.15.32:6816/118188 conn(0x7f602002c200 legacy=0x7f602002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.587+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 msgr2=0x7f6040125fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.587+0000 7f6044b48700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 0x7f6040125fa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f603400c4f0 tx=0x7f603400c800 comp rx=0 tx=0).stop 2023-09-30T22:23:51.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 shutdown_connections 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604013edf0 0x7f60401264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604006e570 0x7f604013beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 --2- 172.21.15.32:0/1815034103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604006ee80 0x7f6040125fa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 >> 172.21.15.32:0/1815034103 conn(0x7f60400a2aa0 msgr2=0x7f60400a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 shutdown_connections 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.588+0000 7f6044b48700 1 -- 172.21.15.32:0/1815034103 wait complete. 2023-09-30T22:23:51.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.589+0000 7f55bbfff700 1 -- 172.21.15.32:0/1836948029 >> v1:172.21.15.32:6816/118188 conn(0x7f55a802c1b0 legacy=0x7f55a802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.603+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55bc074ea0 con 0x7f55bc1335c0 2023-09-30T22:23:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.603+0000 7f55b8ff9700 1 -- 172.21.15.32:0/1836948029 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55b4017020 con 0x7f55bc1335c0 2023-09-30T22:23:51.604 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:51.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.2 2023-09-30T22:23:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 >> v1:172.21.15.32:6816/118188 conn(0x7f55a802c1b0 legacy=0x7f55a802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 msgr2=0x7f55bc133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 0x7f55bc133a60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55b400c7a0 tx=0x7f55b400cab0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 shutdown_connections 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc1335c0 0x7f55bc133a60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc06e570 0x7f55bc140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 --2- 172.21.15.32:0/1836948029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc134610 0x7f55bc133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.606+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 >> 172.21.15.32:0/1836948029 conn(0x7f55bc0a2aa0 msgr2=0x7f55bc137fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.607+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 shutdown_connections 2023-09-30T22:23:51.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.607+0000 7f55c247f700 1 -- 172.21.15.32:0/1836948029 wait complete. 2023-09-30T22:23:51.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:23:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.632+0000 7f22fbfff700 1 -- 172.21.15.32:0/2951978044 >> v1:172.21.15.32:6816/118188 conn(0x7f22e802c1b0 legacy=0x7f22e802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.632+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22fc074ea0 con 0x7f22fc130260 2023-09-30T22:23:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.632+0000 7f22f8ff9700 1 -- 172.21.15.32:0/2951978044 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22f401d400 con 0x7f22fc130260 2023-09-30T22:23:51.633 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 >> v1:172.21.15.32:6816/118188 conn(0x7f22e802c1b0 legacy=0x7f22e802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 msgr2=0x7f22fc1c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 0x7f22fc1c4aa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009e30 tx=0x7f22f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 shutdown_connections 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22fc130260 0x7f22fc1c4aa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22fc12cf60 0x7f22fc1c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 --2- 172.21.15.32:0/2951978044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22fc12f8b0 0x7f22fc1c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 >> 172.21.15.32:0/2951978044 conn(0x7f22fc0a2aa0 msgr2=0x7f22fc06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 shutdown_connections 2023-09-30T22:23:51.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.636+0000 7f23029af700 1 -- 172.21.15.32:0/2951978044 wait complete. 2023-09-30T22:23:51.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:23:51.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.701+0000 7fbf63fff700 1 -- 172.21.15.32:0/2987611239 >> v1:172.21.15.32:6816/118188 conn(0x7fbf500305c0 legacy=0x7fbf50032a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:51.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.736+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbf6406c360 con 0x7fbf6411e810 2023-09-30T22:23:51.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.736+0000 7fbf60ff9700 1 -- 172.21.15.32:0/2987611239 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf58017020 con 0x7fbf6411e810 2023-09-30T22:23:51.737 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:51.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 >> v1:172.21.15.32:6816/118188 conn(0x7fbf500305c0 legacy=0x7fbf50032a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:51.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 msgr2=0x7fbf6411ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 0x7fbf6411ec00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005800 tx=0x7fbf6409d470 comp rx=0 tx=0).stop 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 shutdown_connections 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf641c3f00 0x7fbf641c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf4c0043e0 0x7fbf6411e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 --2- 172.21.15.32:0/2987611239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf6411e810 0x7fbf6411ec00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.738+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 >> 172.21.15.32:0/2987611239 conn(0x7fbf64066f50 msgr2=0x7fbf640a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.739+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 shutdown_connections 2023-09-30T22:23:51.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:51.739+0000 7fbf6a664700 1 -- 172.21.15.32:0/2987611239 wait complete. 2023-09-30T22:23:51.740 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:51.739+0000 7f4e8b1c9240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T22:23:51.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.0 2023-09-30T22:23:51.821 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:51.821+0000 7f4e8b1c9240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T22:23:52.552 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:52.551+0000 7f4e8b1c9240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T22:23:52.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:52.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:52.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:52.696 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:52.695+0000 7f4e8b1c9240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T22:23:52.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.751+0000 7f5dc8c5f700 1 Processor -- start 2023-09-30T22:23:52.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.751+0000 7f5dc8c5f700 1 -- start start 2023-09-30T22:23:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc8c5f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dc41316c0 0x7f5dc4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc8c5f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc406ae80 0x7f5dc4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc8c5f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc409e130 con 0x7f5dc409d9f0 2023-09-30T22:23:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc8c5f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc41312b0 con 0x7f5dc41316c0 2023-09-30T22:23:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc8c5f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc409f6e0 con 0x7f5dc406ae80 2023-09-30T22:23:52.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dc41316c0 0x7f5dc4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dc41316c0 0x7f5dc4131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46290/0 (socket says 172.21.15.32:46290) 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 -- 172.21.15.32:0/2184506396 learned_addr learned my addr 172.21.15.32:0/2184506396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc2d9d700 1 --2- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc406ae80 0x7f5dc4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 -- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc406ae80 msgr2=0x7f5dc4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 --2- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc406ae80 0x7f5dc4139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 -- 172.21.15.32:0/2184506396 >> v1:172.21.15.32:6789/0 conn(0x7f5dc409d9f0 legacy=0x7f5dc406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.752+0000 7f5dc259c700 1 -- 172.21.15.32:0/2184506396 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc409e760 con 0x7f5dc41316c0 2023-09-30T22:23:52.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.753+0000 7f5dc259c700 1 --2- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc41316c0 0x7f5dc4131a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5db40098f0 tx=0x7f5db4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f853b85aa2b19d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.753+0000 7f5dc159a700 1 -- 172.21.15.32:0/2184506396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400b8a0 con 0x7f5dc41316c0 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.753+0000 7f5dc159a700 1 -- 172.21.15.32:0/2184506396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400ba00 con 0x7f5dc41316c0 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.753+0000 7f5dc159a700 1 -- 172.21.15.32:0/2184506396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400bd90 con 0x7f5dc41316c0 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc41316c0 msgr2=0x7f5dc4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc41316c0 0x7f5dc4131a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5db40098f0 tx=0x7f5db4009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/2184506396 shutdown_connections 2023-09-30T22:23:52.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc406ae80 0x7f5dc4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/2184506396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc41316c0 0x7f5dc4131a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/2184506396 >> 172.21.15.32:0/2184506396 conn(0x7f5dc4066f50 msgr2=0x7f5dc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/2184506396 shutdown_connections 2023-09-30T22:23:52.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.754+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/2184506396 wait complete. 2023-09-30T22:23:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.756+0000 7f5dc8c5f700 1 Processor -- start 2023-09-30T22:23:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.756+0000 7f5dc8c5f700 1 -- start start 2023-09-30T22:23:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc8c5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 0x7f5dc4070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc8c5f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc409d9f0 0x7f5dc4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc8c5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc41316c0 0x7f5dc4071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc8c5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc412b900 con 0x7f5dc406ae80 2023-09-30T22:23:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc8c5f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4127e70 con 0x7f5dc409d9f0 2023-09-30T22:23:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc8c5f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40a3540 con 0x7f5dc41316c0 2023-09-30T22:23:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 0x7f5dc4070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 0x7f5dc4070a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46302/0 (socket says 172.21.15.32:46302) 2023-09-30T22:23:52.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 -- 172.21.15.32:0/1450582697 learned_addr learned my addr 172.21.15.32:0/1450582697 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc2d9d700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc41316c0 0x7f5dc4071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc1d9b700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc409d9f0 0x7f5dc4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 -- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc41316c0 msgr2=0x7f5dc4071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc41316c0 0x7f5dc4071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 -- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc409d9f0 msgr2=0x7f5dc4070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc409d9f0 0x7f5dc4070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.757+0000 7f5dc259c700 1 -- 172.21.15.32:0/1450582697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009550 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.758+0000 7f5dc259c700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 0x7f5dc4070a10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5db4000c00 tx=0x7f5db4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.758+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40044d0 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.758+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc411ac20 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.758+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4023070 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.758+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db401fa80 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.758+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc411aee0 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.759+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f5db401fbe0 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.759+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f5db4025070 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.759+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc411aee0 con 0x7f5dc406ae80 2023-09-30T22:23:52.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.760+0000 7f5dc1d9b700 1 -- 172.21.15.32:0/1450582697 >> v1:172.21.15.32:6816/118188 conn(0x7f5da402c1b0 legacy=0x7f5da402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.765+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f5dc411aee0 con 0x7f5dc406ae80 2023-09-30T22:23:52.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.810+0000 7f9a3babd700 1 Processor -- start 2023-09-30T22:23:52.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.810+0000 7f9a3babd700 1 -- start start 2023-09-30T22:23:52.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a3babd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a34131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a3babd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a3409a580 0x7f9a34139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a3babd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a341313c0 con 0x7f9a34099a40 2023-09-30T22:23:52.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a3babd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3406e8c0 con 0x7f9a34131810 2023-09-30T22:23:52.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a3babd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34071d20 con 0x7f9a3409a580 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a39859700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a34131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a39859700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a34131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52118/0 (socket says 172.21.15.32:52118) 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a39859700 1 -- 172.21.15.32:0/3926920584 learned_addr learned my addr 172.21.15.32:0/3926920584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.811+0000 7f9a39859700 1 -- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a3409a580 msgr2=0x7f9a34139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a39859700 1 --2- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a3409a580 0x7f9a34139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a39859700 1 -- 172.21.15.32:0/3926920584 >> v1:172.21.15.32:6789/0 conn(0x7f9a34099a40 legacy=0x7f9a34099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a39859700 1 -- 172.21.15.32:0/3926920584 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340721a0 con 0x7f9a34131810 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a39859700 1 --2- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a34131be0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800f9e0 tx=0x7f9a2800fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27824ae717fe8799 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a38857700 1 -- 172.21.15.32:0/3926920584 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800f460 con 0x7f9a34131810 2023-09-30T22:23:52.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a38857700 1 -- 172.21.15.32:0/3926920584 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28009960 con 0x7f9a34131810 2023-09-30T22:23:52.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.812+0000 7f9a38857700 1 -- 172.21.15.32:0/3926920584 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28009cd0 con 0x7f9a34131810 2023-09-30T22:23:52.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.813+0000 7f9a3babd700 1 -- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 msgr2=0x7f9a34131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.813+0000 7f9a3babd700 1 --2- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a34131be0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800f9e0 tx=0x7f9a2800fcf0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.813+0000 7f9a3babd700 1 -- 172.21.15.32:0/3926920584 shutdown_connections 2023-09-30T22:23:52.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.813+0000 7f9a3babd700 1 --2- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a3409a580 0x7f9a34139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.813+0000 7f9a3babd700 1 --2- 172.21.15.32:0/3926920584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a34131be0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.813+0000 7f9a3babd700 1 -- 172.21.15.32:0/3926920584 >> 172.21.15.32:0/3926920584 conn(0x7f9a340a2aa0 msgr2=0x7f9a340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:52.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.814+0000 7f9a3babd700 1 -- 172.21.15.32:0/3926920584 shutdown_connections 2023-09-30T22:23:52.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.814+0000 7f9a3babd700 1 -- 172.21.15.32:0/3926920584 wait complete. 2023-09-30T22:23:52.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.815+0000 7f9a3babd700 1 Processor -- start 2023-09-30T22:23:52.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.815+0000 7f9a3babd700 1 -- start start 2023-09-30T22:23:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3babd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a34099a40 0x7f9a3411b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3babd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 0x7f9a3411be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3babd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a3411c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3babd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3411eee0 con 0x7f9a3409a580 2023-09-30T22:23:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3babd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3411da50 con 0x7f9a34131810 2023-09-30T22:23:52.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3babd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3411f0c0 con 0x7f9a34099a40 2023-09-30T22:23:52.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a39058700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 0x7f9a3411be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a39058700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 0x7f9a3411be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46314/0 (socket says 172.21.15.32:46314) 2023-09-30T22:23:52.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a39058700 1 -- 172.21.15.32:0/1661182144 learned_addr learned my addr 172.21.15.32:0/1661182144 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a39859700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a34099a40 0x7f9a3411b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.816+0000 7f9a3a05a700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a3411c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a39058700 1 -- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a34099a40 msgr2=0x7f9a3411b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a39058700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a34099a40 0x7f9a3411b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a39058700 1 -- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 msgr2=0x7f9a3411c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a39058700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a3411c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a39058700 1 -- 172.21.15.32:0/1661182144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2800f610 con 0x7f9a3409a580 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a39058700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 0x7f9a3411be10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c4d0 tx=0x7f9a3000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3000cea0 con 0x7f9a3409a580 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a300054e0 con 0x7f9a3409a580 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.817+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3411e3a0 con 0x7f9a3409a580 2023-09-30T22:23:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.818+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3411fb30 con 0x7f9a3409a580 2023-09-30T22:23:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.819+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a300038a0 con 0x7f9a3409a580 2023-09-30T22:23:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.819+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f9a30003a80 con 0x7f9a3409a580 2023-09-30T22:23:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.819+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f9a3003ab70 con 0x7f9a3409a580 2023-09-30T22:23:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.819+0000 7f9a39859700 1 -- 172.21.15.32:0/1661182144 >> v1:172.21.15.32:6816/118188 conn(0x7f9a1802c1b0 legacy=0x7f9a1802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.820+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c0052e0 con 0x7f9a3409a580 2023-09-30T22:23:52.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.824+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f9a30017020 con 0x7f9a3409a580 2023-09-30T22:23:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.854+0000 7f4fea5ec700 1 Processor -- start 2023-09-30T22:23:52.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.854+0000 7f4fea5ec700 1 -- start start 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fea5ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fea5ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe41341d0 0x7f4fe409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fea5ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4fe409e390 0x7f4fe409e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fea5ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe4070c80 con 0x7f4fe409e390 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fea5ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe406f720 con 0x7f4fe41338c0 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fea5ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe406fce0 con 0x7f4fe41341d0 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52128/0 (socket says 172.21.15.32:52128) 2023-09-30T22:23:52.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fe3fff700 1 -- 172.21.15.32:0/2687118855 learned_addr learned my addr 172.21.15.32:0/2687118855 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.855+0000 7f4fe37fe700 1 --2- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe41341d0 0x7f4fe409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe3fff700 1 -- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe41341d0 msgr2=0x7f4fe409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe41341d0 0x7f4fe409de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe3fff700 1 -- 172.21.15.32:0/2687118855 >> v2:172.21.15.32:3300/0 conn(0x7f4fe409e390 msgr2=0x7f4fe409e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/2687118855 >> v2:172.21.15.32:3300/0 conn(0x7f4fe409e390 0x7f4fe409e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe3fff700 1 -- 172.21.15.32:0/2687118855 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe41336a0 con 0x7f4fe41338c0 2023-09-30T22:23:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4133c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40098f0 tx=0x7f4fd4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9e91547e34a0fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe2ffd700 1 -- 172.21.15.32:0/2687118855 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4011830 con 0x7f4fe41338c0 2023-09-30T22:23:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe2ffd700 1 -- 172.21.15.32:0/2687118855 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4011990 con 0x7f4fe41338c0 2023-09-30T22:23:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.856+0000 7f4fe2ffd700 1 -- 172.21.15.32:0/2687118855 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4011d20 con 0x7f4fe41338c0 2023-09-30T22:23:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 -- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 msgr2=0x7f4fe4133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4133c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40098f0 tx=0x7f4fd4009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 -- 172.21.15.32:0/2687118855 shutdown_connections 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4133c90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/2687118855 >> v2:172.21.15.32:3300/0 conn(0x7f4fe409e390 0x7f4fe409e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/2687118855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe41341d0 0x7f4fe409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 -- 172.21.15.32:0/2687118855 >> 172.21.15.32:0/2687118855 conn(0x7f4fe40a2aa0 msgr2=0x7f4fe40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 -- 172.21.15.32:0/2687118855 shutdown_connections 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.857+0000 7f4fea5ec700 1 -- 172.21.15.32:0/2687118855 wait complete. 2023-09-30T22:23:52.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.859+0000 7f4fea5ec700 1 Processor -- start 2023-09-30T22:23:52.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.859+0000 7f4fea5ec700 1 -- start start 2023-09-30T22:23:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fea5ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe409e390 0x7f4fe41255c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fea5ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4123830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fea5ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe41341d0 0x7f4fe4123d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fea5ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41272d0 con 0x7f4fe41341d0 2023-09-30T22:23:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fea5ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4126b30 con 0x7f4fe41338c0 2023-09-30T22:23:52.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fea5ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4127d50 con 0x7f4fe409e390 2023-09-30T22:23:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4123830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4123830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52142/0 (socket says 172.21.15.32:52142) 2023-09-30T22:23:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fe37fe700 1 -- 172.21.15.32:0/1353235236 learned_addr learned my addr 172.21.15.32:0/1353235236 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe409e390 0x7f4fe41255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.860+0000 7f4fe8b89700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe41341d0 0x7f4fe4123d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe3fff700 1 -- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 msgr2=0x7f4fe4123830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4123830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe3fff700 1 -- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe41341d0 msgr2=0x7f4fe4123d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe41341d0 0x7f4fe4123d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe3fff700 1 -- 172.21.15.32:0/1353235236 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4009550 con 0x7f4fe409e390 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe3fff700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe409e390 0x7f4fe41255c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4016000 tx=0x7f4fd4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.861+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004450 con 0x7f4fe409e390 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.862+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd401f070 con 0x7f4fe409e390 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.862+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40038e0 con 0x7f4fe409e390 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.862+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe4126df0 con 0x7f4fe409e390 2023-09-30T22:23:52.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.862+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41242b0 con 0x7f4fe409e390 2023-09-30T22:23:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.863+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe4074ea0 con 0x7f4fe409e390 2023-09-30T22:23:52.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.864+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f4fd4003a40 con 0x7f4fe409e390 2023-09-30T22:23:52.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.864+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7f4fd4021070 con 0x7f4fe409e390 2023-09-30T22:23:52.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.868+0000 7f4fe37fe700 1 -- 172.21.15.32:0/1353235236 >> v1:172.21.15.32:6816/118188 conn(0x7f4fd002c200 legacy=0x7f4fd002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:52.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.868+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f4fd4026480 con 0x7f4fe409e390 2023-09-30T22:23:52.876 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:52.875+0000 7f4e8b1c9240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:23:52.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.961+0000 7f5dc1d9b700 1 -- 172.21.15.32:0/1450582697 >> v1:172.21.15.32:6816/118188 conn(0x7f5da402c1b0 legacy=0x7f5da402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.963+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5dc411aee0 con 0x7f5dc406ae80 2023-09-30T22:23:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.964+0000 7f5db2ffd700 1 -- 172.21.15.32:0/1450582697 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5dc411aee0 con 0x7f5dc406ae80 2023-09-30T22:23:52.964 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.966+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 >> v1:172.21.15.32:6816/118188 conn(0x7f5da402c1b0 legacy=0x7f5da402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:52.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.966+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 msgr2=0x7f5dc4070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.966+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 0x7f5dc4070a10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5db4000c00 tx=0x7f5db4004750 comp rx=0 tx=0).stop 2023-09-30T22:23:52.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 shutdown_connections 2023-09-30T22:23:52.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc41316c0 0x7f5dc4071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc406ae80 0x7f5dc4070a10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 --2- 172.21.15.32:0/1450582697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc409d9f0 0x7f5dc4070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 >> 172.21.15.32:0/1450582697 conn(0x7f5dc4066f50 msgr2=0x7f5dc40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 shutdown_connections 2023-09-30T22:23:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.967+0000 7f5dc8c5f700 1 -- 172.21.15.32:0/1450582697 wait complete. 2023-09-30T22:23:52.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.2 2023-09-30T22:23:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.989+0000 7fcc46857700 1 Processor -- start 2023-09-30T22:23:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.989+0000 7fcc46857700 1 -- start start 2023-09-30T22:23:52.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc46857700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a0a30 0x7fcc400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc46857700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc400a1340 0x7fcc40133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc46857700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4009edd0 con 0x7fcc40133fa0 2023-09-30T22:23:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc46857700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4009e1f0 con 0x7fcc400a1340 2023-09-30T22:23:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc46857700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4009dc50 con 0x7fcc400a0a30 2023-09-30T22:23:52.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc3f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc400a1340 0x7fcc40133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc44df4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc40133fa0 0x7fcc4013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50560/0 (socket says 172.21.15.32:50560) 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc44df4700 1 -- 172.21.15.32:0/108803136 learned_addr learned my addr 172.21.15.32:0/108803136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.990+0000 7fcc3ffff700 1 --2- 172.21.15.32:0/108803136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a0a30 0x7fcc400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3ffff700 1 -- 172.21.15.32:0/108803136 >> v2:172.21.15.32:3300/0 conn(0x7fcc400a1340 msgr2=0x7fcc40133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3ffff700 1 --2- 172.21.15.32:0/108803136 >> v2:172.21.15.32:3300/0 conn(0x7fcc400a1340 0x7fcc40133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3ffff700 1 -- 172.21.15.32:0/108803136 >> v1:172.21.15.32:6789/0 conn(0x7fcc40133fa0 legacy=0x7fcc4013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3ffff700 1 -- 172.21.15.32:0/108803136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400a1fe0 con 0x7fcc400a0a30 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3effd700 1 -- 172.21.15.32:0/108803136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977644862 0 0) 0x7fcc4009edd0 con 0x7fcc40133fa0 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3ffff700 1 --2- 172.21.15.32:0/108803136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a0a30 0x7fcc400a0e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009840 tx=0x7fcc30009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2dc03f829a4c886 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3effd700 1 -- 172.21.15.32:0/108803136 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30011830 con 0x7fcc400a0a30 2023-09-30T22:23:52.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3effd700 1 -- 172.21.15.32:0/108803136 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc30011990 con 0x7fcc400a0a30 2023-09-30T22:23:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.991+0000 7fcc3effd700 1 -- 172.21.15.32:0/108803136 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30011d20 con 0x7fcc400a0a30 2023-09-30T22:23:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 -- 172.21.15.32:0/108803136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a0a30 msgr2=0x7fcc400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 --2- 172.21.15.32:0/108803136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a0a30 0x7fcc400a0e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009840 tx=0x7fcc30009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:52.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 -- 172.21.15.32:0/108803136 shutdown_connections 2023-09-30T22:23:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 --2- 172.21.15.32:0/108803136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a0a30 0x7fcc400a0e00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 --2- 172.21.15.32:0/108803136 >> v2:172.21.15.32:3300/0 conn(0x7fcc400a1340 0x7fcc40133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 -- 172.21.15.32:0/108803136 >> 172.21.15.32:0/108803136 conn(0x7fcc400a6550 msgr2=0x7fcc400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 -- 172.21.15.32:0/108803136 shutdown_connections 2023-09-30T22:23:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.992+0000 7fcc46857700 1 -- 172.21.15.32:0/108803136 wait complete. 2023-09-30T22:23:52.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.994+0000 7fcc46857700 1 Processor -- start 2023-09-30T22:23:52.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.994+0000 7fcc46857700 1 -- start start 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc46857700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc400a0a30 0x7fcc40123410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc46857700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a1340 0x7fcc40123950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc46857700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc40133fa0 0x7fcc401c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc46857700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc401265d0 con 0x7fcc40133fa0 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc46857700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40123020 con 0x7fcc400a0a30 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc46857700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40124c70 con 0x7fcc400a1340 2023-09-30T22:23:52.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc3ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc400a0a30 0x7fcc40123410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc3ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc400a0a30 0x7fcc40123410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52146/0 (socket says 172.21.15.32:52146) 2023-09-30T22:23:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc3ffff700 1 -- 172.21.15.32:0/213627563 learned_addr learned my addr 172.21.15.32:0/213627563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.995+0000 7fcc44df4700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc40133fa0 0x7fcc401c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a1340 0x7fcc40123950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 -- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc400a0a30 msgr2=0x7fcc40123410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc400a0a30 0x7fcc40123410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 -- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc40133fa0 msgr2=0x7fcc401c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc40133fa0 0x7fcc401c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 -- 172.21.15.32:0/213627563 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc300094a0 con 0x7fcc400a1340 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc44df4700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc40133fa0 0x7fcc401c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3f7fe700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a1340 0x7fcc40123950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e900 tx=0x7fcc3400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34013680 con 0x7fcc400a1340 2023-09-30T22:23:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc340137e0 con 0x7fcc400a1340 2023-09-30T22:23:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34013b50 con 0x7fcc400a1340 2023-09-30T22:23:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.996+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc400a3630 con 0x7fcc400a1340 2023-09-30T22:23:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.997+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc401246c0 con 0x7fcc400a1340 2023-09-30T22:23:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.998+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fcc34013cb0 con 0x7fcc400a1340 2023-09-30T22:23:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.998+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc400a6b60 con 0x7fcc400a1340 2023-09-30T22:23:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.998+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3893+0+0 (secure 0 0 0) 0x7fcc3403bb80 con 0x7fcc400a1340 2023-09-30T22:23:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:52.998+0000 7fcc3ffff700 1 -- 172.21.15.32:0/213627563 >> v1:172.21.15.32:6816/118188 conn(0x7fcc2c02c1e0 legacy=0x7fcc2c02e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.006+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fcc34017070 con 0x7fcc400a1340 2023-09-30T22:23:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.020+0000 7f9a39859700 1 -- 172.21.15.32:0/1661182144 >> v1:172.21.15.32:6816/118188 conn(0x7f9a1802c1b0 legacy=0x7f9a1802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.025+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9a1c005590 con 0x7f9a3409a580 2023-09-30T22:23:53.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.026+0000 7f9a267fc700 1 -- 172.21.15.32:0/1661182144 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a30024070 con 0x7f9a3409a580 2023-09-30T22:23:53.027 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 >> v1:172.21.15.32:6816/118188 conn(0x7f9a1802c1b0 legacy=0x7f9a1802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 msgr2=0x7f9a3411be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 0x7f9a3411be10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c4d0 tx=0x7f9a3000c7e0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 shutdown_connections 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a34131810 0x7f9a3411c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a34099a40 0x7f9a3411b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 --2- 172.21.15.32:0/1661182144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a3409a580 0x7f9a3411be10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 >> 172.21.15.32:0/1661182144 conn(0x7f9a340a2aa0 msgr2=0x7f9a340a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 shutdown_connections 2023-09-30T22:23:53.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.028+0000 7f9a3babd700 1 -- 172.21.15.32:0/1661182144 wait complete. 2023-09-30T22:23:53.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:23:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.065+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4fe41253a0 con 0x7f4fe409e390 2023-09-30T22:23:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.066+0000 7f4fe0ff9700 1 -- 172.21.15.32:0/1353235236 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4fd4015020 con 0x7f4fe409e390 2023-09-30T22:23:53.067 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 >> v1:172.21.15.32:6816/118188 conn(0x7f4fd002c200 legacy=0x7f4fd002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe409e390 msgr2=0x7f4fe41255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe409e390 0x7f4fe41255c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4016000 tx=0x7f4fd4004750 comp rx=0 tx=0).stop 2023-09-30T22:23:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 shutdown_connections 2023-09-30T22:23:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe41341d0 0x7f4fe4123d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4fe409e390 0x7f4fe41255c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 --2- 172.21.15.32:0/1353235236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe41338c0 0x7f4fe4123830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 >> 172.21.15.32:0/1353235236 conn(0x7f4fe40a2aa0 msgr2=0x7f4fe40a0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 shutdown_connections 2023-09-30T22:23:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.068+0000 7f4fea5ec700 1 -- 172.21.15.32:0/1353235236 wait complete. 2023-09-30T22:23:53.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:23:53.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.199+0000 7fcc3ffff700 1 -- 172.21.15.32:0/213627563 >> v1:172.21.15.32:6816/118188 conn(0x7fcc2c02c1e0 legacy=0x7fcc2c02e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/118188 2023-09-30T22:23:53.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.210+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcc400a1d40 con 0x7fcc400a1340 2023-09-30T22:23:53.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.210+0000 7fcc3cff9700 1 -- 172.21.15.32:0/213627563 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcc34021070 con 0x7fcc400a1340 2023-09-30T22:23:53.212 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:53.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 >> v1:172.21.15.32:6816/118188 conn(0x7fcc2c02c1e0 legacy=0x7fcc2c02e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a1340 msgr2=0x7fcc40123950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a1340 0x7fcc40123950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e900 tx=0x7fcc3400ec10 comp rx=0 tx=0).stop 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 shutdown_connections 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc40133fa0 0x7fcc401c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc400a0a30 0x7fcc40123410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 --2- 172.21.15.32:0/213627563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc400a1340 0x7fcc40123950 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:53.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.212+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 >> 172.21.15.32:0/213627563 conn(0x7fcc400a6550 msgr2=0x7fcc4011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.213+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 shutdown_connections 2023-09-30T22:23:53.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:53.213+0000 7fcc46857700 1 -- 172.21.15.32:0/213627563 wait complete. 2023-09-30T22:23:53.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.0 2023-09-30T22:23:53.578 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:53.578+0000 7f4e8b1c9240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T22:23:53.715 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:53.714+0000 7f4e8b1c9240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:23:53.889 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:53.888+0000 7f4e8b1c9240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T22:23:53.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:54.005 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.005+0000 7f4e8b1c9240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T22:23:54.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:54.083 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:54.101 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.100+0000 7f4e09020700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.102 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.100+0000 7f4e09020700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.102 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.100+0000 7f4e09020700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.102 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.100+0000 7f4e09020700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.102 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.100+0000 7f4e09020700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.104 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.102+0000 7f4e0601a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.105 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.102+0000 7f4e0601a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.105 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.102+0000 7f4e0601a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.105 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.102+0000 7f4e0601a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.105 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:54.102+0000 7f4e0601a700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:23:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.144+0000 7fae80e5c700 1 Processor -- start 2023-09-30T22:23:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.144+0000 7fae80e5c700 1 -- start start 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae80e5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 0x7fae7c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae80e5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae80e5c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c09b120 con 0x7fae7c09d6f0 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae80e5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c06b460 con 0x7fae7c139170 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae80e5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c09a7e0 con 0x7fae7c09e1d0 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae79d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 0x7fae7c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae79d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 0x7fae7c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41740/0 (socket says 172.21.15.32:41740) 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae7a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fae7c09d6f0 0x7fae7c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50570/0 (socket says 172.21.15.32:50570) 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae79d9b700 1 -- 172.21.15.32:0/616059257 learned_addr learned my addr 172.21.15.32:0/616059257 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.145+0000 7fae7ad9d700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae79d9b700 1 -- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 msgr2=0x7fae7c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae79d9b700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae79d9b700 1 -- 172.21.15.32:0/616059257 >> v1:172.21.15.32:6789/0 conn(0x7fae7c09d6f0 legacy=0x7fae7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae79d9b700 1 -- 172.21.15.32:0/616059257 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c09a500 con 0x7fae7c09e1d0 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae7ad9d700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae79d9b700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 0x7fae7c138c30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fae70009950 tx=0x7fae70009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2a8983a8ae20f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae7959a700 1 -- 172.21.15.32:0/616059257 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70011830 con 0x7fae7c09e1d0 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae7959a700 1 -- 172.21.15.32:0/616059257 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70011990 con 0x7fae7c09e1d0 2023-09-30T22:23:54.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.146+0000 7fae7959a700 1 -- 172.21.15.32:0/616059257 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70011d40 con 0x7fae7c09e1d0 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 -- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 msgr2=0x7fae7c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 0x7fae7c138c30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fae70009950 tx=0x7fae70009c60 comp rx=0 tx=0).stop 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 -- 172.21.15.32:0/616059257 shutdown_connections 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 --2- 172.21.15.32:0/616059257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09e1d0 0x7fae7c138c30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 -- 172.21.15.32:0/616059257 >> 172.21.15.32:0/616059257 conn(0x7fae7c066f50 msgr2=0x7fae7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 -- 172.21.15.32:0/616059257 shutdown_connections 2023-09-30T22:23:54.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.147+0000 7fae80e5c700 1 -- 172.21.15.32:0/616059257 wait complete. 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.148+0000 7fae80e5c700 1 Processor -- start 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.148+0000 7fae80e5c700 1 -- start start 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae80e5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 0x7fae7c11e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae80e5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c09e1d0 0x7fae7c11ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae80e5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c11f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae80e5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c121e70 con 0x7fae7c09e1d0 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae80e5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c122050 con 0x7fae7c139170 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae80e5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c09c710 con 0x7fae7c09d6f0 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 0x7fae7c11e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 0x7fae7c11e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41754/0 (socket says 172.21.15.32:41754) 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 -- 172.21.15.32:0/91564305 learned_addr learned my addr 172.21.15.32:0/91564305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7ad9d700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c11f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae79d9b700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c09e1d0 0x7fae7c11ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 -- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 msgr2=0x7fae7c11f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c11f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 -- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c09e1d0 msgr2=0x7fae7c11ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c09e1d0 0x7fae7c11ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7a59c700 1 -- 172.21.15.32:0/91564305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae74009a70 con 0x7fae7c09d6f0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.149+0000 7fae7ad9d700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c11f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.150+0000 7fae7a59c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 0x7fae7c11e810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c550 tx=0x7fae6c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.150+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c005cb0 con 0x7fae7c09d6f0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.150+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c005e10 con 0x7fae7c09d6f0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.150+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70009550 con 0x7fae7c09d6f0 2023-09-30T22:23:54.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.150+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0072d0 con 0x7fae7c09d6f0 2023-09-30T22:23:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.150+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c120530 con 0x7fae7c09d6f0 2023-09-30T22:23:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.151+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fae6c007430 con 0x7fae7c09d6f0 2023-09-30T22:23:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.151+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c074ea0 con 0x7fae7c09d6f0 2023-09-30T22:23:54.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.151+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7fae6c03e8e0 con 0x7fae7c09d6f0 2023-09-30T22:23:54.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.156+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fae6c01b3b0 con 0x7fae7c09d6f0 2023-09-30T22:23:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 Processor -- start 2023-09-30T22:23:54.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 -- start start 2023-09-30T22:23:54.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f043009d630 0x7f04300a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0430133930 0x7f0430133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04300a2a70 con 0x7f043009d630 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04300a01d0 con 0x7f04300a0a30 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.209+0000 7f043575b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f043009e2b0 con 0x7f0430133930 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04300a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52260/0 (socket says 172.21.15.32:52260) 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 -- 172.21.15.32:0/3793560864 learned_addr learned my addr 172.21.15.32:0/3793560864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042f7fe700 1 --2- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0430133930 0x7f0430133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042e7fc700 1 --2- 172.21.15.32:0/3793560864 >> v2:172.21.15.32:3300/0 conn(0x7f043009d630 0x7f04300a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 -- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0430133930 msgr2=0x7f0430133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:54.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 --2- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0430133930 0x7f0430133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 -- 172.21.15.32:0/3793560864 >> v2:172.21.15.32:3300/0 conn(0x7f043009d630 msgr2=0x7f04300a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 --2- 172.21.15.32:0/3793560864 >> v2:172.21.15.32:3300/0 conn(0x7f043009d630 0x7f04300a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 -- 172.21.15.32:0/3793560864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043009dc50 con 0x7f04300a0a30 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042e7fc700 1 --2- 172.21.15.32:0/3793560864 >> v2:172.21.15.32:3300/0 conn(0x7f043009d630 0x7f04300a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042effd700 1 --2- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04300a0e00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04200098f0 tx=0x7f0420009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57665ea0a62faf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.210+0000 7f042dffb700 1 -- 172.21.15.32:0/3793560864 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420011830 con 0x7f04300a0a30 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f042dffb700 1 -- 172.21.15.32:0/3793560864 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0420011990 con 0x7f04300a0a30 2023-09-30T22:23:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f042dffb700 1 -- 172.21.15.32:0/3793560864 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420011d20 con 0x7f04300a0a30 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 -- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 msgr2=0x7f04300a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 --2- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04300a0e00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04200098f0 tx=0x7f0420009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 -- 172.21.15.32:0/3793560864 shutdown_connections 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 --2- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04300a0e00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 --2- 172.21.15.32:0/3793560864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0430133930 0x7f0430133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 --2- 172.21.15.32:0/3793560864 >> v2:172.21.15.32:3300/0 conn(0x7f043009d630 0x7f04300a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 -- 172.21.15.32:0/3793560864 >> 172.21.15.32:0/3793560864 conn(0x7f04300a3050 msgr2=0x7f04300a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 -- 172.21.15.32:0/3793560864 shutdown_connections 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.211+0000 7f043575b700 1 -- 172.21.15.32:0/3793560864 wait complete. 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.212+0000 7f043575b700 1 Processor -- start 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 -- start start 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f043009d630 0x7f04301c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04301c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0430133930 0x7f04301c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430122100 con 0x7f0430133930 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04301204d0 con 0x7f04300a0a30 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f043575b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0430120a50 con 0x7f043009d630 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f042e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04301c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f042e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04301c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52268/0 (socket says 172.21.15.32:52268) 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f042e7fc700 1 -- 172.21.15.32:0/3382520390 learned_addr learned my addr 172.21.15.32:0/3382520390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f042f7fe700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0430133930 0x7f04301c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.213+0000 7f042effd700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f043009d630 0x7f04301c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f042e7fc700 1 -- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f043009d630 msgr2=0x7f04301c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f042e7fc700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f043009d630 0x7f04301c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f042e7fc700 1 -- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0430133930 msgr2=0x7f04301c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f042e7fc700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0430133930 0x7f04301c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f042e7fc700 1 -- 172.21.15.32:0/3382520390 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240094a0 con 0x7f04300a0a30 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f042e7fc700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04301c85d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f042400ebb0 tx=0x7f042400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400c850 con 0x7f04300a0a30 2023-09-30T22:23:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042400c9b0 con 0x7f04300a0a30 2023-09-30T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f043575b700 1 -- 172.21.15.32:0/3382520390 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0420009550 con 0x7f04300a0a30 2023-09-30T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400cd40 con 0x7f04300a0a30 2023-09-30T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.214+0000 7f043575b700 1 -- 172.21.15.32:0/3382520390 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0430122b70 con 0x7f04300a0a30 2023-09-30T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.215+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04300747f0 con 0x7f04300a0a30 2023-09-30T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.216+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f0424013470 con 0x7f04300a0a30 2023-09-30T22:23:54.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.216+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7f042403f9f0 con 0x7f04300a0a30 2023-09-30T22:23:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.221+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f042401b3b0 con 0x7f04300a0a30 2023-09-30T22:23:54.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:54.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.248+0000 7f15cd6d3700 1 Processor -- start 2023-09-30T22:23:54.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.248+0000 7f15cd6d3700 1 -- start start 2023-09-30T22:23:54.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15cd6d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15c809d6f0 0x7f15c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15cd6d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15cd6d3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c809b120 con 0x7f15c809e000 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15cd6d3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c806b460 con 0x7f15c809d6f0 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15cd6d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c809a7e0 con 0x7f15c809e440 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15be7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15c809e000 0x7f15c81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50580/0 (socket says 172.21.15.32:50580) 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15be7fc700 1 -- 172.21.15.32:0/1760766640 learned_addr learned my addr 172.21.15.32:0/1760766640 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/1760766640 >> v2:172.21.15.32:3300/0 conn(0x7f15c809d6f0 0x7f15c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.249+0000 7f15c77fe700 1 --2- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c6ffd700 1 -- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 msgr2=0x7f15c8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c6ffd700 1 -- 172.21.15.32:0/1760766640 >> v1:172.21.15.32:6789/0 conn(0x7f15c809e000 legacy=0x7f15c81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c6ffd700 1 -- 172.21.15.32:0/1760766640 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c809a500 con 0x7f15c809d6f0 2023-09-30T22:23:54.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c67fc700 1 -- 172.21.15.32:0/1760766640 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595163854 0 0) 0x7f15c809b120 con 0x7f15c809e000 2023-09-30T22:23:54.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c809dac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009870 tx=0x7f15b8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f05563a14e2f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c67fc700 1 -- 172.21.15.32:0/1760766640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8011830 con 0x7f15c809d6f0 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c67fc700 1 -- 172.21.15.32:0/1760766640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b8011990 con 0x7f15c809d6f0 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.250+0000 7f15c67fc700 1 -- 172.21.15.32:0/1760766640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8011d20 con 0x7f15c809d6f0 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 -- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 msgr2=0x7f15c809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c809dac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009870 tx=0x7f15b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 -- 172.21.15.32:0/1760766640 shutdown_connections 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/1760766640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c809dac0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 -- 172.21.15.32:0/1760766640 >> 172.21.15.32:0/1760766640 conn(0x7f15c8066f50 msgr2=0x7f15c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 -- 172.21.15.32:0/1760766640 shutdown_connections 2023-09-30T22:23:54.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.251+0000 7f15cd6d3700 1 -- 172.21.15.32:0/1760766640 wait complete. 2023-09-30T22:23:54.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.253+0000 7f15cd6d3700 1 Processor -- start 2023-09-30T22:23:54.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.253+0000 7f15cd6d3700 1 -- start start 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15cd6d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c81c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15cd6d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15c809e000 0x7f15c81c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15cd6d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c81a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15cd6d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81a5450 con 0x7f15c809d6f0 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15cd6d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81a5260 con 0x7f15c809e440 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15cd6d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81b7fe0 con 0x7f15c809e000 2023-09-30T22:23:54.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c81c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c81c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46526/0 (socket says 172.21.15.32:46526) 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15c6ffd700 1 -- 172.21.15.32:0/976617632 learned_addr learned my addr 172.21.15.32:0/976617632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15c77fe700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c81a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15be7fc700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15c809e000 0x7f15c81c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.254+0000 7f15c6ffd700 1 -- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15c809e000 msgr2=0x7f15c81c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15c809e000 0x7f15c81c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15c6ffd700 1 -- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 msgr2=0x7f15c81a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c81a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15c6ffd700 1 -- 172.21.15.32:0/976617632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b80094a0 con 0x7f15c809d6f0 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15c6ffd700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c81c8410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80036a0 con 0x7f15c809d6f0 2023-09-30T22:23:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b801f070 con 0x7f15c809d6f0 2023-09-30T22:23:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c80a2d60 con 0x7f15c809d6f0 2023-09-30T22:23:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8003a00 con 0x7f15c809d6f0 2023-09-30T22:23:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.255+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c809c770 con 0x7f15c809d6f0 2023-09-30T22:23:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.257+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f15b8003b60 con 0x7f15c809d6f0 2023-09-30T22:23:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.257+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c8074ea0 con 0x7f15c809d6f0 2023-09-30T22:23:54.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.258+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7f15b8021070 con 0x7f15c809d6f0 2023-09-30T22:23:54.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.263+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f15b8024380 con 0x7f15c809d6f0 2023-09-30T22:23:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.350+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fae7c122190 con 0x7fae7c09d6f0 2023-09-30T22:23:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.350+0000 7fae6affd700 1 -- 172.21.15.32:0/91564305 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae7c122190 con 0x7fae7c09d6f0 2023-09-30T22:23:54.352 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.353+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 msgr2=0x7fae7c11e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.353+0000 7fae80e5c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 0x7fae7c11e810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00c550 tx=0x7fae6c00c860 comp rx=0 tx=0).stop 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 shutdown_connections 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae7c139170 0x7fae7c11f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae7c09d6f0 0x7fae7c11e810 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 --2- 172.21.15.32:0/91564305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae7c09e1d0 0x7fae7c11ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 >> 172.21.15.32:0/91564305 conn(0x7fae7c066f50 msgr2=0x7fae7c06f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 shutdown_connections 2023-09-30T22:23:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.354+0000 7fae80e5c700 1 -- 172.21.15.32:0/91564305 wait complete. 2023-09-30T22:23:54.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.2 2023-09-30T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.375+0000 7fdb77a0c700 1 Processor -- start 2023-09-30T22:23:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 -- start start 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 0x7fdb70130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb7006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb7006fb20 0x7fdb7006ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7009a390 con 0x7fdb7006fb20 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700716a0 con 0x7fdb70130540 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb77a0c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70071290 con 0x7fdb70130e50 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb74fa7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb7006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb757a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 0x7fdb70130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.376+0000 7fdb75fa9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb7006fb20 0x7fdb7006ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb7006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41786/0 (socket says 172.21.15.32:41786) 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb757a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 0x7fdb70130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52288/0 (socket says 172.21.15.32:52288) 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 -- 172.21.15.32:0/2865163367 learned_addr learned my addr 172.21.15.32:0/2865163367 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 -- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 msgr2=0x7fdb70130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 --2- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 0x7fdb70130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 -- 172.21.15.32:0/2865163367 >> v2:172.21.15.32:3300/0 conn(0x7fdb7006fb20 msgr2=0x7fdb7006ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 --2- 172.21.15.32:0/2865163367 >> v2:172.21.15.32:3300/0 conn(0x7fdb7006fb20 0x7fdb7006ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 -- 172.21.15.32:0/2865163367 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70130340 con 0x7fdb70130e50 2023-09-30T22:23:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb75fa9700 1 --2- 172.21.15.32:0/2865163367 >> v2:172.21.15.32:3300/0 conn(0x7fdb7006fb20 0x7fdb7006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb757a8700 1 --2- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 0x7fdb70130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb74fa7700 1 --2- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb7006f5e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009920 tx=0x7fdb60009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451e0299560258b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb67fff700 1 -- 172.21.15.32:0/2865163367 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60011830 con 0x7fdb70130e50 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb67fff700 1 -- 172.21.15.32:0/2865163367 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60011990 con 0x7fdb70130e50 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.377+0000 7fdb67fff700 1 -- 172.21.15.32:0/2865163367 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60011d20 con 0x7fdb70130e50 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 -- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 msgr2=0x7fdb7006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb7006f5e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009920 tx=0x7fdb60009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 -- 172.21.15.32:0/2865163367 shutdown_connections 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb70130540 0x7fdb70130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/2865163367 >> v2:172.21.15.32:3300/0 conn(0x7fdb7006fb20 0x7fdb7006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/2865163367 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb7006f5e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 -- 172.21.15.32:0/2865163367 >> 172.21.15.32:0/2865163367 conn(0x7fdb700a2aa0 msgr2=0x7fdb700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 -- 172.21.15.32:0/2865163367 shutdown_connections 2023-09-30T22:23:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.378+0000 7fdb77a0c700 1 -- 172.21.15.32:0/2865163367 wait complete. 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.379+0000 7fdb77a0c700 1 Processor -- start 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.379+0000 7fdb77a0c700 1 -- start start 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb77a0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 0x7fdb701c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb77a0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb70130540 0x7fdb701c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb77a0c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb701c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb77a0c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7012b460 con 0x7fdb70130540 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb77a0c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb701c5980 con 0x7fdb7006fb20 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb77a0c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700718e0 con 0x7fdb70130e50 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 0x7fdb701c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 0x7fdb701c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52304/0 (socket says 172.21.15.32:52304) 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb74fa7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb70130540 0x7fdb701c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 -- 172.21.15.32:0/3761924679 learned_addr learned my addr 172.21.15.32:0/3761924679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 -- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 msgr2=0x7fdb701c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb701c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 -- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb70130540 msgr2=0x7fdb701c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb70130540 0x7fdb701c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 -- 172.21.15.32:0/3761924679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68009a70 con 0x7fdb7006fb20 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb74fa7700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb70130540 0x7fdb701c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb757a8700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 0x7fdb701c4f00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00ebe0 tx=0x7fdb6c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.380+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c009800 con 0x7fdb7006fb20 2023-09-30T22:23:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.381+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c00c910 con 0x7fdb7006fb20 2023-09-30T22:23:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.381+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c0094a0 con 0x7fdb7006fb20 2023-09-30T22:23:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.381+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c00cca0 con 0x7fdb7006fb20 2023-09-30T22:23:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.381+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb60009550 con 0x7fdb7006fb20 2023-09-30T22:23:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.382+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fdb6c013470 con 0x7fdb7006fb20 2023-09-30T22:23:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.382+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4026+0+0 (secure 0 0 0) 0x7fdb6c03f6d0 con 0x7fdb7006fb20 2023-09-30T22:23:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.382+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70074ea0 con 0x7fdb7006fb20 2023-09-30T22:23:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.390+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fdb6c021080 con 0x7fdb7006fb20 2023-09-30T22:23:54.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.419+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0430072980 con 0x7f04300a0a30 2023-09-30T22:23:54.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.420+0000 7f04177fe700 1 -- 172.21.15.32:0/3382520390 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0424017070 con 0x7f04300a0a30 2023-09-30T22:23:54.421 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 msgr2=0x7f04301c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04301c85d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f042400ebb0 tx=0x7f042400eec0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 shutdown_connections 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0430133930 0x7f04301c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f043009d630 0x7f04301c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 --2- 172.21.15.32:0/3382520390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04300a0a30 0x7f04301c85d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 >> 172.21.15.32:0/3382520390 conn(0x7f04300a3050 msgr2=0x7f043009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 shutdown_connections 2023-09-30T22:23:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.422+0000 7f04157fa700 1 -- 172.21.15.32:0/3382520390 wait complete. 2023-09-30T22:23:54.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:23:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.455+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15c81a5a50 con 0x7f15c809d6f0 2023-09-30T22:23:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.455+0000 7f15bffff700 1 -- 172.21.15.32:0/976617632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15b8015020 con 0x7f15c809d6f0 2023-09-30T22:23:54.456 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 msgr2=0x7f15c81c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c81c8410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 shutdown_connections 2023-09-30T22:23:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15c809e440 0x7f15c81a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15c809d6f0 0x7f15c81c8410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 --2- 172.21.15.32:0/976617632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15c809e000 0x7f15c81c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.457+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 >> 172.21.15.32:0/976617632 conn(0x7f15c8066f50 msgr2=0x7f15c81bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.458+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 shutdown_connections 2023-09-30T22:23:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.458+0000 7f15cd6d3700 1 -- 172.21.15.32:0/976617632 wait complete. 2023-09-30T22:23:54.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:23:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.573+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdb7012d3e0 con 0x7fdb7006fb20 2023-09-30T22:23:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.574+0000 7fdb65ffb700 1 -- 172.21.15.32:0/3761924679 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb6c01bdc0 con 0x7fdb7006fb20 2023-09-30T22:23:54.575 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:54.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 msgr2=0x7fdb701c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 0x7fdb701c4f00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00ebe0 tx=0x7fdb6c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 shutdown_connections 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb70130e50 0x7fdb701c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb7006fb20 0x7fdb701c4f00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 --2- 172.21.15.32:0/3761924679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb70130540 0x7fdb701c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 >> 172.21.15.32:0/3761924679 conn(0x7fdb700a2aa0 msgr2=0x7fdb7012f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 shutdown_connections 2023-09-30T22:23:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:54.576+0000 7fdb77a0c700 1 -- 172.21.15.32:0/3761924679 wait complete. 2023-09-30T22:23:54.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.0 2023-09-30T22:23:55.034 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:55.033+0000 7f4e4c5e4700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T22:23:55.034 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:55.033+0000 7f4e4c5e4700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T22:23:55.034 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-09-30T22:23:55.033+0000 7f4e4c5e4700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-09-30T22:23:55.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:55.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:55.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:55.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.523+0000 7faab4a49700 1 Processor -- start 2023-09-30T22:23:55.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.523+0000 7faab4a49700 1 -- start start 2023-09-30T22:23:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faab4a49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 0x7faab0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faab4a49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faab0139250 0x7faab013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faab4a49700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab009edd0 con 0x7faab00a0a30 2023-09-30T22:23:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faab4a49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab00a18e0 con 0x7faab0139250 2023-09-30T22:23:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faab4a49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faab009e1f0 con 0x7faab00a1510 2023-09-30T22:23:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faaa5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 0x7faab0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faaa5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 0x7faab0138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41804/0 (socket says 172.21.15.32:41804) 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2757887427 learned_addr learned my addr 172.21.15.32:0/2757887427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.524+0000 7faaaed9d700 1 --2- 172.21.15.32:0/2757887427 >> v2:172.21.15.32:3300/0 conn(0x7faab0139250 0x7faab013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2757887427 >> v2:172.21.15.32:3300/0 conn(0x7faab0139250 msgr2=0x7faab013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaa5d9b700 1 --2- 172.21.15.32:0/2757887427 >> v2:172.21.15.32:3300/0 conn(0x7faab0139250 0x7faab013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2757887427 >> v1:172.21.15.32:6789/0 conn(0x7faab00a0a30 legacy=0x7faab00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2757887427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab009dc50 con 0x7faab00a1510 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaaed9d700 1 --2- 172.21.15.32:0/2757887427 >> v2:172.21.15.32:3300/0 conn(0x7faab0139250 0x7faab013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaa5d9b700 1 --2- 172.21.15.32:0/2757887427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 0x7faab0138d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa98009920 tx=0x7faa98009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99343d8d049b5a73 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaadd9b700 1 -- 172.21.15.32:0/2757887427 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98011830 con 0x7faab00a1510 2023-09-30T22:23:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaadd9b700 1 -- 172.21.15.32:0/2757887427 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa98011990 con 0x7faab00a1510 2023-09-30T22:23:55.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.525+0000 7faaadd9b700 1 -- 172.21.15.32:0/2757887427 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98011d20 con 0x7faab00a1510 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 -- 172.21.15.32:0/2757887427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 msgr2=0x7faab0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 --2- 172.21.15.32:0/2757887427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 0x7faab0138d10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa98009920 tx=0x7faa98009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 -- 172.21.15.32:0/2757887427 shutdown_connections 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 --2- 172.21.15.32:0/2757887427 >> v2:172.21.15.32:3300/0 conn(0x7faab0139250 0x7faab013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 --2- 172.21.15.32:0/2757887427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a1510 0x7faab0138d10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 -- 172.21.15.32:0/2757887427 >> 172.21.15.32:0/2757887427 conn(0x7faab00a6550 msgr2=0x7faab00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 -- 172.21.15.32:0/2757887427 shutdown_connections 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.526+0000 7faab4a49700 1 -- 172.21.15.32:0/2757887427 wait complete. 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.527+0000 7faab4a49700 1 Processor -- start 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.527+0000 7faab4a49700 1 -- start start 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faab4a49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a0a30 0x7faab01c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faab4a49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 0x7faab01c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faab4a49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab0139250 0x7faab0122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faab4a49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab0125eb0 con 0x7faab0139250 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faab4a49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab0138dd0 con 0x7faab00a1510 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faab4a49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faab01be3d0 con 0x7faab00a0a30 2023-09-30T22:23:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 0x7faab01c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 0x7faab01c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52306/0 (socket says 172.21.15.32:52306) 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2887840530 learned_addr learned my addr 172.21.15.32:0/2887840530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaaed9d700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab0139250 0x7faab0122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaae59c700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a0a30 0x7faab01c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a0a30 msgr2=0x7faab01c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a0a30 0x7faab01c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab0139250 msgr2=0x7faab0122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab0139250 0x7faab0122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaa5d9b700 1 -- 172.21.15.32:0/2887840530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8009a70 con 0x7faab00a1510 2023-09-30T22:23:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.528+0000 7faaaed9d700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab0139250 0x7faab0122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.529+0000 7faaa5d9b700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 0x7faab01c8cf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faa980098f0 tx=0x7faa98004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.529+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98004340 con 0x7faab00a1510 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.529+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9801f070 con 0x7faab00a1510 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.529+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa980036a0 con 0x7faab00a1510 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.529+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98009550 con 0x7faab00a1510 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.529+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab0126450 con 0x7faab00a1510 2023-09-30T22:23:55.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.530+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7faa98021030 con 0x7faab00a1510 2023-09-30T22:23:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.530+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4009+0+0 (secure 0 0 0) 0x7faa980409a0 con 0x7faab00a1510 2023-09-30T22:23:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.530+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab0074ea0 con 0x7faab00a1510 2023-09-30T22:23:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.536+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa98015020 con 0x7faab00a1510 2023-09-30T22:23:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.582+0000 7f90d9889700 1 Processor -- start 2023-09-30T22:23:55.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.582+0000 7f90d9889700 1 -- start start 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d9889700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d9889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d9889700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d409f130 con 0x7f90d409e410 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d9889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d409f520 con 0x7f90d409e000 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d9889700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90d409e930 con 0x7f90d409d6f0 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41818/0 (socket says 172.21.15.32:41818) 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 -- 172.21.15.32:0/2827271052 learned_addr learned my addr 172.21.15.32:0/2827271052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d37fe700 1 --1- 172.21.15.32:0/2827271052 >> v1:172.21.15.32:6789/0 conn(0x7f90d409e410 0x7f90d4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50596/0 (socket says 172.21.15.32:50596) 2023-09-30T22:23:55.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d27fc700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 -- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 msgr2=0x7f90d4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 -- 172.21.15.32:0/2827271052 >> v1:172.21.15.32:6789/0 conn(0x7f90d409e410 legacy=0x7f90d4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d1ffb700 1 -- 172.21.15.32:0/2827271052 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138362211 0 0) 0x7f90d409f130 con 0x7f90d409e410 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d2ffd700 1 -- 172.21.15.32:0/2827271052 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d409b2d0 con 0x7f90d409d6f0 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.583+0000 7f90d27fc700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.584+0000 7f90d2ffd700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d409dac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009950 tx=0x7f90c4009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab1f73a48f4c372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.584+0000 7f90d1ffb700 1 -- 172.21.15.32:0/2827271052 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4011830 con 0x7f90d409d6f0 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.584+0000 7f90d1ffb700 1 -- 172.21.15.32:0/2827271052 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4011990 con 0x7f90d409d6f0 2023-09-30T22:23:55.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.584+0000 7f90d1ffb700 1 -- 172.21.15.32:0/2827271052 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4011d40 con 0x7f90d409d6f0 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 -- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 msgr2=0x7f90d409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d409dac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009950 tx=0x7f90c4009c60 comp rx=0 tx=0).stop 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 -- 172.21.15.32:0/2827271052 shutdown_connections 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d409dac0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 --2- 172.21.15.32:0/2827271052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 -- 172.21.15.32:0/2827271052 >> 172.21.15.32:0/2827271052 conn(0x7f90d4066f50 msgr2=0x7f90d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 -- 172.21.15.32:0/2827271052 shutdown_connections 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.585+0000 7f90d9889700 1 -- 172.21.15.32:0/2827271052 wait complete. 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.586+0000 7f90d9889700 1 Processor -- start 2023-09-30T22:23:55.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.586+0000 7f90d9889700 1 -- start start 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.586+0000 7f90d9889700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d41c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.586+0000 7f90d9889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d41c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d9889700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d409e410 0x7f90d41c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d9889700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d413b570 con 0x7f90d409e410 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d9889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4136020 con 0x7f90d409e000 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d9889700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90d41367d0 con 0x7f90d409d6f0 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d41c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d41c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52334/0 (socket says 172.21.15.32:52334) 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 -- 172.21.15.32:0/1959223587 learned_addr learned my addr 172.21.15.32:0/1959223587 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d37fe700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d409e410 0x7f90d41c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d2ffd700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d41c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 -- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 msgr2=0x7f90d41c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d41c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 -- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d409e410 msgr2=0x7f90d41c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d409e410 0x7f90d41c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 -- 172.21.15.32:0/1959223587 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80094a0 con 0x7f90d409e000 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d37fe700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d409e410 0x7f90d41c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.587+0000 7f90d27fc700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d41c8e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90c800ebe0 tx=0x7f90c800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.588+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800c850 con 0x7f90d409e000 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.588+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800c9b0 con 0x7f90d409e000 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.588+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4009550 con 0x7f90d409e000 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.588+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800cd40 con 0x7f90d409e000 2023-09-30T22:23:55.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.588+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d413b760 con 0x7f90d409e000 2023-09-30T22:23:55.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:55.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.589+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f90c8013470 con 0x7f90d409e000 2023-09-30T22:23:55.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.589+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4009+0+0 (secure 0 0 0) 0x7f90c803fb10 con 0x7f90d409e000 2023-09-30T22:23:55.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.589+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d4136910 con 0x7f90d409e000 2023-09-30T22:23:55.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.594+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90c8017080 con 0x7f90d409e000 2023-09-30T22:23:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.628+0000 7f7142514700 1 Processor -- start 2023-09-30T22:23:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.628+0000 7f7142514700 1 -- start start 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.628+0000 7f7142514700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f713c09eca0 0x7f713c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7142514700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c06f050 0x7f713c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7142514700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713c070ee0 con 0x7f713c06e570 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7142514700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c071980 con 0x7f713c09eca0 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7142514700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c071480 con 0x7f713c06f050 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f713bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f713c06e570 0x7f713c06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50602/0 (socket says 172.21.15.32:50602) 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f713bfff700 1 -- 172.21.15.32:0/3030144421 learned_addr learned my addr 172.21.15.32:0/3030144421 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7140ab1700 1 --2- 172.21.15.32:0/3030144421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c06f050 0x7f713c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f713b7fe700 1 --2- 172.21.15.32:0/3030144421 >> v2:172.21.15.32:3300/0 conn(0x7f713c09eca0 0x7f713c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7140ab1700 1 -- 172.21.15.32:0/3030144421 >> v2:172.21.15.32:3300/0 conn(0x7f713c09eca0 msgr2=0x7f713c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7140ab1700 1 --2- 172.21.15.32:0/3030144421 >> v2:172.21.15.32:3300/0 conn(0x7f713c09eca0 0x7f713c13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7140ab1700 1 -- 172.21.15.32:0/3030144421 >> v1:172.21.15.32:6789/0 conn(0x7f713c06e570 legacy=0x7f713c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.629+0000 7f7140ab1700 1 -- 172.21.15.32:0/3030144421 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c071b90 con 0x7f713c06f050 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.630+0000 7f7140ab1700 1 --2- 172.21.15.32:0/3030144421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c06f050 0x7f713c13f3f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7134009d90 tx=0x7f713400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2999dde8c308ccad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.630+0000 7f713affd700 1 -- 172.21.15.32:0/3030144421 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713400e070 con 0x7f713c06f050 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.630+0000 7f713affd700 1 -- 172.21.15.32:0/3030144421 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713400bb20 con 0x7f713c06f050 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.630+0000 7f713affd700 1 -- 172.21.15.32:0/3030144421 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134012430 con 0x7f713c06f050 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 -- 172.21.15.32:0/3030144421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c06f050 msgr2=0x7f713c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 --2- 172.21.15.32:0/3030144421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c06f050 0x7f713c13f3f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7134009d90 tx=0x7f713400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 -- 172.21.15.32:0/3030144421 shutdown_connections 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 --2- 172.21.15.32:0/3030144421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c06f050 0x7f713c13f3f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 --2- 172.21.15.32:0/3030144421 >> v2:172.21.15.32:3300/0 conn(0x7f713c09eca0 0x7f713c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 -- 172.21.15.32:0/3030144421 >> 172.21.15.32:0/3030144421 conn(0x7f713c0a2aa0 msgr2=0x7f713c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 -- 172.21.15.32:0/3030144421 shutdown_connections 2023-09-30T22:23:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.631+0000 7f7142514700 1 -- 172.21.15.32:0/3030144421 wait complete. 2023-09-30T22:23:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.632+0000 7f7142514700 1 Processor -- start 2023-09-30T22:23:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.632+0000 7f7142514700 1 -- start start 2023-09-30T22:23:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7142514700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f713c06e570 0x7f713c1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7142514700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c06f050 0x7f713c121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7142514700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c121f60 0x7f713c1cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7142514700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c122f70 con 0x7f713c06f050 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7142514700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1248e0 con 0x7f713c121f60 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7142514700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f713c09a200 con 0x7f713c06e570 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f713bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f713c06e570 0x7f713c1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f713bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f713c06e570 0x7f713c1214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41834/0 (socket says 172.21.15.32:41834) 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f713bfff700 1 -- 172.21.15.32:0/4164966416 learned_addr learned my addr 172.21.15.32:0/4164966416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f713b7fe700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c06f050 0x7f713c121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7140ab1700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c121f60 0x7f713c1cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.633+0000 7f7140ab1700 1 -- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f713c06e570 msgr2=0x7f713c1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7140ab1700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f713c06e570 0x7f713c1214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7140ab1700 1 -- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c06f050 msgr2=0x7f713c121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7140ab1700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c06f050 0x7f713c121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7140ab1700 1 -- 172.21.15.32:0/4164966416 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134009a70 con 0x7f713c121f60 2023-09-30T22:23:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f713b7fe700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c06f050 0x7f713c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7140ab1700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c121f60 0x7f713c1cc9f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7134016000 tx=0x7f7134004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134012430 con 0x7f713c121f60 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7134020070 con 0x7f713c121f60 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c070610 con 0x7f713c121f60 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340098f0 con 0x7f713c121f60 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.634+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c13dbe0 con 0x7f713c121f60 2023-09-30T22:23:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.635+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f7134004040 con 0x7f713c121f60 2023-09-30T22:23:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.635+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c074ea0 con 0x7f713c121f60 2023-09-30T22:23:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.636+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4009+0+0 (secure 0 0 0) 0x7f7134022070 con 0x7f713c121f60 2023-09-30T22:23:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.641+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f713401a560 con 0x7f713c121f60 2023-09-30T22:23:55.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.745+0000 7f0b37f02700 1 Processor -- start 2023-09-30T22:23:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.745+0000 7f0b37f02700 1 -- start start 2023-09-30T22:23:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b37f02700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b300a0a30 0x7f0b300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b37f02700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b30133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b37f02700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3009edd0 con 0x7f0b30133fa0 2023-09-30T22:23:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b37f02700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3009e1f0 con 0x7f0b300a0a30 2023-09-30T22:23:55.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b37f02700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3009dc50 con 0x7f0b300a1340 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b3549d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b30133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b3649f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b30133fa0 0x7f0b3013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50612/0 (socket says 172.21.15.32:50612) 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b3649f700 1 -- 172.21.15.32:0/3606120571 learned_addr learned my addr 172.21.15.32:0/3606120571 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b35c9e700 1 --2- 172.21.15.32:0/3606120571 >> v2:172.21.15.32:3300/0 conn(0x7f0b300a0a30 0x7f0b300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b34c9c700 1 -- 172.21.15.32:0/3606120571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338156950 0 0) 0x7f0b3009edd0 con 0x7f0b30133fa0 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.746+0000 7f0b34c9c700 1 -- 172.21.15.32:0/3606120571 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b1c0035b0 con 0x7f0b30133fa0 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 msgr2=0x7f0b30133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b35c9e700 1 --2- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b30133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3606120571 >> v1:172.21.15.32:6789/0 conn(0x7f0b30133fa0 legacy=0x7f0b3013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3606120571 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300a1fe0 con 0x7f0b300a0a30 2023-09-30T22:23:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b35c9e700 1 --2- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b300a0e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009840 tx=0x7f0b2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bca0a3a0ba352fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b34c9c700 1 -- 172.21.15.32:0/3606120571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c011830 con 0x7f0b300a0a30 2023-09-30T22:23:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b34c9c700 1 -- 172.21.15.32:0/3606120571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c011990 con 0x7f0b300a0a30 2023-09-30T22:23:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.747+0000 7f0b34c9c700 1 -- 172.21.15.32:0/3606120571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c011d20 con 0x7f0b300a0a30 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 -- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 msgr2=0x7f0b300a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b300a0e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009840 tx=0x7f0b2c009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 -- 172.21.15.32:0/3606120571 shutdown_connections 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b300a0e00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3606120571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b30133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 -- 172.21.15.32:0/3606120571 >> 172.21.15.32:0/3606120571 conn(0x7f0b300a6550 msgr2=0x7f0b300a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 -- 172.21.15.32:0/3606120571 shutdown_connections 2023-09-30T22:23:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.748+0000 7f0b37f02700 1 -- 172.21.15.32:0/3606120571 wait complete. 2023-09-30T22:23:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.749+0000 7f0b37f02700 1 Processor -- start 2023-09-30T22:23:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.749+0000 7f0b37f02700 1 -- start start 2023-09-30T22:23:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b37f02700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b30123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b37f02700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b301238a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b37f02700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b30133fa0 0x7f0b301c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b37f02700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30125f50 con 0x7f0b300a0a30 2023-09-30T22:23:55.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b37f02700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300a3630 con 0x7f0b300a1340 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b37f02700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30124ba0 con 0x7f0b30133fa0 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b35c9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b30123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b35c9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b30123360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46600/0 (socket says 172.21.15.32:46600) 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b3649f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b30133fa0 0x7f0b301c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3988582215 learned_addr learned my addr 172.21.15.32:0/3988582215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.750+0000 7f0b3549d700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b301238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b30133fa0 msgr2=0x7f0b301c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b35c9e700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b30133fa0 0x7f0b301c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 msgr2=0x7f0b301238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b35c9e700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b301238a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b35c9e700 1 -- 172.21.15.32:0/3988582215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0094a0 con 0x7f0b300a0a30 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b3549d700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b301238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b3649f700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b30133fa0 0x7f0b301c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b35c9e700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b30123360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0090b0 tx=0x7f0b2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0036a0 con 0x7f0b300a0a30 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c01f070 con 0x7f0b300a0a30 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c003a60 con 0x7f0b300a0a30 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b30124680 con 0x7f0b300a0a30 2023-09-30T22:23:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.751+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b30126c80 con 0x7f0b300a0a30 2023-09-30T22:23:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.753+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b301260e0 con 0x7f0b300a0a30 2023-09-30T22:23:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.753+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f0b2c003bc0 con 0x7f0b300a0a30 2023-09-30T22:23:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.754+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4009+0+0 (secure 0 0 0) 0x7f0b2c021070 con 0x7f0b300a0a30 2023-09-30T22:23:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.755+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faab00a28e0 con 0x7faab00a1510 2023-09-30T22:23:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.756+0000 7faaa77fe700 1 -- 172.21.15.32:0/2887840530 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa98023790 con 0x7faab00a1510 2023-09-30T22:23:55.757 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.758+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 >> v1:172.21.15.32:6816/126553 conn(0x7faa9002f240 legacy=0x7faa900316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.758+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 msgr2=0x7faab01c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.758+0000 7faab4a49700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 0x7faab01c8cf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faa980098f0 tx=0x7faa98004750 comp rx=0 tx=0).stop 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 shutdown_connections 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab0139250 0x7faab0122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faab00a0a30 0x7faab01c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 --2- 172.21.15.32:0/2887840530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faab00a1510 0x7faab01c8cf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 >> 172.21.15.32:0/2887840530 conn(0x7faab00a6550 msgr2=0x7faab00a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 shutdown_connections 2023-09-30T22:23:55.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.759+0000 7faab4a49700 1 -- 172.21.15.32:0/2887840530 wait complete. 2023-09-30T22:23:55.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.760+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b2c024d50 con 0x7f0b300a0a30 2023-09-30T22:23:55.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.2 2023-09-30T22:23:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.852+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f713c06b9e0 con 0x7f713c121f60 2023-09-30T22:23:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.853+0000 7f7138ff9700 1 -- 172.21.15.32:0/4164966416 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f713401a380 con 0x7f713c121f60 2023-09-30T22:23:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:55.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 >> v1:172.21.15.32:6816/126553 conn(0x7f712802f270 legacy=0x7f7128031720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c121f60 msgr2=0x7f713c1cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c121f60 0x7f713c1cc9f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7134016000 tx=0x7f7134004ad0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 shutdown_connections 2023-09-30T22:23:55.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f713c121f60 0x7f713c1cc9f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f713c06e570 0x7f713c1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 --2- 172.21.15.32:0/4164966416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f713c06f050 0x7f713c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 >> 172.21.15.32:0/4164966416 conn(0x7f713c0a2aa0 msgr2=0x7f713c0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 shutdown_connections 2023-09-30T22:23:55.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.855+0000 7f7142514700 1 -- 172.21.15.32:0/4164966416 wait complete. 2023-09-30T22:23:55.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.860+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f90d406b600 con 0x7f90d409e000 2023-09-30T22:23:55.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.861+0000 7f90bbfff700 1 -- 172.21.15.32:0/1959223587 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90c8013720 con 0x7f90d409e000 2023-09-30T22:23:55.862 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:55.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 >> v1:172.21.15.32:6816/126553 conn(0x7f90c002f240 legacy=0x7f90c00316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 msgr2=0x7f90d41c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d41c8e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90c800ebe0 tx=0x7f90c800eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 shutdown_connections 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90d409e410 0x7f90d41c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90d409d6f0 0x7f90d41c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.863+0000 7f90d9889700 1 --2- 172.21.15.32:0/1959223587 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90d409e000 0x7f90d41c8e20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.864+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 >> 172.21.15.32:0/1959223587 conn(0x7f90d4066f50 msgr2=0x7f90d4070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.864+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 shutdown_connections 2023-09-30T22:23:55.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:55.864+0000 7f90d9889700 1 -- 172.21.15.32:0/1959223587 wait complete. 2023-09-30T22:23:55.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.3 2023-09-30T22:23:55.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.1 2023-09-30T22:23:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.006+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0b300a1d40 con 0x7f0b300a0a30 2023-09-30T22:23:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.006+0000 7f0b267fc700 1 -- 172.21.15.32:0/3988582215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b2c015020 con 0x7f0b300a0a30 2023-09-30T22:23:56.007 INFO:teuthology.orchestra.run.smithi032.stdout:38654705693 2023-09-30T22:23:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 >> v1:172.21.15.32:6816/126553 conn(0x7f0b1402f240 legacy=0x7f0b140316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:56.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 msgr2=0x7f0b30123360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b30123360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0090b0 tx=0x7f0b2c004750 comp rx=0 tx=0).stop 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 shutdown_connections 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b30133fa0 0x7f0b301c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b300a0a30 0x7f0b30123360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.008+0000 7f0b37f02700 1 --2- 172.21.15.32:0/3988582215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b300a1340 0x7f0b301238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.009+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 >> 172.21.15.32:0/3988582215 conn(0x7f0b300a6550 msgr2=0x7f0b3011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.009+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 shutdown_connections 2023-09-30T22:23:56.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.009+0000 7f0b37f02700 1 -- 172.21.15.32:0/3988582215 wait complete. 2023-09-30T22:23:56.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705693 for osd.0 2023-09-30T22:23:56.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:23:56.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:23:56.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:23:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 Processor -- start 2023-09-30T22:23:56.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 -- start start 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409d6f0 0x7f9e3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e410 0x7f9e34130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3409f130 con 0x7f9e3409e000 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3409f520 con 0x7f9e3409e410 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e39a93700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3409e930 con 0x7f9e3409d6f0 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409d6f0 0x7f9e3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e32ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e3409e000 0x7f9e34130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50624/0 (socket says 172.21.15.32:50624) 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.923+0000 7f9e32ffd700 1 -- 172.21.15.32:0/652362218 learned_addr learned my addr 172.21.15.32:0/652362218 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:56.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e33fff700 1 --2- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e410 0x7f9e34130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:56.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e327fc700 1 -- 172.21.15.32:0/652362218 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023624114 0 0) 0x7f9e3409f130 con 0x7f9e3409e000 2023-09-30T22:23:56.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e337fe700 1 -- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e410 msgr2=0x7f9e34130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:56.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e337fe700 1 --2- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e410 0x7f9e34130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e337fe700 1 -- 172.21.15.32:0/652362218 >> v1:172.21.15.32:6789/0 conn(0x7f9e3409e000 legacy=0x7f9e34130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:56.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e337fe700 1 -- 172.21.15.32:0/652362218 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3409b2d0 con 0x7f9e3409d6f0 2023-09-30T22:23:56.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e337fe700 1 --2- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409d6f0 0x7f9e3409dac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400bb10 tx=0x7f9e2400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fee5664311bcf8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:56.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e327fc700 1 -- 172.21.15.32:0/652362218 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400dc70 con 0x7f9e3409d6f0 2023-09-30T22:23:56.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e327fc700 1 -- 172.21.15.32:0/652362218 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2400ddd0 con 0x7f9e3409d6f0 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.924+0000 7f9e327fc700 1 -- 172.21.15.32:0/652362218 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24010690 con 0x7f9e3409d6f0 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 -- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409d6f0 msgr2=0x7f9e3409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 --2- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409d6f0 0x7f9e3409dac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400bb10 tx=0x7f9e2400be20 comp rx=0 tx=0).stop 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 -- 172.21.15.32:0/652362218 shutdown_connections 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 --2- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e410 0x7f9e34130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 --2- 172.21.15.32:0/652362218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409d6f0 0x7f9e3409dac0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 -- 172.21.15.32:0/652362218 >> 172.21.15.32:0/652362218 conn(0x7f9e34066f50 msgr2=0x7f9e3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 -- 172.21.15.32:0/652362218 shutdown_connections 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.925+0000 7f9e39a93700 1 -- 172.21.15.32:0/652362218 wait complete. 2023-09-30T22:23:56.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.927+0000 7f9e39a93700 1 Processor -- start 2023-09-30T22:23:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.927+0000 7f9e39a93700 1 -- start start 2023-09-30T22:23:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e39a93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 0x7f9e341223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e39a93700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e000 0x7f9e34122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e39a93700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409e410 0x7f9e34122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e39a93700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34125ad0 con 0x7f9e3409d6f0 2023-09-30T22:23:56.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e39a93700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3409c700 con 0x7f9e3409e000 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e39a93700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34123fb0 con 0x7f9e3409e410 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 0x7f9e341223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e000 0x7f9e34122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 0x7f9e341223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46626/0 (socket says 172.21.15.32:46626) 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.928+0000 7f9e337fe700 1 -- 172.21.15.32:0/2880381164 learned_addr learned my addr 172.21.15.32:0/2880381164 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e33fff700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409e410 0x7f9e34122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e32ffd700 1 -- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409e410 msgr2=0x7f9e34122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e32ffd700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409e410 0x7f9e34122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e32ffd700 1 -- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 msgr2=0x7f9e341223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e32ffd700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 0x7f9e341223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e32ffd700 1 -- 172.21.15.32:0/2880381164 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2400b740 con 0x7f9e3409e000 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e337fe700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 0x7f9e341223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e32ffd700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e000 0x7f9e34122930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e340a3e20 tx=0x7f9e2800c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004c80 con 0x7f9e3409e000 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28004de0 con 0x7f9e3409e000 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28007310 con 0x7f9e3409e000 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e34072640 con 0x7f9e3409e000 2023-09-30T22:23:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.929+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e34126540 con 0x7f9e3409e000 2023-09-30T22:23:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.930+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f9e28007470 con 0x7f9e3409e000 2023-09-30T22:23:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.930+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e34074ea0 con 0x7f9e3409e000 2023-09-30T22:23:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.931+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f9e2803f310 con 0x7f9e3409e000 2023-09-30T22:23:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:56.937+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e28007720 con 0x7f9e3409e000 2023-09-30T22:23:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.015+0000 7f154696d700 1 Processor -- start 2023-09-30T22:23:57.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.015+0000 7f154696d700 1 -- start start 2023-09-30T22:23:57.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f154696d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15401345a0 0x7f1540136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f154696d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1540136ec0 0x7f15401392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f154696d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1540099620 con 0x7f15400984f0 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f154696d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154009a9e0 con 0x7f15401345a0 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f154696d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f154009bce0 con 0x7f1540136ec0 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15401345a0 0x7f1540136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f1544f0a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1540136ec0 0x7f15401392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15401345a0 0x7f1540136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46640/0 (socket says 172.21.15.32:46640) 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 -- 172.21.15.32:0/2791765049 learned_addr learned my addr 172.21.15.32:0/2791765049 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 -- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1540136ec0 msgr2=0x7f15401392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 --2- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1540136ec0 0x7f15401392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 -- 172.21.15.32:0/2791765049 >> v1:172.21.15.32:6789/0 conn(0x7f15400984f0 legacy=0x7f15400988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.016+0000 7f153f7fe700 1 -- 172.21.15.32:0/2791765049 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154009aca0 con 0x7f15401345a0 2023-09-30T22:23:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.017+0000 7f153f7fe700 1 --2- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15401345a0 0x7f1540136980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f15340098f0 tx=0x7f1534009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892662f6c9bf16fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.017+0000 7f153effd700 1 -- 172.21.15.32:0/2791765049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400b8a0 con 0x7f15401345a0 2023-09-30T22:23:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.017+0000 7f153effd700 1 -- 172.21.15.32:0/2791765049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400ba00 con 0x7f15401345a0 2023-09-30T22:23:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.017+0000 7f153effd700 1 -- 172.21.15.32:0/2791765049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400bd90 con 0x7f15401345a0 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.017+0000 7f154696d700 1 -- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15401345a0 msgr2=0x7f1540136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.017+0000 7f154696d700 1 --2- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15401345a0 0x7f1540136980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f15340098f0 tx=0x7f1534009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.018+0000 7f154696d700 1 -- 172.21.15.32:0/2791765049 shutdown_connections 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.018+0000 7f154696d700 1 --2- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1540136ec0 0x7f15401392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.018+0000 7f154696d700 1 --2- 172.21.15.32:0/2791765049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15401345a0 0x7f1540136980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.018+0000 7f154696d700 1 -- 172.21.15.32:0/2791765049 >> 172.21.15.32:0/2791765049 conn(0x7f154009f470 msgr2=0x7f15400a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.018+0000 7f154696d700 1 -- 172.21.15.32:0/2791765049 shutdown_connections 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.018+0000 7f154696d700 1 -- 172.21.15.32:0/2791765049 wait complete. 2023-09-30T22:23:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.019+0000 7f154696d700 1 Processor -- start 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.019+0000 7f154696d700 1 -- start start 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.019+0000 7f154696d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15400984f0 0x7f15400974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.019+0000 7f154696d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 0x7f1540097a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f154696d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540136ec0 0x7f15401c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f154696d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154012ba60 con 0x7f1540136ec0 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f154696d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154006f610 con 0x7f15400984f0 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f154696d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f154012b7c0 con 0x7f15401345a0 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 0x7f1540097a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 0x7f1540097a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41896/0 (socket says 172.21.15.32:41896) 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 -- 172.21.15.32:0/2738500918 learned_addr learned my addr 172.21.15.32:0/2738500918 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f1544f0a700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540136ec0 0x7f15401c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 -- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15400984f0 msgr2=0x7f15400974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15400984f0 0x7f15400974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 -- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540136ec0 msgr2=0x7f15401c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540136ec0 0x7f15401c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 -- 172.21.15.32:0/2738500918 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534009550 con 0x7f15401345a0 2023-09-30T22:23:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153f7fe700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 0x7f1540097a20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1534009c40 tx=0x7f1534004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:23:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.020+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534004590 con 0x7f15401345a0 2023-09-30T22:23:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.021+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15340036a0 con 0x7f15401345a0 2023-09-30T22:23:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.021+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154006d700 con 0x7f15401345a0 2023-09-30T22:23:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.021+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534003a50 con 0x7f15401345a0 2023-09-30T22:23:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.021+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154012ec40 con 0x7f15401345a0 2023-09-30T22:23:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.022+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f1534021070 con 0x7f15401345a0 2023-09-30T22:23:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.022+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154009a610 con 0x7f15401345a0 2023-09-30T22:23:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.022+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f1534040ad0 con 0x7f15401345a0 2023-09-30T22:23:57.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.027+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f153401b430 con 0x7f15401345a0 2023-09-30T22:23:57.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.036+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f9e280136a0 con 0x7f9e3409e000 2023-09-30T22:23:57.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.042+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f153402dcf0 con 0x7f15401345a0 2023-09-30T22:23:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.066+0000 7f5826dc6700 1 Processor -- start 2023-09-30T22:23:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.066+0000 7f5826dc6700 1 -- start start 2023-09-30T22:23:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5826dc6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5820136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5826dc6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5820139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5826dc6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582009b4f0 con 0x7f5820131810 2023-09-30T22:23:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5826dc6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58201313c0 con 0x7f5820134630 2023-09-30T22:23:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5826dc6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f582006e8c0 con 0x7f5820136f50 2023-09-30T22:23:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5824b62700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5820131810 0x7f5820131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50636/0 (socket says 172.21.15.32:50636) 2023-09-30T22:23:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5824b62700 1 -- 172.21.15.32:0/3155037018 learned_addr learned my addr 172.21.15.32:0/3155037018 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.067+0000 7f5825363700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5820139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334342683 0 0) 0x7f582009b4f0 con 0x7f5820131810 2023-09-30T22:23:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58100035b0 con 0x7f5820131810 2023-09-30T22:23:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581ffff700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5820136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299597816 0 0) 0x7f58100035b0 con 0x7f5820131810 2023-09-30T22:23:57.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 msgr2=0x7f5820139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5820139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 msgr2=0x7f5820136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5820136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820071d20 con 0x7f5820131810 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1846968574 0 0) 0x7f5814001670 con 0x7f5820131810 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 >> v1:172.21.15.32:6789/0 conn(0x7f5820131810 legacy=0x7f5820131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5810003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f5810004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5810004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58201313c0 con 0x7f5810003f80 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582006e8c0 con 0x7f5820134630 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.068+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5810005350 con 0x7f5820136f50 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f581ffff700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f5810004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5810003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5825363700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5810004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 -- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 msgr2=0x7f5810004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5810004d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 -- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 msgr2=0x7f5810004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f5810004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 -- 172.21.15.32:0/3155037018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100035b0 con 0x7f5820134630 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f581ffff700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f5810004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f5824b62700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5810003a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f581400c320 tx=0x7f5810005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7638a35ea7fa97f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400ce60 con 0x7f5820134630 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814005960 con 0x7f5820134630 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.069+0000 7f581f7fe700 1 -- 172.21.15.32:0/3155037018 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814005cf0 con 0x7f5820134630 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 -- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 msgr2=0x7f5810003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5810003a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f581400c320 tx=0x7f5810005880 comp rx=0 tx=0).stop 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 -- 172.21.15.32:0/3155037018 shutdown_connections 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820136f50 0x7f5810004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f5810004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 --2- 172.21.15.32:0/3155037018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f5810003a00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 -- 172.21.15.32:0/3155037018 >> 172.21.15.32:0/3155037018 conn(0x7f58200a2aa0 msgr2=0x7f58200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 -- 172.21.15.32:0/3155037018 shutdown_connections 2023-09-30T22:23:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.070+0000 7f5826dc6700 1 -- 172.21.15.32:0/3155037018 wait complete. 2023-09-30T22:23:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.071+0000 7f5826dc6700 1 Processor -- start 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.071+0000 7f5826dc6700 1 -- start start 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5826dc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f582009c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5826dc6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820131810 0x7f582009c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5826dc6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f582009cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5826dc6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582011a0d0 con 0x7f5810003f80 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5826dc6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582009de50 con 0x7f5820134630 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5826dc6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58201191e0 con 0x7f5820131810 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5824b62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f582009c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5824b62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f582009c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46670/0 (socket says 172.21.15.32:46670) 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5824b62700 1 -- 172.21.15.32:0/2193611283 learned_addr learned my addr 172.21.15.32:0/2193611283 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.072+0000 7f5825363700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f582009cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f581ffff700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820131810 0x7f582009c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f5824b62700 1 -- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820131810 msgr2=0x7f582009c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f5824b62700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820131810 0x7f582009c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f5824b62700 1 -- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 msgr2=0x7f582009cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f5824b62700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f582009cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f5824b62700 1 -- 172.21.15.32:0/2193611283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582011a2d0 con 0x7f5810003f80 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f5824b62700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f582009c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f581400c910 tx=0x7f5814001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814001680 con 0x7f5810003f80 2023-09-30T22:23:57.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814007ba0 con 0x7f5810003f80 2023-09-30T22:23:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.073+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814004ab0 con 0x7f5810003f80 2023-09-30T22:23:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.074+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582012eb80 con 0x7f5810003f80 2023-09-30T22:23:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.074+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582006eb00 con 0x7f5810003f80 2023-09-30T22:23:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.075+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5820070d80 con 0x7f5810003f80 2023-09-30T22:23:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.075+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f5814004c10 con 0x7f5810003f80 2023-09-30T22:23:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.076+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f5814051e10 con 0x7f5810003f80 2023-09-30T22:23:57.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.085+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58140403a0 con 0x7f5810003f80 2023-09-30T22:23:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.155+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e3406c360 con 0x7f9e3409e000 2023-09-30T22:23:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.156+0000 7f9e1bfff700 1 -- 172.21.15.32:0/2880381164 <== mon.1 v2:172.21.15.32:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e28007720 con 0x7f9e3409e000 2023-09-30T22:23:57.157 INFO:teuthology.orchestra.run.smithi032.stdout:38654705695 2023-09-30T22:23:57.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 >> v1:172.21.15.32:6816/126553 conn(0x7f9e2002f240 legacy=0x7f9e200316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e000 msgr2=0x7f9e34122930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e000 0x7f9e34122930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e340a3e20 tx=0x7f9e2800c750 comp rx=0 tx=0).stop 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 shutdown_connections 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e3409e410 0x7f9e34122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3409d6f0 0x7f9e341223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 --2- 172.21.15.32:0/2880381164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3409e000 0x7f9e34122930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.159+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 >> 172.21.15.32:0/2880381164 conn(0x7f9e34066f50 msgr2=0x7f9e34070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.160+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 shutdown_connections 2023-09-30T22:23:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.160+0000 7f9e39a93700 1 -- 172.21.15.32:0/2880381164 wait complete. 2023-09-30T22:23:57.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705695 for osd.2 2023-09-30T22:23:57.180 DEBUG:teuthology.parallel:result is None 2023-09-30T22:23:57.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.186+0000 7fe31541f700 1 Processor -- start 2023-09-30T22:23:57.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.186+0000 7fe31541f700 1 -- start start 2023-09-30T22:23:57.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe31541f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe310130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe31541f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3100701a0 0x7fe310138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe31541f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe310130260 con 0x7fe31006f680 2023-09-30T22:23:57.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe31541f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31009a350 con 0x7fe3100701a0 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe31541f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe310071ff0 con 0x7fe310130670 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe30e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe31006f680 0x7fe31006fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50644/0 (socket says 172.21.15.32:50644) 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe30e7fc700 1 -- 172.21.15.32:0/373238048 learned_addr learned my addr 172.21.15.32:0/373238048 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe30f7fe700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3100701a0 0x7fe310138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.187+0000 7fe30effd700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe310130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30dffb700 1 -- 172.21.15.32:0/373238048 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620085683 0 0) 0x7fe310130260 con 0x7fe31006f680 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30dffb700 1 -- 172.21.15.32:0/373238048 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f80035b0 con 0x7fe31006f680 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30f7fe700 1 -- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 msgr2=0x7fe310130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30f7fe700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe310130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30f7fe700 1 -- 172.21.15.32:0/373238048 >> v1:172.21.15.32:6789/0 conn(0x7fe31006f680 legacy=0x7fe31006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30f7fe700 1 -- 172.21.15.32:0/373238048 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31009ecf0 con 0x7fe3100701a0 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30effd700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe310130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30f7fe700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3100701a0 0x7fe310138540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe30800be60 tx=0x7fe3080092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccdd29946438650e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30dffb700 1 -- 172.21.15.32:0/373238048 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800c890 con 0x7fe3100701a0 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30dffb700 1 -- 172.21.15.32:0/373238048 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30800c9f0 con 0x7fe3100701a0 2023-09-30T22:23:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.188+0000 7fe30dffb700 1 -- 172.21.15.32:0/373238048 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800cda0 con 0x7fe3100701a0 2023-09-30T22:23:57.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 -- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3100701a0 msgr2=0x7fe310138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3100701a0 0x7fe310138540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe30800be60 tx=0x7fe3080092f0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 -- 172.21.15.32:0/373238048 shutdown_connections 2023-09-30T22:23:57.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3100701a0 0x7fe310138540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 --2- 172.21.15.32:0/373238048 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe310130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 -- 172.21.15.32:0/373238048 >> 172.21.15.32:0/373238048 conn(0x7fe3100a2aa0 msgr2=0x7fe3100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 -- 172.21.15.32:0/373238048 shutdown_connections 2023-09-30T22:23:57.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.189+0000 7fe31541f700 1 -- 172.21.15.32:0/373238048 wait complete. 2023-09-30T22:23:57.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.191+0000 7fe31541f700 1 Processor -- start 2023-09-30T22:23:57.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.191+0000 7fe31541f700 1 -- start start 2023-09-30T22:23:57.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe31541f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 0x7fe31009af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe31541f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100701a0 0x7fe31009b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe31541f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe31009b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe31541f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31009cbf0 con 0x7fe3100701a0 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe31541f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe310118f20 con 0x7fe31006f680 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe31541f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe31009d100 con 0x7fe310130670 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 0x7fe31009af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 0x7fe31009af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52454/0 (socket says 172.21.15.32:52454) 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 -- 172.21.15.32:0/423705779 learned_addr learned my addr 172.21.15.32:0/423705779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30f7fe700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe31009b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30e7fc700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100701a0 0x7fe31009b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 -- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 msgr2=0x7fe31009b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe31009b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 -- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100701a0 msgr2=0x7fe31009b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100701a0 0x7fe31009b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.192+0000 7fe30effd700 1 -- 172.21.15.32:0/423705779 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30000d940 con 0x7fe31006f680 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.193+0000 7fe30effd700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 0x7fe31009af40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe300012fc0 tx=0x7fe3000103a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.193+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300010bb0 con 0x7fe31006f680 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.193+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300013040 con 0x7fe31006f680 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.193+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30800ba90 con 0x7fe31006f680 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.193+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000d2e0 con 0x7fe31006f680 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.193+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe310119200 con 0x7fe31006f680 2023-09-30T22:23:57.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.194+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31006fa90 con 0x7fe31006f680 2023-09-30T22:23:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.195+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7fe300004020 con 0x7fe31006f680 2023-09-30T22:23:57.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.195+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fe3000442c0 con 0x7fe31006f680 2023-09-30T22:23:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.199+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe300020550 con 0x7fe31006f680 2023-09-30T22:23:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.283+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15401317a0 con 0x7f15401345a0 2023-09-30T22:23:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.284+0000 7f153cff9700 1 -- 172.21.15.32:0/2738500918 <== mon.2 v2:172.21.15.32:3302/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1534015030 con 0x7f15401345a0 2023-09-30T22:23:57.285 INFO:teuthology.orchestra.run.smithi032.stdout:38654705695 2023-09-30T22:23:57.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 >> v1:172.21.15.32:6816/126553 conn(0x7f152c02f240 legacy=0x7f152c0316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 msgr2=0x7f1540097a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 0x7f1540097a20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1534009c40 tx=0x7f1534004750 comp rx=0 tx=0).stop 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 shutdown_connections 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1540136ec0 0x7f15401c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15400984f0 0x7f15400974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 --2- 172.21.15.32:0/2738500918 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15401345a0 0x7f1540097a20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.286+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 >> 172.21.15.32:0/2738500918 conn(0x7f154009f470 msgr2=0x7f154009c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.287+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 shutdown_connections 2023-09-30T22:23:57.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.287+0000 7f154696d700 1 -- 172.21.15.32:0/2738500918 wait complete. 2023-09-30T22:23:57.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705695 for osd.3 2023-09-30T22:23:57.301 DEBUG:teuthology.parallel:result is None 2023-09-30T22:23:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.308+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5820074ea0 con 0x7f5810003f80 2023-09-30T22:23:57.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.309+0000 7f581d7fa700 1 -- 172.21.15.32:0/2193611283 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f581403c090 con 0x7f5810003f80 2023-09-30T22:23:57.310 INFO:teuthology.orchestra.run.smithi032.stdout:38654705695 2023-09-30T22:23:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 >> v1:172.21.15.32:6816/126553 conn(0x7f580402f480 legacy=0x7f5804031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 msgr2=0x7f582009c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f582009c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f581400c910 tx=0x7f5814001a00 comp rx=0 tx=0).stop 2023-09-30T22:23:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 shutdown_connections 2023-09-30T22:23:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5820134630 0x7f582009cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5810003f80 0x7f582009c2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 --2- 172.21.15.32:0/2193611283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5820131810 0x7f582009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 >> 172.21.15.32:0/2193611283 conn(0x7f58200a2aa0 msgr2=0x7f5820133b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 shutdown_connections 2023-09-30T22:23:57.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.311+0000 7f5826dc6700 1 -- 172.21.15.32:0/2193611283 wait complete. 2023-09-30T22:23:57.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705695 for osd.1 2023-09-30T22:23:57.326 DEBUG:teuthology.parallel:result is None 2023-09-30T22:23:57.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.420+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe310074ea0 con 0x7fe31006f680 2023-09-30T22:23:57.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.420+0000 7fe2f77fe700 1 -- 172.21.15.32:0/423705779 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe30000dca0 con 0x7fe31006f680 2023-09-30T22:23:57.422 INFO:teuthology.orchestra.run.smithi032.stdout:38654705695 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 >> v1:172.21.15.32:6816/126553 conn(0x7fe2fc02f480 legacy=0x7fe2fc031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 msgr2=0x7fe31009af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 0x7fe31009af40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe300012fc0 tx=0x7fe3000103a0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 shutdown_connections 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe310130670 0x7fe31009b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31006f680 0x7fe31009af40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 --2- 172.21.15.32:0/423705779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3100701a0 0x7fe31009b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 >> 172.21.15.32:0/423705779 conn(0x7fe3100a2aa0 msgr2=0x7fe31012c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 shutdown_connections 2023-09-30T22:23:57.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.423+0000 7fe31541f700 1 -- 172.21.15.32:0/423705779 wait complete. 2023-09-30T22:23:57.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705694 got 38654705695 for osd.0 2023-09-30T22:23:57.437 DEBUG:teuthology.parallel:result is None 2023-09-30T22:23:57.437 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:23:57.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:23:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 Processor -- start 2023-09-30T22:23:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 -- start start 2023-09-30T22:23:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 0x7f5cd8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 0x7f5cd809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 0x7f5cd813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd806fef0 con 0x7f5cd809e760 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd806ef30 con 0x7f5cd81338a0 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.573+0000 7f5cdf9c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd81334b0 con 0x7f5cd809de50 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdcf63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 0x7f5cd809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cddf65700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 0x7f5cd813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 0x7f5cd8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 0x7f5cd8133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52462/0 (socket says 172.21.15.32:52462) 2023-09-30T22:23:57.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdcf63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 0x7f5cd809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41944/0 (socket says 172.21.15.32:41944) 2023-09-30T22:23:57.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 -- 172.21.15.32:0/2994029024 learned_addr learned my addr 172.21.15.32:0/2994029024 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cddf65700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 0x7f5cd813a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46676/0 (socket says 172.21.15.32:46676) 2023-09-30T22:23:57.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 -- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 msgr2=0x7f5cd809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 --2- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 0x7f5cd809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 -- 172.21.15.32:0/2994029024 >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 msgr2=0x7f5cd813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 --2- 172.21.15.32:0/2994029024 >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 0x7f5cd813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 -- 172.21.15.32:0/2994029024 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd806fc30 con 0x7f5cd81338a0 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cddf65700 1 --2- 172.21.15.32:0/2994029024 >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 0x7f5cd813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdcf63700 1 --2- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 0x7f5cd809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5cdd764700 1 --2- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 0x7f5cd8133c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009920 tx=0x7f5cc8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d299568bd7a010fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5ccffff700 1 -- 172.21.15.32:0/2994029024 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8011830 con 0x7f5cd81338a0 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5ccffff700 1 -- 172.21.15.32:0/2994029024 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8011990 con 0x7f5cd81338a0 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.574+0000 7f5ccffff700 1 -- 172.21.15.32:0/2994029024 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8011d20 con 0x7f5cd81338a0 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 msgr2=0x7f5cd8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 0x7f5cd8133c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009920 tx=0x7f5cc8009c30 comp rx=0 tx=0).stop 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/2994029024 shutdown_connections 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd81338a0 0x7f5cd8133c70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/2994029024 >> v2:172.21.15.32:3300/0 conn(0x7f5cd809e760 0x7f5cd813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/2994029024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd809de50 0x7f5cd809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/2994029024 >> 172.21.15.32:0/2994029024 conn(0x7f5cd80a2aa0 msgr2=0x7f5cd80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/2994029024 shutdown_connections 2023-09-30T22:23:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.575+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/2994029024 wait complete. 2023-09-30T22:23:57.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.576+0000 7f5cdf9c8700 1 Processor -- start 2023-09-30T22:23:57.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.576+0000 7f5cdf9c8700 1 -- start start 2023-09-30T22:23:57.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdf9c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 0x7f5cd811eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdf9c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd809e760 0x7f5cd811f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdf9c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd81338a0 0x7f5cd811f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdf9c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd812fad0 con 0x7f5cd809e760 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdf9c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd806f640 con 0x7f5cd809de50 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdf9c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd81222e0 con 0x7f5cd81338a0 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 0x7f5cd811eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdcf63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd809e760 0x7f5cd811f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 0x7f5cd811eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52466/0 (socket says 172.21.15.32:52466) 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 -- 172.21.15.32:0/1637083395 learned_addr learned my addr 172.21.15.32:0/1637083395 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cddf65700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd81338a0 0x7f5cd811f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 -- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd81338a0 msgr2=0x7f5cd811f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd81338a0 0x7f5cd811f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 -- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd809e760 msgr2=0x7f5cd811f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd809e760 0x7f5cd811f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 -- 172.21.15.32:0/1637083395 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40094a0 con 0x7f5cd809de50 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cddf65700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd81338a0 0x7f5cd811f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdcf63700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd809e760 0x7f5cd811f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5cdd764700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 0x7f5cd811eaf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009c70 tx=0x7f5cc8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.577+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004450 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.578+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc80045b0 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.578+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd0009a70 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.578+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80038e0 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.578+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8009550 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.579+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f5cc8003a40 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.579+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f5cc8040a30 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.579+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd81207c0 con 0x7f5cd809de50 2023-09-30T22:23:57.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.584+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5cc8028080 con 0x7f5cd809de50 2023-09-30T22:23:57.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.783+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 --> v1:172.21.15.32:6816/126553 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5cd8066680 con 0x7f5cbc02f480 2023-09-30T22:23:57.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.791+0000 7f5ccdffb700 1 -- 172.21.15.32:0/1637083395 <== mgr.14100 v1:172.21.15.32:6816/126553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270497 (unknown 2965378022 0 1694662702) 0x7f5cd4009c20 con 0x7f5cbc02f480 2023-09-30T22:23:57.795 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 >> v1:172.21.15.32:6816/126553 conn(0x7f5cbc02f480 legacy=0x7f5cbc031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 msgr2=0x7f5cd811eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 0x7f5cd811eaf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009c70 tx=0x7f5cc8004750 comp rx=0 tx=0).stop 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 shutdown_connections 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd81338a0 0x7f5cd811f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd809de50 0x7f5cd811eaf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 --2- 172.21.15.32:0/1637083395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd809e760 0x7f5cd811f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 >> 172.21.15.32:0/1637083395 conn(0x7f5cd80a2aa0 msgr2=0x7f5cd809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 shutdown_connections 2023-09-30T22:23:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.795+0000 7f5cdf9c8700 1 -- 172.21.15.32:0/1637083395 wait complete. 2023-09-30T22:23:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T22:23:57.809 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2023-09-30T22:23:56.031223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189136,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298224,"statfs":{"total":386547056640,"available":385329381376,"internally_reserved":0,"allocated":770048,"data_stored":262506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.995555"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081376+0000","last_change":"2023-09-30T22:21:37.213744+0000","last_active":"2023-09-30T22:23:54.081376+0000","last_peered":"2023-09-30T22:23:54.081376+0000","last_clean":"2023-09-30T22:23:54.081376+0000","last_became_active":"2023-09-30T22:21:37.213432+0000","last_became_peered":"2023-09-30T22:21:37.213432+0000","last_unstale":"2023-09-30T22:23:54.081376+0000","last_undegraded":"2023-09-30T22:23:54.081376+0000","last_fullsized":"2023-09-30T22:23:54.081376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081947+0000","last_change":"2023-09-30T22:21:37.209966+0000","last_active":"2023-09-30T22:23:54.081947+0000","last_peered":"2023-09-30T22:23:54.081947+0000","last_clean":"2023-09-30T22:23:54.081947+0000","last_became_active":"2023-09-30T22:21:37.209700+0000","last_became_peered":"2023-09-30T22:21:37.209700+0000","last_unstale":"2023-09-30T22:23:54.081947+0000","last_undegraded":"2023-09-30T22:23:54.081947+0000","last_fullsized":"2023-09-30T22:23:54.081947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795123+0000","last_change":"2023-09-30T22:21:37.227972+0000","last_active":"2023-09-30T22:22:26.795123+0000","last_peered":"2023-09-30T22:22:26.795123+0000","last_clean":"2023-09-30T22:22:26.795123+0000","last_became_active":"2023-09-30T22:21:37.227698+0000","last_became_peered":"2023-09-30T22:21:37.227698+0000","last_unstale":"2023-09-30T22:22:26.795123+0000","last_undegraded":"2023-09-30T22:22:26.795123+0000","last_fullsized":"2023-09-30T22:22:26.795123+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794740+0000","last_change":"2023-09-30T22:21:37.217682+0000","last_active":"2023-09-30T22:22:26.794740+0000","last_peered":"2023-09-30T22:22:26.794740+0000","last_clean":"2023-09-30T22:22:26.794740+0000","last_became_active":"2023-09-30T22:21:37.217395+0000","last_became_peered":"2023-09-30T22:21:37.217395+0000","last_unstale":"2023-09-30T22:22:26.794740+0000","last_undegraded":"2023-09-30T22:22:26.794740+0000","last_fullsized":"2023-09-30T22:22:26.794740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.805052+0000","last_change":"2023-09-30T22:22:26.804304+0000","last_active":"2023-09-30T22:22:26.805052+0000","last_peered":"2023-09-30T22:22:26.805052+0000","last_clean":"2023-09-30T22:22:26.805052+0000","last_became_active":"2023-09-30T22:22:26.803956+0000","last_became_peered":"2023-09-30T22:22:26.803956+0000","last_unstale":"2023-09-30T22:22:26.805052+0000","last_undegraded":"2023-09-30T22:22:26.805052+0000","last_fullsized":"2023-09-30T22:22:26.805052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159825+0000","last_change":"2023-09-30T22:21:37.224923+0000","last_active":"2023-09-30T22:22:27.159825+0000","last_peered":"2023-09-30T22:22:27.159825+0000","last_clean":"2023-09-30T22:22:27.159825+0000","last_became_active":"2023-09-30T22:21:37.224651+0000","last_became_peered":"2023-09-30T22:21:37.224651+0000","last_unstale":"2023-09-30T22:22:27.159825+0000","last_undegraded":"2023-09-30T22:22:27.159825+0000","last_fullsized":"2023-09-30T22:22:27.159825+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159533+0000","last_change":"2023-09-30T22:21:37.220818+0000","last_active":"2023-09-30T22:22:27.159533+0000","last_peered":"2023-09-30T22:22:27.159533+0000","last_clean":"2023-09-30T22:22:27.159533+0000","last_became_active":"2023-09-30T22:21:37.217825+0000","last_became_peered":"2023-09-30T22:21:37.217825+0000","last_unstale":"2023-09-30T22:22:27.159533+0000","last_undegraded":"2023-09-30T22:22:27.159533+0000","last_fullsized":"2023-09-30T22:22:27.159533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793118+0000","last_change":"2023-09-30T22:21:37.237512+0000","last_active":"2023-09-30T22:22:26.793118+0000","last_peered":"2023-09-30T22:22:26.793118+0000","last_clean":"2023-09-30T22:22:26.793118+0000","last_became_active":"2023-09-30T22:21:37.237217+0000","last_became_peered":"2023-09-30T22:21:37.237217+0000","last_unstale":"2023-09-30T22:22:26.793118+0000","last_undegraded":"2023-09-30T22:22:26.793118+0000","last_fullsized":"2023-09-30T22:22:26.793118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796106+0000","last_change":"2023-09-30T22:21:37.231447+0000","last_active":"2023-09-30T22:22:26.796106+0000","last_peered":"2023-09-30T22:22:26.796106+0000","last_clean":"2023-09-30T22:22:26.796106+0000","last_became_active":"2023-09-30T22:21:37.231106+0000","last_became_peered":"2023-09-30T22:21:37.231106+0000","last_unstale":"2023-09-30T22:22:26.796106+0000","last_undegraded":"2023-09-30T22:22:26.796106+0000","last_fullsized":"2023-09-30T22:22:26.796106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794341+0000","last_change":"2023-09-30T22:21:37.238710+0000","last_active":"2023-09-30T22:22:26.794341+0000","last_peered":"2023-09-30T22:22:26.794341+0000","last_clean":"2023-09-30T22:22:26.794341+0000","last_became_active":"2023-09-30T22:21:37.237230+0000","last_became_peered":"2023-09-30T22:21:37.237230+0000","last_unstale":"2023-09-30T22:22:26.794341+0000","last_undegraded":"2023-09-30T22:22:26.794341+0000","last_fullsized":"2023-09-30T22:22:26.794341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793934+0000","last_change":"2023-09-30T22:21:37.238711+0000","last_active":"2023-09-30T22:22:26.793934+0000","last_peered":"2023-09-30T22:22:26.793934+0000","last_clean":"2023-09-30T22:22:26.793934+0000","last_became_active":"2023-09-30T22:21:37.237399+0000","last_became_peered":"2023-09-30T22:21:37.237399+0000","last_unstale":"2023-09-30T22:22:26.793934+0000","last_undegraded":"2023-09-30T22:22:26.793934+0000","last_fullsized":"2023-09-30T22:22:26.793934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081964+0000","last_change":"2023-09-30T22:21:37.215868+0000","last_active":"2023-09-30T22:23:54.081964+0000","last_peered":"2023-09-30T22:23:54.081964+0000","last_clean":"2023-09-30T22:23:54.081964+0000","last_became_active":"2023-09-30T22:21:37.215601+0000","last_became_peered":"2023-09-30T22:21:37.215601+0000","last_unstale":"2023-09-30T22:23:54.081964+0000","last_undegraded":"2023-09-30T22:23:54.081964+0000","last_fullsized":"2023-09-30T22:23:54.081964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081177+0000","last_change":"2023-09-30T22:21:37.229175+0000","last_active":"2023-09-30T22:23:54.081177+0000","last_peered":"2023-09-30T22:23:54.081177+0000","last_clean":"2023-09-30T22:23:54.081177+0000","last_became_active":"2023-09-30T22:21:37.226712+0000","last_became_peered":"2023-09-30T22:21:37.226712+0000","last_unstale":"2023-09-30T22:23:54.081177+0000","last_undegraded":"2023-09-30T22:23:54.081177+0000","last_fullsized":"2023-09-30T22:23:54.081177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794716+0000","last_change":"2023-09-30T22:21:37.222596+0000","last_active":"2023-09-30T22:22:26.794716+0000","last_peered":"2023-09-30T22:22:26.794716+0000","last_clean":"2023-09-30T22:22:26.794716+0000","last_became_active":"2023-09-30T22:21:37.222295+0000","last_became_peered":"2023-09-30T22:21:37.222295+0000","last_unstale":"2023-09-30T22:22:26.794716+0000","last_undegraded":"2023-09-30T22:22:26.794716+0000","last_fullsized":"2023-09-30T22:22:26.794716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.805515+0000","last_change":"2023-09-30T22:22:26.804397+0000","last_active":"2023-09-30T22:22:26.805515+0000","last_peered":"2023-09-30T22:22:26.805515+0000","last_clean":"2023-09-30T22:22:26.805515+0000","last_became_active":"2023-09-30T22:22:26.804064+0000","last_became_peered":"2023-09-30T22:22:26.804064+0000","last_unstale":"2023-09-30T22:22:26.805515+0000","last_undegraded":"2023-09-30T22:22:26.805515+0000","last_fullsized":"2023-09-30T22:22:26.805515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159159+0000","last_change":"2023-09-30T22:21:37.220367+0000","last_active":"2023-09-30T22:22:27.159159+0000","last_peered":"2023-09-30T22:22:27.159159+0000","last_clean":"2023-09-30T22:22:27.159159+0000","last_became_active":"2023-09-30T22:21:37.217445+0000","last_became_peered":"2023-09-30T22:21:37.217445+0000","last_unstale":"2023-09-30T22:22:27.159159+0000","last_undegraded":"2023-09-30T22:22:27.159159+0000","last_fullsized":"2023-09-30T22:22:27.159159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796112+0000","last_change":"2023-09-30T22:21:37.231390+0000","last_active":"2023-09-30T22:22:26.796112+0000","last_peered":"2023-09-30T22:22:26.796112+0000","last_clean":"2023-09-30T22:22:26.796112+0000","last_became_active":"2023-09-30T22:21:37.231116+0000","last_became_peered":"2023-09-30T22:21:37.231116+0000","last_unstale":"2023-09-30T22:22:26.796112+0000","last_undegraded":"2023-09-30T22:22:26.796112+0000","last_fullsized":"2023-09-30T22:22:26.796112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795532+0000","last_change":"2023-09-30T22:21:37.228923+0000","last_active":"2023-09-30T22:22:26.795532+0000","last_peered":"2023-09-30T22:22:26.795532+0000","last_clean":"2023-09-30T22:22:26.795532+0000","last_became_active":"2023-09-30T22:21:37.220140+0000","last_became_peered":"2023-09-30T22:21:37.220140+0000","last_unstale":"2023-09-30T22:22:26.795532+0000","last_undegraded":"2023-09-30T22:22:26.795532+0000","last_fullsized":"2023-09-30T22:22:26.795532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080992+0000","last_change":"2023-09-30T22:22:26.807130+0000","last_active":"2023-09-30T22:23:54.080992+0000","last_peered":"2023-09-30T22:23:54.080992+0000","last_clean":"2023-09-30T22:23:54.080992+0000","last_became_active":"2023-09-30T22:22:26.806744+0000","last_became_peered":"2023-09-30T22:22:26.806744+0000","last_unstale":"2023-09-30T22:23:54.080992+0000","last_undegraded":"2023-09-30T22:23:54.080992+0000","last_fullsized":"2023-09-30T22:23:54.080992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793712+0000","last_change":"2023-09-30T22:21:37.218484+0000","last_active":"2023-09-30T22:22:26.793712+0000","last_peered":"2023-09-30T22:22:26.793712+0000","last_clean":"2023-09-30T22:22:26.793712+0000","last_became_active":"2023-09-30T22:21:37.218141+0000","last_became_peered":"2023-09-30T22:21:37.218141+0000","last_unstale":"2023-09-30T22:22:26.793712+0000","last_undegraded":"2023-09-30T22:22:26.793712+0000","last_fullsized":"2023-09-30T22:22:26.793712+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.792740+0000","last_change":"2023-09-30T22:21:37.216211+0000","last_active":"2023-09-30T22:22:26.792740+0000","last_peered":"2023-09-30T22:22:26.792740+0000","last_clean":"2023-09-30T22:22:26.792740+0000","last_became_active":"2023-09-30T22:21:37.215922+0000","last_became_peered":"2023-09-30T22:21:37.215922+0000","last_unstale":"2023-09-30T22:22:26.792740+0000","last_undegraded":"2023-09-30T22:22:26.792740+0000","last_fullsized":"2023-09-30T22:22:26.792740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794805+0000","last_change":"2023-09-30T22:21:37.227137+0000","last_active":"2023-09-30T22:22:26.794805+0000","last_peered":"2023-09-30T22:22:26.794805+0000","last_clean":"2023-09-30T22:22:26.794805+0000","last_became_active":"2023-09-30T22:21:37.226848+0000","last_became_peered":"2023-09-30T22:21:37.226848+0000","last_unstale":"2023-09-30T22:22:26.794805+0000","last_undegraded":"2023-09-30T22:22:26.794805+0000","last_fullsized":"2023-09-30T22:22:26.794805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159441+0000","last_change":"2023-09-30T22:21:37.220415+0000","last_active":"2023-09-30T22:22:27.159441+0000","last_peered":"2023-09-30T22:22:27.159441+0000","last_clean":"2023-09-30T22:22:27.159441+0000","last_became_active":"2023-09-30T22:21:37.217468+0000","last_became_peered":"2023-09-30T22:21:37.217468+0000","last_unstale":"2023-09-30T22:22:27.159441+0000","last_undegraded":"2023-09-30T22:22:27.159441+0000","last_fullsized":"2023-09-30T22:22:27.159441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793174+0000","last_change":"2023-09-30T22:21:37.220204+0000","last_active":"2023-09-30T22:22:26.793174+0000","last_peered":"2023-09-30T22:22:26.793174+0000","last_clean":"2023-09-30T22:22:26.793174+0000","last_became_active":"2023-09-30T22:21:37.219099+0000","last_became_peered":"2023-09-30T22:21:37.219099+0000","last_unstale":"2023-09-30T22:22:26.793174+0000","last_undegraded":"2023-09-30T22:22:26.793174+0000","last_fullsized":"2023-09-30T22:22:26.793174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794335+0000","last_change":"2023-09-30T22:21:37.217211+0000","last_active":"2023-09-30T22:22:26.794335+0000","last_peered":"2023-09-30T22:22:26.794335+0000","last_clean":"2023-09-30T22:22:26.794335+0000","last_became_active":"2023-09-30T22:21:37.216537+0000","last_became_peered":"2023-09-30T22:21:37.216537+0000","last_unstale":"2023-09-30T22:22:26.794335+0000","last_undegraded":"2023-09-30T22:22:26.794335+0000","last_fullsized":"2023-09-30T22:22:26.794335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794175+0000","last_change":"2023-09-30T22:21:37.209836+0000","last_active":"2023-09-30T22:22:26.794175+0000","last_peered":"2023-09-30T22:22:26.794175+0000","last_clean":"2023-09-30T22:22:26.794175+0000","last_became_active":"2023-09-30T22:21:37.209561+0000","last_became_peered":"2023-09-30T22:21:37.209561+0000","last_unstale":"2023-09-30T22:22:26.794175+0000","last_undegraded":"2023-09-30T22:22:26.794175+0000","last_fullsized":"2023-09-30T22:22:26.794175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160229+0000","last_change":"2023-09-30T22:21:37.220637+0000","last_active":"2023-09-30T22:22:27.160229+0000","last_peered":"2023-09-30T22:22:27.160229+0000","last_clean":"2023-09-30T22:22:27.160229+0000","last_became_active":"2023-09-30T22:21:37.217741+0000","last_became_peered":"2023-09-30T22:21:37.217741+0000","last_unstale":"2023-09-30T22:22:27.160229+0000","last_undegraded":"2023-09-30T22:22:27.160229+0000","last_fullsized":"2023-09-30T22:22:27.160229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794000+0000","last_change":"2023-09-30T22:21:36.800514+0000","last_active":"2023-09-30T22:22:26.794000+0000","last_peered":"2023-09-30T22:22:26.794000+0000","last_clean":"2023-09-30T22:22:26.794000+0000","last_became_active":"2023-09-30T22:21:36.799775+0000","last_became_peered":"2023-09-30T22:21:36.799775+0000","last_unstale":"2023-09-30T22:22:26.794000+0000","last_undegraded":"2023-09-30T22:22:26.794000+0000","last_fullsized":"2023-09-30T22:22:26.794000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160233+0000","last_change":"2023-09-30T22:21:37.211849+0000","last_active":"2023-09-30T22:22:27.160233+0000","last_peered":"2023-09-30T22:22:27.160233+0000","last_clean":"2023-09-30T22:22:27.160233+0000","last_became_active":"2023-09-30T22:21:37.209128+0000","last_became_peered":"2023-09-30T22:21:37.209128+0000","last_unstale":"2023-09-30T22:22:27.160233+0000","last_undegraded":"2023-09-30T22:22:27.160233+0000","last_fullsized":"2023-09-30T22:22:27.160233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.792774+0000","last_change":"2023-09-30T22:21:36.803467+0000","last_active":"2023-09-30T22:22:26.792774+0000","last_peered":"2023-09-30T22:22:26.792774+0000","last_clean":"2023-09-30T22:22:26.792774+0000","last_became_active":"2023-09-30T22:21:36.803141+0000","last_became_peered":"2023-09-30T22:21:36.803141+0000","last_unstale":"2023-09-30T22:22:26.792774+0000","last_undegraded":"2023-09-30T22:22:26.792774+0000","last_fullsized":"2023-09-30T22:22:26.792774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795643+0000","last_change":"2023-09-30T22:21:37.224566+0000","last_active":"2023-09-30T22:22:26.795643+0000","last_peered":"2023-09-30T22:22:26.795643+0000","last_clean":"2023-09-30T22:22:26.795643+0000","last_became_active":"2023-09-30T22:21:37.219843+0000","last_became_peered":"2023-09-30T22:21:37.219843+0000","last_unstale":"2023-09-30T22:22:26.795643+0000","last_undegraded":"2023-09-30T22:22:26.795643+0000","last_fullsized":"2023-09-30T22:22:26.795643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159449+0000","last_change":"2023-09-30T22:21:36.813865+0000","last_active":"2023-09-30T22:22:27.159449+0000","last_peered":"2023-09-30T22:22:27.159449+0000","last_clean":"2023-09-30T22:22:27.159449+0000","last_became_active":"2023-09-30T22:21:36.812871+0000","last_became_peered":"2023-09-30T22:21:36.812871+0000","last_unstale":"2023-09-30T22:22:27.159449+0000","last_undegraded":"2023-09-30T22:22:27.159449+0000","last_fullsized":"2023-09-30T22:22:27.159449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160854+0000","last_change":"2023-09-30T22:21:37.215838+0000","last_active":"2023-09-30T22:22:27.160854+0000","last_peered":"2023-09-30T22:22:27.160854+0000","last_clean":"2023-09-30T22:22:27.160854+0000","last_became_active":"2023-09-30T22:21:37.215314+0000","last_became_peered":"2023-09-30T22:21:37.215314+0000","last_unstale":"2023-09-30T22:22:27.160854+0000","last_undegraded":"2023-09-30T22:22:27.160854+0000","last_fullsized":"2023-09-30T22:22:27.160854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795977+0000","last_change":"2023-09-30T22:21:36.809280+0000","last_active":"2023-09-30T22:22:26.795977+0000","last_peered":"2023-09-30T22:22:26.795977+0000","last_clean":"2023-09-30T22:22:26.795977+0000","last_became_active":"2023-09-30T22:21:36.808975+0000","last_became_peered":"2023-09-30T22:21:36.808975+0000","last_unstale":"2023-09-30T22:22:26.795977+0000","last_undegraded":"2023-09-30T22:22:26.795977+0000","last_fullsized":"2023-09-30T22:22:26.795977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159548+0000","last_change":"2023-09-30T22:21:37.215010+0000","last_active":"2023-09-30T22:22:27.159548+0000","last_peered":"2023-09-30T22:22:27.159548+0000","last_clean":"2023-09-30T22:22:27.159548+0000","last_became_active":"2023-09-30T22:21:37.212913+0000","last_became_peered":"2023-09-30T22:21:37.212913+0000","last_unstale":"2023-09-30T22:22:27.159548+0000","last_undegraded":"2023-09-30T22:22:27.159548+0000","last_fullsized":"2023-09-30T22:22:27.159548+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081718+0000","last_change":"2023-09-30T22:21:36.800431+0000","last_active":"2023-09-30T22:23:54.081718+0000","last_peered":"2023-09-30T22:23:54.081718+0000","last_clean":"2023-09-30T22:23:54.081718+0000","last_became_active":"2023-09-30T22:21:36.800130+0000","last_became_peered":"2023-09-30T22:21:36.800130+0000","last_unstale":"2023-09-30T22:23:54.081718+0000","last_undegraded":"2023-09-30T22:23:54.081718+0000","last_fullsized":"2023-09-30T22:23:54.081718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079809+0000","last_change":"2023-09-30T22:22:26.805968+0000","last_active":"2023-09-30T22:23:54.079809+0000","last_peered":"2023-09-30T22:23:54.079809+0000","last_clean":"2023-09-30T22:23:54.079809+0000","last_became_active":"2023-09-30T22:22:26.805589+0000","last_became_peered":"2023-09-30T22:22:26.805589+0000","last_unstale":"2023-09-30T22:23:54.079809+0000","last_undegraded":"2023-09-30T22:23:54.079809+0000","last_fullsized":"2023-09-30T22:23:54.079809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795126+0000","last_change":"2023-09-30T22:21:36.810931+0000","last_active":"2023-09-30T22:22:26.795126+0000","last_peered":"2023-09-30T22:22:26.795126+0000","last_clean":"2023-09-30T22:22:26.795126+0000","last_became_active":"2023-09-30T22:21:36.810625+0000","last_became_peered":"2023-09-30T22:21:36.810625+0000","last_unstale":"2023-09-30T22:22:26.795126+0000","last_undegraded":"2023-09-30T22:22:26.795126+0000","last_fullsized":"2023-09-30T22:22:26.795126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794629+0000","last_change":"2023-09-30T22:21:37.217701+0000","last_active":"2023-09-30T22:22:26.794629+0000","last_peered":"2023-09-30T22:22:26.794629+0000","last_clean":"2023-09-30T22:22:26.794629+0000","last_became_active":"2023-09-30T22:21:37.217076+0000","last_became_peered":"2023-09-30T22:21:37.217076+0000","last_unstale":"2023-09-30T22:22:26.794629+0000","last_undegraded":"2023-09-30T22:22:26.794629+0000","last_fullsized":"2023-09-30T22:22:26.794629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.114456+0000","last_change":"2023-09-30T22:21:36.811297+0000","last_active":"2023-09-30T22:23:54.114456+0000","last_peered":"2023-09-30T22:23:54.114456+0000","last_clean":"2023-09-30T22:23:54.114456+0000","last_became_active":"2023-09-30T22:21:36.810987+0000","last_became_peered":"2023-09-30T22:21:36.810987+0000","last_unstale":"2023-09-30T22:23:54.114456+0000","last_undegraded":"2023-09-30T22:23:54.114456+0000","last_fullsized":"2023-09-30T22:23:54.114456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081038+0000","last_change":"2023-09-30T22:21:37.228789+0000","last_active":"2023-09-30T22:23:54.081038+0000","last_peered":"2023-09-30T22:23:54.081038+0000","last_clean":"2023-09-30T22:23:54.081038+0000","last_became_active":"2023-09-30T22:21:37.225220+0000","last_became_peered":"2023-09-30T22:21:37.225220+0000","last_unstale":"2023-09-30T22:23:54.081038+0000","last_undegraded":"2023-09-30T22:23:54.081038+0000","last_fullsized":"2023-09-30T22:23:54.081038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159758+0000","last_change":"2023-09-30T22:21:36.813567+0000","last_active":"2023-09-30T22:22:27.159758+0000","last_peered":"2023-09-30T22:22:27.159758+0000","last_clean":"2023-09-30T22:22:27.159758+0000","last_became_active":"2023-09-30T22:21:36.813257+0000","last_became_peered":"2023-09-30T22:21:36.813257+0000","last_unstale":"2023-09-30T22:22:27.159758+0000","last_undegraded":"2023-09-30T22:22:27.159758+0000","last_fullsized":"2023-09-30T22:22:27.159758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081909+0000","last_change":"2023-09-30T22:22:26.801274+0000","last_active":"2023-09-30T22:23:54.081909+0000","last_peered":"2023-09-30T22:23:54.081909+0000","last_clean":"2023-09-30T22:23:54.081909+0000","last_became_active":"2023-09-30T22:22:26.800913+0000","last_became_peered":"2023-09-30T22:22:26.800913+0000","last_unstale":"2023-09-30T22:23:54.081909+0000","last_undegraded":"2023-09-30T22:23:54.081909+0000","last_fullsized":"2023-09-30T22:23:54.081909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794763+0000","last_change":"2023-09-30T22:21:36.806015+0000","last_active":"2023-09-30T22:22:26.794763+0000","last_peered":"2023-09-30T22:22:26.794763+0000","last_clean":"2023-09-30T22:22:26.794763+0000","last_became_active":"2023-09-30T22:21:36.805679+0000","last_became_peered":"2023-09-30T22:21:36.805679+0000","last_unstale":"2023-09-30T22:22:26.794763+0000","last_undegraded":"2023-09-30T22:22:26.794763+0000","last_fullsized":"2023-09-30T22:22:26.794763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160321+0000","last_change":"2023-09-30T22:21:37.221673+0000","last_active":"2023-09-30T22:22:27.160321+0000","last_peered":"2023-09-30T22:22:27.160321+0000","last_clean":"2023-09-30T22:22:27.160321+0000","last_became_active":"2023-09-30T22:21:37.221388+0000","last_became_peered":"2023-09-30T22:21:37.221388+0000","last_unstale":"2023-09-30T22:22:27.160321+0000","last_undegraded":"2023-09-30T22:22:27.160321+0000","last_fullsized":"2023-09-30T22:22:27.160321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159822+0000","last_change":"2023-09-30T22:21:36.812303+0000","last_active":"2023-09-30T22:22:27.159822+0000","last_peered":"2023-09-30T22:22:27.159822+0000","last_clean":"2023-09-30T22:22:27.159822+0000","last_became_active":"2023-09-30T22:21:36.811887+0000","last_became_peered":"2023-09-30T22:21:36.811887+0000","last_unstale":"2023-09-30T22:22:27.159822+0000","last_undegraded":"2023-09-30T22:22:27.159822+0000","last_fullsized":"2023-09-30T22:22:27.159822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795724+0000","last_change":"2023-09-30T22:21:37.225272+0000","last_active":"2023-09-30T22:22:26.795724+0000","last_peered":"2023-09-30T22:22:26.795724+0000","last_clean":"2023-09-30T22:22:26.795724+0000","last_became_active":"2023-09-30T22:21:37.224827+0000","last_became_peered":"2023-09-30T22:21:37.224827+0000","last_unstale":"2023-09-30T22:22:26.795724+0000","last_undegraded":"2023-09-30T22:22:26.795724+0000","last_fullsized":"2023-09-30T22:22:26.795724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793478+0000","last_change":"2023-09-30T22:21:36.804342+0000","last_active":"2023-09-30T22:22:26.793478+0000","last_peered":"2023-09-30T22:22:26.793478+0000","last_clean":"2023-09-30T22:22:26.793478+0000","last_became_active":"2023-09-30T22:21:36.804002+0000","last_became_peered":"2023-09-30T22:21:36.804002+0000","last_unstale":"2023-09-30T22:22:26.793478+0000","last_undegraded":"2023-09-30T22:22:26.793478+0000","last_fullsized":"2023-09-30T22:22:26.793478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160697+0000","last_change":"2023-09-30T22:21:37.215043+0000","last_active":"2023-09-30T22:22:27.160697+0000","last_peered":"2023-09-30T22:22:27.160697+0000","last_clean":"2023-09-30T22:22:27.160697+0000","last_became_active":"2023-09-30T22:21:37.214750+0000","last_became_peered":"2023-09-30T22:21:37.214750+0000","last_unstale":"2023-09-30T22:22:27.160697+0000","last_undegraded":"2023-09-30T22:22:27.160697+0000","last_fullsized":"2023-09-30T22:22:27.160697+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795845+0000","last_change":"2023-09-30T22:21:36.796103+0000","last_active":"2023-09-30T22:22:26.795845+0000","last_peered":"2023-09-30T22:22:26.795845+0000","last_clean":"2023-09-30T22:22:26.795845+0000","last_became_active":"2023-09-30T22:21:36.795669+0000","last_became_peered":"2023-09-30T22:21:36.795669+0000","last_unstale":"2023-09-30T22:22:26.795845+0000","last_undegraded":"2023-09-30T22:22:26.795845+0000","last_fullsized":"2023-09-30T22:22:26.795845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159638+0000","last_change":"2023-09-30T22:21:37.223782+0000","last_active":"2023-09-30T22:22:27.159638+0000","last_peered":"2023-09-30T22:22:27.159638+0000","last_clean":"2023-09-30T22:22:27.159638+0000","last_became_active":"2023-09-30T22:21:37.223508+0000","last_became_peered":"2023-09-30T22:21:37.223508+0000","last_unstale":"2023-09-30T22:22:27.159638+0000","last_undegraded":"2023-09-30T22:22:27.159638+0000","last_fullsized":"2023-09-30T22:22:27.159638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081790+0000","last_change":"2023-09-30T22:21:36.798241+0000","last_active":"2023-09-30T22:23:54.081790+0000","last_peered":"2023-09-30T22:23:54.081790+0000","last_clean":"2023-09-30T22:23:54.081790+0000","last_became_active":"2023-09-30T22:21:36.797923+0000","last_became_peered":"2023-09-30T22:21:36.797923+0000","last_unstale":"2023-09-30T22:23:54.081790+0000","last_undegraded":"2023-09-30T22:23:54.081790+0000","last_fullsized":"2023-09-30T22:23:54.081790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.792827+0000","last_change":"2023-09-30T22:21:37.223841+0000","last_active":"2023-09-30T22:22:26.792827+0000","last_peered":"2023-09-30T22:22:26.792827+0000","last_clean":"2023-09-30T22:22:26.792827+0000","last_became_active":"2023-09-30T22:21:37.223485+0000","last_became_peered":"2023-09-30T22:21:37.223485+0000","last_unstale":"2023-09-30T22:22:26.792827+0000","last_undegraded":"2023-09-30T22:22:26.792827+0000","last_fullsized":"2023-09-30T22:22:26.792827+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160433+0000","last_change":"2023-09-30T22:21:36.817374+0000","last_active":"2023-09-30T22:22:27.160433+0000","last_peered":"2023-09-30T22:22:27.160433+0000","last_clean":"2023-09-30T22:22:27.160433+0000","last_became_active":"2023-09-30T22:21:36.816913+0000","last_became_peered":"2023-09-30T22:21:36.816913+0000","last_unstale":"2023-09-30T22:22:27.160433+0000","last_undegraded":"2023-09-30T22:22:27.160433+0000","last_fullsized":"2023-09-30T22:22:27.160433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080845+0000","last_change":"2023-09-30T22:21:37.214893+0000","last_active":"2023-09-30T22:23:54.080845+0000","last_peered":"2023-09-30T22:23:54.080845+0000","last_clean":"2023-09-30T22:23:54.080845+0000","last_became_active":"2023-09-30T22:21:37.214616+0000","last_became_peered":"2023-09-30T22:21:37.214616+0000","last_unstale":"2023-09-30T22:23:54.080845+0000","last_undegraded":"2023-09-30T22:23:54.080845+0000","last_fullsized":"2023-09-30T22:23:54.080845+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795120+0000","last_change":"2023-09-30T22:21:36.806594+0000","last_active":"2023-09-30T22:22:26.795120+0000","last_peered":"2023-09-30T22:22:26.795120+0000","last_clean":"2023-09-30T22:22:26.795120+0000","last_became_active":"2023-09-30T22:21:36.806305+0000","last_became_peered":"2023-09-30T22:21:36.806305+0000","last_unstale":"2023-09-30T22:22:26.795120+0000","last_undegraded":"2023-09-30T22:22:26.795120+0000","last_fullsized":"2023-09-30T22:22:26.795120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796168+0000","last_change":"2023-09-30T22:21:37.224756+0000","last_active":"2023-09-30T22:22:26.796168+0000","last_peered":"2023-09-30T22:22:26.796168+0000","last_clean":"2023-09-30T22:22:26.796168+0000","last_became_active":"2023-09-30T22:21:37.224444+0000","last_became_peered":"2023-09-30T22:21:37.224444+0000","last_unstale":"2023-09-30T22:22:26.796168+0000","last_undegraded":"2023-09-30T22:22:26.796168+0000","last_fullsized":"2023-09-30T22:22:26.796168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159654+0000","last_change":"2023-09-30T22:21:36.813533+0000","last_active":"2023-09-30T22:22:27.159654+0000","last_peered":"2023-09-30T22:22:27.159654+0000","last_clean":"2023-09-30T22:22:27.159654+0000","last_became_active":"2023-09-30T22:21:36.813227+0000","last_became_peered":"2023-09-30T22:21:36.813227+0000","last_unstale":"2023-09-30T22:22:27.159654+0000","last_undegraded":"2023-09-30T22:22:27.159654+0000","last_fullsized":"2023-09-30T22:22:27.159654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795194+0000","last_change":"2023-09-30T22:21:37.230482+0000","last_active":"2023-09-30T22:22:26.795194+0000","last_peered":"2023-09-30T22:22:26.795194+0000","last_clean":"2023-09-30T22:22:26.795194+0000","last_became_active":"2023-09-30T22:21:37.229876+0000","last_became_peered":"2023-09-30T22:21:37.229876+0000","last_unstale":"2023-09-30T22:22:26.795194+0000","last_undegraded":"2023-09-30T22:22:26.795194+0000","last_fullsized":"2023-09-30T22:22:26.795194+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159586+0000","last_change":"2023-09-30T22:21:36.811138+0000","last_active":"2023-09-30T22:22:27.159586+0000","last_peered":"2023-09-30T22:22:27.159586+0000","last_clean":"2023-09-30T22:22:27.159586+0000","last_became_active":"2023-09-30T22:21:36.810834+0000","last_became_peered":"2023-09-30T22:21:36.810834+0000","last_unstale":"2023-09-30T22:22:27.159586+0000","last_undegraded":"2023-09-30T22:22:27.159586+0000","last_fullsized":"2023-09-30T22:22:27.159586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080896+0000","last_change":"2023-09-30T22:22:26.800907+0000","last_active":"2023-09-30T22:23:54.080896+0000","last_peered":"2023-09-30T22:23:54.080896+0000","last_clean":"2023-09-30T22:23:54.080896+0000","last_became_active":"2023-09-30T22:22:26.800169+0000","last_became_peered":"2023-09-30T22:22:26.800169+0000","last_unstale":"2023-09-30T22:23:54.080896+0000","last_undegraded":"2023-09-30T22:23:54.080896+0000","last_fullsized":"2023-09-30T22:23:54.080896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159614+0000","last_change":"2023-09-30T22:21:36.812156+0000","last_active":"2023-09-30T22:22:27.159614+0000","last_peered":"2023-09-30T22:22:27.159614+0000","last_clean":"2023-09-30T22:22:27.159614+0000","last_became_active":"2023-09-30T22:21:36.811527+0000","last_became_peered":"2023-09-30T22:21:36.811527+0000","last_unstale":"2023-09-30T22:22:27.159614+0000","last_undegraded":"2023-09-30T22:22:27.159614+0000","last_fullsized":"2023-09-30T22:22:27.159614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081868+0000","last_change":"2023-09-30T22:21:37.226310+0000","last_active":"2023-09-30T22:23:54.081868+0000","last_peered":"2023-09-30T22:23:54.081868+0000","last_clean":"2023-09-30T22:23:54.081868+0000","last_became_active":"2023-09-30T22:21:37.225749+0000","last_became_peered":"2023-09-30T22:21:37.225749+0000","last_unstale":"2023-09-30T22:23:54.081868+0000","last_undegraded":"2023-09-30T22:23:54.081868+0000","last_fullsized":"2023-09-30T22:23:54.081868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159157+0000","last_change":"2023-09-30T22:21:36.811756+0000","last_active":"2023-09-30T22:22:27.159157+0000","last_peered":"2023-09-30T22:22:27.159157+0000","last_clean":"2023-09-30T22:22:27.159157+0000","last_became_active":"2023-09-30T22:21:36.811393+0000","last_became_peered":"2023-09-30T22:21:36.811393+0000","last_unstale":"2023-09-30T22:22:27.159157+0000","last_undegraded":"2023-09-30T22:22:27.159157+0000","last_fullsized":"2023-09-30T22:22:27.159157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794401+0000","last_change":"2023-09-30T22:21:37.217010+0000","last_active":"2023-09-30T22:22:26.794401+0000","last_peered":"2023-09-30T22:22:26.794401+0000","last_clean":"2023-09-30T22:22:26.794401+0000","last_became_active":"2023-09-30T22:21:37.216071+0000","last_became_peered":"2023-09-30T22:21:37.216071+0000","last_unstale":"2023-09-30T22:22:26.794401+0000","last_undegraded":"2023-09-30T22:22:26.794401+0000","last_fullsized":"2023-09-30T22:22:26.794401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793630+0000","last_change":"2023-09-30T22:21:31.794405+0000","last_active":"2023-09-30T22:22:26.793630+0000","last_peered":"2023-09-30T22:22:26.793630+0000","last_clean":"2023-09-30T22:22:26.793630+0000","last_became_active":"2023-09-30T22:21:31.794052+0000","last_became_peered":"2023-09-30T22:21:31.794052+0000","last_unstale":"2023-09-30T22:22:26.793630+0000","last_undegraded":"2023-09-30T22:22:26.793630+0000","last_fullsized":"2023-09-30T22:22:26.793630+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080943+0000","last_change":"2023-09-30T22:21:36.799995+0000","last_active":"2023-09-30T22:23:54.080943+0000","last_peered":"2023-09-30T22:23:54.080943+0000","last_clean":"2023-09-30T22:23:54.080943+0000","last_became_active":"2023-09-30T22:21:36.799674+0000","last_became_peered":"2023-09-30T22:21:36.799674+0000","last_unstale":"2023-09-30T22:23:54.080943+0000","last_undegraded":"2023-09-30T22:23:54.080943+0000","last_fullsized":"2023-09-30T22:23:54.080943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.805177+0000","last_change":"2023-09-30T22:22:26.804169+0000","last_active":"2023-09-30T22:22:26.805177+0000","last_peered":"2023-09-30T22:22:26.805177+0000","last_clean":"2023-09-30T22:22:26.805177+0000","last_became_active":"2023-09-30T22:22:26.803794+0000","last_became_peered":"2023-09-30T22:22:26.803794+0000","last_unstale":"2023-09-30T22:22:26.805177+0000","last_undegraded":"2023-09-30T22:22:26.805177+0000","last_fullsized":"2023-09-30T22:22:26.805177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159668+0000","last_change":"2023-09-30T22:21:31.798962+0000","last_active":"2023-09-30T22:22:27.159668+0000","last_peered":"2023-09-30T22:22:27.159668+0000","last_clean":"2023-09-30T22:22:27.159668+0000","last_became_active":"2023-09-30T22:21:31.798641+0000","last_became_peered":"2023-09-30T22:21:31.798641+0000","last_unstale":"2023-09-30T22:22:27.159668+0000","last_undegraded":"2023-09-30T22:22:27.159668+0000","last_fullsized":"2023-09-30T22:22:27.159668+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795852+0000","last_change":"2023-09-30T22:21:36.793285+0000","last_active":"2023-09-30T22:22:26.795852+0000","last_peered":"2023-09-30T22:22:26.795852+0000","last_clean":"2023-09-30T22:22:26.795852+0000","last_became_active":"2023-09-30T22:21:36.793005+0000","last_became_peered":"2023-09-30T22:21:36.793005+0000","last_unstale":"2023-09-30T22:22:26.795852+0000","last_undegraded":"2023-09-30T22:22:26.795852+0000","last_fullsized":"2023-09-30T22:22:26.795852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793308+0000","last_change":"2023-09-30T22:21:37.236712+0000","last_active":"2023-09-30T22:22:26.793308+0000","last_peered":"2023-09-30T22:22:26.793308+0000","last_clean":"2023-09-30T22:22:26.793308+0000","last_became_active":"2023-09-30T22:21:37.226418+0000","last_became_peered":"2023-09-30T22:21:37.226418+0000","last_unstale":"2023-09-30T22:22:26.793308+0000","last_undegraded":"2023-09-30T22:22:26.793308+0000","last_fullsized":"2023-09-30T22:22:26.793308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079884+0000","last_change":"2023-09-30T22:21:31.797737+0000","last_active":"2023-09-30T22:23:54.079884+0000","last_peered":"2023-09-30T22:23:54.079884+0000","last_clean":"2023-09-30T22:23:54.079884+0000","last_became_active":"2023-09-30T22:21:31.797396+0000","last_became_peered":"2023-09-30T22:21:31.797396+0000","last_unstale":"2023-09-30T22:23:54.079884+0000","last_undegraded":"2023-09-30T22:23:54.079884+0000","last_fullsized":"2023-09-30T22:23:54.079884+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793312+0000","last_change":"2023-09-30T22:21:36.799423+0000","last_active":"2023-09-30T22:22:26.793312+0000","last_peered":"2023-09-30T22:22:26.793312+0000","last_clean":"2023-09-30T22:22:26.793312+0000","last_became_active":"2023-09-30T22:21:36.798817+0000","last_became_peered":"2023-09-30T22:21:36.798817+0000","last_unstale":"2023-09-30T22:22:26.793312+0000","last_undegraded":"2023-09-30T22:22:26.793312+0000","last_fullsized":"2023-09-30T22:22:26.793312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794246+0000","last_change":"2023-09-30T22:21:37.237647+0000","last_active":"2023-09-30T22:22:26.794246+0000","last_peered":"2023-09-30T22:22:26.794246+0000","last_clean":"2023-09-30T22:22:26.794246+0000","last_became_active":"2023-09-30T22:21:37.237287+0000","last_became_peered":"2023-09-30T22:21:37.237287+0000","last_unstale":"2023-09-30T22:22:26.794246+0000","last_undegraded":"2023-09-30T22:22:26.794246+0000","last_fullsized":"2023-09-30T22:22:26.794246+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159690+0000","last_change":"2023-09-30T22:21:31.798294+0000","last_active":"2023-09-30T22:22:27.159690+0000","last_peered":"2023-09-30T22:22:27.159690+0000","last_clean":"2023-09-30T22:22:27.159690+0000","last_became_active":"2023-09-30T22:21:31.797999+0000","last_became_peered":"2023-09-30T22:21:31.797999+0000","last_unstale":"2023-09-30T22:22:27.159690+0000","last_undegraded":"2023-09-30T22:22:27.159690+0000","last_fullsized":"2023-09-30T22:22:27.159690+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793223+0000","last_change":"2023-09-30T22:21:30.796302+0000","last_active":"2023-09-30T22:22:26.793223+0000","last_peered":"2023-09-30T22:22:26.793223+0000","last_clean":"2023-09-30T22:22:26.793223+0000","last_became_active":"2023-09-30T22:21:30.795885+0000","last_became_peered":"2023-09-30T22:21:30.795885+0000","last_unstale":"2023-09-30T22:22:26.793223+0000","last_undegraded":"2023-09-30T22:22:26.793223+0000","last_fullsized":"2023-09-30T22:22:26.793223+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_clean_scrub_stamp":"2023-09-30T22:21:29.785382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159913+0000","last_change":"2023-09-30T22:21:36.811325+0000","last_active":"2023-09-30T22:22:27.159913+0000","last_peered":"2023-09-30T22:22:27.159913+0000","last_clean":"2023-09-30T22:22:27.159913+0000","last_became_active":"2023-09-30T22:21:36.811014+0000","last_became_peered":"2023-09-30T22:21:36.811014+0000","last_unstale":"2023-09-30T22:22:27.159913+0000","last_undegraded":"2023-09-30T22:22:27.159913+0000","last_fullsized":"2023-09-30T22:22:27.159913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160664+0000","last_change":"2023-09-30T22:21:37.220230+0000","last_active":"2023-09-30T22:22:27.160664+0000","last_peered":"2023-09-30T22:22:27.160664+0000","last_clean":"2023-09-30T22:22:27.160664+0000","last_became_active":"2023-09-30T22:21:37.215165+0000","last_became_peered":"2023-09-30T22:21:37.215165+0000","last_unstale":"2023-09-30T22:22:27.160664+0000","last_undegraded":"2023-09-30T22:22:27.160664+0000","last_fullsized":"2023-09-30T22:22:27.160664+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159277+0000","last_change":"2023-09-30T22:21:31.798735+0000","last_active":"2023-09-30T22:22:27.159277+0000","last_peered":"2023-09-30T22:22:27.159277+0000","last_clean":"2023-09-30T22:22:27.159277+0000","last_became_active":"2023-09-30T22:21:31.798405+0000","last_became_peered":"2023-09-30T22:21:31.798405+0000","last_unstale":"2023-09-30T22:22:27.159277+0000","last_undegraded":"2023-09-30T22:22:27.159277+0000","last_fullsized":"2023-09-30T22:22:27.159277+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795327+0000","last_change":"2023-09-30T22:21:36.793235+0000","last_active":"2023-09-30T22:22:26.795327+0000","last_peered":"2023-09-30T22:22:26.795327+0000","last_clean":"2023-09-30T22:22:26.795327+0000","last_became_active":"2023-09-30T22:21:36.792921+0000","last_became_peered":"2023-09-30T22:21:36.792921+0000","last_unstale":"2023-09-30T22:22:26.795327+0000","last_undegraded":"2023-09-30T22:22:26.795327+0000","last_fullsized":"2023-09-30T22:22:26.795327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795776+0000","last_change":"2023-09-30T22:21:37.224362+0000","last_active":"2023-09-30T22:22:26.795776+0000","last_peered":"2023-09-30T22:22:26.795776+0000","last_clean":"2023-09-30T22:22:26.795776+0000","last_became_active":"2023-09-30T22:21:37.219539+0000","last_became_peered":"2023-09-30T22:21:37.219539+0000","last_unstale":"2023-09-30T22:22:26.795776+0000","last_undegraded":"2023-09-30T22:22:26.795776+0000","last_fullsized":"2023-09-30T22:22:26.795776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160545+0000","last_change":"2023-09-30T22:21:31.800121+0000","last_active":"2023-09-30T22:22:27.160545+0000","last_peered":"2023-09-30T22:22:27.160545+0000","last_clean":"2023-09-30T22:22:27.160545+0000","last_became_active":"2023-09-30T22:21:31.799775+0000","last_became_peered":"2023-09-30T22:21:31.799775+0000","last_unstale":"2023-09-30T22:22:27.160545+0000","last_undegraded":"2023-09-30T22:22:27.160545+0000","last_fullsized":"2023-09-30T22:22:27.160545+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793211+0000","last_change":"2023-09-30T22:21:36.799552+0000","last_active":"2023-09-30T22:22:26.793211+0000","last_peered":"2023-09-30T22:22:26.793211+0000","last_clean":"2023-09-30T22:22:26.793211+0000","last_became_active":"2023-09-30T22:21:36.798979+0000","last_became_peered":"2023-09-30T22:21:36.798979+0000","last_unstale":"2023-09-30T22:22:26.793211+0000","last_undegraded":"2023-09-30T22:22:26.793211+0000","last_fullsized":"2023-09-30T22:22:26.793211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796109+0000","last_change":"2023-09-30T22:21:37.227699+0000","last_active":"2023-09-30T22:22:26.796109+0000","last_peered":"2023-09-30T22:22:26.796109+0000","last_clean":"2023-09-30T22:22:26.796109+0000","last_became_active":"2023-09-30T22:21:37.227403+0000","last_became_peered":"2023-09-30T22:21:37.227403+0000","last_unstale":"2023-09-30T22:22:26.796109+0000","last_undegraded":"2023-09-30T22:22:26.796109+0000","last_fullsized":"2023-09-30T22:22:26.796109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160571+0000","last_change":"2023-09-30T22:21:36.817294+0000","last_active":"2023-09-30T22:22:27.160571+0000","last_peered":"2023-09-30T22:22:27.160571+0000","last_clean":"2023-09-30T22:22:27.160571+0000","last_became_active":"2023-09-30T22:21:36.816805+0000","last_became_peered":"2023-09-30T22:21:36.816805+0000","last_unstale":"2023-09-30T22:22:27.160571+0000","last_undegraded":"2023-09-30T22:22:27.160571+0000","last_fullsized":"2023-09-30T22:22:27.160571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794258+0000","last_change":"2023-09-30T22:21:31.794560+0000","last_active":"2023-09-30T22:22:26.794258+0000","last_peered":"2023-09-30T22:22:26.794258+0000","last_clean":"2023-09-30T22:22:26.794258+0000","last_became_active":"2023-09-30T22:21:31.794101+0000","last_became_peered":"2023-09-30T22:21:31.794101+0000","last_unstale":"2023-09-30T22:22:26.794258+0000","last_undegraded":"2023-09-30T22:22:26.794258+0000","last_fullsized":"2023-09-30T22:22:26.794258+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159841+0000","last_change":"2023-09-30T22:21:37.214706+0000","last_active":"2023-09-30T22:22:27.159841+0000","last_peered":"2023-09-30T22:22:27.159841+0000","last_clean":"2023-09-30T22:22:27.159841+0000","last_became_active":"2023-09-30T22:21:37.212393+0000","last_became_peered":"2023-09-30T22:21:37.212393+0000","last_unstale":"2023-09-30T22:22:27.159841+0000","last_undegraded":"2023-09-30T22:22:27.159841+0000","last_fullsized":"2023-09-30T22:22:27.159841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160625+0000","last_change":"2023-09-30T22:21:31.800079+0000","last_active":"2023-09-30T22:22:27.160625+0000","last_peered":"2023-09-30T22:22:27.160625+0000","last_clean":"2023-09-30T22:22:27.160625+0000","last_became_active":"2023-09-30T22:21:31.799718+0000","last_became_peered":"2023-09-30T22:21:31.799718+0000","last_unstale":"2023-09-30T22:22:27.160625+0000","last_undegraded":"2023-09-30T22:22:27.160625+0000","last_fullsized":"2023-09-30T22:22:27.160625+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795588+0000","last_change":"2023-09-30T22:21:36.803700+0000","last_active":"2023-09-30T22:22:26.795588+0000","last_peered":"2023-09-30T22:22:26.795588+0000","last_clean":"2023-09-30T22:22:26.795588+0000","last_became_active":"2023-09-30T22:21:36.803365+0000","last_became_peered":"2023-09-30T22:21:36.803365+0000","last_unstale":"2023-09-30T22:22:26.795588+0000","last_undegraded":"2023-09-30T22:22:26.795588+0000","last_fullsized":"2023-09-30T22:22:26.795588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079977+0000","last_change":"2023-09-30T22:21:37.226913+0000","last_active":"2023-09-30T22:23:54.079977+0000","last_peered":"2023-09-30T22:23:54.079977+0000","last_clean":"2023-09-30T22:23:54.079977+0000","last_became_active":"2023-09-30T22:21:37.225725+0000","last_became_peered":"2023-09-30T22:21:37.225725+0000","last_unstale":"2023-09-30T22:23:54.079977+0000","last_undegraded":"2023-09-30T22:23:54.079977+0000","last_fullsized":"2023-09-30T22:23:54.079977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795979+0000","last_change":"2023-09-30T22:21:36.808872+0000","last_active":"2023-09-30T22:22:26.795979+0000","last_peered":"2023-09-30T22:22:26.795979+0000","last_clean":"2023-09-30T22:22:26.795979+0000","last_became_active":"2023-09-30T22:21:36.806238+0000","last_became_peered":"2023-09-30T22:21:36.806238+0000","last_unstale":"2023-09-30T22:22:26.795979+0000","last_undegraded":"2023-09-30T22:22:26.795979+0000","last_fullsized":"2023-09-30T22:22:26.795979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.082007+0000","last_change":"2023-09-30T22:21:37.208459+0000","last_active":"2023-09-30T22:23:54.082007+0000","last_peered":"2023-09-30T22:23:54.082007+0000","last_clean":"2023-09-30T22:23:54.082007+0000","last_became_active":"2023-09-30T22:21:37.208076+0000","last_became_peered":"2023-09-30T22:21:37.208076+0000","last_unstale":"2023-09-30T22:23:54.082007+0000","last_undegraded":"2023-09-30T22:23:54.082007+0000","last_fullsized":"2023-09-30T22:23:54.082007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793418+0000","last_change":"2023-09-30T22:21:36.806969+0000","last_active":"2023-09-30T22:22:26.793418+0000","last_peered":"2023-09-30T22:22:26.793418+0000","last_clean":"2023-09-30T22:22:26.793418+0000","last_became_active":"2023-09-30T22:21:36.806649+0000","last_became_peered":"2023-09-30T22:21:36.806649+0000","last_unstale":"2023-09-30T22:22:26.793418+0000","last_undegraded":"2023-09-30T22:22:26.793418+0000","last_fullsized":"2023-09-30T22:22:26.793418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081108+0000","last_change":"2023-09-30T22:21:37.229486+0000","last_active":"2023-09-30T22:23:54.081108+0000","last_peered":"2023-09-30T22:23:54.081108+0000","last_clean":"2023-09-30T22:23:54.081108+0000","last_became_active":"2023-09-30T22:21:37.226308+0000","last_became_peered":"2023-09-30T22:21:37.226308+0000","last_unstale":"2023-09-30T22:23:54.081108+0000","last_undegraded":"2023-09-30T22:23:54.081108+0000","last_fullsized":"2023-09-30T22:23:54.081108+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160557+0000","last_change":"2023-09-30T22:21:36.815782+0000","last_active":"2023-09-30T22:22:27.160557+0000","last_peered":"2023-09-30T22:22:27.160557+0000","last_clean":"2023-09-30T22:22:27.160557+0000","last_became_active":"2023-09-30T22:21:36.815431+0000","last_became_peered":"2023-09-30T22:21:36.815431+0000","last_unstale":"2023-09-30T22:22:27.160557+0000","last_undegraded":"2023-09-30T22:22:27.160557+0000","last_fullsized":"2023-09-30T22:22:27.160557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080810+0000","last_change":"2023-09-30T22:21:37.215011+0000","last_active":"2023-09-30T22:23:54.080810+0000","last_peered":"2023-09-30T22:23:54.080810+0000","last_clean":"2023-09-30T22:23:54.080810+0000","last_became_active":"2023-09-30T22:21:37.214739+0000","last_became_peered":"2023-09-30T22:21:37.214739+0000","last_unstale":"2023-09-30T22:23:54.080810+0000","last_undegraded":"2023-09-30T22:23:54.080810+0000","last_fullsized":"2023-09-30T22:23:54.080810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160345+0000","last_change":"2023-09-30T22:21:36.812958+0000","last_active":"2023-09-30T22:22:27.160345+0000","last_peered":"2023-09-30T22:22:27.160345+0000","last_clean":"2023-09-30T22:22:27.160345+0000","last_became_active":"2023-09-30T22:21:36.812662+0000","last_became_peered":"2023-09-30T22:21:36.812662+0000","last_unstale":"2023-09-30T22:22:27.160345+0000","last_undegraded":"2023-09-30T22:22:27.160345+0000","last_fullsized":"2023-09-30T22:22:27.160345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160001+0000","last_change":"2023-09-30T22:21:37.225020+0000","last_active":"2023-09-30T22:22:27.160001+0000","last_peered":"2023-09-30T22:22:27.160001+0000","last_clean":"2023-09-30T22:22:27.160001+0000","last_became_active":"2023-09-30T22:21:37.224728+0000","last_became_peered":"2023-09-30T22:21:37.224728+0000","last_unstale":"2023-09-30T22:22:27.160001+0000","last_undegraded":"2023-09-30T22:22:27.160001+0000","last_fullsized":"2023-09-30T22:22:27.160001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795977+0000","last_change":"2023-09-30T22:21:36.796108+0000","last_active":"2023-09-30T22:22:26.795977+0000","last_peered":"2023-09-30T22:22:26.795977+0000","last_clean":"2023-09-30T22:22:26.795977+0000","last_became_active":"2023-09-30T22:21:36.795665+0000","last_became_peered":"2023-09-30T22:21:36.795665+0000","last_unstale":"2023-09-30T22:22:26.795977+0000","last_undegraded":"2023-09-30T22:22:26.795977+0000","last_fullsized":"2023-09-30T22:22:26.795977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079668+0000","last_change":"2023-09-30T22:21:37.207178+0000","last_active":"2023-09-30T22:23:54.079668+0000","last_peered":"2023-09-30T22:23:54.079668+0000","last_clean":"2023-09-30T22:23:54.079668+0000","last_became_active":"2023-09-30T22:21:37.206889+0000","last_became_peered":"2023-09-30T22:21:37.206889+0000","last_unstale":"2023-09-30T22:23:54.079668+0000","last_undegraded":"2023-09-30T22:23:54.079668+0000","last_fullsized":"2023-09-30T22:23:54.079668+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793393+0000","last_change":"2023-09-30T22:21:37.206551+0000","last_active":"2023-09-30T22:22:26.793393+0000","last_peered":"2023-09-30T22:22:26.793393+0000","last_clean":"2023-09-30T22:22:26.793393+0000","last_became_active":"2023-09-30T22:21:37.206176+0000","last_became_peered":"2023-09-30T22:21:37.206176+0000","last_unstale":"2023-09-30T22:22:26.793393+0000","last_undegraded":"2023-09-30T22:22:26.793393+0000","last_fullsized":"2023-09-30T22:22:26.793393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159699+0000","last_change":"2023-09-30T22:21:37.220615+0000","last_active":"2023-09-30T22:22:27.159699+0000","last_peered":"2023-09-30T22:22:27.159699+0000","last_clean":"2023-09-30T22:22:27.159699+0000","last_became_active":"2023-09-30T22:21:37.217961+0000","last_became_peered":"2023-09-30T22:21:37.217961+0000","last_unstale":"2023-09-30T22:22:27.159699+0000","last_undegraded":"2023-09-30T22:22:27.159699+0000","last_fullsized":"2023-09-30T22:22:27.159699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794866+0000","last_change":"2023-09-30T22:21:37.229799+0000","last_active":"2023-09-30T22:22:26.794866+0000","last_peered":"2023-09-30T22:22:26.794866+0000","last_clean":"2023-09-30T22:22:26.794866+0000","last_became_active":"2023-09-30T22:21:37.229494+0000","last_became_peered":"2023-09-30T22:21:37.229494+0000","last_unstale":"2023-09-30T22:22:26.794866+0000","last_undegraded":"2023-09-30T22:22:26.794866+0000","last_fullsized":"2023-09-30T22:22:26.794866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159832+0000","last_change":"2023-09-30T22:21:37.215694+0000","last_active":"2023-09-30T22:22:27.159832+0000","last_peered":"2023-09-30T22:22:27.159832+0000","last_clean":"2023-09-30T22:22:27.159832+0000","last_became_active":"2023-09-30T22:21:37.215371+0000","last_became_peered":"2023-09-30T22:21:37.215371+0000","last_unstale":"2023-09-30T22:22:27.159832+0000","last_undegraded":"2023-09-30T22:22:27.159832+0000","last_fullsized":"2023-09-30T22:22:27.159832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793525+0000","last_change":"2023-09-30T22:21:37.206797+0000","last_active":"2023-09-30T22:22:26.793525+0000","last_peered":"2023-09-30T22:22:26.793525+0000","last_clean":"2023-09-30T22:22:26.793525+0000","last_became_active":"2023-09-30T22:21:37.206384+0000","last_became_peered":"2023-09-30T22:21:37.206384+0000","last_unstale":"2023-09-30T22:22:26.793525+0000","last_undegraded":"2023-09-30T22:22:26.793525+0000","last_fullsized":"2023-09-30T22:22:26.793525+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":67217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.316,"15min":0.316},"min":{"1min":0.20300000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.39400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.432,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33600000000000002}]},{"osd":1,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.377},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.45700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.37},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.27900000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.26600000000000001}]},{"osd":2,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33400000000000002},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.43099999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.42899999999999999,"5min":0.502,"15min":0.502},"last":0.34999999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.315,"15min":0.315},"min":{"1min":0.24199999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.35599999999999998}]},{"osd":1,"last update":"Sat Sep 30 22:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.28399999999999997,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35599999999999998,"15min":0.35699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33200000000000002}]},{"osd":3,"last update":"Sat Sep 30 22:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36299999999999999,"15min":0.36499999999999999},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.52600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39600000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297284,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074556,"statfs":{"total":96636764160,"available":96332345344,"internally_reserved":0,"allocated":192512,"data_stored":64160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 22:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35599999999999998,"15min":0.35699999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.48199999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Sep 30 22:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.372},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.39200000000000002}]},{"osd":3,"last update":"Sat Sep 30 22:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36199999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40899999999999997,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.376}]}]},{"osd":1,"up_from":9,"seq":38654705695,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":64106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.503},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.39400000000000002}]},{"osd":2,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33200000000000002,"15min":0.33000000000000002},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.374}]},{"osd":3,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.495,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.374,"5min":0.38100000000000001,"15min":0.38200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:23:57.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 Processor -- start 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 -- start start 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb5809d6f0 0x7ffb5809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e000 0x7ffb581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5809b120 con 0x7ffb5809e410 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5806b460 con 0x7ffb5809d6f0 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5e6e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5809a7e0 con 0x7ffb5809e000 2023-09-30T22:23:57.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb57fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb5809d6f0 0x7ffb5809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5cc85700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb5809e410 0x7ffb58133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50654/0 (socket says 172.21.15.32:50654) 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb5cc85700 1 -- 172.21.15.32:0/1818354040 learned_addr learned my addr 172.21.15.32:0/1818354040 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.988+0000 7ffb577fe700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e000 0x7ffb581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb57fff700 1 -- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e000 msgr2=0x7ffb581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb57fff700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e000 0x7ffb581339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb57fff700 1 -- 172.21.15.32:0/1818354040 >> v1:172.21.15.32:6789/0 conn(0x7ffb5809e410 legacy=0x7ffb58133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb57fff700 1 -- 172.21.15.32:0/1818354040 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5809a500 con 0x7ffb5809d6f0 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb56ffd700 1 -- 172.21.15.32:0/1818354040 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818202867 0 0) 0x7ffb5809b120 con 0x7ffb5809e410 2023-09-30T22:23:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb577fe700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e000 0x7ffb581339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb57fff700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb5809dac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009840 tx=0x7ffb48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95626f668d780be7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb56ffd700 1 -- 172.21.15.32:0/1818354040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011830 con 0x7ffb5809d6f0 2023-09-30T22:23:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb56ffd700 1 -- 172.21.15.32:0/1818354040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48011990 con 0x7ffb5809d6f0 2023-09-30T22:23:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.989+0000 7ffb56ffd700 1 -- 172.21.15.32:0/1818354040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48011d20 con 0x7ffb5809d6f0 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 msgr2=0x7ffb5809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb5809dac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009840 tx=0x7ffb48009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1818354040 shutdown_connections 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb5809dac0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1818354040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e000 0x7ffb581339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1818354040 >> 172.21.15.32:0/1818354040 conn(0x7ffb58066f50 msgr2=0x7ffb5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1818354040 shutdown_connections 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.990+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1818354040 wait complete. 2023-09-30T22:23:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 Processor -- start 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 -- start start 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb581c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb5809e000 0x7ffb581c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 0x7ffb581c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb581a7680 con 0x7ffb5809d6f0 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5809c710 con 0x7ffb5809e000 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5e6e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb580a2ec0 con 0x7ffb5809e410 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb5cc85700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 0x7ffb581c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.991+0000 7ffb577fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb5809e000 0x7ffb581c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 0x7ffb581c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41974/0 (socket says 172.21.15.32:41974) 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb577fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb5809e000 0x7ffb581c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52470/0 (socket says 172.21.15.32:52470) 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 -- 172.21.15.32:0/1605522547 learned_addr learned my addr 172.21.15.32:0/1605522547 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb57fff700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb581c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 -- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb5809e000 msgr2=0x7ffb581c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb5809e000 0x7ffb581c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 -- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 msgr2=0x7ffb581c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb581c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 -- 172.21.15.32:0/1605522547 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0094a0 con 0x7ffb5809e410 2023-09-30T22:23:57.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb57fff700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb581c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5cc85700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 0x7ffb581c8c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb50002230 tx=0x7ffb5000cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000aac0 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5000ac40 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480094a0 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500070d0 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.992+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb581a6720 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.993+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5804ea50 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.994+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7ffb50007250 con 0x7ffb5809e410 2023-09-30T22:23:57.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.994+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7ffb5003fbe0 con 0x7ffb5809e410 2023-09-30T22:23:57.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:57.998+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb50007500 con 0x7ffb5809e410 2023-09-30T22:23:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.199+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 --> v1:172.21.15.32:6816/126553 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb5809ace0 con 0x7ffb4402f480 2023-09-30T22:23:58.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.206+0000 7ffb54ff9700 1 -- 172.21.15.32:0/1605522547 <== mgr.14100 v1:172.21.15.32:6816/126553 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270497 (unknown 2965378022 0 4137741278) 0x7ffb48011630 con 0x7ffb4402f480 2023-09-30T22:23:58.210 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 >> v1:172.21.15.32:6816/126553 conn(0x7ffb4402f480 legacy=0x7ffb44031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 msgr2=0x7ffb581c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 0x7ffb581c8c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffb50002230 tx=0x7ffb5000cf50 comp rx=0 tx=0).stop 2023-09-30T22:23:58.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 shutdown_connections 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb5809e410 0x7ffb581c8c40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb5809d6f0 0x7ffb581c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 --2- 172.21.15.32:0/1605522547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb5809e000 0x7ffb581c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 >> 172.21.15.32:0/1605522547 conn(0x7ffb58066f50 msgr2=0x7ffb581bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 shutdown_connections 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.210+0000 7ffb5e6e8700 1 -- 172.21.15.32:0/1605522547 wait complete. 2023-09-30T22:23:58.213 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-09-30T22:23:58.224 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2023-09-30T22:23:58.030006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189136,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298224,"statfs":{"total":386547056640,"available":385329381376,"internally_reserved":0,"allocated":770048,"data_stored":262506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.994337"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081376+0000","last_change":"2023-09-30T22:21:37.213744+0000","last_active":"2023-09-30T22:23:54.081376+0000","last_peered":"2023-09-30T22:23:54.081376+0000","last_clean":"2023-09-30T22:23:54.081376+0000","last_became_active":"2023-09-30T22:21:37.213432+0000","last_became_peered":"2023-09-30T22:21:37.213432+0000","last_unstale":"2023-09-30T22:23:54.081376+0000","last_undegraded":"2023-09-30T22:23:54.081376+0000","last_fullsized":"2023-09-30T22:23:54.081376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081947+0000","last_change":"2023-09-30T22:21:37.209966+0000","last_active":"2023-09-30T22:23:54.081947+0000","last_peered":"2023-09-30T22:23:54.081947+0000","last_clean":"2023-09-30T22:23:54.081947+0000","last_became_active":"2023-09-30T22:21:37.209700+0000","last_became_peered":"2023-09-30T22:21:37.209700+0000","last_unstale":"2023-09-30T22:23:54.081947+0000","last_undegraded":"2023-09-30T22:23:54.081947+0000","last_fullsized":"2023-09-30T22:23:54.081947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795123+0000","last_change":"2023-09-30T22:21:37.227972+0000","last_active":"2023-09-30T22:22:26.795123+0000","last_peered":"2023-09-30T22:22:26.795123+0000","last_clean":"2023-09-30T22:22:26.795123+0000","last_became_active":"2023-09-30T22:21:37.227698+0000","last_became_peered":"2023-09-30T22:21:37.227698+0000","last_unstale":"2023-09-30T22:22:26.795123+0000","last_undegraded":"2023-09-30T22:22:26.795123+0000","last_fullsized":"2023-09-30T22:22:26.795123+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794740+0000","last_change":"2023-09-30T22:21:37.217682+0000","last_active":"2023-09-30T22:22:26.794740+0000","last_peered":"2023-09-30T22:22:26.794740+0000","last_clean":"2023-09-30T22:22:26.794740+0000","last_became_active":"2023-09-30T22:21:37.217395+0000","last_became_peered":"2023-09-30T22:21:37.217395+0000","last_unstale":"2023-09-30T22:22:26.794740+0000","last_undegraded":"2023-09-30T22:22:26.794740+0000","last_fullsized":"2023-09-30T22:22:26.794740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.805052+0000","last_change":"2023-09-30T22:22:26.804304+0000","last_active":"2023-09-30T22:22:26.805052+0000","last_peered":"2023-09-30T22:22:26.805052+0000","last_clean":"2023-09-30T22:22:26.805052+0000","last_became_active":"2023-09-30T22:22:26.803956+0000","last_became_peered":"2023-09-30T22:22:26.803956+0000","last_unstale":"2023-09-30T22:22:26.805052+0000","last_undegraded":"2023-09-30T22:22:26.805052+0000","last_fullsized":"2023-09-30T22:22:26.805052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159825+0000","last_change":"2023-09-30T22:21:37.224923+0000","last_active":"2023-09-30T22:22:27.159825+0000","last_peered":"2023-09-30T22:22:27.159825+0000","last_clean":"2023-09-30T22:22:27.159825+0000","last_became_active":"2023-09-30T22:21:37.224651+0000","last_became_peered":"2023-09-30T22:21:37.224651+0000","last_unstale":"2023-09-30T22:22:27.159825+0000","last_undegraded":"2023-09-30T22:22:27.159825+0000","last_fullsized":"2023-09-30T22:22:27.159825+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159533+0000","last_change":"2023-09-30T22:21:37.220818+0000","last_active":"2023-09-30T22:22:27.159533+0000","last_peered":"2023-09-30T22:22:27.159533+0000","last_clean":"2023-09-30T22:22:27.159533+0000","last_became_active":"2023-09-30T22:21:37.217825+0000","last_became_peered":"2023-09-30T22:21:37.217825+0000","last_unstale":"2023-09-30T22:22:27.159533+0000","last_undegraded":"2023-09-30T22:22:27.159533+0000","last_fullsized":"2023-09-30T22:22:27.159533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793118+0000","last_change":"2023-09-30T22:21:37.237512+0000","last_active":"2023-09-30T22:22:26.793118+0000","last_peered":"2023-09-30T22:22:26.793118+0000","last_clean":"2023-09-30T22:22:26.793118+0000","last_became_active":"2023-09-30T22:21:37.237217+0000","last_became_peered":"2023-09-30T22:21:37.237217+0000","last_unstale":"2023-09-30T22:22:26.793118+0000","last_undegraded":"2023-09-30T22:22:26.793118+0000","last_fullsized":"2023-09-30T22:22:26.793118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796106+0000","last_change":"2023-09-30T22:21:37.231447+0000","last_active":"2023-09-30T22:22:26.796106+0000","last_peered":"2023-09-30T22:22:26.796106+0000","last_clean":"2023-09-30T22:22:26.796106+0000","last_became_active":"2023-09-30T22:21:37.231106+0000","last_became_peered":"2023-09-30T22:21:37.231106+0000","last_unstale":"2023-09-30T22:22:26.796106+0000","last_undegraded":"2023-09-30T22:22:26.796106+0000","last_fullsized":"2023-09-30T22:22:26.796106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794341+0000","last_change":"2023-09-30T22:21:37.238710+0000","last_active":"2023-09-30T22:22:26.794341+0000","last_peered":"2023-09-30T22:22:26.794341+0000","last_clean":"2023-09-30T22:22:26.794341+0000","last_became_active":"2023-09-30T22:21:37.237230+0000","last_became_peered":"2023-09-30T22:21:37.237230+0000","last_unstale":"2023-09-30T22:22:26.794341+0000","last_undegraded":"2023-09-30T22:22:26.794341+0000","last_fullsized":"2023-09-30T22:22:26.794341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793934+0000","last_change":"2023-09-30T22:21:37.238711+0000","last_active":"2023-09-30T22:22:26.793934+0000","last_peered":"2023-09-30T22:22:26.793934+0000","last_clean":"2023-09-30T22:22:26.793934+0000","last_became_active":"2023-09-30T22:21:37.237399+0000","last_became_peered":"2023-09-30T22:21:37.237399+0000","last_unstale":"2023-09-30T22:22:26.793934+0000","last_undegraded":"2023-09-30T22:22:26.793934+0000","last_fullsized":"2023-09-30T22:22:26.793934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081964+0000","last_change":"2023-09-30T22:21:37.215868+0000","last_active":"2023-09-30T22:23:54.081964+0000","last_peered":"2023-09-30T22:23:54.081964+0000","last_clean":"2023-09-30T22:23:54.081964+0000","last_became_active":"2023-09-30T22:21:37.215601+0000","last_became_peered":"2023-09-30T22:21:37.215601+0000","last_unstale":"2023-09-30T22:23:54.081964+0000","last_undegraded":"2023-09-30T22:23:54.081964+0000","last_fullsized":"2023-09-30T22:23:54.081964+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081177+0000","last_change":"2023-09-30T22:21:37.229175+0000","last_active":"2023-09-30T22:23:54.081177+0000","last_peered":"2023-09-30T22:23:54.081177+0000","last_clean":"2023-09-30T22:23:54.081177+0000","last_became_active":"2023-09-30T22:21:37.226712+0000","last_became_peered":"2023-09-30T22:21:37.226712+0000","last_unstale":"2023-09-30T22:23:54.081177+0000","last_undegraded":"2023-09-30T22:23:54.081177+0000","last_fullsized":"2023-09-30T22:23:54.081177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794716+0000","last_change":"2023-09-30T22:21:37.222596+0000","last_active":"2023-09-30T22:22:26.794716+0000","last_peered":"2023-09-30T22:22:26.794716+0000","last_clean":"2023-09-30T22:22:26.794716+0000","last_became_active":"2023-09-30T22:21:37.222295+0000","last_became_peered":"2023-09-30T22:21:37.222295+0000","last_unstale":"2023-09-30T22:22:26.794716+0000","last_undegraded":"2023-09-30T22:22:26.794716+0000","last_fullsized":"2023-09-30T22:22:26.794716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.805515+0000","last_change":"2023-09-30T22:22:26.804397+0000","last_active":"2023-09-30T22:22:26.805515+0000","last_peered":"2023-09-30T22:22:26.805515+0000","last_clean":"2023-09-30T22:22:26.805515+0000","last_became_active":"2023-09-30T22:22:26.804064+0000","last_became_peered":"2023-09-30T22:22:26.804064+0000","last_unstale":"2023-09-30T22:22:26.805515+0000","last_undegraded":"2023-09-30T22:22:26.805515+0000","last_fullsized":"2023-09-30T22:22:26.805515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159159+0000","last_change":"2023-09-30T22:21:37.220367+0000","last_active":"2023-09-30T22:22:27.159159+0000","last_peered":"2023-09-30T22:22:27.159159+0000","last_clean":"2023-09-30T22:22:27.159159+0000","last_became_active":"2023-09-30T22:21:37.217445+0000","last_became_peered":"2023-09-30T22:21:37.217445+0000","last_unstale":"2023-09-30T22:22:27.159159+0000","last_undegraded":"2023-09-30T22:22:27.159159+0000","last_fullsized":"2023-09-30T22:22:27.159159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796112+0000","last_change":"2023-09-30T22:21:37.231390+0000","last_active":"2023-09-30T22:22:26.796112+0000","last_peered":"2023-09-30T22:22:26.796112+0000","last_clean":"2023-09-30T22:22:26.796112+0000","last_became_active":"2023-09-30T22:21:37.231116+0000","last_became_peered":"2023-09-30T22:21:37.231116+0000","last_unstale":"2023-09-30T22:22:26.796112+0000","last_undegraded":"2023-09-30T22:22:26.796112+0000","last_fullsized":"2023-09-30T22:22:26.796112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795532+0000","last_change":"2023-09-30T22:21:37.228923+0000","last_active":"2023-09-30T22:22:26.795532+0000","last_peered":"2023-09-30T22:22:26.795532+0000","last_clean":"2023-09-30T22:22:26.795532+0000","last_became_active":"2023-09-30T22:21:37.220140+0000","last_became_peered":"2023-09-30T22:21:37.220140+0000","last_unstale":"2023-09-30T22:22:26.795532+0000","last_undegraded":"2023-09-30T22:22:26.795532+0000","last_fullsized":"2023-09-30T22:22:26.795532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080992+0000","last_change":"2023-09-30T22:22:26.807130+0000","last_active":"2023-09-30T22:23:54.080992+0000","last_peered":"2023-09-30T22:23:54.080992+0000","last_clean":"2023-09-30T22:23:54.080992+0000","last_became_active":"2023-09-30T22:22:26.806744+0000","last_became_peered":"2023-09-30T22:22:26.806744+0000","last_unstale":"2023-09-30T22:23:54.080992+0000","last_undegraded":"2023-09-30T22:23:54.080992+0000","last_fullsized":"2023-09-30T22:23:54.080992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793712+0000","last_change":"2023-09-30T22:21:37.218484+0000","last_active":"2023-09-30T22:22:26.793712+0000","last_peered":"2023-09-30T22:22:26.793712+0000","last_clean":"2023-09-30T22:22:26.793712+0000","last_became_active":"2023-09-30T22:21:37.218141+0000","last_became_peered":"2023-09-30T22:21:37.218141+0000","last_unstale":"2023-09-30T22:22:26.793712+0000","last_undegraded":"2023-09-30T22:22:26.793712+0000","last_fullsized":"2023-09-30T22:22:26.793712+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.792740+0000","last_change":"2023-09-30T22:21:37.216211+0000","last_active":"2023-09-30T22:22:26.792740+0000","last_peered":"2023-09-30T22:22:26.792740+0000","last_clean":"2023-09-30T22:22:26.792740+0000","last_became_active":"2023-09-30T22:21:37.215922+0000","last_became_peered":"2023-09-30T22:21:37.215922+0000","last_unstale":"2023-09-30T22:22:26.792740+0000","last_undegraded":"2023-09-30T22:22:26.792740+0000","last_fullsized":"2023-09-30T22:22:26.792740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794805+0000","last_change":"2023-09-30T22:21:37.227137+0000","last_active":"2023-09-30T22:22:26.794805+0000","last_peered":"2023-09-30T22:22:26.794805+0000","last_clean":"2023-09-30T22:22:26.794805+0000","last_became_active":"2023-09-30T22:21:37.226848+0000","last_became_peered":"2023-09-30T22:21:37.226848+0000","last_unstale":"2023-09-30T22:22:26.794805+0000","last_undegraded":"2023-09-30T22:22:26.794805+0000","last_fullsized":"2023-09-30T22:22:26.794805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159441+0000","last_change":"2023-09-30T22:21:37.220415+0000","last_active":"2023-09-30T22:22:27.159441+0000","last_peered":"2023-09-30T22:22:27.159441+0000","last_clean":"2023-09-30T22:22:27.159441+0000","last_became_active":"2023-09-30T22:21:37.217468+0000","last_became_peered":"2023-09-30T22:21:37.217468+0000","last_unstale":"2023-09-30T22:22:27.159441+0000","last_undegraded":"2023-09-30T22:22:27.159441+0000","last_fullsized":"2023-09-30T22:22:27.159441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793174+0000","last_change":"2023-09-30T22:21:37.220204+0000","last_active":"2023-09-30T22:22:26.793174+0000","last_peered":"2023-09-30T22:22:26.793174+0000","last_clean":"2023-09-30T22:22:26.793174+0000","last_became_active":"2023-09-30T22:21:37.219099+0000","last_became_peered":"2023-09-30T22:21:37.219099+0000","last_unstale":"2023-09-30T22:22:26.793174+0000","last_undegraded":"2023-09-30T22:22:26.793174+0000","last_fullsized":"2023-09-30T22:22:26.793174+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794335+0000","last_change":"2023-09-30T22:21:37.217211+0000","last_active":"2023-09-30T22:22:26.794335+0000","last_peered":"2023-09-30T22:22:26.794335+0000","last_clean":"2023-09-30T22:22:26.794335+0000","last_became_active":"2023-09-30T22:21:37.216537+0000","last_became_peered":"2023-09-30T22:21:37.216537+0000","last_unstale":"2023-09-30T22:22:26.794335+0000","last_undegraded":"2023-09-30T22:22:26.794335+0000","last_fullsized":"2023-09-30T22:22:26.794335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794175+0000","last_change":"2023-09-30T22:21:37.209836+0000","last_active":"2023-09-30T22:22:26.794175+0000","last_peered":"2023-09-30T22:22:26.794175+0000","last_clean":"2023-09-30T22:22:26.794175+0000","last_became_active":"2023-09-30T22:21:37.209561+0000","last_became_peered":"2023-09-30T22:21:37.209561+0000","last_unstale":"2023-09-30T22:22:26.794175+0000","last_undegraded":"2023-09-30T22:22:26.794175+0000","last_fullsized":"2023-09-30T22:22:26.794175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160229+0000","last_change":"2023-09-30T22:21:37.220637+0000","last_active":"2023-09-30T22:22:27.160229+0000","last_peered":"2023-09-30T22:22:27.160229+0000","last_clean":"2023-09-30T22:22:27.160229+0000","last_became_active":"2023-09-30T22:21:37.217741+0000","last_became_peered":"2023-09-30T22:21:37.217741+0000","last_unstale":"2023-09-30T22:22:27.160229+0000","last_undegraded":"2023-09-30T22:22:27.160229+0000","last_fullsized":"2023-09-30T22:22:27.160229+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794000+0000","last_change":"2023-09-30T22:21:36.800514+0000","last_active":"2023-09-30T22:22:26.794000+0000","last_peered":"2023-09-30T22:22:26.794000+0000","last_clean":"2023-09-30T22:22:26.794000+0000","last_became_active":"2023-09-30T22:21:36.799775+0000","last_became_peered":"2023-09-30T22:21:36.799775+0000","last_unstale":"2023-09-30T22:22:26.794000+0000","last_undegraded":"2023-09-30T22:22:26.794000+0000","last_fullsized":"2023-09-30T22:22:26.794000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160233+0000","last_change":"2023-09-30T22:21:37.211849+0000","last_active":"2023-09-30T22:22:27.160233+0000","last_peered":"2023-09-30T22:22:27.160233+0000","last_clean":"2023-09-30T22:22:27.160233+0000","last_became_active":"2023-09-30T22:21:37.209128+0000","last_became_peered":"2023-09-30T22:21:37.209128+0000","last_unstale":"2023-09-30T22:22:27.160233+0000","last_undegraded":"2023-09-30T22:22:27.160233+0000","last_fullsized":"2023-09-30T22:22:27.160233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.792774+0000","last_change":"2023-09-30T22:21:36.803467+0000","last_active":"2023-09-30T22:22:26.792774+0000","last_peered":"2023-09-30T22:22:26.792774+0000","last_clean":"2023-09-30T22:22:26.792774+0000","last_became_active":"2023-09-30T22:21:36.803141+0000","last_became_peered":"2023-09-30T22:21:36.803141+0000","last_unstale":"2023-09-30T22:22:26.792774+0000","last_undegraded":"2023-09-30T22:22:26.792774+0000","last_fullsized":"2023-09-30T22:22:26.792774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795643+0000","last_change":"2023-09-30T22:21:37.224566+0000","last_active":"2023-09-30T22:22:26.795643+0000","last_peered":"2023-09-30T22:22:26.795643+0000","last_clean":"2023-09-30T22:22:26.795643+0000","last_became_active":"2023-09-30T22:21:37.219843+0000","last_became_peered":"2023-09-30T22:21:37.219843+0000","last_unstale":"2023-09-30T22:22:26.795643+0000","last_undegraded":"2023-09-30T22:22:26.795643+0000","last_fullsized":"2023-09-30T22:22:26.795643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159449+0000","last_change":"2023-09-30T22:21:36.813865+0000","last_active":"2023-09-30T22:22:27.159449+0000","last_peered":"2023-09-30T22:22:27.159449+0000","last_clean":"2023-09-30T22:22:27.159449+0000","last_became_active":"2023-09-30T22:21:36.812871+0000","last_became_peered":"2023-09-30T22:21:36.812871+0000","last_unstale":"2023-09-30T22:22:27.159449+0000","last_undegraded":"2023-09-30T22:22:27.159449+0000","last_fullsized":"2023-09-30T22:22:27.159449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160854+0000","last_change":"2023-09-30T22:21:37.215838+0000","last_active":"2023-09-30T22:22:27.160854+0000","last_peered":"2023-09-30T22:22:27.160854+0000","last_clean":"2023-09-30T22:22:27.160854+0000","last_became_active":"2023-09-30T22:21:37.215314+0000","last_became_peered":"2023-09-30T22:21:37.215314+0000","last_unstale":"2023-09-30T22:22:27.160854+0000","last_undegraded":"2023-09-30T22:22:27.160854+0000","last_fullsized":"2023-09-30T22:22:27.160854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795977+0000","last_change":"2023-09-30T22:21:36.809280+0000","last_active":"2023-09-30T22:22:26.795977+0000","last_peered":"2023-09-30T22:22:26.795977+0000","last_clean":"2023-09-30T22:22:26.795977+0000","last_became_active":"2023-09-30T22:21:36.808975+0000","last_became_peered":"2023-09-30T22:21:36.808975+0000","last_unstale":"2023-09-30T22:22:26.795977+0000","last_undegraded":"2023-09-30T22:22:26.795977+0000","last_fullsized":"2023-09-30T22:22:26.795977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159548+0000","last_change":"2023-09-30T22:21:37.215010+0000","last_active":"2023-09-30T22:22:27.159548+0000","last_peered":"2023-09-30T22:22:27.159548+0000","last_clean":"2023-09-30T22:22:27.159548+0000","last_became_active":"2023-09-30T22:21:37.212913+0000","last_became_peered":"2023-09-30T22:21:37.212913+0000","last_unstale":"2023-09-30T22:22:27.159548+0000","last_undegraded":"2023-09-30T22:22:27.159548+0000","last_fullsized":"2023-09-30T22:22:27.159548+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081718+0000","last_change":"2023-09-30T22:21:36.800431+0000","last_active":"2023-09-30T22:23:54.081718+0000","last_peered":"2023-09-30T22:23:54.081718+0000","last_clean":"2023-09-30T22:23:54.081718+0000","last_became_active":"2023-09-30T22:21:36.800130+0000","last_became_peered":"2023-09-30T22:21:36.800130+0000","last_unstale":"2023-09-30T22:23:54.081718+0000","last_undegraded":"2023-09-30T22:23:54.081718+0000","last_fullsized":"2023-09-30T22:23:54.081718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079809+0000","last_change":"2023-09-30T22:22:26.805968+0000","last_active":"2023-09-30T22:23:54.079809+0000","last_peered":"2023-09-30T22:23:54.079809+0000","last_clean":"2023-09-30T22:23:54.079809+0000","last_became_active":"2023-09-30T22:22:26.805589+0000","last_became_peered":"2023-09-30T22:22:26.805589+0000","last_unstale":"2023-09-30T22:23:54.079809+0000","last_undegraded":"2023-09-30T22:23:54.079809+0000","last_fullsized":"2023-09-30T22:23:54.079809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"16'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795126+0000","last_change":"2023-09-30T22:21:36.810931+0000","last_active":"2023-09-30T22:22:26.795126+0000","last_peered":"2023-09-30T22:22:26.795126+0000","last_clean":"2023-09-30T22:22:26.795126+0000","last_became_active":"2023-09-30T22:21:36.810625+0000","last_became_peered":"2023-09-30T22:21:36.810625+0000","last_unstale":"2023-09-30T22:22:26.795126+0000","last_undegraded":"2023-09-30T22:22:26.795126+0000","last_fullsized":"2023-09-30T22:22:26.795126+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794629+0000","last_change":"2023-09-30T22:21:37.217701+0000","last_active":"2023-09-30T22:22:26.794629+0000","last_peered":"2023-09-30T22:22:26.794629+0000","last_clean":"2023-09-30T22:22:26.794629+0000","last_became_active":"2023-09-30T22:21:37.217076+0000","last_became_peered":"2023-09-30T22:21:37.217076+0000","last_unstale":"2023-09-30T22:22:26.794629+0000","last_undegraded":"2023-09-30T22:22:26.794629+0000","last_fullsized":"2023-09-30T22:22:26.794629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'12","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.114456+0000","last_change":"2023-09-30T22:21:36.811297+0000","last_active":"2023-09-30T22:23:54.114456+0000","last_peered":"2023-09-30T22:23:54.114456+0000","last_clean":"2023-09-30T22:23:54.114456+0000","last_became_active":"2023-09-30T22:21:36.810987+0000","last_became_peered":"2023-09-30T22:21:36.810987+0000","last_unstale":"2023-09-30T22:23:54.114456+0000","last_undegraded":"2023-09-30T22:23:54.114456+0000","last_fullsized":"2023-09-30T22:23:54.114456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081038+0000","last_change":"2023-09-30T22:21:37.228789+0000","last_active":"2023-09-30T22:23:54.081038+0000","last_peered":"2023-09-30T22:23:54.081038+0000","last_clean":"2023-09-30T22:23:54.081038+0000","last_became_active":"2023-09-30T22:21:37.225220+0000","last_became_peered":"2023-09-30T22:21:37.225220+0000","last_unstale":"2023-09-30T22:23:54.081038+0000","last_undegraded":"2023-09-30T22:23:54.081038+0000","last_fullsized":"2023-09-30T22:23:54.081038+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159758+0000","last_change":"2023-09-30T22:21:36.813567+0000","last_active":"2023-09-30T22:22:27.159758+0000","last_peered":"2023-09-30T22:22:27.159758+0000","last_clean":"2023-09-30T22:22:27.159758+0000","last_became_active":"2023-09-30T22:21:36.813257+0000","last_became_peered":"2023-09-30T22:21:36.813257+0000","last_unstale":"2023-09-30T22:22:27.159758+0000","last_undegraded":"2023-09-30T22:22:27.159758+0000","last_fullsized":"2023-09-30T22:22:27.159758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081909+0000","last_change":"2023-09-30T22:22:26.801274+0000","last_active":"2023-09-30T22:23:54.081909+0000","last_peered":"2023-09-30T22:23:54.081909+0000","last_clean":"2023-09-30T22:23:54.081909+0000","last_became_active":"2023-09-30T22:22:26.800913+0000","last_became_peered":"2023-09-30T22:22:26.800913+0000","last_unstale":"2023-09-30T22:23:54.081909+0000","last_undegraded":"2023-09-30T22:23:54.081909+0000","last_fullsized":"2023-09-30T22:23:54.081909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794763+0000","last_change":"2023-09-30T22:21:36.806015+0000","last_active":"2023-09-30T22:22:26.794763+0000","last_peered":"2023-09-30T22:22:26.794763+0000","last_clean":"2023-09-30T22:22:26.794763+0000","last_became_active":"2023-09-30T22:21:36.805679+0000","last_became_peered":"2023-09-30T22:21:36.805679+0000","last_unstale":"2023-09-30T22:22:26.794763+0000","last_undegraded":"2023-09-30T22:22:26.794763+0000","last_fullsized":"2023-09-30T22:22:26.794763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160321+0000","last_change":"2023-09-30T22:21:37.221673+0000","last_active":"2023-09-30T22:22:27.160321+0000","last_peered":"2023-09-30T22:22:27.160321+0000","last_clean":"2023-09-30T22:22:27.160321+0000","last_became_active":"2023-09-30T22:21:37.221388+0000","last_became_peered":"2023-09-30T22:21:37.221388+0000","last_unstale":"2023-09-30T22:22:27.160321+0000","last_undegraded":"2023-09-30T22:22:27.160321+0000","last_fullsized":"2023-09-30T22:22:27.160321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159822+0000","last_change":"2023-09-30T22:21:36.812303+0000","last_active":"2023-09-30T22:22:27.159822+0000","last_peered":"2023-09-30T22:22:27.159822+0000","last_clean":"2023-09-30T22:22:27.159822+0000","last_became_active":"2023-09-30T22:21:36.811887+0000","last_became_peered":"2023-09-30T22:21:36.811887+0000","last_unstale":"2023-09-30T22:22:27.159822+0000","last_undegraded":"2023-09-30T22:22:27.159822+0000","last_fullsized":"2023-09-30T22:22:27.159822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795724+0000","last_change":"2023-09-30T22:21:37.225272+0000","last_active":"2023-09-30T22:22:26.795724+0000","last_peered":"2023-09-30T22:22:26.795724+0000","last_clean":"2023-09-30T22:22:26.795724+0000","last_became_active":"2023-09-30T22:21:37.224827+0000","last_became_peered":"2023-09-30T22:21:37.224827+0000","last_unstale":"2023-09-30T22:22:26.795724+0000","last_undegraded":"2023-09-30T22:22:26.795724+0000","last_fullsized":"2023-09-30T22:22:26.795724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"16'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793478+0000","last_change":"2023-09-30T22:21:36.804342+0000","last_active":"2023-09-30T22:22:26.793478+0000","last_peered":"2023-09-30T22:22:26.793478+0000","last_clean":"2023-09-30T22:22:26.793478+0000","last_became_active":"2023-09-30T22:21:36.804002+0000","last_became_peered":"2023-09-30T22:21:36.804002+0000","last_unstale":"2023-09-30T22:22:26.793478+0000","last_undegraded":"2023-09-30T22:22:26.793478+0000","last_fullsized":"2023-09-30T22:22:26.793478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160697+0000","last_change":"2023-09-30T22:21:37.215043+0000","last_active":"2023-09-30T22:22:27.160697+0000","last_peered":"2023-09-30T22:22:27.160697+0000","last_clean":"2023-09-30T22:22:27.160697+0000","last_became_active":"2023-09-30T22:21:37.214750+0000","last_became_peered":"2023-09-30T22:21:37.214750+0000","last_unstale":"2023-09-30T22:22:27.160697+0000","last_undegraded":"2023-09-30T22:22:27.160697+0000","last_fullsized":"2023-09-30T22:22:27.160697+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795845+0000","last_change":"2023-09-30T22:21:36.796103+0000","last_active":"2023-09-30T22:22:26.795845+0000","last_peered":"2023-09-30T22:22:26.795845+0000","last_clean":"2023-09-30T22:22:26.795845+0000","last_became_active":"2023-09-30T22:21:36.795669+0000","last_became_peered":"2023-09-30T22:21:36.795669+0000","last_unstale":"2023-09-30T22:22:26.795845+0000","last_undegraded":"2023-09-30T22:22:26.795845+0000","last_fullsized":"2023-09-30T22:22:26.795845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159638+0000","last_change":"2023-09-30T22:21:37.223782+0000","last_active":"2023-09-30T22:22:27.159638+0000","last_peered":"2023-09-30T22:22:27.159638+0000","last_clean":"2023-09-30T22:22:27.159638+0000","last_became_active":"2023-09-30T22:21:37.223508+0000","last_became_peered":"2023-09-30T22:21:37.223508+0000","last_unstale":"2023-09-30T22:22:27.159638+0000","last_undegraded":"2023-09-30T22:22:27.159638+0000","last_fullsized":"2023-09-30T22:22:27.159638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"16'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081790+0000","last_change":"2023-09-30T22:21:36.798241+0000","last_active":"2023-09-30T22:23:54.081790+0000","last_peered":"2023-09-30T22:23:54.081790+0000","last_clean":"2023-09-30T22:23:54.081790+0000","last_became_active":"2023-09-30T22:21:36.797923+0000","last_became_peered":"2023-09-30T22:21:36.797923+0000","last_unstale":"2023-09-30T22:23:54.081790+0000","last_undegraded":"2023-09-30T22:23:54.081790+0000","last_fullsized":"2023-09-30T22:23:54.081790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.792827+0000","last_change":"2023-09-30T22:21:37.223841+0000","last_active":"2023-09-30T22:22:26.792827+0000","last_peered":"2023-09-30T22:22:26.792827+0000","last_clean":"2023-09-30T22:22:26.792827+0000","last_became_active":"2023-09-30T22:21:37.223485+0000","last_became_peered":"2023-09-30T22:21:37.223485+0000","last_unstale":"2023-09-30T22:22:26.792827+0000","last_undegraded":"2023-09-30T22:22:26.792827+0000","last_fullsized":"2023-09-30T22:22:26.792827+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160433+0000","last_change":"2023-09-30T22:21:36.817374+0000","last_active":"2023-09-30T22:22:27.160433+0000","last_peered":"2023-09-30T22:22:27.160433+0000","last_clean":"2023-09-30T22:22:27.160433+0000","last_became_active":"2023-09-30T22:21:36.816913+0000","last_became_peered":"2023-09-30T22:21:36.816913+0000","last_unstale":"2023-09-30T22:22:27.160433+0000","last_undegraded":"2023-09-30T22:22:27.160433+0000","last_fullsized":"2023-09-30T22:22:27.160433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080845+0000","last_change":"2023-09-30T22:21:37.214893+0000","last_active":"2023-09-30T22:23:54.080845+0000","last_peered":"2023-09-30T22:23:54.080845+0000","last_clean":"2023-09-30T22:23:54.080845+0000","last_became_active":"2023-09-30T22:21:37.214616+0000","last_became_peered":"2023-09-30T22:21:37.214616+0000","last_unstale":"2023-09-30T22:23:54.080845+0000","last_undegraded":"2023-09-30T22:23:54.080845+0000","last_fullsized":"2023-09-30T22:23:54.080845+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795120+0000","last_change":"2023-09-30T22:21:36.806594+0000","last_active":"2023-09-30T22:22:26.795120+0000","last_peered":"2023-09-30T22:22:26.795120+0000","last_clean":"2023-09-30T22:22:26.795120+0000","last_became_active":"2023-09-30T22:21:36.806305+0000","last_became_peered":"2023-09-30T22:21:36.806305+0000","last_unstale":"2023-09-30T22:22:26.795120+0000","last_undegraded":"2023-09-30T22:22:26.795120+0000","last_fullsized":"2023-09-30T22:22:26.795120+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796168+0000","last_change":"2023-09-30T22:21:37.224756+0000","last_active":"2023-09-30T22:22:26.796168+0000","last_peered":"2023-09-30T22:22:26.796168+0000","last_clean":"2023-09-30T22:22:26.796168+0000","last_became_active":"2023-09-30T22:21:37.224444+0000","last_became_peered":"2023-09-30T22:21:37.224444+0000","last_unstale":"2023-09-30T22:22:26.796168+0000","last_undegraded":"2023-09-30T22:22:26.796168+0000","last_fullsized":"2023-09-30T22:22:26.796168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159654+0000","last_change":"2023-09-30T22:21:36.813533+0000","last_active":"2023-09-30T22:22:27.159654+0000","last_peered":"2023-09-30T22:22:27.159654+0000","last_clean":"2023-09-30T22:22:27.159654+0000","last_became_active":"2023-09-30T22:21:36.813227+0000","last_became_peered":"2023-09-30T22:21:36.813227+0000","last_unstale":"2023-09-30T22:22:27.159654+0000","last_undegraded":"2023-09-30T22:22:27.159654+0000","last_fullsized":"2023-09-30T22:22:27.159654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795194+0000","last_change":"2023-09-30T22:21:37.230482+0000","last_active":"2023-09-30T22:22:26.795194+0000","last_peered":"2023-09-30T22:22:26.795194+0000","last_clean":"2023-09-30T22:22:26.795194+0000","last_became_active":"2023-09-30T22:21:37.229876+0000","last_became_peered":"2023-09-30T22:21:37.229876+0000","last_unstale":"2023-09-30T22:22:26.795194+0000","last_undegraded":"2023-09-30T22:22:26.795194+0000","last_fullsized":"2023-09-30T22:22:26.795194+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159586+0000","last_change":"2023-09-30T22:21:36.811138+0000","last_active":"2023-09-30T22:22:27.159586+0000","last_peered":"2023-09-30T22:22:27.159586+0000","last_clean":"2023-09-30T22:22:27.159586+0000","last_became_active":"2023-09-30T22:21:36.810834+0000","last_became_peered":"2023-09-30T22:21:36.810834+0000","last_unstale":"2023-09-30T22:22:27.159586+0000","last_undegraded":"2023-09-30T22:22:27.159586+0000","last_fullsized":"2023-09-30T22:22:27.159586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080896+0000","last_change":"2023-09-30T22:22:26.800907+0000","last_active":"2023-09-30T22:23:54.080896+0000","last_peered":"2023-09-30T22:23:54.080896+0000","last_clean":"2023-09-30T22:23:54.080896+0000","last_became_active":"2023-09-30T22:22:26.800169+0000","last_became_peered":"2023-09-30T22:22:26.800169+0000","last_unstale":"2023-09-30T22:23:54.080896+0000","last_undegraded":"2023-09-30T22:23:54.080896+0000","last_fullsized":"2023-09-30T22:23:54.080896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159614+0000","last_change":"2023-09-30T22:21:36.812156+0000","last_active":"2023-09-30T22:22:27.159614+0000","last_peered":"2023-09-30T22:22:27.159614+0000","last_clean":"2023-09-30T22:22:27.159614+0000","last_became_active":"2023-09-30T22:21:36.811527+0000","last_became_peered":"2023-09-30T22:21:36.811527+0000","last_unstale":"2023-09-30T22:22:27.159614+0000","last_undegraded":"2023-09-30T22:22:27.159614+0000","last_fullsized":"2023-09-30T22:22:27.159614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081868+0000","last_change":"2023-09-30T22:21:37.226310+0000","last_active":"2023-09-30T22:23:54.081868+0000","last_peered":"2023-09-30T22:23:54.081868+0000","last_clean":"2023-09-30T22:23:54.081868+0000","last_became_active":"2023-09-30T22:21:37.225749+0000","last_became_peered":"2023-09-30T22:21:37.225749+0000","last_unstale":"2023-09-30T22:23:54.081868+0000","last_undegraded":"2023-09-30T22:23:54.081868+0000","last_fullsized":"2023-09-30T22:23:54.081868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159157+0000","last_change":"2023-09-30T22:21:36.811756+0000","last_active":"2023-09-30T22:22:27.159157+0000","last_peered":"2023-09-30T22:22:27.159157+0000","last_clean":"2023-09-30T22:22:27.159157+0000","last_became_active":"2023-09-30T22:21:36.811393+0000","last_became_peered":"2023-09-30T22:21:36.811393+0000","last_unstale":"2023-09-30T22:22:27.159157+0000","last_undegraded":"2023-09-30T22:22:27.159157+0000","last_fullsized":"2023-09-30T22:22:27.159157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794401+0000","last_change":"2023-09-30T22:21:37.217010+0000","last_active":"2023-09-30T22:22:26.794401+0000","last_peered":"2023-09-30T22:22:26.794401+0000","last_clean":"2023-09-30T22:22:26.794401+0000","last_became_active":"2023-09-30T22:21:37.216071+0000","last_became_peered":"2023-09-30T22:21:37.216071+0000","last_unstale":"2023-09-30T22:22:26.794401+0000","last_undegraded":"2023-09-30T22:22:26.794401+0000","last_fullsized":"2023-09-30T22:22:26.794401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793630+0000","last_change":"2023-09-30T22:21:31.794405+0000","last_active":"2023-09-30T22:22:26.793630+0000","last_peered":"2023-09-30T22:22:26.793630+0000","last_clean":"2023-09-30T22:22:26.793630+0000","last_became_active":"2023-09-30T22:21:31.794052+0000","last_became_peered":"2023-09-30T22:21:31.794052+0000","last_unstale":"2023-09-30T22:22:26.793630+0000","last_undegraded":"2023-09-30T22:22:26.793630+0000","last_fullsized":"2023-09-30T22:22:26.793630+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080943+0000","last_change":"2023-09-30T22:21:36.799995+0000","last_active":"2023-09-30T22:23:54.080943+0000","last_peered":"2023-09-30T22:23:54.080943+0000","last_clean":"2023-09-30T22:23:54.080943+0000","last_became_active":"2023-09-30T22:21:36.799674+0000","last_became_peered":"2023-09-30T22:21:36.799674+0000","last_unstale":"2023-09-30T22:23:54.080943+0000","last_undegraded":"2023-09-30T22:23:54.080943+0000","last_fullsized":"2023-09-30T22:23:54.080943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.805177+0000","last_change":"2023-09-30T22:22:26.804169+0000","last_active":"2023-09-30T22:22:26.805177+0000","last_peered":"2023-09-30T22:22:26.805177+0000","last_clean":"2023-09-30T22:22:26.805177+0000","last_became_active":"2023-09-30T22:22:26.803794+0000","last_became_peered":"2023-09-30T22:22:26.803794+0000","last_unstale":"2023-09-30T22:22:26.805177+0000","last_undegraded":"2023-09-30T22:22:26.805177+0000","last_fullsized":"2023-09-30T22:22:26.805177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159668+0000","last_change":"2023-09-30T22:21:31.798962+0000","last_active":"2023-09-30T22:22:27.159668+0000","last_peered":"2023-09-30T22:22:27.159668+0000","last_clean":"2023-09-30T22:22:27.159668+0000","last_became_active":"2023-09-30T22:21:31.798641+0000","last_became_peered":"2023-09-30T22:21:31.798641+0000","last_unstale":"2023-09-30T22:22:27.159668+0000","last_undegraded":"2023-09-30T22:22:27.159668+0000","last_fullsized":"2023-09-30T22:22:27.159668+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795852+0000","last_change":"2023-09-30T22:21:36.793285+0000","last_active":"2023-09-30T22:22:26.795852+0000","last_peered":"2023-09-30T22:22:26.795852+0000","last_clean":"2023-09-30T22:22:26.795852+0000","last_became_active":"2023-09-30T22:21:36.793005+0000","last_became_peered":"2023-09-30T22:21:36.793005+0000","last_unstale":"2023-09-30T22:22:26.795852+0000","last_undegraded":"2023-09-30T22:22:26.795852+0000","last_fullsized":"2023-09-30T22:22:26.795852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793308+0000","last_change":"2023-09-30T22:21:37.236712+0000","last_active":"2023-09-30T22:22:26.793308+0000","last_peered":"2023-09-30T22:22:26.793308+0000","last_clean":"2023-09-30T22:22:26.793308+0000","last_became_active":"2023-09-30T22:21:37.226418+0000","last_became_peered":"2023-09-30T22:21:37.226418+0000","last_unstale":"2023-09-30T22:22:26.793308+0000","last_undegraded":"2023-09-30T22:22:26.793308+0000","last_fullsized":"2023-09-30T22:22:26.793308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079884+0000","last_change":"2023-09-30T22:21:31.797737+0000","last_active":"2023-09-30T22:23:54.079884+0000","last_peered":"2023-09-30T22:23:54.079884+0000","last_clean":"2023-09-30T22:23:54.079884+0000","last_became_active":"2023-09-30T22:21:31.797396+0000","last_became_peered":"2023-09-30T22:21:31.797396+0000","last_unstale":"2023-09-30T22:23:54.079884+0000","last_undegraded":"2023-09-30T22:23:54.079884+0000","last_fullsized":"2023-09-30T22:23:54.079884+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793312+0000","last_change":"2023-09-30T22:21:36.799423+0000","last_active":"2023-09-30T22:22:26.793312+0000","last_peered":"2023-09-30T22:22:26.793312+0000","last_clean":"2023-09-30T22:22:26.793312+0000","last_became_active":"2023-09-30T22:21:36.798817+0000","last_became_peered":"2023-09-30T22:21:36.798817+0000","last_unstale":"2023-09-30T22:22:26.793312+0000","last_undegraded":"2023-09-30T22:22:26.793312+0000","last_fullsized":"2023-09-30T22:22:26.793312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794246+0000","last_change":"2023-09-30T22:21:37.237647+0000","last_active":"2023-09-30T22:22:26.794246+0000","last_peered":"2023-09-30T22:22:26.794246+0000","last_clean":"2023-09-30T22:22:26.794246+0000","last_became_active":"2023-09-30T22:21:37.237287+0000","last_became_peered":"2023-09-30T22:21:37.237287+0000","last_unstale":"2023-09-30T22:22:26.794246+0000","last_undegraded":"2023-09-30T22:22:26.794246+0000","last_fullsized":"2023-09-30T22:22:26.794246+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159690+0000","last_change":"2023-09-30T22:21:31.798294+0000","last_active":"2023-09-30T22:22:27.159690+0000","last_peered":"2023-09-30T22:22:27.159690+0000","last_clean":"2023-09-30T22:22:27.159690+0000","last_became_active":"2023-09-30T22:21:31.797999+0000","last_became_peered":"2023-09-30T22:21:31.797999+0000","last_unstale":"2023-09-30T22:22:27.159690+0000","last_undegraded":"2023-09-30T22:22:27.159690+0000","last_fullsized":"2023-09-30T22:22:27.159690+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793223+0000","last_change":"2023-09-30T22:21:30.796302+0000","last_active":"2023-09-30T22:22:26.793223+0000","last_peered":"2023-09-30T22:22:26.793223+0000","last_clean":"2023-09-30T22:22:26.793223+0000","last_became_active":"2023-09-30T22:21:30.795885+0000","last_became_peered":"2023-09-30T22:21:30.795885+0000","last_unstale":"2023-09-30T22:22:26.793223+0000","last_undegraded":"2023-09-30T22:22:26.793223+0000","last_fullsized":"2023-09-30T22:22:26.793223+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:29.785382+0000","last_clean_scrub_stamp":"2023-09-30T22:21:29.785382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159913+0000","last_change":"2023-09-30T22:21:36.811325+0000","last_active":"2023-09-30T22:22:27.159913+0000","last_peered":"2023-09-30T22:22:27.159913+0000","last_clean":"2023-09-30T22:22:27.159913+0000","last_became_active":"2023-09-30T22:21:36.811014+0000","last_became_peered":"2023-09-30T22:21:36.811014+0000","last_unstale":"2023-09-30T22:22:27.159913+0000","last_undegraded":"2023-09-30T22:22:27.159913+0000","last_fullsized":"2023-09-30T22:22:27.159913+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160664+0000","last_change":"2023-09-30T22:21:37.220230+0000","last_active":"2023-09-30T22:22:27.160664+0000","last_peered":"2023-09-30T22:22:27.160664+0000","last_clean":"2023-09-30T22:22:27.160664+0000","last_became_active":"2023-09-30T22:21:37.215165+0000","last_became_peered":"2023-09-30T22:21:37.215165+0000","last_unstale":"2023-09-30T22:22:27.160664+0000","last_undegraded":"2023-09-30T22:22:27.160664+0000","last_fullsized":"2023-09-30T22:22:27.160664+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159277+0000","last_change":"2023-09-30T22:21:31.798735+0000","last_active":"2023-09-30T22:22:27.159277+0000","last_peered":"2023-09-30T22:22:27.159277+0000","last_clean":"2023-09-30T22:22:27.159277+0000","last_became_active":"2023-09-30T22:21:31.798405+0000","last_became_peered":"2023-09-30T22:21:31.798405+0000","last_unstale":"2023-09-30T22:22:27.159277+0000","last_undegraded":"2023-09-30T22:22:27.159277+0000","last_fullsized":"2023-09-30T22:22:27.159277+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795327+0000","last_change":"2023-09-30T22:21:36.793235+0000","last_active":"2023-09-30T22:22:26.795327+0000","last_peered":"2023-09-30T22:22:26.795327+0000","last_clean":"2023-09-30T22:22:26.795327+0000","last_became_active":"2023-09-30T22:21:36.792921+0000","last_became_peered":"2023-09-30T22:21:36.792921+0000","last_unstale":"2023-09-30T22:22:26.795327+0000","last_undegraded":"2023-09-30T22:22:26.795327+0000","last_fullsized":"2023-09-30T22:22:26.795327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795776+0000","last_change":"2023-09-30T22:21:37.224362+0000","last_active":"2023-09-30T22:22:26.795776+0000","last_peered":"2023-09-30T22:22:26.795776+0000","last_clean":"2023-09-30T22:22:26.795776+0000","last_became_active":"2023-09-30T22:21:37.219539+0000","last_became_peered":"2023-09-30T22:21:37.219539+0000","last_unstale":"2023-09-30T22:22:26.795776+0000","last_undegraded":"2023-09-30T22:22:26.795776+0000","last_fullsized":"2023-09-30T22:22:26.795776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160545+0000","last_change":"2023-09-30T22:21:31.800121+0000","last_active":"2023-09-30T22:22:27.160545+0000","last_peered":"2023-09-30T22:22:27.160545+0000","last_clean":"2023-09-30T22:22:27.160545+0000","last_became_active":"2023-09-30T22:21:31.799775+0000","last_became_peered":"2023-09-30T22:21:31.799775+0000","last_unstale":"2023-09-30T22:22:27.160545+0000","last_undegraded":"2023-09-30T22:22:27.160545+0000","last_fullsized":"2023-09-30T22:22:27.160545+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793211+0000","last_change":"2023-09-30T22:21:36.799552+0000","last_active":"2023-09-30T22:22:26.793211+0000","last_peered":"2023-09-30T22:22:26.793211+0000","last_clean":"2023-09-30T22:22:26.793211+0000","last_became_active":"2023-09-30T22:21:36.798979+0000","last_became_peered":"2023-09-30T22:21:36.798979+0000","last_unstale":"2023-09-30T22:22:26.793211+0000","last_undegraded":"2023-09-30T22:22:26.793211+0000","last_fullsized":"2023-09-30T22:22:26.793211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.796109+0000","last_change":"2023-09-30T22:21:37.227699+0000","last_active":"2023-09-30T22:22:26.796109+0000","last_peered":"2023-09-30T22:22:26.796109+0000","last_clean":"2023-09-30T22:22:26.796109+0000","last_became_active":"2023-09-30T22:21:37.227403+0000","last_became_peered":"2023-09-30T22:21:37.227403+0000","last_unstale":"2023-09-30T22:22:26.796109+0000","last_undegraded":"2023-09-30T22:22:26.796109+0000","last_fullsized":"2023-09-30T22:22:26.796109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160571+0000","last_change":"2023-09-30T22:21:36.817294+0000","last_active":"2023-09-30T22:22:27.160571+0000","last_peered":"2023-09-30T22:22:27.160571+0000","last_clean":"2023-09-30T22:22:27.160571+0000","last_became_active":"2023-09-30T22:21:36.816805+0000","last_became_peered":"2023-09-30T22:21:36.816805+0000","last_unstale":"2023-09-30T22:22:27.160571+0000","last_undegraded":"2023-09-30T22:22:27.160571+0000","last_fullsized":"2023-09-30T22:22:27.160571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794258+0000","last_change":"2023-09-30T22:21:31.794560+0000","last_active":"2023-09-30T22:22:26.794258+0000","last_peered":"2023-09-30T22:22:26.794258+0000","last_clean":"2023-09-30T22:22:26.794258+0000","last_became_active":"2023-09-30T22:21:31.794101+0000","last_became_peered":"2023-09-30T22:21:31.794101+0000","last_unstale":"2023-09-30T22:22:26.794258+0000","last_undegraded":"2023-09-30T22:22:26.794258+0000","last_fullsized":"2023-09-30T22:22:26.794258+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159841+0000","last_change":"2023-09-30T22:21:37.214706+0000","last_active":"2023-09-30T22:22:27.159841+0000","last_peered":"2023-09-30T22:22:27.159841+0000","last_clean":"2023-09-30T22:22:27.159841+0000","last_became_active":"2023-09-30T22:21:37.212393+0000","last_became_peered":"2023-09-30T22:21:37.212393+0000","last_unstale":"2023-09-30T22:22:27.159841+0000","last_undegraded":"2023-09-30T22:22:27.159841+0000","last_fullsized":"2023-09-30T22:22:27.159841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160625+0000","last_change":"2023-09-30T22:21:31.800079+0000","last_active":"2023-09-30T22:22:27.160625+0000","last_peered":"2023-09-30T22:22:27.160625+0000","last_clean":"2023-09-30T22:22:27.160625+0000","last_became_active":"2023-09-30T22:21:31.799718+0000","last_became_peered":"2023-09-30T22:21:31.799718+0000","last_unstale":"2023-09-30T22:22:27.160625+0000","last_undegraded":"2023-09-30T22:22:27.160625+0000","last_fullsized":"2023-09-30T22:22:27.160625+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:30.786527+0000","last_clean_scrub_stamp":"2023-09-30T22:21:30.786527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"17'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795588+0000","last_change":"2023-09-30T22:21:36.803700+0000","last_active":"2023-09-30T22:22:26.795588+0000","last_peered":"2023-09-30T22:22:26.795588+0000","last_clean":"2023-09-30T22:22:26.795588+0000","last_became_active":"2023-09-30T22:21:36.803365+0000","last_became_peered":"2023-09-30T22:21:36.803365+0000","last_unstale":"2023-09-30T22:22:26.795588+0000","last_undegraded":"2023-09-30T22:22:26.795588+0000","last_fullsized":"2023-09-30T22:22:26.795588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079977+0000","last_change":"2023-09-30T22:21:37.226913+0000","last_active":"2023-09-30T22:23:54.079977+0000","last_peered":"2023-09-30T22:23:54.079977+0000","last_clean":"2023-09-30T22:23:54.079977+0000","last_became_active":"2023-09-30T22:21:37.225725+0000","last_became_peered":"2023-09-30T22:21:37.225725+0000","last_unstale":"2023-09-30T22:23:54.079977+0000","last_undegraded":"2023-09-30T22:23:54.079977+0000","last_fullsized":"2023-09-30T22:23:54.079977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795979+0000","last_change":"2023-09-30T22:21:36.808872+0000","last_active":"2023-09-30T22:22:26.795979+0000","last_peered":"2023-09-30T22:22:26.795979+0000","last_clean":"2023-09-30T22:22:26.795979+0000","last_became_active":"2023-09-30T22:21:36.806238+0000","last_became_peered":"2023-09-30T22:21:36.806238+0000","last_unstale":"2023-09-30T22:22:26.795979+0000","last_undegraded":"2023-09-30T22:22:26.795979+0000","last_fullsized":"2023-09-30T22:22:26.795979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.082007+0000","last_change":"2023-09-30T22:21:37.208459+0000","last_active":"2023-09-30T22:23:54.082007+0000","last_peered":"2023-09-30T22:23:54.082007+0000","last_clean":"2023-09-30T22:23:54.082007+0000","last_became_active":"2023-09-30T22:21:37.208076+0000","last_became_peered":"2023-09-30T22:21:37.208076+0000","last_unstale":"2023-09-30T22:23:54.082007+0000","last_undegraded":"2023-09-30T22:23:54.082007+0000","last_fullsized":"2023-09-30T22:23:54.082007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793418+0000","last_change":"2023-09-30T22:21:36.806969+0000","last_active":"2023-09-30T22:22:26.793418+0000","last_peered":"2023-09-30T22:22:26.793418+0000","last_clean":"2023-09-30T22:22:26.793418+0000","last_became_active":"2023-09-30T22:21:36.806649+0000","last_became_peered":"2023-09-30T22:21:36.806649+0000","last_unstale":"2023-09-30T22:22:26.793418+0000","last_undegraded":"2023-09-30T22:22:26.793418+0000","last_fullsized":"2023-09-30T22:22:26.793418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.081108+0000","last_change":"2023-09-30T22:21:37.229486+0000","last_active":"2023-09-30T22:23:54.081108+0000","last_peered":"2023-09-30T22:23:54.081108+0000","last_clean":"2023-09-30T22:23:54.081108+0000","last_became_active":"2023-09-30T22:21:37.226308+0000","last_became_peered":"2023-09-30T22:21:37.226308+0000","last_unstale":"2023-09-30T22:23:54.081108+0000","last_undegraded":"2023-09-30T22:23:54.081108+0000","last_fullsized":"2023-09-30T22:23:54.081108+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"16'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160557+0000","last_change":"2023-09-30T22:21:36.815782+0000","last_active":"2023-09-30T22:22:27.160557+0000","last_peered":"2023-09-30T22:22:27.160557+0000","last_clean":"2023-09-30T22:22:27.160557+0000","last_became_active":"2023-09-30T22:21:36.815431+0000","last_became_peered":"2023-09-30T22:21:36.815431+0000","last_unstale":"2023-09-30T22:22:27.160557+0000","last_undegraded":"2023-09-30T22:22:27.160557+0000","last_fullsized":"2023-09-30T22:22:27.160557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.080810+0000","last_change":"2023-09-30T22:21:37.215011+0000","last_active":"2023-09-30T22:23:54.080810+0000","last_peered":"2023-09-30T22:23:54.080810+0000","last_clean":"2023-09-30T22:23:54.080810+0000","last_became_active":"2023-09-30T22:21:37.214739+0000","last_became_peered":"2023-09-30T22:21:37.214739+0000","last_unstale":"2023-09-30T22:23:54.080810+0000","last_undegraded":"2023-09-30T22:23:54.080810+0000","last_fullsized":"2023-09-30T22:23:54.080810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"16'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160345+0000","last_change":"2023-09-30T22:21:36.812958+0000","last_active":"2023-09-30T22:22:27.160345+0000","last_peered":"2023-09-30T22:22:27.160345+0000","last_clean":"2023-09-30T22:22:27.160345+0000","last_became_active":"2023-09-30T22:21:36.812662+0000","last_became_peered":"2023-09-30T22:21:36.812662+0000","last_unstale":"2023-09-30T22:22:27.160345+0000","last_undegraded":"2023-09-30T22:22:27.160345+0000","last_fullsized":"2023-09-30T22:22:27.160345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.160001+0000","last_change":"2023-09-30T22:21:37.225020+0000","last_active":"2023-09-30T22:22:27.160001+0000","last_peered":"2023-09-30T22:22:27.160001+0000","last_clean":"2023-09-30T22:22:27.160001+0000","last_became_active":"2023-09-30T22:21:37.224728+0000","last_became_peered":"2023-09-30T22:21:37.224728+0000","last_unstale":"2023-09-30T22:22:27.160001+0000","last_undegraded":"2023-09-30T22:22:27.160001+0000","last_fullsized":"2023-09-30T22:22:27.160001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"16'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.795977+0000","last_change":"2023-09-30T22:21:36.796108+0000","last_active":"2023-09-30T22:22:26.795977+0000","last_peered":"2023-09-30T22:22:26.795977+0000","last_clean":"2023-09-30T22:22:26.795977+0000","last_became_active":"2023-09-30T22:21:36.795665+0000","last_became_peered":"2023-09-30T22:21:36.795665+0000","last_unstale":"2023-09-30T22:22:26.795977+0000","last_undegraded":"2023-09-30T22:22:26.795977+0000","last_fullsized":"2023-09-30T22:22:26.795977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:35.773378+0000","last_clean_scrub_stamp":"2023-09-30T22:21:35.773378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-09-30T22:23:54.079668+0000","last_change":"2023-09-30T22:21:37.207178+0000","last_active":"2023-09-30T22:23:54.079668+0000","last_peered":"2023-09-30T22:23:54.079668+0000","last_clean":"2023-09-30T22:23:54.079668+0000","last_became_active":"2023-09-30T22:21:37.206889+0000","last_became_peered":"2023-09-30T22:21:37.206889+0000","last_unstale":"2023-09-30T22:23:54.079668+0000","last_undegraded":"2023-09-30T22:23:54.079668+0000","last_fullsized":"2023-09-30T22:23:54.079668+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793393+0000","last_change":"2023-09-30T22:21:37.206551+0000","last_active":"2023-09-30T22:22:26.793393+0000","last_peered":"2023-09-30T22:22:26.793393+0000","last_clean":"2023-09-30T22:22:26.793393+0000","last_became_active":"2023-09-30T22:21:37.206176+0000","last_became_peered":"2023-09-30T22:21:37.206176+0000","last_unstale":"2023-09-30T22:22:26.793393+0000","last_undegraded":"2023-09-30T22:22:26.793393+0000","last_fullsized":"2023-09-30T22:22:26.793393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159699+0000","last_change":"2023-09-30T22:21:37.220615+0000","last_active":"2023-09-30T22:22:27.159699+0000","last_peered":"2023-09-30T22:22:27.159699+0000","last_clean":"2023-09-30T22:22:27.159699+0000","last_became_active":"2023-09-30T22:21:37.217961+0000","last_became_peered":"2023-09-30T22:21:37.217961+0000","last_unstale":"2023-09-30T22:22:27.159699+0000","last_undegraded":"2023-09-30T22:22:27.159699+0000","last_fullsized":"2023-09-30T22:22:27.159699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.794866+0000","last_change":"2023-09-30T22:21:37.229799+0000","last_active":"2023-09-30T22:22:26.794866+0000","last_peered":"2023-09-30T22:22:26.794866+0000","last_clean":"2023-09-30T22:22:26.794866+0000","last_became_active":"2023-09-30T22:21:37.229494+0000","last_became_peered":"2023-09-30T22:21:37.229494+0000","last_unstale":"2023-09-30T22:22:26.794866+0000","last_undegraded":"2023-09-30T22:22:26.794866+0000","last_fullsized":"2023-09-30T22:22:26.794866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:27.159832+0000","last_change":"2023-09-30T22:21:37.215694+0000","last_active":"2023-09-30T22:22:27.159832+0000","last_peered":"2023-09-30T22:22:27.159832+0000","last_clean":"2023-09-30T22:22:27.159832+0000","last_became_active":"2023-09-30T22:21:37.215371+0000","last_became_peered":"2023-09-30T22:21:37.215371+0000","last_unstale":"2023-09-30T22:22:27.159832+0000","last_undegraded":"2023-09-30T22:22:27.159832+0000","last_fullsized":"2023-09-30T22:22:27.159832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:22:26.793525+0000","last_change":"2023-09-30T22:21:37.206797+0000","last_active":"2023-09-30T22:22:26.793525+0000","last_peered":"2023-09-30T22:22:26.793525+0000","last_clean":"2023-09-30T22:22:26.793525+0000","last_became_active":"2023-09-30T22:21:37.206384+0000","last_became_peered":"2023-09-30T22:21:37.206384+0000","last_unstale":"2023-09-30T22:22:26.793525+0000","last_undegraded":"2023-09-30T22:22:26.793525+0000","last_fullsized":"2023-09-30T22:22:26.793525+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:21:36.775197+0000","last_clean_scrub_stamp":"2023-09-30T22:21:36.775197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":67217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.316,"15min":0.316},"min":{"1min":0.20300000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.39400000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.432,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33600000000000002}]},{"osd":1,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.371,"15min":0.377},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.45700000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.37},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.27900000000000003,"5min":0.215,"15min":0.215},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.26600000000000001}]},{"osd":2,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33200000000000002,"15min":0.33400000000000002},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.43099999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.315},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.42899999999999999,"5min":0.502,"15min":0.502},"last":0.34999999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":67023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.315,"15min":0.315},"min":{"1min":0.24199999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35599999999999998,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.35599999999999998}]},{"osd":1,"last update":"Sat Sep 30 22:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.28399999999999997,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35599999999999998,"15min":0.35699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33200000000000002}]},{"osd":3,"last update":"Sat Sep 30 22:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36299999999999999,"15min":0.36499999999999999},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.52600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39600000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297284,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074556,"statfs":{"total":96636764160,"available":96332345344,"internally_reserved":0,"allocated":192512,"data_stored":64160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 22:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35599999999999998,"15min":0.35699999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.48199999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Sep 30 22:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.37,"15min":0.372},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.39200000000000002}]},{"osd":3,"last update":"Sat Sep 30 22:23:31 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.36199999999999999,"15min":0.36599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.40899999999999997,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.376}]}]},{"osd":1,"up_from":9,"seq":38654705695,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297280,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074560,"statfs":{"total":96636764160,"available":96332349440,"internally_reserved":0,"allocated":188416,"data_stored":64106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.441,"5min":0.441,"15min":0.441},"last":0.503},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.39400000000000002}]},{"osd":2,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.33200000000000002,"15min":0.33000000000000002},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.374}]},{"osd":3,"last update":"Sat Sep 30 22:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.495,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.374,"5min":0.38100000000000001,"15min":0.38200000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:23:58.230 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T22:23:58.230 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T22:23:58.230 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T22:23:58.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T22:23:58.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.402+0000 7f161c57b700 1 Processor -- start 2023-09-30T22:23:58.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.402+0000 7f161c57b700 1 -- start start 2023-09-30T22:23:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161c57b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f161409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161c57b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f161409f240 0x7f16141317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161c57b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1614070940 con 0x7f161409f650 2023-09-30T22:23:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161c57b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161406fd60 con 0x7f161409f240 2023-09-30T22:23:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161c57b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161406fa80 con 0x7f161409e930 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161a317700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f161409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619b16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f161409f240 0x7f16141317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161ab18700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f161409f650 0x7f1614131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50660/0 (socket says 172.21.15.32:50660) 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161ab18700 1 -- 172.21.15.32:0/1472478576 learned_addr learned my addr 172.21.15.32:0/1472478576 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619315700 1 -- 172.21.15.32:0/1472478576 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292511565 0 0) 0x7f1614070940 con 0x7f161409f650 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619315700 1 -- 172.21.15.32:0/1472478576 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16000035b0 con 0x7f161409f650 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619b16700 1 -- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 msgr2=0x7f161409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619b16700 1 --2- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f161409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619b16700 1 -- 172.21.15.32:0/1472478576 >> v1:172.21.15.32:6789/0 conn(0x7f161409f650 legacy=0x7f1614131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f1619b16700 1 -- 172.21.15.32:0/1472478576 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161409fb60 con 0x7f161409f240 2023-09-30T22:23:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.403+0000 7f161a317700 1 --2- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f161409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f1619b16700 1 --2- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141317f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1610009840 tx=0x7f1610009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e934a9a55c0c7bb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f1619315700 1 -- 172.21.15.32:0/1472478576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610011830 con 0x7f161409f240 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f1619315700 1 -- 172.21.15.32:0/1472478576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1610011990 con 0x7f161409f240 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f1619315700 1 -- 172.21.15.32:0/1472478576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610011d20 con 0x7f161409f240 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 -- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 msgr2=0x7f16141317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 --2- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141317f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1610009840 tx=0x7f1610009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 -- 172.21.15.32:0/1472478576 shutdown_connections 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 --2- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f161409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 --2- 172.21.15.32:0/1472478576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141317f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 -- 172.21.15.32:0/1472478576 >> 172.21.15.32:0/1472478576 conn(0x7f16140a4210 msgr2=0x7f16140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.404+0000 7f161c57b700 1 -- 172.21.15.32:0/1472478576 shutdown_connections 2023-09-30T22:23:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.405+0000 7f161c57b700 1 -- 172.21.15.32:0/1472478576 wait complete. 2023-09-30T22:23:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.405+0000 7f161c57b700 1 Processor -- start 2023-09-30T22:23:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 -- start start 2023-09-30T22:23:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f16141a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f161409f650 0x7f16141a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16141a5f00 con 0x7f161409f240 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16141a6bc0 con 0x7f161409e930 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161c57b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16140a32f0 con 0x7f161409f650 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f16141a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f1619b16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f16141a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52490/0 (socket says 172.21.15.32:52490) 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 -- 172.21.15.32:0/621094635 learned_addr learned my addr 172.21.15.32:0/621094635 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161ab18700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f161409f650 0x7f16141a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 -- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f161409f650 msgr2=0x7f16141a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f161409f650 0x7f16141a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 -- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 msgr2=0x7f16141a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f161a317700 1 -- 172.21.15.32:0/621094635 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16040094a0 con 0x7f161409e930 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.406+0000 7f1619b16700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f161a317700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f16141a3cd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f160400ebe0 tx=0x7f160400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160400c850 con 0x7f161409e930 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160400c9b0 con 0x7f161409e930 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16100094a0 con 0x7f161409e930 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160400cd40 con 0x7f161409e930 2023-09-30T22:23:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16140a0190 con 0x7f161409e930 2023-09-30T22:23:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.407+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f1604013470 con 0x7f161409e930 2023-09-30T22:23:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.408+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f160403ff10 con 0x7f161409e930 2023-09-30T22:23:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.408+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16141a0b30 con 0x7f161409e930 2023-09-30T22:23:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.413+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1604021080 con 0x7f161409e930 2023-09-30T22:23:58.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.659+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1614074ea0 con 0x7f161409e930 2023-09-30T22:23:58.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.659+0000 7f160affd700 1 -- 172.21.15.32:0/621094635 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1604017020 con 0x7f161409e930 2023-09-30T22:23:58.661 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:58.661 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 >> v1:172.21.15.32:6816/126553 conn(0x7f15f802f480 legacy=0x7f15f8031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 msgr2=0x7f16141a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f16141a3cd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f160400ebe0 tx=0x7f160400eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 shutdown_connections 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f161409f650 0x7f16141a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f161409e930 0x7f16141a3cd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 --2- 172.21.15.32:0/621094635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161409f240 0x7f16141a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 >> 172.21.15.32:0/621094635 conn(0x7f16140a4210 msgr2=0x7f16141b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 shutdown_connections 2023-09-30T22:23:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.662+0000 7f161c57b700 1 -- 172.21.15.32:0/621094635 wait complete. 2023-09-30T22:23:58.676 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-30T22:23:58.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:23:58.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.811+0000 7f8c234c9700 1 Processor -- start 2023-09-30T22:23:58.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.811+0000 7f8c234c9700 1 -- start start 2023-09-30T22:23:58.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c234c9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 0x7f8c1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c234c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c1c09e000 0x7f8c1c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c234c9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c133f30 0x7f8c1c1343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c234c9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c09b0f0 con 0x7f8c1c09e000 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c234c9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c09d2f0 con 0x7f8c1c133f30 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c234c9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c09a810 con 0x7f8c1c09d6f0 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21265700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 0x7f8c1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21a66700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c133f30 0x7f8c1c1343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21265700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 0x7f8c1c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41994/0 (socket says 172.21.15.32:41994) 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21265700 1 -- 172.21.15.32:0/3663655378 learned_addr learned my addr 172.21.15.32:0/3663655378 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c20a64700 1 --2- 172.21.15.32:0/3663655378 >> v2:172.21.15.32:3300/0 conn(0x7f8c1c09e000 0x7f8c1c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21a66700 1 -- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 msgr2=0x7f8c1c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21a66700 1 --2- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 0x7f8c1c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21a66700 1 -- 172.21.15.32:0/3663655378 >> v2:172.21.15.32:3300/0 conn(0x7f8c1c09e000 msgr2=0x7f8c1c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21a66700 1 --2- 172.21.15.32:0/3663655378 >> v2:172.21.15.32:3300/0 conn(0x7f8c1c09e000 0x7f8c1c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21a66700 1 -- 172.21.15.32:0/3663655378 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c09a470 con 0x7f8c1c133f30 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c21265700 1 --2- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 0x7f8c1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.812+0000 7f8c20a64700 1 --2- 172.21.15.32:0/3663655378 >> v2:172.21.15.32:3300/0 conn(0x7f8c1c09e000 0x7f8c1c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c21a66700 1 --2- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c133f30 0x7f8c1c1343b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009d90 tx=0x7f8c1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c2ed7149dd8589 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c0ffff700 1 -- 172.21.15.32:0/3663655378 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400e070 con 0x7f8c1c133f30 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c0ffff700 1 -- 172.21.15.32:0/3663655378 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1400bb20 con 0x7f8c1c133f30 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c0ffff700 1 -- 172.21.15.32:0/3663655378 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14012430 con 0x7f8c1c133f30 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 -- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c133f30 msgr2=0x7f8c1c1343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 --2- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c133f30 0x7f8c1c1343b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009d90 tx=0x7f8c1400b330 comp rx=0 tx=0).stop 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 -- 172.21.15.32:0/3663655378 shutdown_connections 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 --2- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c133f30 0x7f8c1c1343b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 --2- 172.21.15.32:0/3663655378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c09d6f0 0x7f8c1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 --2- 172.21.15.32:0/3663655378 >> v2:172.21.15.32:3300/0 conn(0x7f8c1c09e000 0x7f8c1c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.813+0000 7f8c234c9700 1 -- 172.21.15.32:0/3663655378 >> 172.21.15.32:0/3663655378 conn(0x7f8c1c066f50 msgr2=0x7f8c1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.814+0000 7f8c234c9700 1 -- 172.21.15.32:0/3663655378 shutdown_connections 2023-09-30T22:23:58.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.814+0000 7f8c234c9700 1 -- 172.21.15.32:0/3663655378 wait complete. 2023-09-30T22:23:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.814+0000 7f8c234c9700 1 Processor -- start 2023-09-30T22:23:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 -- start start 2023-09-30T22:23:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c1c09d6f0 0x7f8c1c1318a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c09e000 0x7f8c1c131de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c133f30 0x7f8c1c1c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:58.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c11e180 con 0x7f8c1c09d6f0 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c09c640 con 0x7f8c1c09e000 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c234c9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c1213e0 con 0x7f8c1c133f30 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c21265700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c1c09d6f0 0x7f8c1c1318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c21265700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c1c09d6f0 0x7f8c1c1318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46746/0 (socket says 172.21.15.32:46746) 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c21265700 1 -- 172.21.15.32:0/2181884178 learned_addr learned my addr 172.21.15.32:0/2181884178 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c20a64700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c09e000 0x7f8c1c131de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c21a66700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c133f30 0x7f8c1c1c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.815+0000 7f8c20a64700 1 -- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c133f30 msgr2=0x7f8c1c1c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c20a64700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c133f30 0x7f8c1c1c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c20a64700 1 -- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c1c09d6f0 msgr2=0x7f8c1c1318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c20a64700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c1c09d6f0 0x7f8c1c1318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:58.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c20a64700 1 -- 172.21.15.32:0/2181884178 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18009550 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c21a66700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c133f30 0x7f8c1c1c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c20a64700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c09e000 0x7f8c1c131de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000e980 tx=0x7f8c1000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000c690 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1000c7f0 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000cb80 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c100094a0 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.816+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c14009a70 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.817+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f8c1000cce0 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.817+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f8c1003fb50 con 0x7f8c1c09e000 2023-09-30T22:23:58.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.817+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c074ea0 con 0x7f8c1c09e000 2023-09-30T22:23:58.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:58.822+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c10017080 con 0x7f8c1c09e000 2023-09-30T22:23:59.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.028+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c1c11fec0 con 0x7f8c1c09e000 2023-09-30T22:23:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.029+0000 7f8c0dffb700 1 -- 172.21.15.32:0/2181884178 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12864 (secure 0 0 0) 0x7f8c1001bdf0 con 0x7f8c1c09e000 2023-09-30T22:23:59.030 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:23:59.030 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:23:56.022608+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.031+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 >> v1:172.21.15.32:6816/126553 conn(0x7f8c0002f480 legacy=0x7f8c00031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.031+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c09e000 msgr2=0x7f8c1c131de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c09e000 0x7f8c1c131de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000e980 tx=0x7f8c1000ec90 comp rx=0 tx=0).stop 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 shutdown_connections 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c1c133f30 0x7f8c1c1c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c1c09d6f0 0x7f8c1c1318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 --2- 172.21.15.32:0/2181884178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c1c09e000 0x7f8c1c131de0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 >> 172.21.15.32:0/2181884178 conn(0x7f8c1c066f50 msgr2=0x7f8c1c0a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 shutdown_connections 2023-09-30T22:23:59.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.032+0000 7f8c234c9700 1 -- 172.21.15.32:0/2181884178 wait complete. 2023-09-30T22:23:59.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:23:59.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:23:59.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.187+0000 7f8afc4cf700 1 Processor -- start 2023-09-30T22:23:59.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.187+0000 7f8afc4cf700 1 -- start start 2023-09-30T22:23:59.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afc4cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af4131b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afc4cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406a770 0x7f8af406abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afc4cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 0x7f8af41385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afc4cf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af409ea40 con 0x7f8af406b100 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afc4cf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4131350 con 0x7f8af406a770 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afc4cf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8af409dfc0 con 0x7f8af4131760 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8af9a6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406a770 0x7f8af406abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afaa6c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 0x7f8af41385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af4131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af4131b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42010/0 (socket says 172.21.15.32:42010) 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 -- 172.21.15.32:0/2139475622 learned_addr learned my addr 172.21.15.32:0/2139475622 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afaa6c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 0x7f8af41385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46762/0 (socket says 172.21.15.32:46762) 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8af9a6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406a770 0x7f8af406abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52522/0 (socket says 172.21.15.32:52522) 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 -- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406a770 msgr2=0x7f8af406abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 --2- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406a770 0x7f8af406abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 -- 172.21.15.32:0/2139475622 >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 msgr2=0x7f8af41385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 --2- 172.21.15.32:0/2139475622 >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 0x7f8af41385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 -- 172.21.15.32:0/2139475622 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af409e210 con 0x7f8af4131760 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afaa6c700 1 --2- 172.21.15.32:0/2139475622 >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 0x7f8af41385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.188+0000 7f8afa26b700 1 --2- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af4131b30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8af00098f0 tx=0x7f8af0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed61f055bc518f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8af9269700 1 -- 172.21.15.32:0/2139475622 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011830 con 0x7f8af4131760 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8af9269700 1 -- 172.21.15.32:0/2139475622 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af0011990 con 0x7f8af4131760 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8af9269700 1 -- 172.21.15.32:0/2139475622 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011d20 con 0x7f8af4131760 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 -- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 msgr2=0x7f8af4131b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af4131b30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8af00098f0 tx=0x7f8af0009c00 comp rx=0 tx=0).stop 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 -- 172.21.15.32:0/2139475622 shutdown_connections 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/2139475622 >> v2:172.21.15.32:3300/0 conn(0x7f8af406b100 0x7f8af41385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af4131b30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/2139475622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406a770 0x7f8af406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 -- 172.21.15.32:0/2139475622 >> 172.21.15.32:0/2139475622 conn(0x7f8af4066d70 msgr2=0x7f8af40a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 -- 172.21.15.32:0/2139475622 shutdown_connections 2023-09-30T22:23:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.189+0000 7f8afc4cf700 1 -- 172.21.15.32:0/2139475622 wait complete. 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.190+0000 7f8afc4cf700 1 Processor -- start 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.190+0000 7f8afc4cf700 1 -- start start 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afc4cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 0x7f8af406e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afc4cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406b100 0x7f8af406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afc4cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afc4cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af412b9a0 con 0x7f8af406a770 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afc4cf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4070ca0 con 0x7f8af406b100 2023-09-30T22:23:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afc4cf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4072140 con 0x7f8af4131760 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 0x7f8af406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afaa6c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 0x7f8af406e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46766/0 (socket says 172.21.15.32:46766) 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 -- 172.21.15.32:0/397029130 learned_addr learned my addr 172.21.15.32:0/397029130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8af9a6a700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406b100 0x7f8af406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 -- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 msgr2=0x7f8af406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af406f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 -- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406b100 msgr2=0x7f8af406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406b100 0x7f8af406eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afa26b700 1 -- 172.21.15.32:0/397029130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec009a70 con 0x7f8af406a770 2023-09-30T22:23:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.191+0000 7f8afaa6c700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af406f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.192+0000 7f8afa26b700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 0x7f8af406e860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8af409e150 tx=0x7f8af0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.192+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0004450 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.192+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00045d0 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.192+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af0009550 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.192+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00038e0 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.192+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af40a3280 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.193+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8af0003a40 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.193+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f8af0040ae0 con 0x7f8af406a770 2023-09-30T22:23:59.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.193+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af409e680 con 0x7f8af406a770 2023-09-30T22:23:59.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.198+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8af0015080 con 0x7f8af406a770 2023-09-30T22:23:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.399+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8af404f2a0 con 0x7f8af406a770 2023-09-30T22:23:59.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.400+0000 7f8aeaffd700 1 -- 172.21.15.32:0/397029130 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8af0028030 con 0x7f8af406a770 2023-09-30T22:23:59.401 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:23:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 >> v1:172.21.15.32:6816/126553 conn(0x7f8ad802f540 legacy=0x7f8ad80319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 msgr2=0x7f8af406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 0x7f8af406e860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8af409e150 tx=0x7f8af0004750 comp rx=0 tx=0).stop 2023-09-30T22:23:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 shutdown_connections 2023-09-30T22:23:59.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4131760 0x7f8af406f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af406a770 0x7f8af406e860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 --2- 172.21.15.32:0/397029130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af406b100 0x7f8af406eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 >> 172.21.15.32:0/397029130 conn(0x7f8af4066d70 msgr2=0x7f8af4130150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 shutdown_connections 2023-09-30T22:23:59.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.402+0000 7f8afc4cf700 1 -- 172.21.15.32:0/397029130 wait complete. 2023-09-30T22:23:59.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:23:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 Processor -- start 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 -- start start 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 0x7f574809f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748133930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574809d6c0 con 0x7f574809fb10 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57480a0a30 con 0x7f574809edd0 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.555+0000 7f574d443700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574809dc50 con 0x7f574809f6e0 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5746ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 0x7f574809f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5746ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 0x7f574809f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46774/0 (socket says 172.21.15.32:46774) 2023-09-30T22:23:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748133930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52536/0 (socket says 172.21.15.32:52536) 2023-09-30T22:23:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57477fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f574809fb10 0x7f5748133e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50668/0 (socket says 172.21.15.32:50668) 2023-09-30T22:23:59.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5746ffd700 1 -- 172.21.15.32:0/4121438427 learned_addr learned my addr 172.21.15.32:0/4121438427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 -- 172.21.15.32:0/4121438427 >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 msgr2=0x7f574809f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:59.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 --2- 172.21.15.32:0/4121438427 >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 0x7f574809f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 -- 172.21.15.32:0/4121438427 >> v1:172.21.15.32:6789/0 conn(0x7f574809fb10 legacy=0x7f5748133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 -- 172.21.15.32:0/4121438427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574809df10 con 0x7f574809f6e0 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5746ffd700 1 --2- 172.21.15.32:0/4121438427 >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 0x7f574809f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5745ffb700 1 -- 172.21.15.32:0/4121438427 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308237192 0 0) 0x7f574809d6c0 con 0x7f574809fb10 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f57467fc700 1 --2- 172.21.15.32:0/4121438427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748133930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5730009840 tx=0x7f5730009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff11fc47ef82216 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5745ffb700 1 -- 172.21.15.32:0/4121438427 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730011830 con 0x7f574809f6e0 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5745ffb700 1 -- 172.21.15.32:0/4121438427 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730011990 con 0x7f574809f6e0 2023-09-30T22:23:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.556+0000 7f5745ffb700 1 -- 172.21.15.32:0/4121438427 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730011d20 con 0x7f574809f6e0 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 -- 172.21.15.32:0/4121438427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 msgr2=0x7f5748133930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 --2- 172.21.15.32:0/4121438427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748133930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5730009840 tx=0x7f5730009b50 comp rx=0 tx=0).stop 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 -- 172.21.15.32:0/4121438427 shutdown_connections 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 --2- 172.21.15.32:0/4121438427 >> v2:172.21.15.32:3300/0 conn(0x7f574809edd0 0x7f574809f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 --2- 172.21.15.32:0/4121438427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748133930 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 -- 172.21.15.32:0/4121438427 >> 172.21.15.32:0/4121438427 conn(0x7f57480a3050 msgr2=0x7f57480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 -- 172.21.15.32:0/4121438427 shutdown_connections 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.557+0000 7f574d443700 1 -- 172.21.15.32:0/4121438427 wait complete. 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 Processor -- start 2023-09-30T22:23:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 -- start start 2023-09-30T22:23:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 0x7f5748122250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574809fb10 0x7f5748122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5748125860 con 0x7f574809edd0 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5748124cf0 con 0x7f574809f6e0 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.559+0000 7f574d443700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57480a0330 con 0x7f574809fb10 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 0x7f5748122250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f57467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 0x7f5748122250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46790/0 (socket says 172.21.15.32:46790) 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 -- 172.21.15.32:0/587358185 learned_addr learned my addr 172.21.15.32:0/587358185 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f57477fe700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574809fb10 0x7f5748122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 -- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574809fb10 msgr2=0x7f5748122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574809fb10 0x7f5748122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 -- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 msgr2=0x7f5748122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 -- 172.21.15.32:0/587358185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738009500 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f57467fc700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f5746ffd700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 0x7f5748122250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f573800ebe0 tx=0x7f573800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738005a20 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738005b80 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57300094a0 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800c850 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.560+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5748125550 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.561+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f573800c9b0 con 0x7f574809edd0 2023-09-30T22:23:59.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.561+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f573803fe40 con 0x7f574809edd0 2023-09-30T22:23:59.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.561+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5748125e00 con 0x7f574809edd0 2023-09-30T22:23:59.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.566+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5738006030 con 0x7f574809edd0 2023-09-30T22:23:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.771+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f574804f2a0 con 0x7f574809edd0 2023-09-30T22:23:59.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.772+0000 7f573f7fe700 1 -- 172.21.15.32:0/587358185 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5738011b40 con 0x7f574809edd0 2023-09-30T22:23:59.773 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:23:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 >> v1:172.21.15.32:6816/126553 conn(0x7f572802f510 legacy=0x7f57280319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 msgr2=0x7f5748122250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 0x7f5748122250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f573800ebe0 tx=0x7f573800eef0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 shutdown_connections 2023-09-30T22:23:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574809fb10 0x7f5748122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574809edd0 0x7f5748122250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 --2- 172.21.15.32:0/587358185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574809f6e0 0x7f5748122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 >> 172.21.15.32:0/587358185 conn(0x7f57480a3050 msgr2=0x7f5748133bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 shutdown_connections 2023-09-30T22:23:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.774+0000 7f574d443700 1 -- 172.21.15.32:0/587358185 wait complete. 2023-09-30T22:23:59.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:23:59.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 Processor -- start 2023-09-30T22:23:59.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 -- start start 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d8136780 0x7f10d8138b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d813b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d81301c0 con 0x7f10d81303c0 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809f620 con 0x7f10d81390a0 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10de3ce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809e1b0 con 0x7f10d8136780 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10d77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d8136780 0x7f10d8138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10d7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10d81303c0 0x7f10d8130790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50682/0 (socket says 172.21.15.32:50682) 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10d7fff700 1 -- 172.21.15.32:0/3360067772 learned_addr learned my addr 172.21.15.32:0/3360067772 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.927+0000 7f10dc96b700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d813b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d77fe700 1 -- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 msgr2=0x7f10d813b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d77fe700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d813b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d77fe700 1 -- 172.21.15.32:0/3360067772 >> v1:172.21.15.32:6789/0 conn(0x7f10d81303c0 legacy=0x7f10d8130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d77fe700 1 -- 172.21.15.32:0/3360067772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80a3a60 con 0x7f10d8136780 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d6ffd700 1 -- 172.21.15.32:0/3360067772 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143145635 0 0) 0x7f10d81301c0 con 0x7f10d81303c0 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10dc96b700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d77fe700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d8136780 0x7f10d8138b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009870 tx=0x7f10cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fdc5cc8792fdafa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d6ffd700 1 -- 172.21.15.32:0/3360067772 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc011830 con 0x7f10d8136780 2023-09-30T22:23:59.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d6ffd700 1 -- 172.21.15.32:0/3360067772 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc011990 con 0x7f10d8136780 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.928+0000 7f10d6ffd700 1 -- 172.21.15.32:0/3360067772 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc011d20 con 0x7f10d8136780 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 -- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d8136780 msgr2=0x7f10d8138b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d8136780 0x7f10d8138b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009870 tx=0x7f10cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 -- 172.21.15.32:0/3360067772 shutdown_connections 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 --2- 172.21.15.32:0/3360067772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d8136780 0x7f10d8138b60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 -- 172.21.15.32:0/3360067772 >> 172.21.15.32:0/3360067772 conn(0x7f10d8066d70 msgr2=0x7f10d80a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 -- 172.21.15.32:0/3360067772 shutdown_connections 2023-09-30T22:23:59.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.929+0000 7f10de3ce700 1 -- 172.21.15.32:0/3360067772 wait complete. 2023-09-30T22:23:59.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.930+0000 7f10de3ce700 1 Processor -- start 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.930+0000 7f10de3ce700 1 -- start start 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.930+0000 7f10de3ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 0x7f10d812cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.930+0000 7f10de3ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d8136780 0x7f10d812d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.930+0000 7f10de3ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d812d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10de3ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8072470 con 0x7f10d8136780 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10de3ce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8070160 con 0x7f10d81390a0 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10de3ce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80707e0 con 0x7f10d81303c0 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 0x7f10d812cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 0x7f10d812cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42052/0 (socket says 172.21.15.32:42052) 2023-09-30T22:23:59.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d8136780 0x7f10d812d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 -- 172.21.15.32:0/2416193154 learned_addr learned my addr 172.21.15.32:0/2416193154 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10dc96b700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d812d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 -- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 msgr2=0x7f10d812d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d812d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 -- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d8136780 msgr2=0x7f10d812d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d8136780 0x7f10d812d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 -- 172.21.15.32:0/2416193154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009a70 con 0x7f10d81303c0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d7fff700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 0x7f10d812cb40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10c800c520 tx=0x7f10c800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8004c80 con 0x7f10d81303c0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c8004e00 con 0x7f10d81303c0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10cc0094a0 con 0x7f10d81303c0 2023-09-30T22:23:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8007310 con 0x7f10d81303c0 2023-09-30T22:23:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.931+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d8070e90 con 0x7f10d81303c0 2023-09-30T22:23:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.932+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f10c8007470 con 0x7f10d81303c0 2023-09-30T22:23:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.932+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f10c803f6e0 con 0x7f10d81303c0 2023-09-30T22:23:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.932+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d8070380 con 0x7f10d81303c0 2023-09-30T22:23:59.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:23:59.938+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10c80123b0 con 0x7f10d81303c0 2023-09-30T22:24:00.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.141+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10d804f2a0 con 0x7f10d81303c0 2023-09-30T22:24:00.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.142+0000 7f10d4ff9700 1 -- 172.21.15.32:0/2416193154 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10c8011360 con 0x7f10d81303c0 2023-09-30T22:24:00.143 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:24:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 >> v1:172.21.15.32:6816/126553 conn(0x7f10c402f540 legacy=0x7f10c40319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 msgr2=0x7f10d812cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 0x7f10d812cb40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10c800c520 tx=0x7f10c800c830 comp rx=0 tx=0).stop 2023-09-30T22:24:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 shutdown_connections 2023-09-30T22:24:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d81390a0 0x7f10d812d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d81303c0 0x7f10d812cb40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 --2- 172.21.15.32:0/2416193154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d8136780 0x7f10d812d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 >> 172.21.15.32:0/2416193154 conn(0x7f10d8066d70 msgr2=0x7f10d80a2610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 shutdown_connections 2023-09-30T22:24:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.144+0000 7f10de3ce700 1 -- 172.21.15.32:0/2416193154 wait complete. 2023-09-30T22:24:00.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:24:00.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.297+0000 7f30e5ad6700 1 Processor -- start 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.297+0000 7f30e5ad6700 1 -- start start 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30e5ad6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e01303c0 0x7f30e0130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30e5ad6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30e006a770 0x7f30e006ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30e5ad6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e009f620 con 0x7f30e006b0c0 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30e5ad6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e0130280 con 0x7f30e006a770 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30e5ad6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e009e1b0 con 0x7f30e01303c0 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30df7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e01303c0 0x7f30e0130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30deffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30e006a770 0x7f30e006ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30dffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f30e006b0c0 0x7f30e013a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50686/0 (socket says 172.21.15.32:50686) 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30dffff700 1 -- 172.21.15.32:0/3918114204 learned_addr learned my addr 172.21.15.32:0/3918114204 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30deffd700 1 -- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e01303c0 msgr2=0x7f30e0130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30deffd700 1 --2- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e01303c0 0x7f30e0130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30deffd700 1 -- 172.21.15.32:0/3918114204 >> v1:172.21.15.32:6789/0 conn(0x7f30e006b0c0 legacy=0x7f30e013a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30deffd700 1 -- 172.21.15.32:0/3918114204 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e006b540 con 0x7f30e006a770 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30de7fc700 1 -- 172.21.15.32:0/3918114204 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838470819 0 0) 0x7f30e009f620 con 0x7f30e006b0c0 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.298+0000 7f30df7fe700 1 --2- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e01303c0 0x7f30e0130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30deffd700 1 --2- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006a770 0x7f30e006ab80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009840 tx=0x7f30d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e7ae831948e5be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30de7fc700 1 -- 172.21.15.32:0/3918114204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011830 con 0x7f30e006a770 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30de7fc700 1 -- 172.21.15.32:0/3918114204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0011990 con 0x7f30e006a770 2023-09-30T22:24:00.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30de7fc700 1 -- 172.21.15.32:0/3918114204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011d20 con 0x7f30e006a770 2023-09-30T22:24:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 -- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006a770 msgr2=0x7f30e006ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006a770 0x7f30e006ab80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009840 tx=0x7f30d0009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 -- 172.21.15.32:0/3918114204 shutdown_connections 2023-09-30T22:24:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e01303c0 0x7f30e0130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/3918114204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006a770 0x7f30e006ab80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 -- 172.21.15.32:0/3918114204 >> 172.21.15.32:0/3918114204 conn(0x7f30e0066d70 msgr2=0x7f30e00a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 -- 172.21.15.32:0/3918114204 shutdown_connections 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.299+0000 7f30e5ad6700 1 -- 172.21.15.32:0/3918114204 wait complete. 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.300+0000 7f30e5ad6700 1 Processor -- start 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 -- start start 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 0x7f30e011d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 0x7f30e011de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e01303c0 0x7f30e011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e011fd40 con 0x7f30e006b0c0 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e0071860 con 0x7f30e01303c0 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30e5ad6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e0071fe0 con 0x7f30e006a770 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30df7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 0x7f30e011d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 0x7f30e011de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30dffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e01303c0 0x7f30e011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30df7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 0x7f30e011d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42068/0 (socket says 172.21.15.32:42068) 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 0x7f30e011de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46826/0 (socket says 172.21.15.32:46826) 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.301+0000 7f30df7fe700 1 -- 172.21.15.32:0/2574183503 learned_addr learned my addr 172.21.15.32:0/2574183503 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30deffd700 1 -- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 msgr2=0x7f30e011d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30deffd700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 0x7f30e011d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30deffd700 1 -- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e01303c0 msgr2=0x7f30e011e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30deffd700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e01303c0 0x7f30e011e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30deffd700 1 -- 172.21.15.32:0/2574183503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80094a0 con 0x7f30e006b0c0 2023-09-30T22:24:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30df7fe700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 0x7f30e011d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30deffd700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 0x7f30e011de70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30d0016000 tx=0x7f30d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00036a0 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d001f070 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d00094a0 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0003a40 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.302+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e011eda0 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.303+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f30d0003ba0 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.303+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f30d0021070 con 0x7f30e006b0c0 2023-09-30T22:24:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.303+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e011f900 con 0x7f30e006b0c0 2023-09-30T22:24:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.308+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30d002b080 con 0x7f30e006b0c0 2023-09-30T22:24:00.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.511+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30e004f2a0 con 0x7f30e006b0c0 2023-09-30T22:24:00.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.512+0000 7f30d7fff700 1 -- 172.21.15.32:0/2574183503 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30d0026480 con 0x7f30e006b0c0 2023-09-30T22:24:00.513 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:24:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 >> v1:172.21.15.32:6816/126553 conn(0x7f30c002f510 legacy=0x7f30c00319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 msgr2=0x7f30e011de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 0x7f30e011de70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30d0016000 tx=0x7f30d0004750 comp rx=0 tx=0).stop 2023-09-30T22:24:00.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 shutdown_connections 2023-09-30T22:24:00.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e01303c0 0x7f30e011e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e006a770 0x7f30e011d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 --2- 172.21.15.32:0/2574183503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e006b0c0 0x7f30e011de70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 >> 172.21.15.32:0/2574183503 conn(0x7f30e0066d70 msgr2=0x7f30e00a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:00.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 shutdown_connections 2023-09-30T22:24:00.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.514+0000 7f30e5ad6700 1 -- 172.21.15.32:0/2574183503 wait complete. 2023-09-30T22:24:00.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:24:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.663+0000 7f55ee8fd700 1 Processor -- start 2023-09-30T22:24:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.663+0000 7f55ee8fd700 1 -- start start 2023-09-30T22:24:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ee8fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55e806b250 0x7f55e8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ee8fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ee8fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e809ece0 con 0x7f55e806a770 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ee8fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e809e760 con 0x7f55e806b250 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ee8fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55e809e040 con 0x7f55e8136d80 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ece9a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55e77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55e806b250 0x7f55e8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ece9a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e8139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55730/0 (socket says 172.21.15.32:55730) 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55ece9a700 1 -- 172.21.15.32:0/2369834157 learned_addr learned my addr 172.21.15.32:0/2369834157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55e7fff700 1 --1- 172.21.15.32:0/2369834157 >> v1:172.21.15.32:6789/0 conn(0x7f55e806a770 0x7f55e806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42832/0 (socket says 172.21.15.32:42832) 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55e77fe700 1 -- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 msgr2=0x7f55e8139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.664+0000 7f55e77fe700 1 --2- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e8139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e77fe700 1 -- 172.21.15.32:0/2369834157 >> v1:172.21.15.32:6789/0 conn(0x7f55e806a770 legacy=0x7f55e806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e77fe700 1 -- 172.21.15.32:0/2369834157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e809f630 con 0x7f55e806b250 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e6ffd700 1 -- 172.21.15.32:0/2369834157 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332863758 0 0) 0x7f55e809ece0 con 0x7f55e806a770 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ece9a700 1 --2- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e77fe700 1 --2- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806b250 0x7f55e8136840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009840 tx=0x7f55dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d1a83fadb574a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e6ffd700 1 -- 172.21.15.32:0/2369834157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc011830 con 0x7f55e806b250 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e6ffd700 1 -- 172.21.15.32:0/2369834157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc011990 con 0x7f55e806b250 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55e6ffd700 1 -- 172.21.15.32:0/2369834157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc011d20 con 0x7f55e806b250 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ee8fd700 1 -- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806b250 msgr2=0x7f55e8136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806b250 0x7f55e8136840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009840 tx=0x7f55dc009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ee8fd700 1 -- 172.21.15.32:0/2369834157 shutdown_connections 2023-09-30T22:24:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/2369834157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806b250 0x7f55e8136840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.665+0000 7f55ee8fd700 1 -- 172.21.15.32:0/2369834157 >> 172.21.15.32:0/2369834157 conn(0x7f55e8066f50 msgr2=0x7f55e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.666+0000 7f55ee8fd700 1 -- 172.21.15.32:0/2369834157 shutdown_connections 2023-09-30T22:24:00.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.666+0000 7f55ee8fd700 1 -- 172.21.15.32:0/2369834157 wait complete. 2023-09-30T22:24:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.666+0000 7f55ee8fd700 1 Processor -- start 2023-09-30T22:24:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 -- start start 2023-09-30T22:24:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 0x7f55e81c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55e806b250 0x7f55e81c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e811c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e811fd00 con 0x7f55e806a770 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e811f670 con 0x7f55e806b250 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ee8fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55e811e010 con 0x7f55e8136d80 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 0x7f55e81c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 0x7f55e81c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45684/0 (socket says 172.21.15.32:45684) 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55e7fff700 1 -- 172.21.15.32:0/1559843369 learned_addr learned my addr 172.21.15.32:0/1559843369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55ece9a700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e811c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.667+0000 7f55e77fe700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55e806b250 0x7f55e81c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:00.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e7fff700 1 -- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 msgr2=0x7f55e811c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e7fff700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e811c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e7fff700 1 -- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55e806b250 msgr2=0x7f55e81c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e7fff700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55e806b250 0x7f55e81c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e7fff700 1 -- 172.21.15.32:0/1559843369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e0009b30 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e7fff700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 0x7f55e81c6150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c520 tx=0x7f55d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8004c80 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8004e00 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8007310 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc0094a0 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.668+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e8071290 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.669+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f55d8007470 con 0x7f55e806a770 2023-09-30T22:24:00.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.669+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e8071e60 con 0x7f55e806a770 2023-09-30T22:24:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.669+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f55d803f7c0 con 0x7f55e806a770 2023-09-30T22:24:00.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.674+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55d80123b0 con 0x7f55e806a770 2023-09-30T22:24:00.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.925+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55e8074ea0 con 0x7f55e806a770 2023-09-30T22:24:00.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.926+0000 7f55e4ff9700 1 -- 172.21.15.32:0/1559843369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2315 (secure 0 0 0) 0x7f55d80102c0 con 0x7f55e806a770 2023-09-30T22:24:00.927 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:00.927 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:58.784702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:24:00.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 >> v1:172.21.15.32:6816/126553 conn(0x7f55d402f510 legacy=0x7f55d40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 msgr2=0x7f55e81c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:00.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 0x7f55e81c6150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c520 tx=0x7f55d800c830 comp rx=0 tx=0).stop 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 shutdown_connections 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55e8136d80 0x7f55e811c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55e806a770 0x7f55e81c6150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 --2- 172.21.15.32:0/1559843369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55e806b250 0x7f55e81c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 >> 172.21.15.32:0/1559843369 conn(0x7f55e8066f50 msgr2=0x7f55e806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 shutdown_connections 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:00.928+0000 7f55ee8fd700 1 -- 172.21.15.32:0/1559843369 wait complete. 2023-09-30T22:24:00.930 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-09-30T22:24:00.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 Processor -- start 2023-09-30T22:24:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 -- start start 2023-09-30T22:24:01.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe398138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe398139140 0x7fe39813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39806bc00 con 0x7fe398133a80 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39806c570 con 0x7fe398139140 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.080+0000 7fe39da64700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980a0040 con 0x7fe398136820 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe397fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe398139140 0x7fe39813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe398138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe3977fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe398133a80 0x7fe398133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42834/0 (socket says 172.21.15.32:42834) 2023-09-30T22:24:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe3977fe700 1 -- 172.21.15.32:0/1289273529 learned_addr learned my addr 172.21.15.32:0/1289273529 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe396ffd700 1 -- 172.21.15.32:0/1289273529 >> v2:172.21.15.32:3300/0 conn(0x7fe398139140 msgr2=0x7fe39813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe396ffd700 1 --2- 172.21.15.32:0/1289273529 >> v2:172.21.15.32:3300/0 conn(0x7fe398139140 0x7fe39813b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe396ffd700 1 -- 172.21.15.32:0/1289273529 >> v1:172.21.15.32:6789/0 conn(0x7fe398133a80 legacy=0x7fe398133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe396ffd700 1 -- 172.21.15.32:0/1289273529 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398133860 con 0x7fe398136820 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe397fff700 1 --2- 172.21.15.32:0/1289273529 >> v2:172.21.15.32:3300/0 conn(0x7fe398139140 0x7fe39813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe3967fc700 1 -- 172.21.15.32:0/1289273529 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776015936 0 0) 0x7fe39806bc00 con 0x7fe398133a80 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe396ffd700 1 --2- 172.21.15.32:0/1289273529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe398138c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe388009840 tx=0x7fe388009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991a2ca4266b7da4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:01.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe3967fc700 1 -- 172.21.15.32:0/1289273529 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388011830 con 0x7fe398136820 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe3967fc700 1 -- 172.21.15.32:0/1289273529 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388011990 con 0x7fe398136820 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.081+0000 7fe3967fc700 1 -- 172.21.15.32:0/1289273529 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388011d20 con 0x7fe398136820 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 -- 172.21.15.32:0/1289273529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 msgr2=0x7fe398138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 --2- 172.21.15.32:0/1289273529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe398138c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe388009840 tx=0x7fe388009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 -- 172.21.15.32:0/1289273529 shutdown_connections 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 --2- 172.21.15.32:0/1289273529 >> v2:172.21.15.32:3300/0 conn(0x7fe398139140 0x7fe39813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 --2- 172.21.15.32:0/1289273529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe398138c00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 -- 172.21.15.32:0/1289273529 >> 172.21.15.32:0/1289273529 conn(0x7fe3980a4210 msgr2=0x7fe3980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 -- 172.21.15.32:0/1289273529 shutdown_connections 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.082+0000 7fe39da64700 1 -- 172.21.15.32:0/1289273529 wait complete. 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.083+0000 7fe39da64700 1 Processor -- start 2023-09-30T22:24:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.083+0000 7fe39da64700 1 -- start start 2023-09-30T22:24:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe39da64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe398133a80 0x7fe39806f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe39da64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe39806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe39da64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe398139140 0x7fe3980703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe39da64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398072380 con 0x7fe398139140 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe39da64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39809bbf0 con 0x7fe398133a80 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe39da64700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe398072560 con 0x7fe398136820 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe398133a80 0x7fe39806f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe39806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe39806fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55756/0 (socket says 172.21.15.32:55756) 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 -- 172.21.15.32:0/4173618764 learned_addr learned my addr 172.21.15.32:0/4173618764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe398133a80 0x7fe39806f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39098/0 (socket says 172.21.15.32:39098) 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe397fff700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe398139140 0x7fe3980703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 -- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe398133a80 msgr2=0x7fe39806f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe398133a80 0x7fe39806f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 -- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe398139140 msgr2=0x7fe3980703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe398139140 0x7fe3980703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 -- 172.21.15.32:0/4173618764 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390009a70 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.084+0000 7fe396ffd700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe39806fe90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe3880090b0 tx=0x7fe388004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.085+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880045d0 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.085+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38801f070 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.085+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3880094a0 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.085+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880038c0 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.085+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39809e3a0 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.085+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe388003a20 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.086+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fe388021070 con 0x7fe398136820 2023-09-30T22:24:01.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.086+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398074ea0 con 0x7fe398136820 2023-09-30T22:24:01.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.091+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe38802b080 con 0x7fe398136820 2023-09-30T22:24:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.299+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe398130aa0 con 0x7fe398136820 2023-09-30T22:24:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.300+0000 7fe38ffff700 1 -- 172.21.15.32:0/4173618764 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12864 (secure 0 0 0) 0x7fe388015020 con 0x7fe398136820 2023-09-30T22:24:01.301 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:01.302 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:23:56.022608+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 >> v1:172.21.15.32:6816/126553 conn(0x7fe37802f510 legacy=0x7fe3780319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 msgr2=0x7fe39806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe39806fe90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe3880090b0 tx=0x7fe388004750 comp rx=0 tx=0).stop 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 shutdown_connections 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe398139140 0x7fe3980703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe398133a80 0x7fe39806f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 --2- 172.21.15.32:0/4173618764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe398136820 0x7fe39806fe90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 >> 172.21.15.32:0/4173618764 conn(0x7fe3980a4210 msgr2=0x7fe398134600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:01.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 shutdown_connections 2023-09-30T22:24:01.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.303+0000 7fe39da64700 1 -- 172.21.15.32:0/4173618764 wait complete. 2023-09-30T22:24:01.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:24:01.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:24:01.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.455+0000 7f6457ea6700 1 Processor -- start 2023-09-30T22:24:01.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6457ea6700 1 -- start start 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6457ea6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64501303c0 0x7f6450130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6457ea6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006a770 0x7f645006ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6457ea6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645009f620 con 0x7f645006b0c0 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6457ea6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6450130250 con 0x7f64501303c0 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6457ea6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645009e1b0 con 0x7f645006a770 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6455c42700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64501303c0 0x7f6450130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6455441700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006a770 0x7f645006ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6456443700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f645006b0c0 0x7f645013a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42840/0 (socket says 172.21.15.32:42840) 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.456+0000 7f6456443700 1 -- 172.21.15.32:0/4175238347 learned_addr learned my addr 172.21.15.32:0/4175238347 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6455441700 1 -- 172.21.15.32:0/4175238347 >> v2:172.21.15.32:3300/0 conn(0x7f64501303c0 msgr2=0x7f6450130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6455441700 1 --2- 172.21.15.32:0/4175238347 >> v2:172.21.15.32:3300/0 conn(0x7f64501303c0 0x7f6450130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6455441700 1 -- 172.21.15.32:0/4175238347 >> v1:172.21.15.32:6789/0 conn(0x7f645006b0c0 legacy=0x7f645013a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6455441700 1 -- 172.21.15.32:0/4175238347 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645006b520 con 0x7f645006a770 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6455c42700 1 --2- 172.21.15.32:0/4175238347 >> v2:172.21.15.32:3300/0 conn(0x7f64501303c0 0x7f6450130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6455441700 1 --2- 172.21.15.32:0/4175238347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006a770 0x7f645006ab80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6444009840 tx=0x7f6444009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa46c92791692ec6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6454c40700 1 -- 172.21.15.32:0/4175238347 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444011830 con 0x7f645006a770 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6454c40700 1 -- 172.21.15.32:0/4175238347 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444011990 con 0x7f645006a770 2023-09-30T22:24:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.457+0000 7f6454c40700 1 -- 172.21.15.32:0/4175238347 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444011d20 con 0x7f645006a770 2023-09-30T22:24:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 -- 172.21.15.32:0/4175238347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006a770 msgr2=0x7f645006ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 --2- 172.21.15.32:0/4175238347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006a770 0x7f645006ab80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6444009840 tx=0x7f6444009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 -- 172.21.15.32:0/4175238347 shutdown_connections 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 --2- 172.21.15.32:0/4175238347 >> v2:172.21.15.32:3300/0 conn(0x7f64501303c0 0x7f6450130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 --2- 172.21.15.32:0/4175238347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006a770 0x7f645006ab80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 -- 172.21.15.32:0/4175238347 >> 172.21.15.32:0/4175238347 conn(0x7f6450066d70 msgr2=0x7f64500a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 -- 172.21.15.32:0/4175238347 shutdown_connections 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.458+0000 7f6457ea6700 1 -- 172.21.15.32:0/4175238347 wait complete. 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 Processor -- start 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 -- start start 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645006a770 0x7f645011d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 0x7f645011de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64501303c0 0x7f645011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6450120360 con 0x7f64501303c0 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645011fcc0 con 0x7f645006b0c0 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6457ea6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f645012d260 con 0x7f645006a770 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6455441700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 0x7f645011de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6455c42700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645006a770 0x7f645011d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6456443700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64501303c0 0x7f645011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6455441700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 0x7f645011de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39126/0 (socket says 172.21.15.32:39126) 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.459+0000 7f6455441700 1 -- 172.21.15.32:0/3014147181 learned_addr learned my addr 172.21.15.32:0/3014147181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6455441700 1 -- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645006a770 msgr2=0x7f645011d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6455441700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645006a770 0x7f645011d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6455441700 1 -- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64501303c0 msgr2=0x7f645011e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6455441700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64501303c0 0x7f645011e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6455441700 1 -- 172.21.15.32:0/3014147181 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0094a0 con 0x7f645006b0c0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6455441700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 0x7f645011de70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6444009fd0 tx=0x7f6444004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440036a0 con 0x7f645006b0c0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644401f070 con 0x7f645006b0c0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64440094a0 con 0x7f645006b0c0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444003a60 con 0x7f645006b0c0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.460+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450130dc0 con 0x7f645006b0c0 2023-09-30T22:24:01.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.461+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6444003bc0 con 0x7f645006b0c0 2023-09-30T22:24:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.461+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f6444021070 con 0x7f645006b0c0 2023-09-30T22:24:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.461+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645004f2a0 con 0x7f645006b0c0 2023-09-30T22:24:01.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.466+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6444015070 con 0x7f645006b0c0 2023-09-30T22:24:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.668+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6450120110 con 0x7f645006b0c0 2023-09-30T22:24:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.669+0000 7f64427fc700 1 -- 172.21.15.32:0/3014147181 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f644401a430 con 0x7f645006b0c0 2023-09-30T22:24:01.670 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:24:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 >> v1:172.21.15.32:6816/126553 conn(0x7f643402f510 legacy=0x7f64340319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 msgr2=0x7f645011de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 0x7f645011de70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6444009fd0 tx=0x7f6444004750 comp rx=0 tx=0).stop 2023-09-30T22:24:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 shutdown_connections 2023-09-30T22:24:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64501303c0 0x7f645011e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f645006a770 0x7f645011d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 --2- 172.21.15.32:0/3014147181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f645006b0c0 0x7f645011de70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 >> 172.21.15.32:0/3014147181 conn(0x7f6450066d70 msgr2=0x7f645009b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:01.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 shutdown_connections 2023-09-30T22:24:01.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.671+0000 7f6457ea6700 1 -- 172.21.15.32:0/3014147181 wait complete. 2023-09-30T22:24:01.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:24:01.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.823+0000 7fa6dbc87700 1 Processor -- start 2023-09-30T22:24:01.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6dbc87700 1 -- start start 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6dbc87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d41303c0 0x7fa6d4130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6dbc87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6d406a770 0x7fa6d406ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6dbc87700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d409f620 con 0x7fa6d406b0c0 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6dbc87700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d4130280 con 0x7fa6d406a770 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6dbc87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d409e1b0 con 0x7fa6d41303c0 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6d9222700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6d406a770 0x7fa6d406ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6da224700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6d406b0c0 0x7fa6d413a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42854/0 (socket says 172.21.15.32:42854) 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6da224700 1 -- 172.21.15.32:0/269826745 learned_addr learned my addr 172.21.15.32:0/269826745 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:01.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.824+0000 7fa6d9a23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d41303c0 0x7fa6d4130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d9222700 1 -- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d41303c0 msgr2=0x7fa6d4130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d9222700 1 --2- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d41303c0 0x7fa6d4130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d9222700 1 -- 172.21.15.32:0/269826745 >> v1:172.21.15.32:6789/0 conn(0x7fa6d406b0c0 legacy=0x7fa6d413a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d8a21700 1 -- 172.21.15.32:0/269826745 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176625233 0 0) 0x7fa6d409f620 con 0x7fa6d406b0c0 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d9222700 1 -- 172.21.15.32:0/269826745 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d406b540 con 0x7fa6d406a770 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d9a23700 1 --2- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d41303c0 0x7fa6d4130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d9222700 1 --2- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406a770 0x7fa6d406ab80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009840 tx=0x7fa6c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4148ded62919356c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d8a21700 1 -- 172.21.15.32:0/269826745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8011830 con 0x7fa6d406a770 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d8a21700 1 -- 172.21.15.32:0/269826745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8011990 con 0x7fa6d406a770 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.825+0000 7fa6d8a21700 1 -- 172.21.15.32:0/269826745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8011d20 con 0x7fa6d406a770 2023-09-30T22:24:01.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 -- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406a770 msgr2=0x7fa6d406ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406a770 0x7fa6d406ab80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009840 tx=0x7fa6c8009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 -- 172.21.15.32:0/269826745 shutdown_connections 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d41303c0 0x7fa6d4130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/269826745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406a770 0x7fa6d406ab80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 -- 172.21.15.32:0/269826745 >> 172.21.15.32:0/269826745 conn(0x7fa6d4066d70 msgr2=0x7fa6d40a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 -- 172.21.15.32:0/269826745 shutdown_connections 2023-09-30T22:24:01.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.826+0000 7fa6dbc87700 1 -- 172.21.15.32:0/269826745 wait complete. 2023-09-30T22:24:01.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.827+0000 7fa6dbc87700 1 Processor -- start 2023-09-30T22:24:01.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.827+0000 7fa6dbc87700 1 -- start start 2023-09-30T22:24:01.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6dbc87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 0x7fa6d411d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6dbc87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406b0c0 0x7fa6d411de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6dbc87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6d41303c0 0x7fa6d411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6dbc87700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d411fd40 con 0x7fa6d406b0c0 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6dbc87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4071860 con 0x7fa6d406a770 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6dbc87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4071fe0 con 0x7fa6d41303c0 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 0x7fa6d411d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 0x7fa6d411d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39132/0 (socket says 172.21.15.32:39132) 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 -- 172.21.15.32:0/2821001 learned_addr learned my addr 172.21.15.32:0/2821001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6da224700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6d41303c0 0x7fa6d411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9222700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406b0c0 0x7fa6d411de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 -- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6d41303c0 msgr2=0x7fa6d411e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6d41303c0 0x7fa6d411e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 -- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406b0c0 msgr2=0x7fa6d411de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406b0c0 0x7fa6d411de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 -- 172.21.15.32:0/2821001 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d00094a0 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9222700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406b0c0 0x7fa6d411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.828+0000 7fa6d9a23700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 0x7fa6d411d930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000ec10 tx=0x7fa6d000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.829+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000c910 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.829+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d000ca70 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.829+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c80094a0 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.829+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000ce00 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.829+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d411eda0 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.829+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa6d0013470 con 0x7fa6d406a770 2023-09-30T22:24:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.830+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d412da40 con 0x7fa6d406a770 2023-09-30T22:24:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.830+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fa6d00400d0 con 0x7fa6d406a770 2023-09-30T22:24:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:01.835+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6d0017020 con 0x7fa6d406a770 2023-09-30T22:24:02.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.038+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6d411f1d0 con 0x7fa6d406a770 2023-09-30T22:24:02.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.038+0000 7fa6c67fc700 1 -- 172.21.15.32:0/2821001 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa6d001b3b0 con 0x7fa6d406a770 2023-09-30T22:24:02.039 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:24:02.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.040+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 >> v1:172.21.15.32:6816/126553 conn(0x7fa6b802f510 legacy=0x7fa6b80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.040+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 msgr2=0x7fa6d411d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.040+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 0x7fa6d411d930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000ec10 tx=0x7fa6d000ef20 comp rx=0 tx=0).stop 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 shutdown_connections 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6d41303c0 0x7fa6d411e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6d406a770 0x7fa6d411d930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 --2- 172.21.15.32:0/2821001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6d406b0c0 0x7fa6d411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 >> 172.21.15.32:0/2821001 conn(0x7fa6d4066d70 msgr2=0x7fa6d40a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 shutdown_connections 2023-09-30T22:24:02.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.041+0000 7fa6dbc87700 1 -- 172.21.15.32:0/2821001 wait complete. 2023-09-30T22:24:02.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:24:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.193+0000 7f01ad8ca700 1 Processor -- start 2023-09-30T22:24:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.193+0000 7f01ad8ca700 1 -- start start 2023-09-30T22:24:02.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.193+0000 7f01ad8ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a809f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.193+0000 7f01ad8ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a8133930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01ad8ca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a809d6c0 con 0x7f01a809fb10 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01ad8ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80a0a30 con 0x7f01a809edd0 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01ad8ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01a809dc50 con 0x7f01a809f6e0 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a8133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a809f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01a809fb10 0x7f01a8133e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42866/0 (socket says 172.21.15.32:42866) 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a77fe700 1 -- 172.21.15.32:0/2342971863 learned_addr learned my addr 172.21.15.32:0/2342971863 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a6ffd700 1 -- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 msgr2=0x7f01a8133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:02.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a6ffd700 1 --2- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a8133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a6ffd700 1 -- 172.21.15.32:0/2342971863 >> v1:172.21.15.32:6789/0 conn(0x7f01a809fb10 legacy=0x7f01a8133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a5ffb700 1 -- 172.21.15.32:0/2342971863 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601419623 0 0) 0x7f01a809d6c0 con 0x7f01a809fb10 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a6ffd700 1 -- 172.21.15.32:0/2342971863 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a809df10 con 0x7f01a809edd0 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a67fc700 1 --2- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a8133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a6ffd700 1 --2- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a809f1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0198009850 tx=0x7f0198009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2453554ca4bd762c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a5ffb700 1 -- 172.21.15.32:0/2342971863 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198005e30 con 0x7f01a809edd0 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a5ffb700 1 -- 172.21.15.32:0/2342971863 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0198006030 con 0x7f01a809edd0 2023-09-30T22:24:02.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.194+0000 7f01a5ffb700 1 -- 172.21.15.32:0/2342971863 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980063e0 con 0x7f01a809edd0 2023-09-30T22:24:02.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 -- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 msgr2=0x7f01a809f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a809f1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0198009850 tx=0x7f0198009b60 comp rx=0 tx=0).stop 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 -- 172.21.15.32:0/2342971863 shutdown_connections 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a809f1a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/2342971863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a8133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 -- 172.21.15.32:0/2342971863 >> 172.21.15.32:0/2342971863 conn(0x7f01a80a3050 msgr2=0x7f01a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 -- 172.21.15.32:0/2342971863 shutdown_connections 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.195+0000 7f01ad8ca700 1 -- 172.21.15.32:0/2342971863 wait complete. 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.196+0000 7f01ad8ca700 1 Processor -- start 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.196+0000 7f01ad8ca700 1 -- start start 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01ad8ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a8122290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01ad8ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a81227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01ad8ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a809fb10 0x7f01a8122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01ad8ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8125ae0 con 0x7f01a809fb10 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01ad8ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81335d0 con 0x7f01a809edd0 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01ad8ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8124ca0 con 0x7f01a809f6e0 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a81227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a809fb10 0x7f01a8122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a8122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a81227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55782/0 (socket says 172.21.15.32:55782) 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a67fc700 1 -- 172.21.15.32:0/4085330623 learned_addr learned my addr 172.21.15.32:0/4085330623 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 -- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 msgr2=0x7f01a81227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a81227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 -- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a809fb10 msgr2=0x7f01a8122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a809fb10 0x7f01a8122d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 -- 172.21.15.32:0/4085330623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900094a0 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a77fe700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a809fb10 0x7f01a8122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.197+0000 7f01a6ffd700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a8122290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0198009850 tx=0x7f0198004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.198+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198006790 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.198+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0198020070 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.198+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0198009430 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.198+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198025670 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.198+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80a0330 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.198+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0198022030 con 0x7f01a809edd0 2023-09-30T22:24:02.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.199+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f0198041010 con 0x7f01a809edd0 2023-09-30T22:24:02.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.199+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a804f2a0 con 0x7f01a809edd0 2023-09-30T22:24:02.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.204+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f019801a390 con 0x7f01a809edd0 2023-09-30T22:24:02.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.405+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01a8072900 con 0x7f01a809edd0 2023-09-30T22:24:02.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.406+0000 7f019ffff700 1 -- 172.21.15.32:0/4085330623 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0198024bc0 con 0x7f01a809edd0 2023-09-30T22:24:02.407 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 >> v1:172.21.15.32:6816/126553 conn(0x7f018802f540 legacy=0x7f01880319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 msgr2=0x7f01a8122290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a8122290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0198009850 tx=0x7f0198004750 comp rx=0 tx=0).stop 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 shutdown_connections 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01a809fb10 0x7f01a8122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01a809edd0 0x7f01a8122290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 --2- 172.21.15.32:0/4085330623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f01a809f6e0 0x7f01a81227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 >> 172.21.15.32:0/4085330623 conn(0x7f01a80a3050 msgr2=0x7f01a8133e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:02.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 shutdown_connections 2023-09-30T22:24:02.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.408+0000 7f01ad8ca700 1 -- 172.21.15.32:0/4085330623 wait complete. 2023-09-30T22:24:02.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:24:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 Processor -- start 2023-09-30T22:24:02.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 -- start start 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8201303c0 0x7fe820130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82013b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82009f620 con 0x7fe82006a770 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820130280 con 0x7fe8201303c0 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe825d98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe82009e1b0 con 0x7fe82006b290 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe81f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8201303c0 0x7fe820130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe81ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82013b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe81effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe82006a770 0x7fe82006ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42880/0 (socket says 172.21.15.32:42880) 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.561+0000 7fe81effd700 1 -- 172.21.15.32:0/997672380 learned_addr learned my addr 172.21.15.32:0/997672380 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81ffff700 1 -- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8201303c0 msgr2=0x7fe820130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81ffff700 1 --2- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8201303c0 0x7fe820130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81ffff700 1 -- 172.21.15.32:0/997672380 >> v1:172.21.15.32:6789/0 conn(0x7fe82006a770 legacy=0x7fe82006ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81ffff700 1 -- 172.21.15.32:0/997672380 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820130cd0 con 0x7fe82006b290 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81e7fc700 1 -- 172.21.15.32:0/997672380 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663378934 0 0) 0x7fe82009f620 con 0x7fe82006a770 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81f7fe700 1 --2- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8201303c0 0x7fe820130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81ffff700 1 --2- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82013b7b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe818009dc0 tx=0x7fe81800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348b2142623d5deb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81e7fc700 1 -- 172.21.15.32:0/997672380 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800e070 con 0x7fe82006b290 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81e7fc700 1 -- 172.21.15.32:0/997672380 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81800bb20 con 0x7fe82006b290 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.562+0000 7fe81e7fc700 1 -- 172.21.15.32:0/997672380 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818012430 con 0x7fe82006b290 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 -- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 msgr2=0x7fe82013b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 --2- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82013b7b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe818009dc0 tx=0x7fe81800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 -- 172.21.15.32:0/997672380 shutdown_connections 2023-09-30T22:24:02.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 --2- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82013b7b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 --2- 172.21.15.32:0/997672380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8201303c0 0x7fe820130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 -- 172.21.15.32:0/997672380 >> 172.21.15.32:0/997672380 conn(0x7fe820066d70 msgr2=0x7fe8200a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:02.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 -- 172.21.15.32:0/997672380 shutdown_connections 2023-09-30T22:24:02.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.563+0000 7fe825d98700 1 -- 172.21.15.32:0/997672380 wait complete. 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 Processor -- start 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 -- start start 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 0x7fe82011d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82011de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8201303c0 0x7fe82011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820072260 con 0x7fe8201303c0 2023-09-30T22:24:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820072070 con 0x7fe82006a770 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.564+0000 7fe825d98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe82011efc0 con 0x7fe82006b290 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 0x7fe82011d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 0x7fe82011d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39162/0 (socket says 172.21.15.32:39162) 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82011de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 -- 172.21.15.32:0/3487030230 learned_addr learned my addr 172.21.15.32:0/3487030230 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81ffff700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8201303c0 0x7fe82011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 -- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 msgr2=0x7fe82011de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82011de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 -- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8201303c0 msgr2=0x7fe82011e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8201303c0 0x7fe82011e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 -- 172.21.15.32:0/3487030230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100094a0 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe81f7fe700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 0x7fe82011d930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ec10 tx=0x7fe81000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000c850 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81000c9b0 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe818009a70 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000cd20 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.565+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8200a0240 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.566+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe810013470 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.566+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82004f2a0 con 0x7fe82006a770 2023-09-30T22:24:02.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.567+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fe810040130 con 0x7fe82006a770 2023-09-30T22:24:02.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.571+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe810013720 con 0x7fe82006a770 2023-09-30T22:24:02.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.774+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe82006c380 con 0x7fe82006a770 2023-09-30T22:24:02.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.774+0000 7fe807fff700 1 -- 172.21.15.32:0/3487030230 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe810021070 con 0x7fe82006a770 2023-09-30T22:24:02.775 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:24:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 >> v1:172.21.15.32:6816/126553 conn(0x7fe80c02f510 legacy=0x7fe80c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 msgr2=0x7fe82011d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 0x7fe82011d930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ec10 tx=0x7fe81000ef20 comp rx=0 tx=0).stop 2023-09-30T22:24:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 shutdown_connections 2023-09-30T22:24:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8201303c0 0x7fe82011e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe82006a770 0x7fe82011d930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 --2- 172.21.15.32:0/3487030230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe82006b290 0x7fe82011de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 >> 172.21.15.32:0/3487030230 conn(0x7fe820066d70 msgr2=0x7fe82009ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 shutdown_connections 2023-09-30T22:24:02.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.777+0000 7fe825d98700 1 -- 172.21.15.32:0/3487030230 wait complete. 2023-09-30T22:24:02.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:24:02.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 Processor -- start 2023-09-30T22:24:02.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 -- start start 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 0x7fd9281337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 0x7fd9281304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92806b990 con 0x7fd928134260 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280a0590 con 0x7fd928133420 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92f138700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd92809fb80 con 0x7fd928133e30 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92d6d5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 0x7fd9281304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92ced4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 0x7fd9281337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92d6d5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 0x7fd9281304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45774/0 (socket says 172.21.15.32:45774) 2023-09-30T22:24:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92ced4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 0x7fd9281337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39170/0 (socket says 172.21.15.32:39170) 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd92d6d5700 1 -- 172.21.15.32:0/2256985582 learned_addr learned my addr 172.21.15.32:0/2256985582 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.927+0000 7fd927fff700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92ced4700 1 -- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 msgr2=0x7fd92812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92ced4700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92ced4700 1 -- 172.21.15.32:0/2256985582 >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 msgr2=0x7fd9281304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92ced4700 1 --2- 172.21.15.32:0/2256985582 >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 0x7fd9281304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92ced4700 1 -- 172.21.15.32:0/2256985582 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280a0030 con 0x7fd928133420 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92d6d5700 1 --2- 172.21.15.32:0/2256985582 >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 0x7fd9281304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd927fff700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd92ced4700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 0x7fd9281337f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd918009840 tx=0x7fd918009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=230c3435603edd91 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd9277fe700 1 -- 172.21.15.32:0/2256985582 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918011830 con 0x7fd928133420 2023-09-30T22:24:02.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd9277fe700 1 -- 172.21.15.32:0/2256985582 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918011990 con 0x7fd928133420 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.928+0000 7fd9277fe700 1 -- 172.21.15.32:0/2256985582 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918011d20 con 0x7fd928133420 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 -- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 msgr2=0x7fd9281337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 0x7fd9281337f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd918009840 tx=0x7fd918009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 -- 172.21.15.32:0/2256985582 shutdown_connections 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928133420 0x7fd9281337f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 --2- 172.21.15.32:0/2256985582 >> v2:172.21.15.32:3300/0 conn(0x7fd928134260 0x7fd9281304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 --2- 172.21.15.32:0/2256985582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 -- 172.21.15.32:0/2256985582 >> 172.21.15.32:0/2256985582 conn(0x7fd9280a4210 msgr2=0x7fd9280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 -- 172.21.15.32:0/2256985582 shutdown_connections 2023-09-30T22:24:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.929+0000 7fd92f138700 1 -- 172.21.15.32:0/2256985582 wait complete. 2023-09-30T22:24:02.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 Processor -- start 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 -- start start 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 0x7fd92809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928134260 0x7fd92809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92811ff70 con 0x7fd928133420 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92809dd60 con 0x7fd928134260 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.930+0000 7fd92f138700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd928131400 con 0x7fd928133e30 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 0x7fd92809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 0x7fd92809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45780/0 (socket says 172.21.15.32:45780) 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd927fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 -- 172.21.15.32:0/720915522 learned_addr learned my addr 172.21.15.32:0/720915522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:02.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92d6d5700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928134260 0x7fd92809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 -- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 msgr2=0x7fd92809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 -- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928134260 msgr2=0x7fd92809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928134260 0x7fd92809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 -- 172.21.15.32:0/720915522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100094a0 con 0x7fd928133420 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92d6d5700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928134260 0x7fd92809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92ced4700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 0x7fd92809c480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9180090b0 tx=0x7fd918004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918004240 con 0x7fd928133420 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91801f070 con 0x7fd928133420 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180036a0 con 0x7fd928133420 2023-09-30T22:24:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd920009b30 con 0x7fd928133420 2023-09-30T22:24:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.931+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9180094a0 con 0x7fd928133420 2023-09-30T22:24:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.932+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd918021030 con 0x7fd928133420 2023-09-30T22:24:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.932+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fd918040a00 con 0x7fd928133420 2023-09-30T22:24:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.932+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92811f460 con 0x7fd928133420 2023-09-30T22:24:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:02.937+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9180239c0 con 0x7fd928133420 2023-09-30T22:24:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.189+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd92811fdb0 con 0x7fd928133420 2023-09-30T22:24:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.190+0000 7fd9257fa700 1 -- 172.21.15.32:0/720915522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2315 (secure 0 0 0) 0x7fd9180233d0 con 0x7fd928133420 2023-09-30T22:24:03.191 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:03.191 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:58.784702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 >> v1:172.21.15.32:6816/126553 conn(0x7fd90802f510 legacy=0x7fd9080319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 msgr2=0x7fd92809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 0x7fd92809c480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9180090b0 tx=0x7fd918004750 comp rx=0 tx=0).stop 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 shutdown_connections 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd928134260 0x7fd92809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd928133420 0x7fd92809c480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 --2- 172.21.15.32:0/720915522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd928133e30 0x7fd92809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 >> 172.21.15.32:0/720915522 conn(0x7fd9280a4210 msgr2=0x7fd9280a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 shutdown_connections 2023-09-30T22:24:03.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.192+0000 7fd92f138700 1 -- 172.21.15.32:0/720915522 wait complete. 2023-09-30T22:24:03.194 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-09-30T22:24:03.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:03.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.341+0000 7fa0cd0ab700 1 Processor -- start 2023-09-30T22:24:03.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.341+0000 7fa0cd0ab700 1 -- start start 2023-09-30T22:24:03.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0cd0ab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0c809d6f0 0x7fa0c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0cd0ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 0x7fa0c8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0cd0ab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c809f130 con 0x7fa0c809e000 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0cd0ab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c809f520 con 0x7fa0c809d6f0 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0cd0ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c809e930 con 0x7fa0c809e410 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0c759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 0x7fa0c8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0c659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0c809e000 0x7fa0c8130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42894/0 (socket says 172.21.15.32:42894) 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0c759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 0x7fa0c8130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39198/0 (socket says 172.21.15.32:39198) 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0c659c700 1 -- 172.21.15.32:0/366467005 learned_addr learned my addr 172.21.15.32:0/366467005 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.342+0000 7fa0c6d9d700 1 --2- 172.21.15.32:0/366467005 >> v2:172.21.15.32:3300/0 conn(0x7fa0c809d6f0 0x7fa0c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c759e700 1 -- 172.21.15.32:0/366467005 >> v2:172.21.15.32:3300/0 conn(0x7fa0c809d6f0 msgr2=0x7fa0c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:03.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c759e700 1 --2- 172.21.15.32:0/366467005 >> v2:172.21.15.32:3300/0 conn(0x7fa0c809d6f0 0x7fa0c809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c759e700 1 -- 172.21.15.32:0/366467005 >> v1:172.21.15.32:6789/0 conn(0x7fa0c809e000 legacy=0x7fa0c8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c759e700 1 -- 172.21.15.32:0/366467005 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c809b2d0 con 0x7fa0c809e410 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c6d9d700 1 --2- 172.21.15.32:0/366467005 >> v2:172.21.15.32:3300/0 conn(0x7fa0c809d6f0 0x7fa0c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c759e700 1 --2- 172.21.15.32:0/366467005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 0x7fa0c8130d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009d90 tx=0x7fa0c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d98d1e1819346a94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c5d9b700 1 -- 172.21.15.32:0/366467005 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000e070 con 0x7fa0c809e410 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c5d9b700 1 -- 172.21.15.32:0/366467005 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c000bb20 con 0x7fa0c809e410 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.343+0000 7fa0c5d9b700 1 -- 172.21.15.32:0/366467005 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0012430 con 0x7fa0c809e410 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/366467005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 msgr2=0x7fa0c8130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/366467005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 0x7fa0c8130d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009d90 tx=0x7fa0c000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/366467005 shutdown_connections 2023-09-30T22:24:03.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/366467005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e410 0x7fa0c8130d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/366467005 >> v2:172.21.15.32:3300/0 conn(0x7fa0c809d6f0 0x7fa0c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/366467005 >> 172.21.15.32:0/366467005 conn(0x7fa0c8066f50 msgr2=0x7fa0c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/366467005 shutdown_connections 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.344+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/366467005 wait complete. 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.345+0000 7fa0cd0ab700 1 Processor -- start 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.345+0000 7fa0cd0ab700 1 -- start start 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0cd0ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 0x7fa0c8122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0cd0ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 0x7fa0c8122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0cd0ab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0c809e410 0x7fa0c8122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0cd0ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8125a70 con 0x7fa0c809d6f0 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0cd0ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8123f50 con 0x7fa0c809e000 2023-09-30T22:24:03.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0cd0ab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c809c700 con 0x7fa0c809e410 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 0x7fa0c8122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 0x7fa0c8122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 0x7fa0c8122230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45800/0 (socket says 172.21.15.32:45800) 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 -- 172.21.15.32:0/3883508022 learned_addr learned my addr 172.21.15.32:0/3883508022 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 0x7fa0c8122770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39200/0 (socket says 172.21.15.32:39200) 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c759e700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0c809e410 0x7fa0c8122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 -- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0c809e410 msgr2=0x7fa0c8122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0c809e410 0x7fa0c8122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 -- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 msgr2=0x7fa0c8122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 0x7fa0c8122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 -- 172.21.15.32:0/3883508022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80094a0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c759e700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0c809e410 0x7fa0c8122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c659c700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 0x7fa0c8122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.346+0000 7fa0c6d9d700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 0x7fa0c8122230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800ebe0 tx=0x7fa0b800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.347+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800c850 con 0x7fa0c809d6f0 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.347+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800c9b0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.347+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c0009a70 con 0x7fa0c809d6f0 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.347+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800cd40 con 0x7fa0c809d6f0 2023-09-30T22:24:03.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.347+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81264e0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.347+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa0b8013470 con 0x7fa0c809d6f0 2023-09-30T22:24:03.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.348+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fa0b803fcd0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.348+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c8074ea0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.353+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0b8017080 con 0x7fa0c809d6f0 2023-09-30T22:24:03.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.558+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c81256f0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.559+0000 7fa0af7fe700 1 -- 172.21.15.32:0/3883508022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+12864 (secure 0 0 0) 0x7fa0b800cea0 con 0x7fa0c809d6f0 2023-09-30T22:24:03.560 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:03.560 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:23:56.022608+0000","last_up_change":"2023-09-30T22:21:28.784040+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.561+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 >> v1:172.21.15.32:6816/126553 conn(0x7fa0b402f510 legacy=0x7fa0b40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.561+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 msgr2=0x7fa0c8122230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.561+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 0x7fa0c8122230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800ebe0 tx=0x7fa0b800eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 shutdown_connections 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0c809e410 0x7fa0c8122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c809d6f0 0x7fa0c8122230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 --2- 172.21.15.32:0/3883508022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0c809e000 0x7fa0c8122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 >> 172.21.15.32:0/3883508022 conn(0x7fa0c8066f50 msgr2=0x7fa0c8070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 shutdown_connections 2023-09-30T22:24:03.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.562+0000 7fa0cd0ab700 1 -- 172.21.15.32:0/3883508022 wait complete. 2023-09-30T22:24:03.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:24:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.711+0000 7fc4206c5700 1 Processor -- start 2023-09-30T22:24:03.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.711+0000 7fc4206c5700 1 -- start start 2023-09-30T22:24:03.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc4206c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc4206c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc418099a40 0x7fc418099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc4206c5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41809a850 con 0x7fc41809a3b0 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc4206c5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41809b580 con 0x7fc418099a40 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc4206c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4181313c0 con 0x7fc418131810 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41dc60700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc418099a40 0x7fc418099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41ec62700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc41809a3b0 0x7fc418138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42906/0 (socket says 172.21.15.32:42906) 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55830/0 (socket says 172.21.15.32:55830) 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41ec62700 1 -- 172.21.15.32:0/231962312 learned_addr learned my addr 172.21.15.32:0/231962312 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 -- 172.21.15.32:0/231962312 >> v2:172.21.15.32:3300/0 conn(0x7fc418099a40 msgr2=0x7fc418099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 --2- 172.21.15.32:0/231962312 >> v2:172.21.15.32:3300/0 conn(0x7fc418099a40 0x7fc418099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 -- 172.21.15.32:0/231962312 >> v1:172.21.15.32:6789/0 conn(0x7fc41809a3b0 legacy=0x7fc418138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 -- 172.21.15.32:0/231962312 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41806e8c0 con 0x7fc418131810 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41d45f700 1 -- 172.21.15.32:0/231962312 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030332675 0 0) 0x7fc41809a850 con 0x7fc41809a3b0 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41dc60700 1 --2- 172.21.15.32:0/231962312 >> v2:172.21.15.32:3300/0 conn(0x7fc418099a40 0x7fc418099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.712+0000 7fc41e461700 1 --2- 172.21.15.32:0/231962312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418131be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc414009870 tx=0x7fc414009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fea2f8850066269 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc41d45f700 1 -- 172.21.15.32:0/231962312 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414011830 con 0x7fc418131810 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc41d45f700 1 -- 172.21.15.32:0/231962312 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414011990 con 0x7fc418131810 2023-09-30T22:24:03.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc41d45f700 1 -- 172.21.15.32:0/231962312 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414011d20 con 0x7fc418131810 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 -- 172.21.15.32:0/231962312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 msgr2=0x7fc418131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 --2- 172.21.15.32:0/231962312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418131be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc414009870 tx=0x7fc414009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 -- 172.21.15.32:0/231962312 shutdown_connections 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 --2- 172.21.15.32:0/231962312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418131be0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 --2- 172.21.15.32:0/231962312 >> v2:172.21.15.32:3300/0 conn(0x7fc418099a40 0x7fc418099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 -- 172.21.15.32:0/231962312 >> 172.21.15.32:0/231962312 conn(0x7fc4180a2aa0 msgr2=0x7fc4180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 -- 172.21.15.32:0/231962312 shutdown_connections 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.713+0000 7fc4206c5700 1 -- 172.21.15.32:0/231962312 wait complete. 2023-09-30T22:24:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.714+0000 7fc4206c5700 1 Processor -- start 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.714+0000 7fc4206c5700 1 -- start start 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc4206c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc418099a40 0x7fc41806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc4206c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc41809a3b0 0x7fc41806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc4206c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc4206c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41806eb00 con 0x7fc41809a3b0 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc4206c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc418070a70 con 0x7fc418099a40 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc4206c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc418123550 con 0x7fc418131810 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41e461700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc418099a40 0x7fc41806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41ec62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41e461700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc418099a40 0x7fc41806f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39204/0 (socket says 172.21.15.32:39204) 2023-09-30T22:24:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41dc60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc41809a3b0 0x7fc41806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41e461700 1 -- 172.21.15.32:0/2066379409 learned_addr learned my addr 172.21.15.32:0/2066379409 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41dc60700 1 -- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 msgr2=0x7fc418070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41dc60700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41dc60700 1 -- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc418099a40 msgr2=0x7fc41806f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41dc60700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc418099a40 0x7fc41806f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.715+0000 7fc41dc60700 1 -- 172.21.15.32:0/2066379409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080094a0 con 0x7fc41809a3b0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.716+0000 7fc41dc60700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc41809a3b0 0x7fc41806fb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc40800e980 tx=0x7fc40800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.716+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800c690 con 0x7fc41809a3b0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.716+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800c7f0 con 0x7fc41809a3b0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.716+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140094a0 con 0x7fc41809a3b0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.716+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800cb80 con 0x7fc41809a3b0 2023-09-30T22:24:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.716+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4180a17c0 con 0x7fc41809a3b0 2023-09-30T22:24:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.717+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc40800cce0 con 0x7fc41809a3b0 2023-09-30T22:24:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.717+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fc40803ff10 con 0x7fc41809a3b0 2023-09-30T22:24:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.717+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc418074ea0 con 0x7fc41809a3b0 2023-09-30T22:24:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.722+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc408017070 con 0x7fc41809a3b0 2023-09-30T22:24:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.970+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc418071db0 con 0x7fc41809a3b0 2023-09-30T22:24:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.970+0000 7fc40effd700 1 -- 172.21.15.32:0/2066379409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2315 (secure 0 0 0) 0x7fc40802a090 con 0x7fc41809a3b0 2023-09-30T22:24:03.971 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:03.971 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:21:37.188029+0000","modified":"2023-09-30T22:21:58.784702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4177":{"gid":4177,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6817/3095020018","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6817","nonce":3095020018}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:24:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 >> v1:172.21.15.32:6816/126553 conn(0x7fc3fc02f510 legacy=0x7fc3fc0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc41809a3b0 msgr2=0x7fc41806fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc41809a3b0 0x7fc41806fb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc40800e980 tx=0x7fc40800ec90 comp rx=0 tx=0).stop 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 shutdown_connections 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc418131810 0x7fc418070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc418099a40 0x7fc41806f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 --2- 172.21.15.32:0/2066379409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc41809a3b0 0x7fc41806fb50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.972+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 >> 172.21.15.32:0/2066379409 conn(0x7fc4180a2aa0 msgr2=0x7fc41809e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.973+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 shutdown_connections 2023-09-30T22:24:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:03.973+0000 7fc4206c5700 1 -- 172.21.15.32:0/2066379409 wait complete. 2023-09-30T22:24:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-09-30T22:24:04.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:21:37.188029+0000', 'modified': '2023-09-30T22:21:58.784702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4177': {'gid': 4177, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6817/3095020018', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.32:6817', 'nonce': 3095020018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-30T22:24:04.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:24:04.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:24:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.230+0000 7fcea15c6700 1 Processor -- start 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.230+0000 7fcea15c6700 1 -- start start 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fcea15c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c0a0a30 0x7fce9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fcea15c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fcea15c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9c0a2a70 con 0x7fce9c133930 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fcea15c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0a01d0 con 0x7fce9c0a0a30 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fcea15c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c09e2b0 con 0x7fce9c09d630 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c0a0a30 0x7fce9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce9c133930 0x7fce9c133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42912/0 (socket says 172.21.15.32:42912) 2023-09-30T22:24:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9b7fe700 1 -- 172.21.15.32:0/927256436 learned_addr learned my addr 172.21.15.32:0/927256436 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce99ffb700 1 -- 172.21.15.32:0/927256436 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336357594 0 0) 0x7fce9c0a2a70 con 0x7fce9c133930 2023-09-30T22:24:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9a7fc700 1 -- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c0a0a30 msgr2=0x7fce9c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c0a0a30 0x7fce9c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9a7fc700 1 -- 172.21.15.32:0/927256436 >> v1:172.21.15.32:6789/0 conn(0x7fce9c133930 legacy=0x7fce9c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9a7fc700 1 -- 172.21.15.32:0/927256436 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c09dc50 con 0x7fce9c09d630 2023-09-30T22:24:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9affd700 1 --2- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c0a0a30 0x7fce9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c0a1340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce90009870 tx=0x7fce90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc86c64576a96f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.231+0000 7fce99ffb700 1 -- 172.21.15.32:0/927256436 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90011830 con 0x7fce9c09d630 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fce99ffb700 1 -- 172.21.15.32:0/927256436 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce90011990 con 0x7fce9c09d630 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fce99ffb700 1 -- 172.21.15.32:0/927256436 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90011d20 con 0x7fce9c09d630 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 -- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 msgr2=0x7fce9c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 --2- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c0a1340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce90009870 tx=0x7fce90009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 -- 172.21.15.32:0/927256436 shutdown_connections 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 --2- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c0a0a30 0x7fce9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 --2- 172.21.15.32:0/927256436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c0a1340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 -- 172.21.15.32:0/927256436 >> 172.21.15.32:0/927256436 conn(0x7fce9c0a3050 msgr2=0x7fce9c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 -- 172.21.15.32:0/927256436 shutdown_connections 2023-09-30T22:24:04.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.232+0000 7fcea15c6700 1 -- 172.21.15.32:0/927256436 wait complete. 2023-09-30T22:24:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.233+0000 7fcea15c6700 1 Processor -- start 2023-09-30T22:24:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.233+0000 7fcea15c6700 1 -- start start 2023-09-30T22:24:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fcea15c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c11f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fcea15c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce9c0a0a30 0x7fce9c11f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fcea15c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c133930 0x7fce9c1c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fcea15c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c11ed60 con 0x7fce9c0a0a30 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fcea15c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c1224a0 con 0x7fce9c133930 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fcea15c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c120490 con 0x7fce9c09d630 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c11f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c11f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55852/0 (socket says 172.21.15.32:55852) 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c133930 0x7fce9c1c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 -- 172.21.15.32:0/1127740278 learned_addr learned my addr 172.21.15.32:0/1127740278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce9c0a0a30 0x7fce9c11f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 -- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c133930 msgr2=0x7fce9c1c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c133930 0x7fce9c1c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 -- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce9c0a0a30 msgr2=0x7fce9c11f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce9c0a0a30 0x7fce9c11f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9affd700 1 -- 172.21.15.32:0/1127740278 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0094a0 con 0x7fce9c09d630 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.234+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce9c0a0a30 0x7fce9c11f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fce9affd700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c11f2e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00ebe0 tx=0x7fce8c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fce837fe700 1 -- 172.21.15.32:0/1127740278 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00c850 con 0x7fce9c09d630 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fce837fe700 1 -- 172.21.15.32:0/1127740278 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c00c9b0 con 0x7fce9c09d630 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce900094a0 con 0x7fce9c09d630 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fce837fe700 1 -- 172.21.15.32:0/1127740278 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00cd40 con 0x7fce9c09d630 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0a0410 con 0x7fce9c09d630 2023-09-30T22:24:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.235+0000 7fce837fe700 1 -- 172.21.15.32:0/1127740278 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fce8c013470 con 0x7fce9c09d630 2023-09-30T22:24:04.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.236+0000 7fce837fe700 1 -- 172.21.15.32:0/1127740278 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fce8c040000 con 0x7fce9c09d630 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 Processor -- start 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 -- start start 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eec3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eedf100 0x55631eec3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eed98f0 0x55631eed9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631ee25c60 con 0x55631eedf100 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x55631ee20a10 con 0x55631eed98f0 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.239+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x55631ee1e210 con 0x55631eec2c50 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eec3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eed98f0 0x55631eed9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eedf100 0x55631eec3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eec3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55868/0 (socket says 172.21.15.32:55868) 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9affd700 1 -- 172.21.15.32:0/2573427370 learned_addr learned my addr 172.21.15.32:0/2573427370 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 -- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 msgr2=0x55631eec3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eec3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 -- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eedf100 msgr2=0x55631eec3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eedf100 0x55631eec3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 -- 172.21.15.32:0/2573427370 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55631ec0d790 con 0x55631eed98f0 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9affd700 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eec3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eed98f0 0x55631eed9d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce94002230 tx=0x7fce940074c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535316a19a9a8b40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce80ff9700 1 -- 172.21.15.32:0/2573427370 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94017070 con 0x55631eed98f0 2023-09-30T22:24:04.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce80ff9700 1 -- 172.21.15.32:0/2573427370 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94012560 con 0x55631eed98f0 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.240+0000 7fce80ff9700 1 -- 172.21.15.32:0/2573427370 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94001eb0 con 0x55631eed98f0 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 -- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eed98f0 msgr2=0x55631eed9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eed98f0 0x55631eed9d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce94002230 tx=0x7fce940074c0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 -- 172.21.15.32:0/2573427370 shutdown_connections 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eed98f0 0x55631eed9d60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eec3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 --2- 172.21.15.32:0/2573427370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eedf100 0x55631eec3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 -- 172.21.15.32:0/2573427370 >> 172.21.15.32:0/2573427370 conn(0x55631eec70f0 msgr2=0x55631eec6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 -- 172.21.15.32:0/2573427370 shutdown_connections 2023-09-30T22:24:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.241+0000 7fcead769b80 1 -- 172.21.15.32:0/2573427370 wait complete. 2023-09-30T22:24:04.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 -- start start 2023-09-30T22:24:04.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 Processor -- start 2023-09-30T22:24:04.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eee7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eee84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 0x55631eee8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631ee19b00 con 0x55631eed98f0 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x55631edb5770 con 0x55631eedf100 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x55631edd86f0 con 0x55631eec2c50 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fcead769b80 10 client.0 _ll_register_callbacks cb 0x55631ec215a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fce9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 0x55631eee8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eee84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fce9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 0x55631eee8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39242/0 (socket says 172.21.15.32:39242) 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.242+0000 7fce9b7fe700 1 -- 172.21.15.32:0/1021971308 learned_addr learned my addr 172.21.15.32:0/1021971308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce9b7fe700 1 -- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 msgr2=0x55631eee7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eee7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce9b7fe700 1 -- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 msgr2=0x55631eee84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eee84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce9b7fe700 1 -- 172.21.15.32:0/1021971308 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0a0410 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 0x55631eee8a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fce94007810 tx=0x7fce94001250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe0ea20f6d2c9db7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3301/0 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce79ffb700 1 -- 172.21.15.32:0/1021971308 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94017070 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce79ffb700 1 -- 172.21.15.32:0/1021971308 <== mon.1 v2:172.21.15.32:3301/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fce94012e70 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55631eebb2b0 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce79ffb700 1 -- 172.21.15.32:0/1021971308 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce94001eb0 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 10 client.14365 fetch_fsmap learned FSMap version 8 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) v3 -- 0x7fce94002010 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce79ffb700 1 -- 172.21.15.32:0/1021971308 <== mon.1 v2:172.21.15.32:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55631eebb2b0 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fce79ffb700 1 -- 172.21.15.32:0/1021971308 <== mon.1 v2:172.21.15.32:3301/0 5 ==== fsmap(e 8) v1 ==== 1436+0+0 (secure 0 0 0) 0x7fce94012070 con 0x55631eedf100 2023-09-30T22:24:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 10 client.14365 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 10 client.14365 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 20 client.14365 populate_metadata read hostname 'smithi032' 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 4 client.14365 mds_command: new command op to 4177 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.243+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 --> v1:172.21.15.32:6817/3095020018 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55631ee97f40 con 0x55631eefc150 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.244+0000 7fce79ffb700 10 client.14365 ms_handle_connect on v1:172.21.15.32:6817/3095020018 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.244+0000 7fce79ffb700 1 -- 172.21.15.32:0/1021971308 <== mds.0 v1:172.21.15.32:6817/3095020018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x55631ee97f40 con 0x55631eefc150 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.244+0000 7fce79ffb700 10 client.14365 handle_command_reply: tid=0 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.244+0000 7fcead769b80 1 client.14365 shutdown 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 msgr2=0x55631eee8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 0x55631eee8a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fce94007810 tx=0x7fce94001250 comp rx=0 tx=0).stop 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 shutdown_connections 2023-09-30T22:24:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eedf100 0x55631eee8a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fce79ffb700 0 client.14365 ms_handle_reset on v1:172.21.15.32:6817/3095020018 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eec2c50 0x55631eee7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 --2- 172.21.15.32:0/1021971308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eee84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 >> 172.21.15.32:0/1021971308 conn(0x55631eec70f0 msgr2=0x55631eec21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 shutdown_connections 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 1 -- 172.21.15.32:0/1021971308 wait complete. 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.245+0000 7fcead769b80 20 client.14365 trim_cache size 0 max 16384 2023-09-30T22:24:04.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.262+0000 7fcead769b80 1 Processor -- start 2023-09-30T22:24:04.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.262+0000 7fcead769b80 1 -- start start 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eeff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eefe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eefea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631eedd2f0 con 0x55631eed98f0 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x55631ec0e390 con 0x55631eec2c50 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x55631eec4000 con 0x55631eedf100 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eeff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eefea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eefe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eefea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55892/0 (socket says 172.21.15.32:55892) 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eeff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39254/0 (socket says 172.21.15.32:39254) 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eefe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45862/0 (socket says 172.21.15.32:45862) 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9b7fe700 1 -- 172.21.15.32:0/2027673166 learned_addr learned my addr 172.21.15.32:0/2027673166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 msgr2=0x55631eefea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eefea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 msgr2=0x55631eeff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eeff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9a7fc700 1 -- 172.21.15.32:0/2027673166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55631ecfa0a0 con 0x55631eed98f0 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eefea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.263+0000 7fce9affd700 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eeff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eefe530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce90016000 tx=0x7fce90010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffcb6ec90a99091 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fce817fa700 1 -- 172.21.15.32:0/2027673166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90022050 con 0x55631eed98f0 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fce817fa700 1 -- 172.21.15.32:0/2027673166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce90023040 con 0x55631eed98f0 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fce817fa700 1 -- 172.21.15.32:0/2027673166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90012ea0 con 0x55631eed98f0 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 -- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 msgr2=0x55631eefe530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eefe530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce90016000 tx=0x7fce90010000 comp rx=0 tx=0).stop 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 -- 172.21.15.32:0/2027673166 shutdown_connections 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eefea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eeff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 --2- 172.21.15.32:0/2027673166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eefe530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 -- 172.21.15.32:0/2027673166 >> 172.21.15.32:0/2027673166 conn(0x55631eec70f0 msgr2=0x55631eec6640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 -- 172.21.15.32:0/2027673166 shutdown_connections 2023-09-30T22:24:04.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.264+0000 7fcead769b80 1 -- 172.21.15.32:0/2027673166 wait complete. 2023-09-30T22:24:04.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 -- start start 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 Processor -- start 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eedf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eedfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eee0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55631eeec100 con 0x55631eed98f0 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x55631ef2c5f0 con 0x55631eec2c50 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x55631eedbd90 con 0x55631eedf100 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fcead769b80 10 client.0 _ll_register_callbacks cb 0x7fce94007060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eedf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eedfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fce9affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eedf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39270/0 (socket says 172.21.15.32:39270) 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fce9a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eedfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45870/0 (socket says 172.21.15.32:45870) 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.265+0000 7fce9affd700 1 -- 172.21.15.32:0/1315153007 learned_addr learned my addr 172.21.15.32:0/1315153007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9b7fe700 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eee0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9affd700 1 -- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 msgr2=0x55631eee0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9affd700 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eee0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9affd700 1 -- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 msgr2=0x55631eedfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9affd700 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eedfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9affd700 1 -- 172.21.15.32:0/1315153007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce900094a0 con 0x55631eec2c50 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9a7fc700 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eedfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce9affd700 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eedf610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce8c03eb50 tx=0x7fce8c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a171a8be5436b3b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3301/0 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce79ffb700 1 -- 172.21.15.32:0/1315153007 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c011730 con 0x55631eec2c50 2023-09-30T22:24:04.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce79ffb700 1 -- 172.21.15.32:0/1315153007 <== mon.1 v2:172.21.15.32:3301/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fce8c02a0b0 con 0x55631eec2c50 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce94012560 con 0x55631eec2c50 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce79ffb700 1 -- 172.21.15.32:0/1315153007 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce8c0119a0 con 0x55631eec2c50 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fcead769b80 10 client.14371 fetch_fsmap learned FSMap version 8 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) v3 -- 0x7fce94002010 con 0x55631eec2c50 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.266+0000 7fce79ffb700 1 -- 172.21.15.32:0/1315153007 <== mon.1 v2:172.21.15.32:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c03a020 con 0x55631eec2c50 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fce79ffb700 1 -- 172.21.15.32:0/1315153007 <== mon.1 v2:172.21.15.32:3301/0 5 ==== fsmap(e 8) v1 ==== 1436+0+0 (secure 0 0 0) 0x7fce8c024950 con 0x55631eec2c50 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 10 client.14371 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 10 client.14371 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 20 client.14371 populate_metadata read hostname 'smithi032' 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 4 client.14371 mds_command: new command op to 4177 tid=0[{"prefix": "status"}] 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 --> v1:172.21.15.32:6817/3095020018 -- command(tid 0: {"prefix": "status"}) v1 -- 0x55631ee12ce0 con 0x55631ef45620 2023-09-30T22:24:04.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fce79ffb700 10 client.14371 ms_handle_connect on v1:172.21.15.32:6817/3095020018 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fce79ffb700 1 -- 172.21.15.32:0/1315153007 <== mds.0 v1:172.21.15.32:6817/3095020018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (unknown 0 0 2437261562) 0x55631ee12ce0 con 0x55631ef45620 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fce79ffb700 10 client.14371 handle_command_reply: tid=0 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 client.14371 shutdown 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 msgr2=0x55631eedf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eedf610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce8c03eb50 tx=0x7fce8c0047c0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 shutdown_connections 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x55631eedf100 0x55631eee0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x55631eec2c50 0x55631eedf610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 --2- 172.21.15.32:0/1315153007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55631eed98f0 0x55631eedfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fce79ffb700 0 client.14371 ms_handle_reset on v1:172.21.15.32:6817/3095020018 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 >> 172.21.15.32:0/1315153007 conn(0x55631eec70f0 msgr2=0x55631eef80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.267+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 shutdown_connections 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcead769b80 1 -- 172.21.15.32:0/1315153007 wait complete. 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcead769b80 20 client.14371 trim_cache size 0 max 16384 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9e20f926-ec7d-45ec-8e27-e5e820815322", 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4177, 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-09-30T22:24:04.275 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-09-30T22:24:04.276 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 147.07925845, 2023-09-30T22:24:04.276 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 8, 2023-09-30T22:24:04.276 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 23, 2023-09-30T22:24:04.276 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 18, 2023-09-30T22:24:04.276 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 152.362589915 2023-09-30T22:24:04.276 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 >> v1:172.21.15.32:6816/126553 conn(0x7fce8802f510 legacy=0x7fce880319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 msgr2=0x7fce9c11f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c11f2e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00ebe0 tx=0x7fce8c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 shutdown_connections 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce9c133930 0x7fce9c1c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce9c09d630 0x7fce9c11f2e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 --2- 172.21.15.32:0/1127740278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce9c0a0a30 0x7fce9c11f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 >> 172.21.15.32:0/1127740278 conn(0x7fce9c0a3050 msgr2=0x7fce9c1bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 shutdown_connections 2023-09-30T22:24:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.268+0000 7fcea15c6700 1 -- 172.21.15.32:0/1127740278 wait complete. 2023-09-30T22:24:04.282 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:24:04.293 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T22:24:04.293 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:24:04.322 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T22:24:04.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2023-09-30T22:24:04.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.504+0000 7f71dda14700 1 Processor -- start 2023-09-30T22:24:04.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.504+0000 7f71dda14700 1 -- start start 2023-09-30T22:24:04.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71dda14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71dda14700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71d809f5b0 0x7f71d8134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71dda14700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d806bec0 con 0x7f71d809f160 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71dda14700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d80a0030 con 0x7f71d809f5b0 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71dda14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80a0830 con 0x7f71d809e850 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71d809f5b0 0x7f71d8134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71d809f160 0x7f71d8133c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42916/0 (socket says 172.21.15.32:42916) 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71d809f5b0 0x7f71d8134370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45874/0 (socket says 172.21.15.32:45874) 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d67fc700 1 -- 172.21.15.32:0/1837438887 learned_addr learned my addr 172.21.15.32:0/1837438887 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d77fe700 1 -- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 msgr2=0x7f71d809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d77fe700 1 --2- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.505+0000 7f71d77fe700 1 -- 172.21.15.32:0/1837438887 >> v1:172.21.15.32:6789/0 conn(0x7f71d809f160 legacy=0x7f71d8133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71d77fe700 1 -- 172.21.15.32:0/1837438887 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d806caa0 con 0x7f71d809f5b0 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71d5ffb700 1 -- 172.21.15.32:0/1837438887 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908022961 0 0) 0x7f71d806bec0 con 0x7f71d809f160 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71d77fe700 1 --2- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f5b0 0x7f71d8134370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009d90 tx=0x7f71d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7166a22a4ca85c5a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71d5ffb700 1 -- 172.21.15.32:0/1837438887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d000e070 con 0x7f71d809f5b0 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71d5ffb700 1 -- 172.21.15.32:0/1837438887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d000bb20 con 0x7f71d809f5b0 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71d5ffb700 1 -- 172.21.15.32:0/1837438887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0012430 con 0x7f71d809f5b0 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71dda14700 1 -- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f5b0 msgr2=0x7f71d8134370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71dda14700 1 --2- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f5b0 0x7f71d8134370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009d90 tx=0x7f71d000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:04.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71dda14700 1 -- 172.21.15.32:0/1837438887 shutdown_connections 2023-09-30T22:24:04.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71dda14700 1 --2- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f5b0 0x7f71d8134370 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71dda14700 1 --2- 172.21.15.32:0/1837438887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.506+0000 7f71dda14700 1 -- 172.21.15.32:0/1837438887 >> 172.21.15.32:0/1837438887 conn(0x7f71d80a3f70 msgr2=0x7f71d80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:04.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.507+0000 7f71dda14700 1 -- 172.21.15.32:0/1837438887 shutdown_connections 2023-09-30T22:24:04.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.507+0000 7f71dda14700 1 -- 172.21.15.32:0/1837438887 wait complete. 2023-09-30T22:24:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.508+0000 7f71dda14700 1 Processor -- start 2023-09-30T22:24:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.508+0000 7f71dda14700 1 -- start start 2023-09-30T22:24:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71dda14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d81204e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71dda14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 0x7f71d8120a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71dda14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71d809f5b0 0x7f71d8120f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71dda14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d8121970 con 0x7f71d809f160 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71dda14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d8122060 con 0x7f71d809f5b0 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71dda14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71d8122680 con 0x7f71d809e850 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 0x7f71d8120a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d81204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71d809f5b0 0x7f71d8120f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d81204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55920/0 (socket says 172.21.15.32:55920) 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 0x7f71d8120a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45882/0 (socket says 172.21.15.32:45882) 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 -- 172.21.15.32:0/3156184489 learned_addr learned my addr 172.21.15.32:0/3156184489 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 -- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71d809f5b0 msgr2=0x7f71d8120f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71d809f5b0 0x7f71d8120f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 -- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 msgr2=0x7f71d8120a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 0x7f71d8120a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d6ffd700 1 -- 172.21.15.32:0/3156184489 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d0009a70 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.509+0000 7f71d67fc700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 0x7f71d8120a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.510+0000 7f71d6ffd700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d81204e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e8d0 tx=0x7f71c800ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.510+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8009590 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.510+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c8013680 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.510+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c80139f0 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.510+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d8123c90 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.510+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d80a3370 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.511+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f71c8013b50 con 0x7f71d809e850 2023-09-30T22:24:04.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.511+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f71c803fa60 con 0x7f71d809e850 2023-09-30T22:24:04.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.511+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d806cd20 con 0x7f71d809e850 2023-09-30T22:24:04.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.516+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71c8017080 con 0x7f71d809e850 2023-09-30T22:24:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:04.730+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7f71d80747f0 con 0x7f71d809e850 2023-09-30T22:24:04.731 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-09-30T22:24:04.730+0000 7f445a3d0700 -1 mon.a@0(leader).osd e23 definitely_dead 0 2023-09-30T22:24:05.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.066+0000 7f71cffff700 1 -- 172.21.15.32:0/3156184489 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 marked down osd.0. v24) v1 ==== 89+0+0 (secure 0 0 0) 0x7f71c8021020 con 0x7f71d809e850 2023-09-30T22:24:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.070+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 >> v1:172.21.15.32:6816/126553 conn(0x7f71b802f510 legacy=0x7f71b80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.070+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 msgr2=0x7f71d81204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:05.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.071+0000 7f71dda14700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d81204e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e8d0 tx=0x7f71c800ebe0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.071+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 shutdown_connections 2023-09-30T22:24:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.071+0000 7f71dda14700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71d809f5b0 0x7f71d8120f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.071+0000 7f71dda14700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71d809e850 0x7f71d81204e0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e8d0 tx=0x7f71c800ebe0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.071+0000 7f71dda14700 1 --2- 172.21.15.32:0/3156184489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71d809f160 0x7f71d8120a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.071+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 >> 172.21.15.32:0/3156184489 conn(0x7f71d80a3f70 msgr2=0x7f71d81b9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.072+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 shutdown_connections 2023-09-30T22:24:05.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.073+0000 7f71dda14700 1 -- 172.21.15.32:0/3156184489 wait complete. 2023-09-30T22:24:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:marked down osd.0. 2023-09-30T22:24:05.089 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T22:24:05.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T22:24:05.091 INFO:tasks.ceph.osd.0:Started 2023-09-30T22:24:05.092 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T22:24:05.092 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:24:05.129 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T22:24:05.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2023-09-30T22:24:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.307+0000 7f2b74261700 1 Processor -- start 2023-09-30T22:24:05.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.307+0000 7f2b74261700 1 -- start start 2023-09-30T22:24:05.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.307+0000 7f2b74261700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b6c0a0090 0x7f2b6c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:05.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.307+0000 7f2b74261700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:05.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b74261700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6c09ba70 con 0x7f2b6c09c650 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b74261700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6c09e8f0 con 0x7f2b6c0a0090 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b74261700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c0a0760 con 0x7f2b6c09d130 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b717fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b6c0a0090 0x7f2b6c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b71ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b6c09c650 0x7f2b6c09ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42930/0 (socket says 172.21.15.32:42930) 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b71ffd700 1 -- 172.21.15.32:0/1523642501 learned_addr learned my addr 172.21.15.32:0/1523642501 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b727fe700 1 --2- 172.21.15.32:0/1523642501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b727fe700 1 -- 172.21.15.32:0/1523642501 >> v2:172.21.15.32:3300/0 conn(0x7f2b6c0a0090 msgr2=0x7f2b6c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b727fe700 1 --2- 172.21.15.32:0/1523642501 >> v2:172.21.15.32:3300/0 conn(0x7f2b6c0a0090 0x7f2b6c136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b727fe700 1 -- 172.21.15.32:0/1523642501 >> v1:172.21.15.32:6789/0 conn(0x7f2b6c09c650 legacy=0x7f2b6c09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b727fe700 1 -- 172.21.15.32:0/1523642501 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c09a080 con 0x7f2b6c09d130 2023-09-30T22:24:05.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b70ffb700 1 -- 172.21.15.32:0/1523642501 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414882350 0 0) 0x7f2b6c09ba70 con 0x7f2b6c09c650 2023-09-30T22:24:05.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b717fc700 1 --2- 172.21.15.32:0/1523642501 >> v2:172.21.15.32:3300/0 conn(0x7f2b6c0a0090 0x7f2b6c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:05.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b727fe700 1 --2- 172.21.15.32:0/1523642501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c139170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009d90 tx=0x7f2b6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57cde3d53d1b7ce1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:05.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b70ffb700 1 -- 172.21.15.32:0/1523642501 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400e070 con 0x7f2b6c09d130 2023-09-30T22:24:05.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b70ffb700 1 -- 172.21.15.32:0/1523642501 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400bb20 con 0x7f2b6c09d130 2023-09-30T22:24:05.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.308+0000 7f2b70ffb700 1 -- 172.21.15.32:0/1523642501 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64012430 con 0x7f2b6c09d130 2023-09-30T22:24:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 -- 172.21.15.32:0/1523642501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 msgr2=0x7f2b6c139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 --2- 172.21.15.32:0/1523642501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c139170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009d90 tx=0x7f2b6400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 -- 172.21.15.32:0/1523642501 shutdown_connections 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 --2- 172.21.15.32:0/1523642501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c139170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 --2- 172.21.15.32:0/1523642501 >> v2:172.21.15.32:3300/0 conn(0x7f2b6c0a0090 0x7f2b6c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 -- 172.21.15.32:0/1523642501 >> 172.21.15.32:0/1523642501 conn(0x7f2b6c0a3f70 msgr2=0x7f2b6c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 -- 172.21.15.32:0/1523642501 shutdown_connections 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.309+0000 7f2b74261700 1 -- 172.21.15.32:0/1523642501 wait complete. 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.310+0000 7f2b74261700 1 Processor -- start 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 -- start start 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 0x7f2b6c135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b6c0a0090 0x7f2b6c135be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c09d980 con 0x7f2b6c09c650 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c136cf0 con 0x7f2b6c09d130 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b74261700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c1372d0 con 0x7f2b6c0a0090 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b71ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 0x7f2b6c135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b71ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 0x7f2b6c135160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45902/0 (socket says 172.21.15.32:45902) 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b71ffd700 1 -- 172.21.15.32:0/2839183867 learned_addr learned my addr 172.21.15.32:0/2839183867 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b727fe700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b6c0a0090 0x7f2b6c135be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.311+0000 7f2b717fc700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b71ffd700 1 -- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b6c0a0090 msgr2=0x7f2b6c135be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b71ffd700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b6c0a0090 0x7f2b6c135be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b71ffd700 1 -- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 msgr2=0x7f2b6c1356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b71ffd700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c1356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b71ffd700 1 -- 172.21.15.32:0/2839183867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0094a0 con 0x7f2b6c09c650 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b71ffd700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 0x7f2b6c135160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c520 tx=0x7f2b6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004c80 con 0x7f2b6c09c650 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b68004e00 con 0x7f2b6c09c650 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b64009a70 con 0x7f2b6c09c650 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68007310 con 0x7f2b6c09c650 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.312+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c0a36a0 con 0x7f2b6c09c650 2023-09-30T22:24:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.313+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2b68007470 con 0x7f2b6c09c650 2023-09-30T22:24:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.313+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f2b6803f780 con 0x7f2b6c09c650 2023-09-30T22:24:05.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.313+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c0747f0 con 0x7f2b6c09c650 2023-09-30T22:24:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.318+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b680123b0 con 0x7f2b6c09c650 2023-09-30T22:24:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:05.535+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7f2b6c072840 con 0x7f2b6c09c650 2023-09-30T22:24:05.536 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-09-30T22:24:05.535+0000 7f445a3d0700 -1 mon.a@0(leader).osd e24 definitely_dead 0 2023-09-30T22:24:05.697 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:24:05.697+0000 7f2ce612f680 -1 Falling back to public interface 2023-09-30T22:24:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.069+0000 7f2b627fc700 1 -- 172.21.15.32:0/2839183867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v25) v1 ==== 89+0+0 (secure 0 0 0) 0x7f2b680102c0 con 0x7f2b6c09c650 2023-09-30T22:24:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.075+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 >> v1:172.21.15.32:6816/126553 conn(0x7f2b5002f510 legacy=0x7f2b500319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.075+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 msgr2=0x7f2b6c135160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.075+0000 7f2b74261700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 0x7f2b6c135160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c520 tx=0x7f2b6800c830 comp rx=0 tx=0).stop 2023-09-30T22:24:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.077+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 shutdown_connections 2023-09-30T22:24:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.077+0000 7f2b74261700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b6c0a0090 0x7f2b6c135be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.077+0000 7f2b74261700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b6c09c650 0x7f2b6c135160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.077+0000 7f2b74261700 1 --2- 172.21.15.32:0/2839183867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b6c09d130 0x7f2b6c1356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.077+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 >> 172.21.15.32:0/2839183867 conn(0x7f2b6c0a3f70 msgr2=0x7f2b6c138d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.078+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 shutdown_connections 2023-09-30T22:24:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.079+0000 7f2b74261700 1 -- 172.21.15.32:0/2839183867 wait complete. 2023-09-30T22:24:06.081 INFO:teuthology.orchestra.run.smithi032.stderr:marked down osd.1. 2023-09-30T22:24:06.097 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T22:24:06.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T22:24:06.100 INFO:tasks.ceph.osd.1:Started 2023-09-30T22:24:06.100 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T22:24:06.100 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:24:06.128 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T22:24:06.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2023-09-30T22:24:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.307+0000 7fb621ce6700 1 Processor -- start 2023-09-30T22:24:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.307+0000 7fb621ce6700 1 -- start start 2023-09-30T22:24:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.307+0000 7fb621ce6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb61c130670 0x7fb61c130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb621ce6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c06f680 0x7fb61c06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb621ce6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 0x7fb61c137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb621ce6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c09a350 con 0x7fb61c130670 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb621ce6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c130260 con 0x7fb61c06ffd0 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb621ce6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c071ff0 con 0x7fb61c06f680 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c06f680 0x7fb61c06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb61c130670 0x7fb61c130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 0x7fb61c137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 0x7fb61c137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39330/0 (socket says 172.21.15.32:39330) 2023-09-30T22:24:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61bfff700 1 -- 172.21.15.32:0/2361359984 learned_addr learned my addr 172.21.15.32:0/2361359984 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c06f680 0x7fb61c06fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55978/0 (socket says 172.21.15.32:55978) 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 -- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c06f680 msgr2=0x7fb61c06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c06f680 0x7fb61c06fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 -- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 msgr2=0x7fb61c137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 0x7fb61c137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 -- 172.21.15.32:0/2361359984 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c09ecf0 con 0x7fb61c130670 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61bfff700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 0x7fb61c137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61b7fe700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c130670 0x7fb61c130a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb614009d90 tx=0x7fb61400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b903645f375aad13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61a7fc700 1 -- 172.21.15.32:0/2361359984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400e070 con 0x7fb61c130670 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61a7fc700 1 -- 172.21.15.32:0/2361359984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61400bb20 con 0x7fb61c130670 2023-09-30T22:24:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.308+0000 7fb61a7fc700 1 -- 172.21.15.32:0/2361359984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614012430 con 0x7fb61c130670 2023-09-30T22:24:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 -- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c130670 msgr2=0x7fb61c130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c130670 0x7fb61c130a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb614009d90 tx=0x7fb61400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 -- 172.21.15.32:0/2361359984 shutdown_connections 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06ffd0 0x7fb61c137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c130670 0x7fb61c130a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2361359984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c06f680 0x7fb61c06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 -- 172.21.15.32:0/2361359984 >> 172.21.15.32:0/2361359984 conn(0x7fb61c0a2aa0 msgr2=0x7fb61c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 -- 172.21.15.32:0/2361359984 shutdown_connections 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.309+0000 7fb621ce6700 1 -- 172.21.15.32:0/2361359984 wait complete. 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.310+0000 7fb621ce6700 1 Processor -- start 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.310+0000 7fb621ce6700 1 -- start start 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb621ce6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06f680 0x7fb61c09a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb621ce6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c06ffd0 0x7fb61c09aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb621ce6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 0x7fb61c09b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb621ce6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c09c190 con 0x7fb61c06ffd0 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb621ce6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c09b9e0 con 0x7fb61c06f680 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb621ce6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c1236e0 con 0x7fb61c130670 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 0x7fb61c09b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 0x7fb61c09b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55988/0 (socket says 172.21.15.32:55988) 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 -- 172.21.15.32:0/2185210357 learned_addr learned my addr 172.21.15.32:0/2185210357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61affd700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c06ffd0 0x7fb61c09aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61b7fe700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06f680 0x7fb61c09a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 -- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06f680 msgr2=0x7fb61c09a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06f680 0x7fb61c09a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 -- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c06ffd0 msgr2=0x7fb61c09aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c06ffd0 0x7fb61c09aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.311+0000 7fb61bfff700 1 -- 172.21.15.32:0/2185210357 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c009550 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.312+0000 7fb61bfff700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 0x7fb61c09b000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00eb60 tx=0x7fb60c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.312+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c009e60 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.312+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c004020 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.312+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb614009a70 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.312+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0043b0 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.312+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c12fc00 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.313+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb60c00c850 con 0x7fb61c130670 2023-09-30T22:24:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.313+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c074ea0 con 0x7fb61c130670 2023-09-30T22:24:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.313+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fb60c03fce0 con 0x7fb61c130670 2023-09-30T22:24:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.318+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb60c017080 con 0x7fb61c130670 2023-09-30T22:24:06.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:06.534+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7fb61c072210 con 0x7fb61c130670 2023-09-30T22:24:06.535 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-09-30T22:24:06.535+0000 7f445a3d0700 -1 mon.a@0(leader).osd e25 definitely_dead 0 2023-09-30T22:24:06.706 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:24:06.706+0000 7f3cb9eaf680 -1 Falling back to public interface 2023-09-30T22:24:07.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.074+0000 7fb613fff700 1 -- 172.21.15.32:0/2185210357 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v26) v1 ==== 89+0+0 (secure 0 0 0) 0x7fb60c020020 con 0x7fb61c130670 2023-09-30T22:24:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.078+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 >> v1:172.21.15.32:6816/126553 conn(0x7fb5fc02f510 legacy=0x7fb5fc0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.078+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 msgr2=0x7fb61c09b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.078+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 0x7fb61c09b000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00eb60 tx=0x7fb60c00ee70 comp rx=0 tx=0).stop 2023-09-30T22:24:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.079+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 shutdown_connections 2023-09-30T22:24:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.079+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb61c130670 0x7fb61c09b000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.079+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb61c06f680 0x7fb61c09a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.079+0000 7fb621ce6700 1 --2- 172.21.15.32:0/2185210357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb61c06ffd0 0x7fb61c09aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.079+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 >> 172.21.15.32:0/2185210357 conn(0x7fb61c0a2aa0 msgr2=0x7fb61c12e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.080+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 shutdown_connections 2023-09-30T22:24:07.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.080+0000 7fb621ce6700 1 -- 172.21.15.32:0/2185210357 wait complete. 2023-09-30T22:24:07.082 INFO:teuthology.orchestra.run.smithi032.stderr:marked down osd.2. 2023-09-30T22:24:07.098 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T22:24:07.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T22:24:07.101 INFO:tasks.ceph.osd.2:Started 2023-09-30T22:24:07.101 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T22:24:07.101 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:24:07.136 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T22:24:07.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2023-09-30T22:24:07.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.312+0000 7f949b37e700 1 Processor -- start 2023-09-30T22:24:07.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.312+0000 7f949b37e700 1 -- start start 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949b37e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f949406a770 0x7f949406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949b37e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f9494131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949b37e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949409ece0 con 0x7f9494131b40 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949b37e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949409e760 con 0x7f949406a770 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949b37e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f949409e040 con 0x7f949406b080 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f9498919700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f9494131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949911a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f949406a770 0x7f949406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f9498919700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f9494131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56000/0 (socket says 172.21.15.32:56000) 2023-09-30T22:24:07.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f9498919700 1 -- 172.21.15.32:0/1206220351 learned_addr learned my addr 172.21.15.32:0/1206220351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949911a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f949406a770 0x7f949406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45960/0 (socket says 172.21.15.32:45960) 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949991b700 1 --1- 172.21.15.32:0/1206220351 >> v1:172.21.15.32:6789/0 conn(0x7f9494131b40 0x7f9494138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42940/0 (socket says 172.21.15.32:42940) 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949911a700 1 -- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 msgr2=0x7f9494131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949911a700 1 --2- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f9494131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949911a700 1 -- 172.21.15.32:0/1206220351 >> v1:172.21.15.32:6789/0 conn(0x7f9494131b40 legacy=0x7f9494138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f948bfff700 1 -- 172.21.15.32:0/1206220351 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697168603 0 0) 0x7f949409ece0 con 0x7f9494131b40 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f949911a700 1 -- 172.21.15.32:0/1206220351 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949409f630 con 0x7f949406a770 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.313+0000 7f9498919700 1 --2- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f9494131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949911a700 1 --2- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f949406a770 0x7f949406ab40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9490009840 tx=0x7f9490009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d290c2d11acce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f948bfff700 1 -- 172.21.15.32:0/1206220351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490011830 con 0x7f949406a770 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f948bfff700 1 -- 172.21.15.32:0/1206220351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490011990 con 0x7f949406a770 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f948bfff700 1 -- 172.21.15.32:0/1206220351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490011d20 con 0x7f949406a770 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 -- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f949406a770 msgr2=0x7f949406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 --2- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f949406a770 0x7f949406ab40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9490009840 tx=0x7f9490009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 -- 172.21.15.32:0/1206220351 shutdown_connections 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 --2- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f949406a770 0x7f949406ab40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 --2- 172.21.15.32:0/1206220351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f9494131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 -- 172.21.15.32:0/1206220351 >> 172.21.15.32:0/1206220351 conn(0x7f9494066f50 msgr2=0x7f949409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:07.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.314+0000 7f949b37e700 1 -- 172.21.15.32:0/1206220351 shutdown_connections 2023-09-30T22:24:07.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.315+0000 7f949b37e700 1 -- 172.21.15.32:0/1206220351 wait complete. 2023-09-30T22:24:07.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.315+0000 7f949b37e700 1 Processor -- start 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.315+0000 7f949b37e700 1 -- start start 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949b37e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f949406a770 0x7f949411c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949b37e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f949411cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949b37e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9494131b40 0x7f949411d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949b37e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949411fcd0 con 0x7f9494131b40 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949b37e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949411fed0 con 0x7f949406a770 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949b37e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9494131580 con 0x7f949406b080 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f949411cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949911a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f949406a770 0x7f949411c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949991b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9494131b40 0x7f949411d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f949411cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56016/0 (socket says 172.21.15.32:56016) 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 -- 172.21.15.32:0/954235275 learned_addr learned my addr 172.21.15.32:0/954235275 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 -- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f949406a770 msgr2=0x7f949411c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f949406a770 0x7f949411c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 -- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9494131b40 msgr2=0x7f949411d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9494131b40 0x7f949411d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f9498919700 1 -- 172.21.15.32:0/954235275 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484009550 con 0x7f949406b080 2023-09-30T22:24:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949991b700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9494131b40 0x7f949411d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.316+0000 7f949911a700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f949406a770 0x7f949411c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.317+0000 7f9498919700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f949411cae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f948400ebe0 tx=0x7f948400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.317+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400c850 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.317+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948400c9b0 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.317+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400cd40 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.317+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94900094a0 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.317+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949411e0b0 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.318+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9484013470 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.318+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f9484040140 con 0x7f949406b080 2023-09-30T22:24:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.318+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494074ea0 con 0x7f949406b080 2023-09-30T22:24:07.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.323+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9484017020 con 0x7f949406b080 2023-09-30T22:24:07.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:07.539+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7f949411e7b0 con 0x7f949406b080 2023-09-30T22:24:07.540 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-09-30T22:24:07.539+0000 7f445a3d0700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2023-09-30T22:24:07.706 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:24:07.706+0000 7f307f22f680 -1 Falling back to public interface 2023-09-30T22:24:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.078+0000 7f9489ffb700 1 -- 172.21.15.32:0/954235275 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v27) v1 ==== 89+0+0 (secure 0 0 0) 0x7f948401b3b0 con 0x7f949406b080 2023-09-30T22:24:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 >> v1:172.21.15.32:6816/126553 conn(0x7f947802f540 legacy=0x7f94780319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 msgr2=0x7f949411cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f949411cae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f948400ebe0 tx=0x7f948400eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 shutdown_connections 2023-09-30T22:24:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9494131b40 0x7f949411d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f949406a770 0x7f949411c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 --2- 172.21.15.32:0/954235275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f949406b080 0x7f949411cae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 >> 172.21.15.32:0/954235275 conn(0x7f9494066f50 msgr2=0x7f949406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 shutdown_connections 2023-09-30T22:24:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.081+0000 7f949b37e700 1 -- 172.21.15.32:0/954235275 wait complete. 2023-09-30T22:24:08.085 INFO:teuthology.orchestra.run.smithi032.stderr:marked down osd.3. 2023-09-30T22:24:08.098 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T22:24:08.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T22:24:08.101 INFO:tasks.ceph.osd.3:Started 2023-09-30T22:24:08.101 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:24:08.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.242+0000 7f5f0802c700 1 Processor -- start 2023-09-30T22:24:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f0802c700 1 -- start start 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f0802c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f000a0a30 0x7f5f000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f0802c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f0009d630 0x7f5f000a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f0802c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f000a2a70 con 0x7f5f00133930 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f0802c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f000a01d0 con 0x7f5f0009d630 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f0802c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0009e2b0 con 0x7f5f000a0a30 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f055c7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f0009d630 0x7f5f000a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f05dc8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f000a0a30 0x7f5f000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f065c9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5f00133930 0x7f5f00133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42956/0 (socket says 172.21.15.32:42956) 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.243+0000 7f5f065c9700 1 -- 172.21.15.32:0/3190905505 learned_addr learned my addr 172.21.15.32:0/3190905505 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f05dc8700 1 -- 172.21.15.32:0/3190905505 >> v2:172.21.15.32:3300/0 conn(0x7f5f0009d630 msgr2=0x7f5f000a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f05dc8700 1 --2- 172.21.15.32:0/3190905505 >> v2:172.21.15.32:3300/0 conn(0x7f5f0009d630 0x7f5f000a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f05dc8700 1 -- 172.21.15.32:0/3190905505 >> v1:172.21.15.32:6789/0 conn(0x7f5f00133930 legacy=0x7f5f00133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f05dc8700 1 -- 172.21.15.32:0/3190905505 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0009dc50 con 0x7f5f000a0a30 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f04dc6700 1 -- 172.21.15.32:0/3190905505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832180124 0 0) 0x7f5f000a2a70 con 0x7f5f00133930 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f055c7700 1 --2- 172.21.15.32:0/3190905505 >> v2:172.21.15.32:3300/0 conn(0x7f5f0009d630 0x7f5f000a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f05dc8700 1 --2- 172.21.15.32:0/3190905505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f000a0a30 0x7f5f000a0e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009840 tx=0x7f5efc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1690757f71e28e27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f04dc6700 1 -- 172.21.15.32:0/3190905505 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc011830 con 0x7f5f000a0a30 2023-09-30T22:24:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f04dc6700 1 -- 172.21.15.32:0/3190905505 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc011990 con 0x7f5f000a0a30 2023-09-30T22:24:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.244+0000 7f5f04dc6700 1 -- 172.21.15.32:0/3190905505 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc011d20 con 0x7f5f000a0a30 2023-09-30T22:24:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 -- 172.21.15.32:0/3190905505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f000a0a30 msgr2=0x7f5f000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 --2- 172.21.15.32:0/3190905505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f000a0a30 0x7f5f000a0e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009840 tx=0x7f5efc009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 -- 172.21.15.32:0/3190905505 shutdown_connections 2023-09-30T22:24:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 --2- 172.21.15.32:0/3190905505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f000a0a30 0x7f5f000a0e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 --2- 172.21.15.32:0/3190905505 >> v2:172.21.15.32:3300/0 conn(0x7f5f0009d630 0x7f5f000a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 -- 172.21.15.32:0/3190905505 >> 172.21.15.32:0/3190905505 conn(0x7f5f000a3050 msgr2=0x7f5f000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 -- 172.21.15.32:0/3190905505 shutdown_connections 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.245+0000 7f5f0802c700 1 -- 172.21.15.32:0/3190905505 wait complete. 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 Processor -- start 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 -- start start 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 0x7f5f001c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f000a0a30 0x7f5f001c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f00133930 0x7f5f001c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00121e90 con 0x7f5f00133930 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00121290 con 0x7f5f000a0a30 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.246+0000 7f5f0802c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00120520 con 0x7f5f0009d630 2023-09-30T22:24:08.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 0x7f5f001c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 0x7f5f001c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56036/0 (socket says 172.21.15.32:56036) 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 -- 172.21.15.32:0/2615327826 learned_addr learned my addr 172.21.15.32:0/2615327826 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f055c7700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f000a0a30 0x7f5f001c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f065c9700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f00133930 0x7f5f001c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 -- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f000a0a30 msgr2=0x7f5f001c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f000a0a30 0x7f5f001c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 -- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f00133930 msgr2=0x7f5f001c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f00133930 0x7f5f001c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 -- 172.21.15.32:0/2615327826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00094a0 con 0x7f5f0009d630 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f065c9700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f00133930 0x7f5f001c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:08.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f055c7700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f000a0a30 0x7f5f001c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f05dc8700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 0x7f5f001c7ef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009fd0 tx=0x7f5efc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc004480 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc01f070 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0038e0 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc0094a0 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.247+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f000a0410 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.248+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5efc021030 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.248+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f5efc040c50 con 0x7f5f0009d630 2023-09-30T22:24:08.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.249+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f000a1800 con 0x7f5f0009d630 2023-09-30T22:24:08.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.254+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5efc015080 con 0x7f5f0009d630 2023-09-30T22:24:08.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.457+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f000747f0 con 0x7f5f0009d630 2023-09-30T22:24:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.458+0000 7f5ef67fc700 1 -- 172.21.15.32:0/2615327826 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+11900 (secure 0 0 0) 0x7f5efc003a40 con 0x7f5f0009d630 2023-09-30T22:24:08.459 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:08.460 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:08.073447+0000","last_up_change":"2023-09-30T22:24:08.073447+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":118763}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":118763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":118763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":118763}]},"public_addr":"172.21.15.32:6804/118763","cluster_addr":"172.21.15.32:6805/118763","heartbeat_back_addr":"172.21.15.32:6807/118763","heartbeat_front_addr":"172.21.15.32:6806/118763","state":["exists"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 >> v1:172.21.15.32:6816/126553 conn(0x7f5ee402f510 legacy=0x7f5ee40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 msgr2=0x7f5f001c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 0x7f5f001c7ef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009fd0 tx=0x7f5efc004750 comp rx=0 tx=0).stop 2023-09-30T22:24:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 shutdown_connections 2023-09-30T22:24:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f00133930 0x7f5f001c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f0009d630 0x7f5f001c7ef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 --2- 172.21.15.32:0/2615327826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f000a0a30 0x7f5f001c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 >> 172.21.15.32:0/2615327826 conn(0x7f5f000a3050 msgr2=0x7f5f001bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 shutdown_connections 2023-09-30T22:24:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:08.461+0000 7f5f0802c700 1 -- 172.21.15.32:0/2615327826 wait complete. 2023-09-30T22:24:08.708 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:24:08.708+0000 7fa4e6ff0680 -1 Falling back to public interface 2023-09-30T22:24:09.497 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:24:09.496+0000 7f2ce612f680 -1 osd.0 23 log_to_monitors true 2023-09-30T22:24:10.546 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:24:10.545+0000 7f3cb9eaf680 -1 osd.1 23 log_to_monitors true 2023-09-30T22:24:10.964 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-09-30T22:24:10.963+0000 7f2cd20aa700 -1 osd.0 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:24:11.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:11.498 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:24:11.498+0000 7f307f22f680 -1 osd.2 25 log_to_monitors true 2023-09-30T22:24:11.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.665+0000 7f4187a09700 1 Processor -- start 2023-09-30T22:24:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.665+0000 7f4187a09700 1 -- start start 2023-09-30T22:24:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4187a09700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f418009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4187a09700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f418009e000 0x7f4180130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4187a09700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f4180130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4187a09700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418009f520 con 0x7f418009e000 2023-09-30T22:24:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4187a09700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418009f130 con 0x7f418009e410 2023-09-30T22:24:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4187a09700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f418009e930 con 0x7f418009d6f0 2023-09-30T22:24:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f4180130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f4180130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35776/0 (socket says 172.21.15.32:35776) 2023-09-30T22:24:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 -- 172.21.15.32:0/3819731341 learned_addr learned my addr 172.21.15.32:0/3819731341 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f41857a5700 1 --2- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f418009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4184fa4700 1 --2- 172.21.15.32:0/3819731341 >> v2:172.21.15.32:3300/0 conn(0x7f418009e000 0x7f4180130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 -- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 msgr2=0x7f418009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 --2- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f418009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 -- 172.21.15.32:0/3819731341 >> v2:172.21.15.32:3300/0 conn(0x7f418009e000 msgr2=0x7f4180130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 --2- 172.21.15.32:0/3819731341 >> v2:172.21.15.32:3300/0 conn(0x7f418009e000 0x7f4180130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.666+0000 7f4185fa6700 1 -- 172.21.15.32:0/3819731341 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418009b2d0 con 0x7f418009e410 2023-09-30T22:24:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.667+0000 7f4185fa6700 1 --2- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f4180130ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4178009ed0 tx=0x7f417800d540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd2878a2e38d8e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.667+0000 7f4177fff700 1 -- 172.21.15.32:0/3819731341 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800dde0 con 0x7f418009e410 2023-09-30T22:24:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.667+0000 7f4177fff700 1 -- 172.21.15.32:0/3819731341 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4178011460 con 0x7f418009e410 2023-09-30T22:24:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.667+0000 7f4177fff700 1 -- 172.21.15.32:0/3819731341 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178011810 con 0x7f418009e410 2023-09-30T22:24:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 -- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 msgr2=0x7f4180130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 --2- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f4180130ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4178009ed0 tx=0x7f417800d540 comp rx=0 tx=0).stop 2023-09-30T22:24:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 -- 172.21.15.32:0/3819731341 shutdown_connections 2023-09-30T22:24:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 --2- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f4180130ba0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 --2- 172.21.15.32:0/3819731341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f418009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 --2- 172.21.15.32:0/3819731341 >> v2:172.21.15.32:3300/0 conn(0x7f418009e000 0x7f4180130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 -- 172.21.15.32:0/3819731341 >> 172.21.15.32:0/3819731341 conn(0x7f4180066f50 msgr2=0x7f418009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 -- 172.21.15.32:0/3819731341 shutdown_connections 2023-09-30T22:24:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.668+0000 7f4187a09700 1 -- 172.21.15.32:0/3819731341 wait complete. 2023-09-30T22:24:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 Processor -- start 2023-09-30T22:24:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 -- start start 2023-09-30T22:24:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f41801b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418009e000 0x7f41801b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f41801c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418009c770 con 0x7f418009e000 2023-09-30T22:24:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418011f6a0 con 0x7f418009e410 2023-09-30T22:24:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.670+0000 7f4187a09700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4180130510 con 0x7f418009d6f0 2023-09-30T22:24:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4185fa6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f41801c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4185fa6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f41801c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35784/0 (socket says 172.21.15.32:35784) 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f41857a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f41801b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4185fa6700 1 -- 172.21.15.32:0/2016728970 learned_addr learned my addr 172.21.15.32:0/2016728970 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418009e000 0x7f41801b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 -- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 msgr2=0x7f41801b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f41801b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 -- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 msgr2=0x7f41801c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f41801c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 -- 172.21.15.32:0/2016728970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178009b30 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.671+0000 7f4184fa4700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418009e000 0x7f41801b89d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f417000e900 tx=0x7f417000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.672+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170013680 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.672+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41700137e0 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.672+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41801b7fb0 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.672+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170013b50 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.672+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418011fe30 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.673+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4170013cb0 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.673+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418011f210 con 0x7f418009e000 2023-09-30T22:24:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.673+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4020+0+0 (secure 0 0 0) 0x7f417003fe50 con 0x7f418009e000 2023-09-30T22:24:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.679+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f417001ab90 con 0x7f418009e000 2023-09-30T22:24:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.967+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4180074ea0 con 0x7f418009e000 2023-09-30T22:24:11.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.968+0000 7f4175ffb700 1 -- 172.21.15.32:0/2016728970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12688 (secure 0 0 0) 0x7f4170017070 con 0x7f418009e000 2023-09-30T22:24:11.969 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:11.969 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:11.083043+0000","last_up_change":"2023-09-30T22:24:11.083043+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"error_message":"pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"error_message":"pool 4 has some PGs where all OSDs are with primary_affinity 0 (at least pg 4.0), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":19,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":118762}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":118762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":118762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":118762}]},"public_addr":"172.21.15.32:6800/118762","cluster_addr":"172.21.15.32:6801/118762","heartbeat_back_addr":"172.21.15.32:6803/118762","heartbeat_front_addr":"172.21.15.32:6802/118762","state":["exists"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":118793}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":118793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":118793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":118793}]},"public_addr":"172.21.15.32:6808/118793","cluster_addr":"172.21.15.32:6810/118793","heartbeat_back_addr":"172.21.15.32:6813/118793","heartbeat_front_addr":"172.21.15.32:6812/118793","state":["exists"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":118794}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":118794}]},"public_addr":"172.21.15.32:6809/118794","cluster_addr":"172.21.15.32:6811/118794","heartbeat_back_addr":"172.21.15.32:6815/118794","heartbeat_front_addr":"172.21.15.32:6814/118794","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:11.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 >> v1:172.21.15.32:6816/126553 conn(0x7f416402f510 legacy=0x7f41640319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:11.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418009e000 msgr2=0x7f41801b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:11.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418009e000 0x7f41801b89d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f417000e900 tx=0x7f417000ec10 comp rx=0 tx=0).stop 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 shutdown_connections 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f418009e410 0x7f41801c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f418009d6f0 0x7f41801b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 --2- 172.21.15.32:0/2016728970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f418009e000 0x7f41801b89d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.970+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 >> 172.21.15.32:0/2016728970 conn(0x7f4180066f50 msgr2=0x7f418006b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.971+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 shutdown_connections 2023-09-30T22:24:11.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:11.971+0000 7f4187a09700 1 -- 172.21.15.32:0/2016728970 wait complete. 2023-09-30T22:24:12.002 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-09-30T22:24:12.002+0000 7f3ca5e2a700 -1 osd.1 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:24:12.507 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:24:12.507+0000 7fa4e6ff0680 -1 osd.3 26 log_to_monitors true 2023-09-30T22:24:12.993 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-09-30T22:24:12.992+0000 7f306b1aa700 -1 osd.2 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:24:14.018 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-09-30T22:24:14.017+0000 7fa4d2f6b700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:24:14.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:15.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.150+0000 7fe7d8e40700 1 Processor -- start 2023-09-30T22:24:15.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d8e40700 1 -- start start 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d8e40700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7d40984f0 0x7fe7d40988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d8e40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d409d6f0 0x7fe7d409db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d8e40700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d4099620 con 0x7fe7d409e040 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d8e40700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d409a9e0 con 0x7fe7d40984f0 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d8e40700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d409bce0 con 0x7fe7d409d6f0 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d409d6f0 0x7fe7d409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:15.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7d40984f0 0x7fe7d40988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7d409e040 0x7fe7d41385c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54838/0 (socket says 172.21.15.32:54838) 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.151+0000 7fe7d2d9d700 1 -- 172.21.15.32:0/2383412545 learned_addr learned my addr 172.21.15.32:0/2383412545 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d1d9b700 1 -- 172.21.15.32:0/2383412545 >> v2:172.21.15.32:3300/0 conn(0x7fe7d40984f0 msgr2=0x7fe7d40988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d1d9b700 1 --2- 172.21.15.32:0/2383412545 >> v2:172.21.15.32:3300/0 conn(0x7fe7d40984f0 0x7fe7d40988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d1d9b700 1 -- 172.21.15.32:0/2383412545 >> v1:172.21.15.32:6789/0 conn(0x7fe7d409e040 legacy=0x7fe7d41385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d159a700 1 -- 172.21.15.32:0/2383412545 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398618568 0 0) 0x7fe7d4099620 con 0x7fe7d409e040 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d1d9b700 1 -- 172.21.15.32:0/2383412545 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d409aca0 con 0x7fe7d409d6f0 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d259c700 1 --2- 172.21.15.32:0/2383412545 >> v2:172.21.15.32:3300/0 conn(0x7fe7d40984f0 0x7fe7d40988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d1d9b700 1 --2- 172.21.15.32:0/2383412545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d409d6f0 0x7fe7d409db00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009840 tx=0x7fe7c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8906a5fc82de8896 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d159a700 1 -- 172.21.15.32:0/2383412545 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011830 con 0x7fe7d409d6f0 2023-09-30T22:24:15.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d159a700 1 -- 172.21.15.32:0/2383412545 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4011990 con 0x7fe7d409d6f0 2023-09-30T22:24:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.152+0000 7fe7d159a700 1 -- 172.21.15.32:0/2383412545 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011d20 con 0x7fe7d409d6f0 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 -- 172.21.15.32:0/2383412545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d409d6f0 msgr2=0x7fe7d409db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/2383412545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d409d6f0 0x7fe7d409db00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009840 tx=0x7fe7c4009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 -- 172.21.15.32:0/2383412545 shutdown_connections 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/2383412545 >> v2:172.21.15.32:3300/0 conn(0x7fe7d40984f0 0x7fe7d40988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/2383412545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d409d6f0 0x7fe7d409db00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 -- 172.21.15.32:0/2383412545 >> 172.21.15.32:0/2383412545 conn(0x7fe7d409f470 msgr2=0x7fe7d40a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 -- 172.21.15.32:0/2383412545 shutdown_connections 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.153+0000 7fe7d8e40700 1 -- 172.21.15.32:0/2383412545 wait complete. 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.154+0000 7fe7d8e40700 1 Processor -- start 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.154+0000 7fe7d8e40700 1 -- start start 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.154+0000 7fe7d8e40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 0x7fe7d406d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.154+0000 7fe7d8e40700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d409d6f0 0x7fe7d406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.154+0000 7fe7d8e40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 0x7fe7d406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.154+0000 7fe7d8e40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d412e310 con 0x7fe7d409e040 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d8e40700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d412e4f0 con 0x7fe7d40984f0 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d8e40700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d406f140 con 0x7fe7d409d6f0 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 0x7fe7d406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 0x7fe7d406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 0x7fe7d406d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35802/0 (socket says 172.21.15.32:35802) 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 0x7fe7d406e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44202/0 (socket says 172.21.15.32:44202) 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d259c700 1 -- 172.21.15.32:0/3552645121 learned_addr learned my addr 172.21.15.32:0/3552645121 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d1d9b700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d409d6f0 0x7fe7d406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 -- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d409d6f0 msgr2=0x7fe7d406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d409d6f0 0x7fe7d406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 -- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 msgr2=0x7fe7d406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 0x7fe7d406d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 -- 172.21.15.32:0/3552645121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0094a0 con 0x7fe7d409e040 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d259c700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 0x7fe7d406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d2d9d700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 0x7fe7d406e0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00cc10 tx=0x7fe7cc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00aa00 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc00ab80 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c40094a0 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0070d0 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.155+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d412df50 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.156+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe7cc007230 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.156+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4020+0+0 (secure 0 0 0) 0x7fe7cc03f8b0 con 0x7fe7d409e040 2023-09-30T22:24:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.156+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d406ea80 con 0x7fe7d409e040 2023-09-30T22:24:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.162+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7cc018080 con 0x7fe7d409e040 2023-09-30T22:24:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.370+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7d412e140 con 0x7fe7d409e040 2023-09-30T22:24:15.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.371+0000 7fe7caffd700 1 -- 172.21.15.32:0/3552645121 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+12902 (secure 0 0 0) 0x7fe7cc0074e0 con 0x7fe7d409e040 2023-09-30T22:24:15.372 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:15.372 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:15.091140+0000","last_up_change":"2023-09-30T22:24:14.089689+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":30,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":129367}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":129367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":129367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":129367}]},"public_addr":"172.21.15.32:6804/129367","cluster_addr":"172.21.15.32:6805/129367","heartbeat_back_addr":"172.21.15.32:6807/129367","heartbeat_front_addr":"172.21.15.32:6806/129367","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":129477}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":129477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":129477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":129477}]},"public_addr":"172.21.15.32:6808/129477","cluster_addr":"172.21.15.32:6809/129477","heartbeat_back_addr":"172.21.15.32:6811/129477","heartbeat_front_addr":"172.21.15.32:6810/129477","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":129817}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":129817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":129817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":129817}]},"public_addr":"172.21.15.32:6812/129817","cluster_addr":"172.21.15.32:6813/129817","heartbeat_back_addr":"172.21.15.32:6815/129817","heartbeat_front_addr":"172.21.15.32:6814/129817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 >> v1:172.21.15.32:6816/126553 conn(0x7fe7b402f510 legacy=0x7fe7b40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 msgr2=0x7fe7d406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 0x7fe7d406e0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00cc10 tx=0x7fe7cc00cf20 comp rx=0 tx=0).stop 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 shutdown_connections 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7d409e040 0x7fe7d406e0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7d40984f0 0x7fe7d406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 --2- 172.21.15.32:0/3552645121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7d409d6f0 0x7fe7d406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 >> 172.21.15.32:0/3552645121 conn(0x7fe7d409f470 msgr2=0x7fe7d41b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 shutdown_connections 2023-09-30T22:24:15.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:15.373+0000 7fe7d8e40700 1 -- 172.21.15.32:0/3552645121 wait complete. 2023-09-30T22:24:15.387 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:24:15.388 INFO:teuthology.run_tasks:Running task ceph.stop... 2023-09-30T22:24:15.399 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-30T22:24:15.399 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:24:15.411 INFO:tasks.ceph.mds.a:Stopped 2023-09-30T22:24:21.065 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog finished 2023-09-30T22:24:21.066 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:24:21.077 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2023-09-30T22:24:21.077 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-30T22:24:21.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-30T22:24:21.080 INFO:tasks.ceph.mds.a:Started 2023-09-30T22:24:21.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:24:21.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:21.177 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-09-30T22:24:21.192 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-09-30T22:24:21.193+0000 7f445a3d0700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T22:24:21.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.236+0000 7f4f13e25700 1 Processor -- start 2023-09-30T22:24:21.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.236+0000 7f4f13e25700 1 -- start start 2023-09-30T22:24:21.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f13e25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f13e25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f13e25700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0c09ece0 con 0x7f4f0c06b080 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f13e25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c09e760 con 0x7f4f0c06a770 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f13e25700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c09e040 con 0x7f4f0c131d10 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f11bc1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f113c0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f0c06b080 0x7f4f0c131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47620/0 (socket says 172.21.15.32:47620) 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f113c0700 1 -- 172.21.15.32:0/4290211512 learned_addr learned my addr 172.21.15.32:0/4290211512 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f123c2700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f11bc1700 1 -- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 msgr2=0x7f4f0c139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f11bc1700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f11bc1700 1 -- 172.21.15.32:0/4290211512 >> v1:172.21.15.32:6789/0 conn(0x7f4f0c06b080 legacy=0x7f4f0c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f10bbf700 1 -- 172.21.15.32:0/4290211512 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668642614 0 0) 0x7f4f0c09ece0 con 0x7f4f0c06b080 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f11bc1700 1 -- 172.21.15.32:0/4290211512 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c09f630 con 0x7f4f0c06a770 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f123c2700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.237+0000 7f4f11bc1700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c06ab40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009870 tx=0x7f4f08009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3860bacdcbb5a49 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f10bbf700 1 -- 172.21.15.32:0/4290211512 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08011830 con 0x7f4f0c06a770 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f10bbf700 1 -- 172.21.15.32:0/4290211512 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f08011990 con 0x7f4f0c06a770 2023-09-30T22:24:21.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f10bbf700 1 -- 172.21.15.32:0/4290211512 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08011d20 con 0x7f4f0c06a770 2023-09-30T22:24:21.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 -- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 msgr2=0x7f4f0c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c06ab40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009870 tx=0x7f4f08009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 -- 172.21.15.32:0/4290211512 shutdown_connections 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 --2- 172.21.15.32:0/4290211512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c06ab40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 -- 172.21.15.32:0/4290211512 >> 172.21.15.32:0/4290211512 conn(0x7f4f0c066f50 msgr2=0x7f4f0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 -- 172.21.15.32:0/4290211512 shutdown_connections 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.238+0000 7f4f13e25700 1 -- 172.21.15.32:0/4290211512 wait complete. 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.239+0000 7f4f13e25700 1 Processor -- start 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.239+0000 7f4f13e25700 1 -- start start 2023-09-30T22:24:21.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f13e25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f13e25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 0x7f4f0c120940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f13e25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c120e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f13e25700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c1218d0 con 0x7f4f0c06b080 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f13e25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c123ed0 con 0x7f4f0c06a770 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f13e25700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c130eb0 con 0x7f4f0c131d10 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 0x7f4f0c120940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f11bc1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 0x7f4f0c120940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35964/0 (socket says 172.21.15.32:35964) 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f123c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c120e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 -- 172.21.15.32:0/1021173203 learned_addr learned my addr 172.21.15.32:0/1021173203 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 -- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 msgr2=0x7f4f0c120e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c120e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 -- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 msgr2=0x7f4f0c120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f113c0700 1 -- 172.21.15.32:0/1021173203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04009a70 con 0x7f4f0c06b080 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f123c2700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c120e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.240+0000 7f4f11bc1700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.241+0000 7f4f113c0700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 0x7f4f0c120940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4efc001eb0 tx=0x7f4efc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.241+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc004c80 con 0x7f4f0c06b080 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.241+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc004de0 con 0x7f4f0c06b080 2023-09-30T22:24:21.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.241+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f080094a0 con 0x7f4f0c06b080 2023-09-30T22:24:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.241+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc007310 con 0x7f4f0c06b080 2023-09-30T22:24:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.241+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0c06ed90 con 0x7f4f0c06b080 2023-09-30T22:24:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.242+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4efc007470 con 0x7f4f0c06b080 2023-09-30T22:24:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.242+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0c071ec0 con 0x7f4f0c06b080 2023-09-30T22:24:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.242+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4efc03f770 con 0x7f4f0c06b080 2023-09-30T22:24:21.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.247+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4efc0106d0 con 0x7f4f0c06b080 2023-09-30T22:24:21.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.455+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f0c074ea0 con 0x7f4f0c06b080 2023-09-30T22:24:21.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.456+0000 7f4f027fc700 1 -- 172.21.15.32:0/1021173203 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+12967 (secure 0 0 0) 0x7f4efc0102c0 con 0x7f4f0c06b080 2023-09-30T22:24:21.457 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:21.457 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:21.192888+0000","last_up_change":"2023-09-30T22:24:14.089689+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":30,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":129367}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":129367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":129367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":129367}]},"public_addr":"172.21.15.32:6804/129367","cluster_addr":"172.21.15.32:6805/129367","heartbeat_back_addr":"172.21.15.32:6807/129367","heartbeat_front_addr":"172.21.15.32:6806/129367","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":129477}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":129477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":129477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":129477}]},"public_addr":"172.21.15.32:6808/129477","cluster_addr":"172.21.15.32:6809/129477","heartbeat_back_addr":"172.21.15.32:6811/129477","heartbeat_front_addr":"172.21.15.32:6810/129477","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":129817}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":129817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":129817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":129817}]},"public_addr":"172.21.15.32:6812/129817","cluster_addr":"172.21.15.32:6813/129817","heartbeat_back_addr":"172.21.15.32:6815/129817","heartbeat_front_addr":"172.21.15.32:6814/129817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6817/3095020018":"2023-10-01T22:24:21.192854+0000","172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 >> v1:172.21.15.32:6816/126553 conn(0x7f4ef002f510 legacy=0x7f4ef00319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 msgr2=0x7f4f0c120940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 0x7f4f0c120940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4efc001eb0 tx=0x7f4efc00c800 comp rx=0 tx=0).stop 2023-09-30T22:24:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 shutdown_connections 2023-09-30T22:24:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f0c131d10 0x7f4f0c120e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f0c06a770 0x7f4f0c120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 --2- 172.21.15.32:0/1021173203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f0c06b080 0x7f4f0c120940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 >> 172.21.15.32:0/1021173203 conn(0x7f4f0c066f50 msgr2=0x7f4f0c0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:21.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 shutdown_connections 2023-09-30T22:24:21.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.459+0000 7f4f13e25700 1 -- 172.21.15.32:0/1021173203 wait complete. 2023-09-30T22:24:21.472 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:24:21.472 INFO:teuthology.run_tasks:Running task exec... 2023-09-30T22:24:21.483 INFO:teuthology.task.exec:Executing custom commands... 2023-09-30T22:24:21.484 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:24:21.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2023-09-30T22:24:21.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.618+0000 7f323c9af700 1 Processor -- start 2023-09-30T22:24:21.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.618+0000 7f323c9af700 1 -- start start 2023-09-30T22:24:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323c9af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809d930 0x7f323809dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323c9af700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f323809e690 0x7f3238134300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323c9af700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323806b640 con 0x7f323809e240 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323c9af700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323809a790 con 0x7f323809e690 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323c9af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323809ef00 con 0x7f323809d930 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f3236d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f323809e690 0x7f3238134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f3235d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f323809e240 0x7f3238133bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47626/0 (socket says 172.21.15.32:47626) 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809d930 0x7f323809dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f3235d9b700 1 -- 172.21.15.32:0/3247400421 learned_addr learned my addr 172.21.15.32:0/3247400421 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:21.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323659c700 1 -- 172.21.15.32:0/3247400421 >> v2:172.21.15.32:3300/0 conn(0x7f323809e690 msgr2=0x7f3238134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323659c700 1 --2- 172.21.15.32:0/3247400421 >> v2:172.21.15.32:3300/0 conn(0x7f323809e690 0x7f3238134300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323559a700 1 -- 172.21.15.32:0/3247400421 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832354250 0 0) 0x7f323806b640 con 0x7f323809e240 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323659c700 1 -- 172.21.15.32:0/3247400421 >> v1:172.21.15.32:6789/0 conn(0x7f323809e240 legacy=0x7f3238133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323659c700 1 -- 172.21.15.32:0/3247400421 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323809f1a0 con 0x7f323809d930 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.619+0000 7f323659c700 1 --2- 172.21.15.32:0/3247400421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809d930 0x7f323809dd00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f322c009840 tx=0x7f322c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed75b8d5e1cb059 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323559a700 1 -- 172.21.15.32:0/3247400421 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c011830 con 0x7f323809d930 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323559a700 1 -- 172.21.15.32:0/3247400421 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c011990 con 0x7f323809d930 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323559a700 1 -- 172.21.15.32:0/3247400421 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c011d20 con 0x7f323809d930 2023-09-30T22:24:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 -- 172.21.15.32:0/3247400421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809d930 msgr2=0x7f323809dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 --2- 172.21.15.32:0/3247400421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809d930 0x7f323809dd00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f322c009840 tx=0x7f322c009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 -- 172.21.15.32:0/3247400421 shutdown_connections 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 --2- 172.21.15.32:0/3247400421 >> v2:172.21.15.32:3300/0 conn(0x7f323809e690 0x7f3238134300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 --2- 172.21.15.32:0/3247400421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809d930 0x7f323809dd00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 -- 172.21.15.32:0/3247400421 >> 172.21.15.32:0/3247400421 conn(0x7f3238099c30 msgr2=0x7f32380a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 -- 172.21.15.32:0/3247400421 shutdown_connections 2023-09-30T22:24:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.620+0000 7f323c9af700 1 -- 172.21.15.32:0/3247400421 wait complete. 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 Processor -- start 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 -- start start 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 0x7f32381c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809e240 0x7f32381c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 0x7f32381c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32381a66d0 con 0x7f323809e690 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32381a64e0 con 0x7f323809e240 2023-09-30T22:24:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323c9af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32381a83f0 con 0x7f323809d930 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 0x7f32381c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f3236d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 0x7f32381c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 0x7f32381c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55408/0 (socket says 172.21.15.32:55408) 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f3236d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 0x7f32381c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35980/0 (socket says 172.21.15.32:35980) 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f323659c700 1 -- 172.21.15.32:0/154429233 learned_addr learned my addr 172.21.15.32:0/154429233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.622+0000 7f3235d9b700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809e240 0x7f32381c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:21.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3236d9d700 1 -- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 msgr2=0x7f32381c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3236d9d700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 0x7f32381c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3236d9d700 1 -- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809e240 msgr2=0x7f32381c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3236d9d700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809e240 0x7f32381c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3236d9d700 1 -- 172.21.15.32:0/154429233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0094a0 con 0x7f323809e690 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f323659c700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 0x7f32381c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3236d9d700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 0x7f32381c8b50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3230009d90 tx=0x7f323000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230009c00 con 0x7f323809e690 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000f040 con 0x7f323809e690 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230014480 con 0x7f323809e690 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32380a2690 con 0x7f323809e690 2023-09-30T22:24:21.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.623+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323809c890 con 0x7f323809e690 2023-09-30T22:24:21.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.624+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f32300145e0 con 0x7f323809e690 2023-09-30T22:24:21.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.624+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32381a7d90 con 0x7f323809e690 2023-09-30T22:24:21.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.624+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f3230040910 con 0x7f323809e690 2023-09-30T22:24:21.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.629+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32381a7d90 con 0x7f323809e690 2023-09-30T22:24:21.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.833+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f32381a7d90 con 0x7f323809e690 2023-09-30T22:24:21.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.835+0000 7f3226ffd700 1 -- 172.21.15.32:0/154429233 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v34) v1 ==== 81+0+24157 (secure 0 0 0) 0x7f32381a7d90 con 0x7f323809e690 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "epoch": 34, 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "fsid": "9e20f926-ec7d-45ec-8e27-e5e820815322", 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "created": "2023-09-30T22:21:19.836852+0000", 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "modified": "2023-09-30T22:24:21.192888+0000", 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "last_up_change": "2023-09-30T22:24:14.089689+0000", 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "last_in_change": "2023-09-30T22:21:22.213069+0000", 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-09-30T22:24:21.835 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_num": 5799936, 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_set": [ 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "pglog_hardlimit", 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "purged_snapdirs", 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_deletes", 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "sortbitwise" 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_version": 8, 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "full_ratio": 0.94999998807907104, 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "nearfull_ratio": 0.85000002384185791, 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_snapshot": "", 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_max": 4, 2023-09-30T22:24:21.836 INFO:teuthology.orchestra.run.smithi032.stdout: "max_osd": 4, 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "require_min_compat_client": "pacific", 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "min_compat_client": "luminous", 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "require_osd_release": "pacific", 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "allow_crimson": false, 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pools": [ 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 1, 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": ".mgr", 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2023-09-30T22:21:28.838521+0000", 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2023-09-30T22:24:21.837 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 1, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 1, 2023-09-30T22:24:21.838 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 1, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 1, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 1, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "23", 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2023-09-30T22:24:21.839 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2023-09-30T22:24:21.840 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.841 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2023-09-30T22:24:21.842 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_max": 32, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 1 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": {}, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr_devicehealth": {} 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 4, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 4, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 0.5, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2, 2023-09-30T22:24:21.843 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 2, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "rbd", 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2023-09-30T22:21:30.226037+0000", 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2023-09-30T22:24:21.844 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 8, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 8, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 8, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 8, 2023-09-30T22:24:21.845 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 8, 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "13", 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2023-09-30T22:24:21.846 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2023-09-30T22:24:21.847 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2023-09-30T22:24:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2023-09-30T22:24:21.849 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "options": {}, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "rbd": {} 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 2.5, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 2.5, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 2.5, 2023-09-30T22:24:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 2.5, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 3, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_metadata", 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2023-09-30T22:21:34.900746+0000", 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2023-09-30T22:24:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 32, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 32, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 32, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 32, 2023-09-30T22:24:21.852 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 32, 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "16", 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:24:21.853 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2023-09-30T22:24:21.854 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2023-09-30T22:24:21.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2023-09-30T22:24:21.856 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_bias": 4, 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 16, 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "recovery_priority": 5 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": "cephfs" 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2023-09-30T22:24:21.857 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.6299999952316284, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.6299999952316284, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.6299999952316284, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.6299999952316284, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "pool": 4, 2023-09-30T22:24:21.858 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_name": "cephfs_data", 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "create_time": "2023-09-30T22:21:36.177318+0000", 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags": 1, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "flags_names": "hashpspool", 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "type": 1, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "size": 2, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "min_size": 1, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_rule": 0, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:24:21.859 INFO:teuthology.orchestra.run.smithi032.stdout: "object_hash": 2, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num": 64, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num": 64, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_placement_num_target": 64, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_target": 64, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_pending": 64, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "last_pg_merge_meta": { 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "source_pgid": "0.0", 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "ready_epoch": 0, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_started": 0, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "last_epoch_clean": 0, 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "source_version": "0'0", 2023-09-30T22:24:21.860 INFO:teuthology.orchestra.run.smithi032.stdout: "target_version": "0'0" 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "last_change": "17", 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend": "0", 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "auid": 0, 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_seq": 0, 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "snap_epoch": 0, 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "pool_snaps": [], 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps": "[]", 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_bytes": 0, 2023-09-30T22:24:21.861 INFO:teuthology.orchestra.run.smithi032.stdout: "quota_max_objects": 0, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tiers": [], 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "tier_of": -1, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "read_tier": -1, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "write_tier": -1, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_mode": "none", 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_bytes": 0, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "target_max_objects": 0, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_flush_age": 0, 2023-09-30T22:24:21.862 INFO:teuthology.orchestra.run.smithi032.stdout: "cache_min_evict_age": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profile": "", 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_params": { 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "none" 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_period": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_count": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "use_gmt_hitset": true, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:24:21.863 INFO:teuthology.orchestra.run.smithi032.stdout: "grade_table": [], 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "stripe_width": 0, 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "expected_num_objects": 0, 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "fast_read": false, 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "options": { 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_num_min": 64, 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "target_size_ratio": 0.89999997615814209 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "application_metadata": { 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "cephfs": { 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: "data": "cephfs" 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "read_balance": { 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "score_acting": 1.059999942779541, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "score_stable": 1.059999942779541, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "optimal_score": 1, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_acting": 1.059999942779541, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "raw_score_stable": 1.059999942779541, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity": 1, 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.865 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "osds": [ 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "762e2f2b-1ff4-4d81-840e-a100cc5ed4e6", 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 9, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 23, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 29, 2023-09-30T22:24:21.866 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 24, 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6800", 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129269 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.867 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6801", 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129269 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6803", 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129269 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6802", 2023-09-30T22:24:21.869 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129269 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6800/129269", 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6801/129269", 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6803/129269", 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6802/129269", 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2023-09-30T22:24:21.870 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "3ff901b4-5d78-49f6-b425-d90361753515", 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 9, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 23, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 30, 2023-09-30T22:24:21.871 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 25, 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6804", 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129367 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.872 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6805", 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129367 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.873 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6807", 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129367 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6806", 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129367 2023-09-30T22:24:21.874 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6804/129367", 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6805/129367", 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6807/129367", 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6806/129367", 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.875 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "bfe588b1-e7fe-46ec-a3f2-fef1646f54c1", 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 9, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 25, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 31, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2023-09-30T22:24:21.876 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 26, 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6808", 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129477 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.877 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6809", 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129477 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.878 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6811", 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129477 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6810", 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129477 2023-09-30T22:24:21.879 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6808/129477", 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6809/129477", 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6811/129477", 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6810/129477", 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.880 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "uuid": "5b3ee2e9-f923-4976-ad87-11ba143ec874", 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "up": 1, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "in": 1, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "weight": 1, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_affinity": 1, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_begin": 9, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "last_clean_end": 26, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "up_from": 32, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "up_thru": 32, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "down_at": 27, 2023-09-30T22:24:21.881 INFO:teuthology.orchestra.run.smithi032.stdout: "lost_at": 0, 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addrs": { 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6812", 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129817 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addrs": { 2023-09-30T22:24:21.882 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6813", 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129817 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addrs": { 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.883 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6815", 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129817 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addrs": { 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addrvec": [ 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "172.21.15.32:6814", 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 129817 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.884 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "public_addr": "172.21.15.32:6812/129817", 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_addr": "172.21.15.32:6813/129817", 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_back_addr": "172.21.15.32:6815/129817", 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "heartbeat_front_addr": "172.21.15.32:6814/129817", 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "state": [ 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "exists", 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "up" 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_xinfo": [ 2023-09-30T22:24:21.885 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 0, 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2023-09-30T22:21:27.629625+0000", 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 24 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.886 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 1, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2023-09-30T22:21:27.321860+0000", 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 25 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 2, 2023-09-30T22:24:21.887 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "0.000000", 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2023-09-30T22:21:27.365072+0000", 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 26 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": 3, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "down_stamp": "2023-09-30T22:24:08.073447+0000", 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_probability": 0, 2023-09-30T22:24:21.888 INFO:teuthology.orchestra.run.smithi032.stdout: "laggy_interval": 0, 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "features": 4540138322906710015, 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "old_weight": 0, 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "last_purged_snaps_scrub": "2023-09-30T22:21:27.445985+0000", 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "dead_epoch": 0 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap": [], 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_items": [ 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.2", 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.889 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 3, 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.d", 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 3, 2023-09-30T22:24:21.890 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 2 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.11", 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 3, 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 2 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.891 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.14", 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 3, 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 2 2023-09-30T22:24:21.892 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.2d", 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.893 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 3, 2023-09-30T22:24:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 0 2023-09-30T22:24:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.894 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.31", 2023-09-30T22:24:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 1, 2023-09-30T22:24:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 2 2023-09-30T22:24:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.895 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "pgid": "4.3b", 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "mappings": [ 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "from": 3, 2023-09-30T22:24:21.896 INFO:teuthology.orchestra.run.smithi032.stdout: "to": 2 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_upmap_primaries": [], 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "pg_temp": [], 2023-09-30T22:24:21.897 INFO:teuthology.orchestra.run.smithi032.stdout: "primary_temp": [], 2023-09-30T22:24:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklist": { 2023-09-30T22:24:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6817/3095020018": "2023-10-01T22:24:21.192854+0000", 2023-09-30T22:24:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:0/2862279367": "2023-10-01T22:23:54.019278+0000", 2023-09-30T22:24:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:0/708160155": "2023-10-01T22:23:54.019278+0000", 2023-09-30T22:24:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: "172.21.15.32:6816/118188": "2023-10-01T22:23:54.019278+0000" 2023-09-30T22:24:21.898 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "range_blocklist": {}, 2023-09-30T22:24:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "erasure_code_profiles": { 2023-09-30T22:24:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "default": { 2023-09-30T22:24:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "crush-failure-domain": "osd", 2023-09-30T22:24:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "k": "2", 2023-09-30T22:24:21.899 INFO:teuthology.orchestra.run.smithi032.stdout: "m": "1", 2023-09-30T22:24:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "plugin": "jerasure", 2023-09-30T22:24:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "technique": "reed_sol_van" 2023-09-30T22:24:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "removed_snaps_queue": [], 2023-09-30T22:24:21.900 INFO:teuthology.orchestra.run.smithi032.stdout: "new_removed_snaps": [], 2023-09-30T22:24:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "new_purged_snaps": [], 2023-09-30T22:24:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "crush_node_flags": {}, 2023-09-30T22:24:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "device_class_flags": {}, 2023-09-30T22:24:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode": { 2023-09-30T22:24:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_enabled": false, 2023-09-30T22:24:21.901 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_bucket_count": 0, 2023-09-30T22:24:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "degraded_stretch_mode": 0, 2023-09-30T22:24:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "recovering_stretch_mode": 0, 2023-09-30T22:24:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: "stretch_mode_bucket": 0 2023-09-30T22:24:21.902 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:21.902 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T22:24:21.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 >> v1:172.21.15.32:6816/126553 conn(0x7f321802f510 legacy=0x7f32180319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 msgr2=0x7f32381c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 0x7f32381c8b50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3230009d90 tx=0x7f323000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 shutdown_connections 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323809e690 0x7f32381c8b50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f323809d930 0x7f32381c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 --2- 172.21.15.32:0/154429233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323809e240 0x7f32381c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 >> 172.21.15.32:0/154429233 conn(0x7f3238099c30 msgr2=0x7f32381bbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 shutdown_connections 2023-09-30T22:24:21.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:21.837+0000 7f323c9af700 1 -- 172.21.15.32:0/154429233 wait complete. 2023-09-30T22:24:21.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2023-09-30T22:24:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 Processor -- start 2023-09-30T22:24:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 -- start start 2023-09-30T22:24:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 0x7fada409f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada41317b0 0x7fada4131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada409abd0 con 0x7fada409f2e0 2023-09-30T22:24:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada406fd90 con 0x7fada41317b0 2023-09-30T22:24:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada8cda700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fada4071e80 con 0x7fada409e9d0 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 0x7fada409f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.040+0000 7fada259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada1d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 0x7fada409f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35986/0 (socket says 172.21.15.32:35986) 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada41317b0 0x7fada4131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada1d9b700 1 -- 172.21.15.32:0/2438923821 learned_addr learned my addr 172.21.15.32:0/2438923821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 -- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 msgr2=0x7fada409eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 --2- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada409eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 -- 172.21.15.32:0/2438923821 >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 msgr2=0x7fada409f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 --2- 172.21.15.32:0/2438923821 >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 0x7fada409f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 -- 172.21.15.32:0/2438923821 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40a0240 con 0x7fada41317b0 2023-09-30T22:24:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada1d9b700 1 --2- 172.21.15.32:0/2438923821 >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 0x7fada409f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada2d9d700 1 --2- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada41317b0 0x7fada4131c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009d90 tx=0x7fad9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0769b52f3c7c550 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada159a700 1 -- 172.21.15.32:0/2438923821 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00e070 con 0x7fada41317b0 2023-09-30T22:24:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada159a700 1 -- 172.21.15.32:0/2438923821 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9c00bb20 con 0x7fada41317b0 2023-09-30T22:24:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.041+0000 7fada159a700 1 -- 172.21.15.32:0/2438923821 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c012430 con 0x7fada41317b0 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 -- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada41317b0 msgr2=0x7fada4131c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 --2- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada41317b0 0x7fada4131c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009d90 tx=0x7fad9c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 -- 172.21.15.32:0/2438923821 shutdown_connections 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 --2- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada41317b0 0x7fada4131c00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 --2- 172.21.15.32:0/2438923821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada409eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 --2- 172.21.15.32:0/2438923821 >> v2:172.21.15.32:3300/0 conn(0x7fada409f2e0 0x7fada409f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 -- 172.21.15.32:0/2438923821 >> 172.21.15.32:0/2438923821 conn(0x7fada40a43c0 msgr2=0x7fada40a47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 -- 172.21.15.32:0/2438923821 shutdown_connections 2023-09-30T22:24:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.042+0000 7fada8cda700 1 -- 172.21.15.32:0/2438923821 wait complete. 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.043+0000 7fada8cda700 1 Processor -- start 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.043+0000 7fada8cda700 1 -- start start 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada8cda700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada411ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada8cda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada409f2e0 0x7fada411d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada8cda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 0x7fada411d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada8cda700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4130710 con 0x7fada41317b0 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada8cda700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada41209a0 con 0x7fada409f2e0 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada8cda700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fada40a2060 con 0x7fada409e9d0 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada411ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 0x7fada411d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada411ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55428/0 (socket says 172.21.15.32:55428) 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 0x7fada411d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35988/0 (socket says 172.21.15.32:35988) 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 -- 172.21.15.32:0/3777329537 learned_addr learned my addr 172.21.15.32:0/3777329537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada1d9b700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada409f2e0 0x7fada411d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 -- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada409f2e0 msgr2=0x7fada411d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada409f2e0 0x7fada411d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 -- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 msgr2=0x7fada411d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 0x7fada411d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 -- 172.21.15.32:0/3777329537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98009550 con 0x7fada409e9d0 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada2d9d700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 0x7fada411d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada1d9b700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada409f2e0 0x7fada411d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fada259c700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada411ce80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e930 tx=0x7fad9400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:22.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.044+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94013680 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.045+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad940137e0 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.045+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c009a70 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.045+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94013b70 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.045+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada411ea90 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.046+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada4120360 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.046+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fad94013cd0 con 0x7fada409e9d0 2023-09-30T22:24:22.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.046+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fad940401f0 con 0x7fada409e9d0 2023-09-30T22:24:22.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.051+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad9400c7d0 con 0x7fada409e9d0 2023-09-30T22:24:22.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.342+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fada409db80 con 0x7fada409e9d0 2023-09-30T22:24:22.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.342+0000 7fad92ffd700 1 -- 172.21.15.32:0/3777329537 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+627 (secure 0 0 0) 0x7fad9401ab50 con 0x7fada409e9d0 2023-09-30T22:24:22.342 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-09-30T22:24:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2023-09-30T22:24:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 3 2023-09-30T22:24:22.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 1 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 4 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 1 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 9 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-09-30T22:24:22.343 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-09-30T22:24:22.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 >> v1:172.21.15.32:6816/126553 conn(0x7fad8402f510 legacy=0x7fad840319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 msgr2=0x7fada411ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada411ce80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e930 tx=0x7fad9400ec40 comp rx=0 tx=0).stop 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 shutdown_connections 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fada41317b0 0x7fada411d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fada409e9d0 0x7fada411ce80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 --2- 172.21.15.32:0/3777329537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fada409f2e0 0x7fada411d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.344+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 >> 172.21.15.32:0/3777329537 conn(0x7fada40a43c0 msgr2=0x7fada4130bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.345+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 shutdown_connections 2023-09-30T22:24:22.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.345+0000 7fada8cda700 1 -- 172.21.15.32:0/3777329537 wait complete. 2023-09-30T22:24:22.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2023-09-30T22:24:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.492+0000 7f878a6c6700 1 Processor -- start 2023-09-30T22:24:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f878a6c6700 1 -- start start 2023-09-30T22:24:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f878a6c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f878409d930 0x7f878409dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f878a6c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e690 0x7f8784134300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f878a6c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878406b640 con 0x7f878409e240 2023-09-30T22:24:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f878a6c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878409a790 con 0x7f878409d930 2023-09-30T22:24:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f878a6c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f878409ef00 con 0x7f878409e690 2023-09-30T22:24:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f8788c63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e690 0x7f8784134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f8783fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f878409d930 0x7f878409dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f87837fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f878409e240 0x7f8784133bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47630/0 (socket says 172.21.15.32:47630) 2023-09-30T22:24:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.493+0000 7f87837fe700 1 -- 172.21.15.32:0/1417505165 learned_addr learned my addr 172.21.15.32:0/1417505165 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:22.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8788c63700 1 -- 172.21.15.32:0/1417505165 >> v2:172.21.15.32:3300/0 conn(0x7f878409d930 msgr2=0x7f878409dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8788c63700 1 --2- 172.21.15.32:0/1417505165 >> v2:172.21.15.32:3300/0 conn(0x7f878409d930 0x7f878409dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8788c63700 1 -- 172.21.15.32:0/1417505165 >> v1:172.21.15.32:6789/0 conn(0x7f878409e240 legacy=0x7f8784133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8788c63700 1 -- 172.21.15.32:0/1417505165 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878409f1a0 con 0x7f878409e690 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8783fff700 1 --2- 172.21.15.32:0/1417505165 >> v2:172.21.15.32:3300/0 conn(0x7f878409d930 0x7f878409dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8788c63700 1 --2- 172.21.15.32:0/1417505165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e690 0x7f8784134300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f877c009d90 tx=0x7f877c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a48c5cf7e4283c04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8782ffd700 1 -- 172.21.15.32:0/1417505165 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c00e070 con 0x7f878409e690 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8782ffd700 1 -- 172.21.15.32:0/1417505165 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c00bb20 con 0x7f878409e690 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.494+0000 7f8782ffd700 1 -- 172.21.15.32:0/1417505165 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c012430 con 0x7f878409e690 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 -- 172.21.15.32:0/1417505165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e690 msgr2=0x7f8784134300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1417505165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e690 0x7f8784134300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f877c009d90 tx=0x7f877c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 -- 172.21.15.32:0/1417505165 shutdown_connections 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1417505165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e690 0x7f8784134300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1417505165 >> v2:172.21.15.32:3300/0 conn(0x7f878409d930 0x7f878409dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 -- 172.21.15.32:0/1417505165 >> 172.21.15.32:0/1417505165 conn(0x7f8784099c30 msgr2=0x7f87840a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 -- 172.21.15.32:0/1417505165 shutdown_connections 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.495+0000 7f878a6c6700 1 -- 172.21.15.32:0/1417505165 wait complete. 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 Processor -- start 2023-09-30T22:24:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 -- start start 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 0x7f87841a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e240 0x7f87841a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f878409e690 0x7f87841a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87841aa4e0 con 0x7f878409d930 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87841a7cb0 con 0x7f878409e690 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f878a6c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87841a9840 con 0x7f878409e240 2023-09-30T22:24:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.496+0000 7f8783fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 0x7f87841a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e240 0x7f87841a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8783fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 0x7f87841a6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36010/0 (socket says 172.21.15.32:36010) 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8783fff700 1 -- 172.21.15.32:0/1339410214 learned_addr learned my addr 172.21.15.32:0/1339410214 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8788c63700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f878409e690 0x7f87841a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 -- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f878409e690 msgr2=0x7f87841a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f878409e690 0x7f87841a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 -- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 msgr2=0x7f87841a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 0x7f87841a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 -- 172.21.15.32:0/1339410214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740094a0 con 0x7f878409e240 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8783fff700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 0x7f87841a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f87837fe700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e240 0x7f87841a6590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f877800c530 tx=0x7f877800c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877800cea0 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778005d70 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87780072d0 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c009a70 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.497+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87841a8e60 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.498+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8778007430 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.498+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f877803efb0 con 0x7f878409e240 2023-09-30T22:24:22.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.498+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878409cc50 con 0x7f878409e240 2023-09-30T22:24:22.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.503+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8778017030 con 0x7f878409e240 2023-09-30T22:24:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.705+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f878404f2a0 con 0x7f878409e240 2023-09-30T22:24:22.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.705+0000 7f8780ff9700 1 -- 172.21.15.32:0/1339410214 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v34) v1 ==== 59+0+37 (secure 0 0 0) 0x7f8778017070 con 0x7f878409e240 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.707+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 >> v1:172.21.15.32:6816/126553 conn(0x7f877002f510 legacy=0x7f87700319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.707+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e240 msgr2=0x7f87841a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e240 0x7f87841a6590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f877800c530 tx=0x7f877800c840 comp rx=0 tx=0).stop 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 shutdown_connections 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f878409e690 0x7f87841a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f878409d930 0x7f87841a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 --2- 172.21.15.32:0/1339410214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409e240 0x7f87841a6590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 >> 172.21.15.32:0/1339410214 conn(0x7f8784099c30 msgr2=0x7f87841bae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 shutdown_connections 2023-09-30T22:24:22.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.708+0000 7f878a6c6700 1 -- 172.21.15.32:0/1339410214 wait complete. 2023-09-30T22:24:22.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 Processor -- start 2023-09-30T22:24:22.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 -- start start 2023-09-30T22:24:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d24139370 0x7f7d2413b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2406b640 con 0x7f7d2409d930 2023-09-30T22:24:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2409a790 con 0x7f7d2409e410 2023-09-30T22:24:22.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d2acac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2409ef00 con 0x7f7d24139370 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d23fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d28a48700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d2409d930 0x7f7d2409dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47640/0 (socket says 172.21.15.32:47640) 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.828+0000 7f7d28a48700 1 -- 172.21.15.32:0/2107040174 learned_addr learned my addr 172.21.15.32:0/2107040174 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d29249700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d24139370 0x7f7d2413b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d237fe700 1 -- 172.21.15.32:0/2107040174 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388816476 0 0) 0x7f7d2406b640 con 0x7f7d2409d930 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d237fe700 1 -- 172.21.15.32:0/2107040174 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0c0035b0 con 0x7f7d2409d930 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d29249700 1 -- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 msgr2=0x7f7d24138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:22.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d29249700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d29249700 1 -- 172.21.15.32:0/2107040174 >> v1:172.21.15.32:6789/0 conn(0x7f7d2409d930 legacy=0x7f7d2409dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d29249700 1 -- 172.21.15.32:0/2107040174 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2409f1a0 con 0x7f7d24139370 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d23fff700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d29249700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d24139370 0x7f7d2413b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009dc0 tx=0x7f7d1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ee2ddf38550671 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d237fe700 1 -- 172.21.15.32:0/2107040174 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00e070 con 0x7f7d24139370 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d237fe700 1 -- 172.21.15.32:0/2107040174 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00bb20 con 0x7f7d24139370 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.829+0000 7f7d237fe700 1 -- 172.21.15.32:0/2107040174 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c012430 con 0x7f7d24139370 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 -- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d24139370 msgr2=0x7f7d2413b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d24139370 0x7f7d2413b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009dc0 tx=0x7f7d1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 -- 172.21.15.32:0/2107040174 shutdown_connections 2023-09-30T22:24:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d24139370 0x7f7d2413b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2107040174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 -- 172.21.15.32:0/2107040174 >> 172.21.15.32:0/2107040174 conn(0x7f7d24099c30 msgr2=0x7f7d240a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 -- 172.21.15.32:0/2107040174 shutdown_connections 2023-09-30T22:24:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.830+0000 7f7d2acac700 1 -- 172.21.15.32:0/2107040174 wait complete. 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.831+0000 7f7d2acac700 1 Processor -- start 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.831+0000 7f7d2acac700 1 -- start start 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 0x7f7d24121980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24121ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d24139370 0x7f7d24122400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24123540 con 0x7f7d24139370 2023-09-30T22:24:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24137820 con 0x7f7d2409e410 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24124be0 con 0x7f7d2409d930 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d28a48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 0x7f7d24121980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d24139370 0x7f7d24122400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d28a48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 0x7f7d24121980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55466/0 (socket says 172.21.15.32:55466) 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d28a48700 1 -- 172.21.15.32:0/2141246620 learned_addr learned my addr 172.21.15.32:0/2141246620 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d23fff700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24121ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 -- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 msgr2=0x7f7d24121980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 0x7f7d24121980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 -- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 msgr2=0x7f7d24121ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24121ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 -- 172.21.15.32:0/2141246620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180094a0 con 0x7f7d24139370 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d28a48700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 0x7f7d24121980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d29249700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d24139370 0x7f7d24122400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c016000 tx=0x7f7d1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c012430 con 0x7f7d24139370 2023-09-30T22:24:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c020070 con 0x7f7d24139370 2023-09-30T22:24:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c009a70 con 0x7f7d24139370 2023-09-30T22:24:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.832+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004280 con 0x7f7d24139370 2023-09-30T22:24:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.833+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2409c830 con 0x7f7d24139370 2023-09-30T22:24:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.833+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7d1c0043e0 con 0x7f7d24139370 2023-09-30T22:24:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.833+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f7d1c022070 con 0x7f7d24139370 2023-09-30T22:24:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.834+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2404f2a0 con 0x7f7d24139370 2023-09-30T22:24:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:22.839+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d1c004690 con 0x7f7d24139370 2023-09-30T22:24:23.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.042+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f7d24071330 con 0x7f7d24139370 2023-09-30T22:24:23.210 INFO:tasks.ceph.mgr.x.smithi032.stderr:terminate called recursively 2023-09-30T22:24:23.211 INFO:tasks.ceph.mgr.x.smithi032.stderr:terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of_buffer' 2023-09-30T22:24:23.211 INFO:tasks.ceph.mgr.x.smithi032.stderr: what(): End of buffer [buffer:2] 2023-09-30T22:24:23.211 INFO:tasks.ceph.mgr.x.smithi032.stderr:*** Caught signal (Aborted) ** 2023-09-30T22:24:23.211 INFO:tasks.ceph.mgr.x.smithi032.stderr: in thread 7f4dfe00a700 thread_name:ms_dispatch 2023-09-30T22:24:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.214+0000 7f7d217fa700 1 -- 172.21.15.32:0/2141246620 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v35) v1 ==== 153+0+0 (secure 0 0 0) 0x7f7d1c02c020 con 0x7f7d24139370 2023-09-30T22:24:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 >> v1:172.21.15.32:6816/126553 conn(0x7f7d1002f510 legacy=0x7f7d100319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d24139370 msgr2=0x7f7d24122400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d24139370 0x7f7d24122400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c016000 tx=0x7f7d1c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 shutdown_connections 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d24139370 0x7f7d24122400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d2409d930 0x7f7d24121980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 --2- 172.21.15.32:0/2141246620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d2409e410 0x7f7d24121ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.217+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 >> 172.21.15.32:0/2141246620 conn(0x7f7d24099c30 msgr2=0x7f7d24070280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.218+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 shutdown_connections 2023-09-30T22:24:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.218+0000 7f7d2acac700 1 -- 172.21.15.32:0/2141246620 wait complete. 2023-09-30T22:24:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 1 pg_autoscale_mode to off 2023-09-30T22:24:23.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.357+0000 7f882be71700 1 Processor -- start 2023-09-30T22:24:23.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882be71700 1 -- start start 2023-09-30T22:24:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882be71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f882409cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882be71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133600 0x7f8824133a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882be71700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88240a1320 con 0x7f8824133f90 2023-09-30T22:24:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882be71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882409f250 con 0x7f8824133600 2023-09-30T22:24:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882be71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8824134400 con 0x7f882409c980 2023-09-30T22:24:23.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882a40e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8824133f90 0x7f882413a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47656/0 (socket says 172.21.15.32:47656) 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882a40e700 1 -- 172.21.15.32:0/1898421118 learned_addr learned my addr 172.21.15.32:0/1898421118 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f882940c700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133600 0x7f8824133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.358+0000 7f8829c0d700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f882409cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8829c0d700 1 -- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133600 msgr2=0x7f8824133a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8829c0d700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133600 0x7f8824133a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8829c0d700 1 -- 172.21.15.32:0/1898421118 >> v1:172.21.15.32:6789/0 conn(0x7f8824133f90 legacy=0x7f882413a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8828c0b700 1 -- 172.21.15.32:0/1898421118 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128802871 0 0) 0x7f88240a1320 con 0x7f8824133f90 2023-09-30T22:24:23.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8829c0d700 1 -- 172.21.15.32:0/1898421118 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882409ebd0 con 0x7f882409c980 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f882940c700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133600 0x7f8824133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8829c0d700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f882409cd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f8820009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d355e8620cd3a87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8828c0b700 1 -- 172.21.15.32:0/1898421118 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011830 con 0x7f882409c980 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8828c0b700 1 -- 172.21.15.32:0/1898421118 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820011990 con 0x7f882409c980 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.359+0000 7f8828c0b700 1 -- 172.21.15.32:0/1898421118 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011d20 con 0x7f882409c980 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 -- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 msgr2=0x7f882409cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:23.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f882409cd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f8820009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 -- 172.21.15.32:0/1898421118 shutdown_connections 2023-09-30T22:24:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f882409cd50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 --2- 172.21.15.32:0/1898421118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133600 0x7f8824133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 -- 172.21.15.32:0/1898421118 >> 172.21.15.32:0/1898421118 conn(0x7f88240a64c0 msgr2=0x7f88240a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 -- 172.21.15.32:0/1898421118 shutdown_connections 2023-09-30T22:24:23.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.360+0000 7f882be71700 1 -- 172.21.15.32:0/1898421118 wait complete. 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.362+0000 7f882be71700 1 Processor -- start 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.362+0000 7f882be71700 1 -- start start 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.362+0000 7f882be71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f88241bb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882be71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 0x7f882406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882be71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 0x7f882406e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882be71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882409e290 con 0x7f8824133600 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882be71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8824071140 con 0x7f8824133f90 2023-09-30T22:24:23.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882be71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f882406fec0 con 0x7f882409c980 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 0x7f882406e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882940c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 0x7f882406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 0x7f882406e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33908/0 (socket says 172.21.15.32:33908) 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f8829c0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f88241bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 -- 172.21.15.32:0/2135915713 learned_addr learned my addr 172.21.15.32:0/2135915713 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882940c700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 0x7f882406dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36028/0 (socket says 172.21.15.32:36028) 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 -- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 msgr2=0x7f88241bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f88241bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 -- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 msgr2=0x7f882406dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 0x7f882406dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 -- 172.21.15.32:0/2135915713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88140094a0 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f8829c0d700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f88241bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882940c700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 0x7f882406dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f882a40e700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 0x7f882406e2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f881c002230 tx=0x7f881c00cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.363+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c00aac0 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.364+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c00ac40 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.364+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88200094a0 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.364+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c0070d0 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.364+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824070aa0 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.364+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f881c007250 con 0x7f8824133f90 2023-09-30T22:24:23.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.365+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f881c03f9d0 con 0x7f8824133f90 2023-09-30T22:24:23.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.365+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88240a5980 con 0x7f8824133f90 2023-09-30T22:24:23.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.370+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f881c018080 con 0x7f8824133f90 2023-09-30T22:24:23.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.485+0000 7f8829c0d700 1 -- 172.21.15.32:0/2135915713 >> v1:172.21.15.32:6816/126553 conn(0x7f880802f510 legacy=0x7f88080319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk reading from fd=16 : -104 Unknown error -104 2023-09-30T22:24:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.485+0000 7f8829c0d700 1 -- 172.21.15.32:0/2135915713 >> v1:172.21.15.32:6816/126553 conn(0x7f880802f510 legacy=0x7f88080319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:24:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.486+0000 7f8829c0d700 1 --1- 172.21.15.32:0/2135915713 >> v1:172.21.15.32:6816/126553 conn(0x7f880802f510 0x7f88080319c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify read banner and identify addresses failed 2023-09-30T22:24:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.486+0000 7f8829c0d700 1 --1- 172.21.15.32:0/2135915713 >> v1:172.21.15.32:6816/126553 conn(0x7f880802f510 0x7f88080319c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).fault on lossy channel, failing 2023-09-30T22:24:23.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.486+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 >> v1:172.21.15.32:6816/126553 conn(0x7f880802f510 legacy=0x7f88080319c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-09-30T22:24:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:23.575+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f88000051e0 con 0x7f8824133f90 2023-09-30T22:24:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.216+0000 7f881a7fc700 1 -- 172.21.15.32:0/2135915713 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v36) v1 ==== 152+0+0 (secure 0 0 0) 0x7f881c007500 con 0x7f8824133f90 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 msgr2=0x7f882406e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 0x7f882406e2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f881c002230 tx=0x7f881c00cf50 comp rx=0 tx=0).stop 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 shutdown_connections 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8824133f90 0x7f882406e2e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f882409c980 0x7f88241bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 --2- 172.21.15.32:0/2135915713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8824133600 0x7f882406dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 >> 172.21.15.32:0/2135915713 conn(0x7f88240a64c0 msgr2=0x7f8824099bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 shutdown_connections 2023-09-30T22:24:24.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.220+0000 7f882be71700 1 -- 172.21.15.32:0/2135915713 wait complete. 2023-09-30T22:24:24.232 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 2 pg_autoscale_mode to off 2023-09-30T22:24:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.347+0000 7fb8582c6700 1 Processor -- start 2023-09-30T22:24:24.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.347+0000 7fb8582c6700 1 -- start start 2023-09-30T22:24:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb8582c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009c910 0x7fb85009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb8582c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 0x7fb85009dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb8582c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 0x7fb85013ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:24.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb8582c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85009ed90 con 0x7fb85009e1c0 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb8582c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500a0880 con 0x7fb85009d850 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb8582c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb85009f200 con 0x7fb85009c910 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb855861700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 0x7fb85009dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb856863700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 0x7fb85013ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb856062700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009c910 0x7fb85009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb855861700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 0x7fb85009dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33924/0 (socket says 172.21.15.32:33924) 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb856863700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 0x7fb85013ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36038/0 (socket says 172.21.15.32:36038) 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.348+0000 7fb855861700 1 -- 172.21.15.32:0/1486569846 learned_addr learned my addr 172.21.15.32:0/1486569846 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856062700 1 -- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 msgr2=0x7fb85009dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:24.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856062700 1 --2- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 0x7fb85009dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856062700 1 -- 172.21.15.32:0/1486569846 >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 msgr2=0x7fb85013ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856062700 1 --2- 172.21.15.32:0/1486569846 >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 0x7fb85013ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856062700 1 -- 172.21.15.32:0/1486569846 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500a05e0 con 0x7fb85009c910 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb855861700 1 --2- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 0x7fb85009dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856863700 1 --2- 172.21.15.32:0/1486569846 >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 0x7fb85013ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb856062700 1 --2- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009c910 0x7fb85009cce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009850 tx=0x7fb84c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8f282120ec8946 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb855060700 1 -- 172.21.15.32:0/1486569846 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c005e30 con 0x7fb85009c910 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb855060700 1 -- 172.21.15.32:0/1486569846 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c006030 con 0x7fb85009c910 2023-09-30T22:24:24.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.349+0000 7fb855060700 1 -- 172.21.15.32:0/1486569846 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0063c0 con 0x7fb85009c910 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 -- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009c910 msgr2=0x7fb85009cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 --2- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009c910 0x7fb85009cce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009850 tx=0x7fb84c009b60 comp rx=0 tx=0).stop 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 -- 172.21.15.32:0/1486569846 shutdown_connections 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 --2- 172.21.15.32:0/1486569846 >> v2:172.21.15.32:3300/0 conn(0x7fb85009e1c0 0x7fb85013ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 --2- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009c910 0x7fb85009cce0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 --2- 172.21.15.32:0/1486569846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009d850 0x7fb85009dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 -- 172.21.15.32:0/1486569846 >> 172.21.15.32:0/1486569846 conn(0x7fb8500a2e50 msgr2=0x7fb8500a3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 -- 172.21.15.32:0/1486569846 shutdown_connections 2023-09-30T22:24:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.350+0000 7fb8582c6700 1 -- 172.21.15.32:0/1486569846 wait complete. 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 Processor -- start 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 -- start start 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009c910 0x7fb8501c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 0x7fb8501c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 0x7fb8501c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85012e4b0 con 0x7fb85009d850 2023-09-30T22:24:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8501c4ca0 con 0x7fb85009c910 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb8582c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8501c4f40 con 0x7fb85009e1c0 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb855861700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 0x7fb8501c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.351+0000 7fb856863700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 0x7fb8501c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856062700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009c910 0x7fb8501c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb855861700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 0x7fb8501c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36040/0 (socket says 172.21.15.32:36040) 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 0x7fb8501c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55514/0 (socket says 172.21.15.32:55514) 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb855861700 1 -- 172.21.15.32:0/2020868631 learned_addr learned my addr 172.21.15.32:0/2020868631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 -- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009c910 msgr2=0x7fb8501c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009c910 0x7fb8501c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 -- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 msgr2=0x7fb8501c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 0x7fb8501c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 -- 172.21.15.32:0/2020868631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009550 con 0x7fb85009e1c0 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb855861700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 0x7fb8501c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb856863700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 0x7fb8501c8210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb84800efc0 tx=0x7fb84800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800ca90 con 0x7fb85009e1c0 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84800f040 con 0x7fb85009e1c0 2023-09-30T22:24:24.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb848009a70 con 0x7fb85009e1c0 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800ce30 con 0x7fb85009e1c0 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.352+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c009430 con 0x7fb85009e1c0 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.353+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb848014480 con 0x7fb85009e1c0 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.353+0000 7fb856062700 1 -- 172.21.15.32:0/2020868631 >> v1:172.21.15.32:6816/126553 conn(0x7fb83402f510 legacy=0x7fb8340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.353+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb8480402c0 con 0x7fb85009e1c0 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.353+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85006baa0 con 0x7fb85009e1c0 2023-09-30T22:24:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.358+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb848018070 con 0x7fb85009e1c0 2023-09-30T22:24:24.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.554+0000 7fb856062700 1 -- 172.21.15.32:0/2020868631 >> v1:172.21.15.32:6816/126553 conn(0x7fb83402f510 legacy=0x7fb8340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.560+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7fb850067ae0 con 0x7fb85009e1c0 2023-09-30T22:24:24.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:24.955+0000 7fb856062700 1 -- 172.21.15.32:0/2020868631 >> v1:172.21.15.32:6816/126553 conn(0x7fb83402f510 legacy=0x7fb8340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.221+0000 7fb846ffd700 1 -- 172.21.15.32:0/2020868631 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 3 pg_autoscale_mode to off v37) v1 ==== 164+0+0 (secure 0 0 0) 0x7fb848012930 con 0x7fb85009e1c0 2023-09-30T22:24:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.224+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 >> v1:172.21.15.32:6816/126553 conn(0x7fb83402f510 legacy=0x7fb8340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.224+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 msgr2=0x7fb8501c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.224+0000 7fb8582c6700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 0x7fb8501c8210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb84800efc0 tx=0x7fb84800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 shutdown_connections 2023-09-30T22:24:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb85009e1c0 0x7fb8501c8210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb85009c910 0x7fb8501c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 --2- 172.21.15.32:0/2020868631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb85009d850 0x7fb8501c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 >> 172.21.15.32:0/2020868631 conn(0x7fb8500a2e50 msgr2=0x7fb8501bade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 shutdown_connections 2023-09-30T22:24:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.225+0000 7fb8582c6700 1 -- 172.21.15.32:0/2020868631 wait complete. 2023-09-30T22:24:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 3 pg_autoscale_mode to off 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.351+0000 7f58e29d0700 1 Processor -- start 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e29d0700 1 -- start start 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e29d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09e9d0 0x7f58dc09eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e29d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 0x7f58dc131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e29d0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc09ab50 con 0x7f58dc09f2e0 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e29d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0a0140 con 0x7f58dc09e9d0 2023-09-30T22:24:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e29d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0a0330 con 0x7f58dc09f6b0 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e0f6d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 0x7f58dc131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e0f6d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 0x7f58dc131ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55524/0 (socket says 172.21.15.32:55524) 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09e9d0 0x7f58dc09eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.352+0000 7f58e0f6d700 1 -- 172.21.15.32:0/976884198 learned_addr learned my addr 172.21.15.32:0/976884198 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58e0f6d700 1 -- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09e9d0 msgr2=0x7f58dc09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58daffd700 1 -- 172.21.15.32:0/976884198 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774391567 0 0) 0x7f58dc09ab50 con 0x7f58dc09f2e0 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58e0f6d700 1 --2- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09e9d0 0x7f58dc09eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58e0f6d700 1 -- 172.21.15.32:0/976884198 >> v1:172.21.15.32:6789/0 conn(0x7f58dc09f2e0 legacy=0x7f58dc1317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58e0f6d700 1 -- 172.21.15.32:0/976884198 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc06ef60 con 0x7f58dc09f6b0 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58dbfff700 1 --2- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09e9d0 0x7f58dc09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58e0f6d700 1 --2- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 0x7f58dc131ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009e80 tx=0x7f58d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b36f42b5cf366c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58daffd700 1 -- 172.21.15.32:0/976884198 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400e070 con 0x7f58dc09f6b0 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58daffd700 1 -- 172.21.15.32:0/976884198 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d400b910 con 0x7f58dc09f6b0 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.353+0000 7f58daffd700 1 -- 172.21.15.32:0/976884198 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400bcc0 con 0x7f58dc09f6b0 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 -- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 msgr2=0x7f58dc131ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 --2- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 0x7f58dc131ec0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009e80 tx=0x7f58d400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 -- 172.21.15.32:0/976884198 shutdown_connections 2023-09-30T22:24:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 --2- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09f6b0 0x7f58dc131ec0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 --2- 172.21.15.32:0/976884198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09e9d0 0x7f58dc09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 -- 172.21.15.32:0/976884198 >> 172.21.15.32:0/976884198 conn(0x7f58dc0a43f0 msgr2=0x7f58dc0a47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 -- 172.21.15.32:0/976884198 shutdown_connections 2023-09-30T22:24:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.354+0000 7f58e29d0700 1 -- 172.21.15.32:0/976884198 wait complete. 2023-09-30T22:24:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 Processor -- start 2023-09-30T22:24:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 -- start start 2023-09-30T22:24:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 0x7f58dc120020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09f2e0 0x7f58dc120560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58dc09f6b0 0x7f58dc120aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:25.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc123b40 con 0x7f58dc09f6b0 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc123630 con 0x7f58dc09f2e0 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e29d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc1232d0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 0x7f58dc120020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58e0f6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58dc09f6b0 0x7f58dc120aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58db7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09f2e0 0x7f58dc120560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58dbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 0x7f58dc120020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55534/0 (socket says 172.21.15.32:55534) 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.355+0000 7f58dbfff700 1 -- 172.21.15.32:0/1858363823 learned_addr learned my addr 172.21.15.32:0/1858363823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58dbfff700 1 -- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09f2e0 msgr2=0x7f58dc120560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58dbfff700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09f2e0 0x7f58dc120560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58dbfff700 1 -- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58dc09f6b0 msgr2=0x7f58dc120aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58dbfff700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58dc09f6b0 0x7f58dc120aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58dbfff700 1 -- 172.21.15.32:0/1858363823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40094a0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58db7fe700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09f2e0 0x7f58dc120560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58e0f6d700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58dc09f6b0 0x7f58dc120aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58dbfff700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 0x7f58dc120020 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58c400ebe0 tx=0x7f58c400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4009760 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c400c850 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d4009b30 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c400cbe0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.356+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc0714a0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.357+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f58c400cd40 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.357+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f58c403ff80 con 0x7f58dc09e9d0 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.357+0000 7f58db7fe700 1 -- 172.21.15.32:0/1858363823 >> v1:172.21.15.32:6816/126553 conn(0x7f58bc02f510 legacy=0x7f58bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.357+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc09a3a0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.362+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58c401b3b0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.558+0000 7f58db7fe700 1 -- 172.21.15.32:0/1858363823 >> v1:172.21.15.32:6816/126553 conn(0x7f58bc02f510 legacy=0x7f58bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:25.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.567+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f58dc04f2a0 con 0x7f58dc09e9d0 2023-09-30T22:24:25.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:25.959+0000 7f58db7fe700 1 -- 172.21.15.32:0/1858363823 >> v1:172.21.15.32:6816/126553 conn(0x7f58bc02f510 legacy=0x7f58bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:26.022 INFO:tasks.ceph.mgr.x.smithi032.stderr:daemon-helper: command crashed with signal 6 2023-09-30T22:24:26.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.223+0000 7f58d8ff9700 1 -- 172.21.15.32:0/1858363823 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 4 pg_autoscale_mode to off v38) v1 ==== 160+0+0 (secure 0 0 0) 0x7f58c4024350 con 0x7f58dc09e9d0 2023-09-30T22:24:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 >> v1:172.21.15.32:6816/126553 conn(0x7f58bc02f510 legacy=0x7f58bc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 msgr2=0x7f58dc120020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 0x7f58dc120020 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58c400ebe0 tx=0x7f58c400eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 shutdown_connections 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58dc09f6b0 0x7f58dc120aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58dc09e9d0 0x7f58dc120020 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58c400ebe0 tx=0x7f58c400eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 --2- 172.21.15.32:0/1858363823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58dc09f2e0 0x7f58dc120560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 >> 172.21.15.32:0/1858363823 conn(0x7f58dc0a43f0 msgr2=0x7f58dc131bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 shutdown_connections 2023-09-30T22:24:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.228+0000 7f58e29d0700 1 -- 172.21.15.32:0/1858363823 wait complete. 2023-09-30T22:24:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:set pool 4 pg_autoscale_mode to off 2023-09-30T22:24:26.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release reef' 2023-09-30T22:24:26.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.384+0000 7f2f346e4700 1 Processor -- start 2023-09-30T22:24:26.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.384+0000 7f2f346e4700 1 -- start start 2023-09-30T22:24:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f346e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f2c09d930 0x7f2f2c09dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f346e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c1307f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f346e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c09b570 con 0x7f2f2c130d30 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f346e4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2c09ed90 con 0x7f2f2c09d930 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f346e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c09a5a0 con 0x7f2f2c09e240 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f31c7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c1307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32480700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f2c09d930 0x7f2f2c09dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f31c7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c1307f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55544/0 (socket says 172.21.15.32:55544) 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32c81700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2f2c130d30 0x7f2f2c137a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47682/0 (socket says 172.21.15.32:47682) 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f31c7f700 1 -- 172.21.15.32:0/1993305846 learned_addr learned my addr 172.21.15.32:0/1993305846 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32480700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f2c09d930 0x7f2f2c09dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36058/0 (socket says 172.21.15.32:36058) 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32480700 1 -- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 msgr2=0x7f2f2c1307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32480700 1 --2- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c1307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32480700 1 -- 172.21.15.32:0/1993305846 >> v1:172.21.15.32:6789/0 conn(0x7f2f2c130d30 legacy=0x7f2f2c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f32480700 1 -- 172.21.15.32:0/1993305846 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c09ac20 con 0x7f2f2c09d930 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f3147e700 1 -- 172.21.15.32:0/1993305846 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961385657 0 0) 0x7f2f2c09b570 con 0x7f2f2c130d30 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.385+0000 7f2f31c7f700 1 --2- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c1307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f32480700 1 --2- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c09dd00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009840 tx=0x7f2f28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3173926070c2f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f3147e700 1 -- 172.21.15.32:0/1993305846 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011830 con 0x7f2f2c09d930 2023-09-30T22:24:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f3147e700 1 -- 172.21.15.32:0/1993305846 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28011990 con 0x7f2f2c09d930 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f3147e700 1 -- 172.21.15.32:0/1993305846 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011d20 con 0x7f2f2c09d930 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 -- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 msgr2=0x7f2f2c09dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 --2- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c09dd00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009840 tx=0x7f2f28009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 -- 172.21.15.32:0/1993305846 shutdown_connections 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 --2- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c09dd00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 --2- 172.21.15.32:0/1993305846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c1307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 -- 172.21.15.32:0/1993305846 >> 172.21.15.32:0/1993305846 conn(0x7f2f2c099c30 msgr2=0x7f2f2c0a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 -- 172.21.15.32:0/1993305846 shutdown_connections 2023-09-30T22:24:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.386+0000 7f2f346e4700 1 -- 172.21.15.32:0/1993305846 wait complete. 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.387+0000 7f2f346e4700 1 Processor -- start 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.387+0000 7f2f346e4700 1 -- start start 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f346e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c11e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f346e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c11e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f346e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f2c130d30 0x7f2f2c11ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f346e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c121e80 con 0x7f2f2c09d930 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f346e4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0a2650 con 0x7f2f2c130d30 2023-09-30T22:24:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f346e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c11ff70 con 0x7f2f2c09e240 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c11e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c11e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55558/0 (socket says 172.21.15.32:55558) 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 -- 172.21.15.32:0/3052860133 learned_addr learned my addr 172.21.15.32:0/3052860133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f32c81700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f2c130d30 0x7f2f2c11ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f32480700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c11e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 -- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f2c130d30 msgr2=0x7f2f2c11ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f2c130d30 0x7f2f2c11ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 -- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 msgr2=0x7f2f2c11e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c11e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f31c7f700 1 -- 172.21.15.32:0/3052860133 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c00b7f0 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.388+0000 7f2f32c81700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f2c130d30 0x7f2f2c11ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.389+0000 7f2f31c7f700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c11e8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c010fa0 tx=0x7f2f1c00e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.389+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00eea0 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.389+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c011650 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.389+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0119c0 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.389+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f280094a0 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.389+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c09c980 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.390+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2f1c011b20 con 0x7f2f2c09e240 2023-09-30T22:24:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.390+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2f1c0428e0 con 0x7f2f2c09e240 2023-09-30T22:24:26.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.390+0000 7f2f32480700 1 -- 172.21.15.32:0/3052860133 >> v1:172.21.15.32:6816/126553 conn(0x7f2f1002f510 legacy=0x7f2f100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:26.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.390+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c04f2a0 con 0x7f2f2c09e240 2023-09-30T22:24:26.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.395+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f1c01e080 con 0x7f2f2c09e240 2023-09-30T22:24:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.591+0000 7f2f32480700 1 -- 172.21.15.32:0/3052860133 >> v1:172.21.15.32:6816/126553 conn(0x7f2f1002f510 legacy=0x7f2f100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:26.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.611+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd require-osd-release", "release": "reef"} v 0) v1 -- 0x7f2f2c0727b0 con 0x7f2f2c09e240 2023-09-30T22:24:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:26.992+0000 7f2f32480700 1 -- 172.21.15.32:0/3052860133 >> v1:172.21.15.32:6816/126553 conn(0x7f2f1002f510 legacy=0x7f2f100319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.233+0000 7f2f22ffd700 1 -- 172.21.15.32:0/3052860133 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "reef"}]=0 v39) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2f1c019440 con 0x7f2f2c09e240 2023-09-30T22:24:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.237+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 >> v1:172.21.15.32:6816/126553 conn(0x7f2f1002f510 legacy=0x7f2f100319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.237+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 msgr2=0x7f2f2c11e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.237+0000 7f2f346e4700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c11e8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c010fa0 tx=0x7f2f1c00e640 comp rx=0 tx=0).stop 2023-09-30T22:24:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 shutdown_connections 2023-09-30T22:24:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f2c130d30 0x7f2f2c11ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f2c09d930 0x7f2f2c11e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 --2- 172.21.15.32:0/3052860133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f2c09e240 0x7f2f2c11e8f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 >> 172.21.15.32:0/3052860133 conn(0x7f2f2c099c30 msgr2=0x7f2f2c0a0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 shutdown_connections 2023-09-30T22:24:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.238+0000 7f2f346e4700 1 -- 172.21.15.32:0/3052860133 wait complete. 2023-09-30T22:24:27.253 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-09-30T22:24:27.265 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:24:27.265 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:24:27.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:24:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.408+0000 7f9abfdfa700 1 Processor -- start 2023-09-30T22:24:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.408+0000 7f9abfdfa700 1 -- start start 2023-09-30T22:24:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abfdfa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 0x7f9ab8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abfdfa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab8135e50 0x7f9ab8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abfdfa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab809f130 con 0x7f9ab809d6f0 2023-09-30T22:24:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abfdfa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab809f520 con 0x7f9ab809e1d0 2023-09-30T22:24:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abfdfa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab809e930 con 0x7f9ab8135e50 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abd395700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 0x7f9ab8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abdb96700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ab809d6f0 0x7f9ab809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47686/0 (socket says 172.21.15.32:47686) 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abd395700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 0x7f9ab8135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36066/0 (socket says 172.21.15.32:36066) 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abdb96700 1 -- 172.21.15.32:0/1171884921 learned_addr learned my addr 172.21.15.32:0/1171884921 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abe397700 1 --2- 172.21.15.32:0/1171884921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab8135e50 0x7f9ab8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abe397700 1 -- 172.21.15.32:0/1171884921 >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 msgr2=0x7f9ab8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abe397700 1 --2- 172.21.15.32:0/1171884921 >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 0x7f9ab8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abe397700 1 -- 172.21.15.32:0/1171884921 >> v1:172.21.15.32:6789/0 conn(0x7f9ab809d6f0 legacy=0x7f9ab809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abe397700 1 -- 172.21.15.32:0/1171884921 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab809b2d0 con 0x7f9ab8135e50 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abd395700 1 --2- 172.21.15.32:0/1171884921 >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 0x7f9ab8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abcb94700 1 -- 172.21.15.32:0/1171884921 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202491322 0 0) 0x7f9ab809f130 con 0x7f9ab809d6f0 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abe397700 1 --2- 172.21.15.32:0/1171884921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab8135e50 0x7f9ab8138230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009d90 tx=0x7f9ab000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9747166e1b092c83 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abcb94700 1 -- 172.21.15.32:0/1171884921 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000e070 con 0x7f9ab8135e50 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.409+0000 7f9abcb94700 1 -- 172.21.15.32:0/1171884921 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab000bb20 con 0x7f9ab8135e50 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abcb94700 1 -- 172.21.15.32:0/1171884921 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0012430 con 0x7f9ab8135e50 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 -- 172.21.15.32:0/1171884921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab8135e50 msgr2=0x7f9ab8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/1171884921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab8135e50 0x7f9ab8138230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009d90 tx=0x7f9ab000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 -- 172.21.15.32:0/1171884921 shutdown_connections 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/1171884921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab8135e50 0x7f9ab8138230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/1171884921 >> v2:172.21.15.32:3300/0 conn(0x7f9ab809e1d0 0x7f9ab8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 -- 172.21.15.32:0/1171884921 >> 172.21.15.32:0/1171884921 conn(0x7f9ab8066f50 msgr2=0x7f9ab809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 -- 172.21.15.32:0/1171884921 shutdown_connections 2023-09-30T22:24:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.410+0000 7f9abfdfa700 1 -- 172.21.15.32:0/1171884921 wait complete. 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 Processor -- start 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 -- start start 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab809d6f0 0x7f9ab811c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 0x7f9ab811c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ab8135e50 0x7f9ab811cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab811f350 con 0x7f9ab809d6f0 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8120960 con 0x7f9ab8135e50 2023-09-30T22:24:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abfdfa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab811e090 con 0x7f9ab809e1d0 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abd395700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 0x7f9ab811c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abdb96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab809d6f0 0x7f9ab811c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abe397700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ab8135e50 0x7f9ab811cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.412+0000 7f9abd395700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 0x7f9ab811c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55578/0 (socket says 172.21.15.32:55578) 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abd395700 1 -- 172.21.15.32:0/87810820 learned_addr learned my addr 172.21.15.32:0/87810820 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abdb96700 1 -- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 msgr2=0x7f9ab811c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abdb96700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 0x7f9ab811c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abdb96700 1 -- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ab8135e50 msgr2=0x7f9ab811cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abdb96700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ab8135e50 0x7f9ab811cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abdb96700 1 -- 172.21.15.32:0/87810820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80094a0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abd395700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 0x7f9ab811c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abdb96700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab809d6f0 0x7f9ab811c480 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400c3f0 tx=0x7f9ab400c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4004a70 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4004c10 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab0009a70 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40070d0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.413+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab809c700 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.414+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9ab4007230 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.414+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f9ab403f5e0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.414+0000 7f9abd395700 1 -- 172.21.15.32:0/87810820 >> v1:172.21.15.32:6816/126553 conn(0x7f9a9c02f510 legacy=0x7f9a9c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:27.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.414+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab811e8f0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.419+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ab40010e0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.615+0000 7f9abd395700 1 -- 172.21.15.32:0/87810820 >> v1:172.21.15.32:6816/126553 conn(0x7f9a9c02f510 legacy=0x7f9a9c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:27.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.685+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9ab8074ea0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.687+0000 7f9aae7fc700 1 -- 172.21.15.32:0/87810820 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v9) v1 ==== 74+0+70483 (secure 0 0 0) 0x7f9ab401b7b0 con 0x7f9ab809d6f0 2023-09-30T22:24:27.688 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:27.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 >> v1:172.21.15.32:6816/126553 conn(0x7f9a9c02f510 legacy=0x7f9a9c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab809d6f0 msgr2=0x7f9ab811c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab809d6f0 0x7f9ab811c480 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400c3f0 tx=0x7f9ab400c700 comp rx=0 tx=0).stop 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 shutdown_connections 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ab8135e50 0x7f9ab811cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab809d6f0 0x7f9ab811c480 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 --2- 172.21.15.32:0/87810820 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ab809e1d0 0x7f9ab811c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.689+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 >> 172.21.15.32:0/87810820 conn(0x7f9ab8066f50 msgr2=0x7f9ab81368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.690+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 shutdown_connections 2023-09-30T22:24:27.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.690+0000 7f9abfdfa700 1 -- 172.21.15.32:0/87810820 wait complete. 2023-09-30T22:24:27.703 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"active_gid":14100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6816","nonce":126553}]},"active_addr":"172.21.15.32:6816/126553","active_change":"2023-09-30T22:23:54.019456+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":21,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1488807751}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2265333326}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1998798597}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2199662186}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":338848985}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3143218070}]}]} 2023-09-30T22:24:27.705 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:24:27.705 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:24:27.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.842+0000 7f629dc1c700 1 Processor -- start 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.842+0000 7f629dc1c700 1 -- start start 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f629dc1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 0x7f629806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f629dc1c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f629806b080 0x7f6298131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f629dc1c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629809ece0 con 0x7f6298131b40 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f629dc1c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629809e760 con 0x7f629806b080 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f629dc1c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629809e040 con 0x7f629806a770 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 0x7f629806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f6296ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f629806b080 0x7f6298131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 0x7f629806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33968/0 (socket says 172.21.15.32:33968) 2023-09-30T22:24:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 -- 172.21.15.32:0/143140684 learned_addr learned my addr 172.21.15.32:0/143140684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f6297fff700 1 --1- 172.21.15.32:0/143140684 >> v1:172.21.15.32:6789/0 conn(0x7f6298131b40 0x7f6298138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47702/0 (socket says 172.21.15.32:47702) 2023-09-30T22:24:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 -- 172.21.15.32:0/143140684 >> v2:172.21.15.32:3300/0 conn(0x7f629806b080 msgr2=0x7f6298131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:27.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 --2- 172.21.15.32:0/143140684 >> v2:172.21.15.32:3300/0 conn(0x7f629806b080 0x7f6298131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 -- 172.21.15.32:0/143140684 >> v1:172.21.15.32:6789/0 conn(0x7f6298131b40 legacy=0x7f6298138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62967fc700 1 -- 172.21.15.32:0/143140684 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43689453 0 0) 0x7f629809ece0 con 0x7f6298131b40 2023-09-30T22:24:27.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f62977fe700 1 -- 172.21.15.32:0/143140684 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629809f630 con 0x7f629806a770 2023-09-30T22:24:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.843+0000 7f6296ffd700 1 --2- 172.21.15.32:0/143140684 >> v2:172.21.15.32:3300/0 conn(0x7f629806b080 0x7f6298131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f62977fe700 1 --2- 172.21.15.32:0/143140684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 0x7f629806ab40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62880098f0 tx=0x7f6288009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849a4425c57936e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f62967fc700 1 -- 172.21.15.32:0/143140684 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288011830 con 0x7f629806a770 2023-09-30T22:24:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f62967fc700 1 -- 172.21.15.32:0/143140684 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288011990 con 0x7f629806a770 2023-09-30T22:24:27.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f62967fc700 1 -- 172.21.15.32:0/143140684 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288011d20 con 0x7f629806a770 2023-09-30T22:24:27.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 -- 172.21.15.32:0/143140684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 msgr2=0x7f629806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 --2- 172.21.15.32:0/143140684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 0x7f629806ab40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62880098f0 tx=0x7f6288009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:27.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 -- 172.21.15.32:0/143140684 shutdown_connections 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 --2- 172.21.15.32:0/143140684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f629806a770 0x7f629806ab40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 --2- 172.21.15.32:0/143140684 >> v2:172.21.15.32:3300/0 conn(0x7f629806b080 0x7f6298131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 -- 172.21.15.32:0/143140684 >> 172.21.15.32:0/143140684 conn(0x7f6298066f50 msgr2=0x7f629809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 -- 172.21.15.32:0/143140684 shutdown_connections 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.844+0000 7f629dc1c700 1 -- 172.21.15.32:0/143140684 wait complete. 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.845+0000 7f629dc1c700 1 Processor -- start 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 -- start start 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f629806a770 0x7f62981c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 0x7f62981c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 0x7f629811d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62981200d0 con 0x7f629806b080 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62981bcef0 con 0x7f6298131b40 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f629dc1c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62981c8e70 con 0x7f629806a770 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f6296ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 0x7f62981c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f6297fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 0x7f629811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f62977fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f629806a770 0x7f62981c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f6296ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 0x7f62981c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36104/0 (socket says 172.21.15.32:36104) 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f6296ffd700 1 -- 172.21.15.32:0/3930256390 learned_addr learned my addr 172.21.15.32:0/3930256390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.846+0000 7f6297fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 0x7f629811d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33976/0 (socket says 172.21.15.32:33976) 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6296ffd700 1 -- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f629806a770 msgr2=0x7f62981c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6296ffd700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f629806a770 0x7f62981c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6296ffd700 1 -- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 msgr2=0x7f629811d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6296ffd700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 0x7f629811d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6296ffd700 1 -- 172.21.15.32:0/3930256390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0094a0 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6297fff700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 0x7f629811d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f6296ffd700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 0x7f62981c8930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f628c00ebe0 tx=0x7f628c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c00c850 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c00c9b0 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c00cd40 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6288009550 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.847+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629809a3d0 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.848+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f628c013470 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.848+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f628c03fc10 con 0x7f629806b080 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.848+0000 7f62977fe700 1 -- 172.21.15.32:0/3930256390 >> v1:172.21.15.32:6816/126553 conn(0x7f628402f540 legacy=0x7f62840319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.848+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298071e60 con 0x7f629806b080 2023-09-30T22:24:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:27.853+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f628c021080 con 0x7f629806b080 2023-09-30T22:24:28.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.049+0000 7f62977fe700 1 -- 172.21.15.32:0/3930256390 >> v1:172.21.15.32:6816/126553 conn(0x7f628402f540 legacy=0x7f62840319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.057+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f629811fce0 con 0x7f629806b080 2023-09-30T22:24:28.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.058+0000 7f627ffff700 1 -- 172.21.15.32:0/3930256390 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12964 (secure 0 0 0) 0x7f629811fce0 con 0x7f629806b080 2023-09-30T22:24:28.059 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:28.059 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:27.227058+0000","last_up_change":"2023-09-30T22:24:14.089689+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":30,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":129367}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":129367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":129367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":129367}]},"public_addr":"172.21.15.32:6804/129367","cluster_addr":"172.21.15.32:6805/129367","heartbeat_back_addr":"172.21.15.32:6807/129367","heartbeat_front_addr":"172.21.15.32:6806/129367","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":129477}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":129477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":129477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":129477}]},"public_addr":"172.21.15.32:6808/129477","cluster_addr":"172.21.15.32:6809/129477","heartbeat_back_addr":"172.21.15.32:6811/129477","heartbeat_front_addr":"172.21.15.32:6810/129477","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":129817}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":129817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":129817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":129817}]},"public_addr":"172.21.15.32:6812/129817","cluster_addr":"172.21.15.32:6813/129817","heartbeat_back_addr":"172.21.15.32:6815/129817","heartbeat_front_addr":"172.21.15.32:6814/129817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6817/3095020018":"2023-10-01T22:24:21.192854+0000","172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 >> v1:172.21.15.32:6816/126553 conn(0x7f628402f540 legacy=0x7f62840319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 msgr2=0x7f62981c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 0x7f62981c8930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f628c00ebe0 tx=0x7f628c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 shutdown_connections 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6298131b40 0x7f629811d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f629806a770 0x7f62981c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 --2- 172.21.15.32:0/3930256390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f629806b080 0x7f62981c8930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 >> 172.21.15.32:0/3930256390 conn(0x7f6298066f50 msgr2=0x7f62980708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 shutdown_connections 2023-09-30T22:24:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.061+0000 7f629dc1c700 1 -- 172.21.15.32:0/3930256390 wait complete. 2023-09-30T22:24:28.075 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:24:28.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:24:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.212+0000 7ff88c775700 1 Processor -- start 2023-09-30T22:24:28.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.212+0000 7ff88c775700 1 -- start start 2023-09-30T22:24:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88c775700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff88409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88c775700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88c775700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88409b120 con 0x7ff88409e000 2023-09-30T22:24:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88c775700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88406b460 con 0x7ff88409d6f0 2023-09-30T22:24:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88c775700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff88409a7e0 con 0x7ff88409e410 2023-09-30T22:24:28.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff889d10700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff88409e000 0x7ff8841339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47706/0 (socket says 172.21.15.32:47706) 2023-09-30T22:24:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88a511700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff88409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88ad12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff889d10700 1 -- 172.21.15.32:0/2404534970 learned_addr learned my addr 172.21.15.32:0/2404534970 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88ad12700 1 -- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 msgr2=0x7ff88409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88ad12700 1 --2- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff88409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88ad12700 1 -- 172.21.15.32:0/2404534970 >> v1:172.21.15.32:6789/0 conn(0x7ff88409e000 legacy=0x7ff8841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88950f700 1 -- 172.21.15.32:0/2404534970 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235866768 0 0) 0x7ff88409b120 con 0x7ff88409e000 2023-09-30T22:24:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88ad12700 1 -- 172.21.15.32:0/2404534970 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88409a500 con 0x7ff88409e410 2023-09-30T22:24:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88a511700 1 --2- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff88409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88ad12700 1 --2- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884134100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009df0 tx=0x7ff87c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794fed98f7b1bde8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88950f700 1 -- 172.21.15.32:0/2404534970 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00e070 con 0x7ff88409e410 2023-09-30T22:24:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88950f700 1 -- 172.21.15.32:0/2404534970 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c00bae0 con 0x7ff88409e410 2023-09-30T22:24:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.213+0000 7ff88950f700 1 -- 172.21.15.32:0/2404534970 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00be90 con 0x7ff88409e410 2023-09-30T22:24:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 -- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 msgr2=0x7ff884134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 --2- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884134100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009df0 tx=0x7ff87c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 -- 172.21.15.32:0/2404534970 shutdown_connections 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 --2- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884134100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 --2- 172.21.15.32:0/2404534970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff88409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 -- 172.21.15.32:0/2404534970 >> 172.21.15.32:0/2404534970 conn(0x7ff884066f50 msgr2=0x7ff88409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 -- 172.21.15.32:0/2404534970 shutdown_connections 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.214+0000 7ff88c775700 1 -- 172.21.15.32:0/2404534970 wait complete. 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.215+0000 7ff88c775700 1 Processor -- start 2023-09-30T22:24:28.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.215+0000 7ff88c775700 1 -- start start 2023-09-30T22:24:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88c775700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff884122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88c775700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88409e000 0x7ff884122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88c775700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88c775700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841237f0 con 0x7ff88409e000 2023-09-30T22:24:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88c775700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff884124e00 con 0x7ff88409d6f0 2023-09-30T22:24:28.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88c775700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff884125e40 con 0x7ff88409e410 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff884122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff884122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33986/0 (socket says 172.21.15.32:33986) 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 -- 172.21.15.32:0/2534076531 learned_addr learned my addr 172.21.15.32:0/2534076531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88ad12700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff889d10700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88409e000 0x7ff884122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 -- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 msgr2=0x7ff884122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 -- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88409e000 msgr2=0x7ff884122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88409e000 0x7ff884122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 -- 172.21.15.32:0/2534076531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780094a0 con 0x7ff88409d6f0 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88ad12700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff889d10700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88409e000 0x7ff884122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff88a511700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff884122320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff87800ec10 tx=0x7ff87800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.216+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800c850 con 0x7ff88409d6f0 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.217+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87800c9b0 con 0x7ff88409d6f0 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.217+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c009a70 con 0x7ff88409d6f0 2023-09-30T22:24:28.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.217+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800cd20 con 0x7ff88409d6f0 2023-09-30T22:24:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.217+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8840a2020 con 0x7ff88409d6f0 2023-09-30T22:24:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.217+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff878013470 con 0x7ff88409d6f0 2023-09-30T22:24:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.218+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7ff87803fce0 con 0x7ff88409d6f0 2023-09-30T22:24:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.218+0000 7ff889d10700 1 -- 172.21.15.32:0/2534076531 >> v1:172.21.15.32:6816/126553 conn(0x7ff86802f510 legacy=0x7ff8680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.218+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88411f030 con 0x7ff88409d6f0 2023-09-30T22:24:28.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.223+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff878013720 con 0x7ff88409d6f0 2023-09-30T22:24:28.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.419+0000 7ff889d10700 1 -- 172.21.15.32:0/2534076531 >> v1:172.21.15.32:6816/126553 conn(0x7ff86802f510 legacy=0x7ff8680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.425+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff884074ea0 con 0x7ff88409d6f0 2023-09-30T22:24:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.426+0000 7ff876ffd700 1 -- 172.21.15.32:0/2534076531 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12964 (secure 0 0 0) 0x7ff87801bba0 con 0x7ff88409d6f0 2023-09-30T22:24:28.426 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:24:28.426 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:27.227058+0000","last_up_change":"2023-09-30T22:24:14.089689+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":30,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":129367}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":129367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":129367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":129367}]},"public_addr":"172.21.15.32:6804/129367","cluster_addr":"172.21.15.32:6805/129367","heartbeat_back_addr":"172.21.15.32:6807/129367","heartbeat_front_addr":"172.21.15.32:6806/129367","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":129477}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":129477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":129477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":129477}]},"public_addr":"172.21.15.32:6808/129477","cluster_addr":"172.21.15.32:6809/129477","heartbeat_back_addr":"172.21.15.32:6811/129477","heartbeat_front_addr":"172.21.15.32:6810/129477","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":129817}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":129817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":129817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":129817}]},"public_addr":"172.21.15.32:6812/129817","cluster_addr":"172.21.15.32:6813/129817","heartbeat_back_addr":"172.21.15.32:6815/129817","heartbeat_front_addr":"172.21.15.32:6814/129817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6817/3095020018":"2023-10-01T22:24:21.192854+0000","172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 >> v1:172.21.15.32:6816/126553 conn(0x7ff86802f510 legacy=0x7ff8680319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 msgr2=0x7ff884122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff884122320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff87800ec10 tx=0x7ff87800ef20 comp rx=0 tx=0).stop 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 shutdown_connections 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff88409e410 0x7ff884122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff88409d6f0 0x7ff884122320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 --2- 172.21.15.32:0/2534076531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88409e000 0x7ff884122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 >> 172.21.15.32:0/2534076531 conn(0x7ff884066f50 msgr2=0x7ff884133de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 shutdown_connections 2023-09-30T22:24:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.429+0000 7ff88c775700 1 -- 172.21.15.32:0/2534076531 wait complete. 2023-09-30T22:24:28.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:24:28.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:24:28.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:24:28.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:24:28.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.591+0000 7ffa47910700 1 Processor -- start 2023-09-30T22:24:28.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.591+0000 7f0e01933700 1 Processor -- start 2023-09-30T22:24:28.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.591+0000 7f0e01933700 1 -- start start 2023-09-30T22:24:28.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0e01933700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc1338a0 0x7f0dfc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0e01933700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 0x7f0dfc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.591+0000 7ffa47910700 1 -- start start 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa47910700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa4006c0e0 0x7ffa4009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa47910700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa40134ec0 0x7ffa401379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa47910700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40137f20 0x7ffa401383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa47910700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4006c4b0 con 0x7ffa40134ec0 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa47910700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa400a09c0 con 0x7ffa4006c0e0 2023-09-30T22:24:28.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa47910700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40134980 con 0x7ffa40137f20 2023-09-30T22:24:28.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0e01933700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc06ef30 con 0x7f0dfc09e760 2023-09-30T22:24:28.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0e01933700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc06fef0 con 0x7f0dfc09de50 2023-09-30T22:24:28.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0e01933700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc1334b0 con 0x7f0dfc1338a0 2023-09-30T22:24:28.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0dfa7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 0x7f0dfc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa456ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa4006c0e0 0x7ffa4009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa456ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa4006c0e0 0x7ffa4009ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34004/0 (socket says 172.21.15.32:34004) 2023-09-30T22:24:28.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa456ac700 1 -- 172.21.15.32:0/2536585779 learned_addr learned my addr 172.21.15.32:0/2536585779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa45ead700 1 --2- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40137f20 0x7ffa401383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0dfa7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 0x7f0dfc09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34002/0 (socket says 172.21.15.32:34002) 2023-09-30T22:24:28.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0dfa7fc700 1 -- 172.21.15.32:0/2900832633 learned_addr learned my addr 172.21.15.32:0/2900832633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa45ead700 1 -- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa4006c0e0 msgr2=0x7ffa4009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa45ead700 1 --2- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa4006c0e0 0x7ffa4009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa45ead700 1 -- 172.21.15.32:0/2536585779 >> v2:172.21.15.32:3300/0 conn(0x7ffa40134ec0 msgr2=0x7ffa401379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa45ead700 1 --2- 172.21.15.32:0/2536585779 >> v2:172.21.15.32:3300/0 conn(0x7ffa40134ec0 0x7ffa401379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7ffa45ead700 1 -- 172.21.15.32:0/2536585779 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400a0600 con 0x7ffa40137f20 2023-09-30T22:24:28.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.592+0000 7f0dfaffd700 1 --2- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc1338a0 0x7f0dfc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0dfa7fc700 1 -- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc1338a0 msgr2=0x7f0dfc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0dfa7fc700 1 --2- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc1338a0 0x7f0dfc133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0dfa7fc700 1 -- 172.21.15.32:0/2900832633 >> v1:172.21.15.32:6789/0 conn(0x7f0dfc09e760 legacy=0x7f0dfc13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0df9ffb700 1 -- 172.21.15.32:0/2900832633 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906533002 0 0) 0x7f0dfc06ef30 con 0x7f0dfc09e760 2023-09-30T22:24:28.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0dfa7fc700 1 -- 172.21.15.32:0/2900832633 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc06fc30 con 0x7f0dfc09de50 2023-09-30T22:24:28.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7ffa45ead700 1 --2- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40137f20 0x7ffa401383c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009d90 tx=0x7ffa3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af3ab89db398df7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0dfa7fc700 1 --2- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 0x7f0dfc09e220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009920 tx=0x7f0df0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6adc80c5bad3a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0df9ffb700 1 -- 172.21.15.32:0/2900832633 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0011830 con 0x7f0dfc09de50 2023-09-30T22:24:28.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0df9ffb700 1 -- 172.21.15.32:0/2900832633 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df0011990 con 0x7f0dfc09de50 2023-09-30T22:24:28.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7f0df9ffb700 1 -- 172.21.15.32:0/2900832633 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0011d20 con 0x7f0dfc09de50 2023-09-30T22:24:28.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7ffa37fff700 1 -- 172.21.15.32:0/2536585779 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800e070 con 0x7ffa40137f20 2023-09-30T22:24:28.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7ffa37fff700 1 -- 172.21.15.32:0/2536585779 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3800bbd0 con 0x7ffa40137f20 2023-09-30T22:24:28.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.593+0000 7ffa37fff700 1 -- 172.21.15.32:0/2536585779 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38012430 con 0x7ffa40137f20 2023-09-30T22:24:28.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 -- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 msgr2=0x7f0dfc09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 --2- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 0x7f0dfc09e220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009920 tx=0x7f0df0009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:28.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 -- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40137f20 msgr2=0x7ffa401383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 --2- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40137f20 0x7ffa401383c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009d90 tx=0x7ffa3800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:28.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 -- 172.21.15.32:0/2900832633 shutdown_connections 2023-09-30T22:24:28.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 --2- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc1338a0 0x7f0dfc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 --2- 172.21.15.32:0/2900832633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc09de50 0x7f0dfc09e220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 -- 172.21.15.32:0/2900832633 >> 172.21.15.32:0/2900832633 conn(0x7f0dfc0a2aa0 msgr2=0x7f0dfc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 -- 172.21.15.32:0/2536585779 shutdown_connections 2023-09-30T22:24:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 --2- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa4006c0e0 0x7ffa4009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 -- 172.21.15.32:0/2900832633 shutdown_connections 2023-09-30T22:24:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 --2- 172.21.15.32:0/2536585779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40137f20 0x7ffa401383c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 --2- 172.21.15.32:0/2536585779 >> v2:172.21.15.32:3300/0 conn(0x7ffa40134ec0 0x7ffa401379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7ffa47910700 1 -- 172.21.15.32:0/2536585779 >> 172.21.15.32:0/2536585779 conn(0x7ffa400a4210 msgr2=0x7ffa400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.594+0000 7f0e01933700 1 -- 172.21.15.32:0/2900832633 wait complete. 2023-09-30T22:24:28.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.595+0000 7ffa47910700 1 -- 172.21.15.32:0/2536585779 shutdown_connections 2023-09-30T22:24:28.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.596+0000 7f0e01933700 1 Processor -- start 2023-09-30T22:24:28.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.596+0000 7ffa47910700 1 -- 172.21.15.32:0/2536585779 wait complete. 2023-09-30T22:24:28.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.596+0000 7f0e01933700 1 -- start start 2023-09-30T22:24:28.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0e01933700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dfc09de50 0x7f0dfc11ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0e01933700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc09e760 0x7f0dfc11f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0e01933700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc1338a0 0x7f0dfc11f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0e01933700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc123110 con 0x7f0dfc09de50 2023-09-30T22:24:28.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0e01933700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc121f90 con 0x7f0dfc1338a0 2023-09-30T22:24:28.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0e01933700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0a0f40 con 0x7f0dfc09e760 2023-09-30T22:24:28.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7ffa47910700 1 Processor -- start 2023-09-30T22:24:28.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7ffa47910700 1 -- start start 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dfc09de50 0x7f0dfc11ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dfc09de50 0x7f0dfc11ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36130/0 (socket says 172.21.15.32:36130) 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 -- 172.21.15.32:0/482255494 learned_addr learned my addr 172.21.15.32:0/482255494 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfa7fc700 1 --2- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc09e760 0x7f0dfc11f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 -- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc09e760 msgr2=0x7f0dfc11f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 --2- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0dfc09e760 0x7f0dfc11f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 -- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc1338a0 msgr2=0x7f0dfc11f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.597+0000 7f0dfaffd700 1 --2- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0dfc1338a0 0x7f0dfc11f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7f0dfaffd700 1 -- 172.21.15.32:0/482255494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0009550 con 0x7f0dfc09de50 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa47910700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4006c0e0 0x7ffa40133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa47910700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40134ec0 0x7ffa4011a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa47910700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa40137f20 0x7ffa4011ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa47910700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4011e280 con 0x7ffa4006c0e0 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa47910700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4011d8c0 con 0x7ffa40137f20 2023-09-30T22:24:28.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa47910700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4013e800 con 0x7ffa40134ec0 2023-09-30T22:24:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7f0dfaffd700 1 --2- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dfc09de50 0x7f0dfc11ead0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e9b0 tx=0x7f0dec00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa456ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4006c0e0 0x7ffa40133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa44eab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40134ec0 0x7ffa4011a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa45ead700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa40137f20 0x7ffa4011ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa456ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4006c0e0 0x7ffa40133df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36144/0 (socket says 172.21.15.32:36144) 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa456ac700 1 -- 172.21.15.32:0/3174160979 learned_addr learned my addr 172.21.15.32:0/3174160979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa44eab700 1 -- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa40137f20 msgr2=0x7ffa4011ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa44eab700 1 --2- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa40137f20 0x7ffa4011ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa44eab700 1 -- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4006c0e0 msgr2=0x7ffa40133df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa44eab700 1 --2- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4006c0e0 0x7ffa40133df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa44eab700 1 -- 172.21.15.32:0/3174160979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38009a70 con 0x7ffa40134ec0 2023-09-30T22:24:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec009710 con 0x7f0dfc09de50 2023-09-30T22:24:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec00c850 con 0x7f0dfc09de50 2023-09-30T22:24:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc06f640 con 0x7f0dfc09de50 2023-09-30T22:24:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.598+0000 7ffa456ac700 1 --2- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa4006c0e0 0x7ffa40133df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00cbc0 con 0x7f0dfc09de50 2023-09-30T22:24:28.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc121c60 con 0x7f0dfc09de50 2023-09-30T22:24:28.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7ffa44eab700 1 --2- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40134ec0 0x7ffa4011a710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa4006f500 tx=0x7ffa3000b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011830 con 0x7ffa40134ec0 2023-09-30T22:24:28.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa30011990 con 0x7ffa40134ec0 2023-09-30T22:24:28.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30011d40 con 0x7ffa40134ec0 2023-09-30T22:24:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4006d210 con 0x7ffa40134ec0 2023-09-30T22:24:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.600+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0dec00cd20 con 0x7f0dfc09de50 2023-09-30T22:24:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.600+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0dec03fce0 con 0x7f0dfc09de50 2023-09-30T22:24:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.600+0000 7f0dfa7fc700 1 -- 172.21.15.32:0/482255494 >> v1:172.21.15.32:6816/126553 conn(0x7f0de802f510 legacy=0x7f0de80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.599+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4011df00 con 0x7ffa40134ec0 2023-09-30T22:24:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.600+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa40134bb0 con 0x7ffa40134ec0 2023-09-30T22:24:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.600+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ffa3001a400 con 0x7ffa40134ec0 2023-09-30T22:24:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.601+0000 7ffa456ac700 1 -- 172.21.15.32:0/3174160979 >> v1:172.21.15.32:6816/126553 conn(0x7ffa2402f540 legacy=0x7ffa240319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.601+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7ffa3001aa30 con 0x7ffa40134ec0 2023-09-30T22:24:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.601+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 --> v1:172.21.15.32:6800/129269 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa2403c210 con 0x7ffa24038510 2023-09-30T22:24:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.601+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa3001a6b0 con 0x7ffa40134ec0 2023-09-30T22:24:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.602+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== osd.0 v1:172.21.15.32:6800/129269 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7ffa2403c210 con 0x7ffa24038510 2023-09-30T22:24:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.602+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 --> v1:172.21.15.32:6808/129477 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0dfc11bcf0 con 0x7f0dfc061190 2023-09-30T22:24:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.604+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== osd.2 v1:172.21.15.32:6808/129477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f0dfc11bcf0 con 0x7f0dfc061190 2023-09-30T22:24:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.604+0000 7f32fe80b700 1 Processor -- start 2023-09-30T22:24:28.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.604+0000 7f4e98a48700 1 Processor -- start 2023-09-30T22:24:28.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.604+0000 7f4e98a48700 1 -- start start 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.604+0000 7f32fe80b700 1 -- start start 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fe80b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 0x7f32f809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fe80b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f81379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fe80b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32f8137f20 0x7f32f8138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fe80b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f806c4b0 con 0x7f32f8137f20 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fe80b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80a09c0 con 0x7f32f8134ec0 2023-09-30T22:24:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fe80b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8134980 con 0x7f32f806c0e0 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 0x7f32f809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 0x7f32f809ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55630/0 (socket says 172.21.15.32:55630) 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32f7fff700 1 -- 172.21.15.32:0/3192810842 learned_addr learned my addr 172.21.15.32:0/3192810842 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f32fcda8700 1 --2- 172.21.15.32:0/3192810842 >> v2:172.21.15.32:3300/0 conn(0x7f32f8137f20 0x7f32f8138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f32f7fff700 1 -- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 msgr2=0x7f32f81379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f32f7fff700 1 --2- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f81379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f32f7fff700 1 -- 172.21.15.32:0/3192810842 >> v2:172.21.15.32:3300/0 conn(0x7f32f8137f20 msgr2=0x7f32f8138390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f32f7fff700 1 --2- 172.21.15.32:0/3192810842 >> v2:172.21.15.32:3300/0 conn(0x7f32f8137f20 0x7f32f8138390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f4e98a48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409d6f0 0x7f4e9409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f4e98a48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e000 0x7f4e941339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f4e98a48700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e9409e410 0x7f4e94133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f4e98a48700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9406b460 con 0x7f4e9409e410 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f4e98a48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9409b120 con 0x7f4e9409d6f0 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.605+0000 7f4e98a48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9409a7e0 con 0x7f4e9409e000 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e9409e410 0x7f4e94133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e9409e410 0x7f4e94133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36154/0 (socket says 172.21.15.32:36154) 2023-09-30T22:24:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 -- 172.21.15.32:0/1273186713 learned_addr learned my addr 172.21.15.32:0/1273186713 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f32f7fff700 1 -- 172.21.15.32:0/3192810842 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80a0600 con 0x7f32f806c0e0 2023-09-30T22:24:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 -- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e000 msgr2=0x7f4e941339f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:24:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e000 0x7f4e941339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 -- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409d6f0 msgr2=0x7f4e9409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409d6f0 0x7f4e9409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 -- 172.21.15.32:0/1273186713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9409a500 con 0x7f4e9409e410 2023-09-30T22:24:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f32f7fff700 1 --2- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 0x7f32f809ac90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32e80098f0 tx=0x7f32e8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f97f6e525672b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.606+0000 7f4e92d9d700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409e410 0x7f4e94133f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009e50 tx=0x7f4e8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a69ce4047230a14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.607+0000 7f4e9159a700 1 -- 172.21.15.32:0/1273186713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00e070 con 0x7f4e9409e410 2023-09-30T22:24:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.607+0000 7f4e9159a700 1 -- 172.21.15.32:0/1273186713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00b9c0 con 0x7f4e9409e410 2023-09-30T22:24:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.607+0000 7f4e9159a700 1 -- 172.21.15.32:0/1273186713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00bd70 con 0x7f4e9409e410 2023-09-30T22:24:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.607+0000 7f32f6ffd700 1 -- 172.21.15.32:0/3192810842 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8011830 con 0x7f32f806c0e0 2023-09-30T22:24:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.607+0000 7f32f6ffd700 1 -- 172.21.15.32:0/3192810842 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8011990 con 0x7f32f806c0e0 2023-09-30T22:24:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.607+0000 7f32f6ffd700 1 -- 172.21.15.32:0/3192810842 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8011d20 con 0x7f32f806c0e0 2023-09-30T22:24:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 -- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409e410 msgr2=0x7f4e94133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409e410 0x7f4e94133f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009e50 tx=0x7f4e8c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 -- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 msgr2=0x7f32f809ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 --2- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 0x7f32f809ac90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32e80098f0 tx=0x7f32e8009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 -- 172.21.15.32:0/1273186713 shutdown_connections 2023-09-30T22:24:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409d6f0 0x7f4e9409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409e410 0x7f4e94133f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 --2- 172.21.15.32:0/1273186713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e000 0x7f4e941339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 -- 172.21.15.32:0/1273186713 >> 172.21.15.32:0/1273186713 conn(0x7f4e94066f50 msgr2=0x7f4e9409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 -- 172.21.15.32:0/3192810842 shutdown_connections 2023-09-30T22:24:28.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 --2- 172.21.15.32:0/3192810842 >> v2:172.21.15.32:3300/0 conn(0x7f32f8137f20 0x7f32f8138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 --2- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f806c0e0 0x7f32f809ac90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 --2- 172.21.15.32:0/3192810842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f81379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f32fe80b700 1 -- 172.21.15.32:0/3192810842 >> 172.21.15.32:0/3192810842 conn(0x7f32f80a4210 msgr2=0x7f32f80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 -- 172.21.15.32:0/1273186713 shutdown_connections 2023-09-30T22:24:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.608+0000 7f4e98a48700 1 -- 172.21.15.32:0/1273186713 wait complete. 2023-09-30T22:24:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.609+0000 7f4e98a48700 1 Processor -- start 2023-09-30T22:24:28.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.609+0000 7f4e98a48700 1 -- start start 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e98a48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409d6f0 0x7f4e94131560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e98a48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409e000 0x7f4e94131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e98a48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e410 0x7f4e941cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e98a48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94133ab0 con 0x7f4e9409d6f0 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e98a48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940a2100 con 0x7f4e9409e000 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e98a48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94131180 con 0x7f4e9409e410 2023-09-30T22:24:28.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409d6f0 0x7f4e94131560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f32fe80b700 1 -- 172.21.15.32:0/3192810842 shutdown_connections 2023-09-30T22:24:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409d6f0 0x7f4e94131560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36156/0 (socket says 172.21.15.32:36156) 2023-09-30T22:24:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 -- 172.21.15.32:0/4202987065 learned_addr learned my addr 172.21.15.32:0/4202987065 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e92d9d700 1 --2- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e410 0x7f4e941cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 -- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e410 msgr2=0x7f4e941cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 --2- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e9409e410 0x7f4e941cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 -- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409e000 msgr2=0x7f4e94131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 --2- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e9409e000 0x7f4e94131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f4e9259c700 1 -- 172.21.15.32:0/4202987065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c009b30 con 0x7f4e9409d6f0 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.611+0000 7f4e9259c700 1 --2- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409d6f0 0x7f4e94131560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400b530 tx=0x7f4e8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.611+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840118f0 con 0x7f4e9409d6f0 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.611+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e84011a70 con 0x7f4e9409d6f0 2023-09-30T22:24:28.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.611+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84011e20 con 0x7f4e9409d6f0 2023-09-30T22:24:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.610+0000 7f32fe80b700 1 -- 172.21.15.32:0/3192810842 wait complete. 2023-09-30T22:24:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.611+0000 7f32fe80b700 1 Processor -- start 2023-09-30T22:24:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.612+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e94133610 con 0x7f4e9409d6f0 2023-09-30T22:24:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.612+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e94131fe0 con 0x7f4e9409d6f0 2023-09-30T22:24:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e9406a9e0 con 0x7f4e9409d6f0 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.612+0000 7f32fe80b700 1 -- start start 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fe80b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f806c0e0 0x7f32f8133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fe80b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f809b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fe80b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f8137f20 0x7f32f809b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fe80b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8129660 con 0x7f32f806c0e0 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fe80b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f809e2c0 con 0x7f32f8134ec0 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fe80b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80726c0 con 0x7f32f8137f20 2023-09-30T22:24:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f809b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fcda8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f8137f20 0x7f32f809b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fcda8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f8137f20 0x7f32f809b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55646/0 (socket says 172.21.15.32:55646) 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f809b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34082/0 (socket says 172.21.15.32:34082) 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32fcda8700 1 -- 172.21.15.32:0/2735164458 learned_addr learned my addr 172.21.15.32:0/2735164458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 -- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f8137f20 msgr2=0x7f32f809b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 --2- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32f8137f20 0x7f32f809b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 -- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f806c0e0 msgr2=0x7f32f8133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 --2- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32f806c0e0 0x7f32f8133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f32f77fe700 1 -- 172.21.15.32:0/2735164458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8009550 con 0x7f32f8134ec0 2023-09-30T22:24:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f32f77fe700 1 --2- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f809b430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32f806f500 tx=0x7f32ec00b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.613+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4e8401a400 con 0x7f4e9409d6f0 2023-09-30T22:24:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4e8403fdb0 con 0x7f4e9409d6f0 2023-09-30T22:24:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f4e91d9b700 1 -- 172.21.15.32:0/4202987065 >> v1:172.21.15.32:6816/126553 conn(0x7f4e7402f510 legacy=0x7f4e740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 --> v1:172.21.15.32:6812/129817 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e7403c260 con 0x7f4e74038560 2023-09-30T22:24:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e8400f8d0 con 0x7f4e9409d6f0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec011830 con 0x7f32f8134ec0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec011990 con 0x7f32f8134ec0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec011d40 con 0x7f32f8134ec0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f812bbf0 con 0x7f32f8134ec0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.614+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f809d000 con 0x7f32f8134ec0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.615+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32f8134bb0 con 0x7f32f8134ec0 2023-09-30T22:24:28.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.615+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f32ec01a400 con 0x7f32f8134ec0 2023-09-30T22:24:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.616+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== osd.3 v1:172.21.15.32:6812/129817 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f4e7403c260 con 0x7f4e74038560 2023-09-30T22:24:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.616+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f32ec03fb80 con 0x7f32f8134ec0 2023-09-30T22:24:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.616+0000 7f32f7fff700 1 -- 172.21.15.32:0/2735164458 >> v1:172.21.15.32:6816/126553 conn(0x7f32e402f510 legacy=0x7f32e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.616+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 --> v1:172.21.15.32:6804/129367 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32e403c2e0 con 0x7f32e40385e0 2023-09-30T22:24:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.616+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32ec0442a0 con 0x7f32f8134ec0 2023-09-30T22:24:28.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.620+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== osd.1 v1:172.21.15.32:6804/129367 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f32e403c2e0 con 0x7f32e40385e0 2023-09-30T22:24:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.621+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 --> v1:172.21.15.32:6800/129269 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa4009a5a0 con 0x7ffa24038510 2023-09-30T22:24:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.622+0000 7ffa35ffb700 1 -- 172.21.15.32:0/3174160979 <== osd.0 v1:172.21.15.32:6800/129269 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 277563570) 0x7ffa4009a5a0 con 0x7ffa24038510 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.622+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 >> v1:172.21.15.32:6800/129269 conn(0x7ffa24038510 legacy=0x7ffa2403a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 >> v1:172.21.15.32:6816/126553 conn(0x7ffa2402f540 legacy=0x7ffa240319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40134ec0 msgr2=0x7ffa4011a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 --2- 172.21.15.32:0/3174160979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa40134ec0 0x7ffa4011a710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffa4006f500 tx=0x7ffa3000b730 comp rx=0 tx=0).stop 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa45ead700 1 -- 172.21.15.32:0/3174160979 reap_dead start 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 shutdown_connections 2023-09-30T22:24:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 >> 172.21.15.32:0/3174160979 conn(0x7ffa400a4210 msgr2=0x7ffa40099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 shutdown_connections 2023-09-30T22:24:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.623+0000 7ffa47910700 1 -- 172.21.15.32:0/3174160979 wait complete. 2023-09-30T22:24:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.624+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 --> v1:172.21.15.32:6808/129477 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0dfc09ebd0 con 0x7f0dfc061190 2023-09-30T22:24:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.626+0000 7f0de3fff700 1 -- 172.21.15.32:0/482255494 <== osd.2 v1:172.21.15.32:6808/129477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3918910179) 0x7f0df4007bd0 con 0x7f0dfc061190 2023-09-30T22:24:28.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.626+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 >> v1:172.21.15.32:6808/129477 conn(0x7f0dfc061190 legacy=0x7f0dfc061560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.627+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 >> v1:172.21.15.32:6816/126553 conn(0x7f0de802f510 legacy=0x7f0de80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:28.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.627+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dfc09de50 msgr2=0x7f0dfc11ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.627+0000 7f0e01933700 1 --2- 172.21.15.32:0/482255494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0dfc09de50 0x7f0dfc11ead0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e9b0 tx=0x7f0dec00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.627+0000 7f0dfb7fe700 1 -- 172.21.15.32:0/482255494 reap_dead start 2023-09-30T22:24:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.633+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 shutdown_connections 2023-09-30T22:24:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.633+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 >> 172.21.15.32:0/482255494 conn(0x7f0dfc0a2aa0 msgr2=0x7f0dfc09ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.633+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 shutdown_connections 2023-09-30T22:24:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.633+0000 7f0e01933700 1 -- 172.21.15.32:0/482255494 wait complete. 2023-09-30T22:24:28.640 INFO:teuthology.orchestra.run.smithi032.stdout:124554051588 2023-09-30T22:24:28.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.646+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 --> v1:172.21.15.32:6812/129817 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4e9409f450 con 0x7f4e74038560 2023-09-30T22:24:28.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.647+0000 7f4e82ffd700 1 -- 172.21.15.32:0/4202987065 <== osd.3 v1:172.21.15.32:6812/129817 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2909512871) 0x7f4e8c003a20 con 0x7f4e74038560 2023-09-30T22:24:28.647 INFO:teuthology.orchestra.run.smithi032.stdout:133143986180 2023-09-30T22:24:28.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.647+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 >> v1:172.21.15.32:6812/129817 conn(0x7f4e74038560 legacy=0x7f4e7403a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.647+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 >> v1:172.21.15.32:6816/126553 conn(0x7f4e7402f510 legacy=0x7f4e740319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.647+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409d6f0 msgr2=0x7f4e94131560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.647+0000 7f4e98a48700 1 --2- 172.21.15.32:0/4202987065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e9409d6f0 0x7f4e94131560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400b530 tx=0x7f4e8400b840 comp rx=0 tx=0).stop 2023-09-30T22:24:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.647+0000 7f4e92d9d700 1 -- 172.21.15.32:0/4202987065 reap_dead start 2023-09-30T22:24:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.648+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 shutdown_connections 2023-09-30T22:24:28.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.648+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 >> 172.21.15.32:0/4202987065 conn(0x7f4e94066f50 msgr2=0x7f4e94071260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.648+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 shutdown_connections 2023-09-30T22:24:28.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.648+0000 7f4e98a48700 1 -- 172.21.15.32:0/4202987065 wait complete. 2023-09-30T22:24:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.651+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 --> v1:172.21.15.32:6804/129367 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32f809f8b0 con 0x7f32e40385e0 2023-09-30T22:24:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.652+0000 7f32f4ff9700 1 -- 172.21.15.32:0/2735164458 <== osd.1 v1:172.21.15.32:6804/129367 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2827348261) 0x7f32f000f040 con 0x7f32e40385e0 2023-09-30T22:24:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.652+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 >> v1:172.21.15.32:6804/129367 conn(0x7f32e40385e0 legacy=0x7f32e403aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.652+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 >> v1:172.21.15.32:6816/126553 conn(0x7f32e402f510 legacy=0x7f32e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 msgr2=0x7f32f809b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fe80b700 1 --2- 172.21.15.32:0/2735164458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32f8134ec0 0x7f32f809b430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32f806f500 tx=0x7f32ec00b730 comp rx=0 tx=0).stop 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fcda8700 1 -- 172.21.15.32:0/2735164458 reap_dead start 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 shutdown_connections 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 >> 172.21.15.32:0/2735164458 conn(0x7f32f80a4210 msgr2=0x7f32f812e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 shutdown_connections 2023-09-30T22:24:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.653+0000 7f32fe80b700 1 -- 172.21.15.32:0/2735164458 wait complete. 2023-09-30T22:24:28.665 INFO:teuthology.orchestra.run.smithi032.stdout:137438953476 2023-09-30T22:24:28.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:28.671 INFO:teuthology.orchestra.run.smithi032.stdout:128849018884 2023-09-30T22:24:28.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:28.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.788+0000 7f274ed21700 1 Processor -- start 2023-09-30T22:24:28.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.788+0000 7f274ed21700 1 -- start start 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274ed21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809f410 0x7f2748136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274ed21700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2748136fe0 0x7f27481393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274ed21700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2748070940 con 0x7f274809e930 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274ed21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274806fd60 con 0x7f274809f410 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274ed21700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f274806fa80 con 0x7f2748136fe0 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f2747fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809f410 0x7f2748136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f2747fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809f410 0x7f2748136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34092/0 (socket says 172.21.15.32:34092) 2023-09-30T22:24:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f2747fff700 1 -- 172.21.15.32:0/3152691052 learned_addr learned my addr 172.21.15.32:0/3152691052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274d2be700 1 --2- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2748136fe0 0x7f27481393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.789+0000 7f274d2be700 1 -- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809f410 msgr2=0x7f2748136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f274d2be700 1 --2- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809f410 0x7f2748136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f274d2be700 1 -- 172.21.15.32:0/3152691052 >> v1:172.21.15.32:6789/0 conn(0x7f274809e930 legacy=0x7f274809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f274d2be700 1 -- 172.21.15.32:0/3152691052 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274809fb60 con 0x7f2748136fe0 2023-09-30T22:24:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f274d2be700 1 --2- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2748136fe0 0x7f27481393c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2740009dc0 tx=0x7f274000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f50926a886c288 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f27477fe700 1 -- 172.21.15.32:0/3152691052 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274000e070 con 0x7f2748136fe0 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f27477fe700 1 -- 172.21.15.32:0/3152691052 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274000bb20 con 0x7f2748136fe0 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.790+0000 7f27477fe700 1 -- 172.21.15.32:0/3152691052 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740012430 con 0x7f2748136fe0 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 -- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2748136fe0 msgr2=0x7f27481393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 --2- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2748136fe0 0x7f27481393c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2740009dc0 tx=0x7f274000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 -- 172.21.15.32:0/3152691052 shutdown_connections 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 --2- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2748136fe0 0x7f27481393c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 --2- 172.21.15.32:0/3152691052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809f410 0x7f2748136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 -- 172.21.15.32:0/3152691052 >> 172.21.15.32:0/3152691052 conn(0x7f27480a4210 msgr2=0x7f27480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 -- 172.21.15.32:0/3152691052 shutdown_connections 2023-09-30T22:24:28.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.791+0000 7f274ed21700 1 -- 172.21.15.32:0/3152691052 wait complete. 2023-09-30T22:24:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.792+0000 7f274ed21700 1 Processor -- start 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.792+0000 7f274ed21700 1 -- start start 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274ed21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809e930 0x7f274806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274ed21700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 0x7f274806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274ed21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2748136fe0 0x7f27481c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274ed21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748122180 con 0x7f2748136fe0 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274ed21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2748132430 con 0x7f274809e930 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274ed21700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f274806cd00 con 0x7f274809f410 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 0x7f274806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 0x7f274806c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55664/0 (socket says 172.21.15.32:55664) 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 -- 172.21.15.32:0/3948408540 learned_addr learned my addr 172.21.15.32:0/3948408540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274cabd700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809e930 0x7f274806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f274d2be700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2748136fe0 0x7f27481c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 -- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809e930 msgr2=0x7f274806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809e930 0x7f274806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 -- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2748136fe0 msgr2=0x7f27481c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2748136fe0 0x7f27481c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.793+0000 7f2747fff700 1 -- 172.21.15.32:0/3948408540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2740009a70 con 0x7f274809f410 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.794+0000 7f2747fff700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 0x7f274806c7c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f273400e940 tx=0x7f273400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.794+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400ee90 con 0x7f274809f410 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.794+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274806d430 con 0x7f274809f410 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.794+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27480a2f90 con 0x7f274809f410 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.794+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273400c850 con 0x7f274809f410 2023-09-30T22:24:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.794+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400cbc0 con 0x7f274809f410 2023-09-30T22:24:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.795+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f273400cd20 con 0x7f274809f410 2023-09-30T22:24:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.795+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27280052e0 con 0x7f274809f410 2023-09-30T22:24:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.796+0000 7f274cabd700 1 -- 172.21.15.32:0/3948408540 >> v1:172.21.15.32:6816/126553 conn(0x7f272c02f510 legacy=0x7f272c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.796+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f273403ff30 con 0x7f274809f410 2023-09-30T22:24:28.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.801+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f273401b3b0 con 0x7f274809f410 2023-09-30T22:24:28.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.813+0000 7f789cfbe700 1 Processor -- start 2023-09-30T22:24:28.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.813+0000 7f789cfbe700 1 -- start start 2023-09-30T22:24:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f789cfbe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f789809d6f0 0x7f789809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f789cfbe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f789cfbe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f789cfbe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789809f520 con 0x7f789809d6f0 2023-09-30T22:24:28.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f789cfbe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789809f130 con 0x7f789809e000 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f789cfbe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f789809e930 con 0x7f789809e410 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f7895d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.814+0000 7f7895d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34106/0 (socket says 172.21.15.32:34106) 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 -- 172.21.15.32:0/3106979145 learned_addr learned my addr 172.21.15.32:0/3106979145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7896d9d700 1 --2- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f789659c700 1 --2- 172.21.15.32:0/3106979145 >> v2:172.21.15.32:3300/0 conn(0x7f789809d6f0 0x7f789809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 -- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 msgr2=0x7f7898130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 --2- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 -- 172.21.15.32:0/3106979145 >> v2:172.21.15.32:3300/0 conn(0x7f789809d6f0 msgr2=0x7f789809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 --2- 172.21.15.32:0/3106979145 >> v2:172.21.15.32:3300/0 conn(0x7f789809d6f0 0x7f789809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 -- 172.21.15.32:0/3106979145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789809b2d0 con 0x7f789809e000 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f789659c700 1 --2- 172.21.15.32:0/3106979145 >> v2:172.21.15.32:3300/0 conn(0x7f789809d6f0 0x7f789809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f7895d9b700 1 --2- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898130660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098f0 tx=0x7f788c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ae5eb394df8e07 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.815+0000 7f789559a700 1 -- 172.21.15.32:0/3106979145 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c011830 con 0x7f789809e000 2023-09-30T22:24:28.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789559a700 1 -- 172.21.15.32:0/3106979145 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c011990 con 0x7f789809e000 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789559a700 1 -- 172.21.15.32:0/3106979145 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c011d20 con 0x7f789809e000 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 -- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 msgr2=0x7f7898130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 --2- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898130660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098f0 tx=0x7f788c009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 -- 172.21.15.32:0/3106979145 shutdown_connections 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 --2- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 --2- 172.21.15.32:0/3106979145 >> v2:172.21.15.32:3300/0 conn(0x7f789809d6f0 0x7f789809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 --2- 172.21.15.32:0/3106979145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898130660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 -- 172.21.15.32:0/3106979145 >> 172.21.15.32:0/3106979145 conn(0x7f7898066f50 msgr2=0x7f789809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.816+0000 7f789cfbe700 1 -- 172.21.15.32:0/3106979145 shutdown_connections 2023-09-30T22:24:28.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.817+0000 7f789cfbe700 1 -- 172.21.15.32:0/3106979145 wait complete. 2023-09-30T22:24:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 Processor -- start 2023-09-30T22:24:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 -- start start 2023-09-30T22:24:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 0x7f78981226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898123170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898123b50 con 0x7f789809d6f0 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78980a3b10 con 0x7f789809e000 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.818+0000 7f789cfbe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7898124410 con 0x7f789809e410 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f789659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 0x7f78981226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7895d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f789659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 0x7f78981226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36194/0 (socket says 172.21.15.32:36194) 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f789659c700 1 -- 172.21.15.32:0/1109681344 learned_addr learned my addr 172.21.15.32:0/1109681344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898123170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 -- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 msgr2=0x7f7898122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 -- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 msgr2=0x7f78981226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 0x7f78981226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 -- 172.21.15.32:0/1109681344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788800b740 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f789659c700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 0x7f78981226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.819+0000 7f7896d9d700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898123170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f789000efc0 tx=0x7f789000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.820+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000cc90 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.820+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789000f040 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.820+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c009550 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.820+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890009e60 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.820+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7898124ac0 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.821+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7890014480 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.821+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f78900406d0 con 0x7f789809e410 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.821+0000 7f789659c700 1 -- 172.21.15.32:0/1109681344 >> v1:172.21.15.32:6816/126553 conn(0x7f787802f510 legacy=0x7f78780319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.821+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898074ea0 con 0x7f789809e410 2023-09-30T22:24:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.830+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7890007900 con 0x7f789809e410 2023-09-30T22:24:28.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.866+0000 7fa514446700 1 Processor -- start 2023-09-30T22:24:28.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.866+0000 7fa514446700 1 -- start start 2023-09-30T22:24:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa514446700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa514446700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa514446700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c06ef30 con 0x7fa50c1338a0 2023-09-30T22:24:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa514446700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c06fef0 con 0x7fa50c136710 2023-09-30T22:24:28.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa514446700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1334b0 con 0x7fa50c139030 2023-09-30T22:24:28.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa5129e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa5129e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55700/0 (socket says 172.21.15.32:55700) 2023-09-30T22:24:28.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa5129e3700 1 -- 172.21.15.32:0/4133698473 learned_addr learned my addr 172.21.15.32:0/4133698473 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.867+0000 7fa5119e1700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5129e3700 1 -- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 msgr2=0x7fa50c138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5129e3700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5129e3700 1 -- 172.21.15.32:0/4133698473 >> v1:172.21.15.32:6789/0 conn(0x7fa50c1338a0 legacy=0x7fa50c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5129e3700 1 -- 172.21.15.32:0/4133698473 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c06fc30 con 0x7fa50c139030 2023-09-30T22:24:28.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5119e1700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:28.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5129e3700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c13b410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa504009e80 tx=0x7fa50400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73c23ccbdb70967 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.868+0000 7fa5111e0700 1 -- 172.21.15.32:0/4133698473 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400e070 con 0x7fa50c139030 2023-09-30T22:24:28.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa5111e0700 1 -- 172.21.15.32:0/4133698473 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400b910 con 0x7fa50c139030 2023-09-30T22:24:28.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa5111e0700 1 -- 172.21.15.32:0/4133698473 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400bcc0 con 0x7fa50c139030 2023-09-30T22:24:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa514446700 1 -- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 msgr2=0x7fa50c13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa514446700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c13b410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa504009e80 tx=0x7fa50400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa514446700 1 -- 172.21.15.32:0/4133698473 shutdown_connections 2023-09-30T22:24:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa514446700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c13b410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa514446700 1 --2- 172.21.15.32:0/4133698473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.869+0000 7fa514446700 1 -- 172.21.15.32:0/4133698473 >> 172.21.15.32:0/4133698473 conn(0x7fa50c0a2aa0 msgr2=0x7fa50c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.870+0000 7fa514446700 1 -- 172.21.15.32:0/4133698473 shutdown_connections 2023-09-30T22:24:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.870+0000 7fa514446700 1 -- 172.21.15.32:0/4133698473 wait complete. 2023-09-30T22:24:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 Processor -- start 2023-09-30T22:24:28.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 -- start start 2023-09-30T22:24:28.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7f23732b3700 1 Processor -- start 2023-09-30T22:24:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c1338a0 0x7fa50c119d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c11a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c11a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c06f640 con 0x7fa50c1338a0 2023-09-30T22:24:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c11bf10 con 0x7fa50c136710 2023-09-30T22:24:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa514446700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c11d570 con 0x7fa50c139030 2023-09-30T22:24:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa5121e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c1338a0 0x7fa50c119d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa5121e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c1338a0 0x7fa50c119d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36208/0 (socket says 172.21.15.32:36208) 2023-09-30T22:24:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa5121e2700 1 -- 172.21.15.32:0/2655628351 learned_addr learned my addr 172.21.15.32:0/2655628351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7fa5129e3700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c11a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.871+0000 7f23732b3700 1 -- start start 2023-09-30T22:24:28.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa5129e3700 1 -- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 msgr2=0x7fa50c11a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa5129e3700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c11a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa5129e3700 1 -- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c1338a0 msgr2=0x7fa50c119d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa5129e3700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c1338a0 0x7fa50c119d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa5129e3700 1 -- 172.21.15.32:0/2655628351 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc0094a0 con 0x7fa50c139030 2023-09-30T22:24:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa5129e3700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c11a7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa504015000 tx=0x7fa504004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004450 con 0x7fa50c139030 2023-09-30T22:24:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f23732b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f236c09e1d0 0x7f236c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f23732b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c135e50 0x7f236c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f237104f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f236c09d6f0 0x7f236c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47726/0 (socket says 172.21.15.32:47726) 2023-09-30T22:24:28.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f237104f700 1 -- 172.21.15.32:0/827407661 learned_addr learned my addr 172.21.15.32:0/827407661 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f237084e700 1 --2- 172.21.15.32:0/827407661 >> v2:172.21.15.32:3300/0 conn(0x7f236c09e1d0 0x7f236c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f2371850700 1 --2- 172.21.15.32:0/827407661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c135e50 0x7f236c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c09f130 con 0x7f236c09d6f0 2023-09-30T22:24:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c09f520 con 0x7f236c09e1d0 2023-09-30T22:24:28.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f236c09e930 con 0x7f236c135e50 2023-09-30T22:24:28.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f2371850700 1 -- 172.21.15.32:0/827407661 >> v2:172.21.15.32:3300/0 conn(0x7f236c09e1d0 msgr2=0x7f236c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:28.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f2371850700 1 --2- 172.21.15.32:0/827407661 >> v2:172.21.15.32:3300/0 conn(0x7f236c09e1d0 0x7f236c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f2371850700 1 -- 172.21.15.32:0/827407661 >> v1:172.21.15.32:6789/0 conn(0x7f236c09d6f0 legacy=0x7f236c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f2371850700 1 -- 172.21.15.32:0/827407661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c09b2d0 con 0x7f236c135e50 2023-09-30T22:24:28.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7f2371850700 1 --2- 172.21.15.32:0/827407661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c135e50 0x7f236c138230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2364009d90 tx=0x7f236400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e305268f423c0d56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5040045f0 con 0x7fa50c139030 2023-09-30T22:24:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa504009b30 con 0x7fa50c139030 2023-09-30T22:24:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.872+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5040092e0 con 0x7fa50c139030 2023-09-30T22:24:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.873+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c11d2b0 con 0x7fa50c139030 2023-09-30T22:24:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.874+0000 7f2363fff700 1 -- 172.21.15.32:0/827407661 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400e070 con 0x7f236c135e50 2023-09-30T22:24:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.874+0000 7f2363fff700 1 -- 172.21.15.32:0/827407661 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400bbd0 con 0x7f236c135e50 2023-09-30T22:24:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.874+0000 7f2363fff700 1 -- 172.21.15.32:0/827407661 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364012430 con 0x7f236c135e50 2023-09-30T22:24:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.874+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c074ea0 con 0x7fa50c139030 2023-09-30T22:24:28.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c135e50 msgr2=0x7f236c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7f23732b3700 1 --2- 172.21.15.32:0/827407661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c135e50 0x7f236c138230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2364009d90 tx=0x7f236400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:28.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa50401c410 con 0x7fa50c139030 2023-09-30T22:24:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 shutdown_connections 2023-09-30T22:24:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7f23732b3700 1 --2- 172.21.15.32:0/827407661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c135e50 0x7f236c138230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7f23732b3700 1 --2- 172.21.15.32:0/827407661 >> v2:172.21.15.32:3300/0 conn(0x7f236c09e1d0 0x7f236c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 >> 172.21.15.32:0/827407661 conn(0x7f236c066f50 msgr2=0x7f236c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7fa5121e2700 1 -- 172.21.15.32:0/2655628351 >> v1:172.21.15.32:6816/126553 conn(0x7fa4f002f510 legacy=0x7fa4f00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.875+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fa504041a40 con 0x7fa50c139030 2023-09-30T22:24:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.878+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 shutdown_connections 2023-09-30T22:24:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.878+0000 7f23732b3700 1 -- 172.21.15.32:0/827407661 wait complete. 2023-09-30T22:24:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.879+0000 7f23732b3700 1 Processor -- start 2023-09-30T22:24:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.879+0000 7f23732b3700 1 -- start start 2023-09-30T22:24:28.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f23732b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 0x7f236c137270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f23732b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c09e1d0 0x7f236c1377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f23732b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f236c135e50 0x7f236c137cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f23732b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c13a870 con 0x7f236c09d6f0 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f23732b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c135550 con 0x7f236c135e50 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f23732b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f236c09c700 con 0x7f236c09e1d0 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f237104f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 0x7f236c137270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f237104f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 0x7f236c137270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36238/0 (socket says 172.21.15.32:36238) 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f237104f700 1 -- 172.21.15.32:0/2082478239 learned_addr learned my addr 172.21.15.32:0/2082478239 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f237084e700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c09e1d0 0x7f236c1377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7f2371850700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f236c135e50 0x7f236c137cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:28.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.880+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa50402d760 con 0x7fa50c139030 2023-09-30T22:24:28.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f237104f700 1 -- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c09e1d0 msgr2=0x7f236c1377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f237104f700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c09e1d0 0x7f236c1377b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f237104f700 1 -- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f236c135e50 msgr2=0x7f236c137cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f237104f700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f236c135e50 0x7f236c137cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f237104f700 1 -- 172.21.15.32:0/2082478239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364009a70 con 0x7f236c09d6f0 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f237104f700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 0x7f236c137270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f236800c520 tx=0x7f236800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368005d70 con 0x7f236c09d6f0 2023-09-30T22:24:28.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23680070d0 con 0x7f236c09d6f0 2023-09-30T22:24:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368007440 con 0x7f236c09d6f0 2023-09-30T22:24:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c138f10 con 0x7f236c09d6f0 2023-09-30T22:24:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.881+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c13b100 con 0x7f236c09d6f0 2023-09-30T22:24:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.883+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c13a4f0 con 0x7f236c09d6f0 2023-09-30T22:24:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.883+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f23680075a0 con 0x7f236c09d6f0 2023-09-30T22:24:28.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.883+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f236803f080 con 0x7f236c09d6f0 2023-09-30T22:24:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.883+0000 7f237084e700 1 -- 172.21.15.32:0/2082478239 >> v1:172.21.15.32:6816/126553 conn(0x7f235002f510 legacy=0x7f23500319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:28.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.893+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f236c13a4f0 con 0x7f236c09d6f0 2023-09-30T22:24:28.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:28.997+0000 7f274cabd700 1 -- 172.21.15.32:0/3948408540 >> v1:172.21.15.32:6816/126553 conn(0x7f272c02f510 legacy=0x7f272c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:29.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.015+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2728005860 con 0x7f274809f410 2023-09-30T22:24:29.015 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.015+0000 7f27457fa700 1 -- 172.21.15.32:0/3948408540 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f273401ab50 con 0x7f274809f410 2023-09-30T22:24:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.017+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 >> v1:172.21.15.32:6816/126553 conn(0x7f272c02f510 legacy=0x7f272c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.017+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 msgr2=0x7f274806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.017+0000 7f274ed21700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 0x7f274806c7c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f273400e940 tx=0x7f273400ec50 comp rx=0 tx=0).stop 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 shutdown_connections 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2748136fe0 0x7f27481c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274809e930 0x7f274806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 --2- 172.21.15.32:0/3948408540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f274809f410 0x7f274806c7c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 >> 172.21.15.32:0/3948408540 conn(0x7f27480a4210 msgr2=0x7f27480a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 shutdown_connections 2023-09-30T22:24:29.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.018+0000 7f274ed21700 1 -- 172.21.15.32:0/3948408540 wait complete. 2023-09-30T22:24:29.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.022+0000 7f789659c700 1 -- 172.21.15.32:0/1109681344 >> v1:172.21.15.32:6816/126553 conn(0x7f787802f510 legacy=0x7f78780319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:29.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.043+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7898126a00 con 0x7f789809e410 2023-09-30T22:24:29.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.043+0000 7f7886ffd700 1 -- 172.21.15.32:0/1109681344 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7890007330 con 0x7f789809e410 2023-09-30T22:24:29.044 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 >> v1:172.21.15.32:6816/126553 conn(0x7f787802f510 legacy=0x7f78780319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 msgr2=0x7f7898123170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898123170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f789000efc0 tx=0x7f789000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 shutdown_connections 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f789809e410 0x7f7898123170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f789809d6f0 0x7f78981226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 --2- 172.21.15.32:0/1109681344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f789809e000 0x7f7898122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 >> 172.21.15.32:0/1109681344 conn(0x7f7898066f50 msgr2=0x7f7898070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 shutdown_connections 2023-09-30T22:24:29.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.046+0000 7f789cfbe700 1 -- 172.21.15.32:0/1109681344 wait complete. 2023-09-30T22:24:29.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.076+0000 7fa5121e2700 1 -- 172.21.15.32:0/2655628351 >> v1:172.21.15.32:6816/126553 conn(0x7fa4f002f510 legacy=0x7fa4f00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.084+0000 7f237084e700 1 -- 172.21.15.32:0/2082478239 >> v1:172.21.15.32:6816/126553 conn(0x7f235002f510 legacy=0x7f23500319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.095+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa50c06e680 con 0x7fa50c139030 2023-09-30T22:24:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.095+0000 7fa502ffd700 1 -- 172.21.15.32:0/2655628351 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa50402a020 con 0x7fa50c139030 2023-09-30T22:24:29.095 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 >> v1:172.21.15.32:6816/126553 conn(0x7fa4f002f510 legacy=0x7fa4f00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 msgr2=0x7fa50c11a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c11a7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa504015000 tx=0x7fa504004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 shutdown_connections 2023-09-30T22:24:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c139030 0x7fa50c11a7d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c1338a0 0x7fa50c119d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 --2- 172.21.15.32:0/2655628351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c136710 0x7fa50c11a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 >> 172.21.15.32:0/2655628351 conn(0x7fa50c0a2aa0 msgr2=0x7fa50c130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:29.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 shutdown_connections 2023-09-30T22:24:29.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.099+0000 7fa514446700 1 -- 172.21.15.32:0/2655628351 wait complete. 2023-09-30T22:24:29.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.105+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f236c13a4d0 con 0x7f236c09d6f0 2023-09-30T22:24:29.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.105+0000 7f2361ffb700 1 -- 172.21.15.32:0/2082478239 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2368017070 con 0x7f236c09d6f0 2023-09-30T22:24:29.105 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:29.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 >> v1:172.21.15.32:6816/126553 conn(0x7f235002f510 legacy=0x7f23500319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:29.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 msgr2=0x7f236c137270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 0x7f236c137270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f236800c520 tx=0x7f236800c830 comp rx=0 tx=0).stop 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 shutdown_connections 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f236c135e50 0x7f236c137cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f236c09d6f0 0x7f236c137270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 --2- 172.21.15.32:0/2082478239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f236c09e1d0 0x7f236c1377b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.109+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 >> 172.21.15.32:0/2082478239 conn(0x7f236c066f50 msgr2=0x7f236c0a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.110+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 shutdown_connections 2023-09-30T22:24:29.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:29.110+0000 7f23732b3700 1 -- 172.21.15.32:0/2082478239 wait complete. 2023-09-30T22:24:29.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:29.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:30.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:30.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:30.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:30.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.187+0000 7f2611a19700 1 Processor -- start 2023-09-30T22:24:30.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.187+0000 7f2611a19700 1 -- start start 2023-09-30T22:24:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f2611a19700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09d6f0 0x7f260c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f2611a19700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f260c09e000 0x7f260c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f2611a19700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c09b120 con 0x7f260c09e410 2023-09-30T22:24:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f2611a19700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260c06b460 con 0x7f260c09e000 2023-09-30T22:24:30.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f2611a19700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c09a7e0 con 0x7f260c09d6f0 2023-09-30T22:24:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f260affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09d6f0 0x7f260c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f260a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f260c09e000 0x7f260c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f260a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f260c09e000 0x7f260c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36248/0 (socket says 172.21.15.32:36248) 2023-09-30T22:24:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f260a7fc700 1 -- 172.21.15.32:0/631952219 learned_addr learned my addr 172.21.15.32:0/631952219 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.188+0000 7f260affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09d6f0 0x7f260c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34142/0 (socket says 172.21.15.32:34142) 2023-09-30T22:24:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f260a7fc700 1 -- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09d6f0 msgr2=0x7f260c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f260a7fc700 1 --2- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09d6f0 0x7f260c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f260a7fc700 1 -- 172.21.15.32:0/631952219 >> v1:172.21.15.32:6789/0 conn(0x7f260c09e410 legacy=0x7f260c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f260a7fc700 1 -- 172.21.15.32:0/631952219 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c09a500 con 0x7f260c09e000 2023-09-30T22:24:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f260a7fc700 1 --2- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1339f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26000098f0 tx=0x7f2600009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de82355ef2f9930 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f2609ffb700 1 -- 172.21.15.32:0/631952219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600011830 con 0x7f260c09e000 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f2609ffb700 1 -- 172.21.15.32:0/631952219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600011990 con 0x7f260c09e000 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.189+0000 7f2609ffb700 1 -- 172.21.15.32:0/631952219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600011d00 con 0x7f260c09e000 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 -- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 msgr2=0x7f260c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 --2- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1339f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26000098f0 tx=0x7f2600009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 -- 172.21.15.32:0/631952219 shutdown_connections 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 --2- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09d6f0 0x7f260c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 --2- 172.21.15.32:0/631952219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1339f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 -- 172.21.15.32:0/631952219 >> 172.21.15.32:0/631952219 conn(0x7f260c066f50 msgr2=0x7f260c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.190+0000 7f2611a19700 1 -- 172.21.15.32:0/631952219 shutdown_connections 2023-09-30T22:24:30.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.191+0000 7f2611a19700 1 -- 172.21.15.32:0/631952219 wait complete. 2023-09-30T22:24:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.192+0000 7f2611a19700 1 Processor -- start 2023-09-30T22:24:30.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.192+0000 7f2611a19700 1 -- start start 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f2611a19700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260c09d6f0 0x7f260c1a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f2611a19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f2611a19700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09e410 0x7f260c1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f2611a19700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c1a93e0 con 0x7f260c09e000 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f2611a19700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c1a7b80 con 0x7f260c09e410 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f2611a19700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f260c09c710 con 0x7f260c09d6f0 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f260a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f260b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09e410 0x7f260c1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f260a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36260/0 (socket says 172.21.15.32:36260) 2023-09-30T22:24:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.193+0000 7f260a7fc700 1 -- 172.21.15.32:0/1107520171 learned_addr learned my addr 172.21.15.32:0/1107520171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f260a7fc700 1 -- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260c09d6f0 msgr2=0x7f260c1a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f260a7fc700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260c09d6f0 0x7f260c1a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f260a7fc700 1 -- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09e410 msgr2=0x7f260c1a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f260a7fc700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09e410 0x7f260c1a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f260a7fc700 1 -- 172.21.15.32:0/1107520171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2600009550 con 0x7f260c09e000 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f260a7fc700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1a6430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2600009c40 tx=0x7f2600004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000036a0 con 0x7f260c09e000 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600003800 con 0x7f260c09e000 2023-09-30T22:24:30.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c1a8d30 con 0x7f260c09e000 2023-09-30T22:24:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600003b90 con 0x7f260c09e000 2023-09-30T22:24:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.194+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c1a9980 con 0x7f260c09e000 2023-09-30T22:24:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.195+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2600003cf0 con 0x7f260c09e000 2023-09-30T22:24:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.196+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c1a9550 con 0x7f260c09e000 2023-09-30T22:24:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.196+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2600021030 con 0x7f260c09e000 2023-09-30T22:24:30.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.196+0000 7f260affd700 1 -- 172.21.15.32:0/1107520171 >> v1:172.21.15.32:6816/126553 conn(0x7f25f802f510 legacy=0x7f25f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.205+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2600010380 con 0x7f260c09e000 2023-09-30T22:24:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.227+0000 7f92474ac700 1 Processor -- start 2023-09-30T22:24:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.227+0000 7f92474ac700 1 -- start start 2023-09-30T22:24:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f92474ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006a770 0x7f924006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f92474ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 0x7f9240131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f92474ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f9240131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f92474ac700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924009ece0 con 0x7f924006b080 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f92474ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924009e760 con 0x7f9240131b40 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f92474ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f924009e040 con 0x7f924006a770 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f9244a47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 0x7f9240131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f9244a47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 0x7f9240131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36264/0 (socket says 172.21.15.32:36264) 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f9244a47700 1 -- 172.21.15.32:0/4293454971 learned_addr learned my addr 172.21.15.32:0/4293454971 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.228+0000 7f9245a49700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f9240131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245248700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006a770 0x7f924006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245a49700 1 -- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006a770 msgr2=0x7f924006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245a49700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006a770 0x7f924006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245a49700 1 -- 172.21.15.32:0/4293454971 >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 msgr2=0x7f9240131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245a49700 1 --2- 172.21.15.32:0/4293454971 >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 0x7f9240131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245a49700 1 -- 172.21.15.32:0/4293454971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924009f630 con 0x7f9240131b40 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9244a47700 1 --2- 172.21.15.32:0/4293454971 >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 0x7f9240131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9245a49700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f9240131fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9238009d90 tx=0x7f923800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a262206926f5cd1b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9237fff700 1 -- 172.21.15.32:0/4293454971 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800bd20 con 0x7f9240131b40 2023-09-30T22:24:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9237fff700 1 -- 172.21.15.32:0/4293454971 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800be80 con 0x7f9240131b40 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.229+0000 7f9237fff700 1 -- 172.21.15.32:0/4293454971 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238010710 con 0x7f9240131b40 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 -- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 msgr2=0x7f9240131fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f9240131fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9238009d90 tx=0x7f923800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 -- 172.21.15.32:0/4293454971 shutdown_connections 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f9240131fc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 --2- 172.21.15.32:0/4293454971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006a770 0x7f924006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 --2- 172.21.15.32:0/4293454971 >> v2:172.21.15.32:3300/0 conn(0x7f924006b080 0x7f9240131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 -- 172.21.15.32:0/4293454971 >> 172.21.15.32:0/4293454971 conn(0x7f9240066f50 msgr2=0x7f924009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 -- 172.21.15.32:0/4293454971 shutdown_connections 2023-09-30T22:24:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.230+0000 7f92474ac700 1 -- 172.21.15.32:0/4293454971 wait complete. 2023-09-30T22:24:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.231+0000 7f92474ac700 1 Processor -- start 2023-09-30T22:24:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.231+0000 7f92474ac700 1 -- start start 2023-09-30T22:24:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f92474ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924006a770 0x7f924011e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f92474ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006b080 0x7f924011e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f92474ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f924011eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f92474ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92401315b0 con 0x7f924006a770 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f92474ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92401202d0 con 0x7f9240131b40 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f92474ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92400a3720 con 0x7f924006b080 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245248700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924006a770 0x7f924011e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245248700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924006a770 0x7f924011e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36270/0 (socket says 172.21.15.32:36270) 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245248700 1 -- 172.21.15.32:0/865617995 learned_addr learned my addr 172.21.15.32:0/865617995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245a49700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f924011eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9244a47700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006b080 0x7f924011e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245a49700 1 -- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006b080 msgr2=0x7f924011e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245a49700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006b080 0x7f924011e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245a49700 1 -- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924006a770 msgr2=0x7f924011e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.232+0000 7f9245a49700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924006a770 0x7f924011e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f9245a49700 1 -- 172.21.15.32:0/865617995 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230009550 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f9245a49700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f924011eb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f923801a000 tx=0x7f9238004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238004800 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9238009a70 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9238004960 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380106f0 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.233+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240121cd0 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.234+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9238010850 con 0x7f9240131b40 2023-09-30T22:24:30.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.235+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924011ac20 con 0x7f9240131b40 2023-09-30T22:24:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.235+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f92380454e0 con 0x7f9240131b40 2023-09-30T22:24:30.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.235+0000 7f9245248700 1 -- 172.21.15.32:0/865617995 >> v1:172.21.15.32:6816/126553 conn(0x7f922402f510 legacy=0x7f92240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.240+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f923801d080 con 0x7f9240131b40 2023-09-30T22:24:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.272+0000 7f42b2b8a700 1 Processor -- start 2023-09-30T22:24:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.272+0000 7f42b2b8a700 1 -- start start 2023-09-30T22:24:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.273+0000 7f42b2b8a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 0x7f42ac06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.273+0000 7f42b2b8a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.273+0000 7f42b2b8a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.273+0000 7f42b2b8a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac12d330 con 0x7f42ac12cf60 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.273+0000 7f42b2b8a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac12cd90 con 0x7f42ac130260 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.273+0000 7f42b2b8a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1306d0 con 0x7f42ac12f8b0 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42b0926700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 0x7f42ac06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42b0926700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 0x7f42ac06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36274/0 (socket says 172.21.15.32:36274) 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55766/0 (socket says 172.21.15.32:55766) 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42b0926700 1 -- 172.21.15.32:0/3649543631 learned_addr learned my addr 172.21.15.32:0/3649543631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42b1127700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 -- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 msgr2=0x7f42ac136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 -- 172.21.15.32:0/3649543631 >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 msgr2=0x7f42ac06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 --2- 172.21.15.32:0/3649543631 >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 0x7f42ac06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 -- 172.21.15.32:0/3649543631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac071780 con 0x7f42ac12f8b0 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42b1127700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42b0926700 1 --2- 172.21.15.32:0/3649543631 >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 0x7f42ac06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42abfff700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac12fd20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4298009870 tx=0x7f4298009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f430d65d13d35a7f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.274+0000 7f42ab7fe700 1 -- 172.21.15.32:0/3649543631 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298011830 con 0x7f42ac12f8b0 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42ab7fe700 1 -- 172.21.15.32:0/3649543631 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4298011990 con 0x7f42ac12f8b0 2023-09-30T22:24:30.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42ab7fe700 1 -- 172.21.15.32:0/3649543631 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298011d20 con 0x7f42ac12f8b0 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 -- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 msgr2=0x7f42ac12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac12fd20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4298009870 tx=0x7f4298009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 -- 172.21.15.32:0/3649543631 shutdown_connections 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/3649543631 >> v2:172.21.15.32:3300/0 conn(0x7f42ac12cf60 0x7f42ac06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/3649543631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac12fd20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 -- 172.21.15.32:0/3649543631 >> 172.21.15.32:0/3649543631 conn(0x7f42ac0a2aa0 msgr2=0x7f42ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.275+0000 7f42b2b8a700 1 -- 172.21.15.32:0/3649543631 shutdown_connections 2023-09-30T22:24:30.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.276+0000 7f42b2b8a700 1 -- 172.21.15.32:0/3649543631 wait complete. 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.277+0000 7f42b2b8a700 1 Processor -- start 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.277+0000 7f42b2b8a700 1 -- start start 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b2b8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 0x7f42ac09b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b2b8a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac09b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b2b8a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac09bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b2b8a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1298d0 con 0x7f42ac12cf60 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b2b8a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac12dab0 con 0x7f42ac130260 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b2b8a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac09e780 con 0x7f42ac12f8b0 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 0x7f42ac09b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 0x7f42ac09b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36284/0 (socket says 172.21.15.32:36284) 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 -- 172.21.15.32:0/1932242423 learned_addr learned my addr 172.21.15.32:0/1932242423 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b1127700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac09bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 -- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 msgr2=0x7f42ac09b580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42abfff700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac09b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac09b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 -- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 msgr2=0x7f42ac09bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac09bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b0926700 1 -- 172.21.15.32:0/1932242423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a000b7f0 con 0x7f42ac12cf60 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.278+0000 7f42b1127700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac09bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42abfff700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac09b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42b0926700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 0x7f42ac09b040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f42a000ded0 tx=0x7f42a00103a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0012040 con 0x7f42ac12cf60 2023-09-30T22:24:30.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a0010e80 con 0x7f42ac12cf60 2023-09-30T22:24:30.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42980094a0 con 0x7f42ac12cf60 2023-09-30T22:24:30.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac09e4c0 con 0x7f42ac12cf60 2023-09-30T22:24:30.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.279+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0004230 con 0x7f42ac12cf60 2023-09-30T22:24:30.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.280+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f42a0004390 con 0x7f42ac12cf60 2023-09-30T22:24:30.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.280+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c0052e0 con 0x7f42ac12cf60 2023-09-30T22:24:30.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.281+0000 7f42abfff700 1 -- 172.21.15.32:0/1932242423 >> v1:172.21.15.32:6816/126553 conn(0x7f429002f510 legacy=0x7f42900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.281+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f42a0042410 con 0x7f42ac12cf60 2023-09-30T22:24:30.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.290+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f42a0015930 con 0x7f42ac12cf60 2023-09-30T22:24:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.314+0000 7f4217f62700 1 Processor -- start 2023-09-30T22:24:30.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4217f62700 1 -- start start 2023-09-30T22:24:30.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4217f62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 0x7f42100a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4217f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f421013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4217f62700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421009edd0 con 0x7f42100a1340 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4217f62700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421009e1f0 con 0x7f4210134170 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4217f62700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f421009dc50 con 0x7f42100a0a30 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.315+0000 7f4215cfe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 0x7f42100a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 0x7f42100a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55782/0 (socket says 172.21.15.32:55782) 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 -- 172.21.15.32:0/3732101083 learned_addr learned my addr 172.21.15.32:0/3732101083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f42154fd700 1 --1- 172.21.15.32:0/3732101083 >> v1:172.21.15.32:6789/0 conn(0x7f42100a1340 0x7f4210133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47738/0 (socket says 172.21.15.32:47738) 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f42164ff700 1 --2- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f421013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4214cfc700 1 -- 172.21.15.32:0/3732101083 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121741093 0 0) 0x7f421009edd0 con 0x7f42100a1340 2023-09-30T22:24:30.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4214cfc700 1 -- 172.21.15.32:0/3732101083 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41fc0035b0 con 0x7f42100a1340 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 -- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 msgr2=0x7f421013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 --2- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f421013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 -- 172.21.15.32:0/3732101083 >> v1:172.21.15.32:6789/0 conn(0x7f42100a1340 legacy=0x7f4210133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 -- 172.21.15.32:0/3732101083 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100a1fe0 con 0x7f42100a0a30 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4215cfe700 1 --2- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 0x7f42100a0e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f420c009920 tx=0x7f420c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7293f70edb7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.316+0000 7f4214cfc700 1 -- 172.21.15.32:0/3732101083 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c011830 con 0x7f42100a0a30 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.317+0000 7f4214cfc700 1 -- 172.21.15.32:0/3732101083 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c011990 con 0x7f42100a0a30 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.317+0000 7f4214cfc700 1 -- 172.21.15.32:0/3732101083 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c011d00 con 0x7f42100a0a30 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.317+0000 7f4217f62700 1 -- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 msgr2=0x7f42100a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.317+0000 7f4217f62700 1 --2- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 0x7f42100a0e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f420c009920 tx=0x7f420c009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.318+0000 7f4217f62700 1 -- 172.21.15.32:0/3732101083 shutdown_connections 2023-09-30T22:24:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.318+0000 7f4217f62700 1 --2- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f421013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.318+0000 7f4217f62700 1 --2- 172.21.15.32:0/3732101083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a0a30 0x7f42100a0e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.318+0000 7f4217f62700 1 -- 172.21.15.32:0/3732101083 >> 172.21.15.32:0/3732101083 conn(0x7f42100a6550 msgr2=0x7f42100a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.318+0000 7f4217f62700 1 -- 172.21.15.32:0/3732101083 shutdown_connections 2023-09-30T22:24:30.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.318+0000 7f4217f62700 1 -- 172.21.15.32:0/3732101083 wait complete. 2023-09-30T22:24:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.319+0000 7f4217f62700 1 Processor -- start 2023-09-30T22:24:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.319+0000 7f4217f62700 1 -- start start 2023-09-30T22:24:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4217f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 0x7f4210123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4217f62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 0x7f4210123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4217f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f42101c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4217f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4210125ef0 con 0x7f42100a0a30 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4217f62700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4210126120 con 0x7f4210134170 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4217f62700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42101244f0 con 0x7f42100a1340 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f42154fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 0x7f4210123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4215cfe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 0x7f4210123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4215cfe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 0x7f4210123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36290/0 (socket says 172.21.15.32:36290) 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f42154fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 0x7f4210123780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55790/0 (socket says 172.21.15.32:55790) 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f4215cfe700 1 -- 172.21.15.32:0/4236961410 learned_addr learned my addr 172.21.15.32:0/4236961410 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:30.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.320+0000 7f42164ff700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f42101c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:30.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42154fd700 1 -- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 msgr2=0x7f42101c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42154fd700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f42101c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42154fd700 1 -- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 msgr2=0x7f4210123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42154fd700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 0x7f4210123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42154fd700 1 -- 172.21.15.32:0/4236961410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c009550 con 0x7f42100a1340 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f4215cfe700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 0x7f4210123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42154fd700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 0x7f4210123780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f420000c4c0 tx=0x7f420000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200005cb0 con 0x7f42100a1340 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420000ce90 con 0x7f42100a1340 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42100a25e0 con 0x7f42100a1340 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200007310 con 0x7f42100a1340 2023-09-30T22:24:30.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.321+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4210124ce0 con 0x7f42100a1340 2023-09-30T22:24:30.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.323+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4200007470 con 0x7f42100a1340 2023-09-30T22:24:30.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.323+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4210074ea0 con 0x7f42100a1340 2023-09-30T22:24:30.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.323+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f420003ebb0 con 0x7f42100a1340 2023-09-30T22:24:30.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.323+0000 7f4215cfe700 1 -- 172.21.15.32:0/4236961410 >> v1:172.21.15.32:6816/126553 conn(0x7f41f402f510 legacy=0x7f41f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.328+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f420001a360 con 0x7f42100a1340 2023-09-30T22:24:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.397+0000 7f260affd700 1 -- 172.21.15.32:0/1107520171 >> v1:172.21.15.32:6816/126553 conn(0x7f25f802f510 legacy=0x7f25f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.424+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f260c074ea0 con 0x7f260c09e000 2023-09-30T22:24:30.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.425+0000 7f25f3fff700 1 -- 172.21.15.32:0/1107520171 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2600015070 con 0x7f260c09e000 2023-09-30T22:24:30.425 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:30.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 >> v1:172.21.15.32:6816/126553 conn(0x7f25f802f510 legacy=0x7f25f80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:30.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 msgr2=0x7f260c1a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1a6430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2600009c40 tx=0x7f2600004750 comp rx=0 tx=0).stop 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 shutdown_connections 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f260c09e410 0x7f260c1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260c09d6f0 0x7f260c1a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 --2- 172.21.15.32:0/1107520171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f260c09e000 0x7f260c1a6430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.427+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 >> 172.21.15.32:0/1107520171 conn(0x7f260c066f50 msgr2=0x7f260c1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.428+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 shutdown_connections 2023-09-30T22:24:30.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.428+0000 7f2611a19700 1 -- 172.21.15.32:0/1107520171 wait complete. 2023-09-30T22:24:30.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.436+0000 7f9245248700 1 -- 172.21.15.32:0/865617995 >> v1:172.21.15.32:6816/126553 conn(0x7f922402f510 legacy=0x7f92240319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:30.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.457+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9240074ea0 con 0x7f9240131b40 2023-09-30T22:24:30.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.457+0000 7f9235ffb700 1 -- 172.21.15.32:0/865617995 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9238026030 con 0x7f9240131b40 2023-09-30T22:24:30.457 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:30.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 >> v1:172.21.15.32:6816/126553 conn(0x7f922402f510 legacy=0x7f92240319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:30.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 msgr2=0x7f924011eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f924011eb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f923801a000 tx=0x7f9238004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 shutdown_connections 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9240131b40 0x7f924011eb90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f924006a770 0x7f924011e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 --2- 172.21.15.32:0/865617995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f924006b080 0x7f924011e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 >> 172.21.15.32:0/865617995 conn(0x7f9240066f50 msgr2=0x7f9240071470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.459+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 shutdown_connections 2023-09-30T22:24:30.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.460+0000 7f92474ac700 1 -- 172.21.15.32:0/865617995 wait complete. 2023-09-30T22:24:30.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:30.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.482+0000 7f42abfff700 1 -- 172.21.15.32:0/1932242423 >> v1:172.21.15.32:6816/126553 conn(0x7f429002f510 legacy=0x7f42900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.506+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f428c005860 con 0x7f42ac12cf60 2023-09-30T22:24:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.506+0000 7f42a97fa700 1 -- 172.21.15.32:0/1932242423 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42a0017370 con 0x7f42ac12cf60 2023-09-30T22:24:30.506 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 >> v1:172.21.15.32:6816/126553 conn(0x7f429002f510 legacy=0x7f42900319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 msgr2=0x7f42ac09b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 0x7f42ac09b040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f42a000ded0 tx=0x7f42a00103a0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 shutdown_connections 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42ac130260 0x7f42ac09bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ac12cf60 0x7f42ac09b040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 --2- 172.21.15.32:0/1932242423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42ac12f8b0 0x7f42ac09b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 >> 172.21.15.32:0/1932242423 conn(0x7f42ac0a2aa0 msgr2=0x7f42ac1b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.508+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 shutdown_connections 2023-09-30T22:24:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.509+0000 7f42b2b8a700 1 -- 172.21.15.32:0/1932242423 wait complete. 2023-09-30T22:24:30.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.524+0000 7f4215cfe700 1 -- 172.21.15.32:0/4236961410 >> v1:172.21.15.32:6816/126553 conn(0x7f41f402f510 legacy=0x7f41f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.539+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f42100a28e0 con 0x7f42100a1340 2023-09-30T22:24:30.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.540+0000 7f42067fc700 1 -- 172.21.15.32:0/4236961410 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42000182b0 con 0x7f42100a1340 2023-09-30T22:24:30.540 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 >> v1:172.21.15.32:6816/126553 conn(0x7f41f402f510 legacy=0x7f41f40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:30.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 msgr2=0x7f4210123780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 0x7f4210123780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f420000c4c0 tx=0x7f420000c7d0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 shutdown_connections 2023-09-30T22:24:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4210134170 0x7f42101c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42100a0a30 0x7f4210123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 --2- 172.21.15.32:0/4236961410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f42100a1340 0x7f4210123780 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:30.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 >> 172.21.15.32:0/4236961410 conn(0x7f42100a6550 msgr2=0x7f421009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 shutdown_connections 2023-09-30T22:24:30.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:30.542+0000 7f4217f62700 1 -- 172.21.15.32:0/4236961410 wait complete. 2023-09-30T22:24:30.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:31.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:31.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:31.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:31.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.596+0000 7f41f3859700 1 Processor -- start 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.597+0000 7f41f3859700 1 -- start start 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f3859700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41ec0a0a30 0x7f41ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f3859700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec134170 0x7f41ec13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f3859700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ec09edd0 con 0x7f41ec0a1340 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f3859700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec09e1f0 con 0x7f41ec0a0a30 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f3859700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec09dc50 con 0x7f41ec134170 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f0df4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41ec0a1340 0x7f41ec133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51348/0 (socket says 172.21.15.32:51348) 2023-09-30T22:24:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f1df6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec134170 0x7f41ec13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f0df4700 1 -- 172.21.15.32:0/2781652168 learned_addr learned my addr 172.21.15.32:0/2781652168 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f15f5700 1 --2- 172.21.15.32:0/2781652168 >> v2:172.21.15.32:3300/0 conn(0x7f41ec0a0a30 0x7f41ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41e3fff700 1 -- 172.21.15.32:0/2781652168 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776820563 0 0) 0x7f41ec09edd0 con 0x7f41ec0a1340 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41e3fff700 1 -- 172.21.15.32:0/2781652168 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d80035b0 con 0x7f41ec0a1340 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f1df6700 1 -- 172.21.15.32:0/2781652168 >> v2:172.21.15.32:3300/0 conn(0x7f41ec0a0a30 msgr2=0x7f41ec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f1df6700 1 --2- 172.21.15.32:0/2781652168 >> v2:172.21.15.32:3300/0 conn(0x7f41ec0a0a30 0x7f41ec0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f1df6700 1 -- 172.21.15.32:0/2781652168 >> v1:172.21.15.32:6789/0 conn(0x7f41ec0a1340 legacy=0x7f41ec133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.598+0000 7f41f1df6700 1 -- 172.21.15.32:0/2781652168 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0a1fe0 con 0x7f41ec134170 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.599+0000 7f41f1df6700 1 --2- 172.21.15.32:0/2781652168 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec134170 0x7f41ec13bd30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009d90 tx=0x7f41e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3002eb5e5f4a195d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.599+0000 7f41e3fff700 1 -- 172.21.15.32:0/2781652168 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400e070 con 0x7f41ec134170 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.599+0000 7f41e3fff700 1 -- 172.21.15.32:0/2781652168 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e400bb20 con 0x7f41ec134170 2023-09-30T22:24:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.599+0000 7f41e3fff700 1 -- 172.21.15.32:0/2781652168 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4012430 con 0x7f41ec134170 2023-09-30T22:24:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 -- 172.21.15.32:0/2781652168 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec134170 msgr2=0x7f41ec13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 --2- 172.21.15.32:0/2781652168 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec134170 0x7f41ec13bd30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009d90 tx=0x7f41e400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 -- 172.21.15.32:0/2781652168 shutdown_connections 2023-09-30T22:24:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 --2- 172.21.15.32:0/2781652168 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec134170 0x7f41ec13bd30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 --2- 172.21.15.32:0/2781652168 >> v2:172.21.15.32:3300/0 conn(0x7f41ec0a0a30 0x7f41ec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 -- 172.21.15.32:0/2781652168 >> 172.21.15.32:0/2781652168 conn(0x7f41ec0a6550 msgr2=0x7f41ec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 -- 172.21.15.32:0/2781652168 shutdown_connections 2023-09-30T22:24:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.600+0000 7f41f3859700 1 -- 172.21.15.32:0/2781652168 wait complete. 2023-09-30T22:24:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.601+0000 7f41f3859700 1 Processor -- start 2023-09-30T22:24:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.601+0000 7f41f3859700 1 -- start start 2023-09-30T22:24:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f3859700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 0x7f41ec1cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f3859700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ec0a1340 0x7f41ec1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f3859700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41ec134170 0x7f41ec1cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f3859700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec125ad0 con 0x7f41ec0a1340 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f3859700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec128e40 con 0x7f41ec134170 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f3859700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1291a0 con 0x7f41ec0a0a30 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 0x7f41ec1cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 0x7f41ec1cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53050/0 (socket says 172.21.15.32:53050) 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 -- 172.21.15.32:0/1269828823 learned_addr learned my addr 172.21.15.32:0/1269828823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f1df6700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41ec134170 0x7f41ec1cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 -- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41ec134170 msgr2=0x7f41ec1cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41ec134170 0x7f41ec1cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 -- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ec0a1340 msgr2=0x7f41ec1ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ec0a1340 0x7f41ec1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.602+0000 7f41f15f5700 1 -- 172.21.15.32:0/1269828823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4009a70 con 0x7f41ec0a0a30 2023-09-30T22:24:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.603+0000 7f41f15f5700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 0x7f41ec1cc810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e810 tx=0x7f41e800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.603+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8013610 con 0x7f41ec0a0a30 2023-09-30T22:24:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.603+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e8013770 con 0x7f41ec0a0a30 2023-09-30T22:24:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.603+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec0a25e0 con 0x7f41ec0a0a30 2023-09-30T22:24:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.603+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec1281d0 con 0x7f41ec0a0a30 2023-09-30T22:24:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.603+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e8013b00 con 0x7f41ec0a0a30 2023-09-30T22:24:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.604+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f41e8013c60 con 0x7f41ec0a0a30 2023-09-30T22:24:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.604+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f41e803fb60 con 0x7f41ec0a0a30 2023-09-30T22:24:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.604+0000 7f41f0df4700 1 -- 172.21.15.32:0/1269828823 >> v1:172.21.15.32:6816/126553 conn(0x7f41d002f510 legacy=0x7f41d00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.605+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc0052e0 con 0x7f41ec0a0a30 2023-09-30T22:24:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.610+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41e8021080 con 0x7f41ec0a0a30 2023-09-30T22:24:31.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.651+0000 7fb72bc54700 1 Processor -- start 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb72bc54700 1 -- start start 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb72bc54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb72406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb72bc54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72406ee80 0x7fb724134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb72bc54700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb724070ee0 con 0x7fb72409eca0 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb72bc54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb724071480 con 0x7fb72406ee80 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb72bc54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb724071b90 con 0x7fb72406e570 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb7291ef700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb72409eca0 0x7fb724134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51352/0 (socket says 172.21.15.32:51352) 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb7291ef700 1 -- 172.21.15.32:0/2462541954 learned_addr learned my addr 172.21.15.32:0/2462541954 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.652+0000 7fb7299f0700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb72406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb72a1f1700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72406ee80 0x7fb724134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb72a1f1700 1 -- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 msgr2=0x7fb72406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb72a1f1700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb72406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb72a1f1700 1 -- 172.21.15.32:0/2462541954 >> v1:172.21.15.32:6789/0 conn(0x7fb72409eca0 legacy=0x7fb724134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb7289ee700 1 -- 172.21.15.32:0/2462541954 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437007882 0 0) 0x7fb724070ee0 con 0x7fb72409eca0 2023-09-30T22:24:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb72a1f1700 1 -- 172.21.15.32:0/2462541954 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240711a0 con 0x7fb72406ee80 2023-09-30T22:24:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb72a1f1700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72406ee80 0x7fb724134d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009dc0 tx=0x7fb71c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8050cebbf12803dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb7289ee700 1 -- 172.21.15.32:0/2462541954 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00e070 con 0x7fb72406ee80 2023-09-30T22:24:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb7289ee700 1 -- 172.21.15.32:0/2462541954 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c00bb20 con 0x7fb72406ee80 2023-09-30T22:24:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.653+0000 7fb7289ee700 1 -- 172.21.15.32:0/2462541954 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c012430 con 0x7fb72406ee80 2023-09-30T22:24:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.654+0000 7fb72bc54700 1 -- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72406ee80 msgr2=0x7fb724134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.654+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72406ee80 0x7fb724134d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009dc0 tx=0x7fb71c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.655+0000 7fb72bc54700 1 -- 172.21.15.32:0/2462541954 shutdown_connections 2023-09-30T22:24:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.655+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72406ee80 0x7fb724134d20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.655+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2462541954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb72406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.655+0000 7fb72bc54700 1 -- 172.21.15.32:0/2462541954 >> 172.21.15.32:0/2462541954 conn(0x7fb7240a2aa0 msgr2=0x7fb7240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.655+0000 7fb72bc54700 1 -- 172.21.15.32:0/2462541954 shutdown_connections 2023-09-30T22:24:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.655+0000 7fb72bc54700 1 -- 172.21.15.32:0/2462541954 wait complete. 2023-09-30T22:24:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.656+0000 7fb72bc54700 1 Processor -- start 2023-09-30T22:24:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.656+0000 7fb72bc54700 1 -- start start 2023-09-30T22:24:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72bc54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb724126600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72bc54700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72406ee80 0x7fb724139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72bc54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72413a430 0x7fb72413a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72bc54700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72413d1b0 con 0x7fb72406ee80 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72bc54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72413d990 con 0x7fb72413a430 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72bc54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb72413be00 con 0x7fb72406e570 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb7299f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb724126600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb7299f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb724126600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53070/0 (socket says 172.21.15.32:53070) 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb7299f0700 1 -- 172.21.15.32:0/2179488621 learned_addr learned my addr 172.21.15.32:0/2179488621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb72a1f1700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72413a430 0x7fb72413a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.657+0000 7fb7291ef700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72406ee80 0x7fb724139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb7299f0700 1 -- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72413a430 msgr2=0x7fb72413a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb7299f0700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72413a430 0x7fb72413a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb7299f0700 1 -- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72406ee80 msgr2=0x7fb724139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb7299f0700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72406ee80 0x7fb724139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb7299f0700 1 -- 172.21.15.32:0/2179488621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c009a70 con 0x7fb72406e570 2023-09-30T22:24:31.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb7299f0700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb724126600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e9b0 tx=0x7fb72000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720009710 con 0x7fb72406e570 2023-09-30T22:24:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000ee50 con 0x7fb72406e570 2023-09-30T22:24:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000c9f0 con 0x7fb72406e570 2023-09-30T22:24:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7241355b0 con 0x7fb72406e570 2023-09-30T22:24:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.658+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7241261b0 con 0x7fb72406e570 2023-09-30T22:24:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.659+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb72000cb50 con 0x7fb72406e570 2023-09-30T22:24:31.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.660+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fb72003fbd0 con 0x7fb72406e570 2023-09-30T22:24:31.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.660+0000 7fb7291ef700 1 -- 172.21.15.32:0/2179488621 >> v1:172.21.15.32:6816/126553 conn(0x7fb70802f510 legacy=0x7fb7080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.660+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb724074ea0 con 0x7fb72406e570 2023-09-30T22:24:31.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.669+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb720021080 con 0x7fb72406e570 2023-09-30T22:24:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.680+0000 7f9ea2cb4700 1 Processor -- start 2023-09-30T22:24:31.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.680+0000 7f9ea2cb4700 1 -- start start 2023-09-30T22:24:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea2cb4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e9c09de00 0x7f9e9c09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea2cb4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 0x7f9e9c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea2cb4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c09a810 con 0x7f9e9c06a770 2023-09-30T22:24:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea2cb4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9c06ab40 con 0x7f9e9c09de00 2023-09-30T22:24:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea2cb4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c09ebc0 con 0x7f9e9c13d1f0 2023-09-30T22:24:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea1251700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 0x7f9e9c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea1251700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 0x7f9e9c13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35268/0 (socket says 172.21.15.32:35268) 2023-09-30T22:24:31.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9ea1251700 1 -- 172.21.15.32:0/2746698053 learned_addr learned my addr 172.21.15.32:0/2746698053 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:31.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.681+0000 7f9e9bfff700 1 --2- 172.21.15.32:0/2746698053 >> v2:172.21.15.32:3300/0 conn(0x7f9e9c09de00 0x7f9e9c09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9e9b7fe700 1 -- 172.21.15.32:0/2746698053 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525254301 0 0) 0x7f9e9c09a810 con 0x7f9e9c06a770 2023-09-30T22:24:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9ea1251700 1 -- 172.21.15.32:0/2746698053 >> v2:172.21.15.32:3300/0 conn(0x7f9e9c09de00 msgr2=0x7f9e9c09e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9ea1251700 1 --2- 172.21.15.32:0/2746698053 >> v2:172.21.15.32:3300/0 conn(0x7f9e9c09de00 0x7f9e9c09e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9ea1251700 1 -- 172.21.15.32:0/2746698053 >> v1:172.21.15.32:6789/0 conn(0x7f9e9c06a770 legacy=0x7f9e9c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9ea1251700 1 -- 172.21.15.32:0/2746698053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c09e920 con 0x7f9e9c13d1f0 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9ea1251700 1 --2- 172.21.15.32:0/2746698053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 0x7f9e9c13f5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009e50 tx=0x7f9e9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc5d91db94c3166 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9e9b7fe700 1 -- 172.21.15.32:0/2746698053 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400e070 con 0x7f9e9c13d1f0 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9e9b7fe700 1 -- 172.21.15.32:0/2746698053 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9400b910 con 0x7f9e9c13d1f0 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.682+0000 7f9e9b7fe700 1 -- 172.21.15.32:0/2746698053 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400bcc0 con 0x7f9e9c13d1f0 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/2746698053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 msgr2=0x7f9e9c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/2746698053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 0x7f9e9c13f5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009e50 tx=0x7f9e9400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/2746698053 shutdown_connections 2023-09-30T22:24:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/2746698053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c13d1f0 0x7f9e9c13f5d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/2746698053 >> v2:172.21.15.32:3300/0 conn(0x7f9e9c09de00 0x7f9e9c09e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/2746698053 >> 172.21.15.32:0/2746698053 conn(0x7f9e9c066f50 msgr2=0x7f9e9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/2746698053 shutdown_connections 2023-09-30T22:24:31.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.683+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/2746698053 wait complete. 2023-09-30T22:24:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.684+0000 7f9ea2cb4700 1 Processor -- start 2023-09-30T22:24:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 -- start start 2023-09-30T22:24:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 0x7f9e9c11b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9c09de00 0x7f9e9c11bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e9c13d1f0 0x7f9e9c11c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c11d280 con 0x7f9e9c09de00 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c119df0 con 0x7f9e9c06a770 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.685+0000 7f9ea2cb4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c071900 con 0x7f9e9c13d1f0 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 0x7f9e9c11b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 0x7f9e9c11b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35278/0 (socket says 172.21.15.32:35278) 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 -- 172.21.15.32:0/3861873118 learned_addr learned my addr 172.21.15.32:0/3861873118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9e9bfff700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9c09de00 0x7f9e9c11bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea1251700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e9c13d1f0 0x7f9e9c11c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 -- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e9c13d1f0 msgr2=0x7f9e9c11c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e9c13d1f0 0x7f9e9c11c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 -- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9c09de00 msgr2=0x7f9e9c11bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9c09de00 0x7f9e9c11bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 -- 172.21.15.32:0/3861873118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e94009b30 con 0x7f9e9c06a770 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea1251700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e9c13d1f0 0x7f9e9c11c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.686+0000 7f9ea0a50700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 0x7f9e9c11b740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c520 tx=0x7f9e9000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.687+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900054e0 con 0x7f9e9c06a770 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.687+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90003680 con 0x7f9e9c06a770 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.687+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c072670 con 0x7f9e9c06a770 2023-09-30T22:24:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.687+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900039f0 con 0x7f9e9c06a770 2023-09-30T22:24:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.687+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c0a3590 con 0x7f9e9c06a770 2023-09-30T22:24:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.688+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9e90003b50 con 0x7f9e9c06a770 2023-09-30T22:24:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.688+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c074ea0 con 0x7f9e9c06a770 2023-09-30T22:24:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.688+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f9e9003efb0 con 0x7f9e9c06a770 2023-09-30T22:24:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.688+0000 7f9e9bfff700 1 -- 172.21.15.32:0/3861873118 >> v1:172.21.15.32:6816/126553 conn(0x7f9e8002f510 legacy=0x7f9e800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.697+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e90070750 con 0x7f9e9c06a770 2023-09-30T22:24:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.753+0000 7f4257ec6700 1 Processor -- start 2023-09-30T22:24:31.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.754+0000 7f4257ec6700 1 -- start start 2023-09-30T22:24:31.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.754+0000 7f4257ec6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f425009d630 0x7f42500a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4257ec6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4250138d80 0x7f425013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4257ec6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500a2a70 con 0x7f42500a0a30 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4257ec6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42500a01d0 con 0x7f4250138d80 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4257ec6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425009e2b0 con 0x7f425009d630 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4255c62700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42500a0a30 0x7f42500a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51358/0 (socket says 172.21.15.32:51358) 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4255c62700 1 -- 172.21.15.32:0/3108082272 learned_addr learned my addr 172.21.15.32:0/3108082272 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4256463700 1 --2- 172.21.15.32:0/3108082272 >> v2:172.21.15.32:3300/0 conn(0x7f4250138d80 0x7f425013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4254c60700 1 -- 172.21.15.32:0/3108082272 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775408317 0 0) 0x7f42500a2a70 con 0x7f42500a0a30 2023-09-30T22:24:31.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4254c60700 1 -- 172.21.15.32:0/3108082272 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f423c0035b0 con 0x7f42500a0a30 2023-09-30T22:24:31.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4256463700 1 -- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f425009d630 msgr2=0x7f42500a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:31.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4256463700 1 --2- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f425009d630 0x7f42500a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4256463700 1 -- 172.21.15.32:0/3108082272 >> v1:172.21.15.32:6789/0 conn(0x7f42500a0a30 legacy=0x7f42500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.755+0000 7f4256463700 1 -- 172.21.15.32:0/3108082272 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425009dc50 con 0x7f4250138d80 2023-09-30T22:24:31.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.756+0000 7f4256463700 1 --2- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4250138d80 0x7f425013b160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4248009d90 tx=0x7f424800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d4bf078540629c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.756+0000 7f4254c60700 1 -- 172.21.15.32:0/3108082272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424800e070 con 0x7f4250138d80 2023-09-30T22:24:31.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.756+0000 7f4254c60700 1 -- 172.21.15.32:0/3108082272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424800bb20 con 0x7f4250138d80 2023-09-30T22:24:31.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.756+0000 7f4254c60700 1 -- 172.21.15.32:0/3108082272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248012430 con 0x7f4250138d80 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 -- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4250138d80 msgr2=0x7f425013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 --2- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4250138d80 0x7f425013b160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4248009d90 tx=0x7f424800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 -- 172.21.15.32:0/3108082272 shutdown_connections 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 --2- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4250138d80 0x7f425013b160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 --2- 172.21.15.32:0/3108082272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f425009d630 0x7f42500a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 -- 172.21.15.32:0/3108082272 >> 172.21.15.32:0/3108082272 conn(0x7f42500a3050 msgr2=0x7f42500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 -- 172.21.15.32:0/3108082272 shutdown_connections 2023-09-30T22:24:31.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.757+0000 7f4257ec6700 1 -- 172.21.15.32:0/3108082272 wait complete. 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 Processor -- start 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 -- start start 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 0x7f42501336e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42500a0a30 0x7f4250133c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4250138d80 0x7f4250134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250138b20 con 0x7f425009d630 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4250136f20 con 0x7f42500a0a30 2023-09-30T22:24:31.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.759+0000 7f4257ec6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42500a4b20 con 0x7f4250138d80 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 0x7f42501336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 0x7f42501336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39112/0 (socket says 172.21.15.32:39112) 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 -- 172.21.15.32:0/2071798576 learned_addr learned my addr 172.21.15.32:0/2071798576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4256463700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4250138d80 0x7f4250134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 -- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4250138d80 msgr2=0x7f4250134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4250138d80 0x7f4250134160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 -- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42500a0a30 msgr2=0x7f4250133c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42500a0a30 0x7f4250133c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 -- 172.21.15.32:0/2071798576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248009a70 con 0x7f425009d630 2023-09-30T22:24:31.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4255c62700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 0x7f42501336e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c700 tx=0x7f424c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:31.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c005d70 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c0070d0 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42500a0410 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.760+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c007460 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.761+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4250137200 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.762+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f424c0075c0 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.762+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250136d60 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.762+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f424c03f0e0 con 0x7f425009d630 2023-09-30T22:24:31.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.762+0000 7f4255461700 1 -- 172.21.15.32:0/2071798576 >> v1:172.21.15.32:6816/126553 conn(0x7f423402f510 legacy=0x7f42340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.771+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f424c00e280 con 0x7f425009d630 2023-09-30T22:24:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.805+0000 7f41f0df4700 1 -- 172.21.15.32:0/1269828823 >> v1:172.21.15.32:6816/126553 conn(0x7f41d002f510 legacy=0x7f41d00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.857+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f41cc005860 con 0x7f41ec0a0a30 2023-09-30T22:24:31.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.858+0000 7f41e1ffb700 1 -- 172.21.15.32:0/1269828823 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41e8012370 con 0x7f41ec0a0a30 2023-09-30T22:24:31.858 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.860+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 >> v1:172.21.15.32:6816/126553 conn(0x7f41d002f510 legacy=0x7f41d00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.860+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 msgr2=0x7f41ec1cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.860+0000 7f41f3859700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 0x7f41ec1cc810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41e800e810 tx=0x7f41e800eb20 comp rx=0 tx=0).stop 2023-09-30T22:24:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7fb7291ef700 1 -- 172.21.15.32:0/2179488621 >> v1:172.21.15.32:6816/126553 conn(0x7fb70802f510 legacy=0x7fb7080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 shutdown_connections 2023-09-30T22:24:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41ec134170 0x7f41ec1cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41ec0a0a30 0x7f41ec1cc810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 --2- 172.21.15.32:0/1269828823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ec0a1340 0x7f41ec1ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 >> 172.21.15.32:0/1269828823 conn(0x7f41ec0a6550 msgr2=0x7f41ec09bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 shutdown_connections 2023-09-30T22:24:31.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.861+0000 7f41f3859700 1 -- 172.21.15.32:0/1269828823 wait complete. 2023-09-30T22:24:31.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.889+0000 7f9e9bfff700 1 -- 172.21.15.32:0/3861873118 >> v1:172.21.15.32:6816/126553 conn(0x7f9e8002f510 legacy=0x7f9e800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.914+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e9c11a7c0 con 0x7f9e9c06a770 2023-09-30T22:24:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.914+0000 7f9e997fa700 1 -- 172.21.15.32:0/3861873118 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e90017070 con 0x7f9e9c06a770 2023-09-30T22:24:31.914 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 >> v1:172.21.15.32:6816/126553 conn(0x7f9e8002f510 legacy=0x7f9e800319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 msgr2=0x7f9e9c11b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 0x7f9e9c11b740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c520 tx=0x7f9e9000c830 comp rx=0 tx=0).stop 2023-09-30T22:24:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 shutdown_connections 2023-09-30T22:24:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e9c13d1f0 0x7f9e9c11c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e9c06a770 0x7f9e9c11b740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 --2- 172.21.15.32:0/3861873118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e9c09de00 0x7f9e9c11bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 >> 172.21.15.32:0/3861873118 conn(0x7f9e9c066f50 msgr2=0x7f9e9c13c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 shutdown_connections 2023-09-30T22:24:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.916+0000 7f9ea2cb4700 1 -- 172.21.15.32:0/3861873118 wait complete. 2023-09-30T22:24:31.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:31.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.963+0000 7f4255461700 1 -- 172.21.15.32:0/2071798576 >> v1:172.21.15.32:6816/126553 conn(0x7f423402f510 legacy=0x7f42340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.967+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb72413b9a0 con 0x7fb72406e570 2023-09-30T22:24:31.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.968+0000 7fb71a7fc700 1 -- 172.21.15.32:0/2179488621 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7200232f0 con 0x7fb72406e570 2023-09-30T22:24:31.968 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 >> v1:172.21.15.32:6816/126553 conn(0x7fb70802f510 legacy=0x7fb7080319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:31.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 msgr2=0x7fb724126600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb724126600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e9b0 tx=0x7fb72000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 shutdown_connections 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb72413a430 0x7fb72413a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb72406e570 0x7fb724126600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 --2- 172.21.15.32:0/2179488621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb72406ee80 0x7fb724139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 >> 172.21.15.32:0/2179488621 conn(0x7fb7240a2aa0 msgr2=0x7fb7240a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 shutdown_connections 2023-09-30T22:24:31.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:31.970+0000 7fb72bc54700 1 -- 172.21.15.32:0/2179488621 wait complete. 2023-09-30T22:24:31.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.024+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f42500747f0 con 0x7f425009d630 2023-09-30T22:24:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.024+0000 7f42467fc700 1 -- 172.21.15.32:0/2071798576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f424c017020 con 0x7f425009d630 2023-09-30T22:24:32.024 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 >> v1:172.21.15.32:6816/126553 conn(0x7f423402f510 legacy=0x7f42340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 msgr2=0x7f42501336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 0x7f42501336e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c700 tx=0x7f424c00ca10 comp rx=0 tx=0).stop 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 shutdown_connections 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4250138d80 0x7f4250134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f425009d630 0x7f42501336e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 --2- 172.21.15.32:0/2071798576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42500a0a30 0x7f4250133c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 >> 172.21.15.32:0/2071798576 conn(0x7f42500a3050 msgr2=0x7f425013acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 shutdown_connections 2023-09-30T22:24:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:32.027+0000 7f4257ec6700 1 -- 172.21.15.32:0/2071798576 wait complete. 2023-09-30T22:24:32.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:32.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:32.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:32.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 Processor -- start 2023-09-30T22:24:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 -- start start 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a5c09e410 0x7f7a5c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c09b120 con 0x7f7a5c09e000 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c06b460 con 0x7f7a5c09e410 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.027+0000 7f7a61e75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c09a7e0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53098/0 (socket says 172.21.15.32:53098) 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/1820497380 learned_addr learned my addr 172.21.15.32:0/1820497380 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/1820497380 >> v2:172.21.15.32:3300/0 conn(0x7f7a5c09e410 0x7f7a5c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/1820497380 >> v2:172.21.15.32:3300/0 conn(0x7f7a5c09e410 msgr2=0x7f7a5c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/1820497380 >> v2:172.21.15.32:3300/0 conn(0x7f7a5c09e410 0x7f7a5c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/1820497380 >> v1:172.21.15.32:6789/0 conn(0x7f7a5c09e000 legacy=0x7f7a5c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/1820497380 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c09a500 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/1820497380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c09dac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009920 tx=0x7f7a4c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad11589ae75dac85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1820497380 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011830 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1820497380 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c011990 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.028+0000 7f7a5a7fc700 1 -- 172.21.15.32:0/1820497380 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011d20 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 -- 172.21.15.32:0/1820497380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 msgr2=0x7f7a5c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 --2- 172.21.15.32:0/1820497380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c09dac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009920 tx=0x7f7a4c009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 -- 172.21.15.32:0/1820497380 shutdown_connections 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 --2- 172.21.15.32:0/1820497380 >> v2:172.21.15.32:3300/0 conn(0x7f7a5c09e410 0x7f7a5c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 --2- 172.21.15.32:0/1820497380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c09dac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 -- 172.21.15.32:0/1820497380 >> 172.21.15.32:0/1820497380 conn(0x7f7a5c066f50 msgr2=0x7f7a5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 -- 172.21.15.32:0/1820497380 shutdown_connections 2023-09-30T22:24:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.029+0000 7f7a61e75700 1 -- 172.21.15.32:0/1820497380 wait complete. 2023-09-30T22:24:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.030+0000 7f7a61e75700 1 Processor -- start 2023-09-30T22:24:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 -- start start 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c1a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c09e000 0x7f7a5c1a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 0x7f7a5c1a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a74c0 con 0x7f7a5c09e000 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a7b90 con 0x7f7a5c09e410 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a61e75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c1a8de0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 0x7f7a5c1a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 0x7f7a5c1a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35302/0 (socket says 172.21.15.32:35302) 2023-09-30T22:24:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a5bfff700 1 -- 172.21.15.32:0/2224866092 learned_addr learned my addr 172.21.15.32:0/2224866092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c1a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.031+0000 7f7a5affd700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c09e000 0x7f7a5c1a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 msgr2=0x7f7a5c1a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 0x7f7a5c1a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c09e000 msgr2=0x7f7a5c1a6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c09e000 0x7f7a5c1a6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5b7fe700 1 -- 172.21.15.32:0/2224866092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009550 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5bfff700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 0x7f7a5c1a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a5b7fe700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c1a6010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c016000 tx=0x7f7a4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c004450 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0036a0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0045f0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a61e75700 1 -- 172.21.15.32:0/2224866092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c1a97c0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.032+0000 7f7a61e75700 1 -- 172.21.15.32:0/2224866092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c0a32d0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.033+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7a4c003a50 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.033+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a340052e0 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.033+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f7a4c021030 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.033+0000 7f7a5affd700 1 -- 172.21.15.32:0/2224866092 >> v1:172.21.15.32:6816/126553 conn(0x7f7a4802f510 legacy=0x7f7a480319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.038+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a4c01a430 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:33.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.079+0000 7fc5f30ef700 1 Processor -- start 2023-09-30T22:24:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.079+0000 7fc5f30ef700 1 -- start start 2023-09-30T22:24:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f30ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5ec09e1d0 0x7fc5ec135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f30ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 0x7fc5ec138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f30ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec09f130 con 0x7fc5ec09d6f0 2023-09-30T22:24:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f30ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec09f520 con 0x7fc5ec09e1d0 2023-09-30T22:24:33.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f30ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec09e930 con 0x7fc5ec135e50 2023-09-30T22:24:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 0x7fc5ec138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 0x7fc5ec138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35308/0 (socket says 172.21.15.32:35308) 2023-09-30T22:24:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 -- 172.21.15.32:0/2165010683 learned_addr learned my addr 172.21.15.32:0/2165010683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5ebfff700 1 --2- 172.21.15.32:0/2165010683 >> v2:172.21.15.32:3300/0 conn(0x7fc5ec09e1d0 0x7fc5ec135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 -- 172.21.15.32:0/2165010683 >> v2:172.21.15.32:3300/0 conn(0x7fc5ec09e1d0 msgr2=0x7fc5ec135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 --2- 172.21.15.32:0/2165010683 >> v2:172.21.15.32:3300/0 conn(0x7fc5ec09e1d0 0x7fc5ec135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 -- 172.21.15.32:0/2165010683 >> v1:172.21.15.32:6789/0 conn(0x7fc5ec09d6f0 legacy=0x7fc5ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5f168c700 1 -- 172.21.15.32:0/2165010683 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec09b2d0 con 0x7fc5ec135e50 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.080+0000 7fc5ebfff700 1 --2- 172.21.15.32:0/2165010683 >> v2:172.21.15.32:3300/0 conn(0x7fc5ec09e1d0 0x7fc5ec135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.081+0000 7fc5f168c700 1 --2- 172.21.15.32:0/2165010683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 0x7fc5ec138230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400d150 tx=0x7fc5e400d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ecde051014d0e22 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.081+0000 7fc5eb7fe700 1 -- 172.21.15.32:0/2165010683 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4010070 con 0x7fc5ec135e50 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.081+0000 7fc5eb7fe700 1 -- 172.21.15.32:0/2165010683 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e400dd80 con 0x7fc5ec135e50 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.081+0000 7fc5eb7fe700 1 -- 172.21.15.32:0/2165010683 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e400b310 con 0x7fc5ec135e50 2023-09-30T22:24:33.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 -- 172.21.15.32:0/2165010683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 msgr2=0x7fc5ec138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/2165010683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 0x7fc5ec138230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400d150 tx=0x7fc5e400d460 comp rx=0 tx=0).stop 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 -- 172.21.15.32:0/2165010683 shutdown_connections 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/2165010683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec135e50 0x7fc5ec138230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/2165010683 >> v2:172.21.15.32:3300/0 conn(0x7fc5ec09e1d0 0x7fc5ec135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 -- 172.21.15.32:0/2165010683 >> 172.21.15.32:0/2165010683 conn(0x7fc5ec066f50 msgr2=0x7fc5ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 -- 172.21.15.32:0/2165010683 shutdown_connections 2023-09-30T22:24:33.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.082+0000 7fc5f30ef700 1 -- 172.21.15.32:0/2165010683 wait complete. 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.083+0000 7fc5f30ef700 1 Processor -- start 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.083+0000 7fc5f30ef700 1 -- start start 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f30ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec09d6f0 0x7fc5ec1c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f30ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ec09e1d0 0x7fc5ec1c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f30ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ec135e50 0x7fc5ec119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f30ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec11b1e0 con 0x7fc5ec135e50 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f30ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec11aff0 con 0x7fc5ec09d6f0 2023-09-30T22:24:33.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f30ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec11c960 con 0x7fc5ec09e1d0 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f0e8b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec09d6f0 0x7fc5ec1c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f0e8b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec09d6f0 0x7fc5ec1c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35312/0 (socket says 172.21.15.32:35312) 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.084+0000 7fc5f0e8b700 1 -- 172.21.15.32:0/751241568 learned_addr learned my addr 172.21.15.32:0/751241568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ec135e50 0x7fc5ec119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5ebfff700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ec09e1d0 0x7fc5ec1c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 -- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ec09e1d0 msgr2=0x7fc5ec1c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ec09e1d0 0x7fc5ec1c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 -- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec09d6f0 msgr2=0x7fc5ec1c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec09d6f0 0x7fc5ec1c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 -- 172.21.15.32:0/751241568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e400bd40 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f168c700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ec135e50 0x7fc5ec119480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec09f6f0 tx=0x7fc5e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e401c070 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.085+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec09c700 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.086+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec11cc20 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.086+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e400b6e0 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.086+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4009a10 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.086+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc5e4009b70 con 0x7fc5ec135e50 2023-09-30T22:24:33.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.087+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc5e40409e0 con 0x7fc5ec135e50 2023-09-30T22:24:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.087+0000 7fc5f0e8b700 1 -- 172.21.15.32:0/751241568 >> v1:172.21.15.32:6816/126553 conn(0x7fc5d002f510 legacy=0x7fc5d00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.092+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc0052e0 con 0x7fc5ec135e50 2023-09-30T22:24:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.098+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5e4025070 con 0x7fc5ec135e50 2023-09-30T22:24:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.133+0000 7fd8304d9700 1 Processor -- start 2023-09-30T22:24:33.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.133+0000 7fd8304d9700 1 -- start start 2023-09-30T22:24:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd8304d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8280a1510 0x7fd828138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd8304d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd828139250 0x7fd82813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd8304d9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82809edd0 con 0x7fd8280a0a30 2023-09-30T22:24:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd8304d9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8280a18e0 con 0x7fd8280a1510 2023-09-30T22:24:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd8304d9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82809e1f0 con 0x7fd828139250 2023-09-30T22:24:33.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd82e275700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd8280a0a30 0x7fd8280a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51394/0 (socket says 172.21.15.32:51394) 2023-09-30T22:24:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd82da74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8280a1510 0x7fd828138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd82e275700 1 -- 172.21.15.32:0/929213284 learned_addr learned my addr 172.21.15.32:0/929213284 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:33.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.134+0000 7fd82ea76700 1 --2- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd828139250 0x7fd82813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82da74700 1 -- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd828139250 msgr2=0x7fd82813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82da74700 1 --2- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd828139250 0x7fd82813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82da74700 1 -- 172.21.15.32:0/929213284 >> v1:172.21.15.32:6789/0 conn(0x7fd8280a0a30 legacy=0x7fd8280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82da74700 1 -- 172.21.15.32:0/929213284 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82809dc50 con 0x7fd8280a1510 2023-09-30T22:24:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82da74700 1 --2- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd828138d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd818009840 tx=0x7fd818009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1766ad8a4e7c6331 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82d273700 1 -- 172.21.15.32:0/929213284 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011830 con 0x7fd8280a1510 2023-09-30T22:24:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82d273700 1 -- 172.21.15.32:0/929213284 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818011990 con 0x7fd8280a1510 2023-09-30T22:24:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.135+0000 7fd82d273700 1 -- 172.21.15.32:0/929213284 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011d20 con 0x7fd8280a1510 2023-09-30T22:24:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 -- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 msgr2=0x7fd828138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 --2- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd828138d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd818009840 tx=0x7fd818009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:33.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 -- 172.21.15.32:0/929213284 shutdown_connections 2023-09-30T22:24:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 --2- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd828139250 0x7fd82813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 --2- 172.21.15.32:0/929213284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd828138d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 -- 172.21.15.32:0/929213284 >> 172.21.15.32:0/929213284 conn(0x7fd8280a6550 msgr2=0x7fd8280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 -- 172.21.15.32:0/929213284 shutdown_connections 2023-09-30T22:24:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.136+0000 7fd8304d9700 1 -- 172.21.15.32:0/929213284 wait complete. 2023-09-30T22:24:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.137+0000 7fd8304d9700 1 Processor -- start 2023-09-30T22:24:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.137+0000 7fd8304d9700 1 -- start start 2023-09-30T22:24:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd8304d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 0x7fd8281c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd8304d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd8281c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd8304d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd828139250 0x7fd828121820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd8304d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828138db0 con 0x7fd8280a1510 2023-09-30T22:24:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd8304d9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd828124da0 con 0x7fd8280a0a30 2023-09-30T22:24:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd8304d9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281be3d0 con 0x7fd828139250 2023-09-30T22:24:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 0x7fd8281c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 0x7fd8281c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35334/0 (socket says 172.21.15.32:35334) 2023-09-30T22:24:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 -- 172.21.15.32:0/3388533674 learned_addr learned my addr 172.21.15.32:0/3388533674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82da74700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd8281c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82ea76700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd828139250 0x7fd828121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 -- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd828139250 msgr2=0x7fd828121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd828139250 0x7fd828121820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 -- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 msgr2=0x7fd8281c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd8281c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 -- 172.21.15.32:0/3388533674 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8180094a0 con 0x7fd8280a0a30 2023-09-30T22:24:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82ea76700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd828139250 0x7fd828121820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:33.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.138+0000 7fd82e275700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 0x7fd8281c8850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd82400c520 tx=0x7fd82400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.139+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824005d70 con 0x7fd8280a0a30 2023-09-30T22:24:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.139+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8240070d0 con 0x7fd8280a0a30 2023-09-30T22:24:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.139+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824007440 con 0x7fd8280a0a30 2023-09-30T22:24:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.139+0000 7fd8304d9700 1 -- 172.21.15.32:0/3388533674 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd828125340 con 0x7fd8280a0a30 2023-09-30T22:24:33.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.139+0000 7fd8304d9700 1 -- 172.21.15.32:0/3388533674 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8280a0220 con 0x7fd8280a0a30 2023-09-30T22:24:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.140+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd828074ea0 con 0x7fd8280a0a30 2023-09-30T22:24:33.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.141+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd8240075a0 con 0x7fd8280a0a30 2023-09-30T22:24:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.141+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fd82403f8d0 con 0x7fd8280a0a30 2023-09-30T22:24:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.141+0000 7fd82da74700 1 -- 172.21.15.32:0/3388533674 >> v1:172.21.15.32:6816/126553 conn(0x7fd80c02f510 legacy=0x7fd80c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.145+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd824011be0 con 0x7fd8280a0a30 2023-09-30T22:24:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.223+0000 7f36b09dd700 1 Processor -- start 2023-09-30T22:24:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.223+0000 7f36b09dd700 1 -- start start 2023-09-30T22:24:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36b09dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36ac09d6f0 0x7f36ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36b09dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 0x7f36ac130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36b09dd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac09f130 con 0x7f36ac09e000 2023-09-30T22:24:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36b09dd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac09f520 con 0x7f36ac09d6f0 2023-09-30T22:24:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36b09dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac09e930 con 0x7f36ac09e410 2023-09-30T22:24:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36aad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 0x7f36ac130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36aad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 0x7f36ac130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53132/0 (socket says 172.21.15.32:53132) 2023-09-30T22:24:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36a9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36ac09e000 0x7f36ac130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51402/0 (socket says 172.21.15.32:51402) 2023-09-30T22:24:33.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36aad9d700 1 -- 172.21.15.32:0/1635182671 learned_addr learned my addr 172.21.15.32:0/1635182671 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.224+0000 7f36aa59c700 1 --2- 172.21.15.32:0/1635182671 >> v2:172.21.15.32:3300/0 conn(0x7f36ac09d6f0 0x7f36ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36aad9d700 1 -- 172.21.15.32:0/1635182671 >> v2:172.21.15.32:3300/0 conn(0x7f36ac09d6f0 msgr2=0x7f36ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36aad9d700 1 --2- 172.21.15.32:0/1635182671 >> v2:172.21.15.32:3300/0 conn(0x7f36ac09d6f0 0x7f36ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36aad9d700 1 -- 172.21.15.32:0/1635182671 >> v1:172.21.15.32:6789/0 conn(0x7f36ac09e000 legacy=0x7f36ac130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36a959a700 1 -- 172.21.15.32:0/1635182671 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170602573 0 0) 0x7f36ac09f130 con 0x7f36ac09e000 2023-09-30T22:24:33.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36aad9d700 1 -- 172.21.15.32:0/1635182671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac09b2d0 con 0x7f36ac09e410 2023-09-30T22:24:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36aa59c700 1 --2- 172.21.15.32:0/1635182671 >> v2:172.21.15.32:3300/0 conn(0x7f36ac09d6f0 0x7f36ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36aad9d700 1 --2- 172.21.15.32:0/1635182671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 0x7f36ac130d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009e50 tx=0x7f36a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e0eb13354f4742 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36a959a700 1 -- 172.21.15.32:0/1635182671 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a400e070 con 0x7f36ac09e410 2023-09-30T22:24:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36a959a700 1 -- 172.21.15.32:0/1635182671 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a400b9c0 con 0x7f36ac09e410 2023-09-30T22:24:33.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.225+0000 7f36a959a700 1 -- 172.21.15.32:0/1635182671 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a400bd70 con 0x7f36ac09e410 2023-09-30T22:24:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 -- 172.21.15.32:0/1635182671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 msgr2=0x7f36ac130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 --2- 172.21.15.32:0/1635182671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 0x7f36ac130d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009e50 tx=0x7f36a400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 -- 172.21.15.32:0/1635182671 shutdown_connections 2023-09-30T22:24:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 --2- 172.21.15.32:0/1635182671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09e410 0x7f36ac130d70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 --2- 172.21.15.32:0/1635182671 >> v2:172.21.15.32:3300/0 conn(0x7f36ac09d6f0 0x7f36ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 -- 172.21.15.32:0/1635182671 >> 172.21.15.32:0/1635182671 conn(0x7f36ac066f50 msgr2=0x7f36ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 -- 172.21.15.32:0/1635182671 shutdown_connections 2023-09-30T22:24:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.226+0000 7f36b09dd700 1 -- 172.21.15.32:0/1635182671 wait complete. 2023-09-30T22:24:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.228+0000 7f36b09dd700 1 Processor -- start 2023-09-30T22:24:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.228+0000 7f36b09dd700 1 -- start start 2023-09-30T22:24:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36b09dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 0x7f36ac131610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36b09dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36ac09e000 0x7f36ac131b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36b09dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36ac09e410 0x7f36ac132090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36b09dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac134340 con 0x7f36ac09e410 2023-09-30T22:24:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36b09dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac09c700 con 0x7f36ac09e000 2023-09-30T22:24:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36b09dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0a39b0 con 0x7f36ac09d6f0 2023-09-30T22:24:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36ac09e410 0x7f36ac132090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 0x7f36ac131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36ac09e410 0x7f36ac132090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39182/0 (socket says 172.21.15.32:39182) 2023-09-30T22:24:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 0x7f36ac131610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53138/0 (socket says 172.21.15.32:53138) 2023-09-30T22:24:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aad9d700 1 -- 172.21.15.32:0/934280123 learned_addr learned my addr 172.21.15.32:0/934280123 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36a9d9b700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36ac09e000 0x7f36ac131b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 -- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36ac09e000 msgr2=0x7f36ac131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36ac09e000 0x7f36ac131b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 -- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36ac09e410 msgr2=0x7f36ac132090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36ac09e410 0x7f36ac132090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.229+0000 7f36aa59c700 1 -- 172.21.15.32:0/934280123 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a4009b30 con 0x7f36ac09d6f0 2023-09-30T22:24:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.230+0000 7f36aa59c700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 0x7f36ac131610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e810 tx=0x7f369c00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.230+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00c070 con 0x7f36ac09d6f0 2023-09-30T22:24:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.230+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c004230 con 0x7f36ac09d6f0 2023-09-30T22:24:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.230+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac132f00 con 0x7f36ac09d6f0 2023-09-30T22:24:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.230+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c0045a0 con 0x7f36ac09d6f0 2023-09-30T22:24:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.230+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac130a60 con 0x7f36ac09d6f0 2023-09-30T22:24:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.231+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f369c0036a0 con 0x7f36ac09d6f0 2023-09-30T22:24:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.231+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac131270 con 0x7f36ac09d6f0 2023-09-30T22:24:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.231+0000 7f36a9d9b700 1 -- 172.21.15.32:0/934280123 >> v1:172.21.15.32:6816/126553 conn(0x7f368c02f510 legacy=0x7f368c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.231+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f369c03f8d0 con 0x7f36ac09d6f0 2023-09-30T22:24:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.235+0000 7f7a5affd700 1 -- 172.21.15.32:0/2224866092 >> v1:172.21.15.32:6816/126553 conn(0x7f7a4802f510 legacy=0x7f7a480319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.241+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f369c01a940 con 0x7f36ac09d6f0 2023-09-30T22:24:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.288+0000 7fc5f0e8b700 1 -- 172.21.15.32:0/751241568 >> v1:172.21.15.32:6816/126553 conn(0x7fc5d002f510 legacy=0x7fc5d00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.303+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7a34005860 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.303+0000 7f7a43fff700 1 -- 172.21.15.32:0/2224866092 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a4c02b020 con 0x7f7a5c09d6f0 2023-09-30T22:24:33.303 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.305+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 >> v1:172.21.15.32:6816/126553 conn(0x7f7a4802f510 legacy=0x7f7a480319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.305+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 msgr2=0x7f7a5c1a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.305+0000 7f7a41ffb700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c1a6010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c016000 tx=0x7f7a4c004750 comp rx=0 tx=0).stop 2023-09-30T22:24:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 shutdown_connections 2023-09-30T22:24:33.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a5c09e410 0x7f7a5c1a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a5c09d6f0 0x7f7a5c1a6010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 --2- 172.21.15.32:0/2224866092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5c09e000 0x7f7a5c1a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 >> 172.21.15.32:0/2224866092 conn(0x7f7a5c066f50 msgr2=0x7f7a5c1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 shutdown_connections 2023-09-30T22:24:33.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.307+0000 7f7a41ffb700 1 -- 172.21.15.32:0/2224866092 wait complete. 2023-09-30T22:24:33.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.312+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc5cc005860 con 0x7fc5ec135e50 2023-09-30T22:24:33.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.312+0000 7fc5e97fa700 1 -- 172.21.15.32:0/751241568 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5e4004030 con 0x7fc5ec135e50 2023-09-30T22:24:33.312 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.314+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 >> v1:172.21.15.32:6816/126553 conn(0x7fc5d002f510 legacy=0x7fc5d00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.314+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ec135e50 msgr2=0x7fc5ec119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.314+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ec135e50 0x7fc5ec119480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec09f6f0 tx=0x7fc5e4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 shutdown_connections 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ec135e50 0x7fc5ec119480 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ec09d6f0 0x7fc5ec1c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 --2- 172.21.15.32:0/751241568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ec09e1d0 0x7fc5ec1c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 >> 172.21.15.32:0/751241568 conn(0x7fc5ec066f50 msgr2=0x7fc5ec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 shutdown_connections 2023-09-30T22:24:33.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.315+0000 7fc5f30ef700 1 -- 172.21.15.32:0/751241568 wait complete. 2023-09-30T22:24:33.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:33.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.342+0000 7fd82da74700 1 -- 172.21.15.32:0/3388533674 >> v1:172.21.15.32:6816/126553 conn(0x7fd80c02f510 legacy=0x7fd80c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.360+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd8280a28e0 con 0x7fd8280a0a30 2023-09-30T22:24:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.361+0000 7fd81effd700 1 -- 172.21.15.32:0/3388533674 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd824017030 con 0x7fd8280a0a30 2023-09-30T22:24:33.361 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 >> v1:172.21.15.32:6816/126553 conn(0x7fd80c02f510 legacy=0x7fd80c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 msgr2=0x7fd8281c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 0x7fd8281c8850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd82400c520 tx=0x7fd82400c830 comp rx=0 tx=0).stop 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 shutdown_connections 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd828139250 0x7fd828121820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8280a0a30 0x7fd8281c8850 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 --2- 172.21.15.32:0/3388533674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8280a1510 0x7fd8281c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.363+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 >> 172.21.15.32:0/3388533674 conn(0x7fd8280a6550 msgr2=0x7fd828137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.364+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 shutdown_connections 2023-09-30T22:24:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.364+0000 7fd81cff9700 1 -- 172.21.15.32:0/3388533674 wait complete. 2023-09-30T22:24:33.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:33.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.432+0000 7f36a9d9b700 1 -- 172.21.15.32:0/934280123 >> v1:172.21.15.32:6816/126553 conn(0x7f368c02f510 legacy=0x7f368c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:33.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.451+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f36ac074ea0 con 0x7f36ac09d6f0 2023-09-30T22:24:33.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.451+0000 7f369affd700 1 -- 172.21.15.32:0/934280123 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f369c01a350 con 0x7f36ac09d6f0 2023-09-30T22:24:33.451 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 >> v1:172.21.15.32:6816/126553 conn(0x7f368c02f510 legacy=0x7f368c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 msgr2=0x7f36ac131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 0x7f36ac131610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e810 tx=0x7f369c00eb20 comp rx=0 tx=0).stop 2023-09-30T22:24:33.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 shutdown_connections 2023-09-30T22:24:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36ac09e410 0x7f36ac132090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36ac09d6f0 0x7f36ac131610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 --2- 172.21.15.32:0/934280123 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36ac09e000 0x7f36ac131b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 >> 172.21.15.32:0/934280123 conn(0x7f36ac066f50 msgr2=0x7f36ac0a2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 shutdown_connections 2023-09-30T22:24:33.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:33.453+0000 7f36b09dd700 1 -- 172.21.15.32:0/934280123 wait complete. 2023-09-30T22:24:33.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:34.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:34.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:34.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:34.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:34.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 Processor -- start 2023-09-30T22:24:34.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 -- start start 2023-09-30T22:24:34.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f056809e1d0 0x7f0568135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056809f130 con 0x7f056809d6f0 2023-09-30T22:24:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056809f520 con 0x7f056809e1d0 2023-09-30T22:24:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.475+0000 7f056dc18700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f056809e930 con 0x7f0568135e50 2023-09-30T22:24:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0567fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f056809e1d0 0x7f0568135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0567fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53140/0 (socket says 172.21.15.32:53140) 2023-09-30T22:24:34.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f056809e1d0 0x7f0568135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39194/0 (socket says 172.21.15.32:39194) 2023-09-30T22:24:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0567fff700 1 -- 172.21.15.32:0/2353634498 learned_addr learned my addr 172.21.15.32:0/2353634498 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 -- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 msgr2=0x7f0568138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 --2- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 -- 172.21.15.32:0/2353634498 >> v1:172.21.15.32:6789/0 conn(0x7f056809d6f0 legacy=0x7f056809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 -- 172.21.15.32:0/2353634498 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056809b2d0 con 0x7f056809e1d0 2023-09-30T22:24:34.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0567fff700 1 --2- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f0566ffd700 1 --2- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568135910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f055c00dc60 tx=0x7f055c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1479afec320672 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.476+0000 7f05667fc700 1 -- 172.21.15.32:0/2353634498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c012070 con 0x7f056809e1d0 2023-09-30T22:24:34.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f05667fc700 1 -- 172.21.15.32:0/2353634498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c00d2e0 con 0x7f056809e1d0 2023-09-30T22:24:34.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f05667fc700 1 -- 172.21.15.32:0/2353634498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00f900 con 0x7f056809e1d0 2023-09-30T22:24:34.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 -- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 msgr2=0x7f0568135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 --2- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568135910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f055c00dc60 tx=0x7f055c00df70 comp rx=0 tx=0).stop 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 -- 172.21.15.32:0/2353634498 shutdown_connections 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 --2- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 --2- 172.21.15.32:0/2353634498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568135910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 -- 172.21.15.32:0/2353634498 >> 172.21.15.32:0/2353634498 conn(0x7f0568066f50 msgr2=0x7f056809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 -- 172.21.15.32:0/2353634498 shutdown_connections 2023-09-30T22:24:34.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.477+0000 7f056dc18700 1 -- 172.21.15.32:0/2353634498 wait complete. 2023-09-30T22:24:34.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.478+0000 7f056dc18700 1 Processor -- start 2023-09-30T22:24:34.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 -- start start 2023-09-30T22:24:34.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f056809d6f0 0x7f05681c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568119900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568119e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056811d4e0 con 0x7f056809e1d0 2023-09-30T22:24:34.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056811d050 con 0x7f056809d6f0 2023-09-30T22:24:34.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f056dc18700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f056811b4a0 con 0x7f0568135e50 2023-09-30T22:24:34.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f0567fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568119e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f0566ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568119900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f0567fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568119e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53142/0 (socket says 172.21.15.32:53142) 2023-09-30T22:24:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f0567fff700 1 -- 172.21.15.32:0/1110244995 learned_addr learned my addr 172.21.15.32:0/1110244995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.479+0000 7f05677fe700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f056809d6f0 0x7f05681c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0566ffd700 1 -- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 msgr2=0x7f0568119e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0566ffd700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568119e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0566ffd700 1 -- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f056809d6f0 msgr2=0x7f05681c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0566ffd700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f056809d6f0 0x7f05681c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0566ffd700 1 -- 172.21.15.32:0/1110244995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056000bb50 con 0x7f056809e1d0 2023-09-30T22:24:34.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0567fff700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568119e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f05677fe700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f056809d6f0 0x7f05681c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:34.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f0566ffd700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568119900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f055c019000 tx=0x7f055c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0036a0 con 0x7f056809e1d0 2023-09-30T22:24:34.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c003800 con 0x7f056809e1d0 2023-09-30T22:24:34.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c00d940 con 0x7f056809e1d0 2023-09-30T22:24:34.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c003bb0 con 0x7f056809e1d0 2023-09-30T22:24:34.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.480+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056811cd50 con 0x7f056809e1d0 2023-09-30T22:24:34.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.481+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f055c003d10 con 0x7f056809e1d0 2023-09-30T22:24:34.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.481+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f055c044c90 con 0x7f056809e1d0 2023-09-30T22:24:34.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.481+0000 7f05677fe700 1 -- 172.21.15.32:0/1110244995 >> v1:172.21.15.32:6816/126553 conn(0x7f055402f510 legacy=0x7f05540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.482+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0568074ea0 con 0x7f056809e1d0 2023-09-30T22:24:34.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.491+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f055c023080 con 0x7f056809e1d0 2023-09-30T22:24:34.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.522+0000 7f29264ee700 1 Processor -- start 2023-09-30T22:24:34.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 -- start start 2023-09-30T22:24:34.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2920130540 0x7f2920130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130e50 0x7f292006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292009a390 con 0x7f2920130540 2023-09-30T22:24:34.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200716a0 con 0x7f2920130e50 2023-09-30T22:24:34.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f29264ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2920071290 con 0x7f292006fb20 2023-09-30T22:24:34.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f291ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2920130540 0x7f2920130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f291ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2920130540 0x7f2920130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39220/0 (socket says 172.21.15.32:39220) 2023-09-30T22:24:34.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f291ffff700 1 -- 172.21.15.32:0/1875750341 learned_addr learned my addr 172.21.15.32:0/1875750341 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:34.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f291f7fe700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130e50 0x7f292006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.523+0000 7f2924a8b700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291ffff700 1 -- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 msgr2=0x7f292006ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:34.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291ffff700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292006ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291ffff700 1 -- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130e50 msgr2=0x7f292006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291ffff700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130e50 0x7f292006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291ffff700 1 -- 172.21.15.32:0/1875750341 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920130340 con 0x7f2920130540 2023-09-30T22:24:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291ffff700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130540 0x7f2920130910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2910009920 tx=0x7f2910009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92735ea9deccbce8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291effd700 1 -- 172.21.15.32:0/1875750341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910010a90 con 0x7f2920130540 2023-09-30T22:24:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291effd700 1 -- 172.21.15.32:0/1875750341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910010bf0 con 0x7f2920130540 2023-09-30T22:24:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.524+0000 7f291effd700 1 -- 172.21.15.32:0/1875750341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910018440 con 0x7f2920130540 2023-09-30T22:24:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 -- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130540 msgr2=0x7f2920130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130540 0x7f2920130910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2910009920 tx=0x7f2910009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 -- 172.21.15.32:0/1875750341 shutdown_connections 2023-09-30T22:24:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130540 0x7f2920130910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 --2- 172.21.15.32:0/1875750341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130e50 0x7f292006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 -- 172.21.15.32:0/1875750341 >> 172.21.15.32:0/1875750341 conn(0x7f29200a2aa0 msgr2=0x7f29200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 -- 172.21.15.32:0/1875750341 shutdown_connections 2023-09-30T22:24:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.525+0000 7f29264ee700 1 -- 172.21.15.32:0/1875750341 wait complete. 2023-09-30T22:24:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.526+0000 7f29264ee700 1 Processor -- start 2023-09-30T22:24:34.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 -- start start 2023-09-30T22:24:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292011a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130540 0x7f292011ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130e50 0x7f292011b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292011e1e0 con 0x7f2920130e50 2023-09-30T22:24:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292011d590 con 0x7f2920130540 2023-09-30T22:24:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f29264ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f292011c370 con 0x7f292006fb20 2023-09-30T22:24:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f291ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292011a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f291ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292011a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53158/0 (socket says 172.21.15.32:53158) 2023-09-30T22:24:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.527+0000 7f291ffff700 1 -- 172.21.15.32:0/2079409588 learned_addr learned my addr 172.21.15.32:0/2079409588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291f7fe700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130540 0x7f292011ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f2924a8b700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130e50 0x7f292011b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291ffff700 1 -- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130540 msgr2=0x7f292011ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291ffff700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130540 0x7f292011ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291ffff700 1 -- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130e50 msgr2=0x7f292011b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291ffff700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130e50 0x7f292011b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291ffff700 1 -- 172.21.15.32:0/2079409588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910009550 con 0x7f292006fb20 2023-09-30T22:24:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f2924a8b700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130e50 0x7f292011b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291ffff700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292011a6c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2910009130 tx=0x7f2910004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29100038a0 con 0x7f292006fb20 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910003a00 con 0x7f292006fb20 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200a0ae0 con 0x7f292006fb20 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910003db0 con 0x7f292006fb20 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.528+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292011e6e0 con 0x7f292006fb20 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.529+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f291001f070 con 0x7f292006fb20 2023-09-30T22:24:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.529+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2910045310 con 0x7f292006fb20 2023-09-30T22:24:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.529+0000 7f291f7fe700 1 -- 172.21.15.32:0/2079409588 >> v1:172.21.15.32:6816/126553 conn(0x7f290c02f510 legacy=0x7f290c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.530+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292009cd40 con 0x7f292006fb20 2023-09-30T22:24:34.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.535+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f291001d080 con 0x7f292006fb20 2023-09-30T22:24:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.560+0000 7f23cab54700 1 Processor -- start 2023-09-30T22:24:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.560+0000 7f23cab54700 1 -- start start 2023-09-30T22:24:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23cab54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23cab54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23cab54700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c409f130 con 0x7f23c409e000 2023-09-30T22:24:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23cab54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c409f520 con 0x7f23c409e410 2023-09-30T22:24:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23cab54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23c409e930 con 0x7f23c409d6f0 2023-09-30T22:24:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23c3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23c409e000 0x7f23c4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51430/0 (socket says 172.21.15.32:51430) 2023-09-30T22:24:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23c88f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23c3fff700 1 -- 172.21.15.32:0/3567890481 learned_addr learned my addr 172.21.15.32:0/3567890481 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.561+0000 7f23c90f1700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c37fe700 1 -- 172.21.15.32:0/3567890481 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385672192 0 0) 0x7f23c409f130 con 0x7f23c409e000 2023-09-30T22:24:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c37fe700 1 -- 172.21.15.32:0/3567890481 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b40035b0 con 0x7f23c409e000 2023-09-30T22:24:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c88f0700 1 -- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 msgr2=0x7f23c4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c88f0700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c88f0700 1 -- 172.21.15.32:0/3567890481 >> v1:172.21.15.32:6789/0 conn(0x7f23c409e000 legacy=0x7f23c4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c88f0700 1 -- 172.21.15.32:0/3567890481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c409b2d0 con 0x7f23c409d6f0 2023-09-30T22:24:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c90f1700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c88f0700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c409dac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009870 tx=0x7f23b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb7e5ce9bc3d999 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c37fe700 1 -- 172.21.15.32:0/3567890481 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8011830 con 0x7f23c409d6f0 2023-09-30T22:24:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c37fe700 1 -- 172.21.15.32:0/3567890481 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b8011990 con 0x7f23c409d6f0 2023-09-30T22:24:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.562+0000 7f23c37fe700 1 -- 172.21.15.32:0/3567890481 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8011d20 con 0x7f23c409d6f0 2023-09-30T22:24:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 -- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 msgr2=0x7f23c409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c409dac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009870 tx=0x7f23b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 -- 172.21.15.32:0/3567890481 shutdown_connections 2023-09-30T22:24:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 --2- 172.21.15.32:0/3567890481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c409dac0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 -- 172.21.15.32:0/3567890481 >> 172.21.15.32:0/3567890481 conn(0x7f23c4066f50 msgr2=0x7f23c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 -- 172.21.15.32:0/3567890481 shutdown_connections 2023-09-30T22:24:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.563+0000 7f23cab54700 1 -- 172.21.15.32:0/3567890481 wait complete. 2023-09-30T22:24:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.564+0000 7f23cab54700 1 Processor -- start 2023-09-30T22:24:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.564+0000 7f23cab54700 1 -- start start 2023-09-30T22:24:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.565+0000 7f23cab54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c4120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.565+0000 7f23cab54700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c409e000 0x7f23c4120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.565+0000 7f23cab54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.565+0000 7f23cab54700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c41215d0 con 0x7f23c409e000 2023-09-30T22:24:34.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.565+0000 7f23cab54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4122260 con 0x7f23c409e410 2023-09-30T22:24:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.565+0000 7f23cab54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40a1c10 con 0x7f23c409d6f0 2023-09-30T22:24:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c88f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c4120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4120b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35390/0 (socket says 172.21.15.32:35390) 2023-09-30T22:24:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c88f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c4120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53176/0 (socket says 172.21.15.32:53176) 2023-09-30T22:24:34.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 -- 172.21.15.32:0/2498296198 learned_addr learned my addr 172.21.15.32:0/2498296198 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c3fff700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c409e000 0x7f23c4120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 -- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 msgr2=0x7f23c4120100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c4120100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 -- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c409e000 msgr2=0x7f23c4120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c409e000 0x7f23c4120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 -- 172.21.15.32:0/2498296198 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc009a70 con 0x7f23c409e410 2023-09-30T22:24:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c3fff700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c409e000 0x7f23c4120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c90f1700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4120b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00efc0 tx=0x7f23bc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.566+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00ca90 con 0x7f23c409e410 2023-09-30T22:24:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.567+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc00f040 con 0x7f23c409e410 2023-09-30T22:24:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.567+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b80094a0 con 0x7f23c409e410 2023-09-30T22:24:34.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.567+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00ce30 con 0x7f23c409e410 2023-09-30T22:24:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.567+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c4123c30 con 0x7f23c409e410 2023-09-30T22:24:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.568+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f23bc014480 con 0x7f23c409e410 2023-09-30T22:24:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.568+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c41224d0 con 0x7f23c409e410 2023-09-30T22:24:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.568+0000 7f23c88f0700 1 -- 172.21.15.32:0/2498296198 >> v1:172.21.15.32:6816/126553 conn(0x7f23a802f510 legacy=0x7f23a80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.568+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f23bc0402d0 con 0x7f23c409e410 2023-09-30T22:24:34.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.577+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23bc012930 con 0x7f23c409e410 2023-09-30T22:24:34.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.682+0000 7f05677fe700 1 -- 172.21.15.32:0/1110244995 >> v1:172.21.15.32:6816/126553 conn(0x7f055402f510 legacy=0x7f05540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.717+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f05681c21b0 con 0x7f056809e1d0 2023-09-30T22:24:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.717+0000 7f054ffff700 1 -- 172.21.15.32:0/1110244995 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f055c015390 con 0x7f056809e1d0 2023-09-30T22:24:34.717 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 >> v1:172.21.15.32:6816/126553 conn(0x7f055402f510 legacy=0x7f05540319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 msgr2=0x7f0568119900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568119900 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f055c019000 tx=0x7f055c004750 comp rx=0 tx=0).stop 2023-09-30T22:24:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 shutdown_connections 2023-09-30T22:24:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0568135e50 0x7f0568119e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f056809d6f0 0x7f05681c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 --2- 172.21.15.32:0/1110244995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f056809e1d0 0x7f0568119900 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 >> 172.21.15.32:0/1110244995 conn(0x7f0568066f50 msgr2=0x7f0568132ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 shutdown_connections 2023-09-30T22:24:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.721+0000 7f056dc18700 1 -- 172.21.15.32:0/1110244995 wait complete. 2023-09-30T22:24:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.730+0000 7f291f7fe700 1 -- 172.21.15.32:0/2079409588 >> v1:172.21.15.32:6816/126553 conn(0x7f290c02f510 legacy=0x7f290c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:34.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.759+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f292011dba0 con 0x7f292006fb20 2023-09-30T22:24:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.760+0000 7f291cff9700 1 -- 172.21.15.32:0/2079409588 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2910027020 con 0x7f292006fb20 2023-09-30T22:24:34.760 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 >> v1:172.21.15.32:6816/126553 conn(0x7f290c02f510 legacy=0x7f290c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 msgr2=0x7f292011a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292011a6c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2910009130 tx=0x7f2910004750 comp rx=0 tx=0).stop 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 shutdown_connections 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2920130e50 0x7f292011b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f292006fb20 0x7f292011a6c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 --2- 172.21.15.32:0/2079409588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920130540 0x7f292011ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 >> 172.21.15.32:0/2079409588 conn(0x7f29200a2aa0 msgr2=0x7f292012d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.762+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 shutdown_connections 2023-09-30T22:24:34.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.763+0000 7f29264ee700 1 -- 172.21.15.32:0/2079409588 wait complete. 2023-09-30T22:24:34.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.769+0000 7f23c88f0700 1 -- 172.21.15.32:0/2498296198 >> v1:172.21.15.32:6816/126553 conn(0x7f23a802f510 legacy=0x7f23a80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:34.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 Processor -- start 2023-09-30T22:24:34.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 -- start start 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c133c30 0x7fdd9c134000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c0723b0 0x7fdd9c09d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c072970 con 0x7fdd9c09d790 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c099680 con 0x7fdd9c133c30 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdda13ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c099e50 con 0x7fdd9c0723b0 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdd9affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c133c30 0x7fdd9c134000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdd9b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd9c09d790 0x7fdd9c13aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51446/0 (socket says 172.21.15.32:51446) 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdd9b7fe700 1 -- 172.21.15.32:0/526582221 learned_addr learned my addr 172.21.15.32:0/526582221 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.783+0000 7fdd9a7fc700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c0723b0 0x7fdd9c09d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd99ffb700 1 -- 172.21.15.32:0/526582221 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299705760 0 0) 0x7fdd94001490 con 0x7fdd9c09d790 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd99ffb700 1 -- 172.21.15.32:0/526582221 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd840035b0 con 0x7fdd9c09d790 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c133c30 msgr2=0x7fdd9c134000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd9a7fc700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c133c30 0x7fdd9c134000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/526582221 >> v1:172.21.15.32:6789/0 conn(0x7fdd9c09d790 legacy=0x7fdd9c13aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/526582221 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c09a060 con 0x7fdd9c0723b0 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd9affd700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c133c30 0x7fdd9c134000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd9a7fc700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c0723b0 0x7fdd9c09d250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009850 tx=0x7fdd90009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a5b72bc2a2e10a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd99ffb700 1 -- 172.21.15.32:0/526582221 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90005e30 con 0x7fdd9c0723b0 2023-09-30T22:24:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd99ffb700 1 -- 172.21.15.32:0/526582221 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd90006030 con 0x7fdd9c0723b0 2023-09-30T22:24:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.784+0000 7fdd99ffb700 1 -- 172.21.15.32:0/526582221 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd900063a0 con 0x7fdd9c0723b0 2023-09-30T22:24:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 -- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c0723b0 msgr2=0x7fdd9c09d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c0723b0 0x7fdd9c09d250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009850 tx=0x7fdd90009b60 comp rx=0 tx=0).stop 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 -- 172.21.15.32:0/526582221 shutdown_connections 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c133c30 0x7fdd9c134000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 --2- 172.21.15.32:0/526582221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c0723b0 0x7fdd9c09d250 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 -- 172.21.15.32:0/526582221 >> 172.21.15.32:0/526582221 conn(0x7fdd9c0a25a0 msgr2=0x7fdd9c0a29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 -- 172.21.15.32:0/526582221 shutdown_connections 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.785+0000 7fdda13ef700 1 -- 172.21.15.32:0/526582221 wait complete. 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.786+0000 7fdda13ef700 1 Processor -- start 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.786+0000 7fdda13ef700 1 -- start start 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.786+0000 7fdda13ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c0723b0 0x7fdd9c12e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.786+0000 7fdda13ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 0x7fdd9c12e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.786+0000 7fdda13ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd9c133c30 0x7fdd9c12eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdda13ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c09c2e0 con 0x7fdd9c133c30 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdda13ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c08b870 con 0x7fdd9c0723b0 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdda13ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c12fb00 con 0x7fdd9c09d790 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 0x7fdd9c12e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 0x7fdd9c12e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53184/0 (socket says 172.21.15.32:53184) 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/1771377632 learned_addr learned my addr 172.21.15.32:0/1771377632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9affd700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c0723b0 0x7fdd9c12e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9b7fe700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd9c133c30 0x7fdd9c12eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c0723b0 msgr2=0x7fdd9c12e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c0723b0 0x7fdd9c12e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd9c133c30 msgr2=0x7fdd9c12eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd9c133c30 0x7fdd9c12eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 -- 172.21.15.32:0/1771377632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009430 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9affd700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c0723b0 0x7fdd9c12e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd9a7fc700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 0x7fdd9c12e620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009ba0 tx=0x7fdd90004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd900067d0 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd90020070 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.787+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90004040 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.788+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd90009430 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.788+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c0a0b50 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.788+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fdd90022030 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.789+0000 7fdd9affd700 1 -- 172.21.15.32:0/1771377632 >> v1:172.21.15.32:6816/126553 conn(0x7fdd8802f510 legacy=0x7fdd880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.789+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fdd90041090 con 0x7fdd9c09d790 2023-09-30T22:24:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.789+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c06ca70 con 0x7fdd9c09d790 2023-09-30T22:24:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.790+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f23c4074ea0 con 0x7f23c409e410 2023-09-30T22:24:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.791+0000 7f23c17fa700 1 -- 172.21.15.32:0/2498296198 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23bc018020 con 0x7f23c409e410 2023-09-30T22:24:34.792 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 >> v1:172.21.15.32:6816/126553 conn(0x7f23a802f510 legacy=0x7f23a80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 msgr2=0x7f23c4120b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4120b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00efc0 tx=0x7f23bc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 shutdown_connections 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c409e410 0x7f23c4120b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c409d6f0 0x7f23c4120100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 --2- 172.21.15.32:0/2498296198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c409e000 0x7f23c4120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 >> 172.21.15.32:0/2498296198 conn(0x7f23c4066f50 msgr2=0x7f23c40710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 shutdown_connections 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.793+0000 7f23cab54700 1 -- 172.21.15.32:0/2498296198 wait complete. 2023-09-30T22:24:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.794+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdd900036a0 con 0x7fdd9c09d790 2023-09-30T22:24:34.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:34.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.990+0000 7fdd9affd700 1 -- 172.21.15.32:0/1771377632 >> v1:172.21.15.32:6816/126553 conn(0x7fdd8802f510 legacy=0x7fdd880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.996+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdd9c130090 con 0x7fdd9c09d790 2023-09-30T22:24:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.997+0000 7fdd837fe700 1 -- 172.21.15.32:0/1771377632 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd9001cbc0 con 0x7fdd9c09d790 2023-09-30T22:24:34.997 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 >> v1:172.21.15.32:6816/126553 conn(0x7fdd8802f510 legacy=0x7fdd880319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 msgr2=0x7fdd9c12e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 0x7fdd9c12e620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009ba0 tx=0x7fdd90004750 comp rx=0 tx=0).stop 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 shutdown_connections 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd9c133c30 0x7fdd9c12eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd9c0723b0 0x7fdd9c12e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 --2- 172.21.15.32:0/1771377632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd9c09d790 0x7fdd9c12e620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 >> 172.21.15.32:0/1771377632 conn(0x7fdd9c0a25a0 msgr2=0x7fdd9c09ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 shutdown_connections 2023-09-30T22:24:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:34.999+0000 7fdda13ef700 1 -- 172.21.15.32:0/1771377632 wait complete. 2023-09-30T22:24:35.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:35.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:35.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:35.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.891+0000 7f91c9cd8700 1 Processor -- start 2023-09-30T22:24:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.891+0000 7f91c9cd8700 1 -- start start 2023-09-30T22:24:35.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.891+0000 7f91c9cd8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91c409e930 0x7f91c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c9cd8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 0x7f91c41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c9cd8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c4070940 con 0x7f91c409f650 2023-09-30T22:24:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c9cd8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c406fd60 con 0x7f91c409e930 2023-09-30T22:24:35.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c9cd8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91c406fa80 con 0x7f91c409f240 2023-09-30T22:24:35.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91baffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 0x7f91c41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91baffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 0x7f91c41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53198/0 (socket says 172.21.15.32:53198) 2023-09-30T22:24:35.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91baffd700 1 -- 172.21.15.32:0/1245795062 learned_addr learned my addr 172.21.15.32:0/1245795062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:35.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1245795062 >> v2:172.21.15.32:3300/0 conn(0x7f91c409e930 0x7f91c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c37fe700 1 -- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 msgr2=0x7f91c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:35.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 0x7f91c41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c37fe700 1 -- 172.21.15.32:0/1245795062 >> v1:172.21.15.32:6789/0 conn(0x7f91c409f650 legacy=0x7f91c4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c37fe700 1 -- 172.21.15.32:0/1245795062 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c409fb60 con 0x7f91c409e930 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91baffd700 1 --2- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 0x7f91c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.892+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409e930 0x7f91c409ed00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009840 tx=0x7f91b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee82d52fe4ecf6aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.893+0000 7f91c2ffd700 1 -- 172.21.15.32:0/1245795062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4011830 con 0x7f91c409e930 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.893+0000 7f91c2ffd700 1 -- 172.21.15.32:0/1245795062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4011990 con 0x7f91c409e930 2023-09-30T22:24:35.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.893+0000 7f91c2ffd700 1 -- 172.21.15.32:0/1245795062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4011d20 con 0x7f91c409e930 2023-09-30T22:24:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409e930 msgr2=0x7f91c409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409e930 0x7f91c409ed00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009840 tx=0x7f91b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1245795062 shutdown_connections 2023-09-30T22:24:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409e930 0x7f91c409ed00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1245795062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409f240 0x7f91c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1245795062 >> 172.21.15.32:0/1245795062 conn(0x7f91c40a4210 msgr2=0x7f91c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1245795062 shutdown_connections 2023-09-30T22:24:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.894+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1245795062 wait complete. 2023-09-30T22:24:35.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.895+0000 7f91c9cd8700 1 Processor -- start 2023-09-30T22:24:35.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.896+0000 7f91c9cd8700 1 -- start start 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c9cd8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 0x7f91c406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c9cd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409f240 0x7f91c406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c9cd8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c409f650 0x7f91c406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c9cd8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c41a8940 con 0x7f91c409f240 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c9cd8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c41a9980 con 0x7f91c409f650 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c9cd8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40a35a0 con 0x7f91c409e930 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 0x7f91c406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 0x7f91c406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53208/0 (socket says 172.21.15.32:53208) 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 -- 172.21.15.32:0/1455692119 learned_addr learned my addr 172.21.15.32:0/1455692119 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c3fff700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c409f650 0x7f91c406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91baffd700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409f240 0x7f91c406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 -- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c409f650 msgr2=0x7f91c406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c409f650 0x7f91c406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 -- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409f240 msgr2=0x7f91c406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409f240 0x7f91c406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 -- 172.21.15.32:0/1455692119 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac009550 con 0x7f91c409e930 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.897+0000 7f91c37fe700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 0x7f91c406b8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009fd0 tx=0x7f91b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.898+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40036a0 con 0x7f91c409e930 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.898+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4027070 con 0x7f91c409e930 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.898+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4003a60 con 0x7f91c409e930 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.898+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b40094a0 con 0x7f91c409e930 2023-09-30T22:24:35.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.898+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c40a0190 con 0x7f91c409e930 2023-09-30T22:24:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.899+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f91b4003bc0 con 0x7f91c409e930 2023-09-30T22:24:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.899+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f91b4029070 con 0x7f91c409e930 2023-09-30T22:24:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.899+0000 7f91baffd700 1 -- 172.21.15.32:0/1455692119 >> v1:172.21.15.32:6816/126553 conn(0x7f91a402f510 legacy=0x7f91a40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:35.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.899+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c41a0b30 con 0x7f91c409e930 2023-09-30T22:24:35.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.904+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91b4015080 con 0x7f91c409e930 2023-09-30T22:24:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.931+0000 7f28808e8700 1 Processor -- start 2023-09-30T22:24:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.931+0000 7f28808e8700 1 -- start start 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f28808e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c0a0a30 0x7f287c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f28808e8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f287c09d630 0x7f287c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f28808e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c133930 0x7f287c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f28808e8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0a2a70 con 0x7f287c09d630 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f28808e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0a01d0 con 0x7f287c0a0a30 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f28808e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f287c09e2b0 con 0x7f287c133930 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f2879d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f287c09d630 0x7f287c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f287ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c133930 0x7f287c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f2879d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f287c09d630 0x7f287c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39244/0 (socket says 172.21.15.32:39244) 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.932+0000 7f2879d9b700 1 -- 172.21.15.32:0/625365273 learned_addr learned my addr 172.21.15.32:0/625365273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f287a59c700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c0a0a30 0x7f287c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f2879d9b700 1 -- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c133930 msgr2=0x7f287c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f2879d9b700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c133930 0x7f287c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f2879d9b700 1 -- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c0a0a30 msgr2=0x7f287c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f2879d9b700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c0a0a30 0x7f287c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f2879d9b700 1 -- 172.21.15.32:0/625365273 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c09dc50 con 0x7f287c09d630 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f2879d9b700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c0a1340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28640098f0 tx=0x7f2864009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf51af40ba87c23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f287959a700 1 -- 172.21.15.32:0/625365273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864011830 con 0x7f287c09d630 2023-09-30T22:24:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f287959a700 1 -- 172.21.15.32:0/625365273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864011990 con 0x7f287c09d630 2023-09-30T22:24:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.933+0000 7f287959a700 1 -- 172.21.15.32:0/625365273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864011d20 con 0x7f287c09d630 2023-09-30T22:24:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 -- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 msgr2=0x7f287c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c0a1340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28640098f0 tx=0x7f2864009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 -- 172.21.15.32:0/625365273 shutdown_connections 2023-09-30T22:24:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c0a0a30 0x7f287c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c133930 0x7f287c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 --2- 172.21.15.32:0/625365273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c0a1340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 -- 172.21.15.32:0/625365273 >> 172.21.15.32:0/625365273 conn(0x7f287c0a3050 msgr2=0x7f287c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 -- 172.21.15.32:0/625365273 shutdown_connections 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.934+0000 7f28808e8700 1 -- 172.21.15.32:0/625365273 wait complete. 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 Processor -- start 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 -- start start 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c0a0a30 0x7f287c1c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c133930 0x7f287c1c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1206b0 con 0x7f287c09d630 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1204c0 con 0x7f287c133930 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.936+0000 7f28808e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1223d0 con 0x7f287c0a0a30 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f287ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c133930 0x7f287c1c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c0a0a30 0x7f287c1c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f287ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c133930 0x7f287c1c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35452/0 (socket says 172.21.15.32:35452) 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f287ad9d700 1 -- 172.21.15.32:0/2230584170 learned_addr learned my addr 172.21.15.32:0/2230584170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f287a59c700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 -- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c133930 msgr2=0x7f287c1c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c133930 0x7f287c1c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 -- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 msgr2=0x7f287c1c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c1c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 -- 172.21.15.32:0/2230584170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864009550 con 0x7f287c0a0a30 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f287a59c700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c1c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2879d9b700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c0a0a30 0x7f287c1c8640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2864016000 tx=0x7f2864004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864004240 con 0x7f287c0a0a30 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286401f070 con 0x7f287c0a0a30 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c0a40c0 con 0x7f287c0a0a30 2023-09-30T22:24:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.937+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28640036a0 con 0x7f287c0a0a30 2023-09-30T22:24:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.938+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c120cb0 con 0x7f287c0a0a30 2023-09-30T22:24:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.938+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2864021030 con 0x7f287c0a0a30 2023-09-30T22:24:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.939+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2864040cd0 con 0x7f287c0a0a30 2023-09-30T22:24:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.939+0000 7f287a59c700 1 -- 172.21.15.32:0/2230584170 >> v1:172.21.15.32:6816/126553 conn(0x7f285c02f510 legacy=0x7f285c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.939+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c121d90 con 0x7f287c0a0a30 2023-09-30T22:24:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:35.943+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f287c121d90 con 0x7f287c0a0a30 2023-09-30T22:24:36.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.001+0000 7f791deb5700 1 Processor -- start 2023-09-30T22:24:36.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.001+0000 7f791deb5700 1 -- start start 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f791deb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79180a0a30 0x7f79180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f791deb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 0x7f79180a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f791deb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7918133930 0x7f7918133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f791deb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180a2a70 con 0x7f7918133930 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f791deb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79180a01d0 con 0x7f791809d630 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f791deb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f791809e2b0 con 0x7f79180a0a30 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f790effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 0x7f79180a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f790effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 0x7f79180a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35456/0 (socket says 172.21.15.32:35456) 2023-09-30T22:24:36.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f790effd700 1 -- 172.21.15.32:0/501447681 learned_addr learned my addr 172.21.15.32:0/501447681 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f7917fff700 1 --2- 172.21.15.32:0/501447681 >> v2:172.21.15.32:3300/0 conn(0x7f7918133930 0x7f7918133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.002+0000 7f79177fe700 1 --2- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79180a0a30 0x7f79180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f790effd700 1 -- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79180a0a30 msgr2=0x7f79180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f790effd700 1 --2- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79180a0a30 0x7f79180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f790effd700 1 -- 172.21.15.32:0/501447681 >> v2:172.21.15.32:3300/0 conn(0x7f7918133930 msgr2=0x7f7918133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f790effd700 1 --2- 172.21.15.32:0/501447681 >> v2:172.21.15.32:3300/0 conn(0x7f7918133930 0x7f7918133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f790effd700 1 -- 172.21.15.32:0/501447681 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791809dc50 con 0x7f791809d630 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f790effd700 1 --2- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 0x7f79180a1340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79000098f0 tx=0x7f7900009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bedaaf451303f21 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f7916ffd700 1 -- 172.21.15.32:0/501447681 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900011830 con 0x7f791809d630 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f7916ffd700 1 -- 172.21.15.32:0/501447681 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7900011990 con 0x7f791809d630 2023-09-30T22:24:36.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.003+0000 7f7916ffd700 1 -- 172.21.15.32:0/501447681 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900011d20 con 0x7f791809d630 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 -- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 msgr2=0x7f79180a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 --2- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 0x7f79180a1340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79000098f0 tx=0x7f7900009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 -- 172.21.15.32:0/501447681 shutdown_connections 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 --2- 172.21.15.32:0/501447681 >> v2:172.21.15.32:3300/0 conn(0x7f7918133930 0x7f7918133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 --2- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79180a0a30 0x7f79180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 --2- 172.21.15.32:0/501447681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f791809d630 0x7f79180a1340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 -- 172.21.15.32:0/501447681 >> 172.21.15.32:0/501447681 conn(0x7f79180a3050 msgr2=0x7f79180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 -- 172.21.15.32:0/501447681 shutdown_connections 2023-09-30T22:24:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.004+0000 7f791deb5700 1 -- 172.21.15.32:0/501447681 wait complete. 2023-09-30T22:24:36.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.005+0000 7f791deb5700 1 Processor -- start 2023-09-30T22:24:36.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.005+0000 7f791deb5700 1 -- start start 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f791deb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791809d630 0x7f79181c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f791deb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 0x7f79181c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f791deb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7918133930 0x7f791811e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f791deb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7918133200 con 0x7f791809d630 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f791deb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7918121e60 con 0x7f79180a0a30 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f791deb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79181bcf70 con 0x7f7918133930 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 0x7f79181c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 0x7f79181c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35460/0 (socket says 172.21.15.32:35460) 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f79177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791809d630 0x7f79181c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 -- 172.21.15.32:0/3743253092 learned_addr learned my addr 172.21.15.32:0/3743253092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f7917fff700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7918133930 0x7f791811e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 -- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7918133930 msgr2=0x7f791811e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7918133930 0x7f791811e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 -- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791809d630 msgr2=0x7f79181c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791809d630 0x7f79181c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.006+0000 7f790effd700 1 -- 172.21.15.32:0/3743253092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900009550 con 0x7f79180a0a30 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.007+0000 7f790effd700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 0x7f79181c8a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7900013000 tx=0x7f7900004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.007+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900004450 con 0x7f79180a0a30 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.007+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79000045b0 con 0x7f79180a0a30 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.007+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79181c8f60 con 0x7f79180a0a30 2023-09-30T22:24:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.007+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000038a0 con 0x7f79180a0a30 2023-09-30T22:24:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.007+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7918120460 con 0x7f79180a0a30 2023-09-30T22:24:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.008+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79180a0410 con 0x7f79180a0a30 2023-09-30T22:24:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.009+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7900021020 con 0x7f79180a0a30 2023-09-30T22:24:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.009+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f7900040e10 con 0x7f79180a0a30 2023-09-30T22:24:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.009+0000 7f79177fe700 1 -- 172.21.15.32:0/3743253092 >> v1:172.21.15.32:6816/126553 conn(0x7f78f802f510 legacy=0x7f78f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:36.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.013+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79000199a0 con 0x7f79180a0a30 2023-09-30T22:24:36.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:36.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.100+0000 7f91baffd700 1 -- 172.21.15.32:0/1455692119 >> v1:172.21.15.32:6816/126553 conn(0x7f91a402f510 legacy=0x7f91a40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.140+0000 7f287a59c700 1 -- 172.21.15.32:0/2230584170 >> v1:172.21.15.32:6816/126553 conn(0x7f285c02f510 legacy=0x7f285c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:36.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.153+0000 7f69d913e700 1 Processor -- start 2023-09-30T22:24:36.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.153+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f287c121d90 con 0x7f287c0a0a30 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.153+0000 7f69d913e700 1 -- start start 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d913e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d406e570 0x7f69d406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d913e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d409eca0 0x7f69d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d913e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d4070ee0 con 0x7f69d4134610 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d913e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4071480 con 0x7f69d409eca0 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d913e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69d406f3c0 con 0x7f69d406e570 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d406e570 0x7f69d406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d409eca0 0x7f69d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d406e570 0x7f69d406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53254/0 (socket says 172.21.15.32:53254) 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d2d9d700 1 -- 172.21.15.32:0/3164539107 learned_addr learned my addr 172.21.15.32:0/3164539107 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f2872ffd700 1 -- 172.21.15.32:0/2230584170 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f287c121d90 con 0x7f287c0a0a30 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d259c700 1 -- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d406e570 msgr2=0x7f69d406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d259c700 1 --2- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d406e570 0x7f69d406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d259c700 1 -- 172.21.15.32:0/3164539107 >> v1:172.21.15.32:6789/0 conn(0x7f69d4134610 legacy=0x7f69d4134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.154+0000 7f69d259c700 1 -- 172.21.15.32:0/3164539107 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4071b90 con 0x7f69d409eca0 2023-09-30T22:24:36.156 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d259c700 1 --2- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d409eca0 0x7f69d406ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009870 tx=0x7f69c8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c3bb36d3635601 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d1d9b700 1 -- 172.21.15.32:0/3164539107 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8011830 con 0x7f69d409eca0 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d1d9b700 1 -- 172.21.15.32:0/3164539107 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c8011990 con 0x7f69d409eca0 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d1d9b700 1 -- 172.21.15.32:0/3164539107 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8011d20 con 0x7f69d409eca0 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d913e700 1 -- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d409eca0 msgr2=0x7f69d406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d913e700 1 --2- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d409eca0 0x7f69d406ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009870 tx=0x7f69c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d913e700 1 -- 172.21.15.32:0/3164539107 shutdown_connections 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d913e700 1 --2- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d406e570 0x7f69d406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d913e700 1 --2- 172.21.15.32:0/3164539107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d409eca0 0x7f69d406ee80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.155+0000 7f69d913e700 1 -- 172.21.15.32:0/3164539107 >> 172.21.15.32:0/3164539107 conn(0x7f69d40a2aa0 msgr2=0x7f69d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:36.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.156+0000 7f69d913e700 1 -- 172.21.15.32:0/3164539107 shutdown_connections 2023-09-30T22:24:36.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.156+0000 7f69d913e700 1 -- 172.21.15.32:0/3164539107 wait complete. 2023-09-30T22:24:36.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.157+0000 7f69d913e700 1 Processor -- start 2023-09-30T22:24:36.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.157+0000 7f69d913e700 1 -- start start 2023-09-30T22:24:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d913e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69d406e570 0x7f69d41232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d913e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 0x7f69d4123820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d913e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d4123d60 0x7f69d41cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d913e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d41247b0 con 0x7f69d406e570 2023-09-30T22:24:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d913e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4133ee0 con 0x7f69d4134610 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d913e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4126500 con 0x7f69d4123d60 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 0x7f69d4123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 0x7f69d4123820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35468/0 (socket says 172.21.15.32:35468) 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 -- 172.21.15.32:0/3122081683 learned_addr learned my addr 172.21.15.32:0/3122081683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d2d9d700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69d406e570 0x7f69d41232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 >> v1:172.21.15.32:6816/126553 conn(0x7f285c02f510 legacy=0x7f285c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c0a0a30 msgr2=0x7f287c1c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c0a0a30 0x7f287c1c8640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2864016000 tx=0x7f2864004750 comp rx=0 tx=0).stop 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 shutdown_connections 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f287c133930 0x7f287c1c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f287c09d630 0x7f287c1c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 --2- 172.21.15.32:0/2230584170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f287c0a0a30 0x7f287c1c8640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 >> 172.21.15.32:0/2230584170 conn(0x7f287c0a3050 msgr2=0x7f287c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 shutdown_connections 2023-09-30T22:24:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f28808e8700 1 -- 172.21.15.32:0/2230584170 wait complete. 2023-09-30T22:24:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d359e700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d4123d60 0x7f69d41cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 -- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d4123d60 msgr2=0x7f69d41cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d4123d60 0x7f69d41cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 -- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69d406e570 msgr2=0x7f69d41232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69d406e570 0x7f69d41232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.158+0000 7f69d259c700 1 -- 172.21.15.32:0/3122081683 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80094a0 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.159+0000 7f69d259c700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 0x7f69d4123820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69c80090b0 tx=0x7f69c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.159+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8004510 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.159+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c801f070 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.159+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d40a2270 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.159+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80038e0 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.159+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d4070610 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.160+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f69c8003a40 con 0x7f69d4134610 2023-09-30T22:24:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.160+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f69c8021070 con 0x7f69d4134610 2023-09-30T22:24:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.160+0000 7f69d2d9d700 1 -- 172.21.15.32:0/3122081683 >> v1:172.21.15.32:6816/126553 conn(0x7f69c002f510 legacy=0x7f69c00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.160+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d4074ea0 con 0x7f69d4134610 2023-09-30T22:24:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.169+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69c8015080 con 0x7f69d4134610 2023-09-30T22:24:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.170+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f91c4074ea0 con 0x7f91c409e930 2023-09-30T22:24:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.170+0000 7f91c0ff9700 1 -- 172.21.15.32:0/1455692119 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91b4033030 con 0x7f91c409e930 2023-09-30T22:24:36.170 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 >> v1:172.21.15.32:6816/126553 conn(0x7f91a402f510 legacy=0x7f91a40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 msgr2=0x7f91c406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 0x7f91c406b8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009fd0 tx=0x7f91b4004750 comp rx=0 tx=0).stop 2023-09-30T22:24:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 shutdown_connections 2023-09-30T22:24:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c409f650 0x7f91c406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c409e930 0x7f91c406b8b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 --2- 172.21.15.32:0/1455692119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c409f240 0x7f91c406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 >> 172.21.15.32:0/1455692119 conn(0x7f91c40a4210 msgr2=0x7f91c41b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 shutdown_connections 2023-09-30T22:24:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.173+0000 7f91c9cd8700 1 -- 172.21.15.32:0/1455692119 wait complete. 2023-09-30T22:24:36.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:36.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:36.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.210+0000 7f79177fe700 1 -- 172.21.15.32:0/3743253092 >> v1:172.21.15.32:6816/126553 conn(0x7f78f802f510 legacy=0x7f78f80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.227+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79180747f0 con 0x7f79180a0a30 2023-09-30T22:24:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.228+0000 7f7914ff9700 1 -- 172.21.15.32:0/3743253092 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7900029070 con 0x7f79180a0a30 2023-09-30T22:24:36.228 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 >> v1:172.21.15.32:6816/126553 conn(0x7f78f802f510 legacy=0x7f78f80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 msgr2=0x7f79181c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 0x7f79181c8a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7900013000 tx=0x7f7900004750 comp rx=0 tx=0).stop 2023-09-30T22:24:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 shutdown_connections 2023-09-30T22:24:36.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7918133930 0x7f791811e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f791809d630 0x7f79181c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 --2- 172.21.15.32:0/3743253092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79180a0a30 0x7f79181c8a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 >> 172.21.15.32:0/3743253092 conn(0x7f79180a3050 msgr2=0x7f791809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 shutdown_connections 2023-09-30T22:24:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.230+0000 7f791deb5700 1 -- 172.21.15.32:0/3743253092 wait complete. 2023-09-30T22:24:36.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:36.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.361+0000 7f69d2d9d700 1 -- 172.21.15.32:0/3122081683 >> v1:172.21.15.32:6816/126553 conn(0x7f69c002f510 legacy=0x7f69c00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.379+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f69d4071100 con 0x7f69d4134610 2023-09-30T22:24:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.379+0000 7f69bb7fe700 1 -- 172.21.15.32:0/3122081683 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f69c802b020 con 0x7f69d4134610 2023-09-30T22:24:36.379 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 >> v1:172.21.15.32:6816/126553 conn(0x7f69c002f510 legacy=0x7f69c00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 msgr2=0x7f69d4123820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:36.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 0x7f69d4123820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69c80090b0 tx=0x7f69c8004750 comp rx=0 tx=0).stop 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 shutdown_connections 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69d4123d60 0x7f69d41cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69d406e570 0x7f69d41232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 --2- 172.21.15.32:0/3122081683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69d4134610 0x7f69d4123820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 >> 172.21.15.32:0/3122081683 conn(0x7f69d40a2aa0 msgr2=0x7f69d40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 shutdown_connections 2023-09-30T22:24:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:36.381+0000 7f69d913e700 1 -- 172.21.15.32:0/3122081683 wait complete. 2023-09-30T22:24:36.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:37.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:37.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:37.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.335+0000 7f4e1218a700 1 Processor -- start 2023-09-30T22:24:37.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.335+0000 7f4e1218a700 1 -- start start 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e1218a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e0c1316c0 0x7f4e0c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e1218a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c09d9f0 0x7f4e0c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e1218a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c09e130 con 0x7f4e0c06acb0 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e1218a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c1312b0 con 0x7f4e0c1316c0 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e1218a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c09f6e0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e0affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c09d9f0 0x7f4e0c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e0bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e0c06acb0 0x7f4e0c138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51482/0 (socket says 172.21.15.32:51482) 2023-09-30T22:24:37.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e0bfff700 1 -- 172.21.15.32:0/1656701513 learned_addr learned my addr 172.21.15.32:0/1656701513 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:37.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.336+0000 7f4e0b7fe700 1 --2- 172.21.15.32:0/1656701513 >> v2:172.21.15.32:3300/0 conn(0x7f4e0c1316c0 0x7f4e0c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0affd700 1 -- 172.21.15.32:0/1656701513 >> v2:172.21.15.32:3300/0 conn(0x7f4e0c1316c0 msgr2=0x7f4e0c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0affd700 1 --2- 172.21.15.32:0/1656701513 >> v2:172.21.15.32:3300/0 conn(0x7f4e0c1316c0 0x7f4e0c131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0affd700 1 -- 172.21.15.32:0/1656701513 >> v1:172.21.15.32:6789/0 conn(0x7f4e0c06acb0 legacy=0x7f4e0c138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0affd700 1 -- 172.21.15.32:0/1656701513 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c09e760 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0b7fe700 1 --2- 172.21.15.32:0/1656701513 >> v2:172.21.15.32:3300/0 conn(0x7f4e0c1316c0 0x7f4e0c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0affd700 1 --2- 172.21.15.32:0/1656701513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c09d9f0 0x7f4e0c06a770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009840 tx=0x7f4e00009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8b69cfb77482a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0a7fc700 1 -- 172.21.15.32:0/1656701513 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004230 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0a7fc700 1 -- 172.21.15.32:0/1656701513 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e00004390 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.337+0000 7f4e0a7fc700 1 -- 172.21.15.32:0/1656701513 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000036a0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 -- 172.21.15.32:0/1656701513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c09d9f0 msgr2=0x7f4e0c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 --2- 172.21.15.32:0/1656701513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c09d9f0 0x7f4e0c06a770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009840 tx=0x7f4e00009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 -- 172.21.15.32:0/1656701513 shutdown_connections 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 --2- 172.21.15.32:0/1656701513 >> v2:172.21.15.32:3300/0 conn(0x7f4e0c1316c0 0x7f4e0c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 --2- 172.21.15.32:0/1656701513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c09d9f0 0x7f4e0c06a770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 -- 172.21.15.32:0/1656701513 >> 172.21.15.32:0/1656701513 conn(0x7f4e0c066f50 msgr2=0x7f4e0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 -- 172.21.15.32:0/1656701513 shutdown_connections 2023-09-30T22:24:37.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.338+0000 7f4e1218a700 1 -- 172.21.15.32:0/1656701513 wait complete. 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.339+0000 7f4e1218a700 1 Processor -- start 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.339+0000 7f4e1218a700 1 -- start start 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e1218a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c06acb0 0x7f4e0c06e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e1218a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 0x7f4e0c06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e1218a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1316c0 0x7f4e0c06f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e1218a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c127ca0 con 0x7f4e0c1316c0 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e1218a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0728d0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e1218a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0a3660 con 0x7f4e0c06acb0 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 0x7f4e0c06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 0x7f4e0c06ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35480/0 (socket says 172.21.15.32:35480) 2023-09-30T22:24:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 -- 172.21.15.32:0/4247147340 learned_addr learned my addr 172.21.15.32:0/4247147340 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0bfff700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1316c0 0x7f4e0c06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0b7fe700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c06acb0 0x7f4e0c06e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 -- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c06acb0 msgr2=0x7f4e0c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c06acb0 0x7f4e0c06e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 -- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1316c0 msgr2=0x7f4e0c06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1316c0 0x7f4e0c06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0affd700 1 -- 172.21.15.32:0/4247147340 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc00d940 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.340+0000 7f4e0bfff700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1316c0 0x7f4e0c06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.341+0000 7f4e0affd700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 0x7f4e0c06ee20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e000090b0 tx=0x7f4e00004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.341+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000045d0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.341+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0001f070 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.341+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e000094a0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.341+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00011850 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.341+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c09ead0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.342+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4e000119b0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.342+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4e00021070 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.342+0000 7f4e0b7fe700 1 -- 172.21.15.32:0/4247147340 >> v1:172.21.15.32:6816/126553 conn(0x7f4df802f510 legacy=0x7f4df80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.343+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c074ea0 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.347+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e00024380 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.375+0000 7fe6078aa700 1 Processor -- start 2023-09-30T22:24:37.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.375+0000 7fe6078aa700 1 -- start start 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe6078aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe6078aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe60006b080 0x7fe600131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe6078aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe600131b40 0x7fe600131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe6078aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60009ece0 con 0x7fe600131b40 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe6078aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60009e760 con 0x7fe60006b080 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe6078aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe60009e040 con 0x7fe60006a770 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe605e47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe600131b40 0x7fe600131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe605e47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe600131b40 0x7fe600131fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39306/0 (socket says 172.21.15.32:39306) 2023-09-30T22:24:37.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe605e47700 1 -- 172.21.15.32:0/269528467 learned_addr learned my addr 172.21.15.32:0/269528467 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe604e45700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe60006b080 0x7fe600131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.376+0000 7fe605646700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe605e47700 1 -- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 msgr2=0x7fe60006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe605e47700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe605e47700 1 -- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe60006b080 msgr2=0x7fe600131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe605e47700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe60006b080 0x7fe600131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe605e47700 1 -- 172.21.15.32:0/269528467 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60009f630 con 0x7fe600131b40 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe604e45700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe60006b080 0x7fe600131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe605e47700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe600131b40 0x7fe600131fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009e50 tx=0x7fe5f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8e68b1d6bfc48f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe5f7fff700 1 -- 172.21.15.32:0/269528467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800e070 con 0x7fe600131b40 2023-09-30T22:24:37.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe5f7fff700 1 -- 172.21.15.32:0/269528467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f800b910 con 0x7fe600131b40 2023-09-30T22:24:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.377+0000 7fe5f7fff700 1 -- 172.21.15.32:0/269528467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800bcc0 con 0x7fe600131b40 2023-09-30T22:24:37.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 -- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe600131b40 msgr2=0x7fe600131fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe600131b40 0x7fe600131fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009e50 tx=0x7fe5f800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 -- 172.21.15.32:0/269528467 shutdown_connections 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe600131b40 0x7fe600131fe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 --2- 172.21.15.32:0/269528467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe60006b080 0x7fe600131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 -- 172.21.15.32:0/269528467 >> 172.21.15.32:0/269528467 conn(0x7fe600066f50 msgr2=0x7fe60009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 -- 172.21.15.32:0/269528467 shutdown_connections 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.378+0000 7fe6078aa700 1 -- 172.21.15.32:0/269528467 wait complete. 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.379+0000 7fe6078aa700 1 Processor -- start 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.379+0000 7fe6078aa700 1 -- start start 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe6078aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60011ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe6078aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe60006b080 0x7fe6001204c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe6078aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 0x7fe600120a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe6078aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe600123aa0 con 0x7fe60006b080 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe6078aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe600122a30 con 0x7fe600131b40 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe6078aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe600123500 con 0x7fe60006a770 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe605e47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 0x7fe600120a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe605e47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 0x7fe600120a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35494/0 (socket says 172.21.15.32:35494) 2023-09-30T22:24:37.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe605646700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60011ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe605e47700 1 -- 172.21.15.32:0/3500976372 learned_addr learned my addr 172.21.15.32:0/3500976372 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.380+0000 7fe604e45700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe60006b080 0x7fe6001204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605e47700 1 -- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 msgr2=0x7fe60011ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605e47700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60011ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605e47700 1 -- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe60006b080 msgr2=0x7fe6001204c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605e47700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe60006b080 0x7fe6001204c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605e47700 1 -- 172.21.15.32:0/3500976372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00094a0 con 0x7fe600131b40 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605646700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60011ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe604e45700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe60006b080 0x7fe6001204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe605e47700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 0x7fe600120a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe60009ee20 tx=0x7fe5f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004450 con 0x7fe600131b40 2023-09-30T22:24:37.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f80045f0 con 0x7fe600131b40 2023-09-30T22:24:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f8009b30 con 0x7fe600131b40 2023-09-30T22:24:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.381+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80099b0 con 0x7fe600131b40 2023-09-30T22:24:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.382+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe600121350 con 0x7fe600131b40 2023-09-30T22:24:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.382+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe5f801c410 con 0x7fe600131b40 2023-09-30T22:24:37.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.383+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60011cc30 con 0x7fe600131b40 2023-09-30T22:24:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.383+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fe5f8041780 con 0x7fe600131b40 2023-09-30T22:24:37.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.383+0000 7fe605646700 1 -- 172.21.15.32:0/3500976372 >> v1:172.21.15.32:6816/126553 conn(0x7fe5e402f510 legacy=0x7fe5e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.388+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5f800ce80 con 0x7fe600131b40 2023-09-30T22:24:37.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.411+0000 7f7f04b50700 1 Processor -- start 2023-09-30T22:24:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.411+0000 7f7f04b50700 1 -- start start 2023-09-30T22:24:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7f04b50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f001309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7f04b50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7f04b50700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f0006ff90 0x7f7f00137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7f04b50700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f001301c0 con 0x7f7f0006ff90 2023-09-30T22:24:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7f04b50700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00071ff0 con 0x7f7f0006f660 2023-09-30T22:24:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7f04b50700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00070e70 con 0x7f7f001305f0 2023-09-30T22:24:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7efe59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f001309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7efdd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7efed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f0006ff90 0x7f7f00137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7efe59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f001309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53296/0 (socket says 172.21.15.32:53296) 2023-09-30T22:24:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7efe59c700 1 -- 172.21.15.32:0/740777250 learned_addr learned my addr 172.21.15.32:0/740777250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.412+0000 7f7efdd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0006fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35510/0 (socket says 172.21.15.32:35510) 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efdd9b700 1 -- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 msgr2=0x7f7f001309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efdd9b700 1 --2- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f001309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efdd9b700 1 -- 172.21.15.32:0/740777250 >> v2:172.21.15.32:3300/0 conn(0x7f7f0006ff90 msgr2=0x7f7f00137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efdd9b700 1 --2- 172.21.15.32:0/740777250 >> v2:172.21.15.32:3300/0 conn(0x7f7f0006ff90 0x7f7f00137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efdd9b700 1 -- 172.21.15.32:0/740777250 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f000722b0 con 0x7f7f0006f660 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efe59c700 1 --2- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f001309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efdd9b700 1 --2- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0006fa50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009840 tx=0x7f7ef4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f576ca56c1c163 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efd59a700 1 -- 172.21.15.32:0/740777250 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4010a90 con 0x7f7f0006f660 2023-09-30T22:24:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efd59a700 1 -- 172.21.15.32:0/740777250 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4010bf0 con 0x7f7f0006f660 2023-09-30T22:24:37.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.413+0000 7f7efd59a700 1 -- 172.21.15.32:0/740777250 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4018440 con 0x7f7f0006f660 2023-09-30T22:24:37.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 -- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 msgr2=0x7f7f0006fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 --2- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0006fa50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009840 tx=0x7f7ef4009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:37.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 -- 172.21.15.32:0/740777250 shutdown_connections 2023-09-30T22:24:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 --2- 172.21.15.32:0/740777250 >> v2:172.21.15.32:3300/0 conn(0x7f7f0006ff90 0x7f7f00137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 --2- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f001309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 --2- 172.21.15.32:0/740777250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0006fa50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.414+0000 7f7f04b50700 1 -- 172.21.15.32:0/740777250 >> 172.21.15.32:0/740777250 conn(0x7f7f000a2aa0 msgr2=0x7f7f000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.415+0000 7f7f04b50700 1 -- 172.21.15.32:0/740777250 shutdown_connections 2023-09-30T22:24:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.415+0000 7f7f04b50700 1 -- 172.21.15.32:0/740777250 wait complete. 2023-09-30T22:24:37.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.416+0000 7f7f04b50700 1 Processor -- start 2023-09-30T22:24:37.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.416+0000 7f7f04b50700 1 -- start start 2023-09-30T22:24:37.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7f04b50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0011b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7f04b50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f0006ff90 0x7f7f0011bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7f04b50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f0011c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7f04b50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0011f500 con 0x7f7f0006ff90 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7f04b50700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0011ed30 con 0x7f7f0006f660 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7f04b50700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00071820 con 0x7f7f001305f0 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7efed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f0011c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7efe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0011b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7efed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f0011c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53304/0 (socket says 172.21.15.32:53304) 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7efed9d700 1 -- 172.21.15.32:0/702057170 learned_addr learned my addr 172.21.15.32:0/702057170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.417+0000 7f7efe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0011b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35526/0 (socket says 172.21.15.32:35526) 2023-09-30T22:24:37.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efdd9b700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f0006ff90 0x7f7f0011bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efed9d700 1 -- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 msgr2=0x7f7f0011b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efed9d700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0011b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efed9d700 1 -- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f0006ff90 msgr2=0x7f7f0011bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efed9d700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f0006ff90 0x7f7f0011bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efed9d700 1 -- 172.21.15.32:0/702057170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0009550 con 0x7f7f001305f0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7efed9d700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f0011c460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800efc0 tx=0x7f7ef800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8009ea0 con 0x7f7f001305f0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef800f040 con 0x7f7f001305f0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800ced0 con 0x7f7f001305f0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.418+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef40094a0 con 0x7f7f001305f0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.419+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0011dba0 con 0x7f7f001305f0 2023-09-30T22:24:37.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.419+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0012eca0 con 0x7f7f001305f0 2023-09-30T22:24:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.420+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f7ef8014480 con 0x7f7f001305f0 2023-09-30T22:24:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.420+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f7ef8040c60 con 0x7f7f001305f0 2023-09-30T22:24:37.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.420+0000 7f7efe59c700 1 -- 172.21.15.32:0/702057170 >> v1:172.21.15.32:6816/126553 conn(0x7f7ee002f510 legacy=0x7f7ee00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.429+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ef8012930 con 0x7f7f001305f0 2023-09-30T22:24:37.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.542+0000 7f0c5979e700 1 Processor -- start 2023-09-30T22:24:37.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f4e0b7fe700 1 -- 172.21.15.32:0/4247147340 >> v1:172.21.15.32:6816/126553 conn(0x7f4df802f510 legacy=0x7f4df80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c5979e700 1 -- start start 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c5979e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5409dba0 0x7f0c5409df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c5979e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c541303e0 0x7f0c54137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c5979e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5409d670 con 0x7f0c5412f8c0 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c5979e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c54099a00 con 0x7f0c541303e0 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c5979e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54099d80 con 0x7f0c5409dba0 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c537fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c5412f8c0 0x7f0c5412fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51490/0 (socket says 172.21.15.32:51490) 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c537fe700 1 -- 172.21.15.32:0/3938343196 learned_addr learned my addr 172.21.15.32:0/3938343196 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c58f9d700 1 --2- 172.21.15.32:0/3938343196 >> v2:172.21.15.32:3300/0 conn(0x7f0c541303e0 0x7f0c54137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.543+0000 7f0c53fff700 1 --2- 172.21.15.32:0/3938343196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5409dba0 0x7f0c5409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c53fff700 1 -- 172.21.15.32:0/3938343196 >> v2:172.21.15.32:3300/0 conn(0x7f0c541303e0 msgr2=0x7f0c54137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c53fff700 1 --2- 172.21.15.32:0/3938343196 >> v2:172.21.15.32:3300/0 conn(0x7f0c541303e0 0x7f0c54137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c53fff700 1 -- 172.21.15.32:0/3938343196 >> v1:172.21.15.32:6789/0 conn(0x7f0c5412f8c0 legacy=0x7f0c5412fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c53fff700 1 -- 172.21.15.32:0/3938343196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5409b020 con 0x7f0c5409dba0 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c58f9d700 1 --2- 172.21.15.32:0/3938343196 >> v2:172.21.15.32:3300/0 conn(0x7f0c541303e0 0x7f0c54137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c53fff700 1 --2- 172.21.15.32:0/3938343196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5409dba0 0x7f0c5409df70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0c44008120 tx=0x7f0c4400abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec8b7c4dc12c56d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c52ffd700 1 -- 172.21.15.32:0/3938343196 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44005c20 con 0x7f0c5409dba0 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c52ffd700 1 -- 172.21.15.32:0/3938343196 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c44005d80 con 0x7f0c5409dba0 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.544+0000 7f0c52ffd700 1 -- 172.21.15.32:0/3938343196 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44004250 con 0x7f0c5409dba0 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 -- 172.21.15.32:0/3938343196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5409dba0 msgr2=0x7f0c5409df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 --2- 172.21.15.32:0/3938343196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5409dba0 0x7f0c5409df70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0c44008120 tx=0x7f0c4400abb0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 -- 172.21.15.32:0/3938343196 shutdown_connections 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 --2- 172.21.15.32:0/3938343196 >> v2:172.21.15.32:3300/0 conn(0x7f0c541303e0 0x7f0c54137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 --2- 172.21.15.32:0/3938343196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5409dba0 0x7f0c5409df70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 -- 172.21.15.32:0/3938343196 >> 172.21.15.32:0/3938343196 conn(0x7f0c540a2540 msgr2=0x7f0c540a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 -- 172.21.15.32:0/3938343196 shutdown_connections 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.545+0000 7f0c5979e700 1 -- 172.21.15.32:0/3938343196 wait complete. 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.547+0000 7f0c5979e700 1 Processor -- start 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.547+0000 7f0c5979e700 1 -- start start 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c5979e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0c5409dba0 0x7f0c541c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c5979e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 0x7f0c541c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c5979e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c541303e0 0x7f0c541c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c5979e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c541c5ce0 con 0x7f0c541303e0 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c5979e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5412d300 con 0x7f0c5409dba0 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c5979e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5406fe30 con 0x7f0c5412f8c0 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 0x7f0c541c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 0x7f0c541c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53322/0 (socket says 172.21.15.32:53322) 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c537fe700 1 -- 172.21.15.32:0/118523870 learned_addr learned my addr 172.21.15.32:0/118523870 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c58f9d700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c541303e0 0x7f0c541c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c58f9d700 1 -- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 msgr2=0x7f0c541c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c58f9d700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 0x7f0c541c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c58f9d700 1 -- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0c5409dba0 msgr2=0x7f0c541c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c58f9d700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0c5409dba0 0x7f0c541c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.548+0000 7f0c58f9d700 1 -- 172.21.15.32:0/118523870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c009a70 con 0x7f0c541303e0 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c537fe700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 0x7f0c541c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c58f9d700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c541303e0 0x7f0c541c8250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00e5b0 tx=0x7f0c4c00e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c003a20 con 0x7f0c541303e0 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c003ba0 con 0x7f0c541303e0 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4400a4e0 con 0x7f0c541303e0 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c004020 con 0x7f0c541303e0 2023-09-30T22:24:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.549+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c540709c0 con 0x7f0c541303e0 2023-09-30T22:24:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.550+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c54099680 con 0x7f0c541303e0 2023-09-30T22:24:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.551+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0c4c004180 con 0x7f0c541303e0 2023-09-30T22:24:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.551+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0c4c041b10 con 0x7f0c541303e0 2023-09-30T22:24:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.551+0000 7f0c53fff700 1 -- 172.21.15.32:0/118523870 >> v1:172.21.15.32:6816/126553 conn(0x7f0c4002f510 legacy=0x7f0c400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.556+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c4c045060 con 0x7f0c541303e0 2023-09-30T22:24:37.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.570+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e0c130270 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.571+0000 7f4df3fff700 1 -- 172.21.15.32:0/4247147340 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e00025b90 con 0x7f4e0c09d9f0 2023-09-30T22:24:37.571 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 >> v1:172.21.15.32:6816/126553 conn(0x7f4df802f510 legacy=0x7f4df80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:37.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 msgr2=0x7f4e0c06ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 0x7f4e0c06ee20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e000090b0 tx=0x7f4e00004750 comp rx=0 tx=0).stop 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 shutdown_connections 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1316c0 0x7f4e0c06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e0c06acb0 0x7f4e0c06e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 --2- 172.21.15.32:0/4247147340 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e0c09d9f0 0x7f4e0c06ee20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 >> 172.21.15.32:0/4247147340 conn(0x7f4e0c066f50 msgr2=0x7f4e0c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 shutdown_connections 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.573+0000 7f4e1218a700 1 -- 172.21.15.32:0/4247147340 wait complete. 2023-09-30T22:24:37.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.584+0000 7fe605646700 1 -- 172.21.15.32:0/3500976372 >> v1:172.21.15.32:6816/126553 conn(0x7fe5e402f510 legacy=0x7fe5e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.606+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe600074ea0 con 0x7fe600131b40 2023-09-30T22:24:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.607+0000 7fe5f5ffb700 1 -- 172.21.15.32:0/3500976372 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5f802a020 con 0x7fe600131b40 2023-09-30T22:24:37.607 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:37.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 >> v1:172.21.15.32:6816/126553 conn(0x7fe5e402f510 legacy=0x7fe5e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:37.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 msgr2=0x7fe600120a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 0x7fe600120a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe60009ee20 tx=0x7fe5f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 shutdown_connections 2023-09-30T22:24:37.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe600131b40 0x7fe600120a00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe60006a770 0x7fe60011ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 --2- 172.21.15.32:0/3500976372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe60006b080 0x7fe6001204c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 >> 172.21.15.32:0/3500976372 conn(0x7fe600066f50 msgr2=0x7fe600071460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 shutdown_connections 2023-09-30T22:24:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.609+0000 7fe6078aa700 1 -- 172.21.15.32:0/3500976372 wait complete. 2023-09-30T22:24:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.621+0000 7f7efe59c700 1 -- 172.21.15.32:0/702057170 >> v1:172.21.15.32:6816/126553 conn(0x7f7ee002f510 legacy=0x7f7ee00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:37.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.641+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f000a0620 con 0x7f7f001305f0 2023-09-30T22:24:37.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.641+0000 7f7eeeffd700 1 -- 172.21.15.32:0/702057170 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ef8012340 con 0x7f7f001305f0 2023-09-30T22:24:37.641 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 >> v1:172.21.15.32:6816/126553 conn(0x7f7ee002f510 legacy=0x7f7ee00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 msgr2=0x7f7f0011c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f0011c460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800efc0 tx=0x7f7ef800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 shutdown_connections 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f001305f0 0x7f7f0011c460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f0006f660 0x7f7f0011b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 --2- 172.21.15.32:0/702057170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f0006ff90 0x7f7f0011bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 >> 172.21.15.32:0/702057170 conn(0x7f7f000a2aa0 msgr2=0x7f7f0006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 shutdown_connections 2023-09-30T22:24:37.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.644+0000 7f7f04b50700 1 -- 172.21.15.32:0/702057170 wait complete. 2023-09-30T22:24:37.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:37.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.752+0000 7f0c53fff700 1 -- 172.21.15.32:0/118523870 >> v1:172.21.15.32:6816/126553 conn(0x7f0c4002f510 legacy=0x7f0c400319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.766+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0c5404f2a0 con 0x7f0c541303e0 2023-09-30T22:24:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.766+0000 7f0c50ff9700 1 -- 172.21.15.32:0/118523870 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c4c045090 con 0x7f0c541303e0 2023-09-30T22:24:37.766 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 >> v1:172.21.15.32:6816/126553 conn(0x7f0c4002f510 legacy=0x7f0c400319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c541303e0 msgr2=0x7f0c541c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c541303e0 0x7f0c541c8250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00e5b0 tx=0x7f0c4c00e8c0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 shutdown_connections 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c541303e0 0x7f0c541c8250 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0c5409dba0 0x7f0c541c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 --2- 172.21.15.32:0/118523870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0c5412f8c0 0x7f0c541c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:37.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 >> 172.21.15.32:0/118523870 conn(0x7f0c540a2540 msgr2=0x7f0c5412d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 shutdown_connections 2023-09-30T22:24:37.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:37.769+0000 7f0c5979e700 1 -- 172.21.15.32:0/118523870 wait complete. 2023-09-30T22:24:37.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:38.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:38.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:38.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.744+0000 7f59bb74f700 1 Processor -- start 2023-09-30T22:24:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.744+0000 7f59bb74f700 1 -- start start 2023-09-30T22:24:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59bb74f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 0x7f59b4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59bb74f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f59b406ae80 0x7f59b4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59bb74f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b409e130 con 0x7f59b409d9f0 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59bb74f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b41312b0 con 0x7f59b406ae80 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59bb74f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b409f6e0 con 0x7f59b41316c0 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 0x7f59b4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 0x7f59b4131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35548/0 (socket says 172.21.15.32:35548) 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 -- 172.21.15.32:0/3125741725 learned_addr learned my addr 172.21.15.32:0/3125741725 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b9cec700 1 --2- 172.21.15.32:0/3125741725 >> v2:172.21.15.32:3300/0 conn(0x7f59b406ae80 0x7f59b4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 -- 172.21.15.32:0/3125741725 >> v2:172.21.15.32:3300/0 conn(0x7f59b406ae80 msgr2=0x7f59b4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 --2- 172.21.15.32:0/3125741725 >> v2:172.21.15.32:3300/0 conn(0x7f59b406ae80 0x7f59b4139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 -- 172.21.15.32:0/3125741725 >> v1:172.21.15.32:6789/0 conn(0x7f59b409d9f0 legacy=0x7f59b406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59abfff700 1 -- 172.21.15.32:0/3125741725 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517959607 0 0) 0x7f59a4001110 con 0x7f59b409d9f0 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.745+0000 7f59b94eb700 1 -- 172.21.15.32:0/3125741725 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b409e760 con 0x7f59b41316c0 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.746+0000 7f59b94eb700 1 --2- 172.21.15.32:0/3125741725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 0x7f59b4131a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59b00098f0 tx=0x7f59b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf13d1005fd4220 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.746+0000 7f59abfff700 1 -- 172.21.15.32:0/3125741725 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011830 con 0x7f59b41316c0 2023-09-30T22:24:38.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.746+0000 7f59abfff700 1 -- 172.21.15.32:0/3125741725 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b0011990 con 0x7f59b41316c0 2023-09-30T22:24:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.746+0000 7f59abfff700 1 -- 172.21.15.32:0/3125741725 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011d20 con 0x7f59b41316c0 2023-09-30T22:24:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.746+0000 7f59bb74f700 1 -- 172.21.15.32:0/3125741725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 msgr2=0x7f59b4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.746+0000 7f59bb74f700 1 --2- 172.21.15.32:0/3125741725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 0x7f59b4131a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59b00098f0 tx=0x7f59b0009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.747+0000 7f59bb74f700 1 -- 172.21.15.32:0/3125741725 shutdown_connections 2023-09-30T22:24:38.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.747+0000 7f59bb74f700 1 --2- 172.21.15.32:0/3125741725 >> v2:172.21.15.32:3300/0 conn(0x7f59b406ae80 0x7f59b4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.747+0000 7f59bb74f700 1 --2- 172.21.15.32:0/3125741725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b41316c0 0x7f59b4131a90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.747+0000 7f59bb74f700 1 -- 172.21.15.32:0/3125741725 >> 172.21.15.32:0/3125741725 conn(0x7f59b4066f50 msgr2=0x7f59b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.747+0000 7f59bb74f700 1 -- 172.21.15.32:0/3125741725 shutdown_connections 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.747+0000 7f59bb74f700 1 -- 172.21.15.32:0/3125741725 wait complete. 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 Processor -- start 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 -- start start 2023-09-30T22:24:38.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 0x7f59b411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b409d9f0 0x7f59b411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b41316c0 0x7f59b411e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b411ec90 con 0x7f59b41316c0 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b412b900 con 0x7f59b409d9f0 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59bb74f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4072500 con 0x7f59b406ae80 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59b94eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 0x7f59b411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59b94eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 0x7f59b411d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53338/0 (socket says 172.21.15.32:53338) 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59b94eb700 1 -- 172.21.15.32:0/2040684961 learned_addr learned my addr 172.21.15.32:0/2040684961 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.748+0000 7f59b9cec700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b41316c0 0x7f59b411e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b8cea700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b409d9f0 0x7f59b411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b94eb700 1 -- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b409d9f0 msgr2=0x7f59b411ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b94eb700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b409d9f0 0x7f59b411ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b94eb700 1 -- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b41316c0 msgr2=0x7f59b411e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b94eb700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b41316c0 0x7f59b411e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b94eb700 1 -- 172.21.15.32:0/2040684961 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0009550 con 0x7f59b406ae80 2023-09-30T22:24:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59b94eb700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 0x7f59b411d890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009c40 tx=0x7f59b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004590 con 0x7f59b406ae80 2023-09-30T22:24:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b00036a0 con 0x7f59b406ae80 2023-09-30T22:24:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b40a3540 con 0x7f59b406ae80 2023-09-30T22:24:38.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.749+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0003a30 con 0x7f59b406ae80 2023-09-30T22:24:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.750+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b4071b30 con 0x7f59b406ae80 2023-09-30T22:24:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.750+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f59b0021070 con 0x7f59b406ae80 2023-09-30T22:24:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.751+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b411fd70 con 0x7f59b406ae80 2023-09-30T22:24:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.751+0000 7f59b8cea700 1 -- 172.21.15.32:0/2040684961 >> v1:172.21.15.32:6816/126553 conn(0x7f599802f510 legacy=0x7f59980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:38.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.751+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f59b0040e50 con 0x7f59b406ae80 2023-09-30T22:24:38.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.759+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59b000b3e0 con 0x7f59b406ae80 2023-09-30T22:24:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.780+0000 7f310f6a7700 1 Processor -- start 2023-09-30T22:24:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310f6a7700 1 -- start start 2023-09-30T22:24:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310f6a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 0x7f310809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310f6a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f31081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310f6a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3108070940 con 0x7f310809f650 2023-09-30T22:24:38.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310f6a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310806fd60 con 0x7f310809e930 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310f6a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f310806fa80 con 0x7f310809f240 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310cc42700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f31081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.781+0000 7f310d443700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 0x7f310809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310cc42700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f31081317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53350/0 (socket says 172.21.15.32:53350) 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310d443700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 0x7f310809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35566/0 (socket says 172.21.15.32:35566) 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310cc42700 1 -- 172.21.15.32:0/2156348247 learned_addr learned my addr 172.21.15.32:0/2156348247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310d443700 1 -- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 msgr2=0x7f31081317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310d443700 1 --2- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f31081317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310d443700 1 -- 172.21.15.32:0/2156348247 >> v1:172.21.15.32:6789/0 conn(0x7f310809f650 legacy=0x7f3108131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310d443700 1 -- 172.21.15.32:0/2156348247 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310809fb60 con 0x7f310809e930 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310cc42700 1 --2- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f31081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f310d443700 1 --2- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 0x7f310809ed00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3104009870 tx=0x7f3104009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ab5c6721d3162d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f30fffff700 1 -- 172.21.15.32:0/2156348247 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104011830 con 0x7f310809e930 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f30fffff700 1 -- 172.21.15.32:0/2156348247 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104011990 con 0x7f310809e930 2023-09-30T22:24:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.782+0000 7f30fffff700 1 -- 172.21.15.32:0/2156348247 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104011d20 con 0x7f310809e930 2023-09-30T22:24:38.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 -- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 msgr2=0x7f310809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 --2- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 0x7f310809ed00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3104009870 tx=0x7f3104009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 -- 172.21.15.32:0/2156348247 shutdown_connections 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 --2- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809e930 0x7f310809ed00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 --2- 172.21.15.32:0/2156348247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f31081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 -- 172.21.15.32:0/2156348247 >> 172.21.15.32:0/2156348247 conn(0x7f31080a4210 msgr2=0x7f31080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 -- 172.21.15.32:0/2156348247 shutdown_connections 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.783+0000 7f310f6a7700 1 -- 172.21.15.32:0/2156348247 wait complete. 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.784+0000 7f310f6a7700 1 Processor -- start 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.784+0000 7f310f6a7700 1 -- start start 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310f6a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f310809e930 0x7f310806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310f6a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f310806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310f6a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 0x7f310806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310f6a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31081a9a00 con 0x7f310809e930 2023-09-30T22:24:38.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310f6a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31081a89a0 con 0x7f310809f650 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310f6a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31081a9190 con 0x7f310809f240 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 0x7f310806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 0x7f310806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35574/0 (socket says 172.21.15.32:35574) 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310cc42700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f310806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 -- 172.21.15.32:0/870800678 learned_addr learned my addr 172.21.15.32:0/870800678 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310d443700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f310809e930 0x7f310806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 -- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 msgr2=0x7f310806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f310806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 -- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f310809e930 msgr2=0x7f310806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f310809e930 0x7f310806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.785+0000 7f310dc44700 1 -- 172.21.15.32:0/870800678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8009550 con 0x7f310809f650 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f310cc42700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f310806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f310dc44700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 0x7f310806c330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f310000cbc0 tx=0x7f310000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100003c00 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100003d60 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310000ab40 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31040094a0 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.786+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31080a0190 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.787+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f310000aca0 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.787+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f310003fb70 con 0x7f310809f650 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.787+0000 7f310d443700 1 -- 172.21.15.32:0/870800678 >> v1:172.21.15.32:6816/126553 conn(0x7f30ec02f510 legacy=0x7f30ec0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.787+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31081a0b30 con 0x7f310809f650 2023-09-30T22:24:38.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.793+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31000198f0 con 0x7f310809f650 2023-09-30T22:24:38.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.826+0000 7f72c0dac700 1 Processor -- start 2023-09-30T22:24:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.827+0000 7f72c0dac700 1 -- start start 2023-09-30T22:24:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.827+0000 7f72c0dac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72bc06a770 0x7f72bc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.827+0000 7f72c0dac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06b080 0x7f72bc131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.827+0000 7f72c0dac700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc09ece0 con 0x7f72bc131b40 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.827+0000 7f72c0dac700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc09e760 con 0x7f72bc06a770 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.827+0000 7f72c0dac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc09e040 con 0x7f72bc06b080 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06b080 0x7f72bc131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72bad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72bc131b40 0x7f72bc138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51518/0 (socket says 172.21.15.32:51518) 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72bad9d700 1 -- 172.21.15.32:0/1604055583 learned_addr learned my addr 172.21.15.32:0/1604055583 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:38.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72ba59c700 1 --2- 172.21.15.32:0/1604055583 >> v2:172.21.15.32:3300/0 conn(0x7f72bc06a770 0x7f72bc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b9d9b700 1 -- 172.21.15.32:0/1604055583 >> v2:172.21.15.32:3300/0 conn(0x7f72bc06a770 msgr2=0x7f72bc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b9d9b700 1 --2- 172.21.15.32:0/1604055583 >> v2:172.21.15.32:3300/0 conn(0x7f72bc06a770 0x7f72bc06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b9d9b700 1 -- 172.21.15.32:0/1604055583 >> v1:172.21.15.32:6789/0 conn(0x7f72bc131b40 legacy=0x7f72bc138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b9d9b700 1 -- 172.21.15.32:0/1604055583 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc09f630 con 0x7f72bc06b080 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b959a700 1 -- 172.21.15.32:0/1604055583 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619001938 0 0) 0x7f72bc09ece0 con 0x7f72bc131b40 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b9d9b700 1 --2- 172.21.15.32:0/1604055583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06b080 0x7f72bc131600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009840 tx=0x7f72b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2aecbe5f8e0568c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.828+0000 7f72b959a700 1 -- 172.21.15.32:0/1604055583 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0011830 con 0x7f72bc06b080 2023-09-30T22:24:38.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72b959a700 1 -- 172.21.15.32:0/1604055583 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0011990 con 0x7f72bc06b080 2023-09-30T22:24:38.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72b959a700 1 -- 172.21.15.32:0/1604055583 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0011d20 con 0x7f72bc06b080 2023-09-30T22:24:38.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72c0dac700 1 -- 172.21.15.32:0/1604055583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06b080 msgr2=0x7f72bc131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72c0dac700 1 --2- 172.21.15.32:0/1604055583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06b080 0x7f72bc131600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009840 tx=0x7f72b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:38.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72c0dac700 1 -- 172.21.15.32:0/1604055583 shutdown_connections 2023-09-30T22:24:38.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72c0dac700 1 --2- 172.21.15.32:0/1604055583 >> v2:172.21.15.32:3300/0 conn(0x7f72bc06a770 0x7f72bc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72c0dac700 1 --2- 172.21.15.32:0/1604055583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06b080 0x7f72bc131600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.829+0000 7f72c0dac700 1 -- 172.21.15.32:0/1604055583 >> 172.21.15.32:0/1604055583 conn(0x7f72bc066f50 msgr2=0x7f72bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.830+0000 7f72c0dac700 1 -- 172.21.15.32:0/1604055583 shutdown_connections 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.830+0000 7f72c0dac700 1 -- 172.21.15.32:0/1604055583 wait complete. 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 Processor -- start 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 -- start start 2023-09-30T22:24:38.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 0x7f72bc1c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72bc06b080 0x7f72bc1c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72bc131b40 0x7f72bc1c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc131690 con 0x7f72bc131b40 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc122e50 con 0x7f72bc06a770 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.831+0000 7f72c0dac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1221e0 con 0x7f72bc06b080 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72ba59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 0x7f72bc1c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72ba59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 0x7f72bc1c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35580/0 (socket says 172.21.15.32:35580) 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72ba59c700 1 -- 172.21.15.32:0/2218312724 learned_addr learned my addr 172.21.15.32:0/2218312724 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:38.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72bc131b40 0x7f72bc1c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72b9d9b700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72bc06b080 0x7f72bc1c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 -- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72bc06b080 msgr2=0x7f72bc1c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72bc06b080 0x7f72bc1c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 -- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 msgr2=0x7f72bc1c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 0x7f72bc1c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 -- 172.21.15.32:0/2218312724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00094a0 con 0x7f72bc131b40 2023-09-30T22:24:38.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72ba59c700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 0x7f72bc1c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.832+0000 7f72bad9d700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72bc131b40 0x7f72bc1c68b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b400cbe0 tx=0x7f72b400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.833+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b400aaf0 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.833+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b400ac70 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.833+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b40070d0 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.833+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc123b50 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.833+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc1310d0 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.834+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc074ea0 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.835+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f72b4007230 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.835+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f72b403f9d0 con 0x7f72bc131b40 2023-09-30T22:24:38.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.835+0000 7f72ba59c700 1 -- 172.21.15.32:0/2218312724 >> v1:172.21.15.32:6816/126553 conn(0x7f729c02f510 legacy=0x7f729c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:38.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.844+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72b40074e0 con 0x7f72bc131b40 2023-09-30T22:24:38.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.929+0000 7f9e20f64700 1 Processor -- start 2023-09-30T22:24:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.930+0000 7f9e20f64700 1 -- start start 2023-09-30T22:24:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.930+0000 7f9e20f64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1c09d6f0 0x7f9e1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.930+0000 7f9e20f64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e000 0x7f9e1c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.930+0000 7f9e20f64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1c09b120 con 0x7f9e1c09e410 2023-09-30T22:24:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.930+0000 7f9e20f64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1c06b460 con 0x7f9e1c09d6f0 2023-09-30T22:24:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.930+0000 7f9e20f64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c09a7e0 con 0x7f9e1c09e000 2023-09-30T22:24:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e1c09d6f0 0x7f9e1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e1c09e410 0x7f9e1c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51530/0 (socket says 172.21.15.32:51530) 2023-09-30T22:24:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1ad9d700 1 -- 172.21.15.32:0/4093397230 learned_addr learned my addr 172.21.15.32:0/4093397230 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:38.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/4093397230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e000 0x7f9e1c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1959a700 1 -- 172.21.15.32:0/4093397230 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774779919 0 0) 0x7f9e1c09b120 con 0x7f9e1c09e410 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1959a700 1 -- 172.21.15.32:0/4093397230 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e040035b0 con 0x7f9e1c09e410 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e19d9b700 1 -- 172.21.15.32:0/4093397230 >> v2:172.21.15.32:3300/0 conn(0x7f9e1c09d6f0 msgr2=0x7f9e1c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/4093397230 >> v2:172.21.15.32:3300/0 conn(0x7f9e1c09d6f0 0x7f9e1c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e19d9b700 1 -- 172.21.15.32:0/4093397230 >> v1:172.21.15.32:6789/0 conn(0x7f9e1c09e410 legacy=0x7f9e1c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e19d9b700 1 -- 172.21.15.32:0/4093397230 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c09a500 con 0x7f9e1c09e000 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1a59c700 1 --2- 172.21.15.32:0/4093397230 >> v2:172.21.15.32:3300/0 conn(0x7f9e1c09d6f0 0x7f9e1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/4093397230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e000 0x7f9e1c1339f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009870 tx=0x7f9e10009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70422f9f51821e00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.931+0000 7f9e1959a700 1 -- 172.21.15.32:0/4093397230 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10011830 con 0x7f9e1c09e000 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e1959a700 1 -- 172.21.15.32:0/4093397230 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e10011990 con 0x7f9e1c09e000 2023-09-30T22:24:38.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e1959a700 1 -- 172.21.15.32:0/4093397230 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10011d20 con 0x7f9e1c09e000 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 -- 172.21.15.32:0/4093397230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e000 msgr2=0x7f9e1c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 --2- 172.21.15.32:0/4093397230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e000 0x7f9e1c1339f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009870 tx=0x7f9e10009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 -- 172.21.15.32:0/4093397230 shutdown_connections 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 --2- 172.21.15.32:0/4093397230 >> v2:172.21.15.32:3300/0 conn(0x7f9e1c09d6f0 0x7f9e1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 --2- 172.21.15.32:0/4093397230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e000 0x7f9e1c1339f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 -- 172.21.15.32:0/4093397230 >> 172.21.15.32:0/4093397230 conn(0x7f9e1c066f50 msgr2=0x7f9e1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 -- 172.21.15.32:0/4093397230 shutdown_connections 2023-09-30T22:24:38.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.932+0000 7f9e20f64700 1 -- 172.21.15.32:0/4093397230 wait complete. 2023-09-30T22:24:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.934+0000 7f9e20f64700 1 Processor -- start 2023-09-30T22:24:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.934+0000 7f9e20f64700 1 -- start start 2023-09-30T22:24:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e20f64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 0x7f9e1c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e20f64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e1c09e000 0x7f9e1c1c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e20f64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e410 0x7f9e1c1c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e20f64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a8160 con 0x7f9e1c09d6f0 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e20f64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a6450 con 0x7f9e1c09e410 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e20f64700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0a2cc0 con 0x7f9e1c09e000 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e1a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 0x7f9e1c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e1a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 0x7f9e1c1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39444/0 (socket says 172.21.15.32:39444) 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e1a59c700 1 -- 172.21.15.32:0/1769494849 learned_addr learned my addr 172.21.15.32:0/1769494849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e1c09e000 0x7f9e1c1c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.935+0000 7f9e1ad9d700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e410 0x7f9e1c1c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:38.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e19d9b700 1 -- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e410 msgr2=0x7f9e1c1c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e410 0x7f9e1c1c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e19d9b700 1 -- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 msgr2=0x7f9e1c1c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 0x7f9e1c1c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e19d9b700 1 -- 172.21.15.32:0/1769494849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0094a0 con 0x7f9e1c09e000 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e1ad9d700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e410 0x7f9e1c1c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e1a59c700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 0x7f9e1c1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e19d9b700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e1c09e000 0x7f9e1c1c8580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c09ac00 tx=0x7f9e10004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10004510 con 0x7f9e1c09e000 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1001f070 con 0x7f9e1c09e000 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e100094a0 con 0x7f9e1c09e000 2023-09-30T22:24:38.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100038c0 con 0x7f9e1c09e000 2023-09-30T22:24:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.936+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1c1a8360 con 0x7f9e1c09e000 2023-09-30T22:24:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.937+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9e10003a20 con 0x7f9e1c09e000 2023-09-30T22:24:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.938+0000 7f9e1a59c700 1 -- 172.21.15.32:0/1769494849 >> v1:172.21.15.32:6816/126553 conn(0x7f9dfc02f510 legacy=0x7f9dfc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.938+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f9e10021070 con 0x7f9e1c09e000 2023-09-30T22:24:38.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.938+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1c1a7cc0 con 0x7f9e1c09e000 2023-09-30T22:24:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.943+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e1001a950 con 0x7f9e1c09e000 2023-09-30T22:24:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.952+0000 7f59b8cea700 1 -- 172.21.15.32:0/2040684961 >> v1:172.21.15.32:6816/126553 conn(0x7f599802f510 legacy=0x7f59980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.984+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f59b4074ea0 con 0x7f59b406ae80 2023-09-30T22:24:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.984+0000 7f59a9ffb700 1 -- 172.21.15.32:0/2040684961 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59b0015020 con 0x7f59b406ae80 2023-09-30T22:24:38.984 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f310d443700 1 -- 172.21.15.32:0/870800678 >> v1:172.21.15.32:6816/126553 conn(0x7f30ec02f510 legacy=0x7f30ec0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 >> v1:172.21.15.32:6816/126553 conn(0x7f599802f510 legacy=0x7f59980319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 msgr2=0x7f59b411d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 0x7f59b411d890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009c40 tx=0x7f59b0004750 comp rx=0 tx=0).stop 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 shutdown_connections 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59b41316c0 0x7f59b411e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59b406ae80 0x7f59b411d890 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 --2- 172.21.15.32:0/2040684961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59b409d9f0 0x7f59b411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 >> 172.21.15.32:0/2040684961 conn(0x7f59b4066f50 msgr2=0x7f59b412db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 shutdown_connections 2023-09-30T22:24:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:38.988+0000 7f59bb74f700 1 -- 172.21.15.32:0/2040684961 wait complete. 2023-09-30T22:24:39.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.036+0000 7f72ba59c700 1 -- 172.21.15.32:0/2218312724 >> v1:172.21.15.32:6816/126553 conn(0x7f729c02f510 legacy=0x7f729c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.043+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3108074ea0 con 0x7f310809f650 2023-09-30T22:24:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.044+0000 7f30fdffb700 1 -- 172.21.15.32:0/870800678 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3100019300 con 0x7f310809f650 2023-09-30T22:24:39.044 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:39.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 >> v1:172.21.15.32:6816/126553 conn(0x7f30ec02f510 legacy=0x7f30ec0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 msgr2=0x7f310806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 0x7f310806c330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f310000cbc0 tx=0x7f310000ced0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 shutdown_connections 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f310809f650 0x7f310806c330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f310809e930 0x7f310806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 --2- 172.21.15.32:0/870800678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f310809f240 0x7f310806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.046+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 >> 172.21.15.32:0/870800678 conn(0x7f31080a4210 msgr2=0x7f31081b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.047+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 shutdown_connections 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.047+0000 7f310f6a7700 1 -- 172.21.15.32:0/870800678 wait complete. 2023-09-30T22:24:39.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:39.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.061+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f72bc06c360 con 0x7f72bc131b40 2023-09-30T22:24:39.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.061+0000 7f72aaffd700 1 -- 172.21.15.32:0/2218312724 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72b4018030 con 0x7f72bc131b40 2023-09-30T22:24:39.062 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 >> v1:172.21.15.32:6816/126553 conn(0x7f729c02f510 legacy=0x7f729c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72bc131b40 msgr2=0x7f72bc1c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72bc131b40 0x7f72bc1c68b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b400cbe0 tx=0x7f72b400cef0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 shutdown_connections 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72bc131b40 0x7f72bc1c68b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72bc06a770 0x7f72bc1c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 --2- 172.21.15.32:0/2218312724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72bc06b080 0x7f72bc1c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 >> 172.21.15.32:0/2218312724 conn(0x7f72bc066f50 msgr2=0x7f72bc06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 shutdown_connections 2023-09-30T22:24:39.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.064+0000 7f72c0dac700 1 -- 172.21.15.32:0/2218312724 wait complete. 2023-09-30T22:24:39.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:39.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.139+0000 7f9e1a59c700 1 -- 172.21.15.32:0/1769494849 >> v1:172.21.15.32:6816/126553 conn(0x7f9dfc02f510 legacy=0x7f9dfc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:39.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.193+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e1c074ea0 con 0x7f9e1c09e000 2023-09-30T22:24:39.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.194+0000 7f9e0affd700 1 -- 172.21.15.32:0/1769494849 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e10010b70 con 0x7f9e1c09e000 2023-09-30T22:24:39.194 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 >> v1:172.21.15.32:6816/126553 conn(0x7f9dfc02f510 legacy=0x7f9dfc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e1c09e000 msgr2=0x7f9e1c1c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e1c09e000 0x7f9e1c1c8580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c09ac00 tx=0x7f9e10004750 comp rx=0 tx=0).stop 2023-09-30T22:24:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 shutdown_connections 2023-09-30T22:24:39.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e1c09e410 0x7f9e1c1c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e1c09d6f0 0x7f9e1c1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 --2- 172.21.15.32:0/1769494849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e1c09e000 0x7f9e1c1c8580 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 >> 172.21.15.32:0/1769494849 conn(0x7f9e1c066f50 msgr2=0x7f9e1c1bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 shutdown_connections 2023-09-30T22:24:39.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:39.196+0000 7f9e20f64700 1 -- 172.21.15.32:0/1769494849 wait complete. 2023-09-30T22:24:39.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:40.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:40.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:40.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:40.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 Processor -- start 2023-09-30T22:24:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 -- start start 2023-09-30T22:24:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 0x7f28880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28880a1340 0x7f2888133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2888133fa0 0x7f2888134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288809edd0 con 0x7f28880a1340 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288809e1f0 con 0x7f28880a0a30 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.155+0000 7f288fbd1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f288809dc50 con 0x7f2888133fa0 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 0x7f28880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 0x7f28880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35614/0 (socket says 172.21.15.32:35614) 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 -- 172.21.15.32:0/3588825654 learned_addr learned my addr 172.21.15.32:0/3588825654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d16c700 1 --2- 172.21.15.32:0/3588825654 >> v2:172.21.15.32:3300/0 conn(0x7f28880a1340 0x7f2888133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288e16e700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2888133fa0 0x7f2888134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 -- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2888133fa0 msgr2=0x7f2888134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2888133fa0 0x7f2888134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 -- 172.21.15.32:0/3588825654 >> v2:172.21.15.32:3300/0 conn(0x7f28880a1340 msgr2=0x7f2888133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 --2- 172.21.15.32:0/3588825654 >> v2:172.21.15.32:3300/0 conn(0x7f28880a1340 0x7f2888133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 -- 172.21.15.32:0/3588825654 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880a1fe0 con 0x7f28880a0a30 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288e16e700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2888133fa0 0x7f2888134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288d96d700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 0x7f28880a0e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2878009e80 tx=0x7f287800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62e8239f619216c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.156+0000 7f288c96b700 1 -- 172.21.15.32:0/3588825654 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287800e070 con 0x7f28880a0a30 2023-09-30T22:24:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288c96b700 1 -- 172.21.15.32:0/3588825654 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287800b8d0 con 0x7f28880a0a30 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288c96b700 1 -- 172.21.15.32:0/3588825654 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287800bc80 con 0x7f28880a0a30 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 -- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 msgr2=0x7f28880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 0x7f28880a0e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2878009e80 tx=0x7f287800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 -- 172.21.15.32:0/3588825654 shutdown_connections 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a0a30 0x7f28880a0e00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 --2- 172.21.15.32:0/3588825654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2888133fa0 0x7f2888134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 --2- 172.21.15.32:0/3588825654 >> v2:172.21.15.32:3300/0 conn(0x7f28880a1340 0x7f2888133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 -- 172.21.15.32:0/3588825654 >> 172.21.15.32:0/3588825654 conn(0x7f28880a6550 msgr2=0x7f28880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 -- 172.21.15.32:0/3588825654 shutdown_connections 2023-09-30T22:24:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.157+0000 7f288fbd1700 1 -- 172.21.15.32:0/3588825654 wait complete. 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.158+0000 7f288fbd1700 1 Processor -- start 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 -- start start 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28880a0a30 0x7f2888120230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 0x7f2888120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2888133fa0 0x7f2888120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888121700 con 0x7f2888133fa0 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2888122440 con 0x7f28880a1340 2023-09-30T22:24:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288fbd1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2888123d00 con 0x7f28880a0a30 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288d16c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 0x7f2888120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288d16c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 0x7f2888120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35616/0 (socket says 172.21.15.32:35616) 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288d16c700 1 -- 172.21.15.32:0/554890469 learned_addr learned my addr 172.21.15.32:0/554890469 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288e16e700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2888133fa0 0x7f2888120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.159+0000 7f288d96d700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28880a0a30 0x7f2888120230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288d16c700 1 -- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28880a0a30 msgr2=0x7f2888120230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288d16c700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28880a0a30 0x7f2888120230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288d16c700 1 -- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2888133fa0 msgr2=0x7f2888120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288d16c700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2888133fa0 0x7f2888120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288d16c700 1 -- 172.21.15.32:0/554890469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28800094a0 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288d16c700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 0x7f2888120770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e870 tx=0x7f287c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c009590 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c013680 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2878009b30 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0139f0 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.160+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28880a37d0 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.161+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f287c013b50 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.161+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f287c03fa70 con 0x7f28880a1340 2023-09-30T22:24:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.161+0000 7f288d96d700 1 -- 172.21.15.32:0/554890469 >> v1:172.21.15.32:6816/126553 conn(0x7f287402f510 legacy=0x7f28740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.161+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888074ea0 con 0x7f28880a1340 2023-09-30T22:24:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.166+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f287c017080 con 0x7f28880a1340 2023-09-30T22:24:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.209+0000 7fa7c5227700 1 Processor -- start 2023-09-30T22:24:40.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.209+0000 7fa7c5227700 1 -- start start 2023-09-30T22:24:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7c5227700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7c5227700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7c0135e50 0x7fa7c0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7c5227700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c009f130 con 0x7fa7c009d6f0 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7c5227700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c009f520 con 0x7fa7c0135e50 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7c5227700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c009e930 con 0x7fa7c009e1d0 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7c0135e50 0x7fa7c0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7c0135e50 0x7fa7c0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39474/0 (socket says 172.21.15.32:39474) 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 -- 172.21.15.32:0/2457658325 learned_addr learned my addr 172.21.15.32:0/2457658325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7be59c700 1 --2- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 -- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 msgr2=0x7fa7c0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:40.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 --2- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c0135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 -- 172.21.15.32:0/2457658325 >> v1:172.21.15.32:6789/0 conn(0x7fa7c009d6f0 legacy=0x7fa7c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bdd9b700 1 -- 172.21.15.32:0/2457658325 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973887482 0 0) 0x7fa7c009f130 con 0x7fa7c009d6f0 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 -- 172.21.15.32:0/2457658325 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c009b2d0 con 0x7fa7c0135e50 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.210+0000 7fa7bf59e700 1 --2- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c0138230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800d180 tx=0x7fa7b800d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f37700d445ea442 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7bdd9b700 1 -- 172.21.15.32:0/2457658325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8010070 con 0x7fa7c0135e50 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7bdd9b700 1 -- 172.21.15.32:0/2457658325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b800dd00 con 0x7fa7c0135e50 2023-09-30T22:24:40.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7bdd9b700 1 -- 172.21.15.32:0/2457658325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800b310 con 0x7fa7c0135e50 2023-09-30T22:24:40.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:40.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 -- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 msgr2=0x7fa7c0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 --2- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c0138230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800d180 tx=0x7fa7b800d490 comp rx=0 tx=0).stop 2023-09-30T22:24:40.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 -- 172.21.15.32:0/2457658325 shutdown_connections 2023-09-30T22:24:40.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 --2- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c0138230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 --2- 172.21.15.32:0/2457658325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 -- 172.21.15.32:0/2457658325 >> 172.21.15.32:0/2457658325 conn(0x7fa7c0066f50 msgr2=0x7fa7c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.211+0000 7fa7c5227700 1 -- 172.21.15.32:0/2457658325 shutdown_connections 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.212+0000 7fa7c5227700 1 -- 172.21.15.32:0/2457658325 wait complete. 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.213+0000 7fa7c5227700 1 Processor -- start 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.213+0000 7fa7c5227700 1 -- start start 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7c5227700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009d6f0 0x7fa7c011e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7c5227700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c011e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7c5227700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c01c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7c5227700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c011eca0 con 0x7fa7c0135e50 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7c5227700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01212a0 con 0x7fa7c009d6f0 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7c5227700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c0120e50 con 0x7fa7c009e1d0 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c011e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c011e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53398/0 (socket says 172.21.15.32:53398) 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 -- 172.21.15.32:0/4237626803 learned_addr learned my addr 172.21.15.32:0/4237626803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7bf59e700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c01c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 -- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009d6f0 msgr2=0x7fa7c011e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009d6f0 0x7fa7c011e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 -- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 msgr2=0x7fa7c01c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c01c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7be59c700 1 -- 172.21.15.32:0/4237626803 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b800bd40 con 0x7fa7c009e1d0 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.214+0000 7fa7bf59e700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c01c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.215+0000 7fa7be59c700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c011e760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9e0 tx=0x7fa7b400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.215+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4009710 con 0x7fa7c009e1d0 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.215+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b400c850 con 0x7fa7c009e1d0 2023-09-30T22:24:40.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.215+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c01301b0 con 0x7fa7c009e1d0 2023-09-30T22:24:40.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.215+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400cbc0 con 0x7fa7c009e1d0 2023-09-30T22:24:40.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.215+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c009c700 con 0x7fa7c009e1d0 2023-09-30T22:24:40.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.216+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa7b400cd20 con 0x7fa7c009e1d0 2023-09-30T22:24:40.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.216+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c0121090 con 0x7fa7c009e1d0 2023-09-30T22:24:40.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.216+0000 7fa7bed9d700 1 -- 172.21.15.32:0/4237626803 >> v1:172.21.15.32:6816/126553 conn(0x7fa7ac02f510 legacy=0x7fa7ac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.216+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fa7b401a360 con 0x7fa7c009e1d0 2023-09-30T22:24:40.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.222+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa7b4045060 con 0x7fa7c009e1d0 2023-09-30T22:24:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.256+0000 7f1b60974700 1 Processor -- start 2023-09-30T22:24:40.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.256+0000 7f1b60974700 1 -- start start 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.257+0000 7f1b60974700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.257+0000 7f1b60974700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.257+0000 7f1b60974700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c0716a0 con 0x7f1b5c06fb20 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b60974700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c09a390 con 0x7f1b5c130540 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b60974700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c071290 con 0x7f1b5c130e50 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53410/0 (socket says 172.21.15.32:53410) 2023-09-30T22:24:40.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 -- 172.21.15.32:0/3860101151 learned_addr learned my addr 172.21.15.32:0/3860101151 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 -- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 msgr2=0x7f1b5c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:40.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 --2- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 -- 172.21.15.32:0/3860101151 >> v1:172.21.15.32:6789/0 conn(0x7f1b5c06fb20 legacy=0x7f1b5c137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.258+0000 7f1b59d9b700 1 -- 172.21.15.32:0/3860101151 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c130340 con 0x7f1b5c130e50 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.259+0000 7f1b59d9b700 1 --2- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c06f5e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009950 tx=0x7f1b50009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299bdacfc8061c75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.259+0000 7f1b5959a700 1 -- 172.21.15.32:0/3860101151 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50009ea0 con 0x7f1b5c130e50 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.259+0000 7f1b5959a700 1 -- 172.21.15.32:0/3860101151 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50004230 con 0x7f1b5c130e50 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.259+0000 7f1b5959a700 1 -- 172.21.15.32:0/3860101151 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000b7e0 con 0x7f1b5c130e50 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 -- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 msgr2=0x7f1b5c06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 --2- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c06f5e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009950 tx=0x7f1b50009c60 comp rx=0 tx=0).stop 2023-09-30T22:24:40.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 -- 172.21.15.32:0/3860101151 shutdown_connections 2023-09-30T22:24:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 --2- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 --2- 172.21.15.32:0/3860101151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c06f5e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 -- 172.21.15.32:0/3860101151 >> 172.21.15.32:0/3860101151 conn(0x7f1b5c0a2aa0 msgr2=0x7f1b5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 -- 172.21.15.32:0/3860101151 shutdown_connections 2023-09-30T22:24:40.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.260+0000 7f1b60974700 1 -- 172.21.15.32:0/3860101151 wait complete. 2023-09-30T22:24:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.261+0000 7f1b60974700 1 Processor -- start 2023-09-30T22:24:40.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.261+0000 7f1b60974700 1 -- start start 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b60974700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c06fb20 0x7f1b5c126160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b60974700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c1266a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b60974700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c1c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b60974700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c12b5b0 con 0x7f1b5c06fb20 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b60974700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c12bf10 con 0x7f1b5c130540 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b60974700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c129660 con 0x7f1b5c130e50 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b5a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c06fb20 0x7f1b5c126160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.262+0000 7f1b5a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c06fb20 0x7f1b5c126160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39494/0 (socket says 172.21.15.32:39494) 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5a59c700 1 -- 172.21.15.32:0/3473632870 learned_addr learned my addr 172.21.15.32:0/3473632870 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c1c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b59d9b700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c1266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 -- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 msgr2=0x7f1b5c1266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c1266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 -- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c06fb20 msgr2=0x7f1b5c126160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c06fb20 0x7f1b5c126160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 -- 172.21.15.32:0/3473632870 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50009550 con 0x7f1b5c130e50 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b5ad9d700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c1c5f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b54010f50 tx=0x7f1b5400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5400ab90 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5400acf0 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54018650 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.263+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c12c900 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.264+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c12ddc0 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.264+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1b540213b0 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.265+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1b54043870 con 0x7f1b5c130e50 2023-09-30T22:24:40.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.265+0000 7f1b5a59c700 1 -- 172.21.15.32:0/3473632870 >> v1:172.21.15.32:6816/126553 conn(0x7f1b3c02f510 legacy=0x7f1b3c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.266+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c12ca90 con 0x7f1b5c130e50 2023-09-30T22:24:40.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.271+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b5c12ca90 con 0x7f1b5c130e50 2023-09-30T22:24:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.362+0000 7f288d96d700 1 -- 172.21.15.32:0/554890469 >> v1:172.21.15.32:6816/126553 conn(0x7f287402f510 legacy=0x7f28740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.380+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28880a28e0 con 0x7f28880a1340 2023-09-30T22:24:40.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.381+0000 7f28867fc700 1 -- 172.21.15.32:0/554890469 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f287c01ba20 con 0x7f28880a1340 2023-09-30T22:24:40.382 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 >> v1:172.21.15.32:6816/126553 conn(0x7f287402f510 legacy=0x7f28740319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 msgr2=0x7f2888120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 0x7f2888120770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f287c00e870 tx=0x7f287c00eb80 comp rx=0 tx=0).stop 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 shutdown_connections 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2888133fa0 0x7f2888120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28880a0a30 0x7f2888120230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 --2- 172.21.15.32:0/554890469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28880a1340 0x7f2888120770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 >> 172.21.15.32:0/554890469 conn(0x7f28880a6550 msgr2=0x7f28881ba800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 shutdown_connections 2023-09-30T22:24:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.383+0000 7f288fbd1700 1 -- 172.21.15.32:0/554890469 wait complete. 2023-09-30T22:24:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.389+0000 7f4479b54700 1 Processor -- start 2023-09-30T22:24:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.389+0000 7f4479b54700 1 -- start start 2023-09-30T22:24:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4479b54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f447409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4479b54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e410 0x7f4474134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4479b54700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447409b120 con 0x7f447409e000 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4479b54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447406b460 con 0x7f447409d6f0 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4479b54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f447409a7e0 con 0x7f447409e410 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4472ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f447409e000 0x7f44741339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51556/0 (socket says 172.21.15.32:51556) 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4472ffd700 1 -- 172.21.15.32:0/1440278292 learned_addr learned my addr 172.21.15.32:0/1440278292 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f4473fff700 1 --2- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e410 0x7f4474134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f44727fc700 1 -- 172.21.15.32:0/1440278292 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284987369 0 0) 0x7f447409b120 con 0x7f447409e000 2023-09-30T22:24:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.390+0000 7f44727fc700 1 -- 172.21.15.32:0/1440278292 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f445c0035b0 con 0x7f447409e000 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f4473fff700 1 -- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 msgr2=0x7f447409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f4473fff700 1 --2- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f447409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f4473fff700 1 -- 172.21.15.32:0/1440278292 >> v1:172.21.15.32:6789/0 conn(0x7f447409e000 legacy=0x7f44741339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f4473fff700 1 -- 172.21.15.32:0/1440278292 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447409a500 con 0x7f447409e410 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f4473fff700 1 --2- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e410 0x7f4474134100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f446c009df0 tx=0x7f446c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c7b356a881e01a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f44727fc700 1 -- 172.21.15.32:0/1440278292 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00e070 con 0x7f447409e410 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f44727fc700 1 -- 172.21.15.32:0/1440278292 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00bae0 con 0x7f447409e410 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.391+0000 7f44727fc700 1 -- 172.21.15.32:0/1440278292 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00be90 con 0x7f447409e410 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 -- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e410 msgr2=0x7f4474134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 --2- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e410 0x7f4474134100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f446c009df0 tx=0x7f446c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 -- 172.21.15.32:0/1440278292 shutdown_connections 2023-09-30T22:24:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 --2- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e410 0x7f4474134100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 --2- 172.21.15.32:0/1440278292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f447409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 -- 172.21.15.32:0/1440278292 >> 172.21.15.32:0/1440278292 conn(0x7f4474066f50 msgr2=0x7f447409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 -- 172.21.15.32:0/1440278292 shutdown_connections 2023-09-30T22:24:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.392+0000 7f4479b54700 1 -- 172.21.15.32:0/1440278292 wait complete. 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.394+0000 7f4479b54700 1 Processor -- start 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.394+0000 7f4479b54700 1 -- start start 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4479b54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f4474122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4479b54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e000 0x7f4474122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4479b54700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447409e410 0x7f44741230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4479b54700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474123ad0 con 0x7f447409e410 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4479b54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447409c710 con 0x7f447409d6f0 2023-09-30T22:24:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4479b54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f447409a0c0 con 0x7f447409e000 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e000 0x7f4474122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e000 0x7f4474122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53418/0 (socket says 172.21.15.32:53418) 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4472ffd700 1 -- 172.21.15.32:0/2788931944 learned_addr learned my addr 172.21.15.32:0/2788931944 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f4473fff700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447409e410 0x7f44741230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.395+0000 7f44737fe700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f4474122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4473fff700 1 -- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e000 msgr2=0x7f4474122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4473fff700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e000 0x7f4474122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4473fff700 1 -- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 msgr2=0x7f4474122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4473fff700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f4474122670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4473fff700 1 -- 172.21.15.32:0/2788931944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c009a70 con 0x7f447409e410 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4473fff700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447409e410 0x7f44741230f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f447406e470 tx=0x7f446c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0045f0 con 0x7f447409e410 2023-09-30T22:24:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4474124a50 con 0x7f447409e410 2023-09-30T22:24:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c020070 con 0x7f447409e410 2023-09-30T22:24:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c004970 con 0x7f447409e410 2023-09-30T22:24:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.396+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4474124240 con 0x7f447409e410 2023-09-30T22:24:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.398+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4474125ac0 con 0x7f447409e410 2023-09-30T22:24:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.398+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f446c027480 con 0x7f447409e410 2023-09-30T22:24:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.398+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f446c022070 con 0x7f447409e410 2023-09-30T22:24:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.398+0000 7f44737fe700 1 -- 172.21.15.32:0/2788931944 >> v1:172.21.15.32:6816/126553 conn(0x7f446002f510 legacy=0x7f44600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.407+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4474125ac0 con 0x7f447409e410 2023-09-30T22:24:40.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.417+0000 7fa7bed9d700 1 -- 172.21.15.32:0/4237626803 >> v1:172.21.15.32:6816/126553 conn(0x7fa7ac02f510 legacy=0x7fa7ac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.460+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa7c0074ea0 con 0x7fa7c009e1d0 2023-09-30T22:24:40.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.460+0000 7fa7a77fe700 1 -- 172.21.15.32:0/4237626803 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7b4045020 con 0x7fa7c009e1d0 2023-09-30T22:24:40.461 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 >> v1:172.21.15.32:6816/126553 conn(0x7fa7ac02f510 legacy=0x7fa7ac0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 msgr2=0x7fa7c011e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c011e760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9e0 tx=0x7fa7b400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 shutdown_connections 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c0135e50 0x7fa7c01c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009d6f0 0x7fa7c011e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 --2- 172.21.15.32:0/4237626803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009e1d0 0x7fa7c011e760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.462+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 >> 172.21.15.32:0/4237626803 conn(0x7fa7c0066f50 msgr2=0x7fa7c01328b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.463+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 shutdown_connections 2023-09-30T22:24:40.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.463+0000 7fa7c5227700 1 -- 172.21.15.32:0/4237626803 wait complete. 2023-09-30T22:24:40.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.466+0000 7f1b5a59c700 1 -- 172.21.15.32:0/3473632870 >> v1:172.21.15.32:6816/126553 conn(0x7f1b3c02f510 legacy=0x7f1b3c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.522+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b5c12ca90 con 0x7f1b5c130e50 2023-09-30T22:24:40.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.522+0000 7f1b4affd700 1 -- 172.21.15.32:0/3473632870 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b5c12ca90 con 0x7f1b5c130e50 2023-09-30T22:24:40.523 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 >> v1:172.21.15.32:6816/126553 conn(0x7f1b3c02f510 legacy=0x7f1b3c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 msgr2=0x7f1b5c1c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c1c5f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b54010f50 tx=0x7f1b5400a340 comp rx=0 tx=0).stop 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 shutdown_connections 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b5c130e50 0x7f1b5c1c5f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b5c06fb20 0x7f1b5c126160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 --2- 172.21.15.32:0/3473632870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b5c130540 0x7f1b5c1266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.524+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 >> 172.21.15.32:0/3473632870 conn(0x7f1b5c0a2aa0 msgr2=0x7f1b5c06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.525+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 shutdown_connections 2023-09-30T22:24:40.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.525+0000 7f1b60974700 1 -- 172.21.15.32:0/3473632870 wait complete. 2023-09-30T22:24:40.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:40.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.599+0000 7f44737fe700 1 -- 172.21.15.32:0/2788931944 >> v1:172.21.15.32:6816/126553 conn(0x7f446002f510 legacy=0x7f44600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.628+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4474125ac0 con 0x7f447409e410 2023-09-30T22:24:40.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.628+0000 7f445bfff700 1 -- 172.21.15.32:0/2788931944 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4474125ac0 con 0x7f447409e410 2023-09-30T22:24:40.629 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:40.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 >> v1:172.21.15.32:6816/126553 conn(0x7f446002f510 legacy=0x7f44600319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447409e410 msgr2=0x7f44741230f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447409e410 0x7f44741230f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f447406e470 tx=0x7f446c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 shutdown_connections 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f447409e410 0x7f44741230f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f447409d6f0 0x7f4474122670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.630+0000 7f4479b54700 1 --2- 172.21.15.32:0/2788931944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f447409e000 0x7f4474122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.631+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 >> 172.21.15.32:0/2788931944 conn(0x7f4474066f50 msgr2=0x7f4474070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.631+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 shutdown_connections 2023-09-30T22:24:40.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:40.631+0000 7f4479b54700 1 -- 172.21.15.32:0/2788931944 wait complete. 2023-09-30T22:24:40.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:41.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:41.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:41.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 Processor -- start 2023-09-30T22:24:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 -- start start 2023-09-30T22:24:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409d6f0 0x7ffb3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3409b120 con 0x7ffb3409e000 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3406b460 con 0x7ffb3409e410 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.551+0000 7ffb391d4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3409a7e0 con 0x7ffb3409d6f0 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51432/0 (socket says 172.21.15.32:51432) 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 -- 172.21.15.32:0/849425909 learned_addr learned my addr 172.21.15.32:0/849425909 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb32d9d700 1 --2- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409d6f0 0x7ffb3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 -- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409d6f0 msgr2=0x7ffb3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 --2- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409d6f0 0x7ffb3409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 -- 172.21.15.32:0/849425909 >> v1:172.21.15.32:6789/0 conn(0x7ffb3409e000 legacy=0x7ffb341339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 -- 172.21.15.32:0/849425909 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3409a500 con 0x7ffb3409e410 2023-09-30T22:24:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.552+0000 7ffb3359e700 1 --2- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34134100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009e80 tx=0x7ffb2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b358abf437a776b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb31d9b700 1 -- 172.21.15.32:0/849425909 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c00e070 con 0x7ffb3409e410 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb31d9b700 1 -- 172.21.15.32:0/849425909 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c003a20 con 0x7ffb3409e410 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb31d9b700 1 -- 172.21.15.32:0/849425909 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c003dd0 con 0x7ffb3409e410 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 -- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 msgr2=0x7ffb34134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 --2- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34134100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009e80 tx=0x7ffb2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 -- 172.21.15.32:0/849425909 shutdown_connections 2023-09-30T22:24:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 --2- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34134100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 --2- 172.21.15.32:0/849425909 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409d6f0 0x7ffb3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 -- 172.21.15.32:0/849425909 >> 172.21.15.32:0/849425909 conn(0x7ffb34066f50 msgr2=0x7ffb3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 -- 172.21.15.32:0/849425909 shutdown_connections 2023-09-30T22:24:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.553+0000 7ffb391d4700 1 -- 172.21.15.32:0/849425909 wait complete. 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.555+0000 7ffb391d4700 1 Processor -- start 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.555+0000 7ffb391d4700 1 -- start start 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.555+0000 7ffb391d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb3409d6f0 0x7ffb341224d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.555+0000 7ffb391d4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409e000 0x7ffb34122a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.555+0000 7ffb391d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34122f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb391d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34124f20 con 0x7ffb3409d6f0 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb391d4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3409c710 con 0x7ffb3409e410 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb391d4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34125fa0 con 0x7ffb3409e000 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb3259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409e000 0x7ffb34122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb32d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb3409d6f0 0x7ffb341224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb3259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409e000 0x7ffb34122a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:39926/0 (socket says 172.21.15.32:39926) 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb3259c700 1 -- 172.21.15.32:0/2302702628 learned_addr learned my addr 172.21.15.32:0/2302702628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb3359e700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb32d9d700 1 -- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409e000 msgr2=0x7ffb34122a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb32d9d700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409e000 0x7ffb34122a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb32d9d700 1 -- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 msgr2=0x7ffb34122f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb32d9d700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34122f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb32d9d700 1 -- 172.21.15.32:0/2302702628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c009b30 con 0x7ffb3409d6f0 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.556+0000 7ffb3359e700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34122f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.557+0000 7ffb32d9d700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb3409d6f0 0x7ffb341224d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e930 tx=0x7ffb2400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.557+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24013680 con 0x7ffb3409d6f0 2023-09-30T22:24:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.557+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb240137e0 con 0x7ffb3409d6f0 2023-09-30T22:24:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.557+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb34133850 con 0x7ffb3409d6f0 2023-09-30T22:24:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.557+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24013b50 con 0x7ffb3409d6f0 2023-09-30T22:24:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.557+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb34125730 con 0x7ffb3409d6f0 2023-09-30T22:24:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.558+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ffb24013cb0 con 0x7ffb3409d6f0 2023-09-30T22:24:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.558+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7ffb2403fff0 con 0x7ffb3409d6f0 2023-09-30T22:24:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.558+0000 7ffb3259c700 1 -- 172.21.15.32:0/2302702628 >> v1:172.21.15.32:6816/126553 conn(0x7ffb2002f510 legacy=0x7ffb200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.558+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb34074ea0 con 0x7ffb3409d6f0 2023-09-30T22:24:41.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.563+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb24017080 con 0x7ffb3409d6f0 2023-09-30T22:24:41.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.623+0000 7fd866088700 1 Processor -- start 2023-09-30T22:24:41.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.623+0000 7fd866088700 1 -- start start 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.623+0000 7fd866088700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8601316c0 0x7fd860131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd866088700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd86006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd866088700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd86006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd866088700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86009e130 con 0x7fd8601316c0 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd866088700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8601312b0 con 0x7fd86009d9f0 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd866088700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd86009f6e0 con 0x7fd86006acb0 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd86006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd86006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51454/0 (socket says 172.21.15.32:51454) 2023-09-30T22:24:41.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 -- 172.21.15.32:0/2052125788 learned_addr learned my addr 172.21.15.32:0/2052125788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85f7fe700 1 --2- 172.21.15.32:0/2052125788 >> v2:172.21.15.32:3300/0 conn(0x7fd8601316c0 0x7fd860131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85ffff700 1 --2- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd86006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 -- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 msgr2=0x7fd86006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 --2- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd86006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 -- 172.21.15.32:0/2052125788 >> v2:172.21.15.32:3300/0 conn(0x7fd8601316c0 msgr2=0x7fd860131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 --2- 172.21.15.32:0/2052125788 >> v2:172.21.15.32:3300/0 conn(0x7fd8601316c0 0x7fd860131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 -- 172.21.15.32:0/2052125788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86009e760 con 0x7fd86009d9f0 2023-09-30T22:24:41.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85f7fe700 1 --2- 172.21.15.32:0/2052125788 >> v2:172.21.15.32:3300/0 conn(0x7fd8601316c0 0x7fd860131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.624+0000 7fd85effd700 1 --2- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd86006a770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8540098f0 tx=0x7fd854009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cc6106ccbb0a79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd85e7fc700 1 -- 172.21.15.32:0/2052125788 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854011830 con 0x7fd86009d9f0 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd85e7fc700 1 -- 172.21.15.32:0/2052125788 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854011990 con 0x7fd86009d9f0 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd85e7fc700 1 -- 172.21.15.32:0/2052125788 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854011d20 con 0x7fd86009d9f0 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 -- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 msgr2=0x7fd86006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 --2- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd86006a770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8540098f0 tx=0x7fd854009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 -- 172.21.15.32:0/2052125788 shutdown_connections 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 --2- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd86006b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 --2- 172.21.15.32:0/2052125788 >> v2:172.21.15.32:3300/0 conn(0x7fd8601316c0 0x7fd860131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 --2- 172.21.15.32:0/2052125788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd86006a770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.625+0000 7fd866088700 1 -- 172.21.15.32:0/2052125788 >> 172.21.15.32:0/2052125788 conn(0x7fd860066f50 msgr2=0x7fd86009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.626+0000 7fd866088700 1 -- 172.21.15.32:0/2052125788 shutdown_connections 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.626+0000 7fd866088700 1 -- 172.21.15.32:0/2052125788 wait complete. 2023-09-30T22:24:41.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.627+0000 7fd866088700 1 Processor -- start 2023-09-30T22:24:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.627+0000 7fd866088700 1 -- start start 2023-09-30T22:24:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd866088700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd860132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd866088700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd860132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd866088700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8601316c0 0x7fd860132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd866088700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860134e10 con 0x7fd8601316c0 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd866088700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd860134910 con 0x7fd86009d9f0 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd866088700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd860072120 con 0x7fd86006acb0 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd85effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd860132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd85effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd860132840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51456/0 (socket says 172.21.15.32:51456) 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd85effd700 1 -- 172.21.15.32:0/3587206580 learned_addr learned my addr 172.21.15.32:0/3587206580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd85f7fe700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd860132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.628+0000 7fd85ffff700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8601316c0 0x7fd860132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd85f7fe700 1 -- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 msgr2=0x7fd860132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd85f7fe700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd860132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd85f7fe700 1 -- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8601316c0 msgr2=0x7fd860132d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd85f7fe700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8601316c0 0x7fd860132d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd85f7fe700 1 -- 172.21.15.32:0/3587206580 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500094a0 con 0x7fd86006acb0 2023-09-30T22:24:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd85f7fe700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd860132300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9b0 tx=0x7fd85000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000ee50 con 0x7fd86006acb0 2023-09-30T22:24:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000c850 con 0x7fd86006acb0 2023-09-30T22:24:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000cc00 con 0x7fd86006acb0 2023-09-30T22:24:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd854009550 con 0x7fd86006acb0 2023-09-30T22:24:41.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.629+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8600a3540 con 0x7fd86006acb0 2023-09-30T22:24:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.630+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd850013470 con 0x7fd86006acb0 2023-09-30T22:24:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.630+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fd85003fee0 con 0x7fd86006acb0 2023-09-30T22:24:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.630+0000 7fd85effd700 1 -- 172.21.15.32:0/3587206580 >> v1:172.21.15.32:6816/126553 conn(0x7fd84c02f510 legacy=0x7fd84c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.630+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd860134590 con 0x7fd86006acb0 2023-09-30T22:24:41.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.635+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd850013720 con 0x7fd86006acb0 2023-09-30T22:24:41.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:41.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.690+0000 7ff798fea700 1 Processor -- start 2023-09-30T22:24:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.690+0000 7ff798fea700 1 -- start start 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff798fea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 0x7ff794131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff798fea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff798fea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79409e130 con 0x7ff79406acb0 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff798fea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7941312b0 con 0x7ff79409d9f0 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff798fea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff79409f6e0 con 0x7ff7941316c0 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff79259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 0x7ff794131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff79259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 0x7ff794131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:39934/0 (socket says 172.21.15.32:39934) 2023-09-30T22:24:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff79259c700 1 -- 172.21.15.32:0/2418604895 learned_addr learned my addr 172.21.15.32:0/2418604895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.691+0000 7ff791d9b700 1 --2- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79259c700 1 -- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 msgr2=0x7ff79406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79259c700 1 --2- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79259c700 1 -- 172.21.15.32:0/2418604895 >> v1:172.21.15.32:6789/0 conn(0x7ff79406acb0 legacy=0x7ff794138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79259c700 1 -- 172.21.15.32:0/2418604895 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79409e760 con 0x7ff7941316c0 2023-09-30T22:24:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79259c700 1 --2- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 0x7ff794131a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff784009920 tx=0x7ff784009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad61c4b725ef40ef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79159a700 1 -- 172.21.15.32:0/2418604895 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400ba60 con 0x7ff7941316c0 2023-09-30T22:24:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79159a700 1 -- 172.21.15.32:0/2418604895 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400bbc0 con 0x7ff7941316c0 2023-09-30T22:24:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.692+0000 7ff79159a700 1 -- 172.21.15.32:0/2418604895 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784009e70 con 0x7ff7941316c0 2023-09-30T22:24:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 -- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 msgr2=0x7ff794131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 --2- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 0x7ff794131a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff784009920 tx=0x7ff784009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 -- 172.21.15.32:0/2418604895 shutdown_connections 2023-09-30T22:24:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 --2- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7941316c0 0x7ff794131a90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 --2- 172.21.15.32:0/2418604895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 -- 172.21.15.32:0/2418604895 >> 172.21.15.32:0/2418604895 conn(0x7ff794066f50 msgr2=0x7ff79409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 -- 172.21.15.32:0/2418604895 shutdown_connections 2023-09-30T22:24:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.693+0000 7ff798fea700 1 -- 172.21.15.32:0/2418604895 wait complete. 2023-09-30T22:24:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.695+0000 7ff798fea700 1 Processor -- start 2023-09-30T22:24:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.695+0000 7ff798fea700 1 -- start start 2023-09-30T22:24:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff798fea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 0x7ff79409a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff798fea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79409a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff798fea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 0x7ff79409ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff798fea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794129040 con 0x7ff7941316c0 2023-09-30T22:24:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff798fea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff794072520 con 0x7ff79409d9f0 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff798fea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff79411dd80 con 0x7ff79406acb0 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff79259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 0x7ff79409a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff792d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 0x7ff79409ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff792d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 0x7ff79409ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52196/0 (socket says 172.21.15.32:52196) 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff79259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 0x7ff79409a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:39944/0 (socket says 172.21.15.32:39944) 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff792d9d700 1 -- 172.21.15.32:0/1416742756 learned_addr learned my addr 172.21.15.32:0/1416742756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.696+0000 7ff791d9b700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff79259c700 1 -- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 msgr2=0x7ff79409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff79259c700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79409a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff79259c700 1 -- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 msgr2=0x7ff79409ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff79259c700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 0x7ff79409ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff79259c700 1 -- 172.21.15.32:0/1416742756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784009550 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff792d9d700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 0x7ff79409ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff79259c700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 0x7ff79409a3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff784016000 tx=0x7ff784004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784004510 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7840036a0 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff794130ac0 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784003a50 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.697+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79409b840 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.698+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff794074ea0 con 0x7ff79406acb0 2023-09-30T22:24:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.699+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff784003bb0 con 0x7ff79406acb0 2023-09-30T22:24:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.699+0000 7ff791d9b700 1 -- 172.21.15.32:0/1416742756 >> v1:172.21.15.32:6816/126553 conn(0x7ff77402f510 legacy=0x7ff7740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.699+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7ff784042ad0 con 0x7ff79406acb0 2023-09-30T22:24:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.708+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff784046060 con 0x7ff79406acb0 2023-09-30T22:24:41.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.759+0000 7ffb3259c700 1 -- 172.21.15.32:0/2302702628 >> v1:172.21.15.32:6816/126553 conn(0x7ffb2002f510 legacy=0x7ffb200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.781+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffb34125b20 con 0x7ffb3409d6f0 2023-09-30T22:24:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.781+0000 7ffb1b7fe700 1 -- 172.21.15.32:0/2302702628 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb34125b20 con 0x7ffb3409d6f0 2023-09-30T22:24:41.782 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 >> v1:172.21.15.32:6816/126553 conn(0x7ffb2002f510 legacy=0x7ffb200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb3409d6f0 msgr2=0x7ffb341224d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb3409d6f0 0x7ffb341224d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e930 tx=0x7ffb2400ec40 comp rx=0 tx=0).stop 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 shutdown_connections 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb3409e410 0x7ffb34122f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb3409d6f0 0x7ffb341224d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 --2- 172.21.15.32:0/2302702628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb3409e000 0x7ffb34122a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 >> 172.21.15.32:0/2302702628 conn(0x7ffb34066f50 msgr2=0x7ffb34070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 shutdown_connections 2023-09-30T22:24:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.785+0000 7ffb391d4700 1 -- 172.21.15.32:0/2302702628 wait complete. 2023-09-30T22:24:41.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.816+0000 7f33169ff700 1 Processor -- start 2023-09-30T22:24:41.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.816+0000 7f33169ff700 1 -- start start 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f33169ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009e930 0x7f331009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f33169ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f650 0x7f3310131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f33169ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3310070940 con 0x7f331009f240 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f33169ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331006fd60 con 0x7f331009e930 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f33169ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f331006fa80 con 0x7f331009f650 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f330f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f331009f240 0x7f33101317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55668/0 (socket says 172.21.15.32:55668) 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f330ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009e930 0x7f331009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f330f7fe700 1 -- 172.21.15.32:0/216650539 learned_addr learned my addr 172.21.15.32:0/216650539 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:41.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f3314f9c700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f650 0x7f3310131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f330effd700 1 -- 172.21.15.32:0/216650539 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266527133 0 0) 0x7f3310070940 con 0x7f331009f240 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.817+0000 7f330effd700 1 -- 172.21.15.32:0/216650539 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f80035b0 con 0x7f331009f240 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f3314f9c700 1 -- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009e930 msgr2=0x7f331009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f3314f9c700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009e930 0x7f331009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f3314f9c700 1 -- 172.21.15.32:0/216650539 >> v1:172.21.15.32:6789/0 conn(0x7f331009f240 legacy=0x7f33101317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f3314f9c700 1 -- 172.21.15.32:0/216650539 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331009fb60 con 0x7f331009f650 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f330ffff700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009e930 0x7f331009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f3314f9c700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f650 0x7f3310131f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3308009dc0 tx=0x7f330800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e654863dd5a8db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f330effd700 1 -- 172.21.15.32:0/216650539 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800e070 con 0x7f331009f650 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f330effd700 1 -- 172.21.15.32:0/216650539 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330800bb20 con 0x7f331009f650 2023-09-30T22:24:41.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.818+0000 7f330effd700 1 -- 172.21.15.32:0/216650539 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308012430 con 0x7f331009f650 2023-09-30T22:24:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 -- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f650 msgr2=0x7f3310131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f650 0x7f3310131f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3308009dc0 tx=0x7f330800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 -- 172.21.15.32:0/216650539 shutdown_connections 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f650 0x7f3310131f00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 --2- 172.21.15.32:0/216650539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009e930 0x7f331009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 -- 172.21.15.32:0/216650539 >> 172.21.15.32:0/216650539 conn(0x7f33100a4210 msgr2=0x7f33100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 -- 172.21.15.32:0/216650539 shutdown_connections 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.819+0000 7f33169ff700 1 -- 172.21.15.32:0/216650539 wait complete. 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.820+0000 7f33169ff700 1 Processor -- start 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.820+0000 7f33169ff700 1 -- start start 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.820+0000 7f33169ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 0x7f331006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f33169ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 0x7f331006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f33169ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009f650 0x7f331006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f33169ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101a92f0 con 0x7f331009e930 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f33169ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33100a32d0 con 0x7f331009f650 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f33169ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3310071d20 con 0x7f331009f240 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 0x7f331006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 0x7f331006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 0x7f331006bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:39972/0 (socket says 172.21.15.32:39972) 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 0x7f331006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52202/0 (socket says 172.21.15.32:52202) 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330f7fe700 1 -- 172.21.15.32:0/1077936779 learned_addr learned my addr 172.21.15.32:0/1077936779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f3314f9c700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009f650 0x7f331006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 -- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 msgr2=0x7f331006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 0x7f331006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 -- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009f650 msgr2=0x7f331006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009f650 0x7f331006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 -- 172.21.15.32:0/1077936779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000094a0 con 0x7f331009e930 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330f7fe700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 0x7f331006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f3314f9c700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009f650 0x7f331006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.821+0000 7f330ffff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 0x7f331006b880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f330000e9b0 tx=0x7f330000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.822+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000ee50 con 0x7f331009e930 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.822+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000c850 con 0x7f331009e930 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.822+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3308009a70 con 0x7f331009e930 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.822+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000cc00 con 0x7f331009e930 2023-09-30T22:24:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.822+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33100a0190 con 0x7f331009e930 2023-09-30T22:24:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.822+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3300013470 con 0x7f331009e930 2023-09-30T22:24:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.823+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f330003fc70 con 0x7f331009e930 2023-09-30T22:24:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.823+0000 7f330f7fe700 1 -- 172.21.15.32:0/1077936779 >> v1:172.21.15.32:6816/126553 conn(0x7f32fc02f510 legacy=0x7f32fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.823+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33101a0b30 con 0x7f331009e930 2023-09-30T22:24:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.828+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3300021080 con 0x7f331009e930 2023-09-30T22:24:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.831+0000 7fd85effd700 1 -- 172.21.15.32:0/3587206580 >> v1:172.21.15.32:6816/126553 conn(0x7fd84c02f510 legacy=0x7fd84c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.855+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd86012b900 con 0x7fd86006acb0 2023-09-30T22:24:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.855+0000 7fd847fff700 1 -- 172.21.15.32:0/3587206580 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd850011b40 con 0x7fd86006acb0 2023-09-30T22:24:41.856 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 >> v1:172.21.15.32:6816/126553 conn(0x7fd84c02f510 legacy=0x7fd84c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 msgr2=0x7fd860132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd860132300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9b0 tx=0x7fd85000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 shutdown_connections 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8601316c0 0x7fd860132d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd86006acb0 0x7fd860132300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 --2- 172.21.15.32:0/3587206580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86009d9f0 0x7fd860132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 >> 172.21.15.32:0/3587206580 conn(0x7fd860066f50 msgr2=0x7fd86009bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 shutdown_connections 2023-09-30T22:24:41.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.858+0000 7fd866088700 1 -- 172.21.15.32:0/3587206580 wait complete. 2023-09-30T22:24:41.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:41.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.900+0000 7ff791d9b700 1 -- 172.21.15.32:0/1416742756 >> v1:172.21.15.32:6816/126553 conn(0x7ff77402f510 legacy=0x7ff7740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:41.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.939+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff79406c360 con 0x7ff79406acb0 2023-09-30T22:24:41.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.939+0000 7ff782ffd700 1 -- 172.21.15.32:0/1416742756 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff784046030 con 0x7ff79406acb0 2023-09-30T22:24:41.940 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 >> v1:172.21.15.32:6816/126553 conn(0x7ff77402f510 legacy=0x7ff7740319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 msgr2=0x7ff79409a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 0x7ff79409a3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff784016000 tx=0x7ff784004750 comp rx=0 tx=0).stop 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 shutdown_connections 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7941316c0 0x7ff79409ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff79406acb0 0x7ff79409a3d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 --2- 172.21.15.32:0/1416742756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff79409d9f0 0x7ff79409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 >> 172.21.15.32:0/1416742756 conn(0x7ff794066f50 msgr2=0x7ff7940a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.941+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 shutdown_connections 2023-09-30T22:24:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:41.942+0000 7ff798fea700 1 -- 172.21.15.32:0/1416742756 wait complete. 2023-09-30T22:24:41.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.024+0000 7f330f7fe700 1 -- 172.21.15.32:0/1077936779 >> v1:172.21.15.32:6816/126553 conn(0x7f32fc02f510 legacy=0x7f32fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.039+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3310074ea0 con 0x7f331009e930 2023-09-30T22:24:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.040+0000 7f330cff9700 1 -- 172.21.15.32:0/1077936779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3300017020 con 0x7f331009e930 2023-09-30T22:24:42.041 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 >> v1:172.21.15.32:6816/126553 conn(0x7f32fc02f510 legacy=0x7f32fc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 msgr2=0x7f331006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 0x7f331006b880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f330000e9b0 tx=0x7f330000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 shutdown_connections 2023-09-30T22:24:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f331009f650 0x7f331006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f331009e930 0x7f331006b880 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 --2- 172.21.15.32:0/1077936779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f331009f240 0x7f331006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 >> 172.21.15.32:0/1077936779 conn(0x7f33100a4210 msgr2=0x7f33101b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.042+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 shutdown_connections 2023-09-30T22:24:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.043+0000 7f33169ff700 1 -- 172.21.15.32:0/1077936779 wait complete. 2023-09-30T22:24:42.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:42.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:42.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:42.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 Processor -- start 2023-09-30T22:24:42.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 -- start start 2023-09-30T22:24:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c180a0a30 0x7f4c180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 0x7f4c180a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c18133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c180a2a70 con 0x7f4c180a0a30 2023-09-30T22:24:42.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180a01d0 con 0x7f4c1809d630 2023-09-30T22:24:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.954+0000 7f4c1df26700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1809e2b0 con 0x7f4c18133930 2023-09-30T22:24:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 0x7f4c180a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c180a0a30 0x7f4c180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 0x7f4c180a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51500/0 (socket says 172.21.15.32:51500) 2023-09-30T22:24:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 -- 172.21.15.32:0/1822303110 learned_addr learned my addr 172.21.15.32:0/1822303110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c17fff700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c18133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 -- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 msgr2=0x7f4c18133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c18133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 -- 172.21.15.32:0/1822303110 >> v2:172.21.15.32:3300/0 conn(0x7f4c180a0a30 msgr2=0x7f4c180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 --2- 172.21.15.32:0/1822303110 >> v2:172.21.15.32:3300/0 conn(0x7f4c180a0a30 0x7f4c180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 -- 172.21.15.32:0/1822303110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1809dc50 con 0x7f4c1809d630 2023-09-30T22:24:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c177fe700 1 --2- 172.21.15.32:0/1822303110 >> v2:172.21.15.32:3300/0 conn(0x7f4c180a0a30 0x7f4c180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c17fff700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c18133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c16ffd700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 0x7f4c180a1340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0098f0 tx=0x7f4c0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d3a9f64d28a814 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c167fc700 1 -- 172.21.15.32:0/1822303110 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c011830 con 0x7f4c1809d630 2023-09-30T22:24:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c167fc700 1 -- 172.21.15.32:0/1822303110 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c011990 con 0x7f4c1809d630 2023-09-30T22:24:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.955+0000 7f4c167fc700 1 -- 172.21.15.32:0/1822303110 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c011d20 con 0x7f4c1809d630 2023-09-30T22:24:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 -- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 msgr2=0x7f4c180a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 0x7f4c180a1340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0098f0 tx=0x7f4c0c009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 -- 172.21.15.32:0/1822303110 shutdown_connections 2023-09-30T22:24:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c18133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 --2- 172.21.15.32:0/1822303110 >> v2:172.21.15.32:3300/0 conn(0x7f4c180a0a30 0x7f4c180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 --2- 172.21.15.32:0/1822303110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c1809d630 0x7f4c180a1340 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 -- 172.21.15.32:0/1822303110 >> 172.21.15.32:0/1822303110 conn(0x7f4c180a3050 msgr2=0x7f4c180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 -- 172.21.15.32:0/1822303110 shutdown_connections 2023-09-30T22:24:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.956+0000 7f4c1df26700 1 -- 172.21.15.32:0/1822303110 wait complete. 2023-09-30T22:24:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.957+0000 7f4c1df26700 1 Processor -- start 2023-09-30T22:24:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 -- start start 2023-09-30T22:24:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 0x7f4c1811f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c180a0a30 0x7f4c1811fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c181c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18122330 con 0x7f4c1809d630 2023-09-30T22:24:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c181333a0 con 0x7f4c180a0a30 2023-09-30T22:24:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c1df26700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18122530 con 0x7f4c18133930 2023-09-30T22:24:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 0x7f4c1811f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 0x7f4c1811f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52220/0 (socket says 172.21.15.32:52220) 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c177fe700 1 -- 172.21.15.32:0/2382712752 learned_addr learned my addr 172.21.15.32:0/2382712752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c16ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c180a0a30 0x7f4c1811fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.958+0000 7f4c17fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c181c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c177fe700 1 -- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 msgr2=0x7f4c181c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c177fe700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c181c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c177fe700 1 -- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c180a0a30 msgr2=0x7f4c1811fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c177fe700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c180a0a30 0x7f4c1811fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:42.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c177fe700 1 -- 172.21.15.32:0/2382712752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080094a0 con 0x7f4c1809d630 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c177fe700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 0x7f4c1811f5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800ebe0 tx=0x7f4c0800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08009800 con 0x7f4c1809d630 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0800c7d0 con 0x7f4c1809d630 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0800cb40 con 0x7f4c1809d630 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c10009a70 con 0x7f4c1809d630 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.959+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c009550 con 0x7f4c1809d630 2023-09-30T22:24:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.960+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4c0800cca0 con 0x7f4c1809d630 2023-09-30T22:24:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.960+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4c0803fc00 con 0x7f4c1809d630 2023-09-30T22:24:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.960+0000 7f4c16ffd700 1 -- 172.21.15.32:0/2382712752 >> v1:172.21.15.32:6816/126553 conn(0x7f4c0402f510 legacy=0x7f4c040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.960+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c180a4030 con 0x7f4c1809d630 2023-09-30T22:24:42.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:42.965+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c08021020 con 0x7f4c1809d630 2023-09-30T22:24:43.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.018+0000 7fe6b47b5700 1 Processor -- start 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.018+0000 7fe6b47b5700 1 -- start start 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b47b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6ac09eca0 0x7fe6ac13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b47b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b47b5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ac070ee0 con 0x7fe6ac06e570 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b47b5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac071980 con 0x7fe6ac09eca0 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b47b5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac071480 con 0x7fe6ac06f050 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b2d52700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b1d50700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6ac09eca0 0x7fe6ac13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b2551700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6ac06e570 0x7fe6ac06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55680/0 (socket says 172.21.15.32:55680) 2023-09-30T22:24:43.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b2551700 1 -- 172.21.15.32:0/1235948919 learned_addr learned my addr 172.21.15.32:0/1235948919 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:43.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b2d52700 1 --2- 172.21.15.32:0/1235948919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac13f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:39994/0 (socket says 172.21.15.32:39994) 2023-09-30T22:24:43.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b154f700 1 -- 172.21.15.32:0/1235948919 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722098537 0 0) 0x7fe6ac070ee0 con 0x7fe6ac06e570 2023-09-30T22:24:43.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b2d52700 1 -- 172.21.15.32:0/1235948919 >> v2:172.21.15.32:3300/0 conn(0x7fe6ac09eca0 msgr2=0x7fe6ac13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:43.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.019+0000 7fe6b2d52700 1 --2- 172.21.15.32:0/1235948919 >> v2:172.21.15.32:3300/0 conn(0x7fe6ac09eca0 0x7fe6ac13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b2d52700 1 -- 172.21.15.32:0/1235948919 >> v1:172.21.15.32:6789/0 conn(0x7fe6ac06e570 legacy=0x7fe6ac06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b2d52700 1 -- 172.21.15.32:0/1235948919 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac071b90 con 0x7fe6ac06f050 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b1d50700 1 --2- 172.21.15.32:0/1235948919 >> v2:172.21.15.32:3300/0 conn(0x7fe6ac09eca0 0x7fe6ac13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b2d52700 1 --2- 172.21.15.32:0/1235948919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac13f3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009d90 tx=0x7fe6a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78a4cf073599868 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b154f700 1 -- 172.21.15.32:0/1235948919 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400e070 con 0x7fe6ac06f050 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b154f700 1 -- 172.21.15.32:0/1235948919 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a400bb20 con 0x7fe6ac06f050 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.020+0000 7fe6b154f700 1 -- 172.21.15.32:0/1235948919 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4012430 con 0x7fe6ac06f050 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1235948919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 msgr2=0x7fe6ac13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1235948919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac13f3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009d90 tx=0x7fe6a400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1235948919 shutdown_connections 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1235948919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac13f3f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1235948919 >> v2:172.21.15.32:3300/0 conn(0x7fe6ac09eca0 0x7fe6ac13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1235948919 >> 172.21.15.32:0/1235948919 conn(0x7fe6ac0a2aa0 msgr2=0x7fe6ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1235948919 shutdown_connections 2023-09-30T22:24:43.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.021+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1235948919 wait complete. 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 Processor -- start 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 -- start start 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 0x7fe6ac1230e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac123620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac123b60 0x7fe6ac1d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac09a190 con 0x7fe6ac123b60 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac125210 con 0x7fe6ac06e570 2023-09-30T22:24:43.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.022+0000 7fe6b47b5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac126b60 con 0x7fe6ac06f050 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 0x7fe6ac1230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 0x7fe6ac1230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51508/0 (socket says 172.21.15.32:51508) 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 -- 172.21.15.32:0/1816641203 learned_addr learned my addr 172.21.15.32:0/1816641203 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b1d50700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac123620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2d52700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac123b60 0x7fe6ac1d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 -- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 msgr2=0x7fe6ac123620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac123620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 -- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac123b60 msgr2=0x7fe6ac1d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac123b60 0x7fe6ac1d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 -- 172.21.15.32:0/1816641203 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c0094a0 con 0x7fe6ac06e570 2023-09-30T22:24:43.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b1d50700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac123620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2d52700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac123b60 0x7fe6ac1d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6b2551700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 0x7fe6ac1230e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c520 tx=0x7fe6a800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8004c80 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.023+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8004de0 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.024+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8007310 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.024+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a4009a70 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.024+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac1265c0 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.024+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac13d030 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.025+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe6a8007470 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.025+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fe6a803fed0 con 0x7fe6ac06e570 2023-09-30T22:24:43.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.025+0000 7fe6b1d50700 1 -- 172.21.15.32:0/1816641203 >> v1:172.21.15.32:6816/126553 conn(0x7fe69002f510 legacy=0x7fe6900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.034+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6a80133c0 con 0x7fe6ac06e570 2023-09-30T22:24:43.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.121+0000 7f0e85b3a700 1 Processor -- start 2023-09-30T22:24:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.121+0000 7f0e85b3a700 1 -- start start 2023-09-30T22:24:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e85b3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 0x7f0e8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e85b3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e85b3a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8009f130 con 0x7f0e8009e410 2023-09-30T22:24:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e85b3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8009f520 con 0x7f0e8009d6f0 2023-09-30T22:24:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e85b3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8009e930 con 0x7f0e8009e000 2023-09-30T22:24:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e7f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 0x7f0e8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e7f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 0x7f0e8009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51522/0 (socket says 172.21.15.32:51522) 2023-09-30T22:24:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e7f7fe700 1 -- 172.21.15.32:0/1218061301 learned_addr learned my addr 172.21.15.32:0/1218061301 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e7ffff700 1 --1- 172.21.15.32:0/1218061301 >> v1:172.21.15.32:6789/0 conn(0x7f0e8009e410 0x7f0e80130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55696/0 (socket says 172.21.15.32:55696) 2023-09-30T22:24:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.122+0000 7f0e7effd700 1 --2- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7e7fc700 1 -- 172.21.15.32:0/1218061301 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116366766 0 0) 0x7f0e8009f130 con 0x7f0e8009e410 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7f7fe700 1 -- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 msgr2=0x7f0e80130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7f7fe700 1 --2- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7f7fe700 1 -- 172.21.15.32:0/1218061301 >> v1:172.21.15.32:6789/0 conn(0x7f0e8009e410 legacy=0x7f0e80130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7f7fe700 1 -- 172.21.15.32:0/1218061301 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8009b2d0 con 0x7f0e8009d6f0 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7f7fe700 1 --2- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 0x7f0e8009dac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009920 tx=0x7f0e70009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130da9367f8a2273 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7e7fc700 1 -- 172.21.15.32:0/1218061301 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011830 con 0x7f0e8009d6f0 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7e7fc700 1 -- 172.21.15.32:0/1218061301 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70011990 con 0x7f0e8009d6f0 2023-09-30T22:24:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.123+0000 7f0e7e7fc700 1 -- 172.21.15.32:0/1218061301 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011d20 con 0x7f0e8009d6f0 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 -- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 msgr2=0x7f0e8009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 0x7f0e8009dac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009920 tx=0x7f0e70009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 -- 172.21.15.32:0/1218061301 shutdown_connections 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009d6f0 0x7f0e8009dac0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/1218061301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 -- 172.21.15.32:0/1218061301 >> 172.21.15.32:0/1218061301 conn(0x7f0e80066f50 msgr2=0x7f0e8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 -- 172.21.15.32:0/1218061301 shutdown_connections 2023-09-30T22:24:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.124+0000 7f0e85b3a700 1 -- 172.21.15.32:0/1218061301 wait complete. 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.125+0000 7f0e85b3a700 1 Processor -- start 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.125+0000 7f0e85b3a700 1 -- start start 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e85b3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e8009d6f0 0x7f0e801365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e85b3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80136af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e85b3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009e410 0x7f0e80137030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e85b3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8009c790 con 0x7f0e8009d6f0 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e85b3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8013adb0 con 0x7f0e8009e410 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e85b3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80139b30 con 0x7f0e8009e000 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80136af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009e410 0x7f0e80137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80136af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40030/0 (socket says 172.21.15.32:40030) 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7effd700 1 -- 172.21.15.32:0/2661490582 learned_addr learned my addr 172.21.15.32:0/2661490582 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7f7fe700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e8009d6f0 0x7f0e801365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7effd700 1 -- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009e410 msgr2=0x7f0e80137030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.126+0000 7f0e7effd700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009e410 0x7f0e80137030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e7effd700 1 -- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e8009d6f0 msgr2=0x7f0e801365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e7effd700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e8009d6f0 0x7f0e801365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e7effd700 1 -- 172.21.15.32:0/2661490582 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e70009550 con 0x7f0e8009e000 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e7effd700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80136af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e9b0 tx=0x7f0e7400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74009710 con 0x7f0e8009e000 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7400ee50 con 0x7f0e8009e000 2023-09-30T22:24:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800a3b10 con 0x7f0e8009e000 2023-09-30T22:24:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400ca70 con 0x7f0e8009e000 2023-09-30T22:24:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.127+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8013a860 con 0x7f0e8009e000 2023-09-30T22:24:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.128+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0e7400cbd0 con 0x7f0e8009e000 2023-09-30T22:24:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.129+0000 7f0e7f7fe700 1 -- 172.21.15.32:0/2661490582 >> v1:172.21.15.32:6816/126553 conn(0x7f0e6c02f510 legacy=0x7f0e6c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.129+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f0e74024350 con 0x7f0e8009e000 2023-09-30T22:24:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.129+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e80074ea0 con 0x7f0e8009e000 2023-09-30T22:24:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.138+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e74021030 con 0x7f0e8009e000 2023-09-30T22:24:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.161+0000 7f4c16ffd700 1 -- 172.21.15.32:0/2382712752 >> v1:172.21.15.32:6816/126553 conn(0x7f4c0402f510 legacy=0x7f4c040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.189+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c180a0410 con 0x7f4c1809d630 2023-09-30T22:24:43.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.190+0000 7f4bfffff700 1 -- 172.21.15.32:0/2382712752 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c080232f0 con 0x7f4c1809d630 2023-09-30T22:24:43.191 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 >> v1:172.21.15.32:6816/126553 conn(0x7f4c0402f510 legacy=0x7f4c040319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 msgr2=0x7f4c1811f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 0x7f4c1811f5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800ebe0 tx=0x7f4c0800eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 shutdown_connections 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c18133930 0x7f4c181c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1809d630 0x7f4c1811f5a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 --2- 172.21.15.32:0/2382712752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c180a0a30 0x7f4c1811fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 >> 172.21.15.32:0/2382712752 conn(0x7f4c180a3050 msgr2=0x7f4c1809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.192+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 shutdown_connections 2023-09-30T22:24:43.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.193+0000 7f4c1df26700 1 -- 172.21.15.32:0/2382712752 wait complete. 2023-09-30T22:24:43.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.204+0000 7f09bbc20700 1 Processor -- start 2023-09-30T22:24:43.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.204+0000 7f09bbc20700 1 -- start start 2023-09-30T22:24:43.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09bbc20700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b406a770 0x7f09b406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09bbc20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09bbc20700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b4131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09bbc20700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b409ece0 con 0x7f09b406a770 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09bbc20700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b409e760 con 0x7f09b406b080 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09bbc20700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09b409e040 con 0x7f09b4131b40 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b4131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09b99bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09b406a770 0x7f09b406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b4131fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40036/0 (socket says 172.21.15.32:40036) 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 -- 172.21.15.32:0/477469796 learned_addr learned my addr 172.21.15.32:0/477469796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09b91bb700 1 --2- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 -- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 msgr2=0x7f09b4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 --2- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b4131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 -- 172.21.15.32:0/477469796 >> v2:172.21.15.32:3300/0 conn(0x7f09b406a770 msgr2=0x7f09b406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 --2- 172.21.15.32:0/477469796 >> v2:172.21.15.32:3300/0 conn(0x7f09b406a770 0x7f09b406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.205+0000 7f09ba1bd700 1 -- 172.21.15.32:0/477469796 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b409f630 con 0x7f09b4131b40 2023-09-30T22:24:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.206+0000 7f09ba1bd700 1 --2- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b4131fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009e50 tx=0x7f09ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da99f41b786e132b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.206+0000 7f09b89ba700 1 -- 172.21.15.32:0/477469796 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00e070 con 0x7f09b4131b40 2023-09-30T22:24:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.206+0000 7f09b89ba700 1 -- 172.21.15.32:0/477469796 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac00b910 con 0x7f09b4131b40 2023-09-30T22:24:43.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.206+0000 7f09b89ba700 1 -- 172.21.15.32:0/477469796 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00bc80 con 0x7f09b4131b40 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 -- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 msgr2=0x7f09b4131fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 --2- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b4131fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009e50 tx=0x7f09ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 -- 172.21.15.32:0/477469796 shutdown_connections 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 --2- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b4131fa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 --2- 172.21.15.32:0/477469796 >> v2:172.21.15.32:3300/0 conn(0x7f09b406a770 0x7f09b406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 --2- 172.21.15.32:0/477469796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b4131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.207+0000 7f09bbc20700 1 -- 172.21.15.32:0/477469796 >> 172.21.15.32:0/477469796 conn(0x7f09b4066f50 msgr2=0x7f09b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.208+0000 7f09bbc20700 1 -- 172.21.15.32:0/477469796 shutdown_connections 2023-09-30T22:24:43.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.208+0000 7f09bbc20700 1 -- 172.21.15.32:0/477469796 wait complete. 2023-09-30T22:24:43.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:43.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 Processor -- start 2023-09-30T22:24:43.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 -- start start 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b406a770 0x7f09b411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b411e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4120680 con 0x7f09b406a770 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4120230 con 0x7f09b406b080 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09bbc20700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4071e60 con 0x7f09b4131b40 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09b91bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09ba1bd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b411e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09b91bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b411ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51536/0 (socket says 172.21.15.32:51536) 2023-09-30T22:24:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.209+0000 7f09b91bb700 1 -- 172.21.15.32:0/649039130 learned_addr learned my addr 172.21.15.32:0/649039130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09b99bc700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b406a770 0x7f09b411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09ba1bd700 1 -- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 msgr2=0x7f09b411ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09ba1bd700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b411ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09ba1bd700 1 -- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b406a770 msgr2=0x7f09b411d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09ba1bd700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b406a770 0x7f09b411d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09ba1bd700 1 -- 172.21.15.32:0/649039130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac009b30 con 0x7f09b4131b40 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09ba1bd700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b411e310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09ac015000 tx=0x7f09ac004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac004450 con 0x7f09b4131b40 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac0045f0 con 0x7f09b4131b40 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00ce80 con 0x7f09b4131b40 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b411ff70 con 0x7f09b4131b40 2023-09-30T22:24:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.210+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b41bc0d0 con 0x7f09b4131b40 2023-09-30T22:24:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.211+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f09ac01c410 con 0x7f09b4131b40 2023-09-30T22:24:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.212+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f09ac041280 con 0x7f09b4131b40 2023-09-30T22:24:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.212+0000 7f09b99bc700 1 -- 172.21.15.32:0/649039130 >> v1:172.21.15.32:6816/126553 conn(0x7f099802f510 legacy=0x7f09980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.212+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b4120470 con 0x7f09b4131b40 2023-09-30T22:24:43.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.221+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09ac01c6c0 con 0x7f09b4131b40 2023-09-30T22:24:43.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.226+0000 7fe6b1d50700 1 -- 172.21.15.32:0/1816641203 >> v1:172.21.15.32:6816/126553 conn(0x7fe69002f510 legacy=0x7fe6900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.254+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe6ac074ea0 con 0x7fe6ac06e570 2023-09-30T22:24:43.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.255+0000 7fe6a2ffd700 1 -- 172.21.15.32:0/1816641203 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6a8011360 con 0x7fe6ac06e570 2023-09-30T22:24:43.256 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 >> v1:172.21.15.32:6816/126553 conn(0x7fe69002f510 legacy=0x7fe6900319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 msgr2=0x7fe6ac1230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 0x7fe6ac1230e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c520 tx=0x7fe6a800c830 comp rx=0 tx=0).stop 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 shutdown_connections 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6ac123b60 0x7fe6ac1d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6ac06e570 0x7fe6ac1230e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 --2- 172.21.15.32:0/1816641203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6ac06f050 0x7fe6ac123620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 >> 172.21.15.32:0/1816641203 conn(0x7fe6ac0a2aa0 msgr2=0x7fe6ac09a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 shutdown_connections 2023-09-30T22:24:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.257+0000 7fe6b47b5700 1 -- 172.21.15.32:0/1816641203 wait complete. 2023-09-30T22:24:43.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.330+0000 7f0e7f7fe700 1 -- 172.21.15.32:0/2661490582 >> v1:172.21.15.32:6816/126553 conn(0x7f0e6c02f510 legacy=0x7f0e6c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.348+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0e80071e90 con 0x7f0e8009e000 2023-09-30T22:24:43.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.349+0000 7f0e67fff700 1 -- 172.21.15.32:0/2661490582 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e74021020 con 0x7f0e8009e000 2023-09-30T22:24:43.350 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 >> v1:172.21.15.32:6816/126553 conn(0x7f0e6c02f510 legacy=0x7f0e6c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 msgr2=0x7f0e80136af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80136af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e9b0 tx=0x7f0e7400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 shutdown_connections 2023-09-30T22:24:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e8009e410 0x7f0e80137030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e8009d6f0 0x7f0e801365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 --2- 172.21.15.32:0/2661490582 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e8009e000 0x7f0e80136af0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 >> 172.21.15.32:0/2661490582 conn(0x7f0e80066f50 msgr2=0x7f0e80070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 shutdown_connections 2023-09-30T22:24:43.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.351+0000 7f0e85b3a700 1 -- 172.21.15.32:0/2661490582 wait complete. 2023-09-30T22:24:43.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:43.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.413+0000 7f09b99bc700 1 -- 172.21.15.32:0/649039130 >> v1:172.21.15.32:6816/126553 conn(0x7f099802f510 legacy=0x7f09980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.430+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f09b4074ea0 con 0x7f09b4131b40 2023-09-30T22:24:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.430+0000 7f09aa7fc700 1 -- 172.21.15.32:0/649039130 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09ac02a020 con 0x7f09b4131b40 2023-09-30T22:24:43.431 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 >> v1:172.21.15.32:6816/126553 conn(0x7f099802f510 legacy=0x7f09980319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 msgr2=0x7f09b411e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b411e310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09ac015000 tx=0x7f09ac004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 shutdown_connections 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b4131b40 0x7f09b411e310 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b406a770 0x7f09b411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 --2- 172.21.15.32:0/649039130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b406b080 0x7f09b411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 >> 172.21.15.32:0/649039130 conn(0x7f09b4066f50 msgr2=0x7f09b40708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 shutdown_connections 2023-09-30T22:24:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:43.433+0000 7f09bbc20700 1 -- 172.21.15.32:0/649039130 wait complete. 2023-09-30T22:24:43.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:44.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:44.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.358+0000 7f68ac524700 1 Processor -- start 2023-09-30T22:24:44.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.358+0000 7f68ac524700 1 -- start start 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68ac524700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68ac524700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68a4138d80 0x7f68a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68ac524700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a40a2a70 con 0x7f68a409d630 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68ac524700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a40a01d0 con 0x7f68a4138d80 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68ac524700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a409e2b0 con 0x7f68a40a0a30 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68aaac1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68a4138d80 0x7f68a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68aa2c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68a9abf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68a409d630 0x7f68a40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55698/0 (socket says 172.21.15.32:55698) 2023-09-30T22:24:44.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.359+0000 7f68a9abf700 1 -- 172.21.15.32:0/3779842318 learned_addr learned my addr 172.21.15.32:0/3779842318 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68aaac1700 1 -- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 msgr2=0x7f68a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68aaac1700 1 --2- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68aaac1700 1 -- 172.21.15.32:0/3779842318 >> v1:172.21.15.32:6789/0 conn(0x7f68a409d630 legacy=0x7f68a40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68aaac1700 1 -- 172.21.15.32:0/3779842318 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a409dc50 con 0x7f68a4138d80 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68aa2c0700 1 --2- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68aaac1700 1 --2- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a4138d80 0x7f68a413b160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f689c009840 tx=0x7f689c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1742ae99d8d9d64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68a92be700 1 -- 172.21.15.32:0/3779842318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011830 con 0x7f68a4138d80 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68a92be700 1 -- 172.21.15.32:0/3779842318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c011990 con 0x7f68a4138d80 2023-09-30T22:24:44.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.360+0000 7f68a92be700 1 -- 172.21.15.32:0/3779842318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011d20 con 0x7f68a4138d80 2023-09-30T22:24:44.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 -- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a4138d80 msgr2=0x7f68a413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 --2- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a4138d80 0x7f68a413b160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f689c009840 tx=0x7f689c009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 -- 172.21.15.32:0/3779842318 shutdown_connections 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 --2- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a4138d80 0x7f68a413b160 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 --2- 172.21.15.32:0/3779842318 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 -- 172.21.15.32:0/3779842318 >> 172.21.15.32:0/3779842318 conn(0x7f68a40a3050 msgr2=0x7f68a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 -- 172.21.15.32:0/3779842318 shutdown_connections 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.361+0000 7f68ac524700 1 -- 172.21.15.32:0/3779842318 wait complete. 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.362+0000 7f68ac524700 1 Processor -- start 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.362+0000 7f68ac524700 1 -- start start 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68ac524700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 0x7f68a4133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68ac524700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a4133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68ac524700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68a4138d80 0x7f68a4134310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68ac524700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a41371b0 con 0x7f68a409d630 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68ac524700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4138030 con 0x7f68a40a0a30 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68ac524700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68a41373b0 con 0x7f68a4138d80 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 0x7f68a4133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 0x7f68a4133890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52258/0 (socket says 172.21.15.32:52258) 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 -- 172.21.15.32:0/1012640282 learned_addr learned my addr 172.21.15.32:0/1012640282 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68a9abf700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a4133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aaac1700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68a4138d80 0x7f68a4134310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 -- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68a4138d80 msgr2=0x7f68a4134310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68a4138d80 0x7f68a4134310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 -- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 msgr2=0x7f68a4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a4133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 -- 172.21.15.32:0/1012640282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894009a70 con 0x7f68a409d630 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68a9abf700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68aa2c0700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 0x7f68a4133890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6894009f60 tx=0x7f6894009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.363+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689400ce70 con 0x7f68a409d630 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.364+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689400f040 con 0x7f68a409d630 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.364+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940146a0 con 0x7f68a409d630 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.364+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c0094a0 con 0x7f68a409d630 2023-09-30T22:24:44.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.364+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a40a0410 con 0x7f68a409d630 2023-09-30T22:24:44.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.364+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6894014800 con 0x7f68a409d630 2023-09-30T22:24:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.365+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a40747f0 con 0x7f68a409d630 2023-09-30T22:24:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.365+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f68940409a0 con 0x7f68a409d630 2023-09-30T22:24:44.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.365+0000 7f68a9abf700 1 -- 172.21.15.32:0/1012640282 >> v1:172.21.15.32:6816/126553 conn(0x7f689002f510 legacy=0x7f68900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.370+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6894007d90 con 0x7f68a409d630 2023-09-30T22:24:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.414+0000 7f63566dc700 1 Processor -- start 2023-09-30T22:24:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.414+0000 7f63566dc700 1 -- start start 2023-09-30T22:24:44.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f63566dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f635009e930 0x7f635009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f63566dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 0x7f63501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f63566dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6350070940 con 0x7f635009f650 2023-09-30T22:24:44.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f63566dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635006fd60 con 0x7f635009e930 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f63566dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f635006fa80 con 0x7f635009f240 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 0x7f63501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f635009e930 0x7f635009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 0x7f63501317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40062/0 (socket says 172.21.15.32:40062) 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 -- 172.21.15.32:0/1112789954 learned_addr learned my addr 172.21.15.32:0/1112789954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 -- 172.21.15.32:0/1112789954 >> v2:172.21.15.32:3300/0 conn(0x7f635009e930 msgr2=0x7f635009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 --2- 172.21.15.32:0/1112789954 >> v2:172.21.15.32:3300/0 conn(0x7f635009e930 0x7f635009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 -- 172.21.15.32:0/1112789954 >> v1:172.21.15.32:6789/0 conn(0x7f635009f650 legacy=0x7f6350131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.415+0000 7f634f7fe700 1 -- 172.21.15.32:0/1112789954 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635009fb60 con 0x7f635009f240 2023-09-30T22:24:44.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.416+0000 7f634f7fe700 1 --2- 172.21.15.32:0/1112789954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 0x7f63501317f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63440098f0 tx=0x7f6344009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc1015b317cfba5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.416+0000 7f634effd700 1 -- 172.21.15.32:0/1112789954 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344011830 con 0x7f635009f240 2023-09-30T22:24:44.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.416+0000 7f634effd700 1 -- 172.21.15.32:0/1112789954 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6344011990 con 0x7f635009f240 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.416+0000 7f634effd700 1 -- 172.21.15.32:0/1112789954 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344011d20 con 0x7f635009f240 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 -- 172.21.15.32:0/1112789954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 msgr2=0x7f63501317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 --2- 172.21.15.32:0/1112789954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 0x7f63501317f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63440098f0 tx=0x7f6344009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 -- 172.21.15.32:0/1112789954 shutdown_connections 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 --2- 172.21.15.32:0/1112789954 >> v2:172.21.15.32:3300/0 conn(0x7f635009e930 0x7f635009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 --2- 172.21.15.32:0/1112789954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f240 0x7f63501317f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 -- 172.21.15.32:0/1112789954 >> 172.21.15.32:0/1112789954 conn(0x7f63500a4210 msgr2=0x7f63500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 -- 172.21.15.32:0/1112789954 shutdown_connections 2023-09-30T22:24:44.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.417+0000 7f63566dc700 1 -- 172.21.15.32:0/1112789954 wait complete. 2023-09-30T22:24:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.418+0000 7f63566dc700 1 Processor -- start 2023-09-30T22:24:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 -- start start 2023-09-30T22:24:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009e930 0x7f635006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 0x7f635006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f650 0x7f635006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6350071d20 con 0x7f635009e930 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635006d580 con 0x7f635009f240 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f63566dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63501ab710 con 0x7f635009f650 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.419+0000 7f634f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 0x7f635006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 0x7f635006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51572/0 (socket says 172.21.15.32:51572) 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f6354c79700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f650 0x7f635006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 -- 172.21.15.32:0/2467655630 learned_addr learned my addr 172.21.15.32:0/2467655630 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634ffff700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009e930 0x7f635006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 -- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f650 msgr2=0x7f635006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f650 0x7f635006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 -- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009e930 msgr2=0x7f635006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009e930 0x7f635006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 -- 172.21.15.32:0/2467655630 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6344009550 con 0x7f635009f240 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634ffff700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009e930 0x7f635006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634f7fe700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 0x7f635006bdf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6344000c00 tx=0x7f6344004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440036a0 con 0x7f635009f240 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6344003800 con 0x7f635009f240 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344003bb0 con 0x7f635009f240 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.420+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63500a0190 con 0x7f635009f240 2023-09-30T22:24:44.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.421+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63501aa2d0 con 0x7f635009f240 2023-09-30T22:24:44.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.421+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6344021070 con 0x7f635009f240 2023-09-30T22:24:44.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.422+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f63440410f0 con 0x7f635009f240 2023-09-30T22:24:44.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.422+0000 7f634ffff700 1 -- 172.21.15.32:0/2467655630 >> v1:172.21.15.32:6816/126553 conn(0x7f633c02f510 legacy=0x7f633c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.422+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63501a2b40 con 0x7f635009f240 2023-09-30T22:24:44.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.431+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6344015080 con 0x7f635009f240 2023-09-30T22:24:44.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.515+0000 7fc9d8519700 1 Processor -- start 2023-09-30T22:24:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.515+0000 7fc9d8519700 1 -- start start 2023-09-30T22:24:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d8519700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 0x7fc9d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d8519700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009e410 0x7fc9d0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d8519700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d009b120 con 0x7fc9d009e000 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d8519700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d006b460 con 0x7fc9d009d6f0 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d8519700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d009a7e0 con 0x7fc9d009e410 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 0x7fc9d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 0x7fc9d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51584/0 (socket says 172.21.15.32:51584) 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 -- 172.21.15.32:0/479046432 learned_addr learned my addr 172.21.15.32:0/479046432 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d6ab6700 1 --2- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009e410 0x7fc9d0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 -- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009e410 msgr2=0x7fc9d0134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 --2- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009e410 0x7fc9d0134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 -- 172.21.15.32:0/479046432 >> v1:172.21.15.32:6789/0 conn(0x7fc9d009e000 legacy=0x7fc9d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:44.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.516+0000 7fc9d62b5700 1 -- 172.21.15.32:0/479046432 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d009a500 con 0x7fc9d009d6f0 2023-09-30T22:24:44.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.517+0000 7fc9d62b5700 1 --2- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 0x7fc9d009dac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009920 tx=0x7fc9cc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db18cfa70795d35 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.517+0000 7fc9d52b3700 1 -- 172.21.15.32:0/479046432 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc004230 con 0x7fc9d009d6f0 2023-09-30T22:24:44.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.517+0000 7fc9d52b3700 1 -- 172.21.15.32:0/479046432 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc004390 con 0x7fc9d009d6f0 2023-09-30T22:24:44.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.517+0000 7fc9d52b3700 1 -- 172.21.15.32:0/479046432 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0036a0 con 0x7fc9d009d6f0 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 -- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 msgr2=0x7fc9d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 --2- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 0x7fc9d009dac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009920 tx=0x7fc9cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 -- 172.21.15.32:0/479046432 shutdown_connections 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 --2- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009e410 0x7fc9d0134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 --2- 172.21.15.32:0/479046432 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009d6f0 0x7fc9d009dac0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 -- 172.21.15.32:0/479046432 >> 172.21.15.32:0/479046432 conn(0x7fc9d0066f50 msgr2=0x7fc9d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 -- 172.21.15.32:0/479046432 shutdown_connections 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.518+0000 7fc9d8519700 1 -- 172.21.15.32:0/479046432 wait complete. 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.519+0000 7fc9d8519700 1 Processor -- start 2023-09-30T22:24:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.519+0000 7fc9d8519700 1 -- start start 2023-09-30T22:24:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d8519700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 0x7fc9d01225c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d8519700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009e000 0x7fc9d0122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d8519700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d009e410 0x7fc9d0123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d8519700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d00a1db0 con 0x7fc9d009e410 2023-09-30T22:24:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d8519700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d009c710 con 0x7fc9d009e000 2023-09-30T22:24:44.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d8519700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d0124f60 con 0x7fc9d009d6f0 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d62b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 0x7fc9d01225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d62b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 0x7fc9d01225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40094/0 (socket says 172.21.15.32:40094) 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d62b5700 1 -- 172.21.15.32:0/614357564 learned_addr learned my addr 172.21.15.32:0/614357564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.520+0000 7fc9d5ab4700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009e000 0x7fc9d0122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d62b5700 1 -- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009e000 msgr2=0x7fc9d0122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d62b5700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009e000 0x7fc9d0122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d6ab6700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d009e410 0x7fc9d0123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d62b5700 1 -- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d009e410 msgr2=0x7fc9d0123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d62b5700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d009e410 0x7fc9d0123040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d62b5700 1 -- 172.21.15.32:0/614357564 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009550 con 0x7fc9d009d6f0 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d6ab6700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d009e410 0x7fc9d0123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d62b5700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 0x7fc9d01225c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098f0 tx=0x7fc9cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc003e60 con 0x7fc9d009d6f0 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d01257e0 con 0x7fc9d009d6f0 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc011630 con 0x7fc9d009d6f0 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d01267e0 con 0x7fc9d009d6f0 2023-09-30T22:24:44.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.521+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0119c0 con 0x7fc9d009d6f0 2023-09-30T22:24:44.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.522+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc9cc021070 con 0x7fc9d009d6f0 2023-09-30T22:24:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.522+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b80052e0 con 0x7fc9d009d6f0 2023-09-30T22:24:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.523+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc9cc041310 con 0x7fc9d009d6f0 2023-09-30T22:24:44.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.524+0000 7fc9d5ab4700 1 -- 172.21.15.32:0/614357564 >> v1:172.21.15.32:6816/126553 conn(0x7fc9b402f510 legacy=0x7fc9b40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.532+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9cc02cd70 con 0x7fc9d009d6f0 2023-09-30T22:24:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.566+0000 7f68a9abf700 1 -- 172.21.15.32:0/1012640282 >> v1:172.21.15.32:6816/126553 conn(0x7f689002f510 legacy=0x7f68900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 Processor -- start 2023-09-30T22:24:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 -- start start 2023-09-30T22:24:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e006e570 0x7f02e006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e009eca0 0x7f02e006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0070ee0 con 0x7f02e0134610 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e0071480 con 0x7f02e006e570 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.593+0000 7f02e8688700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e006f3c0 con 0x7f02e009eca0 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6424700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02e006e570 0x7f02e006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e5c23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e009eca0 0x7f02e006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6c25700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02e0134610 0x7f02e0134a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55712/0 (socket says 172.21.15.32:55712) 2023-09-30T22:24:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6c25700 1 -- 172.21.15.32:0/1963848418 learned_addr learned my addr 172.21.15.32:0/1963848418 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6424700 1 -- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e009eca0 msgr2=0x7f02e006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6424700 1 --2- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e009eca0 0x7f02e006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6424700 1 -- 172.21.15.32:0/1963848418 >> v1:172.21.15.32:6789/0 conn(0x7f02e0134610 legacy=0x7f02e0134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e5422700 1 -- 172.21.15.32:0/1963848418 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507476864 0 0) 0x7f02e0070ee0 con 0x7f02e0134610 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6424700 1 -- 172.21.15.32:0/1963848418 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0071b90 con 0x7f02e006e570 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e5c23700 1 --2- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e009eca0 0x7f02e006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.594+0000 7f02e6424700 1 --2- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e006e570 0x7f02e006e940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009840 tx=0x7f02dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd09a194327ad396 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.595+0000 7f02e5422700 1 -- 172.21.15.32:0/1963848418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc011830 con 0x7f02e006e570 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.595+0000 7f02e5422700 1 -- 172.21.15.32:0/1963848418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc011990 con 0x7f02e006e570 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.595+0000 7f02e5422700 1 -- 172.21.15.32:0/1963848418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc011d20 con 0x7f02e006e570 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.595+0000 7f02e8688700 1 -- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e006e570 msgr2=0x7f02e006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.595+0000 7f02e8688700 1 --2- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e006e570 0x7f02e006e940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009840 tx=0x7f02dc009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.596+0000 7f02e8688700 1 -- 172.21.15.32:0/1963848418 shutdown_connections 2023-09-30T22:24:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.596+0000 7f02e8688700 1 --2- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e006e570 0x7f02e006e940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.596+0000 7f02e8688700 1 --2- 172.21.15.32:0/1963848418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e009eca0 0x7f02e006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.596+0000 7f02e8688700 1 -- 172.21.15.32:0/1963848418 >> 172.21.15.32:0/1963848418 conn(0x7f02e00a2aa0 msgr2=0x7f02e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.596+0000 7f02e8688700 1 -- 172.21.15.32:0/1963848418 shutdown_connections 2023-09-30T22:24:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.596+0000 7f02e8688700 1 -- 172.21.15.32:0/1963848418 wait complete. 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 Processor -- start 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 -- start start 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e006e570 0x7f02e0139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 0x7f02e0132630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02e013a0d0 0x7f02e0132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e013b7e0 con 0x7f02e0134610 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00a1e30 con 0x7f02e013a0d0 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e8688700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0070610 con 0x7f02e006e570 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e5c23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 0x7f02e0132630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e5c23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 0x7f02e0132630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52296/0 (socket says 172.21.15.32:52296) 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e5c23700 1 -- 172.21.15.32:0/769401143 learned_addr learned my addr 172.21.15.32:0/769401143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.597+0000 7f02e6424700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e006e570 0x7f02e0139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6c25700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02e013a0d0 0x7f02e0132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6424700 1 -- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02e013a0d0 msgr2=0x7f02e0132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6424700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02e013a0d0 0x7f02e0132b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6424700 1 -- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 msgr2=0x7f02e0132630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6424700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 0x7f02e0132630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6424700 1 -- 172.21.15.32:0/769401143 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00094a0 con 0x7f02e006e570 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e5c23700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 0x7f02e0132630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e6424700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e006e570 0x7f02e0139b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02dc016000 tx=0x7f02dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0036a0 con 0x7f02e006e570 2023-09-30T22:24:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc01f070 con 0x7f02e006e570 2023-09-30T22:24:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc0094a0 con 0x7f02e006e570 2023-09-30T22:24:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc003a00 con 0x7f02e006e570 2023-09-30T22:24:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.598+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e013abb0 con 0x7f02e006e570 2023-09-30T22:24:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.600+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e0135120 con 0x7f02e006e570 2023-09-30T22:24:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.600+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f02dc003b60 con 0x7f02e006e570 2023-09-30T22:24:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.601+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f02dc021070 con 0x7f02e006e570 2023-09-30T22:24:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.601+0000 7f02e5c23700 1 -- 172.21.15.32:0/769401143 >> v1:172.21.15.32:6816/126553 conn(0x7f02c402f510 legacy=0x7f02c40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.605+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f68a4072980 con 0x7f68a409d630 2023-09-30T22:24:44.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.605+0000 7f68a2ffd700 1 -- 172.21.15.32:0/1012640282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6894007330 con 0x7f68a409d630 2023-09-30T22:24:44.606 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 >> v1:172.21.15.32:6816/126553 conn(0x7f689002f510 legacy=0x7f68900319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 msgr2=0x7f68a4133890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 0x7f68a4133890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6894009f60 tx=0x7f6894009f90 comp rx=0 tx=0).stop 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 shutdown_connections 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68a4138d80 0x7f68a4134310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68a409d630 0x7f68a4133890 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 --2- 172.21.15.32:0/1012640282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68a40a0a30 0x7f68a4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 >> 172.21.15.32:0/1012640282 conn(0x7f68a40a3050 msgr2=0x7f68a409f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 shutdown_connections 2023-09-30T22:24:44.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.608+0000 7f68ac524700 1 -- 172.21.15.32:0/1012640282 wait complete. 2023-09-30T22:24:44.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.609+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02dc003e10 con 0x7f02e006e570 2023-09-30T22:24:44.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.623+0000 7f634ffff700 1 -- 172.21.15.32:0/2467655630 >> v1:172.21.15.32:6816/126553 conn(0x7f633c02f510 legacy=0x7f633c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.651+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6350074ea0 con 0x7f635009f240 2023-09-30T22:24:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.652+0000 7f634cff9700 1 -- 172.21.15.32:0/2467655630 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6344029020 con 0x7f635009f240 2023-09-30T22:24:44.652 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 >> v1:172.21.15.32:6816/126553 conn(0x7f633c02f510 legacy=0x7f633c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 msgr2=0x7f635006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 0x7f635006bdf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6344000c00 tx=0x7f6344004750 comp rx=0 tx=0).stop 2023-09-30T22:24:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 shutdown_connections 2023-09-30T22:24:44.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f635009f650 0x7f635006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009e930 0x7f635006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 --2- 172.21.15.32:0/2467655630 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f240 0x7f635006bdf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 >> 172.21.15.32:0/2467655630 conn(0x7f63500a4210 msgr2=0x7f635009dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 shutdown_connections 2023-09-30T22:24:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.654+0000 7f63566dc700 1 -- 172.21.15.32:0/2467655630 wait complete. 2023-09-30T22:24:44.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.725+0000 7fc9d5ab4700 1 -- 172.21.15.32:0/614357564 >> v1:172.21.15.32:6816/126553 conn(0x7fc9b402f510 legacy=0x7fc9b40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.746+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc9b8005860 con 0x7fc9d009d6f0 2023-09-30T22:24:44.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.747+0000 7fc9c6ffd700 1 -- 172.21.15.32:0/614357564 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9cc02cb90 con 0x7fc9d009d6f0 2023-09-30T22:24:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 >> v1:172.21.15.32:6816/126553 conn(0x7fc9b402f510 legacy=0x7fc9b40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 msgr2=0x7fc9d01225c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 0x7fc9d01225c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098f0 tx=0x7fc9cc004750 comp rx=0 tx=0).stop 2023-09-30T22:24:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 shutdown_connections 2023-09-30T22:24:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d009e410 0x7fc9d0123040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9d009d6f0 0x7fc9d01225c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 --2- 172.21.15.32:0/614357564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9d009e000 0x7fc9d0122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 >> 172.21.15.32:0/614357564 conn(0x7fc9d0066f50 msgr2=0x7fc9d00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 shutdown_connections 2023-09-30T22:24:44.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.749+0000 7fc9d8519700 1 -- 172.21.15.32:0/614357564 wait complete. 2023-09-30T22:24:44.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:44.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.802+0000 7f02e5c23700 1 -- 172.21.15.32:0/769401143 >> v1:172.21.15.32:6816/126553 conn(0x7f02c402f510 legacy=0x7f02c40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.817+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02e0074ea0 con 0x7f02e006e570 2023-09-30T22:24:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.818+0000 7f02d6ffd700 1 -- 172.21.15.32:0/769401143 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02dc015030 con 0x7f02e006e570 2023-09-30T22:24:44.819 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 >> v1:172.21.15.32:6816/126553 conn(0x7f02c402f510 legacy=0x7f02c40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e006e570 msgr2=0x7f02e0139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e006e570 0x7f02e0139b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02dc016000 tx=0x7f02dc004750 comp rx=0 tx=0).stop 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 shutdown_connections 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02e013a0d0 0x7f02e0132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02e006e570 0x7f02e0139b90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 --2- 172.21.15.32:0/769401143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02e0134610 0x7f02e0132630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 >> 172.21.15.32:0/769401143 conn(0x7f02e00a2aa0 msgr2=0x7f02e01253f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 shutdown_connections 2023-09-30T22:24:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:44.820+0000 7f02e8688700 1 -- 172.21.15.32:0/769401143 wait complete. 2023-09-30T22:24:44.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:45.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:45.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:45.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.774+0000 7f7268b12700 1 Processor -- start 2023-09-30T22:24:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.774+0000 7f7268b12700 1 -- start start 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7268b12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409d6f0 0x7f726409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7268b12700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f726409e000 0x7f72641339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7268b12700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726409b120 con 0x7f726409e410 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7268b12700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726406b460 con 0x7f726409e000 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7268b12700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f726409a7e0 con 0x7f726409d6f0 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7261d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f726409e000 0x7f72641339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7262d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f726409e410 0x7f7264133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55716/0 (socket says 172.21.15.32:55716) 2023-09-30T22:24:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f7262d9d700 1 -- 172.21.15.32:0/1480857759 learned_addr learned my addr 172.21.15.32:0/1480857759 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.775+0000 7f726259c700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409d6f0 0x7f726409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f7261d9b700 1 -- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409d6f0 msgr2=0x7f726409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f7261d9b700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409d6f0 0x7f726409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f7261d9b700 1 -- 172.21.15.32:0/1480857759 >> v1:172.21.15.32:6789/0 conn(0x7f726409e410 legacy=0x7f7264133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f7261d9b700 1 -- 172.21.15.32:0/1480857759 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726409a500 con 0x7f726409e000 2023-09-30T22:24:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f726159a700 1 -- 172.21.15.32:0/1480857759 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981806506 0 0) 0x7f726409b120 con 0x7f726409e410 2023-09-30T22:24:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f726259c700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409d6f0 0x7f726409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f7261d9b700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409e000 0x7f72641339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7258009840 tx=0x7f7258009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abfc470fcf54ad0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f726159a700 1 -- 172.21.15.32:0/1480857759 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258011830 con 0x7f726409e000 2023-09-30T22:24:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f726159a700 1 -- 172.21.15.32:0/1480857759 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258011990 con 0x7f726409e000 2023-09-30T22:24:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.776+0000 7f726159a700 1 -- 172.21.15.32:0/1480857759 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258011d20 con 0x7f726409e000 2023-09-30T22:24:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 -- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409e000 msgr2=0x7f72641339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409e000 0x7f72641339f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7258009840 tx=0x7f7258009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 -- 172.21.15.32:0/1480857759 shutdown_connections 2023-09-30T22:24:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409d6f0 0x7f726409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 --2- 172.21.15.32:0/1480857759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409e000 0x7f72641339f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 -- 172.21.15.32:0/1480857759 >> 172.21.15.32:0/1480857759 conn(0x7f7264066f50 msgr2=0x7f726409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 -- 172.21.15.32:0/1480857759 shutdown_connections 2023-09-30T22:24:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.777+0000 7f7268b12700 1 -- 172.21.15.32:0/1480857759 wait complete. 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.779+0000 7f7268b12700 1 Processor -- start 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.779+0000 7f7268b12700 1 -- start start 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7268b12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 0x7f72641a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7268b12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409e000 0x7f72641a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7268b12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f726409e410 0x7f72641a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7268b12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641a9150 con 0x7f726409d6f0 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7268b12700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726409c710 con 0x7f726409e410 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7268b12700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72640a2e40 con 0x7f726409e000 2023-09-30T22:24:45.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f726259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 0x7f72641a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7262d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f726409e410 0x7f72641a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f726259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 0x7f72641a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52304/0 (socket says 172.21.15.32:52304) 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f726259c700 1 -- 172.21.15.32:0/1675316490 learned_addr learned my addr 172.21.15.32:0/1675316490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7261d9b700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409e000 0x7f72641a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7262d9d700 1 -- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409e000 msgr2=0x7f72641a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7262d9d700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409e000 0x7f72641a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7262d9d700 1 -- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 msgr2=0x7f72641a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7262d9d700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 0x7f72641a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f7262d9d700 1 -- 172.21.15.32:0/1675316490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540094a0 con 0x7f726409e410 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.780+0000 7f726259c700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 0x7f72641a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.781+0000 7f7262d9d700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f726409e410 0x7f72641a6920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f725c00cbe0 tx=0x7f725c00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.781+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00aaf0 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.781+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00ac70 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.781+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0070d0 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.781+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72580094a0 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.781+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72641a8d00 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.782+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f725c007230 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.782+0000 7f726259c700 1 -- 172.21.15.32:0/1675316490 >> v1:172.21.15.32:6816/126553 conn(0x7f724402f510 legacy=0x7f72440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.782+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f725c03f5b0 con 0x7f726409e410 2023-09-30T22:24:45.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.782+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72641a9380 con 0x7f726409e410 2023-09-30T22:24:45.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.788+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72641a9380 con 0x7f726409e410 2023-09-30T22:24:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.816+0000 7f2fb6190700 1 Processor -- start 2023-09-30T22:24:45.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.816+0000 7f2fb6190700 1 -- start start 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2fb6190700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0131810 0x7f2fb0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2fb6190700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 0x7f2fb0099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2fb6190700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb01313c0 con 0x7f2fb009a3b0 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2fb6190700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb009b580 con 0x7f2fb0131810 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2fb6190700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb006e8c0 con 0x7f2fb0099a40 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 0x7f2fb0099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0131810 0x7f2fb0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 0x7f2fb0099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40136/0 (socket says 172.21.15.32:40136) 2023-09-30T22:24:45.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faeffd700 1 -- 172.21.15.32:0/757470496 learned_addr learned my addr 172.21.15.32:0/757470496 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faeffd700 1 -- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0131810 msgr2=0x7f2fb0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faeffd700 1 --2- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0131810 0x7f2fb0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2faeffd700 1 -- 172.21.15.32:0/757470496 >> v1:172.21.15.32:6789/0 conn(0x7f2fb009a3b0 legacy=0x7f2fb0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.818+0000 7f2faeffd700 1 -- 172.21.15.32:0/757470496 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb009a820 con 0x7f2fb0099a40 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.817+0000 7f2fae7fc700 1 -- 172.21.15.32:0/757470496 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728581931 0 0) 0x7f2fb01313c0 con 0x7f2fb009a3b0 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.818+0000 7f2faf7fe700 1 --2- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0131810 0x7f2fb0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.818+0000 7f2faeffd700 1 --2- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 0x7f2fb0099e70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009920 tx=0x7f2fa4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b93c9fcc914ea1cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.818+0000 7f2fae7fc700 1 -- 172.21.15.32:0/757470496 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011830 con 0x7f2fb0099a40 2023-09-30T22:24:45.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.818+0000 7f2fae7fc700 1 -- 172.21.15.32:0/757470496 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4011990 con 0x7f2fb0099a40 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.818+0000 7f2fae7fc700 1 -- 172.21.15.32:0/757470496 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011d20 con 0x7f2fb0099a40 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 -- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 msgr2=0x7f2fb0099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 --2- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 0x7f2fb0099e70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009920 tx=0x7f2fa4009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 -- 172.21.15.32:0/757470496 shutdown_connections 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 --2- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0131810 0x7f2fb0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 --2- 172.21.15.32:0/757470496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0099a40 0x7f2fb0099e70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 -- 172.21.15.32:0/757470496 >> 172.21.15.32:0/757470496 conn(0x7f2fb00a2aa0 msgr2=0x7f2fb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 -- 172.21.15.32:0/757470496 shutdown_connections 2023-09-30T22:24:45.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.819+0000 7f2fb6190700 1 -- 172.21.15.32:0/757470496 wait complete. 2023-09-30T22:24:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.821+0000 7f2fb6190700 1 Processor -- start 2023-09-30T22:24:45.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.821+0000 7f2fb6190700 1 -- start start 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2fb6190700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0099a40 0x7f2fb0119a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2fb6190700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 0x7f2fb0119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2fb6190700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0131810 0x7f2fb011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2fb6190700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb011d5a0 con 0x7f2fb009a3b0 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2fb6190700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb011b690 con 0x7f2fb0099a40 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2fb6190700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb011bc40 con 0x7f2fb0131810 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 0x7f2fb0119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0099a40 0x7f2fb0119a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 0x7f2fb0119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52320/0 (socket says 172.21.15.32:52320) 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 -- 172.21.15.32:0/3446361470 learned_addr learned my addr 172.21.15.32:0/3446361470 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:45.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faffff700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0131810 0x7f2fb011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 -- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0131810 msgr2=0x7f2fb011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0131810 0x7f2fb011a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 -- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0099a40 msgr2=0x7f2fb0119a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0099a40 0x7f2fb0119a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 -- 172.21.15.32:0/3446361470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00094a0 con 0x7f2fb009a3b0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faf7fe700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0099a40 0x7f2fb0119a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.822+0000 7f2faeffd700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 0x7f2fb0119fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4000c00 tx=0x7f2fa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.823+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4004450 con 0x7f2fb009a3b0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.823+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa401f070 con 0x7f2fb009a3b0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.823+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa4009550 con 0x7f2fb009a3b0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.823+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40038c0 con 0x7f2fb009a3b0 2023-09-30T22:24:45.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.824+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2fa4003a20 con 0x7f2fb009a3b0 2023-09-30T22:24:45.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.824+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb011d000 con 0x7f2fb009a3b0 2023-09-30T22:24:45.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.824+0000 7f2faf7fe700 1 -- 172.21.15.32:0/3446361470 >> v1:172.21.15.32:6816/126553 conn(0x7f2f9c02f510 legacy=0x7f2f9c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:45.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.824+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f2fa4021070 con 0x7f2fb009a3b0 2023-09-30T22:24:45.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.825+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb006d5a0 con 0x7f2fb009a3b0 2023-09-30T22:24:45.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.829+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2fa4015070 con 0x7f2fb009a3b0 2023-09-30T22:24:45.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.909+0000 7f19801df700 1 Processor -- start 2023-09-30T22:24:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.909+0000 7f19801df700 1 -- start start 2023-09-30T22:24:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f19801df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197812cf60 0x7f197806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f19801df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 0x7f197812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f19801df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197812cd90 con 0x7f1978130260 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f19801df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197812d330 con 0x7f197812f8b0 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f19801df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19781306e0 con 0x7f197812cf60 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197d77a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 0x7f197812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197df7b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197812cf60 0x7f197806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197d77a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 0x7f197812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52322/0 (socket says 172.21.15.32:52322) 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197d77a700 1 -- 172.21.15.32:0/323739232 learned_addr learned my addr 172.21.15.32:0/323739232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197e77c700 1 --1- 172.21.15.32:0/323739232 >> v1:172.21.15.32:6789/0 conn(0x7f1978130260 0x7f1978136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55734/0 (socket says 172.21.15.32:55734) 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197df7b700 1 -- 172.21.15.32:0/323739232 >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 msgr2=0x7f197812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197df7b700 1 --2- 172.21.15.32:0/323739232 >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 0x7f197812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197df7b700 1 -- 172.21.15.32:0/323739232 >> v1:172.21.15.32:6789/0 conn(0x7f1978130260 legacy=0x7f1978136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197df7b700 1 -- 172.21.15.32:0/323739232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978071780 con 0x7f197812cf60 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.910+0000 7f197d77a700 1 --2- 172.21.15.32:0/323739232 >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 0x7f197812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.911+0000 7f197df7b700 1 --2- 172.21.15.32:0/323739232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197812cf60 0x7f197806f5d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1974009840 tx=0x7f1974009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260f73d268bcdcb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.911+0000 7f197cf79700 1 -- 172.21.15.32:0/323739232 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974011830 con 0x7f197812cf60 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.911+0000 7f197cf79700 1 -- 172.21.15.32:0/323739232 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974011990 con 0x7f197812cf60 2023-09-30T22:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.911+0000 7f197cf79700 1 -- 172.21.15.32:0/323739232 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974011d20 con 0x7f197812cf60 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 -- 172.21.15.32:0/323739232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197812cf60 msgr2=0x7f197806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 --2- 172.21.15.32:0/323739232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197812cf60 0x7f197806f5d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1974009840 tx=0x7f1974009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 -- 172.21.15.32:0/323739232 shutdown_connections 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 --2- 172.21.15.32:0/323739232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197812cf60 0x7f197806f5d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 --2- 172.21.15.32:0/323739232 >> v2:172.21.15.32:3300/0 conn(0x7f197812f8b0 0x7f197812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 -- 172.21.15.32:0/323739232 >> 172.21.15.32:0/323739232 conn(0x7f19780a2aa0 msgr2=0x7f19780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 -- 172.21.15.32:0/323739232 shutdown_connections 2023-09-30T22:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.912+0000 7f19801df700 1 -- 172.21.15.32:0/323739232 wait complete. 2023-09-30T22:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 Processor -- start 2023-09-30T22:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 -- start start 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 0x7f19781309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197812f8b0 0x7f1978130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1978130260 0x7f1978131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197809e9b0 con 0x7f197812f8b0 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1978132d90 con 0x7f1978130260 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.914+0000 7f19801df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1978133c10 con 0x7f197812cf60 2023-09-30T22:24:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 0x7f19781309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 0x7f19781309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40164/0 (socket says 172.21.15.32:40164) 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 -- 172.21.15.32:0/4084695824 learned_addr learned my addr 172.21.15.32:0/4084695824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197d77a700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197812f8b0 0x7f1978130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197e77c700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1978130260 0x7f1978131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 -- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1978130260 msgr2=0x7f1978131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1978130260 0x7f1978131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 -- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197812f8b0 msgr2=0x7f1978130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197812f8b0 0x7f1978130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 -- 172.21.15.32:0/4084695824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009550 con 0x7f197812cf60 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197e77c700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1978130260 0x7f1978131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.915+0000 7f197df7b700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 0x7f19781309e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1974009fd0 tx=0x7f1974004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.916+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197401e830 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.916+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974022070 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.916+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19740094a0 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.916+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197401ebf0 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.916+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1978131bf0 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.917+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1978074ea0 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.918+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f197401ed50 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.918+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1974024070 con 0x7f197812cf60 2023-09-30T22:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.918+0000 7f197d77a700 1 -- 172.21.15.32:0/4084695824 >> v1:172.21.15.32:6816/126553 conn(0x7f195c02f540 legacy=0x7f195c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.926+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1974006bd0 con 0x7f197812cf60 2023-09-30T22:24:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.983+0000 7f726259c700 1 -- 172.21.15.32:0/1675316490 >> v1:172.21.15.32:6816/126553 conn(0x7f724402f510 legacy=0x7f72440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:45.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.990+0000 7f210bc71700 1 Processor -- start 2023-09-30T22:24:45.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.990+0000 7f210bc71700 1 -- start start 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210bc71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104133420 0x7f21041337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210bc71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210bc71700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210406b990 con 0x7f2104134260 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210bc71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21040a0590 con 0x7f2104133e30 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210bc71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210409fb80 con 0x7f2104133420 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210a20e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2104134260 0x7f21041304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55736/0 (socket says 172.21.15.32:55736) 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210a20e700 1 -- 172.21.15.32:0/2250138950 learned_addr learned my addr 172.21.15.32:0/2250138950 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f2109a0d700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104133420 0x7f21041337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.991+0000 7f210920c700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2109a0d700 1 -- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 msgr2=0x7f210412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2109a0d700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2109a0d700 1 -- 172.21.15.32:0/2250138950 >> v1:172.21.15.32:6789/0 conn(0x7f2104134260 legacy=0x7f21041304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2108a0b700 1 -- 172.21.15.32:0/2250138950 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052514600 0 0) 0x7f210406b990 con 0x7f2104134260 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2109a0d700 1 -- 172.21.15.32:0/2250138950 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21040a0030 con 0x7f2104133420 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f210920c700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2109a0d700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104133420 0x7f21041337f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2100009870 tx=0x7f2100009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908f6493a9ececf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2108a0b700 1 -- 172.21.15.32:0/2250138950 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100011830 con 0x7f2104133420 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2108a0b700 1 -- 172.21.15.32:0/2250138950 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2100011990 con 0x7f2104133420 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.992+0000 7f2108a0b700 1 -- 172.21.15.32:0/2250138950 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100011d20 con 0x7f2104133420 2023-09-30T22:24:45.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 -- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104133420 msgr2=0x7f21041337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104133420 0x7f21041337f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2100009870 tx=0x7f2100009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 -- 172.21.15.32:0/2250138950 shutdown_connections 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104133420 0x7f21041337f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 --2- 172.21.15.32:0/2250138950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 -- 172.21.15.32:0/2250138950 >> 172.21.15.32:0/2250138950 conn(0x7f21040a4210 msgr2=0x7f21040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 -- 172.21.15.32:0/2250138950 shutdown_connections 2023-09-30T22:24:45.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.993+0000 7f210bc71700 1 -- 172.21.15.32:0/2250138950 wait complete. 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.994+0000 7f210bc71700 1 Processor -- start 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.994+0000 7f210bc71700 1 -- start start 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210bc71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2104133420 0x7f210409b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210bc71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210409bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210bc71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104134260 0x7f210409c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210bc71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210409cc40 con 0x7f2104133420 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210bc71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210412dbf0 con 0x7f2104133e30 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210bc71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210409d2a0 con 0x7f2104134260 2023-09-30T22:24:45.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210409bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210409bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51666/0 (socket says 172.21.15.32:51666) 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 -- 172.21.15.32:0/1259966154 learned_addr learned my addr 172.21.15.32:0/1259966154 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f2109a0d700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2104133420 0x7f210409b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210a20e700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104134260 0x7f210409c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 -- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104134260 msgr2=0x7f210409c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104134260 0x7f210409c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 -- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2104133420 msgr2=0x7f210409b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2104133420 0x7f210409b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.995+0000 7f210920c700 1 -- 172.21.15.32:0/1259966154 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40094a0 con 0x7f2104133e30 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f210920c700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210409bdb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20f400ebb0 tx=0x7f20f400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f210a20e700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104134260 0x7f210409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:24:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400c850 con 0x7f2104133e30 2023-09-30T22:24:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f400c9b0 con 0x7f2104133e30 2023-09-30T22:24:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21000094a0 con 0x7f2104133e30 2023-09-30T22:24:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400cd20 con 0x7f2104133e30 2023-09-30T22:24:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.996+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210411ab40 con 0x7f2104133e30 2023-09-30T22:24:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.997+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f20f4013470 con 0x7f2104133e30 2023-09-30T22:24:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.998+0000 7f2109a0d700 1 -- 172.21.15.32:0/1259966154 >> v1:172.21.15.32:6816/126553 conn(0x7f20e802f560 legacy=0x7f20e8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.998+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f20f4040150 con 0x7f2104133e30 2023-09-30T22:24:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:45.998+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210411ad60 con 0x7f2104133e30 2023-09-30T22:24:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.008+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20f4021080 con 0x7f2104133e30 2023-09-30T22:24:46.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.025+0000 7f2faf7fe700 1 -- 172.21.15.32:0/3446361470 >> v1:172.21.15.32:6816/126553 conn(0x7f2f9c02f510 legacy=0x7f2f9c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:46.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.039+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f72641a9380 con 0x7f726409e410 2023-09-30T22:24:46.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.040+0000 7f7252ffd700 1 -- 172.21.15.32:0/1675316490 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72641a9380 con 0x7f726409e410 2023-09-30T22:24:46.041 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:46.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 >> v1:172.21.15.32:6816/126553 conn(0x7f724402f510 legacy=0x7f72440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:46.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f726409e410 msgr2=0x7f72641a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:46.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f726409e410 0x7f72641a6920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f725c00cbe0 tx=0x7f725c00cef0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 shutdown_connections 2023-09-30T22:24:46.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f726409e410 0x7f72641a6920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726409d6f0 0x7f72641a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 --2- 172.21.15.32:0/1675316490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f726409e000 0x7f72641a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.043+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 >> 172.21.15.32:0/1675316490 conn(0x7f7264066f50 msgr2=0x7f72641bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:46.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.044+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 shutdown_connections 2023-09-30T22:24:46.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.045+0000 7f7268b12700 1 -- 172.21.15.32:0/1675316490 wait complete. 2023-09-30T22:24:46.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.053+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2fb006ece0 con 0x7f2fb009a3b0 2023-09-30T22:24:46.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.053+0000 7f2f97fff700 1 -- 172.21.15.32:0/3446361470 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fa402b070 con 0x7f2fb009a3b0 2023-09-30T22:24:46.054 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:46.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.057+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 >> v1:172.21.15.32:6816/126553 conn(0x7f2f9c02f510 legacy=0x7f2f9c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.057+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 msgr2=0x7f2fb0119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.057+0000 7f2fb6190700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 0x7f2fb0119fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4000c00 tx=0x7f2fa4004750 comp rx=0 tx=0).stop 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 shutdown_connections 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fb0131810 0x7f2fb011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fb0099a40 0x7f2fb0119a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 --2- 172.21.15.32:0/3446361470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fb009a3b0 0x7f2fb0119fc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 >> 172.21.15.32:0/3446361470 conn(0x7f2fb00a2aa0 msgr2=0x7f2fb012e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 shutdown_connections 2023-09-30T22:24:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.058+0000 7f2fb6190700 1 -- 172.21.15.32:0/3446361470 wait complete. 2023-09-30T22:24:46.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:46.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.119+0000 7f197d77a700 1 -- 172.21.15.32:0/4084695824 >> v1:172.21.15.32:6816/126553 conn(0x7f195c02f540 legacy=0x7f195c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.141+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f19780713a0 con 0x7f197812cf60 2023-09-30T22:24:46.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.141+0000 7f196e7fc700 1 -- 172.21.15.32:0/4084695824 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f197402e070 con 0x7f197812cf60 2023-09-30T22:24:46.142 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 >> v1:172.21.15.32:6816/126553 conn(0x7f195c02f540 legacy=0x7f195c0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 msgr2=0x7f19781309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 0x7f19781309e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1974009fd0 tx=0x7f1974004750 comp rx=0 tx=0).stop 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 shutdown_connections 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1978130260 0x7f1978131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197812cf60 0x7f19781309e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 --2- 172.21.15.32:0/4084695824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197812f8b0 0x7f1978130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 >> 172.21.15.32:0/4084695824 conn(0x7f19780a2aa0 msgr2=0x7f197806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 shutdown_connections 2023-09-30T22:24:46.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.144+0000 7f19801df700 1 -- 172.21.15.32:0/4084695824 wait complete. 2023-09-30T22:24:46.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.199+0000 7f2109a0d700 1 -- 172.21.15.32:0/1259966154 >> v1:172.21.15.32:6816/126553 conn(0x7f20e802f560 legacy=0x7f20e8031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:46.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.234+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f210411b090 con 0x7f2104133e30 2023-09-30T22:24:46.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.235+0000 7f20fa7fc700 1 -- 172.21.15.32:0/1259966154 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f210411b090 con 0x7f2104133e30 2023-09-30T22:24:46.236 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 >> v1:172.21.15.32:6816/126553 conn(0x7f20e802f560 legacy=0x7f20e8031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 msgr2=0x7f210409bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210409bdb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20f400ebb0 tx=0x7f20f400eec0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 shutdown_connections 2023-09-30T22:24:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2104134260 0x7f210409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2104133420 0x7f210409b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 --2- 172.21.15.32:0/1259966154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2104133e30 0x7f210409bdb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:46.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 >> 172.21.15.32:0/1259966154 conn(0x7f21040a4210 msgr2=0x7f210412de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:46.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 shutdown_connections 2023-09-30T22:24:46.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:46.237+0000 7f210bc71700 1 -- 172.21.15.32:0/1259966154 wait complete. 2023-09-30T22:24:46.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:47.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:47.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:47.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:47.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.217+0000 7fed6591a700 1 Processor -- start 2023-09-30T22:24:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.217+0000 7fed6591a700 1 -- start start 2023-09-30T22:24:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed6591a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 0x7fed6009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed6591a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 0x7fed601339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed6591a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6009b120 con 0x7fed6009e410 2023-09-30T22:24:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed6591a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6006b460 con 0x7fed6009e000 2023-09-30T22:24:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed6591a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed6009a7e0 con 0x7fed6009d6f0 2023-09-30T22:24:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 0x7fed6009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 0x7fed601339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 0x7fed6009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40204/0 (socket says 172.21.15.32:40204) 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 0x7fed601339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51668/0 (socket says 172.21.15.32:51668) 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5effd700 1 -- 172.21.15.32:0/3824275140 learned_addr learned my addr 172.21.15.32:0/3824275140 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 -- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 msgr2=0x7fed6009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 --2- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 0x7fed6009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 -- 172.21.15.32:0/3824275140 >> v1:172.21.15.32:6789/0 conn(0x7fed6009e410 legacy=0x7fed60133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 -- 172.21.15.32:0/3824275140 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6009a500 con 0x7fed6009e000 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5dffb700 1 -- 172.21.15.32:0/3824275140 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253276622 0 0) 0x7fed6009b120 con 0x7fed6009e410 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5effd700 1 --2- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 0x7fed6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.218+0000 7fed5e7fc700 1 --2- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 0x7fed601339f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed54009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cdcb6d86968141 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.219+0000 7fed5dffb700 1 -- 172.21.15.32:0/3824275140 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54011830 con 0x7fed6009e000 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.219+0000 7fed5dffb700 1 -- 172.21.15.32:0/3824275140 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54011990 con 0x7fed6009e000 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.219+0000 7fed5dffb700 1 -- 172.21.15.32:0/3824275140 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54011d20 con 0x7fed6009e000 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 -- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 msgr2=0x7fed601339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 --2- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 0x7fed601339f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed54009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 -- 172.21.15.32:0/3824275140 shutdown_connections 2023-09-30T22:24:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 --2- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009d6f0 0x7fed6009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 --2- 172.21.15.32:0/3824275140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e000 0x7fed601339f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 -- 172.21.15.32:0/3824275140 >> 172.21.15.32:0/3824275140 conn(0x7fed60066f50 msgr2=0x7fed6009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 -- 172.21.15.32:0/3824275140 shutdown_connections 2023-09-30T22:24:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.220+0000 7fed6591a700 1 -- 172.21.15.32:0/3824275140 wait complete. 2023-09-30T22:24:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.221+0000 7fed6591a700 1 Processor -- start 2023-09-30T22:24:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.221+0000 7fed6591a700 1 -- start start 2023-09-30T22:24:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed6591a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 0x7fed601a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed6591a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009e000 0x7fed601a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed6591a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e410 0x7fed601a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed6591a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed601a7390 con 0x7fed6009d6f0 2023-09-30T22:24:47.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed6591a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed601a8120 con 0x7fed6009e410 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed6591a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed601a91e0 con 0x7fed6009e000 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 0x7fed601a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009e000 0x7fed601a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 0x7fed601a5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52342/0 (socket says 172.21.15.32:52342) 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 -- 172.21.15.32:0/118841398 learned_addr learned my addr 172.21.15.32:0/118841398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5f7fe700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e410 0x7fed601a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 -- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009e000 msgr2=0x7fed601a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009e000 0x7fed601a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 -- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e410 msgr2=0x7fed601a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e410 0x7fed601a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 -- 172.21.15.32:0/118841398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50009550 con 0x7fed6009d6f0 2023-09-30T22:24:47.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.222+0000 7fed5effd700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 0x7fed601a5f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed5000ec10 tx=0x7fed5000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.223+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000c850 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.223+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5000c9b0 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.223+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000cd40 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.223+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed540094a0 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.223+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed600a2eb0 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.224+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed601a7ba0 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.224+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fed50013470 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.224+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fed500400f0 con 0x7fed6009d6f0 2023-09-30T22:24:47.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.224+0000 7fed5e7fc700 1 -- 172.21.15.32:0/118841398 >> v1:172.21.15.32:6816/126553 conn(0x7fed4c02f510 legacy=0x7fed4c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.229+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed50013720 con 0x7fed6009d6f0 2023-09-30T22:24:47.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.257+0000 7f4bd0975700 1 Processor -- start 2023-09-30T22:24:47.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.257+0000 7f4bd0975700 1 -- start start 2023-09-30T22:24:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.258+0000 7f4bd0975700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bcc06f5d0 0x7f4bcc12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.258+0000 7f4bd0975700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc135c20 0x7f4bcc138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bd0975700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc06fa40 con 0x7f4bcc12cf60 2023-09-30T22:24:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bd0975700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc12cd90 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bd0975700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc071780 con 0x7f4bcc135c20 2023-09-30T22:24:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bca59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4bcc12cf60 0x7f4bcc12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55756/0 (socket says 172.21.15.32:55756) 2023-09-30T22:24:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bca59c700 1 -- 172.21.15.32:0/2764889870 learned_addr learned my addr 172.21.15.32:0/2764889870 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc9d9b700 1 --2- 172.21.15.32:0/2764889870 >> v2:172.21.15.32:3300/0 conn(0x7f4bcc06f5d0 0x7f4bcc12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bcad9d700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc135c20 0x7f4bcc138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc959a700 1 -- 172.21.15.32:0/2764889870 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878394316 0 0) 0x7f4bcc06fa40 con 0x7f4bcc12cf60 2023-09-30T22:24:47.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc959a700 1 -- 172.21.15.32:0/2764889870 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb400bdc0 con 0x7f4bcc12cf60 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc9d9b700 1 -- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc135c20 msgr2=0x7f4bcc138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc9d9b700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc135c20 0x7f4bcc138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc959a700 1 -- 172.21.15.32:0/2764889870 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3379416416 0 0) 0x7f4bb400bdc0 con 0x7f4bcc12cf60 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc9d9b700 1 -- 172.21.15.32:0/2764889870 >> v1:172.21.15.32:6789/0 conn(0x7f4bcc12cf60 legacy=0x7f4bcc12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bc9d9b700 1 -- 172.21.15.32:0/2764889870 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc071a40 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.259+0000 7f4bcad9d700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc135c20 0x7f4bcc138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.260+0000 7f4bc9d9b700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc06f5d0 0x7f4bcc12ffc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000dc50 tx=0x7f4bc000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbba576c9ed8dc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.260+0000 7f4bc959a700 1 -- 172.21.15.32:0/2764889870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000c070 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.260+0000 7f4bc959a700 1 -- 172.21.15.32:0/2764889870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc0004030 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.260+0000 7f4bc959a700 1 -- 172.21.15.32:0/2764889870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc001670 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 -- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc06f5d0 msgr2=0x7f4bcc12ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc06f5d0 0x7f4bcc12ffc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000dc50 tx=0x7f4bc000df60 comp rx=0 tx=0).stop 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 -- 172.21.15.32:0/2764889870 shutdown_connections 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc135c20 0x7f4bcc138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 --2- 172.21.15.32:0/2764889870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc06f5d0 0x7f4bcc12ffc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 -- 172.21.15.32:0/2764889870 >> 172.21.15.32:0/2764889870 conn(0x7f4bcc0a2aa0 msgr2=0x7f4bcc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 -- 172.21.15.32:0/2764889870 shutdown_connections 2023-09-30T22:24:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.261+0000 7f4bd0975700 1 -- 172.21.15.32:0/2764889870 wait complete. 2023-09-30T22:24:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.263+0000 7f4bd0975700 1 Processor -- start 2023-09-30T22:24:47.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.263+0000 7f4bd0975700 1 -- start start 2023-09-30T22:24:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bd0975700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc06f5d0 0x7f4bcc12e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bd0975700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bcc12cf60 0x7f4bcc12ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bd0975700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 0x7f4bcc1c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bd0975700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc12f6c0 con 0x7f4bcc135c20 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bd0975700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc130e20 con 0x7f4bcc12cf60 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bd0975700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc12efb0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bcad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 0x7f4bcc1c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bcad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 0x7f4bcc1c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52354/0 (socket says 172.21.15.32:52354) 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bcad9d700 1 -- 172.21.15.32:0/3219250460 learned_addr learned my addr 172.21.15.32:0/3219250460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bca59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc06f5d0 0x7f4bcc12e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.264+0000 7f4bc9d9b700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bcc12cf60 0x7f4bcc12ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bca59c700 1 -- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bcc12cf60 msgr2=0x7f4bcc12ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bca59c700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bcc12cf60 0x7f4bcc12ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bca59c700 1 -- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 msgr2=0x7f4bcc1c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bca59c700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 0x7f4bcc1c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bca59c700 1 -- 172.21.15.32:0/3219250460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc000d930 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bcad9d700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 0x7f4bcc1c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bc9d9b700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bcc12cf60 0x7f4bcc12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bca59c700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc06f5d0 0x7f4bcc12e530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c520 tx=0x7f4bbc00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004c80 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc004de0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.265+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc129fa0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.266+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc007310 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.266+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc0a14f0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.266+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4bbc007470 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.267+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4bbc03f900 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.267+0000 7f4bc9d9b700 1 -- 172.21.15.32:0/3219250460 >> v1:172.21.15.32:6816/126553 conn(0x7f4bac02f510 legacy=0x7f4bac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.267+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc074ea0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.276+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bbc0123b0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.309+0000 7f4ba979c700 1 Processor -- start 2023-09-30T22:24:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.309+0000 7f4ba979c700 1 -- start start 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba979c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ba409d6f0 0x7f4ba409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba979c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e410 0x7f4ba4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba979c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba409f130 con 0x7f4ba409e000 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba979c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba409f520 con 0x7f4ba409d6f0 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba979c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba409e930 con 0x7f4ba409e410 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ba409d6f0 0x7f4ba409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ba409d6f0 0x7f4ba409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52356/0 (socket says 172.21.15.32:52356) 2023-09-30T22:24:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba2ffd700 1 -- 172.21.15.32:0/3440818269 learned_addr learned my addr 172.21.15.32:0/3440818269 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba27fc700 1 --1- 172.21.15.32:0/3440818269 >> v1:172.21.15.32:6789/0 conn(0x7f4ba409e000 0x7f4ba4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55768/0 (socket says 172.21.15.32:55768) 2023-09-30T22:24:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.310+0000 7f4ba37fe700 1 --2- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e410 0x7f4ba4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba2ffd700 1 -- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e410 msgr2=0x7f4ba4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba2ffd700 1 --2- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e410 0x7f4ba4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba2ffd700 1 -- 172.21.15.32:0/3440818269 >> v1:172.21.15.32:6789/0 conn(0x7f4ba409e000 legacy=0x7f4ba4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba2ffd700 1 -- 172.21.15.32:0/3440818269 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba409b2d0 con 0x7f4ba409d6f0 2023-09-30T22:24:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba2ffd700 1 --2- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba409dac0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b940098f0 tx=0x7f4b94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381c596939f391da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba1ffb700 1 -- 172.21.15.32:0/3440818269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94011830 con 0x7f4ba409d6f0 2023-09-30T22:24:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba1ffb700 1 -- 172.21.15.32:0/3440818269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94011990 con 0x7f4ba409d6f0 2023-09-30T22:24:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.311+0000 7f4ba1ffb700 1 -- 172.21.15.32:0/3440818269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94011d20 con 0x7f4ba409d6f0 2023-09-30T22:24:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 -- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 msgr2=0x7f4ba409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba409dac0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b940098f0 tx=0x7f4b94009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 -- 172.21.15.32:0/3440818269 shutdown_connections 2023-09-30T22:24:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e410 0x7f4ba4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3440818269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba409dac0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 -- 172.21.15.32:0/3440818269 >> 172.21.15.32:0/3440818269 conn(0x7f4ba4066f50 msgr2=0x7f4ba409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 -- 172.21.15.32:0/3440818269 shutdown_connections 2023-09-30T22:24:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.312+0000 7f4ba979c700 1 -- 172.21.15.32:0/3440818269 wait complete. 2023-09-30T22:24:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.313+0000 7f4ba979c700 1 Processor -- start 2023-09-30T22:24:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.313+0000 7f4ba979c700 1 -- start start 2023-09-30T22:24:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba979c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba411df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba979c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 0x7f4ba411e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba979c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ba409e410 0x7f4ba411e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba979c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4121790 con 0x7f4ba409d6f0 2023-09-30T22:24:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba979c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4130290 con 0x7f4ba409e000 2023-09-30T22:24:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba979c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4120d10 con 0x7f4ba409e410 2023-09-30T22:24:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 0x7f4ba411e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ba409e410 0x7f4ba411e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 0x7f4ba411e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51694/0 (socket says 172.21.15.32:51694) 2023-09-30T22:24:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 -- 172.21.15.32:0/3688214984 learned_addr learned my addr 172.21.15.32:0/3688214984 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba2ffd700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba411df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 -- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ba409e410 msgr2=0x7f4ba411e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ba409e410 0x7f4ba411e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 -- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 msgr2=0x7f4ba411df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba411df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba27fc700 1 -- 172.21.15.32:0/3688214984 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94009550 con 0x7f4ba409e000 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.314+0000 7f4ba2ffd700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba411df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.315+0000 7f4ba27fc700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 0x7f4ba411e490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e730 tx=0x7f4b9800ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.315+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980090f0 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.315+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b98009250 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.315+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba4121160 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.315+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980095c0 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.315+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba40725c0 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.316+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4b98009720 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.316+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba4074ea0 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.316+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f4b980411b0 con 0x7f4ba409e000 2023-09-30T22:24:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.316+0000 7f4ba2ffd700 1 -- 172.21.15.32:0/3688214984 >> v1:172.21.15.32:6816/126553 conn(0x7f4b9002f510 legacy=0x7f4b900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.325+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b98019030 con 0x7f4ba409e000 2023-09-30T22:24:47.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.412+0000 7faafcb94700 1 Processor -- start 2023-09-30T22:24:47.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.412+0000 7faafcb94700 1 -- start start 2023-09-30T22:24:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faafcb94700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 0x7faaf80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faafcb94700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf809d630 0x7faaf80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faafcb94700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf80a2a70 con 0x7faaf8133930 2023-09-30T22:24:47.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faafcb94700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf80a01d0 con 0x7faaf809d630 2023-09-30T22:24:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faafcb94700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf809e2b0 con 0x7faaf80a0a30 2023-09-30T22:24:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faaf659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 0x7faaf80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.413+0000 7faaf6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faaf8133930 0x7faaf8133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55780/0 (socket says 172.21.15.32:55780) 2023-09-30T22:24:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 0x7faaf80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51706/0 (socket says 172.21.15.32:51706) 2023-09-30T22:24:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf6d9d700 1 -- 172.21.15.32:0/3536655875 learned_addr learned my addr 172.21.15.32:0/3536655875 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:47.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf5d9b700 1 --2- 172.21.15.32:0/3536655875 >> v2:172.21.15.32:3300/0 conn(0x7faaf809d630 0x7faaf80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf659c700 1 -- 172.21.15.32:0/3536655875 >> v2:172.21.15.32:3300/0 conn(0x7faaf809d630 msgr2=0x7faaf80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf659c700 1 --2- 172.21.15.32:0/3536655875 >> v2:172.21.15.32:3300/0 conn(0x7faaf809d630 0x7faaf80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf659c700 1 -- 172.21.15.32:0/3536655875 >> v1:172.21.15.32:6789/0 conn(0x7faaf8133930 legacy=0x7faaf8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf559a700 1 -- 172.21.15.32:0/3536655875 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828191615 0 0) 0x7faaf80a2a70 con 0x7faaf8133930 2023-09-30T22:24:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf659c700 1 -- 172.21.15.32:0/3536655875 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf809dc50 con 0x7faaf80a0a30 2023-09-30T22:24:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf659c700 1 --2- 172.21.15.32:0/3536655875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 0x7faaf80a0e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faae8009840 tx=0x7faae8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3968dec8c251ff4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf559a700 1 -- 172.21.15.32:0/3536655875 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8011830 con 0x7faaf80a0a30 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf559a700 1 -- 172.21.15.32:0/3536655875 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae8011990 con 0x7faaf80a0a30 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.414+0000 7faaf559a700 1 -- 172.21.15.32:0/3536655875 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8011d20 con 0x7faaf80a0a30 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.415+0000 7faafcb94700 1 -- 172.21.15.32:0/3536655875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 msgr2=0x7faaf80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.415+0000 7faafcb94700 1 --2- 172.21.15.32:0/3536655875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 0x7faaf80a0e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faae8009840 tx=0x7faae8009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.416+0000 7faafcb94700 1 -- 172.21.15.32:0/3536655875 shutdown_connections 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.416+0000 7faafcb94700 1 --2- 172.21.15.32:0/3536655875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf80a0a30 0x7faaf80a0e00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.416+0000 7faafcb94700 1 --2- 172.21.15.32:0/3536655875 >> v2:172.21.15.32:3300/0 conn(0x7faaf809d630 0x7faaf80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.416+0000 7faafcb94700 1 -- 172.21.15.32:0/3536655875 >> 172.21.15.32:0/3536655875 conn(0x7faaf80a3050 msgr2=0x7faaf80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.416+0000 7faafcb94700 1 -- 172.21.15.32:0/3536655875 shutdown_connections 2023-09-30T22:24:47.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.416+0000 7faafcb94700 1 -- 172.21.15.32:0/3536655875 wait complete. 2023-09-30T22:24:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.417+0000 7faafcb94700 1 Processor -- start 2023-09-30T22:24:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.417+0000 7faafcb94700 1 -- start start 2023-09-30T22:24:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faafcb94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf809d630 0x7faaf811f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faafcb94700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faaf80a0a30 0x7faaf811f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faafcb94700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf8133930 0x7faaf81c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faafcb94700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf811eeb0 con 0x7faaf809d630 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faafcb94700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80a0410 con 0x7faaf8133930 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faafcb94700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8121180 con 0x7faaf80a0a30 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faaf659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf809d630 0x7faaf811f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faaf5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faaf80a0a30 0x7faaf811f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faaf6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf8133930 0x7faaf81c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faaf659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf809d630 0x7faaf811f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52398/0 (socket says 172.21.15.32:52398) 2023-09-30T22:24:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.418+0000 7faaf659c700 1 -- 172.21.15.32:0/3410745900 learned_addr learned my addr 172.21.15.32:0/3410745900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faaf6d9d700 1 -- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faaf80a0a30 msgr2=0x7faaf811f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faaf6d9d700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faaf80a0a30 0x7faaf811f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faaf6d9d700 1 -- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf809d630 msgr2=0x7faaf811f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faaf6d9d700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf809d630 0x7faaf811f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faaf6d9d700 1 -- 172.21.15.32:0/3410745900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae80094a0 con 0x7faaf8133930 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faaf6d9d700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf8133930 0x7faaf81c7ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faaf000cc10 tx=0x7faaf000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000aa00 con 0x7faaf8133930 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000ab80 con 0x7faaf8133930 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf8121e80 con 0x7faaf8133930 2023-09-30T22:24:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf00070d0 con 0x7faaf8133930 2023-09-30T22:24:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.419+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf8122bb0 con 0x7faaf8133930 2023-09-30T22:24:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.420+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7faaf0007230 con 0x7faaf8133930 2023-09-30T22:24:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.421+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7faaf003f9a0 con 0x7faaf8133930 2023-09-30T22:24:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.421+0000 7faaf659c700 1 -- 172.21.15.32:0/3410745900 >> v1:172.21.15.32:6816/126553 conn(0x7faad802f510 legacy=0x7faad80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.421+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf80747f0 con 0x7faaf8133930 2023-09-30T22:24:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.425+0000 7fed5e7fc700 1 -- 172.21.15.32:0/118841398 >> v1:172.21.15.32:6816/126553 conn(0x7fed4c02f510 legacy=0x7fed4c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.426+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faaf0018020 con 0x7faaf8133930 2023-09-30T22:24:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.450+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fed60074ea0 con 0x7fed6009d6f0 2023-09-30T22:24:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.450+0000 7fed47fff700 1 -- 172.21.15.32:0/118841398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed50011b40 con 0x7fed6009d6f0 2023-09-30T22:24:47.451 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 >> v1:172.21.15.32:6816/126553 conn(0x7fed4c02f510 legacy=0x7fed4c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 msgr2=0x7fed601a5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 0x7fed601a5f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed5000ec10 tx=0x7fed5000ef20 comp rx=0 tx=0).stop 2023-09-30T22:24:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 shutdown_connections 2023-09-30T22:24:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6009e410 0x7fed601a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6009d6f0 0x7fed601a5f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 --2- 172.21.15.32:0/118841398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6009e000 0x7fed601a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 >> 172.21.15.32:0/118841398 conn(0x7fed60066f50 msgr2=0x7fed601bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 shutdown_connections 2023-09-30T22:24:47.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.453+0000 7fed6591a700 1 -- 172.21.15.32:0/118841398 wait complete. 2023-09-30T22:24:47.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:47.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.468+0000 7f4bc9d9b700 1 -- 172.21.15.32:0/3219250460 >> v1:172.21.15.32:6816/126553 conn(0x7f4bac02f510 legacy=0x7f4bac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.495+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4bcc0722f0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.495+0000 7f4bbaffd700 1 -- 172.21.15.32:0/3219250460 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bcc0722f0 con 0x7f4bcc06f5d0 2023-09-30T22:24:47.496 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 >> v1:172.21.15.32:6816/126553 conn(0x7f4bac02f510 legacy=0x7f4bac0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc06f5d0 msgr2=0x7f4bcc12e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc06f5d0 0x7f4bcc12e530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c520 tx=0x7f4bbc00c830 comp rx=0 tx=0).stop 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 shutdown_connections 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bcc135c20 0x7f4bcc1c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bcc06f5d0 0x7f4bcc12e530 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 --2- 172.21.15.32:0/3219250460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bcc12cf60 0x7f4bcc12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 >> 172.21.15.32:0/3219250460 conn(0x7f4bcc0a2aa0 msgr2=0x7f4bcc12a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 shutdown_connections 2023-09-30T22:24:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.497+0000 7f4bd0975700 1 -- 172.21.15.32:0/3219250460 wait complete. 2023-09-30T22:24:47.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.517+0000 7f4ba2ffd700 1 -- 172.21.15.32:0/3688214984 >> v1:172.21.15.32:6816/126553 conn(0x7f4b9002f510 legacy=0x7f4b900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.584+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4ba4121390 con 0x7f4ba409e000 2023-09-30T22:24:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.584+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3688214984 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b98019020 con 0x7f4ba409e000 2023-09-30T22:24:47.585 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 >> v1:172.21.15.32:6816/126553 conn(0x7f4b9002f510 legacy=0x7f4b900319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 msgr2=0x7f4ba411e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 0x7f4ba411e490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800e730 tx=0x7f4b9800ea40 comp rx=0 tx=0).stop 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 shutdown_connections 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ba409e410 0x7f4ba411e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ba409d6f0 0x7f4ba411df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 --2- 172.21.15.32:0/3688214984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ba409e000 0x7f4ba411e490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.586+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 >> 172.21.15.32:0/3688214984 conn(0x7f4ba4066f50 msgr2=0x7f4ba40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.587+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 shutdown_connections 2023-09-30T22:24:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.587+0000 7f4ba979c700 1 -- 172.21.15.32:0/3688214984 wait complete. 2023-09-30T22:24:47.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.622+0000 7faaf659c700 1 -- 172.21.15.32:0/3410745900 >> v1:172.21.15.32:6816/126553 conn(0x7faad802f510 legacy=0x7faad80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:47.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.671+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faaf8072980 con 0x7faaf8133930 2023-09-30T22:24:47.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.671+0000 7faae6ffd700 1 -- 172.21.15.32:0/3410745900 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faaf00074e0 con 0x7faaf8133930 2023-09-30T22:24:47.672 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 >> v1:172.21.15.32:6816/126553 conn(0x7faad802f510 legacy=0x7faad80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf8133930 msgr2=0x7faaf81c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf8133930 0x7faaf81c7ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faaf000cc10 tx=0x7faaf000cf20 comp rx=0 tx=0).stop 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 shutdown_connections 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf8133930 0x7faaf81c7ef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf809d630 0x7faaf811f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 --2- 172.21.15.32:0/3410745900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faaf80a0a30 0x7faaf811f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 >> 172.21.15.32:0/3410745900 conn(0x7faaf80a3050 msgr2=0x7faaf81bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 shutdown_connections 2023-09-30T22:24:47.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:47.674+0000 7faafcb94700 1 -- 172.21.15.32:0/3410745900 wait complete. 2023-09-30T22:24:47.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:48.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:48.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:48.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.620+0000 7f1983283700 1 Processor -- start 2023-09-30T22:24:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.620+0000 7f1983283700 1 -- start start 2023-09-30T22:24:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.621+0000 7f1983283700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c1338a0 0x7f197c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.621+0000 7f1983283700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c09de50 0x7f197c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.621+0000 7f1983283700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197c09e760 0x7f197c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.621+0000 7f1983283700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c06fef0 con 0x7f197c09e760 2023-09-30T22:24:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.621+0000 7f1983283700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c06ef30 con 0x7f197c1338a0 2023-09-30T22:24:48.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.621+0000 7f1983283700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1334b0 con 0x7f197c09de50 2023-09-30T22:24:48.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f198101f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c1338a0 0x7f197c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f197c09e760 0x7f197c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f198101f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c1338a0 0x7f197c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51730/0 (socket says 172.21.15.32:51730) 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f198101f700 1 -- 172.21.15.32:0/1097645124 learned_addr learned my addr 172.21.15.32:0/1097645124 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f198081e700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c09de50 0x7f197c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 -- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c09de50 msgr2=0x7f197c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c09de50 0x7f197c09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 -- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c1338a0 msgr2=0x7f197c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c1338a0 0x7f197c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 -- 172.21.15.32:0/1097645124 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c06fc30 con 0x7f197c09e760 2023-09-30T22:24:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.622+0000 7f1981820700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c13a730 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1974009d90 tx=0x7f197400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca4973f20faace5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.623+0000 7f1973fff700 1 -- 172.21.15.32:0/1097645124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400e070 con 0x7f197c09e760 2023-09-30T22:24:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.623+0000 7f1973fff700 1 -- 172.21.15.32:0/1097645124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400bb20 con 0x7f197c09e760 2023-09-30T22:24:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.623+0000 7f1973fff700 1 -- 172.21.15.32:0/1097645124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974012430 con 0x7f197c09e760 2023-09-30T22:24:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 -- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 msgr2=0x7f197c13a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c13a730 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1974009d90 tx=0x7f197400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 -- 172.21.15.32:0/1097645124 shutdown_connections 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c1338a0 0x7f197c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c13a730 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 --2- 172.21.15.32:0/1097645124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c09de50 0x7f197c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 -- 172.21.15.32:0/1097645124 >> 172.21.15.32:0/1097645124 conn(0x7f197c0a2aa0 msgr2=0x7f197c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 -- 172.21.15.32:0/1097645124 shutdown_connections 2023-09-30T22:24:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.624+0000 7f1983283700 1 -- 172.21.15.32:0/1097645124 wait complete. 2023-09-30T22:24:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 Processor -- start 2023-09-30T22:24:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 -- start start 2023-09-30T22:24:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c09de50 0x7f197c11b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c11bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c1338a0 0x7f197c11c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c11d180 con 0x7f197c09e760 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c11cb60 con 0x7f197c09de50 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.626+0000 7f1983283700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f197c11ebf0 con 0x7f197c1338a0 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c11bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c11bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52422/0 (socket says 172.21.15.32:52422) 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 -- 172.21.15.32:0/2238097297 learned_addr learned my addr 172.21.15.32:0/2238097297 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198101f700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c09de50 0x7f197c11b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f1981820700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c1338a0 0x7f197c11c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 -- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c1338a0 msgr2=0x7f197c11c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c1338a0 0x7f197c11c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 -- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c09de50 msgr2=0x7f197c11b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c09de50 0x7f197c11b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 -- 172.21.15.32:0/2238097297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009a70 con 0x7f197c09e760 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198101f700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c09de50 0x7f197c11b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f198081e700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c11bbf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f196c00e870 tx=0x7f196c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.627+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c009590 con 0x7f197c09e760 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.628+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c013680 con 0x7f197c09e760 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.628+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c11df80 con 0x7f197c09e760 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.628+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c013a10 con 0x7f197c09e760 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.628+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c11f100 con 0x7f197c09e760 2023-09-30T22:24:48.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.629+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f196c013b70 con 0x7f197c09e760 2023-09-30T22:24:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.629+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f196c03fa30 con 0x7f197c09e760 2023-09-30T22:24:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.629+0000 7f198101f700 1 -- 172.21.15.32:0/2238097297 >> v1:172.21.15.32:6816/126553 conn(0x7f196002f510 legacy=0x7f19600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.629+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c11eeb0 con 0x7f197c09e760 2023-09-30T22:24:48.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.638+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f196c017080 con 0x7f197c09e760 2023-09-30T22:24:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.664+0000 7f8c4d006700 1 Processor -- start 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.664+0000 7f8c4d006700 1 -- start start 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.664+0000 7f8c4d006700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.664+0000 7f8c4d006700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c4809e000 0x7f8c481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.664+0000 7f8c4d006700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e410 0x7f8c48133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.664+0000 7f8c4d006700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4806b460 con 0x7f8c4809e000 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c4d006700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4809b120 con 0x7f8c4809e410 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c4d006700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4809a7e0 con 0x7f8c4809d6f0 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c4809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40284/0 (socket says 172.21.15.32:40284) 2023-09-30T22:24:48.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 -- 172.21.15.32:0/3439552070 learned_addr learned my addr 172.21.15.32:0/3439552070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c4759e700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e410 0x7f8c48133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c4659c700 1 --2- 172.21.15.32:0/3439552070 >> v2:172.21.15.32:3300/0 conn(0x7f8c4809e000 0x7f8c481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 -- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e410 msgr2=0x7f8c48133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e410 0x7f8c48133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 -- 172.21.15.32:0/3439552070 >> v2:172.21.15.32:3300/0 conn(0x7f8c4809e000 msgr2=0x7f8c481339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 --2- 172.21.15.32:0/3439552070 >> v2:172.21.15.32:3300/0 conn(0x7f8c4809e000 0x7f8c481339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 -- 172.21.15.32:0/3439552070 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4809a500 con 0x7f8c4809d6f0 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c4759e700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e410 0x7f8c48133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.665+0000 7f8c46d9d700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c4809dac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009920 tx=0x7f8c38009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625075a017748d57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c45d9b700 1 -- 172.21.15.32:0/3439552070 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011830 con 0x7f8c4809d6f0 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c45d9b700 1 -- 172.21.15.32:0/3439552070 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38011990 con 0x7f8c4809d6f0 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c45d9b700 1 -- 172.21.15.32:0/3439552070 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011d20 con 0x7f8c4809d6f0 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 -- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 msgr2=0x7f8c4809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c4809dac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009920 tx=0x7f8c38009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 -- 172.21.15.32:0/3439552070 shutdown_connections 2023-09-30T22:24:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e410 0x7f8c48133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3439552070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c4809dac0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3439552070 >> v2:172.21.15.32:3300/0 conn(0x7f8c4809e000 0x7f8c481339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 -- 172.21.15.32:0/3439552070 >> 172.21.15.32:0/3439552070 conn(0x7f8c48066f50 msgr2=0x7f8c4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 -- 172.21.15.32:0/3439552070 shutdown_connections 2023-09-30T22:24:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.666+0000 7f8c4d006700 1 -- 172.21.15.32:0/3439552070 wait complete. 2023-09-30T22:24:48.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.668+0000 7f8c4d006700 1 Processor -- start 2023-09-30T22:24:48.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.668+0000 7f8c4d006700 1 -- start start 2023-09-30T22:24:48.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4d006700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c481a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4d006700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 0x7f8c481a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4d006700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c4809e410 0x7f8c481a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4d006700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a9a40 con 0x7f8c4809e410 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4d006700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481aa280 con 0x7f8c4809e000 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4d006700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a79b0 con 0x7f8c4809d6f0 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 0x7f8c481a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 0x7f8c481a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51758/0 (socket says 172.21.15.32:51758) 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4659c700 1 -- 172.21.15.32:0/3323730958 learned_addr learned my addr 172.21.15.32:0/3323730958 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c4759e700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c4809e410 0x7f8c481a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.669+0000 7f8c46d9d700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c481a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4659c700 1 -- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 msgr2=0x7f8c481a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4659c700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c481a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4659c700 1 -- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c4809e410 msgr2=0x7f8c481a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4659c700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c4809e410 0x7f8c481a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4659c700 1 -- 172.21.15.32:0/3323730958 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40009a70 con 0x7f8c4809e000 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4759e700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c4809e410 0x7f8c481a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4659c700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 0x7f8c481a6370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e980 tx=0x7f8c3c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c009710 con 0x7f8c4809e000 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00eec0 con 0x7f8c4809e000 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c38009550 con 0x7f8c4809e000 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00ca70 con 0x7f8c4809e000 2023-09-30T22:24:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.670+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4809c710 con 0x7f8c4809e000 2023-09-30T22:24:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.671+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8c3c00cbd0 con 0x7f8c4809e000 2023-09-30T22:24:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.671+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f8c3c0400f0 con 0x7f8c4809e000 2023-09-30T22:24:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.671+0000 7f8c46d9d700 1 -- 172.21.15.32:0/3323730958 >> v1:172.21.15.32:6816/126553 conn(0x7f8c3402f510 legacy=0x7f8c340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:48.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.671+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48074ea0 con 0x7f8c4809e000 2023-09-30T22:24:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.681+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c3c017020 con 0x7f8c4809e000 2023-09-30T22:24:48.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:48.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.764+0000 7fd55624a700 1 Processor -- start 2023-09-30T22:24:48.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.764+0000 7fd55624a700 1 -- start start 2023-09-30T22:24:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd55624a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd55009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd55624a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd550134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd55624a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55009b120 con 0x7fd55009e000 2023-09-30T22:24:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd55624a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55006b460 con 0x7fd55009d6f0 2023-09-30T22:24:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd55624a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd55009a7e0 con 0x7fd55009e410 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd54f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd55009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd54ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd550134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd54f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd55009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51768/0 (socket says 172.21.15.32:51768) 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.765+0000 7fd54f7fe700 1 -- 172.21.15.32:0/3310044715 learned_addr learned my addr 172.21.15.32:0/3310044715 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54f7fe700 1 -- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 msgr2=0x7fd550134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54f7fe700 1 --2- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd550134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54f7fe700 1 -- 172.21.15.32:0/3310044715 >> v1:172.21.15.32:6789/0 conn(0x7fd55009e000 legacy=0x7fd5501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54f7fe700 1 -- 172.21.15.32:0/3310044715 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55009a500 con 0x7fd55009d6f0 2023-09-30T22:24:48.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54f7fe700 1 --2- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd55009dac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd540009920 tx=0x7fd540009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700d06840e1b2956 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54e7fc700 1 -- 172.21.15.32:0/3310044715 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540011830 con 0x7fd55009d6f0 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54e7fc700 1 -- 172.21.15.32:0/3310044715 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd540011990 con 0x7fd55009d6f0 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.766+0000 7fd54e7fc700 1 -- 172.21.15.32:0/3310044715 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540011d20 con 0x7fd55009d6f0 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.767+0000 7fd55624a700 1 -- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 msgr2=0x7fd55009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.767+0000 7fd55624a700 1 --2- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd55009dac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd540009920 tx=0x7fd540009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.767+0000 7fd55624a700 1 -- 172.21.15.32:0/3310044715 shutdown_connections 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.767+0000 7fd55624a700 1 --2- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd550134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.767+0000 7fd55624a700 1 --2- 172.21.15.32:0/3310044715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd55009dac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.767+0000 7fd55624a700 1 -- 172.21.15.32:0/3310044715 >> 172.21.15.32:0/3310044715 conn(0x7fd550066f50 msgr2=0x7fd55009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.768+0000 7fd55624a700 1 -- 172.21.15.32:0/3310044715 shutdown_connections 2023-09-30T22:24:48.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.768+0000 7fd55624a700 1 -- 172.21.15.32:0/3310044715 wait complete. 2023-09-30T22:24:48.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.768+0000 7fd55624a700 1 Processor -- start 2023-09-30T22:24:48.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 -- start start 2023-09-30T22:24:48.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd550122750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd55009e000 0x7fd550122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd5501231d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500a1d90 con 0x7fd55009e000 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd550124390 con 0x7fd55009d6f0 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd55624a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd55009c710 con 0x7fd55009e410 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd54ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd5501231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd54effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd55009e000 0x7fd550122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd54ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd5501231d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40310/0 (socket says 172.21.15.32:40310) 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.769+0000 7fd54ffff700 1 -- 172.21.15.32:0/2246017481 learned_addr learned my addr 172.21.15.32:0/2246017481 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54f7fe700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd550122750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54ffff700 1 -- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 msgr2=0x7fd550122750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54ffff700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd550122750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54ffff700 1 -- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd55009e000 msgr2=0x7fd550122c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54ffff700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd55009e000 0x7fd550122c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54ffff700 1 -- 172.21.15.32:0/2246017481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540009550 con 0x7fd55009e410 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54f7fe700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd550122750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54effd700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd55009e000 0x7fd550122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd54ffff700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd5501231d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b6d0 tx=0x7fd54800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548019070 con 0x7fd55009e410 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548007540 con 0x7fd55009e410 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd550124aa0 con 0x7fd55009e410 2023-09-30T22:24:48.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480078d0 con 0x7fd55009e410 2023-09-30T22:24:48.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.770+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd550126190 con 0x7fd55009e410 2023-09-30T22:24:48.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.771+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd548007a30 con 0x7fd55009e410 2023-09-30T22:24:48.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.772+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd550074ea0 con 0x7fd55009e410 2023-09-30T22:24:48.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.772+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fd548047fb0 con 0x7fd55009e410 2023-09-30T22:24:48.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.772+0000 7fd54f7fe700 1 -- 172.21.15.32:0/2246017481 >> v1:172.21.15.32:6816/126553 conn(0x7fd53c02f510 legacy=0x7fd53c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:48.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.777+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd548079750 con 0x7fd55009e410 2023-09-30T22:24:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.830+0000 7f198101f700 1 -- 172.21.15.32:0/2238097297 >> v1:172.21.15.32:6816/126553 conn(0x7f196002f510 legacy=0x7f19600319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.845+0000 7fe167d5a700 1 Processor -- start 2023-09-30T22:24:48.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.845+0000 7fe167d5a700 1 -- start start 2023-09-30T22:24:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe167d5a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe16009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe167d5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e000 0x7fe1601339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe167d5a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16009b120 con 0x7fe16009e410 2023-09-30T22:24:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe167d5a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16006b460 con 0x7fe16009e000 2023-09-30T22:24:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe167d5a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16009a7e0 con 0x7fe16009d6f0 2023-09-30T22:24:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe165af6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe16009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe165af6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe16009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40326/0 (socket says 172.21.15.32:40326) 2023-09-30T22:24:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe165af6700 1 -- 172.21.15.32:0/161415749 learned_addr learned my addr 172.21.15.32:0/161415749 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe1652f5700 1 --2- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e000 0x7fe1601339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.846+0000 7fe165af6700 1 -- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e000 msgr2=0x7fe1601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe165af6700 1 --2- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e000 0x7fe1601339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe165af6700 1 -- 172.21.15.32:0/161415749 >> v1:172.21.15.32:6789/0 conn(0x7fe16009e410 legacy=0x7fe160133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe165af6700 1 -- 172.21.15.32:0/161415749 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16009a500 con 0x7fe16009d6f0 2023-09-30T22:24:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe165af6700 1 --2- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe16009dac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009950 tx=0x7fe15c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3141d0346a91c280 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe164af4700 1 -- 172.21.15.32:0/161415749 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c011830 con 0x7fe16009d6f0 2023-09-30T22:24:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe164af4700 1 -- 172.21.15.32:0/161415749 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c011990 con 0x7fe16009d6f0 2023-09-30T22:24:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.847+0000 7fe164af4700 1 -- 172.21.15.32:0/161415749 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c011d40 con 0x7fe16009d6f0 2023-09-30T22:24:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 -- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 msgr2=0x7fe16009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 --2- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe16009dac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009950 tx=0x7fe15c009c60 comp rx=0 tx=0).stop 2023-09-30T22:24:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 -- 172.21.15.32:0/161415749 shutdown_connections 2023-09-30T22:24:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 --2- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe16009dac0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 --2- 172.21.15.32:0/161415749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e000 0x7fe1601339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 -- 172.21.15.32:0/161415749 >> 172.21.15.32:0/161415749 conn(0x7fe160066f50 msgr2=0x7fe16009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 -- 172.21.15.32:0/161415749 shutdown_connections 2023-09-30T22:24:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.848+0000 7fe167d5a700 1 -- 172.21.15.32:0/161415749 wait complete. 2023-09-30T22:24:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 Processor -- start 2023-09-30T22:24:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 -- start start 2023-09-30T22:24:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe1601a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16009e000 0x7fe1601a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e410 0x7fe1601a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1601a9880 con 0x7fe16009e000 2023-09-30T22:24:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1601a9a60 con 0x7fe16009e410 2023-09-30T22:24:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe167d5a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16009c710 con 0x7fe16009d6f0 2023-09-30T22:24:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.850+0000 7fe165af6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe1601a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe1601a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40328/0 (socket says 172.21.15.32:40328) 2023-09-30T22:24:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 -- 172.21.15.32:0/3262643922 learned_addr learned my addr 172.21.15.32:0/3262643922 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe1662f7700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e410 0x7fe1601a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe1652f5700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16009e000 0x7fe1601a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 -- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e410 msgr2=0x7fe1601a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e410 0x7fe1601a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 -- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16009e000 msgr2=0x7fe1601a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16009e000 0x7fe1601a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 -- 172.21.15.32:0/3262643922 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009550 con 0x7fe16009d6f0 2023-09-30T22:24:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe1662f7700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e410 0x7fe1601a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe165af6700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe1601a5fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe16009a970 tx=0x7fe15c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c004300 con 0x7fe16009d6f0 2023-09-30T22:24:48.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c01f070 con 0x7fe16009d6f0 2023-09-30T22:24:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1601a7bf0 con 0x7fe16009d6f0 2023-09-30T22:24:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0036a0 con 0x7fe16009d6f0 2023-09-30T22:24:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.851+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1601a82d0 con 0x7fe16009d6f0 2023-09-30T22:24:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.852+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe15c021030 con 0x7fe16009d6f0 2023-09-30T22:24:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.853+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fe15c040e80 con 0x7fe16009d6f0 2023-09-30T22:24:48.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.853+0000 7fe1652f5700 1 -- 172.21.15.32:0/3262643922 >> v1:172.21.15.32:6816/126553 conn(0x7fe14402f510 legacy=0x7fe1440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.853+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe160074ea0 con 0x7fe16009d6f0 2023-09-30T22:24:48.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.858+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe15c015080 con 0x7fe16009d6f0 2023-09-30T22:24:48.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.864+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f197c074ea0 con 0x7f197c09e760 2023-09-30T22:24:48.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.864+0000 7f1971ffb700 1 -- 172.21.15.32:0/2238097297 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f196c01bbe0 con 0x7f197c09e760 2023-09-30T22:24:48.865 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.866+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 >> v1:172.21.15.32:6816/126553 conn(0x7f196002f510 legacy=0x7f19600319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.866+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 msgr2=0x7f197c11bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.866+0000 7f1983283700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c11bbf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f196c00e870 tx=0x7f196c00eb80 comp rx=0 tx=0).stop 2023-09-30T22:24:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 shutdown_connections 2023-09-30T22:24:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f197c1338a0 0x7f197c11c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f197c09de50 0x7f197c11b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 --2- 172.21.15.32:0/2238097297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f197c09e760 0x7f197c11bbf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 >> 172.21.15.32:0/2238097297 conn(0x7f197c0a2aa0 msgr2=0x7f197c130630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:48.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 shutdown_connections 2023-09-30T22:24:48.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.867+0000 7f1983283700 1 -- 172.21.15.32:0/2238097297 wait complete. 2023-09-30T22:24:48.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.872+0000 7f8c46d9d700 1 -- 172.21.15.32:0/3323730958 >> v1:172.21.15.32:6816/126553 conn(0x7f8c3402f510 legacy=0x7f8c340319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:48.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.923+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8c4809ace0 con 0x7f8c4809e000 2023-09-30T22:24:48.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.923+0000 7f8c2f7fe700 1 -- 172.21.15.32:0/3323730958 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c3c01b3b0 con 0x7f8c4809e000 2023-09-30T22:24:48.924 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 >> v1:172.21.15.32:6816/126553 conn(0x7f8c3402f510 legacy=0x7f8c340319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 msgr2=0x7f8c481a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 0x7f8c481a6370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e980 tx=0x7f8c3c00ed40 comp rx=0 tx=0).stop 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 shutdown_connections 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c4809e410 0x7f8c481a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8c4809d6f0 0x7f8c481a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.926+0000 7f8c4d006700 1 --2- 172.21.15.32:0/3323730958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8c4809e000 0x7f8c481a6370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.927+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 >> 172.21.15.32:0/3323730958 conn(0x7f8c48066f50 msgr2=0x7f8c481bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.927+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 shutdown_connections 2023-09-30T22:24:48.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.927+0000 7f8c4d006700 1 -- 172.21.15.32:0/3323730958 wait complete. 2023-09-30T22:24:48.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:48.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:48.973+0000 7fd54f7fe700 1 -- 172.21.15.32:0/2246017481 >> v1:172.21.15.32:6816/126553 conn(0x7fd53c02f510 legacy=0x7fd53c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.025+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd550125f70 con 0x7fd55009e410 2023-09-30T22:24:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.025+0000 7fd537fff700 1 -- 172.21.15.32:0/2246017481 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd548022020 con 0x7fd55009e410 2023-09-30T22:24:49.026 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.027+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 >> v1:172.21.15.32:6816/126553 conn(0x7fd53c02f510 legacy=0x7fd53c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.027+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 msgr2=0x7fd5501231d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.027+0000 7fd55624a700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd5501231d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b6d0 tx=0x7fd54800b9e0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 shutdown_connections 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd55009e410 0x7fd5501231d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd55009d6f0 0x7fd550122750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 --2- 172.21.15.32:0/2246017481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd55009e000 0x7fd550122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 >> 172.21.15.32:0/2246017481 conn(0x7fd550066f50 msgr2=0x7fd5500a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 shutdown_connections 2023-09-30T22:24:49.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.028+0000 7fd55624a700 1 -- 172.21.15.32:0/2246017481 wait complete. 2023-09-30T22:24:49.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:49.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.054+0000 7fe1652f5700 1 -- 172.21.15.32:0/3262643922 >> v1:172.21.15.32:6816/126553 conn(0x7fe14402f510 legacy=0x7fe1440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.070+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe1600a3190 con 0x7fe16009d6f0 2023-09-30T22:24:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.071+0000 7fe1567fc700 1 -- 172.21.15.32:0/3262643922 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe15c023790 con 0x7fe16009d6f0 2023-09-30T22:24:49.072 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 >> v1:172.21.15.32:6816/126553 conn(0x7fe14402f510 legacy=0x7fe1440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 msgr2=0x7fe1601a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe1601a5fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe16009a970 tx=0x7fe15c004750 comp rx=0 tx=0).stop 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 shutdown_connections 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe16009e410 0x7fe1601a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe16009d6f0 0x7fe1601a5fc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 --2- 172.21.15.32:0/3262643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe16009e000 0x7fe1601a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 >> 172.21.15.32:0/3262643922 conn(0x7fe160066f50 msgr2=0x7fe1600a36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 shutdown_connections 2023-09-30T22:24:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:49.073+0000 7fe167d5a700 1 -- 172.21.15.32:0/3262643922 wait complete. 2023-09-30T22:24:49.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:49.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:49.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 Processor -- start 2023-09-30T22:24:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 -- start start 2023-09-30T22:24:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d8134170 0x7fa5d813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d809edd0 con 0x7fa5d80a1340 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d809e1f0 con 0x7fa5d8134170 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.031+0000 7fa5deeb0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d809dc50 con 0x7fa5d80a0a30 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dd44d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d8134170 0x7fa5d813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5d80a1340 0x7fa5d8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55812/0 (socket says 172.21.15.32:55812) 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dcc4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d7fff700 1 -- 172.21.15.32:0/3040676970 learned_addr learned my addr 172.21.15.32:0/3040676970 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d77fe700 1 -- 172.21.15.32:0/3040676970 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091255133 0 0) 0x7fa5d809edd0 con 0x7fa5d80a1340 2023-09-30T22:24:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d77fe700 1 -- 172.21.15.32:0/3040676970 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c80035b0 con 0x7fa5d80a1340 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dd44d700 1 -- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 msgr2=0x7fa5d80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dd44d700 1 --2- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dd44d700 1 -- 172.21.15.32:0/3040676970 >> v1:172.21.15.32:6789/0 conn(0x7fa5d80a1340 legacy=0x7fa5d8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dd44d700 1 -- 172.21.15.32:0/3040676970 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80a1fe0 con 0x7fa5d8134170 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dcc4c700 1 --2- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5dd44d700 1 --2- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d8134170 0x7fa5d813bd30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009dc0 tx=0x7fa5d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c97337fa31ac8e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d77fe700 1 -- 172.21.15.32:0/3040676970 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000e070 con 0x7fa5d8134170 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d77fe700 1 -- 172.21.15.32:0/3040676970 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d000bb20 con 0x7fa5d8134170 2023-09-30T22:24:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.032+0000 7fa5d77fe700 1 -- 172.21.15.32:0/3040676970 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0012430 con 0x7fa5d8134170 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 -- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d8134170 msgr2=0x7fa5d813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d8134170 0x7fa5d813bd30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009dc0 tx=0x7fa5d000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 -- 172.21.15.32:0/3040676970 shutdown_connections 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d8134170 0x7fa5d813bd30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/3040676970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 -- 172.21.15.32:0/3040676970 >> 172.21.15.32:0/3040676970 conn(0x7fa5d80a6550 msgr2=0x7fa5d80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 -- 172.21.15.32:0/3040676970 shutdown_connections 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.033+0000 7fa5deeb0700 1 -- 172.21.15.32:0/3040676970 wait complete. 2023-09-30T22:24:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.034+0000 7fa5deeb0700 1 Processor -- start 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.034+0000 7fa5deeb0700 1 -- start start 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5deeb0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d8122800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5deeb0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 0x7fa5d8122d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5deeb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d8134170 0x7fa5d8123280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5deeb0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8125c50 con 0x7fa5d8134170 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5deeb0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8133690 con 0x7fa5d80a1340 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5deeb0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d81262f0 con 0x7fa5d80a0a30 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 0x7fa5d8122d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5dcc4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d8122800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 0x7fa5d8122d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51806/0 (socket says 172.21.15.32:51806) 2023-09-30T22:24:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 -- 172.21.15.32:0/587954291 learned_addr learned my addr 172.21.15.32:0/587954291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5dd44d700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d8134170 0x7fa5d8123280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 -- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 msgr2=0x7fa5d8122800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d8122800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 -- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d8134170 msgr2=0x7fa5d8123280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d8134170 0x7fa5d8123280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5d7fff700 1 -- 172.21.15.32:0/587954291 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0094a0 con 0x7fa5d80a1340 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.035+0000 7fa5dcc4c700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d8122800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.036+0000 7fa5d7fff700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 0x7fa5d8122d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c700 tx=0x7fa5c400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.036+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4005d70 con 0x7fa5d80a1340 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.036+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40070d0 con 0x7fa5d80a1340 2023-09-30T22:24:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.036+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d0009a70 con 0x7fa5d80a1340 2023-09-30T22:24:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.036+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007460 con 0x7fa5d80a1340 2023-09-30T22:24:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.036+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d80a25e0 con 0x7fa5d80a1340 2023-09-30T22:24:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.037+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa5c40075c0 con 0x7fa5d80a1340 2023-09-30T22:24:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.037+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d8074ea0 con 0x7fa5d80a1340 2023-09-30T22:24:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.037+0000 7fa5dcc4c700 1 -- 172.21.15.32:0/587954291 >> v1:172.21.15.32:6816/126553 conn(0x7fa5bc02f510 legacy=0x7fa5bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.037+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fa5c403f040 con 0x7fa5d80a1340 2023-09-30T22:24:50.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.042+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5c4018260 con 0x7fa5d80a1340 2023-09-30T22:24:50.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:50.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.094+0000 7f9b8d803700 1 Processor -- start 2023-09-30T22:24:50.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.094+0000 7f9b8d803700 1 -- start start 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b8d803700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b8806a770 0x7f9b88133830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b8d803700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133d70 0x7f9b881303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b8d803700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8809f440 con 0x7f9b88133280 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b8d803700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8806ab40 con 0x7f9b8806a770 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b8d803700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8809a2c0 con 0x7f9b88133d70 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b867fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b88133280 0x7f9b8812fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55824/0 (socket says 172.21.15.32:55824) 2023-09-30T22:24:50.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.095+0000 7f9b867fc700 1 -- 172.21.15.32:0/292711868 learned_addr learned my addr 172.21.15.32:0/292711868 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b86ffd700 1 --2- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b8806a770 0x7f9b88133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b86ffd700 1 -- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133d70 msgr2=0x7f9b881303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b86ffd700 1 --2- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133d70 0x7f9b881303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b86ffd700 1 -- 172.21.15.32:0/292711868 >> v1:172.21.15.32:6789/0 conn(0x7f9b88133280 legacy=0x7f9b8812fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b86ffd700 1 -- 172.21.15.32:0/292711868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8806b250 con 0x7f9b8806a770 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b86ffd700 1 --2- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b8806a770 0x7f9b88133830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098a0 tx=0x7f9b78009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1020b89368239f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b85ffb700 1 -- 172.21.15.32:0/292711868 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011830 con 0x7f9b8806a770 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b85ffb700 1 -- 172.21.15.32:0/292711868 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78011990 con 0x7f9b8806a770 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.096+0000 7f9b85ffb700 1 -- 172.21.15.32:0/292711868 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011d40 con 0x7f9b8806a770 2023-09-30T22:24:50.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 -- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b8806a770 msgr2=0x7f9b88133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 --2- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b8806a770 0x7f9b88133830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098a0 tx=0x7f9b78009bb0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 -- 172.21.15.32:0/292711868 shutdown_connections 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 --2- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133d70 0x7f9b881303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 --2- 172.21.15.32:0/292711868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b8806a770 0x7f9b88133830 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 -- 172.21.15.32:0/292711868 >> 172.21.15.32:0/292711868 conn(0x7f9b880a2cc0 msgr2=0x7f9b880a30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 -- 172.21.15.32:0/292711868 shutdown_connections 2023-09-30T22:24:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.097+0000 7f9b8d803700 1 -- 172.21.15.32:0/292711868 wait complete. 2023-09-30T22:24:50.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.099+0000 7f9b8d803700 1 Processor -- start 2023-09-30T22:24:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.099+0000 7f9b8d803700 1 -- start start 2023-09-30T22:24:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b8d803700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 0x7f9b8811cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b8d803700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133280 0x7f9b88119310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b8d803700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b88133d70 0x7f9b88119850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b8d803700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8811b4a0 con 0x7f9b8806a770 2023-09-30T22:24:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b8d803700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8811c910 con 0x7f9b88133d70 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b8d803700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8809c760 con 0x7f9b88133280 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 0x7f9b8811cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 0x7f9b8811cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52494/0 (socket says 172.21.15.32:52494) 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 -- 172.21.15.32:0/3858637503 learned_addr learned my addr 172.21.15.32:0/3858637503 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b877fe700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b88133d70 0x7f9b88119850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b867fc700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133280 0x7f9b88119310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 -- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133280 msgr2=0x7f9b88119310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133280 0x7f9b88119310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 -- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b88133d70 msgr2=0x7f9b88119850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b88133d70 0x7f9b88119850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b86ffd700 1 -- 172.21.15.32:0/3858637503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b780094a0 con 0x7f9b8806a770 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.100+0000 7f9b877fe700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b88133d70 0x7f9b88119850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.101+0000 7f9b86ffd700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 0x7f9b8811cf70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b780090b0 tx=0x7f9b78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.101+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780038a0 con 0x7f9b8806a770 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.101+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7801f070 con 0x7f9b8806a770 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.101+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8811c6a0 con 0x7f9b8806a770 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.101+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8812d900 con 0x7f9b8806a770 2023-09-30T22:24:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.101+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003c60 con 0x7f9b8806a770 2023-09-30T22:24:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.102+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9b7801a430 con 0x7f9b8806a770 2023-09-30T22:24:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.102+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f9b78021070 con 0x7f9b8806a770 2023-09-30T22:24:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.102+0000 7f9b867fc700 1 -- 172.21.15.32:0/3858637503 >> v1:172.21.15.32:6816/126553 conn(0x7f9b7402f540 legacy=0x7f9b740319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.103+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b680052e0 con 0x7f9b8806a770 2023-09-30T22:24:50.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.112+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b78015080 con 0x7f9b8806a770 2023-09-30T22:24:50.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.201+0000 7f39cc87c700 1 Processor -- start 2023-09-30T22:24:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.201+0000 7f39cc87c700 1 -- start start 2023-09-30T22:24:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39cc87c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39cc87c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a1340 0x7f39c8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39cc87c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c809edd0 con 0x7f39c8133fa0 2023-09-30T22:24:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39cc87c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c809e1f0 con 0x7f39c80a1340 2023-09-30T22:24:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39cc87c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39c809dc50 con 0x7f39c80a0a30 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40386/0 (socket says 172.21.15.32:40386) 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c659c700 1 -- 172.21.15.32:0/1861718307 learned_addr learned my addr 172.21.15.32:0/1861718307 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c5d9b700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a1340 0x7f39c8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c5d9b700 1 -- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 msgr2=0x7f39c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c5d9b700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c5d9b700 1 -- 172.21.15.32:0/1861718307 >> v1:172.21.15.32:6789/0 conn(0x7f39c8133fa0 legacy=0x7f39c813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.202+0000 7f39c5d9b700 1 -- 172.21.15.32:0/1861718307 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80a1fe0 con 0x7f39c80a1340 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.203+0000 7f39c659c700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.203+0000 7f39c5d9b700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a1340 0x7f39c8133a60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009870 tx=0x7f39bc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5edf0dab96a51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.203+0000 7f39c559a700 1 -- 172.21.15.32:0/1861718307 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc004230 con 0x7f39c80a1340 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.203+0000 7f39c559a700 1 -- 172.21.15.32:0/1861718307 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc004390 con 0x7f39c80a1340 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.203+0000 7f39c559a700 1 -- 172.21.15.32:0/1861718307 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0036a0 con 0x7f39c80a1340 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 -- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a1340 msgr2=0x7f39c8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a1340 0x7f39c8133a60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009870 tx=0x7f39bc009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 -- 172.21.15.32:0/1861718307 shutdown_connections 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 --2- 172.21.15.32:0/1861718307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a1340 0x7f39c8133a60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 -- 172.21.15.32:0/1861718307 >> 172.21.15.32:0/1861718307 conn(0x7f39c80a6550 msgr2=0x7f39c80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 -- 172.21.15.32:0/1861718307 shutdown_connections 2023-09-30T22:24:50.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.204+0000 7f39cc87c700 1 -- 172.21.15.32:0/1861718307 wait complete. 2023-09-30T22:24:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.206+0000 7f39cc87c700 1 Processor -- start 2023-09-30T22:24:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.206+0000 7f39cc87c700 1 -- start start 2023-09-30T22:24:50.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39cc87c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c81c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39cc87c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a1340 0x7f39c81c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39cc87c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c8133fa0 0x7f39c81c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39cc87c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8120680 con 0x7f39c8133fa0 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39cc87c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8120490 con 0x7f39c80a0a30 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39cc87c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39c81223a0 con 0x7f39c80a1340 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39c659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c81c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39c659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c81c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51838/0 (socket says 172.21.15.32:51838) 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39c659c700 1 -- 172.21.15.32:0/4000862864 learned_addr learned my addr 172.21.15.32:0/4000862864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:50.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39c6d9d700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c8133fa0 0x7f39c81c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.207+0000 7f39c5d9b700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a1340 0x7f39c81c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39c659c700 1 -- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a1340 msgr2=0x7f39c81c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39c659c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a1340 0x7f39c81c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39c659c700 1 -- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c8133fa0 msgr2=0x7f39c81c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39c659c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c8133fa0 0x7f39c81c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39c659c700 1 -- 172.21.15.32:0/4000862864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b8009550 con 0x7f39c80a0a30 2023-09-30T22:24:50.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39c659c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c81c8450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39b800ebe0 tx=0x7f39b800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800c850 con 0x7f39c80a0a30 2023-09-30T22:24:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b800c9b0 con 0x7f39c80a0a30 2023-09-30T22:24:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800cd20 con 0x7f39c80a0a30 2023-09-30T22:24:50.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc0094a0 con 0x7f39c80a0a30 2023-09-30T22:24:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.208+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c80a25e0 con 0x7f39c80a0a30 2023-09-30T22:24:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.209+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f39b8013470 con 0x7f39c80a0a30 2023-09-30T22:24:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.209+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f39b80401e0 con 0x7f39c80a0a30 2023-09-30T22:24:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.209+0000 7f39c5d9b700 1 -- 172.21.15.32:0/4000862864 >> v1:172.21.15.32:6816/126553 conn(0x7f39a802f510 legacy=0x7f39a80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.210+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c8074ea0 con 0x7f39c80a0a30 2023-09-30T22:24:50.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.219+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39b8021080 con 0x7f39c80a0a30 2023-09-30T22:24:50.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.238+0000 7fa5dcc4c700 1 -- 172.21.15.32:0/587954291 >> v1:172.21.15.32:6816/126553 conn(0x7fa5bc02f510 legacy=0x7fa5bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 Processor -- start 2023-09-30T22:24:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 -- start start 2023-09-30T22:24:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc131810 0x7faecc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc1313c0 con 0x7faecc099a40 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc06e8c0 con 0x7faecc131810 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faed160e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faecc071d20 con 0x7faecc09a580 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faecaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc131810 0x7faecc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faecaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc131810 0x7faecc131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:51852/0 (socket says 172.21.15.32:51852) 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.247+0000 7faecaffd700 1 -- 172.21.15.32:0/3044192007 learned_addr learned my addr 172.21.15.32:0/3044192007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faecb7fe700 1 --2- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faecb7fe700 1 -- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc131810 msgr2=0x7faecc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faecb7fe700 1 --2- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc131810 0x7faecc131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faecb7fe700 1 -- 172.21.15.32:0/3044192007 >> v1:172.21.15.32:6789/0 conn(0x7faecc099a40 legacy=0x7faecc099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faec9ffb700 1 -- 172.21.15.32:0/3044192007 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012858711 0 0) 0x7faecc1313c0 con 0x7faecc099a40 2023-09-30T22:24:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faecb7fe700 1 -- 172.21.15.32:0/3044192007 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0721a0 con 0x7faecc09a580 2023-09-30T22:24:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faecb7fe700 1 --2- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc139660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faec4009dc0 tx=0x7faec400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b256ccf3fb67e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.248+0000 7faec9ffb700 1 -- 172.21.15.32:0/3044192007 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400e070 con 0x7faecc09a580 2023-09-30T22:24:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faec9ffb700 1 -- 172.21.15.32:0/3044192007 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec400bb20 con 0x7faecc09a580 2023-09-30T22:24:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faec9ffb700 1 -- 172.21.15.32:0/3044192007 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4012430 con 0x7faecc09a580 2023-09-30T22:24:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 -- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 msgr2=0x7faecc139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 --2- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc139660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faec4009dc0 tx=0x7faec400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 -- 172.21.15.32:0/3044192007 shutdown_connections 2023-09-30T22:24:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 --2- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc139660 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 --2- 172.21.15.32:0/3044192007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc131810 0x7faecc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 -- 172.21.15.32:0/3044192007 >> 172.21.15.32:0/3044192007 conn(0x7faecc0a2aa0 msgr2=0x7faecc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 -- 172.21.15.32:0/3044192007 shutdown_connections 2023-09-30T22:24:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.249+0000 7faed160e700 1 -- 172.21.15.32:0/3044192007 wait complete. 2023-09-30T22:24:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.251+0000 7faed160e700 1 Processor -- start 2023-09-30T22:24:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.251+0000 7faed160e700 1 -- start start 2023-09-30T22:24:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faed160e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc099a40 0x7faecc11b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faed160e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc11bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faed160e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faecc131810 0x7faecc11c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faed160e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc11d280 con 0x7faecc131810 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faed160e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc11d8a0 con 0x7faecc099a40 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faed160e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faecc11e4d0 con 0x7faecc09a580 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faeca7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc11bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc099a40 0x7faecc11b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faeca7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc11bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40420/0 (socket says 172.21.15.32:40420) 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faeca7fc700 1 -- 172.21.15.32:0/2112141643 learned_addr learned my addr 172.21.15.32:0/2112141643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecb7fe700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faecc131810 0x7faecc11c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecaffd700 1 -- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 msgr2=0x7faecc11bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecaffd700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc11bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecaffd700 1 -- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faecc131810 msgr2=0x7faecc11c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecaffd700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faecc131810 0x7faecc11c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.252+0000 7faecaffd700 1 -- 172.21.15.32:0/2112141643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009a70 con 0x7faecc099a40 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.253+0000 7faecaffd700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc099a40 0x7faecc11b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e910 tx=0x7faebc00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.253+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc013680 con 0x7faecc099a40 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.253+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc0137e0 con 0x7faecc099a40 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.253+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc11e920 con 0x7faecc099a40 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.253+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc013b70 con 0x7faecc099a40 2023-09-30T22:24:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.253+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc11cbb0 con 0x7faecc099a40 2023-09-30T22:24:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.254+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7faebc013cd0 con 0x7faecc099a40 2023-09-30T22:24:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.254+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc074ea0 con 0x7faecc099a40 2023-09-30T22:24:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.254+0000 7faeca7fc700 1 -- 172.21.15.32:0/2112141643 >> v1:172.21.15.32:6816/126553 conn(0x7faeb802f510 legacy=0x7faeb80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.254+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7faebc03ffd0 con 0x7faecc099a40 2023-09-30T22:24:50.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.260+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faebc017080 con 0x7faecc099a40 2023-09-30T22:24:50.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.271+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa5d8124380 con 0x7fa5d80a1340 2023-09-30T22:24:50.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.272+0000 7fa5d57fa700 1 -- 172.21.15.32:0/587954291 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5c4017070 con 0x7fa5d80a1340 2023-09-30T22:24:50.273 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 >> v1:172.21.15.32:6816/126553 conn(0x7fa5bc02f510 legacy=0x7fa5bc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 msgr2=0x7fa5d8122d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 0x7fa5d8122d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c700 tx=0x7fa5c400ca10 comp rx=0 tx=0).stop 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 shutdown_connections 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d8134170 0x7fa5d8123280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5d80a0a30 0x7fa5d8122800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 --2- 172.21.15.32:0/587954291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5d80a1340 0x7fa5d8122d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 >> 172.21.15.32:0/587954291 conn(0x7fa5d80a6550 msgr2=0x7fa5d809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.274+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 shutdown_connections 2023-09-30T22:24:50.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.275+0000 7fa5deeb0700 1 -- 172.21.15.32:0/587954291 wait complete. 2023-09-30T22:24:50.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:50.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.303+0000 7f9b867fc700 1 -- 172.21.15.32:0/3858637503 >> v1:172.21.15.32:6816/126553 conn(0x7f9b7402f540 legacy=0x7f9b740319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.333+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9b68005860 con 0x7f9b8806a770 2023-09-30T22:24:50.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.333+0000 7f9b6ffff700 1 -- 172.21.15.32:0/3858637503 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b7802e7b0 con 0x7f9b8806a770 2023-09-30T22:24:50.334 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:50.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 >> v1:172.21.15.32:6816/126553 conn(0x7f9b7402f540 legacy=0x7f9b740319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:50.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 msgr2=0x7f9b8811cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 0x7f9b8811cf70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b780090b0 tx=0x7f9b78004750 comp rx=0 tx=0).stop 2023-09-30T22:24:50.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 shutdown_connections 2023-09-30T22:24:50.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b88133d70 0x7f9b88119850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b8806a770 0x7f9b8811cf70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 --2- 172.21.15.32:0/3858637503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b88133280 0x7f9b88119310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 >> 172.21.15.32:0/3858637503 conn(0x7f9b880a2cc0 msgr2=0x7f9b8809b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 shutdown_connections 2023-09-30T22:24:50.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.337+0000 7f9b8d803700 1 -- 172.21.15.32:0/3858637503 wait complete. 2023-09-30T22:24:50.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.411+0000 7f39c5d9b700 1 -- 172.21.15.32:0/4000862864 >> v1:172.21.15.32:6816/126553 conn(0x7f39a802f510 legacy=0x7f39a80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.430+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f39c80a28e0 con 0x7f39c80a0a30 2023-09-30T22:24:50.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.430+0000 7f39b6ffd700 1 -- 172.21.15.32:0/4000862864 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39b8017070 con 0x7f39c80a0a30 2023-09-30T22:24:50.431 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 >> v1:172.21.15.32:6816/126553 conn(0x7f39a802f510 legacy=0x7f39a80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 msgr2=0x7f39c81c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c81c8450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39b800ebe0 tx=0x7f39b800eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 shutdown_connections 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c8133fa0 0x7f39c81c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c81c8450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 --2- 172.21.15.32:0/4000862864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a1340 0x7f39c81c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 >> 172.21.15.32:0/4000862864 conn(0x7f39c80a6550 msgr2=0x7f39c81b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 shutdown_connections 2023-09-30T22:24:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.432+0000 7f39cc87c700 1 -- 172.21.15.32:0/4000862864 wait complete. 2023-09-30T22:24:50.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:50.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.456+0000 7faeca7fc700 1 -- 172.21.15.32:0/2112141643 >> v1:172.21.15.32:6816/126553 conn(0x7faeb802f510 legacy=0x7faeb80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:50.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.479+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faecc06b9e0 con 0x7faecc099a40 2023-09-30T22:24:50.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.479+0000 7faeb37fe700 1 -- 172.21.15.32:0/2112141643 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faebc011db0 con 0x7faecc099a40 2023-09-30T22:24:50.480 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:50.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 >> v1:172.21.15.32:6816/126553 conn(0x7faeb802f510 legacy=0x7faeb80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc099a40 msgr2=0x7faecc11b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc099a40 0x7faecc11b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e910 tx=0x7faebc00ec20 comp rx=0 tx=0).stop 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 shutdown_connections 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faecc131810 0x7faecc11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faecc099a40 0x7faecc11b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.481+0000 7faed160e700 1 --2- 172.21.15.32:0/2112141643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faecc09a580 0x7faecc11bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.482+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 >> 172.21.15.32:0/2112141643 conn(0x7faecc0a2aa0 msgr2=0x7faecc09d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.482+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 shutdown_connections 2023-09-30T22:24:50.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:50.482+0000 7faed160e700 1 -- 172.21.15.32:0/2112141643 wait complete. 2023-09-30T22:24:50.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:51.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:51.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:51.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.439+0000 7ff3bc9d7700 1 Processor -- start 2023-09-30T22:24:51.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.439+0000 7ff3bc9d7700 1 -- start start 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3bc9d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3bc9d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3b809d630 0x7ff3b80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3bc9d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80a2a70 con 0x7ff3b8133930 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3bc9d7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b80a01d0 con 0x7ff3b809d630 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3bc9d7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b809e2b0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3b809d630 0x7ff3b80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3b8133930 0x7ff3b8133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40194/0 (socket says 172.21.15.32:40194) 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3b809d630 0x7ff3b80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57524/0 (socket says 172.21.15.32:57524) 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59120/0 (socket says 172.21.15.32:59120) 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b6d9d700 1 -- 172.21.15.32:0/145095586 learned_addr learned my addr 172.21.15.32:0/145095586 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b5d9b700 1 -- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 msgr2=0x7ff3b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.440+0000 7ff3b5d9b700 1 --2- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b5d9b700 1 -- 172.21.15.32:0/145095586 >> v1:172.21.15.32:6789/0 conn(0x7ff3b8133930 legacy=0x7ff3b8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b559a700 1 -- 172.21.15.32:0/145095586 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792334827 0 0) 0x7ff3b80a2a70 con 0x7ff3b8133930 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b5d9b700 1 -- 172.21.15.32:0/145095586 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b809dc50 con 0x7ff3b809d630 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b5d9b700 1 --2- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b809d630 0x7ff3b80a1340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009840 tx=0x7ff3a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ce373d55e179c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b559a700 1 -- 172.21.15.32:0/145095586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0011830 con 0x7ff3b809d630 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b559a700 1 -- 172.21.15.32:0/145095586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0011990 con 0x7ff3b809d630 2023-09-30T22:24:51.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.441+0000 7ff3b559a700 1 -- 172.21.15.32:0/145095586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0011d20 con 0x7ff3b809d630 2023-09-30T22:24:51.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b809d630 msgr2=0x7ff3b80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b809d630 0x7ff3b80a1340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009840 tx=0x7ff3a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/145095586 shutdown_connections 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/145095586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b809d630 0x7ff3b80a1340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/145095586 >> 172.21.15.32:0/145095586 conn(0x7ff3b80a3050 msgr2=0x7ff3b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/145095586 shutdown_connections 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.442+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/145095586 wait complete. 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 Processor -- start 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 -- start start 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 0x7ff3b811f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b811f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b8133930 0x7ff3b81c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81223a0 con 0x7ff3b8133930 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8122580 con 0x7ff3b809d630 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.443+0000 7ff3bc9d7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81333f0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 0x7ff3b811f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b811f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 0x7ff3b811f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47960/0 (socket says 172.21.15.32:47960) 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b8133930 0x7ff3b81c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b811f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59134/0 (socket says 172.21.15.32:59134) 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b659c700 1 -- 172.21.15.32:0/495167944 learned_addr learned my addr 172.21.15.32:0/495167944 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 -- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 msgr2=0x7ff3b811f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 0x7ff3b811f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 -- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b8133930 msgr2=0x7ff3b81c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b8133930 0x7ff3b81c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 -- 172.21.15.32:0/495167944 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00094a0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b659c700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 0x7ff3b811f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b6d9d700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b8133930 0x7ff3b81c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3b5d9b700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b811f970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0016000 tx=0x7ff3a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004400 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a001f070 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.444+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b811feb0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.445+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00038a0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.445+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b809d370 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.445+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff3a0021030 con 0x7ff3b80a0a30 2023-09-30T22:24:51.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.445+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7ff3a0040d30 con 0x7ff3b80a0a30 2023-09-30T22:24:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.446+0000 7ff3b659c700 1 -- 172.21.15.32:0/495167944 >> v1:172.21.15.32:6816/126553 conn(0x7ff39802f510 legacy=0x7ff3980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.446+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b8120ec0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:51.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.450+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3a0015020 con 0x7ff3b80a0a30 2023-09-30T22:24:51.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 Processor -- start 2023-09-30T22:24:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 -- start start 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72000a0a30 0x7f72000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f720009d630 0x7f72000a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000a2a70 con 0x7f7200133930 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72000a01d0 con 0x7f720009d630 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.499+0000 7f72085d1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f720009e2b0 con 0x7f72000a0a30 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f720009d630 0x7f72000a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7206b6e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7200133930 0x7f7200133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40210/0 (socket says 172.21.15.32:40210) 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f720636d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72000a0a30 0x7f72000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7206b6e700 1 -- 172.21.15.32:0/1441618153 learned_addr learned my addr 172.21.15.32:0/1441618153 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 --2- 172.21.15.32:0/1441618153 >> v2:172.21.15.32:3300/0 conn(0x7f720009d630 0x7f72000a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57538/0 (socket says 172.21.15.32:57538) 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 -- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72000a0a30 msgr2=0x7f72000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 --2- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72000a0a30 0x7f72000a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 -- 172.21.15.32:0/1441618153 >> v1:172.21.15.32:6789/0 conn(0x7f7200133930 legacy=0x7f7200133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 -- 172.21.15.32:0/1441618153 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720009dc50 con 0x7f720009d630 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.500+0000 7f7205b6c700 1 --2- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f720009d630 0x7f72000a1340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009840 tx=0x7f71f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c364d59d14786f4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f720536b700 1 -- 172.21.15.32:0/1441618153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011830 con 0x7f720009d630 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f720536b700 1 -- 172.21.15.32:0/1441618153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0011990 con 0x7f720009d630 2023-09-30T22:24:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f720536b700 1 -- 172.21.15.32:0/1441618153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011d20 con 0x7f720009d630 2023-09-30T22:24:51.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f72085d1700 1 -- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f720009d630 msgr2=0x7f72000a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f72085d1700 1 --2- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f720009d630 0x7f72000a1340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009840 tx=0x7f71f0009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:51.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f72085d1700 1 -- 172.21.15.32:0/1441618153 shutdown_connections 2023-09-30T22:24:51.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f72085d1700 1 --2- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72000a0a30 0x7f72000a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f72085d1700 1 --2- 172.21.15.32:0/1441618153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f720009d630 0x7f72000a1340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.501+0000 7f72085d1700 1 -- 172.21.15.32:0/1441618153 >> 172.21.15.32:0/1441618153 conn(0x7f72000a3050 msgr2=0x7f72000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.502+0000 7f72085d1700 1 -- 172.21.15.32:0/1441618153 shutdown_connections 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.502+0000 7f72085d1700 1 -- 172.21.15.32:0/1441618153 wait complete. 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 Processor -- start 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 -- start start 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f720009d630 0x7f72001c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000a0a30 0x7f72001c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7200133930 0x7f72001c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7200120a70 con 0x7f72000a0a30 2023-09-30T22:24:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7200120880 con 0x7f7200133930 2023-09-30T22:24:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f72085d1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7200121920 con 0x7f720009d630 2023-09-30T22:24:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f720636d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f720009d630 0x7f72001c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f720636d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f720009d630 0x7f72001c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59156/0 (socket says 172.21.15.32:59156) 2023-09-30T22:24:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f720636d700 1 -- 172.21.15.32:0/3393970061 learned_addr learned my addr 172.21.15.32:0/3393970061 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f7205b6c700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000a0a30 0x7f72001c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.503+0000 7f7206b6e700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7200133930 0x7f72001c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f7205b6c700 1 -- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f720009d630 msgr2=0x7f72001c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f7205b6c700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f720009d630 0x7f72001c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f7205b6c700 1 -- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7200133930 msgr2=0x7f72001c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f7205b6c700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7200133930 0x7f72001c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f7205b6c700 1 -- 172.21.15.32:0/3393970061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00094a0 con 0x7f72000a0a30 2023-09-30T22:24:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f7205b6c700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000a0a30 0x7f72001c8850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71f0016000 tx=0x7f71f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004400 con 0x7f72000a0a30 2023-09-30T22:24:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f001f070 con 0x7f72000a0a30 2023-09-30T22:24:51.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72000a40c0 con 0x7f72000a0a30 2023-09-30T22:24:51.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.504+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00038a0 con 0x7f72000a0a30 2023-09-30T22:24:51.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.505+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7200121e90 con 0x7f72000a0a30 2023-09-30T22:24:51.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.506+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f71f0021030 con 0x7f72000a0a30 2023-09-30T22:24:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.506+0000 7f720636d700 1 -- 172.21.15.32:0/3393970061 >> v1:172.21.15.32:6816/126553 conn(0x7f71e402f510 legacy=0x7f71e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.506+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f71f0004560 con 0x7f72000a0a30 2023-09-30T22:24:51.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.506+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72000008d0 con 0x7f72000a0a30 2023-09-30T22:24:51.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.512+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71f0015020 con 0x7f72000a0a30 2023-09-30T22:24:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 Processor -- start 2023-09-30T22:24:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 -- start start 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 0x7f1f1813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f18070ee0 con 0x7f1f1806e570 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f18071980 con 0x7f1f1809eca0 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.594+0000 7f1f1e895700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18071480 con 0x7f1f1806f050 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 0x7f1f1813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 0x7f1f1813cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57546/0 (socket says 172.21.15.32:57546) 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f177fe700 1 -- 172.21.15.32:0/2689557887 learned_addr learned my addr 172.21.15.32:0/2689557887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f17fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f1806e570 0x7f1f1806e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40216/0 (socket says 172.21.15.32:40216) 2023-09-30T22:24:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f1ce32700 1 --2- 172.21.15.32:0/2689557887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f1ce32700 1 -- 172.21.15.32:0/2689557887 >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 msgr2=0x7f1f1813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f1ce32700 1 --2- 172.21.15.32:0/2689557887 >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 0x7f1f1813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f1ce32700 1 -- 172.21.15.32:0/2689557887 >> v1:172.21.15.32:6789/0 conn(0x7f1f1806e570 legacy=0x7f1f1806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f1ce32700 1 -- 172.21.15.32:0/2689557887 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f18071b90 con 0x7f1f1806f050 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.595+0000 7f1f177fe700 1 --2- 172.21.15.32:0/2689557887 >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 0x7f1f1813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.596+0000 7f1f1ce32700 1 --2- 172.21.15.32:0/2689557887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813f3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009d90 tx=0x7f1f1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31dadcaf686e931 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.596+0000 7f1f16ffd700 1 -- 172.21.15.32:0/2689557887 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000e070 con 0x7f1f1806f050 2023-09-30T22:24:51.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.596+0000 7f1f16ffd700 1 -- 172.21.15.32:0/2689557887 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1000bb20 con 0x7f1f1806f050 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.596+0000 7f1f16ffd700 1 -- 172.21.15.32:0/2689557887 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10012430 con 0x7f1f1806f050 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 -- 172.21.15.32:0/2689557887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 msgr2=0x7f1f1813f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 --2- 172.21.15.32:0/2689557887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813f3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009d90 tx=0x7f1f1000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 -- 172.21.15.32:0/2689557887 shutdown_connections 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 --2- 172.21.15.32:0/2689557887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813f3f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 --2- 172.21.15.32:0/2689557887 >> v2:172.21.15.32:3300/0 conn(0x7f1f1809eca0 0x7f1f1813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 -- 172.21.15.32:0/2689557887 >> 172.21.15.32:0/2689557887 conn(0x7f1f180a2aa0 msgr2=0x7f1f180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 -- 172.21.15.32:0/2689557887 shutdown_connections 2023-09-30T22:24:51.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.597+0000 7f1f1e895700 1 -- 172.21.15.32:0/2689557887 wait complete. 2023-09-30T22:24:51.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.598+0000 7f1f1e895700 1 Processor -- start 2023-09-30T22:24:51.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.598+0000 7f1f1e895700 1 -- start start 2023-09-30T22:24:51.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1e895700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1806e570 0x7f1f1813cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1e895700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1e895700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 0x7f1f1813d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1e895700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1813ecd0 con 0x7f1f1806e570 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1e895700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1812ceb0 con 0x7f1f1806f050 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1e895700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18070610 con 0x7f1f181383a0 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 0x7f1f1813d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 0x7f1f1813d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59158/0 (socket says 172.21.15.32:59158) 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 -- 172.21.15.32:0/3225545792 learned_addr learned my addr 172.21.15.32:0/3225545792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f177fe700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f17fff700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1806e570 0x7f1f1813cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 -- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 msgr2=0x7f1f1813d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 -- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1806e570 msgr2=0x7f1f1813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1806e570 0x7f1f1813cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.599+0000 7f1f1ce32700 1 -- 172.21.15.32:0/3225545792 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c009550 con 0x7f1f181383a0 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.600+0000 7f1f1ce32700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 0x7f1f1813d670 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1f18071ac0 tx=0x7f1f10004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.600+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10012430 con 0x7f1f181383a0 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.600+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10020070 con 0x7f1f181383a0 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.600+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f10009a70 con 0x7f1f181383a0 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.600+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003ea0 con 0x7f1f181383a0 2023-09-30T22:24:51.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.600+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1812d560 con 0x7f1f181383a0 2023-09-30T22:24:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.601+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1812cab0 con 0x7f1f181383a0 2023-09-30T22:24:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.602+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1f10004040 con 0x7f1f181383a0 2023-09-30T22:24:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.602+0000 7f1f17fff700 1 -- 172.21.15.32:0/3225545792 >> v1:172.21.15.32:6816/126553 conn(0x7f1f0402f510 legacy=0x7f1f040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.602+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f1f10022070 con 0x7f1f181383a0 2023-09-30T22:24:51.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.607+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f10025740 con 0x7f1f181383a0 2023-09-30T22:24:51.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.644+0000 7fe228490700 1 Processor -- start 2023-09-30T22:24:51.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.644+0000 7fe228490700 1 -- start start 2023-09-30T22:24:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe228490700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe22009e1d0 0x7fe220138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe228490700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe228490700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22009b120 con 0x7fe22009d6f0 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe228490700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22006b460 con 0x7fe22009e1d0 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe228490700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe22009a7e0 con 0x7fe220139170 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe225a2b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe22009e1d0 0x7fe220138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59164/0 (socket says 172.21.15.32:59164) 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 -- 172.21.15.32:0/80106086 learned_addr learned my addr 172.21.15.32:0/80106086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 -- 172.21.15.32:0/80106086 >> v2:172.21.15.32:3300/0 conn(0x7fe22009e1d0 msgr2=0x7fe220138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 --2- 172.21.15.32:0/80106086 >> v2:172.21.15.32:3300/0 conn(0x7fe22009e1d0 0x7fe220138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 -- 172.21.15.32:0/80106086 >> v1:172.21.15.32:6789/0 conn(0x7fe22009d6f0 legacy=0x7fe22009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe226a2d700 1 -- 172.21.15.32:0/80106086 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22009a500 con 0x7fe220139170 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.645+0000 7fe225a2b700 1 --2- 172.21.15.32:0/80106086 >> v2:172.21.15.32:3300/0 conn(0x7fe22009e1d0 0x7fe220138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.646+0000 7fe226a2d700 1 --2- 172.21.15.32:0/80106086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22013b550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe218009e80 tx=0x7fe21800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915b2cb1ac3c2073 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.646+0000 7fe22522a700 1 -- 172.21.15.32:0/80106086 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800e070 con 0x7fe220139170 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.646+0000 7fe22522a700 1 -- 172.21.15.32:0/80106086 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21800b8d0 con 0x7fe220139170 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.646+0000 7fe22522a700 1 -- 172.21.15.32:0/80106086 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800bc80 con 0x7fe220139170 2023-09-30T22:24:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7ff3b659c700 1 -- 172.21.15.32:0/495167944 >> v1:172.21.15.32:6816/126553 conn(0x7ff39802f510 legacy=0x7ff3980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 -- 172.21.15.32:0/80106086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 msgr2=0x7fe22013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 --2- 172.21.15.32:0/80106086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22013b550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe218009e80 tx=0x7fe21800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 -- 172.21.15.32:0/80106086 shutdown_connections 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 --2- 172.21.15.32:0/80106086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22013b550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 --2- 172.21.15.32:0/80106086 >> v2:172.21.15.32:3300/0 conn(0x7fe22009e1d0 0x7fe220138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 -- 172.21.15.32:0/80106086 >> 172.21.15.32:0/80106086 conn(0x7fe220066f50 msgr2=0x7fe22009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 -- 172.21.15.32:0/80106086 shutdown_connections 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.647+0000 7fe228490700 1 -- 172.21.15.32:0/80106086 wait complete. 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 Processor -- start 2023-09-30T22:24:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 -- start start 2023-09-30T22:24:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe22009d6f0 0x7fe22011e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe22009e1d0 0x7fe22011ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22011f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220121e90 con 0x7fe22009d6f0 2023-09-30T22:24:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22009c710 con 0x7fe22009e1d0 2023-09-30T22:24:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.649+0000 7fe228490700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe220120550 con 0x7fe220139170 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22011f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22011f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59166/0 (socket says 172.21.15.32:59166) 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 -- 172.21.15.32:0/2408113305 learned_addr learned my addr 172.21.15.32:0/2408113305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe22622c700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe22009d6f0 0x7fe22011e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 -- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe22009e1d0 msgr2=0x7fe22011ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe22009e1d0 0x7fe22011ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 -- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe22009d6f0 msgr2=0x7fe22011e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe22009d6f0 0x7fe22011e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 -- 172.21.15.32:0/2408113305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100094a0 con 0x7fe220139170 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe226a2d700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22011f340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe22009a920 tx=0x7fe218004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2180048b0 con 0x7fe220139170 2023-09-30T22:24:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.650+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2180092e0 con 0x7fe220139170 2023-09-30T22:24:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.651+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21801c660 con 0x7fe220139170 2023-09-30T22:24:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.651+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe218009b30 con 0x7fe220139170 2023-09-30T22:24:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.651+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe220122070 con 0x7fe220139170 2023-09-30T22:24:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.652+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe21801c7c0 con 0x7fe220139170 2023-09-30T22:24:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.652+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe220074ea0 con 0x7fe220139170 2023-09-30T22:24:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.652+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fe2180416d0 con 0x7fe220139170 2023-09-30T22:24:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.652+0000 7fe22622c700 1 -- 172.21.15.32:0/2408113305 >> v1:172.21.15.32:6816/126553 conn(0x7fe20402f510 legacy=0x7fe2040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.661+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe21801a330 con 0x7fe220139170 2023-09-30T22:24:51.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.678+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff3b80747f0 con 0x7ff3b80a0a30 2023-09-30T22:24:51.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.679+0000 7ff3aeffd700 1 -- 172.21.15.32:0/495167944 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3a0004560 con 0x7ff3b80a0a30 2023-09-30T22:24:51.679 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 >> v1:172.21.15.32:6816/126553 conn(0x7ff39802f510 legacy=0x7ff3980319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 msgr2=0x7ff3b811f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b811f970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0016000 tx=0x7ff3a0004750 comp rx=0 tx=0).stop 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 shutdown_connections 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3b8133930 0x7ff3b81c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3b809d630 0x7ff3b811f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 --2- 172.21.15.32:0/495167944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3b80a0a30 0x7ff3b811f970 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 >> 172.21.15.32:0/495167944 conn(0x7ff3b80a3050 msgr2=0x7ff3b81bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 shutdown_connections 2023-09-30T22:24:51.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.681+0000 7ff3bc9d7700 1 -- 172.21.15.32:0/495167944 wait complete. 2023-09-30T22:24:51.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:51.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.707+0000 7f720636d700 1 -- 172.21.15.32:0/3393970061 >> v1:172.21.15.32:6816/126553 conn(0x7f71e402f510 legacy=0x7f71e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.740+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7200072980 con 0x7f72000a0a30 2023-09-30T22:24:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.741+0000 7f71f6ffd700 1 -- 172.21.15.32:0/3393970061 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71f0046060 con 0x7f72000a0a30 2023-09-30T22:24:51.742 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 >> v1:172.21.15.32:6816/126553 conn(0x7f71e402f510 legacy=0x7f71e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000a0a30 msgr2=0x7f72001c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000a0a30 0x7f72001c8850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71f0016000 tx=0x7f71f0004750 comp rx=0 tx=0).stop 2023-09-30T22:24:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 shutdown_connections 2023-09-30T22:24:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7200133930 0x7f72001c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f720009d630 0x7f72001c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 --2- 172.21.15.32:0/3393970061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72000a0a30 0x7f72001c8850 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 >> 172.21.15.32:0/3393970061 conn(0x7f72000a3050 msgr2=0x7f72001bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 shutdown_connections 2023-09-30T22:24:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.743+0000 7f72085d1700 1 -- 172.21.15.32:0/3393970061 wait complete. 2023-09-30T22:24:51.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:51.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.803+0000 7f1f17fff700 1 -- 172.21.15.32:0/3225545792 >> v1:172.21.15.32:6816/126553 conn(0x7f1f0402f510 legacy=0x7f1f040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.825+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1f18074ea0 con 0x7f1f181383a0 2023-09-30T22:24:51.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.825+0000 7f1f14ff9700 1 -- 172.21.15.32:0/3225545792 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f10025380 con 0x7f1f181383a0 2023-09-30T22:24:51.826 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:51.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 >> v1:172.21.15.32:6816/126553 conn(0x7f1f0402f510 legacy=0x7f1f040319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:51.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 msgr2=0x7f1f1813d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 0x7f1f1813d670 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1f18071ac0 tx=0x7f1f10004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 shutdown_connections 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f181383a0 0x7f1f1813d670 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f1806e570 0x7f1f1813cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 --2- 172.21.15.32:0/3225545792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f1806f050 0x7f1f1813d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.827+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 >> 172.21.15.32:0/3225545792 conn(0x7f1f180a2aa0 msgr2=0x7f1f1809cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.828+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 shutdown_connections 2023-09-30T22:24:51.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.828+0000 7f1f1e895700 1 -- 172.21.15.32:0/3225545792 wait complete. 2023-09-30T22:24:51.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:51.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.853+0000 7fe22622c700 1 -- 172.21.15.32:0/2408113305 >> v1:172.21.15.32:6816/126553 conn(0x7fe20402f510 legacy=0x7fe2040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.872+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe22009f420 con 0x7fe220139170 2023-09-30T22:24:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.872+0000 7fe216ffd700 1 -- 172.21.15.32:0/2408113305 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe218024ba0 con 0x7fe220139170 2023-09-30T22:24:51.873 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 >> v1:172.21.15.32:6816/126553 conn(0x7fe20402f510 legacy=0x7fe2040319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 msgr2=0x7fe22011f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22011f340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe22009a920 tx=0x7fe218004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 shutdown_connections 2023-09-30T22:24:51.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe220139170 0x7fe22011f340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe22009d6f0 0x7fe22011e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 --2- 172.21.15.32:0/2408113305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe22009e1d0 0x7fe22011ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 >> 172.21.15.32:0/2408113305 conn(0x7fe220066f50 msgr2=0x7fe22009b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 shutdown_connections 2023-09-30T22:24:51.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:51.875+0000 7fe228490700 1 -- 172.21.15.32:0/2408113305 wait complete. 2023-09-30T22:24:51.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:52.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:52.758 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:52.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.844+0000 7f8f4f718700 1 Processor -- start 2023-09-30T22:24:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.844+0000 7f8f4f718700 1 -- start start 2023-09-30T22:24:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4f718700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f4809d630 0x7f8f480a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4f718700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f48138d80 0x7f8f4813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4f718700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480a2a70 con 0x7f8f480a0a30 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4f718700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480a01d0 con 0x7f8f4809d630 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4f718700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4809e2b0 con 0x7f8f48138d80 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4ccb3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f4809d630 0x7f8f480a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4d4b4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f480a0a30 0x7f8f480a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40236/0 (socket says 172.21.15.32:40236) 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4d4b4700 1 -- 172.21.15.32:0/2947745298 learned_addr learned my addr 172.21.15.32:0/2947745298 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4dcb5700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f48138d80 0x7f8f4813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4ccb3700 1 -- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f48138d80 msgr2=0x7f8f4813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4ccb3700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f48138d80 0x7f8f4813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4ccb3700 1 -- 172.21.15.32:0/2947745298 >> v1:172.21.15.32:6789/0 conn(0x7f8f480a0a30 legacy=0x7f8f480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f3ffff700 1 -- 172.21.15.32:0/2947745298 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119441496 0 0) 0x7f8f480a2a70 con 0x7f8f480a0a30 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4ccb3700 1 -- 172.21.15.32:0/2947745298 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4809dc50 con 0x7f8f4809d630 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4dcb5700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f48138d80 0x7f8f4813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.845+0000 7f8f4ccb3700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4809d630 0x7f8f480a1510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009840 tx=0x7f8f38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7895e2568e277470 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f3ffff700 1 -- 172.21.15.32:0/2947745298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38011830 con 0x7f8f4809d630 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f3ffff700 1 -- 172.21.15.32:0/2947745298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38011990 con 0x7f8f4809d630 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f3ffff700 1 -- 172.21.15.32:0/2947745298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38011d20 con 0x7f8f4809d630 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f4f718700 1 -- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4809d630 msgr2=0x7f8f480a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f4f718700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4809d630 0x7f8f480a1510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009840 tx=0x7f8f38009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f4f718700 1 -- 172.21.15.32:0/2947745298 shutdown_connections 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f4f718700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f48138d80 0x7f8f4813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.846+0000 7f8f4f718700 1 --2- 172.21.15.32:0/2947745298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4809d630 0x7f8f480a1510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.847+0000 7f8f4f718700 1 -- 172.21.15.32:0/2947745298 >> 172.21.15.32:0/2947745298 conn(0x7f8f480a3050 msgr2=0x7f8f480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:52.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.847+0000 7f8f4f718700 1 -- 172.21.15.32:0/2947745298 shutdown_connections 2023-09-30T22:24:52.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.847+0000 7f8f4f718700 1 -- 172.21.15.32:0/2947745298 wait complete. 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 Processor -- start 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 -- start start 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4809d630 0x7f8f48133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f480a0a30 0x7f8f48133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 0x7f8f48134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48134bd0 con 0x7f8f48138d80 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48136190 con 0x7f8f480a0a30 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4f718700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48136eb0 con 0x7f8f4809d630 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4dcb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 0x7f8f48134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4d4b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4809d630 0x7f8f48133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4dcb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 0x7f8f48134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57606/0 (socket says 172.21.15.32:57606) 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4dcb5700 1 -- 172.21.15.32:0/1465655743 learned_addr learned my addr 172.21.15.32:0/1465655743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.848+0000 7f8f4ccb3700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f480a0a30 0x7f8f48133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4ccb3700 1 -- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4809d630 msgr2=0x7f8f48133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4ccb3700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4809d630 0x7f8f48133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4ccb3700 1 -- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 msgr2=0x7f8f48134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4ccb3700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 0x7f8f48134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4ccb3700 1 -- 172.21.15.32:0/1465655743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380094a0 con 0x7f8f480a0a30 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4dcb5700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 0x7f8f48134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4ccb3700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f480a0a30 0x7f8f48133c40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f480a1fb0 tx=0x7f8f38004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004400 con 0x7f8f480a0a30 2023-09-30T22:24:52.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3801f070 con 0x7f8f480a0a30 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f480a4ba0 con 0x7f8f480a0a30 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.849+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380038c0 con 0x7f8f480a0a30 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.850+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f48137920 con 0x7f8f480a0a30 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.851+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8f38021030 con 0x7f8f480a0a30 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.851+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f8f38040c40 con 0x7f8f480a0a30 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.851+0000 7f8f4d4b4700 1 -- 172.21.15.32:0/1465655743 >> v1:172.21.15.32:6816/126553 conn(0x7f8f2c02f510 legacy=0x7f8f2c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.851+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f480747f0 con 0x7f8f480a0a30 2023-09-30T22:24:52.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.856+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f38015020 con 0x7f8f480a0a30 2023-09-30T22:24:52.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.926+0000 7fc7b6910700 1 Processor -- start 2023-09-30T22:24:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.926+0000 7fc7b6910700 1 -- start start 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7b6910700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7b6910700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7b6910700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b006bf30 con 0x7fc7b009f5b0 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7b6910700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b009ffb0 con 0x7fc7b009e850 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7b6910700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b006cb10 con 0x7fc7b009f160 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7affff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7affff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57608/0 (socket says 172.21.15.32:57608) 2023-09-30T22:24:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.927+0000 7fc7affff700 1 -- 172.21.15.32:0/1641524105 learned_addr learned my addr 172.21.15.32:0/1641524105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7af7fe700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7affff700 1 -- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f160 msgr2=0x7fc7b0133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7affff700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7affff700 1 -- 172.21.15.32:0/1641524105 >> v1:172.21.15.32:6789/0 conn(0x7fc7b009f5b0 legacy=0x7fc7b0134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7aeffd700 1 -- 172.21.15.32:0/1641524105 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866859991 0 0) 0x7fc7b006bf30 con 0x7fc7b009f5b0 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7affff700 1 -- 172.21.15.32:0/1641524105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00a0270 con 0x7fc7b009e850 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7af7fe700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7affff700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009ec20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098f0 tx=0x7fc7a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbfe12709af0e8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7aeffd700 1 -- 172.21.15.32:0/1641524105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000b8a0 con 0x7fc7b009e850 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7aeffd700 1 -- 172.21.15.32:0/1641524105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000ba00 con 0x7fc7b009e850 2023-09-30T22:24:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.928+0000 7fc7aeffd700 1 -- 172.21.15.32:0/1641524105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000bd90 con 0x7fc7b009e850 2023-09-30T22:24:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.929+0000 7fc7b6910700 1 -- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 msgr2=0x7fc7b009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.929+0000 7fc7b6910700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009ec20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00098f0 tx=0x7fc7a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.929+0000 7fc7b6910700 1 -- 172.21.15.32:0/1641524105 shutdown_connections 2023-09-30T22:24:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.929+0000 7fc7b6910700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009ec20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.930+0000 7fc7b6910700 1 --2- 172.21.15.32:0/1641524105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f160 0x7fc7b0133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.930+0000 7fc7b6910700 1 -- 172.21.15.32:0/1641524105 >> 172.21.15.32:0/1641524105 conn(0x7fc7b00a3f70 msgr2=0x7fc7b00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.930+0000 7fc7b6910700 1 -- 172.21.15.32:0/1641524105 shutdown_connections 2023-09-30T22:24:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.930+0000 7fc7b6910700 1 -- 172.21.15.32:0/1641524105 wait complete. 2023-09-30T22:24:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.931+0000 7fc7b6910700 1 Processor -- start 2023-09-30T22:24:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.931+0000 7fc7b6910700 1 -- start start 2023-09-30T22:24:52.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b6910700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b6910700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7b009f160 0x7fc7b009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b6910700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f5b0 0x7fc7b009cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b6910700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b01b61c0 con 0x7fc7b009e850 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b6910700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b006f930 con 0x7fc7b009f5b0 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b6910700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b006cd30 con 0x7fc7b009f160 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7affff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57616/0 (socket says 172.21.15.32:57616) 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7affff700 1 -- 172.21.15.32:0/4161350889 learned_addr learned my addr 172.21.15.32:0/4161350889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.932+0000 7fc7b4ead700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f5b0 0x7fc7b009cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7af7fe700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7b009f160 0x7fc7b009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b4ead700 1 -- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7b009f160 msgr2=0x7fc7b009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b4ead700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7b009f160 0x7fc7b009ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b4ead700 1 -- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 msgr2=0x7fc7b009c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b4ead700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b4ead700 1 -- 172.21.15.32:0/4161350889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0009550 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7af7fe700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7b009f160 0x7fc7b009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b4ead700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f5b0 0x7fc7b009cfd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c7c0 tx=0x7fc7a800cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8003e30 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8005070 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b009e200 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8005400 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.933+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b01a59c0 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.934+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc7a8005560 con 0x7fc7b009f5b0 2023-09-30T22:24:52.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.935+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7fc7a803f210 con 0x7fc7b009f5b0 2023-09-30T22:24:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.935+0000 7fc7affff700 1 -- 172.21.15.32:0/4161350889 >> v1:172.21.15.32:6816/126553 conn(0x7fc79c02f510 legacy=0x7fc79c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:52.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.935+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b00747f0 con 0x7fc7b009f5b0 2023-09-30T22:24:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.940+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7a8017080 con 0x7fc7b009f5b0 2023-09-30T22:24:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.997+0000 7f0273642700 1 Processor -- start 2023-09-30T22:24:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.997+0000 7f0273642700 1 -- start start 2023-09-30T22:24:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0273642700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c0a0a30 0x7f026c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0273642700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f026c09d630 0x7f026c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0273642700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c0a2a70 con 0x7f026c133930 2023-09-30T22:24:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0273642700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c0a01d0 con 0x7f026c09d630 2023-09-30T22:24:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0273642700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f026c09e2b0 con 0x7f026c0a0a30 2023-09-30T22:24:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0270bdd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f026c09d630 0x7f026c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0270bdd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f026c09d630 0x7f026c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57624/0 (socket says 172.21.15.32:57624) 2023-09-30T22:24:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.998+0000 7f0270bdd700 1 -- 172.21.15.32:0/1119517079 learned_addr learned my addr 172.21.15.32:0/1119517079 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f02713de700 1 --2- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c0a0a30 0x7f026c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0270bdd700 1 -- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c0a0a30 msgr2=0x7f026c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0270bdd700 1 --2- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c0a0a30 0x7f026c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0270bdd700 1 -- 172.21.15.32:0/1119517079 >> v1:172.21.15.32:6789/0 conn(0x7f026c133930 legacy=0x7f026c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0270bdd700 1 -- 172.21.15.32:0/1119517079 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c09dc50 con 0x7f026c09d630 2023-09-30T22:24:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0270bdd700 1 --2- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c09d630 0x7f026c0a1340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f025c0098f0 tx=0x7f025c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7ed04b6b424c03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0263fff700 1 -- 172.21.15.32:0/1119517079 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c011830 con 0x7f026c09d630 2023-09-30T22:24:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0263fff700 1 -- 172.21.15.32:0/1119517079 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c011990 con 0x7f026c09d630 2023-09-30T22:24:53.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:52.999+0000 7f0263fff700 1 -- 172.21.15.32:0/1119517079 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c011d20 con 0x7f026c09d630 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.000+0000 7f0273642700 1 -- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c09d630 msgr2=0x7f026c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.000+0000 7f0273642700 1 --2- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c09d630 0x7f026c0a1340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f025c0098f0 tx=0x7f025c009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.001+0000 7f0273642700 1 -- 172.21.15.32:0/1119517079 shutdown_connections 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.001+0000 7f0273642700 1 --2- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c0a0a30 0x7f026c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.001+0000 7f0273642700 1 --2- 172.21.15.32:0/1119517079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c09d630 0x7f026c0a1340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.001+0000 7f0273642700 1 -- 172.21.15.32:0/1119517079 >> 172.21.15.32:0/1119517079 conn(0x7f026c0a3050 msgr2=0x7f026c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.001+0000 7f0273642700 1 -- 172.21.15.32:0/1119517079 shutdown_connections 2023-09-30T22:24:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.001+0000 7f0273642700 1 -- 172.21.15.32:0/1119517079 wait complete. 2023-09-30T22:24:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 Processor -- start 2023-09-30T22:24:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 -- start start 2023-09-30T22:24:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 0x7f026c1c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c0a0a30 0x7f026c1c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026c133930 0x7f026c1c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c120a70 con 0x7f026c0a0a30 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c120880 con 0x7f026c133930 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.002+0000 7f0273642700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f026c121920 con 0x7f026c09d630 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 0x7f026c1c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 0x7f026c1c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59206/0 (socket says 172.21.15.32:59206) 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 -- 172.21.15.32:0/3585066817 learned_addr learned my addr 172.21.15.32:0/3585066817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f0270bdd700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c0a0a30 0x7f026c1c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 -- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026c133930 msgr2=0x7f026c1c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026c133930 0x7f026c1c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 -- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c0a0a30 msgr2=0x7f026c1c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c0a0a30 0x7f026c1c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 -- 172.21.15.32:0/3585066817 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009550 con 0x7f026c09d630 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f0270bdd700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c0a0a30 0x7f026c1c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f02713de700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 0x7f026c1c8310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f026800e9b0 tx=0x7f026800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.003+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268009710 con 0x7f026c09d630 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.004+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026800c850 con 0x7f026c09d630 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.004+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c0a40c0 con 0x7f026c09d630 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.004+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800cbc0 con 0x7f026c09d630 2023-09-30T22:24:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.004+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c121e90 con 0x7f026c09d630 2023-09-30T22:24:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.005+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c0008d0 con 0x7f026c09d630 2023-09-30T22:24:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.005+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f026800cd20 con 0x7f026c09d630 2023-09-30T22:24:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.005+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4063+0+0 (secure 0 0 0) 0x7f026803fc20 con 0x7f026c09d630 2023-09-30T22:24:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.005+0000 7f0270bdd700 1 -- 172.21.15.32:0/3585066817 >> v1:172.21.15.32:6816/126553 conn(0x7f025002f510 legacy=0x7f02500319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:53.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.014+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f026801bc30 con 0x7f026c09d630 2023-09-30T22:24:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.052+0000 7f8f4d4b4700 1 -- 172.21.15.32:0/1465655743 >> v1:172.21.15.32:6816/126553 conn(0x7f8f2c02f510 legacy=0x7f8f2c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.32:6816/126553 2023-09-30T22:24:53.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.059+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0268013940 con 0x7f026c09d630 2023-09-30T22:24:53.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.059+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 >> v1:172.21.15.32:6816/126553 conn(0x7f025002f510 legacy=0x7f02500319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:53.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.060+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f38019cd0 con 0x7f8f480a0a30 2023-09-30T22:24:53.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.060+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 >> v1:172.21.15.32:6816/126553 conn(0x7f8f2c02f510 legacy=0x7f8f2c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.061+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7a801b8d0 con 0x7fc7b009f5b0 2023-09-30T22:24:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.061+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 >> v1:172.21.15.32:6816/126553 conn(0x7fc79c02f510 legacy=0x7fc79c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:24:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.062+0000 7f030c9c4700 1 Processor -- start 2023-09-30T22:24:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.062+0000 7f030c9c4700 1 -- start start 2023-09-30T22:24:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.064+0000 7f030c9c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809d6f0 0x7f030809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.064+0000 7f030c9c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f0308130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.064+0000 7f030c9c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030809f130 con 0x7f030809e000 2023-09-30T22:24:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.064+0000 7f030c9c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030809f520 con 0x7f030809d6f0 2023-09-30T22:24:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.064+0000 7f030c9c4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f030809e930 con 0x7f030809e410 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f0308130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f0308130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59214/0 (socket says 172.21.15.32:59214) 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 -- 172.21.15.32:0/3352208964 learned_addr learned my addr 172.21.15.32:0/3352208964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 -- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809d6f0 msgr2=0x7f030809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 --2- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809d6f0 0x7f030809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 -- 172.21.15.32:0/3352208964 >> v1:172.21.15.32:6789/0 conn(0x7f030809e000 legacy=0x7f0308130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.065+0000 7f0306d9d700 1 -- 172.21.15.32:0/3352208964 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030809b2d0 con 0x7f030809e410 2023-09-30T22:24:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.066+0000 7f0306d9d700 1 --2- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f0308130d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009920 tx=0x7f02f8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803cc6752b218be7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.068+0000 7f030559a700 1 -- 172.21.15.32:0/3352208964 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004230 con 0x7f030809e410 2023-09-30T22:24:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.068+0000 7f030559a700 1 -- 172.21.15.32:0/3352208964 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8004390 con 0x7f030809e410 2023-09-30T22:24:53.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.068+0000 7f030559a700 1 -- 172.21.15.32:0/3352208964 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8009e70 con 0x7f030809e410 2023-09-30T22:24:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.069+0000 7f030c9c4700 1 -- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 msgr2=0x7f0308130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.069+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f0308130d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009920 tx=0x7f02f8009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:53.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.069+0000 7f030c9c4700 1 -- 172.21.15.32:0/3352208964 shutdown_connections 2023-09-30T22:24:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.069+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f0308130d70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.069+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3352208964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809d6f0 0x7f030809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.069+0000 7f030c9c4700 1 -- 172.21.15.32:0/3352208964 >> 172.21.15.32:0/3352208964 conn(0x7f0308066f50 msgr2=0x7f030809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:53.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.070+0000 7f030c9c4700 1 -- 172.21.15.32:0/3352208964 shutdown_connections 2023-09-30T22:24:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.070+0000 7f030c9c4700 1 -- 172.21.15.32:0/3352208964 wait complete. 2023-09-30T22:24:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.071+0000 7f030c9c4700 1 Processor -- start 2023-09-30T22:24:53.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.072+0000 7f030c9c4700 1 -- start start 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.072+0000 7f030c9c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030809d6f0 0x7f030812ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f030c9c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809e000 0x7f030812f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f030c9c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f030812f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f030c9c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0308132aa0 con 0x7f030809d6f0 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f030c9c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0308132500 con 0x7f030809e000 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f030c9c4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0308132fa0 con 0x7f030809e410 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f030812f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:53.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f030812f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59220/0 (socket says 172.21.15.32:59220) 2023-09-30T22:24:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 -- 172.21.15.32:0/3686052441 learned_addr learned my addr 172.21.15.32:0/3686052441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 -- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809e000 msgr2=0x7f030812f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809e000 0x7f030812f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 -- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030809d6f0 msgr2=0x7f030812ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.073+0000 7f0306d9d700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030809d6f0 0x7f030812ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.074+0000 7f0306d9d700 1 -- 172.21.15.32:0/3686052441 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f8009550 con 0x7f030809e410 2023-09-30T22:24:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.074+0000 7f0306d9d700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f030812f670 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02f80098f0 tx=0x7f02f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.075+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f800bed0 con 0x7f030809e410 2023-09-30T22:24:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.075+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03081327c0 con 0x7f030809e410 2023-09-30T22:24:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.075+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030813b470 con 0x7f030809e410 2023-09-30T22:24:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.076+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f80036a0 con 0x7f030809e410 2023-09-30T22:24:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.076+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8003a30 con 0x7f030809e410 2023-09-30T22:24:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.077+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02f8022070 con 0x7f030809e410 2023-09-30T22:24:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.077+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f02f8041460 con 0x7f030809e410 2023-09-30T22:24:53.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.077+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec0052e0 con 0x7f030809e410 2023-09-30T22:24:53.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.085+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02f801b970 con 0x7f030809e410 2023-09-30T22:24:53.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.117+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f48072980 con 0x7f8f480a0a30 2023-09-30T22:24:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.117+0000 7f8f3dffb700 1 -- 172.21.15.32:0/1465655743 <== mon.1 v2:172.21.15.32:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f3801a450 con 0x7f8f480a0a30 2023-09-30T22:24:53.118 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f480a0a30 msgr2=0x7f8f48133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f480a0a30 0x7f8f48133c40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f480a1fb0 tx=0x7f8f38004750 comp rx=0 tx=0).stop 2023-09-30T22:24:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 shutdown_connections 2023-09-30T22:24:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f48138d80 0x7f8f48134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4809d630 0x7f8f48133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 --2- 172.21.15.32:0/1465655743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f480a0a30 0x7f8f48133c40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 >> 172.21.15.32:0/1465655743 conn(0x7f8f480a3050 msgr2=0x7f8f4809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 shutdown_connections 2023-09-30T22:24:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.121+0000 7f8f4f718700 1 -- 172.21.15.32:0/1465655743 wait complete. 2023-09-30T22:24:53.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:53.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.202+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc7b006bbb0 con 0x7fc7b009f5b0 2023-09-30T22:24:53.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.202+0000 7fc7acff9700 1 -- 172.21.15.32:0/4161350889 <== mon.1 v2:172.21.15.32:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7a800a930 con 0x7fc7b009f5b0 2023-09-30T22:24:53.203 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f5b0 msgr2=0x7fc7b009cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f5b0 0x7fc7b009cfd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c7c0 tx=0x7fc7a800cad0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 shutdown_connections 2023-09-30T22:24:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc7b009f5b0 0x7fc7b009cfd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7b009e850 0x7fc7b009c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 --2- 172.21.15.32:0/4161350889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7b009f160 0x7fc7b009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 >> 172.21.15.32:0/4161350889 conn(0x7fc7b00a3f70 msgr2=0x7fc7b01bbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 shutdown_connections 2023-09-30T22:24:53.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.204+0000 7fc7b6910700 1 -- 172.21.15.32:0/4161350889 wait complete. 2023-09-30T22:24:53.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.254+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f026c072980 con 0x7f026c09d630 2023-09-30T22:24:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.255+0000 7f0261ffb700 1 -- 172.21.15.32:0/3585066817 <== mon.2 v2:172.21.15.32:3302/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0268017090 con 0x7f026c09d630 2023-09-30T22:24:53.255 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 msgr2=0x7f026c1c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 0x7f026c1c8310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f026800e9b0 tx=0x7f026800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 shutdown_connections 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026c133930 0x7f026c1c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026c09d630 0x7f026c1c8310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 --2- 172.21.15.32:0/3585066817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026c0a0a30 0x7f026c1c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 >> 172.21.15.32:0/3585066817 conn(0x7f026c0a3050 msgr2=0x7f026c1bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 shutdown_connections 2023-09-30T22:24:53.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.257+0000 7f0273642700 1 -- 172.21.15.32:0/3585066817 wait complete. 2023-09-30T22:24:53.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:53.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.334+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02ec005590 con 0x7f030809e410 2023-09-30T22:24:53.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.334+0000 7f02f6ffd700 1 -- 172.21.15.32:0/3686052441 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02f8019ba0 con 0x7f030809e410 2023-09-30T22:24:53.335 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 msgr2=0x7f030812f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f030812f670 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02f80098f0 tx=0x7f02f8004750 comp rx=0 tx=0).stop 2023-09-30T22:24:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 shutdown_connections 2023-09-30T22:24:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f030809e410 0x7f030812f670 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f030809d6f0 0x7f030812ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 --2- 172.21.15.32:0/3686052441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f030809e000 0x7f030812f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:53.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 >> 172.21.15.32:0/3686052441 conn(0x7f0308066f50 msgr2=0x7f03080a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:53.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 shutdown_connections 2023-09-30T22:24:53.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:53.337+0000 7f030c9c4700 1 -- 172.21.15.32:0/3686052441 wait complete. 2023-09-30T22:24:53.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:54.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:54.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:54.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.293+0000 7f5c00963700 1 Processor -- start 2023-09-30T22:24:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.293+0000 7f5c00963700 1 -- start start 2023-09-30T22:24:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.293+0000 7f5c00963700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bfc09d6f0 0x7f5bfc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5c00963700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09e000 0x7f5bfc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5c00963700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc09b120 con 0x7f5bfc09e410 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5c00963700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bfc06b460 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5c00963700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc09a7e0 con 0x7f5bfc09e000 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bfad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bfc09e410 0x7f5bfc133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40260/0 (socket says 172.21.15.32:40260) 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bfad9d700 1 -- 172.21.15.32:0/1944588993 learned_addr learned my addr 172.21.15.32:0/1944588993 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf9d9b700 1 --2- 172.21.15.32:0/1944588993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09e000 0x7f5bfc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bfa59c700 1 --2- 172.21.15.32:0/1944588993 >> v2:172.21.15.32:3300/0 conn(0x7f5bfc09d6f0 0x7f5bfc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf959a700 1 -- 172.21.15.32:0/1944588993 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625148298 0 0) 0x7f5bfc09b120 con 0x7f5bfc09e410 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf959a700 1 -- 172.21.15.32:0/1944588993 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be40035b0 con 0x7f5bfc09e410 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf9d9b700 1 -- 172.21.15.32:0/1944588993 >> v2:172.21.15.32:3300/0 conn(0x7f5bfc09d6f0 msgr2=0x7f5bfc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf9d9b700 1 --2- 172.21.15.32:0/1944588993 >> v2:172.21.15.32:3300/0 conn(0x7f5bfc09d6f0 0x7f5bfc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf9d9b700 1 -- 172.21.15.32:0/1944588993 >> v1:172.21.15.32:6789/0 conn(0x7f5bfc09e410 legacy=0x7f5bfc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bf9d9b700 1 -- 172.21.15.32:0/1944588993 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc09a500 con 0x7f5bfc09e000 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.294+0000 7f5bfa59c700 1 --2- 172.21.15.32:0/1944588993 >> v2:172.21.15.32:3300/0 conn(0x7f5bfc09d6f0 0x7f5bfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.295+0000 7f5bf9d9b700 1 --2- 172.21.15.32:0/1944588993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09e000 0x7f5bfc1339f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009870 tx=0x7f5bf0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd94e653ff0820 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.295+0000 7f5bf959a700 1 -- 172.21.15.32:0/1944588993 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0011830 con 0x7f5bfc09e000 2023-09-30T22:24:54.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.295+0000 7f5bf959a700 1 -- 172.21.15.32:0/1944588993 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0011990 con 0x7f5bfc09e000 2023-09-30T22:24:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.295+0000 7f5bf959a700 1 -- 172.21.15.32:0/1944588993 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0011d00 con 0x7f5bfc09e000 2023-09-30T22:24:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.295+0000 7f5c00963700 1 -- 172.21.15.32:0/1944588993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09e000 msgr2=0x7f5bfc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.295+0000 7f5c00963700 1 --2- 172.21.15.32:0/1944588993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09e000 0x7f5bfc1339f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009870 tx=0x7f5bf0009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.296+0000 7f5c00963700 1 -- 172.21.15.32:0/1944588993 shutdown_connections 2023-09-30T22:24:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.296+0000 7f5c00963700 1 --2- 172.21.15.32:0/1944588993 >> v2:172.21.15.32:3300/0 conn(0x7f5bfc09d6f0 0x7f5bfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.296+0000 7f5c00963700 1 --2- 172.21.15.32:0/1944588993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09e000 0x7f5bfc1339f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.296+0000 7f5c00963700 1 -- 172.21.15.32:0/1944588993 >> 172.21.15.32:0/1944588993 conn(0x7f5bfc066f50 msgr2=0x7f5bfc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.296+0000 7f5c00963700 1 -- 172.21.15.32:0/1944588993 shutdown_connections 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.296+0000 7f5c00963700 1 -- 172.21.15.32:0/1944588993 wait complete. 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 Processor -- start 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 -- start start 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 0x7f5bfc1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bfc09e000 0x7f5bfc1c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bfc09e410 0x7f5bfc1c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a65a0 con 0x7f5bfc09e410 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a63b0 con 0x7f5bfc09e000 2023-09-30T22:24:54.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.297+0000 7f5c00963700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a82c0 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 0x7f5bfc1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 0x7f5bfc1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59232/0 (socket says 172.21.15.32:59232) 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 -- 172.21.15.32:0/699788145 learned_addr learned my addr 172.21.15.32:0/699788145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bf9d9b700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bfc09e000 0x7f5bfc1c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 -- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bfc09e000 msgr2=0x7f5bfc1c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bfc09e000 0x7f5bfc1c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 -- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bfc09e410 msgr2=0x7f5bfc1c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bfc09e410 0x7f5bfc1c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 -- 172.21.15.32:0/699788145 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0094a0 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bf9d9b700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bfc09e000 0x7f5bfc1c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5bfa59c700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 0x7f5bfc1c8040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00ebe0 tx=0x7f5bec00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.298+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec009800 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.299+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00c910 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.299+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00cc80 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.299+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf00094a0 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.299+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc09c710 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.300+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be00052e0 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.300+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5bec013470 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.300+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5bec040020 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.305+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5bec013720 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 Processor -- start 2023-09-30T22:24:54.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 -- start start 2023-09-30T22:24:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fb4136710 0x7f1fb4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4139030 0x7f1fb413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb406ef30 con 0x7f1fb41338a0 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb406fef0 con 0x7f1fb4136710 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.368+0000 7f1fbad9f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41334b0 con 0x7f1fb4139030 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fb4136710 0x7f1fb4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fb4136710 0x7f1fb4138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57670/0 (socket says 172.21.15.32:57670) 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 -- 172.21.15.32:0/604396712 learned_addr learned my addr 172.21.15.32:0/604396712 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb8b3b700 1 --1- 172.21.15.32:0/604396712 >> v1:172.21.15.32:6789/0 conn(0x7f1fb41338a0 0x7f1fb4133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40270/0 (socket says 172.21.15.32:40270) 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb933c700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4139030 0x7f1fb413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 -- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4139030 msgr2=0x7f1fb413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4139030 0x7f1fb413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 -- 172.21.15.32:0/604396712 >> v1:172.21.15.32:6789/0 conn(0x7f1fb41338a0 legacy=0x7f1fb4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:54.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 -- 172.21.15.32:0/604396712 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb406fc30 con 0x7f1fb4136710 2023-09-30T22:24:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb933c700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4139030 0x7f1fb413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:24:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb3fff700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4136710 0x7f1fb4138af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00098f0 tx=0x7f1fa0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d78fef24f722642 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb37fe700 1 -- 172.21.15.32:0/604396712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0004230 con 0x7f1fb4136710 2023-09-30T22:24:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb37fe700 1 -- 172.21.15.32:0/604396712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0004390 con 0x7f1fb4136710 2023-09-30T22:24:54.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.369+0000 7f1fb37fe700 1 -- 172.21.15.32:0/604396712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00036a0 con 0x7f1fb4136710 2023-09-30T22:24:54.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 -- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4136710 msgr2=0x7f1fb4138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4136710 0x7f1fb4138af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00098f0 tx=0x7f1fa0009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:54.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 -- 172.21.15.32:0/604396712 shutdown_connections 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4139030 0x7f1fb413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/604396712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4136710 0x7f1fb4138af0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 -- 172.21.15.32:0/604396712 >> 172.21.15.32:0/604396712 conn(0x7f1fb40a2aa0 msgr2=0x7f1fb40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 -- 172.21.15.32:0/604396712 shutdown_connections 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.370+0000 7f1fbad9f700 1 -- 172.21.15.32:0/604396712 wait complete. 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.371+0000 7f1fbad9f700 1 Processor -- start 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.371+0000 7f1fbad9f700 1 -- start start 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fbad9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fb41338a0 0x7f1fb411ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fbad9f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4136710 0x7f1fb411b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fbad9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 0x7f1fb411b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fbad9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4138b00 con 0x7f1fb4139030 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fbad9f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb411e5c0 con 0x7f1fb41338a0 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fbad9f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4130c70 con 0x7f1fb4136710 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fb3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4136710 0x7f1fb411b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fb933c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 0x7f1fb411b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fb933c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 0x7f1fb411b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57676/0 (socket says 172.21.15.32:57676) 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fb933c700 1 -- 172.21.15.32:0/3754500388 learned_addr learned my addr 172.21.15.32:0/3754500388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fb8b3b700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fb41338a0 0x7f1fb411ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.372+0000 7f1fb3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4136710 0x7f1fb411b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59262/0 (socket says 172.21.15.32:59262) 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb933c700 1 -- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4136710 msgr2=0x7f1fb411b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb933c700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4136710 0x7f1fb411b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb933c700 1 -- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fb41338a0 msgr2=0x7f1fb411ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb933c700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fb41338a0 0x7f1fb411ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb933c700 1 -- 172.21.15.32:0/3754500388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0009550 con 0x7f1fb4139030 2023-09-30T22:24:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb933c700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 0x7f1fb411b800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00cef0 tx=0x7f1fac00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00ad00 con 0x7f1fb4139030 2023-09-30T22:24:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00ae60 con 0x7f1fb4139030 2023-09-30T22:24:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.373+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac015680 con 0x7f1fb4139030 2023-09-30T22:24:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.374+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb411cf40 con 0x7f1fb4139030 2023-09-30T22:24:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.374+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb411d180 con 0x7f1fb4139030 2023-09-30T22:24:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.375+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1fac00d430 con 0x7f1fb4139030 2023-09-30T22:24:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.375+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1fac03fc80 con 0x7f1fb4139030 2023-09-30T22:24:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.375+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb411e700 con 0x7f1fb4139030 2023-09-30T22:24:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.383+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1fb411e700 con 0x7f1fb4139030 2023-09-30T22:24:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.463+0000 7fed6c4be700 1 Processor -- start 2023-09-30T22:24:54.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.463+0000 7fed6c4be700 1 -- start start 2023-09-30T22:24:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6c4be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412c2f0 0x7fed64130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6c4be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412cff0 0x7fed64137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6c4be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6409a510 con 0x7fed6412cc20 2023-09-30T22:24:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6c4be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6412c7d0 con 0x7fed6412c2f0 2023-09-30T22:24:54.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6c4be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed6409c190 con 0x7fed6412cff0 2023-09-30T22:24:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6a25a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed6412cc20 0x7fed6412f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40272/0 (socket says 172.21.15.32:40272) 2023-09-30T22:24:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6a25a700 1 -- 172.21.15.32:0/4049349310 learned_addr learned my addr 172.21.15.32:0/4049349310 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed69a59700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412c2f0 0x7fed64130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed6aa5b700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412cff0 0x7fed64137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed69a59700 1 -- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412cff0 msgr2=0x7fed64137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:54.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed69a59700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412cff0 0x7fed64137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.464+0000 7fed69258700 1 -- 172.21.15.32:0/4049349310 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049064102 0 0) 0x7fed6409a510 con 0x7fed6412cc20 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.465+0000 7fed69a59700 1 -- 172.21.15.32:0/4049349310 >> v1:172.21.15.32:6789/0 conn(0x7fed6412cc20 legacy=0x7fed6412f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.465+0000 7fed69a59700 1 -- 172.21.15.32:0/4049349310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6409b540 con 0x7fed6412c2f0 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.465+0000 7fed69a59700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412c2f0 0x7fed64130040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed54009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97672c1b8f93d82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.465+0000 7fed69258700 1 -- 172.21.15.32:0/4049349310 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54010a90 con 0x7fed6412c2f0 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.465+0000 7fed69258700 1 -- 172.21.15.32:0/4049349310 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54010bf0 con 0x7fed6412c2f0 2023-09-30T22:24:54.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.465+0000 7fed69258700 1 -- 172.21.15.32:0/4049349310 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54018440 con 0x7fed6412c2f0 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 -- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412c2f0 msgr2=0x7fed64130040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412c2f0 0x7fed64130040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed54009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 -- 172.21.15.32:0/4049349310 shutdown_connections 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412cff0 0x7fed64137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 --2- 172.21.15.32:0/4049349310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412c2f0 0x7fed64130040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 -- 172.21.15.32:0/4049349310 >> 172.21.15.32:0/4049349310 conn(0x7fed6409f470 msgr2=0x7fed640a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 -- 172.21.15.32:0/4049349310 shutdown_connections 2023-09-30T22:24:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.466+0000 7fed6c4be700 1 -- 172.21.15.32:0/4049349310 wait complete. 2023-09-30T22:24:54.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.468+0000 7fed6c4be700 1 Processor -- start 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.468+0000 7fed6c4be700 1 -- start start 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6c4be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412c2f0 0x7fed64128be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6c4be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6412cc20 0x7fed64129120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6c4be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 0x7fed641c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6c4be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed64129f10 con 0x7fed6412cc20 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6c4be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed64128790 con 0x7fed6412cff0 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6c4be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed6406e540 con 0x7fed6412c2f0 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 0x7fed641c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 0x7fed641c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48106/0 (socket says 172.21.15.32:48106) 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 -- 172.21.15.32:0/2607184873 learned_addr learned my addr 172.21.15.32:0/2607184873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:54.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6a25a700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412c2f0 0x7fed64128be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed69a59700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6412cc20 0x7fed64129120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 -- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412c2f0 msgr2=0x7fed64128be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412c2f0 0x7fed64128be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 -- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6412cc20 msgr2=0x7fed64129120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6412cc20 0x7fed64129120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 -- 172.21.15.32:0/2607184873 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540094a0 con 0x7fed6412cff0 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.469+0000 7fed6aa5b700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 0x7fed641c3f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00efc0 tx=0x7fed5c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.470+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c009ea0 con 0x7fed6412cff0 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.470+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00f040 con 0x7fed6412cff0 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.470+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6406f5f0 con 0x7fed6412cff0 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.470+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00ced0 con 0x7fed6412cff0 2023-09-30T22:24:54.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.470+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6406d900 con 0x7fed6412cff0 2023-09-30T22:24:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.471+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fed5c014480 con 0x7fed6412cff0 2023-09-30T22:24:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.471+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fed5c03ff40 con 0x7fed6412cff0 2023-09-30T22:24:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.471+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6412b9a0 con 0x7fed6412cff0 2023-09-30T22:24:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.477+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed5c018020 con 0x7fed6412cff0 2023-09-30T22:24:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.496+0000 7f359a1ef700 1 Processor -- start 2023-09-30T22:24:54.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.496+0000 7f359a1ef700 1 -- start start 2023-09-30T22:24:54.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f359a1ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f359409e1d0 0x7f3594138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f359a1ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3594139170 0x7f359413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f359a1ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359409b120 con 0x7f359409d6f0 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f359a1ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359406b460 con 0x7f359409e1d0 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f359a1ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f359409a7e0 con 0x7f3594139170 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f35937fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f359409d6f0 0x7f359409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40280/0 (socket says 172.21.15.32:40280) 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f35937fe700 1 -- 172.21.15.32:0/765396287 learned_addr learned my addr 172.21.15.32:0/765396287 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f3593fff700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3594139170 0x7f359413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.497+0000 7f3592ffd700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f359409e1d0 0x7f3594138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f35927fc700 1 -- 172.21.15.32:0/765396287 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148700509 0 0) 0x7f359409b120 con 0x7f359409d6f0 2023-09-30T22:24:54.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f35927fc700 1 -- 172.21.15.32:0/765396287 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f357c0035b0 con 0x7f359409d6f0 2023-09-30T22:24:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f3593fff700 1 -- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f359409e1d0 msgr2=0x7f3594138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f3593fff700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f359409e1d0 0x7f3594138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f3593fff700 1 -- 172.21.15.32:0/765396287 >> v1:172.21.15.32:6789/0 conn(0x7f359409d6f0 legacy=0x7f359409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f3593fff700 1 -- 172.21.15.32:0/765396287 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359409a500 con 0x7f3594139170 2023-09-30T22:24:54.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f3593fff700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3594139170 0x7f359413b550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f358c009df0 tx=0x7f358c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a1a212d8ab56ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.498+0000 7f35927fc700 1 -- 172.21.15.32:0/765396287 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00e070 con 0x7f3594139170 2023-09-30T22:24:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.499+0000 7f35927fc700 1 -- 172.21.15.32:0/765396287 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00bae0 con 0x7f3594139170 2023-09-30T22:24:54.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.499+0000 7f35927fc700 1 -- 172.21.15.32:0/765396287 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00be90 con 0x7f3594139170 2023-09-30T22:24:54.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 -- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3594139170 msgr2=0x7f359413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3594139170 0x7f359413b550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f358c009df0 tx=0x7f358c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:54.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 -- 172.21.15.32:0/765396287 shutdown_connections 2023-09-30T22:24:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3594139170 0x7f359413b550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 --2- 172.21.15.32:0/765396287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f359409e1d0 0x7f3594138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 -- 172.21.15.32:0/765396287 >> 172.21.15.32:0/765396287 conn(0x7f3594066f50 msgr2=0x7f359409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 -- 172.21.15.32:0/765396287 shutdown_connections 2023-09-30T22:24:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.505+0000 7f359a1ef700 1 -- 172.21.15.32:0/765396287 wait complete. 2023-09-30T22:24:54.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.506+0000 7f359a1ef700 1 Processor -- start 2023-09-30T22:24:54.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 -- start start 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f359409d6f0 0x7f3594127a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359409e1d0 0x7f3594127f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3594139170 0x7f35941cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359412a970 con 0x7f359409e1d0 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35941276d0 con 0x7f3594139170 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.507+0000 7f359a1ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3594071eb0 con 0x7f359409d6f0 2023-09-30T22:24:54.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3592ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359409e1d0 0x7f3594127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3593fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3594139170 0x7f35941cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3592ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359409e1d0 0x7f3594127f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57702/0 (socket says 172.21.15.32:57702) 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3592ffd700 1 -- 172.21.15.32:0/3771582305 learned_addr learned my addr 172.21.15.32:0/3771582305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f35937fe700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f359409d6f0 0x7f3594127a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3593fff700 1 -- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f359409d6f0 msgr2=0x7f3594127a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3593fff700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f359409d6f0 0x7f3594127a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3593fff700 1 -- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359409e1d0 msgr2=0x7f3594127f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3593fff700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359409e1d0 0x7f3594127f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.508+0000 7f3593fff700 1 -- 172.21.15.32:0/3771582305 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c009a70 con 0x7f3594139170 2023-09-30T22:24:54.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.509+0000 7f3593fff700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3594139170 0x7f35941cc3a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f358c016000 tx=0x7f358c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:54.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.509+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c004550 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.509+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c020070 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.509+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0048d0 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.509+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35941292f0 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.509+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3594128da0 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.510+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f358c027480 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.510+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f358c022070 con 0x7f3594139170 2023-09-30T22:24:54.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.511+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594074ea0 con 0x7f3594139170 2023-09-30T22:24:54.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.520+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f358c02c080 con 0x7f3594139170 2023-09-30T22:24:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.574+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5be0005860 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.574+0000 7f5beaffd700 1 -- 172.21.15.32:0/699788145 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bec024b00 con 0x7f5bfc09d6f0 2023-09-30T22:24:54.575 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 msgr2=0x7f5bfc1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 0x7f5bfc1c8040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00ebe0 tx=0x7f5bec00eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 shutdown_connections 2023-09-30T22:24:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bfc09e410 0x7f5bfc1c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bfc09d6f0 0x7f5bfc1c8040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 --2- 172.21.15.32:0/699788145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bfc09e000 0x7f5bfc1c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 >> 172.21.15.32:0/699788145 conn(0x7f5bfc066f50 msgr2=0x7f5bfc1bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 shutdown_connections 2023-09-30T22:24:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.577+0000 7f5c00963700 1 -- 172.21.15.32:0/699788145 wait complete. 2023-09-30T22:24:54.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.609+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1fb411e700 con 0x7f1fb4139030 2023-09-30T22:24:54.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.609+0000 7f1fb17fa700 1 -- 172.21.15.32:0/3754500388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1fb411e700 con 0x7f1fb4139030 2023-09-30T22:24:54.610 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:54.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 msgr2=0x7f1fb411b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:54.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 0x7f1fb411b800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00cef0 tx=0x7f1fac00cf20 comp rx=0 tx=0).stop 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 shutdown_connections 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb4139030 0x7f1fb411b800 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fb41338a0 0x7f1fb411ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 --2- 172.21.15.32:0/3754500388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fb4136710 0x7f1fb411b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.612+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 >> 172.21.15.32:0/3754500388 conn(0x7f1fb40a2aa0 msgr2=0x7f1fb4137190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.613+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 shutdown_connections 2023-09-30T22:24:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.613+0000 7f1fbad9f700 1 -- 172.21.15.32:0/3754500388 wait complete. 2023-09-30T22:24:54.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:54.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.687+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fed640747f0 con 0x7fed6412cff0 2023-09-30T22:24:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.688+0000 7fed5affd700 1 -- 172.21.15.32:0/2607184873 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed5c01a680 con 0x7fed6412cff0 2023-09-30T22:24:55.332 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 msgr2=0x7fed641c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 0x7fed641c3f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00efc0 tx=0x7fed5c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 shutdown_connections 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6412cff0 0x7fed641c3f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6412c2f0 0x7fed64128be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 --2- 172.21.15.32:0/2607184873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6412cc20 0x7fed64129120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 >> 172.21.15.32:0/2607184873 conn(0x7fed6409f470 msgr2=0x7fed6412a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 shutdown_connections 2023-09-30T22:24:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.690+0000 7fed6c4be700 1 -- 172.21.15.32:0/2607184873 wait complete. 2023-09-30T22:24:55.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.727+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f359406c360 con 0x7f3594139170 2023-09-30T22:24:55.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.728+0000 7f357bfff700 1 -- 172.21.15.32:0/3771582305 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f358c011980 con 0x7f3594139170 2023-09-30T22:24:55.335 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3594139170 msgr2=0x7f35941cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3594139170 0x7f35941cc3a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f358c016000 tx=0x7f358c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 shutdown_connections 2023-09-30T22:24:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3594139170 0x7f35941cc3a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f359409d6f0 0x7f3594127a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 --2- 172.21.15.32:0/3771582305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359409e1d0 0x7f3594127f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 >> 172.21.15.32:0/3771582305 conn(0x7f3594066f50 msgr2=0x7f3594070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 shutdown_connections 2023-09-30T22:24:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:54.730+0000 7f359a1ef700 1 -- 172.21.15.32:0/3771582305 wait complete. 2023-09-30T22:24:55.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:55.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:55.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.745+0000 7ff5066c3700 1 Processor -- start 2023-09-30T22:24:55.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.745+0000 7ff5066c3700 1 -- start start 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff5066c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff50009d630 0x7ff5000a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff5066c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff500138d80 0x7ff50013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff5066c3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5000a2a70 con 0x7ff5000a0a30 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff5066c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000a01d0 con 0x7ff50009d630 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff5066c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff50009e2b0 con 0x7ff500138d80 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff4fffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff5000a0a30 0x7ff5000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40290/0 (socket says 172.21.15.32:40290) 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff504c60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff500138d80 0x7ff50013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff4fffff700 1 -- 172.21.15.32:0/3533554349 learned_addr learned my addr 172.21.15.32:0/3533554349 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:55.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff4ff7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff50009d630 0x7ff5000a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff4feffd700 1 -- 172.21.15.32:0/3533554349 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504399653 0 0) 0x7ff5000a2a70 con 0x7ff5000a0a30 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff4feffd700 1 -- 172.21.15.32:0/3533554349 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e80035b0 con 0x7ff5000a0a30 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff504c60700 1 -- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff50009d630 msgr2=0x7ff5000a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff504c60700 1 --2- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff50009d630 0x7ff5000a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff504c60700 1 -- 172.21.15.32:0/3533554349 >> v1:172.21.15.32:6789/0 conn(0x7ff5000a0a30 legacy=0x7ff5000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff504c60700 1 -- 172.21.15.32:0/3533554349 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50009dc50 con 0x7ff500138d80 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.746+0000 7ff4ff7fe700 1 --2- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff50009d630 0x7ff5000a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff504c60700 1 --2- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff500138d80 0x7ff50013b160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009dc0 tx=0x7ff4f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f9f634d9405c9e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff4feffd700 1 -- 172.21.15.32:0/3533554349 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800e070 con 0x7ff500138d80 2023-09-30T22:24:55.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff4feffd700 1 -- 172.21.15.32:0/3533554349 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f800bb20 con 0x7ff500138d80 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff4feffd700 1 -- 172.21.15.32:0/3533554349 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800be90 con 0x7ff500138d80 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff5066c3700 1 -- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff500138d80 msgr2=0x7ff50013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff5066c3700 1 --2- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff500138d80 0x7ff50013b160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009dc0 tx=0x7ff4f800b330 comp rx=0 tx=0).stop 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff5066c3700 1 -- 172.21.15.32:0/3533554349 shutdown_connections 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff5066c3700 1 --2- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff500138d80 0x7ff50013b160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.747+0000 7ff5066c3700 1 --2- 172.21.15.32:0/3533554349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff50009d630 0x7ff5000a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.748+0000 7ff5066c3700 1 -- 172.21.15.32:0/3533554349 >> 172.21.15.32:0/3533554349 conn(0x7ff5000a3050 msgr2=0x7ff5000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.748+0000 7ff5066c3700 1 -- 172.21.15.32:0/3533554349 shutdown_connections 2023-09-30T22:24:55.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.748+0000 7ff5066c3700 1 -- 172.21.15.32:0/3533554349 wait complete. 2023-09-30T22:24:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.748+0000 7ff5066c3700 1 Processor -- start 2023-09-30T22:24:55.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 -- start start 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 0x7ff5001336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5000a0a30 0x7ff500133c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 0x7ff500134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5001371b0 con 0x7ff5000a0a30 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff500137cf0 con 0x7ff500138d80 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff5066c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000a4940 con 0x7ff50009d630 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 0x7ff5001336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff504c60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 0x7ff500134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff504c60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 0x7ff500134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48154/0 (socket says 172.21.15.32:48154) 2023-09-30T22:24:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 0x7ff5001336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59322/0 (socket says 172.21.15.32:59322) 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff504c60700 1 -- 172.21.15.32:0/2272354116 learned_addr learned my addr 172.21.15.32:0/2272354116 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.749+0000 7ff4ff7fe700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5000a0a30 0x7ff500133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff504c60700 1 -- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 msgr2=0x7ff5001336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff504c60700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 0x7ff5001336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff504c60700 1 -- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5000a0a30 msgr2=0x7ff500133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff504c60700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5000a0a30 0x7ff500133c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff504c60700 1 -- 172.21.15.32:0/2272354116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f40094a0 con 0x7ff500138d80 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff4fffff700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 0x7ff5001336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff504c60700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 0x7ff500134140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800b370 tx=0x7ff4f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8004880 con 0x7ff500138d80 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f8020070 con 0x7ff500138d80 2023-09-30T22:24:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f8009a70 con 0x7ff500138d80 2023-09-30T22:24:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80276c0 con 0x7ff500138d80 2023-09-30T22:24:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.750+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5000a0410 con 0x7ff500138d80 2023-09-30T22:24:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.751+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff4f8012430 con 0x7ff500138d80 2023-09-30T22:24:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.751+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff4f8022070 con 0x7ff500138d80 2023-09-30T22:24:55.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.751+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5001351c0 con 0x7ff500138d80 2023-09-30T22:24:55.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.756+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4f8003e10 con 0x7ff500138d80 2023-09-30T22:24:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.782+0000 7f077833d700 1 Processor -- start 2023-09-30T22:24:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.782+0000 7f077833d700 1 -- start start 2023-09-30T22:24:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f077833d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07700a0a30 0x7f07700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f077833d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 0x7f0770133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f077833d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 0x7f0770134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f077833d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077009edd0 con 0x7f07700a1340 2023-09-30T22:24:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f077833d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077009e1f0 con 0x7f0770133fa0 2023-09-30T22:24:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f077833d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f077009dc50 con 0x7f07700a0a30 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f07758d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 0x7f0770133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f07768da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 0x7f0770134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f07758d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 0x7f0770133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57722/0 (socket says 172.21.15.32:57722) 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f07760d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07700a0a30 0x7f07700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f07768da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 0x7f0770134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48162/0 (socket says 172.21.15.32:48162) 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.783+0000 7f07758d8700 1 -- 172.21.15.32:0/1074989453 learned_addr learned my addr 172.21.15.32:0/1074989453 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07768da700 1 -- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07700a0a30 msgr2=0x7f07700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07768da700 1 --2- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07700a0a30 0x7f07700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07768da700 1 -- 172.21.15.32:0/1074989453 >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 msgr2=0x7f0770133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07768da700 1 --2- 172.21.15.32:0/1074989453 >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 0x7f0770133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07768da700 1 -- 172.21.15.32:0/1074989453 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700a1fe0 con 0x7f0770133fa0 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07758d8700 1 --2- 172.21.15.32:0/1074989453 >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 0x7f0770133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07768da700 1 --2- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 0x7f0770134410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0768009870 tx=0x7f0768009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1e7e85659038cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07750d7700 1 -- 172.21.15.32:0/1074989453 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011830 con 0x7f0770133fa0 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07750d7700 1 -- 172.21.15.32:0/1074989453 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768011990 con 0x7f0770133fa0 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.784+0000 7f07750d7700 1 -- 172.21.15.32:0/1074989453 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011d20 con 0x7f0770133fa0 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 -- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 msgr2=0x7f0770134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 --2- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 0x7f0770134410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0768009870 tx=0x7f0768009b80 comp rx=0 tx=0).stop 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 -- 172.21.15.32:0/1074989453 shutdown_connections 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 --2- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0770133fa0 0x7f0770134410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 --2- 172.21.15.32:0/1074989453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07700a0a30 0x7f07700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 --2- 172.21.15.32:0/1074989453 >> v2:172.21.15.32:3300/0 conn(0x7f07700a1340 0x7f0770133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 -- 172.21.15.32:0/1074989453 >> 172.21.15.32:0/1074989453 conn(0x7f07700a6550 msgr2=0x7f07700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 -- 172.21.15.32:0/1074989453 shutdown_connections 2023-09-30T22:24:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.785+0000 7f077833d700 1 -- 172.21.15.32:0/1074989453 wait complete. 2023-09-30T22:24:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 Processor -- start 2023-09-30T22:24:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 -- start start 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07700a0a30 0x7f07701200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07700a1340 0x7f07701205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 0x7f0770120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07701233e0 con 0x7f07700a1340 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770122eb0 con 0x7f07700a0a30 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f077833d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0770121d00 con 0x7f0770133fa0 2023-09-30T22:24:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f07768da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 0x7f0770120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f07760d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07700a0a30 0x7f07701200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f07768da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 0x7f0770120b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59352/0 (socket says 172.21.15.32:59352) 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f07760d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07700a0a30 0x7f07701200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48166/0 (socket says 172.21.15.32:48166) 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.786+0000 7f07768da700 1 -- 172.21.15.32:0/3080080632 learned_addr learned my addr 172.21.15.32:0/3080080632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07758d8700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07700a1340 0x7f07701205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07768da700 1 -- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07700a0a30 msgr2=0x7f07701200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07768da700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07700a0a30 0x7f07701200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07768da700 1 -- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07700a1340 msgr2=0x7f07701205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07768da700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07700a1340 0x7f07701205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07768da700 1 -- 172.21.15.32:0/3080080632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009550 con 0x7f0770133fa0 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07758d8700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07700a1340 0x7f07701205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:24:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f07768da700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 0x7f0770120b20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0768009840 tx=0x7f0768004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768004510 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07680036a0 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07680094a0 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768003a50 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.787+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07700a34a0 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.788+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0768003bb0 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.788+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f07680408f0 con 0x7f0770133fa0 2023-09-30T22:24:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.788+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07700a0280 con 0x7f0770133fa0 2023-09-30T22:24:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.793+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f076801a420 con 0x7f0770133fa0 2023-09-30T22:24:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.965+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff5001364c0 con 0x7ff500138d80 2023-09-30T22:24:55.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.965+0000 7ff4fcff9700 1 -- 172.21.15.32:0/2272354116 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4f802c020 con 0x7ff500138d80 2023-09-30T22:24:55.966 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 msgr2=0x7ff500134140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:55.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 0x7ff500134140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800b370 tx=0x7ff4f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 shutdown_connections 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff500138d80 0x7ff500134140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff50009d630 0x7ff5001336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 --2- 172.21.15.32:0/2272354116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5000a0a30 0x7ff500133c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 >> 172.21.15.32:0/2272354116 conn(0x7ff5000a3050 msgr2=0x7ff50013ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.967+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 shutdown_connections 2023-09-30T22:24:55.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:55.968+0000 7ff5066c3700 1 -- 172.21.15.32:0/2272354116 wait complete. 2023-09-30T22:24:55.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.000+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0770074ea0 con 0x7f0770133fa0 2023-09-30T22:24:56.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.000+0000 7f076effd700 1 -- 172.21.15.32:0/3080080632 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0768036020 con 0x7f0770133fa0 2023-09-30T22:24:56.001 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 msgr2=0x7f0770120b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 0x7f0770120b20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0768009840 tx=0x7f0768004750 comp rx=0 tx=0).stop 2023-09-30T22:24:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 shutdown_connections 2023-09-30T22:24:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0770133fa0 0x7f0770120b20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07700a0a30 0x7f07701200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 --2- 172.21.15.32:0/3080080632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07700a1340 0x7f07701205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 >> 172.21.15.32:0/3080080632 conn(0x7f07700a6550 msgr2=0x7f07701baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 shutdown_connections 2023-09-30T22:24:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.003+0000 7f077833d700 1 -- 172.21.15.32:0/3080080632 wait complete. 2023-09-30T22:24:56.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:56.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:56.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.488+0000 7fd5502b9700 1 Processor -- start 2023-09-30T22:24:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.488+0000 7fd5502b9700 1 -- start start 2023-09-30T22:24:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.488+0000 7fba79e7d700 1 Processor -- start 2023-09-30T22:24:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.488+0000 7fba79e7d700 1 -- start start 2023-09-30T22:24:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba79e7d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba79e7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba74099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd5502b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd54809e850 0x7fd54809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba79e7d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba741313c0 con 0x7fba7409a3b0 2023-09-30T22:24:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba79e7d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7409b580 con 0x7fba74131810 2023-09-30T22:24:56.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba79e7d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7406e8c0 con 0x7fba74099a40 2023-09-30T22:24:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd5502b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd548134300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd5502b9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54806bf30 con 0x7fd54809f160 2023-09-30T22:24:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd5502b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54809ffb0 con 0x7fd54809e850 2023-09-30T22:24:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd5502b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd54806cb10 con 0x7fd54809f5b0 2023-09-30T22:24:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd54d854700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd54809f160 0x7fd548133bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40298/0 (socket says 172.21.15.32:40298) 2023-09-30T22:24:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd54d854700 1 -- 172.21.15.32:0/2232114032 learned_addr learned my addr 172.21.15.32:0/2232114032 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba737fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba737fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48182/0 (socket says 172.21.15.32:48182) 2023-09-30T22:24:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba737fe700 1 -- 172.21.15.32:0/1688593365 learned_addr learned my addr 172.21.15.32:0/1688593365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd54e856700 1 --2- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd548134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fd54e055700 1 --2- 172.21.15.32:0/2232114032 >> v2:172.21.15.32:3300/0 conn(0x7fd54809e850 0x7fd54809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.489+0000 7fba72ffd700 1 --2- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba74099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba737fe700 1 -- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 msgr2=0x7fba74099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba737fe700 1 --2- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba74099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba737fe700 1 -- 172.21.15.32:0/1688593365 >> v1:172.21.15.32:6789/0 conn(0x7fba7409a3b0 legacy=0x7fba74138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba727fc700 1 -- 172.21.15.32:0/1688593365 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58388693 0 0) 0x7fba741313c0 con 0x7fba7409a3b0 2023-09-30T22:24:56.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba737fe700 1 -- 172.21.15.32:0/1688593365 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7409a820 con 0x7fba74131810 2023-09-30T22:24:56.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54d053700 1 -- 172.21.15.32:0/2232114032 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743884220 0 0) 0x7fd54806bf30 con 0x7fd54809f160 2023-09-30T22:24:56.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54d053700 1 -- 172.21.15.32:0/2232114032 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5340035b0 con 0x7fd54809f160 2023-09-30T22:24:56.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba737fe700 1 --2- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74131be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fba64009920 tx=0x7fba64009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d96aee8691c18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:56.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54e055700 1 -- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 msgr2=0x7fd548134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:56.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54e055700 1 --2- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd548134300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54e055700 1 -- 172.21.15.32:0/2232114032 >> v1:172.21.15.32:6789/0 conn(0x7fd54809f160 legacy=0x7fd548133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54e055700 1 -- 172.21.15.32:0/2232114032 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480a0270 con 0x7fd54809e850 2023-09-30T22:24:56.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba727fc700 1 -- 172.21.15.32:0/1688593365 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64011830 con 0x7fba74131810 2023-09-30T22:24:56.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54e055700 1 --2- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd54809ec20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd544009840 tx=0x7fd544009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f635c478b073ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba727fc700 1 -- 172.21.15.32:0/1688593365 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64011990 con 0x7fba74131810 2023-09-30T22:24:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fba727fc700 1 -- 172.21.15.32:0/1688593365 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64011d20 con 0x7fba74131810 2023-09-30T22:24:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.490+0000 7fd54d053700 1 -- 172.21.15.32:0/2232114032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544011830 con 0x7fd54809e850 2023-09-30T22:24:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd54d053700 1 -- 172.21.15.32:0/2232114032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544011990 con 0x7fd54809e850 2023-09-30T22:24:56.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd54d053700 1 -- 172.21.15.32:0/2232114032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544011d20 con 0x7fd54809e850 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 -- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 msgr2=0x7fba74131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 --2- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74131be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fba64009920 tx=0x7fba64009c30 comp rx=0 tx=0).stop 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 -- 172.21.15.32:0/1688593365 shutdown_connections 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 --2- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74131be0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 --2- 172.21.15.32:0/1688593365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba74099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 -- 172.21.15.32:0/1688593365 >> 172.21.15.32:0/1688593365 conn(0x7fba740a2aa0 msgr2=0x7fba740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:56.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 -- 172.21.15.32:0/1688593365 shutdown_connections 2023-09-30T22:24:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fba79e7d700 1 -- 172.21.15.32:0/1688593365 wait complete. 2023-09-30T22:24:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 -- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 msgr2=0x7fd54809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd54809ec20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd544009840 tx=0x7fd544009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:56.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 -- 172.21.15.32:0/2232114032 shutdown_connections 2023-09-30T22:24:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd548134300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2232114032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd54809ec20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 -- 172.21.15.32:0/2232114032 >> 172.21.15.32:0/2232114032 conn(0x7fd5480a3f70 msgr2=0x7fd5480a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 -- 172.21.15.32:0/2232114032 shutdown_connections 2023-09-30T22:24:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.491+0000 7fd5502b9700 1 -- 172.21.15.32:0/2232114032 wait complete. 2023-09-30T22:24:56.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.492+0000 7fba79e7d700 1 Processor -- start 2023-09-30T22:24:56.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.492+0000 7fd5502b9700 1 Processor -- start 2023-09-30T22:24:56.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.492+0000 7fd5502b9700 1 -- start start 2023-09-30T22:24:56.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.492+0000 7fba79e7d700 1 -- start start 2023-09-30T22:24:56.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd5502b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd5481a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd5502b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd54809f160 0x7fd5481a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd5502b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd5481a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd5502b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481a8ab0 con 0x7fd54809e850 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd5502b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481a6dd0 con 0x7fd54809f160 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd5502b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481a8cb0 con 0x7fd54809f5b0 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba79e7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba7406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba79e7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7409a3b0 0x7fba7406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba79e7d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba79e7d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba74070a70 con 0x7fba7409a3b0 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba79e7d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7411f160 con 0x7fba74131810 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba79e7d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7411e050 con 0x7fba74099a40 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd54e055700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd5481a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd54e055700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd5481a52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57750/0 (socket says 172.21.15.32:57750) 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd54d854700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd54809f160 0x7fd5481a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd54e055700 1 -- 172.21.15.32:0/2273323494 learned_addr learned my addr 172.21.15.32:0/2273323494 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:56.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba73fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fd54e856700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd5481a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba73fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48204/0 (socket says 172.21.15.32:48204) 2023-09-30T22:24:56.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba73fff700 1 -- 172.21.15.32:0/2230689695 learned_addr learned my addr 172.21.15.32:0/2230689695 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:56.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba72ffd700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7409a3b0 0x7fba7406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.493+0000 7fba737fe700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba7406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54d854700 1 -- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 msgr2=0x7fd5481a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54d854700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd5481a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54d854700 1 -- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 msgr2=0x7fd5481a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54d854700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd5481a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54d854700 1 -- 172.21.15.32:0/2273323494 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440094a0 con 0x7fd54809f160 2023-09-30T22:24:56.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54e856700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd5481a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54e055700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd5481a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba72ffd700 1 -- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 msgr2=0x7fba7406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba72ffd700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba7406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba72ffd700 1 -- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 msgr2=0x7fba74070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba72ffd700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba72ffd700 1 -- 172.21.15.32:0/2230689695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64009550 con 0x7fba7409a3b0 2023-09-30T22:24:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd54d854700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd54809f160 0x7fd5481a57e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd53800c490 tx=0x7fd53800c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba72ffd700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7409a3b0 0x7fba7406fb50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba6800e8d0 tx=0x7fba6800ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:56.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538005d70 con 0x7fd54809f160 2023-09-30T22:24:56.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5380070d0 con 0x7fd54809f160 2023-09-30T22:24:56.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54806cd90 con 0x7fd54809f160 2023-09-30T22:24:56.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538007440 con 0x7fd54809f160 2023-09-30T22:24:56.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5481a6670 con 0x7fd54809f160 2023-09-30T22:24:56.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68013680 con 0x7fba7409a3b0 2023-09-30T22:24:56.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba680137e0 con 0x7fba7409a3b0 2023-09-30T22:24:56.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68013b50 con 0x7fba7409a3b0 2023-09-30T22:24:56.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba740712c0 con 0x7fba7409a3b0 2023-09-30T22:24:56.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.494+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba741306a0 con 0x7fba7409a3b0 2023-09-30T22:24:56.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.495+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba68013cb0 con 0x7fba7409a3b0 2023-09-30T22:24:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.495+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5380075a0 con 0x7fd54809f160 2023-09-30T22:24:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.495+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd53803ead0 con 0x7fd54809f160 2023-09-30T22:24:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.496+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7406eb00 con 0x7fba7409a3b0 2023-09-30T22:24:56.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.496+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba6803f980 con 0x7fba7409a3b0 2023-09-30T22:24:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.496+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5481a8740 con 0x7fd54809f160 2023-09-30T22:24:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.505+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd53801bb10 con 0x7fd54809f160 2023-09-30T22:24:56.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.505+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba6801ab90 con 0x7fba7409a3b0 2023-09-30T22:24:56.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.709+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fba74074ea0 con 0x7fba7409a3b0 2023-09-30T22:24:56.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.710+0000 7fba5bfff700 1 -- 172.21.15.32:0/2230689695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba68021070 con 0x7fba7409a3b0 2023-09-30T22:24:56.711 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:56.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7409a3b0 msgr2=0x7fba7406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7409a3b0 0x7fba7406fb50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba6800e8d0 tx=0x7fba6800ebe0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 shutdown_connections 2023-09-30T22:24:56.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba74131810 0x7fba74070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba74099a40 0x7fba7406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 --2- 172.21.15.32:0/2230689695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba7409a3b0 0x7fba7406fb50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 >> 172.21.15.32:0/2230689695 conn(0x7fba740a2aa0 msgr2=0x7fba7412e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:56.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 shutdown_connections 2023-09-30T22:24:56.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.712+0000 7fba79e7d700 1 -- 172.21.15.32:0/2230689695 wait complete. 2023-09-30T22:24:56.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.715+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd5480747f0 con 0x7fd54809f160 2023-09-30T22:24:56.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.715+0000 7fd53effd700 1 -- 172.21.15.32:0/2273323494 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd53803ce60 con 0x7fd54809f160 2023-09-30T22:24:56.716 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd54809f160 msgr2=0x7fd5481a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd54809f160 0x7fd5481a57e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd53800c490 tx=0x7fd53800c7a0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 shutdown_connections 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd54809f5b0 0x7fd5481a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd54809e850 0x7fd5481a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 --2- 172.21.15.32:0/2273323494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd54809f160 0x7fd5481a57e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 >> 172.21.15.32:0/2273323494 conn(0x7fd5480a3f70 msgr2=0x7fd5481b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:56.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 shutdown_connections 2023-09-30T22:24:56.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:56.718+0000 7fd5502b9700 1 -- 172.21.15.32:0/2273323494 wait complete. 2023-09-30T22:24:56.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:56.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:56.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:57.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:57.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 Processor -- start 2023-09-30T22:24:57.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 -- start start 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08c138d80 0x7fc08c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c0a2a70 con 0x7fc08c0a0a30 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c0a01d0 con 0x7fc08c138d80 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc092121700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c09e2b0 con 0x7fc08c09d630 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc08bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08c138d80 0x7fc08c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc08b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc08c0a0a30 0x7fc08c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40308/0 (socket says 172.21.15.32:40308) 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.134+0000 7fc08b7fe700 1 -- 172.21.15.32:0/3111222091 learned_addr learned my addr 172.21.15.32:0/3111222091 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08affd700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08bfff700 1 -- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 msgr2=0x7fc08c0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08bfff700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08bfff700 1 -- 172.21.15.32:0/3111222091 >> v1:172.21.15.32:6789/0 conn(0x7fc08c0a0a30 legacy=0x7fc08c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08bfff700 1 -- 172.21.15.32:0/3111222091 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c09dc50 con 0x7fc08c138d80 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08affd700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08bfff700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c138d80 0x7fc08c13b160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc084009d90 tx=0x7fc08400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab95ce99422c23f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08a7fc700 1 -- 172.21.15.32:0/3111222091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08400e070 con 0x7fc08c138d80 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08a7fc700 1 -- 172.21.15.32:0/3111222091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08400bbd0 con 0x7fc08c138d80 2023-09-30T22:24:57.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.135+0000 7fc08a7fc700 1 -- 172.21.15.32:0/3111222091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084012430 con 0x7fc08c138d80 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 -- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c138d80 msgr2=0x7fc08c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c138d80 0x7fc08c13b160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc084009d90 tx=0x7fc08400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 -- 172.21.15.32:0/3111222091 shutdown_connections 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c138d80 0x7fc08c13b160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 --2- 172.21.15.32:0/3111222091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 -- 172.21.15.32:0/3111222091 >> 172.21.15.32:0/3111222091 conn(0x7fc08c0a3050 msgr2=0x7fc08c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 -- 172.21.15.32:0/3111222091 shutdown_connections 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.136+0000 7fc092121700 1 -- 172.21.15.32:0/3111222091 wait complete. 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.137+0000 7fc092121700 1 Processor -- start 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.137+0000 7fc092121700 1 -- start start 2023-09-30T22:24:57.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc092121700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c1337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc092121700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 0x7fc08c133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc092121700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c138d80 0x7fc08c134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc092121700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c135b80 con 0x7fc08c0a0a30 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc092121700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c137230 con 0x7fc08c09d630 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc092121700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0a0410 con 0x7fc08c138d80 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 0x7fc08c133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 0x7fc08c133ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57788/0 (socket says 172.21.15.32:57788) 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 -- 172.21.15.32:0/2579391427 learned_addr learned my addr 172.21.15.32:0/2579391427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08bfff700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c138d80 0x7fc08c134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08b7fe700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c1337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 -- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c138d80 msgr2=0x7fc08c134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c138d80 0x7fc08c134220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 -- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 msgr2=0x7fc08c1337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c1337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 -- 172.21.15.32:0/2579391427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084009a70 con 0x7fc08c0a0a30 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.138+0000 7fc08affd700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 0x7fc08c133ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc07400e810 tx=0x7fc07400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.139+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074013610 con 0x7fc08c0a0a30 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.139+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc074013770 con 0x7fc08c0a0a30 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.139+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c0a4bd0 con 0x7fc08c0a0a30 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.139+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074013b20 con 0x7fc08c0a0a30 2023-09-30T22:24:57.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.139+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c136570 con 0x7fc08c0a0a30 2023-09-30T22:24:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.140+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc074013c80 con 0x7fc08c0a0a30 2023-09-30T22:24:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.140+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc07403f2b0 con 0x7fc08c0a0a30 2023-09-30T22:24:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.140+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c136a80 con 0x7fc08c0a0a30 2023-09-30T22:24:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.145+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc074012370 con 0x7fc08c0a0a30 2023-09-30T22:24:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.172+0000 7f84a3a86700 1 Processor -- start 2023-09-30T22:24:57.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.172+0000 7f84a3a86700 1 -- start start 2023-09-30T22:24:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a3a86700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12cf60 0x7f849c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a3a86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f849c12f8b0 0x7f849c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a3a86700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a3a86700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849c12d330 con 0x7f849c12f8b0 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a3a86700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c12cd90 con 0x7f849c130260 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a3a86700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f849c1306d0 con 0x7f849c12cf60 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f849c12f8b0 0x7f849c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f849c12f8b0 0x7f849c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57800/0 (socket says 172.21.15.32:57800) 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 -- 172.21.15.32:0/1656147507 learned_addr learned my addr 172.21.15.32:0/1656147507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a2023700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1822700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12cf60 0x7f849c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 -- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12cf60 msgr2=0x7f849c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12cf60 0x7f849c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 -- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 msgr2=0x7f849c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 -- 172.21.15.32:0/1656147507 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c071780 con 0x7f849c12f8b0 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a2023700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.173+0000 7f84a1021700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12f8b0 0x7f849c12fd20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f848c00aac0 tx=0x7f848c00add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c588f79e339c79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a0820700 1 -- 172.21.15.32:0/1656147507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00cb30 con 0x7f849c12f8b0 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a0820700 1 -- 172.21.15.32:0/1656147507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c00cc90 con 0x7f849c12f8b0 2023-09-30T22:24:57.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a0820700 1 -- 172.21.15.32:0/1656147507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c018400 con 0x7f849c12f8b0 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 -- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12f8b0 msgr2=0x7f849c12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12f8b0 0x7f849c12fd20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f848c00aac0 tx=0x7f848c00add0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 -- 172.21.15.32:0/1656147507 shutdown_connections 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12cf60 0x7f849c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 --2- 172.21.15.32:0/1656147507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12f8b0 0x7f849c12fd20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 -- 172.21.15.32:0/1656147507 >> 172.21.15.32:0/1656147507 conn(0x7f849c0a2aa0 msgr2=0x7f849c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:57.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 -- 172.21.15.32:0/1656147507 shutdown_connections 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.174+0000 7f84a3a86700 1 -- 172.21.15.32:0/1656147507 wait complete. 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.175+0000 7f84a3a86700 1 Processor -- start 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.175+0000 7f84a3a86700 1 -- start start 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a3a86700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 0x7f849c11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a3a86700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12f8b0 0x7f849c11aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a3a86700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c11af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a3a86700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c11c9a0 con 0x7f849c12cf60 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a3a86700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c11e9a0 con 0x7f849c130260 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a3a86700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f849c11c0d0 con 0x7f849c12f8b0 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 0x7f849c11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a2023700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c11af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 0x7f849c11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57808/0 (socket says 172.21.15.32:57808) 2023-09-30T22:24:57.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 -- 172.21.15.32:0/2781326792 learned_addr learned my addr 172.21.15.32:0/2781326792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1021700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12f8b0 0x7f849c11aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 -- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12f8b0 msgr2=0x7f849c11aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12f8b0 0x7f849c11aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 -- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 msgr2=0x7f849c11af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c11af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a1822700 1 -- 172.21.15.32:0/2781326792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494009a70 con 0x7f849c12cf60 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.176+0000 7f84a2023700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c11af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.177+0000 7f84a1822700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 0x7f849c11a4c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f849800ea10 tx=0x7f849800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.177+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498009710 con 0x7f849c12cf60 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.177+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849800eea0 con 0x7f849c12cf60 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.177+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800ca50 con 0x7f849c12cf60 2023-09-30T22:24:57.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.177+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c00a6f0 con 0x7f849c12cf60 2023-09-30T22:24:57.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.177+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c129fa0 con 0x7f849c12cf60 2023-09-30T22:24:57.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.178+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f849800cbb0 con 0x7f849c12cf60 2023-09-30T22:24:57.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.178+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c074ea0 con 0x7f849c12cf60 2023-09-30T22:24:57.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.178+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f849803f6a0 con 0x7f849c12cf60 2023-09-30T22:24:57.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.183+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f84980232f0 con 0x7f849c12cf60 2023-09-30T22:24:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.356+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc08c0747f0 con 0x7fc08c0a0a30 2023-09-30T22:24:57.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.356+0000 7fc083fff700 1 -- 172.21.15.32:0/2579391427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc074021020 con 0x7fc08c0a0a30 2023-09-30T22:24:57.357 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 msgr2=0x7fc08c133ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 0x7fc08c133ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc07400e810 tx=0x7fc07400eb20 comp rx=0 tx=0).stop 2023-09-30T22:24:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 shutdown_connections 2023-09-30T22:24:57.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c138d80 0x7fc08c134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c09d630 0x7fc08c1337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 --2- 172.21.15.32:0/2579391427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c0a0a30 0x7fc08c133ce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 >> 172.21.15.32:0/2579391427 conn(0x7fc08c0a3050 msgr2=0x7fc08c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 shutdown_connections 2023-09-30T22:24:57.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.358+0000 7fc092121700 1 -- 172.21.15.32:0/2579391427 wait complete. 2023-09-30T22:24:57.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.400+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f849c0713a0 con 0x7f849c12cf60 2023-09-30T22:24:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.400+0000 7f84927fc700 1 -- 172.21.15.32:0/2781326792 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f849801ae80 con 0x7f849c12cf60 2023-09-30T22:24:57.401 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 msgr2=0x7f849c11a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 0x7f849c11a4c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f849800ea10 tx=0x7f849800ed20 comp rx=0 tx=0).stop 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 shutdown_connections 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f849c130260 0x7f849c11af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f849c12cf60 0x7f849c11a4c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 --2- 172.21.15.32:0/2781326792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f849c12f8b0 0x7f849c11aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 >> 172.21.15.32:0/2781326792 conn(0x7f849c0a2aa0 msgr2=0x7f849c09e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 shutdown_connections 2023-09-30T22:24:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.403+0000 7f84a3a86700 1 -- 172.21.15.32:0/2781326792 wait complete. 2023-09-30T22:24:57.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:57.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:57.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.877+0000 7f68e028c700 1 Processor -- start 2023-09-30T22:24:57.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 -- start start 2023-09-30T22:24:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68d809d630 0x7f68d80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d8133930 0x7f68d8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d80a2a70 con 0x7f68d809d630 2023-09-30T22:24:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d80a01d0 con 0x7f68d80a0a30 2023-09-30T22:24:57.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68e028c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d809e2b0 con 0x7f68d8133930 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68de028700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68dd827700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68d809d630 0x7f68d80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68dd827700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68d809d630 0x7f68d80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57810/0 (socket says 172.21.15.32:57810) 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68de028700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48264/0 (socket says 172.21.15.32:48264) 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68de829700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d8133930 0x7f68d8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.878+0000 7f68dd827700 1 -- 172.21.15.32:0/1005199342 learned_addr learned my addr 172.21.15.32:0/1005199342 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd827700 1 -- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d8133930 msgr2=0x7f68d8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd827700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d8133930 0x7f68d8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd827700 1 -- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 msgr2=0x7f68d80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd827700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd827700 1 -- 172.21.15.32:0/1005199342 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d809dc50 con 0x7f68d809d630 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68de028700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68de829700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d8133930 0x7f68d8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd827700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d809d630 0x7f68d80a1340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0098f0 tx=0x7f68cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8ad68fffef6f42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd026700 1 -- 172.21.15.32:0/1005199342 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc011830 con 0x7f68d809d630 2023-09-30T22:24:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd026700 1 -- 172.21.15.32:0/1005199342 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc011990 con 0x7f68d809d630 2023-09-30T22:24:57.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.879+0000 7f68dd026700 1 -- 172.21.15.32:0/1005199342 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc011d20 con 0x7f68d809d630 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 -- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d809d630 msgr2=0x7f68d80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d809d630 0x7f68d80a1340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0098f0 tx=0x7f68cc009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 -- 172.21.15.32:0/1005199342 shutdown_connections 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d8133930 0x7f68d8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 --2- 172.21.15.32:0/1005199342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d809d630 0x7f68d80a1340 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 -- 172.21.15.32:0/1005199342 >> 172.21.15.32:0/1005199342 conn(0x7f68d80a3050 msgr2=0x7f68d80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 -- 172.21.15.32:0/1005199342 shutdown_connections 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.880+0000 7f68e028c700 1 -- 172.21.15.32:0/1005199342 wait complete. 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.881+0000 7f68e028c700 1 Processor -- start 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.881+0000 7f68e028c700 1 -- start start 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68e028c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d809d630 0x7f68d8121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68e028c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d8121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68e028c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 0x7f68d8121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68e028c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d81226d0 con 0x7f68d8133930 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68e028c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8124cb0 con 0x7f68d80a0a30 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68e028c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8124710 con 0x7f68d809d630 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68de829700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 0x7f68d8121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68de028700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d809d630 0x7f68d8121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d8121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68de829700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 0x7f68d8121ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57822/0 (socket says 172.21.15.32:57822) 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68de829700 1 -- 172.21.15.32:0/3992765266 learned_addr learned my addr 172.21.15.32:0/3992765266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 -- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d809d630 msgr2=0x7f68d8121220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d809d630 0x7f68d8121220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 -- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 msgr2=0x7f68d8121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 0x7f68d8121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 -- 172.21.15.32:0/3992765266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00094a0 con 0x7f68d80a0a30 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68de028700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d809d630 0x7f68d8121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68de829700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 0x7f68d8121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.882+0000 7f68dd827700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d8121760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68cc016000 tx=0x7f68cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.883+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc004240 con 0x7f68d80a0a30 2023-09-30T22:24:57.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.883+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc01f070 con 0x7f68d80a0a30 2023-09-30T22:24:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.883+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c8009a70 con 0x7f68d80a0a30 2023-09-30T22:24:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.883+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc0036a0 con 0x7f68d80a0a30 2023-09-30T22:24:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.883+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc009550 con 0x7f68d80a0a30 2023-09-30T22:24:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.884+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68cc021030 con 0x7f68d80a0a30 2023-09-30T22:24:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.884+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d80a0410 con 0x7f68d80a0a30 2023-09-30T22:24:57.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.884+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f68cc0407f0 con 0x7f68d80a0a30 2023-09-30T22:24:57.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.889+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68cc0239c0 con 0x7f68d80a0a30 2023-09-30T22:24:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.916+0000 7f5ab779a700 1 Processor -- start 2023-09-30T22:24:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.917+0000 7f5ab779a700 1 -- start start 2023-09-30T22:24:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.917+0000 7f5ab779a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab00a0a30 0x7f5ab00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.917+0000 7f5ab779a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab009d630 0x7f5ab00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.917+0000 7f5ab779a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab00a2a70 con 0x7f5ab0133930 2023-09-30T22:24:57.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.917+0000 7f5ab779a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00a01d0 con 0x7f5ab00a0a30 2023-09-30T22:24:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.917+0000 7f5ab779a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab009e2b0 con 0x7f5ab009d630 2023-09-30T22:24:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5d37700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ab0133930 0x7f5ab0133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40322/0 (socket says 172.21.15.32:40322) 2023-09-30T22:24:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab4d35700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab009d630 0x7f5ab00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5d37700 1 -- 172.21.15.32:0/3604108908 learned_addr learned my addr 172.21.15.32:0/3604108908 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:57.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5536700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab00a0a30 0x7f5ab00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5aa7fff700 1 -- 172.21.15.32:0/3604108908 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861236628 0 0) 0x7f5ab00a2a70 con 0x7f5ab0133930 2023-09-30T22:24:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5536700 1 -- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab009d630 msgr2=0x7f5ab00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:57.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5536700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab009d630 0x7f5ab00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5536700 1 -- 172.21.15.32:0/3604108908 >> v1:172.21.15.32:6789/0 conn(0x7f5ab0133930 legacy=0x7f5ab0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5536700 1 -- 172.21.15.32:0/3604108908 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab009dc50 con 0x7f5ab00a0a30 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab4d35700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab009d630 0x7f5ab00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5ab5536700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab00a0a30 0x7f5ab00a0e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009dc0 tx=0x7f5aac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed3863ac6d7be93 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5aa7fff700 1 -- 172.21.15.32:0/3604108908 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00e070 con 0x7f5ab00a0a30 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5aa7fff700 1 -- 172.21.15.32:0/3604108908 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00bb20 con 0x7f5ab00a0a30 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.918+0000 7f5aa7fff700 1 -- 172.21.15.32:0/3604108908 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac012430 con 0x7f5ab00a0a30 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 -- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab00a0a30 msgr2=0x7f5ab00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab00a0a30 0x7f5ab00a0e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009dc0 tx=0x7f5aac00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 -- 172.21.15.32:0/3604108908 shutdown_connections 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab00a0a30 0x7f5ab00a0e00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 --2- 172.21.15.32:0/3604108908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab009d630 0x7f5ab00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 -- 172.21.15.32:0/3604108908 >> 172.21.15.32:0/3604108908 conn(0x7f5ab00a3050 msgr2=0x7f5ab00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 -- 172.21.15.32:0/3604108908 shutdown_connections 2023-09-30T22:24:57.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.919+0000 7f5ab779a700 1 -- 172.21.15.32:0/3604108908 wait complete. 2023-09-30T22:24:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 Processor -- start 2023-09-30T22:24:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 -- start start 2023-09-30T22:24:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab009d630 0x7f5ab011f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab00a0a30 0x7f5ab011f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab0133930 0x7f5ab01c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab0121ed0 con 0x7f5ab0133930 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab0133250 con 0x7f5ab009d630 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.921+0000 7f5ab779a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab011ee20 con 0x7f5ab00a0a30 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5d37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab0133930 0x7f5ab01c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab009d630 0x7f5ab011f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5d37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab0133930 0x7f5ab01c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57826/0 (socket says 172.21.15.32:57826) 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5d37700 1 -- 172.21.15.32:0/2981809833 learned_addr learned my addr 172.21.15.32:0/2981809833 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab4d35700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab00a0a30 0x7f5ab011f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 -- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab00a0a30 msgr2=0x7f5ab011f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab00a0a30 0x7f5ab011f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 -- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab0133930 msgr2=0x7f5ab01c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab0133930 0x7f5ab01c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 -- 172.21.15.32:0/2981809833 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00094a0 con 0x7f5ab009d630 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab5536700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab009d630 0x7f5ab011f2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5aac016000 tx=0x7f5aac004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac012430 con 0x7f5ab009d630 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac020070 con 0x7f5ab009d630 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac009a70 con 0x7f5ab009d630 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.922+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac004280 con 0x7f5ab009d630 2023-09-30T22:24:57.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.923+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab00a4250 con 0x7f5ab009d630 2023-09-30T22:24:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.923+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5aac0043e0 con 0x7f5ab009d630 2023-09-30T22:24:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.924+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5aac022070 con 0x7f5ab009d630 2023-09-30T22:24:57.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.924+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab011fda0 con 0x7f5ab009d630 2023-09-30T22:24:57.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:57.929+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5aac02c020 con 0x7f5ab009d630 2023-09-30T22:24:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.097+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f68d80747f0 con 0x7f68d80a0a30 2023-09-30T22:24:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.098+0000 7f68d6ffd700 1 -- 172.21.15.32:0/3992765266 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68cc015020 con 0x7f68d80a0a30 2023-09-30T22:24:58.098 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 msgr2=0x7f68d8121760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d8121760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68cc016000 tx=0x7f68cc004750 comp rx=0 tx=0).stop 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 shutdown_connections 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d8133930 0x7f68d8121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d809d630 0x7f68d8121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 --2- 172.21.15.32:0/3992765266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d80a0a30 0x7f68d8121760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 >> 172.21.15.32:0/3992765266 conn(0x7f68d80a3050 msgr2=0x7f68d8099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 shutdown_connections 2023-09-30T22:24:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.100+0000 7f68e028c700 1 -- 172.21.15.32:0/3992765266 wait complete. 2023-09-30T22:24:58.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:58.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.137+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5ab0120a40 con 0x7f5ab009d630 2023-09-30T22:24:58.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.137+0000 7f5aa5ffb700 1 -- 172.21.15.32:0/2981809833 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5aac027480 con 0x7f5ab009d630 2023-09-30T22:24:58.138 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab009d630 msgr2=0x7f5ab011f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab009d630 0x7f5ab011f2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5aac016000 tx=0x7f5aac004ad0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 shutdown_connections 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ab0133930 0x7f5ab01c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ab009d630 0x7f5ab011f2c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 --2- 172.21.15.32:0/2981809833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ab00a0a30 0x7f5ab011f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 >> 172.21.15.32:0/2981809833 conn(0x7f5ab00a3050 msgr2=0x7f5ab01bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 shutdown_connections 2023-09-30T22:24:58.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.140+0000 7f5ab779a700 1 -- 172.21.15.32:0/2981809833 wait complete. 2023-09-30T22:24:58.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:58.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:58.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:58.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.526+0000 7f10f4ff9700 1 Processor -- start 2023-09-30T22:24:58.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.526+0000 7f10f4ff9700 1 -- start start 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10f4ff9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10f4ff9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10f0134170 0x7f10f013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10f4ff9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f009edd0 con 0x7f10f00a1340 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10f4ff9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f009e1f0 con 0x7f10f0134170 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10f4ff9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f009dc50 con 0x7f10f00a0a30 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10eed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10f0134170 0x7f10f013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10edd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10f00a1340 0x7f10f0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40330/0 (socket says 172.21.15.32:40330) 2023-09-30T22:24:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10edd9b700 1 -- 172.21.15.32:0/264083112 learned_addr learned my addr 172.21.15.32:0/264083112 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10ee59c700 1 --2- 172.21.15.32:0/264083112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.527+0000 7f10ed59a700 1 -- 172.21.15.32:0/264083112 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942932867 0 0) 0x7f10f009edd0 con 0x7f10f00a1340 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ee59c700 1 -- 172.21.15.32:0/264083112 >> v2:172.21.15.32:3300/0 conn(0x7f10f0134170 msgr2=0x7f10f013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ee59c700 1 --2- 172.21.15.32:0/264083112 >> v2:172.21.15.32:3300/0 conn(0x7f10f0134170 0x7f10f013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ee59c700 1 -- 172.21.15.32:0/264083112 >> v1:172.21.15.32:6789/0 conn(0x7f10f00a1340 legacy=0x7f10f0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ee59c700 1 -- 172.21.15.32:0/264083112 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00a1fe0 con 0x7f10f00a0a30 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10eed9d700 1 --2- 172.21.15.32:0/264083112 >> v2:172.21.15.32:3300/0 conn(0x7f10f0134170 0x7f10f013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ee59c700 1 --2- 172.21.15.32:0/264083112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f00a0e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009840 tx=0x7f10e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7713048a26d3f1ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ed59a700 1 -- 172.21.15.32:0/264083112 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011830 con 0x7f10f00a0a30 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ed59a700 1 -- 172.21.15.32:0/264083112 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0011990 con 0x7f10f00a0a30 2023-09-30T22:24:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.528+0000 7f10ed59a700 1 -- 172.21.15.32:0/264083112 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011d20 con 0x7f10f00a0a30 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 -- 172.21.15.32:0/264083112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 msgr2=0x7f10f00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/264083112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f00a0e00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009840 tx=0x7f10e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 -- 172.21.15.32:0/264083112 shutdown_connections 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/264083112 >> v2:172.21.15.32:3300/0 conn(0x7f10f0134170 0x7f10f013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/264083112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f00a0e00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 -- 172.21.15.32:0/264083112 >> 172.21.15.32:0/264083112 conn(0x7f10f00a6550 msgr2=0x7f10f00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 -- 172.21.15.32:0/264083112 shutdown_connections 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.529+0000 7f10f4ff9700 1 -- 172.21.15.32:0/264083112 wait complete. 2023-09-30T22:24:58.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 Processor -- start 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 -- start start 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f0123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10f00a1340 0x7f10f0123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0134170 0x7f10f01c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0125ef0 con 0x7f10f0134170 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0126120 con 0x7f10f00a0a30 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.530+0000 7f10f4ff9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10f01244f0 con 0x7f10f00a1340 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f0123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f0123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48310/0 (socket says 172.21.15.32:48310) 2023-09-30T22:24:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 -- 172.21.15.32:0/800955201 learned_addr learned my addr 172.21.15.32:0/800955201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10eed9d700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0134170 0x7f10f01c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10edd9b700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10f00a1340 0x7f10f0123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 -- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10f00a1340 msgr2=0x7f10f0123780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10f00a1340 0x7f10f0123780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 -- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0134170 msgr2=0x7f10f01c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0134170 0x7f10f01c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 -- 172.21.15.32:0/800955201 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009a70 con 0x7f10f00a0a30 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10eed9d700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0134170 0x7f10f01c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10ee59c700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f0123240 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009fd0 tx=0x7f10e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00045d0 con 0x7f10f00a0a30 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e001f070 con 0x7f10f00a0a30 2023-09-30T22:24:58.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e00094a0 con 0x7f10f00a0a30 2023-09-30T22:24:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00038a0 con 0x7f10f00a0a30 2023-09-30T22:24:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.531+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f0124ce0 con 0x7f10f00a0a30 2023-09-30T22:24:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.532+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10e0003a00 con 0x7f10f00a0a30 2023-09-30T22:24:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.532+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f10e0021070 con 0x7f10f00a0a30 2023-09-30T22:24:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.533+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f00a1130 con 0x7f10f00a0a30 2023-09-30T22:24:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.538+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10e0015080 con 0x7f10f00a0a30 2023-09-30T22:24:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.563+0000 7f94eb98b700 1 Processor -- start 2023-09-30T22:24:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.563+0000 7f94eb98b700 1 -- start start 2023-09-30T22:24:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94eb98b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e40a0a30 0x7f94e40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94eb98b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94e409d630 0x7f94e40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94eb98b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40a2a70 con 0x7f94e4133930 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94eb98b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40a01d0 con 0x7f94e409d630 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94eb98b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94e409e2b0 con 0x7f94e40a0a30 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9727700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e40a0a30 0x7f94e40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e8f26700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94e409d630 0x7f94e40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9f28700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f94e4133930 0x7f94e4133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40344/0 (socket says 172.21.15.32:40344) 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9f28700 1 -- 172.21.15.32:0/1972825480 learned_addr learned my addr 172.21.15.32:0/1972825480 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9727700 1 -- 172.21.15.32:0/1972825480 >> v2:172.21.15.32:3300/0 conn(0x7f94e409d630 msgr2=0x7f94e40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9727700 1 --2- 172.21.15.32:0/1972825480 >> v2:172.21.15.32:3300/0 conn(0x7f94e409d630 0x7f94e40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9727700 1 -- 172.21.15.32:0/1972825480 >> v1:172.21.15.32:6789/0 conn(0x7f94e4133930 legacy=0x7f94e4133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9727700 1 -- 172.21.15.32:0/1972825480 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e409dc50 con 0x7f94e40a0a30 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e3fff700 1 -- 172.21.15.32:0/1972825480 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677373550 0 0) 0x7f94e40a2a70 con 0x7f94e4133930 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e8f26700 1 --2- 172.21.15.32:0/1972825480 >> v2:172.21.15.32:3300/0 conn(0x7f94e409d630 0x7f94e40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.564+0000 7f94e9727700 1 --2- 172.21.15.32:0/1972825480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e40a0a30 0x7f94e40a0e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124c648a618ad48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94e3fff700 1 -- 172.21.15.32:0/1972825480 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400e070 con 0x7f94e40a0a30 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94e3fff700 1 -- 172.21.15.32:0/1972825480 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400bb20 con 0x7f94e40a0a30 2023-09-30T22:24:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94e3fff700 1 -- 172.21.15.32:0/1972825480 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4012430 con 0x7f94e40a0a30 2023-09-30T22:24:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 -- 172.21.15.32:0/1972825480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e40a0a30 msgr2=0x7f94e40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 --2- 172.21.15.32:0/1972825480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e40a0a30 0x7f94e40a0e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d400b330 comp rx=0 tx=0).stop 2023-09-30T22:24:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 -- 172.21.15.32:0/1972825480 shutdown_connections 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 --2- 172.21.15.32:0/1972825480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e40a0a30 0x7f94e40a0e00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 --2- 172.21.15.32:0/1972825480 >> v2:172.21.15.32:3300/0 conn(0x7f94e409d630 0x7f94e40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 -- 172.21.15.32:0/1972825480 >> 172.21.15.32:0/1972825480 conn(0x7f94e40a3050 msgr2=0x7f94e40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 -- 172.21.15.32:0/1972825480 shutdown_connections 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.565+0000 7f94eb98b700 1 -- 172.21.15.32:0/1972825480 wait complete. 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.566+0000 7f94eb98b700 1 Processor -- start 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.566+0000 7f94eb98b700 1 -- start start 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94eb98b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e409d630 0x7f94e41c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94eb98b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 0x7f94e41c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94eb98b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94e4133930 0x7f94e41c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94eb98b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4121e90 con 0x7f94e40a0a30 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94eb98b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4121290 con 0x7f94e4133930 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94eb98b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4120520 con 0x7f94e409d630 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 0x7f94e41c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 0x7f94e41c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57874/0 (socket says 172.21.15.32:57874) 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 -- 172.21.15.32:0/4019008286 learned_addr learned my addr 172.21.15.32:0/4019008286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e9f28700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94e4133930 0x7f94e41c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e9727700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e409d630 0x7f94e41c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 -- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e409d630 msgr2=0x7f94e41c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e409d630 0x7f94e41c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 -- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94e4133930 msgr2=0x7f94e41c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94e4133930 0x7f94e41c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e8f26700 1 -- 172.21.15.32:0/4019008286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80094a0 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.567+0000 7f94e9727700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e409d630 0x7f94e41c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.568+0000 7f94e8f26700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 0x7f94e41c8430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94d800ebb0 tx=0x7f94d800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.568+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800c850 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.568+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d800c9b0 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.568+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d4009a70 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.568+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800cd40 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.568+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40a0410 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.569+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94d8013470 con 0x7f94e40a0a30 2023-09-30T22:24:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.569+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e4133420 con 0x7f94e40a0a30 2023-09-30T22:24:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.569+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94d803f8c0 con 0x7f94e40a0a30 2023-09-30T22:24:58.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.574+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94d8017020 con 0x7f94e40a0a30 2023-09-30T22:24:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.746+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f10f0074ea0 con 0x7f10f00a0a30 2023-09-30T22:24:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.746+0000 7f10deffd700 1 -- 172.21.15.32:0/800955201 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10e002b020 con 0x7f10f00a0a30 2023-09-30T22:24:58.748 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:24:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 msgr2=0x7f10f0123240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f0123240 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009fd0 tx=0x7f10e0004750 comp rx=0 tx=0).stop 2023-09-30T22:24:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 shutdown_connections 2023-09-30T22:24:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10f0134170 0x7f10f01c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10f00a0a30 0x7f10f0123240 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 --2- 172.21.15.32:0/800955201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10f00a1340 0x7f10f0123780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 >> 172.21.15.32:0/800955201 conn(0x7f10f00a6550 msgr2=0x7f10f009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 shutdown_connections 2023-09-30T22:24:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.749+0000 7f10f4ff9700 1 -- 172.21.15.32:0/800955201 wait complete. 2023-09-30T22:24:58.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:24:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.780+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f94e40a1800 con 0x7f94e40a0a30 2023-09-30T22:24:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.780+0000 7f94e1ffb700 1 -- 172.21.15.32:0/4019008286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94d80232f0 con 0x7f94e40a0a30 2023-09-30T22:24:58.781 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:24:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 msgr2=0x7f94e41c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 0x7f94e41c8430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94d800ebb0 tx=0x7f94d800eec0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 shutdown_connections 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94e4133930 0x7f94e41c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94e409d630 0x7f94e41c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 --2- 172.21.15.32:0/4019008286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94e40a0a30 0x7f94e41c8430 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.782+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 >> 172.21.15.32:0/4019008286 conn(0x7f94e40a3050 msgr2=0x7f94e41bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.783+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 shutdown_connections 2023-09-30T22:24:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:58.783+0000 7f94eb98b700 1 -- 172.21.15.32:0/4019008286 wait complete. 2023-09-30T22:24:58.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:24:59.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:24:59.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:24:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.264+0000 7f584ad32700 1 Processor -- start 2023-09-30T22:24:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.264+0000 7f584ad32700 1 -- start start 2023-09-30T22:24:59.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.264+0000 7f584ad32700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58440a0a30 0x7f58440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.264+0000 7f584ad32700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5844134170 0x7f584413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f584ad32700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584409edd0 con 0x7f58440a1340 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f584ad32700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584409e1f0 con 0x7f58440a0a30 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f584ad32700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f584409dc50 con 0x7f5844134170 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58440a0a30 0x7f58440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f58492cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5844134170 0x7f584413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58440a0a30 0x7f58440a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57888/0 (socket says 172.21.15.32:57888) 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5843fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58440a1340 0x7f5844133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40358/0 (socket says 172.21.15.32:40358) 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 -- 172.21.15.32:0/2784434729 learned_addr learned my addr 172.21.15.32:0/2784434729 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 -- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5844134170 msgr2=0x7f584413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:59.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 --2- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5844134170 0x7f584413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 -- 172.21.15.32:0/2784434729 >> v1:172.21.15.32:6789/0 conn(0x7f58440a1340 legacy=0x7f5844133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f58437fe700 1 -- 172.21.15.32:0/2784434729 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080276899 0 0) 0x7f584409edd0 con 0x7f58440a1340 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 -- 172.21.15.32:0/2784434729 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440a1fe0 con 0x7f58440a0a30 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f58492cf700 1 --2- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5844134170 0x7f584413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f5848ace700 1 --2- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58440a0a30 0x7f58440a0e00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58380098f0 tx=0x7f5838009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e4fe283840884b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.265+0000 7f58437fe700 1 -- 172.21.15.32:0/2784434729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838011830 con 0x7f58440a0a30 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f58437fe700 1 -- 172.21.15.32:0/2784434729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838011990 con 0x7f58440a0a30 2023-09-30T22:24:59.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f58437fe700 1 -- 172.21.15.32:0/2784434729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838011d20 con 0x7f58440a0a30 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 -- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58440a0a30 msgr2=0x7f58440a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 --2- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58440a0a30 0x7f58440a0e00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f58380098f0 tx=0x7f5838009c00 comp rx=0 tx=0).stop 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 -- 172.21.15.32:0/2784434729 shutdown_connections 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 --2- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5844134170 0x7f584413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 --2- 172.21.15.32:0/2784434729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58440a0a30 0x7f58440a0e00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 -- 172.21.15.32:0/2784434729 >> 172.21.15.32:0/2784434729 conn(0x7f58440a6550 msgr2=0x7f58440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:59.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 -- 172.21.15.32:0/2784434729 shutdown_connections 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.266+0000 7f584ad32700 1 -- 172.21.15.32:0/2784434729 wait complete. 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.267+0000 7f584ad32700 1 Processor -- start 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.267+0000 7f584ad32700 1 -- start start 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f584ad32700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 0x7f58441c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f584ad32700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 0x7f58441c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f584ad32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5844134170 0x7f58441c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f584ad32700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844125260 con 0x7f5844134170 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f584ad32700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5844125e60 con 0x7f58440a0a30 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f584ad32700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58441244f0 con 0x7f58440a1340 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f5848ace700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 0x7f58441c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f5843fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 0x7f58441c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f5848ace700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 0x7f58441c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48340/0 (socket says 172.21.15.32:48340) 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f5843fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 0x7f58441c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59486/0 (socket says 172.21.15.32:59486) 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f5848ace700 1 -- 172.21.15.32:0/713659291 learned_addr learned my addr 172.21.15.32:0/713659291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.268+0000 7f58492cf700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5844134170 0x7f58441c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5843fff700 1 -- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 msgr2=0x7f58441c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5843fff700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 0x7f58441c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5843fff700 1 -- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5844134170 msgr2=0x7f58441c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5843fff700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5844134170 0x7f58441c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5843fff700 1 -- 172.21.15.32:0/713659291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c009a70 con 0x7f58440a1340 2023-09-30T22:24:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5848ace700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 0x7f58441c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f5843fff700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 0x7f58441c8a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f583000c470 tx=0x7f583000c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830004c80 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830004de0 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838009550 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830007310 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.269+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58440a25e0 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.270+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5830007470 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.270+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5844122d00 con 0x7f58440a1340 2023-09-30T22:24:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.270+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f583003f2e0 con 0x7f58440a1340 2023-09-30T22:24:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.279+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5830007720 con 0x7f58440a1340 2023-09-30T22:24:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.306+0000 7f173e431700 1 Processor -- start 2023-09-30T22:24:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.306+0000 7f173e431700 1 -- start start 2023-09-30T22:24:59.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173e431700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809e1d0 0x7f1738138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173e431700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1738139170 0x7f173813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173e431700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173809b120 con 0x7f173809d6f0 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173e431700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173806b460 con 0x7f1738139170 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173e431700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173809a7e0 con 0x7f173809e1d0 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173c9ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1738139170 0x7f173813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f1737fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f173809d6f0 0x7f173809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40364/0 (socket says 172.21.15.32:40364) 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f1737fff700 1 -- 172.21.15.32:0/2029855449 learned_addr learned my addr 172.21.15.32:0/2029855449 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f17377fe700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809e1d0 0x7f1738138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f1736ffd700 1 -- 172.21.15.32:0/2029855449 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012452512 0 0) 0x7f173809b120 con 0x7f173809d6f0 2023-09-30T22:24:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f1736ffd700 1 -- 172.21.15.32:0/2029855449 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17200035b0 con 0x7f173809d6f0 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173c9ce700 1 -- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809e1d0 msgr2=0x7f1738138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173c9ce700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809e1d0 0x7f1738138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173c9ce700 1 -- 172.21.15.32:0/2029855449 >> v1:172.21.15.32:6789/0 conn(0x7f173809d6f0 legacy=0x7f173809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f173c9ce700 1 -- 172.21.15.32:0/2029855449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173809a500 con 0x7f1738139170 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.307+0000 7f17377fe700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809e1d0 0x7f1738138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.308+0000 7f173c9ce700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1738139170 0x7f173813b550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1730009d90 tx=0x7f173000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40c4880d4a9be2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.308+0000 7f1736ffd700 1 -- 172.21.15.32:0/2029855449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000e070 con 0x7f1738139170 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.308+0000 7f1736ffd700 1 -- 172.21.15.32:0/2029855449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173000bb20 con 0x7f1738139170 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.308+0000 7f1736ffd700 1 -- 172.21.15.32:0/2029855449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730012430 con 0x7f1738139170 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.308+0000 7f173e431700 1 -- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1738139170 msgr2=0x7f173813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.308+0000 7f173e431700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1738139170 0x7f173813b550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1730009d90 tx=0x7f173000b330 comp rx=0 tx=0).stop 2023-09-30T22:24:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.309+0000 7f173e431700 1 -- 172.21.15.32:0/2029855449 shutdown_connections 2023-09-30T22:24:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.309+0000 7f173e431700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1738139170 0x7f173813b550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.309+0000 7f173e431700 1 --2- 172.21.15.32:0/2029855449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809e1d0 0x7f1738138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.309+0000 7f173e431700 1 -- 172.21.15.32:0/2029855449 >> 172.21.15.32:0/2029855449 conn(0x7f1738066f50 msgr2=0x7f173809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.309+0000 7f173e431700 1 -- 172.21.15.32:0/2029855449 shutdown_connections 2023-09-30T22:24:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.309+0000 7f173e431700 1 -- 172.21.15.32:0/2029855449 wait complete. 2023-09-30T22:24:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 Processor -- start 2023-09-30T22:24:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 -- start start 2023-09-30T22:24:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809d6f0 0x7f1738120750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 0x7f1738120c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1738139170 0x7f17381211d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173809c710 con 0x7f173809e1d0 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1738123ff0 con 0x7f173809d6f0 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f173e431700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1738122270 con 0x7f1738139170 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f17377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 0x7f1738120c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f17377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 0x7f1738120c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57910/0 (socket says 172.21.15.32:57910) 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f17377fe700 1 -- 172.21.15.32:0/2757740245 learned_addr learned my addr 172.21.15.32:0/2757740245 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.310+0000 7f1737fff700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809d6f0 0x7f1738120750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f173c9ce700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1738139170 0x7f17381211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f17377fe700 1 -- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1738139170 msgr2=0x7f17381211d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f17377fe700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1738139170 0x7f17381211d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f17377fe700 1 -- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809d6f0 msgr2=0x7f1738120750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f17377fe700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809d6f0 0x7f1738120750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f17377fe700 1 -- 172.21.15.32:0/2757740245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0094a0 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f173c9ce700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1738139170 0x7f17381211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f17377fe700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 0x7f1738120c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f172c00ebe0 tx=0x7f172c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c009800 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c00c910 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1730009a70 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c00cc80 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.311+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17381241f0 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.312+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f172c013470 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.312+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1738074ea0 con 0x7f173809e1d0 2023-09-30T22:24:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.312+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f172c03fcd0 con 0x7f173809e1d0 2023-09-30T22:24:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.317+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f172c013720 con 0x7f173809e1d0 2023-09-30T22:24:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.488+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f584409cac0 con 0x7f58440a1340 2023-09-30T22:24:59.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.488+0000 7f58417fa700 1 -- 172.21.15.32:0/713659291 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5830011e80 con 0x7f58440a1340 2023-09-30T22:24:59.489 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:24:59.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 msgr2=0x7f58441c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 0x7f58441c8a30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f583000c470 tx=0x7f583000c780 comp rx=0 tx=0).stop 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 shutdown_connections 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5844134170 0x7f58441c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58440a0a30 0x7f58441c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 --2- 172.21.15.32:0/713659291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58440a1340 0x7f58441c8a30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 >> 172.21.15.32:0/713659291 conn(0x7f58440a6550 msgr2=0x7f584409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 shutdown_connections 2023-09-30T22:24:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.490+0000 7f584ad32700 1 -- 172.21.15.32:0/713659291 wait complete. 2023-09-30T22:24:59.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:24:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.522+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1738123e50 con 0x7f173809e1d0 2023-09-30T22:24:59.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.522+0000 7f1734ff9700 1 -- 172.21.15.32:0/2757740245 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f172c021070 con 0x7f173809e1d0 2023-09-30T22:24:59.524 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 msgr2=0x7f1738120c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 0x7f1738120c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f172c00ebe0 tx=0x7f172c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 shutdown_connections 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1738139170 0x7f17381211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f173809d6f0 0x7f1738120750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 --2- 172.21.15.32:0/2757740245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f173809e1d0 0x7f1738120c90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 >> 172.21.15.32:0/2757740245 conn(0x7f1738066f50 msgr2=0x7f17380a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 shutdown_connections 2023-09-30T22:24:59.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.525+0000 7f173e431700 1 -- 172.21.15.32:0/2757740245 wait complete. 2023-09-30T22:24:59.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:24:59.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:24:59.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:24:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.913+0000 7fefa80f4700 1 Processor -- start 2023-09-30T22:24:59.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.913+0000 7fefa80f4700 1 -- start start 2023-09-30T22:24:59.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa80f4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefa00a0a30 0x7fefa00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa80f4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa80f4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa00a2a70 con 0x7fefa009d630 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa80f4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa00a01d0 con 0x7fefa00a0a30 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa80f4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefa009e2b0 con 0x7fefa0138d80 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa5e90700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefa00a0a30 0x7fefa00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa6691700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa568f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fefa009d630 0x7fefa00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40376/0 (socket says 172.21.15.32:40376) 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa568f700 1 -- 172.21.15.32:0/1420707286 learned_addr learned my addr 172.21.15.32:0/1420707286 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa5e90700 1 -- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 msgr2=0x7fefa013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:59.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa5e90700 1 --2- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa5e90700 1 -- 172.21.15.32:0/1420707286 >> v1:172.21.15.32:6789/0 conn(0x7fefa009d630 legacy=0x7fefa00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa5e90700 1 -- 172.21.15.32:0/1420707286 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa009dc50 con 0x7fefa00a0a30 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa4e8e700 1 -- 172.21.15.32:0/1420707286 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420579357 0 0) 0x7fefa00a2a70 con 0x7fefa009d630 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa6691700 1 --2- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.914+0000 7fefa5e90700 1 --2- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa00a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009d90 tx=0x7fef9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94abdf0aed0ba74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa4e8e700 1 -- 172.21.15.32:0/1420707286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00e070 con 0x7fefa00a0a30 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa4e8e700 1 -- 172.21.15.32:0/1420707286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00bb20 con 0x7fefa00a0a30 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa4e8e700 1 -- 172.21.15.32:0/1420707286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c012430 con 0x7fefa00a0a30 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 -- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 msgr2=0x7fefa00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa00a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009d90 tx=0x7fef9c00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:59.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 -- 172.21.15.32:0/1420707286 shutdown_connections 2023-09-30T22:24:59.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1420707286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa00a0e00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 -- 172.21.15.32:0/1420707286 >> 172.21.15.32:0/1420707286 conn(0x7fefa00a3050 msgr2=0x7fefa00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:59.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 -- 172.21.15.32:0/1420707286 shutdown_connections 2023-09-30T22:24:59.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.915+0000 7fefa80f4700 1 -- 172.21.15.32:0/1420707286 wait complete. 2023-09-30T22:24:59.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.916+0000 7fefa80f4700 1 Processor -- start 2023-09-30T22:24:59.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.916+0000 7fefa80f4700 1 -- start start 2023-09-30T22:24:59.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa80f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefa009d630 0x7fefa0133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa80f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa0133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa80f4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa01341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa80f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0136fa0 con 0x7fefa00a0a30 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa80f4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0136630 con 0x7fefa009d630 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa80f4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00a4b50 con 0x7fefa0138d80 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa0133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa5e90700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefa009d630 0x7fefa0133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa0133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57932/0 (socket says 172.21.15.32:57932) 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 -- 172.21.15.32:0/1632070464 learned_addr learned my addr 172.21.15.32:0/1632070464 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa6691700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa01341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 -- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 msgr2=0x7fefa01341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa01341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 -- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefa009d630 msgr2=0x7fefa0133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.917+0000 7fefa568f700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefa009d630 0x7fefa0133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fefa568f700 1 -- 172.21.15.32:0/1632070464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980094a0 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fefa5e90700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefa009d630 0x7fefa0133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fefa6691700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa01341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fefa568f700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa0133ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c520 tx=0x7fef9000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004c80 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90004e00 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c009a70 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90007310 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.918+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa0137a10 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.919+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef90007470 con 0x7fefa00a0a30 2023-09-30T22:24:59.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.919+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef9003f220 con 0x7fefa00a0a30 2023-09-30T22:24:59.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.919+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa0136c40 con 0x7fefa00a0a30 2023-09-30T22:24:59.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.924+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef900123b0 con 0x7fefa00a0a30 2023-09-30T22:24:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 Processor -- start 2023-09-30T22:24:59.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 -- start start 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf040a0a30 0x7faf040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf04138d80 0x7faf0413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf040a2a70 con 0x7faf0409d630 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf040a01d0 con 0x7faf04138d80 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.951+0000 7faf09136700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf0409e2b0 con 0x7faf040a0a30 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf040a0a30 0x7faf040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf04138d80 0x7faf0413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf02d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf040a0a30 0x7faf040a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:59526/0 (socket says 172.21.15.32:59526) 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faf0409d630 0x7faf040a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40390/0 (socket says 172.21.15.32:40390) 2023-09-30T22:24:59.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf02d9d700 1 -- 172.21.15.32:0/1712019025 learned_addr learned my addr 172.21.15.32:0/1712019025 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0359e700 1 -- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf040a0a30 msgr2=0x7faf040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0359e700 1 --2- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf040a0a30 0x7faf040a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0359e700 1 -- 172.21.15.32:0/1712019025 >> v1:172.21.15.32:6789/0 conn(0x7faf0409d630 legacy=0x7faf040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0359e700 1 -- 172.21.15.32:0/1712019025 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0409dc50 con 0x7faf04138d80 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf01d9b700 1 -- 172.21.15.32:0/1712019025 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871708253 0 0) 0x7faf040a2a70 con 0x7faf0409d630 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf0359e700 1 --2- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04138d80 0x7faf0413b160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faefc009d90 tx=0x7faefc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c49509db1749387 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf01d9b700 1 -- 172.21.15.32:0/1712019025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00e070 con 0x7faf04138d80 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf01d9b700 1 -- 172.21.15.32:0/1712019025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc00bb20 con 0x7faf04138d80 2023-09-30T22:24:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.952+0000 7faf01d9b700 1 -- 172.21.15.32:0/1712019025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc012430 con 0x7faf04138d80 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 -- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04138d80 msgr2=0x7faf0413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 --2- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04138d80 0x7faf0413b160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faefc009d90 tx=0x7faefc00b330 comp rx=0 tx=0).stop 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 -- 172.21.15.32:0/1712019025 shutdown_connections 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 --2- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04138d80 0x7faf0413b160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 --2- 172.21.15.32:0/1712019025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf040a0a30 0x7faf040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 -- 172.21.15.32:0/1712019025 >> 172.21.15.32:0/1712019025 conn(0x7faf040a3050 msgr2=0x7faf040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 -- 172.21.15.32:0/1712019025 shutdown_connections 2023-09-30T22:24:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.953+0000 7faf09136700 1 -- 172.21.15.32:0/1712019025 wait complete. 2023-09-30T22:24:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.955+0000 7faf09136700 1 Processor -- start 2023-09-30T22:24:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.955+0000 7faf09136700 1 -- start start 2023-09-30T22:24:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf09136700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf0409d630 0x7faf04133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf09136700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 0x7faf04133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf09136700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faf04138d80 0x7faf041342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf09136700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf04134d00 con 0x7faf040a0a30 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf09136700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf04136700 con 0x7faf04138d80 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf09136700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf04137070 con 0x7faf0409d630 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 0x7faf04133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faf04138d80 0x7faf041342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 0x7faf04133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57942/0 (socket says 172.21.15.32:57942) 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 -- 172.21.15.32:0/1406756008 learned_addr learned my addr 172.21.15.32:0/1406756008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf02d9d700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf0409d630 0x7faf04133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 -- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf0409d630 msgr2=0x7faf04133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf0409d630 0x7faf04133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 -- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faf04138d80 msgr2=0x7faf041342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faf04138d80 0x7faf041342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 -- 172.21.15.32:0/1406756008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc009a70 con 0x7faf040a0a30 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0359e700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faf04138d80 0x7faf041342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.956+0000 7faf0259c700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 0x7faf04133d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faef800c520 tx=0x7faef800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.957+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8004c80 con 0x7faf040a0a30 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.957+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8004de0 con 0x7faf040a0a30 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.957+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf040a0410 con 0x7faf040a0a30 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.957+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8007310 con 0x7faf040a0a30 2023-09-30T22:24:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.957+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf04135c20 con 0x7faf040a0a30 2023-09-30T22:24:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.958+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf04137f30 con 0x7faf040a0a30 2023-09-30T22:24:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.959+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faef8007470 con 0x7faf040a0a30 2023-09-30T22:24:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.959+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faef803f8b0 con 0x7faf040a0a30 2023-09-30T22:24:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:24:59.963+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faef80133c0 con 0x7faf040a0a30 2023-09-30T22:25:00.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.140+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fefa00747f0 con 0x7fefa00a0a30 2023-09-30T22:25:00.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.141+0000 7fef967fc700 1 -- 172.21.15.32:0/1632070464 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef90011360 con 0x7fefa00a0a30 2023-09-30T22:25:00.142 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:00.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 msgr2=0x7fefa0133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa0133ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c520 tx=0x7fef9000c830 comp rx=0 tx=0).stop 2023-09-30T22:25:00.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 shutdown_connections 2023-09-30T22:25:00.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefa0138d80 0x7fefa01341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefa009d630 0x7fefa0133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 --2- 172.21.15.32:0/1632070464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefa00a0a30 0x7fefa0133ca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 >> 172.21.15.32:0/1632070464 conn(0x7fefa00a3050 msgr2=0x7fefa013ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 shutdown_connections 2023-09-30T22:25:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.143+0000 7fefa80f4700 1 -- 172.21.15.32:0/1632070464 wait complete. 2023-09-30T22:25:00.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.169+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faf040747f0 con 0x7faf040a0a30 2023-09-30T22:25:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.170+0000 7faeeb7fe700 1 -- 172.21.15.32:0/1406756008 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faef8011360 con 0x7faf040a0a30 2023-09-30T22:25:00.170 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 msgr2=0x7faf04133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 0x7faf04133d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faef800c520 tx=0x7faef800c830 comp rx=0 tx=0).stop 2023-09-30T22:25:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 shutdown_connections 2023-09-30T22:25:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faf04138d80 0x7faf041342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faf0409d630 0x7faf04133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 --2- 172.21.15.32:0/1406756008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf040a0a30 0x7faf04133d70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 >> 172.21.15.32:0/1406756008 conn(0x7faf040a3050 msgr2=0x7faf0409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:00.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 shutdown_connections 2023-09-30T22:25:00.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.172+0000 7faf09136700 1 -- 172.21.15.32:0/1406756008 wait complete. 2023-09-30T22:25:00.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:00.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:00.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.655+0000 7feb11ed1700 1 Processor -- start 2023-09-30T22:25:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.655+0000 7feb11ed1700 1 -- start start 2023-09-30T22:25:00.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb11ed1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 0x7feb0c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb11ed1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb11ed1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c09edd0 con 0x7feb0c0a0a30 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb11ed1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0a18e0 con 0x7feb0c0a1510 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb11ed1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c09e1f0 con 0x7feb0c139250 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 0x7feb0c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c13b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49112/0 (socket says 172.21.15.32:49112) 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 -- 172.21.15.32:0/1891900664 learned_addr learned my addr 172.21.15.32:0/1891900664 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:00.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 0x7feb0c138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51298/0 (socket says 172.21.15.32:51298) 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 -- 172.21.15.32:0/1891900664 >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 msgr2=0x7feb0c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 --2- 172.21.15.32:0/1891900664 >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 0x7feb0c138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 -- 172.21.15.32:0/1891900664 >> v1:172.21.15.32:6789/0 conn(0x7feb0c0a0a30 legacy=0x7feb0c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 -- 172.21.15.32:0/1891900664 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c09dc50 con 0x7feb0c139250 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0affd700 1 --2- 172.21.15.32:0/1891900664 >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 0x7feb0c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.656+0000 7feb0bfff700 1 --2- 172.21.15.32:0/1891900664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c13b630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb040098f0 tx=0x7feb04009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d84a55de8f9544a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb0a7fc700 1 -- 172.21.15.32:0/1891900664 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04011830 con 0x7feb0c139250 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb0a7fc700 1 -- 172.21.15.32:0/1891900664 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb04011990 con 0x7feb0c139250 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb0a7fc700 1 -- 172.21.15.32:0/1891900664 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04011d20 con 0x7feb0c139250 2023-09-30T22:25:00.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 -- 172.21.15.32:0/1891900664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 msgr2=0x7feb0c13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 --2- 172.21.15.32:0/1891900664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c13b630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb040098f0 tx=0x7feb04009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 -- 172.21.15.32:0/1891900664 shutdown_connections 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 --2- 172.21.15.32:0/1891900664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c13b630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 --2- 172.21.15.32:0/1891900664 >> v2:172.21.15.32:3300/0 conn(0x7feb0c0a1510 0x7feb0c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 -- 172.21.15.32:0/1891900664 >> 172.21.15.32:0/1891900664 conn(0x7feb0c0a6550 msgr2=0x7feb0c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 -- 172.21.15.32:0/1891900664 shutdown_connections 2023-09-30T22:25:00.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.657+0000 7feb11ed1700 1 -- 172.21.15.32:0/1891900664 wait complete. 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.659+0000 7feb11ed1700 1 Processor -- start 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.659+0000 7feb11ed1700 1 -- start start 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb11ed1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 0x7feb0c1c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb11ed1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 0x7feb0c1c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb11ed1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c1c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb11ed1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c124820 con 0x7feb0c0a1510 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb11ed1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c125d40 con 0x7feb0c139250 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb11ed1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1253a0 con 0x7feb0c0a0a30 2023-09-30T22:25:00.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c1c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 0x7feb0c1c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 0x7feb0c1c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 0x7feb0c1c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51304/0 (socket says 172.21.15.32:51304) 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c1c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49120/0 (socket says 172.21.15.32:49120) 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 -- 172.21.15.32:0/2157556835 learned_addr learned my addr 172.21.15.32:0/2157556835 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 0x7feb0c1c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42806/0 (socket says 172.21.15.32:42806) 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 -- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 msgr2=0x7feb0c1c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 0x7feb0c1c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 -- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 msgr2=0x7feb0c1c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c1c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0affd700 1 -- 172.21.15.32:0/2157556835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40094a0 con 0x7feb0c0a1510 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0b7fe700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 0x7feb0c1c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.660+0000 7feb0bfff700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c1c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.661+0000 7feb0affd700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 0x7feb0c1c8a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feaf400ebe0 tx=0x7feaf400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.661+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400c850 con 0x7feb0c0a1510 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.661+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf400c9b0 con 0x7feb0c0a1510 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.661+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb04009550 con 0x7feb0c0a1510 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.661+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400cd40 con 0x7feb0c0a1510 2023-09-30T22:25:00.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.661+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1262d0 con 0x7feb0c0a1510 2023-09-30T22:25:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.662+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feaf4013470 con 0x7feb0c0a1510 2023-09-30T22:25:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.662+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c1264c0 con 0x7feb0c0a1510 2023-09-30T22:25:00.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.662+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feaf403f790 con 0x7feb0c0a1510 2023-09-30T22:25:00.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.671+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feaf400cea0 con 0x7feb0c0a1510 2023-09-30T22:25:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.695+0000 7f41a3b2b700 1 Processor -- start 2023-09-30T22:25:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a3b2b700 1 -- start start 2023-09-30T22:25:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a3b2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a3b2b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f419c138d80 0x7f419c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a3b2b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419c0a2a70 con 0x7f419c0a0a30 2023-09-30T22:25:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a3b2b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419c0a01d0 con 0x7f419c138d80 2023-09-30T22:25:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a3b2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c09e2b0 con 0x7f419c09d630 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a10c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a10c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49128/0 (socket says 172.21.15.32:49128) 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a10c6700 1 -- 172.21.15.32:0/2084676091 learned_addr learned my addr 172.21.15.32:0/2084676091 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.696+0000 7f41a20c8700 1 --2- 172.21.15.32:0/2084676091 >> v2:172.21.15.32:3300/0 conn(0x7f419c138d80 0x7f419c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a10c6700 1 -- 172.21.15.32:0/2084676091 >> v2:172.21.15.32:3300/0 conn(0x7f419c138d80 msgr2=0x7f419c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a10c6700 1 --2- 172.21.15.32:0/2084676091 >> v2:172.21.15.32:3300/0 conn(0x7f419c138d80 0x7f419c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a10c6700 1 -- 172.21.15.32:0/2084676091 >> v1:172.21.15.32:6789/0 conn(0x7f419c0a0a30 legacy=0x7f419c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a08c5700 1 -- 172.21.15.32:0/2084676091 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049858374 0 0) 0x7f419c0a2a70 con 0x7f419c0a0a30 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a10c6700 1 -- 172.21.15.32:0/2084676091 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c09dc50 con 0x7f419c09d630 2023-09-30T22:25:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a20c8700 1 --2- 172.21.15.32:0/2084676091 >> v2:172.21.15.32:3300/0 conn(0x7f419c138d80 0x7f419c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a10c6700 1 --2- 172.21.15.32:0/2084676091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c0a1510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f41900098f0 tx=0x7f4190009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31b7e02f7714e7d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a08c5700 1 -- 172.21.15.32:0/2084676091 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190011830 con 0x7f419c09d630 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a08c5700 1 -- 172.21.15.32:0/2084676091 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4190011990 con 0x7f419c09d630 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.697+0000 7f41a08c5700 1 -- 172.21.15.32:0/2084676091 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190011d20 con 0x7f419c09d630 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 -- 172.21.15.32:0/2084676091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 msgr2=0x7f419c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/2084676091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c0a1510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f41900098f0 tx=0x7f4190009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 -- 172.21.15.32:0/2084676091 shutdown_connections 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/2084676091 >> v2:172.21.15.32:3300/0 conn(0x7f419c138d80 0x7f419c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/2084676091 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c0a1510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 -- 172.21.15.32:0/2084676091 >> 172.21.15.32:0/2084676091 conn(0x7f419c0a3050 msgr2=0x7f419c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 -- 172.21.15.32:0/2084676091 shutdown_connections 2023-09-30T22:25:00.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.698+0000 7f41a3b2b700 1 -- 172.21.15.32:0/2084676091 wait complete. 2023-09-30T22:25:00.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.699+0000 7f41a3b2b700 1 Processor -- start 2023-09-30T22:25:00.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.699+0000 7f41a3b2b700 1 -- start start 2023-09-30T22:25:00.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a3b2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c1337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a3b2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f419c0a0a30 0x7f419c133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a3b2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 0x7f419c134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a3b2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c1365f0 con 0x7f419c138d80 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a3b2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c136f60 con 0x7f419c09d630 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a3b2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f419c136240 con 0x7f419c0a0a30 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a20c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 0x7f419c134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a18c7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c1337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a20c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 0x7f419c134230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51316/0 (socket says 172.21.15.32:51316) 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a20c8700 1 -- 172.21.15.32:0/168056875 learned_addr learned my addr 172.21.15.32:0/168056875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.700+0000 7f41a10c6700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f419c0a0a30 0x7f419c133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a20c8700 1 -- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f419c0a0a30 msgr2=0x7f419c133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a20c8700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f419c0a0a30 0x7f419c133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a20c8700 1 -- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 msgr2=0x7f419c1337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a20c8700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c1337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a20c8700 1 -- 172.21.15.32:0/168056875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41940094a0 con 0x7f419c138d80 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a18c7700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c1337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a20c8700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 0x7f419c134230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f419400eb60 tx=0x7f419400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194009780 con 0x7f419c138d80 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194009e60 con 0x7f419c138d80 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190009550 con 0x7f419c138d80 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400c9f0 con 0x7f419c138d80 2023-09-30T22:25:00.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.701+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c0a0410 con 0x7f419c138d80 2023-09-30T22:25:00.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.702+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f419400cb70 con 0x7f419c138d80 2023-09-30T22:25:00.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.702+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f419403f760 con 0x7f419c138d80 2023-09-30T22:25:00.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.702+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c137e20 con 0x7f419c138d80 2023-09-30T22:25:00.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.707+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41940232f0 con 0x7f419c138d80 2023-09-30T22:25:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.903+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feb0c0a1d10 con 0x7feb0c0a1510 2023-09-30T22:25:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.903+0000 7feb03fff700 1 -- 172.21.15.32:0/2157556835 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feaf40102b0 con 0x7feb0c0a1510 2023-09-30T22:25:00.904 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:00.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 msgr2=0x7feb0c1c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 0x7feb0c1c8a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feaf400ebe0 tx=0x7feaf400eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 shutdown_connections 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb0c139250 0x7feb0c1c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb0c0a0a30 0x7feb0c1c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 --2- 172.21.15.32:0/2157556835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0a1510 0x7feb0c1c8a80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.905+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 >> 172.21.15.32:0/2157556835 conn(0x7feb0c0a6550 msgr2=0x7feb0c0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.906+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 shutdown_connections 2023-09-30T22:25:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.906+0000 7feb11ed1700 1 -- 172.21.15.32:0/2157556835 wait complete. 2023-09-30T22:25:00.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:00.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.938+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f419c0747f0 con 0x7f419c138d80 2023-09-30T22:25:00.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.939+0000 7f419a7fc700 1 -- 172.21.15.32:0/168056875 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4194017020 con 0x7f419c138d80 2023-09-30T22:25:00.940 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 msgr2=0x7f419c134230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 0x7f419c134230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f419400eb60 tx=0x7f419400ee70 comp rx=0 tx=0).stop 2023-09-30T22:25:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 shutdown_connections 2023-09-30T22:25:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f419c138d80 0x7f419c134230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f419c09d630 0x7f419c1337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 --2- 172.21.15.32:0/168056875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f419c0a0a30 0x7f419c133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 >> 172.21.15.32:0/168056875 conn(0x7f419c0a3050 msgr2=0x7f419c0a6940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 shutdown_connections 2023-09-30T22:25:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:00.941+0000 7f41a3b2b700 1 -- 172.21.15.32:0/168056875 wait complete. 2023-09-30T22:25:00.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:01.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:01.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:01.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.307+0000 7f06a460e700 1 Processor -- start 2023-09-30T22:25:01.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.307+0000 7f06a460e700 1 -- start start 2023-09-30T22:25:01.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a460e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a460e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f069c131d10 0x7f069c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a460e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c09ece0 con 0x7f069c06b080 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a460e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c09e760 con 0x7f069c131d10 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a460e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f069c09e040 con 0x7f069c06a770 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a23aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a2bab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f069c131d10 0x7f069c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a23aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42824/0 (socket says 172.21.15.32:42824) 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.308+0000 7f06a23aa700 1 -- 172.21.15.32:0/3449197363 learned_addr learned my addr 172.21.15.32:0/3449197363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a2bab700 1 -- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 msgr2=0x7f069c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a2bab700 1 --2- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a2bab700 1 -- 172.21.15.32:0/3449197363 >> v1:172.21.15.32:6789/0 conn(0x7f069c06b080 legacy=0x7f069c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a2bab700 1 -- 172.21.15.32:0/3449197363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c09f630 con 0x7f069c131d10 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a13a8700 1 -- 172.21.15.32:0/3449197363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540828876 0 0) 0x7f069c09ece0 con 0x7f069c06b080 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a23aa700 1 --2- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a2bab700 1 --2- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c131d10 0x7f069c139860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0694009d90 tx=0x7f069400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a715513434bff53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a13a8700 1 -- 172.21.15.32:0/3449197363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069400e070 con 0x7f069c131d10 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a13a8700 1 -- 172.21.15.32:0/3449197363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069400bb20 con 0x7f069c131d10 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.309+0000 7f06a13a8700 1 -- 172.21.15.32:0/3449197363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694012430 con 0x7f069c131d10 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 -- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c131d10 msgr2=0x7f069c139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 --2- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c131d10 0x7f069c139860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0694009d90 tx=0x7f069400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 -- 172.21.15.32:0/3449197363 shutdown_connections 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 --2- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c131d10 0x7f069c139860 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 --2- 172.21.15.32:0/3449197363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 -- 172.21.15.32:0/3449197363 >> 172.21.15.32:0/3449197363 conn(0x7f069c066f50 msgr2=0x7f069c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 -- 172.21.15.32:0/3449197363 shutdown_connections 2023-09-30T22:25:01.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.310+0000 7f06a460e700 1 -- 172.21.15.32:0/3449197363 wait complete. 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.311+0000 7f06a460e700 1 Processor -- start 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.311+0000 7f06a460e700 1 -- start start 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a460e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c1c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a460e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c06b080 0x7f069c1c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a460e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 0x7f069c120410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a460e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c1c6db0 con 0x7f069c06b080 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a460e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c124b10 con 0x7f069c131d10 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a460e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f069c071e60 con 0x7f069c06a770 2023-09-30T22:25:01.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a2bab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 0x7f069c120410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a1ba9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c06b080 0x7f069c1c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a2bab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 0x7f069c120410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49136/0 (socket says 172.21.15.32:49136) 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a2bab700 1 -- 172.21.15.32:0/1386154223 learned_addr learned my addr 172.21.15.32:0/1386154223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.312+0000 7f06a23aa700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c1c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a23aa700 1 -- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 msgr2=0x7f069c120410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a23aa700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 0x7f069c120410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a23aa700 1 -- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c06b080 msgr2=0x7f069c1c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a23aa700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c06b080 0x7f069c1c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a23aa700 1 -- 172.21.15.32:0/1386154223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c009550 con 0x7f069c06a770 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a1ba9700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c06b080 0x7f069c1c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a2bab700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 0x7f069c120410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:01.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a23aa700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c1c6330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e9d0 tx=0x7f068c00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00ee70 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c004030 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0694009a70 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c0043c0 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.313+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c131600 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.314+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f068c00c850 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.314+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f068c03f520 con 0x7f069c06a770 2023-09-30T22:25:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.314+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c074ea0 con 0x7f069c06a770 2023-09-30T22:25:01.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.319+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f068c019d40 con 0x7f069c06a770 2023-09-30T22:25:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.345+0000 7ff58f106700 1 Processor -- start 2023-09-30T22:25:01.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.345+0000 7ff58f106700 1 -- start start 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58f106700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 0x7ff5880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58f106700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff588134170 0x7ff58813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58f106700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58809edd0 con 0x7ff5880a1340 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58f106700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58809e1f0 con 0x7ff588134170 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58f106700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff58809dc50 con 0x7ff5880a0a30 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58cea2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 0x7ff5880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58cea2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 0x7ff5880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42840/0 (socket says 172.21.15.32:42840) 2023-09-30T22:25:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58d6a3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff588134170 0x7ff58813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff58cea2700 1 -- 172.21.15.32:0/1014390166 learned_addr learned my addr 172.21.15.32:0/1014390166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.346+0000 7ff587fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff5880a1340 0x7ff588133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44770/0 (socket says 172.21.15.32:44770) 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff58cea2700 1 -- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff588134170 msgr2=0x7ff58813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff5877fe700 1 -- 172.21.15.32:0/1014390166 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113077502 0 0) 0x7ff58809edd0 con 0x7ff5880a1340 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff58cea2700 1 --2- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff588134170 0x7ff58813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff58cea2700 1 -- 172.21.15.32:0/1014390166 >> v1:172.21.15.32:6789/0 conn(0x7ff5880a1340 legacy=0x7ff588133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff58cea2700 1 -- 172.21.15.32:0/1014390166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880a1fe0 con 0x7ff5880a0a30 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff58cea2700 1 --2- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 0x7ff5880a0e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009920 tx=0x7ff57c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40837f1ff046bef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff5877fe700 1 -- 172.21.15.32:0/1014390166 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c011830 con 0x7ff5880a0a30 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff5877fe700 1 -- 172.21.15.32:0/1014390166 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c011990 con 0x7ff5880a0a30 2023-09-30T22:25:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.347+0000 7ff5877fe700 1 -- 172.21.15.32:0/1014390166 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c011d20 con 0x7ff5880a0a30 2023-09-30T22:25:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 -- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 msgr2=0x7ff5880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 --2- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 0x7ff5880a0e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009920 tx=0x7ff57c009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 -- 172.21.15.32:0/1014390166 shutdown_connections 2023-09-30T22:25:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 --2- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff588134170 0x7ff58813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 --2- 172.21.15.32:0/1014390166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a0a30 0x7ff5880a0e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 -- 172.21.15.32:0/1014390166 >> 172.21.15.32:0/1014390166 conn(0x7ff5880a6550 msgr2=0x7ff5880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 -- 172.21.15.32:0/1014390166 shutdown_connections 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.348+0000 7ff58f106700 1 -- 172.21.15.32:0/1014390166 wait complete. 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 Processor -- start 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 -- start start 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 0x7ff588122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a1340 0x7ff588122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff588134170 0x7ff5881232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58809bd90 con 0x7ff588134170 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881332f0 con 0x7ff5880a0a30 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.349+0000 7ff58f106700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881244f0 con 0x7ff5880a1340 2023-09-30T22:25:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 0x7ff588122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff587fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a1340 0x7ff588122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 0x7ff588122860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49146/0 (socket says 172.21.15.32:49146) 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 -- 172.21.15.32:0/2579747037 learned_addr learned my addr 172.21.15.32:0/2579747037 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58d6a3700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff588134170 0x7ff5881232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 -- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a1340 msgr2=0x7ff588122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a1340 0x7ff588122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 -- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff588134170 msgr2=0x7ff5881232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff588134170 0x7ff5881232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 -- 172.21.15.32:0/2579747037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c009550 con 0x7ff5880a0a30 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff587fff700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a1340 0x7ff588122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58d6a3700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff588134170 0x7ff5881232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58cea2700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 0x7ff588122860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009130 tx=0x7ff57c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004450 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c01f070 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0038c0 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5880a25e0 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.350+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff588126010 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.351+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff57c003a20 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.351+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff57c021070 con 0x7ff5880a0a30 2023-09-30T22:25:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.351+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff588074ea0 con 0x7ff5880a0a30 2023-09-30T22:25:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.356+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff57c02b080 con 0x7ff5880a0a30 2023-09-30T22:25:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.525+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f069c130f10 con 0x7f069c06a770 2023-09-30T22:25:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.525+0000 7f069affd700 1 -- 172.21.15.32:0/1386154223 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f068c019300 con 0x7f069c06a770 2023-09-30T22:25:01.526 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:01.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 msgr2=0x7f069c1c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c1c6330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e9d0 tx=0x7f068c00ece0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 shutdown_connections 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f069c131d10 0x7f069c120410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f069c06a770 0x7f069c1c6330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 --2- 172.21.15.32:0/1386154223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f069c06b080 0x7f069c1c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 >> 172.21.15.32:0/1386154223 conn(0x7f069c066f50 msgr2=0x7f069c070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 shutdown_connections 2023-09-30T22:25:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.527+0000 7f06a460e700 1 -- 172.21.15.32:0/1386154223 wait complete. 2023-09-30T22:25:01.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.562+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff5880a28e0 con 0x7ff5880a0a30 2023-09-30T22:25:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.562+0000 7ff5857fa700 1 -- 172.21.15.32:0/2579747037 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff57c015020 con 0x7ff5880a0a30 2023-09-30T22:25:01.563 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 msgr2=0x7ff588122860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 0x7ff588122860 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009130 tx=0x7ff57c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 shutdown_connections 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff588134170 0x7ff5881232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5880a0a30 0x7ff588122860 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 --2- 172.21.15.32:0/2579747037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5880a1340 0x7ff588122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 >> 172.21.15.32:0/2579747037 conn(0x7ff5880a6550 msgr2=0x7ff58809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:01.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 shutdown_connections 2023-09-30T22:25:01.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:01.565+0000 7ff58f106700 1 -- 172.21.15.32:0/2579747037 wait complete. 2023-09-30T22:25:01.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:01.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:01.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 Processor -- start 2023-09-30T22:25:02.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 -- start start 2023-09-30T22:25:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe4640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe46409d630 0x7fe4640a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4640a2a70 con 0x7fe464133930 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640a01d0 con 0x7fe4640a0a30 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.069+0000 7fe46ab8e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe46409e2b0 con 0x7fe46409d630 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe463fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe46409d630 0x7fe4640a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe46912b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe464133930 0x7fe464133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44784/0 (socket says 172.21.15.32:44784) 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe46912b700 1 -- 172.21.15.32:0/3703139481 learned_addr learned my addr 172.21.15.32:0/3703139481 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe46892a700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe4640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe4637fe700 1 -- 172.21.15.32:0/3703139481 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003149758 0 0) 0x7fe4640a2a70 con 0x7fe464133930 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe463fff700 1 -- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 msgr2=0x7fe4640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe463fff700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe4640a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe463fff700 1 -- 172.21.15.32:0/3703139481 >> v1:172.21.15.32:6789/0 conn(0x7fe464133930 legacy=0x7fe464133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe463fff700 1 -- 172.21.15.32:0/3703139481 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46409dc50 con 0x7fe46409d630 2023-09-30T22:25:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe46892a700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe4640a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe463fff700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe46409d630 0x7fe4640a1340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe458009870 tx=0x7fe458009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad5dd6dd221b963 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe4637fe700 1 -- 172.21.15.32:0/3703139481 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458011830 con 0x7fe46409d630 2023-09-30T22:25:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe4637fe700 1 -- 172.21.15.32:0/3703139481 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458011990 con 0x7fe46409d630 2023-09-30T22:25:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.070+0000 7fe4637fe700 1 -- 172.21.15.32:0/3703139481 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458011d20 con 0x7fe46409d630 2023-09-30T22:25:02.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 -- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe46409d630 msgr2=0x7fe4640a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe46409d630 0x7fe4640a1340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe458009870 tx=0x7fe458009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 -- 172.21.15.32:0/3703139481 shutdown_connections 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe4640a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/3703139481 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe46409d630 0x7fe4640a1340 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 -- 172.21.15.32:0/3703139481 >> 172.21.15.32:0/3703139481 conn(0x7fe4640a3050 msgr2=0x7fe4640a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 -- 172.21.15.32:0/3703139481 shutdown_connections 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.071+0000 7fe46ab8e700 1 -- 172.21.15.32:0/3703139481 wait complete. 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.072+0000 7fe46ab8e700 1 Processor -- start 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.072+0000 7fe46ab8e700 1 -- start start 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46ab8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe46409d630 0x7fe46411f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46ab8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe46411f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46ab8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe464133930 0x7fe4641c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46ab8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46411ed60 con 0x7fe46409d630 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46ab8e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4641224a0 con 0x7fe4640a0a30 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46ab8e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe464120490 con 0x7fe464133930 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46912b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe464133930 0x7fe4641c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe463fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe46411f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46912b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe464133930 0x7fe4641c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42872/0 (socket says 172.21.15.32:42872) 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46912b700 1 -- 172.21.15.32:0/826600547 learned_addr learned my addr 172.21.15.32:0/826600547 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46892a700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe46409d630 0x7fe46411f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe463fff700 1 -- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe464133930 msgr2=0x7fe4641c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe463fff700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe464133930 0x7fe4641c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe463fff700 1 -- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe46409d630 msgr2=0x7fe46411f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe463fff700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe46409d630 0x7fe46411f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe463fff700 1 -- 172.21.15.32:0/826600547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540094a0 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.073+0000 7fe46892a700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe46409d630 0x7fe46411f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.074+0000 7fe463fff700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe46411f820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe458016000 tx=0x7fe458004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.074+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458004300 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.074+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45801f070 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.074+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4580036a0 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.074+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4580094a0 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.074+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4640a0410 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.075+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe458021030 con 0x7fe4640a0a30 2023-09-30T22:25:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.075+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe4580405f0 con 0x7fe4640a0a30 2023-09-30T22:25:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.075+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46411fd60 con 0x7fe4640a0a30 2023-09-30T22:25:02.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.080+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe458015080 con 0x7fe4640a0a30 2023-09-30T22:25:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.110+0000 7fe5e4a7a700 1 Processor -- start 2023-09-30T22:25:02.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.110+0000 7fe5e4a7a700 1 -- start start 2023-09-30T22:25:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5e4a7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009d6f0 0x7fe5e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5e4a7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e000 0x7fe5e01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5e4a7a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e009e410 0x7fe5e0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5e4a7a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e006b460 con 0x7fe5e009e410 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5e4a7a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e009b120 con 0x7fe5e009d6f0 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5e4a7a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e009a7e0 con 0x7fe5e009e000 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5de59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009d6f0 0x7fe5e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ddd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e000 0x7fe5e01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5de59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009d6f0 0x7fe5e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49172/0 (socket says 172.21.15.32:49172) 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ded9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e009e410 0x7fe5e0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5de59c700 1 -- 172.21.15.32:0/3735518761 learned_addr learned my addr 172.21.15.32:0/3735518761 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ddd9b700 1 -- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009d6f0 msgr2=0x7fe5e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ddd9b700 1 --2- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009d6f0 0x7fe5e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ddd9b700 1 -- 172.21.15.32:0/3735518761 >> v2:172.21.15.32:3300/0 conn(0x7fe5e009e410 msgr2=0x7fe5e0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:02.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ddd9b700 1 --2- 172.21.15.32:0/3735518761 >> v2:172.21.15.32:3300/0 conn(0x7fe5e009e410 0x7fe5e0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ddd9b700 1 -- 172.21.15.32:0/3735518761 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e009a500 con 0x7fe5e009e000 2023-09-30T22:25:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.111+0000 7fe5ded9d700 1 --2- 172.21.15.32:0/3735518761 >> v2:172.21.15.32:3300/0 conn(0x7fe5e009e410 0x7fe5e0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:02.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.112+0000 7fe5ddd9b700 1 --2- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e000 0x7fe5e01339f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009870 tx=0x7fe5d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb40ca894861b583 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.112+0000 7fe5dd59a700 1 -- 172.21.15.32:0/3735518761 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4011830 con 0x7fe5e009e000 2023-09-30T22:25:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.112+0000 7fe5dd59a700 1 -- 172.21.15.32:0/3735518761 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4011990 con 0x7fe5e009e000 2023-09-30T22:25:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.112+0000 7fe5dd59a700 1 -- 172.21.15.32:0/3735518761 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4011d20 con 0x7fe5e009e000 2023-09-30T22:25:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e000 msgr2=0x7fe5e01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e000 0x7fe5e01339f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009870 tx=0x7fe5d4009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:02.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/3735518761 shutdown_connections 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009d6f0 0x7fe5e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/3735518761 >> v2:172.21.15.32:3300/0 conn(0x7fe5e009e410 0x7fe5e0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/3735518761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e000 0x7fe5e01339f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/3735518761 >> 172.21.15.32:0/3735518761 conn(0x7fe5e0066f50 msgr2=0x7fe5e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/3735518761 shutdown_connections 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.113+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/3735518761 wait complete. 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.114+0000 7fe5e4a7a700 1 Processor -- start 2023-09-30T22:25:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.114+0000 7fe5e4a7a700 1 -- start start 2023-09-30T22:25:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5e4a7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 0x7fe5e01c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5e4a7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009e000 0x7fe5e01c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5e4a7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e410 0x7fe5e01c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5e4a7a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e01aac60 con 0x7fe5e009d6f0 2023-09-30T22:25:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5e4a7a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e009a0c0 con 0x7fe5e009e000 2023-09-30T22:25:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5e4a7a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e009c710 con 0x7fe5e009e410 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 0x7fe5e01c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5ded9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e410 0x7fe5e01c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 0x7fe5e01c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51358/0 (socket says 172.21.15.32:51358) 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 -- 172.21.15.32:0/1367767753 learned_addr learned my addr 172.21.15.32:0/1367767753 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5ddd9b700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009e000 0x7fe5e01c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 -- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e410 msgr2=0x7fe5e01c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e410 0x7fe5e01c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 -- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009e000 msgr2=0x7fe5e01c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009e000 0x7fe5e01c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.115+0000 7fe5de59c700 1 -- 172.21.15.32:0/1367767753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8009a70 con 0x7fe5e009d6f0 2023-09-30T22:25:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.116+0000 7fe5de59c700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 0x7fe5e01c8960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e910 tx=0x7fe5d000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.116+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00096a0 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.116+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0013680 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.116+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40094a0 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.116+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0013a30 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.116+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e01aa020 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.117+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5d0013b90 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.117+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e01aa500 con 0x7fe5e009d6f0 2023-09-30T22:25:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.117+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5d003f490 con 0x7fe5e009d6f0 2023-09-30T22:25:02.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.122+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5e01aa500 con 0x7fe5e009d6f0 2023-09-30T22:25:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.293+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe4641222e0 con 0x7fe4640a0a30 2023-09-30T22:25:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.293+0000 7fe4617fa700 1 -- 172.21.15.32:0/826600547 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4641222e0 con 0x7fe4640a0a30 2023-09-30T22:25:02.295 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 msgr2=0x7fe46411f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe46411f820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe458016000 tx=0x7fe458004750 comp rx=0 tx=0).stop 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 shutdown_connections 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe464133930 0x7fe4641c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe46409d630 0x7fe46411f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 --2- 172.21.15.32:0/826600547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4640a0a30 0x7fe46411f820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 >> 172.21.15.32:0/826600547 conn(0x7fe4640a3050 msgr2=0x7fe4641bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 shutdown_connections 2023-09-30T22:25:02.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.296+0000 7fe46ab8e700 1 -- 172.21.15.32:0/826600547 wait complete. 2023-09-30T22:25:02.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:02.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.332+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5e01aa020 con 0x7fe5e009d6f0 2023-09-30T22:25:02.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.333+0000 7fe5ceffd700 1 -- 172.21.15.32:0/1367767753 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5e01aa020 con 0x7fe5e009d6f0 2023-09-30T22:25:02.334 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 msgr2=0x7fe5e01c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 0x7fe5e01c8960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e910 tx=0x7fe5d000ec20 comp rx=0 tx=0).stop 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 shutdown_connections 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e009e410 0x7fe5e01c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e009d6f0 0x7fe5e01c8960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 --2- 172.21.15.32:0/1367767753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e009e000 0x7fe5e01c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 >> 172.21.15.32:0/1367767753 conn(0x7fe5e0066f50 msgr2=0x7fe5e006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 shutdown_connections 2023-09-30T22:25:02.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.335+0000 7fe5e4a7a700 1 -- 172.21.15.32:0/1367767753 wait complete. 2023-09-30T22:25:02.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:02.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:02.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.691+0000 7fd528a11700 1 Processor -- start 2023-09-30T22:25:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.691+0000 7fd528a11700 1 -- start start 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd528a11700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a0a30 0x7fd5240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd528a11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5240a1340 0x7fd524133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd528a11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd524133fa0 0x7fd524134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd528a11700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52409edd0 con 0x7fd5240a1340 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd528a11700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52409e1f0 con 0x7fd5240a0a30 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd528a11700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd52409dc50 con 0x7fd524133fa0 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd521d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5240a1340 0x7fd524133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd522d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd524133fa0 0x7fd524134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd52259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a0a30 0x7fd5240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd521d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5240a1340 0x7fd524133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51364/0 (socket says 172.21.15.32:51364) 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.692+0000 7fd521d9b700 1 -- 172.21.15.32:0/303566958 learned_addr learned my addr 172.21.15.32:0/303566958 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd521d9b700 1 -- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd524133fa0 msgr2=0x7fd524134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd521d9b700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd524133fa0 0x7fd524134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd521d9b700 1 -- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a0a30 msgr2=0x7fd5240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd521d9b700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a0a30 0x7fd5240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd521d9b700 1 -- 172.21.15.32:0/303566958 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240a1fe0 con 0x7fd5240a1340 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd522d9d700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd524133fa0 0x7fd524134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd521d9b700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5240a1340 0x7fd524133a60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098f0 tx=0x7fd518009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf38e440a3803ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd52159a700 1 -- 172.21.15.32:0/303566958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011830 con 0x7fd5240a1340 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd52159a700 1 -- 172.21.15.32:0/303566958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518011990 con 0x7fd5240a1340 2023-09-30T22:25:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.693+0000 7fd52159a700 1 -- 172.21.15.32:0/303566958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518011d20 con 0x7fd5240a1340 2023-09-30T22:25:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 -- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5240a1340 msgr2=0x7fd524133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5240a1340 0x7fd524133a60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098f0 tx=0x7fd518009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 -- 172.21.15.32:0/303566958 shutdown_connections 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a0a30 0x7fd5240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd524133fa0 0x7fd524134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 --2- 172.21.15.32:0/303566958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5240a1340 0x7fd524133a60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 -- 172.21.15.32:0/303566958 >> 172.21.15.32:0/303566958 conn(0x7fd5240a6550 msgr2=0x7fd5240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 -- 172.21.15.32:0/303566958 shutdown_connections 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.694+0000 7fd528a11700 1 -- 172.21.15.32:0/303566958 wait complete. 2023-09-30T22:25:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.696+0000 7fd528a11700 1 Processor -- start 2023-09-30T22:25:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.696+0000 7fd528a11700 1 -- start start 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd528a11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5240a0a30 0x7fd524121560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd528a11700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a1340 0x7fd524121aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd528a11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd524133fa0 0x7fd524121fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd528a11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524124ff0 con 0x7fd524133fa0 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd528a11700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5241230e0 con 0x7fd5240a1340 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd528a11700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240a3850 con 0x7fd5240a0a30 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd52259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5240a0a30 0x7fd524121560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a1340 0x7fd524121aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd52259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5240a0a30 0x7fd524121560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42916/0 (socket says 172.21.15.32:42916) 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd52259c700 1 -- 172.21.15.32:0/1135764611 learned_addr learned my addr 172.21.15.32:0/1135764611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd522d9d700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd524133fa0 0x7fd524121fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 -- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5240a0a30 msgr2=0x7fd524121560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5240a0a30 0x7fd524121560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 -- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd524133fa0 msgr2=0x7fd524121fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd524133fa0 0x7fd524121fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 -- 172.21.15.32:0/1135764611 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c009a70 con 0x7fd5240a1340 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.697+0000 7fd521d9b700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a1340 0x7fd524121aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098f0 tx=0x7fd518004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.698+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518004450 con 0x7fd5240a1340 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.698+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51801e070 con 0x7fd5240a1340 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.698+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd518009550 con 0x7fd5240a1340 2023-09-30T22:25:02.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.698+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180038c0 con 0x7fd5240a1340 2023-09-30T22:25:02.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.698+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd524131ff0 con 0x7fd5240a1340 2023-09-30T22:25:02.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.699+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd518003a20 con 0x7fd5240a1340 2023-09-30T22:25:02.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.699+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd5180402e0 con 0x7fd5240a1340 2023-09-30T22:25:02.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.699+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5241249b0 con 0x7fd5240a1340 2023-09-30T22:25:02.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.704+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5241249b0 con 0x7fd5240a1340 2023-09-30T22:25:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 Processor -- start 2023-09-30T22:25:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 -- start start 2023-09-30T22:25:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a84138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a84138dc0 0x7f3a8413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8406b990 con 0x7f3a84133420 2023-09-30T22:25:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8409b0e0 con 0x7f3a84138dc0 2023-09-30T22:25:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.735+0000 7f3a8a2e9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84133c20 con 0x7f3a841337f0 2023-09-30T22:25:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a84138dc0 0x7f3a8413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a83fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a84133420 0x7f3a84133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44792/0 (socket says 172.21.15.32:44792) 2023-09-30T22:25:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a837fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a84138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a83fff700 1 -- 172.21.15.32:0/3635557145 learned_addr learned my addr 172.21.15.32:0/3635557145 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3a84138dc0 0x7f3a8413b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51374/0 (socket says 172.21.15.32:51374) 2023-09-30T22:25:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 -- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 msgr2=0x7f3a84138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 --2- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a84138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 -- 172.21.15.32:0/3635557145 >> v1:172.21.15.32:6789/0 conn(0x7f3a84133420 legacy=0x7f3a84133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a82ffd700 1 -- 172.21.15.32:0/3635557145 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561432187 0 0) 0x7f3a8406b990 con 0x7f3a84133420 2023-09-30T22:25:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 -- 172.21.15.32:0/3635557145 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840a0590 con 0x7f3a84138dc0 2023-09-30T22:25:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a837fe700 1 --2- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a84138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a88886700 1 --2- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84138dc0 0x7f3a8413b1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009d90 tx=0x7f3a7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5624b6232fd22fb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a82ffd700 1 -- 172.21.15.32:0/3635557145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00e070 con 0x7f3a84138dc0 2023-09-30T22:25:02.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a82ffd700 1 -- 172.21.15.32:0/3635557145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c00bb20 con 0x7f3a84138dc0 2023-09-30T22:25:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.736+0000 7f3a82ffd700 1 -- 172.21.15.32:0/3635557145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c012430 con 0x7f3a84138dc0 2023-09-30T22:25:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84138dc0 msgr2=0x7f3a8413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84138dc0 0x7f3a8413b1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009d90 tx=0x7f3a7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/3635557145 shutdown_connections 2023-09-30T22:25:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84138dc0 0x7f3a8413b1a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/3635557145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a84138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/3635557145 >> 172.21.15.32:0/3635557145 conn(0x7f3a840a4210 msgr2=0x7f3a840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/3635557145 shutdown_connections 2023-09-30T22:25:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.737+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/3635557145 wait complete. 2023-09-30T22:25:02.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.738+0000 7f3a8a2e9700 1 Processor -- start 2023-09-30T22:25:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.738+0000 7f3a8a2e9700 1 -- start start 2023-09-30T22:25:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a8a2e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 0x7f3a8412f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a8a2e9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a8412f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a8a2e9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a84138dc0 0x7f3a8412fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:02.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a8a2e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8409c4e0 con 0x7f3a84133420 2023-09-30T22:25:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a8a2e9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8409b430 con 0x7f3a84138dc0 2023-09-30T22:25:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a8a2e9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8409d600 con 0x7f3a841337f0 2023-09-30T22:25:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 0x7f3a8412f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a837fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a8412f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 0x7f3a8412f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51376/0 (socket says 172.21.15.32:51376) 2023-09-30T22:25:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a837fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a8412f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42926/0 (socket says 172.21.15.32:42926) 2023-09-30T22:25:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 -- 172.21.15.32:0/1228576051 learned_addr learned my addr 172.21.15.32:0/1228576051 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a88886700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a84138dc0 0x7f3a8412fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 -- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 msgr2=0x7f3a8412f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a8412f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 -- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a84138dc0 msgr2=0x7f3a8412fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a84138dc0 0x7f3a8412fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.739+0000 7f3a83fff700 1 -- 172.21.15.32:0/1228576051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780094a0 con 0x7f3a84133420 2023-09-30T22:25:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.740+0000 7f3a83fff700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 0x7f3a8412f0b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c520 tx=0x7f3a7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:02.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.740+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74004c80 con 0x7f3a84133420 2023-09-30T22:25:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.740+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a74004e00 con 0x7f3a84133420 2023-09-30T22:25:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.740+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c009a70 con 0x7f3a84133420 2023-09-30T22:25:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.740+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74007310 con 0x7f3a84133420 2023-09-30T22:25:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.740+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a84130b10 con 0x7f3a84133420 2023-09-30T22:25:02.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.741+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a74007470 con 0x7f3a84133420 2023-09-30T22:25:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.741+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a7403f290 con 0x7f3a84133420 2023-09-30T22:25:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.741+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a84074ea0 con 0x7f3a84133420 2023-09-30T22:25:02.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.746+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a740123b0 con 0x7f3a84133420 2023-09-30T22:25:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.913+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd5241249b0 con 0x7fd5240a1340 2023-09-30T22:25:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.913+0000 7fd512ffd700 1 -- 172.21.15.32:0/1135764611 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5241249b0 con 0x7fd5240a1340 2023-09-30T22:25:02.914 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:02.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.915+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a1340 msgr2=0x7fd524121aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.915+0000 7fd528a11700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a1340 0x7fd524121aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5180098f0 tx=0x7fd518004750 comp rx=0 tx=0).stop 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 shutdown_connections 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd524133fa0 0x7fd524121fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5240a0a30 0x7fd524121560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 --2- 172.21.15.32:0/1135764611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5240a1340 0x7fd524121aa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 >> 172.21.15.32:0/1135764611 conn(0x7fd5240a6550 msgr2=0x7fd5240a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 shutdown_connections 2023-09-30T22:25:02.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.916+0000 7fd528a11700 1 -- 172.21.15.32:0/1135764611 wait complete. 2023-09-30T22:25:02.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:02.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.952+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3a8406b250 con 0x7f3a84133420 2023-09-30T22:25:02.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.953+0000 7f3a80ff9700 1 -- 172.21.15.32:0/1228576051 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a74011360 con 0x7f3a84133420 2023-09-30T22:25:02.954 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:02.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 msgr2=0x7f3a8412f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:02.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 0x7f3a8412f0b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400c520 tx=0x7f3a7400c830 comp rx=0 tx=0).stop 2023-09-30T22:25:02.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 shutdown_connections 2023-09-30T22:25:02.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a84138dc0 0x7f3a8412fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a84133420 0x7f3a8412f0b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 --2- 172.21.15.32:0/1228576051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a841337f0 0x7f3a8412f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 >> 172.21.15.32:0/1228576051 conn(0x7f3a840a4210 msgr2=0x7f3a840a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 shutdown_connections 2023-09-30T22:25:02.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:02.955+0000 7f3a8a2e9700 1 -- 172.21.15.32:0/1228576051 wait complete. 2023-09-30T22:25:02.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:03.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:03.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:03.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.462+0000 7f306f0eb700 1 Processor -- start 2023-09-30T22:25:03.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.462+0000 7f306f0eb700 1 -- start start 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.462+0000 7f306f0eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 0x7f30680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.462+0000 7f306f0eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30680a1340 0x7f3068133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306f0eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306809edd0 con 0x7f3068133fa0 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306f0eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306809e1f0 con 0x7f30680a1340 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306f0eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f306809dc50 con 0x7f30680a0a30 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 0x7f30680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 0x7f30680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42942/0 (socket says 172.21.15.32:42942) 2023-09-30T22:25:03.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 -- 172.21.15.32:0/3736321835 learned_addr learned my addr 172.21.15.32:0/3736321835 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f3067fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30680a1340 0x7f3068133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 -- 172.21.15.32:0/3736321835 >> v2:172.21.15.32:3300/0 conn(0x7f30680a1340 msgr2=0x7f3068133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 --2- 172.21.15.32:0/3736321835 >> v2:172.21.15.32:3300/0 conn(0x7f30680a1340 0x7f3068133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 -- 172.21.15.32:0/3736321835 >> v1:172.21.15.32:6789/0 conn(0x7f3068133fa0 legacy=0x7f306813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 -- 172.21.15.32:0/3736321835 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680a1fe0 con 0x7f30680a0a30 2023-09-30T22:25:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f306ce87700 1 --2- 172.21.15.32:0/3736321835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 0x7f30680a0e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f305c0098f0 tx=0x7f305c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c742f9aa5f218605 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:03.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.463+0000 7f30677fe700 1 -- 172.21.15.32:0/3736321835 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011830 con 0x7f30680a0a30 2023-09-30T22:25:03.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.464+0000 7f30677fe700 1 -- 172.21.15.32:0/3736321835 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c011990 con 0x7f30680a0a30 2023-09-30T22:25:03.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.464+0000 7f30677fe700 1 -- 172.21.15.32:0/3736321835 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011d20 con 0x7f30680a0a30 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.464+0000 7f306f0eb700 1 -- 172.21.15.32:0/3736321835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 msgr2=0x7f30680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.464+0000 7f306f0eb700 1 --2- 172.21.15.32:0/3736321835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 0x7f30680a0e00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f305c0098f0 tx=0x7f305c009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.465+0000 7f306f0eb700 1 -- 172.21.15.32:0/3736321835 shutdown_connections 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.465+0000 7f306f0eb700 1 --2- 172.21.15.32:0/3736321835 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a0a30 0x7f30680a0e00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.465+0000 7f306f0eb700 1 --2- 172.21.15.32:0/3736321835 >> v2:172.21.15.32:3300/0 conn(0x7f30680a1340 0x7f3068133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.465+0000 7f306f0eb700 1 -- 172.21.15.32:0/3736321835 >> 172.21.15.32:0/3736321835 conn(0x7f30680a6550 msgr2=0x7f30680a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.465+0000 7f306f0eb700 1 -- 172.21.15.32:0/3736321835 shutdown_connections 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.465+0000 7f306f0eb700 1 -- 172.21.15.32:0/3736321835 wait complete. 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.466+0000 7f306f0eb700 1 Processor -- start 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.466+0000 7f306f0eb700 1 -- start start 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306f0eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30680a0a30 0x7f30681c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306f0eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a1340 0x7f30681c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306f0eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 0x7f306811e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306f0eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30681bd3a0 con 0x7f3068133fa0 2023-09-30T22:25:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306f0eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30681333d0 con 0x7f30680a0a30 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306f0eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3068120390 con 0x7f30680a1340 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306d688700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 0x7f306811e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f3067fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a1340 0x7f30681c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306d688700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 0x7f306811e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51402/0 (socket says 172.21.15.32:51402) 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306d688700 1 -- 172.21.15.32:0/1391256372 learned_addr learned my addr 172.21.15.32:0/1391256372 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.467+0000 7f306ce87700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30680a0a30 0x7f30681c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306d688700 1 -- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a1340 msgr2=0x7f30681c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306d688700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a1340 0x7f30681c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306d688700 1 -- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30680a0a30 msgr2=0x7f30681c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306d688700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30680a0a30 0x7f30681c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306d688700 1 -- 172.21.15.32:0/1391256372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009550 con 0x7f3068133fa0 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306d688700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 0x7f306811e8c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f306000cc50 tx=0x7f306000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060003c00 con 0x7f3068133fa0 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060003d60 con 0x7f3068133fa0 2023-09-30T22:25:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30680a3820 con 0x7f3068133fa0 2023-09-30T22:25:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000aa50 con 0x7f3068133fa0 2023-09-30T22:25:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.468+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068122aa0 con 0x7f3068133fa0 2023-09-30T22:25:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.469+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f306000abb0 con 0x7f3068133fa0 2023-09-30T22:25:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.469+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f306003f700 con 0x7f3068133fa0 2023-09-30T22:25:03.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.469+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30680a1d00 con 0x7f3068133fa0 2023-09-30T22:25:03.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.475+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f306000ae60 con 0x7f3068133fa0 2023-09-30T22:25:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.502+0000 7f7e3d249700 1 Processor -- start 2023-09-30T22:25:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.502+0000 7f7e3d249700 1 -- start start 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3d249700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3d249700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3d249700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e380a2a70 con 0x7f7e380a0a30 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3d249700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380a01d0 con 0x7f7e38138d80 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3d249700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3809e2b0 con 0x7f7e3809d630 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e36d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e380a0a30 0x7f7e380a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44802/0 (socket says 172.21.15.32:44802) 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e36d9d700 1 -- 172.21.15.32:0/681952466 learned_addr learned my addr 172.21.15.32:0/681952466 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3759e700 1 --2- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e35d9b700 1 -- 172.21.15.32:0/681952466 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340775718 0 0) 0x7f7e380a2a70 con 0x7f7e380a0a30 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3659c700 1 -- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 msgr2=0x7f7e3813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3659c700 1 --2- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.503+0000 7f7e3659c700 1 -- 172.21.15.32:0/681952466 >> v1:172.21.15.32:6789/0 conn(0x7f7e380a0a30 legacy=0x7f7e380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3659c700 1 -- 172.21.15.32:0/681952466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3809dc50 con 0x7f7e3809d630 2023-09-30T22:25:03.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3659c700 1 --2- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009870 tx=0x7f7e2c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49b058708a67478 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:03.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e35d9b700 1 -- 172.21.15.32:0/681952466 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011830 con 0x7f7e3809d630 2023-09-30T22:25:03.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e35d9b700 1 -- 172.21.15.32:0/681952466 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c011990 con 0x7f7e3809d630 2023-09-30T22:25:03.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e35d9b700 1 -- 172.21.15.32:0/681952466 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c011d20 con 0x7f7e3809d630 2023-09-30T22:25:03.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3d249700 1 -- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e3809d630 msgr2=0x7f7e380a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3d249700 1 --2- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009870 tx=0x7f7e2c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:03.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3d249700 1 -- 172.21.15.32:0/681952466 shutdown_connections 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3d249700 1 --2- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e3813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3d249700 1 --2- 172.21.15.32:0/681952466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e3809d630 0x7f7e380a1510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.504+0000 7f7e3d249700 1 -- 172.21.15.32:0/681952466 >> 172.21.15.32:0/681952466 conn(0x7f7e380a3050 msgr2=0x7f7e380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.505+0000 7f7e3d249700 1 -- 172.21.15.32:0/681952466 shutdown_connections 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.505+0000 7f7e3d249700 1 -- 172.21.15.32:0/681952466 wait complete. 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 Processor -- start 2023-09-30T22:25:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 -- start start 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3809d630 0x7f7e381337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e380a0a30 0x7f7e38133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e38134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38136560 con 0x7f7e3809d630 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38134c00 con 0x7f7e38138d80 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3d249700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38136ed0 con 0x7f7e380a0a30 2023-09-30T22:25:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.506+0000 7f7e3659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e380a0a30 0x7f7e38133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3809d630 0x7f7e381337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e3659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e380a0a30 0x7f7e38133d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42970/0 (socket says 172.21.15.32:42970) 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e3659c700 1 -- 172.21.15.32:0/3073612120 learned_addr learned my addr 172.21.15.32:0/3073612120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e3759e700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e38134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 -- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e380a0a30 msgr2=0x7f7e38133d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e380a0a30 0x7f7e38133d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 -- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 msgr2=0x7f7e38134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e38134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 -- 172.21.15.32:0/3073612120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0094a0 con 0x7f7e3809d630 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e3759e700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e38134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e36d9d700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3809d630 0x7f7e381337c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800cbc0 tx=0x7f7e2800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800aaa0 con 0x7f7e3809d630 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800ac00 con 0x7f7e3809d630 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3813ad00 con 0x7f7e3809d630 2023-09-30T22:25:03.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280070d0 con 0x7f7e3809d630 2023-09-30T22:25:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.507+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e381371b0 con 0x7f7e3809d630 2023-09-30T22:25:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.508+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e28007230 con 0x7f7e3809d630 2023-09-30T22:25:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.508+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7e2803f170 con 0x7f7e3809d630 2023-09-30T22:25:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.509+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3813b620 con 0x7f7e3809d630 2023-09-30T22:25:03.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.518+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e28018070 con 0x7f7e3809d630 2023-09-30T22:25:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.686+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3068121f70 con 0x7f3068133fa0 2023-09-30T22:25:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.687+0000 7f30657fa700 1 -- 172.21.15.32:0/1391256372 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3068121f70 con 0x7f3068133fa0 2023-09-30T22:25:03.688 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 msgr2=0x7f306811e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 0x7f306811e8c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f306000cc50 tx=0x7f306000cf60 comp rx=0 tx=0).stop 2023-09-30T22:25:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 shutdown_connections 2023-09-30T22:25:03.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3068133fa0 0x7f306811e8c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30680a0a30 0x7f30681c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 --2- 172.21.15.32:0/1391256372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30680a1340 0x7f30681c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 >> 172.21.15.32:0/1391256372 conn(0x7f30680a6550 msgr2=0x7f30681b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 shutdown_connections 2023-09-30T22:25:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.689+0000 7f306f0eb700 1 -- 172.21.15.32:0/1391256372 wait complete. 2023-09-30T22:25:03.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.723+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7e380747f0 con 0x7f7e3809d630 2023-09-30T22:25:03.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.724+0000 7f7e1f7fe700 1 -- 172.21.15.32:0/3073612120 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e280074e0 con 0x7f7e3809d630 2023-09-30T22:25:03.725 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3809d630 msgr2=0x7f7e381337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3809d630 0x7f7e381337c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800cbc0 tx=0x7f7e2800ced0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 shutdown_connections 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e38138d80 0x7f7e38134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3809d630 0x7f7e381337c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 --2- 172.21.15.32:0/3073612120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e380a0a30 0x7f7e38133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 >> 172.21.15.32:0/3073612120 conn(0x7f7e380a3050 msgr2=0x7f7e3809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 shutdown_connections 2023-09-30T22:25:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:03.726+0000 7f7e3d249700 1 -- 172.21.15.32:0/3073612120 wait complete. 2023-09-30T22:25:03.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:03.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:03.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 Processor -- start 2023-09-30T22:25:04.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 -- start start 2023-09-30T22:25:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bc00a1510 0x7f3bc0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc0139250 0x7f3bc013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc009edd0 con 0x7f3bc00a0a30 2023-09-30T22:25:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc00a18e0 con 0x7f3bc00a1510 2023-09-30T22:25:04.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.079+0000 7f3bc480d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc009e1f0 con 0x7f3bc0139250 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc0139250 0x7f3bc013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbdd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bc00a1510 0x7f3bc0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbe59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3bc00a0a30 0x7f3bc00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44818/0 (socket says 172.21.15.32:44818) 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbe59c700 1 -- 172.21.15.32:0/4252992563 learned_addr learned my addr 172.21.15.32:0/4252992563 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbdd9b700 1 -- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc0139250 msgr2=0x7f3bc013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbdd9b700 1 --2- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc0139250 0x7f3bc013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbdd9b700 1 -- 172.21.15.32:0/4252992563 >> v1:172.21.15.32:6789/0 conn(0x7f3bc00a0a30 legacy=0x7f3bc00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbdd9b700 1 -- 172.21.15.32:0/4252992563 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc009dc50 con 0x7f3bc00a1510 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbd59a700 1 -- 172.21.15.32:0/4252992563 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431601643 0 0) 0x7f3bc009edd0 con 0x7f3bc00a0a30 2023-09-30T22:25:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbed9d700 1 --2- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc0139250 0x7f3bc013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbdd9b700 1 --2- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a1510 0x7f3bc0138d10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009840 tx=0x7f3bb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c1824be79f517e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbd59a700 1 -- 172.21.15.32:0/4252992563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4011830 con 0x7f3bc00a1510 2023-09-30T22:25:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbd59a700 1 -- 172.21.15.32:0/4252992563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb4011990 con 0x7f3bc00a1510 2023-09-30T22:25:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.080+0000 7f3bbd59a700 1 -- 172.21.15.32:0/4252992563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4011d20 con 0x7f3bc00a1510 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 -- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a1510 msgr2=0x7f3bc0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 --2- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a1510 0x7f3bc0138d10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009840 tx=0x7f3bb4009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 -- 172.21.15.32:0/4252992563 shutdown_connections 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 --2- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc0139250 0x7f3bc013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 --2- 172.21.15.32:0/4252992563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a1510 0x7f3bc0138d10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 -- 172.21.15.32:0/4252992563 >> 172.21.15.32:0/4252992563 conn(0x7f3bc00a6550 msgr2=0x7f3bc00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 -- 172.21.15.32:0/4252992563 shutdown_connections 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.081+0000 7f3bc480d700 1 -- 172.21.15.32:0/4252992563 wait complete. 2023-09-30T22:25:04.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.082+0000 7f3bc480d700 1 Processor -- start 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.082+0000 7f3bc480d700 1 -- start start 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bc480d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a0a30 0x7f3bc01228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bc480d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc00a1510 0x7f3bc0122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bc480d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 0x7f3bc0123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bc480d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01389e0 con 0x7f3bc00a0a30 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bc480d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc00a0220 con 0x7f3bc00a1510 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bc480d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0126060 con 0x7f3bc0139250 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bbed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 0x7f3bc0123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bbed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 0x7f3bc0123340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:42978/0 (socket says 172.21.15.32:42978) 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bbed9d700 1 -- 172.21.15.32:0/1795884457 learned_addr learned my addr 172.21.15.32:0/1795884457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bbdd9b700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc00a1510 0x7f3bc0122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.083+0000 7f3bbe59c700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a0a30 0x7f3bc01228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbed9d700 1 -- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc00a1510 msgr2=0x7f3bc0122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbed9d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc00a1510 0x7f3bc0122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbed9d700 1 -- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a0a30 msgr2=0x7f3bc01228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbed9d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a0a30 0x7f3bc01228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbed9d700 1 -- 172.21.15.32:0/1795884457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb80094a0 con 0x7f3bc0139250 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbe59c700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a0a30 0x7f3bc01228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbdd9b700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc00a1510 0x7f3bc0122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bbed9d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 0x7f3bc0123340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800eb30 tx=0x7f3bb800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8009780 con 0x7f3bc0139250 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8009e60 con 0x7f3bc0139250 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800ca50 con 0x7f3bc0139250 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb40094a0 con 0x7f3bc0139250 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.084+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc0124bb0 con 0x7f3bc0139250 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.085+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bb800cbb0 con 0x7f3bc0139250 2023-09-30T22:25:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.085+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3bb803fac0 con 0x7f3bc0139250 2023-09-30T22:25:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.085+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc0074ea0 con 0x7f3bc0139250 2023-09-30T22:25:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.090+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bb8011b00 con 0x7f3bc0139250 2023-09-30T22:25:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.119+0000 7f0bcfa8a700 1 Processor -- start 2023-09-30T22:25:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.119+0000 7f0bcfa8a700 1 -- start start 2023-09-30T22:25:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.119+0000 7f0bcfa8a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bc80a0a30 0x7f0bc80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.119+0000 7f0bcfa8a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc8134170 0x7f0bc813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcfa8a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc809edd0 con 0x7f0bc80a1340 2023-09-30T22:25:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcfa8a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc809e1f0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcfa8a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc809dc50 con 0x7f0bc8134170 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcd826700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bc80a0a30 0x7f0bc80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bce027700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc8134170 0x7f0bc813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcd025700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0bc80a1340 0x7f0bc8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44832/0 (socket says 172.21.15.32:44832) 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcd025700 1 -- 172.21.15.32:0/486257305 learned_addr learned my addr 172.21.15.32:0/486257305 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bce027700 1 -- 172.21.15.32:0/486257305 >> v2:172.21.15.32:3300/0 conn(0x7f0bc80a0a30 msgr2=0x7f0bc80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bce027700 1 --2- 172.21.15.32:0/486257305 >> v2:172.21.15.32:3300/0 conn(0x7f0bc80a0a30 0x7f0bc80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bce027700 1 -- 172.21.15.32:0/486257305 >> v1:172.21.15.32:6789/0 conn(0x7f0bc80a1340 legacy=0x7f0bc8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bce027700 1 -- 172.21.15.32:0/486257305 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc80a1fe0 con 0x7f0bc8134170 2023-09-30T22:25:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcc824700 1 -- 172.21.15.32:0/486257305 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753656968 0 0) 0x7f0bc809edd0 con 0x7f0bc80a1340 2023-09-30T22:25:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcd826700 1 --2- 172.21.15.32:0/486257305 >> v2:172.21.15.32:3300/0 conn(0x7f0bc80a0a30 0x7f0bc80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bce027700 1 --2- 172.21.15.32:0/486257305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc8134170 0x7f0bc813bd30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009d90 tx=0x7f0bc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=557e941ffc648f8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.120+0000 7f0bcc824700 1 -- 172.21.15.32:0/486257305 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000e070 con 0x7f0bc8134170 2023-09-30T22:25:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcc824700 1 -- 172.21.15.32:0/486257305 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc000bb20 con 0x7f0bc8134170 2023-09-30T22:25:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcc824700 1 -- 172.21.15.32:0/486257305 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0012430 con 0x7f0bc8134170 2023-09-30T22:25:04.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/486257305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc8134170 msgr2=0x7f0bc813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/486257305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc8134170 0x7f0bc813bd30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009d90 tx=0x7f0bc000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/486257305 shutdown_connections 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/486257305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc8134170 0x7f0bc813bd30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/486257305 >> v2:172.21.15.32:3300/0 conn(0x7f0bc80a0a30 0x7f0bc80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/486257305 >> 172.21.15.32:0/486257305 conn(0x7f0bc80a6550 msgr2=0x7f0bc80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/486257305 shutdown_connections 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.121+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/486257305 wait complete. 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.122+0000 7f0bcfa8a700 1 Processor -- start 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.122+0000 7f0bcfa8a700 1 -- start start 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcfa8a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 0x7f0bc81226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcfa8a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc80a1340 0x7f0bc8122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcfa8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc8134170 0x7f0bc8123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcfa8a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8124b80 con 0x7f0bc8134170 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcfa8a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8123b40 con 0x7f0bc80a1340 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcfa8a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8124400 con 0x7f0bc80a0a30 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 0x7f0bc81226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bce027700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc8134170 0x7f0bc8123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 0x7f0bc81226e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43004/0 (socket says 172.21.15.32:43004) 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 -- 172.21.15.32:0/124898348 learned_addr learned my addr 172.21.15.32:0/124898348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd025700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc80a1340 0x7f0bc8122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 -- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc80a1340 msgr2=0x7f0bc8122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc80a1340 0x7f0bc8122c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 -- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc8134170 msgr2=0x7f0bc8123160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc8134170 0x7f0bc8123160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 -- 172.21.15.32:0/124898348 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40094a0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bce027700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc8134170 0x7f0bc8123160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.123+0000 7f0bcd826700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 0x7f0bc81226e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400ebe0 tx=0x7f0bc400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.124+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400c7f0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.124+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc400c950 con 0x7f0bc80a0a30 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.124+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400cce0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.124+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc0009a70 con 0x7f0bc80a0a30 2023-09-30T22:25:04.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.124+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc80a25e0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.125+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc8074ea0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.125+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0bc4013470 con 0x7f0bc80a0a30 2023-09-30T22:25:04.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.125+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0bc403f8a0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.130+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0bc4013720 con 0x7f0bc80a0a30 2023-09-30T22:25:04.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.299+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3bc00a28e0 con 0x7f3bc0139250 2023-09-30T22:25:04.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.299+0000 7f3baeffd700 1 -- 172.21.15.32:0/1795884457 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bb802b090 con 0x7f3bc0139250 2023-09-30T22:25:04.300 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:04.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 msgr2=0x7f3bc0123340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 0x7f3bc0123340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800eb30 tx=0x7f3bb800ee40 comp rx=0 tx=0).stop 2023-09-30T22:25:04.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 shutdown_connections 2023-09-30T22:25:04.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bc0139250 0x7f3bc0123340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bc00a0a30 0x7f3bc01228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 --2- 172.21.15.32:0/1795884457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bc00a1510 0x7f3bc0122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 >> 172.21.15.32:0/1795884457 conn(0x7f3bc00a6550 msgr2=0x7f3bc00a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 shutdown_connections 2023-09-30T22:25:04.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.301+0000 7f3bc480d700 1 -- 172.21.15.32:0/1795884457 wait complete. 2023-09-30T22:25:04.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.336+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0bc8133200 con 0x7f0bc80a0a30 2023-09-30T22:25:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.336+0000 7f0bbe7fc700 1 -- 172.21.15.32:0/124898348 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bc401b3b0 con 0x7f0bc80a0a30 2023-09-30T22:25:04.338 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 msgr2=0x7f0bc81226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 0x7f0bc81226e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400ebe0 tx=0x7f0bc400eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 shutdown_connections 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc8134170 0x7f0bc8123160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc80a0a30 0x7f0bc81226e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 --2- 172.21.15.32:0/124898348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc80a1340 0x7f0bc8122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 >> 172.21.15.32:0/124898348 conn(0x7f0bc80a6550 msgr2=0x7f0bc809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 shutdown_connections 2023-09-30T22:25:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.339+0000 7f0bcfa8a700 1 -- 172.21.15.32:0/124898348 wait complete. 2023-09-30T22:25:04.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:04.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:04.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 Processor -- start 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 -- start start 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 0x7f63140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 0x7f63140a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6314133930 0x7f6314133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140a2a70 con 0x7f63140a0a30 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.851+0000 7f631940c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140a01d0 con 0x7f631409d630 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f631940c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f631409e2b0 con 0x7f6314133930 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 0x7f63140a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f6312ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 0x7f63140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f6312ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 0x7f63140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51454/0 (socket says 172.21.15.32:51454) 2023-09-30T22:25:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 0x7f63140a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49290/0 (socket says 172.21.15.32:49290) 2023-09-30T22:25:04.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f6312ffd700 1 -- 172.21.15.32:0/4014683596 learned_addr learned my addr 172.21.15.32:0/4014683596 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:04.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f63137fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6314133930 0x7f6314133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 -- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6314133930 msgr2=0x7f6314133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 --2- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6314133930 0x7f6314133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 -- 172.21.15.32:0/4014683596 >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 msgr2=0x7f63140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:04.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 --2- 172.21.15.32:0/4014683596 >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 0x7f63140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 -- 172.21.15.32:0/4014683596 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631409dc50 con 0x7f631409d630 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f63137fe700 1 --2- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6314133930 0x7f6314133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f6312ffd700 1 --2- 172.21.15.32:0/4014683596 >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 0x7f63140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f630a7fc700 1 --2- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 0x7f63140a1340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009840 tx=0x7f62fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740053aab2ea6faa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.852+0000 7f63127fc700 1 -- 172.21.15.32:0/4014683596 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc011830 con 0x7f631409d630 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f63127fc700 1 -- 172.21.15.32:0/4014683596 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc011990 con 0x7f631409d630 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f63127fc700 1 -- 172.21.15.32:0/4014683596 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc011d20 con 0x7f631409d630 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 -- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 msgr2=0x7f63140a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 --2- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 0x7f63140a1340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009840 tx=0x7f62fc009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 -- 172.21.15.32:0/4014683596 shutdown_connections 2023-09-30T22:25:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 --2- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6314133930 0x7f6314133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 --2- 172.21.15.32:0/4014683596 >> v2:172.21.15.32:3300/0 conn(0x7f63140a0a30 0x7f63140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 --2- 172.21.15.32:0/4014683596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f631409d630 0x7f63140a1340 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 -- 172.21.15.32:0/4014683596 >> 172.21.15.32:0/4014683596 conn(0x7f63140a3050 msgr2=0x7f63140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 -- 172.21.15.32:0/4014683596 shutdown_connections 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.853+0000 7f631940c700 1 -- 172.21.15.32:0/4014683596 wait complete. 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.854+0000 7f631940c700 1 Processor -- start 2023-09-30T22:25:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 -- start start 2023-09-30T22:25:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 0x7f63141c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63140a0a30 0x7f63141c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6314133930 0x7f63141c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314120e20 con 0x7f6314133930 2023-09-30T22:25:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63141206d0 con 0x7f63140a0a30 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f631940c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63140a4030 con 0x7f631409d630 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 0x7f63141c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f630a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63140a0a30 0x7f63141c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 0x7f63141c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43016/0 (socket says 172.21.15.32:43016) 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f6312ffd700 1 -- 172.21.15.32:0/2098111829 learned_addr learned my addr 172.21.15.32:0/2098111829 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.855+0000 7f63137fe700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6314133930 0x7f63141c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f6312ffd700 1 -- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63140a0a30 msgr2=0x7f63141c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f6312ffd700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63140a0a30 0x7f63141c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f6312ffd700 1 -- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6314133930 msgr2=0x7f63141c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f6312ffd700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6314133930 0x7f63141c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f6312ffd700 1 -- 172.21.15.32:0/2098111829 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304009550 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f630a7fc700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63140a0a30 0x7f63141c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f6312ffd700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 0x7f63141c8100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f630400ec10 tx=0x7f630400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400c850 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400c9b0 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400cd20 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c009a70 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.856+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc0094a0 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.857+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6304013470 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.857+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f630403f7b0 con 0x7f631409d630 2023-09-30T22:25:04.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.857+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6314122210 con 0x7f631409d630 2023-09-30T22:25:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.866+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6304013720 con 0x7f631409d630 2023-09-30T22:25:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.892+0000 7fc71b493700 1 Processor -- start 2023-09-30T22:25:04.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.892+0000 7fc71b493700 1 -- start start 2023-09-30T22:25:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71b493700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 0x7fc7140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71b493700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7140a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71b493700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 0x7fc714133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71b493700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7140a2a70 con 0x7fc714133930 2023-09-30T22:25:04.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71b493700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140a01d0 con 0x7fc71409d630 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71b493700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc71409e2b0 con 0x7fc7140a0a30 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc718a2e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7140a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc719a30700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 0x7fc714133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 0x7fc7140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc718a2e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7140a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49312/0 (socket says 172.21.15.32:49312) 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 0x7fc7140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43022/0 (socket says 172.21.15.32:43022) 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 -- 172.21.15.32:0/915106230 learned_addr learned my addr 172.21.15.32:0/915106230 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc719a30700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 0x7fc714133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51480/0 (socket says 172.21.15.32:51480) 2023-09-30T22:25:04.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 -- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 msgr2=0x7fc7140a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 --2- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7140a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 -- 172.21.15.32:0/915106230 >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 msgr2=0x7fc714133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 --2- 172.21.15.32:0/915106230 >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 0x7fc714133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 -- 172.21.15.32:0/915106230 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71409dc50 con 0x7fc7140a0a30 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc719a30700 1 --2- 172.21.15.32:0/915106230 >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 0x7fc714133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.893+0000 7fc71922f700 1 --2- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 0x7fc7140a0e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc704009e50 tx=0x7fc70400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c9edbd43084e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc713fff700 1 -- 172.21.15.32:0/915106230 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400e070 con 0x7fc7140a0a30 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc713fff700 1 -- 172.21.15.32:0/915106230 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400b910 con 0x7fc7140a0a30 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc713fff700 1 -- 172.21.15.32:0/915106230 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70400bcc0 con 0x7fc7140a0a30 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 -- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 msgr2=0x7fc7140a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 --2- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 0x7fc7140a0e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc704009e50 tx=0x7fc70400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 -- 172.21.15.32:0/915106230 shutdown_connections 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 --2- 172.21.15.32:0/915106230 >> v2:172.21.15.32:3300/0 conn(0x7fc714133930 0x7fc714133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 --2- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc7140a0a30 0x7fc7140a0e00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 --2- 172.21.15.32:0/915106230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7140a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 -- 172.21.15.32:0/915106230 >> 172.21.15.32:0/915106230 conn(0x7fc7140a3050 msgr2=0x7fc7140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 -- 172.21.15.32:0/915106230 shutdown_connections 2023-09-30T22:25:04.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.894+0000 7fc71b493700 1 -- 172.21.15.32:0/915106230 wait complete. 2023-09-30T22:25:04.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.895+0000 7fc71b493700 1 Processor -- start 2023-09-30T22:25:04.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.895+0000 7fc71b493700 1 -- start start 2023-09-30T22:25:04.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71b493700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7141c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71b493700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140a0a30 0x7fc7141c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71b493700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 0x7fc7141c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71b493700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714122430 con 0x7fc7140a0a30 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71b493700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc714120720 con 0x7fc71409d630 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71b493700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140a41c0 con 0x7fc714133930 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc719a30700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 0x7fc7141c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc718a2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140a0a30 0x7fc7141c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71922f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7141c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc719a30700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 0x7fc7141c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43036/0 (socket says 172.21.15.32:43036) 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc719a30700 1 -- 172.21.15.32:0/1930935077 learned_addr learned my addr 172.21.15.32:0/1930935077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71922f700 1 -- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 msgr2=0x7fc7141c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71922f700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 0x7fc7141c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71922f700 1 -- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140a0a30 msgr2=0x7fc7141c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71922f700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140a0a30 0x7fc7141c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.896+0000 7fc71922f700 1 -- 172.21.15.32:0/1930935077 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0094a0 con 0x7fc71409d630 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc719a30700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 0x7fc7141c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc718a2e700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140a0a30 0x7fc7141c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc71922f700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7141c81a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc704015000 tx=0x7fc704004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040092e0 con 0x7fc71409d630 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70400cea0 con 0x7fc71409d630 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704004490 con 0x7fc71409d630 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704009b30 con 0x7fc71409d630 2023-09-30T22:25:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.897+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7140a0410 con 0x7fc71409d630 2023-09-30T22:25:04.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.898+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7040045f0 con 0x7fc71409d630 2023-09-30T22:25:04.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.898+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7140747f0 con 0x7fc71409d630 2023-09-30T22:25:04.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.899+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc704040860 con 0x7fc71409d630 2023-09-30T22:25:04.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:04.903+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc704011390 con 0x7fc71409d630 2023-09-30T22:25:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.073+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f63140747f0 con 0x7f631409d630 2023-09-30T22:25:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.074+0000 7f630bfff700 1 -- 172.21.15.32:0/2098111829 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f630401ae80 con 0x7f631409d630 2023-09-30T22:25:05.075 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 msgr2=0x7f63141c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 0x7f63141c8100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f630400ec10 tx=0x7f630400ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 shutdown_connections 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6314133930 0x7f63141c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f631409d630 0x7f63141c8100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 --2- 172.21.15.32:0/2098111829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63140a0a30 0x7f63141c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 >> 172.21.15.32:0/2098111829 conn(0x7f63140a3050 msgr2=0x7f63141bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:05.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 shutdown_connections 2023-09-30T22:25:05.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.076+0000 7f631940c700 1 -- 172.21.15.32:0/2098111829 wait complete. 2023-09-30T22:25:05.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.108+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc7140a4f80 con 0x7fc71409d630 2023-09-30T22:25:05.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.108+0000 7fc711ffb700 1 -- 172.21.15.32:0/1930935077 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc70401bb90 con 0x7fc71409d630 2023-09-30T22:25:05.109 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.110+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 msgr2=0x7fc7141c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7141c81a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc704015000 tx=0x7fc704004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 shutdown_connections 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc714133930 0x7fc7141c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc71409d630 0x7fc7141c81a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 --2- 172.21.15.32:0/1930935077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7140a0a30 0x7fc7141c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 >> 172.21.15.32:0/1930935077 conn(0x7fc7140a3050 msgr2=0x7fc7141bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 shutdown_connections 2023-09-30T22:25:05.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.111+0000 7fc71b493700 1 -- 172.21.15.32:0/1930935077 wait complete. 2023-09-30T22:25:05.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:05.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:05.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:05.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.465+0000 7f08ae0a6700 1 Processor -- start 2023-09-30T22:25:05.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.465+0000 7f08ae0a6700 1 -- start start 2023-09-30T22:25:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.465+0000 7f08ae0a6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 0x7f08a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08ae0a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 0x7f08a80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08ae0a6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 0x7f08a8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08ae0a6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a80a2a70 con 0x7f08a80a0a30 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08ae0a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80a01d0 con 0x7f08a809d630 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08ae0a6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08a809e2b0 con 0x7f08a8133930 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 0x7f08a80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 0x7f08a8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 0x7f08a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 0x7f08a8133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43044/0 (socket says 172.21.15.32:43044) 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 0x7f08a80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49330/0 (socket says 172.21.15.32:49330) 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 0x7f08a80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51494/0 (socket says 172.21.15.32:51494) 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 -- 172.21.15.32:0/1818648191 learned_addr learned my addr 172.21.15.32:0/1818648191 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 -- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 msgr2=0x7f08a80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 --2- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 0x7f08a80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 -- 172.21.15.32:0/1818648191 >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 msgr2=0x7f08a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 --2- 172.21.15.32:0/1818648191 >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 0x7f08a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 -- 172.21.15.32:0/1818648191 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a809dc50 con 0x7f08a8133930 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a6ffd700 1 --2- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 0x7f08a80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a77fe700 1 --2- 172.21.15.32:0/1818648191 >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 0x7f08a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.466+0000 7f08a7fff700 1 --2- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 0x7f08a8133db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08a00098f0 tx=0x7f08a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be21acfa31542ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.467+0000 7f08a67fc700 1 -- 172.21.15.32:0/1818648191 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0011830 con 0x7f08a8133930 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.467+0000 7f08a67fc700 1 -- 172.21.15.32:0/1818648191 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0011990 con 0x7f08a8133930 2023-09-30T22:25:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.467+0000 7f08a67fc700 1 -- 172.21.15.32:0/1818648191 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0011d20 con 0x7f08a8133930 2023-09-30T22:25:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.467+0000 7f08ae0a6700 1 -- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 msgr2=0x7f08a8133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 0x7f08a8133db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08a00098f0 tx=0x7f08a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 -- 172.21.15.32:0/1818648191 shutdown_connections 2023-09-30T22:25:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a8133930 0x7f08a8133db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/1818648191 >> v2:172.21.15.32:3300/0 conn(0x7f08a80a0a30 0x7f08a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/1818648191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a809d630 0x7f08a80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 -- 172.21.15.32:0/1818648191 >> 172.21.15.32:0/1818648191 conn(0x7f08a80a3050 msgr2=0x7f08a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 -- 172.21.15.32:0/1818648191 shutdown_connections 2023-09-30T22:25:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.468+0000 7f08ae0a6700 1 -- 172.21.15.32:0/1818648191 wait complete. 2023-09-30T22:25:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.469+0000 7f08ae0a6700 1 Processor -- start 2023-09-30T22:25:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.469+0000 7f08ae0a6700 1 -- start start 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08ae0a6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 0x7f08a811e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08ae0a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a80a0a30 0x7f08a811ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08ae0a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 0x7f08a811f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08ae0a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a8120a40 con 0x7f08a80a0a30 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08ae0a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80a0480 con 0x7f08a8133930 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08ae0a6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81205a0 con 0x7f08a809d630 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08a77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 0x7f08a811e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08a7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 0x7f08a811f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08a7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 0x7f08a811f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49332/0 (socket says 172.21.15.32:49332) 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08a77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 0x7f08a811e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43054/0 (socket says 172.21.15.32:43054) 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08a77fe700 1 -- 172.21.15.32:0/3836130350 learned_addr learned my addr 172.21.15.32:0/3836130350 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.470+0000 7f08a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a80a0a30 0x7f08a811ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a77fe700 1 -- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 msgr2=0x7f08a811f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a77fe700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 0x7f08a811f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a77fe700 1 -- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a80a0a30 msgr2=0x7f08a811ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a77fe700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a80a0a30 0x7f08a811ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a77fe700 1 -- 172.21.15.32:0/3836130350 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0094a0 con 0x7f08a809d630 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a7fff700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 0x7f08a811f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08a77fe700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 0x7f08a811e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f089800efc0 tx=0x7f089800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089800cd70 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089800f040 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0898009a70 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08980146a0 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.471+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a0009550 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.472+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0898014800 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.472+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f08980403f0 con 0x7f08a809d630 2023-09-30T22:25:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.473+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a80747f0 con 0x7f08a809d630 2023-09-30T22:25:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.477+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f089801d400 con 0x7f08a809d630 2023-09-30T22:25:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.510+0000 7fbd579ac700 1 Processor -- start 2023-09-30T22:25:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.510+0000 7fbd579ac700 1 -- start start 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd579ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd579ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd500a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd579ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd50133930 0x7fbd50133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd579ac700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd500a2a70 con 0x7fbd50133930 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd579ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500a01d0 con 0x7fbd5009d630 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd579ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5009e2b0 con 0x7fbd500a0a30 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd50133930 0x7fbd50133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd50133930 0x7fbd50133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51504/0 (socket says 172.21.15.32:51504) 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 -- 172.21.15.32:0/3655045832 learned_addr learned my addr 172.21.15.32:0/3655045832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55748700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd54f47700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd500a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 -- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 msgr2=0x7fbd500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd500a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 -- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 msgr2=0x7fbd500a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd500a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55f49700 1 -- 172.21.15.32:0/3655045832 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5009dc50 con 0x7fbd50133930 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.511+0000 7fbd55748700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd55f49700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd50133dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd480098f0 tx=0x7fbd48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6830eb5d296396 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd4ffff700 1 -- 172.21.15.32:0/3655045832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011830 con 0x7fbd50133930 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd4ffff700 1 -- 172.21.15.32:0/3655045832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48011990 con 0x7fbd50133930 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd4ffff700 1 -- 172.21.15.32:0/3655045832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011d20 con 0x7fbd50133930 2023-09-30T22:25:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 -- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 msgr2=0x7fbd50133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd50133dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd480098f0 tx=0x7fbd48009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 -- 172.21.15.32:0/3655045832 shutdown_connections 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd50133dd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 --2- 172.21.15.32:0/3655045832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd500a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.512+0000 7fbd579ac700 1 -- 172.21.15.32:0/3655045832 >> 172.21.15.32:0/3655045832 conn(0x7fbd500a3050 msgr2=0x7fbd500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.513+0000 7fbd579ac700 1 -- 172.21.15.32:0/3655045832 shutdown_connections 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.513+0000 7fbd579ac700 1 -- 172.21.15.32:0/3655045832 wait complete. 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 Processor -- start 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 -- start start 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd5011e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd5011ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd5011f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd50133460 con 0x7fbd50133930 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500a50b0 con 0x7fbd5009d630 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd579ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd501205e0 con 0x7fbd500a0a30 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd55f49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd5011f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd55748700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd5011e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd54f47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd5011ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd55f49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd5011f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51506/0 (socket says 172.21.15.32:51506) 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd55748700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd5011e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49360/0 (socket says 172.21.15.32:49360) 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd55f49700 1 -- 172.21.15.32:0/2559571714 learned_addr learned my addr 172.21.15.32:0/2559571714 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.514+0000 7fbd54f47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd5011ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43064/0 (socket says 172.21.15.32:43064) 2023-09-30T22:25:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd55748700 1 -- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 msgr2=0x7fbd5011ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd55748700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd5011ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd55748700 1 -- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 msgr2=0x7fbd5011f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd55748700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd5011f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd55748700 1 -- 172.21.15.32:0/2559571714 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd40009a70 con 0x7fbd5009d630 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd54f47700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd5011ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd55748700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd5011e8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000efc0 tx=0x7fbd4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000cd10 con 0x7fbd5009d630 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4000f040 con 0x7fbd5009d630 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd48009550 con 0x7fbd5009d630 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd400146a0 con 0x7fbd5009d630 2023-09-30T22:25:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.515+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd50122090 con 0x7fbd5009d630 2023-09-30T22:25:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.516+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd50120cb0 con 0x7fbd5009d630 2023-09-30T22:25:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.516+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd40014800 con 0x7fbd5009d630 2023-09-30T22:25:05.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.517+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbd40040370 con 0x7fbd5009d630 2023-09-30T22:25:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.521+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd40012900 con 0x7fbd5009d630 2023-09-30T22:25:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.693+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08a8122110 con 0x7f08a809d630 2023-09-30T22:25:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.693+0000 7f088ffff700 1 -- 172.21.15.32:0/3836130350 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0898012340 con 0x7f08a809d630 2023-09-30T22:25:05.695 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:05.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.695+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 msgr2=0x7f08a811e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.695+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 0x7f08a811e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f089800efc0 tx=0x7f089800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 shutdown_connections 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a8133930 0x7f08a811f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a809d630 0x7f08a811e810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 --2- 172.21.15.32:0/3836130350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a80a0a30 0x7f08a811ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 >> 172.21.15.32:0/3836130350 conn(0x7f08a80a3050 msgr2=0x7f08a81bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 shutdown_connections 2023-09-30T22:25:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.696+0000 7f08ae0a6700 1 -- 172.21.15.32:0/3836130350 wait complete. 2023-09-30T22:25:05.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:05.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.725+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd500747f0 con 0x7fbd5009d630 2023-09-30T22:25:05.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.726+0000 7fbd4dffb700 1 -- 172.21.15.32:0/2559571714 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd40012340 con 0x7fbd5009d630 2023-09-30T22:25:05.727 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 msgr2=0x7fbd5011e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd5011e8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000efc0 tx=0x7fbd4000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 shutdown_connections 2023-09-30T22:25:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd50133930 0x7fbd5011f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbd5009d630 0x7fbd5011e8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 --2- 172.21.15.32:0/2559571714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbd500a0a30 0x7fbd5011ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 >> 172.21.15.32:0/2559571714 conn(0x7fbd500a3050 msgr2=0x7fbd500a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 shutdown_connections 2023-09-30T22:25:05.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:05.728+0000 7fbd579ac700 1 -- 172.21.15.32:0/2559571714 wait complete. 2023-09-30T22:25:05.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:06.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:06.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.240+0000 7f93b21eb700 1 Processor -- start 2023-09-30T22:25:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.240+0000 7f93b21eb700 1 -- start start 2023-09-30T22:25:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93b21eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93b21eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93ac134170 0x7f93ac13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93b21eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac09edd0 con 0x7f93ac0a1340 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93b21eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac09e1f0 con 0x7f93ac134170 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93b21eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac09dc50 con 0x7f93ac0a0a30 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93abfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93ac134170 0x7f93ac13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93ab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93aaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93ac0a1340 0x7f93ac133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44848/0 (socket says 172.21.15.32:44848) 2023-09-30T22:25:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93aaffd700 1 -- 172.21.15.32:0/1481925028 learned_addr learned my addr 172.21.15.32:0/1481925028 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93abfff700 1 -- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 msgr2=0x7f93ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93abfff700 1 --2- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93abfff700 1 -- 172.21.15.32:0/1481925028 >> v1:172.21.15.32:6789/0 conn(0x7f93ac0a1340 legacy=0x7f93ac133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93abfff700 1 -- 172.21.15.32:0/1481925028 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0a1fe0 con 0x7f93ac134170 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93ab7fe700 1 --2- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93aa7fc700 1 -- 172.21.15.32:0/1481925028 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741959794 0 0) 0x7f93ac09edd0 con 0x7f93ac0a1340 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.241+0000 7f93abfff700 1 --2- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac134170 0x7f93ac13bd30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009d90 tx=0x7f93a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136ff1d7e7219ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93aa7fc700 1 -- 172.21.15.32:0/1481925028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400e070 con 0x7f93ac134170 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93aa7fc700 1 -- 172.21.15.32:0/1481925028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a400bb20 con 0x7f93ac134170 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93aa7fc700 1 -- 172.21.15.32:0/1481925028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a4012430 con 0x7f93ac134170 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 -- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac134170 msgr2=0x7f93ac13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 --2- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac134170 0x7f93ac13bd30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009d90 tx=0x7f93a400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 -- 172.21.15.32:0/1481925028 shutdown_connections 2023-09-30T22:25:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 --2- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac134170 0x7f93ac13bd30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 --2- 172.21.15.32:0/1481925028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 -- 172.21.15.32:0/1481925028 >> 172.21.15.32:0/1481925028 conn(0x7f93ac0a6550 msgr2=0x7f93ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 -- 172.21.15.32:0/1481925028 shutdown_connections 2023-09-30T22:25:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.242+0000 7f93b21eb700 1 -- 172.21.15.32:0/1481925028 wait complete. 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.244+0000 7f93b21eb700 1 Processor -- start 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.244+0000 7f93b21eb700 1 -- start start 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93b21eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93b21eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac0a1340 0x7f93ac122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93b21eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93ac134170 0x7f93ac123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93b21eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac125f30 con 0x7f93ac0a1340 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93b21eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0a4dc0 con 0x7f93ac134170 2023-09-30T22:25:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93b21eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0a0220 con 0x7f93ac0a0a30 2023-09-30T22:25:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93ab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93abfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93ac134170 0x7f93ac123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93ab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43092/0 (socket says 172.21.15.32:43092) 2023-09-30T22:25:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93ab7fe700 1 -- 172.21.15.32:0/2113140387 learned_addr learned my addr 172.21.15.32:0/2113140387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac0a1340 0x7f93ac122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93aaffd700 1 -- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 msgr2=0x7f93ac122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93aaffd700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93aaffd700 1 -- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93ac134170 msgr2=0x7f93ac123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93aaffd700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93ac134170 0x7f93ac123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93aaffd700 1 -- 172.21.15.32:0/2113140387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c0094a0 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93abfff700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93ac134170 0x7f93ac123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.245+0000 7f93ab7fe700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.246+0000 7f93aaffd700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac0a1340 0x7f93ac122cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93a000c470 tx=0x7f93a000c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.246+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0004c80 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.246+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0004de0 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.246+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0007310 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.246+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a4009a70 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.246+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac1269a0 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.247+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93a0007470 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.247+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f93a003f1f0 con 0x7f93ac0a1340 2023-09-30T22:25:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.247+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac074ea0 con 0x7f93ac0a1340 2023-09-30T22:25:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.252+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93a0007720 con 0x7f93ac0a1340 2023-09-30T22:25:06.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 Processor -- start 2023-09-30T22:25:06.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 -- start start 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 0x7f51ac0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51ac09d5c0 0x7f51ac0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0a1810 con 0x7f51ac133920 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac09ea10 con 0x7f51ac09d5c0 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51b17e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac09e210 con 0x7f51ac0a09c0 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.279+0000 7f51aaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 0x7f51ac0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aa7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51ac09d5c0 0x7f51ac0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51ab7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51ac133920 0x7f51ac133da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44854/0 (socket says 172.21.15.32:44854) 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 0x7f51ac0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49384/0 (socket says 172.21.15.32:49384) 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51ab7fe700 1 -- 172.21.15.32:0/411012736 learned_addr learned my addr 172.21.15.32:0/411012736 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aaffd700 1 -- 172.21.15.32:0/411012736 >> v2:172.21.15.32:3300/0 conn(0x7f51ac09d5c0 msgr2=0x7f51ac0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aaffd700 1 --2- 172.21.15.32:0/411012736 >> v2:172.21.15.32:3300/0 conn(0x7f51ac09d5c0 0x7f51ac0a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aaffd700 1 -- 172.21.15.32:0/411012736 >> v1:172.21.15.32:6789/0 conn(0x7f51ac133920 legacy=0x7f51ac133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aaffd700 1 -- 172.21.15.32:0/411012736 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0a26f0 con 0x7f51ac0a09c0 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51aaffd700 1 --2- 172.21.15.32:0/411012736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 0x7f51ac0a0d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f519c009840 tx=0x7f519c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68494474a3371a64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51a9ffb700 1 -- 172.21.15.32:0/411012736 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c011830 con 0x7f51ac0a09c0 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51a9ffb700 1 -- 172.21.15.32:0/411012736 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c011990 con 0x7f51ac0a09c0 2023-09-30T22:25:06.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.280+0000 7f51a9ffb700 1 -- 172.21.15.32:0/411012736 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c011d20 con 0x7f51ac0a09c0 2023-09-30T22:25:06.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 -- 172.21.15.32:0/411012736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 msgr2=0x7f51ac0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 --2- 172.21.15.32:0/411012736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 0x7f51ac0a0d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f519c009840 tx=0x7f519c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:06.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 -- 172.21.15.32:0/411012736 shutdown_connections 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 --2- 172.21.15.32:0/411012736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac0a09c0 0x7f51ac0a0d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 --2- 172.21.15.32:0/411012736 >> v2:172.21.15.32:3300/0 conn(0x7f51ac09d5c0 0x7f51ac0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 -- 172.21.15.32:0/411012736 >> 172.21.15.32:0/411012736 conn(0x7f51ac0a2fe0 msgr2=0x7f51ac0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 -- 172.21.15.32:0/411012736 shutdown_connections 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.281+0000 7f51b17e0700 1 -- 172.21.15.32:0/411012736 wait complete. 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.282+0000 7f51b17e0700 1 Processor -- start 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.282+0000 7f51b17e0700 1 -- start start 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51b17e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac09d5c0 0x7f51ac1c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51b17e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 0x7f51ac1c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51b17e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f51ac133920 0x7f51ac1c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51b17e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac122360 con 0x7f51ac0a09c0 2023-09-30T22:25:06.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51b17e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac120650 con 0x7f51ac09d5c0 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51b17e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0a4000 con 0x7f51ac133920 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 0x7f51ac1c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51ab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f51ac133920 0x7f51ac1c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 0x7f51ac1c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51538/0 (socket says 172.21.15.32:51538) 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 -- 172.21.15.32:0/2267607046 learned_addr learned my addr 172.21.15.32:0/2267607046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aaffd700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac09d5c0 0x7f51ac1c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 -- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f51ac133920 msgr2=0x7f51ac1c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f51ac133920 0x7f51ac1c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 -- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac09d5c0 msgr2=0x7f51ac1c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac09d5c0 0x7f51ac1c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 -- 172.21.15.32:0/2267607046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0094a0 con 0x7f51ac0a09c0 2023-09-30T22:25:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.283+0000 7f51aa7fc700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 0x7f51ac1c8730 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f51a000e980 tx=0x7f51a000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.284+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0009710 con 0x7f51ac0a09c0 2023-09-30T22:25:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.284+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a000c690 con 0x7f51ac0a09c0 2023-09-30T22:25:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.284+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac120e40 con 0x7f51ac0a09c0 2023-09-30T22:25:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.284+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000ca20 con 0x7f51ac0a09c0 2023-09-30T22:25:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.284+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac122560 con 0x7f51ac0a09c0 2023-09-30T22:25:06.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.284+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51a000cb80 con 0x7f51ac0a09c0 2023-09-30T22:25:06.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.285+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f51a003f6c0 con 0x7f51ac0a09c0 2023-09-30T22:25:06.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.285+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac0747f0 con 0x7f51ac0a09c0 2023-09-30T22:25:06.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.290+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51a00232f0 con 0x7f51ac0a09c0 2023-09-30T22:25:06.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.485+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93ac0a28e0 con 0x7f93ac0a1340 2023-09-30T22:25:06.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.486+0000 7f9393fff700 1 -- 172.21.15.32:0/2113140387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93a0011360 con 0x7f93ac0a1340 2023-09-30T22:25:06.487 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:06.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac0a1340 msgr2=0x7f93ac122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac0a1340 0x7f93ac122cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93a000c470 tx=0x7f93a000c780 comp rx=0 tx=0).stop 2023-09-30T22:25:06.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 shutdown_connections 2023-09-30T22:25:06.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93ac134170 0x7f93ac123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93ac0a0a30 0x7f93ac122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 --2- 172.21.15.32:0/2113140387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93ac0a1340 0x7f93ac122cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 >> 172.21.15.32:0/2113140387 conn(0x7f93ac0a6550 msgr2=0x7f93ac09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:06.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 shutdown_connections 2023-09-30T22:25:06.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.488+0000 7f93b21eb700 1 -- 172.21.15.32:0/2113140387 wait complete. 2023-09-30T22:25:06.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.518+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f51ac072980 con 0x7f51ac0a09c0 2023-09-30T22:25:06.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.518+0000 7f51937fe700 1 -- 172.21.15.32:0/2267607046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51a0017020 con 0x7f51ac0a09c0 2023-09-30T22:25:06.519 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 msgr2=0x7f51ac1c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 0x7f51ac1c8730 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f51a000e980 tx=0x7f51a000ec90 comp rx=0 tx=0).stop 2023-09-30T22:25:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 shutdown_connections 2023-09-30T22:25:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f51ac133920 0x7f51ac1c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f51ac09d5c0 0x7f51ac1c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 --2- 172.21.15.32:0/2267607046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51ac0a09c0 0x7f51ac1c8730 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 >> 172.21.15.32:0/2267607046 conn(0x7f51ac0a2fe0 msgr2=0x7f51ac09ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 shutdown_connections 2023-09-30T22:25:06.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.520+0000 7f51b17e0700 1 -- 172.21.15.32:0/2267607046 wait complete. 2023-09-30T22:25:06.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:06.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:06.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.859+0000 7fd214bf4700 1 Processor -- start 2023-09-30T22:25:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.859+0000 7fd214bf4700 1 -- start start 2023-09-30T22:25:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.860+0000 7fd214bf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009d6f0 0x7fd21009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.860+0000 7fd214bf4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21009e000 0x7fd2101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.860+0000 7fd214bf4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21009b120 con 0x7fd21009e410 2023-09-30T22:25:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.860+0000 7fd214bf4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21006b460 con 0x7fd21009e000 2023-09-30T22:25:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.860+0000 7fd214bf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21009a7e0 con 0x7fd21009d6f0 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21009e000 0x7fd2101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009d6f0 0x7fd21009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21009e000 0x7fd2101339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51546/0 (socket says 172.21.15.32:51546) 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 -- 172.21.15.32:0/3572337832 learned_addr learned my addr 172.21.15.32:0/3572337832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20ed9d700 1 --1- 172.21.15.32:0/3572337832 >> v1:172.21.15.32:6789/0 conn(0x7fd21009e410 0x7fd210133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44864/0 (socket says 172.21.15.32:44864) 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 -- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009d6f0 msgr2=0x7fd21009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 --2- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009d6f0 0x7fd21009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 -- 172.21.15.32:0/3572337832 >> v1:172.21.15.32:6789/0 conn(0x7fd21009e410 legacy=0x7fd210133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 -- 172.21.15.32:0/3572337832 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21009a500 con 0x7fd21009e000 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20d59a700 1 -- 172.21.15.32:0/3572337832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002777901 0 0) 0x7fd21009b120 con 0x7fd21009e410 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20e59c700 1 --2- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009d6f0 0x7fd21009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20dd9b700 1 --2- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009e000 0x7fd2101339f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098f0 tx=0x7fd204009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67904f1e4834eb25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.861+0000 7fd20d59a700 1 -- 172.21.15.32:0/3572337832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011830 con 0x7fd21009e000 2023-09-30T22:25:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd20d59a700 1 -- 172.21.15.32:0/3572337832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204011990 con 0x7fd21009e000 2023-09-30T22:25:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd20d59a700 1 -- 172.21.15.32:0/3572337832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204011d20 con 0x7fd21009e000 2023-09-30T22:25:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 -- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009e000 msgr2=0x7fd2101339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 --2- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009e000 0x7fd2101339f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098f0 tx=0x7fd204009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 -- 172.21.15.32:0/3572337832 shutdown_connections 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 --2- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009d6f0 0x7fd21009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 --2- 172.21.15.32:0/3572337832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009e000 0x7fd2101339f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 -- 172.21.15.32:0/3572337832 >> 172.21.15.32:0/3572337832 conn(0x7fd210066f50 msgr2=0x7fd21009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 -- 172.21.15.32:0/3572337832 shutdown_connections 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.862+0000 7fd214bf4700 1 -- 172.21.15.32:0/3572337832 wait complete. 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.864+0000 7fd214bf4700 1 Processor -- start 2023-09-30T22:25:06.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.864+0000 7fd214bf4700 1 -- start start 2023-09-30T22:25:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd214bf4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009d6f0 0x7fd2101a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd214bf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd21009e000 0x7fd2101a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd214bf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 0x7fd2101a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd214bf4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2101a9150 con 0x7fd21009d6f0 2023-09-30T22:25:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd214bf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21009c710 con 0x7fd21009e410 2023-09-30T22:25:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd214bf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100a2e40 con 0x7fd21009e000 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 0x7fd2101a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009d6f0 0x7fd2101a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 0x7fd2101a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49410/0 (socket says 172.21.15.32:49410) 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 -- 172.21.15.32:0/2681410852 learned_addr learned my addr 172.21.15.32:0/2681410852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20dd9b700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd21009e000 0x7fd2101a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 -- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd21009e000 msgr2=0x7fd2101a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd21009e000 0x7fd2101a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 -- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009d6f0 msgr2=0x7fd2101a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009d6f0 0x7fd2101a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20ed9d700 1 -- 172.21.15.32:0/2681410852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000094a0 con 0x7fd21009e410 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20e59c700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009d6f0 0x7fd2101a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.865+0000 7fd20dd9b700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd21009e000 0x7fd2101a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.866+0000 7fd20ed9d700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 0x7fd2101a6920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd208002230 tx=0x7fd20800cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.866+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800aac0 con 0x7fd21009e410 2023-09-30T22:25:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.866+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20800ac40 con 0x7fd21009e410 2023-09-30T22:25:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.866+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080070d0 con 0x7fd21009e410 2023-09-30T22:25:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.866+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd204009550 con 0x7fd21009e410 2023-09-30T22:25:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.866+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2101a8d00 con 0x7fd21009e410 2023-09-30T22:25:06.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.867+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd208007250 con 0x7fd21009e410 2023-09-30T22:25:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.867+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210074ea0 con 0x7fd21009e410 2023-09-30T22:25:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.867+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd20803f110 con 0x7fd21009e410 2023-09-30T22:25:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.872+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd208007500 con 0x7fd21009e410 2023-09-30T22:25:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.907+0000 7fb38d571700 1 Processor -- start 2023-09-30T22:25:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.907+0000 7fb38d571700 1 -- start start 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb38d571700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb38809e850 0x7fb38809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb38d571700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb38d571700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38806bec0 con 0x7fb38809f550 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb38d571700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38806bce0 con 0x7fb38809e850 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb38d571700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38806caa0 con 0x7fb38809f160 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb386ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb38809e850 0x7fb38809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb386ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb38809e850 0x7fb38809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51576/0 (socket says 172.21.15.32:51576) 2023-09-30T22:25:06.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb3877fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb38809f550 0x7fb3881340c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44870/0 (socket says 172.21.15.32:44870) 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb386ffd700 1 -- 172.21.15.32:0/2526219963 learned_addr learned my addr 172.21.15.32:0/2526219963 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.908+0000 7fb3867fc700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb386ffd700 1 -- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 msgr2=0x7fb388133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb386ffd700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388133b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb386ffd700 1 -- 172.21.15.32:0/2526219963 >> v1:172.21.15.32:6789/0 conn(0x7fb38809f550 legacy=0x7fb3881340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb386ffd700 1 -- 172.21.15.32:0/2526219963 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880a00a0 con 0x7fb38809e850 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb385ffb700 1 -- 172.21.15.32:0/2526219963 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217664540 0 0) 0x7fb38806bec0 con 0x7fb38809f550 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb3867fc700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb386ffd700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb38809ec20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb378009e50 tx=0x7fb37800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780e3ffdfedf3cd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb385ffb700 1 -- 172.21.15.32:0/2526219963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800e070 con 0x7fb38809e850 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb385ffb700 1 -- 172.21.15.32:0/2526219963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800b910 con 0x7fb38809e850 2023-09-30T22:25:06.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.909+0000 7fb385ffb700 1 -- 172.21.15.32:0/2526219963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800bcc0 con 0x7fb38809e850 2023-09-30T22:25:06.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 -- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 msgr2=0x7fb38809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb38809ec20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb378009e50 tx=0x7fb37800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 -- 172.21.15.32:0/2526219963 shutdown_connections 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb38809ec20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 --2- 172.21.15.32:0/2526219963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388133b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 -- 172.21.15.32:0/2526219963 >> 172.21.15.32:0/2526219963 conn(0x7fb3880a3f70 msgr2=0x7fb3880a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 -- 172.21.15.32:0/2526219963 shutdown_connections 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.910+0000 7fb38d571700 1 -- 172.21.15.32:0/2526219963 wait complete. 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.911+0000 7fb38d571700 1 Processor -- start 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.911+0000 7fb38d571700 1 -- start start 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb38d571700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb388122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb38d571700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb38d571700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb38809f550 0x7fb388122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb38d571700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388125060 con 0x7fb38809e850 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb38d571700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb388126060 con 0x7fb38809f160 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb38d571700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb388125ac0 con 0x7fb38809f550 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb3867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb388122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb3867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388122a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49428/0 (socket says 172.21.15.32:49428) 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb3867fc700 1 -- 172.21.15.32:0/1597856867 learned_addr learned my addr 172.21.15.32:0/1597856867 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 -- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb38809f550 msgr2=0x7fb388122fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb38809f550 0x7fb388122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 -- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 msgr2=0x7fb388122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388122a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:06.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 -- 172.21.15.32:0/1597856867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0094a0 con 0x7fb38809e850 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb3867fc700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.912+0000 7fb386ffd700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb388122540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb38806b7b0 tx=0x7fb378004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.913+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378003ea0 con 0x7fb38809e850 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.913+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3780092e0 con 0x7fb38809e850 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.913+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb378009b30 con 0x7fb38809e850 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.913+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378004970 con 0x7fb38809e850 2023-09-30T22:25:06.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.913+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388072500 con 0x7fb38809e850 2023-09-30T22:25:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.913+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb37801b630 con 0x7fb38809e850 2023-09-30T22:25:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.914+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb378040bd0 con 0x7fb38809e850 2023-09-30T22:25:06.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.914+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38806cd20 con 0x7fb38809e850 2023-09-30T22:25:06.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:06.919+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb37801f080 con 0x7fb38809e850 2023-09-30T22:25:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.115+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd2101a9560 con 0x7fd21009e410 2023-09-30T22:25:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.116+0000 7fd1feffd700 1 -- 172.21.15.32:0/2681410852 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd208018020 con 0x7fd21009e410 2023-09-30T22:25:07.117 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 msgr2=0x7fd2101a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 0x7fd2101a6920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd208002230 tx=0x7fd20800cf50 comp rx=0 tx=0).stop 2023-09-30T22:25:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 shutdown_connections 2023-09-30T22:25:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd21009e410 0x7fd2101a6920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21009d6f0 0x7fd2101a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 --2- 172.21.15.32:0/2681410852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd21009e000 0x7fd2101a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 >> 172.21.15.32:0/2681410852 conn(0x7fd210066f50 msgr2=0x7fd2101bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 shutdown_connections 2023-09-30T22:25:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.118+0000 7fd214bf4700 1 -- 172.21.15.32:0/2681410852 wait complete. 2023-09-30T22:25:07.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:07.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.149+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb3880747f0 con 0x7fb38809e850 2023-09-30T22:25:07.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.149+0000 7fb36f7fe700 1 -- 172.21.15.32:0/1597856867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb378023770 con 0x7fb38809e850 2023-09-30T22:25:07.150 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 msgr2=0x7fb388122540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb388122540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb38806b7b0 tx=0x7fb378004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 shutdown_connections 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb38809f550 0x7fb388122fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38809e850 0x7fb388122540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 --2- 172.21.15.32:0/1597856867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb38809f160 0x7fb388122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 >> 172.21.15.32:0/1597856867 conn(0x7fb3880a3f70 msgr2=0x7fb388134020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:07.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 shutdown_connections 2023-09-30T22:25:07.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.152+0000 7fb38d571700 1 -- 172.21.15.32:0/1597856867 wait complete. 2023-09-30T22:25:07.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:07.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:07.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:07.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.652+0000 7f92f9c57700 1 Processor -- start 2023-09-30T22:25:07.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.652+0000 7f92f9c57700 1 -- start start 2023-09-30T22:25:07.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f9c57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a0a30 0x7f92f40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f9c57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92f40a1340 0x7f92f4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f9c57700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f409edd0 con 0x7f92f4133fa0 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f9c57700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f409e1f0 con 0x7f92f40a1340 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f9c57700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92f409dc50 con 0x7f92f40a0a30 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92f40a1340 0x7f92f4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92f40a1340 0x7f92f4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51586/0 (socket says 172.21.15.32:51586) 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 -- 172.21.15.32:0/1633217551 learned_addr learned my addr 172.21.15.32:0/1633217551 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f37fe700 1 --2- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a0a30 0x7f92f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 -- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a0a30 msgr2=0x7f92f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 --2- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a0a30 0x7f92f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 -- 172.21.15.32:0/1633217551 >> v1:172.21.15.32:6789/0 conn(0x7f92f4133fa0 legacy=0x7f92f413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 -- 172.21.15.32:0/1633217551 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40a1fe0 con 0x7f92f40a1340 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92eaffd700 1 --2- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a1340 0x7f92f4133a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0098f0 tx=0x7f92dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564b5ec8034b1ced server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.653+0000 7f92f2ffd700 1 -- 172.21.15.32:0/1633217551 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011830 con 0x7f92f40a1340 2023-09-30T22:25:07.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f2ffd700 1 -- 172.21.15.32:0/1633217551 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc011990 con 0x7f92f40a1340 2023-09-30T22:25:07.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f2ffd700 1 -- 172.21.15.32:0/1633217551 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011d20 con 0x7f92f40a1340 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 -- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a1340 msgr2=0x7f92f4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 --2- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a1340 0x7f92f4133a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0098f0 tx=0x7f92dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 -- 172.21.15.32:0/1633217551 shutdown_connections 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 --2- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a0a30 0x7f92f40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 --2- 172.21.15.32:0/1633217551 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a1340 0x7f92f4133a60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 -- 172.21.15.32:0/1633217551 >> 172.21.15.32:0/1633217551 conn(0x7f92f40a6550 msgr2=0x7f92f40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 -- 172.21.15.32:0/1633217551 shutdown_connections 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.654+0000 7f92f9c57700 1 -- 172.21.15.32:0/1633217551 wait complete. 2023-09-30T22:25:07.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.655+0000 7f92f9c57700 1 Processor -- start 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.655+0000 7f92f9c57700 1 -- start start 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f9c57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a0a30 0x7f92f411f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f9c57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 0x7f92f411f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f9c57700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92f4133fa0 0x7f92f41c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f9c57700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f409c030 con 0x7f92f40a0a30 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f9c57700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40a0220 con 0x7f92f4133fa0 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f9c57700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92f41206b0 con 0x7f92f40a1340 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 0x7f92f411f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 0x7f92f411f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43134/0 (socket says 172.21.15.32:43134) 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 -- 172.21.15.32:0/3229511069 learned_addr learned my addr 172.21.15.32:0/3229511069 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f3fff700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92f4133fa0 0x7f92f41c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92f37fe700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a0a30 0x7f92f411f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 -- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92f4133fa0 msgr2=0x7f92f41c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92f4133fa0 0x7f92f41c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 -- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a0a30 msgr2=0x7f92f411f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a0a30 0x7f92f411f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.656+0000 7f92eaffd700 1 -- 172.21.15.32:0/3229511069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc009550 con 0x7f92f40a1340 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.657+0000 7f92eaffd700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 0x7f92f411f9a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000c00 tx=0x7f92dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.657+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0036a0 con 0x7f92f40a1340 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.657+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc003800 con 0x7f92f40a1340 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.657+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc003bb0 con 0x7f92f40a1340 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.657+0000 7f92f9c57700 1 -- 172.21.15.32:0/3229511069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f4120e10 con 0x7f92f40a1340 2023-09-30T22:25:07.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.657+0000 7f92f9c57700 1 -- 172.21.15.32:0/3229511069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f411ef50 con 0x7f92f40a1340 2023-09-30T22:25:07.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.658+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92dc021070 con 0x7f92f40a1340 2023-09-30T22:25:07.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.658+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc0052e0 con 0x7f92f40a1340 2023-09-30T22:25:07.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.658+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f92dc040da0 con 0x7f92f40a1340 2023-09-30T22:25:07.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.663+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92dc02d3a0 con 0x7f92f40a1340 2023-09-30T22:25:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.690+0000 7faa9dd63700 1 Processor -- start 2023-09-30T22:25:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.690+0000 7faa9dd63700 1 -- start start 2023-09-30T22:25:07.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa9dd63700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa9809d6f0 0x7faa9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa9dd63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809e440 0x7faa98134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa9dd63700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa9809b120 con 0x7faa9809e000 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa9dd63700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9806b460 con 0x7faa9809d6f0 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa9dd63700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9809a7e0 con 0x7faa9809e440 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa9809d6f0 0x7faa9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa97fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809e440 0x7faa98134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa9809d6f0 0x7faa9809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51606/0 (socket says 172.21.15.32:51606) 2023-09-30T22:25:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 -- 172.21.15.32:0/106116939 learned_addr learned my addr 172.21.15.32:0/106116939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 -- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809e440 msgr2=0x7faa98134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 --2- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809e440 0x7faa98134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 -- 172.21.15.32:0/106116939 >> v1:172.21.15.32:6789/0 conn(0x7faa9809e000 legacy=0x7faa981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 -- 172.21.15.32:0/106116939 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9809a500 con 0x7faa9809d6f0 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.691+0000 7faa977fe700 1 --2- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809d6f0 0x7faa9809dac0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faa88009920 tx=0x7faa88009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaadb9ff3c67fcf9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa967fc700 1 -- 172.21.15.32:0/106116939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88011830 con 0x7faa9809d6f0 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa967fc700 1 -- 172.21.15.32:0/106116939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88011990 con 0x7faa9809d6f0 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa967fc700 1 -- 172.21.15.32:0/106116939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88011d20 con 0x7faa9809d6f0 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 -- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809d6f0 msgr2=0x7faa9809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 --2- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809d6f0 0x7faa9809dac0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faa88009920 tx=0x7faa88009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 -- 172.21.15.32:0/106116939 shutdown_connections 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 --2- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809e440 0x7faa98134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 --2- 172.21.15.32:0/106116939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809d6f0 0x7faa9809dac0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 -- 172.21.15.32:0/106116939 >> 172.21.15.32:0/106116939 conn(0x7faa98066f50 msgr2=0x7faa9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 -- 172.21.15.32:0/106116939 shutdown_connections 2023-09-30T22:25:07.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.692+0000 7faa9dd63700 1 -- 172.21.15.32:0/106116939 wait complete. 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.693+0000 7faa9dd63700 1 Processor -- start 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 -- start start 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 0x7faa981a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa9809e000 0x7faa981a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809e440 0x7faa981a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9809c710 con 0x7faa9809e440 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa981a81b0 con 0x7faa9809e000 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa9dd63700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa981a7b60 con 0x7faa9809d6f0 2023-09-30T22:25:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa977fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 0x7faa981a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa96ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa9809e000 0x7faa981a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809e440 0x7faa981a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa977fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 0x7faa981a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43158/0 (socket says 172.21.15.32:43158) 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.694+0000 7faa977fe700 1 -- 172.21.15.32:0/2277570693 learned_addr learned my addr 172.21.15.32:0/2277570693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.695+0000 7faa977fe700 1 -- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa9809e000 msgr2=0x7faa981a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.695+0000 7faa977fe700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa9809e000 0x7faa981a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.695+0000 7faa977fe700 1 -- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809e440 msgr2=0x7faa981a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.695+0000 7faa977fe700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809e440 0x7faa981a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.695+0000 7faa977fe700 1 -- 172.21.15.32:0/2277570693 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88009550 con 0x7faa9809d6f0 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.695+0000 7faa97fff700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809e440 0x7faa981a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa977fe700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 0x7faa981a5fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa88009c70 tx=0x7faa88004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88004300 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8801f070 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa981a9a70 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880036a0 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa981a9790 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa88021030 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa88040820 con 0x7faa9809d6f0 2023-09-30T22:25:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.696+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa981a9340 con 0x7faa9809d6f0 2023-09-30T22:25:07.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.701+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa88015080 con 0x7faa9809d6f0 2023-09-30T22:25:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.871+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f92cc005590 con 0x7f92f40a1340 2023-09-30T22:25:07.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.871+0000 7f92f0ff9700 1 -- 172.21.15.32:0/3229511069 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92dc029030 con 0x7f92f40a1340 2023-09-30T22:25:07.872 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 msgr2=0x7f92f411f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 0x7f92f411f9a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92dc000c00 tx=0x7f92dc004750 comp rx=0 tx=0).stop 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 shutdown_connections 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92f4133fa0 0x7f92f41c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92f40a0a30 0x7f92f411f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 --2- 172.21.15.32:0/3229511069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92f40a1340 0x7f92f411f9a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 >> 172.21.15.32:0/3229511069 conn(0x7f92f40a6550 msgr2=0x7f92f41b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:07.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 shutdown_connections 2023-09-30T22:25:07.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.874+0000 7f92e9ffb700 1 -- 172.21.15.32:0/3229511069 wait complete. 2023-09-30T22:25:07.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:07.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.910+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa98074ea0 con 0x7faa9809d6f0 2023-09-30T22:25:07.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.910+0000 7faa7ffff700 1 -- 172.21.15.32:0/2277570693 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa880199a0 con 0x7faa9809d6f0 2023-09-30T22:25:07.911 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 msgr2=0x7faa981a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 0x7faa981a5fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa88009c70 tx=0x7faa88004750 comp rx=0 tx=0).stop 2023-09-30T22:25:07.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 shutdown_connections 2023-09-30T22:25:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa9809e440 0x7faa981a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa9809d6f0 0x7faa981a5fa0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 --2- 172.21.15.32:0/2277570693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa9809e000 0x7faa981a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.912+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 >> 172.21.15.32:0/2277570693 conn(0x7faa98066f50 msgr2=0x7faa981baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.913+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 shutdown_connections 2023-09-30T22:25:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:07.913+0000 7faa9dd63700 1 -- 172.21.15.32:0/2277570693 wait complete. 2023-09-30T22:25:07.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:08.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:08.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.284+0000 7f304d4ee700 1 Processor -- start 2023-09-30T22:25:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.284+0000 7f304d4ee700 1 -- start start 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f304d4ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f304d4ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f3048131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f304d4ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3048070940 con 0x7f304809f240 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f304d4ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304806fd60 con 0x7f304809f650 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f304d4ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f304806fa80 con 0x7f304809e930 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f3046ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f30477fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f3048131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f3046ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43172/0 (socket says 172.21.15.32:43172) 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f30477fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f3048131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49444/0 (socket says 172.21.15.32:49444) 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f3046ffd700 1 -- 172.21.15.32:0/2617716907 learned_addr learned my addr 172.21.15.32:0/2617716907 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f3046ffd700 1 -- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 msgr2=0x7f3048131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f3046ffd700 1 --2- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f3048131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.286+0000 7f3046ffd700 1 -- 172.21.15.32:0/2617716907 >> v1:172.21.15.32:6789/0 conn(0x7f304809f240 legacy=0x7f30481317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.285+0000 7f3045ffb700 1 -- 172.21.15.32:0/2617716907 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467216299 0 0) 0x7f3048070940 con 0x7f304809f240 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.286+0000 7f3046ffd700 1 -- 172.21.15.32:0/2617716907 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304809fb60 con 0x7f304809e930 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.286+0000 7f3046ffd700 1 --2- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304809ed00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3038009e80 tx=0x7f303800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9ccf6fa37bbc62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.286+0000 7f3045ffb700 1 -- 172.21.15.32:0/2617716907 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800e070 con 0x7f304809e930 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.286+0000 7f3045ffb700 1 -- 172.21.15.32:0/2617716907 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303800b910 con 0x7f304809e930 2023-09-30T22:25:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.286+0000 7f3045ffb700 1 -- 172.21.15.32:0/2617716907 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800bcc0 con 0x7f304809e930 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 -- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 msgr2=0x7f304809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304809ed00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3038009e80 tx=0x7f303800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 -- 172.21.15.32:0/2617716907 shutdown_connections 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f3048131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2617716907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304809ed00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 -- 172.21.15.32:0/2617716907 >> 172.21.15.32:0/2617716907 conn(0x7f30480a4210 msgr2=0x7f30480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 -- 172.21.15.32:0/2617716907 shutdown_connections 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.287+0000 7f304d4ee700 1 -- 172.21.15.32:0/2617716907 wait complete. 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 Processor -- start 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 -- start start 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f304809f240 0x7f304806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f304806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30481a9680 con 0x7f304809f240 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30481a8910 con 0x7f304809f650 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.288+0000 7f304d4ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30481b7200 con 0x7f304809e930 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43174/0 (socket says 172.21.15.32:43174) 2023-09-30T22:25:08.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 -- 172.21.15.32:0/2181695486 learned_addr learned my addr 172.21.15.32:0/2181695486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f30477fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f304806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f30467fc700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f304809f240 0x7f304806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 -- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 msgr2=0x7f304806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f304806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 -- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f304809f240 msgr2=0x7f304806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f304809f240 0x7f304806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 -- 172.21.15.32:0/2181695486 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038009b30 con 0x7f304809e930 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f30477fe700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f304806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f3046ffd700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304806b8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f303800b680 tx=0x7f3038004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30380092e0 con 0x7f304809e930 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3038004250 con 0x7f304809e930 2023-09-30T22:25:08.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480a0190 con 0x7f304809e930 2023-09-30T22:25:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30380045e0 con 0x7f304809e930 2023-09-30T22:25:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.289+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3048071d20 con 0x7f304809e930 2023-09-30T22:25:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.290+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3038004740 con 0x7f304809e930 2023-09-30T22:25:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.290+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048074ea0 con 0x7f304809e930 2023-09-30T22:25:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.290+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3038040a80 con 0x7f304809e930 2023-09-30T22:25:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.300+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3038020080 con 0x7f304809e930 2023-09-30T22:25:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 Processor -- start 2023-09-30T22:25:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 -- start start 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb640a0a30 0x7fcb640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a1340 0x7fcb64133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6409edd0 con 0x7fcb64133fa0 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6409e1f0 con 0x7fcb640a0a30 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb6a6d8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6409dc50 con 0x7fcb640a1340 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a1340 0x7fcb64133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb68c75700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb64133fa0 0x7fcb6413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44918/0 (socket says 172.21.15.32:44918) 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb68c75700 1 -- 172.21.15.32:0/3800269667 learned_addr learned my addr 172.21.15.32:0/3800269667 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.326+0000 7fcb63fff700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb640a0a30 0x7fcb640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb62ffd700 1 -- 172.21.15.32:0/3800269667 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764331073 0 0) 0x7fcb6409edd0 con 0x7fcb64133fa0 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb63fff700 1 -- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a1340 msgr2=0x7fcb64133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb63fff700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a1340 0x7fcb64133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb63fff700 1 -- 172.21.15.32:0/3800269667 >> v1:172.21.15.32:6789/0 conn(0x7fcb64133fa0 legacy=0x7fcb6413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb63fff700 1 -- 172.21.15.32:0/3800269667 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640a1fe0 con 0x7fcb640a0a30 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb637fe700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a1340 0x7fcb64133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb63fff700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb640a0a30 0x7fcb640a0e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009870 tx=0x7fcb54009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f4588d0bf415bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb62ffd700 1 -- 172.21.15.32:0/3800269667 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54011830 con 0x7fcb640a0a30 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb62ffd700 1 -- 172.21.15.32:0/3800269667 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54011990 con 0x7fcb640a0a30 2023-09-30T22:25:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.327+0000 7fcb62ffd700 1 -- 172.21.15.32:0/3800269667 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54011d20 con 0x7fcb640a0a30 2023-09-30T22:25:08.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb640a0a30 msgr2=0x7fcb640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb640a0a30 0x7fcb640a0e00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009870 tx=0x7fcb54009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:08.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/3800269667 shutdown_connections 2023-09-30T22:25:08.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb640a0a30 0x7fcb640a0e00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/3800269667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a1340 0x7fcb64133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/3800269667 >> 172.21.15.32:0/3800269667 conn(0x7fcb640a6550 msgr2=0x7fcb640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/3800269667 shutdown_connections 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.328+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/3800269667 wait complete. 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.329+0000 7fcb6a6d8700 1 Processor -- start 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.329+0000 7fcb6a6d8700 1 -- start start 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb6a6d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a0a30 0x7fcb641c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb6a6d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 0x7fcb641c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb6a6d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb64133fa0 0x7fcb641c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb6a6d8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64126370 con 0x7fcb640a1340 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb6a6d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64124660 con 0x7fcb64133fa0 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb6a6d8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64124c10 con 0x7fcb640a0a30 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 0x7fcb641c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb68c75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb64133fa0 0x7fcb641c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 0x7fcb641c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51622/0 (socket says 172.21.15.32:51622) 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 -- 172.21.15.32:0/2446253059 learned_addr learned my addr 172.21.15.32:0/2446253059 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb63fff700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a0a30 0x7fcb641c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 -- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a0a30 msgr2=0x7fcb641c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a0a30 0x7fcb641c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 -- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb64133fa0 msgr2=0x7fcb641c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb64133fa0 0x7fcb641c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 -- 172.21.15.32:0/2446253059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb580094a0 con 0x7fcb640a1340 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb63fff700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a0a30 0x7fcb641c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb68c75700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb64133fa0 0x7fcb641c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb637fe700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 0x7fcb641c88c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800ebe0 tx=0x7fcb5800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.330+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800c850 con 0x7fcb640a1340 2023-09-30T22:25:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.331+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5800c9b0 con 0x7fcb640a1340 2023-09-30T22:25:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.331+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb540094a0 con 0x7fcb640a1340 2023-09-30T22:25:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.331+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800cd40 con 0x7fcb640a1340 2023-09-30T22:25:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.331+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640a25e0 con 0x7fcb640a1340 2023-09-30T22:25:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.332+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb58013470 con 0x7fcb640a1340 2023-09-30T22:25:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.332+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64074ea0 con 0x7fcb640a1340 2023-09-30T22:25:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.332+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb5803f770 con 0x7fcb640a1340 2023-09-30T22:25:08.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.337+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb58013720 con 0x7fcb640a1340 2023-09-30T22:25:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.507+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f30481a8c90 con 0x7f304809e930 2023-09-30T22:25:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.508+0000 7f303f7fe700 1 -- 172.21.15.32:0/2181695486 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3038019350 con 0x7f304809e930 2023-09-30T22:25:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 msgr2=0x7f304806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304806b8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f303800b680 tx=0x7f3038004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 shutdown_connections 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304809f650 0x7f304806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304809e930 0x7f304806b8b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 --2- 172.21.15.32:0/2181695486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f304809f240 0x7f304806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 >> 172.21.15.32:0/2181695486 conn(0x7f30480a4210 msgr2=0x7f30481b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 shutdown_connections 2023-09-30T22:25:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.510+0000 7f304d4ee700 1 -- 172.21.15.32:0/2181695486 wait complete. 2023-09-30T22:25:08.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.543+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb640a28e0 con 0x7fcb640a1340 2023-09-30T22:25:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.543+0000 7fcb60ff9700 1 -- 172.21.15.32:0/2446253059 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb58021070 con 0x7fcb640a1340 2023-09-30T22:25:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 msgr2=0x7fcb641c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 0x7fcb641c88c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800ebe0 tx=0x7fcb5800eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 shutdown_connections 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb64133fa0 0x7fcb641c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb640a0a30 0x7fcb641c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 --2- 172.21.15.32:0/2446253059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb640a1340 0x7fcb641c88c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 >> 172.21.15.32:0/2446253059 conn(0x7fcb640a6550 msgr2=0x7fcb640a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 shutdown_connections 2023-09-30T22:25:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:08.546+0000 7fcb6a6d8700 1 -- 172.21.15.32:0/2446253059 wait complete. 2023-09-30T22:25:08.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:08.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:08.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 Processor -- start 2023-09-30T22:25:09.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 -- start start 2023-09-30T22:25:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 0x7fd1d013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d009edd0 con 0x7fd1d00a1340 2023-09-30T22:25:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d009e1f0 con 0x7fd1d0134170 2023-09-30T22:25:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.037+0000 7fd1d7a7b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d009dc50 con 0x7fd1d00a0a30 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d5817700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 0x7fd1d013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d5817700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43200/0 (socket says 172.21.15.32:43200) 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d5016700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1d00a1340 0x7fd1d0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44928/0 (socket says 172.21.15.32:44928) 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d5817700 1 -- 172.21.15.32:0/3547376497 learned_addr learned my addr 172.21.15.32:0/3547376497 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 0x7fd1d013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49496/0 (socket says 172.21.15.32:49496) 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 -- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 msgr2=0x7fd1d00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 -- 172.21.15.32:0/3547376497 >> v1:172.21.15.32:6789/0 conn(0x7fd1d00a1340 legacy=0x7fd1d0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d4815700 1 -- 172.21.15.32:0/3547376497 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657254575 0 0) 0x7fd1d009edd0 con 0x7fd1d00a1340 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 -- 172.21.15.32:0/3547376497 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00a1fe0 con 0x7fd1d0134170 2023-09-30T22:25:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d5817700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d6018700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 0x7fd1d013bd30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009dc0 tx=0x7fd1c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26f807042f7c84e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d4815700 1 -- 172.21.15.32:0/3547376497 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800e070 con 0x7fd1d0134170 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d4815700 1 -- 172.21.15.32:0/3547376497 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800bb20 con 0x7fd1d0134170 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.038+0000 7fd1d4815700 1 -- 172.21.15.32:0/3547376497 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8012430 con 0x7fd1d0134170 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 msgr2=0x7fd1d013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 0x7fd1d013bd30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009dc0 tx=0x7fd1c800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/3547376497 shutdown_connections 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d0134170 0x7fd1d013bd30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/3547376497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/3547376497 >> 172.21.15.32:0/3547376497 conn(0x7fd1d00a6550 msgr2=0x7fd1d00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/3547376497 shutdown_connections 2023-09-30T22:25:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.039+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/3547376497 wait complete. 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.040+0000 7fd1d7a7b700 1 Processor -- start 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.040+0000 7fd1d7a7b700 1 -- start start 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d7a7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d0122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d7a7b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d00a1340 0x7fd1d0122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d7a7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d0134170 0x7fd1d0123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d7a7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00a4ec0 con 0x7fd1d0134170 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d7a7b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01336b0 con 0x7fd1d00a1340 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d7a7b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0124410 con 0x7fd1d00a0a30 2023-09-30T22:25:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5016700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d00a1340 0x7fd1d0122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d0122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5016700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d00a1340 0x7fd1d0122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49504/0 (socket says 172.21.15.32:49504) 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d6018700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d0134170 0x7fd1d0123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5016700 1 -- 172.21.15.32:0/1651944253 learned_addr learned my addr 172.21.15.32:0/1651944253 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 -- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d00a1340 msgr2=0x7fd1d0122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d00a1340 0x7fd1d0122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 -- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d0134170 msgr2=0x7fd1d0123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d0134170 0x7fd1d0123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 -- 172.21.15.32:0/1651944253 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc009550 con 0x7fd1d00a0a30 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d6018700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d0134170 0x7fd1d0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.041+0000 7fd1d5817700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d0122780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00e9d0 tx=0x7fd1cc00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.042+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00ee70 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.042+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc004030 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.042+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c8009a70 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.042+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc0043c0 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.042+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d00a25e0 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.042+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1cc00c850 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.043+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd1cc03f530 con 0x7fd1d00a0a30 2023-09-30T22:25:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.043+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d0074ea0 con 0x7fd1d00a0a30 2023-09-30T22:25:09.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.052+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1cc019d40 con 0x7fd1d00a0a30 2023-09-30T22:25:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.079+0000 7f1990f1e700 1 Processor -- start 2023-09-30T22:25:09.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.079+0000 7f1990f1e700 1 -- start start 2023-09-30T22:25:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1990f1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c0a0a30 0x7f198c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1990f1e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c134170 0x7f198c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1990f1e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c09edd0 con 0x7f198c0a1340 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1990f1e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c09e1f0 con 0x7f198c0a0a30 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1990f1e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f198c09dc50 con 0x7f198c134170 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c134170 0x7f198c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1989d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f198c0a1340 0x7f198c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44930/0 (socket says 172.21.15.32:44930) 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f1989d9b700 1 -- 172.21.15.32:0/3235556106 learned_addr learned my addr 172.21.15.32:0/3235556106 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198a59c700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c0a0a30 0x7f198c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198a59c700 1 -- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c134170 msgr2=0x7f198c13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198a59c700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c134170 0x7f198c13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198959a700 1 -- 172.21.15.32:0/3235556106 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840473904 0 0) 0x7f198c09edd0 con 0x7f198c0a1340 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198a59c700 1 -- 172.21.15.32:0/3235556106 >> v1:172.21.15.32:6789/0 conn(0x7f198c0a1340 legacy=0x7f198c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198a59c700 1 -- 172.21.15.32:0/3235556106 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0a1fe0 con 0x7f198c0a0a30 2023-09-30T22:25:09.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198ad9d700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c134170 0x7f198c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.080+0000 7f198a59c700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c0a0a30 0x7f198c0a0e00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f197c009870 tx=0x7f197c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a694d0064d8d692 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f198959a700 1 -- 172.21.15.32:0/3235556106 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c011830 con 0x7f198c0a0a30 2023-09-30T22:25:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f198959a700 1 -- 172.21.15.32:0/3235556106 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c011990 con 0x7f198c0a0a30 2023-09-30T22:25:09.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f198959a700 1 -- 172.21.15.32:0/3235556106 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c011d20 con 0x7f198c0a0a30 2023-09-30T22:25:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 -- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c0a0a30 msgr2=0x7f198c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c0a0a30 0x7f198c0a0e00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f197c009870 tx=0x7f197c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:09.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 -- 172.21.15.32:0/3235556106 shutdown_connections 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c134170 0x7f198c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 --2- 172.21.15.32:0/3235556106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c0a0a30 0x7f198c0a0e00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 -- 172.21.15.32:0/3235556106 >> 172.21.15.32:0/3235556106 conn(0x7f198c0a6550 msgr2=0x7f198c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 -- 172.21.15.32:0/3235556106 shutdown_connections 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.081+0000 7f1990f1e700 1 -- 172.21.15.32:0/3235556106 wait complete. 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.082+0000 7f1990f1e700 1 Processor -- start 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 -- start start 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c0a0a30 0x7f198c1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 0x7f198c1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 0x7f198c1c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c126440 con 0x7f198c0a1340 2023-09-30T22:25:09.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1246d0 con 0x7f198c134170 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1990f1e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1244a0 con 0x7f198c0a0a30 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1989d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 0x7f198c1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f198ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 0x7f198c1c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1989d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 0x7f198c1c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51636/0 (socket says 172.21.15.32:51636) 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f198ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 0x7f198c1c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49528/0 (socket says 172.21.15.32:49528) 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1989d9b700 1 -- 172.21.15.32:0/903242688 learned_addr learned my addr 172.21.15.32:0/903242688 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f198a59c700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c0a0a30 0x7f198c1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.083+0000 7f1989d9b700 1 -- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c0a0a30 msgr2=0x7f198c1c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1989d9b700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c0a0a30 0x7f198c1c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1989d9b700 1 -- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 msgr2=0x7f198c1c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1989d9b700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 0x7f198c1c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1989d9b700 1 -- 172.21.15.32:0/903242688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1984009a70 con 0x7f198c0a1340 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f198ad9d700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 0x7f198c1c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1989d9b700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 0x7f198c1c8990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f198000c550 tx=0x7f198000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980004c80 con 0x7f198c0a1340 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980004e00 con 0x7f198c0a1340 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c0094a0 con 0x7f198c0a1340 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980007310 con 0x7f198c0a1340 2023-09-30T22:25:09.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.084+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c0a25e0 con 0x7f198c0a1340 2023-09-30T22:25:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.085+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1980007470 con 0x7f198c0a1340 2023-09-30T22:25:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.085+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c09c7e0 con 0x7f198c0a1340 2023-09-30T22:25:09.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.085+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f198003f2d0 con 0x7f198c0a1340 2023-09-30T22:25:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.090+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19800133c0 con 0x7f198c0a1340 2023-09-30T22:25:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.262+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1d00a28e0 con 0x7fd1d00a0a30 2023-09-30T22:25:09.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.263+0000 7fd1c67fc700 1 -- 172.21.15.32:0/1651944253 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1cc019300 con 0x7fd1d00a0a30 2023-09-30T22:25:09.263 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 msgr2=0x7fd1d0122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d0122780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00e9d0 tx=0x7fd1cc00ece0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 shutdown_connections 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1d0134170 0x7fd1d0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1d00a0a30 0x7fd1d0122780 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 --2- 172.21.15.32:0/1651944253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1d00a1340 0x7fd1d0122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 >> 172.21.15.32:0/1651944253 conn(0x7fd1d00a6550 msgr2=0x7fd1d009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 shutdown_connections 2023-09-30T22:25:09.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.265+0000 7fd1d7a7b700 1 -- 172.21.15.32:0/1651944253 wait complete. 2023-09-30T22:25:09.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:09.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.300+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f198c126260 con 0x7f198c0a1340 2023-09-30T22:25:09.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.300+0000 7f197affd700 1 -- 172.21.15.32:0/903242688 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1980011360 con 0x7f198c0a1340 2023-09-30T22:25:09.300 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:09.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 msgr2=0x7f198c1c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 0x7f198c1c8990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f198000c550 tx=0x7f198000c860 comp rx=0 tx=0).stop 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 shutdown_connections 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f198c134170 0x7f198c1c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f198c0a0a30 0x7f198c1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 --2- 172.21.15.32:0/903242688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f198c0a1340 0x7f198c1c8990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 >> 172.21.15.32:0/903242688 conn(0x7f198c0a6550 msgr2=0x7f198c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 shutdown_connections 2023-09-30T22:25:09.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.302+0000 7f1990f1e700 1 -- 172.21.15.32:0/903242688 wait complete. 2023-09-30T22:25:09.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:09.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:09.561 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.675+0000 7f7be0319700 1 Processor -- start 2023-09-30T22:25:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.675+0000 7f7be0319700 1 -- start start 2023-09-30T22:25:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.675+0000 7f7be0319700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd80a0a30 0x7f7bd80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.675+0000 7f7be0319700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 0x7f7bd8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7be0319700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bd8133fa0 0x7f7bd8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7be0319700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd809edd0 con 0x7f7bd8133fa0 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7be0319700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd809e1f0 con 0x7f7bd80a1340 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7be0319700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd809dc50 con 0x7f7bd80a0a30 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bd8133fa0 0x7f7bd8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde0b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd80a0a30 0x7f7bd80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bdd8b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 0x7f7bd8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde0b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd80a0a30 0x7f7bd80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43226/0 (socket says 172.21.15.32:43226) 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bdd8b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 0x7f7bd8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:49542/0 (socket says 172.21.15.32:49542) 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde0b5700 1 -- 172.21.15.32:0/2349567756 learned_addr learned my addr 172.21.15.32:0/2349567756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bd8133fa0 0x7f7bd8134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51650/0 (socket says 172.21.15.32:51650) 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 -- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd80a0a30 msgr2=0x7f7bd80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd80a0a30 0x7f7bd80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 -- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 msgr2=0x7f7bd8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 0x7f7bd8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 -- 172.21.15.32:0/2349567756 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80a1fe0 con 0x7f7bd8133fa0 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bdd8b4700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 0x7f7bd8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bde8b6700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd8133fa0 0x7f7bd8134410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009840 tx=0x7f7bd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb161f2106564358 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.676+0000 7f7bdd0b3700 1 -- 172.21.15.32:0/2349567756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0011830 con 0x7f7bd8133fa0 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.677+0000 7f7bdd0b3700 1 -- 172.21.15.32:0/2349567756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd0011990 con 0x7f7bd8133fa0 2023-09-30T22:25:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.677+0000 7f7bdd0b3700 1 -- 172.21.15.32:0/2349567756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0011d20 con 0x7f7bd8133fa0 2023-09-30T22:25:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.677+0000 7f7be0319700 1 -- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd8133fa0 msgr2=0x7f7bd8134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.677+0000 7f7be0319700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd8133fa0 0x7f7bd8134410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009840 tx=0x7f7bd0009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 -- 172.21.15.32:0/2349567756 shutdown_connections 2023-09-30T22:25:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd8133fa0 0x7f7bd8134410 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd80a0a30 0x7f7bd80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 --2- 172.21.15.32:0/2349567756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a1340 0x7f7bd8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 -- 172.21.15.32:0/2349567756 >> 172.21.15.32:0/2349567756 conn(0x7f7bd80a6550 msgr2=0x7f7bd80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 -- 172.21.15.32:0/2349567756 shutdown_connections 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.678+0000 7f7be0319700 1 -- 172.21.15.32:0/2349567756 wait complete. 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 Processor -- start 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 -- start start 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a0a30 0x7f7bd8120030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 0x7f7bd8120570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 0x7f7bd8120ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8124510 con 0x7f7bd80a1340 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8123b50 con 0x7f7bd80a0a30 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7be0319700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80a35a0 con 0x7f7bd8133fa0 2023-09-30T22:25:09.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7bde8b6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 0x7f7bd8120ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7bdd8b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 0x7f7bd8120570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7bdd8b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 0x7f7bd8120570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51662/0 (socket says 172.21.15.32:51662) 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7bde8b6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 0x7f7bd8120ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43228/0 (socket says 172.21.15.32:43228) 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7bdd8b4700 1 -- 172.21.15.32:0/4038354215 learned_addr learned my addr 172.21.15.32:0/4038354215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.679+0000 7f7bde0b5700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a0a30 0x7f7bd8120030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bdd8b4700 1 -- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 msgr2=0x7f7bd8120ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bdd8b4700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 0x7f7bd8120ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bdd8b4700 1 -- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a0a30 msgr2=0x7f7bd8120030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bdd8b4700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a0a30 0x7f7bd8120030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bdd8b4700 1 -- 172.21.15.32:0/4038354215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80094a0 con 0x7f7bd80a1340 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bde8b6700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 0x7f7bd8120ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bde0b5700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a0a30 0x7f7bd8120030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bdd8b4700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 0x7f7bd8120570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800ebe0 tx=0x7f7bc800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800c850 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800c9b0 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd00094a0 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800cd40 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.680+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd80a0220 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.681+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7bc8013470 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.681+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7bc803fa70 con 0x7f7bd80a1340 2023-09-30T22:25:09.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.681+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd8123510 con 0x7f7bd80a1340 2023-09-30T22:25:09.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.686+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7bd8123510 con 0x7f7bd80a1340 2023-09-30T22:25:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.715+0000 7ff6e9584700 1 Processor -- start 2023-09-30T22:25:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.715+0000 7ff6e9584700 1 -- start start 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e9584700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e9584700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6e4138d80 0x7ff6e413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e9584700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40a2a70 con 0x7ff6e40a0a30 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e9584700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40a01d0 con 0x7ff6e4138d80 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e9584700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e409e2b0 con 0x7ff6e409d630 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff6e40a0a30 0x7ff6e40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44938/0 (socket says 172.21.15.32:44938) 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e2ffd700 1 -- 172.21.15.32:0/2839200495 learned_addr learned my addr 172.21.15.32:0/2839200495 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e27fc700 1 --2- 172.21.15.32:0/2839200495 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e37fe700 1 --2- 172.21.15.32:0/2839200495 >> v2:172.21.15.32:3300/0 conn(0x7ff6e4138d80 0x7ff6e413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e1ffb700 1 -- 172.21.15.32:0/2839200495 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390744937 0 0) 0x7ff6e40a2a70 con 0x7ff6e40a0a30 2023-09-30T22:25:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e1ffb700 1 -- 172.21.15.32:0/2839200495 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d00035b0 con 0x7ff6e40a0a30 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e27fc700 1 -- 172.21.15.32:0/2839200495 >> v2:172.21.15.32:3300/0 conn(0x7ff6e4138d80 msgr2=0x7ff6e413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e27fc700 1 --2- 172.21.15.32:0/2839200495 >> v2:172.21.15.32:3300/0 conn(0x7ff6e4138d80 0x7ff6e413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e27fc700 1 -- 172.21.15.32:0/2839200495 >> v1:172.21.15.32:6789/0 conn(0x7ff6e40a0a30 legacy=0x7ff6e40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.716+0000 7ff6e27fc700 1 -- 172.21.15.32:0/2839200495 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e409dc50 con 0x7ff6e409d630 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e27fc700 1 --2- 172.21.15.32:0/2839200495 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e40a1510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009840 tx=0x7ff6cc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a466467ddbafd23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e1ffb700 1 -- 172.21.15.32:0/2839200495 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc011830 con 0x7ff6e409d630 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e1ffb700 1 -- 172.21.15.32:0/2839200495 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc011990 con 0x7ff6e409d630 2023-09-30T22:25:09.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e1ffb700 1 -- 172.21.15.32:0/2839200495 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc011d20 con 0x7ff6e409d630 2023-09-30T22:25:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e9584700 1 -- 172.21.15.32:0/2839200495 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 msgr2=0x7ff6e40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e9584700 1 --2- 172.21.15.32:0/2839200495 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e40a1510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009840 tx=0x7ff6cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e9584700 1 -- 172.21.15.32:0/2839200495 shutdown_connections 2023-09-30T22:25:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e9584700 1 --2- 172.21.15.32:0/2839200495 >> v2:172.21.15.32:3300/0 conn(0x7ff6e4138d80 0x7ff6e413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e9584700 1 --2- 172.21.15.32:0/2839200495 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e40a1510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.717+0000 7ff6e9584700 1 -- 172.21.15.32:0/2839200495 >> 172.21.15.32:0/2839200495 conn(0x7ff6e40a3050 msgr2=0x7ff6e40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.718+0000 7ff6e9584700 1 -- 172.21.15.32:0/2839200495 shutdown_connections 2023-09-30T22:25:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.718+0000 7ff6e9584700 1 -- 172.21.15.32:0/2839200495 wait complete. 2023-09-30T22:25:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.719+0000 7ff6e9584700 1 Processor -- start 2023-09-30T22:25:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.719+0000 7ff6e9584700 1 -- start start 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e9584700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e41335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e9584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40a0a30 0x7ff6e4133ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e9584700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6e4138d80 0x7ff6e4134020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e9584700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4136470 con 0x7ff6e40a0a30 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e9584700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4136de0 con 0x7ff6e4138d80 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e9584700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e41360c0 con 0x7ff6e409d630 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e41335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6e4138d80 0x7ff6e4134020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e41335a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43238/0 (socket says 172.21.15.32:43238) 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 -- 172.21.15.32:0/1671497209 learned_addr learned my addr 172.21.15.32:0/1671497209 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e27fc700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40a0a30 0x7ff6e4133ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 -- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6e4138d80 msgr2=0x7ff6e4134020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6e4138d80 0x7ff6e4134020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 -- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40a0a30 msgr2=0x7ff6e4133ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40a0a30 0x7ff6e4133ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e2ffd700 1 -- 172.21.15.32:0/1671497209 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0094a0 con 0x7ff6e409d630 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.720+0000 7ff6e37fe700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6e4138d80 0x7ff6e4134020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.721+0000 7ff6e2ffd700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e41335a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400cbf0 tx=0x7ff6d400cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.721+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400aa60 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.721+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400abc0 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.721+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e413ab80 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.721+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4012400 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.721+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e41375c0 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.722+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6d401b3b0 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.722+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e4136a50 con 0x7ff6e409d630 2023-09-30T22:25:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.722+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff6d403f460 con 0x7ff6e409d630 2023-09-30T22:25:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.727+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6d40113a0 con 0x7ff6e409d630 2023-09-30T22:25:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.912+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7bd8123510 con 0x7f7bd80a1340 2023-09-30T22:25:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.913+0000 7f7bceffd700 1 -- 172.21.15.32:0/4038354215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7bd8123510 con 0x7f7bd80a1340 2023-09-30T22:25:09.913 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 msgr2=0x7f7bd8120570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 0x7f7bd8120570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800ebe0 tx=0x7f7bc800eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 shutdown_connections 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bd8133fa0 0x7f7bd8120ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bd80a0a30 0x7f7bd8120030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 --2- 172.21.15.32:0/4038354215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bd80a1340 0x7f7bd8120570 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 >> 172.21.15.32:0/4038354215 conn(0x7f7bd80a6550 msgr2=0x7f7bd81baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 shutdown_connections 2023-09-30T22:25:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.915+0000 7f7be0319700 1 -- 172.21.15.32:0/4038354215 wait complete. 2023-09-30T22:25:09.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.952+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff6e40747f0 con 0x7ff6e409d630 2023-09-30T22:25:09.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.952+0000 7ff6db7fe700 1 -- 172.21.15.32:0/1671497209 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6d4010300 con 0x7ff6e409d630 2023-09-30T22:25:09.952 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 msgr2=0x7ff6e41335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e41335a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400cbf0 tx=0x7ff6d400cf00 comp rx=0 tx=0).stop 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 shutdown_connections 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6e4138d80 0x7ff6e4134020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6e409d630 0x7ff6e41335a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 --2- 172.21.15.32:0/1671497209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40a0a30 0x7ff6e4133ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 >> 172.21.15.32:0/1671497209 conn(0x7ff6e40a3050 msgr2=0x7ff6e40a6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 shutdown_connections 2023-09-30T22:25:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:09.955+0000 7ff6e9584700 1 -- 172.21.15.32:0/1671497209 wait complete. 2023-09-30T22:25:09.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:10.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:10.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.428+0000 7fa51f299700 1 Processor -- start 2023-09-30T22:25:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.428+0000 7fa51f299700 1 -- start start 2023-09-30T22:25:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51f299700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5180a0a30 0x7fa5180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51f299700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 0x7fa51813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51f299700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51809edd0 con 0x7fa5180a1340 2023-09-30T22:25:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51f299700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51809e1f0 con 0x7fa5180a0a30 2023-09-30T22:25:10.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51f299700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa51809dc50 con 0x7fa518134170 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51d836700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 0x7fa51813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51d035700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5180a0a30 0x7fa5180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51d836700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 0x7fa51813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60316/0 (socket says 172.21.15.32:60316) 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51d035700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5180a0a30 0x7fa5180a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42498/0 (socket says 172.21.15.32:42498) 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.429+0000 7fa51d836700 1 -- 172.21.15.32:0/3578293069 learned_addr learned my addr 172.21.15.32:0/3578293069 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa51d836700 1 -- 172.21.15.32:0/3578293069 >> v2:172.21.15.32:3300/0 conn(0x7fa5180a0a30 msgr2=0x7fa5180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa51d836700 1 --2- 172.21.15.32:0/3578293069 >> v2:172.21.15.32:3300/0 conn(0x7fa5180a0a30 0x7fa5180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa51d836700 1 -- 172.21.15.32:0/3578293069 >> v1:172.21.15.32:6789/0 conn(0x7fa5180a1340 legacy=0x7fa518133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa51d836700 1 -- 172.21.15.32:0/3578293069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5180a1fe0 con 0x7fa518134170 2023-09-30T22:25:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa51d836700 1 --2- 172.21.15.32:0/3578293069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 0x7fa51813bd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa510009e50 tx=0x7fa51000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54634f9b3672f29c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa50ffff700 1 -- 172.21.15.32:0/3578293069 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000e070 con 0x7fa518134170 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa50ffff700 1 -- 172.21.15.32:0/3578293069 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51000b910 con 0x7fa518134170 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.430+0000 7fa50ffff700 1 -- 172.21.15.32:0/3578293069 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000bcc0 con 0x7fa518134170 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 -- 172.21.15.32:0/3578293069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 msgr2=0x7fa51813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 --2- 172.21.15.32:0/3578293069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 0x7fa51813bd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa510009e50 tx=0x7fa51000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 -- 172.21.15.32:0/3578293069 shutdown_connections 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 --2- 172.21.15.32:0/3578293069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa518134170 0x7fa51813bd30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 --2- 172.21.15.32:0/3578293069 >> v2:172.21.15.32:3300/0 conn(0x7fa5180a0a30 0x7fa5180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 -- 172.21.15.32:0/3578293069 >> 172.21.15.32:0/3578293069 conn(0x7fa5180a6550 msgr2=0x7fa5180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 -- 172.21.15.32:0/3578293069 shutdown_connections 2023-09-30T22:25:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.431+0000 7fa51f299700 1 -- 172.21.15.32:0/3578293069 wait complete. 2023-09-30T22:25:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.432+0000 7fa51f299700 1 Processor -- start 2023-09-30T22:25:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.432+0000 7fa51f299700 1 -- start start 2023-09-30T22:25:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51f299700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 0x7fa5181232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51f299700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180a1340 0x7fa518123820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51f299700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa518134170 0x7fa5181c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51f299700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa518122f60 con 0x7fa5180a1340 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51f299700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180a4f40 con 0x7fa518134170 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51f299700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa518126200 con 0x7fa5180a0a30 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51d035700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 0x7fa5181232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51d035700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 0x7fa5181232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60320/0 (socket says 172.21.15.32:60320) 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51d035700 1 -- 172.21.15.32:0/2240219077 learned_addr learned my addr 172.21.15.32:0/2240219077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51d836700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa518134170 0x7fa5181c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.433+0000 7fa51c834700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180a1340 0x7fa518123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51d035700 1 -- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa518134170 msgr2=0x7fa5181c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51d035700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa518134170 0x7fa5181c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51d035700 1 -- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180a1340 msgr2=0x7fa518123820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51d035700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180a1340 0x7fa518123820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51d035700 1 -- 172.21.15.32:0/2240219077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510009b30 con 0x7fa5180a0a30 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51d035700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 0x7fa5181232e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa51400e9b0 tx=0x7fa51400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514009710 con 0x7fa5180a0a30 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51400ee50 con 0x7fa5180a0a30 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5180a25e0 con 0x7fa5180a0a30 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400ca50 con 0x7fa5180a0a30 2023-09-30T22:25:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.434+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa518124dc0 con 0x7fa5180a0a30 2023-09-30T22:25:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.435+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa51400cbb0 con 0x7fa5180a0a30 2023-09-30T22:25:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.435+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa51403f750 con 0x7fa5180a0a30 2023-09-30T22:25:10.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.435+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa518074ea0 con 0x7fa5180a0a30 2023-09-30T22:25:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.440+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa51401be10 con 0x7fa5180a0a30 2023-09-30T22:25:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 Processor -- start 2023-09-30T22:25:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 -- start start 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3604138d80 0x7f360413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040a2a70 con 0x7f360409d630 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36040a01d0 con 0x7f3604138d80 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f360a449700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360409e2b0 con 0x7f36040a0a30 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f3603fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f36089e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3604138d80 0x7f360413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f3603fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:32970/0 (socket says 172.21.15.32:32970) 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.467+0000 7f3603fff700 1 -- 172.21.15.32:0/3029736456 learned_addr learned my addr 172.21.15.32:0/3029736456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f3602ffd700 1 -- 172.21.15.32:0/3029736456 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189983792 0 0) 0x7f36040a2a70 con 0x7f360409d630 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f36089e6700 1 -- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 msgr2=0x7f36040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f36089e6700 1 --2- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f36089e6700 1 -- 172.21.15.32:0/3029736456 >> v1:172.21.15.32:6789/0 conn(0x7f360409d630 legacy=0x7f36040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f36089e6700 1 -- 172.21.15.32:0/3029736456 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360409dc50 con 0x7f3604138d80 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f3603fff700 1 --2- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f36089e6700 1 --2- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3604138d80 0x7f360413b160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009d90 tx=0x7f35fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a825338fa42e861 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f3602ffd700 1 -- 172.21.15.32:0/3029736456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc00e070 con 0x7f3604138d80 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f3602ffd700 1 -- 172.21.15.32:0/3029736456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc00bb20 con 0x7f3604138d80 2023-09-30T22:25:10.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.468+0000 7f3602ffd700 1 -- 172.21.15.32:0/3029736456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc012430 con 0x7f3604138d80 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 -- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3604138d80 msgr2=0x7f360413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 --2- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3604138d80 0x7f360413b160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009d90 tx=0x7f35fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 -- 172.21.15.32:0/3029736456 shutdown_connections 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 --2- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3604138d80 0x7f360413b160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 --2- 172.21.15.32:0/3029736456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f36040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 -- 172.21.15.32:0/3029736456 >> 172.21.15.32:0/3029736456 conn(0x7f36040a3050 msgr2=0x7f36040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 -- 172.21.15.32:0/3029736456 shutdown_connections 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.469+0000 7f360a449700 1 -- 172.21.15.32:0/3029736456 wait complete. 2023-09-30T22:25:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.470+0000 7f360a449700 1 Processor -- start 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.470+0000 7f360a449700 1 -- start start 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f360a449700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 0x7f3604133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f360a449700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f3604133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f360a449700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3604138d80 0x7f3604134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f360a449700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041361b0 con 0x7f360409d630 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f360a449700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3604136ed0 con 0x7f36040a0a30 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f360a449700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36040a4b60 con 0x7f3604138d80 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f36037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f3604133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 0x7f3604133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 0x7f3604133690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42524/0 (socket says 172.21.15.32:42524) 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 -- 172.21.15.32:0/2007154716 learned_addr learned my addr 172.21.15.32:0/2007154716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f36089e6700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3604138d80 0x7f3604134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 -- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3604138d80 msgr2=0x7f3604134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3604138d80 0x7f3604134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 -- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 msgr2=0x7f3604133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f3604133bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f3603fff700 1 -- 172.21.15.32:0/2007154716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4009550 con 0x7f360409d630 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f36089e6700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3604138d80 0x7f3604134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.471+0000 7f36037fe700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f3604133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.472+0000 7f3603fff700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 0x7f3604133690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f35f400ec10 tx=0x7f35f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.472+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400c850 con 0x7f360409d630 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.472+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400c9b0 con 0x7f360409d630 2023-09-30T22:25:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.472+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc009a70 con 0x7f360409d630 2023-09-30T22:25:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.472+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400cd40 con 0x7f360409d630 2023-09-30T22:25:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.472+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3604136560 con 0x7f360409d630 2023-09-30T22:25:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.473+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35f4013470 con 0x7f360409d630 2023-09-30T22:25:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.473+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3604137a00 con 0x7f360409d630 2023-09-30T22:25:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.473+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f35f403fc00 con 0x7f360409d630 2023-09-30T22:25:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.478+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35f4013720 con 0x7f360409d630 2023-09-30T22:25:10.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.646+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa5180a28e0 con 0x7fa5180a0a30 2023-09-30T22:25:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.647+0000 7fa50dffb700 1 -- 172.21.15.32:0/2240219077 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa514017070 con 0x7fa5180a0a30 2023-09-30T22:25:10.647 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 msgr2=0x7fa5181232e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 0x7fa5181232e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa51400e9b0 tx=0x7fa51400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 shutdown_connections 2023-09-30T22:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa518134170 0x7fa5181c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5180a0a30 0x7fa5181232e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 --2- 172.21.15.32:0/2240219077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5180a1340 0x7fa518123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 >> 172.21.15.32:0/2240219077 conn(0x7fa5180a6550 msgr2=0x7fa51809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 shutdown_connections 2023-09-30T22:25:10.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.649+0000 7fa51f299700 1 -- 172.21.15.32:0/2240219077 wait complete. 2023-09-30T22:25:10.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.685+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f36040747f0 con 0x7f360409d630 2023-09-30T22:25:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.686+0000 7f3600ff9700 1 -- 172.21.15.32:0/2007154716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35f4011b00 con 0x7f360409d630 2023-09-30T22:25:10.686 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:10.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 msgr2=0x7f3604133690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:10.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 0x7f3604133690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f35f400ec10 tx=0x7f35f400ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 shutdown_connections 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3604138d80 0x7f3604134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f360409d630 0x7f3604133690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 --2- 172.21.15.32:0/2007154716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36040a0a30 0x7f3604133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 >> 172.21.15.32:0/2007154716 conn(0x7f36040a3050 msgr2=0x7f360413af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 shutdown_connections 2023-09-30T22:25:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:10.688+0000 7f360a449700 1 -- 172.21.15.32:0/2007154716 wait complete. 2023-09-30T22:25:10.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:10.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:10.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:11.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.077+0000 7f0bf4169700 1 Processor -- start 2023-09-30T22:25:11.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.077+0000 7f0bf4169700 1 -- start start 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf4169700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bec0a0a30 0x7f0bec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf4169700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec0a1340 0x7f0bec133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf4169700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec133fa0 0x7f0bec134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf4169700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec09edd0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf4169700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec09e1f0 con 0x7f0bec0a1340 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf4169700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec09dc50 con 0x7f0bec133fa0 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bec0a0a30 0x7f0bec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1704700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec0a1340 0x7f0bec133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bec0a0a30 0x7f0bec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42532/0 (socket says 172.21.15.32:42532) 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 -- 172.21.15.32:0/357104436 learned_addr learned my addr 172.21.15.32:0/357104436 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:11.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf2706700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec133fa0 0x7f0bec134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 -- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec133fa0 msgr2=0x7f0bec134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec133fa0 0x7f0bec134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 -- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec0a1340 msgr2=0x7f0bec133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec0a1340 0x7f0bec133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1f05700 1 -- 172.21.15.32:0/357104436 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0a1fe0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.078+0000 7f0bf1704700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec0a1340 0x7f0bec133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.079+0000 7f0bf1f05700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a0a30 0x7f0bec0a0e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0be80098f0 tx=0x7f0be8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba275f4ff53cc00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.079+0000 7f0bf0f03700 1 -- 172.21.15.32:0/357104436 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8011830 con 0x7f0bec0a0a30 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.079+0000 7f0bf0f03700 1 -- 172.21.15.32:0/357104436 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8011990 con 0x7f0bec0a0a30 2023-09-30T22:25:11.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.079+0000 7f0bf0f03700 1 -- 172.21.15.32:0/357104436 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8011d20 con 0x7f0bec0a0a30 2023-09-30T22:25:11.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 -- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a0a30 msgr2=0x7f0bec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a0a30 0x7f0bec0a0e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0be80098f0 tx=0x7f0be8009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:11.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 -- 172.21.15.32:0/357104436 shutdown_connections 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec133fa0 0x7f0bec134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a0a30 0x7f0bec0a0e00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 --2- 172.21.15.32:0/357104436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec0a1340 0x7f0bec133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 -- 172.21.15.32:0/357104436 >> 172.21.15.32:0/357104436 conn(0x7f0bec0a6550 msgr2=0x7f0bec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 -- 172.21.15.32:0/357104436 shutdown_connections 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.080+0000 7f0bf4169700 1 -- 172.21.15.32:0/357104436 wait complete. 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 Processor -- start 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 -- start start 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 0x7f0bec121480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a1340 0x7f0bec1219c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec133fa0 0x7f0bec121f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec123090 con 0x7f0bec0a1340 2023-09-30T22:25:11.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0a0290 con 0x7f0bec133fa0 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf4169700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec1228c0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf1f05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 0x7f0bec121480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf1f05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 0x7f0bec121480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60352/0 (socket says 172.21.15.32:60352) 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf1f05700 1 -- 172.21.15.32:0/3956912814 learned_addr learned my addr 172.21.15.32:0/3956912814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf2706700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec133fa0 0x7f0bec121f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.081+0000 7f0bf1704700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a1340 0x7f0bec1219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1f05700 1 -- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec133fa0 msgr2=0x7f0bec121f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1f05700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec133fa0 0x7f0bec121f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1f05700 1 -- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a1340 msgr2=0x7f0bec1219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1f05700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a1340 0x7f0bec1219c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1f05700 1 -- 172.21.15.32:0/3956912814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc0094a0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1704700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a1340 0x7f0bec1219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf1f05700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 0x7f0bec121480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009c40 tx=0x7f0be8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8004450 con 0x7f0bec0a0a30 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be801e070 con 0x7f0bec0a0a30 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be8009550 con 0x7f0bec0a0a30 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be80038e0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.082+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec0a3800 con 0x7f0bec0a0a30 2023-09-30T22:25:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.083+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0be8003a40 con 0x7f0bec0a0a30 2023-09-30T22:25:11.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.083+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0be8040530 con 0x7f0bec0a0a30 2023-09-30T22:25:11.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.083+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec074ea0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.092+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0be8003cf0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.124+0000 7f50acbb7700 1 Processor -- start 2023-09-30T22:25:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.124+0000 7f50acbb7700 1 -- start start 2023-09-30T22:25:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50acbb7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50a809d6f0 0x7f50a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50acbb7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809e410 0x7f50a8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50acbb7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a809f130 con 0x7f50a809e000 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50acbb7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a809f520 con 0x7f50a809d6f0 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50acbb7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50a809e930 con 0x7f50a809e410 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50a809d6f0 0x7f50a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809e410 0x7f50a8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a5d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f50a809e000 0x7f50a8130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38940/0 (socket says 172.21.15.32:38940) 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a5d9b700 1 -- 172.21.15.32:0/1727005633 learned_addr learned my addr 172.21.15.32:0/1727005633 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a559a700 1 -- 172.21.15.32:0/1727005633 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504839945 0 0) 0x7f50a809f130 con 0x7f50a809e000 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a6d9d700 1 -- 172.21.15.32:0/1727005633 >> v2:172.21.15.32:3300/0 conn(0x7f50a809d6f0 msgr2=0x7f50a809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:11.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a6d9d700 1 --2- 172.21.15.32:0/1727005633 >> v2:172.21.15.32:3300/0 conn(0x7f50a809d6f0 0x7f50a809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a6d9d700 1 -- 172.21.15.32:0/1727005633 >> v1:172.21.15.32:6789/0 conn(0x7f50a809e000 legacy=0x7f50a8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a6d9d700 1 -- 172.21.15.32:0/1727005633 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a809b2d0 con 0x7f50a809e410 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.125+0000 7f50a659c700 1 --2- 172.21.15.32:0/1727005633 >> v2:172.21.15.32:3300/0 conn(0x7f50a809d6f0 0x7f50a809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50a6d9d700 1 --2- 172.21.15.32:0/1727005633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809e410 0x7f50a8130d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009d90 tx=0x7f50a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5507cc7591e29848 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50a559a700 1 -- 172.21.15.32:0/1727005633 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a000e070 con 0x7f50a809e410 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50a559a700 1 -- 172.21.15.32:0/1727005633 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a000bb20 con 0x7f50a809e410 2023-09-30T22:25:11.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50a559a700 1 -- 172.21.15.32:0/1727005633 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0012430 con 0x7f50a809e410 2023-09-30T22:25:11.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 -- 172.21.15.32:0/1727005633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809e410 msgr2=0x7f50a8130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1727005633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809e410 0x7f50a8130d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009d90 tx=0x7f50a000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 -- 172.21.15.32:0/1727005633 shutdown_connections 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1727005633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809e410 0x7f50a8130d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1727005633 >> v2:172.21.15.32:3300/0 conn(0x7f50a809d6f0 0x7f50a809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 -- 172.21.15.32:0/1727005633 >> 172.21.15.32:0/1727005633 conn(0x7f50a8066f50 msgr2=0x7f50a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 -- 172.21.15.32:0/1727005633 shutdown_connections 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.126+0000 7f50acbb7700 1 -- 172.21.15.32:0/1727005633 wait complete. 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.128+0000 7f50acbb7700 1 Processor -- start 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.128+0000 7f50acbb7700 1 -- start start 2023-09-30T22:25:11.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50acbb7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 0x7f50a811ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50acbb7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a809e000 0x7f50a811f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50acbb7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809e410 0x7f50a811f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50acbb7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8120aa0 con 0x7f50a809e410 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50acbb7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8122770 con 0x7f50a809e000 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50acbb7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8121930 con 0x7f50a809d6f0 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 0x7f50a811ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 0x7f50a811ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60374/0 (socket says 172.21.15.32:60374) 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 -- 172.21.15.32:0/1001511333 learned_addr learned my addr 172.21.15.32:0/1001511333 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a6d9d700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809e410 0x7f50a811f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 -- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a809e000 msgr2=0x7f50a811f470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a809e000 0x7f50a811f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 -- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809e410 msgr2=0x7f50a811f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809e410 0x7f50a811f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 -- 172.21.15.32:0/1001511333 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980094a0 con 0x7f50a809d6f0 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a6d9d700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809e410 0x7f50a811f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.129+0000 7f50a659c700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 0x7f50a811ef30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509800ebe0 tx=0x7f5098009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.130+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800c8b0 con 0x7f50a809d6f0 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.130+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800ca10 con 0x7f50a809d6f0 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.130+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0009a70 con 0x7f50a809d6f0 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.130+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800cd80 con 0x7f50a809d6f0 2023-09-30T22:25:11.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.130+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a806eba0 con 0x7f50a809d6f0 2023-09-30T22:25:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.130+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5098013470 con 0x7f50a809d6f0 2023-09-30T22:25:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.131+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f509803fcf0 con 0x7f50a809d6f0 2023-09-30T22:25:11.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.131+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c0052e0 con 0x7f50a809d6f0 2023-09-30T22:25:11.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.136+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5098017020 con 0x7f50a809d6f0 2023-09-30T22:25:11.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.304+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0bec1323d0 con 0x7f0bec0a0a30 2023-09-30T22:25:11.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.304+0000 7f0be27fc700 1 -- 172.21.15.32:0/3956912814 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0be8035020 con 0x7f0bec0a0a30 2023-09-30T22:25:11.304 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 msgr2=0x7f0bec121480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 0x7f0bec121480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009c40 tx=0x7f0be8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 shutdown_connections 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bec133fa0 0x7f0bec121f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bec0a0a30 0x7f0bec121480 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 --2- 172.21.15.32:0/3956912814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bec0a1340 0x7f0bec1219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.306+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 >> 172.21.15.32:0/3956912814 conn(0x7f0bec0a6550 msgr2=0x7f0bec133630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.307+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 shutdown_connections 2023-09-30T22:25:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.307+0000 7f0bf4169700 1 -- 172.21.15.32:0/3956912814 wait complete. 2023-09-30T22:25:11.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.342+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f508c005590 con 0x7f50a809d6f0 2023-09-30T22:25:11.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.343+0000 7f5096ffd700 1 -- 172.21.15.32:0/1001511333 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f509801b3b0 con 0x7f50a809d6f0 2023-09-30T22:25:11.343 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 msgr2=0x7f50a811ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 0x7f50a811ef30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509800ebe0 tx=0x7f5098009ec0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 shutdown_connections 2023-09-30T22:25:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809e410 0x7f50a811f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a809d6f0 0x7f50a811ef30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 --2- 172.21.15.32:0/1001511333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a809e000 0x7f50a811f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 >> 172.21.15.32:0/1001511333 conn(0x7f50a8066f50 msgr2=0x7f50a8130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 shutdown_connections 2023-09-30T22:25:11.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.345+0000 7f50acbb7700 1 -- 172.21.15.32:0/1001511333 wait complete. 2023-09-30T22:25:11.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:11.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:11.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.812+0000 7fda8d90f700 1 Processor -- start 2023-09-30T22:25:11.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.812+0000 7fda8d90f700 1 -- start start 2023-09-30T22:25:11.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.812+0000 7fda8d90f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 0x7fda880a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.812+0000 7fda8d90f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda88138d80 0x7fda8813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda8d90f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda880a2a70 con 0x7fda880a0a30 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda8d90f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda880a01d0 con 0x7fda88138d80 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda8d90f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8809e2b0 con 0x7fda8809d630 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 0x7fda880a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda877fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda88138d80 0x7fda8813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda86ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fda880a0a30 0x7fda880a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38950/0 (socket says 172.21.15.32:38950) 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 0x7fda880a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33008/0 (socket says 172.21.15.32:33008) 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda86ffd700 1 -- 172.21.15.32:0/1279036508 learned_addr learned my addr 172.21.15.32:0/1279036508 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 -- 172.21.15.32:0/1279036508 >> v2:172.21.15.32:3300/0 conn(0x7fda88138d80 msgr2=0x7fda8813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 --2- 172.21.15.32:0/1279036508 >> v2:172.21.15.32:3300/0 conn(0x7fda88138d80 0x7fda8813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 -- 172.21.15.32:0/1279036508 >> v1:172.21.15.32:6789/0 conn(0x7fda880a0a30 legacy=0x7fda880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 -- 172.21.15.32:0/1279036508 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8809dc50 con 0x7fda8809d630 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda85ffb700 1 -- 172.21.15.32:0/1279036508 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960852173 0 0) 0x7fda880a2a70 con 0x7fda880a0a30 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda877fe700 1 --2- 172.21.15.32:0/1279036508 >> v2:172.21.15.32:3300/0 conn(0x7fda88138d80 0x7fda8813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.813+0000 7fda867fc700 1 --2- 172.21.15.32:0/1279036508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 0x7fda880a1510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009840 tx=0x7fda7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8844b7b80de29a82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda85ffb700 1 -- 172.21.15.32:0/1279036508 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011830 con 0x7fda8809d630 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda85ffb700 1 -- 172.21.15.32:0/1279036508 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c011990 con 0x7fda8809d630 2023-09-30T22:25:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda85ffb700 1 -- 172.21.15.32:0/1279036508 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c011d20 con 0x7fda8809d630 2023-09-30T22:25:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 -- 172.21.15.32:0/1279036508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 msgr2=0x7fda880a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 --2- 172.21.15.32:0/1279036508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 0x7fda880a1510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009840 tx=0x7fda7c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 -- 172.21.15.32:0/1279036508 shutdown_connections 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 --2- 172.21.15.32:0/1279036508 >> v2:172.21.15.32:3300/0 conn(0x7fda88138d80 0x7fda8813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 --2- 172.21.15.32:0/1279036508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda8809d630 0x7fda880a1510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 -- 172.21.15.32:0/1279036508 >> 172.21.15.32:0/1279036508 conn(0x7fda880a3050 msgr2=0x7fda880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 -- 172.21.15.32:0/1279036508 shutdown_connections 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.814+0000 7fda8d90f700 1 -- 172.21.15.32:0/1279036508 wait complete. 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.816+0000 7fda8d90f700 1 Processor -- start 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.816+0000 7fda8d90f700 1 -- start start 2023-09-30T22:25:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda8d90f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda8809d630 0x7fda881337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda8d90f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 0x7fda88133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda8d90f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda88138d80 0x7fda88134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda8d90f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda881365f0 con 0x7fda88138d80 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda8d90f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda88136f60 con 0x7fda880a0a30 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda8d90f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda88136240 con 0x7fda8809d630 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 0x7fda88133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 0x7fda88133cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33022/0 (socket says 172.21.15.32:33022) 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda867fc700 1 -- 172.21.15.32:0/2408537650 learned_addr learned my addr 172.21.15.32:0/2408537650 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda88138d80 0x7fda88134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 -- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda8809d630 msgr2=0x7fda881337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda8809d630 0x7fda881337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 -- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 msgr2=0x7fda88133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 0x7fda88133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 -- 172.21.15.32:0/2408537650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda800094a0 con 0x7fda88138d80 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda867fc700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 0x7fda88133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.817+0000 7fda877fe700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda88138d80 0x7fda88134230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e910 tx=0x7fda8000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.818+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ee60 con 0x7fda88138d80 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.818+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80004020 con 0x7fda88138d80 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.818+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800043d0 con 0x7fda88138d80 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.818+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c0094a0 con 0x7fda88138d80 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.818+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda880a0410 con 0x7fda88138d80 2023-09-30T22:25:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.818+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda8000c7f0 con 0x7fda88138d80 2023-09-30T22:25:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.819+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda8003f540 con 0x7fda88138d80 2023-09-30T22:25:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.819+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda880747f0 con 0x7fda88138d80 2023-09-30T22:25:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.824+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda80017080 con 0x7fda88138d80 2023-09-30T22:25:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.850+0000 7f241a3ff700 1 Processor -- start 2023-09-30T22:25:11.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.850+0000 7f241a3ff700 1 -- start start 2023-09-30T22:25:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241a3ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f241409d630 0x7f24140a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241a3ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f241413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241a3ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140a2a70 con 0x7f24140a0a30 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241a3ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24140a01d0 con 0x7f241409d630 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241a3ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f241409e2b0 con 0x7f2414138d80 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241899c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f241413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f2413fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f24140a0a30 0x7f24140a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38964/0 (socket says 172.21.15.32:38964) 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f2413fff700 1 -- 172.21.15.32:0/1193477427 learned_addr learned my addr 172.21.15.32:0/1193477427 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f24137fe700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f241409d630 0x7f24140a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f2412ffd700 1 -- 172.21.15.32:0/1193477427 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858079119 0 0) 0x7f24140a2a70 con 0x7f24140a0a30 2023-09-30T22:25:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f2412ffd700 1 -- 172.21.15.32:0/1193477427 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23fc0035b0 con 0x7f24140a0a30 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241899c700 1 -- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f241409d630 msgr2=0x7f24140a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241899c700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f241409d630 0x7f24140a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241899c700 1 -- 172.21.15.32:0/1193477427 >> v1:172.21.15.32:6789/0 conn(0x7f24140a0a30 legacy=0x7f24140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f241899c700 1 -- 172.21.15.32:0/1193477427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241409dc50 con 0x7f2414138d80 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.851+0000 7f24137fe700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f241409d630 0x7f24140a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241899c700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f241413b160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f240c009dc0 tx=0x7f240c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50f44110641166a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f2412ffd700 1 -- 172.21.15.32:0/1193477427 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c00e070 con 0x7f2414138d80 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f2412ffd700 1 -- 172.21.15.32:0/1193477427 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c00bb20 con 0x7f2414138d80 2023-09-30T22:25:11.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f2412ffd700 1 -- 172.21.15.32:0/1193477427 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c012430 con 0x7f2414138d80 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 -- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 msgr2=0x7f241413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f241413b160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f240c009dc0 tx=0x7f240c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 -- 172.21.15.32:0/1193477427 shutdown_connections 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f241413b160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 --2- 172.21.15.32:0/1193477427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f241409d630 0x7f24140a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 -- 172.21.15.32:0/1193477427 >> 172.21.15.32:0/1193477427 conn(0x7f24140a3050 msgr2=0x7f24140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.852+0000 7f241a3ff700 1 -- 172.21.15.32:0/1193477427 shutdown_connections 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.853+0000 7f241a3ff700 1 -- 172.21.15.32:0/1193477427 wait complete. 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.853+0000 7f241a3ff700 1 Processor -- start 2023-09-30T22:25:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 -- start start 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 0x7f24141337f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24140a0a30 0x7f2414133d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f2414134270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414134cc0 con 0x7f241409d630 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24141361e0 con 0x7f24140a0a30 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241a3ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2414136590 con 0x7f2414138d80 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f2413fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 0x7f24141337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f241899c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f2414134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f2413fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 0x7f24141337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42578/0 (socket says 172.21.15.32:42578) 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f2413fff700 1 -- 172.21.15.32:0/964597492 learned_addr learned my addr 172.21.15.32:0/964597492 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.854+0000 7f24137fe700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24140a0a30 0x7f2414133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2413fff700 1 -- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 msgr2=0x7f2414134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2413fff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f2414134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2413fff700 1 -- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24140a0a30 msgr2=0x7f2414133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2413fff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24140a0a30 0x7f2414133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2413fff700 1 -- 172.21.15.32:0/964597492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080094a0 con 0x7f241409d630 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f24137fe700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24140a0a30 0x7f2414133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f241899c700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f2414134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2413fff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 0x7f24141337f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f240400c700 tx=0x7f240400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404005d70 con 0x7f241409d630 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040070d0 con 0x7f241409d630 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c009a70 con 0x7f241409d630 2023-09-30T22:25:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404007460 con 0x7f241409d630 2023-09-30T22:25:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.855+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2414136f50 con 0x7f241409d630 2023-09-30T22:25:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.856+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24140747f0 con 0x7f241409d630 2023-09-30T22:25:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.856+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24040075c0 con 0x7f241409d630 2023-09-30T22:25:11.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.857+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f240403e9c0 con 0x7f241409d630 2023-09-30T22:25:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:11.861+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2404019b40 con 0x7f241409d630 2023-09-30T22:25:12.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.032+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fda88137a30 con 0x7fda88138d80 2023-09-30T22:25:12.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.033+0000 7fda6ffff700 1 -- 172.21.15.32:0/2408537650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda80019d50 con 0x7fda88138d80 2023-09-30T22:25:12.033 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda88138d80 msgr2=0x7fda88134230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda88138d80 0x7fda88134230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e910 tx=0x7fda8000ec20 comp rx=0 tx=0).stop 2023-09-30T22:25:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 shutdown_connections 2023-09-30T22:25:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda88138d80 0x7fda88134230 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda8809d630 0x7fda881337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 --2- 172.21.15.32:0/2408537650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda880a0a30 0x7fda88133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 >> 172.21.15.32:0/2408537650 conn(0x7fda880a3050 msgr2=0x7fda880a6940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 shutdown_connections 2023-09-30T22:25:12.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.035+0000 7fda8d90f700 1 -- 172.21.15.32:0/2408537650 wait complete. 2023-09-30T22:25:12.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:12.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.069+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2414136f50 con 0x7f241409d630 2023-09-30T22:25:12.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.069+0000 7f2410ff9700 1 -- 172.21.15.32:0/964597492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2414136f50 con 0x7f241409d630 2023-09-30T22:25:12.069 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:12.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.071+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 msgr2=0x7f24141337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.071+0000 7f241a3ff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 0x7f24141337f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f240400c700 tx=0x7f240400ca10 comp rx=0 tx=0).stop 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.071+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 shutdown_connections 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.072+0000 7f241a3ff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2414138d80 0x7f2414134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.072+0000 7f241a3ff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f241409d630 0x7f24141337f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.072+0000 7f241a3ff700 1 --2- 172.21.15.32:0/964597492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24140a0a30 0x7f2414133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.072+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 >> 172.21.15.32:0/964597492 conn(0x7f24140a3050 msgr2=0x7f2414139700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.072+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 shutdown_connections 2023-09-30T22:25:12.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.072+0000 7f241a3ff700 1 -- 172.21.15.32:0/964597492 wait complete. 2023-09-30T22:25:12.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:12.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:12.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:12.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 Processor -- start 2023-09-30T22:25:12.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 -- start start 2023-09-30T22:25:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f76280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7628134170 0x7f762813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762809edd0 con 0x7f76280a1340 2023-09-30T22:25:12.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762809e1f0 con 0x7f7628134170 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.470+0000 7f762c8f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762809dc50 con 0x7f76280a0a30 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f7626d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7628134170 0x7f762813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f7625d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76280a1340 0x7f7628133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38972/0 (socket says 172.21.15.32:38972) 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f76280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f7625d9b700 1 -- 172.21.15.32:0/1297190645 learned_addr learned my addr 172.21.15.32:0/1297190645 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762559a700 1 -- 172.21.15.32:0/1297190645 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405226900 0 0) 0x7f762809edd0 con 0x7f76280a1340 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762659c700 1 -- 172.21.15.32:0/1297190645 >> v2:172.21.15.32:3300/0 conn(0x7f7628134170 msgr2=0x7f762813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762659c700 1 --2- 172.21.15.32:0/1297190645 >> v2:172.21.15.32:3300/0 conn(0x7f7628134170 0x7f762813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762659c700 1 -- 172.21.15.32:0/1297190645 >> v1:172.21.15.32:6789/0 conn(0x7f76280a1340 legacy=0x7f7628133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762659c700 1 -- 172.21.15.32:0/1297190645 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280a1fe0 con 0x7f76280a0a30 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f7626d9d700 1 --2- 172.21.15.32:0/1297190645 >> v2:172.21.15.32:3300/0 conn(0x7f7628134170 0x7f762813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:12.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762659c700 1 --2- 172.21.15.32:0/1297190645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f76280a0e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7618009840 tx=0x7f7618009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723f016d43f32f96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762559a700 1 -- 172.21.15.32:0/1297190645 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618011830 con 0x7f76280a0a30 2023-09-30T22:25:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762559a700 1 -- 172.21.15.32:0/1297190645 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618011990 con 0x7f76280a0a30 2023-09-30T22:25:12.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.471+0000 7f762559a700 1 -- 172.21.15.32:0/1297190645 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618011d20 con 0x7f76280a0a30 2023-09-30T22:25:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 -- 172.21.15.32:0/1297190645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 msgr2=0x7f76280a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1297190645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f76280a0e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7618009840 tx=0x7f7618009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 -- 172.21.15.32:0/1297190645 shutdown_connections 2023-09-30T22:25:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1297190645 >> v2:172.21.15.32:3300/0 conn(0x7f7628134170 0x7f762813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1297190645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f76280a0e00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 -- 172.21.15.32:0/1297190645 >> 172.21.15.32:0/1297190645 conn(0x7f76280a6550 msgr2=0x7f76280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 -- 172.21.15.32:0/1297190645 shutdown_connections 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.472+0000 7f762c8f0700 1 -- 172.21.15.32:0/1297190645 wait complete. 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.473+0000 7f762c8f0700 1 Processor -- start 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 -- start start 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f7628122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76280a1340 0x7f7628122bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7628134170 0x7f7628123100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628125b70 con 0x7f7628134170 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76281335b0 con 0x7f76280a0a30 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762c8f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76281330f0 con 0x7f76280a1340 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f7628122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f7628122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33062/0 (socket says 172.21.15.32:33062) 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f762659c700 1 -- 172.21.15.32:0/1615760962 learned_addr learned my addr 172.21.15.32:0/1615760962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f7626d9d700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7628134170 0x7f7628123100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.474+0000 7f7625d9b700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76280a1340 0x7f7628122bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762659c700 1 -- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76280a1340 msgr2=0x7f7628122bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762659c700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76280a1340 0x7f7628122bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762659c700 1 -- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7628134170 msgr2=0x7f7628123100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762659c700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7628134170 0x7f7628123100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762659c700 1 -- 172.21.15.32:0/1615760962 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009a70 con 0x7f76280a0a30 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f7626d9d700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7628134170 0x7f7628123100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:12.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762659c700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f7628122680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7618009fd0 tx=0x7f7618004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180045d0 con 0x7f76280a0a30 2023-09-30T22:25:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761801f070 con 0x7f76280a0a30 2023-09-30T22:25:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76180094a0 con 0x7f76280a0a30 2023-09-30T22:25:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180038e0 con 0x7f76280a0a30 2023-09-30T22:25:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.475+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76280a0270 con 0x7f76280a0a30 2023-09-30T22:25:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.476+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7618003a40 con 0x7f76280a0a30 2023-09-30T22:25:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.476+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7618021070 con 0x7f76280a0a30 2023-09-30T22:25:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.476+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7628125980 con 0x7f76280a0a30 2023-09-30T22:25:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.481+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7618015080 con 0x7f76280a0a30 2023-09-30T22:25:12.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 Processor -- start 2023-09-30T22:25:12.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 -- start start 2023-09-30T22:25:12.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe45809e850 0x7fe45809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 0x7fe458133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45806bec0 con 0x7fe45809f5b0 2023-09-30T22:25:12.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580a0030 con 0x7fe45809e850 2023-09-30T22:25:12.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe45d92c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580a0830 con 0x7fe45809f160 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe4567fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 0x7fe458133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe456ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe45809e850 0x7fe45809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe4567fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 0x7fe458133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60414/0 (socket says 172.21.15.32:60414) 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.508+0000 7fe4567fc700 1 -- 172.21.15.32:0/1854763125 learned_addr learned my addr 172.21.15.32:0/1854763125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe456ffd700 1 -- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 msgr2=0x7fe458133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe456ffd700 1 --2- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 0x7fe458133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe456ffd700 1 -- 172.21.15.32:0/1854763125 >> v1:172.21.15.32:6789/0 conn(0x7fe45809f5b0 legacy=0x7fe4581341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe456ffd700 1 -- 172.21.15.32:0/1854763125 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45806caa0 con 0x7fe45809e850 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe455ffb700 1 -- 172.21.15.32:0/1854763125 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765956010 0 0) 0x7fe45806bec0 con 0x7fe45809f5b0 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe4567fc700 1 --2- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 0x7fe458133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe456ffd700 1 --2- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809e850 0x7fe45809ec20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe448009d90 tx=0x7fe44800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513e88b97c0bcc0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:12.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe455ffb700 1 -- 172.21.15.32:0/1854763125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800e070 con 0x7fe45809e850 2023-09-30T22:25:12.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe455ffb700 1 -- 172.21.15.32:0/1854763125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44800bb20 con 0x7fe45809e850 2023-09-30T22:25:12.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.509+0000 7fe455ffb700 1 -- 172.21.15.32:0/1854763125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448012430 con 0x7fe45809e850 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 -- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809e850 msgr2=0x7fe45809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 --2- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809e850 0x7fe45809ec20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe448009d90 tx=0x7fe44800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 -- 172.21.15.32:0/1854763125 shutdown_connections 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 --2- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809e850 0x7fe45809ec20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 --2- 172.21.15.32:0/1854763125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f160 0x7fe458133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 -- 172.21.15.32:0/1854763125 >> 172.21.15.32:0/1854763125 conn(0x7fe4580a3f70 msgr2=0x7fe4580a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:12.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 -- 172.21.15.32:0/1854763125 shutdown_connections 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.510+0000 7fe45d92c700 1 -- 172.21.15.32:0/1854763125 wait complete. 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.511+0000 7fe45d92c700 1 Processor -- start 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.511+0000 7fe45d92c700 1 -- start start 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe45d92c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe45809e850 0x7fe4581205b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe45d92c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809f160 0x7fe458120af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe45d92c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 0x7fe458121030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe45d92c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458122e50 con 0x7fe45809f160 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe45d92c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580a31a0 con 0x7fe45809e850 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe45d92c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe45806ccc0 con 0x7fe45809f5b0 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 0x7fe458121030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe456ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe45809e850 0x7fe4581205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 0x7fe458121030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60416/0 (socket says 172.21.15.32:60416) 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe456ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe45809e850 0x7fe4581205b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33078/0 (socket says 172.21.15.32:33078) 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 -- 172.21.15.32:0/2469350740 learned_addr learned my addr 172.21.15.32:0/2469350740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4567fc700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809f160 0x7fe458120af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 -- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe45809e850 msgr2=0x7fe4581205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe45809e850 0x7fe4581205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 -- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809f160 msgr2=0x7fe458120af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809f160 0x7fe458120af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 -- 172.21.15.32:0/2469350740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c009550 con 0x7fe45809f5b0 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe4577fe700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 0x7fe458121030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c500 tx=0x7fe45000c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500054e0 con 0x7fe45809f5b0 2023-09-30T22:25:12.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.512+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450003680 con 0x7fe45809f5b0 2023-09-30T22:25:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.513+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe448009a70 con 0x7fe45809f5b0 2023-09-30T22:25:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.513+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450003a30 con 0x7fe45809f5b0 2023-09-30T22:25:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.513+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4581220d0 con 0x7fe45809f5b0 2023-09-30T22:25:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.513+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe450003b90 con 0x7fe45809f5b0 2023-09-30T22:25:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.513+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe45003eb00 con 0x7fe45809f5b0 2023-09-30T22:25:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.514+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45809caf0 con 0x7fe45809f5b0 2023-09-30T22:25:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.519+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe450017080 con 0x7fe45809f5b0 2023-09-30T22:25:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.686+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7628074ea0 con 0x7f76280a0a30 2023-09-30T22:25:12.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.686+0000 7f7616ffd700 1 -- 172.21.15.32:0/1615760962 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f761802b020 con 0x7f76280a0a30 2023-09-30T22:25:12.686 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:12.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 msgr2=0x7f7628122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f7628122680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7618009fd0 tx=0x7f7618004750 comp rx=0 tx=0).stop 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 shutdown_connections 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7628134170 0x7f7628123100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76280a0a30 0x7f7628122680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 --2- 172.21.15.32:0/1615760962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76280a1340 0x7f7628122bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.688+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 >> 172.21.15.32:0/1615760962 conn(0x7f76280a6550 msgr2=0x7f762809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.689+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 shutdown_connections 2023-09-30T22:25:12.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.689+0000 7f762c8f0700 1 -- 172.21.15.32:0/1615760962 wait complete. 2023-09-30T22:25:12.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.724+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe4581228a0 con 0x7fe45809f5b0 2023-09-30T22:25:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.724+0000 7fe43ffff700 1 -- 172.21.15.32:0/2469350740 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe450003e40 con 0x7fe45809f5b0 2023-09-30T22:25:12.724 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 msgr2=0x7fe458121030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 0x7fe458121030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe45000c500 tx=0x7fe45000c810 comp rx=0 tx=0).stop 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 shutdown_connections 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe45809f5b0 0x7fe458121030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe45809e850 0x7fe4581205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 --2- 172.21.15.32:0/2469350740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45809f160 0x7fe458120af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.726+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 >> 172.21.15.32:0/2469350740 conn(0x7fe4580a3f70 msgr2=0x7fe4581bade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.727+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 shutdown_connections 2023-09-30T22:25:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:12.727+0000 7fe45d92c700 1 -- 172.21.15.32:0/2469350740 wait complete. 2023-09-30T22:25:12.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:13.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:13.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:13.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 Processor -- start 2023-09-30T22:25:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 -- start start 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b9c0a1340 0x7f3b9c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b9c09edd0 con 0x7f3b9c133fa0 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9c09e1f0 con 0x7f3b9c0a1340 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3ba2237700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c09dc50 con 0x7f3b9c0a0a30 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3b9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3b9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33092/0 (socket says 172.21.15.32:33092) 2023-09-30T22:25:13.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.202+0000 7f3b9b7fe700 1 -- 172.21.15.32:0/3348441026 learned_addr learned my addr 172.21.15.32:0/3348441026 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b92ffd700 1 --2- 172.21.15.32:0/3348441026 >> v2:172.21.15.32:3300/0 conn(0x7f3b9c0a1340 0x7f3b9c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9b7fe700 1 -- 172.21.15.32:0/3348441026 >> v2:172.21.15.32:3300/0 conn(0x7f3b9c0a1340 msgr2=0x7f3b9c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9b7fe700 1 --2- 172.21.15.32:0/3348441026 >> v2:172.21.15.32:3300/0 conn(0x7f3b9c0a1340 0x7f3b9c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9b7fe700 1 -- 172.21.15.32:0/3348441026 >> v1:172.21.15.32:6789/0 conn(0x7f3b9c133fa0 legacy=0x7f3b9c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9affd700 1 -- 172.21.15.32:0/3348441026 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323586757 0 0) 0x7f3b9c09edd0 con 0x7f3b9c133fa0 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9b7fe700 1 -- 172.21.15.32:0/3348441026 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c0a1fe0 con 0x7f3b9c0a0a30 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9b7fe700 1 --2- 172.21.15.32:0/3348441026 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c0a0e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0098f0 tx=0x7f3b8c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a811e96abfa1b1df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9affd700 1 -- 172.21.15.32:0/3348441026 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c011830 con 0x7f3b9c0a0a30 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9affd700 1 -- 172.21.15.32:0/3348441026 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c011990 con 0x7f3b9c0a0a30 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.203+0000 7f3b9affd700 1 -- 172.21.15.32:0/3348441026 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c011d20 con 0x7f3b9c0a0a30 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 -- 172.21.15.32:0/3348441026 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 msgr2=0x7f3b9c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 --2- 172.21.15.32:0/3348441026 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c0a0e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0098f0 tx=0x7f3b8c009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 -- 172.21.15.32:0/3348441026 shutdown_connections 2023-09-30T22:25:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 --2- 172.21.15.32:0/3348441026 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c0a0e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 --2- 172.21.15.32:0/3348441026 >> v2:172.21.15.32:3300/0 conn(0x7f3b9c0a1340 0x7f3b9c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 -- 172.21.15.32:0/3348441026 >> 172.21.15.32:0/3348441026 conn(0x7f3b9c0a6550 msgr2=0x7f3b9c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 -- 172.21.15.32:0/3348441026 shutdown_connections 2023-09-30T22:25:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.204+0000 7f3ba2237700 1 -- 172.21.15.32:0/3348441026 wait complete. 2023-09-30T22:25:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.205+0000 7f3ba2237700 1 Processor -- start 2023-09-30T22:25:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.205+0000 7f3ba2237700 1 -- start start 2023-09-30T22:25:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3ba2237700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3ba2237700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b9c0a1340 0x7f3b9c1238a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3ba2237700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b9c133fa0 0x7f3b9c1c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3ba2237700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c126340 con 0x7f3b9c133fa0 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3ba2237700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c124ee0 con 0x7f3b9c0a0a30 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3ba2237700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c126520 con 0x7f3b9c0a1340 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b9b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c123360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33108/0 (socket says 172.21.15.32:33108) 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b9b7fe700 1 -- 172.21.15.32:0/503657430 learned_addr learned my addr 172.21.15.32:0/503657430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b9c0a1340 0x7f3b9c1238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b9bfff700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b9c133fa0 0x7f3b9c1c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 -- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 msgr2=0x7f3b9c123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c123360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 -- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b9c133fa0 msgr2=0x7f3b9c1c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b9c133fa0 0x7f3b9c1c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 -- 172.21.15.32:0/503657430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c009550 con 0x7f3b9c0a1340 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b9b7fe700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.206+0000 7f3b92ffd700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b9c0a1340 0x7f3b9c1238a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400e930 tx=0x7f3b8400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.207+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84013680 con 0x7f3b9c0a1340 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.207+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b840137e0 con 0x7f3b9c0a1340 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.207+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84013b50 con 0x7f3b9c0a1340 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.207+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c0a3610 con 0x7f3b9c0a1340 2023-09-30T22:25:13.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.207+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c124b40 con 0x7f3b9c0a1340 2023-09-30T22:25:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.208+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c0a1d40 con 0x7f3b9c0a1340 2023-09-30T22:25:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.208+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b84013cb0 con 0x7f3b9c0a1340 2023-09-30T22:25:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.208+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3b8403f660 con 0x7f3b9c0a1340 2023-09-30T22:25:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.213+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b8401bc30 con 0x7f3b9c0a1340 2023-09-30T22:25:13.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.238+0000 7effe4584700 1 Processor -- start 2023-09-30T22:25:13.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.238+0000 7effe4584700 1 -- start start 2023-09-30T22:25:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe4584700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc0a0a30 0x7effdc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe4584700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effdc09d630 0x7effdc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe4584700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc0a2a70 con 0x7effdc133930 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe4584700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc0a01d0 con 0x7effdc09d630 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe4584700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc09e2b0 con 0x7effdc0a0a30 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe1b1f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effdc09d630 0x7effdc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe2b21700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7effdc133930 0x7effdc133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38996/0 (socket says 172.21.15.32:38996) 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe2b21700 1 -- 172.21.15.32:0/3440597482 learned_addr learned my addr 172.21.15.32:0/3440597482 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe2320700 1 --2- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc0a0a30 0x7effdc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe1b1f700 1 -- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc0a0a30 msgr2=0x7effdc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe1b1f700 1 --2- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc0a0a30 0x7effdc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe1b1f700 1 -- 172.21.15.32:0/3440597482 >> v1:172.21.15.32:6789/0 conn(0x7effdc133930 legacy=0x7effdc133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe1b1f700 1 -- 172.21.15.32:0/3440597482 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc09dc50 con 0x7effdc09d630 2023-09-30T22:25:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.239+0000 7effe1b1f700 1 --2- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc09d630 0x7effdc0a1340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7effd0009840 tx=0x7effd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6420b265acf694e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe131e700 1 -- 172.21.15.32:0/3440597482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0011830 con 0x7effdc09d630 2023-09-30T22:25:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe131e700 1 -- 172.21.15.32:0/3440597482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0011990 con 0x7effdc09d630 2023-09-30T22:25:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe131e700 1 -- 172.21.15.32:0/3440597482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0011d20 con 0x7effdc09d630 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 -- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc09d630 msgr2=0x7effdc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 --2- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc09d630 0x7effdc0a1340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7effd0009840 tx=0x7effd0009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 -- 172.21.15.32:0/3440597482 shutdown_connections 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 --2- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc0a0a30 0x7effdc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 --2- 172.21.15.32:0/3440597482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc09d630 0x7effdc0a1340 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 -- 172.21.15.32:0/3440597482 >> 172.21.15.32:0/3440597482 conn(0x7effdc0a3050 msgr2=0x7effdc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 -- 172.21.15.32:0/3440597482 shutdown_connections 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.240+0000 7effe4584700 1 -- 172.21.15.32:0/3440597482 wait complete. 2023-09-30T22:25:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.242+0000 7effe4584700 1 Processor -- start 2023-09-30T22:25:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.242+0000 7effe4584700 1 -- start start 2023-09-30T22:25:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.242+0000 7effe4584700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc09d630 0x7effdc1c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe4584700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 0x7effdc1c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe4584700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc133930 0x7effdc1c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe4584700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc120a70 con 0x7effdc0a0a30 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe4584700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc120880 con 0x7effdc133930 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe4584700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effdc121920 con 0x7effdc09d630 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2320700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc09d630 0x7effdc1c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe1b1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 0x7effdc1c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2320700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc09d630 0x7effdc1c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60436/0 (socket says 172.21.15.32:60436) 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe1b1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 0x7effdc1c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42634/0 (socket says 172.21.15.32:42634) 2023-09-30T22:25:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc133930 0x7effdc1c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2320700 1 -- 172.21.15.32:0/1279793315 learned_addr learned my addr 172.21.15.32:0/1279793315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 -- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc09d630 msgr2=0x7effdc1c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc09d630 0x7effdc1c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 -- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 msgr2=0x7effdc1c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 0x7effdc1c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 -- 172.21.15.32:0/1279793315 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd00094a0 con 0x7effdc133930 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe1b1f700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 0x7effdc1c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe2b21700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc133930 0x7effdc1c8d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effd400c430 tx=0x7effd400c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4005d70 con 0x7effdc133930 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400ce40 con 0x7effdc133930 2023-09-30T22:25:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.243+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc0a40c0 con 0x7effdc133930 2023-09-30T22:25:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.244+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40072d0 con 0x7effdc133930 2023-09-30T22:25:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.244+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc121e90 con 0x7effdc133930 2023-09-30T22:25:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.244+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7effd4007430 con 0x7effdc133930 2023-09-30T22:25:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.245+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc0008d0 con 0x7effdc133930 2023-09-30T22:25:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.245+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7effd403e8e0 con 0x7effdc133930 2023-09-30T22:25:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.250+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7effd401b3b0 con 0x7effdc133930 2023-09-30T22:25:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.423+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3b9c074ea0 con 0x7f3b9c0a1340 2023-09-30T22:25:13.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.423+0000 7f3b98ff9700 1 -- 172.21.15.32:0/503657430 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b84017070 con 0x7f3b9c0a1340 2023-09-30T22:25:13.423 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b9c0a1340 msgr2=0x7f3b9c1238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b9c0a1340 0x7f3b9c1238a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400e930 tx=0x7f3b8400ec40 comp rx=0 tx=0).stop 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 shutdown_connections 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b9c133fa0 0x7f3b9c1c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3b9c0a0a30 0x7f3b9c123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 --2- 172.21.15.32:0/503657430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3b9c0a1340 0x7f3b9c1238a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 >> 172.21.15.32:0/503657430 conn(0x7f3b9c0a6550 msgr2=0x7f3b9c11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 shutdown_connections 2023-09-30T22:25:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.426+0000 7f3ba2237700 1 -- 172.21.15.32:0/503657430 wait complete. 2023-09-30T22:25:13.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:13.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.458+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7effdc072980 con 0x7effdc133930 2023-09-30T22:25:13.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.458+0000 7effdaffd700 1 -- 172.21.15.32:0/1279793315 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7effd4017030 con 0x7effdc133930 2023-09-30T22:25:13.458 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc133930 msgr2=0x7effdc1c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc133930 0x7effdc1c8d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effd400c430 tx=0x7effd400c740 comp rx=0 tx=0).stop 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 shutdown_connections 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc133930 0x7effdc1c8d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc09d630 0x7effdc1c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 --2- 172.21.15.32:0/1279793315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc0a0a30 0x7effdc1c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 >> 172.21.15.32:0/1279793315 conn(0x7effdc0a3050 msgr2=0x7effdc1bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 shutdown_connections 2023-09-30T22:25:13.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.460+0000 7effe4584700 1 -- 172.21.15.32:0/1279793315 wait complete. 2023-09-30T22:25:13.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:13.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:13.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.851+0000 7fdad65dc700 1 Processor -- start 2023-09-30T22:25:13.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.851+0000 7fdad65dc700 1 -- start start 2023-09-30T22:25:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad65dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a0a30 0x7fdad00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad65dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdad00a1340 0x7fdad0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad65dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad0133fa0 0x7fdad0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad65dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad009edd0 con 0x7fdad00a1340 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad65dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad009e1f0 con 0x7fdad0133fa0 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad65dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdad009dc50 con 0x7fdad00a0a30 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdad00a1340 0x7fdad0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad4b79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad0133fa0 0x7fdad0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a0a30 0x7fdad00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdad00a1340 0x7fdad0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42648/0 (socket says 172.21.15.32:42648) 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 -- 172.21.15.32:0/4086965335 learned_addr learned my addr 172.21.15.32:0/4086965335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 -- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a0a30 msgr2=0x7fdad00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:13.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a0a30 0x7fdad00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 -- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad0133fa0 msgr2=0x7fdad0134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad0133fa0 0x7fdad0134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdacf7fe700 1 -- 172.21.15.32:0/4086965335 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad00a1fe0 con 0x7fdad00a1340 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.852+0000 7fdad4b79700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad0133fa0 0x7fdad0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.853+0000 7fdacf7fe700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad00a1340 0x7fdad0133a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdac40098f0 tx=0x7fdac4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ccccb9a5f8cf54 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.853+0000 7fdaceffd700 1 -- 172.21.15.32:0/4086965335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4011830 con 0x7fdad00a1340 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.853+0000 7fdaceffd700 1 -- 172.21.15.32:0/4086965335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac4011990 con 0x7fdad00a1340 2023-09-30T22:25:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.853+0000 7fdaceffd700 1 -- 172.21.15.32:0/4086965335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4011d20 con 0x7fdad00a1340 2023-09-30T22:25:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.853+0000 7fdad65dc700 1 -- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad00a1340 msgr2=0x7fdad0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.853+0000 7fdad65dc700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad00a1340 0x7fdad0133a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdac40098f0 tx=0x7fdac4009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 -- 172.21.15.32:0/4086965335 shutdown_connections 2023-09-30T22:25:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad0133fa0 0x7fdad0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a0a30 0x7fdad00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 --2- 172.21.15.32:0/4086965335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad00a1340 0x7fdad0133a60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 -- 172.21.15.32:0/4086965335 >> 172.21.15.32:0/4086965335 conn(0x7fdad00a6550 msgr2=0x7fdad00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 -- 172.21.15.32:0/4086965335 shutdown_connections 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.854+0000 7fdad65dc700 1 -- 172.21.15.32:0/4086965335 wait complete. 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 Processor -- start 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 -- start start 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad00a0a30 0x7fdad0120270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 0x7fdad01207b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad0133fa0 0x7fdad0120cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0121740 con 0x7fdad0133fa0 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0133660 con 0x7fdad00a0a30 2023-09-30T22:25:13.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad65dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdad01230d0 con 0x7fdad00a1340 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdacf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 0x7fdad01207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdacffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad00a0a30 0x7fdad0120270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdacf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 0x7fdad01207b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60458/0 (socket says 172.21.15.32:60458) 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdacf7fe700 1 -- 172.21.15.32:0/3667395982 learned_addr learned my addr 172.21.15.32:0/3667395982 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdacffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad00a0a30 0x7fdad0120270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33156/0 (socket says 172.21.15.32:33156) 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.855+0000 7fdad4b79700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad0133fa0 0x7fdad0120cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdacf7fe700 1 -- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad00a0a30 msgr2=0x7fdad0120270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdacf7fe700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad00a0a30 0x7fdad0120270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdacf7fe700 1 -- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad0133fa0 msgr2=0x7fdad0120cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdacf7fe700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad0133fa0 0x7fdad0120cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdacf7fe700 1 -- 172.21.15.32:0/3667395982 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8009a70 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdacf7fe700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 0x7fdad01207b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdac4011db0 tx=0x7fdac4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4004240 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac401f070 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac4009550 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40045a0 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.856+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad00a34a0 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.857+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdac4021030 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.857+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdac40407c0 con 0x7fdad00a1340 2023-09-30T22:25:13.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.857+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad0074ea0 con 0x7fdad00a1340 2023-09-30T22:25:13.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.862+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdac401a450 con 0x7fdad00a1340 2023-09-30T22:25:13.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.890+0000 7fcde225a700 1 Processor -- start 2023-09-30T22:25:13.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.890+0000 7fcde225a700 1 -- start start 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcde225a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 0x7fcddc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcde225a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 0x7fcddc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcde225a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc09edd0 con 0x7fcddc0a0a30 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcde225a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0a18e0 con 0x7fcddc0a1510 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcde225a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc09e1f0 con 0x7fcddc139250 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 0x7fcddc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 0x7fcddc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 0x7fcddc138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42664/0 (socket says 172.21.15.32:42664) 2023-09-30T22:25:13.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 0x7fcddc13b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60470/0 (socket says 172.21.15.32:60470) 2023-09-30T22:25:13.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddaffd700 1 -- 172.21.15.32:0/952911341 learned_addr learned my addr 172.21.15.32:0/952911341 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddbfff700 1 -- 172.21.15.32:0/952911341 >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 msgr2=0x7fcddc138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddbfff700 1 --2- 172.21.15.32:0/952911341 >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 0x7fcddc138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddbfff700 1 -- 172.21.15.32:0/952911341 >> v1:172.21.15.32:6789/0 conn(0x7fcddc0a0a30 legacy=0x7fcddc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddbfff700 1 -- 172.21.15.32:0/952911341 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc09dc50 con 0x7fcddc139250 2023-09-30T22:25:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcdda7fc700 1 -- 172.21.15.32:0/952911341 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771877919 0 0) 0x7fcddc09edd0 con 0x7fcddc0a0a30 2023-09-30T22:25:13.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.891+0000 7fcddaffd700 1 --2- 172.21.15.32:0/952911341 >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 0x7fcddc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.892+0000 7fcddbfff700 1 --2- 172.21.15.32:0/952911341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 0x7fcddc13b630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009d90 tx=0x7fcdd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=213c830e128eb170 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.892+0000 7fcdda7fc700 1 -- 172.21.15.32:0/952911341 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400e070 con 0x7fcddc139250 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.892+0000 7fcdda7fc700 1 -- 172.21.15.32:0/952911341 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd400bb20 con 0x7fcddc139250 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.892+0000 7fcdda7fc700 1 -- 172.21.15.32:0/952911341 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4012430 con 0x7fcddc139250 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 -- 172.21.15.32:0/952911341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 msgr2=0x7fcddc13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 --2- 172.21.15.32:0/952911341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 0x7fcddc13b630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009d90 tx=0x7fcdd400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 -- 172.21.15.32:0/952911341 shutdown_connections 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 --2- 172.21.15.32:0/952911341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc139250 0x7fcddc13b630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 --2- 172.21.15.32:0/952911341 >> v2:172.21.15.32:3300/0 conn(0x7fcddc0a1510 0x7fcddc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 -- 172.21.15.32:0/952911341 >> 172.21.15.32:0/952911341 conn(0x7fcddc0a6550 msgr2=0x7fcddc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:13.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 -- 172.21.15.32:0/952911341 shutdown_connections 2023-09-30T22:25:13.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.893+0000 7fcde225a700 1 -- 172.21.15.32:0/952911341 wait complete. 2023-09-30T22:25:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.894+0000 7fcde225a700 1 Processor -- start 2023-09-30T22:25:13.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.894+0000 7fcde225a700 1 -- start start 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcde225a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcddc0a0a30 0x7fcddc11e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcde225a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 0x7fcddc11eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcde225a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 0x7fcddc11f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcde225a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1221f0 con 0x7fcddc0a0a30 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcde225a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1218e0 con 0x7fcddc139250 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcde225a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0a5040 con 0x7fcddc0a1510 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcddaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 0x7fcddc11eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 0x7fcddc11f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcddaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 0x7fcddc11eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60486/0 (socket says 172.21.15.32:60486) 2023-09-30T22:25:13.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcddaffd700 1 -- 172.21.15.32:0/1540436110 learned_addr learned my addr 172.21.15.32:0/1540436110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcddbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 0x7fcddc11f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33168/0 (socket says 172.21.15.32:33168) 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.895+0000 7fcddb7fe700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcddc0a0a30 0x7fcddc11e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddbfff700 1 -- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 msgr2=0x7fcddc11eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddbfff700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 0x7fcddc11eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddbfff700 1 -- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcddc0a0a30 msgr2=0x7fcddc11e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddbfff700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcddc0a0a30 0x7fcddc11e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddbfff700 1 -- 172.21.15.32:0/1540436110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0009550 con 0x7fcddc139250 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddaffd700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 0x7fcddc11eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcddbfff700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 0x7fcddc11f0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4016000 tx=0x7fcdd4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4012430 con 0x7fcddc139250 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd4020070 con 0x7fcddc139250 2023-09-30T22:25:13.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd4009a70 con 0x7fcddc139250 2023-09-30T22:25:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4004280 con 0x7fcddc139250 2023-09-30T22:25:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.896+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc120a10 con 0x7fcddc139250 2023-09-30T22:25:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.897+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdd40043e0 con 0x7fcddc139250 2023-09-30T22:25:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.897+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcdd4022070 con 0x7fcddc139250 2023-09-30T22:25:13.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.897+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc0a1d00 con 0x7fcddc139250 2023-09-30T22:25:13.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:13.902+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdd402c080 con 0x7fcddc139250 2023-09-30T22:25:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.066+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdad00a6c20 con 0x7fdad00a1340 2023-09-30T22:25:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.067+0000 7fdaccff9700 1 -- 172.21.15.32:0/3667395982 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdac40233d0 con 0x7fdad00a1340 2023-09-30T22:25:14.067 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 msgr2=0x7fdad01207b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 0x7fdad01207b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdac4011db0 tx=0x7fdac4004750 comp rx=0 tx=0).stop 2023-09-30T22:25:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 shutdown_connections 2023-09-30T22:25:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad0133fa0 0x7fdad0120cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad00a0a30 0x7fdad0120270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 --2- 172.21.15.32:0/3667395982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdad00a1340 0x7fdad01207b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 >> 172.21.15.32:0/3667395982 conn(0x7fdad00a6550 msgr2=0x7fdad01baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 shutdown_connections 2023-09-30T22:25:14.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.069+0000 7fdad65dc700 1 -- 172.21.15.32:0/3667395982 wait complete. 2023-09-30T22:25:14.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.108+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcddc121cb0 con 0x7fcddc139250 2023-09-30T22:25:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.108+0000 7fcdc3fff700 1 -- 172.21.15.32:0/1540436110 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdd4012e40 con 0x7fcddc139250 2023-09-30T22:25:14.108 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 msgr2=0x7fcddc11f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 0x7fcddc11f0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4016000 tx=0x7fcdd4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 shutdown_connections 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcddc139250 0x7fcddc11f0b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcddc0a0a30 0x7fcddc11e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 --2- 172.21.15.32:0/1540436110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcddc0a1510 0x7fcddc11eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 >> 172.21.15.32:0/1540436110 conn(0x7fcddc0a6550 msgr2=0x7fcddc0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 shutdown_connections 2023-09-30T22:25:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.111+0000 7fcde225a700 1 -- 172.21.15.32:0/1540436110 wait complete. 2023-09-30T22:25:14.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:14.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:14.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 Processor -- start 2023-09-30T22:25:14.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 -- start start 2023-09-30T22:25:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a1510 0x7f23c0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23c0139250 0x7f23c013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c009edd0 con 0x7f23c00a0a30 2023-09-30T22:25:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00a18e0 con 0x7f23c0139250 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.589+0000 7f23c735c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23c009e1f0 con 0x7f23c00a1510 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c58f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23c0139250 0x7f23c013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c50f8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23c00a0a30 0x7f23c00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39004/0 (socket says 172.21.15.32:39004) 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c50f8700 1 -- 172.21.15.32:0/2565345539 learned_addr learned my addr 172.21.15.32:0/2565345539 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c48f7700 1 --2- 172.21.15.32:0/2565345539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a1510 0x7f23c0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23b7fff700 1 -- 172.21.15.32:0/2565345539 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851156353 0 0) 0x7f23c009edd0 con 0x7f23c00a0a30 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23b7fff700 1 -- 172.21.15.32:0/2565345539 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23ac0035b0 con 0x7f23c00a0a30 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c48f7700 1 -- 172.21.15.32:0/2565345539 >> v2:172.21.15.32:3300/0 conn(0x7f23c0139250 msgr2=0x7f23c013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c48f7700 1 --2- 172.21.15.32:0/2565345539 >> v2:172.21.15.32:3300/0 conn(0x7f23c0139250 0x7f23c013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c48f7700 1 -- 172.21.15.32:0/2565345539 >> v1:172.21.15.32:6789/0 conn(0x7f23c00a0a30 legacy=0x7f23c00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c48f7700 1 -- 172.21.15.32:0/2565345539 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c009dc50 con 0x7f23c00a1510 2023-09-30T22:25:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c58f9700 1 --2- 172.21.15.32:0/2565345539 >> v2:172.21.15.32:3300/0 conn(0x7f23c0139250 0x7f23c013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23c48f7700 1 --2- 172.21.15.32:0/2565345539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a1510 0x7f23c0138d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009870 tx=0x7f23b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11f0c4d9647f9c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23b7fff700 1 -- 172.21.15.32:0/2565345539 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0011830 con 0x7f23c00a1510 2023-09-30T22:25:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23b7fff700 1 -- 172.21.15.32:0/2565345539 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0011990 con 0x7f23c00a1510 2023-09-30T22:25:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.590+0000 7f23b7fff700 1 -- 172.21.15.32:0/2565345539 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0011d20 con 0x7f23c00a1510 2023-09-30T22:25:14.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 -- 172.21.15.32:0/2565345539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a1510 msgr2=0x7f23c0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 --2- 172.21.15.32:0/2565345539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a1510 0x7f23c0138d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009870 tx=0x7f23b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:14.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 -- 172.21.15.32:0/2565345539 shutdown_connections 2023-09-30T22:25:14.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 --2- 172.21.15.32:0/2565345539 >> v2:172.21.15.32:3300/0 conn(0x7f23c0139250 0x7f23c013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 --2- 172.21.15.32:0/2565345539 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a1510 0x7f23c0138d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 -- 172.21.15.32:0/2565345539 >> 172.21.15.32:0/2565345539 conn(0x7f23c00a6550 msgr2=0x7f23c00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 -- 172.21.15.32:0/2565345539 shutdown_connections 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.591+0000 7f23c735c700 1 -- 172.21.15.32:0/2565345539 wait complete. 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.592+0000 7f23c735c700 1 Processor -- start 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.592+0000 7f23c735c700 1 -- start start 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c735c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a0a30 0x7f23c01227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c735c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 0x7f23c0122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c735c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 0x7f23c0123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c735c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0138980 con 0x7f23c0139250 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c735c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0124c30 con 0x7f23c00a1510 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c735c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0138060 con 0x7f23c00a0a30 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c58f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 0x7f23c0123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 0x7f23c0122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c58f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 0x7f23c0123260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42698/0 (socket says 172.21.15.32:42698) 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 0x7f23c0122d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33174/0 (socket says 172.21.15.32:33174) 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c58f9700 1 -- 172.21.15.32:0/2632874304 learned_addr learned my addr 172.21.15.32:0/2632874304 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c50f8700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a0a30 0x7f23c01227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 -- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a0a30 msgr2=0x7f23c01227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a0a30 0x7f23c01227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 -- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 msgr2=0x7f23c0123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 0x7f23c0123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c48f7700 1 -- 172.21.15.32:0/2632874304 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8009a70 con 0x7f23c00a1510 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.593+0000 7f23c58f9700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 0x7f23c0123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.594+0000 7f23c48f7700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 0x7f23c0122d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f23b0016000 tx=0x7f23b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.594+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0004590 con 0x7f23c00a1510 2023-09-30T22:25:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.594+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b001f070 con 0x7f23c00a1510 2023-09-30T22:25:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.594+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b00094a0 con 0x7f23c00a1510 2023-09-30T22:25:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.594+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00038c0 con 0x7f23c00a1510 2023-09-30T22:25:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.594+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c0138640 con 0x7f23c00a1510 2023-09-30T22:25:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.595+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23b0003a20 con 0x7f23c00a1510 2023-09-30T22:25:14.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.595+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f23b0021070 con 0x7f23c00a1510 2023-09-30T22:25:14.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.595+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c0125d30 con 0x7f23c00a1510 2023-09-30T22:25:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.600+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23c0125d30 con 0x7f23c00a1510 2023-09-30T22:25:14.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 Processor -- start 2023-09-30T22:25:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 -- start start 2023-09-30T22:25:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 0x7ff7c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7c809d630 0x7ff7c80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c8133930 0x7ff7c8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c80a2a70 con 0x7ff7c809d630 2023-09-30T22:25:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80a01d0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.627+0000 7ff7cfd74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c809e2b0 con 0x7ff7c8133930 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7c809d630 0x7ff7c80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cdb10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 0x7ff7c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7ce311700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c8133930 0x7ff7c8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7c809d630 0x7ff7c80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42714/0 (socket says 172.21.15.32:42714) 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cdb10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 0x7ff7c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33184/0 (socket says 172.21.15.32:33184) 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 -- 172.21.15.32:0/3943453194 learned_addr learned my addr 172.21.15.32:0/3943453194 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 -- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c8133930 msgr2=0x7ff7c8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c8133930 0x7ff7c8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 -- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 msgr2=0x7ff7c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 0x7ff7c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 -- 172.21.15.32:0/3943453194 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c809dc50 con 0x7ff7c809d630 2023-09-30T22:25:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cdb10700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 0x7ff7c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7cd30f700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c809d630 0x7ff7c80a1340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0098f0 tx=0x7ff7bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae322a408621f804 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7ccb0e700 1 -- 172.21.15.32:0/3943453194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc011830 con 0x7ff7c809d630 2023-09-30T22:25:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7ccb0e700 1 -- 172.21.15.32:0/3943453194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc011990 con 0x7ff7c809d630 2023-09-30T22:25:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.628+0000 7ff7ccb0e700 1 -- 172.21.15.32:0/3943453194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc011d20 con 0x7ff7c809d630 2023-09-30T22:25:14.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 -- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c809d630 msgr2=0x7ff7c80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c809d630 0x7ff7c80a1340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0098f0 tx=0x7ff7bc009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:14.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 -- 172.21.15.32:0/3943453194 shutdown_connections 2023-09-30T22:25:14.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c80a0a30 0x7ff7c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c8133930 0x7ff7c8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/3943453194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c809d630 0x7ff7c80a1340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 -- 172.21.15.32:0/3943453194 >> 172.21.15.32:0/3943453194 conn(0x7ff7c80a3050 msgr2=0x7ff7c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 -- 172.21.15.32:0/3943453194 shutdown_connections 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.629+0000 7ff7cfd74700 1 -- 172.21.15.32:0/3943453194 wait complete. 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.631+0000 7ff7cfd74700 1 Processor -- start 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.631+0000 7ff7cfd74700 1 -- start start 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.631+0000 7ff7cfd74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c809d630 0x7ff7c81c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.631+0000 7ff7cfd74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 0x7ff7c81c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.631+0000 7ff7cfd74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c8133930 0x7ff7c81c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cfd74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c81206b0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cfd74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c81204c0 con 0x7ff7c8133930 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cfd74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c81223d0 con 0x7ff7c809d630 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 0x7ff7c81c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 0x7ff7c81c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42720/0 (socket says 172.21.15.32:42720) 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 -- 172.21.15.32:0/2032779900 learned_addr learned my addr 172.21.15.32:0/2032779900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7ce311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c8133930 0x7ff7c81c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cdb10700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c809d630 0x7ff7c81c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 -- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c809d630 msgr2=0x7ff7c81c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c809d630 0x7ff7c81c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 -- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c8133930 msgr2=0x7ff7c81c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c8133930 0x7ff7c81c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 -- 172.21.15.32:0/2032779900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8009a70 con 0x7ff7c80a0a30 2023-09-30T22:25:14.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cd30f700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 0x7ff7c81c8640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc000c00 tx=0x7ff7bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc004240 con 0x7ff7c80a0a30 2023-09-30T22:25:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc01f070 con 0x7ff7c80a0a30 2023-09-30T22:25:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc009550 con 0x7ff7c80a0a30 2023-09-30T22:25:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.632+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0045e0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.633+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c81220f0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.633+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c80747f0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.634+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7bc021030 con 0x7ff7c80a0a30 2023-09-30T22:25:14.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.634+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff7bc040d00 con 0x7ff7c80a0a30 2023-09-30T22:25:14.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.643+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7bc0237e0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.811+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f23c0125d30 con 0x7f23c00a1510 2023-09-30T22:25:14.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.811+0000 7f23b5ffb700 1 -- 172.21.15.32:0/2632874304 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23c0125d30 con 0x7f23c00a1510 2023-09-30T22:25:14.812 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 msgr2=0x7f23c0122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 0x7f23c0122d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f23b0016000 tx=0x7f23b0004750 comp rx=0 tx=0).stop 2023-09-30T22:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 shutdown_connections 2023-09-30T22:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23c0139250 0x7f23c0123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23c00a0a30 0x7f23c01227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 --2- 172.21.15.32:0/2632874304 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23c00a1510 0x7f23c0122d20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 >> 172.21.15.32:0/2632874304 conn(0x7f23c00a6550 msgr2=0x7f23c00a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 shutdown_connections 2023-09-30T22:25:14.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.815+0000 7f23c735c700 1 -- 172.21.15.32:0/2632874304 wait complete. 2023-09-30T22:25:14.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.852+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff7c8120cb0 con 0x7ff7c80a0a30 2023-09-30T22:25:14.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.853+0000 7ff7c67fc700 1 -- 172.21.15.32:0/2032779900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7bc015030 con 0x7ff7c80a0a30 2023-09-30T22:25:14.853 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 msgr2=0x7ff7c81c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:14.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 0x7ff7c81c8640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc000c00 tx=0x7ff7bc004750 comp rx=0 tx=0).stop 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 shutdown_connections 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7c8133930 0x7ff7c81c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7c809d630 0x7ff7c81c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 --2- 172.21.15.32:0/2032779900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7c80a0a30 0x7ff7c81c8640 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 >> 172.21.15.32:0/2032779900 conn(0x7ff7c80a3050 msgr2=0x7ff7c809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 shutdown_connections 2023-09-30T22:25:14.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:14.855+0000 7ff7cfd74700 1 -- 172.21.15.32:0/2032779900 wait complete. 2023-09-30T22:25:14.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:15.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:15.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:15.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.234+0000 7f878c674700 1 Processor -- start 2023-09-30T22:25:15.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.234+0000 7f878c674700 1 -- start start 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878c674700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878c674700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878c674700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8784133930 0x7f8784133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878c674700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87840a2a70 con 0x7f8784133930 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878c674700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87840a01d0 con 0x7f87840a0a30 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878c674700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f878409e2b0 con 0x7f878409d630 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f8789c0f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878a410700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f8789c0f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87840a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60536/0 (socket says 172.21.15.32:60536) 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f8789c0f700 1 -- 172.21.15.32:0/1149642568 learned_addr learned my addr 172.21.15.32:0/1149642568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.235+0000 7f878ac11700 1 --2- 172.21.15.32:0/1149642568 >> v2:172.21.15.32:3300/0 conn(0x7f8784133930 0x7f8784133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f8789c0f700 1 -- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 msgr2=0x7f87840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f8789c0f700 1 --2- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f8789c0f700 1 -- 172.21.15.32:0/1149642568 >> v2:172.21.15.32:3300/0 conn(0x7f8784133930 msgr2=0x7f8784133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f8789c0f700 1 --2- 172.21.15.32:0/1149642568 >> v2:172.21.15.32:3300/0 conn(0x7f8784133930 0x7f8784133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f8789c0f700 1 -- 172.21.15.32:0/1149642568 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878409dc50 con 0x7f878409d630 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f878ac11700 1 --2- 172.21.15.32:0/1149642568 >> v2:172.21.15.32:3300/0 conn(0x7f8784133930 0x7f8784133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f8789c0f700 1 --2- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87840a1340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87740098f0 tx=0x7f8774009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baaf33a91e17782e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f878940e700 1 -- 172.21.15.32:0/1149642568 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774011830 con 0x7f878409d630 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f878940e700 1 -- 172.21.15.32:0/1149642568 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8774011990 con 0x7f878409d630 2023-09-30T22:25:15.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.236+0000 7f878940e700 1 -- 172.21.15.32:0/1149642568 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774011d20 con 0x7f878409d630 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 -- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 msgr2=0x7f87840a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 --2- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87840a1340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87740098f0 tx=0x7f8774009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 -- 172.21.15.32:0/1149642568 shutdown_connections 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 --2- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 --2- 172.21.15.32:0/1149642568 >> v2:172.21.15.32:3300/0 conn(0x7f8784133930 0x7f8784133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 --2- 172.21.15.32:0/1149642568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87840a1340 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 -- 172.21.15.32:0/1149642568 >> 172.21.15.32:0/1149642568 conn(0x7f87840a3050 msgr2=0x7f87840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:15.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 -- 172.21.15.32:0/1149642568 shutdown_connections 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.237+0000 7f878c674700 1 -- 172.21.15.32:0/1149642568 wait complete. 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.238+0000 7f878c674700 1 Processor -- start 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.238+0000 7f878c674700 1 -- start start 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878c674700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87841c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878c674700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87841c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878c674700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8784133930 0x7f87841c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878c674700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784120660 con 0x7f8784133930 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878c674700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8784121e50 con 0x7f87840a0a30 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878c674700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87840a0410 con 0x7f878409d630 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878a410700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87841c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f8789c0f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87841c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878a410700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87841c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60538/0 (socket says 172.21.15.32:60538) 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878a410700 1 -- 172.21.15.32:0/3058641399 learned_addr learned my addr 172.21.15.32:0/3058641399 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.239+0000 7f878ac11700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8784133930 0x7f87841c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f8789c0f700 1 -- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 msgr2=0x7f87841c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f8789c0f700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87841c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f8789c0f700 1 -- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8784133930 msgr2=0x7f87841c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f8789c0f700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8784133930 0x7f87841c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f8789c0f700 1 -- 172.21.15.32:0/3058641399 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c009a70 con 0x7f87840a0a30 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f878a410700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87841c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f878ac11700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8784133930 0x7f87841c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f8789c0f700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87841c8540 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8774016000 tx=0x7f8774004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774004240 con 0x7f87840a0a30 2023-09-30T22:25:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877401f070 con 0x7f87840a0a30 2023-09-30T22:25:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87740045e0 con 0x7f87840a0a30 2023-09-30T22:25:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8774009550 con 0x7f87840a0a30 2023-09-30T22:25:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.240+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8784121150 con 0x7f87840a0a30 2023-09-30T22:25:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.241+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8774021030 con 0x7f87840a0a30 2023-09-30T22:25:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.241+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f87740406a0 con 0x7f87840a0a30 2023-09-30T22:25:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.241+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87840747f0 con 0x7f87840a0a30 2023-09-30T22:25:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.246+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8774015020 con 0x7f87840a0a30 2023-09-30T22:25:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 Processor -- start 2023-09-30T22:25:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 -- start start 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b80a0a30 0x7ff8b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8b80a1340 0x7ff8b8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b8133fa0 0x7ff8b8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b809edd0 con 0x7ff8b80a1340 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b809e1f0 con 0x7ff8b8133fa0 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bedfd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b809dc50 con 0x7ff8b80a0a30 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8b7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8b80a1340 0x7ff8b8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8b80a1340 0x7ff8b8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42750/0 (socket says 172.21.15.32:42750) 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 -- 172.21.15.32:0/1417351006 learned_addr learned my addr 172.21.15.32:0/1417351006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8bcb99700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b80a0a30 0x7ff8b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.272+0000 7ff8bd39a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b8133fa0 0x7ff8b8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 -- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b80a0a30 msgr2=0x7ff8b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b80a0a30 0x7ff8b80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 -- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b8133fa0 msgr2=0x7ff8b8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b8133fa0 0x7ff8b8134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 -- 172.21.15.32:0/1417351006 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b80a1fe0 con 0x7ff8b80a1340 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8bcb99700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b80a0a30 0x7ff8b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b7fff700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a1340 0x7ff8b8133a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0098f0 tx=0x7ff8ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a45a3885b05a14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b77fe700 1 -- 172.21.15.32:0/1417351006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011830 con 0x7ff8b80a1340 2023-09-30T22:25:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b77fe700 1 -- 172.21.15.32:0/1417351006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac011990 con 0x7ff8b80a1340 2023-09-30T22:25:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.273+0000 7ff8b77fe700 1 -- 172.21.15.32:0/1417351006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011d20 con 0x7ff8b80a1340 2023-09-30T22:25:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 -- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a1340 msgr2=0x7ff8b8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a1340 0x7ff8b8133a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0098f0 tx=0x7ff8ac009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 -- 172.21.15.32:0/1417351006 shutdown_connections 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b8133fa0 0x7ff8b8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b80a0a30 0x7ff8b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/1417351006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a1340 0x7ff8b8133a60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 -- 172.21.15.32:0/1417351006 >> 172.21.15.32:0/1417351006 conn(0x7ff8b80a6550 msgr2=0x7ff8b80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 -- 172.21.15.32:0/1417351006 shutdown_connections 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.274+0000 7ff8bedfd700 1 -- 172.21.15.32:0/1417351006 wait complete. 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 Processor -- start 2023-09-30T22:25:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 -- start start 2023-09-30T22:25:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 0x7ff8b8121310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b80a1340 0x7ff8b8121850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b8133fa0 0x7ff8b8121d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8124e30 con 0x7ff8b80a0a30 2023-09-30T22:25:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8123db0 con 0x7ff8b80a1340 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bedfd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b81326e0 con 0x7ff8b8133fa0 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bcb99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 0x7ff8b8121310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bd39a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b8133fa0 0x7ff8b8121d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bcb99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 0x7ff8b8121310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42762/0 (socket says 172.21.15.32:42762) 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8bcb99700 1 -- 172.21.15.32:0/2663794409 learned_addr learned my addr 172.21.15.32:0/2663794409 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.276+0000 7ff8b7fff700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b80a1340 0x7ff8b8121850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bcb99700 1 -- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b8133fa0 msgr2=0x7ff8b8121d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bcb99700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b8133fa0 0x7ff8b8121d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bcb99700 1 -- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b80a1340 msgr2=0x7ff8b8121850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bcb99700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b80a1340 0x7ff8b8121850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bcb99700 1 -- 172.21.15.32:0/2663794409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a80094a0 con 0x7ff8b80a0a30 2023-09-30T22:25:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bd39a700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b8133fa0 0x7ff8b8121d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bcb99700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 0x7ff8b8121310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800ebe0 tx=0x7ff8a800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800c850 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a800c9b0 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac009550 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800cd40 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.277+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b8124170 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.278+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8a8013470 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.278+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff8a803fae0 con 0x7ff8b80a0a30 2023-09-30T22:25:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.278+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b8074ea0 con 0x7ff8b80a0a30 2023-09-30T22:25:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.283+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8a8013720 con 0x7ff8b80a0a30 2023-09-30T22:25:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.455+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8784072980 con 0x7f87840a0a30 2023-09-30T22:25:15.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.455+0000 7f877affd700 1 -- 172.21.15.32:0/3058641399 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87740233d0 con 0x7f87840a0a30 2023-09-30T22:25:15.456 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 msgr2=0x7f87841c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87841c8540 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8774016000 tx=0x7f8774004750 comp rx=0 tx=0).stop 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 shutdown_connections 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8784133930 0x7f87841c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f878409d630 0x7f87841c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 --2- 172.21.15.32:0/3058641399 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87840a0a30 0x7f87841c8540 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 >> 172.21.15.32:0/3058641399 conn(0x7f87840a3050 msgr2=0x7f87841bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:15.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 shutdown_connections 2023-09-30T22:25:15.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.458+0000 7f878c674700 1 -- 172.21.15.32:0/3058641399 wait complete. 2023-09-30T22:25:15.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.489+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8b80a28e0 con 0x7ff8b80a0a30 2023-09-30T22:25:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.489+0000 7ff8b57fa700 1 -- 172.21.15.32:0/2663794409 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8a8021030 con 0x7ff8b80a0a30 2023-09-30T22:25:15.489 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 msgr2=0x7ff8b8121310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 0x7ff8b8121310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800ebe0 tx=0x7ff8a800eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 shutdown_connections 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8b8133fa0 0x7ff8b8121d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8b80a0a30 0x7ff8b8121310 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 --2- 172.21.15.32:0/2663794409 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8b80a1340 0x7ff8b8121850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 >> 172.21.15.32:0/2663794409 conn(0x7ff8b80a6550 msgr2=0x7ff8b80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.491+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 shutdown_connections 2023-09-30T22:25:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.492+0000 7ff8bedfd700 1 -- 172.21.15.32:0/2663794409 wait complete. 2023-09-30T22:25:15.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:15.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:15.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.981+0000 7fd1f27f1700 1 Processor -- start 2023-09-30T22:25:15.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.981+0000 7fd1f27f1700 1 -- start start 2023-09-30T22:25:15.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f27f1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 0x7fd1ec09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f27f1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd1ec09f240 0x7fd1ec1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f27f1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 0x7fd1ec131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f27f1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec070940 con 0x7fd1ec09f240 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f27f1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06fd60 con 0x7fd1ec09e930 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f27f1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06fa80 con 0x7fd1ec09f650 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 0x7fd1ec09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 0x7fd1ec131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 0x7fd1ec131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60564/0 (socket says 172.21.15.32:60564) 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 0x7fd1ec09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33250/0 (socket says 172.21.15.32:33250) 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 -- 172.21.15.32:0/4004919934 learned_addr learned my addr 172.21.15.32:0/4004919934 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:15.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/4004919934 >> v2:172.21.15.32:3300/0 conn(0x7fd1ec09f240 0x7fd1ec1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 -- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 msgr2=0x7fd1ec09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 --2- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 0x7fd1ec09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 -- 172.21.15.32:0/4004919934 >> v2:172.21.15.32:3300/0 conn(0x7fd1ec09f240 msgr2=0x7fd1ec1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 --2- 172.21.15.32:0/4004919934 >> v2:172.21.15.32:3300/0 conn(0x7fd1ec09f240 0x7fd1ec1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1f0d8e700 1 -- 172.21.15.32:0/4004919934 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec09fb60 con 0x7fd1ec09f650 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 0x7fd1ec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.982+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/4004919934 >> v2:172.21.15.32:3300/0 conn(0x7fd1ec09f240 0x7fd1ec1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.983+0000 7fd1f0d8e700 1 --2- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 0x7fd1ec131d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400f370 tx=0x7fd1e400f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4776cb3df258c9d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.983+0000 7fd1eaffd700 1 -- 172.21.15.32:0/4004919934 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4012070 con 0x7fd1ec09f650 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.983+0000 7fd1eaffd700 1 -- 172.21.15.32:0/4004919934 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e400d0d0 con 0x7fd1ec09f650 2023-09-30T22:25:15.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.983+0000 7fd1eaffd700 1 -- 172.21.15.32:0/4004919934 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400d480 con 0x7fd1ec09f650 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 -- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 msgr2=0x7fd1ec131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 0x7fd1ec131d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400f370 tx=0x7fd1e400f680 comp rx=0 tx=0).stop 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 -- 172.21.15.32:0/4004919934 shutdown_connections 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09e930 0x7fd1ec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/4004919934 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09f650 0x7fd1ec131d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/4004919934 >> v2:172.21.15.32:3300/0 conn(0x7fd1ec09f240 0x7fd1ec1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 -- 172.21.15.32:0/4004919934 >> 172.21.15.32:0/4004919934 conn(0x7fd1ec0a4210 msgr2=0x7fd1ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 -- 172.21.15.32:0/4004919934 shutdown_connections 2023-09-30T22:25:15.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.984+0000 7fd1f27f1700 1 -- 172.21.15.32:0/4004919934 wait complete. 2023-09-30T22:25:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 Processor -- start 2023-09-30T22:25:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 -- start start 2023-09-30T22:25:15.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 0x7fd1ec1c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec09f240 0x7fd1ec1c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09f650 0x7fd1ec1c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1a3e00 con 0x7fd1ec09f240 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1a2140 con 0x7fd1ec09f650 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.985+0000 7fd1f27f1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0a31f0 con 0x7fd1ec09e930 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 0x7fd1ec1c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 0x7fd1ec1c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60570/0 (socket says 172.21.15.32:60570) 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 -- 172.21.15.32:0/108530882 learned_addr learned my addr 172.21.15.32:0/108530882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1f0d8e700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09f650 0x7fd1ec1c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1eb7fe700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec09f240 0x7fd1ec1c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:15.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 -- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09f650 msgr2=0x7fd1ec1c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09f650 0x7fd1ec1c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 -- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec09f240 msgr2=0x7fd1ec1c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec09f240 0x7fd1ec1c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 -- 172.21.15.32:0/108530882 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0094a0 con 0x7fd1ec09e930 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1ebfff700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 0x7fd1ec1c3cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00ec10 tx=0x7fd1dc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc009780 con 0x7fd1ec09e930 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc00c850 con 0x7fd1ec09e930 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e00094a0 con 0x7fd1ec09e930 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00cbe0 con 0x7fd1ec09e930 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.986+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e400f050 con 0x7fd1ec09e930 2023-09-30T22:25:15.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.987+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec1a3c40 con 0x7fd1ec09e930 2023-09-30T22:25:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.988+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1dc00cd40 con 0x7fd1ec09e930 2023-09-30T22:25:15.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.988+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd1dc03fdb0 con 0x7fd1ec09e930 2023-09-30T22:25:15.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:15.992+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1dc013470 con 0x7fd1ec09e930 2023-09-30T22:25:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 Processor -- start 2023-09-30T22:25:16.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 -- start start 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c135870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442c09c190 con 0x7f442c130750 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442c09b090 con 0x7f442c133490 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f44316af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f442c09a6c0 con 0x7f442c135db0 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f442a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.020+0000 7f442b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c135870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33266/0 (socket says 172.21.15.32:33266) 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442a7fc700 1 -- 172.21.15.32:0/2122992752 learned_addr learned my addr 172.21.15.32:0/2122992752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442b7fe700 1 -- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 msgr2=0x7f442c135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442b7fe700 1 --2- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f4429ffb700 1 -- 172.21.15.32:0/2122992752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213277541 0 0) 0x7f442c09c190 con 0x7f442c130750 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442b7fe700 1 -- 172.21.15.32:0/2122992752 >> v1:172.21.15.32:6789/0 conn(0x7f442c130750 legacy=0x7f442c130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442b7fe700 1 -- 172.21.15.32:0/2122992752 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c130300 con 0x7f442c135db0 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442a7fc700 1 --2- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f442b7fe700 1 --2- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c138190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4424009870 tx=0x7f4424009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1633045b85fa6d5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f4429ffb700 1 -- 172.21.15.32:0/2122992752 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011830 con 0x7f442c135db0 2023-09-30T22:25:16.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f4429ffb700 1 -- 172.21.15.32:0/2122992752 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424011990 con 0x7f442c135db0 2023-09-30T22:25:16.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.021+0000 7f4429ffb700 1 -- 172.21.15.32:0/2122992752 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011d20 con 0x7f442c135db0 2023-09-30T22:25:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 -- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 msgr2=0x7f442c138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 --2- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c138190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4424009870 tx=0x7f4424009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 -- 172.21.15.32:0/2122992752 shutdown_connections 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 --2- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c138190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 --2- 172.21.15.32:0/2122992752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 -- 172.21.15.32:0/2122992752 >> 172.21.15.32:0/2122992752 conn(0x7f442c09f470 msgr2=0x7f442c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 -- 172.21.15.32:0/2122992752 shutdown_connections 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.022+0000 7f44316af700 1 -- 172.21.15.32:0/2122992752 wait complete. 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.023+0000 7f44316af700 1 Processor -- start 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.023+0000 7f44316af700 1 -- start start 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f44316af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f442c130750 0x7f442c127e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f44316af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c128390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f44316af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c1288d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f44316af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c071ca0 con 0x7f442c130750 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f44316af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442c12a710 con 0x7f442c133490 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f44316af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f442c09b2b0 con 0x7f442c135db0 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c128390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c1288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c128390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33268/0 (socket says 172.21.15.32:33268) 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442a7fc700 1 -- 172.21.15.32:0/3992275914 learned_addr learned my addr 172.21.15.32:0/3992275914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442affd700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f442c130750 0x7f442c127e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 -- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 msgr2=0x7f442c128390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c128390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 -- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f442c130750 msgr2=0x7f442c127e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f442c130750 0x7f442c127e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 -- 172.21.15.32:0/3992275914 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442000b7f0 con 0x7f442c135db0 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.024+0000 7f442b7fe700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c1288d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4424016000 tx=0x7f4424004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.025+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424004450 con 0x7f442c135db0 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.025+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44240045d0 con 0x7f442c135db0 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.025+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44240094a0 con 0x7f442c135db0 2023-09-30T22:25:16.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.025+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240038e0 con 0x7f442c135db0 2023-09-30T22:25:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.025+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c12a170 con 0x7f442c135db0 2023-09-30T22:25:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.026+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442c12ac70 con 0x7f442c135db0 2023-09-30T22:25:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.026+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4424003a40 con 0x7f442c135db0 2023-09-30T22:25:16.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.026+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4424040bd0 con 0x7f442c135db0 2023-09-30T22:25:16.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.031+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f442402c350 con 0x7f442c135db0 2023-09-30T22:25:16.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.201+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1ec1a3c40 con 0x7fd1ec09e930 2023-09-30T22:25:16.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.201+0000 7fd1e8ff9700 1 -- 172.21.15.32:0/108530882 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1dc017070 con 0x7fd1ec09e930 2023-09-30T22:25:16.201 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 msgr2=0x7fd1ec1c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 0x7fd1ec1c3cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00ec10 tx=0x7fd1dc00ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 shutdown_connections 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1ec09f650 0x7fd1ec1c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd1ec09e930 0x7fd1ec1c3cb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 --2- 172.21.15.32:0/108530882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1ec09f240 0x7fd1ec1c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 >> 172.21.15.32:0/108530882 conn(0x7fd1ec0a4210 msgr2=0x7fd1ec1b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 shutdown_connections 2023-09-30T22:25:16.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.205+0000 7fd1f27f1700 1 -- 172.21.15.32:0/108530882 wait complete. 2023-09-30T22:25:16.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.248+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f442c0747f0 con 0x7f442c135db0 2023-09-30T22:25:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.248+0000 7f44137fe700 1 -- 172.21.15.32:0/3992275914 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4424015070 con 0x7f442c135db0 2023-09-30T22:25:16.248 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 msgr2=0x7f442c1288d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c1288d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4424016000 tx=0x7f4424004750 comp rx=0 tx=0).stop 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 shutdown_connections 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f442c135db0 0x7f442c1288d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f442c130750 0x7f442c127e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 --2- 172.21.15.32:0/3992275914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f442c133490 0x7f442c128390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 >> 172.21.15.32:0/3992275914 conn(0x7f442c09f470 msgr2=0x7f442c09c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.250+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 shutdown_connections 2023-09-30T22:25:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.251+0000 7f44316af700 1 -- 172.21.15.32:0/3992275914 wait complete. 2023-09-30T22:25:16.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:16.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:16.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.620+0000 7fc11fd74700 1 Processor -- start 2023-09-30T22:25:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.620+0000 7fc11fd74700 1 -- start start 2023-09-30T22:25:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11fd74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1180a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11fd74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc118138d80 0x7fc11813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11fd74700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1180a2a70 con 0x7fc1180a0a30 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11fd74700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1180a01d0 con 0x7fc118138d80 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11fd74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11809e2b0 con 0x7fc11809d630 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11d30f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1180a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc118138d80 0x7fc11813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc118138d80 0x7fc11813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42796/0 (socket says 172.21.15.32:42796) 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11db10700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc1180a0a30 0x7fc1180a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39016/0 (socket says 172.21.15.32:39016) 2023-09-30T22:25:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11d30f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1180a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33272/0 (socket says 172.21.15.32:33272) 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 -- 172.21.15.32:0/3812929005 learned_addr learned my addr 172.21.15.32:0/3812929005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 -- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 msgr2=0x7fc1180a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 --2- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1180a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 -- 172.21.15.32:0/3812929005 >> v1:172.21.15.32:6789/0 conn(0x7fc1180a0a30 legacy=0x7fc1180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11cb0e700 1 -- 172.21.15.32:0/3812929005 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147280342 0 0) 0x7fc1180a2a70 con 0x7fc1180a0a30 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.621+0000 7fc11e311700 1 -- 172.21.15.32:0/3812929005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11809dc50 con 0x7fc118138d80 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.622+0000 7fc11e311700 1 --2- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc118138d80 0x7fc11813b160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc110009e50 tx=0x7fc11000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29afd593871ee532 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.622+0000 7fc11cb0e700 1 -- 172.21.15.32:0/3812929005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000e070 con 0x7fc118138d80 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.622+0000 7fc11cb0e700 1 -- 172.21.15.32:0/3812929005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000b910 con 0x7fc118138d80 2023-09-30T22:25:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.622+0000 7fc11cb0e700 1 -- 172.21.15.32:0/3812929005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bcc0 con 0x7fc118138d80 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.622+0000 7fc11fd74700 1 -- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc118138d80 msgr2=0x7fc11813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.622+0000 7fc11fd74700 1 --2- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc118138d80 0x7fc11813b160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc110009e50 tx=0x7fc11000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.623+0000 7fc11fd74700 1 -- 172.21.15.32:0/3812929005 shutdown_connections 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.623+0000 7fc11fd74700 1 --2- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc118138d80 0x7fc11813b160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.623+0000 7fc11fd74700 1 --2- 172.21.15.32:0/3812929005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1180a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.623+0000 7fc11fd74700 1 -- 172.21.15.32:0/3812929005 >> 172.21.15.32:0/3812929005 conn(0x7fc1180a3050 msgr2=0x7fc1180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.623+0000 7fc11fd74700 1 -- 172.21.15.32:0/3812929005 shutdown_connections 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.623+0000 7fc11fd74700 1 -- 172.21.15.32:0/3812929005 wait complete. 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.624+0000 7fc11fd74700 1 Processor -- start 2023-09-30T22:25:16.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.624+0000 7fc11fd74700 1 -- start start 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11fd74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1181335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11fd74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 0x7fc118133b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11fd74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc118138d80 0x7fc118134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11fd74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181371a0 con 0x7fc1180a0a30 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11fd74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180a49c0 con 0x7fc11809d630 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11fd74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180a0410 con 0x7fc118138d80 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 0x7fc118133b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11e311700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc118138d80 0x7fc118134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 0x7fc118133b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42798/0 (socket says 172.21.15.32:42798) 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 -- 172.21.15.32:0/4285803190 learned_addr learned my addr 172.21.15.32:0/4285803190 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11db10700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1181335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 -- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc118138d80 msgr2=0x7fc118134070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc118138d80 0x7fc118134070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 -- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 msgr2=0x7fc1181335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1181335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11d30f700 1 -- 172.21.15.32:0/4285803190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110009b30 con 0x7fc1180a0a30 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.625+0000 7fc11e311700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc118138d80 0x7fc118134070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.626+0000 7fc11d30f700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 0x7fc118133b30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e9b0 tx=0x7fc10800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.626+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108009710 con 0x7fc1180a0a30 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.626+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800ee50 con 0x7fc1180a0a30 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.626+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc118136af0 con 0x7fc1180a0a30 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.626+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800ca50 con 0x7fc1180a0a30 2023-09-30T22:25:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.626+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118137820 con 0x7fc1180a0a30 2023-09-30T22:25:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.627+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1180747f0 con 0x7fc1180a0a30 2023-09-30T22:25:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.627+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc10800cbb0 con 0x7fc1180a0a30 2023-09-30T22:25:16.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.628+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc10803fec0 con 0x7fc1180a0a30 2023-09-30T22:25:16.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.632+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc10801a360 con 0x7fc1180a0a30 2023-09-30T22:25:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.658+0000 7f5eaad03700 1 Processor -- start 2023-09-30T22:25:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.658+0000 7f5eaad03700 1 -- start start 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5eaad03700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 0x7f5ea4138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5eaad03700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 0x7f5ea413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5eaad03700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea406b990 con 0x7f5ea4133420 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5eaad03700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea409b0e0 con 0x7f5ea41337f0 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5eaad03700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4133c20 con 0x7f5ea4138dc0 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea92a0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 0x7f5ea413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5e9bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 0x7f5ea4138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea8a9f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ea4133420 0x7f5ea4133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39032/0 (socket says 172.21.15.32:39032) 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea92a0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 0x7f5ea413b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60594/0 (socket says 172.21.15.32:60594) 2023-09-30T22:25:16.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3318479242 learned_addr learned my addr 172.21.15.32:0/3318479242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5e9bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 0x7f5ea4138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42814/0 (socket says 172.21.15.32:42814) 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3318479242 >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 msgr2=0x7f5ea4138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea92a0700 1 --2- 172.21.15.32:0/3318479242 >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 0x7f5ea4138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3318479242 >> v1:172.21.15.32:6789/0 conn(0x7f5ea4133420 legacy=0x7f5ea4133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3318479242 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40a0590 con 0x7f5ea4138dc0 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.659+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/3318479242 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959553098 0 0) 0x7f5ea406b990 con 0x7f5ea4133420 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5e9bfff700 1 --2- 172.21.15.32:0/3318479242 >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 0x7f5ea4138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5ea92a0700 1 --2- 172.21.15.32:0/3318479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 0x7f5ea413b1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009e50 tx=0x7f5e9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d37b5ddfc2b62c2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/3318479242 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00e070 con 0x7f5ea4138dc0 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/3318479242 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c00b910 con 0x7f5ea4138dc0 2023-09-30T22:25:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5e9b7fe700 1 -- 172.21.15.32:0/3318479242 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00bcc0 con 0x7f5ea4138dc0 2023-09-30T22:25:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5eaad03700 1 -- 172.21.15.32:0/3318479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 msgr2=0x7f5ea413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3318479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 0x7f5ea413b1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009e50 tx=0x7f5e9c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5eaad03700 1 -- 172.21.15.32:0/3318479242 shutdown_connections 2023-09-30T22:25:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.660+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3318479242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea4138dc0 0x7f5ea413b1a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.661+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3318479242 >> v2:172.21.15.32:3300/0 conn(0x7f5ea41337f0 0x7f5ea4138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.661+0000 7f5eaad03700 1 -- 172.21.15.32:0/3318479242 >> 172.21.15.32:0/3318479242 conn(0x7f5ea40a4210 msgr2=0x7f5ea40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.661+0000 7f5eaad03700 1 -- 172.21.15.32:0/3318479242 shutdown_connections 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.661+0000 7f5eaad03700 1 -- 172.21.15.32:0/3318479242 wait complete. 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 Processor -- start 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 -- start start 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea4133420 0x7f5ea412ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 0x7f5ea412f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea4138dc0 0x7f5ea412f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea409c620 con 0x7f5ea4138dc0 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea409cc60 con 0x7f5ea4133420 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5eaad03700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea409d090 con 0x7f5ea41337f0 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5e9bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 0x7f5ea412f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5e9bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 0x7f5ea412f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60608/0 (socket says 172.21.15.32:60608) 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5e9bfff700 1 -- 172.21.15.32:0/3301286021 learned_addr learned my addr 172.21.15.32:0/3301286021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5ea8a9f700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea4133420 0x7f5ea412ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.662+0000 7f5ea92a0700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea4138dc0 0x7f5ea412f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 msgr2=0x7f5ea412f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea92a0700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 0x7f5ea412f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea4133420 msgr2=0x7f5ea412ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea92a0700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea4133420 0x7f5ea412ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea92a0700 1 -- 172.21.15.32:0/3301286021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940094a0 con 0x7f5ea4138dc0 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5e9bfff700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 0x7f5ea412f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea8a9f700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea4133420 0x7f5ea412ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5ea92a0700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea4138dc0 0x7f5ea412f9b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c015000 tx=0x7f5e9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c004880 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0092e0 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c009b30 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c01c680 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.663+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4130ab0 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.664+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e9c01c7e0 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.664+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5e9c040d30 con 0x7f5ea4138dc0 2023-09-30T22:25:16.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.664+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea409ca00 con 0x7f5ea4138dc0 2023-09-30T22:25:16.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.674+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ea409ca00 con 0x7f5ea4138dc0 2023-09-30T22:25:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.843+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc118135f30 con 0x7fc1180a0a30 2023-09-30T22:25:16.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.844+0000 7fc10e7fc700 1 -- 172.21.15.32:0/4285803190 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc108021020 con 0x7fc1180a0a30 2023-09-30T22:25:16.844 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 msgr2=0x7fc118133b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 0x7fc118133b30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e9b0 tx=0x7fc10800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 shutdown_connections 2023-09-30T22:25:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc118138d80 0x7fc118134070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc11809d630 0x7fc1181335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 --2- 172.21.15.32:0/4285803190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1180a0a30 0x7fc118133b30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 >> 172.21.15.32:0/4285803190 conn(0x7fc1180a3050 msgr2=0x7fc11813ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 shutdown_connections 2023-09-30T22:25:16.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.846+0000 7fc11fd74700 1 -- 172.21.15.32:0/4285803190 wait complete. 2023-09-30T22:25:16.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:16.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.878+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5ea409ca00 con 0x7f5ea4138dc0 2023-09-30T22:25:16.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.879+0000 7f5e997fa700 1 -- 172.21.15.32:0/3301286021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ea409ca00 con 0x7f5ea4138dc0 2023-09-30T22:25:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea4138dc0 msgr2=0x7f5ea412f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea4138dc0 0x7f5ea412f9b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c015000 tx=0x7f5e9c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 shutdown_connections 2023-09-30T22:25:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea4138dc0 0x7f5ea412f9b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea4133420 0x7f5ea412ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 --2- 172.21.15.32:0/3301286021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea41337f0 0x7f5ea412f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 >> 172.21.15.32:0/3301286021 conn(0x7f5ea40a4210 msgr2=0x7f5ea413a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 shutdown_connections 2023-09-30T22:25:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:16.881+0000 7f5eaad03700 1 -- 172.21.15.32:0/3301286021 wait complete. 2023-09-30T22:25:16.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:17.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:17.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:17.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.372+0000 7fef302fb700 1 Processor -- start 2023-09-30T22:25:17.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.372+0000 7fef302fb700 1 -- start start 2023-09-30T22:25:17.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef302fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 0x7fef280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef302fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef2809d630 0x7fef280a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef302fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef28133930 0x7fef28133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef302fb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef280a2a70 con 0x7fef2809d630 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef302fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280a01d0 con 0x7fef280a0a30 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef302fb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef2809e2b0 con 0x7fef28133930 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2e097700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 0x7fef280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef2809d630 0x7fef280a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2e097700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 0x7fef280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33288/0 (socket says 172.21.15.32:33288) 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2e898700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef28133930 0x7fef28133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef2809d630 0x7fef280a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42834/0 (socket says 172.21.15.32:42834) 2023-09-30T22:25:17.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2e097700 1 -- 172.21.15.32:0/1512604904 learned_addr learned my addr 172.21.15.32:0/1512604904 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 -- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef28133930 msgr2=0x7fef28133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef28133930 0x7fef28133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 -- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 msgr2=0x7fef280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 0x7fef280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2d896700 1 -- 172.21.15.32:0/1512604904 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2809dc50 con 0x7fef2809d630 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2e097700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 0x7fef280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.373+0000 7fef2e898700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef28133930 0x7fef28133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef2d896700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef280a1340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009840 tx=0x7fef1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc514a3fa88156ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef2d095700 1 -- 172.21.15.32:0/1512604904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c011830 con 0x7fef2809d630 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef2d095700 1 -- 172.21.15.32:0/1512604904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c011990 con 0x7fef2809d630 2023-09-30T22:25:17.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef2d095700 1 -- 172.21.15.32:0/1512604904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c011d20 con 0x7fef2809d630 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef302fb700 1 -- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 msgr2=0x7fef280a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef302fb700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef280a1340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009840 tx=0x7fef1c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef302fb700 1 -- 172.21.15.32:0/1512604904 shutdown_connections 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef302fb700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef280a0a30 0x7fef280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef302fb700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef28133930 0x7fef28133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.374+0000 7fef302fb700 1 --2- 172.21.15.32:0/1512604904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef280a1340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.375+0000 7fef302fb700 1 -- 172.21.15.32:0/1512604904 >> 172.21.15.32:0/1512604904 conn(0x7fef280a3050 msgr2=0x7fef280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:17.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.375+0000 7fef302fb700 1 -- 172.21.15.32:0/1512604904 shutdown_connections 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.375+0000 7fef302fb700 1 -- 172.21.15.32:0/1512604904 wait complete. 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.376+0000 7fef302fb700 1 Processor -- start 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.376+0000 7fef302fb700 1 -- start start 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef302fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef281c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef302fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef280a0a30 0x7fef281c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef302fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 0x7fef281c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef302fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef28120820 con 0x7fef2809d630 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef302fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef28122010 con 0x7fef28133930 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef302fb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef280a40c0 con 0x7fef280a0a30 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e898700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 0x7fef281c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef281c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e898700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 0x7fef281c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33292/0 (socket says 172.21.15.32:33292) 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef281c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42842/0 (socket says 172.21.15.32:42842) 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e898700 1 -- 172.21.15.32:0/2806922995 learned_addr learned my addr 172.21.15.32:0/2806922995 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2d896700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef280a0a30 0x7fef281c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 -- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef280a0a30 msgr2=0x7fef281c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef280a0a30 0x7fef281c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 -- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 msgr2=0x7fef281c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 0x7fef281c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 -- 172.21.15.32:0/2806922995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18009b30 con 0x7fef2809d630 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e898700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 0x7fef281c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2d896700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef280a0a30 0x7fef281c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.377+0000 7fef2e097700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef281c8090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fef1800efb0 tx=0x7fef1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.378+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18003c20 con 0x7fef2809d630 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.378+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1800f040 con 0x7fef2809d630 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.378+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800c900 con 0x7fef2809d630 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.378+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef200094a0 con 0x7fef2809d630 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.378+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c0094a0 con 0x7fef2809d630 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.379+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef1800ca60 con 0x7fef2809d630 2023-09-30T22:25:17.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.379+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef1803f9d0 con 0x7fef2809d630 2023-09-30T22:25:17.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.379+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef280747f0 con 0x7fef2809d630 2023-09-30T22:25:17.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.384+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef18014480 con 0x7fef2809d630 2023-09-30T22:25:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.410+0000 7f86f270f700 1 Processor -- start 2023-09-30T22:25:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.410+0000 7f86f270f700 1 -- start start 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f270f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 0x7f86ec09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f270f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f270f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f270f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86ec070940 con 0x7f86ec09e930 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f270f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec06fd60 con 0x7f86ec09f650 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f270f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec06fa80 con 0x7f86ec09f240 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f0cac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 0x7f86ec09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f0cac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33308/0 (socket says 172.21.15.32:33308) 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 0x7f86ec09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42852/0 (socket says 172.21.15.32:42852) 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86eb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.411+0000 7f86f0cac700 1 -- 172.21.15.32:0/3870453259 learned_addr learned my addr 172.21.15.32:0/3870453259 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eb7fe700 1 -- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 msgr2=0x7f86ec131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eb7fe700 1 --2- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eb7fe700 1 -- 172.21.15.32:0/3870453259 >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 msgr2=0x7f86ec09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eb7fe700 1 --2- 172.21.15.32:0/3870453259 >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 0x7f86ec09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eb7fe700 1 -- 172.21.15.32:0/3870453259 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec09fb60 con 0x7f86ec09f240 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86f0cac700 1 --2- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86ebfff700 1 --2- 172.21.15.32:0/3870453259 >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 0x7f86ec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eb7fe700 1 --2- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec1317f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009840 tx=0x7f86e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e83caf222f4ddae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eaffd700 1 -- 172.21.15.32:0/3870453259 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0011830 con 0x7f86ec09f240 2023-09-30T22:25:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eaffd700 1 -- 172.21.15.32:0/3870453259 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0011990 con 0x7f86ec09f240 2023-09-30T22:25:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.412+0000 7f86eaffd700 1 -- 172.21.15.32:0/3870453259 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0011d20 con 0x7f86ec09f240 2023-09-30T22:25:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 -- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 msgr2=0x7f86ec1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 --2- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec1317f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009840 tx=0x7f86e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 -- 172.21.15.32:0/3870453259 shutdown_connections 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 --2- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 --2- 172.21.15.32:0/3870453259 >> v2:172.21.15.32:3300/0 conn(0x7f86ec09e930 0x7f86ec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 --2- 172.21.15.32:0/3870453259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec1317f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 -- 172.21.15.32:0/3870453259 >> 172.21.15.32:0/3870453259 conn(0x7f86ec0a4210 msgr2=0x7f86ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 -- 172.21.15.32:0/3870453259 shutdown_connections 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.413+0000 7f86f270f700 1 -- 172.21.15.32:0/3870453259 wait complete. 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.414+0000 7f86f270f700 1 Processor -- start 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.414+0000 7f86f270f700 1 -- start start 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f270f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 0x7f86ec06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f270f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f270f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f270f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec1a8980 con 0x7f86ec09e930 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f270f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec1a9990 con 0x7f86ec09f650 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f270f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec06cd10 con 0x7f86ec09f240 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 0x7f86ec06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 0x7f86ec06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42856/0 (socket says 172.21.15.32:42856) 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86f0cac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86eb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.415+0000 7f86ebfff700 1 -- 172.21.15.32:0/3212319523 learned_addr learned my addr 172.21.15.32:0/3212319523 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86ebfff700 1 -- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 msgr2=0x7f86ec06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86ebfff700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86ebfff700 1 -- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 msgr2=0x7f86ec06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86ebfff700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86ebfff700 1 -- 172.21.15.32:0/3212319523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0094a0 con 0x7f86ec09e930 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86eb7fe700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86ebfff700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 0x7f86ec06b8b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00ec10 tx=0x7f86dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc009780 con 0x7f86ec09e930 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc00c850 con 0x7f86ec09e930 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00cbe0 con 0x7f86ec09e930 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e4009b30 con 0x7f86ec09e930 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.416+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e00094a0 con 0x7f86ec09e930 2023-09-30T22:25:17.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.417+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f86dc00cd40 con 0x7f86ec09e930 2023-09-30T22:25:17.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.417+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f86dc03f840 con 0x7f86ec09e930 2023-09-30T22:25:17.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.417+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec1a0b30 con 0x7f86ec09e930 2023-09-30T22:25:17.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.422+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f86dc0232f0 con 0x7f86ec09e930 2023-09-30T22:25:17.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.592+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef28072980 con 0x7fef2809d630 2023-09-30T22:25:17.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.592+0000 7fef26ffd700 1 -- 172.21.15.32:0/2806922995 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef1802c090 con 0x7fef2809d630 2023-09-30T22:25:17.592 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:17.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.594+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 msgr2=0x7fef281c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.594+0000 7fef302fb700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef281c8090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fef1800efb0 tx=0x7fef1800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 shutdown_connections 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef28133930 0x7fef281c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef2809d630 0x7fef281c8090 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 --2- 172.21.15.32:0/2806922995 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef280a0a30 0x7fef281c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 >> 172.21.15.32:0/2806922995 conn(0x7fef280a3050 msgr2=0x7fef2809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 shutdown_connections 2023-09-30T22:25:17.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.595+0000 7fef302fb700 1 -- 172.21.15.32:0/2806922995 wait complete. 2023-09-30T22:25:17.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:17.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.625+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86ec0a3550 con 0x7f86ec09e930 2023-09-30T22:25:17.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.625+0000 7f86e8ff9700 1 -- 172.21.15.32:0/3212319523 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86dc02a090 con 0x7f86ec09e930 2023-09-30T22:25:17.625 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:17.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 msgr2=0x7f86ec06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:17.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 0x7f86ec06b8b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00ec10 tx=0x7f86dc00ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:17.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 shutdown_connections 2023-09-30T22:25:17.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86ec09f650 0x7f86ec06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ec09e930 0x7f86ec06b8b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 --2- 172.21.15.32:0/3212319523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86ec09f240 0x7f86ec06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:17.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 >> 172.21.15.32:0/3212319523 conn(0x7f86ec0a4210 msgr2=0x7f86ec1b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:17.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 shutdown_connections 2023-09-30T22:25:17.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:17.627+0000 7f86f270f700 1 -- 172.21.15.32:0/3212319523 wait complete. 2023-09-30T22:25:17.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:17.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:17.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:18.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.011+0000 7f1d76b3f700 1 Processor -- start 2023-09-30T22:25:18.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.011+0000 7f1d76b3f700 1 -- start start 2023-09-30T22:25:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d76b3f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d76b3f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d7013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d76b3f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700a2a70 con 0x7f1d7009d630 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d76b3f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700a01d0 con 0x7f1d700a0a30 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d76b3f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7009e2b0 con 0x7f1d70138d80 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d7013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d7013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60662/0 (socket says 172.21.15.32:60662) 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 -- 172.21.15.32:0/3389908232 learned_addr learned my addr 172.21.15.32:0/3389908232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d748db700 1 --2- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 -- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 msgr2=0x7f1d700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 --2- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 -- 172.21.15.32:0/3389908232 >> v1:172.21.15.32:6789/0 conn(0x7f1d7009d630 legacy=0x7f1d700a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 -- 172.21.15.32:0/3389908232 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7009dc50 con 0x7f1d70138d80 2023-09-30T22:25:18.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d6f7fe700 1 -- 172.21.15.32:0/3389908232 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148975889 0 0) 0x7f1d700a2a70 con 0x7f1d7009d630 2023-09-30T22:25:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.012+0000 7f1d750dc700 1 --2- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d7013b160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009e80 tx=0x7f1d6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa227c9ad3e488d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d6f7fe700 1 -- 172.21.15.32:0/3389908232 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800e070 con 0x7f1d70138d80 2023-09-30T22:25:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d6f7fe700 1 -- 172.21.15.32:0/3389908232 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6800b910 con 0x7f1d70138d80 2023-09-30T22:25:18.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d6f7fe700 1 -- 172.21.15.32:0/3389908232 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800bcc0 con 0x7f1d70138d80 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 -- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 msgr2=0x7f1d7013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d7013b160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009e80 tx=0x7f1d6800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 -- 172.21.15.32:0/3389908232 shutdown_connections 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d7013b160 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/3389908232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 -- 172.21.15.32:0/3389908232 >> 172.21.15.32:0/3389908232 conn(0x7f1d700a3050 msgr2=0x7f1d700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.013+0000 7f1d76b3f700 1 -- 172.21.15.32:0/3389908232 shutdown_connections 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.014+0000 7f1d76b3f700 1 -- 172.21.15.32:0/3389908232 wait complete. 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.014+0000 7f1d76b3f700 1 Processor -- start 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.014+0000 7f1d76b3f700 1 -- start start 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d76b3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7009d630 0x7f1d701204b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d76b3f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d701209f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d76b3f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d70120f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d76b3f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d70123d40 con 0x7f1d7009d630 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d76b3f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d70122e30 con 0x7f1d700a0a30 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d76b3f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d701220d0 con 0x7f1d70138d80 2023-09-30T22:25:18.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d748db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7009d630 0x7f1d701204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d6ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d701209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d748db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7009d630 0x7f1d701204b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42860/0 (socket says 172.21.15.32:42860) 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d6ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d701209f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33344/0 (socket says 172.21.15.32:33344) 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d748db700 1 -- 172.21.15.32:0/2392383032 learned_addr learned my addr 172.21.15.32:0/2392383032 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.015+0000 7f1d750dc700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d70120f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6ffff700 1 -- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 msgr2=0x7f1d70120f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6ffff700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d70120f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6ffff700 1 -- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7009d630 msgr2=0x7f1d701204b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6ffff700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7009d630 0x7f1d701204b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6ffff700 1 -- 172.21.15.32:0/2392383032 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009b30 con 0x7f1d700a0a30 2023-09-30T22:25:18.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6ffff700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d701209f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d7006d980 tx=0x7f1d6400c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d640054e0 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6400ce10 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d700a4800 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d640038c0 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.016+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d701232f0 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.017+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d64003a20 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.017+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d700747f0 con 0x7f1d700a0a30 2023-09-30T22:25:18.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.017+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d6403ea90 con 0x7f1d700a0a30 2023-09-30T22:25:18.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.022+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d64019b20 con 0x7f1d700a0a30 2023-09-30T22:25:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.050+0000 7f5c80a0d700 1 Processor -- start 2023-09-30T22:25:18.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.050+0000 7f5c80a0d700 1 -- start start 2023-09-30T22:25:18.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.050+0000 7f5c80a0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c7c09e930 0x7f5c7c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.050+0000 7f5c80a0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09f650 0x7f5c7c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c80a0d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c070940 con 0x7f5c7c09f240 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c80a0d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c06fd60 con 0x7f5c7c09e930 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c80a0d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c06fa80 con 0x7f5c7c09f650 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c7c09e930 0x7f5c7c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09f650 0x7f5c7c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c7c09e930 0x7f5c7c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42866/0 (socket says 172.21.15.32:42866) 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 -- 172.21.15.32:0/4007989814 learned_addr learned my addr 172.21.15.32:0/4007989814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 -- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09f650 msgr2=0x7f5c7c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 --2- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09f650 0x7f5c7c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 -- 172.21.15.32:0/4007989814 >> v1:172.21.15.32:6789/0 conn(0x7f5c7c09f240 legacy=0x7f5c7c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7959a700 1 -- 172.21.15.32:0/4007989814 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870817125 0 0) 0x7f5c7c070940 con 0x7f5c7c09f240 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 -- 172.21.15.32:0/4007989814 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c09fb60 con 0x7f5c7c09e930 2023-09-30T22:25:18.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7ad9d700 1 --2- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09f650 0x7f5c7c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7a59c700 1 --2- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09e930 0x7f5c7c09ed00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0098f0 tx=0x7f5c6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce43ee7862af44b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7959a700 1 -- 172.21.15.32:0/4007989814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c011830 con 0x7f5c7c09e930 2023-09-30T22:25:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7959a700 1 -- 172.21.15.32:0/4007989814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c011990 con 0x7f5c7c09e930 2023-09-30T22:25:18.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.051+0000 7f5c7959a700 1 -- 172.21.15.32:0/4007989814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c011d20 con 0x7f5c7c09e930 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 -- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09e930 msgr2=0x7f5c7c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09e930 0x7f5c7c09ed00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0098f0 tx=0x7f5c6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 -- 172.21.15.32:0/4007989814 shutdown_connections 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09f650 0x7f5c7c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/4007989814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09e930 0x7f5c7c09ed00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 -- 172.21.15.32:0/4007989814 >> 172.21.15.32:0/4007989814 conn(0x7f5c7c0a4210 msgr2=0x7f5c7c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 -- 172.21.15.32:0/4007989814 shutdown_connections 2023-09-30T22:25:18.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.052+0000 7f5c80a0d700 1 -- 172.21.15.32:0/4007989814 wait complete. 2023-09-30T22:25:18.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.054+0000 7f5c80a0d700 1 Processor -- start 2023-09-30T22:25:18.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.054+0000 7f5c80a0d700 1 -- start start 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c80a0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 0x7f5c7c1c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c80a0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09f240 0x7f5c7c1c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c80a0d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c09f650 0x7f5c7c1c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c80a0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a7400 con 0x7f5c7c09f240 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c80a0d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a4f70 con 0x7f5c7c09f650 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c80a0d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a75e0 con 0x7f5c7c09e930 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 0x7f5c7c1c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 0x7f5c7c1c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60684/0 (socket says 172.21.15.32:60684) 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 -- 172.21.15.32:0/3687312584 learned_addr learned my addr 172.21.15.32:0/3687312584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:18.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c79d9b700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09f240 0x7f5c7c1c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 -- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c09f650 msgr2=0x7f5c7c1c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c09f650 0x7f5c7c1c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 -- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09f240 msgr2=0x7f5c7c1c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09f240 0x7f5c7c1c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 -- 172.21.15.32:0/3687312584 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009a70 con 0x7f5c7c09e930 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c7a59c700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 0x7f5c7c1c2e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009c40 tx=0x7f5c6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.055+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004380 con 0x7f5c7c09e930 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.056+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c01f070 con 0x7f5c7c09e930 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.056+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c009550 con 0x7f5c7c09e930 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.056+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0036a0 con 0x7f5c7c09e930 2023-09-30T22:25:18.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.056+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c0a0190 con 0x7f5c7c09e930 2023-09-30T22:25:18.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.056+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c6c021030 con 0x7f5c7c09e930 2023-09-30T22:25:18.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.057+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c6c040a60 con 0x7f5c7c09e930 2023-09-30T22:25:18.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.057+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c074ea0 con 0x7f5c7c09e930 2023-09-30T22:25:18.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.062+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c6c015070 con 0x7f5c7c09e930 2023-09-30T22:25:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.236+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1d70072980 con 0x7f1d700a0a30 2023-09-30T22:25:18.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.236+0000 7f1d6d7fa700 1 -- 172.21.15.32:0/2392383032 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d64017070 con 0x7f1d700a0a30 2023-09-30T22:25:18.236 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:18.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 msgr2=0x7f1d701209f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d701209f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d7006d980 tx=0x7f1d6400c710 comp rx=0 tx=0).stop 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 shutdown_connections 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d70138d80 0x7f1d70120f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7009d630 0x7f1d701204b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 --2- 172.21.15.32:0/2392383032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d700a0a30 0x7f1d701209f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.238+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 >> 172.21.15.32:0/2392383032 conn(0x7f1d700a3050 msgr2=0x7f1d7009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.239+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 shutdown_connections 2023-09-30T22:25:18.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.239+0000 7f1d76b3f700 1 -- 172.21.15.32:0/2392383032 wait complete. 2023-09-30T22:25:18.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:18.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.271+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5c7c1a6920 con 0x7f5c7c09e930 2023-09-30T22:25:18.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.272+0000 7f5c6affd700 1 -- 172.21.15.32:0/3687312584 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c6c019770 con 0x7f5c7c09e930 2023-09-30T22:25:18.272 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 msgr2=0x7f5c7c1c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 0x7f5c7c1c2e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009c40 tx=0x7f5c6c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 shutdown_connections 2023-09-30T22:25:18.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c09f650 0x7f5c7c1c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c09e930 0x7f5c7c1c2e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 --2- 172.21.15.32:0/3687312584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c09f240 0x7f5c7c1c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 >> 172.21.15.32:0/3687312584 conn(0x7f5c7c0a4210 msgr2=0x7f5c7c1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 shutdown_connections 2023-09-30T22:25:18.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.274+0000 7f5c80a0d700 1 -- 172.21.15.32:0/3687312584 wait complete. 2023-09-30T22:25:18.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:18.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:18.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.756+0000 7fd91bb7c700 1 Processor -- start 2023-09-30T22:25:18.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.756+0000 7fd91bb7c700 1 -- start start 2023-09-30T22:25:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91bb7c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 0x7fd914133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91bb7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 0x7fd91413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91bb7c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91406bc00 con 0x7fd91412ff20 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91bb7c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91406c570 con 0x7fd914133a80 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91bb7c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140a0040 con 0x7fd914130a70 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd919918700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 0x7fd914133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91a119700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 0x7fd91413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd919117700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd91412ff20 0x7fd914130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39058/0 (socket says 172.21.15.32:39058) 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd919918700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 0x7fd914133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42878/0 (socket says 172.21.15.32:42878) 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91a119700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 0x7fd91413b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60696/0 (socket says 172.21.15.32:60696) 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd919117700 1 -- 172.21.15.32:0/864229792 learned_addr learned my addr 172.21.15.32:0/864229792 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91a119700 1 -- 172.21.15.32:0/864229792 >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 msgr2=0x7fd914133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:18.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91a119700 1 --2- 172.21.15.32:0/864229792 >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 0x7fd914133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91a119700 1 -- 172.21.15.32:0/864229792 >> v1:172.21.15.32:6789/0 conn(0x7fd91412ff20 legacy=0x7fd914130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd91a119700 1 -- 172.21.15.32:0/864229792 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914133860 con 0x7fd914130a70 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd918916700 1 -- 172.21.15.32:0/864229792 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774190519 0 0) 0x7fd91406bc00 con 0x7fd91412ff20 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.757+0000 7fd919918700 1 --2- 172.21.15.32:0/864229792 >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 0x7fd914133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.758+0000 7fd91a119700 1 --2- 172.21.15.32:0/864229792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 0x7fd91413b850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009840 tx=0x7fd90c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1362bfc922e432a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.758+0000 7fd918916700 1 -- 172.21.15.32:0/864229792 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011830 con 0x7fd914130a70 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.758+0000 7fd918916700 1 -- 172.21.15.32:0/864229792 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c011990 con 0x7fd914130a70 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.758+0000 7fd918916700 1 -- 172.21.15.32:0/864229792 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011d20 con 0x7fd914130a70 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.758+0000 7fd91bb7c700 1 -- 172.21.15.32:0/864229792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 msgr2=0x7fd91413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.758+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/864229792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 0x7fd91413b850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009840 tx=0x7fd90c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:18.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.759+0000 7fd91bb7c700 1 -- 172.21.15.32:0/864229792 shutdown_connections 2023-09-30T22:25:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.759+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/864229792 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd914130a70 0x7fd91413b850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.759+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/864229792 >> v2:172.21.15.32:3300/0 conn(0x7fd914133a80 0x7fd914133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.759+0000 7fd91bb7c700 1 -- 172.21.15.32:0/864229792 >> 172.21.15.32:0/864229792 conn(0x7fd9140a4210 msgr2=0x7fd9140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.759+0000 7fd91bb7c700 1 -- 172.21.15.32:0/864229792 shutdown_connections 2023-09-30T22:25:18.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.759+0000 7fd91bb7c700 1 -- 172.21.15.32:0/864229792 wait complete. 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 Processor -- start 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 -- start start 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd91412ff20 0x7fd91409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 0x7fd91409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd914133a80 0x7fd91409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91409ddb0 con 0x7fd914130a70 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91411ff00 con 0x7fd914133a80 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91bb7c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd91411fbd0 con 0x7fd91412ff20 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd919117700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 0x7fd91409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd919918700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd91412ff20 0x7fd91409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd919117700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 0x7fd91409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42892/0 (socket says 172.21.15.32:42892) 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd919117700 1 -- 172.21.15.32:0/1959995476 learned_addr learned my addr 172.21.15.32:0/1959995476 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.760+0000 7fd91a119700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd914133a80 0x7fd91409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919117700 1 -- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd91412ff20 msgr2=0x7fd91409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919117700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd91412ff20 0x7fd91409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919117700 1 -- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd914133a80 msgr2=0x7fd91409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919117700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd914133a80 0x7fd91409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919117700 1 -- 172.21.15.32:0/1959995476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904009a70 con 0x7fd914130a70 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919918700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd91412ff20 0x7fd91409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd919117700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 0x7fd91409c9c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c520 tx=0x7fd90800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908004c80 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908004e00 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c0094a0 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908007310 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.761+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91411f010 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.762+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd908007470 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.762+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd90803f290 con 0x7fd914130a70 2023-09-30T22:25:18.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.762+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd914074ea0 con 0x7fd914130a70 2023-09-30T22:25:18.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.767+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9080123b0 con 0x7fd914130a70 2023-09-30T22:25:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.794+0000 7f2e7eb51700 1 Processor -- start 2023-09-30T22:25:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.794+0000 7f2e7eb51700 1 -- start start 2023-09-30T22:25:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7eb51700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e7809d630 0x7f2e780a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7eb51700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e78138d80 0x7f2e7813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7eb51700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e780a2a70 con 0x7f2e780a0a30 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7eb51700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e780a01d0 con 0x7f2e78138d80 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7eb51700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7809e2b0 con 0x7f2e7809d630 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7d0ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e78138d80 0x7f2e7813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7c8ed700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e780a0a30 0x7f2e780a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39068/0 (socket says 172.21.15.32:39068) 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7c8ed700 1 -- 172.21.15.32:0/1423669376 learned_addr learned my addr 172.21.15.32:0/1423669376 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e77fff700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e7809d630 0x7f2e780a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7d0ee700 1 -- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e7809d630 msgr2=0x7f2e780a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7d0ee700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e7809d630 0x7f2e780a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7d0ee700 1 -- 172.21.15.32:0/1423669376 >> v1:172.21.15.32:6789/0 conn(0x7f2e780a0a30 legacy=0x7f2e780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7d0ee700 1 -- 172.21.15.32:0/1423669376 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7809dc50 con 0x7f2e78138d80 2023-09-30T22:25:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e777fe700 1 -- 172.21.15.32:0/1423669376 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505310617 0 0) 0x7f2e780a2a70 con 0x7f2e780a0a30 2023-09-30T22:25:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e77fff700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e7809d630 0x7f2e780a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.795+0000 7f2e7d0ee700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e78138d80 0x7f2e7813b160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009840 tx=0x7f2e70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257e78618e295a71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e777fe700 1 -- 172.21.15.32:0/1423669376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70011830 con 0x7f2e78138d80 2023-09-30T22:25:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e777fe700 1 -- 172.21.15.32:0/1423669376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70011990 con 0x7f2e78138d80 2023-09-30T22:25:18.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e777fe700 1 -- 172.21.15.32:0/1423669376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70011d20 con 0x7f2e78138d80 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 -- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e78138d80 msgr2=0x7f2e7813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e78138d80 0x7f2e7813b160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009840 tx=0x7f2e70009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 -- 172.21.15.32:0/1423669376 shutdown_connections 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e78138d80 0x7f2e7813b160 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/1423669376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e7809d630 0x7f2e780a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 -- 172.21.15.32:0/1423669376 >> 172.21.15.32:0/1423669376 conn(0x7f2e780a3050 msgr2=0x7f2e780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 -- 172.21.15.32:0/1423669376 shutdown_connections 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.796+0000 7f2e7eb51700 1 -- 172.21.15.32:0/1423669376 wait complete. 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.797+0000 7f2e7eb51700 1 Processor -- start 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.797+0000 7f2e7eb51700 1 -- start start 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7eb51700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e7809d630 0x7f2e7812c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7eb51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 0x7f2e7812cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7eb51700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 0x7f2e7812cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7eb51700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e78130190 con 0x7f2e780a0a30 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7eb51700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7812fc10 con 0x7f2e78138d80 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7eb51700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e780a0410 con 0x7f2e7809d630 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 0x7f2e7812cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7d0ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 0x7f2e7812cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 0x7f2e7812cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42906/0 (socket says 172.21.15.32:42906) 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7d0ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 0x7f2e7812cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33386/0 (socket says 172.21.15.32:33386) 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e77fff700 1 -- 172.21.15.32:0/467296901 learned_addr learned my addr 172.21.15.32:0/467296901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7c8ed700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e7809d630 0x7f2e7812c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7d0ee700 1 -- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e7809d630 msgr2=0x7f2e7812c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7d0ee700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e7809d630 0x7f2e7812c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.798+0000 7f2e7d0ee700 1 -- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 msgr2=0x7f2e7812cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e7d0ee700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 0x7f2e7812cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e7d0ee700 1 -- 172.21.15.32:0/467296901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0094a0 con 0x7f2e78138d80 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e77fff700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 0x7f2e7812cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e7d0ee700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 0x7f2e7812cff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e70016000 tx=0x7f2e70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70004380 con 0x7f2e78138d80 2023-09-30T22:25:18.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7001f070 con 0x7f2e78138d80 2023-09-30T22:25:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e700094a0 con 0x7f2e78138d80 2023-09-30T22:25:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e700036a0 con 0x7f2e78138d80 2023-09-30T22:25:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.799+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7812e5a0 con 0x7f2e78138d80 2023-09-30T22:25:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.800+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e70021030 con 0x7f2e78138d80 2023-09-30T22:25:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.800+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e780747f0 con 0x7f2e78138d80 2023-09-30T22:25:18.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.800+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e700407e0 con 0x7f2e78138d80 2023-09-30T22:25:18.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.805+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e700239c0 con 0x7f2e78138d80 2023-09-30T22:25:18.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.973+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd914120070 con 0x7fd914130a70 2023-09-30T22:25:18.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.974+0000 7fd9127fc700 1 -- 172.21.15.32:0/1959995476 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd908011360 con 0x7fd914130a70 2023-09-30T22:25:18.974 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:18.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 msgr2=0x7fd91409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:18.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 0x7fd91409c9c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c520 tx=0x7fd90800c830 comp rx=0 tx=0).stop 2023-09-30T22:25:18.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 shutdown_connections 2023-09-30T22:25:18.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd914133a80 0x7fd91409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd91412ff20 0x7fd91409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 --2- 172.21.15.32:0/1959995476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd914130a70 0x7fd91409c9c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 >> 172.21.15.32:0/1959995476 conn(0x7fd9140a4210 msgr2=0x7fd91409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 shutdown_connections 2023-09-30T22:25:18.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:18.976+0000 7fd91bb7c700 1 -- 172.21.15.32:0/1959995476 wait complete. 2023-09-30T22:25:18.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.021+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e78072980 con 0x7f2e78138d80 2023-09-30T22:25:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.022+0000 7f2e757fa700 1 -- 172.21.15.32:0/467296901 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e70015070 con 0x7f2e78138d80 2023-09-30T22:25:19.022 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:19.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 msgr2=0x7f2e7812cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 0x7f2e7812cff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e70016000 tx=0x7f2e70004750 comp rx=0 tx=0).stop 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 shutdown_connections 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e78138d80 0x7f2e7812cff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e7809d630 0x7f2e7812c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 --2- 172.21.15.32:0/467296901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e780a0a30 0x7f2e7812cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 >> 172.21.15.32:0/467296901 conn(0x7f2e780a3050 msgr2=0x7f2e7809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 shutdown_connections 2023-09-30T22:25:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.024+0000 7f2e7eb51700 1 -- 172.21.15.32:0/467296901 wait complete. 2023-09-30T22:25:19.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:19.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:19.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:19.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 Processor -- start 2023-09-30T22:25:19.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 -- start start 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd20009d630 0x7fd2000a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd200138d80 0x7fd20013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2000a2a70 con 0x7fd2000a0a30 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000a01d0 con 0x7fd200138d80 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd2064f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20009e2b0 con 0x7fd20009d630 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd1fffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd2000a0a30 0x7fd2000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39080/0 (socket says 172.21.15.32:39080) 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd1fffff700 1 -- 172.21.15.32:0/1883532295 learned_addr learned my addr 172.21.15.32:0/1883532295 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd1ff7fe700 1 --2- 172.21.15.32:0/1883532295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd20009d630 0x7fd2000a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.408+0000 7fd204a8d700 1 --2- 172.21.15.32:0/1883532295 >> v2:172.21.15.32:3300/0 conn(0x7fd200138d80 0x7fd20013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1feffd700 1 -- 172.21.15.32:0/1883532295 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534201114 0 0) 0x7fd2000a2a70 con 0x7fd2000a0a30 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1feffd700 1 -- 172.21.15.32:0/1883532295 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1e80035b0 con 0x7fd2000a0a30 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/1883532295 >> v2:172.21.15.32:3300/0 conn(0x7fd200138d80 msgr2=0x7fd20013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1ff7fe700 1 --2- 172.21.15.32:0/1883532295 >> v2:172.21.15.32:3300/0 conn(0x7fd200138d80 0x7fd20013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/1883532295 >> v1:172.21.15.32:6789/0 conn(0x7fd2000a0a30 legacy=0x7fd2000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/1883532295 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20009dc50 con 0x7fd20009d630 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd204a8d700 1 --2- 172.21.15.32:0/1883532295 >> v2:172.21.15.32:3300/0 conn(0x7fd200138d80 0x7fd20013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1ff7fe700 1 --2- 172.21.15.32:0/1883532295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd20009d630 0x7fd2000a1510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009840 tx=0x7fd1f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784534cbae076a95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1feffd700 1 -- 172.21.15.32:0/1883532295 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4011830 con 0x7fd20009d630 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1feffd700 1 -- 172.21.15.32:0/1883532295 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4011990 con 0x7fd20009d630 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.409+0000 7fd1feffd700 1 -- 172.21.15.32:0/1883532295 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4011d20 con 0x7fd20009d630 2023-09-30T22:25:19.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 -- 172.21.15.32:0/1883532295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd20009d630 msgr2=0x7fd2000a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 --2- 172.21.15.32:0/1883532295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd20009d630 0x7fd2000a1510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009840 tx=0x7fd1f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 -- 172.21.15.32:0/1883532295 shutdown_connections 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 --2- 172.21.15.32:0/1883532295 >> v2:172.21.15.32:3300/0 conn(0x7fd200138d80 0x7fd20013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 --2- 172.21.15.32:0/1883532295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd20009d630 0x7fd2000a1510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 -- 172.21.15.32:0/1883532295 >> 172.21.15.32:0/1883532295 conn(0x7fd2000a3050 msgr2=0x7fd2000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 -- 172.21.15.32:0/1883532295 shutdown_connections 2023-09-30T22:25:19.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.410+0000 7fd2064f0700 1 -- 172.21.15.32:0/1883532295 wait complete. 2023-09-30T22:25:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.411+0000 7fd2064f0700 1 Processor -- start 2023-09-30T22:25:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.411+0000 7fd2064f0700 1 -- start start 2023-09-30T22:25:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd2064f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd20009d630 0x7fd20012c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd2064f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 0x7fd20012cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd2064f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd200138d80 0x7fd20012d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd2064f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20012e190 con 0x7fd20009d630 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd2064f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20013abb0 con 0x7fd2000a0a30 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd2064f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000a0410 con 0x7fd200138d80 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 0x7fd20012cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 0x7fd20012cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33404/0 (socket says 172.21.15.32:33404) 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/679822101 learned_addr learned my addr 172.21.15.32:0/679822101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd204a8d700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd200138d80 0x7fd20012d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1fffff700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd20009d630 0x7fd20012c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd200138d80 msgr2=0x7fd20012d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd200138d80 0x7fd20012d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd20009d630 msgr2=0x7fd20012c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd20009d630 0x7fd20012c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1ff7fe700 1 -- 172.21.15.32:0/679822101 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8009a70 con 0x7fd2000a0a30 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd204a8d700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd200138d80 0x7fd20012d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.412+0000 7fd1fffff700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd20009d630 0x7fd20012c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.413+0000 7fd1ff7fe700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 0x7fd20012cc50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40090b0 tx=0x7fd1f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.413+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4004340 con 0x7fd2000a0a30 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.413+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f401f070 con 0x7fd2000a0a30 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.413+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f40094a0 con 0x7fd2000a0a30 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.413+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f40036a0 con 0x7fd2000a0a30 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.413+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20012fcc0 con 0x7fd2000a0a30 2023-09-30T22:25:19.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.414+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1f4021030 con 0x7fd2000a0a30 2023-09-30T22:25:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.414+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd1f4040800 con 0x7fd2000a0a30 2023-09-30T22:25:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.414+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20012e3f0 con 0x7fd2000a0a30 2023-09-30T22:25:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.419+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1f4023790 con 0x7fd2000a0a30 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.443+0000 7f55368c6700 1 Processor -- start 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.443+0000 7f55368c6700 1 -- start start 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f55368c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f553009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f55368c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f553009ffb0 0x7f553009f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f55368c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55300a06b0 con 0x7f5530133b80 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f55368c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553006b920 con 0x7f553009e850 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f55368c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f553009aa30 con 0x7f553009ffb0 2023-09-30T22:25:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f5534e63700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5530133b80 0x7f5530133fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39082/0 (socket says 172.21.15.32:39082) 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f553009ffb0 0x7f553009f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f553009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f5534e63700 1 -- 172.21.15.32:0/1058699167 learned_addr learned my addr 172.21.15.32:0/1058699167 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552ffff700 1 -- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f553009ffb0 msgr2=0x7f553009f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552ffff700 1 --2- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f553009ffb0 0x7f553009f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552ffff700 1 -- 172.21.15.32:0/1058699167 >> v1:172.21.15.32:6789/0 conn(0x7f5530133b80 legacy=0x7f5530133fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552ffff700 1 -- 172.21.15.32:0/1058699167 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553006caa0 con 0x7f553009e850 2023-09-30T22:25:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552effd700 1 -- 172.21.15.32:0/1058699167 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282279414 0 0) 0x7f55300a06b0 con 0x7f5530133b80 2023-09-30T22:25:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.444+0000 7f552ffff700 1 --2- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f553009ec20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5520009870 tx=0x7f5520009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3537ff2289bf215c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f552effd700 1 -- 172.21.15.32:0/1058699167 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520011830 con 0x7f553009e850 2023-09-30T22:25:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f552effd700 1 -- 172.21.15.32:0/1058699167 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520011990 con 0x7f553009e850 2023-09-30T22:25:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f552effd700 1 -- 172.21.15.32:0/1058699167 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520011d20 con 0x7f553009e850 2023-09-30T22:25:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 -- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 msgr2=0x7f553009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 --2- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f553009ec20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5520009870 tx=0x7f5520009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 -- 172.21.15.32:0/1058699167 shutdown_connections 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 --2- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f553009ec20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 --2- 172.21.15.32:0/1058699167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f553009ffb0 0x7f553009f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 -- 172.21.15.32:0/1058699167 >> 172.21.15.32:0/1058699167 conn(0x7f55300a3f70 msgr2=0x7f55300a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 -- 172.21.15.32:0/1058699167 shutdown_connections 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.445+0000 7f55368c6700 1 -- 172.21.15.32:0/1058699167 wait complete. 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.446+0000 7f55368c6700 1 Processor -- start 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 -- start start 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f5530120630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f553009ffb0 0x7f5530120b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5530133b80 0x7f55301210b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:19.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530122720 con 0x7f553009ffb0 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5530123e20 con 0x7f553009e850 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f55368c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5530123100 con 0x7f5530133b80 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f552ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f5530120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f552f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f553009ffb0 0x7f5530120b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f552ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f5530120630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33428/0 (socket says 172.21.15.32:33428) 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f552ffff700 1 -- 172.21.15.32:0/1835171786 learned_addr learned my addr 172.21.15.32:0/1835171786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.447+0000 7f5534e63700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5530133b80 0x7f55301210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552ffff700 1 -- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5530133b80 msgr2=0x7f55301210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552ffff700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5530133b80 0x7f55301210b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552ffff700 1 -- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f553009ffb0 msgr2=0x7f5530120b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552ffff700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f553009ffb0 0x7f5530120b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552ffff700 1 -- 172.21.15.32:0/1835171786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200094a0 con 0x7f553009e850 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f5534e63700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5530133b80 0x7f55301210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552ffff700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f5530120630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5520009ed0 tx=0x7f5520004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520004240 con 0x7f553009e850 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552001f070 con 0x7f553009e850 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553006ccc0 con 0x7f553009e850 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200036a0 con 0x7f553009e850 2023-09-30T22:25:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.448+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5530124890 con 0x7f553009e850 2023-09-30T22:25:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.449+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553009cbf0 con 0x7f553009e850 2023-09-30T22:25:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.450+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5520021030 con 0x7f553009e850 2023-09-30T22:25:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.450+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5520040eb0 con 0x7f553009e850 2023-09-30T22:25:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.455+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55200199a0 con 0x7f553009e850 2023-09-30T22:25:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.627+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd2000747f0 con 0x7fd2000a0a30 2023-09-30T22:25:19.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.627+0000 7fd1fcff9700 1 -- 172.21.15.32:0/679822101 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1f40233d0 con 0x7fd2000a0a30 2023-09-30T22:25:19.628 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 msgr2=0x7fd20012cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 0x7fd20012cc50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40090b0 tx=0x7fd1f4004750 comp rx=0 tx=0).stop 2023-09-30T22:25:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 shutdown_connections 2023-09-30T22:25:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd200138d80 0x7fd20012d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd20009d630 0x7fd20012c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 --2- 172.21.15.32:0/679822101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2000a0a30 0x7fd20012cc50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 >> 172.21.15.32:0/679822101 conn(0x7fd2000a3050 msgr2=0x7fd2000a4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 shutdown_connections 2023-09-30T22:25:19.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.630+0000 7fd2064f0700 1 -- 172.21.15.32:0/679822101 wait complete. 2023-09-30T22:25:19.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.664+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f55300747f0 con 0x7f553009e850 2023-09-30T22:25:19.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.664+0000 7f552cff9700 1 -- 172.21.15.32:0/1835171786 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5520015030 con 0x7f553009e850 2023-09-30T22:25:19.664 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 msgr2=0x7f5530120630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f5530120630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5520009ed0 tx=0x7f5520004750 comp rx=0 tx=0).stop 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 shutdown_connections 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5530133b80 0x7f55301210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f553009e850 0x7f5530120630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 --2- 172.21.15.32:0/1835171786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f553009ffb0 0x7f5530120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 >> 172.21.15.32:0/1835171786 conn(0x7f55300a3f70 msgr2=0x7f55301bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:19.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 shutdown_connections 2023-09-30T22:25:19.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:19.667+0000 7f55368c6700 1 -- 172.21.15.32:0/1835171786 wait complete. 2023-09-30T22:25:19.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:19.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:20.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:20.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.144+0000 7f13f0319700 1 Processor -- start 2023-09-30T22:25:20.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.144+0000 7f13f0319700 1 -- start start 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13f0319700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 0x7f13e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13f0319700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 0x7f13e813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13f0319700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e809edd0 con 0x7f13e80a1340 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13f0319700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e809e1f0 con 0x7f13e80a0a30 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13f0319700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e809dc50 con 0x7f13e8134170 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee0b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 0x7f13e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 0x7f13e813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ed8b4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f13e80a1340 0x7f13e8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39092/0 (socket says 172.21.15.32:39092) 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 0x7f13e813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33432/0 (socket says 172.21.15.32:33432) 2023-09-30T22:25:20.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee0b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 0x7f13e80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42936/0 (socket says 172.21.15.32:42936) 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ed8b4700 1 -- 172.21.15.32:0/3886977507 learned_addr learned my addr 172.21.15.32:0/3886977507 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 -- 172.21.15.32:0/3886977507 >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 msgr2=0x7f13e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 --2- 172.21.15.32:0/3886977507 >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 0x7f13e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 -- 172.21.15.32:0/3886977507 >> v1:172.21.15.32:6789/0 conn(0x7f13e80a1340 legacy=0x7f13e8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 -- 172.21.15.32:0/3886977507 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80a1fe0 con 0x7f13e8134170 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ed0b3700 1 -- 172.21.15.32:0/3886977507 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600628157 0 0) 0x7f13e809edd0 con 0x7f13e80a1340 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee0b5700 1 --2- 172.21.15.32:0/3886977507 >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 0x7f13e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.145+0000 7f13ee8b6700 1 --2- 172.21.15.32:0/3886977507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 0x7f13e813bd30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009840 tx=0x7f13e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d6448281014a78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13ed0b3700 1 -- 172.21.15.32:0/3886977507 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011830 con 0x7f13e8134170 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13ed0b3700 1 -- 172.21.15.32:0/3886977507 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0011990 con 0x7f13e8134170 2023-09-30T22:25:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13ed0b3700 1 -- 172.21.15.32:0/3886977507 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0011d20 con 0x7f13e8134170 2023-09-30T22:25:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13f0319700 1 -- 172.21.15.32:0/3886977507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 msgr2=0x7f13e813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13f0319700 1 --2- 172.21.15.32:0/3886977507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 0x7f13e813bd30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009840 tx=0x7f13e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13f0319700 1 -- 172.21.15.32:0/3886977507 shutdown_connections 2023-09-30T22:25:20.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13f0319700 1 --2- 172.21.15.32:0/3886977507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e8134170 0x7f13e813bd30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13f0319700 1 --2- 172.21.15.32:0/3886977507 >> v2:172.21.15.32:3300/0 conn(0x7f13e80a0a30 0x7f13e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.146+0000 7f13f0319700 1 -- 172.21.15.32:0/3886977507 >> 172.21.15.32:0/3886977507 conn(0x7f13e80a6550 msgr2=0x7f13e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.147+0000 7f13f0319700 1 -- 172.21.15.32:0/3886977507 shutdown_connections 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.147+0000 7f13f0319700 1 -- 172.21.15.32:0/3886977507 wait complete. 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 Processor -- start 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 -- start start 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13e80a0a30 0x7f13e81226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e80a1340 0x7f13e8122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 0x7f13e8123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8126190 con 0x7f13e8134170 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8124400 con 0x7f13e80a1340 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13f0319700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80a0220 con 0x7f13e80a0a30 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13ee8b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 0x7f13e8123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13ed8b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e80a1340 0x7f13e8122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13ee0b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13e80a0a30 0x7f13e81226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13ee8b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 0x7f13e8123160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42942/0 (socket says 172.21.15.32:42942) 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13ee0b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13e80a0a30 0x7f13e81226e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60762/0 (socket says 172.21.15.32:60762) 2023-09-30T22:25:20.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.148+0000 7f13ee8b6700 1 -- 172.21.15.32:0/4263890639 learned_addr learned my addr 172.21.15.32:0/4263890639 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ee8b6700 1 -- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13e80a0a30 msgr2=0x7f13e81226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ee8b6700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13e80a0a30 0x7f13e81226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ee8b6700 1 -- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e80a1340 msgr2=0x7f13e8122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ee8b6700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e80a1340 0x7f13e8122c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ee8b6700 1 -- 172.21.15.32:0/4263890639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8009a70 con 0x7f13e8134170 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ed8b4700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e80a1340 0x7f13e8122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13ee8b6700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 0x7f13e8123160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009fd0 tx=0x7f13e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0004510 con 0x7f13e8134170 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e00036a0 con 0x7f13e8134170 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e00094a0 con 0x7f13e8134170 2023-09-30T22:25:20.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0003a50 con 0x7f13e8134170 2023-09-30T22:25:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.149+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e80a25e0 con 0x7f13e8134170 2023-09-30T22:25:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.150+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13e0003bb0 con 0x7f13e8134170 2023-09-30T22:25:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.150+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e809bcb0 con 0x7f13e8134170 2023-09-30T22:25:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.150+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f13e00409f0 con 0x7f13e8134170 2023-09-30T22:25:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.155+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13e001a420 con 0x7f13e8134170 2023-09-30T22:25:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 Processor -- start 2023-09-30T22:25:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 -- start start 2023-09-30T22:25:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 0x7f1df00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 0x7f1df013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df00a2a70 con 0x7f1df009d630 2023-09-30T22:25:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df00a01d0 con 0x7f1df00a0a30 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1df5f62700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df009e2b0 con 0x7f1df0138d80 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1def7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 0x7f1df00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1deffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 0x7f1df013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1deeffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1df009d630 0x7f1df00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39096/0 (socket says 172.21.15.32:39096) 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1deeffd700 1 -- 172.21.15.32:0/3919440969 learned_addr learned my addr 172.21.15.32:0/3919440969 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1deffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 0x7f1df013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33448/0 (socket says 172.21.15.32:33448) 2023-09-30T22:25:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.183+0000 7f1def7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 0x7f1df00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42946/0 (socket says 172.21.15.32:42946) 2023-09-30T22:25:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1deffff700 1 -- 172.21.15.32:0/3919440969 >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 msgr2=0x7f1df00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1deffff700 1 --2- 172.21.15.32:0/3919440969 >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 0x7f1df00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1deffff700 1 -- 172.21.15.32:0/3919440969 >> v1:172.21.15.32:6789/0 conn(0x7f1df009d630 legacy=0x7f1df00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1deffff700 1 -- 172.21.15.32:0/3919440969 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df009dc50 con 0x7f1df0138d80 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1dee7fc700 1 -- 172.21.15.32:0/3919440969 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421279210 0 0) 0x7f1df00a2a70 con 0x7f1df009d630 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1def7fe700 1 --2- 172.21.15.32:0/3919440969 >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 0x7f1df00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1deffff700 1 --2- 172.21.15.32:0/3919440969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 0x7f1df013b160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009d90 tx=0x7f1de800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab32842e5665698 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1dee7fc700 1 -- 172.21.15.32:0/3919440969 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800e070 con 0x7f1df0138d80 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1dee7fc700 1 -- 172.21.15.32:0/3919440969 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800bb20 con 0x7f1df0138d80 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.184+0000 7f1dee7fc700 1 -- 172.21.15.32:0/3919440969 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8012430 con 0x7f1df0138d80 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 -- 172.21.15.32:0/3919440969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 msgr2=0x7f1df013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 --2- 172.21.15.32:0/3919440969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 0x7f1df013b160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009d90 tx=0x7f1de800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 -- 172.21.15.32:0/3919440969 shutdown_connections 2023-09-30T22:25:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 --2- 172.21.15.32:0/3919440969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df0138d80 0x7f1df013b160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 --2- 172.21.15.32:0/3919440969 >> v2:172.21.15.32:3300/0 conn(0x7f1df00a0a30 0x7f1df00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 -- 172.21.15.32:0/3919440969 >> 172.21.15.32:0/3919440969 conn(0x7f1df00a3050 msgr2=0x7f1df00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 -- 172.21.15.32:0/3919440969 shutdown_connections 2023-09-30T22:25:20.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.185+0000 7f1df5f62700 1 -- 172.21.15.32:0/3919440969 wait complete. 2023-09-30T22:25:20.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.186+0000 7f1df5f62700 1 Processor -- start 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 -- start start 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1df009d630 0x7f1df0133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 0x7f1df0133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df0138d80 0x7f1df01340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0137170 con 0x7f1df0138d80 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0136060 con 0x7f1df00a0a30 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1df5f62700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0136520 con 0x7f1df009d630 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1deeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 0x7f1df0133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1deeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 0x7f1df0133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33450/0 (socket says 172.21.15.32:33450) 2023-09-30T22:25:20.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1deeffd700 1 -- 172.21.15.32:0/4115482730 learned_addr learned my addr 172.21.15.32:0/4115482730 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.187+0000 7f1deffff700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df0138d80 0x7f1df01340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deeffd700 1 -- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1df009d630 msgr2=0x7f1df0133650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deeffd700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1df009d630 0x7f1df0133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deeffd700 1 -- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df0138d80 msgr2=0x7f1df01340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deeffd700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df0138d80 0x7f1df01340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deeffd700 1 -- 172.21.15.32:0/4115482730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00094a0 con 0x7f1df00a0a30 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deffff700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df0138d80 0x7f1df01340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1deeffd700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 0x7f1df0133b90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c520 tx=0x7f1de400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4004c80 con 0x7f1df00a0a30 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de4004de0 con 0x7f1df00a0a30 2023-09-30T22:25:20.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de8009a70 con 0x7f1df00a0a30 2023-09-30T22:25:20.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4007310 con 0x7f1df00a0a30 2023-09-30T22:25:20.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.188+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df0136bd0 con 0x7f1df00a0a30 2023-09-30T22:25:20.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.189+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1de4007470 con 0x7f1df00a0a30 2023-09-30T22:25:20.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.189+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1de403f3c0 con 0x7f1df00a0a30 2023-09-30T22:25:20.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.189+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df00747f0 con 0x7f1df00a0a30 2023-09-30T22:25:20.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.194+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1de40123b0 con 0x7f1df00a0a30 2023-09-30T22:25:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.364+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13e8074ea0 con 0x7f13e8134170 2023-09-30T22:25:20.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.365+0000 7f13e6ffd700 1 -- 172.21.15.32:0/4263890639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13e000de00 con 0x7f13e8134170 2023-09-30T22:25:20.365 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:20.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 msgr2=0x7f13e8123160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 0x7f13e8123160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009fd0 tx=0x7f13e0004750 comp rx=0 tx=0).stop 2023-09-30T22:25:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 shutdown_connections 2023-09-30T22:25:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13e8134170 0x7f13e8123160 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13e80a0a30 0x7f13e81226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 --2- 172.21.15.32:0/4263890639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13e80a1340 0x7f13e8122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 >> 172.21.15.32:0/4263890639 conn(0x7f13e80a6550 msgr2=0x7f13e809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:20.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 shutdown_connections 2023-09-30T22:25:20.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.367+0000 7f13f0319700 1 -- 172.21.15.32:0/4263890639 wait complete. 2023-09-30T22:25:20.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:20.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.399+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1df0072980 con 0x7f1df00a0a30 2023-09-30T22:25:20.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.400+0000 7f1dd7fff700 1 -- 172.21.15.32:0/4115482730 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1de4011360 con 0x7f1df00a0a30 2023-09-30T22:25:20.400 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 msgr2=0x7f1df0133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 0x7f1df0133b90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c520 tx=0x7f1de400c830 comp rx=0 tx=0).stop 2023-09-30T22:25:20.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 shutdown_connections 2023-09-30T22:25:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1df0138d80 0x7f1df01340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1df009d630 0x7f1df0133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 --2- 172.21.15.32:0/4115482730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1df00a0a30 0x7f1df0133b90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 >> 172.21.15.32:0/4115482730 conn(0x7f1df00a3050 msgr2=0x7f1df009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 shutdown_connections 2023-09-30T22:25:20.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.402+0000 7f1df5f62700 1 -- 172.21.15.32:0/4115482730 wait complete. 2023-09-30T22:25:20.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:20.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:20.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 Processor -- start 2023-09-30T22:25:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 -- start start 2023-09-30T22:25:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 0x7f53740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5374138d80 0x7f537413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740a2a70 con 0x7f537409d630 2023-09-30T22:25:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740a01d0 con 0x7f5374138d80 2023-09-30T22:25:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.795+0000 7f537945c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537409e2b0 con 0x7f53740a0a30 2023-09-30T22:25:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5374138d80 0x7f537413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f5372ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 0x7f53740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f536a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f537409d630 0x7f53740a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51458/0 (socket says 172.21.15.32:51458) 2023-09-30T22:25:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f536a7fc700 1 -- 172.21.15.32:0/71124133 learned_addr learned my addr 172.21.15.32:0/71124133 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 --2- 172.21.15.32:0/71124133 >> v2:172.21.15.32:3300/0 conn(0x7f5374138d80 0x7f537413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54814/0 (socket says 172.21.15.32:54814) 2023-09-30T22:25:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f5372ffd700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 0x7f53740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55884/0 (socket says 172.21.15.32:55884) 2023-09-30T22:25:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 -- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 msgr2=0x7f53740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 0x7f53740a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 -- 172.21.15.32:0/71124133 >> v1:172.21.15.32:6789/0 conn(0x7f537409d630 legacy=0x7f53740a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 -- 172.21.15.32:0/71124133 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537409dc50 con 0x7f5374138d80 2023-09-30T22:25:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53727fc700 1 -- 172.21.15.32:0/71124133 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179578803 0 0) 0x7f53740a2a70 con 0x7f537409d630 2023-09-30T22:25:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f5372ffd700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 0x7f53740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53737fe700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5374138d80 0x7f537413b160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f536c009d90 tx=0x7f536c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9664d16b1050afbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53727fc700 1 -- 172.21.15.32:0/71124133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00e070 con 0x7f5374138d80 2023-09-30T22:25:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53727fc700 1 -- 172.21.15.32:0/71124133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c00bb20 con 0x7f5374138d80 2023-09-30T22:25:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.796+0000 7f53727fc700 1 -- 172.21.15.32:0/71124133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c012430 con 0x7f5374138d80 2023-09-30T22:25:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 -- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5374138d80 msgr2=0x7f537413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5374138d80 0x7f537413b160 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f536c009d90 tx=0x7f536c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 -- 172.21.15.32:0/71124133 shutdown_connections 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5374138d80 0x7f537413b160 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 --2- 172.21.15.32:0/71124133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53740a0a30 0x7f53740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 -- 172.21.15.32:0/71124133 >> 172.21.15.32:0/71124133 conn(0x7f53740a3050 msgr2=0x7f53740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 -- 172.21.15.32:0/71124133 shutdown_connections 2023-09-30T22:25:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.797+0000 7f537945c700 1 -- 172.21.15.32:0/71124133 wait complete. 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.798+0000 7f537945c700 1 Processor -- start 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.798+0000 7f537945c700 1 -- start start 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f537945c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f537409d630 0x7f5374133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f537945c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 0x7f5374133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f537945c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5374138d80 0x7f5374134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f537945c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53741361b0 con 0x7f53740a0a30 2023-09-30T22:25:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f537945c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5374136ed0 con 0x7f5374138d80 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f537945c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53740a4b60 con 0x7f537409d630 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 0x7f5374133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 0x7f5374133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54826/0 (socket says 172.21.15.32:54826) 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f5372ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f537409d630 0x7f5374133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 -- 172.21.15.32:0/3502244112 learned_addr learned my addr 172.21.15.32:0/3502244112 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f53737fe700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5374138d80 0x7f5374134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 -- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f537409d630 msgr2=0x7f5374133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f537409d630 0x7f5374133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 -- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5374138d80 msgr2=0x7f5374134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5374138d80 0x7f5374134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.799+0000 7f536a7fc700 1 -- 172.21.15.32:0/3502244112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53640094a0 con 0x7f53740a0a30 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.800+0000 7f536a7fc700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 0x7f5374133bd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c520 tx=0x7f535c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.800+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c004c80 con 0x7f53740a0a30 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.800+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c004de0 con 0x7f53740a0a30 2023-09-30T22:25:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.800+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c009a70 con 0x7f53740a0a30 2023-09-30T22:25:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.800+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c007310 con 0x7f53740a0a30 2023-09-30T22:25:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.800+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5374136560 con 0x7f53740a0a30 2023-09-30T22:25:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.801+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f535c007470 con 0x7f53740a0a30 2023-09-30T22:25:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.801+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f535c03f2a0 con 0x7f53740a0a30 2023-09-30T22:25:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.801+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5374137a00 con 0x7f53740a0a30 2023-09-30T22:25:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.806+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f535c0123b0 con 0x7f53740a0a30 2023-09-30T22:25:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.832+0000 7f0535538700 1 Processor -- start 2023-09-30T22:25:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.833+0000 7f0535538700 1 -- start start 2023-09-30T22:25:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.833+0000 7f0535538700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f053009d6f0 0x7f053009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.833+0000 7f0535538700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 0x7f0530130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.833+0000 7f0535538700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053009f130 con 0x7f053009e410 2023-09-30T22:25:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.833+0000 7f0535538700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053009f520 con 0x7f053009d6f0 2023-09-30T22:25:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.833+0000 7f0535538700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053009e930 con 0x7f053009e000 2023-09-30T22:25:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 0x7f0530130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 0x7f0530130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55904/0 (socket says 172.21.15.32:55904) 2023-09-30T22:25:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 -- 172.21.15.32:0/3373565836 learned_addr learned my addr 172.21.15.32:0/3373565836 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052f7fe700 1 --1- 172.21.15.32:0/3373565836 >> v1:172.21.15.32:6789/0 conn(0x7f053009e410 0x7f0530130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51468/0 (socket says 172.21.15.32:51468) 2023-09-30T22:25:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f053009d6f0 0x7f053009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 -- 172.21.15.32:0/3373565836 >> v2:172.21.15.32:3300/0 conn(0x7f053009d6f0 msgr2=0x7f053009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 --2- 172.21.15.32:0/3373565836 >> v2:172.21.15.32:3300/0 conn(0x7f053009d6f0 0x7f053009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 -- 172.21.15.32:0/3373565836 >> v1:172.21.15.32:6789/0 conn(0x7f053009e410 legacy=0x7f0530130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 -- 172.21.15.32:0/3373565836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053009b2d0 con 0x7f053009e000 2023-09-30T22:25:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052effd700 1 --2- 172.21.15.32:0/3373565836 >> v2:172.21.15.32:3300/0 conn(0x7f053009d6f0 0x7f053009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052e7fc700 1 --2- 172.21.15.32:0/3373565836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 0x7f0530130660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05240098f0 tx=0x7f0524009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb9e5c52252fa5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052dffb700 1 -- 172.21.15.32:0/3373565836 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524011830 con 0x7f053009e000 2023-09-30T22:25:20.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052dffb700 1 -- 172.21.15.32:0/3373565836 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524011990 con 0x7f053009e000 2023-09-30T22:25:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.834+0000 7f052dffb700 1 -- 172.21.15.32:0/3373565836 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524011d20 con 0x7f053009e000 2023-09-30T22:25:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 -- 172.21.15.32:0/3373565836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 msgr2=0x7f0530130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 --2- 172.21.15.32:0/3373565836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 0x7f0530130660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05240098f0 tx=0x7f0524009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 -- 172.21.15.32:0/3373565836 shutdown_connections 2023-09-30T22:25:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 --2- 172.21.15.32:0/3373565836 >> v2:172.21.15.32:3300/0 conn(0x7f053009d6f0 0x7f053009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 --2- 172.21.15.32:0/3373565836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e000 0x7f0530130660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 -- 172.21.15.32:0/3373565836 >> 172.21.15.32:0/3373565836 conn(0x7f0530066f50 msgr2=0x7f053009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 -- 172.21.15.32:0/3373565836 shutdown_connections 2023-09-30T22:25:20.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.835+0000 7f0535538700 1 -- 172.21.15.32:0/3373565836 wait complete. 2023-09-30T22:25:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 Processor -- start 2023-09-30T22:25:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 -- start start 2023-09-30T22:25:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f053009d6f0 0x7f053011f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053009e000 0x7f053011f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e410 0x7f053011fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530122ce0 con 0x7f053009e000 2023-09-30T22:25:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0530120dd0 con 0x7f053009e410 2023-09-30T22:25:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.837+0000 7f0535538700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f053006e8d0 con 0x7f053009d6f0 2023-09-30T22:25:20.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053009e000 0x7f053011f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e410 0x7f053011fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f053009d6f0 0x7f053011f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053009e000 0x7f053011f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54846/0 (socket says 172.21.15.32:54846) 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052e7fc700 1 -- 172.21.15.32:0/2507615314 learned_addr learned my addr 172.21.15.32:0/2507615314 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 -- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f053009d6f0 msgr2=0x7f053011f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f053009d6f0 0x7f053011f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 -- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053009e000 msgr2=0x7f053011f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053009e000 0x7f053011f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 -- 172.21.15.32:0/2507615314 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520009a70 con 0x7f053009e410 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f052f7fe700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e410 0x7f053011fc40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0528001470 tx=0x7f0528001780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528005cb0 con 0x7f053009e410 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0528005e10 con 0x7f053009e410 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524009550 con 0x7f053009e410 2023-09-30T22:25:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.838+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528007270 con 0x7f053009e410 2023-09-30T22:25:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.839+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053012fce0 con 0x7f053009e410 2023-09-30T22:25:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.839+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05280073d0 con 0x7f053009e410 2023-09-30T22:25:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.840+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f052803f720 con 0x7f053009e410 2023-09-30T22:25:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.840+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053009dcd0 con 0x7f053009e410 2023-09-30T22:25:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:20.844+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0528007680 con 0x7f053009e410 2023-09-30T22:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.014+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53740747f0 con 0x7f53740a0a30 2023-09-30T22:25:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.014+0000 7f536bfff700 1 -- 172.21.15.32:0/3502244112 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f535c011360 con 0x7f53740a0a30 2023-09-30T22:25:21.014 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 msgr2=0x7f5374133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 0x7f5374133bd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c520 tx=0x7f535c00c830 comp rx=0 tx=0).stop 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 shutdown_connections 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5374138d80 0x7f5374134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f537409d630 0x7f5374133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 --2- 172.21.15.32:0/3502244112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53740a0a30 0x7f5374133bd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 >> 172.21.15.32:0/3502244112 conn(0x7f53740a3050 msgr2=0x7f537413af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 shutdown_connections 2023-09-30T22:25:21.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.017+0000 7f537945c700 1 -- 172.21.15.32:0/3502244112 wait complete. 2023-09-30T22:25:21.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.054+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0530122650 con 0x7f053009e410 2023-09-30T22:25:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.054+0000 7f05177fe700 1 -- 172.21.15.32:0/2507615314 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f052800f3c0 con 0x7f053009e410 2023-09-30T22:25:21.055 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e410 msgr2=0x7f053011fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e410 0x7f053011fc40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0528001470 tx=0x7f0528001780 comp rx=0 tx=0).stop 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 shutdown_connections 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f053009e410 0x7f053011fc40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f053009d6f0 0x7f053011f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 --2- 172.21.15.32:0/2507615314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f053009e000 0x7f053011f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 >> 172.21.15.32:0/2507615314 conn(0x7f0530066f50 msgr2=0x7f0530070120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 shutdown_connections 2023-09-30T22:25:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.057+0000 7f0535538700 1 -- 172.21.15.32:0/2507615314 wait complete. 2023-09-30T22:25:21.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:21.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:21.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:21.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 Processor -- start 2023-09-30T22:25:21.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 -- start start 2023-09-30T22:25:21.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a1510 0x7fca08138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca08139250 0x7fca0813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0809edd0 con 0x7fca080a0a30 2023-09-30T22:25:21.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080a18e0 con 0x7fca080a1510 2023-09-30T22:25:21.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca0df2c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca0809e1f0 con 0x7fca08139250 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca06ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a1510 0x7fca08138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca077fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca080a0a30 0x7fca080a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51484/0 (socket says 172.21.15.32:51484) 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca077fe700 1 -- 172.21.15.32:0/1366649246 learned_addr learned my addr 172.21.15.32:0/1366649246 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.532+0000 7fca07fff700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca08139250 0x7fca0813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca06ffd700 1 -- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca08139250 msgr2=0x7fca0813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca067fc700 1 -- 172.21.15.32:0/1366649246 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728623842 0 0) 0x7fca0809edd0 con 0x7fca080a0a30 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca06ffd700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca08139250 0x7fca0813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca06ffd700 1 -- 172.21.15.32:0/1366649246 >> v1:172.21.15.32:6789/0 conn(0x7fca080a0a30 legacy=0x7fca080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca06ffd700 1 -- 172.21.15.32:0/1366649246 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0809dc50 con 0x7fca080a1510 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca07fff700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca08139250 0x7fca0813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca06ffd700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a1510 0x7fca08138d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009870 tx=0x7fc9fc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b89113a598469c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca067fc700 1 -- 172.21.15.32:0/1366649246 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc011830 con 0x7fca080a1510 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca067fc700 1 -- 172.21.15.32:0/1366649246 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc011990 con 0x7fca080a1510 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.533+0000 7fca067fc700 1 -- 172.21.15.32:0/1366649246 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc011d20 con 0x7fca080a1510 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 -- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a1510 msgr2=0x7fca08138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a1510 0x7fca08138d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009870 tx=0x7fc9fc009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 -- 172.21.15.32:0/1366649246 shutdown_connections 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca08139250 0x7fca0813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 --2- 172.21.15.32:0/1366649246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a1510 0x7fca08138d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 -- 172.21.15.32:0/1366649246 >> 172.21.15.32:0/1366649246 conn(0x7fca080a6550 msgr2=0x7fca080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 -- 172.21.15.32:0/1366649246 shutdown_connections 2023-09-30T22:25:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.534+0000 7fca0df2c700 1 -- 172.21.15.32:0/1366649246 wait complete. 2023-09-30T22:25:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.535+0000 7fca0df2c700 1 Processor -- start 2023-09-30T22:25:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.535+0000 7fca0df2c700 1 -- start start 2023-09-30T22:25:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca0df2c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 0x7fca081227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca0df2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca080a1510 0x7fca08122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca0df2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08139250 0x7fca08123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca0df2c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca081380f0 con 0x7fca08139250 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca0df2c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080a0220 con 0x7fca080a0a30 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca0df2c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca081386d0 con 0x7fca080a1510 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 0x7fca081227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 0x7fca081227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55924/0 (socket says 172.21.15.32:55924) 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca077fe700 1 -- 172.21.15.32:0/2829948149 learned_addr learned my addr 172.21.15.32:0/2829948149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08139250 0x7fca08123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca06ffd700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca080a1510 0x7fca08122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 -- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca080a1510 msgr2=0x7fca08122d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca080a1510 0x7fca08122d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 -- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 msgr2=0x7fca081227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 0x7fca081227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 -- 172.21.15.32:0/2829948149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca000094a0 con 0x7fca08139250 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca077fe700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 0x7fca081227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.536+0000 7fca07fff700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08139250 0x7fca08123260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fca0000e910 tx=0x7fca0000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.537+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00004020 con 0x7fca08139250 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.537+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca000041a0 con 0x7fca08139250 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.537+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc0094a0 con 0x7fca08139250 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.537+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00004550 con 0x7fca08139250 2023-09-30T22:25:21.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.537+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca08124be0 con 0x7fca08139250 2023-09-30T22:25:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.538+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca08125cc0 con 0x7fca08139250 2023-09-30T22:25:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.538+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca000036a0 con 0x7fca08139250 2023-09-30T22:25:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.538+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca0003f9c0 con 0x7fca08139250 2023-09-30T22:25:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.543+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca00003cd0 con 0x7fca08139250 2023-09-30T22:25:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.569+0000 7fab487ca700 1 Processor -- start 2023-09-30T22:25:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.569+0000 7fab487ca700 1 -- start start 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab487ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab400a1510 0x7fab40138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab487ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab487ca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4009edd0 con 0x7fab400a0a30 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab487ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab400a18e0 con 0x7fab400a1510 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab487ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4009e1f0 con 0x7fab40139250 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab46d67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab45d65700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab400a1510 0x7fab40138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab46d67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55926/0 (socket says 172.21.15.32:55926) 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab46d67700 1 -- 172.21.15.32:0/2293766753 learned_addr learned my addr 172.21.15.32:0/2293766753 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab46d67700 1 -- 172.21.15.32:0/2293766753 >> v2:172.21.15.32:3300/0 conn(0x7fab400a1510 msgr2=0x7fab40138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab46d67700 1 --2- 172.21.15.32:0/2293766753 >> v2:172.21.15.32:3300/0 conn(0x7fab400a1510 0x7fab40138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab46d67700 1 -- 172.21.15.32:0/2293766753 >> v1:172.21.15.32:6789/0 conn(0x7fab400a0a30 legacy=0x7fab400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.571+0000 7fab46d67700 1 -- 172.21.15.32:0/2293766753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4009dc50 con 0x7fab40139250 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.570+0000 7fab45564700 1 -- 172.21.15.32:0/2293766753 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487482820 0 0) 0x7fab4009edd0 con 0x7fab400a0a30 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.571+0000 7fab45d65700 1 --2- 172.21.15.32:0/2293766753 >> v2:172.21.15.32:3300/0 conn(0x7fab400a1510 0x7fab40138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.571+0000 7fab46d67700 1 --2- 172.21.15.32:0/2293766753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4013b630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab38009e50 tx=0x7fab3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f68c0758f1145 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.571+0000 7fab45564700 1 -- 172.21.15.32:0/2293766753 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800e070 con 0x7fab40139250 2023-09-30T22:25:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.571+0000 7fab45564700 1 -- 172.21.15.32:0/2293766753 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3800b910 con 0x7fab40139250 2023-09-30T22:25:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.571+0000 7fab45564700 1 -- 172.21.15.32:0/2293766753 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800bcc0 con 0x7fab40139250 2023-09-30T22:25:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 -- 172.21.15.32:0/2293766753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 msgr2=0x7fab4013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 --2- 172.21.15.32:0/2293766753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4013b630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab38009e50 tx=0x7fab3800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 -- 172.21.15.32:0/2293766753 shutdown_connections 2023-09-30T22:25:21.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 --2- 172.21.15.32:0/2293766753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4013b630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 --2- 172.21.15.32:0/2293766753 >> v2:172.21.15.32:3300/0 conn(0x7fab400a1510 0x7fab40138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 -- 172.21.15.32:0/2293766753 >> 172.21.15.32:0/2293766753 conn(0x7fab400a6550 msgr2=0x7fab400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 -- 172.21.15.32:0/2293766753 shutdown_connections 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.572+0000 7fab487ca700 1 -- 172.21.15.32:0/2293766753 wait complete. 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.573+0000 7fab487ca700 1 Processor -- start 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.573+0000 7fab487ca700 1 -- start start 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab487ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 0x7fab4011e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab487ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400a1510 0x7fab4011ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab487ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4011efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:21.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab487ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40121b90 con 0x7fab400a1510 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab487ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40121050 con 0x7fab40139250 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab487ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab400a0220 con 0x7fab400a0a30 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 0x7fab4011e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 0x7fab4011e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53388/0 (socket says 172.21.15.32:53388) 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 -- 172.21.15.32:0/1728617384 learned_addr learned my addr 172.21.15.32:0/1728617384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab45d65700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400a1510 0x7fab4011ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46d67700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4011efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 -- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 msgr2=0x7fab4011efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4011efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 -- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400a1510 msgr2=0x7fab4011ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400a1510 0x7fab4011ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab46566700 1 -- 172.21.15.32:0/1728617384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340094a0 con 0x7fab400a0a30 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.574+0000 7fab45d65700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400a1510 0x7fab4011ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.575+0000 7fab46566700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 0x7fab4011e530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c520 tx=0x7fab3000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.575+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300054e0 con 0x7fab400a0a30 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.575+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab30003680 con 0x7fab400a0a30 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.575+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab38009b30 con 0x7fab400a0a30 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.575+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300039f0 con 0x7fab400a0a30 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.575+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab40120930 con 0x7fab400a0a30 2023-09-30T22:25:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.576+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab30003b50 con 0x7fab400a0a30 2023-09-30T22:25:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.576+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab3003ea10 con 0x7fab400a0a30 2023-09-30T22:25:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.576+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab40074ea0 con 0x7fab400a0a30 2023-09-30T22:25:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.581+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab3001b3b0 con 0x7fab400a0a30 2023-09-30T22:25:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.750+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fca080a1d00 con 0x7fca08139250 2023-09-30T22:25:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.750+0000 7fc9effff700 1 -- 172.21.15.32:0/2829948149 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca00029090 con 0x7fca08139250 2023-09-30T22:25:21.751 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08139250 msgr2=0x7fca08123260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08139250 0x7fca08123260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fca0000e910 tx=0x7fca0000ec20 comp rx=0 tx=0).stop 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 shutdown_connections 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca08139250 0x7fca08123260 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca080a0a30 0x7fca081227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 --2- 172.21.15.32:0/2829948149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca080a1510 0x7fca08122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 >> 172.21.15.32:0/2829948149 conn(0x7fca080a6550 msgr2=0x7fca0809ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 shutdown_connections 2023-09-30T22:25:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.753+0000 7fca0df2c700 1 -- 172.21.15.32:0/2829948149 wait complete. 2023-09-30T22:25:21.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.789+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fab400a28e0 con 0x7fab400a0a30 2023-09-30T22:25:21.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.790+0000 7fab3effd700 1 -- 172.21.15.32:0/1728617384 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab30017020 con 0x7fab400a0a30 2023-09-30T22:25:21.790 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 msgr2=0x7fab4011e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 0x7fab4011e530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c520 tx=0x7fab3000c830 comp rx=0 tx=0).stop 2023-09-30T22:25:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 shutdown_connections 2023-09-30T22:25:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab40139250 0x7fab4011efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab400a0a30 0x7fab4011e530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 --2- 172.21.15.32:0/1728617384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab400a1510 0x7fab4011ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 >> 172.21.15.32:0/1728617384 conn(0x7fab400a6550 msgr2=0x7fab400a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 shutdown_connections 2023-09-30T22:25:21.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:21.792+0000 7fab487ca700 1 -- 172.21.15.32:0/1728617384 wait complete. 2023-09-30T22:25:21.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:22.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:22.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.183+0000 7f8d1c342700 1 Processor -- start 2023-09-30T22:25:22.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.183+0000 7f8d1c342700 1 -- start start 2023-09-30T22:25:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.183+0000 7f8d1c342700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409d6f0 0x7f8d1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.183+0000 7f8d1c342700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d1409e000 0x7f8d141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1c342700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 0x7f8d14133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1c342700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1406b460 con 0x7f8d1409e000 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1c342700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1409b120 con 0x7f8d1409d6f0 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1c342700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1409a7e0 con 0x7f8d1409e410 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 0x7f8d14133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a0de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409d6f0 0x7f8d1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d198dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d1409e000 0x7f8d141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 0x7f8d14133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53392/0 (socket says 172.21.15.32:53392) 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 -- 172.21.15.32:0/2558734430 learned_addr learned my addr 172.21.15.32:0/2558734430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 -- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409d6f0 msgr2=0x7f8d1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 --2- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409d6f0 0x7f8d1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 -- 172.21.15.32:0/2558734430 >> v2:172.21.15.32:3300/0 conn(0x7f8d1409e000 msgr2=0x7f8d141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 --2- 172.21.15.32:0/2558734430 >> v2:172.21.15.32:3300/0 conn(0x7f8d1409e000 0x7f8d141339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 -- 172.21.15.32:0/2558734430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1409a500 con 0x7f8d1409e410 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d198dd700 1 --2- 172.21.15.32:0/2558734430 >> v2:172.21.15.32:3300/0 conn(0x7f8d1409e000 0x7f8d141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a0de700 1 --2- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409d6f0 0x7f8d1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d1a8df700 1 --2- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 0x7f8d14133f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009e80 tx=0x7f8d0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5f99fa7d6e13be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.184+0000 7f8d190dc700 1 -- 172.21.15.32:0/2558734430 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00e070 con 0x7f8d1409e410 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d190dc700 1 -- 172.21.15.32:0/2558734430 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c00b8d0 con 0x7f8d1409e410 2023-09-30T22:25:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d190dc700 1 -- 172.21.15.32:0/2558734430 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00bc80 con 0x7f8d1409e410 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 -- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 msgr2=0x7f8d14133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 --2- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 0x7f8d14133f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009e80 tx=0x7f8d0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 -- 172.21.15.32:0/2558734430 shutdown_connections 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 --2- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409d6f0 0x7f8d1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 --2- 172.21.15.32:0/2558734430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409e410 0x7f8d14133f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 --2- 172.21.15.32:0/2558734430 >> v2:172.21.15.32:3300/0 conn(0x7f8d1409e000 0x7f8d141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 -- 172.21.15.32:0/2558734430 >> 172.21.15.32:0/2558734430 conn(0x7f8d14066f50 msgr2=0x7f8d1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 -- 172.21.15.32:0/2558734430 shutdown_connections 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.185+0000 7f8d1c342700 1 -- 172.21.15.32:0/2558734430 wait complete. 2023-09-30T22:25:22.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.187+0000 7f8d1c342700 1 Processor -- start 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.187+0000 7f8d1c342700 1 -- start start 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1c342700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409d6f0 0x7f8d141c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1c342700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 0x7f8d141c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1c342700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1409e410 0x7f8d141c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1c342700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d141a8170 con 0x7f8d1409e410 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1c342700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d141a6460 con 0x7f8d1409e000 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1c342700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140a2000 con 0x7f8d1409d6f0 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 0x7f8d141c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 0x7f8d141c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55942/0 (socket says 172.21.15.32:55942) 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 -- 172.21.15.32:0/219982730 learned_addr learned my addr 172.21.15.32:0/219982730 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:22.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1a0de700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409d6f0 0x7f8d141c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d1a8df700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1409e410 0x7f8d141c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 -- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409d6f0 msgr2=0x7f8d141c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409d6f0 0x7f8d141c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 -- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1409e410 msgr2=0x7f8d141c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1409e410 0x7f8d141c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 -- 172.21.15.32:0/219982730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100094a0 con 0x7f8d1409e000 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.188+0000 7f8d198dd700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 0x7f8d141c85f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ebb0 tx=0x7f8d0400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.189+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400c850 con 0x7f8d1409e000 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.189+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0400c9b0 con 0x7f8d1409e000 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.189+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400cd40 con 0x7f8d1409e000 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.189+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d040094a0 con 0x7f8d1409e000 2023-09-30T22:25:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.189+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c009b30 con 0x7f8d1409e000 2023-09-30T22:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.190+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d14074ea0 con 0x7f8d1409e000 2023-09-30T22:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.190+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d04013470 con 0x7f8d1409e000 2023-09-30T22:25:22.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.190+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8d0403fa70 con 0x7f8d1409e000 2023-09-30T22:25:22.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.195+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d04041e60 con 0x7f8d1409e000 2023-09-30T22:25:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.222+0000 7fe0a3258700 1 Processor -- start 2023-09-30T22:25:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.222+0000 7fe0a3258700 1 -- start start 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.222+0000 7fe0a3258700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe09c0a0a30 0x7fe09c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.222+0000 7fe0a3258700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c09d630 0x7fe09c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a3258700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09c0a2a70 con 0x7fe09c133930 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a3258700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c0a01d0 con 0x7fe09c0a0a30 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a3258700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c09e2b0 con 0x7fe09c09d630 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a0ff4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe09c0a0a30 0x7fe09c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe09bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c09d630 0x7fe09c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a17f5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe09c133930 0x7fe09c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51512/0 (socket says 172.21.15.32:51512) 2023-09-30T22:25:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a17f5700 1 -- 172.21.15.32:0/1298535075 learned_addr learned my addr 172.21.15.32:0/1298535075 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a0ff4700 1 -- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c09d630 msgr2=0x7fe09c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a0ff4700 1 --2- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c09d630 0x7fe09c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a0ff4700 1 -- 172.21.15.32:0/1298535075 >> v1:172.21.15.32:6789/0 conn(0x7fe09c133930 legacy=0x7fe09c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a0ff4700 1 -- 172.21.15.32:0/1298535075 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c09dc50 con 0x7fe09c0a0a30 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe09b7fe700 1 -- 172.21.15.32:0/1298535075 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452364768 0 0) 0x7fe09c0a2a70 con 0x7fe09c133930 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe09bfff700 1 --2- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c09d630 0x7fe09c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe0a0ff4700 1 --2- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c0a0a30 0x7fe09c0a0e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe090009840 tx=0x7fe090009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c087106844734c09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.223+0000 7fe09b7fe700 1 -- 172.21.15.32:0/1298535075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090011830 con 0x7fe09c0a0a30 2023-09-30T22:25:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe09b7fe700 1 -- 172.21.15.32:0/1298535075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe090011990 con 0x7fe09c0a0a30 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe09b7fe700 1 -- 172.21.15.32:0/1298535075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090011d20 con 0x7fe09c0a0a30 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 -- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c0a0a30 msgr2=0x7fe09c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 --2- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c0a0a30 0x7fe09c0a0e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe090009840 tx=0x7fe090009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 -- 172.21.15.32:0/1298535075 shutdown_connections 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 --2- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c0a0a30 0x7fe09c0a0e00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 --2- 172.21.15.32:0/1298535075 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c09d630 0x7fe09c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 -- 172.21.15.32:0/1298535075 >> 172.21.15.32:0/1298535075 conn(0x7fe09c0a3050 msgr2=0x7fe09c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 -- 172.21.15.32:0/1298535075 shutdown_connections 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.224+0000 7fe0a3258700 1 -- 172.21.15.32:0/1298535075 wait complete. 2023-09-30T22:25:22.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.225+0000 7fe0a3258700 1 Processor -- start 2023-09-30T22:25:22.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.225+0000 7fe0a3258700 1 -- start start 2023-09-30T22:25:22.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a3258700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c09d630 0x7fe09c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a3258700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 0x7fe09c1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a3258700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe09c133930 0x7fe09c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a3258700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c120620 con 0x7fe09c09d630 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a3258700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0a42c0 con 0x7fe09c133930 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a3258700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0a0410 con 0x7fe09c0a0a30 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe09bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 0x7fe09c1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe09bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 0x7fe09c1c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53428/0 (socket says 172.21.15.32:53428) 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe09bfff700 1 -- 172.21.15.32:0/2466282173 learned_addr learned my addr 172.21.15.32:0/2466282173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a17f5700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe09c133930 0x7fe09c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a0ff4700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c09d630 0x7fe09c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a17f5700 1 -- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 msgr2=0x7fe09c1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a17f5700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 0x7fe09c1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a17f5700 1 -- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c09d630 msgr2=0x7fe09c1c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a17f5700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c09d630 0x7fe09c1c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe0a17f5700 1 -- 172.21.15.32:0/2466282173 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880094a0 con 0x7fe09c133930 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.226+0000 7fe09bfff700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 0x7fe09c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.227+0000 7fe0a17f5700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe09c133930 0x7fe09c1c8900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe09400cc10 tx=0x7fe09400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.227+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400aa00 con 0x7fe09c133930 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.227+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09400ab80 con 0x7fe09c133930 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.227+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0900094a0 con 0x7fe09c133930 2023-09-30T22:25:22.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.227+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940070d0 con 0x7fe09c133930 2023-09-30T22:25:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.227+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1334c0 con 0x7fe09c133930 2023-09-30T22:25:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.228+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe094007230 con 0x7fe09c133930 2023-09-30T22:25:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.228+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe09403f400 con 0x7fe09c133930 2023-09-30T22:25:22.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.228+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c0747f0 con 0x7fe09c133930 2023-09-30T22:25:22.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.233+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe094018080 con 0x7fe09c133930 2023-09-30T22:25:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.402+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d141a7fd0 con 0x7f8d1409e000 2023-09-30T22:25:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.402+0000 7f8d0affd700 1 -- 172.21.15.32:0/219982730 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d141a7fd0 con 0x7f8d1409e000 2023-09-30T22:25:22.402 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:22.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 msgr2=0x7f8d141c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 0x7f8d141c85f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ebb0 tx=0x7f8d0400eec0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 shutdown_connections 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d1409e410 0x7f8d141c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d1409d6f0 0x7f8d141c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 --2- 172.21.15.32:0/219982730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d1409e000 0x7f8d141c85f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.404+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 >> 172.21.15.32:0/219982730 conn(0x7f8d14066f50 msgr2=0x7f8d1406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.405+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 shutdown_connections 2023-09-30T22:25:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.405+0000 7f8d1c342700 1 -- 172.21.15.32:0/219982730 wait complete. 2023-09-30T22:25:22.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.437+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe09c072980 con 0x7fe09c133930 2023-09-30T22:25:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.438+0000 7fe0997fa700 1 -- 172.21.15.32:0/2466282173 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0940074e0 con 0x7fe09c133930 2023-09-30T22:25:22.438 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe09c133930 msgr2=0x7fe09c1c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe09c133930 0x7fe09c1c8900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe09400cc10 tx=0x7fe09400cf20 comp rx=0 tx=0).stop 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 shutdown_connections 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe09c133930 0x7fe09c1c8900 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe09c09d630 0x7fe09c1c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 --2- 172.21.15.32:0/2466282173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe09c0a0a30 0x7fe09c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 >> 172.21.15.32:0/2466282173 conn(0x7fe09c0a3050 msgr2=0x7fe09c1bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 shutdown_connections 2023-09-30T22:25:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.440+0000 7fe0a3258700 1 -- 172.21.15.32:0/2466282173 wait complete. 2023-09-30T22:25:22.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:22.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:22.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:22.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.916+0000 7fe2875c6700 1 Processor -- start 2023-09-30T22:25:22.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.916+0000 7fe2875c6700 1 -- start start 2023-09-30T22:25:22.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.916+0000 7fe2875c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe2875c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe28009f240 0x7fe2801317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe2875c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f650 0x7fe280131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe2875c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe280070940 con 0x7fe28009f240 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe2875c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28006fd60 con 0x7fe28009e930 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe2875c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe28006fa80 con 0x7fe28009f650 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55956/0 (socket says 172.21.15.32:55956) 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 -- 172.21.15.32:0/1293233240 learned_addr learned my addr 172.21.15.32:0/1293233240 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe284b61700 1 --2- 172.21.15.32:0/1293233240 >> v2:172.21.15.32:3300/0 conn(0x7fe28009f240 0x7fe2801317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285b63700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f650 0x7fe280131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 -- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f650 msgr2=0x7fe280131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f650 0x7fe280131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 -- 172.21.15.32:0/1293233240 >> v2:172.21.15.32:3300/0 conn(0x7fe28009f240 msgr2=0x7fe2801317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 --2- 172.21.15.32:0/1293233240 >> v2:172.21.15.32:3300/0 conn(0x7fe28009f240 0x7fe2801317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 -- 172.21.15.32:0/1293233240 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28009fb60 con 0x7fe28009e930 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285b63700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f650 0x7fe280131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe284b61700 1 --2- 172.21.15.32:0/1293233240 >> v2:172.21.15.32:3300/0 conn(0x7fe28009f240 0x7fe2801317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.917+0000 7fe285362700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28009ed00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0098f0 tx=0x7fe27c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bffd175202321d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe277fff700 1 -- 172.21.15.32:0/1293233240 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c011830 con 0x7fe28009e930 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe277fff700 1 -- 172.21.15.32:0/1293233240 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c011990 con 0x7fe28009e930 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe277fff700 1 -- 172.21.15.32:0/1293233240 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c011d20 con 0x7fe28009e930 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 -- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 msgr2=0x7fe28009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28009ed00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0098f0 tx=0x7fe27c009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 -- 172.21.15.32:0/1293233240 shutdown_connections 2023-09-30T22:25:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28009ed00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 --2- 172.21.15.32:0/1293233240 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f650 0x7fe280131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 --2- 172.21.15.32:0/1293233240 >> v2:172.21.15.32:3300/0 conn(0x7fe28009f240 0x7fe2801317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 -- 172.21.15.32:0/1293233240 >> 172.21.15.32:0/1293233240 conn(0x7fe2800a4210 msgr2=0x7fe2800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 -- 172.21.15.32:0/1293233240 shutdown_connections 2023-09-30T22:25:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.918+0000 7fe2875c6700 1 -- 172.21.15.32:0/1293233240 wait complete. 2023-09-30T22:25:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.919+0000 7fe2875c6700 1 Processor -- start 2023-09-30T22:25:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 -- start start 2023-09-30T22:25:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f240 0x7fe28006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 0x7fe28006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801a9a10 con 0x7fe28009f650 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801a89b0 con 0x7fe28009e930 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe2875c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2800a35c0 con 0x7fe28009f240 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285b63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 0x7fe28006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55968/0 (socket says 172.21.15.32:55968) 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 -- 172.21.15.32:0/875712468 learned_addr learned my addr 172.21.15.32:0/875712468 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285b63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 0x7fe28006c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54936/0 (socket says 172.21.15.32:54936) 2023-09-30T22:25:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe284b61700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f240 0x7fe28006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 -- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f240 msgr2=0x7fe28006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f240 0x7fe28006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 -- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 msgr2=0x7fe28006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.920+0000 7fe285362700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 0x7fe28006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe285362700 1 -- 172.21.15.32:0/875712468 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2700094a0 con 0x7fe28009e930 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe285b63700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 0x7fe28006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe284b61700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f240 0x7fe28006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe285362700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28006b880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c004240 con 0x7fe28009e930 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c01f070 con 0x7fe28009e930 2023-09-30T22:25:22.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278009a70 con 0x7fe28009e930 2023-09-30T22:25:22.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0045c0 con 0x7fe28009e930 2023-09-30T22:25:22.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.921+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c009550 con 0x7fe28009e930 2023-09-30T22:25:22.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.922+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe27c021030 con 0x7fe28009e930 2023-09-30T22:25:22.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.922+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe27c0407c0 con 0x7fe28009e930 2023-09-30T22:25:22.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.922+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2801a0b30 con 0x7fe28009e930 2023-09-30T22:25:22.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.927+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe27c015080 con 0x7fe28009e930 2023-09-30T22:25:22.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.955+0000 7f4d01327700 1 Processor -- start 2023-09-30T22:25:22.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.955+0000 7f4d01327700 1 -- start start 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4d01327700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4d01327700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 0x7f4cfc13b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4d01327700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc09ea10 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4d01327700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc09e210 con 0x7f4cfc0a14a0 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4d01327700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0a26f0 con 0x7f4cfc09d5c0 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4cfa7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4cfb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 0x7f4cfc13b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4cfaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4cfc0a09c0 0x7f4cfc0a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51520/0 (socket says 172.21.15.32:51520) 2023-09-30T22:25:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4cfa7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc138d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55984/0 (socket says 172.21.15.32:55984) 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4cfb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 0x7f4cfc13b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54952/0 (socket says 172.21.15.32:54952) 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.956+0000 7f4cfaffd700 1 -- 172.21.15.32:0/905779310 learned_addr learned my addr 172.21.15.32:0/905779310 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cfa7fc700 1 -- 172.21.15.32:0/905779310 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 msgr2=0x7f4cfc13b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cfa7fc700 1 --2- 172.21.15.32:0/905779310 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 0x7f4cfc13b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cfa7fc700 1 -- 172.21.15.32:0/905779310 >> v1:172.21.15.32:6789/0 conn(0x7f4cfc0a09c0 legacy=0x7f4cfc0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cfa7fc700 1 -- 172.21.15.32:0/905779310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc09db70 con 0x7f4cfc09d5c0 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cfb7fe700 1 --2- 172.21.15.32:0/905779310 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 0x7f4cfc13b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cfa7fc700 1 --2- 172.21.15.32:0/905779310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc138d70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009840 tx=0x7f4cf0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f886d037d388a940 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cf9ffb700 1 -- 172.21.15.32:0/905779310 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011830 con 0x7f4cfc09d5c0 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cf9ffb700 1 -- 172.21.15.32:0/905779310 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0011990 con 0x7f4cfc09d5c0 2023-09-30T22:25:22.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.957+0000 7f4cf9ffb700 1 -- 172.21.15.32:0/905779310 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011d20 con 0x7f4cfc09d5c0 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 -- 172.21.15.32:0/905779310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 msgr2=0x7f4cfc138d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 --2- 172.21.15.32:0/905779310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc138d70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009840 tx=0x7f4cf0009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 -- 172.21.15.32:0/905779310 shutdown_connections 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 --2- 172.21.15.32:0/905779310 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc0a14a0 0x7f4cfc13b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 --2- 172.21.15.32:0/905779310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc138d70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 -- 172.21.15.32:0/905779310 >> 172.21.15.32:0/905779310 conn(0x7f4cfc0a2fe0 msgr2=0x7f4cfc0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 -- 172.21.15.32:0/905779310 shutdown_connections 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.958+0000 7f4d01327700 1 -- 172.21.15.32:0/905779310 wait complete. 2023-09-30T22:25:22.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 Processor -- start 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 -- start start 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc11c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc0a09c0 0x7f4cfc11ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 0x7f4cfc11d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc120020 con 0x7f4cfc0a14a0 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc11e1b0 con 0x7f4cfc09d5c0 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.959+0000 7f4d01327700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0a5010 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 0x7f4cfc11d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 0x7f4cfc11d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54958/0 (socket says 172.21.15.32:54958) 2023-09-30T22:25:22.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc11c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfb7fe700 1 -- 172.21.15.32:0/3467197317 learned_addr learned my addr 172.21.15.32:0/3467197317 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc0a09c0 0x7f4cfc11ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 -- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 msgr2=0x7f4cfc11c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc11c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 -- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 msgr2=0x7f4cfc11d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 0x7f4cfc11d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 -- 172.21.15.32:0/3467197317 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4009a70 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfaffd700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc11c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfb7fe700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 0x7f4cfc11d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4cfa7fc700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc0a09c0 0x7f4cfc11ccf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0016000 tx=0x7f4cf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:22.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00036a0 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf001f070 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf00094a0 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.960+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0003a60 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.961+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc11f3d0 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.961+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4cf0003bc0 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.961+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4cf0021070 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.962+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc0747f0 con 0x7f4cfc0a09c0 2023-09-30T22:25:22.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:22.966+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4cf002b020 con 0x7f4cfc0a09c0 2023-09-30T22:25:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.137+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe280071d20 con 0x7fe28009e930 2023-09-30T22:25:23.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.137+0000 7fe275ffb700 1 -- 172.21.15.32:0/875712468 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe27c01a450 con 0x7fe28009e930 2023-09-30T22:25:23.137 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 msgr2=0x7fe28006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28006b880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 shutdown_connections 2023-09-30T22:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe28009f650 0x7fe28006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe28009e930 0x7fe28006b880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 --2- 172.21.15.32:0/875712468 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe28009f240 0x7fe28006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 >> 172.21.15.32:0/875712468 conn(0x7fe2800a4210 msgr2=0x7fe2801b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 shutdown_connections 2023-09-30T22:25:23.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.140+0000 7fe2875c6700 1 -- 172.21.15.32:0/875712468 wait complete. 2023-09-30T22:25:23.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:23.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.177+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4cfc072980 con 0x7f4cfc0a09c0 2023-09-30T22:25:23.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.178+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3467197317 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4cf0015070 con 0x7f4cfc0a09c0 2023-09-30T22:25:23.178 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc0a09c0 msgr2=0x7f4cfc11ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc0a09c0 0x7f4cfc11ccf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0016000 tx=0x7f4cf0004750 comp rx=0 tx=0).stop 2023-09-30T22:25:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 shutdown_connections 2023-09-30T22:25:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc0a14a0 0x7f4cfc11d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d5c0 0x7f4cfc11c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 --2- 172.21.15.32:0/3467197317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc0a09c0 0x7f4cfc11ccf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 >> 172.21.15.32:0/3467197317 conn(0x7f4cfc0a2fe0 msgr2=0x7f4cfc09a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 shutdown_connections 2023-09-30T22:25:23.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.180+0000 7f4d01327700 1 -- 172.21.15.32:0/3467197317 wait complete. 2023-09-30T22:25:23.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:23.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:23.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.567+0000 7f310bb72700 1 Processor -- start 2023-09-30T22:25:23.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310bb72700 1 -- start start 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310bb72700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310bb72700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31040a0d90 0x7f31040a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310bb72700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31040a2a20 con 0x7f31041338a0 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310bb72700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310409e690 con 0x7f31040a0d90 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310bb72700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31040a1cc0 con 0x7f31040a09c0 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310910d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31040a0d90 0x7f31040a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310a10f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31041338a0 0x7f3104133d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51536/0 (socket says 172.21.15.32:51536) 2023-09-30T22:25:23.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310a10f700 1 -- 172.21.15.32:0/140495975 learned_addr learned my addr 172.21.15.32:0/140495975 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.568+0000 7f310990e700 1 --2- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310910d700 1 -- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 msgr2=0x7f310406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310910d700 1 --2- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310910d700 1 -- 172.21.15.32:0/140495975 >> v1:172.21.15.32:6789/0 conn(0x7f31041338a0 legacy=0x7f3104133d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310890c700 1 -- 172.21.15.32:0/140495975 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867525051 0 0) 0x7f30fc001490 con 0x7f31041338a0 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310910d700 1 -- 172.21.15.32:0/140495975 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040a1f40 con 0x7f31040a0d90 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310910d700 1 --2- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f31040a11c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009840 tx=0x7f30f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d5746c6c49f17e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310890c700 1 -- 172.21.15.32:0/140495975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8011830 con 0x7f31040a0d90 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310890c700 1 -- 172.21.15.32:0/140495975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f8011990 con 0x7f31040a0d90 2023-09-30T22:25:23.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.569+0000 7f310890c700 1 -- 172.21.15.32:0/140495975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8011d20 con 0x7f31040a0d90 2023-09-30T22:25:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 -- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 msgr2=0x7f31040a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 --2- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f31040a11c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009840 tx=0x7f30f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 -- 172.21.15.32:0/140495975 shutdown_connections 2023-09-30T22:25:23.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 --2- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 --2- 172.21.15.32:0/140495975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f31040a11c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 -- 172.21.15.32:0/140495975 >> 172.21.15.32:0/140495975 conn(0x7f31040a60c0 msgr2=0x7f31040a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 -- 172.21.15.32:0/140495975 shutdown_connections 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.570+0000 7f310bb72700 1 -- 172.21.15.32:0/140495975 wait complete. 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 Processor -- start 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 -- start start 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310411e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f310411e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31041338a0 0x7f310411eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310411f560 con 0x7f31040a0d90 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310411fc30 con 0x7f31040a09c0 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.571+0000 7f310bb72700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3104121b40 con 0x7f31041338a0 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310411e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310411e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56012/0 (socket says 172.21.15.32:56012) 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310a10f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31041338a0 0x7f310411eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310910d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f310411e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 -- 172.21.15.32:0/1506191373 learned_addr learned my addr 172.21.15.32:0/1506191373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 -- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31041338a0 msgr2=0x7f310411eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31041338a0 0x7f310411eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 -- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 msgr2=0x7f310411e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f310411e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 -- 172.21.15.32:0/1506191373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80094a0 con 0x7f31040a09c0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310910d700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f310411e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310990e700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310411e0b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30f400e9b0 tx=0x7f30f400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4009710 con 0x7f31040a09c0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f400ee50 con 0x7f31040a09c0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310409dea0 con 0x7f31040a09c0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400ca50 con 0x7f31040a09c0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.572+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310409cea0 con 0x7f31040a09c0 2023-09-30T22:25:23.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.573+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310409b1e0 con 0x7f31040a09c0 2023-09-30T22:25:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.573+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f30f400cbb0 con 0x7f31040a09c0 2023-09-30T22:25:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.574+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f30f403fa60 con 0x7f31040a09c0 2023-09-30T22:25:23.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.579+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30f401b3b0 con 0x7f31040a09c0 2023-09-30T22:25:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.606+0000 7f3fcd830700 1 Processor -- start 2023-09-30T22:25:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.606+0000 7f3fcd830700 1 -- start start 2023-09-30T22:25:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fcd830700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e850 0x7f3fc809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fcd830700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fcd830700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8132130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fcd830700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc809b560 con 0x7f3fc809e850 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fcd830700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80a0040 con 0x7f3fc8131cd0 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fcd830700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80a02e0 con 0x7f3fc809f160 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8132130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8132130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56016/0 (socket says 172.21.15.32:56016) 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc77fe700 1 -- 172.21.15.32:0/3824025334 learned_addr learned my addr 172.21.15.32:0/3824025334 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8131790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53474/0 (socket says 172.21.15.32:53474) 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc6ffd700 1 --2- 172.21.15.32:0/3824025334 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e850 0x7f3fc809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 -- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 msgr2=0x7f3fc8132130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8132130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 -- 172.21.15.32:0/3824025334 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e850 msgr2=0x7f3fc809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/3824025334 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e850 0x7f3fc809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 -- 172.21.15.32:0/3824025334 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc809fab0 con 0x7f3fc809f160 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc77fe700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8132130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc6ffd700 1 --2- 172.21.15.32:0/3824025334 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e850 0x7f3fc809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.607+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8131790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009840 tx=0x7f3fbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae471d9ac4ee809 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fc5ffb700 1 -- 172.21.15.32:0/3824025334 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011830 con 0x7f3fc809f160 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fc5ffb700 1 -- 172.21.15.32:0/3824025334 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc011990 con 0x7f3fc809f160 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fc5ffb700 1 -- 172.21.15.32:0/3824025334 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011d20 con 0x7f3fc809f160 2023-09-30T22:25:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 -- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 msgr2=0x7f3fc8131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8131790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009840 tx=0x7f3fbc009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 -- 172.21.15.32:0/3824025334 shutdown_connections 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8132130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 --2- 172.21.15.32:0/3824025334 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e850 0x7f3fc809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 --2- 172.21.15.32:0/3824025334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8131790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 -- 172.21.15.32:0/3824025334 >> 172.21.15.32:0/3824025334 conn(0x7f3fc80a3f70 msgr2=0x7f3fc80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 -- 172.21.15.32:0/3824025334 shutdown_connections 2023-09-30T22:25:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.608+0000 7f3fcd830700 1 -- 172.21.15.32:0/3824025334 wait complete. 2023-09-30T22:25:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.609+0000 7f3fcd830700 1 Processor -- start 2023-09-30T22:25:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 -- start start 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 0x7f3fc8122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8123000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8123540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc809d9a0 con 0x7f3fc809e850 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81265c0 con 0x7f3fc8131cd0 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fcd830700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8125f20 con 0x7f3fc809f160 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fc6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 0x7f3fc8122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fc6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 0x7f3fc8122ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55004/0 (socket says 172.21.15.32:55004) 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fc6ffd700 1 -- 172.21.15.32:0/1195328623 learned_addr learned my addr 172.21.15.32:0/1195328623 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8123000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.610+0000 7f3fc77fe700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8123540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc67fc700 1 -- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 msgr2=0x7f3fc8123540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8123540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc67fc700 1 -- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 msgr2=0x7f3fc8122ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 0x7f3fc8122ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc67fc700 1 -- 172.21.15.32:0/1195328623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80094a0 con 0x7f3fc809f160 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc6ffd700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 0x7f3fc8122ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fc67fc700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8123000 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc016000 tx=0x7f3fbc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc004240 con 0x7f3fc809f160 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc01f070 con 0x7f3fc809f160 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0009b30 con 0x7f3fc809f160 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0036a0 con 0x7f3fc809f160 2023-09-30T22:25:23.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.611+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc0094a0 con 0x7f3fc809f160 2023-09-30T22:25:23.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.612+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fbc021030 con 0x7f3fc809f160 2023-09-30T22:25:23.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.612+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3fbc040720 con 0x7f3fc809f160 2023-09-30T22:25:23.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.612+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc80747f0 con 0x7f3fc809f160 2023-09-30T22:25:23.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.617+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fbc010970 con 0x7f3fc809f160 2023-09-30T22:25:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.791+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f310404f2a0 con 0x7f31040a09c0 2023-09-30T22:25:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.791+0000 7f31027fc700 1 -- 172.21.15.32:0/1506191373 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f30f4021020 con 0x7f31040a09c0 2023-09-30T22:25:23.792 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.793+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 msgr2=0x7f310411e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.793+0000 7f310bb72700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310411e0b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f30f400e9b0 tx=0x7f30f400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.793+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 shutdown_connections 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.793+0000 7f310bb72700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31041338a0 0x7f310411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.793+0000 7f310bb72700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31040a09c0 0x7f310411e0b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.793+0000 7f310bb72700 1 --2- 172.21.15.32:0/1506191373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31040a0d90 0x7f310411e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.794+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 >> 172.21.15.32:0/1506191373 conn(0x7f31040a60c0 msgr2=0x7f3104133020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.794+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 shutdown_connections 2023-09-30T22:25:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.794+0000 7f310bb72700 1 -- 172.21.15.32:0/1506191373 wait complete. 2023-09-30T22:25:23.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.827+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3fc809b780 con 0x7f3fc809f160 2023-09-30T22:25:23.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.827+0000 7f3faffff700 1 -- 172.21.15.32:0/1195328623 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fbc015020 con 0x7f3fc809f160 2023-09-30T22:25:23.827 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 msgr2=0x7f3fc8123000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8123000 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc016000 tx=0x7f3fbc004750 comp rx=0 tx=0).stop 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 shutdown_connections 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc8131cd0 0x7f3fc8123540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e850 0x7f3fc8122ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 --2- 172.21.15.32:0/1195328623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f160 0x7f3fc8123000 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.829+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 >> 172.21.15.32:0/1195328623 conn(0x7f3fc80a3f70 msgr2=0x7f3fc8071800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.830+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 shutdown_connections 2023-09-30T22:25:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:23.830+0000 7f3fcd830700 1 -- 172.21.15.32:0/1195328623 wait complete. 2023-09-30T22:25:23.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:24.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:24.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.303+0000 7fbb14fd5700 1 Processor -- start 2023-09-30T22:25:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 -- start start 2023-09-30T22:25:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009d6f0 0x7fbb1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb1009e000 0x7fbb101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 0x7fbb10133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1006b460 con 0x7fbb1009e000 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1009b120 con 0x7fbb1009e410 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.304+0000 7fbb14fd5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1009a7e0 con 0x7fbb1009d6f0 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb1009e000 0x7fbb101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 0x7fbb10133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009d6f0 0x7fbb1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb1009e000 0x7fbb101339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55012/0 (socket says 172.21.15.32:55012) 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 0x7fbb10133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56034/0 (socket says 172.21.15.32:56034) 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2133496189 learned_addr learned my addr 172.21.15.32:0/2133496189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009d6f0 msgr2=0x7fbb1009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009d6f0 0x7fbb1009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 msgr2=0x7fbb10133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 0x7fbb10133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2133496189 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1009a500 con 0x7fbb1009e000 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0ed9d700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 0x7fbb10133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0e59c700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009d6f0 0x7fbb1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb05d9b700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101339f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80098f0 tx=0x7fbaf8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe5874be498619b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0dd9b700 1 -- 172.21.15.32:0/2133496189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8011830 con 0x7fbb1009e000 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0dd9b700 1 -- 172.21.15.32:0/2133496189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf8011990 con 0x7fbb1009e000 2023-09-30T22:25:24.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.305+0000 7fbb0dd9b700 1 -- 172.21.15.32:0/2133496189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8011d20 con 0x7fbb1009e000 2023-09-30T22:25:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 msgr2=0x7fbb101339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101339f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80098f0 tx=0x7fbaf8009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:24.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2133496189 shutdown_connections 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009e410 0x7fbb10133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009d6f0 0x7fbb1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2133496189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101339f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2133496189 >> 172.21.15.32:0/2133496189 conn(0x7fbb10066f50 msgr2=0x7fbb1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2133496189 shutdown_connections 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.306+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2133496189 wait complete. 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.308+0000 7fbb14fd5700 1 Processor -- start 2023-09-30T22:25:24.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.308+0000 7fbb14fd5700 1 -- start start 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb14fd5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 0x7fbb101a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb14fd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb14fd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 0x7fbb101a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb14fd5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100a2090 con 0x7fbb1009e000 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb14fd5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101a9310 con 0x7fbb1009d6f0 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb14fd5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1009c710 con 0x7fbb1009e410 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb0e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 0x7fbb101a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb0ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 0x7fbb101a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb05d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb0e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 0x7fbb101a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56042/0 (socket says 172.21.15.32:56042) 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb0e59c700 1 -- 172.21.15.32:0/2199892001 learned_addr learned my addr 172.21.15.32:0/2199892001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.309+0000 7fbb0ed9d700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 0x7fbb101a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53504/0 (socket says 172.21.15.32:53504) 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 msgr2=0x7fbb101a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb05d9b700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 0x7fbb101a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 msgr2=0x7fbb101a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb05d9b700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 0x7fbb101a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb05d9b700 1 -- 172.21.15.32:0/2199892001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000094a0 con 0x7fbb1009e000 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb0ed9d700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 0x7fbb101a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb0e59c700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 0x7fbb101a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb05d9b700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101a6480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbb1006e510 tx=0x7fbaf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8004240 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf801f070 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb08009a70 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80045c0 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.310+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf8009550 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.311+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb10074ea0 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.311+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbaf8021030 con 0x7fbb1009e000 2023-09-30T22:25:24.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.312+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbaf8040bd0 con 0x7fbb1009e000 2023-09-30T22:25:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.320+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbaf8003e00 con 0x7fbb1009e000 2023-09-30T22:25:24.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.354+0000 7f2ff8c5f700 1 Processor -- start 2023-09-30T22:25:24.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.354+0000 7f2ff8c5f700 1 -- start start 2023-09-30T22:25:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff8c5f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 0x7f2ff40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff8c5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff4138d80 0x7f2ff413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff8c5f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff40a2a70 con 0x7f2ff40a0a30 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff8c5f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40a01d0 con 0x7f2ff409d630 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff8c5f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff409e2b0 con 0x7f2ff4138d80 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 0x7f2ff40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 0x7f2ff40a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56056/0 (socket says 172.21.15.32:56056) 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ff40a0a30 0x7f2ff40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51544/0 (socket says 172.21.15.32:51544) 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/273570708 learned_addr learned my addr 172.21.15.32:0/273570708 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff2d9d700 1 --2- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff4138d80 0x7f2ff413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff159a700 1 -- 172.21.15.32:0/273570708 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153619361 0 0) 0x7f2ff40a2a70 con 0x7f2ff40a0a30 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff4138d80 msgr2=0x7f2ff413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 --2- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff4138d80 0x7f2ff413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/273570708 >> v1:172.21.15.32:6789/0 conn(0x7f2ff40a0a30 legacy=0x7f2ff40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/273570708 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff409dc50 con 0x7f2ff409d630 2023-09-30T22:25:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff1d9b700 1 --2- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 0x7f2ff40a1510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009920 tx=0x7f2fe8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc0fbc8cea1fb82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.355+0000 7f2ff159a700 1 -- 172.21.15.32:0/273570708 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011830 con 0x7f2ff409d630 2023-09-30T22:25:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff159a700 1 -- 172.21.15.32:0/273570708 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8011990 con 0x7f2ff409d630 2023-09-30T22:25:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff159a700 1 -- 172.21.15.32:0/273570708 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011d20 con 0x7f2ff409d630 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 msgr2=0x7f2ff40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 0x7f2ff40a1510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009920 tx=0x7f2fe8009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/273570708 shutdown_connections 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff4138d80 0x7f2ff413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/273570708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff409d630 0x7f2ff40a1510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/273570708 >> 172.21.15.32:0/273570708 conn(0x7f2ff40a3050 msgr2=0x7f2ff40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/273570708 shutdown_connections 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.356+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/273570708 wait complete. 2023-09-30T22:25:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.357+0000 7f2ff8c5f700 1 Processor -- start 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 -- start start 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff409d630 0x7f2ff41337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 0x7f2ff4133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff4138d80 0x7f2ff4134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4134c10 con 0x7f2ff4138d80 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41365f0 con 0x7f2ff40a0a30 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff8c5f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4136f60 con 0x7f2ff409d630 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 0x7f2ff4133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff4138d80 0x7f2ff4134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 0x7f2ff4133cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56058/0 (socket says 172.21.15.32:56058) 2023-09-30T22:25:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/345778061 learned_addr learned my addr 172.21.15.32:0/345778061 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.358+0000 7f2ff259c700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff409d630 0x7f2ff41337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff409d630 msgr2=0x7f2ff41337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff1d9b700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff409d630 0x7f2ff41337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff4138d80 msgr2=0x7f2ff4134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff1d9b700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff4138d80 0x7f2ff4134230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff1d9b700 1 -- 172.21.15.32:0/345778061 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8009550 con 0x7f2ff40a0a30 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff2d9d700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff4138d80 0x7f2ff4134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff1d9b700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 0x7f2ff4133cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800a010 tx=0x7f2fe8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8004240 con 0x7f2ff40a0a30 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe801f070 con 0x7f2ff40a0a30 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80045c0 con 0x7f2ff40a0a30 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff40a4b70 con 0x7f2ff40a0a30 2023-09-30T22:25:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.359+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff41379d0 con 0x7f2ff40a0a30 2023-09-30T22:25:24.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.360+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff40747f0 con 0x7f2ff40a0a30 2023-09-30T22:25:24.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.360+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2fe8021030 con 0x7f2ff40a0a30 2023-09-30T22:25:24.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.360+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2fe80408d0 con 0x7f2ff40a0a30 2023-09-30T22:25:24.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.369+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2fe80233d0 con 0x7f2ff40a0a30 2023-09-30T22:25:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.542+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbb101a87a0 con 0x7fbb1009e000 2023-09-30T22:25:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.543+0000 7fbb077fe700 1 -- 172.21.15.32:0/2199892001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbaf8015070 con 0x7fbb1009e000 2023-09-30T22:25:24.543 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 msgr2=0x7fbb101a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101a6480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbb1006e510 tx=0x7fbaf8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 shutdown_connections 2023-09-30T22:25:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbb1009e410 0x7fbb101a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb1009d6f0 0x7fbb101a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 --2- 172.21.15.32:0/2199892001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb1009e000 0x7fbb101a6480 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 >> 172.21.15.32:0/2199892001 conn(0x7fbb10066f50 msgr2=0x7fbb101bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 shutdown_connections 2023-09-30T22:25:24.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.545+0000 7fbb14fd5700 1 -- 172.21.15.32:0/2199892001 wait complete. 2023-09-30T22:25:24.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.577+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2ff4072980 con 0x7f2ff40a0a30 2023-09-30T22:25:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.577+0000 7f2fe2ffd700 1 -- 172.21.15.32:0/345778061 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fe8015020 con 0x7f2ff40a0a30 2023-09-30T22:25:24.577 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 msgr2=0x7f2ff4133cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 0x7f2ff4133cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800a010 tx=0x7f2fe8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 shutdown_connections 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff4138d80 0x7f2ff4134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ff409d630 0x7f2ff41337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 --2- 172.21.15.32:0/345778061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ff40a0a30 0x7f2ff4133cf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 >> 172.21.15.32:0/345778061 conn(0x7f2ff40a3050 msgr2=0x7f2ff409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 shutdown_connections 2023-09-30T22:25:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.580+0000 7f2ff8c5f700 1 -- 172.21.15.32:0/345778061 wait complete. 2023-09-30T22:25:24.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:24.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:24.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 Processor -- start 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 -- start start 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90480984f0 0x7f90480988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f904809d6f0 0x7f904809db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 0x7f90481385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9048099620 con 0x7f904809d6f0 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904809aca0 con 0x7f904809e040 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.957+0000 7f904d94b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f904809a9e0 con 0x7f90480984f0 2023-09-30T22:25:24.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90477fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 0x7f90481385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f904809d6f0 0x7f904809db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90477fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 0x7f90481385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56066/0 (socket says 172.21.15.32:56066) 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90477fe700 1 -- 172.21.15.32:0/1693179565 learned_addr learned my addr 172.21.15.32:0/1693179565 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f9046ffd700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90480984f0 0x7f90480988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 -- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90480984f0 msgr2=0x7f90480988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90480984f0 0x7f90480988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 -- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 msgr2=0x7f90481385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 0x7f90481385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 -- 172.21.15.32:0/1693179565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904809bce0 con 0x7f904809d6f0 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90477fe700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 0x7f90481385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f90467fc700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809d6f0 0x7f904809db00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f903c009840 tx=0x7f903c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2d49434595cd27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f9045ffb700 1 -- 172.21.15.32:0/1693179565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c011830 con 0x7f904809d6f0 2023-09-30T22:25:24.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f9045ffb700 1 -- 172.21.15.32:0/1693179565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c011990 con 0x7f904809d6f0 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.958+0000 7f9045ffb700 1 -- 172.21.15.32:0/1693179565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c011d20 con 0x7f904809d6f0 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 -- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809d6f0 msgr2=0x7f904809db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809d6f0 0x7f904809db00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f903c009840 tx=0x7f903c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 -- 172.21.15.32:0/1693179565 shutdown_connections 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904809e040 0x7f90481385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90480984f0 0x7f90480988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 --2- 172.21.15.32:0/1693179565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809d6f0 0x7f904809db00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 -- 172.21.15.32:0/1693179565 >> 172.21.15.32:0/1693179565 conn(0x7f904809f470 msgr2=0x7f90480a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 -- 172.21.15.32:0/1693179565 shutdown_connections 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.959+0000 7f904d94b700 1 -- 172.21.15.32:0/1693179565 wait complete. 2023-09-30T22:25:24.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.960+0000 7f904d94b700 1 Processor -- start 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.960+0000 7f904d94b700 1 -- start start 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f904d94b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90480984f0 0x7f90481c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f904d94b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f904809d6f0 0x7f90481c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f904d94b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 0x7f904812d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f904d94b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904812ed50 con 0x7f904809e040 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f904d94b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480a1b90 con 0x7f90480984f0 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f904d94b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f904806da10 con 0x7f904809d6f0 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 0x7f904812d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90467fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f904809d6f0 0x7f90481c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 0x7f904812d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55046/0 (socket says 172.21.15.32:55046) 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 -- 172.21.15.32:0/503929726 learned_addr learned my addr 172.21.15.32:0/503929726 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f9046ffd700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90480984f0 0x7f90481c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 -- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f904809d6f0 msgr2=0x7f90481c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f904809d6f0 0x7f90481c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 -- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90480984f0 msgr2=0x7f90481c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90480984f0 0x7f90481c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90477fe700 1 -- 172.21.15.32:0/503929726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009b30 con 0x7f904809e040 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f90467fc700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f904809d6f0 0x7f90481c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.961+0000 7f9046ffd700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90480984f0 0x7f90481c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.962+0000 7f90477fe700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 0x7f904812d0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9040009f10 tx=0x7f904000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.962+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040018070 con 0x7f904809e040 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.962+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904000f040 con 0x7f904809e040 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.962+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c0094a0 con 0x7f904809e040 2023-09-30T22:25:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.962+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040003c40 con 0x7f904809e040 2023-09-30T22:25:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.962+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904809a2b0 con 0x7f904809e040 2023-09-30T22:25:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.963+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f904000c940 con 0x7f904809e040 2023-09-30T22:25:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.963+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f904003f6a0 con 0x7f904809e040 2023-09-30T22:25:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.963+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90480747f0 con 0x7f904809e040 2023-09-30T22:25:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.968+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9040014480 con 0x7f904809e040 2023-09-30T22:25:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.995+0000 7f7d4c44f700 1 Processor -- start 2023-09-30T22:25:24.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.995+0000 7f7d4c44f700 1 -- start start 2023-09-30T22:25:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4c44f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 0x7f7d440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4c44f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4c44f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 0x7f7d44134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4c44f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4409edd0 con 0x7f7d44133fa0 2023-09-30T22:25:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4c44f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4409e1f0 con 0x7f7d440a0a30 2023-09-30T22:25:24.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4c44f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4409dc50 con 0x7f7d440a1340 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d499ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4a9ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 0x7f7d44134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4a1eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 0x7f7d440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4a1eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 0x7f7d440a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56078/0 (socket says 172.21.15.32:56078) 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d499ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53562/0 (socket says 172.21.15.32:53562) 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4a1eb700 1 -- 172.21.15.32:0/4220199854 learned_addr learned my addr 172.21.15.32:0/4220199854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:24.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.996+0000 7f7d4a9ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 0x7f7d44134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55062/0 (socket says 172.21.15.32:55062) 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a1eb700 1 -- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 msgr2=0x7f7d44133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a1eb700 1 --2- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a1eb700 1 -- 172.21.15.32:0/4220199854 >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 msgr2=0x7f7d44134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a1eb700 1 --2- 172.21.15.32:0/4220199854 >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 0x7f7d44134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a1eb700 1 -- 172.21.15.32:0/4220199854 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d440a1fe0 con 0x7f7d440a0a30 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a9ec700 1 --2- 172.21.15.32:0/4220199854 >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 0x7f7d44134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d499ea700 1 --2- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d4a1eb700 1 --2- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 0x7f7d440a0e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d400098f0 tx=0x7f7d40009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f20fed5e1f18d71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:24.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d491e9700 1 -- 172.21.15.32:0/4220199854 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40011830 con 0x7f7d440a0a30 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d491e9700 1 -- 172.21.15.32:0/4220199854 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d40011990 con 0x7f7d440a0a30 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.997+0000 7f7d491e9700 1 -- 172.21.15.32:0/4220199854 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40011d20 con 0x7f7d440a0a30 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 -- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 msgr2=0x7f7d440a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 0x7f7d440a0e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d400098f0 tx=0x7f7d40009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 -- 172.21.15.32:0/4220199854 shutdown_connections 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d440a0a30 0x7f7d440a0e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/4220199854 >> v2:172.21.15.32:3300/0 conn(0x7f7d44133fa0 0x7f7d44134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/4220199854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 -- 172.21.15.32:0/4220199854 >> 172.21.15.32:0/4220199854 conn(0x7f7d440a6550 msgr2=0x7f7d440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 -- 172.21.15.32:0/4220199854 shutdown_connections 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.998+0000 7f7d4c44f700 1 -- 172.21.15.32:0/4220199854 wait complete. 2023-09-30T22:25:25.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.999+0000 7f7d4c44f700 1 Processor -- start 2023-09-30T22:25:25.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:24.999+0000 7f7d4c44f700 1 -- start start 2023-09-30T22:25:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4c44f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 0x7f7d441212f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4c44f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44121830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4c44f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 0x7f7d44121d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4c44f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44124e10 con 0x7f7d440a0a30 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4c44f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44123d90 con 0x7f7d44133fa0 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4c44f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44124870 con 0x7f7d440a1340 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44121830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4a1eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 0x7f7d441212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4a9ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 0x7f7d44121d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44121830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53568/0 (socket says 172.21.15.32:53568) 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4a1eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 0x7f7d441212f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55076/0 (socket says 172.21.15.32:55076) 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 -- 172.21.15.32:0/2833426503 learned_addr learned my addr 172.21.15.32:0/2833426503 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4a9ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 0x7f7d44121d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56094/0 (socket says 172.21.15.32:56094) 2023-09-30T22:25:25.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 -- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 msgr2=0x7f7d44121d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 0x7f7d44121d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 -- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 msgr2=0x7f7d441212f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 0x7f7d441212f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 -- 172.21.15.32:0/2833426503 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340094a0 con 0x7f7d440a1340 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4a9ec700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 0x7f7d44121d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d4a1eb700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 0x7f7d441212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.000+0000 7f7d499ea700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44121830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400ec10 tx=0x7f7d3400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.001+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400c850 con 0x7f7d440a1340 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.001+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400c9b0 con 0x7f7d440a1340 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.001+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c009a70 con 0x7f7d440a1340 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.001+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400cd20 con 0x7f7d440a1340 2023-09-30T22:25:25.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.001+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d40009550 con 0x7f7d440a1340 2023-09-30T22:25:25.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.002+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d440a0220 con 0x7f7d440a1340 2023-09-30T22:25:25.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.002+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d34013470 con 0x7f7d440a1340 2023-09-30T22:25:25.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.002+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d3403fb30 con 0x7f7d440a1340 2023-09-30T22:25:25.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.007+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d34013720 con 0x7f7d440a1340 2023-09-30T22:25:25.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.178+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f904806eea0 con 0x7f904809e040 2023-09-30T22:25:25.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.179+0000 7f902ffff700 1 -- 172.21.15.32:0/503929726 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f904001cb60 con 0x7f904809e040 2023-09-30T22:25:25.179 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 msgr2=0x7f904812d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 0x7f904812d0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9040009f10 tx=0x7f904000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 shutdown_connections 2023-09-30T22:25:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f904809e040 0x7f904812d0d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90480984f0 0x7f90481c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 --2- 172.21.15.32:0/503929726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f904809d6f0 0x7f90481c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 >> 172.21.15.32:0/503929726 conn(0x7f904809f470 msgr2=0x7f90481b8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 shutdown_connections 2023-09-30T22:25:25.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.181+0000 7f904d94b700 1 -- 172.21.15.32:0/503929726 wait complete. 2023-09-30T22:25:25.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.212+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7d44074ea0 con 0x7f7d440a1340 2023-09-30T22:25:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.212+0000 7f7d3affd700 1 -- 172.21.15.32:0/2833426503 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d34021020 con 0x7f7d440a1340 2023-09-30T22:25:25.212 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 msgr2=0x7f7d44121830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44121830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400ec10 tx=0x7f7d3400ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 shutdown_connections 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d44133fa0 0x7f7d44121d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d440a0a30 0x7f7d441212f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 --2- 172.21.15.32:0/2833426503 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d440a1340 0x7f7d44121830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 >> 172.21.15.32:0/2833426503 conn(0x7f7d440a6550 msgr2=0x7f7d440a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 shutdown_connections 2023-09-30T22:25:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.215+0000 7f7d4c44f700 1 -- 172.21.15.32:0/2833426503 wait complete. 2023-09-30T22:25:25.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:25.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:25.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:25.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.709+0000 7f0737527700 1 Processor -- start 2023-09-30T22:25:25.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.709+0000 7f0737527700 1 -- start start 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0737527700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009d6f0 0x7f073009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0737527700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 0x7f0730130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0737527700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f073009e410 0x7f0730130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0737527700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073009f520 con 0x7f073009e410 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0737527700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073009f130 con 0x7f073009d6f0 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0737527700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f073009e930 con 0x7f073009e000 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 0x7f0730130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 0x7f0730130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53584/0 (socket says 172.21.15.32:53584) 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0735ac4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f073009e410 0x7f0730130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 -- 172.21.15.32:0/3589299926 learned_addr learned my addr 172.21.15.32:0/3589299926 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f07352c3700 1 --2- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009d6f0 0x7f073009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 -- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009d6f0 msgr2=0x7f073009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 --2- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009d6f0 0x7f073009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 -- 172.21.15.32:0/3589299926 >> v2:172.21.15.32:3300/0 conn(0x7f073009e410 msgr2=0x7f0730130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 --2- 172.21.15.32:0/3589299926 >> v2:172.21.15.32:3300/0 conn(0x7f073009e410 0x7f0730130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.710+0000 7f0734ac2700 1 -- 172.21.15.32:0/3589299926 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073009b2d0 con 0x7f073009e000 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.711+0000 7f0734ac2700 1 --2- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 0x7f0730130660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0720009920 tx=0x7f0720009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3016cc7e113c4b0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.711+0000 7f0727fff700 1 -- 172.21.15.32:0/3589299926 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720011830 con 0x7f073009e000 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.711+0000 7f0727fff700 1 -- 172.21.15.32:0/3589299926 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0720011990 con 0x7f073009e000 2023-09-30T22:25:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.711+0000 7f0727fff700 1 -- 172.21.15.32:0/3589299926 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720011d20 con 0x7f073009e000 2023-09-30T22:25:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 -- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 msgr2=0x7f0730130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 --2- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 0x7f0730130660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0720009920 tx=0x7f0720009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 -- 172.21.15.32:0/3589299926 shutdown_connections 2023-09-30T22:25:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 --2- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009d6f0 0x7f073009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 --2- 172.21.15.32:0/3589299926 >> v2:172.21.15.32:3300/0 conn(0x7f073009e410 0x7f0730130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 --2- 172.21.15.32:0/3589299926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009e000 0x7f0730130660 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 -- 172.21.15.32:0/3589299926 >> 172.21.15.32:0/3589299926 conn(0x7f0730066f50 msgr2=0x7f073009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 -- 172.21.15.32:0/3589299926 shutdown_connections 2023-09-30T22:25:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.712+0000 7f0737527700 1 -- 172.21.15.32:0/3589299926 wait complete. 2023-09-30T22:25:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.713+0000 7f0737527700 1 Processor -- start 2023-09-30T22:25:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.713+0000 7f0737527700 1 -- start start 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0737527700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 0x7f073011df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0737527700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009e000 0x7f073011e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0737527700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f073009e410 0x7f073011e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0737527700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07301217b0 con 0x7f073009e410 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0737527700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07301302b0 con 0x7f073009e000 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0737527700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0730120d30 con 0x7f073009d6f0 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 0x7f073011df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 0x7f073011df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53600/0 (socket says 172.21.15.32:53600) 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 -- 172.21.15.32:0/2088589029 learned_addr learned my addr 172.21.15.32:0/2088589029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0734ac2700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009e000 0x7f073011e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f0735ac4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f073009e410 0x7f073011e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 -- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009e000 msgr2=0x7f073011e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009e000 0x7f073011e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 -- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f073009e410 msgr2=0x7f073011e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f073009e410 0x7f073011e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.714+0000 7f07352c3700 1 -- 172.21.15.32:0/2088589029 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720009550 con 0x7f073009d6f0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.715+0000 7f07352c3700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 0x7f073011df20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f072c012f00 tx=0x7f072c012f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.715+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c014040 con 0x7f073009d6f0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.715+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c00f860 con 0x7f073009d6f0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.715+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07301200c0 con 0x7f073009d6f0 2023-09-30T22:25:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.715+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00dbb0 con 0x7f073009d6f0 2023-09-30T22:25:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.715+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073009c700 con 0x7f073009d6f0 2023-09-30T22:25:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.716+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f072c00fbd0 con 0x7f073009d6f0 2023-09-30T22:25:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.716+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f072c043240 con 0x7f073009d6f0 2023-09-30T22:25:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.716+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0730074ea0 con 0x7f073009d6f0 2023-09-30T22:25:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.721+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f072c021410 con 0x7f073009d6f0 2023-09-30T22:25:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.746+0000 7fb8f04c5700 1 Processor -- start 2023-09-30T22:25:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 -- start start 2023-09-30T22:25:25.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809e930 0x7fb8e809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8e809f650 0x7fb8e8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e8070940 con 0x7fb8e809f650 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e806fd60 con 0x7fb8e809e930 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8f04c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e806fa80 con 0x7fb8e809f240 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8eea62700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8e809f650 0x7fb8e8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.747+0000 7fb8eda60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8e809f650 0x7fb8e8131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55094/0 (socket says 172.21.15.32:55094) 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 -- 172.21.15.32:0/3416264815 learned_addr learned my addr 172.21.15.32:0/3416264815 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8ee261700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809e930 0x7fb8e809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 -- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 msgr2=0x7fb8e81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 -- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809e930 msgr2=0x7fb8e809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809e930 0x7fb8e809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 -- 172.21.15.32:0/3416264815 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e809fb60 con 0x7fb8e809f650 2023-09-30T22:25:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eda60700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8eea62700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809f650 0x7fb8e8131d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009e50 tx=0x7fb8e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c164bb68732b1e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8ed25f700 1 -- 172.21.15.32:0/3416264815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000e070 con 0x7fb8e809f650 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8ed25f700 1 -- 172.21.15.32:0/3416264815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e000b910 con 0x7fb8e809f650 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.748+0000 7fb8ed25f700 1 -- 172.21.15.32:0/3416264815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000bcc0 con 0x7fb8e809f650 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 -- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809f650 msgr2=0x7fb8e8131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809f650 0x7fb8e8131d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009e50 tx=0x7fb8e000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 -- 172.21.15.32:0/3416264815 shutdown_connections 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809e930 0x7fb8e809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809f650 0x7fb8e8131d30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/3416264815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 -- 172.21.15.32:0/3416264815 >> 172.21.15.32:0/3416264815 conn(0x7fb8e80a4210 msgr2=0x7fb8e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 -- 172.21.15.32:0/3416264815 shutdown_connections 2023-09-30T22:25:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.749+0000 7fb8f04c5700 1 -- 172.21.15.32:0/3416264815 wait complete. 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.750+0000 7fb8f04c5700 1 Processor -- start 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.750+0000 7fb8f04c5700 1 -- start start 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 0x7fb8e806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809f650 0x7fb8e806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e81b72b0 con 0x7fb8e809e930 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e81a9460 con 0x7fb8e809f650 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e81a8980 con 0x7fb8e809f240 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 0x7fb8e806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 0x7fb8e806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55102/0 (socket says 172.21.15.32:55102) 2023-09-30T22:25:25.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 -- 172.21.15.32:0/2521933331 learned_addr learned my addr 172.21.15.32:0/2521933331 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8eea62700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809f650 0x7fb8e806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8eda60700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 -- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 msgr2=0x7fb8e806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 -- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809f650 msgr2=0x7fb8e806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809f650 0x7fb8e806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 -- 172.21.15.32:0/2521933331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d80094a0 con 0x7fb8e809e930 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8eda60700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8eea62700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809f650 0x7fb8e806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8ee261700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 0x7fb8e806b8b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400e930 tx=0x7fb8e400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400ee80 con 0x7fb8e809e930 2023-09-30T22:25:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400c850 con 0x7fb8e809e930 2023-09-30T22:25:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.751+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e0009b30 con 0x7fb8e809e930 2023-09-30T22:25:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.752+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400cbe0 con 0x7fb8e809e930 2023-09-30T22:25:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.752+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e80a0190 con 0x7fb8e809e930 2023-09-30T22:25:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.753+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8e400cd40 con 0x7fb8e809e930 2023-09-30T22:25:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.753+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e81a95d0 con 0x7fb8e809e930 2023-09-30T22:25:25.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.753+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb8e403fa70 con 0x7fb8e809e930 2023-09-30T22:25:25.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.758+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8e4011d90 con 0x7fb8e809e930 2023-09-30T22:25:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.946+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f073006c360 con 0x7f073009d6f0 2023-09-30T22:25:25.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.947+0000 7f0725ffb700 1 -- 172.21.15.32:0/2088589029 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f072c020370 con 0x7f073009d6f0 2023-09-30T22:25:25.947 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 msgr2=0x7f073011df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 0x7f073011df20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f072c012f00 tx=0x7f072c012f30 comp rx=0 tx=0).stop 2023-09-30T22:25:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 shutdown_connections 2023-09-30T22:25:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f073009e410 0x7f073011e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f073009d6f0 0x7f073011df20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 --2- 172.21.15.32:0/2088589029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f073009e000 0x7f073011e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 >> 172.21.15.32:0/2088589029 conn(0x7f0730066f50 msgr2=0x7f0730070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 shutdown_connections 2023-09-30T22:25:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.949+0000 7f0737527700 1 -- 172.21.15.32:0/2088589029 wait complete. 2023-09-30T22:25:25.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:25.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.977+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb8e81a0b30 con 0x7fb8e809e930 2023-09-30T22:25:25.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.977+0000 7fb8deffd700 1 -- 172.21.15.32:0/2521933331 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8e4021070 con 0x7fb8e809e930 2023-09-30T22:25:25.977 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:25.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 msgr2=0x7fb8e806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 0x7fb8e806b8b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400e930 tx=0x7fb8e400ec40 comp rx=0 tx=0).stop 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 shutdown_connections 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8e809f650 0x7fb8e806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8e809e930 0x7fb8e806b8b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 --2- 172.21.15.32:0/2521933331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8e809f240 0x7fb8e806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 >> 172.21.15.32:0/2521933331 conn(0x7fb8e80a4210 msgr2=0x7fb8e81b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 shutdown_connections 2023-09-30T22:25:25.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:25.980+0000 7fb8f04c5700 1 -- 172.21.15.32:0/2521933331 wait complete. 2023-09-30T22:25:25.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:26.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:26.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:26.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.343+0000 7f3882a38700 1 Processor -- start 2023-09-30T22:25:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 -- start start 2023-09-30T22:25:26.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 0x7f387c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f387c0a1340 0x7f387c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 0x7f387c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c09edd0 con 0x7f387c0a1340 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c09e1f0 con 0x7f387c0a0a30 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.344+0000 7f3882a38700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f387c09dc50 con 0x7f387c133fa0 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f3880fd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 0x7f387c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f387c0a1340 0x7f387c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 0x7f387c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f3880fd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 0x7f387c134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53626/0 (socket says 172.21.15.32:53626) 2023-09-30T22:25:26.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 0x7f387c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56126/0 (socket says 172.21.15.32:56126) 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f387c0a1340 0x7f387c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55108/0 (socket says 172.21.15.32:55108) 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f3880fd5700 1 -- 172.21.15.32:0/856343012 learned_addr learned my addr 172.21.15.32:0/856343012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 -- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 msgr2=0x7f387c134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 --2- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 0x7f387c134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 -- 172.21.15.32:0/856343012 >> v2:172.21.15.32:3300/0 conn(0x7f387c0a1340 msgr2=0x7f387c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 --2- 172.21.15.32:0/856343012 >> v2:172.21.15.32:3300/0 conn(0x7f387c0a1340 0x7f387c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 -- 172.21.15.32:0/856343012 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0a1fe0 con 0x7f387c0a0a30 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f3880fd5700 1 --2- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 0x7f387c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387bfff700 1 --2- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 0x7f387c0a0e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f386c009870 tx=0x7f386c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e04f872189276fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387affd700 1 -- 172.21.15.32:0/856343012 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c011830 con 0x7f387c0a0a30 2023-09-30T22:25:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387affd700 1 -- 172.21.15.32:0/856343012 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c011990 con 0x7f387c0a0a30 2023-09-30T22:25:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.345+0000 7f387affd700 1 -- 172.21.15.32:0/856343012 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c011d20 con 0x7f387c0a0a30 2023-09-30T22:25:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 -- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 msgr2=0x7f387c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 --2- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 0x7f387c0a0e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f386c009870 tx=0x7f386c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 -- 172.21.15.32:0/856343012 shutdown_connections 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 --2- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a0a30 0x7f387c0a0e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 --2- 172.21.15.32:0/856343012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c133fa0 0x7f387c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 --2- 172.21.15.32:0/856343012 >> v2:172.21.15.32:3300/0 conn(0x7f387c0a1340 0x7f387c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 -- 172.21.15.32:0/856343012 >> 172.21.15.32:0/856343012 conn(0x7f387c0a6550 msgr2=0x7f387c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 -- 172.21.15.32:0/856343012 shutdown_connections 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.346+0000 7f3882a38700 1 -- 172.21.15.32:0/856343012 wait complete. 2023-09-30T22:25:26.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.347+0000 7f3882a38700 1 Processor -- start 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 -- start start 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c0a0a30 0x7f387c120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a1340 0x7f387c120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 0x7f387c120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1215e0 con 0x7f387c133fa0 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c123c00 con 0x7f387c0a1340 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3882a38700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f387c122fb0 con 0x7f387c0a0a30 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3880fd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 0x7f387c120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a1340 0x7f387c120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3880fd5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 0x7f387c120bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55118/0 (socket says 172.21.15.32:55118) 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f387bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c0a0a30 0x7f387c120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.348+0000 7f3880fd5700 1 -- 172.21.15.32:0/3583841008 learned_addr learned my addr 172.21.15.32:0/3583841008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3880fd5700 1 -- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c0a0a30 msgr2=0x7f387c120130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3880fd5700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c0a0a30 0x7f387c120130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3880fd5700 1 -- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a1340 msgr2=0x7f387c120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3880fd5700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a1340 0x7f387c120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3880fd5700 1 -- 172.21.15.32:0/3583841008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009b30 con 0x7f387c133fa0 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f387b7fe700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a1340 0x7f387c120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3880fd5700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 0x7f387c120bb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f387400efc0 tx=0x7f387400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:26.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874018070 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387400f040 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c0094a0 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874003c20 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.349+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c0a0220 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.350+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f387400c880 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.350+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f387403f780 con 0x7f387c133fa0 2023-09-30T22:25:26.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.350+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c121e20 con 0x7f387c133fa0 2023-09-30T22:25:26.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.355+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3874014480 con 0x7f387c133fa0 2023-09-30T22:25:26.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.383+0000 7f3abd0d5700 1 Processor -- start 2023-09-30T22:25:26.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.383+0000 7f3abd0d5700 1 -- start start 2023-09-30T22:25:26.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3abd0d5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab80a0a30 0x7f3ab80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3abd0d5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3abd0d5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab809edd0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3abd0d5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab809e1f0 con 0x7f3ab80a0a30 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3abd0d5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab809dc50 con 0x7f3ab80a1340 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab759e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ab8133fa0 0x7f3ab813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51554/0 (socket says 172.21.15.32:51554) 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab759e700 1 -- 172.21.15.32:0/2835083090 learned_addr learned my addr 172.21.15.32:0/2835083090 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab6d9d700 1 --2- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab80a0a30 0x7f3ab80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab659c700 1 -- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab80a0a30 msgr2=0x7f3ab80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab659c700 1 --2- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab80a0a30 0x7f3ab80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab5d9b700 1 -- 172.21.15.32:0/2835083090 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194606930 0 0) 0x7f3ab809edd0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab659c700 1 -- 172.21.15.32:0/2835083090 >> v1:172.21.15.32:6789/0 conn(0x7f3ab8133fa0 legacy=0x7f3ab813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.384+0000 7f3ab659c700 1 -- 172.21.15.32:0/2835083090 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80a1fe0 con 0x7f3ab80a1340 2023-09-30T22:25:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.385+0000 7f3ab659c700 1 --2- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab8133a60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009870 tx=0x7f3aac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db97283c534c08b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.385+0000 7f3ab5d9b700 1 -- 172.21.15.32:0/2835083090 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac011830 con 0x7f3ab80a1340 2023-09-30T22:25:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.385+0000 7f3ab5d9b700 1 -- 172.21.15.32:0/2835083090 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac011990 con 0x7f3ab80a1340 2023-09-30T22:25:26.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.385+0000 7f3ab5d9b700 1 -- 172.21.15.32:0/2835083090 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac011d20 con 0x7f3ab80a1340 2023-09-30T22:25:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 -- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 msgr2=0x7f3ab8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab8133a60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009870 tx=0x7f3aac009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 -- 172.21.15.32:0/2835083090 shutdown_connections 2023-09-30T22:25:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab80a0a30 0x7f3ab80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/2835083090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab8133a60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 -- 172.21.15.32:0/2835083090 >> 172.21.15.32:0/2835083090 conn(0x7f3ab80a6550 msgr2=0x7f3ab80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 -- 172.21.15.32:0/2835083090 shutdown_connections 2023-09-30T22:25:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.386+0000 7f3abd0d5700 1 -- 172.21.15.32:0/2835083090 wait complete. 2023-09-30T22:25:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 Processor -- start 2023-09-30T22:25:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 -- start start 2023-09-30T22:25:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 0x7f3ab81c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab81c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 0x7f3ab81c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab81244c0 con 0x7f3ab80a0a30 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8124d90 con 0x7f3ab8133fa0 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.388+0000 7f3abd0d5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8122d60 con 0x7f3ab80a1340 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 0x7f3ab81c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 0x7f3ab81c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 0x7f3ab81c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55126/0 (socket says 172.21.15.32:55126) 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab6d9d700 1 -- 172.21.15.32:0/203913471 learned_addr learned my addr 172.21.15.32:0/203913471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 0x7f3ab81c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56154/0 (socket says 172.21.15.32:56154) 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab659c700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab81c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:26.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 -- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 msgr2=0x7f3ab81c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab81c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 -- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 msgr2=0x7f3ab81c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 0x7f3ab81c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 -- 172.21.15.32:0/203913471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0094a0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab6d9d700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 0x7f3ab81c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3ab759e700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 0x7f3ab81c8d00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000cbe0 tx=0x7f3ab000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000aa40 con 0x7f3ab8133fa0 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab000abc0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00070d0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.389+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80a25e0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.390+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80a0220 con 0x7f3ab8133fa0 2023-09-30T22:25:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.390+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ab0007230 con 0x7f3ab8133fa0 2023-09-30T22:25:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.391+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3ab003eff0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.391+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab8125d60 con 0x7f3ab8133fa0 2023-09-30T22:25:26.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.395+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ab0018070 con 0x7f3ab8133fa0 2023-09-30T22:25:26.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.563+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f387c123880 con 0x7f387c133fa0 2023-09-30T22:25:26.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.563+0000 7f3878ff9700 1 -- 172.21.15.32:0/3583841008 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f387c123880 con 0x7f387c133fa0 2023-09-30T22:25:26.563 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 msgr2=0x7f387c120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 0x7f387c120bb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f387400efc0 tx=0x7f387400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 shutdown_connections 2023-09-30T22:25:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387c133fa0 0x7f387c120bb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f387c0a0a30 0x7f387c120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 --2- 172.21.15.32:0/3583841008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f387c0a1340 0x7f387c120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 >> 172.21.15.32:0/3583841008 conn(0x7f387c0a6550 msgr2=0x7f387c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 shutdown_connections 2023-09-30T22:25:26.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.566+0000 7f3882a38700 1 -- 172.21.15.32:0/3583841008 wait complete. 2023-09-30T22:25:26.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.603+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ab8074ea0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.603+0000 7f3a9f7fe700 1 -- 172.21.15.32:0/203913471 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ab00074e0 con 0x7f3ab8133fa0 2023-09-30T22:25:26.603 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:26.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 msgr2=0x7f3ab81c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:26.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 0x7f3ab81c8d00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000cbe0 tx=0x7f3ab000cef0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 shutdown_connections 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab8133fa0 0x7f3ab81c8d00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab80a0a30 0x7f3ab81c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 --2- 172.21.15.32:0/203913471 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab80a1340 0x7f3ab81c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 >> 172.21.15.32:0/203913471 conn(0x7f3ab80a6550 msgr2=0x7f3ab811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 shutdown_connections 2023-09-30T22:25:26.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:26.605+0000 7f3abd0d5700 1 -- 172.21.15.32:0/203913471 wait complete. 2023-09-30T22:25:26.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:26.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:26.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.117+0000 7fabbd29c700 1 Processor -- start 2023-09-30T22:25:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.117+0000 7fabbd29c700 1 -- start start 2023-09-30T22:25:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabbd29c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb8130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabbd29c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb809e250 0x7fabb81396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabbd29c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb809c190 con 0x7fabb809d6f0 2023-09-30T22:25:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabbd29c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb806dd10 con 0x7fabb809e250 2023-09-30T22:25:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabbd29c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb809b090 con 0x7fabb8130750 2023-09-30T22:25:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabb77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb809e250 0x7fabb81396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabb77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb809e250 0x7fabb81396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55138/0 (socket says 172.21.15.32:55138) 2023-09-30T22:25:27.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.118+0000 7fabb77fe700 1 -- 172.21.15.32:0/1490838706 learned_addr learned my addr 172.21.15.32:0/1490838706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb6ffd700 1 --2- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb8130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb5ffb700 1 -- 172.21.15.32:0/1490838706 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117266090 0 0) 0x7fabb809c190 con 0x7fabb809d6f0 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb5ffb700 1 -- 172.21.15.32:0/1490838706 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba00035b0 con 0x7fabb809d6f0 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb77fe700 1 -- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 msgr2=0x7fabb8130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb77fe700 1 --2- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb8130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb77fe700 1 -- 172.21.15.32:0/1490838706 >> v1:172.21.15.32:6789/0 conn(0x7fabb809d6f0 legacy=0x7fabb809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.119+0000 7fabb77fe700 1 -- 172.21.15.32:0/1490838706 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb809a6c0 con 0x7fabb809e250 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.120+0000 7fabb77fe700 1 --2- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809e250 0x7fabb81396c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098f0 tx=0x7fabac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd9595685a121 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.120+0000 7fabb5ffb700 1 -- 172.21.15.32:0/1490838706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac004230 con 0x7fabb809e250 2023-09-30T22:25:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.120+0000 7fabb5ffb700 1 -- 172.21.15.32:0/1490838706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac004390 con 0x7fabb809e250 2023-09-30T22:25:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.120+0000 7fabb5ffb700 1 -- 172.21.15.32:0/1490838706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0036a0 con 0x7fabb809e250 2023-09-30T22:25:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 -- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809e250 msgr2=0x7fabb81396c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 --2- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809e250 0x7fabb81396c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098f0 tx=0x7fabac009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 -- 172.21.15.32:0/1490838706 shutdown_connections 2023-09-30T22:25:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 --2- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809e250 0x7fabb81396c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 --2- 172.21.15.32:0/1490838706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb8130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 -- 172.21.15.32:0/1490838706 >> 172.21.15.32:0/1490838706 conn(0x7fabb809f470 msgr2=0x7fabb80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 -- 172.21.15.32:0/1490838706 shutdown_connections 2023-09-30T22:25:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.121+0000 7fabbd29c700 1 -- 172.21.15.32:0/1490838706 wait complete. 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.122+0000 7fabbd29c700 1 Processor -- start 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.122+0000 7fabbd29c700 1 -- start start 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabbd29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809d6f0 0x7fabb8098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabbd29c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb809e250 0x7fabb808c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabbd29c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb808c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabbd29c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8092fc0 con 0x7fabb809d6f0 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabbd29c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb813a8e0 con 0x7fabb809e250 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabbd29c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb808df10 con 0x7fabb8130750 2023-09-30T22:25:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb808c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809d6f0 0x7fabb8098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb808c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53678/0 (socket says 172.21.15.32:53678) 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 -- 172.21.15.32:0/449858972 learned_addr learned my addr 172.21.15.32:0/449858972 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb67fc700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb809e250 0x7fabb808c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 -- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb809e250 msgr2=0x7fabb808c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb809e250 0x7fabb808c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 -- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809d6f0 msgr2=0x7fabb8098990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809d6f0 0x7fabb8098990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 -- 172.21.15.32:0/449858972 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac009550 con 0x7fabb8130750 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.123+0000 7fabb77fe700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb808c9d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabac009130 tx=0x7fabac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.124+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac003bf0 con 0x7fabb8130750 2023-09-30T22:25:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.124+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac003d70 con 0x7fabb8130750 2023-09-30T22:25:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.124+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac011870 con 0x7fabb8130750 2023-09-30T22:25:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.124+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb8098ed0 con 0x7fabb8130750 2023-09-30T22:25:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.124+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb808e3e0 con 0x7fabb8130750 2023-09-30T22:25:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.125+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fabac0119d0 con 0x7fabb8130750 2023-09-30T22:25:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.125+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb80747f0 con 0x7fabb8130750 2023-09-30T22:25:27.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.125+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fabac040b90 con 0x7fabb8130750 2023-09-30T22:25:27.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.130+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabac00b3e0 con 0x7fabb8130750 2023-09-30T22:25:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.156+0000 7f830ed43700 1 Processor -- start 2023-09-30T22:25:27.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830ed43700 1 -- start start 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830ed43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83080a0a30 0x7f83080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830ed43700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f830809d630 0x7f83080a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830ed43700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83080a2a70 con 0x7f8308133930 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830ed43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83080a01d0 con 0x7f83080a0a30 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830ed43700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f830809e2b0 con 0x7f830809d630 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f8307fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f830809d630 0x7f83080a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830d2e0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8308133930 0x7f8308133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51570/0 (socket says 172.21.15.32:51570) 2023-09-30T22:25:27.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830d2e0700 1 -- 172.21.15.32:0/1271326697 learned_addr learned my addr 172.21.15.32:0/1271326697 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.157+0000 7f830cadf700 1 --2- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83080a0a30 0x7f83080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f83077fe700 1 -- 172.21.15.32:0/1271326697 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101962259 0 0) 0x7f83080a2a70 con 0x7f8308133930 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f8307fff700 1 -- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83080a0a30 msgr2=0x7f83080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f8307fff700 1 --2- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83080a0a30 0x7f83080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f8307fff700 1 -- 172.21.15.32:0/1271326697 >> v1:172.21.15.32:6789/0 conn(0x7f8308133930 legacy=0x7f8308133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f8307fff700 1 -- 172.21.15.32:0/1271326697 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830809dc50 con 0x7f830809d630 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f8307fff700 1 --2- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f830809d630 0x7f83080a1340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009870 tx=0x7f82f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d550852dddee1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f83077fe700 1 -- 172.21.15.32:0/1271326697 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4011830 con 0x7f830809d630 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f83077fe700 1 -- 172.21.15.32:0/1271326697 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4011990 con 0x7f830809d630 2023-09-30T22:25:27.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.158+0000 7f83077fe700 1 -- 172.21.15.32:0/1271326697 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4011d20 con 0x7f830809d630 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 -- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f830809d630 msgr2=0x7f83080a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 --2- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f830809d630 0x7f83080a1340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009870 tx=0x7f82f4009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 -- 172.21.15.32:0/1271326697 shutdown_connections 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 --2- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83080a0a30 0x7f83080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 --2- 172.21.15.32:0/1271326697 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f830809d630 0x7f83080a1340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 -- 172.21.15.32:0/1271326697 >> 172.21.15.32:0/1271326697 conn(0x7f83080a3050 msgr2=0x7f83080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 -- 172.21.15.32:0/1271326697 shutdown_connections 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.159+0000 7f830ed43700 1 -- 172.21.15.32:0/1271326697 wait complete. 2023-09-30T22:25:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 Processor -- start 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 -- start start 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f830809d630 0x7f830811f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 0x7f830811f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8308133930 0x7f83081c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830811ed60 con 0x7f83080a0a30 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83081224a0 con 0x7f830809d630 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.160+0000 7f830ed43700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8308120490 con 0x7f8308133930 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f8307fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 0x7f830811f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f830809d630 0x7f830811f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f8307fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 0x7f830811f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55156/0 (socket says 172.21.15.32:55156) 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f8307fff700 1 -- 172.21.15.32:0/4226678290 learned_addr learned my addr 172.21.15.32:0/4226678290 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830d2e0700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8308133930 0x7f83081c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 -- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8308133930 msgr2=0x7f83081c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8308133930 0x7f83081c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 -- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 msgr2=0x7f830811f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 0x7f830811f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 -- 172.21.15.32:0/4226678290 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40094a0 con 0x7f830809d630 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f8307fff700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 0x7f830811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830cadf700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f830809d630 0x7f830811f2e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e900 tx=0x7f82fc00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc013680 con 0x7f830809d630 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc0137e0 con 0x7f830809d630 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83080a42c0 con 0x7f830809d630 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.161+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc013b50 con 0x7f830809d630 2023-09-30T22:25:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.162+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8308121e00 con 0x7f830809d630 2023-09-30T22:25:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.162+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830811fd60 con 0x7f830809d630 2023-09-30T22:25:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.163+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82fc013cb0 con 0x7f830809d630 2023-09-30T22:25:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.163+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f82fc03f650 con 0x7f830809d630 2023-09-30T22:25:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.168+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82fc01bc10 con 0x7f830809d630 2023-09-30T22:25:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.338+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fabb8093af0 con 0x7fabb8130750 2023-09-30T22:25:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.338+0000 7fab9f7fe700 1 -- 172.21.15.32:0/449858972 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabac028070 con 0x7fabb8130750 2023-09-30T22:25:27.339 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 msgr2=0x7fabb808c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb808c9d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabac009130 tx=0x7fabac004750 comp rx=0 tx=0).stop 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 shutdown_connections 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb8130750 0x7fabb808c9d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb809d6f0 0x7fabb8098990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 --2- 172.21.15.32:0/449858972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb809e250 0x7fabb808c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 >> 172.21.15.32:0/449858972 conn(0x7fabb809f470 msgr2=0x7fabb809c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 shutdown_connections 2023-09-30T22:25:27.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.341+0000 7fabbd29c700 1 -- 172.21.15.32:0/449858972 wait complete. 2023-09-30T22:25:27.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.378+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f83081222e0 con 0x7f830809d630 2023-09-30T22:25:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.379+0000 7f83057fa700 1 -- 172.21.15.32:0/4226678290 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83081222e0 con 0x7f830809d630 2023-09-30T22:25:27.379 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f830809d630 msgr2=0x7f830811f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f830809d630 0x7f830811f2e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e900 tx=0x7f82fc00ec10 comp rx=0 tx=0).stop 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 shutdown_connections 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8308133930 0x7f83081c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f830809d630 0x7f830811f2e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 --2- 172.21.15.32:0/4226678290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83080a0a30 0x7f830811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 >> 172.21.15.32:0/4226678290 conn(0x7f83080a3050 msgr2=0x7f83081bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.381+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 shutdown_connections 2023-09-30T22:25:27.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.382+0000 7f830ed43700 1 -- 172.21.15.32:0/4226678290 wait complete. 2023-09-30T22:25:27.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:27.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:27.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 Processor -- start 2023-09-30T22:25:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 -- start start 2023-09-30T22:25:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbea40a0a30 0x7fbea40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea40a2a70 con 0x7fbea4133930 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea40a01d0 con 0x7fbea40a0a30 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.730+0000 7fbeaadf2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbea409e2b0 con 0x7fbea409d630 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbea40a0a30 0x7fbea40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea938f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbea4133930 0x7fbea4133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51576/0 (socket says 172.21.15.32:51576) 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbea40a0a30 0x7fbea40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55166/0 (socket says 172.21.15.32:55166) 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea938f700 1 -- 172.21.15.32:0/2189557876 learned_addr learned my addr 172.21.15.32:0/2189557876 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea3fff700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 -- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 msgr2=0x7fbea40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 -- 172.21.15.32:0/2189557876 >> v1:172.21.15.32:6789/0 conn(0x7fbea4133930 legacy=0x7fbea4133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 -- 172.21.15.32:0/2189557876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea409dc50 con 0x7fbea40a0a30 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea37fe700 1 -- 172.21.15.32:0/2189557876 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749061836 0 0) 0x7fbea40a2a70 con 0x7fbea4133930 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea3fff700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea8b8e700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40a0a30 0x7fbea40a0e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009840 tx=0x7fbe94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a1fbb4faa210ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea37fe700 1 -- 172.21.15.32:0/2189557876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94011830 con 0x7fbea40a0a30 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea37fe700 1 -- 172.21.15.32:0/2189557876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe94011990 con 0x7fbea40a0a30 2023-09-30T22:25:27.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.731+0000 7fbea37fe700 1 -- 172.21.15.32:0/2189557876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94011d20 con 0x7fbea40a0a30 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 -- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40a0a30 msgr2=0x7fbea40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40a0a30 0x7fbea40a0e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009840 tx=0x7fbe94009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 -- 172.21.15.32:0/2189557876 shutdown_connections 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea40a0a30 0x7fbea40a0e00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/2189557876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 -- 172.21.15.32:0/2189557876 >> 172.21.15.32:0/2189557876 conn(0x7fbea40a3050 msgr2=0x7fbea40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 -- 172.21.15.32:0/2189557876 shutdown_connections 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.732+0000 7fbeaadf2700 1 -- 172.21.15.32:0/2189557876 wait complete. 2023-09-30T22:25:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.733+0000 7fbeaadf2700 1 Processor -- start 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.733+0000 7fbeaadf2700 1 -- start start 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbeaadf2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea41c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbeaadf2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea40a0a30 0x7fbea41c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbeaadf2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 0x7fbea41c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbeaadf2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4120620 con 0x7fbea4133930 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbeaadf2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea40a42c0 con 0x7fbea40a0a30 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbeaadf2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbea40a0410 con 0x7fbea409d630 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 0x7fbea41c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 0x7fbea41c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55178/0 (socket says 172.21.15.32:55178) 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea8b8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea41c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 -- 172.21.15.32:0/103066487 learned_addr learned my addr 172.21.15.32:0/103066487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 -- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 msgr2=0x7fbea41c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea41c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 -- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea40a0a30 msgr2=0x7fbea41c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.734+0000 7fbea938f700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea40a0a30 0x7fbea41c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbea938f700 1 -- 172.21.15.32:0/103066487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe980094a0 con 0x7fbea4133930 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbea8b8e700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea41c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbea938f700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 0x7fbea41c8900 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c002230 tx=0x7fbe9c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c00ab70 con 0x7fbea4133930 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c00acf0 con 0x7fbea4133930 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe940094a0 con 0x7fbea4133930 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007310 con 0x7fbea4133930 2023-09-30T22:25:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.735+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea41334c0 con 0x7fbea4133930 2023-09-30T22:25:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.736+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe9c007470 con 0x7fbea4133930 2023-09-30T22:25:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.736+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbe9c03f590 con 0x7fbea4133930 2023-09-30T22:25:27.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.736+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea40747f0 con 0x7fbea4133930 2023-09-30T22:25:27.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.741+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe9c018020 con 0x7fbea4133930 2023-09-30T22:25:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.767+0000 7fa5ee3f9700 1 Processor -- start 2023-09-30T22:25:27.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.767+0000 7fa5ee3f9700 1 -- start start 2023-09-30T22:25:27.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ee3f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5e80a0a30 0x7fa5e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ee3f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 0x7fa5e8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ee3f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 0x7fa5e8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ee3f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e809edd0 con 0x7fa5e80a0a30 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ee3f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e809e1f0 con 0x7fa5e8133fa0 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ee3f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e809dc50 con 0x7fa5e80a1340 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 0x7fa5e8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ec996700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 0x7fa5e8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5e80a0a30 0x7fa5e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 0x7fa5e8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53732/0 (socket says 172.21.15.32:53732) 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ec996700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 0x7fa5e8134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56200/0 (socket says 172.21.15.32:56200) 2023-09-30T22:25:27.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e77fe700 1 -- 172.21.15.32:0/4204408693 learned_addr learned my addr 172.21.15.32:0/4204408693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 -- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 msgr2=0x7fa5e8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 0x7fa5e8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 -- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 msgr2=0x7fa5e8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 0x7fa5e8134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 -- 172.21.15.32:0/4204408693 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e80a1fe0 con 0x7fa5e80a0a30 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5ec996700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 0x7fa5e8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e77fe700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 0x7fa5e8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.768+0000 7fa5e7fff700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e80a0a30 0x7fa5e80a0e00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009840 tx=0x7fa5d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09b952b2a51e85e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5e6ffd700 1 -- 172.21.15.32:0/4204408693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8011830 con 0x7fa5e80a0a30 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5e6ffd700 1 -- 172.21.15.32:0/4204408693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d8011990 con 0x7fa5e80a0a30 2023-09-30T22:25:27.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5e6ffd700 1 -- 172.21.15.32:0/4204408693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8011d20 con 0x7fa5e80a0a30 2023-09-30T22:25:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e80a0a30 msgr2=0x7fa5e80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e80a0a30 0x7fa5e80a0e00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009840 tx=0x7fa5d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/4204408693 shutdown_connections 2023-09-30T22:25:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e8133fa0 0x7fa5e8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e80a0a30 0x7fa5e80a0e00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/4204408693 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a1340 0x7fa5e8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/4204408693 >> 172.21.15.32:0/4204408693 conn(0x7fa5e80a6550 msgr2=0x7fa5e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/4204408693 shutdown_connections 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.769+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/4204408693 wait complete. 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.770+0000 7fa5ee3f9700 1 Processor -- start 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 -- start start 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a0a30 0x7fa5e8120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e80a1340 0x7fa5e8120940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 0x7fa5e8120e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e81337a0 con 0x7fa5e8133fa0 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8123210 con 0x7fa5e80a1340 2023-09-30T22:25:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ee3f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8123bf0 con 0x7fa5e80a0a30 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ec996700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 0x7fa5e8120e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a0a30 0x7fa5e8120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ec996700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 0x7fa5e8120e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55202/0 (socket says 172.21.15.32:55202) 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5ec996700 1 -- 172.21.15.32:0/196370752 learned_addr learned my addr 172.21.15.32:0/196370752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.771+0000 7fa5e77fe700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e80a1340 0x7fa5e8120940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ec996700 1 -- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a0a30 msgr2=0x7fa5e8120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ec996700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a0a30 0x7fa5e8120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ec996700 1 -- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e80a1340 msgr2=0x7fa5e8120940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ec996700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e80a1340 0x7fa5e8120940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ec996700 1 -- 172.21.15.32:0/196370752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009550 con 0x7fa5e8133fa0 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5e7fff700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a0a30 0x7fa5e8120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:27.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ec996700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 0x7fa5e8120e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000efc0 tx=0x7fa5e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000ca90 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e000f040 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000ce10 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e0009a70 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.772+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d80094a0 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.773+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5e0014480 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.773+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa5e003fc70 con 0x7fa5e8133fa0 2023-09-30T22:25:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.773+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e80a5360 con 0x7fa5e8133fa0 2023-09-30T22:25:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.783+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5e0012930 con 0x7fa5e8133fa0 2023-09-30T22:25:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.949+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbea4072980 con 0x7fbea4133930 2023-09-30T22:25:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.950+0000 7fbea17fa700 1 -- 172.21.15.32:0/103066487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe9c01a680 con 0x7fbea4133930 2023-09-30T22:25:27.950 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 msgr2=0x7fbea41c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 0x7fbea41c8900 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c002230 tx=0x7fbe9c00cf50 comp rx=0 tx=0).stop 2023-09-30T22:25:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 shutdown_connections 2023-09-30T22:25:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea4133930 0x7fbea41c8900 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbea409d630 0x7fbea41c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 --2- 172.21.15.32:0/103066487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea40a0a30 0x7fbea41c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 >> 172.21.15.32:0/103066487 conn(0x7fbea40a3050 msgr2=0x7fbea41bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 shutdown_connections 2023-09-30T22:25:27.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.952+0000 7fbeaadf2700 1 -- 172.21.15.32:0/103066487 wait complete. 2023-09-30T22:25:27.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.987+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa5e80a3410 con 0x7fa5e8133fa0 2023-09-30T22:25:27.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.987+0000 7fa5e4ff9700 1 -- 172.21.15.32:0/196370752 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5e001c3b0 con 0x7fa5e8133fa0 2023-09-30T22:25:27.987 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 msgr2=0x7fa5e8120e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 0x7fa5e8120e80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000efc0 tx=0x7fa5e000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 shutdown_connections 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5e8133fa0 0x7fa5e8120e80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5e80a0a30 0x7fa5e8120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 --2- 172.21.15.32:0/196370752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5e80a1340 0x7fa5e8120940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:27.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 >> 172.21.15.32:0/196370752 conn(0x7fa5e80a6550 msgr2=0x7fa5e81bafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:27.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 shutdown_connections 2023-09-30T22:25:27.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:27.990+0000 7fa5ee3f9700 1 -- 172.21.15.32:0/196370752 wait complete. 2023-09-30T22:25:28.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:28.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:28.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:28.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 Processor -- start 2023-09-30T22:25:28.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 -- start start 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdaf80a0a30 0x7fdaf80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf8134170 0x7fdaf813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf809edd0 con 0x7fdaf80a1340 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf809e1f0 con 0x7fdaf80a0a30 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdafdb3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf809dc50 con 0x7fdaf8134170 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdaf77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdaf80a0a30 0x7fdaf80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdaf6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdaf80a1340 0x7fdaf8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51584/0 (socket says 172.21.15.32:51584) 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdaf77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdaf80a0a30 0x7fdaf80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55218/0 (socket says 172.21.15.32:55218) 2023-09-30T22:25:28.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdaf6ffd700 1 -- 172.21.15.32:0/544269326 learned_addr learned my addr 172.21.15.32:0/544269326 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.505+0000 7fdaf7fff700 1 --2- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf8134170 0x7fdaf813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf67fc700 1 -- 172.21.15.32:0/544269326 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165799828 0 0) 0x7fdaf809edd0 con 0x7fdaf80a1340 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf67fc700 1 -- 172.21.15.32:0/544269326 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae00035b0 con 0x7fdaf80a1340 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf77fe700 1 -- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf8134170 msgr2=0x7fdaf813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf77fe700 1 --2- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf8134170 0x7fdaf813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf77fe700 1 -- 172.21.15.32:0/544269326 >> v1:172.21.15.32:6789/0 conn(0x7fdaf80a1340 legacy=0x7fdaf8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf77fe700 1 -- 172.21.15.32:0/544269326 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf80a1fe0 con 0x7fdaf80a0a30 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf77fe700 1 --2- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf80a0a30 0x7fdaf80a0e00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009840 tx=0x7fdae8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497f795079ee94c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf67fc700 1 -- 172.21.15.32:0/544269326 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8011830 con 0x7fdaf80a0a30 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf67fc700 1 -- 172.21.15.32:0/544269326 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8011990 con 0x7fdaf80a0a30 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.506+0000 7fdaf67fc700 1 -- 172.21.15.32:0/544269326 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8011d20 con 0x7fdaf80a0a30 2023-09-30T22:25:28.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 -- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf80a0a30 msgr2=0x7fdaf80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf80a0a30 0x7fdaf80a0e00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009840 tx=0x7fdae8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 -- 172.21.15.32:0/544269326 shutdown_connections 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf8134170 0x7fdaf813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/544269326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf80a0a30 0x7fdaf80a0e00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 -- 172.21.15.32:0/544269326 >> 172.21.15.32:0/544269326 conn(0x7fdaf80a6550 msgr2=0x7fdaf80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 -- 172.21.15.32:0/544269326 shutdown_connections 2023-09-30T22:25:28.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.507+0000 7fdafdb3b700 1 -- 172.21.15.32:0/544269326 wait complete. 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.508+0000 7fdafdb3b700 1 Processor -- start 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.508+0000 7fdafdb3b700 1 -- start start 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdafdb3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 0x7fdaf8122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdafdb3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf80a1340 0x7fdaf8122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdafdb3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf8134170 0x7fdaf81231f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdafdb3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8124b80 con 0x7fdaf8134170 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdafdb3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8123bd0 con 0x7fdaf80a1340 2023-09-30T22:25:28.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdafdb3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8124400 con 0x7fdaf80a0a30 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 0x7fdaf8122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf8134170 0x7fdaf81231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 0x7fdaf8122770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53754/0 (socket says 172.21.15.32:53754) 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf8134170 0x7fdaf81231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55226/0 (socket says 172.21.15.32:55226) 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 -- 172.21.15.32:0/334296319 learned_addr learned my addr 172.21.15.32:0/334296319 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf6ffd700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf80a1340 0x7fdaf8122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 -- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf80a1340 msgr2=0x7fdaf8122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf80a1340 0x7fdaf8122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 -- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf8134170 msgr2=0x7fdaf81231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf8134170 0x7fdaf81231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 -- 172.21.15.32:0/334296319 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80094a0 con 0x7fdaf80a0a30 2023-09-30T22:25:28.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.509+0000 7fdaf77fe700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 0x7fdaf8122770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdae80090b0 tx=0x7fdae8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.510+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae80036a0 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.510+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae801f070 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.510+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8003a60 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.510+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf80a25e0 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.510+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf8125f70 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.510+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdae8003bc0 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.511+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8133260 con 0x7fdaf80a0a30 2023-09-30T22:25:28.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.511+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdae8021070 con 0x7fdaf80a0a30 2023-09-30T22:25:28.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.516+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdae8015080 con 0x7fdaf80a0a30 2023-09-30T22:25:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.543+0000 7f028e05d700 1 Processor -- start 2023-09-30T22:25:28.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 -- start start 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02880a0a30 0x7f02880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02880a1340 0x7f0288133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0288133fa0 0x7f0288134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028809edd0 con 0x7f02880a1340 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028809e1f0 con 0x7f0288133fa0 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f028e05d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f028809dc50 con 0x7f02880a0a30 2023-09-30T22:25:28.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f02877fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02880a0a30 0x7f02880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f027effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02880a1340 0x7f0288133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f0287fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0288133fa0 0x7f0288134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f02877fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02880a0a30 0x7f02880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53766/0 (socket says 172.21.15.32:53766) 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.544+0000 7f02877fe700 1 -- 172.21.15.32:0/991184107 learned_addr learned my addr 172.21.15.32:0/991184107 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0287fff700 1 -- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02880a0a30 msgr2=0x7f02880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0287fff700 1 --2- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02880a0a30 0x7f02880a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0287fff700 1 -- 172.21.15.32:0/991184107 >> v2:172.21.15.32:3300/0 conn(0x7f02880a1340 msgr2=0x7f0288133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0287fff700 1 --2- 172.21.15.32:0/991184107 >> v2:172.21.15.32:3300/0 conn(0x7f02880a1340 0x7f0288133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0287fff700 1 -- 172.21.15.32:0/991184107 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880a1fe0 con 0x7f0288133fa0 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f027effd700 1 --2- 172.21.15.32:0/991184107 >> v2:172.21.15.32:3300/0 conn(0x7f02880a1340 0x7f0288133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:28.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0287fff700 1 --2- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0288133fa0 0x7f0288134410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0280009dc0 tx=0x7f028000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b1e2f222912cac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:28.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0286ffd700 1 -- 172.21.15.32:0/991184107 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000e070 con 0x7f0288133fa0 2023-09-30T22:25:28.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0286ffd700 1 -- 172.21.15.32:0/991184107 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028000bae0 con 0x7f0288133fa0 2023-09-30T22:25:28.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.545+0000 7f0286ffd700 1 -- 172.21.15.32:0/991184107 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000be90 con 0x7f0288133fa0 2023-09-30T22:25:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 -- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0288133fa0 msgr2=0x7f0288134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 --2- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0288133fa0 0x7f0288134410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0280009dc0 tx=0x7f028000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 -- 172.21.15.32:0/991184107 shutdown_connections 2023-09-30T22:25:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 --2- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0288133fa0 0x7f0288134410 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 --2- 172.21.15.32:0/991184107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02880a0a30 0x7f02880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 --2- 172.21.15.32:0/991184107 >> v2:172.21.15.32:3300/0 conn(0x7f02880a1340 0x7f0288133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 -- 172.21.15.32:0/991184107 >> 172.21.15.32:0/991184107 conn(0x7f02880a6550 msgr2=0x7f02880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 -- 172.21.15.32:0/991184107 shutdown_connections 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.546+0000 7f028e05d700 1 -- 172.21.15.32:0/991184107 wait complete. 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.547+0000 7f028e05d700 1 Processor -- start 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.547+0000 7f028e05d700 1 -- start start 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.547+0000 7f028e05d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 0x7f02881200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.547+0000 7f028e05d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02880a1340 0x7f02881205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f028e05d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0288133fa0 0x7f0288120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f028e05d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02881233e0 con 0x7f02880a1340 2023-09-30T22:25:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f028e05d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0288122eb0 con 0x7f02880a0a30 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f028e05d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0288121d00 con 0x7f0288133fa0 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 0x7f02881200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f027effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02880a1340 0x7f02881205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 0x7f02881200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56246/0 (socket says 172.21.15.32:56246) 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 -- 172.21.15.32:0/2556045548 learned_addr learned my addr 172.21.15.32:0/2556045548 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f0287fff700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0288133fa0 0x7f0288120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 -- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0288133fa0 msgr2=0x7f0288120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0288133fa0 0x7f0288120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 -- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02880a1340 msgr2=0x7f02881205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02880a1340 0x7f02881205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 -- 172.21.15.32:0/2556045548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700094a0 con 0x7f02880a0a30 2023-09-30T22:25:28.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f027effd700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02880a1340 0x7f02881205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f02877fe700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 0x7f02881200a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f027800e910 tx=0x7f027800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02780096a0 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0278013680 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0280009a70 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.548+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278013a30 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.549+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02881238e0 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.549+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0278013b90 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.549+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f027803f790 con 0x7f02880a0a30 2023-09-30T22:25:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.550+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02881238e0 con 0x7f02880a0a30 2023-09-30T22:25:28.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.555+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02881238e0 con 0x7f02880a0a30 2023-09-30T22:25:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.725+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdaf8125f70 con 0x7fdaf80a0a30 2023-09-30T22:25:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.725+0000 7fdadffff700 1 -- 172.21.15.32:0/334296319 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdaf8125f70 con 0x7fdaf80a0a30 2023-09-30T22:25:28.725 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 msgr2=0x7fdaf8122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 0x7fdaf8122770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdae80090b0 tx=0x7fdae8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 shutdown_connections 2023-09-30T22:25:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf8134170 0x7fdaf81231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf80a0a30 0x7fdaf8122770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 --2- 172.21.15.32:0/334296319 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf80a1340 0x7fdaf8122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.727+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 >> 172.21.15.32:0/334296319 conn(0x7fdaf80a6550 msgr2=0x7fdaf809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.728+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 shutdown_connections 2023-09-30T22:25:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.728+0000 7fdafdb3b700 1 -- 172.21.15.32:0/334296319 wait complete. 2023-09-30T22:25:28.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:28.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.761+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02881238e0 con 0x7f02880a0a30 2023-09-30T22:25:28.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.761+0000 7f0284ff9700 1 -- 172.21.15.32:0/2556045548 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02881238e0 con 0x7f02880a0a30 2023-09-30T22:25:28.761 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 msgr2=0x7f02881200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:28.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 0x7f02881200a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f027800e910 tx=0x7f027800ec20 comp rx=0 tx=0).stop 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 shutdown_connections 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0288133fa0 0x7f0288120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02880a0a30 0x7f02881200a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 --2- 172.21.15.32:0/2556045548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02880a1340 0x7f02881205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.763+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 >> 172.21.15.32:0/2556045548 conn(0x7f02880a6550 msgr2=0x7f02881baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.764+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 shutdown_connections 2023-09-30T22:25:28.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:28.764+0000 7f028e05d700 1 -- 172.21.15.32:0/2556045548 wait complete. 2023-09-30T22:25:28.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:28.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:29.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.117+0000 7f247b8ec700 1 Processor -- start 2023-09-30T22:25:29.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.117+0000 7f247b8ec700 1 -- start start 2023-09-30T22:25:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.118+0000 7f247b8ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2474130540 0x7f2474130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.118+0000 7f247b8ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 0x7f2474138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.118+0000 7f247b8ec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24740716a0 con 0x7f2474130e50 2023-09-30T22:25:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.118+0000 7f247b8ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247409a390 con 0x7f2474130540 2023-09-30T22:25:29.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.118+0000 7f247b8ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2474071290 con 0x7f247406fcf0 2023-09-30T22:25:29.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479e89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 0x7f2474138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479688700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2474130540 0x7f2474130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479e89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 0x7f2474138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53792/0 (socket says 172.21.15.32:53792) 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479e89700 1 -- 172.21.15.32:0/3243696991 learned_addr learned my addr 172.21.15.32:0/3243696991 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479688700 1 -- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 msgr2=0x7f2474138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479688700 1 --2- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 0x7f2474138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479688700 1 -- 172.21.15.32:0/3243696991 >> v1:172.21.15.32:6789/0 conn(0x7f2474130e50 legacy=0x7f247406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479688700 1 -- 172.21.15.32:0/3243696991 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474130340 con 0x7f2474130540 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479e89700 1 --2- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 0x7f2474138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.119+0000 7f2479688700 1 --2- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2474130540 0x7f2474130910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f247000dcc0 tx=0x7f247000f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566c7c6fff0aa79f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.120+0000 7f246bfff700 1 -- 172.21.15.32:0/3243696991 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470012070 con 0x7f2474130540 2023-09-30T22:25:29.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.120+0000 7f246bfff700 1 -- 172.21.15.32:0/3243696991 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247000d2e0 con 0x7f2474130540 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.120+0000 7f246bfff700 1 -- 172.21.15.32:0/3243696991 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000f8e0 con 0x7f2474130540 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 -- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2474130540 msgr2=0x7f2474130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 --2- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2474130540 0x7f2474130910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f247000dcc0 tx=0x7f247000f050 comp rx=0 tx=0).stop 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 -- 172.21.15.32:0/3243696991 shutdown_connections 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 --2- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f247406fcf0 0x7f2474138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 --2- 172.21.15.32:0/3243696991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2474130540 0x7f2474130910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 -- 172.21.15.32:0/3243696991 >> 172.21.15.32:0/3243696991 conn(0x7f24740a2aa0 msgr2=0x7f24740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 -- 172.21.15.32:0/3243696991 shutdown_connections 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.121+0000 7f247b8ec700 1 -- 172.21.15.32:0/3243696991 wait complete. 2023-09-30T22:25:29.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 Processor -- start 2023-09-30T22:25:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 -- start start 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f247406fcf0 0x7f247409c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 0x7f247409c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2474130e50 0x7f247409cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2474118ff0 con 0x7f2474130e50 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2474119c50 con 0x7f247406fcf0 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.122+0000 7f247b8ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24740a0eb0 con 0x7f2474130540 2023-09-30T22:25:29.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2479688700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f247406fcf0 0x7f247409c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 0x7f247409c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 0x7f247409c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53808/0 (socket says 172.21.15.32:53808) 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 -- 172.21.15.32:0/1547903709 learned_addr learned my addr 172.21.15.32:0/1547903709 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2479688700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f247406fcf0 0x7f247409c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56276/0 (socket says 172.21.15.32:56276) 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2479e89700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2474130e50 0x7f247409cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 -- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f247406fcf0 msgr2=0x7f247409c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f247406fcf0 0x7f247409c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 -- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2474130e50 msgr2=0x7f247409cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2474130e50 0x7f247409cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 -- 172.21.15.32:0/1547903709 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c009b30 con 0x7f2474130540 2023-09-30T22:25:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2479e89700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2474130e50 0x7f247409cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2478e87700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 0x7f247409c650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f246400c500 tx=0x7f246400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24640052d0 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2464005430 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247000d940 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.123+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464006aa0 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.124+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247409dcd0 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.124+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2464003680 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.125+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f246403e8b0 con 0x7f2474130540 2023-09-30T22:25:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.125+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2474074ea0 con 0x7f2474130540 2023-09-30T22:25:29.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.130+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2464019b60 con 0x7f2474130540 2023-09-30T22:25:29.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 Processor -- start 2023-09-30T22:25:29.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 -- start start 2023-09-30T22:25:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1a00a0a30 0x7fc1a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 0x7fc1a0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a009edd0 con 0x7fc1a0133fa0 2023-09-30T22:25:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a009e1f0 con 0x7fc1a00a0a30 2023-09-30T22:25:29.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a7d2f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a009dc50 con 0x7fc1a00a1340 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a52ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 0x7fc1a0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.156+0000 7fc1a5acb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1a00a0a30 0x7fc1a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 0x7fc1a0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53816/0 (socket says 172.21.15.32:53816) 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 -- 172.21.15.32:0/4108110353 learned_addr learned my addr 172.21.15.32:0/4108110353 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a62cc700 1 --1- 172.21.15.32:0/4108110353 >> v1:172.21.15.32:6789/0 conn(0x7fc1a0133fa0 0x7fc1a013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51588/0 (socket says 172.21.15.32:51588) 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 -- 172.21.15.32:0/4108110353 >> v2:172.21.15.32:3300/0 conn(0x7fc1a00a0a30 msgr2=0x7fc1a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 --2- 172.21.15.32:0/4108110353 >> v2:172.21.15.32:3300/0 conn(0x7fc1a00a0a30 0x7fc1a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 -- 172.21.15.32:0/4108110353 >> v1:172.21.15.32:6789/0 conn(0x7fc1a0133fa0 legacy=0x7fc1a013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 -- 172.21.15.32:0/4108110353 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00a1fe0 con 0x7fc1a00a1340 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a4ac9700 1 -- 172.21.15.32:0/4108110353 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443711939 0 0) 0x7fc1a009edd0 con 0x7fc1a0133fa0 2023-09-30T22:25:29.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a5acb700 1 --2- 172.21.15.32:0/4108110353 >> v2:172.21.15.32:3300/0 conn(0x7fc1a00a0a30 0x7fc1a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a52ca700 1 --2- 172.21.15.32:0/4108110353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 0x7fc1a0133a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1940098f0 tx=0x7fc194009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c8bb65e1ec7d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a4ac9700 1 -- 172.21.15.32:0/4108110353 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194011830 con 0x7fc1a00a1340 2023-09-30T22:25:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a4ac9700 1 -- 172.21.15.32:0/4108110353 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc194011990 con 0x7fc1a00a1340 2023-09-30T22:25:29.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.157+0000 7fc1a4ac9700 1 -- 172.21.15.32:0/4108110353 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194011d20 con 0x7fc1a00a1340 2023-09-30T22:25:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/4108110353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 msgr2=0x7fc1a0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/4108110353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 0x7fc1a0133a60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1940098f0 tx=0x7fc194009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/4108110353 shutdown_connections 2023-09-30T22:25:29.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/4108110353 >> v2:172.21.15.32:3300/0 conn(0x7fc1a00a0a30 0x7fc1a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/4108110353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a00a1340 0x7fc1a0133a60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/4108110353 >> 172.21.15.32:0/4108110353 conn(0x7fc1a00a6550 msgr2=0x7fc1a00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/4108110353 shutdown_connections 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.158+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/4108110353 wait complete. 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.159+0000 7fc1a7d2f700 1 Processor -- start 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.159+0000 7fc1a7d2f700 1 -- start start 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a7d2f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1a00a0a30 0x7fc1a011e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a7d2f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 0x7fc1a011ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a7d2f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a0133fa0 0x7fc1a011f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a7d2f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0120200 con 0x7fc1a00a1340 2023-09-30T22:25:29.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a7d2f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00a0220 con 0x7fc1a00a0a30 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a7d2f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0133580 con 0x7fc1a0133fa0 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a52ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 0x7fc1a011ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a62cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a0133fa0 0x7fc1a011f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a52ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 0x7fc1a011ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55266/0 (socket says 172.21.15.32:55266) 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.160+0000 7fc1a52ca700 1 -- 172.21.15.32:0/1292955736 learned_addr learned my addr 172.21.15.32:0/1292955736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a52ca700 1 -- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a0133fa0 msgr2=0x7fc1a011f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a52ca700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a0133fa0 0x7fc1a011f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a52ca700 1 -- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1a00a0a30 msgr2=0x7fc1a011e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a52ca700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1a00a0a30 0x7fc1a011e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a52ca700 1 -- 172.21.15.32:0/1292955736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1900094a0 con 0x7fc1a00a1340 2023-09-30T22:25:29.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a52ca700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 0x7fc1a011ebd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1a009dd90 tx=0x7fc194004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194004380 con 0x7fc1a00a1340 2023-09-30T22:25:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19401f070 con 0x7fc1a00a1340 2023-09-30T22:25:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc194009550 con 0x7fc1a00a1340 2023-09-30T22:25:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940036a0 con 0x7fc1a00a1340 2023-09-30T22:25:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.161+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a00a35e0 con 0x7fc1a00a1340 2023-09-30T22:25:29.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.162+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc194021030 con 0x7fc1a00a1340 2023-09-30T22:25:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.162+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a0120a30 con 0x7fc1a00a1340 2023-09-30T22:25:29.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.162+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc1940409a0 con 0x7fc1a00a1340 2023-09-30T22:25:29.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.172+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1940199a0 con 0x7fc1a00a1340 2023-09-30T22:25:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.344+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2474119e70 con 0x7f2474130540 2023-09-30T22:25:29.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.344+0000 7f2469ffb700 1 -- 172.21.15.32:0/1547903709 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2474119e70 con 0x7f2474130540 2023-09-30T22:25:29.345 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 msgr2=0x7f247409c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 0x7f247409c650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f246400c500 tx=0x7f246400c810 comp rx=0 tx=0).stop 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 shutdown_connections 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2474130e50 0x7f247409cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f247406fcf0 0x7f247409c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 --2- 172.21.15.32:0/1547903709 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2474130540 0x7f247409c650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 >> 172.21.15.32:0/1547903709 conn(0x7f24740a2aa0 msgr2=0x7f247412b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 shutdown_connections 2023-09-30T22:25:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.348+0000 7f247b8ec700 1 -- 172.21.15.32:0/1547903709 wait complete. 2023-09-30T22:25:29.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:29.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.381+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc1a0074ea0 con 0x7fc1a00a1340 2023-09-30T22:25:29.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.381+0000 7fc19e7fc700 1 -- 172.21.15.32:0/1292955736 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc194015070 con 0x7fc1a00a1340 2023-09-30T22:25:29.381 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 msgr2=0x7fc1a011ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 0x7fc1a011ebd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1a009dd90 tx=0x7fc194004750 comp rx=0 tx=0).stop 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 shutdown_connections 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc1a0133fa0 0x7fc1a011f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1a00a0a30 0x7fc1a011e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 --2- 172.21.15.32:0/1292955736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1a00a1340 0x7fc1a011ebd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 >> 172.21.15.32:0/1292955736 conn(0x7fc1a00a6550 msgr2=0x7fc1a01b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 shutdown_connections 2023-09-30T22:25:29.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.383+0000 7fc1a7d2f700 1 -- 172.21.15.32:0/1292955736 wait complete. 2023-09-30T22:25:29.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:29.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:29.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.891+0000 7f1dbce0c700 1 Processor -- start 2023-09-30T22:25:29.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.891+0000 7f1dbce0c700 1 -- start start 2023-09-30T22:25:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.891+0000 7f1dbce0c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1dbce0c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db80a1340 0x7f1db8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1dbce0c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db809edd0 con 0x7f1db8133fa0 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1dbce0c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db809e1f0 con 0x7f1db80a0a30 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1dbce0c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1db809dc50 con 0x7f1db80a1340 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db80a1340 0x7f1db8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55268/0 (socket says 172.21.15.32:55268) 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db80a1340 0x7f1db8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53824/0 (socket says 172.21.15.32:53824) 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 -- 172.21.15.32:0/4077105501 learned_addr learned my addr 172.21.15.32:0/4077105501 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db6d9d700 1 --1- 172.21.15.32:0/4077105501 >> v1:172.21.15.32:6789/0 conn(0x7f1db8133fa0 0x7f1db813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51600/0 (socket says 172.21.15.32:51600) 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 -- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db80a1340 msgr2=0x7f1db8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 --2- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db80a1340 0x7f1db8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 -- 172.21.15.32:0/4077105501 >> v1:172.21.15.32:6789/0 conn(0x7f1db8133fa0 legacy=0x7f1db813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 -- 172.21.15.32:0/4077105501 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db80a1fe0 con 0x7f1db80a0a30 2023-09-30T22:25:29.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db559a700 1 -- 172.21.15.32:0/4077105501 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55476064 0 0) 0x7f1db809edd0 con 0x7f1db8133fa0 2023-09-30T22:25:29.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db659c700 1 --2- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098f0 tx=0x7f1da8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cde5eb282c6415 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.892+0000 7f1db559a700 1 -- 172.21.15.32:0/4077105501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8011830 con 0x7f1db80a0a30 2023-09-30T22:25:29.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1db559a700 1 -- 172.21.15.32:0/4077105501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da8011990 con 0x7f1db80a0a30 2023-09-30T22:25:29.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1db559a700 1 -- 172.21.15.32:0/4077105501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8011d20 con 0x7f1db80a0a30 2023-09-30T22:25:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 -- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 msgr2=0x7f1db80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098f0 tx=0x7f1da8009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 -- 172.21.15.32:0/4077105501 shutdown_connections 2023-09-30T22:25:29.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/4077105501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db80a1340 0x7f1db8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 -- 172.21.15.32:0/4077105501 >> 172.21.15.32:0/4077105501 conn(0x7f1db80a6550 msgr2=0x7f1db80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 -- 172.21.15.32:0/4077105501 shutdown_connections 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.893+0000 7f1dbce0c700 1 -- 172.21.15.32:0/4077105501 wait complete. 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 Processor -- start 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 -- start start 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db8122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a1340 0x7f1db8122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133fa0 0x7f1db8123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8124cb0 con 0x7f1db80a0a30 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8125d80 con 0x7f1db80a1340 2023-09-30T22:25:29.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1dbce0c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8124520 con 0x7f1db8133fa0 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1db659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db8122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1db5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a1340 0x7f1db8122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1db659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db8122890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55282/0 (socket says 172.21.15.32:55282) 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1db659c700 1 -- 172.21.15.32:0/1372206626 learned_addr learned my addr 172.21.15.32:0/1372206626 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.895+0000 7f1db6d9d700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133fa0 0x7f1db8123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db5d9b700 1 -- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133fa0 msgr2=0x7f1db8123310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db5d9b700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133fa0 0x7f1db8123310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db5d9b700 1 -- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 msgr2=0x7f1db8122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db5d9b700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db8122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db5d9b700 1 -- 172.21.15.32:0/1372206626 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da8009550 con 0x7f1db80a1340 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db6d9d700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133fa0 0x7f1db8123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db659c700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db8122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1db5d9b700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a1340 0x7f1db8122dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e900 tx=0x7f1dac00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac013680 con 0x7f1db80a1340 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac0137e0 con 0x7f1db80a1340 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db80a35a0 con 0x7f1db80a1340 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac013b50 con 0x7f1db80a1340 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.896+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db8133300 con 0x7f1db80a1340 2023-09-30T22:25:29.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.897+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db80a1d40 con 0x7f1db80a1340 2023-09-30T22:25:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.898+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1dac013cb0 con 0x7f1db80a1340 2023-09-30T22:25:29.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.898+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1dac03fec0 con 0x7f1db80a1340 2023-09-30T22:25:29.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.902+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1dac01a360 con 0x7f1db80a1340 2023-09-30T22:25:29.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.930+0000 7ff19631b700 1 Processor -- start 2023-09-30T22:25:29.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.930+0000 7ff19631b700 1 -- start start 2023-09-30T22:25:29.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff19631b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 0x7ff1900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff19631b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff190134170 0x7ff19013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff19631b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19009edd0 con 0x7ff1900a1340 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff19631b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19009e1f0 con 0x7ff190134170 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff19631b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19009dc50 con 0x7ff1900a0a30 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff190134170 0x7ff19013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff18ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 0x7ff1900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff190134170 0x7ff19013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55288/0 (socket says 172.21.15.32:55288) 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff18f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff1900a1340 0x7ff190133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51602/0 (socket says 172.21.15.32:51602) 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 -- 172.21.15.32:0/523645015 learned_addr learned my addr 172.21.15.32:0/523645015 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff18ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 0x7ff1900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56322/0 (socket says 172.21.15.32:56322) 2023-09-30T22:25:29.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 -- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 msgr2=0x7ff1900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 --2- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 0x7ff1900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 -- 172.21.15.32:0/523645015 >> v1:172.21.15.32:6789/0 conn(0x7ff1900a1340 legacy=0x7ff190133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff1948b8700 1 -- 172.21.15.32:0/523645015 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900a1fe0 con 0x7ff190134170 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.931+0000 7ff18ffff700 1 --2- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 0x7ff1900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff1948b8700 1 --2- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19013bd30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff188009e50 tx=0x7ff18800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89ebdd6f90a9f4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff18effd700 1 -- 172.21.15.32:0/523645015 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800e070 con 0x7ff190134170 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff18effd700 1 -- 172.21.15.32:0/523645015 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18800b910 con 0x7ff190134170 2023-09-30T22:25:29.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff18effd700 1 -- 172.21.15.32:0/523645015 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800bcc0 con 0x7ff190134170 2023-09-30T22:25:29.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 -- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 msgr2=0x7ff19013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 --2- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19013bd30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff188009e50 tx=0x7ff18800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:29.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 -- 172.21.15.32:0/523645015 shutdown_connections 2023-09-30T22:25:29.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 --2- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19013bd30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 --2- 172.21.15.32:0/523645015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a0a30 0x7ff1900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 -- 172.21.15.32:0/523645015 >> 172.21.15.32:0/523645015 conn(0x7ff1900a6550 msgr2=0x7ff1900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:29.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.932+0000 7ff19631b700 1 -- 172.21.15.32:0/523645015 shutdown_connections 2023-09-30T22:25:29.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.933+0000 7ff19631b700 1 -- 172.21.15.32:0/523645015 wait complete. 2023-09-30T22:25:29.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.934+0000 7ff19631b700 1 Processor -- start 2023-09-30T22:25:29.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.934+0000 7ff19631b700 1 -- start start 2023-09-30T22:25:29.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff19631b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1900a0a30 0x7ff19011e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff19631b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 0x7ff19011ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff19631b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19011f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:29.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff19631b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190122190 con 0x7ff190134170 2023-09-30T22:25:29.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff19631b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff190121bf0 con 0x7ff1900a1340 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff19631b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff190120280 con 0x7ff1900a0a30 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff18f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 0x7ff19011ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff1948b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19011f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff18f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 0x7ff19011ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56328/0 (socket says 172.21.15.32:56328) 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff18f7fe700 1 -- 172.21.15.32:0/17356055 learned_addr learned my addr 172.21.15.32:0/17356055 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff18ffff700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1900a0a30 0x7ff19011e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff1948b8700 1 -- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1900a0a30 msgr2=0x7ff19011e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.935+0000 7ff1948b8700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1900a0a30 0x7ff19011e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff1948b8700 1 -- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 msgr2=0x7ff19011ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff1948b8700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 0x7ff19011ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:29.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff1948b8700 1 -- 172.21.15.32:0/17356055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800094a0 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff18f7fe700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 0x7ff19011ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff1948b8700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19011f180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff188015000 tx=0x7ff188004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188004800 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188004960 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff188009b30 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18801c680 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.936+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff190120a30 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.937+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff190074ea0 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.937+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff18801c7e0 con 0x7ff190134170 2023-09-30T22:25:29.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.938+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff188041400 con 0x7ff190134170 2023-09-30T22:25:29.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:29.947+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1880243b0 con 0x7ff190134170 2023-09-30T22:25:30.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.114+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1db8074ea0 con 0x7f1db80a1340 2023-09-30T22:25:30.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.114+0000 7f1da6ffd700 1 -- 172.21.15.32:0/1372206626 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1dac021020 con 0x7f1db80a1340 2023-09-30T22:25:30.114 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:30.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a1340 msgr2=0x7f1db8122dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a1340 0x7f1db8122dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e900 tx=0x7f1dac00ec10 comp rx=0 tx=0).stop 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 shutdown_connections 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133fa0 0x7f1db8123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db80a0a30 0x7f1db8122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 --2- 172.21.15.32:0/1372206626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a1340 0x7f1db8122dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.116+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 >> 172.21.15.32:0/1372206626 conn(0x7f1db80a6550 msgr2=0x7f1db811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.117+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 shutdown_connections 2023-09-30T22:25:30.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.117+0000 7f1dbce0c700 1 -- 172.21.15.32:0/1372206626 wait complete. 2023-09-30T22:25:30.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:30.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.152+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff1900a28e0 con 0x7ff190134170 2023-09-30T22:25:30.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.152+0000 7ff18cff9700 1 -- 172.21.15.32:0/17356055 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff188023b50 con 0x7ff190134170 2023-09-30T22:25:30.152 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:30.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.154+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 msgr2=0x7ff19011f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.154+0000 7ff19631b700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19011f180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff188015000 tx=0x7ff188004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.154+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 shutdown_connections 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.154+0000 7ff19631b700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff190134170 0x7ff19011f180 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.155+0000 7ff19631b700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1900a0a30 0x7ff19011e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.155+0000 7ff19631b700 1 --2- 172.21.15.32:0/17356055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1900a1340 0x7ff19011ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.155+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 >> 172.21.15.32:0/17356055 conn(0x7ff1900a6550 msgr2=0x7ff19009c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.155+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 shutdown_connections 2023-09-30T22:25:30.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.155+0000 7ff19631b700 1 -- 172.21.15.32:0/17356055 wait complete. 2023-09-30T22:25:30.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:30.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:30.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.515+0000 7f7ef08fe700 1 Processor -- start 2023-09-30T22:25:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.515+0000 7f7ef08fe700 1 -- start start 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ef08fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7eec0a0a30 0x7f7eec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ef08fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec138d80 0x7f7eec13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ef08fe700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec0a2a70 con 0x7f7eec09d630 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ef08fe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0a01d0 con 0x7f7eec0a0a30 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ef08fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec09e2b0 con 0x7f7eec138d80 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7eead9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec138d80 0x7f7eec13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7eea59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7eec0a0a30 0x7f7eec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ee9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7eec09d630 0x7f7eec0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56416/0 (socket says 172.21.15.32:56416) 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.516+0000 7f7ee9d9b700 1 -- 172.21.15.32:0/4164551383 learned_addr learned my addr 172.21.15.32:0/4164551383 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7ee959a700 1 -- 172.21.15.32:0/4164551383 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842671997 0 0) 0x7f7eec0a2a70 con 0x7f7eec09d630 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7eea59c700 1 -- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec138d80 msgr2=0x7f7eec13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7eea59c700 1 --2- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec138d80 0x7f7eec13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7eea59c700 1 -- 172.21.15.32:0/4164551383 >> v1:172.21.15.32:6789/0 conn(0x7f7eec09d630 legacy=0x7f7eec0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7eea59c700 1 -- 172.21.15.32:0/4164551383 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec09dc50 con 0x7f7eec0a0a30 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7eead9d700 1 --2- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec138d80 0x7f7eec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7eea59c700 1 --2- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec0a0a30 0x7f7eec0a0e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009d90 tx=0x7f7edc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43157ff5e4b2e8df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7ee959a700 1 -- 172.21.15.32:0/4164551383 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00e070 con 0x7f7eec0a0a30 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7ee959a700 1 -- 172.21.15.32:0/4164551383 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc00bb20 con 0x7f7eec0a0a30 2023-09-30T22:25:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.517+0000 7f7ee959a700 1 -- 172.21.15.32:0/4164551383 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc012430 con 0x7f7eec0a0a30 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 -- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec0a0a30 msgr2=0x7f7eec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec0a0a30 0x7f7eec0a0e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009d90 tx=0x7f7edc00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 -- 172.21.15.32:0/4164551383 shutdown_connections 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec138d80 0x7f7eec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/4164551383 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec0a0a30 0x7f7eec0a0e00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 -- 172.21.15.32:0/4164551383 >> 172.21.15.32:0/4164551383 conn(0x7f7eec0a3050 msgr2=0x7f7eec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 -- 172.21.15.32:0/4164551383 shutdown_connections 2023-09-30T22:25:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.518+0000 7f7ef08fe700 1 -- 172.21.15.32:0/4164551383 wait complete. 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.519+0000 7f7ef08fe700 1 Processor -- start 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.519+0000 7f7ef08fe700 1 -- start start 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ef08fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 0x7f7eec133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ef08fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec0a0a30 0x7f7eec133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ef08fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7eec138d80 0x7f7eec1341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ef08fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec136fa0 con 0x7f7eec09d630 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ef08fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec136630 con 0x7f7eec138d80 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ef08fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0a4b50 con 0x7f7eec0a0a30 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 0x7f7eec133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7ee9d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec0a0a30 0x7f7eec133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 0x7f7eec133760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53780/0 (socket says 172.21.15.32:53780) 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 -- 172.21.15.32:0/2706364061 learned_addr learned my addr 172.21.15.32:0/2706364061 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 -- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec0a0a30 msgr2=0x7f7eec133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec0a0a30 0x7f7eec133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 -- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7eec138d80 msgr2=0x7f7eec1341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7eec138d80 0x7f7eec1341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 -- 172.21.15.32:0/2706364061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee40094a0 con 0x7f7eec09d630 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.520+0000 7f7eea59c700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 0x7f7eec133760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7edc014040 tx=0x7f7edc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.521+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc012430 con 0x7f7eec09d630 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.521+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc020070 con 0x7f7eec09d630 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.521+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc009a70 con 0x7f7eec09d630 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.521+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc004280 con 0x7f7eec09d630 2023-09-30T22:25:30.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.521+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec137a10 con 0x7f7eec09d630 2023-09-30T22:25:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.522+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7edc0043e0 con 0x7f7eec09d630 2023-09-30T22:25:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.522+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec137e60 con 0x7f7eec09d630 2023-09-30T22:25:30.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.522+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7edc022070 con 0x7f7eec09d630 2023-09-30T22:25:30.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.527+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7edc0048d0 con 0x7f7eec09d630 2023-09-30T22:25:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 Processor -- start 2023-09-30T22:25:30.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 -- start start 2023-09-30T22:25:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05901337f0 0x7f0590138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059006b990 con 0x7f0590133420 2023-09-30T22:25:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059009b0e0 con 0x7f05901337f0 2023-09-30T22:25:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.553+0000 7f0596c67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0590133c20 con 0x7f0590138dc0 2023-09-30T22:25:30.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05901337f0 0x7f0590138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05901337f0 0x7f0590138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53790/0 (socket says 172.21.15.32:53790) 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 -- 172.21.15.32:0/2090151511 learned_addr learned my addr 172.21.15.32:0/2090151511 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f0595204700 1 --2- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 -- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 msgr2=0x7f059013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 --2- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 -- 172.21.15.32:0/2090151511 >> v1:172.21.15.32:6789/0 conn(0x7f0590133420 legacy=0x7f0590133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 -- 172.21.15.32:0/2090151511 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900a0590 con 0x7f05901337f0 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058f7fe700 1 -- 172.21.15.32:0/2090151511 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323904089 0 0) 0x7f059006b990 con 0x7f0590133420 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058ffff700 1 --2- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f0590138880 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f05840098f0 tx=0x7f0584009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c98ea07a3f7344e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058f7fe700 1 -- 172.21.15.32:0/2090151511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584011830 con 0x7f05901337f0 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058f7fe700 1 -- 172.21.15.32:0/2090151511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0584011990 con 0x7f05901337f0 2023-09-30T22:25:30.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.554+0000 7f058f7fe700 1 -- 172.21.15.32:0/2090151511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584011d20 con 0x7f05901337f0 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 -- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 msgr2=0x7f0590138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 --2- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f0590138880 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f05840098f0 tx=0x7f0584009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 -- 172.21.15.32:0/2090151511 shutdown_connections 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 --2- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 --2- 172.21.15.32:0/2090151511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f0590138880 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 -- 172.21.15.32:0/2090151511 >> 172.21.15.32:0/2090151511 conn(0x7f05900a4210 msgr2=0x7f05900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:30.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 -- 172.21.15.32:0/2090151511 shutdown_connections 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.555+0000 7f0596c67700 1 -- 172.21.15.32:0/2090151511 wait complete. 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.556+0000 7f0596c67700 1 Processor -- start 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.556+0000 7f0596c67700 1 -- start start 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0596c67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0590133420 0x7f059012ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0596c67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f059012f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0596c67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059012f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0596c67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059009c920 con 0x7f05901337f0 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0596c67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059009e400 con 0x7f0590138dc0 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0596c67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05901389c0 con 0x7f0590133420 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f058ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f059012f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0595204700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059012f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f058ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f059012f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53792/0 (socket says 172.21.15.32:53792) 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0595204700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059012f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38518/0 (socket says 172.21.15.32:38518) 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f058ffff700 1 -- 172.21.15.32:0/2946258679 learned_addr learned my addr 172.21.15.32:0/2946258679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.557+0000 7f0594a03700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0590133420 0x7f059012ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0595204700 1 -- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0590133420 msgr2=0x7f059012ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0595204700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0590133420 0x7f059012ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0595204700 1 -- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 msgr2=0x7f059012f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0595204700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f059012f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0595204700 1 -- 172.21.15.32:0/2946258679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0584009550 con 0x7f0590138dc0 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0595204700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059012f8d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f058800ef90 tx=0x7f058800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058800cc90 con 0x7f0590138dc0 2023-09-30T22:25:30.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058800f040 con 0x7f0590138dc0 2023-09-30T22:25:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059009beb0 con 0x7f0590138dc0 2023-09-30T22:25:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880146c0 con 0x7f0590138dc0 2023-09-30T22:25:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.558+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059009d390 con 0x7f0590138dc0 2023-09-30T22:25:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.559+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0588014820 con 0x7f0590138dc0 2023-09-30T22:25:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.559+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0588040200 con 0x7f0590138dc0 2023-09-30T22:25:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.560+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059009c760 con 0x7f0590138dc0 2023-09-30T22:25:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.565+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0588018020 con 0x7f0590138dc0 2023-09-30T22:25:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.733+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7eec0747f0 con 0x7f7eec09d630 2023-09-30T22:25:30.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.734+0000 7f7edaffd700 1 -- 172.21.15.32:0/2706364061 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7edc02c070 con 0x7f7eec09d630 2023-09-30T22:25:30.734 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 msgr2=0x7f7eec133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 0x7f7eec133760 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7edc014040 tx=0x7f7edc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 shutdown_connections 2023-09-30T22:25:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7eec138d80 0x7f7eec1341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eec09d630 0x7f7eec133760 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 --2- 172.21.15.32:0/2706364061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7eec0a0a30 0x7f7eec133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 >> 172.21.15.32:0/2706364061 conn(0x7f7eec0a3050 msgr2=0x7f7eec13ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 shutdown_connections 2023-09-30T22:25:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.736+0000 7f7ef08fe700 1 -- 172.21.15.32:0/2706364061 wait complete. 2023-09-30T22:25:30.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.772+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0590074ea0 con 0x7f0590138dc0 2023-09-30T22:25:30.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.772+0000 7f058d7fa700 1 -- 172.21.15.32:0/2946258679 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f058802b090 con 0x7f0590138dc0 2023-09-30T22:25:30.772 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 msgr2=0x7f059012f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059012f8d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f058800ef90 tx=0x7f058800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 shutdown_connections 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0590138dc0 0x7f059012f8d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0590133420 0x7f059012ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 --2- 172.21.15.32:0/2946258679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05901337f0 0x7f059012f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 >> 172.21.15.32:0/2946258679 conn(0x7f05900a4210 msgr2=0x7f05900a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:30.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 shutdown_connections 2023-09-30T22:25:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:30.775+0000 7f0596c67700 1 -- 172.21.15.32:0/2946258679 wait complete. 2023-09-30T22:25:30.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:31.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:31.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:31.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 Processor -- start 2023-09-30T22:25:31.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 -- start start 2023-09-30T22:25:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e1009d630 0x7f8e100a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e10138d80 0x7f8e1013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e100a2a70 con 0x7f8e100a0a30 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e100a01d0 con 0x7f8e10138d80 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e16644700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1009e2b0 con 0x7f8e1009d630 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e14be1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e10138d80 0x7f8e1013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e0ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8e100a0a30 0x7f8e100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56442/0 (socket says 172.21.15.32:56442) 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.280+0000 7f8e0ffff700 1 -- 172.21.15.32:0/3442202576 learned_addr learned my addr 172.21.15.32:0/3442202576 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e0f7fe700 1 --2- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e1009d630 0x7f8e100a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e14be1700 1 -- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e1009d630 msgr2=0x7f8e100a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e14be1700 1 --2- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e1009d630 0x7f8e100a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e14be1700 1 -- 172.21.15.32:0/3442202576 >> v1:172.21.15.32:6789/0 conn(0x7f8e100a0a30 legacy=0x7f8e100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e14be1700 1 -- 172.21.15.32:0/3442202576 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1009dc50 con 0x7f8e10138d80 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e0effd700 1 -- 172.21.15.32:0/3442202576 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624538225 0 0) 0x7f8e100a2a70 con 0x7f8e100a0a30 2023-09-30T22:25:31.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e14be1700 1 --2- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e10138d80 0x7f8e1013b160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009d90 tx=0x7f8e0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e0b8ce078a4065 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e0effd700 1 -- 172.21.15.32:0/3442202576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800e070 con 0x7f8e10138d80 2023-09-30T22:25:31.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e0effd700 1 -- 172.21.15.32:0/3442202576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0800bb20 con 0x7f8e10138d80 2023-09-30T22:25:31.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.281+0000 7f8e0effd700 1 -- 172.21.15.32:0/3442202576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08012430 con 0x7f8e10138d80 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 -- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e10138d80 msgr2=0x7f8e1013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 --2- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e10138d80 0x7f8e1013b160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009d90 tx=0x7f8e0800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 -- 172.21.15.32:0/3442202576 shutdown_connections 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 --2- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e10138d80 0x7f8e1013b160 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 --2- 172.21.15.32:0/3442202576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e1009d630 0x7f8e100a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 -- 172.21.15.32:0/3442202576 >> 172.21.15.32:0/3442202576 conn(0x7f8e100a3050 msgr2=0x7f8e100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 -- 172.21.15.32:0/3442202576 shutdown_connections 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.282+0000 7f8e16644700 1 -- 172.21.15.32:0/3442202576 wait complete. 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.283+0000 7f8e16644700 1 Processor -- start 2023-09-30T22:25:31.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.283+0000 7f8e16644700 1 -- start start 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e16644700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e1009d630 0x7f8e10133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e16644700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e100a0a30 0x7f8e10133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e16644700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 0x7f8e101340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e16644700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10135aa0 con 0x7f8e100a0a30 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e16644700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100a4a60 con 0x7f8e1009d630 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e16644700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100a0410 con 0x7f8e10138d80 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 0x7f8e101340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e0f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e100a0a30 0x7f8e10133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 0x7f8e101340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54534/0 (socket says 172.21.15.32:54534) 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 -- 172.21.15.32:0/2549910937 learned_addr learned my addr 172.21.15.32:0/2549910937 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e0ffff700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e1009d630 0x7f8e10133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 -- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e1009d630 msgr2=0x7f8e10133650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e1009d630 0x7f8e10133650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 -- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e100a0a30 msgr2=0x7f8e10133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e100a0a30 0x7f8e10133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e14be1700 1 -- 172.21.15.32:0/2549910937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e08009a70 con 0x7f8e10138d80 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.284+0000 7f8e0f7fe700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e100a0a30 0x7f8e10133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.285+0000 7f8e14be1700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 0x7f8e101340d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8e08014040 tx=0x7f8e08004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.285+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08012430 con 0x7f8e10138d80 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.285+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08020070 con 0x7f8e10138d80 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.285+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e10134a40 con 0x7f8e10138d80 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.285+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08004280 con 0x7f8e10138d80 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.285+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e10136420 con 0x7f8e10138d80 2023-09-30T22:25:31.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.286+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e080043e0 con 0x7f8e10138d80 2023-09-30T22:25:31.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.286+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e08022070 con 0x7f8e10138d80 2023-09-30T22:25:31.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.286+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e100747f0 con 0x7f8e10138d80 2023-09-30T22:25:31.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.291+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e080263e0 con 0x7f8e10138d80 2023-09-30T22:25:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.318+0000 7f0977e34700 1 Processor -- start 2023-09-30T22:25:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.318+0000 7f0977e34700 1 -- start start 2023-09-30T22:25:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.318+0000 7f0977e34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f0970136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0977e34700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0970136d80 0x7f0970139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0977e34700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097009ece0 con 0x7f097006a770 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0977e34700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097006b640 con 0x7f097006b250 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0977e34700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f097009e760 con 0x7f0970136d80 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f0970136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09763d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0970136d80 0x7f0970139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f0970136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38522/0 (socket says 172.21.15.32:38522) 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09763d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0970136d80 0x7f0970139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54550/0 (socket says 172.21.15.32:54550) 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 -- 172.21.15.32:0/1057313627 learned_addr learned my addr 172.21.15.32:0/1057313627 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 -- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0970136d80 msgr2=0x7f0970139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 --2- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0970136d80 0x7f0970139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 -- 172.21.15.32:0/1057313627 >> v1:172.21.15.32:6789/0 conn(0x7f097006a770 legacy=0x7f097006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0974bce700 1 -- 172.21.15.32:0/1057313627 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759853918 0 0) 0x7f097009ece0 con 0x7f097006a770 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 -- 172.21.15.32:0/1057313627 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097009e040 con 0x7f097006b250 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f09753cf700 1 --2- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f0970136840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0960009920 tx=0x7f0960009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df84185134b5605 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0974bce700 1 -- 172.21.15.32:0/1057313627 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960011830 con 0x7f097006b250 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.319+0000 7f0974bce700 1 -- 172.21.15.32:0/1057313627 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960011990 con 0x7f097006b250 2023-09-30T22:25:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0974bce700 1 -- 172.21.15.32:0/1057313627 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960011d20 con 0x7f097006b250 2023-09-30T22:25:31.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 -- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 msgr2=0x7f0970136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 --2- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f0970136840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0960009920 tx=0x7f0960009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 -- 172.21.15.32:0/1057313627 shutdown_connections 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 --2- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0970136d80 0x7f0970139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 --2- 172.21.15.32:0/1057313627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f0970136840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 -- 172.21.15.32:0/1057313627 >> 172.21.15.32:0/1057313627 conn(0x7f0970066f50 msgr2=0x7f097009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 -- 172.21.15.32:0/1057313627 shutdown_connections 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.320+0000 7f0977e34700 1 -- 172.21.15.32:0/1057313627 wait complete. 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 Processor -- start 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 -- start start 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f097006a770 0x7f09701c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f09701c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 0x7f097011c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970120260 con 0x7f0970136d80 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09701bbf70 con 0x7f097006b250 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.322+0000 7f0977e34700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f097011fdb0 con 0x7f097006a770 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 0x7f097011c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09753cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f09701c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f0975bd0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f097006a770 0x7f09701c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 0x7f097011c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53818/0 (socket says 172.21.15.32:53818) 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09753cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f09701c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38538/0 (socket says 172.21.15.32:38538) 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 -- 172.21.15.32:0/2651027682 learned_addr learned my addr 172.21.15.32:0/2651027682 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 -- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f097006a770 msgr2=0x7f09701c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f097006a770 0x7f09701c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 -- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 msgr2=0x7f09701c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f09701c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 -- 172.21.15.32:0/2651027682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960009550 con 0x7f0970136d80 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09763d1700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 0x7f097011c500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f096800efc0 tx=0x7f096800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968009ea0 con 0x7f0970136d80 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096800f040 con 0x7f0970136d80 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09701323d0 con 0x7f0970136d80 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.323+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096800ced0 con 0x7f0970136d80 2023-09-30T22:25:31.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.324+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097006e930 con 0x7f0970136d80 2023-09-30T22:25:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.324+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0968014480 con 0x7f0970136d80 2023-09-30T22:25:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.324+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f096803fea0 con 0x7f0970136d80 2023-09-30T22:25:31.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.325+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097011f9e0 con 0x7f0970136d80 2023-09-30T22:25:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.330+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0968018020 con 0x7f0970136d80 2023-09-30T22:25:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.533+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e10072980 con 0x7f8e10138d80 2023-09-30T22:25:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.534+0000 7f8e0cff9700 1 -- 172.21.15.32:0/2549910937 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e0801ab70 con 0x7f8e10138d80 2023-09-30T22:25:31.535 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 msgr2=0x7f8e101340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 0x7f8e101340d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8e08014040 tx=0x7f8e08004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 shutdown_connections 2023-09-30T22:25:31.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e10138d80 0x7f8e101340d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e1009d630 0x7f8e10133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 --2- 172.21.15.32:0/2549910937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e100a0a30 0x7f8e10133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 >> 172.21.15.32:0/2549910937 conn(0x7f8e100a3050 msgr2=0x7f8e1013ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 shutdown_connections 2023-09-30T22:25:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.536+0000 7f8e16644700 1 -- 172.21.15.32:0/2549910937 wait complete. 2023-09-30T22:25:31.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:31.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.571+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f097009f6b0 con 0x7f0970136d80 2023-09-30T22:25:31.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.572+0000 7f09667fc700 1 -- 172.21.15.32:0/2651027682 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0968018020 con 0x7f0970136d80 2023-09-30T22:25:31.573 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 msgr2=0x7f097011c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 0x7f097011c500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f096800efc0 tx=0x7f096800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 shutdown_connections 2023-09-30T22:25:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0970136d80 0x7f097011c500 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f097006a770 0x7f09701c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 --2- 172.21.15.32:0/2651027682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f097006b250 0x7f09701c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 >> 172.21.15.32:0/2651027682 conn(0x7f0970066f50 msgr2=0x7f09700a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 shutdown_connections 2023-09-30T22:25:31.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.574+0000 7f0977e34700 1 -- 172.21.15.32:0/2651027682 wait complete. 2023-09-30T22:25:31.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:31.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:31.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:31.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.898+0000 7f112274d700 1 Processor -- start 2023-09-30T22:25:31.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.898+0000 7f112274d700 1 -- start start 2023-09-30T22:25:31.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f112274d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f112274d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c139170 0x7f111c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f112274d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c09d2f0 con 0x7f111c09d6f0 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f112274d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09b0f0 con 0x7f111c09e1d0 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f112274d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09a810 con 0x7f111c139170 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f111bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f111c09d6f0 0x7f111c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56462/0 (socket says 172.21.15.32:56462) 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f1120cea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c139170 0x7f111c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f111bfff700 1 -- 172.21.15.32:0/2056030894 learned_addr learned my addr 172.21.15.32:0/2056030894 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f111b7fe700 1 --2- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f1120cea700 1 -- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 msgr2=0x7f111c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f1120cea700 1 --2- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f1120cea700 1 -- 172.21.15.32:0/2056030894 >> v1:172.21.15.32:6789/0 conn(0x7f111c09d6f0 legacy=0x7f111c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f111affd700 1 -- 172.21.15.32:0/2056030894 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169726478 0 0) 0x7f111c09d2f0 con 0x7f111c09d6f0 2023-09-30T22:25:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f1120cea700 1 -- 172.21.15.32:0/2056030894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c09a470 con 0x7f111c139170 2023-09-30T22:25:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.899+0000 7f1120cea700 1 --2- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c139170 0x7f111c13b550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1114009dc0 tx=0x7f111400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e23011477ac04f51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f111affd700 1 -- 172.21.15.32:0/2056030894 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400e070 con 0x7f111c139170 2023-09-30T22:25:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f111affd700 1 -- 172.21.15.32:0/2056030894 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111400bb20 con 0x7f111c139170 2023-09-30T22:25:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f111affd700 1 -- 172.21.15.32:0/2056030894 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114012430 con 0x7f111c139170 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 -- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c139170 msgr2=0x7f111c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 --2- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c139170 0x7f111c13b550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1114009dc0 tx=0x7f111400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 -- 172.21.15.32:0/2056030894 shutdown_connections 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 --2- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c139170 0x7f111c13b550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 --2- 172.21.15.32:0/2056030894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 -- 172.21.15.32:0/2056030894 >> 172.21.15.32:0/2056030894 conn(0x7f111c066f50 msgr2=0x7f111c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 -- 172.21.15.32:0/2056030894 shutdown_connections 2023-09-30T22:25:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.900+0000 7f112274d700 1 -- 172.21.15.32:0/2056030894 wait complete. 2023-09-30T22:25:31.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.902+0000 7f112274d700 1 Processor -- start 2023-09-30T22:25:31.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.902+0000 7f112274d700 1 -- start start 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f112274d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 0x7f111c11e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f112274d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c11ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f112274d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c139170 0x7f111c11f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f112274d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c09bf80 con 0x7f111c139170 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f112274d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c122100 con 0x7f111c09e1d0 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f112274d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1347b0 con 0x7f111c09d6f0 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f111bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 0x7f111c11e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f1120cea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c139170 0x7f111c11f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f111bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 0x7f111c11e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54574/0 (socket says 172.21.15.32:54574) 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f111bfff700 1 -- 172.21.15.32:0/3800471346 learned_addr learned my addr 172.21.15.32:0/3800471346 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f111b7fe700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c11ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f1120cea700 1 -- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 msgr2=0x7f111c11e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f1120cea700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 0x7f111c11e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f1120cea700 1 -- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 msgr2=0x7f111c11ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f1120cea700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c11ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f1120cea700 1 -- 172.21.15.32:0/3800471346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1114009a70 con 0x7f111c139170 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f111b7fe700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c11ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.903+0000 7f111bfff700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 0x7f111c11e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.904+0000 7f1120cea700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c139170 0x7f111c11f340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1114009dc0 tx=0x7f1114004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.904+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111401b810 con 0x7f111c139170 2023-09-30T22:25:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.904+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1114020070 con 0x7f111c139170 2023-09-30T22:25:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.904+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c120a80 con 0x7f111c139170 2023-09-30T22:25:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.904+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111401bbf0 con 0x7f111c139170 2023-09-30T22:25:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.904+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c122ae0 con 0x7f111c139170 2023-09-30T22:25:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.905+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f111401bd50 con 0x7f111c139170 2023-09-30T22:25:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.905+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c121d70 con 0x7f111c139170 2023-09-30T22:25:31.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.905+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1114022070 con 0x7f111c139170 2023-09-30T22:25:31.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.910+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1114025790 con 0x7f111c139170 2023-09-30T22:25:31.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.937+0000 7fce0fade700 1 Processor -- start 2023-09-30T22:25:31.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.937+0000 7fce0fade700 1 -- start start 2023-09-30T22:25:31.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0fade700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 0x7fce0809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0fade700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 0x7fce081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0fade700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 0x7fce08131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0fade700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce08070940 con 0x7fce0809e930 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0fade700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0806fd60 con 0x7fce0809f650 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0fade700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce0806fa80 con 0x7fce0809f240 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d079700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 0x7fce081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 0x7fce08131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d87a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 0x7fce0809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d079700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 0x7fce081317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54584/0 (socket says 172.21.15.32:54584) 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d87a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 0x7fce0809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53828/0 (socket says 172.21.15.32:53828) 2023-09-30T22:25:31.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 0x7fce08131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38562/0 (socket says 172.21.15.32:38562) 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d079700 1 -- 172.21.15.32:0/3186736817 learned_addr learned my addr 172.21.15.32:0/3186736817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 -- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 msgr2=0x7fce081317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 --2- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 0x7fce081317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 -- 172.21.15.32:0/3186736817 >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 msgr2=0x7fce0809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 --2- 172.21.15.32:0/3186736817 >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 0x7fce0809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 -- 172.21.15.32:0/3186736817 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0809fb60 con 0x7fce0809f650 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d079700 1 --2- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 0x7fce081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0d87a700 1 --2- 172.21.15.32:0/3186736817 >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 0x7fce0809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.938+0000 7fce0e07b700 1 --2- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 0x7fce08131d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce00009d90 tx=0x7fce0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc35c9d7edec2c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0c878700 1 -- 172.21.15.32:0/3186736817 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000e070 con 0x7fce0809f650 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0c878700 1 -- 172.21.15.32:0/3186736817 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0000bb20 con 0x7fce0809f650 2023-09-30T22:25:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0c878700 1 -- 172.21.15.32:0/3186736817 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00012430 con 0x7fce0809f650 2023-09-30T22:25:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 -- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 msgr2=0x7fce08131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 --2- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 0x7fce08131d30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce00009d90 tx=0x7fce0000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 -- 172.21.15.32:0/3186736817 shutdown_connections 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 --2- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f650 0x7fce08131d30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 --2- 172.21.15.32:0/3186736817 >> v2:172.21.15.32:3300/0 conn(0x7fce0809e930 0x7fce0809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 --2- 172.21.15.32:0/3186736817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809f240 0x7fce081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 -- 172.21.15.32:0/3186736817 >> 172.21.15.32:0/3186736817 conn(0x7fce080a4210 msgr2=0x7fce080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 -- 172.21.15.32:0/3186736817 shutdown_connections 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.939+0000 7fce0fade700 1 -- 172.21.15.32:0/3186736817 wait complete. 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.940+0000 7fce0fade700 1 Processor -- start 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 -- start start 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 0x7fce081a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f240 0x7fce081a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce0809f650 0x7fce081a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce081a81a0 con 0x7fce0809f650 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce081a77e0 con 0x7fce0809f240 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0fade700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce08071d20 con 0x7fce0809e930 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0d87a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 0x7fce081a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0d87a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 0x7fce081a3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54598/0 (socket says 172.21.15.32:54598) 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0d87a700 1 -- 172.21.15.32:0/2923221104 learned_addr learned my addr 172.21.15.32:0/2923221104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0e07b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce0809f650 0x7fce081a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0d079700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f240 0x7fce081a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0d87a700 1 -- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f240 msgr2=0x7fce081a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.941+0000 7fce0d87a700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f240 0x7fce081a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0d87a700 1 -- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce0809f650 msgr2=0x7fce081a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0d87a700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce0809f650 0x7fce081a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0d87a700 1 -- 172.21.15.32:0/2923221104 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040094a0 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0e07b700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce0809f650 0x7fce081a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0d87a700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 0x7fce081a3d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce0400ec10 tx=0x7fce0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04009780 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0400c850 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf8009550 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400cbe0 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.942+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce00009a70 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.943+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce0400cd40 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.943+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce08074ea0 con 0x7fce0809e930 2023-09-30T22:25:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.943+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fce0403f750 con 0x7fce0809e930 2023-09-30T22:25:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:31.948+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce040232f0 con 0x7fce0809e930 2023-09-30T22:25:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.120+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f111c122240 con 0x7f111c139170 2023-09-30T22:25:32.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.122+0000 7f1118ff9700 1 -- 172.21.15.32:0/3800471346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f111c122240 con 0x7f111c139170 2023-09-30T22:25:32.125 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c139170 msgr2=0x7f111c11f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c139170 0x7f111c11f340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1114009dc0 tx=0x7f1114004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 shutdown_connections 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c139170 0x7f111c11f340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f111c09d6f0 0x7f111c11e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 --2- 172.21.15.32:0/3800471346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f111c09e1d0 0x7f111c11ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 >> 172.21.15.32:0/3800471346 conn(0x7f111c066f50 msgr2=0x7f111c136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.124+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 shutdown_connections 2023-09-30T22:25:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.125+0000 7f112274d700 1 -- 172.21.15.32:0/3800471346 wait complete. 2023-09-30T22:25:32.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:32.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.153+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce081a7360 con 0x7fce0809e930 2023-09-30T22:25:32.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.153+0000 7fcdfe7fc700 1 -- 172.21.15.32:0/2923221104 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce081a7360 con 0x7fce0809e930 2023-09-30T22:25:32.154 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 msgr2=0x7fce081a3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 0x7fce081a3d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce0400ec10 tx=0x7fce0400ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 shutdown_connections 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce0809f650 0x7fce081a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809e930 0x7fce081a3d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 --2- 172.21.15.32:0/2923221104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809f240 0x7fce081a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 >> 172.21.15.32:0/2923221104 conn(0x7fce080a4210 msgr2=0x7fce081b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:32.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 shutdown_connections 2023-09-30T22:25:32.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.156+0000 7fce0fade700 1 -- 172.21.15.32:0/2923221104 wait complete. 2023-09-30T22:25:32.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:32.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:32.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.703+0000 7f1115f61700 1 Processor -- start 2023-09-30T22:25:32.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.703+0000 7f1115f61700 1 -- start start 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f1115f61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11100a1510 0x7f1110138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f1115f61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1110139250 0x7f111013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f1115f61700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111009edd0 con 0x7f11100a0a30 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f1115f61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11100a18e0 con 0x7f11100a1510 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f1115f61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f111009e1f0 con 0x7f1110139250 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11100a1510 0x7f1110138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1110139250 0x7f111013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f11100a0a30 0x7f11100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56466/0 (socket says 172.21.15.32:56466) 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110f7fe700 1 -- 172.21.15.32:0/673595562 learned_addr learned my addr 172.21.15.32:0/673595562 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110e7fc700 1 -- 172.21.15.32:0/673595562 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136286998 0 0) 0x7f111009edd0 con 0x7f11100a0a30 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110effd700 1 -- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1110139250 msgr2=0x7f111013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110effd700 1 --2- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1110139250 0x7f111013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110effd700 1 -- 172.21.15.32:0/673595562 >> v1:172.21.15.32:6789/0 conn(0x7f11100a0a30 legacy=0x7f11100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110effd700 1 -- 172.21.15.32:0/673595562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111009dc50 con 0x7f11100a1510 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110ffff700 1 --2- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1110139250 0x7f111013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.704+0000 7f110effd700 1 --2- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11100a1510 0x7f1110138d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1104009840 tx=0x7f1104009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ef4072192685d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f110e7fc700 1 -- 172.21.15.32:0/673595562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011830 con 0x7f11100a1510 2023-09-30T22:25:32.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f110e7fc700 1 -- 172.21.15.32:0/673595562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104011990 con 0x7f11100a1510 2023-09-30T22:25:32.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f110e7fc700 1 -- 172.21.15.32:0/673595562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011d20 con 0x7f11100a1510 2023-09-30T22:25:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 -- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11100a1510 msgr2=0x7f1110138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 --2- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11100a1510 0x7f1110138d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1104009840 tx=0x7f1104009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 -- 172.21.15.32:0/673595562 shutdown_connections 2023-09-30T22:25:32.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 --2- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1110139250 0x7f111013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 --2- 172.21.15.32:0/673595562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11100a1510 0x7f1110138d10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 -- 172.21.15.32:0/673595562 >> 172.21.15.32:0/673595562 conn(0x7f11100a6550 msgr2=0x7f11100a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 -- 172.21.15.32:0/673595562 shutdown_connections 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.705+0000 7f1115f61700 1 -- 172.21.15.32:0/673595562 wait complete. 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.706+0000 7f1115f61700 1 Processor -- start 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 -- start start 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 0x7f11101228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11100a1510 0x7f1110122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1110139250 0x7f1110123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101389e0 con 0x7f1110139250 2023-09-30T22:25:32.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100a0220 con 0x7f11100a1510 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f1115f61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1110126060 con 0x7f11100a0a30 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f110f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 0x7f11101228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f110f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 0x7f11101228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54618/0 (socket says 172.21.15.32:54618) 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f110f7fe700 1 -- 172.21.15.32:0/4028953926 learned_addr learned my addr 172.21.15.32:0/4028953926 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f110ffff700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1110139250 0x7f1110123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.707+0000 7f110effd700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11100a1510 0x7f1110122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110f7fe700 1 -- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11100a1510 msgr2=0x7f1110122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110f7fe700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11100a1510 0x7f1110122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110f7fe700 1 -- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1110139250 msgr2=0x7f1110123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110f7fe700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1110139250 0x7f1110123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110f7fe700 1 -- 172.21.15.32:0/4028953926 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080094a0 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110effd700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11100a1510 0x7f1110122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110ffff700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1110139250 0x7f1110123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f110f7fe700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 0x7f11101228c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f110000cc80 tx=0x7f110000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110000aaf0 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110000ac50 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040094a0 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11000070d0 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.708+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1110124bb0 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.709+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1100007230 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.709+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f110003f540 con 0x7f11100a0a30 2023-09-30T22:25:32.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.709+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1110074ea0 con 0x7f11100a0a30 2023-09-30T22:25:32.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.714+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1100018070 con 0x7f11100a0a30 2023-09-30T22:25:32.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.741+0000 7f36f84e4700 1 Processor -- start 2023-09-30T22:25:32.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.741+0000 7f36f84e4700 1 -- start start 2023-09-30T22:25:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f84e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f84e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36f009f650 0x7f36f0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f84e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f0070940 con 0x7f36f009f240 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f84e4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f006fd60 con 0x7f36f009f650 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f84e4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f006fa80 con 0x7f36f009e930 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f6280700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f5a7f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36f009f240 0x7f36f01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56470/0 (socket says 172.21.15.32:56470) 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f5a7f700 1 -- 172.21.15.32:0/561074228 learned_addr learned my addr 172.21.15.32:0/561074228 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f6a81700 1 --2- 172.21.15.32:0/561074228 >> v2:172.21.15.32:3300/0 conn(0x7f36f009f650 0x7f36f0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f6a81700 1 -- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 msgr2=0x7f36f009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f6a81700 1 --2- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f6a81700 1 -- 172.21.15.32:0/561074228 >> v1:172.21.15.32:6789/0 conn(0x7f36f009f240 legacy=0x7f36f01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f6a81700 1 -- 172.21.15.32:0/561074228 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f009fb60 con 0x7f36f009f650 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.742+0000 7f36f527e700 1 -- 172.21.15.32:0/561074228 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251660529 0 0) 0x7f36f0070940 con 0x7f36f009f240 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.743+0000 7f36f6a81700 1 --2- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f650 0x7f36f0131f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009840 tx=0x7f36e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b695e57b7be6656 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.743+0000 7f36f527e700 1 -- 172.21.15.32:0/561074228 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8011830 con 0x7f36f009f650 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.743+0000 7f36f527e700 1 -- 172.21.15.32:0/561074228 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8011990 con 0x7f36f009f650 2023-09-30T22:25:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.743+0000 7f36f527e700 1 -- 172.21.15.32:0/561074228 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8011d20 con 0x7f36f009f650 2023-09-30T22:25:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.743+0000 7f36f84e4700 1 -- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f650 msgr2=0x7f36f0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.743+0000 7f36f84e4700 1 --2- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f650 0x7f36f0131f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009840 tx=0x7f36e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.744+0000 7f36f84e4700 1 -- 172.21.15.32:0/561074228 shutdown_connections 2023-09-30T22:25:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.744+0000 7f36f84e4700 1 --2- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f650 0x7f36f0131f00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.744+0000 7f36f84e4700 1 --2- 172.21.15.32:0/561074228 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.744+0000 7f36f84e4700 1 -- 172.21.15.32:0/561074228 >> 172.21.15.32:0/561074228 conn(0x7f36f00a4210 msgr2=0x7f36f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.744+0000 7f36f84e4700 1 -- 172.21.15.32:0/561074228 shutdown_connections 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.744+0000 7f36f84e4700 1 -- 172.21.15.32:0/561074228 wait complete. 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.745+0000 7f36f84e4700 1 Processor -- start 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.745+0000 7f36f84e4700 1 -- start start 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f84e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f01a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f84e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f240 0x7f36f01a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f84e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36f009f650 0x7f36f01c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f84e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f01a5f30 con 0x7f36f009f240 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f84e4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f01a6110 con 0x7f36f009e930 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f84e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36f01b6f80 con 0x7f36f009f650 2023-09-30T22:25:32.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6280700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f01a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6a81700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36f009f650 0x7f36f01c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6280700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f01a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38612/0 (socket says 172.21.15.32:38612) 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6280700 1 -- 172.21.15.32:0/2148343396 learned_addr learned my addr 172.21.15.32:0/2148343396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f5a7f700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f240 0x7f36f01a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6a81700 1 -- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 msgr2=0x7f36f01a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6a81700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f01a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6a81700 1 -- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f240 msgr2=0x7f36f01a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6a81700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f240 0x7f36f01a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6a81700 1 -- 172.21.15.32:0/2148343396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80094a0 con 0x7f36f009f650 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f6280700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f01a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:32.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.746+0000 7f36f5a7f700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f240 0x7f36f01a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.747+0000 7f36f6a81700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36f009f650 0x7f36f01c5dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36f00a1bf0 tx=0x7f36e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.747+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004590 con 0x7f36f009f650 2023-09-30T22:25:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.747+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e801f070 con 0x7f36f009f650 2023-09-30T22:25:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.747+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80038e0 con 0x7f36f009f650 2023-09-30T22:25:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.747+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f00a0190 con 0x7f36f009f650 2023-09-30T22:25:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.747+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f0071d20 con 0x7f36f009f650 2023-09-30T22:25:32.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.748+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36e8003a40 con 0x7f36f009f650 2023-09-30T22:25:32.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.748+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f01a3ad0 con 0x7f36f009f650 2023-09-30T22:25:32.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.748+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f36e8021070 con 0x7f36f009f650 2023-09-30T22:25:32.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.753+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36e8026800 con 0x7f36f009f650 2023-09-30T22:25:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.921+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f11100a28e0 con 0x7f11100a0a30 2023-09-30T22:25:32.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.921+0000 7f10f7fff700 1 -- 172.21.15.32:0/4028953926 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f110001a4a0 con 0x7f11100a0a30 2023-09-30T22:25:32.922 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.923+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 msgr2=0x7f11101228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.923+0000 7f1115f61700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 0x7f11101228c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f110000cc80 tx=0x7f110000cf90 comp rx=0 tx=0).stop 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 shutdown_connections 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1110139250 0x7f1110123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11100a0a30 0x7f11101228c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 --2- 172.21.15.32:0/4028953926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11100a1510 0x7f1110122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 >> 172.21.15.32:0/4028953926 conn(0x7f11100a6550 msgr2=0x7f11100a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 shutdown_connections 2023-09-30T22:25:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.924+0000 7f1115f61700 1 -- 172.21.15.32:0/4028953926 wait complete. 2023-09-30T22:25:32.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.962+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f36f0074ea0 con 0x7f36f009f650 2023-09-30T22:25:32.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.962+0000 7f36e6ffd700 1 -- 172.21.15.32:0/2148343396 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36e802b030 con 0x7f36f009f650 2023-09-30T22:25:32.963 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36f009f650 msgr2=0x7f36f01c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36f009f650 0x7f36f01c5dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f36f00a1bf0 tx=0x7f36e8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 shutdown_connections 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36f009f650 0x7f36f01c5dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36f009e930 0x7f36f01a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 --2- 172.21.15.32:0/2148343396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f009f240 0x7f36f01a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 >> 172.21.15.32:0/2148343396 conn(0x7f36f00a4210 msgr2=0x7f36f00a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 shutdown_connections 2023-09-30T22:25:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:32.965+0000 7f36f84e4700 1 -- 172.21.15.32:0/2148343396 wait complete. 2023-09-30T22:25:32.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:33.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:33.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.288+0000 7fd59d5fc700 1 Processor -- start 2023-09-30T22:25:33.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.288+0000 7fd59d5fc700 1 -- start start 2023-09-30T22:25:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd59d5fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd5980a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd59d5fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd59813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd59d5fc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980a2a70 con 0x7fd5980a0a30 2023-09-30T22:25:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd59d5fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980a01d0 con 0x7fd59809d630 2023-09-30T22:25:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd59d5fc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd59809e2b0 con 0x7fd598138d80 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd596ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd5980a0a30 0x7fd5980a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56476/0 (socket says 172.21.15.32:56476) 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd596ffd700 1 -- 172.21.15.32:0/4076894999 learned_addr learned my addr 172.21.15.32:0/4076894999 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5967fc700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd5980a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5977fe700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd59813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd595ffb700 1 -- 172.21.15.32:0/4076894999 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209891582 0 0) 0x7fd5980a2a70 con 0x7fd5980a0a30 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5967fc700 1 -- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 msgr2=0x7fd59813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5967fc700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd59813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5967fc700 1 -- 172.21.15.32:0/4076894999 >> v1:172.21.15.32:6789/0 conn(0x7fd5980a0a30 legacy=0x7fd5980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5967fc700 1 -- 172.21.15.32:0/4076894999 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59809dc50 con 0x7fd59809d630 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5977fe700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd59813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.289+0000 7fd5967fc700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd5980a1510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aca8c7cc472ab65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd595ffb700 1 -- 172.21.15.32:0/4076894999 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011830 con 0x7fd59809d630 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd595ffb700 1 -- 172.21.15.32:0/4076894999 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c011990 con 0x7fd59809d630 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd595ffb700 1 -- 172.21.15.32:0/4076894999 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011d20 con 0x7fd59809d630 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 -- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 msgr2=0x7fd5980a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd5980a1510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 -- 172.21.15.32:0/4076894999 shutdown_connections 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd59813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/4076894999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd5980a1510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 -- 172.21.15.32:0/4076894999 >> 172.21.15.32:0/4076894999 conn(0x7fd5980a3050 msgr2=0x7fd5980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 -- 172.21.15.32:0/4076894999 shutdown_connections 2023-09-30T22:25:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.290+0000 7fd59d5fc700 1 -- 172.21.15.32:0/4076894999 wait complete. 2023-09-30T22:25:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 Processor -- start 2023-09-30T22:25:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 -- start start 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd598133910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5980a0a30 0x7fd598133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd598134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd598134de0 con 0x7fd5980a0a30 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981374b0 con 0x7fd59809d630 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd59d5fc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd598136e00 con 0x7fd598138d80 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd596ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd598133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd5967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5980a0a30 0x7fd598133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd596ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd598133910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38632/0 (socket says 172.21.15.32:38632) 2023-09-30T22:25:33.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd596ffd700 1 -- 172.21.15.32:0/2641130214 learned_addr learned my addr 172.21.15.32:0/2641130214 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.292+0000 7fd5977fe700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd598134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5967fc700 1 -- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 msgr2=0x7fd598134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5967fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd598134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5967fc700 1 -- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 msgr2=0x7fd598133910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5967fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd598133910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5967fc700 1 -- 172.21.15.32:0/2641130214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900094a0 con 0x7fd5980a0a30 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5977fe700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd598134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd596ffd700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd598133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd5967fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5980a0a30 0x7fd598133e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd5980a1fb0 tx=0x7fd58c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0042c0 con 0x7fd5980a0a30 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c01f070 con 0x7fd5980a0a30 2023-09-30T22:25:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c0094a0 con 0x7fd5980a0a30 2023-09-30T22:25:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0036a0 con 0x7fd5980a0a30 2023-09-30T22:25:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.293+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5981370c0 con 0x7fd5980a0a30 2023-09-30T22:25:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.294+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd58c021030 con 0x7fd5980a0a30 2023-09-30T22:25:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.294+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980747f0 con 0x7fd5980a0a30 2023-09-30T22:25:33.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.294+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd58c040680 con 0x7fd5980a0a30 2023-09-30T22:25:33.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.304+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd58c015070 con 0x7fd5980a0a30 2023-09-30T22:25:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 Processor -- start 2023-09-30T22:25:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 -- start start 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 0x7f0d980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d980a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d98133930 0x7f0d98133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d980a2a70 con 0x7f0d980a0a30 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980a01d0 con 0x7f0d98133930 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9e849700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9809e2b0 con 0x7f0d9809d630 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d97fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 0x7f0d980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d97fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 0x7f0d980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53910/0 (socket says 172.21.15.32:53910) 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d97fff700 1 -- 172.21.15.32:0/2975436518 learned_addr learned my addr 172.21.15.32:0/2975436518 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d9cde6700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d98133930 0x7f0d98133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.330+0000 7f0d977fe700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d980a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d9cde6700 1 -- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 msgr2=0x7f0d980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d9cde6700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d980a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d9cde6700 1 -- 172.21.15.32:0/2975436518 >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 msgr2=0x7f0d980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d9cde6700 1 --2- 172.21.15.32:0/2975436518 >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 0x7f0d980a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d9cde6700 1 -- 172.21.15.32:0/2975436518 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9809dc50 con 0x7f0d98133930 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d97fff700 1 --2- 172.21.15.32:0/2975436518 >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 0x7f0d980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d9cde6700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d98133930 0x7f0d98133db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009d90 tx=0x7f0d9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661d3b0e79ad2eae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d96ffd700 1 -- 172.21.15.32:0/2975436518 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000e070 con 0x7f0d98133930 2023-09-30T22:25:33.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d96ffd700 1 -- 172.21.15.32:0/2975436518 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9000bb20 con 0x7f0d98133930 2023-09-30T22:25:33.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.331+0000 7f0d96ffd700 1 -- 172.21.15.32:0/2975436518 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90012430 con 0x7f0d98133930 2023-09-30T22:25:33.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 -- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d98133930 msgr2=0x7f0d98133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d98133930 0x7f0d98133db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009d90 tx=0x7f0d9000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 -- 172.21.15.32:0/2975436518 shutdown_connections 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d98133930 0x7f0d98133db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 --2- 172.21.15.32:0/2975436518 >> v2:172.21.15.32:3300/0 conn(0x7f0d980a0a30 0x7f0d980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 --2- 172.21.15.32:0/2975436518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d980a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 -- 172.21.15.32:0/2975436518 >> 172.21.15.32:0/2975436518 conn(0x7f0d980a3050 msgr2=0x7f0d980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 -- 172.21.15.32:0/2975436518 shutdown_connections 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.332+0000 7f0d9e849700 1 -- 172.21.15.32:0/2975436518 wait complete. 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.333+0000 7f0d9e849700 1 Processor -- start 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.333+0000 7f0d9e849700 1 -- start start 2023-09-30T22:25:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9e849700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d9811e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9e849700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d980a0a30 0x7f0d9811ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9e849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d98133930 0x7f0d9811f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9e849700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98120a40 con 0x7f0d98133930 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9e849700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980a0480 con 0x7f0d980a0a30 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9e849700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d981205a0 con 0x7f0d9809d630 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d9811e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d980a0a30 0x7f0d9811ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d9811e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54656/0 (socket says 172.21.15.32:54656) 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 -- 172.21.15.32:0/736688128 learned_addr learned my addr 172.21.15.32:0/736688128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d9cde6700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d98133930 0x7f0d9811f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 -- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d980a0a30 msgr2=0x7f0d9811ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d980a0a30 0x7f0d9811ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 -- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d98133930 msgr2=0x7f0d9811f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d98133930 0x7f0d9811f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 -- 172.21.15.32:0/736688128 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009550 con 0x7f0d9809d630 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d977fe700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d980a0a30 0x7f0d9811ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.334+0000 7f0d97fff700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d9811e810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800ebe0 tx=0x7f0d8800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.335+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800c7f0 con 0x7f0d9809d630 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.335+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8800c950 con 0x7f0d9809d630 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.335+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d90009a70 con 0x7f0d9809d630 2023-09-30T22:25:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.335+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800cce0 con 0x7f0d9809d630 2023-09-30T22:25:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.335+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d98122050 con 0x7f0d9809d630 2023-09-30T22:25:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.336+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d98120c70 con 0x7f0d9809d630 2023-09-30T22:25:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.336+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d88013470 con 0x7f0d9809d630 2023-09-30T22:25:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.336+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0d8803ff90 con 0x7f0d9809d630 2023-09-30T22:25:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.341+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d88013720 con 0x7f0d9809d630 2023-09-30T22:25:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.510+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd598135c80 con 0x7fd5980a0a30 2023-09-30T22:25:33.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.510+0000 7fd57f7fe700 1 -- 172.21.15.32:0/2641130214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd58c0239c0 con 0x7fd5980a0a30 2023-09-30T22:25:33.511 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:33.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.512+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5980a0a30 msgr2=0x7fd598133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.512+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5980a0a30 0x7fd598133e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd5980a1fb0 tx=0x7fd58c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.512+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 shutdown_connections 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.512+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd598138d80 0x7fd598134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.512+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd59809d630 0x7fd598133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.513+0000 7fd59d5fc700 1 --2- 172.21.15.32:0/2641130214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5980a0a30 0x7fd598133e50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.513+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 >> 172.21.15.32:0/2641130214 conn(0x7fd5980a3050 msgr2=0x7fd59809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.513+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 shutdown_connections 2023-09-30T22:25:33.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.513+0000 7fd59d5fc700 1 -- 172.21.15.32:0/2641130214 wait complete. 2023-09-30T22:25:33.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.545+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0d980747f0 con 0x7f0d9809d630 2023-09-30T22:25:33.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.546+0000 7f0d94ff9700 1 -- 172.21.15.32:0/736688128 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d88017070 con 0x7f0d9809d630 2023-09-30T22:25:33.547 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 msgr2=0x7f0d9811e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d9811e810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800ebe0 tx=0x7f0d8800eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 shutdown_connections 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d98133930 0x7f0d9811f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d9809d630 0x7f0d9811e810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 --2- 172.21.15.32:0/736688128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d980a0a30 0x7f0d9811ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:33.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 >> 172.21.15.32:0/736688128 conn(0x7f0d980a3050 msgr2=0x7f0d981bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:33.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 shutdown_connections 2023-09-30T22:25:33.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:33.548+0000 7f0d9e849700 1 -- 172.21.15.32:0/736688128 wait complete. 2023-09-30T22:25:33.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:33.939 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:33.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:34.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.087+0000 7fa26c108700 1 Processor -- start 2023-09-30T22:25:34.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26c108700 1 -- start start 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26c108700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa2640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26c108700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa26413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26c108700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26409edd0 con 0x7fa2640a1340 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26c108700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26409e1f0 con 0x7fa2640a0a30 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26c108700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa26409dc50 con 0x7fa264134170 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa2696a3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2640a1340 0x7fa264133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56478/0 (socket says 172.21.15.32:56478) 2023-09-30T22:25:34.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa269ea4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa2640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa2696a3700 1 -- 172.21.15.32:0/1988808102 learned_addr learned my addr 172.21.15.32:0/1988808102 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.088+0000 7fa26a6a5700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa26413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa268ea2700 1 -- 172.21.15.32:0/1988808102 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893807066 0 0) 0x7fa26409edd0 con 0x7fa2640a1340 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa268ea2700 1 -- 172.21.15.32:0/1988808102 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa24c0035b0 con 0x7fa2640a1340 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa26a6a5700 1 -- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 msgr2=0x7fa2640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa26a6a5700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa2640a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa26a6a5700 1 -- 172.21.15.32:0/1988808102 >> v1:172.21.15.32:6789/0 conn(0x7fa2640a1340 legacy=0x7fa264133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa26a6a5700 1 -- 172.21.15.32:0/1988808102 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640a1fe0 con 0x7fa264134170 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa269ea4700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa2640a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa26a6a5700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa26413bd30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa254009dc0 tx=0x7fa25400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e961ac0e7392a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa268ea2700 1 -- 172.21.15.32:0/1988808102 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400e070 con 0x7fa264134170 2023-09-30T22:25:34.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa268ea2700 1 -- 172.21.15.32:0/1988808102 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400bb20 con 0x7fa264134170 2023-09-30T22:25:34.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.089+0000 7fa268ea2700 1 -- 172.21.15.32:0/1988808102 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400be90 con 0x7fa264134170 2023-09-30T22:25:34.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 -- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 msgr2=0x7fa26413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa26413bd30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa254009dc0 tx=0x7fa25400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 -- 172.21.15.32:0/1988808102 shutdown_connections 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa26413bd30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 --2- 172.21.15.32:0/1988808102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa2640a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 -- 172.21.15.32:0/1988808102 >> 172.21.15.32:0/1988808102 conn(0x7fa2640a6550 msgr2=0x7fa2640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 -- 172.21.15.32:0/1988808102 shutdown_connections 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.090+0000 7fa26c108700 1 -- 172.21.15.32:0/1988808102 wait complete. 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.091+0000 7fa26c108700 1 Processor -- start 2023-09-30T22:25:34.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 -- start start 2023-09-30T22:25:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa264122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2640a1340 0x7fa264122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa264123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2641262a0 con 0x7fa2640a1340 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264124c20 con 0x7fa2640a0a30 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26c108700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa264124390 con 0x7fa264134170 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa269ea4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa264122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa26a6a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa264123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa269ea4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa264122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38646/0 (socket says 172.21.15.32:38646) 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa269ea4700 1 -- 172.21.15.32:0/348258663 learned_addr learned my addr 172.21.15.32:0/348258663 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.092+0000 7fa2696a3700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2640a1340 0x7fa264122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26a6a5700 1 -- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 msgr2=0x7fa264122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26a6a5700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa264122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26a6a5700 1 -- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2640a1340 msgr2=0x7fa264122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26a6a5700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2640a1340 0x7fa264122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26a6a5700 1 -- 172.21.15.32:0/348258663 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0094a0 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa2696a3700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2640a1340 0x7fa264122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa269ea4700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa264122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26a6a5700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa264123200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa254016000 tx=0x7fa254004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254007000 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254012630 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa254009a70 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2540129e0 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.093+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2640a25e0 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.094+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa254012b40 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.094+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa254041800 con 0x7fa264134170 2023-09-30T22:25:34.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.095+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa264125e40 con 0x7fa264134170 2023-09-30T22:25:34.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.099+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa254003e30 con 0x7fa264134170 2023-09-30T22:25:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.127+0000 7f19410aa700 1 Processor -- start 2023-09-30T22:25:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.127+0000 7f19410aa700 1 -- start start 2023-09-30T22:25:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f19410aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f19410aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f19410aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c09f130 con 0x7f193c09e000 2023-09-30T22:25:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f19410aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c09f520 con 0x7f193c09d6f0 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f19410aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09e930 con 0x7f193c09e410 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193b59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f193c09e000 0x7f193c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56486/0 (socket says 172.21.15.32:56486) 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193b59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54684/0 (socket says 172.21.15.32:54684) 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193a59c700 1 -- 172.21.15.32:0/2589354904 learned_addr learned my addr 172.21.15.32:0/2589354904 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193ad9d700 1 -- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 msgr2=0x7f193c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193ad9d700 1 --2- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193ad9d700 1 -- 172.21.15.32:0/2589354904 >> v1:172.21.15.32:6789/0 conn(0x7f193c09e000 legacy=0x7f193c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193ad9d700 1 -- 172.21.15.32:0/2589354904 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c09b2d0 con 0x7f193c09d6f0 2023-09-30T22:25:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f1939d9b700 1 -- 172.21.15.32:0/2589354904 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084609813 0 0) 0x7f193c09f130 con 0x7f193c09e000 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193b59e700 1 --2- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.128+0000 7f193ad9d700 1 --2- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09d6f0 0x7f193c09dac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f192c009840 tx=0x7f192c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8bb3251d101367 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f1939d9b700 1 -- 172.21.15.32:0/2589354904 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c011830 con 0x7f193c09d6f0 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f1939d9b700 1 -- 172.21.15.32:0/2589354904 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c011990 con 0x7f193c09d6f0 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f1939d9b700 1 -- 172.21.15.32:0/2589354904 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c011d20 con 0x7f193c09d6f0 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 -- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09d6f0 msgr2=0x7f193c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 --2- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09d6f0 0x7f193c09dac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f192c009840 tx=0x7f192c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 -- 172.21.15.32:0/2589354904 shutdown_connections 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 --2- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 --2- 172.21.15.32:0/2589354904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 -- 172.21.15.32:0/2589354904 >> 172.21.15.32:0/2589354904 conn(0x7f193c066f50 msgr2=0x7f193c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 -- 172.21.15.32:0/2589354904 shutdown_connections 2023-09-30T22:25:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.129+0000 7f19410aa700 1 -- 172.21.15.32:0/2589354904 wait complete. 2023-09-30T22:25:34.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 Processor -- start 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 -- start start 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09d6f0 0x7f193c1201f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 0x7f193c120730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c120c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c122390 con 0x7f193c09e000 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09c700 con 0x7f193c09d6f0 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f19410aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c121d70 con 0x7f193c09e410 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f193a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 0x7f193c120730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f193a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 0x7f193c120730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53936/0 (socket says 172.21.15.32:53936) 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f193a59c700 1 -- 172.21.15.32:0/6439636 learned_addr learned my addr 172.21.15.32:0/6439636 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:34.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f193ad9d700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09d6f0 0x7f193c1201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.131+0000 7f193b59e700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c120c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f193a59c700 1 -- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 msgr2=0x7f193c120c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f193a59c700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c120c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f193a59c700 1 -- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09d6f0 msgr2=0x7f193c1201f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f193a59c700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09d6f0 0x7f193c1201f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f193a59c700 1 -- 172.21.15.32:0/6439636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009a70 con 0x7f193c09e000 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f193a59c700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 0x7f193c120730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f193000c520 tx=0x7f193000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930004c80 con 0x7f193c09e000 2023-09-30T22:25:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1930004e00 con 0x7f193c09e000 2023-09-30T22:25:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c0094a0 con 0x7f193c09e000 2023-09-30T22:25:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930007310 con 0x7f193c09e000 2023-09-30T22:25:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.132+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c122560 con 0x7f193c09e000 2023-09-30T22:25:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.133+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1930007470 con 0x7f193c09e000 2023-09-30T22:25:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.133+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f193003f270 con 0x7f193c09e000 2023-09-30T22:25:34.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.134+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c123820 con 0x7f193c09e000 2023-09-30T22:25:34.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.139+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19300123b0 con 0x7f193c09e000 2023-09-30T22:25:34.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.329+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa264074ea0 con 0x7fa264134170 2023-09-30T22:25:34.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.330+0000 7fa2627fc700 1 -- 172.21.15.32:0/348258663 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa254034090 con 0x7fa264134170 2023-09-30T22:25:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:34.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 msgr2=0x7fa264123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa264123200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa254016000 tx=0x7fa254004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 shutdown_connections 2023-09-30T22:25:34.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa264134170 0x7fa264123200 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2640a0a30 0x7fa264122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 --2- 172.21.15.32:0/348258663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2640a1340 0x7fa264122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 >> 172.21.15.32:0/348258663 conn(0x7fa2640a6550 msgr2=0x7fa26409c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 shutdown_connections 2023-09-30T22:25:34.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.332+0000 7fa26c108700 1 -- 172.21.15.32:0/348258663 wait complete. 2023-09-30T22:25:34.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:34.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.361+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f193c070d80 con 0x7f193c09e000 2023-09-30T22:25:34.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.361+0000 7f19237fe700 1 -- 172.21.15.32:0/6439636 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f193000ce60 con 0x7f193c09e000 2023-09-30T22:25:34.362 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 msgr2=0x7f193c120730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 0x7f193c120730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f193000c520 tx=0x7f193000c830 comp rx=0 tx=0).stop 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 shutdown_connections 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e410 0x7f193c120c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09d6f0 0x7f193c1201f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 --2- 172.21.15.32:0/6439636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09e000 0x7f193c120730 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 >> 172.21.15.32:0/6439636 conn(0x7f193c066f50 msgr2=0x7f193c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 shutdown_connections 2023-09-30T22:25:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.364+0000 7f19410aa700 1 -- 172.21.15.32:0/6439636 wait complete. 2023-09-30T22:25:34.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:34.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:34.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.675+0000 7f059a82c700 1 Processor -- start 2023-09-30T22:25:34.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.675+0000 7f059a82c700 1 -- start start 2023-09-30T22:25:34.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f059a82c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0594133a80 0x7f0594133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f059a82c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 0x7f059413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f059a82c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059406bc00 con 0x7f059412ff20 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f059a82c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406c570 con 0x7f0594133a80 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f059a82c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05940a0040 con 0x7f0594130a70 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 0x7f059413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 0x7f059413b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38650/0 (socket says 172.21.15.32:38650) 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 -- 172.21.15.32:0/3122954278 learned_addr learned my addr 172.21.15.32:0/3122954278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0593fff700 1 --2- 172.21.15.32:0/3122954278 >> v2:172.21.15.32:3300/0 conn(0x7f0594133a80 0x7f0594133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0592ffd700 1 -- 172.21.15.32:0/3122954278 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837595330 0 0) 0x7f059406bc00 con 0x7f059412ff20 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 -- 172.21.15.32:0/3122954278 >> v2:172.21.15.32:3300/0 conn(0x7f0594133a80 msgr2=0x7f0594133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 --2- 172.21.15.32:0/3122954278 >> v2:172.21.15.32:3300/0 conn(0x7f0594133a80 0x7f0594133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 -- 172.21.15.32:0/3122954278 >> v1:172.21.15.32:6789/0 conn(0x7f059412ff20 legacy=0x7f0594130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.676+0000 7f0598dc9700 1 -- 172.21.15.32:0/3122954278 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594133860 con 0x7f0594130a70 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.677+0000 7f0593fff700 1 --2- 172.21.15.32:0/3122954278 >> v2:172.21.15.32:3300/0 conn(0x7f0594133a80 0x7f0594133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.677+0000 7f0598dc9700 1 --2- 172.21.15.32:0/3122954278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 0x7f059413b850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f058c009e50 tx=0x7f058c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd611ba1ef39fa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.677+0000 7f0592ffd700 1 -- 172.21.15.32:0/3122954278 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00e070 con 0x7f0594130a70 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.677+0000 7f0592ffd700 1 -- 172.21.15.32:0/3122954278 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00b910 con 0x7f0594130a70 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.677+0000 7f0592ffd700 1 -- 172.21.15.32:0/3122954278 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00bcc0 con 0x7f0594130a70 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 -- 172.21.15.32:0/3122954278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 msgr2=0x7f059413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 --2- 172.21.15.32:0/3122954278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 0x7f059413b850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f058c009e50 tx=0x7f058c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 -- 172.21.15.32:0/3122954278 shutdown_connections 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 --2- 172.21.15.32:0/3122954278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0594130a70 0x7f059413b850 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 --2- 172.21.15.32:0/3122954278 >> v2:172.21.15.32:3300/0 conn(0x7f0594133a80 0x7f0594133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 -- 172.21.15.32:0/3122954278 >> 172.21.15.32:0/3122954278 conn(0x7f05940a4210 msgr2=0x7f05940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 -- 172.21.15.32:0/3122954278 shutdown_connections 2023-09-30T22:25:34.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.678+0000 7f059a82c700 1 -- 172.21.15.32:0/3122954278 wait complete. 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.679+0000 7f059a82c700 1 Processor -- start 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.679+0000 7f059a82c700 1 -- start start 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f059a82c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 0x7f059409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f059a82c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0594130a70 0x7f059409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f059a82c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0594133a80 0x7f059409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f059a82c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059409ddb0 con 0x7f0594130a70 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f059a82c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059411ff00 con 0x7f059412ff20 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f059a82c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f059411fbd0 con 0x7f0594133a80 2023-09-30T22:25:34.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f0598dc9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0594133a80 0x7f059409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f0593fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 0x7f059409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f05937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0594130a70 0x7f059409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f0593fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 0x7f059409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38662/0 (socket says 172.21.15.32:38662) 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f0598dc9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0594133a80 0x7f059409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54704/0 (socket says 172.21.15.32:54704) 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.680+0000 7f0593fff700 1 -- 172.21.15.32:0/2350351135 learned_addr learned my addr 172.21.15.32:0/2350351135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0593fff700 1 -- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0594133a80 msgr2=0x7f059409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0593fff700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0594133a80 0x7f059409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0593fff700 1 -- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0594130a70 msgr2=0x7f059409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0593fff700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0594130a70 0x7f059409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0593fff700 1 -- 172.21.15.32:0/2350351135 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840094a0 con 0x7f059412ff20 2023-09-30T22:25:34.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0593fff700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 0x7f059409c480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f058400ebe0 tx=0x7f058400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058400c850 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400c9b0 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c009b30 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058400cd40 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.681+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059411f010 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.682+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0584013470 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.682+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f058403f770 con 0x7f059412ff20 2023-09-30T22:25:34.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.682+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0594074ea0 con 0x7f059412ff20 2023-09-30T22:25:34.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.687+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0584013720 con 0x7f059412ff20 2023-09-30T22:25:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.714+0000 7f8669e7d700 1 Processor -- start 2023-09-30T22:25:34.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.714+0000 7f8669e7d700 1 -- start start 2023-09-30T22:25:34.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8669e7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a0a30 0x7f86640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8669e7d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 0x7f8664133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8669e7d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866409edd0 con 0x7f8664133fa0 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8669e7d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866409e1f0 con 0x7f86640a1340 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8669e7d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f866409dc50 con 0x7f86640a0a30 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8662ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 0x7f8664133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f86637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a0a30 0x7f86640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8662ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 0x7f8664133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38668/0 (socket says 172.21.15.32:38668) 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8662ffd700 1 -- 172.21.15.32:0/1269382501 learned_addr learned my addr 172.21.15.32:0/1269382501 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8663fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8664133fa0 0x7f866413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56512/0 (socket says 172.21.15.32:56512) 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f86637fe700 1 -- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 msgr2=0x7f8664133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f86637fe700 1 --2- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 0x7f8664133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f86637fe700 1 -- 172.21.15.32:0/1269382501 >> v1:172.21.15.32:6789/0 conn(0x7f8664133fa0 legacy=0x7f866413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f86637fe700 1 -- 172.21.15.32:0/1269382501 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640a1fe0 con 0x7f86640a0a30 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f86627fc700 1 -- 172.21.15.32:0/1269382501 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354915221 0 0) 0x7f866409edd0 con 0x7f8664133fa0 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.715+0000 7f8662ffd700 1 --2- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 0x7f8664133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f86637fe700 1 --2- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a0a30 0x7f86640a0e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8654009dc0 tx=0x7f865400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e2954a9de49b08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f86627fc700 1 -- 172.21.15.32:0/1269382501 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400e070 con 0x7f86640a0a30 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f86627fc700 1 -- 172.21.15.32:0/1269382501 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400bb20 con 0x7f86640a0a30 2023-09-30T22:25:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f86627fc700 1 -- 172.21.15.32:0/1269382501 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654012430 con 0x7f86640a0a30 2023-09-30T22:25:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 -- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a0a30 msgr2=0x7f86640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 --2- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a0a30 0x7f86640a0e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8654009dc0 tx=0x7f865400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 -- 172.21.15.32:0/1269382501 shutdown_connections 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 --2- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a0a30 0x7f86640a0e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 --2- 172.21.15.32:0/1269382501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86640a1340 0x7f8664133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 -- 172.21.15.32:0/1269382501 >> 172.21.15.32:0/1269382501 conn(0x7f86640a6550 msgr2=0x7f86640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.716+0000 7f8669e7d700 1 -- 172.21.15.32:0/1269382501 shutdown_connections 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.717+0000 7f8669e7d700 1 -- 172.21.15.32:0/1269382501 wait complete. 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.717+0000 7f8669e7d700 1 Processor -- start 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 -- start start 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86640a0a30 0x7f86641227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 0x7f8664122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8664133fa0 0x7f8664123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8664124c90 con 0x7f86640a0a30 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8664125d60 con 0x7f8664133fa0 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8669e7d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8664124500 con 0x7f86640a1340 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8662ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 0x7f8664122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8663fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8664133fa0 0x7f8664123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8662ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 0x7f8664122d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54722/0 (socket says 172.21.15.32:54722) 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8662ffd700 1 -- 172.21.15.32:0/2460224800 learned_addr learned my addr 172.21.15.32:0/2460224800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f8663fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8664133fa0 0x7f8664123260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38680/0 (socket says 172.21.15.32:38680) 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.718+0000 7f86637fe700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86640a0a30 0x7f86641227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8662ffd700 1 -- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8664133fa0 msgr2=0x7f8664123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8662ffd700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8664133fa0 0x7f8664123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8662ffd700 1 -- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86640a0a30 msgr2=0x7f86641227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8662ffd700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86640a0a30 0x7f86641227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8662ffd700 1 -- 172.21.15.32:0/2460224800 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009550 con 0x7f86640a1340 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8662ffd700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 0x7f8664122d20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ebe0 tx=0x7f864c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00c850 con 0x7f86640a1340 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c00c9b0 con 0x7f86640a1340 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8654009a70 con 0x7f86640a1340 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00cd40 con 0x7f86640a1340 2023-09-30T22:25:34.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.719+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86640a25e0 con 0x7f86640a1340 2023-09-30T22:25:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.720+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f864c013470 con 0x7f86640a1340 2023-09-30T22:25:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.720+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8664074ea0 con 0x7f86640a1340 2023-09-30T22:25:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.720+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f864c03f830 con 0x7f86640a1340 2023-09-30T22:25:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.725+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f864c0232f0 con 0x7f86640a1340 2023-09-30T22:25:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.896+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f059406b250 con 0x7f059412ff20 2023-09-30T22:25:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.897+0000 7f0590ff9700 1 -- 172.21.15.32:0/2350351135 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0584021020 con 0x7f059412ff20 2023-09-30T22:25:34.898 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 msgr2=0x7f059409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 0x7f059409c480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f058400ebe0 tx=0x7f058400eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 shutdown_connections 2023-09-30T22:25:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0594133a80 0x7f059409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f059412ff20 0x7f059409c480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 --2- 172.21.15.32:0/2350351135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0594130a70 0x7f059409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 >> 172.21.15.32:0/2350351135 conn(0x7f05940a4210 msgr2=0x7f059409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 shutdown_connections 2023-09-30T22:25:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.899+0000 7f059a82c700 1 -- 172.21.15.32:0/2350351135 wait complete. 2023-09-30T22:25:34.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.932+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f86640a28e0 con 0x7f86640a1340 2023-09-30T22:25:34.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.933+0000 7f865bfff700 1 -- 172.21.15.32:0/2460224800 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f864c021070 con 0x7f86640a1340 2023-09-30T22:25:34.933 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 msgr2=0x7f8664122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 0x7f8664122d20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f864c00ebe0 tx=0x7f864c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 shutdown_connections 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8664133fa0 0x7f8664123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86640a0a30 0x7f86641227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 --2- 172.21.15.32:0/2460224800 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f86640a1340 0x7f8664122d20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 >> 172.21.15.32:0/2460224800 conn(0x7f86640a6550 msgr2=0x7f86640a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:34.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 shutdown_connections 2023-09-30T22:25:34.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:34.935+0000 7f8669e7d700 1 -- 172.21.15.32:0/2460224800 wait complete. 2023-09-30T22:25:34.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:35.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:35.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:35.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.497+0000 7fd6c73ef700 1 Processor -- start 2023-09-30T22:25:35.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 -- start start 2023-09-30T22:25:35.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6c00a0a30 0x7fd6c00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a1340 0x7fd6c0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c0133fa0 0x7fd6c0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c009edd0 con 0x7fd6c00a0a30 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c009e1f0 con 0x7fd6c00a1340 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.498+0000 7fd6c73ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c009dc50 con 0x7fd6c0133fa0 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c598c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c0133fa0 0x7fd6c0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6c00a0a30 0x7fd6c00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c498a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a1340 0x7fd6c0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6c00a0a30 0x7fd6c00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53952/0 (socket says 172.21.15.32:53952) 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 -- 172.21.15.32:0/3074249291 learned_addr learned my addr 172.21.15.32:0/3074249291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:35.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c598c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c0133fa0 0x7fd6c0134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54738/0 (socket says 172.21.15.32:54738) 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 -- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c0133fa0 msgr2=0x7fd6c0134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c0133fa0 0x7fd6c0134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 -- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a1340 msgr2=0x7fd6c0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a1340 0x7fd6c0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 -- 172.21.15.32:0/3074249291 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00a1fe0 con 0x7fd6c00a0a30 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c498a700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a1340 0x7fd6c0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6c518b700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c00a0a30 0x7fd6c00a0e00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009e50 tx=0x7fd6b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c69b04f6bf271e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6bffff700 1 -- 172.21.15.32:0/3074249291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000e070 con 0x7fd6c00a0a30 2023-09-30T22:25:35.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6bffff700 1 -- 172.21.15.32:0/3074249291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000b910 con 0x7fd6c00a0a30 2023-09-30T22:25:35.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.499+0000 7fd6bffff700 1 -- 172.21.15.32:0/3074249291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000bcc0 con 0x7fd6c00a0a30 2023-09-30T22:25:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 -- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c00a0a30 msgr2=0x7fd6c00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c00a0a30 0x7fd6c00a0e00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009e50 tx=0x7fd6b000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 -- 172.21.15.32:0/3074249291 shutdown_connections 2023-09-30T22:25:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c0133fa0 0x7fd6c0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c00a0a30 0x7fd6c00a0e00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/3074249291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a1340 0x7fd6c0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 -- 172.21.15.32:0/3074249291 >> 172.21.15.32:0/3074249291 conn(0x7fd6c00a6550 msgr2=0x7fd6c00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 -- 172.21.15.32:0/3074249291 shutdown_connections 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.500+0000 7fd6c73ef700 1 -- 172.21.15.32:0/3074249291 wait complete. 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.501+0000 7fd6c73ef700 1 Processor -- start 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.501+0000 7fd6c73ef700 1 -- start start 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c73ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a0a30 0x7fd6c0120380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c73ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 0x7fd6c01208c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c73ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 0x7fd6c0120e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c73ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0133770 con 0x7fd6c0133fa0 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c73ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01231e0 con 0x7fd6c00a0a30 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c73ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0122570 con 0x7fd6c00a1340 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 0x7fd6c01208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c598c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 0x7fd6c0120e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c518b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a0a30 0x7fd6c0120380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 0x7fd6c01208c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54740/0 (socket says 172.21.15.32:54740) 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 -- 172.21.15.32:0/2194621537 learned_addr learned my addr 172.21.15.32:0/2194621537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c518b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a0a30 0x7fd6c0120380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38696/0 (socket says 172.21.15.32:38696) 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c598c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 0x7fd6c0120e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53956/0 (socket says 172.21.15.32:53956) 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 -- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a0a30 msgr2=0x7fd6c0120380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a0a30 0x7fd6c0120380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 -- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 msgr2=0x7fd6c0120e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 0x7fd6c0120e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c498a700 1 -- 172.21.15.32:0/2194621537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40094a0 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.502+0000 7fd6c598c700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 0x7fd6c0120e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.503+0000 7fd6c498a700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 0x7fd6c01208c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400ebe0 tx=0x7fd6b400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.503+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400c850 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.503+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b400c9b0 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.503+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400cd40 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.503+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0009b30 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.503+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c00a3410 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.504+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6b4013470 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.504+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd6b403f8e0 con 0x7fd6c00a1340 2023-09-30T22:25:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.504+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c00a0220 con 0x7fd6c00a1340 2023-09-30T22:25:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.509+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6b4013720 con 0x7fd6c00a1340 2023-09-30T22:25:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 Processor -- start 2023-09-30T22:25:35.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 -- start start 2023-09-30T22:25:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 0x7f84380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 0x7f84380a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8438133930 0x7f8438133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84380a2a70 con 0x7f84380a0a30 2023-09-30T22:25:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84380a01d0 con 0x7f8438133930 2023-09-30T22:25:35.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843e83c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f843809e2b0 con 0x7f843809d630 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f8437fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 0x7f84380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f84377fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 0x7f84380a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f843cdd9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8438133930 0x7f8438133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f84377fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 0x7f84380a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54746/0 (socket says 172.21.15.32:54746) 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f8437fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 0x7f84380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53958/0 (socket says 172.21.15.32:53958) 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.536+0000 7f84377fe700 1 -- 172.21.15.32:0/3994836928 learned_addr learned my addr 172.21.15.32:0/3994836928 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f84377fe700 1 -- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8438133930 msgr2=0x7f8438133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f84377fe700 1 --2- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8438133930 0x7f8438133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f84377fe700 1 -- 172.21.15.32:0/3994836928 >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 msgr2=0x7f84380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:35.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f84377fe700 1 --2- 172.21.15.32:0/3994836928 >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 0x7f84380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f84377fe700 1 -- 172.21.15.32:0/3994836928 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843809dc50 con 0x7f843809d630 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f843cdd9700 1 --2- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8438133930 0x7f8438133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f8437fff700 1 --2- 172.21.15.32:0/3994836928 >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 0x7f84380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f84377fe700 1 --2- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 0x7f84380a1340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098f0 tx=0x7f842c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c505efcefd8a37b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f8436ffd700 1 -- 172.21.15.32:0/3994836928 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c011830 con 0x7f843809d630 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f8436ffd700 1 -- 172.21.15.32:0/3994836928 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c011990 con 0x7f843809d630 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.537+0000 7f8436ffd700 1 -- 172.21.15.32:0/3994836928 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c011d20 con 0x7f843809d630 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 -- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 msgr2=0x7f84380a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 --2- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 0x7f84380a1340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098f0 tx=0x7f842c009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 -- 172.21.15.32:0/3994836928 shutdown_connections 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 --2- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8438133930 0x7f8438133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 --2- 172.21.15.32:0/3994836928 >> v2:172.21.15.32:3300/0 conn(0x7f84380a0a30 0x7f84380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 --2- 172.21.15.32:0/3994836928 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f843809d630 0x7f84380a1340 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 -- 172.21.15.32:0/3994836928 >> 172.21.15.32:0/3994836928 conn(0x7f84380a3050 msgr2=0x7f84380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 -- 172.21.15.32:0/3994836928 shutdown_connections 2023-09-30T22:25:35.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.538+0000 7f843e83c700 1 -- 172.21.15.32:0/3994836928 wait complete. 2023-09-30T22:25:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.539+0000 7f843e83c700 1 Processor -- start 2023-09-30T22:25:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 -- start start 2023-09-30T22:25:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f843809d630 0x7f84381210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 0x7f8438121600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8438133930 0x7f8438121b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8438125510 con 0x7f84380a0a30 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8438124b50 con 0x7f843809d630 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843e83c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84381242e0 con 0x7f8438133930 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f84377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 0x7f8438121600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f84377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 0x7f8438121600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53972/0 (socket says 172.21.15.32:53972) 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f84377fe700 1 -- 172.21.15.32:0/17977886 learned_addr learned my addr 172.21.15.32:0/17977886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f843cdd9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8438133930 0x7f8438121b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.540+0000 7f8437fff700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f843809d630 0x7f84381210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f84377fe700 1 -- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8438133930 msgr2=0x7f8438121b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f84377fe700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8438133930 0x7f8438121b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f84377fe700 1 -- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f843809d630 msgr2=0x7f84381210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f84377fe700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f843809d630 0x7f84381210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f84377fe700 1 -- 172.21.15.32:0/17977886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280094a0 con 0x7f84380a0a30 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f843cdd9700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8438133930 0x7f8438121b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f84377fe700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 0x7f8438121600 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c004240 con 0x7f84380a0a30 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c01f070 con 0x7f84380a0a30 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8430009a70 con 0x7f84380a0a30 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0045c0 con 0x7f84380a0a30 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.541+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c009550 con 0x7f84380a0a30 2023-09-30T22:25:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.542+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f842c021030 con 0x7f84380a0a30 2023-09-30T22:25:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.542+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f842c040ab0 con 0x7f84380a0a30 2023-09-30T22:25:35.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.542+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843811df60 con 0x7f84380a0a30 2023-09-30T22:25:35.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.547+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f842c01a450 con 0x7f84380a0a30 2023-09-30T22:25:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.718+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd6c0074ea0 con 0x7fd6c00a1340 2023-09-30T22:25:35.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.718+0000 7fd6bdffb700 1 -- 172.21.15.32:0/2194621537 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6b402b090 con 0x7fd6c00a1340 2023-09-30T22:25:35.719 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 msgr2=0x7fd6c01208c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 0x7fd6c01208c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400ebe0 tx=0x7fd6b400eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 shutdown_connections 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6c0133fa0 0x7fd6c0120e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6c00a0a30 0x7fd6c0120380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 --2- 172.21.15.32:0/2194621537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6c00a1340 0x7fd6c01208c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 >> 172.21.15.32:0/2194621537 conn(0x7fd6c00a6550 msgr2=0x7fd6c01baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:35.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 shutdown_connections 2023-09-30T22:25:35.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.721+0000 7fd6c73ef700 1 -- 172.21.15.32:0/2194621537 wait complete. 2023-09-30T22:25:35.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.756+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f84380a0410 con 0x7f84380a0a30 2023-09-30T22:25:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.757+0000 7f8434ff9700 1 -- 172.21.15.32:0/17977886 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f842c0233d0 con 0x7f84380a0a30 2023-09-30T22:25:35.758 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 msgr2=0x7f8438121600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 0x7f8438121600 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 shutdown_connections 2023-09-30T22:25:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8438133930 0x7f8438121b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f843809d630 0x7f84381210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 --2- 172.21.15.32:0/17977886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84380a0a30 0x7f8438121600 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 >> 172.21.15.32:0/17977886 conn(0x7f84380a3050 msgr2=0x7f84381331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 shutdown_connections 2023-09-30T22:25:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:35.759+0000 7f843e83c700 1 -- 172.21.15.32:0/17977886 wait complete. 2023-09-30T22:25:35.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:35.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:36.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 Processor -- start 2023-09-30T22:25:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 -- start start 2023-09-30T22:25:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fe5f3365700 1 Processor -- start 2023-09-30T22:25:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fe5f3365700 1 -- start start 2023-09-30T22:25:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009d6f0 0x7fab1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1009b120 con 0x7fab1009e410 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1006b460 con 0x7fab1009e000 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.230+0000 7fab151c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab1009a7e0 con 0x7fab1009d6f0 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0f59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab1009e410 0x7fab10133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56520/0 (socket says 172.21.15.32:56520) 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0f59e700 1 -- 172.21.15.32:0/4264356512 learned_addr learned my addr 172.21.15.32:0/4264356512 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0ed9d700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009d6f0 0x7fab1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0e59c700 1 -- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009d6f0 msgr2=0x7fab1009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0e59c700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009d6f0 0x7fab1009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f3365700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec09e1d0 0x7fe5ec138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f3365700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5ec139170 0x7fe5ec13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0e59c700 1 -- 172.21.15.32:0/4264356512 >> v1:172.21.15.32:6789/0 conn(0x7fab1009e410 legacy=0x7fab10133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0e59c700 1 -- 172.21.15.32:0/4264356512 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1009a500 con 0x7fab1009e000 2023-09-30T22:25:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f3365700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec09b120 con 0x7fe5ec09d6f0 2023-09-30T22:25:36.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f3365700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec06b460 con 0x7fe5ec139170 2023-09-30T22:25:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f3365700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec09a7e0 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fab0ed9d700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009d6f0 0x7fab1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f1101700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5ec09d6f0 0x7fe5ec09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56526/0 (socket says 172.21.15.32:56526) 2023-09-30T22:25:36.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f1101700 1 -- 172.21.15.32:0/2837638496 learned_addr learned my addr 172.21.15.32:0/2837638496 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f0900700 1 --2- 172.21.15.32:0/2837638496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec09e1d0 0x7fe5ec138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.231+0000 7fe5f1902700 1 --2- 172.21.15.32:0/2837638496 >> v2:172.21.15.32:3300/0 conn(0x7fe5ec139170 0x7fe5ec13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5e3fff700 1 -- 172.21.15.32:0/2837638496 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692316596 0 0) 0x7fe5ec09b120 con 0x7fe5ec09d6f0 2023-09-30T22:25:36.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fab0e59c700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab101339f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab04009870 tx=0x7fab04009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c4c885a77f9647 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5e3fff700 1 -- 172.21.15.32:0/2837638496 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d80035b0 con 0x7fe5ec09d6f0 2023-09-30T22:25:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5f0900700 1 -- 172.21.15.32:0/2837638496 >> v2:172.21.15.32:3300/0 conn(0x7fe5ec139170 msgr2=0x7fe5ec13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5f0900700 1 --2- 172.21.15.32:0/2837638496 >> v2:172.21.15.32:3300/0 conn(0x7fe5ec139170 0x7fe5ec13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5f0900700 1 -- 172.21.15.32:0/2837638496 >> v1:172.21.15.32:6789/0 conn(0x7fe5ec09d6f0 legacy=0x7fe5ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5f0900700 1 -- 172.21.15.32:0/2837638496 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec09a500 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5f1902700 1 --2- 172.21.15.32:0/2837638496 >> v2:172.21.15.32:3300/0 conn(0x7fe5ec139170 0x7fe5ec13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:25:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fab0dd9b700 1 -- 172.21.15.32:0/4264356512 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04011830 con 0x7fab1009e000 2023-09-30T22:25:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fab0dd9b700 1 -- 172.21.15.32:0/4264356512 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab04011990 con 0x7fab1009e000 2023-09-30T22:25:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fab0dd9b700 1 -- 172.21.15.32:0/4264356512 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04011d20 con 0x7fab1009e000 2023-09-30T22:25:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5f0900700 1 --2- 172.21.15.32:0/2837638496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec09e1d0 0x7fe5ec138c30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009870 tx=0x7fe5dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d549d95471b7aa4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5e3fff700 1 -- 172.21.15.32:0/2837638496 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc011830 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5e3fff700 1 -- 172.21.15.32:0/2837638496 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc011990 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.232+0000 7fe5e3fff700 1 -- 172.21.15.32:0/2837638496 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc011d20 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 -- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 msgr2=0x7fab101339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab101339f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab04009870 tx=0x7fab04009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 -- 172.21.15.32:0/4264356512 shutdown_connections 2023-09-30T22:25:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009d6f0 0x7fab1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 --2- 172.21.15.32:0/4264356512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab101339f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 -- 172.21.15.32:0/4264356512 >> 172.21.15.32:0/4264356512 conn(0x7fab10066f50 msgr2=0x7fab1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 -- 172.21.15.32:0/4264356512 shutdown_connections 2023-09-30T22:25:36.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fab151c8700 1 -- 172.21.15.32:0/4264356512 wait complete. 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 -- 172.21.15.32:0/2837638496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec09e1d0 msgr2=0x7fe5ec138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 --2- 172.21.15.32:0/2837638496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec09e1d0 0x7fe5ec138c30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009870 tx=0x7fe5dc009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 -- 172.21.15.32:0/2837638496 shutdown_connections 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 --2- 172.21.15.32:0/2837638496 >> v2:172.21.15.32:3300/0 conn(0x7fe5ec139170 0x7fe5ec13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 --2- 172.21.15.32:0/2837638496 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec09e1d0 0x7fe5ec138c30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 -- 172.21.15.32:0/2837638496 >> 172.21.15.32:0/2837638496 conn(0x7fe5ec066f50 msgr2=0x7fe5ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 -- 172.21.15.32:0/2837638496 shutdown_connections 2023-09-30T22:25:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.233+0000 7fe5f3365700 1 -- 172.21.15.32:0/2837638496 wait complete. 2023-09-30T22:25:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.234+0000 7fe5f3365700 1 Processor -- start 2023-09-30T22:25:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 Processor -- start 2023-09-30T22:25:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 -- start start 2023-09-30T22:25:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 -- start start 2023-09-30T22:25:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ec09d6f0 0x7fe5ec120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5ec09e1d0 0x7fe5ec120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 0x7fe5ec121090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec122410 con 0x7fe5ec09d6f0 2023-09-30T22:25:36.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec122220 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f3365700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec136590 con 0x7fe5ec139170 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 0x7fab10122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab10122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 0x7fab10122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab10126060 con 0x7fab1009d6f0 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1009c710 con 0x7fab1009e000 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fab151c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab10124150 con 0x7fab1009e410 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 0x7fab10122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 0x7fab10122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.235+0000 7fe5f1902700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 0x7fe5ec121090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f1902700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 0x7fe5ec121090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54786/0 (socket says 172.21.15.32:54786) 2023-09-30T22:25:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f1902700 1 -- 172.21.15.32:0/918915105 learned_addr learned my addr 172.21.15.32:0/918915105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5ec09e1d0 0x7fe5ec120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f1101700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ec09d6f0 0x7fe5ec120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 0x7fab10122540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54014/0 (socket says 172.21.15.32:54014) 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0ed9d700 1 -- 172.21.15.32:0/1916821425 learned_addr learned my addr 172.21.15.32:0/1916821425 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab10122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 0x7fab10122fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54802/0 (socket says 172.21.15.32:54802) 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 -- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 msgr2=0x7fab10122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab10122a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 -- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 msgr2=0x7fe5ec121090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 0x7fe5ec121090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 -- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 msgr2=0x7fab10122540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 0x7fab10122540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 -- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ec09d6f0 msgr2=0x7fe5ec120610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ec09d6f0 0x7fe5ec120610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 -- 172.21.15.32:0/918915105 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0094a0 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f1902700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 0x7fe5ec121090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 -- 172.21.15.32:0/1916821425 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000094a0 con 0x7fab1009e410 2023-09-30T22:25:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0e59c700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab10122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0ed9d700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 0x7fab10122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fe5f0900700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5ec09e1d0 0x7fe5ec120b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc016000 tx=0x7fe5dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.236+0000 7fab0f59e700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 0x7fab10122fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab0800ca70 tx=0x7fab0800cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0036a0 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc01f070 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc003a20 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec09c710 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800a9a0 con 0x7fab1009e410 2023-09-30T22:25:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0800ab00 con 0x7fab1009e410 2023-09-30T22:25:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab040094a0 con 0x7fab1009e410 2023-09-30T22:25:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800ae70 con 0x7fab1009e410 2023-09-30T22:25:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab10133490 con 0x7fab1009e410 2023-09-30T22:25:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.237+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec122a10 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.238+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5dc003b80 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.238+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec074ea0 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.238+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab10074ea0 con 0x7fab1009e410 2023-09-30T22:25:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.238+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5dc021070 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.239+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab0801b3b0 con 0x7fab1009e410 2023-09-30T22:25:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.239+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab0803f6b0 con 0x7fab1009e410 2023-09-30T22:25:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.243+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5dc02f3a0 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.243+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab080192c0 con 0x7fab1009e410 2023-09-30T22:25:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.448+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fab10133060 con 0x7fab1009e410 2023-09-30T22:25:36.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.448+0000 7faaf77fe700 1 -- 172.21.15.32:0/1916821425 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab0801a360 con 0x7fab1009e410 2023-09-30T22:25:36.450 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:36.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.450+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe5ec09f420 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.450+0000 7fe5e1ffb700 1 -- 172.21.15.32:0/918915105 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5dc02b070 con 0x7fe5ec09e1d0 2023-09-30T22:25:36.451 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 msgr2=0x7fab10122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 0x7fab10122fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab0800ca70 tx=0x7fab0800cd80 comp rx=0 tx=0).stop 2023-09-30T22:25:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 shutdown_connections 2023-09-30T22:25:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab1009e410 0x7fab10122fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab1009d6f0 0x7fab10122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 --2- 172.21.15.32:0/1916821425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab1009e000 0x7fab10122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 >> 172.21.15.32:0/1916821425 conn(0x7fab10066f50 msgr2=0x7fab1006f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:36.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 shutdown_connections 2023-09-30T22:25:36.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.451+0000 7fab151c8700 1 -- 172.21.15.32:0/1916821425 wait complete. 2023-09-30T22:25:36.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5ec09e1d0 msgr2=0x7fe5ec120b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5ec09e1d0 0x7fe5ec120b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc016000 tx=0x7fe5dc004750 comp rx=0 tx=0).stop 2023-09-30T22:25:36.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 shutdown_connections 2023-09-30T22:25:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5ec139170 0x7fe5ec121090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ec09d6f0 0x7fe5ec120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 --2- 172.21.15.32:0/918915105 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5ec09e1d0 0x7fe5ec120b50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 >> 172.21.15.32:0/918915105 conn(0x7fe5ec066f50 msgr2=0x7fe5ec09b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 shutdown_connections 2023-09-30T22:25:36.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.453+0000 7fe5f3365700 1 -- 172.21.15.32:0/918915105 wait complete. 2023-09-30T22:25:36.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:36.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:36.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:36.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.885+0000 7fa2507c1700 1 Processor -- start 2023-09-30T22:25:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.885+0000 7fa2507c1700 1 -- start start 2023-09-30T22:25:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa2507c1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24809e7e0 0x7fa24809ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa2507c1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa24809fa10 0x7fa24809f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa2507c1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24809b610 con 0x7fa2481316a0 2023-09-30T22:25:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa2507c1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24809f630 con 0x7fa24809fa10 2023-09-30T22:25:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa2507c1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa24809bbb0 con 0x7fa24809e7e0 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa24dd5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa24809fa10 0x7fa24809f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa24dd5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa24809fa10 0x7fa24809f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54028/0 (socket says 172.21.15.32:54028) 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa24e55d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24809e7e0 0x7fa24809ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.886+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1740953084 learned_addr learned my addr 172.21.15.32:0/1740953084 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24809e7e0 msgr2=0x7fa24809ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24dd5c700 1 --2- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24809e7e0 0x7fa24809ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1740953084 >> v1:172.21.15.32:6789/0 conn(0x7fa2481316a0 legacy=0x7fa248131b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24d55b700 1 -- 172.21.15.32:0/1740953084 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205758621 0 0) 0x7fa24809b610 con 0x7fa2481316a0 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1740953084 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2480a0010 con 0x7fa24809fa10 2023-09-30T22:25:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24e55d700 1 --2- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24809e7e0 0x7fa24809ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24dd5c700 1 --2- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24809f0f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa2380098f0 tx=0x7fa238009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ae9d5a9d028c78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24d55b700 1 -- 172.21.15.32:0/1740953084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238011830 con 0x7fa24809fa10 2023-09-30T22:25:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24d55b700 1 -- 172.21.15.32:0/1740953084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa238011990 con 0x7fa24809fa10 2023-09-30T22:25:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.887+0000 7fa24d55b700 1 -- 172.21.15.32:0/1740953084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238011d20 con 0x7fa24809fa10 2023-09-30T22:25:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 -- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 msgr2=0x7fa24809f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24809f0f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa2380098f0 tx=0x7fa238009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 -- 172.21.15.32:0/1740953084 shutdown_connections 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24809e7e0 0x7fa24809ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1740953084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24809f0f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 -- 172.21.15.32:0/1740953084 >> 172.21.15.32:0/1740953084 conn(0x7fa2480a4300 msgr2=0x7fa2480a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 -- 172.21.15.32:0/1740953084 shutdown_connections 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.888+0000 7fa2507c1700 1 -- 172.21.15.32:0/1740953084 wait complete. 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.889+0000 7fa2507c1700 1 Processor -- start 2023-09-30T22:25:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 -- start start 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24809e7e0 0x7fa24806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2481316a0 0x7fa24806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa248121a60 con 0x7fa24809fa10 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa248120a00 con 0x7fa24809e7e0 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa2507c1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa248121550 con 0x7fa2481316a0 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa24dd5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa24dd5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54044/0 (socket says 172.21.15.32:54044) 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1445298357 learned_addr learned my addr 172.21.15.32:0/1445298357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa24e55d700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24809e7e0 0x7fa24806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.890+0000 7fa24ed5e700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2481316a0 0x7fa24806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2481316a0 msgr2=0x7fa24806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24dd5c700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2481316a0 0x7fa24806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24809e7e0 msgr2=0x7fa24806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24dd5c700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24809e7e0 0x7fa24806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24dd5c700 1 -- 172.21.15.32:0/1445298357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2440094a0 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24e55d700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24809e7e0 0x7fa24806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa24dd5c700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24806bdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa238013000 tx=0x7fa238004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238004300 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23801f070 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa238009550 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2380036a0 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.891+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa248121250 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.892+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa238021030 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.892+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa238040780 con 0x7fa24809fa10 2023-09-30T22:25:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.892+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2481211f0 con 0x7fa24809fa10 2023-09-30T22:25:36.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.901+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa23801a450 con 0x7fa24809fa10 2023-09-30T22:25:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 Processor -- start 2023-09-30T22:25:36.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 -- start start 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 0x7f10fc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10fc134170 0x7f10fc13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc09edd0 con 0x7f10fc0a1340 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc09e1f0 con 0x7f10fc134170 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f1101b77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc09dc50 con 0x7f10fc0a0a30 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.926+0000 7f10fb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 0x7f10fc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 0x7f10fc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38758/0 (socket says 172.21.15.32:38758) 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10faffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10fc0a1340 0x7f10fc133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56540/0 (socket says 172.21.15.32:56540) 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10fc134170 0x7f10fc13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 -- 172.21.15.32:0/825180273 learned_addr learned my addr 172.21.15.32:0/825180273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 -- 172.21.15.32:0/825180273 >> v2:172.21.15.32:3300/0 conn(0x7f10fc134170 msgr2=0x7f10fc13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 --2- 172.21.15.32:0/825180273 >> v2:172.21.15.32:3300/0 conn(0x7f10fc134170 0x7f10fc13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 -- 172.21.15.32:0/825180273 >> v1:172.21.15.32:6789/0 conn(0x7f10fc0a1340 legacy=0x7f10fc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 -- 172.21.15.32:0/825180273 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0a1fe0 con 0x7f10fc0a0a30 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fa7fc700 1 -- 172.21.15.32:0/825180273 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556214221 0 0) 0x7f10fc09edd0 con 0x7f10fc0a1340 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fbfff700 1 --2- 172.21.15.32:0/825180273 >> v2:172.21.15.32:3300/0 conn(0x7f10fc134170 0x7f10fc13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fb7fe700 1 --2- 172.21.15.32:0/825180273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 0x7f10fc0a0e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098f0 tx=0x7f10ec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324ff6cd766936cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fa7fc700 1 -- 172.21.15.32:0/825180273 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec011830 con 0x7f10fc0a0a30 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fa7fc700 1 -- 172.21.15.32:0/825180273 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec011990 con 0x7f10fc0a0a30 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.927+0000 7f10fa7fc700 1 -- 172.21.15.32:0/825180273 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec011d20 con 0x7f10fc0a0a30 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 -- 172.21.15.32:0/825180273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 msgr2=0x7f10fc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 --2- 172.21.15.32:0/825180273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 0x7f10fc0a0e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098f0 tx=0x7f10ec009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 -- 172.21.15.32:0/825180273 shutdown_connections 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 --2- 172.21.15.32:0/825180273 >> v2:172.21.15.32:3300/0 conn(0x7f10fc134170 0x7f10fc13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 --2- 172.21.15.32:0/825180273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a0a30 0x7f10fc0a0e00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 -- 172.21.15.32:0/825180273 >> 172.21.15.32:0/825180273 conn(0x7f10fc0a6550 msgr2=0x7f10fc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 -- 172.21.15.32:0/825180273 shutdown_connections 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.928+0000 7f1101b77700 1 -- 172.21.15.32:0/825180273 wait complete. 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.929+0000 7f1101b77700 1 Processor -- start 2023-09-30T22:25:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 -- start start 2023-09-30T22:25:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 0x7f10fc123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a1340 0x7f10fc123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10fc134170 0x7f10fc1c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc125ef0 con 0x7f10fc0a0a30 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc126120 con 0x7f10fc0a1340 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f1101b77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc1244f0 con 0x7f10fc134170 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.930+0000 7f10fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 0x7f10fc123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10fc134170 0x7f10fc1c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 0x7f10fc123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54058/0 (socket says 172.21.15.32:54058) 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 -- 172.21.15.32:0/630383431 learned_addr learned my addr 172.21.15.32:0/630383431 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10faffd700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a1340 0x7f10fc123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 -- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10fc134170 msgr2=0x7f10fc1c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10fc134170 0x7f10fc1c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 -- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a1340 msgr2=0x7f10fc123780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a1340 0x7f10fc123780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 -- 172.21.15.32:0/630383431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4009a70 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10fb7fe700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 0x7f10fc123240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009c40 tx=0x7f10ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004340 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec01f070 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec009550 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0036a0 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.931+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc124ce0 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.932+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10ec021030 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.932+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f10ec040970 con 0x7f10fc0a0a30 2023-09-30T22:25:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.933+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10fc0a1130 con 0x7f10fc0a0a30 2023-09-30T22:25:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:36.937+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10ec015080 con 0x7f10fc0a0a30 2023-09-30T22:25:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.108+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa24804f2a0 con 0x7fa24809fa10 2023-09-30T22:25:37.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.108+0000 7fa23effd700 1 -- 172.21.15.32:0/1445298357 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2380193b0 con 0x7fa24809fa10 2023-09-30T22:25:37.109 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 msgr2=0x7fa24806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24806bdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa238013000 tx=0x7fa238004750 comp rx=0 tx=0).stop 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 shutdown_connections 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2481316a0 0x7fa24806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24809e7e0 0x7fa24806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 --2- 172.21.15.32:0/1445298357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa24809fa10 0x7fa24806bdf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 >> 172.21.15.32:0/1445298357 conn(0x7fa2480a4300 msgr2=0x7fa248130c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 shutdown_connections 2023-09-30T22:25:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.111+0000 7fa2507c1700 1 -- 172.21.15.32:0/1445298357 wait complete. 2023-09-30T22:25:37.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.141+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f10fc074ea0 con 0x7f10fc0a0a30 2023-09-30T22:25:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.142+0000 7f10e3fff700 1 -- 172.21.15.32:0/630383431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10ec019770 con 0x7f10fc0a0a30 2023-09-30T22:25:37.143 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 msgr2=0x7f10fc123240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 0x7f10fc123240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009c40 tx=0x7f10ec004750 comp rx=0 tx=0).stop 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 shutdown_connections 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10fc134170 0x7f10fc1c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10fc0a0a30 0x7f10fc123240 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 --2- 172.21.15.32:0/630383431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10fc0a1340 0x7f10fc123780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 >> 172.21.15.32:0/630383431 conn(0x7f10fc0a6550 msgr2=0x7f10fc09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 shutdown_connections 2023-09-30T22:25:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.144+0000 7f1101b77700 1 -- 172.21.15.32:0/630383431 wait complete. 2023-09-30T22:25:37.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:37.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:37.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:37.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.616+0000 7f923f8c6700 1 Processor -- start 2023-09-30T22:25:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.616+0000 7f4ead89d700 1 Processor -- start 2023-09-30T22:25:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.616+0000 7f4ead89d700 1 -- start start 2023-09-30T22:25:37.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.616+0000 7f923f8c6700 1 -- start start 2023-09-30T22:25:37.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.616+0000 7f923f8c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809d6f0 0x7f923809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.616+0000 7f4ead89d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923f8c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f923809e410 0x7f9238130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ead89d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f650 0x7f4ea8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923f8c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923809f130 con 0x7f923809e000 2023-09-30T22:25:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923f8c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923809f520 con 0x7f923809e410 2023-09-30T22:25:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923f8c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923809e930 con 0x7f923809d6f0 2023-09-30T22:25:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ead89d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea8070940 con 0x7f4ea809f240 2023-09-30T22:25:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ead89d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea806fd60 con 0x7f4ea809f650 2023-09-30T22:25:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ead89d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea806fa80 con 0x7f4ea809e930 2023-09-30T22:25:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923de63700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f923809e410 0x7f9238130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923ce61700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f923809e000 0x7f9238130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56548/0 (socket says 172.21.15.32:56548) 2023-09-30T22:25:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923ce61700 1 -- 172.21.15.32:0/3008181754 learned_addr learned my addr 172.21.15.32:0/3008181754 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923d662700 1 --2- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809d6f0 0x7f923809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f650 0x7f4ea8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54834/0 (socket says 172.21.15.32:54834) 2023-09-30T22:25:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 -- 172.21.15.32:0/3456337343 learned_addr learned my addr 172.21.15.32:0/3456337343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f650 0x7f4ea8131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38774/0 (socket says 172.21.15.32:38774) 2023-09-30T22:25:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 -- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f650 msgr2=0x7f4ea8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 --2- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f650 0x7f4ea8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 -- 172.21.15.32:0/3456337343 >> v1:172.21.15.32:6789/0 conn(0x7f4ea809f240 legacy=0x7f4ea81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f4ea6ffd700 1 -- 172.21.15.32:0/3456337343 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea809fb60 con 0x7f4ea809e930 2023-09-30T22:25:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923de63700 1 -- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809d6f0 msgr2=0x7f923809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923de63700 1 --2- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809d6f0 0x7f923809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923de63700 1 -- 172.21.15.32:0/3008181754 >> v1:172.21.15.32:6789/0 conn(0x7f923809e000 legacy=0x7f9238130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.617+0000 7f923de63700 1 -- 172.21.15.32:0/3008181754 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923809b2d0 con 0x7f923809e410 2023-09-30T22:25:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f4ea6ffd700 1 --2- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea809ed00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800dd40 tx=0x7f4e9800f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b9c4827943f745 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f923de63700 1 --2- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809e410 0x7f9238130d70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f923000bfa0 tx=0x7f923000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464611702b2ea90e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:37.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f4ea5ffb700 1 -- 172.21.15.32:0/3456337343 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98012070 con 0x7f4ea809e930 2023-09-30T22:25:37.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f4ea5ffb700 1 -- 172.21.15.32:0/3456337343 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9800de90 con 0x7f4ea809e930 2023-09-30T22:25:37.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f4ea5ffb700 1 -- 172.21.15.32:0/3456337343 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800fb30 con 0x7f4ea809e930 2023-09-30T22:25:37.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f922ffff700 1 -- 172.21.15.32:0/3008181754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230010070 con 0x7f923809e410 2023-09-30T22:25:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f922ffff700 1 -- 172.21.15.32:0/3008181754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000b2e0 con 0x7f923809e410 2023-09-30T22:25:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f922ffff700 1 -- 172.21.15.32:0/3008181754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000dc40 con 0x7f923809e410 2023-09-30T22:25:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f4ead89d700 1 -- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 msgr2=0x7f4ea809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.618+0000 7f4ead89d700 1 --2- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea809ed00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800dd40 tx=0x7f4e9800f330 comp rx=0 tx=0).stop 2023-09-30T22:25:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f4ead89d700 1 -- 172.21.15.32:0/3456337343 shutdown_connections 2023-09-30T22:25:37.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f4ead89d700 1 --2- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f650 0x7f4ea8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f4ead89d700 1 --2- 172.21.15.32:0/3456337343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea809ed00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f4ead89d700 1 -- 172.21.15.32:0/3456337343 >> 172.21.15.32:0/3456337343 conn(0x7f4ea80a4210 msgr2=0x7f4ea80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:37.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 -- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809e410 msgr2=0x7f9238130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 --2- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809e410 0x7f9238130d70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f923000bfa0 tx=0x7f923000d330 comp rx=0 tx=0).stop 2023-09-30T22:25:37.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f4ead89d700 1 -- 172.21.15.32:0/3456337343 shutdown_connections 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f4ead89d700 1 -- 172.21.15.32:0/3456337343 wait complete. 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 -- 172.21.15.32:0/3008181754 shutdown_connections 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 --2- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809e410 0x7f9238130d70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 --2- 172.21.15.32:0/3008181754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809d6f0 0x7f923809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 -- 172.21.15.32:0/3008181754 >> 172.21.15.32:0/3008181754 conn(0x7f9238066f50 msgr2=0x7f923809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 -- 172.21.15.32:0/3008181754 shutdown_connections 2023-09-30T22:25:37.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.619+0000 7f923f8c6700 1 -- 172.21.15.32:0/3008181754 wait complete. 2023-09-30T22:25:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f4ead89d700 1 Processor -- start 2023-09-30T22:25:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 Processor -- start 2023-09-30T22:25:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 -- start start 2023-09-30T22:25:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f4ead89d700 1 -- start start 2023-09-30T22:25:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 0x7f923811df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923809e000 0x7f923811e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809e410 0x7f923811e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923811f430 con 0x7f923809d6f0 2023-09-30T22:25:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9238130340 con 0x7f923809e410 2023-09-30T22:25:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923f8c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9238120dc0 con 0x7f923809e000 2023-09-30T22:25:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923d662700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 0x7f923811df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f4ead89d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea81a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f4ead89d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 0x7f4ea81a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923d662700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 0x7f923811df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54086/0 (socket says 172.21.15.32:54086) 2023-09-30T22:25:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923d662700 1 -- 172.21.15.32:0/778903126 learned_addr learned my addr 172.21.15.32:0/778903126 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f4ead89d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ea809f650 0x7f4ea81a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ead89d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81a58d0 con 0x7f4ea809f650 2023-09-30T22:25:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923ce61700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923809e000 0x7f923811e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.620+0000 7f923de63700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809e410 0x7f923811e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ead89d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81a3fe0 con 0x7f4ea809f240 2023-09-30T22:25:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ead89d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81a5060 con 0x7f4ea809e930 2023-09-30T22:25:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 0x7f4ea81a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 0x7f4ea81a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38792/0 (socket says 172.21.15.32:38792) 2023-09-30T22:25:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 -- 172.21.15.32:0/1395475221 learned_addr learned my addr 172.21.15.32:0/1395475221 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea77fe700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ea809f650 0x7f4ea81a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923d662700 1 -- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923809e000 msgr2=0x7f923811e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923d662700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923809e000 0x7f923811e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923d662700 1 -- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809e410 msgr2=0x7f923811e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923d662700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809e410 0x7f923811e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923d662700 1 -- 172.21.15.32:0/778903126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923000bc80 con 0x7f923809d6f0 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea6ffd700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea81a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 -- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 msgr2=0x7f4ea81a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea81a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 -- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ea809f650 msgr2=0x7f4ea81a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ea809f650 0x7f4ea81a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 -- 172.21.15.32:0/1395475221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9800d9f0 con 0x7f4ea809f240 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea6ffd700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:25:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea77fe700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ea809f650 0x7f4ea81a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:25:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923d662700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 0x7f923811df60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9234009fc0 tx=0x7f923400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f4ea67fc700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 0x7f4ea81a2380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c4b0 tx=0x7f4e9c00c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234013980 con 0x7f923809d6f0 2023-09-30T22:25:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234013ae0 con 0x7f923809d6f0 2023-09-30T22:25:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234013e50 con 0x7f923809d6f0 2023-09-30T22:25:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.621+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9238121800 con 0x7f923809d6f0 2023-09-30T22:25:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.622+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92380a1d00 con 0x7f923809d6f0 2023-09-30T22:25:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.622+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0052d0 con 0x7f4ea809f240 2023-09-30T22:25:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.622+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c005430 con 0x7f4ea809f240 2023-09-30T22:25:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.622+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea80a0190 con 0x7f4ea809f240 2023-09-30T22:25:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.622+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0038c0 con 0x7f4ea809f240 2023-09-30T22:25:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.622+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea81b51a0 con 0x7f4ea809f240 2023-09-30T22:25:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.623+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92340243b0 con 0x7f923809d6f0 2023-09-30T22:25:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.623+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923809c760 con 0x7f923809d6f0 2023-09-30T22:25:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.623+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9234043370 con 0x7f923809d6f0 2023-09-30T22:25:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.623+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4e9c003a20 con 0x7f4ea809f240 2023-09-30T22:25:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.623+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4e9c03eba0 con 0x7f4ea809f240 2023-09-30T22:25:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.624+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e880052e0 con 0x7f4ea809f240 2023-09-30T22:25:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.628+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9234023360 con 0x7f923809d6f0 2023-09-30T22:25:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.629+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e9c017080 con 0x7f4ea809f240 2023-09-30T22:25:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.834+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f923811ac20 con 0x7f923809d6f0 2023-09-30T22:25:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.834+0000 7f922dffb700 1 -- 172.21.15.32:0/778903126 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9234020070 con 0x7f923809d6f0 2023-09-30T22:25:37.835 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 msgr2=0x7f923811df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 0x7f923811df60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9234009fc0 tx=0x7f923400eb50 comp rx=0 tx=0).stop 2023-09-30T22:25:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 shutdown_connections 2023-09-30T22:25:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f923809e410 0x7f923811e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f923809d6f0 0x7f923811df60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 --2- 172.21.15.32:0/778903126 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f923809e000 0x7f923811e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 >> 172.21.15.32:0/778903126 conn(0x7f9238066f50 msgr2=0x7f9238130a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 shutdown_connections 2023-09-30T22:25:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f923f8c6700 1 -- 172.21.15.32:0/778903126 wait complete. 2023-09-30T22:25:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.836+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e88005590 con 0x7f4ea809f240 2023-09-30T22:25:37.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.837+0000 7f4e8ffff700 1 -- 172.21.15.32:0/1395475221 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e9c00e280 con 0x7f4ea809f240 2023-09-30T22:25:37.839 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.840+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 msgr2=0x7f4ea81a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.840+0000 7f4ead89d700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 0x7f4ea81a2380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c4b0 tx=0x7f4e9c00c870 comp rx=0 tx=0).stop 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 shutdown_connections 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ea809f650 0x7f4ea81a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ea809e930 0x7f4ea81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 --2- 172.21.15.32:0/1395475221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ea809f240 0x7f4ea81a2380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 >> 172.21.15.32:0/1395475221 conn(0x7f4ea80a4210 msgr2=0x7f4ea81b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 shutdown_connections 2023-09-30T22:25:37.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:37.841+0000 7f4ead89d700 1 -- 172.21.15.32:0/1395475221 wait complete. 2023-09-30T22:25:37.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:37.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:38.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:38.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:38.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 Processor -- start 2023-09-30T22:25:38.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 -- start start 2023-09-30T22:25:38.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bc409d630 0x7f0bc40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc4138d80 0x7f0bc413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc40a2a70 con 0x7f0bc40a0a30 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc40a01d0 con 0x7f0bc409d630 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.277+0000 7f0bcc53f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc409e2b0 con 0x7f0bc4138d80 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bcaadc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc4138d80 0x7f0bc413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bca2db700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0bc40a0a30 0x7f0bc40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56570/0 (socket says 172.21.15.32:56570) 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bca2db700 1 -- 172.21.15.32:0/2185576056 learned_addr learned my addr 172.21.15.32:0/2185576056 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bc9ada700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bc409d630 0x7f0bc40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bc92d9700 1 -- 172.21.15.32:0/2185576056 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910388480 0 0) 0x7f0bc40a2a70 con 0x7f0bc40a0a30 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bc92d9700 1 -- 172.21.15.32:0/2185576056 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bb00035b0 con 0x7f0bc40a0a30 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bcaadc700 1 -- 172.21.15.32:0/2185576056 >> v2:172.21.15.32:3300/0 conn(0x7f0bc409d630 msgr2=0x7f0bc40a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bcaadc700 1 --2- 172.21.15.32:0/2185576056 >> v2:172.21.15.32:3300/0 conn(0x7f0bc409d630 0x7f0bc40a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bcaadc700 1 -- 172.21.15.32:0/2185576056 >> v1:172.21.15.32:6789/0 conn(0x7f0bc40a0a30 legacy=0x7f0bc40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bcaadc700 1 -- 172.21.15.32:0/2185576056 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc409dc50 con 0x7f0bc4138d80 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bc9ada700 1 --2- 172.21.15.32:0/2185576056 >> v2:172.21.15.32:3300/0 conn(0x7f0bc409d630 0x7f0bc40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bcaadc700 1 --2- 172.21.15.32:0/2185576056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc4138d80 0x7f0bc413b160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009d90 tx=0x7f0bbc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8b6bea0e88bd34 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.278+0000 7f0bc92d9700 1 -- 172.21.15.32:0/2185576056 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00e070 con 0x7f0bc4138d80 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bc92d9700 1 -- 172.21.15.32:0/2185576056 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc00bb20 con 0x7f0bc4138d80 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bc92d9700 1 -- 172.21.15.32:0/2185576056 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc012430 con 0x7f0bc4138d80 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 -- 172.21.15.32:0/2185576056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc4138d80 msgr2=0x7f0bc413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/2185576056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc4138d80 0x7f0bc413b160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009d90 tx=0x7f0bbc00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 -- 172.21.15.32:0/2185576056 shutdown_connections 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/2185576056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc4138d80 0x7f0bc413b160 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/2185576056 >> v2:172.21.15.32:3300/0 conn(0x7f0bc409d630 0x7f0bc40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 -- 172.21.15.32:0/2185576056 >> 172.21.15.32:0/2185576056 conn(0x7f0bc40a3050 msgr2=0x7f0bc40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 -- 172.21.15.32:0/2185576056 shutdown_connections 2023-09-30T22:25:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.279+0000 7f0bcc53f700 1 -- 172.21.15.32:0/2185576056 wait complete. 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.280+0000 7f0bcc53f700 1 Processor -- start 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.280+0000 7f0bcc53f700 1 -- start start 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcc53f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc409d630 0x7f0bc412c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcc53f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc40a0a30 0x7f0bc412c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcc53f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 0x7f0bc412ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcc53f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc412ecd0 con 0x7f0bc4138d80 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcc53f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc412eeb0 con 0x7f0bc409d630 2023-09-30T22:25:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcc53f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc413abc0 con 0x7f0bc40a0a30 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcaadc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 0x7f0bc412ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcaadc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 0x7f0bc412ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54114/0 (socket says 172.21.15.32:54114) 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bc9ada700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc40a0a30 0x7f0bc412c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bcaadc700 1 -- 172.21.15.32:0/1111958213 learned_addr learned my addr 172.21.15.32:0/1111958213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.281+0000 7f0bca2db700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc409d630 0x7f0bc412c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcaadc700 1 -- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc40a0a30 msgr2=0x7f0bc412c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcaadc700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc40a0a30 0x7f0bc412c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcaadc700 1 -- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc409d630 msgr2=0x7f0bc412c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcaadc700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc409d630 0x7f0bc412c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcaadc700 1 -- 172.21.15.32:0/1111958213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40094a0 con 0x7f0bc4138d80 2023-09-30T22:25:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bc9ada700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc40a0a30 0x7f0bc412c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcaadc700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 0x7f0bc412ccb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc014040 tx=0x7f0bbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc012a90 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc020070 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc009a70 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc012e70 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.282+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc40a0460 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.283+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0bbc004040 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.283+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc40747f0 con 0x7f0bc4138d80 2023-09-30T22:25:38.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.283+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0bbc022070 con 0x7f0bc4138d80 2023-09-30T22:25:38.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.288+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0bbc011980 con 0x7f0bc4138d80 2023-09-30T22:25:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.315+0000 7fb04f0f6700 1 Processor -- start 2023-09-30T22:25:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.315+0000 7fb04f0f6700 1 -- start start 2023-09-30T22:25:38.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04f0f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 0x7fb048138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04f0f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb048139250 0x7fb04813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04f0f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04809edd0 con 0x7fb0480a0a30 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04f0f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480a18e0 con 0x7fb0480a1510 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04f0f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb04809e1f0 con 0x7fb048139250 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 0x7fb048138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 0x7fb048138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38818/0 (socket says 172.21.15.32:38818) 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04ce92700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb0480a0a30 0x7fb0480a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56578/0 (socket says 172.21.15.32:56578) 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 -- 172.21.15.32:0/2078876588 learned_addr learned my addr 172.21.15.32:0/2078876588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb04d693700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb048139250 0x7fb04813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 -- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb048139250 msgr2=0x7fb04813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb048139250 0x7fb04813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 -- 172.21.15.32:0/2078876588 >> v1:172.21.15.32:6789/0 conn(0x7fb0480a0a30 legacy=0x7fb0480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb0477fe700 1 -- 172.21.15.32:0/2078876588 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600770619 0 0) 0x7fb04809edd0 con 0x7fb0480a0a30 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.316+0000 7fb047fff700 1 -- 172.21.15.32:0/2078876588 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04809dc50 con 0x7fb0480a1510 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb04d693700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb048139250 0x7fb04813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb047fff700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 0x7fb048138d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb034009920 tx=0x7fb034009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d51280e21017d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb0477fe700 1 -- 172.21.15.32:0/2078876588 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034011830 con 0x7fb0480a1510 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb0477fe700 1 -- 172.21.15.32:0/2078876588 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034011990 con 0x7fb0480a1510 2023-09-30T22:25:38.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb0477fe700 1 -- 172.21.15.32:0/2078876588 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034011d20 con 0x7fb0480a1510 2023-09-30T22:25:38.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb04f0f6700 1 -- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 msgr2=0x7fb048138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 0x7fb048138d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb034009920 tx=0x7fb034009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:38.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb04f0f6700 1 -- 172.21.15.32:0/2078876588 shutdown_connections 2023-09-30T22:25:38.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb048139250 0x7fb04813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.317+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/2078876588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0480a1510 0x7fb048138d10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.318+0000 7fb04f0f6700 1 -- 172.21.15.32:0/2078876588 >> 172.21.15.32:0/2078876588 conn(0x7fb0480a6550 msgr2=0x7fb0480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:38.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.318+0000 7fb04f0f6700 1 -- 172.21.15.32:0/2078876588 shutdown_connections 2023-09-30T22:25:38.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.318+0000 7fb04f0f6700 1 -- 172.21.15.32:0/2078876588 wait complete. 2023-09-30T22:25:38.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.319+0000 7fb04f0f6700 1 Processor -- start 2023-09-30T22:25:38.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.319+0000 7fb04f0f6700 1 -- start start 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04f0f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0480a0a30 0x7fb048122720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04f0f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0480a1510 0x7fb048122c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04f0f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb048139250 0x7fb0481231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04f0f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048138200 con 0x7fb0480a1510 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04f0f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481261f0 con 0x7fb048139250 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04f0f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb048125970 con 0x7fb0480a0a30 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04ce92700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0480a0a30 0x7fb048122720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb047fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0480a1510 0x7fb048122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb047fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0480a1510 0x7fb048122c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54128/0 (socket says 172.21.15.32:54128) 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb047fff700 1 -- 172.21.15.32:0/3844455499 learned_addr learned my addr 172.21.15.32:0/3844455499 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04d693700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb048139250 0x7fb0481231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04ce92700 1 -- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb048139250 msgr2=0x7fb0481231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04ce92700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb048139250 0x7fb0481231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.320+0000 7fb04ce92700 1 -- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0480a1510 msgr2=0x7fb048122c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb04ce92700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0480a1510 0x7fb048122c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb04ce92700 1 -- 172.21.15.32:0/3844455499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009a70 con 0x7fb0480a0a30 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb04d693700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb048139250 0x7fb0481231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb04ce92700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0480a0a30 0x7fb048122720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c470 tx=0x7fb03c00c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:38.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004c80 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c004de0 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034009550 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c007310 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.321+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb048125c50 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.322+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb03c007470 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.322+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb03c03f2d0 con 0x7fb0480a0a30 2023-09-30T22:25:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.322+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0480a1d00 con 0x7fb0480a0a30 2023-09-30T22:25:38.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.327+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb03c007720 con 0x7fb0480a0a30 2023-09-30T22:25:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.501+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0bc409c710 con 0x7f0bc4138d80 2023-09-30T22:25:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.501+0000 7f0bbaffd700 1 -- 172.21.15.32:0/1111958213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bbc01b400 con 0x7f0bc4138d80 2023-09-30T22:25:38.503 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 msgr2=0x7f0bc412ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 0x7f0bc412ccb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc014040 tx=0x7f0bbc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 shutdown_connections 2023-09-30T22:25:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bc4138d80 0x7f0bc412ccb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bc409d630 0x7f0bc412c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 --2- 172.21.15.32:0/1111958213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bc40a0a30 0x7f0bc412c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 >> 172.21.15.32:0/1111958213 conn(0x7f0bc40a3050 msgr2=0x7f0bc412aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 shutdown_connections 2023-09-30T22:25:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.504+0000 7f0bcc53f700 1 -- 172.21.15.32:0/1111958213 wait complete. 2023-09-30T22:25:38.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.534+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb048125e70 con 0x7fb0480a0a30 2023-09-30T22:25:38.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.535+0000 7fb0457fa700 1 -- 172.21.15.32:0/3844455499 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb03c011e80 con 0x7fb0480a0a30 2023-09-30T22:25:38.536 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0480a0a30 msgr2=0x7fb048122720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0480a0a30 0x7fb048122720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c470 tx=0x7fb03c00c780 comp rx=0 tx=0).stop 2023-09-30T22:25:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 shutdown_connections 2023-09-30T22:25:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb048139250 0x7fb0481231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0480a0a30 0x7fb048122720 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 --2- 172.21.15.32:0/3844455499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0480a1510 0x7fb048122c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 >> 172.21.15.32:0/3844455499 conn(0x7fb0480a6550 msgr2=0x7fb04809aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 shutdown_connections 2023-09-30T22:25:38.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:38.537+0000 7fb04f0f6700 1 -- 172.21.15.32:0/3844455499 wait complete. 2023-09-30T22:25:38.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:38.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:38.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.006+0000 7f2924ba9700 1 Processor -- start 2023-09-30T22:25:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.006+0000 7f2924ba9700 1 -- start start 2023-09-30T22:25:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f2924ba9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f29200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f2924ba9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f292013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f2924ba9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200a2a70 con 0x7f292009d630 2023-09-30T22:25:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f2924ba9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200a01d0 con 0x7f2920138d80 2023-09-30T22:25:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f2924ba9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f292009e2b0 con 0x7f29200a0a30 2023-09-30T22:25:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f291e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f29200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f291e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f29200a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54866/0 (socket says 172.21.15.32:54866) 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f291e59c700 1 -- 172.21.15.32:0/771949339 learned_addr learned my addr 172.21.15.32:0/771949339 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.007+0000 7f291ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f292013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291e59c700 1 -- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 msgr2=0x7f292013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291e59c700 1 --2- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f292013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291e59c700 1 -- 172.21.15.32:0/771949339 >> v1:172.21.15.32:6789/0 conn(0x7f292009d630 legacy=0x7f29200a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291e59c700 1 -- 172.21.15.32:0/771949339 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292009dc50 con 0x7f29200a0a30 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291d59a700 1 -- 172.21.15.32:0/771949339 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240021993 0 0) 0x7f29200a2a70 con 0x7f292009d630 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291ed9d700 1 --2- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f292013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291e59c700 1 --2- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f29200a0e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2910009e80 tx=0x7f291000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf7f0fc9fff3aa1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291d59a700 1 -- 172.21.15.32:0/771949339 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000e070 con 0x7f29200a0a30 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291d59a700 1 -- 172.21.15.32:0/771949339 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291000b910 con 0x7f29200a0a30 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.008+0000 7f291d59a700 1 -- 172.21.15.32:0/771949339 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291000bcc0 con 0x7f29200a0a30 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 -- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 msgr2=0x7f29200a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 --2- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f29200a0e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2910009e80 tx=0x7f291000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 -- 172.21.15.32:0/771949339 shutdown_connections 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 --2- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f292013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 --2- 172.21.15.32:0/771949339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f29200a0e00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 -- 172.21.15.32:0/771949339 >> 172.21.15.32:0/771949339 conn(0x7f29200a3050 msgr2=0x7f29200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 -- 172.21.15.32:0/771949339 shutdown_connections 2023-09-30T22:25:39.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.009+0000 7f2924ba9700 1 -- 172.21.15.32:0/771949339 wait complete. 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.010+0000 7f2924ba9700 1 Processor -- start 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.010+0000 7f2924ba9700 1 -- start start 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f2924ba9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 0x7f2920133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f2924ba9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f2920133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f2924ba9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f2920134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f2924ba9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200a0410 con 0x7f292009d630 2023-09-30T22:25:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f2924ba9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2920136ec0 con 0x7f2920138d80 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f2924ba9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f292013ac70 con 0x7f29200a0a30 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f2920133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f2920134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 0x7f2920133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f2920133cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54868/0 (socket says 172.21.15.32:54868) 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 0x7f2920133790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54144/0 (socket says 172.21.15.32:54144) 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291dd9b700 1 -- 172.21.15.32:0/2604661513 learned_addr learned my addr 172.21.15.32:0/2604661513 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 -- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 msgr2=0x7f2920133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f2920133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 -- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 msgr2=0x7f2920133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 0x7f2920133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 -- 172.21.15.32:0/2604661513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180094a0 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291e59c700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 0x7f2920133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.011+0000 7f291ed9d700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f2920134210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f292009ff90 tx=0x7f291800ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.012+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918013680 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.012+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918013800 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.012+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2910009b30 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.012+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918013bb0 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.012+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29201371a0 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.012+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2918013d10 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.013+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f291803f950 con 0x7f2920138d80 2023-09-30T22:25:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.013+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29200747f0 con 0x7f2920138d80 2023-09-30T22:25:39.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.018+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2918017080 con 0x7f2920138d80 2023-09-30T22:25:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.042+0000 7f6bfe845700 1 Processor -- start 2023-09-30T22:25:39.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfe845700 1 -- start start 2023-09-30T22:25:39.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfe845700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfe845700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bf8138d80 0x7f6bf813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfe845700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf80a2a70 con 0x7f6bf80a0a30 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfe845700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf80a01d0 con 0x7f6bf8138d80 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfe845700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf809e2b0 con 0x7f6bf809d630 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bfcde2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6bf8138d80 0x7f6bf813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bf77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bf7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6bf80a0a30 0x7f6bf80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56590/0 (socket says 172.21.15.32:56590) 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bf7fff700 1 -- 172.21.15.32:0/1474738479 learned_addr learned my addr 172.21.15.32:0/1474738479 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.043+0000 7f6bf77fe700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf80a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38850/0 (socket says 172.21.15.32:38850) 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bf6ffd700 1 -- 172.21.15.32:0/1474738479 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602687515 0 0) 0x7f6bf80a2a70 con 0x7f6bf80a0a30 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bf6ffd700 1 -- 172.21.15.32:0/1474738479 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be00035b0 con 0x7f6bf80a0a30 2023-09-30T22:25:39.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bfcde2700 1 -- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 msgr2=0x7f6bf80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bfcde2700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf80a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bfcde2700 1 -- 172.21.15.32:0/1474738479 >> v1:172.21.15.32:6789/0 conn(0x7f6bf80a0a30 legacy=0x7f6bf80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bfcde2700 1 -- 172.21.15.32:0/1474738479 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf809dc50 con 0x7f6bf8138d80 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bf77fe700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bfcde2700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf813b160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009d90 tx=0x7f6bf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5938d27f5e257979 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bf6ffd700 1 -- 172.21.15.32:0/1474738479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000e070 con 0x7f6bf8138d80 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bf6ffd700 1 -- 172.21.15.32:0/1474738479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf000bb20 con 0x7f6bf8138d80 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.044+0000 7f6bf6ffd700 1 -- 172.21.15.32:0/1474738479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0012430 con 0x7f6bf8138d80 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 -- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 msgr2=0x7f6bf813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf813b160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009d90 tx=0x7f6bf000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 -- 172.21.15.32:0/1474738479 shutdown_connections 2023-09-30T22:25:39.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf813b160 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1474738479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 -- 172.21.15.32:0/1474738479 >> 172.21.15.32:0/1474738479 conn(0x7f6bf80a3050 msgr2=0x7f6bf80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 -- 172.21.15.32:0/1474738479 shutdown_connections 2023-09-30T22:25:39.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.045+0000 7f6bfe845700 1 -- 172.21.15.32:0/1474738479 wait complete. 2023-09-30T22:25:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.046+0000 7f6bfe845700 1 Processor -- start 2023-09-30T22:25:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.046+0000 7f6bfe845700 1 -- start start 2023-09-30T22:25:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfe845700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf811c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfe845700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bf80a0a30 0x7f6bf811cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfe845700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf811d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfe845700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf811e5d0 con 0x7f6bf8138d80 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfe845700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf809d320 con 0x7f6bf809d630 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfe845700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80a0410 con 0x7f6bf80a0a30 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf811c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bf80a0a30 0x7f6bf811cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf811c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38866/0 (socket says 172.21.15.32:38866) 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 -- 172.21.15.32:0/1622078276 learned_addr learned my addr 172.21.15.32:0/1622078276 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bfcde2700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf811d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 -- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bf80a0a30 msgr2=0x7f6bf811cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bf80a0a30 0x7f6bf811cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 -- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 msgr2=0x7f6bf811d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf811d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 -- 172.21.15.32:0/1622078276 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0094a0 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf77fe700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bf80a0a30 0x7f6bf811cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.047+0000 7f6bf7fff700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf811c7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c700 tx=0x7f6be800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.048+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8005d70 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.048+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be80070d0 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.048+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8007460 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.048+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf0009a70 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.048+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf811ffe0 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.049+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6be80075c0 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.049+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6be803eaf0 con 0x7f6bf809d630 2023-09-30T22:25:39.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.049+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf811f8b0 con 0x7f6bf809d630 2023-09-30T22:25:39.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.054+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6be8018260 con 0x7f6bf809d630 2023-09-30T22:25:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.227+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2920072980 con 0x7f2920138d80 2023-09-30T22:25:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.228+0000 7f290effd700 1 -- 172.21.15.32:0/2604661513 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f291801adc0 con 0x7f2920138d80 2023-09-30T22:25:39.229 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 msgr2=0x7f2920134210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f2920134210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f292009ff90 tx=0x7f291800ebe0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 shutdown_connections 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2920138d80 0x7f2920134210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f292009d630 0x7f2920133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 --2- 172.21.15.32:0/2604661513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29200a0a30 0x7f2920133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 >> 172.21.15.32:0/2604661513 conn(0x7f29200a3050 msgr2=0x7f292009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 shutdown_connections 2023-09-30T22:25:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.230+0000 7f2924ba9700 1 -- 172.21.15.32:0/2604661513 wait complete. 2023-09-30T22:25:39.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.260+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6bf811e880 con 0x7f6bf809d630 2023-09-30T22:25:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.261+0000 7f6bf4ff9700 1 -- 172.21.15.32:0/1622078276 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6be8017020 con 0x7f6bf809d630 2023-09-30T22:25:39.262 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 msgr2=0x7f6bf811c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf811c7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6be800c700 tx=0x7f6be800ca10 comp rx=0 tx=0).stop 2023-09-30T22:25:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 shutdown_connections 2023-09-30T22:25:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bf8138d80 0x7f6bf811d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bf809d630 0x7f6bf811c7c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 --2- 172.21.15.32:0/1622078276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bf80a0a30 0x7f6bf811cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 >> 172.21.15.32:0/1622078276 conn(0x7f6bf80a3050 msgr2=0x7f6bf809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 shutdown_connections 2023-09-30T22:25:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.263+0000 7f6bfe845700 1 -- 172.21.15.32:0/1622078276 wait complete. 2023-09-30T22:25:39.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:39.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:39.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.668+0000 7fb3b9f9f700 1 Processor -- start 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.668+0000 7fb3b9f9f700 1 -- start start 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.668+0000 7fb3b9f9f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b40a1510 0x7fb3b4138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.668+0000 7fb3b9f9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b4139250 0x7fb3b413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b9f9f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b409edd0 con 0x7fb3b40a0a30 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b9f9f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b40a18e0 con 0x7fb3b40a1510 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b9f9f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b409e1f0 con 0x7fb3b4139250 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b40a1510 0x7fb3b4138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b40a1510 0x7fb3b4138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54164/0 (socket says 172.21.15.32:54164) 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb3b40a0a30 0x7fb3b40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56598/0 (socket says 172.21.15.32:56598) 2023-09-30T22:25:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b4139250 0x7fb3b413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/356528321 learned_addr learned my addr 172.21.15.32:0/356528321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b4139250 msgr2=0x7fb3b413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 --2- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b4139250 0x7fb3b413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/356528321 >> v1:172.21.15.32:6789/0 conn(0x7fb3b40a0a30 legacy=0x7fb3b40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/356528321 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b409dc50 con 0x7fb3b40a1510 2023-09-30T22:25:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b27fc700 1 -- 172.21.15.32:0/356528321 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699507072 0 0) 0x7fb3b409edd0 con 0x7fb3b40a0a30 2023-09-30T22:25:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b3fff700 1 --2- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b4139250 0x7fb3b413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b2ffd700 1 --2- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4138d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80098f0 tx=0x7fb3a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe50d07ade3ae57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.669+0000 7fb3b27fc700 1 -- 172.21.15.32:0/356528321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8011830 con 0x7fb3b40a1510 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b27fc700 1 -- 172.21.15.32:0/356528321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a8011990 con 0x7fb3b40a1510 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b27fc700 1 -- 172.21.15.32:0/356528321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8011d20 con 0x7fb3b40a1510 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 msgr2=0x7fb3b4138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4138d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80098f0 tx=0x7fb3a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/356528321 shutdown_connections 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b4139250 0x7fb3b413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/356528321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4138d10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/356528321 >> 172.21.15.32:0/356528321 conn(0x7fb3b40a6550 msgr2=0x7fb3b40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/356528321 shutdown_connections 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.670+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/356528321 wait complete. 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.671+0000 7fb3b9f9f700 1 Processor -- start 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 -- start start 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 0x7fb3b41228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3b4139250 0x7fb3b4123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41389e0 con 0x7fb3b40a1510 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40a0220 con 0x7fb3b40a0a30 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b9f9f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4126060 con 0x7fb3b4139250 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 0x7fb3b41228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 0x7fb3b41228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38880/0 (socket says 172.21.15.32:38880) 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b37fe700 1 -- 172.21.15.32:0/2753900231 learned_addr learned my addr 172.21.15.32:0/2753900231 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.672+0000 7fb3b3fff700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3b4139250 0x7fb3b4123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3b4139250 msgr2=0x7fb3b4123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b2ffd700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3b4139250 0x7fb3b4123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 msgr2=0x7fb3b41228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b2ffd700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 0x7fb3b41228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b2ffd700 1 -- 172.21.15.32:0/2753900231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0094a0 con 0x7fb3b40a1510 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b37fe700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 0x7fb3b41228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b2ffd700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4122e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8016000 tx=0x7fb3a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8004340 con 0x7fb3b40a1510 2023-09-30T22:25:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a801f070 con 0x7fb3b40a1510 2023-09-30T22:25:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a8009550 con 0x7fb3b40a1510 2023-09-30T22:25:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80036a0 con 0x7fb3b40a1510 2023-09-30T22:25:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.673+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b4124bb0 con 0x7fb3b40a1510 2023-09-30T22:25:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.674+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb3a8021030 con 0x7fb3b40a1510 2023-09-30T22:25:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.674+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb3a80408c0 con 0x7fb3b40a1510 2023-09-30T22:25:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.674+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b4074ea0 con 0x7fb3b40a1510 2023-09-30T22:25:39.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.679+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb3a8015070 con 0x7fb3b40a1510 2023-09-30T22:25:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.706+0000 7fb6fed9c700 1 Processor -- start 2023-09-30T22:25:39.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.706+0000 7fb6fed9c700 1 -- start start 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fed9c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6f809d630 0x7fb6f80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fed9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fed9c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80a2a70 con 0x7fb6f80a0a30 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fed9c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f80a01d0 con 0x7fb6f809d630 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fed9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f809e2b0 con 0x7fb6f8138d80 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fd339700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fd339700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38890/0 (socket says 172.21.15.32:38890) 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6fd339700 1 -- 172.21.15.32:0/4151980477 learned_addr learned my addr 172.21.15.32:0/4151980477 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.707+0000 7fb6f7fff700 1 --2- 172.21.15.32:0/4151980477 >> v2:172.21.15.32:3300/0 conn(0x7fb6f809d630 0x7fb6f80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6fd339700 1 -- 172.21.15.32:0/4151980477 >> v2:172.21.15.32:3300/0 conn(0x7fb6f809d630 msgr2=0x7fb6f80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6fd339700 1 --2- 172.21.15.32:0/4151980477 >> v2:172.21.15.32:3300/0 conn(0x7fb6f809d630 0x7fb6f80a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6fd339700 1 -- 172.21.15.32:0/4151980477 >> v1:172.21.15.32:6789/0 conn(0x7fb6f80a0a30 legacy=0x7fb6f80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6fd339700 1 -- 172.21.15.32:0/4151980477 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f809dc50 con 0x7fb6f8138d80 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6f7fff700 1 --2- 172.21.15.32:0/4151980477 >> v2:172.21.15.32:3300/0 conn(0x7fb6f809d630 0x7fb6f80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6fd339700 1 --2- 172.21.15.32:0/4151980477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f813b160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009e50 tx=0x7fb6f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54eb85bb11c2c5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6f77fe700 1 -- 172.21.15.32:0/4151980477 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000e070 con 0x7fb6f8138d80 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6f77fe700 1 -- 172.21.15.32:0/4151980477 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f000b910 con 0x7fb6f8138d80 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.708+0000 7fb6f77fe700 1 -- 172.21.15.32:0/4151980477 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000bcc0 con 0x7fb6f8138d80 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 -- 172.21.15.32:0/4151980477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 msgr2=0x7fb6f813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/4151980477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f813b160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009e50 tx=0x7fb6f000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:39.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 -- 172.21.15.32:0/4151980477 shutdown_connections 2023-09-30T22:25:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/4151980477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f813b160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/4151980477 >> v2:172.21.15.32:3300/0 conn(0x7fb6f809d630 0x7fb6f80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 -- 172.21.15.32:0/4151980477 >> 172.21.15.32:0/4151980477 conn(0x7fb6f80a3050 msgr2=0x7fb6f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 -- 172.21.15.32:0/4151980477 shutdown_connections 2023-09-30T22:25:39.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.709+0000 7fb6fed9c700 1 -- 172.21.15.32:0/4151980477 wait complete. 2023-09-30T22:25:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 Processor -- start 2023-09-30T22:25:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 -- start start 2023-09-30T22:25:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 0x7fb6f8120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 0x7fb6f8120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f81210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:39.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8124140 con 0x7fb6f809d630 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81234f0 con 0x7fb6f8138d80 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.710+0000 7fb6fed9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8138b20 con 0x7fb6f80a0a30 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fcb38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 0x7fb6f8120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 0x7fb6f8120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fcb38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 0x7fb6f8120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54178/0 (socket says 172.21.15.32:54178) 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fcb38700 1 -- 172.21.15.32:0/1678978940 learned_addr learned my addr 172.21.15.32:0/1678978940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 0x7fb6f8120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54898/0 (socket says 172.21.15.32:54898) 2023-09-30T22:25:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fd339700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f81210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 -- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 msgr2=0x7fb6f81210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f81210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 -- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 msgr2=0x7fb6f8120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 0x7fb6f8120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 -- 172.21.15.32:0/1678978940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0094a0 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fcb38700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 0x7fb6f8120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f7fff700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 0x7fb6f8120b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e9b0 tx=0x7fb6ec00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00ee50 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec00c850 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f0009b30 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00cc00 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.711+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f80a0460 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.712+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6ec013470 con 0x7fb6f80a0a30 2023-09-30T22:25:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.712+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8124990 con 0x7fb6f80a0a30 2023-09-30T22:25:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.713+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb6ec03fa70 con 0x7fb6f80a0a30 2023-09-30T22:25:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.717+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6ec013720 con 0x7fb6f80a0a30 2023-09-30T22:25:39.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.886+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb3b40a28e0 con 0x7fb3b40a1510 2023-09-30T22:25:39.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.887+0000 7fb39bfff700 1 -- 172.21.15.32:0/2753900231 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3a80239c0 con 0x7fb3b40a1510 2023-09-30T22:25:39.888 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:39.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 msgr2=0x7fb3b4122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4122e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8016000 tx=0x7fb3a8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:39.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 shutdown_connections 2023-09-30T22:25:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3b4139250 0x7fb3b4123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3b40a0a30 0x7fb3b41228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 --2- 172.21.15.32:0/2753900231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b40a1510 0x7fb3b4122e00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 >> 172.21.15.32:0/2753900231 conn(0x7fb3b40a6550 msgr2=0x7fb3b40a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 shutdown_connections 2023-09-30T22:25:39.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.889+0000 7fb3b9f9f700 1 -- 172.21.15.32:0/2753900231 wait complete. 2023-09-30T22:25:39.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:39.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.923+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb6f80747f0 con 0x7fb6f80a0a30 2023-09-30T22:25:39.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.923+0000 7fb6f57fa700 1 -- 172.21.15.32:0/1678978940 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6ec021070 con 0x7fb6f80a0a30 2023-09-30T22:25:39.925 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 msgr2=0x7fb6f8120b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 0x7fb6f8120b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e9b0 tx=0x7fb6ec00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 shutdown_connections 2023-09-30T22:25:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6f8138d80 0x7fb6f81210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f809d630 0x7fb6f8120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 --2- 172.21.15.32:0/1678978940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6f80a0a30 0x7fb6f8120b60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:39.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 >> 172.21.15.32:0/1678978940 conn(0x7fb6f80a3050 msgr2=0x7fb6f8139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:39.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 shutdown_connections 2023-09-30T22:25:39.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:39.926+0000 7fb6fed9c700 1 -- 172.21.15.32:0/1678978940 wait complete. 2023-09-30T22:25:39.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:40.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:40.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 Processor -- start 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 -- start start 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76400a0a30 0x7f76400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76400a1340 0x7f7640133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7640133fa0 0x7f7640134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764009edd0 con 0x7f76400a1340 2023-09-30T22:25:40.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764009e1f0 con 0x7f76400a0a30 2023-09-30T22:25:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f7646c1d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f764009dc50 con 0x7f7640133fa0 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f763ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76400a1340 0x7f7640133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f76449b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76400a0a30 0x7f76400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f763ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76400a1340 0x7f7640133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54182/0 (socket says 172.21.15.32:54182) 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f763ffff700 1 -- 172.21.15.32:0/3211164327 learned_addr learned my addr 172.21.15.32:0/3211164327 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.395+0000 7f76451ba700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7640133fa0 0x7f7640134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763ffff700 1 -- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7640133fa0 msgr2=0x7f7640134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763ffff700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7640133fa0 0x7f7640134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763ffff700 1 -- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76400a0a30 msgr2=0x7f76400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763ffff700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76400a0a30 0x7f76400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763ffff700 1 -- 172.21.15.32:0/3211164327 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400a1fe0 con 0x7f76400a1340 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f76449b9700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76400a0a30 0x7f76400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f76451ba700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7640133fa0 0x7f7640134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763ffff700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a1340 0x7f7640133a60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f76340098f0 tx=0x7f7634009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae34ad3d4fedcfc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763f7fe700 1 -- 172.21.15.32:0/3211164327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011830 con 0x7f76400a1340 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763f7fe700 1 -- 172.21.15.32:0/3211164327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7634011990 con 0x7f76400a1340 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.396+0000 7f763f7fe700 1 -- 172.21.15.32:0/3211164327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011d20 con 0x7f76400a1340 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 -- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a1340 msgr2=0x7f7640133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a1340 0x7f7640133a60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f76340098f0 tx=0x7f7634009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 -- 172.21.15.32:0/3211164327 shutdown_connections 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76400a0a30 0x7f76400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7640133fa0 0x7f7640134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 --2- 172.21.15.32:0/3211164327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a1340 0x7f7640133a60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 -- 172.21.15.32:0/3211164327 >> 172.21.15.32:0/3211164327 conn(0x7f76400a6550 msgr2=0x7f76400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 -- 172.21.15.32:0/3211164327 shutdown_connections 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.397+0000 7f7646c1d700 1 -- 172.21.15.32:0/3211164327 wait complete. 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.398+0000 7f7646c1d700 1 Processor -- start 2023-09-30T22:25:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.398+0000 7f7646c1d700 1 -- start start 2023-09-30T22:25:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f7646c1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a0a30 0x7f76401213a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f7646c1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 0x7f76401218e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f7646c1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7640133fa0 0x7f7640121e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f7646c1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400a0220 con 0x7f76400a0a30 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f7646c1d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76401237d0 con 0x7f7640133fa0 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f7646c1d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76400a36e0 con 0x7f76400a1340 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 0x7f76401218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f76449b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a0a30 0x7f76401213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 0x7f76401218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54916/0 (socket says 172.21.15.32:54916) 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 -- 172.21.15.32:0/2937712136 learned_addr learned my addr 172.21.15.32:0/2937712136 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f76451ba700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7640133fa0 0x7f7640121e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 -- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7640133fa0 msgr2=0x7f7640121e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7640133fa0 0x7f7640121e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 -- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a0a30 msgr2=0x7f76401213a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a0a30 0x7f76401213a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 -- 172.21.15.32:0/2937712136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300094a0 con 0x7f76400a1340 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f76449b9700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a0a30 0x7f76401213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f76451ba700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7640133fa0 0x7f7640121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.399+0000 7f763ffff700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 0x7f76401218e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7634009f50 tx=0x7f7634004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.400+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634004240 con 0x7f76400a1340 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.400+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763401f070 con 0x7f76400a1340 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.400+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7638009a70 con 0x7f76400a1340 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.400+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340045a0 con 0x7f76400a1340 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.400+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7634009550 con 0x7f76400a1340 2023-09-30T22:25:40.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.401+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7634021030 con 0x7f76400a1340 2023-09-30T22:25:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.401+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7634040780 con 0x7f76400a1340 2023-09-30T22:25:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.401+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7640074ea0 con 0x7f76400a1340 2023-09-30T22:25:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.406+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f763401a450 con 0x7f76400a1340 2023-09-30T22:25:40.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.433+0000 7fa90d512700 1 Processor -- start 2023-09-30T22:25:40.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.433+0000 7fa90d512700 1 -- start start 2023-09-30T22:25:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa90d512700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa90809d630 0x7fa9080a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa90d512700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa908138d80 0x7fa90813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa90d512700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9080a2a70 con 0x7fa9080a0a30 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa90d512700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080a01d0 con 0x7fa90809d630 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa90d512700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa90809e2b0 con 0x7fa908138d80 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa9077fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa908138d80 0x7fa90813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa906ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9080a0a30 0x7fa9080a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52086/0 (socket says 172.21.15.32:52086) 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa906ffd700 1 -- 172.21.15.32:0/4102790850 learned_addr learned my addr 172.21.15.32:0/4102790850 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.434+0000 7fa9067fc700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa90809d630 0x7fa9080a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa905ffb700 1 -- 172.21.15.32:0/4102790850 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614598626 0 0) 0x7fa9080a2a70 con 0x7fa9080a0a30 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa9067fc700 1 -- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa908138d80 msgr2=0x7fa90813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa9067fc700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa908138d80 0x7fa90813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa9067fc700 1 -- 172.21.15.32:0/4102790850 >> v1:172.21.15.32:6789/0 conn(0x7fa9080a0a30 legacy=0x7fa9080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa9067fc700 1 -- 172.21.15.32:0/4102790850 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90809dc50 con 0x7fa90809d630 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa9077fe700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa908138d80 0x7fa90813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa9067fc700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa90809d630 0x7fa9080a1510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009870 tx=0x7fa8fc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5516b9b739d6acf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa905ffb700 1 -- 172.21.15.32:0/4102790850 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011830 con 0x7fa90809d630 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa905ffb700 1 -- 172.21.15.32:0/4102790850 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc011990 con 0x7fa90809d630 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.435+0000 7fa905ffb700 1 -- 172.21.15.32:0/4102790850 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011d20 con 0x7fa90809d630 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 -- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa90809d630 msgr2=0x7fa9080a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa90809d630 0x7fa9080a1510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009870 tx=0x7fa8fc009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 -- 172.21.15.32:0/4102790850 shutdown_connections 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa908138d80 0x7fa90813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 --2- 172.21.15.32:0/4102790850 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa90809d630 0x7fa9080a1510 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 -- 172.21.15.32:0/4102790850 >> 172.21.15.32:0/4102790850 conn(0x7fa9080a3050 msgr2=0x7fa9080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 -- 172.21.15.32:0/4102790850 shutdown_connections 2023-09-30T22:25:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.436+0000 7fa90d512700 1 -- 172.21.15.32:0/4102790850 wait complete. 2023-09-30T22:25:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.437+0000 7fa90d512700 1 Processor -- start 2023-09-30T22:25:40.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.437+0000 7fa90d512700 1 -- start start 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa90d512700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa90809d630 0x7fa9081338c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa90d512700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 0x7fa908133e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa90d512700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 0x7fa908134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa90d512700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908137150 con 0x7fa9080a0a30 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa90d512700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa908137350 con 0x7fa908138d80 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa90d512700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa908138ad0 con 0x7fa90809d630 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa9077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 0x7fa908134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa9067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 0x7fa908133e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa9077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 0x7fa908134340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43172/0 (socket says 172.21.15.32:43172) 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa9067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 0x7fa908133e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34992/0 (socket says 172.21.15.32:34992) 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa906ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa90809d630 0x7fa9081338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.438+0000 7fa9077fe700 1 -- 172.21.15.32:0/3094993617 learned_addr learned my addr 172.21.15.32:0/3094993617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9077fe700 1 -- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa90809d630 msgr2=0x7fa9081338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9077fe700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa90809d630 0x7fa9081338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9077fe700 1 -- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 msgr2=0x7fa908133e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9077fe700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 0x7fa908133e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9077fe700 1 -- 172.21.15.32:0/3094993617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000094a0 con 0x7fa908138d80 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9067fc700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 0x7fa908133e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa9077fe700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 0x7fa908134340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa90000eb60 tx=0x7fa90000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:40.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900009e00 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900004020 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc0094a0 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000043d0 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.439+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908136240 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.440+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa90000c850 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.440+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa90003f410 con 0x7fa908138d80 2023-09-30T22:25:40.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.440+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9080747f0 con 0x7fa908138d80 2023-09-30T22:25:40.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.445+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa90000cb00 con 0x7fa908138d80 2023-09-30T22:25:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.615+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f76400a28e0 con 0x7f76400a1340 2023-09-30T22:25:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.615+0000 7f763d7fa700 1 -- 172.21.15.32:0/2937712136 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76340233d0 con 0x7f76400a1340 2023-09-30T22:25:40.616 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.617+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 msgr2=0x7f76401218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.617+0000 7f7646c1d700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 0x7f76401218e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7634009f50 tx=0x7f7634004750 comp rx=0 tx=0).stop 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.617+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 shutdown_connections 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.617+0000 7f7646c1d700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7640133fa0 0x7f7640121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.618+0000 7f7646c1d700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76400a0a30 0x7f76401213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.618+0000 7f7646c1d700 1 --2- 172.21.15.32:0/2937712136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76400a1340 0x7f76401218e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.618+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 >> 172.21.15.32:0/2937712136 conn(0x7f76400a6550 msgr2=0x7f76400a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.618+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 shutdown_connections 2023-09-30T22:25:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.618+0000 7f7646c1d700 1 -- 172.21.15.32:0/2937712136 wait complete. 2023-09-30T22:25:40.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.655+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa908137fd0 con 0x7fa908138d80 2023-09-30T22:25:40.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.655+0000 7fa8ef7fe700 1 -- 172.21.15.32:0/3094993617 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa900017020 con 0x7fa908138d80 2023-09-30T22:25:40.656 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 msgr2=0x7fa908134340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 0x7fa908134340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa90000eb60 tx=0x7fa90000ee70 comp rx=0 tx=0).stop 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 shutdown_connections 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa908138d80 0x7fa908134340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa90809d630 0x7fa9081338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 --2- 172.21.15.32:0/3094993617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080a0a30 0x7fa908133e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 >> 172.21.15.32:0/3094993617 conn(0x7fa9080a3050 msgr2=0x7fa90809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.657+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 shutdown_connections 2023-09-30T22:25:40.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:40.658+0000 7fa90d512700 1 -- 172.21.15.32:0/3094993617 wait complete. 2023-09-30T22:25:40.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:40.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:40.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 Processor -- start 2023-09-30T22:25:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 -- start start 2023-09-30T22:25:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 0x7f71c013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c00a2a70 con 0x7f71c009d630 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00a01d0 con 0x7f71c00a0a30 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c70da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71c009e2b0 con 0x7f71c0138d80 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c5677700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 0x7f71c013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.054+0000 7f71c4e76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 0x7f71c013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46704/0 (socket says 172.21.15.32:46704) 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 -- 172.21.15.32:0/4133858113 learned_addr learned my addr 172.21.15.32:0/4133858113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c4e76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43174/0 (socket says 172.21.15.32:43174) 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 -- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 msgr2=0x7f71c00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 --2- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 -- 172.21.15.32:0/4133858113 >> v1:172.21.15.32:6789/0 conn(0x7f71c009d630 legacy=0x7f71c00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71bf7fe700 1 -- 172.21.15.32:0/4133858113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153808805 0 0) 0x7f71c00a2a70 con 0x7f71c009d630 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 -- 172.21.15.32:0/4133858113 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c009dc50 con 0x7f71c0138d80 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c4e76700 1 --2- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71c5677700 1 --2- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 0x7f71c013b160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009920 tx=0x7f71b8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770d136a1ab89d00 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71bf7fe700 1 -- 172.21.15.32:0/4133858113 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8011830 con 0x7f71c0138d80 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71bf7fe700 1 -- 172.21.15.32:0/4133858113 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b8011990 con 0x7f71c0138d80 2023-09-30T22:25:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.055+0000 7f71bf7fe700 1 -- 172.21.15.32:0/4133858113 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8011d20 con 0x7f71c0138d80 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 -- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 msgr2=0x7f71c013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 --2- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 0x7f71c013b160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71b8009920 tx=0x7f71b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 -- 172.21.15.32:0/4133858113 shutdown_connections 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 --2- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c0138d80 0x7f71c013b160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 --2- 172.21.15.32:0/4133858113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 -- 172.21.15.32:0/4133858113 >> 172.21.15.32:0/4133858113 conn(0x7f71c00a3050 msgr2=0x7f71c00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 -- 172.21.15.32:0/4133858113 shutdown_connections 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.056+0000 7f71c70da700 1 -- 172.21.15.32:0/4133858113 wait complete. 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.057+0000 7f71c70da700 1 Processor -- start 2023-09-30T22:25:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.057+0000 7f71c70da700 1 -- start start 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c70da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 0x7f71c01205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c70da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c0120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c70da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 0x7f71c0121070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c70da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0122a90 con 0x7f71c0138d80 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c70da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01389f0 con 0x7f71c00a0a30 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c70da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00a0410 con 0x7f71c009d630 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c4e76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 0x7f71c01205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 0x7f71c0121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71bffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c0120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c4e76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 0x7f71c01205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46720/0 (socket says 172.21.15.32:46720) 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c4e76700 1 -- 172.21.15.32:0/2298362702 learned_addr learned my addr 172.21.15.32:0/2298362702 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 0x7f71c0121070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35004/0 (socket says 172.21.15.32:35004) 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 -- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 msgr2=0x7f71c01205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 0x7f71c01205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 -- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 msgr2=0x7f71c0120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c0120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c5677700 1 -- 172.21.15.32:0/2298362702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0009a70 con 0x7f71c0138d80 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.058+0000 7f71c4e76700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 0x7f71c01205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71bffff700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c0120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71c5677700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 0x7f71c0121070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71c00672b0 tx=0x7f71b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8004450 con 0x7f71c0138d80 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b8006ab0 con 0x7f71c0138d80 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80045f0 con 0x7f71c0138d80 2023-09-30T22:25:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b8009550 con 0x7f71c0138d80 2023-09-30T22:25:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.059+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c00a4920 con 0x7f71c0138d80 2023-09-30T22:25:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.060+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71b80038e0 con 0x7f71c0138d80 2023-09-30T22:25:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.060+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f71b8040620 con 0x7f71c0138d80 2023-09-30T22:25:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.060+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c00747f0 con 0x7f71c0138d80 2023-09-30T22:25:41.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.065+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71b801a420 con 0x7f71c0138d80 2023-09-30T22:25:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.091+0000 7f92ab02a700 1 Processor -- start 2023-09-30T22:25:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.091+0000 7f92ab02a700 1 -- start start 2023-09-30T22:25:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92ab02a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a409d630 0x7f92a40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92ab02a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a4138d80 0x7f92a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92ab02a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40a2a70 con 0x7f92a40a0a30 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92ab02a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40a01d0 con 0x7f92a409d630 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92ab02a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92a409e2b0 con 0x7f92a4138d80 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a8dc6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92a40a0a30 0x7f92a40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52098/0 (socket says 172.21.15.32:52098) 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a8dc6700 1 -- 172.21.15.32:0/846248667 learned_addr learned my addr 172.21.15.32:0/846248667 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a95c7700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a4138d80 0x7f92a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a3fff700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a409d630 0x7f92a40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a37fe700 1 -- 172.21.15.32:0/846248667 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332152958 0 0) 0x7f92a40a2a70 con 0x7f92a40a0a30 2023-09-30T22:25:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a37fe700 1 -- 172.21.15.32:0/846248667 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f928c0035b0 con 0x7f92a40a0a30 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a95c7700 1 -- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a409d630 msgr2=0x7f92a40a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a95c7700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a409d630 0x7f92a40a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a95c7700 1 -- 172.21.15.32:0/846248667 >> v1:172.21.15.32:6789/0 conn(0x7f92a40a0a30 legacy=0x7f92a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a95c7700 1 -- 172.21.15.32:0/846248667 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a409dc50 con 0x7f92a4138d80 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a3fff700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a409d630 0x7f92a40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.092+0000 7f92a95c7700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a4138d80 0x7f92a413b160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f929c009dc0 tx=0x7f929c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ff02958e0bf93a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92a37fe700 1 -- 172.21.15.32:0/846248667 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00e070 con 0x7f92a4138d80 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92a37fe700 1 -- 172.21.15.32:0/846248667 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00bb20 con 0x7f92a4138d80 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92a37fe700 1 -- 172.21.15.32:0/846248667 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c012430 con 0x7f92a4138d80 2023-09-30T22:25:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 -- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a4138d80 msgr2=0x7f92a413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a4138d80 0x7f92a413b160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f929c009dc0 tx=0x7f929c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 -- 172.21.15.32:0/846248667 shutdown_connections 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a4138d80 0x7f92a413b160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 --2- 172.21.15.32:0/846248667 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a409d630 0x7f92a40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 -- 172.21.15.32:0/846248667 >> 172.21.15.32:0/846248667 conn(0x7f92a40a3050 msgr2=0x7f92a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 -- 172.21.15.32:0/846248667 shutdown_connections 2023-09-30T22:25:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.093+0000 7f92ab02a700 1 -- 172.21.15.32:0/846248667 wait complete. 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.094+0000 7f92ab02a700 1 Processor -- start 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.094+0000 7f92ab02a700 1 -- start start 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92ab02a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 0x7f92a41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92ab02a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a40a0a30 0x7f92a4133d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92ab02a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a4138d80 0x7f92a4134270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92ab02a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4134cc0 con 0x7f92a409d630 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92ab02a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41361e0 con 0x7f92a4138d80 2023-09-30T22:25:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92ab02a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4136590 con 0x7f92a40a0a30 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92a8dc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 0x7f92a41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92a95c7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a4138d80 0x7f92a4134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92a8dc6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 0x7f92a41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35008/0 (socket says 172.21.15.32:35008) 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92a8dc6700 1 -- 172.21.15.32:0/2377646381 learned_addr learned my addr 172.21.15.32:0/2377646381 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.095+0000 7f92a3fff700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a40a0a30 0x7f92a4133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a8dc6700 1 -- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a40a0a30 msgr2=0x7f92a4133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a8dc6700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a40a0a30 0x7f92a4133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a8dc6700 1 -- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a4138d80 msgr2=0x7f92a4134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a8dc6700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a4138d80 0x7f92a4134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a8dc6700 1 -- 172.21.15.32:0/2377646381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980094a0 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a95c7700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a4138d80 0x7f92a4134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a8dc6700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 0x7f92a41337f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f929400c700 tx=0x7f929400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294005d70 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92940070d0 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294007460 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c009a70 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.096+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a4136f50 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.097+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92940075c0 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.097+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f929403e9c0 con 0x7f92a409d630 2023-09-30T22:25:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.097+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a4136f50 con 0x7f92a409d630 2023-09-30T22:25:41.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.102+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92a4136f50 con 0x7f92a409d630 2023-09-30T22:25:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.276+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71c0123410 con 0x7f71c0138d80 2023-09-30T22:25:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.276+0000 7f71bd7fa700 1 -- 172.21.15.32:0/2298362702 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71b8015070 con 0x7f71c0138d80 2023-09-30T22:25:41.278 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 msgr2=0x7f71c0121070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 0x7f71c0121070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71c00672b0 tx=0x7f71b8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 shutdown_connections 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71c0138d80 0x7f71c0121070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71c009d630 0x7f71c01205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 --2- 172.21.15.32:0/2298362702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71c00a0a30 0x7f71c0120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 >> 172.21.15.32:0/2298362702 conn(0x7f71c00a3050 msgr2=0x7f71c009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 shutdown_connections 2023-09-30T22:25:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.279+0000 7f71c70da700 1 -- 172.21.15.32:0/2298362702 wait complete. 2023-09-30T22:25:41.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.311+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f92a4136f50 con 0x7f92a409d630 2023-09-30T22:25:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.312+0000 7f92a17fa700 1 -- 172.21.15.32:0/2377646381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92a4136f50 con 0x7f92a409d630 2023-09-30T22:25:41.313 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 msgr2=0x7f92a41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 0x7f92a41337f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f929400c700 tx=0x7f929400ca10 comp rx=0 tx=0).stop 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 shutdown_connections 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92a4138d80 0x7f92a4134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a409d630 0x7f92a41337f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 --2- 172.21.15.32:0/2377646381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92a40a0a30 0x7f92a4133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 >> 172.21.15.32:0/2377646381 conn(0x7f92a40a3050 msgr2=0x7f92a4139700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 shutdown_connections 2023-09-30T22:25:41.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.314+0000 7f92ab02a700 1 -- 172.21.15.32:0/2377646381 wait complete. 2023-09-30T22:25:41.328 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:41.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:41.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.781+0000 7f9909b36700 1 Processor -- start 2023-09-30T22:25:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 -- start start 2023-09-30T22:25:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f990409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f990409f240 0x7f99041317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f9904131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9904070940 con 0x7f990409f240 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990406fd60 con 0x7f990409e930 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.782+0000 7f9909b36700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f990406fa80 con 0x7f990409f650 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f990409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f9903fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f9904131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f9902ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f990409f240 0x7f99041317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f990409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43196/0 (socket says 172.21.15.32:43196) 2023-09-30T22:25:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f9903fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f9904131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46742/0 (socket says 172.21.15.32:46742) 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 -- 172.21.15.32:0/2254060472 learned_addr learned my addr 172.21.15.32:0/2254060472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 -- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 msgr2=0x7f9904131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 --2- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f9904131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 -- 172.21.15.32:0/2254060472 >> v2:172.21.15.32:3300/0 conn(0x7f990409f240 msgr2=0x7f99041317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 --2- 172.21.15.32:0/2254060472 >> v2:172.21.15.32:3300/0 conn(0x7f990409f240 0x7f99041317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 -- 172.21.15.32:0/2254060472 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990409fb60 con 0x7f990409e930 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f9902ffd700 1 --2- 172.21.15.32:0/2254060472 >> v2:172.21.15.32:3300/0 conn(0x7f990409f240 0x7f99041317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99037fe700 1 --2- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f990409ed00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009e50 tx=0x7f98f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f551b2b6b8151bac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99027fc700 1 -- 172.21.15.32:0/2254060472 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400e070 con 0x7f990409e930 2023-09-30T22:25:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.783+0000 7f99027fc700 1 -- 172.21.15.32:0/2254060472 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400b910 con 0x7f990409e930 2023-09-30T22:25:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.784+0000 7f99027fc700 1 -- 172.21.15.32:0/2254060472 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400bcc0 con 0x7f990409e930 2023-09-30T22:25:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.784+0000 7f9909b36700 1 -- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 msgr2=0x7f990409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.784+0000 7f9909b36700 1 --2- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f990409ed00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009e50 tx=0x7f98f400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 -- 172.21.15.32:0/2254060472 shutdown_connections 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 --2- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f990409ed00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 --2- 172.21.15.32:0/2254060472 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f9904131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 --2- 172.21.15.32:0/2254060472 >> v2:172.21.15.32:3300/0 conn(0x7f990409f240 0x7f99041317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 -- 172.21.15.32:0/2254060472 >> 172.21.15.32:0/2254060472 conn(0x7f99040a4210 msgr2=0x7f99040a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 -- 172.21.15.32:0/2254060472 shutdown_connections 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.785+0000 7f9909b36700 1 -- 172.21.15.32:0/2254060472 wait complete. 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 Processor -- start 2023-09-30T22:25:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 -- start start 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f99041a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409f240 0x7f99041a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f99041a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99041a58a0 con 0x7f990409f240 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99041a5230 con 0x7f990409e930 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.786+0000 7f9909b36700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99041a7310 con 0x7f990409f650 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9903fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f99041a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f99041a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9903fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f99041a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46752/0 (socket says 172.21.15.32:46752) 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9903fff700 1 -- 172.21.15.32:0/3466412625 learned_addr learned my addr 172.21.15.32:0/3466412625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f99041a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43200/0 (socket says 172.21.15.32:43200) 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9902ffd700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409f240 0x7f99041a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 -- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 msgr2=0x7f99041a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f99041a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 -- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409f240 msgr2=0x7f99041a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409f240 0x7f99041a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 -- 172.21.15.32:0/3466412625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec0094a0 con 0x7f990409e930 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9903fff700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f99041a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f99037fe700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f99041a3d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98f4015000 tx=0x7f98f4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40092e0 con 0x7f990409e930 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400cea0 con 0x7f990409e930 2023-09-30T22:25:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4004490 con 0x7f990409e930 2023-09-30T22:25:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f4009b30 con 0x7f990409e930 2023-09-30T22:25:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.787+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99040a0190 con 0x7f990409e930 2023-09-30T22:25:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.788+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98f40045f0 con 0x7f990409e930 2023-09-30T22:25:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.788+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f98f40408b0 con 0x7f990409e930 2023-09-30T22:25:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.788+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9904071d80 con 0x7f990409e930 2023-09-30T22:25:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.793+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98f4011ba0 con 0x7f990409e930 2023-09-30T22:25:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 Processor -- start 2023-09-30T22:25:41.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 -- start start 2023-09-30T22:25:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a0a30 0x7f4ce40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a1340 0x7f4ce4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ce4133fa0 0x7f4ce4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce409edd0 con 0x7f4ce4133fa0 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce409e1f0 con 0x7f4ce40a0a30 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.820+0000 7f4cea9f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce409dc50 con 0x7f4ce40a1340 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a0a30 0x7f4ce40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a1340 0x7f4ce4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce8f93700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ce4133fa0 0x7f4ce4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a0a30 0x7f4ce40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43212/0 (socket says 172.21.15.32:43212) 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce3fff700 1 -- 172.21.15.32:0/4176777705 learned_addr learned my addr 172.21.15.32:0/4176777705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 -- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a0a30 msgr2=0x7f4ce40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:41.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a0a30 0x7f4ce40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 -- 172.21.15.32:0/4176777705 >> v2:172.21.15.32:3300/0 conn(0x7f4ce4133fa0 msgr2=0x7f4ce4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/4176777705 >> v2:172.21.15.32:3300/0 conn(0x7f4ce4133fa0 0x7f4ce4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 -- 172.21.15.32:0/4176777705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40a1fe0 con 0x7f4ce40a1340 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a1340 0x7f4ce4133a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009840 tx=0x7f4cd8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61d545f86a9f9b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce2ffd700 1 -- 172.21.15.32:0/4176777705 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8011830 con 0x7f4ce40a1340 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce2ffd700 1 -- 172.21.15.32:0/4176777705 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd8011990 con 0x7f4ce40a1340 2023-09-30T22:25:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.821+0000 7f4ce2ffd700 1 -- 172.21.15.32:0/4176777705 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8011d20 con 0x7f4ce40a1340 2023-09-30T22:25:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 -- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a1340 msgr2=0x7f4ce4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a1340 0x7f4ce4133a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009840 tx=0x7f4cd8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 -- 172.21.15.32:0/4176777705 shutdown_connections 2023-09-30T22:25:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a0a30 0x7f4ce40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/4176777705 >> v2:172.21.15.32:3300/0 conn(0x7f4ce4133fa0 0x7f4ce4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/4176777705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a1340 0x7f4ce4133a60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 -- 172.21.15.32:0/4176777705 >> 172.21.15.32:0/4176777705 conn(0x7f4ce40a6550 msgr2=0x7f4ce40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 -- 172.21.15.32:0/4176777705 shutdown_connections 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.822+0000 7f4cea9f6700 1 -- 172.21.15.32:0/4176777705 wait complete. 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.823+0000 7f4cea9f6700 1 Processor -- start 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.823+0000 7f4cea9f6700 1 -- start start 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4cea9f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 0x7f4ce41200c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4cea9f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a1340 0x7f4ce4120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4cea9f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce4133fa0 0x7f4ce4120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4cea9f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce4123840 con 0x7f4ce4133fa0 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4cea9f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40a3610 con 0x7f4ce40a1340 2023-09-30T22:25:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4cea9f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce4121d20 con 0x7f4ce40a0a30 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 0x7f4ce41200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 0x7f4ce41200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46778/0 (socket says 172.21.15.32:46778) 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce3fff700 1 -- 172.21.15.32:0/3079215149 learned_addr learned my addr 172.21.15.32:0/3079215149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a1340 0x7f4ce4120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce8f93700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce4133fa0 0x7f4ce4120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 msgr2=0x7f4ce41200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 0x7f4ce41200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce4133fa0 msgr2=0x7f4ce4120b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce4133fa0 0x7f4ce4120b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce37fe700 1 -- 172.21.15.32:0/3079215149 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd80094a0 con 0x7f4ce40a1340 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.824+0000 7f4ce3fff700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 0x7f4ce41200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.825+0000 7f4ce37fe700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a1340 0x7f4ce4120600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ce409d2f0 tx=0x7f4cd8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.825+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8004450 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.825+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd801f070 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.825+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce40a0220 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.825+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80038e0 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.825+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce4123b20 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.826+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4cd8003a40 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.826+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce41233b0 con 0x7f4ce40a1340 2023-09-30T22:25:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.826+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4cd8021070 con 0x7f4ce40a1340 2023-09-30T22:25:41.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.831+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4cd80045b0 con 0x7f4ce40a1340 2023-09-30T22:25:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.999+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f99041a0b30 con 0x7f990409e930 2023-09-30T22:25:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:41.999+0000 7f98fbfff700 1 -- 172.21.15.32:0/3466412625 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98f4024e60 con 0x7f990409e930 2023-09-30T22:25:42.000 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 msgr2=0x7f99041a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f99041a3d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98f4015000 tx=0x7f98f4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 shutdown_connections 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f990409f650 0x7f99041a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409e930 0x7f99041a3d80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 --2- 172.21.15.32:0/3466412625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409f240 0x7f99041a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 >> 172.21.15.32:0/3466412625 conn(0x7f99040a4210 msgr2=0x7f99041b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 shutdown_connections 2023-09-30T22:25:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.002+0000 7f9909b36700 1 -- 172.21.15.32:0/3466412625 wait complete. 2023-09-30T22:25:42.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.034+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4ce4074ea0 con 0x7f4ce40a1340 2023-09-30T22:25:42.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.034+0000 7f4ce0ff9700 1 -- 172.21.15.32:0/3079215149 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4cd8015020 con 0x7f4ce40a1340 2023-09-30T22:25:42.035 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a1340 msgr2=0x7f4ce4120600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a1340 0x7f4ce4120600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ce409d2f0 tx=0x7f4cd8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 shutdown_connections 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ce4133fa0 0x7f4ce4120b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4ce40a0a30 0x7f4ce41200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 --2- 172.21.15.32:0/3079215149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ce40a1340 0x7f4ce4120600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.036+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 >> 172.21.15.32:0/3079215149 conn(0x7f4ce40a6550 msgr2=0x7f4ce40a4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.037+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 shutdown_connections 2023-09-30T22:25:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.037+0000 7f4cea9f6700 1 -- 172.21.15.32:0/3079215149 wait complete. 2023-09-30T22:25:42.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:42.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:42.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.442+0000 7f0767cce700 1 Processor -- start 2023-09-30T22:25:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.442+0000 7f0767cce700 1 -- start start 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0767cce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 0x7f07600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0767cce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 0x7f07600a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0767cce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f0760133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0767cce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07600a2a70 con 0x7f07600a0a30 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0767cce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600a01d0 con 0x7f0760133930 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0767cce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f076009e2b0 con 0x7f076009d630 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0765269700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 0x7f07600a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0765a6a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 0x7f07600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f076626b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f0760133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0765a6a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 0x7f07600a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35036/0 (socket says 172.21.15.32:35036) 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0765269700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 0x7f07600a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46786/0 (socket says 172.21.15.32:46786) 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.443+0000 7f0765a6a700 1 -- 172.21.15.32:0/1609626518 learned_addr learned my addr 172.21.15.32:0/1609626518 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765269700 1 -- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 msgr2=0x7f0760133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765269700 1 --2- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f0760133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765269700 1 -- 172.21.15.32:0/1609626518 >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 msgr2=0x7f07600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765269700 1 --2- 172.21.15.32:0/1609626518 >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 0x7f07600a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765269700 1 -- 172.21.15.32:0/1609626518 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076009dc50 con 0x7f076009d630 2023-09-30T22:25:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765a6a700 1 --2- 172.21.15.32:0/1609626518 >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 0x7f07600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f076626b700 1 --2- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f0760133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0765269700 1 --2- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 0x7f07600a1340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0754009840 tx=0x7f0754009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db672221b24ab55f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0764a68700 1 -- 172.21.15.32:0/1609626518 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754011830 con 0x7f076009d630 2023-09-30T22:25:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0764a68700 1 -- 172.21.15.32:0/1609626518 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0754011990 con 0x7f076009d630 2023-09-30T22:25:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.444+0000 7f0764a68700 1 -- 172.21.15.32:0/1609626518 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754011d20 con 0x7f076009d630 2023-09-30T22:25:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 -- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 msgr2=0x7f07600a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 --2- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 0x7f07600a1340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0754009840 tx=0x7f0754009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 -- 172.21.15.32:0/1609626518 shutdown_connections 2023-09-30T22:25:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 --2- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f0760133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 --2- 172.21.15.32:0/1609626518 >> v2:172.21.15.32:3300/0 conn(0x7f07600a0a30 0x7f07600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 --2- 172.21.15.32:0/1609626518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f076009d630 0x7f07600a1340 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 -- 172.21.15.32:0/1609626518 >> 172.21.15.32:0/1609626518 conn(0x7f07600a3050 msgr2=0x7f07600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 -- 172.21.15.32:0/1609626518 shutdown_connections 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.445+0000 7f0767cce700 1 -- 172.21.15.32:0/1609626518 wait complete. 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.447+0000 7f0767cce700 1 Processor -- start 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.447+0000 7f0767cce700 1 -- start start 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.447+0000 7f0767cce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076009d630 0x7f07601c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.447+0000 7f0767cce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 0x7f07601c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0767cce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f07601c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0767cce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601207f0 con 0x7f076009d630 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0767cce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0760120600 con 0x7f0760133930 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0767cce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07600a4150 con 0x7f07600a0a30 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 0x7f07601c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f076626b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f07601c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 0x7f07601c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46792/0 (socket says 172.21.15.32:46792) 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 -- 172.21.15.32:0/410182754 learned_addr learned my addr 172.21.15.32:0/410182754 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f076626b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f07601c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43242/0 (socket says 172.21.15.32:43242) 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765a6a700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076009d630 0x7f07601c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 -- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 msgr2=0x7f07601c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f07601c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 -- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076009d630 msgr2=0x7f07601c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076009d630 0x7f07601c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 -- 172.21.15.32:0/410182754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0750009b30 con 0x7f07600a0a30 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f076626b700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f07601c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765a6a700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076009d630 0x7f07601c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0765269700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 0x7f07601c8560 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07540090b0 tx=0x7f0754004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754004240 con 0x7f07600a0a30 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075401f070 con 0x7f07600a0a30 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.448+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07580094a0 con 0x7f07600a0a30 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.449+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540036a0 con 0x7f07600a0a30 2023-09-30T22:25:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.449+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07540094a0 con 0x7f07600a0a30 2023-09-30T22:25:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.449+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0754021030 con 0x7f07600a0a30 2023-09-30T22:25:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.449+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f07540405a0 con 0x7f07600a0a30 2023-09-30T22:25:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.450+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07600747f0 con 0x7f07600a0a30 2023-09-30T22:25:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.455+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07540239c0 con 0x7f07600a0a30 2023-09-30T22:25:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.481+0000 7f04c879b700 1 Processor -- start 2023-09-30T22:25:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.481+0000 7f04c879b700 1 -- start start 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c879b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04c0133420 0x7f04c01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c879b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c879b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c006b990 con 0x7f04c0134260 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c879b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c00a0590 con 0x7f04c0133420 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c879b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c009fb80 con 0x7f04c0133e30 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c5d36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c6537700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04c0133420 0x7f04c01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c5d36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43246/0 (socket says 172.21.15.32:43246) 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c5d36700 1 -- 172.21.15.32:0/2461117598 learned_addr learned my addr 172.21.15.32:0/2461117598 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.482+0000 7f04c6d38700 1 --1- 172.21.15.32:0/2461117598 >> v1:172.21.15.32:6789/0 conn(0x7f04c0134260 0x7f04c01304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52110/0 (socket says 172.21.15.32:52110) 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c6537700 1 -- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 msgr2=0x7f04c012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c6537700 1 --2- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c6537700 1 -- 172.21.15.32:0/2461117598 >> v1:172.21.15.32:6789/0 conn(0x7f04c0134260 legacy=0x7f04c01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c6537700 1 -- 172.21.15.32:0/2461117598 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00a0030 con 0x7f04c0133420 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c5d36700 1 --2- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c6537700 1 --2- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c01337f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009840 tx=0x7f04b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7f08ecbe4d4e2c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c5535700 1 -- 172.21.15.32:0/2461117598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8011830 con 0x7f04c0133420 2023-09-30T22:25:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c5535700 1 -- 172.21.15.32:0/2461117598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8011990 con 0x7f04c0133420 2023-09-30T22:25:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.483+0000 7f04c5535700 1 -- 172.21.15.32:0/2461117598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8011d20 con 0x7f04c0133420 2023-09-30T22:25:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 -- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 msgr2=0x7f04c01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 --2- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c01337f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009840 tx=0x7f04b8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 -- 172.21.15.32:0/2461117598 shutdown_connections 2023-09-30T22:25:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 --2- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c01337f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 --2- 172.21.15.32:0/2461117598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 -- 172.21.15.32:0/2461117598 >> 172.21.15.32:0/2461117598 conn(0x7f04c00a4210 msgr2=0x7f04c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 -- 172.21.15.32:0/2461117598 shutdown_connections 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.484+0000 7f04c879b700 1 -- 172.21.15.32:0/2461117598 wait complete. 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 Processor -- start 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 -- start start 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c009c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c009c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04c0134260 0x7f04c009cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0120010 con 0x7f04c0133420 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c01201f0 con 0x7f04c0133e30 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c879b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04c012da70 con 0x7f04c0134260 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c5d36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c009c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c5d36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c009c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43248/0 (socket says 172.21.15.32:43248) 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c5d36700 1 -- 172.21.15.32:0/708326654 learned_addr learned my addr 172.21.15.32:0/708326654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.485+0000 7f04c6d38700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04c0134260 0x7f04c009cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c6537700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c009c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c5d36700 1 -- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04c0134260 msgr2=0x7f04c009cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c5d36700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04c0134260 0x7f04c009cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c5d36700 1 -- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 msgr2=0x7f04c009c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c5d36700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c009c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c5d36700 1 -- 172.21.15.32:0/708326654 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4009550 con 0x7f04c0133e30 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c5d36700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c009c740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04b400ec40 tx=0x7f04b400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400c850 con 0x7f04c0133e30 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b400c9b0 con 0x7f04c0133e30 2023-09-30T22:25:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b80094a0 con 0x7f04c0133e30 2023-09-30T22:25:42.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400cd20 con 0x7f04c0133e30 2023-09-30T22:25:42.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.486+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c012d620 con 0x7f04c0133e30 2023-09-30T22:25:42.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.487+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f04b4013470 con 0x7f04c0133e30 2023-09-30T22:25:42.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.487+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c0074ea0 con 0x7f04c0133e30 2023-09-30T22:25:42.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.487+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f04b403f7b0 con 0x7f04c0133e30 2023-09-30T22:25:42.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.492+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f04b4013720 con 0x7f04c0133e30 2023-09-30T22:25:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.661+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0760120df0 con 0x7f07600a0a30 2023-09-30T22:25:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.662+0000 7f075e7fc700 1 -- 172.21.15.32:0/410182754 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0754015020 con 0x7f07600a0a30 2023-09-30T22:25:42.663 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 msgr2=0x7f07601c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 0x7f07601c8560 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07540090b0 tx=0x7f0754004750 comp rx=0 tx=0).stop 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 shutdown_connections 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0760133930 0x7f07601c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076009d630 0x7f07601c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 --2- 172.21.15.32:0/410182754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f07600a0a30 0x7f07601c8560 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 >> 172.21.15.32:0/410182754 conn(0x7f07600a3050 msgr2=0x7f07601bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 shutdown_connections 2023-09-30T22:25:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.664+0000 7f0767cce700 1 -- 172.21.15.32:0/410182754 wait complete. 2023-09-30T22:25:42.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.699+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f04c011fe20 con 0x7f04c0133e30 2023-09-30T22:25:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.699+0000 7f04beffd700 1 -- 172.21.15.32:0/708326654 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04b401bba0 con 0x7f04c0133e30 2023-09-30T22:25:42.700 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 msgr2=0x7f04c009c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c009c740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04b400ec40 tx=0x7f04b400ef50 comp rx=0 tx=0).stop 2023-09-30T22:25:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 shutdown_connections 2023-09-30T22:25:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04c0134260 0x7f04c009cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04c0133420 0x7f04c009c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 --2- 172.21.15.32:0/708326654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04c0133e30 0x7f04c009c740 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:42.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 >> 172.21.15.32:0/708326654 conn(0x7f04c00a4210 msgr2=0x7f04c012f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:42.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 shutdown_connections 2023-09-30T22:25:42.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:42.702+0000 7f04c879b700 1 -- 172.21.15.32:0/708326654 wait complete. 2023-09-30T22:25:42.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:43.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:43.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 Processor -- start 2023-09-30T22:25:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 -- start start 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 0x7f9ebc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ebc138d80 0x7f9ebc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc0a2a70 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc0a01d0 con 0x7f9ebc138d80 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ec1705700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc09e2b0 con 0x7f9ebc09d630 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9eba7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 0x7f9ebc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9ebb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ebc138d80 0x7f9ebc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.170+0000 7f9eba7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 0x7f9ebc0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43258/0 (socket says 172.21.15.32:43258) 2023-09-30T22:25:43.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eba7fc700 1 -- 172.21.15.32:0/3475473598 learned_addr learned my addr 172.21.15.32:0/3475473598 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9ebaffd700 1 --1- 172.21.15.32:0/3475473598 >> v1:172.21.15.32:6789/0 conn(0x7f9ebc0a0a30 0x7f9ebc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52120/0 (socket says 172.21.15.32:52120) 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eba7fc700 1 -- 172.21.15.32:0/3475473598 >> v2:172.21.15.32:3300/0 conn(0x7f9ebc138d80 msgr2=0x7f9ebc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eba7fc700 1 --2- 172.21.15.32:0/3475473598 >> v2:172.21.15.32:3300/0 conn(0x7f9ebc138d80 0x7f9ebc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eba7fc700 1 -- 172.21.15.32:0/3475473598 >> v1:172.21.15.32:6789/0 conn(0x7f9ebc0a0a30 legacy=0x7f9ebc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eba7fc700 1 -- 172.21.15.32:0/3475473598 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc09dc50 con 0x7f9ebc09d630 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9ebb7fe700 1 --2- 172.21.15.32:0/3475473598 >> v2:172.21.15.32:3300/0 conn(0x7f9ebc138d80 0x7f9ebc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eba7fc700 1 --2- 172.21.15.32:0/3475473598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 0x7f9ebc0a1510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00098f0 tx=0x7f9eb0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b218146b4d6bad72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eb9ffb700 1 -- 172.21.15.32:0/3475473598 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0011830 con 0x7f9ebc09d630 2023-09-30T22:25:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eb9ffb700 1 -- 172.21.15.32:0/3475473598 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb0011990 con 0x7f9ebc09d630 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.171+0000 7f9eb9ffb700 1 -- 172.21.15.32:0/3475473598 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0011d20 con 0x7f9ebc09d630 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 -- 172.21.15.32:0/3475473598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 msgr2=0x7f9ebc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 --2- 172.21.15.32:0/3475473598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 0x7f9ebc0a1510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00098f0 tx=0x7f9eb0009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 -- 172.21.15.32:0/3475473598 shutdown_connections 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 --2- 172.21.15.32:0/3475473598 >> v2:172.21.15.32:3300/0 conn(0x7f9ebc138d80 0x7f9ebc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 --2- 172.21.15.32:0/3475473598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc09d630 0x7f9ebc0a1510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 -- 172.21.15.32:0/3475473598 >> 172.21.15.32:0/3475473598 conn(0x7f9ebc0a3050 msgr2=0x7f9ebc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 -- 172.21.15.32:0/3475473598 shutdown_connections 2023-09-30T22:25:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.172+0000 7f9ec1705700 1 -- 172.21.15.32:0/3475473598 wait complete. 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.173+0000 7f9ec1705700 1 Processor -- start 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.173+0000 7f9ec1705700 1 -- start start 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ec1705700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ebc09d630 0x7f9ebc133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ec1705700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 0x7f9ebc134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ec1705700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc138d80 0x7f9ebc1345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ec1705700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc135010 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ec1705700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc13afc0 con 0x7f9ebc138d80 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ec1705700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc135c30 con 0x7f9ebc09d630 2023-09-30T22:25:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 0x7f9ebc134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ebb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc138d80 0x7f9ebc1345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 0x7f9ebc134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35092/0 (socket says 172.21.15.32:35092) 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 -- 172.21.15.32:0/1090672872 learned_addr learned my addr 172.21.15.32:0/1090672872 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ebaffd700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ebc09d630 0x7f9ebc133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 -- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ebc09d630 msgr2=0x7f9ebc133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ebc09d630 0x7f9ebc133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 -- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc138d80 msgr2=0x7f9ebc1345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc138d80 0x7f9ebc1345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 -- 172.21.15.32:0/1090672872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40094a0 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9ebb7fe700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc138d80 0x7f9ebc1345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.174+0000 7f9eba7fc700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 0x7f9ebc134090 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009c40 tx=0x7f9eb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.175+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00045d0 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.175+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb00036a0 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.175+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb0009550 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.175+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0003a10 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.175+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc137590 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.175+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9eb0021070 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.176+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9eb0040a50 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.176+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc0747f0 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.181+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9eb0015080 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 Processor -- start 2023-09-30T22:25:43.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 -- start start 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bac0a0a30 0x7f0bac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 0x7f0bac13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac09edd0 con 0x7f0bac0a1340 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac09e1f0 con 0x7f0bac0a0a30 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb311d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac09dc50 con 0x7f0bac134170 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb16ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 0x7f0bac13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.209+0000 7f0bb16ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 0x7f0bac13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46810/0 (socket says 172.21.15.32:46810) 2023-09-30T22:25:43.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb16ba700 1 -- 172.21.15.32:0/2094486610 learned_addr learned my addr 172.21.15.32:0/2094486610 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb0eb9700 1 --2- 172.21.15.32:0/2094486610 >> v2:172.21.15.32:3300/0 conn(0x7f0bac0a0a30 0x7f0bac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bab7fe700 1 -- 172.21.15.32:0/2094486610 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090298602 0 0) 0x7f0bac09edd0 con 0x7f0bac0a1340 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bab7fe700 1 -- 172.21.15.32:0/2094486610 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b940035b0 con 0x7f0bac0a1340 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb0eb9700 1 -- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 msgr2=0x7f0bac13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb0eb9700 1 --2- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 0x7f0bac13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb0eb9700 1 -- 172.21.15.32:0/2094486610 >> v1:172.21.15.32:6789/0 conn(0x7f0bac0a1340 legacy=0x7f0bac133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb0eb9700 1 -- 172.21.15.32:0/2094486610 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac0a1fe0 con 0x7f0bac0a0a30 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb16ba700 1 --2- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 0x7f0bac13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bb0eb9700 1 --2- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac0a0a30 0x7f0bac0a0e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6267fe66a93faa50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bab7fe700 1 -- 172.21.15.32:0/2094486610 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400e070 con 0x7f0bac0a0a30 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bab7fe700 1 -- 172.21.15.32:0/2094486610 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba400bb20 con 0x7f0bac0a0a30 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.210+0000 7f0bab7fe700 1 -- 172.21.15.32:0/2094486610 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4012430 con 0x7f0bac0a0a30 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 -- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac0a0a30 msgr2=0x7f0bac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 --2- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac0a0a30 0x7f0bac0a0e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 -- 172.21.15.32:0/2094486610 shutdown_connections 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 --2- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac134170 0x7f0bac13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 --2- 172.21.15.32:0/2094486610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac0a0a30 0x7f0bac0a0e00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 -- 172.21.15.32:0/2094486610 >> 172.21.15.32:0/2094486610 conn(0x7f0bac0a6550 msgr2=0x7f0bac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:43.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 -- 172.21.15.32:0/2094486610 shutdown_connections 2023-09-30T22:25:43.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.211+0000 7f0bb311d700 1 -- 172.21.15.32:0/2094486610 wait complete. 2023-09-30T22:25:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.212+0000 7f0bb311d700 1 Processor -- start 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.212+0000 7f0bb311d700 1 -- start start 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb311d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bac0a0a30 0x7f0bac122740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb311d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 0x7f0bac122c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb311d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac134170 0x7f0bac1231c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb311d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac124b50 con 0x7f0bac134170 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb311d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1242c0 con 0x7f0bac0a0a30 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb311d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac09bd20 con 0x7f0bac0a1340 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 0x7f0bac122c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb0eb9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bac0a0a30 0x7f0bac122740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 0x7f0bac122c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46822/0 (socket says 172.21.15.32:46822) 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb16ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac134170 0x7f0bac1231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 -- 172.21.15.32:0/437567824 learned_addr learned my addr 172.21.15.32:0/437567824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb0eb9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bac0a0a30 0x7f0bac122740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43264/0 (socket says 172.21.15.32:43264) 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 -- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bac0a0a30 msgr2=0x7f0bac122740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bac0a0a30 0x7f0bac122740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 -- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac134170 msgr2=0x7f0bac1231c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac134170 0x7f0bac1231c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0babfff700 1 -- 172.21.15.32:0/437567824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c009550 con 0x7f0bac0a1340 2023-09-30T22:25:43.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.213+0000 7f0bb16ba700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac134170 0x7f0bac1231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.214+0000 7f0babfff700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 0x7f0bac122c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c520 tx=0x7f0ba000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.214+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00052d0 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.214+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba0005430 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.214+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba4009a70 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.214+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00038c0 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.214+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac0a25e0 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.215+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ba0003a20 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.215+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0ba003eac0 con 0x7f0bac0a1340 2023-09-30T22:25:43.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.215+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac074ea0 con 0x7f0bac0a1340 2023-09-30T22:25:43.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.220+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ba0017020 con 0x7f0bac0a1340 2023-09-30T22:25:43.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.388+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ebc072980 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.389+0000 7f9ea37fe700 1 -- 172.21.15.32:0/1090672872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9eb001b7b0 con 0x7f9ebc0a0a30 2023-09-30T22:25:43.390 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 msgr2=0x7f9ebc134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 0x7f9ebc134090 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009c40 tx=0x7f9eb0004750 comp rx=0 tx=0).stop 2023-09-30T22:25:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 shutdown_connections 2023-09-30T22:25:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ebc138d80 0x7f9ebc1345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ebc09d630 0x7f9ebc133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 --2- 172.21.15.32:0/1090672872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ebc0a0a30 0x7f9ebc134090 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 >> 172.21.15.32:0/1090672872 conn(0x7f9ebc0a3050 msgr2=0x7f9ebc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 shutdown_connections 2023-09-30T22:25:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.391+0000 7f9ec1705700 1 -- 172.21.15.32:0/1090672872 wait complete. 2023-09-30T22:25:43.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.424+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0bac1331d0 con 0x7f0bac0a1340 2023-09-30T22:25:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.425+0000 7f0ba97fa700 1 -- 172.21.15.32:0/437567824 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ba000f260 con 0x7f0bac0a1340 2023-09-30T22:25:43.426 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 msgr2=0x7f0bac122c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 0x7f0bac122c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c520 tx=0x7f0ba000c830 comp rx=0 tx=0).stop 2023-09-30T22:25:43.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 shutdown_connections 2023-09-30T22:25:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bac134170 0x7f0bac1231c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bac0a0a30 0x7f0bac122740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 --2- 172.21.15.32:0/437567824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bac0a1340 0x7f0bac122c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 >> 172.21.15.32:0/437567824 conn(0x7f0bac0a6550 msgr2=0x7f0bac09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 shutdown_connections 2023-09-30T22:25:43.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.427+0000 7f0bb311d700 1 -- 172.21.15.32:0/437567824 wait complete. 2023-09-30T22:25:43.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:43.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:43.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.827+0000 7f2a88cf1700 1 Processor -- start 2023-09-30T22:25:43.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.827+0000 7f2a88cf1700 1 -- start start 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a88cf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a0a30 0x7f2a840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a88cf1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 0x7f2a84133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a88cf1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a88cf1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8409edd0 con 0x7f2a840a1340 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a88cf1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8409e1f0 con 0x7f2a840a0a30 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a88cf1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8409dc50 con 0x7f2a84133fa0 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a81d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 0x7f2a84133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a8259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a0a30 0x7f2a840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a81d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 0x7f2a84133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35118/0 (socket says 172.21.15.32:35118) 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a8259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a0a30 0x7f2a840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43278/0 (socket says 172.21.15.32:43278) 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a81d9b700 1 -- 172.21.15.32:0/763946120 learned_addr learned my addr 172.21.15.32:0/763946120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 --2- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 -- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a0a30 msgr2=0x7f2a840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 --2- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a0a30 0x7f2a840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 -- 172.21.15.32:0/763946120 >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 msgr2=0x7f2a84133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 --2- 172.21.15.32:0/763946120 >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 0x7f2a84133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 -- 172.21.15.32:0/763946120 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840a1fe0 con 0x7f2a84133fa0 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a81d9b700 1 --2- 172.21.15.32:0/763946120 >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 0x7f2a84133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.828+0000 7f2a82d9d700 1 --2- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84134410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009dc0 tx=0x7f2a7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d53655ac7122d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a8159a700 1 -- 172.21.15.32:0/763946120 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400e070 con 0x7f2a84133fa0 2023-09-30T22:25:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a8159a700 1 -- 172.21.15.32:0/763946120 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400bae0 con 0x7f2a84133fa0 2023-09-30T22:25:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a8159a700 1 -- 172.21.15.32:0/763946120 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400be90 con 0x7f2a84133fa0 2023-09-30T22:25:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 -- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 msgr2=0x7f2a84134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84134410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009dc0 tx=0x7f2a7400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 -- 172.21.15.32:0/763946120 shutdown_connections 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a0a30 0x7f2a840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/763946120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84134410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/763946120 >> v2:172.21.15.32:3300/0 conn(0x7f2a840a1340 0x7f2a84133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 -- 172.21.15.32:0/763946120 >> 172.21.15.32:0/763946120 conn(0x7f2a840a6550 msgr2=0x7f2a840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 -- 172.21.15.32:0/763946120 shutdown_connections 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.829+0000 7f2a88cf1700 1 -- 172.21.15.32:0/763946120 wait complete. 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 Processor -- start 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 -- start start 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 0x7f2a841202e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a1340 0x7f2a84120820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84120d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84123bc0 con 0x7f2a840a0a30 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a841336d0 con 0x7f2a840a1340 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a88cf1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84123140 con 0x7f2a84133fa0 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 0x7f2a841202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 0x7f2a841202e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35132/0 (socket says 172.21.15.32:35132) 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a8259c700 1 -- 172.21.15.32:0/1210689031 learned_addr learned my addr 172.21.15.32:0/1210689031 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a81d9b700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a1340 0x7f2a84120820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.831+0000 7f2a82d9d700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84120d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a81d9b700 1 -- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 msgr2=0x7f2a84120d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a81d9b700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84120d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a81d9b700 1 -- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 msgr2=0x7f2a841202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a81d9b700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 0x7f2a841202e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a81d9b700 1 -- 172.21.15.32:0/1210689031 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c009550 con 0x7f2a840a1340 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a82d9d700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84120d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a8259c700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 0x7f2a841202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a81d9b700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a1340 0x7f2a84120820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00e9a0 tx=0x7f2a6c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c004030 con 0x7f2a840a1340 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c004190 con 0x7f2a840a1340 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a74009a70 con 0x7f2a840a1340 2023-09-30T22:25:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c004520 con 0x7f2a840a1340 2023-09-30T22:25:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.832+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841235e0 con 0x7f2a840a1340 2023-09-30T22:25:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.833+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2a6c0036a0 con 0x7f2a840a1340 2023-09-30T22:25:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.833+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2a6c03f590 con 0x7f2a840a1340 2023-09-30T22:25:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.833+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a84074ea0 con 0x7f2a840a1340 2023-09-30T22:25:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.838+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a6c019d60 con 0x7f2a840a1340 2023-09-30T22:25:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.865+0000 7feef1a82700 1 Processor -- start 2023-09-30T22:25:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.865+0000 7feef1a82700 1 -- start start 2023-09-30T22:25:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feef1a82700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feef1a82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 0x7feeec13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feef1a82700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec0a2a70 con 0x7feeec09d630 2023-09-30T22:25:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feef1a82700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0a01d0 con 0x7feeec138d80 2023-09-30T22:25:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feef1a82700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeec09e2b0 con 0x7feeec0a0a30 2023-09-30T22:25:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feeeb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feeebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 0x7feeec13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feeeaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feeec09d630 0x7feeec0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52138/0 (socket says 172.21.15.32:52138) 2023-09-30T22:25:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feeeb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46854/0 (socket says 172.21.15.32:46854) 2023-09-30T22:25:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feeebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 0x7feeec13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35148/0 (socket says 172.21.15.32:35148) 2023-09-30T22:25:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.866+0000 7feeeaffd700 1 -- 172.21.15.32:0/1763305482 learned_addr learned my addr 172.21.15.32:0/1763305482 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeeb7fe700 1 -- 172.21.15.32:0/1763305482 >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 msgr2=0x7feeec13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeea7fc700 1 -- 172.21.15.32:0/1763305482 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513431488 0 0) 0x7feeec0a2a70 con 0x7feeec09d630 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeeb7fe700 1 --2- 172.21.15.32:0/1763305482 >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 0x7feeec13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeeb7fe700 1 -- 172.21.15.32:0/1763305482 >> v1:172.21.15.32:6789/0 conn(0x7feeec09d630 legacy=0x7feeec0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeeb7fe700 1 -- 172.21.15.32:0/1763305482 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec09dc50 con 0x7feeec0a0a30 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeebfff700 1 --2- 172.21.15.32:0/1763305482 >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 0x7feeec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeeb7fe700 1 --2- 172.21.15.32:0/1763305482 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec0a0e00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feedc009840 tx=0x7feedc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b2dce376be99f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeea7fc700 1 -- 172.21.15.32:0/1763305482 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc011830 con 0x7feeec0a0a30 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeea7fc700 1 -- 172.21.15.32:0/1763305482 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc011990 con 0x7feeec0a0a30 2023-09-30T22:25:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.867+0000 7feeea7fc700 1 -- 172.21.15.32:0/1763305482 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc011d20 con 0x7feeec0a0a30 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 -- 172.21.15.32:0/1763305482 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 msgr2=0x7feeec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 --2- 172.21.15.32:0/1763305482 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec0a0e00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feedc009840 tx=0x7feedc009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 -- 172.21.15.32:0/1763305482 shutdown_connections 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 --2- 172.21.15.32:0/1763305482 >> v2:172.21.15.32:3300/0 conn(0x7feeec138d80 0x7feeec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 --2- 172.21.15.32:0/1763305482 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec0a0e00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 -- 172.21.15.32:0/1763305482 >> 172.21.15.32:0/1763305482 conn(0x7feeec0a3050 msgr2=0x7feeec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 -- 172.21.15.32:0/1763305482 shutdown_connections 2023-09-30T22:25:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.868+0000 7feef1a82700 1 -- 172.21.15.32:0/1763305482 wait complete. 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 Processor -- start 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 -- start start 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec09d630 0x7feeec1338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeec138d80 0x7feeec134330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec138b40 con 0x7feeec09d630 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec135d10 con 0x7feeec138d80 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.869+0000 7feef1a82700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeec13ae40 con 0x7feeec0a0a30 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec133df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46858/0 (socket says 172.21.15.32:46858) 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 -- 172.21.15.32:0/929644375 learned_addr learned my addr 172.21.15.32:0/929644375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeb7fe700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec09d630 0x7feeec1338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeebfff700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeec138d80 0x7feeec134330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 -- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeec138d80 msgr2=0x7feeec134330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeec138d80 0x7feeec134330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 -- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec09d630 msgr2=0x7feeec1338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec09d630 0x7feeec1338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 -- 172.21.15.32:0/929644375 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009a70 con 0x7feeec0a0a30 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeb7fe700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec09d630 0x7feeec1338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feeeaffd700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec133df0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feee000c520 tx=0x7feee000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0004c80 con 0x7feeec0a0a30 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0004de0 con 0x7feeec0a0a30 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc0094a0 con 0x7feeec0a0a30 2023-09-30T22:25:43.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0007310 con 0x7feeec0a0a30 2023-09-30T22:25:43.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.870+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec136690 con 0x7feeec0a0a30 2023-09-30T22:25:43.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.871+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec0747f0 con 0x7feeec0a0a30 2023-09-30T22:25:43.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.872+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feee0007470 con 0x7feeec0a0a30 2023-09-30T22:25:43.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.872+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feee003f9c0 con 0x7feeec0a0a30 2023-09-30T22:25:43.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:43.877+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feee00133c0 con 0x7feeec0a0a30 2023-09-30T22:25:44.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.046+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2a840a6cb0 con 0x7f2a840a1340 2023-09-30T22:25:44.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.047+0000 7f2a7affd700 1 -- 172.21.15.32:0/1210689031 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a6c019300 con 0x7f2a840a1340 2023-09-30T22:25:44.048 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a1340 msgr2=0x7f2a84120820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a1340 0x7f2a84120820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00e9a0 tx=0x7f2a6c00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 shutdown_connections 2023-09-30T22:25:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a84133fa0 0x7f2a84120d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a840a0a30 0x7f2a841202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 --2- 172.21.15.32:0/1210689031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a840a1340 0x7f2a84120820 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 >> 172.21.15.32:0/1210689031 conn(0x7f2a840a6550 msgr2=0x7f2a841bb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 shutdown_connections 2023-09-30T22:25:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.049+0000 7f2a88cf1700 1 -- 172.21.15.32:0/1210689031 wait complete. 2023-09-30T22:25:44.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:44.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.081+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feeec072980 con 0x7feeec0a0a30 2023-09-30T22:25:44.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.081+0000 7feed3fff700 1 -- 172.21.15.32:0/929644375 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feee0011360 con 0x7feeec0a0a30 2023-09-30T22:25:44.082 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 msgr2=0x7feeec133df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec133df0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feee000c520 tx=0x7feee000c830 comp rx=0 tx=0).stop 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 shutdown_connections 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeec138d80 0x7feeec134330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeec09d630 0x7feeec1338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 --2- 172.21.15.32:0/929644375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeec0a0a30 0x7feeec133df0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 >> 172.21.15.32:0/929644375 conn(0x7feeec0a3050 msgr2=0x7feeec09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.083+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 shutdown_connections 2023-09-30T22:25:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.084+0000 7feef1a82700 1 -- 172.21.15.32:0/929644375 wait complete. 2023-09-30T22:25:44.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:44.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:44.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.553+0000 7f0a9a2ef700 1 Processor -- start 2023-09-30T22:25:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.553+0000 7f0a9a2ef700 1 -- start start 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a9a2ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a9a2ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a9a2ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9409edd0 con 0x7f0a940a1340 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a9a2ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9409e1f0 con 0x7f0a94134170 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a9a2ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9409dc50 con 0x7f0a940a0a30 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a937fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a940a1340 0x7f0a94133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52152/0 (socket says 172.21.15.32:52152) 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a937fe700 1 -- 172.21.15.32:0/2243419884 learned_addr learned my addr 172.21.15.32:0/2243419884 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a93fff700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a9888c700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a92ffd700 1 -- 172.21.15.32:0/2243419884 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301082589 0 0) 0x7f0a9409edd0 con 0x7f0a940a1340 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a92ffd700 1 -- 172.21.15.32:0/2243419884 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a7c0035b0 con 0x7f0a940a1340 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a93fff700 1 -- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a94134170 msgr2=0x7f0a9413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a93fff700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a93fff700 1 -- 172.21.15.32:0/2243419884 >> v1:172.21.15.32:6789/0 conn(0x7f0a940a1340 legacy=0x7f0a94133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.554+0000 7f0a93fff700 1 -- 172.21.15.32:0/2243419884 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940a1fe0 con 0x7f0a940a0a30 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.555+0000 7f0a93fff700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a84009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4daf442d24c701 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.555+0000 7f0a92ffd700 1 -- 172.21.15.32:0/2243419884 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84011830 con 0x7f0a940a0a30 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.555+0000 7f0a92ffd700 1 -- 172.21.15.32:0/2243419884 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84011990 con 0x7f0a940a0a30 2023-09-30T22:25:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.555+0000 7f0a92ffd700 1 -- 172.21.15.32:0/2243419884 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84011d00 con 0x7f0a940a0a30 2023-09-30T22:25:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.555+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 msgr2=0x7f0a940a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.555+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a84009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.556+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/2243419884 shutdown_connections 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.556+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.556+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/2243419884 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.556+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/2243419884 >> 172.21.15.32:0/2243419884 conn(0x7f0a940a6550 msgr2=0x7f0a940a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.556+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/2243419884 shutdown_connections 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.556+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/2243419884 wait complete. 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 Processor -- start 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 -- start start 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940a4e10 con 0x7f0a94134170 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94124430 con 0x7f0a940a0a30 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a9a2ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94125f50 con 0x7f0a940a1340 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a937fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43330/0 (socket says 172.21.15.32:43330) 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.557+0000 7f0a93fff700 1 -- 172.21.15.32:0/4055897947 learned_addr learned my addr 172.21.15.32:0/4055897947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a9888c700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a937fe700 1 -- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 msgr2=0x7f0a941227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a937fe700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a937fe700 1 -- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a94134170 msgr2=0x7f0a94123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a937fe700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a937fe700 1 -- 172.21.15.32:0/4055897947 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840094a0 con 0x7f0a940a1340 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a937fe700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a1340 0x7f0a94122d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c490 tx=0x7f0a8800c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88005d70 con 0x7f0a940a1340 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a880070d0 con 0x7f0a940a1340 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a940a25e0 con 0x7f0a940a1340 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88007440 con 0x7f0a940a1340 2023-09-30T22:25:44.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.558+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a94133230 con 0x7f0a940a1340 2023-09-30T22:25:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.559+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a880075a0 con 0x7f0a940a1340 2023-09-30T22:25:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.559+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a8803e710 con 0x7f0a940a1340 2023-09-30T22:25:44.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.559+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94074ea0 con 0x7f0a940a1340 2023-09-30T22:25:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.568+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a88017020 con 0x7f0a940a1340 2023-09-30T22:25:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.594+0000 7f48a7bf4700 1 Processor -- start 2023-09-30T22:25:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a7bf4700 1 -- start start 2023-09-30T22:25:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a7bf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a7bf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a7bf4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00a2a70 con 0x7f48a009d630 2023-09-30T22:25:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a7bf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00a01d0 con 0x7f48a00a0a30 2023-09-30T22:25:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a7bf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48a009e2b0 con 0x7f48a0138d80 2023-09-30T22:25:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a5990700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a518f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f48a009d630 0x7f48a00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52164/0 (socket says 172.21.15.32:52164) 2023-09-30T22:25:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a518f700 1 -- 172.21.15.32:0/1326638936 learned_addr learned my addr 172.21.15.32:0/1326638936 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.595+0000 7f48a6191700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a498e700 1 -- 172.21.15.32:0/1326638936 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972138348 0 0) 0x7f48a00a2a70 con 0x7f48a009d630 2023-09-30T22:25:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a498e700 1 -- 172.21.15.32:0/1326638936 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f488c0035b0 con 0x7f48a009d630 2023-09-30T22:25:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a6191700 1 -- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 msgr2=0x7f48a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:44.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a6191700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a6191700 1 -- 172.21.15.32:0/1326638936 >> v1:172.21.15.32:6789/0 conn(0x7f48a009d630 legacy=0x7f48a00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a6191700 1 -- 172.21.15.32:0/1326638936 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a009dc50 con 0x7f48a0138d80 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a5990700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a6191700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a013b160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4898009dc0 tx=0x7f489800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fec177f8a47ba96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a498e700 1 -- 172.21.15.32:0/1326638936 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800e070 con 0x7f48a0138d80 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a498e700 1 -- 172.21.15.32:0/1326638936 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800bb20 con 0x7f48a0138d80 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.596+0000 7f48a498e700 1 -- 172.21.15.32:0/1326638936 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898012430 con 0x7f48a0138d80 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 -- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 msgr2=0x7f48a013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a013b160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4898009dc0 tx=0x7f489800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 -- 172.21.15.32:0/1326638936 shutdown_connections 2023-09-30T22:25:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a013b160 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/1326638936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 -- 172.21.15.32:0/1326638936 >> 172.21.15.32:0/1326638936 conn(0x7f48a00a3050 msgr2=0x7f48a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 -- 172.21.15.32:0/1326638936 shutdown_connections 2023-09-30T22:25:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.597+0000 7f48a7bf4700 1 -- 172.21.15.32:0/1326638936 wait complete. 2023-09-30T22:25:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.598+0000 7f48a7bf4700 1 Processor -- start 2023-09-30T22:25:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.598+0000 7f48a7bf4700 1 -- start start 2023-09-30T22:25:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a7bf4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a009d630 0x7f48a012c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a7bf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a012ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a7bf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a012cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a7bf4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a012de40 con 0x7f48a009d630 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a7bf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a012e030 con 0x7f48a00a0a30 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a7bf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00a0410 con 0x7f48a0138d80 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a518f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a012ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a6191700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a012cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a6191700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a012cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46892/0 (socket says 172.21.15.32:46892) 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a518f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a012ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43346/0 (socket says 172.21.15.32:43346) 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a6191700 1 -- 172.21.15.32:0/202468110 learned_addr learned my addr 172.21.15.32:0/202468110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.599+0000 7f48a5990700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a009d630 0x7f48a012c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a518f700 1 -- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 msgr2=0x7f48a012cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a518f700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a012cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a518f700 1 -- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a009d630 msgr2=0x7f48a012c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a518f700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a009d630 0x7f48a012c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a518f700 1 -- 172.21.15.32:0/202468110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0094a0 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a518f700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a012ca00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f489000c490 tx=0x7f489000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890005d70 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48900070d0 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4898009a70 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890007440 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.600+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a012fc80 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.601+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48900075a0 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.601+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a00747f0 con 0x7f48a00a0a30 2023-09-30T22:25:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.601+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f489003e6b0 con 0x7f48a00a0a30 2023-09-30T22:25:44.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.610+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4890017030 con 0x7f48a00a0a30 2023-09-30T22:25:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.780+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0a940a28e0 con 0x7f0a940a1340 2023-09-30T22:25:44.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.781+0000 7f0a90ff9700 1 -- 172.21.15.32:0/4055897947 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a88018af0 con 0x7f0a940a1340 2023-09-30T22:25:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a1340 msgr2=0x7f0a94122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a1340 0x7f0a94122d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c490 tx=0x7f0a8800c7a0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 shutdown_connections 2023-09-30T22:25:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 --2- 172.21.15.32:0/4055897947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 >> 172.21.15.32:0/4055897947 conn(0x7f0a940a6550 msgr2=0x7f0a9409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 shutdown_connections 2023-09-30T22:25:44.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.783+0000 7f0a9a2ef700 1 -- 172.21.15.32:0/4055897947 wait complete. 2023-09-30T22:25:44.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:44.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.816+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f48a012d8b0 con 0x7f48a00a0a30 2023-09-30T22:25:44.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.817+0000 7f48967fc700 1 -- 172.21.15.32:0/202468110 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4890023020 con 0x7f48a00a0a30 2023-09-30T22:25:44.818 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 msgr2=0x7f48a012ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a012ca00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f489000c490 tx=0x7f489000c7a0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 shutdown_connections 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48a0138d80 0x7f48a012cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48a009d630 0x7f48a012c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 --2- 172.21.15.32:0/202468110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48a00a0a30 0x7f48a012ca00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:44.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 >> 172.21.15.32:0/202468110 conn(0x7f48a00a3050 msgr2=0x7f48a012ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 shutdown_connections 2023-09-30T22:25:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:44.819+0000 7f48a7bf4700 1 -- 172.21.15.32:0/202468110 wait complete. 2023-09-30T22:25:44.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:45.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:45.098 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:45.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.214+0000 7fc4d35c5700 1 Processor -- start 2023-09-30T22:25:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.214+0000 7fc4d35c5700 1 -- start start 2023-09-30T22:25:45.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d35c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc133420 0x7fc4cc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d35c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4cc133e30 0x7fc4cc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d35c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 0x7fc4cc1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d35c5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc06b990 con 0x7fc4cc133e30 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d35c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0a0590 con 0x7fc4cc133420 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d35c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc09fb80 con 0x7fc4cc134260 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 0x7fc4cc1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 0x7fc4cc1304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46894/0 (socket says 172.21.15.32:46894) 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 -- 172.21.15.32:0/4024983574 learned_addr learned my addr 172.21.15.32:0/4024983574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d0b60700 1 --2- 172.21.15.32:0/4024983574 >> v2:172.21.15.32:3300/0 conn(0x7fc4cc133e30 0x7fc4cc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1361700 1 --2- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc133420 0x7fc4cc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 -- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc133420 msgr2=0x7fc4cc1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 --2- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc133420 0x7fc4cc1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 -- 172.21.15.32:0/4024983574 >> v2:172.21.15.32:3300/0 conn(0x7fc4cc133e30 msgr2=0x7fc4cc12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 --2- 172.21.15.32:0/4024983574 >> v2:172.21.15.32:3300/0 conn(0x7fc4cc133e30 0x7fc4cc12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 -- 172.21.15.32:0/4024983574 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0a0030 con 0x7fc4cc134260 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d0b60700 1 --2- 172.21.15.32:0/4024983574 >> v2:172.21.15.32:3300/0 conn(0x7fc4cc133e30 0x7fc4cc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.215+0000 7fc4d1b62700 1 --2- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 0x7fc4cc1304d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009e50 tx=0x7fc4c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e4dd086f435b989 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4c3fff700 1 -- 172.21.15.32:0/4024983574 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400e070 con 0x7fc4cc134260 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4c3fff700 1 -- 172.21.15.32:0/4024983574 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400b910 con 0x7fc4cc134260 2023-09-30T22:25:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4c3fff700 1 -- 172.21.15.32:0/4024983574 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400bcc0 con 0x7fc4cc134260 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 -- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 msgr2=0x7fc4cc1304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 0x7fc4cc1304d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009e50 tx=0x7fc4c400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 -- 172.21.15.32:0/4024983574 shutdown_connections 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc133420 0x7fc4cc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/4024983574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc134260 0x7fc4cc1304d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/4024983574 >> v2:172.21.15.32:3300/0 conn(0x7fc4cc133e30 0x7fc4cc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.216+0000 7fc4d35c5700 1 -- 172.21.15.32:0/4024983574 >> 172.21.15.32:0/4024983574 conn(0x7fc4cc0a4210 msgr2=0x7fc4cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.217+0000 7fc4d35c5700 1 -- 172.21.15.32:0/4024983574 shutdown_connections 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.217+0000 7fc4d35c5700 1 -- 172.21.15.32:0/4024983574 wait complete. 2023-09-30T22:25:45.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 Processor -- start 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 -- start start 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 0x7fc4cc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4cc133e30 0x7fc4cc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 0x7fc4cc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc11f990 con 0x7fc4cc133e30 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc11fbd0 con 0x7fc4cc134260 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d35c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc09df00 con 0x7fc4cc133420 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d1361700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 0x7fc4cc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d1b62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 0x7fc4cc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d0b60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4cc133e30 0x7fc4cc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d1b62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 0x7fc4cc09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43374/0 (socket says 172.21.15.32:43374) 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d1361700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 0x7fc4cc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46908/0 (socket says 172.21.15.32:46908) 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.218+0000 7fc4d1b62700 1 -- 172.21.15.32:0/2661945730 learned_addr learned my addr 172.21.15.32:0/2661945730 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1b62700 1 -- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 msgr2=0x7fc4cc09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1b62700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 0x7fc4cc09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1b62700 1 -- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4cc133e30 msgr2=0x7fc4cc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1b62700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4cc133e30 0x7fc4cc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1b62700 1 -- 172.21.15.32:0/2661945730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0094a0 con 0x7fc4cc134260 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d0b60700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4cc133e30 0x7fc4cc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1361700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 0x7fc4cc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d1b62700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 0x7fc4cc09cf00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4012000 tx=0x7fc4c4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4004240 con 0x7fc4cc134260 2023-09-30T22:25:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40043e0 con 0x7fc4cc134260 2023-09-30T22:25:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c4009b30 con 0x7fc4cc134260 2023-09-30T22:25:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4004790 con 0x7fc4cc134260 2023-09-30T22:25:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.219+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc11fe90 con 0x7fc4cc134260 2023-09-30T22:25:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.220+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4c401c410 con 0x7fc4cc134260 2023-09-30T22:25:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.220+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc074ea0 con 0x7fc4cc134260 2023-09-30T22:25:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.221+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc4c4040be0 con 0x7fc4cc134260 2023-09-30T22:25:45.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.225+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc4c400ce80 con 0x7fc4cc134260 2023-09-30T22:25:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.252+0000 7f2791d4f700 1 Processor -- start 2023-09-30T22:25:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.252+0000 7f2791d4f700 1 -- start start 2023-09-30T22:25:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f2791d4f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c0a0a30 0x7f278c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f2791d4f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f278c0a1340 0x7f278c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f2791d4f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c09edd0 con 0x7f278c133fa0 2023-09-30T22:25:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f2791d4f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c09e1f0 con 0x7f278c0a1340 2023-09-30T22:25:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f2791d4f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c09dc50 con 0x7f278c0a0a30 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f278c0a1340 0x7f278c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c0a0a30 0x7f278c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f278c133fa0 0x7f278c13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52170/0 (socket says 172.21.15.32:52170) 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c0a0a30 0x7f278c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43388/0 (socket says 172.21.15.32:43388) 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278bfff700 1 -- 172.21.15.32:0/3238814209 learned_addr learned my addr 172.21.15.32:0/3238814209 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278affd700 1 -- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c0a0a30 msgr2=0x7f278c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278affd700 1 --2- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c0a0a30 0x7f278c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278affd700 1 -- 172.21.15.32:0/3238814209 >> v1:172.21.15.32:6789/0 conn(0x7f278c133fa0 legacy=0x7f278c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278affd700 1 -- 172.21.15.32:0/3238814209 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0a1fe0 con 0x7f278c0a1340 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.253+0000 7f278a7fc700 1 -- 172.21.15.32:0/3238814209 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174699270 0 0) 0x7f278c09edd0 con 0x7f278c133fa0 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f278affd700 1 --2- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a1340 0x7f278c133a60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2780009840 tx=0x7f2780009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b797aac1b06f97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:45.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f278a7fc700 1 -- 172.21.15.32:0/3238814209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780011830 con 0x7f278c0a1340 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f278a7fc700 1 -- 172.21.15.32:0/3238814209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780011990 con 0x7f278c0a1340 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f278a7fc700 1 -- 172.21.15.32:0/3238814209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780011d20 con 0x7f278c0a1340 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f2791d4f700 1 -- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a1340 msgr2=0x7f278c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f2791d4f700 1 --2- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a1340 0x7f278c133a60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2780009840 tx=0x7f2780009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f2791d4f700 1 -- 172.21.15.32:0/3238814209 shutdown_connections 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f2791d4f700 1 --2- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c0a0a30 0x7f278c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f2791d4f700 1 --2- 172.21.15.32:0/3238814209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a1340 0x7f278c133a60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.254+0000 7f2791d4f700 1 -- 172.21.15.32:0/3238814209 >> 172.21.15.32:0/3238814209 conn(0x7f278c0a6550 msgr2=0x7f278c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.255+0000 7f2791d4f700 1 -- 172.21.15.32:0/3238814209 shutdown_connections 2023-09-30T22:25:45.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.255+0000 7f2791d4f700 1 -- 172.21.15.32:0/3238814209 wait complete. 2023-09-30T22:25:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 Processor -- start 2023-09-30T22:25:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 -- start start 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a0a30 0x7f278c1227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f278c0a1340 0x7f278c122d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 0x7f278c123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c123c90 con 0x7f278c0a0a30 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1332e0 con 0x7f278c133fa0 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f2791d4f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1244e0 con 0x7f278c0a1340 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f278bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 0x7f278c123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f278b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a0a30 0x7f278c1227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f278bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 0x7f278c123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43398/0 (socket says 172.21.15.32:43398) 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.256+0000 7f278bfff700 1 -- 172.21.15.32:0/1350126200 learned_addr learned my addr 172.21.15.32:0/1350126200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278b7fe700 1 -- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f278c0a1340 msgr2=0x7f278c122d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278b7fe700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f278c0a1340 0x7f278c122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278b7fe700 1 -- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 msgr2=0x7f278c123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278b7fe700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 0x7f278c123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278b7fe700 1 -- 172.21.15.32:0/1350126200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800094a0 con 0x7f278c0a0a30 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278bfff700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 0x7f278c123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f278b7fe700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a0a30 0x7f278c1227c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f277c009db0 tx=0x7f277c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c009c20 con 0x7f278c0a0a30 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c00f040 con 0x7f278c0a0a30 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c133710 con 0x7f278c0a0a30 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c014480 con 0x7f278c0a0a30 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.257+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c124bc0 con 0x7f278c0a0a30 2023-09-30T22:25:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.258+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f277c0145e0 con 0x7f278c0a0a30 2023-09-30T22:25:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.258+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f277c0404f0 con 0x7f278c0a0a30 2023-09-30T22:25:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.258+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c125e50 con 0x7f278c0a0a30 2023-09-30T22:25:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.263+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f277c012930 con 0x7f278c0a0a30 2023-09-30T22:25:45.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.431+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc4cc120960 con 0x7fc4cc134260 2023-09-30T22:25:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.432+0000 7fc4c1ffb700 1 -- 172.21.15.32:0/2661945730 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4c402a070 con 0x7fc4cc134260 2023-09-30T22:25:45.433 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 msgr2=0x7fc4cc09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 0x7fc4cc09cf00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4012000 tx=0x7fc4c4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 shutdown_connections 2023-09-30T22:25:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4cc134260 0x7fc4cc09cf00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4cc133420 0x7fc4cc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 --2- 172.21.15.32:0/2661945730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4cc133e30 0x7fc4cc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 >> 172.21.15.32:0/2661945730 conn(0x7fc4cc0a4210 msgr2=0x7fc4cc0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:45.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 shutdown_connections 2023-09-30T22:25:45.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.434+0000 7fc4d35c5700 1 -- 172.21.15.32:0/2661945730 wait complete. 2023-09-30T22:25:45.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.470+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f278c0a1d40 con 0x7f278c0a0a30 2023-09-30T22:25:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.471+0000 7f2773fff700 1 -- 172.21.15.32:0/1350126200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f277c02c090 con 0x7f278c0a0a30 2023-09-30T22:25:45.472 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a0a30 msgr2=0x7f278c1227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a0a30 0x7f278c1227c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f277c009db0 tx=0x7f277c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 shutdown_connections 2023-09-30T22:25:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f278c133fa0 0x7f278c123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f278c0a0a30 0x7f278c1227c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 --2- 172.21.15.32:0/1350126200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f278c0a1340 0x7f278c122d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 >> 172.21.15.32:0/1350126200 conn(0x7f278c0a6550 msgr2=0x7f278c11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 shutdown_connections 2023-09-30T22:25:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.473+0000 7f2791d4f700 1 -- 172.21.15.32:0/1350126200 wait complete. 2023-09-30T22:25:45.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:45.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:45.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.947+0000 7f88effeb700 1 Processor -- start 2023-09-30T22:25:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.947+0000 7f88effeb700 1 -- start start 2023-09-30T22:25:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88effeb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 0x7f88e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88effeb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e8134170 0x7f88e813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88effeb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e809edd0 con 0x7f88e80a1340 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88effeb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e809e1f0 con 0x7f88e80a0a30 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88effeb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88e809dc50 con 0x7f88e8134170 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 0x7f88e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88ee588700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e8134170 0x7f88e813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 0x7f88e80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43414/0 (socket says 172.21.15.32:43414) 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 -- 172.21.15.32:0/3021113746 learned_addr learned my addr 172.21.15.32:0/3021113746 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88ecd85700 1 -- 172.21.15.32:0/3021113746 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214861155 0 0) 0x7f88e809edd0 con 0x7f88e80a1340 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 -- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e8134170 msgr2=0x7f88e813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 --2- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e8134170 0x7f88e813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 -- 172.21.15.32:0/3021113746 >> v1:172.21.15.32:6789/0 conn(0x7f88e80a1340 legacy=0x7f88e8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 -- 172.21.15.32:0/3021113746 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80a1fe0 con 0x7f88e80a0a30 2023-09-30T22:25:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88ee588700 1 --2- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e8134170 0x7f88e813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.948+0000 7f88edd87700 1 --2- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 0x7f88e80a0e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009920 tx=0x7f88e4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3eee9684483be0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88ecd85700 1 -- 172.21.15.32:0/3021113746 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4011830 con 0x7f88e80a0a30 2023-09-30T22:25:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88ecd85700 1 -- 172.21.15.32:0/3021113746 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e4011990 con 0x7f88e80a0a30 2023-09-30T22:25:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88ecd85700 1 -- 172.21.15.32:0/3021113746 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4011d20 con 0x7f88e80a0a30 2023-09-30T22:25:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 -- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 msgr2=0x7f88e80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 --2- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 0x7f88e80a0e00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009920 tx=0x7f88e4009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 -- 172.21.15.32:0/3021113746 shutdown_connections 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 --2- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e8134170 0x7f88e813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 --2- 172.21.15.32:0/3021113746 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e80a0a30 0x7f88e80a0e00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 -- 172.21.15.32:0/3021113746 >> 172.21.15.32:0/3021113746 conn(0x7f88e80a6550 msgr2=0x7f88e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 -- 172.21.15.32:0/3021113746 shutdown_connections 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.949+0000 7f88effeb700 1 -- 172.21.15.32:0/3021113746 wait complete. 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.950+0000 7f88effeb700 1 Processor -- start 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.950+0000 7f88effeb700 1 -- start start 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88effeb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 0x7f88e8122750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88effeb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e80a1340 0x7f88e8122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88effeb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e8134170 0x7f88e81231d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88effeb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80a4f50 con 0x7f88e80a0a30 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88effeb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8125c40 con 0x7f88e8134170 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88effeb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80a0220 con 0x7f88e80a1340 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88edd87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 0x7f88e8122750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ed586700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e80a1340 0x7f88e8122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88edd87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 0x7f88e8122750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35224/0 (socket says 172.21.15.32:35224) 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88edd87700 1 -- 172.21.15.32:0/263099470 learned_addr learned my addr 172.21.15.32:0/263099470 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ee588700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e8134170 0x7f88e81231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ed586700 1 -- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e8134170 msgr2=0x7f88e81231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ed586700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e8134170 0x7f88e81231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ed586700 1 -- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 msgr2=0x7f88e8122750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ed586700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 0x7f88e8122750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88ed586700 1 -- 172.21.15.32:0/263099470 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e0009a70 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.951+0000 7f88edd87700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 0x7f88e8122750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.952+0000 7f88ed586700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e80a1340 0x7f88e8122c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c430 tx=0x7f88d800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.952+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80054e0 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.952+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800ce30 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.952+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e4009550 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.952+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80038c0 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.952+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e80a25e0 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.953+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88d8003a20 con 0x7f88e80a1340 2023-09-30T22:25:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.953+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f88d803e9e0 con 0x7f88e80a1340 2023-09-30T22:25:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.953+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e8074ea0 con 0x7f88e80a1340 2023-09-30T22:25:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.958+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f88d8017080 con 0x7f88e80a1340 2023-09-30T22:25:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.995+0000 7fb980ceb700 1 Processor -- start 2023-09-30T22:25:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.995+0000 7fb980ceb700 1 -- start start 2023-09-30T22:25:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb980ceb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09e930 0x7fb97c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb980ceb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:45.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb980ceb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c070940 con 0x7fb97c09f240 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb980ceb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c06fd60 con 0x7fb97c09f650 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb980ceb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c06fa80 con 0x7fb97c09e930 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb971d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb97c09f240 0x7fb97c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52184/0 (socket says 172.21.15.32:52184) 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb971d9b700 1 -- 172.21.15.32:0/1010882704 learned_addr learned my addr 172.21.15.32:0/1010882704 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb97ad9d700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb97a59c700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09e930 0x7fb97c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.996+0000 7fb979d9b700 1 -- 172.21.15.32:0/1010882704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036732846 0 0) 0x7fb97c070940 con 0x7fb97c09f240 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb979d9b700 1 -- 172.21.15.32:0/1010882704 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9680035b0 con 0x7fb97c09f240 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb97ad9d700 1 -- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09e930 msgr2=0x7fb97c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb97ad9d700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09e930 0x7fb97c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:45.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb97ad9d700 1 -- 172.21.15.32:0/1010882704 >> v1:172.21.15.32:6789/0 conn(0x7fb97c09f240 legacy=0x7fb97c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb97ad9d700 1 -- 172.21.15.32:0/1010882704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c09fb60 con 0x7fb97c09f650 2023-09-30T22:25:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb97ad9d700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c131f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb974009dc0 tx=0x7fb97400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a089a51cd74a0199 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb979d9b700 1 -- 172.21.15.32:0/1010882704 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400e070 con 0x7fb97c09f650 2023-09-30T22:25:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb979d9b700 1 -- 172.21.15.32:0/1010882704 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97400bb20 con 0x7fb97c09f650 2023-09-30T22:25:46.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.997+0000 7fb979d9b700 1 -- 172.21.15.32:0/1010882704 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974012430 con 0x7fb97c09f650 2023-09-30T22:25:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 -- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 msgr2=0x7fb97c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c131f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb974009dc0 tx=0x7fb97400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 -- 172.21.15.32:0/1010882704 shutdown_connections 2023-09-30T22:25:46.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c131f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 --2- 172.21.15.32:0/1010882704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09e930 0x7fb97c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 -- 172.21.15.32:0/1010882704 >> 172.21.15.32:0/1010882704 conn(0x7fb97c0a4210 msgr2=0x7fb97c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 -- 172.21.15.32:0/1010882704 shutdown_connections 2023-09-30T22:25:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:45.998+0000 7fb980ceb700 1 -- 172.21.15.32:0/1010882704 wait complete. 2023-09-30T22:25:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.000+0000 7fb980ceb700 1 Processor -- start 2023-09-30T22:25:46.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.000+0000 7fb980ceb700 1 -- start start 2023-09-30T22:25:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb980ceb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c09e930 0x7fb97c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb980ceb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 0x7fb97c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb980ceb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb980ceb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c06cd60 con 0x7fb97c09e930 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb980ceb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c06d430 con 0x7fb97c09f650 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb980ceb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a9240 con 0x7fb97c09f240 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb971d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 0x7fb97c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb971d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 0x7fb97c06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46950/0 (socket says 172.21.15.32:46950) 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb971d9b700 1 -- 172.21.15.32:0/2030763330 learned_addr learned my addr 172.21.15.32:0/2030763330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97a59c700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c09e930 0x7fb97c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97ad9d700 1 -- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 msgr2=0x7fb97c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97ad9d700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 0x7fb97c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97ad9d700 1 -- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c09e930 msgr2=0x7fb97c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97ad9d700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c09e930 0x7fb97c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97ad9d700 1 -- 172.21.15.32:0/2030763330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974009a70 con 0x7fb97c09f650 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb971d9b700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 0x7fb97c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.001+0000 7fb97a59c700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c09e930 0x7fb97c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.002+0000 7fb97ad9d700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c06c330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb974016000 tx=0x7fb974004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.002+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974004800 con 0x7fb97c09f650 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.002+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb974020070 con 0x7fb97c09f650 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.002+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c0a0190 con 0x7fb97c09f650 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.002+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740276c0 con 0x7fb97c09f650 2023-09-30T22:25:46.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.002+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c1a8fc0 con 0x7fb97c09f650 2023-09-30T22:25:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.003+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c071d80 con 0x7fb97c09f650 2023-09-30T22:25:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.003+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb974031430 con 0x7fb97c09f650 2023-09-30T22:25:46.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.003+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb974022070 con 0x7fb97c09f650 2023-09-30T22:25:46.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.008+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb974012430 con 0x7fb97c09f650 2023-09-30T22:25:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.202+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88e80a28e0 con 0x7f88e80a1340 2023-09-30T22:25:46.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.203+0000 7f88de7fc700 1 -- 172.21.15.32:0/263099470 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88d8010770 con 0x7f88e80a1340 2023-09-30T22:25:46.204 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e80a1340 msgr2=0x7f88e8122c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e80a1340 0x7f88e8122c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c430 tx=0x7f88d800c740 comp rx=0 tx=0).stop 2023-09-30T22:25:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 shutdown_connections 2023-09-30T22:25:46.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f88e8134170 0x7f88e81231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f88e80a0a30 0x7f88e8122750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 --2- 172.21.15.32:0/263099470 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f88e80a1340 0x7f88e8122c90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 >> 172.21.15.32:0/263099470 conn(0x7f88e80a6550 msgr2=0x7f88e809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 shutdown_connections 2023-09-30T22:25:46.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.205+0000 7f88effeb700 1 -- 172.21.15.32:0/263099470 wait complete. 2023-09-30T22:25:46.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:46.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.235+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb97c074ea0 con 0x7fb97c09f650 2023-09-30T22:25:46.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.235+0000 7fb9737fe700 1 -- 172.21.15.32:0/2030763330 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb97402c020 con 0x7fb97c09f650 2023-09-30T22:25:46.238 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:46.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 msgr2=0x7fb97c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c06c330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb974016000 tx=0x7fb974004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 shutdown_connections 2023-09-30T22:25:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb97c09f650 0x7fb97c06c330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb97c09e930 0x7fb97c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 --2- 172.21.15.32:0/2030763330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb97c09f240 0x7fb97c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 >> 172.21.15.32:0/2030763330 conn(0x7fb97c0a4210 msgr2=0x7fb97c1b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 shutdown_connections 2023-09-30T22:25:46.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.237+0000 7fb980ceb700 1 -- 172.21.15.32:0/2030763330 wait complete. 2023-09-30T22:25:46.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:46.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:46.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.599+0000 7f10dd4fb700 1 Processor -- start 2023-09-30T22:25:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.599+0000 7f10dd4fb700 1 -- start start 2023-09-30T22:25:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.599+0000 7f10dd4fb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 0x7f10d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10dd4fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 0x7f10d8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10dd4fb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d809f130 con 0x7f10d809e000 2023-09-30T22:25:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10dd4fb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d809f520 con 0x7f10d809d6f0 2023-09-30T22:25:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10dd4fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809e930 con 0x7f10d809e410 2023-09-30T22:25:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 0x7f10d8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 0x7f10d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10d809e000 0x7f10d8130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52188/0 (socket says 172.21.15.32:52188) 2023-09-30T22:25:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 0x7f10d809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35234/0 (socket says 172.21.15.32:35234) 2023-09-30T22:25:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 0x7f10d8130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43460/0 (socket says 172.21.15.32:43460) 2023-09-30T22:25:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d67fc700 1 -- 172.21.15.32:0/3806651932 learned_addr learned my addr 172.21.15.32:0/3806651932 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 -- 172.21.15.32:0/3806651932 >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 msgr2=0x7f10d809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 --2- 172.21.15.32:0/3806651932 >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 0x7f10d809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 -- 172.21.15.32:0/3806651932 >> v1:172.21.15.32:6789/0 conn(0x7f10d809e000 legacy=0x7f10d8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 -- 172.21.15.32:0/3806651932 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d809b2d0 con 0x7f10d809e410 2023-09-30T22:25:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d5ffb700 1 -- 172.21.15.32:0/3806651932 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743782953 0 0) 0x7f10d809f130 con 0x7f10d809e000 2023-09-30T22:25:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d6ffd700 1 --2- 172.21.15.32:0/3806651932 >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 0x7f10d809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:46.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.600+0000 7f10d77fe700 1 --2- 172.21.15.32:0/3806651932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 0x7f10d8130d70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009d90 tx=0x7f10d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1875281f0565d0b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.601+0000 7f10d5ffb700 1 -- 172.21.15.32:0/3806651932 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d000e070 con 0x7f10d809e410 2023-09-30T22:25:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.601+0000 7f10d5ffb700 1 -- 172.21.15.32:0/3806651932 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d000bb20 con 0x7f10d809e410 2023-09-30T22:25:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.601+0000 7f10d5ffb700 1 -- 172.21.15.32:0/3806651932 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0012430 con 0x7f10d809e410 2023-09-30T22:25:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.601+0000 7f10dd4fb700 1 -- 172.21.15.32:0/3806651932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 msgr2=0x7f10d8130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.601+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/3806651932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 0x7f10d8130d70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009d90 tx=0x7f10d000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.602+0000 7f10dd4fb700 1 -- 172.21.15.32:0/3806651932 shutdown_connections 2023-09-30T22:25:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.602+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/3806651932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809e410 0x7f10d8130d70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.602+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/3806651932 >> v2:172.21.15.32:3300/0 conn(0x7f10d809d6f0 0x7f10d809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.602+0000 7f10dd4fb700 1 -- 172.21.15.32:0/3806651932 >> 172.21.15.32:0/3806651932 conn(0x7f10d8066f50 msgr2=0x7f10d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.602+0000 7f10dd4fb700 1 -- 172.21.15.32:0/3806651932 shutdown_connections 2023-09-30T22:25:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.602+0000 7f10dd4fb700 1 -- 172.21.15.32:0/3806651932 wait complete. 2023-09-30T22:25:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.603+0000 7f10dd4fb700 1 Processor -- start 2023-09-30T22:25:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.603+0000 7f10dd4fb700 1 -- start start 2023-09-30T22:25:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10dd4fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809d6f0 0x7f10d811ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10dd4fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 0x7f10d8120450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10dd4fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d809e410 0x7f10d8120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10dd4fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8123a30 con 0x7f10d809e000 2023-09-30T22:25:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10dd4fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d81231c0 con 0x7f10d809d6f0 2023-09-30T22:25:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10dd4fb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809c700 con 0x7f10d809e410 2023-09-30T22:25:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10d67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 0x7f10d8120450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10d77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d809e410 0x7f10d8120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10d67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 0x7f10d8120450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35238/0 (socket says 172.21.15.32:35238) 2023-09-30T22:25:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10d67fc700 1 -- 172.21.15.32:0/1016877666 learned_addr learned my addr 172.21.15.32:0/1016877666 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.604+0000 7f10d6ffd700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809d6f0 0x7f10d811ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d67fc700 1 -- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d809e410 msgr2=0x7f10d8120990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d67fc700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d809e410 0x7f10d8120990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d67fc700 1 -- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809d6f0 msgr2=0x7f10d811ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d67fc700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809d6f0 0x7f10d811ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d67fc700 1 -- 172.21.15.32:0/1016877666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c80094a0 con 0x7f10d809e000 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d77fe700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d809e410 0x7f10d8120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10d67fc700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 0x7f10d8120450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c520 tx=0x7f10cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004c80 con 0x7f10d809e000 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc004e00 con 0x7f10d809e000 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d0009a70 con 0x7f10d809e000 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc007310 con 0x7f10d809e000 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.605+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d8122330 con 0x7f10d809e000 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.606+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10cc007470 con 0x7f10d809e000 2023-09-30T22:25:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.606+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f10cc03f300 con 0x7f10d809e000 2023-09-30T22:25:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.606+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d811cc30 con 0x7f10d809e000 2023-09-30T22:25:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.611+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10cc0123b0 con 0x7f10d809e000 2023-09-30T22:25:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.637+0000 7fa513c96700 1 Processor -- start 2023-09-30T22:25:46.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.637+0000 7fa513c96700 1 -- start start 2023-09-30T22:25:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa513c96700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa50c0a0a30 0x7fa50c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa513c96700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa513c96700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c09edd0 con 0x7fa50c0a1340 2023-09-30T22:25:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa513c96700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c09e1f0 con 0x7fa50c0a0a30 2023-09-30T22:25:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa513c96700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c09dc50 con 0x7fa50c134170 2023-09-30T22:25:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa511a32700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa50c0a0a30 0x7fa50c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa511231700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa50c0a1340 0x7fa50c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52202/0 (socket says 172.21.15.32:52202) 2023-09-30T22:25:46.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa511231700 1 -- 172.21.15.32:0/1937648790 learned_addr learned my addr 172.21.15.32:0/1937648790 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa512233700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa512233700 1 -- 172.21.15.32:0/1937648790 >> v2:172.21.15.32:3300/0 conn(0x7fa50c0a0a30 msgr2=0x7fa50c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa512233700 1 --2- 172.21.15.32:0/1937648790 >> v2:172.21.15.32:3300/0 conn(0x7fa50c0a0a30 0x7fa50c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa512233700 1 -- 172.21.15.32:0/1937648790 >> v1:172.21.15.32:6789/0 conn(0x7fa50c0a1340 legacy=0x7fa50c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa512233700 1 -- 172.21.15.32:0/1937648790 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0a1fe0 con 0x7fa50c134170 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa510a30700 1 -- 172.21.15.32:0/1937648790 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504198528 0 0) 0x7fa50c09edd0 con 0x7fa50c0a1340 2023-09-30T22:25:46.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa511a32700 1 --2- 172.21.15.32:0/1937648790 >> v2:172.21.15.32:3300/0 conn(0x7fa50c0a0a30 0x7fa50c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.638+0000 7fa512233700 1 --2- 172.21.15.32:0/1937648790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c13bd30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa504009d90 tx=0x7fa50400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656be7048869c514 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa510a30700 1 -- 172.21.15.32:0/1937648790 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400e070 con 0x7fa50c134170 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa510a30700 1 -- 172.21.15.32:0/1937648790 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400bb20 con 0x7fa50c134170 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa510a30700 1 -- 172.21.15.32:0/1937648790 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504012430 con 0x7fa50c134170 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 -- 172.21.15.32:0/1937648790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 msgr2=0x7fa50c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 --2- 172.21.15.32:0/1937648790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c13bd30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa504009d90 tx=0x7fa50400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:46.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 -- 172.21.15.32:0/1937648790 shutdown_connections 2023-09-30T22:25:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 --2- 172.21.15.32:0/1937648790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c13bd30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 --2- 172.21.15.32:0/1937648790 >> v2:172.21.15.32:3300/0 conn(0x7fa50c0a0a30 0x7fa50c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 -- 172.21.15.32:0/1937648790 >> 172.21.15.32:0/1937648790 conn(0x7fa50c0a6550 msgr2=0x7fa50c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 -- 172.21.15.32:0/1937648790 shutdown_connections 2023-09-30T22:25:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.639+0000 7fa513c96700 1 -- 172.21.15.32:0/1937648790 wait complete. 2023-09-30T22:25:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.640+0000 7fa513c96700 1 Processor -- start 2023-09-30T22:25:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.640+0000 7fa513c96700 1 -- start start 2023-09-30T22:25:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa513c96700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c0a0a30 0x7fa50c1226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa513c96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 0x7fa50c122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa513c96700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa513c96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c124b80 con 0x7fa50c0a1340 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa513c96700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c123b40 con 0x7fa50c134170 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa513c96700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c124400 con 0x7fa50c0a0a30 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 0x7fa50c122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 0x7fa50c122c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35244/0 (socket says 172.21.15.32:35244) 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 -- 172.21.15.32:0/2564013356 learned_addr learned my addr 172.21.15.32:0/2564013356 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa512233700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 -- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c0a0a30 msgr2=0x7fa50c1226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c0a0a30 0x7fa50c1226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 -- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 msgr2=0x7fa50c123160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c123160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 -- 172.21.15.32:0/2564013356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080094a0 con 0x7fa50c0a1340 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.641+0000 7fa511231700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 0x7fa50c122c20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c520 tx=0x7fa4fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:46.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.642+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc004c80 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.642+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc004e00 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.642+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc007310 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.642+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa504009a70 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.642+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c0a25e0 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.642+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa4fc007470 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.643+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa4fc03f390 con 0x7fa50c0a1340 2023-09-30T22:25:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.643+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c074ea0 con 0x7fa50c0a1340 2023-09-30T22:25:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.648+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa4fc0123b0 con 0x7fa50c0a1340 2023-09-30T22:25:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.818+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f10d8074ea0 con 0x7f10d809e000 2023-09-30T22:25:46.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.818+0000 7f10bf7fe700 1 -- 172.21.15.32:0/1016877666 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10cc011360 con 0x7f10d809e000 2023-09-30T22:25:46.820 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:46.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 msgr2=0x7f10d8120450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 0x7f10d8120450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c520 tx=0x7f10cc00c830 comp rx=0 tx=0).stop 2023-09-30T22:25:46.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 shutdown_connections 2023-09-30T22:25:46.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10d809e410 0x7f10d8120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10d809d6f0 0x7f10d811ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 --2- 172.21.15.32:0/1016877666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10d809e000 0x7f10d8120450 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 >> 172.21.15.32:0/1016877666 conn(0x7f10d8066f50 msgr2=0x7f10d8130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 shutdown_connections 2023-09-30T22:25:46.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.821+0000 7f10dd4fb700 1 -- 172.21.15.32:0/1016877666 wait complete. 2023-09-30T22:25:46.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:46.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.856+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa50c133200 con 0x7fa50c0a1340 2023-09-30T22:25:46.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.857+0000 7fa5027fc700 1 -- 172.21.15.32:0/2564013356 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4fc011360 con 0x7fa50c0a1340 2023-09-30T22:25:46.858 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 msgr2=0x7fa50c122c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 0x7fa50c122c20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c520 tx=0x7fa4fc00c830 comp rx=0 tx=0).stop 2023-09-30T22:25:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 shutdown_connections 2023-09-30T22:25:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa50c134170 0x7fa50c123160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa50c0a0a30 0x7fa50c1226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 --2- 172.21.15.32:0/2564013356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa50c0a1340 0x7fa50c122c20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 >> 172.21.15.32:0/2564013356 conn(0x7fa50c0a6550 msgr2=0x7fa50c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 shutdown_connections 2023-09-30T22:25:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:46.859+0000 7fa513c96700 1 -- 172.21.15.32:0/2564013356 wait complete. 2023-09-30T22:25:46.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:47.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:47.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.369+0000 7f3fceb95700 1 Processor -- start 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.369+0000 7f3fceb95700 1 -- start start 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fceb95700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 0x7f3fc809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fceb95700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fceb95700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8070940 con 0x7f3fc809f240 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fceb95700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc806fd60 con 0x7f3fc809e930 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fceb95700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc806fa80 con 0x7f3fc809f650 2023-09-30T22:25:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcc931700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 0x7f3fc809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcd132700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcc931700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 0x7f3fc809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35258/0 (socket says 172.21.15.32:35258) 2023-09-30T22:25:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fc7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3fc809f240 0x7f3fc81317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52216/0 (socket says 172.21.15.32:52216) 2023-09-30T22:25:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcc931700 1 -- 172.21.15.32:0/1578072165 learned_addr learned my addr 172.21.15.32:0/1578072165 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcd132700 1 -- 172.21.15.32:0/1578072165 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 msgr2=0x7f3fc809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcd132700 1 --2- 172.21.15.32:0/1578072165 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 0x7f3fc809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcd132700 1 -- 172.21.15.32:0/1578072165 >> v1:172.21.15.32:6789/0 conn(0x7f3fc809f240 legacy=0x7f3fc81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcd132700 1 -- 172.21.15.32:0/1578072165 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc809fb60 con 0x7f3fc809f650 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fcc931700 1 --2- 172.21.15.32:0/1578072165 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 0x7f3fc809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.370+0000 7f3fc77fe700 1 -- 172.21.15.32:0/1578072165 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996628542 0 0) 0x7f3fc8070940 con 0x7f3fc809f240 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.371+0000 7f3fcd132700 1 --2- 172.21.15.32:0/1578072165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc8131f00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009d90 tx=0x7f3fc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e0cb49b90593b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.371+0000 7f3fc77fe700 1 -- 172.21.15.32:0/1578072165 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000e070 con 0x7f3fc809f650 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.371+0000 7f3fc77fe700 1 -- 172.21.15.32:0/1578072165 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc000bb20 con 0x7f3fc809f650 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.371+0000 7f3fc77fe700 1 -- 172.21.15.32:0/1578072165 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0012430 con 0x7f3fc809f650 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 -- 172.21.15.32:0/1578072165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 msgr2=0x7f3fc8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 --2- 172.21.15.32:0/1578072165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc8131f00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009d90 tx=0x7f3fc000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 -- 172.21.15.32:0/1578072165 shutdown_connections 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 --2- 172.21.15.32:0/1578072165 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc8131f00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 --2- 172.21.15.32:0/1578072165 >> v2:172.21.15.32:3300/0 conn(0x7f3fc809e930 0x7f3fc809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 -- 172.21.15.32:0/1578072165 >> 172.21.15.32:0/1578072165 conn(0x7f3fc80a4210 msgr2=0x7f3fc80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 -- 172.21.15.32:0/1578072165 shutdown_connections 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.372+0000 7f3fceb95700 1 -- 172.21.15.32:0/1578072165 wait complete. 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 Processor -- start 2023-09-30T22:25:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 -- start start 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 0x7f3fc806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc809f240 0x7f3fc806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc806d4c0 con 0x7f3fc809e930 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81a9460 con 0x7f3fc809f240 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.373+0000 7f3fceb95700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81a91a0 con 0x7f3fc809f650 2023-09-30T22:25:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 0x7f3fc806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 0x7f3fc806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35268/0 (socket says 172.21.15.32:35268) 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcd132700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 -- 172.21.15.32:0/4040255849 learned_addr learned my addr 172.21.15.32:0/4040255849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fc7fff700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc809f240 0x7f3fc806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 -- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 msgr2=0x7f3fc806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 -- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc809f240 msgr2=0x7f3fc806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc809f240 0x7f3fc806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 -- 172.21.15.32:0/4040255849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb8009550 con 0x7f3fc809e930 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcd132700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:47.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fcc931700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 0x7f3fc806b8b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800ebe0 tx=0x7f3fb800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800c850 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb800c9b0 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0009a70 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800cd40 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.374+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc80a0190 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.375+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fb8013470 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.375+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3fb803f860 con 0x7f3fc809e930 2023-09-30T22:25:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.376+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc81a0b30 con 0x7f3fc809e930 2023-09-30T22:25:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.381+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fb8017080 con 0x7f3fc809e930 2023-09-30T22:25:47.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.420+0000 7f8e12984700 1 Processor -- start 2023-09-30T22:25:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.420+0000 7f8e12984700 1 -- start start 2023-09-30T22:25:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.420+0000 7f8e12984700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c09f410 0x7f8e0c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e12984700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e0c136fe0 0x7f8e0c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e12984700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0c070940 con 0x7f8e0c09e930 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e12984700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0c06fd60 con 0x7f8e0c136fe0 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e12984700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c06fa80 con 0x7f8e0c09f410 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e10f21700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e0c136fe0 0x7f8e0c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c09f410 0x7f8e0c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8e0c09e930 0x7f8e0c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52226/0 (socket says 172.21.15.32:52226) 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0bfff700 1 -- 172.21.15.32:0/1916383711 learned_addr learned my addr 172.21.15.32:0/1916383711 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0b7fe700 1 -- 172.21.15.32:0/1916383711 >> v2:172.21.15.32:3300/0 conn(0x7f8e0c136fe0 msgr2=0x7f8e0c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0b7fe700 1 --2- 172.21.15.32:0/1916383711 >> v2:172.21.15.32:3300/0 conn(0x7f8e0c136fe0 0x7f8e0c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0b7fe700 1 -- 172.21.15.32:0/1916383711 >> v1:172.21.15.32:6789/0 conn(0x7f8e0c09e930 legacy=0x7f8e0c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0b7fe700 1 -- 172.21.15.32:0/1916383711 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c09fb60 con 0x7f8e0c09f410 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e10f21700 1 --2- 172.21.15.32:0/1916383711 >> v2:172.21.15.32:3300/0 conn(0x7f8e0c136fe0 0x7f8e0c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0affd700 1 -- 172.21.15.32:0/1916383711 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208812545 0 0) 0x7f8e0c070940 con 0x7f8e0c09e930 2023-09-30T22:25:47.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0b7fe700 1 --2- 172.21.15.32:0/1916383711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c09f410 0x7f8e0c136aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009840 tx=0x7f8e00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42837042fc720731 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:47.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.421+0000 7f8e0affd700 1 -- 172.21.15.32:0/1916383711 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00011830 con 0x7f8e0c09f410 2023-09-30T22:25:47.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e0affd700 1 -- 172.21.15.32:0/1916383711 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00011990 con 0x7f8e0c09f410 2023-09-30T22:25:47.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e0affd700 1 -- 172.21.15.32:0/1916383711 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00011d20 con 0x7f8e0c09f410 2023-09-30T22:25:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 -- 172.21.15.32:0/1916383711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c09f410 msgr2=0x7f8e0c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 --2- 172.21.15.32:0/1916383711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c09f410 0x7f8e0c136aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009840 tx=0x7f8e00009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 -- 172.21.15.32:0/1916383711 shutdown_connections 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 --2- 172.21.15.32:0/1916383711 >> v2:172.21.15.32:3300/0 conn(0x7f8e0c136fe0 0x7f8e0c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 --2- 172.21.15.32:0/1916383711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c09f410 0x7f8e0c136aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 -- 172.21.15.32:0/1916383711 >> 172.21.15.32:0/1916383711 conn(0x7f8e0c0a4210 msgr2=0x7f8e0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 -- 172.21.15.32:0/1916383711 shutdown_connections 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.422+0000 7f8e12984700 1 -- 172.21.15.32:0/1916383711 wait complete. 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.424+0000 7f8e12984700 1 Processor -- start 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.424+0000 7f8e12984700 1 -- start start 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.424+0000 7f8e12984700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e0c09e930 0x7f8e0c1c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.424+0000 7f8e12984700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e0c09f410 0x7f8e0c1c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.424+0000 7f8e12984700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 0x7f8e0c11c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e12984700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c1c7080 con 0x7f8e0c09f410 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e12984700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c11e2f0 con 0x7f8e0c136fe0 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e12984700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c120320 con 0x7f8e0c09e930 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 0x7f8e0c11c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 0x7f8e0c11c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43526/0 (socket says 172.21.15.32:43526) 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 -- 172.21.15.32:0/1859369757 learned_addr learned my addr 172.21.15.32:0/1859369757 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e0bfff700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e0c09e930 0x7f8e0c1c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e0b7fe700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e0c09f410 0x7f8e0c1c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 -- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e0c09e930 msgr2=0x7f8e0c1c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e0c09e930 0x7f8e0c1c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 -- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e0c09f410 msgr2=0x7f8e0c1c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e0c09f410 0x7f8e0c1c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 -- 172.21.15.32:0/1859369757 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04009a70 con 0x7f8e0c136fe0 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e10f21700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 0x7f8e0c11c6e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400efc0 tx=0x7f8e0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400cc90 con 0x7f8e0c136fe0 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0400f040 con 0x7f8e0c136fe0 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e000094a0 con 0x7f8e0c136fe0 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.425+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04014480 con 0x7f8e0c136fe0 2023-09-30T22:25:47.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.426+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c1325d0 con 0x7f8e0c136fe0 2023-09-30T22:25:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.426+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e04014600 con 0x7f8e0c136fe0 2023-09-30T22:25:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.426+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e040400b0 con 0x7f8e0c136fe0 2023-09-30T22:25:47.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.427+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0c071d80 con 0x7f8e0c136fe0 2023-09-30T22:25:47.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.432+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e04018020 con 0x7f8e0c136fe0 2023-09-30T22:25:47.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.623+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3fc8071d20 con 0x7f3fc809e930 2023-09-30T22:25:47.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.624+0000 7f3fc57fa700 1 -- 172.21.15.32:0/4040255849 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fb8021020 con 0x7f3fc809e930 2023-09-30T22:25:47.625 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 msgr2=0x7f3fc806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 0x7f3fc806b8b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800ebe0 tx=0x7f3fb800eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 shutdown_connections 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fc809f650 0x7f3fc806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fc809e930 0x7f3fc806b8b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 --2- 172.21.15.32:0/4040255849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fc809f240 0x7f3fc806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 >> 172.21.15.32:0/4040255849 conn(0x7f3fc80a4210 msgr2=0x7f3fc81b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 shutdown_connections 2023-09-30T22:25:47.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.626+0000 7f3fceb95700 1 -- 172.21.15.32:0/4040255849 wait complete. 2023-09-30T22:25:47.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:47.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.654+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e0c074ea0 con 0x7f8e0c136fe0 2023-09-30T22:25:47.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.654+0000 7f8e08ff9700 1 -- 172.21.15.32:0/1859369757 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e040112c0 con 0x7f8e0c136fe0 2023-09-30T22:25:47.655 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:47.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 msgr2=0x7f8e0c11c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 0x7f8e0c11c6e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400efc0 tx=0x7f8e0400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 shutdown_connections 2023-09-30T22:25:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e0c136fe0 0x7f8e0c11c6e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e0c09e930 0x7f8e0c1c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 --2- 172.21.15.32:0/1859369757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e0c09f410 0x7f8e0c1c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 >> 172.21.15.32:0/1859369757 conn(0x7f8e0c0a4210 msgr2=0x7f8e0c134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 shutdown_connections 2023-09-30T22:25:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.656+0000 7f8e12984700 1 -- 172.21.15.32:0/1859369757 wait complete. 2023-09-30T22:25:47.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:47.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:47.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.983+0000 7fde74c21700 1 Processor -- start 2023-09-30T22:25:47.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.983+0000 7fde74c21700 1 -- start start 2023-09-30T22:25:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde74c21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde74c21700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde7009d630 0x7fde700a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde74c21700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde700a2a70 con 0x7fde70133930 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde74c21700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde700a01d0 con 0x7fde7009d630 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde74c21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7009e2b0 con 0x7fde700a0a30 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fde70133930 0x7fde70133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52240/0 (socket says 172.21.15.32:52240) 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6ed9d700 1 -- 172.21.15.32:0/2060657694 learned_addr learned my addr 172.21.15.32:0/2060657694 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:47.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6dd9b700 1 --2- 172.21.15.32:0/2060657694 >> v2:172.21.15.32:3300/0 conn(0x7fde7009d630 0x7fde700a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6e59c700 1 -- 172.21.15.32:0/2060657694 >> v2:172.21.15.32:3300/0 conn(0x7fde7009d630 msgr2=0x7fde700a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6e59c700 1 --2- 172.21.15.32:0/2060657694 >> v2:172.21.15.32:3300/0 conn(0x7fde7009d630 0x7fde700a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6d59a700 1 -- 172.21.15.32:0/2060657694 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292128703 0 0) 0x7fde700a2a70 con 0x7fde70133930 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6e59c700 1 -- 172.21.15.32:0/2060657694 >> v1:172.21.15.32:6789/0 conn(0x7fde70133930 legacy=0x7fde70133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.984+0000 7fde6e59c700 1 -- 172.21.15.32:0/2060657694 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7009dc50 con 0x7fde700a0a30 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.985+0000 7fde6e59c700 1 --2- 172.21.15.32:0/2060657694 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde700a0e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde60009d90 tx=0x7fde6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c1f442280f603e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.985+0000 7fde6d59a700 1 -- 172.21.15.32:0/2060657694 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000e070 con 0x7fde700a0a30 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.985+0000 7fde6d59a700 1 -- 172.21.15.32:0/2060657694 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000bb20 con 0x7fde700a0a30 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.985+0000 7fde6d59a700 1 -- 172.21.15.32:0/2060657694 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde60012430 con 0x7fde700a0a30 2023-09-30T22:25:47.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.985+0000 7fde74c21700 1 -- 172.21.15.32:0/2060657694 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 msgr2=0x7fde700a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.985+0000 7fde74c21700 1 --2- 172.21.15.32:0/2060657694 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde700a0e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde60009d90 tx=0x7fde6000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.986+0000 7fde74c21700 1 -- 172.21.15.32:0/2060657694 shutdown_connections 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.986+0000 7fde74c21700 1 --2- 172.21.15.32:0/2060657694 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde700a0e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.986+0000 7fde74c21700 1 --2- 172.21.15.32:0/2060657694 >> v2:172.21.15.32:3300/0 conn(0x7fde7009d630 0x7fde700a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.986+0000 7fde74c21700 1 -- 172.21.15.32:0/2060657694 >> 172.21.15.32:0/2060657694 conn(0x7fde700a3050 msgr2=0x7fde700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.986+0000 7fde74c21700 1 -- 172.21.15.32:0/2060657694 shutdown_connections 2023-09-30T22:25:47.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.986+0000 7fde74c21700 1 -- 172.21.15.32:0/2060657694 wait complete. 2023-09-30T22:25:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 Processor -- start 2023-09-30T22:25:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 -- start start 2023-09-30T22:25:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7009d630 0x7fde7011f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde7011faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde70133930 0x7fde701c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:47.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700a0410 con 0x7fde7009d630 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7011f110 con 0x7fde700a0a30 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.987+0000 7fde74c21700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde70121870 con 0x7fde70133930 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde7011faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7009d630 0x7fde7011f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde7011faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43542/0 (socket says 172.21.15.32:43542) 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 -- 172.21.15.32:0/3166388102 learned_addr learned my addr 172.21.15.32:0/3166388102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6ed9d700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde70133930 0x7fde701c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 -- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde70133930 msgr2=0x7fde701c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde70133930 0x7fde701c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 -- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7009d630 msgr2=0x7fde7011f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7009d630 0x7fde7011f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:47.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 -- 172.21.15.32:0/3166388102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60009a70 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6e59c700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7009d630 0x7fde7011f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde6dd9b700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde7011faf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e980 tx=0x7fde5800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58009710 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5800c690 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde70120030 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800ca20 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.988+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde70122f00 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.989+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde5800cb80 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.989+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde70074560 con 0x7fde700a0a30 2023-09-30T22:25:47.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.990+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde5803f6b0 con 0x7fde700a0a30 2023-09-30T22:25:47.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:47.994+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde580232f0 con 0x7fde700a0a30 2023-09-30T22:25:48.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 Processor -- start 2023-09-30T22:25:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 -- start start 2023-09-30T22:25:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c09edd0 con 0x7ff04c0a0a30 2023-09-30T22:25:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c09e1f0 con 0x7ff04c0a1340 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.022+0000 7ff052a76700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c09dc50 con 0x7ff04c133fa0 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff051013700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43544/0 (socket says 172.21.15.32:43544) 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 -- 172.21.15.32:0/2954211908 learned_addr learned my addr 172.21.15.32:0/2954211908 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff050812700 1 --2- 172.21.15.32:0/2954211908 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 -- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 msgr2=0x7ff04c134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 --2- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 -- 172.21.15.32:0/2954211908 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 msgr2=0x7ff04c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:48.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 --2- 172.21.15.32:0/2954211908 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 -- 172.21.15.32:0/2954211908 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0a1fe0 con 0x7ff04c0a1340 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff051013700 1 --2- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff050812700 1 --2- 172.21.15.32:0/2954211908 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04bfff700 1 --2- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c133a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff040009920 tx=0x7ff040009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f3d731ff9d1f66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04b7fe700 1 -- 172.21.15.32:0/2954211908 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040011830 con 0x7ff04c0a1340 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04b7fe700 1 -- 172.21.15.32:0/2954211908 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff040011990 con 0x7ff04c0a1340 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.023+0000 7ff04b7fe700 1 -- 172.21.15.32:0/2954211908 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040011d20 con 0x7ff04c0a1340 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 -- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 msgr2=0x7ff04c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 --2- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c133a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff040009920 tx=0x7ff040009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 -- 172.21.15.32:0/2954211908 shutdown_connections 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 --2- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 --2- 172.21.15.32:0/2954211908 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 --2- 172.21.15.32:0/2954211908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c133a60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 -- 172.21.15.32:0/2954211908 >> 172.21.15.32:0/2954211908 conn(0x7ff04c0a6550 msgr2=0x7ff04c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 -- 172.21.15.32:0/2954211908 shutdown_connections 2023-09-30T22:25:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.024+0000 7ff052a76700 1 -- 172.21.15.32:0/2954211908 wait complete. 2023-09-30T22:25:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.025+0000 7ff052a76700 1 Processor -- start 2023-09-30T22:25:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.025+0000 7ff052a76700 1 -- start start 2023-09-30T22:25:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff052a76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 0x7ff04c1200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff052a76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff052a76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff052a76700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c121d90 con 0x7ff04c0a0a30 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff052a76700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c123920 con 0x7ff04c0a1340 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff052a76700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0a36c0 con 0x7ff04c133fa0 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff050812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 0x7ff04c1200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff050812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 0x7ff04c1200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35310/0 (socket says 172.21.15.32:35310) 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff050812700 1 -- 172.21.15.32:0/1910742790 learned_addr learned my addr 172.21.15.32:0/1910742790 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff051013700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff051013700 1 -- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 msgr2=0x7ff04c120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff051013700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff051013700 1 -- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 msgr2=0x7ff04c1200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff051013700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 0x7ff04c1200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff051013700 1 -- 172.21.15.32:0/1910742790 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0094a0 con 0x7ff04c133fa0 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.026+0000 7ff050812700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 0x7ff04c1200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.027+0000 7ff051013700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c120b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff04400efc0 tx=0x7ff04400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.027+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400ca90 con 0x7ff04c133fa0 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.027+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400f040 con 0x7ff04c133fa0 2023-09-30T22:25:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.027+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400ce30 con 0x7ff04c133fa0 2023-09-30T22:25:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.027+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff044009a70 con 0x7ff04c133fa0 2023-09-30T22:25:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.027+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff040009550 con 0x7ff04c133fa0 2023-09-30T22:25:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.028+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff044014480 con 0x7ff04c133fa0 2023-09-30T22:25:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.028+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff04403fd30 con 0x7ff04c133fa0 2023-09-30T22:25:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.028+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c123540 con 0x7ff04c133fa0 2023-09-30T22:25:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.033+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff044012910 con 0x7ff04c133fa0 2023-09-30T22:25:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.203+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde700747f0 con 0x7fde700a0a30 2023-09-30T22:25:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.203+0000 7fde66ffd700 1 -- 172.21.15.32:0/3166388102 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde5801bd00 con 0x7fde700a0a30 2023-09-30T22:25:48.204 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 msgr2=0x7fde7011faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde7011faf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e980 tx=0x7fde5800ec90 comp rx=0 tx=0).stop 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 shutdown_connections 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde70133930 0x7fde701c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7009d630 0x7fde7011f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 --2- 172.21.15.32:0/3166388102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde700a0a30 0x7fde7011faf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 >> 172.21.15.32:0/3166388102 conn(0x7fde700a3050 msgr2=0x7fde700a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 shutdown_connections 2023-09-30T22:25:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.205+0000 7fde74c21700 1 -- 172.21.15.32:0/3166388102 wait complete. 2023-09-30T22:25:48.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.241+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff04c0a1d20 con 0x7ff04c133fa0 2023-09-30T22:25:48.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.242+0000 7ff0497fa700 1 -- 172.21.15.32:0/1910742790 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff04c0a1d20 con 0x7ff04c133fa0 2023-09-30T22:25:48.243 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 msgr2=0x7ff04c120b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c120b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff04400efc0 tx=0x7ff04400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 shutdown_connections 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c133fa0 0x7ff04c120b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c0a0a30 0x7ff04c1200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 --2- 172.21.15.32:0/1910742790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c0a1340 0x7ff04c120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 >> 172.21.15.32:0/1910742790 conn(0x7ff04c0a6550 msgr2=0x7ff04c1bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.244+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 shutdown_connections 2023-09-30T22:25:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.245+0000 7ff052a76700 1 -- 172.21.15.32:0/1910742790 wait complete. 2023-09-30T22:25:48.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:48.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:48.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 Processor -- start 2023-09-30T22:25:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 -- start start 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09e1d0 0x7fdeac135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeac135e50 0x7fdeac138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac09f130 con 0x7fdeac09d6f0 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac09f520 con 0x7fdeac135e50 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.789+0000 7fdeb2333700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac09e930 con 0x7fdeac09e1d0 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeb08d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeac135e50 0x7fdeac138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09e1d0 0x7fdeac135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeabfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdeac09d6f0 0x7fdeac09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52248/0 (socket says 172.21.15.32:52248) 2023-09-30T22:25:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeabfff700 1 -- 172.21.15.32:0/3166180459 learned_addr learned my addr 172.21.15.32:0/3166180459 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeb08d0700 1 -- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09e1d0 msgr2=0x7fdeac135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeb08d0700 1 --2- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09e1d0 0x7fdeac135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeb08d0700 1 -- 172.21.15.32:0/3166180459 >> v1:172.21.15.32:6789/0 conn(0x7fdeac09d6f0 legacy=0x7fdeac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeb08d0700 1 -- 172.21.15.32:0/3166180459 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac09b2d0 con 0x7fdeac135e50 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09e1d0 0x7fdeac135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeb08d0700 1 --2- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac135e50 0x7fdeac138230 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009dc0 tx=0x7fdea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6134825df81c979 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeaaffd700 1 -- 172.21.15.32:0/3166180459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400e070 con 0x7fdeac135e50 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeaaffd700 1 -- 172.21.15.32:0/3166180459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400bae0 con 0x7fdeac135e50 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.790+0000 7fdeaaffd700 1 -- 172.21.15.32:0/3166180459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400be90 con 0x7fdeac135e50 2023-09-30T22:25:48.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 -- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac135e50 msgr2=0x7fdeac138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac135e50 0x7fdeac138230 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009dc0 tx=0x7fdea400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 -- 172.21.15.32:0/3166180459 shutdown_connections 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac135e50 0x7fdeac138230 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3166180459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09e1d0 0x7fdeac135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 -- 172.21.15.32:0/3166180459 >> 172.21.15.32:0/3166180459 conn(0x7fdeac066f50 msgr2=0x7fdeac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 -- 172.21.15.32:0/3166180459 shutdown_connections 2023-09-30T22:25:48.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.791+0000 7fdeb2333700 1 -- 172.21.15.32:0/3166180459 wait complete. 2023-09-30T22:25:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 Processor -- start 2023-09-30T22:25:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 -- start start 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 0x7fdeac11b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac09e1d0 0x7fdeac11b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac135e50 0x7fdeac11be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac11ec00 con 0x7fdeac09e1d0 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac11d0e0 con 0x7fdeac135e50 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.793+0000 7fdeb2333700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac06e840 con 0x7fdeac09d6f0 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 0x7fdeac11b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac09e1d0 0x7fdeac11b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 0x7fdeac11b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:47032/0 (socket says 172.21.15.32:47032) 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeabfff700 1 -- 172.21.15.32:0/3351047845 learned_addr learned my addr 172.21.15.32:0/3351047845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeb08d0700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac135e50 0x7fdeac11be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 msgr2=0x7fdeac11b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 0x7fdeac11b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac135e50 msgr2=0x7fdeac11be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac135e50 0x7fdeac11be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3351047845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea00094a0 con 0x7fdeac09e1d0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeabfff700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 0x7fdeac11b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac09e1d0 0x7fdeac11b900 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdea0010bc0 tx=0x7fdea0010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea000ed20 con 0x7fdeac09e1d0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea000ee80 con 0x7fdeac09e1d0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.794+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0011690 con 0x7fdeac09e1d0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.795+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea4009a70 con 0x7fdeac09e1d0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.795+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac11f670 con 0x7fdeac09e1d0 2023-09-30T22:25:48.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.795+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac074ea0 con 0x7fdeac09e1d0 2023-09-30T22:25:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.796+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdea00117f0 con 0x7fdeac09e1d0 2023-09-30T22:25:48.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.796+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdea0041df0 con 0x7fdeac09e1d0 2023-09-30T22:25:48.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.801+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdea0018760 con 0x7fdeac09e1d0 2023-09-30T22:25:48.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.827+0000 7f13fe4fe700 1 Processor -- start 2023-09-30T22:25:48.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.827+0000 7f13fe4fe700 1 -- start start 2023-09-30T22:25:48.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fe4fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809f410 0x7f13f8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fe4fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fe4fe700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f8070940 con 0x7f13f809e930 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fe4fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f806fd60 con 0x7f13f809f410 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fe4fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13f806fa80 con 0x7f13f8136fe0 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fca9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809f410 0x7f13f8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f13f809e930 0x7f13f809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52260/0 (socket says 172.21.15.32:52260) 2023-09-30T22:25:48.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f7fff700 1 -- 172.21.15.32:0/112633036 learned_addr learned my addr 172.21.15.32:0/112633036 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f77fe700 1 -- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 msgr2=0x7f13f81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f6ffd700 1 -- 172.21.15.32:0/112633036 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854707784 0 0) 0x7f13f8070940 con 0x7f13f809e930 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f77fe700 1 --2- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f77fe700 1 -- 172.21.15.32:0/112633036 >> v1:172.21.15.32:6789/0 conn(0x7f13f809e930 legacy=0x7f13f809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13f77fe700 1 -- 172.21.15.32:0/112633036 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f809fb60 con 0x7f13f809f410 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.828+0000 7f13fca9b700 1 --2- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.829+0000 7f13f77fe700 1 --2- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809f410 0x7f13f8136aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009870 tx=0x7f13ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b25f04e82f06229 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.829+0000 7f13f6ffd700 1 -- 172.21.15.32:0/112633036 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011830 con 0x7f13f809f410 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.829+0000 7f13f6ffd700 1 -- 172.21.15.32:0/112633036 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec011990 con 0x7f13f809f410 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.829+0000 7f13f6ffd700 1 -- 172.21.15.32:0/112633036 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec011d20 con 0x7f13f809f410 2023-09-30T22:25:48.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 -- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809f410 msgr2=0x7f13f8136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809f410 0x7f13f8136aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009870 tx=0x7f13ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 -- 172.21.15.32:0/112633036 shutdown_connections 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/112633036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809f410 0x7f13f8136aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 -- 172.21.15.32:0/112633036 >> 172.21.15.32:0/112633036 conn(0x7f13f80a4210 msgr2=0x7f13f80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 -- 172.21.15.32:0/112633036 shutdown_connections 2023-09-30T22:25:48.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.830+0000 7f13fe4fe700 1 -- 172.21.15.32:0/112633036 wait complete. 2023-09-30T22:25:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 Processor -- start 2023-09-30T22:25:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 -- start start 2023-09-30T22:25:48.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 0x7f13f806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f809f410 0x7f13f806c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8121160 con 0x7f13f809f410 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f806d5c0 con 0x7f13f809e930 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.831+0000 7f13fe4fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13f806cc40 con 0x7f13f8136fe0 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 0x7f13f806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13fca9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 0x7f13f806c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43586/0 (socket says 172.21.15.32:43586) 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 -- 172.21.15.32:0/1933876794 learned_addr learned my addr 172.21.15.32:0/1933876794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f77fe700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f809f410 0x7f13f806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:48.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 -- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 msgr2=0x7f13f81c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 -- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f809f410 msgr2=0x7f13f806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f809f410 0x7f13f806c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 -- 172.21.15.32:0/1933876794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009a70 con 0x7f13f809e930 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f77fe700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f809f410 0x7f13f806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13fca9b700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f7fff700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 0x7f13f806c1c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13e800c520 tx=0x7f13e800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004c80 con 0x7f13f809e930 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8004e00 con 0x7f13f809e930 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0094a0 con 0x7f13f809e930 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.832+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8007310 con 0x7f13f809e930 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.833+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f8121590 con 0x7f13f809e930 2023-09-30T22:25:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.833+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13e8007470 con 0x7f13f809e930 2023-09-30T22:25:48.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.833+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f809dfd0 con 0x7f13f809e930 2023-09-30T22:25:48.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.834+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f13e803f2b0 con 0x7f13f809e930 2023-09-30T22:25:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:48.839+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13e80133c0 con 0x7f13f809e930 2023-09-30T22:25:49.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.009+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdeac09c700 con 0x7fdeac09e1d0 2023-09-30T22:25:49.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.010+0000 7fdea8ff9700 1 -- 172.21.15.32:0/3351047845 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdea001e020 con 0x7fdeac09e1d0 2023-09-30T22:25:49.011 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac09e1d0 msgr2=0x7fdeac11b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac09e1d0 0x7fdeac11b900 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdea0010bc0 tx=0x7fdea0010ed0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 shutdown_connections 2023-09-30T22:25:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac135e50 0x7fdeac11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac09d6f0 0x7fdeac11b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 --2- 172.21.15.32:0/3351047845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac09e1d0 0x7fdeac11b900 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 >> 172.21.15.32:0/3351047845 conn(0x7fdeac066f50 msgr2=0x7fdeac070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 shutdown_connections 2023-09-30T22:25:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.012+0000 7fdeb2333700 1 -- 172.21.15.32:0/3351047845 wait complete. 2023-09-30T22:25:49.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.043+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f13f80a3d20 con 0x7f13f809e930 2023-09-30T22:25:49.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.043+0000 7f13f4ff9700 1 -- 172.21.15.32:0/1933876794 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13e80102c0 con 0x7f13f809e930 2023-09-30T22:25:49.044 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.045+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 msgr2=0x7f13f806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.045+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 0x7f13f806c1c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13e800c520 tx=0x7f13e800c830 comp rx=0 tx=0).stop 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.045+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 shutdown_connections 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.045+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13f8136fe0 0x7f13f81c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.045+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13f809e930 0x7f13f806c1c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.045+0000 7f13fe4fe700 1 --2- 172.21.15.32:0/1933876794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f809f410 0x7f13f806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.046+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 >> 172.21.15.32:0/1933876794 conn(0x7f13f80a4210 msgr2=0x7f13f8135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.046+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 shutdown_connections 2023-09-30T22:25:49.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.046+0000 7f13fe4fe700 1 -- 172.21.15.32:0/1933876794 wait complete. 2023-09-30T22:25:49.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:49.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:49.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 Processor -- start 2023-09-30T22:25:49.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 -- start start 2023-09-30T22:25:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb340a0a30 0x7ffb340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 0x7ffb34133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb34133fa0 0x7ffb34134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3409edd0 con 0x7ffb340a0a30 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3409e1f0 con 0x7ffb34133fa0 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb39fd7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3409dc50 con 0x7ffb340a1340 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb337fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb340a0a30 0x7ffb340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.369+0000 7ffb2affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 0x7ffb34133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb340a0a30 0x7ffb340a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35352/0 (socket says 172.21.15.32:35352) 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb2affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 0x7ffb34133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:47066/0 (socket says 172.21.15.32:47066) 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 -- 172.21.15.32:0/558657052 learned_addr learned my addr 172.21.15.32:0/558657052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb33fff700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb34133fa0 0x7ffb34134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 -- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 msgr2=0x7ffb34133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 0x7ffb34133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 -- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb34133fa0 msgr2=0x7ffb34134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb34133fa0 0x7ffb34134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 -- 172.21.15.32:0/558657052 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340a1fe0 con 0x7ffb340a0a30 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb2affd700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 0x7ffb34133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb33fff700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb34133fa0 0x7ffb34134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb337fe700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb340a0a30 0x7ffb340a0e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098f0 tx=0x7ffb24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b74b1c5a6b4130d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb32ffd700 1 -- 172.21.15.32:0/558657052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24011830 con 0x7ffb340a0a30 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb32ffd700 1 -- 172.21.15.32:0/558657052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24011990 con 0x7ffb340a0a30 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.370+0000 7ffb32ffd700 1 -- 172.21.15.32:0/558657052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24011d20 con 0x7ffb340a0a30 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 -- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb340a0a30 msgr2=0x7ffb340a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb340a0a30 0x7ffb340a0e00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098f0 tx=0x7ffb24009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:49.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 -- 172.21.15.32:0/558657052 shutdown_connections 2023-09-30T22:25:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb34133fa0 0x7ffb34134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb340a0a30 0x7ffb340a0e00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/558657052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a1340 0x7ffb34133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 -- 172.21.15.32:0/558657052 >> 172.21.15.32:0/558657052 conn(0x7ffb340a6550 msgr2=0x7ffb340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 -- 172.21.15.32:0/558657052 shutdown_connections 2023-09-30T22:25:49.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.371+0000 7ffb39fd7700 1 -- 172.21.15.32:0/558657052 wait complete. 2023-09-30T22:25:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.372+0000 7ffb39fd7700 1 Processor -- start 2023-09-30T22:25:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.372+0000 7ffb39fd7700 1 -- start start 2023-09-30T22:25:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb39fd7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a0a30 0x7ffb341212d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb39fd7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 0x7ffb34121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb39fd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 0x7ffb34121d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb39fd7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341326a0 con 0x7ffb34133fa0 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb39fd7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34123d70 con 0x7ffb340a1340 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb39fd7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34124850 con 0x7ffb340a0a30 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb2affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 0x7ffb34121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 0x7ffb34121d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a0a30 0x7ffb341212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb2affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 0x7ffb34121810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43600/0 (socket says 172.21.15.32:43600) 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb2affd700 1 -- 172.21.15.32:0/1550696508 learned_addr learned my addr 172.21.15.32:0/1550696508 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 0x7ffb34121d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35358/0 (socket says 172.21.15.32:35358) 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 -- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 msgr2=0x7ffb34121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 0x7ffb34121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 -- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 msgr2=0x7ffb34121d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 0x7ffb34121d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 -- 172.21.15.32:0/1550696508 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0094a0 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb33fff700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 0x7ffb34121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb2affd700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 0x7ffb34121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.373+0000 7ffb337fe700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a0a30 0x7ffb341212d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098f0 tx=0x7ffb24004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.374+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24004240 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.374+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2401f070 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.374+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c009a70 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.374+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb240045c0 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.374+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb24009550 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.374+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb24021030 con 0x7ffb340a0a30 2023-09-30T22:25:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.375+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffb240408d0 con 0x7ffb340a0a30 2023-09-30T22:25:49.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.375+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb34124990 con 0x7ffb340a0a30 2023-09-30T22:25:49.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.380+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb24015080 con 0x7ffb340a0a30 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 Processor -- start 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 -- start start 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809e850 0x7f252809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f252809f160 0x7f2528133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f25281340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252806bce0 con 0x7f252809f160 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252806bec0 con 0x7f252809e850 2023-09-30T22:25:49.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.407+0000 7f252d722700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f252806caa0 con 0x7f252809f550 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f25281340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f252809f160 0x7f2528133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f2526ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809e850 0x7f252809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f25281340c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:47086/0 (socket says 172.21.15.32:47086) 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25277fe700 1 -- 172.21.15.32:0/98371842 learned_addr learned my addr 172.21.15.32:0/98371842 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 -- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 msgr2=0x7f25281340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f25281340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 -- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809e850 msgr2=0x7f252809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809e850 0x7f252809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 -- 172.21.15.32:0/98371842 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25280a00a0 con 0x7f252809f160 2023-09-30T22:25:49.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25277fe700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f25281340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f2526ffd700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809e850 0x7f252809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f25267fc700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809f160 0x7f2528133b80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f251c009840 tx=0x7f251c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb5efac5a3eeb4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f2525ffb700 1 -- 172.21.15.32:0/98371842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c011830 con 0x7f252809f160 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f2525ffb700 1 -- 172.21.15.32:0/98371842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c011990 con 0x7f252809f160 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.408+0000 7f2525ffb700 1 -- 172.21.15.32:0/98371842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c011d20 con 0x7f252809f160 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 -- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809f160 msgr2=0x7f2528133b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809f160 0x7f2528133b80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f251c009840 tx=0x7f251c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 -- 172.21.15.32:0/98371842 shutdown_connections 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809e850 0x7f252809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f25281340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 --2- 172.21.15.32:0/98371842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809f160 0x7f2528133b80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 -- 172.21.15.32:0/98371842 >> 172.21.15.32:0/98371842 conn(0x7f25280a3f70 msgr2=0x7f25280a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 -- 172.21.15.32:0/98371842 shutdown_connections 2023-09-30T22:25:49.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.409+0000 7f252d722700 1 -- 172.21.15.32:0/98371842 wait complete. 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.410+0000 7f252d722700 1 Processor -- start 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.410+0000 7f252d722700 1 -- start start 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f252d722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 0x7f25281c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f252d722700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 0x7f25281c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f252d722700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f2528122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f252d722700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2528124710 con 0x7f252809e850 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f252d722700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252806fb10 con 0x7f252809f160 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f252d722700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2528126840 con 0x7f252809f550 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f25277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f2528122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 0x7f25281c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f25267fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 0x7f25281c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 0x7f25281c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35374/0 (socket says 172.21.15.32:35374) 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f25267fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 0x7f25281c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43620/0 (socket says 172.21.15.32:43620) 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f25277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f2528122b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:47098/0 (socket says 172.21.15.32:47098) 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 -- 172.21.15.32:0/3417295654 learned_addr learned my addr 172.21.15.32:0/3417295654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 -- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 msgr2=0x7f2528122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f2528122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 -- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 msgr2=0x7f25281c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 0x7f25281c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f2526ffd700 1 -- 172.21.15.32:0/3417295654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180094a0 con 0x7f252809e850 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f25277fe700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f2528122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.411+0000 7f25267fc700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 0x7f25281c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.412+0000 7f2526ffd700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 0x7f25281c8bd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f251800ebe0 tx=0x7f251800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.412+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251800c7f0 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.412+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251800c950 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.412+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2520009b30 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.412+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251800cce0 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.412+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c0094a0 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.413+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25280747f0 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.413+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2518013470 con 0x7f252809e850 2023-09-30T22:25:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.413+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f251803fb40 con 0x7f252809e850 2023-09-30T22:25:49.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.422+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2518013720 con 0x7f252809e850 2023-09-30T22:25:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.590+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffb340a0220 con 0x7ffb340a0a30 2023-09-30T22:25:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.590+0000 7ffb30ff9700 1 -- 172.21.15.32:0/1550696508 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb2401a450 con 0x7ffb340a0a30 2023-09-30T22:25:49.592 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a0a30 msgr2=0x7ffb341212d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a0a30 0x7ffb341212d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098f0 tx=0x7ffb24004750 comp rx=0 tx=0).stop 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 shutdown_connections 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb34133fa0 0x7ffb34121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb340a0a30 0x7ffb341212d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 --2- 172.21.15.32:0/1550696508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb340a1340 0x7ffb34121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 >> 172.21.15.32:0/1550696508 conn(0x7ffb340a6550 msgr2=0x7ffb340a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 shutdown_connections 2023-09-30T22:25:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.593+0000 7ffb39fd7700 1 -- 172.21.15.32:0/1550696508 wait complete. 2023-09-30T22:25:49.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.626+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f25280a17c0 con 0x7f252809e850 2023-09-30T22:25:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.627+0000 7f250f7fe700 1 -- 172.21.15.32:0/3417295654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2518021030 con 0x7f252809e850 2023-09-30T22:25:49.628 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 msgr2=0x7f25281c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 0x7f25281c8bd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f251800ebe0 tx=0x7f251800eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 shutdown_connections 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f252809f550 0x7f2528122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f252809e850 0x7f25281c8bd0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 --2- 172.21.15.32:0/3417295654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f252809f160 0x7f25281c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 >> 172.21.15.32:0/3417295654 conn(0x7f25280a3f70 msgr2=0x7f2528070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 shutdown_connections 2023-09-30T22:25:49.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:49.629+0000 7f252d722700 1 -- 172.21.15.32:0/3417295654 wait complete. 2023-09-30T22:25:49.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:50.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:50.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 Processor -- start 2023-09-30T22:25:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 -- start start 2023-09-30T22:25:50.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8540a09c0 0x7ff8540a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a12d0 0x7ff854134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85409ea10 con 0x7ff85409d5c0 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85409e210 con 0x7ff8540a09c0 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8594da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540a26f0 con 0x7ff8540a12d0 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff84a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff85409d5c0 0x7ff854133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52266/0 (socket says 172.21.15.32:52266) 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff84a7fc700 1 -- 172.21.15.32:0/3332442961 learned_addr learned my addr 172.21.15.32:0/3332442961 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff8537fe700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a12d0 0x7ff854134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.181+0000 7ff852ffd700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8540a09c0 0x7ff8540a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8527fc700 1 -- 172.21.15.32:0/3332442961 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395860856 0 0) 0x7ff85409ea10 con 0x7ff85409d5c0 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8537fe700 1 -- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8540a09c0 msgr2=0x7ff8540a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8537fe700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8540a09c0 0x7ff8540a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8537fe700 1 -- 172.21.15.32:0/3332442961 >> v1:172.21.15.32:6789/0 conn(0x7ff85409d5c0 legacy=0x7ff854133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8537fe700 1 -- 172.21.15.32:0/3332442961 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85409db70 con 0x7ff8540a12d0 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff852ffd700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8540a09c0 0x7ff8540a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8537fe700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a12d0 0x7ff854134030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009dc0 tx=0x7ff84c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20c4fd00469ea12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8527fc700 1 -- 172.21.15.32:0/3332442961 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00e070 con 0x7ff8540a12d0 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8527fc700 1 -- 172.21.15.32:0/3332442961 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c00bb20 con 0x7ff8540a12d0 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.182+0000 7ff8527fc700 1 -- 172.21.15.32:0/3332442961 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c012430 con 0x7ff8540a12d0 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 -- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a12d0 msgr2=0x7ff854134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a12d0 0x7ff854134030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009dc0 tx=0x7ff84c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 -- 172.21.15.32:0/3332442961 shutdown_connections 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a12d0 0x7ff854134030 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 --2- 172.21.15.32:0/3332442961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8540a09c0 0x7ff8540a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 -- 172.21.15.32:0/3332442961 >> 172.21.15.32:0/3332442961 conn(0x7ff8540a2fe0 msgr2=0x7ff8540a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 -- 172.21.15.32:0/3332442961 shutdown_connections 2023-09-30T22:25:50.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.183+0000 7ff8594da700 1 -- 172.21.15.32:0/3332442961 wait complete. 2023-09-30T22:25:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.184+0000 7ff8594da700 1 Processor -- start 2023-09-30T22:25:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.184+0000 7ff8594da700 1 -- start start 2023-09-30T22:25:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8594da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff85409d5c0 0x7ff854123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8594da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a09c0 0x7ff8541235c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8594da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 0x7ff854123b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8594da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff854126b10 con 0x7ff8540a12d0 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8594da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff854125ab0 con 0x7ff85409d5c0 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8594da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541262a0 con 0x7ff8540a09c0 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 0x7ff854123b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff852ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff85409d5c0 0x7ff854123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 0x7ff854123b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35388/0 (socket says 172.21.15.32:35388) 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 -- 172.21.15.32:0/2096286738 learned_addr learned my addr 172.21.15.32:0/2096286738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:50.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff84a7fc700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a09c0 0x7ff8541235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 -- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a09c0 msgr2=0x7ff8541235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a09c0 0x7ff8541235c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 -- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff85409d5c0 msgr2=0x7ff854123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff85409d5c0 0x7ff854123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff8537fe700 1 -- 172.21.15.32:0/2096286738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440094a0 con 0x7ff8540a12d0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.185+0000 7ff852ffd700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff85409d5c0 0x7ff854123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.186+0000 7ff8537fe700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 0x7ff854123b00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b370 tx=0x7ff84c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.186+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c012430 con 0x7ff8540a12d0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.186+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c020070 con 0x7ff8540a12d0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.186+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c004280 con 0x7ff8540a12d0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.186+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c009a70 con 0x7ff8540a12d0 2023-09-30T22:25:50.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.186+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff854124c00 con 0x7ff8540a12d0 2023-09-30T22:25:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.187+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff84c0043e0 con 0x7ff8540a12d0 2023-09-30T22:25:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.187+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85411ff60 con 0x7ff8540a12d0 2023-09-30T22:25:50.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.187+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff84c022070 con 0x7ff8540a12d0 2023-09-30T22:25:50.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.192+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff84c01b400 con 0x7ff8540a12d0 2023-09-30T22:25:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.219+0000 7f6f7425d700 1 Processor -- start 2023-09-30T22:25:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.219+0000 7f6f7425d700 1 -- start start 2023-09-30T22:25:50.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f7425d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c0a1510 0x7f6f6c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f7425d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c139250 0x7f6f6c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f7425d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c09edd0 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f7425d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0a18e0 con 0x7f6f6c0a1510 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f7425d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c09e1f0 con 0x7f6f6c139250 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f717f8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c0a1510 0x7f6f6c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f71ff9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f6c0a0a30 0x7f6f6c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52278/0 (socket says 172.21.15.32:52278) 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f71ff9700 1 -- 172.21.15.32:0/407192973 learned_addr learned my addr 172.21.15.32:0/407192973 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f727fa700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c139250 0x7f6f6c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f727fa700 1 -- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c0a1510 msgr2=0x7f6f6c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:50.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f70ff7700 1 -- 172.21.15.32:0/407192973 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827882784 0 0) 0x7f6f6c09edd0 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f727fa700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c0a1510 0x7f6f6c138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f727fa700 1 -- 172.21.15.32:0/407192973 >> v1:172.21.15.32:6789/0 conn(0x7f6f6c0a0a30 legacy=0x7f6f6c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f727fa700 1 -- 172.21.15.32:0/407192973 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c09dc50 con 0x7f6f6c139250 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.220+0000 7f6f717f8700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c0a1510 0x7f6f6c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.221+0000 7f6f727fa700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c139250 0x7f6f6c13b630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009dc0 tx=0x7f6f6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18658c138408d2d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.221+0000 7f6f70ff7700 1 -- 172.21.15.32:0/407192973 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400e070 con 0x7f6f6c139250 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.221+0000 7f6f70ff7700 1 -- 172.21.15.32:0/407192973 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400bb20 con 0x7f6f6c139250 2023-09-30T22:25:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.221+0000 7f6f70ff7700 1 -- 172.21.15.32:0/407192973 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64012430 con 0x7f6f6c139250 2023-09-30T22:25:50.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.221+0000 7f6f7425d700 1 -- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c139250 msgr2=0x7f6f6c13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c139250 0x7f6f6c13b630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009dc0 tx=0x7f6f6400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 -- 172.21.15.32:0/407192973 shutdown_connections 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c139250 0x7f6f6c13b630 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 --2- 172.21.15.32:0/407192973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c0a1510 0x7f6f6c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 -- 172.21.15.32:0/407192973 >> 172.21.15.32:0/407192973 conn(0x7f6f6c0a6550 msgr2=0x7f6f6c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 -- 172.21.15.32:0/407192973 shutdown_connections 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.222+0000 7f6f7425d700 1 -- 172.21.15.32:0/407192973 wait complete. 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 Processor -- start 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 -- start start 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 0x7f6f6c133680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f6c0a1510 0x7f6f6c133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c139250 0x7f6f6c11e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c11fd50 con 0x7f6f6c0a1510 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c09c4f0 con 0x7f6f6c139250 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f7425d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0a0220 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f71ff9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 0x7f6f6c133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f71ff9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 0x7f6f6c133680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:47128/0 (socket says 172.21.15.32:47128) 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.223+0000 7f6f71ff9700 1 -- 172.21.15.32:0/2154280967 learned_addr learned my addr 172.21.15.32:0/2154280967 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f717f8700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f6c0a1510 0x7f6f6c133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f727fa700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c139250 0x7f6f6c11e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f71ff9700 1 -- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c139250 msgr2=0x7f6f6c11e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f71ff9700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c139250 0x7f6f6c11e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f71ff9700 1 -- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f6c0a1510 msgr2=0x7f6f6c133bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f71ff9700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f6c0a1510 0x7f6f6c133bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f71ff9700 1 -- 172.21.15.32:0/2154280967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0094a0 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f71ff9700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 0x7f6f6c133680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800c520 tx=0x7f6f6800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004c80 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f68004e00 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f64009a70 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68007310 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.224+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c1220d0 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.225+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f68007470 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.225+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f6803f2f0 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.225+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6c121d70 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.230+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f6c121d70 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.406+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8540747f0 con 0x7ff8540a12d0 2023-09-30T22:25:50.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.406+0000 7ff84bfff700 1 -- 172.21.15.32:0/2096286738 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff84c02c030 con 0x7ff8540a12d0 2023-09-30T22:25:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 msgr2=0x7ff854123b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 0x7ff854123b00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b370 tx=0x7ff84c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 shutdown_connections 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8540a12d0 0x7ff854123b00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff85409d5c0 0x7ff854123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 --2- 172.21.15.32:0/2096286738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8540a09c0 0x7ff8541235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 >> 172.21.15.32:0/2096286738 conn(0x7ff8540a2fe0 msgr2=0x7ff85409b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 shutdown_connections 2023-09-30T22:25:50.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.409+0000 7ff8594da700 1 -- 172.21.15.32:0/2096286738 wait complete. 2023-09-30T22:25:50.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.436+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6f6c121f30 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.437+0000 7f6f627fc700 1 -- 172.21.15.32:0/2154280967 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f6c121f30 con 0x7f6f6c0a0a30 2023-09-30T22:25:50.439 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 msgr2=0x7f6f6c133680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 0x7f6f6c133680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800c520 tx=0x7f6f6800c830 comp rx=0 tx=0).stop 2023-09-30T22:25:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 shutdown_connections 2023-09-30T22:25:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f6c139250 0x7f6f6c11e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f6c0a0a30 0x7f6f6c133680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 --2- 172.21.15.32:0/2154280967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f6c0a1510 0x7f6f6c133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 >> 172.21.15.32:0/2154280967 conn(0x7f6f6c0a6550 msgr2=0x7f6f6c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 shutdown_connections 2023-09-30T22:25:50.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.439+0000 7f6f7425d700 1 -- 172.21.15.32:0/2154280967 wait complete. 2023-09-30T22:25:50.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:50.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:50.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.756+0000 7fd4ad0b7700 1 Processor -- start 2023-09-30T22:25:50.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.756+0000 7fd4ad0b7700 1 -- start start 2023-09-30T22:25:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4ad0b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4a806b250 0x7fd4a8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4ad0b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 0x7fd4a8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4ad0b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a809ece0 con 0x7fd4a806a770 2023-09-30T22:25:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4ad0b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a809e760 con 0x7fd4a806b250 2023-09-30T22:25:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4ad0b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a809e040 con 0x7fd4a8136d80 2023-09-30T22:25:50.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4a659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4a806b250 0x7fd4a8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4a759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 0x7fd4a8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4a6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4a806a770 0x7fd4a806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45254/0 (socket says 172.21.15.32:45254) 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4a759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 0x7fd4a8139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33626/0 (socket says 172.21.15.32:33626) 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4a6d9d700 1 -- 172.21.15.32:0/1091635925 learned_addr learned my addr 172.21.15.32:0/1091635925 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.757+0000 7fd4a659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4a806b250 0x7fd4a8136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53072/0 (socket says 172.21.15.32:53072) 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a659c700 1 -- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 msgr2=0x7fd4a8139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a659c700 1 --2- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 0x7fd4a8139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a659c700 1 -- 172.21.15.32:0/1091635925 >> v1:172.21.15.32:6789/0 conn(0x7fd4a806a770 legacy=0x7fd4a806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a659c700 1 -- 172.21.15.32:0/1091635925 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a809f630 con 0x7fd4a806b250 2023-09-30T22:25:50.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a5d9b700 1 -- 172.21.15.32:0/1091635925 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522913791 0 0) 0x7fd4a809ece0 con 0x7fd4a806a770 2023-09-30T22:25:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a759e700 1 --2- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 0x7fd4a8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a659c700 1 --2- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806b250 0x7fd4a8136840 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd490009840 tx=0x7fd490009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f0394402bfab12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a5d9b700 1 -- 172.21.15.32:0/1091635925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490011830 con 0x7fd4a806b250 2023-09-30T22:25:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a5d9b700 1 -- 172.21.15.32:0/1091635925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd490011990 con 0x7fd4a806b250 2023-09-30T22:25:50.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.758+0000 7fd4a5d9b700 1 -- 172.21.15.32:0/1091635925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490011d20 con 0x7fd4a806b250 2023-09-30T22:25:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806b250 msgr2=0x7fd4a8136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806b250 0x7fd4a8136840 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd490009840 tx=0x7fd490009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/1091635925 shutdown_connections 2023-09-30T22:25:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a8136d80 0x7fd4a8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/1091635925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806b250 0x7fd4a8136840 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/1091635925 >> 172.21.15.32:0/1091635925 conn(0x7fd4a8066f50 msgr2=0x7fd4a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/1091635925 shutdown_connections 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.759+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/1091635925 wait complete. 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.760+0000 7fd4ad0b7700 1 Processor -- start 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.760+0000 7fd4ad0b7700 1 -- start start 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4ad0b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806a770 0x7fd4a81c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4ad0b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 0x7fd4a81c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4ad0b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4a8136d80 0x7fd4a811c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4ad0b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8071340 con 0x7fd4a806a770 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4ad0b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a811fa00 con 0x7fd4a8136d80 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4ad0b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a81bbf00 con 0x7fd4a806b250 2023-09-30T22:25:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 0x7fd4a81c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806a770 0x7fd4a81c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 0x7fd4a81c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33636/0 (socket says 172.21.15.32:33636) 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 -- 172.21.15.32:0/158254099 learned_addr learned my addr 172.21.15.32:0/158254099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a759e700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4a8136d80 0x7fd4a811c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 -- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4a8136d80 msgr2=0x7fd4a811c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4a8136d80 0x7fd4a811c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 -- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806a770 msgr2=0x7fd4a81c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806a770 0x7fd4a81c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a659c700 1 -- 172.21.15.32:0/158254099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00094a0 con 0x7fd4a806b250 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.761+0000 7fd4a6d9d700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806a770 0x7fd4a81c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.762+0000 7fd4a659c700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 0x7fd4a81c6820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd490009fd0 tx=0x7fd490004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.762+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900045d0 con 0x7fd4a806b250 2023-09-30T22:25:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.762+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49001f070 con 0x7fd4a806b250 2023-09-30T22:25:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.762+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900094a0 con 0x7fd4a806b250 2023-09-30T22:25:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.762+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900038e0 con 0x7fd4a806b250 2023-09-30T22:25:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.762+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a8071e60 con 0x7fd4a806b250 2023-09-30T22:25:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.763+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a81c6dc0 con 0x7fd4a806b250 2023-09-30T22:25:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.763+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd490003a40 con 0x7fd4a806b250 2023-09-30T22:25:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.763+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd490021070 con 0x7fd4a806b250 2023-09-30T22:25:50.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.768+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4900253e0 con 0x7fd4a806b250 2023-09-30T22:25:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.794+0000 7fc4a9aa3700 1 Processor -- start 2023-09-30T22:25:50.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.794+0000 7fc4a9aa3700 1 -- start start 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a9aa3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a409d630 0x7fc4a40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a9aa3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a4138d80 0x7fc4a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a9aa3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a40a2a70 con 0x7fc4a40a0a30 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a9aa3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40a01d0 con 0x7fc4a409d630 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a9aa3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a409e2b0 con 0x7fc4a4138d80 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc4a40a0a30 0x7fc4a40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45262/0 (socket says 172.21.15.32:45262) 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a37fe700 1 -- 172.21.15.32:0/4119498037 learned_addr learned my addr 172.21.15.32:0/4119498037 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:50.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a3fff700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a4138d80 0x7fc4a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a2ffd700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a409d630 0x7fc4a40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a27fc700 1 -- 172.21.15.32:0/4119498037 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657026219 0 0) 0x7fc4a40a2a70 con 0x7fc4a40a0a30 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.795+0000 7fc4a27fc700 1 -- 172.21.15.32:0/4119498037 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4900035b0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a2ffd700 1 -- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a4138d80 msgr2=0x7fc4a413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a2ffd700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a4138d80 0x7fc4a413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a2ffd700 1 -- 172.21.15.32:0/4119498037 >> v1:172.21.15.32:6789/0 conn(0x7fc4a40a0a30 legacy=0x7fc4a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a2ffd700 1 -- 172.21.15.32:0/4119498037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a409dc50 con 0x7fc4a409d630 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a3fff700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a4138d80 0x7fc4a413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a2ffd700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a409d630 0x7fc4a40a1510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009870 tx=0x7fc48c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1649f553c9eb3ecc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a27fc700 1 -- 172.21.15.32:0/4119498037 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c011830 con 0x7fc4a409d630 2023-09-30T22:25:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a27fc700 1 -- 172.21.15.32:0/4119498037 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c011990 con 0x7fc4a409d630 2023-09-30T22:25:50.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.796+0000 7fc4a27fc700 1 -- 172.21.15.32:0/4119498037 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c011d20 con 0x7fc4a409d630 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a409d630 msgr2=0x7fc4a40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a409d630 0x7fc4a40a1510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009870 tx=0x7fc48c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/4119498037 shutdown_connections 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a4138d80 0x7fc4a413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/4119498037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a409d630 0x7fc4a40a1510 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/4119498037 >> 172.21.15.32:0/4119498037 conn(0x7fc4a40a3050 msgr2=0x7fc4a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/4119498037 shutdown_connections 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.797+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/4119498037 wait complete. 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 Processor -- start 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 -- start start 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a409d630 0x7fc4a4133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a40a0a30 0x7fc4a4133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 0x7fc4a41343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40a4bb0 con 0x7fc4a409d630 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40a0410 con 0x7fc4a4138d80 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a9aa3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a41366b0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.798+0000 7fc4a3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 0x7fc4a41343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 0x7fc4a41343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48558/0 (socket says 172.21.15.32:48558) 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a40a0a30 0x7fc4a4133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a3fff700 1 -- 172.21.15.32:0/419000005 learned_addr learned my addr 172.21.15.32:0/419000005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a37fe700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a409d630 0x7fc4a4133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 -- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 msgr2=0x7fc4a41343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 0x7fc4a41343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 -- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a409d630 msgr2=0x7fc4a4133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a409d630 0x7fc4a4133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 -- 172.21.15.32:0/419000005 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0094a0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a3fff700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 0x7fc4a41343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a2ffd700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a40a0a30 0x7fc4a4133e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc48c016000 tx=0x7fc48c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c004450 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c01f070 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c0094a0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0038e0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.799+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a41373a0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.800+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc48c003a40 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.800+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a40747f0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.801+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc48c021070 con 0x7fc4a40a0a30 2023-09-30T22:25:50.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.805+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc48c0253e0 con 0x7fc4a40a0a30 2023-09-30T22:25:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.974+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd4a8074ea0 con 0x7fd4a806b250 2023-09-30T22:25:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.975+0000 7fd49f7fe700 1 -- 172.21.15.32:0/158254099 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd490015020 con 0x7fd4a806b250 2023-09-30T22:25:50.976 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 msgr2=0x7fd4a81c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 0x7fd4a81c6820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd490009fd0 tx=0x7fd490004750 comp rx=0 tx=0).stop 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 shutdown_connections 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4a8136d80 0x7fd4a811c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4a806a770 0x7fd4a81c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 --2- 172.21.15.32:0/158254099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd4a806b250 0x7fd4a81c6820 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 >> 172.21.15.32:0/158254099 conn(0x7fd4a8066f50 msgr2=0x7fd4a806ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 shutdown_connections 2023-09-30T22:25:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:50.977+0000 7fd4ad0b7700 1 -- 172.21.15.32:0/158254099 wait complete. 2023-09-30T22:25:50.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:51.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.011+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc4a4072980 con 0x7fc4a40a0a30 2023-09-30T22:25:51.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.011+0000 7fc49bfff700 1 -- 172.21.15.32:0/419000005 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc48c015030 con 0x7fc4a40a0a30 2023-09-30T22:25:51.013 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a40a0a30 msgr2=0x7fc4a4133e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a40a0a30 0x7fc4a4133e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc48c016000 tx=0x7fc48c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 shutdown_connections 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4a4138d80 0x7fc4a41343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a409d630 0x7fc4a4133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 --2- 172.21.15.32:0/419000005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4a40a0a30 0x7fc4a4133e70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.013+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 >> 172.21.15.32:0/419000005 conn(0x7fc4a40a3050 msgr2=0x7fc4a41397d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.014+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 shutdown_connections 2023-09-30T22:25:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.014+0000 7fc4a9aa3700 1 -- 172.21.15.32:0/419000005 wait complete. 2023-09-30T22:25:51.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:51.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:51.455 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:51.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 Processor -- start 2023-09-30T22:25:51.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 -- start start 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10ec139250 0x7f10ec13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ec09edd0 con 0x7f10ec0a0a30 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ec0a18e0 con 0x7f10ec139250 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f2dd3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec09e1f0 con 0x7f10ec0a1510 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10f1370700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10ec139250 0x7f10ec13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33674/0 (socket says 172.21.15.32:33674) 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.573+0000 7f10ebfff700 1 -- 172.21.15.32:0/3998853931 learned_addr learned my addr 172.21.15.32:0/3998853931 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10ebfff700 1 -- 172.21.15.32:0/3998853931 >> v2:172.21.15.32:3300/0 conn(0x7f10ec139250 msgr2=0x7f10ec13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10ebfff700 1 --2- 172.21.15.32:0/3998853931 >> v2:172.21.15.32:3300/0 conn(0x7f10ec139250 0x7f10ec13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10ebfff700 1 -- 172.21.15.32:0/3998853931 >> v1:172.21.15.32:6789/0 conn(0x7f10ec0a0a30 legacy=0x7f10ec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10ebfff700 1 -- 172.21.15.32:0/3998853931 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ec09dc50 con 0x7f10ec0a1510 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10f1370700 1 --2- 172.21.15.32:0/3998853931 >> v2:172.21.15.32:3300/0 conn(0x7f10ec139250 0x7f10ec13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10ebfff700 1 --2- 172.21.15.32:0/3998853931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec138d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009920 tx=0x7f10e0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc198c97f2818a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10eb7fe700 1 -- 172.21.15.32:0/3998853931 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011830 con 0x7f10ec0a1510 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10eb7fe700 1 -- 172.21.15.32:0/3998853931 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0011990 con 0x7f10ec0a1510 2023-09-30T22:25:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.574+0000 7f10eb7fe700 1 -- 172.21.15.32:0/3998853931 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011d20 con 0x7f10ec0a1510 2023-09-30T22:25:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 -- 172.21.15.32:0/3998853931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 msgr2=0x7f10ec138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/3998853931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec138d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009920 tx=0x7f10e0009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 -- 172.21.15.32:0/3998853931 shutdown_connections 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/3998853931 >> v2:172.21.15.32:3300/0 conn(0x7f10ec139250 0x7f10ec13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/3998853931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec138d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 -- 172.21.15.32:0/3998853931 >> 172.21.15.32:0/3998853931 conn(0x7f10ec0a6550 msgr2=0x7f10ec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 -- 172.21.15.32:0/3998853931 shutdown_connections 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.575+0000 7f10f2dd3700 1 -- 172.21.15.32:0/3998853931 wait complete. 2023-09-30T22:25:51.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.576+0000 7f10f2dd3700 1 Processor -- start 2023-09-30T22:25:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.576+0000 7f10f2dd3700 1 -- start start 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.577+0000 7f10f2dd3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10ec0a0a30 0x7f10ec1c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.577+0000 7f10f2dd3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec1c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.577+0000 7f10f2dd3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10ec139250 0x7f10ec122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.577+0000 7f10f2dd3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec125eb0 con 0x7f10ec0a0a30 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.577+0000 7f10f2dd3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec138dd0 con 0x7f10ec139250 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.577+0000 7f10f2dd3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1be3d0 con 0x7f10ec0a1510 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec1c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10f1370700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10ec139250 0x7f10ec122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10f0b6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10ec0a0a30 0x7f10ec1c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec1c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33680/0 (socket says 172.21.15.32:33680) 2023-09-30T22:25:51.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 -- 172.21.15.32:0/1310073135 learned_addr learned my addr 172.21.15.32:0/1310073135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 -- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10ec139250 msgr2=0x7f10ec122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10ec139250 0x7f10ec122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 -- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10ec0a0a30 msgr2=0x7f10ec1c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10ec0a0a30 0x7f10ec1c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 -- 172.21.15.32:0/1310073135 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009a70 con 0x7f10ec0a1510 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10f1370700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10ec139250 0x7f10ec122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10f0b6f700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10ec0a0a30 0x7f10ec1c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10ebfff700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec1c8cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10e00098f0 tx=0x7f10e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00042c0 con 0x7f10ec0a1510 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e001f070 con 0x7f10ec0a1510 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00036a0 con 0x7f10ec0a1510 2023-09-30T22:25:51.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e0009550 con 0x7f10ec0a1510 2023-09-30T22:25:51.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.578+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec126450 con 0x7f10ec0a1510 2023-09-30T22:25:51.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.579+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10e0021030 con 0x7f10ec0a1510 2023-09-30T22:25:51.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.579+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ec0a0280 con 0x7f10ec0a1510 2023-09-30T22:25:51.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.580+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f10e00408b0 con 0x7f10ec0a1510 2023-09-30T22:25:51.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.584+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10e0019770 con 0x7f10ec0a1510 2023-09-30T22:25:51.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.613+0000 7f9b9f478700 1 Processor -- start 2023-09-30T22:25:51.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.613+0000 7f9b9f478700 1 -- start start 2023-09-30T22:25:51.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.613+0000 7f9b9f478700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809d6f0 0x7f9b9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.613+0000 7f9b9f478700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809e000 0x7f9b98130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9f478700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9809f130 con 0x7f9b9809e410 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9f478700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9809f520 con 0x7f9b9809e000 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9f478700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9809e930 con 0x7f9b9809d6f0 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9ca13700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809e000 0x7f9b98130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9da15700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b9809e410 0x7f9b98130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45286/0 (socket says 172.21.15.32:45286) 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9da15700 1 -- 172.21.15.32:0/1642637578 learned_addr learned my addr 172.21.15.32:0/1642637578 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9d214700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809d6f0 0x7f9b9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b8ffff700 1 -- 172.21.15.32:0/1642637578 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920366253 0 0) 0x7f9b9809f130 con 0x7f9b9809e410 2023-09-30T22:25:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b8ffff700 1 -- 172.21.15.32:0/1642637578 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b840057c0 con 0x7f9b9809e410 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9ca13700 1 -- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809d6f0 msgr2=0x7f9b9809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9ca13700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809d6f0 0x7f9b9809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9ca13700 1 -- 172.21.15.32:0/1642637578 >> v1:172.21.15.32:6789/0 conn(0x7f9b9809e410 legacy=0x7f9b98130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9ca13700 1 -- 172.21.15.32:0/1642637578 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9809b2d0 con 0x7f9b9809e000 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9d214700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809d6f0 0x7f9b9809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.614+0000 7f9b9ca13700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809e000 0x7f9b98130660 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b88009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdad21886c3a2ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b8ffff700 1 -- 172.21.15.32:0/1642637578 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88011830 con 0x7f9b9809e000 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b8ffff700 1 -- 172.21.15.32:0/1642637578 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88011990 con 0x7f9b9809e000 2023-09-30T22:25:51.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b8ffff700 1 -- 172.21.15.32:0/1642637578 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88011d20 con 0x7f9b9809e000 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 -- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809e000 msgr2=0x7f9b98130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809e000 0x7f9b98130660 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b88009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 -- 172.21.15.32:0/1642637578 shutdown_connections 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809d6f0 0x7f9b9809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1642637578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809e000 0x7f9b98130660 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 -- 172.21.15.32:0/1642637578 >> 172.21.15.32:0/1642637578 conn(0x7f9b98066f50 msgr2=0x7f9b9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 -- 172.21.15.32:0/1642637578 shutdown_connections 2023-09-30T22:25:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.615+0000 7f9b9f478700 1 -- 172.21.15.32:0/1642637578 wait complete. 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.616+0000 7f9b9f478700 1 Processor -- start 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.616+0000 7f9b9f478700 1 -- start start 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9f478700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 0x7f9b98122210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9f478700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9809e000 0x7f9b98122750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9f478700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809e410 0x7f9b98122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9f478700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98123e00 con 0x7f9b9809e000 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9f478700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98125770 con 0x7f9b9809d6f0 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9f478700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980a3bb0 con 0x7f9b9809e410 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9d214700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 0x7f9b98122210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9d214700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 0x7f9b98122210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48592/0 (socket says 172.21.15.32:48592) 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9ca13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9809e000 0x7f9b98122750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9d214700 1 -- 172.21.15.32:0/1184315417 learned_addr learned my addr 172.21.15.32:0/1184315417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9da15700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809e410 0x7f9b98122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9ca13700 1 -- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809e410 msgr2=0x7f9b98122c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9ca13700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809e410 0x7f9b98122c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9ca13700 1 -- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 msgr2=0x7f9b98122210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9ca13700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 0x7f9b98122210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9ca13700 1 -- 172.21.15.32:0/1184315417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940094a0 con 0x7f9b9809e000 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.617+0000 7f9b9d214700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 0x7f9b98122210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.618+0000 7f9b9ca13700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9809e000 0x7f9b98122750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9b980a3e20 tx=0x7f9b88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.618+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88004450 con 0x7f9b9809e000 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.618+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8801f070 con 0x7f9b9809e000 2023-09-30T22:25:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.618+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880038e0 con 0x7f9b9809e000 2023-09-30T22:25:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.618+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b880094a0 con 0x7f9b9809e000 2023-09-30T22:25:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.618+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9809c700 con 0x7f9b9809e000 2023-09-30T22:25:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.619+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b88003a40 con 0x7f9b9809e000 2023-09-30T22:25:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.619+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b88021070 con 0x7f9b9809e000 2023-09-30T22:25:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.619+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b981254a0 con 0x7f9b9809e000 2023-09-30T22:25:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.624+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b8802b080 con 0x7f9b9809e000 2023-09-30T22:25:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.791+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f10ec074ea0 con 0x7f10ec0a1510 2023-09-30T22:25:51.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.791+0000 7f10e97fa700 1 -- 172.21.15.32:0/1310073135 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10e0015030 con 0x7f10ec0a1510 2023-09-30T22:25:51.792 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 msgr2=0x7f10ec1c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec1c8cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10e00098f0 tx=0x7f10e0004750 comp rx=0 tx=0).stop 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 shutdown_connections 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10ec139250 0x7f10ec122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10ec0a0a30 0x7f10ec1c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 --2- 172.21.15.32:0/1310073135 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10ec0a1510 0x7f10ec1c8cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 >> 172.21.15.32:0/1310073135 conn(0x7f10ec0a6550 msgr2=0x7f10ec0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 shutdown_connections 2023-09-30T22:25:51.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.794+0000 7f10f2dd3700 1 -- 172.21.15.32:0/1310073135 wait complete. 2023-09-30T22:25:51.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.831+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b98074ea0 con 0x7f9b9809e000 2023-09-30T22:25:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.831+0000 7f9b8dffb700 1 -- 172.21.15.32:0/1184315417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b88015020 con 0x7f9b9809e000 2023-09-30T22:25:51.833 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9809e000 msgr2=0x7f9b98122750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9809e000 0x7f9b98122750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9b980a3e20 tx=0x7f9b88004750 comp rx=0 tx=0).stop 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 shutdown_connections 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9809e410 0x7f9b98122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9809d6f0 0x7f9b98122210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 --2- 172.21.15.32:0/1184315417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9809e000 0x7f9b98122750 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:51.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 >> 172.21.15.32:0/1184315417 conn(0x7f9b98066f50 msgr2=0x7f9b98130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:51.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 shutdown_connections 2023-09-30T22:25:51.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:51.834+0000 7f9b9f478700 1 -- 172.21.15.32:0/1184315417 wait complete. 2023-09-30T22:25:51.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:51.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:52.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.139+0000 7fa884722700 1 Processor -- start 2023-09-30T22:25:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.139+0000 7fa884722700 1 -- start start 2023-09-30T22:25:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa884722700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c0a0a30 0x7fa87c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa884722700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c09d630 0x7fa87c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa884722700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa87c133930 0x7fa87c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa884722700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c0a2a70 con 0x7fa87c133930 2023-09-30T22:25:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa884722700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0a01d0 con 0x7fa87c09d630 2023-09-30T22:25:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa884722700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c09e2b0 con 0x7fa87c0a0a30 2023-09-30T22:25:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa87c133930 0x7fa87c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa87c133930 0x7fa87c133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53126/0 (socket says 172.21.15.32:53126) 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 -- 172.21.15.32:0/1815521637 learned_addr learned my addr 172.21.15.32:0/1815521637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa8824be700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c0a0a30 0x7fa87c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa881cbd700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c09d630 0x7fa87c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 -- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c0a0a30 msgr2=0x7fa87c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c0a0a30 0x7fa87c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 -- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c09d630 msgr2=0x7fa87c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c09d630 0x7fa87c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.140+0000 7fa882cbf700 1 -- 172.21.15.32:0/1815521637 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c09dc50 con 0x7fa87c133930 2023-09-30T22:25:52.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.141+0000 7fa882cbf700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c133930 0x7fa87c133dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa874009e50 tx=0x7fa87400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455b0803216b279b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.143+0000 7fa8814bc700 1 -- 172.21.15.32:0/1815521637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400e070 con 0x7fa87c133930 2023-09-30T22:25:52.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.143+0000 7fa8814bc700 1 -- 172.21.15.32:0/1815521637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87400b910 con 0x7fa87c133930 2023-09-30T22:25:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.143+0000 7fa8814bc700 1 -- 172.21.15.32:0/1815521637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400bcc0 con 0x7fa87c133930 2023-09-30T22:25:52.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.143+0000 7fa884722700 1 -- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c133930 msgr2=0x7fa87c133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.143+0000 7fa884722700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c133930 0x7fa87c133dd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa874009e50 tx=0x7fa87400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 -- 172.21.15.32:0/1815521637 shutdown_connections 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c133930 0x7fa87c133dd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c0a0a30 0x7fa87c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 --2- 172.21.15.32:0/1815521637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c09d630 0x7fa87c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 -- 172.21.15.32:0/1815521637 >> 172.21.15.32:0/1815521637 conn(0x7fa87c0a3050 msgr2=0x7fa87c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 -- 172.21.15.32:0/1815521637 shutdown_connections 2023-09-30T22:25:52.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.144+0000 7fa884722700 1 -- 172.21.15.32:0/1815521637 wait complete. 2023-09-30T22:25:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 Processor -- start 2023-09-30T22:25:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 -- start start 2023-09-30T22:25:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 0x7fa87c11f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 0x7fa87c11f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c133930 0x7fa87c1c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c120610 con 0x7fa87c09d630 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c121f80 con 0x7fa87c0a0a30 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa884722700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c122600 con 0x7fa87c133930 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa8824be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 0x7fa87c11f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa881cbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 0x7fa87c11f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa8824be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 0x7fa87c11f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53128/0 (socket says 172.21.15.32:53128) 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa8824be700 1 -- 172.21.15.32:0/4168211298 learned_addr learned my addr 172.21.15.32:0/4168211298 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa881cbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 0x7fa87c11f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48608/0 (socket says 172.21.15.32:48608) 2023-09-30T22:25:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.145+0000 7fa882cbf700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c133930 0x7fa87c1c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa8824be700 1 -- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c133930 msgr2=0x7fa87c1c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa8824be700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c133930 0x7fa87c1c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa8824be700 1 -- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 msgr2=0x7fa87c11f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa8824be700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 0x7fa87c11f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa8824be700 1 -- 172.21.15.32:0/4168211298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874009b30 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa881cbd700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 0x7fa87c11f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa8824be700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 0x7fa87c11f400 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9b0 tx=0x7fa87800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878009710 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87800ee50 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0a6e60 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800ca50 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.146+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c11fe80 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.147+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa87800cbb0 con 0x7fa87c09d630 2023-09-30T22:25:52.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.147+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c0747f0 con 0x7fa87c09d630 2023-09-30T22:25:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.148+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa87803f9f0 con 0x7fa87c09d630 2023-09-30T22:25:52.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.152+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa87801b3b0 con 0x7fa87c09d630 2023-09-30T22:25:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 Processor -- start 2023-09-30T22:25:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 -- start start 2023-09-30T22:25:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 0x7f6fb80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fb8133930 0x7f6fb8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb80a2a70 con 0x7f6fb8133930 2023-09-30T22:25:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80a01d0 con 0x7f6fb80a0a30 2023-09-30T22:25:52.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.180+0000 7f6fbfd48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb809e2b0 con 0x7f6fb809d630 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbdae4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 0x7f6fb80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbd2e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbdae4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 0x7f6fb80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48612/0 (socket says 172.21.15.32:48612) 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fb8133930 0x7f6fb8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbdae4700 1 -- 172.21.15.32:0/3884789216 learned_addr learned my addr 172.21.15.32:0/3884789216 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 -- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 msgr2=0x7f6fb80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 -- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 msgr2=0x7f6fb80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 0x7f6fb80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 -- 172.21.15.32:0/3884789216 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb809dc50 con 0x7f6fb8133930 2023-09-30T22:25:52.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbdae4700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 0x7f6fb80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbd2e3700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbe2e5700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb8133930 0x7f6fb8133dd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009d90 tx=0x7f6fb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa9eb6e2898fd0a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbcae2700 1 -- 172.21.15.32:0/3884789216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000e070 con 0x7f6fb8133930 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbcae2700 1 -- 172.21.15.32:0/3884789216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb000bb20 con 0x7f6fb8133930 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.181+0000 7f6fbcae2700 1 -- 172.21.15.32:0/3884789216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0012430 con 0x7f6fb8133930 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 -- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb8133930 msgr2=0x7f6fb8133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb8133930 0x7f6fb8133dd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009d90 tx=0x7f6fb000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 -- 172.21.15.32:0/3884789216 shutdown_connections 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb80a0a30 0x7f6fb80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb8133930 0x7f6fb8133dd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/3884789216 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 -- 172.21.15.32:0/3884789216 >> 172.21.15.32:0/3884789216 conn(0x7f6fb80a3050 msgr2=0x7f6fb80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 -- 172.21.15.32:0/3884789216 shutdown_connections 2023-09-30T22:25:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.182+0000 7f6fbfd48700 1 -- 172.21.15.32:0/3884789216 wait complete. 2023-09-30T22:25:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.183+0000 7f6fbfd48700 1 Processor -- start 2023-09-30T22:25:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.183+0000 7f6fbfd48700 1 -- start start 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbfd48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb811e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbfd48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb80a0a30 0x7f6fb811ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbfd48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 0x7f6fb811f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbfd48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8120bb0 con 0x7f6fb80a0a30 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbfd48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8121ff0 con 0x7f6fb8133930 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbfd48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb81204d0 con 0x7f6fb809d630 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbe2e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 0x7f6fb811f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbdae4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb811e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb80a0a30 0x7f6fb811ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbe2e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 0x7f6fb811f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48616/0 (socket says 172.21.15.32:48616) 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbdae4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb811e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33738/0 (socket says 172.21.15.32:33738) 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbe2e5700 1 -- 172.21.15.32:0/2562053737 learned_addr learned my addr 172.21.15.32:0/2562053737 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 -- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 msgr2=0x7f6fb811e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb811e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 -- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 msgr2=0x7f6fb811f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 0x7f6fb811f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 -- 172.21.15.32:0/2562053737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80094a0 con 0x7f6fb80a0a30 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbe2e5700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 0x7f6fb811f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbdae4700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb811e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.184+0000 7f6fbd2e3700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb80a0a30 0x7f6fb811ee20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b3c0 tx=0x7f6fa800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.185+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8019070 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.185+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa800be40 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.185+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa801d630 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.185+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4009550 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.185+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb0009a70 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.185+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fa801d790 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.186+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb80747f0 con 0x7f6fb80a0a30 2023-09-30T22:25:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.186+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6fa80472e0 con 0x7f6fb80a0a30 2023-09-30T22:25:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.191+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fa8022070 con 0x7f6fb80a0a30 2023-09-30T22:25:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.359+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa87c1334b0 con 0x7fa87c09d630 2023-09-30T22:25:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.359+0000 7fa872ffd700 1 -- 172.21.15.32:0/4168211298 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa878021030 con 0x7fa87c09d630 2023-09-30T22:25:52.361 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 msgr2=0x7fa87c11f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 0x7fa87c11f400 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e9b0 tx=0x7fa87800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 shutdown_connections 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa87c133930 0x7fa87c1c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa87c09d630 0x7fa87c11f400 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 --2- 172.21.15.32:0/4168211298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa87c0a0a30 0x7fa87c11f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 >> 172.21.15.32:0/4168211298 conn(0x7fa87c0a3050 msgr2=0x7fa87c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 shutdown_connections 2023-09-30T22:25:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.362+0000 7fa884722700 1 -- 172.21.15.32:0/4168211298 wait complete. 2023-09-30T22:25:52.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:52.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.397+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6fb8072980 con 0x7f6fb80a0a30 2023-09-30T22:25:52.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.397+0000 7f6fae7fc700 1 -- 172.21.15.32:0/2562053737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fa801b900 con 0x7f6fb80a0a30 2023-09-30T22:25:52.398 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb80a0a30 msgr2=0x7f6fb811ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb80a0a30 0x7f6fb811ee20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b3c0 tx=0x7f6fa800b6d0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 shutdown_connections 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fb8133930 0x7f6fb811f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fb809d630 0x7f6fb811e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 --2- 172.21.15.32:0/2562053737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fb80a0a30 0x7f6fb811ee20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 >> 172.21.15.32:0/2562053737 conn(0x7f6fb80a3050 msgr2=0x7f6fb809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 shutdown_connections 2023-09-30T22:25:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.400+0000 7f6fbfd48700 1 -- 172.21.15.32:0/2562053737 wait complete. 2023-09-30T22:25:52.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:52.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:52.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:52.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 Processor -- start 2023-09-30T22:25:52.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 -- start start 2023-09-30T22:25:52.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a0a30 0x7fa0280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 0x7fa028133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa028133fa0 0x7fa028134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02809edd0 con 0x7fa028133fa0 2023-09-30T22:25:52.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02809e1f0 con 0x7fa0280a1340 2023-09-30T22:25:52.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02eedc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa02809dc50 con 0x7fa0280a0a30 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa027fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 0x7fa028133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02d479700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa028133fa0 0x7fa028134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa027fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 0x7fa028133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48618/0 (socket says 172.21.15.32:48618) 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa027fff700 1 -- 172.21.15.32:0/3859691849 learned_addr learned my addr 172.21.15.32:0/3859691849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.957+0000 7fa02cc78700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a0a30 0x7fa0280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa02d479700 1 -- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a0a30 msgr2=0x7fa0280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa02d479700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a0a30 0x7fa0280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa02d479700 1 -- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 msgr2=0x7fa028133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa02d479700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 0x7fa028133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa02d479700 1 -- 172.21.15.32:0/3859691849 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280a1fe0 con 0x7fa028133fa0 2023-09-30T22:25:52.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa027fff700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 0x7fa028133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa02d479700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028134410 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa020009d90 tx=0x7fa02000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2601b781dfcf34 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa0277fe700 1 -- 172.21.15.32:0/3859691849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000e070 con 0x7fa028133fa0 2023-09-30T22:25:52.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa0277fe700 1 -- 172.21.15.32:0/3859691849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000bb20 con 0x7fa028133fa0 2023-09-30T22:25:52.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.958+0000 7fa0277fe700 1 -- 172.21.15.32:0/3859691849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020012430 con 0x7fa028133fa0 2023-09-30T22:25:52.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 -- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 msgr2=0x7fa028134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028134410 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa020009d90 tx=0x7fa02000b330 comp rx=0 tx=0).stop 2023-09-30T22:25:52.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 -- 172.21.15.32:0/3859691849 shutdown_connections 2023-09-30T22:25:52.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028134410 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a0a30 0x7fa0280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 --2- 172.21.15.32:0/3859691849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a1340 0x7fa028133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 -- 172.21.15.32:0/3859691849 >> 172.21.15.32:0/3859691849 conn(0x7fa0280a6550 msgr2=0x7fa0280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 -- 172.21.15.32:0/3859691849 shutdown_connections 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.959+0000 7fa02eedc700 1 -- 172.21.15.32:0/3859691849 wait complete. 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.960+0000 7fa02eedc700 1 Processor -- start 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.960+0000 7fa02eedc700 1 -- start start 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02eedc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a0a30 0x7fa0281200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02eedc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 0x7fa028120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02eedc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02eedc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028121d80 con 0x7fa028133fa0 2023-09-30T22:25:52.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02eedc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa028123910 con 0x7fa0280a0a30 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02eedc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa028122f30 con 0x7fa0280a1340 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa027fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 0x7fa028120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa027fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 0x7fa028120610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33758/0 (socket says 172.21.15.32:33758) 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa027fff700 1 -- 172.21.15.32:0/1724662816 learned_addr learned my addr 172.21.15.32:0/1724662816 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02cc78700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a0a30 0x7fa0281200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 -- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 msgr2=0x7fa028120610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 0x7fa028120610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 -- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a0a30 msgr2=0x7fa0281200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a0a30 0x7fa0281200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 -- 172.21.15.32:0/1724662816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c009550 con 0x7fa028133fa0 2023-09-30T22:25:52.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa027fff700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 0x7fa028120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.961+0000 7fa02d479700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028120b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa020016000 tx=0x7fa020004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.962+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020004800 con 0x7fa028133fa0 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.962+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020020070 con 0x7fa028133fa0 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.962+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa020009a70 con 0x7fa028133fa0 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.962+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0200276c0 con 0x7fa028133fa0 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.962+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0280a0220 con 0x7fa028133fa0 2023-09-30T22:25:52.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.963+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0281235b0 con 0x7fa028133fa0 2023-09-30T22:25:52.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.963+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa020012430 con 0x7fa028133fa0 2023-09-30T22:25:52.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.963+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa020022070 con 0x7fa028133fa0 2023-09-30T22:25:52.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.968+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa020012e40 con 0x7fa028133fa0 2023-09-30T22:25:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.996+0000 7f8a7606c700 1 Processor -- start 2023-09-30T22:25:52.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.996+0000 7f8a7606c700 1 -- start start 2023-09-30T22:25:52.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a7606c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a701317f0 0x7f8a70131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a7606c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 0x7f8a7006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a7606c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a7006b100 0x7f8a70138500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a7606c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7009d760 con 0x7f8a701317f0 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a7606c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70132100 con 0x7f8a7006a770 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a7606c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701313a0 con 0x7f8a7006b100 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a7006b100 0x7f8a70138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 0x7f8a7006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 0x7f8a7006abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48632/0 (socket says 172.21.15.32:48632) 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a7006b100 0x7f8a70138500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33764/0 (socket says 172.21.15.32:33764) 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 -- 172.21.15.32:0/3885197206 learned_addr learned my addr 172.21.15.32:0/3885197206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:52.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6f7fe700 1 --2- 172.21.15.32:0/3885197206 >> v2:172.21.15.32:3300/0 conn(0x7f8a701317f0 0x7f8a70131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 -- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a7006b100 msgr2=0x7f8a70138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 --2- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a7006b100 0x7f8a70138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 -- 172.21.15.32:0/3885197206 >> v2:172.21.15.32:3300/0 conn(0x7f8a701317f0 msgr2=0x7f8a70131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 --2- 172.21.15.32:0/3885197206 >> v2:172.21.15.32:3300/0 conn(0x7f8a701317f0 0x7f8a70131bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6effd700 1 -- 172.21.15.32:0/3885197206 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7009e880 con 0x7f8a7006a770 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.997+0000 7f8a6f7fe700 1 --2- 172.21.15.32:0/3885197206 >> v2:172.21.15.32:3300/0 conn(0x7f8a701317f0 0x7f8a70131bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a6effd700 1 --2- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 0x7f8a7006abc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a640098f0 tx=0x7f8a64009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b229fbc37bfa94f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a6e7fc700 1 -- 172.21.15.32:0/3885197206 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64011830 con 0x7f8a7006a770 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a6e7fc700 1 -- 172.21.15.32:0/3885197206 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a64011990 con 0x7f8a7006a770 2023-09-30T22:25:53.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a6e7fc700 1 -- 172.21.15.32:0/3885197206 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64011d20 con 0x7f8a7006a770 2023-09-30T22:25:53.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 -- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 msgr2=0x7f8a7006abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 --2- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 0x7f8a7006abc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a640098f0 tx=0x7f8a64009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 -- 172.21.15.32:0/3885197206 shutdown_connections 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 --2- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a7006b100 0x7f8a70138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 --2- 172.21.15.32:0/3885197206 >> v2:172.21.15.32:3300/0 conn(0x7f8a701317f0 0x7f8a70131bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 --2- 172.21.15.32:0/3885197206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006a770 0x7f8a7006abc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 -- 172.21.15.32:0/3885197206 >> 172.21.15.32:0/3885197206 conn(0x7f8a70099c20 msgr2=0x7f8a700a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 -- 172.21.15.32:0/3885197206 shutdown_connections 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.998+0000 7f8a7606c700 1 -- 172.21.15.32:0/3885197206 wait complete. 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:52.999+0000 7f8a7606c700 1 Processor -- start 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 -- start start 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7006a770 0x7f8a70070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006b100 0x7f8a70071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 0x7f8a700716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700a0db0 con 0x7f8a7006a770 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701195a0 con 0x7f8a7006b100 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a7606c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7011a970 con 0x7f8a701317f0 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 0x7f8a700716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a6f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7006a770 0x7f8a70070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 0x7f8a700716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33774/0 (socket says 172.21.15.32:33774) 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a6ffff700 1 -- 172.21.15.32:0/1306639847 learned_addr learned my addr 172.21.15.32:0/1306639847 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.000+0000 7f8a6effd700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006b100 0x7f8a70071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6ffff700 1 -- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006b100 msgr2=0x7f8a70071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6ffff700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006b100 0x7f8a70071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6ffff700 1 -- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7006a770 msgr2=0x7f8a70070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6ffff700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7006a770 0x7f8a70070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6ffff700 1 -- 172.21.15.32:0/1306639847 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60009a70 con 0x7f8a701317f0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6f7fe700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7006a770 0x7f8a70070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a6ffff700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 0x7f8a700716c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e960 tx=0x7f8a6800ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a680096d0 con 0x7f8a701317f0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68013680 con 0x7f8a701317f0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a64009550 con 0x7f8a701317f0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68013a10 con 0x7f8a701317f0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.001+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7012b8a0 con 0x7f8a701317f0 2023-09-30T22:25:53.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.002+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a68013b70 con 0x7f8a701317f0 2023-09-30T22:25:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.002+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7004f2a0 con 0x7f8a701317f0 2023-09-30T22:25:53.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.002+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a6803f9d0 con 0x7f8a701317f0 2023-09-30T22:25:53.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.007+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a6801adc0 con 0x7f8a701317f0 2023-09-30T22:25:53.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.176+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa028074ea0 con 0x7fa028133fa0 2023-09-30T22:25:53.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.177+0000 7fa0257fa700 1 -- 172.21.15.32:0/1724662816 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa020035090 con 0x7fa028133fa0 2023-09-30T22:25:53.178 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 msgr2=0x7fa028120b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028120b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa020016000 tx=0x7fa020004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 shutdown_connections 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa028133fa0 0x7fa028120b50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0280a0a30 0x7fa0281200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 --2- 172.21.15.32:0/1724662816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0280a1340 0x7fa028120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 >> 172.21.15.32:0/1724662816 conn(0x7fa0280a6550 msgr2=0x7fa0280a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 shutdown_connections 2023-09-30T22:25:53.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.179+0000 7fa02eedc700 1 -- 172.21.15.32:0/1724662816 wait complete. 2023-09-30T22:25:53.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:53.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.215+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a7009eaa0 con 0x7f8a701317f0 2023-09-30T22:25:53.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.216+0000 7f8a57fff700 1 -- 172.21.15.32:0/1306639847 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a68021070 con 0x7f8a701317f0 2023-09-30T22:25:53.217 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:53.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 msgr2=0x7f8a700716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 0x7f8a700716c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e960 tx=0x7f8a6800ec70 comp rx=0 tx=0).stop 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 shutdown_connections 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a701317f0 0x7f8a700716c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a7006a770 0x7f8a70070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 --2- 172.21.15.32:0/1306639847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a7006b100 0x7f8a70071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 >> 172.21.15.32:0/1306639847 conn(0x7f8a70099c20 msgr2=0x7f8a7012ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 shutdown_connections 2023-09-30T22:25:53.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.218+0000 7f8a7606c700 1 -- 172.21.15.32:0/1306639847 wait complete. 2023-09-30T22:25:53.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:53.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:53.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:53.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.522+0000 7f1bec402700 1 Processor -- start 2023-09-30T22:25:53.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.522+0000 7f1bec402700 1 -- start start 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bec402700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 0x7f1be40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bec402700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 0x7f1be4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bec402700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be409edd0 con 0x7f1be4133fa0 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bec402700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be409e1f0 con 0x7f1be40a1340 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bec402700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1be409dc50 con 0x7f1be40a0a30 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 0x7f1be40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1be999d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 0x7f1be4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 0x7f1be40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33786/0 (socket says 172.21.15.32:33786) 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1be999d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 0x7f1be4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53190/0 (socket says 172.21.15.32:53190) 2023-09-30T22:25:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 -- 172.21.15.32:0/3420794207 learned_addr learned my addr 172.21.15.32:0/3420794207 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 -- 172.21.15.32:0/3420794207 >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 msgr2=0x7f1be4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 --2- 172.21.15.32:0/3420794207 >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 0x7f1be4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 -- 172.21.15.32:0/3420794207 >> v1:172.21.15.32:6789/0 conn(0x7f1be4133fa0 legacy=0x7f1be413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1bea19e700 1 -- 172.21.15.32:0/3420794207 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be40a1fe0 con 0x7f1be40a0a30 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.523+0000 7f1be999d700 1 --2- 172.21.15.32:0/3420794207 >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 0x7f1be4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.524+0000 7f1bea19e700 1 --2- 172.21.15.32:0/3420794207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 0x7f1be40a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098f0 tx=0x7f1be0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a77b2de6721ec9e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.524+0000 7f1be919c700 1 -- 172.21.15.32:0/3420794207 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0011830 con 0x7f1be40a0a30 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.524+0000 7f1be919c700 1 -- 172.21.15.32:0/3420794207 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be0011990 con 0x7f1be40a0a30 2023-09-30T22:25:53.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.524+0000 7f1be919c700 1 -- 172.21.15.32:0/3420794207 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0011d20 con 0x7f1be40a0a30 2023-09-30T22:25:53.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.524+0000 7f1bec402700 1 -- 172.21.15.32:0/3420794207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 msgr2=0x7f1be40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.524+0000 7f1bec402700 1 --2- 172.21.15.32:0/3420794207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 0x7f1be40a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098f0 tx=0x7f1be0009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.525+0000 7f1bec402700 1 -- 172.21.15.32:0/3420794207 shutdown_connections 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.525+0000 7f1bec402700 1 --2- 172.21.15.32:0/3420794207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a0a30 0x7f1be40a0e00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.525+0000 7f1bec402700 1 --2- 172.21.15.32:0/3420794207 >> v2:172.21.15.32:3300/0 conn(0x7f1be40a1340 0x7f1be4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.525+0000 7f1bec402700 1 -- 172.21.15.32:0/3420794207 >> 172.21.15.32:0/3420794207 conn(0x7f1be40a6550 msgr2=0x7f1be40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.525+0000 7f1bec402700 1 -- 172.21.15.32:0/3420794207 shutdown_connections 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.525+0000 7f1bec402700 1 -- 172.21.15.32:0/3420794207 wait complete. 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 Processor -- start 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 -- start start 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40a0a30 0x7f1be411e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 0x7f1be411ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1be4133fa0 0x7f1be411f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4121f50 con 0x7f1be40a0a30 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4120430 con 0x7f1be4133fa0 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bec402700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1be40a37e0 con 0x7f1be40a1340 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1be999d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 0x7f1be411ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bea99f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1be4133fa0 0x7f1be411f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1bea99f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1be4133fa0 0x7f1be411f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48644/0 (socket says 172.21.15.32:48644) 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.526+0000 7f1be999d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 0x7f1be411ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33790/0 (socket says 172.21.15.32:33790) 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bea99f700 1 -- 172.21.15.32:0/1356092223 learned_addr learned my addr 172.21.15.32:0/1356092223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bea19e700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40a0a30 0x7f1be411e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1be999d700 1 -- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1be4133fa0 msgr2=0x7f1be411f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1be999d700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1be4133fa0 0x7f1be411f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1be999d700 1 -- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40a0a30 msgr2=0x7f1be411e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1be999d700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40a0a30 0x7f1be411e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1be999d700 1 -- 172.21.15.32:0/1356092223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd40094a0 con 0x7f1be40a1340 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1be999d700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 0x7f1be411ece0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400e9e0 tx=0x7f1bd400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400ee80 con 0x7f1be40a1340 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd400c850 con 0x7f1be40a1340 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0009550 con 0x7f1be40a1340 2023-09-30T22:25:53.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400cbe0 con 0x7f1be40a1340 2023-09-30T22:25:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.527+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be40a0270 con 0x7f1be40a1340 2023-09-30T22:25:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.528+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1bd400cd40 con 0x7f1be40a1340 2023-09-30T22:25:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.528+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1bd403f720 con 0x7f1be40a1340 2023-09-30T22:25:53.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.528+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be4122a80 con 0x7f1be40a1340 2023-09-30T22:25:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.533+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1bd40232f0 con 0x7f1be40a1340 2023-09-30T22:25:53.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.561+0000 7fea71bb8700 1 Processor -- start 2023-09-30T22:25:53.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.561+0000 7fea71bb8700 1 -- start start 2023-09-30T22:25:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea71bb8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea6c0a0a30 0x7fea6c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea71bb8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea71bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 0x7fea6c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea71bb8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c0a2a70 con 0x7fea6c0a0a30 2023-09-30T22:25:53.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea71bb8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0a01d0 con 0x7fea6c09d630 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea71bb8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c09e2b0 con 0x7fea6c133930 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 0x7fea6c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea6c0a0a30 0x7fea6c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 0x7fea6c133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33794/0 (socket says 172.21.15.32:33794) 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6bfff700 1 -- 172.21.15.32:0/3227508267 learned_addr learned my addr 172.21.15.32:0/3227508267 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 -- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 msgr2=0x7fea6c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 --2- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 0x7fea6c133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 -- 172.21.15.32:0/3227508267 >> v2:172.21.15.32:3300/0 conn(0x7fea6c0a0a30 msgr2=0x7fea6c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 --2- 172.21.15.32:0/3227508267 >> v2:172.21.15.32:3300/0 conn(0x7fea6c0a0a30 0x7fea6c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 -- 172.21.15.32:0/3227508267 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c09dc50 con 0x7fea6c09d630 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6bfff700 1 --2- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 0x7fea6c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/3227508267 >> v2:172.21.15.32:3300/0 conn(0x7fea6c0a0a30 0x7fea6c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.562+0000 7fea6affd700 1 --2- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c0a1340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea54009840 tx=0x7fea54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb001f78fe27c72a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea6a7fc700 1 -- 172.21.15.32:0/3227508267 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54011830 con 0x7fea6c09d630 2023-09-30T22:25:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea6a7fc700 1 -- 172.21.15.32:0/3227508267 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54011990 con 0x7fea6c09d630 2023-09-30T22:25:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea6a7fc700 1 -- 172.21.15.32:0/3227508267 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54011d20 con 0x7fea6c09d630 2023-09-30T22:25:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 -- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 msgr2=0x7fea6c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c0a1340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea54009840 tx=0x7fea54009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 -- 172.21.15.32:0/3227508267 shutdown_connections 2023-09-30T22:25:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c133930 0x7fea6c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3227508267 >> v2:172.21.15.32:3300/0 conn(0x7fea6c0a0a30 0x7fea6c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3227508267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c0a1340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 -- 172.21.15.32:0/3227508267 >> 172.21.15.32:0/3227508267 conn(0x7fea6c0a3050 msgr2=0x7fea6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 -- 172.21.15.32:0/3227508267 shutdown_connections 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.563+0000 7fea71bb8700 1 -- 172.21.15.32:0/3227508267 wait complete. 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.564+0000 7fea71bb8700 1 Processor -- start 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 -- start start 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 0x7fea6c1c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c133930 0x7fea6c1c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c121140 con 0x7fea6c133930 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0a40c0 con 0x7fea6c09d630 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea71bb8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c120570 con 0x7fea6c0a0a30 2023-09-30T22:25:53.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea6affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 0x7fea6c1c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c133930 0x7fea6c1c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea6affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 0x7fea6c1c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33806/0 (socket says 172.21.15.32:33806) 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea6affd700 1 -- 172.21.15.32:0/3880192590 learned_addr learned my addr 172.21.15.32:0/3880192590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.565+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6b7fe700 1 -- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 msgr2=0x7fea6c1c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 0x7fea6c1c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6b7fe700 1 -- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c133930 msgr2=0x7fea6c1c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c133930 0x7fea6c1c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6b7fe700 1 -- 172.21.15.32:0/3880192590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c009a70 con 0x7fea6c09d630 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6bfff700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c133930 0x7fea6c1c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6affd700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 0x7fea6c1c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea6b7fe700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c1c8090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00efc0 tx=0x7fea5c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:53.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00cd10 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c00f040 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c0146a0 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea64009550 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.566+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea540094a0 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.567+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea5c014800 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.567+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c121cb0 con 0x7fea6c09d630 2023-09-30T22:25:53.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.567+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea5c040270 con 0x7fea6c09d630 2023-09-30T22:25:53.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.572+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea5c01d400 con 0x7fea6c09d630 2023-09-30T22:25:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.742+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1be4074ea0 con 0x7f1be40a1340 2023-09-30T22:25:53.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.743+0000 7f1bdaffd700 1 -- 172.21.15.32:0/1356092223 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bd401bba0 con 0x7f1be40a1340 2023-09-30T22:25:53.743 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 msgr2=0x7f1be411ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 0x7f1be411ece0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400e9e0 tx=0x7f1bd400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 shutdown_connections 2023-09-30T22:25:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1be4133fa0 0x7f1be411f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1be40a0a30 0x7f1be411e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 --2- 172.21.15.32:0/1356092223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1be40a1340 0x7f1be411ece0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 >> 172.21.15.32:0/1356092223 conn(0x7f1be40a6550 msgr2=0x7f1be41b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 shutdown_connections 2023-09-30T22:25:53.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.745+0000 7f1bec402700 1 -- 172.21.15.32:0/1356092223 wait complete. 2023-09-30T22:25:53.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.780+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fea6c120cb0 con 0x7fea6c09d630 2023-09-30T22:25:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.781+0000 7fea63fff700 1 -- 172.21.15.32:0/3880192590 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea5c02b070 con 0x7fea6c09d630 2023-09-30T22:25:53.782 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 msgr2=0x7fea6c1c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c1c8090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00efc0 tx=0x7fea5c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 shutdown_connections 2023-09-30T22:25:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6c133930 0x7fea6c1c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea6c09d630 0x7fea6c1c8090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 --2- 172.21.15.32:0/3880192590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea6c0a0a30 0x7fea6c1c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 >> 172.21.15.32:0/3880192590 conn(0x7fea6c0a3050 msgr2=0x7fea6c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 shutdown_connections 2023-09-30T22:25:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:53.783+0000 7fea71bb8700 1 -- 172.21.15.32:0/3880192590 wait complete. 2023-09-30T22:25:53.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:54.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:54.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.343+0000 7f545bd28700 1 Processor -- start 2023-09-30T22:25:54.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.343+0000 7f545bd28700 1 -- start start 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545bd28700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 0x7f54540a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545bd28700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 0x7f545413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545bd28700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54540a2a70 con 0x7f54540a0a30 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545bd28700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540a01d0 con 0x7f545409d630 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545bd28700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f545409e2b0 con 0x7f5454138d80 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545a2c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 0x7f545413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f54592c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 0x7f54540a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545a2c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 0x7f545413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33808/0 (socket says 172.21.15.32:33808) 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f54592c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 0x7f54540a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48662/0 (socket says 172.21.15.32:48662) 2023-09-30T22:25:54.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545a2c5700 1 -- 172.21.15.32:0/4184981379 learned_addr learned my addr 172.21.15.32:0/4184981379 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545a2c5700 1 -- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 msgr2=0x7f54540a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f545a2c5700 1 --2- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 0x7f54540a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f545a2c5700 1 -- 172.21.15.32:0/4184981379 >> v1:172.21.15.32:6789/0 conn(0x7f54540a0a30 legacy=0x7f54540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.344+0000 7f5458ac2700 1 -- 172.21.15.32:0/4184981379 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704124976 0 0) 0x7f54540a2a70 con 0x7f54540a0a30 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f545a2c5700 1 -- 172.21.15.32:0/4184981379 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545409dc50 con 0x7f5454138d80 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f54592c3700 1 --2- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 0x7f54540a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f545a2c5700 1 --2- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 0x7f545413b160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f544c009e80 tx=0x7f544c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0415a5fa44a096 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f5458ac2700 1 -- 172.21.15.32:0/4184981379 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00e070 con 0x7f5454138d80 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f5458ac2700 1 -- 172.21.15.32:0/4184981379 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c00b910 con 0x7f5454138d80 2023-09-30T22:25:54.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.345+0000 7f5458ac2700 1 -- 172.21.15.32:0/4184981379 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c00bcc0 con 0x7f5454138d80 2023-09-30T22:25:54.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 -- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 msgr2=0x7f545413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 --2- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 0x7f545413b160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f544c009e80 tx=0x7f544c00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 -- 172.21.15.32:0/4184981379 shutdown_connections 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 --2- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5454138d80 0x7f545413b160 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 --2- 172.21.15.32:0/4184981379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f545409d630 0x7f54540a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 -- 172.21.15.32:0/4184981379 >> 172.21.15.32:0/4184981379 conn(0x7f54540a3050 msgr2=0x7f54540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 -- 172.21.15.32:0/4184981379 shutdown_connections 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.346+0000 7f545bd28700 1 -- 172.21.15.32:0/4184981379 wait complete. 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 Processor -- start 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 -- start start 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f545409d630 0x7f5454133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 0x7f5454133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5454138d80 0x7f54541341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5454134c40 con 0x7f5454138d80 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54541361b0 con 0x7f54540a0a30 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f545bd28700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5454136ed0 con 0x7f545409d630 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f54592c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 0x7f5454133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f54592c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 0x7f5454133cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48666/0 (socket says 172.21.15.32:48666) 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f54592c3700 1 -- 172.21.15.32:0/2560327930 learned_addr learned my addr 172.21.15.32:0/2560327930 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.347+0000 7f5459ac4700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f545409d630 0x7f5454133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f545a2c5700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5454138d80 0x7f54541341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f54592c3700 1 -- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f545409d630 msgr2=0x7f5454133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f54592c3700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f545409d630 0x7f5454133770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f54592c3700 1 -- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5454138d80 msgr2=0x7f54541341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f54592c3700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5454138d80 0x7f54541341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f54592c3700 1 -- 172.21.15.32:0/2560327930 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440094a0 con 0x7f54540a0a30 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f54592c3700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 0x7f5454133cb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f544400ebe0 tx=0x7f544400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400c850 con 0x7f54540a0a30 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544400c9b0 con 0x7f54540a0a30 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544c009b30 con 0x7f54540a0a30 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400cd40 con 0x7f54540a0a30 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.348+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54540a0410 con 0x7f54540a0a30 2023-09-30T22:25:54.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.349+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5444013470 con 0x7f54540a0a30 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.349+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f544403fbe0 con 0x7f54540a0a30 2023-09-30T22:25:54.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.349+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5454136d10 con 0x7f54540a0a30 2023-09-30T22:25:54.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.354+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5444013720 con 0x7f54540a0a30 2023-09-30T22:25:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.381+0000 7f442092e700 1 Processor -- start 2023-09-30T22:25:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.382+0000 7f442092e700 1 -- start start 2023-09-30T22:25:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f442092e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 0x7f441c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f442092e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c09d630 0x7f441c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f442092e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 0x7f441c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f442092e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c0a2a70 con 0x7f441c133930 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f442092e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0a01d0 con 0x7f441c0a0a30 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f442092e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f441c09e2b0 con 0x7f441c09d630 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 0x7f441c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 0x7f441c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 0x7f441c133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53242/0 (socket says 172.21.15.32:53242) 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441ad9d700 1 -- 172.21.15.32:0/663346948 learned_addr learned my addr 172.21.15.32:0/663346948 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c09d630 0x7f441c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 0x7f441c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48676/0 (socket says 172.21.15.32:48676) 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 -- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 msgr2=0x7f441c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 --2- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 0x7f441c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 -- 172.21.15.32:0/663346948 >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 msgr2=0x7f441c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 --2- 172.21.15.32:0/663346948 >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 0x7f441c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 -- 172.21.15.32:0/663346948 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c09dc50 con 0x7f441c09d630 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441ad9d700 1 --2- 172.21.15.32:0/663346948 >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 0x7f441c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f441a59c700 1 --2- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 0x7f441c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.383+0000 7f4419d9b700 1 --2- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c09d630 0x7f441c0a1340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4404009840 tx=0x7f4404009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93823c76c898c240 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f441959a700 1 -- 172.21.15.32:0/663346948 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404011830 con 0x7f441c09d630 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f441959a700 1 -- 172.21.15.32:0/663346948 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404011990 con 0x7f441c09d630 2023-09-30T22:25:54.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f441959a700 1 -- 172.21.15.32:0/663346948 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404011d20 con 0x7f441c09d630 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 -- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c09d630 msgr2=0x7f441c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 --2- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c09d630 0x7f441c0a1340 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4404009840 tx=0x7f4404009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 -- 172.21.15.32:0/663346948 shutdown_connections 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 --2- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c0a0a30 0x7f441c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 --2- 172.21.15.32:0/663346948 >> v2:172.21.15.32:3300/0 conn(0x7f441c133930 0x7f441c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 --2- 172.21.15.32:0/663346948 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c09d630 0x7f441c0a1340 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 -- 172.21.15.32:0/663346948 >> 172.21.15.32:0/663346948 conn(0x7f441c0a3050 msgr2=0x7f441c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 -- 172.21.15.32:0/663346948 shutdown_connections 2023-09-30T22:25:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.384+0000 7f442092e700 1 -- 172.21.15.32:0/663346948 wait complete. 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.386+0000 7f442092e700 1 Processor -- start 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.386+0000 7f442092e700 1 -- start start 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f442092e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 0x7f441c11f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f442092e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 0x7f441c11f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f442092e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c133930 0x7f441c1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f442092e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c122390 con 0x7f441c0a0a30 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f442092e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c122570 con 0x7f441c09d630 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f442092e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1333e0 con 0x7f441c133930 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 0x7f441c11f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f4419d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 0x7f441c11f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 0x7f441c11f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48682/0 (socket says 172.21.15.32:48682) 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441a59c700 1 -- 172.21.15.32:0/3323795230 learned_addr learned my addr 172.21.15.32:0/3323795230 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f4419d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 0x7f441c11f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53254/0 (socket says 172.21.15.32:53254) 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441ad9d700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c133930 0x7f441c1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441ad9d700 1 -- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 msgr2=0x7f441c11f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441ad9d700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 0x7f441c11f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441ad9d700 1 -- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 msgr2=0x7f441c11f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441ad9d700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 0x7f441c11f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441ad9d700 1 -- 172.21.15.32:0/3323795230 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c009a70 con 0x7f441c133930 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f441a59c700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 0x7f441c11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.387+0000 7f4419d9b700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 0x7f441c11f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.388+0000 7f441ad9d700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c133930 0x7f441c1c8100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f441400e930 tx=0x7f441400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.388+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414004020 con 0x7f441c133930 2023-09-30T22:25:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.388+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44140041a0 con 0x7f441c133930 2023-09-30T22:25:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.388+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4414009550 con 0x7f441c133930 2023-09-30T22:25:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.388+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414004550 con 0x7f441c133930 2023-09-30T22:25:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.388+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44040094a0 con 0x7f441c133930 2023-09-30T22:25:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.389+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44140036a0 con 0x7f441c133930 2023-09-30T22:25:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.389+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f441403f390 con 0x7f441c133930 2023-09-30T22:25:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.389+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c0a0410 con 0x7f441c133930 2023-09-30T22:25:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.394+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4414020070 con 0x7f441c133930 2023-09-30T22:25:54.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.563+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5454136b70 con 0x7f54540a0a30 2023-09-30T22:25:54.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.563+0000 7f544a7fc700 1 -- 172.21.15.32:0/2560327930 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5444021030 con 0x7f54540a0a30 2023-09-30T22:25:54.564 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:54.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 msgr2=0x7f5454133cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 0x7f5454133cb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f544400ebe0 tx=0x7f544400eef0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 shutdown_connections 2023-09-30T22:25:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5454138d80 0x7f54541341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f545409d630 0x7f5454133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 --2- 172.21.15.32:0/2560327930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54540a0a30 0x7f5454133cb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 >> 172.21.15.32:0/2560327930 conn(0x7f54540a3050 msgr2=0x7f54540a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 shutdown_connections 2023-09-30T22:25:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.565+0000 7f545bd28700 1 -- 172.21.15.32:0/2560327930 wait complete. 2023-09-30T22:25:54.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.597+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f441c0747f0 con 0x7f441c133930 2023-09-30T22:25:54.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.597+0000 7f4412ffd700 1 -- 172.21.15.32:0/3323795230 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4414017030 con 0x7f441c133930 2023-09-30T22:25:54.598 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c133930 msgr2=0x7f441c1c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c133930 0x7f441c1c8100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f441400e930 tx=0x7f441400ec40 comp rx=0 tx=0).stop 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 shutdown_connections 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f441c133930 0x7f441c1c8100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f441c09d630 0x7f441c11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 --2- 172.21.15.32:0/3323795230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f441c0a0a30 0x7f441c11f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 >> 172.21.15.32:0/3323795230 conn(0x7f441c0a3050 msgr2=0x7f441c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 shutdown_connections 2023-09-30T22:25:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.600+0000 7f442092e700 1 -- 172.21.15.32:0/3323795230 wait complete. 2023-09-30T22:25:54.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:54.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:54.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.907+0000 7f96e1ff3700 1 Processor -- start 2023-09-30T22:25:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.907+0000 7f96e1ff3700 1 -- start start 2023-09-30T22:25:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.907+0000 7f96e1ff3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96dc0a0a30 0x7f96dc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.907+0000 7f96e1ff3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.907+0000 7f96e1ff3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0a2a70 con 0x7f96dc133930 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96e1ff3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc0a01d0 con 0x7f96dc0a0a30 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96e1ff3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc09e2b0 con 0x7f96dc09d630 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96db7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96dc0a0a30 0x7f96dc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96dbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96dc133930 0x7f96dc133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45314/0 (socket says 172.21.15.32:45314) 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96dbfff700 1 -- 172.21.15.32:0/1113083618 learned_addr learned my addr 172.21.15.32:0/1113083618 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96daffd700 1 --2- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96db7fe700 1 -- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 msgr2=0x7f96dc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96db7fe700 1 --2- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96db7fe700 1 -- 172.21.15.32:0/1113083618 >> v1:172.21.15.32:6789/0 conn(0x7f96dc133930 legacy=0x7f96dc133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96db7fe700 1 -- 172.21.15.32:0/1113083618 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc09dc50 con 0x7f96dc0a0a30 2023-09-30T22:25:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96da7fc700 1 -- 172.21.15.32:0/1113083618 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303518730 0 0) 0x7f96dc0a2a70 con 0x7f96dc133930 2023-09-30T22:25:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96db7fe700 1 --2- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0a0a30 0x7f96dc0a0e00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009d90 tx=0x7f96cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705914ee8f77d3ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.908+0000 7f96da7fc700 1 -- 172.21.15.32:0/1113083618 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00e070 con 0x7f96dc0a0a30 2023-09-30T22:25:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96da7fc700 1 -- 172.21.15.32:0/1113083618 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00bb20 con 0x7f96dc0a0a30 2023-09-30T22:25:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96da7fc700 1 -- 172.21.15.32:0/1113083618 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc012430 con 0x7f96dc0a0a30 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 -- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0a0a30 msgr2=0x7f96dc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0a0a30 0x7f96dc0a0e00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009d90 tx=0x7f96cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 -- 172.21.15.32:0/1113083618 shutdown_connections 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc0a0a30 0x7f96dc0a0e00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/1113083618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 -- 172.21.15.32:0/1113083618 >> 172.21.15.32:0/1113083618 conn(0x7f96dc0a3050 msgr2=0x7f96dc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 -- 172.21.15.32:0/1113083618 shutdown_connections 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.909+0000 7f96e1ff3700 1 -- 172.21.15.32:0/1113083618 wait complete. 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.910+0000 7f96e1ff3700 1 Processor -- start 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 -- start start 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc0a0a30 0x7f96dc1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc133930 0x7f96dc1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc120620 con 0x7f96dc133930 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0a42c0 con 0x7f96dc0a0a30 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96e1ff3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0a0410 con 0x7f96dc09d630 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96db7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc133930 0x7f96dc1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96db7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc1c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33854/0 (socket says 172.21.15.32:33854) 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96db7fe700 1 -- 172.21.15.32:0/2786773738 learned_addr learned my addr 172.21.15.32:0/2786773738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.911+0000 7f96daffd700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc0a0a30 0x7f96dc1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96db7fe700 1 -- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc0a0a30 msgr2=0x7f96dc1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96db7fe700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc0a0a30 0x7f96dc1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96db7fe700 1 -- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc133930 msgr2=0x7f96dc1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96db7fe700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc133930 0x7f96dc1c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96db7fe700 1 -- 172.21.15.32:0/2786773738 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009a70 con 0x7f96dc09d630 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96dbfff700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc133930 0x7f96dc1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96db7fe700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc1c7e80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0a2c00 tx=0x7f96cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc012430 con 0x7f96dc09d630 2023-09-30T22:25:54.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc020070 con 0x7f96dc09d630 2023-09-30T22:25:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc120e10 con 0x7f96dc09d630 2023-09-30T22:25:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc004280 con 0x7f96dc09d630 2023-09-30T22:25:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.912+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc122cc0 con 0x7f96dc09d630 2023-09-30T22:25:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.913+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f96cc0043e0 con 0x7f96dc09d630 2023-09-30T22:25:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.913+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f96cc022070 con 0x7f96dc09d630 2023-09-30T22:25:54.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.913+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0747f0 con 0x7f96dc09d630 2023-09-30T22:25:54.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.922+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96cc01b400 con 0x7f96dc09d630 2023-09-30T22:25:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.949+0000 7fa1f6ad7700 1 Processor -- start 2023-09-30T22:25:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f6ad7700 1 -- start start 2023-09-30T22:25:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f6ad7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1f009e230 0x7fa1f0131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f6ad7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 0x7fa1f0139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f6ad7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f009dba0 con 0x7fa1f0131830 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f6ad7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f0132140 con 0x7fa1f009e230 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f6ad7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01313e0 con 0x7fa1f006ae80 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f5074700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 0x7fa1f0139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1effff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1f009e230 0x7fa1f0131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f5074700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 0x7fa1f0139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48700/0 (socket says 172.21.15.32:48700) 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.950+0000 7fa1f5074700 1 -- 172.21.15.32:0/1429855291 learned_addr learned my addr 172.21.15.32:0/1429855291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1f5074700 1 -- 172.21.15.32:0/1429855291 >> v2:172.21.15.32:3300/0 conn(0x7fa1f009e230 msgr2=0x7fa1f0131c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1f5074700 1 --2- 172.21.15.32:0/1429855291 >> v2:172.21.15.32:3300/0 conn(0x7fa1f009e230 0x7fa1f0131c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1f5074700 1 -- 172.21.15.32:0/1429855291 >> v1:172.21.15.32:6789/0 conn(0x7fa1f0131830 legacy=0x7fa1f006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1f5074700 1 -- 172.21.15.32:0/1429855291 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f009d6d0 con 0x7fa1f006ae80 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1effff700 1 --2- 172.21.15.32:0/1429855291 >> v2:172.21.15.32:3300/0 conn(0x7fa1f009e230 0x7fa1f0131c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1f5074700 1 --2- 172.21.15.32:0/1429855291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 0x7fa1f0139590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009e50 tx=0x7fa1e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d47a52be03b68f5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1ef7fe700 1 -- 172.21.15.32:0/1429855291 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800e070 con 0x7fa1f006ae80 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1ef7fe700 1 -- 172.21.15.32:0/1429855291 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800b910 con 0x7fa1f006ae80 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.951+0000 7fa1ef7fe700 1 -- 172.21.15.32:0/1429855291 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800bcc0 con 0x7fa1f006ae80 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/1429855291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 msgr2=0x7fa1f0139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/1429855291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 0x7fa1f0139590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009e50 tx=0x7fa1e800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/1429855291 shutdown_connections 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/1429855291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f006ae80 0x7fa1f0139590 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/1429855291 >> v2:172.21.15.32:3300/0 conn(0x7fa1f009e230 0x7fa1f0131c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/1429855291 >> 172.21.15.32:0/1429855291 conn(0x7fa1f0099c20 msgr2=0x7fa1f00a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/1429855291 shutdown_connections 2023-09-30T22:25:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.952+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/1429855291 wait complete. 2023-09-30T22:25:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 Processor -- start 2023-09-30T22:25:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 -- start start 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1f006ae80 0x7fa1f009a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 0x7fa1f009a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 0x7fa1f009ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f012b660 con 0x7fa1f009e230 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f011fe90 con 0x7fa1f0131830 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.953+0000 7fa1f6ad7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f009bf90 con 0x7fa1f006ae80 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 0x7fa1f009ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 0x7fa1f009a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 0x7fa1f009ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48716/0 (socket says 172.21.15.32:48716) 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 -- 172.21.15.32:0/723954006 learned_addr learned my addr 172.21.15.32:0/723954006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 0x7fa1f009a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53308/0 (socket says 172.21.15.32:53308) 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f4873700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1f006ae80 0x7fa1f009a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 -- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1f006ae80 msgr2=0x7fa1f009a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1f006ae80 0x7fa1f009a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 -- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 msgr2=0x7fa1f009a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 0x7fa1f009a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 -- 172.21.15.32:0/723954006 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80094a0 con 0x7fa1f0131830 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1effff700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 0x7fa1f009a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f5074700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 0x7fa1f009ad80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0131520 tx=0x7fa1e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:54.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80092e0 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8004040 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e8009b30 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80043f0 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.954+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f012b8f0 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.955+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa1e8004550 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.956+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa1e8040910 con 0x7fa1f0131830 2023-09-30T22:25:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.956+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f004f2a0 con 0x7fa1f0131830 2023-09-30T22:25:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:54.960+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa1e8011b80 con 0x7fa1f0131830 2023-09-30T22:25:55.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.130+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f96dc072980 con 0x7f96dc09d630 2023-09-30T22:25:55.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.131+0000 7f96d3fff700 1 -- 172.21.15.32:0/2786773738 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96cc02c030 con 0x7f96dc09d630 2023-09-30T22:25:55.132 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 msgr2=0x7f96dc1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc1c7e80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0a2c00 tx=0x7f96cc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 shutdown_connections 2023-09-30T22:25:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc133930 0x7f96dc1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09d630 0x7f96dc1c7e80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 --2- 172.21.15.32:0/2786773738 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc0a0a30 0x7f96dc1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 >> 172.21.15.32:0/2786773738 conn(0x7f96dc0a3050 msgr2=0x7f96dc1bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 shutdown_connections 2023-09-30T22:25:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.133+0000 7f96e1ff3700 1 -- 172.21.15.32:0/2786773738 wait complete. 2023-09-30T22:25:55.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:55.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.169+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa1f006c230 con 0x7fa1f0131830 2023-09-30T22:25:55.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.170+0000 7fa1ed7fa700 1 -- 172.21.15.32:0/723954006 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1e8011b80 con 0x7fa1f0131830 2023-09-30T22:25:55.171 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:55.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 msgr2=0x7fa1f009ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 0x7fa1f009ad80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0131520 tx=0x7fa1e8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 shutdown_connections 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1f0131830 0x7fa1f009ad80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1f006ae80 0x7fa1f009a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 --2- 172.21.15.32:0/723954006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1f009e230 0x7fa1f009a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 >> 172.21.15.32:0/723954006 conn(0x7fa1f0099c20 msgr2=0x7fa1f012dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 shutdown_connections 2023-09-30T22:25:55.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.172+0000 7fa1f6ad7700 1 -- 172.21.15.32:0/723954006 wait complete. 2023-09-30T22:25:55.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:55.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:55.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.728+0000 7f9e0a657700 1 Processor -- start 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.728+0000 7f9e0a657700 1 -- start start 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e0a657700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e040a0a30 0x7f9e040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e0a657700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e0a657700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e04133fa0 0x7f9e04134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e0a657700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0409edd0 con 0x7f9e040a0a30 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e0a657700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0409e1f0 con 0x7f9e040a1340 2023-09-30T22:25:55.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e0a657700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0409dc50 con 0x7f9e04133fa0 2023-09-30T22:25:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e08bf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e04133fa0 0x7f9e04134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48730/0 (socket says 172.21.15.32:48730) 2023-09-30T22:25:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e08bf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e04133fa0 0x7f9e04134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33884/0 (socket says 172.21.15.32:33884) 2023-09-30T22:25:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 -- 172.21.15.32:0/1751284397 learned_addr learned my addr 172.21.15.32:0/1751284397 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e03fff700 1 --2- 172.21.15.32:0/1751284397 >> v2:172.21.15.32:3300/0 conn(0x7f9e040a0a30 0x7f9e040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 -- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e04133fa0 msgr2=0x7f9e04134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 --2- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e04133fa0 0x7f9e04134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 -- 172.21.15.32:0/1751284397 >> v2:172.21.15.32:3300/0 conn(0x7f9e040a0a30 msgr2=0x7f9e040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 --2- 172.21.15.32:0/1751284397 >> v2:172.21.15.32:3300/0 conn(0x7f9e040a0a30 0x7f9e040a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 -- 172.21.15.32:0/1751284397 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040a1fe0 con 0x7f9e040a1340 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.729+0000 7f9e037fe700 1 --2- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04133a60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009920 tx=0x7f9df8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c20562e8dab2c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e02ffd700 1 -- 172.21.15.32:0/1751284397 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8011830 con 0x7f9e040a1340 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e02ffd700 1 -- 172.21.15.32:0/1751284397 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8011990 con 0x7f9e040a1340 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e02ffd700 1 -- 172.21.15.32:0/1751284397 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8011d20 con 0x7f9e040a1340 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 -- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 msgr2=0x7f9e04133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 --2- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04133a60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009920 tx=0x7f9df8009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 -- 172.21.15.32:0/1751284397 shutdown_connections 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 --2- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e04133fa0 0x7f9e04134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 --2- 172.21.15.32:0/1751284397 >> v2:172.21.15.32:3300/0 conn(0x7f9e040a0a30 0x7f9e040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 --2- 172.21.15.32:0/1751284397 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04133a60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 -- 172.21.15.32:0/1751284397 >> 172.21.15.32:0/1751284397 conn(0x7f9e040a6550 msgr2=0x7f9e040a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.730+0000 7f9e0a657700 1 -- 172.21.15.32:0/1751284397 shutdown_connections 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.731+0000 7f9e0a657700 1 -- 172.21.15.32:0/1751284397 wait complete. 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.731+0000 7f9e0a657700 1 Processor -- start 2023-09-30T22:25:55.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 -- start start 2023-09-30T22:25:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 0x7f9e041206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e04133fa0 0x7f9e04121130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e04121b60 con 0x7f9e04133fa0 2023-09-30T22:25:55.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041222c0 con 0x7f9e040a1340 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e0a657700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041228e0 con 0x7f9e040a0a30 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e03fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 0x7f9e041206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e08bf4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e04133fa0 0x7f9e04121130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e03fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 0x7f9e041206b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33896/0 (socket says 172.21.15.32:33896) 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.732+0000 7f9e03fff700 1 -- 172.21.15.32:0/3685596595 learned_addr learned my addr 172.21.15.32:0/3685596595 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e037fe700 1 -- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 msgr2=0x7f9e041206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e037fe700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 0x7f9e041206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e037fe700 1 -- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e04133fa0 msgr2=0x7f9e04121130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e037fe700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e04133fa0 0x7f9e04121130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e037fe700 1 -- 172.21.15.32:0/3685596595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8009550 con 0x7f9e040a1340 2023-09-30T22:25:55.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e03fff700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 0x7f9e041206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e08bf4700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e04133fa0 0x7f9e04121130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e037fe700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04120bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9df8016000 tx=0x7f9df8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8004450 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df80036a0 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e04123ef0 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80045f0 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.733+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e040a3610 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.734+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9df8003a50 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.734+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e040a0220 con 0x7f9e040a1340 2023-09-30T22:25:55.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.734+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9df8021030 con 0x7f9e040a1340 2023-09-30T22:25:55.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.739+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9df801a430 con 0x7f9e040a1340 2023-09-30T22:25:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.766+0000 7fa6fa30b700 1 Processor -- start 2023-09-30T22:25:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.766+0000 7fa6fa30b700 1 -- start start 2023-09-30T22:25:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6fa30b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4133420 0x7fa6f41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6fa30b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6f4133e30 0x7fa6f412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6fa30b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f406b990 con 0x7fa6f4134260 2023-09-30T22:25:55.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6fa30b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f40a0590 con 0x7fa6f4133e30 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6fa30b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f409fb80 con 0x7fa6f4133420 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6f4133e30 0x7fa6f412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4133420 0x7fa6f41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f88a8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6f4134260 0x7fa6f41304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45336/0 (socket says 172.21.15.32:45336) 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f88a8700 1 -- 172.21.15.32:0/3980587816 learned_addr learned my addr 172.21.15.32:0/3980587816 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f37fe700 1 -- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4133420 msgr2=0x7fa6f41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f37fe700 1 --2- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4133420 0x7fa6f41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f37fe700 1 -- 172.21.15.32:0/3980587816 >> v1:172.21.15.32:6789/0 conn(0x7fa6f4134260 legacy=0x7fa6f41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f2ffd700 1 -- 172.21.15.32:0/3980587816 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012469331 0 0) 0x7fa6f406b990 con 0x7fa6f4134260 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.767+0000 7fa6f37fe700 1 -- 172.21.15.32:0/3980587816 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40a0030 con 0x7fa6f4133e30 2023-09-30T22:25:55.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6f37fe700 1 --2- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f412ff90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009840 tx=0x7fa6e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8d88ae2dde40aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6f2ffd700 1 -- 172.21.15.32:0/3980587816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8011830 con 0x7fa6f4133e30 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6f2ffd700 1 -- 172.21.15.32:0/3980587816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8011990 con 0x7fa6f4133e30 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6f2ffd700 1 -- 172.21.15.32:0/3980587816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8011d20 con 0x7fa6f4133e30 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 -- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 msgr2=0x7fa6f412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f412ff90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009840 tx=0x7fa6e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 -- 172.21.15.32:0/3980587816 shutdown_connections 2023-09-30T22:25:55.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4133420 0x7fa6f41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/3980587816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f412ff90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 -- 172.21.15.32:0/3980587816 >> 172.21.15.32:0/3980587816 conn(0x7fa6f40a4210 msgr2=0x7fa6f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.768+0000 7fa6fa30b700 1 -- 172.21.15.32:0/3980587816 shutdown_connections 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.769+0000 7fa6fa30b700 1 -- 172.21.15.32:0/3980587816 wait complete. 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.769+0000 7fa6fa30b700 1 Processor -- start 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 -- start start 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6f4133420 0x7fa6f412d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4134260 0x7fa6f409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f409d850 con 0x7fa6f4133e30 2023-09-30T22:25:55.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f41205e0 con 0x7fa6f4134260 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6fa30b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f411fa80 con 0x7fa6f4133420 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6f3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6f4133420 0x7fa6f412d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6f37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6f3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6f4133420 0x7fa6f412d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33906/0 (socket says 172.21.15.32:33906) 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6f3fff700 1 -- 172.21.15.32:0/4225147607 learned_addr learned my addr 172.21.15.32:0/4225147607 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.770+0000 7fa6f88a8700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4134260 0x7fa6f409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f88a8700 1 -- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6f4133420 msgr2=0x7fa6f412d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f88a8700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6f4133420 0x7fa6f412d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f88a8700 1 -- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 msgr2=0x7fa6f409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f88a8700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f88a8700 1 -- 172.21.15.32:0/4225147607 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80094a0 con 0x7fa6f4134260 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f37fe700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f88a8700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4134260 0x7fa6f409c9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa6f409f3e0 tx=0x7fa6ec00cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00aa40 con 0x7fa6f4134260 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00abc0 con 0x7fa6f4134260 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f411f2f0 con 0x7fa6f4134260 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0070d0 con 0x7fa6f4134260 2023-09-30T22:25:55.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.771+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f409ddd0 con 0x7fa6f4134260 2023-09-30T22:25:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.772+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6ec007230 con 0x7fa6f4134260 2023-09-30T22:25:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.772+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa6ec03f4f0 con 0x7fa6f4134260 2023-09-30T22:25:55.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.772+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f4074ea0 con 0x7fa6f4134260 2023-09-30T22:25:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.777+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6ec018020 con 0x7fa6f4134260 2023-09-30T22:25:55.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.947+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9e04074ea0 con 0x7f9e040a1340 2023-09-30T22:25:55.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.947+0000 7f9e00ff9700 1 -- 172.21.15.32:0/3685596595 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9df802b070 con 0x7f9e040a1340 2023-09-30T22:25:55.949 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 msgr2=0x7f9e04120bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04120bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9df8016000 tx=0x7f9df8004750 comp rx=0 tx=0).stop 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 shutdown_connections 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e04133fa0 0x7f9e04121130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e040a0a30 0x7f9e041206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 --2- 172.21.15.32:0/3685596595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e040a1340 0x7f9e04120bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 >> 172.21.15.32:0/3685596595 conn(0x7f9e040a6550 msgr2=0x7f9e0409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:55.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 shutdown_connections 2023-09-30T22:25:55.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.950+0000 7f9e0a657700 1 -- 172.21.15.32:0/3685596595 wait complete. 2023-09-30T22:25:55.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.985+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa6f406b250 con 0x7fa6f4134260 2023-09-30T22:25:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.985+0000 7fa6f0ff9700 1 -- 172.21.15.32:0/4225147607 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6ec01a8b0 con 0x7fa6f4134260 2023-09-30T22:25:55.987 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4134260 msgr2=0x7fa6f409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4134260 0x7fa6f409c9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa6f409f3e0 tx=0x7fa6ec00cec0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 shutdown_connections 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6f4134260 0x7fa6f409c9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6f4133420 0x7fa6f412d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 --2- 172.21.15.32:0/4225147607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6f4133e30 0x7fa6f409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 >> 172.21.15.32:0/4225147607 conn(0x7fa6f40a4210 msgr2=0x7fa6f40a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 shutdown_connections 2023-09-30T22:25:55.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:55.988+0000 7fa6fa30b700 1 -- 172.21.15.32:0/4225147607 wait complete. 2023-09-30T22:25:56.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:56.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:56.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:56.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.298+0000 7f3bd29a4700 1 Processor -- start 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.298+0000 7f3bd29a4700 1 -- start start 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd29a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd29a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd29a4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0a2a70 con 0x7f3bcc09d630 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd29a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0a01d0 con 0x7f3bcc138d80 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd29a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc09e2b0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bcbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48778/0 (socket says 172.21.15.32:48778) 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 -- 172.21.15.32:0/2965683097 learned_addr learned my addr 172.21.15.32:0/2965683097 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bcbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33918/0 (socket says 172.21.15.32:33918) 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bcb7fe700 1 --1- 172.21.15.32:0/2965683097 >> v1:172.21.15.32:6789/0 conn(0x7f3bcc09d630 0x7f3bcc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45346/0 (socket says 172.21.15.32:45346) 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bcaffd700 1 -- 172.21.15.32:0/2965683097 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690062839 0 0) 0x7f3bcc0a2a70 con 0x7f3bcc09d630 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 -- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 msgr2=0x7f3bcc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 --2- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 -- 172.21.15.32:0/2965683097 >> v1:172.21.15.32:6789/0 conn(0x7f3bcc09d630 legacy=0x7f3bcc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bd0f41700 1 -- 172.21.15.32:0/2965683097 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc09dc50 con 0x7f3bcc138d80 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.299+0000 7f3bcbfff700 1 --2- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.300+0000 7f3bd0f41700 1 --2- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc13b160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009e80 tx=0x7f3bc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1679afe4442f1e42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:56.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.300+0000 7f3bcaffd700 1 -- 172.21.15.32:0/2965683097 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400e070 con 0x7f3bcc138d80 2023-09-30T22:25:56.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.300+0000 7f3bcaffd700 1 -- 172.21.15.32:0/2965683097 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc400b910 con 0x7f3bcc138d80 2023-09-30T22:25:56.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.300+0000 7f3bcaffd700 1 -- 172.21.15.32:0/2965683097 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400bcc0 con 0x7f3bcc138d80 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.300+0000 7f3bd29a4700 1 -- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 msgr2=0x7f3bcc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.300+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc13b160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009e80 tx=0x7f3bc400b330 comp rx=0 tx=0).stop 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.301+0000 7f3bd29a4700 1 -- 172.21.15.32:0/2965683097 shutdown_connections 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.301+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc13b160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.301+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/2965683097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.301+0000 7f3bd29a4700 1 -- 172.21.15.32:0/2965683097 >> 172.21.15.32:0/2965683097 conn(0x7f3bcc0a3050 msgr2=0x7f3bcc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.301+0000 7f3bd29a4700 1 -- 172.21.15.32:0/2965683097 shutdown_connections 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.301+0000 7f3bd29a4700 1 -- 172.21.15.32:0/2965683097 wait complete. 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 Processor -- start 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 -- start start 2023-09-30T22:25:56.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bcc09d630 0x7f3bcc133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc134bd0 con 0x7f3bcc09d630 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc136140 con 0x7f3bcc138d80 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd29a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc136e60 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33924/0 (socket says 172.21.15.32:33924) 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bcb7fe700 1 -- 172.21.15.32:0/3801546460 learned_addr learned my addr 172.21.15.32:0/3801546460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bcbfff700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bcc09d630 0x7f3bcc133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.302+0000 7f3bd0f41700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcb7fe700 1 -- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 msgr2=0x7f3bcc134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcb7fe700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcb7fe700 1 -- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bcc09d630 msgr2=0x7f3bcc133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcb7fe700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bcc09d630 0x7f3bcc133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcb7fe700 1 -- 172.21.15.32:0/3801546460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0094a0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcbfff700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bcc09d630 0x7f3bcc133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bcb7fe700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc133c40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c520 tx=0x7f3bc000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0004c80 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0004de0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc4009b30 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0007310 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.303+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1364f0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.304+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bc0007470 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.304+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc0747f0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.304+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3bc003f290 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.309+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bc00133c0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.337+0000 7f838c9b8700 1 Processor -- start 2023-09-30T22:25:56.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.337+0000 7f838c9b8700 1 -- start start 2023-09-30T22:25:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838c9b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83880a0a30 0x7f83880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838c9b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f838809d630 0x7f83880a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838c9b8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880a2a70 con 0x7f8388133930 2023-09-30T22:25:56.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838c9b8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83880a01d0 con 0x7f83880a0a30 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838c9b8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838809e2b0 con 0x7f838809d630 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83880a0a30 0x7f83880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f8385d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f838809d630 0x7f83880a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f8386d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8388133930 0x7f8388133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45356/0 (socket says 172.21.15.32:45356) 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f8386d9d700 1 -- 172.21.15.32:0/3210865758 learned_addr learned my addr 172.21.15.32:0/3210865758 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838659c700 1 -- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f838809d630 msgr2=0x7f83880a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838659c700 1 --2- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f838809d630 0x7f83880a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838659c700 1 -- 172.21.15.32:0/3210865758 >> v1:172.21.15.32:6789/0 conn(0x7f8388133930 legacy=0x7f8388133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838659c700 1 -- 172.21.15.32:0/3210865758 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838809dc50 con 0x7f83880a0a30 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f838559a700 1 -- 172.21.15.32:0/3210865758 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913207983 0 0) 0x7f83880a2a70 con 0x7f8388133930 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.338+0000 7f8385d9b700 1 --2- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f838809d630 0x7f83880a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:56.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838659c700 1 --2- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83880a0a30 0x7f83880a0e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8378009840 tx=0x7f8378009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a4c512a87abbdb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838559a700 1 -- 172.21.15.32:0/3210865758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378011830 con 0x7f83880a0a30 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838559a700 1 -- 172.21.15.32:0/3210865758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378011990 con 0x7f83880a0a30 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838559a700 1 -- 172.21.15.32:0/3210865758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378011d20 con 0x7f83880a0a30 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 -- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83880a0a30 msgr2=0x7f83880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 --2- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83880a0a30 0x7f83880a0e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8378009840 tx=0x7f8378009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 -- 172.21.15.32:0/3210865758 shutdown_connections 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 --2- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83880a0a30 0x7f83880a0e00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 --2- 172.21.15.32:0/3210865758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f838809d630 0x7f83880a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 -- 172.21.15.32:0/3210865758 >> 172.21.15.32:0/3210865758 conn(0x7f83880a3050 msgr2=0x7f83880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.339+0000 7f838c9b8700 1 -- 172.21.15.32:0/3210865758 shutdown_connections 2023-09-30T22:25:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.340+0000 7f838c9b8700 1 -- 172.21.15.32:0/3210865758 wait complete. 2023-09-30T22:25:56.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.341+0000 7f838c9b8700 1 Processor -- start 2023-09-30T22:25:56.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.341+0000 7f838c9b8700 1 -- start start 2023-09-30T22:25:56.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838c9b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838809d630 0x7f83881c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838c9b8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 0x7f83881c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838c9b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8388133930 0x7f83881c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838c9b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8388120620 con 0x7f838809d630 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838c9b8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880a42c0 con 0x7f8388133930 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838c9b8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83880a0410 con 0x7f83880a0a30 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f8385d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 0x7f83881c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f8385d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 0x7f83881c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33932/0 (socket says 172.21.15.32:33932) 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f8385d9b700 1 -- 172.21.15.32:0/445078562 learned_addr learned my addr 172.21.15.32:0/445078562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f838659c700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838809d630 0x7f83881c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.342+0000 7f8386d9d700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8388133930 0x7f83881c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f8385d9b700 1 -- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8388133930 msgr2=0x7f83881c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f8385d9b700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8388133930 0x7f83881c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f8385d9b700 1 -- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838809d630 msgr2=0x7f83881c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f8385d9b700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838809d630 0x7f83881c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f8385d9b700 1 -- 172.21.15.32:0/445078562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700094a0 con 0x7f83880a0a30 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f838659c700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838809d630 0x7f83881c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f8385d9b700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 0x7f83881c83c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f837000ec10 tx=0x7f837000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000c850 con 0x7f83880a0a30 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837000c9b0 con 0x7f83880a0a30 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83780094a0 con 0x7f83880a0a30 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000cd20 con 0x7f83880a0a30 2023-09-30T22:25:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.343+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83881334c0 con 0x7f83880a0a30 2023-09-30T22:25:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.344+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83880747f0 con 0x7f83880a0a30 2023-09-30T22:25:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.344+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8370013470 con 0x7f83880a0a30 2023-09-30T22:25:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.345+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8370040090 con 0x7f83880a0a30 2023-09-30T22:25:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.349+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8370013720 con 0x7f83880a0a30 2023-09-30T22:25:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.522+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3bcc072980 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.522+0000 7f3bc8ff9700 1 -- 172.21.15.32:0/3801546460 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bc00102c0 con 0x7f3bcc0a0a30 2023-09-30T22:25:56.523 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 msgr2=0x7f3bcc133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc133c40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c520 tx=0x7f3bc000c830 comp rx=0 tx=0).stop 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 shutdown_connections 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bcc138d80 0x7f3bcc134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bcc09d630 0x7f3bcc133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 --2- 172.21.15.32:0/3801546460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bcc0a0a30 0x7f3bcc133c40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 >> 172.21.15.32:0/3801546460 conn(0x7f3bcc0a3050 msgr2=0x7f3bcc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 shutdown_connections 2023-09-30T22:25:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.525+0000 7f3bd29a4700 1 -- 172.21.15.32:0/3801546460 wait complete. 2023-09-30T22:25:56.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:56.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.558+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8388072980 con 0x7f83880a0a30 2023-09-30T22:25:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.558+0000 7f837effd700 1 -- 172.21.15.32:0/445078562 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8370021020 con 0x7f83880a0a30 2023-09-30T22:25:56.559 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:56.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 msgr2=0x7f83881c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 0x7f83881c83c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f837000ec10 tx=0x7f837000ef20 comp rx=0 tx=0).stop 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 shutdown_connections 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8388133930 0x7f83881c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838809d630 0x7f83881c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 --2- 172.21.15.32:0/445078562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83880a0a30 0x7f83881c83c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 >> 172.21.15.32:0/445078562 conn(0x7f83880a3050 msgr2=0x7f83881bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.560+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 shutdown_connections 2023-09-30T22:25:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:56.561+0000 7f838c9b8700 1 -- 172.21.15.32:0/445078562 wait complete. 2023-09-30T22:25:56.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:56.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:57.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.112+0000 7f7cb284e700 1 Processor -- start 2023-09-30T22:25:57.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.113+0000 7f7cb284e700 1 -- start start 2023-09-30T22:25:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.113+0000 7f7cb284e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.113+0000 7f7cb284e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac06ae80 0x7f7cac139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.113+0000 7f7cb284e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac09e130 con 0x7f7cac09d9f0 2023-09-30T22:25:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.113+0000 7f7cb284e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac1312b0 con 0x7f7cac1316c0 2023-09-30T22:25:57.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.113+0000 7f7cb284e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac09f6e0 con 0x7f7cac06ae80 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cb0deb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac06ae80 0x7f7cac139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cab7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7cac09d9f0 0x7f7cac06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45360/0 (socket says 172.21.15.32:45360) 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cab7fe700 1 -- 172.21.15.32:0/1514607090 learned_addr learned my addr 172.21.15.32:0/1514607090 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cabfff700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7caaffd700 1 -- 172.21.15.32:0/1514607090 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574567677 0 0) 0x7f7ca0001110 con 0x7f7cac09d9f0 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7caaffd700 1 -- 172.21.15.32:0/1514607090 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c940035b0 con 0x7f7cac09d9f0 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cabfff700 1 -- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac06ae80 msgr2=0x7f7cac139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cabfff700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac06ae80 0x7f7cac139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cabfff700 1 -- 172.21.15.32:0/1514607090 >> v1:172.21.15.32:6789/0 conn(0x7f7cac09d9f0 legacy=0x7f7cac06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cabfff700 1 -- 172.21.15.32:0/1514607090 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac09e760 con 0x7f7cac1316c0 2023-09-30T22:25:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cb0deb700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac06ae80 0x7f7cac139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.114+0000 7f7cabfff700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac131a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009870 tx=0x7f7c9c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d71e6b84c17e48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.115+0000 7f7caaffd700 1 -- 172.21.15.32:0/1514607090 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c011830 con 0x7f7cac1316c0 2023-09-30T22:25:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.115+0000 7f7caaffd700 1 -- 172.21.15.32:0/1514607090 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c011990 con 0x7f7cac1316c0 2023-09-30T22:25:57.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.115+0000 7f7caaffd700 1 -- 172.21.15.32:0/1514607090 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c011d20 con 0x7f7cac1316c0 2023-09-30T22:25:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 -- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 msgr2=0x7f7cac131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac131a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009870 tx=0x7f7c9c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:57.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 -- 172.21.15.32:0/1514607090 shutdown_connections 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac06ae80 0x7f7cac139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 --2- 172.21.15.32:0/1514607090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac131a90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 -- 172.21.15.32:0/1514607090 >> 172.21.15.32:0/1514607090 conn(0x7f7cac066f50 msgr2=0x7f7cac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 -- 172.21.15.32:0/1514607090 shutdown_connections 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.116+0000 7f7cb284e700 1 -- 172.21.15.32:0/1514607090 wait complete. 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 Processor -- start 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 -- start start 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 0x7f7cac06e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac09d9f0 0x7f7cac06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac06f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac071090 con 0x7f7cac06ae80 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac071ff0 con 0x7f7cac1316c0 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.117+0000 7f7cb284e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac072750 con 0x7f7cac09d9f0 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 0x7f7cac06e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 0x7f7cac06e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53372/0 (socket says 172.21.15.32:53372) 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 -- 172.21.15.32:0/3135981281 learned_addr learned my addr 172.21.15.32:0/3135981281 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cb0deb700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cab7fe700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac09d9f0 0x7f7cac06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 -- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac09d9f0 msgr2=0x7f7cac06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac09d9f0 0x7f7cac06ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 -- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 msgr2=0x7f7cac06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 -- 172.21.15.32:0/3135981281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4009a70 con 0x7f7cac06ae80 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cb0deb700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cabfff700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 0x7f7cac06e8e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009bc0 tx=0x7f7c9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0044d0 con 0x7f7cac06ae80 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c0036a0 con 0x7f7cac06ae80 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c0094a0 con 0x7f7cac06ae80 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.118+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c003a30 con 0x7f7cac06ae80 2023-09-30T22:25:57.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.119+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac12ac10 con 0x7f7cac06ae80 2023-09-30T22:25:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.119+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac0712d0 con 0x7f7cac06ae80 2023-09-30T22:25:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.120+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c9c003b90 con 0x7f7cac06ae80 2023-09-30T22:25:57.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.120+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c9c040d80 con 0x7f7cac06ae80 2023-09-30T22:25:57.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.125+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c9c003e40 con 0x7f7cac06ae80 2023-09-30T22:25:57.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.152+0000 7f4b1c85e700 1 Processor -- start 2023-09-30T22:25:57.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.152+0000 7f4b1c85e700 1 -- start start 2023-09-30T22:25:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.152+0000 7f4b1c85e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b1809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1c85e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b1809e000 0x7f4b181339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1c85e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b18133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1c85e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1806b460 con 0x7f4b1809e000 2023-09-30T22:25:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1c85e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1809b120 con 0x7f4b1809e410 2023-09-30T22:25:57.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1c85e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1809a7e0 con 0x7f4b1809d6f0 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b1809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b18133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b1809e000 0x7f4b181339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b1809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33956/0 (socket says 172.21.15.32:33956) 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1659c700 1 -- 172.21.15.32:0/3240357103 learned_addr learned my addr 172.21.15.32:0/3240357103 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 -- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 msgr2=0x7f4b1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b1809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 -- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 msgr2=0x7f4b18133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:57.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b18133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 -- 172.21.15.32:0/3240357103 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1809a500 con 0x7f4b1809e000 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1659c700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b1809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b16d9d700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b18133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b15d9b700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181339f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009840 tx=0x7f4b0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c794e204e0b95c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.153+0000 7f4b1559a700 1 -- 172.21.15.32:0/3240357103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011830 con 0x7f4b1809e000 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1559a700 1 -- 172.21.15.32:0/3240357103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c011990 con 0x7f4b1809e000 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1559a700 1 -- 172.21.15.32:0/3240357103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c011d20 con 0x7f4b1809e000 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 -- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 msgr2=0x7f4b181339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181339f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009840 tx=0x7f4b0c009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 -- 172.21.15.32:0/3240357103 shutdown_connections 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b18133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b1809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/3240357103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181339f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 -- 172.21.15.32:0/3240357103 >> 172.21.15.32:0/3240357103 conn(0x7f4b18066f50 msgr2=0x7f4b1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 -- 172.21.15.32:0/3240357103 shutdown_connections 2023-09-30T22:25:57.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.154+0000 7f4b1c85e700 1 -- 172.21.15.32:0/3240357103 wait complete. 2023-09-30T22:25:57.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.156+0000 7f4b1c85e700 1 Processor -- start 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.156+0000 7f4b1c85e700 1 -- start start 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1c85e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b181c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1c85e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1c85e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b181c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1c85e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181a5590 con 0x7f4b1809e000 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1c85e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181a53a0 con 0x7f4b1809e410 2023-09-30T22:25:57.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1c85e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180a1fe0 con 0x7f4b1809d6f0 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b181c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b16d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b181c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48844/0 (socket says 172.21.15.32:48844) 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53382/0 (socket says 172.21.15.32:53382) 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b16d9d700 1 -- 172.21.15.32:0/2707558033 learned_addr learned my addr 172.21.15.32:0/2707558033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b1659c700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b181c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 -- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 msgr2=0x7f4b181c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b181c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 -- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 msgr2=0x7f4b181c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b181c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 -- 172.21.15.32:0/2707558033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08009550 con 0x7f4b1809e000 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b16d9d700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b181c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.157+0000 7f4b15d9b700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181c86d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4b1806e510 tx=0x7f4b0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.158+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c004450 con 0x7f4b1809e000 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.158+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c01f070 con 0x7f4b1809e000 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.158+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b10009a70 con 0x7f4b1809e000 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.158+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0038c0 con 0x7f4b1809e000 2023-09-30T22:25:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.158+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c0094a0 con 0x7f4b1809e000 2023-09-30T22:25:57.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.159+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b0c003a20 con 0x7f4b1809e000 2023-09-30T22:25:57.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.159+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b181a7380 con 0x7f4b1809e000 2023-09-30T22:25:57.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.159+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b0c021070 con 0x7f4b1809e000 2023-09-30T22:25:57.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.164+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b0c0045b0 con 0x7f4b1809e000 2023-09-30T22:25:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.335+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7cac070d30 con 0x7f7cac06ae80 2023-09-30T22:25:57.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.335+0000 7f7ca8ff9700 1 -- 172.21.15.32:0/3135981281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c9c015070 con 0x7f7cac06ae80 2023-09-30T22:25:57.336 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:57.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 msgr2=0x7f7cac06e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 0x7f7cac06e8e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009bc0 tx=0x7f7c9c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 shutdown_connections 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7cac1316c0 0x7f7cac06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cac06ae80 0x7f7cac06e8e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 --2- 172.21.15.32:0/3135981281 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7cac09d9f0 0x7f7cac06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 >> 172.21.15.32:0/3135981281 conn(0x7f7cac066f50 msgr2=0x7f7cac1301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 shutdown_connections 2023-09-30T22:25:57.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.337+0000 7f7cb284e700 1 -- 172.21.15.32:0/3135981281 wait complete. 2023-09-30T22:25:57.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.370+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4b18074ea0 con 0x7f4b1809e000 2023-09-30T22:25:57.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.371+0000 7f4b06ffd700 1 -- 172.21.15.32:0/2707558033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b0c015020 con 0x7f4b1809e000 2023-09-30T22:25:57.372 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 msgr2=0x7f4b181c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181c86d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4b1806e510 tx=0x7f4b0c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 shutdown_connections 2023-09-30T22:25:57.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b1809e410 0x7f4b181c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b1809d6f0 0x7f4b181c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 --2- 172.21.15.32:0/2707558033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b1809e000 0x7f4b181c86d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 >> 172.21.15.32:0/2707558033 conn(0x7f4b18066f50 msgr2=0x7f4b181bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 shutdown_connections 2023-09-30T22:25:57.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.373+0000 7f4b1c85e700 1 -- 172.21.15.32:0/2707558033 wait complete. 2023-09-30T22:25:57.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:57.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:57.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:57.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.688+0000 7f20e5126700 1 Processor -- start 2023-09-30T22:25:57.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.688+0000 7f20e5126700 1 -- start start 2023-09-30T22:25:57.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20e5126700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 0x7f20e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20e5126700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20e5126700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e009edd0 con 0x7f20e00a1340 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20e5126700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e009e1f0 con 0x7f20e0134170 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20e5126700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20e009dc50 con 0x7f20e00a0a30 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 0x7f20e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 0x7f20e00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33962/0 (socket says 172.21.15.32:33962) 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 -- 172.21.15.32:0/1148349377 learned_addr learned my addr 172.21.15.32:0/1148349377 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20df59e700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 -- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 msgr2=0x7f20e013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 -- 172.21.15.32:0/1148349377 >> v1:172.21.15.32:6789/0 conn(0x7f20e00a1340 legacy=0x7f20e0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ded9d700 1 -- 172.21.15.32:0/1148349377 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00a1fe0 con 0x7f20e00a0a30 2023-09-30T22:25:57.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20df59e700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.689+0000 7f20ddd9b700 1 -- 172.21.15.32:0/1148349377 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492052358 0 0) 0x7f20e009edd0 con 0x7f20e00a1340 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20ded9d700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 0x7f20e00a0e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009920 tx=0x7f20d0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6934ad965e0fe52a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20ddd9b700 1 -- 172.21.15.32:0/1148349377 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0011830 con 0x7f20e00a0a30 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20ddd9b700 1 -- 172.21.15.32:0/1148349377 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d0011990 con 0x7f20e00a0a30 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20ddd9b700 1 -- 172.21.15.32:0/1148349377 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0011d20 con 0x7f20e00a0a30 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20e5126700 1 -- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 msgr2=0x7f20e00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20e5126700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 0x7f20e00a0e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009920 tx=0x7f20d0009c30 comp rx=0 tx=0).stop 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20e5126700 1 -- 172.21.15.32:0/1148349377 shutdown_connections 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20e5126700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20e5126700 1 --2- 172.21.15.32:0/1148349377 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a0a30 0x7f20e00a0e00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.690+0000 7f20e5126700 1 -- 172.21.15.32:0/1148349377 >> 172.21.15.32:0/1148349377 conn(0x7f20e00a6550 msgr2=0x7f20e00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.691+0000 7f20e5126700 1 -- 172.21.15.32:0/1148349377 shutdown_connections 2023-09-30T22:25:57.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.691+0000 7f20e5126700 1 -- 172.21.15.32:0/1148349377 wait complete. 2023-09-30T22:25:57.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.692+0000 7f20e5126700 1 Processor -- start 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.692+0000 7f20e5126700 1 -- start start 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.692+0000 7f20e5126700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00a0a30 0x7f20e0122810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.692+0000 7f20e5126700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a1340 0x7f20e0122d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.692+0000 7f20e5126700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e0123290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20e5126700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00a0290 con 0x7f20e00a0a30 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20e5126700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0126050 con 0x7f20e0134170 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20e5126700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20e01245a0 con 0x7f20e00a1340 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20df59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e0123290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20ded9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00a0a30 0x7f20e0122810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20df59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e0123290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48860/0 (socket says 172.21.15.32:48860) 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20df59e700 1 -- 172.21.15.32:0/906401675 learned_addr learned my addr 172.21.15.32:0/906401675 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a1340 0x7f20e0122d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 -- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 msgr2=0x7f20e0123290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e0123290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 -- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00a0a30 msgr2=0x7f20e0122810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00a0a30 0x7f20e0122810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 -- 172.21.15.32:0/906401675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009a70 con 0x7f20e00a1340 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20df59e700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e0123290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20ded9d700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00a0a30 0x7f20e0122810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20de59c700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a1340 0x7f20e0122d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c430 tx=0x7f20d400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40054e0 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d400ce40 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d0009550 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.693+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40038c0 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.694+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e00a25e0 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.694+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20d4003a20 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.694+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f20d403e9f0 con 0x7f20e00a1340 2023-09-30T22:25:57.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.695+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e01337c0 con 0x7f20e00a1340 2023-09-30T22:25:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.700+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20d4017080 con 0x7f20e00a1340 2023-09-30T22:25:57.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.727+0000 7efc7365a700 1 Processor -- start 2023-09-30T22:25:57.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc7365a700 1 -- start start 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc7365a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc7365a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 0x7efc6c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc7365a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6c0a2a70 con 0x7efc6c0a0a30 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc7365a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c0a01d0 con 0x7efc6c09d630 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc7365a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c09e2b0 con 0x7efc6c138d80 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc71bf7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 0x7efc6c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc70bf5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc71bf7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 0x7efc6c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33988/0 (socket says 172.21.15.32:33988) 2023-09-30T22:25:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc713f6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efc6c0a0a30 0x7efc6c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45376/0 (socket says 172.21.15.32:45376) 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc70bf5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48876/0 (socket says 172.21.15.32:48876) 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.728+0000 7efc71bf7700 1 -- 172.21.15.32:0/238023407 learned_addr learned my addr 172.21.15.32:0/238023407 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc70bf5700 1 -- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 msgr2=0x7efc6c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc70bf5700 1 --2- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 0x7efc6c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc70bf5700 1 -- 172.21.15.32:0/238023407 >> v1:172.21.15.32:6789/0 conn(0x7efc6c0a0a30 legacy=0x7efc6c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc6bfff700 1 -- 172.21.15.32:0/238023407 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458918925 0 0) 0x7efc6c0a2a70 con 0x7efc6c0a0a30 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc70bf5700 1 -- 172.21.15.32:0/238023407 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c09dc50 con 0x7efc6c09d630 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc71bf7700 1 --2- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 0x7efc6c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc70bf5700 1 --2- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c0a1510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc60009870 tx=0x7efc60009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f677f40ca092c010 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc6bfff700 1 -- 172.21.15.32:0/238023407 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60011830 con 0x7efc6c09d630 2023-09-30T22:25:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc6bfff700 1 -- 172.21.15.32:0/238023407 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc60011990 con 0x7efc6c09d630 2023-09-30T22:25:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.729+0000 7efc6bfff700 1 -- 172.21.15.32:0/238023407 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60011d20 con 0x7efc6c09d630 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 -- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 msgr2=0x7efc6c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 --2- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c0a1510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc60009870 tx=0x7efc60009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 -- 172.21.15.32:0/238023407 shutdown_connections 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 --2- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c138d80 0x7efc6c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 --2- 172.21.15.32:0/238023407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c0a1510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 -- 172.21.15.32:0/238023407 >> 172.21.15.32:0/238023407 conn(0x7efc6c0a3050 msgr2=0x7efc6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 -- 172.21.15.32:0/238023407 shutdown_connections 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.730+0000 7efc7365a700 1 -- 172.21.15.32:0/238023407 wait complete. 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.731+0000 7efc7365a700 1 Processor -- start 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.731+0000 7efc7365a700 1 -- start start 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc7365a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c133910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc7365a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 0x7efc6c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc7365a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc6c138d80 0x7efc6c134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc7365a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c134de0 con 0x7efc6c138d80 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc7365a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c1374b0 con 0x7efc6c09d630 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc7365a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c136e00 con 0x7efc6c0a0a30 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc713f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc70bf5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 0x7efc6c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc70bf5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 0x7efc6c133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33992/0 (socket says 172.21.15.32:33992) 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc713f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c133910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48878/0 (socket says 172.21.15.32:48878) 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc70bf5700 1 -- 172.21.15.32:0/1245843975 learned_addr learned my addr 172.21.15.32:0/1245843975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.732+0000 7efc71bf7700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc6c138d80 0x7efc6c134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc70bf5700 1 -- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 msgr2=0x7efc6c133910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc70bf5700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c133910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc70bf5700 1 -- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc6c138d80 msgr2=0x7efc6c134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc70bf5700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc6c138d80 0x7efc6c134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc70bf5700 1 -- 172.21.15.32:0/1245843975 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc64009550 con 0x7efc6c0a0a30 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc71bf7700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc6c138d80 0x7efc6c134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc70bf5700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 0x7efc6c133e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc60009840 tx=0x7efc60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60004300 con 0x7efc6c0a0a30 2023-09-30T22:25:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6001f070 con 0x7efc6c0a0a30 2023-09-30T22:25:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc600094a0 con 0x7efc6c0a0a30 2023-09-30T22:25:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc600036a0 con 0x7efc6c0a0a30 2023-09-30T22:25:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.733+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6c1370c0 con 0x7efc6c0a0a30 2023-09-30T22:25:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.734+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc60021030 con 0x7efc6c0a0a30 2023-09-30T22:25:57.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.734+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efc60040700 con 0x7efc6c0a0a30 2023-09-30T22:25:57.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.735+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6c0747f0 con 0x7efc6c0a0a30 2023-09-30T22:25:57.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.740+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc600199a0 con 0x7efc6c0a0a30 2023-09-30T22:25:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.909+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20e0074ea0 con 0x7f20e00a1340 2023-09-30T22:25:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.910+0000 7f20c77fe700 1 -- 172.21.15.32:0/906401675 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20d4019b20 con 0x7f20e00a1340 2023-09-30T22:25:57.911 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a1340 msgr2=0x7f20e0122d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a1340 0x7f20e0122d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c430 tx=0x7f20d400c740 comp rx=0 tx=0).stop 2023-09-30T22:25:57.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 shutdown_connections 2023-09-30T22:25:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20e0134170 0x7f20e0123290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20e00a0a30 0x7f20e0122810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 --2- 172.21.15.32:0/906401675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20e00a1340 0x7f20e0122d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 >> 172.21.15.32:0/906401675 conn(0x7f20e00a6550 msgr2=0x7f20e009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 shutdown_connections 2023-09-30T22:25:57.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.912+0000 7f20e5126700 1 -- 172.21.15.32:0/906401675 wait complete. 2023-09-30T22:25:57.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.947+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efc6c135c80 con 0x7efc6c0a0a30 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.947+0000 7efc69ffb700 1 -- 172.21.15.32:0/1245843975 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc60006cc0 con 0x7efc6c0a0a30 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 msgr2=0x7efc6c133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 0x7efc6c133e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc60009840 tx=0x7efc60004750 comp rx=0 tx=0).stop 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 shutdown_connections 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc6c138d80 0x7efc6c134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc6c09d630 0x7efc6c133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 --2- 172.21.15.32:0/1245843975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc6c0a0a30 0x7efc6c133e50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 >> 172.21.15.32:0/1245843975 conn(0x7efc6c0a3050 msgr2=0x7efc6c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 shutdown_connections 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:57.949+0000 7efc7365a700 1 -- 172.21.15.32:0/1245843975 wait complete. 2023-09-30T22:25:57.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:58.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:58.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:58.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.501+0000 7f75913b3700 1 Processor -- start 2023-09-30T22:25:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.501+0000 7f75913b3700 1 -- start start 2023-09-30T22:25:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f75913b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 0x7f758c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f75913b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f75913b3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c0a1810 con 0x7f758c133920 2023-09-30T22:25:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f75913b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c09ea10 con 0x7f758c0a09c0 2023-09-30T22:25:58.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f75913b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f758c09e210 con 0x7f758c09d5c0 2023-09-30T22:25:58.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f758affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 0x7f758c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f758a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f758b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f758c133920 0x7f758c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45392/0 (socket says 172.21.15.32:45392) 2023-09-30T22:25:58.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f758affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 0x7f758c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48890/0 (socket says 172.21.15.32:48890) 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f758b7fe700 1 -- 172.21.15.32:0/1133453426 learned_addr learned my addr 172.21.15.32:0/1133453426 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f758a7fc700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34006/0 (socket says 172.21.15.32:34006) 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f758affd700 1 -- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 msgr2=0x7f758c0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.502+0000 7f7589ffb700 1 -- 172.21.15.32:0/1133453426 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646705122 0 0) 0x7f758c0a1810 con 0x7f758c133920 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f758affd700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c0a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f758affd700 1 -- 172.21.15.32:0/1133453426 >> v1:172.21.15.32:6789/0 conn(0x7f758c133920 legacy=0x7f758c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f758affd700 1 -- 172.21.15.32:0/1133453426 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0a26f0 con 0x7f758c0a09c0 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f758a7fc700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f758affd700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 0x7f758c0a0d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4d32f0c5631edb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f7589ffb700 1 -- 172.21.15.32:0/1133453426 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c011830 con 0x7f758c0a09c0 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f7589ffb700 1 -- 172.21.15.32:0/1133453426 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c011990 con 0x7f758c0a09c0 2023-09-30T22:25:58.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.503+0000 7f7589ffb700 1 -- 172.21.15.32:0/1133453426 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c011d20 con 0x7f758c0a09c0 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 -- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 msgr2=0x7f758c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 0x7f758c0a0d90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f757c009870 tx=0x7f757c009b80 comp rx=0 tx=0).stop 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 -- 172.21.15.32:0/1133453426 shutdown_connections 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c0a09c0 0x7f758c0a0d90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 --2- 172.21.15.32:0/1133453426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 -- 172.21.15.32:0/1133453426 >> 172.21.15.32:0/1133453426 conn(0x7f758c0a2fe0 msgr2=0x7f758c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 -- 172.21.15.32:0/1133453426 shutdown_connections 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.504+0000 7f75913b3700 1 -- 172.21.15.32:0/1133453426 wait complete. 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.506+0000 7f75913b3700 1 Processor -- start 2023-09-30T22:25:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.506+0000 7f75913b3700 1 -- start start 2023-09-30T22:25:58.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75913b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c11f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75913b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0a09c0 0x7f758c11fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75913b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c133920 0x7f758c1c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75913b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1210d0 con 0x7f758c0a09c0 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75913b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1221c0 con 0x7f758c133920 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75913b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f758c11f090 con 0x7f758c09d5c0 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0a09c0 0x7f758c11fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0a09c0 0x7f758c11fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53422/0 (socket says 172.21.15.32:53422) 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758a7fc700 1 -- 172.21.15.32:0/3890627099 learned_addr learned my addr 172.21.15.32:0/3890627099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c11f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758b7fe700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c133920 0x7f758c1c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 -- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c133920 msgr2=0x7f758c1c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c133920 0x7f758c1c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 -- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0a09c0 msgr2=0x7f758c11fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0a09c0 0x7f758c11fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 -- 172.21.15.32:0/3890627099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800094a0 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f758affd700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c11f630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f757c016000 tx=0x7f757c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.507+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c004340 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.508+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c01f070 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.508+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c0094a0 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.508+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c0036a0 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.508+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c1207e0 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.509+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f757c021030 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.509+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c1200b0 con 0x7f758c09d5c0 2023-09-30T22:25:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.509+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f757c0407c0 con 0x7f758c09d5c0 2023-09-30T22:25:58.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.514+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f757c02b970 con 0x7f758c09d5c0 2023-09-30T22:25:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.539+0000 7f57ff90c700 1 Processor -- start 2023-09-30T22:25:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.539+0000 7f57ff90c700 1 -- start start 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57ff90c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f80a0a30 0x7f57f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57ff90c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57ff90c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80a2a70 con 0x7f57f809d630 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57ff90c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80a01d0 con 0x7f57f80a0a30 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57ff90c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57f809e2b0 con 0x7f57f8138d80 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57fdea9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57fcea7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57f809d630 0x7f57f80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45394/0 (socket says 172.21.15.32:45394) 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57fcea7700 1 -- 172.21.15.32:0/3284244574 learned_addr learned my addr 172.21.15.32:0/3284244574 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57fd6a8700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f80a0a30 0x7f57f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.540+0000 7f57f7fff700 1 -- 172.21.15.32:0/3284244574 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155681778 0 0) 0x7f57f80a2a70 con 0x7f57f809d630 2023-09-30T22:25:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57f7fff700 1 -- 172.21.15.32:0/3284244574 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e00035b0 con 0x7f57f809d630 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57fd6a8700 1 -- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 msgr2=0x7f57f813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57fd6a8700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57fd6a8700 1 -- 172.21.15.32:0/3284244574 >> v1:172.21.15.32:6789/0 conn(0x7f57f809d630 legacy=0x7f57f80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57fd6a8700 1 -- 172.21.15.32:0/3284244574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f809dc50 con 0x7f57f80a0a30 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57fdea9700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57fd6a8700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f80a0a30 0x7f57f80a0e00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009dc0 tx=0x7f57e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5152e4a0ed1f5ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57f7fff700 1 -- 172.21.15.32:0/3284244574 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800e070 con 0x7f57f80a0a30 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57f7fff700 1 -- 172.21.15.32:0/3284244574 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e800bb20 con 0x7f57f80a0a30 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.541+0000 7f57f7fff700 1 -- 172.21.15.32:0/3284244574 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8012430 con 0x7f57f80a0a30 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 -- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f80a0a30 msgr2=0x7f57f80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f80a0a30 0x7f57f80a0e00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009dc0 tx=0x7f57e800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 -- 172.21.15.32:0/3284244574 shutdown_connections 2023-09-30T22:25:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 --2- 172.21.15.32:0/3284244574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f80a0a30 0x7f57f80a0e00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 -- 172.21.15.32:0/3284244574 >> 172.21.15.32:0/3284244574 conn(0x7f57f80a3050 msgr2=0x7f57f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 -- 172.21.15.32:0/3284244574 shutdown_connections 2023-09-30T22:25:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.542+0000 7f57ff90c700 1 -- 172.21.15.32:0/3284244574 wait complete. 2023-09-30T22:25:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 Processor -- start 2023-09-30T22:25:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 -- start start 2023-09-30T22:25:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f809d630 0x7f57f8133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 0x7f57f8133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f81341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f81365a0 con 0x7f57f80a0a30 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8138b10 con 0x7f57f809d630 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57ff90c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8136f10 con 0x7f57f8138d80 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57fdea9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f81341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57fcea7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 0x7f57f8133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57fdea9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f81341e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34036/0 (socket says 172.21.15.32:34036) 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57fdea9700 1 -- 172.21.15.32:0/2133200748 learned_addr learned my addr 172.21.15.32:0/2133200748 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.543+0000 7f57fcea7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 0x7f57f8133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53438/0 (socket says 172.21.15.32:53438) 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fd6a8700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f809d630 0x7f57f8133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fcea7700 1 -- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 msgr2=0x7f57f81341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fcea7700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f81341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fcea7700 1 -- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f809d630 msgr2=0x7f57f8133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fcea7700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f809d630 0x7f57f8133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fcea7700 1 -- 172.21.15.32:0/2133200748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00094a0 con 0x7f57f80a0a30 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fdea9700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f81341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fd6a8700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f809d630 0x7f57f8133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57fcea7700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 0x7f57f8133ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c4c0 tx=0x7f57ec00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec005d70 con 0x7f57f80a0a30 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec0070d0 con 0x7f57f80a0a30 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e8009a70 con 0x7f57f80a0a30 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.544+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007440 con 0x7f57f80a0a30 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.545+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f81371f0 con 0x7f57f80a0a30 2023-09-30T22:25:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.545+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f57ec0075a0 con 0x7f57f80a0a30 2023-09-30T22:25:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.545+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f57ec03ea80 con 0x7f57f80a0a30 2023-09-30T22:25:58.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.545+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f8137050 con 0x7f57f80a0a30 2023-09-30T22:25:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.551+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57f8137050 con 0x7f57f80a0a30 2023-09-30T22:25:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.723+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f758c122030 con 0x7f758c09d5c0 2023-09-30T22:25:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.723+0000 7f75737fe700 1 -- 172.21.15.32:0/3890627099 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f757c015070 con 0x7f758c09d5c0 2023-09-30T22:25:58.724 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 msgr2=0x7f758c11f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c11f630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f757c016000 tx=0x7f757c004750 comp rx=0 tx=0).stop 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 shutdown_connections 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f758c133920 0x7f758c1c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f758c09d5c0 0x7f758c11f630 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 --2- 172.21.15.32:0/3890627099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f758c0a09c0 0x7f758c11fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.725+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 >> 172.21.15.32:0/3890627099 conn(0x7f758c0a2fe0 msgr2=0x7f758c09ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.726+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 shutdown_connections 2023-09-30T22:25:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.726+0000 7f75913b3700 1 -- 172.21.15.32:0/3890627099 wait complete. 2023-09-30T22:25:58.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:25:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.754+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f57f8137050 con 0x7f57f80a0a30 2023-09-30T22:25:58.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.755+0000 7f57f5ffb700 1 -- 172.21.15.32:0/2133200748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57f8137050 con 0x7f57f80a0a30 2023-09-30T22:25:58.756 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:25:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 msgr2=0x7f57f8133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 0x7f57f8133ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c4c0 tx=0x7f57ec00c7d0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 shutdown_connections 2023-09-30T22:25:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f8138d80 0x7f57f81341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f809d630 0x7f57f8133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 --2- 172.21.15.32:0/2133200748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f80a0a30 0x7f57f8133ca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 >> 172.21.15.32:0/2133200748 conn(0x7f57f80a3050 msgr2=0x7f57f813ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 shutdown_connections 2023-09-30T22:25:58.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:58.757+0000 7f57ff90c700 1 -- 172.21.15.32:0/2133200748 wait complete. 2023-09-30T22:25:58.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:25:58.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:25:58.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:25:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.076+0000 7f9d77c63700 1 Processor -- start 2023-09-30T22:25:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.076+0000 7f9d77c63700 1 -- start start 2023-09-30T22:25:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d77c63700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d700a1510 0x7f9d70138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d77c63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d70139250 0x7f9d7013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d77c63700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7009edd0 con 0x7f9d700a0a30 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d77c63700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d700a18e0 con 0x7f9d700a1510 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d77c63700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7009e1f0 con 0x7f9d70139250 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d751fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d700a1510 0x7f9d70138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d76200700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d70139250 0x7f9d7013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d759ff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d700a0a30 0x7f9d700a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45396/0 (socket says 172.21.15.32:45396) 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d759ff700 1 -- 172.21.15.32:0/1915111951 learned_addr learned my addr 172.21.15.32:0/1915111951 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d751fe700 1 -- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d70139250 msgr2=0x7f9d7013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d751fe700 1 --2- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d70139250 0x7f9d7013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d751fe700 1 -- 172.21.15.32:0/1915111951 >> v1:172.21.15.32:6789/0 conn(0x7f9d700a0a30 legacy=0x7f9d700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d751fe700 1 -- 172.21.15.32:0/1915111951 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7009dc50 con 0x7f9d700a1510 2023-09-30T22:25:59.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d749fd700 1 -- 172.21.15.32:0/1915111951 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327697590 0 0) 0x7f9d7009edd0 con 0x7f9d700a0a30 2023-09-30T22:25:59.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d76200700 1 --2- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d70139250 0x7f9d7013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.077+0000 7f9d751fe700 1 --2- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a1510 0x7f9d70138d10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009840 tx=0x7f9d60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8086582be5f60e73 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d749fd700 1 -- 172.21.15.32:0/1915111951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60011830 con 0x7f9d700a1510 2023-09-30T22:25:59.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d749fd700 1 -- 172.21.15.32:0/1915111951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d60011990 con 0x7f9d700a1510 2023-09-30T22:25:59.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d749fd700 1 -- 172.21.15.32:0/1915111951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60011d20 con 0x7f9d700a1510 2023-09-30T22:25:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d77c63700 1 -- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a1510 msgr2=0x7f9d70138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d77c63700 1 --2- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a1510 0x7f9d70138d10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009840 tx=0x7f9d60009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d77c63700 1 -- 172.21.15.32:0/1915111951 shutdown_connections 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d77c63700 1 --2- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d70139250 0x7f9d7013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d77c63700 1 --2- 172.21.15.32:0/1915111951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a1510 0x7f9d70138d10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.078+0000 7f9d77c63700 1 -- 172.21.15.32:0/1915111951 >> 172.21.15.32:0/1915111951 conn(0x7f9d700a6550 msgr2=0x7f9d700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.079+0000 7f9d77c63700 1 -- 172.21.15.32:0/1915111951 shutdown_connections 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.079+0000 7f9d77c63700 1 -- 172.21.15.32:0/1915111951 wait complete. 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.079+0000 7f9d77c63700 1 Processor -- start 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 -- start start 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 0x7f9d701228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d700a1510 0x7f9d70122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d70139250 0x7f9d70123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d701389e0 con 0x7f9d700a0a30 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700a0220 con 0x7f9d700a1510 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d77c63700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70126060 con 0x7f9d70139250 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 0x7f9d701228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d751fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d700a1510 0x7f9d70122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 0x7f9d701228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53456/0 (socket says 172.21.15.32:53456) 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 -- 172.21.15.32:0/946683497 learned_addr learned my addr 172.21.15.32:0/946683497 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d76200700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d70139250 0x7f9d70123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 -- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d70139250 msgr2=0x7f9d70123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d70139250 0x7f9d70123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 -- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d700a1510 msgr2=0x7f9d70122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d700a1510 0x7f9d70122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.080+0000 7f9d759ff700 1 -- 172.21.15.32:0/946683497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d68009a70 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d751fe700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d700a1510 0x7f9d70122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d759ff700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 0x7f9d701228c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00c520 tx=0x7f9d6c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c004c80 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c004e00 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d600094a0 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c007310 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.081+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d70124bb0 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.082+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d6c007470 con 0x7f9d700a0a30 2023-09-30T22:25:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.082+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d6c03f340 con 0x7f9d700a0a30 2023-09-30T22:25:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.082+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70074ea0 con 0x7f9d700a0a30 2023-09-30T22:25:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.087+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d6c0123b0 con 0x7f9d700a0a30 2023-09-30T22:25:59.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.114+0000 7f277a38a700 1 Processor -- start 2023-09-30T22:25:59.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.114+0000 7f277a38a700 1 -- start start 2023-09-30T22:25:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f277a38a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f2774131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f277a38a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 0x7f277406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f277a38a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f277406acb0 0x7f277406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f277a38a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277409e130 con 0x7f277406acb0 2023-09-30T22:25:59.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f277a38a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27741312b0 con 0x7f27741316c0 2023-09-30T22:25:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f277a38a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f277409f6e0 con 0x7f277409d9f0 2023-09-30T22:25:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f27737fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 0x7f277406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.115+0000 7f2773fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f2774131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 0x7f277406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34064/0 (socket says 172.21.15.32:34064) 2023-09-30T22:25:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f2773fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f2774131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48928/0 (socket says 172.21.15.32:48928) 2023-09-30T22:25:59.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 -- 172.21.15.32:0/2754723278 learned_addr learned my addr 172.21.15.32:0/2754723278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 -- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 msgr2=0x7f2774131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 --2- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f2774131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 -- 172.21.15.32:0/2754723278 >> v2:172.21.15.32:3300/0 conn(0x7f277406acb0 msgr2=0x7f277406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:59.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 --2- 172.21.15.32:0/2754723278 >> v2:172.21.15.32:3300/0 conn(0x7f277406acb0 0x7f277406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 -- 172.21.15.32:0/2754723278 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277409e760 con 0x7f277409d9f0 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f27737fe700 1 --2- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 0x7f277406a770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f27680098f0 tx=0x7f2768009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5186218535f6ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f2772ffd700 1 -- 172.21.15.32:0/2754723278 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768011830 con 0x7f277409d9f0 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f2772ffd700 1 -- 172.21.15.32:0/2754723278 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768011990 con 0x7f277409d9f0 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.116+0000 7f2772ffd700 1 -- 172.21.15.32:0/2754723278 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768011d20 con 0x7f277409d9f0 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 -- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 msgr2=0x7f277406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 --2- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 0x7f277406a770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f27680098f0 tx=0x7f2768009c00 comp rx=0 tx=0).stop 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 -- 172.21.15.32:0/2754723278 shutdown_connections 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 --2- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f2774131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 --2- 172.21.15.32:0/2754723278 >> v2:172.21.15.32:3300/0 conn(0x7f277406acb0 0x7f277406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 --2- 172.21.15.32:0/2754723278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277409d9f0 0x7f277406a770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 -- 172.21.15.32:0/2754723278 >> 172.21.15.32:0/2754723278 conn(0x7f2774066f50 msgr2=0x7f277409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 -- 172.21.15.32:0/2754723278 shutdown_connections 2023-09-30T22:25:59.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.117+0000 7f277a38a700 1 -- 172.21.15.32:0/2754723278 wait complete. 2023-09-30T22:25:59.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.119+0000 7f277a38a700 1 Processor -- start 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.119+0000 7f277a38a700 1 -- start start 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.119+0000 7f277a38a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277406acb0 0x7f27741c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f277a38a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 0x7f27741c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f277a38a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f27741c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f277a38a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277411fd70 con 0x7f277409d9f0 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f277a38a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2774071f50 con 0x7f27741316c0 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f277a38a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27741202a0 con 0x7f277406acb0 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f27737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 0x7f27741c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f27737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 0x7f27741c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53486/0 (socket says 172.21.15.32:53486) 2023-09-30T22:25:59.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f27737fe700 1 -- 172.21.15.32:0/1068837303 learned_addr learned my addr 172.21.15.32:0/1068837303 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277406acb0 0x7f27741c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2778927700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f27741c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 -- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 msgr2=0x7f27741c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f27741c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 -- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 msgr2=0x7f27741c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 0x7f27741c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 -- 172.21.15.32:0/1068837303 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009550 con 0x7f277406acb0 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f27737fe700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 0x7f27741c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2778927700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f27741c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.120+0000 7f2773fff700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277406acb0 0x7f27741c8330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f276400e900 tx=0x7f276400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.121+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400ee50 con 0x7f277406acb0 2023-09-30T22:25:59.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.121+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276400c850 con 0x7f277406acb0 2023-09-30T22:25:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.121+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277411eb80 con 0x7f277406acb0 2023-09-30T22:25:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.121+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400cbc0 con 0x7f277406acb0 2023-09-30T22:25:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.121+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27741207a0 con 0x7f277406acb0 2023-09-30T22:25:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.122+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2774074ea0 con 0x7f277406acb0 2023-09-30T22:25:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.122+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f276400cd20 con 0x7f277406acb0 2023-09-30T22:25:59.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.123+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f276403ff90 con 0x7f277406acb0 2023-09-30T22:25:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.127+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f276401a360 con 0x7f277406acb0 2023-09-30T22:25:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.314+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9d70125e50 con 0x7f9d700a0a30 2023-09-30T22:25:59.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.314+0000 7f9d667fc700 1 -- 172.21.15.32:0/946683497 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d6c00ce60 con 0x7f9d700a0a30 2023-09-30T22:25:59.315 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 msgr2=0x7f9d701228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 0x7f9d701228c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00c520 tx=0x7f9d6c00c830 comp rx=0 tx=0).stop 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 shutdown_connections 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d70139250 0x7f9d70123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700a0a30 0x7f9d701228c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 --2- 172.21.15.32:0/946683497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d700a1510 0x7f9d70122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 >> 172.21.15.32:0/946683497 conn(0x7f9d700a6550 msgr2=0x7f9d700a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:59.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 shutdown_connections 2023-09-30T22:25:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.317+0000 7f9d77c63700 1 -- 172.21.15.32:0/946683497 wait complete. 2023-09-30T22:25:59.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:25:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.348+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f277409e980 con 0x7f277406acb0 2023-09-30T22:25:59.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.349+0000 7f2770ff9700 1 -- 172.21.15.32:0/1068837303 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2764017070 con 0x7f277406acb0 2023-09-30T22:25:59.350 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277406acb0 msgr2=0x7f27741c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277406acb0 0x7f27741c8330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f276400e900 tx=0x7f276400ec10 comp rx=0 tx=0).stop 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 shutdown_connections 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f27741316c0 0x7f27741c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277406acb0 0x7f27741c8330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 --2- 172.21.15.32:0/1068837303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277409d9f0 0x7f27741c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 >> 172.21.15.32:0/1068837303 conn(0x7f2774066f50 msgr2=0x7f27740a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 shutdown_connections 2023-09-30T22:25:59.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.351+0000 7f277a38a700 1 -- 172.21.15.32:0/1068837303 wait complete. 2023-09-30T22:25:59.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:25:59.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:25:59.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:25:59.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 Processor -- start 2023-09-30T22:25:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 -- start start 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b80a0a30 0x7f75b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b80a1340 0x7f75b8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b809edd0 con 0x7f75b8133fa0 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b809e1f0 con 0x7f75b80a0a30 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75bd0aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75b809dc50 con 0x7f75b80a1340 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75b6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b80a0a30 0x7f75b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75b659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b80a1340 0x7f75b8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75b759e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75b8133fa0 0x7f75b813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45402/0 (socket says 172.21.15.32:45402) 2023-09-30T22:25:59.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.890+0000 7f75b759e700 1 -- 172.21.15.32:0/4106079713 learned_addr learned my addr 172.21.15.32:0/4106079713 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b6d9d700 1 -- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b80a1340 msgr2=0x7f75b8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b6d9d700 1 --2- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b80a1340 0x7f75b8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b6d9d700 1 -- 172.21.15.32:0/4106079713 >> v1:172.21.15.32:6789/0 conn(0x7f75b8133fa0 legacy=0x7f75b813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b6d9d700 1 -- 172.21.15.32:0/4106079713 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b80a1fe0 con 0x7f75b80a0a30 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b5d9b700 1 -- 172.21.15.32:0/4106079713 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331955733 0 0) 0x7f75b809edd0 con 0x7f75b8133fa0 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b659c700 1 --2- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b80a1340 0x7f75b8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b6d9d700 1 --2- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a0a30 0x7f75b80a0e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009d90 tx=0x7f75a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3274d03bdc539c66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b5d9b700 1 -- 172.21.15.32:0/4106079713 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a800e070 con 0x7f75b80a0a30 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b5d9b700 1 -- 172.21.15.32:0/4106079713 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a800bb20 con 0x7f75b80a0a30 2023-09-30T22:25:59.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.891+0000 7f75b5d9b700 1 -- 172.21.15.32:0/4106079713 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8012430 con 0x7f75b80a0a30 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 -- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a0a30 msgr2=0x7f75b80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a0a30 0x7f75b80a0e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009d90 tx=0x7f75a800b330 comp rx=0 tx=0).stop 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 -- 172.21.15.32:0/4106079713 shutdown_connections 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a0a30 0x7f75b80a0e00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/4106079713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b80a1340 0x7f75b8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 -- 172.21.15.32:0/4106079713 >> 172.21.15.32:0/4106079713 conn(0x7f75b80a6550 msgr2=0x7f75b80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 -- 172.21.15.32:0/4106079713 shutdown_connections 2023-09-30T22:25:59.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.892+0000 7f75bd0aa700 1 -- 172.21.15.32:0/4106079713 wait complete. 2023-09-30T22:25:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.894+0000 7f75bd0aa700 1 Processor -- start 2023-09-30T22:25:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.894+0000 7f75bd0aa700 1 -- start start 2023-09-30T22:25:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75bd0aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75b80a0a30 0x7f75b8122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75bd0aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 0x7f75b8122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75bd0aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b8133fa0 0x7f75b8123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75bd0aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b8124520 con 0x7f75b80a1340 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75bd0aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b8133320 con 0x7f75b80a0a30 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75bd0aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75b81262b0 con 0x7f75b8133fa0 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 0x7f75b8122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b8133fa0 0x7f75b8123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75b80a0a30 0x7f75b8122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 0x7f75b8122cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53510/0 (socket says 172.21.15.32:53510) 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b659c700 1 -- 172.21.15.32:0/3172763809 learned_addr learned my addr 172.21.15.32:0/3172763809 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b759e700 1 -- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75b80a0a30 msgr2=0x7f75b8122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b759e700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75b80a0a30 0x7f75b8122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b759e700 1 -- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 msgr2=0x7f75b8122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b759e700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 0x7f75b8122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b759e700 1 -- 172.21.15.32:0/3172763809 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00094a0 con 0x7f75b8133fa0 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.895+0000 7f75b659c700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 0x7f75b8122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.896+0000 7f75b759e700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b8133fa0 0x7f75b8123210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c460 tx=0x7f75b000c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.896+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004c80 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.896+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0004e00 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.896+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a8009a70 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.896+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0007310 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.896+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b80a0220 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.897+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75b0007470 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.897+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f75b003f2c0 con 0x7f75b8133fa0 2023-09-30T22:25:59.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.897+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b8074ea0 con 0x7f75b8133fa0 2023-09-30T22:25:59.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.902+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75b000ce30 con 0x7f75b8133fa0 2023-09-30T22:25:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.936+0000 7fe08e956700 1 Processor -- start 2023-09-30T22:25:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.936+0000 7fe08e956700 1 -- start start 2023-09-30T22:25:59.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08e956700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe0880a0a30 0x7fe0880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08e956700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 0x7fe08813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08e956700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0880a2a70 con 0x7fe08809d630 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08e956700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0880a01d0 con 0x7fe0880a0a30 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08e956700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe08809e2b0 con 0x7fe088138d80 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe087fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe0880a0a30 0x7fe0880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08cef3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 0x7fe08813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe0877fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe08809d630 0x7fe0880a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45418/0 (socket says 172.21.15.32:45418) 2023-09-30T22:25:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe087fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe0880a0a30 0x7fe0880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53520/0 (socket says 172.21.15.32:53520) 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe0877fe700 1 -- 172.21.15.32:0/2521721688 learned_addr learned my addr 172.21.15.32:0/2521721688 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08cef3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 0x7fe08813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34096/0 (socket says 172.21.15.32:34096) 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe086ffd700 1 -- 172.21.15.32:0/2521721688 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887972959 0 0) 0x7fe0880a2a70 con 0x7fe08809d630 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe086ffd700 1 -- 172.21.15.32:0/2521721688 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0700035b0 con 0x7fe08809d630 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe087fff700 1 -- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 msgr2=0x7fe08813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe087fff700 1 --2- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 0x7fe08813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe087fff700 1 -- 172.21.15.32:0/2521721688 >> v1:172.21.15.32:6789/0 conn(0x7fe08809d630 legacy=0x7fe0880a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe087fff700 1 -- 172.21.15.32:0/2521721688 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08809dc50 con 0x7fe0880a0a30 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.937+0000 7fe08cef3700 1 --2- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 0x7fe08813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe087fff700 1 --2- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe0880a0e00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe078009840 tx=0x7fe078009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bf8ff03c090986 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe086ffd700 1 -- 172.21.15.32:0/2521721688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078011830 con 0x7fe0880a0a30 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe086ffd700 1 -- 172.21.15.32:0/2521721688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe078011990 con 0x7fe0880a0a30 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe086ffd700 1 -- 172.21.15.32:0/2521721688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078011d20 con 0x7fe0880a0a30 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe08e956700 1 -- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 msgr2=0x7fe0880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe08e956700 1 --2- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe0880a0e00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe078009840 tx=0x7fe078009b50 comp rx=0 tx=0).stop 2023-09-30T22:25:59.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe08e956700 1 -- 172.21.15.32:0/2521721688 shutdown_connections 2023-09-30T22:25:59.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe08e956700 1 --2- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe088138d80 0x7fe08813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe08e956700 1 --2- 172.21.15.32:0/2521721688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe0880a0e00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.938+0000 7fe08e956700 1 -- 172.21.15.32:0/2521721688 >> 172.21.15.32:0/2521721688 conn(0x7fe0880a3050 msgr2=0x7fe0880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:25:59.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.939+0000 7fe08e956700 1 -- 172.21.15.32:0/2521721688 shutdown_connections 2023-09-30T22:25:59.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.939+0000 7fe08e956700 1 -- 172.21.15.32:0/2521721688 wait complete. 2023-09-30T22:25:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.939+0000 7fe08e956700 1 Processor -- start 2023-09-30T22:25:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 -- start start 2023-09-30T22:25:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe08809d630 0x7fe088133670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe088133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe088138d80 0x7fe0881340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe088136e60 con 0x7fe0880a0a30 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe088136140 con 0x7fe088138d80 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08e956700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe088134ad0 con 0x7fe08809d630 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe088133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe088133bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53530/0 (socket says 172.21.15.32:53530) 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe0877fe700 1 -- 172.21.15.32:0/458194039 learned_addr learned my addr 172.21.15.32:0/458194039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe087fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe08809d630 0x7fe088133670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.940+0000 7fe08cef3700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe088138d80 0x7fe0881340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe087fff700 1 -- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe088138d80 msgr2=0x7fe0881340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe087fff700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe088138d80 0x7fe0881340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe087fff700 1 -- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 msgr2=0x7fe088133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe087fff700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe088133bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe087fff700 1 -- 172.21.15.32:0/458194039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe080009a70 con 0x7fe08809d630 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe0877fe700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe088133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe087fff700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe08809d630 0x7fe088133670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe078016000 tx=0x7fe078004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078004380 con 0x7fe08809d630 2023-09-30T22:25:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07801f070 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0780094a0 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0780036a0 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.941+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0880a0410 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.942+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe078021030 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.942+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe078040910 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.942+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe088136b00 con 0x7fe08809d630 2023-09-30T22:25:59.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:25:59.947+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe078015070 con 0x7fe08809d630 2023-09-30T22:26:00.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.130+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f75b80a28e0 con 0x7f75b8133fa0 2023-09-30T22:26:00.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.130+0000 7f75af7fe700 1 -- 172.21.15.32:0/3172763809 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75b001bb30 con 0x7f75b8133fa0 2023-09-30T22:26:00.130 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:00.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b8133fa0 msgr2=0x7f75b8123210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b8133fa0 0x7f75b8123210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c460 tx=0x7f75b000c770 comp rx=0 tx=0).stop 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 shutdown_connections 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75b8133fa0 0x7f75b8123210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75b80a0a30 0x7f75b8122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 --2- 172.21.15.32:0/3172763809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b80a1340 0x7f75b8122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.132+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 >> 172.21.15.32:0/3172763809 conn(0x7f75b80a6550 msgr2=0x7f75b80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.133+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 shutdown_connections 2023-09-30T22:26:00.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.133+0000 7f75bd0aa700 1 -- 172.21.15.32:0/3172763809 wait complete. 2023-09-30T22:26:00.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:00.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.167+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe0880747f0 con 0x7fe08809d630 2023-09-30T22:26:00.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.168+0000 7fe084ff9700 1 -- 172.21.15.32:0/458194039 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe078023790 con 0x7fe08809d630 2023-09-30T22:26:00.167 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe08809d630 msgr2=0x7fe088133670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe08809d630 0x7fe088133670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe078016000 tx=0x7fe078004750 comp rx=0 tx=0).stop 2023-09-30T22:26:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 shutdown_connections 2023-09-30T22:26:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe088138d80 0x7fe0881340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe08809d630 0x7fe088133670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 --2- 172.21.15.32:0/458194039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0880a0a30 0x7fe088133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 >> 172.21.15.32:0/458194039 conn(0x7fe0880a3050 msgr2=0x7fe08813af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 shutdown_connections 2023-09-30T22:26:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.170+0000 7fe08e956700 1 -- 172.21.15.32:0/458194039 wait complete. 2023-09-30T22:26:00.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:00.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:00.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.479+0000 7fa84a68c700 1 Processor -- start 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.479+0000 7fa84a68c700 1 -- start start 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa84a68c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa8440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa84a68c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8440a1340 0x7fa844133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa84a68c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84409edd0 con 0x7fa844133fa0 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa84a68c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84409e1f0 con 0x7fa8440a0a30 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa84a68c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa84409dc50 con 0x7fa8440a1340 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa848c29700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa844133fa0 0x7fa84413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43226/0 (socket says 172.21.15.32:43226) 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa843fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa8440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa848c29700 1 -- 172.21.15.32:0/2591711683 learned_addr learned my addr 172.21.15.32:0/2591711683 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa8437fe700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8440a1340 0x7fa844133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa842ffd700 1 -- 172.21.15.32:0/2591711683 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230333760 0 0) 0x7fa84409edd0 con 0x7fa844133fa0 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa842ffd700 1 -- 172.21.15.32:0/2591711683 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa82c0035b0 con 0x7fa844133fa0 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa8437fe700 1 -- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 msgr2=0x7fa8440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa8437fe700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa8440a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa8437fe700 1 -- 172.21.15.32:0/2591711683 >> v1:172.21.15.32:6789/0 conn(0x7fa844133fa0 legacy=0x7fa84413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa8437fe700 1 -- 172.21.15.32:0/2591711683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8440a1fe0 con 0x7fa8440a1340 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa843fff700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa8440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.480+0000 7fa8437fe700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8440a1340 0x7fa844133a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa838009870 tx=0x7fa838009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92cbff6cd2d5b79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa842ffd700 1 -- 172.21.15.32:0/2591711683 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838011830 con 0x7fa8440a1340 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa842ffd700 1 -- 172.21.15.32:0/2591711683 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838011990 con 0x7fa8440a1340 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa842ffd700 1 -- 172.21.15.32:0/2591711683 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838011d20 con 0x7fa8440a1340 2023-09-30T22:26:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 -- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8440a1340 msgr2=0x7fa844133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8440a1340 0x7fa844133a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa838009870 tx=0x7fa838009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 -- 172.21.15.32:0/2591711683 shutdown_connections 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa8440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2591711683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8440a1340 0x7fa844133a60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 -- 172.21.15.32:0/2591711683 >> 172.21.15.32:0/2591711683 conn(0x7fa8440a6550 msgr2=0x7fa8440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 -- 172.21.15.32:0/2591711683 shutdown_connections 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.481+0000 7fa84a68c700 1 -- 172.21.15.32:0/2591711683 wait complete. 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.482+0000 7fa84a68c700 1 Processor -- start 2023-09-30T22:26:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 -- start start 2023-09-30T22:26:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa844122750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 0x7fa844122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa844133fa0 0x7fa8441231d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844133780 con 0x7fa8440a1340 2023-09-30T22:26:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440a3680 con 0x7fa8440a0a30 2023-09-30T22:26:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa84a68c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa844124470 con 0x7fa844133fa0 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa843fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa844122750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 0x7fa844122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa843fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa844122750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46864/0 (socket says 172.21.15.32:46864) 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 0x7fa844122c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54424/0 (socket says 172.21.15.32:54424) 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa843fff700 1 -- 172.21.15.32:0/2105843010 learned_addr learned my addr 172.21.15.32:0/2105843010 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.483+0000 7fa848c29700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa844133fa0 0x7fa8441231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa843fff700 1 -- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa844133fa0 msgr2=0x7fa8441231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa843fff700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa844133fa0 0x7fa8441231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa843fff700 1 -- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 msgr2=0x7fa844122c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa843fff700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 0x7fa844122c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa843fff700 1 -- 172.21.15.32:0/2105843010 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8340094a0 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa8437fe700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 0x7fa844122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa843fff700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa844122750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa83400ebe0 tx=0x7fa83400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400c850 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400c9b0 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8380094a0 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400cd40 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.484+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8440a25e0 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.485+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa834013470 con 0x7fa8440a0a30 2023-09-30T22:26:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.485+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa83403f740 con 0x7fa8440a0a30 2023-09-30T22:26:00.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.485+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa844074ea0 con 0x7fa8440a0a30 2023-09-30T22:26:00.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.490+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa834013720 con 0x7fa8440a0a30 2023-09-30T22:26:00.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.517+0000 7f56d088b700 1 Processor -- start 2023-09-30T22:26:00.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.517+0000 7f56d088b700 1 -- start start 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56d088b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc0a0a30 0x7f56cc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56d088b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56cc138d80 0x7f56cc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56d088b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0a2a70 con 0x7f56cc09d630 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56d088b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc0a01d0 con 0x7f56cc138d80 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56d088b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc09e2b0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56c9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56cc09d630 0x7f56cc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43228/0 (socket says 172.21.15.32:43228) 2023-09-30T22:26:00.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56cad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56cc138d80 0x7f56cc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56c9d9b700 1 -- 172.21.15.32:0/2746326942 learned_addr learned my addr 172.21.15.32:0/2746326942 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.518+0000 7f56ca59c700 1 --2- 172.21.15.32:0/2746326942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc0a0a30 0x7f56cc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56ca59c700 1 -- 172.21.15.32:0/2746326942 >> v2:172.21.15.32:3300/0 conn(0x7f56cc138d80 msgr2=0x7f56cc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56ca59c700 1 --2- 172.21.15.32:0/2746326942 >> v2:172.21.15.32:3300/0 conn(0x7f56cc138d80 0x7f56cc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56ca59c700 1 -- 172.21.15.32:0/2746326942 >> v1:172.21.15.32:6789/0 conn(0x7f56cc09d630 legacy=0x7f56cc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56ca59c700 1 -- 172.21.15.32:0/2746326942 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc09dc50 con 0x7f56cc0a0a30 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56cad9d700 1 --2- 172.21.15.32:0/2746326942 >> v2:172.21.15.32:3300/0 conn(0x7f56cc138d80 0x7f56cc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56c959a700 1 -- 172.21.15.32:0/2746326942 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838443551 0 0) 0x7f56cc0a2a70 con 0x7f56cc09d630 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56ca59c700 1 --2- 172.21.15.32:0/2746326942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc0a0a30 0x7f56cc0a0e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009840 tx=0x7f56bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae0cd896e2a7c1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56c959a700 1 -- 172.21.15.32:0/2746326942 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc011830 con 0x7f56cc0a0a30 2023-09-30T22:26:00.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56c959a700 1 -- 172.21.15.32:0/2746326942 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc011990 con 0x7f56cc0a0a30 2023-09-30T22:26:00.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.519+0000 7f56c959a700 1 -- 172.21.15.32:0/2746326942 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc011d20 con 0x7f56cc0a0a30 2023-09-30T22:26:00.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 -- 172.21.15.32:0/2746326942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc0a0a30 msgr2=0x7f56cc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 --2- 172.21.15.32:0/2746326942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc0a0a30 0x7f56cc0a0e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009840 tx=0x7f56bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:00.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 -- 172.21.15.32:0/2746326942 shutdown_connections 2023-09-30T22:26:00.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 --2- 172.21.15.32:0/2746326942 >> v2:172.21.15.32:3300/0 conn(0x7f56cc138d80 0x7f56cc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 --2- 172.21.15.32:0/2746326942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc0a0a30 0x7f56cc0a0e00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 -- 172.21.15.32:0/2746326942 >> 172.21.15.32:0/2746326942 conn(0x7f56cc0a3050 msgr2=0x7f56cc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 -- 172.21.15.32:0/2746326942 shutdown_connections 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.520+0000 7f56d088b700 1 -- 172.21.15.32:0/2746326942 wait complete. 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.521+0000 7f56d088b700 1 Processor -- start 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.521+0000 7f56d088b700 1 -- start start 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56d088b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc09d630 0x7f56cc1337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56d088b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 0x7f56cc133d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56d088b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 0x7f56cc134260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56d088b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc135a40 con 0x7f56cc09d630 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56d088b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc136d60 con 0x7f56cc0a0a30 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56d088b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc136a90 con 0x7f56cc138d80 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 0x7f56cc133d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56cad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 0x7f56cc134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 0x7f56cc133d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46868/0 (socket says 172.21.15.32:46868) 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56cad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 0x7f56cc134260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49944/0 (socket says 172.21.15.32:49944) 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 -- 172.21.15.32:0/3379222452 learned_addr learned my addr 172.21.15.32:0/3379222452 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56ca59c700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc09d630 0x7f56cc1337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 -- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 msgr2=0x7f56cc134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 0x7f56cc134260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 -- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc09d630 msgr2=0x7f56cc1337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc09d630 0x7f56cc1337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 -- 172.21.15.32:0/3379222452 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4009a70 con 0x7f56cc0a0a30 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56cad9d700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 0x7f56cc134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56ca59c700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc09d630 0x7f56cc1337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.522+0000 7f56c9d9b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 0x7f56cc133d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56b4001eb0 tx=0x7f56b400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.523+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4004c80 con 0x7f56cc0a0a30 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.523+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b4004de0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.523+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc0094a0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.523+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4007310 con 0x7f56cc0a0a30 2023-09-30T22:26:00.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.523+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc13ae00 con 0x7f56cc0a0a30 2023-09-30T22:26:00.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.524+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc0747f0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.524+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56b4007470 con 0x7f56cc0a0a30 2023-09-30T22:26:00.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.524+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f56b403f8b0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.529+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56b40133c0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.696+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa8440a28e0 con 0x7fa8440a0a30 2023-09-30T22:26:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.696+0000 7fa840ff9700 1 -- 172.21.15.32:0/2105843010 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa834021020 con 0x7fa8440a0a30 2023-09-30T22:26:00.696 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 msgr2=0x7fa844122750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa844122750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa83400ebe0 tx=0x7fa83400eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 shutdown_connections 2023-09-30T22:26:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa844133fa0 0x7fa8441231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8440a0a30 0x7fa844122750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 --2- 172.21.15.32:0/2105843010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8440a1340 0x7fa844122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 >> 172.21.15.32:0/2105843010 conn(0x7fa8440a6550 msgr2=0x7fa84411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 shutdown_connections 2023-09-30T22:26:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.698+0000 7fa84a68c700 1 -- 172.21.15.32:0/2105843010 wait complete. 2023-09-30T22:26:00.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:00.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.732+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f56cc135be0 con 0x7f56cc0a0a30 2023-09-30T22:26:00.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.732+0000 7f56c2ffd700 1 -- 172.21.15.32:0/3379222452 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56b401b340 con 0x7f56cc0a0a30 2023-09-30T22:26:00.732 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.734+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 msgr2=0x7f56cc133d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.734+0000 7f56d088b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 0x7f56cc133d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56b4001eb0 tx=0x7f56b400c800 comp rx=0 tx=0).stop 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 shutdown_connections 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56cc138d80 0x7f56cc134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56cc09d630 0x7f56cc1337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 --2- 172.21.15.32:0/3379222452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56cc0a0a30 0x7f56cc133d20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 >> 172.21.15.32:0/3379222452 conn(0x7f56cc0a3050 msgr2=0x7f56cc0a69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:00.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 shutdown_connections 2023-09-30T22:26:00.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:00.735+0000 7f56d088b700 1 -- 172.21.15.32:0/3379222452 wait complete. 2023-09-30T22:26:00.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:01.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:01.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.292+0000 7fb545929700 1 Processor -- start 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.292+0000 7fb545929700 1 -- start start 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb545929700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb540138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb545929700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54013b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb545929700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54009ea10 con 0x7fb5400a09c0 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb545929700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54009e210 con 0x7fb54009d5c0 2023-09-30T22:26:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb545929700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400a26f0 con 0x7fb5400a14a0 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54013b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb5400a09c0 0x7fb5400a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43234/0 (socket says 172.21.15.32:43234) 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb540138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53effd700 1 -- 172.21.15.32:0/1348375010 learned_addr learned my addr 172.21.15.32:0/1348375010 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53f7fe700 1 -- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 msgr2=0x7fb540138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53dffb700 1 -- 172.21.15.32:0/1348375010 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865123492 0 0) 0x7fb54009ea10 con 0x7fb5400a09c0 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.293+0000 7fb53f7fe700 1 --2- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb540138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53f7fe700 1 -- 172.21.15.32:0/1348375010 >> v1:172.21.15.32:6789/0 conn(0x7fb5400a09c0 legacy=0x7fb5400a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53f7fe700 1 -- 172.21.15.32:0/1348375010 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54009db70 con 0x7fb5400a14a0 2023-09-30T22:26:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53e7fc700 1 --2- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb540138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53f7fe700 1 --2- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54013b2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb538009870 tx=0x7fb538009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb3cf1222f323a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53dffb700 1 -- 172.21.15.32:0/1348375010 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538011830 con 0x7fb5400a14a0 2023-09-30T22:26:01.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53dffb700 1 -- 172.21.15.32:0/1348375010 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538011990 con 0x7fb5400a14a0 2023-09-30T22:26:01.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.294+0000 7fb53dffb700 1 -- 172.21.15.32:0/1348375010 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538011d20 con 0x7fb5400a14a0 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 -- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 msgr2=0x7fb54013b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 --2- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54013b2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb538009870 tx=0x7fb538009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 -- 172.21.15.32:0/1348375010 shutdown_connections 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 --2- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54013b2c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 --2- 172.21.15.32:0/1348375010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb540138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 -- 172.21.15.32:0/1348375010 >> 172.21.15.32:0/1348375010 conn(0x7fb5400a2fe0 msgr2=0x7fb5400a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 -- 172.21.15.32:0/1348375010 shutdown_connections 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.295+0000 7fb545929700 1 -- 172.21.15.32:0/1348375010 wait complete. 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.296+0000 7fb545929700 1 Processor -- start 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.296+0000 7fb545929700 1 -- start start 2023-09-30T22:26:01.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb545929700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb54011c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb545929700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 0x7fb54011ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb545929700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54011d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb545929700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54011e3f0 con 0x7fb5400a09c0 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb545929700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54011fd10 con 0x7fb54009d5c0 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb545929700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb54009c0f0 con 0x7fb5400a14a0 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 0x7fb54011ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 0x7fb54011ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54448/0 (socket says 172.21.15.32:54448) 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb54011c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53e7fc700 1 -- 172.21.15.32:0/1297005365 learned_addr learned my addr 172.21.15.32:0/1297005365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53f7fe700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54011d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 -- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 msgr2=0x7fb54011d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54011d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 -- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 msgr2=0x7fb54011ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 0x7fb54011ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 -- 172.21.15.32:0/1297005365 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340094a0 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53e7fc700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 0x7fb54011ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.297+0000 7fb53effd700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb54011c7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb53000cbc0 tx=0x7fb53000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.298+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53000aaa0 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.298+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53000ac00 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.298+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5380094a0 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.298+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300070d0 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.298+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb540134710 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.299+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb530007230 con 0x7fb54009d5c0 2023-09-30T22:26:01.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.299+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb53003f120 con 0x7fb54009d5c0 2023-09-30T22:26:01.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.299+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5400747f0 con 0x7fb54009d5c0 2023-09-30T22:26:01.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.304+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb530018030 con 0x7fb54009d5c0 2023-09-30T22:26:01.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.341+0000 7f14434cc700 1 Processor -- start 2023-09-30T22:26:01.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.341+0000 7f14434cc700 1 -- start start 2023-09-30T22:26:01.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f14434cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 0x7f143c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f14434cc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f143c09e000 0x7f143c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f14434cc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 0x7f143c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f14434cc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c06b460 con 0x7f143c09e000 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f14434cc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c09b120 con 0x7f143c09e410 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f14434cc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f143c09a7e0 con 0x7f143c09d6f0 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441a69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 0x7f143c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 0x7f143c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441a69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 0x7f143c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46898/0 (socket says 172.21.15.32:46898) 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441a69700 1 -- 172.21.15.32:0/3628085063 learned_addr learned my addr 172.21.15.32:0/3628085063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 0x7f143c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2023-09-30T22:26:01.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1440a67700 1 --2- 172.21.15.32:0/3628085063 >> v2:172.21.15.32:3300/0 conn(0x7f143c09e000 0x7f143c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 -- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 msgr2=0x7f143c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 --2- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 0x7f143c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 -- 172.21.15.32:0/3628085063 >> v2:172.21.15.32:3300/0 conn(0x7f143c09e000 msgr2=0x7f143c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 --2- 172.21.15.32:0/3628085063 >> v2:172.21.15.32:3300/0 conn(0x7f143c09e000 0x7f143c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 -- 172.21.15.32:0/3628085063 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c09a500 con 0x7f143c09d6f0 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441a69700 1 --2- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 0x7f143c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1440a67700 1 --2- 172.21.15.32:0/3628085063 >> v2:172.21.15.32:3300/0 conn(0x7f143c09e000 0x7f143c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.342+0000 7f1441268700 1 --2- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 0x7f143c09dac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1438009870 tx=0x7f1438009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee0a09e3f24fb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f1433fff700 1 -- 172.21.15.32:0/3628085063 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438011830 con 0x7f143c09d6f0 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f1433fff700 1 -- 172.21.15.32:0/3628085063 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438011990 con 0x7f143c09d6f0 2023-09-30T22:26:01.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f1433fff700 1 -- 172.21.15.32:0/3628085063 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438011d20 con 0x7f143c09d6f0 2023-09-30T22:26:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 -- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 msgr2=0x7f143c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 --2- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 0x7f143c09dac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1438009870 tx=0x7f1438009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 -- 172.21.15.32:0/3628085063 shutdown_connections 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 --2- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e410 0x7f143c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 --2- 172.21.15.32:0/3628085063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09d6f0 0x7f143c09dac0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 --2- 172.21.15.32:0/3628085063 >> v2:172.21.15.32:3300/0 conn(0x7f143c09e000 0x7f143c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 -- 172.21.15.32:0/3628085063 >> 172.21.15.32:0/3628085063 conn(0x7f143c066f50 msgr2=0x7f143c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 -- 172.21.15.32:0/3628085063 shutdown_connections 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.343+0000 7f14434cc700 1 -- 172.21.15.32:0/3628085063 wait complete. 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.344+0000 7f14434cc700 1 Processor -- start 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.344+0000 7f14434cc700 1 -- start start 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f14434cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c09d6f0 0x7f143c1a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f14434cc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 0x7f143c1a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f14434cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09e410 0x7f143c1c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f14434cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1a81a0 con 0x7f143c09d6f0 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f14434cc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1a8380 con 0x7f143c09e000 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f14434cc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1b91f0 con 0x7f143c09e410 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 0x7f143c1a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 0x7f143c1a5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46904/0 (socket says 172.21.15.32:46904) 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 -- 172.21.15.32:0/2104097993 learned_addr learned my addr 172.21.15.32:0/2104097993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1441a69700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09e410 0x7f143c1c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1441268700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c09d6f0 0x7f143c1a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 -- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09e410 msgr2=0x7f143c1c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09e410 0x7f143c1c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 -- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c09d6f0 msgr2=0x7f143c1a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c09d6f0 0x7f143c1a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.345+0000 7f1440a67700 1 -- 172.21.15.32:0/2104097993 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0094a0 con 0x7f143c09e000 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.346+0000 7f1440a67700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 0x7f143c1a5800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f142c00ec10 tx=0x7f142c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.346+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00c850 con 0x7f143c09e000 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.346+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00c9b0 con 0x7f143c09e000 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.346+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1434009b30 con 0x7f143c09e000 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.346+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00cd20 con 0x7f143c09e000 2023-09-30T22:26:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.346+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14380094a0 con 0x7f143c09e000 2023-09-30T22:26:01.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.347+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f142c013470 con 0x7f143c09e000 2023-09-30T22:26:01.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.347+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f142c03fc40 con 0x7f143c09e000 2023-09-30T22:26:01.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.347+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c1a5d40 con 0x7f143c09e000 2023-09-30T22:26:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.352+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f142c013720 con 0x7f143c09e000 2023-09-30T22:26:01.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.523+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb540072980 con 0x7fb54009d5c0 2023-09-30T22:26:01.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.523+0000 7fb527fff700 1 -- 172.21.15.32:0/1297005365 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5300074e0 con 0x7fb54009d5c0 2023-09-30T22:26:01.523 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 msgr2=0x7fb54011c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb54011c7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb53000cbc0 tx=0x7fb53000ced0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 shutdown_connections 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb5400a14a0 0x7fb54011d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb54009d5c0 0x7fb54011c7b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 --2- 172.21.15.32:0/1297005365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5400a09c0 0x7fb54011ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 >> 172.21.15.32:0/1297005365 conn(0x7fb5400a2fe0 msgr2=0x7fb540136e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 shutdown_connections 2023-09-30T22:26:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.526+0000 7fb545929700 1 -- 172.21.15.32:0/1297005365 wait complete. 2023-09-30T22:26:01.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.558+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f143c074ea0 con 0x7f143c09e000 2023-09-30T22:26:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.558+0000 7f1431ffb700 1 -- 172.21.15.32:0/2104097993 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f142c02b090 con 0x7f143c09e000 2023-09-30T22:26:01.558 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 msgr2=0x7f143c1a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 0x7f143c1a5800 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f142c00ec10 tx=0x7f142c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 shutdown_connections 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f143c09e410 0x7f143c1c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c09d6f0 0x7f143c1a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 --2- 172.21.15.32:0/2104097993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f143c09e000 0x7f143c1a5800 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 >> 172.21.15.32:0/2104097993 conn(0x7f143c066f50 msgr2=0x7f143c0a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 shutdown_connections 2023-09-30T22:26:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.561+0000 7f14434cc700 1 -- 172.21.15.32:0/2104097993 wait complete. 2023-09-30T22:26:01.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:01.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:01.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.858+0000 7fcdc68b1700 1 Processor -- start 2023-09-30T22:26:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.858+0000 7fcdc68b1700 1 -- start start 2023-09-30T22:26:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc68b1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdc00a0a30 0x7fcdc00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc68b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc00a1340 0x7fcdc0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc68b1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc009edd0 con 0x7fcdc0133fa0 2023-09-30T22:26:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc68b1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc009e1f0 con 0x7fcdc00a0a30 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc68b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc009dc50 con 0x7fcdc00a1340 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdbffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdc00a0a30 0x7fcdc00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdbf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc00a1340 0x7fcdc0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc4e4e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcdc0133fa0 0x7fcdc013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43248/0 (socket says 172.21.15.32:43248) 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdc4e4e700 1 -- 172.21.15.32:0/3389803531 learned_addr learned my addr 172.21.15.32:0/3389803531 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdbffff700 1 -- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc00a1340 msgr2=0x7fcdc0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdbffff700 1 --2- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc00a1340 0x7fcdc0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdbffff700 1 -- 172.21.15.32:0/3389803531 >> v1:172.21.15.32:6789/0 conn(0x7fcdc0133fa0 legacy=0x7fcdc013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.860+0000 7fcdbffff700 1 -- 172.21.15.32:0/3389803531 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00a1fe0 con 0x7fcdc00a0a30 2023-09-30T22:26:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.859+0000 7fcdbeffd700 1 -- 172.21.15.32:0/3389803531 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960261062 0 0) 0x7fcdc009edd0 con 0x7fcdc0133fa0 2023-09-30T22:26:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.860+0000 7fcdbf7fe700 1 --2- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc00a1340 0x7fcdc0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.860+0000 7fcdbffff700 1 --2- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a0a30 0x7fcdc00a0e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009840 tx=0x7fcdb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96a3bf395664dfa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.860+0000 7fcdbeffd700 1 -- 172.21.15.32:0/3389803531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0011830 con 0x7fcdc00a0a30 2023-09-30T22:26:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.860+0000 7fcdbeffd700 1 -- 172.21.15.32:0/3389803531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb0011990 con 0x7fcdc00a0a30 2023-09-30T22:26:01.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.860+0000 7fcdbeffd700 1 -- 172.21.15.32:0/3389803531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0011d20 con 0x7fcdc00a0a30 2023-09-30T22:26:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 -- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a0a30 msgr2=0x7fcdc00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a0a30 0x7fcdc00a0e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009840 tx=0x7fcdb0009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 -- 172.21.15.32:0/3389803531 shutdown_connections 2023-09-30T22:26:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a0a30 0x7fcdc00a0e00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/3389803531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc00a1340 0x7fcdc0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 -- 172.21.15.32:0/3389803531 >> 172.21.15.32:0/3389803531 conn(0x7fcdc00a6550 msgr2=0x7fcdc00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 -- 172.21.15.32:0/3389803531 shutdown_connections 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.861+0000 7fcdc68b1700 1 -- 172.21.15.32:0/3389803531 wait complete. 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.863+0000 7fcdc68b1700 1 Processor -- start 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.863+0000 7fcdc68b1700 1 -- start start 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc68b1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 0x7fcdc0122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc68b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a1340 0x7fcdc0122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc68b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 0x7fcdc0123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc68b1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0124cb0 con 0x7fcdc00a1340 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc68b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0125d80 con 0x7fcdc0133fa0 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc68b1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0124520 con 0x7fcdc00a0a30 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc4e4e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 0x7fcdc0123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 0x7fcdc0122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a1340 0x7fcdc0122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc4e4e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 0x7fcdc0123310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46910/0 (socket says 172.21.15.32:46910) 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc4e4e700 1 -- 172.21.15.32:0/121570626 learned_addr learned my addr 172.21.15.32:0/121570626 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 0x7fcdc0122890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50000/0 (socket says 172.21.15.32:50000) 2023-09-30T22:26:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 -- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 msgr2=0x7fcdc0123310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 0x7fcdc0123310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 -- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a1340 msgr2=0x7fcdc0122dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a1340 0x7fcdc0122dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 -- 172.21.15.32:0/121570626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40094a0 con 0x7fcdc00a0a30 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbf7fe700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a1340 0x7fcdc0122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdc4e4e700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 0x7fcdc0123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.864+0000 7fcdbffff700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 0x7fcdc0122890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0016000 tx=0x7fcdb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.865+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00036a0 con 0x7fcdc00a0a30 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.865+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb001f070 con 0x7fcdc00a0a30 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.865+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb00094a0 con 0x7fcdc00a0a30 2023-09-30T22:26:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.865+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0003a20 con 0x7fcdc00a0a30 2023-09-30T22:26:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.865+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc0126040 con 0x7fcdc00a0a30 2023-09-30T22:26:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.866+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc00a1d40 con 0x7fcdc00a0a30 2023-09-30T22:26:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.866+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdb0003b80 con 0x7fcdc00a0a30 2023-09-30T22:26:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.866+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcdb0021070 con 0x7fcdc00a0a30 2023-09-30T22:26:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.871+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdb002f3a0 con 0x7fcdc00a0a30 2023-09-30T22:26:01.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.900+0000 7fd35cd59700 1 Processor -- start 2023-09-30T22:26:01.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.900+0000 7fd35cd59700 1 -- start start 2023-09-30T22:26:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35cd59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 0x7fd35812f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35cd59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812c2f0 0x7fd35812fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35cd59700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35809a510 con 0x7fd35812cff0 2023-09-30T22:26:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35cd59700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35812c7d0 con 0x7fd35812c2f0 2023-09-30T22:26:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35cd59700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd35809c190 con 0x7fd35812cc20 2023-09-30T22:26:01.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 0x7fd35812f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd355d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812c2f0 0x7fd35812fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd356d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd35812cff0 0x7fd358136cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43260/0 (socket says 172.21.15.32:43260) 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd35659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 0x7fd35812f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50014/0 (socket says 172.21.15.32:50014) 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.901+0000 7fd356d9d700 1 -- 172.21.15.32:0/2882708136 learned_addr learned my addr 172.21.15.32:0/2882708136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35659c700 1 -- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812c2f0 msgr2=0x7fd35812fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35659c700 1 --2- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812c2f0 0x7fd35812fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35659c700 1 -- 172.21.15.32:0/2882708136 >> v1:172.21.15.32:6789/0 conn(0x7fd35812cff0 legacy=0x7fd358136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35659c700 1 -- 172.21.15.32:0/2882708136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35809b540 con 0x7fd35812cc20 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd355d9b700 1 --2- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812c2f0 0x7fd35812fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35559a700 1 -- 172.21.15.32:0/2882708136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028071419 0 0) 0x7fd35809a510 con 0x7fd35812cff0 2023-09-30T22:26:01.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35659c700 1 --2- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 0x7fd35812f930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd34800bb10 tx=0x7fd34800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52de93b8d9efba14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35559a700 1 -- 172.21.15.32:0/2882708136 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800dc70 con 0x7fd35812cc20 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35559a700 1 -- 172.21.15.32:0/2882708136 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34800ddd0 con 0x7fd35812cc20 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.902+0000 7fd35559a700 1 -- 172.21.15.32:0/2882708136 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348010690 con 0x7fd35812cc20 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 -- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 msgr2=0x7fd35812f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 --2- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 0x7fd35812f930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd34800bb10 tx=0x7fd34800be20 comp rx=0 tx=0).stop 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 -- 172.21.15.32:0/2882708136 shutdown_connections 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 --2- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cc20 0x7fd35812f930 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 --2- 172.21.15.32:0/2882708136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812c2f0 0x7fd35812fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 -- 172.21.15.32:0/2882708136 >> 172.21.15.32:0/2882708136 conn(0x7fd35809f470 msgr2=0x7fd3580a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:01.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 -- 172.21.15.32:0/2882708136 shutdown_connections 2023-09-30T22:26:01.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.903+0000 7fd35cd59700 1 -- 172.21.15.32:0/2882708136 wait complete. 2023-09-30T22:26:01.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 Processor -- start 2023-09-30T22:26:01.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 -- start start 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35812c2f0 0x7fd3580983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812cc20 0x7fd358098930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 0x7fd358128290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd358129f10 con 0x7fd35812c2f0 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35806ff90 con 0x7fd35812cc20 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd35cd59700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd35806ed00 con 0x7fd35812cff0 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.905+0000 7fd356d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 0x7fd358128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 0x7fd358128290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50030/0 (socket says 172.21.15.32:50030) 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 -- 172.21.15.32:0/3451621291 learned_addr learned my addr 172.21.15.32:0/3451621291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd355d9b700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812cc20 0x7fd358098930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd35659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35812c2f0 0x7fd3580983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 -- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812cc20 msgr2=0x7fd358098930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812cc20 0x7fd358098930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 -- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35812c2f0 msgr2=0x7fd3580983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35812c2f0 0x7fd3580983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 -- 172.21.15.32:0/3451621291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c00a550 con 0x7fd35812cff0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd356d9d700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 0x7fd358128290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd350002230 tx=0x7fd35000cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000aac0 con 0x7fd35812cff0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000ac40 con 0x7fd35812cff0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34800b740 con 0x7fd35812cff0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3500070d0 con 0x7fd35812cff0 2023-09-30T22:26:01.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.906+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35806fa30 con 0x7fd35812cff0 2023-09-30T22:26:01.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.907+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd350007250 con 0x7fd35812cff0 2023-09-30T22:26:01.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.907+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd35003f070 con 0x7fd35812cff0 2023-09-30T22:26:01.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.907+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35806eec0 con 0x7fd35812cff0 2023-09-30T22:26:01.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:01.912+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd350018020 con 0x7fd35812cff0 2023-09-30T22:26:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.079+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcdc0074ea0 con 0x7fcdc00a0a30 2023-09-30T22:26:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.079+0000 7fcdbcff9700 1 -- 172.21.15.32:0/121570626 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdb002b030 con 0x7fcdc00a0a30 2023-09-30T22:26:02.080 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:02.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.081+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 msgr2=0x7fcdc0122890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.081+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 0x7fcdc0122890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0016000 tx=0x7fcdb0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:02.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 shutdown_connections 2023-09-30T22:26:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdc0133fa0 0x7fcdc0123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdc00a0a30 0x7fcdc0122890 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 --2- 172.21.15.32:0/121570626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdc00a1340 0x7fcdc0122dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 >> 172.21.15.32:0/121570626 conn(0x7fcdc00a6550 msgr2=0x7fcdc011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:02.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 shutdown_connections 2023-09-30T22:26:02.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.082+0000 7fcdc68b1700 1 -- 172.21.15.32:0/121570626 wait complete. 2023-09-30T22:26:02.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:02.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.116+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd3580747f0 con 0x7fd35812cff0 2023-09-30T22:26:02.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.117+0000 7fd346ffd700 1 -- 172.21.15.32:0/3451621291 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd350007500 con 0x7fd35812cff0 2023-09-30T22:26:02.117 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 msgr2=0x7fd358128290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 0x7fd358128290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd350002230 tx=0x7fd35000cf50 comp rx=0 tx=0).stop 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 shutdown_connections 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd35812cff0 0x7fd358128290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd35812c2f0 0x7fd3580983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 --2- 172.21.15.32:0/3451621291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd35812cc20 0x7fd358098930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 >> 172.21.15.32:0/3451621291 conn(0x7fd35809f470 msgr2=0x7fd35812a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 shutdown_connections 2023-09-30T22:26:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.119+0000 7fd35cd59700 1 -- 172.21.15.32:0/3451621291 wait complete. 2023-09-30T22:26:02.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:02.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:02.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.691+0000 7f1b308d0700 1 Processor -- start 2023-09-30T22:26:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.691+0000 7f1b308d0700 1 -- start start 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b308d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b2c0a0a30 0x7f1b2c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b308d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 0x7f1b2c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b308d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 0x7f1b2c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b308d0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2c0a2a70 con 0x7f1b2c0a0a30 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b308d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0a01d0 con 0x7f1b2c133930 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b308d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c09e2b0 con 0x7f1b2c09d630 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b29d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 0x7f1b2c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 0x7f1b2c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b2c0a0a30 0x7f1b2c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b29d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 0x7f1b2c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50042/0 (socket says 172.21.15.32:50042) 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 0x7f1b2c133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46950/0 (socket says 172.21.15.32:46950) 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b29d9b700 1 -- 172.21.15.32:0/183208538 learned_addr learned my addr 172.21.15.32:0/183208538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 -- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 msgr2=0x7f1b2c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 0x7f1b2c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 -- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 msgr2=0x7f1b2c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 0x7f1b2c133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 -- 172.21.15.32:0/183208538 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c09dc50 con 0x7f1b2c0a0a30 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b29d9b700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 0x7f1b2c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2ad9d700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 0x7f1b2c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.692+0000 7f1b2a59c700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c0a0a30 0x7f1b2c0a0e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009840 tx=0x7f1b1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ef3cd32d4b4d02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b2959a700 1 -- 172.21.15.32:0/183208538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c011830 con 0x7f1b2c0a0a30 2023-09-30T22:26:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b2959a700 1 -- 172.21.15.32:0/183208538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c011990 con 0x7f1b2c0a0a30 2023-09-30T22:26:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b2959a700 1 -- 172.21.15.32:0/183208538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c011d20 con 0x7f1b2c0a0a30 2023-09-30T22:26:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 -- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c0a0a30 msgr2=0x7f1b2c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c0a0a30 0x7f1b2c0a0e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009840 tx=0x7f1b1c009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 -- 172.21.15.32:0/183208538 shutdown_connections 2023-09-30T22:26:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c133930 0x7f1b2c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c0a0a30 0x7f1b2c0a0e00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 --2- 172.21.15.32:0/183208538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c09d630 0x7f1b2c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.693+0000 7f1b308d0700 1 -- 172.21.15.32:0/183208538 >> 172.21.15.32:0/183208538 conn(0x7f1b2c0a3050 msgr2=0x7f1b2c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.694+0000 7f1b308d0700 1 -- 172.21.15.32:0/183208538 shutdown_connections 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.694+0000 7f1b308d0700 1 -- 172.21.15.32:0/183208538 wait complete. 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 Processor -- start 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 -- start start 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 0x7f1b2c1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c0a0a30 0x7f1b2c1c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c133930 0x7f1b2c1c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c1332b0 con 0x7f1b2c09d630 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c1206b0 con 0x7f1b2c0a0a30 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b308d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0a52d0 con 0x7f1b2c133930 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b2a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 0x7f1b2c1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b2a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 0x7f1b2c1c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54522/0 (socket says 172.21.15.32:54522) 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b2a59c700 1 -- 172.21.15.32:0/3401448937 learned_addr learned my addr 172.21.15.32:0/3401448937 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b2ad9d700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c133930 0x7f1b2c1c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.695+0000 7f1b29d9b700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c0a0a30 0x7f1b2c1c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2ad9d700 1 -- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c0a0a30 msgr2=0x7f1b2c1c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2ad9d700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c0a0a30 0x7f1b2c1c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2ad9d700 1 -- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 msgr2=0x7f1b2c1c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2ad9d700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 0x7f1b2c1c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2ad9d700 1 -- 172.21.15.32:0/3401448937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20009550 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2a59c700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 0x7f1b2c1c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b2ad9d700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c133930 0x7f1b2c1c8b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009e50 tx=0x7f1b2400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400ca90 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2400f040 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b24009a70 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400ce50 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.696+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c0094a0 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.697+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b24014480 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.697+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c120d80 con 0x7f1b2c133930 2023-09-30T22:26:02.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.697+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1b2403fcc0 con 0x7f1b2c133930 2023-09-30T22:26:02.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.702+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b24012930 con 0x7f1b2c133930 2023-09-30T22:26:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.729+0000 7fb0fa7dc700 1 Processor -- start 2023-09-30T22:26:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.729+0000 7fb0fa7dc700 1 -- start start 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0fa7dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0f409e930 0x7fb0f409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0fa7dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0fa7dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f4070940 con 0x7fb0f409f240 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0fa7dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f406fd60 con 0x7fb0f409e930 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0fa7dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f406fa80 con 0x7fb0f409f650 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f4131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46980/0 (socket says 172.21.15.32:46980) 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb0f409f240 0x7fb0f41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43264/0 (socket says 172.21.15.32:43264) 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 -- 172.21.15.32:0/1340102623 learned_addr learned my addr 172.21.15.32:0/1340102623 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:02.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f3fff700 1 --2- 172.21.15.32:0/1340102623 >> v2:172.21.15.32:3300/0 conn(0x7fb0f409e930 0x7fb0f409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 -- 172.21.15.32:0/1340102623 >> v2:172.21.15.32:3300/0 conn(0x7fb0f409e930 msgr2=0x7fb0f409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 --2- 172.21.15.32:0/1340102623 >> v2:172.21.15.32:3300/0 conn(0x7fb0f409e930 0x7fb0f409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 -- 172.21.15.32:0/1340102623 >> v1:172.21.15.32:6789/0 conn(0x7fb0f409f240 legacy=0x7fb0f41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f2ffd700 1 -- 172.21.15.32:0/1340102623 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187104232 0 0) 0x7fb0f4070940 con 0x7fb0f409f240 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.730+0000 7fb0f8d79700 1 -- 172.21.15.32:0/1340102623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f409fb60 con 0x7fb0f409f650 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0f8d79700 1 --2- 172.21.15.32:0/1340102623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f4131f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009e50 tx=0x7fb0ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c576b0b8d2a023 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0f2ffd700 1 -- 172.21.15.32:0/1340102623 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00e070 con 0x7fb0f409f650 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0f2ffd700 1 -- 172.21.15.32:0/1340102623 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec00b910 con 0x7fb0f409f650 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0f2ffd700 1 -- 172.21.15.32:0/1340102623 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00bcc0 con 0x7fb0f409f650 2023-09-30T22:26:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1340102623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 msgr2=0x7fb0f4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1340102623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f4131f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009e50 tx=0x7fb0ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1340102623 shutdown_connections 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1340102623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f4131f00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1340102623 >> v2:172.21.15.32:3300/0 conn(0x7fb0f409e930 0x7fb0f409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1340102623 >> 172.21.15.32:0/1340102623 conn(0x7fb0f40a4210 msgr2=0x7fb0f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.731+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1340102623 shutdown_connections 2023-09-30T22:26:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.732+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1340102623 wait complete. 2023-09-30T22:26:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.732+0000 7fb0fa7dc700 1 Processor -- start 2023-09-30T22:26:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 -- start start 2023-09-30T22:26:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 0x7fb0f406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f409f240 0x7fb0f406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f406d4c0 con 0x7fb0f409f240 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41a9460 con 0x7fb0f409f650 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0fa7dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41a91a0 con 0x7fb0f409e930 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 0x7fb0f406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0f37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f409f240 0x7fb0f406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0f3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 0x7fb0f406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50074/0 (socket says 172.21.15.32:50074) 2023-09-30T22:26:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0f3fff700 1 -- 172.21.15.32:0/1916080080 learned_addr learned my addr 172.21.15.32:0/1916080080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.733+0000 7fb0f8d79700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f37fe700 1 -- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 msgr2=0x7fb0f406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f37fe700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 0x7fb0f406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f37fe700 1 -- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 msgr2=0x7fb0f406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f37fe700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f37fe700 1 -- 172.21.15.32:0/1916080080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec009b30 con 0x7fb0f409f240 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f3fff700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 0x7fb0f406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f37fe700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f409f240 0x7fb0f406bdf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c470 tx=0x7fb0e800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8004c80 con 0x7fb0f409f240 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e8004de0 con 0x7fb0f409f240 2023-09-30T22:26:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f40a0190 con 0x7fb0f409f240 2023-09-30T22:26:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8007310 con 0x7fb0f409f240 2023-09-30T22:26:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.734+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f4071d20 con 0x7fb0f409f240 2023-09-30T22:26:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.735+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0e8007470 con 0x7fb0f409f240 2023-09-30T22:26:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.735+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb0e803f200 con 0x7fb0f409f240 2023-09-30T22:26:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.736+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f41a0b30 con 0x7fb0f409f240 2023-09-30T22:26:02.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.741+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0e8007720 con 0x7fb0f409f240 2023-09-30T22:26:02.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.909+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b2c0747f0 con 0x7f1b2c133930 2023-09-30T22:26:02.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.910+0000 7f1b1affd700 1 -- 172.21.15.32:0/3401448937 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b24012340 con 0x7f1b2c133930 2023-09-30T22:26:02.910 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c133930 msgr2=0x7f1b2c1c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c133930 0x7f1b2c1c8b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009e50 tx=0x7f1b2400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 shutdown_connections 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b2c133930 0x7f1b2c1c8b10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b2c09d630 0x7f1b2c1c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 --2- 172.21.15.32:0/3401448937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b2c0a0a30 0x7f1b2c1c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 >> 172.21.15.32:0/3401448937 conn(0x7f1b2c0a3050 msgr2=0x7f1b2c1bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:02.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 shutdown_connections 2023-09-30T22:26:02.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.912+0000 7f1b308d0700 1 -- 172.21.15.32:0/3401448937 wait complete. 2023-09-30T22:26:02.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:02.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.945+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb0f4074ea0 con 0x7fb0f409f240 2023-09-30T22:26:02.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.946+0000 7fb0f0ff9700 1 -- 172.21.15.32:0/1916080080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0e8011e80 con 0x7fb0f409f240 2023-09-30T22:26:02.946 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:02.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f409f240 msgr2=0x7fb0f406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f409f240 0x7fb0f406bdf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c470 tx=0x7fb0e800c780 comp rx=0 tx=0).stop 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 shutdown_connections 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0f409f650 0x7fb0f406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0f409e930 0x7fb0f406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 --2- 172.21.15.32:0/1916080080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0f409f240 0x7fb0f406bdf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 >> 172.21.15.32:0/1916080080 conn(0x7fb0f40a4210 msgr2=0x7fb0f41b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:02.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 shutdown_connections 2023-09-30T22:26:02.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:02.948+0000 7fb0fa7dc700 1 -- 172.21.15.32:0/1916080080 wait complete. 2023-09-30T22:26:02.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:03.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:03.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.244+0000 7f5ae4149700 1 Processor -- start 2023-09-30T22:26:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae4149700 1 -- start start 2023-09-30T22:26:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae4149700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae4149700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f240 0x7f5adc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae4149700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc070940 con 0x7f5adc09f650 2023-09-30T22:26:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae4149700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc06fd60 con 0x7f5adc09f240 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae4149700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc06fa80 con 0x7f5adc09e930 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae1ee5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae16e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f240 0x7f5adc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae1ee5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50082/0 (socket says 172.21.15.32:50082) 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.245+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/1742561369 learned_addr learned my addr 172.21.15.32:0/1742561369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f240 msgr2=0x7f5adc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae1ee5700 1 --2- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f240 0x7f5adc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/1742561369 >> v1:172.21.15.32:6789/0 conn(0x7f5adc09f650 legacy=0x7f5adc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/1742561369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc09fb60 con 0x7f5adc09e930 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae16e4700 1 --2- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f240 0x7f5adc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae1ee5700 1 --2- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc09ed00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009920 tx=0x7f5ad8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf0a68caece1f4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae0ee3700 1 -- 172.21.15.32:0/1742561369 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8011830 con 0x7f5adc09e930 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae0ee3700 1 -- 172.21.15.32:0/1742561369 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8011990 con 0x7f5adc09e930 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.246+0000 7f5ae0ee3700 1 -- 172.21.15.32:0/1742561369 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8011d20 con 0x7f5adc09e930 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 -- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 msgr2=0x7f5adc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 --2- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc09ed00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009920 tx=0x7f5ad8009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 -- 172.21.15.32:0/1742561369 shutdown_connections 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 --2- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc09ed00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 --2- 172.21.15.32:0/1742561369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f240 0x7f5adc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 -- 172.21.15.32:0/1742561369 >> 172.21.15.32:0/1742561369 conn(0x7f5adc0a4210 msgr2=0x7f5adc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 -- 172.21.15.32:0/1742561369 shutdown_connections 2023-09-30T22:26:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.247+0000 7f5ae4149700 1 -- 172.21.15.32:0/1742561369 wait complete. 2023-09-30T22:26:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.248+0000 7f5ae4149700 1 Processor -- start 2023-09-30T22:26:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.248+0000 7f5ae4149700 1 -- start start 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae4149700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae4149700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5adc09f240 0x7f5adc1a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae4149700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f650 0x7f5adc1a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae4149700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc071ce0 con 0x7f5adc09f240 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae4149700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc1a5810 con 0x7f5adc09f650 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae4149700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0a35c0 con 0x7f5adc09e930 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae1ee5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae26e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f650 0x7f5adc1a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae1ee5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc1a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50088/0 (socket says 172.21.15.32:50088) 2023-09-30T22:26:03.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/2420923676 learned_addr learned my addr 172.21.15.32:0/2420923676 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.249+0000 7f5ae16e4700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5adc09f240 0x7f5adc1a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f650 msgr2=0x7f5adc1a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae1ee5700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f650 0x7f5adc1a2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5adc09f240 msgr2=0x7f5adc1a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae1ee5700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5adc09f240 0x7f5adc1a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae1ee5700 1 -- 172.21.15.32:0/2420923676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc00d940 con 0x7f5adc09e930 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae26e6700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f650 0x7f5adc1a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae16e4700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5adc09f240 0x7f5adc1a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae1ee5700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc1a1cf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8004450 con 0x7f5adc09e930 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad80045b0 con 0x7f5adc09e930 2023-09-30T22:26:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae4149700 1 -- 172.21.15.32:0/2420923676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad8009550 con 0x7f5adc09e930 2023-09-30T22:26:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80038c0 con 0x7f5adc09e930 2023-09-30T22:26:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.250+0000 7f5ae4149700 1 -- 172.21.15.32:0/2420923676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc0a0190 con 0x7f5adc09e930 2023-09-30T22:26:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.251+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ad8003a20 con 0x7f5adc09e930 2023-09-30T22:26:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.251+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ad8021030 con 0x7f5adc09e930 2023-09-30T22:26:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.251+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab80052e0 con 0x7f5adc09e930 2023-09-30T22:26:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.256+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ad8015080 con 0x7f5adc09e930 2023-09-30T22:26:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.282+0000 7fdfcd8cf700 1 Processor -- start 2023-09-30T22:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.282+0000 7fdfcd8cf700 1 -- start start 2023-09-30T22:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfcd8cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 0x7fdfc809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfcd8cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdfc809f160 0x7fdfc809f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfcd8cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc8133b80 0x7fdfc8134020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfcd8cf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc806bec0 con 0x7fdfc809f160 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfcd8cf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80a0030 con 0x7fdfc809e850 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfcd8cf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806caa0 con 0x7fdfc8133b80 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 0x7fdfc809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 0x7fdfc809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47020/0 (socket says 172.21.15.32:47020) 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc6ffd700 1 -- 172.21.15.32:0/1879934538 learned_addr learned my addr 172.21.15.32:0/1879934538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:03.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc77fe700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc8133b80 0x7fdfc8134020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc67fc700 1 --2- 172.21.15.32:0/1879934538 >> v2:172.21.15.32:3300/0 conn(0x7fdfc809f160 0x7fdfc809f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc77fe700 1 -- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 msgr2=0x7fdfc809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc77fe700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 0x7fdfc809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc77fe700 1 -- 172.21.15.32:0/1879934538 >> v2:172.21.15.32:3300/0 conn(0x7fdfc809f160 msgr2=0x7fdfc809f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc77fe700 1 --2- 172.21.15.32:0/1879934538 >> v2:172.21.15.32:3300/0 conn(0x7fdfc809f160 0x7fdfc809f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc77fe700 1 -- 172.21.15.32:0/1879934538 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80a0830 con 0x7fdfc8133b80 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.283+0000 7fdfc6ffd700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 0x7fdfc809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfc77fe700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc8133b80 0x7fdfc8134020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009d90 tx=0x7fdfc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8a638ea4f45be9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:03.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfc5ffb700 1 -- 172.21.15.32:0/1879934538 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000e070 con 0x7fdfc8133b80 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfc5ffb700 1 -- 172.21.15.32:0/1879934538 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000bb20 con 0x7fdfc8133b80 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfc5ffb700 1 -- 172.21.15.32:0/1879934538 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0012430 con 0x7fdfc8133b80 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc8133b80 msgr2=0x7fdfc8134020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc8133b80 0x7fdfc8134020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009d90 tx=0x7fdfc000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/1879934538 shutdown_connections 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc809e850 0x7fdfc809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/1879934538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc8133b80 0x7fdfc8134020 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/1879934538 >> v2:172.21.15.32:3300/0 conn(0x7fdfc809f160 0x7fdfc809f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/1879934538 >> 172.21.15.32:0/1879934538 conn(0x7fdfc80a3f70 msgr2=0x7fdfc80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.284+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/1879934538 shutdown_connections 2023-09-30T22:26:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.285+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/1879934538 wait complete. 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.286+0000 7fdfcd8cf700 1 Processor -- start 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.286+0000 7fdfcd8cf700 1 -- start start 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfcd8cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc809e850 0x7fdfc811ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfcd8cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 0x7fdfc811ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfcd8cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc8133b80 0x7fdfc811f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfcd8cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8121c30 con 0x7fdfc809e850 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfcd8cf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc811ff80 con 0x7fdfc8133b80 2023-09-30T22:26:03.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfcd8cf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806ccc0 con 0x7fdfc809f160 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 0x7fdfc811ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 0x7fdfc811ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50094/0 (socket says 172.21.15.32:50094) 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 -- 172.21.15.32:0/3677751345 learned_addr learned my addr 172.21.15.32:0/3677751345 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc8133b80 0x7fdfc811f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc6ffd700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc809e850 0x7fdfc811ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 -- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc8133b80 msgr2=0x7fdfc811f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc8133b80 0x7fdfc811f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 -- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc809e850 msgr2=0x7fdfc811ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc809e850 0x7fdfc811ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.287+0000 7fdfc67fc700 1 -- 172.21.15.32:0/3677751345 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8009550 con 0x7fdfc809f160 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.288+0000 7fdfc67fc700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 0x7fdfc811ef70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000e9b0 tx=0x7fdfb000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:03.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.288+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0009710 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.288+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb000ee50 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.288+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc0009a70 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.288+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000ca70 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.288+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc81211b0 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.289+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8071d00 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.289+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdfb000cbd0 con 0x7fdfc809f160 2023-09-30T22:26:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.290+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdfb003fe00 con 0x7fdfc809f160 2023-09-30T22:26:03.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.298+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdfb0013470 con 0x7fdfc809f160 2023-09-30T22:26:03.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.468+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5ab8005590 con 0x7f5adc09e930 2023-09-30T22:26:03.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.469+0000 7f5ad27fc700 1 -- 172.21.15.32:0/2420923676 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ad802b020 con 0x7f5adc09e930 2023-09-30T22:26:03.469 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 msgr2=0x7f5adc1a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc1a1cf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad8004750 comp rx=0 tx=0).stop 2023-09-30T22:26:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 shutdown_connections 2023-09-30T22:26:03.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5adc09f650 0x7f5adc1a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5adc09e930 0x7f5adc1a1cf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 --2- 172.21.15.32:0/2420923676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5adc09f240 0x7f5adc1a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 >> 172.21.15.32:0/2420923676 conn(0x7f5adc0a4210 msgr2=0x7f5adc1b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 shutdown_connections 2023-09-30T22:26:03.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.471+0000 7f5abffff700 1 -- 172.21.15.32:0/2420923676 wait complete. 2023-09-30T22:26:03.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.505+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdfc80747f0 con 0x7fdfc809f160 2023-09-30T22:26:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.505+0000 7fdfbffff700 1 -- 172.21.15.32:0/3677751345 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdfb0021070 con 0x7fdfc809f160 2023-09-30T22:26:03.505 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 msgr2=0x7fdfc811ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 0x7fdfc811ef70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000e9b0 tx=0x7fdfb000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 shutdown_connections 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdfc8133b80 0x7fdfc811f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc809e850 0x7fdfc811ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 --2- 172.21.15.32:0/3677751345 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdfc809f160 0x7fdfc811ef70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.507+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 >> 172.21.15.32:0/3677751345 conn(0x7fdfc80a3f70 msgr2=0x7fdfc81332b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.508+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 shutdown_connections 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:03.508+0000 7fdfcd8cf700 1 -- 172.21.15.32:0/3677751345 wait complete. 2023-09-30T22:26:03.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:03.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:03.963 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 Processor -- start 2023-09-30T22:26:04.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 -- start start 2023-09-30T22:26:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28340a0a30 0x7f28340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 0x7f2834133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2834133fa0 0x7f2834134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283409edd0 con 0x7f28340a0a30 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283409e1f0 con 0x7f28340a1340 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.076+0000 7f283bc9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f283409dc50 con 0x7f2834133fa0 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 0x7f2834133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f283a239700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2834133fa0 0x7f2834134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839a38700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28340a0a30 0x7f28340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 0x7f2834133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47040/0 (socket says 172.21.15.32:47040) 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f283a239700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2834133fa0 0x7f2834134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50104/0 (socket says 172.21.15.32:50104) 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 -- 172.21.15.32:0/331125104 learned_addr learned my addr 172.21.15.32:0/331125104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 -- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2834133fa0 msgr2=0x7f2834134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 --2- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2834133fa0 0x7f2834134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 -- 172.21.15.32:0/331125104 >> v2:172.21.15.32:3300/0 conn(0x7f28340a0a30 msgr2=0x7f28340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 --2- 172.21.15.32:0/331125104 >> v2:172.21.15.32:3300/0 conn(0x7f28340a0a30 0x7f28340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 -- 172.21.15.32:0/331125104 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340a1fe0 con 0x7f28340a1340 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839a38700 1 --2- 172.21.15.32:0/331125104 >> v2:172.21.15.32:3300/0 conn(0x7f28340a0a30 0x7f28340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2839237700 1 --2- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 0x7f2834133a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2824009920 tx=0x7f2824009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20cefe37681b76 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2838a36700 1 -- 172.21.15.32:0/331125104 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824011830 con 0x7f28340a1340 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2838a36700 1 -- 172.21.15.32:0/331125104 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2824011990 con 0x7f28340a1340 2023-09-30T22:26:04.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.077+0000 7f2838a36700 1 -- 172.21.15.32:0/331125104 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824011d20 con 0x7f28340a1340 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 -- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 msgr2=0x7f2834133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 --2- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 0x7f2834133a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2824009920 tx=0x7f2824009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 -- 172.21.15.32:0/331125104 shutdown_connections 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 --2- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2834133fa0 0x7f2834134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 --2- 172.21.15.32:0/331125104 >> v2:172.21.15.32:3300/0 conn(0x7f28340a0a30 0x7f28340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 --2- 172.21.15.32:0/331125104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28340a1340 0x7f2834133a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 -- 172.21.15.32:0/331125104 >> 172.21.15.32:0/331125104 conn(0x7f28340a6550 msgr2=0x7f28340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 -- 172.21.15.32:0/331125104 shutdown_connections 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.078+0000 7f283bc9c700 1 -- 172.21.15.32:0/331125104 wait complete. 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 Processor -- start 2023-09-30T22:26:04.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 -- start start 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28340a0a30 0x7f28341206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28340a1340 0x7f2834120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 0x7f2834121130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2834121b60 con 0x7f28340a1340 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28341222c0 con 0x7f2834133fa0 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.080+0000 7f283bc9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28341228e0 con 0x7f28340a0a30 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f283a239700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 0x7f2834121130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28340a0a30 0x7f28341206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f283a239700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 0x7f2834121130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47054/0 (socket says 172.21.15.32:47054) 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f283a239700 1 -- 172.21.15.32:0/2301919294 learned_addr learned my addr 172.21.15.32:0/2301919294 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839237700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28340a1340 0x7f2834120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 -- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 msgr2=0x7f2834121130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 0x7f2834121130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 -- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28340a1340 msgr2=0x7f2834120bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28340a1340 0x7f2834120bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 -- 172.21.15.32:0/2301919294 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300094a0 con 0x7f28340a0a30 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f283a239700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 0x7f2834121130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f2839a38700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28340a0a30 0x7f28341206b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f283000e9e0 tx=0x7f283000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000ee80 con 0x7f28340a0a30 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000c850 con 0x7f28340a0a30 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2824009550 con 0x7f28340a0a30 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.081+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000cc00 con 0x7f28340a0a30 2023-09-30T22:26:04.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.082+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28340a3610 con 0x7f28340a0a30 2023-09-30T22:26:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.083+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f283000cd60 con 0x7f28340a0a30 2023-09-30T22:26:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.083+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28340a0220 con 0x7f28340a0a30 2023-09-30T22:26:04.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.083+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f283003f710 con 0x7f28340a0a30 2023-09-30T22:26:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.088+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f283001be10 con 0x7f28340a0a30 2023-09-30T22:26:04.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 Processor -- start 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 -- start start 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 0x7f2f40130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4009c190 con 0x7f2f4009e080 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4009b090 con 0x7f2f40130750 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.115+0000 7f2f4688b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4009a6c0 con 0x7f2f4009d6f0 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 0x7f2f40130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 0x7f2f40130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54590/0 (socket says 172.21.15.32:54590) 2023-09-30T22:26:04.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50126/0 (socket says 172.21.15.32:50126) 2023-09-30T22:26:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3ffff700 1 -- 172.21.15.32:0/2952564740 learned_addr learned my addr 172.21.15.32:0/2952564740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f44e28700 1 --1- 172.21.15.32:0/2952564740 >> v1:172.21.15.32:6789/0 conn(0x7f2f4009e080 0x7f2f401374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43294/0 (socket says 172.21.15.32:43294) 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/2952564740 >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 msgr2=0x7f2f40130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 --2- 172.21.15.32:0/2952564740 >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 0x7f2f40130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/2952564740 >> v1:172.21.15.32:6789/0 conn(0x7f2f4009e080 legacy=0x7f2f401374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/2952564740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4009e500 con 0x7f2f4009d6f0 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3ffff700 1 --2- 172.21.15.32:0/2952564740 >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 0x7f2f40130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3f7fe700 1 --2- 172.21.15.32:0/2952564740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009db40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009840 tx=0x7f2f34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ce97f0cf78418b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3effd700 1 -- 172.21.15.32:0/2952564740 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011830 con 0x7f2f4009d6f0 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3effd700 1 -- 172.21.15.32:0/2952564740 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34011990 con 0x7f2f4009d6f0 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.116+0000 7f2f3effd700 1 -- 172.21.15.32:0/2952564740 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011d20 con 0x7f2f4009d6f0 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 -- 172.21.15.32:0/2952564740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 msgr2=0x7f2f4009db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 --2- 172.21.15.32:0/2952564740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009db40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009840 tx=0x7f2f34009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 -- 172.21.15.32:0/2952564740 shutdown_connections 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 --2- 172.21.15.32:0/2952564740 >> v2:172.21.15.32:3300/0 conn(0x7f2f40130750 0x7f2f40130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 --2- 172.21.15.32:0/2952564740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f4009d6f0 0x7f2f4009db40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 -- 172.21.15.32:0/2952564740 >> 172.21.15.32:0/2952564740 conn(0x7f2f4009f470 msgr2=0x7f2f400a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 -- 172.21.15.32:0/2952564740 shutdown_connections 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.117+0000 7f2f4688b700 1 -- 172.21.15.32:0/2952564740 wait complete. 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.118+0000 7f2f4688b700 1 Processor -- start 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.118+0000 7f2f4688b700 1 -- start start 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f4688b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 0x7f2f4012df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f4688b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 0x7f2f4012e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f4688b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f40130750 0x7f2f4012e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f4688b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4012f370 con 0x7f2f4009e080 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f4688b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4008c440 con 0x7f2f4009d6f0 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f4688b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400a0c00 con 0x7f2f40130750 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 0x7f2f4012df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 0x7f2f4012e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 0x7f2f4012e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54602/0 (socket says 172.21.15.32:54602) 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/864530841 learned_addr learned my addr 172.21.15.32:0/864530841 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3ffff700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 0x7f2f4012df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47062/0 (socket says 172.21.15.32:47062) 2023-09-30T22:26:04.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f44e28700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f40130750 0x7f2f4012e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f40130750 msgr2=0x7f2f4012e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f40130750 0x7f2f4012e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 msgr2=0x7f2f4012df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 0x7f2f4012df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 -- 172.21.15.32:0/864530841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f30009550 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3ffff700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 0x7f2f4012df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.119+0000 7f2f3f7fe700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 0x7f2f4012e450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009790 tx=0x7f2f34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.120+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f340036a0 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.120+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3401e070 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.120+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34003a40 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.120+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f340094a0 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.120+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4008d170 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.121+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f34003ba0 con 0x7f2f4009e080 2023-09-30T22:26:04.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.121+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f40131020 con 0x7f2f4009e080 2023-09-30T22:26:04.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.121+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2f34020070 con 0x7f2f4009e080 2023-09-30T22:26:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.130+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f3402a020 con 0x7f2f4009e080 2023-09-30T22:26:04.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.301+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2834074ea0 con 0x7f28340a0a30 2023-09-30T22:26:04.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.301+0000 7f282a7fc700 1 -- 172.21.15.32:0/2301919294 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f283001b3b0 con 0x7f28340a0a30 2023-09-30T22:26:04.301 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28340a0a30 msgr2=0x7f28341206b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28340a0a30 0x7f28341206b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f283000e9e0 tx=0x7f283000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 shutdown_connections 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2834133fa0 0x7f2834121130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28340a0a30 0x7f28341206b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 --2- 172.21.15.32:0/2301919294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28340a1340 0x7f2834120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 >> 172.21.15.32:0/2301919294 conn(0x7f28340a6550 msgr2=0x7f283409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 shutdown_connections 2023-09-30T22:26:04.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.304+0000 7f283bc9c700 1 -- 172.21.15.32:0/2301919294 wait complete. 2023-09-30T22:26:04.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.338+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f400747f0 con 0x7f2f4009e080 2023-09-30T22:26:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.338+0000 7f2f3cff9700 1 -- 172.21.15.32:0/864530841 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f34015070 con 0x7f2f4009e080 2023-09-30T22:26:04.338 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 msgr2=0x7f2f4012e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 0x7f2f4012e450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009790 tx=0x7f2f34004750 comp rx=0 tx=0).stop 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 shutdown_connections 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2f40130750 0x7f2f4012e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2f4009d6f0 0x7f2f4012df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 --2- 172.21.15.32:0/864530841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f4009e080 0x7f2f4012e450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 >> 172.21.15.32:0/864530841 conn(0x7f2f4009f470 msgr2=0x7f2f400992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 shutdown_connections 2023-09-30T22:26:04.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.341+0000 7f2f4688b700 1 -- 172.21.15.32:0/864530841 wait complete. 2023-09-30T22:26:04.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:04.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:04.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.633+0000 7f507f433700 1 Processor -- start 2023-09-30T22:26:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.633+0000 7f507f433700 1 -- start start 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507f433700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50780a0a30 0x7f50780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507f433700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50780a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507f433700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780a2a70 con 0x7f5078133930 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507f433700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50780a01d0 con 0x7f50780a0a30 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507f433700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507809e2b0 con 0x7f507809d630 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507c9ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50780a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507d1cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50780a0a30 0x7f50780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507c9ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50780a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50134/0 (socket says 172.21.15.32:50134) 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507c9ce700 1 -- 172.21.15.32:0/2455097120 learned_addr learned my addr 172.21.15.32:0/2455097120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507d9d0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5078133930 0x7f5078133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43302/0 (socket says 172.21.15.32:43302) 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507d1cf700 1 -- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 msgr2=0x7f50780a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507d1cf700 1 --2- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50780a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.634+0000 7f507d1cf700 1 -- 172.21.15.32:0/2455097120 >> v1:172.21.15.32:6789/0 conn(0x7f5078133930 legacy=0x7f5078133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f5077fff700 1 -- 172.21.15.32:0/2455097120 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188283587 0 0) 0x7f50780a2a70 con 0x7f5078133930 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f507d1cf700 1 -- 172.21.15.32:0/2455097120 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507809dc50 con 0x7f50780a0a30 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f507c9ce700 1 --2- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50780a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f507d1cf700 1 --2- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50780a0a30 0x7f50780a0e00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5068009d90 tx=0x7f506800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c07070032a8acf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f5077fff700 1 -- 172.21.15.32:0/2455097120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800e070 con 0x7f50780a0a30 2023-09-30T22:26:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f5077fff700 1 -- 172.21.15.32:0/2455097120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800bb20 con 0x7f50780a0a30 2023-09-30T22:26:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.635+0000 7f5077fff700 1 -- 172.21.15.32:0/2455097120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068012430 con 0x7f50780a0a30 2023-09-30T22:26:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 -- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50780a0a30 msgr2=0x7f50780a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 --2- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50780a0a30 0x7f50780a0e00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5068009d90 tx=0x7f506800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 -- 172.21.15.32:0/2455097120 shutdown_connections 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 --2- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50780a0a30 0x7f50780a0e00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 --2- 172.21.15.32:0/2455097120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50780a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 -- 172.21.15.32:0/2455097120 >> 172.21.15.32:0/2455097120 conn(0x7f50780a3050 msgr2=0x7f50780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 -- 172.21.15.32:0/2455097120 shutdown_connections 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.636+0000 7f507f433700 1 -- 172.21.15.32:0/2455097120 wait complete. 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 Processor -- start 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 -- start start 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50781c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50780a0a30 0x7f50781c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5078133930 0x7f50781c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781206a0 con 0x7f5078133930 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50781204b0 con 0x7f50780a0a30 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507f433700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50781223c0 con 0x7f507809d630 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507d1cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50781c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507c9ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50780a0a30 0x7f50781c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507d1cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50781c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50136/0 (socket says 172.21.15.32:50136) 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.637+0000 7f507d1cf700 1 -- 172.21.15.32:0/4022882275 learned_addr learned my addr 172.21.15.32:0/4022882275 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507c9ce700 1 -- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 msgr2=0x7f50781c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507c9ce700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50781c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507c9ce700 1 -- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5078133930 msgr2=0x7f50781c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507c9ce700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5078133930 0x7f50781c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507c9ce700 1 -- 172.21.15.32:0/4022882275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c009550 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507d1cf700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50781c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507c9ce700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50780a0a30 0x7f50781c84d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9a0 tx=0x7f506c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c004030 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c004190 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5068009a70 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c004520 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.638+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50781220e0 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.639+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f506c00c850 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.639+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f506c03f630 con 0x7f50780a0a30 2023-09-30T22:26:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.640+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50780747f0 con 0x7f50780a0a30 2023-09-30T22:26:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.645+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f506c0036a0 con 0x7f50780a0a30 2023-09-30T22:26:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 Processor -- start 2023-09-30T22:26:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 -- start start 2023-09-30T22:26:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 0x7fa56c13b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c09b150 con 0x7fa56c09d6f0 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c09a500 con 0x7fa56c09e1d0 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa5717d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c09f720 con 0x7fa56c09ace0 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa56b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 0x7fa56c13b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.672+0000 7fa56a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa56c09d6f0 0x7fa56c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43318/0 (socket says 172.21.15.32:43318) 2023-09-30T22:26:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56affd700 1 -- 172.21.15.32:0/1267833511 learned_addr learned my addr 172.21.15.32:0/1267833511 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56a7fc700 1 --2- 172.21.15.32:0/1267833511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47074/0 (socket says 172.21.15.32:47074) 2023-09-30T22:26:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56b7fe700 1 --2- 172.21.15.32:0/1267833511 >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 0x7fa56c13b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54624/0 (socket says 172.21.15.32:54624) 2023-09-30T22:26:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56a7fc700 1 -- 172.21.15.32:0/1267833511 >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 msgr2=0x7fa56c13b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56a7fc700 1 --2- 172.21.15.32:0/1267833511 >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 0x7fa56c13b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56a7fc700 1 -- 172.21.15.32:0/1267833511 >> v1:172.21.15.32:6789/0 conn(0x7fa56c09d6f0 legacy=0x7fa56c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56a7fc700 1 -- 172.21.15.32:0/1267833511 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c06b490 con 0x7fa56c09ace0 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa569ffb700 1 -- 172.21.15.32:0/1267833511 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191151832 0 0) 0x7fa56c09b150 con 0x7fa56c09d6f0 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56b7fe700 1 --2- 172.21.15.32:0/1267833511 >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 0x7fa56c13b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa56a7fc700 1 --2- 172.21.15.32:0/1267833511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c138c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa560009840 tx=0x7fa560009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=588ef39bbffca088 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa569ffb700 1 -- 172.21.15.32:0/1267833511 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560011830 con 0x7fa56c09ace0 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa569ffb700 1 -- 172.21.15.32:0/1267833511 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa560011990 con 0x7fa56c09ace0 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.673+0000 7fa569ffb700 1 -- 172.21.15.32:0/1267833511 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560011d20 con 0x7fa56c09ace0 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 -- 172.21.15.32:0/1267833511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 msgr2=0x7fa56c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 --2- 172.21.15.32:0/1267833511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c138c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa560009840 tx=0x7fa560009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 -- 172.21.15.32:0/1267833511 shutdown_connections 2023-09-30T22:26:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 --2- 172.21.15.32:0/1267833511 >> v2:172.21.15.32:3300/0 conn(0x7fa56c09e1d0 0x7fa56c13b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 --2- 172.21.15.32:0/1267833511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c138c30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 -- 172.21.15.32:0/1267833511 >> 172.21.15.32:0/1267833511 conn(0x7fa56c066f50 msgr2=0x7fa56c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 -- 172.21.15.32:0/1267833511 shutdown_connections 2023-09-30T22:26:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.674+0000 7fa5717d2700 1 -- 172.21.15.32:0/1267833511 wait complete. 2023-09-30T22:26:04.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 Processor -- start 2023-09-30T22:26:04.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 -- start start 2023-09-30T22:26:04.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c11d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa56c09d6f0 0x7fa56c11ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa56c09e1d0 0x7fa56c11e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c11ff10 con 0x7fa56c09d6f0 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c09c760 con 0x7fa56c09ace0 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.676+0000 7fa5717d2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c11ee90 con 0x7fa56c09e1d0 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c11d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c11d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47084/0 (socket says 172.21.15.32:47084) 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 -- 172.21.15.32:0/3797539870 learned_addr learned my addr 172.21.15.32:0/3797539870 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa56c09e1d0 0x7fa56c11e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56a7fc700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa56c09d6f0 0x7fa56c11ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 -- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa56c09e1d0 msgr2=0x7fa56c11e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa56c09e1d0 0x7fa56c11e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 -- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa56c09d6f0 msgr2=0x7fa56c11ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa56c09d6f0 0x7fa56c11ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 -- 172.21.15.32:0/3797539870 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009a70 con 0x7fa56c09ace0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56b7fe700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa56c09e1d0 0x7fa56c11e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56a7fc700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa56c09d6f0 0x7fa56c11ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa56affd700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c11d890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c550 tx=0x7fa55c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c004c80 con 0x7fa56c09ace0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c004e00 con 0x7fa56c09ace0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c007310 con 0x7fa56c09ace0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.677+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5600094a0 con 0x7fa56c09ace0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.678+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c11e950 con 0x7fa56c09ace0 2023-09-30T22:26:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.678+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa55c007470 con 0x7fa56c09ace0 2023-09-30T22:26:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.679+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa55c03f390 con 0x7fa56c09ace0 2023-09-30T22:26:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.679+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c0747f0 con 0x7fa56c09ace0 2023-09-30T22:26:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.683+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa55c0123b0 con 0x7fa56c09ace0 2023-09-30T22:26:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.853+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5078120ca0 con 0x7f50780a0a30 2023-09-30T22:26:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.853+0000 7f5075ffb700 1 -- 172.21.15.32:0/4022882275 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f506c013e60 con 0x7f50780a0a30 2023-09-30T22:26:04.853 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50780a0a30 msgr2=0x7f50781c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50780a0a30 0x7f50781c84d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9a0 tx=0x7f506c00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 shutdown_connections 2023-09-30T22:26:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5078133930 0x7f50781c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507809d630 0x7f50781c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 --2- 172.21.15.32:0/4022882275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50780a0a30 0x7f50781c84d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 >> 172.21.15.32:0/4022882275 conn(0x7f50780a3050 msgr2=0x7f50781bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 shutdown_connections 2023-09-30T22:26:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.857+0000 7f507f433700 1 -- 172.21.15.32:0/4022882275 wait complete. 2023-09-30T22:26:04.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.899+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa56c11f250 con 0x7fa56c09ace0 2023-09-30T22:26:04.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.899+0000 7fa5537fe700 1 -- 172.21.15.32:0/3797539870 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa55c011360 con 0x7fa56c09ace0 2023-09-30T22:26:04.899 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 msgr2=0x7fa56c11d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c11d890 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c550 tx=0x7fa55c00c860 comp rx=0 tx=0).stop 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 shutdown_connections 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa56c09e1d0 0x7fa56c11e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa56c09ace0 0x7fa56c11d890 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 --2- 172.21.15.32:0/3797539870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa56c09d6f0 0x7fa56c11ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 >> 172.21.15.32:0/3797539870 conn(0x7fa56c066f50 msgr2=0x7fa56c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:04.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 shutdown_connections 2023-09-30T22:26:04.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:04.902+0000 7fa5717d2700 1 -- 172.21.15.32:0/3797539870 wait complete. 2023-09-30T22:26:04.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:05.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:05.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:05.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 Processor -- start 2023-09-30T22:26:05.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 -- start start 2023-09-30T22:26:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff0138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 0x7fdff013b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff006bec0 con 0x7fdff009e850 2023-09-30T22:26:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff006bce0 con 0x7fdff0139370 2023-09-30T22:26:05.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdff57fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff006caa0 con 0x7fdff009f330 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdfef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 0x7fdff013b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdfef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 0x7fdff013b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54644/0 (socket says 172.21.15.32:54644) 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdfef7fe700 1 -- 172.21.15.32:0/4244090007 learned_addr learned my addr 172.21.15.32:0/4244090007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.468+0000 7fdfee7fc700 1 --2- 172.21.15.32:0/4244090007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff0138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfee7fc700 1 -- 172.21.15.32:0/4244090007 >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 msgr2=0x7fdff013b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfee7fc700 1 --2- 172.21.15.32:0/4244090007 >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 0x7fdff013b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfee7fc700 1 -- 172.21.15.32:0/4244090007 >> v1:172.21.15.32:6789/0 conn(0x7fdff009e850 legacy=0x7fdff009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfee7fc700 1 -- 172.21.15.32:0/4244090007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00a00a0 con 0x7fdff009f330 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfef7fe700 1 --2- 172.21.15.32:0/4244090007 >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 0x7fdff013b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:05.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfee7fc700 1 --2- 172.21.15.32:0/4244090007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff0138e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009840 tx=0x7fdfd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da31a6447a9196 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfedffb700 1 -- 172.21.15.32:0/4244090007 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8011830 con 0x7fdff009f330 2023-09-30T22:26:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfedffb700 1 -- 172.21.15.32:0/4244090007 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8011990 con 0x7fdff009f330 2023-09-30T22:26:05.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.469+0000 7fdfedffb700 1 -- 172.21.15.32:0/4244090007 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8011d20 con 0x7fdff009f330 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 -- 172.21.15.32:0/4244090007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 msgr2=0x7fdff0138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 --2- 172.21.15.32:0/4244090007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff0138e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009840 tx=0x7fdfd8009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 -- 172.21.15.32:0/4244090007 shutdown_connections 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 --2- 172.21.15.32:0/4244090007 >> v2:172.21.15.32:3300/0 conn(0x7fdff0139370 0x7fdff013b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 --2- 172.21.15.32:0/4244090007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff0138e30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 -- 172.21.15.32:0/4244090007 >> 172.21.15.32:0/4244090007 conn(0x7fdff00a3f70 msgr2=0x7fdff00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 -- 172.21.15.32:0/4244090007 shutdown_connections 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.470+0000 7fdff57fd700 1 -- 172.21.15.32:0/4244090007 wait complete. 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.471+0000 7fdff57fd700 1 Processor -- start 2023-09-30T22:26:05.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.471+0000 7fdff57fd700 1 -- start start 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdff57fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdff009e850 0x7fdff011f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdff57fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff011fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdff57fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 0x7fdff01c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdff57fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff006ccc0 con 0x7fdff0139370 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdff57fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff0120160 con 0x7fdff009f330 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdff57fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdff011f240 con 0x7fdff009e850 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 0x7fdff01c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 0x7fdff01c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54654/0 (socket says 172.21.15.32:54654) 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 -- 172.21.15.32:0/3929628783 learned_addr learned my addr 172.21.15.32:0/3929628783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfeeffd700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdff009e850 0x7fdff011f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfee7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff011fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 -- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdff009e850 msgr2=0x7fdff011f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdff009e850 0x7fdff011f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 -- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 msgr2=0x7fdff011fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff011fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 -- 172.21.15.32:0/3929628783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8009550 con 0x7fdff0139370 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.472+0000 7fdfef7fe700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 0x7fdff01c8560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800eb90 tx=0x7fdfe800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.473+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80097a0 con 0x7fdff0139370 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.473+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe8004020 con 0x7fdff0139370 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.473+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80043b0 con 0x7fdff0139370 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.473+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd80094a0 con 0x7fdff0139370 2023-09-30T22:26:05.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.473+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff01223e0 con 0x7fdff0139370 2023-09-30T22:26:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.474+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0121fa0 con 0x7fdff0139370 2023-09-30T22:26:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.474+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdfe800c850 con 0x7fdff0139370 2023-09-30T22:26:05.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.474+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdfe803f4c0 con 0x7fdff0139370 2023-09-30T22:26:05.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.479+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdfe8019960 con 0x7fdff0139370 2023-09-30T22:26:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.506+0000 7fdc38d16700 1 Processor -- start 2023-09-30T22:26:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.506+0000 7fdc38d16700 1 -- start start 2023-09-30T22:26:05.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc38d16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc340a0a30 0x7fdc340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc38d16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc34133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc38d16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 0x7fdc34134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc38d16700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3409edd0 con 0x7fdc34133fa0 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc38d16700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3409e1f0 con 0x7fdc340a0a30 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc38d16700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3409dc50 con 0x7fdc340a1340 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc32d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 0x7fdc34134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc32d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 0x7fdc34134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54670/0 (socket says 172.21.15.32:54670) 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc32d9d700 1 -- 172.21.15.32:0/2566873875 learned_addr learned my addr 172.21.15.32:0/2566873875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc3259c700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc340a0a30 0x7fdc340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc34133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc340a0a30 msgr2=0x7fdc340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc340a0a30 0x7fdc340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2566873875 >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 msgr2=0x7fdc34134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2566873875 >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 0x7fdc34134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2566873875 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc340a1fe0 con 0x7fdc340a1340 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc32d9d700 1 --2- 172.21.15.32:0/2566873875 >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 0x7fdc34134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.507+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc34133a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009840 tx=0x7fdc28009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0eb9dd055e06480 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc3159a700 1 -- 172.21.15.32:0/2566873875 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011830 con 0x7fdc340a1340 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc3159a700 1 -- 172.21.15.32:0/2566873875 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28011990 con 0x7fdc340a1340 2023-09-30T22:26:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc3159a700 1 -- 172.21.15.32:0/2566873875 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011d20 con 0x7fdc340a1340 2023-09-30T22:26:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 -- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 msgr2=0x7fdc34133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc34133a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009840 tx=0x7fdc28009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 -- 172.21.15.32:0/2566873875 shutdown_connections 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc340a0a30 0x7fdc340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2566873875 >> v2:172.21.15.32:3300/0 conn(0x7fdc34133fa0 0x7fdc34134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2566873875 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc34133a60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 -- 172.21.15.32:0/2566873875 >> 172.21.15.32:0/2566873875 conn(0x7fdc340a6550 msgr2=0x7fdc340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 -- 172.21.15.32:0/2566873875 shutdown_connections 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.508+0000 7fdc38d16700 1 -- 172.21.15.32:0/2566873875 wait complete. 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 Processor -- start 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 -- start start 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 0x7fdc3411e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc3411ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc34133fa0 0x7fdc3411ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34122010 con 0x7fdc340a0a30 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3411f930 con 0x7fdc34133fa0 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc38d16700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34120100 con 0x7fdc340a1340 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc31d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc3411ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc3259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 0x7fdc3411e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc31d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc3411ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50166/0 (socket says 172.21.15.32:50166) 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc3259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 0x7fdc3411e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54684/0 (socket says 172.21.15.32:54684) 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2285247290 learned_addr learned my addr 172.21.15.32:0/2285247290 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:05.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.510+0000 7fdc32d9d700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc34133fa0 0x7fdc3411ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc34133fa0 msgr2=0x7fdc3411ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc34133fa0 0x7fdc3411ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 msgr2=0x7fdc3411e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 0x7fdc3411e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc31d9b700 1 -- 172.21.15.32:0/2285247290 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c009b30 con 0x7fdc340a1340 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc3259c700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 0x7fdc3411e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc32d9d700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc34133fa0 0x7fdc3411ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc31d9b700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc3411ea30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc28000c00 tx=0x7fdc28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28004450 con 0x7fdc340a1340 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2801f070 con 0x7fdc340a1340 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280038e0 con 0x7fdc340a1340 2023-09-30T22:26:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc280094a0 con 0x7fdc340a1340 2023-09-30T22:26:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.511+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc340a0220 con 0x7fdc340a1340 2023-09-30T22:26:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.512+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc28003a40 con 0x7fdc340a1340 2023-09-30T22:26:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.512+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc28021070 con 0x7fdc340a1340 2023-09-30T22:26:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.512+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc34074ea0 con 0x7fdc340a1340 2023-09-30T22:26:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.517+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc28015080 con 0x7fdc340a1340 2023-09-30T22:26:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.691+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdff00747f0 con 0x7fdff0139370 2023-09-30T22:26:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.691+0000 7fdfe77fe700 1 -- 172.21.15.32:0/3929628783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdfe8019310 con 0x7fdff0139370 2023-09-30T22:26:05.691 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 msgr2=0x7fdff01c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 0x7fdff01c8560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800eb90 tx=0x7fdfe800eea0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 shutdown_connections 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdff0139370 0x7fdff01c8560 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdff009e850 0x7fdff011f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 --2- 172.21.15.32:0/3929628783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdff009f330 0x7fdff011fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 >> 172.21.15.32:0/3929628783 conn(0x7fdff00a3f70 msgr2=0x7fdff01383b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 shutdown_connections 2023-09-30T22:26:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.694+0000 7fdff57fd700 1 -- 172.21.15.32:0/3929628783 wait complete. 2023-09-30T22:26:05.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:05.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.724+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdc340a28e0 con 0x7fdc340a1340 2023-09-30T22:26:05.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.725+0000 7fdc22ffd700 1 -- 172.21.15.32:0/2285247290 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc2801a7b0 con 0x7fdc340a1340 2023-09-30T22:26:05.725 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 msgr2=0x7fdc3411ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc3411ea30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc28000c00 tx=0x7fdc28004750 comp rx=0 tx=0).stop 2023-09-30T22:26:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 shutdown_connections 2023-09-30T22:26:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc34133fa0 0x7fdc3411ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc340a0a30 0x7fdc3411e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 --2- 172.21.15.32:0/2285247290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc340a1340 0x7fdc3411ea30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 >> 172.21.15.32:0/2285247290 conn(0x7fdc340a6550 msgr2=0x7fdc34133060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 shutdown_connections 2023-09-30T22:26:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:05.727+0000 7fdc38d16700 1 -- 172.21.15.32:0/2285247290 wait complete. 2023-09-30T22:26:05.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:05.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:05.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 Processor -- start 2023-09-30T22:26:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 -- start start 2023-09-30T22:26:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7dc809f330 0x7f7dc8138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc806bec0 con 0x7f7dc809e850 2023-09-30T22:26:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc80a0030 con 0x7f7dc809f330 2023-09-30T22:26:06.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.024+0000 7f7dcd6eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc806caa0 con 0x7f7dc8139300 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7dc809f330 0x7f7dc8138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7dc809e850 0x7f7dc809ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43338/0 (socket says 172.21.15.32:43338) 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc6ffd700 1 -- 172.21.15.32:0/1573019355 learned_addr learned my addr 172.21.15.32:0/1573019355 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc67fc700 1 -- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 msgr2=0x7f7dc813b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc67fc700 1 --2- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc813b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc67fc700 1 -- 172.21.15.32:0/1573019355 >> v1:172.21.15.32:6789/0 conn(0x7f7dc809e850 legacy=0x7f7dc809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc67fc700 1 -- 172.21.15.32:0/1573019355 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc80a0830 con 0x7f7dc809f330 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc5ffb700 1 -- 172.21.15.32:0/1573019355 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928142989 0 0) 0x7f7dc806bec0 con 0x7f7dc809e850 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc77fe700 1 --2- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc813b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc67fc700 1 --2- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc8138dc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009840 tx=0x7f7dbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5a2bcbe27bc24e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc5ffb700 1 -- 172.21.15.32:0/1573019355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011830 con 0x7f7dc809f330 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc5ffb700 1 -- 172.21.15.32:0/1573019355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc011990 con 0x7f7dc809f330 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.025+0000 7f7dc5ffb700 1 -- 172.21.15.32:0/1573019355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011d20 con 0x7f7dc809f330 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 msgr2=0x7f7dc8138dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc8138dc0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009840 tx=0x7f7dbc009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:06.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/1573019355 shutdown_connections 2023-09-30T22:26:06.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc813b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/1573019355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc8138dc0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/1573019355 >> 172.21.15.32:0/1573019355 conn(0x7f7dc80a3f70 msgr2=0x7f7dc80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:06.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/1573019355 shutdown_connections 2023-09-30T22:26:06.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.026+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/1573019355 wait complete. 2023-09-30T22:26:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.027+0000 7f7dcd6eb700 1 Processor -- start 2023-09-30T22:26:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.027+0000 7f7dcd6eb700 1 -- start start 2023-09-30T22:26:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dcd6eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc809e850 0x7f7dc811f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dcd6eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc811fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dcd6eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc81c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dcd6eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc811f2b0 con 0x7f7dc809f330 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dcd6eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8134650 con 0x7f7dc809e850 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dcd6eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8121410 con 0x7f7dc8139300 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc811fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc809e850 0x7f7dc811f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc811fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54700/0 (socket says 172.21.15.32:54700) 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 -- 172.21.15.32:0/2190790707 learned_addr learned my addr 172.21.15.32:0/2190790707 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc77fe700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc81c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 -- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 msgr2=0x7f7dc81c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc81c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 -- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc809e850 msgr2=0x7f7dc811f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc809e850 0x7f7dc811f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc67fc700 1 -- 172.21.15.32:0/2190790707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc00094a0 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.028+0000 7f7dc6ffd700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc809e850 0x7f7dc811f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.029+0000 7f7dc67fc700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc811fcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009fd0 tx=0x7f7dbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.029+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0045d0 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.029+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc01f070 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.029+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0038a0 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.029+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc0094a0 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.029+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc8122740 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.030+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7dbc003a00 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.030+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc8070540 con 0x7f7dc809f330 2023-09-30T22:26:06.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.030+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7dbc021070 con 0x7f7dc809f330 2023-09-30T22:26:06.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.035+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7dbc010c00 con 0x7f7dc809f330 2023-09-30T22:26:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.063+0000 7f46b4c14700 1 Processor -- start 2023-09-30T22:26:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.063+0000 7f46b4c14700 1 -- start start 2023-09-30T22:26:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46b4c14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46b4c14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 0x7f46b013b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46b4c14700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b0133550 con 0x7f46b009de50 2023-09-30T22:26:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46b4c14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b006fab0 con 0x7f46b0133960 2023-09-30T22:26:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46b4c14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46b0071e20 con 0x7f46b009e990 2023-09-30T22:26:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46aed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 0x7f46b013b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46ae59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46add9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46b009de50 0x7f46b009e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43346/0 (socket says 172.21.15.32:43346) 2023-09-30T22:26:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46aed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 0x7f46b013b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50184/0 (socket says 172.21.15.32:50184) 2023-09-30T22:26:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46ae59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47142/0 (socket says 172.21.15.32:47142) 2023-09-30T22:26:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46add9b700 1 -- 172.21.15.32:0/3699746002 learned_addr learned my addr 172.21.15.32:0/3699746002 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46aed9d700 1 -- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 msgr2=0x7f46b0133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46aed9d700 1 --2- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46aed9d700 1 -- 172.21.15.32:0/3699746002 >> v1:172.21.15.32:6789/0 conn(0x7f46b009de50 legacy=0x7f46b009e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46aed9d700 1 -- 172.21.15.32:0/3699746002 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b006f1e0 con 0x7f46b009e990 2023-09-30T22:26:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.064+0000 7f46ae59c700 1 --2- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46aed9d700 1 --2- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 0x7f46b013b7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f46a80098a0 tx=0x7f46a8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17fe49848f206810 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46ad59a700 1 -- 172.21.15.32:0/3699746002 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8011830 con 0x7f46b009e990 2023-09-30T22:26:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46ad59a700 1 -- 172.21.15.32:0/3699746002 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a8011990 con 0x7f46b009e990 2023-09-30T22:26:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46ad59a700 1 -- 172.21.15.32:0/3699746002 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8011d20 con 0x7f46b009e990 2023-09-30T22:26:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46b4c14700 1 -- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 msgr2=0x7f46b013b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46b4c14700 1 --2- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 0x7f46b013b7f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f46a80098a0 tx=0x7f46a8009bb0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46b4c14700 1 -- 172.21.15.32:0/3699746002 shutdown_connections 2023-09-30T22:26:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46b4c14700 1 --2- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009e990 0x7f46b013b7f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46b4c14700 1 --2- 172.21.15.32:0/3699746002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.065+0000 7f46b4c14700 1 -- 172.21.15.32:0/3699746002 >> 172.21.15.32:0/3699746002 conn(0x7f46b00a2610 msgr2=0x7f46b00a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.066+0000 7f46b4c14700 1 -- 172.21.15.32:0/3699746002 shutdown_connections 2023-09-30T22:26:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.066+0000 7f46b4c14700 1 -- 172.21.15.32:0/3699746002 wait complete. 2023-09-30T22:26:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.067+0000 7f46b4c14700 1 Processor -- start 2023-09-30T22:26:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.067+0000 7f46b4c14700 1 -- start start 2023-09-30T22:26:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46b4c14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009de50 0x7f46b011fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46b4c14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 0x7f46b011fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46b4c14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0120530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46b4c14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b0123350 con 0x7f46b009e990 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46b4c14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b0121640 con 0x7f46b0133960 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46b4c14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46b006f420 con 0x7f46b009de50 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 0x7f46b011fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46ae59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009de50 0x7f46b011fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 0x7f46b011fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54704/0 (socket says 172.21.15.32:54704) 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 -- 172.21.15.32:0/1333049065 learned_addr learned my addr 172.21.15.32:0/1333049065 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:06.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46aed9d700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0120530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 -- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009de50 msgr2=0x7f46b011fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009de50 0x7f46b011fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 -- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 msgr2=0x7f46b0120530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0120530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46add9b700 1 -- 172.21.15.32:0/1333049065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a0009a70 con 0x7f46b009e990 2023-09-30T22:26:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.068+0000 7f46ae59c700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009de50 0x7f46b011fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.069+0000 7f46add9b700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 0x7f46b011fff0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f469800c450 tx=0x7f469800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.069+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698005d70 con 0x7f46b009e990 2023-09-30T22:26:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.069+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800ce60 con 0x7f46b009e990 2023-09-30T22:26:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.069+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a80094a0 con 0x7f46b009e990 2023-09-30T22:26:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.069+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980072d0 con 0x7f46b009e990 2023-09-30T22:26:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.069+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b0123ce0 con 0x7f46b009e990 2023-09-30T22:26:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.070+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4698007430 con 0x7f46b009e990 2023-09-30T22:26:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.070+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f469803e950 con 0x7f46b009e990 2023-09-30T22:26:06.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.070+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b0123190 con 0x7f46b009e990 2023-09-30T22:26:06.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.075+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4698017080 con 0x7f46b009e990 2023-09-30T22:26:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.262+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7dc80747f0 con 0x7f7dc809f330 2023-09-30T22:26:06.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.262+0000 7f7daf7fe700 1 -- 172.21.15.32:0/2190790707 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7dbc02b070 con 0x7f7dc809f330 2023-09-30T22:26:06.263 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 msgr2=0x7f7dc811fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc811fcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009fd0 tx=0x7f7dbc004750 comp rx=0 tx=0).stop 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 shutdown_connections 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7dc8139300 0x7f7dc81c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc809e850 0x7f7dc811f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 --2- 172.21.15.32:0/2190790707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc809f330 0x7f7dc811fcd0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 >> 172.21.15.32:0/2190790707 conn(0x7f7dc80a3f70 msgr2=0x7f7dc80a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 shutdown_connections 2023-09-30T22:26:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.265+0000 7f7dcd6eb700 1 -- 172.21.15.32:0/2190790707 wait complete. 2023-09-30T22:26:06.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.300+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f46b00747f0 con 0x7f46b009e990 2023-09-30T22:26:06.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.301+0000 7f46a6ffd700 1 -- 172.21.15.32:0/1333049065 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f469801b3b0 con 0x7f46b009e990 2023-09-30T22:26:06.300 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 msgr2=0x7f46b011fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 0x7f46b011fff0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f469800c450 tx=0x7f469800c760 comp rx=0 tx=0).stop 2023-09-30T22:26:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 shutdown_connections 2023-09-30T22:26:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46b0133960 0x7f46b0120530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46b009de50 0x7f46b011fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 --2- 172.21.15.32:0/1333049065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46b009e990 0x7f46b011fff0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 >> 172.21.15.32:0/1333049065 conn(0x7f46b00a2610 msgr2=0x7f46b00a0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 shutdown_connections 2023-09-30T22:26:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.303+0000 7f46b4c14700 1 -- 172.21.15.32:0/1333049065 wait complete. 2023-09-30T22:26:06.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:06.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:06.811 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:06.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.959+0000 7f8a77ec1700 1 Processor -- start 2023-09-30T22:26:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.959+0000 7f3fb87f7700 1 Processor -- start 2023-09-30T22:26:06.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.959+0000 7f3fb87f7700 1 -- start start 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.959+0000 7f8a77ec1700 1 -- start start 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a77ec1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 0x7f8a700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a77ec1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a700a1340 0x7f8a70133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a77ec1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 0x7f8a70134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a77ec1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7009edd0 con 0x7f8a700a0a30 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a77ec1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7009e1f0 con 0x7f8a700a1340 2023-09-30T22:26:06.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a77ec1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7009dc50 con 0x7f8a70133fa0 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb87f7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb87f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0134260 0x7f3fb01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb87f7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb006b990 con 0x7f3fb0133e30 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb87f7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00a0590 con 0x7f3fb0134260 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb87f7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb009fb80 con 0x7f3fb0133420 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 0x7f8a70134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a75c5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 0x7f8a700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 0x7f8a70134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50200/0 (socket says 172.21.15.32:50200) 2023-09-30T22:26:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 -- 172.21.15.32:0/3219778594 learned_addr learned my addr 172.21.15.32:0/3219778594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a75c5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 0x7f8a700a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54716/0 (socket says 172.21.15.32:54716) 2023-09-30T22:26:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb5d92700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3fb0133e30 0x7f3fb012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43350/0 (socket says 172.21.15.32:43350) 2023-09-30T22:26:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb5d92700 1 -- 172.21.15.32:0/745145819 learned_addr learned my addr 172.21.15.32:0/745145819 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:06.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb6d94700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0134260 0x7f3fb01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 -- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a700a1340 msgr2=0x7f8a70133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:06.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 --2- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a700a1340 0x7f8a70133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 -- 172.21.15.32:0/3219778594 >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 msgr2=0x7f8a700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:06.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 --2- 172.21.15.32:0/3219778594 >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 0x7f8a700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb6593700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a7645e700 1 -- 172.21.15.32:0/3219778594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700a1fe0 con 0x7f8a70133fa0 2023-09-30T22:26:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f8a75c5d700 1 --2- 172.21.15.32:0/3219778594 >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 0x7f8a700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb5591700 1 -- 172.21.15.32:0/745145819 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398289568 0 0) 0x7f3fb006b990 con 0x7f3fb0133e30 2023-09-30T22:26:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb6d94700 1 -- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 msgr2=0x7f3fb01337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb6d94700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb01337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.960+0000 7f3fb6d94700 1 -- 172.21.15.32:0/745145819 >> v1:172.21.15.32:6789/0 conn(0x7f3fb0133e30 legacy=0x7f3fb012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f3fb6d94700 1 -- 172.21.15.32:0/745145819 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00a0030 con 0x7f3fb0134260 2023-09-30T22:26:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f8a7645e700 1 --2- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 0x7f8a70134410 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009e80 tx=0x7f8a6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4273cf18f874f2fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f3fb6593700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f3fb6d94700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0134260 0x7f3fb01306a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009dc0 tx=0x7f3fa800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec95e129158ed5aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f8a74c5b700 1 -- 172.21.15.32:0/3219778594 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800e070 con 0x7f8a70133fa0 2023-09-30T22:26:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f8a74c5b700 1 -- 172.21.15.32:0/3219778594 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6800b8d0 con 0x7f8a70133fa0 2023-09-30T22:26:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f8a74c5b700 1 -- 172.21.15.32:0/3219778594 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800bc80 con 0x7f8a70133fa0 2023-09-30T22:26:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f3fb5591700 1 -- 172.21.15.32:0/745145819 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800e070 con 0x7f3fb0134260 2023-09-30T22:26:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f3fb5591700 1 -- 172.21.15.32:0/745145819 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa800bbd0 con 0x7f3fb0134260 2023-09-30T22:26:06.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.961+0000 7f3fb5591700 1 -- 172.21.15.32:0/745145819 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8012430 con 0x7f3fb0134260 2023-09-30T22:26:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 -- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 msgr2=0x7f8a70134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 0x7f8a70134410 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009e80 tx=0x7f8a6800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 -- 172.21.15.32:0/3219778594 shutdown_connections 2023-09-30T22:26:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a70133fa0 0x7f8a70134410 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/3219778594 >> v2:172.21.15.32:3300/0 conn(0x7f8a700a0a30 0x7f8a700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/3219778594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a700a1340 0x7f8a70133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 -- 172.21.15.32:0/3219778594 >> 172.21.15.32:0/3219778594 conn(0x7f8a700a6550 msgr2=0x7f8a700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 -- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0134260 msgr2=0x7f3fb01306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0134260 0x7f3fb01306a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009dc0 tx=0x7f3fa800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 -- 172.21.15.32:0/3219778594 shutdown_connections 2023-09-30T22:26:06.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f8a77ec1700 1 -- 172.21.15.32:0/3219778594 wait complete. 2023-09-30T22:26:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 -- 172.21.15.32:0/745145819 shutdown_connections 2023-09-30T22:26:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0134260 0x7f3fb01306a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/745145819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 -- 172.21.15.32:0/745145819 >> 172.21.15.32:0/745145819 conn(0x7f3fb00a4210 msgr2=0x7f3fb00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:06.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 -- 172.21.15.32:0/745145819 shutdown_connections 2023-09-30T22:26:06.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.962+0000 7f3fb87f7700 1 -- 172.21.15.32:0/745145819 wait complete. 2023-09-30T22:26:06.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 Processor -- start 2023-09-30T22:26:06.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 -- start start 2023-09-30T22:26:06.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f8a77ec1700 1 Processor -- start 2023-09-30T22:26:06.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f8a77ec1700 1 -- start start 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0133e30 0x7f3fb009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 0x7f3fb009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb009d850 con 0x7f3fb0134260 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb011ee10 con 0x7f3fb0133e30 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.963+0000 7f3fb87f7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01206c0 con 0x7f3fb0133420 2023-09-30T22:26:06.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 0x7f3fb009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6593700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb5d92700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0133e30 0x7f3fb009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 0x7f3fb009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54718/0 (socket says 172.21.15.32:54718) 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 -- 172.21.15.32:0/148825779 learned_addr learned my addr 172.21.15.32:0/148825779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 -- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 msgr2=0x7f3fb009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 -- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0133e30 msgr2=0x7f3fb009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0133e30 0x7f3fb009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 -- 172.21.15.32:0/148825779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8009a70 con 0x7f3fb0134260 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a77ec1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 0x7f8a7011d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a77ec1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a700a1340 0x7f8a7011d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a77ec1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a70133fa0 0x7f8a7011db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a77ec1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7011ecd0 con 0x7f8a700a0a30 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a77ec1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7011ff20 con 0x7f8a70133fa0 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a77ec1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7011f2c0 con 0x7f8a700a1340 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a75c5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 0x7f8a7011d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a75c5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 0x7f8a7011d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54728/0 (socket says 172.21.15.32:54728) 2023-09-30T22:26:06.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a7545c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a700a1340 0x7f8a7011d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a75c5d700 1 -- 172.21.15.32:0/4026204898 learned_addr learned my addr 172.21.15.32:0/4026204898 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:06.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f3fb6d94700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 0x7f3fb009cf00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8014040 tx=0x7f3fa8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.964+0000 7f8a7645e700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a70133fa0 0x7f8a7011db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:06.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8012430 con 0x7f3fb0134260 2023-09-30T22:26:06.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa8020070 con 0x7f3fb0134260 2023-09-30T22:26:06.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a75c5d700 1 -- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a700a1340 msgr2=0x7f8a7011d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a75c5d700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a700a1340 0x7f8a7011d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8004280 con 0x7f3fb0134260 2023-09-30T22:26:06.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a75c5d700 1 -- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a70133fa0 msgr2=0x7f8a7011db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:06.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a75c5d700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a70133fa0 0x7f8a7011db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:06.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a75c5d700 1 -- 172.21.15.32:0/4026204898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0094a0 con 0x7f8a700a0a30 2023-09-30T22:26:06.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb011fb50 con 0x7f3fb0134260 2023-09-30T22:26:06.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a7645e700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a70133fa0 0x7f8a7011db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:06.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb012db20 con 0x7f3fb0134260 2023-09-30T22:26:06.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.965+0000 7f8a7545c700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a700a1340 0x7f8a7011d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:06.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fa80043e0 con 0x7f3fb0134260 2023-09-30T22:26:06.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f8a75c5d700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 0x7f8a7011d0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00ebe0 tx=0x7f8a6c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:06.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3fa8022070 con 0x7f3fb0134260 2023-09-30T22:26:06.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c009800 con 0x7f8a700a0a30 2023-09-30T22:26:06.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00c910 con 0x7f8a700a0a30 2023-09-30T22:26:06.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a68009b30 con 0x7f8a700a0a30 2023-09-30T22:26:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00cc80 con 0x7f8a700a0a30 2023-09-30T22:26:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb0074ea0 con 0x7f3fb0134260 2023-09-30T22:26:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.966+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a700a0220 con 0x7f8a700a0a30 2023-09-30T22:26:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.967+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70074ea0 con 0x7f8a700a0a30 2023-09-30T22:26:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.967+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a6c013470 con 0x7f8a700a0a30 2023-09-30T22:26:06.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.971+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a6c01b3b0 con 0x7f8a700a0a30 2023-09-30T22:26:06.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.972+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fa801b400 con 0x7f3fb0134260 2023-09-30T22:26:06.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:06.976+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a6c013720 con 0x7f8a700a0a30 2023-09-30T22:26:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.185+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3fb006b250 con 0x7f3fb0134260 2023-09-30T22:26:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.185+0000 7f3fa6ffd700 1 -- 172.21.15.32:0/148825779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fa802c030 con 0x7f3fb0134260 2023-09-30T22:26:07.185 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.186+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a700a28e0 con 0x7f8a700a0a30 2023-09-30T22:26:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.186+0000 7f8a667fc700 1 -- 172.21.15.32:0/4026204898 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a6c021020 con 0x7f8a700a0a30 2023-09-30T22:26:07.186 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 msgr2=0x7f3fb009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 0x7f3fb009cf00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8014040 tx=0x7f3fa8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 shutdown_connections 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fb0134260 0x7f3fb009cf00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3fb0133420 0x7f3fb009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 --2- 172.21.15.32:0/148825779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3fb0133e30 0x7f3fb009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 >> 172.21.15.32:0/148825779 conn(0x7f3fb00a4210 msgr2=0x7f3fb00a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:07.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 shutdown_connections 2023-09-30T22:26:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f3fb87f7700 1 -- 172.21.15.32:0/148825779 wait complete. 2023-09-30T22:26:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 msgr2=0x7f8a7011d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 0x7f8a7011d0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00ebe0 tx=0x7f8a6c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 shutdown_connections 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a70133fa0 0x7f8a7011db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a700a0a30 0x7f8a7011d0c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 --2- 172.21.15.32:0/4026204898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a700a1340 0x7f8a7011d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 >> 172.21.15.32:0/4026204898 conn(0x7f8a700a6550 msgr2=0x7f8a70133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.188+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 shutdown_connections 2023-09-30T22:26:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.189+0000 7f8a77ec1700 1 -- 172.21.15.32:0/4026204898 wait complete. 2023-09-30T22:26:07.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:07.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:07.279 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:07.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.423+0000 7f0a9f8d3700 1 Processor -- start 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.423+0000 7f0a9f8d3700 1 -- start start 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9f8d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a9809d630 0x7f0a980a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9f8d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 0x7f0a9813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9f8d3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a980a2a70 con 0x7f0a980a0a30 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9f8d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980a01d0 con 0x7f0a9809d630 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9f8d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9809e2b0 con 0x7f0a98138d80 2023-09-30T22:26:07.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9de70700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 0x7f0a9813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9d66f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a980a0a30 0x7f0a980a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43360/0 (socket says 172.21.15.32:43360) 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9de70700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 0x7f0a9813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50230/0 (socket says 172.21.15.32:50230) 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9d66f700 1 -- 172.21.15.32:0/2202105363 learned_addr learned my addr 172.21.15.32:0/2202105363 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9ce6e700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a9809d630 0x7f0a980a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a97fff700 1 -- 172.21.15.32:0/2202105363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74071753 0 0) 0x7f0a980a2a70 con 0x7f0a980a0a30 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a97fff700 1 -- 172.21.15.32:0/2202105363 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a800035b0 con 0x7f0a980a0a30 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9de70700 1 -- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a9809d630 msgr2=0x7f0a980a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9de70700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a9809d630 0x7f0a980a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9de70700 1 -- 172.21.15.32:0/2202105363 >> v1:172.21.15.32:6789/0 conn(0x7f0a980a0a30 legacy=0x7f0a980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9de70700 1 -- 172.21.15.32:0/2202105363 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9809dc50 con 0x7f0a98138d80 2023-09-30T22:26:07.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.424+0000 7f0a9ce6e700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a9809d630 0x7f0a980a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.425+0000 7f0a9de70700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 0x7f0a9813b160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009870 tx=0x7f0a90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b443c364c9ad76 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.425+0000 7f0a97fff700 1 -- 172.21.15.32:0/2202105363 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011830 con 0x7f0a98138d80 2023-09-30T22:26:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.425+0000 7f0a97fff700 1 -- 172.21.15.32:0/2202105363 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a90011990 con 0x7f0a98138d80 2023-09-30T22:26:07.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.425+0000 7f0a97fff700 1 -- 172.21.15.32:0/2202105363 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011d20 con 0x7f0a98138d80 2023-09-30T22:26:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 msgr2=0x7f0a9813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 0x7f0a9813b160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009870 tx=0x7f0a90009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2202105363 shutdown_connections 2023-09-30T22:26:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a98138d80 0x7f0a9813b160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2202105363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a9809d630 0x7f0a980a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2202105363 >> 172.21.15.32:0/2202105363 conn(0x7f0a980a3050 msgr2=0x7f0a980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2202105363 shutdown_connections 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.426+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2202105363 wait complete. 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.427+0000 7f0a9f8d3700 1 Processor -- start 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.427+0000 7f0a9f8d3700 1 -- start start 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.427+0000 7f0a9f8d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a9809d630 0x7f0a981337f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9f8d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a980a0a30 0x7f0a98133d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9f8d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a98138d80 0x7f0a98134270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9f8d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98134cc0 con 0x7f0a980a0a30 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9f8d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a981361e0 con 0x7f0a98138d80 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9f8d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98136590 con 0x7f0a9809d630 2023-09-30T22:26:07.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9d66f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a9809d630 0x7f0a981337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9de70700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a98138d80 0x7f0a98134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a980a0a30 0x7f0a98133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9d66f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a9809d630 0x7f0a981337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50244/0 (socket says 172.21.15.32:50244) 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9d66f700 1 -- 172.21.15.32:0/2110859978 learned_addr learned my addr 172.21.15.32:0/2110859978 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 -- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a9809d630 msgr2=0x7f0a981337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a9809d630 0x7f0a981337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 -- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a98138d80 msgr2=0x7f0a98134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a98138d80 0x7f0a98134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 -- 172.21.15.32:0/2110859978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c0094a0 con 0x7f0a980a0a30 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9de70700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a98138d80 0x7f0a98134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a9ce6e700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a980a0a30 0x7f0a98133d30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9b0 tx=0x7f0a8c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:07.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.428+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00ee50 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.429+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c00c7f0 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.429+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a900094a0 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.429+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00cba0 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.429+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980a0410 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.430+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a8c00cd00 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.430+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a8c03fa90 con 0x7f0a980a0a30 2023-09-30T22:26:07.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.430+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a980747f0 con 0x7f0a980a0a30 2023-09-30T22:26:07.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.435+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a8c021020 con 0x7f0a980a0a30 2023-09-30T22:26:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 Processor -- start 2023-09-30T22:26:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 -- start start 2023-09-30T22:26:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 0x7f8798135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8798135e50 0x7f8798138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879809f130 con 0x7f879809d6f0 2023-09-30T22:26:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879809f520 con 0x7f8798135e50 2023-09-30T22:26:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879ebaf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f879809e930 con 0x7f879809e1d0 2023-09-30T22:26:07.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f8797fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 0x7f8798135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879d14c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8798135e50 0x7f8798138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f8797fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 0x7f8798135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50250/0 (socket says 172.21.15.32:50250) 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f879c94b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f879809d6f0 0x7f879809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43364/0 (socket says 172.21.15.32:43364) 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.464+0000 7f8797fff700 1 -- 172.21.15.32:0/2984496237 learned_addr learned my addr 172.21.15.32:0/2984496237 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f8797fff700 1 -- 172.21.15.32:0/2984496237 >> v2:172.21.15.32:3300/0 conn(0x7f8798135e50 msgr2=0x7f8798138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f8797fff700 1 --2- 172.21.15.32:0/2984496237 >> v2:172.21.15.32:3300/0 conn(0x7f8798135e50 0x7f8798138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f8797fff700 1 -- 172.21.15.32:0/2984496237 >> v1:172.21.15.32:6789/0 conn(0x7f879809d6f0 legacy=0x7f879809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f8797fff700 1 -- 172.21.15.32:0/2984496237 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879809b2d0 con 0x7f879809e1d0 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f87977fe700 1 -- 172.21.15.32:0/2984496237 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365516620 0 0) 0x7f879809f130 con 0x7f879809d6f0 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f879d14c700 1 --2- 172.21.15.32:0/2984496237 >> v2:172.21.15.32:3300/0 conn(0x7f8798135e50 0x7f8798138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f8797fff700 1 --2- 172.21.15.32:0/2984496237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 0x7f8798135910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f878c009920 tx=0x7f878c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52b7e8380529320 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f87977fe700 1 -- 172.21.15.32:0/2984496237 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c011830 con 0x7f879809e1d0 2023-09-30T22:26:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f87977fe700 1 -- 172.21.15.32:0/2984496237 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c011990 con 0x7f879809e1d0 2023-09-30T22:26:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.465+0000 7f87977fe700 1 -- 172.21.15.32:0/2984496237 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c011d20 con 0x7f879809e1d0 2023-09-30T22:26:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 -- 172.21.15.32:0/2984496237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 msgr2=0x7f8798135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 --2- 172.21.15.32:0/2984496237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 0x7f8798135910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f878c009920 tx=0x7f878c009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 -- 172.21.15.32:0/2984496237 shutdown_connections 2023-09-30T22:26:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 --2- 172.21.15.32:0/2984496237 >> v2:172.21.15.32:3300/0 conn(0x7f8798135e50 0x7f8798138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 --2- 172.21.15.32:0/2984496237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f879809e1d0 0x7f8798135910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 -- 172.21.15.32:0/2984496237 >> 172.21.15.32:0/2984496237 conn(0x7f8798066f50 msgr2=0x7f879809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 -- 172.21.15.32:0/2984496237 shutdown_connections 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.466+0000 7f879ebaf700 1 -- 172.21.15.32:0/2984496237 wait complete. 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.467+0000 7f879ebaf700 1 Processor -- start 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.467+0000 7f879ebaf700 1 -- start start 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.468+0000 7f879ebaf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f879809d6f0 0x7f87981c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.468+0000 7f879ebaf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f879809e1d0 0x7f87981c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.468+0000 7f879ebaf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 0x7f8798119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.468+0000 7f879ebaf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87981301f0 con 0x7f879809e1d0 2023-09-30T22:26:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.468+0000 7f879ebaf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879811b180 con 0x7f879809d6f0 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.468+0000 7f879ebaf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f879809a0c0 con 0x7f8798135e50 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f879d14c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 0x7f8798119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f879809e1d0 0x7f87981c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f879d14c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 0x7f8798119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50252/0 (socket says 172.21.15.32:50252) 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f879d14c700 1 -- 172.21.15.32:0/937571396 learned_addr learned my addr 172.21.15.32:0/937571396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f879c94b700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f879809d6f0 0x7f87981c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 -- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 msgr2=0x7f8798119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 0x7f8798119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 -- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f879809d6f0 msgr2=0x7f87981c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f879809d6f0 0x7f87981c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 -- 172.21.15.32:0/937571396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790009a70 con 0x7f879809e1d0 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f879d14c700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 0x7f8798119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f8797fff700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f879809e1d0 0x7f87981c4830 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f878c016000 tx=0x7f878c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0045d0 con 0x7f879809e1d0 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c01f070 con 0x7f879809e1d0 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c009550 con 0x7f879809e1d0 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.469+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0038e0 con 0x7f879809e1d0 2023-09-30T22:26:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.470+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879809c700 con 0x7f879809e1d0 2023-09-30T22:26:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.470+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f878c003a40 con 0x7f879809e1d0 2023-09-30T22:26:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.471+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f878c021070 con 0x7f879809e1d0 2023-09-30T22:26:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.471+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8798074ea0 con 0x7f879809e1d0 2023-09-30T22:26:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.476+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f878c015080 con 0x7f879809e1d0 2023-09-30T22:26:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.646+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0a98072980 con 0x7f0a980a0a30 2023-09-30T22:26:07.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.647+0000 7f0a95ffb700 1 -- 172.21.15.32:0/2110859978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a8c01b3b0 con 0x7f0a980a0a30 2023-09-30T22:26:07.647 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a980a0a30 msgr2=0x7f0a98133d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a980a0a30 0x7f0a98133d30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9b0 tx=0x7f0a8c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 shutdown_connections 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a98138d80 0x7f0a98134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a9809d630 0x7f0a981337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 --2- 172.21.15.32:0/2110859978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a980a0a30 0x7f0a98133d30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 >> 172.21.15.32:0/2110859978 conn(0x7f0a980a3050 msgr2=0x7f0a98139700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 shutdown_connections 2023-09-30T22:26:07.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.649+0000 7f0a9f8d3700 1 -- 172.21.15.32:0/2110859978 wait complete. 2023-09-30T22:26:07.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:07.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.680+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f87980a20d0 con 0x7f879809e1d0 2023-09-30T22:26:07.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.681+0000 7f87957fa700 1 -- 172.21.15.32:0/937571396 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f878c02b020 con 0x7f879809e1d0 2023-09-30T22:26:07.680 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:07.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f879809e1d0 msgr2=0x7f87981c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f879809e1d0 0x7f87981c4830 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f878c016000 tx=0x7f878c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 shutdown_connections 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8798135e50 0x7f8798119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f879809d6f0 0x7f87981c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 --2- 172.21.15.32:0/937571396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f879809e1d0 0x7f87981c4830 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 >> 172.21.15.32:0/937571396 conn(0x7f8798066f50 msgr2=0x7f87981329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 shutdown_connections 2023-09-30T22:26:07.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:07.683+0000 7f879ebaf700 1 -- 172.21.15.32:0/937571396 wait complete. 2023-09-30T22:26:07.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:08.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:08.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:08.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.352+0000 7fbe846ae700 1 Processor -- start 2023-09-30T22:26:08.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.352+0000 7fbe846ae700 1 -- start start 2023-09-30T22:26:08.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.352+0000 7f5cd9659700 1 Processor -- start 2023-09-30T22:26:08.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.352+0000 7f5cd9659700 1 -- start start 2023-09-30T22:26:08.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe846ae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe7c0a0a30 0x7fbe7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe846ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c09d630 0x7fbe7c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe846ae700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7c0a2a70 con 0x7fbe7c133930 2023-09-30T22:26:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe846ae700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7c0a01d0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe846ae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c09e2b0 con 0x7fbe7c09d630 2023-09-30T22:26:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd9659700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd40a1510 0x7f5cd4138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd9659700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 0x7f5cd413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd9659700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd409edd0 con 0x7f5cd40a0a30 2023-09-30T22:26:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd9659700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40a18e0 con 0x7f5cd40a1510 2023-09-30T22:26:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd9659700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd409e1f0 con 0x7f5cd4139250 2023-09-30T22:26:08.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe81c49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c09d630 0x7fbe7c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 0x7f5cd413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 0x7f5cd413b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47218/0 (socket says 172.21.15.32:47218) 2023-09-30T22:26:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cd40a1510 0x7f5cd4138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7f5cd37fe700 1 -- 172.21.15.32:0/3646533050 learned_addr learned my addr 172.21.15.32:0/3646533050 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe8244a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe7c0a0a30 0x7fbe7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe81c49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c09d630 0x7fbe7c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47216/0 (socket says 172.21.15.32:47216) 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe81c49700 1 -- 172.21.15.32:0/3508631198 learned_addr learned my addr 172.21.15.32:0/3508631198 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe8244a700 1 -- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c09d630 msgr2=0x7fbe7c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.353+0000 7fbe8244a700 1 --2- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c09d630 0x7fbe7c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7fbe8244a700 1 -- 172.21.15.32:0/3508631198 >> v1:172.21.15.32:6789/0 conn(0x7fbe7c133930 legacy=0x7fbe7c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7fbe8244a700 1 -- 172.21.15.32:0/3508631198 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c09dc50 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd37fe700 1 -- 172.21.15.32:0/3646533050 >> v2:172.21.15.32:3300/0 conn(0x7f5cd40a1510 msgr2=0x7f5cd4138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:08.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd37fe700 1 --2- 172.21.15.32:0/3646533050 >> v2:172.21.15.32:3300/0 conn(0x7f5cd40a1510 0x7f5cd4138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd37fe700 1 -- 172.21.15.32:0/3646533050 >> v1:172.21.15.32:6789/0 conn(0x7f5cd40a0a30 legacy=0x7f5cd40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd37fe700 1 -- 172.21.15.32:0/3646533050 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd409dc50 con 0x7f5cd4139250 2023-09-30T22:26:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7fbe8244a700 1 --2- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c0a0e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009840 tx=0x7fbe78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0e2230ed974f58 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd37fe700 1 --2- 172.21.15.32:0/3646533050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 0x7f5cd413b630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009e50 tx=0x7f5ccc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f133fc5d5706cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7fbe81448700 1 -- 172.21.15.32:0/3508631198 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78011830 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd1ffb700 1 -- 172.21.15.32:0/3646533050 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00e070 con 0x7f5cd4139250 2023-09-30T22:26:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7fbe81448700 1 -- 172.21.15.32:0/3508631198 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78011990 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7fbe81448700 1 -- 172.21.15.32:0/3508631198 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78011d20 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd1ffb700 1 -- 172.21.15.32:0/3646533050 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc003a20 con 0x7f5cd4139250 2023-09-30T22:26:08.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.354+0000 7f5cd1ffb700 1 -- 172.21.15.32:0/3646533050 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc003dd0 con 0x7f5cd4139250 2023-09-30T22:26:08.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 -- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 msgr2=0x7fbe7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 --2- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c0a0e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009840 tx=0x7fbe78009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 -- 172.21.15.32:0/3646533050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 msgr2=0x7f5cd413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 --2- 172.21.15.32:0/3646533050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 0x7f5cd413b630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009e50 tx=0x7f5ccc00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 -- 172.21.15.32:0/3508631198 shutdown_connections 2023-09-30T22:26:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 --2- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c0a0e00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 --2- 172.21.15.32:0/3508631198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c09d630 0x7fbe7c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 -- 172.21.15.32:0/3508631198 >> 172.21.15.32:0/3508631198 conn(0x7fbe7c0a3050 msgr2=0x7fbe7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 -- 172.21.15.32:0/3646533050 shutdown_connections 2023-09-30T22:26:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 --2- 172.21.15.32:0/3646533050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd4139250 0x7f5cd413b630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 --2- 172.21.15.32:0/3646533050 >> v2:172.21.15.32:3300/0 conn(0x7f5cd40a1510 0x7f5cd4138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 -- 172.21.15.32:0/3646533050 >> 172.21.15.32:0/3646533050 conn(0x7f5cd40a6550 msgr2=0x7f5cd40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:08.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 -- 172.21.15.32:0/3508631198 shutdown_connections 2023-09-30T22:26:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 -- 172.21.15.32:0/3646533050 shutdown_connections 2023-09-30T22:26:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7f5cd9659700 1 -- 172.21.15.32:0/3646533050 wait complete. 2023-09-30T22:26:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.355+0000 7fbe846ae700 1 -- 172.21.15.32:0/3508631198 wait complete. 2023-09-30T22:26:08.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.356+0000 7fbe846ae700 1 Processor -- start 2023-09-30T22:26:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.356+0000 7fbe846ae700 1 -- start start 2023-09-30T22:26:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.356+0000 7f5cd9659700 1 Processor -- start 2023-09-30T22:26:08.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.356+0000 7f5cd9659700 1 -- start start 2023-09-30T22:26:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe846ae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe7c09d630 0x7fbe7c1c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe846ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c1c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe846ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c133930 0x7fbe7c1c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe846ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c11eea0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe846ae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1207f0 con 0x7fbe7c133930 2023-09-30T22:26:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe846ae700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c120600 con 0x7fbe7c09d630 2023-09-30T22:26:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd9659700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd40a0a30 0x7f5cd4122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd9659700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 0x7f5cd4122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd9659700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd4139250 0x7f5cd4123500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd9659700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4126280 con 0x7f5cd40a0a30 2023-09-30T22:26:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd9659700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4126480 con 0x7f5cd40a1510 2023-09-30T22:26:08.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd9659700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4138a90 con 0x7f5cd4139250 2023-09-30T22:26:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c1c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 0x7f5cd4122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 0x7f5cd4122fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47248/0 (socket says 172.21.15.32:47248) 2023-09-30T22:26:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 -- 172.21.15.32:0/2835176469 learned_addr learned my addr 172.21.15.32:0/2835176469 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c1c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54780/0 (socket says 172.21.15.32:54780) 2023-09-30T22:26:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 -- 172.21.15.32:0/2828503927 learned_addr learned my addr 172.21.15.32:0/2828503927 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe82c4b700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c133930 0x7fbe7c1c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd2ffd700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd40a0a30 0x7f5cd4122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 -- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd4139250 msgr2=0x7f5cd4123500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd4139250 0x7f5cd4123500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 -- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd40a0a30 msgr2=0x7f5cd4122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd40a0a30 0x7f5cd4122a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 -- 172.21.15.32:0/2835176469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc009b30 con 0x7f5cd40a1510 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 -- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe7c09d630 msgr2=0x7fbe7c1c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe7c09d630 0x7fbe7c1c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 -- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c133930 msgr2=0x7fbe7c1c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c133930 0x7fbe7c1c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 -- 172.21.15.32:0/2828503927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780094a0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7f5cd27fc700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 0x7f5cd4122fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800e9e0 tx=0x7f5cc800eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.357+0000 7fbe81c49700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c1c8660 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00e910 tx=0x7fbe6c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800c850 con 0x7f5cd40a1510 2023-09-30T22:26:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0096a0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc800c9b0 con 0x7f5cd40a1510 2023-09-30T22:26:08.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800cd20 con 0x7f5cd40a1510 2023-09-30T22:26:08.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd4124610 con 0x7f5cd40a1510 2023-09-30T22:26:08.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40a0270 con 0x7f5cd40a1510 2023-09-30T22:26:08.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c00ee60 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c122510 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c0a0460 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.358+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c00ca90 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.359+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5cc8013470 con 0x7f5cd40a1510 2023-09-30T22:26:08.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.359+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd4074ea0 con 0x7f5cd40a1510 2023-09-30T22:26:08.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.359+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5cc803fb60 con 0x7f5cd40a1510 2023-09-30T22:26:08.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.359+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe640052e0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.360+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe6c00cc50 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.360+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbe6c03f6d0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.364+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5cc8013aa0 con 0x7f5cd40a1510 2023-09-30T22:26:08.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.365+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe6c01ba00 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.568+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbe64005590 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.568+0000 7fbe72ffd700 1 -- 172.21.15.32:0/2828503927 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe6c01b3b0 con 0x7fbe7c0a0a30 2023-09-30T22:26:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:08.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 msgr2=0x7fbe7c1c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c1c8660 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00e910 tx=0x7fbe6c00ec20 comp rx=0 tx=0).stop 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 shutdown_connections 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe7c133930 0x7fbe7c1c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe7c09d630 0x7fbe7c1c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 --2- 172.21.15.32:0/2828503927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe7c0a0a30 0x7fbe7c1c8660 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.570+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 >> 172.21.15.32:0/2828503927 conn(0x7fbe7c0a3050 msgr2=0x7fbe7c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.571+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 shutdown_connections 2023-09-30T22:26:08.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.571+0000 7fbe846ae700 1 -- 172.21.15.32:0/2828503927 wait complete. 2023-09-30T22:26:08.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.573+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5cd4125e40 con 0x7f5cd40a1510 2023-09-30T22:26:08.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.573+0000 7f5cbb7fe700 1 -- 172.21.15.32:0/2835176469 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5cc8017030 con 0x7f5cd40a1510 2023-09-30T22:26:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 msgr2=0x7f5cd4122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 0x7f5cd4122fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800e9e0 tx=0x7f5cc800eda0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 shutdown_connections 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cd4139250 0x7f5cd4123500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cd40a0a30 0x7f5cd4122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 --2- 172.21.15.32:0/2835176469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cd40a1510 0x7f5cd4122fc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 >> 172.21.15.32:0/2835176469 conn(0x7f5cd40a6550 msgr2=0x7f5cd40a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:08.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 shutdown_connections 2023-09-30T22:26:08.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.576+0000 7f5cd9659700 1 -- 172.21.15.32:0/2835176469 wait complete. 2023-09-30T22:26:08.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:08.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:08.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:08.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:08.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.808+0000 7f7fa35a9700 1 Processor -- start 2023-09-30T22:26:08.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.808+0000 7f7fa35a9700 1 -- start start 2023-09-30T22:26:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.808+0000 7f7fa35a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f9c0a0a30 0x7f7f9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.808+0000 7f7fa35a9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa35a9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c0a2a70 con 0x7f7f9c09d630 2023-09-30T22:26:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa35a9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0a01d0 con 0x7f7f9c0a0a30 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa35a9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c09e2b0 con 0x7f7f9c138d80 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1345700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f9c0a0a30 0x7f7f9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1b46700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa0b44700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f9c09d630 0x7f7f9c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43374/0 (socket says 172.21.15.32:43374) 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa0b44700 1 -- 172.21.15.32:0/408956159 learned_addr learned my addr 172.21.15.32:0/408956159 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7f93fff700 1 -- 172.21.15.32:0/408956159 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628025499 0 0) 0x7f7f9c0a2a70 con 0x7f7f9c09d630 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7f93fff700 1 -- 172.21.15.32:0/408956159 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f880035b0 con 0x7f7f9c09d630 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1b46700 1 -- 172.21.15.32:0/408956159 >> v2:172.21.15.32:3300/0 conn(0x7f7f9c0a0a30 msgr2=0x7f7f9c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1b46700 1 --2- 172.21.15.32:0/408956159 >> v2:172.21.15.32:3300/0 conn(0x7f7f9c0a0a30 0x7f7f9c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1b46700 1 -- 172.21.15.32:0/408956159 >> v1:172.21.15.32:6789/0 conn(0x7f7f9c09d630 legacy=0x7f7f9c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1b46700 1 -- 172.21.15.32:0/408956159 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c09dc50 con 0x7f7f9c138d80 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1345700 1 --2- 172.21.15.32:0/408956159 >> v2:172.21.15.32:3300/0 conn(0x7f7f9c0a0a30 0x7f7f9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7fa1b46700 1 --2- 172.21.15.32:0/408956159 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c13b160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009840 tx=0x7f7f94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540b5066956b0def server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7f93fff700 1 -- 172.21.15.32:0/408956159 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94011830 con 0x7f7f9c138d80 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.809+0000 7f7f93fff700 1 -- 172.21.15.32:0/408956159 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94011990 con 0x7f7f9c138d80 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7f93fff700 1 -- 172.21.15.32:0/408956159 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94011d20 con 0x7f7f9c138d80 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 -- 172.21.15.32:0/408956159 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 msgr2=0x7f7f9c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/408956159 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c13b160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009840 tx=0x7f7f94009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 -- 172.21.15.32:0/408956159 shutdown_connections 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/408956159 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c13b160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/408956159 >> v2:172.21.15.32:3300/0 conn(0x7f7f9c0a0a30 0x7f7f9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 -- 172.21.15.32:0/408956159 >> 172.21.15.32:0/408956159 conn(0x7f7f9c0a3050 msgr2=0x7f7f9c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 -- 172.21.15.32:0/408956159 shutdown_connections 2023-09-30T22:26:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.810+0000 7f7fa35a9700 1 -- 172.21.15.32:0/408956159 wait complete. 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.811+0000 7f7fa35a9700 1 Processor -- start 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.811+0000 7f7fa35a9700 1 -- start start 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa35a9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f9c09d630 0x7f7f9c133620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa35a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f9c0a0a30 0x7f7f9c133b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa35a9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c1340a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa35a9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c137140 con 0x7f7f9c0a0a30 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa35a9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c136480 con 0x7f7f9c09d630 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa35a9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c13ace0 con 0x7f7f9c138d80 2023-09-30T22:26:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1345700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f9c09d630 0x7f7f9c133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa0b44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f9c0a0a30 0x7f7f9c133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1b46700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c1340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1345700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f9c09d630 0x7f7f9c133620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47258/0 (socket says 172.21.15.32:47258) 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1345700 1 -- 172.21.15.32:0/3664388214 learned_addr learned my addr 172.21.15.32:0/3664388214 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1b46700 1 -- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f9c09d630 msgr2=0x7f7f9c133620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1b46700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f9c09d630 0x7f7f9c133620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1b46700 1 -- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f9c0a0a30 msgr2=0x7f7f9c133b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1b46700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f9c0a0a30 0x7f7f9c133b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa1b46700 1 -- 172.21.15.32:0/3664388214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98009a70 con 0x7f7f9c138d80 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.812+0000 7f7fa0b44700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f9c0a0a30 0x7f7f9c133b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.813+0000 7f7fa1b46700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c1340a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f94016000 tx=0x7f7f94004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.813+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94004510 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.813+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f940036a0 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.813+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94003a30 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.813+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f940094a0 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.813+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c136ba0 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.814+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f94003b90 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.814+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c0747f0 con 0x7f7f9c138d80 2023-09-30T22:26:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.814+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f940400d0 con 0x7f7f9c138d80 2023-09-30T22:26:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.819+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f9401a420 con 0x7f7f9c138d80 2023-09-30T22:26:08.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.849+0000 7fee3268a700 1 Processor -- start 2023-09-30T22:26:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.849+0000 7fee3268a700 1 -- start start 2023-09-30T22:26:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.849+0000 7fee3268a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c0a1510 0x7fee2c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee3268a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee2c139250 0x7fee2c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee3268a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c09edd0 con 0x7fee2c0a0a30 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee3268a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0a18e0 con 0x7fee2c139250 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee3268a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c09e1f0 con 0x7fee2c0a1510 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee30c27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee2c139250 0x7fee2c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee2bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee2c0a0a30 0x7fee2c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43388/0 (socket says 172.21.15.32:43388) 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee2bfff700 1 -- 172.21.15.32:0/2845421740 learned_addr learned my addr 172.21.15.32:0/2845421740 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee2b7fe700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c0a1510 0x7fee2c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee30c27700 1 -- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c0a1510 msgr2=0x7fee2c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee30c27700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c0a1510 0x7fee2c138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee30c27700 1 -- 172.21.15.32:0/2845421740 >> v1:172.21.15.32:6789/0 conn(0x7fee2c0a0a30 legacy=0x7fee2c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee30c27700 1 -- 172.21.15.32:0/2845421740 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c09dc50 con 0x7fee2c139250 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee2b7fe700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c0a1510 0x7fee2c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee30c27700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c139250 0x7fee2c13b630 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fee24009dc0 tx=0x7fee2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8b847e9b241747 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.850+0000 7fee2affd700 1 -- 172.21.15.32:0/2845421740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400e070 con 0x7fee2c139250 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee2affd700 1 -- 172.21.15.32:0/2845421740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2400bae0 con 0x7fee2c139250 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee2affd700 1 -- 172.21.15.32:0/2845421740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400be90 con 0x7fee2c139250 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 -- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c139250 msgr2=0x7fee2c13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c139250 0x7fee2c13b630 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fee24009dc0 tx=0x7fee2400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:08.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 -- 172.21.15.32:0/2845421740 shutdown_connections 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c139250 0x7fee2c13b630 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 --2- 172.21.15.32:0/2845421740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c0a1510 0x7fee2c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 -- 172.21.15.32:0/2845421740 >> 172.21.15.32:0/2845421740 conn(0x7fee2c0a6550 msgr2=0x7fee2c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 -- 172.21.15.32:0/2845421740 shutdown_connections 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.851+0000 7fee3268a700 1 -- 172.21.15.32:0/2845421740 wait complete. 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.853+0000 7fee3268a700 1 Processor -- start 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.853+0000 7fee3268a700 1 -- start start 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.853+0000 7fee3268a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee2c0a0a30 0x7fee2c122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.853+0000 7fee3268a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 0x7fee2c122e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.853+0000 7fee3268a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 0x7fee2c1233a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee3268a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c138250 con 0x7fee2c0a1510 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee3268a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c138810 con 0x7fee2c0a0a30 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee3268a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c124640 con 0x7fee2c139250 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 0x7fee2c1233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee2b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 0x7fee2c122e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee2b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 0x7fee2c122e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54846/0 (socket says 172.21.15.32:54846) 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 0x7fee2c1233a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50308/0 (socket says 172.21.15.32:50308) 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee2b7fe700 1 -- 172.21.15.32:0/214362114 learned_addr learned my addr 172.21.15.32:0/214362114 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee2bfff700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee2c0a0a30 0x7fee2c122920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 -- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee2c0a0a30 msgr2=0x7fee2c122920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee2c0a0a30 0x7fee2c122920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 -- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 msgr2=0x7fee2c122e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 0x7fee2c122e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 -- 172.21.15.32:0/214362114 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200094a0 con 0x7fee2c139250 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee2b7fe700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 0x7fee2c122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee30c27700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 0x7fee2c1233a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee24016000 tx=0x7fee24004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee240045d0 con 0x7fee2c139250 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24020070 con 0x7fee2c139250 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.854+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee24009a70 con 0x7fee2c139250 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.855+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24027480 con 0x7fee2c139250 2023-09-30T22:26:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.855+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c125ef0 con 0x7fee2c139250 2023-09-30T22:26:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.856+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee240275e0 con 0x7fee2c139250 2023-09-30T22:26:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.856+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c074ea0 con 0x7fee2c139250 2023-09-30T22:26:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.856+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fee24022070 con 0x7fee2c139250 2023-09-30T22:26:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:08.861+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee2401a740 con 0x7fee2c139250 2023-09-30T22:26:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.036+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f9c072980 con 0x7f7f9c138d80 2023-09-30T22:26:09.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.036+0000 7f7f91ffb700 1 -- 172.21.15.32:0/3664388214 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f94028030 con 0x7f7f9c138d80 2023-09-30T22:26:09.036 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.038+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 msgr2=0x7f7f9c1340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.038+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c1340a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7f94016000 tx=0x7f7f94004750 comp rx=0 tx=0).stop 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 shutdown_connections 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f9c138d80 0x7f7f9c1340a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f9c09d630 0x7f7f9c133620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 --2- 172.21.15.32:0/3664388214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f9c0a0a30 0x7f7f9c133b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 >> 172.21.15.32:0/3664388214 conn(0x7f7f9c0a3050 msgr2=0x7f7f9c0a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 shutdown_connections 2023-09-30T22:26:09.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.039+0000 7f7fa35a9700 1 -- 172.21.15.32:0/3664388214 wait complete. 2023-09-30T22:26:09.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:09.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.075+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee2c0a28e0 con 0x7fee2c139250 2023-09-30T22:26:09.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.075+0000 7fee28ff9700 1 -- 172.21.15.32:0/214362114 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee2401a380 con 0x7fee2c139250 2023-09-30T22:26:09.075 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 msgr2=0x7fee2c1233a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 0x7fee2c1233a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee24016000 tx=0x7fee24004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 shutdown_connections 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee2c139250 0x7fee2c1233a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee2c0a0a30 0x7fee2c122920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 --2- 172.21.15.32:0/214362114 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee2c0a1510 0x7fee2c122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 >> 172.21.15.32:0/214362114 conn(0x7fee2c0a6550 msgr2=0x7fee2c0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 shutdown_connections 2023-09-30T22:26:09.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.078+0000 7fee3268a700 1 -- 172.21.15.32:0/214362114 wait complete. 2023-09-30T22:26:09.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:09.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:09.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.736+0000 7fe1e50c1700 1 Processor -- start 2023-09-30T22:26:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.736+0000 7fe1e50c1700 1 -- start start 2023-09-30T22:26:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.736+0000 7f4ee27f9700 1 Processor -- start 2023-09-30T22:26:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.736+0000 7f4ee27f9700 1 -- start start 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.736+0000 7fe1e50c1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e00a0a30 0x7fe1e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.736+0000 7fe1e50c1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 0x7fe1e00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1e50c1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 0x7fe1e0133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1e50c1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e00a2a70 con 0x7fe1e0133930 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1e50c1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00a01d0 con 0x7fe1e009d630 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1e50c1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e009e2b0 con 0x7fe1e00a0a30 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 0x7fe1e00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1df59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 0x7fe1e0133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 0x7fe1e00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47282/0 (socket says 172.21.15.32:47282) 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 -- 172.21.15.32:0/3813541132 learned_addr learned my addr 172.21.15.32:0/3813541132 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1df59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 0x7fe1e0133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54856/0 (socket says 172.21.15.32:54856) 2023-09-30T22:26:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1ded9d700 1 --2- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e00a0a30 0x7fe1e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee27f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4edc0a0a30 0x7f4edc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee27f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc0a1340 0x7f4edc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee27f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc09edd0 con 0x7f4edc133fa0 2023-09-30T22:26:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee27f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc09e1f0 con 0x7f4edc0a0a30 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee27f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc09dc50 con 0x7f4edc0a1340 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4edb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc0a1340 0x7f4edc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee0d96700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4edc133fa0 0x7f4edc13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43390/0 (socket says 172.21.15.32:43390) 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4ee0d96700 1 -- 172.21.15.32:0/2990162757 learned_addr learned my addr 172.21.15.32:0/2990162757 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 -- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e00a0a30 msgr2=0x7fe1e00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 --2- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e00a0a30 0x7fe1e00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 -- 172.21.15.32:0/3813541132 >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 msgr2=0x7fe1e0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 --2- 172.21.15.32:0/3813541132 >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 0x7fe1e0133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1de59c700 1 -- 172.21.15.32:0/3813541132 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e009dc50 con 0x7fe1e009d630 2023-09-30T22:26:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7fe1df59e700 1 --2- 172.21.15.32:0/3813541132 >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 0x7fe1e0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.737+0000 7f4edbfff700 1 --2- 172.21.15.32:0/2990162757 >> v2:172.21.15.32:3300/0 conn(0x7f4edc0a0a30 0x7f4edc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edb7fe700 1 -- 172.21.15.32:0/2990162757 >> v2:172.21.15.32:3300/0 conn(0x7f4edc0a0a30 msgr2=0x7f4edc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edb7fe700 1 --2- 172.21.15.32:0/2990162757 >> v2:172.21.15.32:3300/0 conn(0x7f4edc0a0a30 0x7f4edc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edb7fe700 1 -- 172.21.15.32:0/2990162757 >> v1:172.21.15.32:6789/0 conn(0x7f4edc133fa0 legacy=0x7f4edc13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edb7fe700 1 -- 172.21.15.32:0/2990162757 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0a1fe0 con 0x7f4edc0a1340 2023-09-30T22:26:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7fe1de59c700 1 --2- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 0x7fe1e00a1340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40098f0 tx=0x7fe1d4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2e9f258ceced18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7fe1ddd9b700 1 -- 172.21.15.32:0/3813541132 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4011830 con 0x7fe1e009d630 2023-09-30T22:26:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7fe1ddd9b700 1 -- 172.21.15.32:0/3813541132 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d4011990 con 0x7fe1e009d630 2023-09-30T22:26:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edb7fe700 1 --2- 172.21.15.32:0/2990162757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc0a1340 0x7f4edc133a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009840 tx=0x7f4ed0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523392caa70768f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7fe1ddd9b700 1 -- 172.21.15.32:0/3813541132 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4011d20 con 0x7fe1e009d630 2023-09-30T22:26:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edaffd700 1 -- 172.21.15.32:0/2990162757 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0011830 con 0x7f4edc0a1340 2023-09-30T22:26:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edaffd700 1 -- 172.21.15.32:0/2990162757 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed0011990 con 0x7f4edc0a1340 2023-09-30T22:26:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7f4edaffd700 1 -- 172.21.15.32:0/2990162757 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0011d20 con 0x7f4edc0a1340 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7fe1e50c1700 1 -- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 msgr2=0x7fe1e00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.738+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 0x7fe1e00a1340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40098f0 tx=0x7fe1d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 -- 172.21.15.32:0/3813541132 shutdown_connections 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/3813541132 >> v2:172.21.15.32:3300/0 conn(0x7fe1e0133930 0x7fe1e0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e00a0a30 0x7fe1e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/3813541132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e009d630 0x7fe1e00a1340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 -- 172.21.15.32:0/3813541132 >> 172.21.15.32:0/3813541132 conn(0x7fe1e00a3050 msgr2=0x7fe1e00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 -- 172.21.15.32:0/3813541132 shutdown_connections 2023-09-30T22:26:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7fe1e50c1700 1 -- 172.21.15.32:0/3813541132 wait complete. 2023-09-30T22:26:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 -- 172.21.15.32:0/2990162757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc0a1340 msgr2=0x7f4edc133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/2990162757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc0a1340 0x7f4edc133a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009840 tx=0x7f4ed0009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 -- 172.21.15.32:0/2990162757 shutdown_connections 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/2990162757 >> v2:172.21.15.32:3300/0 conn(0x7f4edc0a0a30 0x7f4edc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/2990162757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc0a1340 0x7f4edc133a60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 -- 172.21.15.32:0/2990162757 >> 172.21.15.32:0/2990162757 conn(0x7f4edc0a6550 msgr2=0x7f4edc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 -- 172.21.15.32:0/2990162757 shutdown_connections 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.739+0000 7f4ee27f9700 1 -- 172.21.15.32:0/2990162757 wait complete. 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.740+0000 7f4ee27f9700 1 Processor -- start 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.740+0000 7f4ee27f9700 1 -- start start 2023-09-30T22:26:09.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.740+0000 7fe1e50c1700 1 Processor -- start 2023-09-30T22:26:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.740+0000 7fe1e50c1700 1 -- start start 2023-09-30T22:26:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.740+0000 7f4ee27f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4edc0a0a30 0x7f4edc11e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.740+0000 7f4ee27f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 0x7f4edc11ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4ee27f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc133fa0 0x7f4edc11f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4ee27f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1208f0 con 0x7f4edc0a0a30 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4ee27f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0a0220 con 0x7f4edc133fa0 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4ee27f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1203b0 con 0x7f4edc0a1340 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1e50c1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 0x7fe1e011f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1e50c1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 0x7fe1e011fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1e50c1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0133930 0x7fe1e01c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1e50c1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00a0410 con 0x7fe1e0133930 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1e50c1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e0122640 con 0x7fe1e00a0a30 2023-09-30T22:26:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1e50c1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e011ef20 con 0x7fe1e009d630 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1de59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 0x7fe1e011fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 0x7fe1e011f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1de59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 0x7fe1e011fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47308/0 (socket says 172.21.15.32:47308) 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1de59c700 1 -- 172.21.15.32:0/1121518210 learned_addr learned my addr 172.21.15.32:0/1121518210 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 0x7fe1e011f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50340/0 (socket says 172.21.15.32:50340) 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1df59e700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0133930 0x7fe1e01c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 0x7f4edc11ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 0x7f4edc11ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50324/0 (socket says 172.21.15.32:50324) 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 -- 172.21.15.32:0/4096031297 learned_addr learned my addr 172.21.15.32:0/4096031297 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4ee0d96700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc133fa0 0x7f4edc11f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 -- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc133fa0 msgr2=0x7f4edc11f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc133fa0 0x7f4edc11f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 -- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4edc0a0a30 msgr2=0x7f4edc11e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4edc0a0a30 0x7f4edc11e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7f4edb7fe700 1 -- 172.21.15.32:0/4096031297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00094a0 con 0x7f4edc0a1340 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 -- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 msgr2=0x7fe1e011fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 0x7fe1e011fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 -- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0133930 msgr2=0x7fe1e01c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0133930 0x7fe1e01c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.741+0000 7fe1ded9d700 1 -- 172.21.15.32:0/1121518210 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009a70 con 0x7fe1e009d630 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1de59c700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 0x7fe1e011fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7f4edb7fe700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 0x7f4edc11ec60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00090b0 tx=0x7f4ed0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1ded9d700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 0x7fe1e011f4c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000e930 tx=0x7fe1d000ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00036a0 con 0x7f4edc0a1340 2023-09-30T22:26:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed001f070 con 0x7f4edc0a1340 2023-09-30T22:26:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc121ed0 con 0x7f4edc0a1340 2023-09-30T22:26:09.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0003a20 con 0x7f4edc0a1340 2023-09-30T22:26:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000c640 con 0x7fe1e009d630 2023-09-30T22:26:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d000c7a0 con 0x7fe1e009d630 2023-09-30T22:26:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d4009550 con 0x7fe1e009d630 2023-09-30T22:26:09.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1221b0 con 0x7f4edc0a1340 2023-09-30T22:26:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e0121fa0 con 0x7fe1e009d630 2023-09-30T22:26:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.742+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000cb10 con 0x7fe1e009d630 2023-09-30T22:26:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.743+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc0a1d40 con 0x7f4edc0a1340 2023-09-30T22:26:09.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.743+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c00052e0 con 0x7fe1e009d630 2023-09-30T22:26:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.743+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ed0003b80 con 0x7f4edc0a1340 2023-09-30T22:26:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.744+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1d000cc70 con 0x7fe1e009d630 2023-09-30T22:26:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.744+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ed0021070 con 0x7f4edc0a1340 2023-09-30T22:26:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.744+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe1d003fa10 con 0x7fe1e009d630 2023-09-30T22:26:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.748+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ed0024380 con 0x7f4edc0a1340 2023-09-30T22:26:09.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.753+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1d001b3b0 con 0x7fe1e009d630 2023-09-30T22:26:09.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.970+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4edc074ea0 con 0x7f4edc0a1340 2023-09-30T22:26:09.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.970+0000 7f4ed8ff9700 1 -- 172.21.15.32:0/4096031297 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ed0015030 con 0x7f4edc0a1340 2023-09-30T22:26:09.970 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:09.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 msgr2=0x7f4edc11ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 0x7f4edc11ec60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00090b0 tx=0x7f4ed0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 shutdown_connections 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4edc133fa0 0x7f4edc11f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4edc0a0a30 0x7f4edc11e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 --2- 172.21.15.32:0/4096031297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4edc0a1340 0x7f4edc11ec60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.972+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 >> 172.21.15.32:0/4096031297 conn(0x7f4edc0a6550 msgr2=0x7f4edc1bc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.973+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 shutdown_connections 2023-09-30T22:26:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.973+0000 7f4ee27f9700 1 -- 172.21.15.32:0/4096031297 wait complete. 2023-09-30T22:26:09.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.976+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe1c0005150 con 0x7fe1e009d630 2023-09-30T22:26:09.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.976+0000 7fe1c77fe700 1 -- 172.21.15.32:0/1121518210 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1d001acb0 con 0x7fe1e009d630 2023-09-30T22:26:09.976 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 msgr2=0x7fe1e011f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 0x7fe1e011f4c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000e930 tx=0x7fe1d000ec40 comp rx=0 tx=0).stop 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 shutdown_connections 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1e0133930 0x7fe1e01c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1e009d630 0x7fe1e011f4c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 --2- 172.21.15.32:0/1121518210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1e00a0a30 0x7fe1e011fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 >> 172.21.15.32:0/1121518210 conn(0x7fe1e00a3050 msgr2=0x7fe1e009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 shutdown_connections 2023-09-30T22:26:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:09.981+0000 7fe1e50c1700 1 -- 172.21.15.32:0/1121518210 wait complete. 2023-09-30T22:26:09.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:09.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:10.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:10.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.196+0000 7fba4b697700 1 Processor -- start 2023-09-30T22:26:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.196+0000 7fba4b697700 1 -- start start 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba4b697700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 0x7fba440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba4b697700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 0x7fba44133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba4b697700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 0x7fba44134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba4b697700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4409edd0 con 0x7fba440a1340 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba4b697700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4409e1f0 con 0x7fba440a0a30 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba4b697700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba4409dc50 con 0x7fba44133fa0 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba49433700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 0x7fba440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba48c32700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 0x7fba44133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba49c34700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 0x7fba44134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba48c32700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 0x7fba44133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54892/0 (socket says 172.21.15.32:54892) 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba48c32700 1 -- 172.21.15.32:0/3132746389 learned_addr learned my addr 172.21.15.32:0/3132746389 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba49433700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 0x7fba440a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47314/0 (socket says 172.21.15.32:47314) 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.197+0000 7fba49c34700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 0x7fba44134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50344/0 (socket says 172.21.15.32:50344) 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49433700 1 -- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 msgr2=0x7fba44134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49433700 1 --2- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 0x7fba44134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49433700 1 -- 172.21.15.32:0/3132746389 >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 msgr2=0x7fba44133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49433700 1 --2- 172.21.15.32:0/3132746389 >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 0x7fba44133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49433700 1 -- 172.21.15.32:0/3132746389 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba440a1fe0 con 0x7fba440a0a30 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba48c32700 1 --2- 172.21.15.32:0/3132746389 >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 0x7fba44133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49c34700 1 --2- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 0x7fba44134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba49433700 1 --2- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 0x7fba440a0e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba40009870 tx=0x7fba40009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5fa741c83d391b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba3bfff700 1 -- 172.21.15.32:0/3132746389 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40011830 con 0x7fba440a0a30 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba3bfff700 1 -- 172.21.15.32:0/3132746389 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba40011990 con 0x7fba440a0a30 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.198+0000 7fba3bfff700 1 -- 172.21.15.32:0/3132746389 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40011d20 con 0x7fba440a0a30 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 -- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 msgr2=0x7fba440a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 --2- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 0x7fba440a0e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba40009870 tx=0x7fba40009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 -- 172.21.15.32:0/3132746389 shutdown_connections 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 --2- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba440a0a30 0x7fba440a0e00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 --2- 172.21.15.32:0/3132746389 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba44133fa0 0x7fba44134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 --2- 172.21.15.32:0/3132746389 >> v2:172.21.15.32:3300/0 conn(0x7fba440a1340 0x7fba44133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 -- 172.21.15.32:0/3132746389 >> 172.21.15.32:0/3132746389 conn(0x7fba440a6550 msgr2=0x7fba440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 -- 172.21.15.32:0/3132746389 shutdown_connections 2023-09-30T22:26:10.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.199+0000 7fba4b697700 1 -- 172.21.15.32:0/3132746389 wait complete. 2023-09-30T22:26:10.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 Processor -- start 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 -- start start 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba440a0a30 0x7fba44120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba440a1340 0x7fba44120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba44133fa0 0x7fba44120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba441215e0 con 0x7fba440a1340 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba44123c00 con 0x7fba44133fa0 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.200+0000 7fba4b697700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba44122fb0 con 0x7fba440a0a30 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba48c32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba440a1340 0x7fba44120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49c34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba44133fa0 0x7fba44120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba48c32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba440a1340 0x7fba44120670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54906/0 (socket says 172.21.15.32:54906) 2023-09-30T22:26:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba48c32700 1 -- 172.21.15.32:0/128030130 learned_addr learned my addr 172.21.15.32:0/128030130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba440a0a30 0x7fba44120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 -- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba44133fa0 msgr2=0x7fba44120bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba44133fa0 0x7fba44120bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 -- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba440a1340 msgr2=0x7fba44120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba440a1340 0x7fba44120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 -- 172.21.15.32:0/128030130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba34009550 con 0x7fba440a0a30 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49c34700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba44133fa0 0x7fba44120bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba49433700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba440a0a30 0x7fba44120130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba40016000 tx=0x7fba40004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40004450 con 0x7fba440a0a30 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4001f070 con 0x7fba440a0a30 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c009a70 con 0x7fba440a0a30 2023-09-30T22:26:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.201+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400038a0 con 0x7fba440a0a30 2023-09-30T22:26:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.202+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba400094a0 con 0x7fba440a0a30 2023-09-30T22:26:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.202+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba40003a00 con 0x7fba440a0a30 2023-09-30T22:26:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.203+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba40021070 con 0x7fba440a0a30 2023-09-30T22:26:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.203+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba440a0280 con 0x7fba440a0a30 2023-09-30T22:26:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.212+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba4002b080 con 0x7fba440a0a30 2023-09-30T22:26:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.241+0000 7f67c11d3700 1 Processor -- start 2023-09-30T22:26:10.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.241+0000 7f67c11d3700 1 -- start start 2023-09-30T22:26:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67c11d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67c11d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67bc09d630 0x7f67bc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67c11d3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc0a2a70 con 0x7f67bc133930 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67c11d3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc0a01d0 con 0x7f67bc09d630 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67c11d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc09e2b0 con 0x7f67bc0a0a30 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67ba59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67bc09d630 0x7f67bc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67bad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67bb59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67bc133930 0x7f67bc133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43398/0 (socket says 172.21.15.32:43398) 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.242+0000 7f67bb59e700 1 -- 172.21.15.32:0/877742156 learned_addr learned my addr 172.21.15.32:0/877742156 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67ba59c700 1 -- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 msgr2=0x7f67bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67ba59c700 1 --2- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67ba59c700 1 -- 172.21.15.32:0/877742156 >> v1:172.21.15.32:6789/0 conn(0x7f67bc133930 legacy=0x7f67bc133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67ba59c700 1 -- 172.21.15.32:0/877742156 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc09dc50 con 0x7f67bc09d630 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67b9d9b700 1 -- 172.21.15.32:0/877742156 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165336113 0 0) 0x7f67bc0a2a70 con 0x7f67bc133930 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67bad9d700 1 --2- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67ba59c700 1 --2- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc0a1340 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009840 tx=0x7f67b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f13366030622f75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67b9d9b700 1 -- 172.21.15.32:0/877742156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0011830 con 0x7f67bc09d630 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67b9d9b700 1 -- 172.21.15.32:0/877742156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0011990 con 0x7f67bc09d630 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67b9d9b700 1 -- 172.21.15.32:0/877742156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0011d20 con 0x7f67bc09d630 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67c11d3700 1 -- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 msgr2=0x7f67bc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.243+0000 7f67c11d3700 1 --2- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc0a1340 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009840 tx=0x7f67b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.244+0000 7f67c11d3700 1 -- 172.21.15.32:0/877742156 shutdown_connections 2023-09-30T22:26:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.244+0000 7f67c11d3700 1 --2- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.244+0000 7f67c11d3700 1 --2- 172.21.15.32:0/877742156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc0a1340 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.244+0000 7f67c11d3700 1 -- 172.21.15.32:0/877742156 >> 172.21.15.32:0/877742156 conn(0x7f67bc0a3050 msgr2=0x7f67bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.244+0000 7f67c11d3700 1 -- 172.21.15.32:0/877742156 shutdown_connections 2023-09-30T22:26:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.244+0000 7f67c11d3700 1 -- 172.21.15.32:0/877742156 wait complete. 2023-09-30T22:26:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 Processor -- start 2023-09-30T22:26:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 -- start start 2023-09-30T22:26:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc1c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc1c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67bc133930 0x7f67bc1c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1210e0 con 0x7f67bc09d630 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1207f0 con 0x7f67bc133930 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.245+0000 7f67c11d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1209e0 con 0x7f67bc0a0a30 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc1c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67bb59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67bc133930 0x7f67bc1c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc1c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50378/0 (socket says 172.21.15.32:50378) 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 -- 172.21.15.32:0/1174215717 learned_addr learned my addr 172.21.15.32:0/1174215717 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67bad9d700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc1c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 -- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67bc133930 msgr2=0x7f67bc1c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67bc133930 0x7f67bc1c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 -- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 msgr2=0x7f67bc1c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc1c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 -- 172.21.15.32:0/1174215717 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009a70 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67bb59e700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67bc133930 0x7f67bc1c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67ba59c700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc1c86d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67b00090b0 tx=0x7f67b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004400 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b001f070 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00038e0 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.246+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00094a0 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.247+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc0a0410 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.247+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67b0021030 con 0x7f67bc0a0a30 2023-09-30T22:26:10.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.247+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f67b0040760 con 0x7f67bc0a0a30 2023-09-30T22:26:10.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.248+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc122380 con 0x7f67bc0a0a30 2023-09-30T22:26:10.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.252+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67b0004560 con 0x7f67bc0a0a30 2023-09-30T22:26:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.418+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fba441235c0 con 0x7fba440a0a30 2023-09-30T22:26:10.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.418+0000 7fba39ffb700 1 -- 172.21.15.32:0/128030130 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba40015020 con 0x7fba440a0a30 2023-09-30T22:26:10.419 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba440a0a30 msgr2=0x7fba44120130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba440a0a30 0x7fba44120130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba40016000 tx=0x7fba40004750 comp rx=0 tx=0).stop 2023-09-30T22:26:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 shutdown_connections 2023-09-30T22:26:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba44133fa0 0x7fba44120bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba440a0a30 0x7fba44120130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 --2- 172.21.15.32:0/128030130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba440a1340 0x7fba44120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 >> 172.21.15.32:0/128030130 conn(0x7fba440a6550 msgr2=0x7fba4409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 shutdown_connections 2023-09-30T22:26:10.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.421+0000 7fba4b697700 1 -- 172.21.15.32:0/128030130 wait complete. 2023-09-30T22:26:10.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:10.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.462+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67bc0a5780 con 0x7f67bc0a0a30 2023-09-30T22:26:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.463+0000 7f67a37fe700 1 -- 172.21.15.32:0/1174215717 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67b0015020 con 0x7f67bc0a0a30 2023-09-30T22:26:10.463 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:10.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 msgr2=0x7f67bc1c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:10.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc1c86d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67b00090b0 tx=0x7f67b0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:10.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 shutdown_connections 2023-09-30T22:26:10.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67bc133930 0x7f67bc1c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67bc09d630 0x7f67bc1c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 --2- 172.21.15.32:0/1174215717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67bc0a0a30 0x7f67bc1c86d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 >> 172.21.15.32:0/1174215717 conn(0x7f67bc0a3050 msgr2=0x7f67bc1bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 shutdown_connections 2023-09-30T22:26:10.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:10.465+0000 7f67c11d3700 1 -- 172.21.15.32:0/1174215717 wait complete. 2023-09-30T22:26:10.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:10.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:10.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:11.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.137+0000 7fedb78d1700 1 Processor -- start 2023-09-30T22:26:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.137+0000 7fedb78d1700 1 -- start start 2023-09-30T22:26:11.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.137+0000 7fedb78d1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb00a0a30 0x7fedb00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.137+0000 7fedb78d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 0x7fedb0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.137+0000 7fedb78d1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb009edd0 con 0x7fedb0133fa0 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb78d1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb009e1f0 con 0x7fedb00a0a30 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb78d1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb009dc50 con 0x7fedb00a1340 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 0x7fedb0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 0x7fedb0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37562/0 (socket says 172.21.15.32:37562) 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb566d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb00a0a30 0x7fedb00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 -- 172.21.15.32:0/639984018 learned_addr learned my addr 172.21.15.32:0/639984018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb5e6e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fedb0133fa0 0x7fedb013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57308/0 (socket says 172.21.15.32:57308) 2023-09-30T22:26:11.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7feda7fff700 1 -- 172.21.15.32:0/639984018 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522925072 0 0) 0x7fedb009edd0 con 0x7fedb0133fa0 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 -- 172.21.15.32:0/639984018 >> v2:172.21.15.32:3300/0 conn(0x7fedb00a0a30 msgr2=0x7fedb00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 --2- 172.21.15.32:0/639984018 >> v2:172.21.15.32:3300/0 conn(0x7fedb00a0a30 0x7fedb00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 -- 172.21.15.32:0/639984018 >> v1:172.21.15.32:6789/0 conn(0x7fedb0133fa0 legacy=0x7fedb013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 -- 172.21.15.32:0/639984018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00a1fe0 con 0x7fedb00a1340 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb566d700 1 --2- 172.21.15.32:0/639984018 >> v2:172.21.15.32:3300/0 conn(0x7fedb00a0a30 0x7fedb00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7fedb4e6c700 1 --2- 172.21.15.32:0/639984018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 0x7fedb0133a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feda00098f0 tx=0x7feda0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b43ead162f0c29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.138+0000 7feda7fff700 1 -- 172.21.15.32:0/639984018 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0011830 con 0x7fedb00a1340 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7feda7fff700 1 -- 172.21.15.32:0/639984018 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda0011990 con 0x7fedb00a1340 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7feda7fff700 1 -- 172.21.15.32:0/639984018 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0011d20 con 0x7fedb00a1340 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 -- 172.21.15.32:0/639984018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 msgr2=0x7fedb0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 --2- 172.21.15.32:0/639984018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 0x7fedb0133a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feda00098f0 tx=0x7feda0009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 -- 172.21.15.32:0/639984018 shutdown_connections 2023-09-30T22:26:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 --2- 172.21.15.32:0/639984018 >> v2:172.21.15.32:3300/0 conn(0x7fedb00a0a30 0x7fedb00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 --2- 172.21.15.32:0/639984018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb00a1340 0x7fedb0133a60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 -- 172.21.15.32:0/639984018 >> 172.21.15.32:0/639984018 conn(0x7fedb00a6550 msgr2=0x7fedb00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 -- 172.21.15.32:0/639984018 shutdown_connections 2023-09-30T22:26:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.139+0000 7fedb78d1700 1 -- 172.21.15.32:0/639984018 wait complete. 2023-09-30T22:26:11.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.140+0000 7fedb78d1700 1 Processor -- start 2023-09-30T22:26:11.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.140+0000 7fedb78d1700 1 -- start start 2023-09-30T22:26:11.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb78d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedb00a0a30 0x7fedb0123470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb78d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 0x7fedb01239b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb78d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb0133fa0 0x7fedb01c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb78d1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0126370 con 0x7fedb00a1340 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb78d1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0126550 con 0x7fedb0133fa0 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb78d1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00a3580 con 0x7fedb00a0a30 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 0x7fedb01239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb566d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedb00a0a30 0x7fedb0123470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 0x7fedb01239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38232/0 (socket says 172.21.15.32:38232) 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 -- 172.21.15.32:0/3456055611 learned_addr learned my addr 172.21.15.32:0/3456055611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb5e6e700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb0133fa0 0x7fedb01c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 -- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedb00a0a30 msgr2=0x7fedb0123470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedb00a0a30 0x7fedb0123470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 -- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb0133fa0 msgr2=0x7fedb01c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb0133fa0 0x7fedb01c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.141+0000 7fedb4e6c700 1 -- 172.21.15.32:0/3456055611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0094a0 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.142+0000 7fedb4e6c700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 0x7fedb01239b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fedb009dd90 tx=0x7feda0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.142+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0004340 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.142+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda001f070 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.142+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00036a0 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.142+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda0009550 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.142+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb0133750 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.143+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feda0021030 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.143+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb0074ea0 con 0x7fedb00a1340 2023-09-30T22:26:11.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.143+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feda0040ab0 con 0x7fedb00a1340 2023-09-30T22:26:11.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.148+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feda00199a0 con 0x7fedb00a1340 2023-09-30T22:26:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.184+0000 7f1c2cc68700 1 Processor -- start 2023-09-30T22:26:11.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.184+0000 7f1c2cc68700 1 -- start start 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2cc68700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c2809d630 0x7f1c280a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2cc68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c28138d80 0x7f1c2813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2cc68700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c280a2a70 con 0x7f1c280a0a30 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2cc68700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c280a01d0 con 0x7f1c2809d630 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2cc68700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2809e2b0 con 0x7f1c28138d80 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c25d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c2809d630 0x7f1c280a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c280a0a30 0x7f1c280a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57320/0 (socket says 172.21.15.32:57320) 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c2659c700 1 -- 172.21.15.32:0/3232707798 learned_addr learned my addr 172.21.15.32:0/3232707798 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.185+0000 7f1c26d9d700 1 --2- 172.21.15.32:0/3232707798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c28138d80 0x7f1c2813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c26d9d700 1 -- 172.21.15.32:0/3232707798 >> v2:172.21.15.32:3300/0 conn(0x7f1c2809d630 msgr2=0x7f1c280a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c26d9d700 1 --2- 172.21.15.32:0/3232707798 >> v2:172.21.15.32:3300/0 conn(0x7f1c2809d630 0x7f1c280a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c26d9d700 1 -- 172.21.15.32:0/3232707798 >> v1:172.21.15.32:6789/0 conn(0x7f1c280a0a30 legacy=0x7f1c280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c2559a700 1 -- 172.21.15.32:0/3232707798 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950242283 0 0) 0x7f1c280a2a70 con 0x7f1c280a0a30 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c26d9d700 1 -- 172.21.15.32:0/3232707798 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2809dc50 con 0x7f1c28138d80 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c25d9b700 1 --2- 172.21.15.32:0/3232707798 >> v2:172.21.15.32:3300/0 conn(0x7f1c2809d630 0x7f1c280a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c26d9d700 1 --2- 172.21.15.32:0/3232707798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c28138d80 0x7f1c2813b160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009d90 tx=0x7f1c2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c5f875ed5ef09e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c2559a700 1 -- 172.21.15.32:0/3232707798 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2000e070 con 0x7f1c28138d80 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c2559a700 1 -- 172.21.15.32:0/3232707798 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2000bb20 con 0x7f1c28138d80 2023-09-30T22:26:11.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.186+0000 7f1c2559a700 1 -- 172.21.15.32:0/3232707798 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20012430 con 0x7f1c28138d80 2023-09-30T22:26:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 -- 172.21.15.32:0/3232707798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c28138d80 msgr2=0x7f1c2813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/3232707798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c28138d80 0x7f1c2813b160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009d90 tx=0x7f1c2000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 -- 172.21.15.32:0/3232707798 shutdown_connections 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/3232707798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c28138d80 0x7f1c2813b160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/3232707798 >> v2:172.21.15.32:3300/0 conn(0x7f1c2809d630 0x7f1c280a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 -- 172.21.15.32:0/3232707798 >> 172.21.15.32:0/3232707798 conn(0x7f1c280a3050 msgr2=0x7f1c280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 -- 172.21.15.32:0/3232707798 shutdown_connections 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.187+0000 7f1c2cc68700 1 -- 172.21.15.32:0/3232707798 wait complete. 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.189+0000 7f1c2cc68700 1 Processor -- start 2023-09-30T22:26:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.189+0000 7f1c2cc68700 1 -- start start 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2cc68700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 0x7f1c2812c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2cc68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 0x7f1c2812c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2cc68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 0x7f1c2812cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2cc68700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28130010 con 0x7f1c28138d80 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2cc68700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2813ad00 con 0x7f1c280a0a30 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2cc68700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c280a0410 con 0x7f1c2809d630 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c26d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 0x7f1c2812cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 0x7f1c2812c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 0x7f1c2812c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 0x7f1c2812c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55662/0 (socket says 172.21.15.32:55662) 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c26d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 0x7f1c2812cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38262/0 (socket says 172.21.15.32:38262) 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2659c700 1 -- 172.21.15.32:0/2106572229 learned_addr learned my addr 172.21.15.32:0/2106572229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 0x7f1c2812c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37580/0 (socket says 172.21.15.32:37580) 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 -- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 msgr2=0x7f1c2812c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 0x7f1c2812c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 -- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 msgr2=0x7f1c2812cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 0x7f1c2812cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 -- 172.21.15.32:0/2106572229 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0094a0 con 0x7f1c280a0a30 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c2659c700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 0x7f1c2812c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c26d9d700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 0x7f1c2812cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.190+0000 7f1c25d9b700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 0x7f1c2812c9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00e9b0 tx=0x7f1c1c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.191+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00ee50 con 0x7f1c280a0a30 2023-09-30T22:26:11.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.191+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00c850 con 0x7f1c280a0a30 2023-09-30T22:26:11.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.191+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c20009a70 con 0x7f1c280a0a30 2023-09-30T22:26:11.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.191+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00cc00 con 0x7f1c280a0a30 2023-09-30T22:26:11.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.191+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2812e000 con 0x7f1c280a0a30 2023-09-30T22:26:11.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.191+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c1c013470 con 0x7f1c280a0a30 2023-09-30T22:26:11.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.192+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1c1c03fa70 con 0x7f1c280a0a30 2023-09-30T22:26:11.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.192+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2812fc90 con 0x7f1c280a0a30 2023-09-30T22:26:11.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.197+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c1c013720 con 0x7f1c280a0a30 2023-09-30T22:26:11.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.367+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fedb009bc20 con 0x7fedb00a1340 2023-09-30T22:26:11.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.367+0000 7feda5ffb700 1 -- 172.21.15.32:0/3456055611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feda0015070 con 0x7fedb00a1340 2023-09-30T22:26:11.367 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.369+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 msgr2=0x7fedb01239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.369+0000 7fedb78d1700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 0x7fedb01239b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fedb009dd90 tx=0x7feda0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 shutdown_connections 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedb0133fa0 0x7fedb01c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedb00a0a30 0x7fedb0123470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 --2- 172.21.15.32:0/3456055611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00a1340 0x7fedb01239b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 >> 172.21.15.32:0/3456055611 conn(0x7fedb00a6550 msgr2=0x7fedb011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 shutdown_connections 2023-09-30T22:26:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.370+0000 7fedb78d1700 1 -- 172.21.15.32:0/3456055611 wait complete. 2023-09-30T22:26:11.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:11.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.401+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1c280747f0 con 0x7f1c280a0a30 2023-09-30T22:26:11.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.402+0000 7f1c16ffd700 1 -- 172.21.15.32:0/2106572229 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c1c021030 con 0x7f1c280a0a30 2023-09-30T22:26:11.402 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 msgr2=0x7f1c2812c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 0x7f1c2812c9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00e9b0 tx=0x7f1c1c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 shutdown_connections 2023-09-30T22:26:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c28138d80 0x7f1c2812cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2809d630 0x7f1c2812c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 --2- 172.21.15.32:0/2106572229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c280a0a30 0x7f1c2812c9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 >> 172.21.15.32:0/2106572229 conn(0x7f1c280a3050 msgr2=0x7f1c280a4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 shutdown_connections 2023-09-30T22:26:11.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.404+0000 7f1c2cc68700 1 -- 172.21.15.32:0/2106572229 wait complete. 2023-09-30T22:26:11.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:11.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:11.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 Processor -- start 2023-09-30T22:26:11.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 -- start start 2023-09-30T22:26:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e200a1510 0x7f4e20138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e2013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2009edd0 con 0x7f4e200a0a30 2023-09-30T22:26:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e200a18e0 con 0x7f4e200a1510 2023-09-30T22:26:11.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.600+0000 7f4e24f62700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2009e1f0 con 0x7f4e20139250 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e2013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e200a1510 0x7f4e20138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e200a0a30 0x7f4e200a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57324/0 (socket says 172.21.15.32:57324) 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1e59c700 1 -- 172.21.15.32:0/2719309344 learned_addr learned my addr 172.21.15.32:0/2719309344 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1dd9b700 1 -- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 msgr2=0x7f4e2013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1dd9b700 1 --2- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e2013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1dd9b700 1 -- 172.21.15.32:0/2719309344 >> v1:172.21.15.32:6789/0 conn(0x7f4e200a0a30 legacy=0x7f4e200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1dd9b700 1 -- 172.21.15.32:0/2719309344 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2009dc50 con 0x7f4e200a1510 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1d59a700 1 -- 172.21.15.32:0/2719309344 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430559157 0 0) 0x7f4e2009edd0 con 0x7f4e200a0a30 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1ed9d700 1 --2- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e2013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1dd9b700 1 --2- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20138d10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009840 tx=0x7f4e14009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2157e6b51e77f4d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1d59a700 1 -- 172.21.15.32:0/2719309344 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14011830 con 0x7f4e200a1510 2023-09-30T22:26:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1d59a700 1 -- 172.21.15.32:0/2719309344 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e14011990 con 0x7f4e200a1510 2023-09-30T22:26:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.601+0000 7f4e1d59a700 1 -- 172.21.15.32:0/2719309344 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14011d20 con 0x7f4e200a1510 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 -- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 msgr2=0x7f4e20138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 --2- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20138d10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009840 tx=0x7f4e14009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 -- 172.21.15.32:0/2719309344 shutdown_connections 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 --2- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e2013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 --2- 172.21.15.32:0/2719309344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20138d10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 -- 172.21.15.32:0/2719309344 >> 172.21.15.32:0/2719309344 conn(0x7f4e200a6550 msgr2=0x7f4e200a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 -- 172.21.15.32:0/2719309344 shutdown_connections 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.602+0000 7f4e24f62700 1 -- 172.21.15.32:0/2719309344 wait complete. 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.603+0000 7f4e24f62700 1 Processor -- start 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 -- start start 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e200a0a30 0x7f4e201228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e20123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201389e0 con 0x7f4e200a1510 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200a0220 con 0x7f4e20139250 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e24f62700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20126060 con 0x7f4e200a0a30 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e1e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e200a0a30 0x7f4e201228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e1ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e20123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e1e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e200a0a30 0x7f4e201228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55678/0 (socket says 172.21.15.32:55678) 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e1dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.604+0000 7f4e1e59c700 1 -- 172.21.15.32:0/1450356161 learned_addr learned my addr 172.21.15.32:0/1450356161 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1ed9d700 1 -- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e200a0a30 msgr2=0x7f4e201228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1ed9d700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e200a0a30 0x7f4e201228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1ed9d700 1 -- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 msgr2=0x7f4e20122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1ed9d700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1ed9d700 1 -- 172.21.15.32:0/1450356161 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e18009a70 con 0x7f4e20139250 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1dd9b700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e1ed9d700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e20123340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800ef90 tx=0x7f4e1800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800ca90 con 0x7f4e20139250 2023-09-30T22:26:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1800f040 con 0x7f4e20139250 2023-09-30T22:26:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e140094a0 con 0x7f4e20139250 2023-09-30T22:26:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800ce30 con 0x7f4e20139250 2023-09-30T22:26:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.605+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e20124bb0 con 0x7f4e20139250 2023-09-30T22:26:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.606+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4e18014480 con 0x7f4e20139250 2023-09-30T22:26:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.606+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4e1803fc50 con 0x7f4e20139250 2023-09-30T22:26:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.606+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e20074ea0 con 0x7f4e20139250 2023-09-30T22:26:11.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.611+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e18012930 con 0x7f4e20139250 2023-09-30T22:26:11.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.641+0000 7f7d64ab8700 1 Processor -- start 2023-09-30T22:26:11.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.641+0000 7f7d64ab8700 1 -- start start 2023-09-30T22:26:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.641+0000 7f7d64ab8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6009d630 0x7f7d600a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.641+0000 7f7d64ab8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d60138d80 0x7f7d6013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d64ab8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d600a2a70 con 0x7f7d600a0a30 2023-09-30T22:26:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d64ab8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d600a01d0 con 0x7f7d60138d80 2023-09-30T22:26:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d64ab8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6009e2b0 con 0x7f7d6009d630 2023-09-30T22:26:11.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d60138d80 0x7f7d6013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6009d630 0x7f7d600a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7d600a0a30 0x7f7d600a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57340/0 (socket says 172.21.15.32:57340) 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5e59c700 1 -- 172.21.15.32:0/2151749650 learned_addr learned my addr 172.21.15.32:0/2151749650 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5dd9b700 1 -- 172.21.15.32:0/2151749650 >> v2:172.21.15.32:3300/0 conn(0x7f7d60138d80 msgr2=0x7f7d6013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5dd9b700 1 --2- 172.21.15.32:0/2151749650 >> v2:172.21.15.32:3300/0 conn(0x7f7d60138d80 0x7f7d6013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5dd9b700 1 -- 172.21.15.32:0/2151749650 >> v1:172.21.15.32:6789/0 conn(0x7f7d600a0a30 legacy=0x7f7d600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5dd9b700 1 -- 172.21.15.32:0/2151749650 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6009dc50 con 0x7f7d6009d630 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5ed9d700 1 --2- 172.21.15.32:0/2151749650 >> v2:172.21.15.32:3300/0 conn(0x7f7d60138d80 0x7f7d6013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5dd9b700 1 --2- 172.21.15.32:0/2151749650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6009d630 0x7f7d600a1510 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009840 tx=0x7f7d54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5feec304e8d15772 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5d59a700 1 -- 172.21.15.32:0/2151749650 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54011830 con 0x7f7d6009d630 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.642+0000 7f7d5d59a700 1 -- 172.21.15.32:0/2151749650 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d54011990 con 0x7f7d6009d630 2023-09-30T22:26:11.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d5d59a700 1 -- 172.21.15.32:0/2151749650 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54011d20 con 0x7f7d6009d630 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 -- 172.21.15.32:0/2151749650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6009d630 msgr2=0x7f7d600a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/2151749650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6009d630 0x7f7d600a1510 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009840 tx=0x7f7d54009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 -- 172.21.15.32:0/2151749650 shutdown_connections 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/2151749650 >> v2:172.21.15.32:3300/0 conn(0x7f7d60138d80 0x7f7d6013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/2151749650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6009d630 0x7f7d600a1510 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 -- 172.21.15.32:0/2151749650 >> 172.21.15.32:0/2151749650 conn(0x7f7d600a3050 msgr2=0x7f7d600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 -- 172.21.15.32:0/2151749650 shutdown_connections 2023-09-30T22:26:11.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.643+0000 7f7d64ab8700 1 -- 172.21.15.32:0/2151749650 wait complete. 2023-09-30T22:26:11.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.644+0000 7f7d64ab8700 1 Processor -- start 2023-09-30T22:26:11.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.644+0000 7f7d64ab8700 1 -- start start 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d64ab8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 0x7f7d60133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d64ab8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d600a0a30 0x7f7d60134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d64ab8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d60138d80 0x7f7d601345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d64ab8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60135010 con 0x7f7d6009d630 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d64ab8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6013afc0 con 0x7f7d600a0a30 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d64ab8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60135c30 con 0x7f7d60138d80 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 0x7f7d60133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d600a0a30 0x7f7d60134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 0x7f7d60133b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38304/0 (socket says 172.21.15.32:38304) 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 -- 172.21.15.32:0/4279560111 learned_addr learned my addr 172.21.15.32:0/4279560111 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5ed9d700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d60138d80 0x7f7d601345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:11.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 -- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d60138d80 msgr2=0x7f7d601345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d60138d80 0x7f7d601345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 -- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d600a0a30 msgr2=0x7f7d60134090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d600a0a30 0x7f7d60134090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.645+0000 7f7d5e59c700 1 -- 172.21.15.32:0/4279560111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580094a0 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.646+0000 7f7d5e59c700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 0x7f7d60133b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000cbd0 tx=0x7f7d5000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.646+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000aa00 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.646+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5000ab60 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.646+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d540094a0 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.646+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50012400 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.646+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d60137590 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.647+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d5001b3b0 con 0x7f7d6009d630 2023-09-30T22:26:11.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.647+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d5003f300 con 0x7f7d6009d630 2023-09-30T22:26:11.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.647+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d600747f0 con 0x7f7d6009d630 2023-09-30T22:26:11.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.652+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d500192c0 con 0x7f7d6009d630 2023-09-30T22:26:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.819+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4e200a28e0 con 0x7f4e20139250 2023-09-30T22:26:11.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.819+0000 7f4e0effd700 1 -- 172.21.15.32:0/1450356161 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e18009cc0 con 0x7f4e20139250 2023-09-30T22:26:11.819 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.821+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 msgr2=0x7f4e20123340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.821+0000 7f4e24f62700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e20123340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800ef90 tx=0x7f4e1800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 shutdown_connections 2023-09-30T22:26:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4e20139250 0x7f4e20123340 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4e200a0a30 0x7f4e201228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 --2- 172.21.15.32:0/1450356161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e200a1510 0x7f4e20122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 >> 172.21.15.32:0/1450356161 conn(0x7f4e200a6550 msgr2=0x7f4e200a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 shutdown_connections 2023-09-30T22:26:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.822+0000 7f4e24f62700 1 -- 172.21.15.32:0/1450356161 wait complete. 2023-09-30T22:26:11.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:11.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.858+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7d60072980 con 0x7f7d6009d630 2023-09-30T22:26:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.858+0000 7f7d4effd700 1 -- 172.21.15.32:0/4279560111 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d50017020 con 0x7f7d6009d630 2023-09-30T22:26:11.858 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:11.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 msgr2=0x7f7d60133b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:11.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 0x7f7d60133b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000cbd0 tx=0x7f7d5000cee0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 shutdown_connections 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d60138d80 0x7f7d601345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d6009d630 0x7f7d60133b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 --2- 172.21.15.32:0/4279560111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d600a0a30 0x7f7d60134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 >> 172.21.15.32:0/4279560111 conn(0x7f7d600a3050 msgr2=0x7f7d6009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 shutdown_connections 2023-09-30T22:26:11.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:11.860+0000 7f7d64ab8700 1 -- 172.21.15.32:0/4279560111 wait complete. 2023-09-30T22:26:11.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:12.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:12.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 Processor -- start 2023-09-30T22:26:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 -- start start 2023-09-30T22:26:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2bc09d630 0x7fa2bc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc0a2a70 con 0x7fa2bc0a0a30 2023-09-30T22:26:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc0a01d0 con 0x7fa2bc09d630 2023-09-30T22:26:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c2b61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc09e2b0 con 0x7fa2bc138d80 2023-09-30T22:26:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c10fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2bbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2bc09d630 0x7fa2bc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.533+0000 7fa2c10fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37636/0 (socket says 172.21.15.32:37636) 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2c10fe700 1 -- 172.21.15.32:0/3227684517 learned_addr learned my addr 172.21.15.32:0/3227684517 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2c10fe700 1 -- 172.21.15.32:0/3227684517 >> v2:172.21.15.32:3300/0 conn(0x7fa2bc09d630 msgr2=0x7fa2bc0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2c10fe700 1 --2- 172.21.15.32:0/3227684517 >> v2:172.21.15.32:3300/0 conn(0x7fa2bc09d630 0x7fa2bc0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2c10fe700 1 -- 172.21.15.32:0/3227684517 >> v1:172.21.15.32:6789/0 conn(0x7fa2bc0a0a30 legacy=0x7fa2bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2bb7fe700 1 -- 172.21.15.32:0/3227684517 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203706722 0 0) 0x7fa2bc0a2a70 con 0x7fa2bc0a0a30 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2c10fe700 1 -- 172.21.15.32:0/3227684517 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc09dc50 con 0x7fa2bc138d80 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2bbfff700 1 --2- 172.21.15.32:0/3227684517 >> v2:172.21.15.32:3300/0 conn(0x7fa2bc09d630 0x7fa2bc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2c10fe700 1 --2- 172.21.15.32:0/3227684517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc13b160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009e50 tx=0x7fa2b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e92d009d7210ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2bb7fe700 1 -- 172.21.15.32:0/3227684517 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400e070 con 0x7fa2bc138d80 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2bb7fe700 1 -- 172.21.15.32:0/3227684517 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b400b910 con 0x7fa2bc138d80 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.534+0000 7fa2bb7fe700 1 -- 172.21.15.32:0/3227684517 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400bcc0 con 0x7fa2bc138d80 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 -- 172.21.15.32:0/3227684517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 msgr2=0x7fa2bc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/3227684517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc13b160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009e50 tx=0x7fa2b400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 -- 172.21.15.32:0/3227684517 shutdown_connections 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/3227684517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc13b160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/3227684517 >> v2:172.21.15.32:3300/0 conn(0x7fa2bc09d630 0x7fa2bc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 -- 172.21.15.32:0/3227684517 >> 172.21.15.32:0/3227684517 conn(0x7fa2bc0a3050 msgr2=0x7fa2bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 -- 172.21.15.32:0/3227684517 shutdown_connections 2023-09-30T22:26:12.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.535+0000 7fa2c2b61700 1 -- 172.21.15.32:0/3227684517 wait complete. 2023-09-30T22:26:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.536+0000 7fa2c2b61700 1 Processor -- start 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.536+0000 7fa2c2b61700 1 -- start start 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c2b61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 0x7fa2bc12c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c2b61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2bc0a0a30 0x7fa2bc12c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c2b61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc12cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c2b61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc13ac50 con 0x7fa2bc09d630 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c2b61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc12e990 con 0x7fa2bc138d80 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c2b61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc0a0410 con 0x7fa2bc0a0a30 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c08fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 0x7fa2bc12c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc12cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc12cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37642/0 (socket says 172.21.15.32:37642) 2023-09-30T22:26:12.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 -- 172.21.15.32:0/174883177 learned_addr learned my addr 172.21.15.32:0/174883177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c08fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 0x7fa2bc12c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38332/0 (socket says 172.21.15.32:38332) 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2bbfff700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2bc0a0a30 0x7fa2bc12c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 -- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2bc0a0a30 msgr2=0x7fa2bc12c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2bc0a0a30 0x7fa2bc12c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 -- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 msgr2=0x7fa2bc12c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 0x7fa2bc12c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 -- 172.21.15.32:0/174883177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00094a0 con 0x7fa2bc138d80 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c08fd700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 0x7fa2bc12c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.537+0000 7fa2c10fe700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc12cc20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4012000 tx=0x7fa2b4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:12.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.538+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4003ea0 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.538+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b40048e0 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.538+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b4009b30 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.538+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b401c650 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.538+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc12e330 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.538+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2b401c7b0 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.539+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa2b4040e40 con 0x7fa2bc138d80 2023-09-30T22:26:12.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.539+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc12ebb0 con 0x7fa2bc138d80 2023-09-30T22:26:12.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.544+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa2b40123f0 con 0x7fa2bc138d80 2023-09-30T22:26:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.580+0000 7fdbf617a700 1 Processor -- start 2023-09-30T22:26:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.580+0000 7fdbf617a700 1 -- start start 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.580+0000 7fdbf617a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009e930 0x7fdbf009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.580+0000 7fdbf617a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbf009f240 0x7fdbf01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbf617a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf0070940 con 0x7fdbf009f650 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbf617a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf006fd60 con 0x7fdbf009f240 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbf617a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf006fa80 con 0x7fdbf009e930 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbf009f240 0x7fdbf01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbf009f240 0x7fdbf01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38344/0 (socket says 172.21.15.32:38344) 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 -- 172.21.15.32:0/513751098 learned_addr learned my addr 172.21.15.32:0/513751098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbef7fe700 1 --2- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009e930 0x7fdbf009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 -- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009e930 msgr2=0x7fdbf009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 --2- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009e930 0x7fdbf009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 -- 172.21.15.32:0/513751098 >> v1:172.21.15.32:6789/0 conn(0x7fdbf009f650 legacy=0x7fdbf0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 -- 172.21.15.32:0/513751098 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf009fb60 con 0x7fdbf009f240 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbeeffd700 1 --2- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f240 0x7fdbf01317f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80098f0 tx=0x7fdbd8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e86e43439ddc38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbee7fc700 1 -- 172.21.15.32:0/513751098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8011830 con 0x7fdbf009f240 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.581+0000 7fdbee7fc700 1 -- 172.21.15.32:0/513751098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd8011990 con 0x7fdbf009f240 2023-09-30T22:26:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbee7fc700 1 -- 172.21.15.32:0/513751098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8011d20 con 0x7fdbf009f240 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 -- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f240 msgr2=0x7fdbf01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 --2- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f240 0x7fdbf01317f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80098f0 tx=0x7fdbd8009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 -- 172.21.15.32:0/513751098 shutdown_connections 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 --2- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009e930 0x7fdbf009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 --2- 172.21.15.32:0/513751098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f240 0x7fdbf01317f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 -- 172.21.15.32:0/513751098 >> 172.21.15.32:0/513751098 conn(0x7fdbf00a4210 msgr2=0x7fdbf00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 -- 172.21.15.32:0/513751098 shutdown_connections 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.582+0000 7fdbf617a700 1 -- 172.21.15.32:0/513751098 wait complete. 2023-09-30T22:26:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.584+0000 7fdbf617a700 1 Processor -- start 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.584+0000 7fdbf617a700 1 -- start start 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbf617a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 0x7fdbf01a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbf617a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009f240 0x7fdbf01a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbf617a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f650 0x7fdbf01a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbf617a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf01a6f60 con 0x7fdbf009f650 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbf617a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf0071d20 con 0x7fdbf009f240 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbf617a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00a36e0 con 0x7fdbf009e930 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 0x7fdbf01a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 0x7fdbf01a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55720/0 (socket says 172.21.15.32:55720) 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f650 0x7fdbf01a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbef7fe700 1 -- 172.21.15.32:0/2430724272 learned_addr learned my addr 172.21.15.32:0/2430724272 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeeffd700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009f240 0x7fdbf01a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeffff700 1 -- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 msgr2=0x7fdbf01a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeffff700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 0x7fdbf01a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeffff700 1 -- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009f240 msgr2=0x7fdbf01a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeffff700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009f240 0x7fdbf01a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbeffff700 1 -- 172.21.15.32:0/2430724272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8009550 con 0x7fdbf009f650 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.585+0000 7fdbef7fe700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 0x7fdbf01a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.586+0000 7fdbeffff700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f650 0x7fdbf01a4730 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000f60 tx=0x7fdbe800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.586+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80052d0 con 0x7fdbf009f650 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.586+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8005430 con 0x7fdbf009f650 2023-09-30T22:26:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.586+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf00a0190 con 0x7fdbf009f650 2023-09-30T22:26:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.586+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80038c0 con 0x7fdbf009f650 2023-09-30T22:26:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.586+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf01a6b10 con 0x7fdbf009f650 2023-09-30T22:26:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.587+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdbe8003a20 con 0x7fdbf009f650 2023-09-30T22:26:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.587+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdbe803e950 con 0x7fdbf009f650 2023-09-30T22:26:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.587+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf01a0b30 con 0x7fdbf009f650 2023-09-30T22:26:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.592+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdbe8017080 con 0x7fdbf009f650 2023-09-30T22:26:12.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.776+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa2bc0747f0 con 0x7fa2bc138d80 2023-09-30T22:26:12.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.777+0000 7fa2b97fa700 1 -- 172.21.15.32:0/174883177 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2b4033090 con 0x7fa2bc138d80 2023-09-30T22:26:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 msgr2=0x7fa2bc12cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc12cc20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4012000 tx=0x7fa2b4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 shutdown_connections 2023-09-30T22:26:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2bc138d80 0x7fa2bc12cc20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2bc09d630 0x7fa2bc12c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 --2- 172.21.15.32:0/174883177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2bc0a0a30 0x7fa2bc12c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 >> 172.21.15.32:0/174883177 conn(0x7fa2bc0a3050 msgr2=0x7fa2bc0a4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 shutdown_connections 2023-09-30T22:26:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.779+0000 7fa2c2b61700 1 -- 172.21.15.32:0/174883177 wait complete. 2023-09-30T22:26:12.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:12.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.808+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdbf0074ea0 con 0x7fdbf009f650 2023-09-30T22:26:12.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.808+0000 7fdbe7fff700 1 -- 172.21.15.32:0/2430724272 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdbe8019d20 con 0x7fdbf009f650 2023-09-30T22:26:12.808 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.810+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f650 msgr2=0x7fdbf01a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:12.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.810+0000 7fdbf617a700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f650 0x7fdbf01a4730 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000f60 tx=0x7fdbe800c6e0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.810+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 shutdown_connections 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.810+0000 7fdbf617a700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf009f650 0x7fdbf01a4730 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.810+0000 7fdbf617a700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdbf009e930 0x7fdbf01a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.810+0000 7fdbf617a700 1 --2- 172.21.15.32:0/2430724272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbf009f240 0x7fdbf01a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.811+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 >> 172.21.15.32:0/2430724272 conn(0x7fdbf00a4210 msgr2=0x7fdbf01b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.811+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 shutdown_connections 2023-09-30T22:26:12.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:12.811+0000 7fdbf617a700 1 -- 172.21.15.32:0/2430724272 wait complete. 2023-09-30T22:26:12.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:12.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:12.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.005+0000 7fce0e8b3700 1 Processor -- start 2023-09-30T22:26:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.005+0000 7fce0e8b3700 1 -- start start 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0e8b3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0e8b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0e8b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 0x7fce08133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0e8b3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce080a2a70 con 0x7fce080a0a30 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0e8b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce080a01d0 con 0x7fce08133930 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0e8b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce0809e2b0 con 0x7fce0809d630 2023-09-30T22:26:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce077fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0ce50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 0x7fce08133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce077fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55726/0 (socket says 172.21.15.32:55726) 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce0ce50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 0x7fce08133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37666/0 (socket says 172.21.15.32:37666) 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce077fe700 1 -- 172.21.15.32:0/3907182914 learned_addr learned my addr 172.21.15.32:0/3907182914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.006+0000 7fce07fff700 1 --2- 172.21.15.32:0/3907182914 >> v2:172.21.15.32:3300/0 conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce077fe700 1 -- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 msgr2=0x7fce08133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce077fe700 1 --2- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 0x7fce08133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce077fe700 1 -- 172.21.15.32:0/3907182914 >> v2:172.21.15.32:3300/0 conn(0x7fce080a0a30 msgr2=0x7fce080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce077fe700 1 --2- 172.21.15.32:0/3907182914 >> v2:172.21.15.32:3300/0 conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce077fe700 1 -- 172.21.15.32:0/3907182914 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0809dc50 con 0x7fce0809d630 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce0ce50700 1 --2- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 0x7fce08133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce07fff700 1 --2- 172.21.15.32:0/3907182914 >> v2:172.21.15.32:3300/0 conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce077fe700 1 --2- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 0x7fce080a1340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0098f0 tx=0x7fcdfc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c90d568bf386c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce06ffd700 1 -- 172.21.15.32:0/3907182914 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011830 con 0x7fce0809d630 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce06ffd700 1 -- 172.21.15.32:0/3907182914 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc011990 con 0x7fce0809d630 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.007+0000 7fce06ffd700 1 -- 172.21.15.32:0/3907182914 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011d20 con 0x7fce0809d630 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 -- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 msgr2=0x7fce080a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 0x7fce080a1340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0098f0 tx=0x7fcdfc009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 -- 172.21.15.32:0/3907182914 shutdown_connections 2023-09-30T22:26:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce08133930 0x7fce08133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/3907182914 >> v2:172.21.15.32:3300/0 conn(0x7fce080a0a30 0x7fce080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/3907182914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce0809d630 0x7fce080a1340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 -- 172.21.15.32:0/3907182914 >> 172.21.15.32:0/3907182914 conn(0x7fce080a3050 msgr2=0x7fce080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 -- 172.21.15.32:0/3907182914 shutdown_connections 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.008+0000 7fce0e8b3700 1 -- 172.21.15.32:0/3907182914 wait complete. 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.009+0000 7fce0e8b3700 1 Processor -- start 2023-09-30T22:26:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.009+0000 7fce0e8b3700 1 -- start start 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0e8b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809d630 0x7fce081c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0e8b3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce080a0a30 0x7fce081c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0e8b3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 0x7fce081c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0e8b3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce081207f0 con 0x7fce080a0a30 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0e8b3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce08120600 con 0x7fce0809d630 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0e8b3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce080a4150 con 0x7fce08133930 2023-09-30T22:26:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 0x7fce081c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 0x7fce081c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55742/0 (socket says 172.21.15.32:55742) 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 -- 172.21.15.32:0/2783654756 learned_addr learned my addr 172.21.15.32:0/2783654756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce07fff700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809d630 0x7fce081c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce077fe700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce080a0a30 0x7fce081c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 -- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809d630 msgr2=0x7fce081c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809d630 0x7fce081c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 -- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce080a0a30 msgr2=0x7fce081c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce080a0a30 0x7fce081c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce0ce50700 1 -- 172.21.15.32:0/2783654756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80094a0 con 0x7fce08133930 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.010+0000 7fce077fe700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce080a0a30 0x7fce081c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.011+0000 7fce0ce50700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 0x7fce081c8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce0000efc0 tx=0x7fce0000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.011+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000ca90 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.011+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0000f040 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.011+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0000ce10 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.011+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce00009a70 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.011+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc009550 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.012+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce00014480 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.012+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fce0003fc40 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.012+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce08121ed0 con 0x7fce08133930 2023-09-30T22:26:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.017+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce08121ed0 con 0x7fce08133930 2023-09-30T22:26:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.047+0000 7ff31f8fc700 1 Processor -- start 2023-09-30T22:26:13.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.047+0000 7ff31f8fc700 1 -- start start 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31f8fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff318131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31f8fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff31809d9f0 0x7ff31806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31f8fc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31809e130 con 0x7ff31806acb0 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31f8fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3181312b0 con 0x7ff31809d9f0 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31f8fc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff31809f6e0 con 0x7ff3181316c0 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31d698700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff318131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31ce97700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff31809d9f0 0x7ff31806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31d698700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff318131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55744/0 (socket says 172.21.15.32:55744) 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31d698700 1 -- 172.21.15.32:0/1221537919 learned_addr learned my addr 172.21.15.32:0/1221537919 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.048+0000 7ff31de99700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff31806acb0 0x7ff318138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57374/0 (socket says 172.21.15.32:57374) 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff31ce97700 1 -- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 msgr2=0x7ff318131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff31ce97700 1 --2- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff318131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff31ce97700 1 -- 172.21.15.32:0/1221537919 >> v1:172.21.15.32:6789/0 conn(0x7ff31806acb0 legacy=0x7ff318138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff30ffff700 1 -- 172.21.15.32:0/1221537919 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438122724 0 0) 0x7ff310001490 con 0x7ff31806acb0 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff31ce97700 1 -- 172.21.15.32:0/1221537919 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31809e760 con 0x7ff31809d9f0 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff31d698700 1 --2- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff318131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff31ce97700 1 --2- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff31806a770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff308009840 tx=0x7ff308009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ac087d341471d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff30ffff700 1 -- 172.21.15.32:0/1221537919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308011830 con 0x7ff31809d9f0 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff30ffff700 1 -- 172.21.15.32:0/1221537919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308011990 con 0x7ff31809d9f0 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.049+0000 7ff30ffff700 1 -- 172.21.15.32:0/1221537919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308011d20 con 0x7ff31809d9f0 2023-09-30T22:26:13.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 -- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 msgr2=0x7ff31806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff31806a770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff308009840 tx=0x7ff308009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 -- 172.21.15.32:0/1221537919 shutdown_connections 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff318131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/1221537919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff31806a770 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 -- 172.21.15.32:0/1221537919 >> 172.21.15.32:0/1221537919 conn(0x7ff318066f50 msgr2=0x7ff31809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 -- 172.21.15.32:0/1221537919 shutdown_connections 2023-09-30T22:26:13.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.050+0000 7ff31f8fc700 1 -- 172.21.15.32:0/1221537919 wait complete. 2023-09-30T22:26:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 Processor -- start 2023-09-30T22:26:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 -- start start 2023-09-30T22:26:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff31806acb0 0x7ff318070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff318070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff3180713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31811a800 con 0x7ff31809d9f0 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3180a36d0 con 0x7ff31806acb0 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31f8fc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff318119680 con 0x7ff3181316c0 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31d698700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff31806acb0 0x7ff318070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31de99700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff3180713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31d698700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff31806acb0 0x7ff318070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37676/0 (socket says 172.21.15.32:37676) 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31d698700 1 -- 172.21.15.32:0/651195878 learned_addr learned my addr 172.21.15.32:0/651195878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.051+0000 7ff31ce97700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff318070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31de99700 1 -- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff31806acb0 msgr2=0x7ff318070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31de99700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff31806acb0 0x7ff318070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31de99700 1 -- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 msgr2=0x7ff318070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31de99700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff318070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31de99700 1 -- 172.21.15.32:0/651195878 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314009550 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31de99700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff3180713e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff31000cc10 tx=0x7ff31000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31000aaa0 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31000ac20 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3080094a0 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100070d0 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.052+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff318119be0 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.053+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff310007230 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.053+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff31003f280 con 0x7ff3181316c0 2023-09-30T22:26:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.053+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31809e130 con 0x7ff3181316c0 2023-09-30T22:26:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.058+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff31809e130 con 0x7ff3181316c0 2023-09-30T22:26:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.230+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fce08121ed0 con 0x7fce08133930 2023-09-30T22:26:13.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.230+0000 7fce04ff9700 1 -- 172.21.15.32:0/2783654756 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce08121ed0 con 0x7fce08133930 2023-09-30T22:26:13.231 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:13.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 msgr2=0x7fce081c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 0x7fce081c8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce0000efc0 tx=0x7fce0000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 shutdown_connections 2023-09-30T22:26:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce08133930 0x7fce081c8aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce0809d630 0x7fce081c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 --2- 172.21.15.32:0/2783654756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce080a0a30 0x7fce081c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.232+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 >> 172.21.15.32:0/2783654756 conn(0x7fce080a3050 msgr2=0x7fce081bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.233+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 shutdown_connections 2023-09-30T22:26:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.233+0000 7fce0e8b3700 1 -- 172.21.15.32:0/2783654756 wait complete. 2023-09-30T22:26:13.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.269+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff31809e130 con 0x7ff3181316c0 2023-09-30T22:26:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.269+0000 7ff30dffb700 1 -- 172.21.15.32:0/651195878 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff31809e130 con 0x7ff3181316c0 2023-09-30T22:26:13.269 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 msgr2=0x7ff3180713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff3180713e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff31000cc10 tx=0x7ff31000cf20 comp rx=0 tx=0).stop 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 shutdown_connections 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3181316c0 0x7ff3180713e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff31806acb0 0x7ff318070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 --2- 172.21.15.32:0/651195878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31809d9f0 0x7ff318070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 >> 172.21.15.32:0/651195878 conn(0x7ff318066f50 msgr2=0x7ff3180a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:13.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 shutdown_connections 2023-09-30T22:26:13.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.272+0000 7ff31f8fc700 1 -- 172.21.15.32:0/651195878 wait complete. 2023-09-30T22:26:13.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:13.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:13.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:13.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.941+0000 7f5ea6a69700 1 Processor -- start 2023-09-30T22:26:13.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.941+0000 7f5ea6a69700 1 -- start start 2023-09-30T22:26:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea6a69700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea6a69700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea0138d80 0x7f5ea013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea6a69700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea00a2a70 con 0x7f5ea009d630 2023-09-30T22:26:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea6a69700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00a01d0 con 0x7f5ea0138d80 2023-09-30T22:26:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea6a69700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea009e2b0 con 0x7f5ea00a0a30 2023-09-30T22:26:13.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea5006700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ea0138d80 0x7f5ea013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea4805700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5e9ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ea009d630 0x7f5ea00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57384/0 (socket says 172.21.15.32:57384) 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5e9ffff700 1 -- 172.21.15.32:0/2199915778 learned_addr learned my addr 172.21.15.32:0/2199915778 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea5006700 1 -- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 msgr2=0x7f5ea00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea5006700 1 --2- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea5006700 1 -- 172.21.15.32:0/2199915778 >> v1:172.21.15.32:6789/0 conn(0x7f5ea009d630 legacy=0x7f5ea00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5ea5006700 1 -- 172.21.15.32:0/2199915778 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea009dc50 con 0x7f5ea0138d80 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.942+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2199915778 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916661587 0 0) 0x7f5ea00a2a70 con 0x7f5ea009d630 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.943+0000 7f5ea4805700 1 --2- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.943+0000 7f5ea5006700 1 --2- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea0138d80 0x7f5ea013b160 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009d90 tx=0x7f5e9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3afe73ed465df9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.943+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2199915778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800e070 con 0x7f5ea0138d80 2023-09-30T22:26:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.943+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2199915778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9800bb20 con 0x7f5ea0138d80 2023-09-30T22:26:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.943+0000 7f5e9f7fe700 1 -- 172.21.15.32:0/2199915778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98012430 con 0x7f5ea0138d80 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 -- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea0138d80 msgr2=0x7f5ea013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea0138d80 0x7f5ea013b160 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009d90 tx=0x7f5e9800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 -- 172.21.15.32:0/2199915778 shutdown_connections 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea0138d80 0x7f5ea013b160 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/2199915778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 -- 172.21.15.32:0/2199915778 >> 172.21.15.32:0/2199915778 conn(0x7f5ea00a3050 msgr2=0x7f5ea00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 -- 172.21.15.32:0/2199915778 shutdown_connections 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.944+0000 7f5ea6a69700 1 -- 172.21.15.32:0/2199915778 wait complete. 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.945+0000 7f5ea6a69700 1 Processor -- start 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.945+0000 7f5ea6a69700 1 -- start start 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea6a69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea009d630 0x7f5ea0133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea6a69700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea0133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea6a69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea0138d80 0x7f5ea0134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea6a69700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea01361b0 con 0x7f5ea009d630 2023-09-30T22:26:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea6a69700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0136ed0 con 0x7f5ea0138d80 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea6a69700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00a4b60 con 0x7f5ea00a0a30 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea0133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea5006700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea0138d80 0x7f5ea0134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea0133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55774/0 (socket says 172.21.15.32:55774) 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5e9ffff700 1 -- 172.21.15.32:0/4135986178 learned_addr learned my addr 172.21.15.32:0/4135986178 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.946+0000 7f5ea4805700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea009d630 0x7f5ea0133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea4805700 1 -- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 msgr2=0x7f5ea0133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea4805700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea0133bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea4805700 1 -- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea0138d80 msgr2=0x7f5ea0134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea4805700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea0138d80 0x7f5ea0134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea4805700 1 -- 172.21.15.32:0/4135986178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900094a0 con 0x7f5ea009d630 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5e9ffff700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea0133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea4805700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea009d630 0x7f5ea0133690 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000e9e0 tx=0x7f5e9000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000ee80 con 0x7f5ea009d630 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e98009a70 con 0x7f5ea009d630 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9000c850 con 0x7f5ea009d630 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000cc00 con 0x7f5ea009d630 2023-09-30T22:26:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.947+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea0136560 con 0x7f5ea009d630 2023-09-30T22:26:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.948+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea0137a00 con 0x7f5ea009d630 2023-09-30T22:26:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.948+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e9000cd60 con 0x7f5ea009d630 2023-09-30T22:26:13.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.949+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5e9003fae0 con 0x7f5ea009d630 2023-09-30T22:26:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.954+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e90011d80 con 0x7f5ea009d630 2023-09-30T22:26:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.981+0000 7fdeb278f700 1 Processor -- start 2023-09-30T22:26:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.981+0000 7fdeb278f700 1 -- start start 2023-09-30T22:26:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb278f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb278f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb278f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeac133930 0x7fdeac133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb278f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0a2a70 con 0x7fdeac133930 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb278f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0a01d0 con 0x7fdeac09d630 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb278f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac09e2b0 con 0x7fdeac0a0a30 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb0d2c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeac133930 0x7fdeac133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37684/0 (socket says 172.21.15.32:37684) 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3750417317 learned_addr learned my addr 172.21.15.32:0/3750417317 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 msgr2=0x7fdeac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:13.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3750417317 >> v2:172.21.15.32:3300/0 conn(0x7fdeac133930 msgr2=0x7fdeac133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3750417317 >> v2:172.21.15.32:3300/0 conn(0x7fdeac133930 0x7fdeac133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeab7fe700 1 -- 172.21.15.32:0/3750417317 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac09dc50 con 0x7fdeac09d630 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeb0d2c700 1 --2- 172.21.15.32:0/3750417317 >> v2:172.21.15.32:3300/0 conn(0x7fdeac133930 0x7fdeac133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.982+0000 7fdeabfff700 1 --2- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac0a1340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdea00098f0 tx=0x7fdea0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd1f222fd981e52e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeaaffd700 1 -- 172.21.15.32:0/3750417317 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0011830 con 0x7fdeac09d630 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeaaffd700 1 -- 172.21.15.32:0/3750417317 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea0011990 con 0x7fdeac09d630 2023-09-30T22:26:13.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeaaffd700 1 -- 172.21.15.32:0/3750417317 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0011d20 con 0x7fdeac09d630 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeb278f700 1 -- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 msgr2=0x7fdeac0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeb278f700 1 --2- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac0a1340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdea00098f0 tx=0x7fdea0009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeb278f700 1 -- 172.21.15.32:0/3750417317 shutdown_connections 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeb278f700 1 --2- 172.21.15.32:0/3750417317 >> v2:172.21.15.32:3300/0 conn(0x7fdeac133930 0x7fdeac133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.983+0000 7fdeb278f700 1 --2- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.984+0000 7fdeb278f700 1 --2- 172.21.15.32:0/3750417317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac0a1340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.984+0000 7fdeb278f700 1 -- 172.21.15.32:0/3750417317 >> 172.21.15.32:0/3750417317 conn(0x7fdeac0a3050 msgr2=0x7fdeac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.984+0000 7fdeb278f700 1 -- 172.21.15.32:0/3750417317 shutdown_connections 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.984+0000 7fdeb278f700 1 -- 172.21.15.32:0/3750417317 wait complete. 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 Processor -- start 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 -- start start 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac1c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac133930 0x7fdeac1c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac120690 con 0x7fdeac133930 2023-09-30T22:26:13.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac121ef0 con 0x7fdeac09d630 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb278f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0a0410 con 0x7fdeac0a0a30 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac1c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeb0d2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac133930 0x7fdeac1c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac1c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55792/0 (socket says 172.21.15.32:55792) 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac1c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37694/0 (socket says 172.21.15.32:37694) 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.985+0000 7fdeab7fe700 1 -- 172.21.15.32:0/1859631773 learned_addr learned my addr 172.21.15.32:0/1859631773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeab7fe700 1 -- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 msgr2=0x7fdeac1c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac1c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeab7fe700 1 -- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac133930 msgr2=0x7fdeac1c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac133930 0x7fdeac1c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeab7fe700 1 -- 172.21.15.32:0/1859631773 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0094a0 con 0x7fdeac0a0a30 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeab7fe700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac1c85d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdea0000c00 tx=0x7fdea0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0004450 con 0x7fdeac0a0a30 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea00045b0 con 0x7fdeac0a0a30 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea4009a70 con 0x7fdeac0a0a30 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00038e0 con 0x7fdeac0a0a30 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.986+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea0009550 con 0x7fdeac0a0a30 2023-09-30T22:26:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.987+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdea0021070 con 0x7fdeac0a0a30 2023-09-30T22:26:13.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.987+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac0747f0 con 0x7fdeac0a0a30 2023-09-30T22:26:13.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.987+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdea00408d0 con 0x7fdeac0a0a30 2023-09-30T22:26:13.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:13.997+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdea00199a0 con 0x7fdeac0a0a30 2023-09-30T22:26:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.168+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5ea00747f0 con 0x7f5ea009d630 2023-09-30T22:26:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.169+0000 7f5e9d7fa700 1 -- 172.21.15.32:0/4135986178 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e90011350 con 0x7f5ea009d630 2023-09-30T22:26:14.169 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea009d630 msgr2=0x7f5ea0133690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea009d630 0x7f5ea0133690 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000e9e0 tx=0x7f5e9000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 shutdown_connections 2023-09-30T22:26:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ea0138d80 0x7f5ea0134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ea009d630 0x7f5ea0133690 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 --2- 172.21.15.32:0/4135986178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ea00a0a30 0x7f5ea0133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 >> 172.21.15.32:0/4135986178 conn(0x7f5ea00a3050 msgr2=0x7f5ea013af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:14.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 shutdown_connections 2023-09-30T22:26:14.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.171+0000 7f5ea6a69700 1 -- 172.21.15.32:0/4135986178 wait complete. 2023-09-30T22:26:14.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.207+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdeac072980 con 0x7fdeac0a0a30 2023-09-30T22:26:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.207+0000 7fdea8ff9700 1 -- 172.21.15.32:0/1859631773 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdea0029030 con 0x7fdeac0a0a30 2023-09-30T22:26:14.208 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.209+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 msgr2=0x7fdeac1c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.209+0000 7fdeb278f700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac1c85d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdea0000c00 tx=0x7fdea0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 shutdown_connections 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeac133930 0x7fdeac1c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeac09d630 0x7fdeac1c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 --2- 172.21.15.32:0/1859631773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdeac0a0a30 0x7fdeac1c85d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 >> 172.21.15.32:0/1859631773 conn(0x7fdeac0a3050 msgr2=0x7fdeac09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 shutdown_connections 2023-09-30T22:26:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.210+0000 7fdeb278f700 1 -- 172.21.15.32:0/1859631773 wait complete. 2023-09-30T22:26:14.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:14.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:14.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:14.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.406+0000 7f95b7bf2700 1 Processor -- start 2023-09-30T22:26:14.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.406+0000 7f95b7bf2700 1 -- start start 2023-09-30T22:26:14.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b7bf2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b7bf2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b7bf2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00a2a70 con 0x7f95b00a0a30 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b7bf2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00a01d0 con 0x7f95b0138d80 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b7bf2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95b009e2b0 con 0x7f95b009d630 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b618f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b518d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b598e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f95b00a0a30 0x7f95b00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57394/0 (socket says 172.21.15.32:57394) 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b598e700 1 -- 172.21.15.32:0/41675759 learned_addr learned my addr 172.21.15.32:0/41675759 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b618f700 1 -- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 msgr2=0x7f95b00a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:14.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b618f700 1 --2- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b00a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b618f700 1 -- 172.21.15.32:0/41675759 >> v1:172.21.15.32:6789/0 conn(0x7f95b00a0a30 legacy=0x7f95b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b618f700 1 -- 172.21.15.32:0/41675759 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b009dc50 con 0x7f95b0138d80 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b498c700 1 -- 172.21.15.32:0/41675759 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271354145 0 0) 0x7f95b00a2a70 con 0x7f95b00a0a30 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b518d700 1 --2- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b618f700 1 --2- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b013b160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009dc0 tx=0x7f95a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31dce9fa75cb13c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.407+0000 7f95b498c700 1 -- 172.21.15.32:0/41675759 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800e070 con 0x7f95b0138d80 2023-09-30T22:26:14.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b498c700 1 -- 172.21.15.32:0/41675759 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800bb20 con 0x7f95b0138d80 2023-09-30T22:26:14.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b498c700 1 -- 172.21.15.32:0/41675759 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8012430 con 0x7f95b0138d80 2023-09-30T22:26:14.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 -- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 msgr2=0x7f95b013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b013b160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009dc0 tx=0x7f95a800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:14.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 -- 172.21.15.32:0/41675759 shutdown_connections 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b013b160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/41675759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 -- 172.21.15.32:0/41675759 >> 172.21.15.32:0/41675759 conn(0x7f95b00a3050 msgr2=0x7f95b00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 -- 172.21.15.32:0/41675759 shutdown_connections 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.408+0000 7f95b7bf2700 1 -- 172.21.15.32:0/41675759 wait complete. 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.410+0000 7f95b7bf2700 1 Processor -- start 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.410+0000 7f95b7bf2700 1 -- start start 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.410+0000 7f95b7bf2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b01336a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.410+0000 7f95b7bf2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95b00a0a30 0x7f95b0133be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b7bf2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b0134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b7bf2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0134b00 con 0x7f95b00a0a30 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b7bf2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b01364e0 con 0x7f95b0138d80 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b7bf2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0136e50 con 0x7f95b009d630 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b01336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b618f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b0134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b01336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55814/0 (socket says 172.21.15.32:55814) 2023-09-30T22:26:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 -- 172.21.15.32:0/1612468889 learned_addr learned my addr 172.21.15.32:0/1612468889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b518d700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95b00a0a30 0x7f95b0133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 -- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 msgr2=0x7f95b0134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b0134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 -- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95b00a0a30 msgr2=0x7f95b0133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95b00a0a30 0x7f95b0133be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 -- 172.21.15.32:0/1612468889 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00094a0 con 0x7f95b009d630 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b598e700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b01336a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c520 tx=0x7f95ac00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004c80 con 0x7f95b009d630 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac004e00 con 0x7f95b009d630 2023-09-30T22:26:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac007310 con 0x7f95b009d630 2023-09-30T22:26:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.411+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a8009a70 con 0x7f95b009d630 2023-09-30T22:26:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.412+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00a0410 con 0x7f95b009d630 2023-09-30T22:26:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.412+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95ac007470 con 0x7f95b009d630 2023-09-30T22:26:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.413+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f95ac03f340 con 0x7f95b009d630 2023-09-30T22:26:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.413+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b00747f0 con 0x7f95b009d630 2023-09-30T22:26:14.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.418+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95ac0123b0 con 0x7f95b009d630 2023-09-30T22:26:14.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 Processor -- start 2023-09-30T22:26:14.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 -- start start 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd330134170 0x7fd33013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33009edd0 con 0x7fd3300a1340 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33009e1f0 con 0x7fd330134170 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd33604e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd33009dc50 con 0x7fd3300a0a30 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.448+0000 7fd32ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd330134170 0x7fd33013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3300a1340 0x7fd330133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57404/0 (socket says 172.21.15.32:57404) 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32effd700 1 -- 172.21.15.32:0/2699143144 learned_addr learned my addr 172.21.15.32:0/2699143144 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32ffff700 1 -- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 msgr2=0x7fd3300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32ffff700 1 --2- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3300a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32ffff700 1 -- 172.21.15.32:0/2699143144 >> v1:172.21.15.32:6789/0 conn(0x7fd3300a1340 legacy=0x7fd330133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32ffff700 1 -- 172.21.15.32:0/2699143144 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300a1fe0 con 0x7fd330134170 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32e7fc700 1 -- 172.21.15.32:0/2699143144 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073189241 0 0) 0x7fd33009edd0 con 0x7fd3300a1340 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32ffff700 1 --2- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330134170 0x7fd33013bd30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd328009840 tx=0x7fd328009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5446387fcb37320b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32e7fc700 1 -- 172.21.15.32:0/2699143144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328011830 con 0x7fd330134170 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32e7fc700 1 -- 172.21.15.32:0/2699143144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328011990 con 0x7fd330134170 2023-09-30T22:26:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.449+0000 7fd32e7fc700 1 -- 172.21.15.32:0/2699143144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328011d20 con 0x7fd330134170 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 -- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330134170 msgr2=0x7fd33013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 --2- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330134170 0x7fd33013bd30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd328009840 tx=0x7fd328009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 -- 172.21.15.32:0/2699143144 shutdown_connections 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 --2- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd330134170 0x7fd33013bd30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 --2- 172.21.15.32:0/2699143144 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 -- 172.21.15.32:0/2699143144 >> 172.21.15.32:0/2699143144 conn(0x7fd3300a6550 msgr2=0x7fd3300a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 -- 172.21.15.32:0/2699143144 shutdown_connections 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.450+0000 7fd33604e700 1 -- 172.21.15.32:0/2699143144 wait complete. 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.451+0000 7fd33604e700 1 Processor -- start 2023-09-30T22:26:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.451+0000 7fd33604e700 1 -- start start 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd33604e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3301c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd33604e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3300a1340 0x7fd3301c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd33604e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 0x7fd330122880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd33604e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300a4d60 con 0x7fd3300a1340 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd33604e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300a0220 con 0x7fd330134170 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd33604e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3301260c0 con 0x7fd3300a0a30 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 0x7fd330122880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3300a1340 0x7fd3301c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 0x7fd330122880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37708/0 (socket says 172.21.15.32:37708) 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32ffff700 1 -- 172.21.15.32:0/4139902644 learned_addr learned my addr 172.21.15.32:0/4139902644 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:14.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32f7fe700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3301c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32effd700 1 -- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 msgr2=0x7fd3301c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32effd700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3301c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32effd700 1 -- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 msgr2=0x7fd330122880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32effd700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 0x7fd330122880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32effd700 1 -- 172.21.15.32:0/4139902644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3280094a0 con 0x7fd3300a1340 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.452+0000 7fd32ffff700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 0x7fd330122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.453+0000 7fd32effd700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3300a1340 0x7fd3301c8c70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd31800c470 tx=0x7fd31800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.453+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318004c80 con 0x7fd3300a1340 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.453+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318004de0 con 0x7fd3300a1340 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.453+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330126b30 con 0x7fd3300a1340 2023-09-30T22:26:14.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.453+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318007310 con 0x7fd3300a1340 2023-09-30T22:26:14.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.453+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3301c91b0 con 0x7fd3300a1340 2023-09-30T22:26:14.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.454+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd318007470 con 0x7fd3300a1340 2023-09-30T22:26:14.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.454+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd330074ea0 con 0x7fd3300a1340 2023-09-30T22:26:14.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.454+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd31803f340 con 0x7fd3300a1340 2023-09-30T22:26:14.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.459+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd318007720 con 0x7fd3300a1340 2023-09-30T22:26:14.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.626+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f95b0072980 con 0x7f95b009d630 2023-09-30T22:26:14.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.627+0000 7f95a67fc700 1 -- 172.21.15.32:0/1612468889 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95ac008040 con 0x7f95b009d630 2023-09-30T22:26:14.627 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 msgr2=0x7f95b01336a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b01336a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c520 tx=0x7f95ac00c830 comp rx=0 tx=0).stop 2023-09-30T22:26:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 shutdown_connections 2023-09-30T22:26:14.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95b0138d80 0x7f95b0134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95b009d630 0x7f95b01336a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 --2- 172.21.15.32:0/1612468889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95b00a0a30 0x7f95b0133be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 >> 172.21.15.32:0/1612468889 conn(0x7f95b00a3050 msgr2=0x7f95b00a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 shutdown_connections 2023-09-30T22:26:14.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.629+0000 7f95b7bf2700 1 -- 172.21.15.32:0/1612468889 wait complete. 2023-09-30T22:26:14.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.665+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd3300a28e0 con 0x7fd3300a1340 2023-09-30T22:26:14.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.666+0000 7fd327fff700 1 -- 172.21.15.32:0/4139902644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd318011e80 con 0x7fd3300a1340 2023-09-30T22:26:14.666 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3300a1340 msgr2=0x7fd3301c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3300a1340 0x7fd3301c8c70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd31800c470 tx=0x7fd31800c780 comp rx=0 tx=0).stop 2023-09-30T22:26:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 shutdown_connections 2023-09-30T22:26:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd330134170 0x7fd330122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3300a0a30 0x7fd3301c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 --2- 172.21.15.32:0/4139902644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3300a1340 0x7fd3301c8c70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 >> 172.21.15.32:0/4139902644 conn(0x7fd3300a6550 msgr2=0x7fd33009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 shutdown_connections 2023-09-30T22:26:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:14.668+0000 7fd33604e700 1 -- 172.21.15.32:0/4139902644 wait complete. 2023-09-30T22:26:14.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:15.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:15.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.334+0000 7f918c379700 1 Processor -- start 2023-09-30T22:26:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.334+0000 7f918c379700 1 -- start start 2023-09-30T22:26:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918c379700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 0x7f91840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918c379700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f91840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918c379700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9184133930 0x7f9184133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918c379700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91840a2a70 con 0x7f9184133930 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918c379700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91840a01d0 con 0x7f91840a0a30 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918c379700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f918409e2b0 con 0x7f918409d630 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f9189914700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f91840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 0x7f91840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a916700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9184133930 0x7f9184133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 0x7f91840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37718/0 (socket says 172.21.15.32:37718) 2023-09-30T22:26:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f9189914700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f91840a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55852/0 (socket says 172.21.15.32:55852) 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a916700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9184133930 0x7f9184133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38464/0 (socket says 172.21.15.32:38464) 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 -- 172.21.15.32:0/2577187119 learned_addr learned my addr 172.21.15.32:0/2577187119 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 -- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 msgr2=0x7f91840a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 --2- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f91840a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 -- 172.21.15.32:0/2577187119 >> v2:172.21.15.32:3300/0 conn(0x7f9184133930 msgr2=0x7f9184133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 --2- 172.21.15.32:0/2577187119 >> v2:172.21.15.32:3300/0 conn(0x7f9184133930 0x7f9184133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.335+0000 7f918a115700 1 -- 172.21.15.32:0/2577187119 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918409dc50 con 0x7f91840a0a30 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f918a115700 1 --2- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 0x7f91840a0e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098f0 tx=0x7f917c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fedf370e65b62d43 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f9189113700 1 -- 172.21.15.32:0/2577187119 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011830 con 0x7f91840a0a30 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f9189113700 1 -- 172.21.15.32:0/2577187119 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c011990 con 0x7f91840a0a30 2023-09-30T22:26:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f9189113700 1 -- 172.21.15.32:0/2577187119 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011d20 con 0x7f91840a0a30 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f918c379700 1 -- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 msgr2=0x7f91840a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f918c379700 1 --2- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 0x7f91840a0e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098f0 tx=0x7f917c009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f918c379700 1 -- 172.21.15.32:0/2577187119 shutdown_connections 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f918c379700 1 --2- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91840a0a30 0x7f91840a0e00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.336+0000 7f918c379700 1 --2- 172.21.15.32:0/2577187119 >> v2:172.21.15.32:3300/0 conn(0x7f9184133930 0x7f9184133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.337+0000 7f918c379700 1 --2- 172.21.15.32:0/2577187119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f91840a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.337+0000 7f918c379700 1 -- 172.21.15.32:0/2577187119 >> 172.21.15.32:0/2577187119 conn(0x7f91840a3050 msgr2=0x7f91840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.337+0000 7f918c379700 1 -- 172.21.15.32:0/2577187119 shutdown_connections 2023-09-30T22:26:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.337+0000 7f918c379700 1 -- 172.21.15.32:0/2577187119 wait complete. 2023-09-30T22:26:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 Processor -- start 2023-09-30T22:26:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 -- start start 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f9184121160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 0x7f91841216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9184133930 0x7f9184121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841255a0 con 0x7f91840a0a30 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9184122cd0 con 0x7f9184133930 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918c379700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9184124640 con 0x7f918409d630 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918a115700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f9184121160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918a916700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9184133930 0x7f9184121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f9189914700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 0x7f91841216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f9189914700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 0x7f91841216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38480/0 (socket says 172.21.15.32:38480) 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f918a115700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f9184121160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55856/0 (socket says 172.21.15.32:55856) 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.338+0000 7f9189914700 1 -- 172.21.15.32:0/574768764 learned_addr learned my addr 172.21.15.32:0/574768764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918a916700 1 -- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 msgr2=0x7f9184121160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918a916700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f9184121160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918a916700 1 -- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 msgr2=0x7f91841216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918a916700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 0x7f91841216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918a916700 1 -- 172.21.15.32:0/574768764 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009550 con 0x7f9184133930 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f9189914700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 0x7f91841216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918a916700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9184133930 0x7f9184121be0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9174009d90 tx=0x7f917400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174009c00 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400f040 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9184123b60 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174014480 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.339+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91840a0410 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.340+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91740145e0 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.340+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f91740401b0 con 0x7f9184133930 2023-09-30T22:26:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.340+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918411df60 con 0x7f9184133930 2023-09-30T22:26:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.345+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9174018030 con 0x7f9184133930 2023-09-30T22:26:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.371+0000 7fe8f624e700 1 Processor -- start 2023-09-30T22:26:15.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.371+0000 7fe8f624e700 1 -- start start 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8f624e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0136820 0x7fe8f0138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8f624e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8f0139140 0x7fe8f013b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8f624e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f006bc00 con 0x7fe8f0133a80 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8f624e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f006c570 con 0x7fe8f0139140 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8f624e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00a0040 con 0x7fe8f0136820 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8effff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8f0139140 0x7fe8f013b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8ef7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8f0133a80 0x7fe8f0133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57414/0 (socket says 172.21.15.32:57414) 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8ef7fe700 1 -- 172.21.15.32:0/19858914 learned_addr learned my addr 172.21.15.32:0/19858914 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8eeffd700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0136820 0x7fe8f0138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8effff700 1 -- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0136820 msgr2=0x7fe8f0138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8effff700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0136820 0x7fe8f0138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8effff700 1 -- 172.21.15.32:0/19858914 >> v1:172.21.15.32:6789/0 conn(0x7fe8f0133a80 legacy=0x7fe8f0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8effff700 1 -- 172.21.15.32:0/19858914 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0133860 con 0x7fe8f0139140 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8ee7fc700 1 -- 172.21.15.32:0/19858914 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163919562 0 0) 0x7fe8f006bc00 con 0x7fe8f0133a80 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.372+0000 7fe8eeffd700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0136820 0x7fe8f0138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8effff700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0139140 0x7fe8f013b520 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009d90 tx=0x7fe8e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59384d8b9a2dd4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8ee7fc700 1 -- 172.21.15.32:0/19858914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800e070 con 0x7fe8f0139140 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8ee7fc700 1 -- 172.21.15.32:0/19858914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800bb20 con 0x7fe8f0139140 2023-09-30T22:26:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8ee7fc700 1 -- 172.21.15.32:0/19858914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8012430 con 0x7fe8f0139140 2023-09-30T22:26:15.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8f624e700 1 -- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0139140 msgr2=0x7fe8f013b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8f624e700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0139140 0x7fe8f013b520 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009d90 tx=0x7fe8e800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:15.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.373+0000 7fe8f624e700 1 -- 172.21.15.32:0/19858914 shutdown_connections 2023-09-30T22:26:15.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.374+0000 7fe8f624e700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0139140 0x7fe8f013b520 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.374+0000 7fe8f624e700 1 --2- 172.21.15.32:0/19858914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0136820 0x7fe8f0138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.374+0000 7fe8f624e700 1 -- 172.21.15.32:0/19858914 >> 172.21.15.32:0/19858914 conn(0x7fe8f00a4210 msgr2=0x7fe8f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.374+0000 7fe8f624e700 1 -- 172.21.15.32:0/19858914 shutdown_connections 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.374+0000 7fe8f624e700 1 -- 172.21.15.32:0/19858914 wait complete. 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.375+0000 7fe8f624e700 1 Processor -- start 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.375+0000 7fe8f624e700 1 -- start start 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8f624e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8f0133a80 0x7fe8f012de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8f624e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 0x7fe8f012e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8f624e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 0x7fe8f012e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8f624e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f012f190 con 0x7fe8f0136820 2023-09-30T22:26:15.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8f624e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f009e3a0 con 0x7fe8f0139140 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8f624e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f009cf10 con 0x7fe8f0133a80 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 0x7fe8f012e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 0x7fe8f012e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8eeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 0x7fe8f012e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38500/0 (socket says 172.21.15.32:38500) 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8eeffd700 1 -- 172.21.15.32:0/1288925726 learned_addr learned my addr 172.21.15.32:0/1288925726 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 0x7fe8f012e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37736/0 (socket says 172.21.15.32:37736) 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8ef7fe700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8f0133a80 0x7fe8f012de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 -- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8f0133a80 msgr2=0x7fe8f012de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8f0133a80 0x7fe8f012de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 -- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 msgr2=0x7fe8f012e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 0x7fe8f012e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8effff700 1 -- 172.21.15.32:0/1288925726 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40094a0 con 0x7fe8f0139140 2023-09-30T22:26:15.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.376+0000 7fe8eeffd700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 0x7fe8f012e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.377+0000 7fe8effff700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 0x7fe8f012e890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8016000 tx=0x7fe8e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.377+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8012430 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.377+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8020070 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.377+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e8009a70 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.377+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004280 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.377+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f00726f0 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.378+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f0074ea0 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.378+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8e80043e0 con 0x7fe8f0139140 2023-09-30T22:26:15.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.379+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe8e8022070 con 0x7fe8f0139140 2023-09-30T22:26:15.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.383+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8e8012e60 con 0x7fe8f0139140 2023-09-30T22:26:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.555+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f91840747f0 con 0x7f9184133930 2023-09-30T22:26:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.556+0000 7f9182ffd700 1 -- 172.21.15.32:0/574768764 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f917402b090 con 0x7f9184133930 2023-09-30T22:26:15.556 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9184133930 msgr2=0x7f9184121be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9184133930 0x7f9184121be0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9174009d90 tx=0x7f917400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 shutdown_connections 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9184133930 0x7f9184121be0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f918409d630 0x7f9184121160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 --2- 172.21.15.32:0/574768764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91840a0a30 0x7f91841216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 >> 172.21.15.32:0/574768764 conn(0x7f91840a3050 msgr2=0x7f91841331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 shutdown_connections 2023-09-30T22:26:15.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.558+0000 7f918c379700 1 -- 172.21.15.32:0/574768764 wait complete. 2023-09-30T22:26:15.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.591+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe8f006b250 con 0x7fe8f0139140 2023-09-30T22:26:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.591+0000 7fe8d7fff700 1 -- 172.21.15.32:0/1288925726 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8e80263e0 con 0x7fe8f0139140 2023-09-30T22:26:15.591 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 msgr2=0x7fe8f012e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 0x7fe8f012e890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8016000 tx=0x7fe8e8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 shutdown_connections 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8f0139140 0x7fe8f012e890 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8f0133a80 0x7fe8f012de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 --2- 172.21.15.32:0/1288925726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8f0136820 0x7fe8f012e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 >> 172.21.15.32:0/1288925726 conn(0x7fe8f00a4210 msgr2=0x7fe8f0137310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 shutdown_connections 2023-09-30T22:26:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.593+0000 7fe8f624e700 1 -- 172.21.15.32:0/1288925726 wait complete. 2023-09-30T22:26:15.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:15.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:15.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.788+0000 7f9a71279700 1 Processor -- start 2023-09-30T22:26:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.788+0000 7f9a71279700 1 -- start start 2023-09-30T22:26:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a71279700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a71279700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6c12ff20 0x7f9a6c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a71279700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c130870 0x7f9a6c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a71279700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6c06c570 con 0x7f9a6c12ff20 2023-09-30T22:26:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a71279700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c06bc00 con 0x7f9a6c133a80 2023-09-30T22:26:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a71279700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c0a0040 con 0x7f9a6c130870 2023-09-30T22:26:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6c12ff20 0x7f9a6c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c130870 0x7f9a6c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37740/0 (socket says 172.21.15.32:37740) 2023-09-30T22:26:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a6c12ff20 0x7f9a6c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38516/0 (socket says 172.21.15.32:38516) 2023-09-30T22:26:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c130870 0x7f9a6c13a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55886/0 (socket says 172.21.15.32:55886) 2023-09-30T22:26:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 -- 172.21.15.32:0/3613539028 learned_addr learned my addr 172.21.15.32:0/3613539028 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 -- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c130870 msgr2=0x7f9a6c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c130870 0x7f9a6c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 -- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 msgr2=0x7f9a6c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6a7fc700 1 -- 172.21.15.32:0/3613539028 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c133860 con 0x7f9a6c12ff20 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.789+0000 7f9a6affd700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.790+0000 7f9a6a7fc700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c12ff20 0x7f9a6c130330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9a600098f0 tx=0x7f9a60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75e68c030e7405e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.790+0000 7f9a69ffb700 1 -- 172.21.15.32:0/3613539028 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011830 con 0x7f9a6c12ff20 2023-09-30T22:26:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.790+0000 7f9a69ffb700 1 -- 172.21.15.32:0/3613539028 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a60011990 con 0x7f9a6c12ff20 2023-09-30T22:26:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.790+0000 7f9a69ffb700 1 -- 172.21.15.32:0/3613539028 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011d20 con 0x7f9a6c12ff20 2023-09-30T22:26:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.790+0000 7f9a71279700 1 -- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c12ff20 msgr2=0x7f9a6c130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.790+0000 7f9a71279700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c12ff20 0x7f9a6c130330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9a600098f0 tx=0x7f9a60009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:15.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 -- 172.21.15.32:0/3613539028 shutdown_connections 2023-09-30T22:26:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c130870 0x7f9a6c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 --2- 172.21.15.32:0/3613539028 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c12ff20 0x7f9a6c130330 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 -- 172.21.15.32:0/3613539028 >> 172.21.15.32:0/3613539028 conn(0x7f9a6c0a4210 msgr2=0x7f9a6c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 -- 172.21.15.32:0/3613539028 shutdown_connections 2023-09-30T22:26:15.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.791+0000 7f9a71279700 1 -- 172.21.15.32:0/3613539028 wait complete. 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 Processor -- start 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 -- start start 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 0x7f9a6c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c130870 0x7f9a6c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c11fd20 con 0x7f9a6c130870 2023-09-30T22:26:15.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c09d850 con 0x7f9a6c133a80 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a71279700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c1206e0 con 0x7f9a6c12ff20 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a6b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a6affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 0x7f9a6c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a6b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37748/0 (socket says 172.21.15.32:37748) 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a6affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 0x7f9a6c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55902/0 (socket says 172.21.15.32:55902) 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.792+0000 7f9a6b7fe700 1 -- 172.21.15.32:0/1060526011 learned_addr learned my addr 172.21.15.32:0/1060526011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6a7fc700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c130870 0x7f9a6c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6b7fe700 1 -- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 msgr2=0x7f9a6c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6b7fe700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 0x7f9a6c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6b7fe700 1 -- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c130870 msgr2=0x7f9a6c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6b7fe700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c130870 0x7f9a6c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6b7fe700 1 -- 172.21.15.32:0/1060526011 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c009a70 con 0x7f9a6c133a80 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6affd700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 0x7f9a6c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6a7fc700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c130870 0x7f9a6c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a6b7fe700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c09cf00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400e980 tx=0x7f9a6400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64009730 con 0x7f9a6c133a80 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a64013680 con 0x7f9a6c133a80 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a60009550 con 0x7f9a6c133a80 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64013a10 con 0x7f9a6c133a80 2023-09-30T22:26:15.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.793+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c120920 con 0x7f9a6c133a80 2023-09-30T22:26:15.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.794+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a64013b70 con 0x7f9a6c133a80 2023-09-30T22:26:15.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.794+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9a6403f6a0 con 0x7f9a6c133a80 2023-09-30T22:26:15.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.795+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c12d260 con 0x7f9a6c133a80 2023-09-30T22:26:15.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.800+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a6c12d260 con 0x7f9a6c133a80 2023-09-30T22:26:15.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 Processor -- start 2023-09-30T22:26:15.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 -- start start 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400a2a70 con 0x7f704009d630 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70400a01d0 con 0x7f7040138d80 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7046819700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f704009e2b0 con 0x7f70400a0a30 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f703ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7044db6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f703ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55904/0 (socket says 172.21.15.32:55904) 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f703f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f704009d630 0x7f70400a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57420/0 (socket says 172.21.15.32:57420) 2023-09-30T22:26:15.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f7044db6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38530/0 (socket says 172.21.15.32:38530) 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.830+0000 7f703ffff700 1 -- 172.21.15.32:0/1293396941 learned_addr learned my addr 172.21.15.32:0/1293396941 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703ffff700 1 -- 172.21.15.32:0/1293396941 >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 msgr2=0x7f704013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703ffff700 1 --2- 172.21.15.32:0/1293396941 >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703ffff700 1 -- 172.21.15.32:0/1293396941 >> v1:172.21.15.32:6789/0 conn(0x7f704009d630 legacy=0x7f70400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703ffff700 1 -- 172.21.15.32:0/1293396941 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704009dc50 con 0x7f70400a0a30 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f7044db6700 1 --2- 172.21.15.32:0/1293396941 >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703ffff700 1 --2- 172.21.15.32:0/1293396941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 0x7f70400a0e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70300098f0 tx=0x7f7030009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b7c404486e61a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703effd700 1 -- 172.21.15.32:0/1293396941 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030011830 con 0x7f70400a0a30 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703effd700 1 -- 172.21.15.32:0/1293396941 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030011990 con 0x7f70400a0a30 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.831+0000 7f703effd700 1 -- 172.21.15.32:0/1293396941 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030011d20 con 0x7f70400a0a30 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 -- 172.21.15.32:0/1293396941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 msgr2=0x7f70400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 --2- 172.21.15.32:0/1293396941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 0x7f70400a0e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70300098f0 tx=0x7f7030009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 -- 172.21.15.32:0/1293396941 shutdown_connections 2023-09-30T22:26:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 --2- 172.21.15.32:0/1293396941 >> v2:172.21.15.32:3300/0 conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 --2- 172.21.15.32:0/1293396941 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 -- 172.21.15.32:0/1293396941 >> 172.21.15.32:0/1293396941 conn(0x7f70400a3050 msgr2=0x7f70400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 -- 172.21.15.32:0/1293396941 shutdown_connections 2023-09-30T22:26:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.832+0000 7f7046819700 1 -- 172.21.15.32:0/1293396941 wait complete. 2023-09-30T22:26:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.833+0000 7f7046819700 1 Processor -- start 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.833+0000 7f7046819700 1 -- start start 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7046819700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704009d630 0x7f70401208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7046819700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70400a0a30 0x7f7040120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7046819700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 0x7f7040121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7046819700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040122470 con 0x7f704009d630 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7046819700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70401237c0 con 0x7f70400a0a30 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7046819700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7040121d50 con 0x7f7040138d80 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7044db6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 0x7f7040121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f703f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70400a0a30 0x7f7040120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7044db6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 0x7f7040121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55906/0 (socket says 172.21.15.32:55906) 2023-09-30T22:26:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f7044db6700 1 -- 172.21.15.32:0/2875513443 learned_addr learned my addr 172.21.15.32:0/2875513443 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.834+0000 7f703ffff700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704009d630 0x7f70401208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703f7fe700 1 -- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 msgr2=0x7f7040121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703f7fe700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 0x7f7040121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703f7fe700 1 -- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704009d630 msgr2=0x7f70401208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703f7fe700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704009d630 0x7f70401208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703f7fe700 1 -- 172.21.15.32:0/2875513443 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038009a70 con 0x7f70400a0a30 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f7044db6700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 0x7f7040121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703ffff700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704009d630 0x7f70401208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703f7fe700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70400a0a30 0x7f7040120e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f703400c420 tx=0x7f703400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034005cb0 con 0x7f70400a0a30 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034005e10 con 0x7f70400a0a30 2023-09-30T22:26:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7030009550 con 0x7f70400a0a30 2023-09-30T22:26:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70340072d0 con 0x7f70400a0a30 2023-09-30T22:26:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.835+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70400a0410 con 0x7f70400a0a30 2023-09-30T22:26:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.836+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7034007430 con 0x7f70400a0a30 2023-09-30T22:26:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.836+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f703403e990 con 0x7f70400a0a30 2023-09-30T22:26:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.836+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70400747f0 con 0x7f70400a0a30 2023-09-30T22:26:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:15.841+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7034017080 con 0x7f70400a0a30 2023-09-30T22:26:16.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.007+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9a6c12d260 con 0x7f9a6c133a80 2023-09-30T22:26:16.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.008+0000 7f9a537fe700 1 -- 172.21.15.32:0/1060526011 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a6c12d260 con 0x7f9a6c133a80 2023-09-30T22:26:16.008 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 msgr2=0x7f9a6c09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c09cf00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400e980 tx=0x7f9a6400ec90 comp rx=0 tx=0).stop 2023-09-30T22:26:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 shutdown_connections 2023-09-30T22:26:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a6c133a80 0x7f9a6c09cf00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a6c12ff20 0x7f9a6c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 --2- 172.21.15.32:0/1060526011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6c130870 0x7f9a6c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 >> 172.21.15.32:0/1060526011 conn(0x7f9a6c0a4210 msgr2=0x7f9a6c0a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 shutdown_connections 2023-09-30T22:26:16.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.010+0000 7f9a71279700 1 -- 172.21.15.32:0/1060526011 wait complete. 2023-09-30T22:26:16.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:16.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.046+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7040123ca0 con 0x7f70400a0a30 2023-09-30T22:26:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.046+0000 7f703cff9700 1 -- 172.21.15.32:0/2875513443 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70340076e0 con 0x7f70400a0a30 2023-09-30T22:26:16.046 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70400a0a30 msgr2=0x7f7040120e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70400a0a30 0x7f7040120e30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f703400c420 tx=0x7f703400c730 comp rx=0 tx=0).stop 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 shutdown_connections 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7040138d80 0x7f7040121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704009d630 0x7f70401208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 --2- 172.21.15.32:0/2875513443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70400a0a30 0x7f7040120e30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 >> 172.21.15.32:0/2875513443 conn(0x7f70400a3050 msgr2=0x7f7040139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.048+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 shutdown_connections 2023-09-30T22:26:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.049+0000 7f7046819700 1 -- 172.21.15.32:0/2875513443 wait complete. 2023-09-30T22:26:16.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:16.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:16.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.722+0000 7f5bf9cb5700 1 Processor -- start 2023-09-30T22:26:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf9cb5700 1 -- start start 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf9cb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bf409d6f0 0x7f5bf409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf9cb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf9cb5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf409b120 con 0x7f5bf409e000 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf9cb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf406b460 con 0x7f5bf409d6f0 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf9cb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf409a7e0 con 0x7f5bf409e440 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5bf409d6f0 0x7f5bf409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bf409e000 0x7f5bf41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57432/0 (socket says 172.21.15.32:57432) 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.723+0000 7f5bf2ffd700 1 -- 172.21.15.32:0/3564940049 learned_addr learned my addr 172.21.15.32:0/3564940049 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf37fe700 1 -- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 msgr2=0x7f5bf4134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf37fe700 1 --2- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf4134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf37fe700 1 -- 172.21.15.32:0/3564940049 >> v1:172.21.15.32:6789/0 conn(0x7f5bf409e000 legacy=0x7f5bf41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf37fe700 1 -- 172.21.15.32:0/3564940049 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf409a500 con 0x7f5bf409d6f0 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf27fc700 1 -- 172.21.15.32:0/3564940049 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421061378 0 0) 0x7f5bf409b120 con 0x7f5bf409e000 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf3fff700 1 --2- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf4134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf37fe700 1 --2- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409d6f0 0x7f5bf409dac0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009870 tx=0x7f5be4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58563ab7a92fd95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf27fc700 1 -- 172.21.15.32:0/3564940049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4011830 con 0x7f5bf409d6f0 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf27fc700 1 -- 172.21.15.32:0/3564940049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4011990 con 0x7f5bf409d6f0 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.724+0000 7f5bf27fc700 1 -- 172.21.15.32:0/3564940049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4011d20 con 0x7f5bf409d6f0 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409d6f0 msgr2=0x7f5bf409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409d6f0 0x7f5bf409dac0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009870 tx=0x7f5be4009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/3564940049 shutdown_connections 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf4134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/3564940049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409d6f0 0x7f5bf409dac0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/3564940049 >> 172.21.15.32:0/3564940049 conn(0x7f5bf4066f50 msgr2=0x7f5bf409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/3564940049 shutdown_connections 2023-09-30T22:26:16.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.725+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/3564940049 wait complete. 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.726+0000 7f5bf9cb5700 1 Processor -- start 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.726+0000 7f5bf9cb5700 1 -- start start 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf9cb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bf409d6f0 0x7f5bf41a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf9cb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409e000 0x7f5bf41a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf9cb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf41c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf9cb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41a8110 con 0x7f5bf409e000 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf9cb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41b9180 con 0x7f5bf409e440 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf9cb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41a8310 con 0x7f5bf409d6f0 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409e000 0x7f5bf41a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf41c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409e000 0x7f5bf41a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38558/0 (socket says 172.21.15.32:38558) 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf41c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37782/0 (socket says 172.21.15.32:37782) 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf2ffd700 1 -- 172.21.15.32:0/2713042104 learned_addr learned my addr 172.21.15.32:0/2713042104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf37fe700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bf409d6f0 0x7f5bf41a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 -- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bf409d6f0 msgr2=0x7f5bf41a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bf409d6f0 0x7f5bf41a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 -- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409e000 msgr2=0x7f5bf41a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409e000 0x7f5bf41a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 -- 172.21.15.32:0/2713042104 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec009a70 con 0x7f5bf409e440 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bf3fff700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf41c7fd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00efc0 tx=0x7f5bec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00ca90 con 0x7f5bf409e440 2023-09-30T22:26:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.727+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00f040 con 0x7f5bf409e440 2023-09-30T22:26:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.728+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00ce30 con 0x7f5bf409e440 2023-09-30T22:26:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.728+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be40094a0 con 0x7f5bf409e440 2023-09-30T22:26:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.728+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf41a6400 con 0x7f5bf409e440 2023-09-30T22:26:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.728+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5bec014480 con 0x7f5bf409e440 2023-09-30T22:26:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.728+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf409c710 con 0x7f5bf409e440 2023-09-30T22:26:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.729+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5bec03fc00 con 0x7f5bf409e440 2023-09-30T22:26:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.733+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5bec012930 con 0x7f5bf409e440 2023-09-30T22:26:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.759+0000 7f2b30388700 1 Processor -- start 2023-09-30T22:26:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.759+0000 7f2b30388700 1 -- start start 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.760+0000 7f2b30388700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b2809d6f0 0x7f2b2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b30388700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809e410 0x7f2b28130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b30388700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2809f130 con 0x7f2b2809e000 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b30388700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2809f520 con 0x7f2b2809d6f0 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b30388700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2809e930 con 0x7f2b2809e410 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e925700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809e410 0x7f2b28130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e124700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b2809d6f0 0x7f2b2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2d923700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b2809e000 0x7f2b28130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57436/0 (socket says 172.21.15.32:57436) 2023-09-30T22:26:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2d923700 1 -- 172.21.15.32:0/238312077 learned_addr learned my addr 172.21.15.32:0/238312077 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e925700 1 -- 172.21.15.32:0/238312077 >> v2:172.21.15.32:3300/0 conn(0x7f2b2809d6f0 msgr2=0x7f2b2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e925700 1 --2- 172.21.15.32:0/238312077 >> v2:172.21.15.32:3300/0 conn(0x7f2b2809d6f0 0x7f2b2809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e925700 1 -- 172.21.15.32:0/238312077 >> v1:172.21.15.32:6789/0 conn(0x7f2b2809e000 legacy=0x7f2b28130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e925700 1 -- 172.21.15.32:0/238312077 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2809b2d0 con 0x7f2b2809e410 2023-09-30T22:26:16.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2e925700 1 --2- 172.21.15.32:0/238312077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809e410 0x7f2b28130d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009d90 tx=0x7f2b2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f829fc1ab828817f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.761+0000 7f2b2d122700 1 -- 172.21.15.32:0/238312077 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2000e070 con 0x7f2b2809e410 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b2d122700 1 -- 172.21.15.32:0/238312077 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2000bb20 con 0x7f2b2809e410 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b2d122700 1 -- 172.21.15.32:0/238312077 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20012430 con 0x7f2b2809e410 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 -- 172.21.15.32:0/238312077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809e410 msgr2=0x7f2b28130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 --2- 172.21.15.32:0/238312077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809e410 0x7f2b28130d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009d90 tx=0x7f2b2000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 -- 172.21.15.32:0/238312077 shutdown_connections 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 --2- 172.21.15.32:0/238312077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809e410 0x7f2b28130d70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 --2- 172.21.15.32:0/238312077 >> v2:172.21.15.32:3300/0 conn(0x7f2b2809d6f0 0x7f2b2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 -- 172.21.15.32:0/238312077 >> 172.21.15.32:0/238312077 conn(0x7f2b28066f50 msgr2=0x7f2b2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 -- 172.21.15.32:0/238312077 shutdown_connections 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.762+0000 7f2b30388700 1 -- 172.21.15.32:0/238312077 wait complete. 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.763+0000 7f2b30388700 1 Processor -- start 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.763+0000 7f2b30388700 1 -- start start 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b30388700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809d6f0 0x7f2b28120430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b30388700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b2809e000 0x7f2b28120970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b30388700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 0x7f2b28120eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b30388700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2809c700 con 0x7f2b2809e000 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b30388700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28122150 con 0x7f2b2809e410 2023-09-30T22:26:16.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b30388700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28123c70 con 0x7f2b2809d6f0 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 0x7f2b28120eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2d923700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b2809e000 0x7f2b28120970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 0x7f2b28120eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37788/0 (socket says 172.21.15.32:37788) 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 -- 172.21.15.32:0/458148300 learned_addr learned my addr 172.21.15.32:0/458148300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e124700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809d6f0 0x7f2b28120430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 -- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809d6f0 msgr2=0x7f2b28120430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809d6f0 0x7f2b28120430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 -- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b2809e000 msgr2=0x7f2b28120970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b2809e000 0x7f2b28120970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 -- 172.21.15.32:0/458148300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b20009a70 con 0x7f2b2809e410 2023-09-30T22:26:16.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2d923700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b2809e000 0x7f2b28120970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.764+0000 7f2b2e925700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 0x7f2b28120eb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2b20014040 tx=0x7f2b20004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.765+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20012430 con 0x7f2b2809e410 2023-09-30T22:26:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.765+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20020070 con 0x7f2b2809e410 2023-09-30T22:26:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.765+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b280a1ca0 con 0x7f2b2809e410 2023-09-30T22:26:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.765+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20003ea0 con 0x7f2b2809e410 2023-09-30T22:26:16.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.765+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b281246e0 con 0x7f2b2809e410 2023-09-30T22:26:16.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.766+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b20004040 con 0x7f2b2809e410 2023-09-30T22:26:16.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.766+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b280a2a20 con 0x7f2b2809e410 2023-09-30T22:26:16.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.766+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2b20022070 con 0x7f2b2809e410 2023-09-30T22:26:16.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.775+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b2001a970 con 0x7f2b2809e410 2023-09-30T22:26:16.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.947+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5bf409f420 con 0x7f5bf409e440 2023-09-30T22:26:16.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.947+0000 7f5bdbfff700 1 -- 172.21.15.32:0/2713042104 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bec012340 con 0x7f5bf409e440 2023-09-30T22:26:16.947 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 msgr2=0x7f5bf41c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf41c7fd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00efc0 tx=0x7f5bec00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 shutdown_connections 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5bf409e440 0x7f5bf41c7fd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5bf409d6f0 0x7f5bf41a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 --2- 172.21.15.32:0/2713042104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bf409e000 0x7f5bf41a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 >> 172.21.15.32:0/2713042104 conn(0x7f5bf4066f50 msgr2=0x7f5bf41bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 shutdown_connections 2023-09-30T22:26:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.950+0000 7f5bf9cb5700 1 -- 172.21.15.32:0/2713042104 wait complete. 2023-09-30T22:26:16.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.983+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b28074ea0 con 0x7f2b2809e410 2023-09-30T22:26:16.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.984+0000 7f2b26ffd700 1 -- 172.21.15.32:0/458148300 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b2002c030 con 0x7f2b2809e410 2023-09-30T22:26:16.984 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 msgr2=0x7f2b28120eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 0x7f2b28120eb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2b20014040 tx=0x7f2b20004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 shutdown_connections 2023-09-30T22:26:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b2809e410 0x7f2b28120eb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b2809d6f0 0x7f2b28120430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 --2- 172.21.15.32:0/458148300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b2809e000 0x7f2b28120970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:16.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 >> 172.21.15.32:0/458148300 conn(0x7f2b28066f50 msgr2=0x7f2b2806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 shutdown_connections 2023-09-30T22:26:16.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:16.986+0000 7f2b30388700 1 -- 172.21.15.32:0/458148300 wait complete. 2023-09-30T22:26:16.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:17.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:17.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 Processor -- start 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 -- start start 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f466c0a09c0 0x7f466c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c0a12d0 0x7f466c134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c09ea10 con 0x7f466c09d5c0 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c09e210 con 0x7f466c0a09c0 2023-09-30T22:26:17.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f467181b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0a26f0 con 0x7f466c0a12d0 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f466affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f466c0a09c0 0x7f466c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f466a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f466c09d5c0 0x7f466c133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57438/0 (socket says 172.21.15.32:57438) 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f466a7fc700 1 -- 172.21.15.32:0/761285207 learned_addr learned my addr 172.21.15.32:0/761285207 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.183+0000 7f466b7fe700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c0a12d0 0x7f466c134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f466affd700 1 -- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c0a12d0 msgr2=0x7f466c134030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f466affd700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c0a12d0 0x7f466c134030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f466affd700 1 -- 172.21.15.32:0/761285207 >> v1:172.21.15.32:6789/0 conn(0x7f466c09d5c0 legacy=0x7f466c133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f466affd700 1 -- 172.21.15.32:0/761285207 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c09db70 con 0x7f466c0a09c0 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f4669ffb700 1 -- 172.21.15.32:0/761285207 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131505162 0 0) 0x7f466c09ea10 con 0x7f466c09d5c0 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f466b7fe700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c0a12d0 0x7f466c134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f466affd700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c0a0d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f465c009840 tx=0x7f465c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632c098ab39e4f81 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f4669ffb700 1 -- 172.21.15.32:0/761285207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011830 con 0x7f466c0a09c0 2023-09-30T22:26:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f4669ffb700 1 -- 172.21.15.32:0/761285207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c011990 con 0x7f466c0a09c0 2023-09-30T22:26:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.184+0000 7f4669ffb700 1 -- 172.21.15.32:0/761285207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011d20 con 0x7f466c0a09c0 2023-09-30T22:26:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 -- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 msgr2=0x7f466c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c0a0d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f465c009840 tx=0x7f465c009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 -- 172.21.15.32:0/761285207 shutdown_connections 2023-09-30T22:26:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c0a12d0 0x7f466c134030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 --2- 172.21.15.32:0/761285207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c0a0d90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 -- 172.21.15.32:0/761285207 >> 172.21.15.32:0/761285207 conn(0x7f466c0a2fe0 msgr2=0x7f466c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 -- 172.21.15.32:0/761285207 shutdown_connections 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.185+0000 7f467181b700 1 -- 172.21.15.32:0/761285207 wait complete. 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.186+0000 7f467181b700 1 Processor -- start 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.186+0000 7f467181b700 1 -- start start 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f467181b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 0x7f466c1c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f467181b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c1c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f467181b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f466c0a12d0 0x7f466c1c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f467181b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c123400 con 0x7f466c0a09c0 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f467181b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1216f0 con 0x7f466c09d5c0 2023-09-30T22:26:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f467181b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f466c09c340 con 0x7f466c0a12d0 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 0x7f466c1c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c1c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 0x7f466c1c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37794/0 (socket says 172.21.15.32:37794) 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 -- 172.21.15.32:0/3401432921 learned_addr learned my addr 172.21.15.32:0/3401432921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466b7fe700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f466c0a12d0 0x7f466c1c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 -- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f466c0a12d0 msgr2=0x7f466c1c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f466c0a12d0 0x7f466c1c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 -- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 msgr2=0x7f466c1c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c1c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 -- 172.21.15.32:0/3401432921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664009a70 con 0x7f466c09d5c0 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466a7fc700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c1c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.187+0000 7f466affd700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 0x7f466c1c8080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f465c009fd0 tx=0x7f465c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.188+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0043c0 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.188+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c01f070 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.188+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0036a0 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.188+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c0094a0 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.188+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c121ee0 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.188+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f465c021030 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.189+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f465c0407f0 con 0x7f466c09d5c0 2023-09-30T22:26:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.189+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c0747f0 con 0x7f466c09d5c0 2023-09-30T22:26:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.194+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f465c0239c0 con 0x7f466c09d5c0 2023-09-30T22:26:17.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.224+0000 7fefdeeea700 1 Processor -- start 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.224+0000 7fefdeeea700 1 -- start start 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdeeea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefd8133420 0x7fefd81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdeeea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdeeea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd806b990 con 0x7fefd8134260 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdeeea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd80a0590 con 0x7fefd8133420 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdeeea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefd809fb80 con 0x7fefd8133e30 2023-09-30T22:26:17.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefd8133420 0x7fefd81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdd487700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fefd8134260 0x7fefd81304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57440/0 (socket says 172.21.15.32:57440) 2023-09-30T22:26:17.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdd487700 1 -- 172.21.15.32:0/2789768974 learned_addr learned my addr 172.21.15.32:0/2789768974 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:17.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 --2- 172.21.15.32:0/2789768974 >> v2:172.21.15.32:3300/0 conn(0x7fefd8133420 0x7fefd81337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38600/0 (socket says 172.21.15.32:38600) 2023-09-30T22:26:17.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 -- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 msgr2=0x7fefd812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:17.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 --2- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 -- 172.21.15.32:0/2789768974 >> v1:172.21.15.32:6789/0 conn(0x7fefd8134260 legacy=0x7fefd81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefd77fe700 1 -- 172.21.15.32:0/2789768974 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194191095 0 0) 0x7fefd806b990 con 0x7fefd8134260 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 -- 172.21.15.32:0/2789768974 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80a0030 con 0x7fefd8133420 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefd7fff700 1 --2- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.225+0000 7fefdcc86700 1 --2- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8133420 0x7fefd81337f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009840 tx=0x7fefc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3bef4312c4d0d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefd77fe700 1 -- 172.21.15.32:0/2789768974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011830 con 0x7fefd8133420 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefd77fe700 1 -- 172.21.15.32:0/2789768974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8011990 con 0x7fefd8133420 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefd77fe700 1 -- 172.21.15.32:0/2789768974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011d20 con 0x7fefd8133420 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 -- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8133420 msgr2=0x7fefd81337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8133420 0x7fefd81337f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009840 tx=0x7fefc8009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 -- 172.21.15.32:0/2789768974 shutdown_connections 2023-09-30T22:26:17.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8133420 0x7fefd81337f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2789768974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 -- 172.21.15.32:0/2789768974 >> 172.21.15.32:0/2789768974 conn(0x7fefd80a4210 msgr2=0x7fefd80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 -- 172.21.15.32:0/2789768974 shutdown_connections 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.226+0000 7fefdeeea700 1 -- 172.21.15.32:0/2789768974 wait complete. 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.227+0000 7fefdeeea700 1 Processor -- start 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.227+0000 7fefdeeea700 1 -- start start 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdeeea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd8133420 0x7fefd809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdeeea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdeeea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8134260 0x7fefd809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:17.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdeeea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8120110 con 0x7fefd8134260 2023-09-30T22:26:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdeeea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd811f080 con 0x7fefd8133420 2023-09-30T22:26:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdeeea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefd809deb0 con 0x7fefd8133e30 2023-09-30T22:26:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdd487700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8134260 0x7fefd809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefd7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55954/0 (socket says 172.21.15.32:55954) 2023-09-30T22:26:17.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefd7fff700 1 -- 172.21.15.32:0/2538672083 learned_addr learned my addr 172.21.15.32:0/2538672083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.228+0000 7fefdcc86700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd8133420 0x7fefd809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdd487700 1 -- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 msgr2=0x7fefd809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdd487700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdd487700 1 -- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd8133420 msgr2=0x7fefd809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdd487700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd8133420 0x7fefd809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdd487700 1 -- 172.21.15.32:0/2538672083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00094a0 con 0x7fefd8134260 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdcc86700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd8133420 0x7fefd809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdd487700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8134260 0x7fefd809cf00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cbe0 tx=0x7fefd000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000aa00 con 0x7fefd8134260 2023-09-30T22:26:17.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000ab80 con 0x7fefd8134260 2023-09-30T22:26:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc80094a0 con 0x7fefd8134260 2023-09-30T22:26:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd00070d0 con 0x7fefd8134260 2023-09-30T22:26:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.229+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd811fd90 con 0x7fefd8134260 2023-09-30T22:26:17.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.230+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefd0007230 con 0x7fefd8134260 2023-09-30T22:26:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.230+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fefd003f390 con 0x7fefd8134260 2023-09-30T22:26:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.230+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd811fa30 con 0x7fefd8134260 2023-09-30T22:26:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.235+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefd0018080 con 0x7fefd8134260 2023-09-30T22:26:17.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.404+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f466c123df0 con 0x7f466c09d5c0 2023-09-30T22:26:17.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.404+0000 7f4663fff700 1 -- 172.21.15.32:0/3401432921 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f465c015020 con 0x7f466c09d5c0 2023-09-30T22:26:17.404 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 msgr2=0x7f466c1c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 0x7f466c1c8080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f465c009fd0 tx=0x7f465c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 shutdown_connections 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f466c0a12d0 0x7f466c1c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f466c09d5c0 0x7f466c1c8080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 --2- 172.21.15.32:0/3401432921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0a09c0 0x7f466c1c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 >> 172.21.15.32:0/3401432921 conn(0x7f466c0a2fe0 msgr2=0x7f466c0a4890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.406+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 shutdown_connections 2023-09-30T22:26:17.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.407+0000 7f467181b700 1 -- 172.21.15.32:0/3401432921 wait complete. 2023-09-30T22:26:17.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:17.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.441+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fefd8074ea0 con 0x7fefd8134260 2023-09-30T22:26:17.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.441+0000 7fefd57fa700 1 -- 172.21.15.32:0/2538672083 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefd00074e0 con 0x7fefd8134260 2023-09-30T22:26:17.441 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:17.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8134260 msgr2=0x7fefd809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:17.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8134260 0x7fefd809cf00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cbe0 tx=0x7fefd000cef0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 shutdown_connections 2023-09-30T22:26:17.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd8134260 0x7fefd809cf00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd8133420 0x7fefd809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 --2- 172.21.15.32:0/2538672083 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd8133e30 0x7fefd809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:17.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 >> 172.21.15.32:0/2538672083 conn(0x7fefd80a4210 msgr2=0x7fefd812de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:17.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 shutdown_connections 2023-09-30T22:26:17.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:17.443+0000 7fefdeeea700 1 -- 172.21.15.32:0/2538672083 wait complete. 2023-09-30T22:26:17.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:17.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:18.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.113+0000 7f4170b4c700 1 Processor -- start 2023-09-30T22:26:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.113+0000 7f4170b4c700 1 -- start start 2023-09-30T22:26:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.113+0000 7f4170b4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f4170b4c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f416c134170 0x7f416c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f4170b4c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c09edd0 con 0x7f416c0a1340 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f4170b4c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c09e1f0 con 0x7f416c134170 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f4170b4c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f416c09dc50 con 0x7f416c0a0a30 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f416c134170 0x7f416c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f416c134170 0x7f416c13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38624/0 (socket says 172.21.15.32:38624) 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55970/0 (socket says 172.21.15.32:55970) 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 -- 172.21.15.32:0/633504208 learned_addr learned my addr 172.21.15.32:0/633504208 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 -- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 msgr2=0x7f416c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 --2- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 -- 172.21.15.32:0/633504208 >> v1:172.21.15.32:6789/0 conn(0x7f416c0a1340 legacy=0x7f416c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416959a700 1 -- 172.21.15.32:0/633504208 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003631325 0 0) 0x7f416c09edd0 con 0x7f416c0a1340 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 -- 172.21.15.32:0/633504208 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0a1fe0 con 0x7f416c134170 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416a59c700 1 --2- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.114+0000 7f416ad9d700 1 --2- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c13bd30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f41640098f0 tx=0x7f4164009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7971154cf8985412 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f416959a700 1 -- 172.21.15.32:0/633504208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011830 con 0x7f416c134170 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f416959a700 1 -- 172.21.15.32:0/633504208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164011990 con 0x7f416c134170 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f416959a700 1 -- 172.21.15.32:0/633504208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011d20 con 0x7f416c134170 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 -- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 msgr2=0x7f416c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 --2- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c13bd30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f41640098f0 tx=0x7f4164009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 -- 172.21.15.32:0/633504208 shutdown_connections 2023-09-30T22:26:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 --2- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c13bd30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 --2- 172.21.15.32:0/633504208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 -- 172.21.15.32:0/633504208 >> 172.21.15.32:0/633504208 conn(0x7f416c0a6550 msgr2=0x7f416c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 -- 172.21.15.32:0/633504208 shutdown_connections 2023-09-30T22:26:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.115+0000 7f4170b4c700 1 -- 172.21.15.32:0/633504208 wait complete. 2023-09-30T22:26:18.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 Processor -- start 2023-09-30T22:26:18.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 -- start start 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f416c0a1340 0x7f416c122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c125f30 con 0x7f416c134170 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0a4dc0 con 0x7f416c0a1340 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f4170b4c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0a0220 con 0x7f416c0a0a30 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38626/0 (socket says 172.21.15.32:38626) 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.117+0000 7f416ad9d700 1 -- 172.21.15.32:0/1474330641 learned_addr learned my addr 172.21.15.32:0/1474330641 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4169d9b700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f416c0a1340 0x7f416c122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416ad9d700 1 -- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 msgr2=0x7f416c122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416a59c700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416ad9d700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416ad9d700 1 -- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f416c0a1340 msgr2=0x7f416c122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416ad9d700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f416c0a1340 0x7f416c122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416ad9d700 1 -- 172.21.15.32:0/1474330641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c009a70 con 0x7f416c134170 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4169d9b700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f416c0a1340 0x7f416c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416a59c700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f416ad9d700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c123200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f416c09e330 tx=0x7f4164004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640044f0 con 0x7f416c134170 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416401f070 con 0x7f416c134170 2023-09-30T22:26:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4164009550 con 0x7f416c134170 2023-09-30T22:26:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640038e0 con 0x7f416c134170 2023-09-30T22:26:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.118+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c1269a0 con 0x7f416c134170 2023-09-30T22:26:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.119+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4164021030 con 0x7f416c134170 2023-09-30T22:26:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.119+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f41640409f0 con 0x7f416c134170 2023-09-30T22:26:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.119+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c125bd0 con 0x7f416c134170 2023-09-30T22:26:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.124+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4164015080 con 0x7f416c134170 2023-09-30T22:26:18.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.153+0000 7f1a11bf1700 1 Processor -- start 2023-09-30T22:26:18.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.153+0000 7f1a11bf1700 1 -- start start 2023-09-30T22:26:18.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a11bf1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a0c0a0a30 0x7f1a0c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a11bf1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 0x7f1a0c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a11bf1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0a2a70 con 0x7f1a0c09d630 2023-09-30T22:26:18.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a11bf1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c0a01d0 con 0x7f1a0c0a0a30 2023-09-30T22:26:18.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a11bf1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c09e2b0 con 0x7f1a0c138d80 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 0x7f1a0c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 0x7f1a0c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55986/0 (socket says 172.21.15.32:55986) 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 -- 172.21.15.32:0/385455949 learned_addr learned my addr 172.21.15.32:0/385455949 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0b7fe700 1 --2- 172.21.15.32:0/385455949 >> v2:172.21.15.32:3300/0 conn(0x7f1a0c0a0a30 0x7f1a0c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 -- 172.21.15.32:0/385455949 >> v2:172.21.15.32:3300/0 conn(0x7f1a0c0a0a30 msgr2=0x7f1a0c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 --2- 172.21.15.32:0/385455949 >> v2:172.21.15.32:3300/0 conn(0x7f1a0c0a0a30 0x7f1a0c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 -- 172.21.15.32:0/385455949 >> v1:172.21.15.32:6789/0 conn(0x7f1a0c09d630 legacy=0x7f1a0c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 -- 172.21.15.32:0/385455949 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c09dc50 con 0x7f1a0c138d80 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0b7fe700 1 --2- 172.21.15.32:0/385455949 >> v2:172.21.15.32:3300/0 conn(0x7f1a0c0a0a30 0x7f1a0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.154+0000 7f1a0bfff700 1 --2- 172.21.15.32:0/385455949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 0x7f1a0c13b160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009e50 tx=0x7f1a0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2eb873e3e870e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a0a7fc700 1 -- 172.21.15.32:0/385455949 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400e070 con 0x7f1a0c138d80 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a0a7fc700 1 -- 172.21.15.32:0/385455949 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0400b910 con 0x7f1a0c138d80 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a0a7fc700 1 -- 172.21.15.32:0/385455949 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400bcc0 con 0x7f1a0c138d80 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 -- 172.21.15.32:0/385455949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 msgr2=0x7f1a0c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/385455949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 0x7f1a0c13b160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009e50 tx=0x7f1a0400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 -- 172.21.15.32:0/385455949 shutdown_connections 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/385455949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c138d80 0x7f1a0c13b160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/385455949 >> v2:172.21.15.32:3300/0 conn(0x7f1a0c0a0a30 0x7f1a0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 -- 172.21.15.32:0/385455949 >> 172.21.15.32:0/385455949 conn(0x7f1a0c0a3050 msgr2=0x7f1a0c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 -- 172.21.15.32:0/385455949 shutdown_connections 2023-09-30T22:26:18.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.155+0000 7f1a11bf1700 1 -- 172.21.15.32:0/385455949 wait complete. 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.156+0000 7f1a11bf1700 1 Processor -- start 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 -- start start 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 0x7f1a0c133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c0a0a30 0x7f1a0c133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a0c138d80 0x7f1a0c1344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c136b90 con 0x7f1a0c09d630 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c138090 con 0x7f1a0c138d80 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a11bf1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1374b0 con 0x7f1a0c0a0a30 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a0b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 0x7f1a0c133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a0affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c0a0a30 0x7f1a0c133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a0b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 0x7f1a0c133a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38634/0 (socket says 172.21.15.32:38634) 2023-09-30T22:26:18.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a0b7fe700 1 -- 172.21.15.32:0/2505773911 learned_addr learned my addr 172.21.15.32:0/2505773911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.157+0000 7f1a0bfff700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a0c138d80 0x7f1a0c1344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0b7fe700 1 -- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c0a0a30 msgr2=0x7f1a0c133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0b7fe700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c0a0a30 0x7f1a0c133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0b7fe700 1 -- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a0c138d80 msgr2=0x7f1a0c1344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0b7fe700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a0c138d80 0x7f1a0c1344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0b7fe700 1 -- 172.21.15.32:0/2505773911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0094a0 con 0x7f1a0c09d630 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0affd700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c0a0a30 0x7f1a0c133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a0b7fe700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 0x7f1a0c133a30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00ec10 tx=0x7f19fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00c850 con 0x7f1a0c09d630 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc00c9b0 con 0x7f1a0c09d630 2023-09-30T22:26:18.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00cd20 con 0x7f1a0c09d630 2023-09-30T22:26:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a04009b30 con 0x7f1a0c09d630 2023-09-30T22:26:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.158+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c1371d0 con 0x7f1a0c09d630 2023-09-30T22:26:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.159+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19fc013470 con 0x7f1a0c09d630 2023-09-30T22:26:18.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.159+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c137030 con 0x7f1a0c09d630 2023-09-30T22:26:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.159+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f19fc03fb30 con 0x7f1a0c09d630 2023-09-30T22:26:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.164+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a0c137030 con 0x7f1a0c09d630 2023-09-30T22:26:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.334+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f416c074ea0 con 0x7f416c134170 2023-09-30T22:26:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.335+0000 7f4162ffd700 1 -- 172.21.15.32:0/1474330641 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4164003a40 con 0x7f416c134170 2023-09-30T22:26:18.335 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 msgr2=0x7f416c123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c123200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f416c09e330 tx=0x7f4164004750 comp rx=0 tx=0).stop 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 shutdown_connections 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416c134170 0x7f416c123200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416c0a0a30 0x7f416c122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 --2- 172.21.15.32:0/1474330641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f416c0a1340 0x7f416c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 >> 172.21.15.32:0/1474330641 conn(0x7f416c0a6550 msgr2=0x7f416c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 shutdown_connections 2023-09-30T22:26:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.337+0000 7f4170b4c700 1 -- 172.21.15.32:0/1474330641 wait complete. 2023-09-30T22:26:18.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.372+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a0c137030 con 0x7f1a0c09d630 2023-09-30T22:26:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.372+0000 7f1a03fff700 1 -- 172.21.15.32:0/2505773911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a0c137030 con 0x7f1a0c09d630 2023-09-30T22:26:18.372 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:18.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 msgr2=0x7f1a0c133a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 0x7f1a0c133a30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00ec10 tx=0x7f19fc00ef20 comp rx=0 tx=0).stop 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 shutdown_connections 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a0c138d80 0x7f1a0c1344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a0c09d630 0x7f1a0c133a30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 --2- 172.21.15.32:0/2505773911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a0c0a0a30 0x7f1a0c133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 >> 172.21.15.32:0/2505773911 conn(0x7f1a0c0a3050 msgr2=0x7f1a0c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 shutdown_connections 2023-09-30T22:26:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.374+0000 7f1a11bf1700 1 -- 172.21.15.32:0/2505773911 wait complete. 2023-09-30T22:26:18.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:18.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:18.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:18.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.571+0000 7f248ae98700 1 Processor -- start 2023-09-30T22:26:18.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.571+0000 7f248ae98700 1 -- start start 2023-09-30T22:26:18.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f248ae98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f24840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f248ae98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f24840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f248ae98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2484133930 0x7f2484133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f248ae98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24840a2a70 con 0x7f2484133930 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f248ae98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840a01d0 con 0x7f248409d630 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f248ae98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f248409e2b0 con 0x7f24840a0a30 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2488c34700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f24840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2488c34700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f24840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55998/0 (socket says 172.21.15.32:55998) 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2488c34700 1 -- 172.21.15.32:0/1581627844 learned_addr learned my addr 172.21.15.32:0/1581627844 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 --2- 172.21.15.32:0/1581627844 >> v2:172.21.15.32:3300/0 conn(0x7f2484133930 0x7f2484133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2483fff700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f24840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 -- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 msgr2=0x7f24840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f24840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 -- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 msgr2=0x7f24840a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f24840a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 -- 172.21.15.32:0/1581627844 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248409dc50 con 0x7f2484133930 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2488c34700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f24840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.572+0000 7f2489435700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f2484133dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f247c009d90 tx=0x7f247c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ccaf2bcad0ee64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f24837fe700 1 -- 172.21.15.32:0/1581627844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00e070 con 0x7f2484133930 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f24837fe700 1 -- 172.21.15.32:0/1581627844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c00bb20 con 0x7f2484133930 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f24837fe700 1 -- 172.21.15.32:0/1581627844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c012430 con 0x7f2484133930 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 -- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 msgr2=0x7f2484133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f2484133dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f247c009d90 tx=0x7f247c00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 -- 172.21.15.32:0/1581627844 shutdown_connections 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f2484133dd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f24840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 --2- 172.21.15.32:0/1581627844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f24840a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 -- 172.21.15.32:0/1581627844 >> 172.21.15.32:0/1581627844 conn(0x7f24840a3050 msgr2=0x7f24840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.573+0000 7f248ae98700 1 -- 172.21.15.32:0/1581627844 shutdown_connections 2023-09-30T22:26:18.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.574+0000 7f248ae98700 1 -- 172.21.15.32:0/1581627844 wait complete. 2023-09-30T22:26:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.574+0000 7f248ae98700 1 Processor -- start 2023-09-30T22:26:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 -- start start 2023-09-30T22:26:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f248411e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f248411ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f248411f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484133460 con 0x7f2484133930 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840a50b0 con 0x7f248409d630 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f248ae98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24841205e0 con 0x7f24840a0a30 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f2488c34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f248411e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f2489435700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f248411f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f2488c34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f248411e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37846/0 (socket says 172.21.15.32:37846) 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f2489435700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f248411f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38656/0 (socket says 172.21.15.32:38656) 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.575+0000 7f2488c34700 1 -- 172.21.15.32:0/2366961917 learned_addr learned my addr 172.21.15.32:0/2366961917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2483fff700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f248411ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2488c34700 1 -- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 msgr2=0x7f248411ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2488c34700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f248411ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2488c34700 1 -- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 msgr2=0x7f248411f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2488c34700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f248411f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2488c34700 1 -- 172.21.15.32:0/2366961917 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474009550 con 0x7f248409d630 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2489435700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f248411f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f2488c34700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f248411e8e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f247400ebe0 tx=0x7f247400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247400c850 con 0x7f248409d630 2023-09-30T22:26:18.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247400c9b0 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c009a70 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247400cd40 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.576+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2484122090 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.577+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2474013470 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.577+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f247403fbe0 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.577+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24840747f0 con 0x7f248409d630 2023-09-30T22:26:18.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.582+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f247401b3b0 con 0x7f248409d630 2023-09-30T22:26:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 Processor -- start 2023-09-30T22:26:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 -- start start 2023-09-30T22:26:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f477c0a0a30 0x7f477c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c09d630 0x7f477c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0a2a70 con 0x7f477c133930 2023-09-30T22:26:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0a01d0 con 0x7f477c0a0a30 2023-09-30T22:26:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.613+0000 7f47833ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f477c09e2b0 con 0x7f477c09d630 2023-09-30T22:26:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478118b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f477c0a0a30 0x7f477c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478198c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f477c133930 0x7f477c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57472/0 (socket says 172.21.15.32:57472) 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478198c700 1 -- 172.21.15.32:0/1610125880 learned_addr learned my addr 172.21.15.32:0/1610125880 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478098a700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c09d630 0x7f477c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478118b700 1 -- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c09d630 msgr2=0x7f477c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478118b700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c09d630 0x7f477c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478118b700 1 -- 172.21.15.32:0/1610125880 >> v1:172.21.15.32:6789/0 conn(0x7f477c133930 legacy=0x7f477c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478118b700 1 -- 172.21.15.32:0/1610125880 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c09dc50 con 0x7f477c0a0a30 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f4773fff700 1 -- 172.21.15.32:0/1610125880 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191667567 0 0) 0x7f477c0a2a70 con 0x7f477c133930 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478098a700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c09d630 0x7f477c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f478118b700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c0a0e00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4778009840 tx=0x7f4778009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f880243f679ac9e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f4773fff700 1 -- 172.21.15.32:0/1610125880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778011830 con 0x7f477c0a0a30 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f4773fff700 1 -- 172.21.15.32:0/1610125880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4778011990 con 0x7f477c0a0a30 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.614+0000 7f4773fff700 1 -- 172.21.15.32:0/1610125880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778011d20 con 0x7f477c0a0a30 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 -- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 msgr2=0x7f477c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c0a0e00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4778009840 tx=0x7f4778009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 -- 172.21.15.32:0/1610125880 shutdown_connections 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c0a0e00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 --2- 172.21.15.32:0/1610125880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c09d630 0x7f477c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 -- 172.21.15.32:0/1610125880 >> 172.21.15.32:0/1610125880 conn(0x7f477c0a3050 msgr2=0x7f477c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 -- 172.21.15.32:0/1610125880 shutdown_connections 2023-09-30T22:26:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.615+0000 7f47833ef700 1 -- 172.21.15.32:0/1610125880 wait complete. 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.616+0000 7f47833ef700 1 Processor -- start 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.616+0000 7f47833ef700 1 -- start start 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f47833ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 0x7f477c1c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f47833ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f47833ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c133930 0x7f477c1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f47833ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c121f10 con 0x7f477c0a0a30 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f47833ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c133540 con 0x7f477c09d630 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f47833ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f477c120720 con 0x7f477c133930 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478118b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 0x7f477c1c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c1c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38662/0 (socket says 172.21.15.32:38662) 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478118b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 0x7f477c1c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37858/0 (socket says 172.21.15.32:37858) 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 -- 172.21.15.32:0/1666250148 learned_addr learned my addr 172.21.15.32:0/1666250148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478198c700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c133930 0x7f477c1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 -- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c133930 msgr2=0x7f477c1c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c133930 0x7f477c1c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 -- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 msgr2=0x7f477c1c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 0x7f477c1c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 -- 172.21.15.32:0/1666250148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0094a0 con 0x7f477c0a0a30 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478198c700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c133930 0x7f477c1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478118b700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 0x7f477c1c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:18.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f478098a700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c1c8450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f476c00ec10 tx=0x7f476c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.617+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00c850 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.618+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00c9b0 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.618+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47780094a0 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.618+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00cd20 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.618+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c0a42a0 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.619+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c0747f0 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.619+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f476c013470 con 0x7f477c0a0a30 2023-09-30T22:26:18.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.619+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f476c03ff90 con 0x7f477c0a0a30 2023-09-30T22:26:18.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.624+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f476c013720 con 0x7f477c0a0a30 2023-09-30T22:26:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.790+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2484120c50 con 0x7f248409d630 2023-09-30T22:26:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.791+0000 7f24817fa700 1 -- 172.21.15.32:0/2366961917 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2474024350 con 0x7f248409d630 2023-09-30T22:26:18.791 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 msgr2=0x7f248411e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f248411e8e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f247400ebe0 tx=0x7f247400eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 shutdown_connections 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2484133930 0x7f248411f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f248409d630 0x7f248411e8e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 --2- 172.21.15.32:0/2366961917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24840a0a30 0x7f248411ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.793+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 >> 172.21.15.32:0/2366961917 conn(0x7f24840a3050 msgr2=0x7f24840a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.794+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 shutdown_connections 2023-09-30T22:26:18.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.794+0000 7f248ae98700 1 -- 172.21.15.32:0/2366961917 wait complete. 2023-09-30T22:26:18.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.830+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f477c072980 con 0x7f477c0a0a30 2023-09-30T22:26:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.831+0000 7f4771ffb700 1 -- 172.21.15.32:0/1666250148 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f476c01ab00 con 0x7f477c0a0a30 2023-09-30T22:26:18.831 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 msgr2=0x7f477c1c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c1c8450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f476c00ec10 tx=0x7f476c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 shutdown_connections 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f477c133930 0x7f477c1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f477c09d630 0x7f477c1c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 --2- 172.21.15.32:0/1666250148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477c0a0a30 0x7f477c1c8450 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 >> 172.21.15.32:0/1666250148 conn(0x7f477c0a3050 msgr2=0x7f477c1bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 shutdown_connections 2023-09-30T22:26:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:18.833+0000 7f47833ef700 1 -- 172.21.15.32:0/1666250148 wait complete. 2023-09-30T22:26:18.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:19.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:19.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.501+0000 7feece252700 1 Processor -- start 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.501+0000 7feece252700 1 -- start start 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feece252700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a0a30 0x7feec80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feece252700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feec8134170 0x7feec813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feece252700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec809edd0 con 0x7feec80a1340 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feece252700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec809e1f0 con 0x7feec8134170 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feece252700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec809dc50 con 0x7feec80a0a30 2023-09-30T22:26:19.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feec8134170 0x7feec813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feec80a1340 0x7feec8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57480/0 (socket says 172.21.15.32:57480) 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec6ffd700 1 -- 172.21.15.32:0/669538771 learned_addr learned my addr 172.21.15.32:0/669538771 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec77fe700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a0a30 0x7feec80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec7fff700 1 -- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a0a30 msgr2=0x7feec80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec7fff700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a0a30 0x7feec80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec7fff700 1 -- 172.21.15.32:0/669538771 >> v1:172.21.15.32:6789/0 conn(0x7feec80a1340 legacy=0x7feec8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec7fff700 1 -- 172.21.15.32:0/669538771 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80a1fe0 con 0x7feec8134170 2023-09-30T22:26:19.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec67fc700 1 -- 172.21.15.32:0/669538771 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806807021 0 0) 0x7feec809edd0 con 0x7feec80a1340 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.502+0000 7feec77fe700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a0a30 0x7feec80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.503+0000 7feec7fff700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec813bd30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feec0009d90 tx=0x7feec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb74f8598b10e85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.503+0000 7feec67fc700 1 -- 172.21.15.32:0/669538771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000e070 con 0x7feec8134170 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.503+0000 7feec67fc700 1 -- 172.21.15.32:0/669538771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec000bb20 con 0x7feec8134170 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.503+0000 7feec67fc700 1 -- 172.21.15.32:0/669538771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0012430 con 0x7feec8134170 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 -- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 msgr2=0x7feec813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec813bd30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feec0009d90 tx=0x7feec000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 -- 172.21.15.32:0/669538771 shutdown_connections 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec813bd30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 --2- 172.21.15.32:0/669538771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a0a30 0x7feec80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 -- 172.21.15.32:0/669538771 >> 172.21.15.32:0/669538771 conn(0x7feec80a6550 msgr2=0x7feec80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:19.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 -- 172.21.15.32:0/669538771 shutdown_connections 2023-09-30T22:26:19.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.504+0000 7feece252700 1 -- 172.21.15.32:0/669538771 wait complete. 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 Processor -- start 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 -- start start 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feec80a0a30 0x7feec8122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 0x7feec8122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec8123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec8125f30 con 0x7feec8134170 2023-09-30T22:26:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec80a4dc0 con 0x7feec80a1340 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.505+0000 7feece252700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feec80a0220 con 0x7feec80a0a30 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 0x7feec8122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 0x7feec8122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37884/0 (socket says 172.21.15.32:37884) 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 -- 172.21.15.32:0/965168786 learned_addr learned my addr 172.21.15.32:0/965168786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec77fe700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feec80a0a30 0x7feec8122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec7fff700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec8123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 -- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feec80a0a30 msgr2=0x7feec8122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feec80a0a30 0x7feec8122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 -- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 msgr2=0x7feec8123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec8123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 -- 172.21.15.32:0/965168786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80094a0 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec77fe700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feec80a0a30 0x7feec8122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feec6ffd700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 0x7feec8122cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c520 tx=0x7feebc00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc004c80 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc004e00 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feece252700 1 -- 172.21.15.32:0/965168786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec0009a70 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc007310 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.506+0000 7feece252700 1 -- 172.21.15.32:0/965168786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec81269a0 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.507+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feebc007470 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.507+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feebc03f290 con 0x7feec80a1340 2023-09-30T22:26:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.508+0000 7feece252700 1 -- 172.21.15.32:0/965168786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec8074ea0 con 0x7feec80a1340 2023-09-30T22:26:19.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.513+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feebc0123b0 con 0x7feec80a1340 2023-09-30T22:26:19.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.540+0000 7ff8c1bab700 1 Processor -- start 2023-09-30T22:26:19.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.540+0000 7ff8c1bab700 1 -- start start 2023-09-30T22:26:19.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.540+0000 7ff8c1bab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.540+0000 7ff8c1bab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 0x7ff8bc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8c1bab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8bc0a2a70 con 0x7ff8bc133930 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8c1bab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc0a01d0 con 0x7ff8bc09d630 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8c1bab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc09e2b0 con 0x7ff8bc0a0a30 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8baffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 0x7ff8bc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8baffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 0x7ff8bc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38686/0 (socket says 172.21.15.32:38686) 2023-09-30T22:26:19.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8baffd700 1 -- 172.21.15.32:0/793037134 learned_addr learned my addr 172.21.15.32:0/793037134 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8bb7fe700 1 -- 172.21.15.32:0/793037134 >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 msgr2=0x7ff8bc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8bb7fe700 1 --2- 172.21.15.32:0/793037134 >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 0x7ff8bc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8bb7fe700 1 -- 172.21.15.32:0/793037134 >> v1:172.21.15.32:6789/0 conn(0x7ff8bc133930 legacy=0x7ff8bc133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8ba7fc700 1 -- 172.21.15.32:0/793037134 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866734084 0 0) 0x7ff8bc0a2a70 con 0x7ff8bc133930 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8bb7fe700 1 -- 172.21.15.32:0/793037134 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc09dc50 con 0x7ff8bc0a0a30 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8baffd700 1 --2- 172.21.15.32:0/793037134 >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 0x7ff8bc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8bb7fe700 1 --2- 172.21.15.32:0/793037134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc0a0e00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009d90 tx=0x7ff8ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee45702068938a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:19.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.541+0000 7ff8ba7fc700 1 -- 172.21.15.32:0/793037134 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00e070 con 0x7ff8bc0a0a30 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8ba7fc700 1 -- 172.21.15.32:0/793037134 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00bb20 con 0x7ff8bc0a0a30 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8ba7fc700 1 -- 172.21.15.32:0/793037134 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac012430 con 0x7ff8bc0a0a30 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 -- 172.21.15.32:0/793037134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 msgr2=0x7ff8bc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/793037134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc0a0e00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009d90 tx=0x7ff8ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 -- 172.21.15.32:0/793037134 shutdown_connections 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/793037134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc0a0e00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/793037134 >> v2:172.21.15.32:3300/0 conn(0x7ff8bc09d630 0x7ff8bc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 -- 172.21.15.32:0/793037134 >> 172.21.15.32:0/793037134 conn(0x7ff8bc0a3050 msgr2=0x7ff8bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 -- 172.21.15.32:0/793037134 shutdown_connections 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.542+0000 7ff8c1bab700 1 -- 172.21.15.32:0/793037134 wait complete. 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.544+0000 7ff8c1bab700 1 Processor -- start 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.544+0000 7ff8c1bab700 1 -- start start 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8c1bab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8bc09d630 0x7ff8bc1c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8c1bab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc1c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8c1bab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 0x7ff8bc1c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8c1bab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc121e90 con 0x7ff8bc133930 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8c1bab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc121290 con 0x7ff8bc09d630 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8c1bab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc120520 con 0x7ff8bc0a0a30 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 0x7ff8bc1c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8bc09d630 0x7ff8bc1c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 0x7ff8bc1c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38700/0 (socket says 172.21.15.32:38700) 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 -- 172.21.15.32:0/2943489553 learned_addr learned my addr 172.21.15.32:0/2943489553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8baffd700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc1c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 -- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 msgr2=0x7ff8bc1c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc1c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 -- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8bc09d630 msgr2=0x7ff8bc1c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8bc09d630 0x7ff8bc1c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bbfff700 1 -- 172.21.15.32:0/2943489553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b0009550 con 0x7ff8bc133930 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.545+0000 7ff8bb7fe700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8bc09d630 0x7ff8bc1c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.546+0000 7ff8bbfff700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 0x7ff8bc1c8970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c4d0 tx=0x7ff8b400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.546+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40054e0 con 0x7ff8bc133930 2023-09-30T22:26:19.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.546+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b4003680 con 0x7ff8bc133930 2023-09-30T22:26:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.546+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac009a70 con 0x7ff8bc133930 2023-09-30T22:26:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.546+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003a30 con 0x7ff8bc133930 2023-09-30T22:26:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.546+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc0a0410 con 0x7ff8bc133930 2023-09-30T22:26:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.547+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc0a1800 con 0x7ff8bc133930 2023-09-30T22:26:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.547+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8b4003b90 con 0x7ff8bc133930 2023-09-30T22:26:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.548+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff8b403f0f0 con 0x7ff8bc133930 2023-09-30T22:26:19.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.556+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8b4003e40 con 0x7ff8bc133930 2023-09-30T22:26:19.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.723+0000 7feece252700 1 -- 172.21.15.32:0/965168786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feec80a28e0 con 0x7feec80a1340 2023-09-30T22:26:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.724+0000 7feeaffff700 1 -- 172.21.15.32:0/965168786 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feebc011360 con 0x7feec80a1340 2023-09-30T22:26:19.724 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 -- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 msgr2=0x7feec8122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 0x7feec8122cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c520 tx=0x7feebc00c830 comp rx=0 tx=0).stop 2023-09-30T22:26:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 -- 172.21.15.32:0/965168786 shutdown_connections 2023-09-30T22:26:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feec8134170 0x7feec8123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feec80a0a30 0x7feec8122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 --2- 172.21.15.32:0/965168786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feec80a1340 0x7feec8122cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 -- 172.21.15.32:0/965168786 >> 172.21.15.32:0/965168786 conn(0x7feec80a6550 msgr2=0x7feec809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 -- 172.21.15.32:0/965168786 shutdown_connections 2023-09-30T22:26:19.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.726+0000 7feece252700 1 -- 172.21.15.32:0/965168786 wait complete. 2023-09-30T22:26:19.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.762+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8bc0747f0 con 0x7ff8bc133930 2023-09-30T22:26:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.763+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2943489553 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8b4017090 con 0x7ff8bc133930 2023-09-30T22:26:19.763 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 msgr2=0x7ff8bc1c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 0x7ff8bc1c8970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c4d0 tx=0x7ff8b400c7e0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 shutdown_connections 2023-09-30T22:26:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8bc133930 0x7ff8bc1c8970 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8bc09d630 0x7ff8bc1c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 --2- 172.21.15.32:0/2943489553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8bc0a0a30 0x7ff8bc1c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 >> 172.21.15.32:0/2943489553 conn(0x7ff8bc0a3050 msgr2=0x7ff8bc1bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 shutdown_connections 2023-09-30T22:26:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.765+0000 7ff8c1bab700 1 -- 172.21.15.32:0/2943489553 wait complete. 2023-09-30T22:26:19.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:19.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:19.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:19.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 Processor -- start 2023-09-30T22:26:19.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 -- start start 2023-09-30T22:26:19.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009e850 0x7f2be009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 0x7f2be009f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2be0133b80 0x7f2be0134020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be006bec0 con 0x7f2be0133b80 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00a0030 con 0x7f2be009f160 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2be56ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2be006caa0 con 0x7f2be009e850 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2bde7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 0x7f2be009f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2bdeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009e850 0x7f2be009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2bdf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2be0133b80 0x7f2be0134020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.960+0000 7f2bde7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 0x7f2be009f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37908/0 (socket says 172.21.15.32:37908) 2023-09-30T22:26:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bde7fc700 1 -- 172.21.15.32:0/217098217 learned_addr learned my addr 172.21.15.32:0/217098217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdeffd700 1 -- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 msgr2=0x7f2be009f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdeffd700 1 --2- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 0x7f2be009f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdeffd700 1 -- 172.21.15.32:0/217098217 >> v2:172.21.15.32:3300/0 conn(0x7f2be0133b80 msgr2=0x7f2be0134020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:19.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdeffd700 1 --2- 172.21.15.32:0/217098217 >> v2:172.21.15.32:3300/0 conn(0x7f2be0133b80 0x7f2be0134020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdeffd700 1 -- 172.21.15.32:0/217098217 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00a0830 con 0x7f2be009e850 2023-09-30T22:26:19.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bde7fc700 1 --2- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 0x7f2be009f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdf7fe700 1 --2- 172.21.15.32:0/217098217 >> v2:172.21.15.32:3300/0 conn(0x7f2be0133b80 0x7f2be0134020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bdeffd700 1 --2- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009e850 0x7f2be009ec20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009840 tx=0x7f2bd0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b21fc905c110f1a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:19.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bddffb700 1 -- 172.21.15.32:0/217098217 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0011830 con 0x7f2be009e850 2023-09-30T22:26:19.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bddffb700 1 -- 172.21.15.32:0/217098217 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0011990 con 0x7f2be009e850 2023-09-30T22:26:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.961+0000 7f2bddffb700 1 -- 172.21.15.32:0/217098217 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0011d20 con 0x7f2be009e850 2023-09-30T22:26:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 -- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009e850 msgr2=0x7f2be009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 --2- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009e850 0x7f2be009ec20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009840 tx=0x7f2bd0009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 -- 172.21.15.32:0/217098217 shutdown_connections 2023-09-30T22:26:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 --2- 172.21.15.32:0/217098217 >> v2:172.21.15.32:3300/0 conn(0x7f2be0133b80 0x7f2be0134020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 --2- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009e850 0x7f2be009ec20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 --2- 172.21.15.32:0/217098217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009f160 0x7f2be009f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 -- 172.21.15.32:0/217098217 >> 172.21.15.32:0/217098217 conn(0x7f2be00a3f70 msgr2=0x7f2be00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 -- 172.21.15.32:0/217098217 shutdown_connections 2023-09-30T22:26:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.962+0000 7f2be56ca700 1 -- 172.21.15.32:0/217098217 wait complete. 2023-09-30T22:26:19.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.963+0000 7f2be56ca700 1 Processor -- start 2023-09-30T22:26:19.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.963+0000 7f2be56ca700 1 -- start start 2023-09-30T22:26:19.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2be56ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 0x7f2be0120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2be56ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009f160 0x7f2be0120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2be56ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2be0133b80 0x7f2be0121080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:19.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2be56ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00a1430 con 0x7f2be0133b80 2023-09-30T22:26:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2be56ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be0124120 con 0x7f2be009e850 2023-09-30T22:26:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2be56ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2be006ccc0 con 0x7f2be009f160 2023-09-30T22:26:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bdeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 0x7f2be0120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009f160 0x7f2be0120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bdeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 0x7f2be0120600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37910/0 (socket says 172.21.15.32:37910) 2023-09-30T22:26:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bdeffd700 1 -- 172.21.15.32:0/3587629805 learned_addr learned my addr 172.21.15.32:0/3587629805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bdf7fe700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2be0133b80 0x7f2be0121080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 -- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 msgr2=0x7f2be0120600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 0x7f2be0120600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 -- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2be0133b80 msgr2=0x7f2be0121080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2be0133b80 0x7f2be0121080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 -- 172.21.15.32:0/3587629805 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4009550 con 0x7f2be009f160 2023-09-30T22:26:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bdeffd700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 0x7f2be0120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bdf7fe700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2be0133b80 0x7f2be0121080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.964+0000 7f2bde7fc700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009f160 0x7f2be0120b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2be00a4fc0 tx=0x7f2bd400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:19.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.965+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400eec0 con 0x7f2be009f160 2023-09-30T22:26:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.965+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4004240 con 0x7f2be009f160 2023-09-30T22:26:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.965+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd8009a70 con 0x7f2be009f160 2023-09-30T22:26:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.965+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40045b0 con 0x7f2be009f160 2023-09-30T22:26:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.965+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd00094a0 con 0x7f2be009f160 2023-09-30T22:26:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.966+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2bd40036a0 con 0x7f2be009f160 2023-09-30T22:26:19.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.966+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2bd403fc60 con 0x7f2be009f160 2023-09-30T22:26:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.966+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be009caf0 con 0x7f2be009f160 2023-09-30T22:26:19.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:19.971+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2bd4024350 con 0x7f2be009f160 2023-09-30T22:26:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 Processor -- start 2023-09-30T22:26:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 -- start start 2023-09-30T22:26:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fa009e850 0x7f8fa009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0133b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa006bec0 con 0x7f8fa009f550 2023-09-30T22:26:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa006bce0 con 0x7f8fa009e850 2023-09-30T22:26:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.002+0000 7f8fa6b3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa006caa0 con 0x7f8fa009f160 2023-09-30T22:26:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0133b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56092/0 (socket says 172.21.15.32:56092) 2023-09-30T22:26:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 -- 172.21.15.32:0/721971893 learned_addr learned my addr 172.21.15.32:0/721971893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8fa48d7700 1 --2- 172.21.15.32:0/721971893 >> v2:172.21.15.32:3300/0 conn(0x7f8fa009e850 0x7f8fa009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 -- 172.21.15.32:0/721971893 >> v2:172.21.15.32:3300/0 conn(0x7f8fa009e850 msgr2=0x7f8fa009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9f7fe700 1 -- 172.21.15.32:0/721971893 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235570651 0 0) 0x7f8fa006bec0 con 0x7f8fa009f550 2023-09-30T22:26:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 --2- 172.21.15.32:0/721971893 >> v2:172.21.15.32:3300/0 conn(0x7f8fa009e850 0x7f8fa009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 -- 172.21.15.32:0/721971893 >> v1:172.21.15.32:6789/0 conn(0x7f8fa009f550 legacy=0x7f8fa01340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 -- 172.21.15.32:0/721971893 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00a00a0 con 0x7f8fa009f160 2023-09-30T22:26:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9ffff700 1 --2- 172.21.15.32:0/721971893 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0133b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f940098f0 tx=0x7f8f94009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f790ba869f4c8e74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9f7fe700 1 -- 172.21.15.32:0/721971893 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011830 con 0x7f8fa009f160 2023-09-30T22:26:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9f7fe700 1 -- 172.21.15.32:0/721971893 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94011990 con 0x7f8fa009f160 2023-09-30T22:26:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.003+0000 7f8f9f7fe700 1 -- 172.21.15.32:0/721971893 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011d20 con 0x7f8fa009f160 2023-09-30T22:26:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/721971893 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 msgr2=0x7f8fa0133b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/721971893 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0133b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f940098f0 tx=0x7f8f94009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/721971893 shutdown_connections 2023-09-30T22:26:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/721971893 >> v2:172.21.15.32:3300/0 conn(0x7f8fa009e850 0x7f8fa009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/721971893 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0133b80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/721971893 >> 172.21.15.32:0/721971893 conn(0x7f8fa00a3f70 msgr2=0x7f8fa00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/721971893 shutdown_connections 2023-09-30T22:26:20.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.004+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/721971893 wait complete. 2023-09-30T22:26:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.005+0000 7f8fa6b3b700 1 Processor -- start 2023-09-30T22:26:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.005+0000 7f8fa6b3b700 1 -- start start 2023-09-30T22:26:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa6b3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fa009e850 0x7f8fa01226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa6b3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa6b3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 0x7f8fa0123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa6b3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa0126200 con 0x7f8fa009f550 2023-09-30T22:26:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa6b3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa0124390 con 0x7f8fa009e850 2023-09-30T22:26:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa6b3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa006fa80 con 0x7f8fa009f160 2023-09-30T22:26:20.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 0x7f8fa0123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 0x7f8fa0123160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38718/0 (socket says 172.21.15.32:38718) 2023-09-30T22:26:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa48d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fa009e850 0x7f8fa01226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 -- 172.21.15.32:0/2202405880 learned_addr learned my addr 172.21.15.32:0/2202405880 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:20.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 -- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 msgr2=0x7f8fa0122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0122c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 -- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fa009e850 msgr2=0x7f8fa01226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fa009e850 0x7f8fa01226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 -- 172.21.15.32:0/2202405880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94009550 con 0x7f8fa009f550 2023-09-30T22:26:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8f9ffff700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.006+0000 7f8fa50d8700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 0x7f8fa0123160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8f98001560 tx=0x7f8f9800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.007+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98003e30 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.007+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f98005070 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.007+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa006ccc0 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.007+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f980053e0 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.007+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa0126990 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.008+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f98005540 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.008+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8f9803ec30 con 0x7f8fa009f550 2023-09-30T22:26:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.008+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa0125e40 con 0x7f8fa009f550 2023-09-30T22:26:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.013+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f98017070 con 0x7f8fa009f550 2023-09-30T22:26:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.178+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2be006c430 con 0x7f2be009f160 2023-09-30T22:26:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.178+0000 7f2bc77fe700 1 -- 172.21.15.32:0/3587629805 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bd402a030 con 0x7f2be009f160 2023-09-30T22:26:20.179 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.180+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009f160 msgr2=0x7f2be0120b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.180+0000 7f2be56ca700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009f160 0x7f2be0120b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2be00a4fc0 tx=0x7f2bd400ec80 comp rx=0 tx=0).stop 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 shutdown_connections 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2be0133b80 0x7f2be0121080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2be009e850 0x7f2be0120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 --2- 172.21.15.32:0/3587629805 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2be009f160 0x7f2be0120b40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 >> 172.21.15.32:0/3587629805 conn(0x7f2be00a3f70 msgr2=0x7f2be01bb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 shutdown_connections 2023-09-30T22:26:20.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.181+0000 7f2be56ca700 1 -- 172.21.15.32:0/3587629805 wait complete. 2023-09-30T22:26:20.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:20.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.221+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8fa00747f0 con 0x7f8fa009f550 2023-09-30T22:26:20.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.221+0000 7f8f9d7fa700 1 -- 172.21.15.32:0/2202405880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f98024020 con 0x7f8fa009f550 2023-09-30T22:26:20.221 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 msgr2=0x7f8fa0123160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 0x7f8fa0123160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8f98001560 tx=0x7f8f9800cb50 comp rx=0 tx=0).stop 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 shutdown_connections 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fa009f550 0x7f8fa0123160 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fa009e850 0x7f8fa01226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 --2- 172.21.15.32:0/2202405880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fa009f160 0x7f8fa0122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 >> 172.21.15.32:0/2202405880 conn(0x7f8fa00a3f70 msgr2=0x7f8fa009d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:20.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 shutdown_connections 2023-09-30T22:26:20.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.224+0000 7f8fa6b3b700 1 -- 172.21.15.32:0/2202405880 wait complete. 2023-09-30T22:26:20.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:20.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:20.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.889+0000 7f29f7537700 1 Processor -- start 2023-09-30T22:26:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f7537700 1 -- start start 2023-09-30T22:26:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f7537700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f7537700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f0136fe0 0x7f29f01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f7537700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f0070940 con 0x7f29f009e930 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f7537700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f006fd60 con 0x7f29f009f410 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f7537700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29f006fa80 con 0x7f29f0136fe0 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f5ad4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f0136fe0 0x7f29f01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f52d3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29f009e930 0x7f29f009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49910/0 (socket says 172.21.15.32:49910) 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.890+0000 7f29f52d3700 1 -- 172.21.15.32:0/3978849335 learned_addr learned my addr 172.21.15.32:0/3978849335 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:20.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f4ad2700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f5ad4700 1 -- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 msgr2=0x7f29f0136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f5ad4700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f0136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f5ad4700 1 -- 172.21.15.32:0/3978849335 >> v1:172.21.15.32:6789/0 conn(0x7f29f009e930 legacy=0x7f29f009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29e7fff700 1 -- 172.21.15.32:0/3978849335 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318951209 0 0) 0x7f29f0070940 con 0x7f29f009e930 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f5ad4700 1 -- 172.21.15.32:0/3978849335 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f009fb60 con 0x7f29f0136fe0 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f4ad2700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29f5ad4700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f0136fe0 0x7f29f01393c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009dc0 tx=0x7f29e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e142e0685650f12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29e7fff700 1 -- 172.21.15.32:0/3978849335 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e800e070 con 0x7f29f0136fe0 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29e7fff700 1 -- 172.21.15.32:0/3978849335 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e800bb20 con 0x7f29f0136fe0 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.891+0000 7f29e7fff700 1 -- 172.21.15.32:0/3978849335 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8012430 con 0x7f29f0136fe0 2023-09-30T22:26:20.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 -- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f0136fe0 msgr2=0x7f29f01393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f0136fe0 0x7f29f01393c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009dc0 tx=0x7f29e800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 -- 172.21.15.32:0/3978849335 shutdown_connections 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f0136fe0 0x7f29f01393c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 --2- 172.21.15.32:0/3978849335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 -- 172.21.15.32:0/3978849335 >> 172.21.15.32:0/3978849335 conn(0x7f29f00a4210 msgr2=0x7f29f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 -- 172.21.15.32:0/3978849335 shutdown_connections 2023-09-30T22:26:20.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.892+0000 7f29f7537700 1 -- 172.21.15.32:0/3978849335 wait complete. 2023-09-30T22:26:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.893+0000 7f29f7537700 1 Processor -- start 2023-09-30T22:26:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.893+0000 7f29f7537700 1 -- start start 2023-09-30T22:26:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f7537700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f009e930 0x7f29f01c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f7537700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f01c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f7537700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0136fe0 0x7f29f006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f7537700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01345f0 con 0x7f29f0136fe0 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f7537700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f006d640 con 0x7f29f009f410 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f7537700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0122ae0 con 0x7f29f009e930 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f4ad2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f01c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0136fe0 0x7f29f006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f52d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f009e930 0x7f29f01c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f4ad2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f01c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42396/0 (socket says 172.21.15.32:42396) 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f4ad2700 1 -- 172.21.15.32:0/867774237 learned_addr learned my addr 172.21.15.32:0/867774237 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 -- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f009e930 msgr2=0x7f29f01c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f009e930 0x7f29f01c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 -- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 msgr2=0x7f29f01c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f01c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 -- 172.21.15.32:0/867774237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00094a0 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f4ad2700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f01c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.894+0000 7f29f5ad4700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0136fe0 0x7f29f006b8b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f29e8016000 tx=0x7f29e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.895+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8004840 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.895+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8020070 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.895+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e8009a70 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.895+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e80276c0 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.895+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f01c8fa0 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.896+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29e8012430 con 0x7f29f0136fe0 2023-09-30T22:26:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.896+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f29e8022070 con 0x7f29f0136fe0 2023-09-30T22:26:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.896+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f00a3350 con 0x7f29f0136fe0 2023-09-30T22:26:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.901+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29e801b400 con 0x7f29f0136fe0 2023-09-30T22:26:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.928+0000 7f635540d700 1 Processor -- start 2023-09-30T22:26:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.928+0000 7f635540d700 1 -- start start 2023-09-30T22:26:20.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f635540d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 0x7f635009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f635540d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 0x7f6350131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f635540d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635009b560 con 0x7f6350131cd0 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f635540d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63500a0040 con 0x7f635009e850 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f635540d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63500a02e0 con 0x7f635009f160 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 0x7f6350131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 0x7f635009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 0x7f6350131790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42412/0 (socket says 172.21.15.32:42412) 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 0x7f635009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58704/0 (socket says 172.21.15.32:58704) 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6350131cd0 0x7f6350138bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49922/0 (socket says 172.21.15.32:49922) 2023-09-30T22:26:20.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 -- 172.21.15.32:0/1415185558 learned_addr learned my addr 172.21.15.32:0/1415185558 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634dffb700 1 -- 172.21.15.32:0/1415185558 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875703508 0 0) 0x7f635009b560 con 0x7f6350131cd0 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 -- 172.21.15.32:0/1415185558 >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 msgr2=0x7f635009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 --2- 172.21.15.32:0/1415185558 >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 0x7f635009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 -- 172.21.15.32:0/1415185558 >> v1:172.21.15.32:6789/0 conn(0x7f6350131cd0 legacy=0x7f6350138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634e7fc700 1 -- 172.21.15.32:0/1415185558 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635009fab0 con 0x7f635009f160 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.929+0000 7f634effd700 1 --2- 172.21.15.32:0/1415185558 >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 0x7f635009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f634e7fc700 1 --2- 172.21.15.32:0/1415185558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 0x7f6350131790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63440098f0 tx=0x7f6344009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9f833ac44695ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f634dffb700 1 -- 172.21.15.32:0/1415185558 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344011830 con 0x7f635009f160 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f634dffb700 1 -- 172.21.15.32:0/1415185558 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6344011990 con 0x7f635009f160 2023-09-30T22:26:20.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f634dffb700 1 -- 172.21.15.32:0/1415185558 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344011d20 con 0x7f635009f160 2023-09-30T22:26:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 -- 172.21.15.32:0/1415185558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 msgr2=0x7f6350131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 --2- 172.21.15.32:0/1415185558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 0x7f6350131790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63440098f0 tx=0x7f6344009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 -- 172.21.15.32:0/1415185558 shutdown_connections 2023-09-30T22:26:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 --2- 172.21.15.32:0/1415185558 >> v2:172.21.15.32:3300/0 conn(0x7f635009e850 0x7f635009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 --2- 172.21.15.32:0/1415185558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009f160 0x7f6350131790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 -- 172.21.15.32:0/1415185558 >> 172.21.15.32:0/1415185558 conn(0x7f63500a3f70 msgr2=0x7f63500a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.930+0000 7f635540d700 1 -- 172.21.15.32:0/1415185558 shutdown_connections 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.931+0000 7f635540d700 1 -- 172.21.15.32:0/1415185558 wait complete. 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 Processor -- start 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 -- start start 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009e850 0x7f63501c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009f160 0x7f63501c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6350131cd0 0x7f63501c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635011e6f0 con 0x7f635009f160 2023-09-30T22:26:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635011fee0 con 0x7f635009e850 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f635540d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f635009d3c0 con 0x7f6350131cd0 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f634e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009f160 0x7f63501c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f634effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009e850 0x7f63501c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f634e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009f160 0x7f63501c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58718/0 (socket says 172.21.15.32:58718) 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f634e7fc700 1 -- 172.21.15.32:0/2639494974 learned_addr learned my addr 172.21.15.32:0/2639494974 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.932+0000 7f634f7fe700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6350131cd0 0x7f63501c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f634effd700 1 -- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6350131cd0 msgr2=0x7f63501c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f634effd700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6350131cd0 0x7f63501c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f634effd700 1 -- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009f160 msgr2=0x7f63501c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:20.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f634effd700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009f160 0x7f63501c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f634effd700 1 -- 172.21.15.32:0/2639494974 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400094a0 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f634effd700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009e850 0x7f63501c6100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f634000e900 tx=0x7f634000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634000ee50 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634000c850 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6344009550 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634000cbe0 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.933+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6350120c10 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.934+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f634000cd40 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.934+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f634003fa90 con 0x7f635009e850 2023-09-30T22:26:20.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.934+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63500747f0 con 0x7f635009e850 2023-09-30T22:26:20.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:20.939+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f634001b3b0 con 0x7f635009e850 2023-09-30T22:26:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.108+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29f0071d20 con 0x7f29f0136fe0 2023-09-30T22:26:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.108+0000 7f29e5ffb700 1 -- 172.21.15.32:0/867774237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29e802c020 con 0x7f29f0136fe0 2023-09-30T22:26:21.109 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0136fe0 msgr2=0x7f29f006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0136fe0 0x7f29f006b8b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f29e8016000 tx=0x7f29e8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 shutdown_connections 2023-09-30T22:26:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29f0136fe0 0x7f29f006b8b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29f009e930 0x7f29f01c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 --2- 172.21.15.32:0/867774237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29f009f410 0x7f29f01c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 >> 172.21.15.32:0/867774237 conn(0x7f29f00a4210 msgr2=0x7f29f00a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 shutdown_connections 2023-09-30T22:26:21.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.111+0000 7f29f7537700 1 -- 172.21.15.32:0/867774237 wait complete. 2023-09-30T22:26:21.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.148+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f635009b780 con 0x7f635009e850 2023-09-30T22:26:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.149+0000 7f63377fe700 1 -- 172.21.15.32:0/2639494974 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6340021020 con 0x7f635009e850 2023-09-30T22:26:21.150 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009e850 msgr2=0x7f63501c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009e850 0x7f63501c6100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f634000e900 tx=0x7f634000ec10 comp rx=0 tx=0).stop 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 shutdown_connections 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6350131cd0 0x7f63501c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f635009e850 0x7f63501c6100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 --2- 172.21.15.32:0/2639494974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f635009f160 0x7f63501c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 >> 172.21.15.32:0/2639494974 conn(0x7f63500a3f70 msgr2=0x7f63501b9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 shutdown_connections 2023-09-30T22:26:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.151+0000 7f635540d700 1 -- 172.21.15.32:0/2639494974 wait complete. 2023-09-30T22:26:21.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:21.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:21.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:21.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.350+0000 7fec8579b700 1 Processor -- start 2023-09-30T22:26:21.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.350+0000 7fec8579b700 1 -- start start 2023-09-30T22:26:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec8579b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec8009d630 0x7fec800a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec8579b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec8013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec8579b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800a2a70 con 0x7fec800a0a30 2023-09-30T22:26:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec8579b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec800a01d0 con 0x7fec8009d630 2023-09-30T22:26:21.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec8579b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec8009e2b0 con 0x7fec80138d80 2023-09-30T22:26:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec8009d630 0x7fec800a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec8013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec800a0a30 0x7fec800a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49932/0 (socket says 172.21.15.32:49932) 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7effd700 1 -- 172.21.15.32:0/1900754983 learned_addr learned my addr 172.21.15.32:0/1900754983 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7e7fc700 1 -- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 msgr2=0x7fec8013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7e7fc700 1 --2- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec8013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7e7fc700 1 -- 172.21.15.32:0/1900754983 >> v1:172.21.15.32:6789/0 conn(0x7fec800a0a30 legacy=0x7fec800a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7dffb700 1 -- 172.21.15.32:0/1900754983 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887665357 0 0) 0x7fec800a2a70 con 0x7fec800a0a30 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.351+0000 7fec7e7fc700 1 -- 172.21.15.32:0/1900754983 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8009dc50 con 0x7fec8009d630 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec7e7fc700 1 --2- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8009d630 0x7fec800a1510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fec68009840 tx=0x7fec68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90fdf7a1df61a86 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec7dffb700 1 -- 172.21.15.32:0/1900754983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68011830 con 0x7fec8009d630 2023-09-30T22:26:21.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec7dffb700 1 -- 172.21.15.32:0/1900754983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68011990 con 0x7fec8009d630 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec7dffb700 1 -- 172.21.15.32:0/1900754983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68011d20 con 0x7fec8009d630 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec8579b700 1 -- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8009d630 msgr2=0x7fec800a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec8579b700 1 --2- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8009d630 0x7fec800a1510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fec68009840 tx=0x7fec68009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec8579b700 1 -- 172.21.15.32:0/1900754983 shutdown_connections 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec8579b700 1 --2- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec8013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.352+0000 7fec8579b700 1 --2- 172.21.15.32:0/1900754983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec8009d630 0x7fec800a1510 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.353+0000 7fec8579b700 1 -- 172.21.15.32:0/1900754983 >> 172.21.15.32:0/1900754983 conn(0x7fec800a3050 msgr2=0x7fec800a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.353+0000 7fec8579b700 1 -- 172.21.15.32:0/1900754983 shutdown_connections 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.353+0000 7fec8579b700 1 -- 172.21.15.32:0/1900754983 wait complete. 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 Processor -- start 2023-09-30T22:26:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 -- start start 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 0x7fec801337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 0x7fec80133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec80134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80136270 con 0x7fec800a0a30 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec80136f90 con 0x7fec8009d630 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec8579b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec800a4b10 con 0x7fec80138d80 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec7effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 0x7fec801337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec7e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 0x7fec80133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec7effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 0x7fec801337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42430/0 (socket says 172.21.15.32:42430) 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec7e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 0x7fec80133ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58740/0 (socket says 172.21.15.32:58740) 2023-09-30T22:26:21.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.354+0000 7fec7effd700 1 -- 172.21.15.32:0/68443226 learned_addr learned my addr 172.21.15.32:0/68443226 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7f7fe700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec80134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7e7fc700 1 -- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 msgr2=0x7fec80134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7e7fc700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec80134220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7e7fc700 1 -- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 msgr2=0x7fec801337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7e7fc700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 0x7fec801337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7e7fc700 1 -- 172.21.15.32:0/68443226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680094a0 con 0x7fec800a0a30 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7effd700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 0x7fec801337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec7e7fc700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 0x7fec80133ce0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec68016000 tx=0x7fec68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68004480 con 0x7fec800a0a30 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6801f070 con 0x7fec800a0a30 2023-09-30T22:26:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec800a0410 con 0x7fec800a0a30 2023-09-30T22:26:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680038a0 con 0x7fec800a0a30 2023-09-30T22:26:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.355+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec80137270 con 0x7fec800a0a30 2023-09-30T22:26:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.356+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec68021030 con 0x7fec800a0a30 2023-09-30T22:26:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.356+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fec680408a0 con 0x7fec800a0a30 2023-09-30T22:26:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.357+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec80136dd0 con 0x7fec800a0a30 2023-09-30T22:26:21.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.362+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec68003a00 con 0x7fec800a0a30 2023-09-30T22:26:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.402+0000 7f507c447700 1 Processor -- start 2023-09-30T22:26:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.402+0000 7f507c447700 1 -- start start 2023-09-30T22:26:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.403+0000 7f507c447700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 0x7f50740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.403+0000 7f507c447700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a1340 0x7f5074133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.403+0000 7f507c447700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 0x7f5074134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.403+0000 7f507c447700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507409edd0 con 0x7f5074133fa0 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.403+0000 7f507c447700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507409e1f0 con 0x7f50740a0a30 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.403+0000 7f507c447700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507409dc50 con 0x7f50740a1340 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a9e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 0x7f5074134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 0x7f50740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a9e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 0x7f5074134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58744/0 (socket says 172.21.15.32:58744) 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f50799e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a1340 0x7f5074133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a9e4700 1 -- 172.21.15.32:0/3342721639 learned_addr learned my addr 172.21.15.32:0/3342721639 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 0x7f50740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42432/0 (socket says 172.21.15.32:42432) 2023-09-30T22:26:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 -- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a1340 msgr2=0x7f5074133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 --2- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a1340 0x7f5074133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 -- 172.21.15.32:0/3342721639 >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 msgr2=0x7f5074134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 --2- 172.21.15.32:0/3342721639 >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 0x7f5074134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 -- 172.21.15.32:0/3342721639 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740a1fe0 con 0x7f50740a0a30 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a9e4700 1 --2- 172.21.15.32:0/3342721639 >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 0x7f5074134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f507a1e3700 1 --2- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 0x7f50740a0e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5070009840 tx=0x7f5070009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb7e2fcce4f3197b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f50791e1700 1 -- 172.21.15.32:0/3342721639 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070011830 con 0x7f50740a0a30 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f50791e1700 1 -- 172.21.15.32:0/3342721639 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070011990 con 0x7f50740a0a30 2023-09-30T22:26:21.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.404+0000 7f50791e1700 1 -- 172.21.15.32:0/3342721639 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070011d20 con 0x7f50740a0a30 2023-09-30T22:26:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 -- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 msgr2=0x7f50740a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 --2- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 0x7f50740a0e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5070009840 tx=0x7f5070009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 -- 172.21.15.32:0/3342721639 shutdown_connections 2023-09-30T22:26:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 --2- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a0a30 0x7f50740a0e00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 --2- 172.21.15.32:0/3342721639 >> v2:172.21.15.32:3300/0 conn(0x7f5074133fa0 0x7f5074134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 --2- 172.21.15.32:0/3342721639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a1340 0x7f5074133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 -- 172.21.15.32:0/3342721639 >> 172.21.15.32:0/3342721639 conn(0x7f50740a6550 msgr2=0x7f50740a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 -- 172.21.15.32:0/3342721639 shutdown_connections 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.405+0000 7f507c447700 1 -- 172.21.15.32:0/3342721639 wait complete. 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.406+0000 7f507c447700 1 Processor -- start 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.406+0000 7f507c447700 1 -- start start 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507c447700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a0a30 0x7f5074120160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507c447700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 0x7f50741206a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507c447700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5074133fa0 0x7f5074120be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507c447700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074121d70 con 0x7f5074133fa0 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507c447700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5074122fc0 con 0x7f50740a1340 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507c447700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5074122360 con 0x7f50740a0a30 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 0x7f50741206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 0x7f50741206a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42442/0 (socket says 172.21.15.32:42442) 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 -- 172.21.15.32:0/1097949578 learned_addr learned my addr 172.21.15.32:0/1097949578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507a9e4700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5074133fa0 0x7f5074120be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507a1e3700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a0a30 0x7f5074120160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 -- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a0a30 msgr2=0x7f5074120160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a0a30 0x7f5074120160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 -- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5074133fa0 msgr2=0x7f5074120be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5074133fa0 0x7f5074120be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f50799e2700 1 -- 172.21.15.32:0/1097949578 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c009b30 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.407+0000 7f507a9e4700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5074133fa0 0x7f5074120be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f50799e2700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 0x7f50741206a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9b0 tx=0x7f506400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064009710 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506400ee50 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700094a0 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506400ca50 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50740a0220 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.408+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f506400cbb0 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.409+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f506403fa60 con 0x7f50740a1340 2023-09-30T22:26:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.409+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5074074ea0 con 0x7f50740a1340 2023-09-30T22:26:21.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.414+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5064021080 con 0x7f50740a1340 2023-09-30T22:26:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.583+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fec800747f0 con 0x7fec800a0a30 2023-09-30T22:26:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.583+0000 7fec777fe700 1 -- 172.21.15.32:0/68443226 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec68015020 con 0x7fec800a0a30 2023-09-30T22:26:21.583 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 msgr2=0x7fec80133ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 0x7fec80133ce0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fec68016000 tx=0x7fec68004750 comp rx=0 tx=0).stop 2023-09-30T22:26:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 shutdown_connections 2023-09-30T22:26:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec80138d80 0x7fec80134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec8009d630 0x7fec801337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 --2- 172.21.15.32:0/68443226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec800a0a30 0x7fec80133ce0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 >> 172.21.15.32:0/68443226 conn(0x7fec800a3050 msgr2=0x7fec8009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 shutdown_connections 2023-09-30T22:26:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.586+0000 7fec8579b700 1 -- 172.21.15.32:0/68443226 wait complete. 2023-09-30T22:26:21.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.622+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f50740a28e0 con 0x7f50740a1340 2023-09-30T22:26:21.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.623+0000 7f506affd700 1 -- 172.21.15.32:0/1097949578 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5064017020 con 0x7f50740a1340 2023-09-30T22:26:21.623 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 msgr2=0x7f50741206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 0x7f50741206a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f506400e9b0 tx=0x7f506400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 shutdown_connections 2023-09-30T22:26:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5074133fa0 0x7f5074120be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50740a0a30 0x7f5074120160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 --2- 172.21.15.32:0/1097949578 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50740a1340 0x7f50741206a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 >> 172.21.15.32:0/1097949578 conn(0x7f50740a6550 msgr2=0x7f50741bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 shutdown_connections 2023-09-30T22:26:21.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:21.625+0000 7f507c447700 1 -- 172.21.15.32:0/1097949578 wait complete. 2023-09-30T22:26:21.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:22.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:22.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:22.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.272+0000 7fdd73850700 1 Processor -- start 2023-09-30T22:26:22.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 -- start start 2023-09-30T22:26:22.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd6c0a0a30 0x7fdd6c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 0x7fdd6c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c0a2a70 con 0x7fdd6c0a0a30 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0a01d0 con 0x7fdd6c09d630 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.273+0000 7fdd73850700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09e2b0 con 0x7fdd6c133930 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd70deb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd71ded700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 0x7fdd6c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd6c0a0a30 0x7fdd6c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd71ded700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 0x7fdd6c133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40890/0 (socket says 172.21.15.32:40890) 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd70deb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42446/0 (socket says 172.21.15.32:42446) 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd71ded700 1 -- 172.21.15.32:0/3890882638 learned_addr learned my addr 172.21.15.32:0/3890882638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 -- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 msgr2=0x7fdd6c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 0x7fdd6c133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 -- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 msgr2=0x7fdd6c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 -- 172.21.15.32:0/3890882638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c09dc50 con 0x7fdd6c0a0a30 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd71ded700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 0x7fdd6c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd70deb700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd715ec700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c0a0a30 0x7fdd6c0a0e00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009840 tx=0x7fdd5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b931bbdbfb84acbf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd6bfff700 1 -- 172.21.15.32:0/3890882638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011830 con 0x7fdd6c0a0a30 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd6bfff700 1 -- 172.21.15.32:0/3890882638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c011990 con 0x7fdd6c0a0a30 2023-09-30T22:26:22.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.274+0000 7fdd6bfff700 1 -- 172.21.15.32:0/3890882638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011d20 con 0x7fdd6c0a0a30 2023-09-30T22:26:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 -- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c0a0a30 msgr2=0x7fdd6c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c0a0a30 0x7fdd6c0a0e00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009840 tx=0x7fdd5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 -- 172.21.15.32:0/3890882638 shutdown_connections 2023-09-30T22:26:22.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c133930 0x7fdd6c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c0a0a30 0x7fdd6c0a0e00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 --2- 172.21.15.32:0/3890882638 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 -- 172.21.15.32:0/3890882638 >> 172.21.15.32:0/3890882638 conn(0x7fdd6c0a3050 msgr2=0x7fdd6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 -- 172.21.15.32:0/3890882638 shutdown_connections 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.275+0000 7fdd73850700 1 -- 172.21.15.32:0/3890882638 wait complete. 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.276+0000 7fdd73850700 1 Processor -- start 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.276+0000 7fdd73850700 1 -- start start 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd73850700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c11f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd73850700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c0a0a30 0x7fdd6c11fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd73850700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 0x7fdd6c1c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd73850700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c11f3b0 con 0x7fdd6c133930 2023-09-30T22:26:22.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd73850700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c1222f0 con 0x7fdd6c09d630 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd73850700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c122020 con 0x7fdd6c0a0a30 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd70deb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c0a0a30 0x7fdd6c11fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 0x7fdd6c1c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 0x7fdd6c1c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58770/0 (socket says 172.21.15.32:58770) 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd70deb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c0a0a30 0x7fdd6c11fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40896/0 (socket says 172.21.15.32:40896) 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 -- 172.21.15.32:0/2093734290 learned_addr learned my addr 172.21.15.32:0/2093734290 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd715ec700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c11f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 -- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c0a0a30 msgr2=0x7fdd6c11fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c0a0a30 0x7fdd6c11fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 -- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 msgr2=0x7fdd6c11f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c11f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.277+0000 7fdd71ded700 1 -- 172.21.15.32:0/2093734290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600094a0 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.278+0000 7fdd71ded700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 0x7fdd6c1c8200 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400efc0 tx=0x7fdd6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.278+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64018070 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.278+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6400f040 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.278+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64003c20 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.278+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd64009b30 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.278+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c0094a0 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.279+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdd6400c880 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.279+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdd6403f650 con 0x7fdd6c133930 2023-09-30T22:26:22.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.279+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c0747f0 con 0x7fdd6c133930 2023-09-30T22:26:22.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.284+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdd64014480 con 0x7fdd6c133930 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.322+0000 7f821b16c700 1 Processor -- start 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.322+0000 7f821b16c700 1 -- start start 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f821b16c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 0x7f821409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f821b16c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f821409f240 0x7f82141317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f821b16c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f650 0x7f8214131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f821b16c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8214070940 con 0x7f821409f240 2023-09-30T22:26:22.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f821b16c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821406fd60 con 0x7f821409f650 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f821b16c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f821406fa80 con 0x7f821409e930 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8218f08700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 0x7f821409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8213fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f821409f240 0x7f82141317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8218f08700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 0x7f821409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40904/0 (socket says 172.21.15.32:40904) 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8213fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f821409f240 0x7f82141317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58776/0 (socket says 172.21.15.32:58776) 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8218f08700 1 -- 172.21.15.32:0/4202667908 learned_addr learned my addr 172.21.15.32:0/4202667908 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8219709700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f650 0x7f8214131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.323+0000 7f8213fff700 1 -- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 msgr2=0x7f821409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f8213fff700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 0x7f821409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f8213fff700 1 -- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f650 msgr2=0x7f8214131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f8213fff700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f650 0x7f8214131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f8213fff700 1 -- 172.21.15.32:0/4202667908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821409fb60 con 0x7f821409f240 2023-09-30T22:26:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f8218f08700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 0x7f821409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f8213fff700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409f240 0x7f82141317f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8208009840 tx=0x7f8208009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4595516d8612a6d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f82137fe700 1 -- 172.21.15.32:0/4202667908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208011830 con 0x7f821409f240 2023-09-30T22:26:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f82137fe700 1 -- 172.21.15.32:0/4202667908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208011990 con 0x7f821409f240 2023-09-30T22:26:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.324+0000 7f82137fe700 1 -- 172.21.15.32:0/4202667908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208011d20 con 0x7f821409f240 2023-09-30T22:26:22.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 -- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409f240 msgr2=0x7f82141317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409f240 0x7f82141317f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8208009840 tx=0x7f8208009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:22.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 -- 172.21.15.32:0/4202667908 shutdown_connections 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f650 0x7f8214131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409e930 0x7f821409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 --2- 172.21.15.32:0/4202667908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409f240 0x7f82141317f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 -- 172.21.15.32:0/4202667908 >> 172.21.15.32:0/4202667908 conn(0x7f82140a4210 msgr2=0x7f82140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 -- 172.21.15.32:0/4202667908 shutdown_connections 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.325+0000 7f821b16c700 1 -- 172.21.15.32:0/4202667908 wait complete. 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.326+0000 7f821b16c700 1 Processor -- start 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.326+0000 7f821b16c700 1 -- start start 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f821b16c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 0x7f82141a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f821b16c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f240 0x7f82141a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f821b16c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 0x7f82141a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f821b16c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a7040 con 0x7f821409e930 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f821b16c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a6680 con 0x7f821409f240 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f821b16c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82141a5a50 con 0x7f821409f650 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f8218f08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 0x7f82141a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f8219709700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 0x7f82141a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f8219709700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 0x7f82141a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40912/0 (socket says 172.21.15.32:40912) 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f8218f08700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 0x7f82141a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58784/0 (socket says 172.21.15.32:58784) 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f8219709700 1 -- 172.21.15.32:0/3036174676 learned_addr learned my addr 172.21.15.32:0/3036174676 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.327+0000 7f8213fff700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f240 0x7f82141a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8219709700 1 -- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f240 msgr2=0x7f82141a30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8219709700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f240 0x7f82141a30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8219709700 1 -- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 msgr2=0x7f82141a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8219709700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 0x7f82141a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8219709700 1 -- 172.21.15.32:0/3036174676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009550 con 0x7f821409f650 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8218f08700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 0x7f82141a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f8219709700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 0x7f82141a35e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f820c009e30 tx=0x7f820c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c009ca0 con 0x7f821409f650 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c00f040 con 0x7f821409f650 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c014480 con 0x7f821409f650 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82080094a0 con 0x7f821409f650 2023-09-30T22:26:22.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.328+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82140a0190 con 0x7f821409f650 2023-09-30T22:26:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.329+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f820c0145e0 con 0x7f821409f650 2023-09-30T22:26:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.329+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f820c040220 con 0x7f821409f650 2023-09-30T22:26:22.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.330+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214071d20 con 0x7f821409f650 2023-09-30T22:26:22.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.335+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f820c007920 con 0x7f821409f650 2023-09-30T22:26:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.520+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdd6c1224f0 con 0x7fdd6c133930 2023-09-30T22:26:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.520+0000 7fdd69ffb700 1 -- 172.21.15.32:0/2093734290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd6c1224f0 con 0x7fdd6c133930 2023-09-30T22:26:22.521 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 msgr2=0x7fdd6c1c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 0x7fdd6c1c8200 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400efc0 tx=0x7fdd6400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 shutdown_connections 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd6c133930 0x7fdd6c1c8200 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd6c09d630 0x7fdd6c11f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 --2- 172.21.15.32:0/2093734290 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd6c0a0a30 0x7fdd6c11fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.522+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 >> 172.21.15.32:0/2093734290 conn(0x7fdd6c0a3050 msgr2=0x7fdd6c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.523+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 shutdown_connections 2023-09-30T22:26:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.523+0000 7fdd73850700 1 -- 172.21.15.32:0/2093734290 wait complete. 2023-09-30T22:26:22.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:22.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.549+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8214074ea0 con 0x7f821409f650 2023-09-30T22:26:22.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.549+0000 7f82117fa700 1 -- 172.21.15.32:0/3036174676 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f820c02c090 con 0x7f821409f650 2023-09-30T22:26:22.550 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:22.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.551+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 msgr2=0x7f82141a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.551+0000 7f821b16c700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 0x7f82141a35e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f820c009e30 tx=0x7f820c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 shutdown_connections 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f821409f650 0x7f82141a35e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f821409e930 0x7f82141a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 --2- 172.21.15.32:0/3036174676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f821409f240 0x7f82141a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 >> 172.21.15.32:0/3036174676 conn(0x7f82140a4210 msgr2=0x7f82141b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 shutdown_connections 2023-09-30T22:26:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.552+0000 7f821b16c700 1 -- 172.21.15.32:0/3036174676 wait complete. 2023-09-30T22:26:22.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:22.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:22.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:22.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.748+0000 7f6be80a6700 1 Processor -- start 2023-09-30T22:26:22.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 -- start start 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 0x7f6be00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 0x7f6be0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 0x7f6be0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be009edd0 con 0x7f6be00a0a30 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be009e1f0 con 0x7f6be0133fa0 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be80a6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6be009dc50 con 0x7f6be00a1340 2023-09-30T22:26:22.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be6643700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 0x7f6be0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be5e42700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 0x7f6be00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be5641700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 0x7f6be0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be5641700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 0x7f6be0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40928/0 (socket says 172.21.15.32:40928) 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be6643700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 0x7f6be0134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42474/0 (socket says 172.21.15.32:42474) 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be5e42700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 0x7f6be00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58792/0 (socket says 172.21.15.32:58792) 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.749+0000 7f6be5641700 1 -- 172.21.15.32:0/3711389062 learned_addr learned my addr 172.21.15.32:0/3711389062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be6643700 1 -- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 msgr2=0x7f6be0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be6643700 1 --2- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 0x7f6be0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be6643700 1 -- 172.21.15.32:0/3711389062 >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 msgr2=0x7f6be00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be6643700 1 --2- 172.21.15.32:0/3711389062 >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 0x7f6be00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be6643700 1 -- 172.21.15.32:0/3711389062 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be00a1fe0 con 0x7f6be0133fa0 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be5641700 1 --2- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 0x7f6be0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be5e42700 1 --2- 172.21.15.32:0/3711389062 >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 0x7f6be00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be6643700 1 --2- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 0x7f6be0134410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009dc0 tx=0x7f6bd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d03db43dc518d12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be4e40700 1 -- 172.21.15.32:0/3711389062 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800e070 con 0x7f6be0133fa0 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be4e40700 1 -- 172.21.15.32:0/3711389062 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd800bae0 con 0x7f6be0133fa0 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.750+0000 7f6be4e40700 1 -- 172.21.15.32:0/3711389062 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800be90 con 0x7f6be0133fa0 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 -- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 msgr2=0x7f6be0134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 0x7f6be0134410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009dc0 tx=0x7f6bd800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 -- 172.21.15.32:0/3711389062 shutdown_connections 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be0133fa0 0x7f6be0134410 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3711389062 >> v2:172.21.15.32:3300/0 conn(0x7f6be00a0a30 0x7f6be00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3711389062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a1340 0x7f6be0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 -- 172.21.15.32:0/3711389062 >> 172.21.15.32:0/3711389062 conn(0x7f6be00a6550 msgr2=0x7f6be00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 -- 172.21.15.32:0/3711389062 shutdown_connections 2023-09-30T22:26:22.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.751+0000 7f6be80a6700 1 -- 172.21.15.32:0/3711389062 wait complete. 2023-09-30T22:26:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.752+0000 7f6be80a6700 1 Processor -- start 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.752+0000 7f6be80a6700 1 -- start start 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.752+0000 7f6be80a6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a0a30 0x7f6be01200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.752+0000 7f6be80a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be00a1340 0x7f6be0120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be80a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 0x7f6be0120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be80a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0124530 con 0x7f6be0133fa0 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be80a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0121d00 con 0x7f6be00a1340 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be80a6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0123890 con 0x7f6be00a0a30 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be6643700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 0x7f6be0120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be6643700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 0x7f6be0120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58798/0 (socket says 172.21.15.32:58798) 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a0a30 0x7f6be01200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be6643700 1 -- 172.21.15.32:0/3852824841 learned_addr learned my addr 172.21.15.32:0/3852824841 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5641700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be00a1340 0x7f6be0120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 -- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be00a1340 msgr2=0x7f6be0120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be00a1340 0x7f6be0120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 -- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 msgr2=0x7f6be0120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 0x7f6be0120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 -- 172.21.15.32:0/3852824841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc0094a0 con 0x7f6be00a0a30 2023-09-30T22:26:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be6643700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 0x7f6be0120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be5e42700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a0a30 0x7f6be01200e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00e9e0 tx=0x7f6bdc00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00ee80 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00c850 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd0009550 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.753+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00cc00 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.754+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd8009a70 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.754+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bdc00cd60 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.754+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be00a0280 con 0x7f6be00a0a30 2023-09-30T22:26:22.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.755+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6bdc03fa90 con 0x7f6be00a0a30 2023-09-30T22:26:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.763+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bdc011d80 con 0x7f6be00a0a30 2023-09-30T22:26:22.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 Processor -- start 2023-09-30T22:26:22.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 -- start start 2023-09-30T22:26:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c0a0a30 0x7f568c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f568c138d80 0x7f568c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0a2a70 con 0x7f568c09d630 2023-09-30T22:26:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0a01d0 con 0x7f568c138d80 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.793+0000 7f5693376700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f568c09e2b0 con 0x7f568c0a0a30 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691913700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f568c138d80 0x7f568c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691112700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c0a0a30 0x7f568c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5690911700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f568c09d630 0x7f568c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49942/0 (socket says 172.21.15.32:49942) 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5690911700 1 -- 172.21.15.32:0/402789600 learned_addr learned my addr 172.21.15.32:0/402789600 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691112700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c0a0a30 0x7f568c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40946/0 (socket says 172.21.15.32:40946) 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691913700 1 -- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c0a0a30 msgr2=0x7f568c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691913700 1 --2- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c0a0a30 0x7f568c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691913700 1 -- 172.21.15.32:0/402789600 >> v1:172.21.15.32:6789/0 conn(0x7f568c09d630 legacy=0x7f568c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5683fff700 1 -- 172.21.15.32:0/402789600 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306806298 0 0) 0x7f568c0a2a70 con 0x7f568c09d630 2023-09-30T22:26:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691913700 1 -- 172.21.15.32:0/402789600 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c09dc50 con 0x7f568c138d80 2023-09-30T22:26:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5691913700 1 --2- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c13b160 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5684009d90 tx=0x7f568400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf18b755e2f8d211 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5683fff700 1 -- 172.21.15.32:0/402789600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400e070 con 0x7f568c138d80 2023-09-30T22:26:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5683fff700 1 -- 172.21.15.32:0/402789600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400bb20 con 0x7f568c138d80 2023-09-30T22:26:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.794+0000 7f5683fff700 1 -- 172.21.15.32:0/402789600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684012430 con 0x7f568c138d80 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 -- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 msgr2=0x7f568c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 --2- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c13b160 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5684009d90 tx=0x7f568400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 -- 172.21.15.32:0/402789600 shutdown_connections 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 --2- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c13b160 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 --2- 172.21.15.32:0/402789600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c0a0a30 0x7f568c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 -- 172.21.15.32:0/402789600 >> 172.21.15.32:0/402789600 conn(0x7f568c0a3050 msgr2=0x7f568c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 -- 172.21.15.32:0/402789600 shutdown_connections 2023-09-30T22:26:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.795+0000 7f5693376700 1 -- 172.21.15.32:0/402789600 wait complete. 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.796+0000 7f5693376700 1 Processor -- start 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 -- start start 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c09d630 0x7f568c1336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f568c0a0a30 0x7f568c133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c136130 con 0x7f568c138d80 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c136e50 con 0x7f568c0a0a30 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5693376700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0a4b90 con 0x7f568c09d630 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5690911700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f568c0a0a30 0x7f568c133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5691913700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5691913700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c134170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58812/0 (socket says 172.21.15.32:58812) 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5691913700 1 -- 172.21.15.32:0/1421503971 learned_addr learned my addr 172.21.15.32:0/1421503971 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.797+0000 7f5691112700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c09d630 0x7f568c1336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5690911700 1 -- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c09d630 msgr2=0x7f568c1336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5690911700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c09d630 0x7f568c1336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5690911700 1 -- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 msgr2=0x7f568c134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5690911700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5690911700 1 -- 172.21.15.32:0/1421503971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684009a70 con 0x7f568c0a0a30 2023-09-30T22:26:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5691913700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5691112700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c09d630 0x7f568c1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5690911700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f568c0a0a30 0x7f568c133c30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f568c06d980 tx=0x7f567c00c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c004c80 con 0x7f568c0a0a30 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c004de0 con 0x7f568c0a0a30 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0a0410 con 0x7f568c0a0a30 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c007310 con 0x7f568c0a0a30 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.798+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c1378c0 con 0x7f568c0a0a30 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.799+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f567c007470 con 0x7f568c0a0a30 2023-09-30T22:26:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.799+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f567c03f270 con 0x7f568c0a0a30 2023-09-30T22:26:22.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.800+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c136f90 con 0x7f568c0a0a30 2023-09-30T22:26:22.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.804+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f568c136f90 con 0x7f568c0a0a30 2023-09-30T22:26:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.989+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6be0074ea0 con 0x7f6be00a0a30 2023-09-30T22:26:22.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.990+0000 7f6bd67fc700 1 -- 172.21.15.32:0/3852824841 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bdc023300 con 0x7f6be00a0a30 2023-09-30T22:26:22.991 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a0a30 msgr2=0x7f6be01200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:22.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a0a30 0x7f6be01200e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00e9e0 tx=0x7f6bdc00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 shutdown_connections 2023-09-30T22:26:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6be0133fa0 0x7f6be0120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6be00a0a30 0x7f6be01200e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 --2- 172.21.15.32:0/3852824841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6be00a1340 0x7f6be0120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 >> 172.21.15.32:0/3852824841 conn(0x7f6be00a6550 msgr2=0x7f6be009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:22.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 shutdown_connections 2023-09-30T22:26:22.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:22.992+0000 7f6be80a6700 1 -- 172.21.15.32:0/3852824841 wait complete. 2023-09-30T22:26:23.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:23.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.029+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f568c136f90 con 0x7f568c0a0a30 2023-09-30T22:26:23.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.029+0000 7f5681ffb700 1 -- 172.21.15.32:0/1421503971 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f568c136f90 con 0x7f568c0a0a30 2023-09-30T22:26:23.031 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f568c0a0a30 msgr2=0x7f568c133c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f568c0a0a30 0x7f568c133c30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f568c06d980 tx=0x7f567c00c750 comp rx=0 tx=0).stop 2023-09-30T22:26:23.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 shutdown_connections 2023-09-30T22:26:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f568c138d80 0x7f568c134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f568c09d630 0x7f568c1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 --2- 172.21.15.32:0/1421503971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f568c0a0a30 0x7f568c133c30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 >> 172.21.15.32:0/1421503971 conn(0x7f568c0a3050 msgr2=0x7f568c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 shutdown_connections 2023-09-30T22:26:23.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.032+0000 7f5693376700 1 -- 172.21.15.32:0/1421503971 wait complete. 2023-09-30T22:26:23.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:23.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:23.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 Processor -- start 2023-09-30T22:26:23.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 -- start start 2023-09-30T22:26:23.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f02800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 0x7f0280133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028009edd0 con 0x7f0280133fa0 2023-09-30T22:26:23.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028009e1f0 con 0x7f02800a1340 2023-09-30T22:26:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0287a9b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028009dc50 con 0x7f02800a0a30 2023-09-30T22:26:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0285837700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f02800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.686+0000 7f0285036700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 0x7f0280133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f02800a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42516/0 (socket says 172.21.15.32:42516) 2023-09-30T22:26:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285036700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 0x7f0280133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58828/0 (socket says 172.21.15.32:58828) 2023-09-30T22:26:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 -- 172.21.15.32:0/2179664741 learned_addr learned my addr 172.21.15.32:0/2179664741 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0284835700 1 -- 172.21.15.32:0/2179664741 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366401235 0 0) 0x7f028009edd0 con 0x7f0280133fa0 2023-09-30T22:26:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 -- 172.21.15.32:0/2179664741 >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 msgr2=0x7f0280133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 --2- 172.21.15.32:0/2179664741 >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 0x7f0280133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 -- 172.21.15.32:0/2179664741 >> v1:172.21.15.32:6789/0 conn(0x7f0280133fa0 legacy=0x7f028013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 -- 172.21.15.32:0/2179664741 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800a1fe0 con 0x7f02800a0a30 2023-09-30T22:26:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285036700 1 --2- 172.21.15.32:0/2179664741 >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 0x7f0280133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0285837700 1 --2- 172.21.15.32:0/2179664741 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f02800a0e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f027c0098f0 tx=0x7f027c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a399268928fff39a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0284835700 1 -- 172.21.15.32:0/2179664741 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c011830 con 0x7f02800a0a30 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0284835700 1 -- 172.21.15.32:0/2179664741 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c011990 con 0x7f02800a0a30 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.687+0000 7f0284835700 1 -- 172.21.15.32:0/2179664741 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c011d20 con 0x7f02800a0a30 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 -- 172.21.15.32:0/2179664741 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 msgr2=0x7f02800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2179664741 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f02800a0e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f027c0098f0 tx=0x7f027c009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:23.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 -- 172.21.15.32:0/2179664741 shutdown_connections 2023-09-30T22:26:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2179664741 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f02800a0e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2179664741 >> v2:172.21.15.32:3300/0 conn(0x7f02800a1340 0x7f0280133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 -- 172.21.15.32:0/2179664741 >> 172.21.15.32:0/2179664741 conn(0x7f02800a6550 msgr2=0x7f02800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 -- 172.21.15.32:0/2179664741 shutdown_connections 2023-09-30T22:26:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.688+0000 7f0287a9b700 1 -- 172.21.15.32:0/2179664741 wait complete. 2023-09-30T22:26:23.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.689+0000 7f0287a9b700 1 Processor -- start 2023-09-30T22:26:23.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 -- start start 2023-09-30T22:26:23.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f0280122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02800a1340 0x7f0280122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0280133fa0 0x7f0280123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280125c80 con 0x7f0280133fa0 2023-09-30T22:26:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0280123b80 con 0x7f02800a0a30 2023-09-30T22:26:23.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0287a9b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02801243b0 con 0x7f02800a1340 2023-09-30T22:26:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0285837700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f0280122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0286038700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0280133fa0 0x7f0280123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0285837700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f0280122790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42530/0 (socket says 172.21.15.32:42530) 2023-09-30T22:26:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.690+0000 7f0285837700 1 -- 172.21.15.32:0/2037918223 learned_addr learned my addr 172.21.15.32:0/2037918223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285036700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02800a1340 0x7f0280122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285837700 1 -- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02800a1340 msgr2=0x7f0280122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285837700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02800a1340 0x7f0280122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285837700 1 -- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0280133fa0 msgr2=0x7f0280123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285837700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0280133fa0 0x7f0280123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285837700 1 -- 172.21.15.32:0/2037918223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700094a0 con 0x7f02800a0a30 2023-09-30T22:26:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0286038700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0280133fa0 0x7f0280123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:23.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0285837700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f0280122790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f027c009c40 tx=0x7f027c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:23.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c004300 con 0x7f02800a0a30 2023-09-30T22:26:23.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c01f070 con 0x7f02800a0a30 2023-09-30T22:26:23.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c009550 con 0x7f02800a0a30 2023-09-30T22:26:23.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0036a0 con 0x7f02800a0a30 2023-09-30T22:26:23.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.691+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280133200 con 0x7f02800a0a30 2023-09-30T22:26:23.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.692+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f027c021030 con 0x7f02800a0a30 2023-09-30T22:26:23.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.692+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f027c040b90 con 0x7f02800a0a30 2023-09-30T22:26:23.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.692+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02800a1d40 con 0x7f02800a0a30 2023-09-30T22:26:23.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.697+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f027c019960 con 0x7f02800a0a30 2023-09-30T22:26:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 Processor -- start 2023-09-30T22:26:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 -- start start 2023-09-30T22:26:23.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41a409d630 0x7f41a40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a40a2a70 con 0x7f41a40a0a30 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a40a01d0 con 0x7f41a409d630 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.725+0000 7f41ac400700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41a409e2b0 con 0x7f41a4138d80 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41aa99d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a999b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41a409d630 0x7f41a40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41aa99d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40970/0 (socket says 172.21.15.32:40970) 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41aa99d700 1 -- 172.21.15.32:0/776935277 learned_addr learned my addr 172.21.15.32:0/776935277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a999b700 1 -- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 msgr2=0x7f41a413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a999b700 1 --2- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a999b700 1 -- 172.21.15.32:0/776935277 >> v1:172.21.15.32:6789/0 conn(0x7f41a40a0a30 legacy=0x7f41a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a999b700 1 -- 172.21.15.32:0/776935277 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a409dc50 con 0x7f41a409d630 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a919a700 1 -- 172.21.15.32:0/776935277 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880989736 0 0) 0x7f41a40a2a70 con 0x7f41a40a0a30 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41aa99d700 1 --2- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:23.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a999b700 1 --2- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a409d630 0x7f41a40a1510 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4194009840 tx=0x7f4194009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cb3c2da28d63e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a919a700 1 -- 172.21.15.32:0/776935277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194011830 con 0x7f41a409d630 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a919a700 1 -- 172.21.15.32:0/776935277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194011990 con 0x7f41a409d630 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.726+0000 7f41a919a700 1 -- 172.21.15.32:0/776935277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194011d20 con 0x7f41a409d630 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 -- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a409d630 msgr2=0x7f41a40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 --2- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a409d630 0x7f41a40a1510 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4194009840 tx=0x7f4194009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 -- 172.21.15.32:0/776935277 shutdown_connections 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 --2- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 --2- 172.21.15.32:0/776935277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a409d630 0x7f41a40a1510 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 -- 172.21.15.32:0/776935277 >> 172.21.15.32:0/776935277 conn(0x7f41a40a3050 msgr2=0x7f41a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 -- 172.21.15.32:0/776935277 shutdown_connections 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.727+0000 7f41ac400700 1 -- 172.21.15.32:0/776935277 wait complete. 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.728+0000 7f41ac400700 1 Processor -- start 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.728+0000 7f41ac400700 1 -- start start 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41ac400700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41a409d630 0x7f41a412da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41ac400700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 0x7f41a412df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41ac400700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a412e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41ac400700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a412cef0 con 0x7f41a40a0a30 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41ac400700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a412f700 con 0x7f41a409d630 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41ac400700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41a409b890 con 0x7f41a4138d80 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 0x7f41a412df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 0x7f41a412df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58856/0 (socket says 172.21.15.32:58856) 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 -- 172.21.15.32:0/1173958591 learned_addr learned my addr 172.21.15.32:0/1173958591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41aa19c700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41a409d630 0x7f41a412da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41aa99d700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a412e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 -- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 msgr2=0x7f41a412e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a412e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 -- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41a409d630 msgr2=0x7f41a412da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41a409d630 0x7f41a412da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.729+0000 7f41a999b700 1 -- 172.21.15.32:0/1173958591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c009b30 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.730+0000 7f41a999b700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 0x7f41a412df80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4194009fd0 tx=0x7f4194004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.730+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194004550 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.730+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419401f070 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.730+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41940094a0 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.730+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940038e0 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.730+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a412c8c0 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.731+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4194003a40 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.731+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a40747f0 con 0x7f41a40a0a30 2023-09-30T22:26:23.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.731+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4194021070 con 0x7f41a40a0a30 2023-09-30T22:26:23.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.736+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4194026480 con 0x7f41a40a0a30 2023-09-30T22:26:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.903+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0280074ea0 con 0x7f02800a0a30 2023-09-30T22:26:23.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.903+0000 7f02767fc700 1 -- 172.21.15.32:0/2037918223 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f027c035090 con 0x7f02800a0a30 2023-09-30T22:26:23.904 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 msgr2=0x7f0280122790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f0280122790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f027c009c40 tx=0x7f027c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 shutdown_connections 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0280133fa0 0x7f0280123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02800a0a30 0x7f0280122790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 --2- 172.21.15.32:0/2037918223 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02800a1340 0x7f0280122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 >> 172.21.15.32:0/2037918223 conn(0x7f02800a6550 msgr2=0x7f02800a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.905+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 shutdown_connections 2023-09-30T22:26:23.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.906+0000 7f0287a9b700 1 -- 172.21.15.32:0/2037918223 wait complete. 2023-09-30T22:26:23.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:23.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.943+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f41a4072980 con 0x7f41a40a0a30 2023-09-30T22:26:23.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.943+0000 7f419affd700 1 -- 172.21.15.32:0/1173958591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f419402b070 con 0x7f41a40a0a30 2023-09-30T22:26:23.944 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 msgr2=0x7f41a412df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 0x7f41a412df80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4194009fd0 tx=0x7f4194004750 comp rx=0 tx=0).stop 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 shutdown_connections 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41a4138d80 0x7f41a412e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41a409d630 0x7f41a412da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 --2- 172.21.15.32:0/1173958591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41a40a0a30 0x7f41a412df80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.945+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 >> 172.21.15.32:0/1173958591 conn(0x7f41a40a3050 msgr2=0x7f41a409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.946+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 shutdown_connections 2023-09-30T22:26:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:23.946+0000 7f41ac400700 1 -- 172.21.15.32:0/1173958591 wait complete. 2023-09-30T22:26:23.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:24.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:24.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.149+0000 7f4211ddc700 1 Processor -- start 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.149+0000 7f4211ddc700 1 -- start start 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4211ddc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4211ddc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c09d630 0x7f420c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4211ddc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 0x7f420c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4211ddc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420c0a2a70 con 0x7f420c133930 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4211ddc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0a01d0 con 0x7f420c09d630 2023-09-30T22:26:24.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4211ddc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f420c09e2b0 con 0x7f420c0a0a30 2023-09-30T22:26:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 0x7f420c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40990/0 (socket says 172.21.15.32:40990) 2023-09-30T22:26:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 0x7f420c133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58870/0 (socket says 172.21.15.32:58870) 2023-09-30T22:26:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 -- 172.21.15.32:0/3444722360 learned_addr learned my addr 172.21.15.32:0/3444722360 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:24.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4202ffd700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c09d630 0x7f420c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 -- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c09d630 msgr2=0x7f420c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c09d630 0x7f420c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 -- 172.21.15.32:0/3444722360 >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 msgr2=0x7f420c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:24.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 --2- 172.21.15.32:0/3444722360 >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 0x7f420c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420b7fe700 1 -- 172.21.15.32:0/3444722360 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c09dc50 con 0x7f420c0a0a30 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f4202ffd700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c09d630 0x7f420c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.150+0000 7f420bfff700 1 --2- 172.21.15.32:0/3444722360 >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 0x7f420c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.151+0000 7f420b7fe700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c0a0e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0098f0 tx=0x7f41fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=893210f6c7296306 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.151+0000 7f420affd700 1 -- 172.21.15.32:0/3444722360 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011830 con 0x7f420c0a0a30 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.151+0000 7f420affd700 1 -- 172.21.15.32:0/3444722360 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc011990 con 0x7f420c0a0a30 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.151+0000 7f420affd700 1 -- 172.21.15.32:0/3444722360 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011d20 con 0x7f420c0a0a30 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.151+0000 7f4211ddc700 1 -- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 msgr2=0x7f420c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c0a0e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0098f0 tx=0x7f41fc009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:24.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 -- 172.21.15.32:0/3444722360 shutdown_connections 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 --2- 172.21.15.32:0/3444722360 >> v2:172.21.15.32:3300/0 conn(0x7f420c133930 0x7f420c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c0a0e00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 --2- 172.21.15.32:0/3444722360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c09d630 0x7f420c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 -- 172.21.15.32:0/3444722360 >> 172.21.15.32:0/3444722360 conn(0x7f420c0a3050 msgr2=0x7f420c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 -- 172.21.15.32:0/3444722360 shutdown_connections 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.152+0000 7f4211ddc700 1 -- 172.21.15.32:0/3444722360 wait complete. 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 Processor -- start 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 -- start start 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 0x7f420c1c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c1c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c133930 0x7f420c1c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c122430 con 0x7f420c09d630 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c120720 con 0x7f420c133930 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.153+0000 7f4211ddc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0a41c0 con 0x7f420c0a0a30 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 0x7f420c1c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c133930 0x7f420c1c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 0x7f420c1c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58872/0 (socket says 172.21.15.32:58872) 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 -- 172.21.15.32:0/834462111 learned_addr learned my addr 172.21.15.32:0/834462111 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f4202ffd700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c1c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 -- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 msgr2=0x7f420c1c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c1c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 -- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c133930 msgr2=0x7f420c1c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c133930 0x7f420c1c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 -- 172.21.15.32:0/834462111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40094a0 con 0x7f420c09d630 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f420b7fe700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 0x7f420c1c81a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009c40 tx=0x7f41fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc004450 con 0x7f420c09d630 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc0045b0 con 0x7f420c09d630 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4204009a70 con 0x7f420c09d630 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0038c0 con 0x7f420c09d630 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.154+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc009550 con 0x7f420c09d630 2023-09-30T22:26:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.155+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f41fc021070 con 0x7f420c09d630 2023-09-30T22:26:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.155+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f41fc040990 con 0x7f420c09d630 2023-09-30T22:26:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.155+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c133620 con 0x7f420c09d630 2023-09-30T22:26:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.160+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41fc029070 con 0x7f420c09d630 2023-09-30T22:26:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.192+0000 7f3ab871e700 1 Processor -- start 2023-09-30T22:26:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.192+0000 7f3ab871e700 1 -- start start 2023-09-30T22:26:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab871e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 0x7f3ab00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab871e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab009d630 0x7f3ab00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab871e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab00a2a70 con 0x7f3ab0133930 2023-09-30T22:26:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab871e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab00a01d0 con 0x7f3ab00a0a30 2023-09-30T22:26:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab871e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab009e2b0 con 0x7f3ab009d630 2023-09-30T22:26:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab64ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 0x7f3ab00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab5cb9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab009d630 0x7f3ab00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab6cbb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ab0133930 0x7f3ab0133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab64ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 0x7f3ab00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58882/0 (socket says 172.21.15.32:58882) 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.193+0000 7f3ab6cbb700 1 -- 172.21.15.32:0/1559010021 learned_addr learned my addr 172.21.15.32:0/1559010021 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab5cb9700 1 -- 172.21.15.32:0/1559010021 >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 msgr2=0x7f3ab00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab5cb9700 1 --2- 172.21.15.32:0/1559010021 >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 0x7f3ab00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab5cb9700 1 -- 172.21.15.32:0/1559010021 >> v1:172.21.15.32:6789/0 conn(0x7f3ab0133930 legacy=0x7f3ab0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab5cb9700 1 -- 172.21.15.32:0/1559010021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab009dc50 con 0x7f3ab009d630 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab54b8700 1 -- 172.21.15.32:0/1559010021 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046756639 0 0) 0x7f3ab00a2a70 con 0x7f3ab0133930 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab64ba700 1 --2- 172.21.15.32:0/1559010021 >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 0x7f3ab00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab5cb9700 1 --2- 172.21.15.32:0/1559010021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab009d630 0x7f3ab00a1340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009840 tx=0x7f3aa4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be35e8511179ac5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:24.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab54b8700 1 -- 172.21.15.32:0/1559010021 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4011830 con 0x7f3ab009d630 2023-09-30T22:26:24.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab54b8700 1 -- 172.21.15.32:0/1559010021 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4011990 con 0x7f3ab009d630 2023-09-30T22:26:24.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.194+0000 7f3ab54b8700 1 -- 172.21.15.32:0/1559010021 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4011d20 con 0x7f3ab009d630 2023-09-30T22:26:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 -- 172.21.15.32:0/1559010021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab009d630 msgr2=0x7f3ab00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 --2- 172.21.15.32:0/1559010021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab009d630 0x7f3ab00a1340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009840 tx=0x7f3aa4009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 -- 172.21.15.32:0/1559010021 shutdown_connections 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 --2- 172.21.15.32:0/1559010021 >> v2:172.21.15.32:3300/0 conn(0x7f3ab00a0a30 0x7f3ab00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 --2- 172.21.15.32:0/1559010021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab009d630 0x7f3ab00a1340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 -- 172.21.15.32:0/1559010021 >> 172.21.15.32:0/1559010021 conn(0x7f3ab00a3050 msgr2=0x7f3ab00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 -- 172.21.15.32:0/1559010021 shutdown_connections 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.195+0000 7f3ab871e700 1 -- 172.21.15.32:0/1559010021 wait complete. 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 Processor -- start 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 -- start start 2023-09-30T22:26:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 0x7f3ab01c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 0x7f3ab01c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab0133930 0x7f3ab01c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01221b0 con 0x7f3ab009d630 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0133520 con 0x7f3ab0133930 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.196+0000 7f3ab871e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0120700 con 0x7f3ab00a0a30 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 0x7f3ab01c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab5cb9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 0x7f3ab01c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 0x7f3ab01c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58898/0 (socket says 172.21.15.32:58898) 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab5cb9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 0x7f3ab01c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41022/0 (socket says 172.21.15.32:41022) 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 -- 172.21.15.32:0/2005219868 learned_addr learned my addr 172.21.15.32:0/2005219868 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab6cbb700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab0133930 0x7f3ab01c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 -- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 msgr2=0x7f3ab01c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 0x7f3ab01c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 -- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab0133930 msgr2=0x7f3ab01c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab0133930 0x7f3ab01c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 -- 172.21.15.32:0/2005219868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa00094a0 con 0x7f3ab009d630 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab5cb9700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 0x7f3ab01c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab64ba700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 0x7f3ab01c7f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ebe0 tx=0x7f3aa000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000c850 con 0x7f3ab009d630 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa000c9b0 con 0x7f3ab009d630 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cd40 con 0x7f3ab009d630 2023-09-30T22:26:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa40094a0 con 0x7f3ab009d630 2023-09-30T22:26:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.197+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab0121ef0 con 0x7f3ab009d630 2023-09-30T22:26:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.199+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3aa0013470 con 0x7f3ab009d630 2023-09-30T22:26:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.199+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab00747f0 con 0x7f3ab009d630 2023-09-30T22:26:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.199+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3aa003f780 con 0x7f3ab009d630 2023-09-30T22:26:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.204+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3aa0013720 con 0x7f3ab009d630 2023-09-30T22:26:24.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.368+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f420c0747f0 con 0x7f420c09d630 2023-09-30T22:26:24.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.368+0000 7f4208ff9700 1 -- 172.21.15.32:0/834462111 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41fc015030 con 0x7f420c09d630 2023-09-30T22:26:24.369 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 msgr2=0x7f420c1c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 0x7f420c1c81a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009c40 tx=0x7f41fc004750 comp rx=0 tx=0).stop 2023-09-30T22:26:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 shutdown_connections 2023-09-30T22:26:24.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f420c133930 0x7f420c1c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f420c09d630 0x7f420c1c81a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 --2- 172.21.15.32:0/834462111 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f420c0a0a30 0x7f420c1c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 >> 172.21.15.32:0/834462111 conn(0x7f420c0a3050 msgr2=0x7f420c1bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:24.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 shutdown_connections 2023-09-30T22:26:24.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.371+0000 7f4211ddc700 1 -- 172.21.15.32:0/834462111 wait complete. 2023-09-30T22:26:24.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:24.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.409+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ab0072980 con 0x7f3ab009d630 2023-09-30T22:26:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.409+0000 7f3aaeffd700 1 -- 172.21.15.32:0/2005219868 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3aa0021070 con 0x7f3ab009d630 2023-09-30T22:26:24.410 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 msgr2=0x7f3ab01c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 0x7f3ab01c7f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ebe0 tx=0x7f3aa000eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 shutdown_connections 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ab0133930 0x7f3ab01c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ab009d630 0x7f3ab01c7f80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 --2- 172.21.15.32:0/2005219868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ab00a0a30 0x7f3ab01c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 >> 172.21.15.32:0/2005219868 conn(0x7f3ab00a3050 msgr2=0x7f3ab01bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 shutdown_connections 2023-09-30T22:26:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:24.412+0000 7f3ab871e700 1 -- 172.21.15.32:0/2005219868 wait complete. 2023-09-30T22:26:24.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:24.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:24.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:25.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 Processor -- start 2023-09-30T22:26:25.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 -- start start 2023-09-30T22:26:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76b809e410 0x7f76b8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b806b460 con 0x7f76b809e410 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b809b120 con 0x7f76b809e000 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76bcafd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76b809a7e0 con 0x7f76b809d6f0 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.070+0000 7f76b5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76b809e410 0x7f76b8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42580/0 (socket says 172.21.15.32:42580) 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 -- 172.21.15.32:0/730425416 learned_addr learned my addr 172.21.15.32:0/730425416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 -- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 msgr2=0x7f76b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 --2- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 -- 172.21.15.32:0/730425416 >> v2:172.21.15.32:3300/0 conn(0x7f76b809e410 msgr2=0x7f76b8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:25.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 --2- 172.21.15.32:0/730425416 >> v2:172.21.15.32:3300/0 conn(0x7f76b809e410 0x7f76b8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 -- 172.21.15.32:0/730425416 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b809a500 con 0x7f76b809e000 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b6d9d700 1 --2- 172.21.15.32:0/730425416 >> v2:172.21.15.32:3300/0 conn(0x7f76b809e410 0x7f76b8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b659c700 1 --2- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b5d9b700 1 --2- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81339f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009920 tx=0x7f76a0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ffedd4f4aed629b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b559a700 1 -- 172.21.15.32:0/730425416 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011830 con 0x7f76b809e000 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b559a700 1 -- 172.21.15.32:0/730425416 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0011990 con 0x7f76b809e000 2023-09-30T22:26:25.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.071+0000 7f76b559a700 1 -- 172.21.15.32:0/730425416 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011d20 con 0x7f76b809e000 2023-09-30T22:26:25.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 -- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 msgr2=0x7f76b81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 --2- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81339f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009920 tx=0x7f76a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 -- 172.21.15.32:0/730425416 shutdown_connections 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 --2- 172.21.15.32:0/730425416 >> v2:172.21.15.32:3300/0 conn(0x7f76b809e410 0x7f76b8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 --2- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 --2- 172.21.15.32:0/730425416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81339f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 -- 172.21.15.32:0/730425416 >> 172.21.15.32:0/730425416 conn(0x7f76b8066f50 msgr2=0x7f76b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 -- 172.21.15.32:0/730425416 shutdown_connections 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.072+0000 7f76bcafd700 1 -- 172.21.15.32:0/730425416 wait complete. 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.073+0000 7f76bcafd700 1 Processor -- start 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 -- start start 2023-09-30T22:26:25.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b81b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b809e410 0x7f76b81a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81aa2c0 con 0x7f76b809e410 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81a9900 con 0x7f76b809e000 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76bcafd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81a9090 con 0x7f76b809d6f0 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76b659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b81b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76b5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76b5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42592/0 (socket says 172.21.15.32:42592) 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76b5d9b700 1 -- 172.21.15.32:0/467392672 learned_addr learned my addr 172.21.15.32:0/467392672 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76b6d9d700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b809e410 0x7f76b81a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.074+0000 7f76b5d9b700 1 -- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 msgr2=0x7f76b81b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76b5d9b700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b81b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76b5d9b700 1 -- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b809e410 msgr2=0x7f76b81a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76b5d9b700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b809e410 0x7f76b81a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76b5d9b700 1 -- 172.21.15.32:0/467392672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8009a70 con 0x7f76b809e000 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76b659c700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b81b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76b5d9b700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81a5de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009c70 tx=0x7f76a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0004240 con 0x7f76b809e000 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a001f070 con 0x7f76b809e000 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b00094a0 con 0x7f76b809e000 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a00036a0 con 0x7f76b809e000 2023-09-30T22:26:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.075+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a0009550 con 0x7f76b809e000 2023-09-30T22:26:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.076+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76a0021030 con 0x7f76b809e000 2023-09-30T22:26:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.076+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f76a0040a90 con 0x7f76b809e000 2023-09-30T22:26:25.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.076+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b81a92c0 con 0x7f76b809e000 2023-09-30T22:26:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.081+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76b81a92c0 con 0x7f76b809e000 2023-09-30T22:26:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.108+0000 7f4b9216c700 1 Processor -- start 2023-09-30T22:26:25.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.108+0000 7f4b9216c700 1 -- start start 2023-09-30T22:26:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.108+0000 7f4b9216c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b8c12cc20 0x7f4b8c12f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.108+0000 7f4b9216c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12cff0 0x7f4b8c137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.108+0000 7f4b9216c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c09a510 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b9216c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8c12c7d0 con 0x7f4b8c12cc20 2023-09-30T22:26:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b9216c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c09c190 con 0x7f4b8c12cff0 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b8c12cc20 0x7f4b8c12f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4b8c12c2f0 0x7f4b8c12fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49974/0 (socket says 172.21.15.32:49974) 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8affd700 1 -- 172.21.15.32:0/3350418799 learned_addr learned my addr 172.21.15.32:0/3350418799 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8bfff700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12cff0 0x7f4b8c137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8a7fc700 1 -- 172.21.15.32:0/3350418799 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774652743 0 0) 0x7f4b8c09a510 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8a7fc700 1 -- 172.21.15.32:0/3350418799 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b780035b0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12cff0 msgr2=0x7f4b8c137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8b7fe700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12cff0 0x7f4b8c137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3350418799 >> v1:172.21.15.32:6789/0 conn(0x7f4b8c12c2f0 legacy=0x7f4b8c12fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3350418799 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c09b540 con 0x7f4b8c12cc20 2023-09-30T22:26:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8bfff700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12cff0 0x7f4b8c137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8b7fe700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c12f930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00a8e0 tx=0x7f4b7c00abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d106df9af1042b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.109+0000 7f4b8a7fc700 1 -- 172.21.15.32:0/3350418799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00b6e0 con 0x7f4b8c12cc20 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b8a7fc700 1 -- 172.21.15.32:0/3350418799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c00b840 con 0x7f4b8c12cc20 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b8a7fc700 1 -- 172.21.15.32:0/3350418799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00bbd0 con 0x7f4b8c12cc20 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 -- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 msgr2=0x7f4b8c12f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c12f930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00a8e0 tx=0x7f4b7c00abf0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 -- 172.21.15.32:0/3350418799 shutdown_connections 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12cff0 0x7f4b8c137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3350418799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c12f930 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 -- 172.21.15.32:0/3350418799 >> 172.21.15.32:0/3350418799 conn(0x7f4b8c09f470 msgr2=0x7f4b8c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 -- 172.21.15.32:0/3350418799 shutdown_connections 2023-09-30T22:26:25.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.110+0000 7f4b9216c700 1 -- 172.21.15.32:0/3350418799 wait complete. 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.111+0000 7f4b9216c700 1 Processor -- start 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.111+0000 7f4b9216c700 1 -- start start 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b9216c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 0x7f4b8c128960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b9216c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c128ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b9216c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 0x7f4b8c1c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b9216c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c129d30 con 0x7f4b8c12cc20 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b9216c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c06e060 con 0x7f4b8c12cff0 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b9216c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c12b690 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 0x7f4b8c128960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 0x7f4b8c1c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 0x7f4b8c128960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41064/0 (socket says 172.21.15.32:41064) 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3951102139 learned_addr learned my addr 172.21.15.32:0/3951102139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8bfff700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 0x7f4b8c1c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42594/0 (socket says 172.21.15.32:42594) 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8affd700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c128ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 msgr2=0x7f4b8c1c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 0x7f4b8c1c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 msgr2=0x7f4b8c128ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c128ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8b7fe700 1 -- 172.21.15.32:0/3951102139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840099e0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8bfff700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 0x7f4b8c1c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.112+0000 7f4b8affd700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c128ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.113+0000 7f4b8b7fe700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 0x7f4b8c128960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00a8b0 tx=0x7f4b7c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.113+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c01f070 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.113+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c004350 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.113+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c00a550 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.113+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0036a0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.113+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c06efb0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.114+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b7c003800 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.114+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b7c0409b0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.114+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c0747f0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.119+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b7c00b040 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.289+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f76b81a92c0 con 0x7f76b809e000 2023-09-30T22:26:25.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.289+0000 7f76aeffd700 1 -- 172.21.15.32:0/467392672 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76b81a92c0 con 0x7f76b809e000 2023-09-30T22:26:25.290 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.291+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 msgr2=0x7f76b81a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81a5de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009c70 tx=0x7f76a0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 shutdown_connections 2023-09-30T22:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b809e410 0x7f76b81a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76b809d6f0 0x7f76b81b91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 --2- 172.21.15.32:0/467392672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76b809e000 0x7f76b81a5de0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 >> 172.21.15.32:0/467392672 conn(0x7f76b8066f50 msgr2=0x7f76b81badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 shutdown_connections 2023-09-30T22:26:25.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.292+0000 7f76bcafd700 1 -- 172.21.15.32:0/467392672 wait complete. 2023-09-30T22:26:25.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.327+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4b8c06b250 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.327+0000 7f4b83fff700 1 -- 172.21.15.32:0/3951102139 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b7c0233c0 con 0x7f4b8c12c2f0 2023-09-30T22:26:25.329 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 msgr2=0x7f4b8c128960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 0x7f4b8c128960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00a8b0 tx=0x7f4b7c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 shutdown_connections 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b8c12cff0 0x7f4b8c1c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b8c12c2f0 0x7f4b8c128960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 --2- 172.21.15.32:0/3951102139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b8c12cc20 0x7f4b8c128ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 >> 172.21.15.32:0/3951102139 conn(0x7f4b8c09f470 msgr2=0x7f4b8c12a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 shutdown_connections 2023-09-30T22:26:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.330+0000 7f4b9216c700 1 -- 172.21.15.32:0/3951102139 wait complete. 2023-09-30T22:26:25.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:25.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:25.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 Processor -- start 2023-09-30T22:26:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 -- start start 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6409f130 con 0x7efe6409e410 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6409f520 con 0x7efe6409e000 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe6ab06700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6409e930 con 0x7efe6409d6f0 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe63fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.528+0000 7efe690a3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe6409e410 0x7efe64130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49988/0 (socket says 172.21.15.32:49988) 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe690a3700 1 -- 172.21.15.32:0/1860582707 learned_addr learned my addr 172.21.15.32:0/1860582707 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe688a2700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe63fff700 1 -- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 msgr2=0x7efe6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe63fff700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe63fff700 1 -- 172.21.15.32:0/1860582707 >> v1:172.21.15.32:6789/0 conn(0x7efe6409e410 legacy=0x7efe64130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe637fe700 1 -- 172.21.15.32:0/1860582707 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301345960 0 0) 0x7efe6409f130 con 0x7efe6409e410 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe63fff700 1 -- 172.21.15.32:0/1860582707 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6409b2d0 con 0x7efe6409e000 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe688a2700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe63fff700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe64130660 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efe58009840 tx=0x7efe58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8febb30f36f8dd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe637fe700 1 -- 172.21.15.32:0/1860582707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58011830 con 0x7efe6409e000 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe637fe700 1 -- 172.21.15.32:0/1860582707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe58011990 con 0x7efe6409e000 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.529+0000 7efe637fe700 1 -- 172.21.15.32:0/1860582707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58011d20 con 0x7efe6409e000 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 -- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 msgr2=0x7efe64130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe64130660 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efe58009840 tx=0x7efe58009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 -- 172.21.15.32:0/1860582707 shutdown_connections 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 --2- 172.21.15.32:0/1860582707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe64130660 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 -- 172.21.15.32:0/1860582707 >> 172.21.15.32:0/1860582707 conn(0x7efe64066f50 msgr2=0x7efe6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 -- 172.21.15.32:0/1860582707 shutdown_connections 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.530+0000 7efe6ab06700 1 -- 172.21.15.32:0/1860582707 wait complete. 2023-09-30T22:26:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.531+0000 7efe6ab06700 1 Processor -- start 2023-09-30T22:26:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.531+0000 7efe6ab06700 1 -- start start 2023-09-30T22:26:25.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe6ab06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6411dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe6ab06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe6411e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe6ab06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe6409e410 0x7efe6411ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe6ab06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6411f490 con 0x7efe6409e000 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe6ab06700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe641303a0 con 0x7efe6409d6f0 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe6ab06700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe64120e20 con 0x7efe6409e410 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe6411e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe6411e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58956/0 (socket says 172.21.15.32:58956) 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 -- 172.21.15.32:0/605657958 learned_addr learned my addr 172.21.15.32:0/605657958 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe690a3700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe6409e410 0x7efe6411ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe688a2700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6411dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 -- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe6409e410 msgr2=0x7efe6411ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe6409e410 0x7efe6411ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 -- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 msgr2=0x7efe6411dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6411dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe63fff700 1 -- 172.21.15.32:0/605657958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5400d940 con 0x7efe6409e000 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.532+0000 7efe688a2700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6411dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.533+0000 7efe63fff700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe6411e500 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efe58009fd0 tx=0x7efe58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.533+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe580036a0 con 0x7efe6409e000 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.533+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5801f070 con 0x7efe6409e000 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.533+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe580094a0 con 0x7efe6409e000 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.533+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58003a60 con 0x7efe6409e000 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.533+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe64121850 con 0x7efe6409e000 2023-09-30T22:26:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.534+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe58003bc0 con 0x7efe6409e000 2023-09-30T22:26:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.534+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efe58021070 con 0x7efe6409e000 2023-09-30T22:26:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.534+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe64121850 con 0x7efe6409e000 2023-09-30T22:26:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.539+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe64121850 con 0x7efe6409e000 2023-09-30T22:26:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 Processor -- start 2023-09-30T22:26:25.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 -- start start 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00c0130750 0x7f00c0130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c009db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c009e080 0x7f00c01374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c009c190 con 0x7f00c0130750 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c009b090 con 0x7f00c009e080 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.568+0000 7f00c83ae700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00c009a6c0 con 0x7f00c009d6f0 2023-09-30T22:26:25.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00c0130750 0x7f00c0130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c5949700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c694b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c009e080 0x7f00c01374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00c0130750 0x7f00c0130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58968/0 (socket says 172.21.15.32:58968) 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 -- 172.21.15.32:0/593004837 learned_addr learned my addr 172.21.15.32:0/593004837 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 -- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 msgr2=0x7f00c009db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c009db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 -- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c009e080 msgr2=0x7f00c01374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c009e080 0x7f00c01374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 -- 172.21.15.32:0/593004837 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0131060 con 0x7f00c0130750 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c5949700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c009db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c694b700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c009e080 0x7f00c01374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c614a700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c0130750 0x7f00c0130b20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098f0 tx=0x7f00bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268d49b5718fd9cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c5148700 1 -- 172.21.15.32:0/593004837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc011830 con 0x7f00c0130750 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c5148700 1 -- 172.21.15.32:0/593004837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc011990 con 0x7f00c0130750 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.569+0000 7f00c5148700 1 -- 172.21.15.32:0/593004837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc011d20 con 0x7f00c0130750 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 -- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c0130750 msgr2=0x7f00c0130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c0130750 0x7f00c0130b20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098f0 tx=0x7f00bc009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 -- 172.21.15.32:0/593004837 shutdown_connections 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c009e080 0x7f00c01374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c0130750 0x7f00c0130b20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 --2- 172.21.15.32:0/593004837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c009db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 -- 172.21.15.32:0/593004837 >> 172.21.15.32:0/593004837 conn(0x7f00c009f470 msgr2=0x7f00c00a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 -- 172.21.15.32:0/593004837 shutdown_connections 2023-09-30T22:26:25.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.570+0000 7f00c83ae700 1 -- 172.21.15.32:0/593004837 wait complete. 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.571+0000 7f00c83ae700 1 Processor -- start 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.571+0000 7f00c83ae700 1 -- start start 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c83ae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c012cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c83ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c009e080 0x7f00c012d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c83ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c0130750 0x7f00c012d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c83ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c012e140 con 0x7f00c009e080 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c83ae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c012f010 con 0x7f00c0130750 2023-09-30T22:26:25.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c83ae700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00c009d170 con 0x7f00c009d6f0 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c012cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c012cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41098/0 (socket says 172.21.15.32:41098) 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 -- 172.21.15.32:0/2838549021 learned_addr learned my addr 172.21.15.32:0/2838549021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c5949700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c009e080 0x7f00c012d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c694b700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c0130750 0x7f00c012d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 -- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c0130750 msgr2=0x7f00c012d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c0130750 0x7f00c012d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 -- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c009e080 msgr2=0x7f00c012d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c009e080 0x7f00c012d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 -- 172.21.15.32:0/2838549021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00094a0 con 0x7f00c009d6f0 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c694b700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c0130750 0x7f00c012d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:25.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00c614a700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c012cdd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009c40 tx=0x7f00bc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.572+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004450 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.573+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0045b0 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.573+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b8009a70 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.573+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0038c0 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.573+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc009550 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.574+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f00bc021070 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.574+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c00747f0 con 0x7f00c009d6f0 2023-09-30T22:26:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.574+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f00bc0408d0 con 0x7f00c009d6f0 2023-09-30T22:26:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.578+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f00bc00b9d0 con 0x7f00c009d6f0 2023-09-30T22:26:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.745+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efe64121850 con 0x7efe6409e000 2023-09-30T22:26:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.746+0000 7efe617fa700 1 -- 172.21.15.32:0/605657958 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe64121850 con 0x7efe6409e000 2023-09-30T22:26:25.747 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 msgr2=0x7efe6411e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe6411e500 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efe58009fd0 tx=0x7efe58004750 comp rx=0 tx=0).stop 2023-09-30T22:26:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 shutdown_connections 2023-09-30T22:26:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe6409e410 0x7efe6411ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409d6f0 0x7efe6411dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 --2- 172.21.15.32:0/605657958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e000 0x7efe6411e500 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 >> 172.21.15.32:0/605657958 conn(0x7efe64066f50 msgr2=0x7efe64130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 shutdown_connections 2023-09-30T22:26:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.749+0000 7efe6ab06700 1 -- 172.21.15.32:0/605657958 wait complete. 2023-09-30T22:26:25.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:25.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.794+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f00c006b250 con 0x7f00c009d6f0 2023-09-30T22:26:25.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.794+0000 7f00b6ffd700 1 -- 172.21.15.32:0/2838549021 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00bc015020 con 0x7f00c009d6f0 2023-09-30T22:26:25.795 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 msgr2=0x7f00c012cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c012cdd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009c40 tx=0x7f00bc004750 comp rx=0 tx=0).stop 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 shutdown_connections 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c0130750 0x7f00c012d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c009d6f0 0x7f00c012cdd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 --2- 172.21.15.32:0/2838549021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c009e080 0x7f00c012d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:25.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 >> 172.21.15.32:0/2838549021 conn(0x7f00c009f470 msgr2=0x7f00c006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 shutdown_connections 2023-09-30T22:26:25.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:25.796+0000 7f00c83ae700 1 -- 172.21.15.32:0/2838549021 wait complete. 2023-09-30T22:26:25.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:26.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:26.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:26.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.454+0000 7eff70bdf700 1 Processor -- start 2023-09-30T22:26:26.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.454+0000 7eff70bdf700 1 -- start start 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff70bdf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 0x7eff6c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff70bdf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff70bdf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff6c133930 0x7eff6c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff70bdf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0a2a70 con 0x7eff6c133930 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff70bdf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0a01d0 con 0x7eff6c0a0a30 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff70bdf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c09e2b0 con 0x7eff6c09d630 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 0x7eff6c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff69d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 0x7eff6c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42636/0 (socket says 172.21.15.32:42636) 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff6ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff6c133930 0x7eff6c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff6a59c700 1 -- 172.21.15.32:0/1696785234 learned_addr learned my addr 172.21.15.32:0/1696785234 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff69d9b700 1 -- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 msgr2=0x7eff6c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff69d9b700 1 --2- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 0x7eff6c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff69d9b700 1 -- 172.21.15.32:0/1696785234 >> v2:172.21.15.32:3300/0 conn(0x7eff6c133930 msgr2=0x7eff6c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff69d9b700 1 --2- 172.21.15.32:0/1696785234 >> v2:172.21.15.32:3300/0 conn(0x7eff6c133930 0x7eff6c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff69d9b700 1 -- 172.21.15.32:0/1696785234 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c09dc50 con 0x7eff6c09d630 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff6a59c700 1 --2- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 0x7eff6c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.455+0000 7eff6ad9d700 1 --2- 172.21.15.32:0/1696785234 >> v2:172.21.15.32:3300/0 conn(0x7eff6c133930 0x7eff6c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.456+0000 7eff69d9b700 1 --2- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c0a1340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff54009840 tx=0x7eff54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee633e3bb1c2109 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.456+0000 7eff6959a700 1 -- 172.21.15.32:0/1696785234 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011830 con 0x7eff6c09d630 2023-09-30T22:26:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.456+0000 7eff6959a700 1 -- 172.21.15.32:0/1696785234 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54011990 con 0x7eff6c09d630 2023-09-30T22:26:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.456+0000 7eff6959a700 1 -- 172.21.15.32:0/1696785234 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011d20 con 0x7eff6c09d630 2023-09-30T22:26:26.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 -- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 msgr2=0x7eff6c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 --2- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c0a1340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff54009840 tx=0x7eff54009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:26.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 -- 172.21.15.32:0/1696785234 shutdown_connections 2023-09-30T22:26:26.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 --2- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c0a0a30 0x7eff6c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 --2- 172.21.15.32:0/1696785234 >> v2:172.21.15.32:3300/0 conn(0x7eff6c133930 0x7eff6c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 --2- 172.21.15.32:0/1696785234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c0a1340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 -- 172.21.15.32:0/1696785234 >> 172.21.15.32:0/1696785234 conn(0x7eff6c0a3050 msgr2=0x7eff6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 -- 172.21.15.32:0/1696785234 shutdown_connections 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.457+0000 7eff70bdf700 1 -- 172.21.15.32:0/1696785234 wait complete. 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.458+0000 7eff70bdf700 1 Processor -- start 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.458+0000 7eff70bdf700 1 -- start start 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff70bdf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c11f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff70bdf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6c0a0a30 0x7eff6c11f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff70bdf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 0x7eff6c1c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff70bdf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c11ee60 con 0x7eff6c0a0a30 2023-09-30T22:26:26.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff70bdf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c11fdd0 con 0x7eff6c133930 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff70bdf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c121e70 con 0x7eff6c09d630 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 0x7eff6c1c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c11f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff69d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6c0a0a30 0x7eff6c11f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c11f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41120/0 (socket says 172.21.15.32:41120) 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 0x7eff6c1c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42646/0 (socket says 172.21.15.32:42646) 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.459+0000 7eff6a59c700 1 -- 172.21.15.32:0/4029305600 learned_addr learned my addr 172.21.15.32:0/4029305600 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6ad9d700 1 -- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 msgr2=0x7eff6c11f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6ad9d700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c11f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6ad9d700 1 -- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6c0a0a30 msgr2=0x7eff6c11f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6ad9d700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6c0a0a30 0x7eff6c11f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6ad9d700 1 -- 172.21.15.32:0/4029305600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c009b30 con 0x7eff6c133930 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff69d9b700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6c0a0a30 0x7eff6c11f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6a59c700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c11f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff6ad9d700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 0x7eff6c1c8000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff6400e910 tx=0x7eff6400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64004020 con 0x7eff6c133930 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff640041a0 con 0x7eff6c133930 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff640094a0 con 0x7eff6c133930 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64004550 con 0x7eff6c133930 2023-09-30T22:26:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.460+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff540094a0 con 0x7eff6c133930 2023-09-30T22:26:26.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.461+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff640036a0 con 0x7eff6c133930 2023-09-30T22:26:26.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.461+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7eff6403f750 con 0x7eff6c133930 2023-09-30T22:26:26.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.461+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c0747f0 con 0x7eff6c133930 2023-09-30T22:26:26.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.470+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff64017080 con 0x7eff6c133930 2023-09-30T22:26:26.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.496+0000 7febe3d47700 1 Processor -- start 2023-09-30T22:26:26.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe3d47700 1 -- start start 2023-09-30T22:26:26.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe3d47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7febdc09d630 0x7febdc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe3d47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe3d47700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc0a2a70 con 0x7febdc0a0a30 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe3d47700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febdc0a01d0 con 0x7febdc09d630 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe3d47700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc09e2b0 con 0x7febdc138d80 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe22e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe12e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7febdc09d630 0x7febdc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe22e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42654/0 (socket says 172.21.15.32:42654) 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.497+0000 7febe22e4700 1 -- 172.21.15.32:0/382499936 learned_addr learned my addr 172.21.15.32:0/382499936 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe22e4700 1 -- 172.21.15.32:0/382499936 >> v2:172.21.15.32:3300/0 conn(0x7febdc09d630 msgr2=0x7febdc0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:26.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe22e4700 1 --2- 172.21.15.32:0/382499936 >> v2:172.21.15.32:3300/0 conn(0x7febdc09d630 0x7febdc0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe22e4700 1 -- 172.21.15.32:0/382499936 >> v1:172.21.15.32:6789/0 conn(0x7febdc0a0a30 legacy=0x7febdc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe22e4700 1 -- 172.21.15.32:0/382499936 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc09dc50 con 0x7febdc138d80 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe12e2700 1 --2- 172.21.15.32:0/382499936 >> v2:172.21.15.32:3300/0 conn(0x7febdc09d630 0x7febdc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe22e4700 1 --2- 172.21.15.32:0/382499936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc13b160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e50 tx=0x7febd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50fd625712bde41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe0ae1700 1 -- 172.21.15.32:0/382499936 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400e070 con 0x7febdc138d80 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe0ae1700 1 -- 172.21.15.32:0/382499936 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd400b910 con 0x7febdc138d80 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.498+0000 7febe0ae1700 1 -- 172.21.15.32:0/382499936 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400bcc0 con 0x7febdc138d80 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 -- 172.21.15.32:0/382499936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 msgr2=0x7febdc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 --2- 172.21.15.32:0/382499936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc13b160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e50 tx=0x7febd400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 -- 172.21.15.32:0/382499936 shutdown_connections 2023-09-30T22:26:26.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 --2- 172.21.15.32:0/382499936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc13b160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 --2- 172.21.15.32:0/382499936 >> v2:172.21.15.32:3300/0 conn(0x7febdc09d630 0x7febdc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 -- 172.21.15.32:0/382499936 >> 172.21.15.32:0/382499936 conn(0x7febdc0a3050 msgr2=0x7febdc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 -- 172.21.15.32:0/382499936 shutdown_connections 2023-09-30T22:26:26.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.499+0000 7febe3d47700 1 -- 172.21.15.32:0/382499936 wait complete. 2023-09-30T22:26:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 Processor -- start 2023-09-30T22:26:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 -- start start 2023-09-30T22:26:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febdc09d630 0x7febdc120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 0x7febdc120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc1210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc124140 con 0x7febdc0a0a30 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1234f0 con 0x7febdc138d80 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.500+0000 7febe3d47700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febdc138b20 con 0x7febdc09d630 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe12e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 0x7febdc120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe12e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 0x7febdc120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59016/0 (socket says 172.21.15.32:59016) 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe12e2700 1 -- 172.21.15.32:0/4015040537 learned_addr learned my addr 172.21.15.32:0/4015040537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc1210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe1ae3700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febdc09d630 0x7febdc120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 -- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febdc09d630 msgr2=0x7febdc120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febdc09d630 0x7febdc120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 -- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 msgr2=0x7febdc120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 0x7febdc120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 -- 172.21.15.32:0/4015040537 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00094a0 con 0x7febdc138d80 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe12e2700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 0x7febdc120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe1ae3700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febdc09d630 0x7febdc120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe22e4700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc1210a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7febd4015000 tx=0x7febd4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4004780 con 0x7febdc138d80 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd40092e0 con 0x7febdc138d80 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd4009b30 con 0x7febdc138d80 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.501+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd401c650 con 0x7febdc138d80 2023-09-30T22:26:26.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.502+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc0a0460 con 0x7febdc138d80 2023-09-30T22:26:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.503+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febd401c7b0 con 0x7febdc138d80 2023-09-30T22:26:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.502+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc124990 con 0x7febdc138d80 2023-09-30T22:26:26.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.503+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7febd4040f00 con 0x7febdc138d80 2023-09-30T22:26:26.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.508+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febd40048e0 con 0x7febdc138d80 2023-09-30T22:26:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.684+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7eff6c072980 con 0x7eff6c133930 2023-09-30T22:26:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.684+0000 7eff62ffd700 1 -- 172.21.15.32:0/4029305600 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff64013680 con 0x7eff6c133930 2023-09-30T22:26:26.685 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.686+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 msgr2=0x7eff6c1c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.686+0000 7eff70bdf700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 0x7eff6c1c8000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff6400e910 tx=0x7eff6400ec20 comp rx=0 tx=0).stop 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.686+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 shutdown_connections 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.687+0000 7eff70bdf700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff6c133930 0x7eff6c1c8000 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.687+0000 7eff70bdf700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff6c09d630 0x7eff6c11f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.687+0000 7eff70bdf700 1 --2- 172.21.15.32:0/4029305600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff6c0a0a30 0x7eff6c11f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.687+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 >> 172.21.15.32:0/4029305600 conn(0x7eff6c0a3050 msgr2=0x7eff6c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.687+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 shutdown_connections 2023-09-30T22:26:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.687+0000 7eff70bdf700 1 -- 172.21.15.32:0/4029305600 wait complete. 2023-09-30T22:26:26.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:26.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.718+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7febdc0747f0 con 0x7febdc138d80 2023-09-30T22:26:26.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.718+0000 7febda7fc700 1 -- 172.21.15.32:0/4015040537 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febd4029070 con 0x7febdc138d80 2023-09-30T22:26:26.719 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 msgr2=0x7febdc1210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc1210a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7febd4015000 tx=0x7febd4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 shutdown_connections 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7febdc138d80 0x7febdc1210a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7febdc09d630 0x7febdc120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 --2- 172.21.15.32:0/4015040537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7febdc0a0a30 0x7febdc120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 >> 172.21.15.32:0/4015040537 conn(0x7febdc0a3050 msgr2=0x7febdc139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 shutdown_connections 2023-09-30T22:26:26.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.721+0000 7febe3d47700 1 -- 172.21.15.32:0/4015040537 wait complete. 2023-09-30T22:26:26.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:26.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:26.811 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 Processor -- start 2023-09-30T22:26:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 -- start start 2023-09-30T22:26:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa690139250 0x7fa69013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69009edd0 con 0x7fa6900a0a30 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6900a18e0 con 0x7fa6900a1510 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.918+0000 7fa694dca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa69009e1f0 con 0x7fa690139250 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa690139250 0x7fa69013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42680/0 (socket says 172.21.15.32:42680) 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 -- 172.21.15.32:0/1095835564 learned_addr learned my addr 172.21.15.32:0/1095835564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 -- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa690139250 msgr2=0x7fa69013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 --2- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa690139250 0x7fa69013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 -- 172.21.15.32:0/1095835564 >> v1:172.21.15.32:6789/0 conn(0x7fa6900a0a30 legacy=0x7fa6900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 -- 172.21.15.32:0/1095835564 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69009dc50 con 0x7fa6900a1510 2023-09-30T22:26:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68ed9d700 1 --2- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa690139250 0x7fa69013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68dd9b700 1 --2- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690138d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa678009920 tx=0x7fa678009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b971636ca0bf572 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68d59a700 1 -- 172.21.15.32:0/1095835564 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678011830 con 0x7fa6900a1510 2023-09-30T22:26:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68d59a700 1 -- 172.21.15.32:0/1095835564 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa678011990 con 0x7fa6900a1510 2023-09-30T22:26:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.919+0000 7fa68d59a700 1 -- 172.21.15.32:0/1095835564 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678011d20 con 0x7fa6900a1510 2023-09-30T22:26:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 -- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 msgr2=0x7fa690138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 --2- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690138d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa678009920 tx=0x7fa678009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 -- 172.21.15.32:0/1095835564 shutdown_connections 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 --2- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa690139250 0x7fa69013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 --2- 172.21.15.32:0/1095835564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690138d10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 -- 172.21.15.32:0/1095835564 >> 172.21.15.32:0/1095835564 conn(0x7fa6900a6550 msgr2=0x7fa6900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 -- 172.21.15.32:0/1095835564 shutdown_connections 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.920+0000 7fa694dca700 1 -- 172.21.15.32:0/1095835564 wait complete. 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.921+0000 7fa694dca700 1 Processor -- start 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.921+0000 7fa694dca700 1 -- start start 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa694dca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 0x7fa690123420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa694dca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690123960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa694dca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690139250 0x7fa6901c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa694dca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6900a0220 con 0x7fa690139250 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa694dca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa690126160 con 0x7fa6900a1510 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa694dca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa690138d40 con 0x7fa6900a0a30 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 0x7fa690123420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690123960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690123960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42696/0 (socket says 172.21.15.32:42696) 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 0x7fa690123420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41162/0 (socket says 172.21.15.32:41162) 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 -- 172.21.15.32:0/493592131 learned_addr learned my addr 172.21.15.32:0/493592131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68ed9d700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690139250 0x7fa6901c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 -- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 msgr2=0x7fa690123420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 0x7fa690123420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 -- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690139250 msgr2=0x7fa6901c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690139250 0x7fa6901c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68dd9b700 1 -- 172.21.15.32:0/493592131 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688009a70 con 0x7fa6900a1510 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68e59c700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 0x7fa690123420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.922+0000 7fa68ed9d700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690139250 0x7fa6901c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.923+0000 7fa68dd9b700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690123960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098f0 tx=0x7fa678004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.923+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678004450 con 0x7fa6900a1510 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.923+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6780045b0 con 0x7fa6900a1510 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.923+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa678009550 con 0x7fa6900a1510 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.923+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780038c0 con 0x7fa6900a1510 2023-09-30T22:26:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.923+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa690126700 con 0x7fa6900a1510 2023-09-30T22:26:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.924+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6900a1010 con 0x7fa6900a1510 2023-09-30T22:26:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.925+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa678003a20 con 0x7fa6900a1510 2023-09-30T22:26:26.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.925+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa678021030 con 0x7fa6900a1510 2023-09-30T22:26:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.933+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa678030480 con 0x7fa6900a1510 2023-09-30T22:26:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.962+0000 7fe91f177700 1 Processor -- start 2023-09-30T22:26:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.962+0000 7fe91f177700 1 -- start start 2023-09-30T22:26:26.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91f177700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 0x7fe91806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91f177700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe91812f8b0 0x7fe91812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91f177700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe918130260 0x7fe918136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91f177700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91812d330 con 0x7fe91812f8b0 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91f177700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91812cd90 con 0x7fe91812cf60 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91f177700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9181306d0 con 0x7fe918130260 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91cf13700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 0x7fe91806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe91812f8b0 0x7fe91812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe91812f8b0 0x7fe91812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59040/0 (socket says 172.21.15.32:59040) 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91cf13700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 0x7fe91806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42710/0 (socket says 172.21.15.32:42710) 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91d714700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe918130260 0x7fe918136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 -- 172.21.15.32:0/975883147 learned_addr learned my addr 172.21.15.32:0/975883147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 -- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe918130260 msgr2=0x7fe918136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe918130260 0x7fe918136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 -- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 msgr2=0x7fe91806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 0x7fe91806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe917fff700 1 -- 172.21.15.32:0/975883147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918071780 con 0x7fe91812f8b0 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91cf13700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 0x7fe91806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.963+0000 7fe91d714700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe918130260 0x7fe918136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe917fff700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91812fd20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00aac0 tx=0x7fe90c00add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6747415fd10e7649 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe9177fe700 1 -- 172.21.15.32:0/975883147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00cb30 con 0x7fe91812f8b0 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe9177fe700 1 -- 172.21.15.32:0/975883147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c00cc90 con 0x7fe91812f8b0 2023-09-30T22:26:26.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe9177fe700 1 -- 172.21.15.32:0/975883147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00f470 con 0x7fe91812f8b0 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 -- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 msgr2=0x7fe91812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91812fd20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00aac0 tx=0x7fe90c00add0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 -- 172.21.15.32:0/975883147 shutdown_connections 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe91812cf60 0x7fe91806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe918130260 0x7fe918136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 --2- 172.21.15.32:0/975883147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91812fd20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.964+0000 7fe91f177700 1 -- 172.21.15.32:0/975883147 >> 172.21.15.32:0/975883147 conn(0x7fe9180a2aa0 msgr2=0x7fe9180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.965+0000 7fe91f177700 1 -- 172.21.15.32:0/975883147 shutdown_connections 2023-09-30T22:26:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.965+0000 7fe91f177700 1 -- 172.21.15.32:0/975883147 wait complete. 2023-09-30T22:26:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.966+0000 7fe91f177700 1 Processor -- start 2023-09-30T22:26:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.966+0000 7fe91f177700 1 -- start start 2023-09-30T22:26:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91f177700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe91812cf60 0x7fe91809ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91f177700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91809bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91f177700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe918130260 0x7fe9181c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91f177700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918129a00 con 0x7fe91812f8b0 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91f177700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91809c4f0 con 0x7fe918130260 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91f177700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe91809b560 con 0x7fe91812cf60 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe917fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91809bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91d714700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe918130260 0x7fe9181c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe917fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91809bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59054/0 (socket says 172.21.15.32:59054) 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe917fff700 1 -- 172.21.15.32:0/2437744975 learned_addr learned my addr 172.21.15.32:0/2437744975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe91cf13700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe91812cf60 0x7fe91809ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.967+0000 7fe917fff700 1 -- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe91812cf60 msgr2=0x7fe91809ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe917fff700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe91812cf60 0x7fe91809ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe917fff700 1 -- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe918130260 msgr2=0x7fe9181c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe917fff700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe918130260 0x7fe9181c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe917fff700 1 -- 172.21.15.32:0/2437744975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9080094a0 con 0x7fe91812f8b0 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe917fff700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91809bfb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe90c019040 tx=0x7fe90c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0036a0 con 0x7fe91812f8b0 2023-09-30T22:26:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c003800 con 0x7fe91812f8b0 2023-09-30T22:26:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe910009a70 con 0x7fe91812f8b0 2023-09-30T22:26:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c003bb0 con 0x7fe91812f8b0 2023-09-30T22:26:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.968+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c00a6f0 con 0x7fe91812f8b0 2023-09-30T22:26:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.969+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe90c003d10 con 0x7fe91812f8b0 2023-09-30T22:26:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.969+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe90c013070 con 0x7fe91812f8b0 2023-09-30T22:26:26.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.969+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91809d920 con 0x7fe91812f8b0 2023-09-30T22:26:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:26.974+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe90c026080 con 0x7fe91812f8b0 2023-09-30T22:26:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.143+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa690074ea0 con 0x7fa6900a1510 2023-09-30T22:26:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.143+0000 7fa686ffd700 1 -- 172.21.15.32:0/493592131 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa678015030 con 0x7fa6900a1510 2023-09-30T22:26:27.145 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 msgr2=0x7fa690123960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690123960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098f0 tx=0x7fa678004750 comp rx=0 tx=0).stop 2023-09-30T22:26:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 shutdown_connections 2023-09-30T22:26:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa690139250 0x7fa6901c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6900a0a30 0x7fa690123420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 --2- 172.21.15.32:0/493592131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6900a1510 0x7fa690123960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 >> 172.21.15.32:0/493592131 conn(0x7fa6900a6550 msgr2=0x7fa6900a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 shutdown_connections 2023-09-30T22:26:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.146+0000 7fa694dca700 1 -- 172.21.15.32:0/493592131 wait complete. 2023-09-30T22:26:27.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:27.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.179+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe918074ea0 con 0x7fe91812f8b0 2023-09-30T22:26:27.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.180+0000 7fe9157fa700 1 -- 172.21.15.32:0/2437744975 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe90c00f470 con 0x7fe91812f8b0 2023-09-30T22:26:27.181 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 msgr2=0x7fe91809bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91809bfb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe90c019040 tx=0x7fe90c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 shutdown_connections 2023-09-30T22:26:27.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe918130260 0x7fe9181c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe91812cf60 0x7fe91809ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 --2- 172.21.15.32:0/2437744975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe91812f8b0 0x7fe91809bfb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 >> 172.21.15.32:0/2437744975 conn(0x7fe9180a2aa0 msgr2=0x7fe91806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 shutdown_connections 2023-09-30T22:26:27.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.182+0000 7fe91f177700 1 -- 172.21.15.32:0/2437744975 wait complete. 2023-09-30T22:26:27.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:27.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:27.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.850+0000 7ff9a8bb5700 1 Processor -- start 2023-09-30T22:26:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.850+0000 7ff9a8bb5700 1 -- start start 2023-09-30T22:26:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a8bb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 0x7ff9a4133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a8bb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 0x7ff9a413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a8bb5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a406bc00 con 0x7ff9a412ff20 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a8bb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a406c570 con 0x7ff9a4130a40 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a8bb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40a0040 con 0x7ff9a4133a80 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 0x7ff9a413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 0x7ff9a4133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9a412ff20 0x7ff9a4130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50032/0 (socket says 172.21.15.32:50032) 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a1d9b700 1 -- 172.21.15.32:0/2968801062 learned_addr learned my addr 172.21.15.32:0/2968801062 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 0x7ff9a4133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41190/0 (socket says 172.21.15.32:41190) 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 0x7ff9a413b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59060/0 (socket says 172.21.15.32:59060) 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 -- 172.21.15.32:0/2968801062 >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 msgr2=0x7ff9a413b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 --2- 172.21.15.32:0/2968801062 >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 0x7ff9a413b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 -- 172.21.15.32:0/2968801062 >> v1:172.21.15.32:6789/0 conn(0x7ff9a412ff20 legacy=0x7ff9a4130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 -- 172.21.15.32:0/2968801062 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4133860 con 0x7ff9a4133a80 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a159a700 1 -- 172.21.15.32:0/2968801062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567280691 0 0) 0x7ff9a406bc00 con 0x7ff9a412ff20 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a2d9d700 1 --2- 172.21.15.32:0/2968801062 >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 0x7ff9a413b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.851+0000 7ff9a259c700 1 --2- 172.21.15.32:0/2968801062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 0x7ff9a4133e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff994009840 tx=0x7ff994009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8846c157998f8c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a159a700 1 -- 172.21.15.32:0/2968801062 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994011830 con 0x7ff9a4133a80 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a159a700 1 -- 172.21.15.32:0/2968801062 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994011990 con 0x7ff9a4133a80 2023-09-30T22:26:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a159a700 1 -- 172.21.15.32:0/2968801062 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994011d20 con 0x7ff9a4133a80 2023-09-30T22:26:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/2968801062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 msgr2=0x7ff9a4133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/2968801062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 0x7ff9a4133e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff994009840 tx=0x7ff994009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/2968801062 shutdown_connections 2023-09-30T22:26:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/2968801062 >> v2:172.21.15.32:3300/0 conn(0x7ff9a4130a40 0x7ff9a413b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/2968801062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a4133a80 0x7ff9a4133e50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.852+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/2968801062 >> 172.21.15.32:0/2968801062 conn(0x7ff9a40a4210 msgr2=0x7ff9a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.853+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/2968801062 shutdown_connections 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.853+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/2968801062 wait complete. 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 Processor -- start 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 -- start start 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 0x7ff9a409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9a4130a40 0x7ff9a409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a4133a80 0x7ff9a409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a411ff80 con 0x7ff9a4133a80 2023-09-30T22:26:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a409e460 con 0x7ff9a4130a40 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a8bb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a409de60 con 0x7ff9a412ff20 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 0x7ff9a409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a4133a80 0x7ff9a409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 0x7ff9a409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41192/0 (socket says 172.21.15.32:41192) 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a259c700 1 -- 172.21.15.32:0/67539854 learned_addr learned my addr 172.21.15.32:0/67539854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.854+0000 7ff9a1d9b700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9a4130a40 0x7ff9a409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a259c700 1 -- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9a4130a40 msgr2=0x7ff9a409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a259c700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9a4130a40 0x7ff9a409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a259c700 1 -- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a4133a80 msgr2=0x7ff9a409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a259c700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a4133a80 0x7ff9a409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a259c700 1 -- 172.21.15.32:0/67539854 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c009a70 con 0x7ff9a412ff20 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a2d9d700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a4133a80 0x7ff9a409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a259c700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 0x7ff9a409c480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff9940090b0 tx=0x7ff994004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9940045d0 con 0x7ff9a412ff20 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99401f070 con 0x7ff9a412ff20 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9940038e0 con 0x7ff9a412ff20 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9940094a0 con 0x7ff9a412ff20 2023-09-30T22:26:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.855+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a411fcc0 con 0x7ff9a412ff20 2023-09-30T22:26:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.856+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff994003a40 con 0x7ff9a412ff20 2023-09-30T22:26:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.856+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4074ea0 con 0x7ff9a412ff20 2023-09-30T22:26:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.856+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff994021070 con 0x7ff9a412ff20 2023-09-30T22:26:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.865+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff994047080 con 0x7ff9a412ff20 2023-09-30T22:26:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 Processor -- start 2023-09-30T22:26:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 -- start start 2023-09-30T22:26:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 0x7f76980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 0x7f769813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769809edd0 con 0x7f76980a1340 2023-09-30T22:26:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769809e1f0 con 0x7f7698134170 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.893+0000 7f769e609700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769809dc50 con 0x7f76980a0a30 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f769cba6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 0x7f769813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 0x7f76980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f769cba6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 0x7f769813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59084/0 (socket says 172.21.15.32:59084) 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f76977fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76980a1340 0x7f7698133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50036/0 (socket says 172.21.15.32:50036) 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 0x7f76980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42744/0 (socket says 172.21.15.32:42744) 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f769cba6700 1 -- 172.21.15.32:0/2651860014 learned_addr learned my addr 172.21.15.32:0/2651860014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 -- 172.21.15.32:0/2651860014 >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 msgr2=0x7f769813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 --2- 172.21.15.32:0/2651860014 >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 0x7f769813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 -- 172.21.15.32:0/2651860014 >> v1:172.21.15.32:6789/0 conn(0x7f76980a1340 legacy=0x7f7698133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 -- 172.21.15.32:0/2651860014 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980a1fe0 con 0x7f76980a0a30 2023-09-30T22:26:27.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f769cba6700 1 --2- 172.21.15.32:0/2651860014 >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 0x7f769813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:27.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7697fff700 1 --2- 172.21.15.32:0/2651860014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 0x7f76980a0e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7688009840 tx=0x7f7688009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca4231ba56c9ad9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:27.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7696ffd700 1 -- 172.21.15.32:0/2651860014 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688011830 con 0x7f76980a0a30 2023-09-30T22:26:27.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7696ffd700 1 -- 172.21.15.32:0/2651860014 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7688011990 con 0x7f76980a0a30 2023-09-30T22:26:27.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.894+0000 7f7696ffd700 1 -- 172.21.15.32:0/2651860014 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688011d20 con 0x7f76980a0a30 2023-09-30T22:26:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 -- 172.21.15.32:0/2651860014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 msgr2=0x7f76980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 --2- 172.21.15.32:0/2651860014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 0x7f76980a0e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7688009840 tx=0x7f7688009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 -- 172.21.15.32:0/2651860014 shutdown_connections 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 --2- 172.21.15.32:0/2651860014 >> v2:172.21.15.32:3300/0 conn(0x7f7698134170 0x7f769813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 --2- 172.21.15.32:0/2651860014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a0a30 0x7f76980a0e00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 -- 172.21.15.32:0/2651860014 >> 172.21.15.32:0/2651860014 conn(0x7f76980a6550 msgr2=0x7f76980a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 -- 172.21.15.32:0/2651860014 shutdown_connections 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.895+0000 7f769e609700 1 -- 172.21.15.32:0/2651860014 wait complete. 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.896+0000 7f769e609700 1 Processor -- start 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.896+0000 7f769e609700 1 -- start start 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769e609700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 0x7f7698122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769e609700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a1340 0x7f7698122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769e609700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 0x7f76981232f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769e609700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76981243f0 con 0x7f7698134170 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769e609700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7698125d60 con 0x7f76980a1340 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769e609700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76981263e0 con 0x7f76980a0a30 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769cba6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 0x7f76981232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f7697fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 0x7f7698122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769cba6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 0x7f76981232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59086/0 (socket says 172.21.15.32:59086) 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769cba6700 1 -- 172.21.15.32:0/1586007098 learned_addr learned my addr 172.21.15.32:0/1586007098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f7697fff700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 0x7f7698122870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41196/0 (socket says 172.21.15.32:41196) 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f76977fe700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a1340 0x7f7698122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f76977fe700 1 -- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 msgr2=0x7f7698122870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f76977fe700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 0x7f7698122870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f76977fe700 1 -- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 msgr2=0x7f76981232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f76977fe700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 0x7f76981232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f76977fe700 1 -- 172.21.15.32:0/1586007098 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009b30 con 0x7f76980a1340 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f7697fff700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 0x7f7698122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.897+0000 7f769cba6700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 0x7f76981232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.898+0000 7f76977fe700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a1340 0x7f7698122db0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f768c001b00 tx=0x7f768c001e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.898+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c017070 con 0x7f76980a1340 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.898+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c005ab0 con 0x7f76980a1340 2023-09-30T22:26:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.898+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76880094a0 con 0x7f76980a1340 2023-09-30T22:26:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.898+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c005e20 con 0x7f76980a1340 2023-09-30T22:26:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.898+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76980a0220 con 0x7f76980a1340 2023-09-30T22:26:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.899+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f768c010850 con 0x7f76980a1340 2023-09-30T22:26:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.899+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f768c047050 con 0x7f76980a1340 2023-09-30T22:26:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.899+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7698074ea0 con 0x7f76980a1340 2023-09-30T22:26:27.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:27.904+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f768c010b00 con 0x7f76980a1340 2023-09-30T22:26:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.072+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff9a406b250 con 0x7ff9a412ff20 2023-09-30T22:26:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.073+0000 7ff992ffd700 1 -- 172.21.15.32:0/67539854 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff994015020 con 0x7ff9a412ff20 2023-09-30T22:26:28.074 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 msgr2=0x7ff9a409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 0x7ff9a409c480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff9940090b0 tx=0x7ff994004750 comp rx=0 tx=0).stop 2023-09-30T22:26:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 shutdown_connections 2023-09-30T22:26:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9a4133a80 0x7ff9a409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9a412ff20 0x7ff9a409c480 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 --2- 172.21.15.32:0/67539854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9a4130a40 0x7ff9a409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 >> 172.21.15.32:0/67539854 conn(0x7ff9a40a4210 msgr2=0x7ff9a409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 shutdown_connections 2023-09-30T22:26:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.075+0000 7ff9a8bb5700 1 -- 172.21.15.32:0/67539854 wait complete. 2023-09-30T22:26:28.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:28.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.109+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f76980a28e0 con 0x7f76980a1340 2023-09-30T22:26:28.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.109+0000 7f7694ff9700 1 -- 172.21.15.32:0/1586007098 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f768c01ab60 con 0x7f76980a1340 2023-09-30T22:26:28.110 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a1340 msgr2=0x7f7698122db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a1340 0x7f7698122db0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f768c001b00 tx=0x7f768c001e10 comp rx=0 tx=0).stop 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 shutdown_connections 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7698134170 0x7f76981232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f76980a0a30 0x7f7698122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 --2- 172.21.15.32:0/1586007098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f76980a1340 0x7f7698122db0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 >> 172.21.15.32:0/1586007098 conn(0x7f76980a6550 msgr2=0x7f769809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 shutdown_connections 2023-09-30T22:26:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.112+0000 7f769e609700 1 -- 172.21.15.32:0/1586007098 wait complete. 2023-09-30T22:26:28.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:28.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:28.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:28.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.311+0000 7fda46c67700 1 Processor -- start 2023-09-30T22:26:28.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.311+0000 7fda46c67700 1 -- start start 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda46c67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 0x7fda40136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda46c67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda40136fe0 0x7fda401393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda46c67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda40070940 con 0x7fda4009e930 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda46c67700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4006fd60 con 0x7fda40136fe0 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda46c67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda4006fa80 con 0x7fda4009f410 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 0x7fda40136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 0x7fda40136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41198/0 (socket says 172.21.15.32:41198) 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 -- 172.21.15.32:0/2911630239 learned_addr learned my addr 172.21.15.32:0/2911630239 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda45204700 1 --2- 172.21.15.32:0/2911630239 >> v2:172.21.15.32:3300/0 conn(0x7fda40136fe0 0x7fda401393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 -- 172.21.15.32:0/2911630239 >> v2:172.21.15.32:3300/0 conn(0x7fda40136fe0 msgr2=0x7fda401393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 --2- 172.21.15.32:0/2911630239 >> v2:172.21.15.32:3300/0 conn(0x7fda40136fe0 0x7fda401393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 -- 172.21.15.32:0/2911630239 >> v1:172.21.15.32:6789/0 conn(0x7fda4009e930 legacy=0x7fda4009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3f7fe700 1 -- 172.21.15.32:0/2911630239 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969656850 0 0) 0x7fda40070940 con 0x7fda4009e930 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.312+0000 7fda3ffff700 1 -- 172.21.15.32:0/2911630239 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4009fb60 con 0x7fda4009f410 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.313+0000 7fda3ffff700 1 --2- 172.21.15.32:0/2911630239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 0x7fda40136aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda300098f0 tx=0x7fda30009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c20bdc842c9973 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.313+0000 7fda3f7fe700 1 -- 172.21.15.32:0/2911630239 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30011830 con 0x7fda4009f410 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.313+0000 7fda3f7fe700 1 -- 172.21.15.32:0/2911630239 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda30011990 con 0x7fda4009f410 2023-09-30T22:26:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.313+0000 7fda3f7fe700 1 -- 172.21.15.32:0/2911630239 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30011d20 con 0x7fda4009f410 2023-09-30T22:26:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 -- 172.21.15.32:0/2911630239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 msgr2=0x7fda40136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 --2- 172.21.15.32:0/2911630239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 0x7fda40136aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda300098f0 tx=0x7fda30009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 -- 172.21.15.32:0/2911630239 shutdown_connections 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 --2- 172.21.15.32:0/2911630239 >> v2:172.21.15.32:3300/0 conn(0x7fda40136fe0 0x7fda401393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 --2- 172.21.15.32:0/2911630239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda4009f410 0x7fda40136aa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 -- 172.21.15.32:0/2911630239 >> 172.21.15.32:0/2911630239 conn(0x7fda400a4210 msgr2=0x7fda400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 -- 172.21.15.32:0/2911630239 shutdown_connections 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.314+0000 7fda46c67700 1 -- 172.21.15.32:0/2911630239 wait complete. 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.315+0000 7fda46c67700 1 Processor -- start 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.315+0000 7fda46c67700 1 -- start start 2023-09-30T22:26:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda46c67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda4009e930 0x7fda401c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda46c67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 0x7fda401c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda46c67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 0x7fda401c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda46c67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda40071d90 con 0x7fda4009f410 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda46c67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4006d640 con 0x7fda4009e930 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda46c67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda40122c40 con 0x7fda40136fe0 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda45204700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 0x7fda401c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 0x7fda401c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 0x7fda401c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59096/0 (socket says 172.21.15.32:59096) 2023-09-30T22:26:28.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda45204700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 0x7fda401c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41202/0 (socket says 172.21.15.32:41202) 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.316+0000 7fda3ffff700 1 -- 172.21.15.32:0/2783876215 learned_addr learned my addr 172.21.15.32:0/2783876215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda44a03700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda4009e930 0x7fda401c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda45204700 1 -- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda4009e930 msgr2=0x7fda401c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda45204700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda4009e930 0x7fda401c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda45204700 1 -- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 msgr2=0x7fda401c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda45204700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 0x7fda401c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda45204700 1 -- 172.21.15.32:0/2783876215 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda30009550 con 0x7fda40136fe0 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda3ffff700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 0x7fda401c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda45204700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 0x7fda401c8c70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda4009abe0 tx=0x7fda2800eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:28.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28013680 con 0x7fda40136fe0 2023-09-30T22:26:28.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda280137e0 con 0x7fda40136fe0 2023-09-30T22:26:28.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda401242f0 con 0x7fda40136fe0 2023-09-30T22:26:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28013b90 con 0x7fda40136fe0 2023-09-30T22:26:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.317+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda401230b0 con 0x7fda40136fe0 2023-09-30T22:26:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.318+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda28013cf0 con 0x7fda40136fe0 2023-09-30T22:26:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.318+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda2803f930 con 0x7fda40136fe0 2023-09-30T22:26:28.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.318+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda40124130 con 0x7fda40136fe0 2023-09-30T22:26:28.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.328+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda28021020 con 0x7fda40136fe0 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.356+0000 7f5dc9b5c700 1 Processor -- start 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.356+0000 7f5dc9b5c700 1 -- start start 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc9b5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dc40a1510 0x7f5dc4138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc9b5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc9b5c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc409edd0 con 0x7f5dc40a0a30 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc9b5c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40a18e0 con 0x7f5dc40a1510 2023-09-30T22:26:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc9b5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc409e1f0 con 0x7f5dc4139250 2023-09-30T22:26:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dc40a1510 0x7f5dc4138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5dc40a0a30 0x7f5dc40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50040/0 (socket says 172.21.15.32:50040) 2023-09-30T22:26:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc37fe700 1 -- 172.21.15.32:0/2673135226 learned_addr learned my addr 172.21.15.32:0/2673135226 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc3fff700 1 --2- 172.21.15.32:0/2673135226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc27fc700 1 -- 172.21.15.32:0/2673135226 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666414359 0 0) 0x7f5dc409edd0 con 0x7f5dc40a0a30 2023-09-30T22:26:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc27fc700 1 -- 172.21.15.32:0/2673135226 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db00035b0 con 0x7f5dc40a0a30 2023-09-30T22:26:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc3fff700 1 -- 172.21.15.32:0/2673135226 >> v2:172.21.15.32:3300/0 conn(0x7f5dc40a1510 msgr2=0x7f5dc4138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc3fff700 1 --2- 172.21.15.32:0/2673135226 >> v2:172.21.15.32:3300/0 conn(0x7f5dc40a1510 0x7f5dc4138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc3fff700 1 -- 172.21.15.32:0/2673135226 >> v1:172.21.15.32:6789/0 conn(0x7f5dc40a0a30 legacy=0x7f5dc40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.357+0000 7f5dc3fff700 1 -- 172.21.15.32:0/2673135226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc409dc50 con 0x7f5dc4139250 2023-09-30T22:26:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc3fff700 1 --2- 172.21.15.32:0/2673135226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc413b630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009840 tx=0x7f5dbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a44a20fd6c9901 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc27fc700 1 -- 172.21.15.32:0/2673135226 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011830 con 0x7f5dc4139250 2023-09-30T22:26:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc27fc700 1 -- 172.21.15.32:0/2673135226 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc011990 con 0x7f5dc4139250 2023-09-30T22:26:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc27fc700 1 -- 172.21.15.32:0/2673135226 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011d20 con 0x7f5dc4139250 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/2673135226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 msgr2=0x7f5dc413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/2673135226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc413b630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009840 tx=0x7f5dbc009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/2673135226 shutdown_connections 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/2673135226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc413b630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/2673135226 >> v2:172.21.15.32:3300/0 conn(0x7f5dc40a1510 0x7f5dc4138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/2673135226 >> 172.21.15.32:0/2673135226 conn(0x7f5dc40a6550 msgr2=0x7f5dc40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/2673135226 shutdown_connections 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.358+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/2673135226 wait complete. 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.359+0000 7f5dc9b5c700 1 Processor -- start 2023-09-30T22:26:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 -- start start 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 0x7f5dc4122700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 0x7f5dc4122c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc4123180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4138030 con 0x7f5dc40a1510 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41385f0 con 0x7f5dc40a0a30 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc9b5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40a5480 con 0x7f5dc4139250 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 0x7f5dc4122700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 0x7f5dc4122c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 0x7f5dc4122c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59112/0 (socket says 172.21.15.32:59112) 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc2ffd700 1 -- 172.21.15.32:0/909213895 learned_addr learned my addr 172.21.15.32:0/909213895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 0x7f5dc4122700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42766/0 (socket says 172.21.15.32:42766) 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.360+0000 7f5dc3fff700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc4123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc2ffd700 1 -- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 msgr2=0x7f5dc4123180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc2ffd700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc4123180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc2ffd700 1 -- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 msgr2=0x7f5dc4122700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc2ffd700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 0x7f5dc4122700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc2ffd700 1 -- 172.21.15.32:0/909213895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0094a0 con 0x7f5dc40a1510 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc37fe700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 0x7f5dc4122700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc2ffd700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 0x7f5dc4122c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e9b0 tx=0x7f5dac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac009710 con 0x7f5dc40a1510 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac00ee50 con 0x7f5dc40a1510 2023-09-30T22:26:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc40a0220 con 0x7f5dc40a1510 2023-09-30T22:26:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00ca50 con 0x7f5dc40a1510 2023-09-30T22:26:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.361+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc4125f40 con 0x7f5dc40a1510 2023-09-30T22:26:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.362+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5dac00cbb0 con 0x7f5dc40a1510 2023-09-30T22:26:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.362+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5dac03f720 con 0x7f5dc40a1510 2023-09-30T22:26:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.362+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4126a70 con 0x7f5dc40a1510 2023-09-30T22:26:28.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.367+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5dac0232f0 con 0x7f5dc40a1510 2023-09-30T22:26:28.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.534+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fda40123050 con 0x7fda40136fe0 2023-09-30T22:26:28.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.535+0000 7fda3d7fa700 1 -- 172.21.15.32:0/2783876215 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda28017070 con 0x7fda40136fe0 2023-09-30T22:26:28.536 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 msgr2=0x7fda401c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 0x7fda401c8c70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda4009abe0 tx=0x7fda2800eb30 comp rx=0 tx=0).stop 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 shutdown_connections 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda40136fe0 0x7fda401c8c70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda4009e930 0x7fda401c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 --2- 172.21.15.32:0/2783876215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda4009f410 0x7fda401c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 >> 172.21.15.32:0/2783876215 conn(0x7fda400a4210 msgr2=0x7fda400a47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:28.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 shutdown_connections 2023-09-30T22:26:28.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.537+0000 7fda46c67700 1 -- 172.21.15.32:0/2783876215 wait complete. 2023-09-30T22:26:28.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.574+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5dc4074ea0 con 0x7f5dc40a1510 2023-09-30T22:26:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.574+0000 7f5dbbfff700 1 -- 172.21.15.32:0/909213895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5dac017020 con 0x7f5dc40a1510 2023-09-30T22:26:28.576 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 msgr2=0x7f5dc4122c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 0x7f5dc4122c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e9b0 tx=0x7f5dac00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 shutdown_connections 2023-09-30T22:26:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dc4139250 0x7f5dc4123180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dc40a0a30 0x7f5dc4122700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 --2- 172.21.15.32:0/909213895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dc40a1510 0x7f5dc4122c40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:28.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 >> 172.21.15.32:0/909213895 conn(0x7f5dc40a6550 msgr2=0x7f5dc411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:28.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 shutdown_connections 2023-09-30T22:26:28.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:28.577+0000 7f5dc9b5c700 1 -- 172.21.15.32:0/909213895 wait complete. 2023-09-30T22:26:28.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:29.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:29.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 Processor -- start 2023-09-30T22:26:29.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 -- start start 2023-09-30T22:26:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a1510 0x7f7f14138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f14139250 0x7f7f1413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1409edd0 con 0x7f7f140a0a30 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140a18e0 con 0x7f7f140a1510 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.240+0000 7f7f1b624700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1409e1f0 con 0x7f7f14139250 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f193c0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f140a0a30 0x7f7f140a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50052/0 (socket says 172.21.15.32:50052) 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f19bc1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f14139250 0x7f7f1413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f193c0700 1 -- 172.21.15.32:0/808069033 learned_addr learned my addr 172.21.15.32:0/808069033 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f18bbf700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a1510 0x7f7f14138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f0bfff700 1 -- 172.21.15.32:0/808069033 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809619570 0 0) 0x7f7f1409edd0 con 0x7f7f140a0a30 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f0bfff700 1 -- 172.21.15.32:0/808069033 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f000035b0 con 0x7f7f140a0a30 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f18bbf700 1 -- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f14139250 msgr2=0x7f7f1413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:29.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f18bbf700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f14139250 0x7f7f1413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f18bbf700 1 -- 172.21.15.32:0/808069033 >> v1:172.21.15.32:6789/0 conn(0x7f7f140a0a30 legacy=0x7f7f140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f18bbf700 1 -- 172.21.15.32:0/808069033 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1409dc50 con 0x7f7f140a1510 2023-09-30T22:26:29.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f19bc1700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f14139250 0x7f7f1413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f18bbf700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a1510 0x7f7f14138d10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009870 tx=0x7f7f04009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a969b7d991a62c56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f0bfff700 1 -- 172.21.15.32:0/808069033 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011830 con 0x7f7f140a1510 2023-09-30T22:26:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f0bfff700 1 -- 172.21.15.32:0/808069033 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04011990 con 0x7f7f140a1510 2023-09-30T22:26:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.241+0000 7f7f0bfff700 1 -- 172.21.15.32:0/808069033 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011d20 con 0x7f7f140a1510 2023-09-30T22:26:29.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 -- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a1510 msgr2=0x7f7f14138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a1510 0x7f7f14138d10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009870 tx=0x7f7f04009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 -- 172.21.15.32:0/808069033 shutdown_connections 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f14139250 0x7f7f1413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 --2- 172.21.15.32:0/808069033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a1510 0x7f7f14138d10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 -- 172.21.15.32:0/808069033 >> 172.21.15.32:0/808069033 conn(0x7f7f140a6550 msgr2=0x7f7f140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 -- 172.21.15.32:0/808069033 shutdown_connections 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.242+0000 7f7f1b624700 1 -- 172.21.15.32:0/808069033 wait complete. 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 Processor -- start 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 -- start start 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a0a30 0x7f7f14122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 0x7f7f14122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f14139250 0x7f7f14123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14124390 con 0x7f7f14139250 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141380b0 con 0x7f7f140a0a30 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f1b624700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14124c00 con 0x7f7f140a1510 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f18bbf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 0x7f7f14122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f193c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a0a30 0x7f7f14122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f18bbf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 0x7f7f14122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41244/0 (socket says 172.21.15.32:41244) 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f18bbf700 1 -- 172.21.15.32:0/553551120 learned_addr learned my addr 172.21.15.32:0/553551120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.244+0000 7f7f19bc1700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f14139250 0x7f7f14123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f193c0700 1 -- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 msgr2=0x7f7f14122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f193c0700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 0x7f7f14122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f193c0700 1 -- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f14139250 msgr2=0x7f7f14123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f193c0700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f14139250 0x7f7f14123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f193c0700 1 -- 172.21.15.32:0/553551120 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009a70 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f18bbf700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 0x7f7f14122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f193c0700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a0a30 0x7f7f14122780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c490 tx=0x7f7f1000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10005d70 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100070d0 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f040094a0 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10007440 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.245+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14138670 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.246+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f100075a0 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.246+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f10019af0 con 0x7f7f140a0a30 2023-09-30T22:26:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.247+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14125d60 con 0x7f7f140a0a30 2023-09-30T22:26:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.251+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f14125d60 con 0x7f7f140a0a30 2023-09-30T22:26:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.278+0000 7f1ee12fa700 1 Processor -- start 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.278+0000 7f1ee12fa700 1 -- start start 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.278+0000 7f1ee12fa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1edc09ffb0 0x7f1edc09f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ee12fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc13b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ee12fa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0a06b0 con 0x7f1edc09e850 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ee12fa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1edc06b920 con 0x7f1edc09ffb0 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ee12fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc09aa30 con 0x7f1edc138e30 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1eda7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1edc09ffb0 0x7f1edc09f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1edaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1edc09e850 0x7f1edc09ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50062/0 (socket says 172.21.15.32:50062) 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1edaffd700 1 -- 172.21.15.32:0/3108252026 learned_addr learned my addr 172.21.15.32:0/3108252026 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1edb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc13b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1eda7fc700 1 -- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 msgr2=0x7f1edc13b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1eda7fc700 1 --2- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc13b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1eda7fc700 1 -- 172.21.15.32:0/3108252026 >> v1:172.21.15.32:6789/0 conn(0x7f1edc09e850 legacy=0x7f1edc09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1eda7fc700 1 -- 172.21.15.32:0/3108252026 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc06caa0 con 0x7f1edc09ffb0 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/3108252026 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609288223 0 0) 0x7f1edc0a06b0 con 0x7f1edc09e850 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1eda7fc700 1 --2- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09ffb0 0x7f1edc09f330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009840 tx=0x7f1ec4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc510c9de8f9d1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/3108252026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4011830 con 0x7f1edc09ffb0 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.279+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/3108252026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec4011990 con 0x7f1edc09ffb0 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ed9ffb700 1 -- 172.21.15.32:0/3108252026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4011d20 con 0x7f1edc09ffb0 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 -- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09ffb0 msgr2=0x7f1edc09f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09ffb0 0x7f1edc09f330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009840 tx=0x7f1ec4009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 -- 172.21.15.32:0/3108252026 shutdown_connections 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc13b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/3108252026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09ffb0 0x7f1edc09f330 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 -- 172.21.15.32:0/3108252026 >> 172.21.15.32:0/3108252026 conn(0x7f1edc0a3f70 msgr2=0x7f1edc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 -- 172.21.15.32:0/3108252026 shutdown_connections 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.280+0000 7f1ee12fa700 1 -- 172.21.15.32:0/3108252026 wait complete. 2023-09-30T22:26:29.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.281+0000 7f1ee12fa700 1 Processor -- start 2023-09-30T22:26:29.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.281+0000 7f1ee12fa700 1 -- start start 2023-09-30T22:26:29.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1ee12fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09e850 0x7f1edc1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1ee12fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 0x7f1edc1c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1ee12fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc1c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1ee12fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc1343c0 con 0x7f1edc09e850 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1ee12fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc09dd80 con 0x7f1edc09ffb0 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1ee12fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc135810 con 0x7f1edc138e30 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 0x7f1edc1c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1edb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc1c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 0x7f1edc1c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42800/0 (socket says 172.21.15.32:42800) 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 -- 172.21.15.32:0/2700323802 learned_addr learned my addr 172.21.15.32:0/2700323802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1edaffd700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09e850 0x7f1edc1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 -- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 msgr2=0x7f1edc1c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc1c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 -- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09e850 msgr2=0x7f1edc1c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09e850 0x7f1edc1c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1eda7fc700 1 -- 172.21.15.32:0/2700323802 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40094a0 con 0x7f1edc09ffb0 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1edb7fe700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc1c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.282+0000 7f1edaffd700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09e850 0x7f1edc1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.283+0000 7f1eda7fc700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 0x7f1edc1c8ed0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009fd0 tx=0x7f1ec4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.283+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40036a0 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.283+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec401f070 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.283+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc134720 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.283+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4003a60 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.283+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1edc135530 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.284+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1edc134f80 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.284+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ec4003bc0 con 0x7f1edc09ffb0 2023-09-30T22:26:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.284+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1ec4021070 con 0x7f1edc09ffb0 2023-09-30T22:26:29.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.289+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1edc134f80 con 0x7f1edc09ffb0 2023-09-30T22:26:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.457+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f14125d60 con 0x7f7f140a0a30 2023-09-30T22:26:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.458+0000 7f7f09ffb700 1 -- 172.21.15.32:0/553551120 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f14125d60 con 0x7f7f140a0a30 2023-09-30T22:26:29.459 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a0a30 msgr2=0x7f7f14122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a0a30 0x7f7f14122780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000c490 tx=0x7f7f1000c7a0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 shutdown_connections 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f14139250 0x7f7f14123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f140a0a30 0x7f7f14122780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 --2- 172.21.15.32:0/553551120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f140a1510 0x7f7f14122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 >> 172.21.15.32:0/553551120 conn(0x7f7f140a6550 msgr2=0x7f7f1409aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 shutdown_connections 2023-09-30T22:26:29.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.460+0000 7f7f1b624700 1 -- 172.21.15.32:0/553551120 wait complete. 2023-09-30T22:26:29.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:29.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.493+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1edc1351d0 con 0x7f1edc09ffb0 2023-09-30T22:26:29.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.493+0000 7f1ed37fe700 1 -- 172.21.15.32:0/2700323802 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1edc1351d0 con 0x7f1edc09ffb0 2023-09-30T22:26:29.494 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:29.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 msgr2=0x7f1edc1c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 0x7f1edc1c8ed0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009fd0 tx=0x7f1ec4004750 comp rx=0 tx=0).stop 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 shutdown_connections 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1edc138e30 0x7f1edc1c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1edc09e850 0x7f1edc1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 --2- 172.21.15.32:0/2700323802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1edc09ffb0 0x7f1edc1c8ed0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.495+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 >> 172.21.15.32:0/2700323802 conn(0x7f1edc0a3f70 msgr2=0x7f1edc13a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.496+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 shutdown_connections 2023-09-30T22:26:29.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.496+0000 7f1ee12fa700 1 -- 172.21.15.32:0/2700323802 wait complete. 2023-09-30T22:26:29.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:29.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:29.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:29.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.694+0000 7f98a4231700 1 Processor -- start 2023-09-30T22:26:29.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.694+0000 7f98a4231700 1 -- start start 2023-09-30T22:26:29.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a4231700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a4231700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 0x7f989c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a4231700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c09edd0 con 0x7f989c0a1340 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a4231700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c09e1f0 con 0x7f989c134170 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a4231700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f989c09dc50 con 0x7f989c0a0a30 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a27ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 0x7f989c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41284/0 (socket says 172.21.15.32:41284) 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a27ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 0x7f989c13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59144/0 (socket says 172.21.15.32:59144) 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a17cc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f989c0a1340 0x7f989c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50078/0 (socket says 172.21.15.32:50078) 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 -- 172.21.15.32:0/2292109427 learned_addr learned my addr 172.21.15.32:0/2292109427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:29.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 -- 172.21.15.32:0/2292109427 >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 msgr2=0x7f989c13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 --2- 172.21.15.32:0/2292109427 >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 0x7f989c13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 -- 172.21.15.32:0/2292109427 >> v1:172.21.15.32:6789/0 conn(0x7f989c0a1340 legacy=0x7f989c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a1fcd700 1 -- 172.21.15.32:0/2292109427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0a1fe0 con 0x7f989c0a0a30 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.695+0000 7f98a27ce700 1 --2- 172.21.15.32:0/2292109427 >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 0x7f989c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a1fcd700 1 --2- 172.21.15.32:0/2292109427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c0a0e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98940098f0 tx=0x7f9894009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b93e87bf0b8cc56a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a0fcb700 1 -- 172.21.15.32:0/2292109427 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894011830 con 0x7f989c0a0a30 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a0fcb700 1 -- 172.21.15.32:0/2292109427 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894011990 con 0x7f989c0a0a30 2023-09-30T22:26:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a0fcb700 1 -- 172.21.15.32:0/2292109427 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894011d20 con 0x7f989c0a0a30 2023-09-30T22:26:29.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a4231700 1 -- 172.21.15.32:0/2292109427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 msgr2=0x7f989c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a4231700 1 --2- 172.21.15.32:0/2292109427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c0a0e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98940098f0 tx=0x7f9894009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:29.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.696+0000 7f98a4231700 1 -- 172.21.15.32:0/2292109427 shutdown_connections 2023-09-30T22:26:29.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.697+0000 7f98a4231700 1 --2- 172.21.15.32:0/2292109427 >> v2:172.21.15.32:3300/0 conn(0x7f989c134170 0x7f989c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.697+0000 7f98a4231700 1 --2- 172.21.15.32:0/2292109427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c0a0e00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.697+0000 7f98a4231700 1 -- 172.21.15.32:0/2292109427 >> 172.21.15.32:0/2292109427 conn(0x7f989c0a6550 msgr2=0x7f989c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.697+0000 7f98a4231700 1 -- 172.21.15.32:0/2292109427 shutdown_connections 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.697+0000 7f98a4231700 1 -- 172.21.15.32:0/2292109427 wait complete. 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 Processor -- start 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 -- start start 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f989c0a1340 0x7f989c122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c134170 0x7f989c123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c124690 con 0x7f989c134170 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0a4f40 con 0x7f989c0a1340 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a4231700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f989c126420 con 0x7f989c0a0a30 2023-09-30T22:26:29.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a1fcd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a17cc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f989c0a1340 0x7f989c122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a1fcd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41296/0 (socket says 172.21.15.32:41296) 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a1fcd700 1 -- 172.21.15.32:0/3025247295 learned_addr learned my addr 172.21.15.32:0/3025247295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.698+0000 7f98a27ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c134170 0x7f989c123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a1fcd700 1 -- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f989c0a1340 msgr2=0x7f989c122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a1fcd700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f989c0a1340 0x7f989c122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a1fcd700 1 -- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c134170 msgr2=0x7f989c123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a1fcd700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c134170 0x7f989c123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a1fcd700 1 -- 172.21.15.32:0/3025247295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009a70 con 0x7f989c0a0a30 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a17cc700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f989c0a1340 0x7f989c122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a27ce700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c134170 0x7f989c123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a1fcd700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c122900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940043c0 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989401f070 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9894009550 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940038c0 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.699+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c133420 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.700+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9894021030 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.700+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f98940408f0 con 0x7f989c0a0a30 2023-09-30T22:26:29.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.700+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c0a1d40 con 0x7f989c0a0a30 2023-09-30T22:26:29.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.705+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9894015080 con 0x7f989c0a0a30 2023-09-30T22:26:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.735+0000 7fedcc8d0700 1 Processor -- start 2023-09-30T22:26:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.735+0000 7fedcc8d0700 1 -- start start 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedcc8d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 0x7fedc80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedcc8d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 0x7fedc80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedcc8d0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80a2a70 con 0x7fedc8133930 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedcc8d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80a01d0 con 0x7fedc80a0a30 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedcc8d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc809e2b0 con 0x7fedc809d630 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedc659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 0x7fedc80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedc5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 0x7fedc80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedc659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 0x7fedc80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42816/0 (socket says 172.21.15.32:42816) 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedc659c700 1 -- 172.21.15.32:0/4183575300 learned_addr learned my addr 172.21.15.32:0/4183575300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedc5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 0x7fedc80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:41300/0 (socket says 172.21.15.32:41300) 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.736+0000 7fedc6d9d700 1 --1- 172.21.15.32:0/4183575300 >> v1:172.21.15.32:6789/0 conn(0x7fedc8133930 0x7fedc8133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50080/0 (socket says 172.21.15.32:50080) 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc659c700 1 -- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 msgr2=0x7fedc80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc659c700 1 --2- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 0x7fedc80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc659c700 1 -- 172.21.15.32:0/4183575300 >> v1:172.21.15.32:6789/0 conn(0x7fedc8133930 legacy=0x7fedc8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc559a700 1 -- 172.21.15.32:0/4183575300 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111621252 0 0) 0x7fedc80a2a70 con 0x7fedc8133930 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc659c700 1 -- 172.21.15.32:0/4183575300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc809dc50 con 0x7fedc80a0a30 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc5d9b700 1 --2- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 0x7fedc80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc659c700 1 --2- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 0x7fedc80a0e00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009920 tx=0x7fedc0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25c496da843fc7a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc559a700 1 -- 172.21.15.32:0/4183575300 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011830 con 0x7fedc80a0a30 2023-09-30T22:26:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc559a700 1 -- 172.21.15.32:0/4183575300 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0011990 con 0x7fedc80a0a30 2023-09-30T22:26:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.737+0000 7fedc559a700 1 -- 172.21.15.32:0/4183575300 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011d20 con 0x7fedc80a0a30 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 -- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 msgr2=0x7fedc80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 0x7fedc80a0e00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009920 tx=0x7fedc0009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 -- 172.21.15.32:0/4183575300 shutdown_connections 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc80a0a30 0x7fedc80a0e00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/4183575300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc809d630 0x7fedc80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 -- 172.21.15.32:0/4183575300 >> 172.21.15.32:0/4183575300 conn(0x7fedc80a3050 msgr2=0x7fedc80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 -- 172.21.15.32:0/4183575300 shutdown_connections 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.738+0000 7fedcc8d0700 1 -- 172.21.15.32:0/4183575300 wait complete. 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.739+0000 7fedcc8d0700 1 Processor -- start 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.739+0000 7fedcc8d0700 1 -- start start 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 0x7fedc81c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc80a0a30 0x7fedc81c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc8133930 0x7fedc81c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81208e0 con 0x7fedc80a0a30 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81223c0 con 0x7fedc809d630 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81206b0 con 0x7fedc8133930 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 0x7fedc81c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc8133930 0x7fedc81c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 0x7fedc81c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42826/0 (socket says 172.21.15.32:42826) 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 -- 172.21.15.32:0/894252846 learned_addr learned my addr 172.21.15.32:0/894252846 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc5d9b700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc80a0a30 0x7fedc81c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 -- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc8133930 msgr2=0x7fedc81c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc8133930 0x7fedc81c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 -- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc80a0a30 msgr2=0x7fedc81c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc80a0a30 0x7fedc81c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 -- 172.21.15.32:0/894252846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00094a0 con 0x7fedc809d630 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedc659c700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 0x7fedc81c8020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fedc80a2c00 tx=0x7fedc0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0004450 con 0x7fedc809d630 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc00045b0 con 0x7fedc809d630 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.740+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc0009550 con 0x7fedc809d630 2023-09-30T22:26:29.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.741+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc00038c0 con 0x7fedc809d630 2023-09-30T22:26:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.741+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc8120ea0 con 0x7fedc809d630 2023-09-30T22:26:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.741+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fedc0003a20 con 0x7fedc809d630 2023-09-30T22:26:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.741+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fedc0021030 con 0x7fedc809d630 2023-09-30T22:26:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.742+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc81335b0 con 0x7fedc809d630 2023-09-30T22:26:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.746+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fedc001a430 con 0x7fedc809d630 2023-09-30T22:26:29.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.911+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f989c074ea0 con 0x7f989c0a0a30 2023-09-30T22:26:29.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.912+0000 7f989a7fc700 1 -- 172.21.15.32:0/3025247295 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9894004560 con 0x7f989c0a0a30 2023-09-30T22:26:29.913 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 msgr2=0x7f989c122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c122900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894004750 comp rx=0 tx=0).stop 2023-09-30T22:26:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 shutdown_connections 2023-09-30T22:26:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989c134170 0x7f989c123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f989c0a0a30 0x7f989c122900 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 --2- 172.21.15.32:0/3025247295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f989c0a1340 0x7f989c122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 >> 172.21.15.32:0/3025247295 conn(0x7f989c0a6550 msgr2=0x7f989c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 shutdown_connections 2023-09-30T22:26:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.914+0000 7f98a4231700 1 -- 172.21.15.32:0/3025247295 wait complete. 2023-09-30T22:26:29.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:29.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.950+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fedc80747f0 con 0x7fedc809d630 2023-09-30T22:26:29.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.951+0000 7fedbeffd700 1 -- 172.21.15.32:0/894252846 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fedc002b030 con 0x7fedc809d630 2023-09-30T22:26:29.952 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 msgr2=0x7fedc81c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 0x7fedc81c8020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fedc80a2c00 tx=0x7fedc0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 shutdown_connections 2023-09-30T22:26:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedc8133930 0x7fedc81c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedc809d630 0x7fedc81c8020 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 --2- 172.21.15.32:0/894252846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedc80a0a30 0x7fedc81c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 >> 172.21.15.32:0/894252846 conn(0x7fedc80a3050 msgr2=0x7fedc81bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 shutdown_connections 2023-09-30T22:26:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:29.953+0000 7fedcc8d0700 1 -- 172.21.15.32:0/894252846 wait complete. 2023-09-30T22:26:29.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:30.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:30.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:30.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.624+0000 7f1560331700 1 Processor -- start 2023-09-30T22:26:30.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.624+0000 7f1560331700 1 -- start start 2023-09-30T22:26:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.624+0000 7f1560331700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155809d630 0x7f15580a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.624+0000 7f1560331700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1558138d80 0x7f155813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f1560331700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580a2a70 con 0x7f15580a0a30 2023-09-30T22:26:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f1560331700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15580a01d0 con 0x7f155809d630 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f1560331700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f155809e2b0 con 0x7f1558138d80 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e0cd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15580a0a30 0x7f15580a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43296/0 (socket says 172.21.15.32:43296) 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e0cd700 1 -- 172.21.15.32:0/1380233754 learned_addr learned my addr 172.21.15.32:0/1380233754 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e8ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1558138d80 0x7f155813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155d8cc700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155809d630 0x7f15580a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e8ce700 1 -- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155809d630 msgr2=0x7f15580a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e8ce700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155809d630 0x7f15580a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e8ce700 1 -- 172.21.15.32:0/1380233754 >> v1:172.21.15.32:6789/0 conn(0x7f15580a0a30 legacy=0x7f15580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155d0cb700 1 -- 172.21.15.32:0/1380233754 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514153357 0 0) 0x7f15580a2a70 con 0x7f15580a0a30 2023-09-30T22:26:30.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e8ce700 1 -- 172.21.15.32:0/1380233754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155809dc50 con 0x7f1558138d80 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155d8cc700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155809d630 0x7f15580a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155e8ce700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1558138d80 0x7f155813b160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1550009dc0 tx=0x7f155000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=557e72886b2d257 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.625+0000 7f155d0cb700 1 -- 172.21.15.32:0/1380233754 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155000e070 con 0x7f1558138d80 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f155d0cb700 1 -- 172.21.15.32:0/1380233754 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155000bb20 con 0x7f1558138d80 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f155d0cb700 1 -- 172.21.15.32:0/1380233754 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550012430 con 0x7f1558138d80 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 -- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1558138d80 msgr2=0x7f155813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1558138d80 0x7f155813b160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1550009dc0 tx=0x7f155000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 -- 172.21.15.32:0/1380233754 shutdown_connections 2023-09-30T22:26:30.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1558138d80 0x7f155813b160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 --2- 172.21.15.32:0/1380233754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155809d630 0x7f15580a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 -- 172.21.15.32:0/1380233754 >> 172.21.15.32:0/1380233754 conn(0x7f15580a3050 msgr2=0x7f15580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:30.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 -- 172.21.15.32:0/1380233754 shutdown_connections 2023-09-30T22:26:30.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.626+0000 7f1560331700 1 -- 172.21.15.32:0/1380233754 wait complete. 2023-09-30T22:26:30.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.627+0000 7f1560331700 1 Processor -- start 2023-09-30T22:26:30.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.627+0000 7f1560331700 1 -- start start 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f1560331700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155809d630 0x7f1558133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f1560331700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 0x7f1558133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f1560331700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1558138d80 0x7f15581341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f1560331700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1558134c40 con 0x7f155809d630 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f1560331700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15581361b0 con 0x7f1558138d80 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f1560331700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1558136ed0 con 0x7f15580a0a30 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 0x7f1558133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 0x7f1558133cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49360/0 (socket says 172.21.15.32:49360) 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 -- 172.21.15.32:0/2327811483 learned_addr learned my addr 172.21.15.32:0/2327811483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155e0cd700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155809d630 0x7f1558133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155e8ce700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1558138d80 0x7f15581341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 -- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1558138d80 msgr2=0x7f15581341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1558138d80 0x7f15581341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 -- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155809d630 msgr2=0x7f1558133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155809d630 0x7f1558133770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.628+0000 7f155d8cc700 1 -- 172.21.15.32:0/2327811483 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480094a0 con 0x7f15580a0a30 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.629+0000 7f155d8cc700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 0x7f1558133cb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f154800ebe0 tx=0x7f154800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.629+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154800c7f0 con 0x7f15580a0a30 2023-09-30T22:26:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.629+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154800c950 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.629+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1550009a70 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.629+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154800cce0 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.629+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15580a0410 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.630+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15580747f0 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.630+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1548013470 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.630+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f154803ff90 con 0x7f15580a0a30 2023-09-30T22:26:30.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.635+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1548013720 con 0x7f15580a0a30 2023-09-30T22:26:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.661+0000 7f1644f23700 1 Processor -- start 2023-09-30T22:26:30.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f1644f23700 1 -- start start 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f1644f23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 0x7f16400a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f1644f23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 0x7f164013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f1644f23700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400a2a70 con 0x7f16400a0a30 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f1644f23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400a01d0 con 0x7f164009d630 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f1644f23700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164009e2b0 con 0x7f1640138d80 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f163dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 0x7f16400a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f163ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 0x7f164013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f163ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 0x7f164013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44996/0 (socket says 172.21.15.32:44996) 2023-09-30T22:26:30.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f163dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 0x7f16400a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60846/0 (socket says 172.21.15.32:60846) 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f163e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16400a0a30 0x7f16400a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43304/0 (socket says 172.21.15.32:43304) 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.662+0000 7f163ed9d700 1 -- 172.21.15.32:0/1495479617 learned_addr learned my addr 172.21.15.32:0/1495479617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163ed9d700 1 -- 172.21.15.32:0/1495479617 >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 msgr2=0x7f16400a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163ed9d700 1 --2- 172.21.15.32:0/1495479617 >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 0x7f16400a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163ed9d700 1 -- 172.21.15.32:0/1495479617 >> v1:172.21.15.32:6789/0 conn(0x7f16400a0a30 legacy=0x7f16400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163ed9d700 1 -- 172.21.15.32:0/1495479617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164009dc50 con 0x7f1640138d80 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163dd9b700 1 --2- 172.21.15.32:0/1495479617 >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 0x7f16400a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163ed9d700 1 --2- 172.21.15.32:0/1495479617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 0x7f164013b160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16380098f0 tx=0x7f1638009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f2c4e870f1468b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163d59a700 1 -- 172.21.15.32:0/1495479617 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638011830 con 0x7f1640138d80 2023-09-30T22:26:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163d59a700 1 -- 172.21.15.32:0/1495479617 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638011990 con 0x7f1640138d80 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.663+0000 7f163d59a700 1 -- 172.21.15.32:0/1495479617 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638011d20 con 0x7f1640138d80 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 -- 172.21.15.32:0/1495479617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 msgr2=0x7f164013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 --2- 172.21.15.32:0/1495479617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 0x7f164013b160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16380098f0 tx=0x7f1638009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 -- 172.21.15.32:0/1495479617 shutdown_connections 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 --2- 172.21.15.32:0/1495479617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640138d80 0x7f164013b160 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 --2- 172.21.15.32:0/1495479617 >> v2:172.21.15.32:3300/0 conn(0x7f164009d630 0x7f16400a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 -- 172.21.15.32:0/1495479617 >> 172.21.15.32:0/1495479617 conn(0x7f16400a3050 msgr2=0x7f16400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 -- 172.21.15.32:0/1495479617 shutdown_connections 2023-09-30T22:26:30.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.664+0000 7f1644f23700 1 -- 172.21.15.32:0/1495479617 wait complete. 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.665+0000 7f1644f23700 1 Processor -- start 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 -- start start 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f164009d630 0x7f1640120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 0x7f16401210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1640124140 con 0x7f16400a0a30 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16401234f0 con 0x7f164009d630 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f1644f23700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1640138b20 con 0x7f1640138d80 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f163ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 0x7f16401210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f163dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60854/0 (socket says 172.21.15.32:60854) 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f163e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f164009d630 0x7f1640120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f163dd9b700 1 -- 172.21.15.32:0/3280778088 learned_addr learned my addr 172.21.15.32:0/3280778088 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.666+0000 7f163ed9d700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 0x7f16401210a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49368/0 (socket says 172.21.15.32:49368) 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163ed9d700 1 -- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f164009d630 msgr2=0x7f1640120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163ed9d700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f164009d630 0x7f1640120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163ed9d700 1 -- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 msgr2=0x7f1640120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163ed9d700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163ed9d700 1 -- 172.21.15.32:0/3280778088 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340094a0 con 0x7f1640138d80 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163dd9b700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f163ed9d700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 0x7f16401210a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1638016000 tx=0x7f1638004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638004510 con 0x7f1640138d80 2023-09-30T22:26:30.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16380036a0 con 0x7f1640138d80 2023-09-30T22:26:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1638009550 con 0x7f1640138d80 2023-09-30T22:26:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638003a50 con 0x7f1640138d80 2023-09-30T22:26:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.667+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16400a0460 con 0x7f1640138d80 2023-09-30T22:26:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.668+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1638003bb0 con 0x7f1640138d80 2023-09-30T22:26:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.668+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1638040370 con 0x7f1640138d80 2023-09-30T22:26:30.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.668+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1640124990 con 0x7f1640138d80 2023-09-30T22:26:30.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.678+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1638015080 con 0x7f1640138d80 2023-09-30T22:26:30.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.847+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1558072980 con 0x7f15580a0a30 2023-09-30T22:26:30.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.847+0000 7f154effd700 1 -- 172.21.15.32:0/2327811483 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f154802a090 con 0x7f15580a0a30 2023-09-30T22:26:30.848 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.849+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 msgr2=0x7f1558133cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.849+0000 7f1560331700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 0x7f1558133cb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f154800ebe0 tx=0x7f154800eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 shutdown_connections 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1558138d80 0x7f15581341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f155809d630 0x7f1558133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 --2- 172.21.15.32:0/2327811483 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f15580a0a30 0x7f1558133cb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 >> 172.21.15.32:0/2327811483 conn(0x7f15580a3050 msgr2=0x7f15580a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:30.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 shutdown_connections 2023-09-30T22:26:30.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.850+0000 7f1560331700 1 -- 172.21.15.32:0/2327811483 wait complete. 2023-09-30T22:26:30.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:30.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.883+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f16400747f0 con 0x7f1640138d80 2023-09-30T22:26:30.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.884+0000 7f162effd700 1 -- 172.21.15.32:0/3280778088 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f163801a420 con 0x7f1640138d80 2023-09-30T22:26:30.885 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 msgr2=0x7f16401210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 0x7f16401210a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1638016000 tx=0x7f1638004750 comp rx=0 tx=0).stop 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 shutdown_connections 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1640138d80 0x7f16401210a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f164009d630 0x7f1640120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 --2- 172.21.15.32:0/3280778088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:30.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 >> 172.21.15.32:0/3280778088 conn(0x7f16400a3050 msgr2=0x7f1640139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:30.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 shutdown_connections 2023-09-30T22:26:30.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:30.886+0000 7f1644f23700 1 -- 172.21.15.32:0/3280778088 wait complete. 2023-09-30T22:26:30.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:30.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:30.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.085+0000 7efcaec73700 1 Processor -- start 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.085+0000 7efcaec73700 1 -- start start 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcaec73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca80a0a30 0x7efca80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcaec73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca8138d80 0x7efca813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcaec73700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca80a2a70 con 0x7efca809d630 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcaec73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80a01d0 con 0x7efca8138d80 2023-09-30T22:26:31.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcaec73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efca809e2b0 con 0x7efca80a0a30 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efca7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efca809d630 0x7efca80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43316/0 (socket says 172.21.15.32:43316) 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcaca0f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca80a0a30 0x7efca80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efca7fff700 1 -- 172.21.15.32:0/217208311 learned_addr learned my addr 172.21.15.32:0/217208311 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.086+0000 7efcad210700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca8138d80 0x7efca813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efca77fe700 1 -- 172.21.15.32:0/217208311 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869225617 0 0) 0x7efca80a2a70 con 0x7efca809d630 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efcaca0f700 1 -- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca8138d80 msgr2=0x7efca813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efcaca0f700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca8138d80 0x7efca813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efcaca0f700 1 -- 172.21.15.32:0/217208311 >> v1:172.21.15.32:6789/0 conn(0x7efca809d630 legacy=0x7efca80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efcaca0f700 1 -- 172.21.15.32:0/217208311 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca809dc50 con 0x7efca80a0a30 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efcad210700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca8138d80 0x7efca813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efcaca0f700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca80a0a30 0x7efca80a0e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc98009870 tx=0x7efc98009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f58522741c9bbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:31.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efca77fe700 1 -- 172.21.15.32:0/217208311 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011830 con 0x7efca80a0a30 2023-09-30T22:26:31.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efca77fe700 1 -- 172.21.15.32:0/217208311 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98011990 con 0x7efca80a0a30 2023-09-30T22:26:31.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.087+0000 7efca77fe700 1 -- 172.21.15.32:0/217208311 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011d20 con 0x7efca80a0a30 2023-09-30T22:26:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 -- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca80a0a30 msgr2=0x7efca80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca80a0a30 0x7efca80a0e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc98009870 tx=0x7efc98009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 -- 172.21.15.32:0/217208311 shutdown_connections 2023-09-30T22:26:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca8138d80 0x7efca813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 --2- 172.21.15.32:0/217208311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca80a0a30 0x7efca80a0e00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 -- 172.21.15.32:0/217208311 >> 172.21.15.32:0/217208311 conn(0x7efca80a3050 msgr2=0x7efca80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 -- 172.21.15.32:0/217208311 shutdown_connections 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.088+0000 7efcaec73700 1 -- 172.21.15.32:0/217208311 wait complete. 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.089+0000 7efcaec73700 1 Processor -- start 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.089+0000 7efcaec73700 1 -- start start 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.089+0000 7efcaec73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca809d630 0x7efca8133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaec73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca80a0a30 0x7efca8133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaec73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 0x7efca8134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaec73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80a0410 con 0x7efca8138d80 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaec73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca8136ec0 con 0x7efca80a0a30 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaec73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efca813ac70 con 0x7efca809d630 2023-09-30T22:26:31.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efca7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca80a0a30 0x7efca8133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcad210700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 0x7efca8134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcad210700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 0x7efca8134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60870/0 (socket says 172.21.15.32:60870) 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcad210700 1 -- 172.21.15.32:0/511357675 learned_addr learned my addr 172.21.15.32:0/511357675 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efca7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca80a0a30 0x7efca8133cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45006/0 (socket says 172.21.15.32:45006) 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca809d630 0x7efca8133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 -- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca80a0a30 msgr2=0x7efca8133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca80a0a30 0x7efca8133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 -- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 msgr2=0x7efca8134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 0x7efca8134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 -- 172.21.15.32:0/511357675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca0009a70 con 0x7efca809d630 2023-09-30T22:26:31.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcad210700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 0x7efca8134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efcaca0f700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca809d630 0x7efca8133790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efca80672b0 tx=0x7efc98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.090+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98004300 con 0x7efca809d630 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.091+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9801f070 con 0x7efca809d630 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.091+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc980094a0 con 0x7efca809d630 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.091+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc980036a0 con 0x7efca809d630 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.091+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca81371a0 con 0x7efca809d630 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.091+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc98021030 con 0x7efca809d630 2023-09-30T22:26:31.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.092+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efc980405c0 con 0x7efca809d630 2023-09-30T22:26:31.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.092+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca80747f0 con 0x7efca809d630 2023-09-30T22:26:31.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.097+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc98015080 con 0x7efca809d630 2023-09-30T22:26:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.129+0000 7f3a14fdd700 1 Processor -- start 2023-09-30T22:26:31.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.129+0000 7f3a14fdd700 1 -- start start 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a14fdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a100a1510 0x7f3a10138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a14fdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a10139250 0x7f3a1013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a14fdd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1009edd0 con 0x7f3a100a0a30 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a14fdd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100a18e0 con 0x7f3a10139250 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a14fdd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1009e1f0 con 0x7f3a100a1510 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a0dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a100a1510 0x7f3a10138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a0e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3a100a0a30 0x7f3a100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43324/0 (socket says 172.21.15.32:43324) 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a0e59c700 1 -- 172.21.15.32:0/3739442570 learned_addr learned my addr 172.21.15.32:0/3739442570 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.130+0000 7f3a0ed9d700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a10139250 0x7f3a1013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a10139250 msgr2=0x7f3a1013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0dd9b700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a10139250 0x7f3a1013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/3739442570 >> v1:172.21.15.32:6789/0 conn(0x7f3a100a0a30 legacy=0x7f3a100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0d59a700 1 -- 172.21.15.32:0/3739442570 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266145239 0 0) 0x7f3a1009edd0 con 0x7f3a100a0a30 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/3739442570 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1009dc50 con 0x7f3a100a1510 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0ed9d700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a10139250 0x7f3a1013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0dd9b700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a100a1510 0x7f3a10138d10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009870 tx=0x7f3a04009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe7e4f1e3306655 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0d59a700 1 -- 172.21.15.32:0/3739442570 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04011830 con 0x7f3a100a1510 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0d59a700 1 -- 172.21.15.32:0/3739442570 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a04011990 con 0x7f3a100a1510 2023-09-30T22:26:31.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.131+0000 7f3a0d59a700 1 -- 172.21.15.32:0/3739442570 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04011d20 con 0x7f3a100a1510 2023-09-30T22:26:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 -- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a100a1510 msgr2=0x7f3a10138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a100a1510 0x7f3a10138d10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009870 tx=0x7f3a04009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 -- 172.21.15.32:0/3739442570 shutdown_connections 2023-09-30T22:26:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a10139250 0x7f3a1013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 --2- 172.21.15.32:0/3739442570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a100a1510 0x7f3a10138d10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 -- 172.21.15.32:0/3739442570 >> 172.21.15.32:0/3739442570 conn(0x7f3a100a6550 msgr2=0x7f3a100a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 -- 172.21.15.32:0/3739442570 shutdown_connections 2023-09-30T22:26:31.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.132+0000 7f3a14fdd700 1 -- 172.21.15.32:0/3739442570 wait complete. 2023-09-30T22:26:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.134+0000 7f3a14fdd700 1 Processor -- start 2023-09-30T22:26:31.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.134+0000 7f3a14fdd700 1 -- start start 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.134+0000 7f3a14fdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100a0a30 0x7f3a10122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a14fdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 0x7f3a10122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a14fdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 0x7f3a10123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a14fdd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10125c00 con 0x7f3a100a0a30 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a14fdd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10138180 con 0x7f3a100a1510 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a14fdd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a101389a0 con 0x7f3a10139250 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 0x7f3a10122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 0x7f3a10123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 0x7f3a10122e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45018/0 (socket says 172.21.15.32:45018) 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 0x7f3a10123380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49406/0 (socket says 172.21.15.32:49406) 2023-09-30T22:26:31.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/1386062916 learned_addr learned my addr 172.21.15.32:0/1386062916 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0e59c700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100a0a30 0x7f3a10122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 msgr2=0x7f3a10123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 0x7f3a10123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100a0a30 msgr2=0x7f3a10122900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100a0a30 0x7f3a10122900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 -- 172.21.15.32:0/1386062916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a08009a70 con 0x7f3a100a1510 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0ed9d700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 0x7f3a10123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f3a0dd9b700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 0x7f3a10122e40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a040090b0 tx=0x7f3a04004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.135+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040044d0 con 0x7f3a100a1510 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.136+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0401f070 con 0x7f3a100a1510 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.136+0000 7f3a14fdd700 1 -- 172.21.15.32:0/1386062916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a040094a0 con 0x7f3a100a1510 2023-09-30T22:26:31.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.136+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040038e0 con 0x7f3a100a1510 2023-09-30T22:26:31.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.136+0000 7f3a14fdd700 1 -- 172.21.15.32:0/1386062916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a10138760 con 0x7f3a100a1510 2023-09-30T22:26:31.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.137+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a04003a40 con 0x7f3a100a1510 2023-09-30T22:26:31.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.137+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a04021070 con 0x7f3a100a1510 2023-09-30T22:26:31.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.137+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e80052e0 con 0x7f3a100a1510 2023-09-30T22:26:31.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.142+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a04015020 con 0x7f3a100a1510 2023-09-30T22:26:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.310+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efca8072980 con 0x7efca809d630 2023-09-30T22:26:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.310+0000 7efca57fa700 1 -- 172.21.15.32:0/511357675 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc980239c0 con 0x7efca809d630 2023-09-30T22:26:31.311 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.312+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca809d630 msgr2=0x7efca8133790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.312+0000 7efcaec73700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca809d630 0x7efca8133790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efca80672b0 tx=0x7efc98004750 comp rx=0 tx=0).stop 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 shutdown_connections 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efca8138d80 0x7efca8134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efca809d630 0x7efca8133790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 --2- 172.21.15.32:0/511357675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efca80a0a30 0x7efca8133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 >> 172.21.15.32:0/511357675 conn(0x7efca80a3050 msgr2=0x7efca809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 shutdown_connections 2023-09-30T22:26:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.313+0000 7efcaec73700 1 -- 172.21.15.32:0/511357675 wait complete. 2023-09-30T22:26:31.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.350+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f39e8005590 con 0x7f3a100a1510 2023-09-30T22:26:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.351+0000 7f39feffd700 1 -- 172.21.15.32:0/1386062916 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a04003cf0 con 0x7f3a100a1510 2023-09-30T22:26:31.352 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 msgr2=0x7f3a10122e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 0x7f3a10122e40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a040090b0 tx=0x7f3a04004750 comp rx=0 tx=0).stop 2023-09-30T22:26:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 shutdown_connections 2023-09-30T22:26:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3a10139250 0x7f3a10123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3a100a0a30 0x7f3a10122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 --2- 172.21.15.32:0/1386062916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3a100a1510 0x7f3a10122e40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 >> 172.21.15.32:0/1386062916 conn(0x7f3a100a6550 msgr2=0x7f3a1009ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 shutdown_connections 2023-09-30T22:26:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:31.353+0000 7f39fcff9700 1 -- 172.21.15.32:0/1386062916 wait complete. 2023-09-30T22:26:31.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:31.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:31.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.012+0000 7f5f5d7e8700 1 Processor -- start 2023-09-30T22:26:32.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.012+0000 7f5f5d7e8700 1 -- start start 2023-09-30T22:26:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f5d7e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f581316c0 0x7f5f58131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f5d7e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f5d7e8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5809e130 con 0x7f5f5806acb0 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f5d7e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f581312b0 con 0x7f5f581316c0 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f5d7e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5809f6e0 con 0x7f5f5809d9f0 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f56ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f581316c0 0x7f5f58131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f577fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5f5806acb0 0x7f5f58138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43330/0 (socket says 172.21.15.32:43330) 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f4e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f577fe700 1 -- 172.21.15.32:0/1551760391 learned_addr learned my addr 172.21.15.32:0/1551760391 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f581316c0 msgr2=0x7f5f58131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f4e7fc700 1 --2- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f581316c0 0x7f5f58131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/1551760391 >> v1:172.21.15.32:6789/0 conn(0x7f5f5806acb0 legacy=0x7f5f58138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f567fc700 1 -- 172.21.15.32:0/1551760391 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448218165 0 0) 0x7f5f50001490 con 0x7f5f5806acb0 2023-09-30T22:26:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/1551760391 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5809e760 con 0x7f5f5809d9f0 2023-09-30T22:26:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f56ffd700 1 --2- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f581316c0 0x7f5f58131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.013+0000 7f5f4e7fc700 1 --2- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5806a770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009870 tx=0x7f5f40009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a2498db68f6384 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.014+0000 7f5f567fc700 1 -- 172.21.15.32:0/1551760391 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40011830 con 0x7f5f5809d9f0 2023-09-30T22:26:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.014+0000 7f5f567fc700 1 -- 172.21.15.32:0/1551760391 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f40011990 con 0x7f5f5809d9f0 2023-09-30T22:26:32.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.014+0000 7f5f567fc700 1 -- 172.21.15.32:0/1551760391 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40011d20 con 0x7f5f5809d9f0 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 msgr2=0x7f5f5806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5806a770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009870 tx=0x7f5f40009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/1551760391 shutdown_connections 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f581316c0 0x7f5f58131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/1551760391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5806a770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/1551760391 >> 172.21.15.32:0/1551760391 conn(0x7f5f58066f50 msgr2=0x7f5f5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/1551760391 shutdown_connections 2023-09-30T22:26:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.015+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/1551760391 wait complete. 2023-09-30T22:26:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.016+0000 7f5f5d7e8700 1 Processor -- start 2023-09-30T22:26:32.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.016+0000 7f5f5d7e8700 1 -- start start 2023-09-30T22:26:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f5d7e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f5806acb0 0x7f5f5809a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f5d7e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5809a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f5d7e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f581316c0 0x7f5f5809ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f5d7e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58130b50 con 0x7f5f581316c0 2023-09-30T22:26:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f5d7e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5812b900 con 0x7f5f5806acb0 2023-09-30T22:26:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f5d7e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58129470 con 0x7f5f5809d9f0 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f4e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5809a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f4e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5809a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49432/0 (socket says 172.21.15.32:49432) 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/2890011964 learned_addr learned my addr 172.21.15.32:0/2890011964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f56ffd700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f5806acb0 0x7f5f5809a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.017+0000 7f5f577fe700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f581316c0 0x7f5f5809ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f5806acb0 msgr2=0x7f5f5809a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4e7fc700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f5806acb0 0x7f5f5809a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f581316c0 msgr2=0x7f5f5809ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4e7fc700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f581316c0 0x7f5f5809ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4e7fc700 1 -- 172.21.15.32:0/2890011964 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480094a0 con 0x7f5f5809d9f0 2023-09-30T22:26:32.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f56ffd700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f5806acb0 0x7f5f5809a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4e7fc700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5809a910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f400090b0 tx=0x7f5f40004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40004550 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4001f070 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f400094a0 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f400038e0 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.018+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5811dda0 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.019+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f40003a40 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.019+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5f40021070 con 0x7f5f5809d9f0 2023-09-30T22:26:32.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.019+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f58074ea0 con 0x7f5f5809d9f0 2023-09-30T22:26:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.024+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f40015070 con 0x7f5f5809d9f0 2023-09-30T22:26:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.052+0000 7fa0ab186700 1 Processor -- start 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.052+0000 7fa0ab186700 1 -- start start 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0ab186700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0ab186700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0a409d630 0x7fa0a40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0ab186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a4133930 0x7fa0a4133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0ab186700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a40a2a70 con 0x7fa0a409d630 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0ab186700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40a01d0 con 0x7fa0a40a0a30 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0ab186700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a409e2b0 con 0x7fa0a4133930 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a8f22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0a409d630 0x7fa0a40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a9723700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a4133930 0x7fa0a4133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a8f22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45046/0 (socket says 172.21.15.32:45046) 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a8f22700 1 -- 172.21.15.32:0/859270786 learned_addr learned my addr 172.21.15.32:0/859270786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a3fff700 1 -- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a4133930 msgr2=0x7fa0a4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a3fff700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a4133930 0x7fa0a4133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a3fff700 1 -- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 msgr2=0x7fa0a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a3fff700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a3fff700 1 -- 172.21.15.32:0/859270786 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a409dc50 con 0x7fa0a409d630 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a8f22700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.053+0000 7fa0a9723700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a4133930 0x7fa0a4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0a3fff700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a409d630 0x7fa0a40a1340 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa090009840 tx=0x7fa090009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ac0bf1d7dc191d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0a37fe700 1 -- 172.21.15.32:0/859270786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090011830 con 0x7fa0a409d630 2023-09-30T22:26:32.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0a37fe700 1 -- 172.21.15.32:0/859270786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa090011990 con 0x7fa0a409d630 2023-09-30T22:26:32.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0a37fe700 1 -- 172.21.15.32:0/859270786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090011d20 con 0x7fa0a409d630 2023-09-30T22:26:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 -- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a409d630 msgr2=0x7fa0a40a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a409d630 0x7fa0a40a1340 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa090009840 tx=0x7fa090009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 -- 172.21.15.32:0/859270786 shutdown_connections 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a4133930 0x7fa0a4133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 --2- 172.21.15.32:0/859270786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a409d630 0x7fa0a40a1340 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 -- 172.21.15.32:0/859270786 >> 172.21.15.32:0/859270786 conn(0x7fa0a40a3050 msgr2=0x7fa0a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 -- 172.21.15.32:0/859270786 shutdown_connections 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.054+0000 7fa0ab186700 1 -- 172.21.15.32:0/859270786 wait complete. 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.055+0000 7fa0ab186700 1 Processor -- start 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 -- start start 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a409d630 0x7fa0a4121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a4121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 0x7fa0a4121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41226d0 con 0x7fa0a4133930 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4124cb0 con 0x7fa0a40a0a30 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0ab186700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4124710 con 0x7fa0a409d630 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0a9723700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 0x7fa0a4121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0a8f22700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a409d630 0x7fa0a4121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0a3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a4121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0a9723700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 0x7fa0a4121ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60896/0 (socket says 172.21.15.32:60896) 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.056+0000 7fa0a9723700 1 -- 172.21.15.32:0/3333444861 learned_addr learned my addr 172.21.15.32:0/3333444861 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a9723700 1 -- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a409d630 msgr2=0x7fa0a4121220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a9723700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a409d630 0x7fa0a4121220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a9723700 1 -- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 msgr2=0x7fa0a4121760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a9723700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a4121760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a9723700 1 -- 172.21.15.32:0/3333444861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009550 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a3fff700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a4121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a9723700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 0x7fa0a4121ca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00efc0 tx=0x7fa09c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00ca90 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00f040 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c009a70 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00ce10 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.057+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0900094a0 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.058+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa09c014480 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.058+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa09c03fc40 con 0x7fa0a4133930 2023-09-30T22:26:32.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.058+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a40a0410 con 0x7fa0a4133930 2023-09-30T22:26:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.063+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa09c012930 con 0x7fa0a4133930 2023-09-30T22:26:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.231+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5f5806c360 con 0x7f5f5809d9f0 2023-09-30T22:26:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.231+0000 7f5f4ffff700 1 -- 172.21.15.32:0/2890011964 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f4001a950 con 0x7f5f5809d9f0 2023-09-30T22:26:32.232 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 msgr2=0x7f5f5809a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5809a910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5f400090b0 tx=0x7f5f40004750 comp rx=0 tx=0).stop 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 shutdown_connections 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f581316c0 0x7f5f5809ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f5806acb0 0x7f5f5809a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 --2- 172.21.15.32:0/2890011964 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5f5809d9f0 0x7f5f5809a910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.233+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 >> 172.21.15.32:0/2890011964 conn(0x7f5f58066f50 msgr2=0x7f5f5812bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.234+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 shutdown_connections 2023-09-30T22:26:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.234+0000 7f5f5d7e8700 1 -- 172.21.15.32:0/2890011964 wait complete. 2023-09-30T22:26:32.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:32.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.267+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa0a4124930 con 0x7fa0a4133930 2023-09-30T22:26:32.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.268+0000 7fa0a17fa700 1 -- 172.21.15.32:0/3333444861 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0a4124930 con 0x7fa0a4133930 2023-09-30T22:26:32.269 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 msgr2=0x7fa0a4121ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 0x7fa0a4121ca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00efc0 tx=0x7fa09c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 shutdown_connections 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a4133930 0x7fa0a4121ca0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a409d630 0x7fa0a4121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 --2- 172.21.15.32:0/3333444861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a40a0a30 0x7fa0a4121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 >> 172.21.15.32:0/3333444861 conn(0x7fa0a40a3050 msgr2=0x7fa0a4099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 shutdown_connections 2023-09-30T22:26:32.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.270+0000 7fa0ab186700 1 -- 172.21.15.32:0/3333444861 wait complete. 2023-09-30T22:26:32.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:32.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:32.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.473+0000 7f6b8074b700 1 Processor -- start 2023-09-30T22:26:32.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.473+0000 7f6b8074b700 1 -- start start 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b8074b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a0a30 0x7f6b780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b8074b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b78134170 0x7f6b7813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b8074b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7809edd0 con 0x7f6b780a1340 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b8074b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7809e1f0 con 0x7f6b78134170 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b8074b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7809dc50 con 0x7f6b780a0a30 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7ece8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b78134170 0x7f6b7813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7ece8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b78134170 0x7f6b7813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60898/0 (socket says 172.21.15.32:60898) 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7e4e7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a0a30 0x7f6b780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7ece8700 1 -- 172.21.15.32:0/3289296178 learned_addr learned my addr 172.21.15.32:0/3289296178 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7ece8700 1 -- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a0a30 msgr2=0x7f6b780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7ece8700 1 --2- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a0a30 0x7f6b780a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.474+0000 7f6b7ece8700 1 -- 172.21.15.32:0/3289296178 >> v1:172.21.15.32:6789/0 conn(0x7f6b780a1340 legacy=0x7f6b78133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.475+0000 7f6b7ece8700 1 -- 172.21.15.32:0/3289296178 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780a1fe0 con 0x7f6b78134170 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.475+0000 7f6b7d4e5700 1 -- 172.21.15.32:0/3289296178 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474356226 0 0) 0x7f6b7809edd0 con 0x7f6b780a1340 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.475+0000 7f6b7ece8700 1 --2- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b7813bd30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009e50 tx=0x7f6b7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09c31dd9aae1896 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.475+0000 7f6b7d4e5700 1 -- 172.21.15.32:0/3289296178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000e070 con 0x7f6b78134170 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.475+0000 7f6b7d4e5700 1 -- 172.21.15.32:0/3289296178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7000b910 con 0x7f6b78134170 2023-09-30T22:26:32.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.475+0000 7f6b7d4e5700 1 -- 172.21.15.32:0/3289296178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000bcc0 con 0x7f6b78134170 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 -- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 msgr2=0x7f6b7813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 --2- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b7813bd30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009e50 tx=0x7f6b7000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 -- 172.21.15.32:0/3289296178 shutdown_connections 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 --2- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b7813bd30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 --2- 172.21.15.32:0/3289296178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a0a30 0x7f6b780a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 -- 172.21.15.32:0/3289296178 >> 172.21.15.32:0/3289296178 conn(0x7f6b780a6550 msgr2=0x7f6b780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 -- 172.21.15.32:0/3289296178 shutdown_connections 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.476+0000 7f6b8074b700 1 -- 172.21.15.32:0/3289296178 wait complete. 2023-09-30T22:26:32.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 Processor -- start 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 -- start start 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b780a0a30 0x7f6b781c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 0x7f6b781c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b78122880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780a4d60 con 0x7f6b78134170 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780a0220 con 0x7f6b780a1340 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b8074b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b781260c0 con 0x7f6b780a0a30 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b7dce6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 0x7f6b781c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b7dce6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 0x7f6b781c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45088/0 (socket says 172.21.15.32:45088) 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b7dce6700 1 -- 172.21.15.32:0/2851297344 learned_addr learned my addr 172.21.15.32:0/2851297344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b7ece8700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b78122880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.477+0000 7f6b7e4e7700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b780a0a30 0x7f6b781c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7dce6700 1 -- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b780a0a30 msgr2=0x7f6b781c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7dce6700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b780a0a30 0x7f6b781c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7dce6700 1 -- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 msgr2=0x7f6b78122880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7dce6700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b78122880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7dce6700 1 -- 172.21.15.32:0/2851297344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b70009b30 con 0x7f6b780a1340 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7ece8700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b78122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b7dce6700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 0x7f6b781c8c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b68001eb0 tx=0x7f6b6800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680054e0 con 0x7f6b780a1340 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b68003680 con 0x7f6b780a1340 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b78126b30 con 0x7f6b780a1340 2023-09-30T22:26:32.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680039f0 con 0x7f6b780a1340 2023-09-30T22:26:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.478+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b781c91b0 con 0x7f6b780a1340 2023-09-30T22:26:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.479+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b78074ea0 con 0x7f6b780a1340 2023-09-30T22:26:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.480+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b68003b50 con 0x7f6b780a1340 2023-09-30T22:26:32.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.480+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b6803ef00 con 0x7f6b780a1340 2023-09-30T22:26:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.484+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b6801b3b0 con 0x7f6b780a1340 2023-09-30T22:26:32.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 Processor -- start 2023-09-30T22:26:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 -- start start 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e2c0a0a30 0x7f8e2c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 0x7f8e2c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2c0a2a70 con 0x7f8e2c09d630 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0a01d0 con 0x7f8e2c0a0a30 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.514+0000 7f8e341e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c09e2b0 con 0x7f8e2c138d80 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 0x7f8e2c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e31f7c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e2c0a0a30 0x7f8e2c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 0x7f8e2c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45096/0 (socket says 172.21.15.32:45096) 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 -- 172.21.15.32:0/3830634811 learned_addr learned my addr 172.21.15.32:0/3830634811 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3177b700 1 --1- 172.21.15.32:0/3830634811 >> v1:172.21.15.32:6789/0 conn(0x7f8e2c09d630 0x7f8e2c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43352/0 (socket says 172.21.15.32:43352) 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 -- 172.21.15.32:0/3830634811 >> v2:172.21.15.32:3300/0 conn(0x7f8e2c0a0a30 msgr2=0x7f8e2c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 --2- 172.21.15.32:0/3830634811 >> v2:172.21.15.32:3300/0 conn(0x7f8e2c0a0a30 0x7f8e2c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 -- 172.21.15.32:0/3830634811 >> v1:172.21.15.32:6789/0 conn(0x7f8e2c09d630 legacy=0x7f8e2c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e30f7a700 1 -- 172.21.15.32:0/3830634811 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234322992 0 0) 0x7f8e2c0a2a70 con 0x7f8e2c09d630 2023-09-30T22:26:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 -- 172.21.15.32:0/3830634811 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c09dc50 con 0x7f8e2c138d80 2023-09-30T22:26:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e31f7c700 1 --2- 172.21.15.32:0/3830634811 >> v2:172.21.15.32:3300/0 conn(0x7f8e2c0a0a30 0x7f8e2c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e3277d700 1 --2- 172.21.15.32:0/3830634811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 0x7f8e2c13b160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009e50 tx=0x7f8e2400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=140313a2dbf06c4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e30f7a700 1 -- 172.21.15.32:0/3830634811 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400e070 con 0x7f8e2c138d80 2023-09-30T22:26:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e30f7a700 1 -- 172.21.15.32:0/3830634811 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2400b910 con 0x7f8e2c138d80 2023-09-30T22:26:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.515+0000 7f8e30f7a700 1 -- 172.21.15.32:0/3830634811 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400bcc0 con 0x7f8e2c138d80 2023-09-30T22:26:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 -- 172.21.15.32:0/3830634811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 msgr2=0x7f8e2c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 --2- 172.21.15.32:0/3830634811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 0x7f8e2c13b160 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009e50 tx=0x7f8e2400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 -- 172.21.15.32:0/3830634811 shutdown_connections 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 --2- 172.21.15.32:0/3830634811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c138d80 0x7f8e2c13b160 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 --2- 172.21.15.32:0/3830634811 >> v2:172.21.15.32:3300/0 conn(0x7f8e2c0a0a30 0x7f8e2c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 -- 172.21.15.32:0/3830634811 >> 172.21.15.32:0/3830634811 conn(0x7f8e2c0a3050 msgr2=0x7f8e2c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 -- 172.21.15.32:0/3830634811 shutdown_connections 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.516+0000 7f8e341e0700 1 -- 172.21.15.32:0/3830634811 wait complete. 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.517+0000 7f8e341e0700 1 Processor -- start 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.517+0000 7f8e341e0700 1 -- start start 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 0x7f8e2c133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2c0a0a30 0x7f8e2c133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e2c138d80 0x7f8e2c1340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c137170 con 0x7f8e2c0a0a30 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c136060 con 0x7f8e2c09d630 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c136520 con 0x7f8e2c138d80 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 0x7f8e2c133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 0x7f8e2c133650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45098/0 (socket says 172.21.15.32:45098) 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 -- 172.21.15.32:0/2717654877 learned_addr learned my addr 172.21.15.32:0/2717654877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e3177b700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2c0a0a30 0x7f8e2c133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 -- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e2c138d80 msgr2=0x7f8e2c1340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e2c138d80 0x7f8e2c1340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 -- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2c0a0a30 msgr2=0x7f8e2c133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2c0a0a30 0x7f8e2c133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 -- 172.21.15.32:0/2717654877 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280094a0 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e3177b700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2c0a0a30 0x7f8e2c133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e31f7c700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 0x7f8e2c133650 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800ebe0 tx=0x7f8e2800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28009800 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800c910 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e24009b30 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.518+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800cc80 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.519+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c136bd0 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.519+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e28013470 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.519+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e2803fbb0 con 0x7f8e2c09d630 2023-09-30T22:26:32.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.520+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c137d50 con 0x7f8e2c09d630 2023-09-30T22:26:32.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.524+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e28013720 con 0x7f8e2c09d630 2023-09-30T22:26:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.690+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6b780a28e0 con 0x7f6b780a1340 2023-09-30T22:26:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.691+0000 7f6b6effd700 1 -- 172.21.15.32:0/2851297344 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b68017030 con 0x7f6b780a1340 2023-09-30T22:26:32.692 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 msgr2=0x7f6b781c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 0x7f6b781c8c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b68001eb0 tx=0x7f6b6800c800 comp rx=0 tx=0).stop 2023-09-30T22:26:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 shutdown_connections 2023-09-30T22:26:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b78134170 0x7f6b78122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b780a0a30 0x7f6b781c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 --2- 172.21.15.32:0/2851297344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b780a1340 0x7f6b781c8c70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 >> 172.21.15.32:0/2851297344 conn(0x7f6b780a6550 msgr2=0x7f6b7809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 shutdown_connections 2023-09-30T22:26:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.693+0000 7f6b8074b700 1 -- 172.21.15.32:0/2851297344 wait complete. 2023-09-30T22:26:32.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:32.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.730+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8e2c0747f0 con 0x7f8e2c09d630 2023-09-30T22:26:32.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.730+0000 7f8e227fc700 1 -- 172.21.15.32:0/2717654877 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e28021030 con 0x7f8e2c09d630 2023-09-30T22:26:32.731 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:32.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 msgr2=0x7f8e2c133650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 0x7f8e2c133650 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800ebe0 tx=0x7f8e2800eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 shutdown_connections 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e2c138d80 0x7f8e2c1340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e2c09d630 0x7f8e2c133650 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 --2- 172.21.15.32:0/2717654877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e2c0a0a30 0x7f8e2c133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.732+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 >> 172.21.15.32:0/2717654877 conn(0x7f8e2c0a3050 msgr2=0x7f8e2c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.733+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 shutdown_connections 2023-09-30T22:26:32.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:32.733+0000 7f8e341e0700 1 -- 172.21.15.32:0/2717654877 wait complete. 2023-09-30T22:26:32.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:33.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:33.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:33.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.398+0000 7fb14776f700 1 Processor -- start 2023-09-30T22:26:33.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14776f700 1 -- start start 2023-09-30T22:26:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14776f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400a0a30 0x7fb1400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14776f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 0x7fb1400a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14776f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400a2a70 con 0x7fb140133930 2023-09-30T22:26:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14776f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400a01d0 con 0x7fb1400a0a30 2023-09-30T22:26:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14776f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14009e2b0 con 0x7fb14009d630 2023-09-30T22:26:33.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb144d0a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 0x7fb1400a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb14550b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400a0a30 0x7fb1400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb144d0a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 0x7fb1400a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45110/0 (socket says 172.21.15.32:45110) 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.399+0000 7fb144d0a700 1 -- 172.21.15.32:0/388501075 learned_addr learned my addr 172.21.15.32:0/388501075 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb144d0a700 1 -- 172.21.15.32:0/388501075 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a0a30 msgr2=0x7fb1400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb144d0a700 1 --2- 172.21.15.32:0/388501075 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a0a30 0x7fb1400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb144d0a700 1 -- 172.21.15.32:0/388501075 >> v1:172.21.15.32:6789/0 conn(0x7fb140133930 legacy=0x7fb140133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb144d0a700 1 -- 172.21.15.32:0/388501075 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14009dc50 con 0x7fb14009d630 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb137fff700 1 -- 172.21.15.32:0/388501075 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348396140 0 0) 0x7fb1400a2a70 con 0x7fb140133930 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb14550b700 1 --2- 172.21.15.32:0/388501075 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a0a30 0x7fb1400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb144d0a700 1 --2- 172.21.15.32:0/388501075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 0x7fb1400a1340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1300098f0 tx=0x7fb130009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe432b7fdeb338d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb137fff700 1 -- 172.21.15.32:0/388501075 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130011830 con 0x7fb14009d630 2023-09-30T22:26:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb137fff700 1 -- 172.21.15.32:0/388501075 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb130011990 con 0x7fb14009d630 2023-09-30T22:26:33.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.400+0000 7fb137fff700 1 -- 172.21.15.32:0/388501075 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130011d20 con 0x7fb14009d630 2023-09-30T22:26:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 -- 172.21.15.32:0/388501075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 msgr2=0x7fb1400a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 --2- 172.21.15.32:0/388501075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 0x7fb1400a1340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1300098f0 tx=0x7fb130009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 -- 172.21.15.32:0/388501075 shutdown_connections 2023-09-30T22:26:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 --2- 172.21.15.32:0/388501075 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a0a30 0x7fb1400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 --2- 172.21.15.32:0/388501075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb14009d630 0x7fb1400a1340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 -- 172.21.15.32:0/388501075 >> 172.21.15.32:0/388501075 conn(0x7fb1400a3050 msgr2=0x7fb1400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 -- 172.21.15.32:0/388501075 shutdown_connections 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.401+0000 7fb14776f700 1 -- 172.21.15.32:0/388501075 wait complete. 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.402+0000 7fb14776f700 1 Processor -- start 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 -- start start 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb14009d630 0x7fb1401c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 0x7fb1401c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 0x7fb1401c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1401221b0 con 0x7fb1400a0a30 2023-09-30T22:26:33.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb140133520 con 0x7fb140133930 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14776f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb140120700 con 0x7fb14009d630 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb144d0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 0x7fb1401c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb145d0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 0x7fb1401c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb144d0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 0x7fb1401c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60934/0 (socket says 172.21.15.32:60934) 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb145d0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 0x7fb1401c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45122/0 (socket says 172.21.15.32:45122) 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb144d0a700 1 -- 172.21.15.32:0/269772438 learned_addr learned my addr 172.21.15.32:0/269772438 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.403+0000 7fb14550b700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb14009d630 0x7fb1401c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb145d0c700 1 -- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb14009d630 msgr2=0x7fb1401c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb145d0c700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb14009d630 0x7fb1401c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb145d0c700 1 -- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 msgr2=0x7fb1401c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb145d0c700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 0x7fb1401c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb145d0c700 1 -- 172.21.15.32:0/269772438 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009a70 con 0x7fb140133930 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb144d0a700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 0x7fb1401c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb145d0c700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 0x7fb1401c8a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb1380011e0 tx=0x7fb13800c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1380054e0 con 0x7fb140133930 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800ce10 con 0x7fb140133930 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb130009550 con 0x7fb140133930 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1380038c0 con 0x7fb140133930 2023-09-30T22:26:33.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.404+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb140122c20 con 0x7fb140133930 2023-09-30T22:26:33.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.406+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1400747f0 con 0x7fb140133930 2023-09-30T22:26:33.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.406+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb138003a20 con 0x7fb140133930 2023-09-30T22:26:33.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.406+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb13803ee10 con 0x7fb140133930 2023-09-30T22:26:33.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.411+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb138019b40 con 0x7fb140133930 2023-09-30T22:26:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.436+0000 7f8b9fd57700 1 Processor -- start 2023-09-30T22:26:33.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9fd57700 1 -- start start 2023-09-30T22:26:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9fd57700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9fd57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 0x7f8b9813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9fd57700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b980a2a70 con 0x7f8b9809d630 2023-09-30T22:26:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9fd57700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b980a01d0 con 0x7f8b98138d80 2023-09-30T22:26:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9fd57700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9809e2b0 con 0x7f8b980a0a30 2023-09-30T22:26:33.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9e2f4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 0x7f8b9813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9d2f2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8b9809d630 0x7f8b980a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43362/0 (socket says 172.21.15.32:43362) 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9e2f4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 0x7f8b9813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60946/0 (socket says 172.21.15.32:60946) 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9d2f2700 1 -- 172.21.15.32:0/4219774664 learned_addr learned my addr 172.21.15.32:0/4219774664 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.437+0000 7f8b9daf3700 1 --2- 172.21.15.32:0/4219774664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9caf1700 1 -- 172.21.15.32:0/4219774664 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362434544 0 0) 0x7f8b980a2a70 con 0x7f8b9809d630 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9caf1700 1 -- 172.21.15.32:0/4219774664 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b800035b0 con 0x7f8b9809d630 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9daf3700 1 -- 172.21.15.32:0/4219774664 >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 msgr2=0x7f8b9813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9daf3700 1 --2- 172.21.15.32:0/4219774664 >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 0x7f8b9813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9daf3700 1 -- 172.21.15.32:0/4219774664 >> v1:172.21.15.32:6789/0 conn(0x7f8b9809d630 legacy=0x7f8b980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9daf3700 1 -- 172.21.15.32:0/4219774664 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9809dc50 con 0x7f8b980a0a30 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9e2f4700 1 --2- 172.21.15.32:0/4219774664 >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 0x7f8b9813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9daf3700 1 --2- 172.21.15.32:0/4219774664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b980a0e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009d90 tx=0x7f8b8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa75619b56f430b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9caf1700 1 -- 172.21.15.32:0/4219774664 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800e070 con 0x7f8b980a0a30 2023-09-30T22:26:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9caf1700 1 -- 172.21.15.32:0/4219774664 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8800bb20 con 0x7f8b980a0a30 2023-09-30T22:26:33.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.438+0000 7f8b9caf1700 1 -- 172.21.15.32:0/4219774664 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800be90 con 0x7f8b980a0a30 2023-09-30T22:26:33.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 -- 172.21.15.32:0/4219774664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 msgr2=0x7f8b980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/4219774664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b980a0e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009d90 tx=0x7f8b8800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:33.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 -- 172.21.15.32:0/4219774664 shutdown_connections 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/4219774664 >> v2:172.21.15.32:3300/0 conn(0x7f8b98138d80 0x7f8b9813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/4219774664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b980a0e00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 -- 172.21.15.32:0/4219774664 >> 172.21.15.32:0/4219774664 conn(0x7f8b980a3050 msgr2=0x7f8b980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 -- 172.21.15.32:0/4219774664 shutdown_connections 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.439+0000 7f8b9fd57700 1 -- 172.21.15.32:0/4219774664 wait complete. 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 Processor -- start 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 -- start start 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b9809d630 0x7f8b98133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b98133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98138d80 0x7f8b98134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98136f50 con 0x7f8b98138d80 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98135940 con 0x7f8b980a0a30 2023-09-30T22:26:33.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9fd57700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b980a0480 con 0x7f8b9809d630 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9daf3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b9809d630 0x7f8b98133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.440+0000 7f8b9d2f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b98133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9daf3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b9809d630 0x7f8b98133710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49510/0 (socket says 172.21.15.32:49510) 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9daf3700 1 -- 172.21.15.32:0/2728376977 learned_addr learned my addr 172.21.15.32:0/2728376977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9e2f4700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98138d80 0x7f8b98134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9d2f2700 1 -- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b9809d630 msgr2=0x7f8b98133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9d2f2700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b9809d630 0x7f8b98133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9d2f2700 1 -- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98138d80 msgr2=0x7f8b98134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9d2f2700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98138d80 0x7f8b98134190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9d2f2700 1 -- 172.21.15.32:0/2728376977 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b900094a0 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9d2f2700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b98133c50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00c4c0 tx=0x7f8b8c00c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c005d70 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0070d0 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b88009a70 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c007440 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.441+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b98137e10 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.442+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b8c0075a0 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.442+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b980747f0 con 0x7f8b980a0a30 2023-09-30T22:26:33.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.442+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8b8c03ea40 con 0x7f8b980a0a30 2023-09-30T22:26:33.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.447+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b8c017020 con 0x7f8b980a0a30 2023-09-30T22:26:33.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.618+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb140072980 con 0x7fb140133930 2023-09-30T22:26:33.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.619+0000 7fb135ffb700 1 -- 172.21.15.32:0/269772438 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb138017020 con 0x7fb140133930 2023-09-30T22:26:33.620 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 msgr2=0x7fb1401c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 0x7fb1401c8a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb1380011e0 tx=0x7fb13800c710 comp rx=0 tx=0).stop 2023-09-30T22:26:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 shutdown_connections 2023-09-30T22:26:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb140133930 0x7fb1401c8a00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb14009d630 0x7fb1401c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 --2- 172.21.15.32:0/269772438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1400a0a30 0x7fb1401c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 >> 172.21.15.32:0/269772438 conn(0x7fb1400a3050 msgr2=0x7fb1401bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:33.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 shutdown_connections 2023-09-30T22:26:33.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.621+0000 7fb14776f700 1 -- 172.21.15.32:0/269772438 wait complete. 2023-09-30T22:26:33.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.654+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8b98135b10 con 0x7f8b980a0a30 2023-09-30T22:26:33.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.654+0000 7f8b967fc700 1 -- 172.21.15.32:0/2728376977 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b8c00f260 con 0x7f8b980a0a30 2023-09-30T22:26:33.656 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 msgr2=0x7f8b98133c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b98133c50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00c4c0 tx=0x7f8b8c00c7d0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 shutdown_connections 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98138d80 0x7f8b98134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b9809d630 0x7f8b98133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 --2- 172.21.15.32:0/2728376977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b980a0a30 0x7f8b98133c50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 >> 172.21.15.32:0/2728376977 conn(0x7f8b980a3050 msgr2=0x7f8b980a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:33.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 shutdown_connections 2023-09-30T22:26:33.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.657+0000 7f8b9fd57700 1 -- 172.21.15.32:0/2728376977 wait complete. 2023-09-30T22:26:33.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:33.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:33.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.852+0000 7fab88bc2700 1 Processor -- start 2023-09-30T22:26:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.852+0000 7fab88bc2700 1 -- start start 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab88bc2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab840a0a30 0x7fab840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab88bc2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab88bc2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab840a2a70 con 0x7fab84133930 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab88bc2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840a01d0 con 0x7fab840a0a30 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab88bc2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab8409e2b0 con 0x7fab8409d630 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab840a0a30 0x7fab840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab840a0a30 0x7fab840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60956/0 (socket says 172.21.15.32:60956) 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab81d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab82d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab84133930 0x7fab84133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43372/0 (socket says 172.21.15.32:43372) 2023-09-30T22:26:33.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 -- 172.21.15.32:0/1018844227 learned_addr learned my addr 172.21.15.32:0/1018844227 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 -- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 msgr2=0x7fab840a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 --2- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab840a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 -- 172.21.15.32:0/1018844227 >> v1:172.21.15.32:6789/0 conn(0x7fab84133930 legacy=0x7fab84133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab8259c700 1 -- 172.21.15.32:0/1018844227 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8409dc50 con 0x7fab840a0a30 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.853+0000 7fab81d9b700 1 --2- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab840a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.854+0000 7fab8259c700 1 --2- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab840a0a30 0x7fab840a0e00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fab74009e50 tx=0x7fab7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca9287cd89f10c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.854+0000 7fab8159a700 1 -- 172.21.15.32:0/1018844227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7400e070 con 0x7fab840a0a30 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.854+0000 7fab8159a700 1 -- 172.21.15.32:0/1018844227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7400b910 con 0x7fab840a0a30 2023-09-30T22:26:33.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.854+0000 7fab8159a700 1 -- 172.21.15.32:0/1018844227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7400bcc0 con 0x7fab840a0a30 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.854+0000 7fab88bc2700 1 -- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab840a0a30 msgr2=0x7fab840a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.854+0000 7fab88bc2700 1 --2- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab840a0a30 0x7fab840a0e00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fab74009e50 tx=0x7fab7400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.855+0000 7fab88bc2700 1 -- 172.21.15.32:0/1018844227 shutdown_connections 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.855+0000 7fab88bc2700 1 --2- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab840a0a30 0x7fab840a0e00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.855+0000 7fab88bc2700 1 --2- 172.21.15.32:0/1018844227 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab840a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.855+0000 7fab88bc2700 1 -- 172.21.15.32:0/1018844227 >> 172.21.15.32:0/1018844227 conn(0x7fab840a3050 msgr2=0x7fab840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.855+0000 7fab88bc2700 1 -- 172.21.15.32:0/1018844227 shutdown_connections 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.855+0000 7fab88bc2700 1 -- 172.21.15.32:0/1018844227 wait complete. 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 Processor -- start 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 -- start start 2023-09-30T22:26:33.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab841c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab840a0a30 0x7fab841c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 0x7fab841c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8411eea0 con 0x7fab84133930 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab841207f0 con 0x7fab840a0a30 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab88bc2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab84120600 con 0x7fab8409d630 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.856+0000 7fab82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 0x7fab841c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab8259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab841c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 0x7fab841c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60958/0 (socket says 172.21.15.32:60958) 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 -- 172.21.15.32:0/4075223733 learned_addr learned my addr 172.21.15.32:0/4075223733 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab8259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab841c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49524/0 (socket says 172.21.15.32:49524) 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab81d9b700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab840a0a30 0x7fab841c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 -- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 msgr2=0x7fab841c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab841c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 -- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab840a0a30 msgr2=0x7fab841c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab840a0a30 0x7fab841c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 -- 172.21.15.32:0/4075223733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0094a0 con 0x7fab84133930 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab8259c700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab841c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab82d9d700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 0x7fab841c8ba0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c490 tx=0x7fab7c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00cea0 con 0x7fab84133930 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c005cb0 con 0x7fab84133930 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab74009b30 con 0x7fab84133930 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0072d0 con 0x7fab84133930 2023-09-30T22:26:33.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.857+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab84122230 con 0x7fab84133930 2023-09-30T22:26:33.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.859+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab84122420 con 0x7fab84133930 2023-09-30T22:26:33.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.859+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab7c007430 con 0x7fab84133930 2023-09-30T22:26:33.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.859+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab7c03e990 con 0x7fab84133930 2023-09-30T22:26:33.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.864+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab7c0198b0 con 0x7fab84133930 2023-09-30T22:26:33.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.899+0000 7f1645df3700 1 Processor -- start 2023-09-30T22:26:33.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1645df3700 1 -- start start 2023-09-30T22:26:33.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1645df3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1645df3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f164013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1645df3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164009edd0 con 0x7f16400a0a30 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1645df3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400a18e0 con 0x7f1640139250 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1645df3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f164009e1f0 con 0x7f16400a1510 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f163ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f164013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1636ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1636ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49536/0 (socket says 172.21.15.32:49536) 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f163ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f164013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45162/0 (socket says 172.21.15.32:45162) 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.900+0000 7f1636ffd700 1 -- 172.21.15.32:0/3212008931 learned_addr learned my addr 172.21.15.32:0/3212008931 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f1636ffd700 1 -- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 msgr2=0x7f164013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f1636ffd700 1 --2- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f164013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f1636ffd700 1 -- 172.21.15.32:0/3212008931 >> v1:172.21.15.32:6789/0 conn(0x7f16400a0a30 legacy=0x7f16400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f1636ffd700 1 -- 172.21.15.32:0/3212008931 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164009dc50 con 0x7f16400a1510 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f163effd700 1 -- 172.21.15.32:0/3212008931 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686115975 0 0) 0x7f164009edd0 con 0x7f16400a0a30 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f163ffff700 1 --2- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f164013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f1636ffd700 1 --2- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640138d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1628009920 tx=0x7f1628009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffbea4977b58f2c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f163effd700 1 -- 172.21.15.32:0/3212008931 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011830 con 0x7f16400a1510 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f163effd700 1 -- 172.21.15.32:0/3212008931 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628011990 con 0x7f16400a1510 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.901+0000 7f163effd700 1 -- 172.21.15.32:0/3212008931 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011d20 con 0x7f16400a1510 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 -- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 msgr2=0x7f1640138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 --2- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640138d10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1628009920 tx=0x7f1628009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 -- 172.21.15.32:0/3212008931 shutdown_connections 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 --2- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f164013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 --2- 172.21.15.32:0/3212008931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640138d10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 -- 172.21.15.32:0/3212008931 >> 172.21.15.32:0/3212008931 conn(0x7f16400a6550 msgr2=0x7f16400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:33.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 -- 172.21.15.32:0/3212008931 shutdown_connections 2023-09-30T22:26:33.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.902+0000 7f1645df3700 1 -- 172.21.15.32:0/3212008931 wait complete. 2023-09-30T22:26:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 Processor -- start 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 -- start start 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640122720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640122c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f16401231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1640138200 con 0x7f16400a0a30 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16401261f0 con 0x7f1640139250 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1645df3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1640125970 con 0x7f16400a1510 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f1636ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.903+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640122720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640122c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49542/0 (socket says 172.21.15.32:49542) 2023-09-30T22:26:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 -- 172.21.15.32:0/4010424466 learned_addr learned my addr 172.21.15.32:0/4010424466 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f163ffff700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f16401231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 -- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 msgr2=0x7f16401231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f16401231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 -- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 msgr2=0x7f1640122720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640122720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 -- 172.21.15.32:0/4010424466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638009a70 con 0x7f16400a1510 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f163f7fe700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640122720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1636ffd700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640122c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1628000c00 tx=0x7f1628004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628004450 con 0x7f16400a1510 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16280045b0 con 0x7f16400a1510 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1628009550 con 0x7f16400a1510 2023-09-30T22:26:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280038a0 con 0x7f16400a1510 2023-09-30T22:26:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.904+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1640125c50 con 0x7f16400a1510 2023-09-30T22:26:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.905+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1628003a00 con 0x7f16400a1510 2023-09-30T22:26:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.905+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16400a1d00 con 0x7f16400a1510 2023-09-30T22:26:33.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.905+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1628021030 con 0x7f16400a1510 2023-09-30T22:26:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:33.910+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f162801a430 con 0x7f16400a1510 2023-09-30T22:26:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.079+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fab840747f0 con 0x7fab84133930 2023-09-30T22:26:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.080+0000 7fab7affd700 1 -- 172.21.15.32:0/4075223733 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab7c017020 con 0x7fab84133930 2023-09-30T22:26:34.081 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 msgr2=0x7fab841c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 0x7fab841c8ba0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c490 tx=0x7fab7c00c7a0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 shutdown_connections 2023-09-30T22:26:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab84133930 0x7fab841c8ba0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab8409d630 0x7fab841c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 --2- 172.21.15.32:0/4075223733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab840a0a30 0x7fab841c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 >> 172.21.15.32:0/4075223733 conn(0x7fab840a3050 msgr2=0x7fab8409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 shutdown_connections 2023-09-30T22:26:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.083+0000 7fab88bc2700 1 -- 172.21.15.32:0/4075223733 wait complete. 2023-09-30T22:26:34.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.126+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1640125e70 con 0x7f16400a1510 2023-09-30T22:26:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.126+0000 7f163cff9700 1 -- 172.21.15.32:0/4010424466 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1628015030 con 0x7f16400a1510 2023-09-30T22:26:34.127 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 msgr2=0x7f1640122c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640122c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1628000c00 tx=0x7f1628004750 comp rx=0 tx=0).stop 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 shutdown_connections 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1640139250 0x7f16401231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16400a0a30 0x7f1640122720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 --2- 172.21.15.32:0/4010424466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16400a1510 0x7f1640122c60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 >> 172.21.15.32:0/4010424466 conn(0x7f16400a6550 msgr2=0x7f164009aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 shutdown_connections 2023-09-30T22:26:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.128+0000 7f1645df3700 1 -- 172.21.15.32:0/4010424466 wait complete. 2023-09-30T22:26:34.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:34.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:34.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.783+0000 7f1dc0032700 1 Processor -- start 2023-09-30T22:26:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 -- start start 2023-09-30T22:26:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 0x7f1db80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 0x7f1db8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db80a2a70 con 0x7f1db809d630 2023-09-30T22:26:34.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80a01d0 con 0x7f1db80a0a30 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dc0032700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1db809e2b0 con 0x7f1db8133930 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dbe5cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 0x7f1db8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dbd5cd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 0x7f1db80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dbddce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dbd5cd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 0x7f1db80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60978/0 (socket says 172.21.15.32:60978) 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dbe5cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 0x7f1db8133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49544/0 (socket says 172.21.15.32:49544) 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.784+0000 7f1dbd5cd700 1 -- 172.21.15.32:0/2992510673 learned_addr learned my addr 172.21.15.32:0/2992510673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbe5cf700 1 -- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 msgr2=0x7f1db80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbe5cf700 1 --2- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbe5cf700 1 -- 172.21.15.32:0/2992510673 >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 msgr2=0x7f1db80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbe5cf700 1 --2- 172.21.15.32:0/2992510673 >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 0x7f1db80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbe5cf700 1 -- 172.21.15.32:0/2992510673 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db809dc50 con 0x7f1db8133930 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbd5cd700 1 --2- 172.21.15.32:0/2992510673 >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 0x7f1db80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbddce700 1 --2- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbe5cf700 1 --2- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 0x7f1db8133dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009d90 tx=0x7f1db000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9ae0160572748a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbcdcc700 1 -- 172.21.15.32:0/2992510673 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000e070 con 0x7f1db8133930 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbcdcc700 1 -- 172.21.15.32:0/2992510673 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db000bb20 con 0x7f1db8133930 2023-09-30T22:26:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.785+0000 7f1dbcdcc700 1 -- 172.21.15.32:0/2992510673 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0012430 con 0x7f1db8133930 2023-09-30T22:26:34.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 -- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 msgr2=0x7f1db8133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 --2- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 0x7f1db8133dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009d90 tx=0x7f1db000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 -- 172.21.15.32:0/2992510673 shutdown_connections 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 --2- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 --2- 172.21.15.32:0/2992510673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db8133930 0x7f1db8133dd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 --2- 172.21.15.32:0/2992510673 >> v2:172.21.15.32:3300/0 conn(0x7f1db809d630 0x7f1db80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 -- 172.21.15.32:0/2992510673 >> 172.21.15.32:0/2992510673 conn(0x7f1db80a3050 msgr2=0x7f1db80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 -- 172.21.15.32:0/2992510673 shutdown_connections 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.786+0000 7f1dc0032700 1 -- 172.21.15.32:0/2992510673 wait complete. 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.787+0000 7f1dc0032700 1 Processor -- start 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.787+0000 7f1dc0032700 1 -- start start 2023-09-30T22:26:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 0x7f1db81c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db81c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db8133930 0x7f1db81c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80a0410 con 0x7f1db8133930 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8120a90 con 0x7f1db80a0a30 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80a4150 con 0x7f1db809d630 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbddce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 0x7f1db81c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db81c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbddce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 0x7f1db81c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49548/0 (socket says 172.21.15.32:49548) 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbddce700 1 -- 172.21.15.32:0/4150188622 learned_addr learned my addr 172.21.15.32:0/4150188622 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 -- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 msgr2=0x7f1db81c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 0x7f1db81c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 -- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db8133930 msgr2=0x7f1db81c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db8133930 0x7f1db81c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 -- 172.21.15.32:0/4150188622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac009550 con 0x7f1db80a0a30 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbddce700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 0x7f1db81c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dbd5cd700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db81c8640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9a0 tx=0x7f1dac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac004240 con 0x7f1db80a0a30 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac0043a0 con 0x7f1db80a0a30 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.788+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da80094a0 con 0x7f1db80a0a30 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.789+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0036a0 con 0x7f1db80a0a30 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.789+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db0009a70 con 0x7f1db80a0a30 2023-09-30T22:26:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.789+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1dac003800 con 0x7f1db80a0a30 2023-09-30T22:26:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.789+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db8121db0 con 0x7f1db80a0a30 2023-09-30T22:26:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.790+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1dac03f840 con 0x7f1db80a0a30 2023-09-30T22:26:34.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.794+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1dac011350 con 0x7f1db80a0a30 2023-09-30T22:26:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 Processor -- start 2023-09-30T22:26:34.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 -- start start 2023-09-30T22:26:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 0x7fbabc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbabc133fa0 0x7fbabc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbabc09edd0 con 0x7fbabc133fa0 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc09e1f0 con 0x7fbabc0a0a30 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.824+0000 7fbac0f73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc09dc50 con 0x7fbabc0a1340 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 0x7fbabc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 0x7fbabc133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49562/0 (socket says 172.21.15.32:49562) 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbaba59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45188/0 (socket says 172.21.15.32:45188) 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 -- 172.21.15.32:0/992522253 learned_addr learned my addr 172.21.15.32:0/992522253 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbabad9d700 1 --2- 172.21.15.32:0/992522253 >> v2:172.21.15.32:3300/0 conn(0x7fbabc133fa0 0x7fbabc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 -- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 msgr2=0x7fbabc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:34.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 --2- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 -- 172.21.15.32:0/992522253 >> v2:172.21.15.32:3300/0 conn(0x7fbabc133fa0 msgr2=0x7fbabc134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 --2- 172.21.15.32:0/992522253 >> v2:172.21.15.32:3300/0 conn(0x7fbabc133fa0 0x7fbabc134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 -- 172.21.15.32:0/992522253 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0a1fe0 con 0x7fbabc0a1340 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbaba59c700 1 --2- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbabad9d700 1 --2- 172.21.15.32:0/992522253 >> v2:172.21.15.32:3300/0 conn(0x7fbabc133fa0 0x7fbabc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab9d9b700 1 --2- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 0x7fbabc133a60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098f0 tx=0x7fbaa4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c80859dc1873b08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.825+0000 7fbab959a700 1 -- 172.21.15.32:0/992522253 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011830 con 0x7fbabc0a1340 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbab959a700 1 -- 172.21.15.32:0/992522253 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa4011990 con 0x7fbabc0a1340 2023-09-30T22:26:34.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbab959a700 1 -- 172.21.15.32:0/992522253 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011d20 con 0x7fbabc0a1340 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 -- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 msgr2=0x7fbabc133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 --2- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 0x7fbabc133a60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098f0 tx=0x7fbaa4009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 -- 172.21.15.32:0/992522253 shutdown_connections 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 --2- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 --2- 172.21.15.32:0/992522253 >> v2:172.21.15.32:3300/0 conn(0x7fbabc133fa0 0x7fbabc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 --2- 172.21.15.32:0/992522253 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc0a1340 0x7fbabc133a60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 -- 172.21.15.32:0/992522253 >> 172.21.15.32:0/992522253 conn(0x7fbabc0a6550 msgr2=0x7fbabc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 -- 172.21.15.32:0/992522253 shutdown_connections 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.826+0000 7fbac0f73700 1 -- 172.21.15.32:0/992522253 wait complete. 2023-09-30T22:26:34.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.827+0000 7fbac0f73700 1 Processor -- start 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 -- start start 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc121400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 0x7fbabc121940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc133fa0 0x7fbabc121e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc124e80 con 0x7fbabc0a1340 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0a0290 con 0x7fbabc0a0a30 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbac0f73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0a36e0 con 0x7fbabc133fa0 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbab9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 0x7fbabc121940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbabad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc133fa0 0x7fbabc121e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbab9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 0x7fbabc121940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32770/0 (socket says 172.21.15.32:32770) 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbab9d9b700 1 -- 172.21.15.32:0/2059933594 learned_addr learned my addr 172.21.15.32:0/2059933594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:34.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.828+0000 7fbaba59c700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc121400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab9d9b700 1 -- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc133fa0 msgr2=0x7fbabc121e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab9d9b700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc133fa0 0x7fbabc121e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab9d9b700 1 -- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 msgr2=0x7fbabc121400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab9d9b700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc121400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab9d9b700 1 -- 172.21.15.32:0/2059933594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac0094a0 con 0x7fbabc0a1340 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbabad9d700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc133fa0 0x7fbabc121e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbaba59c700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc121400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab9d9b700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 0x7fbabc121940 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4004240 con 0x7fbabc0a1340 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa401f070 con 0x7fbabc0a1340 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4009a70 con 0x7fbabc0a1340 2023-09-30T22:26:34.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40045c0 con 0x7fbabc0a1340 2023-09-30T22:26:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.829+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa4009550 con 0x7fbabc0a1340 2023-09-30T22:26:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.830+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbaa4021030 con 0x7fbabc0a1340 2023-09-30T22:26:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.830+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbaa4040840 con 0x7fbabc0a1340 2023-09-30T22:26:34.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.830+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc074ea0 con 0x7fbabc0a1340 2023-09-30T22:26:34.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:34.835+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbaa4015070 con 0x7fbabc0a1340 2023-09-30T22:26:35.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.001+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1db80747f0 con 0x7f1db80a0a30 2023-09-30T22:26:35.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.001+0000 7f1db67fc700 1 -- 172.21.15.32:0/4150188622 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1dac01bb60 con 0x7f1db80a0a30 2023-09-30T22:26:35.002 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.003+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 msgr2=0x7f1db81c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.003+0000 7f1dc0032700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db81c8640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9a0 tx=0x7f1dac00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 shutdown_connections 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1db8133930 0x7f1db81c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1db809d630 0x7f1db81c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 --2- 172.21.15.32:0/4150188622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1db80a0a30 0x7f1db81c8640 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 >> 172.21.15.32:0/4150188622 conn(0x7f1db80a3050 msgr2=0x7f1db81bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 shutdown_connections 2023-09-30T22:26:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.004+0000 7f1dc0032700 1 -- 172.21.15.32:0/4150188622 wait complete. 2023-09-30T22:26:35.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:35.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.039+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbabc123d90 con 0x7fbabc0a1340 2023-09-30T22:26:35.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.040+0000 7fbab2ffd700 1 -- 172.21.15.32:0/2059933594 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbaa40199a0 con 0x7fbabc0a1340 2023-09-30T22:26:35.041 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 msgr2=0x7fbabc121940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 0x7fbabc121940 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa4004750 comp rx=0 tx=0).stop 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 shutdown_connections 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbabc133fa0 0x7fbabc121e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbabc0a0a30 0x7fbabc121400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 --2- 172.21.15.32:0/2059933594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbabc0a1340 0x7fbabc121940 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 >> 172.21.15.32:0/2059933594 conn(0x7fbabc0a6550 msgr2=0x7fbabc0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 shutdown_connections 2023-09-30T22:26:35.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.042+0000 7fbac0f73700 1 -- 172.21.15.32:0/2059933594 wait complete. 2023-09-30T22:26:35.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:35.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:35.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:35.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.246+0000 7fa2696f8700 1 Processor -- start 2023-09-30T22:26:35.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.246+0000 7fa2696f8700 1 -- start start 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2696f8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa264133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2696f8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 0x7fa264130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2696f8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26406bc00 con 0x7fa264130870 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2696f8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26406c570 con 0x7fa26412ff20 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2696f8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640a0040 con 0x7fa264133a80 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2627fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 0x7fa264130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa262ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa264133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2627fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 0x7fa264130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32776/0 (socket says 172.21.15.32:32776) 2023-09-30T22:26:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa262ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa264133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45198/0 (socket says 172.21.15.32:45198) 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2627fc700 1 -- 172.21.15.32:0/1847389479 learned_addr learned my addr 172.21.15.32:0/1847389479 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.247+0000 7fa2637fe700 1 --1- 172.21.15.32:0/1847389479 >> v1:172.21.15.32:6789/0 conn(0x7fa264130870 0x7fa26413a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43394/0 (socket says 172.21.15.32:43394) 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa261ffb700 1 -- 172.21.15.32:0/1847389479 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425031658 0 0) 0x7fa26406bc00 con 0x7fa264130870 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa261ffb700 1 -- 172.21.15.32:0/1847389479 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa24c0035b0 con 0x7fa264130870 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa262ffd700 1 -- 172.21.15.32:0/1847389479 >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 msgr2=0x7fa264130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa262ffd700 1 --2- 172.21.15.32:0/1847389479 >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 0x7fa264130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa262ffd700 1 -- 172.21.15.32:0/1847389479 >> v1:172.21.15.32:6789/0 conn(0x7fa264130870 legacy=0x7fa26413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa262ffd700 1 -- 172.21.15.32:0/1847389479 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264133860 con 0x7fa264133a80 2023-09-30T22:26:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa2627fc700 1 --2- 172.21.15.32:0/1847389479 >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 0x7fa264130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa262ffd700 1 --2- 172.21.15.32:0/1847389479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa264133e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa254009840 tx=0x7fa254009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5d855d8c4a2699 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa261ffb700 1 -- 172.21.15.32:0/1847389479 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011830 con 0x7fa264133a80 2023-09-30T22:26:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa261ffb700 1 -- 172.21.15.32:0/1847389479 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254011990 con 0x7fa264133a80 2023-09-30T22:26:35.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.248+0000 7fa261ffb700 1 -- 172.21.15.32:0/1847389479 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254011d20 con 0x7fa264133a80 2023-09-30T22:26:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 -- 172.21.15.32:0/1847389479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 msgr2=0x7fa264133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 --2- 172.21.15.32:0/1847389479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa264133e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa254009840 tx=0x7fa254009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 -- 172.21.15.32:0/1847389479 shutdown_connections 2023-09-30T22:26:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 --2- 172.21.15.32:0/1847389479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa264133e50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 --2- 172.21.15.32:0/1847389479 >> v2:172.21.15.32:3300/0 conn(0x7fa26412ff20 0x7fa264130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 -- 172.21.15.32:0/1847389479 >> 172.21.15.32:0/1847389479 conn(0x7fa2640a4210 msgr2=0x7fa2640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 -- 172.21.15.32:0/1847389479 shutdown_connections 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.249+0000 7fa2696f8700 1 -- 172.21.15.32:0/1847389479 wait complete. 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 Processor -- start 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 -- start start 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26412ff20 0x7fa26409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 0x7fa26409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa26409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26411edb0 con 0x7fa264130870 2023-09-30T22:26:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26411f420 con 0x7fa264133a80 2023-09-30T22:26:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2696f8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa26411fb20 con 0x7fa26412ff20 2023-09-30T22:26:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.251+0000 7fa2627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 0x7fa26409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 0x7fa26409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32778/0 (socket says 172.21.15.32:32778) 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2627fc700 1 -- 172.21.15.32:0/4221690092 learned_addr learned my addr 172.21.15.32:0/4221690092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa26409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa262ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26412ff20 0x7fa26409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 -- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26412ff20 msgr2=0x7fa26409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26412ff20 0x7fa26409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 -- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 msgr2=0x7fa26409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 0x7fa26409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 -- 172.21.15.32:0/4221690092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009550 con 0x7fa264133a80 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa262ffd700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26412ff20 0x7fa26409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2627fc700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 0x7fa26409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2637fe700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa26409cf00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00cb40 tx=0x7fa25c00ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00a9e0 con 0x7fa264133a80 2023-09-30T22:26:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c00ab40 con 0x7fa264133a80 2023-09-30T22:26:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c007070 con 0x7fa264133a80 2023-09-30T22:26:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2540094a0 con 0x7fa264133a80 2023-09-30T22:26:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.252+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26411ea90 con 0x7fa264133a80 2023-09-30T22:26:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.253+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa25c0071d0 con 0x7fa264133a80 2023-09-30T22:26:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.253+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa25c03f350 con 0x7fa264133a80 2023-09-30T22:26:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.253+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26411fce0 con 0x7fa264133a80 2023-09-30T22:26:35.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.258+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa25c01b3b0 con 0x7fa264133a80 2023-09-30T22:26:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 Processor -- start 2023-09-30T22:26:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 -- start start 2023-09-30T22:26:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 0x7f315009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315009e410 0x7f3150134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315009b120 con 0x7f315009e000 2023-09-30T22:26:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315006b460 con 0x7f315009e410 2023-09-30T22:26:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.288+0000 7f3154c78700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315009a7e0 con 0x7f315009d6f0 2023-09-30T22:26:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 0x7f315009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315009e410 0x7f3150134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 0x7f315009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45214/0 (socket says 172.21.15.32:45214) 2023-09-30T22:26:35.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314e59c700 1 -- 172.21.15.32:0/677559402 learned_addr learned my addr 172.21.15.32:0/677559402 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314ed9d700 1 -- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 msgr2=0x7f315009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:35.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314ed9d700 1 --2- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 0x7f315009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314ed9d700 1 -- 172.21.15.32:0/677559402 >> v1:172.21.15.32:6789/0 conn(0x7f315009e000 legacy=0x7f31501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314ed9d700 1 -- 172.21.15.32:0/677559402 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315009a500 con 0x7f315009e410 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314d59a700 1 -- 172.21.15.32:0/677559402 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531059690 0 0) 0x7f315009b120 con 0x7f315009e000 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314e59c700 1 --2- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 0x7f315009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314ed9d700 1 --2- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009e410 0x7f3150134100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3148009d90 tx=0x7f314800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85af66876675a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314d59a700 1 -- 172.21.15.32:0/677559402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314800e070 con 0x7f315009e410 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314d59a700 1 -- 172.21.15.32:0/677559402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314800bb20 con 0x7f315009e410 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.289+0000 7f314d59a700 1 -- 172.21.15.32:0/677559402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148012430 con 0x7f315009e410 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 -- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009e410 msgr2=0x7f3150134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 --2- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009e410 0x7f3150134100 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3148009d90 tx=0x7f314800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 -- 172.21.15.32:0/677559402 shutdown_connections 2023-09-30T22:26:35.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 --2- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009e410 0x7f3150134100 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 --2- 172.21.15.32:0/677559402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009d6f0 0x7f315009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 -- 172.21.15.32:0/677559402 >> 172.21.15.32:0/677559402 conn(0x7f3150066f50 msgr2=0x7f315009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 -- 172.21.15.32:0/677559402 shutdown_connections 2023-09-30T22:26:35.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.290+0000 7f3154c78700 1 -- 172.21.15.32:0/677559402 wait complete. 2023-09-30T22:26:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.291+0000 7f3154c78700 1 Processor -- start 2023-09-30T22:26:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.291+0000 7f3154c78700 1 -- start start 2023-09-30T22:26:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f3154c78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d6f0 0x7f31501c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f3154c78700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009e000 0x7f31501c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f3154c78700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 0x7f31501c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f3154c78700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31501a8a30 con 0x7f315009d6f0 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f3154c78700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31501a9780 con 0x7f315009e410 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f3154c78700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31501c8e80 con 0x7f315009e000 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 0x7f31501c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009e000 0x7f31501c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 0x7f31501c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45216/0 (socket says 172.21.15.32:45216) 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 -- 172.21.15.32:0/3743651118 learned_addr learned my addr 172.21.15.32:0/3743651118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314e59c700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d6f0 0x7f31501c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 -- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009e000 msgr2=0x7f31501c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009e000 0x7f31501c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 -- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d6f0 msgr2=0x7f31501c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d6f0 0x7f31501c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314ed9d700 1 -- 172.21.15.32:0/3743651118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140009550 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314dd9b700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009e000 0x7f31501c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.292+0000 7f314e59c700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d6f0 0x7f31501c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.293+0000 7f314ed9d700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 0x7f31501c49d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f314800b680 tx=0x7f3148004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.293+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314801b810 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.293+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3148020070 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.293+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3148009a70 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.293+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314801bb90 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.293+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31501a85e0 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.294+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f314801bcf0 con 0x7f315009e410 2023-09-30T22:26:35.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.294+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3148022070 con 0x7f315009e410 2023-09-30T22:26:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.294+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3150074ea0 con 0x7f315009e410 2023-09-30T22:26:35.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.299+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f314802f350 con 0x7f315009e410 2023-09-30T22:26:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.470+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa264074ea0 con 0x7fa264133a80 2023-09-30T22:26:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.471+0000 7fa24b7fe700 1 -- 172.21.15.32:0/4221690092 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa25c017020 con 0x7fa264133a80 2023-09-30T22:26:35.472 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 msgr2=0x7fa26409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa26409cf00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00cb40 tx=0x7fa25c00ce50 comp rx=0 tx=0).stop 2023-09-30T22:26:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 shutdown_connections 2023-09-30T22:26:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa264133a80 0x7fa26409cf00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26412ff20 0x7fa26409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 --2- 172.21.15.32:0/4221690092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa264130870 0x7fa26409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 >> 172.21.15.32:0/4221690092 conn(0x7fa2640a4210 msgr2=0x7fa26409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 shutdown_connections 2023-09-30T22:26:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.473+0000 7fa2696f8700 1 -- 172.21.15.32:0/4221690092 wait complete. 2023-09-30T22:26:35.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.509+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31501a7870 con 0x7f315009e410 2023-09-30T22:26:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.510+0000 7f313effd700 1 -- 172.21.15.32:0/3743651118 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f314802c030 con 0x7f315009e410 2023-09-30T22:26:35.511 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 msgr2=0x7f31501c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 0x7f31501c49d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f314800b680 tx=0x7f3148004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 shutdown_connections 2023-09-30T22:26:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315009e410 0x7f31501c49d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d6f0 0x7f31501c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 --2- 172.21.15.32:0/3743651118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009e000 0x7f31501c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 >> 172.21.15.32:0/3743651118 conn(0x7f3150066f50 msgr2=0x7f31501bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 shutdown_connections 2023-09-30T22:26:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:35.512+0000 7f3154c78700 1 -- 172.21.15.32:0/3743651118 wait complete. 2023-09-30T22:26:35.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:36.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:36.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 Processor -- start 2023-09-30T22:26:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 -- start start 2023-09-30T22:26:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ce40a0a30 0x7f7ce40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce409eaf0 con 0x7f7ce40a1340 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce40a27c0 con 0x7f7ce40a0a30 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.167+0000 7f7ce9c82700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce409dbe0 con 0x7f7ce4134000 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce413bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49590/0 (socket says 172.21.15.32:49590) 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce3fff700 1 -- 172.21.15.32:0/3891287876 learned_addr learned my addr 172.21.15.32:0/3891287876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce37fe700 1 --2- 172.21.15.32:0/3891287876 >> v2:172.21.15.32:3300/0 conn(0x7f7ce40a0a30 0x7f7ce40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce37fe700 1 -- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 msgr2=0x7f7ce413bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce37fe700 1 --2- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce413bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce37fe700 1 -- 172.21.15.32:0/3891287876 >> v1:172.21.15.32:6789/0 conn(0x7f7ce40a1340 legacy=0x7f7ce41338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce37fe700 1 -- 172.21.15.32:0/3891287876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40a1f90 con 0x7f7ce40a0a30 2023-09-30T22:26:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce27fc700 1 -- 172.21.15.32:0/3891287876 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574672039 0 0) 0x7f7cd8001110 con 0x7f7ce40a1340 2023-09-30T22:26:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce3fff700 1 --2- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce413bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce37fe700 1 --2- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a0a30 0x7f7ce40a0e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009840 tx=0x7f7cd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658ee4d1fa674b1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce27fc700 1 -- 172.21.15.32:0/3891287876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4011830 con 0x7f7ce40a0a30 2023-09-30T22:26:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce27fc700 1 -- 172.21.15.32:0/3891287876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4011990 con 0x7f7ce40a0a30 2023-09-30T22:26:36.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.168+0000 7f7ce27fc700 1 -- 172.21.15.32:0/3891287876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4011d20 con 0x7f7ce40a0a30 2023-09-30T22:26:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 -- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a0a30 msgr2=0x7f7ce40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a0a30 0x7f7ce40a0e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009840 tx=0x7f7cd4009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 -- 172.21.15.32:0/3891287876 shutdown_connections 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce413bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/3891287876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a0a30 0x7f7ce40a0e00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 -- 172.21.15.32:0/3891287876 >> 172.21.15.32:0/3891287876 conn(0x7f7ce40a6550 msgr2=0x7f7ce40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 -- 172.21.15.32:0/3891287876 shutdown_connections 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.169+0000 7f7ce9c82700 1 -- 172.21.15.32:0/3891287876 wait complete. 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.170+0000 7f7ce9c82700 1 Processor -- start 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.170+0000 7f7ce9c82700 1 -- start start 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce9c82700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ce40a0a30 0x7f7ce4122690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce9c82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a1340 0x7f7ce4122bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce9c82700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce4123110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce9c82700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce41261b0 con 0x7f7ce40a1340 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce9c82700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce41335e0 con 0x7f7ce40a0a30 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce9c82700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce41242a0 con 0x7f7ce4134000 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce4123110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce4123110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49602/0 (socket says 172.21.15.32:49602) 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce3fff700 1 -- 172.21.15.32:0/1642770392 learned_addr learned my addr 172.21.15.32:0/1642770392 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce37fe700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ce40a0a30 0x7f7ce4122690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce2ffd700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a1340 0x7f7ce4122bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce3fff700 1 -- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ce40a0a30 msgr2=0x7f7ce4122690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.171+0000 7f7ce3fff700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ce40a0a30 0x7f7ce4122690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ce3fff700 1 -- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a1340 msgr2=0x7f7ce4122bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ce3fff700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a1340 0x7f7ce4122bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ce3fff700 1 -- 172.21.15.32:0/1642770392 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc009b30 con 0x7f7ce4134000 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ce3fff700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce4123110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00efc0 tx=0x7f7cdc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc018070 con 0x7f7ce4134000 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc00f040 con 0x7f7ce4134000 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd40094a0 con 0x7f7ce4134000 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc003c20 con 0x7f7ce4134000 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.172+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce409d710 con 0x7f7ce4134000 2023-09-30T22:26:36.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.173+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7cdc00c880 con 0x7f7ce4134000 2023-09-30T22:26:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.173+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7cdc03f820 con 0x7f7ce4134000 2023-09-30T22:26:36.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.173+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce4125b20 con 0x7f7ce4134000 2023-09-30T22:26:36.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.182+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7cdc014480 con 0x7f7ce4134000 2023-09-30T22:26:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.210+0000 7f0656fce700 1 Processor -- start 2023-09-30T22:26:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 -- start start 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06500a0a30 0x7f06500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 0x7f06500a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 0x7f0650133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06500a2a70 con 0x7f06500a0a30 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500a01d0 con 0x7f065009d630 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0656fce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f065009e2b0 con 0x7f0650133930 2023-09-30T22:26:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f064ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 0x7f06500a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f065556b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 0x7f0650133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f065556b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 0x7f0650133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49616/0 (socket says 172.21.15.32:49616) 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f064ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 0x7f06500a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45238/0 (socket says 172.21.15.32:45238) 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f065556b700 1 -- 172.21.15.32:0/759375704 learned_addr learned my addr 172.21.15.32:0/759375704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.211+0000 7f0654d6a700 1 --2- 172.21.15.32:0/759375704 >> v2:172.21.15.32:3300/0 conn(0x7f06500a0a30 0x7f06500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064ffff700 1 -- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 msgr2=0x7f0650133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064ffff700 1 --2- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 0x7f0650133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064ffff700 1 -- 172.21.15.32:0/759375704 >> v2:172.21.15.32:3300/0 conn(0x7f06500a0a30 msgr2=0x7f06500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064ffff700 1 --2- 172.21.15.32:0/759375704 >> v2:172.21.15.32:3300/0 conn(0x7f06500a0a30 0x7f06500a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064ffff700 1 -- 172.21.15.32:0/759375704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065009dc50 con 0x7f065009d630 2023-09-30T22:26:36.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f065556b700 1 --2- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 0x7f0650133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f0654d6a700 1 --2- 172.21.15.32:0/759375704 >> v2:172.21.15.32:3300/0 conn(0x7f06500a0a30 0x7f06500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064ffff700 1 --2- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 0x7f06500a1340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0644009840 tx=0x7f0644009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf97812c68e7ee14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064f7fe700 1 -- 172.21.15.32:0/759375704 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011830 con 0x7f065009d630 2023-09-30T22:26:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064f7fe700 1 -- 172.21.15.32:0/759375704 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644011990 con 0x7f065009d630 2023-09-30T22:26:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.212+0000 7f064f7fe700 1 -- 172.21.15.32:0/759375704 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011d20 con 0x7f065009d630 2023-09-30T22:26:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 -- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 msgr2=0x7f06500a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 --2- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 0x7f06500a1340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0644009840 tx=0x7f0644009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 -- 172.21.15.32:0/759375704 shutdown_connections 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 --2- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0650133930 0x7f0650133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 --2- 172.21.15.32:0/759375704 >> v2:172.21.15.32:3300/0 conn(0x7f06500a0a30 0x7f06500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 --2- 172.21.15.32:0/759375704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f065009d630 0x7f06500a1340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 -- 172.21.15.32:0/759375704 >> 172.21.15.32:0/759375704 conn(0x7f06500a3050 msgr2=0x7f06500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 -- 172.21.15.32:0/759375704 shutdown_connections 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.213+0000 7f0656fce700 1 -- 172.21.15.32:0/759375704 wait complete. 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.214+0000 7f0656fce700 1 Processor -- start 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.214+0000 7f0656fce700 1 -- start start 2023-09-30T22:26:36.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.214+0000 7f0656fce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 0x7f06501c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.214+0000 7f0656fce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06500a0a30 0x7f06501c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0656fce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0650133930 0x7f06501c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0656fce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650121140 con 0x7f06500a0a30 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0656fce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500a40c0 con 0x7f0650133930 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0656fce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0650120570 con 0x7f065009d630 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 0x7f06501c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 0x7f06501c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49624/0 (socket says 172.21.15.32:49624) 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 -- 172.21.15.32:0/2622473136 learned_addr learned my addr 172.21.15.32:0/2622473136 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f064ffff700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06500a0a30 0x7f06501c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f065556b700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0650133930 0x7f06501c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 -- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0650133930 msgr2=0x7f06501c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0650133930 0x7f06501c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 -- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06500a0a30 msgr2=0x7f06501c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06500a0a30 0x7f06501c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 -- 172.21.15.32:0/2622473136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400094a0 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0654d6a700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 0x7f06501c8090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f064000ebe0 tx=0x7f064000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064000c850 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064000c9b0 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064000cd40 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.215+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0648009b30 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.216+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06440094a0 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.216+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0640013470 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.216+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f064003f770 con 0x7f065009d630 2023-09-30T22:26:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.217+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0650121cb0 con 0x7f065009d630 2023-09-30T22:26:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.221+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0640013720 con 0x7f065009d630 2023-09-30T22:26:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.391+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ce404f2a0 con 0x7f7ce4134000 2023-09-30T22:26:36.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.391+0000 7f7ccbfff700 1 -- 172.21.15.32:0/1642770392 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7cdc01cb60 con 0x7f7ce4134000 2023-09-30T22:26:36.392 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.393+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 msgr2=0x7f7ce4123110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.393+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce4123110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00efc0 tx=0x7f7cdc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 shutdown_connections 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ce4134000 0x7f7ce4123110 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ce40a0a30 0x7f7ce4122690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 --2- 172.21.15.32:0/1642770392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ce40a1340 0x7f7ce4122bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 >> 172.21.15.32:0/1642770392 conn(0x7f7ce40a6550 msgr2=0x7f7ce40a2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 shutdown_connections 2023-09-30T22:26:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.394+0000 7f7ce9c82700 1 -- 172.21.15.32:0/1642770392 wait complete. 2023-09-30T22:26:36.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:36.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.428+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06500747f0 con 0x7f065009d630 2023-09-30T22:26:36.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.428+0000 7f064d7fa700 1 -- 172.21.15.32:0/2622473136 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0640021030 con 0x7f065009d630 2023-09-30T22:26:36.429 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:36.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 msgr2=0x7f06501c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 0x7f06501c8090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f064000ebe0 tx=0x7f064000eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 shutdown_connections 2023-09-30T22:26:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0650133930 0x7f06501c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f065009d630 0x7f06501c8090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 --2- 172.21.15.32:0/2622473136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06500a0a30 0x7f06501c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 >> 172.21.15.32:0/2622473136 conn(0x7f06500a3050 msgr2=0x7f065009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 shutdown_connections 2023-09-30T22:26:36.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.430+0000 7f0656fce700 1 -- 172.21.15.32:0/2622473136 wait complete. 2023-09-30T22:26:36.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:36.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:36.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:36.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.633+0000 7fc36e407700 1 Processor -- start 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.633+0000 7fc36e407700 1 -- start start 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.633+0000 7fc36e407700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 0x7fc368136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36e407700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc3681393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36e407700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc368070940 con 0x7fc36809e930 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36e407700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36806fd60 con 0x7fc36809f410 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36e407700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc36806fa80 con 0x7fc368136fe0 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc3677fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 0x7fc368136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc3681393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc367fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc36809e930 0x7fc36809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43422/0 (socket says 172.21.15.32:43422) 2023-09-30T22:26:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc3677fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 0x7fc368136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32832/0 (socket says 172.21.15.32:32832) 2023-09-30T22:26:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc3681393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49632/0 (socket says 172.21.15.32:49632) 2023-09-30T22:26:36.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc367fff700 1 -- 172.21.15.32:0/533209915 learned_addr learned my addr 172.21.15.32:0/533209915 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:36.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 -- 172.21.15.32:0/533209915 >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 msgr2=0x7fc368136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 --2- 172.21.15.32:0/533209915 >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 0x7fc368136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 -- 172.21.15.32:0/533209915 >> v1:172.21.15.32:6789/0 conn(0x7fc36809e930 legacy=0x7fc36809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 -- 172.21.15.32:0/533209915 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36809fb60 con 0x7fc368136fe0 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc366ffd700 1 -- 172.21.15.32:0/533209915 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088980789 0 0) 0x7fc368070940 con 0x7fc36809e930 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc3677fe700 1 --2- 172.21.15.32:0/533209915 >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 0x7fc368136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc36c9a4700 1 --2- 172.21.15.32:0/533209915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc3681393c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc360009d90 tx=0x7fc36000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d3aa12d53a4d4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.634+0000 7fc366ffd700 1 -- 172.21.15.32:0/533209915 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000e070 con 0x7fc368136fe0 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc366ffd700 1 -- 172.21.15.32:0/533209915 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000bb20 con 0x7fc368136fe0 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc366ffd700 1 -- 172.21.15.32:0/533209915 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360012430 con 0x7fc368136fe0 2023-09-30T22:26:36.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 -- 172.21.15.32:0/533209915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 msgr2=0x7fc3681393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 --2- 172.21.15.32:0/533209915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc3681393c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc360009d90 tx=0x7fc36000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 -- 172.21.15.32:0/533209915 shutdown_connections 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 --2- 172.21.15.32:0/533209915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc3681393c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 --2- 172.21.15.32:0/533209915 >> v2:172.21.15.32:3300/0 conn(0x7fc36809f410 0x7fc368136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 -- 172.21.15.32:0/533209915 >> 172.21.15.32:0/533209915 conn(0x7fc3680a4210 msgr2=0x7fc3680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 -- 172.21.15.32:0/533209915 shutdown_connections 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.635+0000 7fc36e407700 1 -- 172.21.15.32:0/533209915 wait complete. 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.636+0000 7fc36e407700 1 Processor -- start 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 -- start start 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc36809e930 0x7fc36811e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 0x7fc36811e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc36811ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36811ffc0 con 0x7fc36809f410 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc368121210 con 0x7fc36809e930 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36e407700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680a3260 con 0x7fc368136fe0 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 0x7fc36811e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 0x7fc36811e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32836/0 (socket says 172.21.15.32:32836) 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc3677fe700 1 -- 172.21.15.32:0/2812135722 learned_addr learned my addr 172.21.15.32:0/2812135722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.637+0000 7fc36c9a4700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc36811ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc3677fe700 1 -- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 msgr2=0x7fc36811ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc3677fe700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc36811ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc367fff700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc36809e930 0x7fc36811e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc3677fe700 1 -- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc36809e930 msgr2=0x7fc36811e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc3677fe700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc36809e930 0x7fc36811e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc3677fe700 1 -- 172.21.15.32:0/2812135722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0094a0 con 0x7fc36809f410 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc36c9a4700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc36811ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc367fff700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc36809e930 0x7fc36811e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc3677fe700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 0x7fc36811e8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00ebe0 tx=0x7fc35c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c009800 con 0x7fc36809f410 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c00c910 con 0x7fc36809f410 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc360009a70 con 0x7fc36809f410 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00cc80 con 0x7fc36809f410 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.638+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc368121c40 con 0x7fc36809f410 2023-09-30T22:26:36.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.639+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc35c013470 con 0x7fc36809f410 2023-09-30T22:26:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.639+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc35c03f670 con 0x7fc36809f410 2023-09-30T22:26:36.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.639+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc368121c40 con 0x7fc36809f410 2023-09-30T22:26:36.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.644+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc368121c40 con 0x7fc36809f410 2023-09-30T22:26:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.673+0000 7f7bb009d700 1 Processor -- start 2023-09-30T22:26:36.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.673+0000 7f7bb009d700 1 -- start start 2023-09-30T22:26:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bb009d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 0x7f7ba80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bb009d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 0x7f7ba8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bb009d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bb009d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba809edd0 con 0x7f7ba80a1340 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bb009d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba809e1f0 con 0x7f7ba8133fa0 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bb009d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba809dc50 con 0x7f7ba80a0a30 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bad638700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 0x7f7ba8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bade39700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 0x7f7ba80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bae63a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bade39700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 0x7f7ba80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49652/0 (socket says 172.21.15.32:49652) 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bad638700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 0x7f7ba8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32842/0 (socket says 172.21.15.32:32842) 2023-09-30T22:26:36.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bade39700 1 -- 172.21.15.32:0/3618448507 learned_addr learned my addr 172.21.15.32:0/3618448507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bae63a700 1 -- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 msgr2=0x7f7ba80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bae63a700 1 --2- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 0x7f7ba80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bae63a700 1 -- 172.21.15.32:0/3618448507 >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 msgr2=0x7f7ba8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bae63a700 1 --2- 172.21.15.32:0/3618448507 >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 0x7f7ba8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bae63a700 1 -- 172.21.15.32:0/3618448507 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80a1fe0 con 0x7f7ba8133fa0 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bade39700 1 --2- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 0x7f7ba80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.674+0000 7f7bad638700 1 --2- 172.21.15.32:0/3618448507 >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 0x7f7ba8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bae63a700 1 --2- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8134410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009dc0 tx=0x7f7b9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8180d5d65cd6f24 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bace37700 1 -- 172.21.15.32:0/3618448507 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800e070 con 0x7f7ba8133fa0 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bace37700 1 -- 172.21.15.32:0/3618448507 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9800bae0 con 0x7f7ba8133fa0 2023-09-30T22:26:36.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bace37700 1 -- 172.21.15.32:0/3618448507 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800be90 con 0x7f7ba8133fa0 2023-09-30T22:26:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 -- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 msgr2=0x7f7ba8134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 --2- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8134410 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009dc0 tx=0x7f7b9800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 -- 172.21.15.32:0/3618448507 shutdown_connections 2023-09-30T22:26:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 --2- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8134410 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 --2- 172.21.15.32:0/3618448507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a0a30 0x7f7ba80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 --2- 172.21.15.32:0/3618448507 >> v2:172.21.15.32:3300/0 conn(0x7f7ba80a1340 0x7f7ba8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 -- 172.21.15.32:0/3618448507 >> 172.21.15.32:0/3618448507 conn(0x7f7ba80a6550 msgr2=0x7f7ba80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 -- 172.21.15.32:0/3618448507 shutdown_connections 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.675+0000 7f7bb009d700 1 -- 172.21.15.32:0/3618448507 wait complete. 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 Processor -- start 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 -- start start 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba80a0a30 0x7f7ba81213c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 0x7f7ba8121900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8121e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:36.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8124c00 con 0x7f7ba80a0a30 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8124ee0 con 0x7f7ba8133fa0 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bb009d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80a37e0 con 0x7f7ba80a1340 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.677+0000 7f7bad638700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 0x7f7ba8121900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 0x7f7ba8121900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49666/0 (socket says 172.21.15.32:49666) 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 -- 172.21.15.32:0/4054850266 learned_addr learned my addr 172.21.15.32:0/4054850266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bade39700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba80a0a30 0x7f7ba81213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bae63a700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8121e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 -- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 msgr2=0x7f7ba8121e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8121e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 -- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba80a0a30 msgr2=0x7f7ba81213c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba80a0a30 0x7f7ba81213c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 -- 172.21.15.32:0/4054850266 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0009550 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bad638700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 0x7f7ba8121900 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00ebe0 tx=0x7f7b9c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c009760 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00c850 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00cbe0 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0094a0 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.678+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b98009a70 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.679+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b9c00cd40 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.679+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7b9c03fa20 con 0x7f7ba80a1340 2023-09-30T22:26:36.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.679+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba8074ea0 con 0x7f7ba80a1340 2023-09-30T22:26:36.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.689+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b9c01b3b0 con 0x7f7ba80a1340 2023-09-30T22:26:36.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.856+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc368121c40 con 0x7fc36809f410 2023-09-30T22:26:36.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.857+0000 7fc364ff9700 1 -- 172.21.15.32:0/2812135722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc368121c40 con 0x7fc36809f410 2023-09-30T22:26:36.858 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 msgr2=0x7fc36811e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 0x7fc36811e8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00ebe0 tx=0x7fc35c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 shutdown_connections 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc368136fe0 0x7fc36811ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc36809e930 0x7fc36811e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 --2- 172.21.15.32:0/2812135722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc36809f410 0x7fc36811e8f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 >> 172.21.15.32:0/2812135722 conn(0x7fc3680a4210 msgr2=0x7fc368134890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 shutdown_connections 2023-09-30T22:26:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.859+0000 7fc36e407700 1 -- 172.21.15.32:0/2812135722 wait complete. 2023-09-30T22:26:36.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.895+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7ba8123df0 con 0x7f7ba80a1340 2023-09-30T22:26:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.895+0000 7f7ba67fc700 1 -- 172.21.15.32:0/4054850266 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b9c017030 con 0x7f7ba80a1340 2023-09-30T22:26:36.896 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.897+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 msgr2=0x7f7ba8121900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.897+0000 7f7bb009d700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 0x7f7ba8121900 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00ebe0 tx=0x7f7b9c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 shutdown_connections 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7ba8133fa0 0x7f7ba8121e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba80a0a30 0x7f7ba81213c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 --2- 172.21.15.32:0/4054850266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7ba80a1340 0x7f7ba8121900 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 >> 172.21.15.32:0/4054850266 conn(0x7f7ba80a6550 msgr2=0x7f7ba80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 shutdown_connections 2023-09-30T22:26:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:36.898+0000 7f7bb009d700 1 -- 172.21.15.32:0/4054850266 wait complete. 2023-09-30T22:26:36.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:37.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:37.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.556+0000 7f373c568700 1 Processor -- start 2023-09-30T22:26:37.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.556+0000 7f373c568700 1 -- start start 2023-09-30T22:26:37.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373c568700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373c568700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37340a1340 0x7f37341338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373c568700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373409eaf0 con 0x7f3734133e30 2023-09-30T22:26:37.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373c568700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37340a27c0 con 0x7f37340a1340 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373c568700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f373409dbe0 con 0x7f37340a0a30 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f3739b03700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37340a1340 0x7f37341338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373ab05700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3734133e30 0x7f373413ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43430/0 (socket says 172.21.15.32:43430) 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373ab05700 1 -- 172.21.15.32:0/2682882023 learned_addr learned my addr 172.21.15.32:0/2682882023 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.557+0000 7f373a304700 1 --2- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739b03700 1 -- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37340a0a30 msgr2=0x7f37340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739b03700 1 --2- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739b03700 1 -- 172.21.15.32:0/2682882023 >> v1:172.21.15.32:6789/0 conn(0x7f3734133e30 legacy=0x7f373413ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739b03700 1 -- 172.21.15.32:0/2682882023 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340a1f90 con 0x7f37340a1340 2023-09-30T22:26:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739302700 1 -- 172.21.15.32:0/2682882023 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063210382 0 0) 0x7f372c001110 con 0x7f3734133e30 2023-09-30T22:26:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739b03700 1 --2- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f37341338f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3728009840 tx=0x7f3728009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83791c7b7a97c09a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739302700 1 -- 172.21.15.32:0/2682882023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728011830 con 0x7f37340a1340 2023-09-30T22:26:37.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739302700 1 -- 172.21.15.32:0/2682882023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3728011990 con 0x7f37340a1340 2023-09-30T22:26:37.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.558+0000 7f3739302700 1 -- 172.21.15.32:0/2682882023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728011d20 con 0x7f37340a1340 2023-09-30T22:26:37.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 -- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 msgr2=0x7f37341338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 --2- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f37341338f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3728009840 tx=0x7f3728009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 -- 172.21.15.32:0/2682882023 shutdown_connections 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 --2- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37340a0a30 0x7f37340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 --2- 172.21.15.32:0/2682882023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f37341338f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 -- 172.21.15.32:0/2682882023 >> 172.21.15.32:0/2682882023 conn(0x7f37340a6550 msgr2=0x7f37340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 -- 172.21.15.32:0/2682882023 shutdown_connections 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.559+0000 7f373c568700 1 -- 172.21.15.32:0/2682882023 wait complete. 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.560+0000 7f373c568700 1 Processor -- start 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.560+0000 7f373c568700 1 -- start start 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373c568700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37340a0a30 0x7f373411e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373c568700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f373411ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373c568700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 0x7f373411f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373c568700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734122150 con 0x7f37340a1340 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373c568700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3734120420 con 0x7f37340a0a30 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373c568700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37341209d0 con 0x7f3734133e30 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f373411ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373ab05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 0x7f373411f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f373411ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32870/0 (socket says 172.21.15.32:32870) 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 -- 172.21.15.32:0/1812743181 learned_addr learned my addr 172.21.15.32:0/1812743181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:37.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373ab05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 0x7f373411f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49690/0 (socket says 172.21.15.32:49690) 2023-09-30T22:26:37.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373a304700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37340a0a30 0x7f373411e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 -- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 msgr2=0x7f373411f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 0x7f373411f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 -- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37340a0a30 msgr2=0x7f373411e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37340a0a30 0x7f373411e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 -- 172.21.15.32:0/1812743181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37280094a0 con 0x7f37340a1340 2023-09-30T22:26:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373ab05700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 0x7f373411f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f373a304700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37340a0a30 0x7f373411e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.561+0000 7f3739b03700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f373411ede0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f37280090b0 tx=0x7f3728004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.562+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37280036a0 con 0x7f37340a1340 2023-09-30T22:26:37.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.562+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372801f070 con 0x7f37340a1340 2023-09-30T22:26:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.562+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728003a40 con 0x7f37340a1340 2023-09-30T22:26:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.562+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373409d710 con 0x7f37340a1340 2023-09-30T22:26:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.562+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734122a30 con 0x7f37340a1340 2023-09-30T22:26:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.562+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3728003ba0 con 0x7f37340a1340 2023-09-30T22:26:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.563+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3728021070 con 0x7f37340a1340 2023-09-30T22:26:37.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.563+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3734122bc0 con 0x7f37340a1340 2023-09-30T22:26:37.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.568+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3734122bc0 con 0x7f37340a1340 2023-09-30T22:26:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 Processor -- start 2023-09-30T22:26:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 -- start start 2023-09-30T22:26:37.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74ec0a0a30 0x7f74ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec133fa0 0x7f74ec134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec09edd0 con 0x7f74ec0a0a30 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec09e1f0 con 0x7f74ec0a1340 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f312b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec09dc50 con 0x7f74ec133fa0 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f0ec7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74ec0a0a30 0x7f74ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74ebfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f0ec7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74ec0a0a30 0x7f74ec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32886/0 (socket says 172.21.15.32:32886) 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f0ec7700 1 -- 172.21.15.32:0/4260564080 learned_addr learned my addr 172.21.15.32:0/4260564080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:37.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.595+0000 7f74f16c8700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec133fa0 0x7f74ec134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74f0ec7700 1 -- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec133fa0 msgr2=0x7f74ec134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74f0ec7700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec133fa0 0x7f74ec134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74f0ec7700 1 -- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 msgr2=0x7f74ec133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74f0ec7700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74f0ec7700 1 -- 172.21.15.32:0/4260564080 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec0a1fe0 con 0x7f74ec0a0a30 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74ebfff700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74f0ec7700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec0a0a30 0x7f74ec0a0e00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0098f0 tx=0x7f74dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713baa3d435e6aac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74eb7fe700 1 -- 172.21.15.32:0/4260564080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc011830 con 0x7f74ec0a0a30 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74eb7fe700 1 -- 172.21.15.32:0/4260564080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc011990 con 0x7f74ec0a0a30 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.596+0000 7f74eb7fe700 1 -- 172.21.15.32:0/4260564080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc011d20 con 0x7f74ec0a0a30 2023-09-30T22:26:37.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 -- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec0a0a30 msgr2=0x7f74ec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec0a0a30 0x7f74ec0a0e00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0098f0 tx=0x7f74dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 -- 172.21.15.32:0/4260564080 shutdown_connections 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec133fa0 0x7f74ec134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec0a0a30 0x7f74ec0a0e00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 --2- 172.21.15.32:0/4260564080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 -- 172.21.15.32:0/4260564080 >> 172.21.15.32:0/4260564080 conn(0x7f74ec0a6550 msgr2=0x7f74ec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 -- 172.21.15.32:0/4260564080 shutdown_connections 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.597+0000 7f74f312b700 1 -- 172.21.15.32:0/4260564080 wait complete. 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.598+0000 7f74f312b700 1 Processor -- start 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.598+0000 7f74f312b700 1 -- start start 2023-09-30T22:26:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.598+0000 7f74f312b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec0a0a30 0x7f74ec120380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.598+0000 7f74f312b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec1208c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.598+0000 7f74f312b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 0x7f74ec120e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f312b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec133770 con 0x7f74ec133fa0 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f312b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1231e0 con 0x7f74ec0a1340 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f312b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec122570 con 0x7f74ec0a0a30 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 0x7f74ec120e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f0ec7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec0a0a30 0x7f74ec120380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74ebfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec1208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f0ec7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec0a0a30 0x7f74ec120380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49708/0 (socket says 172.21.15.32:49708) 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 0x7f74ec120e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32894/0 (socket says 172.21.15.32:32894) 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f0ec7700 1 -- 172.21.15.32:0/2339311446 learned_addr learned my addr 172.21.15.32:0/2339311446 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 -- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec0a0a30 msgr2=0x7f74ec120380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec0a0a30 0x7f74ec120380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 -- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 msgr2=0x7f74ec1208c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec1208c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 -- 172.21.15.32:0/2339311446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00094a0 con 0x7f74ec133fa0 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f16c8700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 0x7f74ec120e00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009d90 tx=0x7f74e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4009c00 con 0x7f74ec133fa0 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e400f040 con 0x7f74ec133fa0 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc009550 con 0x7f74ec133fa0 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.599+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4014480 con 0x7f74ec133fa0 2023-09-30T22:26:37.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.600+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec0a3410 con 0x7f74ec133fa0 2023-09-30T22:26:37.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.600+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74e40145e0 con 0x7f74ec133fa0 2023-09-30T22:26:37.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.600+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f74e4040190 con 0x7f74ec133fa0 2023-09-30T22:26:37.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.600+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec0a0220 con 0x7f74ec133fa0 2023-09-30T22:26:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.605+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74e4007900 con 0x7f74ec133fa0 2023-09-30T22:26:37.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.774+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3734122bc0 con 0x7f37340a1340 2023-09-30T22:26:37.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.774+0000 7f3732ffd700 1 -- 172.21.15.32:0/1812743181 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3734122bc0 con 0x7f37340a1340 2023-09-30T22:26:37.775 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 msgr2=0x7f373411ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f373411ede0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f37280090b0 tx=0x7f3728004750 comp rx=0 tx=0).stop 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 shutdown_connections 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3734133e30 0x7f373411f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37340a0a30 0x7f373411e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 --2- 172.21.15.32:0/1812743181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37340a1340 0x7f373411ede0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 >> 172.21.15.32:0/1812743181 conn(0x7f37340a6550 msgr2=0x7f37340a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 shutdown_connections 2023-09-30T22:26:37.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.777+0000 7f373c568700 1 -- 172.21.15.32:0/1812743181 wait complete. 2023-09-30T22:26:37.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:37.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.811+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f74ec074ea0 con 0x7f74ec133fa0 2023-09-30T22:26:37.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.812+0000 7f74e97fa700 1 -- 172.21.15.32:0/2339311446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74e4007330 con 0x7f74ec133fa0 2023-09-30T22:26:37.813 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:37.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 msgr2=0x7f74ec120e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 0x7f74ec120e00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009d90 tx=0x7f74e400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 shutdown_connections 2023-09-30T22:26:37.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74ec133fa0 0x7f74ec120e00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f74ec0a0a30 0x7f74ec120380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 --2- 172.21.15.32:0/2339311446 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f74ec0a1340 0x7f74ec1208c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:37.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 >> 172.21.15.32:0/2339311446 conn(0x7f74ec0a6550 msgr2=0x7f74ec1baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 shutdown_connections 2023-09-30T22:26:37.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:37.814+0000 7f74f312b700 1 -- 172.21.15.32:0/2339311446 wait complete. 2023-09-30T22:26:37.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:37.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:37.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:38.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 Processor -- start 2023-09-30T22:26:38.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 -- start start 2023-09-30T22:26:38.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18f80a0a30 0x7f18f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 0x7f18f80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80a2a70 con 0x7f18f80a0a30 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80a01d0 con 0x7f18f809d630 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.015+0000 7f18fd4bd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18f809e2b0 con 0x7f18f8133930 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 0x7f18f80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 0x7f18f80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45304/0 (socket says 172.21.15.32:45304) 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 -- 172.21.15.32:0/1250936452 learned_addr learned my addr 172.21.15.32:0/1250936452 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f6ffd700 1 --2- 172.21.15.32:0/1250936452 >> v2:172.21.15.32:3300/0 conn(0x7f18f80a0a30 0x7f18f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f77fe700 1 --2- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 -- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 msgr2=0x7f18f8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:38.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 --2- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f8133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 -- 172.21.15.32:0/1250936452 >> v2:172.21.15.32:3300/0 conn(0x7f18f80a0a30 msgr2=0x7f18f80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 --2- 172.21.15.32:0/1250936452 >> v2:172.21.15.32:3300/0 conn(0x7f18f80a0a30 0x7f18f80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 -- 172.21.15.32:0/1250936452 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f809dc50 con 0x7f18f809d630 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f6ffd700 1 --2- 172.21.15.32:0/1250936452 >> v2:172.21.15.32:3300/0 conn(0x7f18f80a0a30 0x7f18f80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f67fc700 1 --2- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 0x7f18f80a1340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098f0 tx=0x7f18ec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2248f96e40bc0a43 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f5ffb700 1 -- 172.21.15.32:0/1250936452 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011830 con 0x7f18f809d630 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.016+0000 7f18f5ffb700 1 -- 172.21.15.32:0/1250936452 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec011990 con 0x7f18f809d630 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18f5ffb700 1 -- 172.21.15.32:0/1250936452 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec011d20 con 0x7f18f809d630 2023-09-30T22:26:38.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 -- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 msgr2=0x7f18f80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 0x7f18f80a1340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098f0 tx=0x7f18ec009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 -- 172.21.15.32:0/1250936452 shutdown_connections 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/1250936452 >> v2:172.21.15.32:3300/0 conn(0x7f18f80a0a30 0x7f18f80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/1250936452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f809d630 0x7f18f80a1340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 -- 172.21.15.32:0/1250936452 >> 172.21.15.32:0/1250936452 conn(0x7f18f80a3050 msgr2=0x7f18f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 -- 172.21.15.32:0/1250936452 shutdown_connections 2023-09-30T22:26:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.017+0000 7f18fd4bd700 1 -- 172.21.15.32:0/1250936452 wait complete. 2023-09-30T22:26:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 Processor -- start 2023-09-30T22:26:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 -- start start 2023-09-30T22:26:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f809d630 0x7f18f811f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 0x7f18f811fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8121fa0 con 0x7f18f809d630 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8122640 con 0x7f18f80a0a30 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18fd4bd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18f811ef70 con 0x7f18f8133930 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 0x7f18f811fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18f77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 0x7f18f811fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45316/0 (socket says 172.21.15.32:45316) 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18f67fc700 1 -- 172.21.15.32:0/77529604 learned_addr learned my addr 172.21.15.32:0/77529604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.019+0000 7f18f6ffd700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f809d630 0x7f18f811f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f77fe700 1 -- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 msgr2=0x7f18f811fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f77fe700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 0x7f18f811fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f77fe700 1 -- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f809d630 msgr2=0x7f18f811f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f77fe700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f809d630 0x7f18f811f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f77fe700 1 -- 172.21.15.32:0/77529604 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8009a70 con 0x7f18f8133930 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f67fc700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 0x7f18f811fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18f77fe700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f81c8090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f18f000e850 tx=0x7f18f000eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0013680 con 0x7f18f8133930 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f00137e0 con 0x7f18f8133930 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec009550 con 0x7f18f8133930 2023-09-30T22:26:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0013b90 con 0x7f18f8133930 2023-09-30T22:26:38.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.020+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f80a0410 con 0x7f18f8133930 2023-09-30T22:26:38.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.021+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18f0013cf0 con 0x7f18f8133930 2023-09-30T22:26:38.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.021+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f18f003f900 con 0x7f18f8133930 2023-09-30T22:26:38.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.021+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f80747f0 con 0x7f18f8133930 2023-09-30T22:26:38.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.026+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18f0021020 con 0x7f18f8133930 2023-09-30T22:26:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.057+0000 7f1a34ae3700 1 Processor -- start 2023-09-30T22:26:38.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.057+0000 7f1a34ae3700 1 -- start start 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a34ae3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a3009f410 0x7f1a30136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a34ae3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 0x7f1a301393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a34ae3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a30070940 con 0x7f1a3009e930 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a34ae3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3006fd60 con 0x7f1a3009f410 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a34ae3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3006fa80 con 0x7f1a30136fe0 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a2ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 0x7f1a301393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a2dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a3009f410 0x7f1a30136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a2ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 0x7f1a301393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49734/0 (socket says 172.21.15.32:49734) 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a2e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1a3009e930 0x7f1a3009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43438/0 (socket says 172.21.15.32:43438) 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a2dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a3009f410 0x7f1a30136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32936/0 (socket says 172.21.15.32:32936) 2023-09-30T22:26:38.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.058+0000 7f1a2e59c700 1 -- 172.21.15.32:0/2870334714 learned_addr learned my addr 172.21.15.32:0/2870334714 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 msgr2=0x7f1a301393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2dd9b700 1 --2- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 0x7f1a301393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/2870334714 >> v1:172.21.15.32:6789/0 conn(0x7f1a3009e930 legacy=0x7f1a3009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/2870334714 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3009fb60 con 0x7f1a3009f410 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2d59a700 1 -- 172.21.15.32:0/2870334714 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848704203 0 0) 0x7f1a30070940 con 0x7f1a3009e930 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2ed9d700 1 --2- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 0x7f1a301393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2dd9b700 1 --2- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009f410 0x7f1a30136aa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009840 tx=0x7f1a24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ec7747cd90630 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2d59a700 1 -- 172.21.15.32:0/2870334714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24011830 con 0x7f1a3009f410 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2d59a700 1 -- 172.21.15.32:0/2870334714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24011990 con 0x7f1a3009f410 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.059+0000 7f1a2d59a700 1 -- 172.21.15.32:0/2870334714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24011d20 con 0x7f1a3009f410 2023-09-30T22:26:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 -- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009f410 msgr2=0x7f1a30136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009f410 0x7f1a30136aa0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009840 tx=0x7f1a24009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 -- 172.21.15.32:0/2870334714 shutdown_connections 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a30136fe0 0x7f1a301393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/2870334714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009f410 0x7f1a30136aa0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 -- 172.21.15.32:0/2870334714 >> 172.21.15.32:0/2870334714 conn(0x7f1a300a4210 msgr2=0x7f1a300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 -- 172.21.15.32:0/2870334714 shutdown_connections 2023-09-30T22:26:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.060+0000 7f1a34ae3700 1 -- 172.21.15.32:0/2870334714 wait complete. 2023-09-30T22:26:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.061+0000 7f1a34ae3700 1 Processor -- start 2023-09-30T22:26:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 -- start start 2023-09-30T22:26:38.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009e930 0x7f1a301c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 0x7f1a301c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a30136fe0 0x7f1a3006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30132270 con 0x7f1a3009e930 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3009de80 con 0x7f1a30136fe0 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a34ae3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a301bc160 con 0x7f1a3009f410 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a2dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 0x7f1a301c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a2ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a30136fe0 0x7f1a3006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a2dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 0x7f1a301c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49748/0 (socket says 172.21.15.32:49748) 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/242112864 learned_addr learned my addr 172.21.15.32:0/242112864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.062+0000 7f1a2e59c700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009e930 0x7f1a301c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a30136fe0 msgr2=0x7f1a3006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2dd9b700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a30136fe0 0x7f1a3006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009e930 msgr2=0x7f1a301c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2dd9b700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009e930 0x7f1a301c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2dd9b700 1 -- 172.21.15.32:0/242112864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280094a0 con 0x7f1a3009f410 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2ed9d700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a30136fe0 0x7f1a3006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a2dd9b700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 0x7f1a301c8a10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009fd0 tx=0x7f1a24004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240045d0 con 0x7f1a3009f410 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2401f070 con 0x7f1a3009f410 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240094a0 con 0x7f1a3009f410 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240038a0 con 0x7f1a3009f410 2023-09-30T22:26:38.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.063+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a301c8f50 con 0x7f1a3009f410 2023-09-30T22:26:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.064+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a24003a00 con 0x7f1a3009f410 2023-09-30T22:26:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.064+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1a24021070 con 0x7f1a3009f410 2023-09-30T22:26:38.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.064+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a30074ea0 con 0x7f1a3009f410 2023-09-30T22:26:38.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.069+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a24015080 con 0x7f1a3009f410 2023-09-30T22:26:38.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.239+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18f8072980 con 0x7f18f8133930 2023-09-30T22:26:38.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.239+0000 7f18df7fe700 1 -- 172.21.15.32:0/77529604 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18f0017070 con 0x7f18f8133930 2023-09-30T22:26:38.240 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 msgr2=0x7f18f81c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f81c8090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f18f000e850 tx=0x7f18f000eb60 comp rx=0 tx=0).stop 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 shutdown_connections 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18f8133930 0x7f18f81c8090 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18f809d630 0x7f18f811f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 --2- 172.21.15.32:0/77529604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18f80a0a30 0x7f18f811fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 >> 172.21.15.32:0/77529604 conn(0x7f18f80a3050 msgr2=0x7f18f81bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 shutdown_connections 2023-09-30T22:26:38.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.242+0000 7f18fd4bd700 1 -- 172.21.15.32:0/77529604 wait complete. 2023-09-30T22:26:38.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.281+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1a30121a10 con 0x7f1a3009f410 2023-09-30T22:26:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.282+0000 7f1a1effd700 1 -- 172.21.15.32:0/242112864 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a2402b020 con 0x7f1a3009f410 2023-09-30T22:26:38.284 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 msgr2=0x7f1a301c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 0x7f1a301c8a10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009fd0 tx=0x7f1a24004750 comp rx=0 tx=0).stop 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 shutdown_connections 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a30136fe0 0x7f1a3006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a3009e930 0x7f1a301c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 --2- 172.21.15.32:0/242112864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a3009f410 0x7f1a301c8a10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 >> 172.21.15.32:0/242112864 conn(0x7f1a300a4210 msgr2=0x7f1a30134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 shutdown_connections 2023-09-30T22:26:38.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.284+0000 7f1a34ae3700 1 -- 172.21.15.32:0/242112864 wait complete. 2023-09-30T22:26:38.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:38.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:38.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:38.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.940+0000 7ffab665c700 1 Processor -- start 2023-09-30T22:26:38.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.940+0000 7ffab665c700 1 -- start start 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab665c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a0a30 0x7ffab00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab665c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab0134170 0x7ffab013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab665c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab009edd0 con 0x7ffab00a1340 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab665c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab009e1f0 con 0x7ffab0134170 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab665c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffab009dc50 con 0x7ffab00a0a30 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a0a30 0x7ffab00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffab00a1340 0x7ffab0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43444/0 (socket says 172.21.15.32:43444) 2023-09-30T22:26:38.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab4bf9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab0134170 0x7ffab013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaf7fe700 1 -- 172.21.15.32:0/2875517992 learned_addr learned my addr 172.21.15.32:0/2875517992 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:38.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaeffd700 1 -- 172.21.15.32:0/2875517992 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793625348 0 0) 0x7ffab009edd0 con 0x7ffab00a1340 2023-09-30T22:26:38.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaffff700 1 -- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab0134170 msgr2=0x7ffab013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:38.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaffff700 1 --2- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab0134170 0x7ffab013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaffff700 1 -- 172.21.15.32:0/2875517992 >> v1:172.21.15.32:6789/0 conn(0x7ffab00a1340 legacy=0x7ffab0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffaaffff700 1 -- 172.21.15.32:0/2875517992 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00a1fe0 con 0x7ffab00a0a30 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.941+0000 7ffab4bf9700 1 --2- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab0134170 0x7ffab013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.942+0000 7ffaaffff700 1 --2- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a0a30 0x7ffab00a0e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009870 tx=0x7ffaa0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b556cbffd5a09f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.942+0000 7ffaaeffd700 1 -- 172.21.15.32:0/2875517992 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0011830 con 0x7ffab00a0a30 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.942+0000 7ffaaeffd700 1 -- 172.21.15.32:0/2875517992 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa0011990 con 0x7ffab00a0a30 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.942+0000 7ffaaeffd700 1 -- 172.21.15.32:0/2875517992 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0011d20 con 0x7ffab00a0a30 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.942+0000 7ffab665c700 1 -- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a0a30 msgr2=0x7ffab00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.942+0000 7ffab665c700 1 --2- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a0a30 0x7ffab00a0e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009870 tx=0x7ffaa0009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.943+0000 7ffab665c700 1 -- 172.21.15.32:0/2875517992 shutdown_connections 2023-09-30T22:26:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.943+0000 7ffab665c700 1 --2- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab0134170 0x7ffab013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.943+0000 7ffab665c700 1 --2- 172.21.15.32:0/2875517992 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a0a30 0x7ffab00a0e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.943+0000 7ffab665c700 1 -- 172.21.15.32:0/2875517992 >> 172.21.15.32:0/2875517992 conn(0x7ffab00a6550 msgr2=0x7ffab00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.943+0000 7ffab665c700 1 -- 172.21.15.32:0/2875517992 shutdown_connections 2023-09-30T22:26:38.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.943+0000 7ffab665c700 1 -- 172.21.15.32:0/2875517992 wait complete. 2023-09-30T22:26:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.944+0000 7ffab665c700 1 Processor -- start 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.944+0000 7ffab665c700 1 -- start start 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab665c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 0x7ffab01227b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab665c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a1340 0x7ffab0122cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab665c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffab0134170 0x7ffab0123230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab665c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0125f60 con 0x7ffab0134170 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab665c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0124440 con 0x7ffab00a0a30 2023-09-30T22:26:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab665c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00a4d60 con 0x7ffab00a1340 2023-09-30T22:26:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 0x7ffab01227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab4bf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffab0134170 0x7ffab0123230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 0x7ffab01227b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45340/0 (socket says 172.21.15.32:45340) 2023-09-30T22:26:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 -- 172.21.15.32:0/2296136966 learned_addr learned my addr 172.21.15.32:0/2296136966 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaf7fe700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a1340 0x7ffab0122cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 -- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a1340 msgr2=0x7ffab0122cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a1340 0x7ffab0122cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 -- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffab0134170 msgr2=0x7ffab0123230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffab0134170 0x7ffab0123230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffaaffff700 1 -- 172.21.15.32:0/2296136966 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8009a70 con 0x7ffab00a0a30 2023-09-30T22:26:38.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.945+0000 7ffab4bf9700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffab0134170 0x7ffab0123230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.946+0000 7ffaaffff700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 0x7ffab01227b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009840 tx=0x7ffaa0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.946+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00045d0 con 0x7ffab00a0a30 2023-09-30T22:26:38.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.946+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa001f070 con 0x7ffab00a0a30 2023-09-30T22:26:38.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.946+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00038e0 con 0x7ffab00a0a30 2023-09-30T22:26:38.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.946+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa00094a0 con 0x7ffab00a0a30 2023-09-30T22:26:38.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.946+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab00a25e0 con 0x7ffab00a0a30 2023-09-30T22:26:38.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.947+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffaa0003a40 con 0x7ffab00a0a30 2023-09-30T22:26:38.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.947+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffaa0021070 con 0x7ffab00a0a30 2023-09-30T22:26:38.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.947+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab01332a0 con 0x7ffab00a0a30 2023-09-30T22:26:38.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.952+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffaa002b080 con 0x7ffab00a0a30 2023-09-30T22:26:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.987+0000 7f407996b700 1 Processor -- start 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.987+0000 7f407996b700 1 -- start start 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f407996b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f407409d5c0 0x7f4074138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f407996b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a14a0 0x7f407413b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f407996b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407409ea10 con 0x7f40740a09c0 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f407996b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407409e210 con 0x7f40740a14a0 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f407996b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40740a26f0 con 0x7f407409d5c0 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40727fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f407409d5c0 0x7f4074138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f4072ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40740a09c0 0x7f40740a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43452/0 (socket says 172.21.15.32:43452) 2023-09-30T22:26:38.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40737fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a14a0 0x7f407413b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f4072ffd700 1 -- 172.21.15.32:0/381018708 learned_addr learned my addr 172.21.15.32:0/381018708 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40727fc700 1 -- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a14a0 msgr2=0x7f407413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40727fc700 1 --2- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a14a0 0x7f407413b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40727fc700 1 -- 172.21.15.32:0/381018708 >> v1:172.21.15.32:6789/0 conn(0x7f40740a09c0 legacy=0x7f40740a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40727fc700 1 -- 172.21.15.32:0/381018708 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407409db70 con 0x7f407409d5c0 2023-09-30T22:26:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f4071ffb700 1 -- 172.21.15.32:0/381018708 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526472592 0 0) 0x7f407409ea10 con 0x7f40740a09c0 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40737fe700 1 --2- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a14a0 0x7f407413b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.988+0000 7f40727fc700 1 --2- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f407409d5c0 0x7f4074138d70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4068009870 tx=0x7f4068009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ee0168c387859c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.989+0000 7f4071ffb700 1 -- 172.21.15.32:0/381018708 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068011830 con 0x7f407409d5c0 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.989+0000 7f4071ffb700 1 -- 172.21.15.32:0/381018708 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068011990 con 0x7f407409d5c0 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.989+0000 7f4071ffb700 1 -- 172.21.15.32:0/381018708 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068011d20 con 0x7f407409d5c0 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.989+0000 7f407996b700 1 -- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f407409d5c0 msgr2=0x7f4074138d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.989+0000 7f407996b700 1 --2- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f407409d5c0 0x7f4074138d70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4068009870 tx=0x7f4068009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.990+0000 7f407996b700 1 -- 172.21.15.32:0/381018708 shutdown_connections 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.990+0000 7f407996b700 1 --2- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a14a0 0x7f407413b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.990+0000 7f407996b700 1 --2- 172.21.15.32:0/381018708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f407409d5c0 0x7f4074138d70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.990+0000 7f407996b700 1 -- 172.21.15.32:0/381018708 >> 172.21.15.32:0/381018708 conn(0x7f40740a2fe0 msgr2=0x7f40740a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.990+0000 7f407996b700 1 -- 172.21.15.32:0/381018708 shutdown_connections 2023-09-30T22:26:38.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.990+0000 7f407996b700 1 -- 172.21.15.32:0/381018708 wait complete. 2023-09-30T22:26:38.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.991+0000 7f407996b700 1 Processor -- start 2023-09-30T22:26:38.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.991+0000 7f407996b700 1 -- start start 2023-09-30T22:26:38.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f407996b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 0x7f407411c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f407996b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a09c0 0x7f407411ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f407996b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40740a14a0 0x7f407411d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:38.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f407996b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4074120550 con 0x7f407409d5c0 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f407996b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407411fbe0 con 0x7f40740a09c0 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f407996b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f407411ead0 con 0x7f40740a14a0 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 0x7f407411c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 0x7f407411c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32956/0 (socket says 172.21.15.32:32956) 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 -- 172.21.15.32:0/3908908550 learned_addr learned my addr 172.21.15.32:0/3908908550 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f40727fc700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a09c0 0x7f407411ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f40737fe700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40740a14a0 0x7f407411d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 -- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40740a14a0 msgr2=0x7f407411d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40740a14a0 0x7f407411d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 -- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a09c0 msgr2=0x7f407411ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a09c0 0x7f407411ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.992+0000 7f4072ffd700 1 -- 172.21.15.32:0/3908908550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0094a0 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f40737fe700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40740a14a0 0x7f407411d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f40727fc700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a09c0 0x7f407411ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f4072ffd700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 0x7f407411c7b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f406400cc80 tx=0x7f406400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406400abb0 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406400ad10 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40680094a0 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064007310 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.993+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407411ddc0 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.994+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4064007470 con 0x7f407409d5c0 2023-09-30T22:26:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.994+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40740747f0 con 0x7f407409d5c0 2023-09-30T22:26:38.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.994+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f406403f590 con 0x7f407409d5c0 2023-09-30T22:26:39.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:38.999+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4064018020 con 0x7f407409d5c0 2023-09-30T22:26:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.178+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffab0074ea0 con 0x7ffab00a0a30 2023-09-30T22:26:39.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.178+0000 7ffaacff9700 1 -- 172.21.15.32:0/2296136966 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffaa0015020 con 0x7ffab00a0a30 2023-09-30T22:26:39.179 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 msgr2=0x7ffab01227b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 0x7ffab01227b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009840 tx=0x7ffaa0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 shutdown_connections 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffab0134170 0x7ffab0123230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffab00a0a30 0x7ffab01227b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 --2- 172.21.15.32:0/2296136966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffab00a1340 0x7ffab0122cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.180+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 >> 172.21.15.32:0/2296136966 conn(0x7ffab00a6550 msgr2=0x7ffab009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.181+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 shutdown_connections 2023-09-30T22:26:39.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.181+0000 7ffab665c700 1 -- 172.21.15.32:0/2296136966 wait complete. 2023-09-30T22:26:39.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:39.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.213+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f407409cec0 con 0x7f407409d5c0 2023-09-30T22:26:39.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.214+0000 7f405bfff700 1 -- 172.21.15.32:0/3908908550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f406401c3b0 con 0x7f407409d5c0 2023-09-30T22:26:39.218 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:39.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 msgr2=0x7f407411c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 0x7f407411c7b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f406400cc80 tx=0x7f406400cf90 comp rx=0 tx=0).stop 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 shutdown_connections 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40740a14a0 0x7f407411d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f407409d5c0 0x7f407411c7b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 --2- 172.21.15.32:0/3908908550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40740a09c0 0x7f407411ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 >> 172.21.15.32:0/3908908550 conn(0x7f40740a2fe0 msgr2=0x7f40740a4000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 shutdown_connections 2023-09-30T22:26:39.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.216+0000 7f407996b700 1 -- 172.21.15.32:0/3908908550 wait complete. 2023-09-30T22:26:39.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:39.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:39.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:39.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 Processor -- start 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 -- start start 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f81680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f816813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816809edd0 con 0x7f81680a1340 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816809e1f0 con 0x7f81680a0a30 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816ea52700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f816809dc50 con 0x7f8168134170 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f81677fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81680a1340 0x7f8168133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43458/0 (socket says 172.21.15.32:43458) 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f81677fe700 1 -- 172.21.15.32:0/1615598637 learned_addr learned my addr 172.21.15.32:0/1615598637 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.413+0000 7f816cfef700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f816813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8167fff700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f81680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8166ffd700 1 -- 172.21.15.32:0/1615598637 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860993392 0 0) 0x7f816809edd0 con 0x7f81680a1340 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8166ffd700 1 -- 172.21.15.32:0/1615598637 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81500035b0 con 0x7f81680a1340 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f816cfef700 1 -- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 msgr2=0x7f81680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f816cfef700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f81680a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f816cfef700 1 -- 172.21.15.32:0/1615598637 >> v1:172.21.15.32:6789/0 conn(0x7f81680a1340 legacy=0x7f8168133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f816cfef700 1 -- 172.21.15.32:0/1615598637 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680a1fe0 con 0x7f8168134170 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8167fff700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f81680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f816cfef700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f816813bd30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8160009dc0 tx=0x7f816000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c246a2e24de9697 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8166ffd700 1 -- 172.21.15.32:0/1615598637 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000e070 con 0x7f8168134170 2023-09-30T22:26:39.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8166ffd700 1 -- 172.21.15.32:0/1615598637 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816000bb20 con 0x7f8168134170 2023-09-30T22:26:39.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.414+0000 7f8166ffd700 1 -- 172.21.15.32:0/1615598637 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000be90 con 0x7f8168134170 2023-09-30T22:26:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 -- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 msgr2=0x7f816813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f816813bd30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8160009dc0 tx=0x7f816000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 -- 172.21.15.32:0/1615598637 shutdown_connections 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f816813bd30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 --2- 172.21.15.32:0/1615598637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f81680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 -- 172.21.15.32:0/1615598637 >> 172.21.15.32:0/1615598637 conn(0x7f81680a6550 msgr2=0x7f81680a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 -- 172.21.15.32:0/1615598637 shutdown_connections 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.415+0000 7f816ea52700 1 -- 172.21.15.32:0/1615598637 wait complete. 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.416+0000 7f816ea52700 1 Processor -- start 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 -- start start 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f8168122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 0x7f8168122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f8168123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81681262a0 con 0x7f81680a1340 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8168124c20 con 0x7f81680a0a30 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816ea52700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8168124390 con 0x7f8168134170 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f81677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 0x7f8168122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f8167fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f8168122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816cfef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f8168123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f8167fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f8168122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45370/0 (socket says 172.21.15.32:45370) 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f81677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 0x7f8168122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32962/0 (socket says 172.21.15.32:32962) 2023-09-30T22:26:39.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f8167fff700 1 -- 172.21.15.32:0/2735214768 learned_addr learned my addr 172.21.15.32:0/2735214768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.417+0000 7f816cfef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f8168123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49786/0 (socket says 172.21.15.32:49786) 2023-09-30T22:26:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f81677fe700 1 -- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 msgr2=0x7f8168123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f81677fe700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f8168123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f81677fe700 1 -- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 msgr2=0x7f8168122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f81677fe700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f8168122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f81677fe700 1 -- 172.21.15.32:0/2735214768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580094a0 con 0x7f81680a1340 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f816cfef700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f8168123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f8167fff700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f8168122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f81677fe700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 0x7f8168122cc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f815c00c490 tx=0x7f815c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c005d70 con 0x7f81680a1340 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c0070d0 con 0x7f81680a1340 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8160009a70 con 0x7f81680a1340 2023-09-30T22:26:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c007440 con 0x7f81680a1340 2023-09-30T22:26:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.418+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81680a25e0 con 0x7f81680a1340 2023-09-30T22:26:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.419+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f815c0075a0 con 0x7f81680a1340 2023-09-30T22:26:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.419+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8168074ea0 con 0x7f81680a1340 2023-09-30T22:26:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.419+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f815c03ea70 con 0x7f81680a1340 2023-09-30T22:26:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.424+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f815c019c90 con 0x7f81680a1340 2023-09-30T22:26:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 Processor -- start 2023-09-30T22:26:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 -- start start 2023-09-30T22:26:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d58133fa0 0x7f0d58134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5809edd0 con 0x7f0d58133fa0 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5809e1f0 con 0x7f0d580a0a30 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.454+0000 7f0d6079c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5809dc50 con 0x7f0d580a1340 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5dd37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5e538700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d58133fa0 0x7f0d58134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d58133fa0 0x7f0d58134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32970/0 (socket says 172.21.15.32:32970) 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5dd37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49800/0 (socket says 172.21.15.32:49800) 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5e538700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45372/0 (socket says 172.21.15.32:45372) 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5dd37700 1 -- 172.21.15.32:0/3347783190 learned_addr learned my addr 172.21.15.32:0/3347783190 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 -- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 msgr2=0x7f0d58133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 -- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 msgr2=0x7f0d580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 -- 172.21.15.32:0/3347783190 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d580a1fe0 con 0x7f0d58133fa0 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5dd37700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5ed39700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d58134410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d90 tx=0x7f0d5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2a667de2b4c8f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5d536700 1 -- 172.21.15.32:0/3347783190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000e070 con 0x7f0d58133fa0 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5d536700 1 -- 172.21.15.32:0/3347783190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000bb20 con 0x7f0d58133fa0 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.455+0000 7f0d5d536700 1 -- 172.21.15.32:0/3347783190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50012430 con 0x7f0d58133fa0 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 -- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 msgr2=0x7f0d58134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d58134410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d90 tx=0x7f0d5000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 -- 172.21.15.32:0/3347783190 shutdown_connections 2023-09-30T22:26:39.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d58134410 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 --2- 172.21.15.32:0/3347783190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 -- 172.21.15.32:0/3347783190 >> 172.21.15.32:0/3347783190 conn(0x7f0d580a6550 msgr2=0x7f0d580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 -- 172.21.15.32:0/3347783190 shutdown_connections 2023-09-30T22:26:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.456+0000 7f0d6079c700 1 -- 172.21.15.32:0/3347783190 wait complete. 2023-09-30T22:26:39.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.457+0000 7f0d6079c700 1 Processor -- start 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.457+0000 7f0d6079c700 1 -- start start 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d6079c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d58120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d6079c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d6079c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d581211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d6079c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d581228c0 con 0x7f0d58133fa0 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d6079c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58123980 con 0x7f0d580a0a30 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d6079c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d580a0220 con 0x7f0d580a1340 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d58120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5dd37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d58120760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45378/0 (socket says 172.21.15.32:45378) 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 -- 172.21.15.32:0/1877380817 learned_addr learned my addr 172.21.15.32:0/1877380817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5ed39700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d581211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 -- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 msgr2=0x7f0d58120ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58120ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 -- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 msgr2=0x7f0d581211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d581211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5e538700 1 -- 172.21.15.32:0/1877380817 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48009550 con 0x7f0d580a0a30 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5dd37700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.458+0000 7f0d5ed39700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d581211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.459+0000 7f0d5e538700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d58120760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e9b0 tx=0x7f0d5400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.459+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54009710 con 0x7f0d580a0a30 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.459+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5400ee50 con 0x7f0d580a0a30 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.459+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50009a70 con 0x7f0d580a0a30 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.459+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400ca50 con 0x7f0d580a0a30 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.459+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d581222e0 con 0x7f0d580a0a30 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.460+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d5400cbb0 con 0x7f0d580a0a30 2023-09-30T22:26:39.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.460+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0d5403f6d0 con 0x7f0d580a0a30 2023-09-30T22:26:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.460+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d58074ea0 con 0x7f0d580a0a30 2023-09-30T22:26:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.465+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d540232f0 con 0x7f0d580a0a30 2023-09-30T22:26:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.630+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f81681336d0 con 0x7f81680a1340 2023-09-30T22:26:39.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.630+0000 7f8164ff9700 1 -- 172.21.15.32:0/2735214768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f815c016170 con 0x7f81680a1340 2023-09-30T22:26:39.632 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:39.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.632+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 msgr2=0x7f8168122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.632+0000 7f816ea52700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 0x7f8168122cc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f815c00c490 tx=0x7f815c00c7a0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 shutdown_connections 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8168134170 0x7f8168123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81680a0a30 0x7f8168122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 --2- 172.21.15.32:0/2735214768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81680a1340 0x7f8168122cc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 >> 172.21.15.32:0/2735214768 conn(0x7f81680a6550 msgr2=0x7f816809c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 shutdown_connections 2023-09-30T22:26:39.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.633+0000 7f816ea52700 1 -- 172.21.15.32:0/2735214768 wait complete. 2023-09-30T22:26:39.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.673+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0d58123530 con 0x7f0d580a0a30 2023-09-30T22:26:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.673+0000 7f0d4effd700 1 -- 172.21.15.32:0/1877380817 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d54021020 con 0x7f0d580a0a30 2023-09-30T22:26:39.675 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 msgr2=0x7f0d58120760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d58120760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e9b0 tx=0x7f0d5400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 shutdown_connections 2023-09-30T22:26:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d58133fa0 0x7f0d581211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d580a0a30 0x7f0d58120760 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 --2- 172.21.15.32:0/1877380817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d580a1340 0x7f0d58120ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 >> 172.21.15.32:0/1877380817 conn(0x7f0d580a6550 msgr2=0x7f0d5809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 shutdown_connections 2023-09-30T22:26:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:39.675+0000 7f0d6079c700 1 -- 172.21.15.32:0/1877380817 wait complete. 2023-09-30T22:26:39.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:40.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:40.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.348+0000 7f57a4ade700 1 Processor -- start 2023-09-30T22:26:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.348+0000 7f57a4ade700 1 -- start start 2023-09-30T22:26:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f57a4ade700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 0x7f57a0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f57a4ade700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57a0139250 0x7f57a013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f57a4ade700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a009edd0 con 0x7f57a00a0a30 2023-09-30T22:26:40.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f57a4ade700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00a18e0 con 0x7f57a0139250 2023-09-30T22:26:40.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f57a4ade700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57a009e1f0 con 0x7f57a00a1510 2023-09-30T22:26:40.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 0x7f57a0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57a0139250 0x7f57a013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 0x7f57a0138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49814/0 (socket says 172.21.15.32:49814) 2023-09-30T22:26:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 -- 172.21.15.32:0/3316423765 learned_addr learned my addr 172.21.15.32:0/3316423765 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 -- 172.21.15.32:0/3316423765 >> v2:172.21.15.32:3300/0 conn(0x7f57a0139250 msgr2=0x7f57a013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 --2- 172.21.15.32:0/3316423765 >> v2:172.21.15.32:3300/0 conn(0x7f57a0139250 0x7f57a013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 -- 172.21.15.32:0/3316423765 >> v1:172.21.15.32:6789/0 conn(0x7f57a00a0a30 legacy=0x7f57a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579dd9b700 1 -- 172.21.15.32:0/3316423765 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a009dc50 con 0x7f57a00a1510 2023-09-30T22:26:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579d59a700 1 -- 172.21.15.32:0/3316423765 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534882023 0 0) 0x7f57a009edd0 con 0x7f57a00a0a30 2023-09-30T22:26:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.349+0000 7f579ed9d700 1 --2- 172.21.15.32:0/3316423765 >> v2:172.21.15.32:3300/0 conn(0x7f57a0139250 0x7f57a013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f579dd9b700 1 --2- 172.21.15.32:0/3316423765 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 0x7f57a0138d10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5794009920 tx=0x7f5794009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f06771166ce19b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f579d59a700 1 -- 172.21.15.32:0/3316423765 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794011830 con 0x7f57a00a1510 2023-09-30T22:26:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f579d59a700 1 -- 172.21.15.32:0/3316423765 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794011990 con 0x7f57a00a1510 2023-09-30T22:26:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f579d59a700 1 -- 172.21.15.32:0/3316423765 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794011d20 con 0x7f57a00a1510 2023-09-30T22:26:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f57a4ade700 1 -- 172.21.15.32:0/3316423765 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 msgr2=0x7f57a0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f57a4ade700 1 --2- 172.21.15.32:0/3316423765 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 0x7f57a0138d10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5794009920 tx=0x7f5794009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f57a4ade700 1 -- 172.21.15.32:0/3316423765 shutdown_connections 2023-09-30T22:26:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f57a4ade700 1 --2- 172.21.15.32:0/3316423765 >> v2:172.21.15.32:3300/0 conn(0x7f57a0139250 0x7f57a013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f57a4ade700 1 --2- 172.21.15.32:0/3316423765 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a1510 0x7f57a0138d10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.350+0000 7f57a4ade700 1 -- 172.21.15.32:0/3316423765 >> 172.21.15.32:0/3316423765 conn(0x7f57a00a6550 msgr2=0x7f57a00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.351+0000 7f57a4ade700 1 -- 172.21.15.32:0/3316423765 shutdown_connections 2023-09-30T22:26:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.351+0000 7f57a4ade700 1 -- 172.21.15.32:0/3316423765 wait complete. 2023-09-30T22:26:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 Processor -- start 2023-09-30T22:26:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 -- start start 2023-09-30T22:26:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a0a30 0x7f57a0122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 0x7f57a0122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 0x7f57a0123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01389b0 con 0x7f57a0139250 2023-09-30T22:26:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01380b0 con 0x7f57a00a1510 2023-09-30T22:26:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f57a4ade700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0124390 con 0x7f57a00a0a30 2023-09-30T22:26:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f579dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 0x7f57a0122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f579ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 0x7f57a0123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f579dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 0x7f57a0122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45394/0 (socket says 172.21.15.32:45394) 2023-09-30T22:26:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f579ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 0x7f57a0123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32992/0 (socket says 172.21.15.32:32992) 2023-09-30T22:26:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f579dd9b700 1 -- 172.21.15.32:0/2622666373 learned_addr learned my addr 172.21.15.32:0/2622666373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.352+0000 7f579e59c700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a0a30 0x7f57a0122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579dd9b700 1 -- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a0a30 msgr2=0x7f57a0122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579dd9b700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a0a30 0x7f57a0122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579dd9b700 1 -- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 msgr2=0x7f57a0123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579dd9b700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 0x7f57a0123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579dd9b700 1 -- 172.21.15.32:0/2622666373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980094a0 con 0x7f57a00a1510 2023-09-30T22:26:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579ed9d700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 0x7f57a0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f579dd9b700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 0x7f57a0122cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57940098f0 tx=0x7f5794004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940045d0 con 0x7f57a00a1510 2023-09-30T22:26:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579401f070 con 0x7f57a00a1510 2023-09-30T22:26:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5794009550 con 0x7f57a00a1510 2023-09-30T22:26:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940038e0 con 0x7f57a00a1510 2023-09-30T22:26:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.353+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a0138670 con 0x7f57a00a1510 2023-09-30T22:26:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.354+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5794003a40 con 0x7f57a00a1510 2023-09-30T22:26:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.354+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5794021070 con 0x7f57a00a1510 2023-09-30T22:26:40.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.354+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a00a0280 con 0x7f57a00a1510 2023-09-30T22:26:40.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.359+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5794015080 con 0x7f57a00a1510 2023-09-30T22:26:40.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 Processor -- start 2023-09-30T22:26:40.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 -- start start 2023-09-30T22:26:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 0x7f89e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e009edd0 con 0x7f89e00a1340 2023-09-30T22:26:40.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e009e1f0 con 0x7f89e00a0a30 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.386+0000 7f89e85cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e009dc50 con 0x7f89e0134170 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6367700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 0x7f89e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6b68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6367700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 0x7f89e00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33002/0 (socket says 172.21.15.32:33002) 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e5b66700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89e00a1340 0x7f89e0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43476/0 (socket says 172.21.15.32:43476) 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6367700 1 -- 172.21.15.32:0/584151937 learned_addr learned my addr 172.21.15.32:0/584151937 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e5365700 1 -- 172.21.15.32:0/584151937 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171819671 0 0) 0x7f89e009edd0 con 0x7f89e00a1340 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e5365700 1 -- 172.21.15.32:0/584151937 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89c80035b0 con 0x7f89e00a1340 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6b68700 1 -- 172.21.15.32:0/584151937 >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 msgr2=0x7f89e00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6b68700 1 --2- 172.21.15.32:0/584151937 >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 0x7f89e00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6b68700 1 -- 172.21.15.32:0/584151937 >> v1:172.21.15.32:6789/0 conn(0x7f89e00a1340 legacy=0x7f89e0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6b68700 1 -- 172.21.15.32:0/584151937 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e00a1fe0 con 0x7f89e0134170 2023-09-30T22:26:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6367700 1 --2- 172.21.15.32:0/584151937 >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 0x7f89e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e6b68700 1 --2- 172.21.15.32:0/584151937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e013bd30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009840 tx=0x7f89d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f514d680852820 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e5365700 1 -- 172.21.15.32:0/584151937 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8011830 con 0x7f89e0134170 2023-09-30T22:26:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e5365700 1 -- 172.21.15.32:0/584151937 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8011990 con 0x7f89e0134170 2023-09-30T22:26:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.387+0000 7f89e5365700 1 -- 172.21.15.32:0/584151937 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8011d20 con 0x7f89e0134170 2023-09-30T22:26:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 -- 172.21.15.32:0/584151937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 msgr2=0x7f89e013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 --2- 172.21.15.32:0/584151937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e013bd30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009840 tx=0x7f89d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 -- 172.21.15.32:0/584151937 shutdown_connections 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 --2- 172.21.15.32:0/584151937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e013bd30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 --2- 172.21.15.32:0/584151937 >> v2:172.21.15.32:3300/0 conn(0x7f89e00a0a30 0x7f89e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 -- 172.21.15.32:0/584151937 >> 172.21.15.32:0/584151937 conn(0x7f89e00a6550 msgr2=0x7f89e00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 -- 172.21.15.32:0/584151937 shutdown_connections 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.388+0000 7f89e85cb700 1 -- 172.21.15.32:0/584151937 wait complete. 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.389+0000 7f89e85cb700 1 Processor -- start 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.389+0000 7f89e85cb700 1 -- start start 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e85cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 0x7f89e0121700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e85cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89e00a1340 0x7f89e0121c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e85cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e0122180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e85cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00a0220 con 0x7f89e00a0a30 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e85cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e01248f0 con 0x7f89e0134170 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e85cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0132900 con 0x7f89e00a1340 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 0x7f89e0121700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 0x7f89e0121700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33014/0 (socket says 172.21.15.32:33014) 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 -- 172.21.15.32:0/527671513 learned_addr learned my addr 172.21.15.32:0/527671513 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6b68700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e0122180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e5b66700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89e00a1340 0x7f89e0121c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 -- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89e00a1340 msgr2=0x7f89e0121c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89e00a1340 0x7f89e0121c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 -- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 msgr2=0x7f89e0122180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e0122180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 -- 172.21.15.32:0/527671513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0009b30 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.390+0000 7f89e6367700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 0x7f89e0121700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f89d000efc0 tx=0x7f89d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.391+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d000cb00 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.391+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d000f040 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.391+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d80094a0 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.391+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d000ce80 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.391+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e00a25e0 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.391+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89d0014480 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.392+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f89d003fc90 con 0x7f89e00a0a30 2023-09-30T22:26:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.392+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e0074ea0 con 0x7f89e00a0a30 2023-09-30T22:26:40.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.401+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89d0012910 con 0x7f89e00a0a30 2023-09-30T22:26:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.570+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f57a0074ea0 con 0x7f57a00a1510 2023-09-30T22:26:40.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.570+0000 7f578effd700 1 -- 172.21.15.32:0/2622666373 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f579402b020 con 0x7f57a00a1510 2023-09-30T22:26:40.571 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 msgr2=0x7f57a0122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 0x7f57a0122cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57940098f0 tx=0x7f5794004750 comp rx=0 tx=0).stop 2023-09-30T22:26:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 shutdown_connections 2023-09-30T22:26:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57a0139250 0x7f57a0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57a00a0a30 0x7f57a0122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 --2- 172.21.15.32:0/2622666373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57a00a1510 0x7f57a0122cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 >> 172.21.15.32:0/2622666373 conn(0x7f57a00a6550 msgr2=0x7f57a00a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 shutdown_connections 2023-09-30T22:26:40.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.572+0000 7f57a4ade700 1 -- 172.21.15.32:0/2622666373 wait complete. 2023-09-30T22:26:40.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.607+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f89e00a28e0 con 0x7f89e00a0a30 2023-09-30T22:26:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.608+0000 7f89deffd700 1 -- 172.21.15.32:0/527671513 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89d002c090 con 0x7f89e00a0a30 2023-09-30T22:26:40.609 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 msgr2=0x7f89e0121700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 0x7f89e0121700 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f89d000efc0 tx=0x7f89d000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 shutdown_connections 2023-09-30T22:26:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89e0134170 0x7f89e0122180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e00a0a30 0x7f89e0121700 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 --2- 172.21.15.32:0/527671513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89e00a1340 0x7f89e0121c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 >> 172.21.15.32:0/527671513 conn(0x7f89e00a6550 msgr2=0x7f89e009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 shutdown_connections 2023-09-30T22:26:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.610+0000 7f89e85cb700 1 -- 172.21.15.32:0/527671513 wait complete. 2023-09-30T22:26:40.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:40.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:40.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.806+0000 7fe85b675700 1 Processor -- start 2023-09-30T22:26:40.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe85b675700 1 -- start start 2023-09-30T22:26:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe85b675700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe854138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe85b675700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe85b675700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85406b990 con 0x7fe854133420 2023-09-30T22:26:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe85b675700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85409b0e0 con 0x7fe8541337f0 2023-09-30T22:26:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe85b675700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe854133c20 con 0x7fe854138dc0 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.807+0000 7fe858c10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe854138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe858c10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe854138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46244/0 (socket says 172.21.15.32:46244) 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859411700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe854133420 0x7fe854133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35304/0 (socket says 172.21.15.32:35304) 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe858c10700 1 -- 172.21.15.32:0/2875897189 learned_addr learned my addr 172.21.15.32:0/2875897189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85413b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52292/0 (socket says 172.21.15.32:52292) 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 -- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 msgr2=0x7fe854138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe84bfff700 1 -- 172.21.15.32:0/2875897189 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613541708 0 0) 0x7fe85406b990 con 0x7fe854133420 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 --2- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe854138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 -- 172.21.15.32:0/2875897189 >> v1:172.21.15.32:6789/0 conn(0x7fe854133420 legacy=0x7fe854133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 -- 172.21.15.32:0/2875897189 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540a0590 con 0x7fe854138dc0 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe858c10700 1 --2- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe854138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe859c12700 1 --2- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85413b1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009dc0 tx=0x7fe84c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ce1d6772a23573 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe84bfff700 1 -- 172.21.15.32:0/2875897189 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00e070 con 0x7fe854138dc0 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe84bfff700 1 -- 172.21.15.32:0/2875897189 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00bb20 con 0x7fe854138dc0 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.808+0000 7fe84bfff700 1 -- 172.21.15.32:0/2875897189 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c012430 con 0x7fe854138dc0 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 -- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 msgr2=0x7fe85413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 --2- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85413b1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009dc0 tx=0x7fe84c00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 -- 172.21.15.32:0/2875897189 shutdown_connections 2023-09-30T22:26:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 --2- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85413b1a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 --2- 172.21.15.32:0/2875897189 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe854138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 -- 172.21.15.32:0/2875897189 >> 172.21.15.32:0/2875897189 conn(0x7fe8540a4210 msgr2=0x7fe8540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 -- 172.21.15.32:0/2875897189 shutdown_connections 2023-09-30T22:26:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.809+0000 7fe85b675700 1 -- 172.21.15.32:0/2875897189 wait complete. 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.810+0000 7fe85b675700 1 Processor -- start 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.810+0000 7fe85b675700 1 -- start start 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe85b675700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 0x7fe85409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe85b675700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe85409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe85b675700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe85b675700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85412eef0 con 0x7fe854133420 2023-09-30T22:26:40.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe85b675700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85412e530 con 0x7fe8541337f0 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe85b675700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe85412f760 con 0x7fe854138dc0 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859411700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 0x7fe85409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52302/0 (socket says 172.21.15.32:52302) 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 -- 172.21.15.32:0/1728505536 learned_addr learned my addr 172.21.15.32:0/1728505536 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859411700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 0x7fe85409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60054/0 (socket says 172.21.15.32:60054) 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe858c10700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe85409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 -- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 msgr2=0x7fe85409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe85409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 -- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 msgr2=0x7fe85409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 0x7fe85409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 -- 172.21.15.32:0/1728505536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009550 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859411700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 0x7fe85409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.811+0000 7fe859c12700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85409cf00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe84c014040 tx=0x7fe84c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.812+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c012430 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.812+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c020070 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.812+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c009a70 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.812+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c004280 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.812+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe854130090 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.813+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe84c0043e0 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.813+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe84c022070 con 0x7fe854138dc0 2023-09-30T22:26:40.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.813+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe854132c50 con 0x7fe854138dc0 2023-09-30T22:26:40.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.822+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe84c01a970 con 0x7fe854138dc0 2023-09-30T22:26:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 Processor -- start 2023-09-30T22:26:40.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 -- start start 2023-09-30T22:26:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad780a1510 0x7fad78138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 0x7fad7813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7809edd0 con 0x7fad780a0a30 2023-09-30T22:26:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780a18e0 con 0x7fad780a1510 2023-09-30T22:26:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7ee3c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7809e1f0 con 0x7fad78139250 2023-09-30T22:26:40.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad77fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad780a1510 0x7fad78138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.852+0000 7fad7d3d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 0x7fad7813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad780a1510 0x7fad78138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60056/0 (socket says 172.21.15.32:60056) 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 -- 172.21.15.32:0/3231550213 learned_addr learned my addr 172.21.15.32:0/3231550213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad7d3d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 0x7fad7813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52310/0 (socket says 172.21.15.32:52310) 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 -- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 msgr2=0x7fad7813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 --2- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 0x7fad7813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 -- 172.21.15.32:0/3231550213 >> v1:172.21.15.32:6789/0 conn(0x7fad780a0a30 legacy=0x7fad780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 -- 172.21.15.32:0/3231550213 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7809dc50 con 0x7fad780a1510 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad777fe700 1 -- 172.21.15.32:0/3231550213 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983996577 0 0) 0x7fad7809edd0 con 0x7fad780a0a30 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad7d3d9700 1 --2- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 0x7fad7813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:40.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad77fff700 1 --2- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad780a1510 0x7fad78138d10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0098f0 tx=0x7fad6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47c1c4dae5ededf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad777fe700 1 -- 172.21.15.32:0/3231550213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c011830 con 0x7fad780a1510 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad777fe700 1 -- 172.21.15.32:0/3231550213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c011990 con 0x7fad780a1510 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.853+0000 7fad777fe700 1 -- 172.21.15.32:0/3231550213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c011d20 con 0x7fad780a1510 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 -- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad780a1510 msgr2=0x7fad78138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad780a1510 0x7fad78138d10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0098f0 tx=0x7fad6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 -- 172.21.15.32:0/3231550213 shutdown_connections 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad78139250 0x7fad7813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/3231550213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad780a1510 0x7fad78138d10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 -- 172.21.15.32:0/3231550213 >> 172.21.15.32:0/3231550213 conn(0x7fad780a6550 msgr2=0x7fad780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 -- 172.21.15.32:0/3231550213 shutdown_connections 2023-09-30T22:26:40.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.854+0000 7fad7ee3c700 1 -- 172.21.15.32:0/3231550213 wait complete. 2023-09-30T22:26:40.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.855+0000 7fad7ee3c700 1 Processor -- start 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.855+0000 7fad7ee3c700 1 -- start start 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7ee3c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad780a0a30 0x7fad781228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7ee3c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad780a1510 0x7fad78122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7ee3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 0x7fad78123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7ee3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad781389e0 con 0x7fad78139250 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7ee3c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780a0220 con 0x7fad780a1510 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7ee3c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad78126060 con 0x7fad780a0a30 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 0x7fad78123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 0x7fad78123340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60062/0 (socket says 172.21.15.32:60062) 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad77fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad780a1510 0x7fad78122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 -- 172.21.15.32:0/1704260427 learned_addr learned my addr 172.21.15.32:0/1704260427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7cbd8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad780a0a30 0x7fad781228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 -- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad780a0a30 msgr2=0x7fad781228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad780a0a30 0x7fad781228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 -- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad780a1510 msgr2=0x7fad78122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad780a1510 0x7fad78122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.856+0000 7fad7d3d9700 1 -- 172.21.15.32:0/1704260427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009a70 con 0x7fad78139250 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.857+0000 7fad7d3d9700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 0x7fad78123340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fad7000efb0 tx=0x7fad7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.857+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000cc90 con 0x7fad78139250 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.857+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000f040 con 0x7fad78139250 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.857+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70014480 con 0x7fad78139250 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.857+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c009550 con 0x7fad78139250 2023-09-30T22:26:40.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.857+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad78124bb0 con 0x7fad78139250 2023-09-30T22:26:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.858+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad700145e0 con 0x7fad78139250 2023-09-30T22:26:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.858+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad70040310 con 0x7fad78139250 2023-09-30T22:26:40.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.858+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78074ea0 con 0x7fad78139250 2023-09-30T22:26:40.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:40.863+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad700076f0 con 0x7fad78139250 2023-09-30T22:26:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.031+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe854130030 con 0x7fe854138dc0 2023-09-30T22:26:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.031+0000 7fe849ffb700 1 -- 172.21.15.32:0/1728505536 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe84c036090 con 0x7fe854138dc0 2023-09-30T22:26:41.032 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 msgr2=0x7fe85409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85409cf00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe84c014040 tx=0x7fe84c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 shutdown_connections 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe854138dc0 0x7fe85409cf00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe854133420 0x7fe85409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 --2- 172.21.15.32:0/1728505536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8541337f0 0x7fe85409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 >> 172.21.15.32:0/1728505536 conn(0x7fe8540a4210 msgr2=0x7fe8540a38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:41.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 shutdown_connections 2023-09-30T22:26:41.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.034+0000 7fe85b675700 1 -- 172.21.15.32:0/1728505536 wait complete. 2023-09-30T22:26:41.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.071+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fad780a28e0 con 0x7fad78139250 2023-09-30T22:26:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.071+0000 7fad757fa700 1 -- 172.21.15.32:0/1704260427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad7002a020 con 0x7fad78139250 2023-09-30T22:26:41.072 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 msgr2=0x7fad78123340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 0x7fad78123340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fad7000efb0 tx=0x7fad7000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 shutdown_connections 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad78139250 0x7fad78123340 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad780a0a30 0x7fad781228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 --2- 172.21.15.32:0/1704260427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad780a1510 0x7fad78122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.073+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 >> 172.21.15.32:0/1704260427 conn(0x7fad780a6550 msgr2=0x7fad780a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.074+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 shutdown_connections 2023-09-30T22:26:41.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.074+0000 7fad7ee3c700 1 -- 172.21.15.32:0/1704260427 wait complete. 2023-09-30T22:26:41.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:41.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:41.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.735+0000 7f5dd74fc700 1 Processor -- start 2023-09-30T22:26:41.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.735+0000 7f5dd74fc700 1 -- start start 2023-09-30T22:26:41.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd74fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 0x7f5dd0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd74fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5dd0139250 0x7f5dd013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd74fc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd009edd0 con 0x7f5dd00a0a30 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd74fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd00a18e0 con 0x7f5dd0139250 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd74fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd009e1f0 con 0x7f5dd00a1510 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 0x7f5dd0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 0x7f5dd0138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46270/0 (socket says 172.21.15.32:46270) 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 -- 172.21.15.32:0/3295571793 learned_addr learned my addr 172.21.15.32:0/3295571793 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd5a99700 1 --2- 172.21.15.32:0/3295571793 >> v2:172.21.15.32:3300/0 conn(0x7f5dd0139250 0x7f5dd013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 -- 172.21.15.32:0/3295571793 >> v2:172.21.15.32:3300/0 conn(0x7f5dd0139250 msgr2=0x7f5dd013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:41.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 --2- 172.21.15.32:0/3295571793 >> v2:172.21.15.32:3300/0 conn(0x7f5dd0139250 0x7f5dd013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 -- 172.21.15.32:0/3295571793 >> v1:172.21.15.32:6789/0 conn(0x7f5dd00a0a30 legacy=0x7f5dd00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd4a97700 1 -- 172.21.15.32:0/3295571793 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd009dc50 con 0x7f5dd00a1510 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.736+0000 7f5dd5a99700 1 --2- 172.21.15.32:0/3295571793 >> v2:172.21.15.32:3300/0 conn(0x7f5dd0139250 0x7f5dd013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd4a97700 1 --2- 172.21.15.32:0/3295571793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 0x7f5dd0138d10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009920 tx=0x7f5dc4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252674caeae20b70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dcffff700 1 -- 172.21.15.32:0/3295571793 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4011830 con 0x7f5dd00a1510 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dcffff700 1 -- 172.21.15.32:0/3295571793 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4011990 con 0x7f5dd00a1510 2023-09-30T22:26:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dcffff700 1 -- 172.21.15.32:0/3295571793 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4011d20 con 0x7f5dd00a1510 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd74fc700 1 -- 172.21.15.32:0/3295571793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 msgr2=0x7f5dd0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/3295571793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 0x7f5dd0138d10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009920 tx=0x7f5dc4009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd74fc700 1 -- 172.21.15.32:0/3295571793 shutdown_connections 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/3295571793 >> v2:172.21.15.32:3300/0 conn(0x7f5dd0139250 0x7f5dd013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/3295571793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd00a1510 0x7f5dd0138d10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.737+0000 7f5dd74fc700 1 -- 172.21.15.32:0/3295571793 >> 172.21.15.32:0/3295571793 conn(0x7f5dd00a6550 msgr2=0x7f5dd00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.738+0000 7f5dd74fc700 1 -- 172.21.15.32:0/3295571793 shutdown_connections 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.738+0000 7f5dd74fc700 1 -- 172.21.15.32:0/3295571793 wait complete. 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 Processor -- start 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 -- start start 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd00a0a30 0x7f5dd01c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 0x7f5dd01c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 0x7f5dd0122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0125eb0 con 0x7f5dd00a0a30 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0138dd0 con 0x7f5dd0139250 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd74fc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd01be3d0 con 0x7f5dd00a1510 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd5a99700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 0x7f5dd0122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd4a97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 0x7f5dd01c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd5a99700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 0x7f5dd0122930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46276/0 (socket says 172.21.15.32:46276) 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd4a97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 0x7f5dd01c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52324/0 (socket says 172.21.15.32:52324) 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd5a99700 1 -- 172.21.15.32:0/4151078202 learned_addr learned my addr 172.21.15.32:0/4151078202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.739+0000 7f5dd5298700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd00a0a30 0x7f5dd01c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd5a99700 1 -- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 msgr2=0x7f5dd01c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd5a99700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 0x7f5dd01c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd5a99700 1 -- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd00a0a30 msgr2=0x7f5dd01c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd5a99700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd00a0a30 0x7f5dd01c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd5a99700 1 -- 172.21.15.32:0/4151078202 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8009a70 con 0x7f5dd0139250 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd4a97700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 0x7f5dd01c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd5a99700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 0x7f5dd0122930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009eb0 tx=0x7f5dc800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:41.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8009d50 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc800f040 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc4009550 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc800ceb0 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.740+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd0126450 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.741+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5dc8014480 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.741+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5dc8040400 con 0x7f5dd0139250 2023-09-30T22:26:41.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.741+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd0074ea0 con 0x7f5dd0139250 2023-09-30T22:26:41.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.746+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5dc8018080 con 0x7f5dd0139250 2023-09-30T22:26:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.772+0000 7fca9f59e700 1 Processor -- start 2023-09-30T22:26:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.772+0000 7fca9f59e700 1 -- start start 2023-09-30T22:26:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 0x7fca980a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a1520 0x7fca98133e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9809e770 con 0x7fca980a1110 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca980a1aa0 con 0x7fca980a1520 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9f59e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9809cc80 con 0x7fca980a0800 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 0x7fca980a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a1520 0x7fca98133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca980a1110 0x7fca98133780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35326/0 (socket says 172.21.15.32:35326) 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9dd9b700 1 -- 172.21.15.32:0/686307031 learned_addr learned my addr 172.21.15.32:0/686307031 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 0x7fca980a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52328/0 (socket says 172.21.15.32:52328) 2023-09-30T22:26:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9d59a700 1 -- 172.21.15.32:0/686307031 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159977771 0 0) 0x7fca9809e770 con 0x7fca980a1110 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9e59c700 1 -- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a1520 msgr2=0x7fca98133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.773+0000 7fca9e59c700 1 --2- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a1520 0x7fca98133e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9e59c700 1 -- 172.21.15.32:0/686307031 >> v1:172.21.15.32:6789/0 conn(0x7fca980a1110 legacy=0x7fca98133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9e59c700 1 -- 172.21.15.32:0/686307031 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca980a1c70 con 0x7fca980a0800 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9ed9d700 1 --2- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a1520 0x7fca98133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9e59c700 1 --2- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 0x7fca980a0bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0098a0 tx=0x7fca8c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a9b7a19e090ac1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9d59a700 1 -- 172.21.15.32:0/686307031 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011830 con 0x7fca980a0800 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9d59a700 1 -- 172.21.15.32:0/686307031 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c011990 con 0x7fca980a0800 2023-09-30T22:26:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9d59a700 1 -- 172.21.15.32:0/686307031 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011d20 con 0x7fca980a0800 2023-09-30T22:26:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9f59e700 1 -- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 msgr2=0x7fca980a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.774+0000 7fca9f59e700 1 --2- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 0x7fca980a0bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0098a0 tx=0x7fca8c009bb0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.775+0000 7fca9f59e700 1 -- 172.21.15.32:0/686307031 shutdown_connections 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.775+0000 7fca9f59e700 1 --2- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a1520 0x7fca98133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.775+0000 7fca9f59e700 1 --2- 172.21.15.32:0/686307031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a0800 0x7fca980a0bd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.775+0000 7fca9f59e700 1 -- 172.21.15.32:0/686307031 >> 172.21.15.32:0/686307031 conn(0x7fca980a6140 msgr2=0x7fca980a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.775+0000 7fca9f59e700 1 -- 172.21.15.32:0/686307031 shutdown_connections 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.775+0000 7fca9f59e700 1 -- 172.21.15.32:0/686307031 wait complete. 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 Processor -- start 2023-09-30T22:26:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 -- start start 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 0x7fca981c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a1110 0x7fca981c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca980a1520 0x7fca981c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9811fdd0 con 0x7fca980a1520 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca981231a0 con 0x7fca980a0800 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9f59e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9809cec0 con 0x7fca980a1110 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.776+0000 7fca9e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 0x7fca981c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 0x7fca981c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46306/0 (socket says 172.21.15.32:46306) 2023-09-30T22:26:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 -- 172.21.15.32:0/4287330620 learned_addr learned my addr 172.21.15.32:0/4287330620 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9ed9d700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca980a1520 0x7fca981c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9dd9b700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a1110 0x7fca981c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 -- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a1110 msgr2=0x7fca981c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a1110 0x7fca981c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 -- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca980a1520 msgr2=0x7fca981c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca980a1520 0x7fca981c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 -- 172.21.15.32:0/4287330620 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94009a70 con 0x7fca980a0800 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9dd9b700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a1110 0x7fca981c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9ed9d700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca980a1520 0x7fca981c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9e59c700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 0x7fca981c7ee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fca8c000c00 tx=0x7fca8c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004340 con 0x7fca980a0800 2023-09-30T22:26:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c01f070 con 0x7fca980a0800 2023-09-30T22:26:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c0094a0 con 0x7fca980a0800 2023-09-30T22:26:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0036a0 con 0x7fca980a0800 2023-09-30T22:26:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.777+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca98121490 con 0x7fca980a0800 2023-09-30T22:26:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.778+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca8c021030 con 0x7fca980a0800 2023-09-30T22:26:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.778+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca98121ce0 con 0x7fca980a0800 2023-09-30T22:26:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.779+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca8c040780 con 0x7fca980a0800 2023-09-30T22:26:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.783+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca8c006bd0 con 0x7fca980a0800 2023-09-30T22:26:41.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.954+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5dd00a28e0 con 0x7f5dd0139250 2023-09-30T22:26:41.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.954+0000 7f5dcdffb700 1 -- 172.21.15.32:0/4151078202 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5dc8012700 con 0x7f5dd0139250 2023-09-30T22:26:41.955 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:41.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 msgr2=0x7f5dd0122930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 0x7f5dd0122930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009eb0 tx=0x7f5dc800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 shutdown_connections 2023-09-30T22:26:41.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5dd0139250 0x7f5dd0122930 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd00a0a30 0x7f5dd01c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 --2- 172.21.15.32:0/4151078202 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5dd00a1510 0x7f5dd01c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 >> 172.21.15.32:0/4151078202 conn(0x7f5dd00a6550 msgr2=0x7f5dd00a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:41.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 shutdown_connections 2023-09-30T22:26:41.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.956+0000 7f5dd74fc700 1 -- 172.21.15.32:0/4151078202 wait complete. 2023-09-30T22:26:41.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:41.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.991+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fca9806cb20 con 0x7fca980a0800 2023-09-30T22:26:41.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.991+0000 7fca8affd700 1 -- 172.21.15.32:0/4287330620 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca8c015030 con 0x7fca980a0800 2023-09-30T22:26:41.993 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.993+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 msgr2=0x7fca981c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.993+0000 7fca9f59e700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 0x7fca981c7ee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fca8c000c00 tx=0x7fca8c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.993+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 shutdown_connections 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.993+0000 7fca9f59e700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca980a1520 0x7fca981c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.993+0000 7fca9f59e700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca980a0800 0x7fca981c7ee0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.993+0000 7fca9f59e700 1 --2- 172.21.15.32:0/4287330620 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca980a1110 0x7fca981c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.994+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 >> 172.21.15.32:0/4287330620 conn(0x7fca980a6140 msgr2=0x7fca981bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.994+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 shutdown_connections 2023-09-30T22:26:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:41.994+0000 7fca9f59e700 1 -- 172.21.15.32:0/4287330620 wait complete. 2023-09-30T22:26:42.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:42.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:42.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 Processor -- start 2023-09-30T22:26:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 -- start start 2023-09-30T22:26:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f480c0a09c0 0x7f480c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 0x7f480c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480c0a1810 con 0x7f480c133920 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c09ea10 con 0x7f480c0a09c0 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.191+0000 7f481148d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f480c09e210 con 0x7f480c09d5c0 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 0x7f480c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 0x7f480c0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52346/0 (socket says 172.21.15.32:52346) 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 -- 172.21.15.32:0/60853538 learned_addr learned my addr 172.21.15.32:0/60853538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480affd700 1 --2- 172.21.15.32:0/60853538 >> v2:172.21.15.32:3300/0 conn(0x7f480c0a09c0 0x7f480c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 -- 172.21.15.32:0/60853538 >> v2:172.21.15.32:3300/0 conn(0x7f480c0a09c0 msgr2=0x7f480c0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 --2- 172.21.15.32:0/60853538 >> v2:172.21.15.32:3300/0 conn(0x7f480c0a09c0 0x7f480c0a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 -- 172.21.15.32:0/60853538 >> v1:172.21.15.32:6789/0 conn(0x7f480c133920 legacy=0x7f480c133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 -- 172.21.15.32:0/60853538 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0a26f0 con 0x7f480c09d5c0 2023-09-30T22:26:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.192+0000 7f480a7fc700 1 --2- 172.21.15.32:0/60853538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 0x7f480c0a12d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1694d2b4aed9287 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.194+0000 7f4809ffb700 1 -- 172.21.15.32:0/60853538 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004230 con 0x7f480c09d5c0 2023-09-30T22:26:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.194+0000 7f4809ffb700 1 -- 172.21.15.32:0/60853538 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800004390 con 0x7f480c09d5c0 2023-09-30T22:26:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.194+0000 7f4809ffb700 1 -- 172.21.15.32:0/60853538 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000036a0 con 0x7f480c09d5c0 2023-09-30T22:26:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 -- 172.21.15.32:0/60853538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 msgr2=0x7f480c0a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 --2- 172.21.15.32:0/60853538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 0x7f480c0a12d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 -- 172.21.15.32:0/60853538 shutdown_connections 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 --2- 172.21.15.32:0/60853538 >> v2:172.21.15.32:3300/0 conn(0x7f480c0a09c0 0x7f480c0a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 --2- 172.21.15.32:0/60853538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c09d5c0 0x7f480c0a12d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 -- 172.21.15.32:0/60853538 >> 172.21.15.32:0/60853538 conn(0x7f480c0a2fe0 msgr2=0x7f480c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 -- 172.21.15.32:0/60853538 shutdown_connections 2023-09-30T22:26:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.195+0000 7f481148d700 1 -- 172.21.15.32:0/60853538 wait complete. 2023-09-30T22:26:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 Processor -- start 2023-09-30T22:26:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 -- start start 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 0x7f480c121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c0a09c0 0x7f480c1217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c133920 0x7f480c121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c122760 con 0x7f480c0a09c0 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.197+0000 7f481148d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c124db0 con 0x7f480c09d5c0 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f481148d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f480c123d50 con 0x7f480c133920 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 0x7f480c121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c133920 0x7f480c121d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c0a09c0 0x7f480c1217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 0x7f480c121290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46308/0 (socket says 172.21.15.32:46308) 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 -- 172.21.15.32:0/3074321220 learned_addr learned my addr 172.21.15.32:0/3074321220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 -- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c133920 msgr2=0x7f480c121d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c133920 0x7f480c121d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 -- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c0a09c0 msgr2=0x7f480c1217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c0a09c0 0x7f480c1217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 -- 172.21.15.32:0/3074321220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009550 con 0x7f480c09d5c0 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480b7fe700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c133920 0x7f480c121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480a7fc700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c0a09c0 0x7f480c1217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f480affd700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 0x7f480c121290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e9b0 tx=0x7f47fc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc009710 con 0x7f480c09d5c0 2023-09-30T22:26:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00ee50 con 0x7f480c09d5c0 2023-09-30T22:26:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.198+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c0a4090 con 0x7f480c09d5c0 2023-09-30T22:26:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.199+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00ca50 con 0x7f480c09d5c0 2023-09-30T22:26:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.199+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c09cec0 con 0x7f480c09d5c0 2023-09-30T22:26:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.199+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f47fc00cbb0 con 0x7f480c09d5c0 2023-09-30T22:26:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.200+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f47fc03f6c0 con 0x7f480c09d5c0 2023-09-30T22:26:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.200+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480c122f00 con 0x7f480c09d5c0 2023-09-30T22:26:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.205+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f47fc0232f0 con 0x7f480c09d5c0 2023-09-30T22:26:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.232+0000 7f0ead8c6700 1 Processor -- start 2023-09-30T22:26:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.233+0000 7f0ead8c6700 1 -- start start 2023-09-30T22:26:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.233+0000 7f0ead8c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.233+0000 7f0ead8c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.233+0000 7f0ead8c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea809b560 con 0x7f0ea809f160 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.233+0000 7f0ead8c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80a0040 con 0x7f0ea8131ea0 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.233+0000 7f0ead8c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80a02e0 con 0x7f0ea809e850 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0ea809f160 0x7f0ea8131790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35348/0 (socket says 172.21.15.32:35348) 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea67fc700 1 -- 172.21.15.32:0/3479303031 learned_addr learned my addr 172.21.15.32:0/3479303031 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea77fe700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea5ffb700 1 -- 172.21.15.32:0/3479303031 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439276715 0 0) 0x7f0ea809b560 con 0x7f0ea809f160 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea6ffd700 1 -- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 msgr2=0x7f0ea8139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea6ffd700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8139c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea6ffd700 1 -- 172.21.15.32:0/3479303031 >> v1:172.21.15.32:6789/0 conn(0x7f0ea809f160 legacy=0x7f0ea8131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea6ffd700 1 -- 172.21.15.32:0/3479303031 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea809fab0 con 0x7f0ea809e850 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea77fe700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea6ffd700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea809ec20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009dc0 tx=0x7f0e9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83b02dd838651b0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea5ffb700 1 -- 172.21.15.32:0/3479303031 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800e070 con 0x7f0ea809e850 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea5ffb700 1 -- 172.21.15.32:0/3479303031 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9800bb20 con 0x7f0ea809e850 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.234+0000 7f0ea5ffb700 1 -- 172.21.15.32:0/3479303031 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98012430 con 0x7f0ea809e850 2023-09-30T22:26:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 msgr2=0x7f0ea809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea809ec20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009dc0 tx=0x7f0e9800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3479303031 shutdown_connections 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3479303031 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea809ec20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3479303031 >> 172.21.15.32:0/3479303031 conn(0x7f0ea80a3f70 msgr2=0x7f0ea80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3479303031 shutdown_connections 2023-09-30T22:26:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.235+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3479303031 wait complete. 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.237+0000 7f0ead8c6700 1 Processor -- start 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.237+0000 7f0ead8c6700 1 -- start start 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ead8c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea8120140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ead8c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 0x7f0ea8120680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ead8c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8120bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ead8c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea8123a60 con 0x7f0ea809f160 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ead8c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea8122bf0 con 0x7f0ea8131ea0 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ead8c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea8131530 con 0x7f0ea809e850 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 0x7f0ea8120680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea8120140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8120bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 0x7f0ea8120680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60112/0 (socket says 172.21.15.32:60112) 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea67fc700 1 -- 172.21.15.32:0/3191364100 learned_addr learned my addr 172.21.15.32:0/3191364100 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea8120140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52358/0 (socket says 172.21.15.32:52358) 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 -- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 msgr2=0x7f0ea8120bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8120bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 -- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 msgr2=0x7f0ea8120680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 0x7f0ea8120680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea6ffd700 1 -- 172.21.15.32:0/3191364100 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00094a0 con 0x7f0ea809e850 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.238+0000 7f0ea67fc700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 0x7f0ea8120680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.239+0000 7f0ea6ffd700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea8120140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80a1ab0 tx=0x7f0e98004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.239+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98012430 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.239+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e98020070 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.239+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98004040 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.239+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e98009a70 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.239+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea81236c0 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.240+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e980041a0 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.240+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e98022070 con 0x7f0ea809e850 2023-09-30T22:26:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.240+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea811cdd0 con 0x7f0ea809e850 2023-09-30T22:26:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.250+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e98004450 con 0x7f0ea809e850 2023-09-30T22:26:42.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.417+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f480c0747f0 con 0x7f480c09d5c0 2023-09-30T22:26:42.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.417+0000 7f47f37fe700 1 -- 172.21.15.32:0/3074321220 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f47fc017020 con 0x7f480c09d5c0 2023-09-30T22:26:42.419 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 msgr2=0x7f480c121290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 0x7f480c121290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e9b0 tx=0x7f47fc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 shutdown_connections 2023-09-30T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f480c133920 0x7f480c121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f480c09d5c0 0x7f480c121290 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 --2- 172.21.15.32:0/3074321220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f480c0a09c0 0x7f480c1217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 >> 172.21.15.32:0/3074321220 conn(0x7f480c0a2fe0 msgr2=0x7f480c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 shutdown_connections 2023-09-30T22:26:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.420+0000 7f481148d700 1 -- 172.21.15.32:0/3074321220 wait complete. 2023-09-30T22:26:42.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.454+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0ea80747f0 con 0x7f0ea809e850 2023-09-30T22:26:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.455+0000 7f0e8ffff700 1 -- 172.21.15.32:0/3191364100 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e9802c020 con 0x7f0ea809e850 2023-09-30T22:26:42.456 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 msgr2=0x7f0ea8120140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea8120140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80a1ab0 tx=0x7f0e98004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 shutdown_connections 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ea8131ea0 0x7f0ea8120bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ea809e850 0x7f0ea8120140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 --2- 172.21.15.32:0/3191364100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea809f160 0x7f0ea8120680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 >> 172.21.15.32:0/3191364100 conn(0x7f0ea80a3f70 msgr2=0x7f0ea80720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 shutdown_connections 2023-09-30T22:26:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:42.457+0000 7f0ead8c6700 1 -- 172.21.15.32:0/3191364100 wait complete. 2023-09-30T22:26:42.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:42.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:43.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 Processor -- start 2023-09-30T22:26:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 -- start start 2023-09-30T22:26:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01400a0a30 0x7f01400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a1340 0x7f0140133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f0140134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014009edd0 con 0x7f01400a0a30 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014009e1f0 con 0x7f01400a1340 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0146abc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f014009dc50 con 0x7f0140133fa0 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0145059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f0140134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f013ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a1340 0x7f0140133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0144858700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01400a0a30 0x7f01400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0145059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f0140134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52368/0 (socket says 172.21.15.32:52368) 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.121+0000 7f0145059700 1 -- 172.21.15.32:0/1414645684 learned_addr learned my addr 172.21.15.32:0/1414645684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0145059700 1 -- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a1340 msgr2=0x7f0140133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0145059700 1 --2- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a1340 0x7f0140133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0145059700 1 -- 172.21.15.32:0/1414645684 >> v2:172.21.15.32:3300/0 conn(0x7f01400a0a30 msgr2=0x7f01400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0145059700 1 --2- 172.21.15.32:0/1414645684 >> v2:172.21.15.32:3300/0 conn(0x7f01400a0a30 0x7f01400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0145059700 1 -- 172.21.15.32:0/1414645684 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01400a1fe0 con 0x7f0140133fa0 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0144858700 1 --2- 172.21.15.32:0/1414645684 >> v2:172.21.15.32:3300/0 conn(0x7f01400a0a30 0x7f01400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f013ffff700 1 --2- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a1340 0x7f0140133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f0145059700 1 --2- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f0140134410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0138009e80 tx=0x7f013800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54af2e4a1f7da4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f013f7fe700 1 -- 172.21.15.32:0/1414645684 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800e070 con 0x7f0140133fa0 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f013f7fe700 1 -- 172.21.15.32:0/1414645684 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013800b8d0 con 0x7f0140133fa0 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.122+0000 7f013f7fe700 1 -- 172.21.15.32:0/1414645684 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013800bc80 con 0x7f0140133fa0 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 -- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 msgr2=0x7f0140134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 --2- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f0140134410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0138009e80 tx=0x7f013800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 -- 172.21.15.32:0/1414645684 shutdown_connections 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 --2- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f0140134410 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 --2- 172.21.15.32:0/1414645684 >> v2:172.21.15.32:3300/0 conn(0x7f01400a0a30 0x7f01400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 --2- 172.21.15.32:0/1414645684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a1340 0x7f0140133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 -- 172.21.15.32:0/1414645684 >> 172.21.15.32:0/1414645684 conn(0x7f01400a6550 msgr2=0x7f01400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 -- 172.21.15.32:0/1414645684 shutdown_connections 2023-09-30T22:26:43.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.123+0000 7f0146abc700 1 -- 172.21.15.32:0/1414645684 wait complete. 2023-09-30T22:26:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.125+0000 7f0146abc700 1 Processor -- start 2023-09-30T22:26:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.125+0000 7f0146abc700 1 -- start start 2023-09-30T22:26:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0146abc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a0a30 0x7f014011e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0146abc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01400a1340 0x7f014011e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0146abc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f014011ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0146abc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0140121b30 con 0x7f01400a1340 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0146abc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0140120240 con 0x7f01400a0a30 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0146abc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01401224d0 con 0x7f0140133fa0 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0145059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f014011ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f013ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01400a1340 0x7f014011e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0145059700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f014011ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52370/0 (socket says 172.21.15.32:52370) 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0145059700 1 -- 172.21.15.32:0/1205588532 learned_addr learned my addr 172.21.15.32:0/1205588532 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0144858700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a0a30 0x7f014011e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f013ffff700 1 -- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 msgr2=0x7f014011ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f013ffff700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f014011ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f013ffff700 1 -- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a0a30 msgr2=0x7f014011e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f013ffff700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a0a30 0x7f014011e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f013ffff700 1 -- 172.21.15.32:0/1205588532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300094a0 con 0x7f01400a1340 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0144858700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a0a30 0x7f014011e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.126+0000 7f0145059700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f014011ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.127+0000 7f013ffff700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01400a1340 0x7f014011e550 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f013400e9e0 tx=0x7f013400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.127+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400ee80 con 0x7f01400a1340 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.127+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013400c7f0 con 0x7f01400a1340 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.127+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01340094a0 con 0x7f01400a1340 2023-09-30T22:26:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.127+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400cba0 con 0x7f01400a1340 2023-09-30T22:26:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.127+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0138009b30 con 0x7f01400a1340 2023-09-30T22:26:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.128+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f013400cd00 con 0x7f01400a1340 2023-09-30T22:26:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.128+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f013403f6a0 con 0x7f01400a1340 2023-09-30T22:26:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.128+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014009b1e0 con 0x7f01400a1340 2023-09-30T22:26:43.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.133+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f013401be10 con 0x7f01400a1340 2023-09-30T22:26:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.161+0000 7fc872501700 1 Processor -- start 2023-09-30T22:26:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.161+0000 7fc872501700 1 -- start start 2023-09-30T22:26:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc872501700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c0a0a30 0x7fc86c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc872501700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c138d80 0x7fc86c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc872501700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0a2a70 con 0x7fc86c09d630 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc872501700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0a01d0 con 0x7fc86c138d80 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc872501700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c09e2b0 con 0x7fc86c0a0a30 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c0a0a30 0x7fc86c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc870a9e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c138d80 0x7fc86c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc86c09d630 0x7fc86c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35358/0 (socket says 172.21.15.32:35358) 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86b7fe700 1 -- 172.21.15.32:0/2922582178 learned_addr learned my addr 172.21.15.32:0/2922582178 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86affd700 1 -- 172.21.15.32:0/2922582178 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323677584 0 0) 0x7fc86c0a2a70 con 0x7fc86c09d630 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86bfff700 1 -- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c138d80 msgr2=0x7fc86c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86bfff700 1 --2- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c138d80 0x7fc86c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86bfff700 1 -- 172.21.15.32:0/2922582178 >> v1:172.21.15.32:6789/0 conn(0x7fc86c09d630 legacy=0x7fc86c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86bfff700 1 -- 172.21.15.32:0/2922582178 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c09dc50 con 0x7fc86c0a0a30 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc870a9e700 1 --2- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c138d80 0x7fc86c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.162+0000 7fc86bfff700 1 --2- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c0a0a30 0x7fc86c0a0e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009870 tx=0x7fc85c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9fc08751b310c5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.163+0000 7fc86affd700 1 -- 172.21.15.32:0/2922582178 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011830 con 0x7fc86c0a0a30 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.163+0000 7fc86affd700 1 -- 172.21.15.32:0/2922582178 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c011990 con 0x7fc86c0a0a30 2023-09-30T22:26:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.163+0000 7fc86affd700 1 -- 172.21.15.32:0/2922582178 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011d20 con 0x7fc86c0a0a30 2023-09-30T22:26:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 -- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c0a0a30 msgr2=0x7fc86c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 --2- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c0a0a30 0x7fc86c0a0e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009870 tx=0x7fc85c009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 -- 172.21.15.32:0/2922582178 shutdown_connections 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 --2- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c138d80 0x7fc86c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 --2- 172.21.15.32:0/2922582178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c0a0a30 0x7fc86c0a0e00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 -- 172.21.15.32:0/2922582178 >> 172.21.15.32:0/2922582178 conn(0x7fc86c0a3050 msgr2=0x7fc86c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 -- 172.21.15.32:0/2922582178 shutdown_connections 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.164+0000 7fc872501700 1 -- 172.21.15.32:0/2922582178 wait complete. 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.165+0000 7fc872501700 1 Processor -- start 2023-09-30T22:26:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.165+0000 7fc872501700 1 -- start start 2023-09-30T22:26:43.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc872501700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c09d630 0x7fc86c133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc872501700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 0x7fc86c133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc872501700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 0x7fc86c134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc872501700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0a0410 con 0x7fc86c0a0a30 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc872501700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c136ec0 con 0x7fc86c09d630 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc872501700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c13ac70 con 0x7fc86c138d80 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 0x7fc86c133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc870a9e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 0x7fc86c134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc870a9e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 0x7fc86c134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52394/0 (socket says 172.21.15.32:52394) 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 0x7fc86c133cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60130/0 (socket says 172.21.15.32:60130) 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc870a9e700 1 -- 172.21.15.32:0/759629276 learned_addr learned my addr 172.21.15.32:0/759629276 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.166+0000 7fc86bfff700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c09d630 0x7fc86c133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc870a9e700 1 -- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c09d630 msgr2=0x7fc86c133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc870a9e700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c09d630 0x7fc86c133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc870a9e700 1 -- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 msgr2=0x7fc86c133cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc870a9e700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 0x7fc86c133cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc870a9e700 1 -- 172.21.15.32:0/759629276 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009a70 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc86b7fe700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 0x7fc86c133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc870a9e700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 0x7fc86c134210 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc86400efc0 tx=0x7fc86400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400cc90 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400f040 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c0094a0 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864014480 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.167+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1371a0 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.168+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc8640145e0 con 0x7fc86c138d80 2023-09-30T22:26:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.168+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c0747f0 con 0x7fc86c138d80 2023-09-30T22:26:43.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.168+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc8640403a0 con 0x7fc86c138d80 2023-09-30T22:26:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.173+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8640076f0 con 0x7fc86c138d80 2023-09-30T22:26:43.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.342+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f01401216b0 con 0x7f01400a1340 2023-09-30T22:26:43.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.343+0000 7f013d7fa700 1 -- 172.21.15.32:0/1205588532 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01401216b0 con 0x7f01400a1340 2023-09-30T22:26:43.344 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01400a1340 msgr2=0x7f014011e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01400a1340 0x7f014011e550 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f013400e9e0 tx=0x7f013400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 shutdown_connections 2023-09-30T22:26:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0140133fa0 0x7f014011ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f01400a0a30 0x7f014011e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 --2- 172.21.15.32:0/1205588532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01400a1340 0x7f014011e550 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 >> 172.21.15.32:0/1205588532 conn(0x7f01400a6550 msgr2=0x7f01401330f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 shutdown_connections 2023-09-30T22:26:43.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.345+0000 7f0146abc700 1 -- 172.21.15.32:0/1205588532 wait complete. 2023-09-30T22:26:43.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:43.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.380+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc86c072980 con 0x7fc86c138d80 2023-09-30T22:26:43.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.381+0000 7fc868ff9700 1 -- 172.21.15.32:0/759629276 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc864018020 con 0x7fc86c138d80 2023-09-30T22:26:43.382 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:43.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 msgr2=0x7fc86c134210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 0x7fc86c134210 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc86400efc0 tx=0x7fc86400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 shutdown_connections 2023-09-30T22:26:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc86c138d80 0x7fc86c134210 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86c09d630 0x7fc86c133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 --2- 172.21.15.32:0/759629276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86c0a0a30 0x7fc86c133cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 >> 172.21.15.32:0/759629276 conn(0x7fc86c0a3050 msgr2=0x7fc86c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 shutdown_connections 2023-09-30T22:26:43.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.383+0000 7fc872501700 1 -- 172.21.15.32:0/759629276 wait complete. 2023-09-30T22:26:43.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:43.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:43.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.577+0000 7fc643d71700 1 Processor -- start 2023-09-30T22:26:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc643d71700 1 -- start start 2023-09-30T22:26:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc643d71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc643d71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 0x7fc63c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc643d71700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c0a2a70 con 0x7fc63c09d630 2023-09-30T22:26:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc643d71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0a01d0 con 0x7fc63c138d80 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc643d71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c09e2b0 con 0x7fc63c0a0a30 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc64230e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 0x7fc63c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc641b0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc64230e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 0x7fc63c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46394/0 (socket says 172.21.15.32:46394) 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc64230e700 1 -- 172.21.15.32:0/4218408127 learned_addr learned my addr 172.21.15.32:0/4218408127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.578+0000 7fc64130c700 1 --1- 172.21.15.32:0/4218408127 >> v1:172.21.15.32:6789/0 conn(0x7fc63c09d630 0x7fc63c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35368/0 (socket says 172.21.15.32:35368) 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc64230e700 1 -- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 msgr2=0x7fc63c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc64230e700 1 --2- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc64230e700 1 -- 172.21.15.32:0/4218408127 >> v1:172.21.15.32:6789/0 conn(0x7fc63c09d630 legacy=0x7fc63c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc640b0b700 1 -- 172.21.15.32:0/4218408127 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362573243 0 0) 0x7fc63c0a2a70 con 0x7fc63c09d630 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc64230e700 1 -- 172.21.15.32:0/4218408127 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c09dc50 con 0x7fc63c138d80 2023-09-30T22:26:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc641b0d700 1 --2- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc64230e700 1 --2- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 0x7fc63c13b160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc634009920 tx=0x7fc634009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6289b0d1f149fe60 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc640b0b700 1 -- 172.21.15.32:0/4218408127 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634011830 con 0x7fc63c138d80 2023-09-30T22:26:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc640b0b700 1 -- 172.21.15.32:0/4218408127 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc634011990 con 0x7fc63c138d80 2023-09-30T22:26:43.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.579+0000 7fc640b0b700 1 -- 172.21.15.32:0/4218408127 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634011d20 con 0x7fc63c138d80 2023-09-30T22:26:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 -- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 msgr2=0x7fc63c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 --2- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 0x7fc63c13b160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc634009920 tx=0x7fc634009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 -- 172.21.15.32:0/4218408127 shutdown_connections 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 --2- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c138d80 0x7fc63c13b160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 --2- 172.21.15.32:0/4218408127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 -- 172.21.15.32:0/4218408127 >> 172.21.15.32:0/4218408127 conn(0x7fc63c0a3050 msgr2=0x7fc63c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 -- 172.21.15.32:0/4218408127 shutdown_connections 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.580+0000 7fc643d71700 1 -- 172.21.15.32:0/4218408127 wait complete. 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.581+0000 7fc643d71700 1 Processor -- start 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.581+0000 7fc643d71700 1 -- start start 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc643d71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c09d630 0x7fc63c133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc643d71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc643d71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 0x7fc63c1341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc643d71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c1371c0 con 0x7fc63c138d80 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc643d71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c136500 con 0x7fc63c09d630 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc643d71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c13acc0 con 0x7fc63c0a0a30 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 0x7fc63c1341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc641b0d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c09d630 0x7fc63c133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 0x7fc63c1341b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60134/0 (socket says 172.21.15.32:60134) 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 -- 172.21.15.32:0/3662804469 learned_addr learned my addr 172.21.15.32:0/3662804469 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64130c700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 -- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 msgr2=0x7fc63c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 -- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c09d630 msgr2=0x7fc63c133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c09d630 0x7fc63c133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 -- 172.21.15.32:0/3662804469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c009a70 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc641b0d700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c09d630 0x7fc63c133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64130c700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.582+0000 7fc64230e700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 0x7fc63c1341b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc634009c70 tx=0x7fc634004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.583+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634004450 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.583+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6340045d0 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.583+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc634009550 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.583+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340038e0 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.583+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c136c20 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.583+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc634003a40 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.584+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc634040aa0 con 0x7fc63c138d80 2023-09-30T22:26:43.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.584+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c0747f0 con 0x7fc63c138d80 2023-09-30T22:26:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.589+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc634019ba0 con 0x7fc63c138d80 2023-09-30T22:26:43.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.620+0000 7fd78b06b700 1 Processor -- start 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.620+0000 7fd78b06b700 1 -- start start 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd78b06b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a1510 0x7fd784138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd78b06b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd784139250 0x7fd78413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd78b06b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78409edd0 con 0x7fd7840a0a30 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd78b06b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7840a18e0 con 0x7fd784139250 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd78b06b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd78409e1f0 con 0x7fd7840a1510 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd783fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a1510 0x7fd784138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd789608700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd784139250 0x7fd78413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd788e07700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd7840a0a30 0x7fd7840a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35374/0 (socket says 172.21.15.32:35374) 2023-09-30T22:26:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd788e07700 1 -- 172.21.15.32:0/1485939276 learned_addr learned my addr 172.21.15.32:0/1485939276 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd783fff700 1 -- 172.21.15.32:0/1485939276 >> v2:172.21.15.32:3300/0 conn(0x7fd784139250 msgr2=0x7fd78413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd783fff700 1 --2- 172.21.15.32:0/1485939276 >> v2:172.21.15.32:3300/0 conn(0x7fd784139250 0x7fd78413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd783fff700 1 -- 172.21.15.32:0/1485939276 >> v1:172.21.15.32:6789/0 conn(0x7fd7840a0a30 legacy=0x7fd7840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd783fff700 1 -- 172.21.15.32:0/1485939276 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78409dc50 con 0x7fd7840a1510 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd7837fe700 1 -- 172.21.15.32:0/1485939276 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734794082 0 0) 0x7fd78409edd0 con 0x7fd7840a0a30 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd789608700 1 --2- 172.21.15.32:0/1485939276 >> v2:172.21.15.32:3300/0 conn(0x7fd784139250 0x7fd78413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.621+0000 7fd783fff700 1 --2- 172.21.15.32:0/1485939276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a1510 0x7fd784138d10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd770009870 tx=0x7fd770009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=790b341b38e8e6db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd7837fe700 1 -- 172.21.15.32:0/1485939276 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd770011830 con 0x7fd7840a1510 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd7837fe700 1 -- 172.21.15.32:0/1485939276 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd770011990 con 0x7fd7840a1510 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd7837fe700 1 -- 172.21.15.32:0/1485939276 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd770011d20 con 0x7fd7840a1510 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 -- 172.21.15.32:0/1485939276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a1510 msgr2=0x7fd784138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 --2- 172.21.15.32:0/1485939276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a1510 0x7fd784138d10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd770009870 tx=0x7fd770009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 -- 172.21.15.32:0/1485939276 shutdown_connections 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 --2- 172.21.15.32:0/1485939276 >> v2:172.21.15.32:3300/0 conn(0x7fd784139250 0x7fd78413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 --2- 172.21.15.32:0/1485939276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a1510 0x7fd784138d10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 -- 172.21.15.32:0/1485939276 >> 172.21.15.32:0/1485939276 conn(0x7fd7840a6550 msgr2=0x7fd7840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 -- 172.21.15.32:0/1485939276 shutdown_connections 2023-09-30T22:26:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.622+0000 7fd78b06b700 1 -- 172.21.15.32:0/1485939276 wait complete. 2023-09-30T22:26:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.623+0000 7fd78b06b700 1 Processor -- start 2023-09-30T22:26:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.623+0000 7fd78b06b700 1 -- start start 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd78b06b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a0a30 0x7fd784122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd78b06b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7840a1510 0x7fd784122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd78b06b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 0x7fd784123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd78b06b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841389b0 con 0x7fd784139250 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd78b06b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd784124390 con 0x7fd7840a1510 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd78b06b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd784124c60 con 0x7fd7840a0a30 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd789608700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 0x7fd784123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd788e07700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a0a30 0x7fd784122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd789608700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 0x7fd784123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60156/0 (socket says 172.21.15.32:60156) 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd789608700 1 -- 172.21.15.32:0/3920789527 learned_addr learned my addr 172.21.15.32:0/3920789527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd783fff700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7840a1510 0x7fd784122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd788e07700 1 -- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7840a1510 msgr2=0x7fd784122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd788e07700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7840a1510 0x7fd784122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd788e07700 1 -- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 msgr2=0x7fd784123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.624+0000 7fd788e07700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 0x7fd784123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd788e07700 1 -- 172.21.15.32:0/3920789527 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009a70 con 0x7fd7840a0a30 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd789608700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 0x7fd784123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd788e07700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a0a30 0x7fd784122780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c4a0 tx=0x7fd77800c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778004c80 con 0x7fd7840a0a30 2023-09-30T22:26:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd778004de0 con 0x7fd7840a0a30 2023-09-30T22:26:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7700094a0 con 0x7fd7840a0a30 2023-09-30T22:26:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778007310 con 0x7fd7840a0a30 2023-09-30T22:26:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.625+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd784125fc0 con 0x7fd7840a0a30 2023-09-30T22:26:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.626+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd778007470 con 0x7fd7840a0a30 2023-09-30T22:26:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.626+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd77803f2e0 con 0x7fd7840a0a30 2023-09-30T22:26:43.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.626+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd784074ea0 con 0x7fd7840a0a30 2023-09-30T22:26:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.631+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd778007720 con 0x7fd7840a0a30 2023-09-30T22:26:43.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.797+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc63c072980 con 0x7fc63c138d80 2023-09-30T22:26:43.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.798+0000 7fc63a7fc700 1 -- 172.21.15.32:0/3662804469 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc634019ba0 con 0x7fc63c138d80 2023-09-30T22:26:43.799 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 msgr2=0x7fc63c1341b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 0x7fc63c1341b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc634009c70 tx=0x7fc634004750 comp rx=0 tx=0).stop 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 shutdown_connections 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc63c138d80 0x7fc63c1341b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc63c09d630 0x7fc63c133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 --2- 172.21.15.32:0/3662804469 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc63c0a0a30 0x7fc63c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 >> 172.21.15.32:0/3662804469 conn(0x7fc63c0a3050 msgr2=0x7fc63c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 shutdown_connections 2023-09-30T22:26:43.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.800+0000 7fc643d71700 1 -- 172.21.15.32:0/3662804469 wait complete. 2023-09-30T22:26:43.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.836+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd7840a28e0 con 0x7fd7840a0a30 2023-09-30T22:26:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.836+0000 7fd7817fa700 1 -- 172.21.15.32:0/3920789527 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd778011360 con 0x7fd7840a0a30 2023-09-30T22:26:43.837 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.838+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a0a30 msgr2=0x7fd784122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.838+0000 7fd78b06b700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a0a30 0x7fd784122780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c4a0 tx=0x7fd77800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 shutdown_connections 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd784139250 0x7fd784123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7840a0a30 0x7fd784122780 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 --2- 172.21.15.32:0/3920789527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7840a1510 0x7fd784122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 >> 172.21.15.32:0/3920789527 conn(0x7fd7840a6550 msgr2=0x7fd7840a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 shutdown_connections 2023-09-30T22:26:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:43.839+0000 7fd78b06b700 1 -- 172.21.15.32:0/3920789527 wait complete. 2023-09-30T22:26:43.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:44.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:44.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:44.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.507+0000 7fce04187700 1 Processor -- start 2023-09-30T22:26:44.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.507+0000 7fce04187700 1 -- start start 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce04187700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 0x7fcdfc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce04187700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcdfc09d630 0x7fcdfc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce04187700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 0x7fcdfc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce04187700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc0a2a70 con 0x7fcdfc09d630 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce04187700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0a01d0 con 0x7fcdfc133930 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce04187700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc09e2b0 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce01f23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 0x7fcdfc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 0x7fcdfc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce01f23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 0x7fcdfc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52442/0 (socket says 172.21.15.32:52442) 2023-09-30T22:26:44.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 0x7fcdfc133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46414/0 (socket says 172.21.15.32:46414) 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce01f23700 1 -- 172.21.15.32:0/796547895 learned_addr learned my addr 172.21.15.32:0/796547895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce01722700 1 --2- 172.21.15.32:0/796547895 >> v2:172.21.15.32:3300/0 conn(0x7fcdfc09d630 0x7fcdfc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 -- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 msgr2=0x7fcdfc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 --2- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 0x7fcdfc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 -- 172.21.15.32:0/796547895 >> v2:172.21.15.32:3300/0 conn(0x7fcdfc09d630 msgr2=0x7fcdfc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 --2- 172.21.15.32:0/796547895 >> v2:172.21.15.32:3300/0 conn(0x7fcdfc09d630 0x7fcdfc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce02724700 1 -- 172.21.15.32:0/796547895 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc09dc50 con 0x7fcdfc133930 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce01f23700 1 --2- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 0x7fcdfc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.508+0000 7fce01722700 1 --2- 172.21.15.32:0/796547895 >> v2:172.21.15.32:3300/0 conn(0x7fcdfc09d630 0x7fcdfc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce02724700 1 --2- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 0x7fcdfc133dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009d90 tx=0x7fcdf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4227566440956f5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce00f21700 1 -- 172.21.15.32:0/796547895 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400e070 con 0x7fcdfc133930 2023-09-30T22:26:44.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce00f21700 1 -- 172.21.15.32:0/796547895 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf400bb20 con 0x7fcdfc133930 2023-09-30T22:26:44.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce00f21700 1 -- 172.21.15.32:0/796547895 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4012430 con 0x7fcdfc133930 2023-09-30T22:26:44.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 -- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 msgr2=0x7fcdfc133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 --2- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 0x7fcdfc133dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009d90 tx=0x7fcdf400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:44.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 -- 172.21.15.32:0/796547895 shutdown_connections 2023-09-30T22:26:44.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 --2- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc133930 0x7fcdfc133dd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 --2- 172.21.15.32:0/796547895 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc0a0a30 0x7fcdfc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 --2- 172.21.15.32:0/796547895 >> v2:172.21.15.32:3300/0 conn(0x7fcdfc09d630 0x7fcdfc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.509+0000 7fce04187700 1 -- 172.21.15.32:0/796547895 >> 172.21.15.32:0/796547895 conn(0x7fcdfc0a3050 msgr2=0x7fcdfc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.510+0000 7fce04187700 1 -- 172.21.15.32:0/796547895 shutdown_connections 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.510+0000 7fce04187700 1 -- 172.21.15.32:0/796547895 wait complete. 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 Processor -- start 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 -- start start 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc09d630 0x7fcdfc1c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 0x7fcdfc1c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc133930 0x7fcdfc1c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc122250 con 0x7fcdfc09d630 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc120540 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce04187700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0a4030 con 0x7fcdfc133930 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce01722700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 0x7fcdfc1c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce01722700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 0x7fcdfc1c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46420/0 (socket says 172.21.15.32:46420) 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce01722700 1 -- 172.21.15.32:0/2720380717 learned_addr learned my addr 172.21.15.32:0/2720380717 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce01f23700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc09d630 0x7fcdfc1c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.511+0000 7fce02724700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc133930 0x7fcdfc1c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce01722700 1 -- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc133930 msgr2=0x7fcdfc1c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce01722700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc133930 0x7fcdfc1c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce01722700 1 -- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc09d630 msgr2=0x7fcdfc1c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce01722700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc09d630 0x7fcdfc1c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce01722700 1 -- 172.21.15.32:0/2720380717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec0094a0 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce01722700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 0x7fcdfc1c8540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00ebb0 tx=0x7fcdec00eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00c850 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec00c9b0 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf8009550 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00cd40 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.512+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf4009a70 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.513+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdec013470 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.513+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc120d90 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.513+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcdec03f7b0 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.518+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdec0232f0 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 Processor -- start 2023-09-30T22:26:44.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 -- start start 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae809d630 0x7f1ae80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae8138d80 0x7f1ae813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae80a2a70 con 0x7f1ae80a0a30 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80a01d0 con 0x7f1ae809d630 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aef40b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae809e2b0 con 0x7f1ae8138d80 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aed9a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae8138d80 0x7f1ae813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aec9a6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ae809d630 0x7f1ae80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aed1a7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ae80a0a30 0x7f1ae80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35388/0 (socket says 172.21.15.32:35388) 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.547+0000 7f1aed9a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae8138d80 0x7f1ae813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46422/0 (socket says 172.21.15.32:46422) 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1aed1a7700 1 -- 172.21.15.32:0/402385634 learned_addr learned my addr 172.21.15.32:0/402385634 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1aec9a6700 1 -- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae8138d80 msgr2=0x7f1ae813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1aec9a6700 1 --2- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae8138d80 0x7f1ae813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1aec9a6700 1 -- 172.21.15.32:0/402385634 >> v1:172.21.15.32:6789/0 conn(0x7f1ae80a0a30 legacy=0x7f1ae80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1aec9a6700 1 -- 172.21.15.32:0/402385634 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae809dc50 con 0x7f1ae809d630 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1ae7fff700 1 -- 172.21.15.32:0/402385634 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487204742 0 0) 0x7f1ae80a2a70 con 0x7f1ae80a0a30 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1aec9a6700 1 --2- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae809d630 0x7f1ae80a1510 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009840 tx=0x7f1adc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16922d5b537747c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1ae7fff700 1 -- 172.21.15.32:0/402385634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc011830 con 0x7f1ae809d630 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1ae7fff700 1 -- 172.21.15.32:0/402385634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc011990 con 0x7f1ae809d630 2023-09-30T22:26:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.548+0000 7f1ae7fff700 1 -- 172.21.15.32:0/402385634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc011d20 con 0x7f1ae809d630 2023-09-30T22:26:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 -- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae809d630 msgr2=0x7f1ae80a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 --2- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae809d630 0x7f1ae80a1510 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009840 tx=0x7f1adc009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 -- 172.21.15.32:0/402385634 shutdown_connections 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 --2- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae8138d80 0x7f1ae813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 --2- 172.21.15.32:0/402385634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae809d630 0x7f1ae80a1510 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 -- 172.21.15.32:0/402385634 >> 172.21.15.32:0/402385634 conn(0x7f1ae80a3050 msgr2=0x7f1ae80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 -- 172.21.15.32:0/402385634 shutdown_connections 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.549+0000 7f1aef40b700 1 -- 172.21.15.32:0/402385634 wait complete. 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 Processor -- start 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 -- start start 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 0x7f1ae81337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae80a0a30 0x7f1ae8133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 0x7f1ae8134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae8136270 con 0x7f1ae8138d80 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae8136f90 con 0x7f1ae80a0a30 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.550+0000 7f1aef40b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80a4b10 con 0x7f1ae809d630 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed1a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 0x7f1ae81337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 0x7f1ae8134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed1a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 0x7f1ae81337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52466/0 (socket says 172.21.15.32:52466) 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 0x7f1ae8134220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60178/0 (socket says 172.21.15.32:60178) 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed1a7700 1 -- 172.21.15.32:0/100391017 learned_addr learned my addr 172.21.15.32:0/100391017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aec9a6700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae80a0a30 0x7f1ae8133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 -- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 msgr2=0x7f1ae81337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 0x7f1ae81337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 -- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae80a0a30 msgr2=0x7f1ae8133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae80a0a30 0x7f1ae8133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 -- 172.21.15.32:0/100391017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0094a0 con 0x7f1ae8138d80 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed1a7700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 0x7f1ae81337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aec9a6700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae80a0a30 0x7f1ae8133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1aed9a8700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 0x7f1ae8134220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000ef90 tx=0x7f1ae000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae000cc90 con 0x7f1ae8138d80 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.551+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae000f040 con 0x7f1ae8138d80 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.552+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae80a0410 con 0x7f1ae8138d80 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.552+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00146c0 con 0x7f1ae8138d80 2023-09-30T22:26:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.552+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae8137270 con 0x7f1ae8138d80 2023-09-30T22:26:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.553+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8136dd0 con 0x7f1ae8138d80 2023-09-30T22:26:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.553+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ae0014820 con 0x7f1ae8138d80 2023-09-30T22:26:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.553+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1ae0040830 con 0x7f1ae8138d80 2023-09-30T22:26:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.558+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ae0014ad0 con 0x7f1ae8138d80 2023-09-30T22:26:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.724+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcdfc1211b0 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.725+0000 7fcdf27fc700 1 -- 172.21.15.32:0/2720380717 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdec017070 con 0x7fcdfc0a0a30 2023-09-30T22:26:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 msgr2=0x7fcdfc1c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 0x7fcdfc1c8540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00ebb0 tx=0x7fcdec00eec0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 shutdown_connections 2023-09-30T22:26:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcdfc133930 0x7fcdfc1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdfc09d630 0x7fcdfc1c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 --2- 172.21.15.32:0/2720380717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcdfc0a0a30 0x7fcdfc1c8540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 >> 172.21.15.32:0/2720380717 conn(0x7fcdfc0a3050 msgr2=0x7fcdfc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 shutdown_connections 2023-09-30T22:26:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.728+0000 7fce04187700 1 -- 172.21.15.32:0/2720380717 wait complete. 2023-09-30T22:26:44.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.772+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1ae8136c30 con 0x7f1ae8138d80 2023-09-30T22:26:44.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.773+0000 7f1ae5ffb700 1 -- 172.21.15.32:0/100391017 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ae8136c30 con 0x7f1ae8138d80 2023-09-30T22:26:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 msgr2=0x7f1ae8134220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 0x7f1ae8134220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000ef90 tx=0x7f1ae000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 shutdown_connections 2023-09-30T22:26:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ae8138d80 0x7f1ae8134220 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1ae809d630 0x7f1ae81337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 --2- 172.21.15.32:0/100391017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ae80a0a30 0x7f1ae8133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 >> 172.21.15.32:0/100391017 conn(0x7f1ae80a3050 msgr2=0x7f1ae809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 shutdown_connections 2023-09-30T22:26:44.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.775+0000 7f1aef40b700 1 -- 172.21.15.32:0/100391017 wait complete. 2023-09-30T22:26:44.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:44.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:44.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 Processor -- start 2023-09-30T22:26:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 -- start start 2023-09-30T22:26:44.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcfc81337f0 0x7fcfc8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc8138dc0 0x7fcfc813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc806b990 con 0x7fcfc8133420 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc809b0e0 con 0x7fcfc81337f0 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.972+0000 7fcfce9aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8133c20 con 0x7fcfc8138dc0 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcfc81337f0 0x7fcfc8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcfc8133420 0x7fcfc8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35396/0 (socket says 172.21.15.32:35396) 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfccf47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc8138dc0 0x7fcfc813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc7fff700 1 -- 172.21.15.32:0/2122065302 learned_addr learned my addr 172.21.15.32:0/2122065302 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfccf47700 1 -- 172.21.15.32:0/2122065302 >> v2:172.21.15.32:3300/0 conn(0x7fcfc81337f0 msgr2=0x7fcfc8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfccf47700 1 --2- 172.21.15.32:0/2122065302 >> v2:172.21.15.32:3300/0 conn(0x7fcfc81337f0 0x7fcfc8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfccf47700 1 -- 172.21.15.32:0/2122065302 >> v1:172.21.15.32:6789/0 conn(0x7fcfc8133420 legacy=0x7fcfc8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfccf47700 1 -- 172.21.15.32:0/2122065302 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80a0590 con 0x7fcfc8138dc0 2023-09-30T22:26:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc6ffd700 1 -- 172.21.15.32:0/2122065302 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955246211 0 0) 0x7fcfc806b990 con 0x7fcfc8133420 2023-09-30T22:26:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc77fe700 1 --2- 172.21.15.32:0/2122065302 >> v2:172.21.15.32:3300/0 conn(0x7fcfc81337f0 0x7fcfc8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfccf47700 1 --2- 172.21.15.32:0/2122065302 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc8138dc0 0x7fcfc813b1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009d90 tx=0x7fcfc000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c688604d2e3d6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc6ffd700 1 -- 172.21.15.32:0/2122065302 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000e070 con 0x7fcfc8138dc0 2023-09-30T22:26:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc6ffd700 1 -- 172.21.15.32:0/2122065302 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc000bb20 con 0x7fcfc8138dc0 2023-09-30T22:26:44.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.973+0000 7fcfc6ffd700 1 -- 172.21.15.32:0/2122065302 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0012430 con 0x7fcfc8138dc0 2023-09-30T22:26:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 -- 172.21.15.32:0/2122065302 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc8138dc0 msgr2=0x7fcfc813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/2122065302 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc8138dc0 0x7fcfc813b1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009d90 tx=0x7fcfc000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 -- 172.21.15.32:0/2122065302 shutdown_connections 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/2122065302 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc8138dc0 0x7fcfc813b1a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/2122065302 >> v2:172.21.15.32:3300/0 conn(0x7fcfc81337f0 0x7fcfc8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 -- 172.21.15.32:0/2122065302 >> 172.21.15.32:0/2122065302 conn(0x7fcfc80a4210 msgr2=0x7fcfc80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 -- 172.21.15.32:0/2122065302 shutdown_connections 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.974+0000 7fcfce9aa700 1 -- 172.21.15.32:0/2122065302 wait complete. 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.975+0000 7fcfce9aa700 1 Processor -- start 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 -- start start 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcfc8133420 0x7fcfc812eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc81337f0 0x7fcfc812f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 0x7fcfc812f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:44.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc809c660 con 0x7fcfc8138dc0 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc809b9d0 con 0x7fcfc81337f0 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfce9aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8130820 con 0x7fcfc8133420 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfc77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc81337f0 0x7fcfc812f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfccf47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 0x7fcfc812f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfc77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc81337f0 0x7fcfc812f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46452/0 (socket says 172.21.15.32:46452) 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfccf47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 0x7fcfc812f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60196/0 (socket says 172.21.15.32:60196) 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfc77fe700 1 -- 172.21.15.32:0/3657118614 learned_addr learned my addr 172.21.15.32:0/3657118614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfc7fff700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcfc8133420 0x7fcfc812eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfccf47700 1 -- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcfc8133420 msgr2=0x7fcfc812eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfccf47700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcfc8133420 0x7fcfc812eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfccf47700 1 -- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc81337f0 msgr2=0x7fcfc812f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.976+0000 7fcfccf47700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc81337f0 0x7fcfc812f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfccf47700 1 -- 172.21.15.32:0/3657118614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0094a0 con 0x7fcfc8138dc0 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfccf47700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 0x7fcfc812f600 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0016000 tx=0x7fcfc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0012430 con 0x7fcfc8138dc0 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc0020070 con 0x7fcfc8138dc0 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0004280 con 0x7fcfc8138dc0 2023-09-30T22:26:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc0009a70 con 0x7fcfc8138dc0 2023-09-30T22:26:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.977+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc809c920 con 0x7fcfc8138dc0 2023-09-30T22:26:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.978+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc809cc50 con 0x7fcfc8138dc0 2023-09-30T22:26:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.978+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfc00043e0 con 0x7fcfc8138dc0 2023-09-30T22:26:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.979+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcfc0022070 con 0x7fcfc8138dc0 2023-09-30T22:26:44.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:44.983+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfc0031430 con 0x7fcfc8138dc0 2023-09-30T22:26:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.012+0000 7fe323954700 1 Processor -- start 2023-09-30T22:26:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.012+0000 7fe323954700 1 -- start start 2023-09-30T22:26:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe323954700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c0a0a30 0x7fe31c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe323954700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c138d80 0x7fe31c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe323954700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c0a2a70 con 0x7fe31c09d630 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe323954700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0a01d0 con 0x7fe31c138d80 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe323954700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c09e2b0 con 0x7fe31c0a0a30 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe320eef700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe31c09d630 0x7fe31c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35398/0 (socket says 172.21.15.32:35398) 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe320eef700 1 -- 172.21.15.32:0/1824155218 learned_addr learned my addr 172.21.15.32:0/1824155218 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe3216f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c0a0a30 0x7fe31c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe321ef1700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c138d80 0x7fe31c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe321ef1700 1 -- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c0a0a30 msgr2=0x7fe31c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe321ef1700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c0a0a30 0x7fe31c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe313fff700 1 -- 172.21.15.32:0/1824155218 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967004459 0 0) 0x7fe31c0a2a70 con 0x7fe31c09d630 2023-09-30T22:26:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe321ef1700 1 -- 172.21.15.32:0/1824155218 >> v1:172.21.15.32:6789/0 conn(0x7fe31c09d630 legacy=0x7fe31c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe321ef1700 1 -- 172.21.15.32:0/1824155218 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c09dc50 con 0x7fe31c138d80 2023-09-30T22:26:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.013+0000 7fe3216f0700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c0a0a30 0x7fe31c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe321ef1700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c138d80 0x7fe31c13b160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe314009dc0 tx=0x7fe31400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe9fbd56d84d057 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe313fff700 1 -- 172.21.15.32:0/1824155218 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400e070 con 0x7fe31c138d80 2023-09-30T22:26:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe313fff700 1 -- 172.21.15.32:0/1824155218 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31400bb20 con 0x7fe31c138d80 2023-09-30T22:26:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe313fff700 1 -- 172.21.15.32:0/1824155218 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314012430 con 0x7fe31c138d80 2023-09-30T22:26:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 -- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c138d80 msgr2=0x7fe31c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c138d80 0x7fe31c13b160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe314009dc0 tx=0x7fe31400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 -- 172.21.15.32:0/1824155218 shutdown_connections 2023-09-30T22:26:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c138d80 0x7fe31c13b160 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 --2- 172.21.15.32:0/1824155218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c0a0a30 0x7fe31c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 -- 172.21.15.32:0/1824155218 >> 172.21.15.32:0/1824155218 conn(0x7fe31c0a3050 msgr2=0x7fe31c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.014+0000 7fe323954700 1 -- 172.21.15.32:0/1824155218 shutdown_connections 2023-09-30T22:26:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.015+0000 7fe323954700 1 -- 172.21.15.32:0/1824155218 wait complete. 2023-09-30T22:26:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.015+0000 7fe323954700 1 Processor -- start 2023-09-30T22:26:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 -- start start 2023-09-30T22:26:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c09d630 0x7fe31c133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 0x7fe31c133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c138d80 0x7fe31c134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c134bd0 con 0x7fe31c0a0a30 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c136140 con 0x7fe31c09d630 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe323954700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c136e60 con 0x7fe31c138d80 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe320eef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 0x7fe31c133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe320eef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 0x7fe31c133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60202/0 (socket says 172.21.15.32:60202) 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe320eef700 1 -- 172.21.15.32:0/428902577 learned_addr learned my addr 172.21.15.32:0/428902577 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe3216f0700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c09d630 0x7fe31c133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.016+0000 7fe321ef1700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c138d80 0x7fe31c134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe3216f0700 1 -- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c138d80 msgr2=0x7fe31c134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe3216f0700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c138d80 0x7fe31c134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe3216f0700 1 -- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 msgr2=0x7fe31c133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe3216f0700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 0x7fe31c133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe3216f0700 1 -- 172.21.15.32:0/428902577 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180094a0 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe320eef700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 0x7fe31c133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe3216f0700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c09d630 0x7fe31c133700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe31800ea10 tx=0x7fe31800ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800c850 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31800c9b0 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe314009a70 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800cd60 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.017+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1364f0 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.018+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe318013470 con 0x7fe31c09d630 2023-09-30T22:26:45.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.018+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe31803fb30 con 0x7fe31c09d630 2023-09-30T22:26:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.018+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c0747f0 con 0x7fe31c09d630 2023-09-30T22:26:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.023+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe31801b3b0 con 0x7fe31c09d630 2023-09-30T22:26:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.191+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcfc8074ea0 con 0x7fcfc8138dc0 2023-09-30T22:26:45.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.192+0000 7fcfc4ff9700 1 -- 172.21.15.32:0/3657118614 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfc002c020 con 0x7fcfc8138dc0 2023-09-30T22:26:45.193 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:45.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 msgr2=0x7fcfc812f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 0x7fcfc812f600 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0016000 tx=0x7fcfc0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 shutdown_connections 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfc8138dc0 0x7fcfc812f600 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcfc8133420 0x7fcfc812eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 --2- 172.21.15.32:0/3657118614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcfc81337f0 0x7fcfc812f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 >> 172.21.15.32:0/3657118614 conn(0x7fcfc80a4210 msgr2=0x7fcfc813aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 shutdown_connections 2023-09-30T22:26:45.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.194+0000 7fcfce9aa700 1 -- 172.21.15.32:0/3657118614 wait complete. 2023-09-30T22:26:45.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:45.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.229+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe31c072980 con 0x7fe31c09d630 2023-09-30T22:26:45.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.230+0000 7fe311ffb700 1 -- 172.21.15.32:0/428902577 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe31802b090 con 0x7fe31c09d630 2023-09-30T22:26:45.231 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c09d630 msgr2=0x7fe31c133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c09d630 0x7fe31c133700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe31800ea10 tx=0x7fe31800ed20 comp rx=0 tx=0).stop 2023-09-30T22:26:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 shutdown_connections 2023-09-30T22:26:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe31c138d80 0x7fe31c134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe31c09d630 0x7fe31c133700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 --2- 172.21.15.32:0/428902577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe31c0a0a30 0x7fe31c133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 >> 172.21.15.32:0/428902577 conn(0x7fe31c0a3050 msgr2=0x7fe31c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 shutdown_connections 2023-09-30T22:26:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.232+0000 7fe323954700 1 -- 172.21.15.32:0/428902577 wait complete. 2023-09-30T22:26:45.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:45.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:45.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.897+0000 7f298b775700 1 Processor -- start 2023-09-30T22:26:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f298b775700 1 -- start start 2023-09-30T22:26:45.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f298b775700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984138b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f298b775700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f298413b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f298b775700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298406d860 con 0x7f29840a0a30 2023-09-30T22:26:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f298b775700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298409eaf0 con 0x7f29840a1510 2023-09-30T22:26:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f298b775700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29840a27c0 con 0x7f2984139070 2023-09-30T22:26:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f2989d12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f298413b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f2989511700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29840a0a30 0x7f29840a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35404/0 (socket says 172.21.15.32:35404) 2023-09-30T22:26:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f2989511700 1 -- 172.21.15.32:0/1301377317 learned_addr learned my addr 172.21.15.32:0/1301377317 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.898+0000 7f2988d10700 1 --2- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984138b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2989d12700 1 -- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 msgr2=0x7f2984138b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2989d12700 1 --2- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984138b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2989d12700 1 -- 172.21.15.32:0/1301377317 >> v1:172.21.15.32:6789/0 conn(0x7f29840a0a30 legacy=0x7f29840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2989d12700 1 -- 172.21.15.32:0/1301377317 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298409dbe0 con 0x7f2984139070 2023-09-30T22:26:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2983fff700 1 -- 172.21.15.32:0/1301377317 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908791113 0 0) 0x7f298406d860 con 0x7f29840a0a30 2023-09-30T22:26:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2989d12700 1 --2- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f298413b450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f297c009870 tx=0x7f297c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1c7cf1482f26a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2983fff700 1 -- 172.21.15.32:0/1301377317 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c011830 con 0x7f2984139070 2023-09-30T22:26:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2983fff700 1 -- 172.21.15.32:0/1301377317 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c011990 con 0x7f2984139070 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.899+0000 7f2983fff700 1 -- 172.21.15.32:0/1301377317 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c011d20 con 0x7f2984139070 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 -- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 msgr2=0x7f298413b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 --2- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f298413b450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f297c009870 tx=0x7f297c009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 -- 172.21.15.32:0/1301377317 shutdown_connections 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 --2- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f298413b450 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 --2- 172.21.15.32:0/1301377317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984138b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 -- 172.21.15.32:0/1301377317 >> 172.21.15.32:0/1301377317 conn(0x7f29840a6550 msgr2=0x7f29840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 -- 172.21.15.32:0/1301377317 shutdown_connections 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.900+0000 7f298b775700 1 -- 172.21.15.32:0/1301377317 wait complete. 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.901+0000 7f298b775700 1 Processor -- start 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.901+0000 7f298b775700 1 -- start start 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.901+0000 7f298b775700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29840a0a30 0x7f2984120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.901+0000 7f298b775700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f298b775700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f29841210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f298b775700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984136530 con 0x7f29840a0a30 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f298b775700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2984124160 con 0x7f29840a1510 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f298b775700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2984122a70 con 0x7f2984139070 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2989511700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29840a0a30 0x7f2984120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2989d12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f29841210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2989511700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29840a0a30 0x7f2984120640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60210/0 (socket says 172.21.15.32:60210) 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2989d12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f29841210c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52518/0 (socket says 172.21.15.32:52518) 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2989511700 1 -- 172.21.15.32:0/819174901 learned_addr learned my addr 172.21.15.32:0/819174901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 -- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 msgr2=0x7f29841210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f29841210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 -- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29840a0a30 msgr2=0x7f2984120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29840a0a30 0x7f2984120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 -- 172.21.15.32:0/819174901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0094a0 con 0x7f29840a1510 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2988d10700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984120b80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f297800e900 tx=0x7f297800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978013680 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29780137e0 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978013b50 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.902+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29840a4bf0 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.903+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2984123e80 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.903+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2978013cb0 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.904+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f297803f650 con 0x7f29840a1510 2023-09-30T22:26:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.904+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298404f2a0 con 0x7f29840a1510 2023-09-30T22:26:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.909+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f297801bc10 con 0x7f29840a1510 2023-09-30T22:26:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.936+0000 7f18702b2700 1 Processor -- start 2023-09-30T22:26:45.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.936+0000 7f18702b2700 1 -- start start 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f18702b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f186809d630 0x7f18680a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f18702b2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1868138d80 0x7f186813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f18702b2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680a2a70 con 0x7f18680a0a30 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f18702b2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18680a01d0 con 0x7f186809d630 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f18702b2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186809e2b0 con 0x7f1868138d80 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d84d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f186809d630 0x7f18680a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186e84f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1868138d80 0x7f186813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186e04e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18680a0a30 0x7f18680a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35414/0 (socket says 172.21.15.32:35414) 2023-09-30T22:26:45.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186e04e700 1 -- 172.21.15.32:0/817050098 learned_addr learned my addr 172.21.15.32:0/817050098 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d84d700 1 -- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1868138d80 msgr2=0x7f186813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d84d700 1 --2- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1868138d80 0x7f186813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d84d700 1 -- 172.21.15.32:0/817050098 >> v1:172.21.15.32:6789/0 conn(0x7f18680a0a30 legacy=0x7f18680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d84d700 1 -- 172.21.15.32:0/817050098 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186809dc50 con 0x7f186809d630 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186e84f700 1 --2- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1868138d80 0x7f186813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d04c700 1 -- 172.21.15.32:0/817050098 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706456088 0 0) 0x7f18680a2a70 con 0x7f18680a0a30 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.937+0000 7f186d84d700 1 --2- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186809d630 0x7f18680a1510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1858009840 tx=0x7f1858009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b24b1a8dfc00db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f186d04c700 1 -- 172.21.15.32:0/817050098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011830 con 0x7f186809d630 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f186d04c700 1 -- 172.21.15.32:0/817050098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858011990 con 0x7f186809d630 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f186d04c700 1 -- 172.21.15.32:0/817050098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011d20 con 0x7f186809d630 2023-09-30T22:26:45.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 -- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186809d630 msgr2=0x7f18680a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 --2- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186809d630 0x7f18680a1510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1858009840 tx=0x7f1858009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 -- 172.21.15.32:0/817050098 shutdown_connections 2023-09-30T22:26:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 --2- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1868138d80 0x7f186813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 --2- 172.21.15.32:0/817050098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186809d630 0x7f18680a1510 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 -- 172.21.15.32:0/817050098 >> 172.21.15.32:0/817050098 conn(0x7f18680a3050 msgr2=0x7f18680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 -- 172.21.15.32:0/817050098 shutdown_connections 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.938+0000 7f18702b2700 1 -- 172.21.15.32:0/817050098 wait complete. 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.940+0000 7f18702b2700 1 Processor -- start 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.940+0000 7f18702b2700 1 -- start start 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f18702b2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186809d630 0x7f1868133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f18702b2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18680a0a30 0x7f1868133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f18702b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 0x7f1868134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f18702b2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1868136190 con 0x7f1868138d80 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f18702b2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1868136eb0 con 0x7f186809d630 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f18702b2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18680a4ba0 con 0x7f18680a0a30 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186e84f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 0x7f1868134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186e84f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 0x7f1868134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60232/0 (socket says 172.21.15.32:60232) 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186e84f700 1 -- 172.21.15.32:0/4279334169 learned_addr learned my addr 172.21.15.32:0/4279334169 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18680a0a30 0x7f1868133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186e04e700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186809d630 0x7f1868133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 -- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186809d630 msgr2=0x7f1868133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186809d630 0x7f1868133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 -- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 msgr2=0x7f1868134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 0x7f1868134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 -- 172.21.15.32:0/4279334169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600094a0 con 0x7f18680a0a30 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186e84f700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 0x7f1868134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f186d84d700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18680a0a30 0x7f1868133c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1858016000 tx=0x7f1858004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.941+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858004400 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.942+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185801f070 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.942+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580038a0 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.942+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18580094a0 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.942+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1868136540 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.942+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1858021030 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.943+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f18580407f0 con 0x7f18680a0a30 2023-09-30T22:26:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.943+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18680747f0 con 0x7f18680a0a30 2023-09-30T22:26:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:45.948+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1858004560 con 0x7f18680a0a30 2023-09-30T22:26:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.120+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2984123c70 con 0x7f29840a1510 2023-09-30T22:26:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.121+0000 7f2981ffb700 1 -- 172.21.15.32:0/819174901 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2978021030 con 0x7f29840a1510 2023-09-30T22:26:46.122 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 msgr2=0x7f2984120b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984120b80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f297800e900 tx=0x7f297800ec10 comp rx=0 tx=0).stop 2023-09-30T22:26:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 shutdown_connections 2023-09-30T22:26:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2984139070 0x7f29841210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29840a0a30 0x7f2984120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 --2- 172.21.15.32:0/819174901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29840a1510 0x7f2984120b80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 >> 172.21.15.32:0/819174901 conn(0x7f29840a6550 msgr2=0x7f29840a2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 shutdown_connections 2023-09-30T22:26:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.124+0000 7f298b775700 1 -- 172.21.15.32:0/819174901 wait complete. 2023-09-30T22:26:46.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.158+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1868137980 con 0x7f18680a0a30 2023-09-30T22:26:46.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.158+0000 7f185effd700 1 -- 172.21.15.32:0/4279334169 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1858015020 con 0x7f18680a0a30 2023-09-30T22:26:46.159 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:46.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18680a0a30 msgr2=0x7f1868133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18680a0a30 0x7f1868133c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1858016000 tx=0x7f1858004750 comp rx=0 tx=0).stop 2023-09-30T22:26:46.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 shutdown_connections 2023-09-30T22:26:46.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1868138d80 0x7f1868134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186809d630 0x7f1868133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 --2- 172.21.15.32:0/4279334169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18680a0a30 0x7f1868133c40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 >> 172.21.15.32:0/4279334169 conn(0x7f18680a3050 msgr2=0x7f186809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 shutdown_connections 2023-09-30T22:26:46.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.161+0000 7f18702b2700 1 -- 172.21.15.32:0/4279334169 wait complete. 2023-09-30T22:26:46.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:46.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:46.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.356+0000 7f34e081c700 1 Processor -- start 2023-09-30T22:26:46.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.356+0000 7f34e081c700 1 -- start start 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34e081c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34e081c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc0a1340 0x7f34dc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34e081c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc09edd0 con 0x7f34dc133fa0 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34e081c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc09e1f0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34e081c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc09dc50 con 0x7f34dc0a1340 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34d9d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc0a1340 0x7f34dc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34da59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34dad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34dc133fa0 0x7f34dc13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35418/0 (socket says 172.21.15.32:35418) 2023-09-30T22:26:46.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34dad9d700 1 -- 172.21.15.32:0/1751857314 learned_addr learned my addr 172.21.15.32:0/1751857314 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.357+0000 7f34d9d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc0a1340 0x7f34dc133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52544/0 (socket says 172.21.15.32:52544) 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34da59c700 1 -- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc0a1340 msgr2=0x7f34dc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34da59c700 1 --2- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc0a1340 0x7f34dc133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34da59c700 1 -- 172.21.15.32:0/1751857314 >> v1:172.21.15.32:6789/0 conn(0x7f34dc133fa0 legacy=0x7f34dc13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34da59c700 1 -- 172.21.15.32:0/1751857314 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0a1fe0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34d959a700 1 -- 172.21.15.32:0/1751857314 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169794236 0 0) 0x7f34dc09edd0 con 0x7f34dc133fa0 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34da59c700 1 --2- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc0a0e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a4c0dbd253cc90 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34d959a700 1 -- 172.21.15.32:0/1751857314 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc011830 con 0x7f34dc0a0a30 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34d959a700 1 -- 172.21.15.32:0/1751857314 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc011990 con 0x7f34dc0a0a30 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.358+0000 7f34d959a700 1 -- 172.21.15.32:0/1751857314 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc011d20 con 0x7f34dc0a0a30 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 -- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 msgr2=0x7f34dc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 --2- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc0a0e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 -- 172.21.15.32:0/1751857314 shutdown_connections 2023-09-30T22:26:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 --2- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc0a0e00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 --2- 172.21.15.32:0/1751857314 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc0a1340 0x7f34dc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 -- 172.21.15.32:0/1751857314 >> 172.21.15.32:0/1751857314 conn(0x7f34dc0a6550 msgr2=0x7f34dc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 -- 172.21.15.32:0/1751857314 shutdown_connections 2023-09-30T22:26:46.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.359+0000 7f34e081c700 1 -- 172.21.15.32:0/1751857314 wait complete. 2023-09-30T22:26:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.360+0000 7f34e081c700 1 Processor -- start 2023-09-30T22:26:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.360+0000 7f34e081c700 1 -- start start 2023-09-30T22:26:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34e081c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc1c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34e081c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 0x7f34dc1c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34e081c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc133fa0 0x7f34dc1c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34e081c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc126370 con 0x7f34dc0a1340 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34e081c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc124660 con 0x7f34dc0a0a30 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34e081c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc124c10 con 0x7f34dc133fa0 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34d9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 0x7f34dc1c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34da59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc1c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34d9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 0x7f34dc1c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60248/0 (socket says 172.21.15.32:60248) 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34d9d9b700 1 -- 172.21.15.32:0/1301849662 learned_addr learned my addr 172.21.15.32:0/1301849662 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34dad9d700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc133fa0 0x7f34dc1c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34da59c700 1 -- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc133fa0 msgr2=0x7f34dc1c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34da59c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc133fa0 0x7f34dc1c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34da59c700 1 -- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 msgr2=0x7f34dc1c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34da59c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 0x7f34dc1c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34da59c700 1 -- 172.21.15.32:0/1301849662 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0094a0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.361+0000 7f34d9d9b700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 0x7f34dc1c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.362+0000 7f34da59c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc1c8380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f34cc016000 tx=0x7f34cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.362+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc004510 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.362+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc01f070 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.362+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc0a25e0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.362+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0038e0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.362+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc0a0220 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.363+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34cc003a40 con 0x7f34dc0a0a30 2023-09-30T22:26:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.363+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f34cc021070 con 0x7f34dc0a0a30 2023-09-30T22:26:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.363+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc074ea0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.368+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34cc02b080 con 0x7f34dc0a0a30 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 Processor -- start 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 -- start start 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59400a0a30 0x7f59400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59400a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5940133930 0x7f5940133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400a2a70 con 0x7f5940133930 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400a01d0 con 0x7f59400a0a30 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f5945c98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594009e2b0 con 0x7f594009d630 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f593ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5940133930 0x7f5940133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f593effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59400a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f593ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5940133930 0x7f5940133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60262/0 (socket says 172.21.15.32:60262) 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f593ffff700 1 -- 172.21.15.32:0/1037318301 learned_addr learned my addr 172.21.15.32:0/1037318301 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.399+0000 7f593f7fe700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59400a0a30 0x7f59400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593ffff700 1 -- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 msgr2=0x7f59400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593ffff700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59400a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593ffff700 1 -- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59400a0a30 msgr2=0x7f59400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593ffff700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59400a0a30 0x7f59400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593ffff700 1 -- 172.21.15.32:0/1037318301 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594009dc50 con 0x7f5940133930 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593effd700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59400a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593f7fe700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59400a0a30 0x7f59400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593ffff700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5940133930 0x7f5940133dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f59380098f0 tx=0x7f5938009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e52f581979d92df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593e7fc700 1 -- 172.21.15.32:0/1037318301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938011830 con 0x7f5940133930 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593e7fc700 1 -- 172.21.15.32:0/1037318301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5938011990 con 0x7f5940133930 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.400+0000 7f593e7fc700 1 -- 172.21.15.32:0/1037318301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938011d20 con 0x7f5940133930 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 -- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5940133930 msgr2=0x7f5940133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5940133930 0x7f5940133dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f59380098f0 tx=0x7f5938009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 -- 172.21.15.32:0/1037318301 shutdown_connections 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59400a0a30 0x7f59400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5940133930 0x7f5940133dd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 --2- 172.21.15.32:0/1037318301 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59400a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 -- 172.21.15.32:0/1037318301 >> 172.21.15.32:0/1037318301 conn(0x7f59400a3050 msgr2=0x7f59400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 -- 172.21.15.32:0/1037318301 shutdown_connections 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.401+0000 7f5945c98700 1 -- 172.21.15.32:0/1037318301 wait complete. 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.402+0000 7f5945c98700 1 Processor -- start 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 -- start start 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59401c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59400a0a30 0x7f59401c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 0x7f59401c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5940121ed0 con 0x7f59400a0a30 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400a4030 con 0x7f5940133930 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f5945c98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5940120670 con 0x7f594009d630 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59401c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f593ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 0x7f59401c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f593ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 0x7f59401c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46504/0 (socket says 172.21.15.32:46504) 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f593ffff700 1 -- 172.21.15.32:0/1576791836 learned_addr learned my addr 172.21.15.32:0/1576791836 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59401c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52566/0 (socket says 172.21.15.32:52566) 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.403+0000 7f593effd700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59400a0a30 0x7f59401c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593ffff700 1 -- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 msgr2=0x7f59401c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593ffff700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59401c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593ffff700 1 -- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59400a0a30 msgr2=0x7f59401c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593ffff700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59400a0a30 0x7f59401c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593ffff700 1 -- 172.21.15.32:0/1576791836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340094a0 con 0x7f5940133930 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593f7fe700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59401c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593effd700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59400a0a30 0x7f59401c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f593ffff700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 0x7f59401c8990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59400a2bb0 tx=0x7f5938004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938004450 con 0x7f5940133930 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59380045b0 con 0x7f5940133930 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5930009a70 con 0x7f5940133930 2023-09-30T22:26:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59380038e0 con 0x7f5940133930 2023-09-30T22:26:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.404+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5938009550 con 0x7f5940133930 2023-09-30T22:26:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.405+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5938003a40 con 0x7f5940133930 2023-09-30T22:26:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.405+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5938040a10 con 0x7f5940133930 2023-09-30T22:26:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.406+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400747f0 con 0x7f5940133930 2023-09-30T22:26:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.410+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5938010380 con 0x7f5940133930 2023-09-30T22:26:46.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.573+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f34dc0a28e0 con 0x7f34dc0a0a30 2023-09-30T22:26:46.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.574+0000 7f34caffd700 1 -- 172.21.15.32:0/1301849662 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34cc015020 con 0x7f34dc0a0a30 2023-09-30T22:26:46.575 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 msgr2=0x7f34dc1c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc1c8380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f34cc016000 tx=0x7f34cc004750 comp rx=0 tx=0).stop 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 shutdown_connections 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc133fa0 0x7f34dc1c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc0a0a30 0x7f34dc1c8380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 --2- 172.21.15.32:0/1301849662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc0a1340 0x7f34dc1c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 >> 172.21.15.32:0/1301849662 conn(0x7f34dc0a6550 msgr2=0x7f34dc0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 shutdown_connections 2023-09-30T22:26:46.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.576+0000 7f34e081c700 1 -- 172.21.15.32:0/1301849662 wait complete. 2023-09-30T22:26:46.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:46.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.618+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5940072980 con 0x7f5940133930 2023-09-30T22:26:46.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.619+0000 7f5927fff700 1 -- 172.21.15.32:0/1576791836 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5938015020 con 0x7f5940133930 2023-09-30T22:26:46.620 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:46.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 msgr2=0x7f59401c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 0x7f59401c8990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f59400a2bb0 tx=0x7f5938004750 comp rx=0 tx=0).stop 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 shutdown_connections 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5940133930 0x7f59401c8990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594009d630 0x7f59401c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 --2- 172.21.15.32:0/1576791836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59400a0a30 0x7f59401c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 >> 172.21.15.32:0/1576791836 conn(0x7f59400a3050 msgr2=0x7f594009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 shutdown_connections 2023-09-30T22:26:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:46.621+0000 7f5945c98700 1 -- 172.21.15.32:0/1576791836 wait complete. 2023-09-30T22:26:46.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:47.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:47.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 Processor -- start 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 -- start start 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809e1d0 0x7fc908138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 0x7fc90813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90809b120 con 0x7fc90809d6f0 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90806b460 con 0x7fc908139170 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.290+0000 7fc90e6d2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc90809a7e0 con 0x7fc90809e1d0 2023-09-30T22:26:47.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc90cc6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 0x7fc90813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc9077fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809e1d0 0x7fc908138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc90cc6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 0x7fc90813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60288/0 (socket says 172.21.15.32:60288) 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc90cc6f700 1 -- 172.21.15.32:0/2249899848 learned_addr learned my addr 172.21.15.32:0/2249899848 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc907fff700 1 --1- 172.21.15.32:0/2249899848 >> v1:172.21.15.32:6789/0 conn(0x7fc90809d6f0 0x7fc90809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35434/0 (socket says 172.21.15.32:35434) 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc9077fe700 1 -- 172.21.15.32:0/2249899848 >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 msgr2=0x7fc90813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc9077fe700 1 --2- 172.21.15.32:0/2249899848 >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 0x7fc90813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc9077fe700 1 -- 172.21.15.32:0/2249899848 >> v1:172.21.15.32:6789/0 conn(0x7fc90809d6f0 legacy=0x7fc90809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc9077fe700 1 -- 172.21.15.32:0/2249899848 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90809a500 con 0x7fc90809e1d0 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc906ffd700 1 -- 172.21.15.32:0/2249899848 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002474879 0 0) 0x7fc90809b120 con 0x7fc90809d6f0 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc90cc6f700 1 --2- 172.21.15.32:0/2249899848 >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 0x7fc90813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc9077fe700 1 --2- 172.21.15.32:0/2249899848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809e1d0 0x7fc908138c30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009870 tx=0x7fc8fc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c148eb6aaf8ee309 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc906ffd700 1 -- 172.21.15.32:0/2249899848 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc011830 con 0x7fc90809e1d0 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc906ffd700 1 -- 172.21.15.32:0/2249899848 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc011990 con 0x7fc90809e1d0 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.291+0000 7fc906ffd700 1 -- 172.21.15.32:0/2249899848 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc011d20 con 0x7fc90809e1d0 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 -- 172.21.15.32:0/2249899848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809e1d0 msgr2=0x7fc908138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/2249899848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809e1d0 0x7fc908138c30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009870 tx=0x7fc8fc009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 -- 172.21.15.32:0/2249899848 shutdown_connections 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/2249899848 >> v2:172.21.15.32:3300/0 conn(0x7fc908139170 0x7fc90813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/2249899848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809e1d0 0x7fc908138c30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 -- 172.21.15.32:0/2249899848 >> 172.21.15.32:0/2249899848 conn(0x7fc908066f50 msgr2=0x7fc90809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 -- 172.21.15.32:0/2249899848 shutdown_connections 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.292+0000 7fc90e6d2700 1 -- 172.21.15.32:0/2249899848 wait complete. 2023-09-30T22:26:47.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.293+0000 7fc90e6d2700 1 Processor -- start 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.293+0000 7fc90e6d2700 1 -- start start 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90e6d2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 0x7fc90811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90e6d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e1d0 0x7fc90811eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90e6d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc908139170 0x7fc90811f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90e6d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908072110 con 0x7fc90809e1d0 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90e6d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc908122100 con 0x7fc908139170 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90e6d2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc908120a80 con 0x7fc90809d6f0 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 0x7fc90811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 0x7fc90811e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52586/0 (socket says 172.21.15.32:52586) 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 -- 172.21.15.32:0/618003535 learned_addr learned my addr 172.21.15.32:0/618003535 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc9077fe700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e1d0 0x7fc90811eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc90cc6f700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc908139170 0x7fc90811f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 -- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc908139170 msgr2=0x7fc90811f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc908139170 0x7fc90811f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 -- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e1d0 msgr2=0x7fc90811eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e1d0 0x7fc90811eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.294+0000 7fc907fff700 1 -- 172.21.15.32:0/618003535 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009b30 con 0x7fc90809d6f0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.295+0000 7fc907fff700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 0x7fc90811e960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c520 tx=0x7fc8f800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.295+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8004c80 con 0x7fc90809d6f0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.295+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f8004e00 con 0x7fc90809d6f0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.295+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc0094a0 con 0x7fc90809d6f0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.295+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8007310 con 0x7fc90809d6f0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.295+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc908120cc0 con 0x7fc90809d6f0 2023-09-30T22:26:47.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.296+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc8f8007470 con 0x7fc90809d6f0 2023-09-30T22:26:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.296+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc8f803f300 con 0x7fc90809d6f0 2023-09-30T22:26:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.296+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc908134810 con 0x7fc90809d6f0 2023-09-30T22:26:47.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.301+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8f80123b0 con 0x7fc90809d6f0 2023-09-30T22:26:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 Processor -- start 2023-09-30T22:26:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 -- start start 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 0x7f61240a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 0x7f61240a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6124133920 0x7f6124133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61240a1810 con 0x7f61240a09c0 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612409ea10 con 0x7f6124133920 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f61291de700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f612409e210 con 0x7f612409d5c0 2023-09-30T22:26:47.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f612259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 0x7f61240a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f6122d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 0x7f61240a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f6122d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 0x7f61240a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60312/0 (socket says 172.21.15.32:60312) 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f6122d9d700 1 -- 172.21.15.32:0/417729773 learned_addr learned my addr 172.21.15.32:0/417729773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f612359e700 1 --2- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6124133920 0x7f6124133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.328+0000 7f612259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 0x7f61240a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52590/0 (socket says 172.21.15.32:52590) 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f612259c700 1 -- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6124133920 msgr2=0x7f6124133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f612259c700 1 --2- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6124133920 0x7f6124133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f612259c700 1 -- 172.21.15.32:0/417729773 >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 msgr2=0x7f61240a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f612259c700 1 --2- 172.21.15.32:0/417729773 >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 0x7f61240a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f612259c700 1 -- 172.21.15.32:0/417729773 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61240a26f0 con 0x7f612409d5c0 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f6122d9d700 1 --2- 172.21.15.32:0/417729773 >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 0x7f61240a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f612259c700 1 --2- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 0x7f61240a12d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6118009840 tx=0x7f6118009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dceb2e5434b3502 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f6121d9b700 1 -- 172.21.15.32:0/417729773 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118011830 con 0x7f612409d5c0 2023-09-30T22:26:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f6121d9b700 1 -- 172.21.15.32:0/417729773 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118011990 con 0x7f612409d5c0 2023-09-30T22:26:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.329+0000 7f6121d9b700 1 -- 172.21.15.32:0/417729773 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118011d20 con 0x7f612409d5c0 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 -- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 msgr2=0x7f61240a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 --2- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 0x7f61240a12d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6118009840 tx=0x7f6118009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 -- 172.21.15.32:0/417729773 shutdown_connections 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 --2- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6124133920 0x7f6124133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 --2- 172.21.15.32:0/417729773 >> v2:172.21.15.32:3300/0 conn(0x7f61240a09c0 0x7f61240a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 --2- 172.21.15.32:0/417729773 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f612409d5c0 0x7f61240a12d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 -- 172.21.15.32:0/417729773 >> 172.21.15.32:0/417729773 conn(0x7f61240a2fe0 msgr2=0x7f61240a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 -- 172.21.15.32:0/417729773 shutdown_connections 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.330+0000 7f61291de700 1 -- 172.21.15.32:0/417729773 wait complete. 2023-09-30T22:26:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.332+0000 7f61291de700 1 Processor -- start 2023-09-30T22:26:47.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.332+0000 7f61291de700 1 -- start start 2023-09-30T22:26:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f61291de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f612409d5c0 0x7f61241215a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f61291de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 0x7f6124121ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f61291de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6124133920 0x7f6124122020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f61291de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124124b20 con 0x7f61240a09c0 2023-09-30T22:26:47.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f61291de700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6124132130 con 0x7f612409d5c0 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f61291de700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61240a59b0 con 0x7f6124133920 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 0x7f6124121ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612359e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6124133920 0x7f6124122020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 0x7f6124121ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60318/0 (socket says 172.21.15.32:60318) 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612259c700 1 -- 172.21.15.32:0/921025330 learned_addr learned my addr 172.21.15.32:0/921025330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f6122d9d700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f612409d5c0 0x7f61241215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612359e700 1 -- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f612409d5c0 msgr2=0x7f61241215a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612359e700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f612409d5c0 0x7f61241215a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612359e700 1 -- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 msgr2=0x7f6124121ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612359e700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 0x7f6124121ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612359e700 1 -- 172.21.15.32:0/921025330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114009550 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.333+0000 7f612259c700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 0x7f6124121ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.334+0000 7f612359e700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6124133920 0x7f6124122020 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f611c009d90 tx=0x7f611c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.334+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c009c00 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.334+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c00f040 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.334+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c014480 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.334+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61180094a0 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.334+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612409cec0 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.335+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f611c0145e0 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.335+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f611c0401b0 con 0x7f6124133920 2023-09-30T22:26:47.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.335+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61240a1540 con 0x7f6124133920 2023-09-30T22:26:47.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.340+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f611c018020 con 0x7f6124133920 2023-09-30T22:26:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.509+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc908074ea0 con 0x7fc90809d6f0 2023-09-30T22:26:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.509+0000 7fc904ff9700 1 -- 172.21.15.32:0/618003535 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8f8011360 con 0x7fc90809d6f0 2023-09-30T22:26:47.510 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 msgr2=0x7fc90811e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 0x7fc90811e960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c520 tx=0x7fc8f800c830 comp rx=0 tx=0).stop 2023-09-30T22:26:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 shutdown_connections 2023-09-30T22:26:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc908139170 0x7fc90811f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809d6f0 0x7fc90811e960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 --2- 172.21.15.32:0/618003535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e1d0 0x7fc90811eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 >> 172.21.15.32:0/618003535 conn(0x7fc908066f50 msgr2=0x7fc908136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 shutdown_connections 2023-09-30T22:26:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.511+0000 7fc90e6d2700 1 -- 172.21.15.32:0/618003535 wait complete. 2023-09-30T22:26:47.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:47.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.545+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f61240a4cb0 con 0x7f6124133920 2023-09-30T22:26:47.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.545+0000 7f610b7fe700 1 -- 172.21.15.32:0/921025330 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f611c02b090 con 0x7f6124133920 2023-09-30T22:26:47.547 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:47.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6124133920 msgr2=0x7f6124122020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6124133920 0x7f6124122020 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f611c009d90 tx=0x7f611c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 shutdown_connections 2023-09-30T22:26:47.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6124133920 0x7f6124122020 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f612409d5c0 0x7f61241215a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 --2- 172.21.15.32:0/921025330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61240a09c0 0x7f6124121ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 >> 172.21.15.32:0/921025330 conn(0x7f61240a2fe0 msgr2=0x7f6124133200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 shutdown_connections 2023-09-30T22:26:47.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.548+0000 7f61291de700 1 -- 172.21.15.32:0/921025330 wait complete. 2023-09-30T22:26:47.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:47.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:47.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:47.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 Processor -- start 2023-09-30T22:26:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 -- start start 2023-09-30T22:26:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc409d630 0x7f8cc40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8cc4138d80 0x7f8cc413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc40a2a70 con 0x7f8cc40a0a30 2023-09-30T22:26:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc40a01d0 con 0x7f8cc4138d80 2023-09-30T22:26:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cca819700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc409e2b0 con 0x7f8cc409d630 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cc37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc409d630 0x7f8cc40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cc3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8cc40a0a30 0x7f8cc40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35440/0 (socket says 172.21.15.32:35440) 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cc3fff700 1 -- 172.21.15.32:0/1658997511 learned_addr learned my addr 172.21.15.32:0/1658997511 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.745+0000 7f8cc8db6700 1 --2- 172.21.15.32:0/1658997511 >> v2:172.21.15.32:3300/0 conn(0x7f8cc4138d80 0x7f8cc413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc8db6700 1 -- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc409d630 msgr2=0x7f8cc40a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc8db6700 1 --2- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc409d630 0x7f8cc40a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc8db6700 1 -- 172.21.15.32:0/1658997511 >> v1:172.21.15.32:6789/0 conn(0x7f8cc40a0a30 legacy=0x7f8cc40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc8db6700 1 -- 172.21.15.32:0/1658997511 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc409dc50 con 0x7f8cc4138d80 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc2ffd700 1 -- 172.21.15.32:0/1658997511 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831301793 0 0) 0x7f8cc40a2a70 con 0x7f8cc40a0a30 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc37fe700 1 --2- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc409d630 0x7f8cc40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc8db6700 1 --2- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc4138d80 0x7f8cc413b160 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009d90 tx=0x7f8cbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b04be0bdcd8b0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc2ffd700 1 -- 172.21.15.32:0/1658997511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00e070 con 0x7f8cc4138d80 2023-09-30T22:26:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc2ffd700 1 -- 172.21.15.32:0/1658997511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc00bb20 con 0x7f8cc4138d80 2023-09-30T22:26:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.746+0000 7f8cc2ffd700 1 -- 172.21.15.32:0/1658997511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc012430 con 0x7f8cc4138d80 2023-09-30T22:26:47.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 -- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc4138d80 msgr2=0x7f8cc413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 --2- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc4138d80 0x7f8cc413b160 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009d90 tx=0x7f8cbc00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:47.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 -- 172.21.15.32:0/1658997511 shutdown_connections 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 --2- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc4138d80 0x7f8cc413b160 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 --2- 172.21.15.32:0/1658997511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc409d630 0x7f8cc40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 -- 172.21.15.32:0/1658997511 >> 172.21.15.32:0/1658997511 conn(0x7f8cc40a3050 msgr2=0x7f8cc40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 -- 172.21.15.32:0/1658997511 shutdown_connections 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.747+0000 7f8cca819700 1 -- 172.21.15.32:0/1658997511 wait complete. 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.748+0000 7f8cca819700 1 Processor -- start 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 -- start start 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 0x7f8cc412c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cc40a0a30 0x7f8cc412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 0x7f8cc412cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4130190 con 0x7f8cc409d630 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc412fc10 con 0x7f8cc40a0a30 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cca819700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40a0410 con 0x7f8cc4138d80 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cc3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 0x7f8cc412c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cc8db6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 0x7f8cc412cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cc3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 0x7f8cc412c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60338/0 (socket says 172.21.15.32:60338) 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cc8db6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 0x7f8cc412cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52600/0 (socket says 172.21.15.32:52600) 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cc3fff700 1 -- 172.21.15.32:0/2762267947 learned_addr learned my addr 172.21.15.32:0/2762267947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.749+0000 7f8cc37fe700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cc40a0a30 0x7f8cc412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc37fe700 1 -- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 msgr2=0x7f8cc412cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc37fe700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 0x7f8cc412cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc37fe700 1 -- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 msgr2=0x7f8cc412c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc37fe700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 0x7f8cc412c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc37fe700 1 -- 172.21.15.32:0/2762267947 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80094a0 con 0x7f8cc40a0a30 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc3fff700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 0x7f8cc412c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc8db6700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 0x7f8cc412cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc37fe700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cc40a0a30 0x7f8cc412cab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e9b0 tx=0x7f8cb800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ee50 con 0x7f8cc40a0a30 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800c850 con 0x7f8cc40a0a30 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc009a70 con 0x7f8cc40a0a30 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800cc00 con 0x7f8cc40a0a30 2023-09-30T22:26:47.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.750+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc412e5a0 con 0x7f8cc40a0a30 2023-09-30T22:26:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.751+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8cb8013470 con 0x7f8cc40a0a30 2023-09-30T22:26:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.751+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8cb803f750 con 0x7f8cc40a0a30 2023-09-30T22:26:47.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.752+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc40747f0 con 0x7f8cc40a0a30 2023-09-30T22:26:47.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.756+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8cb8013720 con 0x7f8cc40a0a30 2023-09-30T22:26:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.787+0000 7f3876a40700 1 Processor -- start 2023-09-30T22:26:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3876a40700 1 -- start start 2023-09-30T22:26:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3876a40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f38700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3876a40700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f3870133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3876a40700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387009edd0 con 0x7f3870133fa0 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3876a40700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387009e1f0 con 0x7f38700a0a30 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3876a40700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f387009dc50 con 0x7f38700a1340 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f3870133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f3870133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52608/0 (socket says 172.21.15.32:52608) 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f386ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f38700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f386f7fe700 1 -- 172.21.15.32:0/1732799930 learned_addr learned my addr 172.21.15.32:0/1732799930 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.788+0000 7f3874fdd700 1 --1- 172.21.15.32:0/1732799930 >> v1:172.21.15.32:6789/0 conn(0x7f3870133fa0 0x7f387013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35448/0 (socket says 172.21.15.32:35448) 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386f7fe700 1 -- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 msgr2=0x7f38700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386f7fe700 1 --2- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f38700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386f7fe700 1 -- 172.21.15.32:0/1732799930 >> v1:172.21.15.32:6789/0 conn(0x7f3870133fa0 legacy=0x7f387013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386f7fe700 1 -- 172.21.15.32:0/1732799930 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700a1fe0 con 0x7f38700a1340 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386effd700 1 -- 172.21.15.32:0/1732799930 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338661800 0 0) 0x7f387009edd0 con 0x7f3870133fa0 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386ffff700 1 --2- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f38700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386f7fe700 1 --2- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f3870133a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3864009920 tx=0x7f3864009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4247e3877237249b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386effd700 1 -- 172.21.15.32:0/1732799930 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864011830 con 0x7f38700a1340 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386effd700 1 -- 172.21.15.32:0/1732799930 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864011990 con 0x7f38700a1340 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.789+0000 7f386effd700 1 -- 172.21.15.32:0/1732799930 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864011d20 con 0x7f38700a1340 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 -- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 msgr2=0x7f3870133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 --2- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f3870133a60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3864009920 tx=0x7f3864009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 -- 172.21.15.32:0/1732799930 shutdown_connections 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 --2- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f38700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 --2- 172.21.15.32:0/1732799930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f3870133a60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 -- 172.21.15.32:0/1732799930 >> 172.21.15.32:0/1732799930 conn(0x7f38700a6550 msgr2=0x7f38700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 -- 172.21.15.32:0/1732799930 shutdown_connections 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.790+0000 7f3876a40700 1 -- 172.21.15.32:0/1732799930 wait complete. 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.791+0000 7f3876a40700 1 Processor -- start 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 -- start start 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f387011e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f387011eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3870133fa0 0x7f387011f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387011fb00 con 0x7f3870133fa0 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3870132eb0 con 0x7f38700a0a30 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3876a40700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3870121b10 con 0x7f38700a1340 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f387011eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f386f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f387011eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52616/0 (socket says 172.21.15.32:52616) 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f3874fdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3870133fa0 0x7f387011f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f386f7fe700 1 -- 172.21.15.32:0/1875932387 learned_addr learned my addr 172.21.15.32:0/1875932387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.792+0000 7f386ffff700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f387011e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386f7fe700 1 -- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 msgr2=0x7f387011e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386f7fe700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f387011e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386f7fe700 1 -- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3870133fa0 msgr2=0x7f387011f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386f7fe700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3870133fa0 0x7f387011f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386f7fe700 1 -- 172.21.15.32:0/1875932387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600094a0 con 0x7f38700a1340 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386f7fe700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f387011eb70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3864000c00 tx=0x7f3864004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864004450 con 0x7f38700a1340 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38640045b0 con 0x7f38700a1340 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640038c0 con 0x7f38700a1340 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3864009550 con 0x7f38700a1340 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.793+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38700a25e0 con 0x7f38700a1340 2023-09-30T22:26:47.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.794+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3864003a20 con 0x7f38700a1340 2023-09-30T22:26:47.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.794+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3870074ea0 con 0x7f38700a1340 2023-09-30T22:26:47.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.794+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3864021030 con 0x7f38700a1340 2023-09-30T22:26:47.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.803+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3864010380 con 0x7f38700a1340 2023-09-30T22:26:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.971+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8cc40a4a70 con 0x7f8cc40a0a30 2023-09-30T22:26:47.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.971+0000 7f8cc0ff9700 1 -- 172.21.15.32:0/2762267947 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8cb8017020 con 0x7f8cc40a0a30 2023-09-30T22:26:47.973 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cc40a0a30 msgr2=0x7f8cc412cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cc40a0a30 0x7f8cc412cab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e9b0 tx=0x7f8cb800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 shutdown_connections 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cc4138d80 0x7f8cc412cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cc409d630 0x7f8cc412c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 --2- 172.21.15.32:0/2762267947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cc40a0a30 0x7f8cc412cab0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 >> 172.21.15.32:0/2762267947 conn(0x7f8cc40a3050 msgr2=0x7f8cc409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 shutdown_connections 2023-09-30T22:26:47.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:47.974+0000 7f8cca819700 1 -- 172.21.15.32:0/2762267947 wait complete. 2023-09-30T22:26:47.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:48.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.008+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3870121d30 con 0x7f38700a1340 2023-09-30T22:26:48.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.009+0000 7f386cff9700 1 -- 172.21.15.32:0/1875932387 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3870121d30 con 0x7f38700a1340 2023-09-30T22:26:48.010 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:48.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 msgr2=0x7f387011eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f387011eb70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3864000c00 tx=0x7f3864004750 comp rx=0 tx=0).stop 2023-09-30T22:26:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 shutdown_connections 2023-09-30T22:26:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3870133fa0 0x7f387011f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38700a0a30 0x7f387011e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 --2- 172.21.15.32:0/1875932387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38700a1340 0x7f387011eb70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 >> 172.21.15.32:0/1875932387 conn(0x7f38700a6550 msgr2=0x7f38701b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 shutdown_connections 2023-09-30T22:26:48.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.011+0000 7f3876a40700 1 -- 172.21.15.32:0/1875932387 wait complete. 2023-09-30T22:26:48.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:48.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:48.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.675+0000 7f1fc9926700 1 Processor -- start 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.675+0000 7f1fc9926700 1 -- start start 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc9926700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fc409d630 0x7f1fc40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc9926700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 0x7f1fc413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc9926700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc40a2a70 con 0x7f1fc40a0a30 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc9926700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc40a01d0 con 0x7f1fc409d630 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc9926700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc409e2b0 con 0x7f1fc4138d80 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 0x7f1fc413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fc409d630 0x7f1fc40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 0x7f1fc413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52626/0 (socket says 172.21.15.32:52626) 2023-09-30T22:26:48.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc37fe700 1 -- 172.21.15.32:0/601187568 learned_addr learned my addr 172.21.15.32:0/601187568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc27fc700 1 -- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 msgr2=0x7f1fc413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc27fc700 1 --2- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 0x7f1fc413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc27fc700 1 -- 172.21.15.32:0/601187568 >> v1:172.21.15.32:6789/0 conn(0x7f1fc40a0a30 legacy=0x7f1fc40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc27fc700 1 -- 172.21.15.32:0/601187568 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc409dc50 con 0x7f1fc409d630 2023-09-30T22:26:48.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc1ffb700 1 -- 172.21.15.32:0/601187568 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439357657 0 0) 0x7f1fc40a2a70 con 0x7f1fc40a0a30 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.676+0000 7f1fc37fe700 1 --2- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 0x7f1fc413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc27fc700 1 --2- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc409d630 0x7f1fc40a1510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009840 tx=0x7f1fb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3809f9cccdb6d90a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc1ffb700 1 -- 172.21.15.32:0/601187568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8011830 con 0x7f1fc409d630 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc1ffb700 1 -- 172.21.15.32:0/601187568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb8011990 con 0x7f1fc409d630 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc1ffb700 1 -- 172.21.15.32:0/601187568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8011d20 con 0x7f1fc409d630 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc9926700 1 -- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc409d630 msgr2=0x7f1fc40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc9926700 1 --2- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc409d630 0x7f1fc40a1510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009840 tx=0x7f1fb8009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc9926700 1 -- 172.21.15.32:0/601187568 shutdown_connections 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc9926700 1 --2- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc4138d80 0x7f1fc413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc9926700 1 --2- 172.21.15.32:0/601187568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc409d630 0x7f1fc40a1510 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.677+0000 7f1fc9926700 1 -- 172.21.15.32:0/601187568 >> 172.21.15.32:0/601187568 conn(0x7f1fc40a3050 msgr2=0x7f1fc40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.678+0000 7f1fc9926700 1 -- 172.21.15.32:0/601187568 shutdown_connections 2023-09-30T22:26:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.678+0000 7f1fc9926700 1 -- 172.21.15.32:0/601187568 wait complete. 2023-09-30T22:26:48.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.678+0000 7f1fc9926700 1 Processor -- start 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 -- start start 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc409d630 0x7f1fc4133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 0x7f1fc4133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc4138d80 0x7f1fc4134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4136190 con 0x7f1fc4138d80 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4136eb0 con 0x7f1fc40a0a30 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc9926700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40a4ba0 con 0x7f1fc409d630 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 0x7f1fc4133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc409d630 0x7f1fc4133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc4138d80 0x7f1fc4134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 0x7f1fc4133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46576/0 (socket says 172.21.15.32:46576) 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc409d630 0x7f1fc4133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52638/0 (socket says 172.21.15.32:52638) 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.679+0000 7f1fc27fc700 1 -- 172.21.15.32:0/949677953 learned_addr learned my addr 172.21.15.32:0/949677953 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc37fe700 1 -- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc409d630 msgr2=0x7f1fc4133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc37fe700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc409d630 0x7f1fc4133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc37fe700 1 -- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 msgr2=0x7f1fc4133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc37fe700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 0x7f1fc4133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc37fe700 1 -- 172.21.15.32:0/949677953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009b30 con 0x7f1fc4138d80 2023-09-30T22:26:48.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc27fc700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 0x7f1fc4133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc37fe700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc4138d80 0x7f1fc4134180 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00efc0 tx=0x7f1fbc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00ca80 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc00f040 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00ce20 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb80094a0 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.680+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc4136540 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.681+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1fbc014480 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.681+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1fbc03fbd0 con 0x7f1fc4138d80 2023-09-30T22:26:48.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.682+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc40747f0 con 0x7f1fc4138d80 2023-09-30T22:26:48.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.687+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1fbc012930 con 0x7f1fc4138d80 2023-09-30T22:26:48.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.714+0000 7f70e048d700 1 Processor -- start 2023-09-30T22:26:48.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.714+0000 7f70e048d700 1 -- start start 2023-09-30T22:26:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70e048d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70e048d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 0x7f70d8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70e048d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70e048d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d81313c0 con 0x7f70d8099a40 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70e048d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d806e8c0 con 0x7f70d809a3b0 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70e048d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70d809a820 con 0x7f70d8131810 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dda28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 0x7f70d8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70de229700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70de229700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d8131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52648/0 (socket says 172.21.15.32:52648) 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46590/0 (socket says 172.21.15.32:46590) 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dda28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 0x7f70d8099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60384/0 (socket says 172.21.15.32:60384) 2023-09-30T22:26:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70de229700 1 -- 172.21.15.32:0/3537548231 learned_addr learned my addr 172.21.15.32:0/3537548231 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 -- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 msgr2=0x7f70d8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d8131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 -- 172.21.15.32:0/3537548231 >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 msgr2=0x7f70d8099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3537548231 >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 0x7f70d8099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 -- 172.21.15.32:0/3537548231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8071d20 con 0x7f70d809a3b0 2023-09-30T22:26:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dda28700 1 --2- 172.21.15.32:0/3537548231 >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 0x7f70d8099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70de229700 1 --2- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.715+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8138650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f70d000dc60 tx=0x7f70d000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765b7bd006cefbc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70dd227700 1 -- 172.21.15.32:0/3537548231 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0012070 con 0x7f70d809a3b0 2023-09-30T22:26:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70dd227700 1 -- 172.21.15.32:0/3537548231 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d000d2e0 con 0x7f70d809a3b0 2023-09-30T22:26:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70dd227700 1 -- 172.21.15.32:0/3537548231 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d000f900 con 0x7f70d809a3b0 2023-09-30T22:26:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 -- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 msgr2=0x7f70d8138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 --2- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8138650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f70d000dc60 tx=0x7f70d000df70 comp rx=0 tx=0).stop 2023-09-30T22:26:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 -- 172.21.15.32:0/3537548231 shutdown_connections 2023-09-30T22:26:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 --2- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8138650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 --2- 172.21.15.32:0/3537548231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 --2- 172.21.15.32:0/3537548231 >> v2:172.21.15.32:3300/0 conn(0x7f70d8099a40 0x7f70d8099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 -- 172.21.15.32:0/3537548231 >> 172.21.15.32:0/3537548231 conn(0x7f70d80a2aa0 msgr2=0x7f70d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 -- 172.21.15.32:0/3537548231 shutdown_connections 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.716+0000 7f70e048d700 1 -- 172.21.15.32:0/3537548231 wait complete. 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.718+0000 7f70e048d700 1 Processor -- start 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.718+0000 7f70e048d700 1 -- start start 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70e048d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 0x7f70d81196c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70e048d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8119c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70e048d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d811a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70e048d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d811ab70 con 0x7f70d8099a40 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70de229700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 0x7f70d81196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70de229700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 0x7f70d81196c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60390/0 (socket says 172.21.15.32:60390) 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70de229700 1 -- 172.21.15.32:0/3861091770 learned_addr learned my addr 172.21.15.32:0/3861091770 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d811b2d0 con 0x7f70d809a3b0 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70d811c520 con 0x7f70d8131810 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d811a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dda28700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8119c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 -- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 msgr2=0x7f70d8119c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8119c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 -- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 msgr2=0x7f70d81196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 0x7f70d81196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc0094a0 con 0x7f70d8131810 2023-09-30T22:26:48.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70de229700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 0x7f70d81196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dda28700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8119c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70dea2a700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d811a140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70d0019000 tx=0x7f70d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.719+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0010c90 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.720+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d0010df0 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.720+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c8009b30 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.720+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d000d940 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.720+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d00038e0 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.720+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f70d0003a40 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.721+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b80052e0 con 0x7f70d8131810 2023-09-30T22:26:48.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.721+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f70d00449b0 con 0x7f70d8131810 2023-09-30T22:26:48.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.726+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70d0024080 con 0x7f70d8131810 2023-09-30T22:26:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.896+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1fc4072980 con 0x7f1fc4138d80 2023-09-30T22:26:48.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.896+0000 7f1fabfff700 1 -- 172.21.15.32:0/949677953 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1fbc012340 con 0x7f1fc4138d80 2023-09-30T22:26:48.897 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc4138d80 msgr2=0x7f1fc4134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc4138d80 0x7f1fc4134180 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00efc0 tx=0x7f1fbc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 shutdown_connections 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fc4138d80 0x7f1fc4134180 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1fc409d630 0x7f1fc4133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 --2- 172.21.15.32:0/949677953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fc40a0a30 0x7f1fc4133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 >> 172.21.15.32:0/949677953 conn(0x7f1fc40a3050 msgr2=0x7f1fc409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 shutdown_connections 2023-09-30T22:26:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.898+0000 7f1fc9926700 1 -- 172.21.15.32:0/949677953 wait complete. 2023-09-30T22:26:48.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.931+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f70b8005590 con 0x7f70d8131810 2023-09-30T22:26:48.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.932+0000 7f70d6ffd700 1 -- 172.21.15.32:0/3861091770 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70d0003cf0 con 0x7f70d8131810 2023-09-30T22:26:48.933 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 msgr2=0x7f70d811a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d811a140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70d0019000 tx=0x7f70d0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 shutdown_connections 2023-09-30T22:26:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f70d8131810 0x7f70d811a140 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d8099a40 0x7f70d81196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 --2- 172.21.15.32:0/3861091770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70d809a3b0 0x7f70d8119c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 >> 172.21.15.32:0/3861091770 conn(0x7f70d80a2aa0 msgr2=0x7f70d812e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 shutdown_connections 2023-09-30T22:26:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:48.934+0000 7f70e048d700 1 -- 172.21.15.32:0/3861091770 wait complete. 2023-09-30T22:26:48.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:48.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:49.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.131+0000 7fc8071ee700 1 Processor -- start 2023-09-30T22:26:49.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.131+0000 7fc8071ee700 1 -- start start 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc8071ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc80009d630 0x7fc8000a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc8071ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 0x7fc80013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc8071ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000a2a70 con 0x7fc8000a0a30 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc8071ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8000a01d0 con 0x7fc80009d630 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc8071ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc80009e2b0 con 0x7fc800138d80 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 0x7fc80013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 0x7fc80013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52678/0 (socket says 172.21.15.32:52678) 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc7fffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc80009d630 0x7fc8000a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 -- 172.21.15.32:0/2684003452 learned_addr learned my addr 172.21.15.32:0/2684003452 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 -- 172.21.15.32:0/2684003452 >> v2:172.21.15.32:3300/0 conn(0x7fc80009d630 msgr2=0x7fc8000a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:49.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 --2- 172.21.15.32:0/2684003452 >> v2:172.21.15.32:3300/0 conn(0x7fc80009d630 0x7fc8000a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 -- 172.21.15.32:0/2684003452 >> v1:172.21.15.32:6789/0 conn(0x7fc8000a0a30 legacy=0x7fc8000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 -- 172.21.15.32:0/2684003452 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80009dc50 con 0x7fc800138d80 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc7ff7fe700 1 -- 172.21.15.32:0/2684003452 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984051508 0 0) 0x7fc8000a2a70 con 0x7fc8000a0a30 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc7fffff700 1 --2- 172.21.15.32:0/2684003452 >> v2:172.21.15.32:3300/0 conn(0x7fc80009d630 0x7fc8000a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.132+0000 7fc80578b700 1 --2- 172.21.15.32:0/2684003452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 0x7fc80013b160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009e50 tx=0x7fc7f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9979cc97b17c46 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc7ff7fe700 1 -- 172.21.15.32:0/2684003452 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800e070 con 0x7fc800138d80 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc7ff7fe700 1 -- 172.21.15.32:0/2684003452 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f800b910 con 0x7fc800138d80 2023-09-30T22:26:49.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc7ff7fe700 1 -- 172.21.15.32:0/2684003452 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800bcc0 con 0x7fc800138d80 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 -- 172.21.15.32:0/2684003452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 msgr2=0x7fc80013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 --2- 172.21.15.32:0/2684003452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 0x7fc80013b160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009e50 tx=0x7fc7f800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 -- 172.21.15.32:0/2684003452 shutdown_connections 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 --2- 172.21.15.32:0/2684003452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc800138d80 0x7fc80013b160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 --2- 172.21.15.32:0/2684003452 >> v2:172.21.15.32:3300/0 conn(0x7fc80009d630 0x7fc8000a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 -- 172.21.15.32:0/2684003452 >> 172.21.15.32:0/2684003452 conn(0x7fc8000a3050 msgr2=0x7fc8000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 -- 172.21.15.32:0/2684003452 shutdown_connections 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.133+0000 7fc8071ee700 1 -- 172.21.15.32:0/2684003452 wait complete. 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.134+0000 7fc8071ee700 1 Processor -- start 2023-09-30T22:26:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 -- start start 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 0x7fc80012c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000a0a30 0x7fc80012c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc800138d80 0x7fc80012cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80013ac50 con 0x7fc8000a0a30 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80012e990 con 0x7fc800138d80 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc8071ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000a0410 con 0x7fc80009d630 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc804f8a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 0x7fc80012c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc804f8a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 0x7fc80012c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52680/0 (socket says 172.21.15.32:52680) 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc80578b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc800138d80 0x7fc80012cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc804f8a700 1 -- 172.21.15.32:0/1834435162 learned_addr learned my addr 172.21.15.32:0/1834435162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:49.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.135+0000 7fc7fffff700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000a0a30 0x7fc80012c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc804f8a700 1 -- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc800138d80 msgr2=0x7fc80012cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc804f8a700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc800138d80 0x7fc80012cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc804f8a700 1 -- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000a0a30 msgr2=0x7fc80012c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc804f8a700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000a0a30 0x7fc80012c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc804f8a700 1 -- 172.21.15.32:0/1834435162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec0094a0 con 0x7fc80009d630 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc804f8a700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 0x7fc80012c1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c520 tx=0x7fc7f400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40054e0 con 0x7fc80009d630 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f4003680 con 0x7fc80009d630 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40039f0 con 0x7fc80009d630 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8009b30 con 0x7fc80009d630 2023-09-30T22:26:49.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.136+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80012e330 con 0x7fc80009d630 2023-09-30T22:26:49.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.137+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7f4003b50 con 0x7fc80009d630 2023-09-30T22:26:49.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.137+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc7f403ead0 con 0x7fc80009d630 2023-09-30T22:26:49.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.137+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80012ebb0 con 0x7fc80009d630 2023-09-30T22:26:49.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.142+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7f4017080 con 0x7fc80009d630 2023-09-30T22:26:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.173+0000 7f761dca5700 1 Processor -- start 2023-09-30T22:26:49.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.173+0000 7f761dca5700 1 -- start start 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f761dca5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f761809e850 0x7f761809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f761dca5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7618131ea0 0x7f7618139c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f761dca5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761809b560 con 0x7f761809f160 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f761dca5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76180a0040 con 0x7f761809e850 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f761dca5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76180a02e0 con 0x7f7618131ea0 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f76177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f761809e850 0x7f761809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f7616ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f761809f160 0x7f7618131790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35480/0 (socket says 172.21.15.32:35480) 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f7616ffd700 1 -- 172.21.15.32:0/3628087874 learned_addr learned my addr 172.21.15.32:0/3628087874 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:49.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f76177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f761809e850 0x7f761809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60424/0 (socket says 172.21.15.32:60424) 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.174+0000 7f7617fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7618131ea0 0x7f7618139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76177fe700 1 -- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7618131ea0 msgr2=0x7f7618139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76177fe700 1 --2- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7618131ea0 0x7f7618139c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76177fe700 1 -- 172.21.15.32:0/3628087874 >> v1:172.21.15.32:6789/0 conn(0x7f761809f160 legacy=0x7f7618131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76177fe700 1 -- 172.21.15.32:0/3628087874 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761809fab0 con 0x7f761809e850 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76167fc700 1 -- 172.21.15.32:0/3628087874 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110601104 0 0) 0x7f761809b560 con 0x7f761809f160 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76177fe700 1 --2- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f761809e850 0x7f761809ec20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7608009d90 tx=0x7f760800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eea1ff5fe08e2f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76167fc700 1 -- 172.21.15.32:0/3628087874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760800e070 con 0x7f761809e850 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76167fc700 1 -- 172.21.15.32:0/3628087874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760800bb20 con 0x7f761809e850 2023-09-30T22:26:49.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.175+0000 7f76167fc700 1 -- 172.21.15.32:0/3628087874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608012430 con 0x7f761809e850 2023-09-30T22:26:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 -- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f761809e850 msgr2=0x7f761809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 --2- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f761809e850 0x7f761809ec20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7608009d90 tx=0x7f760800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 -- 172.21.15.32:0/3628087874 shutdown_connections 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 --2- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7618131ea0 0x7f7618139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 --2- 172.21.15.32:0/3628087874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f761809e850 0x7f761809ec20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 -- 172.21.15.32:0/3628087874 >> 172.21.15.32:0/3628087874 conn(0x7f76180a3f70 msgr2=0x7f76180a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 -- 172.21.15.32:0/3628087874 shutdown_connections 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.176+0000 7f761dca5700 1 -- 172.21.15.32:0/3628087874 wait complete. 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 Processor -- start 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 -- start start 2023-09-30T22:26:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f761809e850 0x7f761811d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f761809f160 0x7f761811d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 0x7f761811db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618120bd0 con 0x7f7618131ea0 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76181206c0 con 0x7f761809f160 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.177+0000 7f761dca5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f761811fb60 con 0x7f761809e850 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f76177fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f761809e850 0x7f761811d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 0x7f761811db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7616ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f761809f160 0x7f761811d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f76177fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f761809e850 0x7f761811d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52700/0 (socket says 172.21.15.32:52700) 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 0x7f761811db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60436/0 (socket says 172.21.15.32:60436) 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f76177fe700 1 -- 172.21.15.32:0/3710984309 learned_addr learned my addr 172.21.15.32:0/3710984309 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 -- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f761809e850 msgr2=0x7f761811d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f761809e850 0x7f761811d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 -- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f761809f160 msgr2=0x7f761811d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f761809f160 0x7f761811d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 -- 172.21.15.32:0/3710984309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7608009a70 con 0x7f7618131ea0 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f7617fff700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 0x7f761811db30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f761000e880 tx=0x7f761000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610013680 con 0x7f7618131ea0 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76100137e0 con 0x7f7618131ea0 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7618131790 con 0x7f7618131ea0 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610013b70 con 0x7f7618131ea0 2023-09-30T22:26:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.178+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761811ecc0 con 0x7f7618131ea0 2023-09-30T22:26:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.179+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7610013cd0 con 0x7f7618131ea0 2023-09-30T22:26:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.180+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f761003f5f0 con 0x7f7618131ea0 2023-09-30T22:26:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.180+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76180747f0 con 0x7f7618131ea0 2023-09-30T22:26:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.185+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f761001be10 con 0x7f7618131ea0 2023-09-30T22:26:49.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.355+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc8000747f0 con 0x7fc80009d630 2023-09-30T22:26:49.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.355+0000 7fc7fd7fa700 1 -- 172.21.15.32:0/1834435162 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7f401b3b0 con 0x7fc80009d630 2023-09-30T22:26:49.356 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.357+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 msgr2=0x7fc80012c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.357+0000 7fc8071ee700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 0x7fc80012c1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c520 tx=0x7fc7f400c830 comp rx=0 tx=0).stop 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 shutdown_connections 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc800138d80 0x7fc80012cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc80009d630 0x7fc80012c1a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 --2- 172.21.15.32:0/1834435162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000a0a30 0x7fc80012c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 >> 172.21.15.32:0/1834435162 conn(0x7fc8000a3050 msgr2=0x7fc8000a4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 shutdown_connections 2023-09-30T22:26:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.358+0000 7fc8071ee700 1 -- 172.21.15.32:0/1834435162 wait complete. 2023-09-30T22:26:49.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.393+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f761809b780 con 0x7f7618131ea0 2023-09-30T22:26:49.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.394+0000 7f75fffff700 1 -- 172.21.15.32:0/3710984309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7610017020 con 0x7f7618131ea0 2023-09-30T22:26:49.395 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 msgr2=0x7f761811db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 0x7f761811db30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f761000e880 tx=0x7f761000eb90 comp rx=0 tx=0).stop 2023-09-30T22:26:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 shutdown_connections 2023-09-30T22:26:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7618131ea0 0x7f761811db30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f761809e850 0x7f761811d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 --2- 172.21.15.32:0/3710984309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f761809f160 0x7f761811d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 >> 172.21.15.32:0/3710984309 conn(0x7f76180a3f70 msgr2=0x7f761806f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 shutdown_connections 2023-09-30T22:26:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:49.396+0000 7f761dca5700 1 -- 172.21.15.32:0/3710984309 wait complete. 2023-09-30T22:26:49.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:49.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:49.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.061+0000 7fcd84b71700 1 Processor -- start 2023-09-30T22:26:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.061+0000 7fcd84b71700 1 -- start start 2023-09-30T22:26:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd84b71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd800a0a30 0x7fcd800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd84b71700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd8009d630 0x7fcd800a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd84b71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 0x7fcd80133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd84b71700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800a2a70 con 0x7fcd8009d630 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd84b71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800a01d0 con 0x7fcd800a0a30 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd84b71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8009e2b0 con 0x7fcd80133930 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 0x7fcd80133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd800a0a30 0x7fcd800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 0x7fcd80133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52710/0 (socket says 172.21.15.32:52710) 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd8009d630 0x7fcd800a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/3783341554 learned_addr learned my addr 172.21.15.32:0/3783341554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd800a0a30 msgr2=0x7fcd800a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd800a0a30 0x7fcd800a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/3783341554 >> v2:172.21.15.32:3300/0 conn(0x7fcd8009d630 msgr2=0x7fcd800a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/3783341554 >> v2:172.21.15.32:3300/0 conn(0x7fcd8009d630 0x7fcd800a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/3783341554 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8009dc50 con 0x7fcd80133930 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.062+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 0x7fcd80133dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd780098f0 tx=0x7fcd78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88251a1b5717b3ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd7d59a700 1 -- 172.21.15.32:0/3783341554 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78011830 con 0x7fcd80133930 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd7d59a700 1 -- 172.21.15.32:0/3783341554 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd78011990 con 0x7fcd80133930 2023-09-30T22:26:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd7d59a700 1 -- 172.21.15.32:0/3783341554 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78011d20 con 0x7fcd80133930 2023-09-30T22:26:50.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 -- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 msgr2=0x7fcd80133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 --2- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 0x7fcd80133dd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd780098f0 tx=0x7fcd78009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 -- 172.21.15.32:0/3783341554 shutdown_connections 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 --2- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd800a0a30 0x7fcd800a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 --2- 172.21.15.32:0/3783341554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd80133930 0x7fcd80133dd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 --2- 172.21.15.32:0/3783341554 >> v2:172.21.15.32:3300/0 conn(0x7fcd8009d630 0x7fcd800a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 -- 172.21.15.32:0/3783341554 >> 172.21.15.32:0/3783341554 conn(0x7fcd800a3050 msgr2=0x7fcd800a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 -- 172.21.15.32:0/3783341554 shutdown_connections 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.063+0000 7fcd84b71700 1 -- 172.21.15.32:0/3783341554 wait complete. 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 Processor -- start 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 -- start start 2023-09-30T22:26:50.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd8009d630 0x7fcd8011f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd800a0a30 0x7fcd8011fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 0x7fcd801c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80133410 con 0x7fcd8009d630 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8011ef60 con 0x7fcd80133930 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd84b71700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd801222c0 con 0x7fcd800a0a30 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 0x7fcd801c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd7e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd8009d630 0x7fcd8011f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd7ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 0x7fcd801c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46634/0 (socket says 172.21.15.32:46634) 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/1850285687 learned_addr learned my addr 172.21.15.32:0/1850285687 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.065+0000 7fcd7dd9b700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd800a0a30 0x7fcd8011fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd800a0a30 msgr2=0x7fcd8011fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd800a0a30 0x7fcd8011fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd8009d630 msgr2=0x7fcd8011f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd8009d630 0x7fcd8011f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7ed9d700 1 -- 172.21.15.32:0/1850285687 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78009550 con 0x7fcd80133930 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7e59c700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd8009d630 0x7fcd8011f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd7ed9d700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 0x7fcd801c7f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009c40 tx=0x7fcd78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78004450 con 0x7fcd80133930 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd780045b0 con 0x7fcd80133930 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd80120c40 con 0x7fcd80133930 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd780038c0 con 0x7fcd80133930 2023-09-30T22:26:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.066+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd80120e80 con 0x7fcd80133930 2023-09-30T22:26:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.067+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8011ff60 con 0x7fcd80133930 2023-09-30T22:26:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.068+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd78003a20 con 0x7fcd80133930 2023-09-30T22:26:50.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.068+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd78040990 con 0x7fcd80133930 2023-09-30T22:26:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.072+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd78010380 con 0x7fcd80133930 2023-09-30T22:26:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.098+0000 7f9cb2696700 1 Processor -- start 2023-09-30T22:26:50.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.098+0000 7f9cb2696700 1 -- start start 2023-09-30T22:26:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cb2696700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 0x7f9cac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cb2696700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cac138d80 0x7f9cac13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cb2696700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0a2a70 con 0x7f9cac09d630 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cb2696700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac0a01d0 con 0x7f9cac138d80 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cb2696700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac09e2b0 con 0x7f9cac0a0a30 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 0x7f9cac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cb0c33700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cac138d80 0x7f9cac13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 0x7f9cac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46644/0 (socket says 172.21.15.32:46644) 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.099+0000 7f9cabfff700 1 -- 172.21.15.32:0/2254936261 learned_addr learned my addr 172.21.15.32:0/2254936261 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cabfff700 1 -- 172.21.15.32:0/2254936261 >> v2:172.21.15.32:3300/0 conn(0x7f9cac138d80 msgr2=0x7f9cac13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cabfff700 1 --2- 172.21.15.32:0/2254936261 >> v2:172.21.15.32:3300/0 conn(0x7f9cac138d80 0x7f9cac13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cabfff700 1 -- 172.21.15.32:0/2254936261 >> v1:172.21.15.32:6789/0 conn(0x7f9cac09d630 legacy=0x7f9cac0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cabfff700 1 -- 172.21.15.32:0/2254936261 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac09dc50 con 0x7f9cac0a0a30 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cb0c33700 1 --2- 172.21.15.32:0/2254936261 >> v2:172.21.15.32:3300/0 conn(0x7f9cac138d80 0x7f9cac13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cabfff700 1 --2- 172.21.15.32:0/2254936261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 0x7f9cac0a0e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098f0 tx=0x7f9c9c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a644a84731fb486 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9caaffd700 1 -- 172.21.15.32:0/2254936261 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c011830 con 0x7f9cac0a0a30 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9caaffd700 1 -- 172.21.15.32:0/2254936261 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c011990 con 0x7f9cac0a0a30 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9caaffd700 1 -- 172.21.15.32:0/2254936261 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c011d20 con 0x7f9cac0a0a30 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cb2696700 1 -- 172.21.15.32:0/2254936261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 msgr2=0x7f9cac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.100+0000 7f9cb2696700 1 --2- 172.21.15.32:0/2254936261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 0x7f9cac0a0e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0098f0 tx=0x7f9c9c009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.101+0000 7f9cb2696700 1 -- 172.21.15.32:0/2254936261 shutdown_connections 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.101+0000 7f9cb2696700 1 --2- 172.21.15.32:0/2254936261 >> v2:172.21.15.32:3300/0 conn(0x7f9cac138d80 0x7f9cac13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.101+0000 7f9cb2696700 1 --2- 172.21.15.32:0/2254936261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac0a0a30 0x7f9cac0a0e00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.101+0000 7f9cb2696700 1 -- 172.21.15.32:0/2254936261 >> 172.21.15.32:0/2254936261 conn(0x7f9cac0a3050 msgr2=0x7f9cac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.101+0000 7f9cb2696700 1 -- 172.21.15.32:0/2254936261 shutdown_connections 2023-09-30T22:26:50.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.101+0000 7f9cb2696700 1 -- 172.21.15.32:0/2254936261 wait complete. 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 Processor -- start 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 -- start start 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9cac09d630 0x7f9cac1208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cac0a0a30 0x7f9cac120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 0x7f9cac121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac122470 con 0x7f9cac0a0a30 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1237c0 con 0x7f9cac138d80 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb2696700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac121d50 con 0x7f9cac09d630 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.102+0000 7f9cb0c33700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 0x7f9cac121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 0x7f9cac121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46648/0 (socket says 172.21.15.32:46648) 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 -- 172.21.15.32:0/3625142900 learned_addr learned my addr 172.21.15.32:0/3625142900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9cac09d630 0x7f9cac1208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cab7fe700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cac0a0a30 0x7f9cac120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 -- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9cac09d630 msgr2=0x7f9cac1208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9cac09d630 0x7f9cac1208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 -- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cac0a0a30 msgr2=0x7f9cac120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cac0a0a30 0x7f9cac120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 -- 172.21.15.32:0/3625142900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4009a70 con 0x7f9cac138d80 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb0c33700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 0x7f9cac121370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400efc0 tx=0x7f9ca400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400cc90 con 0x7f9cac138d80 2023-09-30T22:26:50.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca400f040 con 0x7f9cac138d80 2023-09-30T22:26:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c009550 con 0x7f9cac138d80 2023-09-30T22:26:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40146c0 con 0x7f9cac138d80 2023-09-30T22:26:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.103+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac1243d0 con 0x7f9cac138d80 2023-09-30T22:26:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.104+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ca4014820 con 0x7f9cac138d80 2023-09-30T22:26:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.104+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac0747f0 con 0x7f9cac138d80 2023-09-30T22:26:50.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.105+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9ca40401a0 con 0x7f9cac138d80 2023-09-30T22:26:50.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.114+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ca4014ad0 con 0x7f9cac138d80 2023-09-30T22:26:50.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.283+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd80121f40 con 0x7fcd80133930 2023-09-30T22:26:50.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.284+0000 7fcd6effd700 1 -- 172.21.15.32:0/1850285687 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd80121f40 con 0x7fcd80133930 2023-09-30T22:26:50.285 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:50.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 msgr2=0x7fcd801c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 0x7fcd801c7f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009c40 tx=0x7fcd78004750 comp rx=0 tx=0).stop 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 shutdown_connections 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd80133930 0x7fcd801c7f80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd8009d630 0x7fcd8011f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 --2- 172.21.15.32:0/1850285687 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd800a0a30 0x7fcd8011fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 >> 172.21.15.32:0/1850285687 conn(0x7fcd800a3050 msgr2=0x7fcd8009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 shutdown_connections 2023-09-30T22:26:50.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.286+0000 7fcd84b71700 1 -- 172.21.15.32:0/1850285687 wait complete. 2023-09-30T22:26:50.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:50.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.326+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9cac1243d0 con 0x7f9cac138d80 2023-09-30T22:26:50.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.326+0000 7f9ca8ff9700 1 -- 172.21.15.32:0/3625142900 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9cac1243d0 con 0x7f9cac138d80 2023-09-30T22:26:50.327 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:50.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 msgr2=0x7f9cac121370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 0x7f9cac121370 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400efc0 tx=0x7f9ca400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 shutdown_connections 2023-09-30T22:26:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cac138d80 0x7f9cac121370 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9cac09d630 0x7f9cac1208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 --2- 172.21.15.32:0/3625142900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cac0a0a30 0x7f9cac120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 >> 172.21.15.32:0/3625142900 conn(0x7f9cac0a3050 msgr2=0x7f9cac139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 shutdown_connections 2023-09-30T22:26:50.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.329+0000 7f9cb2696700 1 -- 172.21.15.32:0/3625142900 wait complete. 2023-09-30T22:26:50.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:50.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:50.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.527+0000 7f2fc363f700 1 Processor -- start 2023-09-30T22:26:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.527+0000 7f2fc363f700 1 -- start start 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc363f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d630 0x7f2fbc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc363f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fbc138d80 0x7f2fbc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc363f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0a2a70 con 0x7f2fbc0a0a30 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc363f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc0a01d0 con 0x7f2fbc138d80 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc363f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc09e2b0 con 0x7f2fbc09d630 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc1bdc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fbc138d80 0x7f2fbc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc0bda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d630 0x7f2fbc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc13db700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fbc0a0a30 0x7f2fbc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55942/0 (socket says 172.21.15.32:55942) 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.528+0000 7f2fc13db700 1 -- 172.21.15.32:0/2631182074 learned_addr learned my addr 172.21.15.32:0/2631182074 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d630 msgr2=0x7f2fbc0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fc1bdc700 1 --2- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d630 0x7f2fbc0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2631182074 >> v1:172.21.15.32:6789/0 conn(0x7f2fbc0a0a30 legacy=0x7f2fbc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2631182074 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc09dc50 con 0x7f2fbc138d80 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fb3fff700 1 -- 172.21.15.32:0/2631182074 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242135718 0 0) 0x7f2fbc0a2a70 con 0x7f2fbc0a0a30 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fc0bda700 1 --2- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d630 0x7f2fbc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fc1bdc700 1 --2- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc138d80 0x7f2fbc13b160 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009840 tx=0x7f2fb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79db255a5db0cee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fb3fff700 1 -- 172.21.15.32:0/2631182074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4011830 con 0x7f2fbc138d80 2023-09-30T22:26:50.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fb3fff700 1 -- 172.21.15.32:0/2631182074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4011990 con 0x7f2fbc138d80 2023-09-30T22:26:50.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.529+0000 7f2fb3fff700 1 -- 172.21.15.32:0/2631182074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4011d20 con 0x7f2fbc138d80 2023-09-30T22:26:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 -- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc138d80 msgr2=0x7f2fbc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc138d80 0x7f2fbc13b160 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009840 tx=0x7f2fb4009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 -- 172.21.15.32:0/2631182074 shutdown_connections 2023-09-30T22:26:50.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc138d80 0x7f2fbc13b160 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2631182074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d630 0x7f2fbc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 -- 172.21.15.32:0/2631182074 >> 172.21.15.32:0/2631182074 conn(0x7f2fbc0a3050 msgr2=0x7f2fbc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 -- 172.21.15.32:0/2631182074 shutdown_connections 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.530+0000 7f2fc363f700 1 -- 172.21.15.32:0/2631182074 wait complete. 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 Processor -- start 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 -- start start 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09d630 0x7f2fbc11c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc0a0a30 0x7f2fbc11cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 0x7f2fbc11d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc11f190 con 0x7f2fbc09d630 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc09d390 con 0x7f2fbc138d80 2023-09-30T22:26:50.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.531+0000 7f2fc363f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc11fde0 con 0x7f2fbc0a0a30 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 0x7f2fbc11d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 0x7f2fbc11d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35884/0 (socket says 172.21.15.32:35884) 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2932881628 learned_addr learned my addr 172.21.15.32:0/2932881628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc0a0a30 msgr2=0x7f2fbc11cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc0a0a30 0x7f2fbc11cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09d630 msgr2=0x7f2fbc11c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09d630 0x7f2fbc11c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 -- 172.21.15.32:0/2932881628 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac0094a0 con 0x7f2fbc138d80 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc1bdc700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 0x7f2fbc11d240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009fd0 tx=0x7f2fb4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40038a0 con 0x7f2fbc138d80 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb401f070 con 0x7f2fbc138d80 2023-09-30T22:26:50.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb40094a0 con 0x7f2fbc138d80 2023-09-30T22:26:50.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.532+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4003c60 con 0x7f2fbc138d80 2023-09-30T22:26:50.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.533+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc0a0460 con 0x7f2fbc138d80 2023-09-30T22:26:50.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.533+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2fb4026480 con 0x7f2fbc138d80 2023-09-30T22:26:50.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.534+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2fb4021070 con 0x7f2fbc138d80 2023-09-30T22:26:50.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.534+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc11fc20 con 0x7f2fbc138d80 2023-09-30T22:26:50.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.539+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2fb4015070 con 0x7f2fbc138d80 2023-09-30T22:26:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.568+0000 7f1e8e67a700 1 Processor -- start 2023-09-30T22:26:50.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.568+0000 7f1e8e67a700 1 -- start start 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e8e67a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a1510 0x7f1e88138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e8e67a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e8813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e8e67a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8809edd0 con 0x7f1e880a0a30 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e8e67a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880a18e0 con 0x7f1e88139250 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e8e67a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8809e1f0 con 0x7f1e880a1510 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e87fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e880a0a30 0x7f1e880a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55944/0 (socket says 172.21.15.32:55944) 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e8cc17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e8813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e87fff700 1 -- 172.21.15.32:0/991093284 learned_addr learned my addr 172.21.15.32:0/991093284 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:50.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e877fe700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a1510 0x7f1e88138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e86ffd700 1 -- 172.21.15.32:0/991093284 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431091348 0 0) 0x7f1e8809edd0 con 0x7f1e880a0a30 2023-09-30T22:26:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.569+0000 7f1e877fe700 1 -- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 msgr2=0x7f1e8813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e877fe700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e8813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e877fe700 1 -- 172.21.15.32:0/991093284 >> v1:172.21.15.32:6789/0 conn(0x7f1e880a0a30 legacy=0x7f1e880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e877fe700 1 -- 172.21.15.32:0/991093284 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8809dc50 con 0x7f1e880a1510 2023-09-30T22:26:50.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e8cc17700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e8813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e877fe700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a1510 0x7f1e88138d10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5bb7770f65327dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e86ffd700 1 -- 172.21.15.32:0/991093284 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c011830 con 0x7f1e880a1510 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e86ffd700 1 -- 172.21.15.32:0/991093284 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c011990 con 0x7f1e880a1510 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e86ffd700 1 -- 172.21.15.32:0/991093284 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c011d20 con 0x7f1e880a1510 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e8e67a700 1 -- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a1510 msgr2=0x7f1e88138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a1510 0x7f1e88138d10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e8e67a700 1 -- 172.21.15.32:0/991093284 shutdown_connections 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e8813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.570+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/991093284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a1510 0x7f1e88138d10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.571+0000 7f1e8e67a700 1 -- 172.21.15.32:0/991093284 >> 172.21.15.32:0/991093284 conn(0x7f1e880a6550 msgr2=0x7f1e880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.571+0000 7f1e8e67a700 1 -- 172.21.15.32:0/991093284 shutdown_connections 2023-09-30T22:26:50.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.571+0000 7f1e8e67a700 1 -- 172.21.15.32:0/991093284 wait complete. 2023-09-30T22:26:50.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 Processor -- start 2023-09-30T22:26:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 -- start start 2023-09-30T22:26:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a0a30 0x7f1e88122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 0x7f1e88122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e88123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e88125c00 con 0x7f1e880a1510 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e88138180 con 0x7f1e88139250 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8e67a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881389a0 con 0x7f1e880a0a30 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e87fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a0a30 0x7f1e88122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 0x7f1e88122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e87fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a0a30 0x7f1e88122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35522/0 (socket says 172.21.15.32:35522) 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 0x7f1e88122e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52194/0 (socket says 172.21.15.32:52194) 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e87fff700 1 -- 172.21.15.32:0/3063489043 learned_addr learned my addr 172.21.15.32:0/3063489043 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:50.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.572+0000 7f1e8cc17700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e88123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e877fe700 1 -- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a0a30 msgr2=0x7f1e88122900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e877fe700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a0a30 0x7f1e88122900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e877fe700 1 -- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 msgr2=0x7f1e88123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e877fe700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e88123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e877fe700 1 -- 172.21.15.32:0/3063489043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e80009a70 con 0x7f1e880a1510 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e877fe700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 0x7f1e88122e40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c016000 tx=0x7f1e7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0044d0 con 0x7f1e880a1510 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c01f070 con 0x7f1e880a1510 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c0094a0 con 0x7f1e880a1510 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0038a0 con 0x7f1e880a1510 2023-09-30T22:26:50.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.573+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e88138760 con 0x7f1e880a1510 2023-09-30T22:26:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.574+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e7c003a00 con 0x7f1e880a1510 2023-09-30T22:26:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.574+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e880a1d00 con 0x7f1e880a1510 2023-09-30T22:26:50.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.574+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e7c021070 con 0x7f1e880a1510 2023-09-30T22:26:50.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.579+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e7c0253e0 con 0x7f1e880a1510 2023-09-30T22:26:50.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.746+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2fbc0747f0 con 0x7f2fbc138d80 2023-09-30T22:26:50.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.747+0000 7f2fb1ffb700 1 -- 172.21.15.32:0/2932881628 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fb4010790 con 0x7f2fbc138d80 2023-09-30T22:26:50.747 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 msgr2=0x7f2fbc11d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 0x7f2fbc11d240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009fd0 tx=0x7f2fb4004750 comp rx=0 tx=0).stop 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 shutdown_connections 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc138d80 0x7f2fbc11d240 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09d630 0x7f2fbc11c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 --2- 172.21.15.32:0/2932881628 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc0a0a30 0x7f2fbc11cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 >> 172.21.15.32:0/2932881628 conn(0x7f2fbc0a3050 msgr2=0x7f2fbc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 shutdown_connections 2023-09-30T22:26:50.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.749+0000 7f2fc363f700 1 -- 172.21.15.32:0/2932881628 wait complete. 2023-09-30T22:26:50.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.784+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e88124cd0 con 0x7f1e880a1510 2023-09-30T22:26:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.784+0000 7f1e84ff9700 1 -- 172.21.15.32:0/3063489043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e7c010b70 con 0x7f1e880a1510 2023-09-30T22:26:50.784 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:50.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 msgr2=0x7f1e88122e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 0x7f1e88122e40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c016000 tx=0x7f1e7c004750 comp rx=0 tx=0).stop 2023-09-30T22:26:50.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 shutdown_connections 2023-09-30T22:26:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e88139250 0x7f1e88123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e880a0a30 0x7f1e88122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 --2- 172.21.15.32:0/3063489043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e880a1510 0x7f1e88122e40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 >> 172.21.15.32:0/3063489043 conn(0x7f1e880a6550 msgr2=0x7f1e8809ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 shutdown_connections 2023-09-30T22:26:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:50.786+0000 7f1e8e67a700 1 -- 172.21.15.32:0/3063489043 wait complete. 2023-09-30T22:26:50.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:51.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:51.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 Processor -- start 2023-09-30T22:26:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 -- start start 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 0x7f67cc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67cc133930 0x7f67cc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67cc0a2a70 con 0x7f67cc133930 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc0a01d0 con 0x7f67cc0a0a30 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.453+0000 7f67d2d53700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc09e2b0 con 0x7f67cc09d630 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67d0aef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 0x7f67cc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67d0aef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 0x7f67cc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35918/0 (socket says 172.21.15.32:35918) 2023-09-30T22:26:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35532/0 (socket says 172.21.15.32:35532) 2023-09-30T22:26:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67d0aef700 1 -- 172.21.15.32:0/75195654 learned_addr learned my addr 172.21.15.32:0/75195654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67d12f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67cc133930 0x7f67cc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 -- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 msgr2=0x7f67cc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 --2- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 0x7f67cc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 -- 172.21.15.32:0/75195654 >> v2:172.21.15.32:3300/0 conn(0x7f67cc133930 msgr2=0x7f67cc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 --2- 172.21.15.32:0/75195654 >> v2:172.21.15.32:3300/0 conn(0x7f67cc133930 0x7f67cc133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 -- 172.21.15.32:0/75195654 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc09dc50 con 0x7f67cc09d630 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67d12f0700 1 --2- 172.21.15.32:0/75195654 >> v2:172.21.15.32:3300/0 conn(0x7f67cc133930 0x7f67cc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67d0aef700 1 --2- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 0x7f67cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cbfff700 1 --2- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc0a1340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009840 tx=0x7f67b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279d535b752296e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cb7fe700 1 -- 172.21.15.32:0/75195654 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011830 con 0x7f67cc09d630 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cb7fe700 1 -- 172.21.15.32:0/75195654 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8011990 con 0x7f67cc09d630 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.454+0000 7f67cb7fe700 1 -- 172.21.15.32:0/75195654 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011d20 con 0x7f67cc09d630 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 -- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 msgr2=0x7f67cc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 --2- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc0a1340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009840 tx=0x7f67b8009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 -- 172.21.15.32:0/75195654 shutdown_connections 2023-09-30T22:26:51.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 --2- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc0a0a30 0x7f67cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 --2- 172.21.15.32:0/75195654 >> v2:172.21.15.32:3300/0 conn(0x7f67cc133930 0x7f67cc133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 --2- 172.21.15.32:0/75195654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc0a1340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 -- 172.21.15.32:0/75195654 >> 172.21.15.32:0/75195654 conn(0x7f67cc0a3050 msgr2=0x7f67cc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 -- 172.21.15.32:0/75195654 shutdown_connections 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.455+0000 7f67d2d53700 1 -- 172.21.15.32:0/75195654 wait complete. 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.456+0000 7f67d2d53700 1 Processor -- start 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.456+0000 7f67d2d53700 1 -- start start 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d2d53700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc11f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d2d53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 0x7f67cc11f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d2d53700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc133930 0x7f67cc1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d2d53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc122390 con 0x7f67cc0a0a30 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d2d53700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc122570 con 0x7f67cc133930 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d2d53700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1333e0 con 0x7f67cc09d630 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d0aef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc11f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 0x7f67cc11f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d0aef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc11f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35536/0 (socket says 172.21.15.32:35536) 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 0x7f67cc11f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52200/0 (socket says 172.21.15.32:52200) 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d0aef700 1 -- 172.21.15.32:0/1696215151 learned_addr learned my addr 172.21.15.32:0/1696215151 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67d12f0700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc133930 0x7f67cc1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 -- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 msgr2=0x7f67cc11f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc11f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 -- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc133930 msgr2=0x7f67cc1c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc133930 0x7f67cc1c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.457+0000 7f67cbfff700 1 -- 172.21.15.32:0/1696215151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0009550 con 0x7f67cc0a0a30 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67d0aef700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67cbfff700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 0x7f67cc11f9f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0a1f80 tx=0x7f67b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8004450 con 0x7f67cc0a0a30 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b801f070 con 0x7f67cc0a0a30 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c4009a70 con 0x7f67cc0a0a30 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80038e0 con 0x7f67cc0a0a30 2023-09-30T22:26:51.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.458+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b80094a0 con 0x7f67cc0a0a30 2023-09-30T22:26:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.459+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67b8003a40 con 0x7f67cc0a0a30 2023-09-30T22:26:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.459+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f67b8021070 con 0x7f67cc0a0a30 2023-09-30T22:26:51.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.459+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc0a0410 con 0x7f67cc0a0a30 2023-09-30T22:26:51.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.464+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67b8015080 con 0x7f67cc0a0a30 2023-09-30T22:26:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.490+0000 7f11c8f9e700 1 Processor -- start 2023-09-30T22:26:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.490+0000 7f11c8f9e700 1 -- start start 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c8f9e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c8f9e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c4138d80 0x7f11c413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c8f9e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40a2a70 con 0x7f11c409d630 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c8f9e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40a01d0 con 0x7f11c40a0a30 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c8f9e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11c409e2b0 con 0x7f11c4138d80 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f11c409d630 0x7f11c40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55948/0 (socket says 172.21.15.32:55948) 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c1d9b700 1 -- 172.21.15.32:0/1238079288 learned_addr learned my addr 172.21.15.32:0/1238079288 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c2d9d700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c4138d80 0x7f11c413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.491+0000 7f11c259c700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c259c700 1 -- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c4138d80 msgr2=0x7f11c413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c259c700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c4138d80 0x7f11c413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c259c700 1 -- 172.21.15.32:0/1238079288 >> v1:172.21.15.32:6789/0 conn(0x7f11c409d630 legacy=0x7f11c40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c259c700 1 -- 172.21.15.32:0/1238079288 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c409dc50 con 0x7f11c40a0a30 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c159a700 1 -- 172.21.15.32:0/1238079288 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415288649 0 0) 0x7f11c40a2a70 con 0x7f11c409d630 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c2d9d700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c4138d80 0x7f11c413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c259c700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c40a0e00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009870 tx=0x7f11b4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea51044b30d25c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c159a700 1 -- 172.21.15.32:0/1238079288 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4011830 con 0x7f11c40a0a30 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c159a700 1 -- 172.21.15.32:0/1238079288 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b4011990 con 0x7f11c40a0a30 2023-09-30T22:26:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c159a700 1 -- 172.21.15.32:0/1238079288 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4011d20 con 0x7f11c40a0a30 2023-09-30T22:26:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c8f9e700 1 -- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 msgr2=0x7f11c40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.492+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c40a0e00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009870 tx=0x7f11b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.493+0000 7f11c8f9e700 1 -- 172.21.15.32:0/1238079288 shutdown_connections 2023-09-30T22:26:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.493+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c4138d80 0x7f11c413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.493+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/1238079288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c40a0e00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.493+0000 7f11c8f9e700 1 -- 172.21.15.32:0/1238079288 >> 172.21.15.32:0/1238079288 conn(0x7f11c40a3050 msgr2=0x7f11c40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.493+0000 7f11c8f9e700 1 -- 172.21.15.32:0/1238079288 shutdown_connections 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.493+0000 7f11c8f9e700 1 -- 172.21.15.32:0/1238079288 wait complete. 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 Processor -- start 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 -- start start 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 0x7f11c4133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c4133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c4138d80 0x7f11c4134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4137d30 con 0x7f11c4138d80 2023-09-30T22:26:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40a4bf0 con 0x7f11c40a0a30 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c8f9e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40a0410 con 0x7f11c409d630 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 0x7f11c4133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c4133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 0x7f11c4133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35554/0 (socket says 172.21.15.32:35554) 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.494+0000 7f11c259c700 1 -- 172.21.15.32:0/2100577242 learned_addr learned my addr 172.21.15.32:0/2100577242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c2d9d700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c4138d80 0x7f11c4134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c1d9b700 1 -- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 msgr2=0x7f11c4133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c1d9b700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 0x7f11c4133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c1d9b700 1 -- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c4138d80 msgr2=0x7f11c4134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c1d9b700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c4138d80 0x7f11c4134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c1d9b700 1 -- 172.21.15.32:0/2100577242 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc009a70 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c259c700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 0x7f11c4133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c1d9b700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c4133c40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c470 tx=0x7f11ac00c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac004c80 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac004de0 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b40094a0 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac007310 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.495+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c4136500 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.496+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11ac007470 con 0x7f11c40a0a30 2023-09-30T22:26:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.496+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f11ac03f200 con 0x7f11c40a0a30 2023-09-30T22:26:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.496+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4135f60 con 0x7f11c40a0a30 2023-09-30T22:26:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.501+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11ac00ce40 con 0x7f11c40a0a30 2023-09-30T22:26:51.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.673+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f67cc0747f0 con 0x7f67cc0a0a30 2023-09-30T22:26:51.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.673+0000 7f67c97fa700 1 -- 172.21.15.32:0/1696215151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67b802b020 con 0x7f67cc0a0a30 2023-09-30T22:26:51.673 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:51.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 msgr2=0x7f67cc11f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 0x7f67cc11f9f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0a1f80 tx=0x7f67b8004750 comp rx=0 tx=0).stop 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 shutdown_connections 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67cc133930 0x7f67cc1c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67cc09d630 0x7f67cc11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 --2- 172.21.15.32:0/1696215151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67cc0a0a30 0x7f67cc11f9f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 >> 172.21.15.32:0/1696215151 conn(0x7f67cc0a3050 msgr2=0x7f67cc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 shutdown_connections 2023-09-30T22:26:51.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.675+0000 7f67d2d53700 1 -- 172.21.15.32:0/1696215151 wait complete. 2023-09-30T22:26:51.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.709+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f11c40747f0 con 0x7f11c40a0a30 2023-09-30T22:26:51.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.710+0000 7f11baffd700 1 -- 172.21.15.32:0/2100577242 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11ac01b340 con 0x7f11c40a0a30 2023-09-30T22:26:51.710 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 msgr2=0x7f11c4133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c4133c40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c470 tx=0x7f11ac00c780 comp rx=0 tx=0).stop 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 shutdown_connections 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c4138d80 0x7f11c4134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c409d630 0x7f11c4133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 --2- 172.21.15.32:0/2100577242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c40a0a30 0x7f11c4133c40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 >> 172.21.15.32:0/2100577242 conn(0x7f11c40a3050 msgr2=0x7f11c413add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.712+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 shutdown_connections 2023-09-30T22:26:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.713+0000 7f11c8f9e700 1 -- 172.21.15.32:0/2100577242 wait complete. 2023-09-30T22:26:51.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:51.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:51.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:51.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.907+0000 7f1841299700 1 Processor -- start 2023-09-30T22:26:51.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.907+0000 7f1841299700 1 -- start start 2023-09-30T22:26:51.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.907+0000 7f1841299700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c0a0a30 0x7f183c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f1841299700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f1841299700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c133930 0x7f183c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f1841299700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183c0a2a70 con 0x7f183c133930 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f1841299700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c0a01d0 con 0x7f183c0a0a30 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f1841299700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f183c09e2b0 con 0x7f183c09d630 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c0a0a30 0x7f183c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183c133930 0x7f183c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c0a0a30 0x7f183c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35956/0 (socket says 172.21.15.32:35956) 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183affd700 1 -- 172.21.15.32:0/221889013 learned_addr learned my addr 172.21.15.32:0/221889013 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 -- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 msgr2=0x7f183c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 -- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c0a0a30 msgr2=0x7f183c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c0a0a30 0x7f183c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 -- 172.21.15.32:0/221889013 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c09dc50 con 0x7f183c133930 2023-09-30T22:26:51.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.908+0000 7f183b7fe700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c133930 0x7f183c133dd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1834009d90 tx=0x7f183400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d69c4944063281 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1839ffb700 1 -- 172.21.15.32:0/221889013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183400e070 con 0x7f183c133930 2023-09-30T22:26:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1839ffb700 1 -- 172.21.15.32:0/221889013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183400bb20 con 0x7f183c133930 2023-09-30T22:26:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1839ffb700 1 -- 172.21.15.32:0/221889013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834012430 con 0x7f183c133930 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 -- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c133930 msgr2=0x7f183c133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c133930 0x7f183c133dd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1834009d90 tx=0x7f183400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 -- 172.21.15.32:0/221889013 shutdown_connections 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c0a0a30 0x7f183c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c133930 0x7f183c133dd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 --2- 172.21.15.32:0/221889013 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 -- 172.21.15.32:0/221889013 >> 172.21.15.32:0/221889013 conn(0x7f183c0a3050 msgr2=0x7f183c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 -- 172.21.15.32:0/221889013 shutdown_connections 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.909+0000 7f1841299700 1 -- 172.21.15.32:0/221889013 wait complete. 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.911+0000 7f1841299700 1 Processor -- start 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.911+0000 7f1841299700 1 -- start start 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f1841299700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c1c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f1841299700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 0x7f183c1c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f1841299700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 0x7f183c11e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f1841299700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c1bcf70 con 0x7f183c0a0a30 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f1841299700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c1333a0 con 0x7f183c133930 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f1841299700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f183c122020 con 0x7f183c09d630 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 0x7f183c1c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 0x7f183c11e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 0x7f183c1c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52224/0 (socket says 172.21.15.32:52224) 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 -- 172.21.15.32:0/3539465962 learned_addr learned my addr 172.21.15.32:0/3539465962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 0x7f183c11e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35958/0 (socket says 172.21.15.32:35958) 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183affd700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c1c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 -- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 msgr2=0x7f183c1c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c1c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 -- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 msgr2=0x7f183c11e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 0x7f183c11e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183a7fc700 1 -- 172.21.15.32:0/3539465962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009a70 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.912+0000 7f183b7fe700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 0x7f183c11e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.913+0000 7f183a7fc700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 0x7f183c1c8880 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f183000e980 tx=0x7f183000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.913+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830009710 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.913+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183000c690 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.913+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183c0a5e20 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.913+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000ca20 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.913+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c120c90 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.914+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f183000cb80 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.914+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f183003f700 con 0x7f183c0a0a30 2023-09-30T22:26:51.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.914+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183c1204b0 con 0x7f183c0a0a30 2023-09-30T22:26:51.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.919+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18300232f0 con 0x7f183c0a0a30 2023-09-30T22:26:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.949+0000 7f2524623700 1 Processor -- start 2023-09-30T22:26:51.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.949+0000 7f2524623700 1 -- start start 2023-09-30T22:26:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2524623700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c09d630 0x7f251c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2524623700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f251c138d80 0x7f251c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2524623700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c0a2a70 con 0x7f251c0a0a30 2023-09-30T22:26:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2524623700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c0a01d0 con 0x7f251c138d80 2023-09-30T22:26:51.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2524623700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f251c09e2b0 con 0x7f251c09d630 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2521bbe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c09d630 0x7f251c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f2522bc0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f251c138d80 0x7f251c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f25223bf700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f251c0a0a30 0x7f251c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55960/0 (socket says 172.21.15.32:55960) 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.950+0000 7f25223bf700 1 -- 172.21.15.32:0/3804345966 learned_addr learned my addr 172.21.15.32:0/3804345966 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f2522bc0700 1 -- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c09d630 msgr2=0x7f251c0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f2522bc0700 1 --2- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c09d630 0x7f251c0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f2522bc0700 1 -- 172.21.15.32:0/3804345966 >> v1:172.21.15.32:6789/0 conn(0x7f251c0a0a30 legacy=0x7f251c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f2522bc0700 1 -- 172.21.15.32:0/3804345966 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c09dc50 con 0x7f251c138d80 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f25213bd700 1 -- 172.21.15.32:0/3804345966 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567300003 0 0) 0x7f251c0a2a70 con 0x7f251c0a0a30 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f2521bbe700 1 --2- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c09d630 0x7f251c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f2522bc0700 1 --2- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c138d80 0x7f251c13b160 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2514009d90 tx=0x7f251400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed10189cc342f80a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f25213bd700 1 -- 172.21.15.32:0/3804345966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400e070 con 0x7f251c138d80 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f25213bd700 1 -- 172.21.15.32:0/3804345966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251400bb20 con 0x7f251c138d80 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.951+0000 7f25213bd700 1 -- 172.21.15.32:0/3804345966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514012430 con 0x7f251c138d80 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 -- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c138d80 msgr2=0x7f251c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 --2- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c138d80 0x7f251c13b160 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2514009d90 tx=0x7f251400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 -- 172.21.15.32:0/3804345966 shutdown_connections 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 --2- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c138d80 0x7f251c13b160 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 --2- 172.21.15.32:0/3804345966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c09d630 0x7f251c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 -- 172.21.15.32:0/3804345966 >> 172.21.15.32:0/3804345966 conn(0x7f251c0a3050 msgr2=0x7f251c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 -- 172.21.15.32:0/3804345966 shutdown_connections 2023-09-30T22:26:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.952+0000 7f2524623700 1 -- 172.21.15.32:0/3804345966 wait complete. 2023-09-30T22:26:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.953+0000 7f2524623700 1 Processor -- start 2023-09-30T22:26:51.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.953+0000 7f2524623700 1 -- start start 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2524623700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c09d630 0x7f251c12c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2524623700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 0x7f251c12cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2524623700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251c138d80 0x7f251c12cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2524623700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c130190 con 0x7f251c09d630 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2524623700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c12fc10 con 0x7f251c138d80 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2524623700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0a0410 con 0x7f251c0a0a30 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2521bbe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 0x7f251c12cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2521bbe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 0x7f251c12cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35586/0 (socket says 172.21.15.32:35586) 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f2521bbe700 1 -- 172.21.15.32:0/1018855778 learned_addr learned my addr 172.21.15.32:0/1018855778 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.954+0000 7f25223bf700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c09d630 0x7f251c12c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2521bbe700 1 -- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251c138d80 msgr2=0x7f251c12cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2521bbe700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251c138d80 0x7f251c12cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2521bbe700 1 -- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c09d630 msgr2=0x7f251c12c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2521bbe700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c09d630 0x7f251c12c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2521bbe700 1 -- 172.21.15.32:0/1018855778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c0094a0 con 0x7f251c0a0a30 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f25223bf700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c09d630 0x7f251c12c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2521bbe700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 0x7f251c12cab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f250c00ebe0 tx=0x7f250c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c009800 con 0x7f251c0a0a30 2023-09-30T22:26:51.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c00c910 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2514009a70 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00cc80 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.955+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c12e5a0 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.956+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f250c013470 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.956+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f250c03f880 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.956+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c0747f0 con 0x7f251c0a0a30 2023-09-30T22:26:51.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:51.961+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f250c013720 con 0x7f251c0a0a30 2023-09-30T22:26:52.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.126+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f183c0747f0 con 0x7f183c0a0a30 2023-09-30T22:26:52.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.126+0000 7f18237fe700 1 -- 172.21.15.32:0/3539465962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f183001bd00 con 0x7f183c0a0a30 2023-09-30T22:26:52.127 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:52.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 msgr2=0x7f183c1c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 0x7f183c1c8880 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f183000e980 tx=0x7f183000ec90 comp rx=0 tx=0).stop 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 shutdown_connections 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f183c133930 0x7f183c11e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f183c09d630 0x7f183c1c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 --2- 172.21.15.32:0/3539465962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183c0a0a30 0x7f183c1c8880 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 >> 172.21.15.32:0/3539465962 conn(0x7f183c0a3050 msgr2=0x7f183c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 shutdown_connections 2023-09-30T22:26:52.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.129+0000 7f1841299700 1 -- 172.21.15.32:0/3539465962 wait complete. 2023-09-30T22:26:52.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.167+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f251c0a4a70 con 0x7f251c0a0a30 2023-09-30T22:26:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.167+0000 7f2512ffd700 1 -- 172.21.15.32:0/1018855778 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f250c01b3b0 con 0x7f251c0a0a30 2023-09-30T22:26:52.167 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 msgr2=0x7f251c12cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 0x7f251c12cab0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f250c00ebe0 tx=0x7f250c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 shutdown_connections 2023-09-30T22:26:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251c138d80 0x7f251c12cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f251c09d630 0x7f251c12c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 --2- 172.21.15.32:0/1018855778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251c0a0a30 0x7f251c12cab0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 >> 172.21.15.32:0/1018855778 conn(0x7f251c0a3050 msgr2=0x7f251c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 shutdown_connections 2023-09-30T22:26:52.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.170+0000 7f2524623700 1 -- 172.21.15.32:0/1018855778 wait complete. 2023-09-30T22:26:52.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:52.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:52.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:52.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.838+0000 7f064874a700 1 Processor -- start 2023-09-30T22:26:52.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.838+0000 7f064874a700 1 -- start start 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f064874a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 0x7f06400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f064874a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a1340 0x7f0640133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f064874a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f064874a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064009edd0 con 0x7f06400a0a30 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f064874a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064009e1f0 con 0x7f0640133fa0 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f064874a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f064009dc50 con 0x7f06400a1340 2023-09-30T22:26:52.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0645ce5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a1340 0x7f0640133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f06464e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 0x7f06400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35966/0 (socket says 172.21.15.32:35966) 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f06464e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 0x7f06400a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52256/0 (socket says 172.21.15.32:52256) 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 -- 172.21.15.32:0/3140594625 learned_addr learned my addr 172.21.15.32:0/3140594625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 -- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a1340 msgr2=0x7f0640133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 --2- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a1340 0x7f0640133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 -- 172.21.15.32:0/3140594625 >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 msgr2=0x7f06400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.839+0000 7f0646ce7700 1 --2- 172.21.15.32:0/3140594625 >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 0x7f06400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f0646ce7700 1 -- 172.21.15.32:0/3140594625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400a1fe0 con 0x7f0640133fa0 2023-09-30T22:26:52.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f0645ce5700 1 --2- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a1340 0x7f0640133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f06464e6700 1 --2- 172.21.15.32:0/3140594625 >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 0x7f06400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f0646ce7700 1 --2- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640134410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0638009e80 tx=0x7f063800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ee2f5979163d95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f06454e4700 1 -- 172.21.15.32:0/3140594625 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800e070 con 0x7f0640133fa0 2023-09-30T22:26:52.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f06454e4700 1 -- 172.21.15.32:0/3140594625 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063800b8d0 con 0x7f0640133fa0 2023-09-30T22:26:52.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.840+0000 7f06454e4700 1 -- 172.21.15.32:0/3140594625 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800bc80 con 0x7f0640133fa0 2023-09-30T22:26:52.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 -- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 msgr2=0x7f0640134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 --2- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640134410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0638009e80 tx=0x7f063800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:52.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 -- 172.21.15.32:0/3140594625 shutdown_connections 2023-09-30T22:26:52.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 --2- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640134410 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 --2- 172.21.15.32:0/3140594625 >> v2:172.21.15.32:3300/0 conn(0x7f06400a0a30 0x7f06400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 --2- 172.21.15.32:0/3140594625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a1340 0x7f0640133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 -- 172.21.15.32:0/3140594625 >> 172.21.15.32:0/3140594625 conn(0x7f06400a6550 msgr2=0x7f06400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 -- 172.21.15.32:0/3140594625 shutdown_connections 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.841+0000 7f064874a700 1 -- 172.21.15.32:0/3140594625 wait complete. 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.842+0000 7f064874a700 1 Processor -- start 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.842+0000 7f064874a700 1 -- start start 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f064874a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 0x7f06401200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f064874a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06400a1340 0x7f0640120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f064874a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f064874a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640124530 con 0x7f06400a1340 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f064874a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0640121d00 con 0x7f0640133fa0 2023-09-30T22:26:52.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f064874a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0640123890 con 0x7f06400a0a30 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 0x7f06401200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f0646ce7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 0x7f06401200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35604/0 (socket says 172.21.15.32:35604) 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 -- 172.21.15.32:0/992721517 learned_addr learned my addr 172.21.15.32:0/992721517 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f0645ce5700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06400a1340 0x7f0640120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 -- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 msgr2=0x7f0640120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 -- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06400a1340 msgr2=0x7f0640120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06400a1340 0x7f0640120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.843+0000 7f06464e6700 1 -- 172.21.15.32:0/992721517 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c0094a0 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f0646ce7700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f0645ce5700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06400a1340 0x7f0640120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f06464e6700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 0x7f06401200e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f063c00ec10 tx=0x7f063c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00c850 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c00c9b0 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06300094a0 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00cd20 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.844+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0638009b30 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.845+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f063c013470 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.845+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f063c03fbe0 con 0x7f06400a0a30 2023-09-30T22:26:52.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.845+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06400a0280 con 0x7f06400a0a30 2023-09-30T22:26:52.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.854+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f063c013720 con 0x7f06400a0a30 2023-09-30T22:26:52.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.879+0000 7fa2463c2700 1 Processor -- start 2023-09-30T22:26:52.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.879+0000 7fa2463c2700 1 -- start start 2023-09-30T22:26:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa2463c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2400a0a30 0x7fa2400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa2463c2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24009d630 0x7fa2400a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa2463c2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2400a2a70 con 0x7fa240133930 2023-09-30T22:26:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa2463c2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400a01d0 con 0x7fa24009d630 2023-09-30T22:26:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa2463c2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa24009e2b0 con 0x7fa2400a0a30 2023-09-30T22:26:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa23f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24009d630 0x7fa2400a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa23ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2400a0a30 0x7fa2400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa24495f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa240133930 0x7fa240133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55972/0 (socket says 172.21.15.32:55972) 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.880+0000 7fa24495f700 1 -- 172.21.15.32:0/1608742251 learned_addr learned my addr 172.21.15.32:0/1608742251 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23effd700 1 -- 172.21.15.32:0/1608742251 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068016897 0 0) 0x7fa2400a2a70 con 0x7fa240133930 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23ffff700 1 -- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24009d630 msgr2=0x7fa2400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23ffff700 1 --2- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24009d630 0x7fa2400a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23ffff700 1 -- 172.21.15.32:0/1608742251 >> v1:172.21.15.32:6789/0 conn(0x7fa240133930 legacy=0x7fa240133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23ffff700 1 -- 172.21.15.32:0/1608742251 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24009dc50 con 0x7fa2400a0a30 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23f7fe700 1 --2- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24009d630 0x7fa2400a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23ffff700 1 --2- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2400a0a30 0x7fa2400a0e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa230009870 tx=0x7fa230009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884a094ec82c3388 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23effd700 1 -- 172.21.15.32:0/1608742251 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230011830 con 0x7fa2400a0a30 2023-09-30T22:26:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23effd700 1 -- 172.21.15.32:0/1608742251 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230011990 con 0x7fa2400a0a30 2023-09-30T22:26:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.881+0000 7fa23effd700 1 -- 172.21.15.32:0/1608742251 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230011d20 con 0x7fa2400a0a30 2023-09-30T22:26:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 -- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2400a0a30 msgr2=0x7fa2400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 --2- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2400a0a30 0x7fa2400a0e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa230009870 tx=0x7fa230009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 -- 172.21.15.32:0/1608742251 shutdown_connections 2023-09-30T22:26:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 --2- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2400a0a30 0x7fa2400a0e00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 --2- 172.21.15.32:0/1608742251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa24009d630 0x7fa2400a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 -- 172.21.15.32:0/1608742251 >> 172.21.15.32:0/1608742251 conn(0x7fa2400a3050 msgr2=0x7fa2400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 -- 172.21.15.32:0/1608742251 shutdown_connections 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.882+0000 7fa2463c2700 1 -- 172.21.15.32:0/1608742251 wait complete. 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.883+0000 7fa2463c2700 1 Processor -- start 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.883+0000 7fa2463c2700 1 -- start start 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa2463c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24009d630 0x7fa2401c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa2463c2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2400a0a30 0x7fa2401c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa2463c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa240133930 0x7fa2401c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa2463c2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa240121190 con 0x7fa240133930 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa2463c2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400a5360 con 0x7fa2400a0a30 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa2463c2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa240120630 con 0x7fa24009d630 2023-09-30T22:26:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2400a0a30 0x7fa2401c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa24495f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa240133930 0x7fa2401c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2400a0a30 0x7fa2401c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:35996/0 (socket says 172.21.15.32:35996) 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23f7fe700 1 -- 172.21.15.32:0/681060494 learned_addr learned my addr 172.21.15.32:0/681060494 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23ffff700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24009d630 0x7fa2401c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23ffff700 1 -- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2400a0a30 msgr2=0x7fa2401c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23ffff700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2400a0a30 0x7fa2401c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23ffff700 1 -- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa240133930 msgr2=0x7fa2401c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23ffff700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa240133930 0x7fa2401c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa23ffff700 1 -- 172.21.15.32:0/681060494 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340094a0 con 0x7fa24009d630 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.884+0000 7fa24495f700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa240133930 0x7fa2401c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.885+0000 7fa23ffff700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24009d630 0x7fa2401c7d80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa230016000 tx=0x7fa230004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.885+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230004510 con 0x7fa24009d630 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.885+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23001f070 con 0x7fa24009d630 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.885+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2300094a0 con 0x7fa24009d630 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.885+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2300038e0 con 0x7fa24009d630 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.885+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa240122190 con 0x7fa24009d630 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.886+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa230003a40 con 0x7fa24009d630 2023-09-30T22:26:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.886+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa230021070 con 0x7fa24009d630 2023-09-30T22:26:52.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.886+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa240133410 con 0x7fa24009d630 2023-09-30T22:26:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:52.891+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa240133410 con 0x7fa24009d630 2023-09-30T22:26:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.062+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0640074ea0 con 0x7f06400a0a30 2023-09-30T22:26:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.063+0000 7f0636ffd700 1 -- 172.21.15.32:0/992721517 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f063c02b090 con 0x7f06400a0a30 2023-09-30T22:26:53.063 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 msgr2=0x7f06401200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 0x7f06401200e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f063c00ec10 tx=0x7f063c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:26:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 shutdown_connections 2023-09-30T22:26:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0640133fa0 0x7f0640120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06400a0a30 0x7f06401200e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 --2- 172.21.15.32:0/992721517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06400a1340 0x7f0640120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 >> 172.21.15.32:0/992721517 conn(0x7f06400a6550 msgr2=0x7f064009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 shutdown_connections 2023-09-30T22:26:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.066+0000 7f064874a700 1 -- 172.21.15.32:0/992721517 wait complete. 2023-09-30T22:26:53.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:53.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.106+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa240133410 con 0x7fa24009d630 2023-09-30T22:26:53.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.106+0000 7fa23cff9700 1 -- 172.21.15.32:0/681060494 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa240133410 con 0x7fa24009d630 2023-09-30T22:26:53.106 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24009d630 msgr2=0x7fa2401c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24009d630 0x7fa2401c7d80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa230016000 tx=0x7fa230004750 comp rx=0 tx=0).stop 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 shutdown_connections 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa240133930 0x7fa2401c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa24009d630 0x7fa2401c7d80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 --2- 172.21.15.32:0/681060494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2400a0a30 0x7fa2401c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 >> 172.21.15.32:0/681060494 conn(0x7fa2400a3050 msgr2=0x7fa2401bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 shutdown_connections 2023-09-30T22:26:53.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.109+0000 7fa2463c2700 1 -- 172.21.15.32:0/681060494 wait complete. 2023-09-30T22:26:53.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:53.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:53.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.304+0000 7fc403f6e700 1 Processor -- start 2023-09-30T22:26:53.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.304+0000 7fc403f6e700 1 -- start start 2023-09-30T22:26:53.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc403f6e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc09d630 0x7fc3fc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc403f6e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc138d80 0x7fc3fc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc403f6e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc0a2a70 con 0x7fc3fc0a0a30 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc403f6e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0a01d0 con 0x7fc3fc138d80 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc403f6e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc09e2b0 con 0x7fc3fc09d630 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc401509700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc09d630 0x7fc3fc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc401d0a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3fc0a0a30 0x7fc3fc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55978/0 (socket says 172.21.15.32:55978) 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc401d0a700 1 -- 172.21.15.32:0/2538975433 learned_addr learned my addr 172.21.15.32:0/2538975433 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc40250b700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc138d80 0x7fc3fc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc40250b700 1 -- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc09d630 msgr2=0x7fc3fc0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc40250b700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc09d630 0x7fc3fc0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc400d08700 1 -- 172.21.15.32:0/2538975433 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986116740 0 0) 0x7fc3fc0a2a70 con 0x7fc3fc0a0a30 2023-09-30T22:26:53.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc40250b700 1 -- 172.21.15.32:0/2538975433 >> v1:172.21.15.32:6789/0 conn(0x7fc3fc0a0a30 legacy=0x7fc3fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc40250b700 1 -- 172.21.15.32:0/2538975433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc09dc50 con 0x7fc3fc138d80 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.305+0000 7fc401509700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc09d630 0x7fc3fc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.306+0000 7fc40250b700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc138d80 0x7fc3fc13b160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009dc0 tx=0x7fc3f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e270bc8b252aa47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.306+0000 7fc400d08700 1 -- 172.21.15.32:0/2538975433 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400e070 con 0x7fc3fc138d80 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.306+0000 7fc400d08700 1 -- 172.21.15.32:0/2538975433 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f400bb20 con 0x7fc3fc138d80 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.306+0000 7fc400d08700 1 -- 172.21.15.32:0/2538975433 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4012430 con 0x7fc3fc138d80 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.306+0000 7fc403f6e700 1 -- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc138d80 msgr2=0x7fc3fc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc138d80 0x7fc3fc13b160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009dc0 tx=0x7fc3f400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:53.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 -- 172.21.15.32:0/2538975433 shutdown_connections 2023-09-30T22:26:53.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc138d80 0x7fc3fc13b160 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 --2- 172.21.15.32:0/2538975433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc09d630 0x7fc3fc0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 -- 172.21.15.32:0/2538975433 >> 172.21.15.32:0/2538975433 conn(0x7fc3fc0a3050 msgr2=0x7fc3fc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 -- 172.21.15.32:0/2538975433 shutdown_connections 2023-09-30T22:26:53.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.307+0000 7fc403f6e700 1 -- 172.21.15.32:0/2538975433 wait complete. 2023-09-30T22:26:53.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 Processor -- start 2023-09-30T22:26:53.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 -- start start 2023-09-30T22:26:53.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc09d630 0x7fc3fc1336a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 0x7fc3fc133be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 0x7fc3fc134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc134b00 con 0x7fc3fc0a0a30 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc1364e0 con 0x7fc3fc09d630 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc403f6e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc136e50 con 0x7fc3fc138d80 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc401509700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 0x7fc3fc133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc40250b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 0x7fc3fc134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc401509700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 0x7fc3fc133be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52274/0 (socket says 172.21.15.32:52274) 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc40250b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 0x7fc3fc134120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35638/0 (socket says 172.21.15.32:35638) 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc401509700 1 -- 172.21.15.32:0/4228618154 learned_addr learned my addr 172.21.15.32:0/4228618154 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:53.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.308+0000 7fc401d0a700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc09d630 0x7fc3fc1336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc40250b700 1 -- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc09d630 msgr2=0x7fc3fc1336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc40250b700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc09d630 0x7fc3fc1336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc40250b700 1 -- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 msgr2=0x7fc3fc133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc40250b700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 0x7fc3fc133be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc40250b700 1 -- 172.21.15.32:0/4228618154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0094a0 con 0x7fc3fc138d80 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc401509700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 0x7fc3fc133be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc40250b700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 0x7fc3fc134120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4016000 tx=0x7fc3f4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4012430 con 0x7fc3fc138d80 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f4020070 con 0x7fc3fc138d80 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f4009a70 con 0x7fc3fc138d80 2023-09-30T22:26:53.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4004040 con 0x7fc3fc138d80 2023-09-30T22:26:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.309+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc0a0410 con 0x7fc3fc138d80 2023-09-30T22:26:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.310+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3f40041a0 con 0x7fc3fc138d80 2023-09-30T22:26:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.310+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc0747f0 con 0x7fc3fc138d80 2023-09-30T22:26:53.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.311+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc3f4022070 con 0x7fc3fc138d80 2023-09-30T22:26:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.316+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3f401a970 con 0x7fc3fc138d80 2023-09-30T22:26:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.346+0000 7fbaad264700 1 Processor -- start 2023-09-30T22:26:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaad264700 1 -- start start 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaad264700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809ace0 0x7fbaa8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaad264700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809e1d0 0x7fbaa813b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaad264700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa809b150 con 0x7fbaa809d6f0 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaad264700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa809a500 con 0x7fbaa809e1d0 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaad264700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa809f720 con 0x7fbaa809ace0 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaa77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809e1d0 0x7fbaa813b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaa6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbaa809d6f0 0x7fbaa809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55986/0 (socket says 172.21.15.32:55986) 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaa6ffd700 1 -- 172.21.15.32:0/2481065303 learned_addr learned my addr 172.21.15.32:0/2481065303 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.347+0000 7fbaa67fc700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809ace0 0x7fbaa8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa77fe700 1 -- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809ace0 msgr2=0x7fbaa8138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa77fe700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809ace0 0x7fbaa8138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa77fe700 1 -- 172.21.15.32:0/2481065303 >> v1:172.21.15.32:6789/0 conn(0x7fbaa809d6f0 legacy=0x7fbaa809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa5ffb700 1 -- 172.21.15.32:0/2481065303 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158575166 0 0) 0x7fbaa809b150 con 0x7fbaa809d6f0 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa77fe700 1 -- 172.21.15.32:0/2481065303 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa806b490 con 0x7fbaa809e1d0 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa67fc700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809ace0 0x7fbaa8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa77fe700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809e1d0 0x7fbaa813b180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00098a0 tx=0x7fbaa0009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9a3577a3f2aa50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa5ffb700 1 -- 172.21.15.32:0/2481065303 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0011830 con 0x7fbaa809e1d0 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa5ffb700 1 -- 172.21.15.32:0/2481065303 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa0011990 con 0x7fbaa809e1d0 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.348+0000 7fbaa5ffb700 1 -- 172.21.15.32:0/2481065303 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0011d20 con 0x7fbaa809e1d0 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 -- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809e1d0 msgr2=0x7fbaa813b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809e1d0 0x7fbaa813b180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00098a0 tx=0x7fbaa0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 -- 172.21.15.32:0/2481065303 shutdown_connections 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809e1d0 0x7fbaa813b180 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 --2- 172.21.15.32:0/2481065303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809ace0 0x7fbaa8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 -- 172.21.15.32:0/2481065303 >> 172.21.15.32:0/2481065303 conn(0x7fbaa8066f50 msgr2=0x7fbaa809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 -- 172.21.15.32:0/2481065303 shutdown_connections 2023-09-30T22:26:53.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.349+0000 7fbaad264700 1 -- 172.21.15.32:0/2481065303 wait complete. 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 Processor -- start 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 -- start start 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809ace0 0x7fbaa811d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 0x7fbaa811ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa809e1d0 0x7fbaa811e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa811f900 con 0x7fbaa809e1d0 2023-09-30T22:26:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa81207a0 con 0x7fbaa809ace0 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.350+0000 7fbaad264700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa809c6f0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 0x7fbaa811ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 0x7fbaa811ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35648/0 (socket says 172.21.15.32:35648) 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 -- 172.21.15.32:0/2944190384 learned_addr learned my addr 172.21.15.32:0/2944190384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa77fe700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa809e1d0 0x7fbaa811e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa6ffd700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809ace0 0x7fbaa811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 -- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809ace0 msgr2=0x7fbaa811d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809ace0 0x7fbaa811d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 -- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa809e1d0 msgr2=0x7fbaa811e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa809e1d0 0x7fbaa811e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 -- 172.21.15.32:0/2944190384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c0094a0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa77fe700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa809e1d0 0x7fbaa811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaa67fc700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 0x7fbaa811ddd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00ebe0 tx=0x7fba9c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c00c850 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c00c9b0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c00cd40 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.351+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa00094a0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.352+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa811fbc0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.353+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa80747f0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.353+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba9c013470 con 0x7fbaa809d6f0 2023-09-30T22:26:53.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.353+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba9c03fae0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.358+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba9c013720 con 0x7fbaa809d6f0 2023-09-30T22:26:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.524+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc3fc072980 con 0x7fc3fc138d80 2023-09-30T22:26:53.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.524+0000 7fc3f27fc700 1 -- 172.21.15.32:0/4228618154 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3f402c020 con 0x7fc3fc138d80 2023-09-30T22:26:53.524 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 msgr2=0x7fc3fc134120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 0x7fc3fc134120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4016000 tx=0x7fc3f4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 shutdown_connections 2023-09-30T22:26:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3fc138d80 0x7fc3fc134120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3fc09d630 0x7fc3fc1336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 --2- 172.21.15.32:0/4228618154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3fc0a0a30 0x7fc3fc133be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 >> 172.21.15.32:0/4228618154 conn(0x7fc3fc0a3050 msgr2=0x7fc3fc0a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 shutdown_connections 2023-09-30T22:26:53.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.527+0000 7fc403f6e700 1 -- 172.21.15.32:0/4228618154 wait complete. 2023-09-30T22:26:53.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:53.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.562+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbaa80008d0 con 0x7fbaa809d6f0 2023-09-30T22:26:53.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.562+0000 7fba8f7fe700 1 -- 172.21.15.32:0/2944190384 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba9c017030 con 0x7fbaa809d6f0 2023-09-30T22:26:53.562 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 msgr2=0x7fbaa811ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 0x7fbaa811ddd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00ebe0 tx=0x7fba9c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 shutdown_connections 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa809e1d0 0x7fbaa811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa809ace0 0x7fbaa811d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 --2- 172.21.15.32:0/2944190384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa809d6f0 0x7fbaa811ddd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 >> 172.21.15.32:0/2944190384 conn(0x7fbaa8066f50 msgr2=0x7fbaa80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 shutdown_connections 2023-09-30T22:26:53.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:53.565+0000 7fbaad264700 1 -- 172.21.15.32:0/2944190384 wait complete. 2023-09-30T22:26:53.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:54.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:54.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:54.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.236+0000 7fee70d53700 1 Processor -- start 2023-09-30T22:26:54.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.236+0000 7fee70d53700 1 -- start start 2023-09-30T22:26:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee70d53700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee70d53700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 0x7fee6c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee70d53700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee6c133930 0x7fee6c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee70d53700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c0a2a70 con 0x7fee6c133930 2023-09-30T22:26:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee70d53700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0a01d0 con 0x7fee6c09d630 2023-09-30T22:26:54.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee70d53700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c09e2b0 con 0x7fee6c0a0a30 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee69d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 0x7fee6c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee6a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee6ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee6c133930 0x7fee6c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee69d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 0x7fee6c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36064/0 (socket says 172.21.15.32:36064) 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.237+0000 7fee69d9b700 1 -- 172.21.15.32:0/1257402534 learned_addr learned my addr 172.21.15.32:0/1257402534 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6a59c700 1 -- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 msgr2=0x7fee6c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6a59c700 1 --2- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 0x7fee6c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6a59c700 1 -- 172.21.15.32:0/1257402534 >> v2:172.21.15.32:3300/0 conn(0x7fee6c133930 msgr2=0x7fee6c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6a59c700 1 --2- 172.21.15.32:0/1257402534 >> v2:172.21.15.32:3300/0 conn(0x7fee6c133930 0x7fee6c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6a59c700 1 -- 172.21.15.32:0/1257402534 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c09dc50 con 0x7fee6c0a0a30 2023-09-30T22:26:54.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee69d9b700 1 --2- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 0x7fee6c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6ad9d700 1 --2- 172.21.15.32:0/1257402534 >> v2:172.21.15.32:3300/0 conn(0x7fee6c133930 0x7fee6c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6a59c700 1 --2- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c0a0e00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25d2057367ba8e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6959a700 1 -- 172.21.15.32:0/1257402534 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00e070 con 0x7fee6c0a0a30 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6959a700 1 -- 172.21.15.32:0/1257402534 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c00bb20 con 0x7fee6c0a0a30 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.238+0000 7fee6959a700 1 -- 172.21.15.32:0/1257402534 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c012430 con 0x7fee6c0a0a30 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 -- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 msgr2=0x7fee6c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 --2- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c0a0e00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009d90 tx=0x7fee5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 -- 172.21.15.32:0/1257402534 shutdown_connections 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 --2- 172.21.15.32:0/1257402534 >> v2:172.21.15.32:3300/0 conn(0x7fee6c133930 0x7fee6c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 --2- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c0a0e00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 --2- 172.21.15.32:0/1257402534 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c09d630 0x7fee6c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 -- 172.21.15.32:0/1257402534 >> 172.21.15.32:0/1257402534 conn(0x7fee6c0a3050 msgr2=0x7fee6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 -- 172.21.15.32:0/1257402534 shutdown_connections 2023-09-30T22:26:54.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.239+0000 7fee70d53700 1 -- 172.21.15.32:0/1257402534 wait complete. 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.240+0000 7fee70d53700 1 Processor -- start 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.240+0000 7fee70d53700 1 -- start start 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee70d53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 0x7fee6c1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee70d53700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c1c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee70d53700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c133930 0x7fee6c1c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee70d53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c122500 con 0x7fee6c09d630 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee70d53700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c120790 con 0x7fee6c133930 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee70d53700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c120560 con 0x7fee6c0a0a30 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 0x7fee6c1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 0x7fee6c1c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52304/0 (socket says 172.21.15.32:52304) 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 -- 172.21.15.32:0/338524517 learned_addr learned my addr 172.21.15.32:0/338524517 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6ad9d700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c133930 0x7fee6c1c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee69d9b700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c1c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 -- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 msgr2=0x7fee6c1c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c1c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 -- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c133930 msgr2=0x7fee6c1c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c133930 0x7fee6c1c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.241+0000 7fee6a59c700 1 -- 172.21.15.32:0/338524517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee60009550 con 0x7fee6c09d630 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.242+0000 7fee6a59c700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 0x7fee6c1c8090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee6c09e3f0 tx=0x7fee5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.242+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c012430 con 0x7fee6c09d630 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.242+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c020070 con 0x7fee6c09d630 2023-09-30T22:26:54.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.242+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640094a0 con 0x7fee6c09d630 2023-09-30T22:26:54.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.242+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004280 con 0x7fee6c09d630 2023-09-30T22:26:54.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.242+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c009a70 con 0x7fee6c09d630 2023-09-30T22:26:54.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.243+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee5c0043e0 con 0x7fee6c09d630 2023-09-30T22:26:54.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.243+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c0a0410 con 0x7fee6c09d630 2023-09-30T22:26:54.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.243+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fee5c022070 con 0x7fee6c09d630 2023-09-30T22:26:54.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.252+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee5c01c780 con 0x7fee6c09d630 2023-09-30T22:26:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.276+0000 7f94716fd700 1 Processor -- start 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.276+0000 7f94716fd700 1 -- start start 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f94716fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f946c09d630 0x7f946c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f94716fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c138d80 0x7f946c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f94716fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0a2a70 con 0x7f946c0a0a30 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f94716fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0a01d0 con 0x7f946c09d630 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f94716fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c09e2b0 con 0x7f946c138d80 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f946c09d630 0x7f946c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f946c0a0a30 0x7f946c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55988/0 (socket says 172.21.15.32:55988) 2023-09-30T22:26:54.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946affd700 1 -- 172.21.15.32:0/197429009 learned_addr learned my addr 172.21.15.32:0/197429009 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c138d80 0x7f946c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946a7fc700 1 -- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c138d80 msgr2=0x7f946c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946a7fc700 1 --2- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c138d80 0x7f946c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946a7fc700 1 -- 172.21.15.32:0/197429009 >> v1:172.21.15.32:6789/0 conn(0x7f946c0a0a30 legacy=0x7f946c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946a7fc700 1 -- 172.21.15.32:0/197429009 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c09dc50 con 0x7f946c09d630 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f9469ffb700 1 -- 172.21.15.32:0/197429009 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245725833 0 0) 0x7f946c0a2a70 con 0x7f946c0a0a30 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.277+0000 7f946b7fe700 1 --2- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c138d80 0x7f946c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f946a7fc700 1 --2- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c0a1510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9460009840 tx=0x7f9460009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192a06ea47b05ac0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f9469ffb700 1 -- 172.21.15.32:0/197429009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460011830 con 0x7f946c09d630 2023-09-30T22:26:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f9469ffb700 1 -- 172.21.15.32:0/197429009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460011990 con 0x7f946c09d630 2023-09-30T22:26:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f9469ffb700 1 -- 172.21.15.32:0/197429009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460011d20 con 0x7f946c09d630 2023-09-30T22:26:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 -- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 msgr2=0x7f946c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 --2- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c0a1510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9460009840 tx=0x7f9460009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 -- 172.21.15.32:0/197429009 shutdown_connections 2023-09-30T22:26:54.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 --2- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c138d80 0x7f946c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 --2- 172.21.15.32:0/197429009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c0a1510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 -- 172.21.15.32:0/197429009 >> 172.21.15.32:0/197429009 conn(0x7f946c0a3050 msgr2=0x7f946c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.278+0000 7f94716fd700 1 -- 172.21.15.32:0/197429009 shutdown_connections 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.279+0000 7f94716fd700 1 -- 172.21.15.32:0/197429009 wait complete. 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.280+0000 7f94716fd700 1 Processor -- start 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.280+0000 7f94716fd700 1 -- start start 2023-09-30T22:26:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f94716fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f94716fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c0a0a30 0x7f946c133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f94716fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f946c138d80 0x7f946c134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f94716fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c136190 con 0x7f946c09d630 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f94716fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c136eb0 con 0x7f946c0a0a30 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f94716fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0a4ba0 con 0x7f946c138d80 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f946c138d80 0x7f946c134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52318/0 (socket says 172.21.15.32:52318) 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946affd700 1 -- 172.21.15.32:0/3006558487 learned_addr learned my addr 172.21.15.32:0/3006558487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946a7fc700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c0a0a30 0x7f946c133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946b7fe700 1 -- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c0a0a30 msgr2=0x7f946c133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946b7fe700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c0a0a30 0x7f946c133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946b7fe700 1 -- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 msgr2=0x7f946c133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946b7fe700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946b7fe700 1 -- 172.21.15.32:0/3006558487 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640094a0 con 0x7f946c138d80 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946a7fc700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c0a0a30 0x7f946c133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.281+0000 7f946affd700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.282+0000 7f946b7fe700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f946c138d80 0x7f946c134180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f946400e900 tx=0x7f946400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.282+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464013680 con 0x7f946c138d80 2023-09-30T22:26:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.282+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9464013800 con 0x7f946c138d80 2023-09-30T22:26:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.282+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464013bb0 con 0x7f946c138d80 2023-09-30T22:26:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.282+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600094a0 con 0x7f946c138d80 2023-09-30T22:26:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.282+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c136540 con 0x7f946c138d80 2023-09-30T22:26:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.283+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9464013d10 con 0x7f946c138d80 2023-09-30T22:26:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.283+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f946403fa10 con 0x7f946c138d80 2023-09-30T22:26:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.283+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0747f0 con 0x7f946c138d80 2023-09-30T22:26:54.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.288+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9464017080 con 0x7f946c138d80 2023-09-30T22:26:54.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.460+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fee6c1222b0 con 0x7fee6c09d630 2023-09-30T22:26:54.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.460+0000 7fee5affd700 1 -- 172.21.15.32:0/338524517 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee5c025b70 con 0x7fee6c09d630 2023-09-30T22:26:54.460 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.462+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 msgr2=0x7fee6c1c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.462+0000 7fee70d53700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 0x7fee6c1c8090 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee6c09e3f0 tx=0x7fee5c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.462+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 shutdown_connections 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.462+0000 7fee70d53700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee6c133930 0x7fee6c1c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.463+0000 7fee70d53700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee6c09d630 0x7fee6c1c8090 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.463+0000 7fee70d53700 1 --2- 172.21.15.32:0/338524517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee6c0a0a30 0x7fee6c1c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.463+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 >> 172.21.15.32:0/338524517 conn(0x7fee6c0a3050 msgr2=0x7fee6c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.463+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 shutdown_connections 2023-09-30T22:26:54.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.463+0000 7fee70d53700 1 -- 172.21.15.32:0/338524517 wait complete. 2023-09-30T22:26:54.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:54.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.491+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f946c137980 con 0x7f946c138d80 2023-09-30T22:26:54.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.492+0000 7f94537fe700 1 -- 172.21.15.32:0/3006558487 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f946401adc0 con 0x7f946c138d80 2023-09-30T22:26:54.492 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f946c138d80 msgr2=0x7f946c134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f946c138d80 0x7f946c134180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f946400e900 tx=0x7f946400ec10 comp rx=0 tx=0).stop 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 shutdown_connections 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f946c138d80 0x7f946c134180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f946c09d630 0x7f946c133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 --2- 172.21.15.32:0/3006558487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f946c0a0a30 0x7f946c133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 >> 172.21.15.32:0/3006558487 conn(0x7f946c0a3050 msgr2=0x7f946c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 shutdown_connections 2023-09-30T22:26:54.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.494+0000 7f94716fd700 1 -- 172.21.15.32:0/3006558487 wait complete. 2023-09-30T22:26:54.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:54.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:54.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:54.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.689+0000 7fe728301700 1 Processor -- start 2023-09-30T22:26:54.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.689+0000 7fe728301700 1 -- start start 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe728301700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7200a1510 0x7fe720138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe728301700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe72013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe728301700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72009edd0 con 0x7fe7200a0a30 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe728301700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200a18e0 con 0x7fe7200a1510 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe728301700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72009e1f0 con 0x7fe720139250 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe72589c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7200a1510 0x7fe720138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe72609d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7200a0a30 0x7fe7200a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55992/0 (socket says 172.21.15.32:55992) 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe72689e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe72013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.690+0000 7fe72609d700 1 -- 172.21.15.32:0/3943539892 learned_addr learned my addr 172.21.15.32:0/3943539892 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72589c700 1 -- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 msgr2=0x7fe72013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72589c700 1 --2- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe72013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72589c700 1 -- 172.21.15.32:0/3943539892 >> v1:172.21.15.32:6789/0 conn(0x7fe7200a0a30 legacy=0x7fe7200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72589c700 1 -- 172.21.15.32:0/3943539892 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72009dc50 con 0x7fe7200a1510 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72689e700 1 --2- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe72013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72589c700 1 --2- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a1510 0x7fe720138d10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe710009840 tx=0x7fe710009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31825215ffe4934 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72509b700 1 -- 172.21.15.32:0/3943539892 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710011830 con 0x7fe7200a1510 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72509b700 1 -- 172.21.15.32:0/3943539892 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe710011990 con 0x7fe7200a1510 2023-09-30T22:26:54.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.691+0000 7fe72509b700 1 -- 172.21.15.32:0/3943539892 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710011d20 con 0x7fe7200a1510 2023-09-30T22:26:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 -- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a1510 msgr2=0x7fe720138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 --2- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a1510 0x7fe720138d10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe710009840 tx=0x7fe710009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 -- 172.21.15.32:0/3943539892 shutdown_connections 2023-09-30T22:26:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 --2- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe72013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 --2- 172.21.15.32:0/3943539892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a1510 0x7fe720138d10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 -- 172.21.15.32:0/3943539892 >> 172.21.15.32:0/3943539892 conn(0x7fe7200a6550 msgr2=0x7fe7200a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 -- 172.21.15.32:0/3943539892 shutdown_connections 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.692+0000 7fe728301700 1 -- 172.21.15.32:0/3943539892 wait complete. 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.693+0000 7fe728301700 1 Processor -- start 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.693+0000 7fe728301700 1 -- start start 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.693+0000 7fe728301700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a0a30 0x7fe7201218c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe728301700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 0x7fe720121e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe728301700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe720122340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe728301700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7201236c0 con 0x7fe7200a0a30 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe728301700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7201234d0 con 0x7fe720139250 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe728301700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7201253e0 con 0x7fe7200a1510 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72689e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe720122340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 0x7fe720121e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72689e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe720122340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36100/0 (socket says 172.21.15.32:36100) 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 0x7fe720121e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35676/0 (socket says 172.21.15.32:35676) 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72689e700 1 -- 172.21.15.32:0/846944544 learned_addr learned my addr 172.21.15.32:0/846944544 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 -- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 msgr2=0x7fe720122340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe720122340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 -- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a0a30 msgr2=0x7fe7201218c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a0a30 0x7fe7201218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 -- 172.21.15.32:0/846944544 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718009a70 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe72589c700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 0x7fe720121e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe710016000 tx=0x7fe710004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.694+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7100036a0 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.695+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71001f070 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.695+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710003a60 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.695+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100094a0 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.695+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe720125100 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.695+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe710003bc0 con 0x7fe7200a1510 2023-09-30T22:26:54.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.696+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe710021070 con 0x7fe7200a1510 2023-09-30T22:26:54.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.696+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe720124f60 con 0x7fe7200a1510 2023-09-30T22:26:54.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.701+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe720124f60 con 0x7fe7200a1510 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.731+0000 7fc011541700 1 Processor -- start 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc011541700 1 -- start start 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc011541700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc00c12cc20 0x7fc00c12f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc011541700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12cff0 0x7fc00c137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc011541700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c09a510 con 0x7fc00c12c2f0 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc011541700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c12c7d0 con 0x7fc00c12cc20 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc011541700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c09c190 con 0x7fc00c12cff0 2023-09-30T22:26:54.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc00affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc00c12cc20 0x7fc00c12f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc00b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12cff0 0x7fc00c137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc00a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc00c12c2f0 0x7fc00c12fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56006/0 (socket says 172.21.15.32:56006) 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.732+0000 7fc00a7fc700 1 -- 172.21.15.32:0/1841110880 learned_addr learned my addr 172.21.15.32:0/1841110880 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc00affd700 1 -- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12cff0 msgr2=0x7fc00c137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc00affd700 1 --2- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12cff0 0x7fc00c137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc00affd700 1 -- 172.21.15.32:0/1841110880 >> v1:172.21.15.32:6789/0 conn(0x7fc00c12c2f0 legacy=0x7fc00c12fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc00affd700 1 -- 172.21.15.32:0/1841110880 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c09b540 con 0x7fc00c12cc20 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc009ffb700 1 -- 172.21.15.32:0/1841110880 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028312081 0 0) 0x7fc00c09a510 con 0x7fc00c12c2f0 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc00b7fe700 1 --2- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12cff0 0x7fc00c137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc00affd700 1 --2- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c12f930 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b8f0 tx=0x7fc00400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652ccdf62c5a1b22 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc009ffb700 1 -- 172.21.15.32:0/1841110880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400c9f0 con 0x7fc00c12cc20 2023-09-30T22:26:54.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc009ffb700 1 -- 172.21.15.32:0/1841110880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400cb50 con 0x7fc00c12cc20 2023-09-30T22:26:54.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.733+0000 7fc009ffb700 1 -- 172.21.15.32:0/1841110880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400f460 con 0x7fc00c12cc20 2023-09-30T22:26:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 -- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 msgr2=0x7fc00c12f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 --2- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c12f930 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b8f0 tx=0x7fc00400bc00 comp rx=0 tx=0).stop 2023-09-30T22:26:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 -- 172.21.15.32:0/1841110880 shutdown_connections 2023-09-30T22:26:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 --2- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12cff0 0x7fc00c137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 --2- 172.21.15.32:0/1841110880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c12f930 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 -- 172.21.15.32:0/1841110880 >> 172.21.15.32:0/1841110880 conn(0x7fc00c09f470 msgr2=0x7fc00c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 -- 172.21.15.32:0/1841110880 shutdown_connections 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.734+0000 7fc011541700 1 -- 172.21.15.32:0/1841110880 wait complete. 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 Processor -- start 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 -- start start 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12c2f0 0x7fc00c06f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 0x7fc00c1c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0716c0 con 0x7fc00c12cc20 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c098f20 con 0x7fc00c12cff0 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc011541700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c12b720 con 0x7fc00c12c2f0 2023-09-30T22:26:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.735+0000 7fc00b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 0x7fc00c1c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 0x7fc00c1c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36116/0 (socket says 172.21.15.32:36116) 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 -- 172.21.15.32:0/2508197586 learned_addr learned my addr 172.21.15.32:0/2508197586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00affd700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12c2f0 0x7fc00c06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00a7fc700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 -- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12c2f0 msgr2=0x7fc00c06f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12c2f0 0x7fc00c06f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 -- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 msgr2=0x7fc00c06faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c06faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 -- 172.21.15.32:0/2508197586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc00f050 con 0x7fc00c12cff0 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc00b7fe700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 0x7fc00c1c3e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00f500 tx=0x7fbffc00f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc01d070 con 0x7fc00c12cff0 2023-09-30T22:26:54.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc013040 con 0x7fc00c12cff0 2023-09-30T22:26:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00400b550 con 0x7fc00c12cff0 2023-09-30T22:26:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc012050 con 0x7fc00c12cff0 2023-09-30T22:26:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.736+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c0722e0 con 0x7fc00c12cff0 2023-09-30T22:26:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.737+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbffc022410 con 0x7fc00c12cff0 2023-09-30T22:26:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.737+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbffc043d10 con 0x7fc00c12cff0 2023-09-30T22:26:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.737+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c071f30 con 0x7fc00c12cff0 2023-09-30T22:26:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.742+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbffc0226c0 con 0x7fc00c12cff0 2023-09-30T22:26:54.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.907+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe720124f40 con 0x7fe7200a1510 2023-09-30T22:26:54.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.908+0000 7fe716ffd700 1 -- 172.21.15.32:0/846944544 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe71002b020 con 0x7fe7200a1510 2023-09-30T22:26:54.908 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 msgr2=0x7fe720121e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 0x7fe720121e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe710016000 tx=0x7fe710004750 comp rx=0 tx=0).stop 2023-09-30T22:26:54.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 shutdown_connections 2023-09-30T22:26:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe720139250 0x7fe720122340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7200a0a30 0x7fe7201218c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 --2- 172.21.15.32:0/846944544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7200a1510 0x7fe720121e00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 >> 172.21.15.32:0/846944544 conn(0x7fe7200a6550 msgr2=0x7fe720137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 shutdown_connections 2023-09-30T22:26:54.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.910+0000 7fe728301700 1 -- 172.21.15.32:0/846944544 wait complete. 2023-09-30T22:26:54.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.950+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc00c070f00 con 0x7fc00c12cff0 2023-09-30T22:26:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.950+0000 7fc0037fe700 1 -- 172.21.15.32:0/2508197586 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbffc020320 con 0x7fc00c12cff0 2023-09-30T22:26:54.951 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 msgr2=0x7fc00c1c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 0x7fc00c1c3e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00f500 tx=0x7fbffc00f810 comp rx=0 tx=0).stop 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 shutdown_connections 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc00c12cff0 0x7fc00c1c3e00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc00c12c2f0 0x7fc00c06f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 --2- 172.21.15.32:0/2508197586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc00c12cc20 0x7fc00c06faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 >> 172.21.15.32:0/2508197586 conn(0x7fc00c09f470 msgr2=0x7fc00c12ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 shutdown_connections 2023-09-30T22:26:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:54.953+0000 7fc011541700 1 -- 172.21.15.32:0/2508197586 wait complete. 2023-09-30T22:26:54.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:55.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:55.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:55.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 Processor -- start 2023-09-30T22:26:55.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 -- start start 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 0x7f24a0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a009edd0 con 0x7f24a00a0a30 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a00a18e0 con 0x7f24a00a1510 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a6782700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24a009e1f0 con 0x7f24a0139250 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f249f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 0x7f24a0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a4d1f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f249ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f24a00a0a30 0x7f24a00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56014/0 (socket says 172.21.15.32:56014) 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f249f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 0x7f24a0138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52366/0 (socket says 172.21.15.32:52366) 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f24a4d1f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35698/0 (socket says 172.21.15.32:35698) 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.626+0000 7f249ffff700 1 -- 172.21.15.32:0/1042258795 learned_addr learned my addr 172.21.15.32:0/1042258795 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f24a4d1f700 1 -- 172.21.15.32:0/1042258795 >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 msgr2=0x7f24a0138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f24a4d1f700 1 --2- 172.21.15.32:0/1042258795 >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 0x7f24a0138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f24a4d1f700 1 -- 172.21.15.32:0/1042258795 >> v1:172.21.15.32:6789/0 conn(0x7f24a00a0a30 legacy=0x7f24a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f24a4d1f700 1 -- 172.21.15.32:0/1042258795 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a009dc50 con 0x7f24a0139250 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f249f7fe700 1 --2- 172.21.15.32:0/1042258795 >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 0x7f24a0138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f24a4d1f700 1 --2- 172.21.15.32:0/1042258795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a013b630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2498009d90 tx=0x7f249800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ad115c53ebbf50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f249effd700 1 -- 172.21.15.32:0/1042258795 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800e070 con 0x7f24a0139250 2023-09-30T22:26:55.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f249effd700 1 -- 172.21.15.32:0/1042258795 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800bb20 con 0x7f24a0139250 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.627+0000 7f249effd700 1 -- 172.21.15.32:0/1042258795 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498012430 con 0x7f24a0139250 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 -- 172.21.15.32:0/1042258795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 msgr2=0x7f24a013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 --2- 172.21.15.32:0/1042258795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a013b630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2498009d90 tx=0x7f249800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 -- 172.21.15.32:0/1042258795 shutdown_connections 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 --2- 172.21.15.32:0/1042258795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a013b630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 --2- 172.21.15.32:0/1042258795 >> v2:172.21.15.32:3300/0 conn(0x7f24a00a1510 0x7f24a0138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 -- 172.21.15.32:0/1042258795 >> 172.21.15.32:0/1042258795 conn(0x7f24a00a6550 msgr2=0x7f24a00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 -- 172.21.15.32:0/1042258795 shutdown_connections 2023-09-30T22:26:55.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.628+0000 7f24a6782700 1 -- 172.21.15.32:0/1042258795 wait complete. 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.629+0000 7f24a6782700 1 Processor -- start 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.629+0000 7f24a6782700 1 -- start start 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a6782700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 0x7f24a01c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a6782700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24a00a1510 0x7f24a01c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a6782700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a01c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a6782700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0124820 con 0x7f24a00a0a30 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a6782700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0125d40 con 0x7f24a00a1510 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a6782700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01253a0 con 0x7f24a0139250 2023-09-30T22:26:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 0x7f24a01c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 0x7f24a01c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52372/0 (socket says 172.21.15.32:52372) 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 -- 172.21.15.32:0/204808619 learned_addr learned my addr 172.21.15.32:0/204808619 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249f7fe700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24a00a1510 0x7f24a01c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f24a4d1f700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a01c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 -- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 msgr2=0x7f24a01c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a01c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 -- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24a00a1510 msgr2=0x7f24a01c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24a00a1510 0x7f24a01c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 -- 172.21.15.32:0/204808619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24940094a0 con 0x7f24a00a0a30 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249f7fe700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24a00a1510 0x7f24a01c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.630+0000 7f249ffff700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 0x7f24a01c8540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f249000c520 tx=0x7f249000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.631+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490005d70 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.631+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24900070d0 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.631+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498009a70 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.631+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490007440 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.631+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a01262d0 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.631+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24900075a0 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.632+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f249003ea60 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.632+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a01264c0 con 0x7f24a00a0a30 2023-09-30T22:26:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.637+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2490019b40 con 0x7f24a00a0a30 2023-09-30T22:26:55.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.664+0000 7f31573cd700 1 Processor -- start 2023-09-30T22:26:55.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.664+0000 7f31573cd700 1 -- start start 2023-09-30T22:26:55.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f31573cd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315009d630 0x7f31500a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f31573cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f31573cd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31500a2a70 con 0x7f31500a0a30 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f31573cd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31500a01d0 con 0x7f315009d630 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f31573cd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315009e2b0 con 0x7f3150138d80 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f315596a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3154968700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315009d630 0x7f31500a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3155169700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31500a0a30 0x7f31500a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56028/0 (socket says 172.21.15.32:56028) 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3155169700 1 -- 172.21.15.32:0/3283051663 learned_addr learned my addr 172.21.15.32:0/3283051663 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:55.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3147fff700 1 -- 172.21.15.32:0/3283051663 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233803072 0 0) 0x7f31500a2a70 con 0x7f31500a0a30 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3147fff700 1 -- 172.21.15.32:0/3283051663 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f313c0035b0 con 0x7f31500a0a30 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3154968700 1 -- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 msgr2=0x7f315013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3154968700 1 --2- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3154968700 1 -- 172.21.15.32:0/3283051663 >> v1:172.21.15.32:6789/0 conn(0x7f31500a0a30 legacy=0x7f31500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3154968700 1 -- 172.21.15.32:0/3283051663 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315009dc50 con 0x7f315009d630 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f315596a700 1 --2- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.665+0000 7f3154968700 1 --2- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d630 0x7f31500a1510 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3140009840 tx=0x7f3140009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df58dc1509f03b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f3147fff700 1 -- 172.21.15.32:0/3283051663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140011830 con 0x7f315009d630 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f3147fff700 1 -- 172.21.15.32:0/3283051663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3140011990 con 0x7f315009d630 2023-09-30T22:26:55.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f3147fff700 1 -- 172.21.15.32:0/3283051663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140011d20 con 0x7f315009d630 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 -- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d630 msgr2=0x7f31500a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 --2- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d630 0x7f31500a1510 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3140009840 tx=0x7f3140009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 -- 172.21.15.32:0/3283051663 shutdown_connections 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 --2- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 --2- 172.21.15.32:0/3283051663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315009d630 0x7f31500a1510 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 -- 172.21.15.32:0/3283051663 >> 172.21.15.32:0/3283051663 conn(0x7f31500a3050 msgr2=0x7f31500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 -- 172.21.15.32:0/3283051663 shutdown_connections 2023-09-30T22:26:55.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.666+0000 7f31573cd700 1 -- 172.21.15.32:0/3283051663 wait complete. 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.667+0000 7f31573cd700 1 Processor -- start 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.667+0000 7f31573cd700 1 -- start start 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f31573cd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009d630 0x7f315011c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f31573cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 0x7f315011cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f31573cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315011d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f31573cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315009d320 con 0x7f31500a0a30 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f31573cd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315011f660 con 0x7f3150138d80 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f31573cd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f315011fbf0 con 0x7f315009d630 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 0x7f315011cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 0x7f315011cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52396/0 (socket says 172.21.15.32:52396) 2023-09-30T22:26:55.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3155169700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009d630 0x7f315011c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 -- 172.21.15.32:0/3396531776 learned_addr learned my addr 172.21.15.32:0/3396531776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f315596a700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315011d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 -- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009d630 msgr2=0x7f315011c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009d630 0x7f315011c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 -- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 msgr2=0x7f315011d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315011d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.668+0000 7f3154968700 1 -- 172.21.15.32:0/3396531776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148009a70 con 0x7f31500a0a30 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.669+0000 7f3154968700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 0x7f315011cd00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f31400090b0 tx=0x7f3140004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.669+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140004380 con 0x7f31500a0a30 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.669+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314001f070 con 0x7f31500a0a30 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.669+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31400094a0 con 0x7f31500a0a30 2023-09-30T22:26:55.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.669+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31400036a0 con 0x7f31500a0a30 2023-09-30T22:26:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.669+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3150120660 con 0x7f31500a0a30 2023-09-30T22:26:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.670+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315011f870 con 0x7f31500a0a30 2023-09-30T22:26:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.671+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3140021030 con 0x7f31500a0a30 2023-09-30T22:26:55.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.671+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3140040b70 con 0x7f31500a0a30 2023-09-30T22:26:55.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.679+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3140003d40 con 0x7f31500a0a30 2023-09-30T22:26:55.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.850+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f24a0074ea0 con 0x7f24a00a0a30 2023-09-30T22:26:55.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.851+0000 7f249cff9700 1 -- 172.21.15.32:0/204808619 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2490017030 con 0x7f24a00a0a30 2023-09-30T22:26:55.851 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 msgr2=0x7f24a01c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 0x7f24a01c8540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f249000c520 tx=0x7f249000c830 comp rx=0 tx=0).stop 2023-09-30T22:26:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 shutdown_connections 2023-09-30T22:26:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24a0139250 0x7f24a01c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24a00a0a30 0x7f24a01c8540 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 --2- 172.21.15.32:0/204808619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24a00a1510 0x7f24a01c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 >> 172.21.15.32:0/204808619 conn(0x7f24a00a6550 msgr2=0x7f24a00a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 shutdown_connections 2023-09-30T22:26:55.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.853+0000 7f24a6782700 1 -- 172.21.15.32:0/204808619 wait complete. 2023-09-30T22:26:55.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:55.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.885+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f315011f820 con 0x7f31500a0a30 2023-09-30T22:26:55.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.885+0000 7f3145ffb700 1 -- 172.21.15.32:0/3396531776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3140015030 con 0x7f31500a0a30 2023-09-30T22:26:55.886 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:55.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 msgr2=0x7f315011cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 0x7f315011cd00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f31400090b0 tx=0x7f3140004750 comp rx=0 tx=0).stop 2023-09-30T22:26:55.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 shutdown_connections 2023-09-30T22:26:55.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3150138d80 0x7f315011d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315009d630 0x7f315011c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 --2- 172.21.15.32:0/3396531776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31500a0a30 0x7f315011cd00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:55.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 >> 172.21.15.32:0/3396531776 conn(0x7f31500a3050 msgr2=0x7f315013aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 shutdown_connections 2023-09-30T22:26:55.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:55.888+0000 7f31573cd700 1 -- 172.21.15.32:0/3396531776 wait complete. 2023-09-30T22:26:55.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:55.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:55.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 Processor -- start 2023-09-30T22:26:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 -- start start 2023-09-30T22:26:56.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 0x7f1e5c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c09edd0 con 0x7f1e5c133fa0 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c09e1f0 con 0x7f1e5c0a0a30 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e61e3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c09dc50 con 0x7f1e5c0a1340 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e5b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 0x7f1e5c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e5affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e5b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 0x7f1e5c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36148/0 (socket says 172.21.15.32:36148) 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e5b7fe700 1 -- 172.21.15.32:0/1489406015 learned_addr learned my addr 172.21.15.32:0/1489406015 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:56.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.084+0000 7f1e5affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35738/0 (socket says 172.21.15.32:35738) 2023-09-30T22:26:56.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5b7fe700 1 -- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 msgr2=0x7f1e5c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:56.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5b7fe700 1 --2- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5b7fe700 1 -- 172.21.15.32:0/1489406015 >> v1:172.21.15.32:6789/0 conn(0x7f1e5c133fa0 legacy=0x7f1e5c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5a7fc700 1 -- 172.21.15.32:0/1489406015 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328347672 0 0) 0x7f1e5c09edd0 con 0x7f1e5c133fa0 2023-09-30T22:26:56.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5b7fe700 1 -- 172.21.15.32:0/1489406015 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0a1fe0 con 0x7f1e5c0a0a30 2023-09-30T22:26:56.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5affd700 1 --2- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5b7fe700 1 --2- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 0x7f1e5c0a0e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009e80 tx=0x7f1e4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d45ce43cec2cee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5a7fc700 1 -- 172.21.15.32:0/1489406015 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00e070 con 0x7f1e5c0a0a30 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5a7fc700 1 -- 172.21.15.32:0/1489406015 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c00b910 con 0x7f1e5c0a0a30 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.085+0000 7f1e5a7fc700 1 -- 172.21.15.32:0/1489406015 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00bcc0 con 0x7f1e5c0a0a30 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 -- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 msgr2=0x7f1e5c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 0x7f1e5c0a0e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009e80 tx=0x7f1e4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 -- 172.21.15.32:0/1489406015 shutdown_connections 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c0a0a30 0x7f1e5c0a0e00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/1489406015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 -- 172.21.15.32:0/1489406015 >> 172.21.15.32:0/1489406015 conn(0x7f1e5c0a6550 msgr2=0x7f1e5c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 -- 172.21.15.32:0/1489406015 shutdown_connections 2023-09-30T22:26:56.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.086+0000 7f1e61e3a700 1 -- 172.21.15.32:0/1489406015 wait complete. 2023-09-30T22:26:56.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.087+0000 7f1e61e3a700 1 Processor -- start 2023-09-30T22:26:56.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.087+0000 7f1e61e3a700 1 -- start start 2023-09-30T22:26:56.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e61e3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e5c0a0a30 0x7f1e5c1c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e61e3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c1c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e61e3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c133fa0 0x7f1e5c1c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e61e3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c126370 con 0x7f1e5c0a0a30 2023-09-30T22:26:56.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e61e3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c124660 con 0x7f1e5c133fa0 2023-09-30T22:26:56.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e61e3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c124c10 con 0x7f1e5c0a1340 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c1c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c1c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35748/0 (socket says 172.21.15.32:35748) 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 -- 172.21.15.32:0/616848092 learned_addr learned my addr 172.21.15.32:0/616848092 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5bfff700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c133fa0 0x7f1e5c1c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5b7fe700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e5c0a0a30 0x7f1e5c1c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 -- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c133fa0 msgr2=0x7f1e5c1c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c133fa0 0x7f1e5c1c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 -- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e5c0a0a30 msgr2=0x7f1e5c1c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e5c0a0a30 0x7f1e5c1c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5affd700 1 -- 172.21.15.32:0/616848092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e500094a0 con 0x7f1e5c0a1340 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5bfff700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c133fa0 0x7f1e5c1c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.088+0000 7f1e5b7fe700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e5c0a0a30 0x7f1e5c1c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.089+0000 7f1e5affd700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c1c88c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000ebe0 tx=0x7f1e5000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.089+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000c850 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.089+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5000c9b0 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.089+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000cd40 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.089+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c009b30 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.089+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0a25e0 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.090+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e50013470 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.090+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e5003f710 con 0x7f1e5c0a1340 2023-09-30T22:26:56.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.090+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c074ea0 con 0x7f1e5c0a1340 2023-09-30T22:26:56.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.095+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e50013720 con 0x7f1e5c0a1340 2023-09-30T22:26:56.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.125+0000 7faa25ac9700 1 Processor -- start 2023-09-30T22:26:56.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.125+0000 7faa25ac9700 1 -- start start 2023-09-30T22:26:56.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa25ac9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa200a0a30 0x7faa200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa25ac9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa2009d630 0x7faa200a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa25ac9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa200a2a70 con 0x7faa20133930 2023-09-30T22:26:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa25ac9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200a01d0 con 0x7faa200a0a30 2023-09-30T22:26:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa25ac9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa2009e2b0 con 0x7faa2009d630 2023-09-30T22:26:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa20133930 0x7faa20133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56046/0 (socket says 172.21.15.32:56046) 2023-09-30T22:26:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1ffff700 1 -- 172.21.15.32:0/47596188 learned_addr learned my addr 172.21.15.32:0/47596188 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1f7fe700 1 --2- 172.21.15.32:0/47596188 >> v2:172.21.15.32:3300/0 conn(0x7faa200a0a30 0x7faa200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1effd700 1 --2- 172.21.15.32:0/47596188 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa2009d630 0x7faa200a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1e7fc700 1 -- 172.21.15.32:0/47596188 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633546797 0 0) 0x7faa200a2a70 con 0x7faa20133930 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1e7fc700 1 -- 172.21.15.32:0/47596188 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa080035b0 con 0x7faa20133930 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1effd700 1 -- 172.21.15.32:0/47596188 >> v2:172.21.15.32:3300/0 conn(0x7faa200a0a30 msgr2=0x7faa200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1effd700 1 --2- 172.21.15.32:0/47596188 >> v2:172.21.15.32:3300/0 conn(0x7faa200a0a30 0x7faa200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1effd700 1 -- 172.21.15.32:0/47596188 >> v1:172.21.15.32:6789/0 conn(0x7faa20133930 legacy=0x7faa20133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1effd700 1 -- 172.21.15.32:0/47596188 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2009dc50 con 0x7faa2009d630 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.126+0000 7faa1f7fe700 1 --2- 172.21.15.32:0/47596188 >> v2:172.21.15.32:3300/0 conn(0x7faa200a0a30 0x7faa200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa1effd700 1 --2- 172.21.15.32:0/47596188 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa2009d630 0x7faa200a1340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa14009840 tx=0x7faa14009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70abba9cf2240433 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa1e7fc700 1 -- 172.21.15.32:0/47596188 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14011830 con 0x7faa2009d630 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa1e7fc700 1 -- 172.21.15.32:0/47596188 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14011990 con 0x7faa2009d630 2023-09-30T22:26:56.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa1e7fc700 1 -- 172.21.15.32:0/47596188 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14011d20 con 0x7faa2009d630 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa25ac9700 1 -- 172.21.15.32:0/47596188 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa2009d630 msgr2=0x7faa200a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa25ac9700 1 --2- 172.21.15.32:0/47596188 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa2009d630 0x7faa200a1340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa14009840 tx=0x7faa14009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa25ac9700 1 -- 172.21.15.32:0/47596188 shutdown_connections 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa25ac9700 1 --2- 172.21.15.32:0/47596188 >> v2:172.21.15.32:3300/0 conn(0x7faa200a0a30 0x7faa200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa25ac9700 1 --2- 172.21.15.32:0/47596188 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa2009d630 0x7faa200a1340 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.127+0000 7faa25ac9700 1 -- 172.21.15.32:0/47596188 >> 172.21.15.32:0/47596188 conn(0x7faa200a3050 msgr2=0x7faa200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.128+0000 7faa25ac9700 1 -- 172.21.15.32:0/47596188 shutdown_connections 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.128+0000 7faa25ac9700 1 -- 172.21.15.32:0/47596188 wait complete. 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.128+0000 7faa25ac9700 1 Processor -- start 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 -- start start 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 0x7faa201c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa200a0a30 0x7faa201c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa20133930 0x7faa201c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa20120dc0 con 0x7faa2009d630 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa201211d0 con 0x7faa200a0a30 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa25ac9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa20120650 con 0x7faa20133930 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 0x7faa201c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa1effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa200a0a30 0x7faa201c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 0x7faa201c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52420/0 (socket says 172.21.15.32:52420) 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa20133930 0x7faa201c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.129+0000 7faa1f7fe700 1 -- 172.21.15.32:0/652069022 learned_addr learned my addr 172.21.15.32:0/652069022 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa1f7fe700 1 -- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa20133930 msgr2=0x7faa201c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa1f7fe700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa20133930 0x7faa201c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa1f7fe700 1 -- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa200a0a30 msgr2=0x7faa201c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa1f7fe700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa200a0a30 0x7faa201c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa1f7fe700 1 -- 172.21.15.32:0/652069022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa10009a70 con 0x7faa2009d630 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa1f7fe700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 0x7faa201c8100 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7faa10009f60 tx=0x7faa10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000cdc0 con 0x7faa2009d630 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1000f040 con 0x7faa2009d630 2023-09-30T22:26:56.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa140094a0 con 0x7faa2009d630 2023-09-30T22:26:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100146a0 con 0x7faa2009d630 2023-09-30T22:26:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.130+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200a0410 con 0x7faa2009d630 2023-09-30T22:26:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.131+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa10014800 con 0x7faa2009d630 2023-09-30T22:26:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.131+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa10040380 con 0x7faa2009d630 2023-09-30T22:26:56.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.131+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20122060 con 0x7faa2009d630 2023-09-30T22:26:56.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.136+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa1001d400 con 0x7faa2009d630 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.304+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1e5c0a28e0 con 0x7f1e5c0a1340 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.304+0000 7f1e43fff700 1 -- 172.21.15.32:0/616848092 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e50021020 con 0x7f1e5c0a1340 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 msgr2=0x7f1e5c1c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c1c88c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000ebe0 tx=0x7f1e5000eef0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 shutdown_connections 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e5c133fa0 0x7f1e5c1c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e5c0a0a30 0x7f1e5c1c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 --2- 172.21.15.32:0/616848092 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e5c0a1340 0x7f1e5c1c88c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 >> 172.21.15.32:0/616848092 conn(0x7f1e5c0a6550 msgr2=0x7f1e5c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 shutdown_connections 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.307+0000 7f1e61e3a700 1 -- 172.21.15.32:0/616848092 wait complete. 2023-09-30T22:26:56.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.352+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa200747f0 con 0x7faa2009d630 2023-09-30T22:26:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.352+0000 7faa07fff700 1 -- 172.21.15.32:0/652069022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa1002c090 con 0x7faa2009d630 2023-09-30T22:26:56.352 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 msgr2=0x7faa201c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 0x7faa201c8100 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7faa10009f60 tx=0x7faa10009f90 comp rx=0 tx=0).stop 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 shutdown_connections 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa20133930 0x7faa201c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa2009d630 0x7faa201c8100 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 --2- 172.21.15.32:0/652069022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa200a0a30 0x7faa201c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.354+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 >> 172.21.15.32:0/652069022 conn(0x7faa200a3050 msgr2=0x7faa201bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.355+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 shutdown_connections 2023-09-30T22:26:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:56.355+0000 7faa25ac9700 1 -- 172.21.15.32:0/652069022 wait complete. 2023-09-30T22:26:56.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:56.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:56.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:57.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.017+0000 7fafd79ff700 1 Processor -- start 2023-09-30T22:26:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.017+0000 7fafd79ff700 1 -- start start 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd79ff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafd009e930 0x7fafd009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd79ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd79ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009f650 0x7fafd0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd79ff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd0070940 con 0x7fafd009e930 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd79ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd006fd60 con 0x7fafd009f240 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd79ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafd006fa80 con 0x7fafd009f650 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafd009e930 0x7fafd009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd4f9a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fafd009e930 0x7fafd009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52436/0 (socket says 172.21.15.32:52436) 2023-09-30T22:26:57.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd4f9a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36166/0 (socket says 172.21.15.32:36166) 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 -- 172.21.15.32:0/3507730831 learned_addr learned my addr 172.21.15.32:0/3507730831 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd5f9c700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009f650 0x7fafd0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 -- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009f650 msgr2=0x7fafd0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009f650 0x7fafd0131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 -- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 msgr2=0x7fafd01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd579b700 1 -- 172.21.15.32:0/3507730831 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd009fb60 con 0x7fafd009e930 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.018+0000 7fafd4f9a700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.019+0000 7fafd579b700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009e930 0x7fafd009ed00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009e50 tx=0x7fafc000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f926398b9482d686 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.019+0000 7fafcffff700 1 -- 172.21.15.32:0/3507730831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000e070 con 0x7fafd009e930 2023-09-30T22:26:57.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.019+0000 7fafcffff700 1 -- 172.21.15.32:0/3507730831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000b910 con 0x7fafd009e930 2023-09-30T22:26:57.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.019+0000 7fafcffff700 1 -- 172.21.15.32:0/3507730831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000bcc0 con 0x7fafd009e930 2023-09-30T22:26:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.019+0000 7fafd79ff700 1 -- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009e930 msgr2=0x7fafd009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.019+0000 7fafd79ff700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009e930 0x7fafd009ed00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fafc0009e50 tx=0x7fafc000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 -- 172.21.15.32:0/3507730831 shutdown_connections 2023-09-30T22:26:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009f650 0x7fafd0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009e930 0x7fafd009ed00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 --2- 172.21.15.32:0/3507730831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 -- 172.21.15.32:0/3507730831 >> 172.21.15.32:0/3507730831 conn(0x7fafd00a4210 msgr2=0x7fafd00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 -- 172.21.15.32:0/3507730831 shutdown_connections 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.020+0000 7fafd79ff700 1 -- 172.21.15.32:0/3507730831 wait complete. 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.021+0000 7fafd79ff700 1 Processor -- start 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.021+0000 7fafd79ff700 1 -- start start 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd79ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009e930 0x7fafd01a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd79ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd79ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009f650 0x7fafd01a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd79ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd01a5aa0 con 0x7fafd009f650 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd79ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd01a6cf0 con 0x7fafd009f240 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd79ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafd01a6080 con 0x7fafd009e930 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd4f9a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd5f9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009f650 0x7fafd01a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd4f9a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36174/0 (socket says 172.21.15.32:36174) 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd4f9a700 1 -- 172.21.15.32:0/2523622772 learned_addr learned my addr 172.21.15.32:0/2523622772 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd579b700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009e930 0x7fafd01a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd579b700 1 -- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 msgr2=0x7fafd01a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd579b700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd579b700 1 -- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009f650 msgr2=0x7fafd01a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd579b700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009f650 0x7fafd01a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd579b700 1 -- 172.21.15.32:0/2523622772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40094a0 con 0x7fafd009e930 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd5f9c700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009f650 0x7fafd01a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.022+0000 7fafd4f9a700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.023+0000 7fafd579b700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009e930 0x7fafd01a3ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fafc0015000 tx=0x7fafc0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.023+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00092e0 con 0x7fafd009e930 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.023+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000cea0 con 0x7fafd009e930 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.023+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc0009b30 con 0x7fafd009e930 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.023+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0004490 con 0x7fafd009e930 2023-09-30T22:26:57.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.023+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd00a0190 con 0x7fafd009e930 2023-09-30T22:26:57.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.024+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fafc00045f0 con 0x7fafd009e930 2023-09-30T22:26:57.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.024+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fafc0040970 con 0x7fafd009e930 2023-09-30T22:26:57.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.024+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd00a35b0 con 0x7fafd009e930 2023-09-30T22:26:57.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.029+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafc0011bf0 con 0x7fafd009e930 2023-09-30T22:26:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.056+0000 7f4a3e4dc700 1 Processor -- start 2023-09-30T22:26:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.056+0000 7f4a3e4dc700 1 -- start start 2023-09-30T22:26:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.056+0000 7f4a3e4dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 0x7f4a380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.056+0000 7f4a3e4dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a3813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.056+0000 7f4a3e4dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380a2a70 con 0x7f4a3809d630 2023-09-30T22:26:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.056+0000 7f4a3e4dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380a01d0 con 0x7f4a38138d80 2023-09-30T22:26:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3e4dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3809e2b0 con 0x7f4a380a0a30 2023-09-30T22:26:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 0x7f4a380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3ca79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a3813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 0x7f4a380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35802/0 (socket says 172.21.15.32:35802) 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a37fff700 1 -- 172.21.15.32:0/2758826914 learned_addr learned my addr 172.21.15.32:0/2758826914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3ca79700 1 -- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 msgr2=0x7f4a380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3ca79700 1 --2- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 0x7f4a380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3ca79700 1 -- 172.21.15.32:0/2758826914 >> v1:172.21.15.32:6789/0 conn(0x7f4a3809d630 legacy=0x7f4a380a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3ca79700 1 -- 172.21.15.32:0/2758826914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3809dc50 con 0x7f4a38138d80 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a36ffd700 1 -- 172.21.15.32:0/2758826914 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206627697 0 0) 0x7f4a380a2a70 con 0x7f4a3809d630 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a37fff700 1 --2- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 0x7f4a380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a3ca79700 1 --2- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a3813b160 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009dc0 tx=0x7f4a3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9fcebc4cfec51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a36ffd700 1 -- 172.21.15.32:0/2758826914 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000e070 con 0x7f4a38138d80 2023-09-30T22:26:57.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a36ffd700 1 -- 172.21.15.32:0/2758826914 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000bb20 con 0x7f4a38138d80 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.057+0000 7f4a36ffd700 1 -- 172.21.15.32:0/2758826914 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30012430 con 0x7f4a38138d80 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 msgr2=0x7f4a3813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a3813b160 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009dc0 tx=0x7f4a3000b330 comp rx=0 tx=0).stop 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/2758826914 shutdown_connections 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a3813b160 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/2758826914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a380a0a30 0x7f4a380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/2758826914 >> 172.21.15.32:0/2758826914 conn(0x7f4a380a3050 msgr2=0x7f4a380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/2758826914 shutdown_connections 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.058+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/2758826914 wait complete. 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.059+0000 7f4a3e4dc700 1 Processor -- start 2023-09-30T22:26:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.059+0000 7f4a3e4dc700 1 -- start start 2023-09-30T22:26:57.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3e4dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a3809d630 0x7f4a38133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3e4dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a380a0a30 0x7f4a38133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3e4dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a38134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3e4dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38134bd0 con 0x7f4a380a0a30 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3e4dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38136140 con 0x7f4a38138d80 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3e4dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38136e60 con 0x7f4a3809d630 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a38134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a3809d630 0x7f4a38133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a38134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36204/0 (socket says 172.21.15.32:36204) 2023-09-30T22:26:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 -- 172.21.15.32:0/356558218 learned_addr learned my addr 172.21.15.32:0/356558218 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a377fe700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a380a0a30 0x7f4a38133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 -- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a3809d630 msgr2=0x7f4a38133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a3809d630 0x7f4a38133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 -- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a380a0a30 msgr2=0x7f4a38133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a380a0a30 0x7f4a38133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a3ca79700 1 -- 172.21.15.32:0/356558218 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009550 con 0x7f4a38138d80 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.060+0000 7f4a37fff700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a3809d630 0x7f4a38133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.061+0000 7f4a3ca79700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a38134180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4a380a2c00 tx=0x7f4a30004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.061+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30012430 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.061+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30020070 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.061+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30004040 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.061+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a30009a70 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.061+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a381364f0 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.062+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a300041a0 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.062+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a380747f0 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.062+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a30022070 con 0x7f4a38138d80 2023-09-30T22:26:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.067+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a30004450 con 0x7f4a38138d80 2023-09-30T22:26:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.238+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fafd01a0b30 con 0x7fafd009e930 2023-09-30T22:26:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.239+0000 7fafcdffb700 1 -- 172.21.15.32:0/2523622772 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafc0024e60 con 0x7fafd009e930 2023-09-30T22:26:57.239 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009e930 msgr2=0x7fafd01a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009e930 0x7fafd01a3ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fafc0015000 tx=0x7fafc0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 shutdown_connections 2023-09-30T22:26:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafd009f650 0x7fafd01a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafd009e930 0x7fafd01a3ee0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 --2- 172.21.15.32:0/2523622772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafd009f240 0x7fafd01a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 >> 172.21.15.32:0/2523622772 conn(0x7fafd00a4210 msgr2=0x7fafd01b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 shutdown_connections 2023-09-30T22:26:57.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.241+0000 7fafd79ff700 1 -- 172.21.15.32:0/2523622772 wait complete. 2023-09-30T22:26:57.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.272+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4a38072980 con 0x7f4a38138d80 2023-09-30T22:26:57.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.272+0000 7f4a34ff9700 1 -- 172.21.15.32:0/356558218 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a3001a380 con 0x7f4a38138d80 2023-09-30T22:26:57.273 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.274+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 msgr2=0x7f4a38134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.274+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a38134180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4a380a2c00 tx=0x7f4a30004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 shutdown_connections 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a38138d80 0x7f4a38134180 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a3809d630 0x7f4a38133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 --2- 172.21.15.32:0/356558218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a380a0a30 0x7f4a38133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 >> 172.21.15.32:0/356558218 conn(0x7f4a380a3050 msgr2=0x7f4a3809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 shutdown_connections 2023-09-30T22:26:57.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.275+0000 7f4a3e4dc700 1 -- 172.21.15.32:0/356558218 wait complete. 2023-09-30T22:26:57.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:57.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:57.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:57.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.473+0000 7f2e2a4af700 1 Processor -- start 2023-09-30T22:26:57.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.473+0000 7f2e2a4af700 1 -- start start 2023-09-30T22:26:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e2a4af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e2a4af700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e2409d630 0x7f2e240a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e2a4af700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e240a2a70 con 0x7f2e24133930 2023-09-30T22:26:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e2a4af700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240a01d0 con 0x7f2e2409d630 2023-09-30T22:26:57.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e2a4af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2409e2b0 con 0x7f2e240a0a30 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e237fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e2409d630 0x7f2e240a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35832/0 (socket says 172.21.15.32:35832) 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e28a4c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2e24133930 0x7f2e24133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56052/0 (socket says 172.21.15.32:56052) 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e23fff700 1 -- 172.21.15.32:0/1538055594 learned_addr learned my addr 172.21.15.32:0/1538055594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e23fff700 1 -- 172.21.15.32:0/1538055594 >> v2:172.21.15.32:3300/0 conn(0x7f2e2409d630 msgr2=0x7f2e240a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e23fff700 1 --2- 172.21.15.32:0/1538055594 >> v2:172.21.15.32:3300/0 conn(0x7f2e2409d630 0x7f2e240a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.474+0000 7f2e23fff700 1 -- 172.21.15.32:0/1538055594 >> v1:172.21.15.32:6789/0 conn(0x7f2e24133930 legacy=0x7f2e24133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e23fff700 1 -- 172.21.15.32:0/1538055594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2409dc50 con 0x7f2e240a0a30 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1538055594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509747782 0 0) 0x7f2e240a2a70 con 0x7f2e24133930 2023-09-30T22:26:57.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e237fe700 1 --2- 172.21.15.32:0/1538055594 >> v2:172.21.15.32:3300/0 conn(0x7f2e2409d630 0x7f2e240a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e23fff700 1 --2- 172.21.15.32:0/1538055594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2e140098f0 tx=0x7f2e14009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd20660c5f994803 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1538055594 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14011830 con 0x7f2e240a0a30 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1538055594 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e14011990 con 0x7f2e240a0a30 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.475+0000 7f2e22ffd700 1 -- 172.21.15.32:0/1538055594 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14011d20 con 0x7f2e240a0a30 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 -- 172.21.15.32:0/1538055594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 msgr2=0x7f2e240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/1538055594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2e140098f0 tx=0x7f2e14009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 -- 172.21.15.32:0/1538055594 shutdown_connections 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/1538055594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e240a0a30 0x7f2e240a0e00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/1538055594 >> v2:172.21.15.32:3300/0 conn(0x7f2e2409d630 0x7f2e240a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 -- 172.21.15.32:0/1538055594 >> 172.21.15.32:0/1538055594 conn(0x7f2e240a3050 msgr2=0x7f2e240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 -- 172.21.15.32:0/1538055594 shutdown_connections 2023-09-30T22:26:57.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.476+0000 7f2e2a4af700 1 -- 172.21.15.32:0/1538055594 wait complete. 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.477+0000 7f2e2a4af700 1 Processor -- start 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 -- start start 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2409d630 0x7f2e241c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 0x7f2e241c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e24133930 0x7f2e241c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24121e90 con 0x7f2e2409d630 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24121290 con 0x7f2e240a0a30 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e2a4af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24120520 con 0x7f2e24133930 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 0x7f2e241c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e28a4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e24133930 0x7f2e241c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 0x7f2e241c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36206/0 (socket says 172.21.15.32:36206) 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.478+0000 7f2e237fe700 1 -- 172.21.15.32:0/2843619222 learned_addr learned my addr 172.21.15.32:0/2843619222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e23fff700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2409d630 0x7f2e241c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e28a4c700 1 -- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 msgr2=0x7f2e241c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e28a4c700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 0x7f2e241c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e28a4c700 1 -- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2409d630 msgr2=0x7f2e241c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e28a4c700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2409d630 0x7f2e241c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e28a4c700 1 -- 172.21.15.32:0/2843619222 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e180094a0 con 0x7f2e24133930 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e237fe700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 0x7f2e241c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e28a4c700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e24133930 0x7f2e241c8970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00cbe0 tx=0x7f2e1c00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c00aa40 con 0x7f2e24133930 2023-09-30T22:26:57.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c00abc0 con 0x7f2e24133930 2023-09-30T22:26:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14009550 con 0x7f2e24133930 2023-09-30T22:26:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0070d0 con 0x7f2e24133930 2023-09-30T22:26:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.479+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240a0410 con 0x7f2e24133930 2023-09-30T22:26:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.480+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e1c007230 con 0x7f2e24133930 2023-09-30T22:26:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.480+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e1c03f0b0 con 0x7f2e24133930 2023-09-30T22:26:57.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.480+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24133420 con 0x7f2e24133930 2023-09-30T22:26:57.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.485+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e1c018020 con 0x7f2e24133930 2023-09-30T22:26:57.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.513+0000 7f80dc2c0700 1 Processor -- start 2023-09-30T22:26:57.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.513+0000 7f80dc2c0700 1 -- start start 2023-09-30T22:26:57.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80dc2c0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d40a0a30 0x7f80d40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80dc2c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d409d630 0x7f80d40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80dc2c0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40a2a70 con 0x7f80d4133930 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80dc2c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40a01d0 con 0x7f80d409d630 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80dc2c0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d409e2b0 con 0x7f80d40a0a30 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da05c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d40a0a30 0x7f80d40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da85d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f80d4133930 0x7f80d4133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56054/0 (socket says 172.21.15.32:56054) 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80d985b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d409d630 0x7f80d40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da85d700 1 -- 172.21.15.32:0/680623863 learned_addr learned my addr 172.21.15.32:0/680623863 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da05c700 1 -- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d409d630 msgr2=0x7f80d40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da05c700 1 --2- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d409d630 0x7f80d40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da05c700 1 -- 172.21.15.32:0/680623863 >> v1:172.21.15.32:6789/0 conn(0x7f80d4133930 legacy=0x7f80d4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80d905a700 1 -- 172.21.15.32:0/680623863 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664451104 0 0) 0x7f80d40a2a70 con 0x7f80d4133930 2023-09-30T22:26:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da05c700 1 -- 172.21.15.32:0/680623863 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d409dc50 con 0x7f80d40a0a30 2023-09-30T22:26:57.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80d985b700 1 --2- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d409d630 0x7f80d40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.514+0000 7f80da05c700 1 --2- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d40a0a30 0x7f80d40a0e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009870 tx=0x7f80d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf47cb8708144d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80d905a700 1 -- 172.21.15.32:0/680623863 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0011830 con 0x7f80d40a0a30 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80d905a700 1 -- 172.21.15.32:0/680623863 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0011990 con 0x7f80d40a0a30 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80d905a700 1 -- 172.21.15.32:0/680623863 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0011d20 con 0x7f80d40a0a30 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 -- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d40a0a30 msgr2=0x7f80d40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d40a0a30 0x7f80d40a0e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009870 tx=0x7f80d0009b80 comp rx=0 tx=0).stop 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 -- 172.21.15.32:0/680623863 shutdown_connections 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d40a0a30 0x7f80d40a0e00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/680623863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d409d630 0x7f80d40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 -- 172.21.15.32:0/680623863 >> 172.21.15.32:0/680623863 conn(0x7f80d40a3050 msgr2=0x7f80d40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 -- 172.21.15.32:0/680623863 shutdown_connections 2023-09-30T22:26:57.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.515+0000 7f80dc2c0700 1 -- 172.21.15.32:0/680623863 wait complete. 2023-09-30T22:26:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.516+0000 7f80dc2c0700 1 Processor -- start 2023-09-30T22:26:57.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.516+0000 7f80dc2c0700 1 -- start start 2023-09-30T22:26:57.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80dc2c0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 0x7f80d41c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80dc2c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d40a0a30 0x7f80d41c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80dc2c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133930 0x7f80d41c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:57.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80dc2c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4121190 con 0x7f80d4133930 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80dc2c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40a5360 con 0x7f80d40a0a30 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80dc2c0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4120630 con 0x7f80d409d630 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 0x7f80d41c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 0x7f80d41c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35876/0 (socket says 172.21.15.32:35876) 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 -- 172.21.15.32:0/1111787825 learned_addr learned my addr 172.21.15.32:0/1111787825 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80d985b700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d40a0a30 0x7f80d41c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da85d700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133930 0x7f80d41c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 -- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d40a0a30 msgr2=0x7f80d41c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d40a0a30 0x7f80d41c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 -- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133930 msgr2=0x7f80d41c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133930 0x7f80d41c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da05c700 1 -- 172.21.15.32:0/1111787825 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40094a0 con 0x7f80d409d630 2023-09-30T22:26:57.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.517+0000 7f80da85d700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133930 0x7f80d41c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.518+0000 7f80da05c700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 0x7f80d41c7d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f80d00090b0 tx=0x7f80d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.518+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0004510 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.518+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d001f070 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.518+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d00094a0 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.518+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00038c0 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.518+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d4122190 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.519+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80d0003a20 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.519+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f80d0021070 con 0x7f80d409d630 2023-09-30T22:26:57.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.519+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d4121f60 con 0x7f80d409d630 2023-09-30T22:26:57.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.524+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80d001a430 con 0x7f80d409d630 2023-09-30T22:26:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.697+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e240a1800 con 0x7f2e24133930 2023-09-30T22:26:57.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.698+0000 7f2e20ff9700 1 -- 172.21.15.32:0/2843619222 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e1c019860 con 0x7f2e24133930 2023-09-30T22:26:57.698 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e24133930 msgr2=0x7f2e241c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e24133930 0x7f2e241c8970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00cbe0 tx=0x7f2e1c00cef0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 shutdown_connections 2023-09-30T22:26:57.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e24133930 0x7f2e241c8970 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e2409d630 0x7f2e241c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 --2- 172.21.15.32:0/2843619222 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e240a0a30 0x7f2e241c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 >> 172.21.15.32:0/2843619222 conn(0x7f2e240a3050 msgr2=0x7f2e241bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 shutdown_connections 2023-09-30T22:26:57.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.700+0000 7f2e2a4af700 1 -- 172.21.15.32:0/2843619222 wait complete. 2023-09-30T22:26:57.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:57.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.733+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f80d40747f0 con 0x7f80d409d630 2023-09-30T22:26:57.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.733+0000 7f80caffd700 1 -- 172.21.15.32:0/1111787825 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80d0015020 con 0x7f80d409d630 2023-09-30T22:26:57.733 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:57.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 msgr2=0x7f80d41c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 0x7f80d41c7d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f80d00090b0 tx=0x7f80d0004750 comp rx=0 tx=0).stop 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 shutdown_connections 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133930 0x7f80d41c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d409d630 0x7f80d41c7d80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 --2- 172.21.15.32:0/1111787825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d40a0a30 0x7f80d41c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 >> 172.21.15.32:0/1111787825 conn(0x7f80d40a3050 msgr2=0x7f80d41bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 shutdown_connections 2023-09-30T22:26:57.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:57.736+0000 7f80dc2c0700 1 -- 172.21.15.32:0/1111787825 wait complete. 2023-09-30T22:26:57.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:58.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:58.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:58.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.405+0000 7f9ad4388700 1 Processor -- start 2023-09-30T22:26:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.406+0000 7f9ad4388700 1 -- start start 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad4388700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9acc09e930 0x7f9acc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad4388700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad4388700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc070940 con 0x7f9acc09f240 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad4388700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc06fd60 con 0x7f9acc09e930 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad4388700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc06fa80 con 0x7f9acc09f650 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2124700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9acc09e930 0x7f9acc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad1923700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9acc09f240 0x7f9acc1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56066/0 (socket says 172.21.15.32:56066) 2023-09-30T22:26:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad1923700 1 -- 172.21.15.32:0/2300247899 learned_addr learned my addr 172.21.15.32:0/2300247899 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2925700 1 --2- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2124700 1 -- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 msgr2=0x7f9acc131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2124700 1 --2- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2124700 1 -- 172.21.15.32:0/2300247899 >> v1:172.21.15.32:6789/0 conn(0x7f9acc09f240 legacy=0x7f9acc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2124700 1 -- 172.21.15.32:0/2300247899 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc09fb60 con 0x7f9acc09e930 2023-09-30T22:26:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad1122700 1 -- 172.21.15.32:0/2300247899 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293582114 0 0) 0x7f9acc070940 con 0x7f9acc09f240 2023-09-30T22:26:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.407+0000 7f9ad2124700 1 --2- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc09ed00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009d90 tx=0x7f9ac800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f4ec0fa39bf861 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad1122700 1 -- 172.21.15.32:0/2300247899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800e070 con 0x7f9acc09e930 2023-09-30T22:26:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad1122700 1 -- 172.21.15.32:0/2300247899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac800bb20 con 0x7f9acc09e930 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad1122700 1 -- 172.21.15.32:0/2300247899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8012430 con 0x7f9acc09e930 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad4388700 1 -- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 msgr2=0x7f9acc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad4388700 1 --2- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc09ed00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009d90 tx=0x7f9ac800b330 comp rx=0 tx=0).stop 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad4388700 1 -- 172.21.15.32:0/2300247899 shutdown_connections 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad4388700 1 --2- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad4388700 1 --2- 172.21.15.32:0/2300247899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc09ed00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.408+0000 7f9ad4388700 1 -- 172.21.15.32:0/2300247899 >> 172.21.15.32:0/2300247899 conn(0x7f9acc0a4210 msgr2=0x7f9acc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.409+0000 7f9ad4388700 1 -- 172.21.15.32:0/2300247899 shutdown_connections 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.409+0000 7f9ad4388700 1 -- 172.21.15.32:0/2300247899 wait complete. 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.409+0000 7f9ad4388700 1 Processor -- start 2023-09-30T22:26:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 -- start start 2023-09-30T22:26:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9acc09f240 0x7f9acc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a8780 con 0x7f9acc09e930 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc06d4c0 con 0x7f9acc09f240 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad4388700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a9220 con 0x7f9acc09f650 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2925700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52474/0 (socket says 172.21.15.32:52474) 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 -- 172.21.15.32:0/3356744575 learned_addr learned my addr 172.21.15.32:0/3356744575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 -- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 msgr2=0x7f9acc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 -- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9acc09f240 msgr2=0x7f9acc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9acc09f240 0x7f9acc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.410+0000 7f9ad2124700 1 -- 172.21.15.32:0/3356744575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac8009a70 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.411+0000 7f9ad2124700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc06b8b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0a0b00 tx=0x7f9ac8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.411+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8012430 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.411+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac8020070 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.411+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc0a0190 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.411+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004280 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.411+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc071d20 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.412+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ac80043e0 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.412+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9ac8022070 con 0x7f9acc09e930 2023-09-30T22:26:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.412+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc1a0b30 con 0x7f9acc09e930 2023-09-30T22:26:58.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.417+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ac801b400 con 0x7f9acc09e930 2023-09-30T22:26:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.443+0000 7fd74d7be700 1 Processor -- start 2023-09-30T22:26:58.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.443+0000 7fd74d7be700 1 -- start start 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd74d7be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 0x7fd7480a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd74d7be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd748138d80 0x7fd74813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd74d7be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7480a2a70 con 0x7fd7480a0a30 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd74d7be700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7480a01d0 con 0x7fd748138d80 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd74d7be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74809e2b0 con 0x7fd74809d630 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd7467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 0x7fd7480a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd7477fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd748138d80 0x7fd74813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd7467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 0x7fd7480a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36226/0 (socket says 172.21.15.32:36226) 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.444+0000 7fd7467fc700 1 -- 172.21.15.32:0/3952571895 learned_addr learned my addr 172.21.15.32:0/3952571895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd7467fc700 1 -- 172.21.15.32:0/3952571895 >> v2:172.21.15.32:3300/0 conn(0x7fd748138d80 msgr2=0x7fd74813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd7467fc700 1 --2- 172.21.15.32:0/3952571895 >> v2:172.21.15.32:3300/0 conn(0x7fd748138d80 0x7fd74813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd7467fc700 1 -- 172.21.15.32:0/3952571895 >> v1:172.21.15.32:6789/0 conn(0x7fd7480a0a30 legacy=0x7fd7480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd7467fc700 1 -- 172.21.15.32:0/3952571895 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74809dc50 con 0x7fd74809d630 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd745ffb700 1 -- 172.21.15.32:0/3952571895 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962092909 0 0) 0x7fd7480a2a70 con 0x7fd7480a0a30 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd7477fe700 1 --2- 172.21.15.32:0/3952571895 >> v2:172.21.15.32:3300/0 conn(0x7fd748138d80 0x7fd74813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd7467fc700 1 --2- 172.21.15.32:0/3952571895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 0x7fd7480a1510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0098f0 tx=0x7fd73c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a73ba149dae4eec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd745ffb700 1 -- 172.21.15.32:0/3952571895 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c011830 con 0x7fd74809d630 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd745ffb700 1 -- 172.21.15.32:0/3952571895 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c011990 con 0x7fd74809d630 2023-09-30T22:26:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.445+0000 7fd745ffb700 1 -- 172.21.15.32:0/3952571895 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c011d20 con 0x7fd74809d630 2023-09-30T22:26:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 -- 172.21.15.32:0/3952571895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 msgr2=0x7fd7480a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 --2- 172.21.15.32:0/3952571895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 0x7fd7480a1510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0098f0 tx=0x7fd73c009c00 comp rx=0 tx=0).stop 2023-09-30T22:26:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 -- 172.21.15.32:0/3952571895 shutdown_connections 2023-09-30T22:26:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 --2- 172.21.15.32:0/3952571895 >> v2:172.21.15.32:3300/0 conn(0x7fd748138d80 0x7fd74813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 --2- 172.21.15.32:0/3952571895 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74809d630 0x7fd7480a1510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 -- 172.21.15.32:0/3952571895 >> 172.21.15.32:0/3952571895 conn(0x7fd7480a3050 msgr2=0x7fd7480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 -- 172.21.15.32:0/3952571895 shutdown_connections 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.446+0000 7fd74d7be700 1 -- 172.21.15.32:0/3952571895 wait complete. 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 Processor -- start 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 -- start start 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 0x7fd7481337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 0x7fd748133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd748138d80 0x7fd748134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481365f0 con 0x7fd74809d630 2023-09-30T22:26:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd748136f60 con 0x7fd7480a0a30 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd74d7be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd748136240 con 0x7fd748138d80 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd7467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 0x7fd748133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.447+0000 7fd746ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 0x7fd7481337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd7467fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 0x7fd748133cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36232/0 (socket says 172.21.15.32:36232) 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd7467fc700 1 -- 172.21.15.32:0/1903590095 learned_addr learned my addr 172.21.15.32:0/1903590095 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 0x7fd7481337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52482/0 (socket says 172.21.15.32:52482) 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd7477fe700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd748138d80 0x7fd748134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 -- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd748138d80 msgr2=0x7fd748134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd748138d80 0x7fd748134230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 -- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 msgr2=0x7fd748133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 0x7fd748133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 -- 172.21.15.32:0/1903590095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd740009a70 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd7467fc700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 0x7fd748133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd746ffd700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 0x7fd7481337b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c460 tx=0x7fd73800c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380052d0 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd738005450 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c009550 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380038c0 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.448+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74813acf0 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.449+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd738003a20 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.449+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd73803e9e0 con 0x7fd74809d630 2023-09-30T22:26:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.449+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748137a90 con 0x7fd74809d630 2023-09-30T22:26:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.454+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd738017070 con 0x7fd74809d630 2023-09-30T22:26:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.624+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9acc074ea0 con 0x7f9acc09e930 2023-09-30T22:26:58.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.624+0000 7f9ac2ffd700 1 -- 172.21.15.32:0/3356744575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ac801a380 con 0x7f9acc09e930 2023-09-30T22:26:58.625 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 msgr2=0x7f9acc06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc06b8b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0a0b00 tx=0x7f9ac8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 shutdown_connections 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9acc09f650 0x7f9acc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9acc09e930 0x7f9acc06b8b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 --2- 172.21.15.32:0/3356744575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9acc09f240 0x7f9acc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 >> 172.21.15.32:0/3356744575 conn(0x7f9acc0a4210 msgr2=0x7f9acc1b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 shutdown_connections 2023-09-30T22:26:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.627+0000 7f9ad4388700 1 -- 172.21.15.32:0/3356744575 wait complete. 2023-09-30T22:26:58.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:26:58.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.661+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd7480747f0 con 0x7fd74809d630 2023-09-30T22:26:58.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.661+0000 7fd72f7fe700 1 -- 172.21.15.32:0/1903590095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd738019cd0 con 0x7fd74809d630 2023-09-30T22:26:58.661 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:26:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 msgr2=0x7fd7481337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 0x7fd7481337b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c460 tx=0x7fd73800c770 comp rx=0 tx=0).stop 2023-09-30T22:26:58.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 shutdown_connections 2023-09-30T22:26:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd748138d80 0x7fd748134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74809d630 0x7fd7481337b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 --2- 172.21.15.32:0/1903590095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7480a0a30 0x7fd748133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 >> 172.21.15.32:0/1903590095 conn(0x7fd7480a3050 msgr2=0x7fd7480a6940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 shutdown_connections 2023-09-30T22:26:58.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.663+0000 7fd74d7be700 1 -- 172.21.15.32:0/1903590095 wait complete. 2023-09-30T22:26:58.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:26:58.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:26:58.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.857+0000 7fa019061700 1 Processor -- start 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.857+0000 7fa019061700 1 -- start start 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa019061700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0140a0a30 0x7fa0140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa019061700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014138d80 0x7fa01413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa019061700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0140a2a70 con 0x7fa01409d630 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa019061700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140a01d0 con 0x7fa0140a0a30 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa019061700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01409e2b0 con 0x7fa014138d80 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0140a0a30 0x7fa0140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa01359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014138d80 0x7fa01413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0140a0a30 0x7fa0140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52492/0 (socket says 172.21.15.32:52492) 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 -- 172.21.15.32:0/1058334249 learned_addr learned my addr 172.21.15.32:0/1058334249 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa01259c700 1 --1- 172.21.15.32:0/1058334249 >> v1:172.21.15.32:6789/0 conn(0x7fa01409d630 0x7fa0140a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56092/0 (socket says 172.21.15.32:56092) 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 -- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014138d80 msgr2=0x7fa01413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 --2- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014138d80 0x7fa01413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 -- 172.21.15.32:0/1058334249 >> v1:172.21.15.32:6789/0 conn(0x7fa01409d630 legacy=0x7fa0140a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa012d9d700 1 -- 172.21.15.32:0/1058334249 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01409dc50 con 0x7fa0140a0a30 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa011d9b700 1 -- 172.21.15.32:0/1058334249 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92489550 0 0) 0x7fa0140a2a70 con 0x7fa01409d630 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.858+0000 7fa01359e700 1 --2- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014138d80 0x7fa01413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.859+0000 7fa012d9d700 1 --2- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140a0a30 0x7fa0140a0e00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa004009e50 tx=0x7fa00400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f2adbdb30f2b82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.859+0000 7fa011d9b700 1 -- 172.21.15.32:0/1058334249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400e070 con 0x7fa0140a0a30 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.859+0000 7fa011d9b700 1 -- 172.21.15.32:0/1058334249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00400b910 con 0x7fa0140a0a30 2023-09-30T22:26:58.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.859+0000 7fa011d9b700 1 -- 172.21.15.32:0/1058334249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400bcc0 con 0x7fa0140a0a30 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.859+0000 7fa019061700 1 -- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140a0a30 msgr2=0x7fa0140a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.859+0000 7fa019061700 1 --2- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140a0a30 0x7fa0140a0e00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa004009e50 tx=0x7fa00400b330 comp rx=0 tx=0).stop 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.860+0000 7fa019061700 1 -- 172.21.15.32:0/1058334249 shutdown_connections 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.860+0000 7fa019061700 1 --2- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014138d80 0x7fa01413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.860+0000 7fa019061700 1 --2- 172.21.15.32:0/1058334249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0140a0a30 0x7fa0140a0e00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.860+0000 7fa019061700 1 -- 172.21.15.32:0/1058334249 >> 172.21.15.32:0/1058334249 conn(0x7fa0140a3050 msgr2=0x7fa0140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.860+0000 7fa019061700 1 -- 172.21.15.32:0/1058334249 shutdown_connections 2023-09-30T22:26:58.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.860+0000 7fa019061700 1 -- 172.21.15.32:0/1058334249 wait complete. 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.861+0000 7fa019061700 1 Processor -- start 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.861+0000 7fa019061700 1 -- start start 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa019061700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 0x7fa014133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa019061700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0140a0a30 0x7fa014133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa019061700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014138d80 0x7fa0141341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa019061700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014136fa0 con 0x7fa01409d630 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa019061700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa014136630 con 0x7fa0140a0a30 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa019061700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140a4b50 con 0x7fa014138d80 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 0x7fa014133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 0x7fa014133760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52498/0 (socket says 172.21.15.32:52498) 2023-09-30T22:26:58.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 -- 172.21.15.32:0/822087526 learned_addr learned my addr 172.21.15.32:0/822087526 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa01259c700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0140a0a30 0x7fa014133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa01359e700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014138d80 0x7fa0141341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 -- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014138d80 msgr2=0x7fa0141341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014138d80 0x7fa0141341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 -- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0140a0a30 msgr2=0x7fa014133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0140a0a30 0x7fa014133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 -- 172.21.15.32:0/822087526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0094a0 con 0x7fa01409d630 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.862+0000 7fa012d9d700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 0x7fa014133760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa0140a2c00 tx=0x7fa004004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.863+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004003ea0 con 0x7fa01409d630 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.863+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004004900 con 0x7fa01409d630 2023-09-30T22:26:58.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.863+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa004009b30 con 0x7fa01409d630 2023-09-30T22:26:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.863+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00401c640 con 0x7fa01409d630 2023-09-30T22:26:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.863+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014137a10 con 0x7fa01409d630 2023-09-30T22:26:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.864+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa014136c40 con 0x7fa01409d630 2023-09-30T22:26:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.864+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa00401c7a0 con 0x7fa01409d630 2023-09-30T22:26:58.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.865+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa004041440 con 0x7fa01409d630 2023-09-30T22:26:58.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.869+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa004023950 con 0x7fa01409d630 2023-09-30T22:26:58.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.901+0000 7fe2f9296700 1 Processor -- start 2023-09-30T22:26:58.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.901+0000 7fe2f9296700 1 -- start start 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f9296700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409d6f0 0x7fe2f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f9296700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2f409e000 0x7fe2f4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f9296700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e410 0x7fe2f4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f9296700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f409f520 con 0x7fe2f409e000 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f9296700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f409f130 con 0x7fe2f409e410 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f9296700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f409e930 con 0x7fe2f409d6f0 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2f409e000 0x7fe2f4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409d6f0 0x7fe2f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e410 0x7fe2f4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2f409e000 0x7fe2f4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52508/0 (socket says 172.21.15.32:52508) 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 -- 172.21.15.32:0/1653341590 learned_addr learned my addr 172.21.15.32:0/1653341590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 -- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409d6f0 msgr2=0x7fe2f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409d6f0 0x7fe2f409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 -- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e410 msgr2=0x7fe2f4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e410 0x7fe2f4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f27fc700 1 -- 172.21.15.32:0/1653341590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f409b2d0 con 0x7fe2f409e000 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.902+0000 7fe2f2ffd700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409d6f0 0x7fe2f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.903+0000 7fe2f27fc700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409e000 0x7fe2f4130660 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009920 tx=0x7fe2e8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca10fbf199d8c811 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.903+0000 7fe2f1ffb700 1 -- 172.21.15.32:0/1653341590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8011830 con 0x7fe2f409e000 2023-09-30T22:26:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.903+0000 7fe2f1ffb700 1 -- 172.21.15.32:0/1653341590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e8011990 con 0x7fe2f409e000 2023-09-30T22:26:58.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.903+0000 7fe2f1ffb700 1 -- 172.21.15.32:0/1653341590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8011d20 con 0x7fe2f409e000 2023-09-30T22:26:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.903+0000 7fe2f9296700 1 -- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409e000 msgr2=0x7fe2f4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.903+0000 7fe2f9296700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409e000 0x7fe2f4130660 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009920 tx=0x7fe2e8009c30 comp rx=0 tx=0).stop 2023-09-30T22:26:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 -- 172.21.15.32:0/1653341590 shutdown_connections 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e410 0x7fe2f4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409d6f0 0x7fe2f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 --2- 172.21.15.32:0/1653341590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409e000 0x7fe2f4130660 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 -- 172.21.15.32:0/1653341590 >> 172.21.15.32:0/1653341590 conn(0x7fe2f4066f50 msgr2=0x7fe2f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 -- 172.21.15.32:0/1653341590 shutdown_connections 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.904+0000 7fe2f9296700 1 -- 172.21.15.32:0/1653341590 wait complete. 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 Processor -- start 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 -- start start 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 0x7fe2f411e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e000 0x7fe2f411e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409e410 0x7fe2f411ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f409c700 con 0x7fe2f409d6f0 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4121ab0 con 0x7fe2f409e000 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f9296700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f411ff90 con 0x7fe2f409e410 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.905+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 0x7fe2f411e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 0x7fe2f411e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52518/0 (socket says 172.21.15.32:52518) 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e000 0x7fe2f411e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 -- 172.21.15.32:0/2741835393 learned_addr learned my addr 172.21.15.32:0/2741835393 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f37fe700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409e410 0x7fe2f411ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 -- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409e410 msgr2=0x7fe2f411ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409e410 0x7fe2f411ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 -- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e000 msgr2=0x7fe2f411e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e000 0x7fe2f411e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 -- 172.21.15.32:0/2741835393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e400bc80 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f27fc700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e000 0x7fe2f411e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f2ffd700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 0x7fe2f411e270 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40100f0 tx=0x7fe2e4010400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4012040 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e4010a80 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e8009550 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4010df0 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.906+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f40a1730 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.907+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe2e4009540 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.907+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe2e4041b70 con 0x7fe2f409d6f0 2023-09-30T22:26:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.908+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f4121700 con 0x7fe2f409d6f0 2023-09-30T22:26:58.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:58.913+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe2e40099d0 con 0x7fe2f409d6f0 2023-09-30T22:26:59.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.093+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa0140747f0 con 0x7fa01409d630 2023-09-30T22:26:59.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.094+0000 7fa00b7fe700 1 -- 172.21.15.32:0/822087526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa004023360 con 0x7fa01409d630 2023-09-30T22:26:59.094 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 msgr2=0x7fa014133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 0x7fa014133760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa0140a2c00 tx=0x7fa004004ad0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 shutdown_connections 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014138d80 0x7fa0141341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa01409d630 0x7fa014133760 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 --2- 172.21.15.32:0/822087526 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0140a0a30 0x7fa014133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 >> 172.21.15.32:0/822087526 conn(0x7fa0140a3050 msgr2=0x7fa01413ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 shutdown_connections 2023-09-30T22:26:59.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.096+0000 7fa019061700 1 -- 172.21.15.32:0/822087526 wait complete. 2023-09-30T22:26:59.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:26:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.132+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe2f41225e0 con 0x7fe2f409d6f0 2023-09-30T22:26:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.132+0000 7fe2db7fe700 1 -- 172.21.15.32:0/2741835393 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe2e4015360 con 0x7fe2f409d6f0 2023-09-30T22:26:59.132 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:26:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 msgr2=0x7fe2f411e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 0x7fe2f411e270 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40100f0 tx=0x7fe2e4010400 comp rx=0 tx=0).stop 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 shutdown_connections 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe2f409e410 0x7fe2f411ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2f409d6f0 0x7fe2f411e270 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 --2- 172.21.15.32:0/2741835393 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe2f409e000 0x7fe2f411e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 >> 172.21.15.32:0/2741835393 conn(0x7fe2f4066f50 msgr2=0x7fe2f406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 shutdown_connections 2023-09-30T22:26:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.134+0000 7fe2f9296700 1 -- 172.21.15.32:0/2741835393 wait complete. 2023-09-30T22:26:59.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:26:59.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:26:59.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:26:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.792+0000 7f7e09c19700 1 Processor -- start 2023-09-30T22:26:59.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.792+0000 7f7e09c19700 1 -- start start 2023-09-30T22:26:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e09c19700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409d6f0 0x7f7e0409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e09c19700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e04134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e09c19700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0409b120 con 0x7f7e0409e000 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e09c19700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0406b460 con 0x7f7e0409e410 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e09c19700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0409a7e0 con 0x7f7e0409d6f0 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409d6f0 0x7f7e0409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e02ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7e0409e000 0x7f7e041339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56108/0 (socket says 172.21.15.32:56108) 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e02ffd700 1 -- 172.21.15.32:0/2446329815 learned_addr learned my addr 172.21.15.32:0/2446329815 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e03fff700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e04134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e027fc700 1 -- 172.21.15.32:0/2446329815 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408396225 0 0) 0x7f7e0409b120 con 0x7f7e0409e000 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e037fe700 1 -- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 msgr2=0x7f7e04134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e037fe700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e04134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e037fe700 1 -- 172.21.15.32:0/2446329815 >> v1:172.21.15.32:6789/0 conn(0x7f7e0409e000 legacy=0x7f7e041339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e037fe700 1 -- 172.21.15.32:0/2446329815 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0409a500 con 0x7f7e0409d6f0 2023-09-30T22:26:59.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e03fff700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e04134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.793+0000 7f7e037fe700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409d6f0 0x7f7e0409dac0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7df40098a0 tx=0x7f7df4009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffdd4d8b34aaa56a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e027fc700 1 -- 172.21.15.32:0/2446329815 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4011830 con 0x7f7e0409d6f0 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e027fc700 1 -- 172.21.15.32:0/2446329815 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df4011990 con 0x7f7e0409d6f0 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e027fc700 1 -- 172.21.15.32:0/2446329815 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4011d20 con 0x7f7e0409d6f0 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 -- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409d6f0 msgr2=0x7f7e0409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409d6f0 0x7f7e0409dac0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7df40098a0 tx=0x7f7df4009bb0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 -- 172.21.15.32:0/2446329815 shutdown_connections 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e04134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 --2- 172.21.15.32:0/2446329815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409d6f0 0x7f7e0409dac0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 -- 172.21.15.32:0/2446329815 >> 172.21.15.32:0/2446329815 conn(0x7f7e04066f50 msgr2=0x7f7e0409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:59.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 -- 172.21.15.32:0/2446329815 shutdown_connections 2023-09-30T22:26:59.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.794+0000 7f7e09c19700 1 -- 172.21.15.32:0/2446329815 wait complete. 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.795+0000 7f7e09c19700 1 Processor -- start 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 -- start start 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0409d6f0 0x7f7e04122630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 0x7f7e04122b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e041230b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04125d40 con 0x7f7e0409d6f0 2023-09-30T22:26:59.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04126020 con 0x7f7e0409e410 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e09c19700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040a19d0 con 0x7f7e0409e000 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e02ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 0x7f7e04122b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e03fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e041230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e02ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 0x7f7e04122b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35976/0 (socket says 172.21.15.32:35976) 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e02ffd700 1 -- 172.21.15.32:0/3847378152 learned_addr learned my addr 172.21.15.32:0/3847378152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.796+0000 7f7e037fe700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0409d6f0 0x7f7e04122630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e02ffd700 1 -- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 msgr2=0x7f7e041230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e02ffd700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e041230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e02ffd700 1 -- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0409d6f0 msgr2=0x7f7e04122630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e02ffd700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0409d6f0 0x7f7e04122630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e02ffd700 1 -- 172.21.15.32:0/3847378152 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc009a70 con 0x7f7e0409e000 2023-09-30T22:26:59.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e03fff700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e041230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e02ffd700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 0x7f7e04122b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7df8001eb0 tx=0x7f7df800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8004c80 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df8004de0 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df40094a0 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8007310 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.797+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e04125360 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.798+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7df8007470 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.798+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7df803f2b0 con 0x7f7e0409e000 2023-09-30T22:26:59.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.798+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e04074ea0 con 0x7f7e0409e000 2023-09-30T22:26:59.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.803+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7df80123b0 con 0x7f7e0409e000 2023-09-30T22:26:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.831+0000 7fb905d1a700 1 Processor -- start 2023-09-30T22:26:59.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.831+0000 7fb905d1a700 1 -- start start 2023-09-30T22:26:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb905d1a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9000a0a30 0x7fb9000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb905d1a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb90009d630 0x7fb9000a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb905d1a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9000a2a70 con 0x7fb900133930 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb905d1a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9000a01d0 con 0x7fb90009d630 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb905d1a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90009e2b0 con 0x7fb9000a0a30 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb90009d630 0x7fb9000a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9000a0a30 0x7fb9000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8fffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb900133930 0x7fb900133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56116/0 (socket says 172.21.15.32:56116) 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb90009d630 0x7fb9000a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52532/0 (socket says 172.21.15.32:52532) 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8fffff700 1 -- 172.21.15.32:0/1052895960 learned_addr learned my addr 172.21.15.32:0/1052895960 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 -- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9000a0a30 msgr2=0x7fb9000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 --2- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9000a0a30 0x7fb9000a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 -- 172.21.15.32:0/1052895960 >> v1:172.21.15.32:6789/0 conn(0x7fb900133930 legacy=0x7fb900133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 -- 172.21.15.32:0/1052895960 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90009dc50 con 0x7fb90009d630 2023-09-30T22:26:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8ff7fe700 1 --2- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9000a0a30 0x7fb9000a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8fe7fc700 1 -- 172.21.15.32:0/1052895960 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078039676 0 0) 0x7fb9000a2a70 con 0x7fb900133930 2023-09-30T22:26:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.832+0000 7fb8feffd700 1 --2- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90009d630 0x7fb9000a1340 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009840 tx=0x7fb8e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6c3c5b502ca2bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb8fe7fc700 1 -- 172.21.15.32:0/1052895960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011830 con 0x7fb90009d630 2023-09-30T22:26:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb8fe7fc700 1 -- 172.21.15.32:0/1052895960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8011990 con 0x7fb90009d630 2023-09-30T22:26:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb8fe7fc700 1 -- 172.21.15.32:0/1052895960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8011d20 con 0x7fb90009d630 2023-09-30T22:26:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 -- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90009d630 msgr2=0x7fb9000a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90009d630 0x7fb9000a1340 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009840 tx=0x7fb8e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:26:59.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 -- 172.21.15.32:0/1052895960 shutdown_connections 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9000a0a30 0x7fb9000a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1052895960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90009d630 0x7fb9000a1340 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 -- 172.21.15.32:0/1052895960 >> 172.21.15.32:0/1052895960 conn(0x7fb9000a3050 msgr2=0x7fb9000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 -- 172.21.15.32:0/1052895960 shutdown_connections 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.833+0000 7fb905d1a700 1 -- 172.21.15.32:0/1052895960 wait complete. 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.834+0000 7fb905d1a700 1 Processor -- start 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 -- start start 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb90009d630 0x7fb9001c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 0x7fb9001c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb900133930 0x7fb9001c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:26:59.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001210e0 con 0x7fb9000a0a30 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001207f0 con 0x7fb90009d630 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb905d1a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9001209e0 con 0x7fb900133930 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 0x7fb9001c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 0x7fb9001c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52548/0 (socket says 172.21.15.32:52548) 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 -- 172.21.15.32:0/1202627101 learned_addr learned my addr 172.21.15.32:0/1202627101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8fffff700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb900133930 0x7fb9001c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8ff7fe700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb90009d630 0x7fb9001c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 -- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb900133930 msgr2=0x7fb9001c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb900133930 0x7fb9001c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 -- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb90009d630 msgr2=0x7fb9001c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.835+0000 7fb8feffd700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb90009d630 0x7fb9001c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:26:59.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb8feffd700 1 -- 172.21.15.32:0/1202627101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f0009a70 con 0x7fb9000a0a30 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb8ff7fe700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb90009d630 0x7fb9001c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb8feffd700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 0x7fb9001c86d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80090b0 tx=0x7fb8e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8004400 con 0x7fb9000a0a30 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e801f070 con 0x7fb9000a0a30 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e80094a0 con 0x7fb9000a0a30 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80038e0 con 0x7fb9000a0a30 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.836+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000a0410 con 0x7fb9000a0a30 2023-09-30T22:26:59.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.837+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8e8021030 con 0x7fb9000a0a30 2023-09-30T22:26:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.837+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb8e8040760 con 0x7fb9000a0a30 2023-09-30T22:26:59.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.837+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb900122380 con 0x7fb9000a0a30 2023-09-30T22:26:59.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:26:59.846+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8e8004560 con 0x7fb9000a0a30 2023-09-30T22:27:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.013+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7e040a29a0 con 0x7f7e0409e000 2023-09-30T22:27:00.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.014+0000 7f7debfff700 1 -- 172.21.15.32:0/3847378152 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7df8011360 con 0x7f7e0409e000 2023-09-30T22:27:00.014 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 msgr2=0x7f7e04122b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 0x7f7e04122b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7df8001eb0 tx=0x7f7df800c800 comp rx=0 tx=0).stop 2023-09-30T22:27:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 shutdown_connections 2023-09-30T22:27:00.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7e0409e410 0x7f7e041230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e0409d6f0 0x7f7e04122630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 --2- 172.21.15.32:0/3847378152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7e0409e000 0x7f7e04122b70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 >> 172.21.15.32:0/3847378152 conn(0x7f7e04066f50 msgr2=0x7f7e0409b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 shutdown_connections 2023-09-30T22:27:00.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.016+0000 7f7e09c19700 1 -- 172.21.15.32:0/3847378152 wait complete. 2023-09-30T22:27:00.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:00.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.054+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9000a5780 con 0x7fb9000a0a30 2023-09-30T22:27:00.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.055+0000 7fb8f7fff700 1 -- 172.21.15.32:0/1202627101 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8e8015020 con 0x7fb9000a0a30 2023-09-30T22:27:00.055 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 msgr2=0x7fb9001c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 0x7fb9001c86d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80090b0 tx=0x7fb8e8004750 comp rx=0 tx=0).stop 2023-09-30T22:27:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 shutdown_connections 2023-09-30T22:27:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb900133930 0x7fb9001c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb90009d630 0x7fb9001c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 --2- 172.21.15.32:0/1202627101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9000a0a30 0x7fb9001c86d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 >> 172.21.15.32:0/1202627101 conn(0x7fb9000a3050 msgr2=0x7fb9001bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 shutdown_connections 2023-09-30T22:27:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.057+0000 7fb905d1a700 1 -- 172.21.15.32:0/1202627101 wait complete. 2023-09-30T22:27:00.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:00.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:00.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:00.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.255+0000 7ffbea8dd700 1 Processor -- start 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.255+0000 7ffbea8dd700 1 -- start start 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.255+0000 7ffbea8dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbe409d6f0 0x7ffbe409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.255+0000 7ffbea8dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.255+0000 7ffbea8dd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe409b120 con 0x7ffbe409e410 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.255+0000 7ffbea8dd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe406b460 con 0x7ffbe409d6f0 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbea8dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe409a7e0 con 0x7ffbe409e000 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35984/0 (socket says 172.21.15.32:35984) 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbe409d6f0 0x7ffbe409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 -- 172.21.15.32:0/2764562844 learned_addr learned my addr 172.21.15.32:0/2764562844 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 -- 172.21.15.32:0/2764562844 >> v2:172.21.15.32:3300/0 conn(0x7ffbe409d6f0 msgr2=0x7ffbe409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 --2- 172.21.15.32:0/2764562844 >> v2:172.21.15.32:3300/0 conn(0x7ffbe409d6f0 0x7ffbe409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 -- 172.21.15.32:0/2764562844 >> v1:172.21.15.32:6789/0 conn(0x7ffbe409e410 legacy=0x7ffbe4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe2ffd700 1 -- 172.21.15.32:0/2764562844 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538131578 0 0) 0x7ffbe409b120 con 0x7ffbe409e410 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 -- 172.21.15.32:0/2764562844 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe409a500 con 0x7ffbe409e000 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe3fff700 1 --2- 172.21.15.32:0/2764562844 >> v2:172.21.15.32:3300/0 conn(0x7ffbe409d6f0 0x7ffbe409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe37fe700 1 --2- 172.21.15.32:0/2764562844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41339f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009920 tx=0x7ffbd8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5463c458f251a9e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.256+0000 7ffbe2ffd700 1 -- 172.21.15.32:0/2764562844 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8011830 con 0x7ffbe409e000 2023-09-30T22:27:00.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbe2ffd700 1 -- 172.21.15.32:0/2764562844 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd8011990 con 0x7ffbe409e000 2023-09-30T22:27:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbe2ffd700 1 -- 172.21.15.32:0/2764562844 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8011d20 con 0x7ffbe409e000 2023-09-30T22:27:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 -- 172.21.15.32:0/2764562844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 msgr2=0x7ffbe41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/2764562844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41339f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009920 tx=0x7ffbd8009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 -- 172.21.15.32:0/2764562844 shutdown_connections 2023-09-30T22:27:00.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/2764562844 >> v2:172.21.15.32:3300/0 conn(0x7ffbe409d6f0 0x7ffbe409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/2764562844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41339f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 -- 172.21.15.32:0/2764562844 >> 172.21.15.32:0/2764562844 conn(0x7ffbe4066f50 msgr2=0x7ffbe409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 -- 172.21.15.32:0/2764562844 shutdown_connections 2023-09-30T22:27:00.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.257+0000 7ffbea8dd700 1 -- 172.21.15.32:0/2764562844 wait complete. 2023-09-30T22:27:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.258+0000 7ffbea8dd700 1 Processor -- start 2023-09-30T22:27:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 -- start start 2023-09-30T22:27:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffbe409d6f0 0x7ffbe41c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe409e410 0x7ffbe41c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe41b9560 con 0x7ffbe409e410 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe409c710 con 0x7ffbe409d6f0 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbea8dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe41a8560 con 0x7ffbe409e000 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbe37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbe37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:35988/0 (socket says 172.21.15.32:35988) 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbe37fe700 1 -- 172.21.15.32:0/1129434543 learned_addr learned my addr 172.21.15.32:0/1129434543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbe8e7a700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe409e410 0x7ffbe41c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.259+0000 7ffbe3fff700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffbe409d6f0 0x7ffbe41c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe37fe700 1 -- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffbe409d6f0 msgr2=0x7ffbe41c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe37fe700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffbe409d6f0 0x7ffbe41c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe37fe700 1 -- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe409e410 msgr2=0x7ffbe41c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe37fe700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe409e410 0x7ffbe41c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe37fe700 1 -- 172.21.15.32:0/1129434543 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40094a0 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe37fe700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41c8a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009870 tx=0x7ffbd8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8004300 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd801f070 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80036a0 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd8009550 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.260+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe41a77f0 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.261+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffbd8021030 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.261+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe41a9270 con 0x7ffbe409e000 2023-09-30T22:27:00.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.261+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffbd80408a0 con 0x7ffbe409e000 2023-09-30T22:27:00.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.270+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffbd8023790 con 0x7ffbe409e000 2023-09-30T22:27:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.300+0000 7f804bc48700 1 Processor -- start 2023-09-30T22:27:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804bc48700 1 -- start start 2023-09-30T22:27:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804bc48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f804409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804bc48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f8044131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804bc48700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8044070940 con 0x7f804409f240 2023-09-30T22:27:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804bc48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804406fd60 con 0x7f804409f650 2023-09-30T22:27:00.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804bc48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f804406fa80 con 0x7f804409e930 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f80499e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f804409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f80491e3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f804409f240 0x7f80441317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56146/0 (socket says 172.21.15.32:56146) 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f80491e3700 1 -- 172.21.15.32:0/3282549274 learned_addr learned my addr 172.21.15.32:0/3282549274 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.301+0000 7f804a1e5700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f8044131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80489e2700 1 -- 172.21.15.32:0/3282549274 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382315908 0 0) 0x7f8044070940 con 0x7f804409f240 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80499e4700 1 -- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 msgr2=0x7f8044131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80499e4700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f8044131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80499e4700 1 -- 172.21.15.32:0/3282549274 >> v1:172.21.15.32:6789/0 conn(0x7f804409f240 legacy=0x7f80441317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80499e4700 1 -- 172.21.15.32:0/3282549274 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804409fb60 con 0x7f804409e930 2023-09-30T22:27:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f804a1e5700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f8044131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80499e4700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f804409ed00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8040009870 tx=0x7f8040009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47073b6fbb828c2f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80489e2700 1 -- 172.21.15.32:0/3282549274 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040011830 con 0x7f804409e930 2023-09-30T22:27:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80489e2700 1 -- 172.21.15.32:0/3282549274 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040011990 con 0x7f804409e930 2023-09-30T22:27:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.302+0000 7f80489e2700 1 -- 172.21.15.32:0/3282549274 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040011d20 con 0x7f804409e930 2023-09-30T22:27:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 -- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 msgr2=0x7f804409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f804409ed00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8040009870 tx=0x7f8040009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 -- 172.21.15.32:0/3282549274 shutdown_connections 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f8044131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 --2- 172.21.15.32:0/3282549274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f804409ed00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 -- 172.21.15.32:0/3282549274 >> 172.21.15.32:0/3282549274 conn(0x7f80440a4210 msgr2=0x7f80440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 -- 172.21.15.32:0/3282549274 shutdown_connections 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.303+0000 7f804bc48700 1 -- 172.21.15.32:0/3282549274 wait complete. 2023-09-30T22:27:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.304+0000 7f804bc48700 1 Processor -- start 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.304+0000 7f804bc48700 1 -- start start 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804bc48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f80441a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804bc48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804409f240 0x7f80441a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804bc48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f80441c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804bc48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441a3eb0 con 0x7f804409f240 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804bc48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441a4090 con 0x7f804409f650 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804bc48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80441b5130 con 0x7f804409e930 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f80499e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f80441a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804a1e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f80441c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f80499e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f80441a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:36004/0 (socket says 172.21.15.32:36004) 2023-09-30T22:27:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f804a1e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f80441c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36332/0 (socket says 172.21.15.32:36332) 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f80499e4700 1 -- 172.21.15.32:0/3822520903 learned_addr learned my addr 172.21.15.32:0/3822520903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.305+0000 7f80491e3700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804409f240 0x7f80441a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f80499e4700 1 -- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 msgr2=0x7f80441c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f80499e4700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f80441c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f80499e4700 1 -- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804409f240 msgr2=0x7f80441a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f80499e4700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804409f240 0x7f80441a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f80499e4700 1 -- 172.21.15.32:0/3822520903 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c00bc50 con 0x7f804409e930 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f80499e4700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f80441a10b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f80400090b0 tx=0x7f8040004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040004510 con 0x7f804409e930 2023-09-30T22:27:00.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804001f070 con 0x7f804409e930 2023-09-30T22:27:00.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400094a0 con 0x7f804409e930 2023-09-30T22:27:00.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400038e0 con 0x7f804409e930 2023-09-30T22:27:00.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.306+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80440a0190 con 0x7f804409e930 2023-09-30T22:27:00.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.307+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044071d80 con 0x7f804409e930 2023-09-30T22:27:00.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.307+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8040003a40 con 0x7f804409e930 2023-09-30T22:27:00.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.308+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8040021070 con 0x7f804409e930 2023-09-30T22:27:00.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.312+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8040010380 con 0x7f804409e930 2023-09-30T22:27:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.485+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffbe41a7980 con 0x7ffbe409e000 2023-09-30T22:27:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.485+0000 7ffbe0ff9700 1 -- 172.21.15.32:0/1129434543 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffbd8015070 con 0x7ffbe409e000 2023-09-30T22:27:00.485 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 msgr2=0x7ffbe41c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41c8a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009870 tx=0x7ffbd8004750 comp rx=0 tx=0).stop 2023-09-30T22:27:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 shutdown_connections 2023-09-30T22:27:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe409e410 0x7ffbe41c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffbe409d6f0 0x7ffbe41c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 --2- 172.21.15.32:0/1129434543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffbe409e000 0x7ffbe41c8a30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 >> 172.21.15.32:0/1129434543 conn(0x7ffbe4066f50 msgr2=0x7ffbe41bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 shutdown_connections 2023-09-30T22:27:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.487+0000 7ffbea8dd700 1 -- 172.21.15.32:0/1129434543 wait complete. 2023-09-30T22:27:00.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:00.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.517+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8044074ea0 con 0x7f804409e930 2023-09-30T22:27:00.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.517+0000 7f803a7fc700 1 -- 172.21.15.32:0/3822520903 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8040015070 con 0x7f804409e930 2023-09-30T22:27:00.517 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 msgr2=0x7f80441a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f80441a10b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f80400090b0 tx=0x7f8040004750 comp rx=0 tx=0).stop 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 shutdown_connections 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804409f650 0x7f80441c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804409e930 0x7f80441a10b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 --2- 172.21.15.32:0/3822520903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804409f240 0x7f80441a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 >> 172.21.15.32:0/3822520903 conn(0x7f80440a4210 msgr2=0x7f80441b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 shutdown_connections 2023-09-30T22:27:00.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:00.520+0000 7f804bc48700 1 -- 172.21.15.32:0/3822520903 wait complete. 2023-09-30T22:27:00.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:01.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:01.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:01.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.181+0000 7fb8750ce700 1 Processor -- start 2023-09-30T22:27:01.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.181+0000 7fb8750ce700 1 -- start start 2023-09-30T22:27:01.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb8750ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb8750ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb870139250 0x7fb87013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb8750ce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87009edd0 con 0x7fb8700a0a30 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb8750ce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8700a18e0 con 0x7fb870139250 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb8750ce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb87009e1f0 con 0x7fb8700a1510 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb870139250 0x7fb87013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8700a0a30 0x7fb8700a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47490/0 (socket says 172.21.15.32:47490) 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86ed9d700 1 -- 172.21.15.32:0/3597548683 learned_addr learned my addr 172.21.15.32:0/3597548683 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86f59e700 1 -- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 msgr2=0x7fb870138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86f59e700 1 --2- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86f59e700 1 -- 172.21.15.32:0/3597548683 >> v1:172.21.15.32:6789/0 conn(0x7fb8700a0a30 legacy=0x7fb8700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.182+0000 7fb86dd9b700 1 -- 172.21.15.32:0/3597548683 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890856510 0 0) 0x7fb87009edd0 con 0x7fb8700a0a30 2023-09-30T22:27:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb86f59e700 1 -- 172.21.15.32:0/3597548683 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87009dc50 con 0x7fb870139250 2023-09-30T22:27:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb86f59e700 1 --2- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870139250 0x7fb87013b630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb868009870 tx=0x7fb868009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b92ea6567385e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb86dd9b700 1 -- 172.21.15.32:0/3597548683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011830 con 0x7fb870139250 2023-09-30T22:27:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb86dd9b700 1 -- 172.21.15.32:0/3597548683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868011990 con 0x7fb870139250 2023-09-30T22:27:01.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb86dd9b700 1 -- 172.21.15.32:0/3597548683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011d20 con 0x7fb870139250 2023-09-30T22:27:01.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb8750ce700 1 -- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870139250 msgr2=0x7fb87013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870139250 0x7fb87013b630 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb868009870 tx=0x7fb868009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb8750ce700 1 -- 172.21.15.32:0/3597548683 shutdown_connections 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb870139250 0x7fb87013b630 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3597548683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.183+0000 7fb8750ce700 1 -- 172.21.15.32:0/3597548683 >> 172.21.15.32:0/3597548683 conn(0x7fb8700a6550 msgr2=0x7fb8700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.184+0000 7fb8750ce700 1 -- 172.21.15.32:0/3597548683 shutdown_connections 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.184+0000 7fb8750ce700 1 -- 172.21.15.32:0/3597548683 wait complete. 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.185+0000 7fb8750ce700 1 Processor -- start 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.185+0000 7fb8750ce700 1 -- start start 2023-09-30T22:27:01.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.185+0000 7fb8750ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 0x7fb870122720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.185+0000 7fb8750ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870122c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.185+0000 7fb8750ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 0x7fb8701231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.185+0000 7fb8750ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701262e0 con 0x7fb8700a0a30 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb8750ce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701259d0 con 0x7fb870139250 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb8750ce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700a0220 con 0x7fb8700a1510 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 0x7fb870122720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 0x7fb8701231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 0x7fb870122720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60760/0 (socket says 172.21.15.32:60760) 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870122c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49618/0 (socket says 172.21.15.32:49618) 2023-09-30T22:27:01.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 -- 172.21.15.32:0/3367956691 learned_addr learned my addr 172.21.15.32:0/3367956691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 0x7fb8701231a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:46988/0 (socket says 172.21.15.32:46988) 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 -- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 msgr2=0x7fb870122c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870122c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 -- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 msgr2=0x7fb8701231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 0x7fb8701231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 -- 172.21.15.32:0/3367956691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680094a0 con 0x7fb8700a0a30 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86f59e700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 0x7fb8701231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86e59c700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870122c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.186+0000 7fb86ed9d700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 0x7fb870122720 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb86000ccb0 tx=0x7fb86000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.187+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000aaf0 con 0x7fb8700a0a30 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.187+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86000ac50 con 0x7fb8700a0a30 2023-09-30T22:27:01.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.187+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8700a55d0 con 0x7fb8700a0a30 2023-09-30T22:27:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.187+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8600070d0 con 0x7fb8700a0a30 2023-09-30T22:27:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.187+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb870126740 con 0x7fb8700a0a30 2023-09-30T22:27:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.188+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb860007230 con 0x7fb8700a0a30 2023-09-30T22:27:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.188+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb870125e80 con 0x7fb8700a0a30 2023-09-30T22:27:01.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.188+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb86003f5d0 con 0x7fb8700a0a30 2023-09-30T22:27:01.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.193+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb870125e80 con 0x7fb8700a0a30 2023-09-30T22:27:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.229+0000 7f26d458c700 1 Processor -- start 2023-09-30T22:27:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.229+0000 7f26d458c700 1 -- start start 2023-09-30T22:27:01.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d458c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26cc09f410 0x7f26cc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d458c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc136fe0 0x7f26cc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d458c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc070940 con 0x7f26cc09e930 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d458c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc06fd60 con 0x7f26cc09f410 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d458c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc06fa80 con 0x7f26cc136fe0 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d1b27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26cc09f410 0x7f26cc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d2b29700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc136fe0 0x7f26cc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d2328700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26cc09e930 0x7f26cc09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47500/0 (socket says 172.21.15.32:47500) 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d2328700 1 -- 172.21.15.32:0/3303120441 learned_addr learned my addr 172.21.15.32:0/3303120441 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d2b29700 1 -- 172.21.15.32:0/3303120441 >> v2:172.21.15.32:3300/0 conn(0x7f26cc09f410 msgr2=0x7f26cc136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d2b29700 1 --2- 172.21.15.32:0/3303120441 >> v2:172.21.15.32:3300/0 conn(0x7f26cc09f410 0x7f26cc136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.230+0000 7f26d2b29700 1 -- 172.21.15.32:0/3303120441 >> v1:172.21.15.32:6789/0 conn(0x7f26cc09e930 legacy=0x7f26cc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d2b29700 1 -- 172.21.15.32:0/3303120441 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc09fb60 con 0x7f26cc136fe0 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d1b27700 1 --2- 172.21.15.32:0/3303120441 >> v2:172.21.15.32:3300/0 conn(0x7f26cc09f410 0x7f26cc136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d2b29700 1 --2- 172.21.15.32:0/3303120441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc136fe0 0x7f26cc1393c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009d90 tx=0x7f26c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5103ce6a5de550b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d1326700 1 -- 172.21.15.32:0/3303120441 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c400e070 con 0x7f26cc136fe0 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d1326700 1 -- 172.21.15.32:0/3303120441 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c400bb20 con 0x7f26cc136fe0 2023-09-30T22:27:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d1326700 1 -- 172.21.15.32:0/3303120441 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4012430 con 0x7f26cc136fe0 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.231+0000 7f26d458c700 1 -- 172.21.15.32:0/3303120441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc136fe0 msgr2=0x7f26cc1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 --2- 172.21.15.32:0/3303120441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc136fe0 0x7f26cc1393c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009d90 tx=0x7f26c400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 -- 172.21.15.32:0/3303120441 shutdown_connections 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 --2- 172.21.15.32:0/3303120441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc136fe0 0x7f26cc1393c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 --2- 172.21.15.32:0/3303120441 >> v2:172.21.15.32:3300/0 conn(0x7f26cc09f410 0x7f26cc136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 -- 172.21.15.32:0/3303120441 >> 172.21.15.32:0/3303120441 conn(0x7f26cc0a4210 msgr2=0x7f26cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 -- 172.21.15.32:0/3303120441 shutdown_connections 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.232+0000 7f26d458c700 1 -- 172.21.15.32:0/3303120441 wait complete. 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 Processor -- start 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 -- start start 2023-09-30T22:27:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 0x7f26cc1c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09f410 0x7f26cc1c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc136fe0 0x7f26cc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1bc0c0 con 0x7f26cc09e930 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1224b0 con 0x7f26cc136fe0 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d458c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1320d0 con 0x7f26cc09f410 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d2328700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 0x7f26cc1c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d2328700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 0x7f26cc1c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60782/0 (socket says 172.21.15.32:60782) 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d2328700 1 -- 172.21.15.32:0/1201720256 learned_addr learned my addr 172.21.15.32:0/1201720256 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d2b29700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc136fe0 0x7f26cc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.233+0000 7f26d1b27700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09f410 0x7f26cc1c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d2328700 1 -- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09f410 msgr2=0x7f26cc1c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d2328700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09f410 0x7f26cc1c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d2328700 1 -- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc136fe0 msgr2=0x7f26cc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d2328700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc136fe0 0x7f26cc06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d2328700 1 -- 172.21.15.32:0/1201720256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0094a0 con 0x7f26cc09e930 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d2328700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 0x7f26cc1c86d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c520 tx=0x7f26c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8005d70 con 0x7f26cc09e930 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c80070d0 con 0x7f26cc09e930 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c4009a70 con 0x7f26cc09e930 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8007440 con 0x7f26cc09e930 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.234+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc121d40 con 0x7f26cc09e930 2023-09-30T22:27:01.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.235+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f26c80075a0 con 0x7f26cc09e930 2023-09-30T22:27:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.235+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f26c803ec30 con 0x7f26cc09e930 2023-09-30T22:27:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.235+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc06d4c0 con 0x7f26cc09e930 2023-09-30T22:27:01.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.240+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f26c8017080 con 0x7f26cc09e930 2023-09-30T22:27:01.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.429+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb870125e80 con 0x7fb8700a0a30 2023-09-30T22:27:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.429+0000 7fb8577fe700 1 -- 172.21.15.32:0/3367956691 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb870125e80 con 0x7fb8700a0a30 2023-09-30T22:27:01.429 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 msgr2=0x7fb870122720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 0x7fb870122720 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb86000ccb0 tx=0x7fb86000a060 comp rx=0 tx=0).stop 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 shutdown_connections 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb870139250 0x7fb8701231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8700a0a30 0x7fb870122720 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 --2- 172.21.15.32:0/3367956691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8700a1510 0x7fb870122c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 >> 172.21.15.32:0/3367956691 conn(0x7fb8700a6550 msgr2=0x7fb87011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 shutdown_connections 2023-09-30T22:27:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.431+0000 7fb8750ce700 1 -- 172.21.15.32:0/3367956691 wait complete. 2023-09-30T22:27:01.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.462+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f26cc121490 con 0x7f26cc09e930 2023-09-30T22:27:01.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.463+0000 7f26c2ffd700 1 -- 172.21.15.32:0/1201720256 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26c8019b40 con 0x7f26cc09e930 2023-09-30T22:27:01.463 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 msgr2=0x7f26cc1c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 0x7f26cc1c86d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c520 tx=0x7f26c800c830 comp rx=0 tx=0).stop 2023-09-30T22:27:01.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 shutdown_connections 2023-09-30T22:27:01.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc136fe0 0x7f26cc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e930 0x7f26cc1c86d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 --2- 172.21.15.32:0/1201720256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09f410 0x7f26cc1c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 >> 172.21.15.32:0/1201720256 conn(0x7f26cc0a4210 msgr2=0x7f26cc135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 shutdown_connections 2023-09-30T22:27:01.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.465+0000 7f26d458c700 1 -- 172.21.15.32:0/1201720256 wait complete. 2023-09-30T22:27:01.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:01.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:01.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:01.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.661+0000 7f24c0902700 1 Processor -- start 2023-09-30T22:27:01.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.661+0000 7f24c0902700 1 -- start start 2023-09-30T22:27:01.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24c0902700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24bc09d630 0x7f24bc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24c0902700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24c0902700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24bc0a2a70 con 0x7f24bc0a0a30 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24c0902700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24bc0a01d0 con 0x7f24bc09d630 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24c0902700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc09e2b0 con 0x7f24bc138d80 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24b9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24bc09d630 0x7f24bc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24bad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24ba59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f24bc0a0a30 0x7f24bc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47514/0 (socket says 172.21.15.32:47514) 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24ba59c700 1 -- 172.21.15.32:0/4002230019 learned_addr learned my addr 172.21.15.32:0/4002230019 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24b9d9b700 1 -- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 msgr2=0x7f24bc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24b9d9b700 1 --2- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24b9d9b700 1 -- 172.21.15.32:0/4002230019 >> v1:172.21.15.32:6789/0 conn(0x7f24bc0a0a30 legacy=0x7f24bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24b9d9b700 1 -- 172.21.15.32:0/4002230019 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc09dc50 con 0x7f24bc09d630 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24b959a700 1 -- 172.21.15.32:0/4002230019 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175154113 0 0) 0x7f24bc0a2a70 con 0x7f24bc0a0a30 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.662+0000 7f24bad9d700 1 --2- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.663+0000 7f24b9d9b700 1 --2- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc0a1510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009840 tx=0x7f24b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3dff8181aef7f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.663+0000 7f24b959a700 1 -- 172.21.15.32:0/4002230019 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0011830 con 0x7f24bc09d630 2023-09-30T22:27:01.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.663+0000 7f24b959a700 1 -- 172.21.15.32:0/4002230019 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0011990 con 0x7f24bc09d630 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.663+0000 7f24b959a700 1 -- 172.21.15.32:0/4002230019 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0011d20 con 0x7f24bc09d630 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.663+0000 7f24c0902700 1 -- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 msgr2=0x7f24bc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.663+0000 7f24c0902700 1 --2- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc0a1510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009840 tx=0x7f24b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.664+0000 7f24c0902700 1 -- 172.21.15.32:0/4002230019 shutdown_connections 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.664+0000 7f24c0902700 1 --2- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.664+0000 7f24c0902700 1 --2- 172.21.15.32:0/4002230019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc0a1510 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.664+0000 7f24c0902700 1 -- 172.21.15.32:0/4002230019 >> 172.21.15.32:0/4002230019 conn(0x7f24bc0a3050 msgr2=0x7f24bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.664+0000 7f24c0902700 1 -- 172.21.15.32:0/4002230019 shutdown_connections 2023-09-30T22:27:01.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.664+0000 7f24c0902700 1 -- 172.21.15.32:0/4002230019 wait complete. 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 Processor -- start 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 -- start start 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24bc0a0a30 0x7f24bc133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc136190 con 0x7f24bc09d630 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc136eb0 con 0x7f24bc0a0a30 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.665+0000 7f24c0902700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc0a4ba0 con 0x7f24bc138d80 2023-09-30T22:27:01.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24bc0a0a30 0x7f24bc133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24bad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60800/0 (socket says 172.21.15.32:60800) 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24ba59c700 1 -- 172.21.15.32:0/2014049942 learned_addr learned my addr 172.21.15.32:0/2014049942 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 -- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 msgr2=0x7f24bc134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 -- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 msgr2=0x7f24bc133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 -- 172.21.15.32:0/2014049942 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40094a0 con 0x7f24bc0a0a30 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24ba59c700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24b9d9b700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24bc0a0a30 0x7f24bc133c40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24b0016000 tx=0x7f24b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0004480 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b001f070 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.666+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b00094a0 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.667+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b00038a0 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.667+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc136540 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.667+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24b0021030 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.667+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc1379e0 con 0x7f24bc0a0a30 2023-09-30T22:27:01.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.668+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f24b0040720 con 0x7f24bc0a0a30 2023-09-30T22:27:01.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.672+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24b0003a00 con 0x7f24bc0a0a30 2023-09-30T22:27:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 Processor -- start 2023-09-30T22:27:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 -- start start 2023-09-30T22:27:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 0x7f17c809ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c81391d0 0x7f17c813b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c806bd70 con 0x7f17c809f0f0 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c806cc30 con 0x7f17c81391d0 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ce9ab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17c806d870 con 0x7f17c809e7e0 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 0x7f17c809ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17ccf48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c81391d0 0x7f17c813b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17c77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17c809f0f0 0x7f17c8137ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47520/0 (socket says 172.21.15.32:47520) 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 0x7f17c809ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49664/0 (socket says 172.21.15.32:49664) 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.703+0000 7f17c77fe700 1 -- 172.21.15.32:0/2817754770 learned_addr learned my addr 172.21.15.32:0/2817754770 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c7fff700 1 -- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c81391d0 msgr2=0x7f17c813b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c7fff700 1 --2- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c81391d0 0x7f17c813b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c7fff700 1 -- 172.21.15.32:0/2817754770 >> v1:172.21.15.32:6789/0 conn(0x7f17c809f0f0 legacy=0x7f17c8137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c6ffd700 1 -- 172.21.15.32:0/2817754770 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216703635 0 0) 0x7f17c806bd70 con 0x7f17c809f0f0 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c7fff700 1 -- 172.21.15.32:0/2817754770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c806bab0 con 0x7f17c809e7e0 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17ccf48700 1 --2- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c81391d0 0x7f17c813b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c7fff700 1 --2- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 0x7f17c809ebb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009870 tx=0x7f17b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee781a82644c37f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c6ffd700 1 -- 172.21.15.32:0/2817754770 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8011830 con 0x7f17c809e7e0 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c6ffd700 1 -- 172.21.15.32:0/2817754770 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8011990 con 0x7f17c809e7e0 2023-09-30T22:27:01.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.704+0000 7f17c6ffd700 1 -- 172.21.15.32:0/2817754770 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8011d20 con 0x7f17c809e7e0 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 -- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 msgr2=0x7f17c809ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 0x7f17c809ebb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009870 tx=0x7f17b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 -- 172.21.15.32:0/2817754770 shutdown_connections 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c81391d0 0x7f17c813b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/2817754770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809e7e0 0x7f17c809ebb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 -- 172.21.15.32:0/2817754770 >> 172.21.15.32:0/2817754770 conn(0x7f17c80a4300 msgr2=0x7f17c80a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 -- 172.21.15.32:0/2817754770 shutdown_connections 2023-09-30T22:27:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.705+0000 7f17ce9ab700 1 -- 172.21.15.32:0/2817754770 wait complete. 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 Processor -- start 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 -- start start 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c809e7e0 0x7f17c8121710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 0x7f17c8121c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 0x7f17c8122190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8136fb0 con 0x7f17c81391d0 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8124230 con 0x7f17c809e7e0 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.706+0000 7f17ce9ab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80a05d0 con 0x7f17c809f0f0 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 0x7f17c8121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17ccf48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 0x7f17c8122190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c809e7e0 0x7f17c8121710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 0x7f17c8121c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49668/0 (socket says 172.21.15.32:49668) 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17ccf48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 0x7f17c8122190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60812/0 (socket says 172.21.15.32:60812) 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 -- 172.21.15.32:0/3881789662 learned_addr learned my addr 172.21.15.32:0/3881789662 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 -- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c809e7e0 msgr2=0x7f17c8121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c809e7e0 0x7f17c8121710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 -- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 msgr2=0x7f17c8122190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 0x7f17c8122190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 -- 172.21.15.32:0/3881789662 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0009a70 con 0x7f17c809f0f0 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17ccf48700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 0x7f17c8122190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c77fe700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 0x7f17c8121c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001eb0 tx=0x7f17bc00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00cea0 con 0x7f17c809f0f0 2023-09-30T22:27:01.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc004c80 con 0x7f17c809f0f0 2023-09-30T22:27:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b80094a0 con 0x7f17c809f0f0 2023-09-30T22:27:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0070d0 con 0x7f17c809f0f0 2023-09-30T22:27:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.707+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c8123b10 con 0x7f17c809f0f0 2023-09-30T22:27:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.708+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17bc007230 con 0x7f17c809f0f0 2023-09-30T22:27:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.708+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f17bc03f270 con 0x7f17c809f0f0 2023-09-30T22:27:01.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.709+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c804f2a0 con 0x7f17c809f0f0 2023-09-30T22:27:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.713+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17bc0074e0 con 0x7f17c809f0f0 2023-09-30T22:27:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.880+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24bc0747f0 con 0x7f24bc0a0a30 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.881+0000 7f24aaffd700 1 -- 172.21.15.32:0/2014049942 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24b0032020 con 0x7f24bc0a0a30 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24bc0a0a30 msgr2=0x7f24bc133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24bc0a0a30 0x7f24bc133c40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24b0016000 tx=0x7f24b0004750 comp rx=0 tx=0).stop 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 shutdown_connections 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24bc138d80 0x7f24bc134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24bc09d630 0x7f24bc133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 --2- 172.21.15.32:0/2014049942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24bc0a0a30 0x7f24bc133c40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 >> 172.21.15.32:0/2014049942 conn(0x7f24bc0a3050 msgr2=0x7f24bc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 shutdown_connections 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.883+0000 7f24c0902700 1 -- 172.21.15.32:0/2014049942 wait complete. 2023-09-30T22:27:01.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.923+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f17c806b120 con 0x7f17c809f0f0 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.923+0000 7f17c4ff9700 1 -- 172.21.15.32:0/3881789662 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17bc01b340 con 0x7f17c809f0f0 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.925+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 msgr2=0x7f17c8121c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.925+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 0x7f17c8121c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001eb0 tx=0x7f17bc00c800 comp rx=0 tx=0).stop 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.925+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 shutdown_connections 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.925+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17c81391d0 0x7f17c8122190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.926+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17c809e7e0 0x7f17c8121710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.926+0000 7f17ce9ab700 1 --2- 172.21.15.32:0/3881789662 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17c809f0f0 0x7f17c8121c50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.926+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 >> 172.21.15.32:0/3881789662 conn(0x7f17c80a4300 msgr2=0x7f17c8070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.926+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 shutdown_connections 2023-09-30T22:27:01.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:01.926+0000 7f17ce9ab700 1 -- 172.21.15.32:0/3881789662 wait complete. 2023-09-30T22:27:01.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:02.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:02.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:02.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.594+0000 7f02cec17700 1 Processor -- start 2023-09-30T22:27:02.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.594+0000 7f02cec17700 1 -- start start 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cec17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cec17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c813b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cec17700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c80a2a20 con 0x7f02c80a0d90 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cec17700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c809e690 con 0x7f02c80a09c0 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cec17700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80a1cc0 con 0x7f02c8138cf0 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cd1b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c813b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cc9b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02c7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f02c80a0d90 0x7f02c80a11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47536/0 (socket says 172.21.15.32:47536) 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02c7fff700 1 -- 172.21.15.32:0/3382076392 learned_addr learned my addr 172.21.15.32:0/3382076392 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:02.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cc9b3700 1 -- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 msgr2=0x7f02c813b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cc9b3700 1 --2- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c813b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02c77fe700 1 -- 172.21.15.32:0/3382076392 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749980623 0 0) 0x7f02bc001110 con 0x7f02c80a0d90 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cc9b3700 1 -- 172.21.15.32:0/3382076392 >> v1:172.21.15.32:6789/0 conn(0x7f02c80a0d90 legacy=0x7f02c80a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cc9b3700 1 -- 172.21.15.32:0/3382076392 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80a1f40 con 0x7f02c80a09c0 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.595+0000 7f02cd1b4700 1 --2- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c813b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cc9b3700 1 --2- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c806c5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009dc0 tx=0x7f02b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc37c26a309e24aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02c77fe700 1 -- 172.21.15.32:0/3382076392 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800e070 con 0x7f02c80a09c0 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02c77fe700 1 -- 172.21.15.32:0/3382076392 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b800bb20 con 0x7f02c80a09c0 2023-09-30T22:27:02.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02c77fe700 1 -- 172.21.15.32:0/3382076392 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8012430 con 0x7f02c80a09c0 2023-09-30T22:27:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cec17700 1 -- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 msgr2=0x7f02c806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cec17700 1 --2- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c806c5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009dc0 tx=0x7f02b800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cec17700 1 -- 172.21.15.32:0/3382076392 shutdown_connections 2023-09-30T22:27:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cec17700 1 --2- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c813b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cec17700 1 --2- 172.21.15.32:0/3382076392 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c806c5d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.596+0000 7f02cec17700 1 -- 172.21.15.32:0/3382076392 >> 172.21.15.32:0/3382076392 conn(0x7f02c80a60c0 msgr2=0x7f02c80a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.597+0000 7f02cec17700 1 -- 172.21.15.32:0/3382076392 shutdown_connections 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.597+0000 7f02cec17700 1 -- 172.21.15.32:0/3382076392 wait complete. 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.597+0000 7f02cec17700 1 Processor -- start 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 -- start start 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c8137030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 0x7f02c8137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c8137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8136170 con 0x7f02c80a0d90 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c81380e0 con 0x7f02c80a09c0 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cec17700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8138840 con 0x7f02c8138cf0 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 0x7f02c8137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cd1b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c8137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cc9b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c8137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cd1b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c8137ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49684/0 (socket says 172.21.15.32:49684) 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02cd1b4700 1 -- 172.21.15.32:0/2908046889 learned_addr learned my addr 172.21.15.32:0/2908046889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.598+0000 7f02c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 0x7f02c8137570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60820/0 (socket says 172.21.15.32:60820) 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cc9b3700 1 -- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 msgr2=0x7f02c8137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cc9b3700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c8137ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cc9b3700 1 -- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 msgr2=0x7f02c8137570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cc9b3700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 0x7f02c8137570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cc9b3700 1 -- 172.21.15.32:0/2908046889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c00094a0 con 0x7f02c80a09c0 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cd1b4700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c8137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02c7fff700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 0x7f02c8137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cc9b3700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c8137030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f02b8016000 tx=0x7f02b8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8012430 con 0x7f02c80a09c0 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b8020070 con 0x7f02c80a09c0 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8004280 con 0x7f02c80a09c0 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b8009a70 con 0x7f02c80a09c0 2023-09-30T22:27:02.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.599+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c8135b50 con 0x7f02c80a09c0 2023-09-30T22:27:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.600+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02b80043e0 con 0x7f02c80a09c0 2023-09-30T22:27:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.600+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f02b8022070 con 0x7f02c80a09c0 2023-09-30T22:27:02.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.600+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c8135e10 con 0x7f02c80a09c0 2023-09-30T22:27:02.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.605+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02b801b400 con 0x7f02c80a09c0 2023-09-30T22:27:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.633+0000 7f23d73a1700 1 Processor -- start 2023-09-30T22:27:02.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d73a1700 1 -- start start 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d73a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d73a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d0138d80 0x7f23d013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d73a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00a2a70 con 0x7f23d009d630 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d73a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d00a01d0 con 0x7f23d0138d80 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d73a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d009e2b0 con 0x7f23d00a0a30 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d593e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d0138d80 0x7f23d013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d513d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d493c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23d009d630 0x7f23d00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47538/0 (socket says 172.21.15.32:47538) 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.634+0000 7f23d493c700 1 -- 172.21.15.32:0/4010935083 learned_addr learned my addr 172.21.15.32:0/4010935083 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23d513d700 1 -- 172.21.15.32:0/4010935083 >> v2:172.21.15.32:3300/0 conn(0x7f23d0138d80 msgr2=0x7f23d013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23d513d700 1 --2- 172.21.15.32:0/4010935083 >> v2:172.21.15.32:3300/0 conn(0x7f23d0138d80 0x7f23d013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23d513d700 1 -- 172.21.15.32:0/4010935083 >> v1:172.21.15.32:6789/0 conn(0x7f23d009d630 legacy=0x7f23d00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23d513d700 1 -- 172.21.15.32:0/4010935083 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d009dc50 con 0x7f23d00a0a30 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23d593e700 1 --2- 172.21.15.32:0/4010935083 >> v2:172.21.15.32:3300/0 conn(0x7f23d0138d80 0x7f23d013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23d513d700 1 --2- 172.21.15.32:0/4010935083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d00a0e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009840 tx=0x7f23cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312a5b56e2fca1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23c7fff700 1 -- 172.21.15.32:0/4010935083 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc011830 con 0x7f23d00a0a30 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23c7fff700 1 -- 172.21.15.32:0/4010935083 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc011990 con 0x7f23d00a0a30 2023-09-30T22:27:02.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.635+0000 7f23c7fff700 1 -- 172.21.15.32:0/4010935083 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc011d20 con 0x7f23d00a0a30 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 -- 172.21.15.32:0/4010935083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 msgr2=0x7f23d00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 --2- 172.21.15.32:0/4010935083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d00a0e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009840 tx=0x7f23cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 -- 172.21.15.32:0/4010935083 shutdown_connections 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 --2- 172.21.15.32:0/4010935083 >> v2:172.21.15.32:3300/0 conn(0x7f23d0138d80 0x7f23d013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 --2- 172.21.15.32:0/4010935083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d00a0e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 -- 172.21.15.32:0/4010935083 >> 172.21.15.32:0/4010935083 conn(0x7f23d00a3050 msgr2=0x7f23d00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 -- 172.21.15.32:0/4010935083 shutdown_connections 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.636+0000 7f23d73a1700 1 -- 172.21.15.32:0/4010935083 wait complete. 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 Processor -- start 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 -- start start 2023-09-30T22:27:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23d009d630 0x7f23d01208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d0120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 0x7f23d0121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0122470 con 0x7f23d0138d80 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d01237c0 con 0x7f23d00a0a30 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.638+0000 7f23d73a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0121d50 con 0x7f23d009d630 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d493c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d0120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 0x7f23d0121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d493c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d0120e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47062/0 (socket says 172.21.15.32:47062) 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d493c700 1 -- 172.21.15.32:0/916463080 learned_addr learned my addr 172.21.15.32:0/916463080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 0x7f23d0121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60844/0 (socket says 172.21.15.32:60844) 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d513d700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23d009d630 0x7f23d01208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 -- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23d009d630 msgr2=0x7f23d01208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23d009d630 0x7f23d01208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 -- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 msgr2=0x7f23d0120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d0120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 -- 172.21.15.32:0/916463080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8009a70 con 0x7f23d0138d80 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d493c700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d0120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d593e700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 0x7f23d0121370 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f23d00a2430 tx=0x7f23c800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800ccd0 con 0x7f23d0138d80 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c800f040 con 0x7f23d0138d80 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c80146c0 con 0x7f23d0138d80 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.639+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc0094a0 con 0x7f23d0138d80 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.640+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d013b070 con 0x7f23d0138d80 2023-09-30T22:27:02.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.640+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23c8014820 con 0x7f23d0138d80 2023-09-30T22:27:02.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.640+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f23c8040320 con 0x7f23d0138d80 2023-09-30T22:27:02.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.641+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d00747f0 con 0x7f23d0138d80 2023-09-30T22:27:02.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.646+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23c8007920 con 0x7f23d0138d80 2023-09-30T22:27:02.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.813+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f02c80a23f0 con 0x7f02c80a09c0 2023-09-30T22:27:02.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.813+0000 7f02c57fa700 1 -- 172.21.15.32:0/2908046889 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02b802c030 con 0x7f02c80a09c0 2023-09-30T22:27:02.813 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:02.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 msgr2=0x7f02c8137030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c8137030 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f02b8016000 tx=0x7f02b8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 shutdown_connections 2023-09-30T22:27:02.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f02c8138cf0 0x7f02c8137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02c80a09c0 0x7f02c8137030 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 --2- 172.21.15.32:0/2908046889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c80a0d90 0x7f02c8137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 >> 172.21.15.32:0/2908046889 conn(0x7f02c80a60c0 msgr2=0x7f02c80a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 shutdown_connections 2023-09-30T22:27:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.815+0000 7f02cec17700 1 -- 172.21.15.32:0/2908046889 wait complete. 2023-09-30T22:27:02.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:02.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.853+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f23d0123ca0 con 0x7f23d0138d80 2023-09-30T22:27:02.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.854+0000 7f23c5ffb700 1 -- 172.21.15.32:0/916463080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23c8018020 con 0x7f23d0138d80 2023-09-30T22:27:02.854 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 msgr2=0x7f23d0121370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 0x7f23d0121370 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f23d00a2430 tx=0x7f23c800ef90 comp rx=0 tx=0).stop 2023-09-30T22:27:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 shutdown_connections 2023-09-30T22:27:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d0138d80 0x7f23d0121370 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23d009d630 0x7f23d01208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 --2- 172.21.15.32:0/916463080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d00a0a30 0x7f23d0120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 >> 172.21.15.32:0/916463080 conn(0x7f23d00a3050 msgr2=0x7f23d0139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 shutdown_connections 2023-09-30T22:27:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:02.856+0000 7f23d73a1700 1 -- 172.21.15.32:0/916463080 wait complete. 2023-09-30T22:27:02.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:02.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:02.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.053+0000 7f56eba83700 1 Processor -- start 2023-09-30T22:27:03.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.053+0000 7f56eba83700 1 -- start start 2023-09-30T22:27:03.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56eba83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 0x7f56e40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56eba83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e409d630 0x7f56e40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56eba83700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40a2a70 con 0x7f56e4133930 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56eba83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40a01d0 con 0x7f56e40a0a30 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56eba83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56e409e2b0 con 0x7f56e409d630 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e901e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e409d630 0x7f56e40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 0x7f56e40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56ea020700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56e4133930 0x7f56e4133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47548/0 (socket says 172.21.15.32:47548) 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 0x7f56e40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47064/0 (socket says 172.21.15.32:47064) 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e901e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e409d630 0x7f56e40a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49712/0 (socket says 172.21.15.32:49712) 2023-09-30T22:27:03.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56ea020700 1 -- 172.21.15.32:0/636538226 learned_addr learned my addr 172.21.15.32:0/636538226 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 -- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e409d630 msgr2=0x7f56e40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 --2- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e409d630 0x7f56e40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 -- 172.21.15.32:0/636538226 >> v1:172.21.15.32:6789/0 conn(0x7f56e4133930 legacy=0x7f56e4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e881d700 1 -- 172.21.15.32:0/636538226 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135414296 0 0) 0x7f56e40a2a70 con 0x7f56e4133930 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 -- 172.21.15.32:0/636538226 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e409dc50 con 0x7f56e40a0a30 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.054+0000 7f56e981f700 1 --2- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 0x7f56e40a0e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c944369b15ac78be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56e881d700 1 -- 172.21.15.32:0/636538226 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0011830 con 0x7f56e40a0a30 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56e881d700 1 -- 172.21.15.32:0/636538226 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0011990 con 0x7f56e40a0a30 2023-09-30T22:27:03.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56e881d700 1 -- 172.21.15.32:0/636538226 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0011d20 con 0x7f56e40a0a30 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 -- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 msgr2=0x7f56e40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 --2- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 0x7f56e40a0e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 -- 172.21.15.32:0/636538226 shutdown_connections 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 --2- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e40a0a30 0x7f56e40a0e00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 --2- 172.21.15.32:0/636538226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e409d630 0x7f56e40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 -- 172.21.15.32:0/636538226 >> 172.21.15.32:0/636538226 conn(0x7f56e40a3050 msgr2=0x7f56e40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.055+0000 7f56eba83700 1 -- 172.21.15.32:0/636538226 shutdown_connections 2023-09-30T22:27:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.056+0000 7f56eba83700 1 -- 172.21.15.32:0/636538226 wait complete. 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.056+0000 7f56eba83700 1 Processor -- start 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 -- start start 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e409d630 0x7f56e41c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 0x7f56e41c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e4133930 0x7f56e41c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4120830 con 0x7f56e409d630 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4122020 con 0x7f56e4133930 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56eba83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40a4330 con 0x7f56e40a0a30 2023-09-30T22:27:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e901e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 0x7f56e41c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e901e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 0x7f56e41c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49722/0 (socket says 172.21.15.32:49722) 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e901e700 1 -- 172.21.15.32:0/1648645467 learned_addr learned my addr 172.21.15.32:0/1648645467 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e981f700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e409d630 0x7f56e41c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56ea020700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e4133930 0x7f56e41c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e981f700 1 -- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 msgr2=0x7f56e41c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e981f700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 0x7f56e41c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e981f700 1 -- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e4133930 msgr2=0x7f56e41c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.057+0000 7f56e981f700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e4133930 0x7f56e41c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56e981f700 1 -- 172.21.15.32:0/1648645467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094a0 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56e901e700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 0x7f56e41c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56e981f700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e409d630 0x7f56e41c7f90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f56e0016000 tx=0x7f56e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0004300 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e001f070 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e4133550 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00036a0 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.058+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41225c0 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.059+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56e0021030 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.059+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f56e00405c0 con 0x7f56e409d630 2023-09-30T22:27:03.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.059+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e40747f0 con 0x7f56e409d630 2023-09-30T22:27:03.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.064+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56e0015080 con 0x7f56e409d630 2023-09-30T22:27:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 Processor -- start 2023-09-30T22:27:03.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 -- start start 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a981316c0 0x7f7a98131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9809e130 con 0x7f7a9809d9f0 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a981312b0 con 0x7f7a981316c0 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9e726700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9809f6e0 con 0x7f7a9806ae80 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9ccc3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a97fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a981316c0 0x7f7a98131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.097+0000 7f7a9ccc3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47072/0 (socket says 172.21.15.32:47072) 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a9ccc3700 1 -- 172.21.15.32:0/1307523013 learned_addr learned my addr 172.21.15.32:0/1307523013 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a97fff700 1 -- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 msgr2=0x7f7a98139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a97fff700 1 --2- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a97fff700 1 -- 172.21.15.32:0/1307523013 >> v1:172.21.15.32:6789/0 conn(0x7f7a9809d9f0 legacy=0x7f7a9806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a96ffd700 1 -- 172.21.15.32:0/1307523013 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027783128 0 0) 0x7f7a8c001110 con 0x7f7a9809d9f0 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a97fff700 1 -- 172.21.15.32:0/1307523013 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9809e760 con 0x7f7a981316c0 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a9ccc3700 1 --2- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a97fff700 1 --2- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98131a90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009840 tx=0x7f7a88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc81ecff79d80ee8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a96ffd700 1 -- 172.21.15.32:0/1307523013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011830 con 0x7f7a981316c0 2023-09-30T22:27:03.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a96ffd700 1 -- 172.21.15.32:0/1307523013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88011990 con 0x7f7a981316c0 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.098+0000 7f7a96ffd700 1 -- 172.21.15.32:0/1307523013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011d20 con 0x7f7a981316c0 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 -- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 msgr2=0x7f7a98131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 --2- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98131a90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009840 tx=0x7f7a88009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 -- 172.21.15.32:0/1307523013 shutdown_connections 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 --2- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 --2- 172.21.15.32:0/1307523013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98131a90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 -- 172.21.15.32:0/1307523013 >> 172.21.15.32:0/1307523013 conn(0x7f7a98066f50 msgr2=0x7f7a9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 -- 172.21.15.32:0/1307523013 shutdown_connections 2023-09-30T22:27:03.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.099+0000 7f7a9e726700 1 -- 172.21.15.32:0/1307523013 wait complete. 2023-09-30T22:27:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.100+0000 7f7a9e726700 1 Processor -- start 2023-09-30T22:27:03.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 -- start start 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9809d9f0 0x7f7a98071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9811ac10 con 0x7f7a981316c0 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9811adf0 con 0x7f7a9806ae80 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a9e726700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9812b900 con 0x7f7a9809d9f0 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a97fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a97fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47086/0 (socket says 172.21.15.32:47086) 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a97fff700 1 -- 172.21.15.32:0/2059935897 learned_addr learned my addr 172.21.15.32:0/2059935897 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.101+0000 7f7a977fe700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9809d9f0 0x7f7a98071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a9ccc3700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a97fff700 1 -- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9809d9f0 msgr2=0x7f7a98071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a97fff700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9809d9f0 0x7f7a98071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a97fff700 1 -- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 msgr2=0x7f7a98071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a97fff700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a97fff700 1 -- 172.21.15.32:0/2059935897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009b30 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a97fff700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98070ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009fd0 tx=0x7f7a88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a880036a0 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8801f070 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003a20 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880094a0 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.102+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a980a3760 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.103+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a88003b80 con 0x7f7a9806ae80 2023-09-30T22:27:03.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.103+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7a88021070 con 0x7f7a9806ae80 2023-09-30T22:27:03.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.103+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a98074ea0 con 0x7f7a9806ae80 2023-09-30T22:27:03.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.108+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a88015020 con 0x7f7a9806ae80 2023-09-30T22:27:03.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.280+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f56e4072980 con 0x7f56e409d630 2023-09-30T22:27:03.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.281+0000 7f56da7fc700 1 -- 172.21.15.32:0/1648645467 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56e0003800 con 0x7f56e409d630 2023-09-30T22:27:03.281 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:03.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e409d630 msgr2=0x7f56e41c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e409d630 0x7f56e41c7f90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f56e0016000 tx=0x7f56e0004750 comp rx=0 tx=0).stop 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 shutdown_connections 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e4133930 0x7f56e41c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e409d630 0x7f56e41c7f90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 --2- 172.21.15.32:0/1648645467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e40a0a30 0x7f56e41c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 >> 172.21.15.32:0/1648645467 conn(0x7f56e40a3050 msgr2=0x7f56e409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.283+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 shutdown_connections 2023-09-30T22:27:03.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.284+0000 7f56eba83700 1 -- 172.21.15.32:0/1648645467 wait complete. 2023-09-30T22:27:03.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.323+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a9806c360 con 0x7f7a9806ae80 2023-09-30T22:27:03.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.323+0000 7f7a94ff9700 1 -- 172.21.15.32:0/2059935897 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a8802b020 con 0x7f7a9806ae80 2023-09-30T22:27:03.323 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 msgr2=0x7f7a98070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98070ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009fd0 tx=0x7f7a88004750 comp rx=0 tx=0).stop 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 shutdown_connections 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a981316c0 0x7f7a98071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9806ae80 0x7f7a98070ae0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 --2- 172.21.15.32:0/2059935897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9809d9f0 0x7f7a98071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 >> 172.21.15.32:0/2059935897 conn(0x7f7a98066f50 msgr2=0x7f7a980a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 shutdown_connections 2023-09-30T22:27:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.326+0000 7f7a9e726700 1 -- 172.21.15.32:0/2059935897 wait complete. 2023-09-30T22:27:03.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:03.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:03.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.979+0000 7f72f7bc7700 1 Processor -- start 2023-09-30T22:27:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.979+0000 7f72f7bc7700 1 -- start start 2023-09-30T22:27:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f7bc7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 0x7f72f00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f7bc7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 0x7f72f00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f7bc7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f0133930 0x7f72f0133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f7bc7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f00a2a70 con 0x7f72f009d630 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f7bc7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00a01d0 con 0x7f72f0133930 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f7bc7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f009e2b0 con 0x7f72f00a0a30 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5162700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 0x7f72f00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 0x7f72f00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5162700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 0x7f72f00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60882/0 (socket says 172.21.15.32:60882) 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 0x7f72f00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49730/0 (socket says 172.21.15.32:49730) 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5162700 1 -- 172.21.15.32:0/727089169 learned_addr learned my addr 172.21.15.32:0/727089169 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f6164700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f0133930 0x7f72f0133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 -- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f0133930 msgr2=0x7f72f0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f0133930 0x7f72f0133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 -- 172.21.15.32:0/727089169 >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 msgr2=0x7f72f00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 --2- 172.21.15.32:0/727089169 >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 0x7f72f00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5963700 1 -- 172.21.15.32:0/727089169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f009dc50 con 0x7f72f00a0a30 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f5162700 1 --2- 172.21.15.32:0/727089169 >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 0x7f72f00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.980+0000 7f72f6164700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f0133930 0x7f72f0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.981+0000 7f72f5963700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 0x7f72f00a0e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009d90 tx=0x7f72e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a701004ec974faaf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.981+0000 7f72f4961700 1 -- 172.21.15.32:0/727089169 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e000e070 con 0x7f72f00a0a30 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.981+0000 7f72f4961700 1 -- 172.21.15.32:0/727089169 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e000bb20 con 0x7f72f00a0a30 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.981+0000 7f72f4961700 1 -- 172.21.15.32:0/727089169 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0012430 con 0x7f72f00a0a30 2023-09-30T22:27:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.981+0000 7f72f7bc7700 1 -- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 msgr2=0x7f72f00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.981+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 0x7f72f00a0e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009d90 tx=0x7f72e000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 -- 172.21.15.32:0/727089169 shutdown_connections 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f0133930 0x7f72f0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/727089169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f00a0a30 0x7f72f00a0e00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/727089169 >> v2:172.21.15.32:3300/0 conn(0x7f72f009d630 0x7f72f00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 -- 172.21.15.32:0/727089169 >> 172.21.15.32:0/727089169 conn(0x7f72f00a3050 msgr2=0x7f72f00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 -- 172.21.15.32:0/727089169 shutdown_connections 2023-09-30T22:27:03.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.982+0000 7f72f7bc7700 1 -- 172.21.15.32:0/727089169 wait complete. 2023-09-30T22:27:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.983+0000 7f72f7bc7700 1 Processor -- start 2023-09-30T22:27:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.983+0000 7f72f7bc7700 1 -- start start 2023-09-30T22:27:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f7bc7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f009d630 0x7f72f011f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f7bc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 0x7f72f011f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f7bc7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f0133930 0x7f72f01c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f7bc7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0120940 con 0x7f72f00a0a30 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f7bc7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0120750 con 0x7f72f009d630 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f7bc7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f01334d0 con 0x7f72f0133930 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f6164700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f0133930 0x7f72f01c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 0x7f72f011f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 0x7f72f011f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60900/0 (socket says 172.21.15.32:60900) 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f6164700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f0133930 0x7f72f01c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49732/0 (socket says 172.21.15.32:49732) 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 -- 172.21.15.32:0/586546586 learned_addr learned my addr 172.21.15.32:0/586546586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5963700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f009d630 0x7f72f011f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 -- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f0133930 msgr2=0x7f72f01c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f0133930 0x7f72f01c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 -- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f009d630 msgr2=0x7f72f011f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f009d630 0x7f72f011f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.984+0000 7f72f5162700 1 -- 172.21.15.32:0/586546586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4009550 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.985+0000 7f72f5162700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 0x7f72f011f9c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f72e400ebe0 tx=0x7f72e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.985+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400c850 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.985+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e400c9b0 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.985+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e80094a0 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.985+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400cd60 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.985+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e0009a70 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.986+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72e4013470 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.986+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f72e403f740 con 0x7f72f00a0a30 2023-09-30T22:27:03.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.986+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f00747f0 con 0x7f72f00a0a30 2023-09-30T22:27:03.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:03.995+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72e4021080 con 0x7f72f00a0a30 2023-09-30T22:27:04.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 Processor -- start 2023-09-30T22:27:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 -- start start 2023-09-30T22:27:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 0x7f5c7806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c78131d10 0x7f5c78139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7809ece0 con 0x7f5c7806b080 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7809e760 con 0x7f5c78131d10 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.021+0000 7f5c7caa8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7809e040 con 0x7f5c7806a770 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 0x7f5c7806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c76d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c78131d10 0x7f5c78139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 0x7f5c7806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49746/0 (socket says 172.21.15.32:49746) 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 -- 172.21.15.32:0/3544327118 learned_addr learned my addr 172.21.15.32:0/3544327118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c75d9b700 1 --1- 172.21.15.32:0/3544327118 >> v1:172.21.15.32:6789/0 conn(0x7f5c7806b080 0x7f5c78131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47562/0 (socket says 172.21.15.32:47562) 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 -- 172.21.15.32:0/3544327118 >> v2:172.21.15.32:3300/0 conn(0x7f5c78131d10 msgr2=0x7f5c78139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 --2- 172.21.15.32:0/3544327118 >> v2:172.21.15.32:3300/0 conn(0x7f5c78131d10 0x7f5c78139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 -- 172.21.15.32:0/3544327118 >> v1:172.21.15.32:6789/0 conn(0x7f5c7806b080 legacy=0x7f5c78131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 -- 172.21.15.32:0/3544327118 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7809f630 con 0x7f5c7806a770 2023-09-30T22:27:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7559a700 1 -- 172.21.15.32:0/3544327118 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514824399 0 0) 0x7f5c7809ece0 con 0x7f5c7806b080 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c76d9d700 1 --2- 172.21.15.32:0/3544327118 >> v2:172.21.15.32:3300/0 conn(0x7f5c78131d10 0x7f5c78139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7659c700 1 --2- 172.21.15.32:0/3544327118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 0x7f5c7806ab40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c680098f0 tx=0x7f5c68009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee63bf27f3c3602 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7559a700 1 -- 172.21.15.32:0/3544327118 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011830 con 0x7f5c7806a770 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7559a700 1 -- 172.21.15.32:0/3544327118 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68011990 con 0x7f5c7806a770 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.022+0000 7f5c7559a700 1 -- 172.21.15.32:0/3544327118 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011d20 con 0x7f5c7806a770 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 -- 172.21.15.32:0/3544327118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 msgr2=0x7f5c7806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/3544327118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 0x7f5c7806ab40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c680098f0 tx=0x7f5c68009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 -- 172.21.15.32:0/3544327118 shutdown_connections 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/3544327118 >> v2:172.21.15.32:3300/0 conn(0x7f5c78131d10 0x7f5c78139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/3544327118 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7806a770 0x7f5c7806ab40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 -- 172.21.15.32:0/3544327118 >> 172.21.15.32:0/3544327118 conn(0x7f5c78066f50 msgr2=0x7f5c7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 -- 172.21.15.32:0/3544327118 shutdown_connections 2023-09-30T22:27:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.023+0000 7f5c7caa8700 1 -- 172.21.15.32:0/3544327118 wait complete. 2023-09-30T22:27:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.024+0000 7f5c7caa8700 1 Processor -- start 2023-09-30T22:27:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.024+0000 7f5c7caa8700 1 -- start start 2023-09-30T22:27:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7caa8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7806a770 0x7f5c7809abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7caa8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 0x7f5c7809b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7caa8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c78131d10 0x7f5c781c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7caa8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781311d0 con 0x7f5c7806a770 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7caa8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78071e60 con 0x7f5c7806b080 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7caa8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78124ed0 con 0x7f5c78131d10 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c75d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 0x7f5c7809b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7806a770 0x7f5c7809abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c76d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c78131d10 0x7f5c781c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c75d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 0x7f5c7809b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47098/0 (socket says 172.21.15.32:47098) 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c75d9b700 1 -- 172.21.15.32:0/2311467735 learned_addr learned my addr 172.21.15.32:0/2311467735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7659c700 1 -- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c78131d10 msgr2=0x7f5c781c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7659c700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c78131d10 0x7f5c781c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7659c700 1 -- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 msgr2=0x7f5c7809b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7659c700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 0x7f5c7809b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c7659c700 1 -- 172.21.15.32:0/2311467735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70009a70 con 0x7f5c7806a770 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c76d9d700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c78131d10 0x7f5c781c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.025+0000 7f5c75d9b700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 0x7f5c7809b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.026+0000 7f5c7659c700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7806a770 0x7f5c7809abb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5c68016000 tx=0x7f5c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.026+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68004340 con 0x7f5c7806a770 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.026+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6801f070 con 0x7f5c7806a770 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.026+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c68009550 con 0x7f5c7806a770 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.026+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680036a0 con 0x7f5c7806a770 2023-09-30T22:27:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.026+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c78125e90 con 0x7f5c7806a770 2023-09-30T22:27:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.027+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78131580 con 0x7f5c7806a770 2023-09-30T22:27:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.027+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c68021030 con 0x7f5c7806a770 2023-09-30T22:27:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.027+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c68040870 con 0x7f5c7806a770 2023-09-30T22:27:04.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.032+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c680239c0 con 0x7f5c7806a770 2023-09-30T22:27:04.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.207+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f72f0120f40 con 0x7f72f00a0a30 2023-09-30T22:27:04.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.208+0000 7f72ee7fc700 1 -- 172.21.15.32:0/586546586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72e4017020 con 0x7f72f00a0a30 2023-09-30T22:27:04.208 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:04.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 msgr2=0x7f72f011f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 0x7f72f011f9c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f72e400ebe0 tx=0x7f72e400eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 shutdown_connections 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f0133930 0x7f72f01c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f009d630 0x7f72f011f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 --2- 172.21.15.32:0/586546586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f00a0a30 0x7f72f011f9c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 >> 172.21.15.32:0/586546586 conn(0x7f72f00a3050 msgr2=0x7f72f009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 shutdown_connections 2023-09-30T22:27:04.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.210+0000 7f72f7bc7700 1 -- 172.21.15.32:0/586546586 wait complete. 2023-09-30T22:27:04.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.239+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5c78074ea0 con 0x7f5c7806a770 2023-09-30T22:27:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.239+0000 7f5c66ffd700 1 -- 172.21.15.32:0/2311467735 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c68015030 con 0x7f5c7806a770 2023-09-30T22:27:04.239 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:04.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7806a770 msgr2=0x7f5c7809abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7806a770 0x7f5c7809abb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5c68016000 tx=0x7f5c68004750 comp rx=0 tx=0).stop 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 shutdown_connections 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c78131d10 0x7f5c781c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7806a770 0x7f5c7809abb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 --2- 172.21.15.32:0/2311467735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7806b080 0x7f5c7809b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 >> 172.21.15.32:0/2311467735 conn(0x7f5c78066f50 msgr2=0x7f5c78070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 shutdown_connections 2023-09-30T22:27:04.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.241+0000 7f5c7caa8700 1 -- 172.21.15.32:0/2311467735 wait complete. 2023-09-30T22:27:04.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:04.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:04.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:04.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.441+0000 7f3ec9e4b700 1 Processor -- start 2023-09-30T22:27:04.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.441+0000 7f3ec9e4b700 1 -- start start 2023-09-30T22:27:04.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.441+0000 7f3ec9e4b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ec40a0a30 0x7f3ec40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.441+0000 7f3ec9e4b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.441+0000 7f3ec9e4b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec409edd0 con 0x7f3ec4133fa0 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec9e4b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec409e1f0 con 0x7f3ec40a0a30 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec9e4b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec409dc50 con 0x7f3ec40a1340 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ec40a0a30 0x7f3ec40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ec4133fa0 0x7f3ec413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47576/0 (socket says 172.21.15.32:47576) 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec3fff700 1 -- 172.21.15.32:0/2313851085 learned_addr learned my addr 172.21.15.32:0/2313851085 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47112/0 (socket says 172.21.15.32:47112) 2023-09-30T22:27:04.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec37fe700 1 -- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 msgr2=0x7f3ec4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec37fe700 1 --2- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec37fe700 1 -- 172.21.15.32:0/2313851085 >> v1:172.21.15.32:6789/0 conn(0x7f3ec4133fa0 legacy=0x7f3ec413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec37fe700 1 -- 172.21.15.32:0/2313851085 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec40a1fe0 con 0x7f3ec40a0a30 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec27fc700 1 -- 172.21.15.32:0/2313851085 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644863857 0 0) 0x7f3ec409edd0 con 0x7f3ec4133fa0 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec2ffd700 1 --2- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec37fe700 1 --2- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec40a0a30 0x7f3ec40a0e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009840 tx=0x7f3eb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be7b3fe2216a7f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec27fc700 1 -- 172.21.15.32:0/2313851085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4011830 con 0x7f3ec40a0a30 2023-09-30T22:27:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec27fc700 1 -- 172.21.15.32:0/2313851085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb4011990 con 0x7f3ec40a0a30 2023-09-30T22:27:04.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.442+0000 7f3ec27fc700 1 -- 172.21.15.32:0/2313851085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4011d20 con 0x7f3ec40a0a30 2023-09-30T22:27:04.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec40a0a30 msgr2=0x7f3ec40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec40a0a30 0x7f3ec40a0e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009840 tx=0x7f3eb4009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:04.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/2313851085 shutdown_connections 2023-09-30T22:27:04.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec40a0a30 0x7f3ec40a0e00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/2313851085 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/2313851085 >> 172.21.15.32:0/2313851085 conn(0x7f3ec40a6550 msgr2=0x7f3ec40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/2313851085 shutdown_connections 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.443+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/2313851085 wait complete. 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.444+0000 7f3ec9e4b700 1 Processor -- start 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 -- start start 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ec40a0a30 0x7f3ec4122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec4133fa0 0x7f3ec4123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4124cb0 con 0x7f3ec4133fa0 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4125d80 con 0x7f3ec40a1340 2023-09-30T22:27:04.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec9e4b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4124520 con 0x7f3ec40a0a30 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ec40a0a30 0x7f3ec4122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ec40a0a30 0x7f3ec4122890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49768/0 (socket says 172.21.15.32:49768) 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec37fe700 1 -- 172.21.15.32:0/699942776 learned_addr learned my addr 172.21.15.32:0/699942776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4122dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47124/0 (socket says 172.21.15.32:47124) 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.445+0000 7f3ec3fff700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec4133fa0 0x7f3ec4123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec2ffd700 1 -- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ec40a0a30 msgr2=0x7f3ec4122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec2ffd700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ec40a0a30 0x7f3ec4122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec2ffd700 1 -- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec4133fa0 msgr2=0x7f3ec4123310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec2ffd700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec4133fa0 0x7f3ec4123310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec2ffd700 1 -- 172.21.15.32:0/699942776 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80094a0 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec3fff700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec4133fa0 0x7f3ec4123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec2ffd700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4122dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800e9b0 tx=0x7f3eb800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb800ee50 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb800c850 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb40094a0 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb800cc00 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.446+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec4126040 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.447+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec40a1d40 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.448+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3eb8013470 con 0x7f3ec40a1340 2023-09-30T22:27:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.448+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3eb803ff90 con 0x7f3ec40a1340 2023-09-30T22:27:04.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.453+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3eb8013720 con 0x7f3ec40a1340 2023-09-30T22:27:04.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.483+0000 7efe8a8d6700 1 Processor -- start 2023-09-30T22:27:04.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.483+0000 7efe8a8d6700 1 -- start start 2023-09-30T22:27:04.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.483+0000 7efe8a8d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.483+0000 7efe8a8d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe8a8d6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8409edd0 con 0x7efe840a0a30 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe8a8d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840a18e0 con 0x7efe840a1510 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe8a8d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe8409e1f0 con 0x7efe84139250 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe837fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe88e73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe83fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe840a0a30 0x7efe840a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47584/0 (socket says 172.21.15.32:47584) 2023-09-30T22:27:04.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe83fff700 1 -- 172.21.15.32:0/261433832 learned_addr learned my addr 172.21.15.32:0/261433832 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe88e73700 1 -- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 msgr2=0x7efe84138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe88e73700 1 --2- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe88e73700 1 -- 172.21.15.32:0/261433832 >> v1:172.21.15.32:6789/0 conn(0x7efe840a0a30 legacy=0x7efe840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe82ffd700 1 -- 172.21.15.32:0/261433832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890435256 0 0) 0x7efe8409edd0 con 0x7efe840a0a30 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe88e73700 1 -- 172.21.15.32:0/261433832 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8409dc50 con 0x7efe84139250 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe837fe700 1 --2- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe88e73700 1 --2- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8413b630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009dc0 tx=0x7efe7c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f641cc50dab2c82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe82ffd700 1 -- 172.21.15.32:0/261433832 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00e070 con 0x7efe84139250 2023-09-30T22:27:04.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.484+0000 7efe82ffd700 1 -- 172.21.15.32:0/261433832 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c00bb20 con 0x7efe84139250 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe82ffd700 1 -- 172.21.15.32:0/261433832 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c012430 con 0x7efe84139250 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 -- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 msgr2=0x7efe8413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8413b630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009dc0 tx=0x7efe7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 -- 172.21.15.32:0/261433832 shutdown_connections 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8413b630 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/261433832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 -- 172.21.15.32:0/261433832 >> 172.21.15.32:0/261433832 conn(0x7efe840a6550 msgr2=0x7efe840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 -- 172.21.15.32:0/261433832 shutdown_connections 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.485+0000 7efe8a8d6700 1 -- 172.21.15.32:0/261433832 wait complete. 2023-09-30T22:27:04.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.486+0000 7efe8a8d6700 1 Processor -- start 2023-09-30T22:27:04.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.486+0000 7efe8a8d6700 1 -- start start 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe8a8d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe840a0a30 0x7efe84133680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe8a8d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe8a8d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8411e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe8a8d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8411fd50 con 0x7efe840a0a30 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe8a8d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8409c4f0 con 0x7efe840a1510 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe8a8d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe840a0220 con 0x7efe84139250 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe837fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe88e73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8411e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe837fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84133bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47152/0 (socket says 172.21.15.32:47152) 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe837fe700 1 -- 172.21.15.32:0/751257522 learned_addr learned my addr 172.21.15.32:0/751257522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:04.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.487+0000 7efe83fff700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe840a0a30 0x7efe84133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe837fe700 1 -- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 msgr2=0x7efe8411e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe837fe700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8411e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe837fe700 1 -- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe840a0a30 msgr2=0x7efe84133680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe837fe700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe840a0a30 0x7efe84133680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe837fe700 1 -- 172.21.15.32:0/751257522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe780094a0 con 0x7efe840a1510 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe88e73700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe837fe700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84133bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efe7800ebe0 tx=0x7efe7800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800c850 con 0x7efe840a1510 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7800c9b0 con 0x7efe840a1510 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c009a70 con 0x7efe840a1510 2023-09-30T22:27:04.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800cd40 con 0x7efe840a1510 2023-09-30T22:27:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.488+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe841220d0 con 0x7efe840a1510 2023-09-30T22:27:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.489+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe84121f30 con 0x7efe840a1510 2023-09-30T22:27:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.490+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe78013470 con 0x7efe840a1510 2023-09-30T22:27:04.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.490+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efe7803fb30 con 0x7efe840a1510 2023-09-30T22:27:04.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.494+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe84121f30 con 0x7efe840a1510 2023-09-30T22:27:04.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.659+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ec4074ea0 con 0x7f3ec40a1340 2023-09-30T22:27:04.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.660+0000 7f3eabfff700 1 -- 172.21.15.32:0/699942776 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3eb8017070 con 0x7f3ec40a1340 2023-09-30T22:27:04.660 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 msgr2=0x7f3ec4122dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4122dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800e9b0 tx=0x7f3eb800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 shutdown_connections 2023-09-30T22:27:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ec4133fa0 0x7f3ec4123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ec40a0a30 0x7f3ec4122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 --2- 172.21.15.32:0/699942776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ec40a1340 0x7f3ec4122dd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 >> 172.21.15.32:0/699942776 conn(0x7f3ec40a6550 msgr2=0x7f3ec411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 shutdown_connections 2023-09-30T22:27:04.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.662+0000 7f3ec9e4b700 1 -- 172.21.15.32:0/699942776 wait complete. 2023-09-30T22:27:04.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.701+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efe84121f30 con 0x7efe840a1510 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.702+0000 7efe80ff9700 1 -- 172.21.15.32:0/751257522 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe84121f30 con 0x7efe840a1510 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 msgr2=0x7efe84133bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84133bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efe7800ebe0 tx=0x7efe7800eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 shutdown_connections 2023-09-30T22:27:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe84139250 0x7efe8411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe840a0a30 0x7efe84133680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 --2- 172.21.15.32:0/751257522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe840a1510 0x7efe84133bc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 >> 172.21.15.32:0/751257522 conn(0x7efe840a6550 msgr2=0x7efe840a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 shutdown_connections 2023-09-30T22:27:04.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:04.704+0000 7efe8a8d6700 1 -- 172.21.15.32:0/751257522 wait complete. 2023-09-30T22:27:04.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:05.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:05.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.375+0000 7ff2617de700 1 Processor -- start 2023-09-30T22:27:05.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.375+0000 7ff2617de700 1 -- start start 2023-09-30T22:27:05.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.375+0000 7ff2617de700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff25c0a0a30 0x7ff25c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff2617de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 0x7ff25c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff2617de700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0a2a70 con 0x7ff25c133930 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff2617de700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0a01d0 con 0x7ff25c0a0a30 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff2617de700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c09e2b0 con 0x7ff25c09d630 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 0x7ff25c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff25c0a0a30 0x7ff25c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff25c133930 0x7ff25c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47586/0 (socket says 172.21.15.32:47586) 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25b7fe700 1 -- 172.21.15.32:0/1099687951 learned_addr learned my addr 172.21.15.32:0/1099687951 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 0x7ff25c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47156/0 (socket says 172.21.15.32:47156) 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 -- 172.21.15.32:0/1099687951 >> v2:172.21.15.32:3300/0 conn(0x7ff25c0a0a30 msgr2=0x7ff25c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 --2- 172.21.15.32:0/1099687951 >> v2:172.21.15.32:3300/0 conn(0x7ff25c0a0a30 0x7ff25c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 -- 172.21.15.32:0/1099687951 >> v1:172.21.15.32:6789/0 conn(0x7ff25c133930 legacy=0x7ff25c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff259ffb700 1 -- 172.21.15.32:0/1099687951 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379558853 0 0) 0x7ff25c0a2a70 con 0x7ff25c133930 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 -- 172.21.15.32:0/1099687951 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c09dc50 con 0x7ff25c09d630 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25affd700 1 --2- 172.21.15.32:0/1099687951 >> v2:172.21.15.32:3300/0 conn(0x7ff25c0a0a30 0x7ff25c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff25a7fc700 1 --2- 172.21.15.32:0/1099687951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 0x7ff25c0a1340 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff250009840 tx=0x7ff250009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fc50efba5ca81a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff259ffb700 1 -- 172.21.15.32:0/1099687951 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250011830 con 0x7ff25c09d630 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.376+0000 7ff259ffb700 1 -- 172.21.15.32:0/1099687951 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250011990 con 0x7ff25c09d630 2023-09-30T22:27:05.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff259ffb700 1 -- 172.21.15.32:0/1099687951 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250011d20 con 0x7ff25c09d630 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 -- 172.21.15.32:0/1099687951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 msgr2=0x7ff25c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 --2- 172.21.15.32:0/1099687951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 0x7ff25c0a1340 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff250009840 tx=0x7ff250009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 -- 172.21.15.32:0/1099687951 shutdown_connections 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 --2- 172.21.15.32:0/1099687951 >> v2:172.21.15.32:3300/0 conn(0x7ff25c0a0a30 0x7ff25c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 --2- 172.21.15.32:0/1099687951 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c09d630 0x7ff25c0a1340 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 -- 172.21.15.32:0/1099687951 >> 172.21.15.32:0/1099687951 conn(0x7ff25c0a3050 msgr2=0x7ff25c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 -- 172.21.15.32:0/1099687951 shutdown_connections 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.377+0000 7ff2617de700 1 -- 172.21.15.32:0/1099687951 wait complete. 2023-09-30T22:27:05.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.378+0000 7ff2617de700 1 Processor -- start 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 -- start start 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff25c09d630 0x7ff25c1c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 0x7ff25c1c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c133930 0x7ff25c1c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c1221b0 con 0x7ff25c133930 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c133520 con 0x7ff25c0a0a30 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff2617de700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c120700 con 0x7ff25c09d630 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 0x7ff25c1c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 0x7ff25c1c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47166/0 (socket says 172.21.15.32:47166) 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25a7fc700 1 -- 172.21.15.32:0/2600554575 learned_addr learned my addr 172.21.15.32:0/2600554575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:05.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25affd700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff25c09d630 0x7ff25c1c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25b7fe700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c133930 0x7ff25c1c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25a7fc700 1 -- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff25c09d630 msgr2=0x7ff25c1c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25a7fc700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff25c09d630 0x7ff25c1c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.379+0000 7ff25a7fc700 1 -- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c133930 msgr2=0x7ff25c1c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff25a7fc700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c133930 0x7ff25c1c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff25a7fc700 1 -- 172.21.15.32:0/2600554575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009a70 con 0x7ff25c0a0a30 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff25a7fc700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 0x7ff25c1c84c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff250009fd0 tx=0x7ff250004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2500036a0 con 0x7ff25c0a0a30 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25001f070 con 0x7ff25c0a0a30 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2500094a0 con 0x7ff25c0a0a30 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250003a60 con 0x7ff25c0a0a30 2023-09-30T22:27:05.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.380+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c122c20 con 0x7ff25c0a0a30 2023-09-30T22:27:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.381+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff250003bc0 con 0x7ff25c0a0a30 2023-09-30T22:27:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.381+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff250021070 con 0x7ff25c0a0a30 2023-09-30T22:27:05.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.381+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c0747f0 con 0x7ff25c0a0a30 2023-09-30T22:27:05.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.386+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff250015080 con 0x7ff25c0a0a30 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.413+0000 7f1d1bd74700 1 Processor -- start 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1bd74700 1 -- start start 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1bd74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1bd74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d140a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1bd74700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d140a2a70 con 0x7f1d14133930 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1bd74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140a01d0 con 0x7f1d140a0a30 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1bd74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1409e2b0 con 0x7f1d1409d630 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d19b10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d1930f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d140a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d19b10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47170/0 (socket says 172.21.15.32:47170) 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.414+0000 7f1d19b10700 1 -- 172.21.15.32:0/3317494691 learned_addr learned my addr 172.21.15.32:0/3317494691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d18b0e700 1 -- 172.21.15.32:0/3317494691 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091990033 0 0) 0x7f1d140a2a70 con 0x7f1d14133930 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d19b10700 1 -- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 msgr2=0x7f1d140a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d19b10700 1 --2- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d140a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d19b10700 1 -- 172.21.15.32:0/3317494691 >> v1:172.21.15.32:6789/0 conn(0x7f1d14133930 legacy=0x7f1d14133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d19b10700 1 -- 172.21.15.32:0/3317494691 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1409dc50 con 0x7f1d140a0a30 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d1930f700 1 --2- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d140a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d19b10700 1 --2- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d140a0e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009e80 tx=0x7f1d0400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624ae19a3b18b195 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d18b0e700 1 -- 172.21.15.32:0/3317494691 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400e070 con 0x7f1d140a0a30 2023-09-30T22:27:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d18b0e700 1 -- 172.21.15.32:0/3317494691 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0400b910 con 0x7f1d140a0a30 2023-09-30T22:27:05.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.415+0000 7f1d18b0e700 1 -- 172.21.15.32:0/3317494691 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400bcc0 con 0x7f1d140a0a30 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 msgr2=0x7f1d140a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d140a0e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009e80 tx=0x7f1d0400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3317494691 shutdown_connections 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d140a0e00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3317494691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d140a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3317494691 >> 172.21.15.32:0/3317494691 conn(0x7f1d140a3050 msgr2=0x7f1d140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3317494691 shutdown_connections 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.416+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3317494691 wait complete. 2023-09-30T22:27:05.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 Processor -- start 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 -- start start 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d1411f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d1411f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d14133930 0x7f1d141c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14121ed0 con 0x7f1d14133930 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14133250 con 0x7f1d140a0a30 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.417+0000 7f1d1bd74700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1411ee20 con 0x7f1d1409d630 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d1411f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d1411f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49824/0 (socket says 172.21.15.32:49824) 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 -- 172.21.15.32:0/3569331300 learned_addr learned my addr 172.21.15.32:0/3569331300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d1a311700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d14133930 0x7f1d141c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d1930f700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d1411f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 -- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 msgr2=0x7f1d1411f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d1411f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 -- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d14133930 msgr2=0x7f1d141c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d14133930 0x7f1d141c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 -- 172.21.15.32:0/3569331300 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080094a0 con 0x7f1d1409d630 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d19b10700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d1411f2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400b680 tx=0x7f1d04004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d040092e0 con 0x7f1d1409d630 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04004250 con 0x7f1d1409d630 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d04009b30 con 0x7f1d1409d630 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04004600 con 0x7f1d1409d630 2023-09-30T22:27:05.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.418+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14120520 con 0x7f1d1409d630 2023-09-30T22:27:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.419+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1411fda0 con 0x7f1d1409d630 2023-09-30T22:27:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.420+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d04004760 con 0x7f1d1409d630 2023-09-30T22:27:05.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.420+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d04040f70 con 0x7f1d1409d630 2023-09-30T22:27:05.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.424+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d04011390 con 0x7f1d1409d630 2023-09-30T22:27:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.597+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff25c072980 con 0x7ff25c0a0a30 2023-09-30T22:27:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.597+0000 7ff2437fe700 1 -- 172.21.15.32:0/2600554575 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff250035090 con 0x7ff25c0a0a30 2023-09-30T22:27:05.597 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:05.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 msgr2=0x7ff25c1c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 0x7ff25c1c84c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff250009fd0 tx=0x7ff250004750 comp rx=0 tx=0).stop 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 shutdown_connections 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c133930 0x7ff25c1c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff25c09d630 0x7ff25c1c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 --2- 172.21.15.32:0/2600554575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff25c0a0a30 0x7ff25c1c84c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.599+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 >> 172.21.15.32:0/2600554575 conn(0x7ff25c0a3050 msgr2=0x7ff25c1bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.600+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 shutdown_connections 2023-09-30T22:27:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.600+0000 7ff2617de700 1 -- 172.21.15.32:0/2600554575 wait complete. 2023-09-30T22:27:05.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.630+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d14120a40 con 0x7f1d1409d630 2023-09-30T22:27:05.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.631+0000 7f1d127fc700 1 -- 172.21.15.32:0/3569331300 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d04033090 con 0x7f1d1409d630 2023-09-30T22:27:05.630 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 msgr2=0x7f1d1411f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d1411f2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400b680 tx=0x7f1d04004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 shutdown_connections 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d14133930 0x7f1d141c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d1409d630 0x7f1d1411f2c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 --2- 172.21.15.32:0/3569331300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d140a0a30 0x7f1d1411f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 >> 172.21.15.32:0/3569331300 conn(0x7f1d140a3050 msgr2=0x7f1d141bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 shutdown_connections 2023-09-30T22:27:05.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.633+0000 7f1d1bd74700 1 -- 172.21.15.32:0/3569331300 wait complete. 2023-09-30T22:27:05.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:05.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:05.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.828+0000 7f260fd97700 1 Processor -- start 2023-09-30T22:27:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.828+0000 7f260fd97700 1 -- start start 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260fd97700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 0x7f26080a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260fd97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 0x7f260813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260fd97700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080a2a70 con 0x7f26080a0a30 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260fd97700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26080a01d0 con 0x7f260809d630 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260fd97700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260809e2b0 con 0x7f2608138d80 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260e334700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 0x7f260813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260d332700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 0x7f26080a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260d332700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 0x7f26080a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60952/0 (socket says 172.21.15.32:60952) 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260e334700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 0x7f260813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47184/0 (socket says 172.21.15.32:47184) 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260d332700 1 -- 172.21.15.32:0/196055062 learned_addr learned my addr 172.21.15.32:0/196055062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260db33700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26080a0a30 0x7f26080a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47604/0 (socket says 172.21.15.32:47604) 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260e334700 1 -- 172.21.15.32:0/196055062 >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 msgr2=0x7f26080a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260e334700 1 --2- 172.21.15.32:0/196055062 >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 0x7f26080a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260e334700 1 -- 172.21.15.32:0/196055062 >> v1:172.21.15.32:6789/0 conn(0x7f26080a0a30 legacy=0x7f26080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260cb31700 1 -- 172.21.15.32:0/196055062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179834124 0 0) 0x7f26080a2a70 con 0x7f26080a0a30 2023-09-30T22:27:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260e334700 1 -- 172.21.15.32:0/196055062 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260809dc50 con 0x7f2608138d80 2023-09-30T22:27:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.829+0000 7f260d332700 1 --2- 172.21.15.32:0/196055062 >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 0x7f26080a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.830+0000 7f260e334700 1 --2- 172.21.15.32:0/196055062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 0x7f260813b160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2600009d90 tx=0x7f260000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50705d30402c1aa1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.830+0000 7f260cb31700 1 -- 172.21.15.32:0/196055062 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260000e070 con 0x7f2608138d80 2023-09-30T22:27:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.830+0000 7f260cb31700 1 -- 172.21.15.32:0/196055062 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260000bb20 con 0x7f2608138d80 2023-09-30T22:27:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.830+0000 7f260cb31700 1 -- 172.21.15.32:0/196055062 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600012430 con 0x7f2608138d80 2023-09-30T22:27:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.830+0000 7f260fd97700 1 -- 172.21.15.32:0/196055062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 msgr2=0x7f260813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.830+0000 7f260fd97700 1 --2- 172.21.15.32:0/196055062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 0x7f260813b160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2600009d90 tx=0x7f260000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.831+0000 7f260fd97700 1 -- 172.21.15.32:0/196055062 shutdown_connections 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.831+0000 7f260fd97700 1 --2- 172.21.15.32:0/196055062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2608138d80 0x7f260813b160 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.831+0000 7f260fd97700 1 --2- 172.21.15.32:0/196055062 >> v2:172.21.15.32:3300/0 conn(0x7f260809d630 0x7f26080a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.831+0000 7f260fd97700 1 -- 172.21.15.32:0/196055062 >> 172.21.15.32:0/196055062 conn(0x7f26080a3050 msgr2=0x7f26080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.831+0000 7f260fd97700 1 -- 172.21.15.32:0/196055062 shutdown_connections 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.831+0000 7f260fd97700 1 -- 172.21.15.32:0/196055062 wait complete. 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 Processor -- start 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 -- start start 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260809d630 0x7f2608133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26080a0a30 0x7f2608133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 0x7f2608134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608137180 con 0x7f2608138d80 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2608136070 con 0x7f26080a0a30 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260fd97700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2608136530 con 0x7f260809d630 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260e334700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 0x7f2608134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260e334700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 0x7f2608134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60960/0 (socket says 172.21.15.32:60960) 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260e334700 1 -- 172.21.15.32:0/3296112306 learned_addr learned my addr 172.21.15.32:0/3296112306 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260d332700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26080a0a30 0x7f2608133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.832+0000 7f260db33700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260809d630 0x7f2608133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260e334700 1 -- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260809d630 msgr2=0x7f2608133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260e334700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260809d630 0x7f2608133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260e334700 1 -- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26080a0a30 msgr2=0x7f2608133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260e334700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26080a0a30 0x7f2608133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260e334700 1 -- 172.21.15.32:0/3296112306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8009550 con 0x7f2608138d80 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260d332700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26080a0a30 0x7f2608133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260e334700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 0x7f2608134180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2600013000 tx=0x7f2600004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600012430 con 0x7f2608138d80 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600020070 con 0x7f2608138d80 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2600009a70 con 0x7f2608138d80 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600004280 con 0x7f2608138d80 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.833+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2608136be0 con 0x7f2608138d80 2023-09-30T22:27:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.834+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f26000043e0 con 0x7f2608138d80 2023-09-30T22:27:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.834+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2600022070 con 0x7f2608138d80 2023-09-30T22:27:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.834+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26080747f0 con 0x7f2608138d80 2023-09-30T22:27:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.839+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f260001b400 con 0x7f2608138d80 2023-09-30T22:27:05.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.869+0000 7f6390d49700 1 Processor -- start 2023-09-30T22:27:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6390d49700 1 -- start start 2023-09-30T22:27:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6390d49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6390d49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f638c09e410 0x7f638c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6390d49700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c09b120 con 0x7f638c09e000 2023-09-30T22:27:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6390d49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c06b460 con 0x7f638c09e410 2023-09-30T22:27:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6390d49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c09a7e0 con 0x7f638c09d6f0 2023-09-30T22:27:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f638a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6381d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f638c09e000 0x7f638c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47606/0 (socket says 172.21.15.32:47606) 2023-09-30T22:27:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f638ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f638c09e410 0x7f638c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.870+0000 7f6381d9b700 1 -- 172.21.15.32:0/2464475471 learned_addr learned my addr 172.21.15.32:0/2464475471 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f6389d9b700 1 -- 172.21.15.32:0/2464475471 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639151400 0 0) 0x7f638c09b120 con 0x7f638c09e000 2023-09-30T22:27:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f6389d9b700 1 -- 172.21.15.32:0/2464475471 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63780035b0 con 0x7f638c09e000 2023-09-30T22:27:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f638a59c700 1 -- 172.21.15.32:0/2464475471 >> v2:172.21.15.32:3300/0 conn(0x7f638c09e410 msgr2=0x7f638c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f638a59c700 1 --2- 172.21.15.32:0/2464475471 >> v2:172.21.15.32:3300/0 conn(0x7f638c09e410 0x7f638c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f638a59c700 1 -- 172.21.15.32:0/2464475471 >> v1:172.21.15.32:6789/0 conn(0x7f638c09e000 legacy=0x7f638c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f638a59c700 1 -- 172.21.15.32:0/2464475471 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c09a500 con 0x7f638c09d6f0 2023-09-30T22:27:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f638ad9d700 1 --2- 172.21.15.32:0/2464475471 >> v2:172.21.15.32:3300/0 conn(0x7f638c09e410 0x7f638c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f638a59c700 1 --2- 172.21.15.32:0/2464475471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c09dac0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f637c009870 tx=0x7f637c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1b962d67d40758 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f6389d9b700 1 -- 172.21.15.32:0/2464475471 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c004230 con 0x7f638c09d6f0 2023-09-30T22:27:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f6389d9b700 1 -- 172.21.15.32:0/2464475471 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c004390 con 0x7f638c09d6f0 2023-09-30T22:27:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.871+0000 7f6389d9b700 1 -- 172.21.15.32:0/2464475471 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0036a0 con 0x7f638c09d6f0 2023-09-30T22:27:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 -- 172.21.15.32:0/2464475471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 msgr2=0x7f638c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 --2- 172.21.15.32:0/2464475471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c09dac0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f637c009870 tx=0x7f637c009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 -- 172.21.15.32:0/2464475471 shutdown_connections 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 --2- 172.21.15.32:0/2464475471 >> v2:172.21.15.32:3300/0 conn(0x7f638c09e410 0x7f638c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 --2- 172.21.15.32:0/2464475471 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c09dac0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 -- 172.21.15.32:0/2464475471 >> 172.21.15.32:0/2464475471 conn(0x7f638c066f50 msgr2=0x7f638c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 -- 172.21.15.32:0/2464475471 shutdown_connections 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.872+0000 7f6390d49700 1 -- 172.21.15.32:0/2464475471 wait complete. 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.873+0000 7f6390d49700 1 Processor -- start 2023-09-30T22:27:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.873+0000 7f6390d49700 1 -- start start 2023-09-30T22:27:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6390d49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c1a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6390d49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e000 0x7f638c1a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6390d49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e410 0x7f638c1c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6390d49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1b92f0 con 0x7f638c09e410 2023-09-30T22:27:05.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6390d49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1a4e40 con 0x7f638c09d6f0 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6390d49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0a2f10 con 0x7f638c09e000 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c1a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6381d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e000 0x7f638c1a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c1a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47220/0 (socket says 172.21.15.32:47220) 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 -- 172.21.15.32:0/2627780430 learned_addr learned my addr 172.21.15.32:0/2627780430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638ad9d700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e410 0x7f638c1c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 -- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e000 msgr2=0x7f638c1a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e000 0x7f638c1a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 -- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e410 msgr2=0x7f638c1c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e410 0x7f638c1c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638a59c700 1 -- 172.21.15.32:0/2627780430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384009a70 con 0x7f638c09d6f0 2023-09-30T22:27:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f638ad9d700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e410 0x7f638c1c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.874+0000 7f6381d9b700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e000 0x7f638c1a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.875+0000 7f638a59c700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c1a52c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f638c09a920 tx=0x7f637c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.875+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c003c60 con 0x7f638c09d6f0 2023-09-30T22:27:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.875+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c01e070 con 0x7f638c09d6f0 2023-09-30T22:27:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.875+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c0094a0 con 0x7f638c09d6f0 2023-09-30T22:27:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.875+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c011630 con 0x7f638c09d6f0 2023-09-30T22:27:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.875+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c1a5d40 con 0x7f638c09d6f0 2023-09-30T22:27:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.876+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f637c011790 con 0x7f638c09d6f0 2023-09-30T22:27:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.876+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f637c0407d0 con 0x7f638c09d6f0 2023-09-30T22:27:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.876+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c1a7e10 con 0x7f638c09d6f0 2023-09-30T22:27:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:05.881+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f637c010970 con 0x7f638c09d6f0 2023-09-30T22:27:06.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.047+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2608137970 con 0x7f2608138d80 2023-09-30T22:27:06.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.048+0000 7f25fe7fc700 1 -- 172.21.15.32:0/3296112306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f260002c020 con 0x7f2608138d80 2023-09-30T22:27:06.048 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 msgr2=0x7f2608134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 0x7f2608134180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2600013000 tx=0x7f2600004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 shutdown_connections 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2608138d80 0x7f2608134180 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f260809d630 0x7f2608133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 --2- 172.21.15.32:0/3296112306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26080a0a30 0x7f2608133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 >> 172.21.15.32:0/3296112306 conn(0x7f26080a3050 msgr2=0x7f260809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 shutdown_connections 2023-09-30T22:27:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.050+0000 7f260fd97700 1 -- 172.21.15.32:0/3296112306 wait complete. 2023-09-30T22:27:06.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:06.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.086+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f638c1a7fe0 con 0x7f638c09d6f0 2023-09-30T22:27:06.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.086+0000 7f63837fe700 1 -- 172.21.15.32:0/2627780430 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f637c035070 con 0x7f638c09d6f0 2023-09-30T22:27:06.086 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:06.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 msgr2=0x7f638c1a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c1a52c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f638c09a920 tx=0x7f637c004750 comp rx=0 tx=0).stop 2023-09-30T22:27:06.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 shutdown_connections 2023-09-30T22:27:06.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e410 0x7f638c1c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09d6f0 0x7f638c1a52c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 --2- 172.21.15.32:0/2627780430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e000 0x7f638c1a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 >> 172.21.15.32:0/2627780430 conn(0x7f638c066f50 msgr2=0x7f638c1bb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 shutdown_connections 2023-09-30T22:27:06.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.089+0000 7f6390d49700 1 -- 172.21.15.32:0/2627780430 wait complete. 2023-09-30T22:27:06.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:06.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:06.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:06.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.762+0000 7fb62578b700 1 Processor -- start 2023-09-30T22:27:06.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.762+0000 7fb62578b700 1 -- start start 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb62578b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb62009f410 0x7fb620136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb62578b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 0x7fb6201393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb62578b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb620070940 con 0x7fb62009e930 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb62578b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62006fd60 con 0x7fb62009f410 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb62578b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb62006fa80 con 0x7fb620136fe0 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb61e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb62009f410 0x7fb620136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb61f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 0x7fb6201393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb61e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb62009f410 0x7fb620136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60994/0 (socket says 172.21.15.32:60994) 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb61f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 0x7fb6201393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49846/0 (socket says 172.21.15.32:49846) 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb61e7fc700 1 -- 172.21.15.32:0/4198408821 learned_addr learned my addr 172.21.15.32:0/4198408821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.763+0000 7fb61effd700 1 --1- 172.21.15.32:0/4198408821 >> v1:172.21.15.32:6789/0 conn(0x7fb62009e930 0x7fb62009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47618/0 (socket says 172.21.15.32:47618) 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61e7fc700 1 -- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 msgr2=0x7fb6201393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61e7fc700 1 --2- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 0x7fb6201393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61e7fc700 1 -- 172.21.15.32:0/4198408821 >> v1:172.21.15.32:6789/0 conn(0x7fb62009e930 legacy=0x7fb62009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61dffb700 1 -- 172.21.15.32:0/4198408821 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795685417 0 0) 0x7fb620070940 con 0x7fb62009e930 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61e7fc700 1 -- 172.21.15.32:0/4198408821 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62009fb60 con 0x7fb62009f410 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61f7fe700 1 --2- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 0x7fb6201393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61e7fc700 1 --2- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb62009f410 0x7fb620136aa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098f0 tx=0x7fb614009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9509d0199a55984a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61dffb700 1 -- 172.21.15.32:0/4198408821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011830 con 0x7fb62009f410 2023-09-30T22:27:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61dffb700 1 -- 172.21.15.32:0/4198408821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614011990 con 0x7fb62009f410 2023-09-30T22:27:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.764+0000 7fb61dffb700 1 -- 172.21.15.32:0/4198408821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011d20 con 0x7fb62009f410 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 -- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb62009f410 msgr2=0x7fb620136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 --2- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb62009f410 0x7fb620136aa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098f0 tx=0x7fb614009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 -- 172.21.15.32:0/4198408821 shutdown_connections 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 --2- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb620136fe0 0x7fb6201393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 --2- 172.21.15.32:0/4198408821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb62009f410 0x7fb620136aa0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 -- 172.21.15.32:0/4198408821 >> 172.21.15.32:0/4198408821 conn(0x7fb6200a4210 msgr2=0x7fb6200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 -- 172.21.15.32:0/4198408821 shutdown_connections 2023-09-30T22:27:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.765+0000 7fb62578b700 1 -- 172.21.15.32:0/4198408821 wait complete. 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.766+0000 7fb62578b700 1 Processor -- start 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.766+0000 7fb62578b700 1 -- start start 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb62578b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 0x7fb620122a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb62578b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb62009f410 0x7fb620122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb62578b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb620136fe0 0x7fb620123510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb62578b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb620071d90 con 0x7fb620136fe0 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb62578b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200a31b0 con 0x7fb62009f410 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb62578b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb620124f00 con 0x7fb62009e930 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 0x7fb620122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 0x7fb620122a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49862/0 (socket says 172.21.15.32:49862) 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 -- 172.21.15.32:0/1880665894 learned_addr learned my addr 172.21.15.32:0/1880665894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61f7fe700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb620136fe0 0x7fb620123510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61e7fc700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb62009f410 0x7fb620122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 -- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb62009f410 msgr2=0x7fb620122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb62009f410 0x7fb620122fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 -- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb620136fe0 msgr2=0x7fb620123510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb620136fe0 0x7fb620123510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 -- 172.21.15.32:0/1880665894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6180094a0 con 0x7fb62009e930 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61f7fe700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb620136fe0 0x7fb620123510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.767+0000 7fb61effd700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 0x7fb620122a90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb61000cc80 tx=0x7fb61000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.768+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61000aaf0 con 0x7fb62009e930 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.768+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb614009550 con 0x7fb62009e930 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.768+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb620124c10 con 0x7fb62009e930 2023-09-30T22:27:06.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.768+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61000ac50 con 0x7fb62009e930 2023-09-30T22:27:06.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.768+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6100070d0 con 0x7fb62009e930 2023-09-30T22:27:06.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.769+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000052e0 con 0x7fb62009e930 2023-09-30T22:27:06.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.769+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb610007230 con 0x7fb62009e930 2023-09-30T22:27:06.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.770+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb61003fa80 con 0x7fb62009e930 2023-09-30T22:27:06.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.778+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb61001b360 con 0x7fb62009e930 2023-09-30T22:27:06.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.804+0000 7f89270cf700 1 Processor -- start 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 -- start start 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 0x7f8920133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89200a2a70 con 0x7f8920133930 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89200a01d0 con 0x7f89200a0a30 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f89270cf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f892009e2b0 con 0x7f892009d630 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f891ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f892566c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 0x7f8920133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f892566c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 0x7f8920133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32792/0 (socket says 172.21.15.32:32792) 2023-09-30T22:27:06.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f892566c700 1 -- 172.21.15.32:0/1551860647 learned_addr learned my addr 172.21.15.32:0/1551860647 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f891ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49864/0 (socket says 172.21.15.32:49864) 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.805+0000 7f8924e6b700 1 --2- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891ffff700 1 -- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 msgr2=0x7f89200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891ffff700 1 --2- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891ffff700 1 -- 172.21.15.32:0/1551860647 >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 msgr2=0x7f8920133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891ffff700 1 --2- 172.21.15.32:0/1551860647 >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 0x7f8920133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891ffff700 1 -- 172.21.15.32:0/1551860647 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892009dc50 con 0x7f892009d630 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f892566c700 1 --2- 172.21.15.32:0/1551860647 >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 0x7f8920133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891ffff700 1 --2- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 0x7f89200a1340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8914009840 tx=0x7f8914009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f062566b4ea9c649 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891f7fe700 1 -- 172.21.15.32:0/1551860647 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011830 con 0x7f892009d630 2023-09-30T22:27:06.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891f7fe700 1 -- 172.21.15.32:0/1551860647 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8914011990 con 0x7f892009d630 2023-09-30T22:27:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.806+0000 7f891f7fe700 1 -- 172.21.15.32:0/1551860647 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011d20 con 0x7f892009d630 2023-09-30T22:27:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 -- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 msgr2=0x7f89200a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 --2- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 0x7f89200a1340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8914009840 tx=0x7f8914009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 -- 172.21.15.32:0/1551860647 shutdown_connections 2023-09-30T22:27:06.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 --2- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 --2- 172.21.15.32:0/1551860647 >> v2:172.21.15.32:3300/0 conn(0x7f8920133930 0x7f8920133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 --2- 172.21.15.32:0/1551860647 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 -- 172.21.15.32:0/1551860647 >> 172.21.15.32:0/1551860647 conn(0x7f89200a3050 msgr2=0x7f89200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 -- 172.21.15.32:0/1551860647 shutdown_connections 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.807+0000 7f89270cf700 1 -- 172.21.15.32:0/1551860647 wait complete. 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.808+0000 7f89270cf700 1 Processor -- start 2023-09-30T22:27:06.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.808+0000 7f89270cf700 1 -- start start 2023-09-30T22:27:06.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f89270cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f892011f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f89270cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f892011fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f89270cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f89270cf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892011ef80 con 0x7f892009d630 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f89270cf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8920122630 con 0x7f89200a0a30 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f89270cf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8920120620 con 0x7f8920133930 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f892566c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f8924e6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f892011f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f8924e6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f892011f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32796/0 (socket says 172.21.15.32:32796) 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f892566c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49876/0 (socket says 172.21.15.32:49876) 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f8924e6b700 1 -- 172.21.15.32:0/184754394 learned_addr learned my addr 172.21.15.32:0/184754394 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:06.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.809+0000 7f891ffff700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f892011fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f8924e6b700 1 -- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 msgr2=0x7f89201c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f8924e6b700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f8924e6b700 1 -- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 msgr2=0x7f892011fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f8924e6b700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f892011fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f8924e6b700 1 -- 172.21.15.32:0/184754394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918009b30 con 0x7f892009d630 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f891ffff700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f892011fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f8924e6b700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f892011f500 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f891000e9b0 tx=0x7f891000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910009710 con 0x7f892009d630 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000ee50 con 0x7f892009d630 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89140094a0 con 0x7f892009d630 2023-09-30T22:27:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000ca50 con 0x7f892009d630 2023-09-30T22:27:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.810+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89200a0410 con 0x7f892009d630 2023-09-30T22:27:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.811+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f891000cbb0 con 0x7f892009d630 2023-09-30T22:27:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.811+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89200747f0 con 0x7f892009d630 2023-09-30T22:27:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.811+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f891003fb10 con 0x7f892009d630 2023-09-30T22:27:06.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.816+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f891001b3b0 con 0x7f892009d630 2023-09-30T22:27:06.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.984+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb600005150 con 0x7fb62009e930 2023-09-30T22:27:06.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.984+0000 7fb6077fe700 1 -- 172.21.15.32:0/1880665894 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb610022020 con 0x7fb62009e930 2023-09-30T22:27:06.985 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 msgr2=0x7fb620122a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 0x7fb620122a90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb61000cc80 tx=0x7fb61000cf90 comp rx=0 tx=0).stop 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 shutdown_connections 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb620136fe0 0x7fb620123510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb62009e930 0x7fb620122a90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 --2- 172.21.15.32:0/1880665894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb62009f410 0x7fb620122fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 >> 172.21.15.32:0/1880665894 conn(0x7fb6200a4210 msgr2=0x7fb62006bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 shutdown_connections 2023-09-30T22:27:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:06.987+0000 7fb62578b700 1 -- 172.21.15.32:0/1880665894 wait complete. 2023-09-30T22:27:07.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:07.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.022+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f89201334c0 con 0x7f892009d630 2023-09-30T22:27:07.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.022+0000 7f891d7fa700 1 -- 172.21.15.32:0/184754394 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8910017070 con 0x7f892009d630 2023-09-30T22:27:07.022 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 msgr2=0x7f892011f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f892011f500 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f891000e9b0 tx=0x7f891000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 shutdown_connections 2023-09-30T22:27:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f892011f500 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 --2- 172.21.15.32:0/184754394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f892011fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 >> 172.21.15.32:0/184754394 conn(0x7f89200a3050 msgr2=0x7f89201bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 shutdown_connections 2023-09-30T22:27:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.025+0000 7f89270cf700 1 -- 172.21.15.32:0/184754394 wait complete. 2023-09-30T22:27:07.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:07.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:07.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:07.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.221+0000 7fca62c2b700 1 Processor -- start 2023-09-30T22:27:07.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca62c2b700 1 -- start start 2023-09-30T22:27:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca62c2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c06a770 0x7fca5c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca62c2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c06b080 0x7fca5c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca62c2b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c09ece0 con 0x7fca5c131b40 2023-09-30T22:27:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca62c2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c09e760 con 0x7fca5c06b080 2023-09-30T22:27:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca62c2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c09e040 con 0x7fca5c06a770 2023-09-30T22:27:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca611c8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca5c131b40 0x7fca5c138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47626/0 (socket says 172.21.15.32:47626) 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca611c8700 1 -- 172.21.15.32:0/518249793 learned_addr learned my addr 172.21.15.32:0/518249793 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca5bfff700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c06b080 0x7fca5c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.222+0000 7fca609c7700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c06a770 0x7fca5c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5bfff700 1 -- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c06a770 msgr2=0x7fca5c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5bfff700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c06a770 0x7fca5c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5b7fe700 1 -- 172.21.15.32:0/518249793 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156487081 0 0) 0x7fca5c09ece0 con 0x7fca5c131b40 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5bfff700 1 -- 172.21.15.32:0/518249793 >> v1:172.21.15.32:6789/0 conn(0x7fca5c131b40 legacy=0x7fca5c138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5bfff700 1 -- 172.21.15.32:0/518249793 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c09f630 con 0x7fca5c06b080 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca609c7700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c06a770 0x7fca5c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5bfff700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c06b080 0x7fca5c131600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fca48009870 tx=0x7fca48009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa764ca9cf7c325 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5b7fe700 1 -- 172.21.15.32:0/518249793 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48011830 con 0x7fca5c06b080 2023-09-30T22:27:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5b7fe700 1 -- 172.21.15.32:0/518249793 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca48011990 con 0x7fca5c06b080 2023-09-30T22:27:07.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.223+0000 7fca5b7fe700 1 -- 172.21.15.32:0/518249793 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48011d20 con 0x7fca5c06b080 2023-09-30T22:27:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 -- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c06b080 msgr2=0x7fca5c131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c06b080 0x7fca5c131600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fca48009870 tx=0x7fca48009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 -- 172.21.15.32:0/518249793 shutdown_connections 2023-09-30T22:27:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c06a770 0x7fca5c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 --2- 172.21.15.32:0/518249793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c06b080 0x7fca5c131600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 -- 172.21.15.32:0/518249793 >> 172.21.15.32:0/518249793 conn(0x7fca5c066f50 msgr2=0x7fca5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 -- 172.21.15.32:0/518249793 shutdown_connections 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.224+0000 7fca62c2b700 1 -- 172.21.15.32:0/518249793 wait complete. 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.225+0000 7fca62c2b700 1 Processor -- start 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.225+0000 7fca62c2b700 1 -- start start 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca62c2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca5c06a770 0x7fca5c09af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca62c2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 0x7fca5c09b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca62c2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c1c8390 0x7fca5c1c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca62c2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c131250 con 0x7fca5c06a770 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca62c2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c121ef0 con 0x7fca5c1c8390 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca62c2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c122120 con 0x7fca5c131b40 2023-09-30T22:27:07.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca5bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 0x7fca5c09b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca5bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 0x7fca5c09b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49906/0 (socket says 172.21.15.32:49906) 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca5bfff700 1 -- 172.21.15.32:0/36134372 learned_addr learned my addr 172.21.15.32:0/36134372 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca611c8700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c1c8390 0x7fca5c1c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca609c7700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca5c06a770 0x7fca5c09af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca611c8700 1 -- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 msgr2=0x7fca5c09b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca611c8700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 0x7fca5c09b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca611c8700 1 -- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca5c06a770 msgr2=0x7fca5c09af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca611c8700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca5c06a770 0x7fca5c09af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca611c8700 1 -- 172.21.15.32:0/36134372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca500094a0 con 0x7fca5c1c8390 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.226+0000 7fca5bfff700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 0x7fca5c09b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.227+0000 7fca611c8700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c1c8390 0x7fca5c1c8810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fca5400cbe0 tx=0x7fca5400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.227+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400aa40 con 0x7fca5c1c8390 2023-09-30T22:27:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.227+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5400abc0 con 0x7fca5c1c8390 2023-09-30T22:27:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.227+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca480094a0 con 0x7fca5c1c8390 2023-09-30T22:27:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.227+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540070d0 con 0x7fca5c1c8390 2023-09-30T22:27:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.227+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c09c010 con 0x7fca5c1c8390 2023-09-30T22:27:07.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.228+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca54007230 con 0x7fca5c1c8390 2023-09-30T22:27:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.228+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca5403f040 con 0x7fca5c1c8390 2023-09-30T22:27:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.228+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c121d20 con 0x7fca5c1c8390 2023-09-30T22:27:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.233+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca54018070 con 0x7fca5c1c8390 2023-09-30T22:27:07.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.262+0000 7fc0a8897700 1 Processor -- start 2023-09-30T22:27:07.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.262+0000 7fc0a8897700 1 -- start start 2023-09-30T22:27:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a8897700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a8897700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0a4138d80 0x7fc0a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a8897700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a40a2a70 con 0x7fc0a409d630 2023-09-30T22:27:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a8897700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a40a01d0 con 0x7fc0a4138d80 2023-09-30T22:27:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a8897700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a409e2b0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc0a409d630 0x7fc0a40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47640/0 (socket says 172.21.15.32:47640) 2023-09-30T22:27:07.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0a4138d80 0x7fc0a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a1d9b700 1 -- 172.21.15.32:0/3804489047 learned_addr learned my addr 172.21.15.32:0/3804489047 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a2d9d700 1 -- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 msgr2=0x7fc0a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a2d9d700 1 --2- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a159a700 1 -- 172.21.15.32:0/3804489047 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633665162 0 0) 0x7fc0a40a2a70 con 0x7fc0a409d630 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a2d9d700 1 -- 172.21.15.32:0/3804489047 >> v1:172.21.15.32:6789/0 conn(0x7fc0a409d630 legacy=0x7fc0a40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.263+0000 7fc0a2d9d700 1 -- 172.21.15.32:0/3804489047 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a409dc50 con 0x7fc0a4138d80 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a259c700 1 --2- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a2d9d700 1 --2- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a4138d80 0x7fc0a413b160 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568541ad33222400 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a159a700 1 -- 172.21.15.32:0/3804489047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00e070 con 0x7fc0a4138d80 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a159a700 1 -- 172.21.15.32:0/3804489047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c00bb20 con 0x7fc0a4138d80 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a159a700 1 -- 172.21.15.32:0/3804489047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c012430 con 0x7fc0a4138d80 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a8897700 1 -- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a4138d80 msgr2=0x7fc0a413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.264+0000 7fc0a8897700 1 --2- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a4138d80 0x7fc0a413b160 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.265+0000 7fc0a8897700 1 -- 172.21.15.32:0/3804489047 shutdown_connections 2023-09-30T22:27:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.265+0000 7fc0a8897700 1 --2- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a4138d80 0x7fc0a413b160 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.265+0000 7fc0a8897700 1 --2- 172.21.15.32:0/3804489047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.265+0000 7fc0a8897700 1 -- 172.21.15.32:0/3804489047 >> 172.21.15.32:0/3804489047 conn(0x7fc0a40a3050 msgr2=0x7fc0a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.265+0000 7fc0a8897700 1 -- 172.21.15.32:0/3804489047 shutdown_connections 2023-09-30T22:27:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.265+0000 7fc0a8897700 1 -- 172.21.15.32:0/3804489047 wait complete. 2023-09-30T22:27:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 Processor -- start 2023-09-30T22:27:07.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 -- start start 2023-09-30T22:27:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a409d630 0x7fc0a4133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a4133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a4138d80 0x7fc0a4134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a41361b0 con 0x7fc0a409d630 2023-09-30T22:27:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4136ed0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a8897700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40a4b60 con 0x7fc0a4138d80 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a4133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a4133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47300/0 (socket says 172.21.15.32:47300) 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a1d9b700 1 -- 172.21.15.32:0/4035323518 learned_addr learned my addr 172.21.15.32:0/4035323518 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a259c700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a409d630 0x7fc0a4133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.266+0000 7fc0a2d9d700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a4138d80 0x7fc0a4134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a1d9b700 1 -- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a4138d80 msgr2=0x7fc0a4134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a1d9b700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a4138d80 0x7fc0a4134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a1d9b700 1 -- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a409d630 msgr2=0x7fc0a4133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a1d9b700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a409d630 0x7fc0a4133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a1d9b700 1 -- 172.21.15.32:0/4035323518 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940094a0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a2d9d700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a4138d80 0x7fc0a4134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a259c700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a409d630 0x7fc0a4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a1d9b700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a4133bd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc098001eb0 tx=0x7fc09800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098004c80 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc098004de0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098007310 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c009a70 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.267+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a4136560 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.268+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc098007470 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.268+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc09803f2b0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.268+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a40747f0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.273+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0980123b0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.442+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fca5c074ea0 con 0x7fca5c1c8390 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.443+0000 7fca597fa700 1 -- 172.21.15.32:0/36134372 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca540074e0 con 0x7fca5c1c8390 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c1c8390 msgr2=0x7fca5c1c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c1c8390 0x7fca5c1c8810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fca5400cbe0 tx=0x7fca5400cef0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 shutdown_connections 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca5c1c8390 0x7fca5c1c8810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca5c06a770 0x7fca5c09af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 --2- 172.21.15.32:0/36134372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca5c131b40 0x7fca5c09b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 >> 172.21.15.32:0/36134372 conn(0x7fca5c066f50 msgr2=0x7fca5c0700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 shutdown_connections 2023-09-30T22:27:07.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.445+0000 7fca62c2b700 1 -- 172.21.15.32:0/36134372 wait complete. 2023-09-30T22:27:07.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:07.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.480+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc0a41379a0 con 0x7fc0a40a0a30 2023-09-30T22:27:07.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.481+0000 7fc092ffd700 1 -- 172.21.15.32:0/4035323518 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc098011360 con 0x7fc0a40a0a30 2023-09-30T22:27:07.481 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:07.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 msgr2=0x7fc0a4133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a4133bd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc098001eb0 tx=0x7fc09800c800 comp rx=0 tx=0).stop 2023-09-30T22:27:07.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 shutdown_connections 2023-09-30T22:27:07.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a4138d80 0x7fc0a4134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a409d630 0x7fc0a4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 --2- 172.21.15.32:0/4035323518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a40a0a30 0x7fc0a4133bd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 >> 172.21.15.32:0/4035323518 conn(0x7fc0a40a3050 msgr2=0x7fc0a413af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 shutdown_connections 2023-09-30T22:27:07.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:07.483+0000 7fc0a8897700 1 -- 172.21.15.32:0/4035323518 wait complete. 2023-09-30T22:27:07.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:08.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:08.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.149+0000 7f8a1f5bc700 1 Processor -- start 2023-09-30T22:27:08.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.149+0000 7f8a1f5bc700 1 -- start start 2023-09-30T22:27:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1f5bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1f5bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a18138d80 0x7f8a1813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1f5bc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a180a2a70 con 0x7f8a1809d630 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1f5bc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a180a01d0 con 0x7f8a18138d80 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1f5bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1809e2b0 con 0x7f8a180a0a30 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1db59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a18138d80 0x7f8a1813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1cb57700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a1809d630 0x7f8a180a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47642/0 (socket says 172.21.15.32:47642) 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1cb57700 1 -- 172.21.15.32:0/3482979719 learned_addr learned my addr 172.21.15.32:0/3482979719 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a180a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49928/0 (socket says 172.21.15.32:49928) 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 -- 172.21.15.32:0/3482979719 >> v2:172.21.15.32:3300/0 conn(0x7f8a18138d80 msgr2=0x7f8a1813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 --2- 172.21.15.32:0/3482979719 >> v2:172.21.15.32:3300/0 conn(0x7f8a18138d80 0x7f8a1813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 -- 172.21.15.32:0/3482979719 >> v1:172.21.15.32:6789/0 conn(0x7f8a1809d630 legacy=0x7f8a180a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 -- 172.21.15.32:0/3482979719 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1809dc50 con 0x7f8a180a0a30 2023-09-30T22:27:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1db59700 1 --2- 172.21.15.32:0/3482979719 >> v2:172.21.15.32:3300/0 conn(0x7f8a18138d80 0x7f8a1813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.150+0000 7f8a1d358700 1 --2- 172.21.15.32:0/3482979719 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a180a0e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009840 tx=0x7f8a14009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47645252f95fa3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a0ffff700 1 -- 172.21.15.32:0/3482979719 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14011830 con 0x7f8a180a0a30 2023-09-30T22:27:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a0ffff700 1 -- 172.21.15.32:0/3482979719 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14011990 con 0x7f8a180a0a30 2023-09-30T22:27:08.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a0ffff700 1 -- 172.21.15.32:0/3482979719 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14011d20 con 0x7f8a180a0a30 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/3482979719 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 msgr2=0x7f8a180a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/3482979719 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a180a0e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009840 tx=0x7f8a14009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/3482979719 shutdown_connections 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/3482979719 >> v2:172.21.15.32:3300/0 conn(0x7f8a18138d80 0x7f8a1813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/3482979719 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a180a0e00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/3482979719 >> 172.21.15.32:0/3482979719 conn(0x7f8a180a3050 msgr2=0x7f8a180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/3482979719 shutdown_connections 2023-09-30T22:27:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.151+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/3482979719 wait complete. 2023-09-30T22:27:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.153+0000 7f8a1f5bc700 1 Processor -- start 2023-09-30T22:27:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.153+0000 7f8a1f5bc700 1 -- start start 2023-09-30T22:27:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1f5bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1809d630 0x7f8a181208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1f5bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a18120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1f5bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 0x7f8a18121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1f5bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18122470 con 0x7f8a18138d80 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1f5bc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181237c0 con 0x7f8a1809d630 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1f5bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18121d50 con 0x7f8a180a0a30 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 0x7f8a18121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 0x7f8a18121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32842/0 (socket says 172.21.15.32:32842) 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 -- 172.21.15.32:0/1613490373 learned_addr learned my addr 172.21.15.32:0/1613490373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1cb57700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a18120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1d358700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1809d630 0x7f8a181208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 -- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 msgr2=0x7f8a18120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a18120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 -- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1809d630 msgr2=0x7f8a181208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1809d630 0x7f8a181208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 -- 172.21.15.32:0/1613490373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10009a70 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a1db59700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 0x7f8a18121370 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000efc0 tx=0x7f8a1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.154+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000cc90 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.155+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1000f040 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.155+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a140094a0 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.155+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a100146c0 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.155+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181243d0 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.156+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a10014820 con 0x7f8a18138d80 2023-09-30T22:27:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.156+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a100401a0 con 0x7f8a18138d80 2023-09-30T22:27:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.156+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a181243d0 con 0x7f8a18138d80 2023-09-30T22:27:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.161+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a181243d0 con 0x7f8a18138d80 2023-09-30T22:27:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.229+0000 7f5de6853700 1 Processor -- start 2023-09-30T22:27:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.229+0000 7f5de6853700 1 -- start start 2023-09-30T22:27:08.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.229+0000 7f5de6853700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 0x7f5de0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.229+0000 7f5de6853700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5de0136fe0 0x7f5de01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.229+0000 7f5de6853700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de0070940 con 0x7f5de009e930 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de6853700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de006fd60 con 0x7f5de0136fe0 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de6853700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de006fa80 con 0x7f5de009f410 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 0x7f5de0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5de0136fe0 0x7f5de01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 0x7f5de0136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47322/0 (socket says 172.21.15.32:47322) 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5de009e930 0x7f5de009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47658/0 (socket says 172.21.15.32:47658) 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddf7fe700 1 -- 172.21.15.32:0/57007349 learned_addr learned my addr 172.21.15.32:0/57007349 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5de0136fe0 0x7f5de01393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32860/0 (socket says 172.21.15.32:32860) 2023-09-30T22:27:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 -- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 msgr2=0x7f5de0136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 --2- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 0x7f5de0136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 -- 172.21.15.32:0/57007349 >> v1:172.21.15.32:6789/0 conn(0x7f5de009e930 legacy=0x7f5de009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 -- 172.21.15.32:0/57007349 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de009fb60 con 0x7f5de0136fe0 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddf7fe700 1 --2- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 0x7f5de0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5de4df0700 1 --2- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de01393c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009d90 tx=0x7f5dd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f776415b0dc6bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddeffd700 1 -- 172.21.15.32:0/57007349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800e070 con 0x7f5de0136fe0 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddeffd700 1 -- 172.21.15.32:0/57007349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800bb20 con 0x7f5de0136fe0 2023-09-30T22:27:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.230+0000 7f5ddeffd700 1 -- 172.21.15.32:0/57007349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012430 con 0x7f5de0136fe0 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 -- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 msgr2=0x7f5de01393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 --2- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de01393c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009d90 tx=0x7f5dd800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 -- 172.21.15.32:0/57007349 shutdown_connections 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 --2- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de01393c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 --2- 172.21.15.32:0/57007349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009f410 0x7f5de0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 -- 172.21.15.32:0/57007349 >> 172.21.15.32:0/57007349 conn(0x7f5de00a4210 msgr2=0x7f5de00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 -- 172.21.15.32:0/57007349 shutdown_connections 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.231+0000 7f5de6853700 1 -- 172.21.15.32:0/57007349 wait complete. 2023-09-30T22:27:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.232+0000 7f5de6853700 1 Processor -- start 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 -- start start 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009e930 0x7f5de01c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 0x7f5de01c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0124000 con 0x7f5de0136fe0 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01bc0c0 con 0x7f5de009e930 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de6853700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0134370 con 0x7f5de009f410 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5ddf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 0x7f5de01c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de4df0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de4df0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32872/0 (socket says 172.21.15.32:32872) 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5ddf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 0x7f5de01c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49938/0 (socket says 172.21.15.32:49938) 2023-09-30T22:27:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5de4df0700 1 -- 172.21.15.32:0/94524192 learned_addr learned my addr 172.21.15.32:0/94524192 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.233+0000 7f5ddffff700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009e930 0x7f5de01c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddf7fe700 1 -- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009e930 msgr2=0x7f5de01c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddf7fe700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009e930 0x7f5de01c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddf7fe700 1 -- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 msgr2=0x7f5de006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddf7fe700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddf7fe700 1 -- 172.21.15.32:0/94524192 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4009550 con 0x7f5de009f410 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5de4df0700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddffff700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009e930 0x7f5de01c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddf7fe700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 0x7f5de01c8a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e9a0 tx=0x7f5dd400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004030 con 0x7f5de009f410 2023-09-30T22:27:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd4004190 con 0x7f5de009f410 2023-09-30T22:27:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd8009a70 con 0x7f5de009f410 2023-09-30T22:27:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004520 con 0x7f5de009f410 2023-09-30T22:27:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.234+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de0122bc0 con 0x7f5de009f410 2023-09-30T22:27:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.235+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5dd40036a0 con 0x7f5de009f410 2023-09-30T22:27:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.235+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5dd403f8f0 con 0x7f5de009f410 2023-09-30T22:27:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.235+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de0124140 con 0x7f5de009f410 2023-09-30T22:27:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.240+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5de0124140 con 0x7f5de009f410 2023-09-30T22:27:08.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.377+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a181243d0 con 0x7f8a18138d80 2023-09-30T22:27:08.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.377+0000 7f8a0dffb700 1 -- 172.21.15.32:0/1613490373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a181243d0 con 0x7f8a18138d80 2023-09-30T22:27:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.379+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 msgr2=0x7f8a18121370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.379+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 0x7f8a18121370 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000efc0 tx=0x7f8a1000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 shutdown_connections 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a18138d80 0x7f8a18121370 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1809d630 0x7f8a181208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 --2- 172.21.15.32:0/1613490373 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a180a0a30 0x7f8a18120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 >> 172.21.15.32:0/1613490373 conn(0x7f8a180a3050 msgr2=0x7f8a18139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 shutdown_connections 2023-09-30T22:27:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.380+0000 7f8a1f5bc700 1 -- 172.21.15.32:0/1613490373 wait complete. 2023-09-30T22:27:08.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.446+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5de0124140 con 0x7f5de009f410 2023-09-30T22:27:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.446+0000 7f5ddcff9700 1 -- 172.21.15.32:0/94524192 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5de0124140 con 0x7f5de009f410 2023-09-30T22:27:08.447 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.448+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 msgr2=0x7f5de01c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.448+0000 7f5de6853700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 0x7f5de01c8a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e9a0 tx=0x7f5dd400ecb0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 shutdown_connections 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5de0136fe0 0x7f5de006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5de009e930 0x7f5de01c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 --2- 172.21.15.32:0/94524192 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5de009f410 0x7f5de01c8a80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 >> 172.21.15.32:0/94524192 conn(0x7f5de00a4210 msgr2=0x7f5de00a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 shutdown_connections 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.449+0000 7f5de6853700 1 -- 172.21.15.32:0/94524192 wait complete. 2023-09-30T22:27:08.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:08.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:08.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.597+0000 7fedec10d700 1 Processor -- start 2023-09-30T22:27:08.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.597+0000 7fedec10d700 1 -- start start 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedec10d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede40a0a30 0x7fede40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedec10d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 0x7fede413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedec10d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40a2a70 con 0x7fede409d630 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedec10d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede40a01d0 con 0x7fede4138d80 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedec10d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fede409e2b0 con 0x7fede40a0a30 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fede9ea9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede40a0a30 0x7fede40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedea6aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 0x7fede413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fede9ea9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede40a0a30 0x7fede40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49940/0 (socket says 172.21.15.32:49940) 2023-09-30T22:27:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fede96a8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fede409d630 0x7fede40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47674/0 (socket says 172.21.15.32:47674) 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fede9ea9700 1 -- 172.21.15.32:0/2123654338 learned_addr learned my addr 172.21.15.32:0/2123654338 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.598+0000 7fedea6aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 0x7fede413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47338/0 (socket says 172.21.15.32:47338) 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fedea6aa700 1 -- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede40a0a30 msgr2=0x7fede40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fedea6aa700 1 --2- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede40a0a30 0x7fede40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fede8ea7700 1 -- 172.21.15.32:0/2123654338 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115622381 0 0) 0x7fede40a2a70 con 0x7fede409d630 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fedea6aa700 1 -- 172.21.15.32:0/2123654338 >> v1:172.21.15.32:6789/0 conn(0x7fede409d630 legacy=0x7fede40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fedea6aa700 1 -- 172.21.15.32:0/2123654338 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede409dc50 con 0x7fede4138d80 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fedea6aa700 1 --2- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 0x7fede413b160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feddc009dc0 tx=0x7feddc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2224a734a0607ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fede8ea7700 1 -- 172.21.15.32:0/2123654338 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc00e070 con 0x7fede4138d80 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fede8ea7700 1 -- 172.21.15.32:0/2123654338 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc00bb20 con 0x7fede4138d80 2023-09-30T22:27:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.599+0000 7fede8ea7700 1 -- 172.21.15.32:0/2123654338 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc012430 con 0x7fede4138d80 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 -- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 msgr2=0x7fede413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 --2- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 0x7fede413b160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feddc009dc0 tx=0x7feddc00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 -- 172.21.15.32:0/2123654338 shutdown_connections 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 --2- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede4138d80 0x7fede413b160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 --2- 172.21.15.32:0/2123654338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede40a0a30 0x7fede40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 -- 172.21.15.32:0/2123654338 >> 172.21.15.32:0/2123654338 conn(0x7fede40a3050 msgr2=0x7fede40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 -- 172.21.15.32:0/2123654338 shutdown_connections 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.600+0000 7fedec10d700 1 -- 172.21.15.32:0/2123654338 wait complete. 2023-09-30T22:27:08.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 Processor -- start 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 -- start start 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fede409d630 0x7fede41336c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede40a0a30 0x7fede4133c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 0x7fede4134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4136e70 con 0x7fede409d630 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede4135aa0 con 0x7fede40a0a30 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedec10d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fede4137150 con 0x7fede4138d80 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedea6aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 0x7fede4134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fede9ea9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fede409d630 0x7fede41336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedea6aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 0x7fede4134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49944/0 (socket says 172.21.15.32:49944) 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fedea6aa700 1 -- 172.21.15.32:0/2740892116 learned_addr learned my addr 172.21.15.32:0/2740892116 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:08.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.601+0000 7fede96a8700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede40a0a30 0x7fede4133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedea6aa700 1 -- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede40a0a30 msgr2=0x7fede4133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedea6aa700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede40a0a30 0x7fede4133c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedea6aa700 1 -- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fede409d630 msgr2=0x7fede41336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedea6aa700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fede409d630 0x7fede41336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedea6aa700 1 -- 172.21.15.32:0/2740892116 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc009a70 con 0x7fede4138d80 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fede9ea9700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fede409d630 0x7fede41336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedea6aa700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 0x7fede4134140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feddc00b370 tx=0x7feddc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc012430 con 0x7fede4138d80 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc020070 con 0x7fede4138d80 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede40a0410 con 0x7fede4138d80 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc004280 con 0x7fede4138d80 2023-09-30T22:27:08.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.602+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede4136490 con 0x7fede4138d80 2023-09-30T22:27:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.603+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feddc0043e0 con 0x7fede4138d80 2023-09-30T22:27:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.603+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feddc022070 con 0x7fede4138d80 2023-09-30T22:27:08.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.604+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede40747f0 con 0x7fede4138d80 2023-09-30T22:27:08.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.608+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feddc01b400 con 0x7fede4138d80 2023-09-30T22:27:08.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.641+0000 7fba616ea700 1 Processor -- start 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.641+0000 7fba616ea700 1 -- start start 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba616ea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba5c0984f0 0x7fba5c0988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba616ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09d6f0 0x7fba5c09db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba616ea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5c099620 con 0x7fba5c09e040 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba616ea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c09a9e0 con 0x7fba5c0984f0 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba616ea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c09bce0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba5c0984f0 0x7fba5c0988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba5c09e040 0x7fba5c1385c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47678/0 (socket says 172.21.15.32:47678) 2023-09-30T22:27:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5b7fe700 1 -- 172.21.15.32:0/1070273488 learned_addr learned my addr 172.21.15.32:0/1070273488 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5a7fc700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09d6f0 0x7fba5c09db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba59ffb700 1 -- 172.21.15.32:0/1070273488 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198366358 0 0) 0x7fba5c099620 con 0x7fba5c09e040 2023-09-30T22:27:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5affd700 1 -- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09d6f0 msgr2=0x7fba5c09db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5affd700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09d6f0 0x7fba5c09db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5affd700 1 -- 172.21.15.32:0/1070273488 >> v1:172.21.15.32:6789/0 conn(0x7fba5c09e040 legacy=0x7fba5c1385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5affd700 1 -- 172.21.15.32:0/1070273488 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c09aca0 con 0x7fba5c0984f0 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5a7fc700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09d6f0 0x7fba5c09db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.642+0000 7fba5affd700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c0988c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009d90 tx=0x7fba4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f17a213d4f3c0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.643+0000 7fba59ffb700 1 -- 172.21.15.32:0/1070273488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00e070 con 0x7fba5c0984f0 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.643+0000 7fba59ffb700 1 -- 172.21.15.32:0/1070273488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00bb20 con 0x7fba5c0984f0 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.643+0000 7fba59ffb700 1 -- 172.21.15.32:0/1070273488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c012430 con 0x7fba5c0984f0 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.643+0000 7fba616ea700 1 -- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 msgr2=0x7fba5c0988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.643+0000 7fba616ea700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c0988c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009d90 tx=0x7fba4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.644+0000 7fba616ea700 1 -- 172.21.15.32:0/1070273488 shutdown_connections 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.644+0000 7fba616ea700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c0988c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.644+0000 7fba616ea700 1 --2- 172.21.15.32:0/1070273488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09d6f0 0x7fba5c09db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.644+0000 7fba616ea700 1 -- 172.21.15.32:0/1070273488 >> 172.21.15.32:0/1070273488 conn(0x7fba5c09f470 msgr2=0x7fba5c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.644+0000 7fba616ea700 1 -- 172.21.15.32:0/1070273488 shutdown_connections 2023-09-30T22:27:08.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.644+0000 7fba616ea700 1 -- 172.21.15.32:0/1070273488 wait complete. 2023-09-30T22:27:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 Processor -- start 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 -- start start 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c06d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 0x7fba5c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 0x7fba5c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c06ea80 con 0x7fba5c0984f0 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c06f370 con 0x7fba5c09e040 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.645+0000 7fba616ea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c12e4b0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 0x7fba5c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 0x7fba5c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 0x7fba5c06db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49950/0 (socket says 172.21.15.32:49950) 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 0x7fba5c06e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47360/0 (socket says 172.21.15.32:47360) 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 -- 172.21.15.32:0/2107619292 learned_addr learned my addr 172.21.15.32:0/2107619292 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5affd700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 -- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 msgr2=0x7fba5c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 0x7fba5c06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 -- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 msgr2=0x7fba5c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c06d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 -- 172.21.15.32:0/2107619292 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba440094a0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5b7fe700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 0x7fba5c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5affd700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba5a7fc700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 0x7fba5c06db60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fba4400ebe0 tx=0x7fba4400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400c850 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4400c9b0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400cd40 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.646+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4c009a70 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.647+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c09a2b0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.647+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba44013470 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.647+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba4403f7e0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.648+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c130930 con 0x7fba5c09d6f0 2023-09-30T22:27:08.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.653+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba44013720 con 0x7fba5c09d6f0 2023-09-30T22:27:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.816+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fede4072980 con 0x7fede4138d80 2023-09-30T22:27:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.817+0000 7fedda7fc700 1 -- 172.21.15.32:0/2740892116 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feddc02c020 con 0x7fede4138d80 2023-09-30T22:27:08.817 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 msgr2=0x7fede4134140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 0x7fede4134140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feddc00b370 tx=0x7feddc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 shutdown_connections 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fede4138d80 0x7fede4134140 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fede409d630 0x7fede41336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 --2- 172.21.15.32:0/2740892116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fede40a0a30 0x7fede4133c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 >> 172.21.15.32:0/2740892116 conn(0x7fede40a3050 msgr2=0x7fede409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 shutdown_connections 2023-09-30T22:27:08.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.819+0000 7fedec10d700 1 -- 172.21.15.32:0/2740892116 wait complete. 2023-09-30T22:27:08.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:08.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.857+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fba5c0747f0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.857+0000 7fba537fe700 1 -- 172.21.15.32:0/2107619292 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba4401b9c0 con 0x7fba5c09d6f0 2023-09-30T22:27:08.857 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 msgr2=0x7fba5c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 0x7fba5c06db60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fba4400ebe0 tx=0x7fba4400eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 shutdown_connections 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba5c09e040 0x7fba5c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba5c0984f0 0x7fba5c06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 --2- 172.21.15.32:0/2107619292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba5c09d6f0 0x7fba5c06db60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 >> 172.21.15.32:0/2107619292 conn(0x7fba5c09f470 msgr2=0x7fba5c1b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 shutdown_connections 2023-09-30T22:27:08.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:08.859+0000 7fba616ea700 1 -- 172.21.15.32:0/2107619292 wait complete. 2023-09-30T22:27:08.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:09.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:09.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.544+0000 7f847836a700 1 Processor -- start 2023-09-30T22:27:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.544+0000 7f847836a700 1 -- start start 2023-09-30T22:27:09.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f847836a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84700a1510 0x7f8470138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f847836a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 0x7f847013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f847836a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847009edd0 con 0x7f84700a0a30 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f847836a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700a18e0 con 0x7f84700a1510 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f847836a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f847009e1f0 con 0x7f8470139250 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475905700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84700a1510 0x7f8470138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8476907700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 0x7f847013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8476106700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84700a0a30 0x7f84700a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47688/0 (socket says 172.21.15.32:47688) 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8476907700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 0x7f847013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49956/0 (socket says 172.21.15.32:49956) 2023-09-30T22:27:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8476106700 1 -- 172.21.15.32:0/3281898350 learned_addr learned my addr 172.21.15.32:0/3281898350 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475905700 1 -- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 msgr2=0x7f847013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475905700 1 --2- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 0x7f847013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475905700 1 -- 172.21.15.32:0/3281898350 >> v1:172.21.15.32:6789/0 conn(0x7f84700a0a30 legacy=0x7f84700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475905700 1 -- 172.21.15.32:0/3281898350 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847009dc50 con 0x7f84700a1510 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475104700 1 -- 172.21.15.32:0/3281898350 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599621547 0 0) 0x7f847009edd0 con 0x7f84700a0a30 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8476907700 1 --2- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 0x7f847013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.545+0000 7f8475905700 1 --2- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84700a1510 0x7f8470138d10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8460009840 tx=0x7f8460009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12131ed1824c7140 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f8475104700 1 -- 172.21.15.32:0/3281898350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460011830 con 0x7f84700a1510 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f8475104700 1 -- 172.21.15.32:0/3281898350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460011990 con 0x7f84700a1510 2023-09-30T22:27:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f8475104700 1 -- 172.21.15.32:0/3281898350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460011d20 con 0x7f84700a1510 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 -- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84700a1510 msgr2=0x7f8470138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 --2- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84700a1510 0x7f8470138d10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8460009840 tx=0x7f8460009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 -- 172.21.15.32:0/3281898350 shutdown_connections 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 --2- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8470139250 0x7f847013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 --2- 172.21.15.32:0/3281898350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84700a1510 0x7f8470138d10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 -- 172.21.15.32:0/3281898350 >> 172.21.15.32:0/3281898350 conn(0x7f84700a6550 msgr2=0x7f84700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 -- 172.21.15.32:0/3281898350 shutdown_connections 2023-09-30T22:27:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.546+0000 7f847836a700 1 -- 172.21.15.32:0/3281898350 wait complete. 2023-09-30T22:27:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.547+0000 7f847836a700 1 Processor -- start 2023-09-30T22:27:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.547+0000 7f847836a700 1 -- start start 2023-09-30T22:27:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f847836a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84700a0a30 0x7f84701228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f847836a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 0x7f8470122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f847836a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8470139250 0x7f8470123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f847836a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84701389e0 con 0x7f8470139250 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f847836a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700a0220 con 0x7f84700a0a30 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f847836a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8470126060 con 0x7f84700a1510 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 0x7f8470122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8476907700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8470139250 0x7f8470123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 0x7f8470122e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49960/0 (socket says 172.21.15.32:49960) 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 -- 172.21.15.32:0/3927786449 learned_addr learned my addr 172.21.15.32:0/3927786449 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8476106700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84700a0a30 0x7f84701228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 -- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84700a0a30 msgr2=0x7f84701228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84700a0a30 0x7f84701228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 -- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8470139250 msgr2=0x7f8470123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8470139250 0x7f8470123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8475905700 1 -- 172.21.15.32:0/3927786449 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009a70 con 0x7f84700a1510 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.548+0000 7f8476907700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8470139250 0x7f8470123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.549+0000 7f8475905700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 0x7f8470122e00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8460009fd0 tx=0x7f8460004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.549+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84600045d0 con 0x7f84700a1510 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.549+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846001f070 con 0x7f84700a1510 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.549+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84600094a0 con 0x7f84700a1510 2023-09-30T22:27:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.549+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84600038a0 con 0x7f84700a1510 2023-09-30T22:27:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.549+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8470124bb0 con 0x7f84700a1510 2023-09-30T22:27:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.550+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8460003a00 con 0x7f84700a1510 2023-09-30T22:27:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.550+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8460021070 con 0x7f84700a1510 2023-09-30T22:27:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.550+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470074ea0 con 0x7f84700a1510 2023-09-30T22:27:09.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.559+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f846002b080 con 0x7f84700a1510 2023-09-30T22:27:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.609+0000 7f6efd6cc700 1 Processor -- start 2023-09-30T22:27:09.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.609+0000 7f6efd6cc700 1 -- start start 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6efd6cc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef806b250 0x7f6ef8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6efd6cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef8136d80 0x7f6ef8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6efd6cc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef809e810 con 0x7f6ef806a770 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6efd6cc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef809e070 con 0x7f6ef806b250 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6efd6cc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef809ead0 con 0x7f6ef8136d80 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef806b250 0x7f6ef8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef8136d80 0x7f6ef8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ef806a770 0x7f6ef806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47704/0 (socket says 172.21.15.32:47704) 2023-09-30T22:27:09.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef6ffd700 1 -- 172.21.15.32:0/3804088419 learned_addr learned my addr 172.21.15.32:0/3804088419 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef5ffb700 1 -- 172.21.15.32:0/3804088419 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192532836 0 0) 0x7f6ef809e810 con 0x7f6ef806a770 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef77fe700 1 -- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef806b250 msgr2=0x7f6ef8136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef77fe700 1 --2- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef806b250 0x7f6ef8136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef77fe700 1 -- 172.21.15.32:0/3804088419 >> v1:172.21.15.32:6789/0 conn(0x7f6ef806a770 legacy=0x7f6ef806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef77fe700 1 -- 172.21.15.32:0/3804088419 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef809daa0 con 0x7f6ef8136d80 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.610+0000 7f6ef67fc700 1 --2- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef806b250 0x7f6ef8136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.611+0000 7f6ef77fe700 1 --2- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef8136d80 0x7f6ef8139160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009df0 tx=0x7f6ef000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7984869df3e289 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.611+0000 7f6ef5ffb700 1 -- 172.21.15.32:0/3804088419 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000e070 con 0x7f6ef8136d80 2023-09-30T22:27:09.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.611+0000 7f6ef5ffb700 1 -- 172.21.15.32:0/3804088419 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef000bae0 con 0x7f6ef8136d80 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.611+0000 7f6ef5ffb700 1 -- 172.21.15.32:0/3804088419 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000be90 con 0x7f6ef8136d80 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 -- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef8136d80 msgr2=0x7f6ef8139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef8136d80 0x7f6ef8139160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009df0 tx=0x7f6ef000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 -- 172.21.15.32:0/3804088419 shutdown_connections 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef8136d80 0x7f6ef8139160 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/3804088419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef806b250 0x7f6ef8136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 -- 172.21.15.32:0/3804088419 >> 172.21.15.32:0/3804088419 conn(0x7f6ef8066f50 msgr2=0x7f6ef809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 -- 172.21.15.32:0/3804088419 shutdown_connections 2023-09-30T22:27:09.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.612+0000 7f6efd6cc700 1 -- 172.21.15.32:0/3804088419 wait complete. 2023-09-30T22:27:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.613+0000 7f6efd6cc700 1 Processor -- start 2023-09-30T22:27:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 -- start start 2023-09-30T22:27:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef806a770 0x7f6ef809ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 0x7f6ef809b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef8136d80 0x7f6ef81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81263a0 con 0x7f6ef806b250 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8126580 con 0x7f6ef8136d80 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6efd6cc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81363e0 con 0x7f6ef806a770 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6ef67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 0x7f6ef809b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6ef67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 0x7f6ef809b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32942/0 (socket says 172.21.15.32:32942) 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6ef77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef8136d80 0x7f6ef81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.614+0000 7f6ef67fc700 1 -- 172.21.15.32:0/2545264232 learned_addr learned my addr 172.21.15.32:0/2545264232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:09.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef6ffd700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef806a770 0x7f6ef809ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef77fe700 1 -- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef806a770 msgr2=0x7f6ef809ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef77fe700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef806a770 0x7f6ef809ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef77fe700 1 -- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 msgr2=0x7f6ef809b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef77fe700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 0x7f6ef809b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef77fe700 1 -- 172.21.15.32:0/2545264232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0094a0 con 0x7f6ef8136d80 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef67fc700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 0x7f6ef809b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6ef77fe700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef8136d80 0x7f6ef81c8090 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0016000 tx=0x7f6ef0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00045d0 con 0x7f6ef8136d80 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0020070 con 0x7f6ef8136d80 2023-09-30T22:27:09.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef0009a70 con 0x7f6ef8136d80 2023-09-30T22:27:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0027480 con 0x7f6ef8136d80 2023-09-30T22:27:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.615+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef8071270 con 0x7f6ef8136d80 2023-09-30T22:27:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.616+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ef00275e0 con 0x7f6ef8136d80 2023-09-30T22:27:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.616+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef80747f0 con 0x7f6ef8136d80 2023-09-30T22:27:09.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.617+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6ef0022070 con 0x7f6ef8136d80 2023-09-30T22:27:09.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.621+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ef002f750 con 0x7f6ef8136d80 2023-09-30T22:27:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.769+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f84700a28e0 con 0x7f84700a1510 2023-09-30T22:27:09.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.769+0000 7f8466ffd700 1 -- 172.21.15.32:0/3927786449 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8460015020 con 0x7f84700a1510 2023-09-30T22:27:09.769 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.771+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 msgr2=0x7f8470122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.771+0000 7f847836a700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 0x7f8470122e00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8460009fd0 tx=0x7f8460004750 comp rx=0 tx=0).stop 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.771+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 shutdown_connections 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.771+0000 7f847836a700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8470139250 0x7f8470123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.771+0000 7f847836a700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84700a0a30 0x7f84701228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.771+0000 7f847836a700 1 --2- 172.21.15.32:0/3927786449 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84700a1510 0x7f8470122e00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.772+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 >> 172.21.15.32:0/3927786449 conn(0x7f84700a6550 msgr2=0x7f84700a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.772+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 shutdown_connections 2023-09-30T22:27:09.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.772+0000 7f847836a700 1 -- 172.21.15.32:0/3927786449 wait complete. 2023-09-30T22:27:09.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:09.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.828+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6ef806c360 con 0x7f6ef8136d80 2023-09-30T22:27:09.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.829+0000 7f6edf7fe700 1 -- 172.21.15.32:0/2545264232 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ef002f390 con 0x7f6ef8136d80 2023-09-30T22:27:09.829 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:09.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef8136d80 msgr2=0x7f6ef81c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef8136d80 0x7f6ef81c8090 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0016000 tx=0x7f6ef0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 shutdown_connections 2023-09-30T22:27:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ef8136d80 0x7f6ef81c8090 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ef806a770 0x7f6ef809ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 --2- 172.21.15.32:0/2545264232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ef806b250 0x7f6ef809b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 >> 172.21.15.32:0/2545264232 conn(0x7f6ef8066f50 msgr2=0x7f6ef80a2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 shutdown_connections 2023-09-30T22:27:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.831+0000 7f6efd6cc700 1 -- 172.21.15.32:0/2545264232 wait complete. 2023-09-30T22:27:09.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:09.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:09.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:09.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.973+0000 7fb70bacd700 1 Processor -- start 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.973+0000 7fb70bacd700 1 -- start start 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70bacd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7040a0a30 0x7fb7040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70bacd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7040a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70bacd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 0x7fb704133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70bacd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7040a2a70 con 0x7fb7040a0a30 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70bacd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040a01d0 con 0x7fb704133930 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70bacd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb70409e2b0 con 0x7fb70409d630 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7040a0a30 0x7fb7040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70a06a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 0x7fb704133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709068700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7040a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7040a0a30 0x7fb7040a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32958/0 (socket says 172.21.15.32:32958) 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 -- 172.21.15.32:0/1527384563 learned_addr learned my addr 172.21.15.32:0/1527384563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70a06a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 0x7fb704133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47396/0 (socket says 172.21.15.32:47396) 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709068700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7040a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49996/0 (socket says 172.21.15.32:49996) 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 -- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 msgr2=0x7fb7040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7040a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 -- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 msgr2=0x7fb704133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 0x7fb704133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709869700 1 -- 172.21.15.32:0/1527384563 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70409dc50 con 0x7fb7040a0a30 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb70a06a700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 0x7fb704133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.974+0000 7fb709068700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7040a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb709869700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7040a0a30 0x7fb7040a0e00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009e50 tx=0x7fb6fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fd3b05da6a9f29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb708867700 1 -- 172.21.15.32:0/1527384563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00e070 con 0x7fb7040a0a30 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb708867700 1 -- 172.21.15.32:0/1527384563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00b910 con 0x7fb7040a0a30 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb708867700 1 -- 172.21.15.32:0/1527384563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00bcc0 con 0x7fb7040a0a30 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 -- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7040a0a30 msgr2=0x7fb7040a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7040a0a30 0x7fb7040a0e00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009e50 tx=0x7fb6fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 -- 172.21.15.32:0/1527384563 shutdown_connections 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb704133930 0x7fb704133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7040a0a30 0x7fb7040a0e00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 --2- 172.21.15.32:0/1527384563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7040a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.975+0000 7fb70bacd700 1 -- 172.21.15.32:0/1527384563 >> 172.21.15.32:0/1527384563 conn(0x7fb7040a3050 msgr2=0x7fb7040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.976+0000 7fb70bacd700 1 -- 172.21.15.32:0/1527384563 shutdown_connections 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.976+0000 7fb70bacd700 1 -- 172.21.15.32:0/1527384563 wait complete. 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 Processor -- start 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 -- start start 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7041c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7040a0a30 0x7fb7041c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb704133930 0x7fb7041c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041332b0 con 0x7fb704133930 2023-09-30T22:27:09.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041206b0 con 0x7fb7040a0a30 2023-09-30T22:27:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70bacd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040a52d0 con 0x7fb70409d630 2023-09-30T22:27:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb709869700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7041c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb709869700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7041c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50004/0 (socket says 172.21.15.32:50004) 2023-09-30T22:27:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb709869700 1 -- 172.21.15.32:0/612638021 learned_addr learned my addr 172.21.15.32:0/612638021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb70a06a700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb704133930 0x7fb7041c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.977+0000 7fb709068700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7040a0a30 0x7fb7041c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709869700 1 -- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7040a0a30 msgr2=0x7fb7041c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709869700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7040a0a30 0x7fb7041c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709869700 1 -- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb704133930 msgr2=0x7fb7041c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709869700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb704133930 0x7fb7041c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709869700 1 -- 172.21.15.32:0/612638021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f80094a0 con 0x7fb70409d630 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709068700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7040a0a30 0x7fb7041c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb70a06a700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb704133930 0x7fb7041c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb709869700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7041c8090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb7040a2bb0 tx=0x7fb6fc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ce80 con 0x7fb70409d630 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc004250 con 0x7fb70409d630 2023-09-30T22:27:09.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f40094a0 con 0x7fb70409d630 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc004600 con 0x7fb70409d630 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.978+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc009b30 con 0x7fb70409d630 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.979+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6fc004760 con 0x7fb70409d630 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.979+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb704121fc0 con 0x7fb70409d630 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.979+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb6fc040d10 con 0x7fb70409d630 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:09.984+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb704121fc0 con 0x7fb70409d630 2023-09-30T22:27:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.024+0000 7f45bc953700 1 Processor -- start 2023-09-30T22:27:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.024+0000 7f45bc953700 1 -- start start 2023-09-30T22:27:10.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45bc953700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 0x7f45b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45bc953700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b809d630 0x7f45b80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45bc953700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45b8133930 0x7f45b8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45bc953700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80a2a70 con 0x7f45b8133930 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45bc953700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80a01d0 con 0x7f45b80a0a30 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45bc953700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45b809e2b0 con 0x7f45b809d630 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b809d630 0x7f45b80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 0x7f45b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b809d630 0x7f45b80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50020/0 (socket says 172.21.15.32:50020) 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 0x7f45b80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47422/0 (socket says 172.21.15.32:47422) 2023-09-30T22:27:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b5d9b700 1 -- 172.21.15.32:0/1632310351 learned_addr learned my addr 172.21.15.32:0/1632310351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b6d9d700 1 --2- 172.21.15.32:0/1632310351 >> v2:172.21.15.32:3300/0 conn(0x7f45b8133930 0x7f45b8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 -- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b809d630 msgr2=0x7f45b80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 --2- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b809d630 0x7f45b80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 -- 172.21.15.32:0/1632310351 >> v2:172.21.15.32:3300/0 conn(0x7f45b8133930 msgr2=0x7f45b8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 --2- 172.21.15.32:0/1632310351 >> v2:172.21.15.32:3300/0 conn(0x7f45b8133930 0x7f45b8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 -- 172.21.15.32:0/1632310351 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b809dc50 con 0x7f45b80a0a30 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.025+0000 7f45b659c700 1 --2- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 0x7f45b80a0e00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009840 tx=0x7f45a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3820b4ccf621e85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45b559a700 1 -- 172.21.15.32:0/1632310351 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011830 con 0x7f45b80a0a30 2023-09-30T22:27:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45b559a700 1 -- 172.21.15.32:0/1632310351 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8011990 con 0x7f45b80a0a30 2023-09-30T22:27:10.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45b559a700 1 -- 172.21.15.32:0/1632310351 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011d20 con 0x7f45b80a0a30 2023-09-30T22:27:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 -- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 msgr2=0x7f45b80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 --2- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 0x7f45b80a0e00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009840 tx=0x7f45a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 -- 172.21.15.32:0/1632310351 shutdown_connections 2023-09-30T22:27:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 --2- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b80a0a30 0x7f45b80a0e00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 --2- 172.21.15.32:0/1632310351 >> v2:172.21.15.32:3300/0 conn(0x7f45b8133930 0x7f45b8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 --2- 172.21.15.32:0/1632310351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b809d630 0x7f45b80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 -- 172.21.15.32:0/1632310351 >> 172.21.15.32:0/1632310351 conn(0x7f45b80a3050 msgr2=0x7f45b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 -- 172.21.15.32:0/1632310351 shutdown_connections 2023-09-30T22:27:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.026+0000 7f45bc953700 1 -- 172.21.15.32:0/1632310351 wait complete. 2023-09-30T22:27:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 Processor -- start 2023-09-30T22:27:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 -- start start 2023-09-30T22:27:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 0x7f45b8121160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b80a0a30 0x7f45b81216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8133930 0x7f45b8121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b81255a0 con 0x7f45b8133930 2023-09-30T22:27:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8122cd0 con 0x7f45b809d630 2023-09-30T22:27:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45bc953700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8124640 con 0x7f45b80a0a30 2023-09-30T22:27:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45b659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 0x7f45b8121160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45b659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 0x7f45b8121160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47426/0 (socket says 172.21.15.32:47426) 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45b6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8133930 0x7f45b8121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45b659c700 1 -- 172.21.15.32:0/859864313 learned_addr learned my addr 172.21.15.32:0/859864313 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.028+0000 7f45b5d9b700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b80a0a30 0x7f45b81216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b6d9d700 1 -- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b80a0a30 msgr2=0x7f45b81216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b6d9d700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b80a0a30 0x7f45b81216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b6d9d700 1 -- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 msgr2=0x7f45b8121160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b6d9d700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 0x7f45b8121160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b6d9d700 1 -- 172.21.15.32:0/859864313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a80094a0 con 0x7f45b8133930 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b659c700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 0x7f45b8121160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45b6d9d700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8133930 0x7f45b8121be0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009d90 tx=0x7f45b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0009c00 con 0x7f45b8133930 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b000f040 con 0x7f45b8133930 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b8123b60 con 0x7f45b8133930 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0014480 con 0x7f45b8133930 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.029+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b80a0410 con 0x7f45b8133930 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.030+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b811df60 con 0x7f45b8133930 2023-09-30T22:27:10.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.030+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f45b00145e0 con 0x7f45b8133930 2023-09-30T22:27:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.030+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f45b0040210 con 0x7f45b8133930 2023-09-30T22:27:10.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.035+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45b0007920 con 0x7f45b8133930 2023-09-30T22:27:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.193+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb704121fc0 con 0x7fb70409d630 2023-09-30T22:27:10.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.194+0000 7fb7027fc700 1 -- 172.21.15.32:0/612638021 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb704121fc0 con 0x7fb70409d630 2023-09-30T22:27:10.194 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 msgr2=0x7fb7041c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7041c8090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb7040a2bb0 tx=0x7fb6fc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 shutdown_connections 2023-09-30T22:27:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb704133930 0x7fb7041c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70409d630 0x7fb7041c8090 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 --2- 172.21.15.32:0/612638021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7040a0a30 0x7fb7041c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 >> 172.21.15.32:0/612638021 conn(0x7fb7040a3050 msgr2=0x7fb7041bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 shutdown_connections 2023-09-30T22:27:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.196+0000 7fb70bacd700 1 -- 172.21.15.32:0/612638021 wait complete. 2023-09-30T22:27:10.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:10.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.242+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f45b80747f0 con 0x7f45b8133930 2023-09-30T22:27:10.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.243+0000 7f45a6ffd700 1 -- 172.21.15.32:0/859864313 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f45b0007330 con 0x7f45b8133930 2023-09-30T22:27:10.243 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8133930 msgr2=0x7f45b8121be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8133930 0x7f45b8121be0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009d90 tx=0x7f45b000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 shutdown_connections 2023-09-30T22:27:10.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45b8133930 0x7f45b8121be0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f45b809d630 0x7f45b8121160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 --2- 172.21.15.32:0/859864313 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f45b80a0a30 0x7f45b81216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 >> 172.21.15.32:0/859864313 conn(0x7f45b80a3050 msgr2=0x7f45b81331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 shutdown_connections 2023-09-30T22:27:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.245+0000 7f45bc953700 1 -- 172.21.15.32:0/859864313 wait complete. 2023-09-30T22:27:10.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:10.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:10.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.933+0000 7f8bd3a82700 1 Processor -- start 2023-09-30T22:27:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.933+0000 7f8bd3a82700 1 -- start start 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd3a82700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc0a0a30 0x7f8bcc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd3a82700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc138d80 0x7f8bcc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd3a82700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0a2a70 con 0x7f8bcc09d630 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd3a82700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0a01d0 con 0x7f8bcc138d80 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd3a82700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc09e2b0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd101d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8bcc09d630 0x7f8bcc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43320/0 (socket says 172.21.15.32:43320) 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd101d700 1 -- 172.21.15.32:0/238491437 learned_addr learned my addr 172.21.15.32:0/238491437 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd181e700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc0a0a30 0x7f8bcc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd201f700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc138d80 0x7f8bcc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd081c700 1 -- 172.21.15.32:0/238491437 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711145590 0 0) 0x7f8bcc0a2a70 con 0x7f8bcc09d630 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd181e700 1 -- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc138d80 msgr2=0x7f8bcc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd181e700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc138d80 0x7f8bcc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd181e700 1 -- 172.21.15.32:0/238491437 >> v1:172.21.15.32:6789/0 conn(0x7f8bcc09d630 legacy=0x7f8bcc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd181e700 1 -- 172.21.15.32:0/238491437 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc09dc50 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd201f700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc138d80 0x7f8bcc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.934+0000 7f8bd181e700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc0a0a30 0x7f8bcc0a0e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009dc0 tx=0x7f8bc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b39bc9c4a36b372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd081c700 1 -- 172.21.15.32:0/238491437 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800e070 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd081c700 1 -- 172.21.15.32:0/238491437 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc800bb20 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd081c700 1 -- 172.21.15.32:0/238491437 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8012430 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 -- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc0a0a30 msgr2=0x7f8bcc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc0a0a30 0x7f8bcc0a0e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009dc0 tx=0x7f8bc800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 -- 172.21.15.32:0/238491437 shutdown_connections 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc138d80 0x7f8bcc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/238491437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc0a0a30 0x7f8bcc0a0e00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 -- 172.21.15.32:0/238491437 >> 172.21.15.32:0/238491437 conn(0x7f8bcc0a3050 msgr2=0x7f8bcc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.935+0000 7f8bd3a82700 1 -- 172.21.15.32:0/238491437 shutdown_connections 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.936+0000 7f8bd3a82700 1 -- 172.21.15.32:0/238491437 wait complete. 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.936+0000 7f8bd3a82700 1 Processor -- start 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 -- start start 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bcc09d630 0x7f8bcc133790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 0x7f8bcc133cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 0x7f8bcc134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0a0410 con 0x7f8bcc09d630 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc136ec0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd3a82700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc13ac70 con 0x7f8bcc138d80 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 0x7f8bcc133cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd201f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 0x7f8bcc134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 0x7f8bcc133cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53888/0 (socket says 172.21.15.32:53888) 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd201f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 0x7f8bcc134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33992/0 (socket says 172.21.15.32:33992) 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 -- 172.21.15.32:0/680998978 learned_addr learned my addr 172.21.15.32:0/680998978 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd181e700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bcc09d630 0x7f8bcc133790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 -- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 msgr2=0x7f8bcc134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 0x7f8bcc134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 -- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bcc09d630 msgr2=0x7f8bcc133790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bcc09d630 0x7f8bcc133790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd101d700 1 -- 172.21.15.32:0/680998978 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40094a0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.937+0000 7f8bd201f700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 0x7f8bcc134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.938+0000 7f8bd101d700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 0x7f8bcc133cd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c520 tx=0x7f8bbc00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.938+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc004c80 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.938+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc004de0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.938+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc8009a70 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.938+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc007310 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.938+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1371a0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.939+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bbc007470 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.939+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc0747f0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.939+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8bbc03f380 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.944+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bbc0123b0 con 0x7f8bcc0a0a30 2023-09-30T22:27:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.988+0000 7f0714f78700 1 Processor -- start 2023-09-30T22:27:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.988+0000 7f0714f78700 1 -- start start 2023-09-30T22:27:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f0714f78700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 0x7f071009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f0714f78700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f071009e000 0x7f0710130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f0714f78700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f0710130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f0714f78700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071009f520 con 0x7f071009e000 2023-09-30T22:27:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f0714f78700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071009f130 con 0x7f071009e410 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f0714f78700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f071009e930 con 0x7f071009d6f0 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f070e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 0x7f071009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f070ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f0710130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f070dd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f071009e000 0x7f0710130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f070ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f0710130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53890/0 (socket says 172.21.15.32:53890) 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f070e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 0x7f071009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33996/0 (socket says 172.21.15.32:33996) 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.989+0000 7f070ed9d700 1 -- 172.21.15.32:0/219621131 learned_addr learned my addr 172.21.15.32:0/219621131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070e59c700 1 -- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 msgr2=0x7f0710130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070e59c700 1 --2- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f0710130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070e59c700 1 -- 172.21.15.32:0/219621131 >> v2:172.21.15.32:3300/0 conn(0x7f071009e000 msgr2=0x7f0710130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070e59c700 1 --2- 172.21.15.32:0/219621131 >> v2:172.21.15.32:3300/0 conn(0x7f071009e000 0x7f0710130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070e59c700 1 -- 172.21.15.32:0/219621131 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071009b2d0 con 0x7f071009d6f0 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070ed9d700 1 --2- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f0710130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070dd9b700 1 --2- 172.21.15.32:0/219621131 >> v2:172.21.15.32:3300/0 conn(0x7f071009e000 0x7f0710130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070e59c700 1 --2- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 0x7f071009dac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0700009dc0 tx=0x7f070000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b1bdb28f76e9d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070d59a700 1 -- 172.21.15.32:0/219621131 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000bce0 con 0x7f071009d6f0 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070d59a700 1 -- 172.21.15.32:0/219621131 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000be40 con 0x7f071009d6f0 2023-09-30T22:27:10.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.990+0000 7f070d59a700 1 -- 172.21.15.32:0/219621131 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000116a0 con 0x7f071009d6f0 2023-09-30T22:27:10.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 -- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 msgr2=0x7f071009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 --2- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 0x7f071009dac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0700009dc0 tx=0x7f070000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 -- 172.21.15.32:0/219621131 shutdown_connections 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 --2- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f0710130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 --2- 172.21.15.32:0/219621131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009d6f0 0x7f071009dac0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 --2- 172.21.15.32:0/219621131 >> v2:172.21.15.32:3300/0 conn(0x7f071009e000 0x7f0710130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 -- 172.21.15.32:0/219621131 >> 172.21.15.32:0/219621131 conn(0x7f0710066f50 msgr2=0x7f071009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 -- 172.21.15.32:0/219621131 shutdown_connections 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.991+0000 7f0714f78700 1 -- 172.21.15.32:0/219621131 wait complete. 2023-09-30T22:27:10.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.993+0000 7f0714f78700 1 Processor -- start 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.993+0000 7f0714f78700 1 -- start start 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f0714f78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 0x7f071011e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f0714f78700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009e000 0x7f071011e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f0714f78700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f071011eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f0714f78700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710121b80 con 0x7f071009d6f0 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f0714f78700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07100a1a50 con 0x7f071009e410 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f0714f78700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f071009c700 con 0x7f071009e000 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 0x7f071011e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f071011eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009e000 0x7f071011e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 0x7f071011e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50746/0 (socket says 172.21.15.32:50746) 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 -- 172.21.15.32:0/1443250728 learned_addr learned my addr 172.21.15.32:0/1443250728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 -- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009e000 msgr2=0x7f071011e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009e000 0x7f071011e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 -- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 msgr2=0x7f071011eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f071011eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 -- 172.21.15.32:0/1443250728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80094a0 con 0x7f071009d6f0 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070dd9b700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009e000 0x7f071011e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.994+0000 7f070e59c700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 0x7f071011e060 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f070000b370 tx=0x7f0700004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.995+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700026070 con 0x7f071009d6f0 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.995+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700003e90 con 0x7f071009d6f0 2023-09-30T22:27:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.995+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0708009550 con 0x7f071009d6f0 2023-09-30T22:27:10.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.995+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000046a0 con 0x7f071009d6f0 2023-09-30T22:27:10.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.995+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0700009a70 con 0x7f071009d6f0 2023-09-30T22:27:10.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.996+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0700004800 con 0x7f071009d6f0 2023-09-30T22:27:10.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.996+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0710120f50 con 0x7f071009d6f0 2023-09-30T22:27:10.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:10.996+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0700044670 con 0x7f071009d6f0 2023-09-30T22:27:11.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.001+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0700011460 con 0x7f071009d6f0 2023-09-30T22:27:11.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.159+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8bcc072980 con 0x7f8bcc0a0a30 2023-09-30T22:27:11.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.159+0000 7f8bc27fc700 1 -- 172.21.15.32:0/680998978 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bbc03f020 con 0x7f8bcc0a0a30 2023-09-30T22:27:11.159 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 msgr2=0x7f8bcc133cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 0x7f8bcc133cd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00c520 tx=0x7f8bbc00c830 comp rx=0 tx=0).stop 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 shutdown_connections 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bcc138d80 0x7f8bcc134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bcc09d630 0x7f8bcc133790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 --2- 172.21.15.32:0/680998978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bcc0a0a30 0x7f8bcc133cd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 >> 172.21.15.32:0/680998978 conn(0x7f8bcc0a3050 msgr2=0x7f8bcc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.161+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 shutdown_connections 2023-09-30T22:27:11.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.162+0000 7f8bd3a82700 1 -- 172.21.15.32:0/680998978 wait complete. 2023-09-30T22:27:11.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:11.211 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:11.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.212+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0710074ea0 con 0x7f071009d6f0 2023-09-30T22:27:11.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.212+0000 7f0706ffd700 1 -- 172.21.15.32:0/1443250728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f070002a370 con 0x7f071009d6f0 2023-09-30T22:27:11.213 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:11.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 msgr2=0x7f071011e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 0x7f071011e060 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f070000b370 tx=0x7f0700004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 shutdown_connections 2023-09-30T22:27:11.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f071009e410 0x7f071011eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f071009d6f0 0x7f071011e060 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 --2- 172.21.15.32:0/1443250728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f071009e000 0x7f071011e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 >> 172.21.15.32:0/1443250728 conn(0x7f0710066f50 msgr2=0x7f071006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 shutdown_connections 2023-09-30T22:27:11.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.214+0000 7f0714f78700 1 -- 172.21.15.32:0/1443250728 wait complete. 2023-09-30T22:27:11.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:11.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.351+0000 7fcbc6607700 1 Processor -- start 2023-09-30T22:27:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.351+0000 7fcbc6607700 1 -- start start 2023-09-30T22:27:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc6607700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 0x7fcbc0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc6607700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 0x7fcbc013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc6607700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc009edd0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc6607700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00a18e0 con 0x7fcbc0139250 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc6607700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc009e1f0 con 0x7fcbc00a1510 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc4ba4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 0x7fcbc013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbbf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 0x7fcbc0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbbffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcbc00a0a30 0x7fcbc00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43322/0 (socket says 172.21.15.32:43322) 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbbf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 0x7fcbc0138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53898/0 (socket says 172.21.15.32:53898) 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbbffff700 1 -- 172.21.15.32:0/2718353821 learned_addr learned my addr 172.21.15.32:0/2718353821 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:11.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbc4ba4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 0x7fcbc013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50758/0 (socket says 172.21.15.32:50758) 2023-09-30T22:27:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbbf7fe700 1 -- 172.21.15.32:0/2718353821 >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 msgr2=0x7fcbc013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:11.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.352+0000 7fcbbf7fe700 1 --2- 172.21.15.32:0/2718353821 >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 0x7fcbc013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbbf7fe700 1 -- 172.21.15.32:0/2718353821 >> v1:172.21.15.32:6789/0 conn(0x7fcbc00a0a30 legacy=0x7fcbc00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbbf7fe700 1 -- 172.21.15.32:0/2718353821 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc009dc50 con 0x7fcbc00a1510 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbc4ba4700 1 --2- 172.21.15.32:0/2718353821 >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 0x7fcbc013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbbf7fe700 1 --2- 172.21.15.32:0/2718353821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 0x7fcbc0138d10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009870 tx=0x7fcbb4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d78e53f425cff7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbbeffd700 1 -- 172.21.15.32:0/2718353821 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4011830 con 0x7fcbc00a1510 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbbeffd700 1 -- 172.21.15.32:0/2718353821 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4011990 con 0x7fcbc00a1510 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.353+0000 7fcbbeffd700 1 -- 172.21.15.32:0/2718353821 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4011d20 con 0x7fcbc00a1510 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 -- 172.21.15.32:0/2718353821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 msgr2=0x7fcbc0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 --2- 172.21.15.32:0/2718353821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 0x7fcbc0138d10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009870 tx=0x7fcbb4009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 -- 172.21.15.32:0/2718353821 shutdown_connections 2023-09-30T22:27:11.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 --2- 172.21.15.32:0/2718353821 >> v2:172.21.15.32:3300/0 conn(0x7fcbc0139250 0x7fcbc013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 --2- 172.21.15.32:0/2718353821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a1510 0x7fcbc0138d10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 -- 172.21.15.32:0/2718353821 >> 172.21.15.32:0/2718353821 conn(0x7fcbc00a6550 msgr2=0x7fcbc00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 -- 172.21.15.32:0/2718353821 shutdown_connections 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.354+0000 7fcbc6607700 1 -- 172.21.15.32:0/2718353821 wait complete. 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 Processor -- start 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 -- start start 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a0a30 0x7fcbc01c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbc00a1510 0x7fcbc01c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 0x7fcbc0122a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc01389b0 con 0x7fcbc0139250 2023-09-30T22:27:11.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc0125cd0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc6607700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc0126710 con 0x7fcbc00a1510 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.355+0000 7fcbc4ba4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 0x7fcbc0122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a0a30 0x7fcbc01c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbc4ba4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 0x7fcbc0122a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50772/0 (socket says 172.21.15.32:50772) 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbc4ba4700 1 -- 172.21.15.32:0/479259832 learned_addr learned my addr 172.21.15.32:0/479259832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbf7fe700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbc00a1510 0x7fcbc01c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 -- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbc00a1510 msgr2=0x7fcbc01c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbc00a1510 0x7fcbc01c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 -- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 msgr2=0x7fcbc0122a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 0x7fcbc0122a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 -- 172.21.15.32:0/479259832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb8009a70 con 0x7fcbc00a0a30 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbf7fe700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbc00a1510 0x7fcbc01c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbc4ba4700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 0x7fcbc0122a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbffff700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a0a30 0x7fcbc01c8800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000c500 tx=0x7fcbb000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0005d70 con 0x7fcbc00a0a30 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb00070d0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb40094a0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0007440 con 0x7fcbc00a0a30 2023-09-30T22:27:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.356+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc0125fa0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.357+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc01c92e0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.358+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcbb00075a0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.358+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcbb003eea0 con 0x7fcbc00a0a30 2023-09-30T22:27:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.367+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcbb0019b40 con 0x7fcbc00a0a30 2023-09-30T22:27:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.414+0000 7f7c4b527700 1 Processor -- start 2023-09-30T22:27:11.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.414+0000 7f7c4b527700 1 -- start start 2023-09-30T22:27:11.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c4b527700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44133420 0x7f7c441337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c4b527700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44134260 0x7f7c441306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c4b527700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4406b990 con 0x7f7c44133e30 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c4b527700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440a0590 con 0x7f7c44134260 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c4b527700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4409fb80 con 0x7f7c44133420 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c49ac4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44134260 0x7f7c441306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c48ac2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c44133e30 0x7f7c4412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43332/0 (socket says 172.21.15.32:43332) 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c48ac2700 1 -- 172.21.15.32:0/2539953022 learned_addr learned my addr 172.21.15.32:0/2539953022 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c492c3700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44133420 0x7f7c441337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c49ac4700 1 -- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44133420 msgr2=0x7f7c441337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:11.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c49ac4700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44133420 0x7f7c441337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c3bfff700 1 -- 172.21.15.32:0/2539953022 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588570691 0 0) 0x7f7c4406b990 con 0x7f7c44133e30 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c49ac4700 1 -- 172.21.15.32:0/2539953022 >> v1:172.21.15.32:6789/0 conn(0x7f7c44133e30 legacy=0x7f7c4412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c49ac4700 1 -- 172.21.15.32:0/2539953022 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440a0030 con 0x7f7c44134260 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.415+0000 7f7c492c3700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44133420 0x7f7c441337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c49ac4700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44134260 0x7f7c441306a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009dc0 tx=0x7f7c3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bb05f3241dbc1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c3bfff700 1 -- 172.21.15.32:0/2539953022 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00e070 con 0x7f7c44134260 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c3bfff700 1 -- 172.21.15.32:0/2539953022 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c00bb20 con 0x7f7c44134260 2023-09-30T22:27:11.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c3bfff700 1 -- 172.21.15.32:0/2539953022 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c012430 con 0x7f7c44134260 2023-09-30T22:27:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c4b527700 1 -- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44134260 msgr2=0x7f7c441306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c4b527700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44134260 0x7f7c441306a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009dc0 tx=0x7f7c3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c4b527700 1 -- 172.21.15.32:0/2539953022 shutdown_connections 2023-09-30T22:27:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c4b527700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44134260 0x7f7c441306a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c4b527700 1 --2- 172.21.15.32:0/2539953022 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44133420 0x7f7c441337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.416+0000 7f7c4b527700 1 -- 172.21.15.32:0/2539953022 >> 172.21.15.32:0/2539953022 conn(0x7f7c440a4210 msgr2=0x7f7c440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.417+0000 7f7c4b527700 1 -- 172.21.15.32:0/2539953022 shutdown_connections 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.417+0000 7f7c4b527700 1 -- 172.21.15.32:0/2539953022 wait complete. 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 Processor -- start 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 -- start start 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 0x7f7c4409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44133e30 0x7f7c4409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 0x7f7c4409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4411feb0 con 0x7f7c44133420 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4412d9b0 con 0x7f7c44133e30 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c4b527700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44120100 con 0x7f7c44134260 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c492c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 0x7f7c4409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c49ac4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 0x7f7c4409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c49ac4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 0x7f7c4409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34026/0 (socket says 172.21.15.32:34026) 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c492c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 0x7f7c4409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50782/0 (socket says 172.21.15.32:50782) 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c49ac4700 1 -- 172.21.15.32:0/508738398 learned_addr learned my addr 172.21.15.32:0/508738398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.418+0000 7f7c48ac2700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44133e30 0x7f7c4409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c49ac4700 1 -- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44133e30 msgr2=0x7f7c4409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c49ac4700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44133e30 0x7f7c4409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c49ac4700 1 -- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 msgr2=0x7f7c4409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c49ac4700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 0x7f7c4409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c49ac4700 1 -- 172.21.15.32:0/508738398 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c400094a0 con 0x7f7c44134260 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c48ac2700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44133e30 0x7f7c4409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c492c3700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 0x7f7c4409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c49ac4700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 0x7f7c4409cf00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c013000 tx=0x7f7c3c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c012430 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c020070 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c009a70 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004280 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.419+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c44131860 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.420+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c3c0043e0 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.420+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c3c022070 con 0x7f7c44134260 2023-09-30T22:27:11.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.420+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c44074ea0 con 0x7f7c44134260 2023-09-30T22:27:11.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.425+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c3c01b400 con 0x7f7c44134260 2023-09-30T22:27:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.576+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcbc0126190 con 0x7fcbc00a0a30 2023-09-30T22:27:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.576+0000 7fcbbcff9700 1 -- 172.21.15.32:0/479259832 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcbb0017030 con 0x7fcbc00a0a30 2023-09-30T22:27:11.576 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:11.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a0a30 msgr2=0x7fcbc01c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a0a30 0x7fcbc01c8800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000c500 tx=0x7fcbb000c810 comp rx=0 tx=0).stop 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 shutdown_connections 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0139250 0x7fcbc0122a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbc00a0a30 0x7fcbc01c8800 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 --2- 172.21.15.32:0/479259832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbc00a1510 0x7fcbc01c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.578+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 >> 172.21.15.32:0/479259832 conn(0x7fcbc00a6550 msgr2=0x7fcbc00a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.579+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 shutdown_connections 2023-09-30T22:27:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.579+0000 7fcbc6607700 1 -- 172.21.15.32:0/479259832 wait complete. 2023-09-30T22:27:11.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.629+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7c4406b250 con 0x7f7c44134260 2023-09-30T22:27:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.629+0000 7f7c39ffb700 1 -- 172.21.15.32:0/508738398 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c3c01a380 con 0x7f7c44134260 2023-09-30T22:27:11.630 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:11.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 msgr2=0x7f7c4409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 0x7f7c4409cf00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c013000 tx=0x7f7c3c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 shutdown_connections 2023-09-30T22:27:11.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c44134260 0x7f7c4409cf00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c44133420 0x7f7c4409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 --2- 172.21.15.32:0/508738398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c44133e30 0x7f7c4409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:11.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 >> 172.21.15.32:0/508738398 conn(0x7f7c440a4210 msgr2=0x7f7c440a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 shutdown_connections 2023-09-30T22:27:11.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:11.632+0000 7f7c4b527700 1 -- 172.21.15.32:0/508738398 wait complete. 2023-09-30T22:27:11.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:12.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:12.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:12.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.326+0000 7f894763f700 1 Processor -- start 2023-09-30T22:27:12.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f894763f700 1 -- start start 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f894763f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f894009e1d0 0x7f8940138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f894763f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8940139170 0x7f894013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f894763f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894009b120 con 0x7f894009d6f0 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f894763f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894006b460 con 0x7f8940139170 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f894763f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894009a7e0 con 0x7f894009e1d0 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f8945bdc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8940139170 0x7f894013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f8944bda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f894009e1d0 0x7f8940138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f89453db700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f894009d6f0 0x7f894009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43346/0 (socket says 172.21.15.32:43346) 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.327+0000 7f89453db700 1 -- 172.21.15.32:0/117473760 learned_addr learned my addr 172.21.15.32:0/117473760 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8944bda700 1 -- 172.21.15.32:0/117473760 >> v2:172.21.15.32:3300/0 conn(0x7f8940139170 msgr2=0x7f894013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8944bda700 1 --2- 172.21.15.32:0/117473760 >> v2:172.21.15.32:3300/0 conn(0x7f8940139170 0x7f894013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8944bda700 1 -- 172.21.15.32:0/117473760 >> v1:172.21.15.32:6789/0 conn(0x7f894009d6f0 legacy=0x7f894009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8944bda700 1 -- 172.21.15.32:0/117473760 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894009a500 con 0x7f894009e1d0 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8945bdc700 1 --2- 172.21.15.32:0/117473760 >> v2:172.21.15.32:3300/0 conn(0x7f8940139170 0x7f894013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8944bda700 1 --2- 172.21.15.32:0/117473760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f894009e1d0 0x7f8940138c30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8930009870 tx=0x7f8930009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa167ff4ae02b53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8937fff700 1 -- 172.21.15.32:0/117473760 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930011830 con 0x7f894009e1d0 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8937fff700 1 -- 172.21.15.32:0/117473760 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8930011990 con 0x7f894009e1d0 2023-09-30T22:27:12.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.328+0000 7f8937fff700 1 -- 172.21.15.32:0/117473760 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930011d20 con 0x7f894009e1d0 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 -- 172.21.15.32:0/117473760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f894009e1d0 msgr2=0x7f8940138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 --2- 172.21.15.32:0/117473760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f894009e1d0 0x7f8940138c30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8930009870 tx=0x7f8930009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 -- 172.21.15.32:0/117473760 shutdown_connections 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 --2- 172.21.15.32:0/117473760 >> v2:172.21.15.32:3300/0 conn(0x7f8940139170 0x7f894013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 --2- 172.21.15.32:0/117473760 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f894009e1d0 0x7f8940138c30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 -- 172.21.15.32:0/117473760 >> 172.21.15.32:0/117473760 conn(0x7f8940066f50 msgr2=0x7f894009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 -- 172.21.15.32:0/117473760 shutdown_connections 2023-09-30T22:27:12.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.329+0000 7f894763f700 1 -- 172.21.15.32:0/117473760 wait complete. 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.330+0000 7f894763f700 1 Processor -- start 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.330+0000 7f894763f700 1 -- start start 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f894763f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f894009d6f0 0x7f89401c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f894763f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894009e1d0 0x7f89401c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f894763f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 0x7f894011e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f894763f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401220a0 con 0x7f894009e1d0 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f894763f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8940122280 con 0x7f8940139170 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f894763f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89401be250 con 0x7f894009d6f0 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 0x7f894011e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8944bda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894009e1d0 0x7f89401c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 0x7f894011e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53938/0 (socket says 172.21.15.32:53938) 2023-09-30T22:27:12.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 -- 172.21.15.32:0/4214658852 learned_addr learned my addr 172.21.15.32:0/4214658852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f89453db700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f894009d6f0 0x7f89401c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 -- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f894009d6f0 msgr2=0x7f89401c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f894009d6f0 0x7f89401c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 -- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894009e1d0 msgr2=0x7f89401c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894009e1d0 0x7f89401c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8945bdc700 1 -- 172.21.15.32:0/4214658852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938009a70 con 0x7f8940139170 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.331+0000 7f8944bda700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894009e1d0 0x7f89401c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.332+0000 7f8945bdc700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 0x7f894011e760 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f893800efc0 tx=0x7f893800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.332+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893800ca90 con 0x7f8940139170 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.332+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893800f040 con 0x7f8940139170 2023-09-30T22:27:12.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.332+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89300094a0 con 0x7f8940139170 2023-09-30T22:27:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.332+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893800ce10 con 0x7f8940139170 2023-09-30T22:27:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.332+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89400a29f0 con 0x7f8940139170 2023-09-30T22:27:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.333+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8938014480 con 0x7f8940139170 2023-09-30T22:27:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.333+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f893803fd00 con 0x7f8940139170 2023-09-30T22:27:12.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.333+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89401c9130 con 0x7f8940139170 2023-09-30T22:27:12.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.338+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8938012930 con 0x7f8940139170 2023-09-30T22:27:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.376+0000 7f1b188c7700 1 Processor -- start 2023-09-30T22:27:12.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.376+0000 7f1b188c7700 1 -- start start 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b188c7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b140a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b188c7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b14138d80 0x7f1b1413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b188c7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b140a2a70 con 0x7f1b140a0a30 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b188c7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b140a01d0 con 0x7f1b14138d80 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b188c7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1409e2b0 con 0x7f1b1409d630 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b14138d80 0x7f1b1413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b11d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b140a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b1259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b140a0a30 0x7f1b140a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43360/0 (socket says 172.21.15.32:43360) 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b14138d80 0x7f1b1413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50808/0 (socket says 172.21.15.32:50808) 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b1259c700 1 -- 172.21.15.32:0/393515299 learned_addr learned my addr 172.21.15.32:0/393515299 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b11d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b140a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53952/0 (socket says 172.21.15.32:53952) 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 -- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 msgr2=0x7f1b140a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 --2- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b140a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 -- 172.21.15.32:0/393515299 >> v1:172.21.15.32:6789/0 conn(0x7f1b140a0a30 legacy=0x7f1b140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 -- 172.21.15.32:0/393515299 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1409dc50 con 0x7f1b14138d80 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b11d9b700 1 --2- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b140a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:12.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b1159a700 1 -- 172.21.15.32:0/393515299 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991675401 0 0) 0x7f1b140a2a70 con 0x7f1b140a0a30 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.377+0000 7f1b12d9d700 1 --2- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1413b160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009d90 tx=0x7f1b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e24f280feea94f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b1159a700 1 -- 172.21.15.32:0/393515299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00e070 con 0x7f1b14138d80 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b1159a700 1 -- 172.21.15.32:0/393515299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c00bb20 con 0x7f1b14138d80 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b1159a700 1 -- 172.21.15.32:0/393515299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c012430 con 0x7f1b14138d80 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 -- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 msgr2=0x7f1b1413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 --2- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1413b160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009d90 tx=0x7f1b0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 -- 172.21.15.32:0/393515299 shutdown_connections 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 --2- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1413b160 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 --2- 172.21.15.32:0/393515299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b140a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 -- 172.21.15.32:0/393515299 >> 172.21.15.32:0/393515299 conn(0x7f1b140a3050 msgr2=0x7f1b140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 -- 172.21.15.32:0/393515299 shutdown_connections 2023-09-30T22:27:12.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.378+0000 7f1b188c7700 1 -- 172.21.15.32:0/393515299 wait complete. 2023-09-30T22:27:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 Processor -- start 2023-09-30T22:27:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 -- start start 2023-09-30T22:27:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b1412c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b140a0a30 0x7f1b1412cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1412cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14130190 con 0x7f1b14138d80 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1412fc10 con 0x7f1b1409d630 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b188c7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140a0410 con 0x7f1b140a0a30 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b12d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1412cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b1259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b1412c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b12d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1412cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50816/0 (socket says 172.21.15.32:50816) 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b12d9d700 1 -- 172.21.15.32:0/3294163725 learned_addr learned my addr 172.21.15.32:0/3294163725 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.380+0000 7f1b11d9b700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b140a0a30 0x7f1b1412cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b1259c700 1 -- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b140a0a30 msgr2=0x7f1b1412cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b1259c700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b140a0a30 0x7f1b1412cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b1259c700 1 -- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 msgr2=0x7f1b1412cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b1259c700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1412cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b1259c700 1 -- 172.21.15.32:0/3294163725 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080094a0 con 0x7f1b1409d630 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b12d9d700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1412cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b1259c700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b1412c570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c470 tx=0x7f1b0400c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004c80 con 0x7f1b1409d630 2023-09-30T22:27:12.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04004de0 con 0x7f1b1409d630 2023-09-30T22:27:12.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c009a70 con 0x7f1b1409d630 2023-09-30T22:27:12.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04007310 con 0x7f1b1409d630 2023-09-30T22:27:12.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.381+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1412e5a0 con 0x7f1b1409d630 2023-09-30T22:27:12.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.382+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b04007470 con 0x7f1b1409d630 2023-09-30T22:27:12.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.382+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b140747f0 con 0x7f1b1409d630 2023-09-30T22:27:12.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.382+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1b0403f260 con 0x7f1b1409d630 2023-09-30T22:27:12.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.387+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b04007720 con 0x7f1b1409d630 2023-09-30T22:27:12.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.550+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f894009f420 con 0x7f8940139170 2023-09-30T22:27:12.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.550+0000 7f8935ffb700 1 -- 172.21.15.32:0/4214658852 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8938012340 con 0x7f8940139170 2023-09-30T22:27:12.552 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 msgr2=0x7f894011e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 0x7f894011e760 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f893800efc0 tx=0x7f893800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 shutdown_connections 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8940139170 0x7f894011e760 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f894009d6f0 0x7f89401c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 --2- 172.21.15.32:0/4214658852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894009e1d0 0x7f89401c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 >> 172.21.15.32:0/4214658852 conn(0x7f8940066f50 msgr2=0x7f89400a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 shutdown_connections 2023-09-30T22:27:12.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.553+0000 7f894763f700 1 -- 172.21.15.32:0/4214658852 wait complete. 2023-09-30T22:27:12.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.591+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b14072980 con 0x7f1b1409d630 2023-09-30T22:27:12.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.592+0000 7f1b02ffd700 1 -- 172.21.15.32:0/3294163725 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b04011360 con 0x7f1b1409d630 2023-09-30T22:27:12.593 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:12.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 msgr2=0x7f1b1412c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b1412c570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400c470 tx=0x7f1b0400c780 comp rx=0 tx=0).stop 2023-09-30T22:27:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 shutdown_connections 2023-09-30T22:27:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b14138d80 0x7f1b1412cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1b1409d630 0x7f1b1412c570 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 --2- 172.21.15.32:0/3294163725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1b140a0a30 0x7f1b1412cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 >> 172.21.15.32:0/3294163725 conn(0x7f1b140a3050 msgr2=0x7f1b1409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 shutdown_connections 2023-09-30T22:27:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.594+0000 7f1b188c7700 1 -- 172.21.15.32:0/3294163725 wait complete. 2023-09-30T22:27:12.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:12.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:12.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.731+0000 7f7f6eb22700 1 Processor -- start 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.731+0000 7f7f6eb22700 1 -- start start 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.731+0000 7f7f6eb22700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f680a0a30 0x7f7f680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.731+0000 7f7f6eb22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f680a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.731+0000 7f7f6eb22700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f68133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6eb22700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680a2a70 con 0x7f7f680a0a30 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6eb22700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680a01d0 con 0x7f7f6809d630 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6eb22700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6809e2b0 con 0x7f7f68133930 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f680a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f680a0a30 0x7f7f680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f680a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53970/0 (socket says 172.21.15.32:53970) 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f680a0a30 0x7f7f680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50822/0 (socket says 172.21.15.32:50822) 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f67fff700 1 -- 172.21.15.32:0/3795885422 learned_addr learned my addr 172.21.15.32:0/3795885422 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6d0bf700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f68133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 -- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 msgr2=0x7f7f68133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f68133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 -- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 msgr2=0x7f7f680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f680a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 -- 172.21.15.32:0/3795885422 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6809dc50 con 0x7f7f680a0a30 2023-09-30T22:27:12.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f67fff700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:12.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f6c8be700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f680a0e00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009840 tx=0x7f7f5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ab1ff13cd08ec3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.732+0000 7f7f677fe700 1 -- 172.21.15.32:0/3795885422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c011830 con 0x7f7f680a0a30 2023-09-30T22:27:12.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f677fe700 1 -- 172.21.15.32:0/3795885422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c011990 con 0x7f7f680a0a30 2023-09-30T22:27:12.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f677fe700 1 -- 172.21.15.32:0/3795885422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c011d20 con 0x7f7f680a0a30 2023-09-30T22:27:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 -- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 msgr2=0x7f7f680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f680a0e00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009840 tx=0x7f7f5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 -- 172.21.15.32:0/3795885422 shutdown_connections 2023-09-30T22:27:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f68133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f680a0e00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/3795885422 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 -- 172.21.15.32:0/3795885422 >> 172.21.15.32:0/3795885422 conn(0x7f7f680a3050 msgr2=0x7f7f680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 -- 172.21.15.32:0/3795885422 shutdown_connections 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.733+0000 7f7f6eb22700 1 -- 172.21.15.32:0/3795885422 wait complete. 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.734+0000 7f7f6eb22700 1 Processor -- start 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.734+0000 7f7f6eb22700 1 -- start start 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6eb22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f68132940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6eb22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f68132e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6eb22700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f681c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6eb22700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68122310 con 0x7f7f680a0a30 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6eb22700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f681333c0 con 0x7f7f6809d630 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6eb22700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68120650 con 0x7f7f68133930 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6c8be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f68132940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6d0bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f681c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6c8be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f68132940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53982/0 (socket says 172.21.15.32:53982) 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6c8be700 1 -- 172.21.15.32:0/1064713338 learned_addr learned my addr 172.21.15.32:0/1064713338 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f67fff700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f68132e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6d0bf700 1 -- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 msgr2=0x7f7f68132940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6d0bf700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f68132940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6d0bf700 1 -- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 msgr2=0x7f7f68132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6d0bf700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f68132e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.735+0000 7f7f6d0bf700 1 -- 172.21.15.32:0/1064713338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54009550 con 0x7f7f68133930 2023-09-30T22:27:12.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.736+0000 7f7f6d0bf700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f681c7f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000efc0 tx=0x7f7f6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.736+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000cc90 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.736+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6000f040 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.736+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c0094a0 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.736+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600146c0 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.736+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f680a0410 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.737+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f60014820 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.737+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f60040260 con 0x7f7f68133930 2023-09-30T22:27:12.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.737+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f680747f0 con 0x7f7f68133930 2023-09-30T22:27:12.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.742+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f6001d400 con 0x7f7f68133930 2023-09-30T22:27:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.789+0000 7f7d6dba5700 1 Processor -- start 2023-09-30T22:27:12.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.789+0000 7f7d6dba5700 1 -- start start 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d6dba5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 0x7f7d68136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d6dba5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d68136fe0 0x7f7d681393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d6dba5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d68070940 con 0x7f7d6809e930 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d6dba5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6806fd60 con 0x7f7d68136fe0 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d6dba5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6806fa80 con 0x7f7d6809f410 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 0x7f7d68136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 0x7f7d68136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34060/0 (socket says 172.21.15.32:34060) 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d67fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d68136fe0 0x7f7d681393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 -- 172.21.15.32:0/849732138 learned_addr learned my addr 172.21.15.32:0/849732138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 -- 172.21.15.32:0/849732138 >> v2:172.21.15.32:3300/0 conn(0x7f7d68136fe0 msgr2=0x7f7d681393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 --2- 172.21.15.32:0/849732138 >> v2:172.21.15.32:3300/0 conn(0x7f7d68136fe0 0x7f7d681393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 -- 172.21.15.32:0/849732138 >> v1:172.21.15.32:6789/0 conn(0x7f7d6809e930 legacy=0x7f7d6809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.790+0000 7f7d66ffd700 1 -- 172.21.15.32:0/849732138 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6809fb60 con 0x7f7d6809f410 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.791+0000 7f7d66ffd700 1 --2- 172.21.15.32:0/849732138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 0x7f7d68136aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7d500098f0 tx=0x7f7d50009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=febecbb2c567522f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.791+0000 7f7d667fc700 1 -- 172.21.15.32:0/849732138 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50011830 con 0x7f7d6809f410 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.791+0000 7f7d667fc700 1 -- 172.21.15.32:0/849732138 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50011990 con 0x7f7d6809f410 2023-09-30T22:27:12.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.791+0000 7f7d667fc700 1 -- 172.21.15.32:0/849732138 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50011d20 con 0x7f7d6809f410 2023-09-30T22:27:12.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.791+0000 7f7d6dba5700 1 -- 172.21.15.32:0/849732138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 msgr2=0x7f7d68136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.791+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/849732138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 0x7f7d68136aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7d500098f0 tx=0x7f7d50009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.792+0000 7f7d6dba5700 1 -- 172.21.15.32:0/849732138 shutdown_connections 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.792+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/849732138 >> v2:172.21.15.32:3300/0 conn(0x7f7d68136fe0 0x7f7d681393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.792+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/849732138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809f410 0x7f7d68136aa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.792+0000 7f7d6dba5700 1 -- 172.21.15.32:0/849732138 >> 172.21.15.32:0/849732138 conn(0x7f7d680a4210 msgr2=0x7f7d680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.792+0000 7f7d6dba5700 1 -- 172.21.15.32:0/849732138 shutdown_connections 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.792+0000 7f7d6dba5700 1 -- 172.21.15.32:0/849732138 wait complete. 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 Processor -- start 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 -- start start 2023-09-30T22:27:12.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 0x7f7d681c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6809f410 0x7f7d681c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 0x7f7d681c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681221e0 con 0x7f7d68136fe0 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68071d20 con 0x7f7d6809f410 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.793+0000 7f7d6dba5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681204d0 con 0x7f7d6809e930 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 0x7f7d681c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 0x7f7d681c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 0x7f7d681c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34070/0 (socket says 172.21.15.32:34070) 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 0x7f7d681c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50852/0 (socket says 172.21.15.32:50852) 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 -- 172.21.15.32:0/241627427 learned_addr learned my addr 172.21.15.32:0/241627427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 -- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6809f410 msgr2=0x7f7d681c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6809f410 0x7f7d681c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 -- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 msgr2=0x7f7d681c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 0x7f7d681c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 -- 172.21.15.32:0/241627427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50009550 con 0x7f7d6809e930 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d67fff700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 0x7f7d681c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d677fe700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 0x7f7d681c6240 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7d58001ed0 tx=0x7f7d580021e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003c00 con 0x7f7d6809e930 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58003d60 con 0x7f7d6809e930 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d680a2f40 con 0x7f7d6809e930 2023-09-30T22:27:12.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800abf0 con 0x7f7d6809e930 2023-09-30T22:27:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.794+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d68121140 con 0x7f7d6809e930 2023-09-30T22:27:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.796+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68074ea0 con 0x7f7d6809e930 2023-09-30T22:27:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.796+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d5800ad50 con 0x7f7d6809e930 2023-09-30T22:27:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.796+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7d58040260 con 0x7f7d6809e930 2023-09-30T22:27:12.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.801+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d580050d0 con 0x7f7d6809e930 2023-09-30T22:27:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.951+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f68121f20 con 0x7f7f68133930 2023-09-30T22:27:12.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.951+0000 7f7f657fa700 1 -- 172.21.15.32:0/1064713338 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f60012340 con 0x7f7f68133930 2023-09-30T22:27:12.953 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 msgr2=0x7f7f681c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f681c7f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000efc0 tx=0x7f7f6000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 shutdown_connections 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f68133930 0x7f7f681c7f80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f6809d630 0x7f7f68132940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 --2- 172.21.15.32:0/1064713338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f680a0a30 0x7f7f68132e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 >> 172.21.15.32:0/1064713338 conn(0x7f7f680a3050 msgr2=0x7f7f6809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:12.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 shutdown_connections 2023-09-30T22:27:12.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:12.954+0000 7f7f6eb22700 1 -- 172.21.15.32:0/1064713338 wait complete. 2023-09-30T22:27:12.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.005+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7d6809e100 con 0x7f7d6809e930 2023-09-30T22:27:13.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.006+0000 7f7d5ffff700 1 -- 172.21.15.32:0/241627427 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d58011380 con 0x7f7d6809e930 2023-09-30T22:27:13.007 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:13.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 msgr2=0x7f7d681c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 0x7f7d681c6240 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7d58001ed0 tx=0x7f7d580021e0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 shutdown_connections 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d68136fe0 0x7f7d681c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7d6809e930 0x7f7d681c6240 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 --2- 172.21.15.32:0/241627427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d6809f410 0x7f7d681c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 >> 172.21.15.32:0/241627427 conn(0x7f7d680a4210 msgr2=0x7f7d680a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.008+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 shutdown_connections 2023-09-30T22:27:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.009+0000 7f7d6dba5700 1 -- 172.21.15.32:0/241627427 wait complete. 2023-09-30T22:27:13.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:13.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:13.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.716+0000 7fd79a995700 1 Processor -- start 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.716+0000 7fd79a995700 1 -- start start 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd79a995700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 0x7fd79409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd79a995700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 0x7fd7941317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd79a995700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd794070940 con 0x7fd79409f650 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd79a995700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79406fd60 con 0x7fd79409e930 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd79a995700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd79406fa80 con 0x7fd79409f240 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd793fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 0x7fd79409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 0x7fd7941317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 0x7fd7941317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34080/0 (socket says 172.21.15.32:34080) 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd793fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 0x7fd79409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50862/0 (socket says 172.21.15.32:50862) 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 -- 172.21.15.32:0/1228279956 learned_addr learned my addr 172.21.15.32:0/1228279956 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd792ffd700 1 -- 172.21.15.32:0/1228279956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272199545 0 0) 0x7fd794070940 con 0x7fd79409f650 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 -- 172.21.15.32:0/1228279956 >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 msgr2=0x7fd79409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 --2- 172.21.15.32:0/1228279956 >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 0x7fd79409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 -- 172.21.15.32:0/1228279956 >> v1:172.21.15.32:6789/0 conn(0x7fd79409f650 legacy=0x7fd794131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd7937fe700 1 -- 172.21.15.32:0/1228279956 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79409fb60 con 0x7fd79409f240 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.717+0000 7fd793fff700 1 --2- 172.21.15.32:0/1228279956 >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 0x7fd79409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd7937fe700 1 --2- 172.21.15.32:0/1228279956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 0x7fd7941317f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd7880098f0 tx=0x7fd788009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e694cfd64d625483 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd792ffd700 1 -- 172.21.15.32:0/1228279956 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011830 con 0x7fd79409f240 2023-09-30T22:27:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd792ffd700 1 -- 172.21.15.32:0/1228279956 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788011990 con 0x7fd79409f240 2023-09-30T22:27:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd792ffd700 1 -- 172.21.15.32:0/1228279956 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788011d20 con 0x7fd79409f240 2023-09-30T22:27:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd79a995700 1 -- 172.21.15.32:0/1228279956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 msgr2=0x7fd7941317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd79a995700 1 --2- 172.21.15.32:0/1228279956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 0x7fd7941317f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd7880098f0 tx=0x7fd788009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd79a995700 1 -- 172.21.15.32:0/1228279956 shutdown_connections 2023-09-30T22:27:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd79a995700 1 --2- 172.21.15.32:0/1228279956 >> v2:172.21.15.32:3300/0 conn(0x7fd79409e930 0x7fd79409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd79a995700 1 --2- 172.21.15.32:0/1228279956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f240 0x7fd7941317f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.718+0000 7fd79a995700 1 -- 172.21.15.32:0/1228279956 >> 172.21.15.32:0/1228279956 conn(0x7fd7940a4210 msgr2=0x7fd7940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.719+0000 7fd79a995700 1 -- 172.21.15.32:0/1228279956 shutdown_connections 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.719+0000 7fd79a995700 1 -- 172.21.15.32:0/1228279956 wait complete. 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.719+0000 7fd79a995700 1 Processor -- start 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 -- start start 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 0x7fd7941c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 0x7fd7941c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f650 0x7fd7941c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941a43f0 con 0x7fd79409f240 2023-09-30T22:27:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941a41c0 con 0x7fd79409e930 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd79a995700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941a4e30 con 0x7fd79409f650 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 0x7fd7941c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd793fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 0x7fd7941c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd793fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 0x7fd7941c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54000/0 (socket says 172.21.15.32:54000) 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd793fff700 1 -- 172.21.15.32:0/832945767 learned_addr learned my addr 172.21.15.32:0/832945767 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 0x7fd7941c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50872/0 (socket says 172.21.15.32:50872) 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.720+0000 7fd798f32700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f650 0x7fd7941c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd793fff700 1 -- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f650 msgr2=0x7fd7941c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd793fff700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f650 0x7fd7941c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd793fff700 1 -- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 msgr2=0x7fd7941c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd793fff700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 0x7fd7941c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd793fff700 1 -- 172.21.15.32:0/832945767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840094a0 con 0x7fd79409e930 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd798f32700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f650 0x7fd7941c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd7937fe700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 0x7fd7941c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd793fff700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 0x7fd7941c5e20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd784010ef0 tx=0x7fd78400b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784015070 con 0x7fd79409e930 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400bb90 con 0x7fd79409e930 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788009550 con 0x7fd79409e930 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784019410 con 0x7fd79409e930 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.721+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7940a0190 con 0x7fd79409e930 2023-09-30T22:27:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.722+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd794071d80 con 0x7fd79409e930 2023-09-30T22:27:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.722+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd784019570 con 0x7fd79409e930 2023-09-30T22:27:13.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.722+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd784042fd0 con 0x7fd79409e930 2023-09-30T22:27:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.727+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7840223b0 con 0x7fd79409e930 2023-09-30T22:27:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 Processor -- start 2023-09-30T22:27:13.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 -- start start 2023-09-30T22:27:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 0x7fab540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab5409d630 0x7fab540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 0x7fab54133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab540a2a70 con 0x7fab5409d630 2023-09-30T22:27:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab540a01d0 con 0x7fab54133930 2023-09-30T22:27:13.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5bec8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab5409e2b0 con 0x7fab540a0a30 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab5a465700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 0x7fab54133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59c64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 0x7fab540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.754+0000 7fab59463700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab5409d630 0x7fab540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59c64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 0x7fab540a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34104/0 (socket says 172.21.15.32:34104) 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab5a465700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 0x7fab54133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54002/0 (socket says 172.21.15.32:54002) 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59c64700 1 -- 172.21.15.32:0/270473116 learned_addr learned my addr 172.21.15.32:0/270473116 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59463700 1 -- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 msgr2=0x7fab540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59463700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 0x7fab540a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59463700 1 -- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 msgr2=0x7fab54133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59463700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 0x7fab54133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59463700 1 -- 172.21.15.32:0/270473116 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5409dc50 con 0x7fab5409d630 2023-09-30T22:27:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab5a465700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 0x7fab54133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59c64700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 0x7fab540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab59463700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab540a1340 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fab48009840 tx=0x7fab48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d923addb864b032b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab58c62700 1 -- 172.21.15.32:0/270473116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48011830 con 0x7fab5409d630 2023-09-30T22:27:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab58c62700 1 -- 172.21.15.32:0/270473116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab48011990 con 0x7fab5409d630 2023-09-30T22:27:13.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.755+0000 7fab58c62700 1 -- 172.21.15.32:0/270473116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48011d20 con 0x7fab5409d630 2023-09-30T22:27:13.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 -- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 msgr2=0x7fab540a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab540a1340 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fab48009840 tx=0x7fab48009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 -- 172.21.15.32:0/270473116 shutdown_connections 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab54133930 0x7fab54133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab540a0a30 0x7fab540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 --2- 172.21.15.32:0/270473116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab540a1340 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 -- 172.21.15.32:0/270473116 >> 172.21.15.32:0/270473116 conn(0x7fab540a3050 msgr2=0x7fab540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 -- 172.21.15.32:0/270473116 shutdown_connections 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.756+0000 7fab5bec8700 1 -- 172.21.15.32:0/270473116 wait complete. 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.757+0000 7fab5bec8700 1 Processor -- start 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.757+0000 7fab5bec8700 1 -- start start 2023-09-30T22:27:13.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5bec8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab541c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5bec8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 0x7fab541c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5bec8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 0x7fab541c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5bec8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab54121140 con 0x7fab5409d630 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5bec8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab540a5160 con 0x7fab540a0a30 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5bec8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab54120570 con 0x7fab54133930 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 0x7fab541c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab59463700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 0x7fab541c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 0x7fab541c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34110/0 (socket says 172.21.15.32:34110) 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab59463700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 0x7fab541c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54014/0 (socket says 172.21.15.32:54014) 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 -- 172.21.15.32:0/819909451 learned_addr learned my addr 172.21.15.32:0/819909451 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab59c64700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab541c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 -- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 msgr2=0x7fab541c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 0x7fab541c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 -- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 msgr2=0x7fab541c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab541c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab5a465700 1 -- 172.21.15.32:0/819909451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab44009b30 con 0x7fab54133930 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab59c64700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab541c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.758+0000 7fab59463700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 0x7fab541c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.759+0000 7fab5a465700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 0x7fab541c8aa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00eb60 tx=0x7fab4c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.759+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0097a0 con 0x7fab54133930 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.759+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c009e60 con 0x7fab54133930 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.759+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c0094a0 con 0x7fab54133930 2023-09-30T22:27:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.759+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c004240 con 0x7fab54133930 2023-09-30T22:27:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.759+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab480094a0 con 0x7fab54133930 2023-09-30T22:27:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.760+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab4c0043c0 con 0x7fab54133930 2023-09-30T22:27:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.760+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab4c03f350 con 0x7fab54133930 2023-09-30T22:27:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.760+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab54121cb0 con 0x7fab54133930 2023-09-30T22:27:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.765+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab4c020080 con 0x7fab54133930 2023-09-30T22:27:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.933+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd794074ea0 con 0x7fd79409e930 2023-09-30T22:27:13.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.934+0000 7fd790ff9700 1 -- 172.21.15.32:0/832945767 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd78401e020 con 0x7fd79409e930 2023-09-30T22:27:13.935 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 msgr2=0x7fd7941c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 0x7fd7941c5e20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd784010ef0 tx=0x7fd78400b420 comp rx=0 tx=0).stop 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 shutdown_connections 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd79409f650 0x7fd7941c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd79409e930 0x7fd7941c5e20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 --2- 172.21.15.32:0/832945767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd79409f240 0x7fd7941c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 >> 172.21.15.32:0/832945767 conn(0x7fd7940a4210 msgr2=0x7fd7941b9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 shutdown_connections 2023-09-30T22:27:13.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.936+0000 7fd79a995700 1 -- 172.21.15.32:0/832945767 wait complete. 2023-09-30T22:27:13.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:13.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:13.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.969+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fab540747f0 con 0x7fab54133930 2023-09-30T22:27:13.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.970+0000 7fab527fc700 1 -- 172.21.15.32:0/819909451 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab4c017020 con 0x7fab54133930 2023-09-30T22:27:13.971 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 msgr2=0x7fab541c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 0x7fab541c8aa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00eb60 tx=0x7fab4c00ee70 comp rx=0 tx=0).stop 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 shutdown_connections 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab54133930 0x7fab541c8aa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab5409d630 0x7fab541c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 --2- 172.21.15.32:0/819909451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab540a0a30 0x7fab541c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:13.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 >> 172.21.15.32:0/819909451 conn(0x7fab540a3050 msgr2=0x7fab540a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:13.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 shutdown_connections 2023-09-30T22:27:13.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:13.972+0000 7fab5bec8700 1 -- 172.21.15.32:0/819909451 wait complete. 2023-09-30T22:27:13.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:14.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.111+0000 7f8bb8ae6700 1 Processor -- start 2023-09-30T22:27:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.111+0000 7f8bb8ae6700 1 -- start start 2023-09-30T22:27:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.112+0000 7f8bb8ae6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.112+0000 7f8bb8ae6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bb4138d80 0x7f8bb413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.112+0000 7f8bb8ae6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40a2a70 con 0x7f8bb409d630 2023-09-30T22:27:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.112+0000 7f8bb8ae6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40a01d0 con 0x7f8bb4138d80 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb8ae6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb409e2b0 con 0x7f8bb40a0a30 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bb4138d80 0x7f8bb413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8bb409d630 0x7f8bb40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43382/0 (socket says 172.21.15.32:43382) 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb1d9b700 1 -- 172.21.15.32:0/2367063927 learned_addr learned my addr 172.21.15.32:0/2367063927 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 msgr2=0x7f8bb40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb2d9d700 1 --2- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/2367063927 >> v1:172.21.15.32:6789/0 conn(0x7f8bb409d630 legacy=0x7f8bb40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb159a700 1 -- 172.21.15.32:0/2367063927 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513767618 0 0) 0x7f8bb40a2a70 con 0x7f8bb409d630 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/2367063927 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb409dc50 con 0x7f8bb4138d80 2023-09-30T22:27:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb259c700 1 --2- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb2d9d700 1 --2- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb413b160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009d90 tx=0x7f8bac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c17c401a867397 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.113+0000 7f8bb159a700 1 -- 172.21.15.32:0/2367063927 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00e070 con 0x7f8bb4138d80 2023-09-30T22:27:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.114+0000 7f8bb159a700 1 -- 172.21.15.32:0/2367063927 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00bb20 con 0x7f8bb4138d80 2023-09-30T22:27:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.114+0000 7f8bb159a700 1 -- 172.21.15.32:0/2367063927 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac012430 con 0x7f8bb4138d80 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.114+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 msgr2=0x7f8bb413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.114+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb413b160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009d90 tx=0x7f8bac00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.115+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/2367063927 shutdown_connections 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.115+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb413b160 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.115+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/2367063927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.115+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/2367063927 >> 172.21.15.32:0/2367063927 conn(0x7f8bb40a3050 msgr2=0x7f8bb40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.115+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/2367063927 shutdown_connections 2023-09-30T22:27:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.115+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/2367063927 wait complete. 2023-09-30T22:27:14.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.116+0000 7f8bb8ae6700 1 Processor -- start 2023-09-30T22:27:14.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.116+0000 7f8bb8ae6700 1 -- start start 2023-09-30T22:27:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb8ae6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bb409d630 0x7f8bb4133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb8ae6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb4133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb8ae6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb4134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb8ae6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41361b0 con 0x7f8bb4138d80 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb8ae6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4136ed0 con 0x7f8bb40a0a30 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb8ae6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40a4b60 con 0x7f8bb409d630 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb4134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb4133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb4134110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50898/0 (socket says 172.21.15.32:50898) 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bb409d630 0x7f8bb4133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb4133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54020/0 (socket says 172.21.15.32:54020) 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.117+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/1371385421 learned_addr learned my addr 172.21.15.32:0/1371385421 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bb409d630 msgr2=0x7f8bb4133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb2d9d700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bb409d630 0x7f8bb4133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 msgr2=0x7f8bb4133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb2d9d700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb4133bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb2d9d700 1 -- 172.21.15.32:0/1371385421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40094a0 con 0x7f8bb4138d80 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb2d9d700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb4134110 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40a2c00 tx=0x7f8bac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac012430 con 0x7f8bb4138d80 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac020070 con 0x7f8bb4138d80 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac009a70 con 0x7f8bb4138d80 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004280 con 0x7f8bb4138d80 2023-09-30T22:27:14.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.118+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb40a0460 con 0x7f8bb4138d80 2023-09-30T22:27:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.119+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bac0043e0 con 0x7f8bb4138d80 2023-09-30T22:27:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.119+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8bac022070 con 0x7f8bb4138d80 2023-09-30T22:27:14.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.120+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb4137a00 con 0x7f8bb4138d80 2023-09-30T22:27:14.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.129+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bac0263e0 con 0x7f8bb4138d80 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.174+0000 7fae1c185700 1 Processor -- start 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.174+0000 7fae1c185700 1 -- start start 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1c185700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a1510 0x7fae14138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1c185700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae14139250 0x7fae1413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1c185700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1409edd0 con 0x7fae140a0a30 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1c185700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae140a18e0 con 0x7fae14139250 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1c185700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae1409e1f0 con 0x7fae140a1510 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1a722700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae14139250 0x7fae1413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae19720700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a1510 0x7fae14138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae19f21700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fae140a0a30 0x7fae140a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43396/0 (socket says 172.21.15.32:43396) 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae19f21700 1 -- 172.21.15.32:0/3104876426 learned_addr learned my addr 172.21.15.32:0/3104876426 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:14.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1a722700 1 -- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a1510 msgr2=0x7fae14138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1a722700 1 --2- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a1510 0x7fae14138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.175+0000 7fae1a722700 1 -- 172.21.15.32:0/3104876426 >> v1:172.21.15.32:6789/0 conn(0x7fae140a0a30 legacy=0x7fae140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae1a722700 1 -- 172.21.15.32:0/3104876426 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1409dc50 con 0x7fae14139250 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae1a722700 1 --2- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae1413b630 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009870 tx=0x7fae0c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08f2fe46ee8d9e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae18f1f700 1 -- 172.21.15.32:0/3104876426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011830 con 0x7fae14139250 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae18f1f700 1 -- 172.21.15.32:0/3104876426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c011990 con 0x7fae14139250 2023-09-30T22:27:14.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae18f1f700 1 -- 172.21.15.32:0/3104876426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011d20 con 0x7fae14139250 2023-09-30T22:27:14.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae1c185700 1 -- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 msgr2=0x7fae1413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.176+0000 7fae1c185700 1 --2- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae1413b630 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009870 tx=0x7fae0c009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.177+0000 7fae1c185700 1 -- 172.21.15.32:0/3104876426 shutdown_connections 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.177+0000 7fae1c185700 1 --2- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae1413b630 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.177+0000 7fae1c185700 1 --2- 172.21.15.32:0/3104876426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a1510 0x7fae14138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.177+0000 7fae1c185700 1 -- 172.21.15.32:0/3104876426 >> 172.21.15.32:0/3104876426 conn(0x7fae140a6550 msgr2=0x7fae140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.177+0000 7fae1c185700 1 -- 172.21.15.32:0/3104876426 shutdown_connections 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.177+0000 7fae1c185700 1 -- 172.21.15.32:0/3104876426 wait complete. 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 Processor -- start 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 -- start start 2023-09-30T22:27:14.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a0a30 0x7fae141c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae140a1510 0x7fae141c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae14122970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae141be330 con 0x7fae14139250 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae141387b0 con 0x7fae140a1510 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1c185700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae14125e60 con 0x7fae140a0a30 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1a722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae14122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae19720700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae140a1510 0x7fae141c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1a722700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae14122970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50908/0 (socket says 172.21.15.32:50908) 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae1a722700 1 -- 172.21.15.32:0/3623749855 learned_addr learned my addr 172.21.15.32:0/3623749855 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.178+0000 7fae19f21700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a0a30 0x7fae141c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:14.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1a722700 1 -- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a0a30 msgr2=0x7fae141c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1a722700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a0a30 0x7fae141c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1a722700 1 -- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae140a1510 msgr2=0x7fae141c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1a722700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae140a1510 0x7fae141c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1a722700 1 -- 172.21.15.32:0/3623749855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0094a0 con 0x7fae14139250 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae19720700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae140a1510 0x7fae141c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1a722700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae14122970 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0090b0 tx=0x7fae0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c004510 con 0x7fae14139250 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c0036a0 con 0x7fae14139250 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae14126120 con 0x7fae14139250 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c003a50 con 0x7fae14139250 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.179+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae14126b90 con 0x7fae14139250 2023-09-30T22:27:14.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.180+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae0c003bb0 con 0x7fae14139250 2023-09-30T22:27:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.180+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae0c040880 con 0x7fae14139250 2023-09-30T22:27:14.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.180+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14074ea0 con 0x7fae14139250 2023-09-30T22:27:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.185+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae0c015080 con 0x7fae14139250 2023-09-30T22:27:14.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.338+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8bb40747f0 con 0x7f8bb4138d80 2023-09-30T22:27:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.339+0000 7f8ba2ffd700 1 -- 172.21.15.32:0/1371385421 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bac01ab70 con 0x7f8bb4138d80 2023-09-30T22:27:14.340 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 msgr2=0x7f8bb4134110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb4134110 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40a2c00 tx=0x7f8bac004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 shutdown_connections 2023-09-30T22:27:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bb4138d80 0x7f8bb4134110 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bb409d630 0x7f8bb4133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 --2- 172.21.15.32:0/1371385421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bb40a0a30 0x7f8bb4133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 >> 172.21.15.32:0/1371385421 conn(0x7f8bb40a3050 msgr2=0x7f8bb413af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 shutdown_connections 2023-09-30T22:27:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.341+0000 7f8bb8ae6700 1 -- 172.21.15.32:0/1371385421 wait complete. 2023-09-30T22:27:14.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.394+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae140a28e0 con 0x7fae14139250 2023-09-30T22:27:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.395+0000 7fae127fc700 1 -- 172.21.15.32:0/3623749855 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae0c02e970 con 0x7fae14139250 2023-09-30T22:27:14.396 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 msgr2=0x7fae14122970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae14122970 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0090b0 tx=0x7fae0c004750 comp rx=0 tx=0).stop 2023-09-30T22:27:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 shutdown_connections 2023-09-30T22:27:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae14139250 0x7fae14122970 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae140a0a30 0x7fae141c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 --2- 172.21.15.32:0/3623749855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae140a1510 0x7fae141c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 >> 172.21.15.32:0/3623749855 conn(0x7fae140a6550 msgr2=0x7fae140a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 shutdown_connections 2023-09-30T22:27:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:14.397+0000 7fae1c185700 1 -- 172.21.15.32:0/3623749855 wait complete. 2023-09-30T22:27:14.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:14.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:14.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.100+0000 7fd43b527700 1 Processor -- start 2023-09-30T22:27:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd43b527700 1 -- start start 2023-09-30T22:27:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd43b527700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4340a0a30 0x7fd4340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd43b527700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd43413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd43b527700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43409edd0 con 0x7fd4340a1340 2023-09-30T22:27:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd43b527700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43409e1f0 con 0x7fd4340a0a30 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd43b527700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd43409dc50 con 0x7fd434134170 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd4392c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4340a0a30 0x7fd4340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd439ac4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd43413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd438ac2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4340a1340 0x7fd434133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43402/0 (socket says 172.21.15.32:43402) 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.101+0000 7fd438ac2700 1 -- 172.21.15.32:0/3045825254 learned_addr learned my addr 172.21.15.32:0/3045825254 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd4392c3700 1 -- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 msgr2=0x7fd43413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd4392c3700 1 --2- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd43413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd4392c3700 1 -- 172.21.15.32:0/3045825254 >> v1:172.21.15.32:6789/0 conn(0x7fd4340a1340 legacy=0x7fd434133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd433fff700 1 -- 172.21.15.32:0/3045825254 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183306736 0 0) 0x7fd43409edd0 con 0x7fd4340a1340 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd4392c3700 1 -- 172.21.15.32:0/3045825254 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340a1fe0 con 0x7fd4340a0a30 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd439ac4700 1 --2- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd43413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd4392c3700 1 --2- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a0a30 0x7fd4340a0e00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd424009840 tx=0x7fd424009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28d7245fe60a52b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd433fff700 1 -- 172.21.15.32:0/3045825254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424011830 con 0x7fd4340a0a30 2023-09-30T22:27:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd433fff700 1 -- 172.21.15.32:0/3045825254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd424011990 con 0x7fd4340a0a30 2023-09-30T22:27:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.102+0000 7fd433fff700 1 -- 172.21.15.32:0/3045825254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424011d20 con 0x7fd4340a0a30 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 -- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a0a30 msgr2=0x7fd4340a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 --2- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a0a30 0x7fd4340a0e00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd424009840 tx=0x7fd424009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 -- 172.21.15.32:0/3045825254 shutdown_connections 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 --2- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd43413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 --2- 172.21.15.32:0/3045825254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a0a30 0x7fd4340a0e00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 -- 172.21.15.32:0/3045825254 >> 172.21.15.32:0/3045825254 conn(0x7fd4340a6550 msgr2=0x7fd4340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 -- 172.21.15.32:0/3045825254 shutdown_connections 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.103+0000 7fd43b527700 1 -- 172.21.15.32:0/3045825254 wait complete. 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 Processor -- start 2023-09-30T22:27:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 -- start start 2023-09-30T22:27:15.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4340a0a30 0x7fd434122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a1340 0x7fd434122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd4341230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434126190 con 0x7fd4340a1340 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd434125bf0 con 0x7fd4340a0a30 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd43b527700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd434124320 con 0x7fd434134170 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.105+0000 7fd438ac2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a1340 0x7fd434122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4340a0a30 0x7fd434122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd438ac2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a1340 0x7fd434122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50924/0 (socket says 172.21.15.32:50924) 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd438ac2700 1 -- 172.21.15.32:0/138210089 learned_addr learned my addr 172.21.15.32:0/138210089 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd439ac4700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd4341230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 -- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 msgr2=0x7fd4341230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd4341230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 -- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a1340 msgr2=0x7fd434122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a1340 0x7fd434122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 -- 172.21.15.32:0/138210089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c009a70 con 0x7fd4340a0a30 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd439ac4700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd4341230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd4392c3700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4340a0a30 0x7fd434122670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd43409d2f0 tx=0x7fd424004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4240045d0 con 0x7fd4340a0a30 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42401f070 con 0x7fd4340a0a30 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4240038a0 con 0x7fd4340a0a30 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.106+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4240094a0 con 0x7fd4340a0a30 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.107+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4340a25e0 con 0x7fd4340a0a30 2023-09-30T22:27:15.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.107+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd424003a00 con 0x7fd4340a0a30 2023-09-30T22:27:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.108+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4340a0220 con 0x7fd4340a0a30 2023-09-30T22:27:15.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.108+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd424021070 con 0x7fd4340a0a30 2023-09-30T22:27:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.112+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd42402f760 con 0x7fd4340a0a30 2023-09-30T22:27:15.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.139+0000 7ff6913a4700 1 Processor -- start 2023-09-30T22:27:15.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.139+0000 7ff6913a4700 1 -- start start 2023-09-30T22:27:15.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff6913a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c130ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff6913a4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff68c072590 0x7ff68c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff6913a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c09dc30 0x7ff68c09e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff6913a4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68c130ff0 con 0x7ff68c072590 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff6913a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c130290 con 0x7ff68c09dc30 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff6913a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c09bad0 con 0x7ff68c1306e0 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c130ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c09dc30 0x7ff68c09e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c130ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34188/0 (socket says 172.21.15.32:34188) 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 -- 172.21.15.32:0/2317834996 learned_addr learned my addr 172.21.15.32:0/2317834996 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68a7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff68c072590 0x7ff68c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 -- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c09dc30 msgr2=0x7ff68c09e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 --2- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c09dc30 0x7ff68c09e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 -- 172.21.15.32:0/2317834996 >> v2:172.21.15.32:3300/0 conn(0x7ff68c072590 msgr2=0x7ff68c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 --2- 172.21.15.32:0/2317834996 >> v2:172.21.15.32:3300/0 conn(0x7ff68c072590 0x7ff68c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 -- 172.21.15.32:0/2317834996 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c09a8e0 con 0x7ff68c1306e0 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.140+0000 7ff68affd700 1 --2- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c130ab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0098f0 tx=0x7ff67c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bb4daa20e5dcdd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff689ffb700 1 -- 172.21.15.32:0/2317834996 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c011830 con 0x7ff68c1306e0 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff689ffb700 1 -- 172.21.15.32:0/2317834996 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c011990 con 0x7ff68c1306e0 2023-09-30T22:27:15.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff689ffb700 1 -- 172.21.15.32:0/2317834996 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c011d20 con 0x7ff68c1306e0 2023-09-30T22:27:15.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 -- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 msgr2=0x7ff68c130ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 --2- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c130ab0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0098f0 tx=0x7ff67c009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:15.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 -- 172.21.15.32:0/2317834996 shutdown_connections 2023-09-30T22:27:15.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 --2- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c09dc30 0x7ff68c09e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 --2- 172.21.15.32:0/2317834996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c130ab0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 --2- 172.21.15.32:0/2317834996 >> v2:172.21.15.32:3300/0 conn(0x7ff68c072590 0x7ff68c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 -- 172.21.15.32:0/2317834996 >> 172.21.15.32:0/2317834996 conn(0x7ff68c09f470 msgr2=0x7ff68c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 -- 172.21.15.32:0/2317834996 shutdown_connections 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.141+0000 7ff6913a4700 1 -- 172.21.15.32:0/2317834996 wait complete. 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.142+0000 7ff6913a4700 1 Processor -- start 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 -- start start 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c072590 0x7ff68c1288d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 0x7ff68c128e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c129350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c12cc90 con 0x7ff68c09dc30 2023-09-30T22:27:15.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c12c500 con 0x7ff68c072590 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff6913a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c12c8a0 con 0x7ff68c1306e0 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 0x7ff68c128e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c129350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 0x7ff68c128e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50946/0 (socket says 172.21.15.32:50946) 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 -- 172.21.15.32:0/840633195 learned_addr learned my addr 172.21.15.32:0/840633195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68affd700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c072590 0x7ff68c1288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 -- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 msgr2=0x7ff68c129350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c129350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 -- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c072590 msgr2=0x7ff68c1288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c072590 0x7ff68c1288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.143+0000 7ff68a7fc700 1 -- 172.21.15.32:0/840633195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c009550 con 0x7ff68c09dc30 2023-09-30T22:27:15.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.144+0000 7ff68a7fc700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 0x7ff68c128e10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff68000fdb0 tx=0x7ff68000e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.144+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000ed20 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.144+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68000ee80 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.144+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c12bcf0 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.144+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680019670 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.144+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c12a380 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.145+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff680016070 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.145+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff68003feb0 con 0x7ff68c09dc30 2023-09-30T22:27:15.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.145+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c12c0c0 con 0x7ff68c09dc30 2023-09-30T22:27:15.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.150+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff68001e020 con 0x7ff68c09dc30 2023-09-30T22:27:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.322+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd434074ea0 con 0x7fd4340a0a30 2023-09-30T22:27:15.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.322+0000 7fd431ffb700 1 -- 172.21.15.32:0/138210089 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd42402b030 con 0x7fd4340a0a30 2023-09-30T22:27:15.323 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:15.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4340a0a30 msgr2=0x7fd434122670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4340a0a30 0x7fd434122670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd43409d2f0 tx=0x7fd424004750 comp rx=0 tx=0).stop 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 shutdown_connections 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd434134170 0x7fd4341230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd4340a0a30 0x7fd434122670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 --2- 172.21.15.32:0/138210089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4340a1340 0x7fd434122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.324+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 >> 172.21.15.32:0/138210089 conn(0x7fd4340a6550 msgr2=0x7fd43409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.325+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 shutdown_connections 2023-09-30T22:27:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.325+0000 7fd43b527700 1 -- 172.21.15.32:0/138210089 wait complete. 2023-09-30T22:27:15.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:15.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.356+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff68c0747f0 con 0x7ff68c09dc30 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.357+0000 7ff6737fe700 1 -- 172.21.15.32:0/840633195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff68000d350 con 0x7ff68c09dc30 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 msgr2=0x7ff68c128e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 0x7ff68c128e10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff68000fdb0 tx=0x7ff68000e640 comp rx=0 tx=0).stop 2023-09-30T22:27:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 shutdown_connections 2023-09-30T22:27:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff68c1306e0 0x7ff68c129350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff68c072590 0x7ff68c1288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 --2- 172.21.15.32:0/840633195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c09dc30 0x7ff68c128e10 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 >> 172.21.15.32:0/840633195 conn(0x7ff68c09f470 msgr2=0x7ff68c098920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 shutdown_connections 2023-09-30T22:27:15.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.359+0000 7ff6913a4700 1 -- 172.21.15.32:0/840633195 wait complete. 2023-09-30T22:27:15.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:15.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.497+0000 7f2d2910f700 1 Processor -- start 2023-09-30T22:27:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.497+0000 7f2d2910f700 1 -- start start 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2910f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 0x7f2d240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2910f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d2409d630 0x7f2d240a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2910f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d24133930 0x7f2d24133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2910f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d240a2a70 con 0x7f2d24133930 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2910f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240a01d0 con 0x7f2d240a0a30 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2910f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2409e2b0 con 0x7f2d2409d630 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 0x7f2d240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 0x7f2d240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54074/0 (socket says 172.21.15.32:54074) 2023-09-30T22:27:15.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d22d9d700 1 -- 172.21.15.32:0/3043720060 learned_addr learned my addr 172.21.15.32:0/3043720060 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2259c700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d2409d630 0x7f2d240a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2359e700 1 --2- 172.21.15.32:0/3043720060 >> v2:172.21.15.32:3300/0 conn(0x7f2d24133930 0x7f2d24133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2259c700 1 -- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 msgr2=0x7f2d240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2259c700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 0x7f2d240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2259c700 1 -- 172.21.15.32:0/3043720060 >> v2:172.21.15.32:3300/0 conn(0x7f2d24133930 msgr2=0x7f2d24133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2259c700 1 --2- 172.21.15.32:0/3043720060 >> v2:172.21.15.32:3300/0 conn(0x7f2d24133930 0x7f2d24133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d2259c700 1 -- 172.21.15.32:0/3043720060 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2409dc50 con 0x7f2d2409d630 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.498+0000 7f2d22d9d700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 0x7f2d240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2259c700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d2409d630 0x7f2d240a1340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009840 tx=0x7f2d18009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466cefd99a3af035 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d21d9b700 1 -- 172.21.15.32:0/3043720060 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011830 con 0x7f2d2409d630 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d21d9b700 1 -- 172.21.15.32:0/3043720060 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18011990 con 0x7f2d2409d630 2023-09-30T22:27:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d21d9b700 1 -- 172.21.15.32:0/3043720060 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011d20 con 0x7f2d2409d630 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 -- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d2409d630 msgr2=0x7f2d240a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d2409d630 0x7f2d240a1340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009840 tx=0x7f2d18009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 -- 172.21.15.32:0/3043720060 shutdown_connections 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d240a0a30 0x7f2d240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 --2- 172.21.15.32:0/3043720060 >> v2:172.21.15.32:3300/0 conn(0x7f2d24133930 0x7f2d24133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 --2- 172.21.15.32:0/3043720060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d2409d630 0x7f2d240a1340 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 -- 172.21.15.32:0/3043720060 >> 172.21.15.32:0/3043720060 conn(0x7f2d240a3050 msgr2=0x7f2d240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 -- 172.21.15.32:0/3043720060 shutdown_connections 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.499+0000 7f2d2910f700 1 -- 172.21.15.32:0/3043720060 wait complete. 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.501+0000 7f2d2910f700 1 Processor -- start 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.501+0000 7f2d2910f700 1 -- start start 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2910f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d2409d630 0x7f2d241c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2910f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d240a0a30 0x7f2d241c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2910f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 0x7f2d241c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2910f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24120dc0 con 0x7f2d2409d630 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2910f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241211d0 con 0x7f2d24133930 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2910f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24120650 con 0x7f2d240a0a30 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 0x7f2d241c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d22d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d2409d630 0x7f2d241c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 0x7f2d241c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54080/0 (socket says 172.21.15.32:54080) 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2359e700 1 -- 172.21.15.32:0/2966455411 learned_addr learned my addr 172.21.15.32:0/2966455411 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.502+0000 7f2d2259c700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d240a0a30 0x7f2d241c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d22d9d700 1 -- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d240a0a30 msgr2=0x7f2d241c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d22d9d700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d240a0a30 0x7f2d241c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d22d9d700 1 -- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 msgr2=0x7f2d241c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d22d9d700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 0x7f2d241c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d22d9d700 1 -- 172.21.15.32:0/2966455411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d14009550 con 0x7f2d2409d630 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d2259c700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d240a0a30 0x7f2d241c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d2359e700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 0x7f2d241c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d22d9d700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d2409d630 0x7f2d241c8090 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9d0 tx=0x7f2d1400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ee70 con 0x7f2d2409d630 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d14004030 con 0x7f2d2409d630 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d180094a0 con 0x7f2d2409d630 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d140043c0 con 0x7f2d2409d630 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.503+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d240a0410 con 0x7f2d2409d630 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.504+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d1400c850 con 0x7f2d2409d630 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.504+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2d1403f640 con 0x7f2d2409d630 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.504+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d24122060 con 0x7f2d2409d630 2023-09-30T22:27:15.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.509+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d14019d40 con 0x7f2d2409d630 2023-09-30T22:27:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.552+0000 7fa258b20700 1 Processor -- start 2023-09-30T22:27:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.552+0000 7fa258b20700 1 -- start start 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa258b20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 0x7fa2540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa258b20700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2540a1340 0x7fa254133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa258b20700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 0x7fa254134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa258b20700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25409edd0 con 0x7fa254133fa0 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa258b20700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25409e1f0 con 0x7fa2540a0a30 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa258b20700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa25409dc50 con 0x7fa2540a1340 2023-09-30T22:27:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 0x7fa2540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa252d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 0x7fa254134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 0x7fa2540a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54088/0 (socket says 172.21.15.32:54088) 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa252d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 0x7fa254134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50980/0 (socket says 172.21.15.32:50980) 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa251d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2540a1340 0x7fa254133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 -- 172.21.15.32:0/818238417 learned_addr learned my addr 172.21.15.32:0/818238417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 -- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2540a1340 msgr2=0x7fa254133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 --2- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2540a1340 0x7fa254133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 -- 172.21.15.32:0/818238417 >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 msgr2=0x7fa254134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 --2- 172.21.15.32:0/818238417 >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 0x7fa254134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa25259c700 1 -- 172.21.15.32:0/818238417 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540a1fe0 con 0x7fa2540a0a30 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa251d9b700 1 --2- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2540a1340 0x7fa254133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.553+0000 7fa252d9d700 1 --2- 172.21.15.32:0/818238417 >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 0x7fa254134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa25259c700 1 --2- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 0x7fa2540a0e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098f0 tx=0x7fa244009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8534aa2274828dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa25159a700 1 -- 172.21.15.32:0/818238417 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244011830 con 0x7fa2540a0a30 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa25159a700 1 -- 172.21.15.32:0/818238417 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa244011990 con 0x7fa2540a0a30 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa25159a700 1 -- 172.21.15.32:0/818238417 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244011d20 con 0x7fa2540a0a30 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa258b20700 1 -- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 msgr2=0x7fa2540a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa258b20700 1 --2- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 0x7fa2540a0e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098f0 tx=0x7fa244009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa258b20700 1 -- 172.21.15.32:0/818238417 shutdown_connections 2023-09-30T22:27:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa258b20700 1 --2- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a0a30 0x7fa2540a0e00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa258b20700 1 --2- 172.21.15.32:0/818238417 >> v2:172.21.15.32:3300/0 conn(0x7fa254133fa0 0x7fa254134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.554+0000 7fa258b20700 1 --2- 172.21.15.32:0/818238417 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2540a1340 0x7fa254133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.555+0000 7fa258b20700 1 -- 172.21.15.32:0/818238417 >> 172.21.15.32:0/818238417 conn(0x7fa2540a6550 msgr2=0x7fa2540a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.555+0000 7fa258b20700 1 -- 172.21.15.32:0/818238417 shutdown_connections 2023-09-30T22:27:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.555+0000 7fa258b20700 1 -- 172.21.15.32:0/818238417 wait complete. 2023-09-30T22:27:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 Processor -- start 2023-09-30T22:27:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 -- start start 2023-09-30T22:27:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2540a0a30 0x7fa254120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a1340 0x7fa254120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 0x7fa254120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa254121d40 con 0x7fa2540a0a30 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa254122f90 con 0x7fa2540a1340 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa258b20700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa254122330 con 0x7fa254133fa0 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa251d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a1340 0x7fa254120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa252d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 0x7fa254120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa251d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a1340 0x7fa254120670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54098/0 (socket says 172.21.15.32:54098) 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa252d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 0x7fa254120bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34240/0 (socket says 172.21.15.32:34240) 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.556+0000 7fa251d9b700 1 -- 172.21.15.32:0/806507079 learned_addr learned my addr 172.21.15.32:0/806507079 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa25259c700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2540a0a30 0x7fa254120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa252d9d700 1 -- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a1340 msgr2=0x7fa254120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa252d9d700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a1340 0x7fa254120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa252d9d700 1 -- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2540a0a30 msgr2=0x7fa254120130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa252d9d700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2540a0a30 0x7fa254120130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa252d9d700 1 -- 172.21.15.32:0/806507079 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0094a0 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa252d9d700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 0x7fa254120bb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009e50 tx=0x7fa24c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00ca90 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c00f040 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c009a70 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00ce50 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.557+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa244009550 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.558+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa24c014480 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.558+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa24c03fbd0 con 0x7fa254133fa0 2023-09-30T22:27:15.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.558+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2540a0220 con 0x7fa254133fa0 2023-09-30T22:27:15.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.563+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa24c012930 con 0x7fa254133fa0 2023-09-30T22:27:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.716+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2d240747f0 con 0x7f2d2409d630 2023-09-30T22:27:15.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.716+0000 7f2d0b7fe700 1 -- 172.21.15.32:0/2966455411 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d14019300 con 0x7f2d2409d630 2023-09-30T22:27:15.719 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d2409d630 msgr2=0x7f2d241c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d2409d630 0x7f2d241c8090 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9d0 tx=0x7f2d1400ece0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 shutdown_connections 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d24133930 0x7f2d241c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d2409d630 0x7f2d241c8090 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 --2- 172.21.15.32:0/2966455411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d240a0a30 0x7f2d241c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 >> 172.21.15.32:0/2966455411 conn(0x7f2d240a3050 msgr2=0x7f2d241bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 shutdown_connections 2023-09-30T22:27:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.719+0000 7f2d2910f700 1 -- 172.21.15.32:0/2966455411 wait complete. 2023-09-30T22:27:15.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.768+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa254074ea0 con 0x7fa254133fa0 2023-09-30T22:27:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.769+0000 7fa24affd700 1 -- 172.21.15.32:0/806507079 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa24c012340 con 0x7fa254133fa0 2023-09-30T22:27:15.770 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 msgr2=0x7fa254120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 0x7fa254120bb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009e50 tx=0x7fa24c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 shutdown_connections 2023-09-30T22:27:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa254133fa0 0x7fa254120bb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2540a0a30 0x7fa254120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 --2- 172.21.15.32:0/806507079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2540a1340 0x7fa254120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 >> 172.21.15.32:0/806507079 conn(0x7fa2540a6550 msgr2=0x7fa2541bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 shutdown_connections 2023-09-30T22:27:15.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:15.771+0000 7fa258b20700 1 -- 172.21.15.32:0/806507079 wait complete. 2023-09-30T22:27:15.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:16.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:16.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:16.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.488+0000 7effcc681700 1 Processor -- start 2023-09-30T22:27:16.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.488+0000 7effcc681700 1 -- start start 2023-09-30T22:27:16.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effcc681700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc40a0a30 0x7effc40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effcc681700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effc409d630 0x7effc40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effcc681700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc40a2a70 con 0x7effc4133930 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effcc681700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc40a01d0 con 0x7effc409d630 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effcc681700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effc409e2b0 con 0x7effc40a0a30 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effc409d630 0x7effc40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effca41d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc40a0a30 0x7effc40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effc409d630 0x7effc40a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50998/0 (socket says 172.21.15.32:50998) 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 -- 172.21.15.32:0/1901283288 learned_addr learned my addr 172.21.15.32:0/1901283288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effcac1e700 1 --1- 172.21.15.32:0/1901283288 >> v1:172.21.15.32:6789/0 conn(0x7effc4133930 0x7effc4133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43414/0 (socket says 172.21.15.32:43414) 2023-09-30T22:27:16.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 -- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc40a0a30 msgr2=0x7effc40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 --2- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc40a0a30 0x7effc40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 -- 172.21.15.32:0/1901283288 >> v1:172.21.15.32:6789/0 conn(0x7effc4133930 legacy=0x7effc4133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc9c1c700 1 -- 172.21.15.32:0/1901283288 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc409dc50 con 0x7effc409d630 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effca41d700 1 --2- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc40a0a30 0x7effc40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.489+0000 7effc941b700 1 -- 172.21.15.32:0/1901283288 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429974351 0 0) 0x7effc40a2a70 con 0x7effc4133930 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effc9c1c700 1 --2- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc40a1340 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7effb40098f0 tx=0x7effb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fde21db0192052 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effc941b700 1 -- 172.21.15.32:0/1901283288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4011830 con 0x7effc409d630 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effc941b700 1 -- 172.21.15.32:0/1901283288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb4011990 con 0x7effc409d630 2023-09-30T22:27:16.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effc941b700 1 -- 172.21.15.32:0/1901283288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4011d20 con 0x7effc409d630 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effcc681700 1 -- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 msgr2=0x7effc40a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effcc681700 1 --2- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc40a1340 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7effb40098f0 tx=0x7effb4009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effcc681700 1 -- 172.21.15.32:0/1901283288 shutdown_connections 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effcc681700 1 --2- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc40a0a30 0x7effc40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effcc681700 1 --2- 172.21.15.32:0/1901283288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc40a1340 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.490+0000 7effcc681700 1 -- 172.21.15.32:0/1901283288 >> 172.21.15.32:0/1901283288 conn(0x7effc40a3050 msgr2=0x7effc40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.491+0000 7effcc681700 1 -- 172.21.15.32:0/1901283288 shutdown_connections 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.491+0000 7effcc681700 1 -- 172.21.15.32:0/1901283288 wait complete. 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 Processor -- start 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 -- start start 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc41c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 0x7effc41c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc4133930 0x7effc41c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41210e0 con 0x7effc409d630 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc41207f0 con 0x7effc40a0a30 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effcc681700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effc41209e0 con 0x7effc4133930 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effc9c1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 0x7effc41c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effca41d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc41c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effc9c1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 0x7effc41c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54100/0 (socket says 172.21.15.32:54100) 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.492+0000 7effc9c1c700 1 -- 172.21.15.32:0/3064339739 learned_addr learned my addr 172.21.15.32:0/3064339739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effc9c1c700 1 -- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc4133930 msgr2=0x7effc41c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effc9c1c700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc4133930 0x7effc41c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effc9c1c700 1 -- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 msgr2=0x7effc41c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effc9c1c700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc41c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effc9c1c700 1 -- 172.21.15.32:0/3064339739 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc0009a70 con 0x7effc40a0a30 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effca41d700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc41c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effc9c1c700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 0x7effc41c85f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7effb4009c40 tx=0x7effb4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb40045d0 con 0x7effc40a0a30 2023-09-30T22:27:16.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40036a0 con 0x7effc40a0a30 2023-09-30T22:27:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb4009550 con 0x7effc40a0a30 2023-09-30T22:27:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4003a30 con 0x7effc40a0a30 2023-09-30T22:27:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.493+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc40a0410 con 0x7effc40a0a30 2023-09-30T22:27:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.494+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7effb4021070 con 0x7effc40a0a30 2023-09-30T22:27:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.494+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7effb4040ba0 con 0x7effc40a0a30 2023-09-30T22:27:16.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.494+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc4122380 con 0x7effc40a0a30 2023-09-30T22:27:16.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.499+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7effb4029080 con 0x7effc40a0a30 2023-09-30T22:27:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.535+0000 7f1ad32ea700 1 Processor -- start 2023-09-30T22:27:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.535+0000 7f1ad32ea700 1 -- start start 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad32ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 0x7f1acc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad32ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad32ea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1acc09e410 0x7f1acc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad32ea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc06b460 con 0x7f1acc09e410 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad32ea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc09b120 con 0x7f1acc09d6f0 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad32ea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc09a7e0 con 0x7f1acc09e000 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1086700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 0x7f1acc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1887700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1acc09e410 0x7f1acc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad0885700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1086700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 0x7f1acc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54116/0 (socket says 172.21.15.32:54116) 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1086700 1 -- 172.21.15.32:0/866245241 learned_addr learned my addr 172.21.15.32:0/866245241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1887700 1 -- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 msgr2=0x7f1acc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1887700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1887700 1 -- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 msgr2=0x7f1acc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1887700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 0x7f1acc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1887700 1 -- 172.21.15.32:0/866245241 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc09a500 con 0x7f1acc09e410 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad0885700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.536+0000 7f1ad1086700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 0x7f1acc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.537+0000 7f1ad1887700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09e410 0x7f1acc133f30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009d90 tx=0x7f1ac400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af93e56770905b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.537+0000 7f1ac3fff700 1 -- 172.21.15.32:0/866245241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400e070 con 0x7f1acc09e410 2023-09-30T22:27:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.537+0000 7f1ac3fff700 1 -- 172.21.15.32:0/866245241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac400bb20 con 0x7f1acc09e410 2023-09-30T22:27:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.537+0000 7f1ac3fff700 1 -- 172.21.15.32:0/866245241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4012430 con 0x7f1acc09e410 2023-09-30T22:27:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.537+0000 7f1ad32ea700 1 -- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09e410 msgr2=0x7f1acc133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.537+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09e410 0x7f1acc133f30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009d90 tx=0x7f1ac400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 -- 172.21.15.32:0/866245241 shutdown_connections 2023-09-30T22:27:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09d6f0 0x7f1acc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09e410 0x7f1acc133f30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/866245241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 -- 172.21.15.32:0/866245241 >> 172.21.15.32:0/866245241 conn(0x7f1acc066f50 msgr2=0x7f1acc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 -- 172.21.15.32:0/866245241 shutdown_connections 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.538+0000 7f1ad32ea700 1 -- 172.21.15.32:0/866245241 wait complete. 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.539+0000 7f1ad32ea700 1 Processor -- start 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.539+0000 7f1ad32ea700 1 -- start start 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad32ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09d6f0 0x7f1acc1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad32ea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad32ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 0x7f1acc1a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad32ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1a98d0 con 0x7f1acc09d6f0 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad32ea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1a9060 con 0x7f1acc09e410 2023-09-30T22:27:16.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad32ea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0a1f00 con 0x7f1acc09e000 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad1887700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 0x7f1acc1a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad0885700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad0885700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34284/0 (socket says 172.21.15.32:34284) 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad0885700 1 -- 172.21.15.32:0/1294610566 learned_addr learned my addr 172.21.15.32:0/1294610566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad1887700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 0x7f1acc1a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54122/0 (socket says 172.21.15.32:54122) 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.540+0000 7f1ad1086700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09d6f0 0x7f1acc1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1887700 1 -- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 msgr2=0x7f1acc1a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1887700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1887700 1 -- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09d6f0 msgr2=0x7f1acc1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1887700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09d6f0 0x7f1acc1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1887700 1 -- 172.21.15.32:0/1294610566 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8009550 con 0x7f1acc09e410 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad0885700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1086700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09d6f0 0x7f1acc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad1887700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 0x7f1acc1a6830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1acc09e8b0 tx=0x7f1ac4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4004880 con 0x7f1acc09e410 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac4020070 con 0x7f1acc09e410 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc0094a0 con 0x7f1acc09e410 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40276c0 con 0x7f1acc09e410 2023-09-30T22:27:16.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.541+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac4009a70 con 0x7f1acc09e410 2023-09-30T22:27:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.542+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ac4012430 con 0x7f1acc09e410 2023-09-30T22:27:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.542+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1ac4022070 con 0x7f1acc09e410 2023-09-30T22:27:16.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.543+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc074ea0 con 0x7f1acc09e410 2023-09-30T22:27:16.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.548+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ac401b400 con 0x7f1acc09e410 2023-09-30T22:27:16.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.717+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7effc404f2a0 con 0x7effc40a0a30 2023-09-30T22:27:16.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.718+0000 7effbaffd700 1 -- 172.21.15.32:0/3064339739 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7effb4034090 con 0x7effc40a0a30 2023-09-30T22:27:16.719 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 msgr2=0x7effc41c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 0x7effc41c85f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7effb4009c40 tx=0x7effb4004750 comp rx=0 tx=0).stop 2023-09-30T22:27:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 shutdown_connections 2023-09-30T22:27:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effc4133930 0x7effc41c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effc409d630 0x7effc41c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 --2- 172.21.15.32:0/3064339739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effc40a0a30 0x7effc41c85f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 >> 172.21.15.32:0/3064339739 conn(0x7effc40a3050 msgr2=0x7effc409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 shutdown_connections 2023-09-30T22:27:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.720+0000 7effcc681700 1 -- 172.21.15.32:0/3064339739 wait complete. 2023-09-30T22:27:16.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:16.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.754+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1acc1a79c0 con 0x7f1acc09e410 2023-09-30T22:27:16.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.755+0000 7f1ac1ffb700 1 -- 172.21.15.32:0/1294610566 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ac402f3a0 con 0x7f1acc09e410 2023-09-30T22:27:16.755 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 msgr2=0x7f1acc1a6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 0x7f1acc1a6830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1acc09e8b0 tx=0x7f1ac4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 shutdown_connections 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1acc09e410 0x7f1acc1a6830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1acc09d6f0 0x7f1acc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 --2- 172.21.15.32:0/1294610566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1acc09e000 0x7f1acc1a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 >> 172.21.15.32:0/1294610566 conn(0x7f1acc066f50 msgr2=0x7f1acc1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 shutdown_connections 2023-09-30T22:27:16.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.757+0000 7f1ad32ea700 1 -- 172.21.15.32:0/1294610566 wait complete. 2023-09-30T22:27:16.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:16.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.875+0000 7f7f4b58c700 1 Processor -- start 2023-09-30T22:27:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.875+0000 7f7f4b58c700 1 -- start start 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f4b58c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 0x7f7f440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f4b58c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f440a1340 0x7f7f44133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f4b58c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4409edd0 con 0x7f7f44133fa0 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f4b58c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4409e1f0 con 0x7f7f440a1340 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f4b58c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4409dc50 con 0x7f7f440a0a30 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f48b27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f440a1340 0x7f7f44133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f49328700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 0x7f7f440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f49328700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 0x7f7f440a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34290/0 (socket says 172.21.15.32:34290) 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f48b27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f440a1340 0x7f7f44133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51026/0 (socket says 172.21.15.32:51026) 2023-09-30T22:27:16.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f49b29700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f44133fa0 0x7f7f4413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43424/0 (socket says 172.21.15.32:43424) 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.876+0000 7f7f49328700 1 -- 172.21.15.32:0/3534342172 learned_addr learned my addr 172.21.15.32:0/3534342172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f43fff700 1 -- 172.21.15.32:0/3534342172 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259408896 0 0) 0x7f7f4409edd0 con 0x7f7f44133fa0 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f43fff700 1 -- 172.21.15.32:0/3534342172 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2c0035b0 con 0x7f7f44133fa0 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f48b27700 1 -- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 msgr2=0x7f7f440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f48b27700 1 --2- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 0x7f7f440a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f48b27700 1 -- 172.21.15.32:0/3534342172 >> v1:172.21.15.32:6789/0 conn(0x7f7f44133fa0 legacy=0x7f7f4413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f48b27700 1 -- 172.21.15.32:0/3534342172 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440a1fe0 con 0x7f7f440a1340 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f49328700 1 --2- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 0x7f7f440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f48b27700 1 --2- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f440a1340 0x7f7f44133a60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009840 tx=0x7f7f38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85e2e1bfcd578fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f43fff700 1 -- 172.21.15.32:0/3534342172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38011830 con 0x7f7f440a1340 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f43fff700 1 -- 172.21.15.32:0/3534342172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f38011990 con 0x7f7f440a1340 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.877+0000 7f7f43fff700 1 -- 172.21.15.32:0/3534342172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38011d20 con 0x7f7f440a1340 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 -- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f440a1340 msgr2=0x7f7f44133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f440a1340 0x7f7f44133a60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009840 tx=0x7f7f38009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 -- 172.21.15.32:0/3534342172 shutdown_connections 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a0a30 0x7f7f440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/3534342172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f440a1340 0x7f7f44133a60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 -- 172.21.15.32:0/3534342172 >> 172.21.15.32:0/3534342172 conn(0x7f7f440a6550 msgr2=0x7f7f440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 -- 172.21.15.32:0/3534342172 shutdown_connections 2023-09-30T22:27:16.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.878+0000 7f7f4b58c700 1 -- 172.21.15.32:0/3534342172 wait complete. 2023-09-30T22:27:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 Processor -- start 2023-09-30T22:27:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 -- start start 2023-09-30T22:27:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f440a0a30 0x7f7f441232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 0x7f7f44123820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f44133fa0 0x7f7f441c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44125fd0 con 0x7f7f44133fa0 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44124e60 con 0x7f7f440a0a30 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.879+0000 7f7f4b58c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44122ef0 con 0x7f7f440a1340 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 0x7f7f44123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 0x7f7f44123820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34292/0 (socket says 172.21.15.32:34292) 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 -- 172.21.15.32:0/1142178673 learned_addr learned my addr 172.21.15.32:0/1142178673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f49328700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f440a0a30 0x7f7f441232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f49b29700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f44133fa0 0x7f7f441c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 -- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f440a0a30 msgr2=0x7f7f441232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f440a0a30 0x7f7f441232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 -- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f44133fa0 msgr2=0x7f7f441c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f44133fa0 0x7f7f441c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 -- 172.21.15.32:0/1142178673 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34009b30 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f48b27700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 0x7f7f44123820 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7f380090b0 tx=0x7f7f38004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004380 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3801f070 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380036a0 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.880+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f380094a0 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.881+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f441245d0 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.882+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f38021030 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.882+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f44125e20 con 0x7f7f440a1340 2023-09-30T22:27:16.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.882+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f380406e0 con 0x7f7f440a1340 2023-09-30T22:27:16.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.887+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f380239c0 con 0x7f7f440a1340 2023-09-30T22:27:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.952+0000 7ff7ad29b700 1 Processor -- start 2023-09-30T22:27:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.952+0000 7ff7ad29b700 1 -- start start 2023-09-30T22:27:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7ad29b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a0a30 0x7ff7a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7ad29b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7a8134170 0x7ff7a813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7ad29b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a809edd0 con 0x7ff7a80a1340 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7ad29b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a809e1f0 con 0x7ff7a8134170 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7ad29b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a809dc50 con 0x7ff7a80a0a30 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7a77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7a8134170 0x7ff7a813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7a67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7a80a1340 0x7ff7a8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43440/0 (socket says 172.21.15.32:43440) 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7a67fc700 1 -- 172.21.15.32:0/715933944 learned_addr learned my addr 172.21.15.32:0/715933944 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7a6ffd700 1 --2- 172.21.15.32:0/715933944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a0a30 0x7ff7a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7a6ffd700 1 -- 172.21.15.32:0/715933944 >> v2:172.21.15.32:3300/0 conn(0x7ff7a8134170 msgr2=0x7ff7a813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:16.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.953+0000 7ff7a6ffd700 1 --2- 172.21.15.32:0/715933944 >> v2:172.21.15.32:3300/0 conn(0x7ff7a8134170 0x7ff7a813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a6ffd700 1 -- 172.21.15.32:0/715933944 >> v1:172.21.15.32:6789/0 conn(0x7ff7a80a1340 legacy=0x7ff7a8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a6ffd700 1 -- 172.21.15.32:0/715933944 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a80a1fe0 con 0x7ff7a80a0a30 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a5ffb700 1 -- 172.21.15.32:0/715933944 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151312822 0 0) 0x7ff7a809edd0 con 0x7ff7a80a1340 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a77fe700 1 --2- 172.21.15.32:0/715933944 >> v2:172.21.15.32:3300/0 conn(0x7ff7a8134170 0x7ff7a813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a6ffd700 1 --2- 172.21.15.32:0/715933944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a0a30 0x7ff7a80a0e00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5073eed714bc6845 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a5ffb700 1 -- 172.21.15.32:0/715933944 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800e070 con 0x7ff7a80a0a30 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a5ffb700 1 -- 172.21.15.32:0/715933944 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79800bb20 con 0x7ff7a80a0a30 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7a5ffb700 1 -- 172.21.15.32:0/715933944 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798012430 con 0x7ff7a80a0a30 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7ad29b700 1 -- 172.21.15.32:0/715933944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a0a30 msgr2=0x7ff7a80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.954+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/715933944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a0a30 0x7ff7a80a0e00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.955+0000 7ff7ad29b700 1 -- 172.21.15.32:0/715933944 shutdown_connections 2023-09-30T22:27:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.955+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/715933944 >> v2:172.21.15.32:3300/0 conn(0x7ff7a8134170 0x7ff7a813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.955+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/715933944 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a0a30 0x7ff7a80a0e00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.955+0000 7ff7ad29b700 1 -- 172.21.15.32:0/715933944 >> 172.21.15.32:0/715933944 conn(0x7ff7a80a6550 msgr2=0x7ff7a80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.955+0000 7ff7ad29b700 1 -- 172.21.15.32:0/715933944 shutdown_connections 2023-09-30T22:27:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.955+0000 7ff7ad29b700 1 -- 172.21.15.32:0/715933944 wait complete. 2023-09-30T22:27:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.956+0000 7ff7ad29b700 1 Processor -- start 2023-09-30T22:27:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.956+0000 7ff7ad29b700 1 -- start start 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7ad29b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 0x7ff7a8123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7ad29b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a1340 0x7ff7a8123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7ad29b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7a8134170 0x7ff7a81c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7ad29b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a8125ef0 con 0x7ff7a8134170 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7ad29b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a8126120 con 0x7ff7a80a0a30 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7ad29b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81244f0 con 0x7ff7a80a1340 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 0x7ff7a8123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 0x7ff7a8123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54136/0 (socket says 172.21.15.32:54136) 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7a6ffd700 1 -- 172.21.15.32:0/2394062736 learned_addr learned my addr 172.21.15.32:0/2394062736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7a77fe700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7a8134170 0x7ff7a81c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.957+0000 7ff7a67fc700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a1340 0x7ff7a8123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a77fe700 1 -- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a1340 msgr2=0x7ff7a8123780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a77fe700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a1340 0x7ff7a8123780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a77fe700 1 -- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 msgr2=0x7ff7a8123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a77fe700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 0x7ff7a8123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a77fe700 1 -- 172.21.15.32:0/2394062736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00094a0 con 0x7ff7a8134170 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a6ffd700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 0x7ff7a8123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7a77fe700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7a8134170 0x7ff7a81c8480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e910 tx=0x7ff7a000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0004020 con 0x7ff7a8134170 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a00041a0 con 0x7ff7a8134170 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff798009a70 con 0x7ff7a8134170 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0004550 con 0x7ff7a8134170 2023-09-30T22:27:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.958+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a8124ce0 con 0x7ff7a8134170 2023-09-30T22:27:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.959+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a80a1130 con 0x7ff7a8134170 2023-09-30T22:27:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.959+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7a00036a0 con 0x7ff7a8134170 2023-09-30T22:27:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.960+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff7a003f740 con 0x7ff7a8134170 2023-09-30T22:27:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:16.964+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7a0013680 con 0x7ff7a8134170 2023-09-30T22:27:17.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.096+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f44124b80 con 0x7f7f440a1340 2023-09-30T22:27:17.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.097+0000 7f7f41ffb700 1 -- 172.21.15.32:0/1142178673 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f38032070 con 0x7f7f440a1340 2023-09-30T22:27:17.098 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 msgr2=0x7f7f44123820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 0x7f7f44123820 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7f380090b0 tx=0x7f7f38004750 comp rx=0 tx=0).stop 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 shutdown_connections 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f44133fa0 0x7f7f441c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f440a0a30 0x7f7f441232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 --2- 172.21.15.32:0/1142178673 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f440a1340 0x7f7f44123820 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 >> 172.21.15.32:0/1142178673 conn(0x7f7f440a6550 msgr2=0x7f7f4411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:17.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 shutdown_connections 2023-09-30T22:27:17.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.100+0000 7f7f4b58c700 1 -- 172.21.15.32:0/1142178673 wait complete. 2023-09-30T22:27:17.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.179+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff7a8074ea0 con 0x7ff7a8134170 2023-09-30T22:27:17.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.179+0000 7ff78f7fe700 1 -- 172.21.15.32:0/2394062736 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7a0017070 con 0x7ff7a8134170 2023-09-30T22:27:17.180 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7a8134170 msgr2=0x7ff7a81c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7a8134170 0x7ff7a81c8480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e910 tx=0x7ff7a000ec20 comp rx=0 tx=0).stop 2023-09-30T22:27:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 shutdown_connections 2023-09-30T22:27:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7a8134170 0x7ff7a81c8480 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7a80a0a30 0x7ff7a8123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 --2- 172.21.15.32:0/2394062736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff7a80a1340 0x7ff7a8123780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 >> 172.21.15.32:0/2394062736 conn(0x7ff7a80a6550 msgr2=0x7ff7a809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 shutdown_connections 2023-09-30T22:27:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.181+0000 7ff7ad29b700 1 -- 172.21.15.32:0/2394062736 wait complete. 2023-09-30T22:27:17.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:17.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:17.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.884+0000 7f5d319fe700 1 Processor -- start 2023-09-30T22:27:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.884+0000 7f5d319fe700 1 -- start start 2023-09-30T22:27:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d319fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d319fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d319fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d2c133930 0x7f5d2c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d319fe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c0a2a70 con 0x7f5d2c133930 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d319fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0a01d0 con 0x7f5d2c0a0a30 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d319fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c09e2b0 con 0x7f5d2c09d630 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d2c133930 0x7f5d2c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54142/0 (socket says 172.21.15.32:54142) 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d2c133930 0x7f5d2c133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51064/0 (socket says 172.21.15.32:51064) 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2affd700 1 -- 172.21.15.32:0/182512594 learned_addr learned my addr 172.21.15.32:0/182512594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2b7fe700 1 -- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 msgr2=0x7f5d2c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2b7fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2b7fe700 1 -- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 msgr2=0x7f5d2c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:17.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.885+0000 7f5d2b7fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d2b7fe700 1 -- 172.21.15.32:0/182512594 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c09dc50 con 0x7f5d2c133930 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d2affd700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d2a7fc700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d2b7fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c133dd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009840 tx=0x7f5d24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa2324a24a011bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d29ffb700 1 -- 172.21.15.32:0/182512594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24011830 con 0x7f5d2c133930 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d29ffb700 1 -- 172.21.15.32:0/182512594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d24011990 con 0x7f5d2c133930 2023-09-30T22:27:17.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.886+0000 7f5d29ffb700 1 -- 172.21.15.32:0/182512594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24011d20 con 0x7f5d2c133930 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 -- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 msgr2=0x7f5d2c133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c133dd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009840 tx=0x7f5d24009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 -- 172.21.15.32:0/182512594 shutdown_connections 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c133dd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 --2- 172.21.15.32:0/182512594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 -- 172.21.15.32:0/182512594 >> 172.21.15.32:0/182512594 conn(0x7f5d2c0a3050 msgr2=0x7f5d2c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 -- 172.21.15.32:0/182512594 shutdown_connections 2023-09-30T22:27:17.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.887+0000 7f5d319fe700 1 -- 172.21.15.32:0/182512594 wait complete. 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 Processor -- start 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 -- start start 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c1c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c121ed0 con 0x7f5d2c133930 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0a4030 con 0x7f5d2c0a0a30 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.888+0000 7f5d319fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c120670 con 0x7f5d2c09d630 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c1c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54154/0 (socket says 172.21.15.32:54154) 2023-09-30T22:27:17.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2a7fc700 1 -- 172.21.15.32:0/3374023456 learned_addr learned my addr 172.21.15.32:0/3374023456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2affd700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c1c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 -- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 msgr2=0x7f5d2c1c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c1c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 -- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 msgr2=0x7f5d2c1c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c1c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 -- 172.21.15.32:0/3374023456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200094a0 con 0x7f5d2c133930 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2a7fc700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2affd700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c1c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d2b7fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c1c8990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5d24016000 tx=0x7f5d24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24004450 con 0x7f5d2c133930 2023-09-30T22:27:17.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d240045d0 con 0x7f5d2c133930 2023-09-30T22:27:17.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c009b30 con 0x7f5d2c133930 2023-09-30T22:27:17.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d240038e0 con 0x7f5d2c133930 2023-09-30T22:27:17.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.889+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d240094a0 con 0x7f5d2c133930 2023-09-30T22:27:17.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.890+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d24003a40 con 0x7f5d2c133930 2023-09-30T22:27:17.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.890+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5d24040160 con 0x7f5d2c133930 2023-09-30T22:27:17.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.890+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c0747f0 con 0x7f5d2c133930 2023-09-30T22:27:17.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.895+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d2401a420 con 0x7f5d2c133930 2023-09-30T22:27:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.922+0000 7f473e705700 1 Processor -- start 2023-09-30T22:27:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.922+0000 7f473e705700 1 -- start start 2023-09-30T22:27:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.922+0000 7f473e705700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47380a0a30 0x7f47380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.922+0000 7f473e705700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a1340 0x7f4738133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f473e705700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473809edd0 con 0x7f4738133fa0 2023-09-30T22:27:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f473e705700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473809e1f0 con 0x7f47380a1340 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f473e705700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f473809dc50 con 0x7f47380a0a30 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f4737fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47380a0a30 0x7f47380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f473cca2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4738133fa0 0x7f473813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43454/0 (socket says 172.21.15.32:43454) 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f473cca2700 1 -- 172.21.15.32:0/526409421 learned_addr learned my addr 172.21.15.32:0/526409421 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f47377fe700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a1340 0x7f4738133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f4736ffd700 1 -- 172.21.15.32:0/526409421 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756407294 0 0) 0x7f473809edd0 con 0x7f4738133fa0 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f47377fe700 1 -- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47380a0a30 msgr2=0x7f47380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:17.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f47377fe700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47380a0a30 0x7f47380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f47377fe700 1 -- 172.21.15.32:0/526409421 >> v1:172.21.15.32:6789/0 conn(0x7f4738133fa0 legacy=0x7f473813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f47377fe700 1 -- 172.21.15.32:0/526409421 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380a1fe0 con 0x7f47380a1340 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f4737fff700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47380a0a30 0x7f47380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f47377fe700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a1340 0x7f4738133a60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f472c009870 tx=0x7f472c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b0acd3f984c3e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.923+0000 7f4736ffd700 1 -- 172.21.15.32:0/526409421 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c011830 con 0x7f47380a1340 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f4736ffd700 1 -- 172.21.15.32:0/526409421 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c011990 con 0x7f47380a1340 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f4736ffd700 1 -- 172.21.15.32:0/526409421 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c011d20 con 0x7f47380a1340 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 -- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a1340 msgr2=0x7f4738133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a1340 0x7f4738133a60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f472c009870 tx=0x7f472c009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 -- 172.21.15.32:0/526409421 shutdown_connections 2023-09-30T22:27:17.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47380a0a30 0x7f47380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 --2- 172.21.15.32:0/526409421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a1340 0x7f4738133a60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 -- 172.21.15.32:0/526409421 >> 172.21.15.32:0/526409421 conn(0x7f47380a6550 msgr2=0x7f47380a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 -- 172.21.15.32:0/526409421 shutdown_connections 2023-09-30T22:27:17.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.924+0000 7f473e705700 1 -- 172.21.15.32:0/526409421 wait complete. 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.925+0000 7f473e705700 1 Processor -- start 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.925+0000 7f473e705700 1 -- start start 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473e705700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a0a30 0x7f47381c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473e705700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 0x7f47381c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473e705700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4738133fa0 0x7f47381c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473e705700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380a0220 con 0x7f47380a1340 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473e705700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4738125170 con 0x7f47380a0a30 2023-09-30T22:27:17.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473e705700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4738124ab0 con 0x7f4738133fa0 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 0x7f47381c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 0x7f47381c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51086/0 (socket says 172.21.15.32:51086) 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f4737fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a0a30 0x7f47381c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 -- 172.21.15.32:0/3018504241 learned_addr learned my addr 172.21.15.32:0/3018504241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f473cca2700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4738133fa0 0x7f47381c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 -- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4738133fa0 msgr2=0x7f47381c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4738133fa0 0x7f47381c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 -- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a0a30 msgr2=0x7f47381c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a0a30 0x7f47381c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f47377fe700 1 -- 172.21.15.32:0/3018504241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280094a0 con 0x7f47380a1340 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.926+0000 7f4737fff700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a0a30 0x7f47381c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.927+0000 7f47377fe700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 0x7f47381c87c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f472c0090b0 tx=0x7f472c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.927+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c004510 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.927+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c01f070 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.927+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c0094a0 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.927+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c0038c0 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.927+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380a25e0 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.928+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f472c003a20 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.928+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f472c021070 con 0x7f47380a1340 2023-09-30T22:27:17.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.928+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738125dd0 con 0x7f47380a1340 2023-09-30T22:27:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:17.933+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f472c015020 con 0x7f47380a1340 2023-09-30T22:27:18.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:18.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.130+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5d2c072980 con 0x7f5d2c133930 2023-09-30T22:27:18.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.130+0000 7f5d13fff700 1 -- 172.21.15.32:0/3374023456 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d24028020 con 0x7f5d2c133930 2023-09-30T22:27:18.131 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 msgr2=0x7f5d2c1c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c1c8990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5d24016000 tx=0x7f5d24004750 comp rx=0 tx=0).stop 2023-09-30T22:27:18.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 shutdown_connections 2023-09-30T22:27:18.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d2c133930 0x7f5d2c1c8990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d2c09d630 0x7f5d2c1c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 --2- 172.21.15.32:0/3374023456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d2c0a0a30 0x7f5d2c1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.132+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 >> 172.21.15.32:0/3374023456 conn(0x7f5d2c0a3050 msgr2=0x7f5d2c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.133+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 shutdown_connections 2023-09-30T22:27:18.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.133+0000 7f5d319fe700 1 -- 172.21.15.32:0/3374023456 wait complete. 2023-09-30T22:27:18.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.165+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4738074ea0 con 0x7f47380a1340 2023-09-30T22:27:18.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.165+0000 7f4734ff9700 1 -- 172.21.15.32:0/3018504241 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f472c010380 con 0x7f47380a1340 2023-09-30T22:27:18.166 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 msgr2=0x7f47381c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 0x7f47381c87c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f472c0090b0 tx=0x7f472c004750 comp rx=0 tx=0).stop 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 shutdown_connections 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4738133fa0 0x7f47381c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47380a0a30 0x7f47381c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 --2- 172.21.15.32:0/3018504241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47380a1340 0x7f47381c87c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 >> 172.21.15.32:0/3018504241 conn(0x7f47380a6550 msgr2=0x7f473809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.167+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 shutdown_connections 2023-09-30T22:27:18.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.168+0000 7f473e705700 1 -- 172.21.15.32:0/3018504241 wait complete. 2023-09-30T22:27:18.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:18.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 Processor -- start 2023-09-30T22:27:18.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 -- start start 2023-09-30T22:27:18.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09e930 0x7f594c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f594c09f240 0x7f594c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f650 0x7f594c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594c070940 con 0x7f594c09f240 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c06fd60 con 0x7f594c09f650 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.259+0000 7f59537d2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594c06fa80 con 0x7f594c09e930 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f594c09f240 0x7f594c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5951d6f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f650 0x7f594c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f595156e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09e930 0x7f594c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f594c09f240 0x7f594c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51092/0 (socket says 172.21.15.32:51092) 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 -- 172.21.15.32:0/1584272128 learned_addr learned my addr 172.21.15.32:0/1584272128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 -- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09e930 msgr2=0x7f594c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09e930 0x7f594c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 -- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f650 msgr2=0x7f594c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f650 0x7f594c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 -- 172.21.15.32:0/1584272128 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c09fb60 con 0x7f594c09f240 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5951d6f700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f650 0x7f594c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f595156e700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09e930 0x7f594c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5950d6d700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09f240 0x7f594c1317f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f593c0098f0 tx=0x7f593c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb07aa8adc95794 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5943fff700 1 -- 172.21.15.32:0/1584272128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011830 con 0x7f594c09f240 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5943fff700 1 -- 172.21.15.32:0/1584272128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c011990 con 0x7f594c09f240 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.260+0000 7f5943fff700 1 -- 172.21.15.32:0/1584272128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011d20 con 0x7f594c09f240 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 -- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09f240 msgr2=0x7f594c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09f240 0x7f594c1317f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f593c0098f0 tx=0x7f593c009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:18.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 -- 172.21.15.32:0/1584272128 shutdown_connections 2023-09-30T22:27:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f650 0x7f594c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09e930 0x7f594c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 --2- 172.21.15.32:0/1584272128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09f240 0x7f594c1317f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 -- 172.21.15.32:0/1584272128 >> 172.21.15.32:0/1584272128 conn(0x7f594c0a4210 msgr2=0x7f594c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 -- 172.21.15.32:0/1584272128 shutdown_connections 2023-09-30T22:27:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.261+0000 7f59537d2700 1 -- 172.21.15.32:0/1584272128 wait complete. 2023-09-30T22:27:18.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.262+0000 7f59537d2700 1 Processor -- start 2023-09-30T22:27:18.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.262+0000 7f59537d2700 1 -- start start 2023-09-30T22:27:18.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f59537d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09e930 0x7f594c1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f59537d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f240 0x7f594c1a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f59537d2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 0x7f594c1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f59537d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c1a7040 con 0x7f594c09e930 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f59537d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c1a6680 con 0x7f594c09f240 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f59537d2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594c1a5a50 con 0x7f594c09f650 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 0x7f594c1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 0x7f594c1a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34366/0 (socket says 172.21.15.32:34366) 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 -- 172.21.15.32:0/319445512 learned_addr learned my addr 172.21.15.32:0/319445512 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f595156e700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09e930 0x7f594c1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5950d6d700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f240 0x7f594c1a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 -- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f240 msgr2=0x7f594c1a30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f240 0x7f594c1a30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 -- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09e930 msgr2=0x7f594c1a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09e930 0x7f594c1a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f5951d6f700 1 -- 172.21.15.32:0/319445512 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480094a0 con 0x7f594c09f650 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.263+0000 7f595156e700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09e930 0x7f594c1a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.264+0000 7f5951d6f700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 0x7f594c1a35e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f594400efc0 tx=0x7f594400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.264+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594400cc90 con 0x7f594c09f650 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.264+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594400f040 con 0x7f594c09f650 2023-09-30T22:27:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.264+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5944009a70 con 0x7f594c09f650 2023-09-30T22:27:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.264+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944014480 con 0x7f594c09f650 2023-09-30T22:27:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.264+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c009550 con 0x7f594c09f650 2023-09-30T22:27:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.265+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5944014600 con 0x7f594c09f650 2023-09-30T22:27:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.265+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c071d20 con 0x7f594c09f650 2023-09-30T22:27:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.265+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f59440400c0 con 0x7f594c09f650 2023-09-30T22:27:18.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.270+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5944018080 con 0x7f594c09f650 2023-09-30T22:27:18.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.361+0000 7f75cb68b700 1 Processor -- start 2023-09-30T22:27:18.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75cb68b700 1 -- start start 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75cb68b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a1510 0x7f75c4138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75cb68b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c4139250 0x7f75c413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75cb68b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c409edd0 con 0x7f75c40a0a30 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75cb68b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40a18e0 con 0x7f75c40a1510 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75cb68b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75c409e1f0 con 0x7f75c4139250 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75c9c28700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c4139250 0x7f75c413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75c9427700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75c40a0a30 0x7f75c40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43464/0 (socket says 172.21.15.32:43464) 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75c8c26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a1510 0x7f75c4138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.362+0000 7f75c9427700 1 -- 172.21.15.32:0/3690384113 learned_addr learned my addr 172.21.15.32:0/3690384113 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75c8c26700 1 -- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c4139250 msgr2=0x7f75c413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75c8c26700 1 --2- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c4139250 0x7f75c413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75bbfff700 1 -- 172.21.15.32:0/3690384113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740120079 0 0) 0x7f75c409edd0 con 0x7f75c40a0a30 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75c8c26700 1 -- 172.21.15.32:0/3690384113 >> v1:172.21.15.32:6789/0 conn(0x7f75c40a0a30 legacy=0x7f75c40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75c8c26700 1 -- 172.21.15.32:0/3690384113 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c409dc50 con 0x7f75c40a1510 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75c9c28700 1 --2- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c4139250 0x7f75c413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75c8c26700 1 --2- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a1510 0x7f75c4138d10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009870 tx=0x7f75b4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efeebc0c8540895c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75bbfff700 1 -- 172.21.15.32:0/3690384113 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4011830 con 0x7f75c40a1510 2023-09-30T22:27:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75bbfff700 1 -- 172.21.15.32:0/3690384113 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b4011990 con 0x7f75c40a1510 2023-09-30T22:27:18.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.363+0000 7f75bbfff700 1 -- 172.21.15.32:0/3690384113 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4011d20 con 0x7f75c40a1510 2023-09-30T22:27:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 -- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a1510 msgr2=0x7f75c4138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a1510 0x7f75c4138d10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009870 tx=0x7f75b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 -- 172.21.15.32:0/3690384113 shutdown_connections 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c4139250 0x7f75c413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3690384113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a1510 0x7f75c4138d10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 -- 172.21.15.32:0/3690384113 >> 172.21.15.32:0/3690384113 conn(0x7f75c40a6550 msgr2=0x7f75c40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 -- 172.21.15.32:0/3690384113 shutdown_connections 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.364+0000 7f75cb68b700 1 -- 172.21.15.32:0/3690384113 wait complete. 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 Processor -- start 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 -- start start 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 0x7f75c41227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c40a1510 0x7f75c4122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75c4139250 0x7f75c4123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c41380f0 con 0x7f75c4139250 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40a0220 con 0x7f75c40a0a30 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75cb68b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75c41386d0 con 0x7f75c40a1510 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75c9427700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 0x7f75c41227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75c9427700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 0x7f75c41227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54204/0 (socket says 172.21.15.32:54204) 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75c9c28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75c4139250 0x7f75c4123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75c9427700 1 -- 172.21.15.32:0/3087135862 learned_addr learned my addr 172.21.15.32:0/3087135862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.365+0000 7f75c8c26700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c40a1510 0x7f75c4122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c9427700 1 -- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c40a1510 msgr2=0x7f75c4122d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c9427700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c40a1510 0x7f75c4122d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c9427700 1 -- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75c4139250 msgr2=0x7f75c4123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c9427700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75c4139250 0x7f75c4123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c9427700 1 -- 172.21.15.32:0/3087135862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc009a70 con 0x7f75c40a0a30 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c8c26700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c40a1510 0x7f75c4122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75c9427700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 0x7f75c41227e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75c000c520 tx=0x7f75c000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0004c80 con 0x7f75c40a0a30 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0004e00 con 0x7f75c40a0a30 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b40094a0 con 0x7f75c40a0a30 2023-09-30T22:27:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.366+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0007310 con 0x7f75c40a0a30 2023-09-30T22:27:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.367+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c4124be0 con 0x7f75c40a0a30 2023-09-30T22:27:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.367+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75c0007470 con 0x7f75c40a0a30 2023-09-30T22:27:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.367+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f75c003f300 con 0x7f75c40a0a30 2023-09-30T22:27:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.367+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c4125e80 con 0x7f75c40a0a30 2023-09-30T22:27:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.372+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75c4125e80 con 0x7f75c40a0a30 2023-09-30T22:27:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.478+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f594c074ea0 con 0x7f594c09f650 2023-09-30T22:27:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.479+0000 7f5941ffb700 1 -- 172.21.15.32:0/319445512 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5944011680 con 0x7f594c09f650 2023-09-30T22:27:18.480 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 msgr2=0x7f594c1a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 0x7f594c1a35e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f594400efc0 tx=0x7f594400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 shutdown_connections 2023-09-30T22:27:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f594c09f650 0x7f594c1a35e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594c09e930 0x7f594c1a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 --2- 172.21.15.32:0/319445512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594c09f240 0x7f594c1a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.481+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 >> 172.21.15.32:0/319445512 conn(0x7f594c0a4210 msgr2=0x7f594c1b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.482+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 shutdown_connections 2023-09-30T22:27:18.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.482+0000 7f59537d2700 1 -- 172.21.15.32:0/319445512 wait complete. 2023-09-30T22:27:18.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.579+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f75c4125e80 con 0x7f75c40a0a30 2023-09-30T22:27:18.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.579+0000 7f75b9ffb700 1 -- 172.21.15.32:0/3087135862 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75c4125e80 con 0x7f75c40a0a30 2023-09-30T22:27:18.580 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.581+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 msgr2=0x7f75c41227e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.581+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 0x7f75c41227e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75c000c520 tx=0x7f75c000c830 comp rx=0 tx=0).stop 2023-09-30T22:27:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 shutdown_connections 2023-09-30T22:27:18.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75c4139250 0x7f75c4123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75c40a0a30 0x7f75c41227e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 --2- 172.21.15.32:0/3087135862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75c40a1510 0x7f75c4122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 >> 172.21.15.32:0/3087135862 conn(0x7f75c40a6550 msgr2=0x7f75c409ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 shutdown_connections 2023-09-30T22:27:18.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:18.582+0000 7f75cb68b700 1 -- 172.21.15.32:0/3087135862 wait complete. 2023-09-30T22:27:18.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:19.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:19.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.296+0000 7f7768bf1700 1 Processor -- start 2023-09-30T22:27:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.296+0000 7f7768bf1700 1 -- start start 2023-09-30T22:27:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7768bf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 0x7f776406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7768bf1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 0x7f7764131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7768bf1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7764131b40 0x7f7764131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7768bf1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776409ece0 con 0x7f7764131b40 2023-09-30T22:27:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7768bf1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776409e760 con 0x7f776406a770 2023-09-30T22:27:19.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7768bf1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f776409e040 con 0x7f776406b080 2023-09-30T22:27:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7761d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 0x7f7764131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 0x7f776406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 0x7f776406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54212/0 (socket says 172.21.15.32:54212) 2023-09-30T22:27:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7761d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 0x7f7764131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34400/0 (socket says 172.21.15.32:34400) 2023-09-30T22:27:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 -- 172.21.15.32:0/1482992716 learned_addr learned my addr 172.21.15.32:0/1482992716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7762d9d700 1 --2- 172.21.15.32:0/1482992716 >> v2:172.21.15.32:3300/0 conn(0x7f7764131b40 0x7f7764131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 -- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 msgr2=0x7f7764131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 --2- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 0x7f7764131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 -- 172.21.15.32:0/1482992716 >> v2:172.21.15.32:3300/0 conn(0x7f7764131b40 msgr2=0x7f7764131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 --2- 172.21.15.32:0/1482992716 >> v2:172.21.15.32:3300/0 conn(0x7f7764131b40 0x7f7764131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f776259c700 1 -- 172.21.15.32:0/1482992716 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776409f630 con 0x7f776406a770 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7762d9d700 1 --2- 172.21.15.32:0/1482992716 >> v2:172.21.15.32:3300/0 conn(0x7f7764131b40 0x7f7764131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.297+0000 7f7761d9b700 1 --2- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 0x7f7764131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.298+0000 7f776259c700 1 --2- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 0x7f776406ab40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7754009e50 tx=0x7f775400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16a1cb8f5616997 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.298+0000 7f776159a700 1 -- 172.21.15.32:0/1482992716 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400e070 con 0x7f776406a770 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.298+0000 7f776159a700 1 -- 172.21.15.32:0/1482992716 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775400b910 con 0x7f776406a770 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.298+0000 7f776159a700 1 -- 172.21.15.32:0/1482992716 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400bcc0 con 0x7f776406a770 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.298+0000 7f7768bf1700 1 -- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 msgr2=0x7f776406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.298+0000 7f7768bf1700 1 --2- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 0x7f776406ab40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7754009e50 tx=0x7f775400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 -- 172.21.15.32:0/1482992716 shutdown_connections 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 --2- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406a770 0x7f776406ab40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 --2- 172.21.15.32:0/1482992716 >> v2:172.21.15.32:3300/0 conn(0x7f7764131b40 0x7f7764131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 --2- 172.21.15.32:0/1482992716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406b080 0x7f7764131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 -- 172.21.15.32:0/1482992716 >> 172.21.15.32:0/1482992716 conn(0x7f7764066f50 msgr2=0x7f776409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 -- 172.21.15.32:0/1482992716 shutdown_connections 2023-09-30T22:27:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.299+0000 7f7768bf1700 1 -- 172.21.15.32:0/1482992716 wait complete. 2023-09-30T22:27:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 Processor -- start 2023-09-30T22:27:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 -- start start 2023-09-30T22:27:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 0x7f77641200c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 0x7f7764120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7764131b40 0x7f7764120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7764121cd0 con 0x7f7764131b40 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7764122f90 con 0x7f776406b080 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.300+0000 7f7768bf1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77641314b0 con 0x7f776406a770 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f7761d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 0x7f7764120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 0x7f77641200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f7762d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7764131b40 0x7f7764120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 0x7f77641200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34402/0 (socket says 172.21.15.32:34402) 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f7761d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 0x7f7764120600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54222/0 (socket says 172.21.15.32:54222) 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 -- 172.21.15.32:0/2546098510 learned_addr learned my addr 172.21.15.32:0/2546098510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 -- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 msgr2=0x7f7764120600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 0x7f7764120600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 -- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7764131b40 msgr2=0x7f7764120b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7764131b40 0x7f7764120b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 -- 172.21.15.32:0/2546098510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c0094a0 con 0x7f776406a770 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f7761d9b700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 0x7f7764120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f776259c700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 0x7f77641200c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f775400b370 tx=0x7f7754004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400ce80 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7754004250 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c0094a0 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754004600 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.301+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7754009b30 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.302+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7754004760 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.303+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7754040c40 con 0x7f776406a770 2023-09-30T22:27:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.303+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7764071e60 con 0x7f776406a770 2023-09-30T22:27:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.308+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77540117a0 con 0x7f776406a770 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.333+0000 7efdd68c0700 1 Processor -- start 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd68c0700 1 -- start start 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd68c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 0x7efdd00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd68c0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdd0138d80 0x7efdd013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd68c0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00a2a70 con 0x7efdd009d630 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd68c0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd00a01d0 con 0x7efdd0138d80 2023-09-30T22:27:19.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd68c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd009e2b0 con 0x7efdd00a0a30 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd4e5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdd0138d80 0x7efdd013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdcffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 0x7efdd00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd4e5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdd0138d80 0x7efdd013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51134/0 (socket says 172.21.15.32:51134) 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdcffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 0x7efdd00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54238/0 (socket says 172.21.15.32:54238) 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdd4e5d700 1 -- 172.21.15.32:0/1614329012 learned_addr learned my addr 172.21.15.32:0/1614329012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.334+0000 7efdcf7fe700 1 --1- 172.21.15.32:0/1614329012 >> v1:172.21.15.32:6789/0 conn(0x7efdd009d630 0x7efdd00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43468/0 (socket says 172.21.15.32:43468) 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdd4e5d700 1 -- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 msgr2=0x7efdd00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdd4e5d700 1 --2- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 0x7efdd00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdd4e5d700 1 -- 172.21.15.32:0/1614329012 >> v1:172.21.15.32:6789/0 conn(0x7efdd009d630 legacy=0x7efdd00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdd4e5d700 1 -- 172.21.15.32:0/1614329012 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd009dc50 con 0x7efdd0138d80 2023-09-30T22:27:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdceffd700 1 -- 172.21.15.32:0/1614329012 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668519095 0 0) 0x7efdd00a2a70 con 0x7efdd009d630 2023-09-30T22:27:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdcffff700 1 --2- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 0x7efdd00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdd4e5d700 1 --2- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd0138d80 0x7efdd013b160 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009e50 tx=0x7efdc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc75a36f4ccf751 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdceffd700 1 -- 172.21.15.32:0/1614329012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc800e070 con 0x7efdd0138d80 2023-09-30T22:27:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdceffd700 1 -- 172.21.15.32:0/1614329012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc800b910 con 0x7efdd0138d80 2023-09-30T22:27:19.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.335+0000 7efdceffd700 1 -- 172.21.15.32:0/1614329012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc800bcc0 con 0x7efdd0138d80 2023-09-30T22:27:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 -- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd0138d80 msgr2=0x7efdd013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 --2- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd0138d80 0x7efdd013b160 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009e50 tx=0x7efdc800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 -- 172.21.15.32:0/1614329012 shutdown_connections 2023-09-30T22:27:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 --2- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd0138d80 0x7efdd013b160 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 --2- 172.21.15.32:0/1614329012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd00a0a30 0x7efdd00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 -- 172.21.15.32:0/1614329012 >> 172.21.15.32:0/1614329012 conn(0x7efdd00a3050 msgr2=0x7efdd00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 -- 172.21.15.32:0/1614329012 shutdown_connections 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.336+0000 7efdd68c0700 1 -- 172.21.15.32:0/1614329012 wait complete. 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.337+0000 7efdd68c0700 1 Processor -- start 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.337+0000 7efdd68c0700 1 -- start start 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.337+0000 7efdd68c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 0x7efdd0133750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.337+0000 7efdd68c0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdd00a0a30 0x7efdd0133c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.337+0000 7efdd68c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 0x7efdd01341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd68c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0136270 con 0x7efdd009d630 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd68c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0136f90 con 0x7efdd0138d80 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd68c0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00a4ad0 con 0x7efdd00a0a30 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd4e5d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 0x7efdd01341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 0x7efdd0133750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 0x7efdd0133750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51144/0 (socket says 172.21.15.32:51144) 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd4e5d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 0x7efdd01341d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54248/0 (socket says 172.21.15.32:54248) 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 -- 172.21.15.32:0/3618502306 learned_addr learned my addr 172.21.15.32:0/3618502306 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcf7fe700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdd00a0a30 0x7efdd0133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 -- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdd00a0a30 msgr2=0x7efdd0133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdd00a0a30 0x7efdd0133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 -- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 msgr2=0x7efdd01341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 0x7efdd01341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 -- 172.21.15.32:0/3618502306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00094a0 con 0x7efdd009d630 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd4e5d700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 0x7efdd01341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdcffff700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 0x7efdd0133750 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7efdc000ebe0 tx=0x7efdc000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc000c850 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc000c9b0 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc8009b30 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.338+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc000cd40 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.339+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd00a0460 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.339+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efdc0013470 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.339+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd00747f0 con 0x7efdd009d630 2023-09-30T22:27:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.340+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efdc003fae0 con 0x7efdd009d630 2023-09-30T22:27:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.345+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efdc0013720 con 0x7efdd009d630 2023-09-30T22:27:19.497 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.514+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7764074ea0 con 0x7f776406a770 2023-09-30T22:27:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.514+0000 7f775affd700 1 -- 172.21.15.32:0/2546098510 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7754034090 con 0x7f776406a770 2023-09-30T22:27:19.516 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 msgr2=0x7f77641200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 0x7f77641200c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f775400b370 tx=0x7f7754004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 shutdown_connections 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7764131b40 0x7f7764120b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776406a770 0x7f77641200c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 --2- 172.21.15.32:0/2546098510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776406b080 0x7f7764120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.517+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 >> 172.21.15.32:0/2546098510 conn(0x7f7764066f50 msgr2=0x7f77640a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.518+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 shutdown_connections 2023-09-30T22:27:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.518+0000 7f7768bf1700 1 -- 172.21.15.32:0/2546098510 wait complete. 2023-09-30T22:27:19.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:19.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.553+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efdd0072980 con 0x7efdd009d630 2023-09-30T22:27:19.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.554+0000 7efdccff9700 1 -- 172.21.15.32:0/3618502306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdc0017030 con 0x7efdd009d630 2023-09-30T22:27:19.555 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:19.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 msgr2=0x7efdd0133750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 0x7efdd0133750 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7efdc000ebe0 tx=0x7efdc000eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 shutdown_connections 2023-09-30T22:27:19.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efdd0138d80 0x7efdd01341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdd009d630 0x7efdd0133750 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 --2- 172.21.15.32:0/3618502306 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efdd00a0a30 0x7efdd0133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 >> 172.21.15.32:0/3618502306 conn(0x7efdd00a3050 msgr2=0x7efdd013af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 shutdown_connections 2023-09-30T22:27:19.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.556+0000 7efdd68c0700 1 -- 172.21.15.32:0/3618502306 wait complete. 2023-09-30T22:27:19.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:19.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.640+0000 7f7bd38a5700 1 Processor -- start 2023-09-30T22:27:19.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.640+0000 7f7bd38a5700 1 -- start start 2023-09-30T22:27:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.640+0000 7f7bd38a5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bcc1338a0 0x7f7bcc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd38a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc09e930 0x7f7bcc13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd38a5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc06ef30 con 0x7f7bcc09de50 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd38a5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc06fef0 con 0x7f7bcc1338a0 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd38a5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1334b0 con 0x7f7bcc09e930 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1641700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7bcc1338a0 0x7f7bcc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1e42700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc09e930 0x7f7bcc13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd0e40700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7bcc09de50 0x7f7bcc09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43484/0 (socket says 172.21.15.32:43484) 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd0e40700 1 -- 172.21.15.32:0/3433871335 learned_addr learned my addr 172.21.15.32:0/3433871335 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bc3fff700 1 -- 172.21.15.32:0/3433871335 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677201517 0 0) 0x7f7bcc06ef30 con 0x7f7bcc09de50 2023-09-30T22:27:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bc3fff700 1 -- 172.21.15.32:0/3433871335 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb80035b0 con 0x7f7bcc09de50 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1e42700 1 -- 172.21.15.32:0/3433871335 >> v2:172.21.15.32:3300/0 conn(0x7f7bcc1338a0 msgr2=0x7f7bcc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1e42700 1 --2- 172.21.15.32:0/3433871335 >> v2:172.21.15.32:3300/0 conn(0x7f7bcc1338a0 0x7f7bcc133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1e42700 1 -- 172.21.15.32:0/3433871335 >> v1:172.21.15.32:6789/0 conn(0x7f7bcc09de50 legacy=0x7f7bcc09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1e42700 1 -- 172.21.15.32:0/3433871335 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc06fc30 con 0x7f7bcc09e930 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1641700 1 --2- 172.21.15.32:0/3433871335 >> v2:172.21.15.32:3300/0 conn(0x7f7bcc1338a0 0x7f7bcc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.641+0000 7f7bd1e42700 1 --2- 172.21.15.32:0/3433871335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc09e930 0x7f7bcc13b740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009d90 tx=0x7f7bc400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3381ea6e5ee9d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bc3fff700 1 -- 172.21.15.32:0/3433871335 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400e070 con 0x7f7bcc09e930 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bc3fff700 1 -- 172.21.15.32:0/3433871335 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc400bb20 con 0x7f7bcc09e930 2023-09-30T22:27:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bc3fff700 1 -- 172.21.15.32:0/3433871335 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4012430 con 0x7f7bcc09e930 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bd38a5700 1 -- 172.21.15.32:0/3433871335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc09e930 msgr2=0x7f7bcc13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/3433871335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc09e930 0x7f7bcc13b740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009d90 tx=0x7f7bc400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bd38a5700 1 -- 172.21.15.32:0/3433871335 shutdown_connections 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/3433871335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc09e930 0x7f7bcc13b740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/3433871335 >> v2:172.21.15.32:3300/0 conn(0x7f7bcc1338a0 0x7f7bcc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.642+0000 7f7bd38a5700 1 -- 172.21.15.32:0/3433871335 >> 172.21.15.32:0/3433871335 conn(0x7f7bcc0a2aa0 msgr2=0x7f7bcc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.643+0000 7f7bd38a5700 1 -- 172.21.15.32:0/3433871335 shutdown_connections 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.643+0000 7f7bd38a5700 1 -- 172.21.15.32:0/3433871335 wait complete. 2023-09-30T22:27:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 Processor -- start 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 -- start start 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bcc09de50 0x7f7bcc11d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 0x7f7bcc11dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc1338a0 0x7f7bcc11e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc1208f0 con 0x7f7bcc09e930 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc06f640 con 0x7f7bcc09de50 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd38a5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0a0c40 con 0x7f7bcc1338a0 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd0e40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 0x7f7bcc11dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd0e40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 0x7f7bcc11dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51172/0 (socket says 172.21.15.32:51172) 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd0e40700 1 -- 172.21.15.32:0/2511968797 learned_addr learned my addr 172.21.15.32:0/2511968797 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.644+0000 7f7bd1e42700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc1338a0 0x7f7bcc11e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd0e40700 1 -- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc1338a0 msgr2=0x7f7bcc11e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd0e40700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc1338a0 0x7f7bcc11e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd0e40700 1 -- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bcc09de50 msgr2=0x7f7bcc11d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd0e40700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bcc09de50 0x7f7bcc11d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd0e40700 1 -- 172.21.15.32:0/2511968797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80094a0 con 0x7f7bcc09e930 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd1e42700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc1338a0 0x7f7bcc11e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd0e40700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 0x7f7bcc11dcb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc001eb0 tx=0x7f7bbc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc005d70 con 0x7f7bcc09e930 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc0070d0 con 0x7f7bcc09e930 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc4009a70 con 0x7f7bcc09e930 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc007460 con 0x7f7bcc09e930 2023-09-30T22:27:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.645+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1204a0 con 0x7f7bcc09e930 2023-09-30T22:27:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.646+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7bbc0075c0 con 0x7f7bcc09e930 2023-09-30T22:27:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.646+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7bbc03ec90 con 0x7f7bcc09e930 2023-09-30T22:27:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.647+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc11a470 con 0x7f7bcc09e930 2023-09-30T22:27:19.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.652+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7bbc01bb10 con 0x7f7bcc09e930 2023-09-30T22:27:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.762+0000 7fe6d7779700 1 Processor -- start 2023-09-30T22:27:19.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.762+0000 7fe6d7779700 1 -- start start 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d7779700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009d6f0 0x7fe6d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d7779700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6d009e000 0x7fe6d0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d7779700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e410 0x7fe6d0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d7779700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d009f520 con 0x7fe6d009e000 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d7779700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d009f130 con 0x7fe6d009e410 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d7779700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d009e930 con 0x7fe6d009d6f0 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d5515700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009d6f0 0x7fe6d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6d009e000 0x7fe6d0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d5515700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009d6f0 0x7fe6d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:34452/0 (socket says 172.21.15.32:34452) 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6d009e000 0x7fe6d0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51186/0 (socket says 172.21.15.32:51186) 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d5515700 1 -- 172.21.15.32:0/2480657566 learned_addr learned my addr 172.21.15.32:0/2480657566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d5d16700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e410 0x7fe6d0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 -- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009d6f0 msgr2=0x7fe6d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009d6f0 0x7fe6d009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 -- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e410 msgr2=0x7fe6d0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e410 0x7fe6d0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d4d14700 1 -- 172.21.15.32:0/2480657566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d009b2d0 con 0x7fe6d009e000 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.763+0000 7fe6d5d16700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e410 0x7fe6d0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d4d14700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009e000 0x7fe6d0130660 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009870 tx=0x7fe6cc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f164f77e75940b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6c3fff700 1 -- 172.21.15.32:0/2480657566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc011830 con 0x7fe6d009e000 2023-09-30T22:27:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6c3fff700 1 -- 172.21.15.32:0/2480657566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc011990 con 0x7fe6d009e000 2023-09-30T22:27:19.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6c3fff700 1 -- 172.21.15.32:0/2480657566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc011d20 con 0x7fe6d009e000 2023-09-30T22:27:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 -- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009e000 msgr2=0x7fe6d0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009e000 0x7fe6d0130660 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009870 tx=0x7fe6cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 -- 172.21.15.32:0/2480657566 shutdown_connections 2023-09-30T22:27:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e410 0x7fe6d0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009d6f0 0x7fe6d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 --2- 172.21.15.32:0/2480657566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009e000 0x7fe6d0130660 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.764+0000 7fe6d7779700 1 -- 172.21.15.32:0/2480657566 >> 172.21.15.32:0/2480657566 conn(0x7fe6d0066f50 msgr2=0x7fe6d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.765+0000 7fe6d7779700 1 -- 172.21.15.32:0/2480657566 shutdown_connections 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.765+0000 7fe6d7779700 1 -- 172.21.15.32:0/2480657566 wait complete. 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.766+0000 7fe6d7779700 1 Processor -- start 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.766+0000 7fe6d7779700 1 -- start start 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d7779700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 0x7fe6d011deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d7779700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e000 0x7fe6d011e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d7779700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009e410 0x7fe6d011e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d7779700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0120110 con 0x7fe6d009d6f0 2023-09-30T22:27:19.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d7779700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d01219d0 con 0x7fe6d009e000 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d7779700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0120da0 con 0x7fe6d009e410 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 0x7fe6d011deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 0x7fe6d011deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51198/0 (socket says 172.21.15.32:51198) 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 -- 172.21.15.32:0/884564141 learned_addr learned my addr 172.21.15.32:0/884564141 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d4d14700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e000 0x7fe6d011e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5d16700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009e410 0x7fe6d011e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 -- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009e410 msgr2=0x7fe6d011e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009e410 0x7fe6d011e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 -- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e000 msgr2=0x7fe6d011e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e000 0x7fe6d011e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d5515700 1 -- 172.21.15.32:0/884564141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009a70 con 0x7fe6d009d6f0 2023-09-30T22:27:19.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.767+0000 7fe6d4d14700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e000 0x7fe6d011e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.768+0000 7fe6d5515700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 0x7fe6d011deb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4010e40 tx=0x7fe6c400e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.768+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4011650 con 0x7fe6d009d6f0 2023-09-30T22:27:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.768+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4010ed0 con 0x7fe6d009d6f0 2023-09-30T22:27:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.768+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc0094a0 con 0x7fe6d009d6f0 2023-09-30T22:27:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.768+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40119b0 con 0x7fe6d009d6f0 2023-09-30T22:27:19.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.768+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d011fac0 con 0x7fe6d009d6f0 2023-09-30T22:27:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.769+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6c4011b10 con 0x7fe6d009d6f0 2023-09-30T22:27:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.769+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe6c4042230 con 0x7fe6d009d6f0 2023-09-30T22:27:19.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.769+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d011ac20 con 0x7fe6d009d6f0 2023-09-30T22:27:19.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.774+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6c401e020 con 0x7fe6d009d6f0 2023-09-30T22:27:19.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.897+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7bcc120d00 con 0x7f7bcc09e930 2023-09-30T22:27:19.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.898+0000 7f7bc1ffb700 1 -- 172.21.15.32:0/2511968797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7bbc0192c0 con 0x7f7bcc09e930 2023-09-30T22:27:19.899 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 msgr2=0x7f7bcc11dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 0x7f7bcc11dcb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc001eb0 tx=0x7f7bbc00c860 comp rx=0 tx=0).stop 2023-09-30T22:27:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 shutdown_connections 2023-09-30T22:27:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7bcc1338a0 0x7f7bcc11e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7bcc09de50 0x7f7bcc11d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 --2- 172.21.15.32:0/2511968797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7bcc09e930 0x7f7bcc11dcb0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 >> 172.21.15.32:0/2511968797 conn(0x7f7bcc0a2aa0 msgr2=0x7f7bcc1baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 shutdown_connections 2023-09-30T22:27:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.900+0000 7f7bd38a5700 1 -- 172.21.15.32:0/2511968797 wait complete. 2023-09-30T22:27:19.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:20.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:19.999+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe6d0074ea0 con 0x7fe6d009d6f0 2023-09-30T22:27:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.000+0000 7fe6c1ffb700 1 -- 172.21.15.32:0/884564141 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6c4019440 con 0x7fe6d009d6f0 2023-09-30T22:27:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 msgr2=0x7fe6d011deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 0x7fe6d011deb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4010e40 tx=0x7fe6c400e640 comp rx=0 tx=0).stop 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 shutdown_connections 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6d009e410 0x7fe6d011e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6d009d6f0 0x7fe6d011deb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 --2- 172.21.15.32:0/884564141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6d009e000 0x7fe6d011e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 >> 172.21.15.32:0/884564141 conn(0x7fe6d0066f50 msgr2=0x7fe6d0130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 shutdown_connections 2023-09-30T22:27:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.002+0000 7fe6d7779700 1 -- 172.21.15.32:0/884564141 wait complete. 2023-09-30T22:27:20.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:20.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:20.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:20.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.681+0000 7f312da75700 1 Processor -- start 2023-09-30T22:27:20.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.681+0000 7f312da75700 1 -- start start 2023-09-30T22:27:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f312da75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128136820 0x7f3128138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f312da75700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3128139140 0x7f312813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f312da75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312806bc00 con 0x7f3128133a80 2023-09-30T22:27:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f312da75700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312806c570 con 0x7f3128139140 2023-09-30T22:27:20.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f312da75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31280a0040 con 0x7f3128136820 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f3127fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3128139140 0x7f312813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f31277fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3128133a80 0x7f3128133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46346/0 (socket says 172.21.15.32:46346) 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f31277fe700 1 -- 172.21.15.32:0/2485366068 learned_addr learned my addr 172.21.15.32:0/2485366068 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f311ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128136820 0x7f3128138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f311ffff700 1 -- 172.21.15.32:0/2485366068 >> v2:172.21.15.32:3300/0 conn(0x7f3128139140 msgr2=0x7f312813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f311ffff700 1 --2- 172.21.15.32:0/2485366068 >> v2:172.21.15.32:3300/0 conn(0x7f3128139140 0x7f312813b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f311ffff700 1 -- 172.21.15.32:0/2485366068 >> v1:172.21.15.32:6789/0 conn(0x7f3128133a80 legacy=0x7f3128133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f311ffff700 1 -- 172.21.15.32:0/2485366068 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128133860 con 0x7f3128136820 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f3126ffd700 1 -- 172.21.15.32:0/2485366068 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796978812 0 0) 0x7f312806bc00 con 0x7f3128133a80 2023-09-30T22:27:20.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.682+0000 7f3127fff700 1 --2- 172.21.15.32:0/2485366068 >> v2:172.21.15.32:3300/0 conn(0x7f3128139140 0x7f312813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.683+0000 7f311ffff700 1 --2- 172.21.15.32:0/2485366068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128136820 0x7f3128138c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3110009840 tx=0x7f3110009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cae4147b5c87ef3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.683+0000 7f3126ffd700 1 -- 172.21.15.32:0/2485366068 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110011830 con 0x7f3128136820 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.683+0000 7f3126ffd700 1 -- 172.21.15.32:0/2485366068 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3110011990 con 0x7f3128136820 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.683+0000 7f3126ffd700 1 -- 172.21.15.32:0/2485366068 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110011d20 con 0x7f3128136820 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 -- 172.21.15.32:0/2485366068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128136820 msgr2=0x7f3128138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 --2- 172.21.15.32:0/2485366068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128136820 0x7f3128138c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3110009840 tx=0x7f3110009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 -- 172.21.15.32:0/2485366068 shutdown_connections 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 --2- 172.21.15.32:0/2485366068 >> v2:172.21.15.32:3300/0 conn(0x7f3128139140 0x7f312813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 --2- 172.21.15.32:0/2485366068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128136820 0x7f3128138c00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 -- 172.21.15.32:0/2485366068 >> 172.21.15.32:0/2485366068 conn(0x7f31280a4210 msgr2=0x7f31280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:20.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 -- 172.21.15.32:0/2485366068 shutdown_connections 2023-09-30T22:27:20.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.684+0000 7f312da75700 1 -- 172.21.15.32:0/2485366068 wait complete. 2023-09-30T22:27:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.685+0000 7f312da75700 1 Processor -- start 2023-09-30T22:27:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.685+0000 7f312da75700 1 -- start start 2023-09-30T22:27:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f312da75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128133a80 0x7f312809b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f312da75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 0x7f312809b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f312da75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 0x7f312809beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f312da75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312809dcd0 con 0x7f3128139140 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f312da75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31281363d0 con 0x7f3128133a80 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f312da75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f312809deb0 con 0x7f3128136820 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f3127fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 0x7f312809beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 0x7f312809b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 0x7f312809b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51898/0 (socket says 172.21.15.32:51898) 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f3127fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 0x7f312809beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33416/0 (socket says 172.21.15.32:33416) 2023-09-30T22:27:20.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 -- 172.21.15.32:0/536147272 learned_addr learned my addr 172.21.15.32:0/536147272 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f31277fe700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128133a80 0x7f312809b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 -- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128133a80 msgr2=0x7f312809b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128133a80 0x7f312809b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 -- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 msgr2=0x7f312809beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 0x7f312809beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f311ffff700 1 -- 172.21.15.32:0/536147272 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120009a70 con 0x7f3128136820 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.686+0000 7f3127fff700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 0x7f312809beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:20.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.687+0000 7f311ffff700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 0x7f312809b970 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3110009fd0 tx=0x7f3110004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.687+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31100045d0 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.687+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311001f070 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.687+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31100094a0 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.687+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31100038a0 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.687+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312809ca70 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.688+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3110003a00 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.688+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312809db30 con 0x7f3128136820 2023-09-30T22:27:20.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.688+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3110021070 con 0x7f3128136820 2023-09-30T22:27:20.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.693+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f312809db30 con 0x7f3128136820 2023-09-30T22:27:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.719+0000 7fe43148e700 1 Processor -- start 2023-09-30T22:27:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.719+0000 7fe43148e700 1 -- start start 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe43148e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 0x7fe42c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe43148e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe42c138d80 0x7fe42c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe43148e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c0a2a70 con 0x7fe42c09d630 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe43148e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0a01d0 con 0x7fe42c138d80 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe43148e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c09e2b0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe42b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe42c138d80 0x7fe42c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe42affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 0x7fe42c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe42a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe42c09d630 0x7fe42c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46348/0 (socket says 172.21.15.32:46348) 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe42b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe42c138d80 0x7fe42c13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33418/0 (socket says 172.21.15.32:33418) 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe42affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 0x7fe42c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51908/0 (socket says 172.21.15.32:51908) 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.720+0000 7fe42a7fc700 1 -- 172.21.15.32:0/801529518 learned_addr learned my addr 172.21.15.32:0/801529518 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe42b7fe700 1 -- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 msgr2=0x7fe42c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe42b7fe700 1 --2- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 0x7fe42c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe42b7fe700 1 -- 172.21.15.32:0/801529518 >> v1:172.21.15.32:6789/0 conn(0x7fe42c09d630 legacy=0x7fe42c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe42b7fe700 1 -- 172.21.15.32:0/801529518 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c09dc50 con 0x7fe42c138d80 2023-09-30T22:27:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe429ffb700 1 -- 172.21.15.32:0/801529518 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19489189 0 0) 0x7fe42c0a2a70 con 0x7fe42c09d630 2023-09-30T22:27:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe42affd700 1 --2- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 0x7fe42c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe42b7fe700 1 --2- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c138d80 0x7fe42c13b160 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe424009840 tx=0x7fe424009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451f2366ea1efca0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe429ffb700 1 -- 172.21.15.32:0/801529518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424011830 con 0x7fe42c138d80 2023-09-30T22:27:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe429ffb700 1 -- 172.21.15.32:0/801529518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe424011990 con 0x7fe42c138d80 2023-09-30T22:27:20.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.721+0000 7fe429ffb700 1 -- 172.21.15.32:0/801529518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424011d20 con 0x7fe42c138d80 2023-09-30T22:27:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 -- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c138d80 msgr2=0x7fe42c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 --2- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c138d80 0x7fe42c13b160 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe424009840 tx=0x7fe424009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:20.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 -- 172.21.15.32:0/801529518 shutdown_connections 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 --2- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c138d80 0x7fe42c13b160 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 --2- 172.21.15.32:0/801529518 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c0a0a30 0x7fe42c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 -- 172.21.15.32:0/801529518 >> 172.21.15.32:0/801529518 conn(0x7fe42c0a3050 msgr2=0x7fe42c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 -- 172.21.15.32:0/801529518 shutdown_connections 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.722+0000 7fe43148e700 1 -- 172.21.15.32:0/801529518 wait complete. 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.723+0000 7fe43148e700 1 Processor -- start 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.723+0000 7fe43148e700 1 -- start start 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe43148e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe42c09d630 0x7fe42c133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe43148e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c0a0a30 0x7fe42c133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe43148e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 0x7fe42c134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:20.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe43148e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c1361b0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe43148e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c136ed0 con 0x7fe42c09d630 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe43148e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0a4b60 con 0x7fe42c138d80 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 0x7fe42c134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c0a0a30 0x7fe42c133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 0x7fe42c134110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51916/0 (socket says 172.21.15.32:51916) 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42b7fe700 1 -- 172.21.15.32:0/3071768604 learned_addr learned my addr 172.21.15.32:0/3071768604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 -- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 msgr2=0x7fe42c134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 0x7fe42c134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 -- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe42c09d630 msgr2=0x7fe42c133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe42c09d630 0x7fe42c133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 -- 172.21.15.32:0/3071768604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c009a70 con 0x7fe42c0a0a30 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42b7fe700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 0x7fe42c134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:20.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe42a7fc700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c0a0a30 0x7fe42c133bd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c470 tx=0x7fe41400c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.724+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414004c80 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.725+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe414004de0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.725+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4240094a0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.725+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414007310 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.725+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c0a0460 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.725+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe414007470 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.726+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe41403f2d0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.726+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c0747f0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.731+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe414007720 con 0x7fe42c0a0a30 2023-09-30T22:27:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.900+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f312809db30 con 0x7f3128136820 2023-09-30T22:27:20.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.900+0000 7f3124ff9700 1 -- 172.21.15.32:0/536147272 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f312809db30 con 0x7f3128136820 2023-09-30T22:27:20.901 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:20.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.902+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 msgr2=0x7f312809b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.902+0000 7f312da75700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 0x7f312809b970 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3110009fd0 tx=0x7f3110004750 comp rx=0 tx=0).stop 2023-09-30T22:27:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.902+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 shutdown_connections 2023-09-30T22:27:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.902+0000 7f312da75700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3128139140 0x7f312809beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.903+0000 7f312da75700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3128133a80 0x7f312809b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.903+0000 7f312da75700 1 --2- 172.21.15.32:0/536147272 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3128136820 0x7f312809b970 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.903+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 >> 172.21.15.32:0/536147272 conn(0x7f31280a4210 msgr2=0x7f31280a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.903+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 shutdown_connections 2023-09-30T22:27:20.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.903+0000 7f312da75700 1 -- 172.21.15.32:0/536147272 wait complete. 2023-09-30T22:27:20.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:20.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.941+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe42c072980 con 0x7fe42c0a0a30 2023-09-30T22:27:20.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.941+0000 7fe4237fe700 1 -- 172.21.15.32:0/3071768604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe41403f0d0 con 0x7fe42c0a0a30 2023-09-30T22:27:20.942 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c0a0a30 msgr2=0x7fe42c133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c0a0a30 0x7fe42c133bd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c470 tx=0x7fe41400c780 comp rx=0 tx=0).stop 2023-09-30T22:27:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 shutdown_connections 2023-09-30T22:27:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe42c138d80 0x7fe42c134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe42c09d630 0x7fe42c133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 --2- 172.21.15.32:0/3071768604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe42c0a0a30 0x7fe42c133bd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:20.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 >> 172.21.15.32:0/3071768604 conn(0x7fe42c0a3050 msgr2=0x7fe42c13af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:20.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 shutdown_connections 2023-09-30T22:27:20.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:20.944+0000 7fe43148e700 1 -- 172.21.15.32:0/3071768604 wait complete. 2023-09-30T22:27:20.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:21.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.053+0000 7fdba019d700 1 Processor -- start 2023-09-30T22:27:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.053+0000 7fdba019d700 1 -- start start 2023-09-30T22:27:21.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdba019d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9809d6f0 0x7fdb9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdba019d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb98134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdba019d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9809b120 con 0x7fdb9809e000 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdba019d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9806b460 con 0x7fdb9809d6f0 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdba019d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9809a7e0 con 0x7fdb9809e440 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb98134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9df39700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9809d6f0 0x7fdb9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb98134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51924/0 (socket says 172.21.15.32:51924) 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 -- 172.21.15.32:0/265775787 learned_addr learned my addr 172.21.15.32:0/265775787 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9d738700 1 --1- 172.21.15.32:0/265775787 >> v1:172.21.15.32:6789/0 conn(0x7fdb9809e000 0x7fdb981339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46352/0 (socket says 172.21.15.32:46352) 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 -- 172.21.15.32:0/265775787 >> v2:172.21.15.32:3300/0 conn(0x7fdb9809d6f0 msgr2=0x7fdb9809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 --2- 172.21.15.32:0/265775787 >> v2:172.21.15.32:3300/0 conn(0x7fdb9809d6f0 0x7fdb9809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 -- 172.21.15.32:0/265775787 >> v1:172.21.15.32:6789/0 conn(0x7fdb9809e000 legacy=0x7fdb981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 -- 172.21.15.32:0/265775787 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9809a500 con 0x7fdb9809e440 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9cf37700 1 -- 172.21.15.32:0/265775787 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325664279 0 0) 0x7fdb9809b120 con 0x7fdb9809e000 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9df39700 1 --2- 172.21.15.32:0/265775787 >> v2:172.21.15.32:3300/0 conn(0x7fdb9809d6f0 0x7fdb9809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.054+0000 7fdb9e73a700 1 --2- 172.21.15.32:0/265775787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb98134100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009e80 tx=0x7fdb9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b28bcc032821a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdb9cf37700 1 -- 172.21.15.32:0/265775787 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000e070 con 0x7fdb9809e440 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdb9cf37700 1 -- 172.21.15.32:0/265775787 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9000b8d0 con 0x7fdb9809e440 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdb9cf37700 1 -- 172.21.15.32:0/265775787 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000bc80 con 0x7fdb9809e440 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 -- 172.21.15.32:0/265775787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 msgr2=0x7fdb98134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 --2- 172.21.15.32:0/265775787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb98134100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009e80 tx=0x7fdb9000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 -- 172.21.15.32:0/265775787 shutdown_connections 2023-09-30T22:27:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 --2- 172.21.15.32:0/265775787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb98134100 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 --2- 172.21.15.32:0/265775787 >> v2:172.21.15.32:3300/0 conn(0x7fdb9809d6f0 0x7fdb9809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 -- 172.21.15.32:0/265775787 >> 172.21.15.32:0/265775787 conn(0x7fdb98066f50 msgr2=0x7fdb9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 -- 172.21.15.32:0/265775787 shutdown_connections 2023-09-30T22:27:21.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.055+0000 7fdba019d700 1 -- 172.21.15.32:0/265775787 wait complete. 2023-09-30T22:27:21.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.056+0000 7fdba019d700 1 Processor -- start 2023-09-30T22:27:21.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 -- start start 2023-09-30T22:27:21.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb9809d6f0 0x7fdb981a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 0x7fdb981a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb981c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb981a52c0 con 0x7fdb9809e000 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb981b8040 con 0x7fdb9809d6f0 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdba019d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9809c710 con 0x7fdb9809e440 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 0x7fdb981a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 0x7fdb981a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33452/0 (socket says 172.21.15.32:33452) 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9e73a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb981c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 -- 172.21.15.32:0/3066760957 learned_addr learned my addr 172.21.15.32:0/3066760957 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 -- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 msgr2=0x7fdb981c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb981c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 -- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb9809d6f0 msgr2=0x7fdb981a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb9809d6f0 0x7fdb981a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.057+0000 7fdb9d738700 1 -- 172.21.15.32:0/3066760957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940094a0 con 0x7fdb9809e000 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.058+0000 7fdb9d738700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 0x7fdb981a46a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c520 tx=0x7fdb8800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.058+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88004c80 con 0x7fdb9809e000 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.058+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88004de0 con 0x7fdb9809e000 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.058+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb90009b30 con 0x7fdb9809e000 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.058+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88007310 con 0x7fdb9809e000 2023-09-30T22:27:21.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.058+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb981a5ab0 con 0x7fdb9809e000 2023-09-30T22:27:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.059+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdb88007470 con 0x7fdb9809e000 2023-09-30T22:27:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.059+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdb8803f3a0 con 0x7fdb9809e000 2023-09-30T22:27:21.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.059+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb98074ea0 con 0x7fdb9809e000 2023-09-30T22:27:21.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.064+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb880123b0 con 0x7fdb9809e000 2023-09-30T22:27:21.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.158+0000 7fdb030c8700 1 Processor -- start 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.158+0000 7fdb030c8700 1 -- start start 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb030c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb030c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09e000 0x7fdafc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb030c8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdafc09b120 con 0x7fdafc09e410 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb030c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc06b460 con 0x7fdafc09d6f0 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb030c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc09a7e0 con 0x7fdafc09e000 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb01665700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdafc09e410 0x7fdafc133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46358/0 (socket says 172.21.15.32:46358) 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09e000 0x7fdafc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb01665700 1 -- 172.21.15.32:0/3343813574 learned_addr learned my addr 172.21.15.32:0/3343813574 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb00e64700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafb7fe700 1 -- 172.21.15.32:0/3343813574 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761720460 0 0) 0x7fdafc09b120 con 0x7fdafc09e410 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafbfff700 1 -- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09d6f0 msgr2=0x7fdafc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafbfff700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafbfff700 1 -- 172.21.15.32:0/3343813574 >> v1:172.21.15.32:6789/0 conn(0x7fdafc09e410 legacy=0x7fdafc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafbfff700 1 -- 172.21.15.32:0/3343813574 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc09a500 con 0x7fdafc09e000 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdb00e64700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.159+0000 7fdafbfff700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09e000 0x7fdafc1339f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdae80098a0 tx=0x7fdae8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f3ea9ed19291d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdafb7fe700 1 -- 172.21.15.32:0/3343813574 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8011830 con 0x7fdafc09e000 2023-09-30T22:27:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdafb7fe700 1 -- 172.21.15.32:0/3343813574 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8011990 con 0x7fdafc09e000 2023-09-30T22:27:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdafb7fe700 1 -- 172.21.15.32:0/3343813574 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8011d20 con 0x7fdafc09e000 2023-09-30T22:27:21.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 -- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09e000 msgr2=0x7fdafc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09e000 0x7fdafc1339f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdae80098a0 tx=0x7fdae8009bb0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 -- 172.21.15.32:0/3343813574 shutdown_connections 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 --2- 172.21.15.32:0/3343813574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09e000 0x7fdafc1339f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 -- 172.21.15.32:0/3343813574 >> 172.21.15.32:0/3343813574 conn(0x7fdafc066f50 msgr2=0x7fdafc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 -- 172.21.15.32:0/3343813574 shutdown_connections 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.160+0000 7fdb030c8700 1 -- 172.21.15.32:0/3343813574 wait complete. 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.161+0000 7fdb030c8700 1 Processor -- start 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.161+0000 7fdb030c8700 1 -- start start 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb030c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 0x7fdafc1a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb030c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdafc09e000 0x7fdafc1a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb030c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09e410 0x7fdafc1c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb030c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1a4f70 con 0x7fdafc09e000 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb030c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1b93f0 con 0x7fdafc09e410 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb030c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc09c710 con 0x7fdafc09d6f0 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 0x7fdafc1a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 0x7fdafc1a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51944/0 (socket says 172.21.15.32:51944) 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdafbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdafc09e000 0x7fdafc1a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 -- 172.21.15.32:0/1354497478 learned_addr learned my addr 172.21.15.32:0/1354497478 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb01665700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09e410 0x7fdafc1c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 -- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09e410 msgr2=0x7fdafc1c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09e410 0x7fdafc1c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 -- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdafc09e000 msgr2=0x7fdafc1a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdafc09e000 0x7fdafc1a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdb00e64700 1 -- 172.21.15.32:0/1354497478 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf00094a0 con 0x7fdafc09d6f0 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.162+0000 7fdafbfff700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdafc09e000 0x7fdafc1a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.163+0000 7fdb00e64700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 0x7fdafc1a5330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000ebe0 tx=0x7fdaf000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.163+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000c850 con 0x7fdafc09d6f0 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.163+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf000c9b0 con 0x7fdafc09d6f0 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.163+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae80094a0 con 0x7fdafc09d6f0 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.163+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000cd40 con 0x7fdafc09d6f0 2023-09-30T22:27:21.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.163+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc1a6670 con 0x7fdafc09d6f0 2023-09-30T22:27:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.164+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdaf0013470 con 0x7fdafc09d6f0 2023-09-30T22:27:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.164+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc074ea0 con 0x7fdafc09d6f0 2023-09-30T22:27:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.164+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdaf003f770 con 0x7fdafc09d6f0 2023-09-30T22:27:21.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.169+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdaf000cea0 con 0x7fdafc09d6f0 2023-09-30T22:27:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.280+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdb9809f420 con 0x7fdb9809e000 2023-09-30T22:27:21.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.281+0000 7fdb8e7fc700 1 -- 172.21.15.32:0/3066760957 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb880102c0 con 0x7fdb9809e000 2023-09-30T22:27:21.282 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 msgr2=0x7fdb981a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 0x7fdb981a46a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c520 tx=0x7fdb8800c830 comp rx=0 tx=0).stop 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 shutdown_connections 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdb9809e440 0x7fdb981c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdb9809d6f0 0x7fdb981a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 --2- 172.21.15.32:0/3066760957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9809e000 0x7fdb981a46a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 >> 172.21.15.32:0/3066760957 conn(0x7fdb98066f50 msgr2=0x7fdb980a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 shutdown_connections 2023-09-30T22:27:21.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.283+0000 7fdba019d700 1 -- 172.21.15.32:0/3066760957 wait complete. 2023-09-30T22:27:21.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.376+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdafc072600 con 0x7fdafc09d6f0 2023-09-30T22:27:21.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.377+0000 7fdaf97fa700 1 -- 172.21.15.32:0/1354497478 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdaf0021070 con 0x7fdafc09d6f0 2023-09-30T22:27:21.378 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 msgr2=0x7fdafc1a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 0x7fdafc1a5330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000ebe0 tx=0x7fdaf000eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 shutdown_connections 2023-09-30T22:27:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdafc09e410 0x7fdafc1c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdafc09d6f0 0x7fdafc1a5330 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 --2- 172.21.15.32:0/1354497478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdafc09e000 0x7fdafc1a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 >> 172.21.15.32:0/1354497478 conn(0x7fdafc066f50 msgr2=0x7fdafc06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 shutdown_connections 2023-09-30T22:27:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:21.379+0000 7fdb030c8700 1 -- 172.21.15.32:0/1354497478 wait complete. 2023-09-30T22:27:21.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:21.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:21.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:22.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.064+0000 7f7c791a1700 1 Processor -- start 2023-09-30T22:27:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 -- start start 2023-09-30T22:27:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c740a0a30 0x7f7c740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 0x7f7c740a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c74133930 0x7f7c74133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740a2a70 con 0x7f7c74133930 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740a01d0 con 0x7f7c740a0a30 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c791a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7409e2b0 con 0x7f7c7409d630 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c7259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 0x7f7c740a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c7359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c74133930 0x7f7c74133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c7259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 0x7f7c740a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51952/0 (socket says 172.21.15.32:51952) 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c7259c700 1 -- 172.21.15.32:0/3257403733 learned_addr learned my addr 172.21.15.32:0/3257403733 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c7359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c74133930 0x7f7c74133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33478/0 (socket says 172.21.15.32:33478) 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.065+0000 7f7c72d9d700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c740a0a30 0x7f7c740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7359e700 1 -- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 msgr2=0x7f7c740a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7359e700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 0x7f7c740a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7359e700 1 -- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c740a0a30 msgr2=0x7f7c740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7359e700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c740a0a30 0x7f7c740a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7359e700 1 -- 172.21.15.32:0/3257403733 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7409dc50 con 0x7f7c74133930 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7259c700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 0x7f7c740a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c72d9d700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c740a0a30 0x7f7c740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c7359e700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74133930 0x7f7c74133dd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009840 tx=0x7f7c6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4757f42e21616f13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c71d9b700 1 -- 172.21.15.32:0/3257403733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c011830 con 0x7f7c74133930 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c71d9b700 1 -- 172.21.15.32:0/3257403733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c011990 con 0x7f7c74133930 2023-09-30T22:27:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.066+0000 7f7c71d9b700 1 -- 172.21.15.32:0/3257403733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c011d20 con 0x7f7c74133930 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 -- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74133930 msgr2=0x7f7c74133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74133930 0x7f7c74133dd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009840 tx=0x7f7c6c009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 -- 172.21.15.32:0/3257403733 shutdown_connections 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c740a0a30 0x7f7c740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74133930 0x7f7c74133dd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3257403733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c7409d630 0x7f7c740a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 -- 172.21.15.32:0/3257403733 >> 172.21.15.32:0/3257403733 conn(0x7f7c740a3050 msgr2=0x7f7c740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 -- 172.21.15.32:0/3257403733 shutdown_connections 2023-09-30T22:27:22.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.067+0000 7f7c791a1700 1 -- 172.21.15.32:0/3257403733 wait complete. 2023-09-30T22:27:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.068+0000 7f7c791a1700 1 Processor -- start 2023-09-30T22:27:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.068+0000 7f7c791a1700 1 -- start start 2023-09-30T22:27:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c791a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c7409d630 0x7f7c7411f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c791a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 0x7f7c7411fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c791a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c74133930 0x7f7c741c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c791a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74122320 con 0x7f7c740a0a30 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c791a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74120c40 con 0x7f7c7409d630 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c791a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74122500 con 0x7f7c74133930 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c7259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 0x7f7c7411fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c7359e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c74133930 0x7f7c741c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c7259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 0x7f7c7411fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33490/0 (socket says 172.21.15.32:33490) 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c7259c700 1 -- 172.21.15.32:0/3991504574 learned_addr learned my addr 172.21.15.32:0/3991504574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c72d9d700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c7409d630 0x7f7c7411f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c72d9d700 1 -- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c74133930 msgr2=0x7f7c741c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c72d9d700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c74133930 0x7f7c741c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c72d9d700 1 -- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 msgr2=0x7f7c7411fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c72d9d700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 0x7f7c7411fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c72d9d700 1 -- 172.21.15.32:0/3991504574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68009550 con 0x7f7c7409d630 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.069+0000 7f7c7259c700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 0x7f7c7411fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.070+0000 7f7c72d9d700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c7409d630 0x7f7c7411f4e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400efc0 tx=0x7f7c6400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.070+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400cd10 con 0x7f7c7409d630 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.070+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6400f040 con 0x7f7c7409d630 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.070+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c64009a70 con 0x7f7c7409d630 2023-09-30T22:27:22.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.070+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640146a0 con 0x7f7c7409d630 2023-09-30T22:27:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.070+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c0094a0 con 0x7f7c7409d630 2023-09-30T22:27:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.071+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c64014800 con 0x7f7c7409d630 2023-09-30T22:27:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.071+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c640402f0 con 0x7f7c7409d630 2023-09-30T22:27:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.071+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7411ee90 con 0x7f7c7409d630 2023-09-30T22:27:22.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.076+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c6401d400 con 0x7f7c7409d630 2023-09-30T22:27:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.105+0000 7f4f7ffb1700 1 Processor -- start 2023-09-30T22:27:22.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7ffb1700 1 -- start start 2023-09-30T22:27:22.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7ffb1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f780a0a30 0x7f4f780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7ffb1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f78138d80 0x7f4f7813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7ffb1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f780a2a70 con 0x7f4f7809d630 2023-09-30T22:27:22.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7ffb1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f780a01d0 con 0x7f4f78138d80 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7ffb1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7809e2b0 con 0x7f4f780a0a30 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7e54e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f78138d80 0x7f4f7813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7dd4d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f780a0a30 0x7f4f780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7e54e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f78138d80 0x7f4f7813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33500/0 (socket says 172.21.15.32:33500) 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.106+0000 7f4f7d54c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f7809d630 0x7f4f780a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46360/0 (socket says 172.21.15.32:46360) 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7e54e700 1 -- 172.21.15.32:0/1904846326 learned_addr learned my addr 172.21.15.32:0/1904846326 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7e54e700 1 -- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f780a0a30 msgr2=0x7f4f780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7e54e700 1 --2- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f780a0a30 0x7f4f780a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7e54e700 1 -- 172.21.15.32:0/1904846326 >> v1:172.21.15.32:6789/0 conn(0x7f4f7809d630 legacy=0x7f4f780a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7e54e700 1 -- 172.21.15.32:0/1904846326 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7809dc50 con 0x7f4f78138d80 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7cd4b700 1 -- 172.21.15.32:0/1904846326 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358674034 0 0) 0x7f4f780a2a70 con 0x7f4f7809d630 2023-09-30T22:27:22.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7e54e700 1 --2- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f78138d80 0x7f4f7813b160 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4f700098f0 tx=0x7f4f70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6cc51e7e505aac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7cd4b700 1 -- 172.21.15.32:0/1904846326 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70011830 con 0x7f4f78138d80 2023-09-30T22:27:22.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7cd4b700 1 -- 172.21.15.32:0/1904846326 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f70011990 con 0x7f4f78138d80 2023-09-30T22:27:22.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.107+0000 7f4f7cd4b700 1 -- 172.21.15.32:0/1904846326 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70011d20 con 0x7f4f78138d80 2023-09-30T22:27:22.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f78138d80 msgr2=0x7f4f7813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f78138d80 0x7f4f7813b160 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4f700098f0 tx=0x7f4f70009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:22.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/1904846326 shutdown_connections 2023-09-30T22:27:22.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f78138d80 0x7f4f7813b160 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/1904846326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f780a0a30 0x7f4f780a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/1904846326 >> 172.21.15.32:0/1904846326 conn(0x7f4f780a3050 msgr2=0x7f4f780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/1904846326 shutdown_connections 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.108+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/1904846326 wait complete. 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.109+0000 7f4f7ffb1700 1 Processor -- start 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.109+0000 7f4f7ffb1700 1 -- start start 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7ffb1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f7809d630 0x7f4f78133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7ffb1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f780a0a30 0x7f4f78133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7ffb1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 0x7f4f781342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7ffb1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78134d00 con 0x7f4f780a0a30 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7ffb1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78136700 con 0x7f4f78138d80 2023-09-30T22:27:22.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7ffb1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78137070 con 0x7f4f7809d630 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7e54e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 0x7f4f781342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7d54c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f780a0a30 0x7f4f78133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7dd4d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f7809d630 0x7f4f78133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7e54e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 0x7f4f781342b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48382/0 (socket says 172.21.15.32:48382) 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.110+0000 7f4f7e54e700 1 -- 172.21.15.32:0/3387863246 learned_addr learned my addr 172.21.15.32:0/3387863246 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7e54e700 1 -- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f7809d630 msgr2=0x7f4f78133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7e54e700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f7809d630 0x7f4f78133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7e54e700 1 -- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f780a0a30 msgr2=0x7f4f78133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7e54e700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f780a0a30 0x7f4f78133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7e54e700 1 -- 172.21.15.32:0/3387863246 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f70009550 con 0x7f4f78138d80 2023-09-30T22:27:22.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7d54c700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f780a0a30 0x7f4f78133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7e54e700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 0x7f4f781342b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4f70013000 tx=0x7f4f70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70004380 con 0x7f4f78138d80 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7001f070 con 0x7f4f78138d80 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f780a0410 con 0x7f4f78138d80 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f700036a0 con 0x7f4f78138d80 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.111+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f78136e10 con 0x7f4f78138d80 2023-09-30T22:27:22.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.112+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f70021030 con 0x7f4f78138d80 2023-09-30T22:27:22.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.112+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4f70040870 con 0x7f4f78138d80 2023-09-30T22:27:22.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.112+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f78135ce0 con 0x7f4f78138d80 2023-09-30T22:27:22.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.117+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f70003820 con 0x7f4f78138d80 2023-09-30T22:27:22.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.284+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7c740747f0 con 0x7f7c7409d630 2023-09-30T22:27:22.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.285+0000 7f7c5b7fe700 1 -- 172.21.15.32:0/3991504574 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c6402a030 con 0x7f7c7409d630 2023-09-30T22:27:22.286 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:22.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c7409d630 msgr2=0x7f7c7411f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c7409d630 0x7f7c7411f4e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400efc0 tx=0x7f7c6400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 shutdown_connections 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c74133930 0x7f7c741c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c7409d630 0x7f7c7411f4e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 --2- 172.21.15.32:0/3991504574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c740a0a30 0x7f7c7411fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 >> 172.21.15.32:0/3991504574 conn(0x7f7c740a3050 msgr2=0x7f7c7409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 shutdown_connections 2023-09-30T22:27:22.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.287+0000 7f7c791a1700 1 -- 172.21.15.32:0/3991504574 wait complete. 2023-09-30T22:27:22.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:22.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.325+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4f780747f0 con 0x7f4f78138d80 2023-09-30T22:27:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.326+0000 7f4f767fc700 1 -- 172.21.15.32:0/3387863246 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f700193b0 con 0x7f4f78138d80 2023-09-30T22:27:22.327 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:22.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 msgr2=0x7f4f781342b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 0x7f4f781342b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4f70013000 tx=0x7f4f70004750 comp rx=0 tx=0).stop 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 shutdown_connections 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f78138d80 0x7f4f781342b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f7809d630 0x7f4f78133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 --2- 172.21.15.32:0/3387863246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f780a0a30 0x7f4f78133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 >> 172.21.15.32:0/3387863246 conn(0x7f4f780a3050 msgr2=0x7f4f7809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 shutdown_connections 2023-09-30T22:27:22.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.328+0000 7f4f7ffb1700 1 -- 172.21.15.32:0/3387863246 wait complete. 2023-09-30T22:27:22.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:22.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.438+0000 7f0a47802700 1 Processor -- start 2023-09-30T22:27:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.438+0000 7f0a47802700 1 -- start start 2023-09-30T22:27:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a47802700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 0x7f0a400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a47802700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 0x7f0a40133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a47802700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4009edd0 con 0x7f0a40133fa0 2023-09-30T22:27:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a47802700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4009e1f0 con 0x7f0a400a1340 2023-09-30T22:27:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a47802700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4009dc50 con 0x7f0a400a0a30 2023-09-30T22:27:22.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a44d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 0x7f0a40133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a4559e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 0x7f0a400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a44d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 0x7f0a40133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33518/0 (socket says 172.21.15.32:33518) 2023-09-30T22:27:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a4559e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 0x7f0a400a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48396/0 (socket says 172.21.15.32:48396) 2023-09-30T22:27:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a45d9f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a40133fa0 0x7f0a4013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46374/0 (socket says 172.21.15.32:46374) 2023-09-30T22:27:22.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.439+0000 7f0a44d9d700 1 -- 172.21.15.32:0/3936029389 learned_addr learned my addr 172.21.15.32:0/3936029389 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a4559e700 1 -- 172.21.15.32:0/3936029389 >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 msgr2=0x7f0a40133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a4559e700 1 --2- 172.21.15.32:0/3936029389 >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 0x7f0a40133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a4559e700 1 -- 172.21.15.32:0/3936029389 >> v1:172.21.15.32:6789/0 conn(0x7f0a40133fa0 legacy=0x7f0a4013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a4559e700 1 -- 172.21.15.32:0/3936029389 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a400a1fe0 con 0x7f0a400a0a30 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a44d9d700 1 --2- 172.21.15.32:0/3936029389 >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 0x7f0a40133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a4559e700 1 --2- 172.21.15.32:0/3936029389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 0x7f0a400a0e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009d90 tx=0x7f0a3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc0b08ea80f5a48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a37fff700 1 -- 172.21.15.32:0/3936029389 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00e070 con 0x7f0a400a0a30 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a37fff700 1 -- 172.21.15.32:0/3936029389 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c00bb20 con 0x7f0a400a0a30 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.440+0000 7f0a37fff700 1 -- 172.21.15.32:0/3936029389 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c012430 con 0x7f0a400a0a30 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 -- 172.21.15.32:0/3936029389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 msgr2=0x7f0a400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 --2- 172.21.15.32:0/3936029389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 0x7f0a400a0e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009d90 tx=0x7f0a3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 -- 172.21.15.32:0/3936029389 shutdown_connections 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 --2- 172.21.15.32:0/3936029389 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a0a30 0x7f0a400a0e00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 --2- 172.21.15.32:0/3936029389 >> v2:172.21.15.32:3300/0 conn(0x7f0a400a1340 0x7f0a40133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 -- 172.21.15.32:0/3936029389 >> 172.21.15.32:0/3936029389 conn(0x7f0a400a6550 msgr2=0x7f0a400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 -- 172.21.15.32:0/3936029389 shutdown_connections 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.441+0000 7f0a47802700 1 -- 172.21.15.32:0/3936029389 wait complete. 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 Processor -- start 2023-09-30T22:27:22.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 -- start start 2023-09-30T22:27:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 0x7f0a4011e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a1340 0x7f0a4011ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a40133fa0 0x7f0a4011f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a40121f50 con 0x7f0a400a0a30 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a40120430 con 0x7f0a400a1340 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a47802700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a400a37e0 con 0x7f0a40133fa0 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a4559e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 0x7f0a4011e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a45d9f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a40133fa0 0x7f0a4011f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a4559e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 0x7f0a4011e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33530/0 (socket says 172.21.15.32:33530) 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a44d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a1340 0x7f0a4011ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.442+0000 7f0a4559e700 1 -- 172.21.15.32:0/1239529516 learned_addr learned my addr 172.21.15.32:0/1239529516 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a4559e700 1 -- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a40133fa0 msgr2=0x7f0a4011f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a4559e700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a40133fa0 0x7f0a4011f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a4559e700 1 -- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a1340 msgr2=0x7f0a4011ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a4559e700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a1340 0x7f0a4011ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a4559e700 1 -- 172.21.15.32:0/1239529516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30009550 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a44d9d700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a1340 0x7f0a4011ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a45d9f700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a40133fa0 0x7f0a4011f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a4559e700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 0x7f0a4011e7a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c014040 tx=0x7f0a3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c012430 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c020070 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c009a70 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.443+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c004280 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.444+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a400a0270 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.445+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a3c0043e0 con 0x7f0a400a0a30 2023-09-30T22:27:22.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.445+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a40122a80 con 0x7f0a400a0a30 2023-09-30T22:27:22.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.445+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a3c022070 con 0x7f0a400a0a30 2023-09-30T22:27:22.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.450+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a3c02c080 con 0x7f0a400a0a30 2023-09-30T22:27:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 Processor -- start 2023-09-30T22:27:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 -- start start 2023-09-30T22:27:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f543809f330 0x7f5438138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5438139370 0x7f543813b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543806bec0 con 0x7f543809e850 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543806bce0 con 0x7f543809f330 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f543d9c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543806caa0 con 0x7f5438139370 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f542e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f543809f330 0x7f5438138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f5436ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f543809e850 0x7f543809ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46376/0 (socket says 172.21.15.32:46376) 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f54377fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5438139370 0x7f543813b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.535+0000 7f5436ffd700 1 -- 172.21.15.32:0/768458527 learned_addr learned my addr 172.21.15.32:0/768458527 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f542e7fc700 1 -- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5438139370 msgr2=0x7f543813b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f542e7fc700 1 --2- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5438139370 0x7f543813b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f542e7fc700 1 -- 172.21.15.32:0/768458527 >> v1:172.21.15.32:6789/0 conn(0x7f543809e850 legacy=0x7f543809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f54367fc700 1 -- 172.21.15.32:0/768458527 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423891660 0 0) 0x7f543806bec0 con 0x7f543809e850 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f542e7fc700 1 -- 172.21.15.32:0/768458527 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380a00a0 con 0x7f543809f330 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f542e7fc700 1 --2- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809f330 0x7f5438138e30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5420009840 tx=0x7f5420009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea6286b62b81942 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f54367fc700 1 -- 172.21.15.32:0/768458527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420011830 con 0x7f543809f330 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f54367fc700 1 -- 172.21.15.32:0/768458527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5420011990 con 0x7f543809f330 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.536+0000 7f54367fc700 1 -- 172.21.15.32:0/768458527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420011d20 con 0x7f543809f330 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 -- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809f330 msgr2=0x7f5438138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 --2- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809f330 0x7f5438138e30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5420009840 tx=0x7f5420009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:22.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 -- 172.21.15.32:0/768458527 shutdown_connections 2023-09-30T22:27:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 --2- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5438139370 0x7f543813b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 --2- 172.21.15.32:0/768458527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809f330 0x7f5438138e30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 -- 172.21.15.32:0/768458527 >> 172.21.15.32:0/768458527 conn(0x7f54380a3f70 msgr2=0x7f54380a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 -- 172.21.15.32:0/768458527 shutdown_connections 2023-09-30T22:27:22.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.537+0000 7f543d9c4700 1 -- 172.21.15.32:0/768458527 wait complete. 2023-09-30T22:27:22.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.538+0000 7f543d9c4700 1 Processor -- start 2023-09-30T22:27:22.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.538+0000 7f543d9c4700 1 -- start start 2023-09-30T22:27:22.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f543d9c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 0x7f54381c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f543d9c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f543809f330 0x7f54381c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f543d9c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5438139370 0x7f543811ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f543d9c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54381206b0 con 0x7f543809e850 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f543d9c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54381204c0 con 0x7f543809f330 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f543d9c4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5438120d70 con 0x7f5438139370 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f5436ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 0x7f54381c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f5436ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 0x7f54381c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33544/0 (socket says 172.21.15.32:33544) 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f5436ffd700 1 -- 172.21.15.32:0/2810800834 learned_addr learned my addr 172.21.15.32:0/2810800834 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f54377fe700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5438139370 0x7f543811ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.539+0000 7f542e7fc700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f543809f330 0x7f54381c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5436ffd700 1 -- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5438139370 msgr2=0x7f543811ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5436ffd700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5438139370 0x7f543811ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5436ffd700 1 -- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f543809f330 msgr2=0x7f54381c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5436ffd700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f543809f330 0x7f54381c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5436ffd700 1 -- 172.21.15.32:0/2810800834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200094a0 con 0x7f543809e850 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5436ffd700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 0x7f54381c86d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f542800c520 tx=0x7f542800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428004c80 con 0x7f543809e850 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428004e00 con 0x7f543809e850 2023-09-30T22:27:22.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5438134760 con 0x7f543809e850 2023-09-30T22:27:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428007310 con 0x7f543809e850 2023-09-30T22:27:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.540+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54381220d0 con 0x7f543809e850 2023-09-30T22:27:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.541+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5428007470 con 0x7f543809e850 2023-09-30T22:27:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.541+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f542803f340 con 0x7f543809e850 2023-09-30T22:27:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.541+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54380747f0 con 0x7f543809e850 2023-09-30T22:27:22.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.546+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54280123b0 con 0x7f543809e850 2023-09-30T22:27:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.655+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0a40074ea0 con 0x7f0a400a0a30 2023-09-30T22:27:22.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.656+0000 7f0a35ffb700 1 -- 172.21.15.32:0/1239529516 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a3c012e40 con 0x7f0a400a0a30 2023-09-30T22:27:22.657 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 msgr2=0x7f0a4011e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 0x7f0a4011e7a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c014040 tx=0x7f0a3c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 shutdown_connections 2023-09-30T22:27:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a40133fa0 0x7f0a4011f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a400a0a30 0x7f0a4011e7a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 --2- 172.21.15.32:0/1239529516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a400a1340 0x7f0a4011ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 >> 172.21.15.32:0/1239529516 conn(0x7f0a400a6550 msgr2=0x7f0a401b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 shutdown_connections 2023-09-30T22:27:22.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.658+0000 7f0a47802700 1 -- 172.21.15.32:0/1239529516 wait complete. 2023-09-30T22:27:22.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.753+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5438072820 con 0x7f543809e850 2023-09-30T22:27:22.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.753+0000 7f5434ff9700 1 -- 172.21.15.32:0/2810800834 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5428008040 con 0x7f543809e850 2023-09-30T22:27:22.754 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:22.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 msgr2=0x7f54381c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:22.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 0x7f54381c86d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f542800c520 tx=0x7f542800c830 comp rx=0 tx=0).stop 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 shutdown_connections 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5438139370 0x7f543811ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f543809e850 0x7f54381c86d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 --2- 172.21.15.32:0/2810800834 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f543809f330 0x7f54381c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 >> 172.21.15.32:0/2810800834 conn(0x7f54380a3f70 msgr2=0x7f5438136db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 shutdown_connections 2023-09-30T22:27:22.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:22.755+0000 7f543d9c4700 1 -- 172.21.15.32:0/2810800834 wait complete. 2023-09-30T22:27:22.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:23.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:23.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:23.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.449+0000 7fe5983b9700 1 Processor -- start 2023-09-30T22:27:23.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.450+0000 7fe5983b9700 1 -- start start 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe5983b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a0a30 0x7fe5900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe5983b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5900a1340 0x7fe590133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe5983b9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59009edd0 con 0x7fe590133fa0 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe5983b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59009e1f0 con 0x7fe5900a1340 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe5983b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59009dc50 con 0x7fe5900a0a30 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe595954700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5900a1340 0x7fe590133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596155700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a0a30 0x7fe5900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596956700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe590133fa0 0x7fe59013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46384/0 (socket says 172.21.15.32:46384) 2023-09-30T22:27:23.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596956700 1 -- 172.21.15.32:0/307477440 learned_addr learned my addr 172.21.15.32:0/307477440 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596155700 1 -- 172.21.15.32:0/307477440 >> v2:172.21.15.32:3300/0 conn(0x7fe5900a1340 msgr2=0x7fe590133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596155700 1 --2- 172.21.15.32:0/307477440 >> v2:172.21.15.32:3300/0 conn(0x7fe5900a1340 0x7fe590133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596155700 1 -- 172.21.15.32:0/307477440 >> v1:172.21.15.32:6789/0 conn(0x7fe590133fa0 legacy=0x7fe59013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596155700 1 -- 172.21.15.32:0/307477440 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900a1fe0 con 0x7fe5900a0a30 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe595153700 1 -- 172.21.15.32:0/307477440 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877920459 0 0) 0x7fe59009edd0 con 0x7fe590133fa0 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe595954700 1 --2- 172.21.15.32:0/307477440 >> v2:172.21.15.32:3300/0 conn(0x7fe5900a1340 0x7fe590133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.451+0000 7fe596155700 1 --2- 172.21.15.32:0/307477440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a0a30 0x7fe5900a0e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009840 tx=0x7fe58c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef0a7b477632f3f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.452+0000 7fe595153700 1 -- 172.21.15.32:0/307477440 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c011830 con 0x7fe5900a0a30 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.452+0000 7fe595153700 1 -- 172.21.15.32:0/307477440 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c011990 con 0x7fe5900a0a30 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.452+0000 7fe595153700 1 -- 172.21.15.32:0/307477440 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c011d20 con 0x7fe5900a0a30 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.452+0000 7fe5983b9700 1 -- 172.21.15.32:0/307477440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a0a30 msgr2=0x7fe5900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.452+0000 7fe5983b9700 1 --2- 172.21.15.32:0/307477440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a0a30 0x7fe5900a0e00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009840 tx=0x7fe58c009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.453+0000 7fe5983b9700 1 -- 172.21.15.32:0/307477440 shutdown_connections 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.453+0000 7fe5983b9700 1 --2- 172.21.15.32:0/307477440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a0a30 0x7fe5900a0e00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.453+0000 7fe5983b9700 1 --2- 172.21.15.32:0/307477440 >> v2:172.21.15.32:3300/0 conn(0x7fe5900a1340 0x7fe590133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.453+0000 7fe5983b9700 1 -- 172.21.15.32:0/307477440 >> 172.21.15.32:0/307477440 conn(0x7fe5900a6550 msgr2=0x7fe5900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.453+0000 7fe5983b9700 1 -- 172.21.15.32:0/307477440 shutdown_connections 2023-09-30T22:27:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.453+0000 7fe5983b9700 1 -- 172.21.15.32:0/307477440 wait complete. 2023-09-30T22:27:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 Processor -- start 2023-09-30T22:27:23.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 -- start start 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5900a0a30 0x7fe5901c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a1340 0x7fe5901c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe590133fa0 0x7fe590122850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590125d40 con 0x7fe590133fa0 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5901263e0 con 0x7fe5900a1340 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe5983b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900a35a0 con 0x7fe5900a0a30 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe596155700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5900a0a30 0x7fe5901c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe595954700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a1340 0x7fe5901c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe596956700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe590133fa0 0x7fe590122850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe596155700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5900a0a30 0x7fe5901c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:51996/0 (socket says 172.21.15.32:51996) 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.454+0000 7fe596155700 1 -- 172.21.15.32:0/2831362924 learned_addr learned my addr 172.21.15.32:0/2831362924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe595954700 1 -- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5900a0a30 msgr2=0x7fe5901c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe595954700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5900a0a30 0x7fe5901c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe595954700 1 -- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe590133fa0 msgr2=0x7fe590122850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe595954700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe590133fa0 0x7fe590122850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe595954700 1 -- 172.21.15.32:0/2831362924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800094a0 con 0x7fe5900a1340 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe596956700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe590133fa0 0x7fe590122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe595954700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a1340 0x7fe5901c8ad0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe58000e9b0 tx=0x7fe58000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000ee50 con 0x7fe5900a1340 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58000c850 con 0x7fe5900a1340 2023-09-30T22:27:23.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c0094a0 con 0x7fe5900a1340 2023-09-30T22:27:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000cc00 con 0x7fe5900a1340 2023-09-30T22:27:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.455+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5901243d0 con 0x7fe5900a1340 2023-09-30T22:27:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.456+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe580013470 con 0x7fe5900a1340 2023-09-30T22:27:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.456+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe58003fa70 con 0x7fe5900a1340 2023-09-30T22:27:23.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.456+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe590074ea0 con 0x7fe5900a1340 2023-09-30T22:27:23.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.461+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe580013720 con 0x7fe5900a1340 2023-09-30T22:27:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.488+0000 7fb46008e700 1 Processor -- start 2023-09-30T22:27:23.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.489+0000 7fb46008e700 1 -- start start 2023-09-30T22:27:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.489+0000 7fb46008e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.489+0000 7fb46008e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 0x7fb45813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.489+0000 7fb46008e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580a2a70 con 0x7fb45809d630 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.489+0000 7fb46008e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580a01d0 con 0x7fb4580a0a30 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.489+0000 7fb46008e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45809e2b0 con 0x7fb458138d80 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 0x7fb45813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 0x7fb45813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48438/0 (socket says 172.21.15.32:48438) 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 -- 172.21.15.32:0/2676635350 learned_addr learned my addr 172.21.15.32:0/2676635350 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45de2a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 -- 172.21.15.32:0/2676635350 >> v2:172.21.15.32:3300/0 conn(0x7fb4580a0a30 msgr2=0x7fb4580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 --2- 172.21.15.32:0/2676635350 >> v2:172.21.15.32:3300/0 conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 -- 172.21.15.32:0/2676635350 >> v1:172.21.15.32:6789/0 conn(0x7fb45809d630 legacy=0x7fb4580a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 -- 172.21.15.32:0/2676635350 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45809dc50 con 0x7fb458138d80 2023-09-30T22:27:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45e62b700 1 --2- 172.21.15.32:0/2676635350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 0x7fb45813b160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb4500098f0 tx=0x7fb450009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73189ee4de6edb80 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45ce28700 1 -- 172.21.15.32:0/2676635350 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450011830 con 0x7fb458138d80 2023-09-30T22:27:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45ce28700 1 -- 172.21.15.32:0/2676635350 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb450011990 con 0x7fb458138d80 2023-09-30T22:27:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.490+0000 7fb45ce28700 1 -- 172.21.15.32:0/2676635350 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450011d20 con 0x7fb458138d80 2023-09-30T22:27:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 -- 172.21.15.32:0/2676635350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 msgr2=0x7fb45813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 --2- 172.21.15.32:0/2676635350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 0x7fb45813b160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb4500098f0 tx=0x7fb450009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 -- 172.21.15.32:0/2676635350 shutdown_connections 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 --2- 172.21.15.32:0/2676635350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb458138d80 0x7fb45813b160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 --2- 172.21.15.32:0/2676635350 >> v2:172.21.15.32:3300/0 conn(0x7fb4580a0a30 0x7fb4580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 -- 172.21.15.32:0/2676635350 >> 172.21.15.32:0/2676635350 conn(0x7fb4580a3050 msgr2=0x7fb4580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 -- 172.21.15.32:0/2676635350 shutdown_connections 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.491+0000 7fb46008e700 1 -- 172.21.15.32:0/2676635350 wait complete. 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.493+0000 7fb46008e700 1 Processor -- start 2023-09-30T22:27:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.493+0000 7fb46008e700 1 -- start start 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb46008e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 0x7fb458120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb46008e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4580a0a30 0x7fb458120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb46008e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb458138d80 0x7fb4581210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb46008e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb458123e60 con 0x7fb4580a0a30 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb46008e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb458122ff0 con 0x7fb45809d630 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb46008e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb458124060 con 0x7fb458138d80 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 0x7fb458120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 0x7fb458120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48450/0 (socket says 172.21.15.32:48450) 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 -- 172.21.15.32:0/3797931275 learned_addr learned my addr 172.21.15.32:0/3797931275 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45d629700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4580a0a30 0x7fb458120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45e62b700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb458138d80 0x7fb4581210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 -- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb458138d80 msgr2=0x7fb4581210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb458138d80 0x7fb4581210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 -- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4580a0a30 msgr2=0x7fb458120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4580a0a30 0x7fb458120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 -- 172.21.15.32:0/3797931275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009550 con 0x7fb45809d630 2023-09-30T22:27:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.494+0000 7fb45de2a700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 0x7fb458120620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb44800efc0 tx=0x7fb44800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.495+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800cd10 con 0x7fb45809d630 2023-09-30T22:27:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.495+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800f040 con 0x7fb45809d630 2023-09-30T22:27:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.495+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb458123410 con 0x7fb45809d630 2023-09-30T22:27:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.495+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480146a0 con 0x7fb45809d630 2023-09-30T22:27:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.495+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581389a0 con 0x7fb45809d630 2023-09-30T22:27:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.496+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb448014800 con 0x7fb45809d630 2023-09-30T22:27:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.496+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45813b620 con 0x7fb45809d630 2023-09-30T22:27:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.496+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb44803fee0 con 0x7fb45809d630 2023-09-30T22:27:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.500+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb44801a6a0 con 0x7fb45809d630 2023-09-30T22:27:23.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.669+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe5900a28e0 con 0x7fe5900a1340 2023-09-30T22:27:23.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.669+0000 7fe586ffd700 1 -- 172.21.15.32:0/2831362924 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe580017020 con 0x7fe5900a1340 2023-09-30T22:27:23.670 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a1340 msgr2=0x7fe5901c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a1340 0x7fe5901c8ad0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe58000e9b0 tx=0x7fe58000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 shutdown_connections 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe590133fa0 0x7fe590122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5900a0a30 0x7fe5901c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 --2- 172.21.15.32:0/2831362924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5900a1340 0x7fe5901c8ad0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 >> 172.21.15.32:0/2831362924 conn(0x7fe5900a6550 msgr2=0x7fe59011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:23.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 shutdown_connections 2023-09-30T22:27:23.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.672+0000 7fe5983b9700 1 -- 172.21.15.32:0/2831362924 wait complete. 2023-09-30T22:27:23.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:23.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:23.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.706+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb458123cb0 con 0x7fb45809d630 2023-09-30T22:27:23.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.707+0000 7fb4567fc700 1 -- 172.21.15.32:0/3797931275 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb44801a2c0 con 0x7fb45809d630 2023-09-30T22:27:23.708 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 msgr2=0x7fb458120620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 0x7fb458120620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb44800efc0 tx=0x7fb44800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 shutdown_connections 2023-09-30T22:27:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb458138d80 0x7fb4581210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb45809d630 0x7fb458120620 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 --2- 172.21.15.32:0/3797931275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4580a0a30 0x7fb458120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 >> 172.21.15.32:0/3797931275 conn(0x7fb4580a3050 msgr2=0x7fb45809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 shutdown_connections 2023-09-30T22:27:23.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.709+0000 7fb46008e700 1 -- 172.21.15.32:0/3797931275 wait complete. 2023-09-30T22:27:23.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:23.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 Processor -- start 2023-09-30T22:27:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 -- start start 2023-09-30T22:27:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea0136710 0x7f9ea0138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9ea0139030 0x7f9ea013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea006ef30 con 0x7f9ea01338a0 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea006fef0 con 0x7f9ea0139030 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea65d5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01334b0 con 0x7f9ea0136710 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9e9ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ea01338a0 0x7f9ea0133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46402/0 (socket says 172.21.15.32:46402) 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9e9ffff700 1 -- 172.21.15.32:0/4123998300 learned_addr learned my addr 172.21.15.32:0/4123998300 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9e9f7fe700 1 --2- 172.21.15.32:0/4123998300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea0136710 0x7f9ea0138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.812+0000 7f9ea4b72700 1 --2- 172.21.15.32:0/4123998300 >> v2:172.21.15.32:3300/0 conn(0x7f9ea0139030 0x7f9ea013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9effd700 1 -- 172.21.15.32:0/4123998300 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104329060 0 0) 0x7f9ea006ef30 con 0x7f9ea01338a0 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9effd700 1 -- 172.21.15.32:0/4123998300 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e880035b0 con 0x7f9ea01338a0 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9f7fe700 1 -- 172.21.15.32:0/4123998300 >> v2:172.21.15.32:3300/0 conn(0x7f9ea0139030 msgr2=0x7f9ea013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9f7fe700 1 --2- 172.21.15.32:0/4123998300 >> v2:172.21.15.32:3300/0 conn(0x7f9ea0139030 0x7f9ea013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9f7fe700 1 -- 172.21.15.32:0/4123998300 >> v1:172.21.15.32:6789/0 conn(0x7f9ea01338a0 legacy=0x7f9ea0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9f7fe700 1 -- 172.21.15.32:0/4123998300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea006fc30 con 0x7f9ea0136710 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9ea4b72700 1 --2- 172.21.15.32:0/4123998300 >> v2:172.21.15.32:3300/0 conn(0x7f9ea0139030 0x7f9ea013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9f7fe700 1 --2- 172.21.15.32:0/4123998300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea0136710 0x7f9ea0138af0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009840 tx=0x7f9e94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d0b0f3e0e1b751 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9effd700 1 -- 172.21.15.32:0/4123998300 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94011830 con 0x7f9ea0136710 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9effd700 1 -- 172.21.15.32:0/4123998300 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94011990 con 0x7f9ea0136710 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.813+0000 7f9e9effd700 1 -- 172.21.15.32:0/4123998300 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94011d20 con 0x7f9ea0136710 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 -- 172.21.15.32:0/4123998300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea0136710 msgr2=0x7f9ea0138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/4123998300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea0136710 0x7f9ea0138af0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009840 tx=0x7f9e94009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 -- 172.21.15.32:0/4123998300 shutdown_connections 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/4123998300 >> v2:172.21.15.32:3300/0 conn(0x7f9ea0139030 0x7f9ea013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/4123998300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea0136710 0x7f9ea0138af0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 -- 172.21.15.32:0/4123998300 >> 172.21.15.32:0/4123998300 conn(0x7f9ea00a2aa0 msgr2=0x7f9ea00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 -- 172.21.15.32:0/4123998300 shutdown_connections 2023-09-30T22:27:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.814+0000 7f9ea65d5700 1 -- 172.21.15.32:0/4123998300 wait complete. 2023-09-30T22:27:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.815+0000 7f9ea65d5700 1 Processor -- start 2023-09-30T22:27:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 -- start start 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea01338a0 0x7f9ea011ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ea0136710 0x7f9ea011b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea0139030 0x7f9ea011b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea011e770 con 0x7f9ea0139030 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea006f640 con 0x7f9ea01338a0 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.816+0000 7f9ea65d5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00a22b0 con 0x7f9ea0136710 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9ea4b72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea0139030 0x7f9ea011b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea01338a0 0x7f9ea011ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9ea4b72700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea0139030 0x7f9ea011b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33576/0 (socket says 172.21.15.32:33576) 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9ea4b72700 1 -- 172.21.15.32:0/340218804 learned_addr learned my addr 172.21.15.32:0/340218804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9f7fe700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ea0136710 0x7f9ea011b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 -- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ea0136710 msgr2=0x7f9ea011b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ea0136710 0x7f9ea011b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 -- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea0139030 msgr2=0x7f9ea011b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea0139030 0x7f9ea011b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 -- 172.21.15.32:0/340218804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98009a70 con 0x7f9ea01338a0 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9ffff700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea01338a0 0x7f9ea011ac50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c490 tx=0x7f9e9000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90005d70 con 0x7f9ea01338a0 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e900070d0 con 0x7f9ea01338a0 2023-09-30T22:27:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90007440 con 0x7f9ea01338a0 2023-09-30T22:27:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.817+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e940094a0 con 0x7f9ea01338a0 2023-09-30T22:27:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.818+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea011e1d0 con 0x7f9ea01338a0 2023-09-30T22:27:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.818+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e900075a0 con 0x7f9ea01338a0 2023-09-30T22:27:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.819+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0074ea0 con 0x7f9ea01338a0 2023-09-30T22:27:23.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.819+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e9003e710 con 0x7f9ea01338a0 2023-09-30T22:27:23.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.824+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e90018af0 con 0x7f9ea01338a0 2023-09-30T22:27:23.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.910+0000 7f77204d7700 1 Processor -- start 2023-09-30T22:27:23.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.911+0000 7f77204d7700 1 -- start start 2023-09-30T22:27:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f77204d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 0x7f7718136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f77204d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7718136a70 0x7f7718138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f77204d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771809e130 con 0x7f77181316c0 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f77204d7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77181321a0 con 0x7f7718136a70 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f77204d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77181312b0 con 0x7f771809d9f0 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 0x7f7718136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 0x7f7718136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48460/0 (socket says 172.21.15.32:48460) 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771ea74700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7718136a70 0x7f7718138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 -- 172.21.15.32:0/4132093464 learned_addr learned my addr 172.21.15.32:0/4132093464 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 -- 172.21.15.32:0/4132093464 >> v2:172.21.15.32:3300/0 conn(0x7f7718136a70 msgr2=0x7f7718138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 --2- 172.21.15.32:0/4132093464 >> v2:172.21.15.32:3300/0 conn(0x7f7718136a70 0x7f7718138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 -- 172.21.15.32:0/4132093464 >> v1:172.21.15.32:6789/0 conn(0x7f77181316c0 legacy=0x7f7718131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 -- 172.21.15.32:0/4132093464 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771809f6e0 con 0x7f771809d9f0 2023-09-30T22:27:23.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771ea74700 1 --2- 172.21.15.32:0/4132093464 >> v2:172.21.15.32:3300/0 conn(0x7f7718136a70 0x7f7718138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:23.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.912+0000 7f771da72700 1 --2- 172.21.15.32:0/4132093464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 0x7f7718136530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f77080098f0 tx=0x7f7708009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27b7e3a894a7b43f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.913+0000 7f771d271700 1 -- 172.21.15.32:0/4132093464 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708011830 con 0x7f771809d9f0 2023-09-30T22:27:23.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.913+0000 7f771d271700 1 -- 172.21.15.32:0/4132093464 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7708011990 con 0x7f771809d9f0 2023-09-30T22:27:23.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.913+0000 7f771d271700 1 -- 172.21.15.32:0/4132093464 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708011d20 con 0x7f771809d9f0 2023-09-30T22:27:23.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.913+0000 7f77204d7700 1 -- 172.21.15.32:0/4132093464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 msgr2=0x7f7718136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.913+0000 7f77204d7700 1 --2- 172.21.15.32:0/4132093464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 0x7f7718136530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f77080098f0 tx=0x7f7708009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:23.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.914+0000 7f77204d7700 1 -- 172.21.15.32:0/4132093464 shutdown_connections 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.914+0000 7f77204d7700 1 --2- 172.21.15.32:0/4132093464 >> v2:172.21.15.32:3300/0 conn(0x7f7718136a70 0x7f7718138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.914+0000 7f77204d7700 1 --2- 172.21.15.32:0/4132093464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f771809d9f0 0x7f7718136530 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.914+0000 7f77204d7700 1 -- 172.21.15.32:0/4132093464 >> 172.21.15.32:0/4132093464 conn(0x7f7718066f50 msgr2=0x7f771809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.914+0000 7f77204d7700 1 -- 172.21.15.32:0/4132093464 shutdown_connections 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.914+0000 7f77204d7700 1 -- 172.21.15.32:0/4132093464 wait complete. 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 Processor -- start 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 -- start start 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f771809d9f0 0x7f77181c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77181316c0 0x7f77181c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 0x7f77181c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718135970 con 0x7f7718136a70 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77181361d0 con 0x7f77181316c0 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f77204d7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f771812b900 con 0x7f771809d9f0 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f771ea74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 0x7f77181c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f771ea74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 0x7f77181c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33604/0 (socket says 172.21.15.32:33604) 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f771ea74700 1 -- 172.21.15.32:0/1968112243 learned_addr learned my addr 172.21.15.32:0/1968112243 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f771da72700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77181316c0 0x7f77181c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.915+0000 7f771e273700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f771809d9f0 0x7f77181c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f771ea74700 1 -- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f771809d9f0 msgr2=0x7f77181c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f771ea74700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f771809d9f0 0x7f77181c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f771ea74700 1 -- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77181316c0 msgr2=0x7f77181c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f771ea74700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77181316c0 0x7f77181c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f771ea74700 1 -- 172.21.15.32:0/1968112243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100094a0 con 0x7f7718136a70 2023-09-30T22:27:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f771ea74700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 0x7f77181c6db0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f771000eb30 tx=0x7f771000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710009780 con 0x7f7718136a70 2023-09-30T22:27:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710009e60 con 0x7f7718136a70 2023-09-30T22:27:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000c9f0 con 0x7f7718136a70 2023-09-30T22:27:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7708009550 con 0x7f7718136a70 2023-09-30T22:27:23.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.916+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7718138e60 con 0x7f7718136a70 2023-09-30T22:27:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.917+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f771000cb50 con 0x7f7718136a70 2023-09-30T22:27:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.917+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718074ea0 con 0x7f7718136a70 2023-09-30T22:27:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.917+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f771003fe20 con 0x7f7718136a70 2023-09-30T22:27:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:23.922+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7710011350 con 0x7f7718136a70 2023-09-30T22:27:24.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.032+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9ea011d050 con 0x7f9ea01338a0 2023-09-30T22:27:24.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.033+0000 7f9e9cff9700 1 -- 172.21.15.32:0/340218804 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e90017030 con 0x7f9ea01338a0 2023-09-30T22:27:24.037 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea01338a0 msgr2=0x7f9ea011ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea01338a0 0x7f9ea011ac50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000c490 tx=0x7f9e9000c7a0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 shutdown_connections 2023-09-30T22:27:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ea0139030 0x7f9ea011b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9ea01338a0 0x7f9ea011ac50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 --2- 172.21.15.32:0/340218804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9ea0136710 0x7f9ea011b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 >> 172.21.15.32:0/340218804 conn(0x7f9ea00a2aa0 msgr2=0x7f9ea01344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 shutdown_connections 2023-09-30T22:27:24.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.035+0000 7f9ea65d5700 1 -- 172.21.15.32:0/340218804 wait complete. 2023-09-30T22:27:24.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.127+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f771806fe80 con 0x7f7718136a70 2023-09-30T22:27:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.127+0000 7f770effd700 1 -- 172.21.15.32:0/1968112243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7710017070 con 0x7f7718136a70 2023-09-30T22:27:24.129 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 msgr2=0x7f77181c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 0x7f77181c6db0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f771000eb30 tx=0x7f771000ee40 comp rx=0 tx=0).stop 2023-09-30T22:27:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 shutdown_connections 2023-09-30T22:27:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7718136a70 0x7f77181c6db0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f771809d9f0 0x7f77181c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 --2- 172.21.15.32:0/1968112243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77181316c0 0x7f77181c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 >> 172.21.15.32:0/1968112243 conn(0x7f7718066f50 msgr2=0x7f7718136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 shutdown_connections 2023-09-30T22:27:24.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.130+0000 7f77204d7700 1 -- 172.21.15.32:0/1968112243 wait complete. 2023-09-30T22:27:24.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:24.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:24.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:24.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 Processor -- start 2023-09-30T22:27:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 -- start start 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6240a0a30 0x7ff6240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 0x7ff624133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62409edd0 con 0x7ff624133fa0 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62409e1f0 con 0x7ff6240a0a30 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff62aa57700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62409dc50 con 0x7ff6240a1340 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff6237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 0x7ff624133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff6237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 0x7ff624133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48466/0 (socket says 172.21.15.32:48466) 2023-09-30T22:27:24.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff623fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6240a0a30 0x7ff6240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff6237fe700 1 -- 172.21.15.32:0/2595997574 learned_addr learned my addr 172.21.15.32:0/2595997574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.835+0000 7ff628ff4700 1 --1- 172.21.15.32:0/2595997574 >> v1:172.21.15.32:6789/0 conn(0x7ff624133fa0 0x7ff62413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46406/0 (socket says 172.21.15.32:46406) 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff623fff700 1 -- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 msgr2=0x7ff624133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff623fff700 1 --2- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 0x7ff624133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff623fff700 1 -- 172.21.15.32:0/2595997574 >> v1:172.21.15.32:6789/0 conn(0x7ff624133fa0 legacy=0x7ff62413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff623fff700 1 -- 172.21.15.32:0/2595997574 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6240a1fe0 con 0x7ff6240a0a30 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff6237fe700 1 --2- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 0x7ff624133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff623fff700 1 --2- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff6240a0e00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff614009840 tx=0x7ff614009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b775d115d20f7bbc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff622ffd700 1 -- 172.21.15.32:0/2595997574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614011830 con 0x7ff6240a0a30 2023-09-30T22:27:24.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff622ffd700 1 -- 172.21.15.32:0/2595997574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614011990 con 0x7ff6240a0a30 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.836+0000 7ff622ffd700 1 -- 172.21.15.32:0/2595997574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614011d20 con 0x7ff6240a0a30 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 -- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 msgr2=0x7ff6240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff6240a0e00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff614009840 tx=0x7ff614009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 -- 172.21.15.32:0/2595997574 shutdown_connections 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff6240a0e00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2595997574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6240a1340 0x7ff624133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 -- 172.21.15.32:0/2595997574 >> 172.21.15.32:0/2595997574 conn(0x7ff6240a6550 msgr2=0x7ff6240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 -- 172.21.15.32:0/2595997574 shutdown_connections 2023-09-30T22:27:24.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.837+0000 7ff62aa57700 1 -- 172.21.15.32:0/2595997574 wait complete. 2023-09-30T22:27:24.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.838+0000 7ff62aa57700 1 Processor -- start 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.838+0000 7ff62aa57700 1 -- start start 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff62aa57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff62411e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff62aa57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 0x7ff62411ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff62aa57700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff624133fa0 0x7ff62411f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff62aa57700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff624121f50 con 0x7ff6240a0a30 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff62aa57700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff624120430 con 0x7ff624133fa0 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff62aa57700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240a37e0 con 0x7ff6240a1340 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff6237fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 0x7ff62411ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff6237fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 0x7ff62411ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52026/0 (socket says 172.21.15.32:52026) 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff623fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff62411e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff6237fe700 1 -- 172.21.15.32:0/2116806372 learned_addr learned my addr 172.21.15.32:0/2116806372 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff628ff4700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff624133fa0 0x7ff62411f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff628ff4700 1 -- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 msgr2=0x7ff62411ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff628ff4700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 0x7ff62411ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff628ff4700 1 -- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 msgr2=0x7ff62411e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff628ff4700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff62411e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff628ff4700 1 -- 172.21.15.32:0/2116806372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618009550 con 0x7ff624133fa0 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff623fff700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff62411e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:24.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.839+0000 7ff6237fe700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 0x7ff62411ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.840+0000 7ff628ff4700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff624133fa0 0x7ff62411f220 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00cb40 tx=0x7ff61c00ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.840+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00a9a0 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.840+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c00ab00 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.840+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6140094a0 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.840+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00ae70 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.840+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6240a0270 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.841+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff61c007070 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.841+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff624122a80 con 0x7ff624133fa0 2023-09-30T22:27:24.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.841+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff61c03f2e0 con 0x7ff624133fa0 2023-09-30T22:27:24.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.846+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff61c017030 con 0x7ff624133fa0 2023-09-30T22:27:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.873+0000 7fdc62b9d700 1 Processor -- start 2023-09-30T22:27:24.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.873+0000 7fdc62b9d700 1 -- start start 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc62b9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09cbb0 0x7fdc5c09cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc62b9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c133ac0 0x7fdc5c13b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc62b9d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5c09b0e0 con 0x7fdc5c09d4c0 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc62b9d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c09b370 con 0x7fdc5c09cbb0 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc62b9d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c09a1f0 con 0x7fdc5c133ac0 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc6239c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c133ac0 0x7fdc5c13b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc6139a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc5c09d4c0 0x7fdc5c1333b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46420/0 (socket says 172.21.15.32:46420) 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc6139a700 1 -- 172.21.15.32:0/1142480988 learned_addr learned my addr 172.21.15.32:0/1142480988 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.874+0000 7fdc61b9b700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09cbb0 0x7fdc5c09cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc60b99700 1 -- 172.21.15.32:0/1142480988 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643713861 0 0) 0x7fdc5c09b0e0 con 0x7fdc5c09d4c0 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc60b99700 1 -- 172.21.15.32:0/1142480988 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc480035b0 con 0x7fdc5c09d4c0 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc61b9b700 1 -- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c133ac0 msgr2=0x7fdc5c13b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc61b9b700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c133ac0 0x7fdc5c13b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc61b9b700 1 -- 172.21.15.32:0/1142480988 >> v1:172.21.15.32:6789/0 conn(0x7fdc5c09d4c0 legacy=0x7fdc5c1333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc61b9b700 1 -- 172.21.15.32:0/1142480988 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c099d10 con 0x7fdc5c09cbb0 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc6239c700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c133ac0 0x7fdc5c13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc61b9b700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09cbb0 0x7fdc5c09cf80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009870 tx=0x7fdc58009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501da2c4d2dd34b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc60b99700 1 -- 172.21.15.32:0/1142480988 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58011830 con 0x7fdc5c09cbb0 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc60b99700 1 -- 172.21.15.32:0/1142480988 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc58011990 con 0x7fdc5c09cbb0 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.875+0000 7fdc60b99700 1 -- 172.21.15.32:0/1142480988 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58011d20 con 0x7fdc5c09cbb0 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 -- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09cbb0 msgr2=0x7fdc5c09cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09cbb0 0x7fdc5c09cf80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009870 tx=0x7fdc58009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 -- 172.21.15.32:0/1142480988 shutdown_connections 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c133ac0 0x7fdc5c13b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/1142480988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09cbb0 0x7fdc5c09cf80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 -- 172.21.15.32:0/1142480988 >> 172.21.15.32:0/1142480988 conn(0x7fdc5c0a1fa0 msgr2=0x7fdc5c0a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 -- 172.21.15.32:0/1142480988 shutdown_connections 2023-09-30T22:27:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.876+0000 7fdc62b9d700 1 -- 172.21.15.32:0/1142480988 wait complete. 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.877+0000 7fdc62b9d700 1 Processor -- start 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.877+0000 7fdc62b9d700 1 -- start start 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc62b9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c09cbb0 0x7fdc5c11ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc62b9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09d4c0 0x7fdc5c11f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc62b9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 0x7fdc5c11f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc62b9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1208e0 con 0x7fdc5c133ac0 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc62b9d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c121a90 con 0x7fdc5c09d4c0 2023-09-30T22:27:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc62b9d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c09b850 con 0x7fdc5c09cbb0 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc6239c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 0x7fdc5c11f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc6239c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 0x7fdc5c11f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33634/0 (socket says 172.21.15.32:33634) 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc6139a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09d4c0 0x7fdc5c11f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc6239c700 1 -- 172.21.15.32:0/609530107 learned_addr learned my addr 172.21.15.32:0/609530107 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.878+0000 7fdc61b9b700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c09cbb0 0x7fdc5c11ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6239c700 1 -- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c09cbb0 msgr2=0x7fdc5c11ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6239c700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c09cbb0 0x7fdc5c11ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6239c700 1 -- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09d4c0 msgr2=0x7fdc5c11f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6239c700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09d4c0 0x7fdc5c11f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6239c700 1 -- 172.21.15.32:0/609530107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc54009a70 con 0x7fdc5c133ac0 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6139a700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09d4c0 0x7fdc5c11f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc6239c700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 0x7fdc5c11f640 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400efc0 tx=0x7fdc5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400ca90 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5400f040 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc580094a0 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400ce10 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.879+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c1226e0 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.880+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc54014480 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.880+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdc5403fe40 con 0x7fdc5c133ac0 2023-09-30T22:27:24.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.880+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5c04f2a0 con 0x7fdc5c133ac0 2023-09-30T22:27:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:24.885+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc54012930 con 0x7fdc5c133ac0 2023-09-30T22:27:25.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:25.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.055+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff624074ea0 con 0x7ff624133fa0 2023-09-30T22:27:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.055+0000 7ff620ff9700 1 -- 172.21.15.32:0/2116806372 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff61c0196d0 con 0x7ff624133fa0 2023-09-30T22:27:25.056 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff624133fa0 msgr2=0x7ff62411f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff624133fa0 0x7ff62411f220 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00cb40 tx=0x7ff61c00ce50 comp rx=0 tx=0).stop 2023-09-30T22:27:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 shutdown_connections 2023-09-30T22:27:25.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff624133fa0 0x7ff62411f220 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6240a0a30 0x7ff62411e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 --2- 172.21.15.32:0/2116806372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6240a1340 0x7ff62411ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 >> 172.21.15.32:0/2116806372 conn(0x7ff6240a6550 msgr2=0x7ff6241b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 shutdown_connections 2023-09-30T22:27:25.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.058+0000 7ff62aa57700 1 -- 172.21.15.32:0/2116806372 wait complete. 2023-09-30T22:27:25.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.096+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdc5c09a410 con 0x7fdc5c133ac0 2023-09-30T22:27:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.096+0000 7fdc527fc700 1 -- 172.21.15.32:0/609530107 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc54012340 con 0x7fdc5c133ac0 2023-09-30T22:27:25.097 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 msgr2=0x7fdc5c11f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 0x7fdc5c11f640 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400efc0 tx=0x7fdc5400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 shutdown_connections 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc5c133ac0 0x7fdc5c11f640 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdc5c09cbb0 0x7fdc5c11ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 --2- 172.21.15.32:0/609530107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdc5c09d4c0 0x7fdc5c11f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 >> 172.21.15.32:0/609530107 conn(0x7fdc5c0a1fa0 msgr2=0x7fdc5c1baa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 shutdown_connections 2023-09-30T22:27:25.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.099+0000 7fdc62b9d700 1 -- 172.21.15.32:0/609530107 wait complete. 2023-09-30T22:27:25.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:25.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:25.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.193+0000 7f171c3be700 1 Processor -- start 2023-09-30T22:27:25.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171c3be700 1 -- start start 2023-09-30T22:27:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171c3be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714136820 0x7f1714138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171c3be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714139140 0x7f171413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171c3be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171406bc00 con 0x7f1714133a80 2023-09-30T22:27:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171c3be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171406c570 con 0x7f1714136820 2023-09-30T22:27:25.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171c3be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17140a0040 con 0x7f1714139140 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171a95b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714139140 0x7f171413b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171a15a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1714133a80 0x7f1714133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46426/0 (socket says 172.21.15.32:46426) 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f171a15a700 1 -- 172.21.15.32:0/3422536580 learned_addr learned my addr 172.21.15.32:0/3422536580 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.194+0000 7f1719959700 1 --2- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714136820 0x7f1714138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f171a95b700 1 -- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714136820 msgr2=0x7f1714138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f1719158700 1 -- 172.21.15.32:0/3422536580 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594383499 0 0) 0x7f171406bc00 con 0x7f1714133a80 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f171a95b700 1 --2- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714136820 0x7f1714138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f171a95b700 1 -- 172.21.15.32:0/3422536580 >> v1:172.21.15.32:6789/0 conn(0x7f1714133a80 legacy=0x7f1714133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f171a95b700 1 -- 172.21.15.32:0/3422536580 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714133860 con 0x7f1714139140 2023-09-30T22:27:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f171a95b700 1 --2- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714139140 0x7f171413b520 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f170c009dc0 tx=0x7f170c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985e3c21c4052faa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f1719158700 1 -- 172.21.15.32:0/3422536580 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00e070 con 0x7f1714139140 2023-09-30T22:27:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f1719158700 1 -- 172.21.15.32:0/3422536580 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00bb20 con 0x7f1714139140 2023-09-30T22:27:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.195+0000 7f1719158700 1 -- 172.21.15.32:0/3422536580 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c012430 con 0x7f1714139140 2023-09-30T22:27:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 -- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714139140 msgr2=0x7f171413b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 --2- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714139140 0x7f171413b520 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f170c009dc0 tx=0x7f170c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 -- 172.21.15.32:0/3422536580 shutdown_connections 2023-09-30T22:27:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 --2- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714139140 0x7f171413b520 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 --2- 172.21.15.32:0/3422536580 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714136820 0x7f1714138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 -- 172.21.15.32:0/3422536580 >> 172.21.15.32:0/3422536580 conn(0x7f17140a4210 msgr2=0x7f17140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 -- 172.21.15.32:0/3422536580 shutdown_connections 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.196+0000 7f171c3be700 1 -- 172.21.15.32:0/3422536580 wait complete. 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.197+0000 7f171c3be700 1 Processor -- start 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.197+0000 7f171c3be700 1 -- start start 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171c3be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 0x7f171406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171c3be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714136820 0x7f171406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171c3be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 0x7f17140703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171c3be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1714072290 con 0x7f1714133a80 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171c3be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1714072470 con 0x7f1714139140 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171c3be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f171409baf0 con 0x7f1714136820 2023-09-30T22:27:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 0x7f171406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a95b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 0x7f17140703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a95b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 0x7f17140703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48486/0 (socket says 172.21.15.32:48486) 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 0x7f171406f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33644/0 (socket says 172.21.15.32:33644) 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a95b700 1 -- 172.21.15.32:0/3600712912 learned_addr learned my addr 172.21.15.32:0/3600712912 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f1719959700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714136820 0x7f171406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 -- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714136820 msgr2=0x7f171406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714136820 0x7f171406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 -- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 msgr2=0x7f17140703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 0x7f17140703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 -- 172.21.15.32:0/3600712912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c009a70 con 0x7f1714133a80 2023-09-30T22:27:25.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a95b700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 0x7f17140703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f1719959700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714136820 0x7f171406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.198+0000 7f171a15a700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 0x7f171406f950 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f170400c4a0 tx=0x7f170400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.199+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704004c80 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.199+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1714070ac0 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.199+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1704004de0 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.199+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704007310 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.199+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171409e2a0 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.200+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1704007470 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.200+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f170403f260 con 0x7f1714133a80 2023-09-30T22:27:25.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.200+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17141309f0 con 0x7f1714133a80 2023-09-30T22:27:25.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.205+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1704007720 con 0x7f1714133a80 2023-09-30T22:27:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.291+0000 7f5e7c789700 1 Processor -- start 2023-09-30T22:27:25.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7c789700 1 -- start start 2023-09-30T22:27:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7c789700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e74138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7c789700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 0x7f5e7413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7c789700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7409edd0 con 0x7f5e740a0a30 2023-09-30T22:27:25.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7c789700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740a18e0 con 0x7f5e740a1510 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7c789700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7409e1f0 con 0x7f5e74139250 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7ad26700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 0x7f5e7413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7ad26700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 0x7f5e7413b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52066/0 (socket says 172.21.15.32:52066) 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7ad26700 1 -- 172.21.15.32:0/2814313437 learned_addr learned my addr 172.21.15.32:0/2814313437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.292+0000 7f5e7a525700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5e740a0a30 0x7f5e740a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46432/0 (socket says 172.21.15.32:46432) 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e79d24700 1 --2- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e74138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e7ad26700 1 -- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 msgr2=0x7f5e74138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e7ad26700 1 --2- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e74138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e7ad26700 1 -- 172.21.15.32:0/2814313437 >> v1:172.21.15.32:6789/0 conn(0x7f5e740a0a30 legacy=0x7f5e740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e79523700 1 -- 172.21.15.32:0/2814313437 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548024126 0 0) 0x7f5e7409edd0 con 0x7f5e740a0a30 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e7ad26700 1 -- 172.21.15.32:0/2814313437 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7409dc50 con 0x7f5e74139250 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e7ad26700 1 --2- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 0x7f5e7413b630 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009920 tx=0x7f5e6c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7344623ed00c61b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e79523700 1 -- 172.21.15.32:0/2814313437 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011830 con 0x7f5e74139250 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e79523700 1 -- 172.21.15.32:0/2814313437 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c011990 con 0x7f5e74139250 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.293+0000 7f5e79523700 1 -- 172.21.15.32:0/2814313437 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011d20 con 0x7f5e74139250 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 -- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 msgr2=0x7f5e7413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 --2- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 0x7f5e7413b630 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009920 tx=0x7f5e6c009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 -- 172.21.15.32:0/2814313437 shutdown_connections 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 --2- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e74139250 0x7f5e7413b630 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 --2- 172.21.15.32:0/2814313437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e74138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 -- 172.21.15.32:0/2814313437 >> 172.21.15.32:0/2814313437 conn(0x7f5e740a6550 msgr2=0x7f5e740a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:25.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 -- 172.21.15.32:0/2814313437 shutdown_connections 2023-09-30T22:27:25.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.294+0000 7f5e7c789700 1 -- 172.21.15.32:0/2814313437 wait complete. 2023-09-30T22:27:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 Processor -- start 2023-09-30T22:27:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 -- start start 2023-09-30T22:27:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e740a0a30 0x7f5e7411e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e7411ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 0x7f5e7411f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e741205c0 con 0x7f5e74139250 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74121e20 con 0x7f5e740a1510 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.295+0000 7f5e7c789700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74120a60 con 0x7f5e740a0a30 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e7411ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e7ad26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 0x7f5e7411f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e7411ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48502/0 (socket says 172.21.15.32:48502) 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e7ad26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 0x7f5e7411f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33646/0 (socket says 172.21.15.32:33646) 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 -- 172.21.15.32:0/1693429182 learned_addr learned my addr 172.21.15.32:0/1693429182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e7a525700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e740a0a30 0x7f5e7411e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 -- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e740a0a30 msgr2=0x7f5e7411e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e740a0a30 0x7f5e7411e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 -- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 msgr2=0x7f5e7411f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 0x7f5e7411f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 -- 172.21.15.32:0/1693429182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c009550 con 0x7f5e740a1510 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e7ad26700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 0x7f5e7411f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e79d24700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e7411ee70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800e9b0 tx=0x7f5e6800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68009710 con 0x7f5e740a1510 2023-09-30T22:27:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6800ee50 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7409ca10 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.296+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800ca50 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.297+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e74120ca0 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.297+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e6800cbb0 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.298+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e74074ea0 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.298+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5e6803fa90 con 0x7f5e740a1510 2023-09-30T22:27:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.303+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e6801b3b0 con 0x7f5e740a1510 2023-09-30T22:27:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.411+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1714074ea0 con 0x7f1714133a80 2023-09-30T22:27:25.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.411+0000 7f1712ffd700 1 -- 172.21.15.32:0/3600712912 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1704011e80 con 0x7f1714133a80 2023-09-30T22:27:25.412 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.413+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 msgr2=0x7f171406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.413+0000 7f171c3be700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 0x7f171406f950 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f170400c4a0 tx=0x7f170400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 shutdown_connections 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1714139140 0x7f17140703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1714133a80 0x7f171406f950 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 --2- 172.21.15.32:0/3600712912 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1714136820 0x7f171406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 >> 172.21.15.32:0/3600712912 conn(0x7f17140a4210 msgr2=0x7f17140a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 shutdown_connections 2023-09-30T22:27:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.414+0000 7f171c3be700 1 -- 172.21.15.32:0/3600712912 wait complete. 2023-09-30T22:27:25.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.506+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5e740a28e0 con 0x7f5e740a1510 2023-09-30T22:27:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.506+0000 7f5e72ffd700 1 -- 172.21.15.32:0/1693429182 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e68017070 con 0x7f5e740a1510 2023-09-30T22:27:25.508 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 msgr2=0x7f5e7411ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e7411ee70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800e9b0 tx=0x7f5e6800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 shutdown_connections 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e74139250 0x7f5e7411f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e740a0a30 0x7f5e7411e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 --2- 172.21.15.32:0/1693429182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e740a1510 0x7f5e7411ee70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:25.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 >> 172.21.15.32:0/1693429182 conn(0x7f5e740a6550 msgr2=0x7f5e740a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 shutdown_connections 2023-09-30T22:27:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:25.509+0000 7f5e7c789700 1 -- 172.21.15.32:0/1693429182 wait complete. 2023-09-30T22:27:25.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:26.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:26.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:26.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.225+0000 7fa997764700 1 Processor -- start 2023-09-30T22:27:26.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.225+0000 7fa997764700 1 -- start start 2023-09-30T22:27:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa997764700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009e850 0x7fa99009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa997764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa990133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa997764700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99006bf30 con 0x7fa99009f5b0 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa997764700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99009ffb0 con 0x7fa99009f160 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa997764700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa99006cb10 con 0x7fa99009e850 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995500700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009e850 0x7fa99009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995d01700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa99009f5b0 0x7fa990134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46444/0 (socket says 172.21.15.32:46444) 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995d01700 1 -- 172.21.15.32:0/2092619145 learned_addr learned my addr 172.21.15.32:0/2092619145 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa994cff700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa990133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa98ffff700 1 -- 172.21.15.32:0/2092619145 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680907391 0 0) 0x7fa99006bf30 con 0x7fa99009f5b0 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa98ffff700 1 -- 172.21.15.32:0/2092619145 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9780035b0 con 0x7fa99009f5b0 2023-09-30T22:27:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995500700 1 -- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 msgr2=0x7fa990133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995500700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa990133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995500700 1 -- 172.21.15.32:0/2092619145 >> v1:172.21.15.32:6789/0 conn(0x7fa99009f5b0 legacy=0x7fa990134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995500700 1 -- 172.21.15.32:0/2092619145 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900a0270 con 0x7fa99009e850 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa994cff700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa990133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.226+0000 7fa995500700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009e850 0x7fa99009ec20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa980009dc0 tx=0x7fa98000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46953e70c65429d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa98ffff700 1 -- 172.21.15.32:0/2092619145 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000e070 con 0x7fa99009e850 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa98ffff700 1 -- 172.21.15.32:0/2092619145 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000bb20 con 0x7fa99009e850 2023-09-30T22:27:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa98ffff700 1 -- 172.21.15.32:0/2092619145 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980012430 con 0x7fa99009e850 2023-09-30T22:27:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 -- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009e850 msgr2=0x7fa99009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009e850 0x7fa99009ec20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa980009dc0 tx=0x7fa98000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 -- 172.21.15.32:0/2092619145 shutdown_connections 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009e850 0x7fa99009ec20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 --2- 172.21.15.32:0/2092619145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa990133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 -- 172.21.15.32:0/2092619145 >> 172.21.15.32:0/2092619145 conn(0x7fa9900a3f70 msgr2=0x7fa9900a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 -- 172.21.15.32:0/2092619145 shutdown_connections 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.227+0000 7fa997764700 1 -- 172.21.15.32:0/2092619145 wait complete. 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.229+0000 7fa997764700 1 Processor -- start 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.229+0000 7fa997764700 1 -- start start 2023-09-30T22:27:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa997764700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99009e850 0x7fa990122390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa997764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa9901228d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa997764700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009f5b0 0x7fa990122e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa997764700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa990125eb0 con 0x7fa99009e850 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa997764700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa990125910 con 0x7fa99009f160 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa997764700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa990124040 con 0x7fa99009f5b0 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa995d01700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009f5b0 0x7fa990122e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa9901228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa995d01700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009f5b0 0x7fa990122e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52092/0 (socket says 172.21.15.32:52092) 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa995d01700 1 -- 172.21.15.32:0/3770096576 learned_addr learned my addr 172.21.15.32:0/3770096576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa9901228d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48524/0 (socket says 172.21.15.32:48524) 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa995500700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99009e850 0x7fa990122390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 -- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009f5b0 msgr2=0x7fa990122e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009f5b0 0x7fa990122e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 -- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99009e850 msgr2=0x7fa990122390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99009e850 0x7fa990122390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 -- 172.21.15.32:0/3770096576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840094a0 con 0x7fa99009f160 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa994cff700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa9901228d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e980 tx=0x7fa98400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.230+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98400c690 con 0x7fa99009f160 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.231+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98400c7f0 con 0x7fa99009f160 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.231+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa980009a70 con 0x7fa99009f160 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.231+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98400cb80 con 0x7fa99009f160 2023-09-30T22:27:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.231+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa990124da0 con 0x7fa99009f160 2023-09-30T22:27:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.232+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa98400cce0 con 0x7fa99009f160 2023-09-30T22:27:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.232+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9900747f0 con 0x7fa99009f160 2023-09-30T22:27:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.232+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa98403fb80 con 0x7fa99009f160 2023-09-30T22:27:26.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.237+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa984041e60 con 0x7fa99009f160 2023-09-30T22:27:26.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.261+0000 7f8219f06700 1 Processor -- start 2023-09-30T22:27:26.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8219f06700 1 -- start start 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8219f06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f82140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8219f06700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82140a1340 0x7f8214133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8219f06700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821409edd0 con 0x7f8214133fa0 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8219f06700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821409e1f0 con 0x7f82140a1340 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8219f06700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821409dc50 con 0x7f82140a0a30 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8212ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82140a1340 0x7f8214133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8212ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82140a1340 0x7f8214133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33674/0 (socket says 172.21.15.32:33674) 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8212ffd700 1 -- 172.21.15.32:0/1634070513 learned_addr learned my addr 172.21.15.32:0/1634070513 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f8213fff700 1 --1- 172.21.15.32:0/1634070513 >> v1:172.21.15.32:6789/0 conn(0x7f8214133fa0 0x7f821413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46460/0 (socket says 172.21.15.32:46460) 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.262+0000 7f82137fe700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f82140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f8212ffd700 1 -- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 msgr2=0x7f82140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f8212ffd700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f82140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f8212ffd700 1 -- 172.21.15.32:0/1634070513 >> v1:172.21.15.32:6789/0 conn(0x7f8214133fa0 legacy=0x7f821413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f8212ffd700 1 -- 172.21.15.32:0/1634070513 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82140a1fe0 con 0x7f82140a1340 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f82127fc700 1 -- 172.21.15.32:0/1634070513 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647405992 0 0) 0x7f821409edd0 con 0x7f8214133fa0 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f82137fe700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f82140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f8212ffd700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214133a60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f82080098f0 tx=0x7f8208009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a830183b3c359e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f82127fc700 1 -- 172.21.15.32:0/1634070513 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208011830 con 0x7f82140a1340 2023-09-30T22:27:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f82127fc700 1 -- 172.21.15.32:0/1634070513 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208011990 con 0x7f82140a1340 2023-09-30T22:27:26.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.263+0000 7f82127fc700 1 -- 172.21.15.32:0/1634070513 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208011d20 con 0x7f82140a1340 2023-09-30T22:27:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 -- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 msgr2=0x7f8214133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214133a60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f82080098f0 tx=0x7f8208009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 -- 172.21.15.32:0/1634070513 shutdown_connections 2023-09-30T22:27:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f82140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 --2- 172.21.15.32:0/1634070513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214133a60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 -- 172.21.15.32:0/1634070513 >> 172.21.15.32:0/1634070513 conn(0x7f82140a6550 msgr2=0x7f82140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 -- 172.21.15.32:0/1634070513 shutdown_connections 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.264+0000 7f8219f06700 1 -- 172.21.15.32:0/1634070513 wait complete. 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.265+0000 7f8219f06700 1 Processor -- start 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.265+0000 7f8219f06700 1 -- start start 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.265+0000 7f8219f06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f8214122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.265+0000 7f8219f06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8219f06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8214133fa0 0x7f82141232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8219f06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214124620 con 0x7f82140a1340 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8219f06700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8214124430 con 0x7f82140a0a30 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8219f06700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8214124cf0 con 0x7f8214133fa0 2023-09-30T22:27:26.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214122d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33678/0 (socket says 172.21.15.32:33678) 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 -- 172.21.15.32:0/750318857 learned_addr learned my addr 172.21.15.32:0/750318857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f82137fe700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f8214122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8213fff700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8214133fa0 0x7f82141232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 -- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8214133fa0 msgr2=0x7f82141232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8214133fa0 0x7f82141232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 -- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 msgr2=0x7f8214122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f8214122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 -- 172.21.15.32:0/750318857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040094a0 con 0x7f82140a1340 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f82137fe700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f8214122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8212ffd700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214122d60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8208009c40 tx=0x7f8208004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208004550 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82080036a0 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8208009550 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.266+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208003a30 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.267+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8214133340 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.267+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8208021070 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.268+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8208040b00 con 0x7f82140a1340 2023-09-30T22:27:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.268+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214074ea0 con 0x7f82140a1340 2023-09-30T22:27:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.273+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8208040e60 con 0x7f82140a1340 2023-09-30T22:27:26.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.446+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa990125260 con 0x7fa99009f160 2023-09-30T22:27:26.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.447+0000 7fa98dffb700 1 -- 172.21.15.32:0/3770096576 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa984021090 con 0x7fa99009f160 2023-09-30T22:27:26.447 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:26.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 msgr2=0x7fa9901228d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa9901228d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e980 tx=0x7fa98400ec90 comp rx=0 tx=0).stop 2023-09-30T22:27:26.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 shutdown_connections 2023-09-30T22:27:26.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa99009f5b0 0x7fa990122e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa99009e850 0x7fa990122390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 --2- 172.21.15.32:0/3770096576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa99009f160 0x7fa9901228d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 >> 172.21.15.32:0/3770096576 conn(0x7fa9900a3f70 msgr2=0x7fa9900a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 shutdown_connections 2023-09-30T22:27:26.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.449+0000 7fa997764700 1 -- 172.21.15.32:0/3770096576 wait complete. 2023-09-30T22:27:26.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:26.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.481+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f82140a28e0 con 0x7f82140a1340 2023-09-30T22:27:26.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.482+0000 7f81fbfff700 1 -- 172.21.15.32:0/750318857 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8208015080 con 0x7f82140a1340 2023-09-30T22:27:26.482 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 msgr2=0x7f8214122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214122d60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8208009c40 tx=0x7f8208004750 comp rx=0 tx=0).stop 2023-09-30T22:27:26.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 shutdown_connections 2023-09-30T22:27:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8214133fa0 0x7f82141232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f82140a0a30 0x7f8214122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 --2- 172.21.15.32:0/750318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82140a1340 0x7f8214122d60 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 >> 172.21.15.32:0/750318857 conn(0x7f82140a6550 msgr2=0x7f821411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 shutdown_connections 2023-09-30T22:27:26.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.484+0000 7f8219f06700 1 -- 172.21.15.32:0/750318857 wait complete. 2023-09-30T22:27:26.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:26.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.572+0000 7f494b0a1700 1 Processor -- start 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.572+0000 7f494b0a1700 1 -- start start 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494b0a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f494409f410 0x7f4944136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494b0a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f49441393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494b0a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4944070940 con 0x7f494409e930 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494b0a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494406fd60 con 0x7f494409f410 2023-09-30T22:27:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494b0a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f494406fa80 con 0x7f4944136fe0 2023-09-30T22:27:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494963e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f49441393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f4943fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f494409f410 0x7f4944136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494963e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f49441393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52100/0 (socket says 172.21.15.32:52100) 2023-09-30T22:27:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f4948e3d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f494409e930 0x7f494409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46468/0 (socket says 172.21.15.32:46468) 2023-09-30T22:27:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.573+0000 7f494963e700 1 -- 172.21.15.32:0/1181815099 learned_addr learned my addr 172.21.15.32:0/1181815099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f4943fff700 1 -- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 msgr2=0x7f49441393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f4943fff700 1 --2- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f49441393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f4943fff700 1 -- 172.21.15.32:0/1181815099 >> v1:172.21.15.32:6789/0 conn(0x7f494409e930 legacy=0x7f494409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f4943fff700 1 -- 172.21.15.32:0/1181815099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494409fb60 con 0x7f494409f410 2023-09-30T22:27:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f49437fe700 1 -- 172.21.15.32:0/1181815099 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489608914 0 0) 0x7f4944070940 con 0x7f494409e930 2023-09-30T22:27:26.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f494963e700 1 --2- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f49441393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f4943fff700 1 --2- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409f410 0x7f4944136aa0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4938009840 tx=0x7f4938009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23946465b8459da2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f49437fe700 1 -- 172.21.15.32:0/1181815099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938011830 con 0x7f494409f410 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f49437fe700 1 -- 172.21.15.32:0/1181815099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4938011990 con 0x7f494409f410 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.574+0000 7f49437fe700 1 -- 172.21.15.32:0/1181815099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938011d20 con 0x7f494409f410 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 -- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409f410 msgr2=0x7f4944136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409f410 0x7f4944136aa0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4938009840 tx=0x7f4938009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 -- 172.21.15.32:0/1181815099 shutdown_connections 2023-09-30T22:27:26.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f49441393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1181815099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409f410 0x7f4944136aa0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 -- 172.21.15.32:0/1181815099 >> 172.21.15.32:0/1181815099 conn(0x7f49440a4210 msgr2=0x7f49440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 -- 172.21.15.32:0/1181815099 shutdown_connections 2023-09-30T22:27:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.575+0000 7f494b0a1700 1 -- 172.21.15.32:0/1181815099 wait complete. 2023-09-30T22:27:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.576+0000 7f494b0a1700 1 Processor -- start 2023-09-30T22:27:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.576+0000 7f494b0a1700 1 -- start start 2023-09-30T22:27:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494b0a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409e930 0x7f494411c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494b0a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f494409f410 0x7f494411ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494b0a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f494411cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494b0a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494411fd80 con 0x7f494409e930 2023-09-30T22:27:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494b0a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944120b60 con 0x7f494409f410 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494b0a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f494411ff80 con 0x7f4944136fe0 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f494411cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f494411cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52114/0 (socket says 172.21.15.32:52114) 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 -- 172.21.15.32:0/1496470999 learned_addr learned my addr 172.21.15.32:0/1496470999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f4943fff700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f494409f410 0x7f494411ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f4948e3d700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409e930 0x7f494411c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 -- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f494409f410 msgr2=0x7f494411ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f494409f410 0x7f494411ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 -- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409e930 msgr2=0x7f494411c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409e930 0x7f494411c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 -- 172.21.15.32:0/1496470999 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c009b30 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.577+0000 7f494963e700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f494411cf60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f493c009f10 tx=0x7f493c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.578+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c018070 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.578+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c00f040 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.578+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49380094a0 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.578+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c003c40 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.578+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4944071d20 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.578+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f493c00c940 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.579+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f493c03f7d0 con 0x7f4944136fe0 2023-09-30T22:27:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.579+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494411fa40 con 0x7f4944136fe0 2023-09-30T22:27:26.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.584+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f494411fa40 con 0x7f4944136fe0 2023-09-30T22:27:26.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 Processor -- start 2023-09-30T22:27:26.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 -- start start 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6200a1510 0x7fd620138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd620139250 0x7fd62013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62009edd0 con 0x7fd6200a0a30 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200a18e0 con 0x7fd6200a1510 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd6266b0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62009e1f0 con 0x7fd620139250 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd61f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6200a1510 0x7fd620138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd624c4d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd620139250 0x7fd62013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd61ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6200a0a30 0x7fd6200a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46482/0 (socket says 172.21.15.32:46482) 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.677+0000 7fd61ffff700 1 -- 172.21.15.32:0/3820460029 learned_addr learned my addr 172.21.15.32:0/3820460029 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd620139250 msgr2=0x7fd62013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61f7fe700 1 --2- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd620139250 0x7fd62013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3820460029 >> v1:172.21.15.32:6789/0 conn(0x7fd6200a0a30 legacy=0x7fd6200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3820460029 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62009dc50 con 0x7fd6200a1510 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61effd700 1 -- 172.21.15.32:0/3820460029 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747498371 0 0) 0x7fd62009edd0 con 0x7fd6200a0a30 2023-09-30T22:27:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd624c4d700 1 --2- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd620139250 0x7fd62013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61f7fe700 1 --2- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6200a1510 0x7fd620138d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd614009840 tx=0x7fd614009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9bf6cf2062fa6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61effd700 1 -- 172.21.15.32:0/3820460029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614011830 con 0x7fd6200a1510 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61effd700 1 -- 172.21.15.32:0/3820460029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614011990 con 0x7fd6200a1510 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.678+0000 7fd61effd700 1 -- 172.21.15.32:0/3820460029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614011d20 con 0x7fd6200a1510 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 -- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6200a1510 msgr2=0x7fd620138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6200a1510 0x7fd620138d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd614009840 tx=0x7fd614009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 -- 172.21.15.32:0/3820460029 shutdown_connections 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd620139250 0x7fd62013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3820460029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6200a1510 0x7fd620138d10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 -- 172.21.15.32:0/3820460029 >> 172.21.15.32:0/3820460029 conn(0x7fd6200a6550 msgr2=0x7fd6200a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 -- 172.21.15.32:0/3820460029 shutdown_connections 2023-09-30T22:27:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.679+0000 7fd6266b0700 1 -- 172.21.15.32:0/3820460029 wait complete. 2023-09-30T22:27:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.680+0000 7fd6266b0700 1 Processor -- start 2023-09-30T22:27:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.680+0000 7fd6266b0700 1 -- start start 2023-09-30T22:27:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd6266b0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6200a0a30 0x7fd6201227a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd6266b0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 0x7fd620122ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd6266b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd620139250 0x7fd620123220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd6266b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6201389d0 con 0x7fd620139250 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd6266b0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd620124c80 con 0x7fd6200a1510 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd6266b0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6201243b0 con 0x7fd6200a0a30 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 0x7fd620122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6200a0a30 0x7fd6201227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd624c4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd620139250 0x7fd620123220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 0x7fd620122ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48568/0 (socket says 172.21.15.32:48568) 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3359899753 learned_addr learned my addr 172.21.15.32:0/3359899753 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6200a0a30 msgr2=0x7fd6201227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6200a0a30 0x7fd6201227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd620139250 msgr2=0x7fd620123220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd620139250 0x7fd620123220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 -- 172.21.15.32:0/3359899753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618009a70 con 0x7fd6200a1510 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd624c4d700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd620139250 0x7fd620123220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61f7fe700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 0x7fd620122ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd614009fd0 tx=0x7fd614004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.681+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140045d0 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.682+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61401f070 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.682+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6140094a0 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.682+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140038e0 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.682+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6200a0220 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.683+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd614003a40 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.683+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd620074ea0 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.683+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd614021070 con 0x7fd6200a1510 2023-09-30T22:27:26.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.688+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd61401a430 con 0x7fd6200a1510 2023-09-30T22:27:26.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.788+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f494411fa40 con 0x7f4944136fe0 2023-09-30T22:27:26.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.789+0000 7f49417fa700 1 -- 172.21.15.32:0/1496470999 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f494411fa40 con 0x7f4944136fe0 2023-09-30T22:27:26.790 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 msgr2=0x7f494411cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f494411cf60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f493c009f10 tx=0x7f493c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 shutdown_connections 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4944136fe0 0x7f494411cf60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f494409e930 0x7f494411c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 --2- 172.21.15.32:0/1496470999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f494409f410 0x7f494411ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 >> 172.21.15.32:0/1496470999 conn(0x7f49440a4210 msgr2=0x7f4944134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 shutdown_connections 2023-09-30T22:27:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.791+0000 7f494b0a1700 1 -- 172.21.15.32:0/1496470999 wait complete. 2023-09-30T22:27:26.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.898+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd6200a28e0 con 0x7fd6200a1510 2023-09-30T22:27:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.898+0000 7fd61cff9700 1 -- 172.21.15.32:0/3359899753 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd61402b020 con 0x7fd6200a1510 2023-09-30T22:27:26.899 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.900+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 msgr2=0x7fd620122ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.900+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 0x7fd620122ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd614009fd0 tx=0x7fd614004750 comp rx=0 tx=0).stop 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.900+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 shutdown_connections 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.900+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd620139250 0x7fd620123220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.900+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6200a0a30 0x7fd6201227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.900+0000 7fd6266b0700 1 --2- 172.21.15.32:0/3359899753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6200a1510 0x7fd620122ce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.901+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 >> 172.21.15.32:0/3359899753 conn(0x7fd6200a6550 msgr2=0x7fd6200a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.901+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 shutdown_connections 2023-09-30T22:27:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:26.901+0000 7fd6266b0700 1 -- 172.21.15.32:0/3359899753 wait complete. 2023-09-30T22:27:26.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:27.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:27.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.615+0000 7f80d9d45700 1 Processor -- start 2023-09-30T22:27:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.615+0000 7f80d9d45700 1 -- start start 2023-09-30T22:27:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d9d45700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d9d45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4134260 0x7f80d41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d9d45700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d406b990 con 0x7f80d4133e30 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d9d45700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40a0590 con 0x7f80d4133420 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d9d45700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d409fb80 con 0x7f80d4134260 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4134260 0x7f80d41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f80d4133e30 0x7f80d412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46496/0 (socket says 172.21.15.32:46496) 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d2ffd700 1 -- 172.21.15.32:0/3723439810 learned_addr learned my addr 172.21.15.32:0/3723439810 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d3fff700 1 -- 172.21.15.32:0/3723439810 >> v2:172.21.15.32:3300/0 conn(0x7f80d4133420 msgr2=0x7f80d41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d3fff700 1 --2- 172.21.15.32:0/3723439810 >> v2:172.21.15.32:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d3fff700 1 -- 172.21.15.32:0/3723439810 >> v1:172.21.15.32:6789/0 conn(0x7f80d4133e30 legacy=0x7f80d412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.616+0000 7f80d3fff700 1 -- 172.21.15.32:0/3723439810 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40a0030 con 0x7f80d4134260 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d3fff700 1 --2- 172.21.15.32:0/3723439810 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4134260 0x7f80d41306a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009d90 tx=0x7f80cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0576ae294f97ea2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d27fc700 1 -- 172.21.15.32:0/3723439810 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00e070 con 0x7f80d4134260 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d27fc700 1 -- 172.21.15.32:0/3723439810 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc00bb20 con 0x7f80d4134260 2023-09-30T22:27:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d27fc700 1 -- 172.21.15.32:0/3723439810 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc012430 con 0x7f80d4134260 2023-09-30T22:27:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d9d45700 1 -- 172.21.15.32:0/3723439810 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4134260 msgr2=0x7f80d41306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d9d45700 1 --2- 172.21.15.32:0/3723439810 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4134260 0x7f80d41306a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009d90 tx=0x7f80cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d9d45700 1 -- 172.21.15.32:0/3723439810 shutdown_connections 2023-09-30T22:27:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d9d45700 1 --2- 172.21.15.32:0/3723439810 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4134260 0x7f80d41306a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d9d45700 1 --2- 172.21.15.32:0/3723439810 >> v2:172.21.15.32:3300/0 conn(0x7f80d4133420 0x7f80d41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.617+0000 7f80d9d45700 1 -- 172.21.15.32:0/3723439810 >> 172.21.15.32:0/3723439810 conn(0x7f80d40a4210 msgr2=0x7f80d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.618+0000 7f80d9d45700 1 -- 172.21.15.32:0/3723439810 shutdown_connections 2023-09-30T22:27:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.618+0000 7f80d9d45700 1 -- 172.21.15.32:0/3723439810 wait complete. 2023-09-30T22:27:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.619+0000 7f80d9d45700 1 Processor -- start 2023-09-30T22:27:27.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.619+0000 7f80d9d45700 1 -- start start 2023-09-30T22:27:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d9d45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d9d45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d9d45700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d9d45700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d412d1b0 con 0x7f80d4133420 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d9d45700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d412d670 con 0x7f80d4133e30 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d9d45700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d409ddc0 con 0x7f80d4134260 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33720/0 (socket says 172.21.15.32:33720) 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d37fe700 1 -- 172.21.15.32:0/2714463838 learned_addr learned my addr 172.21.15.32:0/2714463838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d3fff700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d2ffd700 1 -- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d4134260 msgr2=0x7f80d409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d2ffd700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d2ffd700 1 -- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 msgr2=0x7f80d409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d2ffd700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d2ffd700 1 -- 172.21.15.32:0/2714463838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc009a70 con 0x7f80d4133e30 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.620+0000 7f80d37fe700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.621+0000 7f80d2ffd700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4133e30 0x7f80d409c9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c500 tx=0x7f80c800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.621+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8005d70 con 0x7f80d4133e30 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.621+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80070d0 con 0x7f80d4133e30 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.621+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8007440 con 0x7f80d4133e30 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.621+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d411ff80 con 0x7f80d4133e30 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.621+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d4120760 con 0x7f80d4133e30 2023-09-30T22:27:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.622+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80c80075a0 con 0x7f80d4133e30 2023-09-30T22:27:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.622+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d4074ea0 con 0x7f80d4133e30 2023-09-30T22:27:27.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.622+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f80c803ead0 con 0x7f80d4133e30 2023-09-30T22:27:27.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.627+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80c801b8d0 con 0x7f80d4133e30 2023-09-30T22:27:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.652+0000 7fa3ec875700 1 Processor -- start 2023-09-30T22:27:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.652+0000 7fa3ec875700 1 -- start start 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3ec875700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3ec875700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3e809d630 0x7fa3e80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3ec875700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e8133930 0x7fa3e8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3ec875700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e80a2a70 con 0x7fa3e809d630 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3ec875700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80a01d0 con 0x7fa3e8133930 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3ec875700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e809e2b0 con 0x7fa3e80a0a30 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e8133930 0x7fa3e8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3e809d630 0x7fa3e80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e5d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3e809d630 0x7fa3e80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33726/0 (socket says 172.21.15.32:33726) 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52150/0 (socket says 172.21.15.32:52150) 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e8133930 0x7fa3e8133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48582/0 (socket says 172.21.15.32:48582) 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.653+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/1782665166 learned_addr learned my addr 172.21.15.32:0/1782665166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 msgr2=0x7fa3e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e5d9b700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e8133930 msgr2=0x7fa3e8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e5d9b700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e8133930 0x7fa3e8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/1782665166 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e809dc50 con 0x7fa3e809d630 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e659c700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e5d9b700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e809d630 0x7fa3e80a1340 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00098f0 tx=0x7fa3d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6e0da286cacc6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e559a700 1 -- 172.21.15.32:0/1782665166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0011830 con 0x7fa3e809d630 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e559a700 1 -- 172.21.15.32:0/1782665166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0011990 con 0x7fa3e809d630 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.654+0000 7fa3e559a700 1 -- 172.21.15.32:0/1782665166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0011d20 con 0x7fa3e809d630 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 -- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e809d630 msgr2=0x7fa3e80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e809d630 0x7fa3e80a1340 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00098f0 tx=0x7fa3d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 -- 172.21.15.32:0/1782665166 shutdown_connections 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e8133930 0x7fa3e8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 --2- 172.21.15.32:0/1782665166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e809d630 0x7fa3e80a1340 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 -- 172.21.15.32:0/1782665166 >> 172.21.15.32:0/1782665166 conn(0x7fa3e80a3050 msgr2=0x7fa3e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 -- 172.21.15.32:0/1782665166 shutdown_connections 2023-09-30T22:27:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.655+0000 7fa3ec875700 1 -- 172.21.15.32:0/1782665166 wait complete. 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.656+0000 7fa3ec875700 1 Processor -- start 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.656+0000 7fa3ec875700 1 -- start start 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3ec875700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e809d630 0x7fa3e8121120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3ec875700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e8121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3ec875700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e8133930 0x7fa3e8121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3ec875700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8123c00 con 0x7fa3e8133930 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3ec875700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8124c40 con 0x7fa3e809d630 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3ec875700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80a51d0 con 0x7fa3e80a0a30 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e8133930 0x7fa3e8121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e8121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e8133930 0x7fa3e8121ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33732/0 (socket says 172.21.15.32:33732) 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e8121660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52158/0 (socket says 172.21.15.32:52158) 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e6d9d700 1 -- 172.21.15.32:0/3313176178 learned_addr learned my addr 172.21.15.32:0/3313176178 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e659c700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e809d630 0x7fa3e8121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e809d630 msgr2=0x7fa3e8121120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e809d630 0x7fa3e8121120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e8133930 msgr2=0x7fa3e8121ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e8133930 0x7fa3e8121ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 -- 172.21.15.32:0/3313176178 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d80094a0 con 0x7fa3e80a0a30 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.657+0000 7fa3e5d9b700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e8121660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0016000 tx=0x7fa3d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:27.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.658+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004240 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.658+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d001f070 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.658+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d0009550 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.658+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00036a0 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.658+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e8122d30 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.658+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3d0021030 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.659+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa3d0040840 con 0x7fa3e80a0a30 2023-09-30T22:27:27.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.659+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e811df60 con 0x7fa3e80a0a30 2023-09-30T22:27:27.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.664+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3d0015080 con 0x7fa3e80a0a30 2023-09-30T22:27:27.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.839+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f80d406b250 con 0x7f80d4133e30 2023-09-30T22:27:27.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.839+0000 7f80bbfff700 1 -- 172.21.15.32:0/2714463838 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80c8017070 con 0x7f80d4133e30 2023-09-30T22:27:27.840 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4133e30 msgr2=0x7f80d409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4133e30 0x7f80d409c9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c500 tx=0x7f80c800c810 comp rx=0 tx=0).stop 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 shutdown_connections 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80d4134260 0x7f80d409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80d4133420 0x7f80d409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 --2- 172.21.15.32:0/2714463838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f80d4133e30 0x7f80d409c9c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 >> 172.21.15.32:0/2714463838 conn(0x7f80d40a4210 msgr2=0x7f80d40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 shutdown_connections 2023-09-30T22:27:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.842+0000 7f80d9d45700 1 -- 172.21.15.32:0/2714463838 wait complete. 2023-09-30T22:27:27.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.871+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa3e80747f0 con 0x7fa3e80a0a30 2023-09-30T22:27:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.871+0000 7fa3deffd700 1 -- 172.21.15.32:0/3313176178 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3d00239c0 con 0x7fa3e80a0a30 2023-09-30T22:27:27.872 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.873+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 msgr2=0x7fa3e8121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.873+0000 7fa3ec875700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e8121660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0016000 tx=0x7fa3d0004750 comp rx=0 tx=0).stop 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.873+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 shutdown_connections 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.873+0000 7fa3ec875700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3e8133930 0x7fa3e8121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.874+0000 7fa3ec875700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3e809d630 0x7fa3e8121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.874+0000 7fa3ec875700 1 --2- 172.21.15.32:0/3313176178 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3e80a0a30 0x7fa3e8121660 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.874+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 >> 172.21.15.32:0/3313176178 conn(0x7fa3e80a3050 msgr2=0x7fa3e81331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.874+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 shutdown_connections 2023-09-30T22:27:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.874+0000 7fa3ec875700 1 -- 172.21.15.32:0/3313176178 wait complete. 2023-09-30T22:27:27.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:27.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:27.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.946+0000 7fd1a20d4700 1 Processor -- start 2023-09-30T22:27:27.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.946+0000 7fd1a20d4700 1 -- start start 2023-09-30T22:27:27.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd1a20d4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09d6f0 0x7fd19c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd1a20d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd1a20d4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c09f130 con 0x7fd19c09e000 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd1a20d4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c09f520 con 0x7fd19c09e410 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd1a20d4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c09e930 con 0x7fd19c09d6f0 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09d6f0 0x7fd19c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48602/0 (socket says 172.21.15.32:48602) 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd19c09e000 0x7fd19c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46508/0 (socket says 172.21.15.32:46508) 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 -- 172.21.15.32:0/2635013689 learned_addr learned my addr 172.21.15.32:0/2635013689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 -- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09d6f0 msgr2=0x7fd19c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 --2- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09d6f0 0x7fd19c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 -- 172.21.15.32:0/2635013689 >> v1:172.21.15.32:6789/0 conn(0x7fd19c09e000 legacy=0x7fd19c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19a7fc700 1 -- 172.21.15.32:0/2635013689 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168355452 0 0) 0x7fd19c09f130 con 0x7fd19c09e000 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19bfff700 1 -- 172.21.15.32:0/2635013689 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c09b2d0 con 0x7fd19c09e410 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.947+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09d6f0 0x7fd19c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.948+0000 7fd19bfff700 1 --2- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c130d70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd19400f3a0 tx=0x7fd19400f6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=246049aac9883365 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.948+0000 7fd19a7fc700 1 -- 172.21.15.32:0/2635013689 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194012070 con 0x7fd19c09e410 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.948+0000 7fd19a7fc700 1 -- 172.21.15.32:0/2635013689 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400d0d0 con 0x7fd19c09e410 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.948+0000 7fd19a7fc700 1 -- 172.21.15.32:0/2635013689 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400d480 con 0x7fd19c09e410 2023-09-30T22:27:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.948+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 msgr2=0x7fd19c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.948+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c130d70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd19400f3a0 tx=0x7fd19400f6b0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.949+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2635013689 shutdown_connections 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.949+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c130d70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.949+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2635013689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09d6f0 0x7fd19c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.949+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2635013689 >> 172.21.15.32:0/2635013689 conn(0x7fd19c066f50 msgr2=0x7fd19c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.949+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2635013689 shutdown_connections 2023-09-30T22:27:27.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.949+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2635013689 wait complete. 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 Processor -- start 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 -- start start 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 0x7fd19c1300b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09e000 0x7fd19c11dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c11e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c1221b0 con 0x7fd19c09d6f0 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c1217f0 con 0x7fd19c09e410 2023-09-30T22:27:27.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.950+0000 7fd1a20d4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c120f80 con 0x7fd19c09e000 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 0x7fd19c1300b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 0x7fd19c1300b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33736/0 (socket says 172.21.15.32:33736) 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c11e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 -- 172.21.15.32:0/2580054557 learned_addr learned my addr 172.21.15.32:0/2580054557 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19affd700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09e000 0x7fd19c11dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 -- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09e000 msgr2=0x7fd19c11dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09e000 0x7fd19c11dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 -- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 msgr2=0x7fd19c11e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:27.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c11e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 -- 172.21.15.32:0/2580054557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c00b550 con 0x7fd19c09d6f0 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19bfff700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c11e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd19b7fe700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 0x7fd19c1300b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00fde0 tx=0x7fd18c00d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00fed0 con 0x7fd19c09d6f0 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c018440 con 0x7fd19c09d6f0 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19400f050 con 0x7fd19c09d6f0 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0187d0 con 0x7fd19c09d6f0 2023-09-30T22:27:27.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.951+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c09c700 con 0x7fd19c09d6f0 2023-09-30T22:27:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.952+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd18c018930 con 0x7fd19c09d6f0 2023-09-30T22:27:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.953+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c11ac20 con 0x7fd19c09d6f0 2023-09-30T22:27:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.953+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd18c03fda0 con 0x7fd19c09d6f0 2023-09-30T22:27:27.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:27.958+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd18c017990 con 0x7fd19c09d6f0 2023-09-30T22:27:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.066+0000 7f4387e3a700 1 Processor -- start 2023-09-30T22:27:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.066+0000 7f4387e3a700 1 -- start start 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4387e3a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43800a0a30 0x7f43800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4387e3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4387e3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4387e3a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438009edd0 con 0x7f43800a0a30 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4387e3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438009e1f0 con 0x7f43800a1340 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4387e3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f438009dc50 con 0x7f4380133fa0 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43863d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4385bd6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43800a0a30 0x7f43800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48610/0 (socket says 172.21.15.32:48610) 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 -- 172.21.15.32:0/4024963712 learned_addr learned my addr 172.21.15.32:0/4024963712 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 -- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 msgr2=0x7f4380134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 --2- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 -- 172.21.15.32:0/4024963712 >> v2:172.21.15.32:3300/0 conn(0x7f43800a0a30 msgr2=0x7f43800a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 --2- 172.21.15.32:0/4024963712 >> v2:172.21.15.32:3300/0 conn(0x7f43800a0a30 0x7f43800a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43853d5700 1 -- 172.21.15.32:0/4024963712 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800a1fe0 con 0x7f43800a1340 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f43863d7700 1 --2- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.067+0000 7f4385bd6700 1 --2- 172.21.15.32:0/4024963712 >> v2:172.21.15.32:3300/0 conn(0x7f43800a0a30 0x7f43800a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f43853d5700 1 --2- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380133a60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4370009920 tx=0x7f4370009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0122c813209a215 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4384bd4700 1 -- 172.21.15.32:0/4024963712 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011830 con 0x7f43800a1340 2023-09-30T22:27:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4384bd4700 1 -- 172.21.15.32:0/4024963712 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370011990 con 0x7f43800a1340 2023-09-30T22:27:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4384bd4700 1 -- 172.21.15.32:0/4024963712 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011d20 con 0x7f43800a1340 2023-09-30T22:27:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 -- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 msgr2=0x7f4380133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 --2- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380133a60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4370009920 tx=0x7f4370009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 -- 172.21.15.32:0/4024963712 shutdown_connections 2023-09-30T22:27:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 --2- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 --2- 172.21.15.32:0/4024963712 >> v2:172.21.15.32:3300/0 conn(0x7f43800a0a30 0x7f43800a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 --2- 172.21.15.32:0/4024963712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380133a60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.068+0000 7f4387e3a700 1 -- 172.21.15.32:0/4024963712 >> 172.21.15.32:0/4024963712 conn(0x7f43800a6550 msgr2=0x7f43800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.069+0000 7f4387e3a700 1 -- 172.21.15.32:0/4024963712 shutdown_connections 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.069+0000 7f4387e3a700 1 -- 172.21.15.32:0/4024963712 wait complete. 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 Processor -- start 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 -- start start 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 0x7f43801200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4380121d90 con 0x7f43800a0a30 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4380123920 con 0x7f43800a1340 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4387e3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43800a36c0 con 0x7f4380133fa0 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4385bd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 0x7f43801200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4385bd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 0x7f43801200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33746/0 (socket says 172.21.15.32:33746) 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f4385bd6700 1 -- 172.21.15.32:0/873286341 learned_addr learned my addr 172.21.15.32:0/873286341 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f43863d7700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.070+0000 7f43853d5700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4385bd6700 1 -- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 msgr2=0x7f4380120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4385bd6700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4385bd6700 1 -- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 msgr2=0x7f4380120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4385bd6700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4385bd6700 1 -- 172.21.15.32:0/873286341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c009a70 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4385bd6700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 0x7f43801200e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f437c00efc0 tx=0x7f437c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c00cdc0 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437c00f040 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c0146a0 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43780094a0 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.071+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4370009550 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.072+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f437c014800 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.072+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f437c040470 con 0x7f43800a0a30 2023-09-30T22:27:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.072+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43801235a0 con 0x7f43800a0a30 2023-09-30T22:27:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.077+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43801235a0 con 0x7f43800a0a30 2023-09-30T22:27:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.170+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd19c121390 con 0x7fd19c09d6f0 2023-09-30T22:27:28.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.171+0000 7fd183fff700 1 -- 172.21.15.32:0/2580054557 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd18c0173a0 con 0x7fd19c09d6f0 2023-09-30T22:27:28.172 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 msgr2=0x7fd19c1300b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 0x7fd19c1300b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00fde0 tx=0x7fd18c00d630 comp rx=0 tx=0).stop 2023-09-30T22:27:28.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 shutdown_connections 2023-09-30T22:27:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd19c09e410 0x7fd19c11e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd19c09d6f0 0x7fd19c1300b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 --2- 172.21.15.32:0/2580054557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd19c09e000 0x7fd19c11dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 >> 172.21.15.32:0/2580054557 conn(0x7fd19c066f50 msgr2=0x7fd19c130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 shutdown_connections 2023-09-30T22:27:28.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.173+0000 7fd1a20d4700 1 -- 172.21.15.32:0/2580054557 wait complete. 2023-09-30T22:27:28.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:28.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.285+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f43801235a0 con 0x7f43800a0a30 2023-09-30T22:27:28.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.286+0000 7f43767fc700 1 -- 172.21.15.32:0/873286341 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43801235a0 con 0x7f43800a0a30 2023-09-30T22:27:28.287 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 msgr2=0x7f43801200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 0x7f43801200e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f437c00efc0 tx=0x7f437c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 shutdown_connections 2023-09-30T22:27:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4380133fa0 0x7f4380120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43800a0a30 0x7f43801200e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 --2- 172.21.15.32:0/873286341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43800a1340 0x7f4380120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 >> 172.21.15.32:0/873286341 conn(0x7f43800a6550 msgr2=0x7f43801bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 shutdown_connections 2023-09-30T22:27:28.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:28.288+0000 7f4387e3a700 1 -- 172.21.15.32:0/873286341 wait complete. 2023-09-30T22:27:28.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:28.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:28.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:29.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 Processor -- start 2023-09-30T22:27:29.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 -- start start 2023-09-30T22:27:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d200a0a30 0x7f3d200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a1340 0x7f3d20133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d20133fa0 0x7f3d20134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2009edd0 con 0x7f3d20133fa0 2023-09-30T22:27:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2009e1f0 con 0x7f3d200a1340 2023-09-30T22:27:29.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d2525d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2009dc50 con 0x7f3d200a0a30 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d1ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d200a0a30 0x7f3d200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d1e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a1340 0x7f3d20133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d1f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d20133fa0 0x7f3d20134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d1ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d200a0a30 0x7f3d200a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52202/0 (socket says 172.21.15.32:52202) 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.007+0000 7f3d1ed9d700 1 -- 172.21.15.32:0/3348235433 learned_addr learned my addr 172.21.15.32:0/3348235433 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1e59c700 1 -- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d200a0a30 msgr2=0x7f3d200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1e59c700 1 --2- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d200a0a30 0x7f3d200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1e59c700 1 -- 172.21.15.32:0/3348235433 >> v2:172.21.15.32:3300/0 conn(0x7f3d20133fa0 msgr2=0x7f3d20134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1e59c700 1 --2- 172.21.15.32:0/3348235433 >> v2:172.21.15.32:3300/0 conn(0x7f3d20133fa0 0x7f3d20134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1e59c700 1 -- 172.21.15.32:0/3348235433 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200a1fe0 con 0x7f3d200a1340 2023-09-30T22:27:29.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1f59e700 1 --2- 172.21.15.32:0/3348235433 >> v2:172.21.15.32:3300/0 conn(0x7f3d20133fa0 0x7f3d20134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1e59c700 1 --2- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a1340 0x7f3d20133a60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009840 tx=0x7f3d08009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e29be63f49c445 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1dd9b700 1 -- 172.21.15.32:0/3348235433 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08011830 con 0x7f3d200a1340 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1dd9b700 1 -- 172.21.15.32:0/3348235433 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08011990 con 0x7f3d200a1340 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.008+0000 7f3d1dd9b700 1 -- 172.21.15.32:0/3348235433 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08011d20 con 0x7f3d200a1340 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 -- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a1340 msgr2=0x7f3d20133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 --2- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a1340 0x7f3d20133a60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009840 tx=0x7f3d08009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 -- 172.21.15.32:0/3348235433 shutdown_connections 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 --2- 172.21.15.32:0/3348235433 >> v2:172.21.15.32:3300/0 conn(0x7f3d20133fa0 0x7f3d20134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 --2- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d200a0a30 0x7f3d200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 --2- 172.21.15.32:0/3348235433 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a1340 0x7f3d20133a60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 -- 172.21.15.32:0/3348235433 >> 172.21.15.32:0/3348235433 conn(0x7f3d200a6550 msgr2=0x7f3d200a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 -- 172.21.15.32:0/3348235433 shutdown_connections 2023-09-30T22:27:29.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.009+0000 7f3d2525d700 1 -- 172.21.15.32:0/3348235433 wait complete. 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.010+0000 7f3d2525d700 1 Processor -- start 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.010+0000 7f3d2525d700 1 -- start start 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d2525d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a0a30 0x7f3d20120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d2525d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 0x7f3d20120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d2525d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d20133fa0 0x7f3d20120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d2525d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20123c50 con 0x7f3d200a1340 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d2525d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d200a3530 con 0x7f3d200a0a30 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d2525d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201233e0 con 0x7f3d20133fa0 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 0x7f3d20120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a0a30 0x7f3d20120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 0x7f3d20120670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33762/0 (socket says 172.21.15.32:33762) 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1e59c700 1 -- 172.21.15.32:0/1392309826 learned_addr learned my addr 172.21.15.32:0/1392309826 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1f59e700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d20133fa0 0x7f3d20120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1f59e700 1 -- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a0a30 msgr2=0x7f3d20120130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1f59e700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a0a30 0x7f3d20120130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1f59e700 1 -- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 msgr2=0x7f3d20120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1f59e700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 0x7f3d20120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1f59e700 1 -- 172.21.15.32:0/1392309826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180094a0 con 0x7f3d20133fa0 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.011+0000 7f3d1e59c700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 0x7f3d20120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.012+0000 7f3d1f59e700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d20133fa0 0x7f3d20120bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800e910 tx=0x7f3d1800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.012+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18004020 con 0x7f3d20133fa0 2023-09-30T22:27:29.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.012+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d180041a0 con 0x7f3d20133fa0 2023-09-30T22:27:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.012+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d080094a0 con 0x7f3d20133fa0 2023-09-30T22:27:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.012+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18004550 con 0x7f3d20133fa0 2023-09-30T22:27:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.012+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d20121d40 con 0x7f3d20133fa0 2023-09-30T22:27:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.013+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3d180036a0 con 0x7f3d20133fa0 2023-09-30T22:27:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.013+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3d1803f7b0 con 0x7f3d20133fa0 2023-09-30T22:27:29.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.013+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d20122ff0 con 0x7f3d20133fa0 2023-09-30T22:27:29.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.018+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3d18017080 con 0x7f3d20133fa0 2023-09-30T22:27:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.045+0000 7fdf80dd4700 1 Processor -- start 2023-09-30T22:27:29.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.045+0000 7fdf80dd4700 1 -- start start 2023-09-30T22:27:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.046+0000 7fdf80dd4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 0x7fdf7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.046+0000 7fdf80dd4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf7c138d80 0x7fdf7c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.046+0000 7fdf80dd4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0a2a70 con 0x7fdf7c09d630 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.046+0000 7fdf80dd4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0a01d0 con 0x7fdf7c138d80 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.046+0000 7fdf80dd4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c09e2b0 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 0x7fdf7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 0x7fdf7c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48658/0 (socket says 172.21.15.32:48658) 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 -- 172.21.15.32:0/2765631553 learned_addr learned my addr 172.21.15.32:0/2765631553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7ad9d700 1 --2- 172.21.15.32:0/2765631553 >> v2:172.21.15.32:3300/0 conn(0x7fdf7c138d80 0x7fdf7c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 -- 172.21.15.32:0/2765631553 >> v2:172.21.15.32:3300/0 conn(0x7fdf7c138d80 msgr2=0x7fdf7c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 --2- 172.21.15.32:0/2765631553 >> v2:172.21.15.32:3300/0 conn(0x7fdf7c138d80 0x7fdf7c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 -- 172.21.15.32:0/2765631553 >> v1:172.21.15.32:6789/0 conn(0x7fdf7c09d630 legacy=0x7fdf7c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 -- 172.21.15.32:0/2765631553 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c09dc50 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7ad9d700 1 --2- 172.21.15.32:0/2765631553 >> v2:172.21.15.32:3300/0 conn(0x7fdf7c138d80 0x7fdf7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7a59c700 1 --2- 172.21.15.32:0/2765631553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 0x7fdf7c0a0e00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0098f0 tx=0x7fdf6c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d62788fa26fd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7959a700 1 -- 172.21.15.32:0/2765631553 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c011830 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.047+0000 7fdf7959a700 1 -- 172.21.15.32:0/2765631553 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c011990 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf7959a700 1 -- 172.21.15.32:0/2765631553 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c011d20 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf80dd4700 1 -- 172.21.15.32:0/2765631553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 msgr2=0x7fdf7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/2765631553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 0x7fdf7c0a0e00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0098f0 tx=0x7fdf6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf80dd4700 1 -- 172.21.15.32:0/2765631553 shutdown_connections 2023-09-30T22:27:29.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/2765631553 >> v2:172.21.15.32:3300/0 conn(0x7fdf7c138d80 0x7fdf7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/2765631553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c0a0a30 0x7fdf7c0a0e00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.048+0000 7fdf80dd4700 1 -- 172.21.15.32:0/2765631553 >> 172.21.15.32:0/2765631553 conn(0x7fdf7c0a3050 msgr2=0x7fdf7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.049+0000 7fdf80dd4700 1 -- 172.21.15.32:0/2765631553 shutdown_connections 2023-09-30T22:27:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.049+0000 7fdf80dd4700 1 -- 172.21.15.32:0/2765631553 wait complete. 2023-09-30T22:27:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.050+0000 7fdf80dd4700 1 Processor -- start 2023-09-30T22:27:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.050+0000 7fdf80dd4700 1 -- start start 2023-09-30T22:27:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf80dd4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c09d630 0x7fdf7c1208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf80dd4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 0x7fdf7c120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf80dd4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf7c138d80 0x7fdf7c121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf80dd4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c122470 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf80dd4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c1237c0 con 0x7fdf7c09d630 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf80dd4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c121d50 con 0x7fdf7c138d80 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf79d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 0x7fdf7c120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf7a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c09d630 0x7fdf7c1208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf79d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 0x7fdf7c120e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33786/0 (socket says 172.21.15.32:33786) 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf79d9b700 1 -- 172.21.15.32:0/1406373784 learned_addr learned my addr 172.21.15.32:0/1406373784 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf79d9b700 1 -- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf7c138d80 msgr2=0x7fdf7c121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.051+0000 7fdf79d9b700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf7c138d80 0x7fdf7c121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf79d9b700 1 -- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c09d630 msgr2=0x7fdf7c1208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf79d9b700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c09d630 0x7fdf7c1208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf79d9b700 1 -- 172.21.15.32:0/1406373784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74009a70 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf7a59c700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c09d630 0x7fdf7c1208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf79d9b700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 0x7fdf7c120e30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400c4b0 tx=0x7fdf6400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640052d0 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64005430 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640038c0 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c009550 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.052+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c0a0410 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.053+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf64003a20 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.053+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf6403ea50 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.053+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c0747f0 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.058+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf64017080 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:29.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.226+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3d20074ea0 con 0x7f3d20133fa0 2023-09-30T22:27:29.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.227+0000 7f3d177fe700 1 -- 172.21.15.32:0/1392309826 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d18020020 con 0x7f3d20133fa0 2023-09-30T22:27:29.228 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:29.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d20133fa0 msgr2=0x7f3d20120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d20133fa0 0x7f3d20120bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800e910 tx=0x7f3d1800ec20 comp rx=0 tx=0).stop 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 shutdown_connections 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d20133fa0 0x7f3d20120bb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d200a0a30 0x7f3d20120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 --2- 172.21.15.32:0/1392309826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d200a1340 0x7f3d20120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 >> 172.21.15.32:0/1392309826 conn(0x7f3d200a6550 msgr2=0x7f3d201bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 shutdown_connections 2023-09-30T22:27:29.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.230+0000 7f3d2525d700 1 -- 172.21.15.32:0/1392309826 wait complete. 2023-09-30T22:27:29.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:29.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.268+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdf7c123cc0 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.269+0000 7fdf72ffd700 1 -- 172.21.15.32:0/1406373784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf64019e60 con 0x7fdf7c0a0a30 2023-09-30T22:27:29.270 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:29.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 msgr2=0x7fdf7c120e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 0x7fdf7c120e30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400c4b0 tx=0x7fdf6400c7c0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 shutdown_connections 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf7c138d80 0x7fdf7c121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf7c09d630 0x7fdf7c1208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 --2- 172.21.15.32:0/1406373784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf7c0a0a30 0x7fdf7c120e30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 >> 172.21.15.32:0/1406373784 conn(0x7fdf7c0a3050 msgr2=0x7fdf7c139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 shutdown_connections 2023-09-30T22:27:29.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.271+0000 7fdf80dd4700 1 -- 172.21.15.32:0/1406373784 wait complete. 2023-09-30T22:27:29.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:29.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.329+0000 7f8943580700 1 Processor -- start 2023-09-30T22:27:29.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8943580700 1 -- start start 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8943580700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893c06a770 0x7f893c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8943580700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8943580700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c09ece0 con 0x7f893c06b080 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8943580700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c09e760 con 0x7f893c06a770 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8943580700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f893c09e040 con 0x7f893c131d10 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8941b1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8941b1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52222/0 (socket says 172.21.15.32:52222) 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f894131c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893c06a770 0x7f893c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.330+0000 7f8941b1d700 1 -- 172.21.15.32:0/1188533758 learned_addr learned my addr 172.21.15.32:0/1188533758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8941b1d700 1 -- 172.21.15.32:0/1188533758 >> v2:172.21.15.32:3300/0 conn(0x7f893c06a770 msgr2=0x7f893c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8941b1d700 1 --2- 172.21.15.32:0/1188533758 >> v2:172.21.15.32:3300/0 conn(0x7f893c06a770 0x7f893c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8941b1d700 1 -- 172.21.15.32:0/1188533758 >> v1:172.21.15.32:6789/0 conn(0x7f893c06b080 legacy=0x7f893c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8941b1d700 1 -- 172.21.15.32:0/1188533758 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c09f630 con 0x7f893c131d10 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f894131c700 1 --2- 172.21.15.32:0/1188533758 >> v2:172.21.15.32:3300/0 conn(0x7f893c06a770 0x7f893c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8941b1d700 1 --2- 172.21.15.32:0/1188533758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c139860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8934009e50 tx=0x7f893400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50fac29bba132d1c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8933fff700 1 -- 172.21.15.32:0/1188533758 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400e070 con 0x7f893c131d10 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8933fff700 1 -- 172.21.15.32:0/1188533758 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893400b910 con 0x7f893c131d10 2023-09-30T22:27:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.331+0000 7f8933fff700 1 -- 172.21.15.32:0/1188533758 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400bcc0 con 0x7f893c131d10 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 -- 172.21.15.32:0/1188533758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 msgr2=0x7f893c139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 --2- 172.21.15.32:0/1188533758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c139860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8934009e50 tx=0x7f893400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 -- 172.21.15.32:0/1188533758 shutdown_connections 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 --2- 172.21.15.32:0/1188533758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c139860 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 --2- 172.21.15.32:0/1188533758 >> v2:172.21.15.32:3300/0 conn(0x7f893c06a770 0x7f893c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 -- 172.21.15.32:0/1188533758 >> 172.21.15.32:0/1188533758 conn(0x7f893c066f50 msgr2=0x7f893c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 -- 172.21.15.32:0/1188533758 shutdown_connections 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.332+0000 7f8943580700 1 -- 172.21.15.32:0/1188533758 wait complete. 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.333+0000 7f8943580700 1 Processor -- start 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.333+0000 7f8943580700 1 -- start start 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.333+0000 7f8943580700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893c06a770 0x7f893c1c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.333+0000 7f8943580700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 0x7f893c1c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.333+0000 7f8943580700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c1c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8943580700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c11e1c0 con 0x7f893c06b080 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8943580700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c11e7e0 con 0x7f893c06a770 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8943580700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f893c071e60 con 0x7f893c131d10 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8940b1b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 0x7f893c1c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c1c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8940b1b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 0x7f893c1c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33804/0 (socket says 172.21.15.32:33804) 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8940b1b700 1 -- 172.21.15.32:0/3001448412 learned_addr learned my addr 172.21.15.32:0/3001448412 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f894131c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893c06a770 0x7f893c1c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 -- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893c06a770 msgr2=0x7f893c1c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893c06a770 0x7f893c1c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 -- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 msgr2=0x7f893c1c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 0x7f893c1c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 -- 172.21.15.32:0/3001448412 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380094a0 con 0x7f893c131d10 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8940b1b700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 0x7f893c1c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8941b1d700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c1c6c10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8934015000 tx=0x7f8934004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934004800 con 0x7f893c131d10 2023-09-30T22:27:29.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934004960 con 0x7f893c131d10 2023-09-30T22:27:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.334+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8934009b30 con 0x7f893c131d10 2023-09-30T22:27:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.335+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893401c680 con 0x7f893c131d10 2023-09-30T22:27:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.335+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c11fdf0 con 0x7f893c131d10 2023-09-30T22:27:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.335+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f893401c7e0 con 0x7f893c131d10 2023-09-30T22:27:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.336+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8934040ec0 con 0x7f893c131d10 2023-09-30T22:27:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.336+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c11fa90 con 0x7f893c131d10 2023-09-30T22:27:29.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.341+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f893c11fa90 con 0x7f893c131d10 2023-09-30T22:27:29.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 Processor -- start 2023-09-30T22:27:29.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 -- start start 2023-09-30T22:27:29.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00c40a1340 0x7f00c4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c409edd0 con 0x7f00c40a1340 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c409e1f0 con 0x7f00c40a0a30 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.463+0000 7f00c8f61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00c409dc50 con 0x7f00c4133fa0 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48686/0 (socket says 172.21.15.32:48686) 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 -- 172.21.15.32:0/2933022338 learned_addr learned my addr 172.21.15.32:0/2933022338 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52236/0 (socket says 172.21.15.32:52236) 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c1d9b700 1 --2- 172.21.15.32:0/2933022338 >> v2:172.21.15.32:3300/0 conn(0x7f00c40a1340 0x7f00c4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 -- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 msgr2=0x7f00c4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 --2- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 -- 172.21.15.32:0/2933022338 >> v2:172.21.15.32:3300/0 conn(0x7f00c40a1340 msgr2=0x7f00c4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 --2- 172.21.15.32:0/2933022338 >> v2:172.21.15.32:3300/0 conn(0x7f00c40a1340 0x7f00c4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 -- 172.21.15.32:0/2933022338 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40a1fe0 con 0x7f00c40a0a30 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c2d9d700 1 --2- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c1d9b700 1 --2- 172.21.15.32:0/2933022338 >> v2:172.21.15.32:3300/0 conn(0x7f00c40a1340 0x7f00c4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c259c700 1 --2- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c40a0e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009e80 tx=0x7f00b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d18a81f2733ab2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c159a700 1 -- 172.21.15.32:0/2933022338 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b400e070 con 0x7f00c40a0a30 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c159a700 1 -- 172.21.15.32:0/2933022338 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b400b8d0 con 0x7f00c40a0a30 2023-09-30T22:27:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.464+0000 7f00c159a700 1 -- 172.21.15.32:0/2933022338 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b400bc80 con 0x7f00c40a0a30 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 -- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 msgr2=0x7f00c40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 --2- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c40a0e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009e80 tx=0x7f00b400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 -- 172.21.15.32:0/2933022338 shutdown_connections 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 --2- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c40a0e00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 --2- 172.21.15.32:0/2933022338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 --2- 172.21.15.32:0/2933022338 >> v2:172.21.15.32:3300/0 conn(0x7f00c40a1340 0x7f00c4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 -- 172.21.15.32:0/2933022338 >> 172.21.15.32:0/2933022338 conn(0x7f00c40a6550 msgr2=0x7f00c40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 -- 172.21.15.32:0/2933022338 shutdown_connections 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.465+0000 7f00c8f61700 1 -- 172.21.15.32:0/2933022338 wait complete. 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.466+0000 7f00c8f61700 1 Processor -- start 2023-09-30T22:27:29.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 -- start start 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c4120150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 0x7f00c4120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41238d0 con 0x7f00c40a1340 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40a3610 con 0x7f00c40a0a30 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c8f61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00c4121db0 con 0x7f00c4133fa0 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 0x7f00c4120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 0x7f00c4120690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33826/0 (socket says 172.21.15.32:33826) 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c1d9b700 1 -- 172.21.15.32:0/47896917 learned_addr learned my addr 172.21.15.32:0/47896917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.467+0000 7f00c259c700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c4120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c1d9b700 1 -- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 msgr2=0x7f00c4120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c1d9b700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4120bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c1d9b700 1 -- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 msgr2=0x7f00c4120150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c1d9b700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c4120150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c1d9b700 1 -- 172.21.15.32:0/47896917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80094a0 con 0x7f00c40a1340 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c1d9b700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 0x7f00c4120690 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f00b800ebe0 tx=0x7f00b800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8009800 con 0x7f00c40a1340 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b800c850 con 0x7f00c40a1340 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc0094a0 con 0x7f00c40a1340 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800cbc0 con 0x7f00c40a1340 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.468+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b4009b30 con 0x7f00c40a1340 2023-09-30T22:27:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.469+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f00b800cd20 con 0x7f00c40a1340 2023-09-30T22:27:29.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.469+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f00b803fb40 con 0x7f00c40a1340 2023-09-30T22:27:29.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.469+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c4074ea0 con 0x7f00c40a1340 2023-09-30T22:27:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.474+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f00b8021080 con 0x7f00c40a1340 2023-09-30T22:27:29.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.553+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f893c11fa90 con 0x7f893c131d10 2023-09-30T22:27:29.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.553+0000 7f8931ffb700 1 -- 172.21.15.32:0/3001448412 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f893c11fa90 con 0x7f893c131d10 2023-09-30T22:27:29.554 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 msgr2=0x7f893c1c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c1c6c10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8934015000 tx=0x7f8934004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 shutdown_connections 2023-09-30T22:27:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893c131d10 0x7f893c1c6c10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893c06a770 0x7f893c1c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 --2- 172.21.15.32:0/3001448412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893c06b080 0x7f893c1c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 >> 172.21.15.32:0/3001448412 conn(0x7f893c066f50 msgr2=0x7f893c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 shutdown_connections 2023-09-30T22:27:29.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.556+0000 7f8943580700 1 -- 172.21.15.32:0/3001448412 wait complete. 2023-09-30T22:27:29.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.679+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f00c40a28e0 con 0x7f00c40a1340 2023-09-30T22:27:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.679+0000 7f00b2ffd700 1 -- 172.21.15.32:0/47896917 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00b8017020 con 0x7f00c40a1340 2023-09-30T22:27:29.680 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 msgr2=0x7f00c4120690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 0x7f00c4120690 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f00b800ebe0 tx=0x7f00b800eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 shutdown_connections 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00c4133fa0 0x7f00c4120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00c40a0a30 0x7f00c4120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 --2- 172.21.15.32:0/47896917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00c40a1340 0x7f00c4120690 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 >> 172.21.15.32:0/47896917 conn(0x7f00c40a6550 msgr2=0x7f00c41bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 shutdown_connections 2023-09-30T22:27:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:29.682+0000 7f00c8f61700 1 -- 172.21.15.32:0/47896917 wait complete. 2023-09-30T22:27:29.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:30.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:30.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.395+0000 7fea8ed6a700 1 Processor -- start 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.395+0000 7fea8ed6a700 1 -- start start 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8ed6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8ed6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea88134170 0x7fea8813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8ed6a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8809edd0 con 0x7fea880a1340 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8ed6a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8809e1f0 con 0x7fea88134170 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8ed6a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea8809dc50 con 0x7fea880a0a30 2023-09-30T22:27:30.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8cb06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea87fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea880a1340 0x7fea88133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46540/0 (socket says 172.21.15.32:46540) 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea87fff700 1 -- 172.21.15.32:0/3876066043 learned_addr learned my addr 172.21.15.32:0/3876066043 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8d307700 1 --2- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea88134170 0x7fea8813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea877fe700 1 -- 172.21.15.32:0/3876066043 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60208832 0 0) 0x7fea8809edd0 con 0x7fea880a1340 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8cb06700 1 -- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea88134170 msgr2=0x7fea8813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8cb06700 1 --2- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea88134170 0x7fea8813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8cb06700 1 -- 172.21.15.32:0/3876066043 >> v1:172.21.15.32:6789/0 conn(0x7fea880a1340 legacy=0x7fea88133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.396+0000 7fea8cb06700 1 -- 172.21.15.32:0/3876066043 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea880a1fe0 con 0x7fea880a0a30 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.397+0000 7fea8cb06700 1 --2- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea880a0a30 0x7fea880a0e00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea78009870 tx=0x7fea78009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d948ff9529f0d16 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.397+0000 7fea877fe700 1 -- 172.21.15.32:0/3876066043 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78011830 con 0x7fea880a0a30 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.397+0000 7fea877fe700 1 -- 172.21.15.32:0/3876066043 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78011990 con 0x7fea880a0a30 2023-09-30T22:27:30.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.397+0000 7fea877fe700 1 -- 172.21.15.32:0/3876066043 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78011d20 con 0x7fea880a0a30 2023-09-30T22:27:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.397+0000 7fea8ed6a700 1 -- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea880a0a30 msgr2=0x7fea880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.397+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea880a0a30 0x7fea880a0e00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea78009870 tx=0x7fea78009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.398+0000 7fea8ed6a700 1 -- 172.21.15.32:0/3876066043 shutdown_connections 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.398+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea88134170 0x7fea8813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.398+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/3876066043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea880a0a30 0x7fea880a0e00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.398+0000 7fea8ed6a700 1 -- 172.21.15.32:0/3876066043 >> 172.21.15.32:0/3876066043 conn(0x7fea880a6550 msgr2=0x7fea880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.398+0000 7fea8ed6a700 1 -- 172.21.15.32:0/3876066043 shutdown_connections 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.398+0000 7fea8ed6a700 1 -- 172.21.15.32:0/3876066043 wait complete. 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 Processor -- start 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 -- start start 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea880a0a30 0x7fea88122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea880a1340 0x7fea88122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea88134170 0x7fea881231f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881336a0 con 0x7fea880a0a30 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea88124400 con 0x7fea880a1340 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8ed6a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea880a0220 con 0x7fea88134170 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.399+0000 7fea8d307700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea88134170 0x7fea881231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea880a0a30 0x7fea88122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8d307700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea88134170 0x7fea881231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:52276/0 (socket says 172.21.15.32:52276) 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8d307700 1 -- 172.21.15.32:0/2331361004 learned_addr learned my addr 172.21.15.32:0/2331361004 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea87fff700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea880a1340 0x7fea88122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 -- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea88134170 msgr2=0x7fea881231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea88134170 0x7fea881231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 -- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea880a1340 msgr2=0x7fea88122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea880a1340 0x7fea88122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 -- 172.21.15.32:0/2331361004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780094a0 con 0x7fea880a0a30 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8cb06700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea880a0a30 0x7fea88122770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fea78016000 tx=0x7fea78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78004550 con 0x7fea880a0a30 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7801f070 con 0x7fea880a0a30 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea880a25e0 con 0x7fea880a0a30 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780038e0 con 0x7fea880a0a30 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.400+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea881331e0 con 0x7fea880a0a30 2023-09-30T22:27:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.401+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea78003a40 con 0x7fea880a0a30 2023-09-30T22:27:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.401+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea78021070 con 0x7fea880a0a30 2023-09-30T22:27:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.401+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88074ea0 con 0x7fea880a0a30 2023-09-30T22:27:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.410+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea78015020 con 0x7fea880a0a30 2023-09-30T22:27:30.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.435+0000 7f54db709700 1 Processor -- start 2023-09-30T22:27:30.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.435+0000 7f54db709700 1 -- start start 2023-09-30T22:27:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54db709700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54db709700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d4135e50 0x7f54d4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54db709700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d409f130 con 0x7f54d409d6f0 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54db709700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d409f520 con 0x7f54d409e1d0 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54db709700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54d409e930 con 0x7f54d4135e50 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54d9ca6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d4135e50 0x7f54d4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54d94a5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f54d409d6f0 0x7f54d409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34948/0 (socket says 172.21.15.32:34948) 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54d94a5700 1 -- 172.21.15.32:0/1860811696 learned_addr learned my addr 172.21.15.32:0/1860811696 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:30.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.436+0000 7f54d8ca4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d8ca4700 1 -- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d4135e50 msgr2=0x7f54d4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d3fff700 1 -- 172.21.15.32:0/1860811696 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231927235 0 0) 0x7f54d409f130 con 0x7f54d409d6f0 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d8ca4700 1 --2- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d4135e50 0x7f54d4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d8ca4700 1 -- 172.21.15.32:0/1860811696 >> v1:172.21.15.32:6789/0 conn(0x7f54d409d6f0 legacy=0x7f54d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d8ca4700 1 -- 172.21.15.32:0/1860811696 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d409b2d0 con 0x7f54d409e1d0 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d9ca6700 1 --2- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d4135e50 0x7f54d4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d8ca4700 1 --2- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d4135910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009870 tx=0x7f54c8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e349e3e88311062 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d3fff700 1 -- 172.21.15.32:0/1860811696 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011830 con 0x7f54d409e1d0 2023-09-30T22:27:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d3fff700 1 -- 172.21.15.32:0/1860811696 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c8011990 con 0x7f54d409e1d0 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.437+0000 7f54d3fff700 1 -- 172.21.15.32:0/1860811696 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011d20 con 0x7f54d409e1d0 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 -- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 msgr2=0x7f54d4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 --2- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d4135910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009870 tx=0x7f54c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 -- 172.21.15.32:0/1860811696 shutdown_connections 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 --2- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d4135e50 0x7f54d4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 --2- 172.21.15.32:0/1860811696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d4135910 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 -- 172.21.15.32:0/1860811696 >> 172.21.15.32:0/1860811696 conn(0x7f54d4066f50 msgr2=0x7f54d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 -- 172.21.15.32:0/1860811696 shutdown_connections 2023-09-30T22:27:30.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.438+0000 7f54db709700 1 -- 172.21.15.32:0/1860811696 wait complete. 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 Processor -- start 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 -- start start 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d409d6f0 0x7f54d411e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d411e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 0x7f54d41c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d4121550 con 0x7f54d4135e50 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d41303a0 con 0x7f54d409e1d0 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.440+0000 7f54db709700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54d411e020 con 0x7f54d409d6f0 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 0x7f54d41c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d94a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d409d6f0 0x7f54d411e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 0x7f54d41c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39102/0 (socket says 172.21.15.32:39102) 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 -- 172.21.15.32:0/3793071149 learned_addr learned my addr 172.21.15.32:0/3793071149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d8ca4700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d411e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 -- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d409d6f0 msgr2=0x7f54d411e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d409d6f0 0x7f54d411e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 -- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 msgr2=0x7f54d411e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d411e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 -- 172.21.15.32:0/3793071149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0094a0 con 0x7f54d4135e50 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d8ca4700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d411e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d9ca6700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 0x7f54d41c8460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00eb30 tx=0x7f54cc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc009780 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc009e60 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c80094a0 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.441+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00ca50 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.442+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d4072150 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.443+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f54cc00cbb0 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.443+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f54cc03fa90 con 0x7f54d4135e50 2023-09-30T22:27:30.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.443+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d409a520 con 0x7f54d4135e50 2023-09-30T22:27:30.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.448+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54cc017080 con 0x7f54d4135e50 2023-09-30T22:27:30.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:30.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.621+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea880a28e0 con 0x7fea880a0a30 2023-09-30T22:27:30.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.621+0000 7fea857fa700 1 -- 172.21.15.32:0/2331361004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea7801a950 con 0x7fea880a0a30 2023-09-30T22:27:30.622 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea880a0a30 msgr2=0x7fea88122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea880a0a30 0x7fea88122770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fea78016000 tx=0x7fea78004750 comp rx=0 tx=0).stop 2023-09-30T22:27:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 shutdown_connections 2023-09-30T22:27:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea88134170 0x7fea881231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea880a0a30 0x7fea88122770 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 --2- 172.21.15.32:0/2331361004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea880a1340 0x7fea88122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 >> 172.21.15.32:0/2331361004 conn(0x7fea880a6550 msgr2=0x7fea8809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 shutdown_connections 2023-09-30T22:27:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.625+0000 7fea8ed6a700 1 -- 172.21.15.32:0/2331361004 wait complete. 2023-09-30T22:27:30.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.655+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54d41211d0 con 0x7f54d4135e50 2023-09-30T22:27:30.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.656+0000 7f54d1ffb700 1 -- 172.21.15.32:0/3793071149 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f54cc021020 con 0x7f54d4135e50 2023-09-30T22:27:30.657 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:30.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 msgr2=0x7f54d41c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 0x7f54d41c8460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00eb30 tx=0x7f54cc00ee40 comp rx=0 tx=0).stop 2023-09-30T22:27:30.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 shutdown_connections 2023-09-30T22:27:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54d4135e50 0x7f54d41c8460 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54d409d6f0 0x7f54d411e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 --2- 172.21.15.32:0/3793071149 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54d409e1d0 0x7f54d411e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 >> 172.21.15.32:0/3793071149 conn(0x7f54d4066f50 msgr2=0x7f54d4132780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 shutdown_connections 2023-09-30T22:27:30.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.658+0000 7f54db709700 1 -- 172.21.15.32:0/3793071149 wait complete. 2023-09-30T22:27:30.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:30.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:30.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.719+0000 7f8af9c68700 1 Processor -- start 2023-09-30T22:27:30.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.719+0000 7f8af9c68700 1 -- start start 2023-09-30T22:27:30.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af9c68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af9c68700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4139170 0x7f8af413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af9c68700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af409b120 con 0x7f8af409d6f0 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af9c68700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af406b460 con 0x7f8af409e1d0 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af9c68700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8af409a7e0 con 0x7f8af4139170 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8af409d6f0 0x7f8af409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34962/0 (socket says 172.21.15.32:34962) 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af37fe700 1 -- 172.21.15.32:0/1459030950 learned_addr learned my addr 172.21.15.32:0/1459030950 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af3fff700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4139170 0x7f8af413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4139170 msgr2=0x7f8af413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:30.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4139170 0x7f8af413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1459030950 >> v1:172.21.15.32:6789/0 conn(0x7f8af409d6f0 legacy=0x7f8af409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1459030950 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af409a500 con 0x7f8af409e1d0 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.720+0000 7f8af27fc700 1 -- 172.21.15.32:0/1459030950 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019835912 0 0) 0x7f8af409b120 con 0x7f8af409d6f0 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.721+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4138c30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009870 tx=0x7f8ae4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=127d37f56bd52215 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.721+0000 7f8af27fc700 1 -- 172.21.15.32:0/1459030950 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4011830 con 0x7f8af409e1d0 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.721+0000 7f8af27fc700 1 -- 172.21.15.32:0/1459030950 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4011990 con 0x7f8af409e1d0 2023-09-30T22:27:30.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.721+0000 7f8af27fc700 1 -- 172.21.15.32:0/1459030950 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4011d20 con 0x7f8af409e1d0 2023-09-30T22:27:30.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.721+0000 7f8af9c68700 1 -- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 msgr2=0x7f8af4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.721+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4138c30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009870 tx=0x7f8ae4009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.722+0000 7f8af9c68700 1 -- 172.21.15.32:0/1459030950 shutdown_connections 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.722+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af4139170 0x7f8af413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.722+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1459030950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4138c30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.722+0000 7f8af9c68700 1 -- 172.21.15.32:0/1459030950 >> 172.21.15.32:0/1459030950 conn(0x7f8af4066f50 msgr2=0x7f8af409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.722+0000 7f8af9c68700 1 -- 172.21.15.32:0/1459030950 shutdown_connections 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.722+0000 7f8af9c68700 1 -- 172.21.15.32:0/1459030950 wait complete. 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 Processor -- start 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 -- start start 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409d6f0 0x7f8af41217b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4121cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4139170 0x7f8af4122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4124230 con 0x7f8af4139170 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4124f50 con 0x7f8af409e1d0 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af9c68700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40a2090 con 0x7f8af409d6f0 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4121cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4139170 0x7f8af4122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4121cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52622/0 (socket says 172.21.15.32:52622) 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1049691954 learned_addr learned my addr 172.21.15.32:0/1049691954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.723+0000 7f8af37fe700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409d6f0 0x7f8af41217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409d6f0 msgr2=0x7f8af41217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409d6f0 0x7f8af41217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4139170 msgr2=0x7f8af4122230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4139170 0x7f8af4122230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af2ffd700 1 -- 172.21.15.32:0/1049691954 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40094a0 con 0x7f8af409e1d0 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af3fff700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4139170 0x7f8af4122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af37fe700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409d6f0 0x7f8af41217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4121cf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4000c00 tx=0x7f8ae4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4024620 con 0x7f8af409e1d0 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4024780 con 0x7f8af409e1d0 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af409c710 con 0x7f8af409e1d0 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4024b30 con 0x7f8af409e1d0 2023-09-30T22:27:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.724+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41259c0 con 0x7f8af409e1d0 2023-09-30T22:27:30.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.725+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ae4022070 con 0x7f8af409e1d0 2023-09-30T22:27:30.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.725+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8ae4040a00 con 0x7f8af409e1d0 2023-09-30T22:27:30.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.725+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4074ea0 con 0x7f8af409e1d0 2023-09-30T22:27:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.730+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ae4015080 con 0x7f8af409e1d0 2023-09-30T22:27:30.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.851+0000 7f39d0501700 1 Processor -- start 2023-09-30T22:27:30.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 -- start start 2023-09-30T22:27:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 0x7f39c80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39c8133930 0x7f39c8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c80a2a70 con 0x7f39c8133930 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80a01d0 con 0x7f39c80a0a30 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39d0501700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39c809e2b0 con 0x7f39c809d630 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.852+0000 7f39cda9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 0x7f39c80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cea9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39c8133930 0x7f39c8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cda9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 0x7f39c80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40398/0 (socket says 172.21.15.32:40398) 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cda9c700 1 -- 172.21.15.32:0/2938436997 learned_addr learned my addr 172.21.15.32:0/2938436997 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 -- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 msgr2=0x7f39c80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 --2- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 0x7f39c80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 -- 172.21.15.32:0/2938436997 >> v2:172.21.15.32:3300/0 conn(0x7f39c8133930 msgr2=0x7f39c8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 --2- 172.21.15.32:0/2938436997 >> v2:172.21.15.32:3300/0 conn(0x7f39c8133930 0x7f39c8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 -- 172.21.15.32:0/2938436997 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c809dc50 con 0x7f39c80a0a30 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cda9c700 1 --2- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 0x7f39c80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cea9e700 1 --2- 172.21.15.32:0/2938436997 >> v2:172.21.15.32:3300/0 conn(0x7f39c8133930 0x7f39c8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39ce29d700 1 --2- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009840 tx=0x7f39b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c20154a6c2e675 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cd29b700 1 -- 172.21.15.32:0/2938436997 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8011830 con 0x7f39c80a0a30 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cd29b700 1 -- 172.21.15.32:0/2938436997 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b8011990 con 0x7f39c80a0a30 2023-09-30T22:27:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.853+0000 7f39cd29b700 1 -- 172.21.15.32:0/2938436997 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8011d20 con 0x7f39c80a0a30 2023-09-30T22:27:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 -- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 msgr2=0x7f39c80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 --2- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009840 tx=0x7f39b8009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 -- 172.21.15.32:0/2938436997 shutdown_connections 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 --2- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c80a0a30 0x7f39c80a0e00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 --2- 172.21.15.32:0/2938436997 >> v2:172.21.15.32:3300/0 conn(0x7f39c8133930 0x7f39c8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 --2- 172.21.15.32:0/2938436997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c809d630 0x7f39c80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 -- 172.21.15.32:0/2938436997 >> 172.21.15.32:0/2938436997 conn(0x7f39c80a3050 msgr2=0x7f39c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 -- 172.21.15.32:0/2938436997 shutdown_connections 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.854+0000 7f39d0501700 1 -- 172.21.15.32:0/2938436997 wait complete. 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.855+0000 7f39d0501700 1 Processor -- start 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.855+0000 7f39d0501700 1 -- start start 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.855+0000 7f39d0501700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c809d630 0x7f39c811f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.855+0000 7f39d0501700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c811f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39d0501700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 0x7f39c81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39d0501700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c81204f0 con 0x7f39c809d630 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39d0501700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c811fde0 con 0x7f39c8133930 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39d0501700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8133270 con 0x7f39c80a0a30 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 0x7f39c81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39ce29d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c809d630 0x7f39c811f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 0x7f39c81c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52646/0 (socket says 172.21.15.32:52646) 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 -- 172.21.15.32:0/1146565266 learned_addr learned my addr 172.21.15.32:0/1146565266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cda9c700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c811f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 -- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 msgr2=0x7f39c811f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c811f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 -- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c809d630 msgr2=0x7f39c811f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c809d630 0x7f39c811f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 -- 172.21.15.32:0/1146565266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc009550 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39cea9e700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 0x7f39c81c8090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efc0 tx=0x7f39c000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000ca90 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c000f040 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000ce30 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.856+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c0009a70 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.857+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b80094a0 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.857+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f39c0014480 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.857+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f39c003fe60 con 0x7f39c8133930 2023-09-30T22:27:30.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.858+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c8121dc0 con 0x7f39c8133930 2023-09-30T22:27:30.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.863+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39c8121dc0 con 0x7f39c8133930 2023-09-30T22:27:30.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.937+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8af406c360 con 0x7f8af409e1d0 2023-09-30T22:27:30.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.938+0000 7f8adbfff700 1 -- 172.21.15.32:0/1049691954 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ae4029020 con 0x7f8af409e1d0 2023-09-30T22:27:30.939 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 msgr2=0x7f8af4121cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4121cf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4000c00 tx=0x7f8ae4004750 comp rx=0 tx=0).stop 2023-09-30T22:27:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 shutdown_connections 2023-09-30T22:27:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af4139170 0x7f8af4122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409d6f0 0x7f8af41217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 --2- 172.21.15.32:0/1049691954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409e1d0 0x7f8af4121cf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 >> 172.21.15.32:0/1049691954 conn(0x7f8af4066f50 msgr2=0x7f8af4070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 shutdown_connections 2023-09-30T22:27:30.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:30.940+0000 7f8af9c68700 1 -- 172.21.15.32:0/1049691954 wait complete. 2023-09-30T22:27:30.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.067+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f39c8121dc0 con 0x7f39c8133930 2023-09-30T22:27:31.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.068+0000 7f39c6ffd700 1 -- 172.21.15.32:0/1146565266 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39c8121dc0 con 0x7f39c8133930 2023-09-30T22:27:31.069 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 msgr2=0x7f39c81c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 0x7f39c81c8090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39c000efc0 tx=0x7f39c000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 shutdown_connections 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39c8133930 0x7f39c81c8090 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39c809d630 0x7f39c811f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 --2- 172.21.15.32:0/1146565266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39c80a0a30 0x7f39c811f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 >> 172.21.15.32:0/1146565266 conn(0x7f39c80a3050 msgr2=0x7f39c809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 shutdown_connections 2023-09-30T22:27:31.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.070+0000 7f39d0501700 1 -- 172.21.15.32:0/1146565266 wait complete. 2023-09-30T22:27:31.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:31.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:31.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:31.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.796+0000 7f03b3b94700 1 Processor -- start 2023-09-30T22:27:31.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.796+0000 7f03b3b94700 1 -- start start 2023-09-30T22:27:31.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.796+0000 7f03b3b94700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.796+0000 7f03b3b94700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03ac138d80 0x7f03ac13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b3b94700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac0a2a70 con 0x7f03ac0a0a30 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b3b94700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0a01d0 con 0x7f03ac138d80 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b3b94700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac09e2b0 con 0x7f03ac09d630 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b112f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b2131700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03ac138d80 0x7f03ac13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b112f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52650/0 (socket says 172.21.15.32:52650) 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b1930700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03ac0a0a30 0x7f03ac0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34970/0 (socket says 172.21.15.32:34970) 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b112f700 1 -- 172.21.15.32:0/1318995279 learned_addr learned my addr 172.21.15.32:0/1318995279 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b092e700 1 -- 172.21.15.32:0/1318995279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383063315 0 0) 0x7f03ac0a2a70 con 0x7f03ac0a0a30 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b092e700 1 -- 172.21.15.32:0/1318995279 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03980035b0 con 0x7f03ac0a0a30 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b2131700 1 -- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 msgr2=0x7f03ac0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:31.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b2131700 1 --2- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b2131700 1 -- 172.21.15.32:0/1318995279 >> v1:172.21.15.32:6789/0 conn(0x7f03ac0a0a30 legacy=0x7f03ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b2131700 1 -- 172.21.15.32:0/1318995279 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac09dc50 con 0x7f03ac138d80 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b112f700 1 --2- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.797+0000 7f03b2131700 1 --2- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac138d80 0x7f03ac13b160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009d90 tx=0x7f03a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e3bf02dc96bc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b092e700 1 -- 172.21.15.32:0/1318995279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400e070 con 0x7f03ac138d80 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b092e700 1 -- 172.21.15.32:0/1318995279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a400bb20 con 0x7f03ac138d80 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b092e700 1 -- 172.21.15.32:0/1318995279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4012430 con 0x7f03ac138d80 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 -- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac138d80 msgr2=0x7f03ac13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 --2- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac138d80 0x7f03ac13b160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009d90 tx=0x7f03a400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 -- 172.21.15.32:0/1318995279 shutdown_connections 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 --2- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac138d80 0x7f03ac13b160 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 --2- 172.21.15.32:0/1318995279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 -- 172.21.15.32:0/1318995279 >> 172.21.15.32:0/1318995279 conn(0x7f03ac0a3050 msgr2=0x7f03ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 -- 172.21.15.32:0/1318995279 shutdown_connections 2023-09-30T22:27:31.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.798+0000 7f03b3b94700 1 -- 172.21.15.32:0/1318995279 wait complete. 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.799+0000 7f03b3b94700 1 Processor -- start 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.799+0000 7f03b3b94700 1 -- start start 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b3b94700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b3b94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 0x7f03ac133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b3b94700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03ac138d80 0x7f03ac134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b3b94700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1356d0 con 0x7f03ac0a0a30 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b3b94700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac137c20 con 0x7f03ac09d630 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b3b94700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0a4990 con 0x7f03ac138d80 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 0x7f03ac133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b1930700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 0x7f03ac133c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39132/0 (socket says 172.21.15.32:39132) 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 -- 172.21.15.32:0/4213523653 learned_addr learned my addr 172.21.15.32:0/4213523653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 -- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03ac138d80 msgr2=0x7f03ac134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03ac138d80 0x7f03ac134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 -- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 msgr2=0x7f03ac133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 -- 172.21.15.32:0/4213523653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c009550 con 0x7f03ac0a0a30 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.800+0000 7f03b112f700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 0x7f03ac133c50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f039c00ec10 tx=0x7f039c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.801+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00c7f0 con 0x7f03ac0a0a30 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.801+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c00c950 con 0x7f03ac0a0a30 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.801+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a4009a70 con 0x7f03ac0a0a30 2023-09-30T22:27:31.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.801+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00ccc0 con 0x7f03ac0a0a30 2023-09-30T22:27:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.801+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac0a0410 con 0x7f03ac0a0a30 2023-09-30T22:27:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.801+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f039c013470 con 0x7f03ac0a0a30 2023-09-30T22:27:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.802+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f039c03f830 con 0x7f03ac0a0a30 2023-09-30T22:27:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.802+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac136450 con 0x7f03ac0a0a30 2023-09-30T22:27:31.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.807+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f039c013720 con 0x7f03ac0a0a30 2023-09-30T22:27:31.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 Processor -- start 2023-09-30T22:27:31.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 -- start start 2023-09-30T22:27:31.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 0x7f09b40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 0x7f09b40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b40a2a70 con 0x7f09b4133930 2023-09-30T22:27:31.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40a01d0 con 0x7f09b40a0a30 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.833+0000 7f09bbdf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09b409e2b0 con 0x7f09b409d630 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 0x7f09b40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b938f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 0x7f09b40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09ba391700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09b4133930 0x7f09b4133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34980/0 (socket says 172.21.15.32:34980) 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 0x7f09b40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52680/0 (socket says 172.21.15.32:52680) 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b938f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 0x7f09b40a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40420/0 (socket says 172.21.15.32:40420) 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09ba391700 1 -- 172.21.15.32:0/748809604 learned_addr learned my addr 172.21.15.32:0/748809604 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b8b8e700 1 -- 172.21.15.32:0/748809604 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799372839 0 0) 0x7f09b40a2a70 con 0x7f09b4133930 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 -- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 msgr2=0x7f09b40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 --2- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 0x7f09b40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 -- 172.21.15.32:0/748809604 >> v1:172.21.15.32:6789/0 conn(0x7f09b4133930 legacy=0x7f09b4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 -- 172.21.15.32:0/748809604 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b409dc50 con 0x7f09b40a0a30 2023-09-30T22:27:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b938f700 1 --2- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 0x7f09b40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b9b90700 1 --2- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 0x7f09b40a0e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009870 tx=0x7f09b0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7424f8e4fe0ee093 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b8b8e700 1 -- 172.21.15.32:0/748809604 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0011830 con 0x7f09b40a0a30 2023-09-30T22:27:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.834+0000 7f09b8b8e700 1 -- 172.21.15.32:0/748809604 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b0011990 con 0x7f09b40a0a30 2023-09-30T22:27:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09b8b8e700 1 -- 172.21.15.32:0/748809604 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0011d20 con 0x7f09b40a0a30 2023-09-30T22:27:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 -- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 msgr2=0x7f09b40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 0x7f09b40a0e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009870 tx=0x7f09b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 -- 172.21.15.32:0/748809604 shutdown_connections 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b40a0a30 0x7f09b40a0e00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/748809604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b409d630 0x7f09b40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 -- 172.21.15.32:0/748809604 >> 172.21.15.32:0/748809604 conn(0x7f09b40a3050 msgr2=0x7f09b40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 -- 172.21.15.32:0/748809604 shutdown_connections 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.835+0000 7f09bbdf4700 1 -- 172.21.15.32:0/748809604 wait complete. 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.836+0000 7f09bbdf4700 1 Processor -- start 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.836+0000 7f09bbdf4700 1 -- start start 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09bbdf4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b409d630 0x7f09b411f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09bbdf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b40a0a30 0x7f09b411f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09bbdf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b4133930 0x7f09b41c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09bbdf4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4121ed0 con 0x7f09b409d630 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09bbdf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4133250 con 0x7f09b4133930 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09bbdf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09b411ee20 con 0x7f09b40a0a30 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09ba391700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b4133930 0x7f09b41c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b40a0a30 0x7f09b411f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09ba391700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b4133930 0x7f09b41c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52684/0 (socket says 172.21.15.32:52684) 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09ba391700 1 -- 172.21.15.32:0/2872366581 learned_addr learned my addr 172.21.15.32:0/2872366581 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b9b90700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b409d630 0x7f09b411f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 -- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b4133930 msgr2=0x7f09b41c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b4133930 0x7f09b41c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 -- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b409d630 msgr2=0x7f09b411f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b409d630 0x7f09b411f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 -- 172.21.15.32:0/2872366581 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40094a0 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b9b90700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b409d630 0x7f09b411f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.837+0000 7f09b938f700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b40a0a30 0x7f09b411f800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09a400e980 tx=0x7f09a400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.838+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400eed0 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.838+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a400c7f0 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.838+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b00094a0 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.838+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400cb80 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.838+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b4120520 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.838+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09a400cce0 con 0x7f09b40a0a30 2023-09-30T22:27:31.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.839+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f09a403fac0 con 0x7f09b40a0a30 2023-09-30T22:27:31.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.839+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b4120a40 con 0x7f09b40a0a30 2023-09-30T22:27:31.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:31.844+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09a4021080 con 0x7f09b40a0a30 2023-09-30T22:27:31.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.022+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03ac13b620 con 0x7f03ac0a0a30 2023-09-30T22:27:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.023+0000 7f03a27fc700 1 -- 172.21.15.32:0/4213523653 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f039c01bba0 con 0x7f03ac0a0a30 2023-09-30T22:27:32.024 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 msgr2=0x7f03ac133c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 0x7f03ac133c50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f039c00ec10 tx=0x7f039c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 shutdown_connections 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03ac138d80 0x7f03ac134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03ac09d630 0x7f03ac133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 --2- 172.21.15.32:0/4213523653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03ac0a0a30 0x7f03ac133c50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 >> 172.21.15.32:0/4213523653 conn(0x7f03ac0a3050 msgr2=0x7f03ac13add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 shutdown_connections 2023-09-30T22:27:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.025+0000 7f03b3b94700 1 -- 172.21.15.32:0/4213523653 wait complete. 2023-09-30T22:27:32.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.050+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09b40a0410 con 0x7f09b40a0a30 2023-09-30T22:27:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.050+0000 7f09aa7fc700 1 -- 172.21.15.32:0/2872366581 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09a401b3b0 con 0x7f09b40a0a30 2023-09-30T22:27:32.051 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b40a0a30 msgr2=0x7f09b411f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b40a0a30 0x7f09b411f800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09a400e980 tx=0x7f09a400ec90 comp rx=0 tx=0).stop 2023-09-30T22:27:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 shutdown_connections 2023-09-30T22:27:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09b4133930 0x7f09b41c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09b409d630 0x7f09b411f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 --2- 172.21.15.32:0/2872366581 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09b40a0a30 0x7f09b411f800 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 >> 172.21.15.32:0/2872366581 conn(0x7f09b40a3050 msgr2=0x7f09b41bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.052+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 shutdown_connections 2023-09-30T22:27:32.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.053+0000 7f09bbdf4700 1 -- 172.21.15.32:0/2872366581 wait complete. 2023-09-30T22:27:32.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:32.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:32.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.097+0000 7fe1049eb700 1 Processor -- start 2023-09-30T22:27:32.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.097+0000 7fe1049eb700 1 -- start start 2023-09-30T22:27:32.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe1049eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1000a0a30 0x7fe1000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe1049eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe10009d630 0x7fe1000a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe1049eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 0x7fe100133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe1049eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1000a2a70 con 0x7fe1000a0a30 2023-09-30T22:27:32.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe1049eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000a01d0 con 0x7fe100133930 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe1049eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe10009e2b0 con 0x7fe10009d630 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1000a0a30 0x7fe1000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 0x7fe100133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fdd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe10009d630 0x7fe1000a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1000a0a30 0x7fe1000a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39150/0 (socket says 172.21.15.32:39150) 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 0x7fe100133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52690/0 (socket says 172.21.15.32:52690) 2023-09-30T22:27:32.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 -- 172.21.15.32:0/1394188547 learned_addr learned my addr 172.21.15.32:0/1394188547 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 -- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe10009d630 msgr2=0x7fe1000a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe10009d630 0x7fe1000a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 -- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 msgr2=0x7fe100133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 0x7fe100133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 -- 172.21.15.32:0/1394188547 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10009dc50 con 0x7fe1000a0a30 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fed9d700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 0x7fe100133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.098+0000 7fe0fe59c700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1000a0a30 0x7fe1000a0e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00098f0 tx=0x7fe0f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d4fdbf4f6d6e8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.099+0000 7fe0fd59a700 1 -- 172.21.15.32:0/1394188547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0011830 con 0x7fe1000a0a30 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.099+0000 7fe0fd59a700 1 -- 172.21.15.32:0/1394188547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0011990 con 0x7fe1000a0a30 2023-09-30T22:27:32.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.099+0000 7fe0fd59a700 1 -- 172.21.15.32:0/1394188547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0011d20 con 0x7fe1000a0a30 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.099+0000 7fe1049eb700 1 -- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1000a0a30 msgr2=0x7fe1000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.099+0000 7fe1049eb700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1000a0a30 0x7fe1000a0e00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00098f0 tx=0x7fe0f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 -- 172.21.15.32:0/1394188547 shutdown_connections 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe100133930 0x7fe100133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1000a0a30 0x7fe1000a0e00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 --2- 172.21.15.32:0/1394188547 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe10009d630 0x7fe1000a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 -- 172.21.15.32:0/1394188547 >> 172.21.15.32:0/1394188547 conn(0x7fe1000a3050 msgr2=0x7fe1000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 -- 172.21.15.32:0/1394188547 shutdown_connections 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.100+0000 7fe1049eb700 1 -- 172.21.15.32:0/1394188547 wait complete. 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 Processor -- start 2023-09-30T22:27:32.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 -- start start 2023-09-30T22:27:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 0x7fe1001c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1000a0a30 0x7fe1001c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe100133930 0x7fe1001c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001332b0 con 0x7fe100133930 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001206b0 con 0x7fe10009d630 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.101+0000 7fe1049eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000a52d0 con 0x7fe1000a0a30 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 0x7fe1001c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 0x7fe1001c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52698/0 (socket says 172.21.15.32:52698) 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe100133930 0x7fe1001c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fdd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1000a0a30 0x7fe1001c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fe59c700 1 -- 172.21.15.32:0/2487075817 learned_addr learned my addr 172.21.15.32:0/2487075817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 -- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1000a0a30 msgr2=0x7fe1001c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1000a0a30 0x7fe1001c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 -- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 msgr2=0x7fe1001c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 0x7fe1001c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 -- 172.21.15.32:0/2487075817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8009a70 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fe59c700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 0x7fe1001c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fdd9b700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1000a0a30 0x7fe1001c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0fed9d700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe100133930 0x7fe1001c8b10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009e50 tx=0x7fe0f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800ca90 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f800f040 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f0009550 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.102+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800ce50 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.103+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe100120d20 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.103+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe0f8014480 con 0x7fe100133930 2023-09-30T22:27:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.103+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe0f803fbf0 con 0x7fe100133930 2023-09-30T22:27:32.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.104+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1000747f0 con 0x7fe100133930 2023-09-30T22:27:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.109+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe0f8012930 con 0x7fe100133930 2023-09-30T22:27:32.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 Processor -- start 2023-09-30T22:27:32.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 -- start start 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f782809d630 0x7f78280a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 0x7f782813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78280a2a70 con 0x7f78280a0a30 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280a01d0 con 0x7f782809d630 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f782dd83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f782809e2b0 con 0x7f7828138d80 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f7827fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 0x7f782813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f7826ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f782809d630 0x7f78280a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f7826ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f782809d630 0x7f78280a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39162/0 (socket says 172.21.15.32:39162) 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f7827fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 0x7f782813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40466/0 (socket says 172.21.15.32:40466) 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f78277fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f78280a0a30 0x7f78280a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34992/0 (socket says 172.21.15.32:34992) 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.237+0000 7f7826ffd700 1 -- 172.21.15.32:0/382957071 learned_addr learned my addr 172.21.15.32:0/382957071 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f7826ffd700 1 -- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 msgr2=0x7f782813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f7826ffd700 1 --2- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 0x7f782813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f7826ffd700 1 -- 172.21.15.32:0/382957071 >> v1:172.21.15.32:6789/0 conn(0x7f78280a0a30 legacy=0x7f78280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f7826ffd700 1 -- 172.21.15.32:0/382957071 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782809dc50 con 0x7f782809d630 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f78267fc700 1 -- 172.21.15.32:0/382957071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817148626 0 0) 0x7f78280a2a70 con 0x7f78280a0a30 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f7827fff700 1 --2- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 0x7f782813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f7826ffd700 1 --2- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782809d630 0x7f78280a1510 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f78100098f0 tx=0x7f7810009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62b818e0be278cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f78267fc700 1 -- 172.21.15.32:0/382957071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810011830 con 0x7f782809d630 2023-09-30T22:27:32.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f78267fc700 1 -- 172.21.15.32:0/382957071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810011990 con 0x7f782809d630 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.238+0000 7f78267fc700 1 -- 172.21.15.32:0/382957071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810011d20 con 0x7f782809d630 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 -- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782809d630 msgr2=0x7f78280a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 --2- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782809d630 0x7f78280a1510 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f78100098f0 tx=0x7f7810009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 -- 172.21.15.32:0/382957071 shutdown_connections 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 --2- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7828138d80 0x7f782813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 --2- 172.21.15.32:0/382957071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f782809d630 0x7f78280a1510 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 -- 172.21.15.32:0/382957071 >> 172.21.15.32:0/382957071 conn(0x7f78280a3050 msgr2=0x7f78280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:32.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 -- 172.21.15.32:0/382957071 shutdown_connections 2023-09-30T22:27:32.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.239+0000 7f782dd83700 1 -- 172.21.15.32:0/382957071 wait complete. 2023-09-30T22:27:32.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 Processor -- start 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 -- start start 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f782809d630 0x7f7828133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 0x7f7828133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7828138d80 0x7f7828134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7828136190 con 0x7f78280a0a30 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7828136eb0 con 0x7f7828138d80 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.240+0000 7f782dd83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78280a4ba0 con 0x7f782809d630 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7826ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 0x7f7828133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f78277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f782809d630 0x7f7828133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7826ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 0x7f7828133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39176/0 (socket says 172.21.15.32:39176) 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7826ffd700 1 -- 172.21.15.32:0/1044083548 learned_addr learned my addr 172.21.15.32:0/1044083548 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:32.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7828138d80 0x7f7828134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 -- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f782809d630 msgr2=0x7f7828133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f782809d630 0x7f7828133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 -- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 msgr2=0x7f7828133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 0x7f7828133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 -- 172.21.15.32:0/1044083548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200094a0 con 0x7f7828138d80 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7826ffd700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 0x7f7828133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f7827fff700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7828138d80 0x7f7828134180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7820009fd0 tx=0x7f782000ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820013680 con 0x7f7828138d80 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820013800 con 0x7f7828138d80 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7810009550 con 0x7f7828138d80 2023-09-30T22:27:32.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820013bb0 con 0x7f7828138d80 2023-09-30T22:27:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.241+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7828136540 con 0x7f7828138d80 2023-09-30T22:27:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.242+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7820013d10 con 0x7f7828138d80 2023-09-30T22:27:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.243+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78281379e0 con 0x7f7828138d80 2023-09-30T22:27:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.243+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f782003fb20 con 0x7f7828138d80 2023-09-30T22:27:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.247+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7820017020 con 0x7f7828138d80 2023-09-30T22:27:32.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.324+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe1001211a0 con 0x7fe100133930 2023-09-30T22:27:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.325+0000 7fe0f6ffd700 1 -- 172.21.15.32:0/2487075817 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0f8012340 con 0x7fe100133930 2023-09-30T22:27:32.326 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe100133930 msgr2=0x7fe1001c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe100133930 0x7fe1001c8b10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009e50 tx=0x7fe0f800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 shutdown_connections 2023-09-30T22:27:32.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe100133930 0x7fe1001c8b10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe10009d630 0x7fe1001c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 --2- 172.21.15.32:0/2487075817 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1000a0a30 0x7fe1001c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 >> 172.21.15.32:0/2487075817 conn(0x7fe1000a3050 msgr2=0x7fe1001bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:32.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 shutdown_connections 2023-09-30T22:27:32.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.327+0000 7fe1049eb700 1 -- 172.21.15.32:0/2487075817 wait complete. 2023-09-30T22:27:32.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.454+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78280747f0 con 0x7f7828138d80 2023-09-30T22:27:32.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.454+0000 7f781ffff700 1 -- 172.21.15.32:0/1044083548 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f782001ada0 con 0x7f7828138d80 2023-09-30T22:27:32.456 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.456+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7828138d80 msgr2=0x7f7828134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.456+0000 7f782dd83700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7828138d80 0x7f7828134180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7820009fd0 tx=0x7f782000ebe0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 shutdown_connections 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7828138d80 0x7f7828134180 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f782809d630 0x7f7828133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 --2- 172.21.15.32:0/1044083548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78280a0a30 0x7f7828133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 >> 172.21.15.32:0/1044083548 conn(0x7f78280a3050 msgr2=0x7f782809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 shutdown_connections 2023-09-30T22:27:32.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:32.457+0000 7f782dd83700 1 -- 172.21.15.32:0/1044083548 wait complete. 2023-09-30T22:27:32.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:33.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:33.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.193+0000 7f7c52d38700 1 Processor -- start 2023-09-30T22:27:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.193+0000 7f7c52d38700 1 -- start start 2023-09-30T22:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c52d38700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 0x7f7c4c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c52d38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c0a1340 0x7f7c4c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c52d38700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c133fa0 0x7f7c4c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c52d38700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c09edd0 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c52d38700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c09e1f0 con 0x7f7c4c133fa0 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c52d38700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c09dc50 con 0x7f7c4c0a1340 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c50ad4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 0x7f7c4c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c0a1340 0x7f7c4c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c50ad4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 0x7f7c4c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39192/0 (socket says 172.21.15.32:39192) 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c50ad4700 1 -- 172.21.15.32:0/2904290082 learned_addr learned my addr 172.21.15.32:0/2904290082 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c512d5700 1 --2- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c133fa0 0x7f7c4c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 -- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c133fa0 msgr2=0x7f7c4c134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 --2- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c133fa0 0x7f7c4c134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 -- 172.21.15.32:0/2904290082 >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 msgr2=0x7f7c4c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:33.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 --2- 172.21.15.32:0/2904290082 >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 0x7f7c4c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 -- 172.21.15.32:0/2904290082 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c0a1fe0 con 0x7f7c4c0a1340 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c50ad4700 1 --2- 172.21.15.32:0/2904290082 >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 0x7f7c4c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.194+0000 7f7c4bfff700 1 --2- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c0a1340 0x7f7c4c133a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009870 tx=0x7f7c38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254a7ab0e4507211 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c4b7fe700 1 -- 172.21.15.32:0/2904290082 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38011830 con 0x7f7c4c0a1340 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c4b7fe700 1 -- 172.21.15.32:0/2904290082 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c38011990 con 0x7f7c4c0a1340 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c4b7fe700 1 -- 172.21.15.32:0/2904290082 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38011d20 con 0x7f7c4c0a1340 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 -- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c0a1340 msgr2=0x7f7c4c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 --2- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c0a1340 0x7f7c4c133a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009870 tx=0x7f7c38009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:33.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 -- 172.21.15.32:0/2904290082 shutdown_connections 2023-09-30T22:27:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 --2- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c133fa0 0x7f7c4c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 --2- 172.21.15.32:0/2904290082 >> v2:172.21.15.32:3300/0 conn(0x7f7c4c0a0a30 0x7f7c4c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 --2- 172.21.15.32:0/2904290082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c0a1340 0x7f7c4c133a60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 -- 172.21.15.32:0/2904290082 >> 172.21.15.32:0/2904290082 conn(0x7f7c4c0a6550 msgr2=0x7f7c4c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 -- 172.21.15.32:0/2904290082 shutdown_connections 2023-09-30T22:27:33.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.195+0000 7f7c52d38700 1 -- 172.21.15.32:0/2904290082 wait complete. 2023-09-30T22:27:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.196+0000 7f7c52d38700 1 Processor -- start 2023-09-30T22:27:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.196+0000 7f7c52d38700 1 -- start start 2023-09-30T22:27:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c52d38700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 0x7f7c4c120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c52d38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c4c0a1340 0x7f7c4c120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c52d38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c133fa0 0x7f7c4c120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c52d38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c1238c0 con 0x7f7c4c0a1340 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c52d38700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0a3630 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c52d38700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0a0220 con 0x7f7c4c133fa0 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c50ad4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 0x7f7c4c120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c4c0a1340 0x7f7c4c120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c50ad4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 0x7f7c4c120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52726/0 (socket says 172.21.15.32:52726) 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c4bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c4c0a1340 0x7f7c4c120640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39200/0 (socket says 172.21.15.32:39200) 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c50ad4700 1 -- 172.21.15.32:0/3915656492 learned_addr learned my addr 172.21.15.32:0/3915656492 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.197+0000 7f7c512d5700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c133fa0 0x7f7c4c120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c50ad4700 1 -- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c133fa0 msgr2=0x7f7c4c120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c50ad4700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c133fa0 0x7f7c4c120b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c50ad4700 1 -- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c4c0a1340 msgr2=0x7f7c4c120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c50ad4700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c4c0a1340 0x7f7c4c120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c50ad4700 1 -- 172.21.15.32:0/3915656492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40009550 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c50ad4700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 0x7f7c4c120100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000ec10 tx=0x7f7c4000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000c850 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4000c9b0 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c380094a0 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000cd40 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.198+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c122ee0 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.199+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c40013470 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.199+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c4003fc20 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.199+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c074ea0 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.204+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c40021080 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.230+0000 7f2316d9c700 1 Processor -- start 2023-09-30T22:27:33.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.230+0000 7f2316d9c700 1 -- start start 2023-09-30T22:27:33.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f2316d9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 0x7f23100a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f2316d9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2310138d80 0x7f231013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f2316d9c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23100a2a70 con 0x7f23100a0a30 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f2316d9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100a01d0 con 0x7f231009d630 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f2316d9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f231009e2b0 con 0x7f2310138d80 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 0x7f23100a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 0x7f23100a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52742/0 (socket says 172.21.15.32:52742) 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 -- 172.21.15.32:0/4269642651 learned_addr learned my addr 172.21.15.32:0/4269642651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f2315339700 1 --2- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2310138d80 0x7f231013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 -- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2310138d80 msgr2=0x7f231013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 --2- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2310138d80 0x7f231013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 -- 172.21.15.32:0/4269642651 >> v1:172.21.15.32:6789/0 conn(0x7f23100a0a30 legacy=0x7f23100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230ffff700 1 -- 172.21.15.32:0/4269642651 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231009dc50 con 0x7f231009d630 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.231+0000 7f230f7fe700 1 -- 172.21.15.32:0/4269642651 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983675563 0 0) 0x7f23100a2a70 con 0x7f23100a0a30 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f230ffff700 1 --2- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 0x7f23100a1510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009920 tx=0x7f22fc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85812c60b10e89e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f230f7fe700 1 -- 172.21.15.32:0/4269642651 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc011830 con 0x7f231009d630 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f230f7fe700 1 -- 172.21.15.32:0/4269642651 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc011990 con 0x7f231009d630 2023-09-30T22:27:33.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f230f7fe700 1 -- 172.21.15.32:0/4269642651 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc011d20 con 0x7f231009d630 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 -- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 msgr2=0x7f23100a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 --2- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 0x7f23100a1510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009920 tx=0x7f22fc009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 -- 172.21.15.32:0/4269642651 shutdown_connections 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 --2- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2310138d80 0x7f231013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 --2- 172.21.15.32:0/4269642651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f231009d630 0x7f23100a1510 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 -- 172.21.15.32:0/4269642651 >> 172.21.15.32:0/4269642651 conn(0x7f23100a3050 msgr2=0x7f23100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 -- 172.21.15.32:0/4269642651 shutdown_connections 2023-09-30T22:27:33.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.232+0000 7f2316d9c700 1 -- 172.21.15.32:0/4269642651 wait complete. 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.233+0000 7f2316d9c700 1 Processor -- start 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.233+0000 7f2316d9c700 1 -- start start 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2316d9c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 0x7f2310133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2316d9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 0x7f2310133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2316d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2310138d80 0x7f23101341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2316d9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310137220 con 0x7f2310138d80 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2316d9c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2310137d60 con 0x7f23100a0a30 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2316d9c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23100a4cb0 con 0x7f231009d630 2023-09-30T22:27:33.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2314b38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 0x7f2310133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f230ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 0x7f2310133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f230ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 0x7f2310133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52750/0 (socket says 172.21.15.32:52750) 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2314b38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 0x7f2310133730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40528/0 (socket says 172.21.15.32:40528) 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f230ffff700 1 -- 172.21.15.32:0/2027416816 learned_addr learned my addr 172.21.15.32:0/2027416816 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.234+0000 7f2315339700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2310138d80 0x7f23101341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230ffff700 1 -- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 msgr2=0x7f2310133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230ffff700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 0x7f2310133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230ffff700 1 -- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2310138d80 msgr2=0x7f23101341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230ffff700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2310138d80 0x7f23101341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230ffff700 1 -- 172.21.15.32:0/2027416816 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc009550 con 0x7f23100a0a30 2023-09-30T22:27:33.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f2314b38700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 0x7f2310133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230ffff700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 0x7f2310133c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00a010 tx=0x7f22fc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc004240 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc01f070 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0045e0 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23100a0410 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.235+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2310136530 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.236+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22fc021030 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.236+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f22fc040900 con 0x7f23100a0a30 2023-09-30T22:27:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.236+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23100747f0 con 0x7f23100a0a30 2023-09-30T22:27:33.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.241+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22fc015070 con 0x7f23100a0a30 2023-09-30T22:27:33.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:33.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.414+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7c4c0a28e0 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.414+0000 7f7c497fa700 1 -- 172.21.15.32:0/3915656492 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c40013720 con 0x7f7c4c0a0a30 2023-09-30T22:27:33.415 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 msgr2=0x7f7c4c120100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 0x7f7c4c120100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000ec10 tx=0x7f7c4000ef20 comp rx=0 tx=0).stop 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 shutdown_connections 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c4c133fa0 0x7f7c4c120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c4c0a0a30 0x7f7c4c120100 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 --2- 172.21.15.32:0/3915656492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c4c0a1340 0x7f7c4c120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 >> 172.21.15.32:0/3915656492 conn(0x7f7c4c0a6550 msgr2=0x7f7c4c0a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 shutdown_connections 2023-09-30T22:27:33.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.417+0000 7f7c52d38700 1 -- 172.21.15.32:0/3915656492 wait complete. 2023-09-30T22:27:33.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.454+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2310072980 con 0x7f23100a0a30 2023-09-30T22:27:33.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.454+0000 7f230d7fa700 1 -- 172.21.15.32:0/2027416816 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22fc01a450 con 0x7f23100a0a30 2023-09-30T22:27:33.456 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 msgr2=0x7f2310133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 0x7f2310133c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00a010 tx=0x7f22fc004750 comp rx=0 tx=0).stop 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 shutdown_connections 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2310138d80 0x7f23101341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f231009d630 0x7f2310133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 --2- 172.21.15.32:0/2027416816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23100a0a30 0x7f2310133c70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 >> 172.21.15.32:0/2027416816 conn(0x7f23100a3050 msgr2=0x7f231009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 shutdown_connections 2023-09-30T22:27:33.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.457+0000 7f2316d9c700 1 -- 172.21.15.32:0/2027416816 wait complete. 2023-09-30T22:27:33.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:33.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.486+0000 7fe4d88ef700 1 Processor -- start 2023-09-30T22:27:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.486+0000 7fe4d88ef700 1 -- start start 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d88ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 0x7fe4d4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d88ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4d4135e50 0x7fe4d4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d88ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d409f130 con 0x7fe4d409d6f0 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d88ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d409f520 con 0x7fe4d4135e50 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d88ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d409e930 con 0x7fe4d409e1d0 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 0x7fe4d4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe4d409d6f0 0x7fe4d409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35014/0 (socket says 172.21.15.32:35014) 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d259c700 1 -- 172.21.15.32:0/2928255005 learned_addr learned my addr 172.21.15.32:0/2928255005 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:33.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 0x7fe4d4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40530/0 (socket says 172.21.15.32:40530) 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d2d9d700 1 --2- 172.21.15.32:0/2928255005 >> v2:172.21.15.32:3300/0 conn(0x7fe4d4135e50 0x7fe4d4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d1d9b700 1 -- 172.21.15.32:0/2928255005 >> v2:172.21.15.32:3300/0 conn(0x7fe4d4135e50 msgr2=0x7fe4d4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/2928255005 >> v2:172.21.15.32:3300/0 conn(0x7fe4d4135e50 0x7fe4d4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d1d9b700 1 -- 172.21.15.32:0/2928255005 >> v1:172.21.15.32:6789/0 conn(0x7fe4d409d6f0 legacy=0x7fe4d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d159a700 1 -- 172.21.15.32:0/2928255005 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455282498 0 0) 0x7fe4d409f130 con 0x7fe4d409d6f0 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d1d9b700 1 -- 172.21.15.32:0/2928255005 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d409b2d0 con 0x7fe4d409e1d0 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.487+0000 7fe4d2d9d700 1 --2- 172.21.15.32:0/2928255005 >> v2:172.21.15.32:3300/0 conn(0x7fe4d4135e50 0x7fe4d4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.488+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/2928255005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 0x7fe4d4135910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400bb10 tx=0x7fe4c400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70aaf6190d5baa0a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.488+0000 7fe4d159a700 1 -- 172.21.15.32:0/2928255005 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400dc30 con 0x7fe4d409e1d0 2023-09-30T22:27:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.488+0000 7fe4d159a700 1 -- 172.21.15.32:0/2928255005 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c400dd90 con 0x7fe4d409e1d0 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.488+0000 7fe4d159a700 1 -- 172.21.15.32:0/2928255005 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4010690 con 0x7fe4d409e1d0 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.488+0000 7fe4d88ef700 1 -- 172.21.15.32:0/2928255005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 msgr2=0x7fe4d4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.488+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/2928255005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 0x7fe4d4135910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400bb10 tx=0x7fe4c400be20 comp rx=0 tx=0).stop 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.489+0000 7fe4d88ef700 1 -- 172.21.15.32:0/2928255005 shutdown_connections 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.489+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/2928255005 >> v2:172.21.15.32:3300/0 conn(0x7fe4d4135e50 0x7fe4d4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.489+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/2928255005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e1d0 0x7fe4d4135910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.489+0000 7fe4d88ef700 1 -- 172.21.15.32:0/2928255005 >> 172.21.15.32:0/2928255005 conn(0x7fe4d4066f50 msgr2=0x7fe4d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.489+0000 7fe4d88ef700 1 -- 172.21.15.32:0/2928255005 shutdown_connections 2023-09-30T22:27:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.489+0000 7fe4d88ef700 1 -- 172.21.15.32:0/2928255005 wait complete. 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.490+0000 7fe4d88ef700 1 Processor -- start 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.490+0000 7fe4d88ef700 1 -- start start 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d88ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 0x7fe4d411b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d88ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e1d0 0x7fe4d411b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d88ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d4135e50 0x7fe4d411bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d88ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d411d520 con 0x7fe4d409e1d0 2023-09-30T22:27:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d88ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d411ee30 con 0x7fe4d409d6f0 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d88ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4072630 con 0x7fe4d4135e50 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 0x7fe4d411b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e1d0 0x7fe4d411b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 0x7fe4d411b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52752/0 (socket says 172.21.15.32:52752) 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d259c700 1 -- 172.21.15.32:0/689640363 learned_addr learned my addr 172.21.15.32:0/689640363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.491+0000 7fe4d2d9d700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d4135e50 0x7fe4d411bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d259c700 1 -- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d4135e50 msgr2=0x7fe4d411bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d259c700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d4135e50 0x7fe4d411bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d259c700 1 -- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e1d0 msgr2=0x7fe4d411b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d259c700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e1d0 0x7fe4d411b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d259c700 1 -- 172.21.15.32:0/689640363 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009a70 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d1d9b700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e1d0 0x7fe4d411b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d2d9d700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d4135e50 0x7fe4d411bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d259c700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 0x7fe4d411b310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c550 tx=0x7fe4c800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004c80 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8004e00 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c400b740 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8007310 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.492+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d411e890 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.493+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe4c8007470 con 0x7fe4d409d6f0 2023-09-30T22:27:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.493+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe4c803f300 con 0x7fe4d409d6f0 2023-09-30T22:27:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.494+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d411e200 con 0x7fe4d409d6f0 2023-09-30T22:27:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.499+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe4c80123b0 con 0x7fe4d409d6f0 2023-09-30T22:27:33.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.623+0000 7f416a972700 1 Processor -- start 2023-09-30T22:27:33.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.623+0000 7f416a972700 1 -- start start 2023-09-30T22:27:33.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.623+0000 7f416a972700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41640a1510 0x7f4164138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f416a972700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4164139250 0x7f416413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f416a972700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416409edd0 con 0x7f41640a0a30 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f416a972700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41640a18e0 con 0x7f41640a1510 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f416a972700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416409e1f0 con 0x7f4164139250 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f41637fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41640a1510 0x7f4164138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4168f0f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4164139250 0x7f416413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4163fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41640a0a30 0x7f41640a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35022/0 (socket says 172.21.15.32:35022) 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4163fff700 1 -- 172.21.15.32:0/748935712 learned_addr learned my addr 172.21.15.32:0/748935712 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:33.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f41637fe700 1 -- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4164139250 msgr2=0x7f416413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:33.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f41637fe700 1 --2- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4164139250 0x7f416413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f41637fe700 1 -- 172.21.15.32:0/748935712 >> v1:172.21.15.32:6789/0 conn(0x7f41640a0a30 legacy=0x7f41640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f41637fe700 1 -- 172.21.15.32:0/748935712 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416409dc50 con 0x7f41640a1510 2023-09-30T22:27:33.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4162ffd700 1 -- 172.21.15.32:0/748935712 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072283790 0 0) 0x7f416409edd0 con 0x7f41640a0a30 2023-09-30T22:27:33.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4168f0f700 1 --2- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4164139250 0x7f416413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f41637fe700 1 --2- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164138d10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4158009840 tx=0x7f4158009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b03c67dbdc7321 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4162ffd700 1 -- 172.21.15.32:0/748935712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158011830 con 0x7f41640a1510 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.624+0000 7f4162ffd700 1 -- 172.21.15.32:0/748935712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4158011990 con 0x7f41640a1510 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f4162ffd700 1 -- 172.21.15.32:0/748935712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158011d20 con 0x7f41640a1510 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 -- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 msgr2=0x7f4164138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 --2- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164138d10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4158009840 tx=0x7f4158009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 -- 172.21.15.32:0/748935712 shutdown_connections 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 --2- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4164139250 0x7f416413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 --2- 172.21.15.32:0/748935712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164138d10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 -- 172.21.15.32:0/748935712 >> 172.21.15.32:0/748935712 conn(0x7f41640a6550 msgr2=0x7f41640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 -- 172.21.15.32:0/748935712 shutdown_connections 2023-09-30T22:27:33.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.625+0000 7f416a972700 1 -- 172.21.15.32:0/748935712 wait complete. 2023-09-30T22:27:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.626+0000 7f416a972700 1 Processor -- start 2023-09-30T22:27:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 -- start start 2023-09-30T22:27:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 0x7f41641228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4164139250 0x7f4164123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:33.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641389e0 con 0x7f41640a1510 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41640a0220 con 0x7f41640a0a30 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.627+0000 7f416a972700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4164126060 con 0x7f4164139250 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 0x7f41641228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4168f0f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4164139250 0x7f4164123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 0x7f41641228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52768/0 (socket says 172.21.15.32:52768) 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 -- 172.21.15.32:0/458533486 learned_addr learned my addr 172.21.15.32:0/458533486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f41637fe700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 -- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4164139250 msgr2=0x7f4164123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4164139250 0x7f4164123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 -- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 msgr2=0x7f4164122e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164122e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 -- 172.21.15.32:0/458533486 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c009a70 con 0x7f41640a0a30 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4168f0f700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4164139250 0x7f4164123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f41637fe700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4163fff700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 0x7f41641228c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f415400c520 tx=0x7f415400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154004c80 con 0x7f41640a0a30 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154004e00 con 0x7f41640a0a30 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41580094a0 con 0x7f41640a0a30 2023-09-30T22:27:33.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.628+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154007310 con 0x7f41640a0a30 2023-09-30T22:27:33.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.629+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4164124bb0 con 0x7f41640a0a30 2023-09-30T22:27:33.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.629+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4164074ea0 con 0x7f41640a0a30 2023-09-30T22:27:33.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.629+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4154007470 con 0x7f41640a0a30 2023-09-30T22:27:33.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.630+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f415403f300 con 0x7f41640a0a30 2023-09-30T22:27:33.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.634+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41540133c0 con 0x7f41640a0a30 2023-09-30T22:27:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.709+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe4d411e680 con 0x7fe4d409d6f0 2023-09-30T22:27:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.710+0000 7fe4c2ffd700 1 -- 172.21.15.32:0/689640363 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4c8011360 con 0x7fe4d409d6f0 2023-09-30T22:27:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 msgr2=0x7fe4d411b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 0x7fe4d411b310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c550 tx=0x7fe4c800c860 comp rx=0 tx=0).stop 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 shutdown_connections 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d4135e50 0x7fe4d411bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d409d6f0 0x7fe4d411b310 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 --2- 172.21.15.32:0/689640363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e1d0 0x7fe4d411b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 >> 172.21.15.32:0/689640363 conn(0x7fe4d4066f50 msgr2=0x7fe4d40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 shutdown_connections 2023-09-30T22:27:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.712+0000 7fe4d88ef700 1 -- 172.21.15.32:0/689640363 wait complete. 2023-09-30T22:27:33.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.838+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f41640a28e0 con 0x7f41640a0a30 2023-09-30T22:27:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.838+0000 7f4160ff9700 1 -- 172.21.15.32:0/458533486 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f415400ce60 con 0x7f41640a0a30 2023-09-30T22:27:33.840 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 msgr2=0x7f41641228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 0x7f41641228c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f415400c520 tx=0x7f415400c830 comp rx=0 tx=0).stop 2023-09-30T22:27:33.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 shutdown_connections 2023-09-30T22:27:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4164139250 0x7f4164123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41640a0a30 0x7f41641228c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 --2- 172.21.15.32:0/458533486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41640a1510 0x7f4164122e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 >> 172.21.15.32:0/458533486 conn(0x7f41640a6550 msgr2=0x7f41640a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 shutdown_connections 2023-09-30T22:27:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:33.840+0000 7f416a972700 1 -- 172.21.15.32:0/458533486 wait complete. 2023-09-30T22:27:33.855 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:34.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:34.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:34.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.581+0000 7faffe80d700 1 Processor -- start 2023-09-30T22:27:34.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.581+0000 7faffe80d700 1 -- start start 2023-09-30T22:27:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffe80d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff80a1510 0x7faff8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffe80d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff8139250 0x7faff813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffe80d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff809edd0 con 0x7faff80a0a30 2023-09-30T22:27:34.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffe80d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff80a18e0 con 0x7faff80a1510 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffe80d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff809e1f0 con 0x7faff8139250 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faff80a1510 0x7faff8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffcdaa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff8139250 0x7faff813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faff80a0a30 0x7faff80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35024/0 (socket says 172.21.15.32:35024) 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff7fff700 1 -- 172.21.15.32:0/1604377361 learned_addr learned my addr 172.21.15.32:0/1604377361 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff77fe700 1 -- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff8139250 msgr2=0x7faff813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff77fe700 1 --2- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff8139250 0x7faff813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff77fe700 1 -- 172.21.15.32:0/1604377361 >> v1:172.21.15.32:6789/0 conn(0x7faff80a0a30 legacy=0x7faff80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff77fe700 1 -- 172.21.15.32:0/1604377361 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff809dc50 con 0x7faff80a1510 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff6ffd700 1 -- 172.21.15.32:0/1604377361 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289061464 0 0) 0x7faff809edd0 con 0x7faff80a0a30 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faffcdaa700 1 --2- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff8139250 0x7faff813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff77fe700 1 --2- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80a1510 0x7faff8138d10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fafec009840 tx=0x7fafec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3523e054b853129 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.582+0000 7faff6ffd700 1 -- 172.21.15.32:0/1604377361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec011830 con 0x7faff80a1510 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faff6ffd700 1 -- 172.21.15.32:0/1604377361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec011990 con 0x7faff80a1510 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faff6ffd700 1 -- 172.21.15.32:0/1604377361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec011d20 con 0x7faff80a1510 2023-09-30T22:27:34.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 -- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80a1510 msgr2=0x7faff8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 --2- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80a1510 0x7faff8138d10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fafec009840 tx=0x7fafec009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 -- 172.21.15.32:0/1604377361 shutdown_connections 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 --2- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff8139250 0x7faff813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 --2- 172.21.15.32:0/1604377361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff80a1510 0x7faff8138d10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 -- 172.21.15.32:0/1604377361 >> 172.21.15.32:0/1604377361 conn(0x7faff80a6550 msgr2=0x7faff80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 -- 172.21.15.32:0/1604377361 shutdown_connections 2023-09-30T22:27:34.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.583+0000 7faffe80d700 1 -- 172.21.15.32:0/1604377361 wait complete. 2023-09-30T22:27:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.584+0000 7faffe80d700 1 Processor -- start 2023-09-30T22:27:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 -- start start 2023-09-30T22:27:34.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff80a0a30 0x7faff81228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 0x7faff8122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 0x7faff8123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff81389e0 con 0x7faff8139250 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80a0220 con 0x7faff80a0a30 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffe80d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faff8126060 con 0x7faff80a1510 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffcdaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 0x7faff8123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faff77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 0x7faff8122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faff77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 0x7faff8122e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40568/0 (socket says 172.21.15.32:40568) 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faffcdaa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 0x7faff8123340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39250/0 (socket says 172.21.15.32:39250) 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faff77fe700 1 -- 172.21.15.32:0/1007521148 learned_addr learned my addr 172.21.15.32:0/1007521148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.585+0000 7faff7fff700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff80a0a30 0x7faff81228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff77fe700 1 -- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff80a0a30 msgr2=0x7faff81228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff77fe700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff80a0a30 0x7faff81228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff77fe700 1 -- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 msgr2=0x7faff8123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff77fe700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 0x7faff8123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff77fe700 1 -- 172.21.15.32:0/1007521148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff0009a70 con 0x7faff80a1510 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff7fff700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff80a0a30 0x7faff81228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faffcdaa700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 0x7faff8123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff77fe700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 0x7faff8122e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fafec009fd0 tx=0x7fafec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0045d0 con 0x7faff80a1510 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec01f070 con 0x7faff80a1510 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0038a0 con 0x7faff80a1510 2023-09-30T22:27:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec0094a0 con 0x7faff80a1510 2023-09-30T22:27:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.586+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff8124bb0 con 0x7faff80a1510 2023-09-30T22:27:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.587+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fafec003a00 con 0x7faff80a1510 2023-09-30T22:27:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.587+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fafec021070 con 0x7faff80a1510 2023-09-30T22:27:34.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.587+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff8074ea0 con 0x7faff80a1510 2023-09-30T22:27:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.592+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafec015080 con 0x7faff80a1510 2023-09-30T22:27:34.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 Processor -- start 2023-09-30T22:27:34.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 -- start start 2023-09-30T22:27:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1700a1510 0x7fa170138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 0x7fa17013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17009edd0 con 0x7fa1700a0a30 2023-09-30T22:27:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1700a18e0 con 0x7fa1700a1510 2023-09-30T22:27:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa176c93700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa17009e1f0 con 0x7fa170139250 2023-09-30T22:27:34.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa167fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1700a1510 0x7fa170138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa175230700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 0x7fa17013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa174a2f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1700a0a30 0x7fa1700a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35038/0 (socket says 172.21.15.32:35038) 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa167fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1700a1510 0x7fa170138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39258/0 (socket says 172.21.15.32:39258) 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa175230700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 0x7fa17013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40574/0 (socket says 172.21.15.32:40574) 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.620+0000 7fa174a2f700 1 -- 172.21.15.32:0/576954775 learned_addr learned my addr 172.21.15.32:0/576954775 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa167fff700 1 -- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 msgr2=0x7fa17013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa167fff700 1 --2- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 0x7fa17013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa167fff700 1 -- 172.21.15.32:0/576954775 >> v1:172.21.15.32:6789/0 conn(0x7fa1700a0a30 legacy=0x7fa1700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa167fff700 1 -- 172.21.15.32:0/576954775 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17009dc50 con 0x7fa1700a1510 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa175230700 1 --2- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 0x7fa17013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa1677fe700 1 -- 172.21.15.32:0/576954775 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489614105 0 0) 0x7fa17009edd0 con 0x7fa1700a0a30 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa167fff700 1 --2- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1700a1510 0x7fa170138d10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa160009840 tx=0x7fa160009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98b45cd66fd216a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa1677fe700 1 -- 172.21.15.32:0/576954775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160011830 con 0x7fa1700a1510 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa1677fe700 1 -- 172.21.15.32:0/576954775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160011990 con 0x7fa1700a1510 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.621+0000 7fa1677fe700 1 -- 172.21.15.32:0/576954775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160011d20 con 0x7fa1700a1510 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 -- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1700a1510 msgr2=0x7fa170138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 --2- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1700a1510 0x7fa170138d10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa160009840 tx=0x7fa160009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 -- 172.21.15.32:0/576954775 shutdown_connections 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 --2- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa170139250 0x7fa17013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 --2- 172.21.15.32:0/576954775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1700a1510 0x7fa170138d10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 -- 172.21.15.32:0/576954775 >> 172.21.15.32:0/576954775 conn(0x7fa1700a6550 msgr2=0x7fa1700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:34.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 -- 172.21.15.32:0/576954775 shutdown_connections 2023-09-30T22:27:34.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.622+0000 7fa176c93700 1 -- 172.21.15.32:0/576954775 wait complete. 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.623+0000 7fa176c93700 1 Processor -- start 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.623+0000 7fa176c93700 1 -- start start 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa176c93700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1700a0a30 0x7fa1701228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa176c93700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1700a1510 0x7fa170122e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa176c93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 0x7fa170123340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa176c93700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1701389e0 con 0x7fa170139250 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa176c93700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700a0220 con 0x7fa1700a0a30 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa176c93700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa170126060 con 0x7fa1700a1510 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa175230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 0x7fa170123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa175230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 0x7fa170123340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39274/0 (socket says 172.21.15.32:39274) 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa175230700 1 -- 172.21.15.32:0/1933234757 learned_addr learned my addr 172.21.15.32:0/1933234757 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1700a1510 0x7fa170122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa174a2f700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1700a0a30 0x7fa1701228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 -- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1700a0a30 msgr2=0x7fa1701228c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1700a0a30 0x7fa1701228c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 -- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 msgr2=0x7fa170123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 0x7fa170123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 -- 172.21.15.32:0/1933234757 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009a70 con 0x7fa1700a1510 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa175230700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 0x7fa170123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.624+0000 7fa167fff700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1700a1510 0x7fa170122e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa160016000 tx=0x7fa160004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.625+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600036a0 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.625+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16001f070 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.625+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1600094a0 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.625+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160003a20 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.625+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa170124bb0 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.625+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa160003b80 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.626+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa160021070 con 0x7fa1700a1510 2023-09-30T22:27:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.626+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa170074ea0 con 0x7fa1700a1510 2023-09-30T22:27:34.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.631+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa16002b020 con 0x7fa1700a1510 2023-09-30T22:27:34.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:34.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.801+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faff80a28e0 con 0x7faff80a1510 2023-09-30T22:27:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.801+0000 7faff4ff9700 1 -- 172.21.15.32:0/1007521148 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafec02b020 con 0x7faff80a1510 2023-09-30T22:27:34.802 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 msgr2=0x7faff8122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 0x7faff8122e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fafec009fd0 tx=0x7fafec004750 comp rx=0 tx=0).stop 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 shutdown_connections 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faff8139250 0x7faff8123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faff80a0a30 0x7faff81228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 --2- 172.21.15.32:0/1007521148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faff80a1510 0x7faff8122e00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 >> 172.21.15.32:0/1007521148 conn(0x7faff80a6550 msgr2=0x7faff80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 shutdown_connections 2023-09-30T22:27:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.803+0000 7faffe80d700 1 -- 172.21.15.32:0/1007521148 wait complete. 2023-09-30T22:27:34.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:34.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.837+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa1700a28e0 con 0x7fa1700a1510 2023-09-30T22:27:34.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.837+0000 7fa1657fa700 1 -- 172.21.15.32:0/1933234757 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa160015070 con 0x7fa1700a1510 2023-09-30T22:27:34.838 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:34.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1700a1510 msgr2=0x7fa170122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1700a1510 0x7fa170122e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa160016000 tx=0x7fa160004750 comp rx=0 tx=0).stop 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 shutdown_connections 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa170139250 0x7fa170123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1700a0a30 0x7fa1701228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 --2- 172.21.15.32:0/1933234757 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1700a1510 0x7fa170122e00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 >> 172.21.15.32:0/1933234757 conn(0x7fa1700a6550 msgr2=0x7fa1700a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 shutdown_connections 2023-09-30T22:27:34.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.839+0000 7fa176c93700 1 -- 172.21.15.32:0/1933234757 wait complete. 2023-09-30T22:27:34.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:34.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.866+0000 7fb35a9c0700 1 Processor -- start 2023-09-30T22:27:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.866+0000 7fb35a9c0700 1 -- start start 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb35a9c0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3540a0a30 0x7fb3540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb35a9c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb35409d630 0x7fb3540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb35a9c0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3540a2a70 con 0x7fb354133930 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb35a9c0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3540a01d0 con 0x7fb3540a0a30 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb35a9c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35409e2b0 con 0x7fb35409d630 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb3537fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb35409d630 0x7fb3540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb358f5d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb354133930 0x7fb354133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35048/0 (socket says 172.21.15.32:35048) 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb358f5d700 1 -- 172.21.15.32:0/795868903 learned_addr learned my addr 172.21.15.32:0/795868903 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:34.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb353fff700 1 --2- 172.21.15.32:0/795868903 >> v2:172.21.15.32:3300/0 conn(0x7fb3540a0a30 0x7fb3540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb3537fe700 1 -- 172.21.15.32:0/795868903 >> v2:172.21.15.32:3300/0 conn(0x7fb3540a0a30 msgr2=0x7fb3540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb3537fe700 1 --2- 172.21.15.32:0/795868903 >> v2:172.21.15.32:3300/0 conn(0x7fb3540a0a30 0x7fb3540a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb3537fe700 1 -- 172.21.15.32:0/795868903 >> v1:172.21.15.32:6789/0 conn(0x7fb354133930 legacy=0x7fb354133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb352ffd700 1 -- 172.21.15.32:0/795868903 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014545269 0 0) 0x7fb3540a2a70 con 0x7fb354133930 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb3537fe700 1 -- 172.21.15.32:0/795868903 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35409dc50 con 0x7fb35409d630 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.867+0000 7fb353fff700 1 --2- 172.21.15.32:0/795868903 >> v2:172.21.15.32:3300/0 conn(0x7fb3540a0a30 0x7fb3540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb3537fe700 1 --2- 172.21.15.32:0/795868903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb35409d630 0x7fb3540a1340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009840 tx=0x7fb33c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e8b21e0068f13f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb352ffd700 1 -- 172.21.15.32:0/795868903 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c011830 con 0x7fb35409d630 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb352ffd700 1 -- 172.21.15.32:0/795868903 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c011990 con 0x7fb35409d630 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb352ffd700 1 -- 172.21.15.32:0/795868903 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c011d20 con 0x7fb35409d630 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb35a9c0700 1 -- 172.21.15.32:0/795868903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb35409d630 msgr2=0x7fb3540a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/795868903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb35409d630 0x7fb3540a1340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009840 tx=0x7fb33c009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb35a9c0700 1 -- 172.21.15.32:0/795868903 shutdown_connections 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/795868903 >> v2:172.21.15.32:3300/0 conn(0x7fb3540a0a30 0x7fb3540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/795868903 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb35409d630 0x7fb3540a1340 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.868+0000 7fb35a9c0700 1 -- 172.21.15.32:0/795868903 >> 172.21.15.32:0/795868903 conn(0x7fb3540a3050 msgr2=0x7fb3540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.869+0000 7fb35a9c0700 1 -- 172.21.15.32:0/795868903 shutdown_connections 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.869+0000 7fb35a9c0700 1 -- 172.21.15.32:0/795868903 wait complete. 2023-09-30T22:27:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.870+0000 7fb35a9c0700 1 Processor -- start 2023-09-30T22:27:34.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.870+0000 7fb35a9c0700 1 -- start start 2023-09-30T22:27:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb35a9c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35409d630 0x7fb3541c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb35a9c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3540a0a30 0x7fb3541c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb35a9c0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb354133930 0x7fb3541c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb35a9c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541221b0 con 0x7fb35409d630 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb35a9c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb354133520 con 0x7fb3540a0a30 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb35a9c0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb354120700 con 0x7fb354133930 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb3537fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3540a0a30 0x7fb3541c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb358f5d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb354133930 0x7fb3541c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb3537fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3540a0a30 0x7fb3541c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52808/0 (socket says 172.21.15.32:52808) 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb3537fe700 1 -- 172.21.15.32:0/3311941229 learned_addr learned my addr 172.21.15.32:0/3311941229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.871+0000 7fb353fff700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35409d630 0x7fb3541c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb358f5d700 1 -- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3540a0a30 msgr2=0x7fb3541c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb358f5d700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3540a0a30 0x7fb3541c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb358f5d700 1 -- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35409d630 msgr2=0x7fb3541c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb358f5d700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35409d630 0x7fb3541c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb358f5d700 1 -- 172.21.15.32:0/3311941229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440094a0 con 0x7fb354133930 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb353fff700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35409d630 0x7fb3541c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb358f5d700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb354133930 0x7fb3541c8a00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00cbe0 tx=0x7fb34c00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c00aa40 con 0x7fb354133930 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c00abc0 con 0x7fb354133930 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c0094a0 con 0x7fb354133930 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0070d0 con 0x7fb354133930 2023-09-30T22:27:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.872+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb354122c20 con 0x7fb354133930 2023-09-30T22:27:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.873+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb34c007230 con 0x7fb354133930 2023-09-30T22:27:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.873+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb34c03f390 con 0x7fb354133930 2023-09-30T22:27:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.873+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3540747f0 con 0x7fb354133930 2023-09-30T22:27:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:34.878+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb34c018070 con 0x7fb354133930 2023-09-30T22:27:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.004+0000 7f6e568dd700 1 Processor -- start 2023-09-30T22:27:35.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.004+0000 7f6e568dd700 1 -- start start 2023-09-30T22:27:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e568dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 0x7f6e5009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e568dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009f240 0x7f6e501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e568dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e50131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e568dd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e50070940 con 0x7f6e5009e930 2023-09-30T22:27:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e568dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5006fd60 con 0x7f6e5009f650 2023-09-30T22:27:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e568dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5006fa80 con 0x7f6e5009f240 2023-09-30T22:27:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e54e7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e50131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 0x7f6e5009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009f240 0x7f6e501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 0x7f6e5009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39296/0 (socket says 172.21.15.32:39296) 2023-09-30T22:27:35.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e54e7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e50131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52810/0 (socket says 172.21.15.32:52810) 2023-09-30T22:27:35.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4ffff700 1 -- 172.21.15.32:0/3232081879 learned_addr learned my addr 172.21.15.32:0/3232081879 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:35.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4f7fe700 1 -- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 msgr2=0x7f6e50131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4f7fe700 1 --2- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e50131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4f7fe700 1 -- 172.21.15.32:0/3232081879 >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 msgr2=0x7f6e5009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4f7fe700 1 --2- 172.21.15.32:0/3232081879 >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 0x7f6e5009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4f7fe700 1 -- 172.21.15.32:0/3232081879 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5009fb60 con 0x7f6e5009f240 2023-09-30T22:27:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e4ffff700 1 --2- 172.21.15.32:0/3232081879 >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 0x7f6e5009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.005+0000 7f6e54e7a700 1 --2- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e50131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.006+0000 7f6e4f7fe700 1 --2- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009f240 0x7f6e501317f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009840 tx=0x7f6e44009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b64981d650389c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.006+0000 7f6e4effd700 1 -- 172.21.15.32:0/3232081879 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44011830 con 0x7f6e5009f240 2023-09-30T22:27:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.006+0000 7f6e4effd700 1 -- 172.21.15.32:0/3232081879 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44011990 con 0x7f6e5009f240 2023-09-30T22:27:35.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.006+0000 7f6e4effd700 1 -- 172.21.15.32:0/3232081879 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44011d20 con 0x7f6e5009f240 2023-09-30T22:27:35.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.006+0000 7f6e568dd700 1 -- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009f240 msgr2=0x7f6e501317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.006+0000 7f6e568dd700 1 --2- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009f240 0x7f6e501317f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009840 tx=0x7f6e44009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:35.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 -- 172.21.15.32:0/3232081879 shutdown_connections 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 --2- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e50131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 --2- 172.21.15.32:0/3232081879 >> v2:172.21.15.32:3300/0 conn(0x7f6e5009e930 0x7f6e5009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 --2- 172.21.15.32:0/3232081879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009f240 0x7f6e501317f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 -- 172.21.15.32:0/3232081879 >> 172.21.15.32:0/3232081879 conn(0x7f6e500a4210 msgr2=0x7f6e500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 -- 172.21.15.32:0/3232081879 shutdown_connections 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.007+0000 7f6e568dd700 1 -- 172.21.15.32:0/3232081879 wait complete. 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.008+0000 7f6e568dd700 1 Processor -- start 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.008+0000 7f6e568dd700 1 -- start start 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e568dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 0x7f6e501c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e568dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5009f240 0x7f6e501c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e568dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e501c0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e568dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e501a6880 con 0x7f6e5009f240 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e568dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5009b220 con 0x7f6e5009f650 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e568dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e50071d20 con 0x7f6e5009e930 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e501c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e4ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 0x7f6e501c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e501c0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52820/0 (socket says 172.21.15.32:52820) 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e4ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 0x7f6e501c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40620/0 (socket says 172.21.15.32:40620) 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 -- 172.21.15.32:0/67243631 learned_addr learned my addr 172.21.15.32:0/67243631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e4f7fe700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5009f240 0x7f6e501c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 -- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 msgr2=0x7f6e501c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 0x7f6e501c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 -- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5009f240 msgr2=0x7f6e501c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5009f240 0x7f6e501c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e54e7a700 1 -- 172.21.15.32:0/67243631 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40009550 con 0x7f6e5009f650 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.009+0000 7f6e4ffff700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 0x7f6e501c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.010+0000 7f6e54e7a700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e501c0840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800f500 tx=0x7f6e4800f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.010+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48012050 con 0x7f6e5009f650 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.010+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e48013040 con 0x7f6e5009f650 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.010+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4800f050 con 0x7f6e5009f650 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.010+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48003e60 con 0x7f6e5009f650 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.010+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440094a0 con 0x7f6e5009f650 2023-09-30T22:27:35.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.011+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e48004030 con 0x7f6e5009f650 2023-09-30T22:27:35.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.011+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e501a6120 con 0x7f6e5009f650 2023-09-30T22:27:35.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.011+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6e48044080 con 0x7f6e5009f650 2023-09-30T22:27:35.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.016+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e48075750 con 0x7f6e5009f650 2023-09-30T22:27:35.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.088+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb354072980 con 0x7fb354133930 2023-09-30T22:27:35.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.088+0000 7fb350ff9700 1 -- 172.21.15.32:0/3311941229 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb34c0074e0 con 0x7fb354133930 2023-09-30T22:27:35.089 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb354133930 msgr2=0x7fb3541c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb354133930 0x7fb3541c8a00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00cbe0 tx=0x7fb34c00cef0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 shutdown_connections 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb354133930 0x7fb3541c8a00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb35409d630 0x7fb3541c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 --2- 172.21.15.32:0/3311941229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3540a0a30 0x7fb3541c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 >> 172.21.15.32:0/3311941229 conn(0x7fb3540a3050 msgr2=0x7fb3541bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:35.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 shutdown_connections 2023-09-30T22:27:35.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.091+0000 7fb35a9c0700 1 -- 172.21.15.32:0/3311941229 wait complete. 2023-09-30T22:27:35.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.226+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6e50074ea0 con 0x7f6e5009f650 2023-09-30T22:27:35.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.226+0000 7f6e4cff9700 1 -- 172.21.15.32:0/67243631 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e501a6120 con 0x7f6e5009f650 2023-09-30T22:27:35.228 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 msgr2=0x7f6e501c0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e501c0840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800f500 tx=0x7f6e4800f810 comp rx=0 tx=0).stop 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 shutdown_connections 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6e5009f650 0x7f6e501c0840 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6e5009e930 0x7f6e501c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 --2- 172.21.15.32:0/67243631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e5009f240 0x7f6e501c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.228+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 >> 172.21.15.32:0/67243631 conn(0x7f6e500a4210 msgr2=0x7f6e5006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.229+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 shutdown_connections 2023-09-30T22:27:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.229+0000 7f6e568dd700 1 -- 172.21.15.32:0/67243631 wait complete. 2023-09-30T22:27:35.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:35.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:35.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 Processor -- start 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 -- start start 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4b580a0a30 0x7f4b580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5809edd0 con 0x7f4b580a0a30 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5809e1f0 con 0x7f4b58133fa0 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.966+0000 7f4b5d0ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5809dc50 con 0x7f4b580a1340 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40626/0 (socket says 172.21.15.32:40626) 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 -- 172.21.15.32:0/4213896882 learned_addr learned my addr 172.21.15.32:0/4213896882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:35.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b56d9d700 1 --2- 172.21.15.32:0/4213896882 >> v2:172.21.15.32:3300/0 conn(0x7f4b580a0a30 0x7f4b580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5759e700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 -- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 msgr2=0x7f4b58134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 -- 172.21.15.32:0/4213896882 >> v2:172.21.15.32:3300/0 conn(0x7f4b580a0a30 msgr2=0x7f4b580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 --2- 172.21.15.32:0/4213896882 >> v2:172.21.15.32:3300/0 conn(0x7f4b580a0a30 0x7f4b580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 -- 172.21.15.32:0/4213896882 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580a1fe0 con 0x7f4b580a1340 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5759e700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b5659c700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58133a60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009920 tx=0x7f4b4c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb15d97622b5e7e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b55d9b700 1 -- 172.21.15.32:0/4213896882 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c011830 con 0x7f4b580a1340 2023-09-30T22:27:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.967+0000 7f4b55d9b700 1 -- 172.21.15.32:0/4213896882 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c011990 con 0x7f4b580a1340 2023-09-30T22:27:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b55d9b700 1 -- 172.21.15.32:0/4213896882 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c011d20 con 0x7f4b580a1340 2023-09-30T22:27:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 msgr2=0x7f4b58133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58133a60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009920 tx=0x7f4b4c009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/4213896882 shutdown_connections 2023-09-30T22:27:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/4213896882 >> v2:172.21.15.32:3300/0 conn(0x7f4b580a0a30 0x7f4b580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/4213896882 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58133a60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/4213896882 >> 172.21.15.32:0/4213896882 conn(0x7f4b580a6550 msgr2=0x7f4b580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/4213896882 shutdown_connections 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.968+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/4213896882 wait complete. 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.970+0000 7f4b5d0ec700 1 Processor -- start 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.970+0000 7f4b5d0ec700 1 -- start start 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5d0ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b580a0a30 0x7f4b58120110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5d0ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58120650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5d0ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58120b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5d0ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581234a0 con 0x7f4b580a0a30 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5d0ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581245f0 con 0x7f4b58133fa0 2023-09-30T22:27:35.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5d0ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b58121e40 con 0x7f4b580a1340 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58120b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58120650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58120b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52828/0 (socket says 172.21.15.32:52828) 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b56d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b580a0a30 0x7f4b58120110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5759e700 1 -- 172.21.15.32:0/1903048653 learned_addr learned my addr 172.21.15.32:0/1903048653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5659c700 1 -- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 msgr2=0x7f4b58120b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5659c700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58120b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5659c700 1 -- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b580a0a30 msgr2=0x7f4b58120110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5659c700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b580a0a30 0x7f4b58120110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5659c700 1 -- 172.21.15.32:0/1903048653 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500094a0 con 0x7f4b580a1340 2023-09-30T22:27:35.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b5759e700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58120b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.971+0000 7f4b56d9d700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b580a0a30 0x7f4b58120110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.972+0000 7f4b5659c700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58120650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4b5806c3a0 tx=0x7f4b4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.972+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004450 con 0x7f4b580a1340 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.972+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0045b0 con 0x7f4b580a1340 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.972+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c009550 con 0x7f4b580a1340 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.972+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0038e0 con 0x7f4b580a1340 2023-09-30T22:27:35.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.972+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b58122fc0 con 0x7f4b580a1340 2023-09-30T22:27:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.973+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b4c003a40 con 0x7f4b580a1340 2023-09-30T22:27:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.973+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b4c021030 con 0x7f4b580a1340 2023-09-30T22:27:35.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.973+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b580a0280 con 0x7f4b580a1340 2023-09-30T22:27:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:35.978+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b4c02b020 con 0x7f4b580a1340 2023-09-30T22:27:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.005+0000 7f162669f700 1 Processor -- start 2023-09-30T22:27:36.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.005+0000 7f162669f700 1 -- start start 2023-09-30T22:27:36.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f162669f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 0x7f162009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f162669f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 0x7f1620131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f162669f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1620070940 con 0x7f162009f240 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f162669f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162006fd60 con 0x7f162009f650 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f162669f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f162006fa80 con 0x7f162009e930 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 0x7f162009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f1624c3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 0x7f1620131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f1624c3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 0x7f1620131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39334/0 (socket says 172.21.15.32:39334) 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 0x7f162009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40652/0 (socket says 172.21.15.32:40652) 2023-09-30T22:27:36.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f1624c3c700 1 -- 172.21.15.32:0/1246598009 learned_addr learned my addr 172.21.15.32:0/1246598009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f162009f240 0x7f16201317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35062/0 (socket says 172.21.15.32:35062) 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161ffff700 1 -- 172.21.15.32:0/1246598009 >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 msgr2=0x7f1620131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161ffff700 1 --2- 172.21.15.32:0/1246598009 >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 0x7f1620131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161ffff700 1 -- 172.21.15.32:0/1246598009 >> v1:172.21.15.32:6789/0 conn(0x7f162009f240 legacy=0x7f16201317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161effd700 1 -- 172.21.15.32:0/1246598009 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628989982 0 0) 0x7f1620070940 con 0x7f162009f240 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f161ffff700 1 -- 172.21.15.32:0/1246598009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162009fb60 con 0x7f162009e930 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.006+0000 7f1624c3c700 1 --2- 172.21.15.32:0/1246598009 >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 0x7f1620131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f161ffff700 1 --2- 172.21.15.32:0/1246598009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 0x7f162009ed00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f161000dc60 tx=0x7f161000df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f853e61cb7052c3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f161effd700 1 -- 172.21.15.32:0/1246598009 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610012070 con 0x7f162009e930 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f161effd700 1 -- 172.21.15.32:0/1246598009 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161000d2e0 con 0x7f162009e930 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f161effd700 1 -- 172.21.15.32:0/1246598009 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000f900 con 0x7f162009e930 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 -- 172.21.15.32:0/1246598009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 msgr2=0x7f162009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 --2- 172.21.15.32:0/1246598009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 0x7f162009ed00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f161000dc60 tx=0x7f161000df70 comp rx=0 tx=0).stop 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 -- 172.21.15.32:0/1246598009 shutdown_connections 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 --2- 172.21.15.32:0/1246598009 >> v2:172.21.15.32:3300/0 conn(0x7f162009f650 0x7f1620131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 --2- 172.21.15.32:0/1246598009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009e930 0x7f162009ed00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 -- 172.21.15.32:0/1246598009 >> 172.21.15.32:0/1246598009 conn(0x7f16200a4210 msgr2=0x7f16200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 -- 172.21.15.32:0/1246598009 shutdown_connections 2023-09-30T22:27:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.007+0000 7f162669f700 1 -- 172.21.15.32:0/1246598009 wait complete. 2023-09-30T22:27:36.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.009+0000 7f162669f700 1 Processor -- start 2023-09-30T22:27:36.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.009+0000 7f162669f700 1 -- start start 2023-09-30T22:27:36.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f162669f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 0x7f16201a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f162669f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162009f240 0x7f16201a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f162669f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009f650 0x7f16201c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f162669f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16201a1b50 con 0x7f162009e930 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f162669f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16201a0b70 con 0x7f162009f240 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f162669f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16201b4ff0 con 0x7f162009f650 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 0x7f16201a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 0x7f16201a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39342/0 (socket says 172.21.15.32:39342) 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 -- 172.21.15.32:0/3612236528 learned_addr learned my addr 172.21.15.32:0/3612236528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f1624c3c700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009f650 0x7f16201c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161f7fe700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162009f240 0x7f16201a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 -- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009f650 msgr2=0x7f16201c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009f650 0x7f16201c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 -- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162009f240 msgr2=0x7f16201a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162009f240 0x7f16201a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.010+0000 7f161ffff700 1 -- 172.21.15.32:0/3612236528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618009b30 con 0x7f162009e930 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.011+0000 7f161ffff700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 0x7f16201a10d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f161000c690 tx=0x7f1610004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.011+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000cc80 con 0x7f162009e930 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.011+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161000ce00 con 0x7f162009e930 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.011+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161000d940 con 0x7f162009e930 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.011+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610020680 con 0x7f162009e930 2023-09-30T22:27:36.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.011+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16200a0190 con 0x7f162009e930 2023-09-30T22:27:36.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.012+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16100207e0 con 0x7f162009e930 2023-09-30T22:27:36.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.012+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1610044c90 con 0x7f162009e930 2023-09-30T22:27:36.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.012+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162009a3c0 con 0x7f162009e930 2023-09-30T22:27:36.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.017+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16100163f0 con 0x7f162009e930 2023-09-30T22:27:36.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.188+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4b58074ea0 con 0x7f4b580a1340 2023-09-30T22:27:36.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.189+0000 7f4b3f7fe700 1 -- 172.21.15.32:0/1903048653 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b4c01a430 con 0x7f4b580a1340 2023-09-30T22:27:36.190 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 msgr2=0x7f4b58120650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58120650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4b5806c3a0 tx=0x7f4b4c004750 comp rx=0 tx=0).stop 2023-09-30T22:27:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 shutdown_connections 2023-09-30T22:27:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b58133fa0 0x7f4b58120b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b580a0a30 0x7f4b58120110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 --2- 172.21.15.32:0/1903048653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b580a1340 0x7f4b58120650 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 >> 172.21.15.32:0/1903048653 conn(0x7f4b580a6550 msgr2=0x7f4b581bb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 shutdown_connections 2023-09-30T22:27:36.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.191+0000 7f4b5d0ec700 1 -- 172.21.15.32:0/1903048653 wait complete. 2023-09-30T22:27:36.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.226+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1620074ea0 con 0x7f162009e930 2023-09-30T22:27:36.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.226+0000 7f161cff9700 1 -- 172.21.15.32:0/3612236528 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f161002a3b0 con 0x7f162009e930 2023-09-30T22:27:36.227 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 msgr2=0x7f16201a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 0x7f16201a10d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f161000c690 tx=0x7f1610004750 comp rx=0 tx=0).stop 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 shutdown_connections 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162009f650 0x7f16201c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162009e930 0x7f16201a10d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 --2- 172.21.15.32:0/3612236528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162009f240 0x7f16201a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.228+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 >> 172.21.15.32:0/3612236528 conn(0x7f16200a4210 msgr2=0x7f162006c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.229+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 shutdown_connections 2023-09-30T22:27:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.229+0000 7f162669f700 1 -- 172.21.15.32:0/3612236528 wait complete. 2023-09-30T22:27:36.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:36.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.249+0000 7fa32fc95700 1 Processor -- start 2023-09-30T22:27:36.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.249+0000 7fa32fc95700 1 -- start start 2023-09-30T22:27:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32fc95700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809d6f0 0x7fa32809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32fc95700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa32809e410 0x7fa328134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32fc95700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32809b120 con 0x7fa32809e000 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32fc95700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32806b460 con 0x7fa32809e410 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32fc95700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa32809a7e0 con 0x7fa32809d6f0 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32da31700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809d6f0 0x7fa32809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32e232700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa32809e410 0x7fa328134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32da31700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809d6f0 0x7fa32809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40666/0 (socket says 172.21.15.32:40666) 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32da31700 1 -- 172.21.15.32:0/1458952146 learned_addr learned my addr 172.21.15.32:0/1458952146 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32e232700 1 -- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809d6f0 msgr2=0x7fa32809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32e232700 1 --2- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809d6f0 0x7fa32809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32e232700 1 -- 172.21.15.32:0/1458952146 >> v1:172.21.15.32:6789/0 conn(0x7fa32809e000 legacy=0x7fa3281339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32e232700 1 -- 172.21.15.32:0/1458952146 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32809a500 con 0x7fa32809e410 2023-09-30T22:27:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.250+0000 7fa32e232700 1 --2- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa328134100 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa320009d90 tx=0x7fa32000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada311dda34a41d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32ca2f700 1 -- 172.21.15.32:0/1458952146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32000e070 con 0x7fa32809e410 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32ca2f700 1 -- 172.21.15.32:0/1458952146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32000bb20 con 0x7fa32809e410 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32ca2f700 1 -- 172.21.15.32:0/1458952146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320012430 con 0x7fa32809e410 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 -- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 msgr2=0x7fa328134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa328134100 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa320009d90 tx=0x7fa32000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 -- 172.21.15.32:0/1458952146 shutdown_connections 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa328134100 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1458952146 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809d6f0 0x7fa32809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 -- 172.21.15.32:0/1458952146 >> 172.21.15.32:0/1458952146 conn(0x7fa328066f50 msgr2=0x7fa32809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 -- 172.21.15.32:0/1458952146 shutdown_connections 2023-09-30T22:27:36.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.251+0000 7fa32fc95700 1 -- 172.21.15.32:0/1458952146 wait complete. 2023-09-30T22:27:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.252+0000 7fa32fc95700 1 Processor -- start 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.252+0000 7fa32fc95700 1 -- start start 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32fc95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa32809d6f0 0x7fa3281a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32fc95700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 0x7fa3281a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32fc95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa3281a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32fc95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281a9870 con 0x7fa32809e410 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32fc95700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281a7b90 con 0x7fa32809d6f0 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32fc95700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281a9a70 con 0x7fa32809e000 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32d230700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 0x7fa3281a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32e232700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa3281a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32d230700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 0x7fa3281a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40682/0 (socket says 172.21.15.32:40682) 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32d230700 1 -- 172.21.15.32:0/1695249236 learned_addr learned my addr 172.21.15.32:0/1695249236 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32da31700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa32809d6f0 0x7fa3281a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32e232700 1 -- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 msgr2=0x7fa3281a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32e232700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 0x7fa3281a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32e232700 1 -- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa32809d6f0 msgr2=0x7fa3281a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32e232700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa32809d6f0 0x7fa3281a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.253+0000 7fa32e232700 1 -- 172.21.15.32:0/1695249236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009a70 con 0x7fa32809e410 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa32d230700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 0x7fa3281a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa32e232700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa3281a6b20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa320009d90 tx=0x7fa320004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320012430 con 0x7fa32809e410 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320020070 con 0x7fa32809e410 2023-09-30T22:27:36.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3280a2f40 con 0x7fa32809e410 2023-09-30T22:27:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320004280 con 0x7fa32809e410 2023-09-30T22:27:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.254+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32809c760 con 0x7fa32809e410 2023-09-30T22:27:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.255+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3200043e0 con 0x7fa32809e410 2023-09-30T22:27:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.255+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa320022070 con 0x7fa32809e410 2023-09-30T22:27:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.255+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3281aa400 con 0x7fa32809e410 2023-09-30T22:27:36.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.261+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa32001b400 con 0x7fa32809e410 2023-09-30T22:27:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.386+0000 7f6a86469700 1 Processor -- start 2023-09-30T22:27:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.386+0000 7f6a86469700 1 -- start start 2023-09-30T22:27:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a86469700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a801337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a86469700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80134260 0x7f6a801306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a86469700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8006b990 con 0x7f6a80133e30 2023-09-30T22:27:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a86469700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800a0590 con 0x7f6a80134260 2023-09-30T22:27:36.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a86469700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8009fb80 con 0x7f6a80133420 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a801337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a80133e30 0x7f6a8012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35088/0 (socket says 172.21.15.32:35088) 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7f7fe700 1 -- 172.21.15.32:0/2507251369 learned_addr learned my addr 172.21.15.32:0/2507251369 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a84a06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80134260 0x7f6a801306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7ffff700 1 -- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80134260 msgr2=0x7f6a801306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7ffff700 1 --2- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80134260 0x7f6a801306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7ffff700 1 -- 172.21.15.32:0/2507251369 >> v1:172.21.15.32:6789/0 conn(0x7f6a80133e30 legacy=0x7f6a8012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7ffff700 1 -- 172.21.15.32:0/2507251369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800a0030 con 0x7f6a80133420 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7effd700 1 -- 172.21.15.32:0/2507251369 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732624292 0 0) 0x7f6a8006b990 con 0x7f6a80133e30 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.387+0000 7f6a7ffff700 1 --2- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a801337f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009870 tx=0x7f6a70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309b68a393789eeb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a7effd700 1 -- 172.21.15.32:0/2507251369 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011830 con 0x7f6a80133420 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a7effd700 1 -- 172.21.15.32:0/2507251369 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70011990 con 0x7f6a80133420 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a7effd700 1 -- 172.21.15.32:0/2507251369 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011d20 con 0x7f6a80133420 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 -- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 msgr2=0x7f6a801337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 --2- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a801337f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009870 tx=0x7f6a70009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 -- 172.21.15.32:0/2507251369 shutdown_connections 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 --2- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80134260 0x7f6a801306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 --2- 172.21.15.32:0/2507251369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a801337f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 -- 172.21.15.32:0/2507251369 >> 172.21.15.32:0/2507251369 conn(0x7f6a800a4210 msgr2=0x7f6a800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 -- 172.21.15.32:0/2507251369 shutdown_connections 2023-09-30T22:27:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.388+0000 7f6a86469700 1 -- 172.21.15.32:0/2507251369 wait complete. 2023-09-30T22:27:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.389+0000 7f6a86469700 1 Processor -- start 2023-09-30T22:27:36.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 -- start start 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a8009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 0x7f6a8009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 0x7f6a8009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8009de30 con 0x7f6a80134260 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8009dbd0 con 0x7f6a80133e30 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a86469700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8011fe20 con 0x7f6a80133420 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a7f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 0x7f6a8009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a84a06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 0x7f6a8009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a84a06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 0x7f6a8009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39382/0 (socket says 172.21.15.32:39382) 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a7ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a8009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:36.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a84a06700 1 -- 172.21.15.32:0/1972911777 learned_addr learned my addr 172.21.15.32:0/1972911777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.390+0000 7f6a7f7fe700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 0x7f6a8009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52840/0 (socket says 172.21.15.32:52840) 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a84a06700 1 -- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 msgr2=0x7f6a8009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a84a06700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a8009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a84a06700 1 -- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 msgr2=0x7f6a8009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a84a06700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 0x7f6a8009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a84a06700 1 -- 172.21.15.32:0/1972911777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700094a0 con 0x7f6a80134260 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a7ffff700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a8009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a7f7fe700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 0x7f6a8009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a84a06700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 0x7f6a8009cf00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800efc0 tx=0x7f6a7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7800cc90 con 0x7f6a80134260 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7800f040 con 0x7f6a80134260 2023-09-30T22:27:36.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a80120090 con 0x7f6a80134260 2023-09-30T22:27:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780146c0 con 0x7f6a80134260 2023-09-30T22:27:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.391+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8011fb40 con 0x7f6a80134260 2023-09-30T22:27:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.392+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a78014820 con 0x7f6a80134260 2023-09-30T22:27:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.392+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6a78040310 con 0x7f6a80134260 2023-09-30T22:27:36.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.392+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80074ea0 con 0x7f6a80134260 2023-09-30T22:27:36.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.397+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a78018020 con 0x7f6a80134260 2023-09-30T22:27:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.472+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa328074ea0 con 0x7fa32809e410 2023-09-30T22:27:36.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.473+0000 7fa31e7fc700 1 -- 172.21.15.32:0/1695249236 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa320035090 con 0x7fa32809e410 2023-09-30T22:27:36.474 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:36.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 msgr2=0x7fa3281a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa3281a6b20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa320009d90 tx=0x7fa320004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 shutdown_connections 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa32809e410 0x7fa3281a6b20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa32809d6f0 0x7fa3281a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 --2- 172.21.15.32:0/1695249236 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa32809e000 0x7fa3281a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 >> 172.21.15.32:0/1695249236 conn(0x7fa328066f50 msgr2=0x7fa3281baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 shutdown_connections 2023-09-30T22:27:36.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.475+0000 7fa32fc95700 1 -- 172.21.15.32:0/1695249236 wait complete. 2023-09-30T22:27:36.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.601+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6a8006b250 con 0x7f6a80134260 2023-09-30T22:27:36.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.601+0000 7f6a7cff9700 1 -- 172.21.15.32:0/1972911777 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a7802b090 con 0x7f6a80134260 2023-09-30T22:27:36.603 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.603+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 msgr2=0x7f6a8009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 0x7f6a8009cf00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800efc0 tx=0x7f6a7800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 shutdown_connections 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a80134260 0x7f6a8009cf00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a80133420 0x7f6a8009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 --2- 172.21.15.32:0/1972911777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a80133e30 0x7f6a8009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 >> 172.21.15.32:0/1972911777 conn(0x7f6a800a4210 msgr2=0x7f6a801303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 shutdown_connections 2023-09-30T22:27:36.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:36.604+0000 7f6a86469700 1 -- 172.21.15.32:0/1972911777 wait complete. 2023-09-30T22:27:36.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:37.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:37.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.356+0000 7fa98bbfd700 1 Processor -- start 2023-09-30T22:27:37.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.356+0000 7fa98bbfd700 1 -- start start 2023-09-30T22:27:37.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.356+0000 7fa98bbfd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a0a30 0x7fa9840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.356+0000 7fa98bbfd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 0x7fa98413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98bbfd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98409d920 con 0x7fa9840a1340 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98bbfd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98409d020 con 0x7fa9840a0a30 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98bbfd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840a2010 con 0x7fa98413d820 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 0x7fa98413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa989999700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a0a30 0x7fa9840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa989999700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a0a30 0x7fa9840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52848/0 (socket says 172.21.15.32:52848) 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 0x7fa98413fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40696/0 (socket says 172.21.15.32:40696) 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa989198700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9840a1340 0x7fa98413c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35100/0 (socket says 172.21.15.32:35100) 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa989999700 1 -- 172.21.15.32:0/1314025887 learned_addr learned my addr 172.21.15.32:0/1314025887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 -- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a0a30 msgr2=0x7fa9840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:37.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 --2- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a0a30 0x7fa9840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 -- 172.21.15.32:0/1314025887 >> v1:172.21.15.32:6789/0 conn(0x7fa9840a1340 legacy=0x7fa98413c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 -- 172.21.15.32:0/1314025887 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98409dbe0 con 0x7fa98413d820 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa988997700 1 -- 172.21.15.32:0/1314025887 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402279487 0 0) 0x7fa98409d920 con 0x7fa9840a1340 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa98a19a700 1 --2- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 0x7fa98413fc00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009dc0 tx=0x7fa97c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c72e900069e0f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa988997700 1 -- 172.21.15.32:0/1314025887 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00e070 con 0x7fa98413d820 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa988997700 1 -- 172.21.15.32:0/1314025887 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c00bb20 con 0x7fa98413d820 2023-09-30T22:27:37.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.357+0000 7fa988997700 1 -- 172.21.15.32:0/1314025887 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c012430 con 0x7fa98413d820 2023-09-30T22:27:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 -- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 msgr2=0x7fa98413fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 0x7fa98413fc00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009dc0 tx=0x7fa97c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 -- 172.21.15.32:0/1314025887 shutdown_connections 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa98413d820 0x7fa98413fc00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/1314025887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a0a30 0x7fa9840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 -- 172.21.15.32:0/1314025887 >> 172.21.15.32:0/1314025887 conn(0x7fa9840a6550 msgr2=0x7fa9840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 -- 172.21.15.32:0/1314025887 shutdown_connections 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.358+0000 7fa98bbfd700 1 -- 172.21.15.32:0/1314025887 wait complete. 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.359+0000 7fa98bbfd700 1 Processor -- start 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.359+0000 7fa98bbfd700 1 -- start start 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98bbfd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9840a0a30 0x7fa9841271d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98bbfd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 0x7fa984127710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98bbfd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa98413d820 0x7fa984127c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98bbfd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98412acf0 con 0x7fa98413d820 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98bbfd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98412a030 con 0x7fa9840a1340 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98bbfd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840a5170 con 0x7fa9840a0a30 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 0x7fa984127710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 0x7fa984127710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52850/0 (socket says 172.21.15.32:52850) 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98a19a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa98413d820 0x7fa984127c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 -- 172.21.15.32:0/4120594691 learned_addr learned my addr 172.21.15.32:0/4120594691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989999700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9840a0a30 0x7fa9841271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 -- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9840a0a30 msgr2=0x7fa9841271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9840a0a30 0x7fa9841271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 -- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa98413d820 msgr2=0x7fa984127c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa98413d820 0x7fa984127c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 -- 172.21.15.32:0/4120594691 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c009a70 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa98a19a700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa98413d820 0x7fa984127c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.360+0000 7fa989198700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 0x7fa984127710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa974001eb0 tx=0x7fa97400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.361+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974004c80 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.361+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974004de0 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.361+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa984129640 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.361+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974007310 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.361+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98413d640 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.362+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa974007470 con 0x7fa9840a1340 2023-09-30T22:27:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.362+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa97403f3d0 con 0x7fa9840a1340 2023-09-30T22:27:37.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.362+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa984071e20 con 0x7fa9840a1340 2023-09-30T22:27:37.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.367+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9740123b0 con 0x7fa9840a1340 2023-09-30T22:27:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.393+0000 7fc60cf5c700 1 Processor -- start 2023-09-30T22:27:37.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60cf5c700 1 -- start start 2023-09-30T22:27:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60cf5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6080a0a30 0x7fc6080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60cf5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc60809d630 0x7fc6080a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60cf5c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080a2a70 con 0x7fc608133930 2023-09-30T22:27:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60cf5c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080a01d0 con 0x7fc6080a0a30 2023-09-30T22:27:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60cf5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60809e2b0 con 0x7fc60809d630 2023-09-30T22:27:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc60659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6080a0a30 0x7fc6080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc606d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc608133930 0x7fc608133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35108/0 (socket says 172.21.15.32:35108) 2023-09-30T22:27:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.394+0000 7fc606d9d700 1 -- 172.21.15.32:0/2243103669 learned_addr learned my addr 172.21.15.32:0/2243103669 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc605d9b700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc60809d630 0x7fc6080a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60659c700 1 -- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc60809d630 msgr2=0x7fc6080a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60659c700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc60809d630 0x7fc6080a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60659c700 1 -- 172.21.15.32:0/2243103669 >> v1:172.21.15.32:6789/0 conn(0x7fc608133930 legacy=0x7fc608133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60659c700 1 -- 172.21.15.32:0/2243103669 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60809dc50 con 0x7fc6080a0a30 2023-09-30T22:27:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc605d9b700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc60809d630 0x7fc6080a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60659c700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6080a0a30 0x7fc6080a0e00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009d90 tx=0x7fc5f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ede94b2926f5888 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60559a700 1 -- 172.21.15.32:0/2243103669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800e070 con 0x7fc6080a0a30 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60559a700 1 -- 172.21.15.32:0/2243103669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800bb20 con 0x7fc6080a0a30 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.395+0000 7fc60559a700 1 -- 172.21.15.32:0/2243103669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8012430 con 0x7fc6080a0a30 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6080a0a30 msgr2=0x7fc6080a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6080a0a30 0x7fc6080a0e00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009d90 tx=0x7fc5f800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2243103669 shutdown_connections 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6080a0a30 0x7fc6080a0e00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2243103669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc60809d630 0x7fc6080a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2243103669 >> 172.21.15.32:0/2243103669 conn(0x7fc6080a3050 msgr2=0x7fc6080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2243103669 shutdown_connections 2023-09-30T22:27:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.396+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2243103669 wait complete. 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.397+0000 7fc60cf5c700 1 Processor -- start 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.397+0000 7fc60cf5c700 1 -- start start 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60cf5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc60809d630 0x7fc6081c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60cf5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6080a0a30 0x7fc6081c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60cf5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 0x7fc6081c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60cf5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60811eea0 con 0x7fc60809d630 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60cf5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081207f0 con 0x7fc608133930 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60cf5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc608120600 con 0x7fc6080a0a30 2023-09-30T22:27:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 0x7fc6081c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 0x7fc6081c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52870/0 (socket says 172.21.15.32:52870) 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 -- 172.21.15.32:0/2094984007 learned_addr learned my addr 172.21.15.32:0/2094984007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60659c700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc60809d630 0x7fc6081c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc605d9b700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6080a0a30 0x7fc6081c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 -- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6080a0a30 msgr2=0x7fc6081c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6080a0a30 0x7fc6081c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 -- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc60809d630 msgr2=0x7fc6081c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc60809d630 0x7fc6081c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc606d9d700 1 -- 172.21.15.32:0/2094984007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc0094a0 con 0x7fc608133930 2023-09-30T22:27:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.398+0000 7fc60659c700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc60809d630 0x7fc6081c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.399+0000 7fc606d9d700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 0x7fc6081c8ba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c500 tx=0x7fc60000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.399+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600005d70 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.399+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6000070d0 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.399+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f8009a70 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.399+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600007440 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.399+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc608122230 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.400+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6000075a0 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.400+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc608122420 con 0x7fc608133930 2023-09-30T22:27:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.400+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc60003e960 con 0x7fc608133930 2023-09-30T22:27:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.405+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc600019b40 con 0x7fc608133930 2023-09-30T22:27:37.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.574+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa984074ea0 con 0x7fa9840a1340 2023-09-30T22:27:37.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.575+0000 7fa97a7fc700 1 -- 172.21.15.32:0/4120594691 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa974011360 con 0x7fa9840a1340 2023-09-30T22:27:37.576 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.578+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 msgr2=0x7fa984127710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.578+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 0x7fa984127710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa974001eb0 tx=0x7fa97400c800 comp rx=0 tx=0).stop 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.578+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 shutdown_connections 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.578+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa98413d820 0x7fa984127c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.579+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9840a0a30 0x7fa9841271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.579+0000 7fa98bbfd700 1 --2- 172.21.15.32:0/4120594691 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9840a1340 0x7fa984127710 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.579+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 >> 172.21.15.32:0/4120594691 conn(0x7fa9840a6550 msgr2=0x7fa9840a2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.579+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 shutdown_connections 2023-09-30T22:27:37.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.579+0000 7fa98bbfd700 1 -- 172.21.15.32:0/4120594691 wait complete. 2023-09-30T22:27:37.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:37.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.613+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc6080747f0 con 0x7fc608133930 2023-09-30T22:27:37.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.613+0000 7fc5f6ffd700 1 -- 172.21.15.32:0/2094984007 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc600017030 con 0x7fc608133930 2023-09-30T22:27:37.614 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 msgr2=0x7fc6081c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 0x7fc6081c8ba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c500 tx=0x7fc60000c810 comp rx=0 tx=0).stop 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 shutdown_connections 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc608133930 0x7fc6081c8ba0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc60809d630 0x7fc6081c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 --2- 172.21.15.32:0/2094984007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6080a0a30 0x7fc6081c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 >> 172.21.15.32:0/2094984007 conn(0x7fc6080a3050 msgr2=0x7fc60809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 shutdown_connections 2023-09-30T22:27:37.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.616+0000 7fc60cf5c700 1 -- 172.21.15.32:0/2094984007 wait complete. 2023-09-30T22:27:37.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:37.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.637+0000 7ff9d3e41700 1 Processor -- start 2023-09-30T22:27:37.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 -- start start 2023-09-30T22:27:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 0x7ff9cc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e410 0x7ff9cc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc06b460 con 0x7ff9cc09e000 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc09b120 con 0x7ff9cc09e410 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d3e41700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc09a7e0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d1bdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d13dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 0x7ff9cc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d23de700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e410 0x7ff9cc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d13dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 0x7ff9cc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39414/0 (socket says 172.21.15.32:39414) 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d1bdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40720/0 (socket says 172.21.15.32:40720) 2023-09-30T22:27:37.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.638+0000 7ff9d13dc700 1 -- 172.21.15.32:0/3188843130 learned_addr learned my addr 172.21.15.32:0/3188843130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d1bdd700 1 -- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e410 msgr2=0x7ff9cc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d1bdd700 1 --2- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e410 0x7ff9cc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d1bdd700 1 -- 172.21.15.32:0/3188843130 >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 msgr2=0x7ff9cc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d1bdd700 1 --2- 172.21.15.32:0/3188843130 >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 0x7ff9cc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d1bdd700 1 -- 172.21.15.32:0/3188843130 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc09a500 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d23de700 1 --2- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e410 0x7ff9cc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d13dc700 1 --2- 172.21.15.32:0/3188843130 >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 0x7ff9cc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d1bdd700 1 --2- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc09dac0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca48cb75b92b5072 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d0bdb700 1 -- 172.21.15.32:0/3188843130 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8011830 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d0bdb700 1 -- 172.21.15.32:0/3188843130 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8011990 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.639+0000 7ff9d0bdb700 1 -- 172.21.15.32:0/3188843130 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8011d20 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 -- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 msgr2=0x7ff9cc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc09dac0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 -- 172.21.15.32:0/3188843130 shutdown_connections 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e410 0x7ff9cc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/3188843130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc09dac0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/3188843130 >> v2:172.21.15.32:3300/0 conn(0x7ff9cc09e000 0x7ff9cc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 -- 172.21.15.32:0/3188843130 >> 172.21.15.32:0/3188843130 conn(0x7ff9cc066f50 msgr2=0x7ff9cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 -- 172.21.15.32:0/3188843130 shutdown_connections 2023-09-30T22:27:37.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.640+0000 7ff9d3e41700 1 -- 172.21.15.32:0/3188843130 wait complete. 2023-09-30T22:27:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 Processor -- start 2023-09-30T22:27:37.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 -- start start 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 0x7ff9cc1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9cc09e410 0x7ff9cc1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1aa280 con 0x7ff9cc09e410 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a79b0 con 0x7ff9cc09e000 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.642+0000 7ff9d3e41700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a7fd0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d13dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 0x7ff9cc1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d13dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 0x7ff9cc1a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52888/0 (socket says 172.21.15.32:52888) 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d13dc700 1 -- 172.21.15.32:0/2875455689 learned_addr learned my addr 172.21.15.32:0/2875455689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:37.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d23de700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9cc09e410 0x7ff9cc1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 -- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 msgr2=0x7ff9cc1a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 0x7ff9cc1a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 -- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9cc09e410 msgr2=0x7ff9cc1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9cc09e410 0x7ff9cc1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 -- 172.21.15.32:0/2875455689 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc009550 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d13dc700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 0x7ff9cc1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d1bdd700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc1a5e30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc06e470 tx=0x7ff9c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8004450 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c801f070 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c4009a70 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.643+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80038a0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.644+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c80094a0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.645+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9c8003a00 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.645+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc074ea0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.645+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff9c8021070 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.650+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9c8003cb0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.756+0000 7f7c3d080700 1 Processor -- start 2023-09-30T22:27:37.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.756+0000 7f7c3d080700 1 -- start start 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3d080700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c3809d630 0x7f7c380a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3d080700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c38138d80 0x7f7c3813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3d080700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c380a2a70 con 0x7f7c380a0a30 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3d080700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c380a01d0 con 0x7f7c38138d80 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3d080700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3809e2b0 con 0x7f7c3809d630 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c38138d80 0x7f7c3813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c36d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c380a0a30 0x7f7c380a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35112/0 (socket says 172.21.15.32:35112) 2023-09-30T22:27:37.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c36d9d700 1 -- 172.21.15.32:0/3477212415 learned_addr learned my addr 172.21.15.32:0/3477212415 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.757+0000 7f7c3659c700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c3809d630 0x7f7c380a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c3759e700 1 -- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c3809d630 msgr2=0x7f7c380a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c3759e700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c3809d630 0x7f7c380a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c3759e700 1 -- 172.21.15.32:0/3477212415 >> v1:172.21.15.32:6789/0 conn(0x7f7c380a0a30 legacy=0x7f7c380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c3759e700 1 -- 172.21.15.32:0/3477212415 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3809dc50 con 0x7f7c38138d80 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c35d9b700 1 -- 172.21.15.32:0/3477212415 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062392348 0 0) 0x7f7c380a2a70 con 0x7f7c380a0a30 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c3659c700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c3809d630 0x7f7c380a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c3759e700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c38138d80 0x7f7c3813b160 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009840 tx=0x7f7c30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cddab59a6dd905 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c35d9b700 1 -- 172.21.15.32:0/3477212415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30011830 con 0x7f7c38138d80 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c35d9b700 1 -- 172.21.15.32:0/3477212415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c30011990 con 0x7f7c38138d80 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.758+0000 7f7c35d9b700 1 -- 172.21.15.32:0/3477212415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30011d20 con 0x7f7c38138d80 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 -- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c38138d80 msgr2=0x7f7c3813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c38138d80 0x7f7c3813b160 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009840 tx=0x7f7c30009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 -- 172.21.15.32:0/3477212415 shutdown_connections 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c38138d80 0x7f7c3813b160 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 --2- 172.21.15.32:0/3477212415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c3809d630 0x7f7c380a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 -- 172.21.15.32:0/3477212415 >> 172.21.15.32:0/3477212415 conn(0x7f7c380a3050 msgr2=0x7f7c380a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 -- 172.21.15.32:0/3477212415 shutdown_connections 2023-09-30T22:27:37.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.759+0000 7f7c3d080700 1 -- 172.21.15.32:0/3477212415 wait complete. 2023-09-30T22:27:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.760+0000 7f7c3d080700 1 Processor -- start 2023-09-30T22:27:37.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.760+0000 7f7c3d080700 1 -- start start 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3d080700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 0x7f7c3811c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3d080700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 0x7f7c3811cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3d080700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 0x7f7c3811d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3d080700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3811f190 con 0x7f7c3809d630 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3d080700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3809d390 con 0x7f7c380a0a30 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3d080700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3811fde0 con 0x7f7c38138d80 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c36d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 0x7f7c3811c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 0x7f7c3811cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 0x7f7c3811d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 0x7f7c3811cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52902/0 (socket says 172.21.15.32:52902) 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 0x7f7c3811d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40730/0 (socket says 172.21.15.32:40730) 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3659c700 1 -- 172.21.15.32:0/331192020 learned_addr learned my addr 172.21.15.32:0/331192020 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c36d9d700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 0x7f7c3811c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39446/0 (socket says 172.21.15.32:39446) 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 -- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 msgr2=0x7f7c3811cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 0x7f7c3811cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 -- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 msgr2=0x7f7c3811c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 0x7f7c3811c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 -- 172.21.15.32:0/331192020 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200094a0 con 0x7f7c38138d80 2023-09-30T22:27:37.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3659c700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 0x7f7c3811cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c36d9d700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 0x7f7c3811c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.761+0000 7f7c3759e700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 0x7f7c3811d240 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c30016000 tx=0x7f7c30004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.762+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004380 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.762+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3001f070 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.762+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c300094a0 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.762+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c300036a0 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.762+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3811fb00 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.763+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c380747f0 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.763+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c30021030 con 0x7f7c38138d80 2023-09-30T22:27:37.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.763+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c30040f40 con 0x7f7c38138d80 2023-09-30T22:27:37.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.768+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c30010970 con 0x7f7c38138d80 2023-09-30T22:27:37.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.855+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff9cc09c6d0 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.856+0000 7ff9c27fc700 1 -- 172.21.15.32:0/2875455689 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9c8015030 con 0x7ff9cc09d6f0 2023-09-30T22:27:37.857 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:37.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 msgr2=0x7ff9cc1a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc1a5e30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc06e470 tx=0x7ff9c8004750 comp rx=0 tx=0).stop 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 shutdown_connections 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9cc09e410 0x7ff9cc1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9cc09d6f0 0x7ff9cc1a5e30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 --2- 172.21.15.32:0/2875455689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9cc09e000 0x7ff9cc1a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 >> 172.21.15.32:0/2875455689 conn(0x7ff9cc066f50 msgr2=0x7ff9cc1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 shutdown_connections 2023-09-30T22:27:37.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.858+0000 7ff9d3e41700 1 -- 172.21.15.32:0/2875455689 wait complete. 2023-09-30T22:27:37.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:37.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.976+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7c380a4e30 con 0x7f7c38138d80 2023-09-30T22:27:37.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.977+0000 7f7c2f7fe700 1 -- 172.21.15.32:0/331192020 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c300193b0 con 0x7f7c38138d80 2023-09-30T22:27:37.978 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 msgr2=0x7f7c3811d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 0x7f7c3811d240 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c30016000 tx=0x7f7c30004750 comp rx=0 tx=0).stop 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 shutdown_connections 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c38138d80 0x7f7c3811d240 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c3809d630 0x7f7c3811c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 --2- 172.21.15.32:0/331192020 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c380a0a30 0x7f7c3811cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 >> 172.21.15.32:0/331192020 conn(0x7f7c380a3050 msgr2=0x7f7c3809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 shutdown_connections 2023-09-30T22:27:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:37.979+0000 7f7c3d080700 1 -- 172.21.15.32:0/331192020 wait complete. 2023-09-30T22:27:37.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:38.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:38.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.747+0000 7f4a575e2700 1 Processor -- start 2023-09-30T22:27:38.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.747+0000 7f4a575e2700 1 -- start start 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a575e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a500a0a30 0x7f4a500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a575e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 0x7f4a500a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a575e2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a500a2a70 con 0x7f4a50133930 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a575e2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a500a01d0 con 0x7f4a500a0a30 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a575e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5009e2b0 con 0x7f4a5009d630 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a5537e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a500a0a30 0x7f4a500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a54b7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 0x7f4a500a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a5537e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a500a0a30 0x7f4a500a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39458/0 (socket says 172.21.15.32:39458) 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a54b7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 0x7f4a500a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40734/0 (socket says 172.21.15.32:40734) 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a5537e700 1 -- 172.21.15.32:0/884723000 learned_addr learned my addr 172.21.15.32:0/884723000 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:38.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.748+0000 7f4a55b7f700 1 --1- 172.21.15.32:0/884723000 >> v1:172.21.15.32:6789/0 conn(0x7f4a50133930 0x7f4a50133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35124/0 (socket says 172.21.15.32:35124) 2023-09-30T22:27:38.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a5537e700 1 -- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 msgr2=0x7f4a500a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a5537e700 1 --2- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 0x7f4a500a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a5537e700 1 -- 172.21.15.32:0/884723000 >> v1:172.21.15.32:6789/0 conn(0x7f4a50133930 legacy=0x7f4a50133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a5537e700 1 -- 172.21.15.32:0/884723000 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5009dc50 con 0x7f4a500a0a30 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a54b7d700 1 --2- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 0x7f4a500a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a5537e700 1 --2- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a500a0a30 0x7f4a500a0e00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009e50 tx=0x7f4a4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca4c44e98169177 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a47fff700 1 -- 172.21.15.32:0/884723000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00e070 con 0x7f4a500a0a30 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a47fff700 1 -- 172.21.15.32:0/884723000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4c00b910 con 0x7f4a500a0a30 2023-09-30T22:27:38.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.749+0000 7f4a47fff700 1 -- 172.21.15.32:0/884723000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00bcc0 con 0x7f4a500a0a30 2023-09-30T22:27:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 -- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a500a0a30 msgr2=0x7f4a500a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 --2- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a500a0a30 0x7f4a500a0e00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009e50 tx=0x7f4a4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 -- 172.21.15.32:0/884723000 shutdown_connections 2023-09-30T22:27:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 --2- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a500a0a30 0x7f4a500a0e00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 --2- 172.21.15.32:0/884723000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a5009d630 0x7f4a500a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 -- 172.21.15.32:0/884723000 >> 172.21.15.32:0/884723000 conn(0x7f4a500a3050 msgr2=0x7f4a500a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 -- 172.21.15.32:0/884723000 shutdown_connections 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.750+0000 7f4a575e2700 1 -- 172.21.15.32:0/884723000 wait complete. 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.751+0000 7f4a575e2700 1 Processor -- start 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.751+0000 7f4a575e2700 1 -- start start 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a575e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a5009d630 0x7f4a501c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a575e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 0x7f4a501c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a575e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 0x7f4a501c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a575e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5011eea0 con 0x7f4a5009d630 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a575e2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501207f0 con 0x7f4a500a0a30 2023-09-30T22:27:38.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a575e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a50120600 con 0x7f4a50133930 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a54b7d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 0x7f4a501c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a55b7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 0x7f4a501c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a54b7d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 0x7f4a501c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52914/0 (socket says 172.21.15.32:52914) 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a55b7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 0x7f4a501c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40748/0 (socket says 172.21.15.32:40748) 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a54b7d700 1 -- 172.21.15.32:0/1959156124 learned_addr learned my addr 172.21.15.32:0/1959156124 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.752+0000 7f4a5537e700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a5009d630 0x7f4a501c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a55b7f700 1 -- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 msgr2=0x7f4a501c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a55b7f700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 0x7f4a501c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a55b7f700 1 -- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a5009d630 msgr2=0x7f4a501c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a55b7f700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a5009d630 0x7f4a501c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a55b7f700 1 -- 172.21.15.32:0/1959156124 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400094a0 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a54b7d700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 0x7f4a501c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a5537e700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a5009d630 0x7f4a501c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a55b7f700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 0x7f4a501c8ba0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800c430 tx=0x7f4a4800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48004a70 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a48004bd0 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c009b30 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480070d0 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.753+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a50122230 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.754+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a48007440 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.754+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a4803ef50 con 0x7f4a50133930 2023-09-30T22:27:38.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.754+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a500747f0 con 0x7f4a50133930 2023-09-30T22:27:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.759+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a480196d0 con 0x7f4a50133930 2023-09-30T22:27:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 Processor -- start 2023-09-30T22:27:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 -- start start 2023-09-30T22:27:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee88138d80 0x7fee8813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee880a2a70 con 0x7fee8809d630 2023-09-30T22:27:38.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee880a01d0 con 0x7fee88138d80 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ef2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee8809e2b0 con 0x7fee880a0a30 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ccc7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ccc7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40752/0 (socket says 172.21.15.32:40752) 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.785+0000 7fee8ccc7700 1 -- 172.21.15.32:0/753737660 learned_addr learned my addr 172.21.15.32:0/753737660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8d4c8700 1 --2- 172.21.15.32:0/753737660 >> v2:172.21.15.32:3300/0 conn(0x7fee88138d80 0x7fee8813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8ccc7700 1 -- 172.21.15.32:0/753737660 >> v2:172.21.15.32:3300/0 conn(0x7fee88138d80 msgr2=0x7fee8813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8ccc7700 1 --2- 172.21.15.32:0/753737660 >> v2:172.21.15.32:3300/0 conn(0x7fee88138d80 0x7fee8813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8ccc7700 1 -- 172.21.15.32:0/753737660 >> v1:172.21.15.32:6789/0 conn(0x7fee8809d630 legacy=0x7fee880a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8ccc7700 1 -- 172.21.15.32:0/753737660 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8809dc50 con 0x7fee880a0a30 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8d4c8700 1 --2- 172.21.15.32:0/753737660 >> v2:172.21.15.32:3300/0 conn(0x7fee88138d80 0x7fee8813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:38.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee8ccc7700 1 --2- 172.21.15.32:0/753737660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee880a0e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fee780098f0 tx=0x7fee78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01ab0a3b01e0f4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee877fe700 1 -- 172.21.15.32:0/753737660 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800b8a0 con 0x7fee880a0a30 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee877fe700 1 -- 172.21.15.32:0/753737660 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7800ba00 con 0x7fee880a0a30 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.786+0000 7fee877fe700 1 -- 172.21.15.32:0/753737660 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800bd90 con 0x7fee880a0a30 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 -- 172.21.15.32:0/753737660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 msgr2=0x7fee880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/753737660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee880a0e00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fee780098f0 tx=0x7fee78009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 -- 172.21.15.32:0/753737660 shutdown_connections 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/753737660 >> v2:172.21.15.32:3300/0 conn(0x7fee88138d80 0x7fee8813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/753737660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee880a0e00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 -- 172.21.15.32:0/753737660 >> 172.21.15.32:0/753737660 conn(0x7fee880a3050 msgr2=0x7fee880a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 -- 172.21.15.32:0/753737660 shutdown_connections 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.787+0000 7fee8ef2b700 1 -- 172.21.15.32:0/753737660 wait complete. 2023-09-30T22:27:38.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.788+0000 7fee8ef2b700 1 Processor -- start 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.788+0000 7fee8ef2b700 1 -- start start 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ef2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 0x7fee881208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ef2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee88120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ef2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee88138d80 0x7fee88121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ef2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8813b070 con 0x7fee8809d630 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ef2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee880a0410 con 0x7fee88138d80 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ef2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee88123b10 con 0x7fee880a0a30 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 0x7fee881208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8d4c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee88138d80 0x7fee88121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 0x7fee881208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39484/0 (socket says 172.21.15.32:39484) 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee87fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee88120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 -- 172.21.15.32:0/4130941997 learned_addr learned my addr 172.21.15.32:0/4130941997 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 -- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 msgr2=0x7fee88120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee88120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 -- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee88138d80 msgr2=0x7fee88121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee88138d80 0x7fee88121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 -- 172.21.15.32:0/4130941997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78009550 con 0x7fee8809d630 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee8ccc7700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 0x7fee881208f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.789+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78004510 con 0x7fee8809d630 2023-09-30T22:27:38.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.790+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee780036a0 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.790+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee881240a0 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.790+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78003a30 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.790+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee88124b10 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.791+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee78021070 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.791+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee880747f0 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.791+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fee78040a80 con 0x7fee8809d630 2023-09-30T22:27:38.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.796+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee78010380 con 0x7fee8809d630 2023-09-30T22:27:38.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:38.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.971+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4a50120df0 con 0x7f4a50133930 2023-09-30T22:27:38.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.972+0000 7f4a45ffb700 1 -- 172.21.15.32:0/1959156124 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a480113a0 con 0x7f4a50133930 2023-09-30T22:27:38.973 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:38.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 msgr2=0x7f4a501c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 0x7f4a501c8ba0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800c430 tx=0x7f4a4800c740 comp rx=0 tx=0).stop 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 shutdown_connections 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a50133930 0x7f4a501c8ba0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a5009d630 0x7f4a501c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 --2- 172.21.15.32:0/1959156124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a500a0a30 0x7f4a501c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.974+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 >> 172.21.15.32:0/1959156124 conn(0x7f4a500a3050 msgr2=0x7f4a5009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:38.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.975+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 shutdown_connections 2023-09-30T22:27:38.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:38.975+0000 7f4a575e2700 1 -- 172.21.15.32:0/1959156124 wait complete. 2023-09-30T22:27:38.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:38.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.005+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fee88072980 con 0x7fee8809d630 2023-09-30T22:27:39.007 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.006+0000 7fee857fa700 1 -- 172.21.15.32:0/4130941997 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee78019ba0 con 0x7fee8809d630 2023-09-30T22:27:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 msgr2=0x7fee881208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 0x7fee881208f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78004750 comp rx=0 tx=0).stop 2023-09-30T22:27:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 shutdown_connections 2023-09-30T22:27:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee88138d80 0x7fee88121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee8809d630 0x7fee881208f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 --2- 172.21.15.32:0/4130941997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fee880a0a30 0x7fee88120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 >> 172.21.15.32:0/4130941997 conn(0x7fee880a3050 msgr2=0x7fee88139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 shutdown_connections 2023-09-30T22:27:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.008+0000 7fee8ef2b700 1 -- 172.21.15.32:0/4130941997 wait complete. 2023-09-30T22:27:39.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.023+0000 7fc2bcbdd700 1 Processor -- start 2023-09-30T22:27:39.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2bcbdd700 1 -- start start 2023-09-30T22:27:39.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2bcbdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b8133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2bcbdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b809e560 0x7fc2b813b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2bcbdd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b806f720 con 0x7fc2b81341d0 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2bcbdd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8070c80 con 0x7fc2b809e560 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2bcbdd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b806fce0 con 0x7fc2b81338c0 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2b5d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2b81341d0 0x7fc2b809de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35140/0 (socket says 172.21.15.32:35140) 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.024+0000 7fc2b5d9b700 1 -- 172.21.15.32:0/566926731 learned_addr learned my addr 172.21.15.32:0/566926731 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b659c700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b8133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b6d9d700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b809e560 0x7fc2b813b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b659c700 1 -- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b809e560 msgr2=0x7fc2b813b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b659c700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b809e560 0x7fc2b813b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b659c700 1 -- 172.21.15.32:0/566926731 >> v1:172.21.15.32:6789/0 conn(0x7fc2b81341d0 legacy=0x7fc2b809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b559a700 1 -- 172.21.15.32:0/566926731 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589368870 0 0) 0x7fc2b806f720 con 0x7fc2b81341d0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b659c700 1 -- 172.21.15.32:0/566926731 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b81336a0 con 0x7fc2b81338c0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b659c700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b8133c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009870 tx=0x7fc2a8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5024dcb00b44d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b559a700 1 -- 172.21.15.32:0/566926731 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8011830 con 0x7fc2b81338c0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b559a700 1 -- 172.21.15.32:0/566926731 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a8011990 con 0x7fc2b81338c0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.025+0000 7fc2b559a700 1 -- 172.21.15.32:0/566926731 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8011d20 con 0x7fc2b81338c0 2023-09-30T22:27:39.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 msgr2=0x7fc2b8133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b8133c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009870 tx=0x7fc2a8009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/566926731 shutdown_connections 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b809e560 0x7fc2b813b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/566926731 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b8133c90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/566926731 >> 172.21.15.32:0/566926731 conn(0x7fc2b80a2aa0 msgr2=0x7fc2b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/566926731 shutdown_connections 2023-09-30T22:27:39.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.026+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/566926731 wait complete. 2023-09-30T22:27:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.027+0000 7fc2bcbdd700 1 Processor -- start 2023-09-30T22:27:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 -- start start 2023-09-30T22:27:39.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 0x7fc2b811c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b811cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b81341d0 0x7fc2b811d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8120350 con 0x7fc2b809e560 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b811f2d0 con 0x7fc2b81341d0 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2bcbdd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b812dc00 con 0x7fc2b81338c0 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2b5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b811cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 0x7fc2b811c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2b5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b811cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40766/0 (socket says 172.21.15.32:40766) 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 0x7fc2b811c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39490/0 (socket says 172.21.15.32:39490) 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2b5d9b700 1 -- 172.21.15.32:0/4137284849 learned_addr learned my addr 172.21.15.32:0/4137284849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:39.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.028+0000 7fc2b6d9d700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b81341d0 0x7fc2b811d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b5d9b700 1 -- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b81341d0 msgr2=0x7fc2b811d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b5d9b700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b81341d0 0x7fc2b811d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b5d9b700 1 -- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 msgr2=0x7fc2b811c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b5d9b700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 0x7fc2b811c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b5d9b700 1 -- 172.21.15.32:0/4137284849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80094a0 con 0x7fc2b81338c0 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b659c700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 0x7fc2b811c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2b5d9b700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b811cdc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c520 tx=0x7fc2ac00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004c80 con 0x7fc2b81338c0 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac004de0 con 0x7fc2b81338c0 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac007310 con 0x7fc2b81338c0 2023-09-30T22:27:39.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b8130e50 con 0x7fc2b81338c0 2023-09-30T22:27:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.029+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b806f9b0 con 0x7fc2b81338c0 2023-09-30T22:27:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.030+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b8074ea0 con 0x7fc2b81338c0 2023-09-30T22:27:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.031+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2ac007470 con 0x7fc2b81338c0 2023-09-30T22:27:39.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.031+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc2ac03f860 con 0x7fc2b81338c0 2023-09-30T22:27:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.036+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2ac0133c0 con 0x7fc2b81338c0 2023-09-30T22:27:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 Processor -- start 2023-09-30T22:27:39.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 -- start start 2023-09-30T22:27:39.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880134900 0x7f7880136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 0x7f7880139600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7880096db0 con 0x7f788009d640 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788009d2c0 con 0x7f7880134900 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.146+0000 7f78852a6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f788009bd50 con 0x7f7880137220 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 0x7f7880139600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880134900 0x7f7880136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f788009d640 0x7f788009da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35144/0 (socket says 172.21.15.32:35144) 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 0x7f7880139600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40768/0 (socket says 172.21.15.32:40768) 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787effd700 1 -- 172.21.15.32:0/3132748339 learned_addr learned my addr 172.21.15.32:0/3132748339 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:39.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787dffb700 1 -- 172.21.15.32:0/3132748339 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669814981 0 0) 0x7f7880096db0 con 0x7f788009d640 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787dffb700 1 -- 172.21.15.32:0/3132748339 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c0035b0 con 0x7f788009d640 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787e7fc700 1 -- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 msgr2=0x7f7880139600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787e7fc700 1 --2- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 0x7f7880139600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787e7fc700 1 -- 172.21.15.32:0/3132748339 >> v1:172.21.15.32:6789/0 conn(0x7f788009d640 legacy=0x7f788009da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787e7fc700 1 -- 172.21.15.32:0/3132748339 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788009e5d0 con 0x7f7880134900 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787f7fe700 1 --2- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 0x7f7880139600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787e7fc700 1 --2- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880134900 0x7f7880136ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7868009850 tx=0x7f7868009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4ec91515af401a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787dffb700 1 -- 172.21.15.32:0/3132748339 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868005e30 con 0x7f7880134900 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787dffb700 1 -- 172.21.15.32:0/3132748339 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868006030 con 0x7f7880134900 2023-09-30T22:27:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.147+0000 7f787dffb700 1 -- 172.21.15.32:0/3132748339 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680063e0 con 0x7f7880134900 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 -- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880134900 msgr2=0x7f7880136ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 --2- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880134900 0x7f7880136ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7868009850 tx=0x7f7868009b60 comp rx=0 tx=0).stop 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 -- 172.21.15.32:0/3132748339 shutdown_connections 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 --2- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880137220 0x7f7880139600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 --2- 172.21.15.32:0/3132748339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880134900 0x7f7880136ce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 -- 172.21.15.32:0/3132748339 >> 172.21.15.32:0/3132748339 conn(0x7f78800a25a0 msgr2=0x7f78800a29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 -- 172.21.15.32:0/3132748339 shutdown_connections 2023-09-30T22:27:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.148+0000 7f78852a6700 1 -- 172.21.15.32:0/3132748339 wait complete. 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 Processor -- start 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 -- start start 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788009d640 0x7f7880099680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880134900 0x7f7880099bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880137220 0x7f788009a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7880136e80 con 0x7f788009d640 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800908f0 con 0x7f7880137220 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f78852a6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7880098ce0 con 0x7f7880134900 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f787f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880137220 0x7f788009a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f787effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788009d640 0x7f7880099680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f787f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880137220 0x7f788009a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:52978/0 (socket says 172.21.15.32:52978) 2023-09-30T22:27:39.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f787f7fe700 1 -- 172.21.15.32:0/3772727027 learned_addr learned my addr 172.21.15.32:0/3772727027 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.150+0000 7f787e7fc700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880134900 0x7f7880099bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f787effd700 1 -- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880134900 msgr2=0x7f7880099bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f787effd700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880134900 0x7f7880099bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f787effd700 1 -- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880137220 msgr2=0x7f788009a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f787effd700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880137220 0x7f788009a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f787effd700 1 -- 172.21.15.32:0/3772727027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780094a0 con 0x7f788009d640 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f787effd700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788009d640 0x7f7880099680 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f787000c760 tx=0x7f787000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78700070f0 con 0x7f788009d640 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870007250 con 0x7f788009d640 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7868009430 con 0x7f788009d640 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78700075e0 con 0x7f788009d640 2023-09-30T22:27:39.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.151+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7880090bb0 con 0x7f788009d640 2023-09-30T22:27:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.152+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7870007740 con 0x7f788009d640 2023-09-30T22:27:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.152+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788006ca70 con 0x7f788009d640 2023-09-30T22:27:39.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.152+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f787003ed30 con 0x7f788009d640 2023-09-30T22:27:39.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.157+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7870017080 con 0x7f788009d640 2023-09-30T22:27:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.245+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc2b806b9e0 con 0x7fc2b81338c0 2023-09-30T22:27:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.245+0000 7fc2a6ffd700 1 -- 172.21.15.32:0/4137284849 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2ac011360 con 0x7fc2b81338c0 2023-09-30T22:27:39.246 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 msgr2=0x7fc2b811cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b811cdc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c520 tx=0x7fc2ac00c830 comp rx=0 tx=0).stop 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 shutdown_connections 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2b81341d0 0x7fc2b811d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2b809e560 0x7fc2b811c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 --2- 172.21.15.32:0/4137284849 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc2b81338c0 0x7fc2b811cdc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 >> 172.21.15.32:0/4137284849 conn(0x7fc2b80a2aa0 msgr2=0x7fc2b809f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 shutdown_connections 2023-09-30T22:27:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.248+0000 7fc2bcbdd700 1 -- 172.21.15.32:0/4137284849 wait complete. 2023-09-30T22:27:39.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.363+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78800747f0 con 0x7f788009d640 2023-09-30T22:27:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.363+0000 7f78777fe700 1 -- 172.21.15.32:0/3772727027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7870019cd0 con 0x7f788009d640 2023-09-30T22:27:39.364 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788009d640 msgr2=0x7f7880099680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788009d640 0x7f7880099680 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f787000c760 tx=0x7f787000ca70 comp rx=0 tx=0).stop 2023-09-30T22:27:39.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 shutdown_connections 2023-09-30T22:27:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7880137220 0x7f788009a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788009d640 0x7f7880099680 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 --2- 172.21.15.32:0/3772727027 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7880134900 0x7f7880099bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 >> 172.21.15.32:0/3772727027 conn(0x7f78800a25a0 msgr2=0x7f78801326e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 shutdown_connections 2023-09-30T22:27:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:39.365+0000 7f78852a6700 1 -- 172.21.15.32:0/3772727027 wait complete. 2023-09-30T22:27:39.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:39.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:40.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.149+0000 7f89d9c59700 1 Processor -- start 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.149+0000 7f89d9c59700 1 -- start start 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d9c59700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89d40a0a30 0x7f89d40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d9c59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d4134170 0x7f89d413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d9c59700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d409edd0 con 0x7f89d40a1340 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d9c59700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d409e1f0 con 0x7f89d40a0a30 2023-09-30T22:27:40.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d9c59700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d409dc50 con 0x7f89d4134170 2023-09-30T22:27:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89d40a0a30 0x7f89d40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d4134170 0x7f89d413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89d40a0a30 0x7f89d40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39500/0 (socket says 172.21.15.32:39500) 2023-09-30T22:27:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 -- 172.21.15.32:0/1289431033 learned_addr learned my addr 172.21.15.32:0/1289431033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d2ffd700 1 --1- 172.21.15.32:0/1289431033 >> v1:172.21.15.32:6789/0 conn(0x7f89d40a1340 0x7f89d4133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35146/0 (socket says 172.21.15.32:35146) 2023-09-30T22:27:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 -- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d4134170 msgr2=0x7f89d413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:40.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 --2- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d4134170 0x7f89d413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 -- 172.21.15.32:0/1289431033 >> v1:172.21.15.32:6789/0 conn(0x7f89d40a1340 legacy=0x7f89d4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.150+0000 7f89d37fe700 1 -- 172.21.15.32:0/1289431033 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d40a1fe0 con 0x7f89d40a0a30 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.151+0000 7f89d37fe700 1 --2- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a0a30 0x7f89d40a0e00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f89c40098f0 tx=0x7f89c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34939d54bc2eb2cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.151+0000 7f89d27fc700 1 -- 172.21.15.32:0/1289431033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4011830 con 0x7f89d40a0a30 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.151+0000 7f89d27fc700 1 -- 172.21.15.32:0/1289431033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4011990 con 0x7f89d40a0a30 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.151+0000 7f89d27fc700 1 -- 172.21.15.32:0/1289431033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4011d20 con 0x7f89d40a0a30 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 -- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a0a30 msgr2=0x7f89d40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 --2- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a0a30 0x7f89d40a0e00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f89c40098f0 tx=0x7f89c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:40.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 -- 172.21.15.32:0/1289431033 shutdown_connections 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 --2- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d4134170 0x7f89d413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 --2- 172.21.15.32:0/1289431033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a0a30 0x7f89d40a0e00 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 -- 172.21.15.32:0/1289431033 >> 172.21.15.32:0/1289431033 conn(0x7f89d40a6550 msgr2=0x7f89d40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 -- 172.21.15.32:0/1289431033 shutdown_connections 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.152+0000 7f89d9c59700 1 -- 172.21.15.32:0/1289431033 wait complete. 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 Processor -- start 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 -- start start 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 0x7f89d411e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 0x7f89d411edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89d4134170 0x7f89d411f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4120a50 con 0x7f89d40a1340 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40a0290 con 0x7f89d40a0a30 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.153+0000 7f89d9c59700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41205b0 con 0x7f89d4134170 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 0x7f89d411edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 0x7f89d411e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 0x7f89d411e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53004/0 (socket says 172.21.15.32:53004) 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 -- 172.21.15.32:0/3349569857 learned_addr learned my addr 172.21.15.32:0/3349569857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 0x7f89d411edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39502/0 (socket says 172.21.15.32:39502) 2023-09-30T22:27:40.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d3fff700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89d4134170 0x7f89d411f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 -- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89d4134170 msgr2=0x7f89d411f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89d4134170 0x7f89d411f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 -- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 msgr2=0x7f89d411edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 0x7f89d411edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 -- 172.21.15.32:0/3349569857 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009550 con 0x7f89d40a0a30 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d2ffd700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 0x7f89d411edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d3fff700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89d4134170 0x7f89d411f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d37fe700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 0x7f89d411e8b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89c4000c00 tx=0x7f89c4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40036a0 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4003800 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4003bb0 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d40a25e0 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.154+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d4120c20 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.155+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89c4021070 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.155+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4074ea0 con 0x7f89d40a0a30 2023-09-30T22:27:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.156+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f89c4040dc0 con 0x7f89d40a0a30 2023-09-30T22:27:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.161+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89c4024470 con 0x7f89d40a0a30 2023-09-30T22:27:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 Processor -- start 2023-09-30T22:27:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 -- start start 2023-09-30T22:27:40.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b900a0a30 0x7f6b900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9009d630 0x7f6b900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900a2a70 con 0x7f6b90133930 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900a01d0 con 0x7f6b900a0a30 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b95b2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9009e2b0 con 0x7f6b9009d630 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.187+0000 7f6b8f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b900a0a30 0x7f6b900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b90133930 0x7f6b90133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35152/0 (socket says 172.21.15.32:35152) 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8ffff700 1 -- 172.21.15.32:0/1056015718 learned_addr learned my addr 172.21.15.32:0/1056015718 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:40.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b86ffd700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9009d630 0x7f6b900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8effd700 1 -- 172.21.15.32:0/1056015718 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286736712 0 0) 0x7f6b900a2a70 con 0x7f6b90133930 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8f7fe700 1 -- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9009d630 msgr2=0x7f6b900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8f7fe700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9009d630 0x7f6b900a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8f7fe700 1 -- 172.21.15.32:0/1056015718 >> v1:172.21.15.32:6789/0 conn(0x7f6b90133930 legacy=0x7f6b90133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8f7fe700 1 -- 172.21.15.32:0/1056015718 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9009dc50 con 0x7f6b900a0a30 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b86ffd700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9009d630 0x7f6b900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8f7fe700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b900a0a30 0x7f6b900a0e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b80009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952974fdd4dc2ff8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8effd700 1 -- 172.21.15.32:0/1056015718 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80011830 con 0x7f6b900a0a30 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8effd700 1 -- 172.21.15.32:0/1056015718 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80011990 con 0x7f6b900a0a30 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.188+0000 7f6b8effd700 1 -- 172.21.15.32:0/1056015718 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80011d20 con 0x7f6b900a0a30 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 -- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b900a0a30 msgr2=0x7f6b900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b900a0a30 0x7f6b900a0e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b80009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 -- 172.21.15.32:0/1056015718 shutdown_connections 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b900a0a30 0x7f6b900a0e00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/1056015718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9009d630 0x7f6b900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 -- 172.21.15.32:0/1056015718 >> 172.21.15.32:0/1056015718 conn(0x7f6b900a3050 msgr2=0x7f6b900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 -- 172.21.15.32:0/1056015718 shutdown_connections 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.189+0000 7f6b95b2b700 1 -- 172.21.15.32:0/1056015718 wait complete. 2023-09-30T22:27:40.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.190+0000 7f6b95b2b700 1 Processor -- start 2023-09-30T22:27:40.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.190+0000 7f6b95b2b700 1 -- start start 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b95b2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9009d630 0x7f6b9011f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b95b2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 0x7f6b9011f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b95b2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b90133930 0x7f6b901c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b95b2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90121ed0 con 0x7f6b9009d630 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b95b2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90133250 con 0x7f6b90133930 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b95b2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9011ee20 con 0x7f6b900a0a30 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 0x7f6b9011f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 0x7f6b9011f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40814/0 (socket says 172.21.15.32:40814) 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 -- 172.21.15.32:0/138260887 learned_addr learned my addr 172.21.15.32:0/138260887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b8ffff700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b90133930 0x7f6b901c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b8f7fe700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9009d630 0x7f6b9011f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 -- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b90133930 msgr2=0x7f6b901c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b90133930 0x7f6b901c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 -- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9009d630 msgr2=0x7f6b9011f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9009d630 0x7f6b9011f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b86ffd700 1 -- 172.21.15.32:0/138260887 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780094a0 con 0x7f6b900a0a30 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b8ffff700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b90133930 0x7f6b901c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.191+0000 7f6b8f7fe700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9009d630 0x7f6b9011f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.192+0000 7f6b86ffd700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 0x7f6b9011f800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800ebb0 tx=0x7f6b7800eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.192+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800c7f0 con 0x7f6b900a0a30 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.192+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7800c950 con 0x7f6b900a0a30 2023-09-30T22:27:40.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.192+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b800094a0 con 0x7f6b900a0a30 2023-09-30T22:27:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.192+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800cce0 con 0x7f6b900a0a30 2023-09-30T22:27:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.192+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90120520 con 0x7f6b900a0a30 2023-09-30T22:27:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.193+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b78013470 con 0x7f6b900a0a30 2023-09-30T22:27:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.193+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b7803fad0 con 0x7f6b900a0a30 2023-09-30T22:27:40.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.193+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b90120a40 con 0x7f6b900a0a30 2023-09-30T22:27:40.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.198+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b78021080 con 0x7f6b900a0a30 2023-09-30T22:27:40.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:40.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.373+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f89d40a28e0 con 0x7f89d40a0a30 2023-09-30T22:27:40.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.374+0000 7f89bbfff700 1 -- 172.21.15.32:0/3349569857 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89c4029070 con 0x7f89d40a0a30 2023-09-30T22:27:40.375 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 msgr2=0x7f89d411e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 0x7f89d411e8b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89c4000c00 tx=0x7f89c4004750 comp rx=0 tx=0).stop 2023-09-30T22:27:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 shutdown_connections 2023-09-30T22:27:40.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89d4134170 0x7f89d411f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89d40a0a30 0x7f89d411e8b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 --2- 172.21.15.32:0/3349569857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89d40a1340 0x7f89d411edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 >> 172.21.15.32:0/3349569857 conn(0x7f89d40a6550 msgr2=0x7f89d41b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 shutdown_connections 2023-09-30T22:27:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.376+0000 7f89d9c59700 1 -- 172.21.15.32:0/3349569857 wait complete. 2023-09-30T22:27:40.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:40.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.402+0000 7fb4db2e2700 1 Processor -- start 2023-09-30T22:27:40.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4db2e2700 1 -- start start 2023-09-30T22:27:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4db2e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4db2e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4d406b080 0x7fb4d4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4db2e2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d409ece0 con 0x7fb4d4131b40 2023-09-30T22:27:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4db2e2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d409e760 con 0x7fb4d406b080 2023-09-30T22:27:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4db2e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d409e040 con 0x7fb4d406a770 2023-09-30T22:27:40.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4d887d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4d406b080 0x7fb4d4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4d907e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4d887d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4d406b080 0x7fb4d4131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39520/0 (socket says 172.21.15.32:39520) 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4d887d700 1 -- 172.21.15.32:0/3035050411 learned_addr learned my addr 172.21.15.32:0/3035050411 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.403+0000 7fb4d907e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:40828/0 (socket says 172.21.15.32:40828) 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4d887d700 1 -- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 msgr2=0x7fb4d406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4d887d700 1 --2- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4d887d700 1 -- 172.21.15.32:0/3035050411 >> v1:172.21.15.32:6789/0 conn(0x7fb4d4131b40 legacy=0x7fb4d4138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4d887d700 1 -- 172.21.15.32:0/3035050411 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d409f630 con 0x7fb4d406b080 2023-09-30T22:27:40.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4d887d700 1 --2- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d406b080 0x7fb4d4131600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40098f0 tx=0x7fb4c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e88c64ed1062dff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4cbfff700 1 -- 172.21.15.32:0/3035050411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4011830 con 0x7fb4d406b080 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4cbfff700 1 -- 172.21.15.32:0/3035050411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c4011990 con 0x7fb4d406b080 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.404+0000 7fb4cbfff700 1 -- 172.21.15.32:0/3035050411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4011d20 con 0x7fb4d406b080 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 -- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d406b080 msgr2=0x7fb4d4131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d406b080 0x7fb4d4131600 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40098f0 tx=0x7fb4c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 -- 172.21.15.32:0/3035050411 shutdown_connections 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/3035050411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d406b080 0x7fb4d4131600 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 -- 172.21.15.32:0/3035050411 >> 172.21.15.32:0/3035050411 conn(0x7fb4d4066f50 msgr2=0x7fb4d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 -- 172.21.15.32:0/3035050411 shutdown_connections 2023-09-30T22:27:40.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.405+0000 7fb4db2e2700 1 -- 172.21.15.32:0/3035050411 wait complete. 2023-09-30T22:27:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.406+0000 7fb4db2e2700 1 Processor -- start 2023-09-30T22:27:40.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.406+0000 7fb4db2e2700 1 -- start start 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4db2e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d409add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4db2e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4d406b080 0x7fb4d409b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4db2e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d4131b40 0x7fb4d41c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4db2e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4071e60 con 0x7fb4d4131b40 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4db2e2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d409a8e0 con 0x7fb4d406b080 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4db2e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d409b850 con 0x7fb4d406a770 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d887d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4d406b080 0x7fb4d409b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d907e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d409add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d987f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d4131b40 0x7fb4d41c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d887d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4d406b080 0x7fb4d409b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:53028/0 (socket says 172.21.15.32:53028) 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d887d700 1 -- 172.21.15.32:0/366387288 learned_addr learned my addr 172.21.15.32:0/366387288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d907e700 1 -- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4d406b080 msgr2=0x7fb4d409b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d907e700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4d406b080 0x7fb4d409b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d907e700 1 -- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d4131b40 msgr2=0x7fb4d41c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d907e700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d4131b40 0x7fb4d41c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d907e700 1 -- 172.21.15.32:0/366387288 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4009550 con 0x7fb4d406a770 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.407+0000 7fb4d987f700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d4131b40 0x7fb4d41c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.408+0000 7fb4d907e700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d409add0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000e900 tx=0x7fb4d000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.408+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0013680 con 0x7fb4d406a770 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.408+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d00137e0 con 0x7fb4d406a770 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.408+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0013b50 con 0x7fb4d406a770 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.408+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d4121620 con 0x7fb4d406a770 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.408+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d409c010 con 0x7fb4d406a770 2023-09-30T22:27:40.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.409+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4d0013cb0 con 0x7fb4d406a770 2023-09-30T22:27:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.409+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb4d003fa50 con 0x7fb4d406a770 2023-09-30T22:27:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.409+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d404ea50 con 0x7fb4d406a770 2023-09-30T22:27:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.412+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6b90120cc0 con 0x7f6b900a0a30 2023-09-30T22:27:40.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.414+0000 7f6b8cff9700 1 -- 172.21.15.32:0/138260887 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b7803de60 con 0x7f6b900a0a30 2023-09-30T22:27:40.416 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:40.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.414+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4d001adc0 con 0x7fb4d406a770 2023-09-30T22:27:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 msgr2=0x7f6b9011f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 0x7f6b9011f800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800ebb0 tx=0x7f6b7800eec0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 shutdown_connections 2023-09-30T22:27:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b90133930 0x7f6b901c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9009d630 0x7f6b9011f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 --2- 172.21.15.32:0/138260887 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b900a0a30 0x7f6b9011f800 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 >> 172.21.15.32:0/138260887 conn(0x7f6b900a3050 msgr2=0x7f6b901bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 shutdown_connections 2023-09-30T22:27:40.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.417+0000 7f6b95b2b700 1 -- 172.21.15.32:0/138260887 wait complete. 2023-09-30T22:27:40.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:40.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.522+0000 7fa6c9bb5700 1 Processor -- start 2023-09-30T22:27:40.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 -- start start 2023-09-30T22:27:40.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409d6f0 0x7fa6c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6c409e410 0x7fa6c4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c409f520 con 0x7fa6c409e410 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c409f130 con 0x7fa6c409e000 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c9bb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c409e930 con 0x7fa6c409d6f0 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa6c409e410 0x7fa6c4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33528/0 (socket says 172.21.15.32:33528) 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c2ffd700 1 -- 172.21.15.32:0/3641657000 learned_addr learned my addr 172.21.15.32:0/3641657000 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.523+0000 7fa6c37fe700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409d6f0 0x7fa6c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c3fff700 1 -- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409d6f0 msgr2=0x7fa6c409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c3fff700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409d6f0 0x7fa6c409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c3fff700 1 -- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 msgr2=0x7fa6c4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:40.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c3fff700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c3fff700 1 -- 172.21.15.32:0/3641657000 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c409b2d0 con 0x7fa6c409e410 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c2ffd700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c37fe700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409d6f0 0x7fa6c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c3fff700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409e410 0x7fa6c4130ba0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009840 tx=0x7fa6bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d5834e79e9ded0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c27fc700 1 -- 172.21.15.32:0/3641657000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc011830 con 0x7fa6c409e410 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c27fc700 1 -- 172.21.15.32:0/3641657000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc011990 con 0x7fa6c409e410 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.524+0000 7fa6c27fc700 1 -- 172.21.15.32:0/3641657000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc011d20 con 0x7fa6c409e410 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409e410 msgr2=0x7fa6c4130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409e410 0x7fa6c4130ba0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009840 tx=0x7fa6bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3641657000 shutdown_connections 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409e410 0x7fa6c4130ba0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409d6f0 0x7fa6c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3641657000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3641657000 >> 172.21.15.32:0/3641657000 conn(0x7fa6c4066f50 msgr2=0x7fa6c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3641657000 shutdown_connections 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.525+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3641657000 wait complete. 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 Processor -- start 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 -- start start 2023-09-30T22:27:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 0x7fa6c4137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409e410 0x7fa6c41383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c411a5b0 con 0x7fa6c409d6f0 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4072530 con 0x7fa6c409e000 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.526+0000 7fa6c9bb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c409c700 con 0x7fa6c409e410 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 0x7fa6c4137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 0x7fa6c4137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55736/0 (socket says 172.21.15.32:55736) 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409e410 0x7fa6c41383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 -- 172.21.15.32:0/3836215037 learned_addr learned my addr 172.21.15.32:0/3836215037 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 -- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409e410 msgr2=0x7fa6c41383a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409e410 0x7fa6c41383a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 -- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 msgr2=0x7fa6c4137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 -- 172.21.15.32:0/3836215037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4009a70 con 0x7fa6c409d6f0 2023-09-30T22:27:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c3fff700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409e410 0x7fa6c41383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c37fe700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 0x7fa6c4137920 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400eed0 tx=0x7fa6b400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4010040 con 0x7fa6c409d6f0 2023-09-30T22:27:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b400bd00 con 0x7fa6c409d6f0 2023-09-30T22:27:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4014620 con 0x7fa6c409d6f0 2023-09-30T22:27:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac009550 con 0x7fa6c409d6f0 2023-09-30T22:27:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.527+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc0094a0 con 0x7fa6c409d6f0 2023-09-30T22:27:40.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.528+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6b4014780 con 0x7fa6c409d6f0 2023-09-30T22:27:40.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.528+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa6b403ff00 con 0x7fa6c409d6f0 2023-09-30T22:27:40.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.528+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c411a1a0 con 0x7fa6c409d6f0 2023-09-30T22:27:40.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.534+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6c411a1a0 con 0x7fa6c409d6f0 2023-09-30T22:27:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.621+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb4d40a3540 con 0x7fb4d406a770 2023-09-30T22:27:40.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.622+0000 7fb4c9ffb700 1 -- 172.21.15.32:0/366387288 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4d002b090 con 0x7fb4d406a770 2023-09-30T22:27:40.623 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:40.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 msgr2=0x7fb4d409add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d409add0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000e900 tx=0x7fb4d000ec10 comp rx=0 tx=0).stop 2023-09-30T22:27:40.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 shutdown_connections 2023-09-30T22:27:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4d4131b40 0x7fb4d41c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4d406a770 0x7fb4d409add0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 --2- 172.21.15.32:0/366387288 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4d406b080 0x7fb4d409b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 >> 172.21.15.32:0/366387288 conn(0x7fb4d4066f50 msgr2=0x7fb4d40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 shutdown_connections 2023-09-30T22:27:40.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.624+0000 7fb4db2e2700 1 -- 172.21.15.32:0/366387288 wait complete. 2023-09-30T22:27:40.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:40.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.740+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa6c4119480 con 0x7fa6c409d6f0 2023-09-30T22:27:40.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.740+0000 7fa6bbfff700 1 -- 172.21.15.32:0/3836215037 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6b402c090 con 0x7fa6c409d6f0 2023-09-30T22:27:40.741 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 msgr2=0x7fa6c4137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 0x7fa6c4137920 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400eed0 tx=0x7fa6b400b390 comp rx=0 tx=0).stop 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 shutdown_connections 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6c409e410 0x7fa6c41383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6c409d6f0 0x7fa6c4137920 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 --2- 172.21.15.32:0/3836215037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6c409e000 0x7fa6c4137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:40.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.742+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 >> 172.21.15.32:0/3836215037 conn(0x7fa6c4066f50 msgr2=0x7fa6c406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:40.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.743+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 shutdown_connections 2023-09-30T22:27:40.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:40.743+0000 7fa6c9bb5700 1 -- 172.21.15.32:0/3836215037 wait complete. 2023-09-30T22:27:40.757 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:41.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:41.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.542+0000 7f3ba796a700 1 Processor -- start 2023-09-30T22:27:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.542+0000 7f3ba796a700 1 -- start start 2023-09-30T22:27:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba796a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba00a0a30 0x7f3ba00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba796a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba0138d80 0x7f3ba013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba796a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba00a2a70 con 0x7f3ba009d630 2023-09-30T22:27:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba796a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00a01d0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba796a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba009e2b0 con 0x7f3ba0138d80 2023-09-30T22:27:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba5706700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba00a0a30 0x7f3ba00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba5f07700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba0138d80 0x7f3ba013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba4f05700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ba009d630 0x7f3ba00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48576/0 (socket says 172.21.15.32:48576) 2023-09-30T22:27:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.543+0000 7f3ba4f05700 1 -- 172.21.15.32:0/3558042174 learned_addr learned my addr 172.21.15.32:0/3558042174 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3b9ffff700 1 -- 172.21.15.32:0/3558042174 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975390162 0 0) 0x7f3ba00a2a70 con 0x7f3ba009d630 2023-09-30T22:27:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3ba5706700 1 -- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba0138d80 msgr2=0x7f3ba013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3ba5706700 1 --2- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba0138d80 0x7f3ba013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3ba5706700 1 -- 172.21.15.32:0/3558042174 >> v1:172.21.15.32:6789/0 conn(0x7f3ba009d630 legacy=0x7f3ba00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3ba5706700 1 -- 172.21.15.32:0/3558042174 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba009dc50 con 0x7f3ba00a0a30 2023-09-30T22:27:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3ba5f07700 1 --2- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba0138d80 0x7f3ba013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3ba5706700 1 --2- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba00a0a30 0x7f3ba00a0e00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009870 tx=0x7f3b90009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f5650c29a6b17f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3b9ffff700 1 -- 172.21.15.32:0/3558042174 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90011830 con 0x7f3ba00a0a30 2023-09-30T22:27:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3b9ffff700 1 -- 172.21.15.32:0/3558042174 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b90011990 con 0x7f3ba00a0a30 2023-09-30T22:27:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.544+0000 7f3b9ffff700 1 -- 172.21.15.32:0/3558042174 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90011d20 con 0x7f3ba00a0a30 2023-09-30T22:27:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 -- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba00a0a30 msgr2=0x7f3ba00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 --2- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba00a0a30 0x7f3ba00a0e00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009870 tx=0x7f3b90009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 -- 172.21.15.32:0/3558042174 shutdown_connections 2023-09-30T22:27:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 --2- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba0138d80 0x7f3ba013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 --2- 172.21.15.32:0/3558042174 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba00a0a30 0x7f3ba00a0e00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 -- 172.21.15.32:0/3558042174 >> 172.21.15.32:0/3558042174 conn(0x7f3ba00a3050 msgr2=0x7f3ba00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 -- 172.21.15.32:0/3558042174 shutdown_connections 2023-09-30T22:27:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.545+0000 7f3ba796a700 1 -- 172.21.15.32:0/3558042174 wait complete. 2023-09-30T22:27:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 Processor -- start 2023-09-30T22:27:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 -- start start 2023-09-30T22:27:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba009d630 0x7f3ba0133620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 0x7f3ba0133b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ba0138d80 0x7f3ba01340a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0136460 con 0x7f3ba0138d80 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0136dd0 con 0x7f3ba009d630 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba796a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00a4ab0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba4f05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 0x7f3ba0133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba5f07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ba0138d80 0x7f3ba01340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba4f05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 0x7f3ba0133b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57180/0 (socket says 172.21.15.32:57180) 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba4f05700 1 -- 172.21.15.32:0/4063385326 learned_addr learned my addr 172.21.15.32:0/4063385326 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.546+0000 7f3ba5f07700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ba0138d80 0x7f3ba01340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55748/0 (socket says 172.21.15.32:55748) 2023-09-30T22:27:41.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba5706700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba009d630 0x7f3ba0133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba4f05700 1 -- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba009d630 msgr2=0x7f3ba0133620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba4f05700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba009d630 0x7f3ba0133620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba4f05700 1 -- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ba0138d80 msgr2=0x7f3ba01340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba4f05700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ba0138d80 0x7f3ba01340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba4f05700 1 -- 172.21.15.32:0/4063385326 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98009a70 con 0x7f3ba00a0a30 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba4f05700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 0x7f3ba0133b60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b94001eb0 tx=0x7f3b9400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94004c80 con 0x7f3ba00a0a30 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94004de0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b900094a0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.547+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94007310 con 0x7f3ba00a0a30 2023-09-30T22:27:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.548+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba01349f0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.548+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b94007470 con 0x7f3ba00a0a30 2023-09-30T22:27:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.549+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3b9403f2d0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.549+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba00747f0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.553+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b940123b0 con 0x7f3ba00a0a30 2023-09-30T22:27:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.580+0000 7f3359e89700 1 Processor -- start 2023-09-30T22:27:41.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.580+0000 7f3359e89700 1 -- start start 2023-09-30T22:27:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3359e89700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 0x7f33541337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3359e89700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 0x7f335412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3359e89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354134260 0x7f33541304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3359e89700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335406b990 con 0x7f3354133e30 2023-09-30T22:27:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3359e89700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33540a0590 con 0x7f3354133420 2023-09-30T22:27:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3359e89700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335409fb80 con 0x7f3354134260 2023-09-30T22:27:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3352ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 0x7f335412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f33537fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 0x7f33541337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3352ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 0x7f335412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55752/0 (socket says 172.21.15.32:55752) 2023-09-30T22:27:41.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3352ffd700 1 -- 172.21.15.32:0/2846907514 learned_addr learned my addr 172.21.15.32:0/2846907514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f33537fe700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 0x7f33541337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33564/0 (socket says 172.21.15.32:33564) 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.581+0000 7f3353fff700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354134260 0x7f33541304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33537fe700 1 -- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354134260 msgr2=0x7f33541304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33537fe700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354134260 0x7f33541304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33537fe700 1 -- 172.21.15.32:0/2846907514 >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 msgr2=0x7f335412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33537fe700 1 --2- 172.21.15.32:0/2846907514 >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 0x7f335412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33537fe700 1 -- 172.21.15.32:0/2846907514 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33540a0030 con 0x7f3354133420 2023-09-30T22:27:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f3352ffd700 1 --2- 172.21.15.32:0/2846907514 >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 0x7f335412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f3353fff700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354134260 0x7f33541304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33537fe700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 0x7f33541337f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f334c009d90 tx=0x7f334c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308b82b86c38e7ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33527fc700 1 -- 172.21.15.32:0/2846907514 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00e070 con 0x7f3354133420 2023-09-30T22:27:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33527fc700 1 -- 172.21.15.32:0/2846907514 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c00bb20 con 0x7f3354133420 2023-09-30T22:27:41.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.582+0000 7f33527fc700 1 -- 172.21.15.32:0/2846907514 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c012430 con 0x7f3354133420 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 -- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 msgr2=0x7f33541337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 0x7f33541337f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f334c009d90 tx=0x7f334c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 -- 172.21.15.32:0/2846907514 shutdown_connections 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133420 0x7f33541337f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 --2- 172.21.15.32:0/2846907514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354134260 0x7f33541304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 --2- 172.21.15.32:0/2846907514 >> v2:172.21.15.32:3300/0 conn(0x7f3354133e30 0x7f335412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 -- 172.21.15.32:0/2846907514 >> 172.21.15.32:0/2846907514 conn(0x7f33540a4210 msgr2=0x7f33540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 -- 172.21.15.32:0/2846907514 shutdown_connections 2023-09-30T22:27:41.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.583+0000 7f3359e89700 1 -- 172.21.15.32:0/2846907514 wait complete. 2023-09-30T22:27:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.584+0000 7f3359e89700 1 Processor -- start 2023-09-30T22:27:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.584+0000 7f3359e89700 1 -- start start 2023-09-30T22:27:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3359e89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354133420 0x7f33540709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3359e89700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 0x7f3354070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3359e89700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 0x7f3354071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3359e89700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335411ff90 con 0x7f3354134260 2023-09-30T22:27:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3359e89700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3354120830 con 0x7f3354133e30 2023-09-30T22:27:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3359e89700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335411fbf0 con 0x7f3354133420 2023-09-30T22:27:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 0x7f3354071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3352ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 0x7f3354070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 0x7f3354071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55766/0 (socket says 172.21.15.32:55766) 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 -- 172.21.15.32:0/2332583453 learned_addr learned my addr 172.21.15.32:0/2332583453 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3352ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 0x7f3354070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33576/0 (socket says 172.21.15.32:33576) 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f33537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354133420 0x7f33540709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 -- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354133420 msgr2=0x7f33540709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354133420 0x7f33540709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 -- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 msgr2=0x7f3354070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 0x7f3354070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 -- 172.21.15.32:0/2332583453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009550 con 0x7f3354134260 2023-09-30T22:27:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3352ffd700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 0x7f3354070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.585+0000 7f3353fff700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 0x7f3354071420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f334400eb30 tx=0x7f334400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.586+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344009780 con 0x7f3354134260 2023-09-30T22:27:41.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.586+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344009e60 con 0x7f3354134260 2023-09-30T22:27:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.586+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33440094a0 con 0x7f3354134260 2023-09-30T22:27:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.586+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400ca50 con 0x7f3354134260 2023-09-30T22:27:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.586+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c009a70 con 0x7f3354134260 2023-09-30T22:27:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.586+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f334400cbb0 con 0x7f3354134260 2023-09-30T22:27:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.587+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f334403f9f0 con 0x7f3354134260 2023-09-30T22:27:41.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.587+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335411fdb0 con 0x7f3354134260 2023-09-30T22:27:41.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.592+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3344017080 con 0x7f3354134260 2023-09-30T22:27:41.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:41.758 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.763+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3ba0072980 con 0x7f3ba00a0a30 2023-09-30T22:27:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.763+0000 7f3b9dffb700 1 -- 172.21.15.32:0/4063385326 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b9401b340 con 0x7f3ba00a0a30 2023-09-30T22:27:41.763 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 msgr2=0x7f3ba0133b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 0x7f3ba0133b60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b94001eb0 tx=0x7f3b9400c800 comp rx=0 tx=0).stop 2023-09-30T22:27:41.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 shutdown_connections 2023-09-30T22:27:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ba0138d80 0x7f3ba01340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ba009d630 0x7f3ba0133620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 --2- 172.21.15.32:0/4063385326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ba00a0a30 0x7f3ba0133b60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 >> 172.21.15.32:0/4063385326 conn(0x7f3ba00a3050 msgr2=0x7f3ba009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 shutdown_connections 2023-09-30T22:27:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.767+0000 7f3ba796a700 1 -- 172.21.15.32:0/4063385326 wait complete. 2023-09-30T22:27:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.782+0000 7fb1a09b1700 1 Processor -- start 2023-09-30T22:27:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.782+0000 7fb1a09b1700 1 -- start start 2023-09-30T22:27:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb1a09b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09d6f0 0x7fb19c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb1a09b1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09e000 0x7fb19c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb1a09b1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb19c09e410 0x7fb19c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb1a09b1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19c06b460 con 0x7fb19c09e410 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb1a09b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c09b120 con 0x7fb19c09d6f0 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb1a09b1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c09a7e0 con 0x7fb19c09e000 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.783+0000 7fb19ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb19c09e410 0x7fb19c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb19c09e410 0x7fb19c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55768/0 (socket says 172.21.15.32:55768) 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 -- 172.21.15.32:0/3288777311 learned_addr learned my addr 172.21.15.32:0/3288777311 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19a59c700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09d6f0 0x7fb19c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb199d9b700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09e000 0x7fb19c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 -- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09e000 msgr2=0x7fb19c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09e000 0x7fb19c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 -- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09d6f0 msgr2=0x7fb19c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09d6f0 0x7fb19c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 -- 172.21.15.32:0/3288777311 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c09a500 con 0x7fb19c09e410 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19a59c700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09d6f0 0x7fb19c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19ad9d700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e410 0x7fb19c133f30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb194009e50 tx=0x7fb19400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c495a9cde92a9809 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.784+0000 7fb19959a700 1 -- 172.21.15.32:0/3288777311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400e070 con 0x7fb19c09e410 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb19959a700 1 -- 172.21.15.32:0/3288777311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19400b910 con 0x7fb19c09e410 2023-09-30T22:27:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb19959a700 1 -- 172.21.15.32:0/3288777311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400bcc0 con 0x7fb19c09e410 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 -- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e410 msgr2=0x7fb19c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e410 0x7fb19c133f30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb194009e50 tx=0x7fb19400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 -- 172.21.15.32:0/3288777311 shutdown_connections 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09d6f0 0x7fb19c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e410 0x7fb19c133f30 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/3288777311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09e000 0x7fb19c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 -- 172.21.15.32:0/3288777311 >> 172.21.15.32:0/3288777311 conn(0x7fb19c066f50 msgr2=0x7fb19c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 -- 172.21.15.32:0/3288777311 shutdown_connections 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.785+0000 7fb1a09b1700 1 -- 172.21.15.32:0/3288777311 wait complete. 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 Processor -- start 2023-09-30T22:27:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 -- start start 2023-09-30T22:27:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09d6f0 0x7fb19c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 0x7fb19c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09e410 0x7fb19c1a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1a62f0 con 0x7fb19c09e000 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1a82e0 con 0x7fb19c09e410 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb1a09b1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0a1e00 con 0x7fb19c09d6f0 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.787+0000 7fb199d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 0x7fb19c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 0x7fb19c1a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55778/0 (socket says 172.21.15.32:55778) 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 -- 172.21.15.32:0/2517362070 learned_addr learned my addr 172.21.15.32:0/2517362070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb19a59c700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09d6f0 0x7fb19c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb19ad9d700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09e410 0x7fb19c1a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 -- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09d6f0 msgr2=0x7fb19c1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09d6f0 0x7fb19c1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 -- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09e410 msgr2=0x7fb19c1a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09e410 0x7fb19c1a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 -- 172.21.15.32:0/2517362070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0094a0 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb199d9b700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 0x7fb19c1a4c70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e7e0 tx=0x7fb19000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190013610 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb190013770 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb194009b30 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.788+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190013ae0 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.789+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1a7c60 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.789+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb190013c40 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.790+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c074ea0 con 0x7fb19c09e000 2023-09-30T22:27:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.790+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb19003f300 con 0x7fb19c09e000 2023-09-30T22:27:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.795+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb190012370 con 0x7fb19c09e000 2023-09-30T22:27:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.805+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3354074ea0 con 0x7f3354134260 2023-09-30T22:27:41.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.805+0000 7f334bfff700 1 -- 172.21.15.32:0/2332583453 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3344021020 con 0x7f3354134260 2023-09-30T22:27:41.805 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 msgr2=0x7f3354071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 0x7f3354071420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f334400eb30 tx=0x7f334400ee40 comp rx=0 tx=0).stop 2023-09-30T22:27:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 shutdown_connections 2023-09-30T22:27:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3354134260 0x7f3354071420 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3354133420 0x7f33540709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 --2- 172.21.15.32:0/2332583453 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3354133e30 0x7f3354070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 >> 172.21.15.32:0/2332583453 conn(0x7f33540a4210 msgr2=0x7f33540a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:41.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 shutdown_connections 2023-09-30T22:27:41.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.808+0000 7f3359e89700 1 -- 172.21.15.32:0/2332583453 wait complete. 2023-09-30T22:27:41.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.901+0000 7f71b2bcf700 1 Processor -- start 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b2bcf700 1 -- start start 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b2bcf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b2bcf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71ac134170 0x7f71ac13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b2bcf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac09edd0 con 0x7f71ac0a1340 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b2bcf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac09e1f0 con 0x7f71ac134170 2023-09-30T22:27:41.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b2bcf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac09dc50 con 0x7f71ac0a0a30 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b096b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b116c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71ac134170 0x7f71ac13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b096b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33608/0 (socket says 172.21.15.32:33608) 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.902+0000 7f71b096b700 1 -- 172.21.15.32:0/1772188514 learned_addr learned my addr 172.21.15.32:0/1772188514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71b116c700 1 -- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 msgr2=0x7f71ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71b116c700 1 --2- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71b116c700 1 -- 172.21.15.32:0/1772188514 >> v1:172.21.15.32:6789/0 conn(0x7f71ac0a1340 legacy=0x7f71ac133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71b116c700 1 -- 172.21.15.32:0/1772188514 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0a1fe0 con 0x7f71ac134170 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71b096b700 1 --2- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71b116c700 1 --2- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac134170 0x7f71ac13bd30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009d90 tx=0x7f71a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0241e11e5d6a912 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71ab7fe700 1 -- 172.21.15.32:0/1772188514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a400e070 con 0x7f71ac134170 2023-09-30T22:27:41.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71ab7fe700 1 -- 172.21.15.32:0/1772188514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a400bb20 con 0x7f71ac134170 2023-09-30T22:27:41.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.903+0000 7f71ab7fe700 1 -- 172.21.15.32:0/1772188514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4012430 con 0x7f71ac134170 2023-09-30T22:27:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 -- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac134170 msgr2=0x7f71ac13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac134170 0x7f71ac13bd30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009d90 tx=0x7f71a400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 -- 172.21.15.32:0/1772188514 shutdown_connections 2023-09-30T22:27:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac134170 0x7f71ac13bd30 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/1772188514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 -- 172.21.15.32:0/1772188514 >> 172.21.15.32:0/1772188514 conn(0x7f71ac0a6550 msgr2=0x7f71ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 -- 172.21.15.32:0/1772188514 shutdown_connections 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.904+0000 7f71b2bcf700 1 -- 172.21.15.32:0/1772188514 wait complete. 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 Processor -- start 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 -- start start 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac11e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac0a1340 0x7f71ac11ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71ac134170 0x7f71ac11f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac122190 con 0x7f71ac0a1340 2023-09-30T22:27:41.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac121bf0 con 0x7f71ac0a0a30 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b2bcf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac120280 con 0x7f71ac134170 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b096b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac11e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac0a1340 0x7f71ac11ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.905+0000 7f71b096b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac11e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33622/0 (socket says 172.21.15.32:33622) 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 -- 172.21.15.32:0/2616886585 learned_addr learned my addr 172.21.15.32:0/2616886585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b116c700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71ac134170 0x7f71ac11f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 -- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71ac134170 msgr2=0x7f71ac11f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71ac134170 0x7f71ac11f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 -- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac0a1340 msgr2=0x7f71ac11ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:41.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac0a1340 0x7f71ac11ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 -- 172.21.15.32:0/2616886585 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c009550 con 0x7f71ac0a0a30 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71abfff700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac0a1340 0x7f71ac11ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b116c700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71ac134170 0x7f71ac11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b096b700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac11e700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f719c00ebe0 tx=0x7f719c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00c850 con 0x7f71ac0a0a30 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c00c9b0 con 0x7f71ac0a0a30 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a4009a70 con 0x7f71ac0a0a30 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00cd40 con 0x7f71ac0a0a30 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.906+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac120a30 con 0x7f71ac0a0a30 2023-09-30T22:27:41.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.907+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f719c013470 con 0x7f71ac0a0a30 2023-09-30T22:27:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.907+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac074ea0 con 0x7f71ac0a0a30 2023-09-30T22:27:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.908+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f719c03fc30 con 0x7f71ac0a0a30 2023-09-30T22:27:41.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:41.912+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f719c046080 con 0x7f71ac0a0a30 2023-09-30T22:27:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.003+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb19c1a8140 con 0x7fb19c09e000 2023-09-30T22:27:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.003+0000 7fb18affd700 1 -- 172.21.15.32:0/2517362070 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb19c1a8140 con 0x7fb19c09e000 2023-09-30T22:27:42.003 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 msgr2=0x7fb19c1a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 0x7fb19c1a4c70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb19000e7e0 tx=0x7fb19000eaf0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 shutdown_connections 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb19c09e410 0x7fb19c1a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb19c09d6f0 0x7fb19c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 --2- 172.21.15.32:0/2517362070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb19c09e000 0x7fb19c1a4c70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 >> 172.21.15.32:0/2517362070 conn(0x7fb19c066f50 msgr2=0x7fb19c1bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 shutdown_connections 2023-09-30T22:27:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.005+0000 7fb1a09b1700 1 -- 172.21.15.32:0/2517362070 wait complete. 2023-09-30T22:27:42.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.122+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f71ac0a28e0 con 0x7f71ac0a0a30 2023-09-30T22:27:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.122+0000 7f71a97fa700 1 -- 172.21.15.32:0/2616886585 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f719c013c40 con 0x7f71ac0a0a30 2023-09-30T22:27:42.122 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.124+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 msgr2=0x7f71ac11e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.124+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac11e700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f719c00ebe0 tx=0x7f719c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 shutdown_connections 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71ac134170 0x7f71ac11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71ac0a0a30 0x7f71ac11e700 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 --2- 172.21.15.32:0/2616886585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71ac0a1340 0x7f71ac11ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 >> 172.21.15.32:0/2616886585 conn(0x7f71ac0a6550 msgr2=0x7f71ac09c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 shutdown_connections 2023-09-30T22:27:42.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.125+0000 7f71b2bcf700 1 -- 172.21.15.32:0/2616886585 wait complete. 2023-09-30T22:27:42.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:42.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:42.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.935+0000 7feaa3009700 1 Processor -- start 2023-09-30T22:27:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.935+0000 7feaa3009700 1 -- start start 2023-09-30T22:27:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa3009700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 0x7fea9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa3009700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c138d80 0x7fea9c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa3009700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c0a2a70 con 0x7fea9c09d630 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa3009700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c0a01d0 con 0x7fea9c0a0a30 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa3009700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c09e2b0 con 0x7fea9c138d80 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa0da5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 0x7fea9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa15a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c138d80 0x7fea9c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa0da5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 0x7fea9c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55806/0 (socket says 172.21.15.32:55806) 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7fea9bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea9c09d630 0x7fea9c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48602/0 (socket says 172.21.15.32:48602) 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa0da5700 1 -- 172.21.15.32:0/1344398776 learned_addr learned my addr 172.21.15.32:0/1344398776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa15a6700 1 -- 172.21.15.32:0/1344398776 >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 msgr2=0x7fea9c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa15a6700 1 --2- 172.21.15.32:0/1344398776 >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 0x7fea9c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa15a6700 1 -- 172.21.15.32:0/1344398776 >> v1:172.21.15.32:6789/0 conn(0x7fea9c09d630 legacy=0x7fea9c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7fea9b7fe700 1 -- 172.21.15.32:0/1344398776 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989523269 0 0) 0x7fea9c0a2a70 con 0x7fea9c09d630 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa15a6700 1 -- 172.21.15.32:0/1344398776 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c09dc50 con 0x7fea9c138d80 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa0da5700 1 --2- 172.21.15.32:0/1344398776 >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 0x7fea9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.936+0000 7feaa15a6700 1 --2- 172.21.15.32:0/1344398776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c138d80 0x7fea9c13b160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea94009d90 tx=0x7fea9400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89fa749d8c14a71a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7fea9b7fe700 1 -- 172.21.15.32:0/1344398776 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400e070 con 0x7fea9c138d80 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7fea9b7fe700 1 -- 172.21.15.32:0/1344398776 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9400bb20 con 0x7fea9c138d80 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7fea9b7fe700 1 -- 172.21.15.32:0/1344398776 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94012430 con 0x7fea9c138d80 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 -- 172.21.15.32:0/1344398776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c138d80 msgr2=0x7fea9c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 --2- 172.21.15.32:0/1344398776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c138d80 0x7fea9c13b160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea94009d90 tx=0x7fea9400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 -- 172.21.15.32:0/1344398776 shutdown_connections 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 --2- 172.21.15.32:0/1344398776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c138d80 0x7fea9c13b160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 --2- 172.21.15.32:0/1344398776 >> v2:172.21.15.32:3300/0 conn(0x7fea9c0a0a30 0x7fea9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 -- 172.21.15.32:0/1344398776 >> 172.21.15.32:0/1344398776 conn(0x7fea9c0a3050 msgr2=0x7fea9c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 -- 172.21.15.32:0/1344398776 shutdown_connections 2023-09-30T22:27:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.937+0000 7feaa3009700 1 -- 172.21.15.32:0/1344398776 wait complete. 2023-09-30T22:27:42.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.938+0000 7feaa3009700 1 Processor -- start 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.938+0000 7feaa3009700 1 -- start start 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa3009700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 0x7fea9c133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa3009700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea9c0a0a30 0x7fea9c133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa3009700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea9c138d80 0x7fea9c1340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa3009700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c137170 con 0x7fea9c138d80 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa3009700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c136060 con 0x7fea9c09d630 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa3009700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c136520 con 0x7fea9c0a0a30 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa0da5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 0x7fea9c133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa0da5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 0x7fea9c133650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33636/0 (socket says 172.21.15.32:33636) 2023-09-30T22:27:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa0da5700 1 -- 172.21.15.32:0/583084755 learned_addr learned my addr 172.21.15.32:0/583084755 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7feaa15a6700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea9c138d80 0x7fea9c1340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.939+0000 7fea9bfff700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea9c0a0a30 0x7fea9c133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa0da5700 1 -- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea9c0a0a30 msgr2=0x7fea9c133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa0da5700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea9c0a0a30 0x7fea9c133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa0da5700 1 -- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea9c138d80 msgr2=0x7fea9c1340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa0da5700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea9c138d80 0x7fea9c1340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa0da5700 1 -- 172.21.15.32:0/583084755 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90009550 con 0x7fea9c09d630 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa0da5700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 0x7fea9c133650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ebe0 tx=0x7fea9000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9000c850 con 0x7fea9c09d630 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9000c9b0 con 0x7fea9c09d630 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea94009a70 con 0x7fea9c09d630 2023-09-30T22:27:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9000cd40 con 0x7fea9c09d630 2023-09-30T22:27:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.940+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c136bd0 con 0x7fea9c09d630 2023-09-30T22:27:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.941+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea90013470 con 0x7fea9c09d630 2023-09-30T22:27:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.941+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea9003fbe0 con 0x7fea9c09d630 2023-09-30T22:27:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.941+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c0747f0 con 0x7fea9c09d630 2023-09-30T22:27:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.946+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea9001b3b0 con 0x7fea9c09d630 2023-09-30T22:27:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 Processor -- start 2023-09-30T22:27:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 -- start start 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400a2a70 con 0x7f704009d630 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70400a01d0 con 0x7f70400a0a30 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f7045a77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704009e2b0 con 0x7f7040138d80 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.977+0000 7f703f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f704009d630 0x7f70400a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48616/0 (socket says 172.21.15.32:48616) 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703effd700 1 -- 172.21.15.32:0/3898682300 learned_addr learned my addr 172.21.15.32:0/3898682300 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703ffff700 1 --2- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703f7fe700 1 -- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 msgr2=0x7f704013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703f7fe700 1 --2- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703f7fe700 1 -- 172.21.15.32:0/3898682300 >> v1:172.21.15.32:6789/0 conn(0x7f704009d630 legacy=0x7f70400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703f7fe700 1 -- 172.21.15.32:0/3898682300 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704009dc50 con 0x7f70400a0a30 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703e7fc700 1 -- 172.21.15.32:0/3898682300 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611648810 0 0) 0x7f70400a2a70 con 0x7f704009d630 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703f7fe700 1 --2- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f70400a0e00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7030009840 tx=0x7f7030009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23213269a4653654 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703e7fc700 1 -- 172.21.15.32:0/3898682300 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030011830 con 0x7f70400a0a30 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703e7fc700 1 -- 172.21.15.32:0/3898682300 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030011990 con 0x7f70400a0a30 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.978+0000 7f703e7fc700 1 -- 172.21.15.32:0/3898682300 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030011d20 con 0x7f70400a0a30 2023-09-30T22:27:42.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 -- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 msgr2=0x7f70400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 --2- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f70400a0e00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7030009840 tx=0x7f7030009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 -- 172.21.15.32:0/3898682300 shutdown_connections 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 --2- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f704013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 --2- 172.21.15.32:0/3898682300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f70400a0e00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 -- 172.21.15.32:0/3898682300 >> 172.21.15.32:0/3898682300 conn(0x7f70400a3050 msgr2=0x7f70400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 -- 172.21.15.32:0/3898682300 shutdown_connections 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.979+0000 7f7045a77700 1 -- 172.21.15.32:0/3898682300 wait complete. 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.980+0000 7f7045a77700 1 Processor -- start 2023-09-30T22:27:42.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.980+0000 7f7045a77700 1 -- start start 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f7045a77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 0x7f7040133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f7045a77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f7040133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f7045a77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f7040134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f7045a77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040137d00 con 0x7f70400a0a30 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f7045a77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70400a4b00 con 0x7f7040138d80 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f7045a77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70400a0410 con 0x7f704009d630 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f7040134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 0x7f7040133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f7040134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33656/0 (socket says 172.21.15.32:33656) 2023-09-30T22:27:42.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 0x7f7040133710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57250/0 (socket says 172.21.15.32:57250) 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 -- 172.21.15.32:0/4124071057 learned_addr learned my addr 172.21.15.32:0/4124071057 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703effd700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f7040133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 -- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 msgr2=0x7f7040133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 0x7f7040133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 -- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 msgr2=0x7f7040133c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f7040133c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 -- 172.21.15.32:0/4124071057 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70300094a0 con 0x7f7040138d80 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703f7fe700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 0x7f7040133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.981+0000 7f703ffff700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f7040134190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f703800efc0 tx=0x7f703800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:42.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.982+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703800cc90 con 0x7f7040138d80 2023-09-30T22:27:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.982+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703800f040 con 0x7f7040138d80 2023-09-30T22:27:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.982+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70401364d0 con 0x7f7040138d80 2023-09-30T22:27:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.982+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70380146c0 con 0x7f7040138d80 2023-09-30T22:27:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.982+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7040137120 con 0x7f7040138d80 2023-09-30T22:27:42.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.983+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7038014820 con 0x7f7040138d80 2023-09-30T22:27:42.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.983+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70400747f0 con 0x7f7040138d80 2023-09-30T22:27:42.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.983+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7038040610 con 0x7f7040138d80 2023-09-30T22:27:42.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:42.992+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f703801d400 con 0x7f7040138d80 2023-09-30T22:27:43.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:43.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.166+0000 7f7c25bd9700 1 Processor -- start 2023-09-30T22:27:43.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.166+0000 7f7c25bd9700 1 -- start start 2023-09-30T22:27:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.167+0000 7f7c25bd9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c2009d6f0 0x7f7c2009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.167+0000 7f7c25bd9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c20134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.167+0000 7f7c25bd9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2009b120 con 0x7f7c2009e000 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.167+0000 7f7c25bd9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2006b460 con 0x7f7c2009d6f0 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.167+0000 7f7c25bd9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2009a7e0 con 0x7f7c2009e440 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.167+0000 7f7c1f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c2009d6f0 0x7f7c2009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c2009d6f0 0x7f7c2009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55854/0 (socket says 172.21.15.32:55854) 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c2009e000 0x7f7c201339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48618/0 (socket says 172.21.15.32:48618) 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1244663885 learned_addr learned my addr 172.21.15.32:0/1244663885 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:43.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1ffff700 1 --2- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c20134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 msgr2=0x7f7c20134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 --2- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c20134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1244663885 >> v1:172.21.15.32:6789/0 conn(0x7f7c2009e000 legacy=0x7f7c201339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1244663885 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2009a500 con 0x7f7c2009d6f0 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1f7fe700 1 --2- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009d6f0 0x7f7c2009dac0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009920 tx=0x7f7c10009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b6d560640b68c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.168+0000 7f7c1e7fc700 1 -- 172.21.15.32:0/1244663885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10004230 con 0x7f7c2009d6f0 2023-09-30T22:27:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.169+0000 7f7c1e7fc700 1 -- 172.21.15.32:0/1244663885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10004390 con 0x7f7c2009d6f0 2023-09-30T22:27:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.169+0000 7f7c1e7fc700 1 -- 172.21.15.32:0/1244663885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c100036a0 con 0x7f7c2009d6f0 2023-09-30T22:27:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.169+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea9c072980 con 0x7fea9c09d630 2023-09-30T22:27:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009d6f0 msgr2=0x7f7c2009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009d6f0 0x7f7c2009dac0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009920 tx=0x7f7c10009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:43.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7fea997fa700 1 -- 172.21.15.32:0/583084755 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea90011b00 con 0x7fea9c09d630 2023-09-30T22:27:43.172 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1244663885 shutdown_connections 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c20134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1244663885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009d6f0 0x7f7c2009dac0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1244663885 >> 172.21.15.32:0/1244663885 conn(0x7f7c20066f50 msgr2=0x7f7c2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1244663885 shutdown_connections 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.170+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1244663885 wait complete. 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.171+0000 7f7c25bd9700 1 Processor -- start 2023-09-30T22:27:43.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.172+0000 7f7c25bd9700 1 -- start start 2023-09-30T22:27:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 msgr2=0x7fea9c133650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 0x7fea9c133650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ebe0 tx=0x7fea9000eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c25bd9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 0x7f7c201cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c25bd9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009e000 0x7f7c201cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c25bd9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c201194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c25bd9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2009c710 con 0x7f7c2009e000 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c25bd9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2011ccf0 con 0x7f7c2009d6f0 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c25bd9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2011b350 con 0x7f7c2009e440 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 shutdown_connections 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea9c138d80 0x7fea9c1340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea9c09d630 0x7fea9c133650 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 --2- 172.21.15.32:0/583084755 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea9c0a0a30 0x7fea9c133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 >> 172.21.15.32:0/583084755 conn(0x7fea9c0a3050 msgr2=0x7fea9c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 0x7f7c201cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 0x7f7c201cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33670/0 (socket says 172.21.15.32:33670) 2023-09-30T22:27:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1155965194 learned_addr learned my addr 172.21.15.32:0/1155965194 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1effd700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009e000 0x7f7c201cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1ffff700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c201194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 shutdown_connections 2023-09-30T22:27:43.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7feaa3009700 1 -- 172.21.15.32:0/583084755 wait complete. 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 msgr2=0x7f7c201194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c201194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009e000 msgr2=0x7f7c201cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009e000 0x7f7c201cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.173+0000 7f7c1f7fe700 1 -- 172.21.15.32:0/1155965194 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10009550 con 0x7f7c2009d6f0 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c1effd700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009e000 0x7f7c201cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c1f7fe700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 0x7f7c201cccc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000a010 tx=0x7f7c10004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:43.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10003c10 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1001f070 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c201ae790 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2011cfd0 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.174+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011630 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.175+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c10021030 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.175+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c10040a60 con 0x7f7c2009d6f0 2023-09-30T22:27:43.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.175+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c000052e0 con 0x7f7c2009d6f0 2023-09-30T22:27:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.181+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c10023790 con 0x7f7c2009d6f0 2023-09-30T22:27:43.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.214+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7040072980 con 0x7f7040138d80 2023-09-30T22:27:43.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.215+0000 7f7027fff700 1 -- 172.21.15.32:0/4124071057 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7038012340 con 0x7f7040138d80 2023-09-30T22:27:43.215 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 msgr2=0x7f7040134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f7040134190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f703800efc0 tx=0x7f703800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 shutdown_connections 2023-09-30T22:27:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7040138d80 0x7f7040134190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704009d630 0x7f7040133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 --2- 172.21.15.32:0/4124071057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70400a0a30 0x7f7040133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 >> 172.21.15.32:0/4124071057 conn(0x7f70400a3050 msgr2=0x7f704013ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 shutdown_connections 2023-09-30T22:27:43.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.217+0000 7f7045a77700 1 -- 172.21.15.32:0/4124071057 wait complete. 2023-09-30T22:27:43.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.284+0000 7fd13b7f0700 1 Processor -- start 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.284+0000 7fd13b7f0700 1 -- start start 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd13b7f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 0x7fd134136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd13b7f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd134136fe0 0x7fd1341393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd13b7f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd134070940 con 0x7fd13409e930 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd13b7f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13406fd60 con 0x7fd13409f410 2023-09-30T22:27:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd13b7f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd13406fa80 con 0x7fd134136fe0 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd138d8b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 0x7fd134136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd139d8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd134136fe0 0x7fd1341393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd138d8b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 0x7fd134136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55876/0 (socket says 172.21.15.32:55876) 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd138d8b700 1 -- 172.21.15.32:0/4052559537 learned_addr learned my addr 172.21.15.32:0/4052559537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd13958c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd13409e930 0x7fd13409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48630/0 (socket says 172.21.15.32:48630) 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd139d8d700 1 -- 172.21.15.32:0/4052559537 >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 msgr2=0x7fd134136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd139d8d700 1 --2- 172.21.15.32:0/4052559537 >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 0x7fd134136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd139d8d700 1 -- 172.21.15.32:0/4052559537 >> v1:172.21.15.32:6789/0 conn(0x7fd13409e930 legacy=0x7fd13409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd12bfff700 1 -- 172.21.15.32:0/4052559537 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177583780 0 0) 0x7fd134070940 con 0x7fd13409e930 2023-09-30T22:27:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd139d8d700 1 -- 172.21.15.32:0/4052559537 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13409fb60 con 0x7fd134136fe0 2023-09-30T22:27:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.285+0000 7fd138d8b700 1 --2- 172.21.15.32:0/4052559537 >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 0x7fd134136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd139d8d700 1 --2- 172.21.15.32:0/4052559537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd134136fe0 0x7fd1341393c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009d90 tx=0x7fd12c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345818afb619ebde server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd12bfff700 1 -- 172.21.15.32:0/4052559537 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c00e070 con 0x7fd134136fe0 2023-09-30T22:27:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd12bfff700 1 -- 172.21.15.32:0/4052559537 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c00bb20 con 0x7fd134136fe0 2023-09-30T22:27:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd12bfff700 1 -- 172.21.15.32:0/4052559537 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c012430 con 0x7fd134136fe0 2023-09-30T22:27:43.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd13b7f0700 1 -- 172.21.15.32:0/4052559537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd134136fe0 msgr2=0x7fd1341393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/4052559537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd134136fe0 0x7fd1341393c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009d90 tx=0x7fd12c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd13b7f0700 1 -- 172.21.15.32:0/4052559537 shutdown_connections 2023-09-30T22:27:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/4052559537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd134136fe0 0x7fd1341393c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.286+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/4052559537 >> v2:172.21.15.32:3300/0 conn(0x7fd13409f410 0x7fd134136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.287+0000 7fd13b7f0700 1 -- 172.21.15.32:0/4052559537 >> 172.21.15.32:0/4052559537 conn(0x7fd1340a4210 msgr2=0x7fd1340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:43.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.287+0000 7fd13b7f0700 1 -- 172.21.15.32:0/4052559537 shutdown_connections 2023-09-30T22:27:43.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.287+0000 7fd13b7f0700 1 -- 172.21.15.32:0/4052559537 wait complete. 2023-09-30T22:27:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 Processor -- start 2023-09-30T22:27:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 -- start start 2023-09-30T22:27:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13409e930 0x7fd13406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd13409f410 0x7fd13406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd134136fe0 0x7fd13406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1341240d0 con 0x7fd13409e930 2023-09-30T22:27:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd134134190 con 0x7fd134136fe0 2023-09-30T22:27:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13b7f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340a3260 con 0x7fd13409f410 2023-09-30T22:27:43.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd138d8b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd13409f410 0x7fd13406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd13958c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13409e930 0x7fd13406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd138d8b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd13409f410 0x7fd13406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57298/0 (socket says 172.21.15.32:57298) 2023-09-30T22:27:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.288+0000 7fd138d8b700 1 -- 172.21.15.32:0/3680124860 learned_addr learned my addr 172.21.15.32:0/3680124860 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13958c700 1 -- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd13409f410 msgr2=0x7fd13406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13958c700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd13409f410 0x7fd13406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13958c700 1 -- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd134136fe0 msgr2=0x7fd13406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13958c700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd134136fe0 0x7fd13406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13958c700 1 -- 172.21.15.32:0/3680124860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124009550 con 0x7fd13409e930 2023-09-30T22:27:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13958c700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13409e930 0x7fd13406b8b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c470 tx=0x7fd13000c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130004c80 con 0x7fd13409e930 2023-09-30T22:27:43.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd130004de0 con 0x7fd13409e930 2023-09-30T22:27:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c009a70 con 0x7fd13409e930 2023-09-30T22:27:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130007310 con 0x7fd13409e930 2023-09-30T22:27:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.289+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd134123b30 con 0x7fd13409e930 2023-09-30T22:27:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.290+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd130007470 con 0x7fd13409e930 2023-09-30T22:27:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.290+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd13003f3d0 con 0x7fd13409e930 2023-09-30T22:27:43.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.290+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13406d4c0 con 0x7fd13409e930 2023-09-30T22:27:43.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.295+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd13406d4c0 con 0x7fd13409e930 2023-09-30T22:27:43.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.393+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c00005590 con 0x7f7c2009d6f0 2023-09-30T22:27:43.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.393+0000 7f7c07fff700 1 -- 172.21.15.32:0/1155965194 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c100233d0 con 0x7f7c2009d6f0 2023-09-30T22:27:43.393 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.395+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 msgr2=0x7f7c201cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.395+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 0x7f7c201cccc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000a010 tx=0x7f7c10004750 comp rx=0 tx=0).stop 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 shutdown_connections 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c2009e440 0x7f7c201194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c2009d6f0 0x7f7c201cccc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 --2- 172.21.15.32:0/1155965194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c2009e000 0x7f7c201cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 >> 172.21.15.32:0/1155965194 conn(0x7f7c20066f50 msgr2=0x7f7c200a2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 shutdown_connections 2023-09-30T22:27:43.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.396+0000 7f7c25bd9700 1 -- 172.21.15.32:0/1155965194 wait complete. 2023-09-30T22:27:43.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.503+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd13406d4c0 con 0x7fd13409e930 2023-09-30T22:27:43.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.503+0000 7fd129ffb700 1 -- 172.21.15.32:0/3680124860 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd13406d4c0 con 0x7fd13409e930 2023-09-30T22:27:43.503 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13409e930 msgr2=0x7fd13406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:43.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13409e930 0x7fd13406b8b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c470 tx=0x7fd13000c780 comp rx=0 tx=0).stop 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 shutdown_connections 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd134136fe0 0x7fd13406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13409e930 0x7fd13406b8b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 --2- 172.21.15.32:0/3680124860 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd13409f410 0x7fd13406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 >> 172.21.15.32:0/3680124860 conn(0x7fd1340a4210 msgr2=0x7fd134134890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 shutdown_connections 2023-09-30T22:27:43.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:43.506+0000 7fd13b7f0700 1 -- 172.21.15.32:0/3680124860 wait complete. 2023-09-30T22:27:43.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:44.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:44.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.339+0000 7f04dd348700 1 Processor -- start 2023-09-30T22:27:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.339+0000 7f04dd348700 1 -- start start 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04dd348700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 0x7f04d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04dd348700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04d809d630 0x7f04d80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04dd348700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d80a2a70 con 0x7f04d8133930 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04dd348700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d80a01d0 con 0x7f04d809d630 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04dd348700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04d809e2b0 con 0x7f04d80a0a30 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 0x7f04d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 0x7f04d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57308/0 (socket says 172.21.15.32:57308) 2023-09-30T22:27:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04d6ffd700 1 -- 172.21.15.32:0/3059892598 learned_addr learned my addr 172.21.15.32:0/3059892598 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04d77fe700 1 --1- 172.21.15.32:0/3059892598 >> v1:172.21.15.32:6789/0 conn(0x7f04d8133930 0x7f04d8133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48636/0 (socket says 172.21.15.32:48636) 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.340+0000 7f04d67fc700 1 --2- 172.21.15.32:0/3059892598 >> v2:172.21.15.32:3300/0 conn(0x7f04d809d630 0x7f04d80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d67fc700 1 -- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 msgr2=0x7f04d80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d67fc700 1 --2- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 0x7f04d80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d67fc700 1 -- 172.21.15.32:0/3059892598 >> v1:172.21.15.32:6789/0 conn(0x7f04d8133930 legacy=0x7f04d8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d5ffb700 1 -- 172.21.15.32:0/3059892598 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45434493 0 0) 0x7f04d80a2a70 con 0x7f04d8133930 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d67fc700 1 -- 172.21.15.32:0/3059892598 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d809dc50 con 0x7f04d809d630 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d6ffd700 1 --2- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 0x7f04d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d67fc700 1 --2- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d80a1340 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009840 tx=0x7f04cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515fd3a28f00ae8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d5ffb700 1 -- 172.21.15.32:0/3059892598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc011830 con 0x7f04d809d630 2023-09-30T22:27:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d5ffb700 1 -- 172.21.15.32:0/3059892598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc011990 con 0x7f04d809d630 2023-09-30T22:27:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.341+0000 7f04d5ffb700 1 -- 172.21.15.32:0/3059892598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc011d20 con 0x7f04d809d630 2023-09-30T22:27:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 -- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 msgr2=0x7f04d80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 --2- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d80a1340 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009840 tx=0x7f04cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 -- 172.21.15.32:0/3059892598 shutdown_connections 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 --2- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d80a0a30 0x7f04d80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 --2- 172.21.15.32:0/3059892598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d80a1340 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 -- 172.21.15.32:0/3059892598 >> 172.21.15.32:0/3059892598 conn(0x7f04d80a3050 msgr2=0x7f04d80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 -- 172.21.15.32:0/3059892598 shutdown_connections 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.342+0000 7f04dd348700 1 -- 172.21.15.32:0/3059892598 wait complete. 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 Processor -- start 2023-09-30T22:27:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 -- start start 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d81c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04d80a0a30 0x7f04d81c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 0x7f04d81c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d81210e0 con 0x7f04d809d630 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d81207f0 con 0x7f04d80a0a30 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.344+0000 7f04dd348700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04d81209e0 con 0x7f04d8133930 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d81c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 0x7f04d81c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d81c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55908/0 (socket says 172.21.15.32:55908) 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 0x7f04d81c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57312/0 (socket says 172.21.15.32:57312) 2023-09-30T22:27:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d6ffd700 1 -- 172.21.15.32:0/1041603898 learned_addr learned my addr 172.21.15.32:0/1041603898 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d67fc700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04d80a0a30 0x7f04d81c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 -- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04d80a0a30 msgr2=0x7f04d81c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04d80a0a30 0x7f04d81c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 -- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 msgr2=0x7f04d81c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d81c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 -- 172.21.15.32:0/1041603898 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c8009550 con 0x7f04d8133930 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d6ffd700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d81c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04d77fe700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 0x7f04d81c8c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f04d000cc10 tx=0x7f04d000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000aa00 con 0x7f04d8133930 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d000ab80 con 0x7f04d8133930 2023-09-30T22:27:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc0094a0 con 0x7f04d8133930 2023-09-30T22:27:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d00070d0 con 0x7f04d8133930 2023-09-30T22:27:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.345+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d80a0410 con 0x7f04d8133930 2023-09-30T22:27:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.346+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f04d0007230 con 0x7f04d8133930 2023-09-30T22:27:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.346+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f04d003f400 con 0x7f04d8133930 2023-09-30T22:27:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.347+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d8122380 con 0x7f04d8133930 2023-09-30T22:27:44.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.352+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f04d0018080 con 0x7f04d8133930 2023-09-30T22:27:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.379+0000 7fc64a226700 1 Processor -- start 2023-09-30T22:27:44.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.379+0000 7fc64a226700 1 -- start start 2023-09-30T22:27:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.379+0000 7fc64a226700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6440a0a30 0x7fc6440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.379+0000 7fc64a226700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a1340 0x7fc644133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.379+0000 7fc64a226700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64409edd0 con 0x7fc644133fa0 2023-09-30T22:27:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc64a226700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64409e1f0 con 0x7fc6440a0a30 2023-09-30T22:27:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc64a226700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc64409dc50 con 0x7fc6440a1340 2023-09-30T22:27:44.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc643fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc644133fa0 0x7fc64413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48652/0 (socket says 172.21.15.32:48652) 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc643fff700 1 -- 172.21.15.32:0/633008270 learned_addr learned my addr 172.21.15.32:0/633008270 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6437fe700 1 --2- 172.21.15.32:0/633008270 >> v2:172.21.15.32:3300/0 conn(0x7fc6440a0a30 0x7fc6440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc642ffd700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a1340 0x7fc644133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6437fe700 1 -- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a1340 msgr2=0x7fc644133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6437fe700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a1340 0x7fc644133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6437fe700 1 -- 172.21.15.32:0/633008270 >> v1:172.21.15.32:6789/0 conn(0x7fc644133fa0 legacy=0x7fc64413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6437fe700 1 -- 172.21.15.32:0/633008270 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440a1fe0 con 0x7fc6440a0a30 2023-09-30T22:27:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6427fc700 1 -- 172.21.15.32:0/633008270 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639213186 0 0) 0x7fc64409edd0 con 0x7fc644133fa0 2023-09-30T22:27:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc642ffd700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a1340 0x7fc644133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6437fe700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a0a30 0x7fc6440a0e00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc634009840 tx=0x7fc634009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224721a5cec810cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6427fc700 1 -- 172.21.15.32:0/633008270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634011830 con 0x7fc6440a0a30 2023-09-30T22:27:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6427fc700 1 -- 172.21.15.32:0/633008270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc634011990 con 0x7fc6440a0a30 2023-09-30T22:27:44.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.380+0000 7fc6427fc700 1 -- 172.21.15.32:0/633008270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634011d20 con 0x7fc6440a0a30 2023-09-30T22:27:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 -- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a0a30 msgr2=0x7fc6440a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a0a30 0x7fc6440a0e00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc634009840 tx=0x7fc634009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 -- 172.21.15.32:0/633008270 shutdown_connections 2023-09-30T22:27:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a0a30 0x7fc6440a0e00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 --2- 172.21.15.32:0/633008270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a1340 0x7fc644133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 -- 172.21.15.32:0/633008270 >> 172.21.15.32:0/633008270 conn(0x7fc6440a6550 msgr2=0x7fc6440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 -- 172.21.15.32:0/633008270 shutdown_connections 2023-09-30T22:27:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.381+0000 7fc64a226700 1 -- 172.21.15.32:0/633008270 wait complete. 2023-09-30T22:27:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.382+0000 7fc64a226700 1 Processor -- start 2023-09-30T22:27:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.382+0000 7fc64a226700 1 -- start start 2023-09-30T22:27:44.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc64a226700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 0x7fc644122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc64a226700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a1340 0x7fc644122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc64a226700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc644133fa0 0x7fc644123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc64a226700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc644124520 con 0x7fc6440a1340 2023-09-30T22:27:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc64a226700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc644133320 con 0x7fc644133fa0 2023-09-30T22:27:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc64a226700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441262b0 con 0x7fc6440a0a30 2023-09-30T22:27:44.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 0x7fc644122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc642ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a1340 0x7fc644122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 0x7fc644122790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57336/0 (socket says 172.21.15.32:57336) 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 -- 172.21.15.32:0/4257604086 learned_addr learned my addr 172.21.15.32:0/4257604086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc643fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc644133fa0 0x7fc644123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 -- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc644133fa0 msgr2=0x7fc644123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc644133fa0 0x7fc644123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 -- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a1340 msgr2=0x7fc644122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a1340 0x7fc644122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.383+0000 7fc6437fe700 1 -- 172.21.15.32:0/4257604086 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380094a0 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.384+0000 7fc6437fe700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 0x7fc644122790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc6340090b0 tx=0x7fc634004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.384+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634004550 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.384+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63401f070 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.384+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6340094a0 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.384+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340038e0 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.384+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6440a0220 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.385+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc634003a40 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.385+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc634021070 con 0x7fc6440a0a30 2023-09-30T22:27:44.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.385+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6441336e0 con 0x7fc6440a0a30 2023-09-30T22:27:44.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.390+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc63401a950 con 0x7fc6440a0a30 2023-09-30T22:27:44.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:44.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.550+0000 7ff9db840700 1 Processor -- start 2023-09-30T22:27:44.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.550+0000 7ff9db840700 1 -- start start 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9db840700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d409e1d0 0x7ff9d4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9db840700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d4135e50 0x7ff9d4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9db840700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d409f130 con 0x7ff9d409d6f0 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9db840700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d409f520 con 0x7ff9d4135e50 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9db840700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d409e930 con 0x7ff9d409e1d0 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9d95dc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff9d409d6f0 0x7ff9d409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48658/0 (socket says 172.21.15.32:48658) 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.551+0000 7ff9d95dc700 1 -- 172.21.15.32:0/2188638804 learned_addr learned my addr 172.21.15.32:0/2188638804 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:44.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d4135e50 0x7ff9d4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d8ddb700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d409e1d0 0x7ff9d4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9cbfff700 1 -- 172.21.15.32:0/2188638804 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639102579 0 0) 0x7ff9d409f130 con 0x7ff9d409d6f0 2023-09-30T22:27:44.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9cbfff700 1 -- 172.21.15.32:0/2188638804 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c00035b0 con 0x7ff9d409d6f0 2023-09-30T22:27:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d9ddd700 1 -- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d409e1d0 msgr2=0x7ff9d4135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d409e1d0 0x7ff9d4135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d9ddd700 1 -- 172.21.15.32:0/2188638804 >> v1:172.21.15.32:6789/0 conn(0x7ff9d409d6f0 legacy=0x7ff9d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d9ddd700 1 -- 172.21.15.32:0/2188638804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d409b2d0 con 0x7ff9d4135e50 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d8ddb700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d409e1d0 0x7ff9d4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.552+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d4135e50 0x7ff9d4138230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00bfd0 tx=0x7ff9cc00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b8138de538ad85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.553+0000 7ff9cbfff700 1 -- 172.21.15.32:0/2188638804 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc010070 con 0x7ff9d4135e50 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.553+0000 7ff9cbfff700 1 -- 172.21.15.32:0/2188638804 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc00b2e0 con 0x7ff9d4135e50 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.553+0000 7ff9cbfff700 1 -- 172.21.15.32:0/2188638804 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00dc70 con 0x7ff9d4135e50 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 -- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d4135e50 msgr2=0x7ff9d4138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d4135e50 0x7ff9d4138230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00bfd0 tx=0x7ff9cc00d330 comp rx=0 tx=0).stop 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 -- 172.21.15.32:0/2188638804 shutdown_connections 2023-09-30T22:27:44.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d4135e50 0x7ff9d4138230 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 --2- 172.21.15.32:0/2188638804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d409e1d0 0x7ff9d4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 -- 172.21.15.32:0/2188638804 >> 172.21.15.32:0/2188638804 conn(0x7ff9d4066f50 msgr2=0x7ff9d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 -- 172.21.15.32:0/2188638804 shutdown_connections 2023-09-30T22:27:44.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.554+0000 7ff9db840700 1 -- 172.21.15.32:0/2188638804 wait complete. 2023-09-30T22:27:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.555+0000 7ff9db840700 1 Processor -- start 2023-09-30T22:27:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 -- start start 2023-09-30T22:27:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d409d6f0 0x7ff9d4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d409e1d0 0x7ff9d41199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d4135e50 0x7ff9d4119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d411ad40 con 0x7ff9d409d6f0 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d411a8e0 con 0x7ff9d409e1d0 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9db840700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40a2120 con 0x7ff9d4135e50 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.556+0000 7ff9d95dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d409d6f0 0x7ff9d4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d8ddb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d409e1d0 0x7ff9d41199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d95dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d409d6f0 0x7ff9d4119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55934/0 (socket says 172.21.15.32:55934) 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d95dc700 1 -- 172.21.15.32:0/2954368250 learned_addr learned my addr 172.21.15.32:0/2954368250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:44.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d4135e50 0x7ff9d4119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 -- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d409e1d0 msgr2=0x7ff9d41199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d409e1d0 0x7ff9d41199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 -- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d409d6f0 msgr2=0x7ff9d4119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d409d6f0 0x7ff9d4119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 -- 172.21.15.32:0/2954368250 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40094a0 con 0x7ff9d4135e50 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d8ddb700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d409e1d0 0x7ff9d41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.557+0000 7ff9d9ddd700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d4135e50 0x7ff9d4119f00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff9d409f6f0 tx=0x7ff9cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.558+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc009300 con 0x7ff9d4135e50 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.558+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc003e30 con 0x7ff9d4135e50 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.558+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc004470 con 0x7ff9d4135e50 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.559+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc00bc80 con 0x7ff9d4135e50 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.559+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d411bb60 con 0x7ff9d4135e50 2023-09-30T22:27:44.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.560+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9cc009050 con 0x7ff9d4135e50 2023-09-30T22:27:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.560+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff9cc0409c0 con 0x7ff9d4135e50 2023-09-30T22:27:44.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.560+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d411c940 con 0x7ff9d4135e50 2023-09-30T22:27:44.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.565+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f04d80a5780 con 0x7f04d8133930 2023-09-30T22:27:44.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.565+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9cc009460 con 0x7ff9d4135e50 2023-09-30T22:27:44.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.566+0000 7f04bf7fe700 1 -- 172.21.15.32:0/1041603898 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04d00074e0 con 0x7f04d8133930 2023-09-30T22:27:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.569+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 msgr2=0x7f04d81c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.569+0000 7f04dd348700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 0x7f04d81c8c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f04d000cc10 tx=0x7f04d000cf20 comp rx=0 tx=0).stop 2023-09-30T22:27:44.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.570+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 shutdown_connections 2023-09-30T22:27:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.570+0000 7f04dd348700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04d8133930 0x7f04d81c8c10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.570+0000 7f04dd348700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04d809d630 0x7f04d81c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.570+0000 7f04dd348700 1 --2- 172.21.15.32:0/1041603898 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04d80a0a30 0x7f04d81c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.570+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 >> 172.21.15.32:0/1041603898 conn(0x7f04d80a3050 msgr2=0x7f04d81bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.571+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 shutdown_connections 2023-09-30T22:27:44.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.571+0000 7f04dd348700 1 -- 172.21.15.32:0/1041603898 wait complete. 2023-09-30T22:27:44.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.600+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc644074ea0 con 0x7fc6440a0a30 2023-09-30T22:27:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.601+0000 7fc62bfff700 1 -- 172.21.15.32:0/4257604086 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc634010b70 con 0x7fc6440a0a30 2023-09-30T22:27:44.601 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 msgr2=0x7fc644122790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 0x7fc644122790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc6340090b0 tx=0x7fc634004750 comp rx=0 tx=0).stop 2023-09-30T22:27:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 shutdown_connections 2023-09-30T22:27:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc644133fa0 0x7fc644123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6440a0a30 0x7fc644122790 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 --2- 172.21.15.32:0/4257604086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6440a1340 0x7fc644122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 >> 172.21.15.32:0/4257604086 conn(0x7fc6440a6550 msgr2=0x7fc6440a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 shutdown_connections 2023-09-30T22:27:44.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.603+0000 7fc64a226700 1 -- 172.21.15.32:0/4257604086 wait complete. 2023-09-30T22:27:44.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.691+0000 7f2d10ff6700 1 Processor -- start 2023-09-30T22:27:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.691+0000 7f2d10ff6700 1 -- start start 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d10ff6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 0x7f2d0c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d10ff6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 0x7f2d0c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d10ff6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c070940 con 0x7f2d0c09f240 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d10ff6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c06fd60 con 0x7f2d0c09e930 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d10ff6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c06fa80 con 0x7f2d0c09f650 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d0a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 0x7f2d0c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d0ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 0x7f2d0c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d0ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 0x7f2d0c131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57364/0 (socket says 172.21.15.32:57364) 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d0a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 0x7f2d0c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55938/0 (socket says 172.21.15.32:55938) 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d09d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d0c09f240 0x7f2d0c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48674/0 (socket says 172.21.15.32:48674) 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.692+0000 7f2d0ad9d700 1 -- 172.21.15.32:0/2576807161 learned_addr learned my addr 172.21.15.32:0/2576807161 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0ad9d700 1 -- 172.21.15.32:0/2576807161 >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 msgr2=0x7f2d0c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0ad9d700 1 --2- 172.21.15.32:0/2576807161 >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 0x7f2d0c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0ad9d700 1 -- 172.21.15.32:0/2576807161 >> v1:172.21.15.32:6789/0 conn(0x7f2d0c09f240 legacy=0x7f2d0c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0ad9d700 1 -- 172.21.15.32:0/2576807161 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c09fb60 con 0x7f2d0c09f650 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0a59c700 1 --2- 172.21.15.32:0/2576807161 >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 0x7f2d0c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0ad9d700 1 --2- 172.21.15.32:0/2576807161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 0x7f2d0c131f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76596444f4280c96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0959a700 1 -- 172.21.15.32:0/2576807161 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400e070 con 0x7f2d0c09f650 2023-09-30T22:27:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0959a700 1 -- 172.21.15.32:0/2576807161 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0400b910 con 0x7f2d0c09f650 2023-09-30T22:27:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.693+0000 7f2d0959a700 1 -- 172.21.15.32:0/2576807161 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400bcc0 con 0x7f2d0c09f650 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 -- 172.21.15.32:0/2576807161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 msgr2=0x7f2d0c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/2576807161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 0x7f2d0c131f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 -- 172.21.15.32:0/2576807161 shutdown_connections 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/2576807161 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09f650 0x7f2d0c131f00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/2576807161 >> v2:172.21.15.32:3300/0 conn(0x7f2d0c09e930 0x7f2d0c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 -- 172.21.15.32:0/2576807161 >> 172.21.15.32:0/2576807161 conn(0x7f2d0c0a4210 msgr2=0x7f2d0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 -- 172.21.15.32:0/2576807161 shutdown_connections 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.694+0000 7f2d10ff6700 1 -- 172.21.15.32:0/2576807161 wait complete. 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.696+0000 7f2d10ff6700 1 Processor -- start 2023-09-30T22:27:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.696+0000 7f2d10ff6700 1 -- start start 2023-09-30T22:27:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.696+0000 7f2d10ff6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 0x7f2d0c06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.696+0000 7f2d10ff6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d0c09f240 0x7f2d0c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.696+0000 7f2d10ff6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c09f650 0x7f2d0c06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.696+0000 7f2d10ff6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1a91d0 con 0x7f2d0c09f650 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d10ff6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0a34f0 con 0x7f2d0c09f240 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d10ff6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1b7320 con 0x7f2d0c09e930 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 0x7f2d0c06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 0x7f2d0c06b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57378/0 (socket says 172.21.15.32:57378) 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c09f650 0x7f2d0c06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 -- 172.21.15.32:0/213572772 learned_addr learned my addr 172.21.15.32:0/213572772 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d09d9b700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d0c09f240 0x7f2d0c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 -- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d0c09f240 msgr2=0x7f2d0c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d0c09f240 0x7f2d0c06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 -- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c09f650 msgr2=0x7f2d0c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c09f650 0x7f2d0c06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 -- 172.21.15.32:0/213572772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0094a0 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0ad9d700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c09f650 0x7f2d0c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d0a59c700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 0x7f2d0c06b880 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00ebe0 tx=0x7f2cfc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00c850 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc00c9b0 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00cd40 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.697+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d04009b30 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.698+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c0a0190 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.698+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cfc013470 con 0x7f2d0c09e930 2023-09-30T22:27:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.699+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2cfc03fb30 con 0x7f2d0c09e930 2023-09-30T22:27:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.699+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c1a0b30 con 0x7f2d0c09e930 2023-09-30T22:27:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.704+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cfc013720 con 0x7f2d0c09e930 2023-09-30T22:27:44.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.808+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff9d411c680 con 0x7ff9d4135e50 2023-09-30T22:27:44.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.809+0000 7ff9c9ffb700 1 -- 172.21.15.32:0/2954368250 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9cc021020 con 0x7ff9d4135e50 2023-09-30T22:27:44.809 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d4135e50 msgr2=0x7ff9d4119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d4135e50 0x7ff9d4119f00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff9d409f6f0 tx=0x7ff9cc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 shutdown_connections 2023-09-30T22:27:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff9d4135e50 0x7ff9d4119f00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9d409d6f0 0x7ff9d4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 --2- 172.21.15.32:0/2954368250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9d409e1d0 0x7ff9d41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 >> 172.21.15.32:0/2954368250 conn(0x7ff9d4066f50 msgr2=0x7ff9d4132810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 shutdown_connections 2023-09-30T22:27:44.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.812+0000 7ff9db840700 1 -- 172.21.15.32:0/2954368250 wait complete. 2023-09-30T22:27:44.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:44.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.916+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d0c074ea0 con 0x7f2d0c09e930 2023-09-30T22:27:44.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.917+0000 7f2cfaffd700 1 -- 172.21.15.32:0/213572772 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cfc021030 con 0x7f2d0c09e930 2023-09-30T22:27:44.917 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 msgr2=0x7f2d0c06b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 0x7f2d0c06b880 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00ebe0 tx=0x7f2cfc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 shutdown_connections 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c09f650 0x7f2d0c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d0c09e930 0x7f2d0c06b880 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 --2- 172.21.15.32:0/213572772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d0c09f240 0x7f2d0c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:44.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 >> 172.21.15.32:0/213572772 conn(0x7f2d0c0a4210 msgr2=0x7f2d0c1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 shutdown_connections 2023-09-30T22:27:44.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:44.919+0000 7f2d10ff6700 1 -- 172.21.15.32:0/213572772 wait complete. 2023-09-30T22:27:44.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:45.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:45.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:45.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.735+0000 7f44d3086700 1 Processor -- start 2023-09-30T22:27:45.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.735+0000 7f44d3086700 1 -- start start 2023-09-30T22:27:45.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.735+0000 7f44d3086700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc0a0a30 0x7f44cc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d3086700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 0x7f44cc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d3086700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44cc133fa0 0x7f44cc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d3086700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc09edd0 con 0x7f44cc133fa0 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d3086700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc09e1f0 con 0x7f44cc0a1340 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d3086700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc09dc50 con 0x7f44cc0a0a30 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 0x7f44cc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d0e22700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc0a0a30 0x7f44cc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 0x7f44cc133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33748/0 (socket says 172.21.15.32:33748) 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 -- 172.21.15.32:0/61880274 learned_addr learned my addr 172.21.15.32:0/61880274 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d1623700 1 --2- 172.21.15.32:0/61880274 >> v2:172.21.15.32:3300/0 conn(0x7f44cc133fa0 0x7f44cc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 -- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc0a0a30 msgr2=0x7f44cc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:45.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 --2- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc0a0a30 0x7f44cc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 -- 172.21.15.32:0/61880274 >> v2:172.21.15.32:3300/0 conn(0x7f44cc133fa0 msgr2=0x7f44cc134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 --2- 172.21.15.32:0/61880274 >> v2:172.21.15.32:3300/0 conn(0x7f44cc133fa0 0x7f44cc134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 -- 172.21.15.32:0/61880274 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0a1fe0 con 0x7f44cc0a1340 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d1623700 1 --2- 172.21.15.32:0/61880274 >> v2:172.21.15.32:3300/0 conn(0x7f44cc133fa0 0x7f44cc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44d0e22700 1 --2- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc0a0a30 0x7f44cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cbfff700 1 --2- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 0x7f44cc133a60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f44c00098f0 tx=0x7f44c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9313537967889e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.736+0000 7f44cb7fe700 1 -- 172.21.15.32:0/61880274 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0011830 con 0x7f44cc0a1340 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44cb7fe700 1 -- 172.21.15.32:0/61880274 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0011990 con 0x7f44cc0a1340 2023-09-30T22:27:45.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44cb7fe700 1 -- 172.21.15.32:0/61880274 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0011d20 con 0x7f44cc0a1340 2023-09-30T22:27:45.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 -- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 msgr2=0x7f44cc133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 --2- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 0x7f44cc133a60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f44c00098f0 tx=0x7f44c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:45.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 -- 172.21.15.32:0/61880274 shutdown_connections 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 --2- 172.21.15.32:0/61880274 >> v2:172.21.15.32:3300/0 conn(0x7f44cc133fa0 0x7f44cc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 --2- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc0a0a30 0x7f44cc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 --2- 172.21.15.32:0/61880274 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a1340 0x7f44cc133a60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 -- 172.21.15.32:0/61880274 >> 172.21.15.32:0/61880274 conn(0x7f44cc0a6550 msgr2=0x7f44cc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 -- 172.21.15.32:0/61880274 shutdown_connections 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.737+0000 7f44d3086700 1 -- 172.21.15.32:0/61880274 wait complete. 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.738+0000 7f44d3086700 1 Processor -- start 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.738+0000 7f44d3086700 1 -- start start 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d3086700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 0x7f44cc1212c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d3086700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 0x7f44cc121800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d3086700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc133fa0 0x7f44cc121d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d3086700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc124be0 con 0x7f44cc0a1340 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d3086700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc123d70 con 0x7f44cc0a0a30 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d3086700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc124de0 con 0x7f44cc133fa0 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 0x7f44cc121800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d0e22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 0x7f44cc1212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 0x7f44cc121800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55970/0 (socket says 172.21.15.32:55970) 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d0e22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 0x7f44cc1212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33754/0 (socket says 172.21.15.32:33754) 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44cbfff700 1 -- 172.21.15.32:0/4220775819 learned_addr learned my addr 172.21.15.32:0/4220775819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.739+0000 7f44d1623700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc133fa0 0x7f44cc121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44cbfff700 1 -- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc133fa0 msgr2=0x7f44cc121d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44cbfff700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc133fa0 0x7f44cc121d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44cbfff700 1 -- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 msgr2=0x7f44cc1212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44cbfff700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 0x7f44cc1212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44cbfff700 1 -- 172.21.15.32:0/4220775819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0094a0 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44d0e22700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 0x7f44cc1212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44cbfff700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 0x7f44cc121800 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f44c0000c00 tx=0x7f44c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0004240 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c001f070 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c4009a70 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00045c0 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.740+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c0009550 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.741+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc0a0220 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.741+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44c0021030 con 0x7f44cc0a1340 2023-09-30T22:27:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.742+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f44c0040820 con 0x7f44cc0a1340 2023-09-30T22:27:45.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.746+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44c00197c0 con 0x7f44cc0a1340 2023-09-30T22:27:45.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.773+0000 7fe5b56f9700 1 Processor -- start 2023-09-30T22:27:45.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.773+0000 7fe5b56f9700 1 -- start start 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5b56f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 0x7fe5b0130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5b56f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5b009d6f0 0x7fe5b009db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5b56f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b009e080 0x7fe5b01374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5b56f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b009c190 con 0x7fe5b009d6f0 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5b56f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b009e520 con 0x7fe5b009e080 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5b56f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b009b090 con 0x7fe5b0130750 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5ae7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5b009d6f0 0x7fe5b009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5aeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 0x7fe5b0130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.774+0000 7fe5af7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b009e080 0x7fe5b01374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5b009d6f0 0x7fe5b009db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55976/0 (socket says 172.21.15.32:55976) 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5aeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 0x7fe5b0130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57418/0 (socket says 172.21.15.32:57418) 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 -- 172.21.15.32:0/2128307874 learned_addr learned my addr 172.21.15.32:0/2128307874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 -- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 msgr2=0x7fe5b0130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 0x7fe5b0130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 -- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b009e080 msgr2=0x7fe5b01374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b009e080 0x7fe5b01374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 -- 172.21.15.32:0/2128307874 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b009a6c0 con 0x7fe5b009d6f0 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5af7fe700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b009e080 0x7fe5b01374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5aeffd700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 0x7fe5b0130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5ae7fc700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b009db40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe59800bb90 tx=0x7fe59800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f1bdee02147797 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5adffb700 1 -- 172.21.15.32:0/2128307874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59800dd30 con 0x7fe5b009d6f0 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5adffb700 1 -- 172.21.15.32:0/2128307874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59800de90 con 0x7fe5b009d6f0 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.775+0000 7fe5adffb700 1 -- 172.21.15.32:0/2128307874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598010690 con 0x7fe5b009d6f0 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 -- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 msgr2=0x7fe5b009db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b009db40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe59800bb90 tx=0x7fe59800bea0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 -- 172.21.15.32:0/2128307874 shutdown_connections 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b009e080 0x7fe5b01374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b0130750 0x7fe5b0130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/2128307874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b009db40 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 -- 172.21.15.32:0/2128307874 >> 172.21.15.32:0/2128307874 conn(0x7fe5b009f470 msgr2=0x7fe5b00a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 -- 172.21.15.32:0/2128307874 shutdown_connections 2023-09-30T22:27:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.776+0000 7fe5b56f9700 1 -- 172.21.15.32:0/2128307874 wait complete. 2023-09-30T22:27:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.778+0000 7fe5b56f9700 1 Processor -- start 2023-09-30T22:27:45.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.778+0000 7fe5b56f9700 1 -- start start 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5b56f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b01c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5b56f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b009e080 0x7fe5b01c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5b56f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 0x7fe5b012bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5b56f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b012f4a0 con 0x7fe5b009d6f0 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5b56f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b0098ce0 con 0x7fe5b0130750 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5b56f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00a0680 con 0x7fe5b009e080 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5ae7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b009e080 0x7fe5b01c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 0x7fe5b012bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 0x7fe5b012bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33772/0 (socket says 172.21.15.32:33772) 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5ae7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b009e080 0x7fe5b01c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57422/0 (socket says 172.21.15.32:57422) 2023-09-30T22:27:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 -- 172.21.15.32:0/709451827 learned_addr learned my addr 172.21.15.32:0/709451827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5aeffd700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b01c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 -- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b009e080 msgr2=0x7fe5b01c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b009e080 0x7fe5b01c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 -- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 msgr2=0x7fe5b01c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b01c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 -- 172.21.15.32:0/709451827 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a0009a70 con 0x7fe5b0130750 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.779+0000 7fe5af7fe700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 0x7fe5b012bec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009e60 tx=0x7fe5a000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.780+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0009cd0 con 0x7fe5b0130750 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.780+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a000f040 con 0x7fe5b0130750 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.780+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a80094a0 con 0x7fe5b0130750 2023-09-30T22:27:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.780+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000ccb0 con 0x7fe5b0130750 2023-09-30T22:27:45.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.780+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59800b7f0 con 0x7fe5b0130750 2023-09-30T22:27:45.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.781+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5a0014480 con 0x7fe5b0130750 2023-09-30T22:27:45.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.781+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5a003fc20 con 0x7fe5b0130750 2023-09-30T22:27:45.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.781+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b01310e0 con 0x7fe5b0130750 2023-09-30T22:27:45.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.786+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5a001d410 con 0x7fe5b0130750 2023-09-30T22:27:45.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:45.933 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:45.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.960+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f44cc074ea0 con 0x7f44cc0a1340 2023-09-30T22:27:45.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.961+0000 7f44c97fa700 1 -- 172.21.15.32:0/4220775819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44c00193b0 con 0x7f44cc0a1340 2023-09-30T22:27:45.961 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 msgr2=0x7f44cc121800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 0x7f44cc121800 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f44c0000c00 tx=0x7f44c0004750 comp rx=0 tx=0).stop 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 shutdown_connections 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44cc133fa0 0x7f44cc121d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44cc0a0a30 0x7f44cc1212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 --2- 172.21.15.32:0/4220775819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44cc0a1340 0x7f44cc121800 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 >> 172.21.15.32:0/4220775819 conn(0x7f44cc0a6550 msgr2=0x7f44cc0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 shutdown_connections 2023-09-30T22:27:45.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.964+0000 7f44d3086700 1 -- 172.21.15.32:0/4220775819 wait complete. 2023-09-30T22:27:45.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.974+0000 7f59f4be5700 1 Processor -- start 2023-09-30T22:27:45.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.974+0000 7f59f4be5700 1 -- start start 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59f4be5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 0x7f59f0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59f4be5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59f4be5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f01313c0 con 0x7f59f0099a40 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59f4be5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f006e8c0 con 0x7f59f009a580 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59f4be5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0071d20 con 0x7f59f0131810 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59eed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59ee59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 0x7f59f0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59eed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f0139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33778/0 (socket says 172.21.15.32:33778) 2023-09-30T22:27:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59ee59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 0x7f59f0131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57430/0 (socket says 172.21.15.32:57430) 2023-09-30T22:27:45.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.975+0000 7f59eed9d700 1 -- 172.21.15.32:0/1300178607 learned_addr learned my addr 172.21.15.32:0/1300178607 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59eed9d700 1 -- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 msgr2=0x7f59f0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59eed9d700 1 --2- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 0x7f59f0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59eed9d700 1 -- 172.21.15.32:0/1300178607 >> v1:172.21.15.32:6789/0 conn(0x7f59f0099a40 legacy=0x7f59f0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59eed9d700 1 -- 172.21.15.32:0/1300178607 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00721a0 con 0x7f59f009a580 2023-09-30T22:27:45.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59ee59c700 1 --2- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 0x7f59f0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59eed9d700 1 --2- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f0139660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009e80 tx=0x7f59e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e07093167c6eab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59ed59a700 1 -- 172.21.15.32:0/1300178607 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800e070 con 0x7f59f009a580 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59ed59a700 1 -- 172.21.15.32:0/1300178607 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e800b910 con 0x7f59f009a580 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.976+0000 7f59ed59a700 1 -- 172.21.15.32:0/1300178607 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800bcc0 con 0x7f59f009a580 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 -- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 msgr2=0x7f59f0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f0139660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009e80 tx=0x7f59e800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 -- 172.21.15.32:0/1300178607 shutdown_connections 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f0139660 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1300178607 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0131810 0x7f59f0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 -- 172.21.15.32:0/1300178607 >> 172.21.15.32:0/1300178607 conn(0x7f59f00a2aa0 msgr2=0x7f59f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:45.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 -- 172.21.15.32:0/1300178607 shutdown_connections 2023-09-30T22:27:45.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.977+0000 7f59f4be5700 1 -- 172.21.15.32:0/1300178607 wait complete. 2023-09-30T22:27:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.978+0000 7f59f4be5700 1 Processor -- start 2023-09-30T22:27:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.978+0000 7f59f4be5700 1 -- start start 2023-09-30T22:27:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59f4be5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 0x7f59f006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59f4be5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59f4be5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f0131810 0x7f59f0070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59f4be5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f01213d0 con 0x7f59f0131810 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59f4be5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f012ec80 con 0x7f59f009a580 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59f4be5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0120350 con 0x7f59f0099a40 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59ee59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 0x7f59f006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59ee59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 0x7f59f006f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57432/0 (socket says 172.21.15.32:57432) 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59ee59c700 1 -- 172.21.15.32:0/1449229295 learned_addr learned my addr 172.21.15.32:0/1449229295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59edd9b700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.979+0000 7f59eed9d700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f0131810 0x7f59f0070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59ee59c700 1 -- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 msgr2=0x7f59f006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59ee59c700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59ee59c700 1 -- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f0131810 msgr2=0x7f59f0070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59ee59c700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f0131810 0x7f59f0070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59ee59c700 1 -- 172.21.15.32:0/1449229295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e00094a0 con 0x7f59f0099a40 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59ee59c700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 0x7f59f006f610 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009ef0 tx=0x7f59e000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0017c60 con 0x7f59f0099a40 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e0017dc0 con 0x7f59f0099a40 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e8009b30 con 0x7f59f0099a40 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000c860 con 0x7f59f0099a40 2023-09-30T22:27:45.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.980+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f006eb00 con 0x7f59f0099a40 2023-09-30T22:27:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.981+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59e000c9c0 con 0x7f59f0099a40 2023-09-30T22:27:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.981+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f59e00479f0 con 0x7f59f0099a40 2023-09-30T22:27:45.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.982+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f0074ea0 con 0x7f59f0099a40 2023-09-30T22:27:45.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:45.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:45.987+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59e001e990 con 0x7f59f0099a40 2023-09-30T22:27:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.010+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe5b00747f0 con 0x7fe5b0130750 2023-09-30T22:27:46.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.010+0000 7fe5a77fe700 1 -- 172.21.15.32:0/709451827 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5a001c3b0 con 0x7fe5b0130750 2023-09-30T22:27:46.010 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:46.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 msgr2=0x7fe5b012bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 0x7fe5b012bec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009e60 tx=0x7fe5a000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 shutdown_connections 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5b0130750 0x7fe5b012bec0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5b009d6f0 0x7fe5b01c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 --2- 172.21.15.32:0/709451827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5b009e080 0x7fe5b01c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.012+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 >> 172.21.15.32:0/709451827 conn(0x7fe5b009f470 msgr2=0x7fe5b00992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.013+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 shutdown_connections 2023-09-30T22:27:46.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.013+0000 7fe5b56f9700 1 -- 172.21.15.32:0/709451827 wait complete. 2023-09-30T22:27:46.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:46.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.079+0000 7f11a74ee700 1 Processor -- start 2023-09-30T22:27:46.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.079+0000 7f11a74ee700 1 -- start start 2023-09-30T22:27:46.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a74ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 0x7f11a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:46.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a74ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11a009d630 0x7f11a00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a74ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a0133930 0x7f11a0133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a74ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a00a2a70 con 0x7f11a009d630 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a74ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00a01d0 con 0x7f11a0133930 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a74ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009e2b0 con 0x7f11a00a0a30 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a528a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 0x7f11a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a4a89700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11a009d630 0x7f11a00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a528a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 0x7f11a00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57436/0 (socket says 172.21.15.32:57436) 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a528a700 1 -- 172.21.15.32:0/2629921926 learned_addr learned my addr 172.21.15.32:0/2629921926 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a0133930 0x7f11a0133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 -- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 msgr2=0x7f11a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 0x7f11a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 -- 172.21.15.32:0/2629921926 >> v2:172.21.15.32:3300/0 conn(0x7f11a009d630 msgr2=0x7f11a00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 --2- 172.21.15.32:0/2629921926 >> v2:172.21.15.32:3300/0 conn(0x7f11a009d630 0x7f11a00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 -- 172.21.15.32:0/2629921926 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a009dc50 con 0x7f11a0133930 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a528a700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 0x7f11a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a4a89700 1 --2- 172.21.15.32:0/2629921926 >> v2:172.21.15.32:3300/0 conn(0x7f11a009d630 0x7f11a00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.080+0000 7f11a5a8b700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a0133930 0x7f11a0133dd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1198009840 tx=0x7f1198009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46763048d90b30a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f1197fff700 1 -- 172.21.15.32:0/2629921926 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011830 con 0x7f11a0133930 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f1197fff700 1 -- 172.21.15.32:0/2629921926 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198011990 con 0x7f11a0133930 2023-09-30T22:27:46.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f1197fff700 1 -- 172.21.15.32:0/2629921926 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011d20 con 0x7f11a0133930 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 -- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a0133930 msgr2=0x7f11a0133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a0133930 0x7f11a0133dd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1198009840 tx=0x7f1198009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 -- 172.21.15.32:0/2629921926 shutdown_connections 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a0133930 0x7f11a0133dd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 --2- 172.21.15.32:0/2629921926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a00a0a30 0x7f11a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 --2- 172.21.15.32:0/2629921926 >> v2:172.21.15.32:3300/0 conn(0x7f11a009d630 0x7f11a00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 -- 172.21.15.32:0/2629921926 >> 172.21.15.32:0/2629921926 conn(0x7f11a00a3050 msgr2=0x7f11a00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 -- 172.21.15.32:0/2629921926 shutdown_connections 2023-09-30T22:27:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.081+0000 7f11a74ee700 1 -- 172.21.15.32:0/2629921926 wait complete. 2023-09-30T22:27:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.082+0000 7f11a74ee700 1 Processor -- start 2023-09-30T22:27:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.082+0000 7f11a74ee700 1 -- start start 2023-09-30T22:27:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a74ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a009d630 0x7f11a01c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:46.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a74ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 0x7f11a01c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a74ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a0133930 0x7f11a01c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a74ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00a0410 con 0x7f11a0133930 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a74ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0120630 con 0x7f11a00a0a30 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a74ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00a4030 con 0x7f11a009d630 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 0x7f11a01c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 0x7f11a01c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33800/0 (socket says 172.21.15.32:33800) 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 -- 172.21.15.32:0/4099401330 learned_addr learned my addr 172.21.15.32:0/4099401330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a5a8b700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a0133930 0x7f11a01c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a528a700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a009d630 0x7f11a01c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 -- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a009d630 msgr2=0x7f11a01c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a009d630 0x7f11a01c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 -- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a0133930 msgr2=0x7f11a01c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a0133930 0x7f11a01c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.083+0000 7f11a4a89700 1 -- 172.21.15.32:0/4099401330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900094a0 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.084+0000 7f11a4a89700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 0x7f11a01c83c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f119000ebb0 tx=0x7f119000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.084+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000c850 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.084+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119000c9b0 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.084+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c009b30 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.084+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000cd40 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.084+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11980094a0 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.085+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1190013470 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.085+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f119003f760 con 0x7f11a00a0a30 2023-09-30T22:27:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.085+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a0121d80 con 0x7f11a00a0a30 2023-09-30T22:27:46.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.090+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11a0121d80 con 0x7f11a00a0a30 2023-09-30T22:27:46.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.195+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f59f006b9e0 con 0x7f59f0099a40 2023-09-30T22:27:46.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.196+0000 7f59deffd700 1 -- 172.21.15.32:0/1449229295 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59e0024020 con 0x7f59f0099a40 2023-09-30T22:27:46.196 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:46.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 msgr2=0x7f59f006f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 0x7f59f006f610 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009ef0 tx=0x7f59e000ee90 comp rx=0 tx=0).stop 2023-09-30T22:27:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 shutdown_connections 2023-09-30T22:27:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59f0131810 0x7f59f0070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f59f0099a40 0x7f59f006f610 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 --2- 172.21.15.32:0/1449229295 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59f009a580 0x7f59f006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 >> 172.21.15.32:0/1449229295 conn(0x7f59f00a2aa0 msgr2=0x7f59f01b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 shutdown_connections 2023-09-30T22:27:46.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.198+0000 7f59f4be5700 1 -- 172.21.15.32:0/1449229295 wait complete. 2023-09-30T22:27:46.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.298+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f11a0121d80 con 0x7f11a00a0a30 2023-09-30T22:27:46.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.298+0000 7f1195ffb700 1 -- 172.21.15.32:0/4099401330 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1190021080 con 0x7f11a00a0a30 2023-09-30T22:27:46.299 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 msgr2=0x7f11a01c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 0x7f11a01c83c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f119000ebb0 tx=0x7f119000eec0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 shutdown_connections 2023-09-30T22:27:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11a0133930 0x7f11a01c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11a009d630 0x7f11a01c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 --2- 172.21.15.32:0/4099401330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11a00a0a30 0x7f11a01c83c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 >> 172.21.15.32:0/4099401330 conn(0x7f11a00a3050 msgr2=0x7f11a009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 shutdown_connections 2023-09-30T22:27:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:46.301+0000 7f11a74ee700 1 -- 172.21.15.32:0/4099401330 wait complete. 2023-09-30T22:27:46.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:46.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:47.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 Processor -- start 2023-09-30T22:27:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 -- start start 2023-09-30T22:27:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc09d630 0x7f0cfc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc0a2a70 con 0x7f0cfc133930 2023-09-30T22:27:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0a01d0 con 0x7f0cfc09d630 2023-09-30T22:27:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.136+0000 7f0d0333e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc09e2b0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57450/0 (socket says 172.21.15.32:57450) 2023-09-30T22:27:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d008d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc09d630 0x7f0cfc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 -- 172.21.15.32:0/3250500763 learned_addr learned my addr 172.21.15.32:0/3250500763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 -- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc09d630 msgr2=0x7f0cfc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 --2- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc09d630 0x7f0cfc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 -- 172.21.15.32:0/3250500763 >> v1:172.21.15.32:6789/0 conn(0x7f0cfc133930 legacy=0x7f0cfc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0cf3fff700 1 -- 172.21.15.32:0/3250500763 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361152244 0 0) 0x7f0cfc0a2a70 con 0x7f0cfc133930 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 -- 172.21.15.32:0/3250500763 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc09dc50 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d008d9700 1 --2- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc09d630 0x7f0cfc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0d010da700 1 --2- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc0a0e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009920 tx=0x7f0cf8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7437ee80376303c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0cf3fff700 1 -- 172.21.15.32:0/3250500763 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8011830 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0cf3fff700 1 -- 172.21.15.32:0/3250500763 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8011990 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.137+0000 7f0cf3fff700 1 -- 172.21.15.32:0/3250500763 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8011d20 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 -- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 msgr2=0x7f0cfc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 --2- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc0a0e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009920 tx=0x7f0cf8009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 -- 172.21.15.32:0/3250500763 shutdown_connections 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 --2- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc0a0e00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 --2- 172.21.15.32:0/3250500763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc09d630 0x7f0cfc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 -- 172.21.15.32:0/3250500763 >> 172.21.15.32:0/3250500763 conn(0x7f0cfc0a3050 msgr2=0x7f0cfc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 -- 172.21.15.32:0/3250500763 shutdown_connections 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.138+0000 7f0d0333e700 1 -- 172.21.15.32:0/3250500763 wait complete. 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.139+0000 7f0d0333e700 1 Processor -- start 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.139+0000 7f0d0333e700 1 -- start start 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d0333e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc09d630 0x7f0cfc1c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d0333e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc1c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d0333e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 0x7f0cfc1c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d0333e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc122240 con 0x7f0cfc09d630 2023-09-30T22:27:47.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d0333e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0a0410 con 0x7f0cfc133930 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d0333e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc120530 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d018db700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 0x7f0cfc1c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d008d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc1c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d018db700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 0x7f0cfc1c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33814/0 (socket says 172.21.15.32:33814) 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d018db700 1 -- 172.21.15.32:0/2121148007 learned_addr learned my addr 172.21.15.32:0/2121148007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d010da700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc09d630 0x7f0cfc1c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d008d9700 1 -- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 msgr2=0x7f0cfc1c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d008d9700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 0x7f0cfc1c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d008d9700 1 -- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc09d630 msgr2=0x7f0cfc1c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d008d9700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc09d630 0x7f0cfc1c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d008d9700 1 -- 172.21.15.32:0/2121148007 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec0094a0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d018db700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 0x7f0cfc1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.140+0000 7f0d010da700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc09d630 0x7f0cfc1c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.141+0000 7f0d008d9700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc1c8540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e980 tx=0x7f0cec00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.141+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00c690 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.141+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec00c7f0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.141+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf8009550 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.141+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00cb80 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.141+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc120d20 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.142+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cec00cce0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.142+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cfc121200 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.142+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0cec03f6b0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.147+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cec01bb50 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 Processor -- start 2023-09-30T22:27:47.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 -- start start 2023-09-30T22:27:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52b80a0a30 0x7f52b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b809d630 0x7f52b80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b80a2a70 con 0x7f52b8133930 2023-09-30T22:27:47.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b80a01d0 con 0x7f52b80a0a30 2023-09-30T22:27:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.176+0000 7f52c021a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52b809e2b0 con 0x7f52b809d630 2023-09-30T22:27:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bdfb6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52b80a0a30 0x7f52b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52be7b7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52b8133930 0x7f52b8133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48714/0 (socket says 172.21.15.32:48714) 2023-09-30T22:27:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bd7b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b809d630 0x7f52b80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52be7b7700 1 -- 172.21.15.32:0/581108739 learned_addr learned my addr 172.21.15.32:0/581108739 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:47.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bdfb6700 1 -- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b809d630 msgr2=0x7f52b80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bdfb6700 1 --2- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b809d630 0x7f52b80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bdfb6700 1 -- 172.21.15.32:0/581108739 >> v1:172.21.15.32:6789/0 conn(0x7f52b8133930 legacy=0x7f52b8133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bdfb6700 1 -- 172.21.15.32:0/581108739 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b809dc50 con 0x7f52b80a0a30 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bdfb6700 1 --2- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b80a0e00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009840 tx=0x7f52b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d565f10959d2f4a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bcfb4700 1 -- 172.21.15.32:0/581108739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4011830 con 0x7f52b80a0a30 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bcfb4700 1 -- 172.21.15.32:0/581108739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4011990 con 0x7f52b80a0a30 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.177+0000 7f52bcfb4700 1 -- 172.21.15.32:0/581108739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4011d20 con 0x7f52b80a0a30 2023-09-30T22:27:47.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 -- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 msgr2=0x7f52b80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 --2- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b80a0e00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009840 tx=0x7f52b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 -- 172.21.15.32:0/581108739 shutdown_connections 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 --2- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b80a0e00 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 --2- 172.21.15.32:0/581108739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b809d630 0x7f52b80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 -- 172.21.15.32:0/581108739 >> 172.21.15.32:0/581108739 conn(0x7f52b80a3050 msgr2=0x7f52b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 -- 172.21.15.32:0/581108739 shutdown_connections 2023-09-30T22:27:47.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.178+0000 7f52c021a700 1 -- 172.21.15.32:0/581108739 wait complete. 2023-09-30T22:27:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.179+0000 7f52c021a700 1 Processor -- start 2023-09-30T22:27:47.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 -- start start 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52b809d630 0x7f52b8132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b8132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b8133930 0x7f52b81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8120670 con 0x7f52b80a0a30 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8121fe0 con 0x7f52b809d630 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52c021a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8122660 con 0x7f52b8133930 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52bd7b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b8132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52be7b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b8133930 0x7f52b81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52bd7b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b8132d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56028/0 (socket says 172.21.15.32:56028) 2023-09-30T22:27:47.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52bd7b5700 1 -- 172.21.15.32:0/396034341 learned_addr learned my addr 172.21.15.32:0/396034341 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.180+0000 7f52bdfb6700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52b809d630 0x7f52b8132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52bd7b5700 1 -- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b8133930 msgr2=0x7f52b81c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52bd7b5700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b8133930 0x7f52b81c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52bd7b5700 1 -- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52b809d630 msgr2=0x7f52b8132840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52bd7b5700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52b809d630 0x7f52b8132840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52bd7b5700 1 -- 172.21.15.32:0/396034341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40094a0 con 0x7f52b80a0a30 2023-09-30T22:27:47.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52be7b7700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b8133930 0x7f52b81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52bd7b5700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b8132d80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f52a800e9b0 tx=0x7f52a800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8009710 con 0x7f52b80a0a30 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a800ee50 con 0x7f52b80a0a30 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b80a40c0 con 0x7f52b80a0a30 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a800ca50 con 0x7f52b80a0a30 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.181+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b81332c0 con 0x7f52b80a0a30 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.182+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f52a800cbb0 con 0x7f52b80a0a30 2023-09-30T22:27:47.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.182+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b80747f0 con 0x7f52b80a0a30 2023-09-30T22:27:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.182+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f52a803faa0 con 0x7f52b80a0a30 2023-09-30T22:27:47.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.187+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52a801b3b0 con 0x7f52b80a0a30 2023-09-30T22:27:47.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:47.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 Processor -- start 2023-09-30T22:27:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 -- start start 2023-09-30T22:27:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18fc131810 0x7f18fc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 0x7f18fc099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc09a3b0 0x7f18fc138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18fc1313c0 con 0x7f18fc131810 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc06e8c0 con 0x7f18fc099a40 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.354+0000 7f19027b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc071d20 con 0x7f18fc09a3b0 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 0x7f18fc099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 0x7f18fc099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33828/0 (socket says 172.21.15.32:33828) 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 -- 172.21.15.32:0/620920328 learned_addr learned my addr 172.21.15.32:0/620920328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fbfff700 1 --2- 172.21.15.32:0/620920328 >> v2:172.21.15.32:3300/0 conn(0x7f18fc131810 0x7f18fc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 -- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc09a3b0 msgr2=0x7f18fc138650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:27:47.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc09a3b0 0x7f18fc138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 -- 172.21.15.32:0/620920328 >> v2:172.21.15.32:3300/0 conn(0x7f18fc131810 msgr2=0x7f18fc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/620920328 >> v2:172.21.15.32:3300/0 conn(0x7f18fc131810 0x7f18fc131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 -- 172.21.15.32:0/620920328 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0721a0 con 0x7f18fc099a40 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fbfff700 1 --2- 172.21.15.32:0/620920328 >> v2:172.21.15.32:3300/0 conn(0x7f18fc131810 0x7f18fc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.355+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 0x7f18fc099e70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f18f00098f0 tx=0x7f18f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808d236b61ee4d55 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.356+0000 7f18faffd700 1 -- 172.21.15.32:0/620920328 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0011830 con 0x7f18fc099a40 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.356+0000 7f18faffd700 1 -- 172.21.15.32:0/620920328 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f0011990 con 0x7f18fc099a40 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.356+0000 7f18faffd700 1 -- 172.21.15.32:0/620920328 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0011d20 con 0x7f18fc099a40 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 -- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 msgr2=0x7f18fc099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 --2- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 0x7f18fc099e70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f18f00098f0 tx=0x7f18f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 -- 172.21.15.32:0/620920328 shutdown_connections 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 --2- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc09a3b0 0x7f18fc138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 --2- 172.21.15.32:0/620920328 >> v2:172.21.15.32:3300/0 conn(0x7f18fc131810 0x7f18fc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 --2- 172.21.15.32:0/620920328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc099a40 0x7f18fc099e70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 -- 172.21.15.32:0/620920328 >> 172.21.15.32:0/620920328 conn(0x7f18fc0a2aa0 msgr2=0x7f18fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 -- 172.21.15.32:0/620920328 shutdown_connections 2023-09-30T22:27:47.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.357+0000 7f19027b7700 1 -- 172.21.15.32:0/620920328 wait complete. 2023-09-30T22:27:47.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.358+0000 7f19027b7700 1 Processor -- start 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.358+0000 7f19027b7700 1 -- start start 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f19027b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc099a40 0x7f18fc11b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f19027b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 0x7f18fc11bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f19027b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc131810 0x7f18fc11c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f19027b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc11f220 con 0x7f18fc09a3b0 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f19027b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc11e5d0 con 0x7f18fc131810 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f19027b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc11d3b0 con 0x7f18fc099a40 2023-09-30T22:27:47.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f18fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 0x7f18fc11bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f18fb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 0x7f18fc11bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56038/0 (socket says 172.21.15.32:56038) 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f18fb7fe700 1 -- 172.21.15.32:0/3640924986 learned_addr learned my addr 172.21.15.32:0/3640924986 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f1900d54700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc131810 0x7f18fc11c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.359+0000 7f18fbfff700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc099a40 0x7f18fc11b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18fb7fe700 1 -- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc099a40 msgr2=0x7f18fc11b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc099a40 0x7f18fc11b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18fb7fe700 1 -- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc131810 msgr2=0x7f18fc11c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc131810 0x7f18fc11c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18fb7fe700 1 -- 172.21.15.32:0/3640924986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec00d940 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 0x7f18fc11bc40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f18f0000c00 tx=0x7f18f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0004300 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f001f070 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f0009550 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00036a0 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.360+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc11caf0 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.361+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc074ea0 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.362+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18f0021030 con 0x7f18fc09a3b0 2023-09-30T22:27:47.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.362+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f18f0040c60 con 0x7f18fc09a3b0 2023-09-30T22:27:47.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.366+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0cfc1220a0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.366+0000 7f0cf1ffb700 1 -- 172.21.15.32:0/2121148007 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cfc1220a0 con 0x7f0cfc0a0a30 2023-09-30T22:27:47.367 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:47.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.367+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18f0006ce0 con 0x7f18fc09a3b0 2023-09-30T22:27:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.370+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 msgr2=0x7f0cfc1c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.370+0000 7f0d0333e700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc1c8540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e980 tx=0x7f0cec00ec90 comp rx=0 tx=0).stop 2023-09-30T22:27:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.372+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 shutdown_connections 2023-09-30T22:27:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.372+0000 7f0d0333e700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0cfc133930 0x7f0cfc1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.372+0000 7f0d0333e700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cfc09d630 0x7f0cfc1c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.372+0000 7f0d0333e700 1 --2- 172.21.15.32:0/2121148007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0cfc0a0a30 0x7f0cfc1c8540 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.372+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 >> 172.21.15.32:0/2121148007 conn(0x7f0cfc0a3050 msgr2=0x7f0cfc1bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.373+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 shutdown_connections 2023-09-30T22:27:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.373+0000 7f0d0333e700 1 -- 172.21.15.32:0/2121148007 wait complete. 2023-09-30T22:27:47.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.406+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f52b8072980 con 0x7f52b80a0a30 2023-09-30T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.407+0000 7f52ae7fc700 1 -- 172.21.15.32:0/396034341 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52a8017020 con 0x7f52b80a0a30 2023-09-30T22:27:47.407 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 msgr2=0x7f52b8132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b8132d80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f52a800e9b0 tx=0x7f52a800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 shutdown_connections 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52b8133930 0x7f52b81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52b809d630 0x7f52b8132840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 --2- 172.21.15.32:0/396034341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52b80a0a30 0x7f52b8132d80 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 >> 172.21.15.32:0/396034341 conn(0x7f52b80a3050 msgr2=0x7f52b81bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 shutdown_connections 2023-09-30T22:27:47.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.409+0000 7f52c021a700 1 -- 172.21.15.32:0/396034341 wait complete. 2023-09-30T22:27:47.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.461+0000 7fe6fe95c700 1 Processor -- start 2023-09-30T22:27:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.461+0000 7fe6fe95c700 1 -- start start 2023-09-30T22:27:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6fe95c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 0x7fe6f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6fe95c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f8138d80 0x7fe6f813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6fe95c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80a2a70 con 0x7fe6f809d630 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6fe95c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80a01d0 con 0x7fe6f80a0a30 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6fe95c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f809e2b0 con 0x7fe6f8138d80 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6f7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 0x7fe6f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6f7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 0x7fe6f80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56044/0 (socket says 172.21.15.32:56044) 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6f7fff700 1 -- 172.21.15.32:0/2770432143 learned_addr learned my addr 172.21.15.32:0/2770432143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6fcef9700 1 --2- 172.21.15.32:0/2770432143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f8138d80 0x7fe6f813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.462+0000 7fe6f77fe700 1 --1- 172.21.15.32:0/2770432143 >> v1:172.21.15.32:6789/0 conn(0x7fe6f809d630 0x7fe6f80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48720/0 (socket says 172.21.15.32:48720) 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6fcef9700 1 -- 172.21.15.32:0/2770432143 >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 msgr2=0x7fe6f80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6fcef9700 1 --2- 172.21.15.32:0/2770432143 >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 0x7fe6f80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6fcef9700 1 -- 172.21.15.32:0/2770432143 >> v1:172.21.15.32:6789/0 conn(0x7fe6f809d630 legacy=0x7fe6f80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6fcef9700 1 -- 172.21.15.32:0/2770432143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f809dc50 con 0x7fe6f8138d80 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6f7fff700 1 --2- 172.21.15.32:0/2770432143 >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 0x7fe6f80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6fcef9700 1 --2- 172.21.15.32:0/2770432143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f8138d80 0x7fe6f813b160 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009d90 tx=0x7fe6f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28897c0e1239b2ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6f6ffd700 1 -- 172.21.15.32:0/2770432143 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000e070 con 0x7fe6f8138d80 2023-09-30T22:27:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6f6ffd700 1 -- 172.21.15.32:0/2770432143 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f000bb20 con 0x7fe6f8138d80 2023-09-30T22:27:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.463+0000 7fe6f6ffd700 1 -- 172.21.15.32:0/2770432143 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0012430 con 0x7fe6f8138d80 2023-09-30T22:27:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 -- 172.21.15.32:0/2770432143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f8138d80 msgr2=0x7fe6f813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/2770432143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f8138d80 0x7fe6f813b160 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009d90 tx=0x7fe6f000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 -- 172.21.15.32:0/2770432143 shutdown_connections 2023-09-30T22:27:47.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/2770432143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f8138d80 0x7fe6f813b160 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/2770432143 >> v2:172.21.15.32:3300/0 conn(0x7fe6f80a0a30 0x7fe6f80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 -- 172.21.15.32:0/2770432143 >> 172.21.15.32:0/2770432143 conn(0x7fe6f80a3050 msgr2=0x7fe6f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 -- 172.21.15.32:0/2770432143 shutdown_connections 2023-09-30T22:27:47.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.464+0000 7fe6fe95c700 1 -- 172.21.15.32:0/2770432143 wait complete. 2023-09-30T22:27:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.465+0000 7fe6fe95c700 1 Processor -- start 2023-09-30T22:27:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.465+0000 7fe6fe95c700 1 -- start start 2023-09-30T22:27:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fe95c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f809d630 0x7fe6f8133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fe95c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f80a0a30 0x7fe6f8133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fe95c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 0x7fe6f81344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fe95c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f8136b90 con 0x7fe6f80a0a30 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fe95c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f8138090 con 0x7fe6f809d630 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fe95c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81374b0 con 0x7fe6f8138d80 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fcef9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 0x7fe6f81344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fcef9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 0x7fe6f81344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57500/0 (socket says 172.21.15.32:57500) 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6f77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f80a0a30 0x7fe6f8133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6fcef9700 1 -- 172.21.15.32:0/3788537839 learned_addr learned my addr 172.21.15.32:0/3788537839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.466+0000 7fe6f7fff700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f809d630 0x7fe6f8133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fcef9700 1 -- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f809d630 msgr2=0x7fe6f8133a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fcef9700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f809d630 0x7fe6f8133a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fcef9700 1 -- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f80a0a30 msgr2=0x7fe6f8133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fcef9700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f80a0a30 0x7fe6f8133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fcef9700 1 -- 172.21.15.32:0/3788537839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8009550 con 0x7fe6f8138d80 2023-09-30T22:27:47.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6f77fe700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f80a0a30 0x7fe6f8133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fcef9700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 0x7fe6f81344b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0013000 tx=0x7fe6f0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0012430 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f0020070 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0003ea0 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f0009a70 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.467+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f81371d0 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.468+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6f0004280 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.468+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe6f0022070 con 0x7fe6f8138d80 2023-09-30T22:27:47.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.468+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f8136e70 con 0x7fe6f8138d80 2023-09-30T22:27:47.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.473+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6f8136e70 con 0x7fe6f8138d80 2023-09-30T22:27:47.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.585+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18fc11fa10 con 0x7f18fc09a3b0 2023-09-30T22:27:47.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.586+0000 7f18f8ff9700 1 -- 172.21.15.32:0/3640924986 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18f0015030 con 0x7f18fc09a3b0 2023-09-30T22:27:47.586 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 msgr2=0x7f18fc11bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 0x7f18fc11bc40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f18f0000c00 tx=0x7f18f0004750 comp rx=0 tx=0).stop 2023-09-30T22:27:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 shutdown_connections 2023-09-30T22:27:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18fc131810 0x7f18fc11c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f18fc099a40 0x7f18fc11b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 --2- 172.21.15.32:0/3640924986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc09a3b0 0x7f18fc11bc40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 >> 172.21.15.32:0/3640924986 conn(0x7f18fc0a2aa0 msgr2=0x7f18fc09d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 shutdown_connections 2023-09-30T22:27:47.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.588+0000 7f19027b7700 1 -- 172.21.15.32:0/3640924986 wait complete. 2023-09-30T22:27:47.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.678+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe6f8136e70 con 0x7fe6f8138d80 2023-09-30T22:27:47.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.678+0000 7fe6f4ff9700 1 -- 172.21.15.32:0/3788537839 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6f8136e70 con 0x7fe6f8138d80 2023-09-30T22:27:47.679 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 msgr2=0x7fe6f81344b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 0x7fe6f81344b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0013000 tx=0x7fe6f0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 shutdown_connections 2023-09-30T22:27:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe6f8138d80 0x7fe6f81344b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe6f809d630 0x7fe6f8133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 --2- 172.21.15.32:0/3788537839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f80a0a30 0x7fe6f8133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.680+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 >> 172.21.15.32:0/3788537839 conn(0x7fe6f80a3050 msgr2=0x7fe6f809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.681+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 shutdown_connections 2023-09-30T22:27:47.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:47.681+0000 7fe6fe95c700 1 -- 172.21.15.32:0/3788537839 wait complete. 2023-09-30T22:27:47.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:48.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:48.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.543+0000 7f36e7067700 1 Processor -- start 2023-09-30T22:27:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.543+0000 7f36e7067700 1 -- start start 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e7067700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009e930 0x7f36e009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e7067700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36e009f650 0x7f36e0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e7067700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e0070940 con 0x7f36e009f240 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e7067700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e006fd60 con 0x7f36e009f650 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e7067700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36e006fa80 con 0x7f36e009e930 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36dffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36e009f240 0x7f36e01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48728/0 (socket says 172.21.15.32:48728) 2023-09-30T22:27:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36dffff700 1 -- 172.21.15.32:0/716415744 learned_addr learned my addr 172.21.15.32:0/716415744 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e5604700 1 --2- 172.21.15.32:0/716415744 >> v2:172.21.15.32:3300/0 conn(0x7f36e009f650 0x7f36e0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36e4e03700 1 --2- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009e930 0x7f36e009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36df7fe700 1 -- 172.21.15.32:0/716415744 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256692177 0 0) 0x7f36e0070940 con 0x7f36e009f240 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.544+0000 7f36df7fe700 1 -- 172.21.15.32:0/716415744 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36c80035b0 con 0x7f36e009f240 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36e5604700 1 -- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009e930 msgr2=0x7f36e009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36e5604700 1 --2- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009e930 0x7f36e009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36e5604700 1 -- 172.21.15.32:0/716415744 >> v1:172.21.15.32:6789/0 conn(0x7f36e009f240 legacy=0x7f36e01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36e5604700 1 -- 172.21.15.32:0/716415744 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e009fb60 con 0x7f36e009f650 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36e5604700 1 --2- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e0131f00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009d90 tx=0x7f36d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9005894576da9ddd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36df7fe700 1 -- 172.21.15.32:0/716415744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800e070 con 0x7f36e009f650 2023-09-30T22:27:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36df7fe700 1 -- 172.21.15.32:0/716415744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d800bb20 con 0x7f36e009f650 2023-09-30T22:27:48.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.545+0000 7f36df7fe700 1 -- 172.21.15.32:0/716415744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8012430 con 0x7f36e009f650 2023-09-30T22:27:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 -- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 msgr2=0x7f36e0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 --2- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e0131f00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009d90 tx=0x7f36d800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 -- 172.21.15.32:0/716415744 shutdown_connections 2023-09-30T22:27:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 --2- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e0131f00 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 --2- 172.21.15.32:0/716415744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009e930 0x7f36e009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 -- 172.21.15.32:0/716415744 >> 172.21.15.32:0/716415744 conn(0x7f36e00a4210 msgr2=0x7f36e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 -- 172.21.15.32:0/716415744 shutdown_connections 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.546+0000 7f36e7067700 1 -- 172.21.15.32:0/716415744 wait complete. 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.547+0000 7f36e7067700 1 Processor -- start 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.547+0000 7f36e7067700 1 -- start start 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e7067700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 0x7f36e006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e7067700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009f240 0x7f36e006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e7067700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e7067700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e006d4c0 con 0x7f36e009f650 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e7067700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01a91d0 con 0x7f36e009e930 2023-09-30T22:27:48.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e7067700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01a86f0 con 0x7f36e009f240 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 0x7f36e006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 0x7f36e006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33854/0 (socket says 172.21.15.32:33854) 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 -- 172.21.15.32:0/532243643 learned_addr learned my addr 172.21.15.32:0/532243643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36dffff700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009f240 0x7f36e006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e5604700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 -- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009f240 msgr2=0x7f36e006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009f240 0x7f36e006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 -- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 msgr2=0x7f36e006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 -- 172.21.15.32:0/532243643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d8009a70 con 0x7f36e009e930 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e5604700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.548+0000 7f36e4e03700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 0x7f36e006b8b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e9e0 tx=0x7f36d000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.549+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0009710 con 0x7f36e009e930 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.549+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d000ee80 con 0x7f36e009e930 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.549+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e00a0190 con 0x7f36e009e930 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.549+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d000ca50 con 0x7f36e009e930 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.549+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e01a94e0 con 0x7f36e009e930 2023-09-30T22:27:48.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.549+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36d000cbb0 con 0x7f36e009e930 2023-09-30T22:27:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.550+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f36d003f8b0 con 0x7f36e009e930 2023-09-30T22:27:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.550+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e01a94e0 con 0x7f36e009e930 2023-09-30T22:27:48.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.555+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f36e01a94e0 con 0x7f36e009e930 2023-09-30T22:27:48.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.582+0000 7fb82b541700 1 Processor -- start 2023-09-30T22:27:48.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb82b541700 1 -- start start 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb82b541700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 0x7fb82409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb82b541700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 0x7fb824131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb82b541700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb824070940 con 0x7fb82409f240 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb82b541700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82406fd60 con 0x7fb82409f650 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb82b541700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82406fa80 con 0x7fb82409e930 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb8292dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 0x7fb82409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb829ade700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 0x7fb824131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb828adc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb82409f240 0x7fb8241317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48736/0 (socket says 172.21.15.32:48736) 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb828adc700 1 -- 172.21.15.32:0/593685000 learned_addr learned my addr 172.21.15.32:0/593685000 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb829ade700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 0x7fb824131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56090/0 (socket says 172.21.15.32:56090) 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.583+0000 7fb8292dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 0x7fb82409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33864/0 (socket says 172.21.15.32:33864) 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb81bfff700 1 -- 172.21.15.32:0/593685000 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547578452 0 0) 0x7fb824070940 con 0x7fb82409f240 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb8292dd700 1 -- 172.21.15.32:0/593685000 >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 msgr2=0x7fb824131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb8292dd700 1 --2- 172.21.15.32:0/593685000 >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 0x7fb824131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb8292dd700 1 -- 172.21.15.32:0/593685000 >> v1:172.21.15.32:6789/0 conn(0x7fb82409f240 legacy=0x7fb8241317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb8292dd700 1 -- 172.21.15.32:0/593685000 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82409fb60 con 0x7fb82409e930 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb829ade700 1 --2- 172.21.15.32:0/593685000 >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 0x7fb824131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb8292dd700 1 --2- 172.21.15.32:0/593685000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 0x7fb82409ed00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb820009840 tx=0x7fb820009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5646febd451ec11 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb81bfff700 1 -- 172.21.15.32:0/593685000 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820011830 con 0x7fb82409e930 2023-09-30T22:27:48.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb81bfff700 1 -- 172.21.15.32:0/593685000 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820011990 con 0x7fb82409e930 2023-09-30T22:27:48.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.584+0000 7fb81bfff700 1 -- 172.21.15.32:0/593685000 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820011d20 con 0x7fb82409e930 2023-09-30T22:27:48.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 -- 172.21.15.32:0/593685000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 msgr2=0x7fb82409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 --2- 172.21.15.32:0/593685000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 0x7fb82409ed00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb820009840 tx=0x7fb820009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:48.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 -- 172.21.15.32:0/593685000 shutdown_connections 2023-09-30T22:27:48.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 --2- 172.21.15.32:0/593685000 >> v2:172.21.15.32:3300/0 conn(0x7fb82409f650 0x7fb824131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 --2- 172.21.15.32:0/593685000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409e930 0x7fb82409ed00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 -- 172.21.15.32:0/593685000 >> 172.21.15.32:0/593685000 conn(0x7fb8240a4210 msgr2=0x7fb8240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 -- 172.21.15.32:0/593685000 shutdown_connections 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.585+0000 7fb82b541700 1 -- 172.21.15.32:0/593685000 wait complete. 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 Processor -- start 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 -- start start 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb82409e930 0x7fb82406c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409f240 0x7fb82406c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 0x7fb8241c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82406bd70 con 0x7fb82409f650 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240a3220 con 0x7fb82409f240 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.586+0000 7fb82b541700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8241a8080 con 0x7fb82409e930 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb829ade700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 0x7fb8241c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb82409e930 0x7fb82406c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb829ade700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 0x7fb8241c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56102/0 (socket says 172.21.15.32:56102) 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb829ade700 1 -- 172.21.15.32:0/2743430139 learned_addr learned my addr 172.21.15.32:0/2743430139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb828adc700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409f240 0x7fb82406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 -- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409f240 msgr2=0x7fb82406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409f240 0x7fb82406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 -- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 msgr2=0x7fb8241c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 0x7fb8241c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 -- 172.21.15.32:0/2743430139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009a70 con 0x7fb82409e930 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb828adc700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409f240 0x7fb82406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb829ade700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 0x7fb8241c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb8292dd700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb82409e930 0x7fb82406c0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb820016000 tx=0x7fb820004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200036a0 con 0x7fb82409e930 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82001f070 con 0x7fb82409e930 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8200094a0 con 0x7fb82409e930 2023-09-30T22:27:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.587+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820003a20 con 0x7fb82409e930 2023-09-30T22:27:48.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.588+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240a0190 con 0x7fb82409e930 2023-09-30T22:27:48.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.589+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8241a7cf0 con 0x7fb82409e930 2023-09-30T22:27:48.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.589+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb820003b80 con 0x7fb82409e930 2023-09-30T22:27:48.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.589+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb820021070 con 0x7fb82409e930 2023-09-30T22:27:48.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.594+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb820010b70 con 0x7fb82409e930 2023-09-30T22:27:48.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:48.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.750+0000 7f91ce3f6700 1 Processor -- start 2023-09-30T22:27:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.750+0000 7f91ce3f6700 1 -- start start 2023-09-30T22:27:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91ce3f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91ce3f6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91c8139170 0x7f91c813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91ce3f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c809b120 con 0x7f91c809d6f0 2023-09-30T22:27:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91ce3f6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c806b460 con 0x7f91c8139170 2023-09-30T22:27:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91ce3f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c809a7e0 con 0x7f91c809e1d0 2023-09-30T22:27:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91c77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91c77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33878/0 (socket says 172.21.15.32:33878) 2023-09-30T22:27:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.751+0000 7f91c77fe700 1 -- 172.21.15.32:0/1954082323 learned_addr learned my addr 172.21.15.32:0/1954082323 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91cc993700 1 --2- 172.21.15.32:0/1954082323 >> v2:172.21.15.32:3300/0 conn(0x7f91c8139170 0x7f91c813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c77fe700 1 -- 172.21.15.32:0/1954082323 >> v2:172.21.15.32:3300/0 conn(0x7f91c8139170 msgr2=0x7f91c813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c77fe700 1 --2- 172.21.15.32:0/1954082323 >> v2:172.21.15.32:3300/0 conn(0x7f91c8139170 0x7f91c813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c77fe700 1 -- 172.21.15.32:0/1954082323 >> v1:172.21.15.32:6789/0 conn(0x7f91c809d6f0 legacy=0x7f91c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c77fe700 1 -- 172.21.15.32:0/1954082323 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c809a500 con 0x7f91c809e1d0 2023-09-30T22:27:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c77fe700 1 --2- 172.21.15.32:0/1954082323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c8138c30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009920 tx=0x7f91bc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ef8cefac5e4800 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c6ffd700 1 -- 172.21.15.32:0/1954082323 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc011830 con 0x7f91c809e1d0 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c6ffd700 1 -- 172.21.15.32:0/1954082323 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc011990 con 0x7f91c809e1d0 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.752+0000 7f91c6ffd700 1 -- 172.21.15.32:0/1954082323 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc011d20 con 0x7f91c809e1d0 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 -- 172.21.15.32:0/1954082323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 msgr2=0x7f91c8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/1954082323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c8138c30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009920 tx=0x7f91bc009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 -- 172.21.15.32:0/1954082323 shutdown_connections 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/1954082323 >> v2:172.21.15.32:3300/0 conn(0x7f91c8139170 0x7f91c813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/1954082323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c8138c30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 -- 172.21.15.32:0/1954082323 >> 172.21.15.32:0/1954082323 conn(0x7f91c8066f50 msgr2=0x7f91c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 -- 172.21.15.32:0/1954082323 shutdown_connections 2023-09-30T22:27:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.753+0000 7f91ce3f6700 1 -- 172.21.15.32:0/1954082323 wait complete. 2023-09-30T22:27:48.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.755+0000 7f91ce3f6700 1 Processor -- start 2023-09-30T22:27:48.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.755+0000 7f91ce3f6700 1 -- start start 2023-09-30T22:27:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91ce3f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c809d6f0 0x7f91c81c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91ce3f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c81c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91ce3f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c8139170 0x7f91c8120790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91ce3f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8124060 con 0x7f91c809d6f0 2023-09-30T22:27:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91ce3f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c81223a0 con 0x7f91c809e1d0 2023-09-30T22:27:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91ce3f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80a2b50 con 0x7f91c8139170 2023-09-30T22:27:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c809d6f0 0x7f91c81c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c809d6f0 0x7f91c81c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56128/0 (socket says 172.21.15.32:56128) 2023-09-30T22:27:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91c7fff700 1 -- 172.21.15.32:0/4196989308 learned_addr learned my addr 172.21.15.32:0/4196989308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91cc993700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c8139170 0x7f91c8120790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.756+0000 7f91c77fe700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c81c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91cc993700 1 -- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 msgr2=0x7f91c81c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91cc993700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c81c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91cc993700 1 -- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c809d6f0 msgr2=0x7f91c81c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91cc993700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c809d6f0 0x7f91c81c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91cc993700 1 -- 172.21.15.32:0/4196989308 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc009550 con 0x7f91c8139170 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91c77fe700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c81c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91cc993700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c8139170 0x7f91c8120790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009d90 tx=0x7f91c000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0009c00 con 0x7f91c8139170 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c000f040 con 0x7f91c8139170 2023-09-30T22:27:48.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c809c710 con 0x7f91c8139170 2023-09-30T22:27:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0014480 con 0x7f91c8139170 2023-09-30T22:27:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.757+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c81c9080 con 0x7f91c8139170 2023-09-30T22:27:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.758+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91c00145e0 con 0x7f91c8139170 2023-09-30T22:27:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.758+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c8123ea0 con 0x7f91c8139170 2023-09-30T22:27:48.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.759+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f91c00401b0 con 0x7f91c8139170 2023-09-30T22:27:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.768+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91c8123ea0 con 0x7f91c8139170 2023-09-30T22:27:48.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.779+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f36e01a94e0 con 0x7f36e009e930 2023-09-30T22:27:48.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.780+0000 7f36dd7fa700 1 -- 172.21.15.32:0/532243643 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f36e01a94e0 con 0x7f36e009e930 2023-09-30T22:27:48.780 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 msgr2=0x7f36e006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 0x7f36e006b8b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e9e0 tx=0x7f36d000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 shutdown_connections 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e009f650 0x7f36e006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36e009e930 0x7f36e006b8b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 --2- 172.21.15.32:0/532243643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36e009f240 0x7f36e006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.784+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 >> 172.21.15.32:0/532243643 conn(0x7f36e00a4210 msgr2=0x7f36e01b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.785+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 shutdown_connections 2023-09-30T22:27:48.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.785+0000 7f36e7067700 1 -- 172.21.15.32:0/532243643 wait complete. 2023-09-30T22:27:48.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:48.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.813+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb8241a7ec0 con 0x7fb82409e930 2023-09-30T22:27:48.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.814+0000 7fb819ffb700 1 -- 172.21.15.32:0/2743430139 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb820040e60 con 0x7fb82409e930 2023-09-30T22:27:48.814 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb82409e930 msgr2=0x7fb82406c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb82409e930 0x7fb82406c0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb820016000 tx=0x7fb820004750 comp rx=0 tx=0).stop 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 shutdown_connections 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb82409f650 0x7fb8241c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb82409e930 0x7fb82406c0e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 --2- 172.21.15.32:0/2743430139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb82409f240 0x7fb82406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 >> 172.21.15.32:0/2743430139 conn(0x7fb8240a4210 msgr2=0x7fb8241b9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 shutdown_connections 2023-09-30T22:27:48.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.818+0000 7fb82b541700 1 -- 172.21.15.32:0/2743430139 wait complete. 2023-09-30T22:27:48.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 Processor -- start 2023-09-30T22:27:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 -- start start 2023-09-30T22:27:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09f410 0x7fae9c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c136fe0 0x7fae9c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c070940 con 0x7fae9c09e930 2023-09-30T22:27:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c06fd60 con 0x7fae9c136fe0 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea45a5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c06fa80 con 0x7fae9c09f410 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.849+0000 7faea2b42700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c136fe0 0x7fae9c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2341700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fae9c09e930 0x7fae9c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48758/0 (socket says 172.21.15.32:48758) 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea1b40700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09f410 0x7fae9c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2341700 1 -- 172.21.15.32:0/1903275953 learned_addr learned my addr 172.21.15.32:0/1903275953 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea133f700 1 -- 172.21.15.32:0/1903275953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403577277 0 0) 0x7fae9c070940 con 0x7fae9c09e930 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2b42700 1 -- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09f410 msgr2=0x7fae9c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2b42700 1 --2- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09f410 0x7fae9c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2b42700 1 -- 172.21.15.32:0/1903275953 >> v1:172.21.15.32:6789/0 conn(0x7fae9c09e930 legacy=0x7fae9c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2b42700 1 -- 172.21.15.32:0/1903275953 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c09fb60 con 0x7fae9c136fe0 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea1b40700 1 --2- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09f410 0x7fae9c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea2b42700 1 --2- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c136fe0 0x7fae9c1393c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009dc0 tx=0x7fae8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ddeb2bc97e3d60 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea133f700 1 -- 172.21.15.32:0/1903275953 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00e070 con 0x7fae9c136fe0 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea133f700 1 -- 172.21.15.32:0/1903275953 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c00bb20 con 0x7fae9c136fe0 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.850+0000 7faea133f700 1 -- 172.21.15.32:0/1903275953 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c012430 con 0x7fae9c136fe0 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 -- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c136fe0 msgr2=0x7fae9c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 --2- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c136fe0 0x7fae9c1393c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009dc0 tx=0x7fae8c00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 -- 172.21.15.32:0/1903275953 shutdown_connections 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 --2- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c136fe0 0x7fae9c1393c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 --2- 172.21.15.32:0/1903275953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09f410 0x7fae9c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 -- 172.21.15.32:0/1903275953 >> 172.21.15.32:0/1903275953 conn(0x7fae9c0a4210 msgr2=0x7fae9c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 -- 172.21.15.32:0/1903275953 shutdown_connections 2023-09-30T22:27:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.851+0000 7faea45a5700 1 -- 172.21.15.32:0/1903275953 wait complete. 2023-09-30T22:27:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.852+0000 7faea45a5700 1 Processor -- start 2023-09-30T22:27:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.852+0000 7faea45a5700 1 -- start start 2023-09-30T22:27:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea45a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09e930 0x7fae9c1c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea45a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 0x7fae9c1c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea45a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c136fe0 0x7fae9c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:48.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea45a5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1325c0 con 0x7fae9c136fe0 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea45a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c06d640 con 0x7fae9c09f410 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea45a5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c120ab0 con 0x7fae9c09e930 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea1b40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 0x7fae9c1c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c136fe0 0x7fae9c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea1b40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 0x7fae9c1c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33908/0 (socket says 172.21.15.32:33908) 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea1b40700 1 -- 172.21.15.32:0/389582050 learned_addr learned my addr 172.21.15.32:0/389582050 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2341700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09e930 0x7fae9c1c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 -- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09e930 msgr2=0x7fae9c1c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09e930 0x7fae9c1c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 -- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 msgr2=0x7fae9c1c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 0x7fae9c1c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 -- 172.21.15.32:0/389582050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900094a0 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea1b40700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 0x7fae9c1c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2341700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09e930 0x7fae9c1c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.853+0000 7faea2b42700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c136fe0 0x7fae9c06b8b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fae8c016000 tx=0x7fae8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.854+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c004840 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.854+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c020070 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.854+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c009a70 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.854+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0276c0 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.854+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c1c8fa0 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.855+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c0a3330 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.855+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae8c012430 con 0x7fae9c136fe0 2023-09-30T22:27:48.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.855+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae8c022070 con 0x7fae9c136fe0 2023-09-30T22:27:48.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.860+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae8c01a740 con 0x7fae9c136fe0 2023-09-30T22:27:48.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.979+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91c8123ea0 con 0x7f91c8139170 2023-09-30T22:27:48.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.979+0000 7f91c4ff9700 1 -- 172.21.15.32:0/4196989308 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91c8123ea0 con 0x7f91c8139170 2023-09-30T22:27:48.980 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c8139170 msgr2=0x7f91c8120790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c8139170 0x7f91c8120790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009d90 tx=0x7f91c000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 shutdown_connections 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91c8139170 0x7f91c8120790 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91c809d6f0 0x7f91c81c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 --2- 172.21.15.32:0/4196989308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91c809e1d0 0x7f91c81c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 >> 172.21.15.32:0/4196989308 conn(0x7f91c8066f50 msgr2=0x7f91c806f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 shutdown_connections 2023-09-30T22:27:48.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:48.982+0000 7f91ce3f6700 1 -- 172.21.15.32:0/4196989308 wait complete. 2023-09-30T22:27:48.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.067+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae9c071d20 con 0x7fae9c136fe0 2023-09-30T22:27:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.067+0000 7fae9affd700 1 -- 172.21.15.32:0/389582050 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae8c030390 con 0x7fae9c136fe0 2023-09-30T22:27:49.068 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c136fe0 msgr2=0x7fae9c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c136fe0 0x7fae9c06b8b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fae8c016000 tx=0x7fae8c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 shutdown_connections 2023-09-30T22:27:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae9c136fe0 0x7fae9c06b8b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae9c09e930 0x7fae9c1c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 --2- 172.21.15.32:0/389582050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae9c09f410 0x7fae9c1c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 >> 172.21.15.32:0/389582050 conn(0x7fae9c0a4210 msgr2=0x7fae9c135c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 shutdown_connections 2023-09-30T22:27:49.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.070+0000 7faea45a5700 1 -- 172.21.15.32:0/389582050 wait complete. 2023-09-30T22:27:49.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:49.800 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:49.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.947+0000 7fbdf951f700 1 Processor -- start 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 -- start start 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdf409e850 0x7fbdf409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 0x7fbdf409f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf4133b80 0x7fbdf4134000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf406bec0 con 0x7fbdf409e850 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40a0030 con 0x7fbdf409f160 2023-09-30T22:27:49.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf951f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf406caa0 con 0x7fbdf4133b80 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 0x7fbdf409f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf4133b80 0x7fbdf4134000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 0x7fbdf409f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33922/0 (socket says 172.21.15.32:33922) 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3329131527 learned_addr learned my addr 172.21.15.32:0/3329131527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.948+0000 7fbdf2ffd700 1 --2- 172.21.15.32:0/3329131527 >> v2:172.21.15.32:3300/0 conn(0x7fbdf409e850 0x7fbdf409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf4133b80 msgr2=0x7fbdf4134000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf27fc700 1 --2- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf4133b80 0x7fbdf4134000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3329131527 >> v2:172.21.15.32:3300/0 conn(0x7fbdf409e850 msgr2=0x7fbdf409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf27fc700 1 --2- 172.21.15.32:0/3329131527 >> v2:172.21.15.32:3300/0 conn(0x7fbdf409e850 0x7fbdf409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3329131527 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40a0830 con 0x7fbdf409f160 2023-09-30T22:27:49.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf2ffd700 1 --2- 172.21.15.32:0/3329131527 >> v2:172.21.15.32:3300/0 conn(0x7fbdf409e850 0x7fbdf409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:49.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf27fc700 1 --2- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 0x7fbdf409f570 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbde80098f0 tx=0x7fbde8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a767eeda9358c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:49.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf1ffb700 1 -- 172.21.15.32:0/3329131527 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8011830 con 0x7fbdf409f160 2023-09-30T22:27:49.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf1ffb700 1 -- 172.21.15.32:0/3329131527 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde8011990 con 0x7fbdf409f160 2023-09-30T22:27:49.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.949+0000 7fbdf1ffb700 1 -- 172.21.15.32:0/3329131527 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8011d20 con 0x7fbdf409f160 2023-09-30T22:27:49.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 -- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 msgr2=0x7fbdf409f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 0x7fbdf409f570 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbde80098f0 tx=0x7fbde8009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 -- 172.21.15.32:0/3329131527 shutdown_connections 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf4133b80 0x7fbdf4134000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3329131527 >> v2:172.21.15.32:3300/0 conn(0x7fbdf409e850 0x7fbdf409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3329131527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf409f160 0x7fbdf409f570 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 -- 172.21.15.32:0/3329131527 >> 172.21.15.32:0/3329131527 conn(0x7fbdf40a3f70 msgr2=0x7fbdf40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 -- 172.21.15.32:0/3329131527 shutdown_connections 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.950+0000 7fbdf951f700 1 -- 172.21.15.32:0/3329131527 wait complete. 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.951+0000 7fbdf951f700 1 Processor -- start 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.951+0000 7fbdf951f700 1 -- start start 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf951f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 0x7fbdf4120270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf951f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf409f160 0x7fbdf41207b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf951f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf4133b80 0x7fbdf4120cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf951f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4121e40 con 0x7fbdf409f160 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf951f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf406ccc0 con 0x7fbdf4133b80 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf951f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf41237b0 con 0x7fbdf409e850 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 0x7fbdf4120270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf409f160 0x7fbdf41207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 0x7fbdf4120270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57588/0 (socket says 172.21.15.32:57588) 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf2ffd700 1 -- 172.21.15.32:0/3767541293 learned_addr learned my addr 172.21.15.32:0/3767541293 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf37fe700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf4133b80 0x7fbdf4120cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 msgr2=0x7fbdf4120270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 0x7fbdf4120270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf4133b80 msgr2=0x7fbdf4120cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf4133b80 0x7fbdf4120cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 -- 172.21.15.32:0/3767541293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009a70 con 0x7fbdf409f160 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf2ffd700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 0x7fbdf4120270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf37fe700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf4133b80 0x7fbdf4120cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbdf27fc700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf409f160 0x7fbdf41207b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbde8016000 tx=0x7fbde8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.952+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8004240 con 0x7fbdf409f160 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.953+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde801f070 con 0x7fbdf409f160 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.953+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde8009550 con 0x7fbdf409f160 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.953+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde80045e0 con 0x7fbdf409f160 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.953+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf4123de0 con 0x7fbdf409f160 2023-09-30T22:27:49.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.953+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbde8021030 con 0x7fbdf409f160 2023-09-30T22:27:49.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.954+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbde8040760 con 0x7fbdf409f160 2023-09-30T22:27:49.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.954+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf409caf0 con 0x7fbdf409f160 2023-09-30T22:27:49.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.959+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbde801a450 con 0x7fbdf409f160 2023-09-30T22:27:49.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.987+0000 7f809e360700 1 Processor -- start 2023-09-30T22:27:49.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.987+0000 7f809e360700 1 -- start start 2023-09-30T22:27:49.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809e360700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80980a0a30 0x7f80980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809e360700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f809813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809e360700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980a2a70 con 0x7f809809d630 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809e360700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80980a01d0 con 0x7f8098138d80 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809e360700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f809809e2b0 con 0x7f80980a0a30 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f809813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f809813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33940/0 (socket says 172.21.15.32:33940) 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 -- 172.21.15.32:0/48260609 learned_addr learned my addr 172.21.15.32:0/48260609 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f8097fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80980a0a30 0x7f80980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 -- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80980a0a30 msgr2=0x7f80980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 --2- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80980a0a30 0x7f80980a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 -- 172.21.15.32:0/48260609 >> v1:172.21.15.32:6789/0 conn(0x7f809809d630 legacy=0x7f80980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 -- 172.21.15.32:0/48260609 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809809dc50 con 0x7f8098138d80 2023-09-30T22:27:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f8096ffd700 1 -- 172.21.15.32:0/48260609 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671452639 0 0) 0x7f80980a2a70 con 0x7f809809d630 2023-09-30T22:27:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f8097fff700 1 --2- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80980a0a30 0x7f80980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.988+0000 7f809c8fd700 1 --2- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f809813b160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8090009e80 tx=0x7f809000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7dbd9ec0614822f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.989+0000 7f8096ffd700 1 -- 172.21.15.32:0/48260609 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000e070 con 0x7f8098138d80 2023-09-30T22:27:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.989+0000 7f8096ffd700 1 -- 172.21.15.32:0/48260609 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000b910 con 0x7f8098138d80 2023-09-30T22:27:49.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.989+0000 7f8096ffd700 1 -- 172.21.15.32:0/48260609 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000bcc0 con 0x7f8098138d80 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.989+0000 7f809e360700 1 -- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 msgr2=0x7f809813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.989+0000 7f809e360700 1 --2- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f809813b160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8090009e80 tx=0x7f809000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.990+0000 7f809e360700 1 -- 172.21.15.32:0/48260609 shutdown_connections 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.990+0000 7f809e360700 1 --2- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f809813b160 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.990+0000 7f809e360700 1 --2- 172.21.15.32:0/48260609 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f80980a0a30 0x7f80980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.990+0000 7f809e360700 1 -- 172.21.15.32:0/48260609 >> 172.21.15.32:0/48260609 conn(0x7f80980a3050 msgr2=0x7f80980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.990+0000 7f809e360700 1 -- 172.21.15.32:0/48260609 shutdown_connections 2023-09-30T22:27:49.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.990+0000 7f809e360700 1 -- 172.21.15.32:0/48260609 wait complete. 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 Processor -- start 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 -- start start 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 0x7f8098120450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 0x7f8098120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f8098120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8098121920 con 0x7f80980a0a30 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8098123f70 con 0x7f8098138d80 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f809e360700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8098123700 con 0x7f809809d630 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f80977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 0x7f8098120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.991+0000 7f8097fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 0x7f8098120450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 0x7f8098120990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56160/0 (socket says 172.21.15.32:56160) 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f8097fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 0x7f8098120450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57598/0 (socket says 172.21.15.32:57598) 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 -- 172.21.15.32:0/3176432372 learned_addr learned my addr 172.21.15.32:0/3176432372 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f809c8fd700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f8098120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 -- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 msgr2=0x7f8098120450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 0x7f8098120450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 -- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 msgr2=0x7f8098120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f8098120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 -- 172.21.15.32:0/3176432372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880094a0 con 0x7f80980a0a30 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f8097fff700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 0x7f8098120450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f809c8fd700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f8098120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f80977fe700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 0x7f8098120990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c580 tx=0x7f808c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:49.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0054e0 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c003680 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c003a30 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8090009b30 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.992+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80980a0460 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.993+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f808c003b90 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.993+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8098123af0 con 0x7f80980a0a30 2023-09-30T22:27:49.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:49.993+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f808c03ea80 con 0x7f80980a0a30 2023-09-30T22:27:49.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:50.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.003+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8098123af0 con 0x7f80980a0a30 2023-09-30T22:27:50.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.139+0000 7fda6110e700 1 Processor -- start 2023-09-30T22:27:50.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.139+0000 7fda6110e700 1 -- start start 2023-09-30T22:27:50.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda6110e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda6110e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e000 0x7fda5c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda6110e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda5c09e410 0x7fda5c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda6110e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c06b460 con 0x7fda5c09e410 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda6110e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c09b120 con 0x7fda5c09d6f0 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda6110e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c09a7e0 con 0x7fda5c09e000 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda5c09e410 0x7fda5c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda5c09e410 0x7fda5c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56166/0 (socket says 172.21.15.32:56166) 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 -- 172.21.15.32:0/1469240682 learned_addr learned my addr 172.21.15.32:0/1469240682 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 -- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e000 msgr2=0x7fda5c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:50.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e000 0x7fda5c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 -- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 msgr2=0x7fda5c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5b59e700 1 -- 172.21.15.32:0/1469240682 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c09a500 con 0x7fda5c09e410 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.140+0000 7fda5ad9d700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.141+0000 7fda5b59e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e410 0x7fda5c133f30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fda54009e50 tx=0x7fda5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baee4d947a98375c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.141+0000 7fda59d9b700 1 -- 172.21.15.32:0/1469240682 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400e070 con 0x7fda5c09e410 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.141+0000 7fda59d9b700 1 -- 172.21.15.32:0/1469240682 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5400b910 con 0x7fda5c09e410 2023-09-30T22:27:50.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.141+0000 7fda59d9b700 1 -- 172.21.15.32:0/1469240682 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400bcc0 con 0x7fda5c09e410 2023-09-30T22:27:50.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.141+0000 7fda6110e700 1 -- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e410 msgr2=0x7fda5c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.141+0000 7fda6110e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e410 0x7fda5c133f30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fda54009e50 tx=0x7fda5400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:50.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 -- 172.21.15.32:0/1469240682 shutdown_connections 2023-09-30T22:27:50.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e410 0x7fda5c133f30 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 --2- 172.21.15.32:0/1469240682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e000 0x7fda5c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 -- 172.21.15.32:0/1469240682 >> 172.21.15.32:0/1469240682 conn(0x7fda5c066f50 msgr2=0x7fda5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 -- 172.21.15.32:0/1469240682 shutdown_connections 2023-09-30T22:27:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.142+0000 7fda6110e700 1 -- 172.21.15.32:0/1469240682 wait complete. 2023-09-30T22:27:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.143+0000 7fda6110e700 1 Processor -- start 2023-09-30T22:27:50.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.143+0000 7fda6110e700 1 -- start start 2023-09-30T22:27:50.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda6110e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda6110e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e000 0x7fda5c1a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda6110e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e410 0x7fda5c1a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda6110e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c1a72d0 con 0x7fda5c09e000 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda6110e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c1a7a30 con 0x7fda5c09d6f0 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda6110e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c1a8c80 con 0x7fda5c09e410 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c1a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:33960/0 (socket says 172.21.15.32:33960) 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5ad9d700 1 -- 172.21.15.32:0/1225312347 learned_addr learned my addr 172.21.15.32:0/1225312347 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5b59e700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e410 0x7fda5c1a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5a59c700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e000 0x7fda5c1a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5ad9d700 1 -- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e410 msgr2=0x7fda5c1a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.144+0000 7fda5ad9d700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e410 0x7fda5c1a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda5ad9d700 1 -- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e000 msgr2=0x7fda5c1a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda5ad9d700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e000 0x7fda5c1a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda5ad9d700 1 -- 172.21.15.32:0/1225312347 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0094a0 con 0x7fda5c09d6f0 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda5a59c700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e000 0x7fda5c1a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda5ad9d700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c1a5e20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00ec40 tx=0x7fda4c009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00c970 con 0x7fda5c09d6f0 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00cad0 con 0x7fda5c09d6f0 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54009b30 con 0x7fda5c09d6f0 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00ce40 con 0x7fda5c09d6f0 2023-09-30T22:27:50.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.145+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c1a96b0 con 0x7fda5c09d6f0 2023-09-30T22:27:50.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.147+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda4c013470 con 0x7fda5c09d6f0 2023-09-30T22:27:50.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.147+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c09c710 con 0x7fda5c09d6f0 2023-09-30T22:27:50.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.147+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda4c03f7e0 con 0x7fda5c09d6f0 2023-09-30T22:27:50.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.156+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda4c013720 con 0x7fda5c09d6f0 2023-09-30T22:27:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.184+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbdf40747f0 con 0x7fbdf409f160 2023-09-30T22:27:50.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.185+0000 7fbddb7fe700 1 -- 172.21.15.32:0/3767541293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbde80233d0 con 0x7fbdf409f160 2023-09-30T22:27:50.184 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf409f160 msgr2=0x7fbdf41207b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf409f160 0x7fbdf41207b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbde8016000 tx=0x7fbde8004750 comp rx=0 tx=0).stop 2023-09-30T22:27:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 shutdown_connections 2023-09-30T22:27:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdf4133b80 0x7fbdf4120cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdf409e850 0x7fbdf4120270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 --2- 172.21.15.32:0/3767541293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdf409f160 0x7fbdf41207b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 >> 172.21.15.32:0/3767541293 conn(0x7fbdf40a3f70 msgr2=0x7fbdf41baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 shutdown_connections 2023-09-30T22:27:50.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.187+0000 7fbdf951f700 1 -- 172.21.15.32:0/3767541293 wait complete. 2023-09-30T22:27:50.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.228+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8098123af0 con 0x7f80980a0a30 2023-09-30T22:27:50.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.229+0000 7f8094ff9700 1 -- 172.21.15.32:0/3176432372 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8098123af0 con 0x7f80980a0a30 2023-09-30T22:27:50.229 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 msgr2=0x7f8098120990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 0x7f8098120990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c580 tx=0x7f808c00c890 comp rx=0 tx=0).stop 2023-09-30T22:27:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 shutdown_connections 2023-09-30T22:27:50.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8098138d80 0x7f8098120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f809809d630 0x7f8098120450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 --2- 172.21.15.32:0/3176432372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f80980a0a30 0x7f8098120990 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 >> 172.21.15.32:0/3176432372 conn(0x7f80980a3050 msgr2=0x7f809809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 shutdown_connections 2023-09-30T22:27:50.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.231+0000 7f809e360700 1 -- 172.21.15.32:0/3176432372 wait complete. 2023-09-30T22:27:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.241+0000 7fd6ae7da700 1 Processor -- start 2023-09-30T22:27:50.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.241+0000 7fd6ae7da700 1 -- start start 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6ae7da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6a8134630 0x7fd6a8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6ae7da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6ae7da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a809b4f0 con 0x7fd6a8131810 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6ae7da700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a81313c0 con 0x7fd6a8134630 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6ae7da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a806e8c0 con 0x7fd6a8136f50 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd6a8134630 0x7fd6a8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6a8131810 0x7fd6a8131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48784/0 (socket says 172.21.15.32:48784) 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a7fff700 1 -- 172.21.15.32:0/3422800350 learned_addr learned my addr 172.21.15.32:0/3422800350 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:50.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6acd77700 1 --2- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a77fe700 1 -- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 msgr2=0x7fd6a8139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a77fe700 1 --2- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a8139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a77fe700 1 -- 172.21.15.32:0/3422800350 >> v1:172.21.15.32:6789/0 conn(0x7fd6a8131810 legacy=0x7fd6a8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a6ffd700 1 -- 172.21.15.32:0/3422800350 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788914054 0 0) 0x7fd6a809b4f0 con 0x7fd6a8131810 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.242+0000 7fd6a77fe700 1 -- 172.21.15.32:0/3422800350 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8071d20 con 0x7fd6a8134630 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.243+0000 7fd6a77fe700 1 --2- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8134630 0x7fd6a8136a10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009840 tx=0x7fd69c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2951648597f80cc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.243+0000 7fd6a6ffd700 1 -- 172.21.15.32:0/3422800350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c011830 con 0x7fd6a8134630 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.243+0000 7fd6a6ffd700 1 -- 172.21.15.32:0/3422800350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c011990 con 0x7fd6a8134630 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.243+0000 7fd6a6ffd700 1 -- 172.21.15.32:0/3422800350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c011d20 con 0x7fd6a8134630 2023-09-30T22:27:50.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 -- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8134630 msgr2=0x7fd6a8136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8134630 0x7fd6a8136a10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009840 tx=0x7fd69c009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 -- 172.21.15.32:0/3422800350 shutdown_connections 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a8139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/3422800350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8134630 0x7fd6a8136a10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 -- 172.21.15.32:0/3422800350 >> 172.21.15.32:0/3422800350 conn(0x7fd6a80a2aa0 msgr2=0x7fd6a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 -- 172.21.15.32:0/3422800350 shutdown_connections 2023-09-30T22:27:50.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.244+0000 7fd6ae7da700 1 -- 172.21.15.32:0/3422800350 wait complete. 2023-09-30T22:27:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.245+0000 7fd6ae7da700 1 Processor -- start 2023-09-30T22:27:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 -- start start 2023-09-30T22:27:50.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 0x7fd6a806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6a8134630 0x7fd6a80704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a81c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a812e940 con 0x7fd6a8131810 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a806fac0 con 0x7fd6a8136f50 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6ae7da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a806e6b0 con 0x7fd6a8134630 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 0x7fd6a806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.246+0000 7fd6a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 0x7fd6a806ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56194/0 (socket says 172.21.15.32:56194) 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 -- 172.21.15.32:0/1583534195 learned_addr learned my addr 172.21.15.32:0/1583534195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6acd77700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a81c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a77fe700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6a8134630 0x7fd6a80704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 -- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6a8134630 msgr2=0x7fd6a80704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6a8134630 0x7fd6a80704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 -- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 msgr2=0x7fd6a81c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a81c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 -- 172.21.15.32:0/1583534195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0094a0 con 0x7fd6a8131810 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6acd77700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a81c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a7fff700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 0x7fd6a806ffb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c520 tx=0x7fd69800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698004c80 con 0x7fd6a8131810 2023-09-30T22:27:50.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd698004e00 con 0x7fd6a8131810 2023-09-30T22:27:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a811e540 con 0x7fd6a8131810 2023-09-30T22:27:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698007310 con 0x7fd6a8131810 2023-09-30T22:27:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.247+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a80710a0 con 0x7fd6a8131810 2023-09-30T22:27:50.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.248+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd698007470 con 0x7fd6a8131810 2023-09-30T22:27:50.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.249+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd69803f2d0 con 0x7fd6a8131810 2023-09-30T22:27:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.249+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a80721a0 con 0x7fd6a8131810 2023-09-30T22:27:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.254+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6980123b0 con 0x7fd6a8131810 2023-09-30T22:27:50.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.367+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fda5c1a96b0 con 0x7fda5c09d6f0 2023-09-30T22:27:50.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.367+0000 7fda437fe700 1 -- 172.21.15.32:0/1225312347 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda5c1a96b0 con 0x7fda5c09d6f0 2023-09-30T22:27:50.367 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:50.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 msgr2=0x7fda5c1a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c1a5e20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00ec40 tx=0x7fda4c009ec0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 shutdown_connections 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda5c09e410 0x7fda5c1a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda5c09d6f0 0x7fda5c1a5e20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 --2- 172.21.15.32:0/1225312347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda5c09e000 0x7fda5c1a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 >> 172.21.15.32:0/1225312347 conn(0x7fda5c066f50 msgr2=0x7fda5c1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 shutdown_connections 2023-09-30T22:27:50.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.369+0000 7fda6110e700 1 -- 172.21.15.32:0/1225312347 wait complete. 2023-09-30T22:27:50.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:50.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.459+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd6a8074ea0 con 0x7fd6a8131810 2023-09-30T22:27:50.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.459+0000 7fd6a4ff9700 1 -- 172.21.15.32:0/1583534195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6980102c0 con 0x7fd6a8131810 2023-09-30T22:27:50.459 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:50.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 msgr2=0x7fd6a806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:50.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 0x7fd6a806ffb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c520 tx=0x7fd69800c830 comp rx=0 tx=0).stop 2023-09-30T22:27:50.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 shutdown_connections 2023-09-30T22:27:50.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6a8136f50 0x7fd6a81c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6a8131810 0x7fd6a806ffb0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 --2- 172.21.15.32:0/1583534195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd6a8134630 0x7fd6a80704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 >> 172.21.15.32:0/1583534195 conn(0x7fd6a80a2aa0 msgr2=0x7fd6a8133bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 shutdown_connections 2023-09-30T22:27:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:50.461+0000 7fd6ae7da700 1 -- 172.21.15.32:0/1583534195 wait complete. 2023-09-30T22:27:50.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:51.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:51.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.359+0000 7fef1e38b700 1 Processor -- start 2023-09-30T22:27:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.359+0000 7fef1e38b700 1 -- start start 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.359+0000 7fef1e38b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1e38b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef18134170 0x7fef1813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1e38b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1809edd0 con 0x7fef180a1340 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1e38b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1809e1f0 con 0x7fef180a0a30 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1e38b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef1809dc50 con 0x7fef18134170 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef177fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef180a1340 0x7fef18133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60134/0 (socket says 172.21.15.32:60134) 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef177fe700 1 -- 172.21.15.32:0/4246669516 learned_addr learned my addr 172.21.15.32:0/4246669516 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1c928700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef18134170 0x7fef1813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef17fff700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef16ffd700 1 -- 172.21.15.32:0/4246669516 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512139012 0 0) 0x7fef1809edd0 con 0x7fef180a1340 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1c928700 1 -- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 msgr2=0x7fef180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1c928700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1c928700 1 -- 172.21.15.32:0/4246669516 >> v1:172.21.15.32:6789/0 conn(0x7fef180a1340 legacy=0x7fef18133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1c928700 1 -- 172.21.15.32:0/4246669516 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180a1fe0 con 0x7fef18134170 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef17fff700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.360+0000 7fef1c928700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef18134170 0x7fef1813bd30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fef10009dc0 tx=0x7fef1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d080d074300632a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.361+0000 7fef16ffd700 1 -- 172.21.15.32:0/4246669516 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000e070 con 0x7fef18134170 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.361+0000 7fef16ffd700 1 -- 172.21.15.32:0/4246669516 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1000bb20 con 0x7fef18134170 2023-09-30T22:27:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.361+0000 7fef16ffd700 1 -- 172.21.15.32:0/4246669516 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10012430 con 0x7fef18134170 2023-09-30T22:27:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 -- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef18134170 msgr2=0x7fef1813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef18134170 0x7fef1813bd30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fef10009dc0 tx=0x7fef1000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 -- 172.21.15.32:0/4246669516 shutdown_connections 2023-09-30T22:27:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef18134170 0x7fef1813bd30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 --2- 172.21.15.32:0/4246669516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 -- 172.21.15.32:0/4246669516 >> 172.21.15.32:0/4246669516 conn(0x7fef180a6550 msgr2=0x7fef180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 -- 172.21.15.32:0/4246669516 shutdown_connections 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.362+0000 7fef1e38b700 1 -- 172.21.15.32:0/4246669516 wait complete. 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.363+0000 7fef1e38b700 1 Processor -- start 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.363+0000 7fef1e38b700 1 -- start start 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1e38b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef181227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1e38b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef180a1340 0x7fef18122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1e38b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef18134170 0x7fef18123250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1e38b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181244d0 con 0x7fef18134170 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1e38b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef18133790 con 0x7fef180a0a30 2023-09-30T22:27:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1e38b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef18125cc0 con 0x7fef180a1340 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef17fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef181227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef17fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef181227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44798/0 (socket says 172.21.15.32:44798) 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef17fff700 1 -- 172.21.15.32:0/1248433763 learned_addr learned my addr 172.21.15.32:0/1248433763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef1c928700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef18134170 0x7fef18123250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.364+0000 7fef177fe700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef180a1340 0x7fef18122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef17fff700 1 -- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef180a1340 msgr2=0x7fef18122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef17fff700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef180a1340 0x7fef18122d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef17fff700 1 -- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef18134170 msgr2=0x7fef18123250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef17fff700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef18134170 0x7fef18123250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef17fff700 1 -- 172.21.15.32:0/1248433763 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080094a0 con 0x7fef180a0a30 2023-09-30T22:27:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef17fff700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef181227d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fef0800ebe0 tx=0x7fef0800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0800c850 con 0x7fef180a0a30 2023-09-30T22:27:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0800c9b0 con 0x7fef180a0a30 2023-09-30T22:27:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef10009a70 con 0x7fef180a0a30 2023-09-30T22:27:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0800cd40 con 0x7fef180a0a30 2023-09-30T22:27:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.365+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef180a25e0 con 0x7fef180a0a30 2023-09-30T22:27:51.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.366+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef08013470 con 0x7fef180a0a30 2023-09-30T22:27:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.366+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef0803fae0 con 0x7fef180a0a30 2023-09-30T22:27:51.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.366+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef180a0280 con 0x7fef180a0a30 2023-09-30T22:27:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.371+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef08013720 con 0x7fef180a0a30 2023-09-30T22:27:51.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.398+0000 7ff564df2700 1 Processor -- start 2023-09-30T22:27:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.398+0000 7ff564df2700 1 -- start start 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff564df2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff560138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff564df2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff56013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff564df2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56006ef30 con 0x7ff5601338a0 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff564df2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56006fef0 con 0x7ff560139030 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff564df2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5601334b0 con 0x7ff560136710 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff560138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff560138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43260/0 (socket says 172.21.15.32:43260) 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 -- 172.21.15.32:0/181892097 learned_addr learned my addr 172.21.15.32:0/181892097 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55ed9d700 1 --2- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff56013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 -- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 msgr2=0x7ff56013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 --2- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff56013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 -- 172.21.15.32:0/181892097 >> v1:172.21.15.32:6789/0 conn(0x7ff5601338a0 legacy=0x7ff560133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 -- 172.21.15.32:0/181892097 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56006fc30 con 0x7ff560136710 2023-09-30T22:27:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.399+0000 7ff55dd9b700 1 --2- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff560138af0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff554009920 tx=0x7ff554009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f8f80a48e6235d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.400+0000 7ff55d59a700 1 -- 172.21.15.32:0/181892097 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554004230 con 0x7ff560136710 2023-09-30T22:27:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.400+0000 7ff55d59a700 1 -- 172.21.15.32:0/181892097 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554004390 con 0x7ff560136710 2023-09-30T22:27:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.400+0000 7ff55d59a700 1 -- 172.21.15.32:0/181892097 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5540036a0 con 0x7ff560136710 2023-09-30T22:27:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 -- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 msgr2=0x7ff560138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 --2- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff560138af0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff554009920 tx=0x7ff554009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:51.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 -- 172.21.15.32:0/181892097 shutdown_connections 2023-09-30T22:27:51.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 --2- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff56013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 --2- 172.21.15.32:0/181892097 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff560138af0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 -- 172.21.15.32:0/181892097 >> 172.21.15.32:0/181892097 conn(0x7ff5600a2aa0 msgr2=0x7ff5600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 -- 172.21.15.32:0/181892097 shutdown_connections 2023-09-30T22:27:51.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.401+0000 7ff564df2700 1 -- 172.21.15.32:0/181892097 wait complete. 2023-09-30T22:27:51.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.402+0000 7ff564df2700 1 Processor -- start 2023-09-30T22:27:51.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 -- start start 2023-09-30T22:27:51.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5601338a0 0x7ff56011aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff56011afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff5601c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff560138b70 con 0x7ff5601338a0 2023-09-30T22:27:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56011d680 con 0x7ff560139030 2023-09-30T22:27:51.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff564df2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff56011b510 con 0x7ff560136710 2023-09-30T22:27:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.403+0000 7ff55dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff56011afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff56011afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43276/0 (socket says 172.21.15.32:43276) 2023-09-30T22:27:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55dd9b700 1 -- 172.21.15.32:0/5072401 learned_addr learned my addr 172.21.15.32:0/5072401 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff5601c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55e59c700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5601338a0 0x7ff56011aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 -- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 msgr2=0x7ff56011afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff56011afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 -- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5601338a0 msgr2=0x7ff56011aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5601338a0 0x7ff56011aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 -- 172.21.15.32:0/5072401 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554009550 con 0x7ff560139030 2023-09-30T22:27:51.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55e59c700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5601338a0 0x7ff56011aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff55ed9d700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff5601c8240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff55800efc0 tx=0x7ff55800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55800cc30 con 0x7ff560139030 2023-09-30T22:27:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55800f040 con 0x7ff560139030 2023-09-30T22:27:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.404+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56011c6c0 con 0x7ff560139030 2023-09-30T22:27:51.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.405+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558009e60 con 0x7ff560139030 2023-09-30T22:27:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.405+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56011d3c0 con 0x7ff560139030 2023-09-30T22:27:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.405+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff558014480 con 0x7ff560139030 2023-09-30T22:27:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.406+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff558041750 con 0x7ff560139030 2023-09-30T22:27:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.407+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56011e1b0 con 0x7ff560139030 2023-09-30T22:27:51.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.416+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff558018020 con 0x7ff560139030 2023-09-30T22:27:51.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:51.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.527+0000 7fa57bcc5700 1 Processor -- start 2023-09-30T22:27:51.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 -- start start 2023-09-30T22:27:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5741338a0 0x7fa574133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57406fef0 con 0x7fa5741338a0 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57406ef30 con 0x7fa57409e760 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.528+0000 7fa57bcc5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741334b0 con 0x7fa57409de50 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43282/0 (socket says 172.21.15.32:43282) 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 -- 172.21.15.32:0/3135552815 learned_addr learned my addr 172.21.15.32:0/3135552815 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa57a262700 1 --2- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579a61700 1 --2- 172.21.15.32:0/3135552815 >> v2:172.21.15.32:3300/0 conn(0x7fa5741338a0 0x7fa574133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 -- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 msgr2=0x7fa57413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 --2- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 -- 172.21.15.32:0/3135552815 >> v2:172.21.15.32:3300/0 conn(0x7fa5741338a0 msgr2=0x7fa574133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 --2- 172.21.15.32:0/3135552815 >> v2:172.21.15.32:3300/0 conn(0x7fa5741338a0 0x7fa574133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 -- 172.21.15.32:0/3135552815 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57406fc30 con 0x7fa57409de50 2023-09-30T22:27:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa579260700 1 --2- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57409e220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa5640098f0 tx=0x7fa564009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce05b88e7b25258 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.529+0000 7fa578a5f700 1 -- 172.21.15.32:0/3135552815 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011830 con 0x7fa57409de50 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa578a5f700 1 -- 172.21.15.32:0/3135552815 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564011990 con 0x7fa57409de50 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa578a5f700 1 -- 172.21.15.32:0/3135552815 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011d20 con 0x7fa57409de50 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 msgr2=0x7fa57409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57409e220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa5640098f0 tx=0x7fa564009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3135552815 shutdown_connections 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3135552815 >> v2:172.21.15.32:3300/0 conn(0x7fa5741338a0 0x7fa574133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3135552815 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57409e220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3135552815 >> 172.21.15.32:0/3135552815 conn(0x7fa5740a2aa0 msgr2=0x7fa5740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.530+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3135552815 shutdown_connections 2023-09-30T22:27:51.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.531+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3135552815 wait complete. 2023-09-30T22:27:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.531+0000 7fa57bcc5700 1 Processor -- start 2023-09-30T22:27:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 -- start start 2023-09-30T22:27:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57411b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57411bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 0x7fa57411c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57411cb50 con 0x7fa5741338a0 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57411d2b0 con 0x7fa57409e760 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57bcc5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa57411d8d0 con 0x7fa57409de50 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57a262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 0x7fa57411c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57a262700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 0x7fa57411c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58548/0 (socket says 172.21.15.32:58548) 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa57a262700 1 -- 172.21.15.32:0/3706470608 learned_addr learned my addr 172.21.15.32:0/3706470608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.532+0000 7fa579260700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57411bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57a262700 1 -- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 msgr2=0x7fa57411b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57a262700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57411b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57a262700 1 -- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 msgr2=0x7fa57411bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57a262700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57411bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57a262700 1 -- 172.21.15.32:0/3706470608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009550 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57a262700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 0x7fa57411c120 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009dd0 tx=0x7fa56c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c009c40 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00f040 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57411eee0 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c014480 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.533+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740a1000 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.535+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57406f6a0 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.535+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa56c0145e0 con 0x7fa5741338a0 2023-09-30T22:27:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.536+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa56c0406e0 con 0x7fa5741338a0 2023-09-30T22:27:51.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.544+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa56c012930 con 0x7fa5741338a0 2023-09-30T22:27:51.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.590+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fef18074ea0 con 0x7fef180a0a30 2023-09-30T22:27:51.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.591+0000 7fef14ff9700 1 -- 172.21.15.32:0/1248433763 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef08017020 con 0x7fef180a0a30 2023-09-30T22:27:51.591 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 msgr2=0x7fef181227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef181227d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fef0800ebe0 tx=0x7fef0800eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 shutdown_connections 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef18134170 0x7fef18123250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef180a0a30 0x7fef181227d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 --2- 172.21.15.32:0/1248433763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef180a1340 0x7fef18122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 >> 172.21.15.32:0/1248433763 conn(0x7fef180a6550 msgr2=0x7fef1809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 shutdown_connections 2023-09-30T22:27:51.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.594+0000 7fef1e38b700 1 -- 172.21.15.32:0/1248433763 wait complete. 2023-09-30T22:27:51.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.620+0000 7fe4da95c700 1 Processor -- start 2023-09-30T22:27:51.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.620+0000 7fe4da95c700 1 -- start start 2023-09-30T22:27:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4da95c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 0x7fe4d4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4da95c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e560 0x7fe4d413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4da95c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d406f720 con 0x7fe4d41341d0 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4da95c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d4070c80 con 0x7fe4d41338c0 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4da95c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d406fce0 con 0x7fe4d409e560 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 0x7fe4d4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 0x7fe4d4133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44838/0 (socket says 172.21.15.32:44838) 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 -- 172.21.15.32:0/1174007861 learned_addr learned my addr 172.21.15.32:0/1174007861 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d8ef9700 1 --2- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e560 0x7fe4d413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 -- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e560 msgr2=0x7fe4d413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 --2- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e560 0x7fe4d413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 -- 172.21.15.32:0/1174007861 >> v1:172.21.15.32:6789/0 conn(0x7fe4d41341d0 legacy=0x7fe4d409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d2ffd700 1 -- 172.21.15.32:0/1174007861 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250155112 0 0) 0x7fe4d406f720 con 0x7fe4d41341d0 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.621+0000 7fe4d3fff700 1 -- 172.21.15.32:0/1174007861 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d41336a0 con 0x7fe4d41338c0 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.622+0000 7fe4d3fff700 1 --2- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 0x7fe4d4133c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009920 tx=0x7fe4c4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875fc90a2592914b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.622+0000 7fe4d2ffd700 1 -- 172.21.15.32:0/1174007861 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4011830 con 0x7fe4d41338c0 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.622+0000 7fe4d2ffd700 1 -- 172.21.15.32:0/1174007861 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c4011990 con 0x7fe4d41338c0 2023-09-30T22:27:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.622+0000 7fe4d2ffd700 1 -- 172.21.15.32:0/1174007861 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4011d20 con 0x7fe4d41338c0 2023-09-30T22:27:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.622+0000 7fe4da95c700 1 -- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 msgr2=0x7fe4d4133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.622+0000 7fe4da95c700 1 --2- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 0x7fe4d4133c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009920 tx=0x7fe4c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.623+0000 7fe4da95c700 1 -- 172.21.15.32:0/1174007861 shutdown_connections 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.623+0000 7fe4da95c700 1 --2- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d409e560 0x7fe4d413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.623+0000 7fe4da95c700 1 --2- 172.21.15.32:0/1174007861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41338c0 0x7fe4d4133c90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.623+0000 7fe4da95c700 1 -- 172.21.15.32:0/1174007861 >> 172.21.15.32:0/1174007861 conn(0x7fe4d40a2aa0 msgr2=0x7fe4d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.623+0000 7fe4da95c700 1 -- 172.21.15.32:0/1174007861 shutdown_connections 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.623+0000 7fe4da95c700 1 -- 172.21.15.32:0/1174007861 wait complete. 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 Processor -- start 2023-09-30T22:27:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 -- start start 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 0x7fe4d411e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d41338c0 0x7fe4d411ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41341d0 0x7fe4d411f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d412fdc0 con 0x7fe4d409e560 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d411fcc0 con 0x7fe4d41341d0 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.624+0000 7fe4da95c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d406f960 con 0x7fe4d41338c0 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 0x7fe4d411e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d41338c0 0x7fe4d411ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 0x7fe4d411e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58552/0 (socket says 172.21.15.32:58552) 2023-09-30T22:27:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d3fff700 1 -- 172.21.15.32:0/3728733736 learned_addr learned my addr 172.21.15.32:0/3728733736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d8ef9700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41341d0 0x7fe4d411f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 -- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41341d0 msgr2=0x7fe4d411f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41341d0 0x7fe4d411f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 -- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 msgr2=0x7fe4d411e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 0x7fe4d411e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 -- 172.21.15.32:0/3728733736 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c4009550 con 0x7fe4d41338c0 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d3fff700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 0x7fe4d411e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d37fe700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d41338c0 0x7fe4d411ee10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40a35d0 tx=0x7fe4c800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80052d0 con 0x7fe4d41338c0 2023-09-30T22:27:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8005450 con 0x7fe4d41338c0 2023-09-30T22:27:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.625+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d4121720 con 0x7fe4d41338c0 2023-09-30T22:27:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.626+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80038c0 con 0x7fe4d41338c0 2023-09-30T22:27:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.626+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d4121e20 con 0x7fe4d41338c0 2023-09-30T22:27:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.626+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe4c8003a20 con 0x7fe4d41338c0 2023-09-30T22:27:51.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.627+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d4074ea0 con 0x7fe4d41338c0 2023-09-30T22:27:51.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.627+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe4c803e9f0 con 0x7fe4d41338c0 2023-09-30T22:27:51.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.631+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe4c8040e60 con 0x7fe4d41338c0 2023-09-30T22:27:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.646+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff56006ea40 con 0x7ff560139030 2023-09-30T22:27:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.647+0000 7ff54effd700 1 -- 172.21.15.32:0/5072401 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff558045060 con 0x7ff560139030 2023-09-30T22:27:51.647 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 msgr2=0x7ff5601c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff5601c8240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff55800efc0 tx=0x7ff55800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 shutdown_connections 2023-09-30T22:27:51.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff560139030 0x7ff5601c8240 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5601338a0 0x7ff56011aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 --2- 172.21.15.32:0/5072401 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff560136710 0x7ff56011afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 >> 172.21.15.32:0/5072401 conn(0x7ff5600a2aa0 msgr2=0x7ff560130050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 shutdown_connections 2023-09-30T22:27:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.649+0000 7ff564df2700 1 -- 172.21.15.32:0/5072401 wait complete. 2023-09-30T22:27:51.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:51.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.756+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa574074ea0 con 0x7fa5741338a0 2023-09-30T22:27:51.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.757+0000 7fa56a7fc700 1 -- 172.21.15.32:0/3706470608 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa56c012340 con 0x7fa5741338a0 2023-09-30T22:27:51.757 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:51.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 msgr2=0x7fa57411c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 0x7fa57411c120 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009dd0 tx=0x7fa56c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 shutdown_connections 2023-09-30T22:27:51.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5741338a0 0x7fa57411c120 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa57409de50 0x7fa57411b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 --2- 172.21.15.32:0/3706470608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57409e760 0x7fa57411bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 >> 172.21.15.32:0/3706470608 conn(0x7fa5740a2aa0 msgr2=0x7fa5741306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 shutdown_connections 2023-09-30T22:27:51.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.759+0000 7fa57bcc5700 1 -- 172.21.15.32:0/3706470608 wait complete. 2023-09-30T22:27:51.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.844+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe4d406b9e0 con 0x7fe4d41338c0 2023-09-30T22:27:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.845+0000 7fe4d0ff9700 1 -- 172.21.15.32:0/3728733736 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4c801b880 con 0x7fe4d41338c0 2023-09-30T22:27:51.845 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d41338c0 msgr2=0x7fe4d411ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d41338c0 0x7fe4d411ee10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40a35d0 tx=0x7fe4c800c710 comp rx=0 tx=0).stop 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 shutdown_connections 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4d41341d0 0x7fe4d411f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4d409e560 0x7fe4d411e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 --2- 172.21.15.32:0/3728733736 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4d41338c0 0x7fe4d411ee10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.847+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 >> 172.21.15.32:0/3728733736 conn(0x7fe4d40a2aa0 msgr2=0x7fe4d409f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.848+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 shutdown_connections 2023-09-30T22:27:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:51.848+0000 7fe4da95c700 1 -- 172.21.15.32:0/3728733736 wait complete. 2023-09-30T22:27:51.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:52.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:52.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.765+0000 7ff95ed18700 1 Processor -- start 2023-09-30T22:27:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.765+0000 7ff95ed18700 1 -- start start 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95ed18700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 0x7ff9580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95ed18700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff95809d630 0x7ff9580a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95ed18700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff958133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95ed18700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9580a2a70 con 0x7ff95809d630 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95ed18700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580a01d0 con 0x7ff9580a0a30 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95ed18700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff95809e2b0 con 0x7ff958133930 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff95809d630 0x7ff9580a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95cab4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 0x7ff9580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff95809d630 0x7ff9580a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58566/0 (socket says 172.21.15.32:58566) 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95cab4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 0x7ff9580a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44860/0 (socket says 172.21.15.32:44860) 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 -- 172.21.15.32:0/2684268939 learned_addr learned my addr 172.21.15.32:0/2684268939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95d2b5700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff958133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 -- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 msgr2=0x7ff958133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff958133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 -- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 msgr2=0x7ff9580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 0x7ff9580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 -- 172.21.15.32:0/2684268939 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95809dc50 con 0x7ff95809d630 2023-09-30T22:27:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff95cab4700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 0x7ff9580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.766+0000 7ff957fff700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95809d630 0x7ff9580a1340 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff9440098f0 tx=0x7ff944009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a49d7dcc2ebc36 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff9577fe700 1 -- 172.21.15.32:0/2684268939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944011830 con 0x7ff95809d630 2023-09-30T22:27:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff9577fe700 1 -- 172.21.15.32:0/2684268939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944011990 con 0x7ff95809d630 2023-09-30T22:27:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff9577fe700 1 -- 172.21.15.32:0/2684268939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944011d20 con 0x7ff95809d630 2023-09-30T22:27:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 -- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95809d630 msgr2=0x7ff9580a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95809d630 0x7ff9580a1340 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff9440098f0 tx=0x7ff944009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 -- 172.21.15.32:0/2684268939 shutdown_connections 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff9580a0a30 0x7ff9580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff958133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 --2- 172.21.15.32:0/2684268939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff95809d630 0x7ff9580a1340 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 -- 172.21.15.32:0/2684268939 >> 172.21.15.32:0/2684268939 conn(0x7ff9580a3050 msgr2=0x7ff9580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.767+0000 7ff95ed18700 1 -- 172.21.15.32:0/2684268939 shutdown_connections 2023-09-30T22:27:52.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.768+0000 7ff95ed18700 1 -- 172.21.15.32:0/2684268939 wait complete. 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 Processor -- start 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 -- start start 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff95809d630 0x7ff9581c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 0x7ff9581c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff9581c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9581206b0 con 0x7ff9580a0a30 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9581204c0 con 0x7ff95809d630 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95ed18700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9581223d0 con 0x7ff958133930 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff957fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 0x7ff9581c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95d2b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff9581c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95cab4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff95809d630 0x7ff9581c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff957fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 0x7ff9581c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58580/0 (socket says 172.21.15.32:58580) 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff957fff700 1 -- 172.21.15.32:0/905228437 learned_addr learned my addr 172.21.15.32:0/905228437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.769+0000 7ff95d2b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff9581c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43314/0 (socket says 172.21.15.32:43314) 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff957fff700 1 -- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 msgr2=0x7ff9581c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff957fff700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff9581c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff957fff700 1 -- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff95809d630 msgr2=0x7ff9581c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff957fff700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff95809d630 0x7ff9581c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff957fff700 1 -- 172.21.15.32:0/905228437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c0094a0 con 0x7ff9580a0a30 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff957fff700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 0x7ff9581c8640 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff944004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944004240 con 0x7ff9580a0a30 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94401f070 con 0x7ff9580a0a30 2023-09-30T22:27:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff944009550 con 0x7ff9580a0a30 2023-09-30T22:27:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440045e0 con 0x7ff9580a0a30 2023-09-30T22:27:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.770+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9581220f0 con 0x7ff9580a0a30 2023-09-30T22:27:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.771+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff944021030 con 0x7ff9580a0a30 2023-09-30T22:27:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.771+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9580747f0 con 0x7ff9580a0a30 2023-09-30T22:27:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.771+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff9440406e0 con 0x7ff9580a0a30 2023-09-30T22:27:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.776+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff944023810 con 0x7ff9580a0a30 2023-09-30T22:27:52.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.819+0000 7f06a7c40700 1 Processor -- start 2023-09-30T22:27:52.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 -- start start 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06a01338c0 0x7f06a0133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a009de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a009e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a0070c80 con 0x7f06a01338c0 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a009ed20 con 0x7f06a009e390 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.820+0000 7f06a7c40700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06a006f720 con 0x7f06a01341d0 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06a01338c0 0x7f06a0133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06a01338c0 0x7f06a0133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58596/0 (socket says 172.21.15.32:58596) 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 -- 172.21.15.32:0/460109615 learned_addr learned my addr 172.21.15.32:0/460109615 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a61dd700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a009e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a51db700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a009de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 -- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 msgr2=0x7f06a009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a009de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 -- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 msgr2=0x7f06a009e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a009e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 -- 172.21.15.32:0/460109615 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a006fce0 con 0x7f06a01338c0 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a61dd700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a009e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a59dc700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a0133c90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f069c0098f0 tx=0x7f069c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dbc45115003d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.821+0000 7f06a49da700 1 -- 172.21.15.32:0/460109615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c011830 con 0x7f06a01338c0 2023-09-30T22:27:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.822+0000 7f06a49da700 1 -- 172.21.15.32:0/460109615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c011990 con 0x7f06a01338c0 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.822+0000 7f06a49da700 1 -- 172.21.15.32:0/460109615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c011d20 con 0x7f06a01338c0 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 -- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 msgr2=0x7f06a0133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a0133c90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f069c0098f0 tx=0x7f069c009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 -- 172.21.15.32:0/460109615 shutdown_connections 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a009e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a0133c90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 --2- 172.21.15.32:0/460109615 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a009de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 -- 172.21.15.32:0/460109615 >> 172.21.15.32:0/460109615 conn(0x7f06a00a2aa0 msgr2=0x7f06a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 -- 172.21.15.32:0/460109615 shutdown_connections 2023-09-30T22:27:52.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.823+0000 7f06a7c40700 1 -- 172.21.15.32:0/460109615 wait complete. 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 Processor -- start 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 -- start start 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a011b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a011bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a011c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a011dfa0 con 0x7f06a01338c0 2023-09-30T22:27:52.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a011cad0 con 0x7f06a009e390 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a7c40700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06a011d1a0 con 0x7f06a01341d0 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.824+0000 7f06a59dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a011b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a59dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a011b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44886/0 (socket says 172.21.15.32:44886) 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a011bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a59dc700 1 -- 172.21.15.32:0/807232768 learned_addr learned my addr 172.21.15.32:0/807232768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a61dd700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a011c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 -- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 msgr2=0x7f06a011c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a011c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 -- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 msgr2=0x7f06a011b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a011b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 -- 172.21.15.32:0/807232768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0698009a70 con 0x7f06a01338c0 2023-09-30T22:27:52.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a59dc700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a011b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a51db700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a011bb60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f069000e900 tx=0x7f069000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000c640 con 0x7f06a01338c0 2023-09-30T22:27:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000c7a0 con 0x7f06a01338c0 2023-09-30T22:27:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c009550 con 0x7f06a01338c0 2023-09-30T22:27:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.825+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000cb10 con 0x7f06a01338c0 2023-09-30T22:27:52.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.826+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a006f960 con 0x7f06a01338c0 2023-09-30T22:27:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.827+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a00a1d20 con 0x7f06a01338c0 2023-09-30T22:27:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.827+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f069000cc70 con 0x7f06a01338c0 2023-09-30T22:27:52.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.827+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f069003f900 con 0x7f06a01338c0 2023-09-30T22:27:52.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.832+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f069001ad70 con 0x7f06a01338c0 2023-09-30T22:27:52.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.923+0000 7fc10d7eb700 1 Processor -- start 2023-09-30T22:27:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.923+0000 7fc10d7eb700 1 -- start start 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc10d7eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 0x7fc10809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc10d7eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809f240 0x7fc1081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc10d7eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc108070940 con 0x7fc10809f650 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc10d7eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10806fd60 con 0x7fc10809e930 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc10d7eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc10806fa80 con 0x7fc10809f240 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 0x7fc10809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 0x7fc10809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44894/0 (socket says 172.21.15.32:44894) 2023-09-30T22:27:52.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc106ffd700 1 -- 172.21.15.32:0/1624452065 learned_addr learned my addr 172.21.15.32:0/1624452065 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:52.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.924+0000 7fc1067fc700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809f240 0x7fc1081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc106ffd700 1 -- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809f240 msgr2=0x7fc1081317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:52.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc106ffd700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809f240 0x7fc1081317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc106ffd700 1 -- 172.21.15.32:0/1624452065 >> v1:172.21.15.32:6789/0 conn(0x7fc10809f650 legacy=0x7fc108131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:52.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc106ffd700 1 -- 172.21.15.32:0/1624452065 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10809fb60 con 0x7fc10809e930 2023-09-30T22:27:52.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc1067fc700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809f240 0x7fc1081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc106ffd700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 0x7fc10809ed00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009920 tx=0x7fc0f8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a9e560877f78fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc105ffb700 1 -- 172.21.15.32:0/1624452065 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8004230 con 0x7fc10809e930 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc105ffb700 1 -- 172.21.15.32:0/1624452065 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f8004390 con 0x7fc10809e930 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.925+0000 7fc105ffb700 1 -- 172.21.15.32:0/1624452065 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80036a0 con 0x7fc10809e930 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 -- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 msgr2=0x7fc10809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 0x7fc10809ed00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009920 tx=0x7fc0f8009c30 comp rx=0 tx=0).stop 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 -- 172.21.15.32:0/1624452065 shutdown_connections 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809e930 0x7fc10809ed00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/1624452065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809f240 0x7fc1081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 -- 172.21.15.32:0/1624452065 >> 172.21.15.32:0/1624452065 conn(0x7fc1080a4210 msgr2=0x7fc1080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 -- 172.21.15.32:0/1624452065 shutdown_connections 2023-09-30T22:27:52.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.926+0000 7fc10d7eb700 1 -- 172.21.15.32:0/1624452065 wait complete. 2023-09-30T22:27:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 Processor -- start 2023-09-30T22:27:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 -- start start 2023-09-30T22:27:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809e930 0x7fc1081a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 0x7fc1081a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809f650 0x7fc1081a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:52.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1081a6f60 con 0x7fc10809f240 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc108071d20 con 0x7fc10809f650 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.928+0000 7fc10d7eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1080a3700 con 0x7fc10809e930 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 0x7fc1081a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1067fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 0x7fc1081a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58622/0 (socket says 172.21.15.32:58622) 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1067fc700 1 -- 172.21.15.32:0/2976922479 learned_addr learned my addr 172.21.15.32:0/2976922479 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809f650 0x7fc1081a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809e930 0x7fc1081a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 -- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809e930 msgr2=0x7fc1081a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809e930 0x7fc1081a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 -- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 msgr2=0x7fc1081a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 0x7fc1081a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:52.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 -- 172.21.15.32:0/2976922479 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0094a0 con 0x7fc10809f650 2023-09-30T22:27:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1067fc700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 0x7fc1081a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.929+0000 7fc1077fe700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809f650 0x7fc1081a4780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc10000cae0 tx=0x7fc10000cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:52.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.930+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10000aaa0 con 0x7fc10809f650 2023-09-30T22:27:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.930+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10000ac00 con 0x7fc10809f650 2023-09-30T22:27:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.930+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f8009550 con 0x7fc10809f650 2023-09-30T22:27:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.930+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100012420 con 0x7fc10809f650 2023-09-30T22:27:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.930+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1080a0190 con 0x7fc10809f650 2023-09-30T22:27:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.931+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1081a0b30 con 0x7fc10809f650 2023-09-30T22:27:52.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.931+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc100012580 con 0x7fc10809f650 2023-09-30T22:27:52.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.931+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc10003fca0 con 0x7fc10809f650 2023-09-30T22:27:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:52.940+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1000228f0 con 0x7fc10809f650 2023-09-30T22:27:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.007+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff958120cb0 con 0x7ff9580a0a30 2023-09-30T22:27:53.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.007+0000 7ff9557fa700 1 -- 172.21.15.32:0/905228437 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff944015020 con 0x7ff9580a0a30 2023-09-30T22:27:53.007 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 msgr2=0x7ff9581c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 0x7ff9581c8640 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff944004750 comp rx=0 tx=0).stop 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 shutdown_connections 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff958133930 0x7ff9581c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff95809d630 0x7ff9581c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 --2- 172.21.15.32:0/905228437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9580a0a30 0x7ff9581c8640 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 >> 172.21.15.32:0/905228437 conn(0x7ff9580a3050 msgr2=0x7ff95809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:53.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 shutdown_connections 2023-09-30T22:27:53.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.011+0000 7ff95ed18700 1 -- 172.21.15.32:0/905228437 wait complete. 2023-09-30T22:27:53.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.012+0000 7f4b4f8e8700 1 Processor -- start 2023-09-30T22:27:53.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.012+0000 7f4b4f8e8700 1 -- start start 2023-09-30T22:27:53.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.013+0000 7f4b4f8e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b48130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.013+0000 7f4b4f8e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b48130e50 0x7f4b4806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.013+0000 7f4b4f8e8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b480716a0 con 0x7f4b4806fb20 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.013+0000 7f4b4f8e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4809a390 con 0x7f4b48130540 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.013+0000 7f4b4f8e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48071290 con 0x7f4b48130e50 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4de85700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4b4806fb20 0x7f4b48137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60170/0 (socket says 172.21.15.32:60170) 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4de85700 1 -- 172.21.15.32:0/1888649333 learned_addr learned my addr 172.21.15.32:0/1888649333 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4d684700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b48130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:53.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4ce83700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b48130e50 0x7f4b4806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4d684700 1 -- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b48130e50 msgr2=0x7f4b4806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4d684700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b48130e50 0x7f4b4806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4d684700 1 -- 172.21.15.32:0/1888649333 >> v1:172.21.15.32:6789/0 conn(0x7f4b4806fb20 legacy=0x7f4b48137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4d684700 1 -- 172.21.15.32:0/1888649333 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48130340 con 0x7f4b48130540 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b3ffff700 1 -- 172.21.15.32:0/1888649333 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722772684 0 0) 0x7f4b480716a0 con 0x7f4b4806fb20 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4ce83700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b48130e50 0x7f4b4806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.014+0000 7f4b4d684700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b48130910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098a0 tx=0x7f4b44009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c818c91986158b9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:53.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.015+0000 7f4b3ffff700 1 -- 172.21.15.32:0/1888649333 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011830 con 0x7f4b48130540 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.015+0000 7f4b3ffff700 1 -- 172.21.15.32:0/1888649333 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b44011990 con 0x7f4b48130540 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.015+0000 7f4b3ffff700 1 -- 172.21.15.32:0/1888649333 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011d20 con 0x7f4b48130540 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 msgr2=0x7f4b48130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b48130910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098a0 tx=0x7f4b44009bb0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/1888649333 shutdown_connections 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b48130910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/1888649333 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b48130e50 0x7f4b4806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/1888649333 >> 172.21.15.32:0/1888649333 conn(0x7f4b480a2aa0 msgr2=0x7f4b480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/1888649333 shutdown_connections 2023-09-30T22:27:53.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.016+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/1888649333 wait complete. 2023-09-30T22:27:53.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.017+0000 7f4b4f8e8700 1 Processor -- start 2023-09-30T22:27:53.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.017+0000 7f4b4f8e8700 1 -- start start 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.017+0000 7f4b4f8e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b4806fb20 0x7f4b481c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.017+0000 7f4b4f8e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b481c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.017+0000 7f4b4f8e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b48130e50 0x7f4b481c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4f8e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480718e0 con 0x7f4b48130e50 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4f8e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4812be50 con 0x7f4b48130540 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4f8e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4812c940 con 0x7f4b4806fb20 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b481c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b481c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44926/0 (socket says 172.21.15.32:44926) 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 -- 172.21.15.32:0/2569303322 learned_addr learned my addr 172.21.15.32:0/2569303322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:53.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4de85700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b48130e50 0x7f4b481c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4d684700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b4806fb20 0x7f4b481c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 -- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b4806fb20 msgr2=0x7f4b481c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b4806fb20 0x7f4b481c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 -- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b48130e50 msgr2=0x7f4b481c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b48130e50 0x7f4b481c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4ce83700 1 -- 172.21.15.32:0/2569303322 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b380094a0 con 0x7f4b48130540 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4de85700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b48130e50 0x7f4b481c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.018+0000 7f4b4d684700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b4806fb20 0x7f4b481c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.019+0000 7f4b4ce83700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b481c6220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ea0 tx=0x7f4b3800ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.019+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38013920 con 0x7f4b48130540 2023-09-30T22:27:53.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.019+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38013a80 con 0x7f4b48130540 2023-09-30T22:27:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.019+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b440094a0 con 0x7f4b48130540 2023-09-30T22:27:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.019+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38013e10 con 0x7f4b48130540 2023-09-30T22:27:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.019+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4812dc50 con 0x7f4b48130540 2023-09-30T22:27:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.020+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4812d860 con 0x7f4b48130540 2023-09-30T22:27:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.021+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b3800c640 con 0x7f4b48130540 2023-09-30T22:27:53.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.021+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4b38043c10 con 0x7f4b48130540 2023-09-30T22:27:53.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:53.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.030+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b3801b440 con 0x7f4b48130540 2023-09-30T22:27:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.056+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f06a0074ea0 con 0x7f06a01338c0 2023-09-30T22:27:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.056+0000 7f06967fc700 1 -- 172.21.15.32:0/807232768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0690021020 con 0x7f06a01338c0 2023-09-30T22:27:53.056 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 msgr2=0x7f06a011bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a011bb60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f069000e900 tx=0x7f069000ec10 comp rx=0 tx=0).stop 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 shutdown_connections 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f06a01341d0 0x7f06a011c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f06a009e390 0x7f06a011b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 --2- 172.21.15.32:0/807232768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06a01338c0 0x7f06a011bb60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 >> 172.21.15.32:0/807232768 conn(0x7f06a00a2aa0 msgr2=0x7f06a0132db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:53.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 shutdown_connections 2023-09-30T22:27:53.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.059+0000 7f06a7c40700 1 -- 172.21.15.32:0/807232768 wait complete. 2023-09-30T22:27:53.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:53.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.204+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc108074ea0 con 0x7fc10809f650 2023-09-30T22:27:53.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.205+0000 7fc0ef7fe700 1 -- 172.21.15.32:0/2976922479 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc100022300 con 0x7fc10809f650 2023-09-30T22:27:53.205 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:53.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809f650 msgr2=0x7fc1081a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809f650 0x7fc1081a4780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc10000cae0 tx=0x7fc10000cdf0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 shutdown_connections 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc10809f650 0x7fc1081a4780 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc10809e930 0x7fc1081a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 --2- 172.21.15.32:0/2976922479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc10809f240 0x7fc1081a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 >> 172.21.15.32:0/2976922479 conn(0x7fc1080a4210 msgr2=0x7fc1081b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 shutdown_connections 2023-09-30T22:27:53.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.207+0000 7fc10d7eb700 1 -- 172.21.15.32:0/2976922479 wait complete. 2023-09-30T22:27:53.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:53.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.286+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4b48074ea0 con 0x7f4b48130540 2023-09-30T22:27:53.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.286+0000 7f4b3dffb700 1 -- 172.21.15.32:0/2569303322 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b38020030 con 0x7f4b48130540 2023-09-30T22:27:53.287 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 msgr2=0x7f4b481c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b481c6220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ea0 tx=0x7f4b3800ee90 comp rx=0 tx=0).stop 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 shutdown_connections 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b48130e50 0x7f4b481c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4b4806fb20 0x7f4b481c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 --2- 172.21.15.32:0/2569303322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4b48130540 0x7f4b481c6220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 >> 172.21.15.32:0/2569303322 conn(0x7f4b480a2aa0 msgr2=0x7f4b4806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 shutdown_connections 2023-09-30T22:27:53.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:53.289+0000 7f4b4f8e8700 1 -- 172.21.15.32:0/2569303322 wait complete. 2023-09-30T22:27:53.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:54.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:54.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:54.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.176+0000 7f416d586700 1 Processor -- start 2023-09-30T22:27:54.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.176+0000 7f416d586700 1 -- start start 2023-09-30T22:27:54.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f416d586700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 0x7f4168136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f416d586700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f41681393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f416d586700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4168070940 con 0x7f416809e930 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f416d586700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416806fd60 con 0x7f416809f410 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f416d586700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416806fa80 con 0x7f4168136fe0 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41667fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 0x7f4168136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f41681393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41667fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 0x7f4168136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58640/0 (socket says 172.21.15.32:58640) 2023-09-30T22:27:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41667fc700 1 -- 172.21.15.32:0/3321039620 learned_addr learned my addr 172.21.15.32:0/3321039620 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41677fe700 1 -- 172.21.15.32:0/3321039620 >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 msgr2=0x7f4168136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41677fe700 1 --2- 172.21.15.32:0/3321039620 >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 0x7f4168136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41677fe700 1 -- 172.21.15.32:0/3321039620 >> v1:172.21.15.32:6789/0 conn(0x7f416809e930 legacy=0x7f416809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.177+0000 7f41677fe700 1 -- 172.21.15.32:0/3321039620 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416809fb60 con 0x7f4168136fe0 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.178+0000 7f41667fc700 1 --2- 172.21.15.32:0/3321039620 >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 0x7f4168136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.178+0000 7f41677fe700 1 --2- 172.21.15.32:0/3321039620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f41681393c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4160009840 tx=0x7f4160009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9b96fb19c26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.178+0000 7f4165ffb700 1 -- 172.21.15.32:0/3321039620 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160011830 con 0x7f4168136fe0 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.178+0000 7f4165ffb700 1 -- 172.21.15.32:0/3321039620 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160011990 con 0x7f4168136fe0 2023-09-30T22:27:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.178+0000 7f4165ffb700 1 -- 172.21.15.32:0/3321039620 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160011d20 con 0x7f4168136fe0 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 -- 172.21.15.32:0/3321039620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 msgr2=0x7f41681393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 --2- 172.21.15.32:0/3321039620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f41681393c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4160009840 tx=0x7f4160009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 -- 172.21.15.32:0/3321039620 shutdown_connections 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 --2- 172.21.15.32:0/3321039620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f41681393c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 --2- 172.21.15.32:0/3321039620 >> v2:172.21.15.32:3300/0 conn(0x7f416809f410 0x7f4168136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 -- 172.21.15.32:0/3321039620 >> 172.21.15.32:0/3321039620 conn(0x7f41680a4210 msgr2=0x7f41680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 -- 172.21.15.32:0/3321039620 shutdown_connections 2023-09-30T22:27:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.179+0000 7f416d586700 1 -- 172.21.15.32:0/3321039620 wait complete. 2023-09-30T22:27:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 Processor -- start 2023-09-30T22:27:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 -- start start 2023-09-30T22:27:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416809e930 0x7f416811c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416809f410 0x7f416811ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f416811cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168071db0 con 0x7f416809e930 2023-09-30T22:27:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41680a33f0 con 0x7f4168136fe0 2023-09-30T22:27:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f416d586700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f416811f820 con 0x7f416809f410 2023-09-30T22:27:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f416811cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.180+0000 7f4166ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416809e930 0x7f416811c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f416811cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44938/0 (socket says 172.21.15.32:44938) 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 -- 172.21.15.32:0/3347795543 learned_addr learned my addr 172.21.15.32:0/3347795543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41667fc700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416809f410 0x7f416811ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 -- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416809f410 msgr2=0x7f416811ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416809f410 0x7f416811ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 -- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416809e930 msgr2=0x7f416811c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416809e930 0x7f416811c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 -- 172.21.15.32:0/3347795543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c009550 con 0x7f4168136fe0 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f4166ffd700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416809e930 0x7f416811c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f41677fe700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f416811cfd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4160009fd0 tx=0x7f4160004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160004510 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41600036a0 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41600094a0 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160003a50 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.181+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416811fd90 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.182+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4160003bb0 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.182+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168074ea0 con 0x7f4168136fe0 2023-09-30T22:27:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.183+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4160040950 con 0x7f4168136fe0 2023-09-30T22:27:54.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.187+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4160015070 con 0x7f4168136fe0 2023-09-30T22:27:54.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.217+0000 7f076f6fe700 1 Processor -- start 2023-09-30T22:27:54.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.218+0000 7f076f6fe700 1 -- start start 2023-09-30T22:27:54.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076f6fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 0x7f07680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076f6fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f076809d630 0x7f07680a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076f6fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f0768133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076f6fe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680a2a70 con 0x7f07680a0a30 2023-09-30T22:27:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076f6fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680a01d0 con 0x7f076809d630 2023-09-30T22:27:54.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076f6fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f076809e2b0 con 0x7f0768133930 2023-09-30T22:27:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076d49a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 0x7f07680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076d49a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 0x7f07680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58652/0 (socket says 172.21.15.32:58652) 2023-09-30T22:27:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076d49a700 1 -- 172.21.15.32:0/2909718750 learned_addr learned my addr 172.21.15.32:0/2909718750 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f076809d630 0x7f07680a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076dc9b700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f0768133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 -- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 msgr2=0x7f0768133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f0768133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 -- 172.21.15.32:0/2909718750 >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 msgr2=0x7f07680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 --2- 172.21.15.32:0/2909718750 >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 0x7f07680a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 -- 172.21.15.32:0/2909718750 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076809dc50 con 0x7f076809d630 2023-09-30T22:27:54.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076d49a700 1 --2- 172.21.15.32:0/2909718750 >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 0x7f07680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.219+0000 7f076cc99700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f076809d630 0x7f07680a1340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0758009840 tx=0x7f0758009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4017aee6a55988e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f075ffff700 1 -- 172.21.15.32:0/2909718750 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758011830 con 0x7f076809d630 2023-09-30T22:27:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f075ffff700 1 -- 172.21.15.32:0/2909718750 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0758011990 con 0x7f076809d630 2023-09-30T22:27:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f075ffff700 1 -- 172.21.15.32:0/2909718750 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758011d20 con 0x7f076809d630 2023-09-30T22:27:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 -- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f076809d630 msgr2=0x7f07680a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f076809d630 0x7f07680a1340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0758009840 tx=0x7f0758009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 -- 172.21.15.32:0/2909718750 shutdown_connections 2023-09-30T22:27:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f0768133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 --2- 172.21.15.32:0/2909718750 >> v2:172.21.15.32:3300/0 conn(0x7f07680a0a30 0x7f07680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 --2- 172.21.15.32:0/2909718750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f076809d630 0x7f07680a1340 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 -- 172.21.15.32:0/2909718750 >> 172.21.15.32:0/2909718750 conn(0x7f07680a3050 msgr2=0x7f07680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 -- 172.21.15.32:0/2909718750 shutdown_connections 2023-09-30T22:27:54.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.220+0000 7f076f6fe700 1 -- 172.21.15.32:0/2909718750 wait complete. 2023-09-30T22:27:54.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.222+0000 7f076f6fe700 1 Processor -- start 2023-09-30T22:27:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.222+0000 7f076f6fe700 1 -- start start 2023-09-30T22:27:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076f6fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076809d630 0x7f076811f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076f6fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 0x7f076811fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076f6fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f07681c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076f6fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0768121fa0 con 0x7f076809d630 2023-09-30T22:27:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076f6fe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0768122640 con 0x7f07680a0a30 2023-09-30T22:27:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076f6fe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f076811ef70 con 0x7f0768133930 2023-09-30T22:27:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 0x7f076811fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 0x7f076811fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44966/0 (socket says 172.21.15.32:44966) 2023-09-30T22:27:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076dc9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f07681c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 -- 172.21.15.32:0/122508379 learned_addr learned my addr 172.21.15.32:0/122508379 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076d49a700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076809d630 0x7f076811f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 -- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 msgr2=0x7f07681c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f07681c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 -- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076809d630 msgr2=0x7f076811f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076809d630 0x7f076811f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 -- 172.21.15.32:0/122508379 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009b30 con 0x7f07680a0a30 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076dc9b700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f07681c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f076cc99700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 0x7f076811fa50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f07580090b0 tx=0x7f0758004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758004240 con 0x7f07680a0a30 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.223+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075801f070 con 0x7f07680a0a30 2023-09-30T22:27:54.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.224+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07580094a0 con 0x7f07680a0a30 2023-09-30T22:27:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.224+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07580036a0 con 0x7f07680a0a30 2023-09-30T22:27:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.224+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07680a0410 con 0x7f07680a0a30 2023-09-30T22:27:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.225+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0758021030 con 0x7f07680a0a30 2023-09-30T22:27:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.225+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f07580405c0 con 0x7f07680a0a30 2023-09-30T22:27:54.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.225+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07681206f0 con 0x7f07680a0a30 2023-09-30T22:27:54.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.230+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07580239c0 con 0x7f07680a0a30 2023-09-30T22:27:54.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:54.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.377+0000 7fcb5318b700 1 Processor -- start 2023-09-30T22:27:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.377+0000 7fcb5318b700 1 -- start start 2023-09-30T22:27:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb5318b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb4c1337f0 0x7fcb4c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb5318b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c138dc0 0x7fcb4c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb5318b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4c06b990 con 0x7fcb4c133420 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb5318b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4c09b0e0 con 0x7fcb4c1337f0 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb5318b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c133c20 con 0x7fcb4c138dc0 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb50f27700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb4c133420 0x7fcb4c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60180/0 (socket says 172.21.15.32:60180) 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb50f27700 1 -- 172.21.15.32:0/694639986 learned_addr learned my addr 172.21.15.32:0/694639986 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb4bfff700 1 --2- 172.21.15.32:0/694639986 >> v2:172.21.15.32:3300/0 conn(0x7fcb4c1337f0 0x7fcb4c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.378+0000 7fcb51728700 1 --2- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c138dc0 0x7fcb4c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4b7fe700 1 -- 172.21.15.32:0/694639986 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146612330 0 0) 0x7fcb4c06b990 con 0x7fcb4c133420 2023-09-30T22:27:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4b7fe700 1 -- 172.21.15.32:0/694639986 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb3c0035b0 con 0x7fcb4c133420 2023-09-30T22:27:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4bfff700 1 -- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c138dc0 msgr2=0x7fcb4c13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4bfff700 1 --2- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c138dc0 0x7fcb4c13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4bfff700 1 -- 172.21.15.32:0/694639986 >> v1:172.21.15.32:6789/0 conn(0x7fcb4c133420 legacy=0x7fcb4c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4bfff700 1 -- 172.21.15.32:0/694639986 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c0a0590 con 0x7fcb4c1337f0 2023-09-30T22:27:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4bfff700 1 --2- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c1337f0 0x7fcb4c138880 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009840 tx=0x7fcb38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e566d42bf08c38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4b7fe700 1 -- 172.21.15.32:0/694639986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38011830 con 0x7fcb4c1337f0 2023-09-30T22:27:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4b7fe700 1 -- 172.21.15.32:0/694639986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb38011990 con 0x7fcb4c1337f0 2023-09-30T22:27:54.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.379+0000 7fcb4b7fe700 1 -- 172.21.15.32:0/694639986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38011d20 con 0x7fcb4c1337f0 2023-09-30T22:27:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 -- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c1337f0 msgr2=0x7fcb4c138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 --2- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c1337f0 0x7fcb4c138880 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009840 tx=0x7fcb38009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 -- 172.21.15.32:0/694639986 shutdown_connections 2023-09-30T22:27:54.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 --2- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c138dc0 0x7fcb4c13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 --2- 172.21.15.32:0/694639986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c1337f0 0x7fcb4c138880 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 -- 172.21.15.32:0/694639986 >> 172.21.15.32:0/694639986 conn(0x7fcb4c0a4210 msgr2=0x7fcb4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 -- 172.21.15.32:0/694639986 shutdown_connections 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.380+0000 7fcb5318b700 1 -- 172.21.15.32:0/694639986 wait complete. 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.382+0000 7fcb5318b700 1 Processor -- start 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.382+0000 7fcb5318b700 1 -- start start 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb5318b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 0x7fcb4c12ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb5318b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb4c1337f0 0x7fcb4c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb5318b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c138dc0 0x7fcb4c12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb5318b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c09c9c0 con 0x7fcb4c138dc0 2023-09-30T22:27:54.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb5318b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c130240 con 0x7fcb4c1337f0 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb5318b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c130690 con 0x7fcb4c133420 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 0x7fcb4c12ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 0x7fcb4c12ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43418/0 (socket says 172.21.15.32:43418) 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 -- 172.21.15.32:0/2355797172 learned_addr learned my addr 172.21.15.32:0/2355797172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb4bfff700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb4c1337f0 0x7fcb4c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb51728700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c138dc0 0x7fcb4c12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 -- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb4c1337f0 msgr2=0x7fcb4c12f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb4c1337f0 0x7fcb4c12f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 -- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c138dc0 msgr2=0x7fcb4c12f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c138dc0 0x7fcb4c12f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb50f27700 1 -- 172.21.15.32:0/2355797172 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380094a0 con 0x7fcb4c133420 2023-09-30T22:27:54.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.383+0000 7fcb51728700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c138dc0 0x7fcb4c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.384+0000 7fcb50f27700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 0x7fcb4c12ee50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000c700 tx=0x7fcb4000cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.384+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40005d70 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.384+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb400070d0 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.384+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c09cdc0 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.384+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40007440 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.384+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c09c350 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.385+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb400075a0 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.385+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb40040cb0 con 0x7fcb4c133420 2023-09-30T22:27:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.385+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c09be40 con 0x7fcb4c133420 2023-09-30T22:27:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.395+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb40017030 con 0x7fcb4c133420 2023-09-30T22:27:54.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.419+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f416806b250 con 0x7f4168136fe0 2023-09-30T22:27:54.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.419+0000 7f414f7fe700 1 -- 172.21.15.32:0/3347795543 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4160036020 con 0x7f4168136fe0 2023-09-30T22:27:54.419 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:54.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 msgr2=0x7f416811cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f416811cfd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4160009fd0 tx=0x7f4160004750 comp rx=0 tx=0).stop 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 shutdown_connections 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4168136fe0 0x7f416811cfd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f416809e930 0x7f416811c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 --2- 172.21.15.32:0/3347795543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f416809f410 0x7f416811ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 >> 172.21.15.32:0/3347795543 conn(0x7f41680a4210 msgr2=0x7f4168135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 shutdown_connections 2023-09-30T22:27:54.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.423+0000 7f416d586700 1 -- 172.21.15.32:0/3347795543 wait complete. 2023-09-30T22:27:54.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.447+0000 7f2023e16700 1 Processor -- start 2023-09-30T22:27:54.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.447+0000 7f2023e16700 1 -- start start 2023-09-30T22:27:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2023e16700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f201c09de00 0x7f201c09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2023e16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 0x7f201c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2023e16700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201c09a810 con 0x7f201c06a770 2023-09-30T22:27:54.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2023e16700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201c06ab40 con 0x7f201c09de00 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2023e16700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201c09ebc0 con 0x7f201c13d1f0 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20223b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 0x7f201c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20213b1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f201c09de00 0x7f201c09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2021bb2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f201c06a770 0x7f201c09d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60194/0 (socket says 172.21.15.32:60194) 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2021bb2700 1 -- 172.21.15.32:0/161664902 learned_addr learned my addr 172.21.15.32:0/161664902 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20223b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 0x7f201c13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44980/0 (socket says 172.21.15.32:44980) 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20213b1700 1 -- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 msgr2=0x7f201c13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20213b1700 1 --2- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 0x7f201c13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20213b1700 1 -- 172.21.15.32:0/161664902 >> v1:172.21.15.32:6789/0 conn(0x7f201c06a770 legacy=0x7f201c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20213b1700 1 -- 172.21.15.32:0/161664902 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c09e920 con 0x7f201c09de00 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f2020bb0700 1 -- 172.21.15.32:0/161664902 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845944282 0 0) 0x7f201c09a810 con 0x7f201c06a770 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.448+0000 7f20223b3700 1 --2- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 0x7f201c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:54.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.449+0000 7f20213b1700 1 --2- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c09de00 0x7f201c09e250 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f200c009840 tx=0x7f200c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed109eca88134b57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.449+0000 7f2020bb0700 1 -- 172.21.15.32:0/161664902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c011830 con 0x7f201c09de00 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.449+0000 7f2020bb0700 1 -- 172.21.15.32:0/161664902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200c011990 con 0x7f201c09de00 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.449+0000 7f2020bb0700 1 -- 172.21.15.32:0/161664902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c011d20 con 0x7f201c09de00 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.449+0000 7f2023e16700 1 -- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c09de00 msgr2=0x7f201c09e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.449+0000 7f2023e16700 1 --2- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c09de00 0x7f201c09e250 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f200c009840 tx=0x7f200c009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.450+0000 7f2023e16700 1 -- 172.21.15.32:0/161664902 shutdown_connections 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.450+0000 7f2023e16700 1 --2- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c13d1f0 0x7f201c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.450+0000 7f2023e16700 1 --2- 172.21.15.32:0/161664902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c09de00 0x7f201c09e250 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.450+0000 7f2023e16700 1 -- 172.21.15.32:0/161664902 >> 172.21.15.32:0/161664902 conn(0x7f201c066f50 msgr2=0x7f201c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.450+0000 7f2023e16700 1 -- 172.21.15.32:0/161664902 shutdown_connections 2023-09-30T22:27:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.450+0000 7f2023e16700 1 -- 172.21.15.32:0/161664902 wait complete. 2023-09-30T22:27:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.451+0000 7f2023e16700 1 Processor -- start 2023-09-30T22:27:54.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.451+0000 7f2023e16700 1 -- start start 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2023e16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 0x7f201c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2023e16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c09de00 0x7f201c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2023e16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f201c13d1f0 0x7f201c071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2023e16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c06fa00 con 0x7f201c06a770 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2023e16700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201c06ee60 con 0x7f201c09de00 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2023e16700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f201c09f1a0 con 0x7f201c13d1f0 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 0x7f201c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 0x7f201c070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58714/0 (socket says 172.21.15.32:58714) 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 -- 172.21.15.32:0/1305097015 learned_addr learned my addr 172.21.15.32:0/1305097015 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f20223b3700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f201c13d1f0 0x7f201c071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f20213b1700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c09de00 0x7f201c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 -- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f201c13d1f0 msgr2=0x7f201c071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f201c13d1f0 0x7f201c071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 -- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c09de00 msgr2=0x7f201c0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c09de00 0x7f201c0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 -- 172.21.15.32:0/1305097015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014009a70 con 0x7f201c06a770 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f20213b1700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c09de00 0x7f201c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.452+0000 7f2021bb2700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 0x7f201c070b80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f201800c520 tx=0x7f201800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.453+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018004c80 con 0x7f201c06a770 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.453+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2018004e00 con 0x7f201c06a770 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.453+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c0094a0 con 0x7f201c06a770 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.453+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018007310 con 0x7f201c06a770 2023-09-30T22:27:54.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.453+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201c070730 con 0x7f201c06a770 2023-09-30T22:27:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.453+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2018007470 con 0x7f201c06a770 2023-09-30T22:27:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.454+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f201803f350 con 0x7f201c06a770 2023-09-30T22:27:54.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.454+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201c074ea0 con 0x7f201c06a770 2023-09-30T22:27:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.459+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0768122480 con 0x7f07680a0a30 2023-09-30T22:27:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.459+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20180123b0 con 0x7f201c06a770 2023-09-30T22:27:54.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.459+0000 7f075dffb700 1 -- 172.21.15.32:0/122508379 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0768122480 con 0x7f07680a0a30 2023-09-30T22:27:54.460 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 msgr2=0x7f076811fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 0x7f076811fa50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f07580090b0 tx=0x7f0758004750 comp rx=0 tx=0).stop 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 shutdown_connections 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0768133930 0x7f07681c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076809d630 0x7f076811f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 --2- 172.21.15.32:0/122508379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f07680a0a30 0x7f076811fa50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 >> 172.21.15.32:0/122508379 conn(0x7f07680a3050 msgr2=0x7f07681bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 shutdown_connections 2023-09-30T22:27:54.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.462+0000 7f076f6fe700 1 -- 172.21.15.32:0/122508379 wait complete. 2023-09-30T22:27:54.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:54.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.614+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcb4c074ea0 con 0x7fcb4c133420 2023-09-30T22:27:54.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.615+0000 7fcb497fa700 1 -- 172.21.15.32:0/2355797172 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb40017030 con 0x7fcb4c133420 2023-09-30T22:27:54.615 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:54.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 msgr2=0x7fcb4c12ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 0x7fcb4c12ee50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000c700 tx=0x7fcb4000cac0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 shutdown_connections 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb4c138dc0 0x7fcb4c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb4c133420 0x7fcb4c12ee50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 --2- 172.21.15.32:0/2355797172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb4c1337f0 0x7fcb4c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 >> 172.21.15.32:0/2355797172 conn(0x7fcb4c0a4210 msgr2=0x7fcb4c0a22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 shutdown_connections 2023-09-30T22:27:54.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.617+0000 7fcb5318b700 1 -- 172.21.15.32:0/2355797172 wait complete. 2023-09-30T22:27:54.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.666+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f201c09a3d0 con 0x7f201c06a770 2023-09-30T22:27:54.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.666+0000 7f20127fc700 1 -- 172.21.15.32:0/1305097015 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2018011360 con 0x7f201c06a770 2023-09-30T22:27:54.666 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 msgr2=0x7f201c070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 0x7f201c070b80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f201800c520 tx=0x7f201800c8e0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 shutdown_connections 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f201c13d1f0 0x7f201c071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f201c06a770 0x7f201c070b80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 --2- 172.21.15.32:0/1305097015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f201c09de00 0x7f201c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 >> 172.21.15.32:0/1305097015 conn(0x7f201c066f50 msgr2=0x7f201c13d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 shutdown_connections 2023-09-30T22:27:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:54.668+0000 7f2023e16700 1 -- 172.21.15.32:0/1305097015 wait complete. 2023-09-30T22:27:54.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:55.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:55.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:55.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.595+0000 7f6ddcb38700 1 Processor -- start 2023-09-30T22:27:55.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.595+0000 7f6ddcb38700 1 -- start start 2023-09-30T22:27:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6ddcb38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 0x7f6dd81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6ddcb38700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6ddcb38700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd806b990 con 0x7f6dd8134260 2023-09-30T22:27:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6ddcb38700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80a0590 con 0x7f6dd8133e30 2023-09-30T22:27:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6ddcb38700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd809fb80 con 0x7f6dd8133420 2023-09-30T22:27:55.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 0x7f6dd81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6dd8134260 0x7f6dd81304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60198/0 (socket says 172.21.15.32:60198) 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 0x7f6dd81337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43438/0 (socket says 172.21.15.32:43438) 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd6d9d700 1 -- 172.21.15.32:0/3281345761 learned_addr learned my addr 172.21.15.32:0/3281345761 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45004/0 (socket says 172.21.15.32:45004) 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd659c700 1 -- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 msgr2=0x7f6dd812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd559a700 1 -- 172.21.15.32:0/3281345761 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855566835 0 0) 0x7f6dd806b990 con 0x7f6dd8134260 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd659c700 1 --2- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd659c700 1 -- 172.21.15.32:0/3281345761 >> v1:172.21.15.32:6789/0 conn(0x7f6dd8134260 legacy=0x7f6dd81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd659c700 1 -- 172.21.15.32:0/3281345761 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd80a0030 con 0x7f6dd8133420 2023-09-30T22:27:55.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.596+0000 7f6dd5d9b700 1 --2- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.597+0000 7f6dd659c700 1 --2- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 0x7f6dd81337f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009dc0 tx=0x7f6dc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0472976bed54b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.597+0000 7f6dd559a700 1 -- 172.21.15.32:0/3281345761 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800e070 con 0x7f6dd8133420 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.597+0000 7f6dd559a700 1 -- 172.21.15.32:0/3281345761 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc800bb20 con 0x7f6dd8133420 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.597+0000 7f6dd559a700 1 -- 172.21.15.32:0/3281345761 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8012430 con 0x7f6dd8133420 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.597+0000 7f6ddcb38700 1 -- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 msgr2=0x7f6dd81337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.597+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 0x7f6dd81337f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009dc0 tx=0x7f6dc800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.598+0000 7f6ddcb38700 1 -- 172.21.15.32:0/3281345761 shutdown_connections 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.598+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8133420 0x7f6dd81337f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.598+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/3281345761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.598+0000 7f6ddcb38700 1 -- 172.21.15.32:0/3281345761 >> 172.21.15.32:0/3281345761 conn(0x7f6dd80a4210 msgr2=0x7f6dd80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:55.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.598+0000 7f6ddcb38700 1 -- 172.21.15.32:0/3281345761 shutdown_connections 2023-09-30T22:27:55.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.598+0000 7f6ddcb38700 1 -- 172.21.15.32:0/3281345761 wait complete. 2023-09-30T22:27:55.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 Processor -- start 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 -- start start 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 0x7f6dd809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8134260 0x7f6dd809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd811ef10 con 0x7f6dd8133420 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd812d8c0 con 0x7f6dd8133e30 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.599+0000 7f6ddcb38700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd811fa60 con 0x7f6dd8134260 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 0x7f6dd809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8134260 0x7f6dd809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45018/0 (socket says 172.21.15.32:45018) 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 0x7f6dd809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58716/0 (socket says 172.21.15.32:58716) 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 -- 172.21.15.32:0/1469897252 learned_addr learned my addr 172.21.15.32:0/1469897252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 -- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8134260 msgr2=0x7f6dd809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8134260 0x7f6dd809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 -- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 msgr2=0x7f6dd809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 0x7f6dd809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 -- 172.21.15.32:0/1469897252 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00094a0 con 0x7f6dd8133e30 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd659c700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 0x7f6dd809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd6d9d700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8134260 0x7f6dd809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dd5d9b700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd809c9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000ebb0 tx=0x7f6dc000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000c850 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.600+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc000c9b0 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.601+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc8009a70 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.601+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000cd40 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.601+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd811fd20 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.601+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6dc0013470 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.602+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd8074ea0 con 0x7f6dd8133e30 2023-09-30T22:27:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.602+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6dc003f880 con 0x7f6dd8133e30 2023-09-30T22:27:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.606+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6dc00232f0 con 0x7f6dd8133e30 2023-09-30T22:27:55.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.632+0000 7ff29a9f0700 1 Processor -- start 2023-09-30T22:27:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.632+0000 7ff29a9f0700 1 -- start start 2023-09-30T22:27:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff29a9f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 0x7ff2940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff29a9f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff29409d630 0x7ff2940a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff29a9f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff294133930 0x7ff294133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff29a9f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2940a2a70 con 0x7ff29409d630 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff29a9f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940a01d0 con 0x7ff294133930 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff29a9f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff29409e2b0 con 0x7ff2940a0a30 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff293fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 0x7ff2940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff2937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff29409d630 0x7ff2940a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff2937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff29409d630 0x7ff2940a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58728/0 (socket says 172.21.15.32:58728) 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff293fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 0x7ff2940a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43458/0 (socket says 172.21.15.32:43458) 2023-09-30T22:27:55.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff2937fe700 1 -- 172.21.15.32:0/2308699243 learned_addr learned my addr 172.21.15.32:0/2308699243 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.633+0000 7ff298f8d700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff294133930 0x7ff294133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff2937fe700 1 -- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 msgr2=0x7ff2940a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff2937fe700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 0x7ff2940a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff2937fe700 1 -- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff294133930 msgr2=0x7ff294133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:55.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff2937fe700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff294133930 0x7ff294133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff2937fe700 1 -- 172.21.15.32:0/2308699243 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29409dc50 con 0x7ff29409d630 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff293fff700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 0x7ff2940a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff2937fe700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29409d630 0x7ff2940a1340 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff2880098f0 tx=0x7ff288009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670138f9e684182b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff292ffd700 1 -- 172.21.15.32:0/2308699243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288011830 con 0x7ff29409d630 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff292ffd700 1 -- 172.21.15.32:0/2308699243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288011990 con 0x7ff29409d630 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.634+0000 7ff292ffd700 1 -- 172.21.15.32:0/2308699243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288011d20 con 0x7ff29409d630 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 -- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29409d630 msgr2=0x7ff2940a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29409d630 0x7ff2940a1340 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff2880098f0 tx=0x7ff288009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 -- 172.21.15.32:0/2308699243 shutdown_connections 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff294133930 0x7ff294133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2940a0a30 0x7ff2940a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/2308699243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff29409d630 0x7ff2940a1340 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 -- 172.21.15.32:0/2308699243 >> 172.21.15.32:0/2308699243 conn(0x7ff2940a3050 msgr2=0x7ff2940a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 -- 172.21.15.32:0/2308699243 shutdown_connections 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.635+0000 7ff29a9f0700 1 -- 172.21.15.32:0/2308699243 wait complete. 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.636+0000 7ff29a9f0700 1 Processor -- start 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.636+0000 7ff29a9f0700 1 -- start start 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff29a9f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff29409d630 0x7ff294121120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff29a9f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2940a0a30 0x7ff294121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff29a9f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff294133930 0x7ff294121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff29a9f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff294123c00 con 0x7ff2940a0a30 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff29a9f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff294124c40 con 0x7ff29409d630 2023-09-30T22:27:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff29a9f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940a51d0 con 0x7ff294133930 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff293fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff29409d630 0x7ff294121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2940a0a30 0x7ff294121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff293fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff29409d630 0x7ff294121120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45026/0 (socket says 172.21.15.32:45026) 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff293fff700 1 -- 172.21.15.32:0/3966904093 learned_addr learned my addr 172.21.15.32:0/3966904093 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff298f8d700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff294133930 0x7ff294121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 -- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff294133930 msgr2=0x7ff294121ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff294133930 0x7ff294121ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 -- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff29409d630 msgr2=0x7ff294121120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff29409d630 0x7ff294121120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 -- 172.21.15.32:0/3966904093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840094a0 con 0x7ff2940a0a30 2023-09-30T22:27:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.637+0000 7ff2937fe700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2940a0a30 0x7ff294121660 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff288016000 tx=0x7ff288004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.638+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288004240 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.638+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28801f070 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.638+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880036a0 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.638+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff288009550 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.638+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff294124420 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.639+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29411df60 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.639+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff288021030 con 0x7ff2940a0a30 2023-09-30T22:27:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.639+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff288040ae0 con 0x7ff2940a0a30 2023-09-30T22:27:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.645+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2880239c0 con 0x7ff2940a0a30 2023-09-30T22:27:55.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.777+0000 7f22e3ead700 1 Processor -- start 2023-09-30T22:27:55.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.777+0000 7f22e3ead700 1 -- start start 2023-09-30T22:27:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e3ead700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc1338a0 0x7f22dc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e3ead700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22dc09de50 0x7f22dc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e3ead700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc06ef30 con 0x7f22dc09e760 2023-09-30T22:27:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e3ead700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc06fef0 con 0x7f22dc09de50 2023-09-30T22:27:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e3ead700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1334b0 con 0x7f22dc1338a0 2023-09-30T22:27:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1448700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22dc09de50 0x7f22dc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e244a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f22dc09e760 0x7f22dc13a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60200/0 (socket says 172.21.15.32:60200) 2023-09-30T22:27:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e244a700 1 -- 172.21.15.32:0/954381778 learned_addr learned my addr 172.21.15.32:0/954381778 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1c49700 1 --2- 172.21.15.32:0/954381778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc1338a0 0x7f22dc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1c49700 1 -- 172.21.15.32:0/954381778 >> v2:172.21.15.32:3300/0 conn(0x7f22dc09de50 msgr2=0x7f22dc09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1c49700 1 --2- 172.21.15.32:0/954381778 >> v2:172.21.15.32:3300/0 conn(0x7f22dc09de50 0x7f22dc09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e0c47700 1 -- 172.21.15.32:0/954381778 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746440856 0 0) 0x7f22dc06ef30 con 0x7f22dc09e760 2023-09-30T22:27:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1c49700 1 -- 172.21.15.32:0/954381778 >> v1:172.21.15.32:6789/0 conn(0x7f22dc09e760 legacy=0x7f22dc13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1c49700 1 -- 172.21.15.32:0/954381778 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc06fc30 con 0x7f22dc1338a0 2023-09-30T22:27:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.778+0000 7f22e1448700 1 --2- 172.21.15.32:0/954381778 >> v2:172.21.15.32:3300/0 conn(0x7f22dc09de50 0x7f22dc09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.779+0000 7f22e1c49700 1 --2- 172.21.15.32:0/954381778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc1338a0 0x7f22dc133c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009840 tx=0x7f22d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb2a9d4308ce1f41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.779+0000 7f22e0c47700 1 -- 172.21.15.32:0/954381778 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8011830 con 0x7f22dc1338a0 2023-09-30T22:27:55.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.779+0000 7f22e0c47700 1 -- 172.21.15.32:0/954381778 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8011990 con 0x7f22dc1338a0 2023-09-30T22:27:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.779+0000 7f22e0c47700 1 -- 172.21.15.32:0/954381778 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8011d20 con 0x7f22dc1338a0 2023-09-30T22:27:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.779+0000 7f22e3ead700 1 -- 172.21.15.32:0/954381778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc1338a0 msgr2=0x7f22dc133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.779+0000 7f22e3ead700 1 --2- 172.21.15.32:0/954381778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc1338a0 0x7f22dc133c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009840 tx=0x7f22d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:27:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.780+0000 7f22e3ead700 1 -- 172.21.15.32:0/954381778 shutdown_connections 2023-09-30T22:27:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.780+0000 7f22e3ead700 1 --2- 172.21.15.32:0/954381778 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc1338a0 0x7f22dc133c70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.780+0000 7f22e3ead700 1 --2- 172.21.15.32:0/954381778 >> v2:172.21.15.32:3300/0 conn(0x7f22dc09de50 0x7f22dc09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.780+0000 7f22e3ead700 1 -- 172.21.15.32:0/954381778 >> 172.21.15.32:0/954381778 conn(0x7f22dc0a2aa0 msgr2=0x7f22dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.780+0000 7f22e3ead700 1 -- 172.21.15.32:0/954381778 shutdown_connections 2023-09-30T22:27:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.780+0000 7f22e3ead700 1 -- 172.21.15.32:0/954381778 wait complete. 2023-09-30T22:27:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 Processor -- start 2023-09-30T22:27:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 -- start start 2023-09-30T22:27:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc09de50 0x7f22dc11a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 0x7f22dc11ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22dc1338a0 0x7f22dc11b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc11c890 con 0x7f22dc09de50 2023-09-30T22:27:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc11e150 con 0x7f22dc1338a0 2023-09-30T22:27:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e3ead700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc11d520 con 0x7f22dc09e760 2023-09-30T22:27:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e1448700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 0x7f22dc11ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.781+0000 7f22e1448700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 0x7f22dc11ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43484/0 (socket says 172.21.15.32:43484) 2023-09-30T22:27:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 -- 172.21.15.32:0/610495594 learned_addr learned my addr 172.21.15.32:0/610495594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e244a700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22dc1338a0 0x7f22dc11b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1c49700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc09de50 0x7f22dc11a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 -- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22dc1338a0 msgr2=0x7f22dc11b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22dc1338a0 0x7f22dc11b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 -- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc09de50 msgr2=0x7f22dc11a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc09de50 0x7f22dc11a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 -- 172.21.15.32:0/610495594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0094a0 con 0x7f22dc09e760 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e1448700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 0x7f22dc11ab70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00ebe0 tx=0x7f22cc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00c850 con 0x7f22dc09e760 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc00c9b0 con 0x7f22dc09e760 2023-09-30T22:27:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80094a0 con 0x7f22dc09e760 2023-09-30T22:27:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc0a0ba0 con 0x7f22dc09e760 2023-09-30T22:27:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.782+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00cd40 con 0x7f22dc09e760 2023-09-30T22:27:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.783+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22cc013470 con 0x7f22dc09e760 2023-09-30T22:27:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.784+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f22cc03f800 con 0x7f22dc09e760 2023-09-30T22:27:55.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.784+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c40052e0 con 0x7f22dc09e760 2023-09-30T22:27:55.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.793+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22cc013720 con 0x7f22dc09e760 2023-09-30T22:27:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.829+0000 7f48f1c3b700 1 Processor -- start 2023-09-30T22:27:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.829+0000 7f48f1c3b700 1 -- start start 2023-09-30T22:27:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.829+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6dd806b250 con 0x7f6dd8133e30 2023-09-30T22:27:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.829+0000 7f6dceffd700 1 -- 172.21.15.32:0/1469897252 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6dc0021020 con 0x7f6dd8133e30 2023-09-30T22:27:55.830 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48f1c3b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48ec09d6f0 0x7f48ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48f1c3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 0x7f48ec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48f1c3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09e410 0x7f48ec133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48f1c3b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec06b460 con 0x7f48ec09d6f0 2023-09-30T22:27:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48f1c3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec09b120 con 0x7f48ec09e410 2023-09-30T22:27:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48f1c3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec09a7e0 con 0x7f48ec09e000 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48ec09d6f0 0x7f48ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48ec09d6f0 0x7f48ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58758/0 (socket says 172.21.15.32:58758) 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 -- 172.21.15.32:0/278117979 learned_addr learned my addr 172.21.15.32:0/278117979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48ebfff700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09e410 0x7f48ec133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 -- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 msgr2=0x7f48ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 0x7f48ec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 -- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09e410 msgr2=0x7f48ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09e410 0x7f48ec133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 -- 172.21.15.32:0/278117979 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec09a500 con 0x7f48ec09d6f0 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48ebfff700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09e410 0x7f48ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.830+0000 7f48eb7fe700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09d6f0 0x7f48ec09dac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0098f0 tx=0x7f48dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d592a767c823fb1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.831+0000 7f48ea7fc700 1 -- 172.21.15.32:0/278117979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc011830 con 0x7f48ec09d6f0 2023-09-30T22:27:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.831+0000 7f48ea7fc700 1 -- 172.21.15.32:0/278117979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc011990 con 0x7f48ec09d6f0 2023-09-30T22:27:55.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.831+0000 7f48ea7fc700 1 -- 172.21.15.32:0/278117979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc011d20 con 0x7f48ec09d6f0 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.831+0000 7f48f1c3b700 1 -- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09d6f0 msgr2=0x7f48ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.831+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09d6f0 0x7f48ec09dac0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0098f0 tx=0x7f48dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 -- 172.21.15.32:0/278117979 shutdown_connections 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09e410 0x7f48ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09d6f0 0x7f48ec09dac0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/278117979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 0x7f48ec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 -- 172.21.15.32:0/278117979 >> 172.21.15.32:0/278117979 conn(0x7f48ec066f50 msgr2=0x7f48ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 -- 172.21.15.32:0/278117979 shutdown_connections 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.832+0000 7f48f1c3b700 1 -- 172.21.15.32:0/278117979 wait complete. 2023-09-30T22:27:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.833+0000 7f48f1c3b700 1 Processor -- start 2023-09-30T22:27:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.833+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 msgr2=0x7f6dd809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.833+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd809c9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000ebb0 tx=0x7f6dc000eec0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.833+0000 7f48f1c3b700 1 -- start start 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 shutdown_connections 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6dd8134260 0x7f6dd809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6dd8133420 0x7f6dd809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 --2- 172.21.15.32:0/1469897252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6dd8133e30 0x7f6dd809c9c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 >> 172.21.15.32:0/1469897252 conn(0x7f6dd80a4210 msgr2=0x7f6dd80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 shutdown_connections 2023-09-30T22:27:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f6ddcb38700 1 -- 172.21.15.32:0/1469897252 wait complete. 2023-09-30T22:27:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48f1c3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09d6f0 0x7f48ec1c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48f1c3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 0x7f48ec1c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48f1c3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 0x7f48ec1c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48f1c3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec1a9600 con 0x7f48ec09e410 2023-09-30T22:27:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48f1c3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec1ab920 con 0x7f48ec09d6f0 2023-09-30T22:27:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48f1c3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec09c710 con 0x7f48ec09e000 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 0x7f48ec1c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 0x7f48ec1c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58760/0 (socket says 172.21.15.32:58760) 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48ebfff700 1 -- 172.21.15.32:0/4002287877 learned_addr learned my addr 172.21.15.32:0/4002287877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.834+0000 7f48eb7fe700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09d6f0 0x7f48ec1c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48ebfff700 1 -- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 msgr2=0x7f48ec1c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48ebfff700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 0x7f48ec1c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48ebfff700 1 -- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09d6f0 msgr2=0x7f48ec1c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48ebfff700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09d6f0 0x7f48ec1c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48ebfff700 1 -- 172.21.15.32:0/4002287877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4009a70 con 0x7f48ec09e410 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48eb7fe700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09d6f0 0x7f48ec1c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:55.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48ebfff700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 0x7f48ec1c4c80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f48e400efc0 tx=0x7f48e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4009d70 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e400f040 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400ce00 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc009550 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.835+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec1aab10 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.836+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48e4014480 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.836+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec074ea0 con 0x7f48ec09e410 2023-09-30T22:27:55.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.837+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f48e403fdf0 con 0x7f48ec09e410 2023-09-30T22:27:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.842+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48e4012930 con 0x7f48ec09e410 2023-09-30T22:27:55.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:55.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.868+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff2940747f0 con 0x7ff2940a0a30 2023-09-30T22:27:55.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.869+0000 7ff290ff9700 1 -- 172.21.15.32:0/3966904093 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff288015070 con 0x7ff2940a0a30 2023-09-30T22:27:55.869 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:55.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2940a0a30 msgr2=0x7ff294121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:55.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2940a0a30 0x7ff294121660 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff288016000 tx=0x7ff288004750 comp rx=0 tx=0).stop 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 shutdown_connections 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff294133930 0x7ff294121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff29409d630 0x7ff294121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 --2- 172.21.15.32:0/3966904093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2940a0a30 0x7ff294121660 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 >> 172.21.15.32:0/3966904093 conn(0x7ff2940a3050 msgr2=0x7ff2941331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 shutdown_connections 2023-09-30T22:27:55.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:55.871+0000 7ff29a9f0700 1 -- 172.21.15.32:0/3966904093 wait complete. 2023-09-30T22:27:55.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.002+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f22c4005590 con 0x7f22dc09e760 2023-09-30T22:27:56.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.003+0000 7f22d27fc700 1 -- 172.21.15.32:0/610495594 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22cc02a090 con 0x7f22dc09e760 2023-09-30T22:27:56.003 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 msgr2=0x7f22dc11ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:56.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 0x7f22dc11ab70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00ebe0 tx=0x7f22cc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 shutdown_connections 2023-09-30T22:27:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22dc1338a0 0x7f22dc11b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22dc09de50 0x7f22dc11a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 --2- 172.21.15.32:0/610495594 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22dc09e760 0x7f22dc11ab70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 >> 172.21.15.32:0/610495594 conn(0x7f22dc0a2aa0 msgr2=0x7f22dc130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 shutdown_connections 2023-09-30T22:27:56.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.005+0000 7f22e3ead700 1 -- 172.21.15.32:0/610495594 wait complete. 2023-09-30T22:27:56.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.047+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f48ec0a2570 con 0x7f48ec09e410 2023-09-30T22:27:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.047+0000 7f48d3fff700 1 -- 172.21.15.32:0/4002287877 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48e4012340 con 0x7f48ec09e410 2023-09-30T22:27:56.048 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:56.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.049+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 msgr2=0x7f48ec1c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.049+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 0x7f48ec1c4c80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f48e400efc0 tx=0x7f48e400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 shutdown_connections 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48ec09e410 0x7f48ec1c4c80 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48ec09d6f0 0x7f48ec1c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 --2- 172.21.15.32:0/4002287877 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f48ec09e000 0x7f48ec1c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 >> 172.21.15.32:0/4002287877 conn(0x7f48ec066f50 msgr2=0x7f48ec06f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 shutdown_connections 2023-09-30T22:27:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:56.050+0000 7f48f1c3b700 1 -- 172.21.15.32:0/4002287877 wait complete. 2023-09-30T22:27:56.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:56.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:56.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:57.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.002+0000 7f31d19b9700 1 Processor -- start 2023-09-30T22:27:57.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.002+0000 7f31d19b9700 1 -- start start 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31d19b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 0x7f31cc0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31d19b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31cc09d5c0 0x7f31cc0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31d19b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc133920 0x7f31cc133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31d19b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31cc0a1810 con 0x7f31cc09d5c0 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31d19b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc09ea10 con 0x7f31cc133920 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31d19b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc09e210 con 0x7f31cc0a09c0 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 0x7f31cc0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31cc09d5c0 0x7f31cc0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31cc09d5c0 0x7f31cc0a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58772/0 (socket says 172.21.15.32:58772) 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 0x7f31cc0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43504/0 (socket says 172.21.15.32:43504) 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 -- 172.21.15.32:0/776632706 learned_addr learned my addr 172.21.15.32:0/776632706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc133920 0x7f31cc133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 -- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 msgr2=0x7f31cc0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 0x7f31cc0a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 -- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc133920 msgr2=0x7f31cc133dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.003+0000 7f31ca7fc700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc133920 0x7f31cc133dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.004+0000 7f31ca7fc700 1 -- 172.21.15.32:0/776632706 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc0a26f0 con 0x7f31cc09d5c0 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.004+0000 7f31caffd700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 0x7f31cc0a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:57.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.004+0000 7f31ca7fc700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc09d5c0 0x7f31cc0a12d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f31c00098f0 tx=0x7f31c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b372df1ba76255b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.004+0000 7f31c9ffb700 1 -- 172.21.15.32:0/776632706 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0011830 con 0x7f31cc09d5c0 2023-09-30T22:27:57.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.004+0000 7f31c9ffb700 1 -- 172.21.15.32:0/776632706 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0011990 con 0x7f31cc09d5c0 2023-09-30T22:27:57.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.004+0000 7f31c9ffb700 1 -- 172.21.15.32:0/776632706 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0011d20 con 0x7f31cc09d5c0 2023-09-30T22:27:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 -- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc09d5c0 msgr2=0x7f31cc0a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc09d5c0 0x7f31cc0a12d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f31c00098f0 tx=0x7f31c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 -- 172.21.15.32:0/776632706 shutdown_connections 2023-09-30T22:27:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc133920 0x7f31cc133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc0a09c0 0x7f31cc0a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 --2- 172.21.15.32:0/776632706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc09d5c0 0x7f31cc0a12d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 -- 172.21.15.32:0/776632706 >> 172.21.15.32:0/776632706 conn(0x7f31cc0a2fe0 msgr2=0x7f31cc0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 -- 172.21.15.32:0/776632706 shutdown_connections 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.005+0000 7f31d19b9700 1 -- 172.21.15.32:0/776632706 wait complete. 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.006+0000 7f31d19b9700 1 Processor -- start 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.006+0000 7f31d19b9700 1 -- start start 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.006+0000 7f31d19b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc09d5c0 0x7f31cc121170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.006+0000 7f31d19b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc0a09c0 0x7f31cc1216b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31d19b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc133920 0x7f31cc121bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31d19b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc122d80 con 0x7f31cc133920 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31d19b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc123c10 con 0x7f31cc0a09c0 2023-09-30T22:27:57.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31d19b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1246f0 con 0x7f31cc09d5c0 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31ca7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc0a09c0 0x7f31cc1216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc09d5c0 0x7f31cc121170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31ca7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc0a09c0 0x7f31cc1216b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45076/0 (socket says 172.21.15.32:45076) 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31ca7fc700 1 -- 172.21.15.32:0/4148691549 learned_addr learned my addr 172.21.15.32:0/4148691549 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31cb7fe700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc133920 0x7f31cc121bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 -- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc0a09c0 msgr2=0x7f31cc1216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc0a09c0 0x7f31cc1216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 -- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc133920 msgr2=0x7f31cc121bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc133920 0x7f31cc121bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 -- 172.21.15.32:0/4148691549 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc009a70 con 0x7f31cc09d5c0 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31caffd700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc09d5c0 0x7f31cc121170 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00efc0 tx=0x7f31bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00cd10 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31bc00f040 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c0009550 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.007+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0146a0 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.008+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc09cec0 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.008+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31bc014800 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.009+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc11df50 con 0x7f31cc09d5c0 2023-09-30T22:27:57.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.009+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f31bc0405b0 con 0x7f31cc09d5c0 2023-09-30T22:27:57.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.013+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31bc014d40 con 0x7f31cc09d5c0 2023-09-30T22:27:57.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:57.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.045+0000 7f9378448700 1 Processor -- start 2023-09-30T22:27:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.045+0000 7f9378448700 1 -- start start 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f9378448700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f9378448700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f937009e410 0x7f9370130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f9378448700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937009f130 con 0x7f937009e000 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f9378448700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937009f520 con 0x7f937009e410 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f9378448700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937009e930 con 0x7f937009d6f0 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f93769e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f937009e410 0x7f9370130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f93759e3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f937009e000 0x7f9370130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60216/0 (socket says 172.21.15.32:60216) 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.046+0000 7f93759e3700 1 -- 172.21.15.32:0/3787749912 learned_addr learned my addr 172.21.15.32:0/3787749912 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93761e4700 1 --2- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93769e5700 1 -- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 msgr2=0x7f937009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93769e5700 1 --2- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93769e5700 1 -- 172.21.15.32:0/3787749912 >> v1:172.21.15.32:6789/0 conn(0x7f937009e000 legacy=0x7f9370130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93751e2700 1 -- 172.21.15.32:0/3787749912 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193452115 0 0) 0x7f937009f130 con 0x7f937009e000 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93769e5700 1 -- 172.21.15.32:0/3787749912 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937009b2d0 con 0x7f937009e410 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93769e5700 1 --2- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e410 0x7f9370130d70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f936800bfa0 tx=0x7f936800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2825d81ef889028 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93751e2700 1 -- 172.21.15.32:0/3787749912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800b2d0 con 0x7f937009e410 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93751e2700 1 -- 172.21.15.32:0/3787749912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9368010070 con 0x7f937009e410 2023-09-30T22:27:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.047+0000 7f93751e2700 1 -- 172.21.15.32:0/3787749912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800db80 con 0x7f937009e410 2023-09-30T22:27:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 -- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e410 msgr2=0x7f9370130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 --2- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e410 0x7f9370130d70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f936800bfa0 tx=0x7f936800d330 comp rx=0 tx=0).stop 2023-09-30T22:27:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 -- 172.21.15.32:0/3787749912 shutdown_connections 2023-09-30T22:27:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 --2- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e410 0x7f9370130d70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 --2- 172.21.15.32:0/3787749912 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 -- 172.21.15.32:0/3787749912 >> 172.21.15.32:0/3787749912 conn(0x7f9370066f50 msgr2=0x7f937009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 -- 172.21.15.32:0/3787749912 shutdown_connections 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.048+0000 7f9378448700 1 -- 172.21.15.32:0/3787749912 wait complete. 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.049+0000 7f9378448700 1 Processor -- start 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.049+0000 7f9378448700 1 -- start start 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f9378448700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937011e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f9378448700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 0x7f937011e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f9378448700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f937009e410 0x7f937011ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f9378448700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937009c790 con 0x7f937009e000 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f9378448700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937009a0c0 con 0x7f937009d6f0 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f9378448700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9370120480 con 0x7f937009e410 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 0x7f937011e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 0x7f937011e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58788/0 (socket says 172.21.15.32:58788) 2023-09-30T22:27:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 -- 172.21.15.32:0/2972445489 learned_addr learned my addr 172.21.15.32:0/2972445489 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93761e4700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937011e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93769e5700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f937009e410 0x7f937011ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 -- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f937009e410 msgr2=0x7f937011ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f937009e410 0x7f937011ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 -- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 msgr2=0x7f937011e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937011e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.050+0000 7f93759e3700 1 -- 172.21.15.32:0/2972445489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936800bc80 con 0x7f937009e000 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.051+0000 7f93759e3700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 0x7f937011e550 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f936000c520 tx=0x7f936000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.051+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360015d70 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.051+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9360019070 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.051+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93701212c0 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.051+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360004ab0 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.051+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9370120170 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.052+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9360004c10 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.052+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9370074ea0 con 0x7f937009e000 2023-09-30T22:27:57.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.052+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9360047200 con 0x7f937009e000 2023-09-30T22:27:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.058+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f936001b830 con 0x7f937009e000 2023-09-30T22:27:57.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:57.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 Processor -- start 2023-09-30T22:27:57.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 -- start start 2023-09-30T22:27:57.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f803409d6f0 0x7f803409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 0x7f8034130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f8034130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803409f520 con 0x7f803409d6f0 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803409f130 con 0x7f803409e000 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.166+0000 7f803a4c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f803409e930 con 0x7f803409e410 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 0x7f8034130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f8038a63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f8034130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 0x7f8034130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45100/0 (socket says 172.21.15.32:45100) 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 -- 172.21.15.32:0/1274971972 learned_addr learned my addr 172.21.15.32:0/1274971972 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f8033fff700 1 --2- 172.21.15.32:0/1274971972 >> v2:172.21.15.32:3300/0 conn(0x7f803409d6f0 0x7f803409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 -- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 msgr2=0x7f8034130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 --2- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f8034130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 -- 172.21.15.32:0/1274971972 >> v2:172.21.15.32:3300/0 conn(0x7f803409d6f0 msgr2=0x7f803409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 --2- 172.21.15.32:0/1274971972 >> v2:172.21.15.32:3300/0 conn(0x7f803409d6f0 0x7f803409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 -- 172.21.15.32:0/1274971972 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803409b2d0 con 0x7f803409e000 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f80337fe700 1 --2- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 0x7f8034130660 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f80280098f0 tx=0x7f8028009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940e9a99c557e1ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.167+0000 7f8032ffd700 1 -- 172.21.15.32:0/1274971972 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028011830 con 0x7f803409e000 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.168+0000 7f8032ffd700 1 -- 172.21.15.32:0/1274971972 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028011990 con 0x7f803409e000 2023-09-30T22:27:57.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.168+0000 7f8032ffd700 1 -- 172.21.15.32:0/1274971972 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028011d20 con 0x7f803409e000 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 -- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 msgr2=0x7f8034130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 --2- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 0x7f8034130660 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f80280098f0 tx=0x7f8028009c00 comp rx=0 tx=0).stop 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 -- 172.21.15.32:0/1274971972 shutdown_connections 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 --2- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f8034130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 --2- 172.21.15.32:0/1274971972 >> v2:172.21.15.32:3300/0 conn(0x7f803409d6f0 0x7f803409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 --2- 172.21.15.32:0/1274971972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409e000 0x7f8034130660 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 -- 172.21.15.32:0/1274971972 >> 172.21.15.32:0/1274971972 conn(0x7f8034066f50 msgr2=0x7f803409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 -- 172.21.15.32:0/1274971972 shutdown_connections 2023-09-30T22:27:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.169+0000 7f803a4c6700 1 -- 172.21.15.32:0/1274971972 wait complete. 2023-09-30T22:27:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.170+0000 7f803a4c6700 1 Processor -- start 2023-09-30T22:27:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.170+0000 7f803a4c6700 1 -- start start 2023-09-30T22:27:57.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f803a4c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 0x7f80341c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f803a4c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803409e000 0x7f80341c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f803a4c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f80341c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f803a4c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8034121820 con 0x7f803409e000 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f803a4c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803411efe0 con 0x7f803409d6f0 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f803a4c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f803409fd10 con 0x7f803409e410 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f8033fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 0x7f80341c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f8033fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 0x7f80341c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45116/0 (socket says 172.21.15.32:45116) 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f8033fff700 1 -- 172.21.15.32:0/866422952 learned_addr learned my addr 172.21.15.32:0/866422952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.171+0000 7f8038a63700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f80341c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f80337fe700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803409e000 0x7f80341c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8033fff700 1 -- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 msgr2=0x7f80341c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8033fff700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f80341c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8033fff700 1 -- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803409e000 msgr2=0x7f80341c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8033fff700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803409e000 0x7f80341c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8033fff700 1 -- 172.21.15.32:0/866422952 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028009550 con 0x7f803409d6f0 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f80337fe700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803409e000 0x7f80341c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8033fff700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 0x7f80341c62f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f802400e840 tx=0x7f802400eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400c070 con 0x7f803409d6f0 2023-09-30T22:27:57.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024013610 con 0x7f803409d6f0 2023-09-30T22:27:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80341213b0 con 0x7f803409d6f0 2023-09-30T22:27:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240139a0 con 0x7f803409d6f0 2023-09-30T22:27:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.172+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80341c6d70 con 0x7f803409d6f0 2023-09-30T22:27:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.174+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803409c760 con 0x7f803409d6f0 2023-09-30T22:27:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.174+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8024013b00 con 0x7f803409d6f0 2023-09-30T22:27:57.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.174+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f802403f310 con 0x7f803409d6f0 2023-09-30T22:27:57.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.179+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80240104f0 con 0x7f803409d6f0 2023-09-30T22:27:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.207+0000 7f2fc2dad700 1 Processor -- start 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.207+0000 7f2fc2dad700 1 -- start start 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fc2dad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fbc09d6f0 0x7f2fbc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fc2dad700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fc2dad700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc09f130 con 0x7f2fbc09e410 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fc2dad700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc09f520 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fc2dad700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc09e930 con 0x7f2fbc09e000 2023-09-30T22:27:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fbbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fc0b49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fbc09d6f0 0x7f2fbc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fbbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43566/0 (socket says 172.21.15.32:43566) 2023-09-30T22:27:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.208+0000 7f2fbbfff700 1 -- 172.21.15.32:0/2341874384 learned_addr learned my addr 172.21.15.32:0/2341874384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbbfff700 1 -- 172.21.15.32:0/2341874384 >> v2:172.21.15.32:3300/0 conn(0x7f2fbc09d6f0 msgr2=0x7f2fbc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbbfff700 1 --2- 172.21.15.32:0/2341874384 >> v2:172.21.15.32:3300/0 conn(0x7f2fbc09d6f0 0x7f2fbc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbbfff700 1 -- 172.21.15.32:0/2341874384 >> v1:172.21.15.32:6789/0 conn(0x7f2fbc09e410 legacy=0x7f2fbc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbbfff700 1 -- 172.21.15.32:0/2341874384 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc09b2d0 con 0x7f2fbc09e000 2023-09-30T22:27:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbb7fe700 1 -- 172.21.15.32:0/2341874384 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365839331 0 0) 0x7f2fbc09f130 con 0x7f2fbc09e410 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbbfff700 1 --2- 172.21.15.32:0/2341874384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc130660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800b9a0 tx=0x7f2fa800bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb9fbce799fe17f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbb7fe700 1 -- 172.21.15.32:0/2341874384 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800caa0 con 0x7f2fbc09e000 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbb7fe700 1 -- 172.21.15.32:0/2341874384 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa800cc00 con 0x7f2fbc09e000 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.209+0000 7f2fbb7fe700 1 -- 172.21.15.32:0/2341874384 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800f460 con 0x7f2fbc09e000 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2341874384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 msgr2=0x7f2fbc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2341874384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc130660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800b9a0 tx=0x7f2fa800bcb0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2341874384 shutdown_connections 2023-09-30T22:27:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2341874384 >> v2:172.21.15.32:3300/0 conn(0x7f2fbc09d6f0 0x7f2fbc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2341874384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc130660 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2341874384 >> 172.21.15.32:0/2341874384 conn(0x7f2fbc066f50 msgr2=0x7f2fbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2341874384 shutdown_connections 2023-09-30T22:27:57.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.210+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2341874384 wait complete. 2023-09-30T22:27:57.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.211+0000 7f2fc2dad700 1 Processor -- start 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.211+0000 7f2fc2dad700 1 -- start start 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc2dad700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d6f0 0x7f2fbc11e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc2dad700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc11e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc2dad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09e410 0x7f2fbc11eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc2dad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc121810 con 0x7f2fbc09e410 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc2dad700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc120200 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc2dad700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc121280 con 0x7f2fbc09e000 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fbbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc11e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fbbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc11e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43570/0 (socket says 172.21.15.32:43570) 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fbbfff700 1 -- 172.21.15.32:0/2043125239 learned_addr learned my addr 172.21.15.32:0/2043125239 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc0b49700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d6f0 0x7f2fbc11e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.212+0000 7f2fc134a700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09e410 0x7f2fbc11eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc0b49700 1 -- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 msgr2=0x7f2fbc11e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc0b49700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc11e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc0b49700 1 -- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09e410 msgr2=0x7f2fbc11eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc0b49700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09e410 0x7f2fbc11eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc0b49700 1 -- 172.21.15.32:0/2043125239 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa800b600 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fbbfff700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc11e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc134a700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09e410 0x7f2fbc11eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc0b49700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d6f0 0x7f2fbc11e070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012c10 tx=0x7f2fb0012f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:57.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0010870 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0013040 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc09c700 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0010bd0 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.213+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc120e30 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.214+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc121950 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.214+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2fb0010d30 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.215+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2fb00438c0 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.219+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2fbc121950 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.235+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f31cc124830 con 0x7f31cc09d5c0 2023-09-30T22:27:57.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.235+0000 7f31b3fff700 1 -- 172.21.15.32:0/4148691549 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31bc012340 con 0x7f31cc09d5c0 2023-09-30T22:27:57.236 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc09d5c0 msgr2=0x7f31cc121170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc09d5c0 0x7f31cc121170 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00efc0 tx=0x7f31bc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 shutdown_connections 2023-09-30T22:27:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31cc133920 0x7f31cc121bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31cc09d5c0 0x7f31cc121170 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 --2- 172.21.15.32:0/4148691549 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31cc0a09c0 0x7f31cc1216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 >> 172.21.15.32:0/4148691549 conn(0x7f31cc0a2fe0 msgr2=0x7f31cc133190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 shutdown_connections 2023-09-30T22:27:57.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.238+0000 7f31d19b9700 1 -- 172.21.15.32:0/4148691549 wait complete. 2023-09-30T22:27:57.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:57.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.279+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9370121690 con 0x7f937009e000 2023-09-30T22:27:57.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.280+0000 7f9366ffd700 1 -- 172.21.15.32:0/2972445489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9360024030 con 0x7f937009e000 2023-09-30T22:27:57.280 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 msgr2=0x7f937011e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 0x7f937011e550 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f936000c520 tx=0x7f936000c830 comp rx=0 tx=0).stop 2023-09-30T22:27:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 shutdown_connections 2023-09-30T22:27:57.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f937009e410 0x7f937011ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f937009d6f0 0x7f937011e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 --2- 172.21.15.32:0/2972445489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f937009e000 0x7f937011e550 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 >> 172.21.15.32:0/2972445489 conn(0x7f9370066f50 msgr2=0x7f9370070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 shutdown_connections 2023-09-30T22:27:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.282+0000 7f9378448700 1 -- 172.21.15.32:0/2972445489 wait complete. 2023-09-30T22:27:57.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.401+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8034120fd0 con 0x7f803409d6f0 2023-09-30T22:27:57.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.402+0000 7f8030ff9700 1 -- 172.21.15.32:0/866422952 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8024010310 con 0x7f803409d6f0 2023-09-30T22:27:57.402 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 msgr2=0x7f80341c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 0x7f80341c62f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f802400e840 tx=0x7f802400eb50 comp rx=0 tx=0).stop 2023-09-30T22:27:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 shutdown_connections 2023-09-30T22:27:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f803409e410 0x7f80341c28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f803409d6f0 0x7f80341c62f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 --2- 172.21.15.32:0/866422952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803409e000 0x7f80341c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 >> 172.21.15.32:0/866422952 conn(0x7f8034066f50 msgr2=0x7f8034130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 shutdown_connections 2023-09-30T22:27:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.404+0000 7f803a4c6700 1 -- 172.21.15.32:0/866422952 wait complete. 2023-09-30T22:27:57.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.431+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2fbc121950 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.431+0000 7f2fb97fa700 1 -- 172.21.15.32:0/2043125239 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fbc121950 con 0x7f2fbc09d6f0 2023-09-30T22:27:57.431 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:57.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d6f0 msgr2=0x7f2fbc11e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d6f0 0x7f2fbc11e070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012c10 tx=0x7f2fb0012f20 comp rx=0 tx=0).stop 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 shutdown_connections 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fbc09e410 0x7f2fbc11eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fbc09d6f0 0x7f2fbc11e070 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 --2- 172.21.15.32:0/2043125239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fbc09e000 0x7f2fbc11e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 >> 172.21.15.32:0/2043125239 conn(0x7f2fbc066f50 msgr2=0x7f2fbc130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 shutdown_connections 2023-09-30T22:27:57.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:57.434+0000 7f2fc2dad700 1 -- 172.21.15.32:0/2043125239 wait complete. 2023-09-30T22:27:57.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:58.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:58.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:58.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.403+0000 7f296588c700 1 Processor -- start 2023-09-30T22:27:58.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.403+0000 7f296588c700 1 -- start start 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f296588c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29600a0a30 0x7f29600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f296588c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 0x7f296013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f296588c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600a2a70 con 0x7f296009d630 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f296588c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29600a01d0 con 0x7f29600a0a30 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f296588c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f296009e2b0 con 0x7f2960138d80 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 0x7f296013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 0x7f296013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43584/0 (socket says 172.21.15.32:43584) 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 -- 172.21.15.32:0/2948129956 learned_addr learned my addr 172.21.15.32:0/2948129956 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295effd700 1 --2- 172.21.15.32:0/2948129956 >> v2:172.21.15.32:3300/0 conn(0x7f29600a0a30 0x7f29600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 -- 172.21.15.32:0/2948129956 >> v2:172.21.15.32:3300/0 conn(0x7f29600a0a30 msgr2=0x7f29600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 --2- 172.21.15.32:0/2948129956 >> v2:172.21.15.32:3300/0 conn(0x7f29600a0a30 0x7f29600a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 -- 172.21.15.32:0/2948129956 >> v1:172.21.15.32:6789/0 conn(0x7f296009d630 legacy=0x7f29600a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295f7fe700 1 -- 172.21.15.32:0/2948129956 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296009dc50 con 0x7f2960138d80 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295effd700 1 --2- 172.21.15.32:0/2948129956 >> v2:172.21.15.32:3300/0 conn(0x7f29600a0a30 0x7f29600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.404+0000 7f295e7fc700 1 -- 172.21.15.32:0/2948129956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359729209 0 0) 0x7f29600a2a70 con 0x7f296009d630 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.405+0000 7f295f7fe700 1 --2- 172.21.15.32:0/2948129956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 0x7f296013b160 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2958009e50 tx=0x7f295800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd74fbe9a0bc5f2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.405+0000 7f295e7fc700 1 -- 172.21.15.32:0/2948129956 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295800e070 con 0x7f2960138d80 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.405+0000 7f295e7fc700 1 -- 172.21.15.32:0/2948129956 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295800b910 con 0x7f2960138d80 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.405+0000 7f295e7fc700 1 -- 172.21.15.32:0/2948129956 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295800bcc0 con 0x7f2960138d80 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.405+0000 7f296588c700 1 -- 172.21.15.32:0/2948129956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 msgr2=0x7f296013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.405+0000 7f296588c700 1 --2- 172.21.15.32:0/2948129956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 0x7f296013b160 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2958009e50 tx=0x7f295800b330 comp rx=0 tx=0).stop 2023-09-30T22:27:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.406+0000 7f296588c700 1 -- 172.21.15.32:0/2948129956 shutdown_connections 2023-09-30T22:27:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.406+0000 7f296588c700 1 --2- 172.21.15.32:0/2948129956 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2960138d80 0x7f296013b160 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.406+0000 7f296588c700 1 --2- 172.21.15.32:0/2948129956 >> v2:172.21.15.32:3300/0 conn(0x7f29600a0a30 0x7f29600a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.406+0000 7f296588c700 1 -- 172.21.15.32:0/2948129956 >> 172.21.15.32:0/2948129956 conn(0x7f29600a3050 msgr2=0x7f29600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.406+0000 7f296588c700 1 -- 172.21.15.32:0/2948129956 shutdown_connections 2023-09-30T22:27:58.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.406+0000 7f296588c700 1 -- 172.21.15.32:0/2948129956 wait complete. 2023-09-30T22:27:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 Processor -- start 2023-09-30T22:27:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 -- start start 2023-09-30T22:27:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 0x7f2960133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29600a0a30 0x7f2960133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2960138d80 0x7f29601340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2960137170 con 0x7f296009d630 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2960136060 con 0x7f2960138d80 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f296588c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2960136520 con 0x7f29600a0a30 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.407+0000 7f295effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 0x7f2960133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29600a0a30 0x7f2960133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 0x7f2960133650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58854/0 (socket says 172.21.15.32:58854) 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295effd700 1 -- 172.21.15.32:0/2167655330 learned_addr learned my addr 172.21.15.32:0/2167655330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295f7fe700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2960138d80 0x7f29601340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 -- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2960138d80 msgr2=0x7f29601340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2960138d80 0x7f29601340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 -- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 msgr2=0x7f2960133650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 0x7f2960133650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 -- 172.21.15.32:0/2167655330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500094a0 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295effd700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 0x7f2960133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f29567fc700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29600a0a30 0x7f2960133b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f294800c430 tx=0x7f294800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480054e0 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948003680 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2958009b30 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948003a30 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.408+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2960136bd0 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.409+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2948003b90 con 0x7f29600a0a30 2023-09-30T22:27:58.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.410+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f294803ead0 con 0x7f29600a0a30 2023-09-30T22:27:58.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.410+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2960137d50 con 0x7f29600a0a30 2023-09-30T22:27:58.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.415+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2948003e40 con 0x7f29600a0a30 2023-09-30T22:27:58.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.446+0000 7fb64b889700 1 Processor -- start 2023-09-30T22:27:58.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.446+0000 7fb64b889700 1 -- start start 2023-09-30T22:27:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb64b889700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb64409d6f0 0x7fb64409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb64b889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 0x7fb644130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb64b889700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64409f130 con 0x7fb64409e000 2023-09-30T22:27:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb64b889700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64409f520 con 0x7fb64409d6f0 2023-09-30T22:27:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb64b889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64409e930 con 0x7fb64409e410 2023-09-30T22:27:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb649e26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 0x7fb644130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb649e26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 0x7fb644130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45152/0 (socket says 172.21.15.32:45152) 2023-09-30T22:27:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb649e26700 1 -- 172.21.15.32:0/3872915731 learned_addr learned my addr 172.21.15.32:0/3872915731 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.447+0000 7fb649625700 1 --2- 172.21.15.32:0/3872915731 >> v2:172.21.15.32:3300/0 conn(0x7fb64409d6f0 0x7fb64409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb649e26700 1 -- 172.21.15.32:0/3872915731 >> v2:172.21.15.32:3300/0 conn(0x7fb64409d6f0 msgr2=0x7fb64409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb649e26700 1 --2- 172.21.15.32:0/3872915731 >> v2:172.21.15.32:3300/0 conn(0x7fb64409d6f0 0x7fb64409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb649e26700 1 -- 172.21.15.32:0/3872915731 >> v1:172.21.15.32:6789/0 conn(0x7fb64409e000 legacy=0x7fb644130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb649e26700 1 -- 172.21.15.32:0/3872915731 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64409b2d0 con 0x7fb64409e410 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb649625700 1 --2- 172.21.15.32:0/3872915731 >> v2:172.21.15.32:3300/0 conn(0x7fb64409d6f0 0x7fb64409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb649e26700 1 --2- 172.21.15.32:0/3872915731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 0x7fb644130d70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00d150 tx=0x7fb63c00d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54d2b1a655fcc52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb63bfff700 1 -- 172.21.15.32:0/3872915731 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c010070 con 0x7fb64409e410 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb63bfff700 1 -- 172.21.15.32:0/3872915731 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c00ddc0 con 0x7fb64409e410 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.448+0000 7fb63bfff700 1 -- 172.21.15.32:0/3872915731 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00b310 con 0x7fb64409e410 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 -- 172.21.15.32:0/3872915731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 msgr2=0x7fb644130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 --2- 172.21.15.32:0/3872915731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 0x7fb644130d70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00d150 tx=0x7fb63c00d460 comp rx=0 tx=0).stop 2023-09-30T22:27:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 -- 172.21.15.32:0/3872915731 shutdown_connections 2023-09-30T22:27:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 --2- 172.21.15.32:0/3872915731 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409e410 0x7fb644130d70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 --2- 172.21.15.32:0/3872915731 >> v2:172.21.15.32:3300/0 conn(0x7fb64409d6f0 0x7fb64409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 -- 172.21.15.32:0/3872915731 >> 172.21.15.32:0/3872915731 conn(0x7fb644066f50 msgr2=0x7fb64409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 -- 172.21.15.32:0/3872915731 shutdown_connections 2023-09-30T22:27:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.449+0000 7fb64b889700 1 -- 172.21.15.32:0/3872915731 wait complete. 2023-09-30T22:27:58.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.450+0000 7fb64b889700 1 Processor -- start 2023-09-30T22:27:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.450+0000 7fb64b889700 1 -- start start 2023-09-30T22:27:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb64b889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 0x7fb644122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb64b889700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb64409e000 0x7fb644122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb64b889700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb64409e410 0x7fb644122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb64b889700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb644125a70 con 0x7fb64409e000 2023-09-30T22:27:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb64b889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb644123f50 con 0x7fb64409d6f0 2023-09-30T22:27:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb64b889700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb64409c700 con 0x7fb64409e410 2023-09-30T22:27:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb649625700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 0x7fb644122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb649625700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 0x7fb644122230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45160/0 (socket says 172.21.15.32:45160) 2023-09-30T22:27:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb649625700 1 -- 172.21.15.32:0/3356806521 learned_addr learned my addr 172.21.15.32:0/3356806521 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.451+0000 7fb649e26700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb64409e410 0x7fb644122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb649625700 1 -- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb64409e410 msgr2=0x7fb644122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb649625700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb64409e410 0x7fb644122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb649625700 1 -- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb64409e000 msgr2=0x7fb644122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb649625700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb64409e000 0x7fb644122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb649625700 1 -- 172.21.15.32:0/3356806521 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c00bd40 con 0x7fb64409d6f0 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb649625700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 0x7fb644122230 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb64000e810 tx=0x7fb64000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640004230 con 0x7fb64409d6f0 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6400043b0 con 0x7fb64409d6f0 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6400038e0 con 0x7fb64409d6f0 2023-09-30T22:27:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644072500 con 0x7fb64409d6f0 2023-09-30T22:27:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.452+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb644125d50 con 0x7fb64409d6f0 2023-09-30T22:27:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.453+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb640003a40 con 0x7fb64409d6f0 2023-09-30T22:27:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.453+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb64003f180 con 0x7fb64409d6f0 2023-09-30T22:27:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.453+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644125bb0 con 0x7fb64409d6f0 2023-09-30T22:27:58.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.458+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb644125bb0 con 0x7fb64409d6f0 2023-09-30T22:27:58.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.562+0000 7fa851481700 1 Processor -- start 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.562+0000 7fa851481700 1 -- start start 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa851481700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 0x7fa84c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa851481700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa84c135e50 0x7fa84c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa851481700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c09f130 con 0x7fa84c09d6f0 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa851481700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c09f520 con 0x7fa84c135e50 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa851481700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c09e930 con 0x7fa84c09e1d0 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa84a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 0x7fa84c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa84b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa84c135e50 0x7fa84c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa84a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 0x7fa84c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45164/0 (socket says 172.21.15.32:45164) 2023-09-30T22:27:58.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.563+0000 7fa84a7fc700 1 -- 172.21.15.32:0/3512261050 learned_addr learned my addr 172.21.15.32:0/3512261050 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa84b7fe700 1 -- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 msgr2=0x7fa84c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa84b7fe700 1 --2- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 0x7fa84c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa84b7fe700 1 -- 172.21.15.32:0/3512261050 >> v1:172.21.15.32:6789/0 conn(0x7fa84c09d6f0 legacy=0x7fa84c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa84b7fe700 1 -- 172.21.15.32:0/3512261050 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c09b2d0 con 0x7fa84c135e50 2023-09-30T22:27:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa84a7fc700 1 --2- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 0x7fa84c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:58.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa84b7fe700 1 --2- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c135e50 0x7fa84c138230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa844009dc0 tx=0x7fa84400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe3b5513d9c8704 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa849ffb700 1 -- 172.21.15.32:0/3512261050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400bce0 con 0x7fa84c135e50 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa849ffb700 1 -- 172.21.15.32:0/3512261050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84400be40 con 0x7fa84c135e50 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.564+0000 7fa849ffb700 1 -- 172.21.15.32:0/3512261050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440116a0 con 0x7fa84c135e50 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 -- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c135e50 msgr2=0x7fa84c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 --2- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c135e50 0x7fa84c138230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa844009dc0 tx=0x7fa84400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 -- 172.21.15.32:0/3512261050 shutdown_connections 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 --2- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c135e50 0x7fa84c138230 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 --2- 172.21.15.32:0/3512261050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09e1d0 0x7fa84c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 -- 172.21.15.32:0/3512261050 >> 172.21.15.32:0/3512261050 conn(0x7fa84c066f50 msgr2=0x7fa84c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 -- 172.21.15.32:0/3512261050 shutdown_connections 2023-09-30T22:27:58.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.565+0000 7fa851481700 1 -- 172.21.15.32:0/3512261050 wait complete. 2023-09-30T22:27:58.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 Processor -- start 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 -- start start 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09d6f0 0x7fa84c1c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c09e1d0 0x7fa84c1c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 0x7fa84c1c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c11b140 con 0x7fa84c09e1d0 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c11c8b0 con 0x7fa84c135e50 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.567+0000 7fa851481700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c12ff70 con 0x7fa84c09d6f0 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 0x7fa84c1c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 0x7fa84c1c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45168/0 (socket says 172.21.15.32:45168) 2023-09-30T22:27:58.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 -- 172.21.15.32:0/539129732 learned_addr learned my addr 172.21.15.32:0/539129732 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84affd700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09d6f0 0x7fa84c1c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84a7fc700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c09e1d0 0x7fa84c1c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 -- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09d6f0 msgr2=0x7fa84c1c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09d6f0 0x7fa84c1c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 -- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c09e1d0 msgr2=0x7fa84c1c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c09e1d0 0x7fa84c1c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 -- 172.21.15.32:0/539129732 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840009550 con 0x7fa84c135e50 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84a7fc700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c09e1d0 0x7fa84c1c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:58.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.568+0000 7fa84b7fe700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 0x7fa84c1c4ab0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa844013000 tx=0x7fa844004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.569+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440048b0 con 0x7fa84c135e50 2023-09-30T22:27:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.569+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844011460 con 0x7fa84c135e50 2023-09-30T22:27:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.569+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844011810 con 0x7fa84c135e50 2023-09-30T22:27:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.569+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa844009a70 con 0x7fa84c135e50 2023-09-30T22:27:58.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.569+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c09c700 con 0x7fa84c135e50 2023-09-30T22:27:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.570+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c11c9f0 con 0x7fa84c135e50 2023-09-30T22:27:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.570+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa844011970 con 0x7fa84c135e50 2023-09-30T22:27:58.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.571+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa8440418e0 con 0x7fa84c135e50 2023-09-30T22:27:58.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.579+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa844019770 con 0x7fa84c135e50 2023-09-30T22:27:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.605+0000 7f8afa148700 1 Processor -- start 2023-09-30T22:27:58.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.605+0000 7f8afa148700 1 -- start start 2023-09-30T22:27:58.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8afa148700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8af409e930 0x7f8af409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8afa148700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 0x7f8af4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8afa148700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af4070940 con 0x7f8af409f240 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8afa148700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af406fd60 con 0x7f8af409e930 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8afa148700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8af406fa80 con 0x7f8af409f650 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 0x7f8af4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 0x7f8af4131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43618/0 (socket says 172.21.15.32:43618) 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af3fff700 1 -- 172.21.15.32:0/3095154611 learned_addr learned my addr 172.21.15.32:0/3095154611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af37fe700 1 --2- 172.21.15.32:0/3095154611 >> v2:172.21.15.32:3300/0 conn(0x7f8af409e930 0x7f8af409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af3fff700 1 -- 172.21.15.32:0/3095154611 >> v2:172.21.15.32:3300/0 conn(0x7f8af409e930 msgr2=0x7f8af409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af3fff700 1 --2- 172.21.15.32:0/3095154611 >> v2:172.21.15.32:3300/0 conn(0x7f8af409e930 0x7f8af409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.606+0000 7f8af3fff700 1 -- 172.21.15.32:0/3095154611 >> v1:172.21.15.32:6789/0 conn(0x7f8af409f240 legacy=0x7f8af41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:58.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.607+0000 7f8af3fff700 1 -- 172.21.15.32:0/3095154611 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af409fb60 con 0x7f8af409f650 2023-09-30T22:27:58.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.607+0000 7f8af3fff700 1 --2- 172.21.15.32:0/3095154611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 0x7f8af4131f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009e50 tx=0x7f8aec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520d257771fa55ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.607+0000 7f8af27fc700 1 -- 172.21.15.32:0/3095154611 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00e070 con 0x7f8af409f650 2023-09-30T22:27:58.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.607+0000 7f8af27fc700 1 -- 172.21.15.32:0/3095154611 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec00b910 con 0x7f8af409f650 2023-09-30T22:27:58.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.607+0000 7f8af27fc700 1 -- 172.21.15.32:0/3095154611 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00bcc0 con 0x7f8af409f650 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 -- 172.21.15.32:0/3095154611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 msgr2=0x7f8af4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 --2- 172.21.15.32:0/3095154611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 0x7f8af4131f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009e50 tx=0x7f8aec00b330 comp rx=0 tx=0).stop 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 -- 172.21.15.32:0/3095154611 shutdown_connections 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 --2- 172.21.15.32:0/3095154611 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409f650 0x7f8af4131f00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 --2- 172.21.15.32:0/3095154611 >> v2:172.21.15.32:3300/0 conn(0x7f8af409e930 0x7f8af409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 -- 172.21.15.32:0/3095154611 >> 172.21.15.32:0/3095154611 conn(0x7f8af40a4210 msgr2=0x7f8af40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 -- 172.21.15.32:0/3095154611 shutdown_connections 2023-09-30T22:27:58.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.608+0000 7f8afa148700 1 -- 172.21.15.32:0/3095154611 wait complete. 2023-09-30T22:27:58.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.610+0000 7f8afa148700 1 Processor -- start 2023-09-30T22:27:58.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.610+0000 7f8afa148700 1 -- start start 2023-09-30T22:27:58.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8afa148700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409e930 0x7f8af41a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8afa148700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409f240 0x7f8af41a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8afa148700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 0x7f8af41a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8afa148700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41a7630 con 0x7f8af409f650 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8afa148700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41a7810 con 0x7f8af409f240 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8afa148700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4071d20 con 0x7f8af409e930 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 0x7f8af41a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8af37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409e930 0x7f8af41a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 0x7f8af41a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58896/0 (socket says 172.21.15.32:58896) 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8af3fff700 1 -- 172.21.15.32:0/1190706247 learned_addr learned my addr 172.21.15.32:0/1190706247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.611+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409f240 0x7f8af41a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af3fff700 1 -- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409e930 msgr2=0x7f8af41a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af3fff700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409e930 0x7f8af41a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af3fff700 1 -- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409f240 msgr2=0x7f8af41a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af3fff700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409f240 0x7f8af41a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af3fff700 1 -- 172.21.15.32:0/1190706247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec009b30 con 0x7f8af409f650 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af2ffd700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409f240 0x7f8af41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8af3fff700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 0x7f8af41a4830 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8af406fbc0 tx=0x7f8aec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0048b0 con 0x7f8af409f650 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec0092e0 con 0x7f8af409f650 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af40a0190 con 0x7f8af409f650 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec01c660 con 0x7f8af409f650 2023-09-30T22:27:58.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.612+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41a6690 con 0x7f8af409f650 2023-09-30T22:27:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.613+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af41a0b30 con 0x7f8af409f650 2023-09-30T22:27:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.614+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8aec01c7c0 con 0x7f8af409f650 2023-09-30T22:27:58.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.614+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8aec0413e0 con 0x7f8af409f650 2023-09-30T22:27:58.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.623+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8aec025450 con 0x7f8af409f650 2023-09-30T22:27:58.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.636+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29600747f0 con 0x7f29600a0a30 2023-09-30T22:27:58.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.636+0000 7f295cff9700 1 -- 172.21.15.32:0/2167655330 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29480192c0 con 0x7f29600a0a30 2023-09-30T22:27:58.636 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:27:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.639+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29600a0a30 msgr2=0x7f2960133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.639+0000 7f296588c700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29600a0a30 0x7f2960133b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f294800c430 tx=0x7f294800c740 comp rx=0 tx=0).stop 2023-09-30T22:27:58.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 shutdown_connections 2023-09-30T22:27:58.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2960138d80 0x7f29601340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f296009d630 0x7f2960133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 --2- 172.21.15.32:0/2167655330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29600a0a30 0x7f2960133b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 >> 172.21.15.32:0/2167655330 conn(0x7f29600a3050 msgr2=0x7f296009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 shutdown_connections 2023-09-30T22:27:58.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.640+0000 7f296588c700 1 -- 172.21.15.32:0/2167655330 wait complete. 2023-09-30T22:27:58.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:27:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.681+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb644125bb0 con 0x7fb64409d6f0 2023-09-30T22:27:58.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.682+0000 7fb639ffb700 1 -- 172.21.15.32:0/3356806521 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb644125bb0 con 0x7fb64409d6f0 2023-09-30T22:27:58.682 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:27:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 msgr2=0x7fb644122230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 0x7fb644122230 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb64000e810 tx=0x7fb64000eb20 comp rx=0 tx=0).stop 2023-09-30T22:27:58.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 shutdown_connections 2023-09-30T22:27:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb64409e410 0x7fb644122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb64409d6f0 0x7fb644122230 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 --2- 172.21.15.32:0/3356806521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb64409e000 0x7fb644122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 >> 172.21.15.32:0/3356806521 conn(0x7fb644066f50 msgr2=0x7fb644070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 shutdown_connections 2023-09-30T22:27:58.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.684+0000 7fb64b889700 1 -- 172.21.15.32:0/3356806521 wait complete. 2023-09-30T22:27:58.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:27:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.790+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa84c074ea0 con 0x7fa84c135e50 2023-09-30T22:27:58.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.791+0000 7fa8337fe700 1 -- 172.21.15.32:0/539129732 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa844034090 con 0x7fa84c135e50 2023-09-30T22:27:58.791 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:27:58.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 msgr2=0x7fa84c1c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 0x7fa84c1c4ab0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa844013000 tx=0x7fa844004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 shutdown_connections 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c135e50 0x7fa84c1c4ab0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09d6f0 0x7fa84c1c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 --2- 172.21.15.32:0/539129732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c09e1d0 0x7fa84c1c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 >> 172.21.15.32:0/539129732 conn(0x7fa84c066f50 msgr2=0x7fa84c09a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 shutdown_connections 2023-09-30T22:27:58.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.793+0000 7fa851481700 1 -- 172.21.15.32:0/539129732 wait complete. 2023-09-30T22:27:58.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:27:58.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.830+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8af409ffc0 con 0x7f8af409f650 2023-09-30T22:27:58.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.830+0000 7f8adbfff700 1 -- 172.21.15.32:0/1190706247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8aec02db40 con 0x7f8af409f650 2023-09-30T22:27:58.831 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:27:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 msgr2=0x7f8af41a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 0x7f8af41a4830 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8af406fbc0 tx=0x7f8aec004ad0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 shutdown_connections 2023-09-30T22:27:58.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8af409f650 0x7f8af41a4830 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8af409e930 0x7f8af41a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 --2- 172.21.15.32:0/1190706247 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8af409f240 0x7f8af41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 >> 172.21.15.32:0/1190706247 conn(0x7f8af40a4210 msgr2=0x7f8af41b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 shutdown_connections 2023-09-30T22:27:58.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:58.832+0000 7f8afa148700 1 -- 172.21.15.32:0/1190706247 wait complete. 2023-09-30T22:27:58.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:27:59.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:59.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.805+0000 7f9e3775c700 1 Processor -- start 2023-09-30T22:27:59.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.805+0000 7f9e3775c700 1 -- start start 2023-09-30T22:27:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e3775c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e3775c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e30139250 0x7f9e3013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e3775c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3009edd0 con 0x7f9e300a0a30 2023-09-30T22:27:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e3775c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300a18e0 con 0x7f9e30139250 2023-09-30T22:27:59.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e3775c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3009e1f0 con 0x7f9e300a1510 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e35cf9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e30139250 0x7f9e3013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e354f8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e300a0a30 0x7f9e300a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60268/0 (socket says 172.21.15.32:60268) 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e354f8700 1 -- 172.21.15.32:0/1598534825 learned_addr learned my addr 172.21.15.32:0/1598534825 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e34cf7700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e2ffff700 1 -- 172.21.15.32:0/1598534825 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153799365 0 0) 0x7f9e3009edd0 con 0x7f9e300a0a30 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e2ffff700 1 -- 172.21.15.32:0/1598534825 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e180035b0 con 0x7f9e300a0a30 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e35cf9700 1 -- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 msgr2=0x7f9e30138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e35cf9700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e35cf9700 1 -- 172.21.15.32:0/1598534825 >> v1:172.21.15.32:6789/0 conn(0x7f9e300a0a30 legacy=0x7f9e300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e35cf9700 1 -- 172.21.15.32:0/1598534825 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3009dc50 con 0x7f9e30139250 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.806+0000 7f9e34cf7700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.807+0000 7f9e35cf9700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e30139250 0x7f9e3013b630 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009dc0 tx=0x7f9e2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f1e6ae4cf8b286 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.807+0000 7f9e2ffff700 1 -- 172.21.15.32:0/1598534825 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000e070 con 0x7f9e30139250 2023-09-30T22:27:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.807+0000 7f9e2ffff700 1 -- 172.21.15.32:0/1598534825 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000bb20 con 0x7f9e30139250 2023-09-30T22:27:59.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.807+0000 7f9e2ffff700 1 -- 172.21.15.32:0/1598534825 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20012430 con 0x7f9e30139250 2023-09-30T22:27:59.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 -- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e30139250 msgr2=0x7f9e3013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e30139250 0x7f9e3013b630 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009dc0 tx=0x7f9e2000b330 comp rx=0 tx=0).stop 2023-09-30T22:27:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 -- 172.21.15.32:0/1598534825 shutdown_connections 2023-09-30T22:27:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e30139250 0x7f9e3013b630 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1598534825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 -- 172.21.15.32:0/1598534825 >> 172.21.15.32:0/1598534825 conn(0x7f9e300a6550 msgr2=0x7f9e300a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 -- 172.21.15.32:0/1598534825 shutdown_connections 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.808+0000 7f9e3775c700 1 -- 172.21.15.32:0/1598534825 wait complete. 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.809+0000 7f9e3775c700 1 Processor -- start 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.809+0000 7f9e3775c700 1 -- start start 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e3775c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e300a0a30 0x7f9e30122700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e3775c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30122c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e3775c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e30139250 0x7f9e30123180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e3775c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e301262c0 con 0x7f9e30139250 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e3775c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e301385f0 con 0x7f9e300a0a30 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e3775c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e30124b30 con 0x7f9e300a1510 2023-09-30T22:27:59.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e34cf7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30122c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e354f8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e300a0a30 0x7f9e30122700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e34cf7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30122c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43642/0 (socket says 172.21.15.32:43642) 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e34cf7700 1 -- 172.21.15.32:0/1564443426 learned_addr learned my addr 172.21.15.32:0/1564443426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.810+0000 7f9e35cf9700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e30139250 0x7f9e30123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e34cf7700 1 -- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e300a0a30 msgr2=0x7f9e30122700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e34cf7700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e300a0a30 0x7f9e30122700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e34cf7700 1 -- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e30139250 msgr2=0x7f9e30123180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e34cf7700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e30139250 0x7f9e30123180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e34cf7700 1 -- 172.21.15.32:0/1564443426 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240094a0 con 0x7f9e300a1510 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e35cf9700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e30139250 0x7f9e30123180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e354f8700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e300a0a30 0x7f9e30122700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e34cf7700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30122c40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400ec10 tx=0x7f9e2400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400c850 con 0x7f9e300a1510 2023-09-30T22:27:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2400c9b0 con 0x7f9e300a1510 2023-09-30T22:27:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e20009a70 con 0x7f9e300a1510 2023-09-30T22:27:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400cd20 con 0x7f9e300a1510 2023-09-30T22:27:59.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.811+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e30138a90 con 0x7f9e300a1510 2023-09-30T22:27:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.812+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e24013470 con 0x7f9e300a1510 2023-09-30T22:27:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.812+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9e2403fb30 con 0x7f9e300a1510 2023-09-30T22:27:59.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.813+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e30126060 con 0x7f9e300a1510 2023-09-30T22:27:59.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.817+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e24021080 con 0x7f9e300a1510 2023-09-30T22:27:59.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 Processor -- start 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 -- start start 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09d6f0 0x7f0b9c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b9c09e410 0x7f0b9c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c09b120 con 0x7f0b9c09e000 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c06b460 con 0x7f0b9c09e410 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0ba14c0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c09a7e0 con 0x7f0b9c09d6f0 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0b9affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09d6f0 0x7f0b9c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0b9a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b9c09e000 0x7f0b9c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60282/0 (socket says 172.21.15.32:60282) 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0b9a7fc700 1 -- 172.21.15.32:0/2449996963 learned_addr learned my addr 172.21.15.32:0/2449996963 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.847+0000 7f0b9b7fe700 1 --2- 172.21.15.32:0/2449996963 >> v2:172.21.15.32:3300/0 conn(0x7f0b9c09e410 0x7f0b9c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09d6f0 msgr2=0x7f0b9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b9b7fe700 1 --2- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09d6f0 0x7f0b9c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/2449996963 >> v1:172.21.15.32:6789/0 conn(0x7f0b9c09e000 legacy=0x7f0b9c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b99ffb700 1 -- 172.21.15.32:0/2449996963 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436622080 0 0) 0x7f0b9c09b120 con 0x7f0b9c09e000 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/2449996963 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c09a500 con 0x7f0b9c09e410 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b9b7fe700 1 --2- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09e410 0x7f0b9c134100 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009d90 tx=0x7f0b9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc3666e2de73b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b99ffb700 1 -- 172.21.15.32:0/2449996963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400e070 con 0x7f0b9c09e410 2023-09-30T22:27:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b99ffb700 1 -- 172.21.15.32:0/2449996963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9400bb20 con 0x7f0b9c09e410 2023-09-30T22:27:59.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.848+0000 7f0b99ffb700 1 -- 172.21.15.32:0/2449996963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94012430 con 0x7f0b9c09e410 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 -- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09e410 msgr2=0x7f0b9c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09e410 0x7f0b9c134100 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009d90 tx=0x7f0b9400b330 comp rx=0 tx=0).stop 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 -- 172.21.15.32:0/2449996963 shutdown_connections 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09e410 0x7f0b9c134100 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/2449996963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09d6f0 0x7f0b9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 -- 172.21.15.32:0/2449996963 >> 172.21.15.32:0/2449996963 conn(0x7f0b9c066f50 msgr2=0x7f0b9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 -- 172.21.15.32:0/2449996963 shutdown_connections 2023-09-30T22:27:59.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.849+0000 7f0ba14c0700 1 -- 172.21.15.32:0/2449996963 wait complete. 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.850+0000 7f0ba14c0700 1 Processor -- start 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.850+0000 7f0ba14c0700 1 -- start start 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0ba14c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09d6f0 0x7f0b9c1a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0ba14c0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b9c09e000 0x7f0b9c1a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0ba14c0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 0x7f0b9c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0ba14c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a8120 con 0x7f0b9c09d6f0 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0ba14c0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a8300 con 0x7f0b9c09e000 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0ba14c0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1b9170 con 0x7f0b9c09e410 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 0x7f0b9c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 0x7f0b9c1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43662/0 (socket says 172.21.15.32:43662) 2023-09-30T22:27:59.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/3349269892 learned_addr learned my addr 172.21.15.32:0/3349269892 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0b9affd700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09d6f0 0x7f0b9c1a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.851+0000 7f0b9a7fc700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b9c09e000 0x7f0b9c1a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b9c09e000 msgr2=0x7f0b9c1a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9b7fe700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b9c09e000 0x7f0b9c1a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09d6f0 msgr2=0x7f0b9c1a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9b7fe700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09d6f0 0x7f0b9c1a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9b7fe700 1 -- 172.21.15.32:0/3349269892 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94009a70 con 0x7f0b9c09e410 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9a7fc700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b9c09e000 0x7f0b9c1a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9affd700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09d6f0 0x7f0b9c1a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b9b7fe700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 0x7f0b9c1c8040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400b680 tx=0x7f0b94004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94012430 con 0x7f0b9c09e410 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94020070 con 0x7f0b9c09e410 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c1a4ca0 con 0x7f0b9c09e410 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9401ba30 con 0x7f0b9c09e410 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.852+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c09c710 con 0x7f0b9c09e410 2023-09-30T22:27:59.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.853+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b9401bb90 con 0x7f0b9c09e410 2023-09-30T22:27:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.853+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b94022070 con 0x7f0b9c09e410 2023-09-30T22:27:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.854+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c074ea0 con 0x7f0b9c09e410 2023-09-30T22:27:59.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.859+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b94026c60 con 0x7f0b9c09e410 2023-09-30T22:27:59.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 Processor -- start 2023-09-30T22:27:59.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 -- start start 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa89009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa890130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89009f130 con 0x7fa89009e000 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89009f520 con 0x7fa89009e410 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.955+0000 7fa894a5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa89009e930 con 0x7fa89009d6f0 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa89009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa89009e000 0x7fa890130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60288/0 (socket says 172.21.15.32:60288) 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88dd9b700 1 -- 172.21.15.32:0/455892650 learned_addr learned my addr 172.21.15.32:0/455892650 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:27:59.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa890130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88e59c700 1 -- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 msgr2=0x7fa890130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88e59c700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa890130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88e59c700 1 -- 172.21.15.32:0/455892650 >> v1:172.21.15.32:6789/0 conn(0x7fa89009e000 legacy=0x7fa890130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88e59c700 1 -- 172.21.15.32:0/455892650 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89009b2d0 con 0x7fa89009d6f0 2023-09-30T22:27:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88d59a700 1 -- 172.21.15.32:0/455892650 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734751778 0 0) 0x7fa89009f130 con 0x7fa89009e000 2023-09-30T22:27:59.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.956+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa890130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.957+0000 7fa88e59c700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa89009dac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa880009870 tx=0x7fa880009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2f720e0d17e673 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.957+0000 7fa88d59a700 1 -- 172.21.15.32:0/455892650 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880011830 con 0x7fa89009d6f0 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.957+0000 7fa88d59a700 1 -- 172.21.15.32:0/455892650 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880011990 con 0x7fa89009d6f0 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.957+0000 7fa88d59a700 1 -- 172.21.15.32:0/455892650 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880011d20 con 0x7fa89009d6f0 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 -- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 msgr2=0x7fa89009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa89009dac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa880009870 tx=0x7fa880009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 -- 172.21.15.32:0/455892650 shutdown_connections 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa890130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 --2- 172.21.15.32:0/455892650 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa89009dac0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 -- 172.21.15.32:0/455892650 >> 172.21.15.32:0/455892650 conn(0x7fa890066f50 msgr2=0x7fa89009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 -- 172.21.15.32:0/455892650 shutdown_connections 2023-09-30T22:27:59.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.958+0000 7fa894a5c700 1 -- 172.21.15.32:0/455892650 wait complete. 2023-09-30T22:27:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.959+0000 7fa894a5c700 1 Processor -- start 2023-09-30T22:27:59.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.959+0000 7fa894a5c700 1 -- start start 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa894a5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa890127d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa894a5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009e000 0x7fa8901282a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa894a5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa8901cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa894a5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901292c0 con 0x7fa89009e000 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa894a5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901290d0 con 0x7fa89009e410 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa894a5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa890072150 con 0x7fa89009d6f0 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa88ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa8901cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa88dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009e000 0x7fa8901282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa88ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa8901cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45234/0 (socket says 172.21.15.32:45234) 2023-09-30T22:27:59.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa88ed9d700 1 -- 172.21.15.32:0/4105067259 learned_addr learned my addr 172.21.15.32:0/4105067259 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.960+0000 7fa88e59c700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa890127d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88dd9b700 1 -- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 msgr2=0x7fa890127d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88dd9b700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa890127d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88dd9b700 1 -- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 msgr2=0x7fa8901cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88dd9b700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa8901cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88dd9b700 1 -- 172.21.15.32:0/4105067259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888009a70 con 0x7fa89009e000 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa8901cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa88dd9b700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009e000 0x7fa8901282a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa8900a3e20 tx=0x7fa88400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884004a70 con 0x7fa89009e000 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884004bd0 con 0x7fa89009e000 2023-09-30T22:27:59.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8800094a0 con 0x7fa89009e000 2023-09-30T22:27:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.961+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840070d0 con 0x7fa89009e000 2023-09-30T22:27:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.962+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa884007230 con 0x7fa89009e000 2023-09-30T22:27:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.962+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8901298c0 con 0x7fa89009e000 2023-09-30T22:27:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.963+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa8840118a0 con 0x7fa89009e000 2023-09-30T22:27:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.963+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89006b800 con 0x7fa89009e000 2023-09-30T22:27:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.968+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa884044060 con 0x7fa89009e000 2023-09-30T22:27:59.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.991+0000 7f1942051700 1 Processor -- start 2023-09-30T22:27:59.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.992+0000 7f1942051700 1 -- start start 2023-09-30T22:27:59.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.992+0000 7f1942051700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 0x7f193c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.992+0000 7f1942051700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09f650 0x7f193c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.992+0000 7f1942051700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c070940 con 0x7f193c09f240 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.992+0000 7f1942051700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c06fd60 con 0x7f193c09e930 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.992+0000 7f1942051700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c06fa80 con 0x7f193c09f650 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 0x7f193c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09f650 0x7f193c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09f650 0x7f193c131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43694/0 (socket says 172.21.15.32:43694) 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193bfff700 1 -- 172.21.15.32:0/3753110388 learned_addr learned my addr 172.21.15.32:0/3753110388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 0x7f193c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45246/0 (socket says 172.21.15.32:45246) 2023-09-30T22:27:59.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f193c09f240 0x7f193c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60298/0 (socket says 172.21.15.32:60298) 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 -- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09f650 msgr2=0x7f193c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 --2- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09f650 0x7f193c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 -- 172.21.15.32:0/3753110388 >> v1:172.21.15.32:6789/0 conn(0x7f193c09f240 legacy=0x7f193c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193a7fc700 1 -- 172.21.15.32:0/3753110388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539319974 0 0) 0x7f193c070940 con 0x7f193c09f240 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 -- 172.21.15.32:0/3753110388 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c09fb60 con 0x7f193c09e930 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.993+0000 7f193b7fe700 1 --2- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 0x7f193c09ed00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f192c009870 tx=0x7f192c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2c30620188e359 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.994+0000 7f193a7fc700 1 -- 172.21.15.32:0/3753110388 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c011830 con 0x7f193c09e930 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.994+0000 7f193a7fc700 1 -- 172.21.15.32:0/3753110388 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c011990 con 0x7f193c09e930 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.994+0000 7f193a7fc700 1 -- 172.21.15.32:0/3753110388 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c011d20 con 0x7f193c09e930 2023-09-30T22:27:59.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 -- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 msgr2=0x7f193c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 --2- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 0x7f193c09ed00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f192c009870 tx=0x7f192c009b80 comp rx=0 tx=0).stop 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 -- 172.21.15.32:0/3753110388 shutdown_connections 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 --2- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09f650 0x7f193c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 --2- 172.21.15.32:0/3753110388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09e930 0x7f193c09ed00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 -- 172.21.15.32:0/3753110388 >> 172.21.15.32:0/3753110388 conn(0x7f193c0a4210 msgr2=0x7f193c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 -- 172.21.15.32:0/3753110388 shutdown_connections 2023-09-30T22:27:59.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.995+0000 7f1942051700 1 -- 172.21.15.32:0/3753110388 wait complete. 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.996+0000 7f1942051700 1 Processor -- start 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.996+0000 7f1942051700 1 -- start start 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f1942051700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e930 0x7f193c1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f1942051700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09f240 0x7f193c1a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f1942051700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 0x7f193c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f1942051700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a80a0 con 0x7f193c09f650 2023-09-30T22:27:59.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f1942051700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a57d0 con 0x7f193c09f240 2023-09-30T22:27:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f1942051700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a6a20 con 0x7f193c09e930 2023-09-30T22:27:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 0x7f193c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 0x7f193c1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58924/0 (socket says 172.21.15.32:58924) 2023-09-30T22:27:59.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193bfff700 1 -- 172.21.15.32:0/2981359890 learned_addr learned my addr 172.21.15.32:0/2981359890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193b7fe700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e930 0x7f193c1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193affd700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09f240 0x7f193c1a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193b7fe700 1 -- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09f240 msgr2=0x7f193c1a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193b7fe700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09f240 0x7f193c1a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193b7fe700 1 -- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 msgr2=0x7f193c1a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193b7fe700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 0x7f193c1a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193b7fe700 1 -- 172.21.15.32:0/2981359890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c0094a0 con 0x7f193c09e930 2023-09-30T22:27:59.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.997+0000 7f193bfff700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 0x7f193c1a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.998+0000 7f193b7fe700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e930 0x7f193c1a3bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f193c0a1bf0 tx=0x7f192c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.998+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c004450 con 0x7f193c09e930 2023-09-30T22:28:00.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.998+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c01f070 con 0x7f193c09e930 2023-09-30T22:28:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.998+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c0a0190 con 0x7f193c09e930 2023-09-30T22:28:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.998+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0038a0 con 0x7f193c09e930 2023-09-30T22:28:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.998+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c0a3460 con 0x7f193c09e930 2023-09-30T22:28:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.999+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f192c003a00 con 0x7f193c09e930 2023-09-30T22:28:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.999+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c071d80 con 0x7f193c09e930 2023-09-30T22:28:00.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:27:59.999+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f192c021070 con 0x7f193c09e930 2023-09-30T22:28:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.004+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f192c0045b0 con 0x7f193c09e930 2023-09-30T22:28:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.038+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e30074ea0 con 0x7f9e300a1510 2023-09-30T22:28:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.039+0000 7f9e2dffb700 1 -- 172.21.15.32:0/1564443426 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e24013720 con 0x7f9e300a1510 2023-09-30T22:28:00.039 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 msgr2=0x7f9e30122c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30122c40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400ec10 tx=0x7f9e2400ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 shutdown_connections 2023-09-30T22:28:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e30139250 0x7f9e30123180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e300a0a30 0x7f9e30122700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 --2- 172.21.15.32:0/1564443426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9e300a1510 0x7f9e30122c40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 >> 172.21.15.32:0/1564443426 conn(0x7f9e300a6550 msgr2=0x7f9e3011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 shutdown_connections 2023-09-30T22:28:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.042+0000 7f9e3775c700 1 -- 172.21.15.32:0/1564443426 wait complete. 2023-09-30T22:28:00.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:00.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.085+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0b9c1a63f0 con 0x7f0b9c09e410 2023-09-30T22:28:00.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.086+0000 7f0b837fe700 1 -- 172.21.15.32:0/3349269892 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b9402c020 con 0x7f0b9c09e410 2023-09-30T22:28:00.087 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:00.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 msgr2=0x7f0b9c1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:00.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 0x7f0b9c1c8040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400b680 tx=0x7f0b94004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 shutdown_connections 2023-09-30T22:28:00.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b9c09e410 0x7f0b9c1c8040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b9c09d6f0 0x7f0b9c1a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 --2- 172.21.15.32:0/3349269892 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b9c09e000 0x7f0b9c1a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.089+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 >> 172.21.15.32:0/3349269892 conn(0x7f0b9c066f50 msgr2=0x7f0b9c06f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:00.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.090+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 shutdown_connections 2023-09-30T22:28:00.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.090+0000 7f0ba14c0700 1 -- 172.21.15.32:0/3349269892 wait complete. 2023-09-30T22:28:00.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.179+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa89012a9a0 con 0x7fa89009e000 2023-09-30T22:28:00.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.179+0000 7fa87effd700 1 -- 172.21.15.32:0/4105067259 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa89012a9a0 con 0x7fa89009e000 2023-09-30T22:28:00.179 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:00.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009e000 msgr2=0x7fa8901282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009e000 0x7fa8901282a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa8900a3e20 tx=0x7fa88400c750 comp rx=0 tx=0).stop 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 shutdown_connections 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009e410 0x7fa8901cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89009d6f0 0x7fa890127d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 --2- 172.21.15.32:0/4105067259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009e000 0x7fa8901282a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.181+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 >> 172.21.15.32:0/4105067259 conn(0x7fa890066f50 msgr2=0x7fa890070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.182+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 shutdown_connections 2023-09-30T22:28:00.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.182+0000 7fa894a5c700 1 -- 172.21.15.32:0/4105067259 wait complete. 2023-09-30T22:28:00.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.221+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f193c074ea0 con 0x7f193c09e930 2023-09-30T22:28:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.221+0000 7f1923fff700 1 -- 172.21.15.32:0/2981359890 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f192c015030 con 0x7f193c09e930 2023-09-30T22:28:00.222 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e930 msgr2=0x7f193c1a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e930 0x7f193c1a3bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f193c0a1bf0 tx=0x7f192c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 shutdown_connections 2023-09-30T22:28:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c09f650 0x7f193c1a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c09e930 0x7f193c1a3bc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 --2- 172.21.15.32:0/2981359890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c09f240 0x7f193c1a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 >> 172.21.15.32:0/2981359890 conn(0x7f193c0a4210 msgr2=0x7f193c1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 shutdown_connections 2023-09-30T22:28:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:00.224+0000 7f1942051700 1 -- 172.21.15.32:0/2981359890 wait complete. 2023-09-30T22:28:00.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:01.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:01.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:01.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.214+0000 7f93cee47700 1 Processor -- start 2023-09-30T22:28:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93cee47700 1 -- start start 2023-09-30T22:28:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93cee47700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93c809d6f0 0x7f93c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93cee47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93cee47700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c809f130 con 0x7f93c809e410 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93cee47700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c809f520 con 0x7f93c809d6f0 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93cee47700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93c809e930 con 0x7f93c809e000 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55574/0 (socket says 172.21.15.32:55574) 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93c7fff700 1 -- 172.21.15.32:0/1097649929 learned_addr learned my addr 172.21.15.32:0/1097649929 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.215+0000 7f93ccbe3700 1 --2- 172.21.15.32:0/1097649929 >> v2:172.21.15.32:3300/0 conn(0x7f93c809d6f0 0x7f93c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c7fff700 1 -- 172.21.15.32:0/1097649929 >> v2:172.21.15.32:3300/0 conn(0x7f93c809d6f0 msgr2=0x7f93c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c7fff700 1 --2- 172.21.15.32:0/1097649929 >> v2:172.21.15.32:3300/0 conn(0x7f93c809d6f0 0x7f93c809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c7fff700 1 -- 172.21.15.32:0/1097649929 >> v1:172.21.15.32:6789/0 conn(0x7f93c809e410 legacy=0x7f93c8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c7fff700 1 -- 172.21.15.32:0/1097649929 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c809b2d0 con 0x7f93c809e000 2023-09-30T22:28:01.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93ccbe3700 1 --2- 172.21.15.32:0/1097649929 >> v2:172.21.15.32:3300/0 conn(0x7f93c809d6f0 0x7f93c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c7fff700 1 --2- 172.21.15.32:0/1097649929 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c8130660 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f93b40098f0 tx=0x7f93b4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf27eebb50d3ca1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c77fe700 1 -- 172.21.15.32:0/1097649929 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4011830 con 0x7f93c809e000 2023-09-30T22:28:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c77fe700 1 -- 172.21.15.32:0/1097649929 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4011990 con 0x7f93c809e000 2023-09-30T22:28:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.216+0000 7f93c77fe700 1 -- 172.21.15.32:0/1097649929 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4011d20 con 0x7f93c809e000 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 -- 172.21.15.32:0/1097649929 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 msgr2=0x7f93c8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 --2- 172.21.15.32:0/1097649929 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c8130660 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f93b40098f0 tx=0x7f93b4009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 -- 172.21.15.32:0/1097649929 shutdown_connections 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 --2- 172.21.15.32:0/1097649929 >> v2:172.21.15.32:3300/0 conn(0x7f93c809d6f0 0x7f93c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 --2- 172.21.15.32:0/1097649929 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c8130660 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 -- 172.21.15.32:0/1097649929 >> 172.21.15.32:0/1097649929 conn(0x7f93c8066f50 msgr2=0x7f93c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 -- 172.21.15.32:0/1097649929 shutdown_connections 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.217+0000 7f93cee47700 1 -- 172.21.15.32:0/1097649929 wait complete. 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.218+0000 7f93cee47700 1 Processor -- start 2023-09-30T22:28:01.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 -- start start 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93c809d6f0 0x7f93c806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93c809e410 0x7f93c806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8125b40 con 0x7f93c809d6f0 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8070390 con 0x7f93c809e410 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.219+0000 7f93cee47700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8125d90 con 0x7f93c809e000 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c806ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55586/0 (socket says 172.21.15.32:55586) 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 -- 172.21.15.32:0/1606725151 learned_addr learned my addr 172.21.15.32:0/1606725151 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93cd3e4700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93c809e410 0x7f93c806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93ccbe3700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93c809d6f0 0x7f93c806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 -- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93c809e410 msgr2=0x7f93c806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93c809e410 0x7f93c806f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 -- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93c809d6f0 msgr2=0x7f93c806e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93c809d6f0 0x7f93c806e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 -- 172.21.15.32:0/1606725151 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc0094a0 con 0x7f93c809e000 2023-09-30T22:28:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c7fff700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c806ed80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000c00 tx=0x7f93b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.220+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40036a0 con 0x7f93c809e000 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.221+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4003800 con 0x7f93c809e000 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.221+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b4009550 con 0x7f93c809e000 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.221+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4003bb0 con 0x7f93c809e000 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.221+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c8124950 con 0x7f93c809e000 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.222+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93b4021070 con 0x7f93c809e000 2023-09-30T22:28:01.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.222+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f93b4040bb0 con 0x7f93c809e000 2023-09-30T22:28:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.222+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c8124c00 con 0x7f93c809e000 2023-09-30T22:28:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.229+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93b4015020 con 0x7f93c809e000 2023-09-30T22:28:01.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.255+0000 7fc09431c700 1 Processor -- start 2023-09-30T22:28:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.256+0000 7fc09431c700 1 -- start start 2023-09-30T22:28:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.256+0000 7fc09431c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 0x7fc08c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.256+0000 7fc09431c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c136f50 0x7fc08c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.256+0000 7fc09431c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c09b4f0 con 0x7fc08c131810 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.256+0000 7fc09431c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c1313c0 con 0x7fc08c134630 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.256+0000 7fc09431c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c06e8c0 con 0x7fc08c136f50 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0918b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 0x7fc08c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0920b8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc08c131810 0x7fc08c131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44372/0 (socket says 172.21.15.32:44372) 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0920b8700 1 -- 172.21.15.32:0/3581723971 learned_addr learned my addr 172.21.15.32:0/3581723971 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0918b7700 1 --2- 172.21.15.32:0/3581723971 >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 0x7fc08c136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42046/0 (socket says 172.21.15.32:42046) 2023-09-30T22:28:01.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0928b9700 1 --2- 172.21.15.32:0/3581723971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c136f50 0x7fc08c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0928b9700 1 -- 172.21.15.32:0/3581723971 >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 msgr2=0x7fc08c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0928b9700 1 --2- 172.21.15.32:0/3581723971 >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 0x7fc08c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0928b9700 1 -- 172.21.15.32:0/3581723971 >> v1:172.21.15.32:6789/0 conn(0x7fc08c131810 legacy=0x7fc08c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0928b9700 1 -- 172.21.15.32:0/3581723971 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c071d20 con 0x7fc08c136f50 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0918b7700 1 --2- 172.21.15.32:0/3581723971 >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 0x7fc08c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0928b9700 1 --2- 172.21.15.32:0/3581723971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c136f50 0x7fc08c139330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc084009d90 tx=0x7fc08400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ca0a2a579c127c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.257+0000 7fc0910b6700 1 -- 172.21.15.32:0/3581723971 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08400e070 con 0x7fc08c136f50 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc0910b6700 1 -- 172.21.15.32:0/3581723971 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08400bb20 con 0x7fc08c136f50 2023-09-30T22:28:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc0910b6700 1 -- 172.21.15.32:0/3581723971 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084012430 con 0x7fc08c136f50 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 -- 172.21.15.32:0/3581723971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c136f50 msgr2=0x7fc08c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 --2- 172.21.15.32:0/3581723971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c136f50 0x7fc08c139330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc084009d90 tx=0x7fc08400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 -- 172.21.15.32:0/3581723971 shutdown_connections 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 --2- 172.21.15.32:0/3581723971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c136f50 0x7fc08c139330 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 --2- 172.21.15.32:0/3581723971 >> v2:172.21.15.32:3300/0 conn(0x7fc08c134630 0x7fc08c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 -- 172.21.15.32:0/3581723971 >> 172.21.15.32:0/3581723971 conn(0x7fc08c0a2aa0 msgr2=0x7fc08c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.258+0000 7fc09431c700 1 -- 172.21.15.32:0/3581723971 shutdown_connections 2023-09-30T22:28:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.259+0000 7fc09431c700 1 -- 172.21.15.32:0/3581723971 wait complete. 2023-09-30T22:28:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.260+0000 7fc09431c700 1 Processor -- start 2023-09-30T22:28:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.260+0000 7fc09431c700 1 -- start start 2023-09-30T22:28:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc09431c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c131810 0x7fc08c119b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc09431c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 0x7fc08c11a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc09431c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c136f50 0x7fc08c1c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc09431c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c11c670 con 0x7fc08c134630 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc09431c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c136cb0 con 0x7fc08c136f50 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc09431c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c06eb00 con 0x7fc08c131810 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 0x7fc08c11a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 0x7fc08c11a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42056/0 (socket says 172.21.15.32:42056) 2023-09-30T22:28:01.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 -- 172.21.15.32:0/4167141785 learned_addr learned my addr 172.21.15.32:0/4167141785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0928b9700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c136f50 0x7fc08c1c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 -- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c131810 msgr2=0x7fc08c119b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c131810 0x7fc08c119b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 -- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c136f50 msgr2=0x7fc08c1c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c136f50 0x7fc08c1c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.261+0000 7fc0918b7700 1 -- 172.21.15.32:0/4167141785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0094a0 con 0x7fc08c134630 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.262+0000 7fc0918b7700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 0x7fc08c11a090 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00ebe0 tx=0x7fc07c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.262+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00c850 con 0x7fc08c134630 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.262+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c00c9b0 con 0x7fc08c134630 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.262+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc084009a70 con 0x7fc08c134630 2023-09-30T22:28:01.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.262+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c1195d0 con 0x7fc08c134630 2023-09-30T22:28:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.263+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cd60 con 0x7fc08c134630 2023-09-30T22:28:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.263+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0740052e0 con 0x7fc08c134630 2023-09-30T22:28:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.264+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc07c004020 con 0x7fc08c134630 2023-09-30T22:28:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.265+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc07c03fc10 con 0x7fc08c134630 2023-09-30T22:28:01.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.269+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc07c01abc0 con 0x7fc08c134630 2023-09-30T22:28:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.352+0000 7f474b832700 1 Processor -- start 2023-09-30T22:28:01.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.353+0000 7f474b832700 1 -- start start 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.353+0000 7f474b832700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f474409d630 0x7f47440a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.353+0000 7f474b832700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4744138d80 0x7f474413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.353+0000 7f474b832700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47440a2a70 con 0x7f47440a0a30 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.353+0000 7f474b832700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47440a01d0 con 0x7f4744138d80 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.353+0000 7f474b832700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f474409e2b0 con 0x7f474409d630 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f47495ce700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f47440a0a30 0x7f47440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44378/0 (socket says 172.21.15.32:44378) 2023-09-30T22:28:01.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f47495ce700 1 -- 172.21.15.32:0/2974752107 learned_addr learned my addr 172.21.15.32:0/2974752107 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f4748dcd700 1 --2- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f474409d630 0x7f47440a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f473bfff700 1 -- 172.21.15.32:0/2974752107 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610751912 0 0) 0x7f47440a2a70 con 0x7f47440a0a30 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f473bfff700 1 -- 172.21.15.32:0/2974752107 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47300035b0 con 0x7f47440a0a30 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f4748dcd700 1 -- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4744138d80 msgr2=0x7f474413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f4748dcd700 1 --2- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4744138d80 0x7f474413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f4748dcd700 1 -- 172.21.15.32:0/2974752107 >> v1:172.21.15.32:6789/0 conn(0x7f47440a0a30 legacy=0x7f47440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f4748dcd700 1 -- 172.21.15.32:0/2974752107 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474409dc50 con 0x7f474409d630 2023-09-30T22:28:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.354+0000 7f4748dcd700 1 --2- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f474409d630 0x7f47440a1510 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4734009870 tx=0x7f4734009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b145556fbb21f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f473bfff700 1 -- 172.21.15.32:0/2974752107 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734011830 con 0x7f474409d630 2023-09-30T22:28:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f473bfff700 1 -- 172.21.15.32:0/2974752107 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4734011990 con 0x7f474409d630 2023-09-30T22:28:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f473bfff700 1 -- 172.21.15.32:0/2974752107 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734011d00 con 0x7f474409d630 2023-09-30T22:28:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f474b832700 1 -- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f474409d630 msgr2=0x7f47440a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f474b832700 1 --2- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f474409d630 0x7f47440a1510 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4734009870 tx=0x7f4734009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f474b832700 1 -- 172.21.15.32:0/2974752107 shutdown_connections 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f474b832700 1 --2- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4744138d80 0x7f474413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f474b832700 1 --2- 172.21.15.32:0/2974752107 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f474409d630 0x7f47440a1510 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.355+0000 7f474b832700 1 -- 172.21.15.32:0/2974752107 >> 172.21.15.32:0/2974752107 conn(0x7f47440a3050 msgr2=0x7f47440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.356+0000 7f474b832700 1 -- 172.21.15.32:0/2974752107 shutdown_connections 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.356+0000 7f474b832700 1 -- 172.21.15.32:0/2974752107 wait complete. 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 Processor -- start 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 -- start start 2023-09-30T22:28:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f474409d630 0x7f47441338c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 0x7f4744133e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4744138d80 0x7f4744134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744136700 con 0x7f474409d630 2023-09-30T22:28:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47441359b0 con 0x7f47440a0a30 2023-09-30T22:28:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.357+0000 7f474b832700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4744137070 con 0x7f4744138d80 2023-09-30T22:28:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 0x7f4744133e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 0x7f4744133e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58658/0 (socket says 172.21.15.32:58658) 2023-09-30T22:28:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 -- 172.21.15.32:0/1783897576 learned_addr learned my addr 172.21.15.32:0/1783897576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f47495ce700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f474409d630 0x7f47441338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4749dcf700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4744138d80 0x7f4744134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 -- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4744138d80 msgr2=0x7f4744134340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4744138d80 0x7f4744134340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 -- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f474409d630 msgr2=0x7f47441338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f474409d630 0x7f47441338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 -- 172.21.15.32:0/1783897576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47340094a0 con 0x7f47440a0a30 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4749dcf700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4744138d80 0x7f4744134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f47495ce700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f474409d630 0x7f47441338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.358+0000 7f4748dcd700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 0x7f4744133e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f47340090b0 tx=0x7f4734004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.359+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340045d0 con 0x7f47440a0a30 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.359+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473401f070 con 0x7f47440a0a30 2023-09-30T22:28:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.359+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4744135c60 con 0x7f47440a0a30 2023-09-30T22:28:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.359+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340038c0 con 0x7f47440a0a30 2023-09-30T22:28:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.359+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47440a0410 con 0x7f47440a0a30 2023-09-30T22:28:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.360+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4734003a20 con 0x7f47440a0a30 2023-09-30T22:28:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.360+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4734021070 con 0x7f47440a0a30 2023-09-30T22:28:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.360+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47440747f0 con 0x7f47440a0a30 2023-09-30T22:28:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.365+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4734015070 con 0x7f47440a0a30 2023-09-30T22:28:01.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.385+0000 7f9c2ecf4700 1 Processor -- start 2023-09-30T22:28:01.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.385+0000 7f9c2ecf4700 1 -- start start 2023-09-30T22:28:01.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2ecf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c2809d630 0x7f9c280a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2ecf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 0x7f9c2813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2ecf4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c280a2a70 con 0x7f9c280a0a30 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2ecf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280a01d0 con 0x7f9c28138d80 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2ecf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2809e2b0 con 0x7f9c2809d630 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 0x7f9c2813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 0x7f9c2813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58672/0 (socket says 172.21.15.32:58672) 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 -- 172.21.15.32:0/3838648779 learned_addr learned my addr 172.21.15.32:0/3838648779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 -- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c2809d630 msgr2=0x7f9c280a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 --2- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c2809d630 0x7f9c280a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 -- 172.21.15.32:0/3838648779 >> v1:172.21.15.32:6789/0 conn(0x7f9c280a0a30 legacy=0x7f9c280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 -- 172.21.15.32:0/3838648779 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2809dc50 con 0x7f9c28138d80 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.386+0000 7f9c2d291700 1 --2- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 0x7f9c2813b160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009e80 tx=0x7f9c2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ed36edefb115b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c277fe700 1 -- 172.21.15.32:0/3838648779 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000e070 con 0x7f9c28138d80 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c277fe700 1 -- 172.21.15.32:0/3838648779 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c20003a20 con 0x7f9c28138d80 2023-09-30T22:28:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c277fe700 1 -- 172.21.15.32:0/3838648779 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20003dd0 con 0x7f9c28138d80 2023-09-30T22:28:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 msgr2=0x7f9c2813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 0x7f9c2813b160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009e80 tx=0x7f9c2000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/3838648779 shutdown_connections 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c28138d80 0x7f9c2813b160 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/3838648779 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c2809d630 0x7f9c280a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/3838648779 >> 172.21.15.32:0/3838648779 conn(0x7f9c280a3050 msgr2=0x7f9c280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/3838648779 shutdown_connections 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.387+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/3838648779 wait complete. 2023-09-30T22:28:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.388+0000 7f9c2ecf4700 1 Processor -- start 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.388+0000 7f9c2ecf4700 1 -- start start 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ecf4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c2809d630 0x7f9c281337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ecf4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c280a0a30 0x7f9c28133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ecf4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 0x7f9c28134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ecf4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28134c80 con 0x7f9c280a0a30 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ecf4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281365e0 con 0x7f9c2809d630 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ecf4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28136f50 con 0x7f9c28138d80 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2d291700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 0x7f9c28134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2d291700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 0x7f9c28134230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55636/0 (socket says 172.21.15.32:55636) 2023-09-30T22:28:01.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2d291700 1 -- 172.21.15.32:0/1404574226 learned_addr learned my addr 172.21.15.32:0/1404574226 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c27fff700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c280a0a30 0x7f9c28133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.389+0000 7f9c2ca90700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c2809d630 0x7f9c281337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2d291700 1 -- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c2809d630 msgr2=0x7f9c281337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2d291700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c2809d630 0x7f9c281337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2d291700 1 -- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c280a0a30 msgr2=0x7f9c28133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2d291700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c280a0a30 0x7f9c28133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2d291700 1 -- 172.21.15.32:0/1404574226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20009b30 con 0x7f9c28138d80 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2ca90700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c2809d630 0x7f9c281337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2d291700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 0x7f9c28134230 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9c20015040 tx=0x7f9c20004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004440 con 0x7f9c28138d80 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c200045e0 con 0x7f9c28138d80 2023-09-30T22:28:01.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c280a4c70 con 0x7f9c28138d80 2023-09-30T22:28:01.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.390+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c281379c0 con 0x7f9c28138d80 2023-09-30T22:28:01.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.391+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20012430 con 0x7f9c28138d80 2023-09-30T22:28:01.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.391+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c20019070 con 0x7f9c28138d80 2023-09-30T22:28:01.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.391+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9c20040960 con 0x7f9c28138d80 2023-09-30T22:28:01.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.392+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c280747f0 con 0x7f9c28138d80 2023-09-30T22:28:01.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.397+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c20019320 con 0x7f9c28138d80 2023-09-30T22:28:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.454+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93c809a500 con 0x7f93c809e000 2023-09-30T22:28:01.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.455+0000 7f93c57fa700 1 -- 172.21.15.32:0/1606725151 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93b400b3e0 con 0x7f93c809e000 2023-09-30T22:28:01.455 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.457+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 msgr2=0x7f93c806ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.457+0000 7f93cee47700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c806ed80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f93b4000c00 tx=0x7f93b4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 shutdown_connections 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93c809e410 0x7f93c806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93c809d6f0 0x7f93c806e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 --2- 172.21.15.32:0/1606725151 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93c809e000 0x7f93c806ed80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 >> 172.21.15.32:0/1606725151 conn(0x7f93c8066f50 msgr2=0x7f93c806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 shutdown_connections 2023-09-30T22:28:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.458+0000 7f93cee47700 1 -- 172.21.15.32:0/1606725151 wait complete. 2023-09-30T22:28:01.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.526+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc074005860 con 0x7fc08c134630 2023-09-30T22:28:01.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.526+0000 7fc082ffd700 1 -- 172.21.15.32:0/4167141785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc07c017020 con 0x7fc08c134630 2023-09-30T22:28:01.526 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:01.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 msgr2=0x7fc08c11a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 0x7fc08c11a090 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00ebe0 tx=0x7fc07c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 shutdown_connections 2023-09-30T22:28:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc08c136f50 0x7fc08c1c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc08c131810 0x7fc08c119b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 --2- 172.21.15.32:0/4167141785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc08c134630 0x7fc08c11a090 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 >> 172.21.15.32:0/4167141785 conn(0x7fc08c0a2aa0 msgr2=0x7fc08c12f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 shutdown_connections 2023-09-30T22:28:01.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.528+0000 7fc09431c700 1 -- 172.21.15.32:0/4167141785 wait complete. 2023-09-30T22:28:01.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:01.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.606+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c28072980 con 0x7f9c28138d80 2023-09-30T22:28:01.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.606+0000 7f9c257fa700 1 -- 172.21.15.32:0/1404574226 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c20017020 con 0x7f9c28138d80 2023-09-30T22:28:01.606 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 msgr2=0x7f9c28134230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 0x7f9c28134230 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9c20015040 tx=0x7f9c20004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 shutdown_connections 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9c28138d80 0x7f9c28134230 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9c2809d630 0x7f9c281337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 --2- 172.21.15.32:0/1404574226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c280a0a30 0x7f9c28133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 >> 172.21.15.32:0/1404574226 conn(0x7f9c280a3050 msgr2=0x7f9c2809f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.608+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 shutdown_connections 2023-09-30T22:28:01.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.609+0000 7f9c2ecf4700 1 -- 172.21.15.32:0/1404574226 wait complete. 2023-09-30T22:28:01.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.616+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4744072980 con 0x7f47440a0a30 2023-09-30T22:28:01.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.617+0000 7f4739ffb700 1 -- 172.21.15.32:0/1783897576 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4734010380 con 0x7f47440a0a30 2023-09-30T22:28:01.617 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:01.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 msgr2=0x7f4744133e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 0x7f4744133e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f47340090b0 tx=0x7f4734004750 comp rx=0 tx=0).stop 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 shutdown_connections 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4744138d80 0x7f4744134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f474409d630 0x7f47441338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 --2- 172.21.15.32:0/1783897576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47440a0a30 0x7f4744133e00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 >> 172.21.15.32:0/1783897576 conn(0x7f47440a3050 msgr2=0x7f47440a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 shutdown_connections 2023-09-30T22:28:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:01.621+0000 7f474b832700 1 -- 172.21.15.32:0/1783897576 wait complete. 2023-09-30T22:28:01.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:01.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:02.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:02.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.621+0000 7f6c37e7b700 1 Processor -- start 2023-09-30T22:28:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.621+0000 7f6c37e7b700 1 -- start start 2023-09-30T22:28:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c37e7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c300a0a30 0x7f6c300a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c37e7b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 0x7f6c300a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c37e7b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c30133930 0x7f6c30133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c37e7b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300a2a70 con 0x7f6c30133930 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c37e7b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300a01d0 con 0x7f6c3009d630 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c37e7b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3009e2b0 con 0x7f6c300a0a30 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c35416700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 0x7f6c300a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c36418700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c30133930 0x7f6c30133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c35c17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c300a0a30 0x7f6c300a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c35416700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 0x7f6c300a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58694/0 (socket says 172.21.15.32:58694) 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c35416700 1 -- 172.21.15.32:0/1913114485 learned_addr learned my addr 172.21.15.32:0/1913114485 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c36418700 1 -- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c300a0a30 msgr2=0x7f6c300a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c36418700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c300a0a30 0x7f6c300a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c36418700 1 -- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 msgr2=0x7f6c300a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c36418700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 0x7f6c300a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c36418700 1 -- 172.21.15.32:0/1913114485 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3009dc50 con 0x7f6c30133930 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c35c17700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c300a0a30 0x7f6c300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.622+0000 7f6c35416700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 0x7f6c300a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.623+0000 7f6c36418700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c30133930 0x7f6c30133dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009840 tx=0x7f6c28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432c454692406561 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.623+0000 7f6c34c15700 1 -- 172.21.15.32:0/1913114485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28011830 con 0x7f6c30133930 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.623+0000 7f6c34c15700 1 -- 172.21.15.32:0/1913114485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28011990 con 0x7f6c30133930 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.623+0000 7f6c34c15700 1 -- 172.21.15.32:0/1913114485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28011d20 con 0x7f6c30133930 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 -- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c30133930 msgr2=0x7f6c30133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c30133930 0x7f6c30133dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009840 tx=0x7f6c28009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 -- 172.21.15.32:0/1913114485 shutdown_connections 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c30133930 0x7f6c30133dd0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c300a0a30 0x7f6c300a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/1913114485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3009d630 0x7f6c300a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 -- 172.21.15.32:0/1913114485 >> 172.21.15.32:0/1913114485 conn(0x7f6c300a3050 msgr2=0x7f6c300a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:02.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 -- 172.21.15.32:0/1913114485 shutdown_connections 2023-09-30T22:28:02.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.624+0000 7f6c37e7b700 1 -- 172.21.15.32:0/1913114485 wait complete. 2023-09-30T22:28:02.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 Processor -- start 2023-09-30T22:28:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 -- start start 2023-09-30T22:28:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3009d630 0x7f6c3011e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c300a0a30 0x7f6c3011ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 0x7f6c3011f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30133460 con 0x7f6c3009d630 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300a50b0 con 0x7f6c300a0a30 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c37e7b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c301205e0 con 0x7f6c30133930 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c36418700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 0x7f6c3011f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c35c17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3009d630 0x7f6c3011e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c36418700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 0x7f6c3011f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55664/0 (socket says 172.21.15.32:55664) 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c36418700 1 -- 172.21.15.32:0/3318233344 learned_addr learned my addr 172.21.15.32:0/3318233344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.625+0000 7f6c35416700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c300a0a30 0x7f6c3011ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c36418700 1 -- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c300a0a30 msgr2=0x7f6c3011ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c36418700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c300a0a30 0x7f6c3011ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c36418700 1 -- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3009d630 msgr2=0x7f6c3011e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c36418700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3009d630 0x7f6c3011e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c36418700 1 -- 172.21.15.32:0/3318233344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24009550 con 0x7f6c30133930 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c35c17700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3009d630 0x7f6c3011e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c36418700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 0x7f6c3011f360 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009fd0 tx=0x7f6c28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28004450 con 0x7f6c30133930 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c280045b0 con 0x7f6c30133930 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c20009a70 con 0x7f6c30133930 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280038e0 con 0x7f6c30133930 2023-09-30T22:28:02.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.626+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280094a0 con 0x7f6c30133930 2023-09-30T22:28:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.627+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c28003a40 con 0x7f6c30133930 2023-09-30T22:28:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.627+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6c28040830 con 0x7f6c30133930 2023-09-30T22:28:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.627+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c300747f0 con 0x7f6c30133930 2023-09-30T22:28:02.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.632+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c28015020 con 0x7f6c30133930 2023-09-30T22:28:02.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.690+0000 7f81bc176700 1 Processor -- start 2023-09-30T22:28:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81bc176700 1 -- start start 2023-09-30T22:28:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81bc176700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81bc176700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 0x7f81b4138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81bc176700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b40716a0 con 0x7f81b4130e50 2023-09-30T22:28:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81bc176700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b409a390 con 0x7f81b4130540 2023-09-30T22:28:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81bc176700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b4071290 con 0x7f81b406fcf0 2023-09-30T22:28:02.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.691+0000 7f81ba713700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 0x7f81b4138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 0x7f81b4138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55674/0 (socket says 172.21.15.32:55674) 2023-09-30T22:28:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81b9f12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 -- 172.21.15.32:0/2464755657 learned_addr learned my addr 172.21.15.32:0/2464755657 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81b9711700 1 --1- 172.21.15.32:0/2464755657 >> v1:172.21.15.32:6789/0 conn(0x7f81b4130e50 0x7f81b406f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44388/0 (socket says 172.21.15.32:44388) 2023-09-30T22:28:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 -- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 msgr2=0x7f81b4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 --2- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 -- 172.21.15.32:0/2464755657 >> v1:172.21.15.32:6789/0 conn(0x7f81b4130e50 legacy=0x7f81b406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 -- 172.21.15.32:0/2464755657 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4130340 con 0x7f81b406fcf0 2023-09-30T22:28:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81ba713700 1 --2- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 0x7f81b4138570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009e80 tx=0x7f81ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4704d5a800e6186 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81b8f10700 1 -- 172.21.15.32:0/2464755657 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00e070 con 0x7f81b406fcf0 2023-09-30T22:28:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81b8f10700 1 -- 172.21.15.32:0/2464755657 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac00b910 con 0x7f81b406fcf0 2023-09-30T22:28:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.692+0000 7f81b8f10700 1 -- 172.21.15.32:0/2464755657 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00bcc0 con 0x7f81b406fcf0 2023-09-30T22:28:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 -- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 msgr2=0x7f81b4138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 --2- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 0x7f81b4138570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009e80 tx=0x7f81ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 -- 172.21.15.32:0/2464755657 shutdown_connections 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 --2- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b406fcf0 0x7f81b4138570 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 --2- 172.21.15.32:0/2464755657 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 -- 172.21.15.32:0/2464755657 >> 172.21.15.32:0/2464755657 conn(0x7f81b40a2aa0 msgr2=0x7f81b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 -- 172.21.15.32:0/2464755657 shutdown_connections 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.693+0000 7f81bc176700 1 -- 172.21.15.32:0/2464755657 wait complete. 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.694+0000 7f81bc176700 1 Processor -- start 2023-09-30T22:28:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 -- start start 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b406fcf0 0x7f81b4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b4130e50 0x7f81b4119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b41b9110 con 0x7f81b406fcf0 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b411bfd0 con 0x7f81b4130540 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81bc176700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b411c1d0 con 0x7f81b4130e50 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81b9f12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b406fcf0 0x7f81b4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81b9711700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81b9f12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b406fcf0 0x7f81b4118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42094/0 (socket says 172.21.15.32:42094) 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81b9711700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58716/0 (socket says 172.21.15.32:58716) 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81b9f12700 1 -- 172.21.15.32:0/107972370 learned_addr learned my addr 172.21.15.32:0/107972370 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.695+0000 7f81ba713700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b4130e50 0x7f81b4119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81b9711700 1 -- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b4130e50 msgr2=0x7f81b4119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81b9711700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b4130e50 0x7f81b4119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81b9711700 1 -- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b406fcf0 msgr2=0x7f81b4118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81b9711700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b406fcf0 0x7f81b4118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81b9711700 1 -- 172.21.15.32:0/107972370 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac009b30 con 0x7f81b4130540 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81b9711700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4119210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c420 tx=0x7f81a400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4005cb0 con 0x7f81b4130540 2023-09-30T22:28:02.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4005e10 con 0x7f81b4130540 2023-09-30T22:28:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b411b0c0 con 0x7f81b4130540 2023-09-30T22:28:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40072d0 con 0x7f81b4130540 2023-09-30T22:28:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.696+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b40718e0 con 0x7f81b4130540 2023-09-30T22:28:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.697+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b412f8d0 con 0x7f81b4130540 2023-09-30T22:28:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.698+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81a4007430 con 0x7f81b4130540 2023-09-30T22:28:02.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.698+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f81a403efc0 con 0x7f81b4130540 2023-09-30T22:28:02.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.703+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81a401b3b0 con 0x7f81b4130540 2023-09-30T22:28:02.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.782+0000 7fd66c3d6700 1 Processor -- start 2023-09-30T22:28:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.782+0000 7fd66c3d6700 1 -- start start 2023-09-30T22:28:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.783+0000 7fd66c3d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 0x7fd66409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.783+0000 7fd66c3d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409e410 0x7fd664130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.783+0000 7fd66c3d6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66409f130 con 0x7fd66409e000 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.783+0000 7fd66c3d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66409f520 con 0x7fd66409d6f0 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.783+0000 7fd66c3d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd66409e930 con 0x7fd66409e410 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 0x7fd66409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 0x7fd66409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58728/0 (socket says 172.21.15.32:58728) 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd669971700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd66409e000 0x7fd664130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44398/0 (socket says 172.21.15.32:44398) 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a973700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409e410 0x7fd664130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 -- 172.21.15.32:0/2353287310 learned_addr learned my addr 172.21.15.32:0/2353287310 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd669170700 1 -- 172.21.15.32:0/2353287310 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726434676 0 0) 0x7fd66409f130 con 0x7fd66409e000 2023-09-30T22:28:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 -- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409e410 msgr2=0x7fd664130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 --2- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409e410 0x7fd664130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 -- 172.21.15.32:0/2353287310 >> v1:172.21.15.32:6789/0 conn(0x7fd66409e000 legacy=0x7fd664130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 -- 172.21.15.32:0/2353287310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66409b2d0 con 0x7fd66409d6f0 2023-09-30T22:28:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.784+0000 7fd66a172700 1 --2- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 0x7fd66409dac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd660009950 tx=0x7fd660009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75eaf3c8d0ac144d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd669170700 1 -- 172.21.15.32:0/2353287310 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660011830 con 0x7fd66409d6f0 2023-09-30T22:28:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd669170700 1 -- 172.21.15.32:0/2353287310 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660011990 con 0x7fd66409d6f0 2023-09-30T22:28:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd669170700 1 -- 172.21.15.32:0/2353287310 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660011d40 con 0x7fd66409d6f0 2023-09-30T22:28:02.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 -- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 msgr2=0x7fd66409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 0x7fd66409dac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd660009950 tx=0x7fd660009c60 comp rx=0 tx=0).stop 2023-09-30T22:28:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 -- 172.21.15.32:0/2353287310 shutdown_connections 2023-09-30T22:28:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409e410 0x7fd664130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/2353287310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409d6f0 0x7fd66409dac0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 -- 172.21.15.32:0/2353287310 >> 172.21.15.32:0/2353287310 conn(0x7fd664066f50 msgr2=0x7fd66409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:02.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 -- 172.21.15.32:0/2353287310 shutdown_connections 2023-09-30T22:28:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.785+0000 7fd66c3d6700 1 -- 172.21.15.32:0/2353287310 wait complete. 2023-09-30T22:28:02.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.787+0000 7fd66c3d6700 1 Processor -- start 2023-09-30T22:28:02.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.787+0000 7fd66c3d6700 1 -- start start 2023-09-30T22:28:02.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66c3d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409d6f0 0x7fd6641c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66c3d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409e000 0x7fd6641c3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66c3d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 0x7fd664131610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66c3d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664133620 con 0x7fd66409e410 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66c3d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd664135130 con 0x7fd66409e000 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66c3d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641c2a00 con 0x7fd66409d6f0 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 0x7fd664131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a172700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409d6f0 0x7fd6641c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 0x7fd664131610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42098/0 (socket says 172.21.15.32:42098) 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 -- 172.21.15.32:0/1671546505 learned_addr learned my addr 172.21.15.32:0/1671546505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd669971700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409e000 0x7fd6641c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 -- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409d6f0 msgr2=0x7fd6641c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409d6f0 0x7fd6641c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 -- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409e000 msgr2=0x7fd6641c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.788+0000 7fd66a973700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409e000 0x7fd6641c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd66a973700 1 -- 172.21.15.32:0/1671546505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009550 con 0x7fd66409e410 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd669971700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409e000 0x7fd6641c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:02.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd66a973700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 0x7fd664131610 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00efc0 tx=0x7fd65c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c009ea0 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c00f040 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00ced0 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd664133a30 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.789+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6641344e0 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.790+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd65c014480 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.790+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd664074ea0 con 0x7fd66409e410 2023-09-30T22:28:02.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.791+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd65c040140 con 0x7fd66409e410 2023-09-30T22:28:02.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.796+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd65c007920 con 0x7fd66409e410 2023-09-30T22:28:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.819+0000 7f5e6c654700 1 Processor -- start 2023-09-30T22:28:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.819+0000 7f5e6c654700 1 -- start start 2023-09-30T22:28:02.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e6c654700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 0x7f5e64135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e6c654700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e64135e50 0x7f5e64138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e6c654700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6409f130 con 0x7f5e6409d6f0 2023-09-30T22:28:02.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e6c654700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6409f520 con 0x7f5e64135e50 2023-09-30T22:28:02.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e6c654700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6409e930 con 0x7f5e6409e1d0 2023-09-30T22:28:02.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e69bef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 0x7f5e64135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e69bef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 0x7f5e64135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58736/0 (socket says 172.21.15.32:58736) 2023-09-30T22:28:02.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.820+0000 7f5e69bef700 1 -- 172.21.15.32:0/2406011666 learned_addr learned my addr 172.21.15.32:0/2406011666 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e6abf1700 1 --2- 172.21.15.32:0/2406011666 >> v2:172.21.15.32:3300/0 conn(0x7f5e64135e50 0x7f5e64138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e69bef700 1 -- 172.21.15.32:0/2406011666 >> v2:172.21.15.32:3300/0 conn(0x7f5e64135e50 msgr2=0x7f5e64138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:02.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e69bef700 1 --2- 172.21.15.32:0/2406011666 >> v2:172.21.15.32:3300/0 conn(0x7f5e64135e50 0x7f5e64138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e69bef700 1 -- 172.21.15.32:0/2406011666 >> v1:172.21.15.32:6789/0 conn(0x7f5e6409d6f0 legacy=0x7f5e6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e69bef700 1 -- 172.21.15.32:0/2406011666 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6409b2d0 con 0x7f5e6409e1d0 2023-09-30T22:28:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e69bef700 1 --2- 172.21.15.32:0/2406011666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 0x7f5e64135910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009920 tx=0x7f5e54009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94c7c3d33b33422 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e693ee700 1 -- 172.21.15.32:0/2406011666 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54011830 con 0x7f5e6409e1d0 2023-09-30T22:28:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e693ee700 1 -- 172.21.15.32:0/2406011666 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54011990 con 0x7f5e6409e1d0 2023-09-30T22:28:02.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.821+0000 7f5e693ee700 1 -- 172.21.15.32:0/2406011666 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54011d20 con 0x7f5e6409e1d0 2023-09-30T22:28:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 -- 172.21.15.32:0/2406011666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 msgr2=0x7f5e64135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 --2- 172.21.15.32:0/2406011666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 0x7f5e64135910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009920 tx=0x7f5e54009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 -- 172.21.15.32:0/2406011666 shutdown_connections 2023-09-30T22:28:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 --2- 172.21.15.32:0/2406011666 >> v2:172.21.15.32:3300/0 conn(0x7f5e64135e50 0x7f5e64138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 --2- 172.21.15.32:0/2406011666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e6409e1d0 0x7f5e64135910 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 -- 172.21.15.32:0/2406011666 >> 172.21.15.32:0/2406011666 conn(0x7f5e64066f50 msgr2=0x7f5e6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 -- 172.21.15.32:0/2406011666 shutdown_connections 2023-09-30T22:28:02.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.822+0000 7f5e6c654700 1 -- 172.21.15.32:0/2406011666 wait complete. 2023-09-30T22:28:02.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.823+0000 7f5e6c654700 1 Processor -- start 2023-09-30T22:28:02.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.823+0000 7f5e6c654700 1 -- start start 2023-09-30T22:28:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6c654700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e6409d6f0 0x7f5e641c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6c654700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6409e1d0 0x7f5e641c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6c654700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e64135e50 0x7f5e641c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6c654700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64123e00 con 0x7f5e6409e1d0 2023-09-30T22:28:02.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6c654700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6409c700 con 0x7f5e64135e50 2023-09-30T22:28:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6c654700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6411d680 con 0x7f5e6409d6f0 2023-09-30T22:28:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6abf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e64135e50 0x7f5e641c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6a3f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e6409d6f0 0x7f5e641c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6abf1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e64135e50 0x7f5e641c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58744/0 (socket says 172.21.15.32:58744) 2023-09-30T22:28:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6abf1700 1 -- 172.21.15.32:0/1774085007 learned_addr learned my addr 172.21.15.32:0/1774085007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e69bef700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6409e1d0 0x7f5e641c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e69bef700 1 -- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e6409d6f0 msgr2=0x7f5e641c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e69bef700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e6409d6f0 0x7f5e641c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e69bef700 1 -- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e64135e50 msgr2=0x7f5e641c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e69bef700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e64135e50 0x7f5e641c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e69bef700 1 -- 172.21.15.32:0/1774085007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e54009550 con 0x7f5e6409e1d0 2023-09-30T22:28:02.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.824+0000 7f5e6a3f0700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e6409d6f0 0x7f5e641c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.825+0000 7f5e69bef700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6409e1d0 0x7f5e641c88c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5e54016000 tx=0x7f5e54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.825+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004510 con 0x7f5e6409e1d0 2023-09-30T22:28:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.825+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e540036a0 con 0x7f5e6409e1d0 2023-09-30T22:28:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.825+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6406a9e0 con 0x7f5e6409e1d0 2023-09-30T22:28:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.825+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54003a50 con 0x7f5e6409e1d0 2023-09-30T22:28:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.825+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6406b460 con 0x7f5e6409e1d0 2023-09-30T22:28:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.826+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e54003bb0 con 0x7f5e6409e1d0 2023-09-30T22:28:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.826+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5e54021030 con 0x7f5e6409e1d0 2023-09-30T22:28:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.826+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6406b460 con 0x7f5e6409e1d0 2023-09-30T22:28:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.831+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e6406b460 con 0x7f5e6409e1d0 2023-09-30T22:28:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.857+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c30120c50 con 0x7f6c30133930 2023-09-30T22:28:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.858+0000 7f6c2e7fc700 1 -- 172.21.15.32:0/3318233344 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c2801a420 con 0x7f6c30133930 2023-09-30T22:28:02.858 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:02.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 msgr2=0x7f6c3011f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 0x7f6c3011f360 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009fd0 tx=0x7f6c28004750 comp rx=0 tx=0).stop 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 shutdown_connections 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c30133930 0x7f6c3011f360 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3009d630 0x7f6c3011e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 --2- 172.21.15.32:0/3318233344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c300a0a30 0x7f6c3011ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 >> 172.21.15.32:0/3318233344 conn(0x7f6c300a3050 msgr2=0x7f6c300a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:02.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.860+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 shutdown_connections 2023-09-30T22:28:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.861+0000 7f6c37e7b700 1 -- 172.21.15.32:0/3318233344 wait complete. 2023-09-30T22:28:02.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:02.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.976+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f81b4074ea0 con 0x7f81b4130540 2023-09-30T22:28:02.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.977+0000 7f81aa7fc700 1 -- 172.21.15.32:0/107972370 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81a4017030 con 0x7f81b4130540 2023-09-30T22:28:02.977 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 msgr2=0x7f81b4119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4119210 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c420 tx=0x7f81a400c730 comp rx=0 tx=0).stop 2023-09-30T22:28:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 shutdown_connections 2023-09-30T22:28:02.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81b4130e50 0x7f81b4119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b406fcf0 0x7f81b4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 --2- 172.21.15.32:0/107972370 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81b4130540 0x7f81b4119210 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 >> 172.21.15.32:0/107972370 conn(0x7f81b40a2aa0 msgr2=0x7f81b412d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 shutdown_connections 2023-09-30T22:28:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:02.979+0000 7f81bc176700 1 -- 172.21.15.32:0/107972370 wait complete. 2023-09-30T22:28:02.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:03.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.041+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5e6406b460 con 0x7f5e6409e1d0 2023-09-30T22:28:03.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.041+0000 7f5e5affd700 1 -- 172.21.15.32:0/1774085007 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e6406b460 con 0x7f5e6409e1d0 2023-09-30T22:28:03.041 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:03.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6409e1d0 msgr2=0x7f5e641c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6409e1d0 0x7f5e641c88c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5e54016000 tx=0x7f5e54004750 comp rx=0 tx=0).stop 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 shutdown_connections 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e64135e50 0x7f5e641c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5e6409d6f0 0x7f5e641c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 --2- 172.21.15.32:0/1774085007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e6409e1d0 0x7f5e641c88c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.043+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 >> 172.21.15.32:0/1774085007 conn(0x7f5e64066f50 msgr2=0x7f5e640a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.044+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 shutdown_connections 2023-09-30T22:28:03.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.044+0000 7f5e6c654700 1 -- 172.21.15.32:0/1774085007 wait complete. 2023-09-30T22:28:03.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:03.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.057+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd664131260 con 0x7fd66409e410 2023-09-30T22:28:03.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.057+0000 7fd65affd700 1 -- 172.21.15.32:0/1671546505 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd65c007330 con 0x7fd66409e410 2023-09-30T22:28:03.058 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:03.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 msgr2=0x7fd664131610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:03.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 0x7fd664131610 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00efc0 tx=0x7fd65c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 shutdown_connections 2023-09-30T22:28:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd66409e410 0x7fd664131610 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd66409d6f0 0x7fd6641c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 --2- 172.21.15.32:0/1671546505 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd66409e000 0x7fd6641c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 >> 172.21.15.32:0/1671546505 conn(0x7fd664066f50 msgr2=0x7fd664070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 shutdown_connections 2023-09-30T22:28:03.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:03.059+0000 7fd66c3d6700 1 -- 172.21.15.32:0/1671546505 wait complete. 2023-09-30T22:28:03.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:03.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:03.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.020+0000 7f6fb4796700 1 Processor -- start 2023-09-30T22:28:04.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.020+0000 7f6fb4796700 1 -- start start 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb4796700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 0x7f6fac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb4796700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb4796700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6fac09e410 0x7f6fac133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb4796700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac06b460 con 0x7f6fac09e410 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb4796700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac09b120 con 0x7f6fac09d6f0 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb4796700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac09a7e0 con 0x7f6fac09e000 2023-09-30T22:28:04.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 0x7f6fac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 0x7f6fac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58750/0 (socket says 172.21.15.32:58750) 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 -- 172.21.15.32:0/813825759 learned_addr learned my addr 172.21.15.32:0/813825759 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2d33700 1 --2- 172.21.15.32:0/813825759 >> v2:172.21.15.32:3300/0 conn(0x7f6fac09e410 0x7f6fac133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb1d31700 1 --2- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 -- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 msgr2=0x7f6fac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 --2- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 -- 172.21.15.32:0/813825759 >> v2:172.21.15.32:3300/0 conn(0x7f6fac09e410 msgr2=0x7f6fac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 --2- 172.21.15.32:0/813825759 >> v2:172.21.15.32:3300/0 conn(0x7f6fac09e410 0x7f6fac133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2532700 1 -- 172.21.15.32:0/813825759 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac09a500 con 0x7f6fac09d6f0 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.021+0000 7f6fb2d33700 1 --2- 172.21.15.32:0/813825759 >> v2:172.21.15.32:3300/0 conn(0x7f6fac09e410 0x7f6fac133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:04.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.022+0000 7f6fb2532700 1 --2- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 0x7f6fac09dac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009920 tx=0x7f6fa8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f907d092ff60c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.022+0000 7f6fb1530700 1 -- 172.21.15.32:0/813825759 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011830 con 0x7f6fac09d6f0 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.022+0000 7f6fb1530700 1 -- 172.21.15.32:0/813825759 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa8011990 con 0x7f6fac09d6f0 2023-09-30T22:28:04.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.022+0000 7f6fb1530700 1 -- 172.21.15.32:0/813825759 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011d20 con 0x7f6fac09d6f0 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 -- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 msgr2=0x7f6fac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 --2- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 0x7f6fac09dac0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009920 tx=0x7f6fa8009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 -- 172.21.15.32:0/813825759 shutdown_connections 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 --2- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09d6f0 0x7f6fac09dac0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 --2- 172.21.15.32:0/813825759 >> v2:172.21.15.32:3300/0 conn(0x7f6fac09e410 0x7f6fac133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 --2- 172.21.15.32:0/813825759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 -- 172.21.15.32:0/813825759 >> 172.21.15.32:0/813825759 conn(0x7f6fac066f50 msgr2=0x7f6fac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 -- 172.21.15.32:0/813825759 shutdown_connections 2023-09-30T22:28:04.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.023+0000 7f6fb4796700 1 -- 172.21.15.32:0/813825759 wait complete. 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 Processor -- start 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 -- start start 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 0x7f6fac1a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09e410 0x7f6fac1a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1a7ac0 con 0x7f6fac09d6f0 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1a80e0 con 0x7f6fac09e410 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb4796700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1a8d10 con 0x7f6fac09e000 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.024+0000 7f6fb2532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 0x7f6fac1a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb1d31700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 0x7f6fac1a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42136/0 (socket says 172.21.15.32:42136) 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2d33700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09e410 0x7f6fac1a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 -- 172.21.15.32:0/2756851977 learned_addr learned my addr 172.21.15.32:0/2756851977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 -- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 msgr2=0x7f6fac1a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 -- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09e410 msgr2=0x7f6fac1a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09e410 0x7f6fac1a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 -- 172.21.15.32:0/2756851977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4009a70 con 0x7f6fac09d6f0 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2d33700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09e410 0x7f6fac1a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb2532700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 0x7f6fac1a5f40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009c70 tx=0x7f6fa8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004240 con 0x7f6fac09d6f0 2023-09-30T22:28:04.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa801f070 con 0x7f6fac09d6f0 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa8009550 con 0x7f6fac09d6f0 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80036a0 con 0x7f6fac09d6f0 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.025+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac0a2140 con 0x7f6fac09d6f0 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.026+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fa8021030 con 0x7f6fac09d6f0 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.026+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6fa80407a0 con 0x7f6fac09d6f0 2023-09-30T22:28:04.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.027+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac09c710 con 0x7f6fac09d6f0 2023-09-30T22:28:04.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.032+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fa8015080 con 0x7f6fac09d6f0 2023-09-30T22:28:04.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:04.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:04.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.142+0000 7f0199958700 1 Processor -- start 2023-09-30T22:28:04.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.142+0000 7f0199958700 1 -- start start 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0199958700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194134630 0x7f0194136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0199958700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0194136f50 0x7f0194139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0199958700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019409b4f0 con 0x7f0194131810 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0199958700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01941313c0 con 0x7f0194136f50 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0199958700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f019406e8c0 con 0x7f0194134630 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f01937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0194136f50 0x7f0194139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f01937fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0194136f50 0x7f0194139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42142/0 (socket says 172.21.15.32:42142) 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0192ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0194131810 0x7f0194131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44414/0 (socket says 172.21.15.32:44414) 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f01937fe700 1 -- 172.21.15.32:0/1585531769 learned_addr learned my addr 172.21.15.32:0/1585531769 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f01937fe700 1 -- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194134630 msgr2=0x7f0194136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f01937fe700 1 --2- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194134630 0x7f0194136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f01937fe700 1 -- 172.21.15.32:0/1585531769 >> v1:172.21.15.32:6789/0 conn(0x7f0194131810 legacy=0x7f0194131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.144+0000 7f01937fe700 1 -- 172.21.15.32:0/1585531769 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194071d20 con 0x7f0194136f50 2023-09-30T22:28:04.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.143+0000 7f0191ffb700 1 -- 172.21.15.32:0/1585531769 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698623433 0 0) 0x7f019409b4f0 con 0x7f0194131810 2023-09-30T22:28:04.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.144+0000 7f01937fe700 1 --2- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f0194139330 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f018c009e50 tx=0x7f018c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2381973f13db9cee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.144+0000 7f0191ffb700 1 -- 172.21.15.32:0/1585531769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00e070 con 0x7f0194136f50 2023-09-30T22:28:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.144+0000 7f0191ffb700 1 -- 172.21.15.32:0/1585531769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c00b9c0 con 0x7f0194136f50 2023-09-30T22:28:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.144+0000 7f0191ffb700 1 -- 172.21.15.32:0/1585531769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00bd70 con 0x7f0194136f50 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 -- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 msgr2=0x7f0194139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 --2- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f0194139330 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f018c009e50 tx=0x7f018c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 -- 172.21.15.32:0/1585531769 shutdown_connections 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 --2- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f0194139330 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 --2- 172.21.15.32:0/1585531769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194134630 0x7f0194136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 -- 172.21.15.32:0/1585531769 >> 172.21.15.32:0/1585531769 conn(0x7f01940a2aa0 msgr2=0x7f01940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 -- 172.21.15.32:0/1585531769 shutdown_connections 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.145+0000 7f0199958700 1 -- 172.21.15.32:0/1585531769 wait complete. 2023-09-30T22:28:04.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.146+0000 7f0199958700 1 Processor -- start 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.146+0000 7f0199958700 1 -- start start 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0199958700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194131810 0x7f019406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0199958700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0194134630 0x7f0194070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0199958700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f01941c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0199958700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01940a1e90 con 0x7f0194136f50 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0199958700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019411d2b0 con 0x7f0194134630 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0199958700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f019406faa0 con 0x7f0194131810 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f01937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f01941c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194131810 0x7f019406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f01937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f01941c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42144/0 (socket says 172.21.15.32:42144) 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f01937fe700 1 -- 172.21.15.32:0/3553001722 learned_addr learned my addr 172.21.15.32:0/3553001722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 -- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0194134630 msgr2=0x7f0194070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0194134630 0x7f0194070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 -- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 msgr2=0x7f01941c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f01941c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 -- 172.21.15.32:0/3553001722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c009b30 con 0x7f0194131810 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f01937fe700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f01941c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.147+0000 7f0192ffd700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194131810 0x7f019406fdd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f018400c3f0 tx=0x7f018400c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.148+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184004c80 con 0x7f0194131810 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.148+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0184004de0 con 0x7f0194131810 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.148+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184007310 con 0x7f0194131810 2023-09-30T22:28:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.148+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019411e640 con 0x7f0194131810 2023-09-30T22:28:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.148+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019411d680 con 0x7f0194131810 2023-09-30T22:28:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.149+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0194074ea0 con 0x7f0194131810 2023-09-30T22:28:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.150+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0184007470 con 0x7f0194131810 2023-09-30T22:28:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.150+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f018403f8b0 con 0x7f0194131810 2023-09-30T22:28:04.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.158+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01840133c0 con 0x7f0194131810 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.227+0000 7fb4bc44b700 1 Processor -- start 2023-09-30T22:28:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.227+0000 7fb4bc44b700 1 -- start start 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4bc44b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4bc44b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b4138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4bc44b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b406fa40 con 0x7fb4b412cf60 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4bc44b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b412cd90 con 0x7fb4b406f5d0 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4bc44b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4071780 con 0x7fb4b4135c20 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58782/0 (socket says 172.21.15.32:58782) 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 -- 172.21.15.32:0/831728441 learned_addr learned my addr 172.21.15.32:0/831728441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4ba9e8700 1 --2- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 -- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 msgr2=0x7fb4b4138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 --2- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b4138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 -- 172.21.15.32:0/831728441 >> v1:172.21.15.32:6789/0 conn(0x7fb4b412cf60 legacy=0x7fb4b412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.229+0000 7fb4b99e6700 1 -- 172.21.15.32:0/831728441 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4071a40 con 0x7fb4b406f5d0 2023-09-30T22:28:04.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.230+0000 7fb4b99e6700 1 --2- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b412ffc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400bbc0 tx=0x7fb4a400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2231a4764982927b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.230+0000 7fb4b91e5700 1 -- 172.21.15.32:0/831728441 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400dc80 con 0x7fb4b406f5d0 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.230+0000 7fb4b91e5700 1 -- 172.21.15.32:0/831728441 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a400dde0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.230+0000 7fb4b91e5700 1 -- 172.21.15.32:0/831728441 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4017630 con 0x7fb4b406f5d0 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 -- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 msgr2=0x7fb4b412ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b412ffc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400bbc0 tx=0x7fb4a400bed0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 -- 172.21.15.32:0/831728441 shutdown_connections 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b4138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/831728441 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b412ffc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 -- 172.21.15.32:0/831728441 >> 172.21.15.32:0/831728441 conn(0x7fb4b40a2aa0 msgr2=0x7fb4b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 -- 172.21.15.32:0/831728441 shutdown_connections 2023-09-30T22:28:04.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.231+0000 7fb4bc44b700 1 -- 172.21.15.32:0/831728441 wait complete. 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.233+0000 7fb4bc44b700 1 Processor -- start 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.233+0000 7fb4bc44b700 1 -- start start 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.233+0000 7fb4bc44b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b41c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4bc44b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b412cf60 0x7fb4b41c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4bc44b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b41c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4bc44b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b41c92d0 con 0x7fb4b412cf60 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4bc44b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b41c8420 con 0x7fb4b406f5d0 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4bc44b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4118ee0 con 0x7fb4b4135c20 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b41c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba9e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b41c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba9e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b41c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55756/0 (socket says 172.21.15.32:55756) 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba9e8700 1 -- 172.21.15.32:0/1804164988 learned_addr learned my addr 172.21.15.32:0/1804164988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4b99e6700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b412cf60 0x7fb4b41c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 -- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 msgr2=0x7fb4b41c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b41c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 -- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b412cf60 msgr2=0x7fb4b41c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b412cf60 0x7fb4b41c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 -- 172.21.15.32:0/1804164988 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a400b7f0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba9e8700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b41c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.234+0000 7fb4ba1e7700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b41c8850 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c450 tx=0x7fb4b000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.235+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00052d0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.235+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0005430 con 0x7fb4b406f5d0 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.235+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b4129fa0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.235+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00038c0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.235+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b409e1d0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.236+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4b0003a20 con 0x7fb4b406f5d0 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.237+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb4b000ce20 con 0x7fb4b406f5d0 2023-09-30T22:28:04.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.237+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b41350f0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.245+0000 7ff92842a700 1 Processor -- start 2023-09-30T22:28:04.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 -- start start 2023-09-30T22:28:04.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4b0017070 con 0x7fb4b406f5d0 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 0x7ff920130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130e50 0x7ff92006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 0x7ff92006ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92009a390 con 0x7ff92006fb20 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200716a0 con 0x7ff920130540 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.246+0000 7ff92842a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff920071290 con 0x7ff920130e50 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9269c7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 0x7ff92006ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 0x7ff920130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9269c7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 0x7ff92006ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42174/0 (socket says 172.21.15.32:42174) 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 0x7ff920130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58806/0 (socket says 172.21.15.32:58806) 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9269c7700 1 -- 172.21.15.32:0/3338348058 learned_addr learned my addr 172.21.15.32:0/3338348058 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9259c5700 1 --2- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130e50 0x7ff92006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 -- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130e50 msgr2=0x7ff92006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 --2- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130e50 0x7ff92006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 -- 172.21.15.32:0/3338348058 >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 msgr2=0x7ff92006ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 --2- 172.21.15.32:0/3338348058 >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 0x7ff92006ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 -- 172.21.15.32:0/3338348058 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920130340 con 0x7ff920130540 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9269c7700 1 --2- 172.21.15.32:0/3338348058 >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 0x7ff92006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.247+0000 7ff9261c6700 1 --2- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 0x7ff920130910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009870 tx=0x7ff91c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7867397fa044e2c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.248+0000 7ff9251c4700 1 -- 172.21.15.32:0/3338348058 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c011830 con 0x7ff920130540 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.248+0000 7ff9251c4700 1 -- 172.21.15.32:0/3338348058 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c011990 con 0x7ff920130540 2023-09-30T22:28:04.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.248+0000 7ff9251c4700 1 -- 172.21.15.32:0/3338348058 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c011d20 con 0x7ff920130540 2023-09-30T22:28:04.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 -- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 msgr2=0x7ff920130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 --2- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 0x7ff920130910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009870 tx=0x7ff91c009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 -- 172.21.15.32:0/3338348058 shutdown_connections 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 --2- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130540 0x7ff920130910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 --2- 172.21.15.32:0/3338348058 >> v2:172.21.15.32:3300/0 conn(0x7ff92006fb20 0x7ff92006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 --2- 172.21.15.32:0/3338348058 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130e50 0x7ff92006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 -- 172.21.15.32:0/3338348058 >> 172.21.15.32:0/3338348058 conn(0x7ff9200a2aa0 msgr2=0x7ff9200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 -- 172.21.15.32:0/3338348058 shutdown_connections 2023-09-30T22:28:04.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.249+0000 7ff92842a700 1 -- 172.21.15.32:0/3338348058 wait complete. 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.250+0000 7ff92842a700 1 Processor -- start 2023-09-30T22:28:04.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 -- start start 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 0x7ff920118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130540 0x7ff920119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130e50 0x7ff920119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92011bb70 con 0x7ff92006fb20 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92011ae70 con 0x7ff920130e50 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff92842a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9200a1c80 con 0x7ff920130540 2023-09-30T22:28:04.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff9261c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 0x7ff920118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff9261c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 0x7ff920118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42180/0 (socket says 172.21.15.32:42180) 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.251+0000 7ff9261c6700 1 -- 172.21.15.32:0/3500771244 learned_addr learned my addr 172.21.15.32:0/3500771244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9269c7700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130e50 0x7ff920119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9259c5700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130540 0x7ff920119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9261c6700 1 -- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130540 msgr2=0x7ff920119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9261c6700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130540 0x7ff920119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9261c6700 1 -- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130e50 msgr2=0x7ff920119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9261c6700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130e50 0x7ff920119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9261c6700 1 -- 172.21.15.32:0/3500771244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009b30 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff9261c6700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 0x7ff920118cd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff91c000c00 tx=0x7ff91c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.252+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c004240 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.253+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c01f070 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.253+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0045a0 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.253+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c0094a0 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.253+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92011a2a0 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.254+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff91c021030 con 0x7ff92006fb20 2023-09-30T22:28:04.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.254+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff91c0406a0 con 0x7ff92006fb20 2023-09-30T22:28:04.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.256+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff920074ea0 con 0x7ff92006fb20 2023-09-30T22:28:04.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.266+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff91c015080 con 0x7ff92006fb20 2023-09-30T22:28:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.278+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6fac074ea0 con 0x7f6fac09d6f0 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.278+0000 7f6fa2ffd700 1 -- 172.21.15.32:0/2756851977 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fa80199a0 con 0x7f6fac09d6f0 2023-09-30T22:28:04.279 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.280+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 msgr2=0x7f6fac1a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.280+0000 7f6fb4796700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 0x7f6fac1a5f40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009c70 tx=0x7f6fa8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 shutdown_connections 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fac09e410 0x7f6fac1a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fac09d6f0 0x7f6fac1a5f40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 --2- 172.21.15.32:0/2756851977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fac09e000 0x7f6fac1a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 >> 172.21.15.32:0/2756851977 conn(0x7f6fac066f50 msgr2=0x7f6fac1bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 shutdown_connections 2023-09-30T22:28:04.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.281+0000 7f6fb4796700 1 -- 172.21.15.32:0/2756851977 wait complete. 2023-09-30T22:28:04.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:04.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.403+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f019406b9e0 con 0x7f0194131810 2023-09-30T22:28:04.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.403+0000 7f017bfff700 1 -- 172.21.15.32:0/3553001722 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0184011360 con 0x7f0194131810 2023-09-30T22:28:04.425 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194131810 msgr2=0x7f019406fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194131810 0x7f019406fdd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f018400c3f0 tx=0x7f018400c700 comp rx=0 tx=0).stop 2023-09-30T22:28:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 shutdown_connections 2023-09-30T22:28:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0194136f50 0x7f01941c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0194131810 0x7f019406fdd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 --2- 172.21.15.32:0/3553001722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0194134630 0x7f0194070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 >> 172.21.15.32:0/3553001722 conn(0x7f01940a2aa0 msgr2=0x7f01940a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 shutdown_connections 2023-09-30T22:28:04.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.406+0000 7f0199958700 1 -- 172.21.15.32:0/3553001722 wait complete. 2023-09-30T22:28:04.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.460+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb4b4074ea0 con 0x7fb4b406f5d0 2023-09-30T22:28:04.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.461+0000 7fb4aaffd700 1 -- 172.21.15.32:0/1804164988 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4b0017030 con 0x7fb4b406f5d0 2023-09-30T22:28:04.462 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 msgr2=0x7fb4b41c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b41c8850 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c450 tx=0x7fb4b000c760 comp rx=0 tx=0).stop 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 shutdown_connections 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb4b4135c20 0x7fb4b41c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb4b406f5d0 0x7fb4b41c8850 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 --2- 172.21.15.32:0/1804164988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b412cf60 0x7fb4b41c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 >> 172.21.15.32:0/1804164988 conn(0x7fb4b40a2aa0 msgr2=0x7fb4b406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 shutdown_connections 2023-09-30T22:28:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.463+0000 7fb4bc44b700 1 -- 172.21.15.32:0/1804164988 wait complete. 2023-09-30T22:28:04.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.473+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff9200a14c0 con 0x7ff92006fb20 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.473+0000 7ff916ffd700 1 -- 172.21.15.32:0/3500771244 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff91c01a450 con 0x7ff92006fb20 2023-09-30T22:28:04.474 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.475+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 msgr2=0x7ff920118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.475+0000 7ff92842a700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 0x7ff920118cd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff91c000c00 tx=0x7ff91c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 shutdown_connections 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff920130e50 0x7ff920119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff92006fb20 0x7ff920118cd0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 --2- 172.21.15.32:0/3500771244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff920130540 0x7ff920119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 >> 172.21.15.32:0/3500771244 conn(0x7ff9200a2aa0 msgr2=0x7ff92012d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 shutdown_connections 2023-09-30T22:28:04.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:04.476+0000 7ff92842a700 1 -- 172.21.15.32:0/3500771244 wait complete. 2023-09-30T22:28:04.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:04.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:05.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:05.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:05.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 Processor -- start 2023-09-30T22:28:05.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 -- start start 2023-09-30T22:28:05.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f032812cf60 0x7f032806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 0x7f0328136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032812d330 con 0x7f032812cf60 2023-09-30T22:28:05.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032812cd90 con 0x7f032812f8b0 2023-09-30T22:28:05.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.443+0000 7f032d028700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03281306d0 con 0x7f0328130260 2023-09-30T22:28:05.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 0x7f0328136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f031ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f032812cf60 0x7f032806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 0x7f0328136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55788/0 (socket says 172.21.15.32:55788) 2023-09-30T22:28:05.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f031ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58816/0 (socket says 172.21.15.32:58816) 2023-09-30T22:28:05.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032759e700 1 -- 172.21.15.32:0/2392686509 learned_addr learned my addr 172.21.15.32:0/2392686509 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f032812cf60 0x7f032806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42192/0 (socket says 172.21.15.32:42192) 2023-09-30T22:28:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 -- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 msgr2=0x7f0328136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 0x7f0328136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 -- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 msgr2=0x7f032812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 -- 172.21.15.32:0/2392686509 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328071780 con 0x7f032812cf60 2023-09-30T22:28:05.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f031ffff700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:05.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032759e700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 0x7f0328136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:05.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f0326d9d700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032812cf60 0x7f032806f5d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0318009870 tx=0x7f0318009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49e834b53045cc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032659c700 1 -- 172.21.15.32:0/2392686509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318011830 con 0x7f032812cf60 2023-09-30T22:28:05.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032659c700 1 -- 172.21.15.32:0/2392686509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318011990 con 0x7f032812cf60 2023-09-30T22:28:05.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.444+0000 7f032659c700 1 -- 172.21.15.32:0/2392686509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318011d20 con 0x7f032812cf60 2023-09-30T22:28:05.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 -- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032812cf60 msgr2=0x7f032806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032812cf60 0x7f032806f5d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0318009870 tx=0x7f0318009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 -- 172.21.15.32:0/2392686509 shutdown_connections 2023-09-30T22:28:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0328130260 0x7f0328136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f032812cf60 0x7f032806f5d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 --2- 172.21.15.32:0/2392686509 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 -- 172.21.15.32:0/2392686509 >> 172.21.15.32:0/2392686509 conn(0x7f03280a2aa0 msgr2=0x7f03280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 -- 172.21.15.32:0/2392686509 shutdown_connections 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.445+0000 7f032d028700 1 -- 172.21.15.32:0/2392686509 wait complete. 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.446+0000 7f032d028700 1 Processor -- start 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 -- start start 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f032812cf60 0x7f032809bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032809c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 0x7f03281c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032809ea10 con 0x7f0328130260 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032809d330 con 0x7f032812f8b0 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032d028700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f032809ebf0 con 0x7f032812cf60 2023-09-30T22:28:05.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 0x7f03281c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f031ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032809c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 0x7f03281c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42200/0 (socket says 172.21.15.32:42200) 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f031ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032809c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58818/0 (socket says 172.21.15.32:58818) 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f032759e700 1 -- 172.21.15.32:0/2599688816 learned_addr learned my addr 172.21.15.32:0/2599688816 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.447+0000 7f0326d9d700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f032812cf60 0x7f032809bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032759e700 1 -- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f032812cf60 msgr2=0x7f032809bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032759e700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f032812cf60 0x7f032809bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032759e700 1 -- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 msgr2=0x7f032809c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032759e700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032809c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032759e700 1 -- 172.21.15.32:0/2599688816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031000d940 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f031ffff700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032809c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032759e700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 0x7f03281c3eb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0320009ef0 tx=0x7f032000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000cb40 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032000f040 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0320009b30 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000cee0 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.448+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180094a0 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.449+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0320014480 con 0x7f0328130260 2023-09-30T22:28:05.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.449+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f032003fe10 con 0x7f0328130260 2023-09-30T22:28:05.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.449+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032809b580 con 0x7f0328130260 2023-09-30T22:28:05.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.454+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f032001c950 con 0x7f0328130260 2023-09-30T22:28:05.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:05.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:05.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.573+0000 7fe295f48700 1 Processor -- start 2023-09-30T22:28:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe295f48700 1 -- start start 2023-09-30T22:28:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe295f48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe29009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe295f48700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe29009e410 0x7fe290130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe295f48700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29009f130 con 0x7fe29009e000 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe295f48700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29009f520 con 0x7fe29009e410 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe295f48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29009e930 con 0x7fe29009d6f0 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe28ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe29009e410 0x7fe290130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe28ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe29009e410 0x7fe290130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42206/0 (socket says 172.21.15.32:42206) 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe28ffff700 1 -- 172.21.15.32:0/406567777 learned_addr learned my addr 172.21.15.32:0/406567777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.574+0000 7fe28f7fe700 1 --2- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe29009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28ffff700 1 -- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 msgr2=0x7fe29009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28ffff700 1 --2- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe29009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28ffff700 1 -- 172.21.15.32:0/406567777 >> v1:172.21.15.32:6789/0 conn(0x7fe29009e000 legacy=0x7fe290130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28ffff700 1 -- 172.21.15.32:0/406567777 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29009b2d0 con 0x7fe29009e410 2023-09-30T22:28:05.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28ffff700 1 --2- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290130d70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe288009e50 tx=0x7fe28800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e75b16347cbf4a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28e7fc700 1 -- 172.21.15.32:0/406567777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800e070 con 0x7fe29009e410 2023-09-30T22:28:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28e7fc700 1 -- 172.21.15.32:0/406567777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800b910 con 0x7fe29009e410 2023-09-30T22:28:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.575+0000 7fe28e7fc700 1 -- 172.21.15.32:0/406567777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800bcc0 con 0x7fe29009e410 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.576+0000 7fe295f48700 1 -- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 msgr2=0x7fe290130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.576+0000 7fe295f48700 1 --2- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290130d70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe288009e50 tx=0x7fe28800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.576+0000 7fe295f48700 1 -- 172.21.15.32:0/406567777 shutdown_connections 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.576+0000 7fe295f48700 1 --2- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290130d70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.576+0000 7fe295f48700 1 --2- 172.21.15.32:0/406567777 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe29009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.576+0000 7fe295f48700 1 -- 172.21.15.32:0/406567777 >> 172.21.15.32:0/406567777 conn(0x7fe290066f50 msgr2=0x7fe29009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.577+0000 7fe295f48700 1 -- 172.21.15.32:0/406567777 shutdown_connections 2023-09-30T22:28:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.577+0000 7fe295f48700 1 -- 172.21.15.32:0/406567777 wait complete. 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.578+0000 7fe295f48700 1 Processor -- start 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.578+0000 7fe295f48700 1 -- start start 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe295f48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe290139360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe295f48700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe29009e000 0x7fe290131620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe295f48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290131b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe295f48700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290132f20 con 0x7fe29009e410 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe295f48700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe290132d30 con 0x7fe29009d6f0 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe295f48700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901335c0 con 0x7fe29009e000 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe29009e000 0x7fe290131620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42210/0 (socket says 172.21.15.32:42210) 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28ffff700 1 -- 172.21.15.32:0/1289717461 learned_addr learned my addr 172.21.15.32:0/1289717461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28f7fe700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe290139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28effd700 1 -- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 msgr2=0x7fe290139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28effd700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe290139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28effd700 1 -- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 msgr2=0x7fe290131b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28effd700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290131b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28effd700 1 -- 172.21.15.32:0/1289717461 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009b30 con 0x7fe29009e000 2023-09-30T22:28:05.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.579+0000 7fe28ffff700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290131b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.580+0000 7fe28effd700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe29009e000 0x7fe290131620 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe284001760 tx=0x7fe284001a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.581+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284013070 con 0x7fe29009e000 2023-09-30T22:28:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.581+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28400e2e0 con 0x7fe29009e000 2023-09-30T22:28:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.581+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840052f0 con 0x7fe29009e000 2023-09-30T22:28:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.581+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2901343f0 con 0x7fe29009e000 2023-09-30T22:28:05.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.581+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29009c700 con 0x7fe29009e000 2023-09-30T22:28:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.582+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290134600 con 0x7fe29009e000 2023-09-30T22:28:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.582+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe284003680 con 0x7fe29009e000 2023-09-30T22:28:05.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.582+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe284042c30 con 0x7fe29009e000 2023-09-30T22:28:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.591+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe290134600 con 0x7fe29009e000 2023-09-30T22:28:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.634+0000 7f117679a700 1 Processor -- start 2023-09-30T22:28:05.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.634+0000 7f117679a700 1 -- start start 2023-09-30T22:28:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f117679a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f11700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f117679a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1170138d80 0x7f117013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f117679a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11700a2a70 con 0x7f117009d630 2023-09-30T22:28:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f117679a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11700a01d0 con 0x7f1170138d80 2023-09-30T22:28:05.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f117679a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117009e2b0 con 0x7f11700a0a30 2023-09-30T22:28:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f1174d37700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1170138d80 0x7f117013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f116f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f117009d630 0x7f11700a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44444/0 (socket says 172.21.15.32:44444) 2023-09-30T22:28:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f116f7fe700 1 -- 172.21.15.32:0/1193770601 learned_addr learned my addr 172.21.15.32:0/1193770601 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:05.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.635+0000 7f116ffff700 1 --2- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f11700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f1174d37700 1 -- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 msgr2=0x7f11700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f1174d37700 1 --2- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f11700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f1174d37700 1 -- 172.21.15.32:0/1193770601 >> v1:172.21.15.32:6789/0 conn(0x7f117009d630 legacy=0x7f11700a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f1174d37700 1 -- 172.21.15.32:0/1193770601 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117009dc50 con 0x7f1170138d80 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f116effd700 1 -- 172.21.15.32:0/1193770601 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805781640 0 0) 0x7f11700a2a70 con 0x7f117009d630 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f1174d37700 1 --2- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f117013b160 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1168009d90 tx=0x7f116800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e376d0498d079b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f116effd700 1 -- 172.21.15.32:0/1193770601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800e070 con 0x7f1170138d80 2023-09-30T22:28:05.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f116effd700 1 -- 172.21.15.32:0/1193770601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800bb20 con 0x7f1170138d80 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.636+0000 7f116effd700 1 -- 172.21.15.32:0/1193770601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168012430 con 0x7f1170138d80 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 -- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 msgr2=0x7f117013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 --2- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f117013b160 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1168009d90 tx=0x7f116800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 -- 172.21.15.32:0/1193770601 shutdown_connections 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 --2- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f117013b160 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 --2- 172.21.15.32:0/1193770601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f11700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 -- 172.21.15.32:0/1193770601 >> 172.21.15.32:0/1193770601 conn(0x7f11700a3050 msgr2=0x7f11700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 -- 172.21.15.32:0/1193770601 shutdown_connections 2023-09-30T22:28:05.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.637+0000 7f117679a700 1 -- 172.21.15.32:0/1193770601 wait complete. 2023-09-30T22:28:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.638+0000 7f117679a700 1 Processor -- start 2023-09-30T22:28:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.638+0000 7f117679a700 1 -- start start 2023-09-30T22:28:05.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f117679a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f117009d630 0x7f1170133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f117679a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f1170133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f117679a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f11701342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f117679a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1170134d00 con 0x7f1170138d80 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f117679a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1170136700 con 0x7f11700a0a30 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f117679a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1170137070 con 0x7f117009d630 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f1174d37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f11701342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f1174d37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f11701342b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42228/0 (socket says 172.21.15.32:42228) 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f1174d37700 1 -- 172.21.15.32:0/84847588 learned_addr learned my addr 172.21.15.32:0/84847588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f116f7fe700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f1170133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f116f7fe700 1 -- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f117009d630 msgr2=0x7f1170133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f116f7fe700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f117009d630 0x7f1170133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f116f7fe700 1 -- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 msgr2=0x7f11701342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f116f7fe700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f11701342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f116f7fe700 1 -- 172.21.15.32:0/84847588 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168009a70 con 0x7f11700a0a30 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.639+0000 7f1174d37700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f11701342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.640+0000 7f116f7fe700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f1170133d70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f117006d980 tx=0x7f116400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.640+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164004c80 con 0x7f11700a0a30 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.640+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164004de0 con 0x7f11700a0a30 2023-09-30T22:28:05.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.640+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11700a0410 con 0x7f11700a0a30 2023-09-30T22:28:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.640+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164007310 con 0x7f11700a0a30 2023-09-30T22:28:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.640+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1170136e00 con 0x7f11700a0a30 2023-09-30T22:28:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.641+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1170137f30 con 0x7f11700a0a30 2023-09-30T22:28:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.642+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1164007470 con 0x7f11700a0a30 2023-09-30T22:28:05.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.642+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f116403f350 con 0x7f11700a0a30 2023-09-30T22:28:05.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.651+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1164007720 con 0x7f11700a0a30 2023-09-30T22:28:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.680+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f032809e870 con 0x7f0328130260 2023-09-30T22:28:05.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.680+0000 7f031f7fe700 1 -- 172.21.15.32:0/2599688816 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f032809e870 con 0x7f0328130260 2023-09-30T22:28:05.681 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.682+0000 7fb376302700 1 Processor -- start 2023-09-30T22:28:05.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.682+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 msgr2=0x7f03281c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.682+0000 7f032d028700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 0x7f03281c3eb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0320009ef0 tx=0x7f032000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.682+0000 7fb376302700 1 -- start start 2023-09-30T22:28:05.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.682+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 shutdown_connections 2023-09-30T22:28:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7f032d028700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0328130260 0x7f03281c3eb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7f032d028700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f032812cf60 0x7f032809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7f032d028700 1 --2- 172.21.15.32:0/2599688816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f032812f8b0 0x7f032809c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 >> 172.21.15.32:0/2599688816 conn(0x7f03280a2aa0 msgr2=0x7f03281b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 shutdown_connections 2023-09-30T22:28:05.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7f032d028700 1 -- 172.21.15.32:0/2599688816 wait complete. 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb376302700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb37009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb376302700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb37013d1f0 0x7fb37013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb376302700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37009a810 con 0x7fb37006a770 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb376302700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37006ab40 con 0x7fb37013d1f0 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb376302700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb37009ebc0 con 0x7fb37009de00 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb37489f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb37013d1f0 0x7fb37013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb37489f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb37013d1f0 0x7fb37013f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42238/0 (socket says 172.21.15.32:42238) 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.683+0000 7fb37489f700 1 -- 172.21.15.32:0/3632091507 learned_addr learned my addr 172.21.15.32:0/3632091507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb36f7fe700 1 --2- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb37009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb37489f700 1 -- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 msgr2=0x7fb37009e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb37489f700 1 --2- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb37009e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb37489f700 1 -- 172.21.15.32:0/3632091507 >> v1:172.21.15.32:6789/0 conn(0x7fb37006a770 legacy=0x7fb37009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb37489f700 1 -- 172.21.15.32:0/3632091507 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37009e920 con 0x7fb37013d1f0 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb37489f700 1 --2- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb37013f5d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb368009e50 tx=0x7fb36800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee34f353f40bb15 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb36effd700 1 -- 172.21.15.32:0/3632091507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800e070 con 0x7fb37013d1f0 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb36effd700 1 -- 172.21.15.32:0/3632091507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb368003a20 con 0x7fb37013d1f0 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.684+0000 7fb36effd700 1 -- 172.21.15.32:0/3632091507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368003dd0 con 0x7fb37013d1f0 2023-09-30T22:28:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 -- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 msgr2=0x7fb37013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 --2- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb37013f5d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb368009e50 tx=0x7fb36800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 -- 172.21.15.32:0/3632091507 shutdown_connections 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 --2- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb37013f5d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 --2- 172.21.15.32:0/3632091507 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb37009e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 -- 172.21.15.32:0/3632091507 >> 172.21.15.32:0/3632091507 conn(0x7fb370066f50 msgr2=0x7fb37009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 -- 172.21.15.32:0/3632091507 shutdown_connections 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.685+0000 7fb376302700 1 -- 172.21.15.32:0/3632091507 wait complete. 2023-09-30T22:28:05.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.686+0000 7fb376302700 1 Processor -- start 2023-09-30T22:28:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 -- start start 2023-09-30T22:28:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb37006a770 0x7fb3701260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb3701265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb370126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370128570 con 0x7fb37013d1f0 2023-09-30T22:28:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37009f1a0 con 0x7fb37006a770 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb376302700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700a3990 con 0x7fb37009de00 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb37489f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb370126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb37489f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb370126b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42244/0 (socket says 172.21.15.32:42244) 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb37489f700 1 -- 172.21.15.32:0/3625632490 learned_addr learned my addr 172.21.15.32:0/3625632490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.687+0000 7fb36ffff700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb37006a770 0x7fb3701260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb37489f700 1 -- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 msgr2=0x7fb3701265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb37489f700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb3701265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb37489f700 1 -- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb37006a770 msgr2=0x7fb3701260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb37489f700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb37006a770 0x7fb3701260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb37489f700 1 -- 172.21.15.32:0/3625632490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb368009b30 con 0x7fb37013d1f0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb36ffff700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb37006a770 0x7fb3701260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb37489f700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb370126b20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb368015040 tx=0x7fb368004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368004440 con 0x7fb37013d1f0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3680045c0 con 0x7fb37013d1f0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.688+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368004970 con 0x7fb37013d1f0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.689+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3701281c0 con 0x7fb37013d1f0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.689+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb370127310 con 0x7fb37013d1f0 2023-09-30T22:28:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.690+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb368025450 con 0x7fb37013d1f0 2023-09-30T22:28:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.690+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb368040cb0 con 0x7fb37013d1f0 2023-09-30T22:28:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.690+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb370074ea0 con 0x7fb37013d1f0 2023-09-30T22:28:05.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.695+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb368017020 con 0x7fb37013d1f0 2023-09-30T22:28:05.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.807+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe290134600 con 0x7fe29009e000 2023-09-30T22:28:05.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.807+0000 7fe277fff700 1 -- 172.21.15.32:0/1289717461 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe290134600 con 0x7fe29009e000 2023-09-30T22:28:05.808 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.809+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe29009e000 msgr2=0x7fe290131620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe29009e000 0x7fe290131620 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe284001760 tx=0x7fe284001a70 comp rx=0 tx=0).stop 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 shutdown_connections 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe29009e410 0x7fe290131b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe29009d6f0 0x7fe290139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 --2- 172.21.15.32:0/1289717461 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe29009e000 0x7fe290131620 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 >> 172.21.15.32:0/1289717461 conn(0x7fe290066f50 msgr2=0x7fe2900a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 shutdown_connections 2023-09-30T22:28:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.810+0000 7fe295f48700 1 -- 172.21.15.32:0/1289717461 wait complete. 2023-09-30T22:28:05.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.891+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f11700747f0 con 0x7f11700a0a30 2023-09-30T22:28:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.892+0000 7f116cff9700 1 -- 172.21.15.32:0/84847588 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f116401bb30 con 0x7f11700a0a30 2023-09-30T22:28:05.894 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:05.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 msgr2=0x7f1170133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f1170133d70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f117006d980 tx=0x7f116400c800 comp rx=0 tx=0).stop 2023-09-30T22:28:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 shutdown_connections 2023-09-30T22:28:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1170138d80 0x7f11701342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f117009d630 0x7f1170133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 --2- 172.21.15.32:0/84847588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11700a0a30 0x7f1170133d70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 >> 172.21.15.32:0/84847588 conn(0x7f11700a3050 msgr2=0x7f117009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 shutdown_connections 2023-09-30T22:28:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.894+0000 7f117679a700 1 -- 172.21.15.32:0/84847588 wait complete. 2023-09-30T22:28:05.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:05.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.919+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb37009a3d0 con 0x7fb37013d1f0 2023-09-30T22:28:05.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.920+0000 7fb36cff9700 1 -- 172.21.15.32:0/3625632490 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb368034090 con 0x7fb37013d1f0 2023-09-30T22:28:05.921 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 msgr2=0x7fb370126b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:05.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb370126b20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb368015040 tx=0x7fb368004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 shutdown_connections 2023-09-30T22:28:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb37013d1f0 0x7fb370126b20 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb37006a770 0x7fb3701260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 --2- 172.21.15.32:0/3625632490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb37009de00 0x7fb3701265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 >> 172.21.15.32:0/3625632490 conn(0x7fb370066f50 msgr2=0x7fb3700a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:05.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 shutdown_connections 2023-09-30T22:28:05.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:05.922+0000 7fb376302700 1 -- 172.21.15.32:0/3625632490 wait complete. 2023-09-30T22:28:05.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:06.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:06.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 Processor -- start 2023-09-30T22:28:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 -- start start 2023-09-30T22:28:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09e1d0 0x7f09ec135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 0x7f09ec138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec09f130 con 0x7f09ec09d6f0 2023-09-30T22:28:06.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec09f520 con 0x7f09ec135e50 2023-09-30T22:28:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f2f7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec09e930 con 0x7f09ec09e1d0 2023-09-30T22:28:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f151c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 0x7f09ec138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09e1d0 0x7f09ec135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f151c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 0x7f09ec138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42250/0 (socket says 172.21.15.32:42250) 2023-09-30T22:28:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.851+0000 7f09f151c700 1 -- 172.21.15.32:0/190544464 learned_addr learned my addr 172.21.15.32:0/190544464 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09ebfff700 1 -- 172.21.15.32:0/190544464 >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 msgr2=0x7f09ec138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09ebfff700 1 --2- 172.21.15.32:0/190544464 >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 0x7f09ec138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09ebfff700 1 -- 172.21.15.32:0/190544464 >> v1:172.21.15.32:6789/0 conn(0x7f09ec09d6f0 legacy=0x7f09ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09ebfff700 1 -- 172.21.15.32:0/190544464 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec09b2d0 con 0x7f09ec09e1d0 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09f151c700 1 --2- 172.21.15.32:0/190544464 >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 0x7f09ec138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09ebfff700 1 --2- 172.21.15.32:0/190544464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09e1d0 0x7f09ec135910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009870 tx=0x7f09d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a805e05193ee50 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09eb7fe700 1 -- 172.21.15.32:0/190544464 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8011830 con 0x7f09ec09e1d0 2023-09-30T22:28:06.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09eb7fe700 1 -- 172.21.15.32:0/190544464 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d8011990 con 0x7f09ec09e1d0 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.852+0000 7f09eb7fe700 1 -- 172.21.15.32:0/190544464 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8011d20 con 0x7f09ec09e1d0 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 -- 172.21.15.32:0/190544464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09e1d0 msgr2=0x7f09ec135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/190544464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09e1d0 0x7f09ec135910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009870 tx=0x7f09d8009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 -- 172.21.15.32:0/190544464 shutdown_connections 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/190544464 >> v2:172.21.15.32:3300/0 conn(0x7f09ec135e50 0x7f09ec138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/190544464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09e1d0 0x7f09ec135910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 -- 172.21.15.32:0/190544464 >> 172.21.15.32:0/190544464 conn(0x7f09ec066f50 msgr2=0x7f09ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 -- 172.21.15.32:0/190544464 shutdown_connections 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.853+0000 7f09f2f7f700 1 -- 172.21.15.32:0/190544464 wait complete. 2023-09-30T22:28:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.854+0000 7f09f2f7f700 1 Processor -- start 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.854+0000 7f09f2f7f700 1 -- start start 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f2f7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09d6f0 0x7f09ec11b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f2f7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 0x7f09ec11ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f2f7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09ec135e50 0x7f09ec11bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f2f7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec06e8b0 con 0x7f09ec09e1d0 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f2f7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec09c700 con 0x7f09ec135e50 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f2f7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec11d1f0 con 0x7f09ec09d6f0 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 0x7f09ec11ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f0d1b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09d6f0 0x7f09ec11b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 0x7f09ec11ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42260/0 (socket says 172.21.15.32:42260) 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09ebfff700 1 -- 172.21.15.32:0/2642953674 learned_addr learned my addr 172.21.15.32:0/2642953674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.855+0000 7f09f151c700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09ec135e50 0x7f09ec11bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09ebfff700 1 -- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09d6f0 msgr2=0x7f09ec11b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09ebfff700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09d6f0 0x7f09ec11b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09ebfff700 1 -- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09ec135e50 msgr2=0x7f09ec11bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09ebfff700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09ec135e50 0x7f09ec11bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09ebfff700 1 -- 172.21.15.32:0/2642953674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4009b30 con 0x7f09ec09e1d0 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09ebfff700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 0x7f09ec11ba60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d80036a0 con 0x7f09ec09e1d0 2023-09-30T22:28:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d801f070 con 0x7f09ec09e1d0 2023-09-30T22:28:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d80094a0 con 0x7f09ec09e1d0 2023-09-30T22:28:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8003a60 con 0x7f09ec09e1d0 2023-09-30T22:28:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.856+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec11ed60 con 0x7f09ec09e1d0 2023-09-30T22:28:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.857+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09d8003bc0 con 0x7f09ec09e1d0 2023-09-30T22:28:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.857+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f09d8021070 con 0x7f09ec09e1d0 2023-09-30T22:28:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.857+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec11ed60 con 0x7f09ec09e1d0 2023-09-30T22:28:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.862+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09ec11ed60 con 0x7f09ec09e1d0 2023-09-30T22:28:06.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:06.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:06.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.977+0000 7fe4a5275700 1 Processor -- start 2023-09-30T22:28:06.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.977+0000 7fe4a5275700 1 -- start start 2023-09-30T22:28:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe4a5275700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009d6f0 0x7fe4a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe4a5275700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e410 0x7fe4a0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe4a5275700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a009b120 con 0x7fe4a009e000 2023-09-30T22:28:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe4a5275700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a006b460 con 0x7fe4a009e410 2023-09-30T22:28:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe4a5275700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a009a7e0 con 0x7fe4a009d6f0 2023-09-30T22:28:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe49e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe4a009e000 0x7fe4a01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44466/0 (socket says 172.21.15.32:44466) 2023-09-30T22:28:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe49e7fc700 1 -- 172.21.15.32:0/2334983177 learned_addr learned my addr 172.21.15.32:0/2334983177 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe49f7fe700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e410 0x7fe4a0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.978+0000 7fe49effd700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009d6f0 0x7fe4a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49f7fe700 1 -- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009d6f0 msgr2=0x7fe4a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49f7fe700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009d6f0 0x7fe4a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49f7fe700 1 -- 172.21.15.32:0/2334983177 >> v1:172.21.15.32:6789/0 conn(0x7fe4a009e000 legacy=0x7fe4a01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49f7fe700 1 -- 172.21.15.32:0/2334983177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a009a500 con 0x7fe4a009e410 2023-09-30T22:28:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49effd700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009d6f0 0x7fe4a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49f7fe700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e410 0x7fe4a0134100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe498009dc0 tx=0x7fe49800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91836c2a5090c8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:06.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49dffb700 1 -- 172.21.15.32:0/2334983177 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800e070 con 0x7fe4a009e410 2023-09-30T22:28:06.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49dffb700 1 -- 172.21.15.32:0/2334983177 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49800bb20 con 0x7fe4a009e410 2023-09-30T22:28:06.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.979+0000 7fe49dffb700 1 -- 172.21.15.32:0/2334983177 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498012430 con 0x7fe4a009e410 2023-09-30T22:28:06.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.980+0000 7fe4a5275700 1 -- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e410 msgr2=0x7fe4a0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.980+0000 7fe4a5275700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e410 0x7fe4a0134100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe498009dc0 tx=0x7fe49800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.980+0000 7fe4a5275700 1 -- 172.21.15.32:0/2334983177 shutdown_connections 2023-09-30T22:28:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.980+0000 7fe4a5275700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e410 0x7fe4a0134100 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.980+0000 7fe4a5275700 1 --2- 172.21.15.32:0/2334983177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009d6f0 0x7fe4a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.980+0000 7fe4a5275700 1 -- 172.21.15.32:0/2334983177 >> 172.21.15.32:0/2334983177 conn(0x7fe4a0066f50 msgr2=0x7fe4a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.981+0000 7fe4a5275700 1 -- 172.21.15.32:0/2334983177 shutdown_connections 2023-09-30T22:28:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.981+0000 7fe4a5275700 1 -- 172.21.15.32:0/2334983177 wait complete. 2023-09-30T22:28:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.982+0000 7fe4a5275700 1 Processor -- start 2023-09-30T22:28:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.982+0000 7fe4a5275700 1 -- start start 2023-09-30T22:28:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe4a5275700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a009d6f0 0x7fe4a01273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe4a5275700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e000 0x7fe4a0127900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe4a5275700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 0x7fe4a0127e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe4a5275700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a0134050 con 0x7fe4a009d6f0 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe4a5275700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a012b7c0 con 0x7fe4a009e000 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe4a5275700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a012a900 con 0x7fe4a009e410 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe49f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 0x7fe4a0127e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe49f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 0x7fe4a0127e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55870/0 (socket says 172.21.15.32:55870) 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe49f7fe700 1 -- 172.21.15.32:0/3098791996 learned_addr learned my addr 172.21.15.32:0/3098791996 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.983+0000 7fe49e7fc700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e000 0x7fe4a0127900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49effd700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a009d6f0 0x7fe4a01273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49f7fe700 1 -- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e000 msgr2=0x7fe4a0127900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49f7fe700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e000 0x7fe4a0127900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49f7fe700 1 -- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a009d6f0 msgr2=0x7fe4a01273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49f7fe700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a009d6f0 0x7fe4a01273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49f7fe700 1 -- 172.21.15.32:0/3098791996 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900094a0 con 0x7fe4a009e410 2023-09-30T22:28:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49effd700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a009d6f0 0x7fe4a01273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe49f7fe700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 0x7fe4a0127e40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b680 tx=0x7fe498004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.984+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498004920 con 0x7fe4a009e410 2023-09-30T22:28:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.985+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498020070 con 0x7fe4a009e410 2023-09-30T22:28:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.985+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe498009a70 con 0x7fe4a009e410 2023-09-30T22:28:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.985+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4980276c0 con 0x7fe4a009e410 2023-09-30T22:28:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.985+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a009c6d0 con 0x7fe4a009e410 2023-09-30T22:28:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.986+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe498027820 con 0x7fe4a009e410 2023-09-30T22:28:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.986+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a0074ea0 con 0x7fe4a009e410 2023-09-30T22:28:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.986+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe498022070 con 0x7fe4a009e410 2023-09-30T22:28:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:06.991+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe498046060 con 0x7fe4a009e410 2023-09-30T22:28:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.072+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f09ec11ed60 con 0x7f09ec09e1d0 2023-09-30T22:28:07.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.072+0000 7f09e97fa700 1 -- 172.21.15.32:0/2642953674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09ec11ed60 con 0x7f09ec09e1d0 2023-09-30T22:28:07.074 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 msgr2=0x7f09ec11ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 0x7f09ec11ba60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000c00 tx=0x7f09d8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 shutdown_connections 2023-09-30T22:28:07.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09ec135e50 0x7f09ec11bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09ec09d6f0 0x7f09ec11b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 --2- 172.21.15.32:0/2642953674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09ec09e1d0 0x7f09ec11ba60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 >> 172.21.15.32:0/2642953674 conn(0x7f09ec066f50 msgr2=0x7f09ec1339c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 shutdown_connections 2023-09-30T22:28:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.075+0000 7f09f2f7f700 1 -- 172.21.15.32:0/2642953674 wait complete. 2023-09-30T22:28:07.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.097+0000 7f1f4b59e700 1 Processor -- start 2023-09-30T22:28:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.097+0000 7f1f4b59e700 1 -- start start 2023-09-30T22:28:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f2905311700 1 Processor -- start 2023-09-30T22:28:07.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f2905311700 1 -- start start 2023-09-30T22:28:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4b59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 0x7f1f4c0a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f4c0a1520 0x7f1f4c133e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4b59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c09e770 con 0x7f1f4c0a1110 2023-09-30T22:28:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c0a1aa0 con 0x7f1f4c0a1520 2023-09-30T22:28:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4b59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c09cc80 con 0x7f1f4c0a0800 2023-09-30T22:28:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 0x7f1f4c0a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 0x7f1f4c0a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58892/0 (socket says 172.21.15.32:58892) 2023-09-30T22:28:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.098+0000 7f1f4a59c700 1 -- 172.21.15.32:0/2024654335 learned_addr learned my addr 172.21.15.32:0/2024654335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4ad9d700 1 --2- 172.21.15.32:0/2024654335 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c0a1520 0x7f1f4c133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f2905311700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 0x7f290009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f2905311700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f2905311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009e410 0x7f2900130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f2905311700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290009f520 con 0x7f290009d6f0 2023-09-30T22:28:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f2905311700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290009f130 con 0x7f290009e410 2023-09-30T22:28:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f2905311700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f290009e930 con 0x7f290009e000 2023-09-30T22:28:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4a59c700 1 -- 172.21.15.32:0/2024654335 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c0a1520 msgr2=0x7f1f4c133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4a59c700 1 --2- 172.21.15.32:0/2024654335 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c0a1520 0x7f1f4c133e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4a59c700 1 -- 172.21.15.32:0/2024654335 >> v1:172.21.15.32:6789/0 conn(0x7f1f4c0a1110 legacy=0x7f1f4c133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4a59c700 1 -- 172.21.15.32:0/2024654335 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0a1c70 con 0x7f1f4c0a0800 2023-09-30T22:28:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28feffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 0x7f290009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28feffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 0x7f290009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42284/0 (socket says 172.21.15.32:42284) 2023-09-30T22:28:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28feffd700 1 -- 172.21.15.32:0/1263825546 learned_addr learned my addr 172.21.15.32:0/1263825546 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28fe7fc700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28ff7fe700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009e410 0x7f2900130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4a59c700 1 --2- 172.21.15.32:0/2024654335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 0x7f1f4c0a0bd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009920 tx=0x7f1f3c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eaaba26fa9c87da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f1f4959a700 1 -- 172.21.15.32:0/2024654335 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00ba20 con 0x7f1f4c0a0800 2023-09-30T22:28:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f1f4959a700 1 -- 172.21.15.32:0/2024654335 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00bb80 con 0x7f1f4c0a0800 2023-09-30T22:28:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28ff7fe700 1 -- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 msgr2=0x7f2900130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28ff7fe700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28ff7fe700 1 -- 172.21.15.32:0/1263825546 >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 msgr2=0x7f290009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.099+0000 7f28ff7fe700 1 --2- 172.21.15.32:0/1263825546 >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 0x7f290009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f28ff7fe700 1 -- 172.21.15.32:0/1263825546 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290009b2d0 con 0x7f290009e410 2023-09-30T22:28:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f1f4959a700 1 -- 172.21.15.32:0/2024654335 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c009e70 con 0x7f1f4c0a0800 2023-09-30T22:28:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f28ff7fe700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009e410 0x7f2900130ba0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009d90 tx=0x7f28f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ea1884f43af5e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f28feffd700 1 --2- 172.21.15.32:0/1263825546 >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 0x7f290009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f28fdffb700 1 -- 172.21.15.32:0/1263825546 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800e070 con 0x7f290009e410 2023-09-30T22:28:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f28fdffb700 1 -- 172.21.15.32:0/1263825546 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f800bb20 con 0x7f290009e410 2023-09-30T22:28:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f28fdffb700 1 -- 172.21.15.32:0/1263825546 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8012430 con 0x7f290009e410 2023-09-30T22:28:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f1f4b59e700 1 -- 172.21.15.32:0/2024654335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 msgr2=0x7f1f4c0a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.100+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/2024654335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 0x7f1f4c0a0bd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009920 tx=0x7f1f3c009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f1f4b59e700 1 -- 172.21.15.32:0/2024654335 shutdown_connections 2023-09-30T22:28:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/2024654335 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c0a1520 0x7f1f4c133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/2024654335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a0800 0x7f1f4c0a0bd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f1f4b59e700 1 -- 172.21.15.32:0/2024654335 >> 172.21.15.32:0/2024654335 conn(0x7f1f4c0a6140 msgr2=0x7f1f4c0a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f1f4b59e700 1 -- 172.21.15.32:0/2024654335 shutdown_connections 2023-09-30T22:28:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f1f4b59e700 1 -- 172.21.15.32:0/2024654335 wait complete. 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 -- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009e410 msgr2=0x7f2900130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009e410 0x7f2900130ba0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009d90 tx=0x7f28f800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 -- 172.21.15.32:0/1263825546 shutdown_connections 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009e410 0x7f2900130ba0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 --2- 172.21.15.32:0/1263825546 >> v2:172.21.15.32:3300/0 conn(0x7f290009d6f0 0x7f290009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 --2- 172.21.15.32:0/1263825546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 -- 172.21.15.32:0/1263825546 >> 172.21.15.32:0/1263825546 conn(0x7f2900066f50 msgr2=0x7f290009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 -- 172.21.15.32:0/1263825546 shutdown_connections 2023-09-30T22:28:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.101+0000 7f2905311700 1 -- 172.21.15.32:0/1263825546 wait complete. 2023-09-30T22:28:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 Processor -- start 2023-09-30T22:28:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 -- start start 2023-09-30T22:28:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c0a0800 0x7f1f4c1c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a1110 0x7f1f4c1c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 0x7f1f4c1c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c134670 con 0x7f1f4c0a0800 2023-09-30T22:28:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c123af0 con 0x7f1f4c0a1110 2023-09-30T22:28:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f1f4b59e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c124810 con 0x7f1f4c0a1520 2023-09-30T22:28:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.102+0000 7f2905311700 1 Processor -- start 2023-09-30T22:28:07.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 0x7f1f4c1c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 -- start start 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 0x7f1f4c1c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55886/0 (socket says 172.21.15.32:55886) 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4ad9d700 1 -- 172.21.15.32:0/3334550822 learned_addr learned my addr 172.21.15.32:0/3334550822 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c0a0800 0x7f1f4c1c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 -- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 msgr2=0x7f1f4c1c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 0x7f1f4c1c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 -- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a1110 msgr2=0x7f1f4c1c8ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a1110 0x7f1f4c1c8ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 -- 172.21.15.32:0/3334550822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c009550 con 0x7f1f4c0a0800 2023-09-30T22:28:07.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4ad9d700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 0x7f1f4c1c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f1f4a59c700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c0a0800 0x7f1f4c1c8570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c016000 tx=0x7f1f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009d6f0 0x7f29001225b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900122af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290009e410 0x7f2900123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290009c770 con 0x7f290009e410 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2900124340 con 0x7f290009d6f0 2023-09-30T22:28:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.103+0000 7f2905311700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29000a3ad0 con 0x7f290009e000 2023-09-30T22:28:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c004510 con 0x7f1f4c0a0800 2023-09-30T22:28:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c0036a0 con 0x7f1f4c0a0800 2023-09-30T22:28:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c124270 con 0x7f1f4c0a0800 2023-09-30T22:28:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c003a50 con 0x7f1f4c0a0800 2023-09-30T22:28:07.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c122900 con 0x7f1f4c0a0800 2023-09-30T22:28:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290009e410 0x7f2900123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290009e410 0x7f2900123030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42306/0 (socket says 172.21.15.32:42306) 2023-09-30T22:28:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28ff7fe700 1 -- 172.21.15.32:0/2850979034 learned_addr learned my addr 172.21.15.32:0/2850979034 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28feffd700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009d6f0 0x7f29001225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:07.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 -- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009d6f0 msgr2=0x7f29001225b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009d6f0 0x7f29001225b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 -- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290009e410 msgr2=0x7f2900123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290009e410 0x7f2900123030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 -- 172.21.15.32:0/2850979034 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8009a70 con 0x7f290009e000 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.104+0000 7f28fe7fc700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900122af0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f28f400e900 tx=0x7f28f400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.105+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4013680 con 0x7f290009e000 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.105+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f40137e0 con 0x7f290009e000 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.105+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4013b50 con 0x7f290009e000 2023-09-30T22:28:07.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.105+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2900125df0 con 0x7f290009e000 2023-09-30T22:28:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.105+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c122a90 con 0x7f1f4c0a0800 2023-09-30T22:28:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.105+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f3c021070 con 0x7f1f4c0a0800 2023-09-30T22:28:07.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.106+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2900126860 con 0x7f290009e000 2023-09-30T22:28:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.107+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2900074ea0 con 0x7f290009e000 2023-09-30T22:28:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.107+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28f4013cb0 con 0x7f290009e000 2023-09-30T22:28:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.108+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f3c0115e0 con 0x7f1f4c0a0800 2023-09-30T22:28:07.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.108+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f28f403f820 con 0x7f290009e000 2023-09-30T22:28:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.110+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f3c02e9e0 con 0x7f1f4c0a0800 2023-09-30T22:28:07.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.113+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28f401be10 con 0x7f290009e000 2023-09-30T22:28:07.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.275+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe4a012ab40 con 0x7fe4a009e410 2023-09-30T22:28:07.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.275+0000 7fe4877fe700 1 -- 172.21.15.32:0/3098791996 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe49801a380 con 0x7fe4a009e410 2023-09-30T22:28:07.277 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 msgr2=0x7fe4a0127e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 0x7fe4a0127e40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b680 tx=0x7fe498004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 shutdown_connections 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe4a009e410 0x7fe4a0127e40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a009d6f0 0x7fe4a01273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 --2- 172.21.15.32:0/3098791996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe4a009e000 0x7fe4a0127900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 >> 172.21.15.32:0/3098791996 conn(0x7fe4a0066f50 msgr2=0x7fe4a0070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 shutdown_connections 2023-09-30T22:28:07.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.278+0000 7fe4a5275700 1 -- 172.21.15.32:0/3098791996 wait complete. 2023-09-30T22:28:07.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.321+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2900071d00 con 0x7f290009e000 2023-09-30T22:28:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.322+0000 7f28e77fe700 1 -- 172.21.15.32:0/2850979034 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28f401b3b0 con 0x7f290009e000 2023-09-30T22:28:07.323 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 msgr2=0x7f2900122af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900122af0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f28f400e900 tx=0x7f28f400ec10 comp rx=0 tx=0).stop 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 shutdown_connections 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290009e410 0x7f2900123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f290009d6f0 0x7f29001225b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 --2- 172.21.15.32:0/2850979034 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f290009e000 0x7f2900122af0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.324+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 >> 172.21.15.32:0/2850979034 conn(0x7f2900066f50 msgr2=0x7f2900130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.325+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 shutdown_connections 2023-09-30T22:28:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.325+0000 7f2905311700 1 -- 172.21.15.32:0/2850979034 wait complete. 2023-09-30T22:28:07.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.345+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1f4c06caf0 con 0x7f1f4c0a0800 2023-09-30T22:28:07.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.345+0000 7f1f3affd700 1 -- 172.21.15.32:0/3334550822 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f3c015020 con 0x7f1f4c0a0800 2023-09-30T22:28:07.346 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:07.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.347+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c0a0800 msgr2=0x7f1f4c1c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.347+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c0a0800 0x7f1f4c1c8570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c016000 tx=0x7f1f3c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.347+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 shutdown_connections 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.347+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c0a1520 0x7f1f4c1c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.347+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c0a0800 0x7f1f4c1c8570 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.348+0000 7f1f4b59e700 1 --2- 172.21.15.32:0/3334550822 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c0a1110 0x7f1f4c1c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.348+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 >> 172.21.15.32:0/3334550822 conn(0x7f1f4c0a6140 msgr2=0x7f1f4c1bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.348+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 shutdown_connections 2023-09-30T22:28:07.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:07.348+0000 7f1f4b59e700 1 -- 172.21.15.32:0/3334550822 wait complete. 2023-09-30T22:28:07.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:08.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.236+0000 7fe1f1a7f700 1 Processor -- start 2023-09-30T22:28:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.236+0000 7fe1f1a7f700 1 -- start start 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.236+0000 7fe1f1a7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 0x7fe1ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.236+0000 7fe1f1a7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 0x7fe1ec134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.236+0000 7fe1f1a7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec09b120 con 0x7fe1ec09e000 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1f1a7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec06b460 con 0x7fe1ec09e410 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1f1a7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec09a7e0 con 0x7fe1ec09d6f0 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 0x7fe1ec134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 0x7fe1ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ebfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 0x7fe1ec134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42322/0 (socket says 172.21.15.32:42322) 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ebfff700 1 -- 172.21.15.32:0/3667806087 learned_addr learned my addr 172.21.15.32:0/3667806087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 --2- 172.21.15.32:0/3667806087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 0x7fe1ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55914/0 (socket says 172.21.15.32:55914) 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eaffd700 1 --1- 172.21.15.32:0/3667806087 >> v1:172.21.15.32:6789/0 conn(0x7fe1ec09e000 0x7fe1ec1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44486/0 (socket says 172.21.15.32:44486) 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 -- 172.21.15.32:0/3667806087 >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 msgr2=0x7fe1ec134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 --2- 172.21.15.32:0/3667806087 >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 0x7fe1ec134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 -- 172.21.15.32:0/3667806087 >> v1:172.21.15.32:6789/0 conn(0x7fe1ec09e000 legacy=0x7fe1ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 -- 172.21.15.32:0/3667806087 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec09a500 con 0x7fe1ec09d6f0 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ebfff700 1 --2- 172.21.15.32:0/3667806087 >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 0x7fe1ec134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1eb7fe700 1 --2- 172.21.15.32:0/3667806087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 0x7fe1ec09dac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009870 tx=0x7fe1dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ade95a92aee634e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ea7fc700 1 -- 172.21.15.32:0/3667806087 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc011830 con 0x7fe1ec09d6f0 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ea7fc700 1 -- 172.21.15.32:0/3667806087 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc011990 con 0x7fe1ec09d6f0 2023-09-30T22:28:08.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.237+0000 7fe1ea7fc700 1 -- 172.21.15.32:0/3667806087 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc011d20 con 0x7fe1ec09d6f0 2023-09-30T22:28:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/3667806087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 msgr2=0x7fe1ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/3667806087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 0x7fe1ec09dac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009870 tx=0x7fe1dc009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/3667806087 shutdown_connections 2023-09-30T22:28:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/3667806087 >> v2:172.21.15.32:3300/0 conn(0x7fe1ec09e410 0x7fe1ec134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/3667806087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09d6f0 0x7fe1ec09dac0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/3667806087 >> 172.21.15.32:0/3667806087 conn(0x7fe1ec066f50 msgr2=0x7fe1ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/3667806087 shutdown_connections 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.238+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/3667806087 wait complete. 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.239+0000 7fe1f1a7f700 1 Processor -- start 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.239+0000 7fe1f1a7f700 1 -- start start 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1f1a7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 0x7fe1ec1a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1f1a7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 0x7fe1ec1a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1f1a7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 0x7fe1ec1a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1f1a7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec1a9b00 con 0x7fe1ec09e000 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1f1a7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec1a81d0 con 0x7fe1ec09d6f0 2023-09-30T22:28:08.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1f1a7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec1a9290 con 0x7fe1ec09e410 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 0x7fe1ec1a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 0x7fe1ec1a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 0x7fe1ec1a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 0x7fe1ec1a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55922/0 (socket says 172.21.15.32:55922) 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 0x7fe1ec1a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58920/0 (socket says 172.21.15.32:58920) 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 0x7fe1ec1a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42324/0 (socket says 172.21.15.32:42324) 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1ebfff700 1 -- 172.21.15.32:0/2301450711 learned_addr learned my addr 172.21.15.32:0/2301450711 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 -- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 msgr2=0x7fe1ec1a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 0x7fe1ec1a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 -- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 msgr2=0x7fe1ec1a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 0x7fe1ec1a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 -- 172.21.15.32:0/2301450711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009b30 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1ebfff700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 0x7fe1ec1a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eb7fe700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 0x7fe1ec1a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.240+0000 7fe1eaffd700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 0x7fe1ec1a65b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400c450 tx=0x7fe1d400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.241+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4005cb0 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.241+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d4005e10 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.241+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0094a0 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.241+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40072d0 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.241+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec09c760 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.241+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1d4007430 con 0x7fe1ec09e000 2023-09-30T22:28:08.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.242+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe1d403ea20 con 0x7fe1ec09e000 2023-09-30T22:28:08.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.242+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec074ea0 con 0x7fe1ec09e000 2023-09-30T22:28:08.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.247+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1d4017070 con 0x7fe1ec09e000 2023-09-30T22:28:08.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:08.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:08.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:08.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.443+0000 7f8a1cd2d700 1 Processor -- start 2023-09-30T22:28:08.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.444+0000 7f8a1cd2d700 1 -- start start 2023-09-30T22:28:08.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.444+0000 7f8a1cd2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a1809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.444+0000 7f8a1cd2d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a1809dc30 0x7f8a1809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.444+0000 7f8a1cd2d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1809a810 con 0x7f8a1813c1e0 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.444+0000 7f8a1cd2d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1806ab40 con 0x7f8a1809dc30 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.444+0000 7f8a1cd2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1809ebc0 con 0x7f8a1806a770 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a1809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a1809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55924/0 (socket says 172.21.15.32:55924) 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 -- 172.21.15.32:0/1341565443 learned_addr learned my addr 172.21.15.32:0/1341565443 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a0ffff700 1 --2- 172.21.15.32:0/1341565443 >> v2:172.21.15.32:3300/0 conn(0x7f8a1809dc30 0x7f8a1809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 -- 172.21.15.32:0/1341565443 >> v2:172.21.15.32:3300/0 conn(0x7f8a1809dc30 msgr2=0x7f8a1809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 --2- 172.21.15.32:0/1341565443 >> v2:172.21.15.32:3300/0 conn(0x7f8a1809dc30 0x7f8a1809e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 -- 172.21.15.32:0/1341565443 >> v1:172.21.15.32:6789/0 conn(0x7f8a1813c1e0 legacy=0x7f8a1813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 -- 172.21.15.32:0/1341565443 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1809e920 con 0x7f8a1806a770 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a0ffff700 1 --2- 172.21.15.32:0/1341565443 >> v2:172.21.15.32:3300/0 conn(0x7f8a1809dc30 0x7f8a1809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a1659c700 1 --2- 172.21.15.32:0/1341565443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a1809d6f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009920 tx=0x7f8a08009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3cf8b836e660844 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.445+0000 7f8a15d9b700 1 -- 172.21.15.32:0/1341565443 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08011830 con 0x7f8a1806a770 2023-09-30T22:28:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.446+0000 7f8a15d9b700 1 -- 172.21.15.32:0/1341565443 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a08011990 con 0x7f8a1806a770 2023-09-30T22:28:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.446+0000 7f8a15d9b700 1 -- 172.21.15.32:0/1341565443 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08011d20 con 0x7f8a1806a770 2023-09-30T22:28:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/1341565443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 msgr2=0x7f8a1809d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/1341565443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a1809d6f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009920 tx=0x7f8a08009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/1341565443 shutdown_connections 2023-09-30T22:28:08.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/1341565443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a1809d6f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/1341565443 >> v2:172.21.15.32:3300/0 conn(0x7f8a1809dc30 0x7f8a1809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/1341565443 >> 172.21.15.32:0/1341565443 conn(0x7f8a18066f50 msgr2=0x7f8a1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/1341565443 shutdown_connections 2023-09-30T22:28:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.447+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/1341565443 wait complete. 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.448+0000 7f8a1cd2d700 1 Processor -- start 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.448+0000 7f8a1cd2d700 1 -- start start 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1cd2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a18127e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1cd2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 0x7f8a181cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1cd2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1813c1e0 0x7f8a181cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1cd2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18127750 con 0x7f8a1809dc30 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1cd2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1812aea0 con 0x7f8a1813c1e0 2023-09-30T22:28:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1cd2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18128e90 con 0x7f8a1806a770 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a18127e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 0x7f8a181cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a18127e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55932/0 (socket says 172.21.15.32:55932) 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 0x7f8a181cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42340/0 (socket says 172.21.15.32:42340) 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1659c700 1 -- 172.21.15.32:0/3300513832 learned_addr learned my addr 172.21.15.32:0/3300513832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a16d9d700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1813c1e0 0x7f8a181cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 -- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 msgr2=0x7f8a18127e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a18127e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 -- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1813c1e0 msgr2=0x7f8a181cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1813c1e0 0x7f8a181cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a0ffff700 1 -- 172.21.15.32:0/3300513832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000094a0 con 0x7f8a1809dc30 2023-09-30T22:28:08.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.449+0000 7f8a1659c700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a18127e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.450+0000 7f8a0ffff700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 0x7f8a181cc4e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9b0 tx=0x7f8a0000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.450+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000ee50 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.450+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0000c850 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.450+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a08009550 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.450+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000cbe0 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.450+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1809f1a0 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.451+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1812acd0 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.451+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a0000cd40 con 0x7f8a1809dc30 2023-09-30T22:28:08.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.451+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8a0003f720 con 0x7f8a1809dc30 2023-09-30T22:28:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.461+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a000232f0 con 0x7f8a1809dc30 2023-09-30T22:28:08.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.466+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe1ec1a94c0 con 0x7fe1ec09e000 2023-09-30T22:28:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.466+0000 7fe1e3fff700 1 -- 172.21.15.32:0/2301450711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1ec1a94c0 con 0x7fe1ec09e000 2023-09-30T22:28:08.467 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.468+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 msgr2=0x7fe1ec1a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.468+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 0x7fe1ec1a65b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400c450 tx=0x7fe1d400c760 comp rx=0 tx=0).stop 2023-09-30T22:28:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.469+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 shutdown_connections 2023-09-30T22:28:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.469+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1ec09e410 0x7fe1ec1a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.469+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1ec09d6f0 0x7fe1ec1a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.469+0000 7fe1f1a7f700 1 --2- 172.21.15.32:0/2301450711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1ec09e000 0x7fe1ec1a65b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.469+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 >> 172.21.15.32:0/2301450711 conn(0x7fe1ec066f50 msgr2=0x7fe1ec1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.470+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 shutdown_connections 2023-09-30T22:28:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.471+0000 7fe1f1a7f700 1 -- 172.21.15.32:0/2301450711 wait complete. 2023-09-30T22:28:08.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:08.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.498+0000 7fcb77733700 1 Processor -- start 2023-09-30T22:28:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.498+0000 7fcb77733700 1 -- start start 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb77733700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 0x7fcb7006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb77733700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb70131d10 0x7fcb70139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb77733700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7009ece0 con 0x7fcb7006b080 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb77733700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7009e760 con 0x7fcb70131d10 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb77733700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7009e040 con 0x7fcb7006a770 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb754cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 0x7fcb7006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb75cd0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb70131d10 0x7fcb70139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb74cce700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb7006b080 0x7fcb70131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44506/0 (socket says 172.21.15.32:44506) 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb754cf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 0x7fcb7006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58940/0 (socket says 172.21.15.32:58940) 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb75cd0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb70131d10 0x7fcb70139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42346/0 (socket says 172.21.15.32:42346) 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb74cce700 1 -- 172.21.15.32:0/237926852 learned_addr learned my addr 172.21.15.32:0/237926852 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb67fff700 1 -- 172.21.15.32:0/237926852 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454750646 0 0) 0x7fcb7009ece0 con 0x7fcb7006b080 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb67fff700 1 -- 172.21.15.32:0/237926852 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb5c0035b0 con 0x7fcb7006b080 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb754cf700 1 -- 172.21.15.32:0/237926852 >> v2:172.21.15.32:3300/0 conn(0x7fcb70131d10 msgr2=0x7fcb70139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb754cf700 1 --2- 172.21.15.32:0/237926852 >> v2:172.21.15.32:3300/0 conn(0x7fcb70131d10 0x7fcb70139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb754cf700 1 -- 172.21.15.32:0/237926852 >> v1:172.21.15.32:6789/0 conn(0x7fcb7006b080 legacy=0x7fcb70131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.499+0000 7fcb754cf700 1 -- 172.21.15.32:0/237926852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7009f630 con 0x7fcb7006a770 2023-09-30T22:28:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.500+0000 7fcb754cf700 1 --2- 172.21.15.32:0/237926852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 0x7fcb7006ab40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009840 tx=0x7fcb6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9e1c619a3ab749 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.500+0000 7fcb67fff700 1 -- 172.21.15.32:0/237926852 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c011830 con 0x7fcb7006a770 2023-09-30T22:28:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.500+0000 7fcb67fff700 1 -- 172.21.15.32:0/237926852 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c011990 con 0x7fcb7006a770 2023-09-30T22:28:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.500+0000 7fcb67fff700 1 -- 172.21.15.32:0/237926852 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c011d20 con 0x7fcb7006a770 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.500+0000 7fcb77733700 1 -- 172.21.15.32:0/237926852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 msgr2=0x7fcb7006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.500+0000 7fcb77733700 1 --2- 172.21.15.32:0/237926852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 0x7fcb7006ab40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009840 tx=0x7fcb6c009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.501+0000 7fcb77733700 1 -- 172.21.15.32:0/237926852 shutdown_connections 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.501+0000 7fcb77733700 1 --2- 172.21.15.32:0/237926852 >> v2:172.21.15.32:3300/0 conn(0x7fcb70131d10 0x7fcb70139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.501+0000 7fcb77733700 1 --2- 172.21.15.32:0/237926852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006a770 0x7fcb7006ab40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.501+0000 7fcb77733700 1 -- 172.21.15.32:0/237926852 >> 172.21.15.32:0/237926852 conn(0x7fcb70066f50 msgr2=0x7fcb7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.501+0000 7fcb77733700 1 -- 172.21.15.32:0/237926852 shutdown_connections 2023-09-30T22:28:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.501+0000 7fcb77733700 1 -- 172.21.15.32:0/237926852 wait complete. 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.502+0000 7fcb77733700 1 Processor -- start 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.502+0000 7fcb77733700 1 -- start start 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb77733700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7006a770 0x7fcb7009aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb77733700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006b080 0x7fcb7009b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb77733700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb70131d10 0x7fcb701c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb77733700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70126170 con 0x7fcb70131d10 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb77733700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7009aa90 con 0x7fcb7006b080 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb77733700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7006ed40 con 0x7fcb7006a770 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb74cce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006b080 0x7fcb7009b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb754cf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7006a770 0x7fcb7009aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb74cce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006b080 0x7fcb7009b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58944/0 (socket says 172.21.15.32:58944) 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb74cce700 1 -- 172.21.15.32:0/3954464009 learned_addr learned my addr 172.21.15.32:0/3954464009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.503+0000 7fcb75cd0700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb70131d10 0x7fcb701c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb754cf700 1 -- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006b080 msgr2=0x7fcb7009b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb754cf700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006b080 0x7fcb7009b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb754cf700 1 -- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb70131d10 msgr2=0x7fcb701c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb754cf700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb70131d10 0x7fcb701c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb754cf700 1 -- 172.21.15.32:0/3954464009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c0094a0 con 0x7fcb7006a770 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb754cf700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7006a770 0x7fcb7009aec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c016000 tx=0x7fcb6c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c004590 con 0x7fcb7006a770 2023-09-30T22:28:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c01f070 con 0x7fcb7006a770 2023-09-30T22:28:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb70071e60 con 0x7fcb7006a770 2023-09-30T22:28:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.504+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0038e0 con 0x7fcb7006a770 2023-09-30T22:28:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.505+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb70126800 con 0x7fcb7006a770 2023-09-30T22:28:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.505+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb6c003a40 con 0x7fcb7006a770 2023-09-30T22:28:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.506+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70125fb0 con 0x7fcb7006a770 2023-09-30T22:28:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.506+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb6c021070 con 0x7fcb7006a770 2023-09-30T22:28:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.511+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb70125fb0 con 0x7fcb7006a770 2023-09-30T22:28:08.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.535+0000 7f6c091d3700 1 Processor -- start 2023-09-30T22:28:08.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 -- start start 2023-09-30T22:28:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409e850 0x7f6c0409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409f160 0x7f6c04133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 0x7f6c04134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0406bf30 con 0x7f6c0409f5b0 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0409ffb0 con 0x7f6c0409e850 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c091d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0406cb10 con 0x7f6c0409f160 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c0359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 0x7f6c04134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c0259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409f160 0x7f6c04133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409e850 0x7f6c0409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c0359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 0x7f6c04134130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42356/0 (socket says 172.21.15.32:42356) 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.536+0000 7f6c0359e700 1 -- 172.21.15.32:0/482138585 learned_addr learned my addr 172.21.15.32:0/482138585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0259c700 1 -- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409e850 msgr2=0x7f6c0409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0259c700 1 --2- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409e850 0x7f6c0409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0259c700 1 -- 172.21.15.32:0/482138585 >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 msgr2=0x7f6c04134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0259c700 1 --2- 172.21.15.32:0/482138585 >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 0x7f6c04134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0259c700 1 -- 172.21.15.32:0/482138585 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040a0270 con 0x7f6c0409f160 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0359e700 1 --2- 172.21.15.32:0/482138585 >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 0x7f6c04134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c0259c700 1 --2- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409f160 0x7f6c04133bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009840 tx=0x7f6bf8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ad5208d316b9c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c01d9b700 1 -- 172.21.15.32:0/482138585 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8011830 con 0x7f6c0409f160 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c01d9b700 1 -- 172.21.15.32:0/482138585 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8011990 con 0x7f6c0409f160 2023-09-30T22:28:08.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.537+0000 7f6c01d9b700 1 -- 172.21.15.32:0/482138585 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8011d20 con 0x7f6c0409f160 2023-09-30T22:28:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 -- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409f160 msgr2=0x7f6c04133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 --2- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409f160 0x7f6c04133bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009840 tx=0x7f6bf8009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 -- 172.21.15.32:0/482138585 shutdown_connections 2023-09-30T22:28:08.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 --2- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409e850 0x7f6c0409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 --2- 172.21.15.32:0/482138585 >> v2:172.21.15.32:3300/0 conn(0x7f6c0409f5b0 0x7f6c04134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 --2- 172.21.15.32:0/482138585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409f160 0x7f6c04133bf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 -- 172.21.15.32:0/482138585 >> 172.21.15.32:0/482138585 conn(0x7f6c040a3f70 msgr2=0x7f6c040a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 -- 172.21.15.32:0/482138585 shutdown_connections 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.538+0000 7f6c091d3700 1 -- 172.21.15.32:0/482138585 wait complete. 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.539+0000 7f6c091d3700 1 Processor -- start 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.539+0000 7f6c091d3700 1 -- start start 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c091d3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 0x7f6c04122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c091d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 0x7f6c04122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c091d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c0409f5b0 0x7f6c041230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c091d3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04125bf0 con 0x7f6c0409f5b0 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c091d3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04124ac0 con 0x7f6c0409f160 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c091d3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0406f810 con 0x7f6c0409e850 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c0259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 0x7f6c04122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 0x7f6c04122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 0x7f6c04122670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55974/0 (socket says 172.21.15.32:55974) 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c0259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 0x7f6c04122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58968/0 (socket says 172.21.15.32:58968) 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 -- 172.21.15.32:0/1119957839 learned_addr learned my addr 172.21.15.32:0/1119957839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:08.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c0359e700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c0409f5b0 0x7f6c041230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 -- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 msgr2=0x7f6c04122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 0x7f6c04122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 -- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c0409f5b0 msgr2=0x7f6c041230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c0409f5b0 0x7f6c041230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c02d9d700 1 -- 172.21.15.32:0/1119957839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc009b30 con 0x7f6c0409e850 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.540+0000 7f6c0259c700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 0x7f6c04122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.541+0000 7f6c02d9d700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 0x7f6c04122670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400e9b0 tx=0x7f6bf400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.541+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4009710 con 0x7f6c0409e850 2023-09-30T22:28:08.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.541+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf400ee50 con 0x7f6c0409e850 2023-09-30T22:28:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.541+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400ca50 con 0x7f6c0409e850 2023-09-30T22:28:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.541+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf80094a0 con 0x7f6c0409e850 2023-09-30T22:28:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.541+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c04125eb0 con 0x7f6c0409e850 2023-09-30T22:28:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.542+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bf400cbb0 con 0x7f6c0409e850 2023-09-30T22:28:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.542+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6bf403fa90 con 0x7f6c0409e850 2023-09-30T22:28:08.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.542+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c040747f0 con 0x7f6c0409e850 2023-09-30T22:28:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.547+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bf401b3b0 con 0x7f6c0409e850 2023-09-30T22:28:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.676+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a18074ea0 con 0x7f8a1809dc30 2023-09-30T22:28:08.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.676+0000 7f8a0effd700 1 -- 172.21.15.32:0/3300513832 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a00021030 con 0x7f8a1809dc30 2023-09-30T22:28:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 msgr2=0x7f8a181cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 0x7f8a181cc4e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9b0 tx=0x7f8a0000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 shutdown_connections 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8a1813c1e0 0x7f8a181cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8a1806a770 0x7f8a18127e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 --2- 172.21.15.32:0/3300513832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1809dc30 0x7f8a181cc4e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 >> 172.21.15.32:0/3300513832 conn(0x7f8a18066f50 msgr2=0x7f8a18070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 shutdown_connections 2023-09-30T22:28:08.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.679+0000 7f8a1cd2d700 1 -- 172.21.15.32:0/3300513832 wait complete. 2023-09-30T22:28:08.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:08.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.717+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb70125fb0 con 0x7fcb7006a770 2023-09-30T22:28:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.718+0000 7fcb65ffb700 1 -- 172.21.15.32:0/3954464009 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb70125fb0 con 0x7fcb7006a770 2023-09-30T22:28:08.719 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:08.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7006a770 msgr2=0x7fcb7009aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7006a770 0x7fcb7009aec0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c016000 tx=0x7fcb6c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 shutdown_connections 2023-09-30T22:28:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb70131d10 0x7fcb701c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7006a770 0x7fcb7009aec0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 --2- 172.21.15.32:0/3954464009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7006b080 0x7fcb7009b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 >> 172.21.15.32:0/3954464009 conn(0x7fcb70066f50 msgr2=0x7fcb70070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 shutdown_connections 2023-09-30T22:28:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.720+0000 7fcb77733700 1 -- 172.21.15.32:0/3954464009 wait complete. 2023-09-30T22:28:08.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.756+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6c0406b9e0 con 0x7f6c0409e850 2023-09-30T22:28:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.757+0000 7f6beb7fe700 1 -- 172.21.15.32:0/1119957839 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bf4017020 con 0x7f6c0409e850 2023-09-30T22:28:08.758 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 msgr2=0x7f6c04122670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 0x7f6c04122670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400e9b0 tx=0x7f6bf400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 shutdown_connections 2023-09-30T22:28:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c0409f5b0 0x7f6c041230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c0409e850 0x7f6c04122670 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 --2- 172.21.15.32:0/1119957839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c0409f160 0x7f6c04122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 >> 172.21.15.32:0/1119957839 conn(0x7f6c040a3f70 msgr2=0x7f6c040a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 shutdown_connections 2023-09-30T22:28:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:08.759+0000 7f6c091d3700 1 -- 172.21.15.32:0/1119957839 wait complete. 2023-09-30T22:28:08.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:09.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.639+0000 7f3aeace5700 1 Processor -- start 2023-09-30T22:28:09.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.639+0000 7f3aeace5700 1 -- start start 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.639+0000 7f3aeace5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409d6f0 0x7f3ae409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.639+0000 7f3aeace5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ae409e410 0x7f3ae4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.639+0000 7f3aeace5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae409b120 con 0x7f3ae409e000 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3aeace5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae406b460 con 0x7f3ae409e410 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3aeace5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae409a7e0 con 0x7f3ae409d6f0 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae8a81700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409d6f0 0x7f3ae409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae9282700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ae409e410 0x7f3ae4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3ae409e000 0x7f3ae41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44508/0 (socket says 172.21.15.32:44508) 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae3fff700 1 -- 172.21.15.32:0/860945816 learned_addr learned my addr 172.21.15.32:0/860945816 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:09.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae9282700 1 -- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409d6f0 msgr2=0x7f3ae409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae9282700 1 --2- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409d6f0 0x7f3ae409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae9282700 1 -- 172.21.15.32:0/860945816 >> v1:172.21.15.32:6789/0 conn(0x7f3ae409e000 legacy=0x7f3ae41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae9282700 1 -- 172.21.15.32:0/860945816 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae409a500 con 0x7f3ae409e410 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae8a81700 1 --2- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409d6f0 0x7f3ae409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae37fe700 1 -- 172.21.15.32:0/860945816 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900840262 0 0) 0x7f3ae409b120 con 0x7f3ae409e000 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae9282700 1 --2- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae4134100 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d90 tx=0x7f3adc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2145515e182222 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae37fe700 1 -- 172.21.15.32:0/860945816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00e070 con 0x7f3ae409e410 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae37fe700 1 -- 172.21.15.32:0/860945816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00bb20 con 0x7f3ae409e410 2023-09-30T22:28:09.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.640+0000 7f3ae37fe700 1 -- 172.21.15.32:0/860945816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc012430 con 0x7f3ae409e410 2023-09-30T22:28:09.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 -- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 msgr2=0x7f3ae4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 --2- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae4134100 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d90 tx=0x7f3adc00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 -- 172.21.15.32:0/860945816 shutdown_connections 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 --2- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae4134100 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 --2- 172.21.15.32:0/860945816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409d6f0 0x7f3ae409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 -- 172.21.15.32:0/860945816 >> 172.21.15.32:0/860945816 conn(0x7f3ae4066f50 msgr2=0x7f3ae409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 -- 172.21.15.32:0/860945816 shutdown_connections 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.641+0000 7f3aeace5700 1 -- 172.21.15.32:0/860945816 wait complete. 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.642+0000 7f3aeace5700 1 Processor -- start 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.642+0000 7f3aeace5700 1 -- start start 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3aeace5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae409d6f0 0x7f3ae41c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3aeace5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409e000 0x7f3ae41c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3aeace5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae41c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3aeace5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41a8a30 con 0x7f3ae409e410 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3aeace5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41a9780 con 0x7f3ae409e000 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3aeace5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41c8e80 con 0x7f3ae409d6f0 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae8a81700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae409d6f0 0x7f3ae41c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae41c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae8a81700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae409d6f0 0x7f3ae41c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55980/0 (socket says 172.21.15.32:55980) 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae8a81700 1 -- 172.21.15.32:0/2009057872 learned_addr learned my addr 172.21.15.32:0/2009057872 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae3fff700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409e000 0x7f3ae41c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 -- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae409d6f0 msgr2=0x7f3ae41c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae409d6f0 0x7f3ae41c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 -- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409e000 msgr2=0x7f3ae41c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409e000 0x7f3ae41c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 -- 172.21.15.32:0/2009057872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80094a0 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.643+0000 7f3ae9282700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae41c49d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d90 tx=0x7f3adc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.644+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc012430 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.644+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc020070 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.644+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc009a70 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.644+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc01ba30 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.644+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae41a85e0 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.645+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3adc01bb90 con 0x7f3ae409e410 2023-09-30T22:28:09.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.645+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae4074ea0 con 0x7f3ae409e410 2023-09-30T22:28:09.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.645+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3adc022070 con 0x7f3ae409e410 2023-09-30T22:28:09.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.650+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3adc01be40 con 0x7f3ae409e410 2023-09-30T22:28:09.695 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:09.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:09.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:09.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.849+0000 7f3becf50700 1 Processor -- start 2023-09-30T22:28:09.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.849+0000 7f3becf50700 1 -- start start 2023-09-30T22:28:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.850+0000 7f3becf50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.850+0000 7f3becf50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8138dc0 0x7f3be813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.850+0000 7f3becf50700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be806b990 con 0x7f3be8133420 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.850+0000 7f3becf50700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be809b0e0 con 0x7f3be8138dc0 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3becf50700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8133c20 con 0x7f3be81337f0 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3be8133420 0x7f3be8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44518/0 (socket says 172.21.15.32:44518) 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be659c700 1 -- 172.21.15.32:0/3561738963 learned_addr learned my addr 172.21.15.32:0/3561738963 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8138dc0 0x7f3be813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be5d9b700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be559a700 1 -- 172.21.15.32:0/3561738963 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451977119 0 0) 0x7f3be806b990 con 0x7f3be8133420 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be6d9d700 1 -- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 msgr2=0x7f3be8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be6d9d700 1 -- 172.21.15.32:0/3561738963 >> v1:172.21.15.32:6789/0 conn(0x7f3be8133420 legacy=0x7f3be8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be6d9d700 1 -- 172.21.15.32:0/3561738963 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80a0590 con 0x7f3be8138dc0 2023-09-30T22:28:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be5d9b700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.851+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8138dc0 0x7f3be813b1a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009dc0 tx=0x7f3be000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe94de37c4e8f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.852+0000 7f3be559a700 1 -- 172.21.15.32:0/3561738963 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000e070 con 0x7f3be8138dc0 2023-09-30T22:28:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.852+0000 7f3be559a700 1 -- 172.21.15.32:0/3561738963 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000bb20 con 0x7f3be8138dc0 2023-09-30T22:28:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.852+0000 7f3be559a700 1 -- 172.21.15.32:0/3561738963 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0012430 con 0x7f3be8138dc0 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 -- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8138dc0 msgr2=0x7f3be813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8138dc0 0x7f3be813b1a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009dc0 tx=0x7f3be000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 -- 172.21.15.32:0/3561738963 shutdown_connections 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8138dc0 0x7f3be813b1a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 --2- 172.21.15.32:0/3561738963 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 -- 172.21.15.32:0/3561738963 >> 172.21.15.32:0/3561738963 conn(0x7f3be80a4210 msgr2=0x7f3be80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 -- 172.21.15.32:0/3561738963 shutdown_connections 2023-09-30T22:28:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.853+0000 7f3becf50700 1 -- 172.21.15.32:0/3561738963 wait complete. 2023-09-30T22:28:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.855+0000 7f3becf50700 1 Processor -- start 2023-09-30T22:28:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.855+0000 7f3becf50700 1 -- start start 2023-09-30T22:28:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3becf50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 0x7f3be80709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3becf50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3becf50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be8138dc0 0x7f3be8071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3becf50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be809b800 con 0x7f3be8138dc0 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3becf50700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be809c6d0 con 0x7f3be8133420 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3becf50700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3be81388a0 con 0x7f3be81337f0 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 0x7f3be80709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 0x7f3be80709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59014/0 (socket says 172.21.15.32:59014) 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be659c700 1 -- 172.21.15.32:0/1082741897 learned_addr learned my addr 172.21.15.32:0/1082741897 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be8138dc0 0x7f3be8071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be6d9d700 1 -- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 msgr2=0x7f3be8070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be6d9d700 1 -- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 msgr2=0x7f3be80709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 0x7f3be80709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be6d9d700 1 -- 172.21.15.32:0/1082741897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0094a0 con 0x7f3be8138dc0 2023-09-30T22:28:09.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.856+0000 7f3be659c700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 0x7f3be80709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:09.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.857+0000 7f3be6d9d700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be8138dc0 0x7f3be8071420 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3be0016000 tx=0x7f3be0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.857+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0012430 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.857+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0020070 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.857+0000 7f3becf50700 1 -- 172.21.15.32:0/1082741897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be0009a70 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.857+0000 7f3becf50700 1 -- 172.21.15.32:0/1082741897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be809c990 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.857+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0004280 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.858+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be8074ea0 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.859+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3be00043e0 con 0x7f3be8138dc0 2023-09-30T22:28:09.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.859+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3be0022070 con 0x7f3be8138dc0 2023-09-30T22:28:09.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.864+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3be001a380 con 0x7f3be8138dc0 2023-09-30T22:28:09.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.874+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3ae41a7870 con 0x7f3ae409e410 2023-09-30T22:28:09.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.874+0000 7f3ae17fa700 1 -- 172.21.15.32:0/2009057872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3adc025b70 con 0x7f3ae409e410 2023-09-30T22:28:09.875 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:09.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 msgr2=0x7f3ae41c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae41c49d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d90 tx=0x7f3adc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 shutdown_connections 2023-09-30T22:28:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae409e410 0x7f3ae41c49d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae409d6f0 0x7f3ae41c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 --2- 172.21.15.32:0/2009057872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae409e000 0x7f3ae41c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.878+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 >> 172.21.15.32:0/2009057872 conn(0x7f3ae4066f50 msgr2=0x7f3ae41bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:09.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.879+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 shutdown_connections 2023-09-30T22:28:09.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.879+0000 7f3aeace5700 1 -- 172.21.15.32:0/2009057872 wait complete. 2023-09-30T22:28:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.892+0000 7f0a03946700 1 Processor -- start 2023-09-30T22:28:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.893+0000 7f0a03946700 1 -- start start 2023-09-30T22:28:09.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a03946700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09fc1316d0 0x7f09fc131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a03946700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc131fe0 0x7f09fc06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a03946700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a03946700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc09a5f0 con 0x7f09fc1316d0 2023-09-30T22:28:09.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a03946700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc131510 con 0x7f09fc131fe0 2023-09-30T22:28:09.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a03946700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc099e70 con 0x7f09fc06ea90 2023-09-30T22:28:09.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09fc1316d0 0x7f09fc131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09fc1316d0 0x7f09fc131aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42408/0 (socket says 172.21.15.32:42408) 2023-09-30T22:28:09.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 -- 172.21.15.32:0/584911789 learned_addr learned my addr 172.21.15.32:0/584911789 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:09.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a00ee1700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc131fe0 0x7f09fc06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a01ee3700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 -- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 msgr2=0x7f09fc06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc06eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 -- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc131fe0 msgr2=0x7f09fc06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc131fe0 0x7f09fc06e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a016e2700 1 -- 172.21.15.32:0/584911789 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc066680 con 0x7f09fc1316d0 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.894+0000 7f0a00ee1700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc131fe0 0x7f09fc06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.895+0000 7f0a016e2700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc1316d0 0x7f09fc131aa0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f09f80098f0 tx=0x7f09f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d511ca25b1bcaf9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.895+0000 7f09f3fff700 1 -- 172.21.15.32:0/584911789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011830 con 0x7f09fc1316d0 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.895+0000 7f09f3fff700 1 -- 172.21.15.32:0/584911789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8011990 con 0x7f09fc1316d0 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.895+0000 7f09f3fff700 1 -- 172.21.15.32:0/584911789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011d20 con 0x7f09fc1316d0 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 -- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc1316d0 msgr2=0x7f09fc131aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc1316d0 0x7f09fc131aa0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f09f80098f0 tx=0x7f09f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 -- 172.21.15.32:0/584911789 shutdown_connections 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc06eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc1316d0 0x7f09fc131aa0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 --2- 172.21.15.32:0/584911789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc131fe0 0x7f09fc06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 -- 172.21.15.32:0/584911789 >> 172.21.15.32:0/584911789 conn(0x7f09fc0a2aa0 msgr2=0x7f09fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 -- 172.21.15.32:0/584911789 shutdown_connections 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.896+0000 7f0a03946700 1 -- 172.21.15.32:0/584911789 wait complete. 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 Processor -- start 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 -- start start 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc11a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc1316d0 0x7f09fc11aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc131fe0 0x7f09fc11b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc11e020 con 0x7f09fc131fe0 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc11d7b0 con 0x7f09fc1316d0 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a03946700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc11d360 con 0x7f09fc06ea90 2023-09-30T22:28:09.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a016e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc11a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a016e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc11a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56006/0 (socket says 172.21.15.32:56006) 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a016e2700 1 -- 172.21.15.32:0/10719562 learned_addr learned my addr 172.21.15.32:0/10719562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.897+0000 7f0a01ee3700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc131fe0 0x7f09fc11b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a00ee1700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc1316d0 0x7f09fc11aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a016e2700 1 -- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc1316d0 msgr2=0x7f09fc11aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a016e2700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc1316d0 0x7f09fc11aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a016e2700 1 -- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc131fe0 msgr2=0x7f09fc11b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a016e2700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc131fe0 0x7f09fc11b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a016e2700 1 -- 172.21.15.32:0/10719562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0094a0 con 0x7f09fc06ea90 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a016e2700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc11a590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000c00 tx=0x7f09f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8004300 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f801f070 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f8009550 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.898+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80036a0 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.899+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc09a0b0 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.899+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09f8021030 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.900+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f09f8040760 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.900+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc09cf10 con 0x7f09fc06ea90 2023-09-30T22:28:09.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.905+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09f80199a0 con 0x7f09fc06ea90 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.936+0000 7f9961fde700 1 Processor -- start 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.936+0000 7f9961fde700 1 -- start start 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f9961fde700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f9961fde700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f9961fde700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995c0a2a70 con 0x7f995c133930 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f9961fde700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0a01d0 con 0x7f995c0a0a30 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f9961fde700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f995c09e2b0 con 0x7f995c09d630 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f995c133930 0x7f995c133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44528/0 (socket says 172.21.15.32:44528) 2023-09-30T22:28:09.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995bfff700 1 -- 172.21.15.32:0/3034592346 learned_addr learned my addr 172.21.15.32:0/3034592346 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995b7fe700 1 --2- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995affd700 1 -- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 msgr2=0x7f995c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995affd700 1 --2- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995affd700 1 -- 172.21.15.32:0/3034592346 >> v1:172.21.15.32:6789/0 conn(0x7f995c133930 legacy=0x7f995c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995a7fc700 1 -- 172.21.15.32:0/3034592346 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718507211 0 0) 0x7f995c0a2a70 con 0x7f995c133930 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.937+0000 7f995affd700 1 -- 172.21.15.32:0/3034592346 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c09dc50 con 0x7f995c09d630 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.938+0000 7f995affd700 1 --2- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c0a1340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9950009870 tx=0x7f9950009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961f3f25625a53a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.938+0000 7f995a7fc700 1 -- 172.21.15.32:0/3034592346 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950011830 con 0x7f995c09d630 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.938+0000 7f995a7fc700 1 -- 172.21.15.32:0/3034592346 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950011990 con 0x7f995c09d630 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.938+0000 7f995a7fc700 1 -- 172.21.15.32:0/3034592346 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950011d20 con 0x7f995c09d630 2023-09-30T22:28:09.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.938+0000 7f9961fde700 1 -- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 msgr2=0x7f995c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.938+0000 7f9961fde700 1 --2- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c0a1340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9950009870 tx=0x7f9950009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.939+0000 7f9961fde700 1 -- 172.21.15.32:0/3034592346 shutdown_connections 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.939+0000 7f9961fde700 1 --2- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.939+0000 7f9961fde700 1 --2- 172.21.15.32:0/3034592346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c0a1340 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.939+0000 7f9961fde700 1 -- 172.21.15.32:0/3034592346 >> 172.21.15.32:0/3034592346 conn(0x7f995c0a3050 msgr2=0x7f995c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.939+0000 7f9961fde700 1 -- 172.21.15.32:0/3034592346 shutdown_connections 2023-09-30T22:28:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.939+0000 7f9961fde700 1 -- 172.21.15.32:0/3034592346 wait complete. 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 Processor -- start 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 -- start start 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c11e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c11ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f995c133930 0x7f995c11f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c1205a0 con 0x7f995c133930 2023-09-30T22:28:09.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0a4350 con 0x7f995c0a0a30 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f9961fde700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f995c122050 con 0x7f995c09d630 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f995affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c11ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f995affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c11ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59042/0 (socket says 172.21.15.32:59042) 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f995affd700 1 -- 172.21.15.32:0/660330743 learned_addr learned my addr 172.21.15.32:0/660330743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.940+0000 7f995bfff700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f995c133930 0x7f995c11f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995b7fe700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c11e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995affd700 1 -- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 msgr2=0x7f995c11e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995affd700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c11e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995affd700 1 -- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f995c133930 msgr2=0x7f995c11f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995affd700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f995c133930 0x7f995c11f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995affd700 1 -- 172.21.15.32:0/660330743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500094a0 con 0x7f995c0a0a30 2023-09-30T22:28:09.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995bfff700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f995c133930 0x7f995c11f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f995affd700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c11ed50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9950000c00 tx=0x7f9950004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99500044d0 con 0x7f995c0a0a30 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99500036a0 con 0x7f995c0a0a30 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c120c10 con 0x7f995c0a0a30 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950003a50 con 0x7f995c0a0a30 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.941+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c122a30 con 0x7f995c0a0a30 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.942+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c0008d0 con 0x7f995c0a0a30 2023-09-30T22:28:09.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.943+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9950021070 con 0x7f995c0a0a30 2023-09-30T22:28:09.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.943+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9950040bc0 con 0x7f995c0a0a30 2023-09-30T22:28:09.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:09.948+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f995002d760 con 0x7f995c0a0a30 2023-09-30T22:28:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.077+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3be806b250 con 0x7f3be8138dc0 2023-09-30T22:28:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.078+0000 7f3bd6ffd700 1 -- 172.21.15.32:0/1082741897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3be0011b80 con 0x7f3be8138dc0 2023-09-30T22:28:10.079 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:10.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be8138dc0 msgr2=0x7f3be8071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:10.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be8138dc0 0x7f3be8071420 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3be0016000 tx=0x7f3be0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 shutdown_connections 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be8138dc0 0x7f3be8071420 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be8133420 0x7f3be80709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 --2- 172.21.15.32:0/1082741897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be81337f0 0x7f3be8070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 >> 172.21.15.32:0/1082741897 conn(0x7f3be80a4210 msgr2=0x7f3be80a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 shutdown_connections 2023-09-30T22:28:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.080+0000 7f3bd4ff9700 1 -- 172.21.15.32:0/1082741897 wait complete. 2023-09-30T22:28:10.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.138+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f09fc074ea0 con 0x7f09fc06ea90 2023-09-30T22:28:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.138+0000 7f09f1ffb700 1 -- 172.21.15.32:0/10719562 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09f8015020 con 0x7f09fc06ea90 2023-09-30T22:28:10.139 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 msgr2=0x7f09fc11a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc11a590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000c00 tx=0x7f09f8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 shutdown_connections 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09fc131fe0 0x7f09fc11b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09fc06ea90 0x7f09fc11a590 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 --2- 172.21.15.32:0/10719562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09fc1316d0 0x7f09fc11aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 >> 172.21.15.32:0/10719562 conn(0x7f09fc0a2aa0 msgr2=0x7f09fc1b8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 shutdown_connections 2023-09-30T22:28:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.141+0000 7f0a03946700 1 -- 172.21.15.32:0/10719562 wait complete. 2023-09-30T22:28:10.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:10.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.175+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f995c072980 con 0x7f995c0a0a30 2023-09-30T22:28:10.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.176+0000 7f9943fff700 1 -- 172.21.15.32:0/660330743 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9950029020 con 0x7f995c0a0a30 2023-09-30T22:28:10.177 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 msgr2=0x7f995c11ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c11ed50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9950000c00 tx=0x7f9950004750 comp rx=0 tx=0).stop 2023-09-30T22:28:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 shutdown_connections 2023-09-30T22:28:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f995c133930 0x7f995c11f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f995c09d630 0x7f995c11e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 --2- 172.21.15.32:0/660330743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f995c0a0a30 0x7f995c11ed50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 >> 172.21.15.32:0/660330743 conn(0x7f995c0a3050 msgr2=0x7f995c1bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 shutdown_connections 2023-09-30T22:28:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:10.178+0000 7f9961fde700 1 -- 172.21.15.32:0/660330743 wait complete. 2023-09-30T22:28:10.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:10.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.043+0000 7fa01b1d8700 1 Processor -- start 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.043+0000 7fa01b1d8700 1 -- start start 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa01b1d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa014135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa01b1d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014135e60 0x7fa014138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa01b1d8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0140716a0 con 0x7fa014130540 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa01b1d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa014071290 con 0x7fa014133540 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa01b1d8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa014130340 con 0x7fa014135e60 2023-09-30T22:28:11.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa019775700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014135e60 0x7fa014138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa013fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa014135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa018f74700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa014130540 0x7fa014130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41666/0 (socket says 172.21.15.32:41666) 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa018f74700 1 -- 172.21.15.32:0/4068600801 learned_addr learned my addr 172.21.15.32:0/4068600801 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa013fff700 1 -- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014135e60 msgr2=0x7fa014138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa0137fe700 1 -- 172.21.15.32:0/4068600801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643240194 0 0) 0x7fa0140716a0 con 0x7fa014130540 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa013fff700 1 --2- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014135e60 0x7fa014138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa013fff700 1 -- 172.21.15.32:0/4068600801 >> v1:172.21.15.32:6789/0 conn(0x7fa014130540 legacy=0x7fa014130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa013fff700 1 -- 172.21.15.32:0/4068600801 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014070b80 con 0x7fa014133540 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.044+0000 7fa019775700 1 --2- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014135e60 0x7fa014138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa013fff700 1 --2- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa014135920 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009870 tx=0x7f9ffc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b988b6b81707c4b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa0137fe700 1 -- 172.21.15.32:0/4068600801 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc011830 con 0x7fa014133540 2023-09-30T22:28:11.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa0137fe700 1 -- 172.21.15.32:0/4068600801 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc011990 con 0x7fa014133540 2023-09-30T22:28:11.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa0137fe700 1 -- 172.21.15.32:0/4068600801 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc011d20 con 0x7fa014133540 2023-09-30T22:28:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa01b1d8700 1 -- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 msgr2=0x7fa014135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa014135920 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009870 tx=0x7f9ffc009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa01b1d8700 1 -- 172.21.15.32:0/4068600801 shutdown_connections 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014135e60 0x7fa014138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.045+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/4068600801 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa014135920 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.046+0000 7fa01b1d8700 1 -- 172.21.15.32:0/4068600801 >> 172.21.15.32:0/4068600801 conn(0x7fa0140a2aa0 msgr2=0x7fa0140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.046+0000 7fa01b1d8700 1 -- 172.21.15.32:0/4068600801 shutdown_connections 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.046+0000 7fa01b1d8700 1 -- 172.21.15.32:0/4068600801 wait complete. 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.046+0000 7fa01b1d8700 1 Processor -- start 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 -- start start 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014130540 0x7fa0141c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa0141c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa014135e60 0x7fa0141c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01411a9e0 con 0x7fa014135e60 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140718e0 con 0x7fa014133540 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa01b1d8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa014132e50 con 0x7fa014130540 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa013fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa0141c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa019775700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa014135e60 0x7fa0141c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa018f74700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014130540 0x7fa0141c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa013fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa0141c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55358/0 (socket says 172.21.15.32:55358) 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa013fff700 1 -- 172.21.15.32:0/2362562690 learned_addr learned my addr 172.21.15.32:0/2362562690 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.047+0000 7fa019775700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa014135e60 0x7fa0141c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53892/0 (socket says 172.21.15.32:53892) 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa013fff700 1 -- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014130540 msgr2=0x7fa0141c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa013fff700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014130540 0x7fa0141c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa013fff700 1 -- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa014135e60 msgr2=0x7fa0141c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa013fff700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa014135e60 0x7fa0141c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa013fff700 1 -- 172.21.15.32:0/2362562690 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c009a70 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa018f74700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014130540 0x7fa0141c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa013fff700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa0141c65a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009840 tx=0x7f9ffc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc004240 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc01f070 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc0094a0 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0045c0 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.048+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01411c960 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.049+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ffc021030 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.049+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0140a1ad0 con 0x7fa014133540 2023-09-30T22:28:11.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.049+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9ffc040620 con 0x7fa014133540 2023-09-30T22:28:11.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.054+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ffc0197f0 con 0x7fa014133540 2023-09-30T22:28:11.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:11.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:11.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:11.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.245+0000 7fef9f089700 1 Processor -- start 2023-09-30T22:28:11.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.245+0000 7fef9f089700 1 -- start start 2023-09-30T22:28:11.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9f089700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef9809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9f089700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef98131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9f089700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef98070940 con 0x7fef9809f240 2023-09-30T22:28:11.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9f089700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9806fd60 con 0x7fef9809f650 2023-09-30T22:28:11.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9f089700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef9806fa80 con 0x7fef9809e930 2023-09-30T22:28:11.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9ce25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef9809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef97fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fef9809f240 0x7fef981317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41680/0 (socket says 172.21.15.32:41680) 2023-09-30T22:28:11.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef9d626700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef98131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.246+0000 7fef97fff700 1 -- 172.21.15.32:0/114521467 learned_addr learned my addr 172.21.15.32:0/114521467 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:11.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef9ce25700 1 -- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 msgr2=0x7fef98131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:11.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef9ce25700 1 --2- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef98131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef9ce25700 1 -- 172.21.15.32:0/114521467 >> v1:172.21.15.32:6789/0 conn(0x7fef9809f240 legacy=0x7fef981317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef9ce25700 1 -- 172.21.15.32:0/114521467 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9809fb60 con 0x7fef9809e930 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef9ce25700 1 --2- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef9809ed00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb3b1cbffee9120 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef977fe700 1 -- 172.21.15.32:0/114521467 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c004230 con 0x7fef9809e930 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef977fe700 1 -- 172.21.15.32:0/114521467 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c004390 con 0x7fef9809e930 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.247+0000 7fef977fe700 1 -- 172.21.15.32:0/114521467 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0036a0 con 0x7fef9809e930 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 -- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 msgr2=0x7fef9809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 --2- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef9809ed00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 -- 172.21.15.32:0/114521467 shutdown_connections 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 --2- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef98131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 --2- 172.21.15.32:0/114521467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef9809ed00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 -- 172.21.15.32:0/114521467 >> 172.21.15.32:0/114521467 conn(0x7fef980a4210 msgr2=0x7fef980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 -- 172.21.15.32:0/114521467 shutdown_connections 2023-09-30T22:28:11.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.248+0000 7fef9f089700 1 -- 172.21.15.32:0/114521467 wait complete. 2023-09-30T22:28:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.250+0000 7fef9f089700 1 Processor -- start 2023-09-30T22:28:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.250+0000 7fef9f089700 1 -- start start 2023-09-30T22:28:11.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9f089700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef981a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9f089700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef9809f240 0x7fef981a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9f089700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef981a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9f089700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981a6f50 con 0x7fef9809f240 2023-09-30T22:28:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9f089700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef98071d20 con 0x7fef9809f650 2023-09-30T22:28:11.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9f089700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef980a3680 con 0x7fef9809e930 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9d626700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef981a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9ce25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef981a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9d626700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef981a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55368/0 (socket says 172.21.15.32:55368) 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9ce25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef981a3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:45820/0 (socket says 172.21.15.32:45820) 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9d626700 1 -- 172.21.15.32:0/413940914 learned_addr learned my addr 172.21.15.32:0/413940914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef9809f240 0x7fef981a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9d626700 1 -- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 msgr2=0x7fef981a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.251+0000 7fef9d626700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef981a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef9d626700 1 -- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef9809f240 msgr2=0x7fef981a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef9d626700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef9809f240 0x7fef981a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef9d626700 1 -- 172.21.15.32:0/413940914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094a0 con 0x7fef9809f650 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef9d626700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef981a4720 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fef9000efc0 tx=0x7fef9000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9000cd40 con 0x7fef9809f650 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9000f040 con 0x7fef9809f650 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900146c0 con 0x7fef9809f650 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef980a0190 con 0x7fef9809f650 2023-09-30T22:28:11.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.252+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef981a6b00 con 0x7fef9809f650 2023-09-30T22:28:11.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.253+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef90014820 con 0x7fef9809f650 2023-09-30T22:28:11.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.254+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef981a7180 con 0x7fef9809f650 2023-09-30T22:28:11.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.254+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef90040420 con 0x7fef9809f650 2023-09-30T22:28:11.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.259+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef981a7180 con 0x7fef9809f650 2023-09-30T22:28:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.306+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa014074ea0 con 0x7fa014133540 2023-09-30T22:28:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.307+0000 7fa0117fa700 1 -- 172.21.15.32:0/2362562690 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ffc015070 con 0x7fa014133540 2023-09-30T22:28:11.308 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:11.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 Processor -- start 2023-09-30T22:28:11.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 -- start start 2023-09-30T22:28:11.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e000 0x7f94c4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 0x7f94c4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c409f520 con 0x7f94c409e410 2023-09-30T22:28:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c409f130 con 0x7f94c409d6f0 2023-09-30T22:28:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94cadb2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c409e930 con 0x7f94c409e000 2023-09-30T22:28:11.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.308+0000 7f94c934f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 0x7f94c4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c934f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 0x7f94c4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53920/0 (socket says 172.21.15.32:53920) 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c934f700 1 -- 172.21.15.32:0/1454488416 learned_addr learned my addr 172.21.15.32:0/1454488416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e000 0x7f94c4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c8b4e700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 -- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 msgr2=0x7f94c409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 -- 172.21.15.32:0/1454488416 >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 msgr2=0x7f94c4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1454488416 >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 0x7f94c4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 -- 172.21.15.32:0/1454488416 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c409b2d0 con 0x7f94c409e000 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c934f700 1 --2- 172.21.15.32:0/1454488416 >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 0x7f94c4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e000 0x7f94c4130660 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009870 tx=0x7f94b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1a6766628d1eac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c37fe700 1 -- 172.21.15.32:0/1454488416 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011830 con 0x7f94c409e000 2023-09-30T22:28:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.309+0000 7f94c37fe700 1 -- 172.21.15.32:0/1454488416 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0011990 con 0x7f94c409e000 2023-09-30T22:28:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94c37fe700 1 -- 172.21.15.32:0/1454488416 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011d20 con 0x7f94c409e000 2023-09-30T22:28:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 msgr2=0x7fa0141c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa0141c65a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009840 tx=0x7f9ffc004750 comp rx=0 tx=0).stop 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 shutdown_connections 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa014135e60 0x7fa0141c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa014130540 0x7fa0141c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 --2- 172.21.15.32:0/2362562690 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa014133540 0x7fa0141c65a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 >> 172.21.15.32:0/2362562690 conn(0x7fa0140a2aa0 msgr2=0x7fa0140a09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 shutdown_connections 2023-09-30T22:28:11.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7fa01b1d8700 1 -- 172.21.15.32:0/2362562690 wait complete. 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 -- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e000 msgr2=0x7f94c4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e000 0x7f94c4130660 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009870 tx=0x7f94b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 -- 172.21.15.32:0/1454488416 shutdown_connections 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1454488416 >> v2:172.21.15.32:3300/0 conn(0x7f94c409e410 0x7f94c4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1454488416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e000 0x7f94c4130660 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.310+0000 7f94cadb2700 1 -- 172.21.15.32:0/1454488416 >> 172.21.15.32:0/1454488416 conn(0x7f94c4066f50 msgr2=0x7f94c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.311+0000 7f94cadb2700 1 -- 172.21.15.32:0/1454488416 shutdown_connections 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.311+0000 7f94cadb2700 1 -- 172.21.15.32:0/1454488416 wait complete. 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.312+0000 7f94cadb2700 1 Processor -- start 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.312+0000 7f94cadb2700 1 -- start start 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94cadb2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c411e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94cadb2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 0x7f94c411e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94cadb2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e410 0x7f94c411ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94cadb2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4122880 con 0x7f94c409e000 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94cadb2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40a16b0 con 0x7f94c409d6f0 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94cadb2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94c409c700 con 0x7f94c409e410 2023-09-30T22:28:11.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 0x7f94c411e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94c8b4e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c411e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94c8b4e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c411e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55396/0 (socket says 172.21.15.32:55396) 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 0x7f94c411e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53934/0 (socket says 172.21.15.32:53934) 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94c8b4e700 1 -- 172.21.15.32:0/1336054837 learned_addr learned my addr 172.21.15.32:0/1336054837 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.313+0000 7f94c934f700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e410 0x7f94c411ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c3fff700 1 -- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e410 msgr2=0x7f94c411ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e410 0x7f94c411ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c3fff700 1 -- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 msgr2=0x7f94c411e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c411e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c3fff700 1 -- 172.21.15.32:0/1336054837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc014660 con 0x7f94c409e000 2023-09-30T22:28:11.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c934f700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e410 0x7f94c411ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c8b4e700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c411e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c3fff700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 0x7f94c411e740 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f94b0016000 tx=0x7f94b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0004450 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b001f070 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b00094a0 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00038e0 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.314+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c411fd20 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.315+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94b0003a40 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.315+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94b0021070 con 0x7f94c409e000 2023-09-30T22:28:11.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.316+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c4074ea0 con 0x7f94c409e000 2023-09-30T22:28:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.324+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94b0010380 con 0x7f94c409e000 2023-09-30T22:28:11.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 Processor -- start 2023-09-30T22:28:11.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 -- start start 2023-09-30T22:28:11.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47140a0090 0x7f4714136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f471409d130 0x7f4714139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471409ba70 con 0x7f471409c650 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471409e8f0 con 0x7f47140a0090 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4719d10700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140a0760 con 0x7f471409d130 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4712ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47140a0090 0x7f4714136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f47137fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f471409c650 0x7f471409ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41692/0 (socket says 172.21.15.32:41692) 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f47137fe700 1 -- 172.21.15.32:0/2748415589 learned_addr learned my addr 172.21.15.32:0/2748415589 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.365+0000 7f4713fff700 1 --2- 172.21.15.32:0/2748415589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f471409d130 0x7f4714139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f4713fff700 1 -- 172.21.15.32:0/2748415589 >> v2:172.21.15.32:3300/0 conn(0x7f47140a0090 msgr2=0x7f4714136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f4713fff700 1 --2- 172.21.15.32:0/2748415589 >> v2:172.21.15.32:3300/0 conn(0x7f47140a0090 0x7f4714136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f4713fff700 1 -- 172.21.15.32:0/2748415589 >> v1:172.21.15.32:6789/0 conn(0x7f471409c650 legacy=0x7f471409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f4713fff700 1 -- 172.21.15.32:0/2748415589 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471409a080 con 0x7f471409d130 2023-09-30T22:28:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f47127fc700 1 -- 172.21.15.32:0/2748415589 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13564539 0 0) 0x7f471409ba70 con 0x7f471409c650 2023-09-30T22:28:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f4712ffd700 1 --2- 172.21.15.32:0/2748415589 >> v2:172.21.15.32:3300/0 conn(0x7f47140a0090 0x7f4714136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f4713fff700 1 --2- 172.21.15.32:0/2748415589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f471409d130 0x7f4714139170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc03f79a00dcc246 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f47127fc700 1 -- 172.21.15.32:0/2748415589 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00e070 con 0x7f471409d130 2023-09-30T22:28:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f47127fc700 1 -- 172.21.15.32:0/2748415589 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00bb20 con 0x7f471409d130 2023-09-30T22:28:11.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.366+0000 7f47127fc700 1 -- 172.21.15.32:0/2748415589 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c012430 con 0x7f471409d130 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 -- 172.21.15.32:0/2748415589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f471409d130 msgr2=0x7f4714139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 --2- 172.21.15.32:0/2748415589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f471409d130 0x7f4714139170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 -- 172.21.15.32:0/2748415589 shutdown_connections 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 --2- 172.21.15.32:0/2748415589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f471409d130 0x7f4714139170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 --2- 172.21.15.32:0/2748415589 >> v2:172.21.15.32:3300/0 conn(0x7f47140a0090 0x7f4714136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 -- 172.21.15.32:0/2748415589 >> 172.21.15.32:0/2748415589 conn(0x7f47140a3f70 msgr2=0x7f47140a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 -- 172.21.15.32:0/2748415589 shutdown_connections 2023-09-30T22:28:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.367+0000 7f4719d10700 1 -- 172.21.15.32:0/2748415589 wait complete. 2023-09-30T22:28:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.369+0000 7f4719d10700 1 Processor -- start 2023-09-30T22:28:11.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.369+0000 7f4719d10700 1 -- start start 2023-09-30T22:28:11.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4719d10700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f471409c650 0x7f4714137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4719d10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f471409d130 0x7f4714137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4719d10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47140a0090 0x7f4714137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4719d10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714138620 con 0x7f471409d130 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4719d10700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4714136740 con 0x7f47140a0090 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4719d10700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4714136990 con 0x7f471409c650 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4712ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f471409d130 0x7f4714137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4712ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f471409d130 0x7f4714137ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53950/0 (socket says 172.21.15.32:53950) 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4712ffd700 1 -- 172.21.15.32:0/1173675907 learned_addr learned my addr 172.21.15.32:0/1173675907 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f47137fe700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f471409c650 0x7f4714137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f4713fff700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47140a0090 0x7f4714137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:11.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f47137fe700 1 -- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47140a0090 msgr2=0x7f4714137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f47137fe700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47140a0090 0x7f4714137ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f47137fe700 1 -- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f471409d130 msgr2=0x7f4714137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f47137fe700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f471409d130 0x7f4714137ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.370+0000 7f47137fe700 1 -- 172.21.15.32:0/1173675907 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080094a0 con 0x7f471409c650 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f4713fff700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47140a0090 0x7f4714137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f47137fe700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f471409c650 0x7f4714137570 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f470400c470 tx=0x7f470400c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004c80 con 0x7f471409c650 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704004de0 con 0x7f471409c650 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704007310 con 0x7f471409c650 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c009a70 con 0x7f471409c650 2023-09-30T22:28:11.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.371+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4714135cd0 con 0x7f471409c650 2023-09-30T22:28:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.372+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471409d980 con 0x7f471409c650 2023-09-30T22:28:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.373+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4704007470 con 0x7f471409c650 2023-09-30T22:28:11.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.373+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f470403f340 con 0x7f471409c650 2023-09-30T22:28:11.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.378+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4704007720 con 0x7f471409c650 2023-09-30T22:28:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.497+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fef981a7180 con 0x7fef9809f650 2023-09-30T22:28:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.497+0000 7fef957fa700 1 -- 172.21.15.32:0/413940914 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef981a7180 con 0x7fef9809f650 2023-09-30T22:28:11.499 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 msgr2=0x7fef981a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef981a4720 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fef9000efc0 tx=0x7fef9000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 shutdown_connections 2023-09-30T22:28:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef9809f650 0x7fef981a4720 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef9809e930 0x7fef981a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 --2- 172.21.15.32:0/413940914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef9809f240 0x7fef981a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 >> 172.21.15.32:0/413940914 conn(0x7fef980a4210 msgr2=0x7fef981b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 shutdown_connections 2023-09-30T22:28:11.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.500+0000 7fef9f089700 1 -- 172.21.15.32:0/413940914 wait complete. 2023-09-30T22:28:11.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:11.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.549+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f94c40722f0 con 0x7f94c409e000 2023-09-30T22:28:11.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.550+0000 7f94c17fa700 1 -- 172.21.15.32:0/1336054837 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94b002eb90 con 0x7f94c409e000 2023-09-30T22:28:11.551 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 msgr2=0x7f94c411e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 0x7f94c411e740 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f94b0016000 tx=0x7f94b0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 shutdown_connections 2023-09-30T22:28:11.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94c409e410 0x7f94c411ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94c409d6f0 0x7f94c411e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 --2- 172.21.15.32:0/1336054837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94c409e000 0x7f94c411e740 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 >> 172.21.15.32:0/1336054837 conn(0x7f94c4066f50 msgr2=0x7f94c406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 shutdown_connections 2023-09-30T22:28:11.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.552+0000 7f94cadb2700 1 -- 172.21.15.32:0/1336054837 wait complete. 2023-09-30T22:28:11.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:11.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.584+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f47140747f0 con 0x7f471409c650 2023-09-30T22:28:11.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.585+0000 7f46fbfff700 1 -- 172.21.15.32:0/1173675907 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4704011e80 con 0x7f471409c650 2023-09-30T22:28:11.586 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:11.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f471409c650 msgr2=0x7f4714137570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:11.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f471409c650 0x7f4714137570 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f470400c470 tx=0x7f470400c780 comp rx=0 tx=0).stop 2023-09-30T22:28:11.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 shutdown_connections 2023-09-30T22:28:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f47140a0090 0x7f4714137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f471409c650 0x7f4714137570 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 --2- 172.21.15.32:0/1173675907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f471409d130 0x7f4714137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 >> 172.21.15.32:0/1173675907 conn(0x7f47140a3f70 msgr2=0x7f4714138ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 shutdown_connections 2023-09-30T22:28:11.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:11.587+0000 7f4719d10700 1 -- 172.21.15.32:0/1173675907 wait complete. 2023-09-30T22:28:11.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:12.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:12.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 Processor -- start 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 -- start start 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 0x7ffa7c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 0x7ffa7c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7c09ece0 con 0x7ffa7c131b40 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c09e760 con 0x7ffa7c06a770 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.477+0000 7ffa82998700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c09e040 con 0x7ffa7c06b080 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 0x7ffa7c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 0x7ffa7c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 0x7ffa7c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53960/0 (socket says 172.21.15.32:53960) 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa80f35700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa7c131b40 0x7ffa7c138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41702/0 (socket says 172.21.15.32:41702) 2023-09-30T22:28:12.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7bfff700 1 -- 172.21.15.32:0/407647461 learned_addr learned my addr 172.21.15.32:0/407647461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 0x7ffa7c131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55424/0 (socket says 172.21.15.32:55424) 2023-09-30T22:28:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 -- 172.21.15.32:0/407647461 >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 msgr2=0x7ffa7c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 --2- 172.21.15.32:0/407647461 >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 0x7ffa7c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 -- 172.21.15.32:0/407647461 >> v1:172.21.15.32:6789/0 conn(0x7ffa7c131b40 legacy=0x7ffa7c138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 -- 172.21.15.32:0/407647461 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c09f630 con 0x7ffa7c06b080 2023-09-30T22:28:12.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7affd700 1 -- 172.21.15.32:0/407647461 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216705736 0 0) 0x7ffa7c09ece0 con 0x7ffa7c131b40 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7bfff700 1 --2- 172.21.15.32:0/407647461 >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 0x7ffa7c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7b7fe700 1 --2- 172.21.15.32:0/407647461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 0x7ffa7c131600 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009840 tx=0x7ffa6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e7c68c6f9947f72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7affd700 1 -- 172.21.15.32:0/407647461 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c011830 con 0x7ffa7c06b080 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7affd700 1 -- 172.21.15.32:0/407647461 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c011990 con 0x7ffa7c06b080 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.478+0000 7ffa7affd700 1 -- 172.21.15.32:0/407647461 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c011d20 con 0x7ffa7c06b080 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 -- 172.21.15.32:0/407647461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 msgr2=0x7ffa7c131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 --2- 172.21.15.32:0/407647461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 0x7ffa7c131600 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009840 tx=0x7ffa6c009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 -- 172.21.15.32:0/407647461 shutdown_connections 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 --2- 172.21.15.32:0/407647461 >> v2:172.21.15.32:3300/0 conn(0x7ffa7c06a770 0x7ffa7c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 --2- 172.21.15.32:0/407647461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c06b080 0x7ffa7c131600 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 -- 172.21.15.32:0/407647461 >> 172.21.15.32:0/407647461 conn(0x7ffa7c066f50 msgr2=0x7ffa7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 -- 172.21.15.32:0/407647461 shutdown_connections 2023-09-30T22:28:12.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.479+0000 7ffa82998700 1 -- 172.21.15.32:0/407647461 wait complete. 2023-09-30T22:28:12.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.480+0000 7ffa82998700 1 Processor -- start 2023-09-30T22:28:12.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.480+0000 7ffa82998700 1 -- start start 2023-09-30T22:28:12.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa82998700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa7c06a770 0x7ffa7c1c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa82998700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 0x7ffa7c1c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa82998700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 0x7ffa7c1c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa82998700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c1c5c10 con 0x7ffa7c06a770 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa82998700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c09c010 con 0x7ffa7c131b40 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa82998700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c125c40 con 0x7ffa7c06b080 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa80f35700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 0x7ffa7c1c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 0x7ffa7c1c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa7c06a770 0x7ffa7c1c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 0x7ffa7c1c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:45868/0 (socket says 172.21.15.32:45868) 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa80f35700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 0x7ffa7c1c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55440/0 (socket says 172.21.15.32:55440) 2023-09-30T22:28:12.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7b7fe700 1 -- 172.21.15.32:0/2085676511 learned_addr learned my addr 172.21.15.32:0/2085676511 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 -- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 msgr2=0x7ffa7c1c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 0x7ffa7c1c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 -- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 msgr2=0x7ffa7c1c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 0x7ffa7c1c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 -- 172.21.15.32:0/2085676511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64009550 con 0x7ffa7c06a770 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa80f35700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 0x7ffa7c1c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7b7fe700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 0x7ffa7c1c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.481+0000 7ffa7bfff700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa7c06a770 0x7ffa7c1c5190 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400e9a0 tx=0x7ffa6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.482+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004030 con 0x7ffa7c06a770 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.482+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64004190 con 0x7ffa7c06a770 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.482+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004520 con 0x7ffa7c06a770 2023-09-30T22:28:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.482+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c0094a0 con 0x7ffa7c06a770 2023-09-30T22:28:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.482+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c1315e0 con 0x7ffa7c06a770 2023-09-30T22:28:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.482+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa640036a0 con 0x7ffa7c06a770 2023-09-30T22:28:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.483+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffa6403f870 con 0x7ffa7c06a770 2023-09-30T22:28:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.483+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c074ea0 con 0x7ffa7c06a770 2023-09-30T22:28:12.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.488+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa64011db0 con 0x7ffa7c06a770 2023-09-30T22:28:12.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:12.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:12.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:12.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.657+0000 7f60e423d700 1 Processor -- start 2023-09-30T22:28:12.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 -- start start 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 0x7f60dc1307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc09e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60dc09e7c0 0x7f60dc13a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60dc06f1c0 con 0x7f60dc09e7c0 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc070700 con 0x7f60dc130410 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.658+0000 7f60e423d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc071f20 con 0x7f60dc09de50 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 0x7f60dc1307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 0x7f60dc1307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55450/0 (socket says 172.21.15.32:55450) 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 -- 172.21.15.32:0/3419620040 learned_addr learned my addr 172.21.15.32:0/3419620040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e17d8700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc09e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e27da700 1 --2- 172.21.15.32:0/3419620040 >> v2:172.21.15.32:3300/0 conn(0x7f60dc09e7c0 0x7f60dc13a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 -- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 msgr2=0x7f60dc09e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc09e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 -- 172.21.15.32:0/3419620040 >> v2:172.21.15.32:3300/0 conn(0x7f60dc09e7c0 msgr2=0x7f60dc13a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 --2- 172.21.15.32:0/3419620040 >> v2:172.21.15.32:3300/0 conn(0x7f60dc09e7c0 0x7f60dc13a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 -- 172.21.15.32:0/3419620040 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc12ffe0 con 0x7f60dc130410 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e17d8700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc09e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e1fd9700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 0x7f60dc1307e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098f0 tx=0x7f60d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5495f157305b49 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.659+0000 7f60e0fd7700 1 -- 172.21.15.32:0/3419620040 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8011830 con 0x7f60dc130410 2023-09-30T22:28:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e0fd7700 1 -- 172.21.15.32:0/3419620040 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d8011990 con 0x7f60dc130410 2023-09-30T22:28:12.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e0fd7700 1 -- 172.21.15.32:0/3419620040 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8011d20 con 0x7f60dc130410 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 -- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 msgr2=0x7f60dc1307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 0x7f60dc1307e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098f0 tx=0x7f60d8009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 -- 172.21.15.32:0/3419620040 shutdown_connections 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc130410 0x7f60dc1307e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 --2- 172.21.15.32:0/3419620040 >> v2:172.21.15.32:3300/0 conn(0x7f60dc09e7c0 0x7f60dc13a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 --2- 172.21.15.32:0/3419620040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc09e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 -- 172.21.15.32:0/3419620040 >> 172.21.15.32:0/3419620040 conn(0x7f60dc0a2610 msgr2=0x7f60dc0a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.660+0000 7f60e423d700 1 -- 172.21.15.32:0/3419620040 shutdown_connections 2023-09-30T22:28:12.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.661+0000 7f60e423d700 1 -- 172.21.15.32:0/3419620040 wait complete. 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.662+0000 7f60e423d700 1 Processor -- start 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.662+0000 7f60e423d700 1 -- start start 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e423d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc11a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e423d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 0x7f60dc11ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e423d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc130410 0x7f60dc11b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e423d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc11d510 con 0x7f60dc130410 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e423d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc11bb90 con 0x7f60dc09e7c0 2023-09-30T22:28:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e423d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc11df60 con 0x7f60dc09de50 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e17d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 0x7f60dc11ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e27da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc130410 0x7f60dc11b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e1fd9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc11a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e17d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 0x7f60dc11ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55464/0 (socket says 172.21.15.32:55464) 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e17d8700 1 -- 172.21.15.32:0/3286332672 learned_addr learned my addr 172.21.15.32:0/3286332672 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e27da700 1 -- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 msgr2=0x7f60dc11a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e27da700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc11a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e27da700 1 -- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 msgr2=0x7f60dc11ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e27da700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 0x7f60dc11ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e27da700 1 -- 172.21.15.32:0/3286332672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc0094a0 con 0x7f60dc130410 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.663+0000 7f60e17d8700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 0x7f60dc11ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.664+0000 7f60e27da700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc130410 0x7f60dc11b160 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f60d400efc0 tx=0x7f60d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.664+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d400cc90 con 0x7f60dc130410 2023-09-30T22:28:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.664+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d8009550 con 0x7f60dc130410 2023-09-30T22:28:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.664+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d400f040 con 0x7f60dc130410 2023-09-30T22:28:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.664+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc071620 con 0x7f60dc130410 2023-09-30T22:28:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.664+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4009e60 con 0x7f60dc130410 2023-09-30T22:28:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.665+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60d4014480 con 0x7f60dc130410 2023-09-30T22:28:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.665+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60d4040080 con 0x7f60dc130410 2023-09-30T22:28:12.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.665+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c40052e0 con 0x7f60dc130410 2023-09-30T22:28:12.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.675+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60d4018020 con 0x7f60dc130410 2023-09-30T22:28:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.699+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffa7c1311a0 con 0x7ffa7c06a770 2023-09-30T22:28:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.699+0000 7ffa78ff9700 1 -- 172.21.15.32:0/2085676511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa64011350 con 0x7ffa7c06a770 2023-09-30T22:28:12.700 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa7c06a770 msgr2=0x7ffa7c1c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa7c06a770 0x7ffa7c1c5190 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400e9a0 tx=0x7ffa6400ecb0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 shutdown_connections 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa7c131b40 0x7ffa7c1c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa7c06a770 0x7ffa7c1c5190 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 --2- 172.21.15.32:0/2085676511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa7c06b080 0x7ffa7c1c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 >> 172.21.15.32:0/2085676511 conn(0x7ffa7c066f50 msgr2=0x7ffa7c06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 shutdown_connections 2023-09-30T22:28:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.703+0000 7ffa82998700 1 -- 172.21.15.32:0/2085676511 wait complete. 2023-09-30T22:28:12.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.718+0000 7f8a039f0700 1 Processor -- start 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.718+0000 7f8a039f0700 1 -- start start 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.718+0000 7f8a039f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89fc09d6f0 0x7f89fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a039f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a039f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc09f130 con 0x7f89fc09e000 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a039f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc09f520 con 0x7f89fc09d6f0 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a039f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc09e930 con 0x7f89fc09e410 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89fc09d6f0 0x7f89fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89fc09d6f0 0x7f89fc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53988/0 (socket says 172.21.15.32:53988) 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 -- 172.21.15.32:0/2119387359 learned_addr learned my addr 172.21.15.32:0/2119387359 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a00f8b700 1 --1- 172.21.15.32:0/2119387359 >> v1:172.21.15.32:6789/0 conn(0x7f89fc09e000 0x7f89fc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41706/0 (socket says 172.21.15.32:41706) 2023-09-30T22:28:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a01f8d700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 -- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 msgr2=0x7f89fc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 -- 172.21.15.32:0/2119387359 >> v1:172.21.15.32:6789/0 conn(0x7f89fc09e000 legacy=0x7f89fc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 -- 172.21.15.32:0/2119387359 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc09b2d0 con 0x7f89fc09d6f0 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f89f3fff700 1 -- 172.21.15.32:0/2119387359 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978233057 0 0) 0x7f89fc09f130 con 0x7f89fc09e000 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a01f8d700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.719+0000 7f8a0178c700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc09dac0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098f0 tx=0x7f89f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc964b6d8c0e387f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f89f3fff700 1 -- 172.21.15.32:0/2119387359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011830 con 0x7f89fc09d6f0 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f89f3fff700 1 -- 172.21.15.32:0/2119387359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8011990 con 0x7f89fc09d6f0 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f89f3fff700 1 -- 172.21.15.32:0/2119387359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011d20 con 0x7f89fc09d6f0 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f8a039f0700 1 -- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 msgr2=0x7f89fc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc09dac0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098f0 tx=0x7f89f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f8a039f0700 1 -- 172.21.15.32:0/2119387359 shutdown_connections 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2119387359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc09dac0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.720+0000 7f8a039f0700 1 -- 172.21.15.32:0/2119387359 >> 172.21.15.32:0/2119387359 conn(0x7f89fc066f50 msgr2=0x7f89fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.721+0000 7f8a039f0700 1 -- 172.21.15.32:0/2119387359 shutdown_connections 2023-09-30T22:28:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.721+0000 7f8a039f0700 1 -- 172.21.15.32:0/2119387359 wait complete. 2023-09-30T22:28:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 Processor -- start 2023-09-30T22:28:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 -- start start 2023-09-30T22:28:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89fc09e000 0x7f89fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc125c30 con 0x7f89fc09d6f0 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc06fcd0 con 0x7f89fc09e000 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a039f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc125e80 con 0x7f89fc09e410 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a0178c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a0178c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53990/0 (socket says 172.21.15.32:53990) 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a0178c700 1 -- 172.21.15.32:0/2022107133 learned_addr learned my addr 172.21.15.32:0/2022107133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a00f8b700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89fc09e000 0x7f89fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.722+0000 7f8a01f8d700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a0178c700 1 -- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 msgr2=0x7f89fc06f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a0178c700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc06f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a0178c700 1 -- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89fc09e000 msgr2=0x7f89fc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a0178c700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89fc09e000 0x7f89fc06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a0178c700 1 -- 172.21.15.32:0/2022107133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4009a70 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a0178c700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc06e870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0a3e80 tx=0x7f89f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80042c0 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f801f070 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f8009550 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80036a0 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.723+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc09c700 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.724+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89f8021030 con 0x7f89fc09d6f0 2023-09-30T22:28:12.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.725+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f89f8040950 con 0x7f89fc09d6f0 2023-09-30T22:28:12.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.725+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc070d60 con 0x7f89fc09d6f0 2023-09-30T22:28:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.730+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89f8015080 con 0x7f89fc09d6f0 2023-09-30T22:28:12.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.764+0000 7f85c3ff1700 1 Processor -- start 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.764+0000 7f85c3ff1700 1 -- start start 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c3ff1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85bc06a770 0x7f85bc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c3ff1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 0x7f85bc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c3ff1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c3ff1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc06ab40 con 0x7f85bc06a770 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c3ff1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc09ebc0 con 0x7f85bc13c1e0 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c3ff1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc09a810 con 0x7f85bc09dc30 2023-09-30T22:28:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c158c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 0x7f85bc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c1d8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85bc06a770 0x7f85bc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c158c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 0x7f85bc09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:45918/0 (socket says 172.21.15.32:45918) 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c158c700 1 -- 172.21.15.32:0/2987864982 learned_addr learned my addr 172.21.15.32:0/2987864982 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.765+0000 7f85c258e700 1 --2- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c158c700 1 -- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 msgr2=0x7f85bc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c158c700 1 --2- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c158c700 1 -- 172.21.15.32:0/2987864982 >> v2:172.21.15.32:3300/0 conn(0x7f85bc06a770 msgr2=0x7f85bc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c158c700 1 --2- 172.21.15.32:0/2987864982 >> v2:172.21.15.32:3300/0 conn(0x7f85bc06a770 0x7f85bc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c158c700 1 -- 172.21.15.32:0/2987864982 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc09e920 con 0x7f85bc09dc30 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c1d8d700 1 --2- 172.21.15.32:0/2987864982 >> v2:172.21.15.32:3300/0 conn(0x7f85bc06a770 0x7f85bc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c158c700 1 --2- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 0x7f85bc09e080 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009920 tx=0x7f85ac009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6730ee69de6110f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c0d8b700 1 -- 172.21.15.32:0/2987864982 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac011830 con 0x7f85bc09dc30 2023-09-30T22:28:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c0d8b700 1 -- 172.21.15.32:0/2987864982 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac011990 con 0x7f85bc09dc30 2023-09-30T22:28:12.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.766+0000 7f85c0d8b700 1 -- 172.21.15.32:0/2987864982 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac011d20 con 0x7f85bc09dc30 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 msgr2=0x7f85bc09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 0x7f85bc09e080 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009920 tx=0x7f85ac009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2987864982 shutdown_connections 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2987864982 >> v2:172.21.15.32:3300/0 conn(0x7f85bc06a770 0x7f85bc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2987864982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc09dc30 0x7f85bc09e080 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2987864982 >> 172.21.15.32:0/2987864982 conn(0x7f85bc066f50 msgr2=0x7f85bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2987864982 shutdown_connections 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.767+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2987864982 wait complete. 2023-09-30T22:28:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 Processor -- start 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 -- start start 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 0x7f85bc126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc09dc30 0x7f85bc127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc1cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc129b80 con 0x7f85bc09dc30 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc13abd0 con 0x7f85bc13c1e0 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.768+0000 7f85c3ff1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1267f0 con 0x7f85bc06a770 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 0x7f85bc126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 0x7f85bc126e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:45922/0 (socket says 172.21.15.32:45922) 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 -- 172.21.15.32:0/2287687103 learned_addr learned my addr 172.21.15.32:0/2287687103 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c258e700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc1cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c158c700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc09dc30 0x7f85bc127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 -- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 msgr2=0x7f85bc1cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc1cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 -- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc09dc30 msgr2=0x7f85bc127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc09dc30 0x7f85bc127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 -- 172.21.15.32:0/2287687103 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80094a0 con 0x7f85bc06a770 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c158c700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc09dc30 0x7f85bc127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c1d8d700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 0x7f85bc126e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f85b800ec10 tx=0x7f85b800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800c910 con 0x7f85bc06a770 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800ca70 con 0x7f85bc06a770 2023-09-30T22:28:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac009550 con 0x7f85bc06a770 2023-09-30T22:28:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800cde0 con 0x7f85bc06a770 2023-09-30T22:28:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.769+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc127e50 con 0x7f85bc06a770 2023-09-30T22:28:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.770+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85b8013470 con 0x7f85bc06a770 2023-09-30T22:28:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.770+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f85b803f860 con 0x7f85bc06a770 2023-09-30T22:28:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.771+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc074ea0 con 0x7f85bc06a770 2023-09-30T22:28:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.780+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85b80232f0 con 0x7f85bc06a770 2023-09-30T22:28:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.890+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f60c4005590 con 0x7f60dc130410 2023-09-30T22:28:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.890+0000 7f60d27fc700 1 -- 172.21.15.32:0/3286332672 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60d402b090 con 0x7f60dc130410 2023-09-30T22:28:12.891 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc130410 msgr2=0x7f60dc11b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc130410 0x7f60dc11b160 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f60d400efc0 tx=0x7f60d400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 shutdown_connections 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60dc130410 0x7f60dc11b160 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60dc09de50 0x7f60dc11a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 --2- 172.21.15.32:0/3286332672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60dc09e7c0 0x7f60dc11ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 >> 172.21.15.32:0/3286332672 conn(0x7f60dc0a2610 msgr2=0x7f60dc12f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 shutdown_connections 2023-09-30T22:28:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.893+0000 7f60e423d700 1 -- 172.21.15.32:0/3286332672 wait complete. 2023-09-30T22:28:12.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:12.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.941+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f89fc074ea0 con 0x7f89fc09d6f0 2023-09-30T22:28:12.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.942+0000 7f89f1ffb700 1 -- 172.21.15.32:0/2022107133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89f80199a0 con 0x7f89fc09d6f0 2023-09-30T22:28:12.943 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 msgr2=0x7f89fc06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc06e870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0a3e80 tx=0x7f89f8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 shutdown_connections 2023-09-30T22:28:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f89fc09e410 0x7f89fc06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89fc09d6f0 0x7f89fc06e870 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 --2- 172.21.15.32:0/2022107133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89fc09e000 0x7f89fc06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 >> 172.21.15.32:0/2022107133 conn(0x7f89fc066f50 msgr2=0x7f89fc071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 shutdown_connections 2023-09-30T22:28:12.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.944+0000 7f8a039f0700 1 -- 172.21.15.32:0/2022107133 wait complete. 2023-09-30T22:28:12.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.991+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f85bc1299e0 con 0x7f85bc06a770 2023-09-30T22:28:12.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.991+0000 7f85b27fc700 1 -- 172.21.15.32:0/2287687103 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85bc1299e0 con 0x7f85bc06a770 2023-09-30T22:28:12.992 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 msgr2=0x7f85bc126e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 0x7f85bc126e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f85b800ec10 tx=0x7f85b800ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 shutdown_connections 2023-09-30T22:28:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85bc13c1e0 0x7f85bc1cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85bc06a770 0x7f85bc126e00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 --2- 172.21.15.32:0/2287687103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85bc09dc30 0x7f85bc127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 >> 172.21.15.32:0/2287687103 conn(0x7f85bc066f50 msgr2=0x7f85bc13b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 shutdown_connections 2023-09-30T22:28:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:12.994+0000 7f85c3ff1700 1 -- 172.21.15.32:0/2287687103 wait complete. 2023-09-30T22:28:13.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:13.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.865+0000 7f8d54c7f700 1 Processor -- start 2023-09-30T22:28:13.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.865+0000 7f8d54c7f700 1 -- start start 2023-09-30T22:28:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d54c7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d5009d5c0 0x7f8d50138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d54c7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d500a14a0 0x7f8d5013b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:13.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d54c7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5009ea10 con 0x7f8d500a09c0 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d54c7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5009e210 con 0x7f8d500a14a0 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d54c7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500a26f0 con 0x7f8d5009d5c0 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d500a14a0 0x7f8d5013b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d5009d5c0 0x7f8d50138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8d500a09c0 0x7f8d500a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41710/0 (socket says 172.21.15.32:41710) 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4e59c700 1 -- 172.21.15.32:0/672914644 learned_addr learned my addr 172.21.15.32:0/672914644 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4dd9b700 1 -- 172.21.15.32:0/672914644 >> v2:172.21.15.32:3300/0 conn(0x7f8d500a14a0 msgr2=0x7f8d5013b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4dd9b700 1 --2- 172.21.15.32:0/672914644 >> v2:172.21.15.32:3300/0 conn(0x7f8d500a14a0 0x7f8d5013b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4dd9b700 1 -- 172.21.15.32:0/672914644 >> v1:172.21.15.32:6789/0 conn(0x7f8d500a09c0 legacy=0x7f8d500a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:13.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4dd9b700 1 -- 172.21.15.32:0/672914644 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5009db70 con 0x7f8d5009d5c0 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4ed9d700 1 --2- 172.21.15.32:0/672914644 >> v2:172.21.15.32:3300/0 conn(0x7f8d500a14a0 0x7f8d5013b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.866+0000 7f8d4dd9b700 1 --2- 172.21.15.32:0/672914644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d5009d5c0 0x7f8d50138d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009840 tx=0x7f8d40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43d2b6d96e7b31e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d4d59a700 1 -- 172.21.15.32:0/672914644 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40011830 con 0x7f8d5009d5c0 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d4d59a700 1 -- 172.21.15.32:0/672914644 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d40011990 con 0x7f8d5009d5c0 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d4d59a700 1 -- 172.21.15.32:0/672914644 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40011d20 con 0x7f8d5009d5c0 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 -- 172.21.15.32:0/672914644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d5009d5c0 msgr2=0x7f8d50138d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/672914644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d5009d5c0 0x7f8d50138d70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009840 tx=0x7f8d40009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 -- 172.21.15.32:0/672914644 shutdown_connections 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/672914644 >> v2:172.21.15.32:3300/0 conn(0x7f8d500a14a0 0x7f8d5013b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:13.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/672914644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d5009d5c0 0x7f8d50138d70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 -- 172.21.15.32:0/672914644 >> 172.21.15.32:0/672914644 conn(0x7f8d500a2fe0 msgr2=0x7f8d500a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 -- 172.21.15.32:0/672914644 shutdown_connections 2023-09-30T22:28:13.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.867+0000 7f8d54c7f700 1 -- 172.21.15.32:0/672914644 wait complete. 2023-09-30T22:28:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.868+0000 7f8d54c7f700 1 Processor -- start 2023-09-30T22:28:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 -- start start 2023-09-30T22:28:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 0x7f8d5011c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d500a09c0 0x7f8d5011ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:13.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d500a14a0 0x7f8d5011d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50120020 con 0x7f8d5009d5c0 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5011e1b0 con 0x7f8d500a14a0 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d54c7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500a5010 con 0x7f8d500a09c0 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 0x7f8d5011c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d500a14a0 0x7f8d5011d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d500a09c0 0x7f8d5011ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 0x7f8d5011c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54038/0 (socket says 172.21.15.32:54038) 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4e59c700 1 -- 172.21.15.32:0/1787863592 learned_addr learned my addr 172.21.15.32:0/1787863592 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:13.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4ed9d700 1 -- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d500a09c0 msgr2=0x7f8d5011ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4ed9d700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d500a09c0 0x7f8d5011ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4ed9d700 1 -- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 msgr2=0x7f8d5011c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4ed9d700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 0x7f8d5011c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.869+0000 7f8d4ed9d700 1 -- 172.21.15.32:0/1787863592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009a70 con 0x7f8d500a14a0 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d4dd9b700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d500a09c0 0x7f8d5011ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d4e59c700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 0x7f8d5011c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d4ed9d700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d500a14a0 0x7f8d5011d230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800efc0 tx=0x7f8d4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800ca90 con 0x7f8d500a14a0 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4800f040 con 0x7f8d500a14a0 2023-09-30T22:28:13.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400094a0 con 0x7f8d500a14a0 2023-09-30T22:28:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800ce30 con 0x7f8d500a14a0 2023-09-30T22:28:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.870+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5009cf20 con 0x7f8d500a14a0 2023-09-30T22:28:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.871+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d48014480 con 0x7f8d500a14a0 2023-09-30T22:28:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.871+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8d4803fbd0 con 0x7f8d500a14a0 2023-09-30T22:28:13.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.871+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d500747f0 con 0x7f8d500a14a0 2023-09-30T22:28:13.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:13.876+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d48012930 con 0x7f8d500a14a0 2023-09-30T22:28:13.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:13.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:14.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.056+0000 7f3bdaea7700 1 Processor -- start 2023-09-30T22:28:14.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.056+0000 7f3bdaea7700 1 -- start start 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bdaea7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bdaea7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd409eca0 0x7f3bd406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bdaea7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bd4134610 0x7f3bd4134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bdaea7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd4070ee0 con 0x7f3bd4134610 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bdaea7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4071480 con 0x7f3bd406e570 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bdaea7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd406f3c0 con 0x7f3bd409eca0 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bd4134610 0x7f3bd4134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3bd4134610 0x7f3bd4134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54040/0 (socket says 172.21.15.32:54040) 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 -- 172.21.15.32:0/1460169176 learned_addr learned my addr 172.21.15.32:0/1460169176 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 -- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd409eca0 msgr2=0x7f3bd406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd409eca0 0x7f3bd406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 -- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 msgr2=0x7f3bd406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.057+0000 7f3bd9444700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.058+0000 7f3bd9444700 1 -- 172.21.15.32:0/1460169176 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd4071b90 con 0x7f3bd4134610 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.058+0000 7f3bd9444700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4134610 0x7f3bd4134ab0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009e50 tx=0x7f3bcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e24b6976037f51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.058+0000 7f3bd37fe700 1 -- 172.21.15.32:0/1460169176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00e070 con 0x7f3bd4134610 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.058+0000 7f3bd37fe700 1 -- 172.21.15.32:0/1460169176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00b9c0 con 0x7f3bd4134610 2023-09-30T22:28:14.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.058+0000 7f3bd37fe700 1 -- 172.21.15.32:0/1460169176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00bd70 con 0x7f3bd4134610 2023-09-30T22:28:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 -- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4134610 msgr2=0x7f3bd4134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4134610 0x7f3bd4134ab0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009e50 tx=0x7f3bcc00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 -- 172.21.15.32:0/1460169176 shutdown_connections 2023-09-30T22:28:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4134610 0x7f3bd4134ab0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/1460169176 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd409eca0 0x7f3bd406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 -- 172.21.15.32:0/1460169176 >> 172.21.15.32:0/1460169176 conn(0x7f3bd40a2aa0 msgr2=0x7f3bd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 -- 172.21.15.32:0/1460169176 shutdown_connections 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.059+0000 7f3bdaea7700 1 -- 172.21.15.32:0/1460169176 wait complete. 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.060+0000 7f3bdaea7700 1 Processor -- start 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.060+0000 7f3bdaea7700 1 -- start start 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bdaea7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd41214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bdaea7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd4134610 0x7f3bd4121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bdaea7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 0x7f3bd41d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bdaea7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4124820 con 0x7f3bd4121f60 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bdaea7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4124a20 con 0x7f3bd406e570 2023-09-30T22:28:14.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bdaea7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4123910 con 0x7f3bd4134610 2023-09-30T22:28:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bd9444700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 0x7f3bd41d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bd9444700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 0x7f3bd41d2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54044/0 (socket says 172.21.15.32:54044) 2023-09-30T22:28:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bd9444700 1 -- 172.21.15.32:0/289229452 learned_addr learned my addr 172.21.15.32:0/289229452 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bd3fff700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd4134610 0x7f3bd4121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.061+0000 7f3bd8c43700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd41214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd3fff700 1 -- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 msgr2=0x7f3bd41214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd3fff700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd41214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd3fff700 1 -- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 msgr2=0x7f3bd41d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd3fff700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 0x7f3bd41d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd3fff700 1 -- 172.21.15.32:0/289229452 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc009b30 con 0x7f3bd4134610 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd9444700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 0x7f3bd41d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd3fff700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd4134610 0x7f3bd4121a20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4070e10 tx=0x7f3bc000b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011630 con 0x7f3bd4134610 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0011790 con 0x7f3bd4134610 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011b40 con 0x7f3bd4134610 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd41343b0 con 0x7f3bd4134610 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.062+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd40a1f30 con 0x7f3bd4134610 2023-09-30T22:28:14.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.063+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bc0011ca0 con 0x7f3bd4134610 2023-09-30T22:28:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.064+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd4070670 con 0x7f3bd4134610 2023-09-30T22:28:14.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.064+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3bc003f530 con 0x7f3bd4134610 2023-09-30T22:28:14.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.069+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bc0021c90 con 0x7f3bd4134610 2023-09-30T22:28:14.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.093+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d50072980 con 0x7f8d500a14a0 2023-09-30T22:28:14.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.093+0000 7f8d46ffd700 1 -- 172.21.15.32:0/1787863592 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d48012340 con 0x7f8d500a14a0 2023-09-30T22:28:14.095 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d500a14a0 msgr2=0x7f8d5011d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d500a14a0 0x7f8d5011d230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800efc0 tx=0x7f8d4800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 shutdown_connections 2023-09-30T22:28:14.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d500a14a0 0x7f8d5011d230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d5009d5c0 0x7f8d5011c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 --2- 172.21.15.32:0/1787863592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d500a09c0 0x7f8d5011ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.096+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 >> 172.21.15.32:0/1787863592 conn(0x7f8d500a2fe0 msgr2=0x7f8d5009a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.099+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 shutdown_connections 2023-09-30T22:28:14.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.099+0000 7f8d54c7f700 1 -- 172.21.15.32:0/1787863592 wait complete. 2023-09-30T22:28:14.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.105+0000 7fd8da8fd700 1 Processor -- start 2023-09-30T22:28:14.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.105+0000 7fd8da8fd700 1 -- start start 2023-09-30T22:28:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8da8fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 0x7fd8d409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8da8fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d409ffb0 0x7fd8d409f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8da8fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d40a06b0 con 0x7fd8d4133b80 2023-09-30T22:28:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8da8fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d406b920 con 0x7fd8d409e850 2023-09-30T22:28:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8da8fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d409aa30 con 0x7fd8d409ffb0 2023-09-30T22:28:14.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8d3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 0x7fd8d409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8d3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 0x7fd8d409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55540/0 (socket says 172.21.15.32:55540) 2023-09-30T22:28:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8d3fff700 1 -- 172.21.15.32:0/3242784053 learned_addr learned my addr 172.21.15.32:0/3242784053 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:14.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8d37fe700 1 --2- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d409ffb0 0x7fd8d409f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8d3fff700 1 -- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d409ffb0 msgr2=0x7fd8d409f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.106+0000 7fd8d3fff700 1 --2- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d409ffb0 0x7fd8d409f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.107+0000 7fd8d3fff700 1 -- 172.21.15.32:0/3242784053 >> v1:172.21.15.32:6789/0 conn(0x7fd8d4133b80 legacy=0x7fd8d4133fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.107+0000 7fd8d3fff700 1 -- 172.21.15.32:0/3242784053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d406caa0 con 0x7fd8d409e850 2023-09-30T22:28:14.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.107+0000 7fd8d3fff700 1 --2- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 0x7fd8d409ec20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009920 tx=0x7fd8c4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4d225cc4e4574d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.107+0000 7fd8d2ffd700 1 -- 172.21.15.32:0/3242784053 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400ba60 con 0x7fd8d409e850 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.107+0000 7fd8d2ffd700 1 -- 172.21.15.32:0/3242784053 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c400bbc0 con 0x7fd8d409e850 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.107+0000 7fd8d2ffd700 1 -- 172.21.15.32:0/3242784053 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4009e70 con 0x7fd8d409e850 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 -- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 msgr2=0x7fd8d409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 0x7fd8d409ec20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009920 tx=0x7fd8c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 -- 172.21.15.32:0/3242784053 shutdown_connections 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409e850 0x7fd8d409ec20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/3242784053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d409ffb0 0x7fd8d409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 -- 172.21.15.32:0/3242784053 >> 172.21.15.32:0/3242784053 conn(0x7fd8d40a3f70 msgr2=0x7fd8d40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 -- 172.21.15.32:0/3242784053 shutdown_connections 2023-09-30T22:28:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.108+0000 7fd8da8fd700 1 -- 172.21.15.32:0/3242784053 wait complete. 2023-09-30T22:28:14.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.109+0000 7fd8da8fd700 1 Processor -- start 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 -- start start 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d409e850 0x7fd8d411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 0x7fd8d411ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d4133b80 0x7fd8d411b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d411db60 con 0x7fd8d409e850 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4129f40 con 0x7fd8d409ffb0 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.110+0000 7fd8da8fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40a3940 con 0x7fd8d4133b80 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 0x7fd8d411ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 0x7fd8d411ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55542/0 (socket says 172.21.15.32:55542) 2023-09-30T22:28:14.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d37fe700 1 -- 172.21.15.32:0/4003601357 learned_addr learned my addr 172.21.15.32:0/4003601357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d3fff700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d409e850 0x7fd8d411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d8e9a700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d4133b80 0x7fd8d411b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d3fff700 1 -- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d4133b80 msgr2=0x7fd8d411b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d3fff700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d4133b80 0x7fd8d411b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d3fff700 1 -- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 msgr2=0x7fd8d411ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d3fff700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 0x7fd8d411ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d3fff700 1 -- 172.21.15.32:0/4003601357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c4009550 con 0x7fd8d409e850 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.111+0000 7fd8d37fe700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 0x7fd8d411ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.112+0000 7fd8d3fff700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d409e850 0x7fd8d411a650 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4016000 tx=0x7fd8c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.112+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4004510 con 0x7fd8d409e850 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.112+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c40036a0 con 0x7fd8d409e850 2023-09-30T22:28:14.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.112+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4003a50 con 0x7fd8d409e850 2023-09-30T22:28:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.112+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d411c300 con 0x7fd8d409e850 2023-09-30T22:28:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.112+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d4132610 con 0x7fd8d409e850 2023-09-30T22:28:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.113+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8c4021070 con 0x7fd8d409e850 2023-09-30T22:28:14.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.114+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd8c4040a20 con 0x7fd8d409e850 2023-09-30T22:28:14.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.115+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d40747f0 con 0x7fd8d409e850 2023-09-30T22:28:14.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:14.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.125+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8c4029080 con 0x7fd8d409e850 2023-09-30T22:28:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.158+0000 7fd7b07ee700 1 Processor -- start 2023-09-30T22:28:14.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.158+0000 7fd7b07ee700 1 -- start start 2023-09-30T22:28:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.158+0000 7fd7b07ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a80a0a30 0x7fd7a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.158+0000 7fd7b07ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 0x7fd7a80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7b07ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a80a2a70 con 0x7fd7a8133930 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7b07ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80a01d0 con 0x7fd7a80a0a30 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7b07ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a809e2b0 con 0x7fd7a809d630 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 0x7fd7a80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7aed8b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd7a8133930 0x7fd7a8133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41716/0 (socket says 172.21.15.32:41716) 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7aed8b700 1 -- 172.21.15.32:0/1052413476 learned_addr learned my addr 172.21.15.32:0/1052413476 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 0x7fd7a80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46006/0 (socket says 172.21.15.32:46006) 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7ae58a700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a80a0a30 0x7fd7a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7ad588700 1 -- 172.21.15.32:0/1052413476 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454808176 0 0) 0x7fd7a80a2a70 con 0x7fd7a8133930 2023-09-30T22:28:14.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 -- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a80a0a30 msgr2=0x7fd7a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a80a0a30 0x7fd7a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 -- 172.21.15.32:0/1052413476 >> v1:172.21.15.32:6789/0 conn(0x7fd7a8133930 legacy=0x7fd7a8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 -- 172.21.15.32:0/1052413476 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a809dc50 con 0x7fd7a809d630 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7ae58a700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a80a0a30 0x7fd7a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.159+0000 7fd7add89700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 0x7fd7a80a1340 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd798009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ef67511114a13d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7ad588700 1 -- 172.21.15.32:0/1052413476 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798011830 con 0x7fd7a809d630 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7ad588700 1 -- 172.21.15.32:0/1052413476 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798011990 con 0x7fd7a809d630 2023-09-30T22:28:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7ad588700 1 -- 172.21.15.32:0/1052413476 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798011d20 con 0x7fd7a809d630 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 -- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 msgr2=0x7fd7a80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 0x7fd7a80a1340 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd798009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 -- 172.21.15.32:0/1052413476 shutdown_connections 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a80a0a30 0x7fd7a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/1052413476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a809d630 0x7fd7a80a1340 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 -- 172.21.15.32:0/1052413476 >> 172.21.15.32:0/1052413476 conn(0x7fd7a80a3050 msgr2=0x7fd7a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 -- 172.21.15.32:0/1052413476 shutdown_connections 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.160+0000 7fd7b07ee700 1 -- 172.21.15.32:0/1052413476 wait complete. 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.161+0000 7fd7b07ee700 1 Processor -- start 2023-09-30T22:28:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.161+0000 7fd7b07ee700 1 -- start start 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7b07ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 0x7fd7a811f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7b07ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80a0a30 0x7fd7a811f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7b07ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a8133930 0x7fd7a81c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7b07ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a811ed60 con 0x7fd7a80a0a30 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7b07ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a81224a0 con 0x7fd7a809d630 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7b07ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a8120490 con 0x7fd7a8133930 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7aed8b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a8133930 0x7fd7a81c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 0x7fd7a811f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7add89700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80a0a30 0x7fd7a811f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 0x7fd7a811f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55552/0 (socket says 172.21.15.32:55552) 2023-09-30T22:28:14.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 -- 172.21.15.32:0/653551598 learned_addr learned my addr 172.21.15.32:0/653551598 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 -- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a8133930 msgr2=0x7fd7a81c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a8133930 0x7fd7a81c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 -- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80a0a30 msgr2=0x7fd7a811f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80a0a30 0x7fd7a811f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7ae58a700 1 -- 172.21.15.32:0/653551598 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40094a0 con 0x7fd7a809d630 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.162+0000 7fd7add89700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80a0a30 0x7fd7a811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.163+0000 7fd7ae58a700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 0x7fd7a811f2e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400ebe0 tx=0x7fd7a400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.163+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400c850 con 0x7fd7a809d630 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.163+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400c9b0 con 0x7fd7a809d630 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.163+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7980094a0 con 0x7fd7a809d630 2023-09-30T22:28:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.163+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400cd40 con 0x7fd7a809d630 2023-09-30T22:28:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.163+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a80a0410 con 0x7fd7a809d630 2023-09-30T22:28:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.164+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd7a4013470 con 0x7fd7a809d630 2023-09-30T22:28:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.164+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd7a403fb70 con 0x7fd7a809d630 2023-09-30T22:28:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.164+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a81222e0 con 0x7fd7a809d630 2023-09-30T22:28:14.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.169+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7a81222e0 con 0x7fd7a809d630 2023-09-30T22:28:14.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.319+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3bd409b1e0 con 0x7f3bd4134610 2023-09-30T22:28:14.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.319+0000 7f3bd17fa700 1 -- 172.21.15.32:0/289229452 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bc0015020 con 0x7f3bd4134610 2023-09-30T22:28:14.320 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:14.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd4134610 msgr2=0x7f3bd4121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd4134610 0x7f3bd4121a20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4070e10 tx=0x7f3bc000b730 comp rx=0 tx=0).stop 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 shutdown_connections 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bd4121f60 0x7f3bd41d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3bd406e570 0x7f3bd41214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 --2- 172.21.15.32:0/289229452 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3bd4134610 0x7f3bd4121a20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 >> 172.21.15.32:0/289229452 conn(0x7f3bd40a2aa0 msgr2=0x7f3bd40a0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.321+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 shutdown_connections 2023-09-30T22:28:14.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.322+0000 7f3bdaea7700 1 -- 172.21.15.32:0/289229452 wait complete. 2023-09-30T22:28:14.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.354+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd8d4072820 con 0x7fd8d409e850 2023-09-30T22:28:14.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.355+0000 7fd8d0ff9700 1 -- 172.21.15.32:0/4003601357 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8c4015020 con 0x7fd8d409e850 2023-09-30T22:28:14.356 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d409e850 msgr2=0x7fd8d411a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d409e850 0x7fd8d411a650 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4016000 tx=0x7fd8c4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:14.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 shutdown_connections 2023-09-30T22:28:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd8d4133b80 0x7fd8d411b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d409e850 0x7fd8d411a650 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 --2- 172.21.15.32:0/4003601357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8d409ffb0 0x7fd8d411ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 >> 172.21.15.32:0/4003601357 conn(0x7fd8d40a3f70 msgr2=0x7fd8d40a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 shutdown_connections 2023-09-30T22:28:14.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.357+0000 7fd8da8fd700 1 -- 172.21.15.32:0/4003601357 wait complete. 2023-09-30T22:28:14.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:14.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.388+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd7a81222e0 con 0x7fd7a809d630 2023-09-30T22:28:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.389+0000 7fd79effd700 1 -- 172.21.15.32:0/653551598 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7a81222e0 con 0x7fd7a809d630 2023-09-30T22:28:14.390 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:14.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 msgr2=0x7fd7a811f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:14.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 0x7fd7a811f2e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400ebe0 tx=0x7fd7a400eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 shutdown_connections 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd7a8133930 0x7fd7a81c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7a809d630 0x7fd7a811f2e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 --2- 172.21.15.32:0/653551598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7a80a0a30 0x7fd7a811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 >> 172.21.15.32:0/653551598 conn(0x7fd7a80a3050 msgr2=0x7fd7a81bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 shutdown_connections 2023-09-30T22:28:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:14.391+0000 7fd7b07ee700 1 -- 172.21.15.32:0/653551598 wait complete. 2023-09-30T22:28:14.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:15.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.267+0000 7f6a979be700 1 Processor -- start 2023-09-30T22:28:15.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.267+0000 7f6a979be700 1 -- start start 2023-09-30T22:28:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a979be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009d6f0 0x7f6a9009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a979be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e410 0x7f6a90130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a979be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9009f130 con 0x7f6a9009e000 2023-09-30T22:28:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a979be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9009f520 con 0x7f6a9009e410 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a979be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9009e930 con 0x7f6a9009d6f0 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a94f59700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a9009e000 0x7f6a90130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41732/0 (socket says 172.21.15.32:41732) 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a94f59700 1 -- 172.21.15.32:0/2251645478 learned_addr learned my addr 172.21.15.32:0/2251645478 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a9575a700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009d6f0 0x7f6a9009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a95f5b700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e410 0x7f6a90130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a83fff700 1 -- 172.21.15.32:0/2251645478 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107287906 0 0) 0x7f6a9009f130 con 0x7f6a9009e000 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a83fff700 1 -- 172.21.15.32:0/2251645478 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a7c0035b0 con 0x7f6a9009e000 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a95f5b700 1 -- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009d6f0 msgr2=0x7f6a9009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a95f5b700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009d6f0 0x7f6a9009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a95f5b700 1 -- 172.21.15.32:0/2251645478 >> v1:172.21.15.32:6789/0 conn(0x7f6a9009e000 legacy=0x7f6a90130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a95f5b700 1 -- 172.21.15.32:0/2251645478 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9009b2d0 con 0x7f6a9009e410 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.268+0000 7f6a9575a700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009d6f0 0x7f6a9009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a95f5b700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e410 0x7f6a90130d70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009dc0 tx=0x7f6a8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18aa27a633a7ab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a83fff700 1 -- 172.21.15.32:0/2251645478 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800e070 con 0x7f6a9009e410 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a83fff700 1 -- 172.21.15.32:0/2251645478 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8800bb20 con 0x7f6a9009e410 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a83fff700 1 -- 172.21.15.32:0/2251645478 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88012430 con 0x7f6a9009e410 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a979be700 1 -- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e410 msgr2=0x7f6a90130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a979be700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e410 0x7f6a90130d70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009dc0 tx=0x7f6a8800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:15.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a979be700 1 -- 172.21.15.32:0/2251645478 shutdown_connections 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a979be700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e410 0x7f6a90130d70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a979be700 1 --2- 172.21.15.32:0/2251645478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009d6f0 0x7f6a9009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.269+0000 7f6a979be700 1 -- 172.21.15.32:0/2251645478 >> 172.21.15.32:0/2251645478 conn(0x7f6a90066f50 msgr2=0x7f6a9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.270+0000 7f6a979be700 1 -- 172.21.15.32:0/2251645478 shutdown_connections 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.270+0000 7f6a979be700 1 -- 172.21.15.32:0/2251645478 wait complete. 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.270+0000 7f6a979be700 1 Processor -- start 2023-09-30T22:28:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 -- start start 2023-09-30T22:28:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 0x7f6a9011ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 0x7f6a90120500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009e410 0x7f6a90120a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a90130330 con 0x7f6a9009d6f0 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901234f0 con 0x7f6a9009e000 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a979be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901223e0 con 0x7f6a9009e410 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 0x7f6a9011ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a94f59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 0x7f6a90120500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a95f5b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009e410 0x7f6a90120a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 0x7f6a9011ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54076/0 (socket says 172.21.15.32:54076) 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 -- 172.21.15.32:0/1826989568 learned_addr learned my addr 172.21.15.32:0/1826989568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a94f59700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 0x7f6a90120500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55576/0 (socket says 172.21.15.32:55576) 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a95f5b700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009e410 0x7f6a90120a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46042/0 (socket says 172.21.15.32:46042) 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 -- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009e410 msgr2=0x7f6a90120a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009e410 0x7f6a90120a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 -- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 msgr2=0x7f6a90120500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 0x7f6a90120500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.271+0000 7f6a9575a700 1 -- 172.21.15.32:0/1826989568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c0094a0 con 0x7f6a9009d6f0 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a94f59700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 0x7f6a90120500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a9575a700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 0x7f6a9011ffc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00ec10 tx=0x7f6a8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00c850 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c00c9b0 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a88009a70 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00cd20 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.272+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9009c700 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.273+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a8c013470 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.273+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6a8c03f7f0 con 0x7f6a9009d6f0 2023-09-30T22:28:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.273+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a90074ea0 con 0x7f6a9009d6f0 2023-09-30T22:28:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.278+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a8c013720 con 0x7f6a9009d6f0 2023-09-30T22:28:15.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:15.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:15.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:15.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.485+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6a9006c360 con 0x7f6a9009d6f0 2023-09-30T22:28:15.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.486+0000 7f6a81ffb700 1 -- 172.21.15.32:0/1826989568 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a8c01bb60 con 0x7f6a9009d6f0 2023-09-30T22:28:15.487 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:15.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 Processor -- start 2023-09-30T22:28:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 msgr2=0x7f6a9011ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7f6a979be700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 0x7f6a9011ffc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00ec10 tx=0x7f6a8c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:15.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 -- start start 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412cf60 0x7fe7b406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7b4130430 0x7fe7b4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b412cd90 con 0x7fe7b412f8b0 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b412d330 con 0x7fe7b4130430 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b980e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4071780 con 0x7fe7b412cf60 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.488+0000 7fe7b37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7b4130430 0x7fe7b4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 shutdown_connections 2023-09-30T22:28:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6a9009e410 0x7f6a90120a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a9009d6f0 0x7f6a9011ffc0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 --2- 172.21.15.32:0/1826989568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a9009e000 0x7f6a90120500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 >> 172.21.15.32:0/1826989568 conn(0x7f6a90066f50 msgr2=0x7f6a90071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7b4130430 0x7fe7b4137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54086/0 (socket says 172.21.15.32:54086) 2023-09-30T22:28:15.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 -- 172.21.15.32:0/332267605 learned_addr learned my addr 172.21.15.32:0/332267605 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 shutdown_connections 2023-09-30T22:28:15.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7f6a979be700 1 -- 172.21.15.32:0/1826989568 wait complete. 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 -- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412cf60 msgr2=0x7fe7b406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 --2- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412cf60 0x7fe7b406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 -- 172.21.15.32:0/332267605 >> v1:172.21.15.32:6789/0 conn(0x7fe7b412f8b0 legacy=0x7fe7b412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 -- 172.21.15.32:0/332267605 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4071a40 con 0x7fe7b4130430 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b37fe700 1 --2- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b4137c30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009e50 tx=0x7fe7ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c56608a24608f52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b1ffb700 1 -- 172.21.15.32:0/332267605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac00b9c0 con 0x7fe7b4130430 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b1ffb700 1 -- 172.21.15.32:0/332267605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac00bb20 con 0x7fe7b4130430 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.489+0000 7fe7b1ffb700 1 -- 172.21.15.32:0/332267605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011400 con 0x7fe7b4130430 2023-09-30T22:28:15.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 -- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 msgr2=0x7fe7b4137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 --2- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b4137c30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009e50 tx=0x7fe7ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 -- 172.21.15.32:0/332267605 shutdown_connections 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 --2- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b4137c30 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 --2- 172.21.15.32:0/332267605 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412cf60 0x7fe7b406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 -- 172.21.15.32:0/332267605 >> 172.21.15.32:0/332267605 conn(0x7fe7b40a2aa0 msgr2=0x7fe7b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.490+0000 7fe7b980e700 1 -- 172.21.15.32:0/332267605 shutdown_connections 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.491+0000 7fe7b980e700 1 -- 172.21.15.32:0/332267605 wait complete. 2023-09-30T22:28:15.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.492+0000 7fe7b980e700 1 Processor -- start 2023-09-30T22:28:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.492+0000 7fe7b980e700 1 -- start start 2023-09-30T22:28:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b980e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7b412cf60 0x7fe7b411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b980e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412f8b0 0x7fe7b411ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b980e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b411b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b980e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b411bb00 con 0x7fe7b4130430 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b980e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b411e0e0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b980e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b411db40 con 0x7fe7b412cf60 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7b412cf60 0x7fe7b411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7b412cf60 0x7fe7b411a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46044/0 (socket says 172.21.15.32:46044) 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b2ffd700 1 -- 172.21.15.32:0/852373210 learned_addr learned my addr 172.21.15.32:0/852373210 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.493+0000 7fe7b27fc700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412f8b0 0x7fe7b411ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b37fe700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b411b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b27fc700 1 -- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7b412cf60 msgr2=0x7fe7b411a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b27fc700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7b412cf60 0x7fe7b411a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b27fc700 1 -- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 msgr2=0x7fe7b411b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b27fc700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b411b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b27fc700 1 -- 172.21.15.32:0/852373210 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac009b30 con 0x7fe7b412f8b0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b37fe700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b411b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b27fc700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412f8b0 0x7fe7b411ab90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8000f60 tx=0x7fe7a800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80050d0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a8005230 con 0x7fe7b412f8b0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b412d7b0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.494+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8003680 con 0x7fe7b412f8b0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.495+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b4129ff0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.495+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe7a80037e0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.496+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe7a803ea60 con 0x7fe7b412f8b0 2023-09-30T22:28:15.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.496+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b411c270 con 0x7fe7b412f8b0 2023-09-30T22:28:15.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.501+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7a8011590 con 0x7fe7b412f8b0 2023-09-30T22:28:15.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 Processor -- start 2023-09-30T22:28:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 -- start start 2023-09-30T22:28:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c131810 0x7f633c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 0x7f633c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c1313c0 con 0x7f633c099a40 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c06e8c0 con 0x7f633c09a580 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.552+0000 7f6344247700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f633c071d20 con 0x7f633c131810 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 0x7f633c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63417e2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f633c099a40 0x7f633c099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41746/0 (socket says 172.21.15.32:41746) 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 0x7f633c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55598/0 (socket says 172.21.15.32:55598) 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f6341fe3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c131810 0x7f633c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63417e2700 1 -- 172.21.15.32:0/444687729 learned_addr learned my addr 172.21.15.32:0/444687729 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 -- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c131810 msgr2=0x7f633c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 --2- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c131810 0x7f633c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 -- 172.21.15.32:0/444687729 >> v1:172.21.15.32:6789/0 conn(0x7f633c099a40 legacy=0x7f633c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 -- 172.21.15.32:0/444687729 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0721a0 con 0x7f633c09a580 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f6341fe3700 1 --2- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c131810 0x7f633c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f63427e4700 1 --2- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 0x7f633c139660 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6334009dc0 tx=0x7f633400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b953360726beb66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f6340fe1700 1 -- 172.21.15.32:0/444687729 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400e070 con 0x7f633c09a580 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f6340fe1700 1 -- 172.21.15.32:0/444687729 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633400bb20 con 0x7f633c09a580 2023-09-30T22:28:15.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.553+0000 7f6340fe1700 1 -- 172.21.15.32:0/444687729 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334012430 con 0x7f633c09a580 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 -- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 msgr2=0x7f633c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 --2- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 0x7f633c139660 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6334009dc0 tx=0x7f633400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 -- 172.21.15.32:0/444687729 shutdown_connections 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 --2- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c09a580 0x7f633c139660 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 --2- 172.21.15.32:0/444687729 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c131810 0x7f633c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 -- 172.21.15.32:0/444687729 >> 172.21.15.32:0/444687729 conn(0x7f633c0a2aa0 msgr2=0x7f633c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 -- 172.21.15.32:0/444687729 shutdown_connections 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.554+0000 7f6344247700 1 -- 172.21.15.32:0/444687729 wait complete. 2023-09-30T22:28:15.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.556+0000 7f6344247700 1 Processor -- start 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.556+0000 7f6344247700 1 -- start start 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6344247700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c099a40 0x7f633c06f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6344247700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c09a580 0x7f633c06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6344247700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c131810 0x7f633c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6344247700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0a1e30 con 0x7f633c131810 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6344247700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c120200 con 0x7f633c099a40 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6344247700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1207d0 con 0x7f633c09a580 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f63417e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c09a580 0x7f633c06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6341fe3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c099a40 0x7f633c06f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f63417e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c09a580 0x7f633c06fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46068/0 (socket says 172.21.15.32:46068) 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f63417e2700 1 -- 172.21.15.32:0/2509867457 learned_addr learned my addr 172.21.15.32:0/2509867457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:15.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f63427e4700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c131810 0x7f633c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6341fe3700 1 -- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c09a580 msgr2=0x7f633c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6341fe3700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c09a580 0x7f633c06fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6341fe3700 1 -- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c131810 msgr2=0x7f633c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6341fe3700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c131810 0x7f633c0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f6341fe3700 1 -- 172.21.15.32:0/2509867457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380094a0 con 0x7f633c099a40 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.557+0000 7f63427e4700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c131810 0x7f633c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.558+0000 7f6341fe3700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c099a40 0x7f633c06f660 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f633800e9b0 tx=0x7f633800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.558+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800ee50 con 0x7f633c099a40 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.558+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633800c850 con 0x7f633c099a40 2023-09-30T22:28:15.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.558+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800cc00 con 0x7f633c099a40 2023-09-30T22:28:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.558+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6334009a70 con 0x7f633c099a40 2023-09-30T22:28:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.558+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c121400 con 0x7f633c099a40 2023-09-30T22:28:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.559+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6338013470 con 0x7f633c099a40 2023-09-30T22:28:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.559+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f633803f6c0 con 0x7f633c099a40 2023-09-30T22:28:15.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.559+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c06eb60 con 0x7f633c099a40 2023-09-30T22:28:15.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.564+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6338013720 con 0x7f633c099a40 2023-09-30T22:28:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.570+0000 7fa182a49700 1 Processor -- start 2023-09-30T22:28:15.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.570+0000 7fa182a49700 1 -- start start 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa182a49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa17c09f410 0x7fa17c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa182a49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa182a49700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c070940 con 0x7fa17c09e930 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa182a49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c06fd60 con 0x7fa17c09f410 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa182a49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c06fa80 con 0x7fa17c136fe0 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa17bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa17c09e930 0x7fa17c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41762/0 (socket says 172.21.15.32:41762) 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa17bfff700 1 -- 172.21.15.32:0/1479687184 learned_addr learned my addr 172.21.15.32:0/1479687184 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa180fe6700 1 --2- 172.21.15.32:0/1479687184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa17b7fe700 1 --2- 172.21.15.32:0/1479687184 >> v2:172.21.15.32:3300/0 conn(0x7fa17c09f410 0x7fa17c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa17affd700 1 -- 172.21.15.32:0/1479687184 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056608922 0 0) 0x7fa17c070940 con 0x7fa17c09e930 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa180fe6700 1 -- 172.21.15.32:0/1479687184 >> v2:172.21.15.32:3300/0 conn(0x7fa17c09f410 msgr2=0x7fa17c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa180fe6700 1 --2- 172.21.15.32:0/1479687184 >> v2:172.21.15.32:3300/0 conn(0x7fa17c09f410 0x7fa17c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa180fe6700 1 -- 172.21.15.32:0/1479687184 >> v1:172.21.15.32:6789/0 conn(0x7fa17c09e930 legacy=0x7fa17c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa180fe6700 1 -- 172.21.15.32:0/1479687184 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c09fb60 con 0x7fa17c136fe0 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.571+0000 7fa180fe6700 1 --2- 172.21.15.32:0/1479687184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c1393c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa174009d90 tx=0x7fa17400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac659ad7d3a743d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa17affd700 1 -- 172.21.15.32:0/1479687184 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400e070 con 0x7fa17c136fe0 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa17affd700 1 -- 172.21.15.32:0/1479687184 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400bb20 con 0x7fa17c136fe0 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa17affd700 1 -- 172.21.15.32:0/1479687184 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174012430 con 0x7fa17c136fe0 2023-09-30T22:28:15.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa182a49700 1 -- 172.21.15.32:0/1479687184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 msgr2=0x7fa17c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa182a49700 1 --2- 172.21.15.32:0/1479687184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c1393c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa174009d90 tx=0x7fa17400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa182a49700 1 -- 172.21.15.32:0/1479687184 shutdown_connections 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa182a49700 1 --2- 172.21.15.32:0/1479687184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c1393c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa182a49700 1 --2- 172.21.15.32:0/1479687184 >> v2:172.21.15.32:3300/0 conn(0x7fa17c09f410 0x7fa17c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.572+0000 7fa182a49700 1 -- 172.21.15.32:0/1479687184 >> 172.21.15.32:0/1479687184 conn(0x7fa17c0a4210 msgr2=0x7fa17c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.573+0000 7fa182a49700 1 -- 172.21.15.32:0/1479687184 shutdown_connections 2023-09-30T22:28:15.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.573+0000 7fa182a49700 1 -- 172.21.15.32:0/1479687184 wait complete. 2023-09-30T22:28:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.573+0000 7fa182a49700 1 Processor -- start 2023-09-30T22:28:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 -- start start 2023-09-30T22:28:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17c09e930 0x7fa17c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa17c09f410 0x7fa17c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:15.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c06d560 con 0x7fa17c09e930 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c121df0 con 0x7fa17c136fe0 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa182a49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c09de30 con 0x7fa17c09f410 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa180fe6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa180fe6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55630/0 (socket says 172.21.15.32:55630) 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa180fe6700 1 -- 172.21.15.32:0/4038449266 learned_addr learned my addr 172.21.15.32:0/4038449266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa17b7fe700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa17c09f410 0x7fa17c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.574+0000 7fa17bfff700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17c09e930 0x7fa17c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa180fe6700 1 -- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa17c09f410 msgr2=0x7fa17c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa180fe6700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa17c09f410 0x7fa17c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa180fe6700 1 -- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17c09e930 msgr2=0x7fa17c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa180fe6700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17c09e930 0x7fa17c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa180fe6700 1 -- 172.21.15.32:0/4038449266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009a70 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa180fe6700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c06c330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa17c06fbc0 tx=0x7fa174004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174012430 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174020070 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c071d20 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174004280 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.575+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c1220d0 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.576+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa1740043e0 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.576+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa174022070 con 0x7fa17c136fe0 2023-09-30T22:28:15.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.576+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c074ea0 con 0x7fa17c136fe0 2023-09-30T22:28:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.581+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa17401b400 con 0x7fa17c136fe0 2023-09-30T22:28:15.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.742+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe7b4074ea0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.742+0000 7fe79bfff700 1 -- 172.21.15.32:0/852373210 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe7a80192c0 con 0x7fe7b412f8b0 2023-09-30T22:28:15.743 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412f8b0 msgr2=0x7fe7b411ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412f8b0 0x7fe7b411ab90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8000f60 tx=0x7fe7a800c730 comp rx=0 tx=0).stop 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 shutdown_connections 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7b4130430 0x7fe7b411b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7b412cf60 0x7fe7b411a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 --2- 172.21.15.32:0/852373210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7b412f8b0 0x7fe7b411ab90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.744+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 >> 172.21.15.32:0/852373210 conn(0x7fe7b40a2aa0 msgr2=0x7fe7b409e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.745+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 shutdown_connections 2023-09-30T22:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.745+0000 7fe7b980e700 1 -- 172.21.15.32:0/852373210 wait complete. 2023-09-30T22:28:15.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:15.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.782+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f633c074ea0 con 0x7f633c099a40 2023-09-30T22:28:15.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.782+0000 7f63327fc700 1 -- 172.21.15.32:0/2509867457 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6338021030 con 0x7f633c099a40 2023-09-30T22:28:15.783 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c099a40 msgr2=0x7f633c06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c099a40 0x7f633c06f660 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f633800e9b0 tx=0x7f633800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 shutdown_connections 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c131810 0x7f633c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f633c099a40 0x7f633c06f660 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 --2- 172.21.15.32:0/2509867457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f633c09a580 0x7f633c06fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 >> 172.21.15.32:0/2509867457 conn(0x7f633c0a2aa0 msgr2=0x7f633c09d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 shutdown_connections 2023-09-30T22:28:15.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.785+0000 7f6344247700 1 -- 172.21.15.32:0/2509867457 wait complete. 2023-09-30T22:28:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.787+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa17c120990 con 0x7fa17c136fe0 2023-09-30T22:28:15.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.787+0000 7fa178ff9700 1 -- 172.21.15.32:0/4038449266 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa17401a380 con 0x7fa17c136fe0 2023-09-30T22:28:15.788 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 msgr2=0x7fa17c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c06c330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa17c06fbc0 tx=0x7fa174004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 shutdown_connections 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa17c136fe0 0x7fa17c06c330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa17c09e930 0x7fa17c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 --2- 172.21.15.32:0/4038449266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa17c09f410 0x7fa17c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 >> 172.21.15.32:0/4038449266 conn(0x7fa17c0a4210 msgr2=0x7fa17c134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 shutdown_connections 2023-09-30T22:28:15.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:15.790+0000 7fa182a49700 1 -- 172.21.15.32:0/4038449266 wait complete. 2023-09-30T22:28:15.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:15.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:16.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.654+0000 7fde9f666700 1 Processor -- start 2023-09-30T22:28:16.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.654+0000 7fde9f666700 1 -- start start 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9f666700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9f666700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde9809e000 0x7fde981339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9f666700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde981343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9f666700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9809b0f0 con 0x7fde9809e000 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9f666700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9809d2f0 con 0x7fde98133f30 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9f666700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9809a810 con 0x7fde9809d6f0 2023-09-30T22:28:16.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9d402700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9d402700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46090/0 (socket says 172.21.15.32:46090) 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9d402700 1 -- 172.21.15.32:0/4031641390 learned_addr learned my addr 172.21.15.32:0/4031641390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9cc01700 1 --2- 172.21.15.32:0/4031641390 >> v2:172.21.15.32:3300/0 conn(0x7fde9809e000 0x7fde981339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.655+0000 7fde9dc03700 1 --2- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde981343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde9d402700 1 -- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 msgr2=0x7fde981343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde9d402700 1 --2- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde981343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde9d402700 1 -- 172.21.15.32:0/4031641390 >> v2:172.21.15.32:3300/0 conn(0x7fde9809e000 msgr2=0x7fde981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde9d402700 1 --2- 172.21.15.32:0/4031641390 >> v2:172.21.15.32:3300/0 conn(0x7fde9809e000 0x7fde981339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde9d402700 1 -- 172.21.15.32:0/4031641390 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9809a470 con 0x7fde9809d6f0 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde9d402700 1 --2- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9809dac0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fde940098f0 tx=0x7fde94009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6cd3057d732aa6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde8ffff700 1 -- 172.21.15.32:0/4031641390 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94011830 con 0x7fde9809d6f0 2023-09-30T22:28:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde8ffff700 1 -- 172.21.15.32:0/4031641390 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde94011990 con 0x7fde9809d6f0 2023-09-30T22:28:16.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.656+0000 7fde8ffff700 1 -- 172.21.15.32:0/4031641390 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94011d20 con 0x7fde9809d6f0 2023-09-30T22:28:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 -- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 msgr2=0x7fde9809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 --2- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9809dac0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fde940098f0 tx=0x7fde94009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:16.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 -- 172.21.15.32:0/4031641390 shutdown_connections 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 --2- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde981343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 --2- 172.21.15.32:0/4031641390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9809dac0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 --2- 172.21.15.32:0/4031641390 >> v2:172.21.15.32:3300/0 conn(0x7fde9809e000 0x7fde981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 -- 172.21.15.32:0/4031641390 >> 172.21.15.32:0/4031641390 conn(0x7fde98066f50 msgr2=0x7fde9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 -- 172.21.15.32:0/4031641390 shutdown_connections 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.657+0000 7fde9f666700 1 -- 172.21.15.32:0/4031641390 wait complete. 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.658+0000 7fde9f666700 1 Processor -- start 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.658+0000 7fde9f666700 1 -- start start 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9f666700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9811d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9f666700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9809e000 0x7fde9811ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9f666700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde9811e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9f666700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98120050 con 0x7fde9809e000 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9f666700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde98120aa0 con 0x7fde98133f30 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9f666700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde98120ca0 con 0x7fde9809d6f0 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9dc03700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde9811e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9811d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46092/0 (socket says 172.21.15.32:46092) 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 -- 172.21.15.32:0/3425986669 learned_addr learned my addr 172.21.15.32:0/3425986669 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9dc03700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde9811e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55656/0 (socket says 172.21.15.32:55656) 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9cc01700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9809e000 0x7fde9811ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 -- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 msgr2=0x7fde9811e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde9811e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 -- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9809e000 msgr2=0x7fde9811ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9809e000 0x7fde9811ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 -- 172.21.15.32:0/3425986669 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94009550 con 0x7fde9809d6f0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde9d402700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9811d890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fde94009c40 tx=0x7fde94004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94004240 con 0x7fde9809d6f0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.659+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9401f070 con 0x7fde9809d6f0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.660+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9811fb90 con 0x7fde9809d6f0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.660+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde940045e0 con 0x7fde9809d6f0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.660+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9809c640 con 0x7fde9809d6f0 2023-09-30T22:28:16.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.661+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde981343b0 con 0x7fde9809d6f0 2023-09-30T22:28:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.661+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde94021030 con 0x7fde9809d6f0 2023-09-30T22:28:16.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.661+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde940406d0 con 0x7fde9809d6f0 2023-09-30T22:28:16.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.666+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde940197c0 con 0x7fde9809d6f0 2023-09-30T22:28:16.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:16.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:16.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.872+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fde98074ea0 con 0x7fde9809d6f0 2023-09-30T22:28:16.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.872+0000 7fde8dffb700 1 -- 172.21.15.32:0/3425986669 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde94015020 con 0x7fde9809d6f0 2023-09-30T22:28:16.874 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 msgr2=0x7fde9811d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9811d890 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fde94009c40 tx=0x7fde94004750 comp rx=0 tx=0).stop 2023-09-30T22:28:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 shutdown_connections 2023-09-30T22:28:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fde98133f30 0x7fde9811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fde9809d6f0 0x7fde9811d890 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 --2- 172.21.15.32:0/3425986669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde9809e000 0x7fde9811ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 >> 172.21.15.32:0/3425986669 conn(0x7fde98066f50 msgr2=0x7fde98131e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:16.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 shutdown_connections 2023-09-30T22:28:16.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.875+0000 7fde9f666700 1 -- 172.21.15.32:0/3425986669 wait complete. 2023-09-30T22:28:16.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.908+0000 7fa26e07f700 1 Processor -- start 2023-09-30T22:28:16.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.909+0000 7fa26e07f700 1 -- start start 2023-09-30T22:28:16.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.909+0000 7fa26e07f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.909+0000 7fa26e07f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa26806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.909+0000 7fa26e07f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa268130260 con 0x7fa26806ffd0 2023-09-30T22:28:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.909+0000 7fa26e07f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26809a350 con 0x7fa268130670 2023-09-30T22:28:16.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.909+0000 7fa26e07f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa268071ff0 con 0x7fa26806f680 2023-09-30T22:28:16.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa266ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa26806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55662/0 (socket says 172.21.15.32:55662) 2023-09-30T22:28:16.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa266ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa26806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46102/0 (socket says 172.21.15.32:46102) 2023-09-30T22:28:16.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 -- 172.21.15.32:0/4082818225 learned_addr learned my addr 172.21.15.32:0/4082818225 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 -- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 msgr2=0x7fa26806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 --2- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa26806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 -- 172.21.15.32:0/4082818225 >> v1:172.21.15.32:6789/0 conn(0x7fa26806ffd0 legacy=0x7fa268137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 -- 172.21.15.32:0/4082818225 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268070470 con 0x7fa268130670 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2677fe700 1 --2- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268130a40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa258009920 tx=0x7fa258009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c75a193cbd25dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2667fc700 1 -- 172.21.15.32:0/4082818225 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258011830 con 0x7fa268130670 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2667fc700 1 -- 172.21.15.32:0/4082818225 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa258011990 con 0x7fa268130670 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.910+0000 7fa2667fc700 1 -- 172.21.15.32:0/4082818225 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258011d20 con 0x7fa268130670 2023-09-30T22:28:16.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 -- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 msgr2=0x7fa268130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 --2- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268130a40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa258009920 tx=0x7fa258009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 -- 172.21.15.32:0/4082818225 shutdown_connections 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 --2- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268130a40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 --2- 172.21.15.32:0/4082818225 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa26806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 -- 172.21.15.32:0/4082818225 >> 172.21.15.32:0/4082818225 conn(0x7fa2680a2aa0 msgr2=0x7fa2680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 -- 172.21.15.32:0/4082818225 shutdown_connections 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.911+0000 7fa26e07f700 1 -- 172.21.15.32:0/4082818225 wait complete. 2023-09-30T22:28:16.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.913+0000 7fa26e07f700 1 Processor -- start 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.913+0000 7fa26e07f700 1 -- start start 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa26e07f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa2681c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa26e07f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26806ffd0 0x7fa2681c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa26e07f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa26e07f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa268118e70 con 0x7fa26806ffd0 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa26e07f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26812de90 con 0x7fa268130670 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa26e07f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa268138f80 con 0x7fa26806f680 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa2681c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa2681c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46118/0 (socket says 172.21.15.32:46118) 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 -- 172.21.15.32:0/2213623645 learned_addr learned my addr 172.21.15.32:0/2213623645 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa266ffd700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26806ffd0 0x7fa2681c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa267fff700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 -- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 msgr2=0x7fa268139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 -- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26806ffd0 msgr2=0x7fa2681c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26806ffd0 0x7fa2681c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa2677fe700 1 -- 172.21.15.32:0/2213623645 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009550 con 0x7fa26806f680 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.914+0000 7fa266ffd700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26806ffd0 0x7fa2681c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:16.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.915+0000 7fa2677fe700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa2681c8770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa25800a010 tx=0x7fa258004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.915+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258004450 con 0x7fa26806f680 2023-09-30T22:28:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.915+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2580045b0 con 0x7fa26806f680 2023-09-30T22:28:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.915+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26812e620 con 0x7fa26806f680 2023-09-30T22:28:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.915+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2580038e0 con 0x7fa26806f680 2023-09-30T22:28:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.915+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26813bc40 con 0x7fa26806f680 2023-09-30T22:28:16.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.916+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa258003a40 con 0x7fa26806f680 2023-09-30T22:28:16.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.916+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa268074ea0 con 0x7fa26806f680 2023-09-30T22:28:16.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.916+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa2580407d0 con 0x7fa26806f680 2023-09-30T22:28:16.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.921+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa258015080 con 0x7fa26806f680 2023-09-30T22:28:16.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.948+0000 7f7dfb0c3700 1 Processor -- start 2023-09-30T22:28:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.948+0000 7f7dfb0c3700 1 -- start start 2023-09-30T22:28:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7dfb0c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7df409e930 0x7f7df409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7dfb0c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 0x7f7df41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7dfb0c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f650 0x7f7df4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7dfb0c3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df4070940 con 0x7f7df409e930 2023-09-30T22:28:16.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7dfb0c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df406fd60 con 0x7f7df409f650 2023-09-30T22:28:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7dfb0c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df406fa80 con 0x7f7df409f240 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 0x7f7df41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 0x7f7df41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46132/0 (socket says 172.21.15.32:46132) 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 -- 172.21.15.32:0/1859363984 learned_addr learned my addr 172.21.15.32:0/1859363984 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df8e5f700 1 --2- 172.21.15.32:0/1859363984 >> v2:172.21.15.32:3300/0 conn(0x7f7df409e930 0x7f7df409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df9660700 1 --2- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f650 0x7f7df4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 -- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f650 msgr2=0x7f7df4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 --2- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f650 0x7f7df4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 -- 172.21.15.32:0/1859363984 >> v2:172.21.15.32:3300/0 conn(0x7f7df409e930 msgr2=0x7f7df409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 --2- 172.21.15.32:0/1859363984 >> v2:172.21.15.32:3300/0 conn(0x7f7df409e930 0x7f7df409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 -- 172.21.15.32:0/1859363984 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df409fb60 con 0x7f7df409f240 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df8e5f700 1 --2- 172.21.15.32:0/1859363984 >> v2:172.21.15.32:3300/0 conn(0x7f7df409e930 0x7f7df409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.949+0000 7f7df3fff700 1 --2- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 0x7f7df41317f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7de00098f0 tx=0x7f7de0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb083a774667ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7df37fe700 1 -- 172.21.15.32:0/1859363984 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0011830 con 0x7f7df409f240 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7df37fe700 1 -- 172.21.15.32:0/1859363984 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0011990 con 0x7f7df409f240 2023-09-30T22:28:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7df37fe700 1 -- 172.21.15.32:0/1859363984 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0011d20 con 0x7f7df409f240 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 msgr2=0x7f7df41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 0x7f7df41317f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7de00098f0 tx=0x7f7de0009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/1859363984 shutdown_connections 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f650 0x7f7df4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/1859363984 >> v2:172.21.15.32:3300/0 conn(0x7f7df409e930 0x7f7df409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/1859363984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409f240 0x7f7df41317f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/1859363984 >> 172.21.15.32:0/1859363984 conn(0x7f7df40a4210 msgr2=0x7f7df40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.950+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/1859363984 shutdown_connections 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.951+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/1859363984 wait complete. 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.951+0000 7f7dfb0c3700 1 Processor -- start 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 -- start start 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 0x7f7df41a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f240 0x7f7df41a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df409f650 0x7f7df41a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41a58a0 con 0x7f7df409f650 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41a52c0 con 0x7f7df409f240 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7dfb0c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7df41a7310 con 0x7f7df409e930 2023-09-30T22:28:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7df8e5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 0x7f7df41a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7df8e5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 0x7f7df41a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46140/0 (socket says 172.21.15.32:46140) 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7df8e5f700 1 -- 172.21.15.32:0/3596022945 learned_addr learned my addr 172.21.15.32:0/3596022945 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.952+0000 7f7df9660700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df409f650 0x7f7df41a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df3fff700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f240 0x7f7df41a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df8e5f700 1 -- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f240 msgr2=0x7f7df41a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df8e5f700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f240 0x7f7df41a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df8e5f700 1 -- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df409f650 msgr2=0x7f7df41a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df8e5f700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df409f650 0x7f7df41a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df8e5f700 1 -- 172.21.15.32:0/3596022945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80094a0 con 0x7f7df409e930 2023-09-30T22:28:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7fbdde426700 1 Processor -- start 2023-09-30T22:28:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df9660700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df409f650 0x7f7df41a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df8e5f700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 0x7f7df41a3e10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7de800ec10 tx=0x7f7de800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7fbdde426700 1 -- start start 2023-09-30T22:28:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8009780 con 0x7f7df409e930 2023-09-30T22:28:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de800c850 con 0x7f7df409e930 2023-09-30T22:28:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de800cbe0 con 0x7f7df409e930 2023-09-30T22:28:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de0009550 con 0x7f7df409e930 2023-09-30T22:28:16.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.953+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df40a0190 con 0x7f7df409e930 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbdde426700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8131140 0x7fbdd809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbdde426700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd809e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbdde426700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd809f130 con 0x7fbdd8130660 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbdde426700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd809f520 con 0x7fbdd8131140 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbdde426700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd809e930 con 0x7fbdd809dc30 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd809e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd809e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46148/0 (socket says 172.21.15.32:46148) 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 -- 172.21.15.32:0/1954030915 learned_addr learned my addr 172.21.15.32:0/1954030915 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 -- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8131140 msgr2=0x7fbdd809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 --2- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8131140 0x7fbdd809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 -- 172.21.15.32:0/1954030915 >> v1:172.21.15.32:6789/0 conn(0x7fbdd8130660 legacy=0x7fbdd8130a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7fbddc9c3700 1 -- 172.21.15.32:0/1954030915 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd809b2d0 con 0x7fbdd809dc30 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.954+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7de800cd40 con 0x7f7df409e930 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.955+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7de803f810 con 0x7f7df409e930 2023-09-30T22:28:16.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.955+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df41a0b30 con 0x7f7df409e930 2023-09-30T22:28:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.955+0000 7fbddc9c3700 1 --2- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd809e0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009e80 tx=0x7fbdd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a48b2f8ec9adc3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.960+0000 7fbdd6ffd700 1 -- 172.21.15.32:0/1954030915 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000e070 con 0x7fbdd809dc30 2023-09-30T22:28:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.960+0000 7fbdd6ffd700 1 -- 172.21.15.32:0/1954030915 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd000b9c0 con 0x7fbdd809dc30 2023-09-30T22:28:16.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.960+0000 7fbdd6ffd700 1 -- 172.21.15.32:0/1954030915 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000bd70 con 0x7fbdd809dc30 2023-09-30T22:28:16.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.960+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7de8017080 con 0x7f7df409e930 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.960+0000 7fbdde426700 1 -- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 msgr2=0x7fbdd809e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.960+0000 7fbdde426700 1 --2- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd809e0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009e80 tx=0x7fbdd000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.961+0000 7fbdde426700 1 -- 172.21.15.32:0/1954030915 shutdown_connections 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.961+0000 7fbdde426700 1 --2- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd809e0b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.961+0000 7fbdde426700 1 --2- 172.21.15.32:0/1954030915 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8131140 0x7fbdd809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.961+0000 7fbdde426700 1 -- 172.21.15.32:0/1954030915 >> 172.21.15.32:0/1954030915 conn(0x7fbdd8066f50 msgr2=0x7fbdd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.961+0000 7fbdde426700 1 -- 172.21.15.32:0/1954030915 shutdown_connections 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.961+0000 7fbdde426700 1 -- 172.21.15.32:0/1954030915 wait complete. 2023-09-30T22:28:16.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.962+0000 7fbdde426700 1 Processor -- start 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.962+0000 7fbdde426700 1 -- start start 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdde426700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd8125d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdde426700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8130660 0x7fbdd812efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdde426700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdd8131140 0x7fbdd812f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdde426700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd812d8c0 con 0x7fbdd8131140 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdde426700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd812e620 con 0x7fbdd8130660 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdde426700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd806b1e0 con 0x7fbdd809dc30 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdd7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd8125d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbddc9c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdd8131140 0x7fbdd812f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdd7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd8125d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46158/0 (socket says 172.21.15.32:46158) 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.963+0000 7fbdd7fff700 1 -- 172.21.15.32:0/160741046 learned_addr learned my addr 172.21.15.32:0/160741046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:16.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdd77fe700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8130660 0x7fbdd812efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbddc9c3700 1 -- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 msgr2=0x7fbdd8125d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbddc9c3700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd8125d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbddc9c3700 1 -- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8130660 msgr2=0x7fbdd812efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbddc9c3700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8130660 0x7fbdd812efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbddc9c3700 1 -- 172.21.15.32:0/160741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0009b30 con 0x7fbdd8131140 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbddc9c3700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdd8131140 0x7fbdd812f500 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0015000 tx=0x7fbdd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdd7fff700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd8125d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0004250 con 0x7fbdd8131140 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00043d0 con 0x7fbdd8131140 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0004780 con 0x7fbdd8131140 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd812ca80 con 0x7fbdd8131140 2023-09-30T22:28:16.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.964+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd812ccc0 con 0x7fbdd8131140 2023-09-30T22:28:16.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.966+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8074ea0 con 0x7fbdd8131140 2023-09-30T22:28:16.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.966+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbdd001c440 con 0x7fbdd8131140 2023-09-30T22:28:16.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.967+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbdd0041190 con 0x7fbdd8131140 2023-09-30T22:28:16.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:16.971+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbdd00117a0 con 0x7fbdd8131140 2023-09-30T22:28:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.145+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa268072210 con 0x7fa26806f680 2023-09-30T22:28:17.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.146+0000 7fa24ffff700 1 -- 172.21.15.32:0/2213623645 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa25802c970 con 0x7fa26806f680 2023-09-30T22:28:17.147 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:17.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.150+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 msgr2=0x7fa2681c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.150+0000 7fa26e07f700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa2681c8770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa25800a010 tx=0x7fa258004750 comp rx=0 tx=0).stop 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 shutdown_connections 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa268130670 0x7fa268139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26806f680 0x7fa2681c8770 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 --2- 172.21.15.32:0/2213623645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26806ffd0 0x7fa2681c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 >> 172.21.15.32:0/2213623645 conn(0x7fa2680a2aa0 msgr2=0x7fa2680a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 shutdown_connections 2023-09-30T22:28:17.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.151+0000 7fa26e07f700 1 -- 172.21.15.32:0/2213623645 wait complete. 2023-09-30T22:28:17.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:17.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.182+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7df4074ea0 con 0x7f7df409e930 2023-09-30T22:28:17.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.182+0000 7f7df17fa700 1 -- 172.21.15.32:0/3596022945 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7de8021020 con 0x7f7df409e930 2023-09-30T22:28:17.183 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 msgr2=0x7f7df41a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 0x7f7df41a3e10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7de800ec10 tx=0x7f7de800ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 shutdown_connections 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7df409f650 0x7f7df41a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7df409e930 0x7f7df41a3e10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 --2- 172.21.15.32:0/3596022945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7df409f240 0x7f7df41a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 >> 172.21.15.32:0/3596022945 conn(0x7f7df40a4210 msgr2=0x7f7df41b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:17.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 shutdown_connections 2023-09-30T22:28:17.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.185+0000 7f7dfb0c3700 1 -- 172.21.15.32:0/3596022945 wait complete. 2023-09-30T22:28:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.190+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbdd809c700 con 0x7fbdd8131140 2023-09-30T22:28:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.191+0000 7fbdd4ff9700 1 -- 172.21.15.32:0/160741046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbdd00243c0 con 0x7fbdd8131140 2023-09-30T22:28:17.192 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdd8131140 msgr2=0x7fbdd812f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdd8131140 0x7fbdd812f500 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0015000 tx=0x7fbdd0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 shutdown_connections 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdd8131140 0x7fbdd812f500 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbdd809dc30 0x7fbdd8125d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 --2- 172.21.15.32:0/160741046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbdd8130660 0x7fbdd812efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.193+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 >> 172.21.15.32:0/160741046 conn(0x7fbdd8066f50 msgr2=0x7fbdd8070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.194+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 shutdown_connections 2023-09-30T22:28:17.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:17.194+0000 7fbdde426700 1 -- 172.21.15.32:0/160741046 wait complete. 2023-09-30T22:28:17.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:17.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:17.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.038+0000 7fe56a166700 1 Processor -- start 2023-09-30T22:28:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 -- start start 2023-09-30T22:28:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 0x7fe56409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 0x7fe564130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56409f520 con 0x7fe56409d6f0 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56409f130 con 0x7fe56409e410 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe56a166700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe56409e930 con 0x7fe56409e000 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe5637fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 0x7fe56409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe562ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 0x7fe564130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe563fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe5637fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 0x7fe56409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54178/0 (socket says 172.21.15.32:54178) 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe562ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 0x7fe564130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46166/0 (socket says 172.21.15.32:46166) 2023-09-30T22:28:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.039+0000 7fe5637fe700 1 -- 172.21.15.32:0/1322852303 learned_addr learned my addr 172.21.15.32:0/1322852303 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe563fff700 1 -- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 msgr2=0x7fe564130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe563fff700 1 --2- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 0x7fe564130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe563fff700 1 -- 172.21.15.32:0/1322852303 >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 msgr2=0x7fe56409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe563fff700 1 --2- 172.21.15.32:0/1322852303 >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 0x7fe56409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe563fff700 1 -- 172.21.15.32:0/1322852303 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56409b2d0 con 0x7fe56409e410 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe562ffd700 1 --2- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 0x7fe564130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe5637fe700 1 --2- 172.21.15.32:0/1322852303 >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 0x7fe56409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe563fff700 1 --2- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564130ba0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009d90 tx=0x7fe55c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5232448d7bda5b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe5627fc700 1 -- 172.21.15.32:0/1322852303 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00e070 con 0x7fe56409e410 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe5627fc700 1 -- 172.21.15.32:0/1322852303 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c00bb20 con 0x7fe56409e410 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.040+0000 7fe5627fc700 1 -- 172.21.15.32:0/1322852303 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c012430 con 0x7fe56409e410 2023-09-30T22:28:18.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 -- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 msgr2=0x7fe564130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 --2- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564130ba0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009d90 tx=0x7fe55c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 -- 172.21.15.32:0/1322852303 shutdown_connections 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 --2- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564130ba0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 --2- 172.21.15.32:0/1322852303 >> v2:172.21.15.32:3300/0 conn(0x7fe56409d6f0 0x7fe56409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 --2- 172.21.15.32:0/1322852303 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409e000 0x7fe564130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 -- 172.21.15.32:0/1322852303 >> 172.21.15.32:0/1322852303 conn(0x7fe564066f50 msgr2=0x7fe56409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 -- 172.21.15.32:0/1322852303 shutdown_connections 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.041+0000 7fe56a166700 1 -- 172.21.15.32:0/1322852303 wait complete. 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 Processor -- start 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 -- start start 2023-09-30T22:28:18.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 0x7fe564120070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 0x7fe5641205b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564120af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564121540 con 0x7fe56409e000 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56409c700 con 0x7fe56409e410 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe56a166700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe564122e40 con 0x7fe56409d6f0 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe5637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 0x7fe564120070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe562ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 0x7fe5641205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe5637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 0x7fe564120070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46170/0 (socket says 172.21.15.32:46170) 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.042+0000 7fe562ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 0x7fe5641205b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54184/0 (socket says 172.21.15.32:54184) 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 -- 172.21.15.32:0/1668664364 learned_addr learned my addr 172.21.15.32:0/1668664364 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe563fff700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564120af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 -- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 msgr2=0x7fe564120af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564120af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 -- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 msgr2=0x7fe5641205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 0x7fe5641205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 -- 172.21.15.32:0/1668664364 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe554009550 con 0x7fe56409d6f0 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe562ffd700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 0x7fe5641205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe5637fe700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 0x7fe564120070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe55400ebe0 tx=0x7fe55400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554004030 con 0x7fe56409d6f0 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554004190 con 0x7fe56409d6f0 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554004520 con 0x7fe56409d6f0 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5580094a0 con 0x7fe56409d6f0 2023-09-30T22:28:18.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.043+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c009a70 con 0x7fe56409d6f0 2023-09-30T22:28:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.044+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe55400c850 con 0x7fe56409d6f0 2023-09-30T22:28:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.044+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe55403f520 con 0x7fe56409d6f0 2023-09-30T22:28:18.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.044+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe564074ea0 con 0x7fe56409d6f0 2023-09-30T22:28:18.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.049+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5540036a0 con 0x7fe56409d6f0 2023-09-30T22:28:18.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:18.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:18.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.264+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe56406c360 con 0x7fe56409d6f0 2023-09-30T22:28:18.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.264+0000 7fe54bfff700 1 -- 172.21.15.32:0/1668664364 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe554019ab0 con 0x7fe56409d6f0 2023-09-30T22:28:18.265 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 msgr2=0x7fe564120070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 0x7fe564120070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe55400ebe0 tx=0x7fe55400eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 shutdown_connections 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe56409e410 0x7fe564120af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe56409d6f0 0x7fe564120070 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 --2- 172.21.15.32:0/1668664364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe56409e000 0x7fe5641205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 >> 172.21.15.32:0/1668664364 conn(0x7fe564066f50 msgr2=0x7fe5640713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 shutdown_connections 2023-09-30T22:28:18.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.267+0000 7fe56a166700 1 -- 172.21.15.32:0/1668664364 wait complete. 2023-09-30T22:28:18.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:18.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.325+0000 7f136b207700 1 Processor -- start 2023-09-30T22:28:18.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.325+0000 7f136b207700 1 -- start start 2023-09-30T22:28:18.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f136b207700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13640a0a30 0x7f13640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f136b207700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136413d820 0x7f136413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f136b207700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136409d920 con 0x7f13640a1340 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f136b207700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136409d020 con 0x7f13640a0a30 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f136b207700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13640a2010 con 0x7f136413d820 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13640a0a30 0x7f13640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f13640a0a30 0x7f13640a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54188/0 (socket says 172.21.15.32:54188) 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 -- 172.21.15.32:0/2640382311 learned_addr learned my addr 172.21.15.32:0/2640382311 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f13697a4700 1 --2- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136413d820 0x7f136413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1363fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f13640a1340 0x7f136413c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41788/0 (socket says 172.21.15.32:41788) 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 -- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136413d820 msgr2=0x7f136413fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:18.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 --2- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136413d820 0x7f136413fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 -- 172.21.15.32:0/2640382311 >> v1:172.21.15.32:6789/0 conn(0x7f13640a1340 legacy=0x7f136413c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.326+0000 7f1368fa3700 1 -- 172.21.15.32:0/2640382311 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136409dbe0 con 0x7f13640a0a30 2023-09-30T22:28:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.327+0000 7f1368fa3700 1 --2- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13640a0a30 0x7f13640a0e00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f13580098f0 tx=0x7f1358009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89edf2daa24de75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.327+0000 7f13637fe700 1 -- 172.21.15.32:0/2640382311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358011830 con 0x7f13640a0a30 2023-09-30T22:28:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.327+0000 7f13637fe700 1 -- 172.21.15.32:0/2640382311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1358011990 con 0x7f13640a0a30 2023-09-30T22:28:18.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.327+0000 7f13637fe700 1 -- 172.21.15.32:0/2640382311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358011d20 con 0x7f13640a0a30 2023-09-30T22:28:18.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.327+0000 7f136b207700 1 -- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13640a0a30 msgr2=0x7f13640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.327+0000 7f136b207700 1 --2- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13640a0a30 0x7f13640a0e00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f13580098f0 tx=0x7f1358009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.328+0000 7f136b207700 1 -- 172.21.15.32:0/2640382311 shutdown_connections 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.328+0000 7f136b207700 1 --2- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136413d820 0x7f136413fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.328+0000 7f136b207700 1 --2- 172.21.15.32:0/2640382311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13640a0a30 0x7f13640a0e00 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.328+0000 7f136b207700 1 -- 172.21.15.32:0/2640382311 >> 172.21.15.32:0/2640382311 conn(0x7f13640a6550 msgr2=0x7f13640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.328+0000 7f136b207700 1 -- 172.21.15.32:0/2640382311 shutdown_connections 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.328+0000 7f136b207700 1 -- 172.21.15.32:0/2640382311 wait complete. 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 Processor -- start 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 -- start start 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13640a0a30 0x7f1364124fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 0x7f13641254f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136413d820 0x7f1364125a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364127110 con 0x7f136413d820 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13641289e0 con 0x7f13640a0a30 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.329+0000 7f136b207700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1364126ad0 con 0x7f13640a1340 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 0x7f13641254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1368fa3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13640a0a30 0x7f1364124fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 0x7f13641254f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46182/0 (socket says 172.21.15.32:46182) 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 -- 172.21.15.32:0/3289910967 learned_addr learned my addr 172.21.15.32:0/3289910967 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f13697a4700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136413d820 0x7f1364125a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 -- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13640a0a30 msgr2=0x7f1364124fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13640a0a30 0x7f1364124fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 -- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136413d820 msgr2=0x7f1364125a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136413d820 0x7f1364125a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 -- 172.21.15.32:0/3289910967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358009550 con 0x7f13640a1340 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f13697a4700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136413d820 0x7f1364125a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f1363fff700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 0x7f13641254f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f135000c550 tx=0x7f135000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1350005cb0 con 0x7f13640a1340 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.330+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1350005e10 con 0x7f13640a1340 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.331+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13640a4aa0 con 0x7f13640a1340 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.331+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13641272c0 con 0x7f13640a1340 2023-09-30T22:28:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.331+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13500072d0 con 0x7f13640a1340 2023-09-30T22:28:18.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.332+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1350007430 con 0x7f13640a1340 2023-09-30T22:28:18.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.332+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13440052e0 con 0x7f13640a1340 2023-09-30T22:28:18.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.332+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f135003ead0 con 0x7f13640a1340 2023-09-30T22:28:18.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.341+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1350012400 con 0x7f13640a1340 2023-09-30T22:28:18.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.361+0000 7f79f59fa700 1 Processor -- start 2023-09-30T22:28:18.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.361+0000 7f79f59fa700 1 -- start start 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79f59fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79f59fa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f009e000 0x7f79f01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79f59fa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f009b120 con 0x7f79f009e410 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79f59fa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f006b460 con 0x7f79f009e000 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79f59fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f009a7e0 con 0x7f79f009d6f0 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79ee7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79f009e000 0x7f79f01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79ef7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f79f009e410 0x7f79f0133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41798/0 (socket says 172.21.15.32:41798) 2023-09-30T22:28:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.362+0000 7f79ef7fe700 1 -- 172.21.15.32:0/2724980225 learned_addr learned my addr 172.21.15.32:0/2724980225 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79ee7fc700 1 -- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 msgr2=0x7f79f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79ee7fc700 1 --2- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79ee7fc700 1 -- 172.21.15.32:0/2724980225 >> v1:172.21.15.32:6789/0 conn(0x7f79f009e410 legacy=0x7f79f0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79ee7fc700 1 -- 172.21.15.32:0/2724980225 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f009a500 con 0x7f79f009e000 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79eeffd700 1 --2- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79ee7fc700 1 --2- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01339f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009840 tx=0x7f79e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88b1780853f5db9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79edffb700 1 -- 172.21.15.32:0/2724980225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4011830 con 0x7f79f009e000 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79edffb700 1 -- 172.21.15.32:0/2724980225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4011990 con 0x7f79f009e000 2023-09-30T22:28:18.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.363+0000 7f79edffb700 1 -- 172.21.15.32:0/2724980225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4011d20 con 0x7f79f009e000 2023-09-30T22:28:18.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 Processor -- start 2023-09-30T22:28:18.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 -- start start 2023-09-30T22:28:18.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 -- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 msgr2=0x7f79f01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 --2- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01339f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009840 tx=0x7f79e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 -- 172.21.15.32:0/2724980225 shutdown_connections 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 --2- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 --2- 172.21.15.32:0/2724980225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01339f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 -- 172.21.15.32:0/2724980225 >> 172.21.15.32:0/2724980225 conn(0x7f79f0066f50 msgr2=0x7f79f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 -- 172.21.15.32:0/2724980225 shutdown_connections 2023-09-30T22:28:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f79f59fa700 1 -- 172.21.15.32:0/2724980225 wait complete. 2023-09-30T22:28:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f275c099a40 0x7f275c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c1313c0 con 0x7f275c099a40 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.364+0000 7f2762f3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c06e8c0 con 0x7f275c09a3b0 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f2762f3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f275c09a820 con 0x7f275c131810 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f2760cd7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f275c099a40 0x7f275c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f2760cd7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46194/0 (socket says 172.21.15.32:46194) 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f2760cd7700 1 -- 172.21.15.32:0/2245261860 learned_addr learned my addr 172.21.15.32:0/2245261860 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 -- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 msgr2=0x7f275c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 -- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 msgr2=0x7f275c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 -- 172.21.15.32:0/2245261860 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c071d20 con 0x7f275c099a40 2023-09-30T22:28:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275bfff700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c099e70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2748009840 tx=0x7f2748009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9a89c36f2e3e59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.365+0000 7f275b7fe700 1 -- 172.21.15.32:0/2245261860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011830 con 0x7f275c099a40 2023-09-30T22:28:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f79f59fa700 1 Processor -- start 2023-09-30T22:28:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f275b7fe700 1 -- 172.21.15.32:0/2245261860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2748011990 con 0x7f275c099a40 2023-09-30T22:28:18.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f275b7fe700 1 -- 172.21.15.32:0/2245261860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011d20 con 0x7f275c099a40 2023-09-30T22:28:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f79f59fa700 1 -- start start 2023-09-30T22:28:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f79f59fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f01a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f79f59fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.366+0000 7f79f59fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79f009e410 0x7f79f01a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79f59fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01a91f0 con 0x7f79f009e000 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79f59fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f009c710 con 0x7f79f009d6f0 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79f59fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00a3160 con 0x7f79f009e410 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 -- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 msgr2=0x7f275c099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c099e70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2748009840 tx=0x7f2748009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79f009e410 0x7f79f01a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f01a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54222/0 (socket says 172.21.15.32:54222) 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 -- 172.21.15.32:0/3906176702 learned_addr learned my addr 172.21.15.32:0/3906176702 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 -- 172.21.15.32:0/2245261860 shutdown_connections 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 --2- 172.21.15.32:0/2245261860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c099e70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 -- 172.21.15.32:0/2245261860 >> 172.21.15.32:0/2245261860 conn(0x7f275c0a2aa0 msgr2=0x7f275c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 -- 172.21.15.32:0/2245261860 shutdown_connections 2023-09-30T22:28:18.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f2762f3b700 1 -- 172.21.15.32:0/2245261860 wait complete. 2023-09-30T22:28:18.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 -- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79f009e410 msgr2=0x7f79f01a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79f009e410 0x7f79f01a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 -- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 msgr2=0x7f79f01a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f01a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 -- 172.21.15.32:0/3906176702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00094a0 con 0x7f79f009e000 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ef7fe700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79f009e410 0x7f79f01a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79eeffd700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f01a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.367+0000 7f79ee7fc700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01a6510 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009fd0 tx=0x7f79e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40036a0 con 0x7f79f009e000 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e401f070 con 0x7f79f009e000 2023-09-30T22:28:18.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003a60 con 0x7f79f009e000 2023-09-30T22:28:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e40094a0 con 0x7f79f009e000 2023-09-30T22:28:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f01a8da0 con 0x7f79f009e000 2023-09-30T22:28:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f2762f3b700 1 Processor -- start 2023-09-30T22:28:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.368+0000 7f2762f3b700 1 -- start start 2023-09-30T22:28:18.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79e4003bc0 con 0x7f79f009e000 2023-09-30T22:28:18.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f79e4021070 con 0x7f79f009e000 2023-09-30T22:28:18.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2762f3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c124920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2762f3b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c124e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2762f3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c1cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2762f3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1260c0 con 0x7f275c099a40 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2762f3b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1244f0 con 0x7f275c09a3b0 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2762f3b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0a1d10 con 0x7f275c131810 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2760cd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c124920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2760cd7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c124920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54226/0 (socket says 172.21.15.32:54226) 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f27614d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c1cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.369+0000 7f2760cd7700 1 -- 172.21.15.32:0/1869821788 learned_addr learned my addr 172.21.15.32:0/1869821788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f275bfff700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c124e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27614d8700 1 -- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 msgr2=0x7f275c124e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27614d8700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c124e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27614d8700 1 -- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 msgr2=0x7f275c124920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27614d8700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c124920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27614d8700 1 -- 172.21.15.32:0/1869821788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27480094a0 con 0x7f275c131810 2023-09-30T22:28:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27614d8700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c1cc560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f275400be20 tx=0x7f2754009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:18.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0074ea0 con 0x7f79f009e000 2023-09-30T22:28:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275400cc10 con 0x7f275c131810 2023-09-30T22:28:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275400cd90 con 0x7f275c131810 2023-09-30T22:28:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c1274b0 con 0x7f275c131810 2023-09-30T22:28:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754014680 con 0x7f275c131810 2023-09-30T22:28:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.370+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c1281e0 con 0x7f275c131810 2023-09-30T22:28:18.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.371+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c074ea0 con 0x7f275c131810 2023-09-30T22:28:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.372+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f275401d3b0 con 0x7f275c131810 2023-09-30T22:28:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.372+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2754006e40 con 0x7f275c131810 2023-09-30T22:28:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.379+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79e4015080 con 0x7f79f009e000 2023-09-30T22:28:18.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.380+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f275401c360 con 0x7f275c131810 2023-09-30T22:28:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.551+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1344005590 con 0x7f13640a1340 2023-09-30T22:28:18.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.551+0000 7f13617fa700 1 -- 172.21.15.32:0/3289910967 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1350011b40 con 0x7f13640a1340 2023-09-30T22:28:18.554 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 msgr2=0x7f13641254f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 0x7f13641254f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f135000c550 tx=0x7f135000c860 comp rx=0 tx=0).stop 2023-09-30T22:28:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 shutdown_connections 2023-09-30T22:28:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136413d820 0x7f1364125a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f13640a0a30 0x7f1364124fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 --2- 172.21.15.32:0/3289910967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f13640a1340 0x7f13641254f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 >> 172.21.15.32:0/3289910967 conn(0x7f13640a6550 msgr2=0x7f13640a2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 shutdown_connections 2023-09-30T22:28:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.554+0000 7f136b207700 1 -- 172.21.15.32:0/3289910967 wait complete. 2023-09-30T22:28:18.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:18.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.611+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f79f006c360 con 0x7f79f009e000 2023-09-30T22:28:18.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.612+0000 7f79d7fff700 1 -- 172.21.15.32:0/3906176702 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79e402b020 con 0x7f79f009e000 2023-09-30T22:28:18.613 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 msgr2=0x7f79f01a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01a6510 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009fd0 tx=0x7f79e4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:18.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 shutdown_connections 2023-09-30T22:28:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f79f009e410 0x7f79f01a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f79f009d6f0 0x7f79f01a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 --2- 172.21.15.32:0/3906176702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79f009e000 0x7f79f01a6510 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 >> 172.21.15.32:0/3906176702 conn(0x7f79f0066f50 msgr2=0x7f79f01baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 shutdown_connections 2023-09-30T22:28:18.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.614+0000 7f79f59fa700 1 -- 172.21.15.32:0/3906176702 wait complete. 2023-09-30T22:28:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.624+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f275c06b9e0 con 0x7f275c131810 2023-09-30T22:28:18.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.625+0000 7f27597fa700 1 -- 172.21.15.32:0/1869821788 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2754019030 con 0x7f275c131810 2023-09-30T22:28:18.626 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 msgr2=0x7f275c1cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c1cc560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f275400be20 tx=0x7f2754009ed0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 shutdown_connections 2023-09-30T22:28:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f275c131810 0x7f275c1cc560 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f275c099a40 0x7f275c124920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 --2- 172.21.15.32:0/1869821788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f275c09a3b0 0x7f275c124e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 >> 172.21.15.32:0/1869821788 conn(0x7f275c0a2aa0 msgr2=0x7f275c09d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.627+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 shutdown_connections 2023-09-30T22:28:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:18.628+0000 7f2762f3b700 1 -- 172.21.15.32:0/1869821788 wait complete. 2023-09-30T22:28:18.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:18.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:19.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:19.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.436+0000 7f33919b4700 1 Processor -- start 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.436+0000 7f33919b4700 1 -- start start 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f33919b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 0x7f338c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f33919b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 0x7f338c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f33919b4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c09ece0 con 0x7f338c131b40 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f33919b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c09e760 con 0x7f338c06a770 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f33919b4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f338c09e040 con 0x7f338c06b080 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 0x7f338c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 0x7f338c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 0x7f338c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55790/0 (socket says 172.21.15.32:55790) 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 0x7f338c131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46234/0 (socket says 172.21.15.32:46234) 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f338c131b40 0x7f338c138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41806/0 (socket says 172.21.15.32:41806) 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338affd700 1 -- 172.21.15.32:0/4014671326 learned_addr learned my addr 172.21.15.32:0/4014671326 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f3389ffb700 1 -- 172.21.15.32:0/4014671326 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806400648 0 0) 0x7f338c09ece0 con 0x7f338c131b40 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338a7fc700 1 -- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 msgr2=0x7f338c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338a7fc700 1 --2- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 0x7f338c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338a7fc700 1 -- 172.21.15.32:0/4014671326 >> v1:172.21.15.32:6789/0 conn(0x7f338c131b40 legacy=0x7f338c138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338a7fc700 1 -- 172.21.15.32:0/4014671326 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c09f630 con 0x7f338c06b080 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.437+0000 7f338affd700 1 --2- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 0x7f338c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f338a7fc700 1 --2- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 0x7f338c131600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f3374009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da17e25f43c1be4e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f3389ffb700 1 -- 172.21.15.32:0/4014671326 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374011830 con 0x7f338c06b080 2023-09-30T22:28:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f3389ffb700 1 -- 172.21.15.32:0/4014671326 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3374011990 con 0x7f338c06b080 2023-09-30T22:28:19.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f3389ffb700 1 -- 172.21.15.32:0/4014671326 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374011d20 con 0x7f338c06b080 2023-09-30T22:28:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 -- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 msgr2=0x7f338c131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 --2- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 0x7f338c131600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f3374009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 -- 172.21.15.32:0/4014671326 shutdown_connections 2023-09-30T22:28:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 --2- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c06a770 0x7f338c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 --2- 172.21.15.32:0/4014671326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06b080 0x7f338c131600 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 -- 172.21.15.32:0/4014671326 >> 172.21.15.32:0/4014671326 conn(0x7f338c066f50 msgr2=0x7f338c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 -- 172.21.15.32:0/4014671326 shutdown_connections 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.438+0000 7f33919b4700 1 -- 172.21.15.32:0/4014671326 wait complete. 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.439+0000 7f33919b4700 1 Processor -- start 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 -- start start 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 0x7f338c09ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f338c06b080 0x7f338c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c131b40 0x7f338c1c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c06ece0 con 0x7f338c06b080 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c125f50 con 0x7f338c131b40 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f33919b4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1311f0 con 0x7f338c06a770 2023-09-30T22:28:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f338affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 0x7f338c09ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f338affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 0x7f338c09ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46250/0 (socket says 172.21.15.32:46250) 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f338affd700 1 -- 172.21.15.32:0/2877201320 learned_addr learned my addr 172.21.15.32:0/2877201320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f338b7fe700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c131b40 0x7f338c1c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.440+0000 7f338a7fc700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f338c06b080 0x7f338c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f338affd700 1 -- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c131b40 msgr2=0x7f338c1c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f338affd700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c131b40 0x7f338c1c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f338affd700 1 -- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f338c06b080 msgr2=0x7f338c09b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f338affd700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f338c06b080 0x7f338c09b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f338affd700 1 -- 172.21.15.32:0/2877201320 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c009550 con 0x7f338c06a770 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f338affd700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 0x7f338c09ad70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f337c00ebe0 tx=0x7f337c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00c850 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c00c9b0 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00cd40 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33740094a0 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.441+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c09b7f0 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.442+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f337c013470 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.442+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f337c03f860 con 0x7f338c06a770 2023-09-30T22:28:19.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.442+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c074ea0 con 0x7f338c06a770 2023-09-30T22:28:19.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.447+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f337c021020 con 0x7f338c06a770 2023-09-30T22:28:19.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:19.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:19.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.653+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f338c126090 con 0x7f338c06a770 2023-09-30T22:28:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.653+0000 7f3383fff700 1 -- 172.21.15.32:0/2877201320 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f337c017070 con 0x7f338c06a770 2023-09-30T22:28:19.654 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 msgr2=0x7f338c09ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 0x7f338c09ad70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f337c00ebe0 tx=0x7f337c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 shutdown_connections 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f338c131b40 0x7f338c1c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f338c06a770 0x7f338c09ad70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 --2- 172.21.15.32:0/2877201320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f338c06b080 0x7f338c09b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 >> 172.21.15.32:0/2877201320 conn(0x7f338c066f50 msgr2=0x7f338c06ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 shutdown_connections 2023-09-30T22:28:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.656+0000 7f33919b4700 1 -- 172.21.15.32:0/2877201320 wait complete. 2023-09-30T22:28:19.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.725+0000 7faae1aa4700 1 Processor -- start 2023-09-30T22:28:19.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.725+0000 7faae1aa4700 1 -- start start 2023-09-30T22:28:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faae1aa4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faae1aa4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faadc12f8b0 0x7faadc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faae1aa4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc130260 0x7faadc136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faae1aa4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faadc12d330 con 0x7faadc12f8b0 2023-09-30T22:28:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faae1aa4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc12cd90 con 0x7faadc130260 2023-09-30T22:28:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faae1aa4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1306d0 con 0x7faadc12cf60 2023-09-30T22:28:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faadb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faadb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46256/0 (socket says 172.21.15.32:46256) 2023-09-30T22:28:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faadb7fe700 1 -- 172.21.15.32:0/3107103246 learned_addr learned my addr 172.21.15.32:0/3107103246 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faadaffd700 1 --2- 172.21.15.32:0/3107103246 >> v2:172.21.15.32:3300/0 conn(0x7faadc12f8b0 0x7faadc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.726+0000 7faadbfff700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc130260 0x7faadc136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadaffd700 1 -- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 msgr2=0x7faadc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadaffd700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadaffd700 1 -- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc130260 msgr2=0x7faadc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadaffd700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc130260 0x7faadc136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadaffd700 1 -- 172.21.15.32:0/3107103246 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc071780 con 0x7faadc12f8b0 2023-09-30T22:28:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadb7fe700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faadaffd700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc12f8b0 0x7faadc12fd20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7faad0009870 tx=0x7faad0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60716a55ea6564f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faada7fc700 1 -- 172.21.15.32:0/3107103246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0011830 con 0x7faadc12f8b0 2023-09-30T22:28:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faada7fc700 1 -- 172.21.15.32:0/3107103246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad0011990 con 0x7faadc12f8b0 2023-09-30T22:28:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.727+0000 7faada7fc700 1 -- 172.21.15.32:0/3107103246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0011d20 con 0x7faadc12f8b0 2023-09-30T22:28:19.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 -- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc12f8b0 msgr2=0x7faadc12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc12f8b0 0x7faadc12fd20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7faad0009870 tx=0x7faad0009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 -- 172.21.15.32:0/3107103246 shutdown_connections 2023-09-30T22:28:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc130260 0x7faadc136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 --2- 172.21.15.32:0/3107103246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc12f8b0 0x7faadc12fd20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 -- 172.21.15.32:0/3107103246 >> 172.21.15.32:0/3107103246 conn(0x7faadc0a2aa0 msgr2=0x7faadc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 -- 172.21.15.32:0/3107103246 shutdown_connections 2023-09-30T22:28:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.728+0000 7faae1aa4700 1 -- 172.21.15.32:0/3107103246 wait complete. 2023-09-30T22:28:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.729+0000 7faae1aa4700 1 Processor -- start 2023-09-30T22:28:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 -- start start 2023-09-30T22:28:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc1c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 0x7faadc1c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc130260 0x7faadc1c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc129cd0 con 0x7faadc130260 2023-09-30T22:28:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc09c650 con 0x7faadc12f8b0 2023-09-30T22:28:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.730+0000 7faae1aa4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1c5c40 con 0x7faadc12cf60 2023-09-30T22:28:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 0x7faadc1c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc1c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 0x7faadc1c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55822/0 (socket says 172.21.15.32:55822) 2023-09-30T22:28:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadaffd700 1 -- 172.21.15.32:0/1617982155 learned_addr learned my addr 172.21.15.32:0/1617982155 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc130260 0x7faadc1c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 -- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 msgr2=0x7faadc1c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc1c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 -- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 msgr2=0x7faadc1c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 0x7faadc1c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 -- 172.21.15.32:0/1617982155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc00b7f0 con 0x7faadc130260 2023-09-30T22:28:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadb7fe700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc1c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadaffd700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 0x7faadc1c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faadbfff700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc130260 0x7faadc1c81e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d90 tx=0x7faad400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4009c00 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.731+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad400f040 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.732+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4014480 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.732+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad00094a0 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.732+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc09ce50 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.733+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faad40145e0 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.733+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faad4040140 con 0x7faadc130260 2023-09-30T22:28:19.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.733+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc074ea0 con 0x7faadc130260 2023-09-30T22:28:19.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.738+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faad4007900 con 0x7faadc130260 2023-09-30T22:28:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.782+0000 7f87283f7700 1 Processor -- start 2023-09-30T22:28:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.782+0000 7f87283f7700 1 -- start start 2023-09-30T22:28:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f87283f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8720131810 0x7f8720131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f87283f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8720099a40 0x7f8720099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f87283f7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87201313c0 con 0x7f872009a3b0 2023-09-30T22:28:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f87283f7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872009b580 con 0x7f8720131810 2023-09-30T22:28:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f87283f7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872006e8c0 con 0x7f8720099a40 2023-09-30T22:28:19.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726193700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8720131810 0x7f8720131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8725992700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8720099a40 0x7f8720099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726994700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f872009a3b0 0x7f8720138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41814/0 (socket says 172.21.15.32:41814) 2023-09-30T22:28:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726994700 1 -- 172.21.15.32:0/378915357 learned_addr learned my addr 172.21.15.32:0/378915357 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726193700 1 -- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8720099a40 msgr2=0x7f8720099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726193700 1 --2- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8720099a40 0x7f8720099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726193700 1 -- 172.21.15.32:0/378915357 >> v1:172.21.15.32:6789/0 conn(0x7f872009a3b0 legacy=0x7f8720138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726193700 1 -- 172.21.15.32:0/378915357 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872009a830 con 0x7f8720131810 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.783+0000 7f8726193700 1 --2- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f8720131be0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f871c009840 tx=0x7f871c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779afc3361dd0451 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f8725191700 1 -- 172.21.15.32:0/378915357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c011830 con 0x7f8720131810 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f8725191700 1 -- 172.21.15.32:0/378915357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c011990 con 0x7f8720131810 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f8725191700 1 -- 172.21.15.32:0/378915357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c011d20 con 0x7f8720131810 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 -- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 msgr2=0x7f8720131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 --2- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f8720131be0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f871c009840 tx=0x7f871c009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:19.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 -- 172.21.15.32:0/378915357 shutdown_connections 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 --2- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f8720131be0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 --2- 172.21.15.32:0/378915357 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8720099a40 0x7f8720099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 -- 172.21.15.32:0/378915357 >> 172.21.15.32:0/378915357 conn(0x7f87200a2aa0 msgr2=0x7f87200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 -- 172.21.15.32:0/378915357 shutdown_connections 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.784+0000 7f87283f7700 1 -- 172.21.15.32:0/378915357 wait complete. 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.785+0000 7f87283f7700 1 Processor -- start 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.785+0000 7f87283f7700 1 -- start start 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f87283f7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8720099a40 0x7f872011b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f87283f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 0x7f872011bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f87283f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f872011c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f87283f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872011d940 con 0x7f8720131810 2023-09-30T22:28:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f87283f7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872011f200 con 0x7f872009a3b0 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f87283f7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f872011e5d0 con 0x7f8720099a40 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f8725992700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 0x7f872011bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f8725992700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 0x7f872011bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55840/0 (socket says 172.21.15.32:55840) 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f8725992700 1 -- 172.21.15.32:0/2717795836 learned_addr learned my addr 172.21.15.32:0/2717795836 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f8726193700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8720099a40 0x7f872011b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.786+0000 7f8726994700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f872011c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8725992700 1 -- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8720099a40 msgr2=0x7f872011b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8725992700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8720099a40 0x7f872011b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8725992700 1 -- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 msgr2=0x7f872011c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8725992700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f872011c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8725992700 1 -- 172.21.15.32:0/2717795836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c0094a0 con 0x7f872009a3b0 2023-09-30T22:28:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8726193700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8720099a40 0x7f872011b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8725992700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 0x7f872011bc20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8710012fc0 tx=0x7f87100100c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000dbb0 con 0x7f872009a3b0 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710013040 con 0x7f872009a3b0 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872011cb40 con 0x7f872009a3b0 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710010b40 con 0x7f872009a3b0 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.787+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872011d2f0 con 0x7f872009a3b0 2023-09-30T22:28:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.788+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8720074ea0 con 0x7f872009a3b0 2023-09-30T22:28:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.788+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8710010ca0 con 0x7f872009a3b0 2023-09-30T22:28:19.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.788+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f87100435a0 con 0x7f872009a3b0 2023-09-30T22:28:19.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.794+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8710016750 con 0x7f872009a3b0 2023-09-30T22:28:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.800+0000 7f32a9974700 1 Processor -- start 2023-09-30T22:28:19.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.800+0000 7f32a9974700 1 -- start start 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.800+0000 7f32a9974700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 0x7f32a412f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.800+0000 7f32a9974700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412c2f0 0x7f32a412fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a9974700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32a412cff0 0x7f32a41303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a9974700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a409a510 con 0x7f32a412cff0 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a9974700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a412c7d0 con 0x7f32a412cc20 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a9974700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32a409c190 con 0x7f32a412c2f0 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 0x7f32a412f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 0x7f32a412f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55850/0 (socket says 172.21.15.32:55850) 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 -- 172.21.15.32:0/468572066 learned_addr learned my addr 172.21.15.32:0/468572066 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a37fe700 1 --2- 172.21.15.32:0/468572066 >> v2:172.21.15.32:3300/0 conn(0x7f32a412cff0 0x7f32a41303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a27fc700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412c2f0 0x7f32a412fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 -- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412c2f0 msgr2=0x7f32a412fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412c2f0 0x7f32a412fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 -- 172.21.15.32:0/468572066 >> v2:172.21.15.32:3300/0 conn(0x7f32a412cff0 msgr2=0x7f32a41303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 --2- 172.21.15.32:0/468572066 >> v2:172.21.15.32:3300/0 conn(0x7f32a412cff0 0x7f32a41303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 -- 172.21.15.32:0/468572066 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a409b540 con 0x7f32a412cc20 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a37fe700 1 --2- 172.21.15.32:0/468572066 >> v2:172.21.15.32:3300/0 conn(0x7f32a412cff0 0x7f32a41303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a27fc700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412c2f0 0x7f32a412fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.801+0000 7f32a2ffd700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 0x7f32a412f930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f329400bb90 tx=0x7f329400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82121ce45e0756de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a1ffb700 1 -- 172.21.15.32:0/468572066 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400dc20 con 0x7f32a412cc20 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a1ffb700 1 -- 172.21.15.32:0/468572066 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400dd80 con 0x7f32a412cc20 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a1ffb700 1 -- 172.21.15.32:0/468572066 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294017630 con 0x7f32a412cc20 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 -- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 msgr2=0x7f32a412f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 0x7f32a412f930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f329400bb90 tx=0x7f329400bea0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 -- 172.21.15.32:0/468572066 shutdown_connections 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cc20 0x7f32a412f930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 --2- 172.21.15.32:0/468572066 >> v2:172.21.15.32:3300/0 conn(0x7f32a412cff0 0x7f32a41303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 --2- 172.21.15.32:0/468572066 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412c2f0 0x7f32a412fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.802+0000 7f32a9974700 1 -- 172.21.15.32:0/468572066 >> 172.21.15.32:0/468572066 conn(0x7f32a409f470 msgr2=0x7f32a40a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.803+0000 7f32a9974700 1 -- 172.21.15.32:0/468572066 shutdown_connections 2023-09-30T22:28:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.803+0000 7f32a9974700 1 -- 172.21.15.32:0/468572066 wait complete. 2023-09-30T22:28:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 Processor -- start 2023-09-30T22:28:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 -- start start 2023-09-30T22:28:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a412c2f0 0x7f32a412f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412cc20 0x7f32a412f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cff0 0x7f32a41c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a41305f0 con 0x7f32a412c2f0 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a412ef90 con 0x7f32a412cff0 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a9974700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32a409a100 con 0x7f32a412cc20 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.804+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a412c2f0 0x7f32a412f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a412c2f0 0x7f32a412f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54270/0 (socket says 172.21.15.32:54270) 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a2ffd700 1 -- 172.21.15.32:0/1960123854 learned_addr learned my addr 172.21.15.32:0/1960123854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cff0 0x7f32a41c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a27fc700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412cc20 0x7f32a412f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 -- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412cc20 msgr2=0x7f32a412f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412cc20 0x7f32a412f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 -- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a412c2f0 msgr2=0x7f32a412f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a412c2f0 0x7f32a412f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 -- 172.21.15.32:0/1960123854 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980094a0 con 0x7f32a412cff0 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f32a37fe700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cff0 0x7f32a41c3d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f329c00df80 tx=0x7f329c00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00ba80 con 0x7f32a412cff0 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c00f040 con 0x7f32a412cff0 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.805+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00be00 con 0x7f32a412cff0 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.806+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c009a70 con 0x7f32a412cff0 2023-09-30T22:28:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.806+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329400b7f0 con 0x7f32a412cff0 2023-09-30T22:28:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.807+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f329c01c410 con 0x7f32a412cff0 2023-09-30T22:28:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.807+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f329c03f870 con 0x7f32a412cff0 2023-09-30T22:28:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.807+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a40747f0 con 0x7f32a412cff0 2023-09-30T22:28:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.812+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f329c007920 con 0x7f32a412cff0 2023-09-30T22:28:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.963+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faadc0713a0 con 0x7faadc130260 2023-09-30T22:28:19.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.963+0000 7faac3fff700 1 -- 172.21.15.32:0/1617982155 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faad402a070 con 0x7faadc130260 2023-09-30T22:28:19.965 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc130260 msgr2=0x7faadc1c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc130260 0x7faadc1c81e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d90 tx=0x7faad400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 shutdown_connections 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faadc130260 0x7faadc1c81e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faadc12cf60 0x7faadc1c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 --2- 172.21.15.32:0/1617982155 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faadc12f8b0 0x7faadc1c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:19.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 >> 172.21.15.32:0/1617982155 conn(0x7faadc0a2aa0 msgr2=0x7faadc12a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 shutdown_connections 2023-09-30T22:28:19.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:19.966+0000 7faae1aa4700 1 -- 172.21.15.32:0/1617982155 wait complete. 2023-09-30T22:28:19.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.016+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f872006b9e0 con 0x7f872009a3b0 2023-09-30T22:28:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.016+0000 7f8716ffd700 1 -- 172.21.15.32:0/2717795836 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87100173e0 con 0x7f872009a3b0 2023-09-30T22:28:20.017 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 msgr2=0x7f872011bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 0x7f872011bc20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8710012fc0 tx=0x7f87100100c0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 shutdown_connections 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8720131810 0x7f872011c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8720099a40 0x7f872011b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 --2- 172.21.15.32:0/2717795836 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f872009a3b0 0x7f872011bc20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 >> 172.21.15.32:0/2717795836 conn(0x7f87200a2aa0 msgr2=0x7f8720071010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 shutdown_connections 2023-09-30T22:28:20.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.019+0000 7f87283f7700 1 -- 172.21.15.32:0/2717795836 wait complete. 2023-09-30T22:28:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.021+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f32a406d970 con 0x7f32a412cff0 2023-09-30T22:28:20.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.021+0000 7f328bfff700 1 -- 172.21.15.32:0/1960123854 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f329c007330 con 0x7f32a412cff0 2023-09-30T22:28:20.022 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cff0 msgr2=0x7f32a41c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cff0 0x7f32a41c3d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f329c00df80 tx=0x7f329c00b390 comp rx=0 tx=0).stop 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 shutdown_connections 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f32a412cff0 0x7f32a41c3d00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a412c2f0 0x7f32a412f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 --2- 172.21.15.32:0/1960123854 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f32a412cc20 0x7f32a412f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.023+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 >> 172.21.15.32:0/1960123854 conn(0x7f32a409f470 msgr2=0x7f32a41b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.024+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 shutdown_connections 2023-09-30T22:28:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.024+0000 7f32a9974700 1 -- 172.21.15.32:0/1960123854 wait complete. 2023-09-30T22:28:20.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:20.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:20.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.821+0000 7f9d5c247700 1 Processor -- start 2023-09-30T22:28:20.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.821+0000 7f9d5c247700 1 -- start start 2023-09-30T22:28:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5c247700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5c247700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d5409e000 0x7f9d54130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5c247700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5409f130 con 0x7f9d5409e410 2023-09-30T22:28:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5c247700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5409f520 con 0x7f9d5409e000 2023-09-30T22:28:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5c247700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5409e930 con 0x7f9d5409d6f0 2023-09-30T22:28:20.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d59fe3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d597e2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d5409e000 0x7f9d54130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5a7e4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d5409e410 0x7f9d54130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48050/0 (socket says 172.21.15.32:48050) 2023-09-30T22:28:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d59fe3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43338/0 (socket says 172.21.15.32:43338) 2023-09-30T22:28:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.822+0000 7f9d5a7e4700 1 -- 172.21.15.32:0/1556293807 learned_addr learned my addr 172.21.15.32:0/1556293807 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d597e2700 1 -- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 msgr2=0x7f9d5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:20.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d597e2700 1 --2- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d597e2700 1 -- 172.21.15.32:0/1556293807 >> v1:172.21.15.32:6789/0 conn(0x7f9d5409e410 legacy=0x7f9d54130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d597e2700 1 -- 172.21.15.32:0/1556293807 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5409b2d0 con 0x7f9d5409e000 2023-09-30T22:28:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d59fe3700 1 --2- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d597e2700 1 --2- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d54130660 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009840 tx=0x7f9d48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da50a2b0de930a97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d58fe1700 1 -- 172.21.15.32:0/1556293807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48011830 con 0x7f9d5409e000 2023-09-30T22:28:20.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d58fe1700 1 -- 172.21.15.32:0/1556293807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d48011990 con 0x7f9d5409e000 2023-09-30T22:28:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.823+0000 7f9d58fe1700 1 -- 172.21.15.32:0/1556293807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48011d20 con 0x7f9d5409e000 2023-09-30T22:28:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 -- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 msgr2=0x7f9d54130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 --2- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d54130660 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009840 tx=0x7f9d48009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 -- 172.21.15.32:0/1556293807 shutdown_connections 2023-09-30T22:28:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 --2- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 --2- 172.21.15.32:0/1556293807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d54130660 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 -- 172.21.15.32:0/1556293807 >> 172.21.15.32:0/1556293807 conn(0x7f9d54066f50 msgr2=0x7f9d5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 -- 172.21.15.32:0/1556293807 shutdown_connections 2023-09-30T22:28:20.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.824+0000 7f9d5c247700 1 -- 172.21.15.32:0/1556293807 wait complete. 2023-09-30T22:28:20.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 Processor -- start 2023-09-30T22:28:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 -- start start 2023-09-30T22:28:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5409e410 0x7f9d5406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54125ca0 con 0x7f9d5409e000 2023-09-30T22:28:20.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54125ea0 con 0x7f9d5409e410 2023-09-30T22:28:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d5c247700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54125610 con 0x7f9d5409d6f0 2023-09-30T22:28:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d59fe3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d597e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d597e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d5406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45922/0 (socket says 172.21.15.32:45922) 2023-09-30T22:28:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d59fe3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5406e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43352/0 (socket says 172.21.15.32:43352) 2023-09-30T22:28:20.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.825+0000 7f9d597e2700 1 -- 172.21.15.32:0/2055483764 learned_addr learned my addr 172.21.15.32:0/2055483764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d5a7e4700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5409e410 0x7f9d5406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d597e2700 1 -- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 msgr2=0x7f9d5406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d597e2700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5406e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d597e2700 1 -- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5409e410 msgr2=0x7f9d5406f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d597e2700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5409e410 0x7f9d5406f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:20.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d597e2700 1 -- 172.21.15.32:0/2055483764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d500094a0 con 0x7f9d5409e000 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d59fe3700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d5a7e4700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5409e410 0x7f9d5406f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d597e2700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d5406edb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9d480090b0 tx=0x7f9d48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480036a0 con 0x7f9d5409e000 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4801f070 con 0x7f9d5409e000 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480094a0 con 0x7f9d5409e000 2023-09-30T22:28:20.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48003a60 con 0x7f9d5409e000 2023-09-30T22:28:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.826+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d540a1f50 con 0x7f9d5409e000 2023-09-30T22:28:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.827+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d48003bc0 con 0x7f9d5409e000 2023-09-30T22:28:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.827+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d48021070 con 0x7f9d5409e000 2023-09-30T22:28:20.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.827+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d541302a0 con 0x7f9d5409e000 2023-09-30T22:28:20.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:20.832+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d48015080 con 0x7f9d5409e000 2023-09-30T22:28:20.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:21.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:21.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.042+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9d54074ea0 con 0x7f9d5409e000 2023-09-30T22:28:21.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.043+0000 7f9d467fc700 1 -- 172.21.15.32:0/2055483764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d48035090 con 0x7f9d5409e000 2023-09-30T22:28:21.045 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.045+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 msgr2=0x7f9d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.045+0000 7f9d5c247700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d5406edb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9d480090b0 tx=0x7f9d48004750 comp rx=0 tx=0).stop 2023-09-30T22:28:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.045+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 shutdown_connections 2023-09-30T22:28:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.046+0000 7f9d5c247700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5409e410 0x7f9d5406f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.046+0000 7f9d5c247700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5409d6f0 0x7f9d5406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.046+0000 7f9d5c247700 1 --2- 172.21.15.32:0/2055483764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5409e000 0x7f9d5406edb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.046+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 >> 172.21.15.32:0/2055483764 conn(0x7f9d54066f50 msgr2=0x7f9d5406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.046+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 shutdown_connections 2023-09-30T22:28:21.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.046+0000 7f9d5c247700 1 -- 172.21.15.32:0/2055483764 wait complete. 2023-09-30T22:28:21.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.130+0000 7f25ef222700 1 Processor -- start 2023-09-30T22:28:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 -- start start 2023-09-30T22:28:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e806b460 con 0x7f25e809d6f0 2023-09-30T22:28:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809b120 con 0x7f25e809e000 2023-09-30T22:28:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ef222700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25e809a7e0 con 0x7f25e809e410 2023-09-30T22:28:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ecfbe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ecfbe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45928/0 (socket says 172.21.15.32:45928) 2023-09-30T22:28:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ecfbe700 1 -- 172.21.15.32:0/3260802920 learned_addr learned my addr 172.21.15.32:0/3260802920 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25e7fff700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.131+0000 7f25ed7bf700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e7fff700 1 -- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 msgr2=0x7f25e8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e7fff700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e7fff700 1 -- 172.21.15.32:0/3260802920 >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 msgr2=0x7f25e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e7fff700 1 --2- 172.21.15.32:0/3260802920 >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e7fff700 1 -- 172.21.15.32:0/3260802920 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e809a500 con 0x7f25e809e000 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25ecfbe700 1 --2- 172.21.15.32:0/3260802920 >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e7fff700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009870 tx=0x7f25d4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d17dc07de7f7adf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e77fe700 1 -- 172.21.15.32:0/3260802920 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4011830 con 0x7f25e809e000 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e77fe700 1 -- 172.21.15.32:0/3260802920 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4011990 con 0x7f25e809e000 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.132+0000 7f25e77fe700 1 -- 172.21.15.32:0/3260802920 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4011d20 con 0x7f25e809e000 2023-09-30T22:28:21.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 -- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809e000 msgr2=0x7f25e81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009870 tx=0x7f25d4009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 -- 172.21.15.32:0/3260802920 shutdown_connections 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 --2- 172.21.15.32:0/3260802920 >> v2:172.21.15.32:3300/0 conn(0x7f25e809d6f0 0x7f25e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 --2- 172.21.15.32:0/3260802920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809e000 0x7f25e81339f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 -- 172.21.15.32:0/3260802920 >> 172.21.15.32:0/3260802920 conn(0x7f25e8066f50 msgr2=0x7f25e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 -- 172.21.15.32:0/3260802920 shutdown_connections 2023-09-30T22:28:21.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.133+0000 7f25ef222700 1 -- 172.21.15.32:0/3260802920 wait complete. 2023-09-30T22:28:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.134+0000 7f25ef222700 1 Processor -- start 2023-09-30T22:28:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.134+0000 7f25ef222700 1 -- start start 2023-09-30T22:28:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ef222700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809d6f0 0x7f25e81a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ef222700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e809e000 0x7f25e81a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ef222700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e81a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ef222700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81aa210 con 0x7f25e809e000 2023-09-30T22:28:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ef222700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81a7f70 con 0x7f25e809d6f0 2023-09-30T22:28:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ef222700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81a7940 con 0x7f25e809e410 2023-09-30T22:28:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e809e000 0x7f25e81a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ecfbe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809d6f0 0x7f25e81a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e809e000 0x7f25e81a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45934/0 (socket says 172.21.15.32:45934) 2023-09-30T22:28:21.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25e7fff700 1 -- 172.21.15.32:0/1677918868 learned_addr learned my addr 172.21.15.32:0/1677918868 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.135+0000 7f25ed7bf700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e81a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ecfbe700 1 -- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 msgr2=0x7f25e81a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ecfbe700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e81a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ecfbe700 1 -- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e809e000 msgr2=0x7f25e81a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ecfbe700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e809e000 0x7f25e81a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ecfbe700 1 -- 172.21.15.32:0/1677918868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc009550 con 0x7f25e809d6f0 2023-09-30T22:28:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ecfbe700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809d6f0 0x7f25e81a5dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00ea00 tx=0x7f25dc00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00eea0 con 0x7f25e809d6f0 2023-09-30T22:28:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc004030 con 0x7f25e809d6f0 2023-09-30T22:28:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d40094a0 con 0x7f25e809d6f0 2023-09-30T22:28:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc0043c0 con 0x7f25e809d6f0 2023-09-30T22:28:21.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.136+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81a8b90 con 0x7f25e809d6f0 2023-09-30T22:28:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.137+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8074ea0 con 0x7f25e809d6f0 2023-09-30T22:28:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.137+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25dc00c850 con 0x7f25e809d6f0 2023-09-30T22:28:21.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.137+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f25dc03f520 con 0x7f25e809d6f0 2023-09-30T22:28:21.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.146+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25dc019d60 con 0x7f25e809d6f0 2023-09-30T22:28:21.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.187+0000 7f31fc1df700 1 Processor -- start 2023-09-30T22:28:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31fc1df700 1 -- start start 2023-09-30T22:28:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31fc1df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f409d6f0 0x7f31f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31fc1df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 0x7f31f4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31fc1df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f409f130 con 0x7f31f409e410 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31fc1df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f409f520 con 0x7f31f409d6f0 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31fc1df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31f409e930 con 0x7f31f409e000 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.188+0000 7f31f977a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 0x7f31f4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f977a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 0x7f31f4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43382/0 (socket says 172.21.15.32:43382) 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f977a700 1 -- 172.21.15.32:0/3664892823 learned_addr learned my addr 172.21.15.32:0/3664892823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f9f7b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f409d6f0 0x7f31f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f9f7b700 1 -- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 msgr2=0x7f31f4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f9f7b700 1 --2- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 0x7f31f4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f9f7b700 1 -- 172.21.15.32:0/3664892823 >> v1:172.21.15.32:6789/0 conn(0x7f31f409e410 legacy=0x7f31f4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f9f7b700 1 -- 172.21.15.32:0/3664892823 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f409b2d0 con 0x7f31f409d6f0 2023-09-30T22:28:21.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f977a700 1 --2- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 0x7f31f4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f9f7b700 1 --2- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409d6f0 0x7f31f409dac0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009840 tx=0x7f31f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a806e23a17e084f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f8f79700 1 -- 172.21.15.32:0/3664892823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0011830 con 0x7f31f409d6f0 2023-09-30T22:28:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f8f79700 1 -- 172.21.15.32:0/3664892823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0011990 con 0x7f31f409d6f0 2023-09-30T22:28:21.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.189+0000 7f31f8f79700 1 -- 172.21.15.32:0/3664892823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0011d20 con 0x7f31f409d6f0 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 -- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409d6f0 msgr2=0x7f31f409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 --2- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409d6f0 0x7f31f409dac0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009840 tx=0x7f31f0009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 -- 172.21.15.32:0/3664892823 shutdown_connections 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 --2- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409d6f0 0x7f31f409dac0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 --2- 172.21.15.32:0/3664892823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409e000 0x7f31f4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 -- 172.21.15.32:0/3664892823 >> 172.21.15.32:0/3664892823 conn(0x7f31f4066f50 msgr2=0x7f31f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 -- 172.21.15.32:0/3664892823 shutdown_connections 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.190+0000 7f31fc1df700 1 -- 172.21.15.32:0/3664892823 wait complete. 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.191+0000 7f31fc1df700 1 Processor -- start 2023-09-30T22:28:21.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.191+0000 7f31fc1df700 1 -- start start 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fc1df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 0x7f31f4131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fc1df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409e000 0x7f31f4131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fc1df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e410 0x7f31f41320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fc1df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f4132a00 con 0x7f31f409e410 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fc1df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f409c700 con 0x7f31f409e000 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fc1df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31f4132e30 con 0x7f31f409d6f0 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 0x7f31f4131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 0x7f31f4131630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43392/0 (socket says 172.21.15.32:43392) 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 -- 172.21.15.32:0/94366074 learned_addr learned my addr 172.21.15.32:0/94366074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f977a700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409e000 0x7f31f4131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fa77c700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e410 0x7f31f41320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 -- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409e000 msgr2=0x7f31f4131b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409e000 0x7f31f4131b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 -- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e410 msgr2=0x7f31f41320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e410 0x7f31f41320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31f9f7b700 1 -- 172.21.15.32:0/94366074 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4009550 con 0x7f31f409d6f0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.192+0000 7f31fa77c700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e410 0x7f31f41320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f31f9f7b700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 0x7f31f4131630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f31f00090b0 tx=0x7f31f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f00036a0 con 0x7f31f409d6f0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f001f070 con 0x7f31f409d6f0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00094a0 con 0x7f31f409d6f0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0003a60 con 0x7f31f409d6f0 2023-09-30T22:28:21.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f41334a0 con 0x7f31f409d6f0 2023-09-30T22:28:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f726ab5b700 1 Processor -- start 2023-09-30T22:28:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.193+0000 7f726ab5b700 1 -- start start 2023-09-30T22:28:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f726ab5b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264130b30 0x7f7264138890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f726ab5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264138dd0 0x7f726413b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f726ab5b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7264099ef0 con 0x7f7264071b40 2023-09-30T22:28:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31f0003bc0 con 0x7f31f409d6f0 2023-09-30T22:28:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f72688f7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7264071b40 0x7f7264130420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48074/0 (socket says 172.21.15.32:48074) 2023-09-30T22:28:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f726ab5b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7264099500 con 0x7f7264138dd0 2023-09-30T22:28:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f72688f7700 1 -- 172.21.15.32:0/2030082069 learned_addr learned my addr 172.21.15.32:0/2030082069 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f726ab5b700 1 -- 172.21.15.32:0/2030082069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7264099c10 con 0x7f7264130b30 2023-09-30T22:28:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f72690f8700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264138dd0 0x7f726413b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f7263fff700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264130b30 0x7f7264138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.194+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f31f0021070 con 0x7f31f409d6f0 2023-09-30T22:28:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f7263fff700 1 -- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264138dd0 msgr2=0x7f726413b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f7263fff700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264138dd0 0x7f726413b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f7263fff700 1 -- 172.21.15.32:0/2030082069 >> v1:172.21.15.32:6789/0 conn(0x7f7264071b40 legacy=0x7f7264130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f7263fff700 1 -- 172.21.15.32:0/2030082069 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7264130f20 con 0x7f7264130b30 2023-09-30T22:28:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f72637fe700 1 -- 172.21.15.32:0/2030082069 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905794718 0 0) 0x7f7264099ef0 con 0x7f7264071b40 2023-09-30T22:28:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f4131260 con 0x7f31f409d6f0 2023-09-30T22:28:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f72690f8700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264138dd0 0x7f726413b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.195+0000 7f7263fff700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264130b30 0x7f7264138890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7250009870 tx=0x7f7250009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0074adcfd6e312b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.199+0000 7f72637fe700 1 -- 172.21.15.32:0/2030082069 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250011830 con 0x7f7264130b30 2023-09-30T22:28:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f72637fe700 1 -- 172.21.15.32:0/2030082069 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250011990 con 0x7f7264130b30 2023-09-30T22:28:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f72637fe700 1 -- 172.21.15.32:0/2030082069 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250011d20 con 0x7f7264130b30 2023-09-30T22:28:21.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31f0015020 con 0x7f31f409d6f0 2023-09-30T22:28:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f726ab5b700 1 -- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264130b30 msgr2=0x7f7264138890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f726ab5b700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264130b30 0x7f7264138890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7250009870 tx=0x7f7250009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f726ab5b700 1 -- 172.21.15.32:0/2030082069 shutdown_connections 2023-09-30T22:28:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f726ab5b700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264138dd0 0x7f726413b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f726ab5b700 1 --2- 172.21.15.32:0/2030082069 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264130b30 0x7f7264138890 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.200+0000 7f726ab5b700 1 -- 172.21.15.32:0/2030082069 >> 172.21.15.32:0/2030082069 conn(0x7f726409e7e0 msgr2=0x7f726409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.205+0000 7f726ab5b700 1 -- 172.21.15.32:0/2030082069 shutdown_connections 2023-09-30T22:28:21.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.205+0000 7f726ab5b700 1 -- 172.21.15.32:0/2030082069 wait complete. 2023-09-30T22:28:21.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.206+0000 7f726ab5b700 1 Processor -- start 2023-09-30T22:28:21.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.206+0000 7f726ab5b700 1 -- start start 2023-09-30T22:28:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f726ab5b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264071b40 0x7f7264126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f726ab5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264130b30 0x7f7264127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f726ab5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 0x7f7264127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f726ab5b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726409c1c0 con 0x7f7264138dd0 2023-09-30T22:28:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f726ab5b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7264129e60 con 0x7f7264130b30 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f726ab5b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7264128c80 con 0x7f7264071b40 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f72690f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 0x7f7264127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f72690f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 0x7f7264127880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45966/0 (socket says 172.21.15.32:45966) 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f72690f8700 1 -- 172.21.15.32:0/3340574514 learned_addr learned my addr 172.21.15.32:0/3340574514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f72688f7700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264071b40 0x7f7264126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.207+0000 7f7263fff700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264130b30 0x7f7264127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72690f8700 1 -- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264071b40 msgr2=0x7f7264126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72690f8700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264071b40 0x7f7264126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72690f8700 1 -- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264130b30 msgr2=0x7f7264127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72690f8700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264130b30 0x7f7264127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72690f8700 1 -- 172.21.15.32:0/3340574514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009a70 con 0x7f7264138dd0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72690f8700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 0x7f7264127880 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f725c009eb0 tx=0x7f725c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c009d50 con 0x7f7264138dd0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00f040 con 0x7f7264138dd0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72500094a0 con 0x7f7264138dd0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c014480 con 0x7f7264138dd0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.208+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264128ee0 con 0x7f7264138dd0 2023-09-30T22:28:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.209+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f725c0145e0 con 0x7f7264138dd0 2023-09-30T22:28:21.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.209+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f725c0403b0 con 0x7f7264138dd0 2023-09-30T22:28:21.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.209+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72640747f0 con 0x7f7264138dd0 2023-09-30T22:28:21.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.215+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f725c018030 con 0x7f7264138dd0 2023-09-30T22:28:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.360+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25e806c360 con 0x7f25e809d6f0 2023-09-30T22:28:21.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.360+0000 7f25e57fa700 1 -- 172.21.15.32:0/1677918868 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25dc019300 con 0x7f25e809d6f0 2023-09-30T22:28:21.362 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809d6f0 msgr2=0x7f25e81a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809d6f0 0x7f25e81a5dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00ea00 tx=0x7f25dc00ed10 comp rx=0 tx=0).stop 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 shutdown_connections 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25e809e410 0x7f25e81a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25e809d6f0 0x7f25e81a5dc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 --2- 172.21.15.32:0/1677918868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25e809e000 0x7f25e81a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 >> 172.21.15.32:0/1677918868 conn(0x7f25e8066f50 msgr2=0x7f25e806f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 shutdown_connections 2023-09-30T22:28:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.364+0000 7f25ef222700 1 -- 172.21.15.32:0/1677918868 wait complete. 2023-09-30T22:28:21.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:21.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.413+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31f4074ea0 con 0x7f31f409d6f0 2023-09-30T22:28:21.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.413+0000 7f31ea7fc700 1 -- 172.21.15.32:0/94366074 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31f001a430 con 0x7f31f409d6f0 2023-09-30T22:28:21.415 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 msgr2=0x7f31f4131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 0x7f31f4131630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f31f00090b0 tx=0x7f31f0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 shutdown_connections 2023-09-30T22:28:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e410 0x7f31f41320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409d6f0 0x7f31f4131630 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 --2- 172.21.15.32:0/94366074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409e000 0x7f31f4131b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 >> 172.21.15.32:0/94366074 conn(0x7f31f4066f50 msgr2=0x7f31f40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 shutdown_connections 2023-09-30T22:28:21.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.416+0000 7f31fc1df700 1 -- 172.21.15.32:0/94366074 wait complete. 2023-09-30T22:28:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.423+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f726406b250 con 0x7f7264138dd0 2023-09-30T22:28:21.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.423+0000 7f72617fa700 1 -- 172.21.15.32:0/3340574514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f725c02b090 con 0x7f7264138dd0 2023-09-30T22:28:21.424 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.425+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 msgr2=0x7f7264127880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.425+0000 7f726ab5b700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 0x7f7264127880 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f725c009eb0 tx=0x7f725c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 shutdown_connections 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7264138dd0 0x7f7264127880 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7264071b40 0x7f7264126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 --2- 172.21.15.32:0/3340574514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7264130b30 0x7f7264127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 >> 172.21.15.32:0/3340574514 conn(0x7f726409e7e0 msgr2=0x7f72640a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:21.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 shutdown_connections 2023-09-30T22:28:21.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:21.426+0000 7f726ab5b700 1 -- 172.21.15.32:0/3340574514 wait complete. 2023-09-30T22:28:21.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:21.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:22.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 Processor -- start 2023-09-30T22:28:22.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 -- start start 2023-09-30T22:28:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f492009d6f0 0x7f492009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009e000 0x7f4920130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492009f130 con 0x7f492009e410 2023-09-30T22:28:22.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492009f520 con 0x7f492009d6f0 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.213+0000 7f49281eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f492009e930 con 0x7f492009e000 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f492009d6f0 0x7f492009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925786700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009e000 0x7f4920130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f492009d6f0 0x7f492009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45968/0 (socket says 172.21.15.32:45968) 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 -- 172.21.15.32:0/1762293858 learned_addr learned my addr 172.21.15.32:0/1762293858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925786700 1 --2- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009e000 0x7f4920130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43426/0 (socket says 172.21.15.32:43426) 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4926788700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f492009e410 0x7f4920130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48076/0 (socket says 172.21.15.32:48076) 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 -- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009e000 msgr2=0x7f4920130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 --2- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009e000 0x7f4920130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 -- 172.21.15.32:0/1762293858 >> v1:172.21.15.32:6789/0 conn(0x7f492009e410 legacy=0x7f4920130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 -- 172.21.15.32:0/1762293858 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492009b2d0 con 0x7f492009d6f0 2023-09-30T22:28:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4925f87700 1 --2- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009d6f0 0x7f492009dac0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f491c0098f0 tx=0x7f491c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876847c980e4bd2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4924f85700 1 -- 172.21.15.32:0/1762293858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c011830 con 0x7f492009d6f0 2023-09-30T22:28:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4924f85700 1 -- 172.21.15.32:0/1762293858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c011990 con 0x7f492009d6f0 2023-09-30T22:28:22.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.214+0000 7f4924f85700 1 -- 172.21.15.32:0/1762293858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c011d20 con 0x7f492009d6f0 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 -- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009d6f0 msgr2=0x7f492009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 --2- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009d6f0 0x7f492009dac0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f491c0098f0 tx=0x7f491c009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 -- 172.21.15.32:0/1762293858 shutdown_connections 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 --2- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009d6f0 0x7f492009dac0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 --2- 172.21.15.32:0/1762293858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009e000 0x7f4920130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 -- 172.21.15.32:0/1762293858 >> 172.21.15.32:0/1762293858 conn(0x7f4920066f50 msgr2=0x7f492009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 -- 172.21.15.32:0/1762293858 shutdown_connections 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.215+0000 7f49281eb700 1 -- 172.21.15.32:0/1762293858 wait complete. 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 Processor -- start 2023-09-30T22:28:22.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 -- start start 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 0x7f49201222c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009e000 0x7f4920122800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f492009e410 0x7f4920122d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201254c0 con 0x7f492009e000 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.216+0000 7f49281eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4920124770 con 0x7f492009e410 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f49281eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4920123ed0 con 0x7f492009d6f0 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 0x7f49201222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925786700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009e000 0x7f4920122800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 0x7f49201222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43440/0 (socket says 172.21.15.32:43440) 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925786700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009e000 0x7f4920122800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45970/0 (socket says 172.21.15.32:45970) 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 -- 172.21.15.32:0/1809869588 learned_addr learned my addr 172.21.15.32:0/1809869588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:22.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4926788700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f492009e410 0x7f4920122d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 -- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f492009e410 msgr2=0x7f4920122d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f492009e410 0x7f4920122d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 -- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009e000 msgr2=0x7f4920122800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009e000 0x7f4920122800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 -- 172.21.15.32:0/1809869588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c009550 con 0x7f492009d6f0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f4925f87700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 0x7f49201222c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f491c009c40 tx=0x7f491c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c004400 con 0x7f492009d6f0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c01f070 con 0x7f492009d6f0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4920125de0 con 0x7f492009d6f0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0038e0 con 0x7f492009d6f0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.217+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4920125b00 con 0x7f492009d6f0 2023-09-30T22:28:22.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.218+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f491c021030 con 0x7f492009d6f0 2023-09-30T22:28:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.218+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f491c040b00 con 0x7f492009d6f0 2023-09-30T22:28:22.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.219+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4920074ea0 con 0x7f492009d6f0 2023-09-30T22:28:22.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.223+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f491c004560 con 0x7f492009d6f0 2023-09-30T22:28:22.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:22.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:22.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.433+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f492006c360 con 0x7f492009d6f0 2023-09-30T22:28:22.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.434+0000 7f49167fc700 1 -- 172.21.15.32:0/1809869588 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f491c015020 con 0x7f492009d6f0 2023-09-30T22:28:22.435 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 msgr2=0x7f49201222c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 0x7f49201222c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f491c009c40 tx=0x7f491c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 shutdown_connections 2023-09-30T22:28:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f492009e410 0x7f4920122d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f492009d6f0 0x7f49201222c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 --2- 172.21.15.32:0/1809869588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492009e000 0x7f4920122800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 >> 172.21.15.32:0/1809869588 conn(0x7f4920066f50 msgr2=0x7f4920130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 shutdown_connections 2023-09-30T22:28:22.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.436+0000 7f49281eb700 1 -- 172.21.15.32:0/1809869588 wait complete. 2023-09-30T22:28:22.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:22.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.527+0000 7fa273d7f700 1 Processor -- start 2023-09-30T22:28:22.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.527+0000 7fa273d7f700 1 -- start start 2023-09-30T22:28:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa273d7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa273d7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c09cf60 0x7fa26c132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa273d7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c09ba70 con 0x7fa26c0a0090 2023-09-30T22:28:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa273d7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c09e8f0 con 0x7fa26c09c650 2023-09-30T22:28:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa273d7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c09d400 con 0x7fa26c09cf60 2023-09-30T22:28:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa27131a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa26c0a0090 0x7fa26c131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48092/0 (socket says 172.21.15.32:48092) 2023-09-30T22:28:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa27131a700 1 -- 172.21.15.32:0/976248717 learned_addr learned my addr 172.21.15.32:0/976248717 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa271b1b700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.528+0000 7fa27231c700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c09cf60 0x7fa26c132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa271b1b700 1 -- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c09cf60 msgr2=0x7fa26c132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa271b1b700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c09cf60 0x7fa26c132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa270b19700 1 -- 172.21.15.32:0/976248717 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26732657 0 0) 0x7fa26c09ba70 con 0x7fa26c0a0090 2023-09-30T22:28:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa271b1b700 1 -- 172.21.15.32:0/976248717 >> v1:172.21.15.32:6789/0 conn(0x7fa26c0a0090 legacy=0x7fa26c131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa271b1b700 1 -- 172.21.15.32:0/976248717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c0a0760 con 0x7fa26c09c650 2023-09-30T22:28:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa271b1b700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c09ca20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa268009870 tx=0x7fa268009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7aa58eea2e7256 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa270b19700 1 -- 172.21.15.32:0/976248717 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268011830 con 0x7fa26c09c650 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa270b19700 1 -- 172.21.15.32:0/976248717 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa268011990 con 0x7fa26c09c650 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.529+0000 7fa270b19700 1 -- 172.21.15.32:0/976248717 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268011d20 con 0x7fa26c09c650 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 -- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 msgr2=0x7fa26c09ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c09ca20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa268009870 tx=0x7fa268009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 -- 172.21.15.32:0/976248717 shutdown_connections 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c09cf60 0x7fa26c132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 --2- 172.21.15.32:0/976248717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c09ca20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 -- 172.21.15.32:0/976248717 >> 172.21.15.32:0/976248717 conn(0x7fa26c0a3f70 msgr2=0x7fa26c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 -- 172.21.15.32:0/976248717 shutdown_connections 2023-09-30T22:28:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.530+0000 7fa273d7f700 1 -- 172.21.15.32:0/976248717 wait complete. 2023-09-30T22:28:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.531+0000 7fa273d7f700 1 Processor -- start 2023-09-30T22:28:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.531+0000 7fa273d7f700 1 -- start start 2023-09-30T22:28:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa273d7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c11a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa273d7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26c09cf60 0x7fa26c11ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa273d7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c0a0090 0x7fa26c11b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa273d7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c11e330 con 0x7fa26c09cf60 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa273d7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c11ba90 con 0x7fa26c09c650 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa273d7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c11d970 con 0x7fa26c0a0090 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c11a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa27131a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26c09cf60 0x7fa26c11ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c11a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42728/0 (socket says 172.21.15.32:42728) 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 -- 172.21.15.32:0/1755870427 learned_addr learned my addr 172.21.15.32:0/1755870427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa27231c700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c0a0090 0x7fa26c11b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 -- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c0a0090 msgr2=0x7fa26c11b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c0a0090 0x7fa26c11b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 -- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26c09cf60 msgr2=0x7fa26c11ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26c09cf60 0x7fa26c11ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa271b1b700 1 -- 172.21.15.32:0/1755870427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680094a0 con 0x7fa26c09c650 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.532+0000 7fa27131a700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26c09cf60 0x7fa26c11ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.533+0000 7fa271b1b700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c11a620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa26c09bc00 tx=0x7fa268004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.533+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268004510 con 0x7fa26c09c650 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.533+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26801f070 con 0x7fa26c09c650 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.533+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2680038c0 con 0x7fa26c09c650 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.533+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c12fc00 con 0x7fa26c09c650 2023-09-30T22:28:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.533+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c11c1e0 con 0x7fa26c09c650 2023-09-30T22:28:22.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.534+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa268003a20 con 0x7fa26c09c650 2023-09-30T22:28:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.534+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa268021070 con 0x7fa26c09c650 2023-09-30T22:28:22.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.534+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c0747f0 con 0x7fa26c09c650 2023-09-30T22:28:22.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.543+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa268015020 con 0x7fa26c09c650 2023-09-30T22:28:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.579+0000 7fb0968df700 1 Processor -- start 2023-09-30T22:28:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb0968df700 1 -- start start 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb0968df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0900a0a30 0x7fb0900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb0968df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb09013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb0968df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900a2a70 con 0x7fb09009d630 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb0968df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0900a01d0 con 0x7fb0900a0a30 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb0968df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb09009e2b0 con 0x7fb090138d80 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb08ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0900a0a30 0x7fb0900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb08f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb09009d630 0x7fb0900a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48106/0 (socket says 172.21.15.32:48106) 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.580+0000 7fb08f7fe700 1 -- 172.21.15.32:0/1291699505 learned_addr learned my addr 172.21.15.32:0/1291699505 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08ffff700 1 -- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 msgr2=0x7fb09013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08ffff700 1 --2- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb09013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08ffff700 1 -- 172.21.15.32:0/1291699505 >> v1:172.21.15.32:6789/0 conn(0x7fb09009d630 legacy=0x7fb0900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08effd700 1 -- 172.21.15.32:0/1291699505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889271604 0 0) 0x7fb0900a2a70 con 0x7fb09009d630 2023-09-30T22:28:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08ffff700 1 -- 172.21.15.32:0/1291699505 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09009dc50 con 0x7fb0900a0a30 2023-09-30T22:28:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08ffff700 1 --2- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb0900a0e00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb080009840 tx=0x7fb080009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6f00e5b18fc781 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08effd700 1 -- 172.21.15.32:0/1291699505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011830 con 0x7fb0900a0a30 2023-09-30T22:28:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08effd700 1 -- 172.21.15.32:0/1291699505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080011990 con 0x7fb0900a0a30 2023-09-30T22:28:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.581+0000 7fb08effd700 1 -- 172.21.15.32:0/1291699505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011d20 con 0x7fb0900a0a30 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 -- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 msgr2=0x7fb0900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 --2- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb0900a0e00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb080009840 tx=0x7fb080009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 -- 172.21.15.32:0/1291699505 shutdown_connections 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 --2- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb09013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 --2- 172.21.15.32:0/1291699505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb0900a0e00 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 -- 172.21.15.32:0/1291699505 >> 172.21.15.32:0/1291699505 conn(0x7fb0900a3050 msgr2=0x7fb0900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 -- 172.21.15.32:0/1291699505 shutdown_connections 2023-09-30T22:28:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.582+0000 7fb0968df700 1 -- 172.21.15.32:0/1291699505 wait complete. 2023-09-30T22:28:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.583+0000 7fb0968df700 1 Processor -- start 2023-09-30T22:28:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.583+0000 7fb0968df700 1 -- start start 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb0968df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09009d630 0x7fb0901206d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb0968df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb090120c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb0968df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb090121150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb0968df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090121ba0 con 0x7fb0900a0a30 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb0968df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb090123510 con 0x7fb09009d630 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb0968df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb090138a80 con 0x7fb090138d80 2023-09-30T22:28:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb090121150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb090121150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43488/0 (socket says 172.21.15.32:43488) 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb08ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09009d630 0x7fb0901206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 -- 172.21.15.32:0/4078253162 learned_addr learned my addr 172.21.15.32:0/4078253162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb08f7fe700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb090120c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 -- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09009d630 msgr2=0x7fb0901206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09009d630 0x7fb0901206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 -- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 msgr2=0x7fb090120c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb090120c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 -- 172.21.15.32:0/4078253162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0800094a0 con 0x7fb090138d80 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb08ffff700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09009d630 0x7fb0901206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:22.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb08f7fe700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb090120c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.584+0000 7fb094e7c700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb090121150 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb08800ba10 tx=0x7fb08800bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.585+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800d8e0 con 0x7fb090138d80 2023-09-30T22:28:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.585+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800da80 con 0x7fb090138d80 2023-09-30T22:28:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.585+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0901238f0 con 0x7fb090138d80 2023-09-30T22:28:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.585+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800de30 con 0x7fb090138d80 2023-09-30T22:28:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.585+0000 7f98e0135700 1 Processor -- start 2023-09-30T22:28:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.585+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0900a0410 con 0x7fb090138d80 2023-09-30T22:28:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7f98e0135700 1 -- start start 2023-09-30T22:28:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb088015420 con 0x7fb090138d80 2023-09-30T22:28:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090122a90 con 0x7fb090138d80 2023-09-30T22:28:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb088011070 con 0x7fb090138d80 2023-09-30T22:28:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7f98e0135700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98d8130b30 0x7f98d8138890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7f98e0135700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8138dd0 0x7f98d813b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7f98e0135700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d8099ef0 con 0x7f98d8071b40 2023-09-30T22:28:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7f98e0135700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d8099500 con 0x7f98d8130b30 2023-09-30T22:28:22.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.586+0000 7f98e0135700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8130f50 con 0x7f98d8138dd0 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dd6d0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98d8130b30 0x7f98d8138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dded1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98d8071b40 0x7f98d8130420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48112/0 (socket says 172.21.15.32:48112) 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dded1700 1 -- 172.21.15.32:0/3649304233 learned_addr learned my addr 172.21.15.32:0/3649304233 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98de6d2700 1 --2- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8138dd0 0x7f98d813b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dd6d0700 1 -- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8138dd0 msgr2=0x7f98d813b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dd6d0700 1 --2- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8138dd0 0x7f98d813b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dd6d0700 1 -- 172.21.15.32:0/3649304233 >> v1:172.21.15.32:6789/0 conn(0x7f98d8071b40 legacy=0x7f98d8130420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dd6d0700 1 -- 172.21.15.32:0/3649304233 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d8099c10 con 0x7f98d8130b30 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dcecf700 1 -- 172.21.15.32:0/3649304233 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950078030 0 0) 0x7f98d8099ef0 con 0x7f98d8071b40 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dd6d0700 1 --2- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8130b30 0x7f98d8138890 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009840 tx=0x7f98d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7174b76f85d121b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dcecf700 1 -- 172.21.15.32:0/3649304233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011830 con 0x7f98d8130b30 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dcecf700 1 -- 172.21.15.32:0/3649304233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4011990 con 0x7f98d8130b30 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.587+0000 7f98dcecf700 1 -- 172.21.15.32:0/3649304233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011d20 con 0x7f98d8130b30 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 -- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8130b30 msgr2=0x7f98d8138890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 --2- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8130b30 0x7f98d8138890 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009840 tx=0x7f98d4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 -- 172.21.15.32:0/3649304233 shutdown_connections 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 --2- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8138dd0 0x7f98d813b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 --2- 172.21.15.32:0/3649304233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8130b30 0x7f98d8138890 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 -- 172.21.15.32:0/3649304233 >> 172.21.15.32:0/3649304233 conn(0x7f98d809e7e0 msgr2=0x7f98d809ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 -- 172.21.15.32:0/3649304233 shutdown_connections 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.588+0000 7f98e0135700 1 -- 172.21.15.32:0/3649304233 wait complete. 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.589+0000 7f98e0135700 1 Processor -- start 2023-09-30T22:28:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.589+0000 7f98e0135700 1 -- start start 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98e0135700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98d8071b40 0x7f98d8126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98e0135700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 0x7f98d8127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98e0135700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8138dd0 0x7f98d8127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98e0135700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8129530 con 0x7f98d8138dd0 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98e0135700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d8129730 con 0x7f98d8071b40 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98e0135700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98d812f6a0 con 0x7f98d8130b30 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98dd6d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 0x7f98d8127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98de6d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8138dd0 0x7f98d8127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98dd6d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 0x7f98d8127340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43516/0 (socket says 172.21.15.32:43516) 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98dded1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98d8071b40 0x7f98d8126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98dd6d0700 1 -- 172.21.15.32:0/923003949 learned_addr learned my addr 172.21.15.32:0/923003949 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:22.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98de6d2700 1 -- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 msgr2=0x7f98d8127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98de6d2700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 0x7f98d8127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98de6d2700 1 -- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98d8071b40 msgr2=0x7f98d8126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98de6d2700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98d8071b40 0x7f98d8126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98de6d2700 1 -- 172.21.15.32:0/923003949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40094a0 con 0x7f98d8138dd0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.590+0000 7f98dd6d0700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 0x7f98d8127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.591+0000 7f98de6d2700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8138dd0 0x7f98d8127880 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e850 tx=0x7f98c800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.591+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8013680 con 0x7f98d8138dd0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.591+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c80137e0 con 0x7f98d8138dd0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.591+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d8138300 con 0x7f98d8138dd0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.591+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8013b90 con 0x7f98d8138dd0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.591+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d809c150 con 0x7f98d8138dd0 2023-09-30T22:28:22.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.592+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d80747f0 con 0x7f98d8138dd0 2023-09-30T22:28:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.593+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98c8013cf0 con 0x7f98d8138dd0 2023-09-30T22:28:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.593+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f98c803fc50 con 0x7f98d8138dd0 2023-09-30T22:28:22.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.593+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb088013920 con 0x7fb090138d80 2023-09-30T22:28:22.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.597+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98c803fe60 con 0x7f98d8138dd0 2023-09-30T22:28:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.757+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa26c072840 con 0x7fa26c09c650 2023-09-30T22:28:22.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.757+0000 7fa2627fc700 1 -- 172.21.15.32:0/1755870427 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa26801a430 con 0x7fa26c09c650 2023-09-30T22:28:22.759 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 msgr2=0x7fa26c11a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c11a620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa26c09bc00 tx=0x7fa268004750 comp rx=0 tx=0).stop 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 shutdown_connections 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa26c0a0090 0x7fa26c11b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa26c09c650 0x7fa26c11a620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 --2- 172.21.15.32:0/1755870427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa26c09cf60 0x7fa26c11ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 >> 172.21.15.32:0/1755870427 conn(0x7fa26c0a3f70 msgr2=0x7fa26c12ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 shutdown_connections 2023-09-30T22:28:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.760+0000 7fa273d7f700 1 -- 172.21.15.32:0/1755870427 wait complete. 2023-09-30T22:28:22.774 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:22.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.801+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb0900747f0 con 0x7fb090138d80 2023-09-30T22:28:22.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.802+0000 7fb08cff9700 1 -- 172.21.15.32:0/4078253162 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb088013330 con 0x7fb090138d80 2023-09-30T22:28:22.803 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:22.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 msgr2=0x7fb090121150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb090121150 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb08800ba10 tx=0x7fb08800bd20 comp rx=0 tx=0).stop 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 shutdown_connections 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb090138d80 0x7fb090121150 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09009d630 0x7fb0901206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 --2- 172.21.15.32:0/4078253162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0900a0a30 0x7fb090120c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.805+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 >> 172.21.15.32:0/4078253162 conn(0x7fb0900a3050 msgr2=0x7fb090139260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.806+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 shutdown_connections 2023-09-30T22:28:22.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.806+0000 7fb0968df700 1 -- 172.21.15.32:0/4078253162 wait complete. 2023-09-30T22:28:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.817+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98d81382a0 con 0x7f98d8138dd0 2023-09-30T22:28:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.817+0000 7f98ce7fc700 1 -- 172.21.15.32:0/923003949 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98c8017070 con 0x7f98d8138dd0 2023-09-30T22:28:22.818 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.819+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8138dd0 msgr2=0x7f98d8127880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.819+0000 7f98e0135700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8138dd0 0x7f98d8127880 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e850 tx=0x7f98c800eb60 comp rx=0 tx=0).stop 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 shutdown_connections 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98d8138dd0 0x7f98d8127880 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98d8071b40 0x7f98d8126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 --2- 172.21.15.32:0/923003949 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98d8130b30 0x7f98d8127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 >> 172.21.15.32:0/923003949 conn(0x7f98d809e7e0 msgr2=0x7f98d809c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 shutdown_connections 2023-09-30T22:28:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:22.820+0000 7f98e0135700 1 -- 172.21.15.32:0/923003949 wait complete. 2023-09-30T22:28:22.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:22.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:23.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:23.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.602+0000 7fd993dba700 1 Processor -- start 2023-09-30T22:28:23.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.602+0000 7fd993dba700 1 -- start start 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd993dba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd98c09d6f0 0x7fd98c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd993dba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd993dba700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c09d2f0 con 0x7fd98c09e000 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd993dba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98c09a810 con 0x7fd98c09d6f0 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd993dba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c09a470 con 0x7fd98c134100 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991b56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd98c09d6f0 0x7fd98c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd992357700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991355700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd98c09e000 0x7fd98c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48118/0 (socket says 172.21.15.32:48118) 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991355700 1 -- 172.21.15.32:0/1929643112 learned_addr learned my addr 172.21.15.32:0/1929643112 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991b56700 1 -- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 msgr2=0x7fd98c13bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:23.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991b56700 1 --2- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c13bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991b56700 1 -- 172.21.15.32:0/1929643112 >> v1:172.21.15.32:6789/0 conn(0x7fd98c09e000 legacy=0x7fd98c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd991b56700 1 -- 172.21.15.32:0/1929643112 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c09ec40 con 0x7fd98c09d6f0 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd990b54700 1 -- 172.21.15.32:0/1929643112 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424062211 0 0) 0x7fd98c09d2f0 con 0x7fd98c09e000 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.603+0000 7fd992357700 1 --2- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd991b56700 1 --2- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c09dac0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009840 tx=0x7fd97c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2af9aa502e8edc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd990b54700 1 -- 172.21.15.32:0/1929643112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c011830 con 0x7fd98c09d6f0 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd990b54700 1 -- 172.21.15.32:0/1929643112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c011990 con 0x7fd98c09d6f0 2023-09-30T22:28:23.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd990b54700 1 -- 172.21.15.32:0/1929643112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c011d20 con 0x7fd98c09d6f0 2023-09-30T22:28:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 -- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 msgr2=0x7fd98c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 --2- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c09dac0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009840 tx=0x7fd97c009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 -- 172.21.15.32:0/1929643112 shutdown_connections 2023-09-30T22:28:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 --2- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 --2- 172.21.15.32:0/1929643112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c09dac0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 -- 172.21.15.32:0/1929643112 >> 172.21.15.32:0/1929643112 conn(0x7fd98c066f50 msgr2=0x7fd98c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 -- 172.21.15.32:0/1929643112 shutdown_connections 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.604+0000 7fd993dba700 1 -- 172.21.15.32:0/1929643112 wait complete. 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.605+0000 7fd993dba700 1 Processor -- start 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 -- start start 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c122630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd98c09e000 0x7fd98c122b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c1230b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c123a20 con 0x7fd98c09d6f0 2023-09-30T22:28:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c1242e0 con 0x7fd98c134100 2023-09-30T22:28:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd993dba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c125bb0 con 0x7fd98c09e000 2023-09-30T22:28:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c122630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c122630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46046/0 (socket says 172.21.15.32:46046) 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 -- 172.21.15.32:0/64653776 learned_addr learned my addr 172.21.15.32:0/64653776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991355700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd98c09e000 0x7fd98c122b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd992357700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c1230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 -- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd98c09e000 msgr2=0x7fd98c122b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd98c09e000 0x7fd98c122b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 -- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 msgr2=0x7fd98c1230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c1230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.606+0000 7fd991b56700 1 -- 172.21.15.32:0/64653776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984009a70 con 0x7fd98c09d6f0 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.607+0000 7fd991b56700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c122630 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009fd0 tx=0x7fd97c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:23.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.607+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0045d0 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.607+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c01f070 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.607+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0094a0 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.607+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0038a0 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.607+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c1268e0 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.608+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd97c003a00 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.608+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd97c021070 con 0x7fd98c09d6f0 2023-09-30T22:28:23.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.608+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c074ea0 con 0x7fd98c09d6f0 2023-09-30T22:28:23.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.613+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd97c010380 con 0x7fd98c09d6f0 2023-09-30T22:28:23.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.816+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd98c125dd0 con 0x7fd98c09d6f0 2023-09-30T22:28:23.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.817+0000 7fd9827fc700 1 -- 172.21.15.32:0/64653776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd97c02eb90 con 0x7fd98c09d6f0 2023-09-30T22:28:23.818 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 msgr2=0x7fd98c122630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c122630 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009fd0 tx=0x7fd97c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:23.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 shutdown_connections 2023-09-30T22:28:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd98c134100 0x7fd98c1230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98c09d6f0 0x7fd98c122630 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 --2- 172.21.15.32:0/64653776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd98c09e000 0x7fd98c122b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 >> 172.21.15.32:0/64653776 conn(0x7fd98c066f50 msgr2=0x7fd98c070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 shutdown_connections 2023-09-30T22:28:23.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.819+0000 7fd993dba700 1 -- 172.21.15.32:0/64653776 wait complete. 2023-09-30T22:28:23.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:23.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:23.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.916+0000 7f92edf77700 1 Processor -- start 2023-09-30T22:28:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.916+0000 7f92edf77700 1 -- start start 2023-09-30T22:28:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92edf77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 0x7f92e8138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92edf77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 0x7f92e813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92edf77700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e806bc00 con 0x7f92e8133a80 2023-09-30T22:28:23.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92edf77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e806c570 con 0x7f92e8139140 2023-09-30T22:28:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92edf77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80a0040 con 0x7f92e8136820 2023-09-30T22:28:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 0x7f92e813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 0x7f92e8138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 0x7f92e8138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43534/0 (socket says 172.21.15.32:43534) 2023-09-30T22:28:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 0x7f92e813b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46062/0 (socket says 172.21.15.32:46062) 2023-09-30T22:28:23.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 -- 172.21.15.32:0/47633339 learned_addr learned my addr 172.21.15.32:0/47633339 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 -- 172.21.15.32:0/47633339 >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 msgr2=0x7f92e813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 --2- 172.21.15.32:0/47633339 >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 0x7f92e813b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 -- 172.21.15.32:0/47633339 >> v1:172.21.15.32:6789/0 conn(0x7f92e8133a80 legacy=0x7f92e8133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e6ffd700 1 -- 172.21.15.32:0/47633339 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8133860 con 0x7f92e8136820 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.917+0000 7f92e7fff700 1 --2- 172.21.15.32:0/47633339 >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 0x7f92e813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.918+0000 7f92e6ffd700 1 --2- 172.21.15.32:0/47633339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 0x7f92e8138c00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0098f0 tx=0x7f92dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1cdbec40e1b2a69 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.918+0000 7f92e67fc700 1 -- 172.21.15.32:0/47633339 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011830 con 0x7f92e8136820 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.918+0000 7f92e67fc700 1 -- 172.21.15.32:0/47633339 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc011990 con 0x7f92e8136820 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.918+0000 7f92e67fc700 1 -- 172.21.15.32:0/47633339 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011d20 con 0x7f92e8136820 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 -- 172.21.15.32:0/47633339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 msgr2=0x7f92e8138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 --2- 172.21.15.32:0/47633339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 0x7f92e8138c00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0098f0 tx=0x7f92dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 -- 172.21.15.32:0/47633339 shutdown_connections 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 --2- 172.21.15.32:0/47633339 >> v2:172.21.15.32:3300/0 conn(0x7f92e8139140 0x7f92e813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 --2- 172.21.15.32:0/47633339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8136820 0x7f92e8138c00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 -- 172.21.15.32:0/47633339 >> 172.21.15.32:0/47633339 conn(0x7f92e80a4210 msgr2=0x7f92e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 -- 172.21.15.32:0/47633339 shutdown_connections 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.919+0000 7f92edf77700 1 -- 172.21.15.32:0/47633339 wait complete. 2023-09-30T22:28:23.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.920+0000 7f92edf77700 1 Processor -- start 2023-09-30T22:28:23.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.920+0000 7f92edf77700 1 -- start start 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92edf77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92e8133a80 0x7f92e806f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92edf77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 0x7f92e806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92edf77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8139140 0x7f92e80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92edf77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e809bdd0 con 0x7f92e8136820 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92edf77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8138d10 con 0x7f92e8133a80 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92edf77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92e809e080 con 0x7f92e8139140 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 0x7f92e806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8139140 0x7f92e80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 0x7f92e806fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46076/0 (socket says 172.21.15.32:46076) 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.921+0000 7f92e6ffd700 1 -- 172.21.15.32:0/4093677267 learned_addr learned my addr 172.21.15.32:0/4093677267 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:23.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e7fff700 1 -- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92e8133a80 msgr2=0x7f92e806f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e7fff700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92e8133a80 0x7f92e806f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e7fff700 1 -- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 msgr2=0x7f92e806fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e7fff700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 0x7f92e806fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e7fff700 1 -- 172.21.15.32:0/4093677267 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009a70 con 0x7f92e8139140 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e6ffd700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 0x7f92e806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92e7fff700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8139140 0x7f92e80703d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f92e000ef90 tx=0x7f92e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000cae0 con 0x7f92e8139140 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e000f040 con 0x7f92e8139140 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc009550 con 0x7f92e8139140 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000ce80 con 0x7f92e8139140 2023-09-30T22:28:23.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.922+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e8130380 con 0x7f92e8139140 2023-09-30T22:28:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.923+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e8074ea0 con 0x7f92e8139140 2023-09-30T22:28:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.924+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f92e0014480 con 0x7f92e8139140 2023-09-30T22:28:23.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.924+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f92e003fce0 con 0x7f92e8139140 2023-09-30T22:28:23.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.929+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92e0012930 con 0x7f92e8139140 2023-09-30T22:28:23.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.966+0000 7f8045325700 1 Processor -- start 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.966+0000 7f8045325700 1 -- start start 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.966+0000 7f8045325700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f804012cc20 0x7f804012f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.966+0000 7f8045325700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012cff0 0x7f8040137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f8045325700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804009a510 con 0x7f804012c2f0 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f8045325700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804012c7d0 con 0x7f804012cc20 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f8045325700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f804009c190 con 0x7f804012cff0 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f804012cc20 0x7f804012f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f804012cc20 0x7f804012f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46090/0 (socket says 172.21.15.32:46090) 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 -- 172.21.15.32:0/1372509130 learned_addr learned my addr 172.21.15.32:0/1372509130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803e7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f804012c2f0 0x7f804012fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48130/0 (socket says 172.21.15.32:48130) 2023-09-30T22:28:23.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803f7fe700 1 --2- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012cff0 0x7f8040137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 -- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012cff0 msgr2=0x7f8040137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 --2- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012cff0 0x7f8040137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 -- 172.21.15.32:0/1372509130 >> v1:172.21.15.32:6789/0 conn(0x7f804012c2f0 legacy=0x7f804012fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 -- 172.21.15.32:0/1372509130 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804009b540 con 0x7f804012cc20 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.967+0000 7f803effd700 1 --2- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804012f930 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f80300098f0 tx=0x7f8030009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2fc6f30ea1af08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f803dffb700 1 -- 172.21.15.32:0/1372509130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000f8a0 con 0x7f804012cc20 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f803dffb700 1 -- 172.21.15.32:0/1372509130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803000fa00 con 0x7f804012cc20 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f803dffb700 1 -- 172.21.15.32:0/1372509130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000fd90 con 0x7f804012cc20 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 -- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 msgr2=0x7f804012f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 --2- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804012f930 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f80300098f0 tx=0x7f8030009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 -- 172.21.15.32:0/1372509130 shutdown_connections 2023-09-30T22:28:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 --2- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012cff0 0x7f8040137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 --2- 172.21.15.32:0/1372509130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804012f930 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 -- 172.21.15.32:0/1372509130 >> 172.21.15.32:0/1372509130 conn(0x7f804009f470 msgr2=0x7f80400a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 -- 172.21.15.32:0/1372509130 shutdown_connections 2023-09-30T22:28:23.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.968+0000 7f8045325700 1 -- 172.21.15.32:0/1372509130 wait complete. 2023-09-30T22:28:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 Processor -- start 2023-09-30T22:28:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 -- start start 2023-09-30T22:28:23.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 0x7f804006ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804012cff0 0x7f804006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400715e0 con 0x7f804012cc20 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8040129f10 con 0x7f804012cff0 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f8045325700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8040070880 con 0x7f804012c2f0 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f803e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f803effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 0x7f804006ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f803effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 0x7f804006ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43560/0 (socket says 172.21.15.32:43560) 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f803effd700 1 -- 172.21.15.32:0/3936813254 learned_addr learned my addr 172.21.15.32:0/3936813254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f803e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46092/0 (socket says 172.21.15.32:46092) 2023-09-30T22:28:23.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.970+0000 7f803f7fe700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804012cff0 0x7f804006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803effd700 1 -- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804012cff0 msgr2=0x7f804006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803effd700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804012cff0 0x7f804006f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803effd700 1 -- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 msgr2=0x7f804006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803effd700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804006f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803effd700 1 -- 172.21.15.32:0/3936813254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009550 con 0x7f804012c2f0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803e7fc700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f803effd700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 0x7f804006ed80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8030009c40 tx=0x7f8030004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300038a0 con 0x7f804012c2f0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030003a00 con 0x7f804012c2f0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804008f510 con 0x7f804012c2f0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030003db0 con 0x7f804012c2f0 2023-09-30T22:28:23.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.971+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8040072480 con 0x7f804012c2f0 2023-09-30T22:28:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.972+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8030019400 con 0x7f804012c2f0 2023-09-30T22:28:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.972+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f80300404a0 con 0x7f804012c2f0 2023-09-30T22:28:23.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.972+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80401285a0 con 0x7f804012c2f0 2023-09-30T22:28:23.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:23.977+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f803002a3f0 con 0x7f804012c2f0 2023-09-30T22:28:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.017+0000 7fc6fe23e700 1 Processor -- start 2023-09-30T22:28:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.017+0000 7fc6fe23e700 1 -- start start 2023-09-30T22:28:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6fe23e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6fe23e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 0x7fc6f81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6fe23e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f809b120 con 0x7fc6f809e410 2023-09-30T22:28:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6fe23e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f806b460 con 0x7fc6f809d6f0 2023-09-30T22:28:24.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6fe23e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f809a7e0 con 0x7fc6f809e000 2023-09-30T22:28:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 0x7fc6f81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 0x7fc6f81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43564/0 (socket says 172.21.15.32:43564) 2023-09-30T22:28:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f6ffd700 1 -- 172.21.15.32:0/3911978704 learned_addr learned my addr 172.21.15.32:0/3911978704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f77fe700 1 -- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 msgr2=0x7fc6f81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f77fe700 1 --2- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 0x7fc6f81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f77fe700 1 -- 172.21.15.32:0/3911978704 >> v1:172.21.15.32:6789/0 conn(0x7fc6f809e410 legacy=0x7fc6f8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f67fc700 1 -- 172.21.15.32:0/3911978704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474399952 0 0) 0x7fc6f809b120 con 0x7fc6f809e410 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f77fe700 1 -- 172.21.15.32:0/3911978704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f809a500 con 0x7fc6f809d6f0 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.018+0000 7fc6f6ffd700 1 --2- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 0x7fc6f81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.019+0000 7fc6f77fe700 1 --2- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f809dac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009870 tx=0x7fc6e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7345097925cf6675 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.019+0000 7fc6f67fc700 1 -- 172.21.15.32:0/3911978704 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8011830 con 0x7fc6f809d6f0 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.019+0000 7fc6f67fc700 1 -- 172.21.15.32:0/3911978704 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8011990 con 0x7fc6f809d6f0 2023-09-30T22:28:24.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.019+0000 7fc6f67fc700 1 -- 172.21.15.32:0/3911978704 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8011d20 con 0x7fc6f809d6f0 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 msgr2=0x7fc6f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f809dac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009870 tx=0x7fc6e8009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3911978704 shutdown_connections 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f809dac0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3911978704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e000 0x7fc6f81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3911978704 >> 172.21.15.32:0/3911978704 conn(0x7fc6f8066f50 msgr2=0x7fc6f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3911978704 shutdown_connections 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.020+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3911978704 wait complete. 2023-09-30T22:28:24.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.021+0000 7fc6fe23e700 1 Processor -- start 2023-09-30T22:28:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 -- start start 2023-09-30T22:28:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f8122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 0x7fc6f8122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e410 0x7fc6f8122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f8124980 con 0x7fc6f809e000 2023-09-30T22:28:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f809c710 con 0x7fc6f809d6f0 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6fe23e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f80a2950 con 0x7fc6f809e410 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6f6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 0x7fc6f8122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6f6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 0x7fc6f8122a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46104/0 (socket says 172.21.15.32:46104) 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6f6ffd700 1 -- 172.21.15.32:0/3842304712 learned_addr learned my addr 172.21.15.32:0/3842304712 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.022+0000 7fc6f7fff700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e410 0x7fc6f8122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f77fe700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f8122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f7fff700 1 -- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 msgr2=0x7fc6f8122540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f7fff700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f8122540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f7fff700 1 -- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 msgr2=0x7fc6f8122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f7fff700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 0x7fc6f8122a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f7fff700 1 -- 172.21.15.32:0/3842304712 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec009550 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f6ffd700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 0x7fc6f8122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6f7fff700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e410 0x7fc6f8122fc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc6f809e8b0 tx=0x7fc6f000caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0003e30 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0005070 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e80094a0 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0005420 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.023+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f8126060 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.024+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6f0005580 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.024+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f80a3a60 con 0x7fc6f809e410 2023-09-30T22:28:24.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.025+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc6f003ec10 con 0x7fc6f809e410 2023-09-30T22:28:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.029+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6f0005830 con 0x7fc6f809e410 2023-09-30T22:28:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.138+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f92e806b250 con 0x7f92e8139140 2023-09-30T22:28:24.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.138+0000 7f92cffff700 1 -- 172.21.15.32:0/4093677267 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92e0009cc0 con 0x7f92e8139140 2023-09-30T22:28:24.139 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:24.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8139140 msgr2=0x7f92e80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8139140 0x7f92e80703d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f92e000ef90 tx=0x7f92e000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 shutdown_connections 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92e8139140 0x7f92e80703d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f92e8133a80 0x7f92e806f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 --2- 172.21.15.32:0/4093677267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e8136820 0x7f92e806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.140+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 >> 172.21.15.32:0/4093677267 conn(0x7f92e80a4210 msgr2=0x7f92e80a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.141+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 shutdown_connections 2023-09-30T22:28:24.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.141+0000 7f92edf77700 1 -- 172.21.15.32:0/4093677267 wait complete. 2023-09-30T22:28:24.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:24.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.188+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f80400722c0 con 0x7f804012c2f0 2023-09-30T22:28:24.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.188+0000 7f80277fe700 1 -- 172.21.15.32:0/3936813254 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8030029360 con 0x7f804012c2f0 2023-09-30T22:28:24.189 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 msgr2=0x7f804006ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 0x7f804006ed80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8030009c40 tx=0x7f8030004750 comp rx=0 tx=0).stop 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 shutdown_connections 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f804012cff0 0x7f804006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f804012c2f0 0x7f804006ed80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 --2- 172.21.15.32:0/3936813254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804012cc20 0x7f804006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 >> 172.21.15.32:0/3936813254 conn(0x7f804009f470 msgr2=0x7f804012f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 shutdown_connections 2023-09-30T22:28:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.191+0000 7f8045325700 1 -- 172.21.15.32:0/3936813254 wait complete. 2023-09-30T22:28:24.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.238+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc6f8074ea0 con 0x7fc6f809e410 2023-09-30T22:28:24.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.238+0000 7fc6dffff700 1 -- 172.21.15.32:0/3842304712 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6f0017020 con 0x7fc6f809e410 2023-09-30T22:28:24.239 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:24.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.240+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e410 msgr2=0x7fc6f8122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.240+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e410 0x7fc6f8122fc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc6f809e8b0 tx=0x7fc6f000caa0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.240+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 shutdown_connections 2023-09-30T22:28:24.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.241+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f809e410 0x7fc6f8122fc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.241+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f809d6f0 0x7fc6f8122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.241+0000 7fc6fe23e700 1 --2- 172.21.15.32:0/3842304712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f809e000 0x7fc6f8122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.241+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 >> 172.21.15.32:0/3842304712 conn(0x7fc6f8066f50 msgr2=0x7fc6f80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:24.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.241+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 shutdown_connections 2023-09-30T22:28:24.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.241+0000 7fc6fe23e700 1 -- 172.21.15.32:0/3842304712 wait complete. 2023-09-30T22:28:24.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:24.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:24.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.980+0000 7fa0fbf69700 1 Processor -- start 2023-09-30T22:28:24.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.980+0000 7fa0fbf69700 1 -- start start 2023-09-30T22:28:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0fbf69700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0f409e930 0x7fa0f409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0fbf69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0fbf69700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f4070940 con 0x7fa0f409f650 2023-09-30T22:28:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0fbf69700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f406fd60 con 0x7fa0f409e930 2023-09-30T22:28:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0fbf69700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f406fa80 con 0x7fa0f409f240 2023-09-30T22:28:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0f409e930 0x7fa0f409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9504700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0f409e930 0x7fa0f409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46108/0 (socket says 172.21.15.32:46108) 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 -- 172.21.15.32:0/2293915501 learned_addr learned my addr 172.21.15.32:0/2293915501 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9504700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42782/0 (socket says 172.21.15.32:42782) 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 -- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 msgr2=0x7fa0f41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 --2- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 -- 172.21.15.32:0/2293915501 >> v1:172.21.15.32:6789/0 conn(0x7fa0f409f650 legacy=0x7fa0f4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 -- 172.21.15.32:0/2293915501 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f409fb60 con 0x7fa0f409e930 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9504700 1 --2- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f9d05700 1 --2- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409e930 0x7fa0f409ed00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009e50 tx=0x7fa0e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b734c58df65b8e29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.981+0000 7fa0f8d03700 1 -- 172.21.15.32:0/2293915501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400e070 con 0x7fa0f409e930 2023-09-30T22:28:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0f8d03700 1 -- 172.21.15.32:0/2293915501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400b910 con 0x7fa0f409e930 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0f8d03700 1 -- 172.21.15.32:0/2293915501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400bcc0 con 0x7fa0f409e930 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 -- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409e930 msgr2=0x7fa0f409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409e930 0x7fa0f409ed00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009e50 tx=0x7fa0e400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 -- 172.21.15.32:0/2293915501 shutdown_connections 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409e930 0x7fa0f409ed00 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/2293915501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 -- 172.21.15.32:0/2293915501 >> 172.21.15.32:0/2293915501 conn(0x7fa0f40a4210 msgr2=0x7fa0f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 -- 172.21.15.32:0/2293915501 shutdown_connections 2023-09-30T22:28:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.982+0000 7fa0fbf69700 1 -- 172.21.15.32:0/2293915501 wait complete. 2023-09-30T22:28:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.983+0000 7fa0fbf69700 1 Processor -- start 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.983+0000 7fa0fbf69700 1 -- start start 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fbf69700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 0x7fa0f406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fbf69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fbf69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 0x7fa0f406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fbf69700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41b7350 con 0x7fa0f409f650 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fbf69700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41a9500 con 0x7fa0f409f240 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fbf69700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f41a8a20 con 0x7fa0f409e930 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fa506700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 0x7fa0f406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9d05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 0x7fa0f406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42794/0 (socket says 172.21.15.32:42794) 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9d05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 0x7fa0f406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43586/0 (socket says 172.21.15.32:43586) 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 -- 172.21.15.32:0/4052744525 learned_addr learned my addr 172.21.15.32:0/4052744525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fa506700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 0x7fa0f406c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46124/0 (socket says 172.21.15.32:46124) 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 -- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 msgr2=0x7fa0f406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 0x7fa0f406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 -- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 msgr2=0x7fa0f406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 0x7fa0f406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 -- 172.21.15.32:0/4052744525 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80094a0 con 0x7fa0f409f240 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9d05700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 0x7fa0f406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0fa506700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 0x7fa0f406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:24.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.984+0000 7fa0f9504700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f406bdf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800ec10 tx=0x7fa0e800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.985+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800c850 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.985+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e800c9b0 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.985+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e4009b30 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.985+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800cd20 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.985+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f40a0190 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.985+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa0e8013470 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.986+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa0e803fb10 con 0x7fa0f409f240 2023-09-30T22:28:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.986+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f40a3530 con 0x7fa0f409f240 2023-09-30T22:28:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:24.991+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0e8013720 con 0x7fa0f409f240 2023-09-30T22:28:25.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.197+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa0f4071d20 con 0x7fa0f409f240 2023-09-30T22:28:25.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.197+0000 7fa0f27fc700 1 -- 172.21.15.32:0/4052744525 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0e8011ab0 con 0x7fa0f409f240 2023-09-30T22:28:25.198 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 msgr2=0x7fa0f406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f406bdf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800ec10 tx=0x7fa0e800ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 shutdown_connections 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0f409f650 0x7fa0f406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0f409e930 0x7fa0f406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 --2- 172.21.15.32:0/4052744525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0f409f240 0x7fa0f406bdf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 >> 172.21.15.32:0/4052744525 conn(0x7fa0f40a4210 msgr2=0x7fa0f41b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 shutdown_connections 2023-09-30T22:28:25.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.200+0000 7fa0fbf69700 1 -- 172.21.15.32:0/4052744525 wait complete. 2023-09-30T22:28:25.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:25.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:25.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.299+0000 7f98777b6700 1 Processor -- start 2023-09-30T22:28:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.299+0000 7f98777b6700 1 -- start start 2023-09-30T22:28:25.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f98777b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 0x7f9870133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f98777b6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 0x7f987009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f98777b6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987006ef30 con 0x7f987009e760 2023-09-30T22:28:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f98777b6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987006fef0 con 0x7f98701338a0 2023-09-30T22:28:25.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f98777b6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98701334b0 con 0x7f987009de50 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9874d51700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 0x7f987009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 0x7f9870133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 0x7f9870133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42804/0 (socket says 172.21.15.32:42804) 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9874d51700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 0x7f987009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43594/0 (socket says 172.21.15.32:43594) 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 -- 172.21.15.32:0/4237785132 learned_addr learned my addr 172.21.15.32:0/4237785132 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 -- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 msgr2=0x7f987009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:25.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 --2- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 0x7f987009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 -- 172.21.15.32:0/4237785132 >> v1:172.21.15.32:6789/0 conn(0x7f987009e760 legacy=0x7f987013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9875552700 1 -- 172.21.15.32:0/4237785132 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987006fc30 con 0x7f98701338a0 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.300+0000 7f9867fff700 1 -- 172.21.15.32:0/4237785132 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561406652 0 0) 0x7f987006ef30 con 0x7f987009e760 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.301+0000 7f9874d51700 1 --2- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 0x7f987009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.301+0000 7f9875552700 1 --2- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 0x7f9870133c70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f986c009920 tx=0x7f986c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1cc88204eccaae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.301+0000 7f9867fff700 1 -- 172.21.15.32:0/4237785132 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c011830 con 0x7f98701338a0 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.301+0000 7f9867fff700 1 -- 172.21.15.32:0/4237785132 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c011990 con 0x7f98701338a0 2023-09-30T22:28:25.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.301+0000 7f9867fff700 1 -- 172.21.15.32:0/4237785132 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c011d20 con 0x7f98701338a0 2023-09-30T22:28:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 -- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 msgr2=0x7f9870133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 --2- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 0x7f9870133c70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f986c009920 tx=0x7f986c009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 -- 172.21.15.32:0/4237785132 shutdown_connections 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 --2- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98701338a0 0x7f9870133c70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 --2- 172.21.15.32:0/4237785132 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009de50 0x7f987009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 -- 172.21.15.32:0/4237785132 >> 172.21.15.32:0/4237785132 conn(0x7f98700a2aa0 msgr2=0x7f98700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 -- 172.21.15.32:0/4237785132 shutdown_connections 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.302+0000 7f98777b6700 1 -- 172.21.15.32:0/4237785132 wait complete. 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 Processor -- start 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 -- start start 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 0x7f987011b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009e760 0x7f987011bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98701338a0 0x7f987011c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987011cbe0 con 0x7f98701338a0 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987011d2a0 con 0x7f987009de50 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.303+0000 7f98777b6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987011d8c0 con 0x7f987009e760 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 0x7f987011b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 0x7f987011b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42808/0 (socket says 172.21.15.32:42808) 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 -- 172.21.15.32:0/2079829890 learned_addr learned my addr 172.21.15.32:0/2079829890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875d53700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98701338a0 0x7f987011c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 -- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009e760 msgr2=0x7f987011bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009e760 0x7f987011bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 -- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98701338a0 msgr2=0x7f987011c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98701338a0 0x7f987011c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 -- 172.21.15.32:0/2079829890 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600094a0 con 0x7f987009de50 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9875552700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 0x7f987011b730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f986c009c70 tx=0x7f986c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0036a0 con 0x7f987009de50 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c003800 con 0x7f987009de50 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c003b90 con 0x7f987009de50 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.304+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c009550 con 0x7f987009de50 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.305+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98700a0eb0 con 0x7f987009de50 2023-09-30T22:28:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.306+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f986c003cf0 con 0x7f987009de50 2023-09-30T22:28:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.306+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f986c040a00 con 0x7f987009de50 2023-09-30T22:28:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.306+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987006f640 con 0x7f987009de50 2023-09-30T22:28:25.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.311+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f986c028080 con 0x7f987009de50 2023-09-30T22:28:25.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.347+0000 7f0eb337e700 1 Processor -- start 2023-09-30T22:28:25.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.347+0000 7f0eb337e700 1 -- start start 2023-09-30T22:28:25.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb337e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09f410 0x7f0eac136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb337e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac136fe0 0x7f0eac1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb337e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac070940 con 0x7f0eac09e930 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb337e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac06fd60 con 0x7f0eac09f410 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb337e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac06fa80 con 0x7f0eac136fe0 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb111a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0eac09e930 0x7f0eac09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48174/0 (socket says 172.21.15.32:48174) 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb111a700 1 -- 172.21.15.32:0/4190875896 learned_addr learned my addr 172.21.15.32:0/4190875896 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb191b700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac136fe0 0x7f0eac1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.348+0000 7f0eb0919700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09f410 0x7f0eac136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0eb191b700 1 -- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09f410 msgr2=0x7f0eac136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0ea3fff700 1 -- 172.21.15.32:0/4190875896 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032406730 0 0) 0x7f0eac070940 con 0x7f0eac09e930 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0eb191b700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09f410 0x7f0eac136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0eb191b700 1 -- 172.21.15.32:0/4190875896 >> v1:172.21.15.32:6789/0 conn(0x7f0eac09e930 legacy=0x7f0eac09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0eb191b700 1 -- 172.21.15.32:0/4190875896 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac09fb60 con 0x7f0eac136fe0 2023-09-30T22:28:25.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0eb191b700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac136fe0 0x7f0eac1393c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009dc0 tx=0x7f0ea400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83a923c79c6a946 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:25.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0ea3fff700 1 -- 172.21.15.32:0/4190875896 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400e070 con 0x7f0eac136fe0 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0ea3fff700 1 -- 172.21.15.32:0/4190875896 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea400bbd0 con 0x7f0eac136fe0 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.349+0000 7f0ea3fff700 1 -- 172.21.15.32:0/4190875896 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4012430 con 0x7f0eac136fe0 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 -- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac136fe0 msgr2=0x7f0eac1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac136fe0 0x7f0eac1393c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009dc0 tx=0x7f0ea400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 -- 172.21.15.32:0/4190875896 shutdown_connections 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac136fe0 0x7f0eac1393c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 --2- 172.21.15.32:0/4190875896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09f410 0x7f0eac136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 -- 172.21.15.32:0/4190875896 >> 172.21.15.32:0/4190875896 conn(0x7f0eac0a4210 msgr2=0x7f0eac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 -- 172.21.15.32:0/4190875896 shutdown_connections 2023-09-30T22:28:25.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.350+0000 7f0eb337e700 1 -- 172.21.15.32:0/4190875896 wait complete. 2023-09-30T22:28:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.351+0000 7f0eb337e700 1 Processor -- start 2023-09-30T22:28:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.351+0000 7f0eb337e700 1 -- start start 2023-09-30T22:28:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb337e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 0x7f0eac11e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb337e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 0x7f0eac11e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb337e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eac136fe0 0x7f0eac11eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb337e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac121f90 con 0x7f0eac136fe0 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb337e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1206a0 con 0x7f0eac09e930 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb337e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac071d20 con 0x7f0eac09f410 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb111a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 0x7f0eac11e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb0919700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 0x7f0eac11e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb111a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 0x7f0eac11e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42836/0 (socket says 172.21.15.32:42836) 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb111a700 1 -- 172.21.15.32:0/2549130751 learned_addr learned my addr 172.21.15.32:0/2549130751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb0919700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 0x7f0eac11e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43632/0 (socket says 172.21.15.32:43632) 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.352+0000 7f0eb191b700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eac136fe0 0x7f0eac11eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb111a700 1 -- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 msgr2=0x7f0eac11e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb111a700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 0x7f0eac11e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb111a700 1 -- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eac136fe0 msgr2=0x7f0eac11eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb111a700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eac136fe0 0x7f0eac11eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb111a700 1 -- 172.21.15.32:0/2549130751 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4009a70 con 0x7f0eac09e930 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb0919700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 0x7f0eac11e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb111a700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 0x7f0eac11e470 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800c550 tx=0x7f0ea800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8004c80 con 0x7f0eac09e930 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea8004e00 con 0x7f0eac09e930 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac120080 con 0x7f0eac09e930 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8007310 con 0x7f0eac09e930 2023-09-30T22:28:25.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.353+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac120870 con 0x7f0eac09e930 2023-09-30T22:28:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.354+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ea8007470 con 0x7f0eac09e930 2023-09-30T22:28:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.354+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0ea803f270 con 0x7f0eac09e930 2023-09-30T22:28:25.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.355+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac1343f0 con 0x7f0eac09e930 2023-09-30T22:28:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.364+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ea80123b0 con 0x7f0eac09e930 2023-09-30T22:28:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.410+0000 7f56ee40f700 1 Processor -- start 2023-09-30T22:28:25.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.410+0000 7f56ee40f700 1 -- start start 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ee40f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 0x7f56e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ee40f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 0x7f56e80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ee40f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e8133930 0x7f56e8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ee40f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80a2a70 con 0x7f56e809d630 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ee40f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80a01d0 con 0x7f56e8133930 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ee40f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56e809e2b0 con 0x7f56e80a0a30 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 0x7f56e80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 0x7f56e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 0x7f56e80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46136/0 (socket says 172.21.15.32:46136) 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e77fe700 1 -- 172.21.15.32:0/2670101823 learned_addr learned my addr 172.21.15.32:0/2670101823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 0x7f56e80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43640/0 (socket says 172.21.15.32:43640) 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56ec9ac700 1 --2- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e8133930 0x7f56e8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 -- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e8133930 msgr2=0x7f56e8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 --2- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e8133930 0x7f56e8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 -- 172.21.15.32:0/2670101823 >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 msgr2=0x7f56e80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 --2- 172.21.15.32:0/2670101823 >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 0x7f56e80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e7fff700 1 -- 172.21.15.32:0/2670101823 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e809dc50 con 0x7f56e80a0a30 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.411+0000 7f56e77fe700 1 --2- 172.21.15.32:0/2670101823 >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 0x7f56e80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:25.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.412+0000 7f56e7fff700 1 --2- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 0x7f56e80a0e00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009840 tx=0x7f56d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca2910d57392b9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.412+0000 7f56e6ffd700 1 -- 172.21.15.32:0/2670101823 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8011830 con 0x7f56e80a0a30 2023-09-30T22:28:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.412+0000 7f56e6ffd700 1 -- 172.21.15.32:0/2670101823 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8011990 con 0x7f56e80a0a30 2023-09-30T22:28:25.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.412+0000 7f56e6ffd700 1 -- 172.21.15.32:0/2670101823 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8011d20 con 0x7f56e80a0a30 2023-09-30T22:28:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.412+0000 7f56ee40f700 1 -- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 msgr2=0x7f56e80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.412+0000 7f56ee40f700 1 --2- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 0x7f56e80a0e00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009840 tx=0x7f56d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 -- 172.21.15.32:0/2670101823 shutdown_connections 2023-09-30T22:28:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 --2- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e8133930 0x7f56e8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 --2- 172.21.15.32:0/2670101823 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e80a0a30 0x7f56e80a0e00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 --2- 172.21.15.32:0/2670101823 >> v2:172.21.15.32:3300/0 conn(0x7f56e809d630 0x7f56e80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 -- 172.21.15.32:0/2670101823 >> 172.21.15.32:0/2670101823 conn(0x7f56e80a3050 msgr2=0x7f56e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 -- 172.21.15.32:0/2670101823 shutdown_connections 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.413+0000 7f56ee40f700 1 -- 172.21.15.32:0/2670101823 wait complete. 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.414+0000 7f56ee40f700 1 Processor -- start 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.414+0000 7f56ee40f700 1 -- start start 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ee40f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 0x7f56e81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ee40f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e80a0a30 0x7f56e81c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ee40f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e8133930 0x7f56e81c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ee40f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81207d0 con 0x7f56e8133930 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ee40f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81205e0 con 0x7f56e80a0a30 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ee40f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81224f0 con 0x7f56e809d630 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 0x7f56e81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 0x7f56e81c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43654/0 (socket says 172.21.15.32:43654) 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56e7fff700 1 -- 172.21.15.32:0/368884924 learned_addr learned my addr 172.21.15.32:0/368884924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56e77fe700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e80a0a30 0x7f56e81c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.415+0000 7f56ec9ac700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e8133930 0x7f56e81c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e7fff700 1 -- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e80a0a30 msgr2=0x7f56e81c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e7fff700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e80a0a30 0x7f56e81c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e7fff700 1 -- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e8133930 msgr2=0x7f56e81c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e7fff700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e8133930 0x7f56e81c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e7fff700 1 -- 172.21.15.32:0/368884924 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc009550 con 0x7f56e809d630 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e77fe700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e80a0a30 0x7f56e81c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56ec9ac700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e8133930 0x7f56e81c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e7fff700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 0x7f56e81c8090 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f56d8000c00 tx=0x7f56d8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8004450 con 0x7f56e809d630 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d801f070 con 0x7f56e809d630 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d80094a0 con 0x7f56e809d630 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80038e0 con 0x7f56e809d630 2023-09-30T22:28:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.416+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e8122210 con 0x7f56e809d630 2023-09-30T22:28:25.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.417+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56d8003a40 con 0x7f56e809d630 2023-09-30T22:28:25.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.417+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8120dd0 con 0x7f56e809d630 2023-09-30T22:28:25.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.417+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f56d8021070 con 0x7f56e809d630 2023-09-30T22:28:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.422+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56d8010380 con 0x7f56e809d630 2023-09-30T22:28:25.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.526+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9870074ea0 con 0x7f987009de50 2023-09-30T22:28:25.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.527+0000 7f9865ffb700 1 -- 172.21.15.32:0/2079829890 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f986c015020 con 0x7f987009de50 2023-09-30T22:28:25.528 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 msgr2=0x7f987011b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 0x7f987011b730 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f986c009c70 tx=0x7f986c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 shutdown_connections 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98701338a0 0x7f987011c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f987009de50 0x7f987011b730 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 --2- 172.21.15.32:0/2079829890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f987009e760 0x7f987011bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 >> 172.21.15.32:0/2079829890 conn(0x7f98700a2aa0 msgr2=0x7f98701306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 shutdown_connections 2023-09-30T22:28:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.529+0000 7f98777b6700 1 -- 172.21.15.32:0/2079829890 wait complete. 2023-09-30T22:28:25.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:25.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.586+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0eac074ea0 con 0x7f0eac09e930 2023-09-30T22:28:25.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.586+0000 7f0ea1ffb700 1 -- 172.21.15.32:0/2549130751 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ea8011360 con 0x7f0eac09e930 2023-09-30T22:28:25.587 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 msgr2=0x7f0eac11e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 0x7f0eac11e470 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800c550 tx=0x7f0ea800c860 comp rx=0 tx=0).stop 2023-09-30T22:28:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 shutdown_connections 2023-09-30T22:28:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0eac136fe0 0x7f0eac11eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0eac09e930 0x7f0eac11e470 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 --2- 172.21.15.32:0/2549130751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0eac09f410 0x7f0eac11e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.588+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 >> 172.21.15.32:0/2549130751 conn(0x7f0eac0a4210 msgr2=0x7f0eac0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.589+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 shutdown_connections 2023-09-30T22:28:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.589+0000 7f0eb337e700 1 -- 172.21.15.32:0/2549130751 wait complete. 2023-09-30T22:28:25.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.628+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f56e8074770 con 0x7f56e809d630 2023-09-30T22:28:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.629+0000 7f56e4ff9700 1 -- 172.21.15.32:0/368884924 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56d8024b70 con 0x7f56e809d630 2023-09-30T22:28:25.630 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 msgr2=0x7f56e81c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 0x7f56e81c8090 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f56d8000c00 tx=0x7f56d8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 shutdown_connections 2023-09-30T22:28:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56e8133930 0x7f56e81c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56e809d630 0x7f56e81c8090 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 --2- 172.21.15.32:0/368884924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56e80a0a30 0x7f56e81c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 >> 172.21.15.32:0/368884924 conn(0x7f56e80a3050 msgr2=0x7f56e809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 shutdown_connections 2023-09-30T22:28:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:25.631+0000 7f56ee40f700 1 -- 172.21.15.32:0/368884924 wait complete. 2023-09-30T22:28:25.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:26.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.362+0000 7ff28b6ff700 1 Processor -- start 2023-09-30T22:28:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.362+0000 7ff28b6ff700 1 -- start start 2023-09-30T22:28:26.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28b6ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284133420 0x7ff2841337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28b6ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28b6ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28406b990 con 0x7ff284134260 2023-09-30T22:28:26.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28b6ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840a0590 con 0x7ff284133420 2023-09-30T22:28:26.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28b6ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff28409fb80 con 0x7ff284133e30 2023-09-30T22:28:26.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff289c9c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff284134260 0x7ff2841304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48190/0 (socket says 172.21.15.32:48190) 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff289c9c700 1 -- 172.21.15.32:0/95041494 learned_addr learned my addr 172.21.15.32:0/95041494 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff288c9a700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28949b700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284133420 0x7ff2841337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff277fff700 1 -- 172.21.15.32:0/95041494 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130268642 0 0) 0x7ff28406b990 con 0x7ff284134260 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff288c9a700 1 -- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284133420 msgr2=0x7ff2841337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff288c9a700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284133420 0x7ff2841337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff288c9a700 1 -- 172.21.15.32:0/95041494 >> v1:172.21.15.32:6789/0 conn(0x7ff284134260 legacy=0x7ff2841304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff288c9a700 1 -- 172.21.15.32:0/95041494 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840a0030 con 0x7ff284133e30 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff28949b700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284133420 0x7ff2841337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff288c9a700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28412ff90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff278009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a567b428e12d840f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.363+0000 7ff277fff700 1 -- 172.21.15.32:0/95041494 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278011830 con 0x7ff284133e30 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff277fff700 1 -- 172.21.15.32:0/95041494 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278011990 con 0x7ff284133e30 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff277fff700 1 -- 172.21.15.32:0/95041494 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278011d20 con 0x7ff284133e30 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 -- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 msgr2=0x7ff28412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28412ff90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff278009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 -- 172.21.15.32:0/95041494 shutdown_connections 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284133420 0x7ff2841337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/95041494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28412ff90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 -- 172.21.15.32:0/95041494 >> 172.21.15.32:0/95041494 conn(0x7ff2840a4210 msgr2=0x7ff2840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 -- 172.21.15.32:0/95041494 shutdown_connections 2023-09-30T22:28:26.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.364+0000 7ff28b6ff700 1 -- 172.21.15.32:0/95041494 wait complete. 2023-09-30T22:28:26.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.365+0000 7ff28b6ff700 1 Processor -- start 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.365+0000 7ff28b6ff700 1 -- start start 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28b6ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 0x7ff28409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28b6ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28b6ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 0x7ff28409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28b6ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28411f050 con 0x7ff284133420 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28b6ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28409de50 con 0x7ff284134260 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28b6ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff28411fc50 con 0x7ff284133e30 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28949b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 0x7ff28409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff289c9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 0x7ff28409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28949b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 0x7ff28409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46158/0 (socket says 172.21.15.32:46158) 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43676/0 (socket says 172.21.15.32:43676) 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff289c9c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 0x7ff28409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42876/0 (socket says 172.21.15.32:42876) 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28949b700 1 -- 172.21.15.32:0/583967754 learned_addr learned my addr 172.21.15.32:0/583967754 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 -- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 msgr2=0x7ff28409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 0x7ff28409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 -- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 msgr2=0x7ff28409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 0x7ff28409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 -- 172.21.15.32:0/583967754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800094a0 con 0x7ff284133e30 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff28949b700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 0x7ff28409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff289c9c700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 0x7ff28409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.366+0000 7ff288c9a700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28409c9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff278016000 tx=0x7ff278004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.367+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278004510 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.367+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27801f070 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.367+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2780094a0 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.367+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780038e0 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.367+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28409e3e0 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.368+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff284074ea0 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.368+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff278003a40 con 0x7ff284133e30 2023-09-30T22:28:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.368+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff278021070 con 0x7ff284133e30 2023-09-30T22:28:26.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.373+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff278010d50 con 0x7ff284133e30 2023-09-30T22:28:26.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.578+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff284120100 con 0x7ff284133e30 2023-09-30T22:28:26.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.578+0000 7ff275ffb700 1 -- 172.21.15.32:0/583967754 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff278015030 con 0x7ff284133e30 2023-09-30T22:28:26.579 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:26.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 msgr2=0x7ff28409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28409c9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff278016000 tx=0x7ff278004750 comp rx=0 tx=0).stop 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 shutdown_connections 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff284134260 0x7ff28409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff284133420 0x7ff28409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 --2- 172.21.15.32:0/583967754 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff284133e30 0x7ff28409c9c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 >> 172.21.15.32:0/583967754 conn(0x7ff2840a4210 msgr2=0x7ff2840a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 shutdown_connections 2023-09-30T22:28:26.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.582+0000 7ff28b6ff700 1 -- 172.21.15.32:0/583967754 wait complete. 2023-09-30T22:28:26.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:26.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:26.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.690+0000 7f2819ce7700 1 Processor -- start 2023-09-30T22:28:26.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.690+0000 7f2819ce7700 1 -- start start 2023-09-30T22:28:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2819ce7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f2814138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2819ce7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2819ce7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281409b120 con 0x7f281409d6f0 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2819ce7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281406b460 con 0x7f281409e1d0 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2819ce7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f281409a7e0 con 0x7f2814139170 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f2814138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f2814138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42888/0 (socket says 172.21.15.32:42888) 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f28137fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f281409d6f0 0x7f281409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48204/0 (socket says 172.21.15.32:48204) 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 -- 172.21.15.32:0/540022827 learned_addr learned my addr 172.21.15.32:0/540022827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:26.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2813fff700 1 --2- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 -- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 msgr2=0x7f281413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 --2- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f28127fc700 1 -- 172.21.15.32:0/540022827 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693916745 0 0) 0x7f281409b120 con 0x7f281409d6f0 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 -- 172.21.15.32:0/540022827 >> v1:172.21.15.32:6789/0 conn(0x7f281409d6f0 legacy=0x7f281409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 -- 172.21.15.32:0/540022827 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281409a500 con 0x7f281409e1d0 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.691+0000 7f2812ffd700 1 --2- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f2814138c30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2808009920 tx=0x7f2808009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2931c1ce5e8f5ebf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f28127fc700 1 -- 172.21.15.32:0/540022827 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808011830 con 0x7f281409e1d0 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f28127fc700 1 -- 172.21.15.32:0/540022827 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808011990 con 0x7f281409e1d0 2023-09-30T22:28:26.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f28127fc700 1 -- 172.21.15.32:0/540022827 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808011d20 con 0x7f281409e1d0 2023-09-30T22:28:26.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f2819ce7700 1 -- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 msgr2=0x7f2814138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f2819ce7700 1 --2- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f2814138c30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2808009920 tx=0x7f2808009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f2819ce7700 1 -- 172.21.15.32:0/540022827 shutdown_connections 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f2819ce7700 1 --2- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.692+0000 7f2819ce7700 1 --2- 172.21.15.32:0/540022827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f2814138c30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.693+0000 7f2819ce7700 1 -- 172.21.15.32:0/540022827 >> 172.21.15.32:0/540022827 conn(0x7f2814066f50 msgr2=0x7f281409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.693+0000 7f2819ce7700 1 -- 172.21.15.32:0/540022827 shutdown_connections 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.693+0000 7f2819ce7700 1 -- 172.21.15.32:0/540022827 wait complete. 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 Processor -- start 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 -- start start 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 0x7f281411e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f281411ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281411f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28141221d0 con 0x7f281409d6f0 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28141345a0 con 0x7f281409e1d0 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f2819ce7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f281409c710 con 0x7f2814139170 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.694+0000 7f28137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 0x7f281411e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f2813fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281411f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 0x7f281411e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46166/0 (socket says 172.21.15.32:46166) 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 -- 172.21.15.32:0/1308962542 learned_addr learned my addr 172.21.15.32:0/1308962542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f2812ffd700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f281411ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 -- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 msgr2=0x7f281411f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281411f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 -- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 msgr2=0x7f281411ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f281411ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 -- 172.21.15.32:0/1308962542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808009550 con 0x7f281409d6f0 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f28137fe700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 0x7f281411e6b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f280400c520 tx=0x7f280400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804004c80 con 0x7f281409d6f0 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804004e00 con 0x7f281409d6f0 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.695+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804007310 con 0x7f281409d6f0 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.696+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2814120360 con 0x7f281409d6f0 2023-09-30T22:28:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.696+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2814122960 con 0x7f281409d6f0 2023-09-30T22:28:26.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.697+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2804007470 con 0x7f281409d6f0 2023-09-30T22:28:26.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.697+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814121e50 con 0x7f281409d6f0 2023-09-30T22:28:26.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.697+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f280403f300 con 0x7f281409d6f0 2023-09-30T22:28:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.702+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28040133c0 con 0x7f281409d6f0 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 Processor -- start 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 -- start start 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50134630 0x7eff50136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50136f50 0x7eff50139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5009b4f0 con 0x7eff50131810 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff501313c0 con 0x7eff50136f50 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.788+0000 7eff55326700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff5006e8c0 con 0x7eff50134630 2023-09-30T22:28:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7eff50131810 0x7eff50131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48218/0 (socket says 172.21.15.32:48218) 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4effd700 1 -- 172.21.15.32:0/165018037 learned_addr learned my addr 172.21.15.32:0/165018037 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4f7fe700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50136f50 0x7eff50139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4e7fc700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50134630 0x7eff50136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4dffb700 1 -- 172.21.15.32:0/165018037 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305374697 0 0) 0x7eff5009b4f0 con 0x7eff50131810 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4dffb700 1 -- 172.21.15.32:0/165018037 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff380035b0 con 0x7eff50131810 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4f7fe700 1 -- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50134630 msgr2=0x7eff50136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4f7fe700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50134630 0x7eff50136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4f7fe700 1 -- 172.21.15.32:0/165018037 >> v1:172.21.15.32:6789/0 conn(0x7eff50131810 legacy=0x7eff50131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4f7fe700 1 -- 172.21.15.32:0/165018037 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff50071d20 con 0x7eff50136f50 2023-09-30T22:28:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4e7fc700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50134630 0x7eff50136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.789+0000 7eff4f7fe700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50136f50 0x7eff50139330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7eff48009dc0 tx=0x7eff4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e52a1e64cdd981 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff4dffb700 1 -- 172.21.15.32:0/165018037 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4800e070 con 0x7eff50136f50 2023-09-30T22:28:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff4dffb700 1 -- 172.21.15.32:0/165018037 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4800bb20 con 0x7eff50136f50 2023-09-30T22:28:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff4dffb700 1 -- 172.21.15.32:0/165018037 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48012430 con 0x7eff50136f50 2023-09-30T22:28:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 -- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50136f50 msgr2=0x7eff50139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50136f50 0x7eff50139330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7eff48009dc0 tx=0x7eff4800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 -- 172.21.15.32:0/165018037 shutdown_connections 2023-09-30T22:28:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50136f50 0x7eff50139330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 --2- 172.21.15.32:0/165018037 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50134630 0x7eff50136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 -- 172.21.15.32:0/165018037 >> 172.21.15.32:0/165018037 conn(0x7eff500a2aa0 msgr2=0x7eff500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 -- 172.21.15.32:0/165018037 shutdown_connections 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.790+0000 7eff55326700 1 -- 172.21.15.32:0/165018037 wait complete. 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.792+0000 7eff55326700 1 Processor -- start 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.792+0000 7eff55326700 1 -- start start 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff55326700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff50131810 0x7eff50118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff55326700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 0x7eff50119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff55326700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50136f50 0x7eff50119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff55326700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5011c2f0 con 0x7eff50131810 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff55326700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5011b560 con 0x7eff50134630 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff55326700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff50130b00 con 0x7eff50136f50 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 0x7eff50119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 0x7eff50119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42912/0 (socket says 172.21.15.32:42912) 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 -- 172.21.15.32:0/1559373054 learned_addr learned my addr 172.21.15.32:0/1559373054 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4f7fe700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50136f50 0x7eff50119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4effd700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff50131810 0x7eff50118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 -- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50136f50 msgr2=0x7eff50119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50136f50 0x7eff50119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 -- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff50131810 msgr2=0x7eff50118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff50131810 0x7eff50118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.793+0000 7eff4e7fc700 1 -- 172.21.15.32:0/1559373054 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff440094a0 con 0x7eff50134630 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.794+0000 7eff4e7fc700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 0x7eff50119210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7eff4400ebe0 tx=0x7eff4400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.794+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400c850 con 0x7eff50134630 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.794+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4400c9b0 con 0x7eff50134630 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.794+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff48009a70 con 0x7eff50134630 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.794+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400cd40 con 0x7eff50134630 2023-09-30T22:28:26.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.794+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5006eb50 con 0x7eff50134630 2023-09-30T22:28:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.795+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff44013470 con 0x7eff50134630 2023-09-30T22:28:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.795+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7eff4403fb30 con 0x7eff50134630 2023-09-30T22:28:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.795+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5011ce20 con 0x7eff50134630 2023-09-30T22:28:26.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.804+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff44017020 con 0x7eff50134630 2023-09-30T22:28:26.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.833+0000 7fc6f8aa3700 1 Processor -- start 2023-09-30T22:28:26.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.833+0000 7fc6f8aa3700 1 -- start start 2023-09-30T22:28:26.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f8aa3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6f4133420 0x7fc6f41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f8aa3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f8aa3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f406b990 con 0x7fc6f4134260 2023-09-30T22:28:26.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f8aa3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40a0590 con 0x7fc6f4133420 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f8aa3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f409fb80 con 0x7fc6f4133e30 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc6f4134260 0x7fc6f41304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48232/0 (socket says 172.21.15.32:48232) 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f2d9d700 1 -- 172.21.15.32:0/1363640572 learned_addr learned my addr 172.21.15.32:0/1363640572 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.834+0000 7fc6f1d9b700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f259c700 1 --2- 172.21.15.32:0/1363640572 >> v2:172.21.15.32:3300/0 conn(0x7fc6f4133420 0x7fc6f41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f259c700 1 -- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 msgr2=0x7fc6f412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f259c700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f259c700 1 -- 172.21.15.32:0/1363640572 >> v1:172.21.15.32:6789/0 conn(0x7fc6f4134260 legacy=0x7fc6f41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f159a700 1 -- 172.21.15.32:0/1363640572 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413245608 0 0) 0x7fc6f406b990 con 0x7fc6f4134260 2023-09-30T22:28:26.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f259c700 1 -- 172.21.15.32:0/1363640572 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40a0030 con 0x7fc6f4133420 2023-09-30T22:28:26.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f1d9b700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:26.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f259c700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4133420 0x7fc6f41337f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009840 tx=0x7fc6e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b5029424a7144 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f159a700 1 -- 172.21.15.32:0/1363640572 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011830 con 0x7fc6f4133420 2023-09-30T22:28:26.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f159a700 1 -- 172.21.15.32:0/1363640572 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4011990 con 0x7fc6f4133420 2023-09-30T22:28:26.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.835+0000 7fc6f159a700 1 -- 172.21.15.32:0/1363640572 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011d20 con 0x7fc6f4133420 2023-09-30T22:28:26.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4133420 msgr2=0x7fc6f41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4133420 0x7fc6f41337f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009840 tx=0x7fc6e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:26.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/1363640572 shutdown_connections 2023-09-30T22:28:26.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4133420 0x7fc6f41337f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/1363640572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/1363640572 >> 172.21.15.32:0/1363640572 conn(0x7fc6f40a4210 msgr2=0x7fc6f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.836+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/1363640572 shutdown_connections 2023-09-30T22:28:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.837+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/1363640572 wait complete. 2023-09-30T22:28:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.838+0000 7fc6f8aa3700 1 Processor -- start 2023-09-30T22:28:26.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.838+0000 7fc6f8aa3700 1 -- start start 2023-09-30T22:28:26.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f8aa3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 0x7fc6f409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f8aa3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f8aa3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 0x7fc6f409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:26.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f8aa3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4131470 con 0x7fc6f4134260 2023-09-30T22:28:26.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f8aa3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f411eed0 con 0x7fc6f4133e30 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f8aa3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4120590 con 0x7fc6f4133420 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 0x7fc6f409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 0x7fc6f409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 0x7fc6f409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43732/0 (socket says 172.21.15.32:43732) 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 0x7fc6f409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46182/0 (socket says 172.21.15.32:46182) 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f259c700 1 -- 172.21.15.32:0/447162325 learned_addr learned my addr 172.21.15.32:0/447162325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f2d9d700 1 -- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 msgr2=0x7fc6f409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f2d9d700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 0x7fc6f409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f2d9d700 1 -- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 msgr2=0x7fc6f409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.839+0000 7fc6f2d9d700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6f2d9d700 1 -- 172.21.15.32:0/447162325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80094a0 con 0x7fc6f4134260 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6f259c700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 0x7fc6f409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6f2d9d700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 0x7fc6f409cf00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc6f409f3e0 tx=0x7fc6ec00cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00aa40 con 0x7fc6f4134260 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec00abc0 con 0x7fc6f4134260 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e40094a0 con 0x7fc6f4134260 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0070d0 con 0x7fc6f4134260 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.840+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f409dc90 con 0x7fc6f4134260 2023-09-30T22:28:26.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.841+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6ec007230 con 0x7fc6f4134260 2023-09-30T22:28:26.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.841+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc6ec03f390 con 0x7fc6f4134260 2023-09-30T22:28:26.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.841+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f41200f0 con 0x7fc6f4134260 2023-09-30T22:28:26.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.846+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6ec018080 con 0x7fc6f4134260 2023-09-30T22:28:26.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.911+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2814074ea0 con 0x7f281409d6f0 2023-09-30T22:28:26.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.912+0000 7f27fbfff700 1 -- 172.21.15.32:0/1308962542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2804011360 con 0x7f281409d6f0 2023-09-30T22:28:26.913 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 msgr2=0x7f281411e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 0x7f281411e6b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f280400c520 tx=0x7f280400c830 comp rx=0 tx=0).stop 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 shutdown_connections 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2814139170 0x7f281411f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f281409d6f0 0x7f281411e6b0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 --2- 172.21.15.32:0/1308962542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f281409e1d0 0x7f281411ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 >> 172.21.15.32:0/1308962542 conn(0x7f2814066f50 msgr2=0x7f2814136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 shutdown_connections 2023-09-30T22:28:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:26.914+0000 7f2819ce7700 1 -- 172.21.15.32:0/1308962542 wait complete. 2023-09-30T22:28:26.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:27.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.015+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff50074ea0 con 0x7eff50134630 2023-09-30T22:28:27.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.015+0000 7eff377fe700 1 -- 172.21.15.32:0/1559373054 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff4400cea0 con 0x7eff50134630 2023-09-30T22:28:27.016 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 msgr2=0x7eff50119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 0x7eff50119210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7eff4400ebe0 tx=0x7eff4400eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 shutdown_connections 2023-09-30T22:28:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff50136f50 0x7eff50119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff50131810 0x7eff50118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 --2- 172.21.15.32:0/1559373054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff50134630 0x7eff50119210 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.017+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 >> 172.21.15.32:0/1559373054 conn(0x7eff500a2aa0 msgr2=0x7eff5009f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.018+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 shutdown_connections 2023-09-30T22:28:27.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.018+0000 7eff55326700 1 -- 172.21.15.32:0/1559373054 wait complete. 2023-09-30T22:28:27.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:27.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.060+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc6f4074ea0 con 0x7fc6f4134260 2023-09-30T22:28:27.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.060+0000 7fc6e2ffd700 1 -- 172.21.15.32:0/447162325 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6ec0074e0 con 0x7fc6f4134260 2023-09-30T22:28:27.061 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:27.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 msgr2=0x7fc6f409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 0x7fc6f409cf00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc6f409f3e0 tx=0x7fc6ec00cec0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 shutdown_connections 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6f4134260 0x7fc6f409cf00 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6f4133420 0x7fc6f409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 --2- 172.21.15.32:0/447162325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6f4133e30 0x7fc6f409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 >> 172.21.15.32:0/447162325 conn(0x7fc6f40a4210 msgr2=0x7fc6f41303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 shutdown_connections 2023-09-30T22:28:27.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.062+0000 7fc6f8aa3700 1 -- 172.21.15.32:0/447162325 wait complete. 2023-09-30T22:28:27.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:27.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 Processor -- start 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 -- start start 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf581338a0 0x7fbf58133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 0x7fbf5809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 0x7fbf5813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5806fef0 con 0x7fbf581338a0 2023-09-30T22:28:27.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5806ef30 con 0x7fbf5809e760 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5e596700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581334b0 con 0x7fbf5809de50 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 0x7fbf5809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf5cb33700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 0x7fbf5813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.749+0000 7fbf57fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf581338a0 0x7fbf58133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 0x7fbf5809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43734/0 (socket says 172.21.15.32:43734) 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf5cb33700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 0x7fbf5813a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42936/0 (socket says 172.21.15.32:42936) 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf577fe700 1 -- 172.21.15.32:0/1462108676 learned_addr learned my addr 172.21.15.32:0/1462108676 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf57fff700 1 -- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 msgr2=0x7fbf5809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf57fff700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 0x7fbf5809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf57fff700 1 -- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 msgr2=0x7fbf5813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf57fff700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 0x7fbf5813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf57fff700 1 -- 172.21.15.32:0/1462108676 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5806fc30 con 0x7fbf581338a0 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf577fe700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 0x7fbf5809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf5cb33700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 0x7fbf5813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf57fff700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf58133c70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009840 tx=0x7fbf48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad93f15ee16e8bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf56ffd700 1 -- 172.21.15.32:0/1462108676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48011830 con 0x7fbf581338a0 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf56ffd700 1 -- 172.21.15.32:0/1462108676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf48011990 con 0x7fbf581338a0 2023-09-30T22:28:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.750+0000 7fbf56ffd700 1 -- 172.21.15.32:0/1462108676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48011d20 con 0x7fbf581338a0 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 -- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 msgr2=0x7fbf58133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf58133c70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009840 tx=0x7fbf48009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 -- 172.21.15.32:0/1462108676 shutdown_connections 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809e760 0x7fbf5813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf58133c70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 --2- 172.21.15.32:0/1462108676 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809de50 0x7fbf5809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 -- 172.21.15.32:0/1462108676 >> 172.21.15.32:0/1462108676 conn(0x7fbf580a2aa0 msgr2=0x7fbf580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 -- 172.21.15.32:0/1462108676 shutdown_connections 2023-09-30T22:28:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.751+0000 7fbf5e596700 1 -- 172.21.15.32:0/1462108676 wait complete. 2023-09-30T22:28:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.752+0000 7fbf5e596700 1 Processor -- start 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.752+0000 7fbf5e596700 1 -- start start 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5e596700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809de50 0x7fbf5811d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5e596700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809e760 0x7fbf5811dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5e596700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf5811e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5e596700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5811f200 con 0x7fbf581338a0 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5e596700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58120450 con 0x7fbf5809de50 2023-09-30T22:28:27.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5e596700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5811f7e0 con 0x7fbf5809e760 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf5811e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf57fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809de50 0x7fbf5811d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809e760 0x7fbf5811dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf5811e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46202/0 (socket says 172.21.15.32:46202) 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf57fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809de50 0x7fbf5811d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42942/0 (socket says 172.21.15.32:42942) 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 -- 172.21.15.32:0/3243951643 learned_addr learned my addr 172.21.15.32:0/3243951643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 -- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809e760 msgr2=0x7fbf5811dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:27.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809e760 0x7fbf5811dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 -- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809de50 msgr2=0x7fbf5811d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809de50 0x7fbf5811d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 -- 172.21.15.32:0/3243951643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c009550 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf5cb33700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf5811e100 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009f00 tx=0x7fbf5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.753+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50009d70 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5000f040 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf50009a70 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000cd70 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf480094a0 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf50014480 con 0x7fbf581338a0 2023-09-30T22:28:27.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbf5003fc40 con 0x7fbf581338a0 2023-09-30T22:28:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.754+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5811eb00 con 0x7fbf581338a0 2023-09-30T22:28:27.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.759+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf50012930 con 0x7fbf581338a0 2023-09-30T22:28:27.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:27.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.962+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbf580a0f20 con 0x7fbf581338a0 2023-09-30T22:28:27.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.963+0000 7fbf54ff9700 1 -- 172.21.15.32:0/3243951643 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf5001c3b0 con 0x7fbf581338a0 2023-09-30T22:28:27.964 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 msgr2=0x7fbf5811e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:27.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf5811e100 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009f00 tx=0x7fbf5000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 shutdown_connections 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf581338a0 0x7fbf5811e100 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf5809de50 0x7fbf5811d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 --2- 172.21.15.32:0/3243951643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf5809e760 0x7fbf5811dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 >> 172.21.15.32:0/3243951643 conn(0x7fbf580a2aa0 msgr2=0x7fbf581b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 shutdown_connections 2023-09-30T22:28:27.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:27.965+0000 7fbf5e596700 1 -- 172.21.15.32:0/3243951643 wait complete. 2023-09-30T22:28:27.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:28.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.075+0000 7ff318539700 1 Processor -- start 2023-09-30T22:28:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.075+0000 7ff318539700 1 -- start start 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff318539700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff31009d630 0x7ff3100a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff318539700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff310138d80 0x7ff31013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff318539700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3100a2a70 con 0x7ff3100a0a30 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff318539700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3100a01d0 con 0x7ff310138d80 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff318539700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff31009e2b0 con 0x7ff31009d630 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff316ad6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff310138d80 0x7ff31013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff315ad4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff31009d630 0x7ff3100a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff3162d5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3100a0a30 0x7ff3100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48244/0 (socket says 172.21.15.32:48244) 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.076+0000 7ff3162d5700 1 -- 172.21.15.32:0/3150380796 learned_addr learned my addr 172.21.15.32:0/3150380796 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff315ad4700 1 -- 172.21.15.32:0/3150380796 >> v2:172.21.15.32:3300/0 conn(0x7ff310138d80 msgr2=0x7ff31013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff315ad4700 1 --2- 172.21.15.32:0/3150380796 >> v2:172.21.15.32:3300/0 conn(0x7ff310138d80 0x7ff31013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff315ad4700 1 -- 172.21.15.32:0/3150380796 >> v1:172.21.15.32:6789/0 conn(0x7ff3100a0a30 legacy=0x7ff3100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff315ad4700 1 -- 172.21.15.32:0/3150380796 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31009dc50 con 0x7ff31009d630 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff3152d3700 1 -- 172.21.15.32:0/3150380796 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866234051 0 0) 0x7ff3100a2a70 con 0x7ff3100a0a30 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff316ad6700 1 --2- 172.21.15.32:0/3150380796 >> v2:172.21.15.32:3300/0 conn(0x7ff310138d80 0x7ff31013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff315ad4700 1 --2- 172.21.15.32:0/3150380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff31009d630 0x7ff3100a1510 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff300009840 tx=0x7ff300009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be2d69f2b65ac982 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff3152d3700 1 -- 172.21.15.32:0/3150380796 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300011830 con 0x7ff31009d630 2023-09-30T22:28:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff3152d3700 1 -- 172.21.15.32:0/3150380796 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff300011990 con 0x7ff31009d630 2023-09-30T22:28:28.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.077+0000 7ff3152d3700 1 -- 172.21.15.32:0/3150380796 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300011d20 con 0x7ff31009d630 2023-09-30T22:28:28.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 -- 172.21.15.32:0/3150380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff31009d630 msgr2=0x7ff3100a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 --2- 172.21.15.32:0/3150380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff31009d630 0x7ff3100a1510 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff300009840 tx=0x7ff300009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 -- 172.21.15.32:0/3150380796 shutdown_connections 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 --2- 172.21.15.32:0/3150380796 >> v2:172.21.15.32:3300/0 conn(0x7ff310138d80 0x7ff31013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 --2- 172.21.15.32:0/3150380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff31009d630 0x7ff3100a1510 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 -- 172.21.15.32:0/3150380796 >> 172.21.15.32:0/3150380796 conn(0x7ff3100a3050 msgr2=0x7ff3100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 -- 172.21.15.32:0/3150380796 shutdown_connections 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.078+0000 7ff318539700 1 -- 172.21.15.32:0/3150380796 wait complete. 2023-09-30T22:28:28.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 Processor -- start 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 -- start start 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31009d630 0x7ff3101337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3100a0a30 0x7ff310133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 0x7ff310134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3101365f0 con 0x7ff31009d630 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff310136f60 con 0x7ff3100a0a30 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.079+0000 7ff318539700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff310136240 con 0x7ff310138d80 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff316ad6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 0x7ff310134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3100a0a30 0x7ff310133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff316ad6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 0x7ff310134230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43760/0 (socket says 172.21.15.32:43760) 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff316ad6700 1 -- 172.21.15.32:0/807891852 learned_addr learned my addr 172.21.15.32:0/807891852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:28.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff3162d5700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31009d630 0x7ff3101337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 -- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 msgr2=0x7ff310134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 0x7ff310134230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 -- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31009d630 msgr2=0x7ff3101337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31009d630 0x7ff3101337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 -- 172.21.15.32:0/807891852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080094a0 con 0x7ff3100a0a30 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff316ad6700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 0x7ff310134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff315ad4700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3100a0a30 0x7ff310133cf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff300016000 tx=0x7ff300004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300004480 con 0x7ff3100a0a30 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30001f070 con 0x7ff3100a0a30 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.080+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3000094a0 con 0x7ff3100a0a30 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.081+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3000038a0 con 0x7ff3100a0a30 2023-09-30T22:28:28.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.081+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3100a0410 con 0x7ff3100a0a30 2023-09-30T22:28:28.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.081+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff300021030 con 0x7ff3100a0a30 2023-09-30T22:28:28.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.082+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff310137e80 con 0x7ff3100a0a30 2023-09-30T22:28:28.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.082+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff300040850 con 0x7ff3100a0a30 2023-09-30T22:28:28.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.091+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff300003a00 con 0x7ff3100a0a30 2023-09-30T22:28:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.181+0000 7fb01fe5c700 1 Processor -- start 2023-09-30T22:28:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.181+0000 7fb01fe5c700 1 -- start start 2023-09-30T22:28:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.182+0000 7fb01fe5c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0180a0a30 0x7fb0180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.182+0000 7fb01fe5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb018133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.182+0000 7fb01fe5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb018133fa0 0x7fb018134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.182+0000 7fb01fe5c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01809edd0 con 0x7fb0180a0a30 2023-09-30T22:28:28.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.182+0000 7fb01fe5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01809e1f0 con 0x7fb0180a1340 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.182+0000 7fb01fe5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb01809dc50 con 0x7fb018133fa0 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0180a0a30 0x7fb0180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0180a0a30 0x7fb0180a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46226/0 (socket says 172.21.15.32:46226) 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 -- 172.21.15.32:0/2228195553 learned_addr learned my addr 172.21.15.32:0/2228195553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01d3f7700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb018133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01e3f9700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb018133fa0 0x7fb018134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 -- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb018133fa0 msgr2=0x7fb018134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb018133fa0 0x7fb018134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 -- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 msgr2=0x7fb018133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb018133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 -- 172.21.15.32:0/2228195553 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180a1fe0 con 0x7fb0180a0a30 2023-09-30T22:28:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01d3f7700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb018133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01dbf8700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0180a0a30 0x7fb0180a0e00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb0140098f0 tx=0x7fb014009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803848115d27d6fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01cbf6700 1 -- 172.21.15.32:0/2228195553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014011830 con 0x7fb0180a0a30 2023-09-30T22:28:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01cbf6700 1 -- 172.21.15.32:0/2228195553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014011990 con 0x7fb0180a0a30 2023-09-30T22:28:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.183+0000 7fb01cbf6700 1 -- 172.21.15.32:0/2228195553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014011d20 con 0x7fb0180a0a30 2023-09-30T22:28:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 -- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0180a0a30 msgr2=0x7fb0180a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0180a0a30 0x7fb0180a0e00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb0140098f0 tx=0x7fb014009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 -- 172.21.15.32:0/2228195553 shutdown_connections 2023-09-30T22:28:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb018133fa0 0x7fb018134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0180a0a30 0x7fb0180a0e00 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/2228195553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb018133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 -- 172.21.15.32:0/2228195553 >> 172.21.15.32:0/2228195553 conn(0x7fb0180a6550 msgr2=0x7fb0180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 -- 172.21.15.32:0/2228195553 shutdown_connections 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.184+0000 7fb01fe5c700 1 -- 172.21.15.32:0/2228195553 wait complete. 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.185+0000 7fb01fe5c700 1 Processor -- start 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 -- start start 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 0x7fb018120380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb0181208c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb018133fa0 0x7fb018120e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018133770 con 0x7fb018133fa0 2023-09-30T22:28:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0181231e0 con 0x7fb0180a1340 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01fe5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb018122570 con 0x7fb0180a0a30 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01dbf8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 0x7fb018120380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01d3f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb0181208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01dbf8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 0x7fb018120380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43774/0 (socket says 172.21.15.32:43774) 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01dbf8700 1 -- 172.21.15.32:0/3111420300 learned_addr learned my addr 172.21.15.32:0/3111420300 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.186+0000 7fb01e3f9700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb018133fa0 0x7fb018120e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01d3f7700 1 -- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 msgr2=0x7fb018120380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01d3f7700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 0x7fb018120380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01d3f7700 1 -- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb018133fa0 msgr2=0x7fb018120e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01d3f7700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb018133fa0 0x7fb018120e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01d3f7700 1 -- 172.21.15.32:0/3111420300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080094a0 con 0x7fb0180a1340 2023-09-30T22:28:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01dbf8700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 0x7fb018120380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01d3f7700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb0181208c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb00800e9e0 tx=0x7fb00800ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800ee80 con 0x7fb0180a1340 2023-09-30T22:28:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00800c850 con 0x7fb0180a1340 2023-09-30T22:28:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb014009550 con 0x7fb0180a1340 2023-09-30T22:28:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00800cc00 con 0x7fb0180a1340 2023-09-30T22:28:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.187+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0180a3410 con 0x7fb0180a1340 2023-09-30T22:28:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.188+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0180a0280 con 0x7fb0180a1340 2023-09-30T22:28:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.189+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb00800cd60 con 0x7fb0180a1340 2023-09-30T22:28:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.189+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb00803ffe0 con 0x7fb0180a1340 2023-09-30T22:28:28.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.198+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb008011350 con 0x7fb0180a1340 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 Processor -- start 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 -- start start 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c0a0a30 0x7f811c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f811c138d80 0x7f811c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c0a2a70 con 0x7f811c09d630 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c0a01d0 con 0x7f811c138d80 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.225+0000 7f81208cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c09e2b0 con 0x7f811c0a0a30 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c0a0a30 0x7f811c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f8113fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f811c09d630 0x7f811c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48250/0 (socket says 172.21.15.32:48250) 2023-09-30T22:28:28.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c0a0a30 0x7f811c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42980/0 (socket says 172.21.15.32:42980) 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f8113fff700 1 -- 172.21.15.32:0/2886143373 learned_addr learned my addr 172.21.15.32:0/2886143373 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811ad9d700 1 --2- 172.21.15.32:0/2886143373 >> v2:172.21.15.32:3300/0 conn(0x7f811c138d80 0x7f811c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811ad9d700 1 -- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c0a0a30 msgr2=0x7f811c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811ad9d700 1 --2- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c0a0a30 0x7f811c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811ad9d700 1 -- 172.21.15.32:0/2886143373 >> v1:172.21.15.32:6789/0 conn(0x7f811c09d630 legacy=0x7f811c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811ad9d700 1 -- 172.21.15.32:0/2886143373 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c09dc50 con 0x7f811c138d80 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f8119d9b700 1 -- 172.21.15.32:0/2886143373 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270276968 0 0) 0x7f811c0a2a70 con 0x7f811c09d630 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.226+0000 7f811ad9d700 1 --2- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c138d80 0x7f811c13b160 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8114009d90 tx=0x7f811400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1224bb060c0627 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.227+0000 7f8119d9b700 1 -- 172.21.15.32:0/2886143373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400e070 con 0x7f811c138d80 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.227+0000 7f8119d9b700 1 -- 172.21.15.32:0/2886143373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811400bb20 con 0x7f811c138d80 2023-09-30T22:28:28.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.227+0000 7f8119d9b700 1 -- 172.21.15.32:0/2886143373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114012430 con 0x7f811c138d80 2023-09-30T22:28:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.227+0000 7f81208cb700 1 -- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c138d80 msgr2=0x7f811c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.227+0000 7f81208cb700 1 --2- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c138d80 0x7f811c13b160 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8114009d90 tx=0x7f811400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:28.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.228+0000 7f81208cb700 1 -- 172.21.15.32:0/2886143373 shutdown_connections 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.228+0000 7f81208cb700 1 --2- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c138d80 0x7f811c13b160 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.228+0000 7f81208cb700 1 --2- 172.21.15.32:0/2886143373 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c0a0a30 0x7f811c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.228+0000 7f81208cb700 1 -- 172.21.15.32:0/2886143373 >> 172.21.15.32:0/2886143373 conn(0x7f811c0a3050 msgr2=0x7f811c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.228+0000 7f81208cb700 1 -- 172.21.15.32:0/2886143373 shutdown_connections 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.228+0000 7f81208cb700 1 -- 172.21.15.32:0/2886143373 wait complete. 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.229+0000 7f81208cb700 1 Processor -- start 2023-09-30T22:28:28.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.229+0000 7f81208cb700 1 -- start start 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f81208cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 0x7f811c1336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f81208cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f811c0a0a30 0x7f811c133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f81208cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c138d80 0x7f811c134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f81208cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c136130 con 0x7f811c09d630 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f81208cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c136e50 con 0x7f811c138d80 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f81208cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0a4b90 con 0x7f811c0a0a30 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f811a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 0x7f811c1336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f8113fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f811c0a0a30 0x7f811c133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f811a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 0x7f811c1336f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46240/0 (socket says 172.21.15.32:46240) 2023-09-30T22:28:28.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f811a59c700 1 -- 172.21.15.32:0/2077664044 learned_addr learned my addr 172.21.15.32:0/2077664044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f8113fff700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f811c0a0a30 0x7f811c133c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43788/0 (socket says 172.21.15.32:43788) 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.230+0000 7f811ad9d700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c138d80 0x7f811c134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f811a59c700 1 -- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f811c0a0a30 msgr2=0x7f811c133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f811a59c700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f811c0a0a30 0x7f811c133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f811a59c700 1 -- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c138d80 msgr2=0x7f811c134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f811a59c700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c138d80 0x7f811c134170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f811a59c700 1 -- 172.21.15.32:0/2077664044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114009a70 con 0x7f811c09d630 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f811a59c700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 0x7f811c1336f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9b0 tx=0x7f810c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c009710 con 0x7f811c09d630 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c00ee50 con 0x7f811c09d630 2023-09-30T22:28:28.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c0a0410 con 0x7f811c09d630 2023-09-30T22:28:28.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00ca50 con 0x7f811c09d630 2023-09-30T22:28:28.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.231+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c1378c0 con 0x7f811c09d630 2023-09-30T22:28:28.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.232+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f810c00cbb0 con 0x7f811c09d630 2023-09-30T22:28:28.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.232+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f810c03fba0 con 0x7f811c09d630 2023-09-30T22:28:28.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.233+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c136f90 con 0x7f811c09d630 2023-09-30T22:28:28.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.238+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f811c136f90 con 0x7f811c09d630 2023-09-30T22:28:28.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.312+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff3100747f0 con 0x7ff3100a0a30 2023-09-30T22:28:28.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.313+0000 7ff306ffd700 1 -- 172.21.15.32:0/807891852 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff300015020 con 0x7ff3100a0a30 2023-09-30T22:28:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3100a0a30 msgr2=0x7ff310133cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3100a0a30 0x7ff310133cf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff300016000 tx=0x7ff300004750 comp rx=0 tx=0).stop 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 shutdown_connections 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff310138d80 0x7ff310134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff31009d630 0x7ff3101337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 --2- 172.21.15.32:0/807891852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3100a0a30 0x7ff310133cf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 >> 172.21.15.32:0/807891852 conn(0x7ff3100a3050 msgr2=0x7ff3100a6940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 shutdown_connections 2023-09-30T22:28:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.315+0000 7ff318539700 1 -- 172.21.15.32:0/807891852 wait complete. 2023-09-30T22:28:28.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:28.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.406+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb018074ea0 con 0x7fb0180a1340 2023-09-30T22:28:28.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.406+0000 7fb00e7fc700 1 -- 172.21.15.32:0/3111420300 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb008017020 con 0x7fb0180a1340 2023-09-30T22:28:28.407 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:28.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 msgr2=0x7fb0181208c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb0181208c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb00800e9e0 tx=0x7fb00800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 shutdown_connections 2023-09-30T22:28:28.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb018133fa0 0x7fb018120e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0180a0a30 0x7fb018120380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 --2- 172.21.15.32:0/3111420300 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0180a1340 0x7fb0181208c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 >> 172.21.15.32:0/3111420300 conn(0x7fb0180a6550 msgr2=0x7fb0181baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 shutdown_connections 2023-09-30T22:28:28.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.408+0000 7fb01fe5c700 1 -- 172.21.15.32:0/3111420300 wait complete. 2023-09-30T22:28:28.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:28.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.445+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f811c136f90 con 0x7f811c09d630 2023-09-30T22:28:28.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.445+0000 7f8112ffd700 1 -- 172.21.15.32:0/2077664044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f811c136f90 con 0x7f811c09d630 2023-09-30T22:28:28.447 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:28.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 msgr2=0x7f811c1336f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:28.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 0x7f811c1336f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9b0 tx=0x7f810c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 shutdown_connections 2023-09-30T22:28:28.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f811c138d80 0x7f811c134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f811c09d630 0x7f811c1336f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 --2- 172.21.15.32:0/2077664044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f811c0a0a30 0x7f811c133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:28.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 >> 172.21.15.32:0/2077664044 conn(0x7f811c0a3050 msgr2=0x7f811c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:28.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 shutdown_connections 2023-09-30T22:28:28.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:28.448+0000 7f81208cb700 1 -- 172.21.15.32:0/2077664044 wait complete. 2023-09-30T22:28:28.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:28.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:29.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.127+0000 7fdf7065d700 1 Processor -- start 2023-09-30T22:28:29.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.127+0000 7fdf7065d700 1 -- start start 2023-09-30T22:28:29.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf7065d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf7065d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf7065d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf7065d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6809edd0 con 0x7fdf680a1340 2023-09-30T22:28:29.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf7065d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6809e1f0 con 0x7fdf68133fa0 2023-09-30T22:28:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf7065d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6809dc50 con 0x7fdf680a0a30 2023-09-30T22:28:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf6e3f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf6dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf6ebfa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf6e3f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43800/0 (socket says 172.21.15.32:43800) 2023-09-30T22:28:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf6dbf8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46254/0 (socket says 172.21.15.32:46254) 2023-09-30T22:28:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.128+0000 7fdf6e3f9700 1 -- 172.21.15.32:0/2281730283 learned_addr learned my addr 172.21.15.32:0/2281730283 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 msgr2=0x7fdf680a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6dbf8700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 msgr2=0x7fdf68134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6dbf8700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/2281730283 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf680a1fe0 con 0x7fdf680a1340 2023-09-30T22:28:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6e3f9700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6ebfa700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6dbf8700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf68133a60 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009840 tx=0x7fdf64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c50586facb1cdd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6d3f7700 1 -- 172.21.15.32:0/2281730283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64011830 con 0x7fdf680a1340 2023-09-30T22:28:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6d3f7700 1 -- 172.21.15.32:0/2281730283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64011990 con 0x7fdf680a1340 2023-09-30T22:28:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.129+0000 7fdf6d3f7700 1 -- 172.21.15.32:0/2281730283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64011d20 con 0x7fdf680a1340 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 -- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 msgr2=0x7fdf68133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf68133a60 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009840 tx=0x7fdf64009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 -- 172.21.15.32:0/2281730283 shutdown_connections 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf680a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 --2- 172.21.15.32:0/2281730283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf68133a60 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 -- 172.21.15.32:0/2281730283 >> 172.21.15.32:0/2281730283 conn(0x7fdf680a6550 msgr2=0x7fdf680a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 -- 172.21.15.32:0/2281730283 shutdown_connections 2023-09-30T22:28:29.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.130+0000 7fdf7065d700 1 -- 172.21.15.32:0/2281730283 wait complete. 2023-09-30T22:28:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 Processor -- start 2023-09-30T22:28:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 -- start start 2023-09-30T22:28:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf681200b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf681205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681238b0 con 0x7fdf680a1340 2023-09-30T22:28:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680a3630 con 0x7fdf68133fa0 2023-09-30T22:28:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf7065d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680a0220 con 0x7fdf680a0a30 2023-09-30T22:28:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf6dbf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf681205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf6dbf8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf681205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46256/0 (socket says 172.21.15.32:46256) 2023-09-30T22:28:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/771251771 learned_addr learned my addr 172.21.15.32:0/771251771 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf6e3f9700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf681200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.131+0000 7fdf6ebfa700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 msgr2=0x7fdf681200b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf6dbf8700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf681200b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 msgr2=0x7fdf68120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf6dbf8700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf6dbf8700 1 -- 172.21.15.32:0/771251771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c009550 con 0x7fdf680a1340 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf6dbf8700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf681205f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009fd0 tx=0x7fdf64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64004450 con 0x7fdf680a1340 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6401f070 con 0x7fdf680a1340 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf60009a70 con 0x7fdf680a1340 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640038e0 con 0x7fdf680a1340 2023-09-30T22:28:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.132+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640094a0 con 0x7fdf680a1340 2023-09-30T22:28:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.133+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf64003a40 con 0x7fdf680a1340 2023-09-30T22:28:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.133+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf64021070 con 0x7fdf680a1340 2023-09-30T22:28:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.133+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf68122f30 con 0x7fdf680a1340 2023-09-30T22:28:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.138+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf64015080 con 0x7fdf680a1340 2023-09-30T22:28:29.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:29.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.345+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdf68074ea0 con 0x7fdf680a1340 2023-09-30T22:28:29.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.345+0000 7fdf5affd700 1 -- 172.21.15.32:0/771251771 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf6402b020 con 0x7fdf680a1340 2023-09-30T22:28:29.346 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 msgr2=0x7fdf681205f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf681205f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009fd0 tx=0x7fdf64004750 comp rx=0 tx=0).stop 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 shutdown_connections 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf68133fa0 0x7fdf68120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf680a0a30 0x7fdf681200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 --2- 172.21.15.32:0/771251771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf680a1340 0x7fdf681205f0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 >> 172.21.15.32:0/771251771 conn(0x7fdf680a6550 msgr2=0x7fdf680a4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 shutdown_connections 2023-09-30T22:28:29.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.347+0000 7fdf7065d700 1 -- 172.21.15.32:0/771251771 wait complete. 2023-09-30T22:28:29.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:29.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:29.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.470+0000 7f20f62da700 1 Processor -- start 2023-09-30T22:28:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.470+0000 7f20f62da700 1 -- start start 2023-09-30T22:28:29.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20f62da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 0x7f20f009ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20f62da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20f009f0f0 0x7f20f0137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20f62da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f006bd70 con 0x7f20f0137ff0 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20f62da700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f006cc30 con 0x7f20f009f0f0 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20f62da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f006d870 con 0x7f20f009e7e0 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20effff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 0x7f20f009ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20ef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20f009f0f0 0x7f20f0137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20effff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 0x7f20f009ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43030/0 (socket says 172.21.15.32:43030) 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20effff700 1 -- 172.21.15.32:0/69763308 learned_addr learned my addr 172.21.15.32:0/69763308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20f4877700 1 --1- 172.21.15.32:0/69763308 >> v1:172.21.15.32:6789/0 conn(0x7f20f0137ff0 0x7f20f013a3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48262/0 (socket says 172.21.15.32:48262) 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20ef7fe700 1 -- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 msgr2=0x7f20f009ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20ef7fe700 1 --2- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 0x7f20f009ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20ef7fe700 1 -- 172.21.15.32:0/69763308 >> v1:172.21.15.32:6789/0 conn(0x7f20f0137ff0 legacy=0x7f20f013a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20ef7fe700 1 -- 172.21.15.32:0/69763308 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f006bab0 con 0x7f20f009f0f0 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20effff700 1 --2- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 0x7f20f009ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.471+0000 7f20ef7fe700 1 --2- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0137ab0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009840 tx=0x7f20e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b282bd06809bfdf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.472+0000 7f20eeffd700 1 -- 172.21.15.32:0/69763308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4011830 con 0x7f20f009f0f0 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.472+0000 7f20eeffd700 1 -- 172.21.15.32:0/69763308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e4011990 con 0x7f20f009f0f0 2023-09-30T22:28:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.472+0000 7f20eeffd700 1 -- 172.21.15.32:0/69763308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4011d20 con 0x7f20f009f0f0 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 -- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 msgr2=0x7f20f0137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 --2- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0137ab0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009840 tx=0x7f20e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 -- 172.21.15.32:0/69763308 shutdown_connections 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 --2- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f009e7e0 0x7f20f009ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 --2- 172.21.15.32:0/69763308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0137ab0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 -- 172.21.15.32:0/69763308 >> 172.21.15.32:0/69763308 conn(0x7f20f00a4300 msgr2=0x7f20f00a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 -- 172.21.15.32:0/69763308 shutdown_connections 2023-09-30T22:28:29.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.473+0000 7f20f62da700 1 -- 172.21.15.32:0/69763308 wait complete. 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.474+0000 7f20f62da700 1 Processor -- start 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.474+0000 7f20f62da700 1 -- start start 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f62da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20f009e7e0 0x7f20f0121710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f62da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0121c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f62da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f0137ff0 0x7f20f01c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f62da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0124680 con 0x7f20f009f0f0 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f62da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0121320 con 0x7f20f0137ff0 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f62da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00a05d0 con 0x7f20f009e7e0 2023-09-30T22:28:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0121c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46270/0 (socket says 172.21.15.32:46270) 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20ef7fe700 1 -- 172.21.15.32:0/87412751 learned_addr learned my addr 172.21.15.32:0/87412751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20effff700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20f009e7e0 0x7f20f0121710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20f4877700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f0137ff0 0x7f20f01c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20ef7fe700 1 -- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20f009e7e0 msgr2=0x7f20f0121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20ef7fe700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20f009e7e0 0x7f20f0121710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.475+0000 7f20ef7fe700 1 -- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f0137ff0 msgr2=0x7f20f01c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20ef7fe700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f0137ff0 0x7f20f01c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20ef7fe700 1 -- 172.21.15.32:0/87412751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009550 con 0x7f20f009f0f0 2023-09-30T22:28:29.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20effff700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20f009e7e0 0x7f20f0121710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20ef7fe700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0121c50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f20e40090b0 tx=0x7f20e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4004400 con 0x7f20f009f0f0 2023-09-30T22:28:29.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e40094a0 con 0x7f20f009f0f0 2023-09-30T22:28:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e401f070 con 0x7f20f009f0f0 2023-09-30T22:28:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e40038e0 con 0x7f20f009f0f0 2023-09-30T22:28:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.476+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f0123af0 con 0x7f20f009f0f0 2023-09-30T22:28:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.477+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20e4021030 con 0x7f20f009f0f0 2023-09-30T22:28:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.477+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f20e40407d0 con 0x7f20f009f0f0 2023-09-30T22:28:29.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.478+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f01247c0 con 0x7f20f009f0f0 2023-09-30T22:28:29.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.487+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20f01247c0 con 0x7f20f009f0f0 2023-09-30T22:28:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.578+0000 7f7b7522b700 1 Processor -- start 2023-09-30T22:28:29.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.578+0000 7f7b7522b700 1 -- start start 2023-09-30T22:28:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b7522b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 0x7f7b700a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b7522b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b70138d80 0x7f7b7013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b7522b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b700a2a70 con 0x7f7b700a0a30 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b7522b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b700a01d0 con 0x7f7b70138d80 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b7522b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7009e2b0 con 0x7f7b7009d630 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b6e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 0x7f7b700a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b6f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b70138d80 0x7f7b7013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b6f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b70138d80 0x7f7b7013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46280/0 (socket says 172.21.15.32:46280) 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b6e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 0x7f7b700a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43050/0 (socket says 172.21.15.32:43050) 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.579+0000 7f7b6f59e700 1 -- 172.21.15.32:0/203395474 learned_addr learned my addr 172.21.15.32:0/203395474 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6f59e700 1 -- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 msgr2=0x7f7b700a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6f59e700 1 --2- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 0x7f7b700a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6f59e700 1 -- 172.21.15.32:0/203395474 >> v1:172.21.15.32:6789/0 conn(0x7f7b700a0a30 legacy=0x7f7b700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6f59e700 1 -- 172.21.15.32:0/203395474 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7009dc50 con 0x7f7b70138d80 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6e59c700 1 --2- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 0x7f7b700a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6f59e700 1 --2- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b70138d80 0x7f7b7013b160 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009e50 tx=0x7f7b6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb86f408e4251a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6dd9b700 1 -- 172.21.15.32:0/203395474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800e070 con 0x7f7b70138d80 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6dd9b700 1 -- 172.21.15.32:0/203395474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6800b910 con 0x7f7b70138d80 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.580+0000 7f7b6dd9b700 1 -- 172.21.15.32:0/203395474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800bcc0 con 0x7f7b70138d80 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 -- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b70138d80 msgr2=0x7f7b7013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 --2- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b70138d80 0x7f7b7013b160 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009e50 tx=0x7f7b6800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:29.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 -- 172.21.15.32:0/203395474 shutdown_connections 2023-09-30T22:28:29.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 --2- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b70138d80 0x7f7b7013b160 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 --2- 172.21.15.32:0/203395474 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b7009d630 0x7f7b700a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 -- 172.21.15.32:0/203395474 >> 172.21.15.32:0/203395474 conn(0x7f7b700a3050 msgr2=0x7f7b700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 -- 172.21.15.32:0/203395474 shutdown_connections 2023-09-30T22:28:29.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.581+0000 7f7b7522b700 1 -- 172.21.15.32:0/203395474 wait complete. 2023-09-30T22:28:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.582+0000 7f7b7522b700 1 Processor -- start 2023-09-30T22:28:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.582+0000 7f7b7522b700 1 -- start start 2023-09-30T22:28:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b7522b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 0x7f7b70133850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b7522b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b700a0a30 0x7f7b70133d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b7522b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b70138d80 0x7f7b701342d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b7522b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70137000 con 0x7f7b7009d630 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b7522b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b701359f0 con 0x7f7b700a0a30 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b7522b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700a0480 con 0x7f7b70138d80 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 0x7f7b70133850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 0x7f7b70133850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46296/0 (socket says 172.21.15.32:46296) 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 -- 172.21.15.32:0/3955476591 learned_addr learned my addr 172.21.15.32:0/3955476591 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6f59e700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b70138d80 0x7f7b701342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6e59c700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b700a0a30 0x7f7b70133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 -- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b70138d80 msgr2=0x7f7b701342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b70138d80 0x7f7b701342d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 -- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b700a0a30 msgr2=0x7f7b70133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b700a0a30 0x7f7b70133d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.583+0000 7f7b6ed9d700 1 -- 172.21.15.32:0/3955476591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640094a0 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.584+0000 7f7b6ed9d700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 0x7f7b70133850 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c510 tx=0x7f7b6000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.584+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b600052d0 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.584+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b60005430 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.584+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b600038c0 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.584+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b68009b30 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.584+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b70137ec0 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.585+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b60003a20 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.585+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7b6003ea00 con 0x7f7b7009d630 2023-09-30T22:28:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.585+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b70136620 con 0x7f7b7009d630 2023-09-30T22:28:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.590+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b60018260 con 0x7f7b7009d630 2023-09-30T22:28:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.609+0000 7f58ca173700 1 Processor -- start 2023-09-30T22:28:29.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.609+0000 7f58ca173700 1 -- start start 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58ca173700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58c4133420 0x7f58c41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58ca173700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4134260 0x7f58c41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58ca173700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c406b990 con 0x7f58c4133e30 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58ca173700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40a0590 con 0x7f58c4133420 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58ca173700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c409fb80 con 0x7f58c4134260 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58c4133420 0x7f58c41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58c37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58c4133420 0x7f58c41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46300/0 (socket says 172.21.15.32:46300) 2023-09-30T22:28:29.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58c2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58c4133e30 0x7f58c412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48274/0 (socket says 172.21.15.32:48274) 2023-09-30T22:28:29.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58c3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4134260 0x7f58c41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.610+0000 7f58c37fe700 1 -- 172.21.15.32:0/4049995622 learned_addr learned my addr 172.21.15.32:0/4049995622 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c37fe700 1 -- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4134260 msgr2=0x7f58c41306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c37fe700 1 --2- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4134260 0x7f58c41306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c37fe700 1 -- 172.21.15.32:0/4049995622 >> v1:172.21.15.32:6789/0 conn(0x7f58c4133e30 legacy=0x7f58c412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c37fe700 1 -- 172.21.15.32:0/4049995622 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40a0030 con 0x7f58c4133420 2023-09-30T22:28:29.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c3fff700 1 --2- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4134260 0x7f58c41306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c37fe700 1 --2- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4133420 0x7f58c41337f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f58b40098f0 tx=0x7f58b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ea7efef2c6f6fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c27fc700 1 -- 172.21.15.32:0/4049995622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4004230 con 0x7f58c4133420 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c27fc700 1 -- 172.21.15.32:0/4049995622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4004390 con 0x7f58c4133420 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.611+0000 7f58c27fc700 1 -- 172.21.15.32:0/4049995622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40036a0 con 0x7f58c4133420 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 -- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4133420 msgr2=0x7f58c41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 --2- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4133420 0x7f58c41337f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f58b40098f0 tx=0x7f58b4009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 -- 172.21.15.32:0/4049995622 shutdown_connections 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 --2- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4134260 0x7f58c41306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 --2- 172.21.15.32:0/4049995622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4133420 0x7f58c41337f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 -- 172.21.15.32:0/4049995622 >> 172.21.15.32:0/4049995622 conn(0x7f58c40a4210 msgr2=0x7f58c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 -- 172.21.15.32:0/4049995622 shutdown_connections 2023-09-30T22:28:29.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.612+0000 7f58ca173700 1 -- 172.21.15.32:0/4049995622 wait complete. 2023-09-30T22:28:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.613+0000 7f58ca173700 1 Processor -- start 2023-09-30T22:28:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.613+0000 7f58ca173700 1 -- start start 2023-09-30T22:28:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58ca173700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 0x7f58c409c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58ca173700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 0x7f58c409c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58ca173700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4134260 0x7f58c409cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58ca173700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c409df10 con 0x7f58c4134260 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58ca173700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41203b0 con 0x7f58c4133420 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58ca173700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c411fd30 con 0x7f58c4133e30 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 0x7f58c409c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 0x7f58c409c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 0x7f58c409c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43080/0 (socket says 172.21.15.32:43080) 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 0x7f58c409c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:43846/0 (socket says 172.21.15.32:43846) 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c37fe700 1 -- 172.21.15.32:0/437829713 learned_addr learned my addr 172.21.15.32:0/437829713 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c3fff700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4134260 0x7f58c409cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.614+0000 7f58c37fe700 1 -- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 msgr2=0x7f58c409c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c37fe700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 0x7f58c409c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c37fe700 1 -- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4134260 msgr2=0x7f58c409cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c37fe700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4134260 0x7f58c409cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c37fe700 1 -- 172.21.15.32:0/437829713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc009a70 con 0x7f58c4133420 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c2ffd700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 0x7f58c409c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c3fff700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4134260 0x7f58c409cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58c37fe700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 0x7f58c409c440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009130 tx=0x7f58b4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40045d0 con 0x7f58c4133420 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4011630 con 0x7f58c4133420 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b4009550 con 0x7f58c4133420 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40119a0 con 0x7f58c4133420 2023-09-30T22:28:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.615+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c411fa20 con 0x7f58c4133420 2023-09-30T22:28:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.616+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4074ea0 con 0x7f58c4133420 2023-09-30T22:28:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.617+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58b4022020 con 0x7f58c4133420 2023-09-30T22:28:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.617+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f58b4041440 con 0x7f58c4133420 2023-09-30T22:28:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.621+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58b402f480 con 0x7f58c4133420 2023-09-30T22:28:29.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.712+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f20f01247c0 con 0x7f20f009f0f0 2023-09-30T22:28:29.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.713+0000 7f20ecff9700 1 -- 172.21.15.32:0/87412751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20f01247c0 con 0x7f20f009f0f0 2023-09-30T22:28:29.714 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 msgr2=0x7f20f0121c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0121c50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f20e40090b0 tx=0x7f20e4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 shutdown_connections 2023-09-30T22:28:29.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20f0137ff0 0x7f20f01c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20f009e7e0 0x7f20f0121710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 --2- 172.21.15.32:0/87412751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20f009f0f0 0x7f20f0121c50 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 >> 172.21.15.32:0/87412751 conn(0x7f20f00a4300 msgr2=0x7f20f0135b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 shutdown_connections 2023-09-30T22:28:29.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.715+0000 7f20f62da700 1 -- 172.21.15.32:0/87412751 wait complete. 2023-09-30T22:28:29.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:29.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.798+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7b70136c80 con 0x7f7b7009d630 2023-09-30T22:28:29.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.799+0000 7f7b577fe700 1 -- 172.21.15.32:0/3955476591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b60017020 con 0x7f7b7009d630 2023-09-30T22:28:29.800 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:29.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 msgr2=0x7f7b70133850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 0x7f7b70133850 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c510 tx=0x7f7b6000c820 comp rx=0 tx=0).stop 2023-09-30T22:28:29.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 shutdown_connections 2023-09-30T22:28:29.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b70138d80 0x7f7b701342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b7009d630 0x7f7b70133850 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 --2- 172.21.15.32:0/3955476591 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b700a0a30 0x7f7b70133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 >> 172.21.15.32:0/3955476591 conn(0x7f7b700a3050 msgr2=0x7f7b7009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 shutdown_connections 2023-09-30T22:28:29.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.801+0000 7f7b7522b700 1 -- 172.21.15.32:0/3955476591 wait complete. 2023-09-30T22:28:29.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.830+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f58c406b250 con 0x7f58c4133420 2023-09-30T22:28:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.831+0000 7f58abfff700 1 -- 172.21.15.32:0/437829713 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58b402a020 con 0x7f58c4133420 2023-09-30T22:28:29.832 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:29.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 msgr2=0x7f58c409c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 0x7f58c409c440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009130 tx=0x7f58b4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 shutdown_connections 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c4134260 0x7f58c409cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c4133420 0x7f58c409c440 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 --2- 172.21.15.32:0/437829713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c4133e30 0x7f58c409c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 >> 172.21.15.32:0/437829713 conn(0x7f58c40a4210 msgr2=0x7f58c40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:29.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 shutdown_connections 2023-09-30T22:28:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:29.833+0000 7f58ca173700 1 -- 172.21.15.32:0/437829713 wait complete. 2023-09-30T22:28:29.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:30.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.510+0000 7f46d49ef700 1 Processor -- start 2023-09-30T22:28:30.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 -- start start 2023-09-30T22:28:30.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 0x7f46d01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 0x7f46d0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d006b460 con 0x7f46d009e410 2023-09-30T22:28:30.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d009b120 con 0x7f46d009d6f0 2023-09-30T22:28:30.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46d49ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46d009a7e0 con 0x7f46d009e000 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46cdd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 0x7f46d01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46ced9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 0x7f46d0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46ce59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46ce59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41260/0 (socket says 172.21.15.32:41260) 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46cdd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 0x7f46d01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53752/0 (socket says 172.21.15.32:53752) 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46ce59c700 1 -- 172.21.15.32:0/1044758861 learned_addr learned my addr 172.21.15.32:0/1044758861 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.511+0000 7f46ced9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 0x7f46d0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50994/0 (socket says 172.21.15.32:50994) 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ce59c700 1 -- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 msgr2=0x7f46d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ce59c700 1 --2- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 0x7f46d01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ce59c700 1 -- 172.21.15.32:0/1044758861 >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 msgr2=0x7f46d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ce59c700 1 --2- 172.21.15.32:0/1044758861 >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 0x7f46d0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ce59c700 1 -- 172.21.15.32:0/1044758861 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d009a500 con 0x7f46d009d6f0 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46cdd9b700 1 --2- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 0x7f46d01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ced9d700 1 --2- 172.21.15.32:0/1044758861 >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 0x7f46d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46ce59c700 1 --2- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d009dac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009920 tx=0x7f46c0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e671649723441729 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46cd59a700 1 -- 172.21.15.32:0/1044758861 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011830 con 0x7f46d009d6f0 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46cd59a700 1 -- 172.21.15.32:0/1044758861 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0011990 con 0x7f46d009d6f0 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.512+0000 7f46cd59a700 1 -- 172.21.15.32:0/1044758861 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011d20 con 0x7f46d009d6f0 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 -- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 msgr2=0x7f46d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 --2- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d009dac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009920 tx=0x7f46c0009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 -- 172.21.15.32:0/1044758861 shutdown_connections 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 --2- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d009dac0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 --2- 172.21.15.32:0/1044758861 >> v2:172.21.15.32:3300/0 conn(0x7f46d009e410 0x7f46d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 --2- 172.21.15.32:0/1044758861 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e000 0x7f46d01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 -- 172.21.15.32:0/1044758861 >> 172.21.15.32:0/1044758861 conn(0x7f46d0066f50 msgr2=0x7f46d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 -- 172.21.15.32:0/1044758861 shutdown_connections 2023-09-30T22:28:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.513+0000 7f46d49ef700 1 -- 172.21.15.32:0/1044758861 wait complete. 2023-09-30T22:28:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.514+0000 7f46d49ef700 1 Processor -- start 2023-09-30T22:28:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.514+0000 7f46d49ef700 1 -- start start 2023-09-30T22:28:30.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46d49ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d01a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46d49ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 0x7f46d01a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46d49ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e410 0x7f46d01a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46d49ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d01a7ac0 con 0x7f46d009e000 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46d49ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d01a80e0 con 0x7f46d009d6f0 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46d49ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46d01a8d10 con 0x7f46d009e410 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46cdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 0x7f46d01a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d01a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ced9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e410 0x7f46d01a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46cdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 0x7f46d01a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50998/0 (socket says 172.21.15.32:50998) 2023-09-30T22:28:30.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46cdd9b700 1 -- 172.21.15.32:0/4207246344 learned_addr learned my addr 172.21.15.32:0/4207246344 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 -- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e410 msgr2=0x7f46d01a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e410 0x7f46d01a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 -- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 msgr2=0x7f46d01a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 0x7f46d01a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 -- 172.21.15.32:0/4207246344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b80094a0 con 0x7f46d009d6f0 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46cdd9b700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 0x7f46d01a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.515+0000 7f46ce59c700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d01a5f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f46c0016000 tx=0x7f46c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.516+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0004240 con 0x7f46d009d6f0 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.516+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c001f070 con 0x7f46d009d6f0 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.516+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c8009a70 con 0x7f46d009d6f0 2023-09-30T22:28:30.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.516+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00036a0 con 0x7f46d009d6f0 2023-09-30T22:28:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.516+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c0009550 con 0x7f46d009d6f0 2023-09-30T22:28:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.516+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f46c0021030 con 0x7f46d009d6f0 2023-09-30T22:28:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.517+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f46c00405f0 con 0x7f46d009d6f0 2023-09-30T22:28:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.517+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d00a2140 con 0x7f46d009d6f0 2023-09-30T22:28:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.521+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46c0015020 con 0x7f46d009d6f0 2023-09-30T22:28:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.727+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f46d009c710 con 0x7f46d009d6f0 2023-09-30T22:28:30.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.727+0000 7f46c6ffd700 1 -- 172.21.15.32:0/4207246344 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f46c0003800 con 0x7f46d009d6f0 2023-09-30T22:28:30.728 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 msgr2=0x7f46d01a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d01a5f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f46c0016000 tx=0x7f46c0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 shutdown_connections 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46d009e410 0x7f46d01a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46d009d6f0 0x7f46d01a5f40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 --2- 172.21.15.32:0/4207246344 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46d009e000 0x7f46d01a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 >> 172.21.15.32:0/4207246344 conn(0x7f46d0066f50 msgr2=0x7f46d01bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 shutdown_connections 2023-09-30T22:28:30.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.730+0000 7f46d49ef700 1 -- 172.21.15.32:0/4207246344 wait complete. 2023-09-30T22:28:30.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:30.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:30.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:30.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.870+0000 7feba4e55700 1 Processor -- start 2023-09-30T22:28:30.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.870+0000 7feba4e55700 1 -- start start 2023-09-30T22:28:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feba4e55700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feba4e55700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feba0135c20 0x7feba0138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feba4e55700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba006fa40 con 0x7feba012cf60 2023-09-30T22:28:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feba4e55700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba012cd90 con 0x7feba0135c20 2023-09-30T22:28:30.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feba4e55700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba0071780 con 0x7feba006f5d0 2023-09-30T22:28:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feb9dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feb9dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba012ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41276/0 (socket says 172.21.15.32:41276) 2023-09-30T22:28:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feb9dd9b700 1 -- 172.21.15.32:0/2643453585 learned_addr learned my addr 172.21.15.32:0/2643453585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:30.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.871+0000 7feb9ed9d700 1 --2- 172.21.15.32:0/2643453585 >> v2:172.21.15.32:3300/0 conn(0x7feba0135c20 0x7feba0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9ed9d700 1 -- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 msgr2=0x7feba012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:30.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9ed9d700 1 --2- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9ed9d700 1 -- 172.21.15.32:0/2643453585 >> v1:172.21.15.32:6789/0 conn(0x7feba012cf60 legacy=0x7feba012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9ed9d700 1 -- 172.21.15.32:0/2643453585 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0071a40 con 0x7feba0135c20 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9dd9b700 1 --2- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9ed9d700 1 --2- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba0135c20 0x7feba0138000 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7feb98009d90 tx=0x7feb9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce82af9511a36463 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9d59a700 1 -- 172.21.15.32:0/2643453585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9800e070 con 0x7feba0135c20 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9d59a700 1 -- 172.21.15.32:0/2643453585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9800bae0 con 0x7feba0135c20 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.872+0000 7feb9d59a700 1 -- 172.21.15.32:0/2643453585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9800be90 con 0x7feba0135c20 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 -- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba0135c20 msgr2=0x7feba0138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 --2- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba0135c20 0x7feba0138000 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7feb98009d90 tx=0x7feb9800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 -- 172.21.15.32:0/2643453585 shutdown_connections 2023-09-30T22:28:30.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 --2- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba0135c20 0x7feba0138000 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 --2- 172.21.15.32:0/2643453585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 -- 172.21.15.32:0/2643453585 >> 172.21.15.32:0/2643453585 conn(0x7feba00a2aa0 msgr2=0x7feba00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 -- 172.21.15.32:0/2643453585 shutdown_connections 2023-09-30T22:28:30.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.873+0000 7feba4e55700 1 -- 172.21.15.32:0/2643453585 wait complete. 2023-09-30T22:28:30.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.874+0000 7feba4e55700 1 Processor -- start 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 -- start start 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba0099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 0x7feba009a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feba0135c20 0x7feba009a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba009c270 con 0x7feba012cf60 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba009eb50 con 0x7feba006f5d0 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feba4e55700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feba009c4c0 con 0x7feba0135c20 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feb9e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba0099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feb9ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feba0135c20 0x7feba009a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feb9dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 0x7feba009a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feb9ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feba0135c20 0x7feba009a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53768/0 (socket says 172.21.15.32:53768) 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feb9dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 0x7feba009a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51014/0 (socket says 172.21.15.32:51014) 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.875+0000 7feb9ed9d700 1 -- 172.21.15.32:0/575329606 learned_addr learned my addr 172.21.15.32:0/575329606 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9e59c700 1 -- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feba0135c20 msgr2=0x7feba009a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9e59c700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feba0135c20 0x7feba009a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9e59c700 1 -- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 msgr2=0x7feba009a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9e59c700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 0x7feba009a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9e59c700 1 -- 172.21.15.32:0/575329606 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb94009550 con 0x7feba006f5d0 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9dd9b700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 0x7feba009a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb9e59c700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba0099d50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7feb9000c3e0 tx=0x7feb9000c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb900052d0 con 0x7feba006f5d0 2023-09-30T22:28:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb90005450 con 0x7feba006f5d0 2023-09-30T22:28:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb98009a70 con 0x7feba006f5d0 2023-09-30T22:28:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb900038c0 con 0x7feba006f5d0 2023-09-30T22:28:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.876+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba013c660 con 0x7feba006f5d0 2023-09-30T22:28:30.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.878+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb90003a20 con 0x7feba006f5d0 2023-09-30T22:28:30.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.878+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba012a000 con 0x7feba006f5d0 2023-09-30T22:28:30.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.878+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb9003eb00 con 0x7feba006f5d0 2023-09-30T22:28:30.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.883+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb9001b8d0 con 0x7feba006f5d0 2023-09-30T22:28:30.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.958+0000 7f12e2a4a700 1 Processor -- start 2023-09-30T22:28:30.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.958+0000 7f12e2a4a700 1 -- start start 2023-09-30T22:28:30.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12e2a4a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12e2a4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e000 0x7f12dc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12e2a4a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc09b120 con 0x7f12dc09e410 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12e2a4a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc06b460 con 0x7f12dc09e000 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12dbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12e2a4a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc09a7e0 con 0x7f12dc09d6f0 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e000 0x7f12dc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12dbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53776/0 (socket says 172.21.15.32:53776) 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12dbfff700 1 -- 172.21.15.32:0/793608423 learned_addr learned my addr 172.21.15.32:0/793608423 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12db7fe700 1 -- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 msgr2=0x7f12dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12db7fe700 1 --2- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12db7fe700 1 -- 172.21.15.32:0/793608423 >> v1:172.21.15.32:6789/0 conn(0x7f12dc09e410 legacy=0x7f12dc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12db7fe700 1 -- 172.21.15.32:0/793608423 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc09a500 con 0x7f12dc09e000 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12daffd700 1 -- 172.21.15.32:0/793608423 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846250798 0 0) 0x7f12dc09b120 con 0x7f12dc09e410 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.959+0000 7f12dbfff700 1 --2- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.960+0000 7f12db7fe700 1 --2- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e000 0x7f12dc1339f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009870 tx=0x7f12d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d2fbb499cadc3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.960+0000 7f12daffd700 1 -- 172.21.15.32:0/793608423 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0011830 con 0x7f12dc09e000 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.960+0000 7f12daffd700 1 -- 172.21.15.32:0/793608423 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0011990 con 0x7f12dc09e000 2023-09-30T22:28:30.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.960+0000 7f12daffd700 1 -- 172.21.15.32:0/793608423 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0011d20 con 0x7f12dc09e000 2023-09-30T22:28:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 -- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e000 msgr2=0x7f12dc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e000 0x7f12dc1339f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009870 tx=0x7f12d0009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 -- 172.21.15.32:0/793608423 shutdown_connections 2023-09-30T22:28:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/793608423 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e000 0x7f12dc1339f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 -- 172.21.15.32:0/793608423 >> 172.21.15.32:0/793608423 conn(0x7f12dc066f50 msgr2=0x7f12dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 -- 172.21.15.32:0/793608423 shutdown_connections 2023-09-30T22:28:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.961+0000 7f12e2a4a700 1 -- 172.21.15.32:0/793608423 wait complete. 2023-09-30T22:28:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.962+0000 7f12e2a4a700 1 Processor -- start 2023-09-30T22:28:30.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.962+0000 7f12e2a4a700 1 -- start start 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e2a4a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e2a4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 0x7f12dc1229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e2a4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e410 0x7f12dc122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e2a4a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc125cb0 con 0x7f12dc09e000 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e2a4a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc125f90 con 0x7f12dc09e410 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e2a4a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0a30d0 con 0x7f12dc09d6f0 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 0x7f12dc1229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12e0fe7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e410 0x7f12dc122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 0x7f12dc1229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51030/0 (socket says 172.21.15.32:51030) 2023-09-30T22:28:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12db7fe700 1 -- 172.21.15.32:0/2866328102 learned_addr learned my addr 172.21.15.32:0/2866328102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.963+0000 7f12dbfff700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e0fe7700 1 -- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 msgr2=0x7f12dc122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e0fe7700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e0fe7700 1 -- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 msgr2=0x7f12dc1229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e0fe7700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 0x7f12dc1229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e0fe7700 1 -- 172.21.15.32:0/2866328102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc009550 con 0x7f12dc09e410 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12dbfff700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12db7fe700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 0x7f12dc1229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e0fe7700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e410 0x7f12dc122ef0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f12dc09e8b0 tx=0x7f12d400caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4003e30 con 0x7f12dc09e410 2023-09-30T22:28:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4005070 con 0x7f12dc09e410 2023-09-30T22:28:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00094a0 con 0x7f12dc09e410 2023-09-30T22:28:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4005420 con 0x7f12dc09e410 2023-09-30T22:28:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.964+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc124ea0 con 0x7f12dc09e410 2023-09-30T22:28:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.965+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12d4005580 con 0x7f12dc09e410 2023-09-30T22:28:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.965+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f12d403ec60 con 0x7f12dc09e410 2023-09-30T22:28:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.966+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc074ea0 con 0x7f12dc09e410 2023-09-30T22:28:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:30.970+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12d4017030 con 0x7f12dc09e410 2023-09-30T22:28:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 Processor -- start 2023-09-30T22:28:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 -- start start 2023-09-30T22:28:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 0x7f87b00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b0138d80 0x7f87b013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00a2a70 con 0x7f87b00a0a30 2023-09-30T22:28:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b00a01d0 con 0x7f87b009d630 2023-09-30T22:28:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.002+0000 7f87b6f60700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b009e2b0 con 0x7f87b0138d80 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87affff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 0x7f87b00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87affff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 0x7f87b00a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51042/0 (socket says 172.21.15.32:51042) 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b4cfc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f87b00a0a30 0x7f87b00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33502/0 (socket says 172.21.15.32:33502) 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87affff700 1 -- 172.21.15.32:0/4289428217 learned_addr learned my addr 172.21.15.32:0/4289428217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b54fd700 1 --2- 172.21.15.32:0/4289428217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b0138d80 0x7f87b013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b54fd700 1 -- 172.21.15.32:0/4289428217 >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 msgr2=0x7f87b00a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b54fd700 1 --2- 172.21.15.32:0/4289428217 >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 0x7f87b00a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b54fd700 1 -- 172.21.15.32:0/4289428217 >> v1:172.21.15.32:6789/0 conn(0x7f87b00a0a30 legacy=0x7f87b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b54fd700 1 -- 172.21.15.32:0/4289428217 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b009dc50 con 0x7f87b0138d80 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87af7fe700 1 -- 172.21.15.32:0/4289428217 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433306095 0 0) 0x7f87b00a2a70 con 0x7f87b00a0a30 2023-09-30T22:28:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87affff700 1 --2- 172.21.15.32:0/4289428217 >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 0x7f87b00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87b54fd700 1 --2- 172.21.15.32:0/4289428217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b0138d80 0x7f87b013b160 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9c1bd8e760d932 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87af7fe700 1 -- 172.21.15.32:0/4289428217 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800e070 con 0x7f87b0138d80 2023-09-30T22:28:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87af7fe700 1 -- 172.21.15.32:0/4289428217 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a800bb20 con 0x7f87b0138d80 2023-09-30T22:28:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.003+0000 7f87af7fe700 1 -- 172.21.15.32:0/4289428217 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8012430 con 0x7f87b0138d80 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 -- 172.21.15.32:0/4289428217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b0138d80 msgr2=0x7f87b013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 --2- 172.21.15.32:0/4289428217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b0138d80 0x7f87b013b160 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 -- 172.21.15.32:0/4289428217 shutdown_connections 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 --2- 172.21.15.32:0/4289428217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b0138d80 0x7f87b013b160 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 --2- 172.21.15.32:0/4289428217 >> v2:172.21.15.32:3300/0 conn(0x7f87b009d630 0x7f87b00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 -- 172.21.15.32:0/4289428217 >> 172.21.15.32:0/4289428217 conn(0x7f87b00a3050 msgr2=0x7f87b00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 -- 172.21.15.32:0/4289428217 shutdown_connections 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.004+0000 7f87b6f60700 1 -- 172.21.15.32:0/4289428217 wait complete. 2023-09-30T22:28:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.006+0000 7f87b6f60700 1 Processor -- start 2023-09-30T22:28:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.006+0000 7f87b6f60700 1 -- start start 2023-09-30T22:28:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b6f60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 0x7f87b0133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b6f60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87b00a0a30 0x7f87b0133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b6f60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87b0138d80 0x7f87b0134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b6f60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0137180 con 0x7f87b0138d80 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b6f60700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0136070 con 0x7f87b009d630 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b6f60700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0136530 con 0x7f87b00a0a30 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 0x7f87b0133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b54fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87b0138d80 0x7f87b0134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 0x7f87b0133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41342/0 (socket says 172.21.15.32:41342) 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 -- 172.21.15.32:0/1115673703 learned_addr learned my addr 172.21.15.32:0/1115673703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 -- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87b00a0a30 msgr2=0x7f87b0133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87b00a0a30 0x7f87b0133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 -- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87b0138d80 msgr2=0x7f87b0134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87b0138d80 0x7f87b0134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 -- 172.21.15.32:0/1115673703 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4009550 con 0x7f87b009d630 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b54fd700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87b0138d80 0x7f87b0134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.007+0000 7f87b4cfc700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 0x7f87b0133700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c520 tx=0x7f87a000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.008+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0004c80 con 0x7f87b009d630 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.008+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a0004e00 con 0x7f87b009d630 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.008+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a8009a70 con 0x7f87b009d630 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.008+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0007310 con 0x7f87b009d630 2023-09-30T22:28:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.008+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b0136be0 con 0x7f87b009d630 2023-09-30T22:28:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.009+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b0137d60 con 0x7f87b009d630 2023-09-30T22:28:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.010+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87a0007470 con 0x7f87b009d630 2023-09-30T22:28:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.010+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f87a003f9e0 con 0x7f87b009d630 2023-09-30T22:28:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.014+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87a00133c0 con 0x7f87b009d630 2023-09-30T22:28:31.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.097+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feba0074ea0 con 0x7feba006f5d0 2023-09-30T22:28:31.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.097+0000 7feb8effd700 1 -- 172.21.15.32:0/575329606 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb90017070 con 0x7feba006f5d0 2023-09-30T22:28:31.098 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.099+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 msgr2=0x7feba0099d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.099+0000 7feba4e55700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba0099d50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7feb9000c3e0 tx=0x7feb9000c6f0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 shutdown_connections 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feba0135c20 0x7feba009a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feba006f5d0 0x7feba0099d50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 --2- 172.21.15.32:0/575329606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba012cf60 0x7feba009a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 >> 172.21.15.32:0/575329606 conn(0x7feba00a2aa0 msgr2=0x7feba00a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:31.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 shutdown_connections 2023-09-30T22:28:31.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.100+0000 7feba4e55700 1 -- 172.21.15.32:0/575329606 wait complete. 2023-09-30T22:28:31.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.201+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f12dc06c360 con 0x7f12dc09e410 2023-09-30T22:28:31.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.201+0000 7f12d8ff9700 1 -- 172.21.15.32:0/2866328102 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12d4005830 con 0x7f12dc09e410 2023-09-30T22:28:31.203 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e410 msgr2=0x7f12dc122ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e410 0x7f12dc122ef0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f12dc09e8b0 tx=0x7f12d400caa0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 shutdown_connections 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12dc09e410 0x7f12dc122ef0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12dc09d6f0 0x7f12dc122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 --2- 172.21.15.32:0/2866328102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12dc09e000 0x7f12dc1229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 >> 172.21.15.32:0/2866328102 conn(0x7f12dc066f50 msgr2=0x7f12dc070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 shutdown_connections 2023-09-30T22:28:31.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.204+0000 7f12e2a4a700 1 -- 172.21.15.32:0/2866328102 wait complete. 2023-09-30T22:28:31.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.248+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f87b00747f0 con 0x7f87b009d630 2023-09-30T22:28:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.248+0000 7f87ad7fa700 1 -- 172.21.15.32:0/1115673703 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87a0011360 con 0x7f87b009d630 2023-09-30T22:28:31.249 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 msgr2=0x7f87b0133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 0x7f87b0133700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c520 tx=0x7f87a000c830 comp rx=0 tx=0).stop 2023-09-30T22:28:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 shutdown_connections 2023-09-30T22:28:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87b0138d80 0x7f87b0134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87b009d630 0x7f87b0133700 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 --2- 172.21.15.32:0/1115673703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87b00a0a30 0x7f87b0133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 >> 172.21.15.32:0/1115673703 conn(0x7f87b00a3050 msgr2=0x7f87b009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 shutdown_connections 2023-09-30T22:28:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.251+0000 7f87b6f60700 1 -- 172.21.15.32:0/1115673703 wait complete. 2023-09-30T22:28:31.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:31.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:31.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.891+0000 7f4c2f47d700 1 Processor -- start 2023-09-30T22:28:31.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.891+0000 7f4c2f47d700 1 -- start start 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2f47d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2f47d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c2809e410 0x7f4c28130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2f47d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2809f130 con 0x7f4c2809e000 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2f47d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2809f520 con 0x7f4c2809e410 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2f47d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2809e930 con 0x7f4c2809d6f0 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2d219700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2da1a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c2809e410 0x7f4c28130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2d219700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c2809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53800/0 (socket says 172.21.15.32:53800) 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2ca18700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4c2809e000 0x7f4c28130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33512/0 (socket says 172.21.15.32:33512) 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2d219700 1 -- 172.21.15.32:0/1617702857 learned_addr learned my addr 172.21.15.32:0/1617702857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2da1a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c2809e410 0x7f4c28130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51068/0 (socket says 172.21.15.32:51068) 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2da1a700 1 -- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 msgr2=0x7f4c2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2da1a700 1 --2- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c2809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2da1a700 1 -- 172.21.15.32:0/1617702857 >> v1:172.21.15.32:6789/0 conn(0x7f4c2809e000 legacy=0x7f4c28130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2da1a700 1 -- 172.21.15.32:0/1617702857 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2809b2d0 con 0x7f4c2809e410 2023-09-30T22:28:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.892+0000 7f4c2d219700 1 --2- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2da1a700 1 --2- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e410 0x7f4c28130d70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000bfa0 tx=0x7f4c2000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181fcd44bf3bff58 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c1bfff700 1 -- 172.21.15.32:0/1617702857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20010070 con 0x7f4c2809e410 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c1bfff700 1 -- 172.21.15.32:0/1617702857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2000b2e0 con 0x7f4c2809e410 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c1bfff700 1 -- 172.21.15.32:0/1617702857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000dc40 con 0x7f4c2809e410 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2f47d700 1 -- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e410 msgr2=0x7f4c28130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e410 0x7f4c28130d70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000bfa0 tx=0x7f4c2000d330 comp rx=0 tx=0).stop 2023-09-30T22:28:31.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2f47d700 1 -- 172.21.15.32:0/1617702857 shutdown_connections 2023-09-30T22:28:31.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e410 0x7f4c28130d70 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/1617702857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.893+0000 7f4c2f47d700 1 -- 172.21.15.32:0/1617702857 >> 172.21.15.32:0/1617702857 conn(0x7f4c28066f50 msgr2=0x7f4c2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:31.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.894+0000 7f4c2f47d700 1 -- 172.21.15.32:0/1617702857 shutdown_connections 2023-09-30T22:28:31.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.894+0000 7f4c2f47d700 1 -- 172.21.15.32:0/1617702857 wait complete. 2023-09-30T22:28:31.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.894+0000 7f4c2f47d700 1 Processor -- start 2023-09-30T22:28:31.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 -- start start 2023-09-30T22:28:31.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c281223b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e000 0x7f4c281228f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c2809e410 0x7f4c28122e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28125cd0 con 0x7f4c2809e000 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28125eb0 con 0x7f4c2809e410 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2f47d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28125910 con 0x7f4c2809d6f0 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c281223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2ca18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e000 0x7f4c281228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c281223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53804/0 (socket says 172.21.15.32:53804) 2023-09-30T22:28:31.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 -- 172.21.15.32:0/4039545626 learned_addr learned my addr 172.21.15.32:0/4039545626 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2da1a700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c2809e410 0x7f4c28122e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 -- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c2809e410 msgr2=0x7f4c28122e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c2809e410 0x7f4c28122e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 -- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e000 msgr2=0x7f4c281228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e000 0x7f4c281228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:31.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.895+0000 7f4c2d219700 1 -- 172.21.15.32:0/4039545626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c24009550 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.896+0000 7f4c2d219700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c281223b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9e0 tx=0x7f4c2400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.896+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ee80 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.896+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400c850 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.896+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2000bc80 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.896+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400cbc0 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.896+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2809c700 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.897+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c28074ea0 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.898+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4c2400cd20 con 0x7f4c2809d6f0 2023-09-30T22:28:31.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.898+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4c2403ff60 con 0x7f4c2809d6f0 2023-09-30T22:28:31.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:31.902+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c2406b750 con 0x7f4c2809d6f0 2023-09-30T22:28:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.107+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c2806c360 con 0x7f4c2809d6f0 2023-09-30T22:28:32.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.107+0000 7f4c19ffb700 1 -- 172.21.15.32:0/4039545626 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c24021020 con 0x7f4c2809d6f0 2023-09-30T22:28:32.107 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 msgr2=0x7f4c281223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c281223b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9e0 tx=0x7f4c2400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 shutdown_connections 2023-09-30T22:28:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c2809e410 0x7f4c28122e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c2809d6f0 0x7f4c281223b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 --2- 172.21.15.32:0/4039545626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c2809e000 0x7f4c281228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 >> 172.21.15.32:0/4039545626 conn(0x7f4c28066f50 msgr2=0x7f4c280710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 shutdown_connections 2023-09-30T22:28:32.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.110+0000 7f4c2f47d700 1 -- 172.21.15.32:0/4039545626 wait complete. 2023-09-30T22:28:32.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:32.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:32.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:32.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.256+0000 7f3549e23700 1 Processor -- start 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.256+0000 7f3549e23700 1 -- start start 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3549e23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35440a1510 0x7f3544138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3549e23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3544139250 0x7f354413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3549e23700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354409edd0 con 0x7f35440a0a30 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3549e23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35440a18e0 con 0x7f3544139250 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3549e23700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f354409e1f0 con 0x7f35440a1510 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3543fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3544139250 0x7f354413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3542ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35440a1510 0x7f3544138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f35437fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f35440a0a30 0x7f35440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33524/0 (socket says 172.21.15.32:33524) 2023-09-30T22:28:32.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f35437fe700 1 -- 172.21.15.32:0/3545031375 learned_addr learned my addr 172.21.15.32:0/3545031375 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3542ffd700 1 -- 172.21.15.32:0/3545031375 >> v2:172.21.15.32:3300/0 conn(0x7f3544139250 msgr2=0x7f354413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3542ffd700 1 --2- 172.21.15.32:0/3545031375 >> v2:172.21.15.32:3300/0 conn(0x7f3544139250 0x7f354413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3542ffd700 1 -- 172.21.15.32:0/3545031375 >> v1:172.21.15.32:6789/0 conn(0x7f35440a0a30 legacy=0x7f35440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3542ffd700 1 -- 172.21.15.32:0/3545031375 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354409dc50 con 0x7f35440a1510 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3543fff700 1 --2- 172.21.15.32:0/3545031375 >> v2:172.21.15.32:3300/0 conn(0x7f3544139250 0x7f354413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f3542ffd700 1 --2- 172.21.15.32:0/3545031375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35440a1510 0x7f3544138d10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f352c009870 tx=0x7f352c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=981dce5039333c59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.257+0000 7f35427fc700 1 -- 172.21.15.32:0/3545031375 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c011830 con 0x7f35440a1510 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f35427fc700 1 -- 172.21.15.32:0/3545031375 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c011990 con 0x7f35440a1510 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f35427fc700 1 -- 172.21.15.32:0/3545031375 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c011d20 con 0x7f35440a1510 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 -- 172.21.15.32:0/3545031375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35440a1510 msgr2=0x7f3544138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 --2- 172.21.15.32:0/3545031375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35440a1510 0x7f3544138d10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f352c009870 tx=0x7f352c009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 -- 172.21.15.32:0/3545031375 shutdown_connections 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 --2- 172.21.15.32:0/3545031375 >> v2:172.21.15.32:3300/0 conn(0x7f3544139250 0x7f354413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 --2- 172.21.15.32:0/3545031375 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35440a1510 0x7f3544138d10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 -- 172.21.15.32:0/3545031375 >> 172.21.15.32:0/3545031375 conn(0x7f35440a6550 msgr2=0x7f35440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 -- 172.21.15.32:0/3545031375 shutdown_connections 2023-09-30T22:28:32.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.258+0000 7f3549e23700 1 -- 172.21.15.32:0/3545031375 wait complete. 2023-09-30T22:28:32.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.259+0000 7f3549e23700 1 Processor -- start 2023-09-30T22:28:32.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.259+0000 7f3549e23700 1 -- start start 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3549e23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35440a0a30 0x7f35441c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3549e23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 0x7f35441c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3549e23700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3544139250 0x7f3544122a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3549e23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35441389b0 con 0x7f35440a1510 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3549e23700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3544125cd0 con 0x7f35440a0a30 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3549e23700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3544126710 con 0x7f3544139250 2023-09-30T22:28:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3542ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 0x7f35441c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f35437fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35440a0a30 0x7f35441c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3542ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 0x7f35441c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51106/0 (socket says 172.21.15.32:51106) 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3542ffd700 1 -- 172.21.15.32:0/4218295325 learned_addr learned my addr 172.21.15.32:0/4218295325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3543fff700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3544139250 0x7f3544122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f35437fe700 1 -- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3544139250 msgr2=0x7f3544122a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f35437fe700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3544139250 0x7f3544122a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f35437fe700 1 -- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 msgr2=0x7f35441c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f35437fe700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 0x7f35441c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f35437fe700 1 -- 172.21.15.32:0/4218295325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c009a70 con 0x7f35440a0a30 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.260+0000 7f3542ffd700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 0x7f35441c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.261+0000 7f35437fe700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35440a0a30 0x7f35441c8800 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f353400c500 tx=0x7f353400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.261+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353400cea0 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.261+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3534005d70 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.261+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c0094a0 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.261+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35340072d0 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.261+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544125fa0 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.262+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3534007430 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.262+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35441c92e0 con 0x7f35440a0a30 2023-09-30T22:28:32.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.262+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f353403e960 con 0x7f35440a0a30 2023-09-30T22:28:32.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:32.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.267+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3534017020 con 0x7f35440a0a30 2023-09-30T22:28:32.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.365+0000 7f0a302ba700 1 Processor -- start 2023-09-30T22:28:32.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.365+0000 7f0a302ba700 1 -- start start 2023-09-30T22:28:32.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a302ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130540 0x7f0a28130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a302ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 0x7f0a28138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a302ba700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280716a0 con 0x7f0a28130e50 2023-09-30T22:28:32.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a302ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2809a390 con 0x7f0a2806fcf0 2023-09-30T22:28:32.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a302ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28071290 con 0x7f0a28130540 2023-09-30T22:28:32.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 0x7f0a28138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 0x7f0a28138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41370/0 (socket says 172.21.15.32:41370) 2023-09-30T22:28:32.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 -- 172.21.15.32:0/1546790703 learned_addr learned my addr 172.21.15.32:0/1546790703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:32.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e056700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130540 0x7f0a28130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 -- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130540 msgr2=0x7f0a28130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130540 0x7f0a28130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 -- 172.21.15.32:0/1546790703 >> v1:172.21.15.32:6789/0 conn(0x7f0a28130e50 legacy=0x7f0a2806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e857700 1 -- 172.21.15.32:0/1546790703 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28130340 con 0x7f0a2806fcf0 2023-09-30T22:28:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.366+0000 7f0a2e056700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130540 0x7f0a28130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.367+0000 7f0a2e857700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 0x7f0a28138570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009eb0 tx=0x7f0a2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5faf13a2bfa97edd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:32.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.367+0000 7f0a2d054700 1 -- 172.21.15.32:0/1546790703 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000e070 con 0x7f0a2806fcf0 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.367+0000 7f0a2d054700 1 -- 172.21.15.32:0/1546790703 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2000b8d0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.367+0000 7f0a2d054700 1 -- 172.21.15.32:0/1546790703 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000bc80 con 0x7f0a2806fcf0 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.367+0000 7f0a302ba700 1 -- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 msgr2=0x7f0a28138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.367+0000 7f0a302ba700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 0x7f0a28138570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009eb0 tx=0x7f0a2000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.368+0000 7f0a302ba700 1 -- 172.21.15.32:0/1546790703 shutdown_connections 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.368+0000 7f0a302ba700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a2806fcf0 0x7f0a28138570 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.368+0000 7f0a302ba700 1 --2- 172.21.15.32:0/1546790703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130540 0x7f0a28130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.368+0000 7f0a302ba700 1 -- 172.21.15.32:0/1546790703 >> 172.21.15.32:0/1546790703 conn(0x7f0a280a2aa0 msgr2=0x7f0a280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.368+0000 7f0a302ba700 1 -- 172.21.15.32:0/1546790703 shutdown_connections 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.368+0000 7f0a302ba700 1 -- 172.21.15.32:0/1546790703 wait complete. 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 Processor -- start 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 -- start start 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 0x7f0a2809a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a28130540 0x7f0a2809aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130e50 0x7f0a2809b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2809dde0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2809cae0 con 0x7f0a28130540 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a302ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2809e4c0 con 0x7f0a28130e50 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a2e056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 0x7f0a2809a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a2d855700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a28130540 0x7f0a2809aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a2e056700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 0x7f0a2809a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51122/0 (socket says 172.21.15.32:51122) 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.369+0000 7f0a2e056700 1 -- 172.21.15.32:0/3652072043 learned_addr learned my addr 172.21.15.32:0/3652072043 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e857700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130e50 0x7f0a2809b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e056700 1 -- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130e50 msgr2=0x7f0a2809b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e056700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130e50 0x7f0a2809b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e056700 1 -- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a28130540 msgr2=0x7f0a2809aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e056700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a28130540 0x7f0a2809aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e056700 1 -- 172.21.15.32:0/3652072043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240094a0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a2e056700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 0x7f0a2809a590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400ebe0 tx=0x7f0a2400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400c850 con 0x7f0a2806fcf0 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2400c9b0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a20009b30 con 0x7f0a2806fcf0 2023-09-30T22:28:32.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.370+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400cd40 con 0x7f0a2806fcf0 2023-09-30T22:28:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.371+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2809d340 con 0x7f0a2806fcf0 2023-09-30T22:28:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.372+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a24013470 con 0x7f0a2806fcf0 2023-09-30T22:28:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.372+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a2403f7a0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.373+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a28074ea0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.378+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a24021080 con 0x7f0a2806fcf0 2023-09-30T22:28:32.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.407+0000 7f9a47a7e700 1 Processor -- start 2023-09-30T22:28:32.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.408+0000 7f9a47a7e700 1 -- start start 2023-09-30T22:28:32.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.408+0000 7f9a47a7e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 0x7f9a40135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.408+0000 7f9a47a7e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a40135e50 0x7f9a40138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.408+0000 7f9a47a7e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4009f130 con 0x7f9a4009d6f0 2023-09-30T22:28:32.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.408+0000 7f9a47a7e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4009f520 con 0x7f9a4009e1d0 2023-09-30T22:28:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.408+0000 7f9a47a7e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4009e930 con 0x7f9a40135e50 2023-09-30T22:28:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 0x7f9a40135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a4581a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a4009d6f0 0x7f9a4009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33552/0 (socket says 172.21.15.32:33552) 2023-09-30T22:28:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 0x7f9a40135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41386/0 (socket says 172.21.15.32:41386) 2023-09-30T22:28:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a4581a700 1 -- 172.21.15.32:0/3635733962 learned_addr learned my addr 172.21.15.32:0/3635733962 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:32.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a4601b700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a40135e50 0x7f9a40138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 -- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a40135e50 msgr2=0x7f9a40138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a40135e50 0x7f9a40138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 -- 172.21.15.32:0/3635733962 >> v1:172.21.15.32:6789/0 conn(0x7f9a4009d6f0 legacy=0x7f9a4009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a44818700 1 -- 172.21.15.32:0/3635733962 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867802703 0 0) 0x7f9a4009f130 con 0x7f9a4009d6f0 2023-09-30T22:28:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 -- 172.21.15.32:0/3635733962 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4009b2d0 con 0x7f9a4009e1d0 2023-09-30T22:28:32.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a4601b700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a40135e50 0x7f9a40138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a45019700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 0x7f9a40135910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009870 tx=0x7f9a30009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e16205e84f71dec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a44818700 1 -- 172.21.15.32:0/3635733962 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30011830 con 0x7f9a4009e1d0 2023-09-30T22:28:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a44818700 1 -- 172.21.15.32:0/3635733962 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a30011990 con 0x7f9a4009e1d0 2023-09-30T22:28:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.409+0000 7f9a44818700 1 -- 172.21.15.32:0/3635733962 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30011d20 con 0x7f9a4009e1d0 2023-09-30T22:28:32.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.410+0000 7f9a47a7e700 1 -- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 msgr2=0x7f9a40135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.410+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 0x7f9a40135910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009870 tx=0x7f9a30009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.410+0000 7f9a47a7e700 1 -- 172.21.15.32:0/3635733962 shutdown_connections 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.410+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a40135e50 0x7f9a40138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.410+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/3635733962 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a4009e1d0 0x7f9a40135910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.410+0000 7f9a47a7e700 1 -- 172.21.15.32:0/3635733962 >> 172.21.15.32:0/3635733962 conn(0x7f9a40066f50 msgr2=0x7f9a4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.411+0000 7f9a47a7e700 1 -- 172.21.15.32:0/3635733962 shutdown_connections 2023-09-30T22:28:32.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.411+0000 7f9a47a7e700 1 -- 172.21.15.32:0/3635733962 wait complete. 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.412+0000 7f9a47a7e700 1 Processor -- start 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.412+0000 7f9a47a7e700 1 -- start start 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a47a7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 0x7f9a401c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a47a7e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a4009e1d0 0x7f9a401c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a47a7e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a40135e50 0x7f9a401c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a47a7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a401209c0 con 0x7f9a4009d6f0 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a47a7e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4006e8d0 con 0x7f9a40135e50 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a47a7e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4011fae0 con 0x7f9a4009e1d0 2023-09-30T22:28:32.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 0x7f9a401c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 0x7f9a401c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51136/0 (socket says 172.21.15.32:51136) 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 -- 172.21.15.32:0/2564009970 learned_addr learned my addr 172.21.15.32:0/2564009970 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a45019700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a4009e1d0 0x7f9a401c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4601b700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a40135e50 0x7f9a401c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 -- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a4009e1d0 msgr2=0x7f9a401c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a4009e1d0 0x7f9a401c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 -- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a40135e50 msgr2=0x7f9a401c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a40135e50 0x7f9a401c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a4581a700 1 -- 172.21.15.32:0/2564009970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38009a70 con 0x7f9a4009d6f0 2023-09-30T22:28:32.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.413+0000 7f9a45019700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a4009e1d0 0x7f9a401c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.414+0000 7f9a4581a700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 0x7f9a401c6050 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00c520 tx=0x7f9a3c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.414+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c004c80 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.414+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c004de0 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.414+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a300094a0 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.414+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c007310 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.414+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4009c750 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.415+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a3c007470 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.415+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9a3c03f220 con 0x7f9a4009d6f0 2023-09-30T22:28:32.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.415+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4006b5d0 con 0x7f9a4009d6f0 2023-09-30T22:28:32.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.424+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a3c0123b0 con 0x7f9a4009d6f0 2023-09-30T22:28:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.481+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3544126190 con 0x7f35440a0a30 2023-09-30T22:28:32.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.481+0000 7f353bfff700 1 -- 172.21.15.32:0/4218295325 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f353401b3b0 con 0x7f35440a0a30 2023-09-30T22:28:32.481 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:32.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.484+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35440a0a30 msgr2=0x7f35441c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.484+0000 7f3549e23700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35440a0a30 0x7f35441c8800 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f353400c500 tx=0x7f353400c810 comp rx=0 tx=0).stop 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.484+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 shutdown_connections 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.484+0000 7f3549e23700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3544139250 0x7f3544122a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.484+0000 7f3549e23700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35440a0a30 0x7f35441c8800 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.484+0000 7f3549e23700 1 --2- 172.21.15.32:0/4218295325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35440a1510 0x7f35441c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.485+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 >> 172.21.15.32:0/4218295325 conn(0x7f35440a6550 msgr2=0x7f35440a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.485+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 shutdown_connections 2023-09-30T22:28:32.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.485+0000 7f3549e23700 1 -- 172.21.15.32:0/4218295325 wait complete. 2023-09-30T22:28:32.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:32.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.588+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a2812ddb0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.588+0000 7f0a1effd700 1 -- 172.21.15.32:0/3652072043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a2400cea0 con 0x7f0a2806fcf0 2023-09-30T22:28:32.589 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:32.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 msgr2=0x7f0a2809a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 0x7f0a2809a590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400ebe0 tx=0x7f0a2400eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 shutdown_connections 2023-09-30T22:28:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a28130e50 0x7f0a2809b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a2806fcf0 0x7f0a2809a590 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 --2- 172.21.15.32:0/3652072043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a28130540 0x7f0a2809aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 >> 172.21.15.32:0/3652072043 conn(0x7f0a280a2aa0 msgr2=0x7f0a28124020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 shutdown_connections 2023-09-30T22:28:32.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.590+0000 7f0a302ba700 1 -- 172.21.15.32:0/3652072043 wait complete. 2023-09-30T22:28:32.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:32.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.634+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9a4011fd00 con 0x7f9a4009d6f0 2023-09-30T22:28:32.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.634+0000 7f9a367fc700 1 -- 172.21.15.32:0/2564009970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a3c00ce60 con 0x7f9a4009d6f0 2023-09-30T22:28:32.635 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 msgr2=0x7f9a401c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 0x7f9a401c6050 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00c520 tx=0x7f9a3c00c830 comp rx=0 tx=0).stop 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 shutdown_connections 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a40135e50 0x7f9a401c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a4009d6f0 0x7f9a401c6050 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 --2- 172.21.15.32:0/2564009970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a4009e1d0 0x7f9a401c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:32.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.636+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 >> 172.21.15.32:0/2564009970 conn(0x7f9a40066f50 msgr2=0x7f9a40134e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.637+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 shutdown_connections 2023-09-30T22:28:32.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:32.637+0000 7f9a47a7e700 1 -- 172.21.15.32:0/2564009970 wait complete. 2023-09-30T22:28:32.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:33.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:33.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.270+0000 7ff987b8c700 1 Processor -- start 2023-09-30T22:28:33.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.270+0000 7ff987b8c700 1 -- start start 2023-09-30T22:28:33.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff987b8c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff987b8c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9801380f0 0x7ff98013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff987b8c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff980134980 con 0x7ff980134ec0 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff987b8c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98006c4b0 con 0x7ff9801380f0 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff987b8c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800a09c0 con 0x7ff98006c0e0 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9801380f0 0x7ff98013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff985928700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9801380f0 0x7ff98013fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51142/0 (socket says 172.21.15.32:51142) 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff985127700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff980134ec0 0x7ff9801379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33564/0 (socket says 172.21.15.32:33564) 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 -- 172.21.15.32:0/1248358703 learned_addr learned my addr 172.21.15.32:0/1248358703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff985928700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98009ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53874/0 (socket says 172.21.15.32:53874) 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 -- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 msgr2=0x7ff98009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 --2- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 -- 172.21.15.32:0/1248358703 >> v1:172.21.15.32:6789/0 conn(0x7ff980134ec0 legacy=0x7ff9801379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 -- 172.21.15.32:0/1248358703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800a0600 con 0x7ff9801380f0 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff984926700 1 -- 172.21.15.32:0/1248358703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953860298 0 0) 0x7ff980134980 con 0x7ff980134ec0 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff985928700 1 --2- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.271+0000 7ff986129700 1 --2- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98013fd20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff978009e50 tx=0x7ff97800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dd4d35667c2006 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff984926700 1 -- 172.21.15.32:0/1248358703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800e070 con 0x7ff9801380f0 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff984926700 1 -- 172.21.15.32:0/1248358703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800b910 con 0x7ff9801380f0 2023-09-30T22:28:33.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff984926700 1 -- 172.21.15.32:0/1248358703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800bcc0 con 0x7ff9801380f0 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 -- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 msgr2=0x7ff98013fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 --2- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98013fd20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff978009e50 tx=0x7ff97800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 -- 172.21.15.32:0/1248358703 shutdown_connections 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 --2- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98013fd20 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 --2- 172.21.15.32:0/1248358703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 -- 172.21.15.32:0/1248358703 >> 172.21.15.32:0/1248358703 conn(0x7ff9800a4210 msgr2=0x7ff9800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 -- 172.21.15.32:0/1248358703 shutdown_connections 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.272+0000 7ff987b8c700 1 -- 172.21.15.32:0/1248358703 wait complete. 2023-09-30T22:28:33.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.273+0000 7ff987b8c700 1 Processor -- start 2023-09-30T22:28:33.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.273+0000 7ff987b8c700 1 -- start start 2023-09-30T22:28:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff987b8c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff987b8c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff980134ec0 0x7ff98011abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff987b8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98011b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff987b8c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff980134270 con 0x7ff9801380f0 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff987b8c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98006d460 con 0x7ff980134ec0 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff987b8c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff980133fd0 con 0x7ff98006c0e0 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff986129700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98011b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53886/0 (socket says 172.21.15.32:53886) 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 -- 172.21.15.32:0/991400570 learned_addr learned my addr 172.21.15.32:0/991400570 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985127700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff980134ec0 0x7ff98011abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 -- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff980134ec0 msgr2=0x7ff98011abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff980134ec0 0x7ff98011abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 -- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 msgr2=0x7ff98011b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98011b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985928700 1 -- 172.21.15.32:0/991400570 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0094a0 con 0x7ff98006c0e0 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff986129700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98011b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.274+0000 7ff985127700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff980134ec0 0x7ff98011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.275+0000 7ff985928700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98011a680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00ec10 tx=0x7ff97c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.275+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c009760 con 0x7ff98006c0e0 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.275+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c00c850 con 0x7ff98006c0e0 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.275+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff978009b30 con 0x7ff98006c0e0 2023-09-30T22:28:33.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.275+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00cbe0 con 0x7ff98006c0e0 2023-09-30T22:28:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.275+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98011d610 con 0x7ff98006c0e0 2023-09-30T22:28:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.276+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff97c00cd40 con 0x7ff98006c0e0 2023-09-30T22:28:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.276+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980074ea0 con 0x7ff98006c0e0 2023-09-30T22:28:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.276+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff97c03fa70 con 0x7ff98006c0e0 2023-09-30T22:28:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.281+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff97c01b3b0 con 0x7ff98006c0e0 2023-09-30T22:28:33.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.483+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff98006b250 con 0x7ff98006c0e0 2023-09-30T22:28:33.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.484+0000 7ff9727fc700 1 -- 172.21.15.32:0/991400570 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff97c011eb0 con 0x7ff98006c0e0 2023-09-30T22:28:33.484 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 msgr2=0x7ff98011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98011a680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00ec10 tx=0x7ff97c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 shutdown_connections 2023-09-30T22:28:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9801380f0 0x7ff98011b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff98006c0e0 0x7ff98011a680 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 --2- 172.21.15.32:0/991400570 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff980134ec0 0x7ff98011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 >> 172.21.15.32:0/991400570 conn(0x7ff9800a4210 msgr2=0x7ff980071960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 shutdown_connections 2023-09-30T22:28:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.486+0000 7ff987b8c700 1 -- 172.21.15.32:0/991400570 wait complete. 2023-09-30T22:28:33.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:33.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:33.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:33.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.640+0000 7f00f033a700 1 Processor -- start 2023-09-30T22:28:33.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.640+0000 7f00f033a700 1 -- start start 2023-09-30T22:28:33.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00f033a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00e80a0a30 0x7f00e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00f033a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00f033a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e80a2a70 con 0x7f00e809d630 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00f033a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e80a01d0 con 0x7f00e80a0a30 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00f033a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00e809e2b0 con 0x7f00e8138d80 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee0d6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00e80a0a30 0x7f00e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee8d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ed8d5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f00e809d630 0x7f00e80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33568/0 (socket says 172.21.15.32:33568) 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ed8d5700 1 -- 172.21.15.32:0/3195825267 learned_addr learned my addr 172.21.15.32:0/3195825267 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee0d6700 1 -- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 msgr2=0x7f00e813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:33.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee0d6700 1 --2- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee0d6700 1 -- 172.21.15.32:0/3195825267 >> v1:172.21.15.32:6789/0 conn(0x7f00e809d630 legacy=0x7f00e80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee0d6700 1 -- 172.21.15.32:0/3195825267 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e809dc50 con 0x7f00e80a0a30 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ed0d4700 1 -- 172.21.15.32:0/3195825267 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735565458 0 0) 0x7f00e80a2a70 con 0x7f00e809d630 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.641+0000 7f00ee8d7700 1 --2- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.642+0000 7f00ee0d6700 1 --2- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e80a0e00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009840 tx=0x7f00e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf074e9b75591c96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.642+0000 7f00ed0d4700 1 -- 172.21.15.32:0/3195825267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4011830 con 0x7f00e80a0a30 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.642+0000 7f00ed0d4700 1 -- 172.21.15.32:0/3195825267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e4011990 con 0x7f00e80a0a30 2023-09-30T22:28:33.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.642+0000 7f00ed0d4700 1 -- 172.21.15.32:0/3195825267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4011d20 con 0x7f00e80a0a30 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.642+0000 7f00f033a700 1 -- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 msgr2=0x7f00e80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.642+0000 7f00f033a700 1 --2- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e80a0e00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009840 tx=0x7f00e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.643+0000 7f00f033a700 1 -- 172.21.15.32:0/3195825267 shutdown_connections 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.643+0000 7f00f033a700 1 --2- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.643+0000 7f00f033a700 1 --2- 172.21.15.32:0/3195825267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e80a0e00 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.643+0000 7f00f033a700 1 -- 172.21.15.32:0/3195825267 >> 172.21.15.32:0/3195825267 conn(0x7f00e80a3050 msgr2=0x7f00e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.643+0000 7f00f033a700 1 -- 172.21.15.32:0/3195825267 shutdown_connections 2023-09-30T22:28:33.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.643+0000 7f00f033a700 1 -- 172.21.15.32:0/3195825267 wait complete. 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 Processor -- start 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 -- start start 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e809d630 0x7f00e81337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e8133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e8134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8135bd0 con 0x7f00e80a0a30 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8134bb0 con 0x7f00e809d630 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00f033a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00e80a4ae0 con 0x7f00e8138d80 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00ed8d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e8133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00ed8d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e8133d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51176/0 (socket says 172.21.15.32:51176) 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00ed8d5700 1 -- 172.21.15.32:0/609342308 learned_addr learned my addr 172.21.15.32:0/609342308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00ee8d7700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e8134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.644+0000 7f00ee0d6700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e809d630 0x7f00e81337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ed8d5700 1 -- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 msgr2=0x7f00e8134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ed8d5700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e8134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ed8d5700 1 -- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e809d630 msgr2=0x7f00e81337c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ed8d5700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e809d630 0x7f00e81337c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ed8d5700 1 -- 172.21.15.32:0/609342308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0009a70 con 0x7f00e80a0a30 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ee8d7700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e8134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ee0d6700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e809d630 0x7f00e81337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00ed8d5700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e8133d00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c550 tx=0x7f00d800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8004c80 con 0x7f00e80a0a30 2023-09-30T22:28:33.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d8004e00 con 0x7f00e80a0a30 2023-09-30T22:28:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e40094a0 con 0x7f00e80a0a30 2023-09-30T22:28:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8007310 con 0x7f00e80a0a30 2023-09-30T22:28:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.645+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8137210 con 0x7f00e80a0a30 2023-09-30T22:28:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.646+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f00d8007470 con 0x7f00e80a0a30 2023-09-30T22:28:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.646+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f00d803f2d0 con 0x7f00e80a0a30 2023-09-30T22:28:33.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.647+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e81365b0 con 0x7f00e80a0a30 2023-09-30T22:28:33.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:33.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.656+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f00d80123b0 con 0x7f00e80a0a30 2023-09-30T22:28:33.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.751+0000 7ffa78051700 1 Processor -- start 2023-09-30T22:28:33.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa78051700 1 -- start start 2023-09-30T22:28:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa78051700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 0x7ffa700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa78051700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 0x7ffa7013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa78051700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7009edd0 con 0x7ffa700a1340 2023-09-30T22:28:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa78051700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7009e1f0 con 0x7ffa70134170 2023-09-30T22:28:33.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa78051700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7009dc50 con 0x7ffa700a0a30 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa75ded700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 0x7ffa700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa765ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 0x7ffa7013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa75ded700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 0x7ffa700a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53930/0 (socket says 172.21.15.32:53930) 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa755ec700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffa700a1340 0x7ffa70133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33570/0 (socket says 172.21.15.32:33570) 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa75ded700 1 -- 172.21.15.32:0/1009778430 learned_addr learned my addr 172.21.15.32:0/1009778430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.752+0000 7ffa765ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 0x7ffa7013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41436/0 (socket says 172.21.15.32:41436) 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa75ded700 1 -- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 msgr2=0x7ffa7013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa75ded700 1 --2- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 0x7ffa7013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa75ded700 1 -- 172.21.15.32:0/1009778430 >> v1:172.21.15.32:6789/0 conn(0x7ffa700a1340 legacy=0x7ffa70133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa75ded700 1 -- 172.21.15.32:0/1009778430 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700a1fe0 con 0x7ffa700a0a30 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa765ee700 1 --2- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 0x7ffa7013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa75ded700 1 --2- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 0x7ffa700a0e00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009920 tx=0x7ffa6c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1686da6184bb5d71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa74deb700 1 -- 172.21.15.32:0/1009778430 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c011830 con 0x7ffa700a0a30 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa74deb700 1 -- 172.21.15.32:0/1009778430 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c011990 con 0x7ffa700a0a30 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.753+0000 7ffa74deb700 1 -- 172.21.15.32:0/1009778430 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c011d20 con 0x7ffa700a0a30 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 -- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 msgr2=0x7ffa700a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 --2- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 0x7ffa700a0e00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009920 tx=0x7ffa6c009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 -- 172.21.15.32:0/1009778430 shutdown_connections 2023-09-30T22:28:33.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 --2- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa70134170 0x7ffa7013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 --2- 172.21.15.32:0/1009778430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa700a0a30 0x7ffa700a0e00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 -- 172.21.15.32:0/1009778430 >> 172.21.15.32:0/1009778430 conn(0x7ffa700a6550 msgr2=0x7ffa700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 -- 172.21.15.32:0/1009778430 shutdown_connections 2023-09-30T22:28:33.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.754+0000 7ffa78051700 1 -- 172.21.15.32:0/1009778430 wait complete. 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.755+0000 7ffa78051700 1 Processor -- start 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.755+0000 7ffa78051700 1 -- start start 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa78051700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700a0a30 0x7ffa7011e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa78051700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa700a1340 0x7ffa7011ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa78051700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa70134170 0x7ffa7011f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa78051700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa701222f0 con 0x7ffa700a0a30 2023-09-30T22:28:33.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa78051700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa701203e0 con 0x7ffa700a1340 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa78051700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700a5000 con 0x7ffa70134170 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa75ded700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700a0a30 0x7ffa7011e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa75ded700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700a0a30 0x7ffa7011e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51180/0 (socket says 172.21.15.32:51180) 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa75ded700 1 -- 172.21.15.32:0/885005660 learned_addr learned my addr 172.21.15.32:0/885005660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa765ee700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa70134170 0x7ffa7011f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa755ec700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa700a1340 0x7ffa7011ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa755ec700 1 -- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa70134170 msgr2=0x7ffa7011f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa755ec700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa70134170 0x7ffa7011f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa755ec700 1 -- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700a0a30 msgr2=0x7ffa7011e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa755ec700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700a0a30 0x7ffa7011e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa755ec700 1 -- 172.21.15.32:0/885005660 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa68009a70 con 0x7ffa700a1340 2023-09-30T22:28:33.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.756+0000 7ffa765ee700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa70134170 0x7ffa7011f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.757+0000 7ffa755ec700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa700a1340 0x7ffa7011ed60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000c450 tx=0x7ffa6000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.757+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60005cb0 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.757+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60005e10 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.757+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c009550 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.757+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600072d0 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.757+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700a25e0 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.758+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa60007430 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.758+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffa6003e990 con 0x7ffa700a1340 2023-09-30T22:28:33.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.758+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70121e10 con 0x7ffa700a1340 2023-09-30T22:28:33.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.763+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa60017080 con 0x7ffa700a1340 2023-09-30T22:28:33.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.792+0000 7f26a8e3c700 1 Processor -- start 2023-09-30T22:28:33.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.793+0000 7f26a8e3c700 1 -- start start 2023-09-30T22:28:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a8e3c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a8e3c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 0x7f26a412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a8e3c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26a4134260 0x7f26a41304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a8e3c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a406b990 con 0x7f26a4134260 2023-09-30T22:28:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a8e3c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40a0590 con 0x7f26a4133420 2023-09-30T22:28:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a8e3c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26a409fb80 con 0x7f26a4133e30 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 0x7f26a412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41464/0 (socket says 172.21.15.32:41464) 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 0x7f26a412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53954/0 (socket says 172.21.15.32:53954) 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a259c700 1 -- 172.21.15.32:0/927278035 learned_addr learned my addr 172.21.15.32:0/927278035 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a2d9d700 1 --2- 172.21.15.32:0/927278035 >> v2:172.21.15.32:3300/0 conn(0x7f26a4134260 0x7f26a41304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 -- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 msgr2=0x7f26a41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 --2- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 -- 172.21.15.32:0/927278035 >> v2:172.21.15.32:3300/0 conn(0x7f26a4134260 msgr2=0x7f26a41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 --2- 172.21.15.32:0/927278035 >> v2:172.21.15.32:3300/0 conn(0x7f26a4134260 0x7f26a41304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 -- 172.21.15.32:0/927278035 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40a0030 con 0x7f26a4133e30 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a2d9d700 1 --2- 172.21.15.32:0/927278035 >> v2:172.21.15.32:3300/0 conn(0x7f26a4134260 0x7f26a41304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a259c700 1 --2- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.794+0000 7f26a1d9b700 1 --2- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 0x7f26a412ff90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2698009840 tx=0x7f2698009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7d3c1b27f6e74e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a159a700 1 -- 172.21.15.32:0/927278035 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011830 con 0x7f26a4133e30 2023-09-30T22:28:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a159a700 1 -- 172.21.15.32:0/927278035 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698011990 con 0x7f26a4133e30 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a159a700 1 -- 172.21.15.32:0/927278035 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698011d20 con 0x7f26a4133e30 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 -- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 msgr2=0x7f26a412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 0x7f26a412ff90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2698009840 tx=0x7f2698009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 -- 172.21.15.32:0/927278035 shutdown_connections 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/927278035 >> v2:172.21.15.32:3300/0 conn(0x7f26a4134260 0x7f26a41304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/927278035 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4133e30 0x7f26a412ff90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.795+0000 7f26a8e3c700 1 -- 172.21.15.32:0/927278035 >> 172.21.15.32:0/927278035 conn(0x7f26a40a4210 msgr2=0x7f26a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.796+0000 7f26a8e3c700 1 -- 172.21.15.32:0/927278035 shutdown_connections 2023-09-30T22:28:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.796+0000 7f26a8e3c700 1 -- 172.21.15.32:0/927278035 wait complete. 2023-09-30T22:28:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.797+0000 7f26a8e3c700 1 Processor -- start 2023-09-30T22:28:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.797+0000 7f26a8e3c700 1 -- start start 2023-09-30T22:28:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a8e3c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a8e3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a4133e30 0x7f26a409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a8e3c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 0x7f26a409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a8e3c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a412d750 con 0x7f26a4133e30 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a8e3c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a409e000 con 0x7f26a4133420 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a8e3c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26a411fd80 con 0x7f26a4134260 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 0x7f26a409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 0x7f26a409d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53970/0 (socket says 172.21.15.32:53970) 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a409c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41478/0 (socket says 172.21.15.32:41478) 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 -- 172.21.15.32:0/1765398618 learned_addr learned my addr 172.21.15.32:0/1765398618 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a1d9b700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a4133e30 0x7f26a409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 -- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 msgr2=0x7f26a409c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a409c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 -- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a4133e30 msgr2=0x7f26a409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a4133e30 0x7f26a409cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 -- 172.21.15.32:0/1765398618 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694009550 con 0x7f26a4134260 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a259c700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a409c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.798+0000 7f26a2d9d700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 0x7f26a409d160 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f269c00efc0 tx=0x7f269c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.799+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00ca90 con 0x7f26a4134260 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.799+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00f040 con 0x7f26a4134260 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.799+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c009a70 con 0x7f26a4134260 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.799+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00ce10 con 0x7f26a4134260 2023-09-30T22:28:33.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.799+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26980094a0 con 0x7f26a4134260 2023-09-30T22:28:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.800+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a409ef90 con 0x7f26a4134260 2023-09-30T22:28:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.800+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f269c014480 con 0x7f26a4134260 2023-09-30T22:28:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.800+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f269c03fd30 con 0x7f26a4134260 2023-09-30T22:28:33.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.805+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f269c012930 con 0x7f26a4134260 2023-09-30T22:28:33.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.867+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f00e80747f0 con 0x7f00e80a0a30 2023-09-30T22:28:33.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.868+0000 7f00deffd700 1 -- 172.21.15.32:0/609342308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f00d8011360 con 0x7f00e80a0a30 2023-09-30T22:28:33.868 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:33.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 msgr2=0x7f00e8133d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e8133d00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c550 tx=0x7f00d800c860 comp rx=0 tx=0).stop 2023-09-30T22:28:33.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 shutdown_connections 2023-09-30T22:28:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f00e8138d80 0x7f00e8134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00e809d630 0x7f00e81337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 --2- 172.21.15.32:0/609342308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00e80a0a30 0x7f00e8133d00 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 >> 172.21.15.32:0/609342308 conn(0x7f00e80a3050 msgr2=0x7f00e813aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 shutdown_connections 2023-09-30T22:28:33.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.870+0000 7f00f033a700 1 -- 172.21.15.32:0/609342308 wait complete. 2023-09-30T22:28:33.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:33.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.975+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa70074ea0 con 0x7ffa700a1340 2023-09-30T22:28:33.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.976+0000 7ffa667fc700 1 -- 172.21.15.32:0/885005660 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa600076e0 con 0x7ffa700a1340 2023-09-30T22:28:33.976 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa700a1340 msgr2=0x7ffa7011ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa700a1340 0x7ffa7011ed60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000c450 tx=0x7ffa6000c760 comp rx=0 tx=0).stop 2023-09-30T22:28:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 shutdown_connections 2023-09-30T22:28:33.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffa70134170 0x7ffa7011f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa700a0a30 0x7ffa7011e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 --2- 172.21.15.32:0/885005660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa700a1340 0x7ffa7011ed60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 >> 172.21.15.32:0/885005660 conn(0x7ffa700a6550 msgr2=0x7ffa701b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 shutdown_connections 2023-09-30T22:28:33.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:33.978+0000 7ffa78051700 1 -- 172.21.15.32:0/885005660 wait complete. 2023-09-30T22:28:33.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:34.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.012+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f26a4074ea0 con 0x7f26a4134260 2023-09-30T22:28:34.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.012+0000 7f2692ffd700 1 -- 172.21.15.32:0/1765398618 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f269c01c3b0 con 0x7f26a4134260 2023-09-30T22:28:34.012 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 msgr2=0x7f26a409d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 0x7f26a409d160 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f269c00efc0 tx=0x7f269c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 shutdown_connections 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26a4134260 0x7f26a409d160 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26a4133420 0x7f26a409c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 --2- 172.21.15.32:0/1765398618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26a4133e30 0x7f26a409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.014+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 >> 172.21.15.32:0/1765398618 conn(0x7f26a40a4210 msgr2=0x7f26a40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.015+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 shutdown_connections 2023-09-30T22:28:34.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.015+0000 7f26a8e3c700 1 -- 172.21.15.32:0/1765398618 wait complete. 2023-09-30T22:28:34.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:34.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:34.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.647+0000 7f1da6ff6700 1 Processor -- start 2023-09-30T22:28:34.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.647+0000 7f1da6ff6700 1 -- start start 2023-09-30T22:28:34.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da6ff6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 0x7f1da0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:34.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da6ff6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:34.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da6ff6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da006acb0 0x7f1da006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:34.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da6ff6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da009e130 con 0x7f1da01316c0 2023-09-30T22:28:34.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da6ff6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da01312b0 con 0x7f1da009d9f0 2023-09-30T22:28:34.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da6ff6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1da009f6e0 con 0x7f1da006acb0 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da4d92700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 0x7f1da0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da4d92700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 0x7f1da0131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51192/0 (socket says 172.21.15.32:51192) 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da4d92700 1 -- 172.21.15.32:0/1447806100 learned_addr learned my addr 172.21.15.32:0/1447806100 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da006acb0 0x7f1da006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 -- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 msgr2=0x7f1da006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 -- 172.21.15.32:0/1447806100 >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 msgr2=0x7f1da0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 --2- 172.21.15.32:0/1447806100 >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 0x7f1da0131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 -- 172.21.15.32:0/1447806100 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da009e760 con 0x7f1da006acb0 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da4d92700 1 --2- 172.21.15.32:0/1447806100 >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 0x7f1da0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.648+0000 7f1da5593700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da006acb0 0x7f1da006b150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009d90 tx=0x7f1d9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15c3bbdd73ef693 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1d9f7fe700 1 -- 172.21.15.32:0/1447806100 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800e070 con 0x7f1da006acb0 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1d9f7fe700 1 -- 172.21.15.32:0/1447806100 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9800bb20 con 0x7f1da006acb0 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1d9f7fe700 1 -- 172.21.15.32:0/1447806100 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98012430 con 0x7f1da006acb0 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 -- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da006acb0 msgr2=0x7f1da006b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da006acb0 0x7f1da006b150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009d90 tx=0x7f1d9800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:34.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 -- 172.21.15.32:0/1447806100 shutdown_connections 2023-09-30T22:28:34.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da006acb0 0x7f1da006b150 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/1447806100 >> v2:172.21.15.32:3300/0 conn(0x7f1da01316c0 0x7f1da0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/1447806100 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 -- 172.21.15.32:0/1447806100 >> 172.21.15.32:0/1447806100 conn(0x7f1da0066f50 msgr2=0x7f1da009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:34.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 -- 172.21.15.32:0/1447806100 shutdown_connections 2023-09-30T22:28:34.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.649+0000 7f1da6ff6700 1 -- 172.21.15.32:0/1447806100 wait complete. 2023-09-30T22:28:34.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.650+0000 7f1da6ff6700 1 Processor -- start 2023-09-30T22:28:34.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.650+0000 7f1da6ff6700 1 -- start start 2023-09-30T22:28:34.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da6ff6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da006acb0 0x7f1da0070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da6ff6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da6ff6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da01316c0 0x7f1da0071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da6ff6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0128290 con 0x7f1da006acb0 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da6ff6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00726d0 con 0x7f1da009d9f0 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da6ff6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1da011a830 con 0x7f1da01316c0 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da4d92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da006acb0 0x7f1da0070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da4d92700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da006acb0 0x7f1da0070ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51204/0 (socket says 172.21.15.32:51204) 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da4d92700 1 -- 172.21.15.32:0/333340689 learned_addr learned my addr 172.21.15.32:0/333340689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:34.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.651+0000 7f1da5593700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da01316c0 0x7f1da0071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9ffff700 1 -- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da01316c0 msgr2=0x7f1da0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da01316c0 0x7f1da0071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9ffff700 1 -- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da006acb0 msgr2=0x7f1da0070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da006acb0 0x7f1da0070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9ffff700 1 -- 172.21.15.32:0/333340689 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940094a0 con 0x7f1da009d9f0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da0070ff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1d94010cc0 tx=0x7f1d9400b140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94015070 con 0x7f1da009d9f0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9400b9a0 con 0x7f1da009d9f0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90009550 con 0x7f1da009d9f0 2023-09-30T22:28:34.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400bd30 con 0x7f1da009d9f0 2023-09-30T22:28:34.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.652+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d98009a70 con 0x7f1da009d9f0 2023-09-30T22:28:34.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.653+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d94019410 con 0x7f1da009d9f0 2023-09-30T22:28:34.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.653+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d94042f50 con 0x7f1da009d9f0 2023-09-30T22:28:34.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.653+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00a3c20 con 0x7f1da009d9f0 2023-09-30T22:28:34.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.658+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d9401e020 con 0x7f1da009d9f0 2023-09-30T22:28:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.861+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1da0074ea0 con 0x7f1da009d9f0 2023-09-30T22:28:34.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.861+0000 7f1d9d7fa700 1 -- 172.21.15.32:0/333340689 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d94017900 con 0x7f1da009d9f0 2023-09-30T22:28:34.862 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.863+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 msgr2=0x7f1da0070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.863+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da0070ff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1d94010cc0 tx=0x7f1d9400b140 comp rx=0 tx=0).stop 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 shutdown_connections 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da01316c0 0x7f1da0071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da006acb0 0x7f1da0070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 --2- 172.21.15.32:0/333340689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da009d9f0 0x7f1da0070ff0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 >> 172.21.15.32:0/333340689 conn(0x7f1da0066f50 msgr2=0x7f1da012b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:34.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 shutdown_connections 2023-09-30T22:28:34.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:34.864+0000 7f1da6ff6700 1 -- 172.21.15.32:0/333340689 wait complete. 2023-09-30T22:28:34.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:34.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:34.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.023+0000 7fdaf7136700 1 Processor -- start 2023-09-30T22:28:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.023+0000 7fdaf7136700 1 -- start start 2023-09-30T22:28:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf7136700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf0136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf7136700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf0139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf7136700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf009ece0 con 0x7fdaf006a770 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf7136700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf006b640 con 0x7fdaf0136d80 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf7136700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf009e760 con 0x7fdaf006b250 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdae7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf0136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf4ed2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdaf006a770 0x7fdaf006ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33582/0 (socket says 172.21.15.32:33582) 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf4ed2700 1 -- 172.21.15.32:0/1461482040 learned_addr learned my addr 172.21.15.32:0/1461482040 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdaf56d3700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf0139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.024+0000 7fdae77fe700 1 -- 172.21.15.32:0/1461482040 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158047369 0 0) 0x7fdaf009ece0 con 0x7fdaf006a770 2023-09-30T22:28:35.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae7fff700 1 -- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 msgr2=0x7fdaf0139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae7fff700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf0139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae7fff700 1 -- 172.21.15.32:0/1461482040 >> v1:172.21.15.32:6789/0 conn(0x7fdaf006a770 legacy=0x7fdaf006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae7fff700 1 -- 172.21.15.32:0/1461482040 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf009e040 con 0x7fdaf006b250 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdaf56d3700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf0139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae7fff700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf0136840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c0bc53cc19376c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae77fe700 1 -- 172.21.15.32:0/1461482040 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0011830 con 0x7fdaf006b250 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae77fe700 1 -- 172.21.15.32:0/1461482040 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0011990 con 0x7fdaf006b250 2023-09-30T22:28:35.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.025+0000 7fdae77fe700 1 -- 172.21.15.32:0/1461482040 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0011d20 con 0x7fdaf006b250 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 -- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 msgr2=0x7fdaf0136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf0136840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae0009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 -- 172.21.15.32:0/1461482040 shutdown_connections 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf0139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 --2- 172.21.15.32:0/1461482040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf0136840 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 -- 172.21.15.32:0/1461482040 >> 172.21.15.32:0/1461482040 conn(0x7fdaf0066f50 msgr2=0x7fdaf009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 -- 172.21.15.32:0/1461482040 shutdown_connections 2023-09-30T22:28:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.026+0000 7fdaf7136700 1 -- 172.21.15.32:0/1461482040 wait complete. 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.027+0000 7fdaf7136700 1 Processor -- start 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.027+0000 7fdaf7136700 1 -- start start 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.027+0000 7fdaf7136700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 0x7fdaf01c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf7136700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf01c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf7136700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf01c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf7136700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0071e60 con 0x7fdaf006a770 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf7136700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0124fd0 con 0x7fdaf0136d80 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf7136700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0136590 con 0x7fdaf006b250 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf56d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf01c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 0x7fdaf01c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf56d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf01c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41522/0 (socket says 172.21.15.32:41522) 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf56d3700 1 -- 172.21.15.32:0/2588241346 learned_addr learned my addr 172.21.15.32:0/2588241346 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 0x7fdaf01c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51208/0 (socket says 172.21.15.32:51208) 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdae7fff700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf01c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 -- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 msgr2=0x7fdaf01c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf01c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 -- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 msgr2=0x7fdaf01c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf01c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 -- 172.21.15.32:0/2588241346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8009a70 con 0x7fdaf006a770 2023-09-30T22:28:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf56d3700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf01c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.028+0000 7fdaf4ed2700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 0x7fdaf01c82a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c3c0 tx=0x7fdaec00c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.029+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004a70 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.029+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec004bf0 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.029+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae00094a0 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.029+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0070d0 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.029+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf0125d00 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.029+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdaec007230 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.030+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdaec03f170 con 0x7fdaf006a770 2023-09-30T22:28:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.030+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf009f130 con 0x7fdaf006a770 2023-09-30T22:28:35.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:35.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.035+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdaec0074e0 con 0x7fdaf006a770 2023-09-30T22:28:35.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.134+0000 7f1796a05700 1 Processor -- start 2023-09-30T22:28:35.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.134+0000 7f1796a05700 1 -- start start 2023-09-30T22:28:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f1796a05700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 0x7f179009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f1796a05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179009cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f1796a05700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1790131970 0x7f1790131df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f1796a05700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179009ba70 con 0x7f179009c650 2023-09-30T22:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f1796a05700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179009d4a0 con 0x7f1790131970 2023-09-30T22:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f1796a05700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f179009e8f0 con 0x7f17900a0090 2023-09-30T22:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.135+0000 7f178ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 0x7f179009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1790131970 0x7f1790131df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 0x7f179009ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51218/0 (socket says 172.21.15.32:51218) 2023-09-30T22:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178ffff700 1 -- 172.21.15.32:0/2264289224 learned_addr learned my addr 172.21.15.32:0/2264289224 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178f7fe700 1 --2- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179009cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 -- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 msgr2=0x7f179009cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 --2- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179009cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 -- 172.21.15.32:0/2264289224 >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 msgr2=0x7f179009ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 --2- 172.21.15.32:0/2264289224 >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 0x7f179009ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 -- 172.21.15.32:0/2264289224 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900a0760 con 0x7f1790131970 2023-09-30T22:28:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178ffff700 1 --2- 172.21.15.32:0/2264289224 >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 0x7f179009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f1794fa2700 1 --2- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1790131970 0x7f1790131df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f178800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d375c05e6185d6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178effd700 1 -- 172.21.15.32:0/2264289224 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800e070 con 0x7f1790131970 2023-09-30T22:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178effd700 1 -- 172.21.15.32:0/2264289224 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178800bb20 con 0x7f1790131970 2023-09-30T22:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.136+0000 7f178effd700 1 -- 172.21.15.32:0/2264289224 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788012430 con 0x7f1790131970 2023-09-30T22:28:35.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 -- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1790131970 msgr2=0x7f1790131df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 --2- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1790131970 0x7f1790131df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d90 tx=0x7f178800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:35.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 -- 172.21.15.32:0/2264289224 shutdown_connections 2023-09-30T22:28:35.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 --2- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1790131970 0x7f1790131df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 --2- 172.21.15.32:0/2264289224 >> v2:172.21.15.32:3300/0 conn(0x7f179009c650 0x7f179009ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 --2- 172.21.15.32:0/2264289224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179009cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 -- 172.21.15.32:0/2264289224 >> 172.21.15.32:0/2264289224 conn(0x7f17900a3f70 msgr2=0x7f17900a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 -- 172.21.15.32:0/2264289224 shutdown_connections 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.137+0000 7f1796a05700 1 -- 172.21.15.32:0/2264289224 wait complete. 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.138+0000 7f1796a05700 1 Processor -- start 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 -- start start 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 0x7f179006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790131970 0x7f179006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179006d940 con 0x7f1790131970 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179011f500 con 0x7f179009c650 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1796a05700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f179009d980 con 0x7f17900a0090 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f178ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 0x7f179006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f1794fa2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790131970 0x7f179006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f178ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 0x7f179006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41544/0 (socket says 172.21.15.32:41544) 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f178ffff700 1 -- 172.21.15.32:0/808411205 learned_addr learned my addr 172.21.15.32:0/808411205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.139+0000 7f178f7fe700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178ffff700 1 -- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 msgr2=0x7f179006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178ffff700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178ffff700 1 -- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790131970 msgr2=0x7f179006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178ffff700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790131970 0x7f179006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178ffff700 1 -- 172.21.15.32:0/808411205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009550 con 0x7f179009c650 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f1794fa2700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790131970 0x7f179006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178f7fe700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178ffff700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 0x7f179006b880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f178000ec10 tx=0x7f178000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178000c850 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178000c9b0 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178000cd40 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1788009a70 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.140+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179011f240 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.142+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17900747f0 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.142+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1780013470 con 0x7f179009c650 2023-09-30T22:28:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.142+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f178003f950 con 0x7f179009c650 2023-09-30T22:28:35.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.147+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1780013720 con 0x7f179009c650 2023-09-30T22:28:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.178+0000 7fe714d9d700 1 Processor -- start 2023-09-30T22:28:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.178+0000 7fe714d9d700 1 -- start start 2023-09-30T22:28:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe714d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009d6f0 0x7fe71009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe714d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 0x7fe710130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe714d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe71009e410 0x7fe710130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe714d9d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71009f520 con 0x7fe71009e410 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe714d9d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71009f130 con 0x7fe71009d6f0 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe714d9d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe71009e930 con 0x7fe71009e000 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 0x7fe710130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 0x7fe710130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54036/0 (socket says 172.21.15.32:54036) 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 -- 172.21.15.32:0/1233528268 learned_addr learned my addr 172.21.15.32:0/1233528268 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70e59c700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009d6f0 0x7fe71009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70ed9d700 1 --2- 172.21.15.32:0/1233528268 >> v2:172.21.15.32:3300/0 conn(0x7fe71009e410 0x7fe710130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 -- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009d6f0 msgr2=0x7fe71009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009d6f0 0x7fe71009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 -- 172.21.15.32:0/1233528268 >> v2:172.21.15.32:3300/0 conn(0x7fe71009e410 msgr2=0x7fe710130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 --2- 172.21.15.32:0/1233528268 >> v2:172.21.15.32:3300/0 conn(0x7fe71009e410 0x7fe710130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70dd9b700 1 -- 172.21.15.32:0/1233528268 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71009b2d0 con 0x7fe71009e000 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70ed9d700 1 --2- 172.21.15.32:0/1233528268 >> v2:172.21.15.32:3300/0 conn(0x7fe71009e410 0x7fe710130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.179+0000 7fe70e59c700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009d6f0 0x7fe71009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.180+0000 7fe70dd9b700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 0x7fe710130660 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe70400b9d0 tx=0x7fe70400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101f1ad113b7059a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.180+0000 7fe70d59a700 1 -- 172.21.15.32:0/1233528268 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400cc20 con 0x7fe71009e000 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.180+0000 7fe70d59a700 1 -- 172.21.15.32:0/1233528268 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70400cd80 con 0x7fe71009e000 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.180+0000 7fe70d59a700 1 -- 172.21.15.32:0/1233528268 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400f680 con 0x7fe71009e000 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.180+0000 7fe714d9d700 1 -- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 msgr2=0x7fe710130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.180+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 0x7fe710130660 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe70400b9d0 tx=0x7fe70400bce0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 -- 172.21.15.32:0/1233528268 shutdown_connections 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009d6f0 0x7fe71009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1233528268 >> v2:172.21.15.32:3300/0 conn(0x7fe71009e410 0x7fe710130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1233528268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009e000 0x7fe710130660 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 -- 172.21.15.32:0/1233528268 >> 172.21.15.32:0/1233528268 conn(0x7fe710066f50 msgr2=0x7fe71009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 -- 172.21.15.32:0/1233528268 shutdown_connections 2023-09-30T22:28:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.181+0000 7fe714d9d700 1 -- 172.21.15.32:0/1233528268 wait complete. 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.182+0000 7fe714d9d700 1 Processor -- start 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 -- start start 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009d6f0 0x7fe71011e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 0x7fe71011e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71009e410 0x7fe71011eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710120590 con 0x7fe71009e410 2023-09-30T22:28:35.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71009c700 con 0x7fe71009e000 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe714d9d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe710120270 con 0x7fe71009d6f0 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe70dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 0x7fe71011e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe70ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71009e410 0x7fe71011eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe70dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 0x7fe71011e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41566/0 (socket says 172.21.15.32:41566) 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.183+0000 7fe70dd9b700 1 -- 172.21.15.32:0/1290196768 learned_addr learned my addr 172.21.15.32:0/1290196768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70e59c700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009d6f0 0x7fe71011e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70ed9d700 1 -- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009d6f0 msgr2=0x7fe71011e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70ed9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009d6f0 0x7fe71011e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70ed9d700 1 -- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 msgr2=0x7fe71011e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70ed9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 0x7fe71011e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70ed9d700 1 -- 172.21.15.32:0/1290196768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000094a0 con 0x7fe71009e410 2023-09-30T22:28:35.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70dd9b700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 0x7fe71011e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe70ed9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71009e410 0x7fe71011eb30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe708011700 tx=0x7fe708011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708003a20 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708003ba0 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70800bc50 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.184+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708010680 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.185+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70400b600 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.185+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe7080107e0 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.186+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe710121540 con 0x7fe71009e410 2023-09-30T22:28:35.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.186+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe708044980 con 0x7fe71009e410 2023-09-30T22:28:35.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.191+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe70800fcc0 con 0x7fe71009e410 2023-09-30T22:28:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.250+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdaf009f6b0 con 0x7fdaf006a770 2023-09-30T22:28:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.250+0000 7fdae57fa700 1 -- 172.21.15.32:0/2588241346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdaec01b340 con 0x7fdaf006a770 2023-09-30T22:28:35.250 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.252+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 msgr2=0x7fdaf01c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.252+0000 7fdaf7136700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 0x7fdaf01c82a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c3c0 tx=0x7fdaec00c6d0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 shutdown_connections 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdaf0136d80 0x7fdaf01c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaf006a770 0x7fdaf01c82a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 --2- 172.21.15.32:0/2588241346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdaf006b250 0x7fdaf01c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 >> 172.21.15.32:0/2588241346 conn(0x7fdaf0066f50 msgr2=0x7fdaf00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 shutdown_connections 2023-09-30T22:28:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.253+0000 7fdaf7136700 1 -- 172.21.15.32:0/2588241346 wait complete. 2023-09-30T22:28:35.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.356+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1790072840 con 0x7f179009c650 2023-09-30T22:28:35.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.357+0000 7f178cff9700 1 -- 172.21.15.32:0/808411205 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1780021020 con 0x7f179009c650 2023-09-30T22:28:35.357 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 msgr2=0x7f179006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 0x7f179006b880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f178000ec10 tx=0x7f178000ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 shutdown_connections 2023-09-30T22:28:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790131970 0x7f179006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f179009c650 0x7f179006b880 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 --2- 172.21.15.32:0/808411205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17900a0090 0x7f179006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 >> 172.21.15.32:0/808411205 conn(0x7f17900a3f70 msgr2=0x7f179012ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 shutdown_connections 2023-09-30T22:28:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.359+0000 7f1796a05700 1 -- 172.21.15.32:0/808411205 wait complete. 2023-09-30T22:28:35.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:35.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.398+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe710074ea0 con 0x7fe71009e410 2023-09-30T22:28:35.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.398+0000 7fe6feffd700 1 -- 172.21.15.32:0/1290196768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe70800fae0 con 0x7fe71009e410 2023-09-30T22:28:35.398 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71009e410 msgr2=0x7fe71011eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71009e410 0x7fe71011eb30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe708011700 tx=0x7fe708011a10 comp rx=0 tx=0).stop 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 shutdown_connections 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe71009e410 0x7fe71011eb30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe71009d6f0 0x7fe71011e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 --2- 172.21.15.32:0/1290196768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe71009e000 0x7fe71011e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.400+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 >> 172.21.15.32:0/1290196768 conn(0x7fe710066f50 msgr2=0x7fe71006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.401+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 shutdown_connections 2023-09-30T22:28:35.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:35.401+0000 7fe714d9d700 1 -- 172.21.15.32:0/1290196768 wait complete. 2023-09-30T22:28:35.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:35.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:36.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.025+0000 7f26d3190700 1 Processor -- start 2023-09-30T22:28:36.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 -- start start 2023-09-30T22:28:36.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26cc09e000 0x7f26cc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 0x7f26cc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc09f520 con 0x7f26cc09e000 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc09f130 con 0x7f26cc09e410 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d3190700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc09e930 con 0x7f26cc09d6f0 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d0f2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d172d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 0x7f26cc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26cbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26cc09e000 0x7f26cc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d172d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 0x7f26cc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41574/0 (socket says 172.21.15.32:41574) 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d0f2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54060/0 (socket says 172.21.15.32:54060) 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.026+0000 7f26d172d700 1 -- 172.21.15.32:0/3456358521 learned_addr learned my addr 172.21.15.32:0/3456358521 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cbfff700 1 -- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 msgr2=0x7f26cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cbfff700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cbfff700 1 -- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 msgr2=0x7f26cc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cbfff700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 0x7f26cc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cbfff700 1 -- 172.21.15.32:0/3456358521 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc09b2d0 con 0x7f26cc09e000 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26d172d700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 0x7f26cc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26d0f2c700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cbfff700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e000 0x7f26cc130660 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009870 tx=0x7f26b4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eb19319bdb497e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cb7fe700 1 -- 172.21.15.32:0/3456358521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011830 con 0x7f26cc09e000 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cb7fe700 1 -- 172.21.15.32:0/3456358521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b4011990 con 0x7f26cc09e000 2023-09-30T22:28:36.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.027+0000 7f26cb7fe700 1 -- 172.21.15.32:0/3456358521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4011d20 con 0x7f26cc09e000 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 -- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e000 msgr2=0x7f26cc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e000 0x7f26cc130660 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009870 tx=0x7f26b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 -- 172.21.15.32:0/3456358521 shutdown_connections 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e410 0x7f26cc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 --2- 172.21.15.32:0/3456358521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e000 0x7f26cc130660 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 -- 172.21.15.32:0/3456358521 >> 172.21.15.32:0/3456358521 conn(0x7f26cc066f50 msgr2=0x7f26cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 -- 172.21.15.32:0/3456358521 shutdown_connections 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.028+0000 7f26d3190700 1 -- 172.21.15.32:0/3456358521 wait complete. 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 Processor -- start 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 -- start start 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc1c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e000 0x7f26cc1c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e410 0x7f26cc1c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1258c0 con 0x7f26cc09e410 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc120140 con 0x7f26cc09e000 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.029+0000 7f26d3190700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc09c700 con 0x7f26cc09d6f0 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc1c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d172d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e410 0x7f26cc1c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc1c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54070/0 (socket says 172.21.15.32:54070) 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e000 0x7f26cc1c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 -- 172.21.15.32:0/1320568342 learned_addr learned my addr 172.21.15.32:0/1320568342 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 -- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e000 msgr2=0x7f26cc1c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e000 0x7f26cc1c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 -- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e410 msgr2=0x7f26cc1c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e410 0x7f26cc1c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 -- 172.21.15.32:0/1320568342 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0094a0 con 0x7f26cc09d6f0 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d0f2c700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc1c8480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00ec10 tx=0x7f26bc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc009780 con 0x7f26cc09d6f0 2023-09-30T22:28:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc00c850 con 0x7f26cc09d6f0 2023-09-30T22:28:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c4009b30 con 0x7f26cc09d6f0 2023-09-30T22:28:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00cbe0 con 0x7f26cc09d6f0 2023-09-30T22:28:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.030+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b40094a0 con 0x7f26cc09d6f0 2023-09-30T22:28:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.031+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f26bc00cd40 con 0x7f26cc09d6f0 2023-09-30T22:28:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.031+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f26bc03f720 con 0x7f26cc09d6f0 2023-09-30T22:28:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.031+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc120320 con 0x7f26cc09d6f0 2023-09-30T22:28:36.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.036+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f26bc0232f0 con 0x7f26cc09d6f0 2023-09-30T22:28:36.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.241+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f26cc074ea0 con 0x7f26cc09d6f0 2023-09-30T22:28:36.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.242+0000 7f26c97fa700 1 -- 172.21.15.32:0/1320568342 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26bc01bb50 con 0x7f26cc09d6f0 2023-09-30T22:28:36.242 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 msgr2=0x7f26cc1c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc1c8480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00ec10 tx=0x7f26bc00ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 shutdown_connections 2023-09-30T22:28:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26cc09e410 0x7f26cc1c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26cc09d6f0 0x7f26cc1c8480 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 --2- 172.21.15.32:0/1320568342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f26cc09e000 0x7f26cc1c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 >> 172.21.15.32:0/1320568342 conn(0x7f26cc066f50 msgr2=0x7f26cc119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 shutdown_connections 2023-09-30T22:28:36.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.244+0000 7f26d3190700 1 -- 172.21.15.32:0/1320568342 wait complete. 2023-09-30T22:28:36.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:36.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:36.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:36.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.410+0000 7fd67c041700 1 Processor -- start 2023-09-30T22:28:36.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.410+0000 7fd67c041700 1 -- start start 2023-09-30T22:28:36.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67c041700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409d6f0 0x7fd67409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67c041700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd67409e000 0x7fd6741339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67c041700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67409d2f0 con 0x7fd674133f30 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67c041700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67409a810 con 0x7fd67409d6f0 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67c041700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd67409a470 con 0x7fd67409e000 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd6795dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd67409e000 0x7fd6741339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67a5de700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd674133f30 0x7fd67413ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33590/0 (socket says 172.21.15.32:33590) 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd679ddd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409d6f0 0x7fd67409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd67a5de700 1 -- 172.21.15.32:0/2648224622 learned_addr learned my addr 172.21.15.32:0/2648224622 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd6795dc700 1 -- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409d6f0 msgr2=0x7fd67409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd678ddb700 1 -- 172.21.15.32:0/2648224622 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747552549 0 0) 0x7fd67409d2f0 con 0x7fd674133f30 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd6795dc700 1 --2- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409d6f0 0x7fd67409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd6795dc700 1 -- 172.21.15.32:0/2648224622 >> v1:172.21.15.32:6789/0 conn(0x7fd674133f30 legacy=0x7fd67413ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd6795dc700 1 -- 172.21.15.32:0/2648224622 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67409ec40 con 0x7fd67409e000 2023-09-30T22:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd679ddd700 1 --2- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409d6f0 0x7fd67409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.411+0000 7fd6795dc700 1 --2- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd67409e000 0x7fd6741339f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd664009870 tx=0x7fd664009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5afd883189fc16bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd678ddb700 1 -- 172.21.15.32:0/2648224622 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664011830 con 0x7fd67409e000 2023-09-30T22:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd678ddb700 1 -- 172.21.15.32:0/2648224622 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664011990 con 0x7fd67409e000 2023-09-30T22:28:36.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd678ddb700 1 -- 172.21.15.32:0/2648224622 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664011d20 con 0x7fd67409e000 2023-09-30T22:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd67c041700 1 -- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd67409e000 msgr2=0x7fd6741339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd67c041700 1 --2- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd67409e000 0x7fd6741339f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd664009870 tx=0x7fd664009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd67c041700 1 -- 172.21.15.32:0/2648224622 shutdown_connections 2023-09-30T22:28:36.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd67c041700 1 --2- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409d6f0 0x7fd67409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd67c041700 1 --2- 172.21.15.32:0/2648224622 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd67409e000 0x7fd6741339f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.412+0000 7fd67c041700 1 -- 172.21.15.32:0/2648224622 >> 172.21.15.32:0/2648224622 conn(0x7fd674066f50 msgr2=0x7fd67409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.413+0000 7fd67c041700 1 -- 172.21.15.32:0/2648224622 shutdown_connections 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.413+0000 7fd67c041700 1 -- 172.21.15.32:0/2648224622 wait complete. 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.413+0000 7fd67c041700 1 Processor -- start 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 -- start start 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 0x7fd674122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 0x7fd674122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd674133f30 0x7fd6741231f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd674133730 con 0x7fd67409d6f0 2023-09-30T22:28:36.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd674124490 con 0x7fd67409e000 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67c041700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd67409c640 con 0x7fd674133f30 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd679ddd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 0x7fd674122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd6795dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 0x7fd674122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd679ddd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 0x7fd674122770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51258/0 (socket says 172.21.15.32:51258) 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd679ddd700 1 -- 172.21.15.32:0/1468591513 learned_addr learned my addr 172.21.15.32:0/1468591513 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd6795dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 0x7fd674122cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41604/0 (socket says 172.21.15.32:41604) 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.414+0000 7fd67a5de700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd674133f30 0x7fd6741231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd6795dc700 1 -- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd674133f30 msgr2=0x7fd6741231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd6795dc700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd674133f30 0x7fd6741231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd6795dc700 1 -- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 msgr2=0x7fd674122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd6795dc700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 0x7fd674122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd6795dc700 1 -- 172.21.15.32:0/1468591513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700094a0 con 0x7fd67409e000 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd679ddd700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 0x7fd674122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd6795dc700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 0x7fd674122cb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd664016000 tx=0x7fd664004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664004300 con 0x7fd67409e000 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66401f070 con 0x7fd67409e000 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6640094a0 con 0x7fd67409e000 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6640036a0 con 0x7fd67409e000 2023-09-30T22:28:36.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.415+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd674126a20 con 0x7fd67409e000 2023-09-30T22:28:36.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.416+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd664021030 con 0x7fd67409e000 2023-09-30T22:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.416+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd664040640 con 0x7fd67409e000 2023-09-30T22:28:36.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.416+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd674074ea0 con 0x7fd67409e000 2023-09-30T22:28:36.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.421+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd664015020 con 0x7fd67409e000 2023-09-30T22:28:36.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 Processor -- start 2023-09-30T22:28:36.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 -- start start 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 0x7fbab8133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbab809f5b0 0x7fbab8134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab806bf30 con 0x7fbab809f5b0 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab809ffb0 con 0x7fbab809f160 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.525+0000 7fbabf747700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbab806cb10 con 0x7fbab809e850 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 0x7fbab8133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabd4e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 0x7fbab8133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41608/0 (socket says 172.21.15.32:41608) 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabdce4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbab809f5b0 0x7fbab8134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 -- 172.21.15.32:0/355858338 learned_addr learned my addr 172.21.15.32:0/355858338 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabd4e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54098/0 (socket says 172.21.15.32:54098) 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 -- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 msgr2=0x7fbab809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 --2- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 -- 172.21.15.32:0/355858338 >> v2:172.21.15.32:3300/0 conn(0x7fbab809f5b0 msgr2=0x7fbab8134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 --2- 172.21.15.32:0/355858338 >> v2:172.21.15.32:3300/0 conn(0x7fbab809f5b0 0x7fbab8134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 -- 172.21.15.32:0/355858338 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab80a0270 con 0x7fbab809f160 2023-09-30T22:28:36.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabdce4700 1 --2- 172.21.15.32:0/355858338 >> v2:172.21.15.32:3300/0 conn(0x7fbab809f5b0 0x7fbab8134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbabcce2700 1 --2- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 0x7fbab8133bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098f0 tx=0x7fbaa8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1ecf023ad99fe3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.526+0000 7fbaaffff700 1 -- 172.21.15.32:0/355858338 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8011830 con 0x7fbab809f160 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbaaffff700 1 -- 172.21.15.32:0/355858338 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa8011990 con 0x7fbab809f160 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbaaffff700 1 -- 172.21.15.32:0/355858338 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8011d20 con 0x7fbab809f160 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 -- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 msgr2=0x7fbab8133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 --2- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 0x7fbab8133bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098f0 tx=0x7fbaa8009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 -- 172.21.15.32:0/355858338 shutdown_connections 2023-09-30T22:28:36.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 --2- 172.21.15.32:0/355858338 >> v2:172.21.15.32:3300/0 conn(0x7fbab809f5b0 0x7fbab8134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 --2- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 --2- 172.21.15.32:0/355858338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f160 0x7fbab8133bf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 -- 172.21.15.32:0/355858338 >> 172.21.15.32:0/355858338 conn(0x7fbab80a3f70 msgr2=0x7fbab80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 -- 172.21.15.32:0/355858338 shutdown_connections 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.527+0000 7fbabf747700 1 -- 172.21.15.32:0/355858338 wait complete. 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.528+0000 7fbabf747700 1 Processor -- start 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.528+0000 7fbabf747700 1 -- start start 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabf747700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab81227b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabf747700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 0x7fbab8122cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabf747700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f5b0 0x7fbab8123230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabf747700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8125d30 con 0x7fbab809f160 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabf747700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab81263d0 con 0x7fbab809f5b0 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabf747700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbab81243c0 con 0x7fbab809e850 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabcce2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 0x7fbab8122cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabd4e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab81227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabdce4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f5b0 0x7fbab8123230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabcce2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 0x7fbab8122cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51268/0 (socket says 172.21.15.32:51268) 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabcce2700 1 -- 172.21.15.32:0/729781813 learned_addr learned my addr 172.21.15.32:0/729781813 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabd4e3700 1 -- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f5b0 msgr2=0x7fbab8123230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabd4e3700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f5b0 0x7fbab8123230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabd4e3700 1 -- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 msgr2=0x7fbab8122cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabd4e3700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 0x7fbab8122cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabd4e3700 1 -- 172.21.15.32:0/729781813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0009a70 con 0x7fbab809e850 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.529+0000 7fbabcce2700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 0x7fbab8122cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.530+0000 7fbabd4e3700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab81227b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbab400e900 tx=0x7fbab400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.530+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400ee50 con 0x7fbab809e850 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.530+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab400c850 con 0x7fbab809e850 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.530+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa8009550 con 0x7fbab809e850 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.530+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400cbc0 con 0x7fbab809e850 2023-09-30T22:28:36.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.530+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab806cd30 con 0x7fbab809e850 2023-09-30T22:28:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.531+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbab400cd20 con 0x7fbab809e850 2023-09-30T22:28:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.531+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbab403fa80 con 0x7fbab809e850 2023-09-30T22:28:36.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.532+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab80747f0 con 0x7fbab809e850 2023-09-30T22:28:36.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.536+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbab401a960 con 0x7fbab809e850 2023-09-30T22:28:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.561+0000 7f5ef7c39700 1 Processor -- start 2023-09-30T22:28:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.561+0000 7f5ef7c39700 1 -- start start 2023-09-30T22:28:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef7c39700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ef009d630 0x7f5ef00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef7c39700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef0138d80 0x7f5ef013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef7c39700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00a2a70 con 0x7f5ef00a0a30 2023-09-30T22:28:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef7c39700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef00a01d0 con 0x7f5ef009d630 2023-09-30T22:28:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef7c39700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef009e2b0 con 0x7f5ef0138d80 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef61d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef0138d80 0x7f5ef013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef59d5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ef00a0a30 0x7f5ef00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33606/0 (socket says 172.21.15.32:33606) 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef59d5700 1 -- 172.21.15.32:0/3778274291 learned_addr learned my addr 172.21.15.32:0/3778274291 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef51d4700 1 --2- 172.21.15.32:0/3778274291 >> v2:172.21.15.32:3300/0 conn(0x7f5ef009d630 0x7f5ef00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef49d3700 1 -- 172.21.15.32:0/3778274291 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431500093 0 0) 0x7f5ef00a2a70 con 0x7f5ef00a0a30 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef49d3700 1 -- 172.21.15.32:0/3778274291 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5edc0035b0 con 0x7f5ef00a0a30 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef51d4700 1 -- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef0138d80 msgr2=0x7f5ef013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef51d4700 1 --2- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef0138d80 0x7f5ef013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef51d4700 1 -- 172.21.15.32:0/3778274291 >> v1:172.21.15.32:6789/0 conn(0x7f5ef00a0a30 legacy=0x7f5ef00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef51d4700 1 -- 172.21.15.32:0/3778274291 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef009dc50 con 0x7f5ef009d630 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef61d6700 1 --2- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef0138d80 0x7f5ef013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.562+0000 7f5ef51d4700 1 --2- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef009d630 0x7f5ef00a1510 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009840 tx=0x7f5ee0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85deda4fbc26aee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef49d3700 1 -- 172.21.15.32:0/3778274291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011830 con 0x7f5ef009d630 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef49d3700 1 -- 172.21.15.32:0/3778274291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee0011990 con 0x7f5ef009d630 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef49d3700 1 -- 172.21.15.32:0/3778274291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011d20 con 0x7f5ef009d630 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 -- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef009d630 msgr2=0x7f5ef00a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef009d630 0x7f5ef00a1510 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009840 tx=0x7f5ee0009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 -- 172.21.15.32:0/3778274291 shutdown_connections 2023-09-30T22:28:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef0138d80 0x7f5ef013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/3778274291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef009d630 0x7f5ef00a1510 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 -- 172.21.15.32:0/3778274291 >> 172.21.15.32:0/3778274291 conn(0x7f5ef00a3050 msgr2=0x7f5ef00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 -- 172.21.15.32:0/3778274291 shutdown_connections 2023-09-30T22:28:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.563+0000 7f5ef7c39700 1 -- 172.21.15.32:0/3778274291 wait complete. 2023-09-30T22:28:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.564+0000 7f5ef7c39700 1 Processor -- start 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.564+0000 7f5ef7c39700 1 -- start start 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef7c39700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef009d630 0x7f5ef0133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef7c39700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ef00a0a30 0x7f5ef0133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef7c39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 0x7f5ef0134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef7c39700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0136540 con 0x7f5ef0138d80 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef7c39700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0136eb0 con 0x7f5ef00a0a30 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef7c39700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0136190 con 0x7f5ef009d630 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef61d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 0x7f5ef0134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef51d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ef00a0a30 0x7f5ef0133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef61d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 0x7f5ef0134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51274/0 (socket says 172.21.15.32:51274) 2023-09-30T22:28:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef61d6700 1 -- 172.21.15.32:0/1557891515 learned_addr learned my addr 172.21.15.32:0/1557891515 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.565+0000 7f5ef59d5700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef009d630 0x7f5ef0133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef59d5700 1 -- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ef00a0a30 msgr2=0x7f5ef0133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef59d5700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ef00a0a30 0x7f5ef0133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef59d5700 1 -- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 msgr2=0x7f5ef0134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef59d5700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 0x7f5ef0134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef59d5700 1 -- 172.21.15.32:0/1557891515 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8009a70 con 0x7f5ef009d630 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef61d6700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 0x7f5ef0134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef51d4700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ef00a0a30 0x7f5ef0133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef59d5700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef009d630 0x7f5ef0133700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00c4c0 tx=0x7f5eec00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec005d70 con 0x7f5ef009d630 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0070d0 con 0x7f5ef009d630 2023-09-30T22:28:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee00094a0 con 0x7f5ef009d630 2023-09-30T22:28:36.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec007440 con 0x7f5ef009d630 2023-09-30T22:28:36.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.566+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef0137190 con 0x7f5ef009d630 2023-09-30T22:28:36.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.567+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5eec0075a0 con 0x7f5ef009d630 2023-09-30T22:28:36.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.567+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5eec03eac0 con 0x7f5ef009d630 2023-09-30T22:28:36.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.567+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0136ff0 con 0x7f5ef009d630 2023-09-30T22:28:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.572+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ef0136ff0 con 0x7f5ef009d630 2023-09-30T22:28:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.675+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd67406c360 con 0x7fd67409e000 2023-09-30T22:28:36.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.675+0000 7fd66a7fc700 1 -- 172.21.15.32:0/1468591513 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6640239c0 con 0x7fd67409e000 2023-09-30T22:28:36.676 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 msgr2=0x7fd674122cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 0x7fd674122cb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd664016000 tx=0x7fd664004750 comp rx=0 tx=0).stop 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 shutdown_connections 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd674133f30 0x7fd6741231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd67409d6f0 0x7fd674122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 --2- 172.21.15.32:0/1468591513 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd67409e000 0x7fd674122cb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 >> 172.21.15.32:0/1468591513 conn(0x7fd674066f50 msgr2=0x7fd674070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 shutdown_connections 2023-09-30T22:28:36.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.678+0000 7fd67c041700 1 -- 172.21.15.32:0/1468591513 wait complete. 2023-09-30T22:28:36.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:36.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.762+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbab806b9e0 con 0x7fbab809e850 2023-09-30T22:28:36.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.763+0000 7fbaadffb700 1 -- 172.21.15.32:0/729781813 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbab4017020 con 0x7fbab809e850 2023-09-30T22:28:36.763 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 msgr2=0x7fbab81227b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab81227b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbab400e900 tx=0x7fbab400ec10 comp rx=0 tx=0).stop 2023-09-30T22:28:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 shutdown_connections 2023-09-30T22:28:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbab809f5b0 0x7fbab8123230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbab809e850 0x7fbab81227b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 --2- 172.21.15.32:0/729781813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbab809f160 0x7fbab8122cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 >> 172.21.15.32:0/729781813 conn(0x7fbab80a3f70 msgr2=0x7fbab806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 shutdown_connections 2023-09-30T22:28:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.765+0000 7fbabf747700 1 -- 172.21.15.32:0/729781813 wait complete. 2023-09-30T22:28:36.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.796+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5ef0136ff0 con 0x7f5ef009d630 2023-09-30T22:28:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.796+0000 7f5ee67fc700 1 -- 172.21.15.32:0/1557891515 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ef0136ff0 con 0x7f5ef009d630 2023-09-30T22:28:36.796 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:36.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef009d630 msgr2=0x7f5ef0133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef009d630 0x7f5ef0133700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00c4c0 tx=0x7f5eec00c7d0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 shutdown_connections 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ef0138d80 0x7f5ef0134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ef009d630 0x7f5ef0133700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 --2- 172.21.15.32:0/1557891515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ef00a0a30 0x7f5ef0133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 >> 172.21.15.32:0/1557891515 conn(0x7f5ef00a3050 msgr2=0x7f5ef009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 shutdown_connections 2023-09-30T22:28:36.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:36.798+0000 7f5ef7c39700 1 -- 172.21.15.32:0/1557891515 wait complete. 2023-09-30T22:28:36.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:37.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.405+0000 7f0e2ae5f700 1 Processor -- start 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.405+0000 7f0e2ae5f700 1 -- start start 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e2ae5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e240a0a30 0x7f0e240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e2ae5f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e24134170 0x7f0e2413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e2ae5f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2409edd0 con 0x7f0e240a1340 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e2ae5f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2409e1f0 con 0x7f0e24134170 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e2ae5f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2409dc50 con 0x7f0e240a0a30 2023-09-30T22:28:37.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e293fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e24134170 0x7f0e2413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e23fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e240a1340 0x7f0e24133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33620/0 (socket says 172.21.15.32:33620) 2023-09-30T22:28:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e23fff700 1 -- 172.21.15.32:0/45666821 learned_addr learned my addr 172.21.15.32:0/45666821 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:37.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e28bfb700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e240a0a30 0x7f0e240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e293fc700 1 -- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e240a0a30 msgr2=0x7f0e240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e293fc700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e240a0a30 0x7f0e240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e293fc700 1 -- 172.21.15.32:0/45666821 >> v1:172.21.15.32:6789/0 conn(0x7f0e240a1340 legacy=0x7f0e24133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e237fe700 1 -- 172.21.15.32:0/45666821 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158487447 0 0) 0x7f0e2409edd0 con 0x7f0e240a1340 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e293fc700 1 -- 172.21.15.32:0/45666821 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240a1fe0 con 0x7f0e24134170 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.406+0000 7f0e28bfb700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e240a0a30 0x7f0e240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e293fc700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e24134170 0x7f0e2413bd30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009dc0 tx=0x7f0e1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86e02ce333c77882 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e237fe700 1 -- 172.21.15.32:0/45666821 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00e070 con 0x7f0e24134170 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e237fe700 1 -- 172.21.15.32:0/45666821 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c00bb20 con 0x7f0e24134170 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e237fe700 1 -- 172.21.15.32:0/45666821 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c012430 con 0x7f0e24134170 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e24134170 msgr2=0x7f0e2413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e24134170 0x7f0e2413bd30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009dc0 tx=0x7f0e1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:37.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/45666821 shutdown_connections 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e24134170 0x7f0e2413bd30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/45666821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e240a0a30 0x7f0e240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.407+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/45666821 >> 172.21.15.32:0/45666821 conn(0x7f0e240a6550 msgr2=0x7f0e240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.408+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/45666821 shutdown_connections 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.408+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/45666821 wait complete. 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.408+0000 7f0e2ae5f700 1 Processor -- start 2023-09-30T22:28:37.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 -- start start 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e240a0a30 0x7f0e24122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e240a1340 0x7f0e24122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 0x7f0e24123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240a4ec0 con 0x7f0e240a0a30 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241336b0 con 0x7f0e240a1340 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e2ae5f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24124410 con 0x7f0e24134170 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e293fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 0x7f0e24123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e28bfb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e240a0a30 0x7f0e24122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e23fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e240a1340 0x7f0e24122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e293fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 0x7f0e24123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54152/0 (socket says 172.21.15.32:54152) 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.409+0000 7f0e293fc700 1 -- 172.21.15.32:0/3042669506 learned_addr learned my addr 172.21.15.32:0/3042669506 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e293fc700 1 -- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e240a1340 msgr2=0x7f0e24122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e293fc700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e240a1340 0x7f0e24122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e293fc700 1 -- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e240a0a30 msgr2=0x7f0e24122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e293fc700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e240a0a30 0x7f0e24122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e293fc700 1 -- 172.21.15.32:0/3042669506 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140094a0 con 0x7f0e24134170 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e23fff700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e240a1340 0x7f0e24122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e293fc700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 0x7f0e24123200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0e2409e380 tx=0x7f0e1c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004840 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c020070 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c009a70 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0276c0 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.410+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e240a25e0 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.411+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e1c012430 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.411+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e1c022070 con 0x7f0e24134170 2023-09-30T22:28:37.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.411+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e24133250 con 0x7f0e24134170 2023-09-30T22:28:37.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.416+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e1c01b400 con 0x7f0e24134170 2023-09-30T22:28:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.620+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e24074ea0 con 0x7f0e24134170 2023-09-30T22:28:37.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.620+0000 7f0e217fa700 1 -- 172.21.15.32:0/3042669506 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e1c02c030 con 0x7f0e24134170 2023-09-30T22:28:37.620 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 msgr2=0x7f0e24123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 0x7f0e24123200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0e2409e380 tx=0x7f0e1c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 shutdown_connections 2023-09-30T22:28:37.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e24134170 0x7f0e24123200 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e240a0a30 0x7f0e24122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 --2- 172.21.15.32:0/3042669506 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e240a1340 0x7f0e24122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 >> 172.21.15.32:0/3042669506 conn(0x7f0e240a6550 msgr2=0x7f0e2409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 shutdown_connections 2023-09-30T22:28:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.622+0000 7f0e2ae5f700 1 -- 172.21.15.32:0/3042669506 wait complete. 2023-09-30T22:28:37.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:37.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:37.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:37.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:37.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.833+0000 7fa27c370700 1 Processor -- start 2023-09-30T22:28:37.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.833+0000 7fa27c370700 1 -- start start 2023-09-30T22:28:37.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27c370700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 0x7fa274133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27c370700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa27409de50 0x7fa27409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27c370700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27406ef30 con 0x7fa27409e760 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27c370700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27406fef0 con 0x7fa27409de50 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27c370700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741334b0 con 0x7fa2741338a0 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27a10c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 0x7fa274133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27a10c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 0x7fa274133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54158/0 (socket says 172.21.15.32:54158) 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.834+0000 7fa27a10c700 1 -- 172.21.15.32:0/2160480063 learned_addr learned my addr 172.21.15.32:0/2160480063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27990b700 1 --2- 172.21.15.32:0/2160480063 >> v2:172.21.15.32:3300/0 conn(0x7fa27409de50 0x7fa27409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27a10c700 1 -- 172.21.15.32:0/2160480063 >> v2:172.21.15.32:3300/0 conn(0x7fa27409de50 msgr2=0x7fa27409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:37.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27a10c700 1 --2- 172.21.15.32:0/2160480063 >> v2:172.21.15.32:3300/0 conn(0x7fa27409de50 0x7fa27409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27a10c700 1 -- 172.21.15.32:0/2160480063 >> v1:172.21.15.32:6789/0 conn(0x7fa27409e760 legacy=0x7fa27413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27a10c700 1 -- 172.21.15.32:0/2160480063 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27406fc30 con 0x7fa2741338a0 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27a10c700 1 --2- 172.21.15.32:0/2160480063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 0x7fa274133c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098f0 tx=0x7fa270009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c2901aedfde004 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.835+0000 7fa27910a700 1 -- 172.21.15.32:0/2160480063 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270011830 con 0x7fa2741338a0 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27910a700 1 -- 172.21.15.32:0/2160480063 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa270011990 con 0x7fa2741338a0 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27910a700 1 -- 172.21.15.32:0/2160480063 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270011d20 con 0x7fa2741338a0 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 -- 172.21.15.32:0/2160480063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 msgr2=0x7fa274133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 --2- 172.21.15.32:0/2160480063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 0x7fa274133c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098f0 tx=0x7fa270009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 -- 172.21.15.32:0/2160480063 shutdown_connections 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 --2- 172.21.15.32:0/2160480063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2741338a0 0x7fa274133c70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 --2- 172.21.15.32:0/2160480063 >> v2:172.21.15.32:3300/0 conn(0x7fa27409de50 0x7fa27409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 -- 172.21.15.32:0/2160480063 >> 172.21.15.32:0/2160480063 conn(0x7fa2740a2aa0 msgr2=0x7fa2740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.836+0000 7fa27c370700 1 -- 172.21.15.32:0/2160480063 shutdown_connections 2023-09-30T22:28:37.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.837+0000 7fa27c370700 1 -- 172.21.15.32:0/2160480063 wait complete. 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.837+0000 7fa27c370700 1 Processor -- start 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 -- start start 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 0x7fa27411ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa27409e760 0x7fa27411efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2741338a0 0x7fa27411f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274122010 con 0x7fa27409de50 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741223b0 con 0x7fa2741338a0 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27c370700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741206a0 con 0x7fa27409e760 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27a10c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 0x7fa27411ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27990b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa27409e760 0x7fa27411efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27a90d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2741338a0 0x7fa27411f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27990b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa27409e760 0x7fa27411efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54162/0 (socket says 172.21.15.32:54162) 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27990b700 1 -- 172.21.15.32:0/1973985561 learned_addr learned my addr 172.21.15.32:0/1973985561 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.838+0000 7fa27a10c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 0x7fa27411ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51312/0 (socket says 172.21.15.32:51312) 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a10c700 1 -- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa27409e760 msgr2=0x7fa27411efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a10c700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa27409e760 0x7fa27411efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a10c700 1 -- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2741338a0 msgr2=0x7fa27411f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a10c700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2741338a0 0x7fa27411f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a10c700 1 -- 172.21.15.32:0/1973985561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270009550 con 0x7fa27409de50 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a90d700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2741338a0 0x7fa27411f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:37.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27a10c700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 0x7fa27411ea90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa27406fae0 tx=0x7fa270004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270004400 con 0x7fa27409de50 2023-09-30T22:28:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27001f070 con 0x7fa27409de50 2023-09-30T22:28:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27406f640 con 0x7fa27409de50 2023-09-30T22:28:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2700038e0 con 0x7fa27409de50 2023-09-30T22:28:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.839+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27412f610 con 0x7fa27409de50 2023-09-30T22:28:37.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.840+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa274122210 con 0x7fa27409de50 2023-09-30T22:28:37.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.841+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa270021030 con 0x7fa27409de50 2023-09-30T22:28:37.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.841+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa270040cb0 con 0x7fa27409de50 2023-09-30T22:28:37.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.846+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa270003a40 con 0x7fa27409de50 2023-09-30T22:28:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.923+0000 7fc3d8cae700 1 Processor -- start 2023-09-30T22:28:37.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.923+0000 7fc3d8cae700 1 -- start start 2023-09-30T22:28:37.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d8cae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 0x7fc3d4133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d8cae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d4130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d8cae700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d406bc00 con 0x7fc3d4130870 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d8cae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d406c570 con 0x7fc3d412ff20 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d8cae700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40a0040 con 0x7fc3d4133a80 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 0x7fc3d4133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d4130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d4130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41654/0 (socket says 172.21.15.32:41654) 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 0x7fc3d4133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54164/0 (socket says 172.21.15.32:54164) 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 -- 172.21.15.32:0/496586521 learned_addr learned my addr 172.21.15.32:0/496586521 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:37.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 -- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 msgr2=0x7fc3d4133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 --2- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 0x7fc3d4133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 -- 172.21.15.32:0/496586521 >> v1:172.21.15.32:6789/0 conn(0x7fc3d4130870 legacy=0x7fc3d413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 -- 172.21.15.32:0/496586521 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4133860 con 0x7fc3d412ff20 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d259c700 1 --2- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 0x7fc3d4133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.924+0000 7fc3d1d9b700 1 --2- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d4130330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009920 tx=0x7fc3c8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f66acdafe56687 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.925+0000 7fc3d159a700 1 -- 172.21.15.32:0/496586521 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8011830 con 0x7fc3d412ff20 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.925+0000 7fc3d159a700 1 -- 172.21.15.32:0/496586521 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8011990 con 0x7fc3d412ff20 2023-09-30T22:28:37.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.925+0000 7fc3d159a700 1 -- 172.21.15.32:0/496586521 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8011d20 con 0x7fc3d412ff20 2023-09-30T22:28:37.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.925+0000 7fc3d8cae700 1 -- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 msgr2=0x7fc3d4130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.925+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d4130330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009920 tx=0x7fc3c8009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.926+0000 7fc3d8cae700 1 -- 172.21.15.32:0/496586521 shutdown_connections 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.926+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4133a80 0x7fc3d4133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.926+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/496586521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d4130330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.926+0000 7fc3d8cae700 1 -- 172.21.15.32:0/496586521 >> 172.21.15.32:0/496586521 conn(0x7fc3d40a4210 msgr2=0x7fc3d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.926+0000 7fc3d8cae700 1 -- 172.21.15.32:0/496586521 shutdown_connections 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.926+0000 7fc3d8cae700 1 -- 172.21.15.32:0/496586521 wait complete. 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 Processor -- start 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 -- start start 2023-09-30T22:28:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4130870 0x7fc3d409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d4133a80 0x7fc3d409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d41200c0 con 0x7fc3d4133a80 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d4120960 con 0x7fc3d412ff20 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.927+0000 7fc3d8cae700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d409e2d0 con 0x7fc3d4130870 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d4133a80 0x7fc3d409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d409c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41666/0 (socket says 172.21.15.32:41666) 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 -- 172.21.15.32:0/805406109 learned_addr learned my addr 172.21.15.32:0/805406109 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d1d9b700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4130870 0x7fc3d409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 -- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4130870 msgr2=0x7fc3d409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4130870 0x7fc3d409cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 -- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d4133a80 msgr2=0x7fc3d409d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d4133a80 0x7fc3d409d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 -- 172.21.15.32:0/805406109 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40094a0 con 0x7fc3d412ff20 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d2d9d700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d4133a80 0x7fc3d409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d259c700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d409c6e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400ebe0 tx=0x7fc3c400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400c850 con 0x7fc3d412ff20 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c400c9b0 con 0x7fc3d412ff20 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.928+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c8009550 con 0x7fc3d412ff20 2023-09-30T22:28:37.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.929+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400cd40 con 0x7fc3d412ff20 2023-09-30T22:28:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.929+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d412daa0 con 0x7fc3d412ff20 2023-09-30T22:28:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.929+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3c4013470 con 0x7fc3d412ff20 2023-09-30T22:28:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.930+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d4074ea0 con 0x7fc3d412ff20 2023-09-30T22:28:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.930+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc3c403f770 con 0x7fc3d412ff20 2023-09-30T22:28:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.934+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3c4013720 con 0x7fc3d412ff20 2023-09-30T22:28:37.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.965+0000 7f9b1d1fa700 1 Processor -- start 2023-09-30T22:28:37.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.965+0000 7f9b1d1fa700 1 -- start start 2023-09-30T22:28:37.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1d1fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b180a1510 0x7f9b18138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1d1fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b18139250 0x7f9b1813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1d1fa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1809edd0 con 0x7f9b180a0a30 2023-09-30T22:28:37.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1d1fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180a18e0 con 0x7f9b180a1510 2023-09-30T22:28:37.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1d1fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1809e1f0 con 0x7f9b18139250 2023-09-30T22:28:37.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b16d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b180a0a30 0x7f9b180a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33634/0 (socket says 172.21.15.32:33634) 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b16d9d700 1 -- 172.21.15.32:0/2580694899 learned_addr learned my addr 172.21.15.32:0/2580694899 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1659c700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b180a1510 0x7f9b18138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1759e700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b18139250 0x7f9b1813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1659c700 1 -- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b18139250 msgr2=0x7f9b1813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1659c700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b18139250 0x7f9b1813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b15d9b700 1 -- 172.21.15.32:0/2580694899 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970032383 0 0) 0x7f9b1809edd0 con 0x7f9b180a0a30 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1659c700 1 -- 172.21.15.32:0/2580694899 >> v1:172.21.15.32:6789/0 conn(0x7f9b180a0a30 legacy=0x7f9b180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.966+0000 7f9b1659c700 1 -- 172.21.15.32:0/2580694899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1809dc50 con 0x7f9b180a1510 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.967+0000 7f9b1659c700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b180a1510 0x7f9b18138d10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009870 tx=0x7f9b0c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe2a03c47851b7f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.967+0000 7f9b15d9b700 1 -- 172.21.15.32:0/2580694899 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011830 con 0x7f9b180a1510 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.967+0000 7f9b15d9b700 1 -- 172.21.15.32:0/2580694899 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c011990 con 0x7f9b180a1510 2023-09-30T22:28:37.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.967+0000 7f9b15d9b700 1 -- 172.21.15.32:0/2580694899 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011d20 con 0x7f9b180a1510 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b180a1510 msgr2=0x7f9b18138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b180a1510 0x7f9b18138d10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009870 tx=0x7f9b0c009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2580694899 shutdown_connections 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b18139250 0x7f9b1813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2580694899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b180a1510 0x7f9b18138d10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2580694899 >> 172.21.15.32:0/2580694899 conn(0x7f9b180a6550 msgr2=0x7f9b180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:37.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2580694899 shutdown_connections 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.968+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2580694899 wait complete. 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.969+0000 7f9b1d1fa700 1 Processor -- start 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.969+0000 7f9b1d1fa700 1 -- start start 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1d1fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 0x7f9b18122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1d1fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 0x7f9b18122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1d1fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b18139250 0x7f9b181232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1d1fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b18126140 con 0x7f9b180a1510 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1d1fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b18125ab0 con 0x7f9b18139250 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1d1fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b18138950 con 0x7f9b180a0a30 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 0x7f9b18122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 0x7f9b18122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 0x7f9b18122d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51338/0 (socket says 172.21.15.32:51338) 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 0x7f9b18122820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54204/0 (socket says 172.21.15.32:54204) 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 -- 172.21.15.32:0/2292437134 learned_addr learned my addr 172.21.15.32:0/2292437134 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1759e700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b18139250 0x7f9b181232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 -- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 msgr2=0x7f9b18122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 0x7f9b18122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 -- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b18139250 msgr2=0x7f9b181232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:37.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b18139250 0x7f9b181232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1659c700 1 -- 172.21.15.32:0/2292437134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0094a0 con 0x7f9b180a1510 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b16d9d700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 0x7f9b18122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.970+0000 7f9b1759e700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b18139250 0x7f9b181232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.971+0000 7f9b1659c700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 0x7f9b18122d60 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c000c00 tx=0x7f9b0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.971+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c004590 con 0x7f9b180a1510 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.971+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c01f070 con 0x7f9b180a1510 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.971+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b18124430 con 0x7f9b180a1510 2023-09-30T22:28:37.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.971+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0038e0 con 0x7f9b180a1510 2023-09-30T22:28:37.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.971+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b18124be0 con 0x7f9b180a1510 2023-09-30T22:28:37.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.972+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b0c003a40 con 0x7f9b180a1510 2023-09-30T22:28:37.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.972+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b0c021070 con 0x7f9b180a1510 2023-09-30T22:28:37.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.972+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b180a1d00 con 0x7f9b180a1510 2023-09-30T22:28:37.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:37.977+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b0c015080 con 0x7f9b180a1510 2023-09-30T22:28:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.059+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa274122210 con 0x7fa27409de50 2023-09-30T22:28:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.059+0000 7fa26affd700 1 -- 172.21.15.32:0/1973985561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa270015070 con 0x7fa27409de50 2023-09-30T22:28:38.060 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 msgr2=0x7fa27411ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 0x7fa27411ea90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa27406fae0 tx=0x7fa270004750 comp rx=0 tx=0).stop 2023-09-30T22:28:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 shutdown_connections 2023-09-30T22:28:38.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2741338a0 0x7fa27411f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27409de50 0x7fa27411ea90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 --2- 172.21.15.32:0/1973985561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa27409e760 0x7fa27411efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 >> 172.21.15.32:0/1973985561 conn(0x7fa2740a2aa0 msgr2=0x7fa27409edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 shutdown_connections 2023-09-30T22:28:38.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.062+0000 7fa27c370700 1 -- 172.21.15.32:0/1973985561 wait complete. 2023-09-30T22:28:38.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:38.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.148+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc3d406b250 con 0x7fc3d412ff20 2023-09-30T22:28:38.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.148+0000 7fc3c2ffd700 1 -- 172.21.15.32:0/805406109 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3c4021070 con 0x7fc3d412ff20 2023-09-30T22:28:38.148 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 msgr2=0x7fc3d409c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d409c6e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400ebe0 tx=0x7fc3c400eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 shutdown_connections 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3d4133a80 0x7fc3d409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3d412ff20 0x7fc3d409c6e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 --2- 172.21.15.32:0/805406109 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3d4130870 0x7fc3d409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 >> 172.21.15.32:0/805406109 conn(0x7fc3d40a4210 msgr2=0x7fc3d40a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:38.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 shutdown_connections 2023-09-30T22:28:38.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.151+0000 7fc3d8cae700 1 -- 172.21.15.32:0/805406109 wait complete. 2023-09-30T22:28:38.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:38.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.187+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b18074ea0 con 0x7f9b180a1510 2023-09-30T22:28:38.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.187+0000 7f9aff7fe700 1 -- 172.21.15.32:0/2292437134 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b0c02b020 con 0x7f9b180a1510 2023-09-30T22:28:38.188 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.189+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 msgr2=0x7f9b18122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.189+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 0x7f9b18122d60 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c000c00 tx=0x7f9b0c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 shutdown_connections 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b18139250 0x7f9b181232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b180a0a30 0x7f9b18122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 --2- 172.21.15.32:0/2292437134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b180a1510 0x7f9b18122d60 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 >> 172.21.15.32:0/2292437134 conn(0x7f9b180a6550 msgr2=0x7f9b1809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:38.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 shutdown_connections 2023-09-30T22:28:38.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.190+0000 7f9b1d1fa700 1 -- 172.21.15.32:0/2292437134 wait complete. 2023-09-30T22:28:38.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:38.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 Processor -- start 2023-09-30T22:28:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 -- start start 2023-09-30T22:28:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddc409d6f0 0x7fddc409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 0x7fddc41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc409b120 con 0x7fddc409e410 2023-09-30T22:28:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc406b460 con 0x7fddc409d6f0 2023-09-30T22:28:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc9dd6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc409a7e0 con 0x7fddc409e000 2023-09-30T22:28:38.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 0x7fddc41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddc409d6f0 0x7fddc409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddc409d6f0 0x7fddc409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51348/0 (socket says 172.21.15.32:51348) 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc37fe700 1 -- 172.21.15.32:0/3318739162 learned_addr learned my addr 172.21.15.32:0/3318739162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 0x7fddc41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54210/0 (socket says 172.21.15.32:54210) 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.784+0000 7fddc3fff700 1 --1- 172.21.15.32:0/3318739162 >> v1:172.21.15.32:6789/0 conn(0x7fddc409e410 0x7fddc4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33648/0 (socket says 172.21.15.32:33648) 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc37fe700 1 -- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 msgr2=0x7fddc41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc37fe700 1 --2- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 0x7fddc41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc37fe700 1 -- 172.21.15.32:0/3318739162 >> v1:172.21.15.32:6789/0 conn(0x7fddc409e410 legacy=0x7fddc4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc37fe700 1 -- 172.21.15.32:0/3318739162 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc409a500 con 0x7fddc409d6f0 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 0x7fddc41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:38.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc27fc700 1 -- 172.21.15.32:0/3318739162 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583415247 0 0) 0x7fddc409b120 con 0x7fddc409e410 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc37fe700 1 --2- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc409dac0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098f0 tx=0x7fddb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390727802548ea71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc27fc700 1 -- 172.21.15.32:0/3318739162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4011830 con 0x7fddc409d6f0 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc27fc700 1 -- 172.21.15.32:0/3318739162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb4011990 con 0x7fddc409d6f0 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.785+0000 7fddc27fc700 1 -- 172.21.15.32:0/3318739162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4011d20 con 0x7fddc409d6f0 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 -- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 msgr2=0x7fddc409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc409dac0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098f0 tx=0x7fddb4009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 -- 172.21.15.32:0/3318739162 shutdown_connections 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc409dac0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/3318739162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e000 0x7fddc41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 -- 172.21.15.32:0/3318739162 >> 172.21.15.32:0/3318739162 conn(0x7fddc4066f50 msgr2=0x7fddc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 -- 172.21.15.32:0/3318739162 shutdown_connections 2023-09-30T22:28:38.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.786+0000 7fddc9dd6700 1 -- 172.21.15.32:0/3318739162 wait complete. 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.787+0000 7fddc9dd6700 1 Processor -- start 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.787+0000 7fddc9dd6700 1 -- start start 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.787+0000 7fddc9dd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc41c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.787+0000 7fddc9dd6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 0x7fddc41c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc9dd6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e410 0x7fddc41c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc9dd6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41a7680 con 0x7fddc409d6f0 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc9dd6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc409c710 con 0x7fddc409e000 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc9dd6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40a2ec0 con 0x7fddc409e410 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 0x7fddc41c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:38.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e410 0x7fddc41c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc41c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 0x7fddc41c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41688/0 (socket says 172.21.15.32:41688) 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc2ffd700 1 -- 172.21.15.32:0/1587939596 learned_addr learned my addr 172.21.15.32:0/1587939596 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 -- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e410 msgr2=0x7fddc41c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e410 0x7fddc41c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 -- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 msgr2=0x7fddc41c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 0x7fddc41c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 -- 172.21.15.32:0/1587939596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80094a0 con 0x7fddc409d6f0 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc3fff700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e410 0x7fddc41c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc2ffd700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 0x7fddc41c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:38.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc37fe700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc41c81c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098f0 tx=0x7fddb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb401a8f0 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb401e070 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb4009550 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.788+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb401ac50 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.789+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc41a8d10 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.789+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fddb402d450 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.789+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fddb40404d0 con 0x7fddc409d6f0 2023-09-30T22:28:38.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.790+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc404ea50 con 0x7fddc409d6f0 2023-09-30T22:28:38.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:38.795+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fddb4015080 con 0x7fddc409d6f0 2023-09-30T22:28:39.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.000+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fddc4074ea0 con 0x7fddc409d6f0 2023-09-30T22:28:39.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.000+0000 7fddabfff700 1 -- 172.21.15.32:0/1587939596 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fddb402d700 con 0x7fddc409d6f0 2023-09-30T22:28:39.000 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 msgr2=0x7fddc41c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc41c81c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098f0 tx=0x7fddb4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 shutdown_connections 2023-09-30T22:28:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fddc409e410 0x7fddc41c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc409d6f0 0x7fddc41c81c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 --2- 172.21.15.32:0/1587939596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fddc409e000 0x7fddc41c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 >> 172.21.15.32:0/1587939596 conn(0x7fddc4066f50 msgr2=0x7fddc41bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 shutdown_connections 2023-09-30T22:28:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.002+0000 7fddc9dd6700 1 -- 172.21.15.32:0/1587939596 wait complete. 2023-09-30T22:28:39.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:39.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:39.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:39.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.225+0000 7ff227357700 1 Processor -- start 2023-09-30T22:28:39.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.225+0000 7ff227357700 1 -- start start 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff227357700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff22006b250 0x7ff220136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff227357700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff220136d80 0x7ff220139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff227357700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22009ece0 con 0x7ff22006a770 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff227357700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22006b640 con 0x7ff220136d80 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff227357700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff22009e760 con 0x7ff22006b250 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff2250f3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff22006a770 0x7ff22006ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33664/0 (socket says 172.21.15.32:33664) 2023-09-30T22:28:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff2250f3700 1 -- 172.21.15.32:0/2255215053 learned_addr learned my addr 172.21.15.32:0/2255215053 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff2248f2700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff22006b250 0x7ff220136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.226+0000 7ff2258f4700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff220136d80 0x7ff220139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff217fff700 1 -- 172.21.15.32:0/2255215053 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035594915 0 0) 0x7ff22009ece0 con 0x7ff22006a770 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff217fff700 1 -- 172.21.15.32:0/2255215053 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff20c0035b0 con 0x7ff22006a770 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff2258f4700 1 -- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff22006b250 msgr2=0x7ff220136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff2258f4700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff22006b250 0x7ff220136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff2258f4700 1 -- 172.21.15.32:0/2255215053 >> v1:172.21.15.32:6789/0 conn(0x7ff22006a770 legacy=0x7ff22006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff2258f4700 1 -- 172.21.15.32:0/2255215053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22009e040 con 0x7ff220136d80 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff2258f4700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff220136d80 0x7ff220139160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff218009dc0 tx=0x7ff21800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c261b79032e173 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff217fff700 1 -- 172.21.15.32:0/2255215053 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21800e070 con 0x7ff220136d80 2023-09-30T22:28:39.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff217fff700 1 -- 172.21.15.32:0/2255215053 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21800bb20 con 0x7ff220136d80 2023-09-30T22:28:39.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.227+0000 7ff217fff700 1 -- 172.21.15.32:0/2255215053 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff218012430 con 0x7ff220136d80 2023-09-30T22:28:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 -- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff220136d80 msgr2=0x7ff220139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff220136d80 0x7ff220139160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff218009dc0 tx=0x7ff21800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 -- 172.21.15.32:0/2255215053 shutdown_connections 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff220136d80 0x7ff220139160 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 --2- 172.21.15.32:0/2255215053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff22006b250 0x7ff220136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 -- 172.21.15.32:0/2255215053 >> 172.21.15.32:0/2255215053 conn(0x7ff220066f50 msgr2=0x7ff22009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 -- 172.21.15.32:0/2255215053 shutdown_connections 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.228+0000 7ff227357700 1 -- 172.21.15.32:0/2255215053 wait complete. 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.229+0000 7ff227357700 1 Processor -- start 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.229+0000 7ff227357700 1 -- start start 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff227357700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff22006a770 0x7ff2201c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff227357700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 0x7ff2201c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff227357700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff220136d80 0x7ff22011f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff227357700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff220122ac0 con 0x7ff22006b250 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff227357700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22006edb0 con 0x7ff22006a770 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff227357700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201bbf00 con 0x7ff220136d80 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2248f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 0x7ff2201c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2248f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 0x7ff2201c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51368/0 (socket says 172.21.15.32:51368) 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2248f2700 1 -- 172.21.15.32:0/349351406 learned_addr learned my addr 172.21.15.32:0/349351406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2258f4700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff220136d80 0x7ff22011f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2250f3700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff22006a770 0x7ff2201c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2258f4700 1 -- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff22006a770 msgr2=0x7ff2201c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2258f4700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff22006a770 0x7ff2201c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2258f4700 1 -- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 msgr2=0x7ff2201c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2258f4700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 0x7ff2201c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.230+0000 7ff2258f4700 1 -- 172.21.15.32:0/349351406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218009a70 con 0x7ff220136d80 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff2248f2700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 0x7ff2201c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff2258f4700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff220136d80 0x7ff22011f540 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff218016000 tx=0x7ff218004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff218004800 con 0x7ff220136d80 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff218020070 con 0x7ff220136d80 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff220135410 con 0x7ff220136d80 2023-09-30T22:28:39.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180276c0 con 0x7ff220136d80 2023-09-30T22:28:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.231+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2201237f0 con 0x7ff220136d80 2023-09-30T22:28:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.232+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff218012430 con 0x7ff220136d80 2023-09-30T22:28:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.232+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff218022070 con 0x7ff220136d80 2023-09-30T22:28:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.232+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff220074ea0 con 0x7ff220136d80 2023-09-30T22:28:39.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.238+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff218003e30 con 0x7ff220136d80 2023-09-30T22:28:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.309+0000 7fc77f9db700 1 Processor -- start 2023-09-30T22:28:39.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.309+0000 7fc77f9db700 1 -- start start 2023-09-30T22:28:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77f9db700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc77809e930 0x7fc77809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77f9db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 0x7fc778133be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77f9db700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77806c9f0 con 0x7fc778134120 2023-09-30T22:28:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77f9db700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77806c670 con 0x7fc77809e930 2023-09-30T22:28:39.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77f9db700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc77806bc90 con 0x7fc77809f240 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77cf76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 0x7fc778133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77d777700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc77809e930 0x7fc77809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77cf76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 0x7fc778133be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54236/0 (socket says 172.21.15.32:54236) 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.310+0000 7fc77cf76700 1 -- 172.21.15.32:0/3183713039 learned_addr learned my addr 172.21.15.32:0/3183713039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc77cf76700 1 -- 172.21.15.32:0/3183713039 >> v2:172.21.15.32:3300/0 conn(0x7fc77809e930 msgr2=0x7fc77809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc77cf76700 1 --2- 172.21.15.32:0/3183713039 >> v2:172.21.15.32:3300/0 conn(0x7fc77809e930 0x7fc77809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc77cf76700 1 -- 172.21.15.32:0/3183713039 >> v1:172.21.15.32:6789/0 conn(0x7fc778134120 legacy=0x7fc77813ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc77cf76700 1 -- 172.21.15.32:0/3183713039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77806d860 con 0x7fc77809f240 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc77d777700 1 --2- 172.21.15.32:0/3183713039 >> v2:172.21.15.32:3300/0 conn(0x7fc77809e930 0x7fc77809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc77cf76700 1 --2- 172.21.15.32:0/3183713039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 0x7fc778133be0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc7680098f0 tx=0x7fc768009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebd7dfb7a41e439 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc76ffff700 1 -- 172.21.15.32:0/3183713039 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768011830 con 0x7fc77809f240 2023-09-30T22:28:39.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc76ffff700 1 -- 172.21.15.32:0/3183713039 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc768011990 con 0x7fc77809f240 2023-09-30T22:28:39.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.311+0000 7fc76ffff700 1 -- 172.21.15.32:0/3183713039 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768011d20 con 0x7fc77809f240 2023-09-30T22:28:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 -- 172.21.15.32:0/3183713039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 msgr2=0x7fc778133be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3183713039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 0x7fc778133be0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc7680098f0 tx=0x7fc768009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 -- 172.21.15.32:0/3183713039 shutdown_connections 2023-09-30T22:28:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3183713039 >> v2:172.21.15.32:3300/0 conn(0x7fc77809e930 0x7fc77809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3183713039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809f240 0x7fc778133be0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 -- 172.21.15.32:0/3183713039 >> 172.21.15.32:0/3183713039 conn(0x7fc7780a4210 msgr2=0x7fc7780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 -- 172.21.15.32:0/3183713039 shutdown_connections 2023-09-30T22:28:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.312+0000 7fc77f9db700 1 -- 172.21.15.32:0/3183713039 wait complete. 2023-09-30T22:28:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.313+0000 7fc77f9db700 1 Processor -- start 2023-09-30T22:28:39.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.313+0000 7fc77f9db700 1 -- start start 2023-09-30T22:28:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77f9db700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809e930 0x7fc778122af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77f9db700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 0x7fc778123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77f9db700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778134120 0x7fc778123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77f9db700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7781247e0 con 0x7fc778134120 2023-09-30T22:28:39.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77f9db700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7781335e0 con 0x7fc77809f240 2023-09-30T22:28:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77f9db700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc778126570 con 0x7fc77809e930 2023-09-30T22:28:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 0x7fc778123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77d777700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809e930 0x7fc778122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 0x7fc778123030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41722/0 (socket says 172.21.15.32:41722) 2023-09-30T22:28:39.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 -- 172.21.15.32:0/3804828003 learned_addr learned my addr 172.21.15.32:0/3804828003 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77df78700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778134120 0x7fc778123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 -- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809e930 msgr2=0x7fc778122af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809e930 0x7fc778122af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 -- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778134120 msgr2=0x7fc778123570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778134120 0x7fc778123570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 -- 172.21.15.32:0/3804828003 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7740094a0 con 0x7fc77809f240 2023-09-30T22:28:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77d777700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809e930 0x7fc778122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.314+0000 7fc77cf76700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 0x7fc778123030 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc768000c00 tx=0x7fc768004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.315+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768004400 con 0x7fc77809f240 2023-09-30T22:28:39.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.315+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76801f070 con 0x7fc77809f240 2023-09-30T22:28:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.315+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc768009550 con 0x7fc77809f240 2023-09-30T22:28:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.315+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680038e0 con 0x7fc77809f240 2023-09-30T22:28:39.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.315+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77806cbb0 con 0x7fc77809f240 2023-09-30T22:28:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.316+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc778074ea0 con 0x7fc77809f240 2023-09-30T22:28:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.317+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc768021030 con 0x7fc77809f240 2023-09-30T22:28:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.317+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc768040df0 con 0x7fc77809f240 2023-09-30T22:28:39.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.321+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7680199a0 con 0x7fc77809f240 2023-09-30T22:28:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 Processor -- start 2023-09-30T22:28:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 -- start start 2023-09-30T22:28:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56ac133420 0x7f56ac1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac134260 0x7f56ac1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ac06b990 con 0x7f56ac133e30 2023-09-30T22:28:39.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0a0590 con 0x7f56ac133420 2023-09-30T22:28:39.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.367+0000 7f56b190c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac09fb80 con 0x7f56ac134260 2023-09-30T22:28:39.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56aa7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56ac133e30 0x7f56ac12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33686/0 (socket says 172.21.15.32:33686) 2023-09-30T22:28:39.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56aa7fc700 1 -- 172.21.15.32:0/3015341004 learned_addr learned my addr 172.21.15.32:0/3015341004 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56ab7fe700 1 --2- 172.21.15.32:0/3015341004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac134260 0x7f56ac1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56aaffd700 1 --2- 172.21.15.32:0/3015341004 >> v2:172.21.15.32:3300/0 conn(0x7f56ac133420 0x7f56ac1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56ab7fe700 1 -- 172.21.15.32:0/3015341004 >> v2:172.21.15.32:3300/0 conn(0x7f56ac133420 msgr2=0x7f56ac1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56a9ffb700 1 -- 172.21.15.32:0/3015341004 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177497776 0 0) 0x7f56ac06b990 con 0x7f56ac133e30 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56ab7fe700 1 --2- 172.21.15.32:0/3015341004 >> v2:172.21.15.32:3300/0 conn(0x7f56ac133420 0x7f56ac1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56ab7fe700 1 -- 172.21.15.32:0/3015341004 >> v1:172.21.15.32:6789/0 conn(0x7f56ac133e30 legacy=0x7f56ac12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56ab7fe700 1 -- 172.21.15.32:0/3015341004 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0a0030 con 0x7f56ac134260 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56ab7fe700 1 --2- 172.21.15.32:0/3015341004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac134260 0x7f56ac1306a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009d90 tx=0x7f56a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb52f1be77b22016 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:39.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56a9ffb700 1 -- 172.21.15.32:0/3015341004 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400e070 con 0x7f56ac134260 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56a9ffb700 1 -- 172.21.15.32:0/3015341004 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a400bb20 con 0x7f56ac134260 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.368+0000 7f56a9ffb700 1 -- 172.21.15.32:0/3015341004 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4012430 con 0x7f56ac134260 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 -- 172.21.15.32:0/3015341004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac134260 msgr2=0x7f56ac1306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 --2- 172.21.15.32:0/3015341004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac134260 0x7f56ac1306a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009d90 tx=0x7f56a400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 -- 172.21.15.32:0/3015341004 shutdown_connections 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 --2- 172.21.15.32:0/3015341004 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac134260 0x7f56ac1306a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 --2- 172.21.15.32:0/3015341004 >> v2:172.21.15.32:3300/0 conn(0x7f56ac133420 0x7f56ac1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 -- 172.21.15.32:0/3015341004 >> 172.21.15.32:0/3015341004 conn(0x7f56ac0a4210 msgr2=0x7f56ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 -- 172.21.15.32:0/3015341004 shutdown_connections 2023-09-30T22:28:39.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.369+0000 7f56b190c700 1 -- 172.21.15.32:0/3015341004 wait complete. 2023-09-30T22:28:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.371+0000 7f56b190c700 1 Processor -- start 2023-09-30T22:28:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.371+0000 7f56b190c700 1 -- start start 2023-09-30T22:28:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56b190c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56ac133420 0x7f56ac09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56b190c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 0x7f56ac09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56b190c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56ac134260 0x7f56ac09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:39.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56b190c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac131470 con 0x7f56ac133420 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56b190c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac1205e0 con 0x7f56ac134260 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56b190c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac11ee60 con 0x7f56ac133e30 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 0x7f56ac09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 0x7f56ac09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54246/0 (socket says 172.21.15.32:54246) 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 -- 172.21.15.32:0/1215642661 learned_addr learned my addr 172.21.15.32:0/1215642661 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aaffd700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56ac133420 0x7f56ac09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 -- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56ac134260 msgr2=0x7f56ac09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56ac134260 0x7f56ac09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 -- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56ac133420 msgr2=0x7f56ac09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56ac133420 0x7f56ac09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 -- 172.21.15.32:0/1215642661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4009a70 con 0x7f56ac133e30 2023-09-30T22:28:39.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.372+0000 7f56aa7fc700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 0x7f56ac09c9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c550 tx=0x7f56a000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.373+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004c80 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.373+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a0004e00 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.373+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007310 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.373+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac120020 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.373+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac12da60 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.374+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac11fae0 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.374+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f56a0007470 con 0x7f56ac133e30 2023-09-30T22:28:39.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.375+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f56a003f7d0 con 0x7f56ac133e30 2023-09-30T22:28:39.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.383+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f56a00102c0 con 0x7f56ac133e30 2023-09-30T22:28:39.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.448+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff22009f6b0 con 0x7ff220136d80 2023-09-30T22:28:39.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.449+0000 7ff215ffb700 1 -- 172.21.15.32:0/349351406 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff21801ab20 con 0x7ff220136d80 2023-09-30T22:28:39.449 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:39.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff220136d80 msgr2=0x7ff22011f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff220136d80 0x7ff22011f540 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff218016000 tx=0x7ff218004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 shutdown_connections 2023-09-30T22:28:39.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff220136d80 0x7ff22011f540 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff22006a770 0x7ff2201c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 --2- 172.21.15.32:0/349351406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff22006b250 0x7ff2201c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 >> 172.21.15.32:0/349351406 conn(0x7ff220066f50 msgr2=0x7ff22006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 shutdown_connections 2023-09-30T22:28:39.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.451+0000 7ff227357700 1 -- 172.21.15.32:0/349351406 wait complete. 2023-09-30T22:28:39.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.552+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc77806b8f0 con 0x7fc77809f240 2023-09-30T22:28:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.553+0000 7fc76dffb700 1 -- 172.21.15.32:0/3804828003 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc768015030 con 0x7fc77809f240 2023-09-30T22:28:39.553 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 msgr2=0x7fc778123030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 0x7fc778123030 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc768000c00 tx=0x7fc768004750 comp rx=0 tx=0).stop 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 shutdown_connections 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc778134120 0x7fc778123570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc77809e930 0x7fc778122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 --2- 172.21.15.32:0/3804828003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc77809f240 0x7fc778123030 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 >> 172.21.15.32:0/3804828003 conn(0x7fc7780a4210 msgr2=0x7fc77809cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 shutdown_connections 2023-09-30T22:28:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.555+0000 7fc77f9db700 1 -- 172.21.15.32:0/3804828003 wait complete. 2023-09-30T22:28:39.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:39.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.592+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f56ac11fae0 con 0x7f56ac133e30 2023-09-30T22:28:39.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.592+0000 7f5693fff700 1 -- 172.21.15.32:0/1215642661 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56ac11fae0 con 0x7f56ac133e30 2023-09-30T22:28:39.592 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:39.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 msgr2=0x7f56ac09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:39.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 0x7f56ac09c9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c550 tx=0x7f56a000c860 comp rx=0 tx=0).stop 2023-09-30T22:28:39.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 shutdown_connections 2023-09-30T22:28:39.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56ac134260 0x7f56ac09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56ac133420 0x7f56ac09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 --2- 172.21.15.32:0/1215642661 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56ac133e30 0x7f56ac09c9c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:39.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 >> 172.21.15.32:0/1215642661 conn(0x7f56ac0a4210 msgr2=0x7f56ac1302a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:39.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 shutdown_connections 2023-09-30T22:28:39.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:39.595+0000 7f56b190c700 1 -- 172.21.15.32:0/1215642661 wait complete. 2023-09-30T22:28:39.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:40.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 Processor -- start 2023-09-30T22:28:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 -- start start 2023-09-30T22:28:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7009e1d0 0x7fcb70138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb70139170 0x7fcb7013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7009b120 con 0x7fcb7009d6f0 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7006b460 con 0x7fcb7009e1d0 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.164+0000 7fcb74ff7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7009a7e0 con 0x7fcb70139170 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb7009d6f0 0x7fcb7009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33690/0 (socket says 172.21.15.32:33690) 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7009e1d0 0x7fcb70138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb70139170 0x7fcb7013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6e59c700 1 -- 172.21.15.32:0/514749546 learned_addr learned my addr 172.21.15.32:0/514749546 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6dd9b700 1 -- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb70139170 msgr2=0x7fcb7013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6dd9b700 1 --2- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb70139170 0x7fcb7013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6dd9b700 1 -- 172.21.15.32:0/514749546 >> v1:172.21.15.32:6789/0 conn(0x7fcb7009d6f0 legacy=0x7fcb7009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6dd9b700 1 -- 172.21.15.32:0/514749546 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7009a500 con 0x7fcb7009e1d0 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6d59a700 1 -- 172.21.15.32:0/514749546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933204234 0 0) 0x7fcb7009b120 con 0x7fcb7009d6f0 2023-09-30T22:28:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6ed9d700 1 --2- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb70139170 0x7fcb7013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6dd9b700 1 --2- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7009e1d0 0x7fcb70138c30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb64009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60656fb9c57fe33e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6d59a700 1 -- 172.21.15.32:0/514749546 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011830 con 0x7fcb7009e1d0 2023-09-30T22:28:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6d59a700 1 -- 172.21.15.32:0/514749546 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64011990 con 0x7fcb7009e1d0 2023-09-30T22:28:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.165+0000 7fcb6d59a700 1 -- 172.21.15.32:0/514749546 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64011d20 con 0x7fcb7009e1d0 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 -- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7009e1d0 msgr2=0x7fcb70138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7009e1d0 0x7fcb70138c30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb64009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 -- 172.21.15.32:0/514749546 shutdown_connections 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb70139170 0x7fcb7013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/514749546 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7009e1d0 0x7fcb70138c30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 -- 172.21.15.32:0/514749546 >> 172.21.15.32:0/514749546 conn(0x7fcb70066f50 msgr2=0x7fcb7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 -- 172.21.15.32:0/514749546 shutdown_connections 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.166+0000 7fcb74ff7700 1 -- 172.21.15.32:0/514749546 wait complete. 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.167+0000 7fcb74ff7700 1 Processor -- start 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.167+0000 7fcb74ff7700 1 -- start start 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb74ff7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7009d6f0 0x7fcb70120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb74ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 0x7fcb70120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb74ff7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb70139170 0x7fcb701211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb74ff7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70122ba0 con 0x7fcb7009e1d0 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb74ff7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70136610 con 0x7fcb70139170 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb74ff7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7009c710 con 0x7fcb7009d6f0 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 0x7fcb70120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb70139170 0x7fcb701211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 0x7fcb70120cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51428/0 (socket says 172.21.15.32:51428) 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6dd9b700 1 -- 172.21.15.32:0/3864336183 learned_addr learned my addr 172.21.15.32:0/3864336183 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6e59c700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7009d6f0 0x7fcb70120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 -- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7009d6f0 msgr2=0x7fcb70120770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7009d6f0 0x7fcb70120770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 -- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 msgr2=0x7fcb70120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 0x7fcb70120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 -- 172.21.15.32:0/3864336183 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68009a70 con 0x7fcb70139170 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6dd9b700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 0x7fcb70120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.168+0000 7fcb6ed9d700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb70139170 0x7fcb701211f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800efc0 tx=0x7fcb6800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.169+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800ca90 con 0x7fcb70139170 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.169+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6800f040 con 0x7fcb70139170 2023-09-30T22:28:40.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.169+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640094a0 con 0x7fcb70139170 2023-09-30T22:28:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.169+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800ce30 con 0x7fcb70139170 2023-09-30T22:28:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.169+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb70072150 con 0x7fcb70139170 2023-09-30T22:28:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.169+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb68014480 con 0x7fcb70139170 2023-09-30T22:28:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.170+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb6803fcb0 con 0x7fcb70139170 2023-09-30T22:28:40.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.170+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70123e80 con 0x7fcb70139170 2023-09-30T22:28:40.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.175+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb68012910 con 0x7fcb70139170 2023-09-30T22:28:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.378+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcb70074ea0 con 0x7fcb70139170 2023-09-30T22:28:40.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.378+0000 7fcb5effd700 1 -- 172.21.15.32:0/3864336183 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb68012340 con 0x7fcb70139170 2023-09-30T22:28:40.379 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.380+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb70139170 msgr2=0x7fcb701211f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.380+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb70139170 0x7fcb701211f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800efc0 tx=0x7fcb6800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 shutdown_connections 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb70139170 0x7fcb701211f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7009d6f0 0x7fcb70120770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 --2- 172.21.15.32:0/3864336183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7009e1d0 0x7fcb70120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 >> 172.21.15.32:0/3864336183 conn(0x7fcb70066f50 msgr2=0x7fcb70136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 shutdown_connections 2023-09-30T22:28:40.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.381+0000 7fcb74ff7700 1 -- 172.21.15.32:0/3864336183 wait complete. 2023-09-30T22:28:40.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:40.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:40.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.609+0000 7fcbbcf11700 1 Processor -- start 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.609+0000 7fcbbcf11700 1 -- start start 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbbcf11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbb806a770 0x7fcbb806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbbcf11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 0x7fcbb8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbbcf11700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb8131b40 0x7fcbb8131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbbcf11700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb809ece0 con 0x7fcbb806a770 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbbcf11700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb809e760 con 0x7fcbb8131b40 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbbcf11700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb809e040 con 0x7fcbb806b080 2023-09-30T22:28:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbadd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 0x7fcbb8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbb806a770 0x7fcbb806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcbb806a770 0x7fcbb806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52780/0 (socket says 172.21.15.32:52780) 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 -- 172.21.15.32:0/1425800296 learned_addr learned my addr 172.21.15.32:0/1425800296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb6d9d700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb8131b40 0x7fcbb8131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbadd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 0x7fcbb8131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46542/0 (socket says 172.21.15.32:46542) 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 -- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 msgr2=0x7fcbb8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 0x7fcbb8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 -- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb8131b40 msgr2=0x7fcbb8131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb8131b40 0x7fcbb8131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb659c700 1 -- 172.21.15.32:0/1425800296 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb809f630 con 0x7fcbb806a770 2023-09-30T22:28:40.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbadd9b700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 0x7fcbb8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.610+0000 7fcbb6d9d700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb8131b40 0x7fcbb8131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb806ab40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcba80098f0 tx=0x7fcba8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899e8d936a49bfd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbb5d9b700 1 -- 172.21.15.32:0/1425800296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8011830 con 0x7fcbb806a770 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbb5d9b700 1 -- 172.21.15.32:0/1425800296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8011990 con 0x7fcbb806a770 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbb5d9b700 1 -- 172.21.15.32:0/1425800296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8011d20 con 0x7fcbb806a770 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 msgr2=0x7fcbb806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb806ab40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcba80098f0 tx=0x7fcba8009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1425800296 shutdown_connections 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb8131b40 0x7fcbb8131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb806ab40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1425800296 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb806b080 0x7fcbb8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1425800296 >> 172.21.15.32:0/1425800296 conn(0x7fcbb8066f50 msgr2=0x7fcbb809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1425800296 shutdown_connections 2023-09-30T22:28:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.611+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1425800296 wait complete. 2023-09-30T22:28:40.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.613+0000 7fcbbcf11700 1 Processor -- start 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.613+0000 7fcbbcf11700 1 -- start start 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbbcf11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb81c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbbcf11700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 0x7fcbb81c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbbcf11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb8131b40 0x7fcbb81206d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbbcf11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb81311e0 con 0x7fcbb806a770 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbbcf11700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8123bd0 con 0x7fcbb806b080 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbbcf11700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8124270 con 0x7fcbb8131b40 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbadd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 0x7fcbb81c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbadd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 0x7fcbb81c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34332/0 (socket says 172.21.15.32:34332) 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbadd9b700 1 -- 172.21.15.32:0/1580201520 learned_addr learned my addr 172.21.15.32:0/1580201520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:40.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbb6d9d700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb8131b40 0x7fcbb81206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.614+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb81c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb659c700 1 -- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb8131b40 msgr2=0x7fcbb81206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb8131b40 0x7fcbb81206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb659c700 1 -- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 msgr2=0x7fcbb81c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 0x7fcbb81c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb659c700 1 -- 172.21.15.32:0/1580201520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00094a0 con 0x7fcbb806a770 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbadd9b700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 0x7fcbb81c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb6d9d700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb8131b40 0x7fcbb81206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbb659c700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb81c61f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcba8016000 tx=0x7fcba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004450 con 0x7fcbb806a770 2023-09-30T22:28:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba801f070 con 0x7fcbb806a770 2023-09-30T22:28:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb0009a70 con 0x7fcbb806a770 2023-09-30T22:28:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80038e0 con 0x7fcbb806a770 2023-09-30T22:28:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.615+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba8009550 con 0x7fcbb806a770 2023-09-30T22:28:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.616+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcba8006ab0 con 0x7fcbb806a770 2023-09-30T22:28:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.616+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcba8021070 con 0x7fcbb806a770 2023-09-30T22:28:40.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.616+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb8074ea0 con 0x7fcbb806a770 2023-09-30T22:28:40.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.621+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcba802f990 con 0x7fcbb806a770 2023-09-30T22:28:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.712+0000 7f6b44cca700 1 Processor -- start 2023-09-30T22:28:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.712+0000 7f6b44cca700 1 -- start start 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b44cca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b44cca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b400a1340 0x7f6b40133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b44cca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4009edd0 con 0x7f6b40133fa0 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b44cca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4009e1f0 con 0x7f6b400a1340 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b44cca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4009dc50 con 0x7f6b400a0a30 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b3e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b3e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b400a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46562/0 (socket says 172.21.15.32:46562) 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b3e59c700 1 -- 172.21.15.32:0/1557780509 learned_addr learned my addr 172.21.15.32:0/1557780509 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b37fff700 1 --2- 172.21.15.32:0/1557780509 >> v2:172.21.15.32:3300/0 conn(0x7f6b400a1340 0x7f6b40133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b3dd9b700 1 -- 172.21.15.32:0/1557780509 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154180262 0 0) 0x7f6b4009edd0 con 0x7f6b40133fa0 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b3dd9b700 1 -- 172.21.15.32:0/1557780509 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2c0035b0 con 0x7f6b40133fa0 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.713+0000 7f6b3e59c700 1 -- 172.21.15.32:0/1557780509 >> v2:172.21.15.32:3300/0 conn(0x7f6b400a1340 msgr2=0x7f6b40133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3e59c700 1 --2- 172.21.15.32:0/1557780509 >> v2:172.21.15.32:3300/0 conn(0x7f6b400a1340 0x7f6b40133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3e59c700 1 -- 172.21.15.32:0/1557780509 >> v1:172.21.15.32:6789/0 conn(0x7f6b40133fa0 legacy=0x7f6b4013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3e59c700 1 -- 172.21.15.32:0/1557780509 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400a1fe0 con 0x7f6b400a0a30 2023-09-30T22:28:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b37fff700 1 --2- 172.21.15.32:0/1557780509 >> v2:172.21.15.32:3300/0 conn(0x7f6b400a1340 0x7f6b40133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3e59c700 1 --2- 172.21.15.32:0/1557780509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b400a0e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6b300098f0 tx=0x7f6b30009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7778927b3ea36f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3dd9b700 1 -- 172.21.15.32:0/1557780509 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011830 con 0x7f6b400a0a30 2023-09-30T22:28:40.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3dd9b700 1 -- 172.21.15.32:0/1557780509 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30011990 con 0x7f6b400a0a30 2023-09-30T22:28:40.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.714+0000 7f6b3dd9b700 1 -- 172.21.15.32:0/1557780509 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011d20 con 0x7f6b400a0a30 2023-09-30T22:28:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 -- 172.21.15.32:0/1557780509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 msgr2=0x7f6b400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1557780509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b400a0e00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6b300098f0 tx=0x7f6b30009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 -- 172.21.15.32:0/1557780509 shutdown_connections 2023-09-30T22:28:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1557780509 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b400a0e00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1557780509 >> v2:172.21.15.32:3300/0 conn(0x7f6b400a1340 0x7f6b40133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 -- 172.21.15.32:0/1557780509 >> 172.21.15.32:0/1557780509 conn(0x7f6b400a6550 msgr2=0x7f6b400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 -- 172.21.15.32:0/1557780509 shutdown_connections 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.715+0000 7f6b44cca700 1 -- 172.21.15.32:0/1557780509 wait complete. 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.716+0000 7f6b44cca700 1 Processor -- start 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.716+0000 7f6b44cca700 1 -- start start 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b44cca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b40122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b44cca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 0x7f6b40122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b44cca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b40133fa0 0x7f6b401232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b44cca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401245b0 con 0x7f6b40133fa0 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b44cca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b400a3580 con 0x7f6b400a1340 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b44cca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40126340 con 0x7f6b400a0a30 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 0x7f6b40122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b40133fa0 0x7f6b401232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 0x7f6b40122d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34334/0 (socket says 172.21.15.32:34334) 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 -- 172.21.15.32:0/1409785515 learned_addr learned my addr 172.21.15.32:0/1409785515 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b3e59c700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b40122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 -- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 msgr2=0x7f6b40122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b40122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 -- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b40133fa0 msgr2=0x7f6b401232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b40133fa0 0x7f6b401232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b37fff700 1 -- 172.21.15.32:0/1409785515 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280094a0 con 0x7f6b400a1340 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b3e59c700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b40122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.717+0000 7f6b3ed9d700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b40133fa0 0x7f6b401232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.718+0000 7f6b37fff700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 0x7f6b40122d60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ebe0 tx=0x7f6b2800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.718+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800c850 con 0x7f6b400a1340 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.718+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2800c9b0 con 0x7f6b400a1340 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.718+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b30009550 con 0x7f6b400a1340 2023-09-30T22:28:40.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.718+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800cd40 con 0x7f6b400a1340 2023-09-30T22:28:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.718+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b40133340 con 0x7f6b400a1340 2023-09-30T22:28:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.719+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b28013470 con 0x7f6b400a1340 2023-09-30T22:28:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.719+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b400a1d40 con 0x7f6b400a1340 2023-09-30T22:28:40.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.719+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b2803fae0 con 0x7f6b400a1340 2023-09-30T22:28:40.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.724+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b2800cea0 con 0x7f6b400a1340 2023-09-30T22:28:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 Processor -- start 2023-09-30T22:28:40.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 -- start start 2023-09-30T22:28:40.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d7409e930 0x7f1d7409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409f240 0x7f1d741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d74070940 con 0x7f1d7409f650 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7406fd60 con 0x7f1d7409e930 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.763+0000 7f1d79c14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7406fa80 con 0x7f1d7409f240 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d73fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d7409f650 0x7f1d74131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43658/0 (socket says 172.21.15.32:43658) 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d73fff700 1 -- 172.21.15.32:0/111211536 learned_addr learned my addr 172.21.15.32:0/111211536 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d72ffd700 1 --2- 172.21.15.32:0/111211536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409f240 0x7f1d741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d737fe700 1 --2- 172.21.15.32:0/111211536 >> v2:172.21.15.32:3300/0 conn(0x7f1d7409e930 0x7f1d7409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d72ffd700 1 -- 172.21.15.32:0/111211536 >> v2:172.21.15.32:3300/0 conn(0x7f1d7409e930 msgr2=0x7f1d7409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d72ffd700 1 --2- 172.21.15.32:0/111211536 >> v2:172.21.15.32:3300/0 conn(0x7f1d7409e930 0x7f1d7409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d72ffd700 1 -- 172.21.15.32:0/111211536 >> v1:172.21.15.32:6789/0 conn(0x7f1d7409f650 legacy=0x7f1d74131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d72ffd700 1 -- 172.21.15.32:0/111211536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7409fb60 con 0x7f1d7409f240 2023-09-30T22:28:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d727fc700 1 -- 172.21.15.32:0/111211536 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754647285 0 0) 0x7f1d74070940 con 0x7f1d7409f650 2023-09-30T22:28:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d737fe700 1 --2- 172.21.15.32:0/111211536 >> v2:172.21.15.32:3300/0 conn(0x7f1d7409e930 0x7f1d7409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d72ffd700 1 --2- 172.21.15.32:0/111211536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409f240 0x7f1d741317f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009840 tx=0x7f1d68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf121ac77ef5fad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.764+0000 7f1d727fc700 1 -- 172.21.15.32:0/111211536 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68011830 con 0x7f1d7409f240 2023-09-30T22:28:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d727fc700 1 -- 172.21.15.32:0/111211536 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d68011990 con 0x7f1d7409f240 2023-09-30T22:28:40.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d727fc700 1 -- 172.21.15.32:0/111211536 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68011d20 con 0x7f1d7409f240 2023-09-30T22:28:40.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 -- 172.21.15.32:0/111211536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409f240 msgr2=0x7f1d741317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 --2- 172.21.15.32:0/111211536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409f240 0x7f1d741317f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009840 tx=0x7f1d68009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:40.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 -- 172.21.15.32:0/111211536 shutdown_connections 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 --2- 172.21.15.32:0/111211536 >> v2:172.21.15.32:3300/0 conn(0x7f1d7409e930 0x7f1d7409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 --2- 172.21.15.32:0/111211536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409f240 0x7f1d741317f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 -- 172.21.15.32:0/111211536 >> 172.21.15.32:0/111211536 conn(0x7f1d740a4210 msgr2=0x7f1d740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 -- 172.21.15.32:0/111211536 shutdown_connections 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.765+0000 7f1d79c14700 1 -- 172.21.15.32:0/111211536 wait complete. 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.766+0000 7f1d79c14700 1 Processor -- start 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 -- start start 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409e930 0x7f1d741c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 0x7f1d741c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7409f650 0x7f1d741c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a7410 con 0x7f1d7409f240 2023-09-30T22:28:40.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741a75f0 con 0x7f1d7409f650 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d79c14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741c3880 con 0x7f1d7409e930 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d72ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 0x7f1d741c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d72ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 0x7f1d741c3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52840/0 (socket says 172.21.15.32:52840) 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d72ffd700 1 -- 172.21.15.32:0/1531280238 learned_addr learned my addr 172.21.15.32:0/1531280238 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d737fe700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409e930 0x7f1d741c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.767+0000 7f1d73fff700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7409f650 0x7f1d741c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d737fe700 1 -- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7409f650 msgr2=0x7f1d741c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d737fe700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7409f650 0x7f1d741c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d737fe700 1 -- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 msgr2=0x7f1d741c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d737fe700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 0x7f1d741c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d737fe700 1 -- 172.21.15.32:0/1531280238 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d640094a0 con 0x7f1d7409e930 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d72ffd700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 0x7f1d741c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d737fe700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409e930 0x7f1d741c2e00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400ea10 tx=0x7f1d6400ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400eeb0 con 0x7f1d7409e930 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6400c850 con 0x7f1d7409e930 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d680094a0 con 0x7f1d7409e930 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400cc00 con 0x7f1d7409e930 2023-09-30T22:28:40.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.768+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d740a0190 con 0x7f1d7409e930 2023-09-30T22:28:40.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.769+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d741a6990 con 0x7f1d7409e930 2023-09-30T22:28:40.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.769+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d6400cd60 con 0x7f1d7409e930 2023-09-30T22:28:40.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.769+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d6403f710 con 0x7f1d7409e930 2023-09-30T22:28:40.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.774+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d6401be10 con 0x7f1d7409e930 2023-09-30T22:28:40.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.834+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcbb81c6c70 con 0x7fcbb806a770 2023-09-30T22:28:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.834+0000 7fcbaf7fe700 1 -- 172.21.15.32:0/1580201520 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcba802b030 con 0x7fcbb806a770 2023-09-30T22:28:40.835 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 msgr2=0x7fcbb81c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb81c61f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcba8016000 tx=0x7fcba8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 shutdown_connections 2023-09-30T22:28:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcbb8131b40 0x7fcbb81206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbb806a770 0x7fcbb81c61f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 --2- 172.21.15.32:0/1580201520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcbb806b080 0x7fcbb81c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 >> 172.21.15.32:0/1580201520 conn(0x7fcbb8066f50 msgr2=0x7fcbb80700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 shutdown_connections 2023-09-30T22:28:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.837+0000 7fcbbcf11700 1 -- 172.21.15.32:0/1580201520 wait complete. 2023-09-30T22:28:40.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:40.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.932+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b40074ea0 con 0x7f6b400a1340 2023-09-30T22:28:40.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.933+0000 7f6b36ffd700 1 -- 172.21.15.32:0/1409785515 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b28017070 con 0x7f6b400a1340 2023-09-30T22:28:40.933 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 msgr2=0x7f6b40122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 0x7f6b40122d60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800ebe0 tx=0x7f6b2800eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 shutdown_connections 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b40133fa0 0x7f6b401232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b400a0a30 0x7f6b40122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 --2- 172.21.15.32:0/1409785515 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b400a1340 0x7f6b40122d60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 >> 172.21.15.32:0/1409785515 conn(0x7f6b400a6550 msgr2=0x7f6b400a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 shutdown_connections 2023-09-30T22:28:40.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.935+0000 7f6b44cca700 1 -- 172.21.15.32:0/1409785515 wait complete. 2023-09-30T22:28:40.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:40.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.983+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1d74074ea0 con 0x7f1d7409e930 2023-09-30T22:28:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.984+0000 7f1d5bfff700 1 -- 172.21.15.32:0/1531280238 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d6401b3b0 con 0x7f1d7409e930 2023-09-30T22:28:40.984 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:40.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409e930 msgr2=0x7f1d741c2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409e930 0x7f1d741c2e00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400ea10 tx=0x7f1d6400ed20 comp rx=0 tx=0).stop 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 shutdown_connections 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7409f650 0x7f1d741c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7409e930 0x7f1d741c2e00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 --2- 172.21.15.32:0/1531280238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7409f240 0x7f1d741c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.986+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 >> 172.21.15.32:0/1531280238 conn(0x7f1d740a4210 msgr2=0x7f1d741b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.987+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 shutdown_connections 2023-09-30T22:28:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:40.987+0000 7f1d79c14700 1 -- 172.21.15.32:0/1531280238 wait complete. 2023-09-30T22:28:41.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:41.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.541+0000 7f764082c700 1 Processor -- start 2023-09-30T22:28:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f764082c700 1 -- start start 2023-09-30T22:28:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f764082c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f764082c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f764082c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c1313c0 con 0x7f763c099a40 2023-09-30T22:28:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f764082c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c06e8c0 con 0x7f763c131810 2023-09-30T22:28:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f764082c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f763c071d20 con 0x7f763c09a580 2023-09-30T22:28:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f763a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f763ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:41.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f7639d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f763c099a40 0x7f763c099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43662/0 (socket says 172.21.15.32:43662) 2023-09-30T22:28:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.542+0000 7f7639d9b700 1 -- 172.21.15.32:0/283190484 learned_addr learned my addr 172.21.15.32:0/283190484 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763959a700 1 -- 172.21.15.32:0/283190484 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84058347 0 0) 0x7f763c1313c0 con 0x7f763c099a40 2023-09-30T22:28:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763ad9d700 1 -- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 msgr2=0x7f763c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763ad9d700 1 --2- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763ad9d700 1 -- 172.21.15.32:0/283190484 >> v1:172.21.15.32:6789/0 conn(0x7f763c099a40 legacy=0x7f763c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763ad9d700 1 -- 172.21.15.32:0/283190484 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0721a0 con 0x7f763c09a580 2023-09-30T22:28:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763a59c700 1 --2- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763ad9d700 1 --2- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c139660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7634009dc0 tx=0x7f763400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ca35d5140629e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763959a700 1 -- 172.21.15.32:0/283190484 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400e070 con 0x7f763c09a580 2023-09-30T22:28:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763959a700 1 -- 172.21.15.32:0/283190484 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763400bb20 con 0x7f763c09a580 2023-09-30T22:28:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.543+0000 7f763959a700 1 -- 172.21.15.32:0/283190484 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634012430 con 0x7f763c09a580 2023-09-30T22:28:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 -- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 msgr2=0x7f763c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 --2- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c139660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7634009dc0 tx=0x7f763400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 -- 172.21.15.32:0/283190484 shutdown_connections 2023-09-30T22:28:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 --2- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c139660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 --2- 172.21.15.32:0/283190484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 -- 172.21.15.32:0/283190484 >> 172.21.15.32:0/283190484 conn(0x7f763c0a2aa0 msgr2=0x7f763c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 -- 172.21.15.32:0/283190484 shutdown_connections 2023-09-30T22:28:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.544+0000 7f764082c700 1 -- 172.21.15.32:0/283190484 wait complete. 2023-09-30T22:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.545+0000 7f764082c700 1 Processor -- start 2023-09-30T22:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.545+0000 7f764082c700 1 -- start start 2023-09-30T22:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763c099a40 0x7f763c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:41.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c070a70 con 0x7f763c099a40 2023-09-30T22:28:41.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c06eb00 con 0x7f763c131810 2023-09-30T22:28:41.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0a1af0 con 0x7f763c09a580 2023-09-30T22:28:41.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f763a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763c099a40 0x7f763c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c06fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46590/0 (socket says 172.21.15.32:46590) 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 -- 172.21.15.32:0/177206528 learned_addr learned my addr 172.21.15.32:0/177206528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f763ad9d700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 -- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 msgr2=0x7f763c070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 -- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763c099a40 msgr2=0x7f763c06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763c099a40 0x7f763c06f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 -- 172.21.15.32:0/177206528 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c0094a0 con 0x7f763c09a580 2023-09-30T22:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f7639d9b700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c06fb50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7630001eb0 tx=0x7f763000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630004c80 con 0x7f763c09a580 2023-09-30T22:28:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7630004de0 con 0x7f763c09a580 2023-09-30T22:28:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630007310 con 0x7f763c09a580 2023-09-30T22:28:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.546+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7634009a70 con 0x7f763c09a580 2023-09-30T22:28:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.547+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c120c00 con 0x7f763c09a580 2023-09-30T22:28:41.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.547+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7630007470 con 0x7f763c09a580 2023-09-30T22:28:41.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.547+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f763003f3d0 con 0x7f763c09a580 2023-09-30T22:28:41.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.548+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c0710f0 con 0x7f763c09a580 2023-09-30T22:28:41.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.552+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76300123b0 con 0x7f763c09a580 2023-09-30T22:28:41.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.759+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f763c074ea0 con 0x7f763c09a580 2023-09-30T22:28:41.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.759+0000 7f762affd700 1 -- 172.21.15.32:0/177206528 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7630011360 con 0x7f763c09a580 2023-09-30T22:28:41.759 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 msgr2=0x7f763c06fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c06fb50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7630001eb0 tx=0x7f763000c800 comp rx=0 tx=0).stop 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 shutdown_connections 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763c131810 0x7f763c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763c099a40 0x7f763c06f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 --2- 172.21.15.32:0/177206528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763c09a580 0x7f763c06fb50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 >> 172.21.15.32:0/177206528 conn(0x7f763c0a2aa0 msgr2=0x7f763c1b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 shutdown_connections 2023-09-30T22:28:41.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.761+0000 7f764082c700 1 -- 172.21.15.32:0/177206528 wait complete. 2023-09-30T22:28:41.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:41.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:41.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:41.999+0000 7fe80307a700 1 Processor -- start 2023-09-30T22:28:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe80307a700 1 -- start start 2023-09-30T22:28:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe80307a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe80307a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e000 0x7fe7fc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe80307a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc09f130 con 0x7fe7fc09e410 2023-09-30T22:28:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe80307a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc09f520 con 0x7fe7fc09d6f0 2023-09-30T22:28:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe80307a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc09e930 con 0x7fe7fc09e000 2023-09-30T22:28:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe800e16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe801617700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7fc09e410 0x7fe7fc130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43674/0 (socket says 172.21.15.32:43674) 2023-09-30T22:28:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe801617700 1 -- 172.21.15.32:0/476625258 learned_addr learned my addr 172.21.15.32:0/476625258 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.000+0000 7fe7f3fff700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e000 0x7fe7fc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe800e16700 1 -- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e000 msgr2=0x7fe7fc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe800e16700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e000 0x7fe7fc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe7f37fe700 1 -- 172.21.15.32:0/476625258 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403450388 0 0) 0x7fe7fc09f130 con 0x7fe7fc09e410 2023-09-30T22:28:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe800e16700 1 -- 172.21.15.32:0/476625258 >> v1:172.21.15.32:6789/0 conn(0x7fe7fc09e410 legacy=0x7fe7fc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe800e16700 1 -- 172.21.15.32:0/476625258 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc09b2d0 con 0x7fe7fc09d6f0 2023-09-30T22:28:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe7f3fff700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e000 0x7fe7fc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe800e16700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009870 tx=0x7fe7f8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bddb87cd37f639d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe7f37fe700 1 -- 172.21.15.32:0/476625258 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011830 con 0x7fe7fc09d6f0 2023-09-30T22:28:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe7f37fe700 1 -- 172.21.15.32:0/476625258 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8011990 con 0x7fe7fc09d6f0 2023-09-30T22:28:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.001+0000 7fe7f37fe700 1 -- 172.21.15.32:0/476625258 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011d20 con 0x7fe7fc09d6f0 2023-09-30T22:28:42.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 -- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 msgr2=0x7fe7fc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009870 tx=0x7fe7f8009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 -- 172.21.15.32:0/476625258 shutdown_connections 2023-09-30T22:28:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 --2- 172.21.15.32:0/476625258 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e000 0x7fe7fc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 -- 172.21.15.32:0/476625258 >> 172.21.15.32:0/476625258 conn(0x7fe7fc066f50 msgr2=0x7fe7fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 -- 172.21.15.32:0/476625258 shutdown_connections 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.002+0000 7fe80307a700 1 -- 172.21.15.32:0/476625258 wait complete. 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.003+0000 7fe80307a700 1 Processor -- start 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.003+0000 7fe80307a700 1 -- start start 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe80307a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe80307a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 0x7fe7fc06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe80307a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e410 0x7fe7fc06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe80307a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc125680 con 0x7fe7fc09e000 2023-09-30T22:28:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe80307a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc09c700 con 0x7fe7fc09d6f0 2023-09-30T22:28:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe80307a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0721a0 con 0x7fe7fc09e410 2023-09-30T22:28:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe7f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 0x7fe7fc06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe800e16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e410 0x7fe7fc06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe7f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 0x7fe7fc06ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52868/0 (socket says 172.21.15.32:52868) 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe800e16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc06e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34374/0 (socket says 172.21.15.32:34374) 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe7f3fff700 1 -- 172.21.15.32:0/3278737419 learned_addr learned my addr 172.21.15.32:0/3278737419 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 -- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 msgr2=0x7fe7fc06e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc06e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 -- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 msgr2=0x7fe7fc06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 0x7fe7fc06ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 -- 172.21.15.32:0/3278737419 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0094a0 con 0x7fe7fc09e410 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe800e16700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc06e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe7f3fff700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 0x7fe7fc06ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe801617700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e410 0x7fe7fc06f2c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400cbe0 tx=0x7fe7f400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.004+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400aa40 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.005+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400abc0 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.005+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f80094a0 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.005+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40070d0 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.005+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc124700 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.006+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe7f4007230 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.006+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc074ea0 con 0x7fe7fc09e410 2023-09-30T22:28:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.006+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe7f403f390 con 0x7fe7fc09e410 2023-09-30T22:28:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.011+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7f40074e0 con 0x7fe7fc09e410 2023-09-30T22:28:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.120+0000 7f31aeb2b700 1 Processor -- start 2023-09-30T22:28:42.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31aeb2b700 1 -- start start 2023-09-30T22:28:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31aeb2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 0x7f31a8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31aeb2b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31a8136fe0 0x7f31a81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31aeb2b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a8070940 con 0x7f31a809e930 2023-09-30T22:28:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31aeb2b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a806fd60 con 0x7f31a8136fe0 2023-09-30T22:28:42.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31aeb2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a806fa80 con 0x7f31a809f410 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31a7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 0x7f31a8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31ac8c7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31a809e930 0x7f31a809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43684/0 (socket says 172.21.15.32:43684) 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31ac8c7700 1 -- 172.21.15.32:0/23676586 learned_addr learned my addr 172.21.15.32:0/23676586 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31a7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 0x7f31a8136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34380/0 (socket says 172.21.15.32:34380) 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.121+0000 7f31ad0c8700 1 --2- 172.21.15.32:0/23676586 >> v2:172.21.15.32:3300/0 conn(0x7f31a8136fe0 0x7f31a81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31ad0c8700 1 -- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 msgr2=0x7f31a8136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31ad0c8700 1 --2- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 0x7f31a8136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31ad0c8700 1 -- 172.21.15.32:0/23676586 >> v1:172.21.15.32:6789/0 conn(0x7f31a809e930 legacy=0x7f31a809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31a77fe700 1 -- 172.21.15.32:0/23676586 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721973979 0 0) 0x7f31a8070940 con 0x7f31a809e930 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31ad0c8700 1 -- 172.21.15.32:0/23676586 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a809fb60 con 0x7f31a8136fe0 2023-09-30T22:28:42.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31a7fff700 1 --2- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 0x7f31a8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31ad0c8700 1 --2- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a8136fe0 0x7f31a81393c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f31a0010350 tx=0x7f31a0010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8e8e4ba00976a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31a77fe700 1 -- 172.21.15.32:0/23676586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a001b680 con 0x7f31a8136fe0 2023-09-30T22:28:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31a77fe700 1 -- 172.21.15.32:0/23676586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a001b7e0 con 0x7f31a8136fe0 2023-09-30T22:28:42.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.122+0000 7f31a77fe700 1 -- 172.21.15.32:0/23676586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a001bb70 con 0x7f31a8136fe0 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 -- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a8136fe0 msgr2=0x7f31a81393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a8136fe0 0x7f31a81393c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f31a0010350 tx=0x7f31a0010660 comp rx=0 tx=0).stop 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 -- 172.21.15.32:0/23676586 shutdown_connections 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a8136fe0 0x7f31a81393c0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/23676586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809f410 0x7f31a8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 -- 172.21.15.32:0/23676586 >> 172.21.15.32:0/23676586 conn(0x7f31a80a4210 msgr2=0x7f31a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 -- 172.21.15.32:0/23676586 shutdown_connections 2023-09-30T22:28:42.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.123+0000 7f31aeb2b700 1 -- 172.21.15.32:0/23676586 wait complete. 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.124+0000 7f31aeb2b700 1 Processor -- start 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 -- start start 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809e930 0x7f31a811b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 0x7f31a811b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31a8136fe0 0x7f31a811bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a811c6d0 con 0x7f31a809f410 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8071d20 con 0x7f31a809e930 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.125+0000 7f31aeb2b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80a31d0 con 0x7f31a8136fe0 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 0x7f31a811b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 0x7f31a811b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52880/0 (socket says 172.21.15.32:52880) 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 -- 172.21.15.32:0/3896681890 learned_addr learned my addr 172.21.15.32:0/3896681890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31ac8c7700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809e930 0x7f31a811b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 -- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31a8136fe0 msgr2=0x7f31a811bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31a8136fe0 0x7f31a811bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 -- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809e930 msgr2=0x7f31a811b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809e930 0x7f31a811b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 -- 172.21.15.32:0/3896681890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940094a0 con 0x7f31a809f410 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a7fff700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 0x7f31a811b7b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f319400ec10 tx=0x7f319400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319400c8b0 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319400ca10 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a000bc50 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.126+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319400cd80 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.127+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a811ee30 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.128+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3194013470 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.128+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3194045060 con 0x7f31a809f410 2023-09-30T22:28:42.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.128+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a811e700 con 0x7f31a809f410 2023-09-30T22:28:42.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.133+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3194017070 con 0x7f31a809f410 2023-09-30T22:28:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.154+0000 7f468aca7700 1 Processor -- start 2023-09-30T22:28:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.154+0000 7f468aca7700 1 -- start start 2023-09-30T22:28:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f468aca7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f46840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f468aca7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f46840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f468aca7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46840a2a70 con 0x7f4684133930 2023-09-30T22:28:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f468aca7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46840a01d0 con 0x7f468409d630 2023-09-30T22:28:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f468aca7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f468409e2b0 con 0x7f46840a0a30 2023-09-30T22:28:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f4688a43700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f46840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f4688a43700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f46840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46630/0 (socket says 172.21.15.32:46630) 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f4688a43700 1 -- 172.21.15.32:0/2173418868 learned_addr learned my addr 172.21.15.32:0/2173418868 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.155+0000 7f4683fff700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f46840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f4683fff700 1 -- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 msgr2=0x7f46840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f4683fff700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f46840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f4683fff700 1 -- 172.21.15.32:0/2173418868 >> v1:172.21.15.32:6789/0 conn(0x7f4684133930 legacy=0x7f4684133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f4683fff700 1 -- 172.21.15.32:0/2173418868 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468409dc50 con 0x7f468409d630 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f4688a43700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f46840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f4683fff700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f46840a1340 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4678009870 tx=0x7f4678009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be01a847c5a3884 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f46837fe700 1 -- 172.21.15.32:0/2173418868 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678011830 con 0x7f468409d630 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f46837fe700 1 -- 172.21.15.32:0/2173418868 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678011990 con 0x7f468409d630 2023-09-30T22:28:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.156+0000 7f46837fe700 1 -- 172.21.15.32:0/2173418868 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678011d20 con 0x7f468409d630 2023-09-30T22:28:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 -- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 msgr2=0x7f46840a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f46840a1340 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4678009870 tx=0x7f4678009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 -- 172.21.15.32:0/2173418868 shutdown_connections 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f46840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 --2- 172.21.15.32:0/2173418868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f46840a1340 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 -- 172.21.15.32:0/2173418868 >> 172.21.15.32:0/2173418868 conn(0x7f46840a3050 msgr2=0x7f46840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 -- 172.21.15.32:0/2173418868 shutdown_connections 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.157+0000 7f468aca7700 1 -- 172.21.15.32:0/2173418868 wait complete. 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.158+0000 7f468aca7700 1 Processor -- start 2023-09-30T22:28:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.158+0000 7f468aca7700 1 -- start start 2023-09-30T22:28:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f468aca7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f468411f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f468aca7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f468411fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f468aca7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4684133930 0x7f46841c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f468aca7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840a50d0 con 0x7f4684133930 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f468aca7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46841222e0 con 0x7f468409d630 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f468aca7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4684122800 con 0x7f46840a0a30 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f4688a43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f468411f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f4683fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f468411fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f4688a43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f468411f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34390/0 (socket says 172.21.15.32:34390) 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.159+0000 7f4688a43700 1 -- 172.21.15.32:0/3239895166 learned_addr learned my addr 172.21.15.32:0/3239895166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4689244700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4684133930 0x7f46841c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4683fff700 1 -- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 msgr2=0x7f468411f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4683fff700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f468411f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4683fff700 1 -- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4684133930 msgr2=0x7f46841c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4683fff700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4684133930 0x7f46841c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4683fff700 1 -- 172.21.15.32:0/3239895166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674009550 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f4683fff700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f468411fa40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4678016000 tx=0x7f4678004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678004240 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467801f070 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46780094a0 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46780045e0 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.160+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4684120ea0 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.161+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4678021030 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.161+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4678040760 con 0x7f46840a0a30 2023-09-30T22:28:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.162+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4684122420 con 0x7f46840a0a30 2023-09-30T22:28:42.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.166+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4684122420 con 0x7f46840a0a30 2023-09-30T22:28:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.306+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe7fc06c360 con 0x7fe7fc09e410 2023-09-30T22:28:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.307+0000 7fe7f17fa700 1 -- 172.21.15.32:0/3278737419 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe7f4018020 con 0x7fe7fc09e410 2023-09-30T22:28:42.307 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e410 msgr2=0x7fe7fc06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e410 0x7fe7fc06f2c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400cbe0 tx=0x7fe7f400cef0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 shutdown_connections 2023-09-30T22:28:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe7fc09e410 0x7fe7fc06f2c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7fc09d6f0 0x7fe7fc06e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 --2- 172.21.15.32:0/3278737419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7fc09e000 0x7fe7fc06ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 >> 172.21.15.32:0/3278737419 conn(0x7fe7fc066f50 msgr2=0x7fe7fc070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 shutdown_connections 2023-09-30T22:28:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.309+0000 7fe80307a700 1 -- 172.21.15.32:0/3278737419 wait complete. 2023-09-30T22:28:42.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.373+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4684122420 con 0x7f46840a0a30 2023-09-30T22:28:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.373+0000 7f46817fa700 1 -- 172.21.15.32:0/3239895166 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4684122420 con 0x7f46840a0a30 2023-09-30T22:28:42.374 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.375+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 msgr2=0x7f468411fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.375+0000 7f468aca7700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f468411fa40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4678016000 tx=0x7f4678004750 comp rx=0 tx=0).stop 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 shutdown_connections 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4684133930 0x7f46841c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f468409d630 0x7f468411f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 --2- 172.21.15.32:0/3239895166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f46840a0a30 0x7f468411fa40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 >> 172.21.15.32:0/3239895166 conn(0x7f46840a3050 msgr2=0x7f46841bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 shutdown_connections 2023-09-30T22:28:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.376+0000 7f468aca7700 1 -- 172.21.15.32:0/3239895166 wait complete. 2023-09-30T22:28:42.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.390+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31a8074ea0 con 0x7f31a809f410 2023-09-30T22:28:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.391+0000 7f31a57fa700 1 -- 172.21.15.32:0/3896681890 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3194017020 con 0x7f31a809f410 2023-09-30T22:28:42.391 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 msgr2=0x7f31a811b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 0x7f31a811b7b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f319400ec10 tx=0x7f319400ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 shutdown_connections 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31a8136fe0 0x7f31a811bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a809e930 0x7f31a811b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 --2- 172.21.15.32:0/3896681890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a809f410 0x7f31a811b7b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 >> 172.21.15.32:0/3896681890 conn(0x7f31a80a4210 msgr2=0x7f31a806bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 shutdown_connections 2023-09-30T22:28:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.393+0000 7f31aeb2b700 1 -- 172.21.15.32:0/3896681890 wait complete. 2023-09-30T22:28:42.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:42.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.922+0000 7fd87d85e700 1 Processor -- start 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd87d85e700 1 -- start start 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd87d85e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd87809ace0 0x7fd878138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd87d85e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809e1d0 0x7fd87813b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd87d85e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87809b150 con 0x7fd87809d6f0 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd87d85e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87809a500 con 0x7fd87809ace0 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd87d85e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd87809f720 con 0x7fd87809e1d0 2023-09-30T22:28:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd8767fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd87809ace0 0x7fd878138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809e1d0 0x7fd87813b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd876ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd87809d6f0 0x7fd87809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43710/0 (socket says 172.21.15.32:43710) 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.923+0000 7fd876ffd700 1 -- 172.21.15.32:0/3913200070 learned_addr learned my addr 172.21.15.32:0/3913200070 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd8767fc700 1 -- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809e1d0 msgr2=0x7fd87813b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd8767fc700 1 --2- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809e1d0 0x7fd87813b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd8767fc700 1 -- 172.21.15.32:0/3913200070 >> v1:172.21.15.32:6789/0 conn(0x7fd87809d6f0 legacy=0x7fd87809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd8767fc700 1 -- 172.21.15.32:0/3913200070 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87806b490 con 0x7fd87809ace0 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd875ffb700 1 -- 172.21.15.32:0/3913200070 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912917108 0 0) 0x7fd87809b150 con 0x7fd87809d6f0 2023-09-30T22:28:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd8777fe700 1 --2- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809e1d0 0x7fd87813b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd8767fc700 1 --2- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd878138c30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009840 tx=0x7fd86c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aad4b7643bba65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd875ffb700 1 -- 172.21.15.32:0/3913200070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c011830 con 0x7fd87809ace0 2023-09-30T22:28:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd875ffb700 1 -- 172.21.15.32:0/3913200070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c011990 con 0x7fd87809ace0 2023-09-30T22:28:42.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.924+0000 7fd875ffb700 1 -- 172.21.15.32:0/3913200070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c011d20 con 0x7fd87809ace0 2023-09-30T22:28:42.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 -- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 msgr2=0x7fd878138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 --2- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd878138c30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009840 tx=0x7fd86c009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 -- 172.21.15.32:0/3913200070 shutdown_connections 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 --2- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809e1d0 0x7fd87813b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 --2- 172.21.15.32:0/3913200070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd878138c30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 -- 172.21.15.32:0/3913200070 >> 172.21.15.32:0/3913200070 conn(0x7fd878066f50 msgr2=0x7fd87809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 -- 172.21.15.32:0/3913200070 shutdown_connections 2023-09-30T22:28:42.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.925+0000 7fd87d85e700 1 -- 172.21.15.32:0/3913200070 wait complete. 2023-09-30T22:28:42.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 Processor -- start 2023-09-30T22:28:42.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 -- start start 2023-09-30T22:28:42.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd87811d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 0x7fd87811ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd87809e1d0 0x7fd87811e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87811eeb0 con 0x7fd87809ace0 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87811ff10 con 0x7fd87809e1d0 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.926+0000 7fd87d85e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780a1d00 con 0x7fd87809d6f0 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 0x7fd87811ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd876ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd87811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8777fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd87809e1d0 0x7fd87811e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 0x7fd87811ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46656/0 (socket says 172.21.15.32:46656) 2023-09-30T22:28:42.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 -- 172.21.15.32:0/1419774466 learned_addr learned my addr 172.21.15.32:0/1419774466 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 -- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd87809e1d0 msgr2=0x7fd87811e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd87809e1d0 0x7fd87811e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 -- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 msgr2=0x7fd87811d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd87811d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 -- 172.21.15.32:0/1419774466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870009a70 con 0x7fd87809d6f0 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd876ffd700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd87811d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd8767fc700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 0x7fd87811ddd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009fd0 tx=0x7fd86c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0045d0 con 0x7fd87809d6f0 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c01f070 con 0x7fd87809d6f0 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0038a0 con 0x7fd87809d6f0 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c0094a0 con 0x7fd87809d6f0 2023-09-30T22:28:42.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.927+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87811f9c0 con 0x7fd87809d6f0 2023-09-30T22:28:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.928+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd86c003a00 con 0x7fd87809d6f0 2023-09-30T22:28:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.928+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd86c021070 con 0x7fd87809d6f0 2023-09-30T22:28:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.928+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87811f2e0 con 0x7fd87809d6f0 2023-09-30T22:28:42.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:42.933+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd86c015080 con 0x7fd87809d6f0 2023-09-30T22:28:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.138+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd8780747f0 con 0x7fd87809d6f0 2023-09-30T22:28:43.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.139+0000 7fd85ffff700 1 -- 172.21.15.32:0/1419774466 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd86c02b020 con 0x7fd87809d6f0 2023-09-30T22:28:43.139 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 msgr2=0x7fd87811ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 0x7fd87811ddd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009fd0 tx=0x7fd86c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 shutdown_connections 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd87809e1d0 0x7fd87811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd87809ace0 0x7fd87811d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 --2- 172.21.15.32:0/1419774466 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd87809d6f0 0x7fd87811ddd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 >> 172.21.15.32:0/1419774466 conn(0x7fd878066f50 msgr2=0x7fd878136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 shutdown_connections 2023-09-30T22:28:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.141+0000 7fd87d85e700 1 -- 172.21.15.32:0/1419774466 wait complete. 2023-09-30T22:28:43.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:43.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:43.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:43.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.481+0000 7fe380749700 1 Processor -- start 2023-09-30T22:28:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.481+0000 7fe380749700 1 -- start start 2023-09-30T22:28:43.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe380749700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3781316c0 0x7fe378131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe380749700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37806ae80 0x7fe378139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe380749700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37809e130 con 0x7fe37809d9f0 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe380749700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3781312b0 con 0x7fe37806ae80 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe380749700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe37809f6e0 con 0x7fe3781316c0 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37e4e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3781316c0 0x7fe378131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37dce4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe37809d9f0 0x7fe37806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43726/0 (socket says 172.21.15.32:43726) 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37dce4700 1 -- 172.21.15.32:0/2956851119 learned_addr learned my addr 172.21.15.32:0/2956851119 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37ece6700 1 --2- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37806ae80 0x7fe378139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37e4e5700 1 -- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37806ae80 msgr2=0x7fe378139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37e4e5700 1 --2- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37806ae80 0x7fe378139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37e4e5700 1 -- 172.21.15.32:0/2956851119 >> v1:172.21.15.32:6789/0 conn(0x7fe37809d9f0 legacy=0x7fe37806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37d4e3700 1 -- 172.21.15.32:0/2956851119 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316549525 0 0) 0x7fe368001110 con 0x7fe37809d9f0 2023-09-30T22:28:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37e4e5700 1 -- 172.21.15.32:0/2956851119 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37809e760 con 0x7fe3781316c0 2023-09-30T22:28:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.482+0000 7fe37e4e5700 1 --2- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3781316c0 0x7fe378131a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe374009870 tx=0x7fe374009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d2781d80508b63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.483+0000 7fe37d4e3700 1 -- 172.21.15.32:0/2956851119 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374011830 con 0x7fe3781316c0 2023-09-30T22:28:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.483+0000 7fe37d4e3700 1 -- 172.21.15.32:0/2956851119 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374011990 con 0x7fe3781316c0 2023-09-30T22:28:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.483+0000 7fe37d4e3700 1 -- 172.21.15.32:0/2956851119 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374011d20 con 0x7fe3781316c0 2023-09-30T22:28:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.483+0000 7fe380749700 1 -- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3781316c0 msgr2=0x7fe378131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.483+0000 7fe380749700 1 --2- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3781316c0 0x7fe378131a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe374009870 tx=0x7fe374009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.484+0000 7fe380749700 1 -- 172.21.15.32:0/2956851119 shutdown_connections 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.484+0000 7fe380749700 1 --2- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37806ae80 0x7fe378139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.484+0000 7fe380749700 1 --2- 172.21.15.32:0/2956851119 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3781316c0 0x7fe378131a90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.484+0000 7fe380749700 1 -- 172.21.15.32:0/2956851119 >> 172.21.15.32:0/2956851119 conn(0x7fe378066f50 msgr2=0x7fe37809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.484+0000 7fe380749700 1 -- 172.21.15.32:0/2956851119 shutdown_connections 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.484+0000 7fe380749700 1 -- 172.21.15.32:0/2956851119 wait complete. 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 Processor -- start 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 -- start start 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 0x7fe37809a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37809d9f0 0x7fe37809a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3781316c0 0x7fe37809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37811f6a0 con 0x7fe3781316c0 2023-09-30T22:28:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37809c010 con 0x7fe37809d9f0 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe380749700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe37812b460 con 0x7fe37806ae80 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe37e4e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 0x7fe37809a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe37e4e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 0x7fe37809a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46682/0 (socket says 172.21.15.32:46682) 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe37e4e5700 1 -- 172.21.15.32:0/1648014456 learned_addr learned my addr 172.21.15.32:0/1648014456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.485+0000 7fe37dce4700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37809d9f0 0x7fe37809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37ece6700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3781316c0 0x7fe37809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37e4e5700 1 -- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37809d9f0 msgr2=0x7fe37809a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37e4e5700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37809d9f0 0x7fe37809a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37e4e5700 1 -- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3781316c0 msgr2=0x7fe37809ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37e4e5700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3781316c0 0x7fe37809ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37e4e5700 1 -- 172.21.15.32:0/1648014456 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740094a0 con 0x7fe37806ae80 2023-09-30T22:28:43.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe37e4e5700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 0x7fe37809a400 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3740090b0 tx=0x7fe374004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:43.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374004590 con 0x7fe37806ae80 2023-09-30T22:28:43.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3740036a0 con 0x7fe37806ae80 2023-09-30T22:28:43.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.486+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374003a50 con 0x7fe37806ae80 2023-09-30T22:28:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.487+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3781200c0 con 0x7fe37806ae80 2023-09-30T22:28:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.487+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37812b960 con 0x7fe37806ae80 2023-09-30T22:28:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.488+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe374003bb0 con 0x7fe37806ae80 2023-09-30T22:28:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.488+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe374040940 con 0x7fe37806ae80 2023-09-30T22:28:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.488+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37812b060 con 0x7fe37806ae80 2023-09-30T22:28:43.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.497+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe37812b060 con 0x7fe37806ae80 2023-09-30T22:28:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.545+0000 7fd11a924700 1 Processor -- start 2023-09-30T22:28:43.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.545+0000 7fd11a924700 1 -- start start 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd11a924700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd11409e1d0 0x7fd114138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd11a924700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 0x7fd11413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd11a924700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11409b120 con 0x7fd11409d6f0 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd11a924700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11406b460 con 0x7fd11409e1d0 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd11a924700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd11409a7e0 con 0x7fd114139170 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 0x7fd11413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 0x7fd11413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46690/0 (socket says 172.21.15.32:46690) 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 -- 172.21.15.32:0/2043158050 learned_addr learned my addr 172.21.15.32:0/2043158050 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd1137fe700 1 --2- 172.21.15.32:0/2043158050 >> v2:172.21.15.32:3300/0 conn(0x7fd11409e1d0 0x7fd114138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 -- 172.21.15.32:0/2043158050 >> v2:172.21.15.32:3300/0 conn(0x7fd11409e1d0 msgr2=0x7fd114138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 --2- 172.21.15.32:0/2043158050 >> v2:172.21.15.32:3300/0 conn(0x7fd11409e1d0 0x7fd114138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 -- 172.21.15.32:0/2043158050 >> v1:172.21.15.32:6789/0 conn(0x7fd11409d6f0 legacy=0x7fd11409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.546+0000 7fd118ec1700 1 -- 172.21.15.32:0/2043158050 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11409a500 con 0x7fd114139170 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.547+0000 7fd118ec1700 1 --2- 172.21.15.32:0/2043158050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 0x7fd11413b550 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009e80 tx=0x7fd10c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9a75b00ce4044 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.547+0000 7fd112ffd700 1 -- 172.21.15.32:0/2043158050 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00e070 con 0x7fd114139170 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.547+0000 7fd112ffd700 1 -- 172.21.15.32:0/2043158050 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10c00b8d0 con 0x7fd114139170 2023-09-30T22:28:43.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.547+0000 7fd112ffd700 1 -- 172.21.15.32:0/2043158050 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00bc80 con 0x7fd114139170 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 -- 172.21.15.32:0/2043158050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 msgr2=0x7fd11413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 --2- 172.21.15.32:0/2043158050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 0x7fd11413b550 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009e80 tx=0x7fd10c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 -- 172.21.15.32:0/2043158050 shutdown_connections 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 --2- 172.21.15.32:0/2043158050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd114139170 0x7fd11413b550 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 --2- 172.21.15.32:0/2043158050 >> v2:172.21.15.32:3300/0 conn(0x7fd11409e1d0 0x7fd114138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 -- 172.21.15.32:0/2043158050 >> 172.21.15.32:0/2043158050 conn(0x7fd114066f50 msgr2=0x7fd11409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 -- 172.21.15.32:0/2043158050 shutdown_connections 2023-09-30T22:28:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.548+0000 7fd11a924700 1 -- 172.21.15.32:0/2043158050 wait complete. 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 Processor -- start 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 -- start start 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd11409d6f0 0x7fd11411e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 0x7fd11411ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd114139170 0x7fd11411f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114121e90 con 0x7fd11409d6f0 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11409c710 con 0x7fd114139170 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.549+0000 7fd11a924700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd114120550 con 0x7fd11409e1d0 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 0x7fd11411ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 0x7fd11411ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46694/0 (socket says 172.21.15.32:46694) 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 -- 172.21.15.32:0/760138920 learned_addr learned my addr 172.21.15.32:0/760138920 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd113fff700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd11409d6f0 0x7fd11411e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd118ec1700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd114139170 0x7fd11411f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 -- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd114139170 msgr2=0x7fd11411f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd114139170 0x7fd11411f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 -- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd11409d6f0 msgr2=0x7fd11411e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd11409d6f0 0x7fd11411e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 -- 172.21.15.32:0/760138920 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c009b30 con 0x7fd11409e1d0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.550+0000 7fd1137fe700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 0x7fd11411ee00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd10800e9e0 tx=0x7fd10800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.551+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108009710 con 0x7fd11409e1d0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.551+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10800ee80 con 0x7fd11409e1d0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.551+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1140a1a10 con 0x7fd11409e1d0 2023-09-30T22:28:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.551+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10800ca50 con 0x7fd11409e1d0 2023-09-30T22:28:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.551+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd114120c30 con 0x7fd11409e1d0 2023-09-30T22:28:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.552+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd114074ea0 con 0x7fd11409e1d0 2023-09-30T22:28:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.552+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd10800cbb0 con 0x7fd11409e1d0 2023-09-30T22:28:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.552+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd10803f6b0 con 0x7fd11409e1d0 2023-09-30T22:28:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.557+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1080232f0 con 0x7fd11409e1d0 2023-09-30T22:28:43.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.588+0000 7f4849fde700 1 Processor -- start 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.588+0000 7f4849fde700 1 -- start start 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.588+0000 7f4849fde700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48440a0a30 0x7f48440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.588+0000 7f4849fde700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f484413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.588+0000 7f4849fde700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48440a2a70 con 0x7f484409d630 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4849fde700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48440a01d0 con 0x7f48440a0a30 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4849fde700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f484409e2b0 con 0x7f4844138d80 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f484413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f484413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46710/0 (socket says 172.21.15.32:46710) 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 -- 172.21.15.32:0/3964824348 learned_addr learned my addr 172.21.15.32:0/3964824348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f48437fe700 1 --2- 172.21.15.32:0/3964824348 >> v2:172.21.15.32:3300/0 conn(0x7f48440a0a30 0x7f48440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 -- 172.21.15.32:0/3964824348 >> v2:172.21.15.32:3300/0 conn(0x7f48440a0a30 msgr2=0x7f48440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 --2- 172.21.15.32:0/3964824348 >> v2:172.21.15.32:3300/0 conn(0x7f48440a0a30 0x7f48440a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 -- 172.21.15.32:0/3964824348 >> v1:172.21.15.32:6789/0 conn(0x7f484409d630 legacy=0x7f48440a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 -- 172.21.15.32:0/3964824348 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484409dc50 con 0x7f4844138d80 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.589+0000 7f4843fff700 1 --2- 172.21.15.32:0/3964824348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f484413b160 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f483c009e50 tx=0x7f483c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce162d0aeb8e949 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f48427fc700 1 -- 172.21.15.32:0/3964824348 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00e070 con 0x7f4844138d80 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f48427fc700 1 -- 172.21.15.32:0/3964824348 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c00b910 con 0x7f4844138d80 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f48427fc700 1 -- 172.21.15.32:0/3964824348 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00bcc0 con 0x7f4844138d80 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 -- 172.21.15.32:0/3964824348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 msgr2=0x7f484413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 --2- 172.21.15.32:0/3964824348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f484413b160 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f483c009e50 tx=0x7f483c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:43.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 -- 172.21.15.32:0/3964824348 shutdown_connections 2023-09-30T22:28:43.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 --2- 172.21.15.32:0/3964824348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f484413b160 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 --2- 172.21.15.32:0/3964824348 >> v2:172.21.15.32:3300/0 conn(0x7f48440a0a30 0x7f48440a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 -- 172.21.15.32:0/3964824348 >> 172.21.15.32:0/3964824348 conn(0x7f48440a3050 msgr2=0x7f48440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 -- 172.21.15.32:0/3964824348 shutdown_connections 2023-09-30T22:28:43.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.590+0000 7f4849fde700 1 -- 172.21.15.32:0/3964824348 wait complete. 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.592+0000 7f4849fde700 1 Processor -- start 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.592+0000 7f4849fde700 1 -- start start 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4849fde700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 0x7f4844120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4849fde700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48440a0a30 0x7f4844120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4849fde700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f48441210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4849fde700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844123e60 con 0x7f48440a0a30 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4849fde700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4844122ff0 con 0x7f484409d630 2023-09-30T22:28:43.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4849fde700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4844124060 con 0x7f4844138d80 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4842ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48440a0a30 0x7f4844120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 0x7f4844120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 0x7f4844120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34444/0 (socket says 172.21.15.32:34444) 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4842ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48440a0a30 0x7f4844120b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52942/0 (socket says 172.21.15.32:52942) 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 -- 172.21.15.32:0/1396801923 learned_addr learned my addr 172.21.15.32:0/1396801923 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f4843fff700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f48441210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 -- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 msgr2=0x7f48441210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f48441210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 -- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48440a0a30 msgr2=0x7f4844120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48440a0a30 0x7f4844120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.593+0000 7f48437fe700 1 -- 172.21.15.32:0/1396801923 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c009b30 con 0x7f484409d630 2023-09-30T22:28:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.594+0000 7f48437fe700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 0x7f4844120620 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f483400e9b0 tx=0x7f483400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.594+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834009710 con 0x7f484409d630 2023-09-30T22:28:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.594+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483400ee50 con 0x7f484409d630 2023-09-30T22:28:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.594+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4844123410 con 0x7f484409d630 2023-09-30T22:28:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.594+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400ca50 con 0x7f484409d630 2023-09-30T22:28:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.594+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48441389a0 con 0x7f484409d630 2023-09-30T22:28:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.595+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484413b620 con 0x7f484409d630 2023-09-30T22:28:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.595+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f483400cbb0 con 0x7f484409d630 2023-09-30T22:28:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.596+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f483403faa0 con 0x7f484409d630 2023-09-30T22:28:43.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.605+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f483401b3b0 con 0x7f484409d630 2023-09-30T22:28:43.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.727+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe37812b060 con 0x7fe37806ae80 2023-09-30T22:28:43.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.728+0000 7fe36effd700 1 -- 172.21.15.32:0/1648014456 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe37812b060 con 0x7fe37806ae80 2023-09-30T22:28:43.728 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:43.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 msgr2=0x7fe37809a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 0x7fe37809a400 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3740090b0 tx=0x7fe374004750 comp rx=0 tx=0).stop 2023-09-30T22:28:43.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 shutdown_connections 2023-09-30T22:28:43.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3781316c0 0x7fe37809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe37806ae80 0x7fe37809a400 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 --2- 172.21.15.32:0/1648014456 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe37809d9f0 0x7fe37809a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 >> 172.21.15.32:0/1648014456 conn(0x7fe378066f50 msgr2=0x7fe3780a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 shutdown_connections 2023-09-30T22:28:43.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.730+0000 7fe380749700 1 -- 172.21.15.32:0/1648014456 wait complete. 2023-09-30T22:28:43.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.768+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd11406c360 con 0x7fd11409e1d0 2023-09-30T22:28:43.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.768+0000 7fd110ff9700 1 -- 172.21.15.32:0/760138920 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd108021030 con 0x7fd11409e1d0 2023-09-30T22:28:43.768 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 msgr2=0x7fd11411ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 0x7fd11411ee00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd10800e9e0 tx=0x7fd10800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 shutdown_connections 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd114139170 0x7fd11411f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd11409d6f0 0x7fd11411e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 --2- 172.21.15.32:0/760138920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd11409e1d0 0x7fd11411ee00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.770+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 >> 172.21.15.32:0/760138920 conn(0x7fd114066f50 msgr2=0x7fd11409b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.771+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 shutdown_connections 2023-09-30T22:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.771+0000 7fd11a924700 1 -- 172.21.15.32:0/760138920 wait complete. 2023-09-30T22:28:43.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.813+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4844123cb0 con 0x7f484409d630 2023-09-30T22:28:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.814+0000 7f482bfff700 1 -- 172.21.15.32:0/1396801923 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4834017020 con 0x7f484409d630 2023-09-30T22:28:43.814 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 msgr2=0x7f4844120620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 0x7f4844120620 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f483400e9b0 tx=0x7f483400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 shutdown_connections 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4844138d80 0x7f48441210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f484409d630 0x7f4844120620 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 --2- 172.21.15.32:0/1396801923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48440a0a30 0x7f4844120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 >> 172.21.15.32:0/1396801923 conn(0x7f48440a3050 msgr2=0x7f484409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 shutdown_connections 2023-09-30T22:28:43.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:43.816+0000 7f4849fde700 1 -- 172.21.15.32:0/1396801923 wait complete. 2023-09-30T22:28:43.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:44.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.301+0000 7f5c8274e700 1 Processor -- start 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.301+0000 7f5c8274e700 1 -- start start 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c8274e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c133a80 0x7f5c7c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c8274e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c8274e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c06bc00 con 0x7f5c7c12ff20 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c8274e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c06c570 con 0x7f5c7c133a80 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c8274e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c0a0040 con 0x7f5c7c130a40 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c80ceb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c133a80 0x7f5c7c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c80ceb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c13b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46732/0 (socket says 172.21.15.32:46732) 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c80ceb700 1 -- 172.21.15.32:0/2749263326 learned_addr learned my addr 172.21.15.32:0/2749263326 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7b7fe700 1 --1- 172.21.15.32:0/2749263326 >> v1:172.21.15.32:6789/0 conn(0x7f5c7c12ff20 0x7f5c7c130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43764/0 (socket says 172.21.15.32:43764) 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7bfff700 1 -- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 msgr2=0x7f5c7c13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c13b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7bfff700 1 -- 172.21.15.32:0/2749263326 >> v1:172.21.15.32:6789/0 conn(0x7f5c7c12ff20 legacy=0x7f5c7c130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7affd700 1 -- 172.21.15.32:0/2749263326 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405312657 0 0) 0x7f5c7c06bc00 con 0x7f5c7c12ff20 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7bfff700 1 -- 172.21.15.32:0/2749263326 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c133860 con 0x7f5c7c133a80 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c80ceb700 1 --2- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c133a80 0x7f5c7c133e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009870 tx=0x7f5c6c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0fb698fcfcc94a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.302+0000 7f5c7affd700 1 -- 172.21.15.32:0/2749263326 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c011830 con 0x7f5c7c133a80 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c7affd700 1 -- 172.21.15.32:0/2749263326 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c011990 con 0x7f5c7c133a80 2023-09-30T22:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c7affd700 1 -- 172.21.15.32:0/2749263326 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c011d20 con 0x7f5c7c133a80 2023-09-30T22:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 -- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c133a80 msgr2=0x7f5c7c133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c133a80 0x7f5c7c133e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009870 tx=0x7f5c6c009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 -- 172.21.15.32:0/2749263326 shutdown_connections 2023-09-30T22:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2749263326 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c133a80 0x7f5c7c133e50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 -- 172.21.15.32:0/2749263326 >> 172.21.15.32:0/2749263326 conn(0x7f5c7c0a4210 msgr2=0x7f5c7c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 -- 172.21.15.32:0/2749263326 shutdown_connections 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.303+0000 7f5c8274e700 1 -- 172.21.15.32:0/2749263326 wait complete. 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.304+0000 7f5c8274e700 1 Processor -- start 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.304+0000 7f5c8274e700 1 -- start start 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c8274e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c12ff20 0x7f5c7c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c8274e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c8274e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c133a80 0x7f5c7c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c8274e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c09de50 con 0x7f5c7c133a80 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c8274e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c09e080 con 0x7f5c7c12ff20 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c8274e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c11fb90 con 0x7f5c7c130a40 2023-09-30T22:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c80ceb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c133a80 0x7f5c7c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46736/0 (socket says 172.21.15.32:46736) 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c12ff20 0x7f5c7c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 -- 172.21.15.32:0/2105700142 learned_addr learned my addr 172.21.15.32:0/2105700142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 -- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c12ff20 msgr2=0x7f5c7c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c12ff20 0x7f5c7c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 -- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c133a80 msgr2=0x7f5c7c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c133a80 0x7f5c7c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7b7fe700 1 -- 172.21.15.32:0/2105700142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009b30 con 0x7f5c7c130a40 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c7bfff700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c12ff20 0x7f5c7c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.305+0000 7f5c80ceb700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c133a80 0x7f5c7c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.306+0000 7f5c7b7fe700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c09c9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5c70001eb0 tx=0x7f5c7000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.306+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70004c80 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.306+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70004de0 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.306+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c0094a0 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.306+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007310 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.306+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c12d990 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.307+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5c70007470 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.307+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5c7003f2c0 con 0x7f5c7c130a40 2023-09-30T22:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.307+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c074ea0 con 0x7f5c7c130a40 2023-09-30T22:28:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.312+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5c700123b0 con 0x7f5c7c130a40 2023-09-30T22:28:44.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.517+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5c7c11feb0 con 0x7f5c7c130a40 2023-09-30T22:28:44.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.518+0000 7f5c78ff9700 1 -- 172.21.15.32:0/2105700142 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5c70011360 con 0x7f5c7c130a40 2023-09-30T22:28:44.518 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:44.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 msgr2=0x7f5c7c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c09c9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5c70001eb0 tx=0x7f5c7000c800 comp rx=0 tx=0).stop 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 shutdown_connections 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c7c133a80 0x7f5c7c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c7c12ff20 0x7f5c7c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 --2- 172.21.15.32:0/2105700142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c7c130a40 0x7f5c7c09c9c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 >> 172.21.15.32:0/2105700142 conn(0x7f5c7c0a4210 msgr2=0x7f5c7c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 shutdown_connections 2023-09-30T22:28:44.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.520+0000 7f5c8274e700 1 -- 172.21.15.32:0/2105700142 wait complete. 2023-09-30T22:28:44.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:44.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:44.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:44.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.898+0000 7f08a9c56700 1 Processor -- start 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.898+0000 7f08a9c56700 1 -- start start 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a9c56700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a40a0a30 0x7f08a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a9c56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a4134170 0x7f08a413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a9c56700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a409edd0 con 0x7f08a40a1340 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a9c56700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a409e1f0 con 0x7f08a4134170 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a9c56700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a409dc50 con 0x7f08a40a0a30 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f08a40a1340 0x7f08a4133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43774/0 (socket says 172.21.15.32:43774) 2023-09-30T22:28:44.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08a4134170 0x7f08a413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a2ffd700 1 -- 172.21.15.32:0/2407938832 learned_addr learned my addr 172.21.15.32:0/2407938832 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:44.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a37fe700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a40a0a30 0x7f08a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a3fff700 1 -- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a40a0a30 msgr2=0x7f08a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a3fff700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a40a0a30 0x7f08a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a3fff700 1 -- 172.21.15.32:0/2407938832 >> v1:172.21.15.32:6789/0 conn(0x7f08a40a1340 legacy=0x7f08a4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a3fff700 1 -- 172.21.15.32:0/2407938832 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40a1fe0 con 0x7f08a4134170 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a27fc700 1 -- 172.21.15.32:0/2407938832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223337528 0 0) 0x7f08a409edd0 con 0x7f08a40a1340 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.899+0000 7f08a37fe700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a40a0a30 0x7f08a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a3fff700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a4134170 0x7f08a413bd30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f089c009d90 tx=0x7f089c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b121fd919bfb90 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a27fc700 1 -- 172.21.15.32:0/2407938832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c00e070 con 0x7f08a4134170 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a27fc700 1 -- 172.21.15.32:0/2407938832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c00bb20 con 0x7f08a4134170 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a27fc700 1 -- 172.21.15.32:0/2407938832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c012430 con 0x7f08a4134170 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a9c56700 1 -- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a4134170 msgr2=0x7f08a413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a9c56700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a4134170 0x7f08a413bd30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f089c009d90 tx=0x7f089c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a9c56700 1 -- 172.21.15.32:0/2407938832 shutdown_connections 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.900+0000 7f08a9c56700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a4134170 0x7f08a413bd30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.901+0000 7f08a9c56700 1 --2- 172.21.15.32:0/2407938832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a40a0a30 0x7f08a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.901+0000 7f08a9c56700 1 -- 172.21.15.32:0/2407938832 >> 172.21.15.32:0/2407938832 conn(0x7f08a40a6550 msgr2=0x7f08a40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.901+0000 7f08a9c56700 1 -- 172.21.15.32:0/2407938832 shutdown_connections 2023-09-30T22:28:44.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.901+0000 7f08a9c56700 1 -- 172.21.15.32:0/2407938832 wait complete. 2023-09-30T22:28:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.902+0000 7f08a9c56700 1 Processor -- start 2023-09-30T22:28:44.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.902+0000 7f08a9c56700 1 -- start start 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a9c56700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a40a0a30 0x7f08a4122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a9c56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a40a1340 0x7f08a4122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a9c56700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 0x7f08a4123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a9c56700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4126220 con 0x7f08a40a1340 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a9c56700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4124310 con 0x7f08a4134170 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a9c56700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40a4d70 con 0x7f08a40a0a30 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 0x7f08a4123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 0x7f08a4123210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34484/0 (socket says 172.21.15.32:34484) 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a3fff700 1 -- 172.21.15.32:0/3581215550 learned_addr learned my addr 172.21.15.32:0/3581215550 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a2ffd700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a40a1340 0x7f08a4122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.903+0000 7f08a37fe700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a40a0a30 0x7f08a4122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a2ffd700 1 -- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a40a0a30 msgr2=0x7f08a4122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a2ffd700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a40a0a30 0x7f08a4122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a2ffd700 1 -- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 msgr2=0x7f08a4123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a2ffd700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 0x7f08a4123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a2ffd700 1 -- 172.21.15.32:0/3581215550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08940094a0 con 0x7f08a40a1340 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a37fe700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a40a0a30 0x7f08a4122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a3fff700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 0x7f08a4123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a2ffd700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a40a1340 0x7f08a4122cd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f089800c470 tx=0x7f089800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898004c80 con 0x7f08a40a1340 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898004de0 con 0x7f08a40a1340 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c009a70 con 0x7f08a40a1340 2023-09-30T22:28:44.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898007310 con 0x7f08a40a1340 2023-09-30T22:28:44.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.904+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a4133220 con 0x7f08a40a1340 2023-09-30T22:28:44.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.905+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0898007470 con 0x7f08a40a1340 2023-09-30T22:28:44.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.905+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f089803f340 con 0x7f08a40a1340 2023-09-30T22:28:44.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.906+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a4125be0 con 0x7f08a40a1340 2023-09-30T22:28:44.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.915+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08a4125be0 con 0x7f08a40a1340 2023-09-30T22:28:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.937+0000 7fd894314700 1 Processor -- start 2023-09-30T22:28:44.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd894314700 1 -- start start 2023-09-30T22:28:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd894314700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c0a0a30 0x7fd88c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd894314700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd88c09d630 0x7fd88c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd894314700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c0a2a70 con 0x7fd88c133930 2023-09-30T22:28:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd894314700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0a01d0 con 0x7fd88c09d630 2023-09-30T22:28:44.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd894314700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c09e2b0 con 0x7fd88c0a0a30 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd8928b1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd88c133930 0x7fd88c133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43778/0 (socket says 172.21.15.32:43778) 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.938+0000 7fd8928b1700 1 -- 172.21.15.32:0/3023068883 learned_addr learned my addr 172.21.15.32:0/3023068883 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8918af700 1 --2- 172.21.15.32:0/3023068883 >> v2:172.21.15.32:3300/0 conn(0x7fd88c09d630 0x7fd88c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8920b0700 1 --2- 172.21.15.32:0/3023068883 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c0a0a30 0x7fd88c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8920b0700 1 -- 172.21.15.32:0/3023068883 >> v2:172.21.15.32:3300/0 conn(0x7fd88c09d630 msgr2=0x7fd88c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8920b0700 1 --2- 172.21.15.32:0/3023068883 >> v2:172.21.15.32:3300/0 conn(0x7fd88c09d630 0x7fd88c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8920b0700 1 -- 172.21.15.32:0/3023068883 >> v1:172.21.15.32:6789/0 conn(0x7fd88c133930 legacy=0x7fd88c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8920b0700 1 -- 172.21.15.32:0/3023068883 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c09dc50 con 0x7fd88c0a0a30 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8920b0700 1 --2- 172.21.15.32:0/3023068883 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c0a0a30 0x7fd88c0a0e00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd888009840 tx=0x7fd888009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef2139bf2064654 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8910ae700 1 -- 172.21.15.32:0/3023068883 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888011830 con 0x7fd88c0a0a30 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8910ae700 1 -- 172.21.15.32:0/3023068883 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888011990 con 0x7fd88c0a0a30 2023-09-30T22:28:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.939+0000 7fd8910ae700 1 -- 172.21.15.32:0/3023068883 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888011d20 con 0x7fd88c0a0a30 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 -- 172.21.15.32:0/3023068883 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c0a0a30 msgr2=0x7fd88c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 --2- 172.21.15.32:0/3023068883 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c0a0a30 0x7fd88c0a0e00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd888009840 tx=0x7fd888009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 -- 172.21.15.32:0/3023068883 shutdown_connections 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 --2- 172.21.15.32:0/3023068883 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c0a0a30 0x7fd88c0a0e00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 --2- 172.21.15.32:0/3023068883 >> v2:172.21.15.32:3300/0 conn(0x7fd88c09d630 0x7fd88c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 -- 172.21.15.32:0/3023068883 >> 172.21.15.32:0/3023068883 conn(0x7fd88c0a3050 msgr2=0x7fd88c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 -- 172.21.15.32:0/3023068883 shutdown_connections 2023-09-30T22:28:44.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.940+0000 7fd894314700 1 -- 172.21.15.32:0/3023068883 wait complete. 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.941+0000 7fd894314700 1 Processor -- start 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.941+0000 7fd894314700 1 -- start start 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd894314700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c09d630 0x7fd88c121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd894314700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 0x7fd88c1217d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd894314700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd88c133930 0x7fd88c121d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd894314700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c122dc0 con 0x7fd88c133930 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd894314700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c123c50 con 0x7fd88c0a0a30 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd894314700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c124730 con 0x7fd88c09d630 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 0x7fd88c1217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 0x7fd88c1217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34492/0 (socket says 172.21.15.32:34492) 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8928b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd88c133930 0x7fd88c121d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 -- 172.21.15.32:0/3963790018 learned_addr learned my addr 172.21.15.32:0/3963790018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8920b0700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c09d630 0x7fd88c121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 -- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c09d630 msgr2=0x7fd88c121290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c09d630 0x7fd88c121290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 -- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd88c133930 msgr2=0x7fd88c121d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd88c133930 0x7fd88c121d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.942+0000 7fd8918af700 1 -- 172.21.15.32:0/3963790018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880094a0 con 0x7fd88c0a0a30 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.943+0000 7fd8918af700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 0x7fd88c1217d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9b0 tx=0x7fd87c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.943+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c009710 con 0x7fd88c0a0a30 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.943+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00ee50 con 0x7fd88c0a0a30 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.943+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c1249f0 con 0x7fd88c0a0a30 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.943+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00ca70 con 0x7fd88c0a0a30 2023-09-30T22:28:44.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.943+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c124460 con 0x7fd88c0a0a30 2023-09-30T22:28:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.944+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd87c00cbd0 con 0x7fd88c0a0a30 2023-09-30T22:28:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.944+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd87c03fa40 con 0x7fd88c0a0a30 2023-09-30T22:28:44.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.944+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c11df60 con 0x7fd88c0a0a30 2023-09-30T22:28:44.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.953+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd87c021080 con 0x7fd88c0a0a30 2023-09-30T22:28:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.978+0000 7f28dd5ec700 1 Processor -- start 2023-09-30T22:28:44.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.978+0000 7f28dd5ec700 1 -- start start 2023-09-30T22:28:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28dd5ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d80a1510 0x7f28d8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28dd5ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d8139250 0x7f28d813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28dd5ec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d809edd0 con 0x7f28d80a0a30 2023-09-30T22:28:44.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28dd5ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80a18e0 con 0x7f28d8139250 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28dd5ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d809e1f0 con 0x7f28d80a1510 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28d6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f28d80a0a30 0x7f28d80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43786/0 (socket says 172.21.15.32:43786) 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28d6ffd700 1 -- 172.21.15.32:0/3744027160 learned_addr learned my addr 172.21.15.32:0/3744027160 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28ce7fc700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d80a1510 0x7f28d8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28d77fe700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d8139250 0x7f28d813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28d67fc700 1 -- 172.21.15.32:0/3744027160 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608943602 0 0) 0x7f28d809edd0 con 0x7f28d80a0a30 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28d67fc700 1 -- 172.21.15.32:0/3744027160 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28c40035b0 con 0x7f28d80a0a30 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28ce7fc700 1 -- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d8139250 msgr2=0x7f28d813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28ce7fc700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d8139250 0x7f28d813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28ce7fc700 1 -- 172.21.15.32:0/3744027160 >> v1:172.21.15.32:6789/0 conn(0x7f28d80a0a30 legacy=0x7f28d80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28ce7fc700 1 -- 172.21.15.32:0/3744027160 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d809dc50 con 0x7f28d80a1510 2023-09-30T22:28:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.979+0000 7f28d77fe700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d8139250 0x7f28d813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.980+0000 7f28ce7fc700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d80a1510 0x7f28d8138d10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009870 tx=0x7f28c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abc7762a39226388 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.980+0000 7f28d67fc700 1 -- 172.21.15.32:0/3744027160 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0011830 con 0x7f28d80a1510 2023-09-30T22:28:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.980+0000 7f28d67fc700 1 -- 172.21.15.32:0/3744027160 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0011990 con 0x7f28d80a1510 2023-09-30T22:28:44.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.980+0000 7f28d67fc700 1 -- 172.21.15.32:0/3744027160 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0011d20 con 0x7f28d80a1510 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 -- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d80a1510 msgr2=0x7f28d8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d80a1510 0x7f28d8138d10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009870 tx=0x7f28c0009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 -- 172.21.15.32:0/3744027160 shutdown_connections 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d8139250 0x7f28d813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/3744027160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d80a1510 0x7f28d8138d10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 -- 172.21.15.32:0/3744027160 >> 172.21.15.32:0/3744027160 conn(0x7f28d80a6550 msgr2=0x7f28d80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 -- 172.21.15.32:0/3744027160 shutdown_connections 2023-09-30T22:28:44.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.981+0000 7f28dd5ec700 1 -- 172.21.15.32:0/3744027160 wait complete. 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.982+0000 7f28dd5ec700 1 Processor -- start 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.983+0000 7f28dd5ec700 1 -- start start 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.983+0000 7f28dd5ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28d80a0a30 0x7f28d81227d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28dd5ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 0x7f28d8122d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28dd5ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d8139250 0x7f28d8123250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28dd5ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d81243e0 con 0x7f28d80a0a30 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28dd5ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d8124a40 con 0x7f28d80a1510 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28dd5ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d81386c0 con 0x7f28d8139250 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28ce7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 0x7f28d8122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28d80a0a30 0x7f28d81227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28ce7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 0x7f28d8122d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34518/0 (socket says 172.21.15.32:34518) 2023-09-30T22:28:44.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28ce7fc700 1 -- 172.21.15.32:0/2635170051 learned_addr learned my addr 172.21.15.32:0/2635170051 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d77fe700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d8139250 0x7f28d8123250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 -- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d8139250 msgr2=0x7f28d8123250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d8139250 0x7f28d8123250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 -- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 msgr2=0x7f28d8122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 0x7f28d8122d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 -- 172.21.15.32:0/2635170051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0009a70 con 0x7f28d80a0a30 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28ce7fc700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 0x7f28d8122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.984+0000 7f28d6ffd700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28d80a0a30 0x7f28d81227d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c4c0 tx=0x7f28c800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.985+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8005d70 con 0x7f28d80a0a30 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.985+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c80070d0 con 0x7f28d80a0a30 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.985+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c00094a0 con 0x7f28d80a0a30 2023-09-30T22:28:44.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.985+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8007440 con 0x7f28d80a0a30 2023-09-30T22:28:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.985+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d8126010 con 0x7f28d80a0a30 2023-09-30T22:28:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.986+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28c80075a0 con 0x7f28d80a0a30 2023-09-30T22:28:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.986+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f28c803ea80 con 0x7f28d80a0a30 2023-09-30T22:28:44.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.986+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8074ea0 con 0x7f28d80a0a30 2023-09-30T22:28:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:44.995+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28c8017030 con 0x7f28d80a0a30 2023-09-30T22:28:45.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.129+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f08a4125be0 con 0x7f08a40a1340 2023-09-30T22:28:45.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.129+0000 7f088bfff700 1 -- 172.21.15.32:0/3581215550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08a4125be0 con 0x7f08a40a1340 2023-09-30T22:28:45.129 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:45.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.131+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a40a1340 msgr2=0x7f08a4122cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.131+0000 7f08a9c56700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a40a1340 0x7f08a4122cd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f089800c470 tx=0x7f089800c780 comp rx=0 tx=0).stop 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 shutdown_connections 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08a4134170 0x7f08a4123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08a40a0a30 0x7f08a4122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 --2- 172.21.15.32:0/3581215550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08a40a1340 0x7f08a4122cd0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 >> 172.21.15.32:0/3581215550 conn(0x7f08a40a6550 msgr2=0x7f08a409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 shutdown_connections 2023-09-30T22:28:45.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.132+0000 7f08a9c56700 1 -- 172.21.15.32:0/3581215550 wait complete. 2023-09-30T22:28:45.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:45.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.168+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd88c0747f0 con 0x7fd88c0a0a30 2023-09-30T22:28:45.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.169+0000 7fd882ffd700 1 -- 172.21.15.32:0/3963790018 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd87c011db0 con 0x7fd88c0a0a30 2023-09-30T22:28:45.169 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 msgr2=0x7fd88c1217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 0x7fd88c1217d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9b0 tx=0x7fd87c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 shutdown_connections 2023-09-30T22:28:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd88c133930 0x7fd88c121d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd88c09d630 0x7fd88c121290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 --2- 172.21.15.32:0/3963790018 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd88c0a0a30 0x7fd88c1217d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 >> 172.21.15.32:0/3963790018 conn(0x7fd88c0a3050 msgr2=0x7fd88c1331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 shutdown_connections 2023-09-30T22:28:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.172+0000 7fd894314700 1 -- 172.21.15.32:0/3963790018 wait complete. 2023-09-30T22:28:45.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.205+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f28d8124bd0 con 0x7f28d80a0a30 2023-09-30T22:28:45.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.205+0000 7f28cffff700 1 -- 172.21.15.32:0/2635170051 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28c801bb10 con 0x7f28d80a0a30 2023-09-30T22:28:45.206 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28d80a0a30 msgr2=0x7f28d81227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28d80a0a30 0x7f28d81227d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c4c0 tx=0x7f28c800c7d0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 shutdown_connections 2023-09-30T22:28:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28d8139250 0x7f28d8123250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28d80a0a30 0x7f28d81227d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 --2- 172.21.15.32:0/2635170051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28d80a1510 0x7f28d8122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 >> 172.21.15.32:0/2635170051 conn(0x7f28d80a6550 msgr2=0x7f28d80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:45.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 shutdown_connections 2023-09-30T22:28:45.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.208+0000 7f28dd5ec700 1 -- 172.21.15.32:0/2635170051 wait complete. 2023-09-30T22:28:45.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:45.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.681+0000 7f3d1d75d700 1 Processor -- start 2023-09-30T22:28:45.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.682+0000 7f3d1d75d700 1 -- start start 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.682+0000 7f3d1d75d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d18138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.682+0000 7f3d1d75d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d180a14a0 0x7f3d1813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.682+0000 7f3d1d75d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1809ea10 con 0x7f3d180a09c0 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.682+0000 7f3d1d75d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1809e210 con 0x7f3d180a14a0 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.682+0000 7f3d1d75d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180a26f0 con 0x7f3d1809d5c0 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d16ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3d180a09c0 0x7f3d180a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43790/0 (socket says 172.21.15.32:43790) 2023-09-30T22:28:45.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d16ffd700 1 -- 172.21.15.32:0/90299592 learned_addr learned my addr 172.21.15.32:0/90299592 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d177fe700 1 --2- 172.21.15.32:0/90299592 >> v2:172.21.15.32:3300/0 conn(0x7f3d180a14a0 0x7f3d1813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d167fc700 1 --2- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d18138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d177fe700 1 -- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 msgr2=0x7f3d18138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d177fe700 1 --2- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d18138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d177fe700 1 -- 172.21.15.32:0/90299592 >> v1:172.21.15.32:6789/0 conn(0x7f3d180a09c0 legacy=0x7f3d180a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d15ffb700 1 -- 172.21.15.32:0/90299592 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204306526 0 0) 0x7f3d1809ea10 con 0x7f3d180a09c0 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d177fe700 1 -- 172.21.15.32:0/90299592 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1809db70 con 0x7f3d180a14a0 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d177fe700 1 --2- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a14a0 0x7f3d1813b2c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009d90 tx=0x7f3d1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4809e5efe38797de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:45.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d15ffb700 1 -- 172.21.15.32:0/90299592 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1000e070 con 0x7f3d180a14a0 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d15ffb700 1 -- 172.21.15.32:0/90299592 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1000bb20 con 0x7f3d180a14a0 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.683+0000 7f3d15ffb700 1 -- 172.21.15.32:0/90299592 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10012430 con 0x7f3d180a14a0 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 -- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a14a0 msgr2=0x7f3d1813b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a14a0 0x7f3d1813b2c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009d90 tx=0x7f3d1000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 -- 172.21.15.32:0/90299592 shutdown_connections 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a14a0 0x7f3d1813b2c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/90299592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d18138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 -- 172.21.15.32:0/90299592 >> 172.21.15.32:0/90299592 conn(0x7f3d180a2fe0 msgr2=0x7f3d180a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 -- 172.21.15.32:0/90299592 shutdown_connections 2023-09-30T22:28:45.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.684+0000 7f3d1d75d700 1 -- 172.21.15.32:0/90299592 wait complete. 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.685+0000 7f3d1d75d700 1 Processor -- start 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.685+0000 7f3d1d75d700 1 -- start start 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d1d75d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d1811c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d1d75d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a09c0 0x7f3d1811ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d1d75d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d180a14a0 0x7f3d1811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d1d75d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1809c1f0 con 0x7f3d180a09c0 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d1d75d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1809cec0 con 0x7f3d1809d5c0 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d1d75d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d181347b0 con 0x7f3d180a14a0 2023-09-30T22:28:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d1811c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d177fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d180a14a0 0x7f3d1811d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d1811c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34546/0 (socket says 172.21.15.32:34546) 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 -- 172.21.15.32:0/1661310775 learned_addr learned my addr 172.21.15.32:0/1661310775 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d167fc700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a09c0 0x7f3d1811ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 -- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d180a14a0 msgr2=0x7f3d1811d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d180a14a0 0x7f3d1811d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 -- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a09c0 msgr2=0x7f3d1811ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a09c0 0x7f3d1811ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 -- 172.21.15.32:0/1661310775 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10009a70 con 0x7f3d1809d5c0 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d167fc700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a09c0 0x7f3d1811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:45.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d16ffd700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d1811c7b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c520 tx=0x7f3d0800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.686+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08004c80 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.687+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08004e00 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.687+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1811f620 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.687+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08007310 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.687+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1811fbb0 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.687+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3d08007470 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.688+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3d0803f420 con 0x7f3d1809d5c0 2023-09-30T22:28:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.688+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d180747f0 con 0x7f3d1809d5c0 2023-09-30T22:28:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.692+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3d080123b0 con 0x7f3d1809d5c0 2023-09-30T22:28:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.897+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3d1811e750 con 0x7f3d1809d5c0 2023-09-30T22:28:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.897+0000 7f3d0f7fe700 1 -- 172.21.15.32:0/1661310775 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d08011360 con 0x7f3d1809d5c0 2023-09-30T22:28:45.898 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 msgr2=0x7f3d1811c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d1811c7b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800c520 tx=0x7f3d0800c830 comp rx=0 tx=0).stop 2023-09-30T22:28:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 shutdown_connections 2023-09-30T22:28:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d180a14a0 0x7f3d1811d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d1809d5c0 0x7f3d1811c7b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 --2- 172.21.15.32:0/1661310775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d180a09c0 0x7f3d1811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 >> 172.21.15.32:0/1661310775 conn(0x7f3d180a2fe0 msgr2=0x7f3d18136e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 shutdown_connections 2023-09-30T22:28:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:45.900+0000 7f3d1d75d700 1 -- 172.21.15.32:0/1661310775 wait complete. 2023-09-30T22:28:45.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:46.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:46.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:46.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.303+0000 7f90e731f700 1 Processor -- start 2023-09-30T22:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.303+0000 7f90e731f700 1 -- start start 2023-09-30T22:28:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e731f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e731f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e731f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f90e0131b40 0x7f90e0131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e731f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e009ece0 con 0x7f90e0131b40 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e731f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e009e760 con 0x7f90e006a770 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e731f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90e009e040 con 0x7f90e006b080 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e50bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e0131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46788/0 (socket says 172.21.15.32:46788) 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e50bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34554/0 (socket says 172.21.15.32:34554) 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 -- 172.21.15.32:0/1086764248 learned_addr learned my addr 172.21.15.32:0/1086764248 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e58bc700 1 --2- 172.21.15.32:0/1086764248 >> v2:172.21.15.32:3300/0 conn(0x7f90e0131b40 0x7f90e0131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 -- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 msgr2=0x7f90e006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 --2- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 -- 172.21.15.32:0/1086764248 >> v2:172.21.15.32:3300/0 conn(0x7f90e0131b40 msgr2=0x7f90e0131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 --2- 172.21.15.32:0/1086764248 >> v2:172.21.15.32:3300/0 conn(0x7f90e0131b40 0x7f90e0131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 -- 172.21.15.32:0/1086764248 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e009f630 con 0x7f90e006b080 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e50bb700 1 --2- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e58bc700 1 --2- 172.21.15.32:0/1086764248 >> v2:172.21.15.32:3300/0 conn(0x7f90e0131b40 0x7f90e0131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.304+0000 7f90e48ba700 1 --2- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e0131600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f90d000dd10 tx=0x7f90d000f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355020dc69fb0f39 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90d7fff700 1 -- 172.21.15.32:0/1086764248 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0012070 con 0x7f90e006b080 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90d7fff700 1 -- 172.21.15.32:0/1086764248 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d000d2e0 con 0x7f90e006b080 2023-09-30T22:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90d7fff700 1 -- 172.21.15.32:0/1086764248 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000fb10 con 0x7f90e006b080 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 -- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 msgr2=0x7f90e0131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 --2- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e0131600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f90d000dd10 tx=0x7f90d000f330 comp rx=0 tx=0).stop 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 -- 172.21.15.32:0/1086764248 shutdown_connections 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 --2- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 --2- 172.21.15.32:0/1086764248 >> v2:172.21.15.32:3300/0 conn(0x7f90e0131b40 0x7f90e0131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 --2- 172.21.15.32:0/1086764248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e0131600 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.305+0000 7f90e731f700 1 -- 172.21.15.32:0/1086764248 >> 172.21.15.32:0/1086764248 conn(0x7f90e0066f50 msgr2=0x7f90e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.306+0000 7f90e731f700 1 -- 172.21.15.32:0/1086764248 shutdown_connections 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.306+0000 7f90e731f700 1 -- 172.21.15.32:0/1086764248 wait complete. 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 Processor -- start 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 -- start start 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e011be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e011c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90e0131b40 0x7f90e011c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e011f930 con 0x7f90e0131b40 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e011ece0 con 0x7f90e006a770 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e731f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90e011dac0 con 0x7f90e006b080 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e48ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e011c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e48ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e011c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46794/0 (socket says 172.21.15.32:46794) 2023-09-30T22:28:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e48ba700 1 -- 172.21.15.32:0/438150606 learned_addr learned my addr 172.21.15.32:0/438150606 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e50bb700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e011be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.307+0000 7f90e58bc700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90e0131b40 0x7f90e011c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e48ba700 1 -- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 msgr2=0x7f90e011be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e48ba700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e011be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e48ba700 1 -- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90e0131b40 msgr2=0x7f90e011c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e48ba700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90e0131b40 0x7f90e011c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e48ba700 1 -- 172.21.15.32:0/438150606 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc0094a0 con 0x7f90e006b080 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e50bb700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e011be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e58bc700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90e0131b40 0x7f90e011c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e48ba700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e011c350 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f90d000f370 tx=0x7f90d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000d870 con 0x7f90e006b080 2023-09-30T22:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d000de60 con 0x7f90e006b080 2023-09-30T22:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d8009a70 con 0x7f90e006b080 2023-09-30T22:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.308+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00038e0 con 0x7f90e006b080 2023-09-30T22:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.309+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d000d9f0 con 0x7f90e006b080 2023-09-30T22:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.310+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0074ea0 con 0x7f90e006b080 2023-09-30T22:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.310+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f90d0003a40 con 0x7f90e006b080 2023-09-30T22:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.310+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f90d00447e0 con 0x7f90e006b080 2023-09-30T22:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.319+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90d0006c60 con 0x7f90e006b080 2023-09-30T22:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.337+0000 7f727caff700 1 Processor -- start 2023-09-30T22:28:46.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.338+0000 7f727caff700 1 -- start start 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727caff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 0x7f7278138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727caff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278139150 0x7f727813b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727caff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727806f720 con 0x7f72781338c0 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727caff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7278070c80 con 0x7f7278139150 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727caff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f727806fce0 con 0x7f7278136830 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 0x7f7278138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 0x7f7278138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46802/0 (socket says 172.21.15.32:46802) 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72781338c0 0x7f7278133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43806/0 (socket says 172.21.15.32:43806) 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 -- 172.21.15.32:0/1141204639 learned_addr learned my addr 172.21.15.32:0/1141204639 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7276d9d700 1 --2- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278139150 0x7f727813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 -- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278139150 msgr2=0x7f727813b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 --2- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278139150 0x7f727813b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 -- 172.21.15.32:0/1141204639 >> v1:172.21.15.32:6789/0 conn(0x7f72781338c0 legacy=0x7f7278133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f727559a700 1 -- 172.21.15.32:0/1141204639 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941623587 0 0) 0x7f727806f720 con 0x7f72781338c0 2023-09-30T22:28:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.339+0000 7f7275d9b700 1 -- 172.21.15.32:0/1141204639 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72781336a0 con 0x7f7278136830 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.340+0000 7f7275d9b700 1 --2- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 0x7f7278138c10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f726c009920 tx=0x7f726c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67154cc03270217b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.340+0000 7f727559a700 1 -- 172.21.15.32:0/1141204639 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c011830 con 0x7f7278136830 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.340+0000 7f727559a700 1 -- 172.21.15.32:0/1141204639 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c011990 con 0x7f7278136830 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.340+0000 7f727559a700 1 -- 172.21.15.32:0/1141204639 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c011d20 con 0x7f7278136830 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.340+0000 7f727caff700 1 -- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 msgr2=0x7f7278138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.340+0000 7f727caff700 1 --2- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 0x7f7278138c10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f726c009920 tx=0x7f726c009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.341+0000 7f727caff700 1 -- 172.21.15.32:0/1141204639 shutdown_connections 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.341+0000 7f727caff700 1 --2- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278139150 0x7f727813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.341+0000 7f727caff700 1 --2- 172.21.15.32:0/1141204639 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278136830 0x7f7278138c10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.341+0000 7f727caff700 1 -- 172.21.15.32:0/1141204639 >> 172.21.15.32:0/1141204639 conn(0x7f72780a2aa0 msgr2=0x7f72780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.341+0000 7f727caff700 1 -- 172.21.15.32:0/1141204639 shutdown_connections 2023-09-30T22:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.341+0000 7f727caff700 1 -- 172.21.15.32:0/1141204639 wait complete. 2023-09-30T22:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 Processor -- start 2023-09-30T22:28:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 -- start start 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72781338c0 0x7f7278118f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278136830 0x7f7278119470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 0x7f72781199b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727811c870 con 0x7f72781338c0 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727806f960 con 0x7f7278136830 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.342+0000 7f727caff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f727811bec0 con 0x7f7278139150 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 0x7f72781199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 0x7f72781199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46810/0 (socket says 172.21.15.32:46810) 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 -- 172.21.15.32:0/3072728993 learned_addr learned my addr 172.21.15.32:0/3072728993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f727659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72781338c0 0x7f7278118f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7275d9b700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278136830 0x7f7278119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 -- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278136830 msgr2=0x7f7278119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278136830 0x7f7278119470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 -- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72781338c0 msgr2=0x7f7278118f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72781338c0 0x7f7278118f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 -- 172.21.15.32:0/3072728993 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c009550 con 0x7f7278139150 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f727659c700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72781338c0 0x7f7278118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7276d9d700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 0x7f72781199b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f727000ef90 tx=0x7f727000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.343+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727000cc90 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.344+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727000f040 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.344+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72781360d0 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.344+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270014480 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.344+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727811b030 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.344+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72700145e0 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.345+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7270040260 con 0x7f7278139150 2023-09-30T22:28:46.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.345+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727811c590 con 0x7f7278139150 2023-09-30T22:28:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.350+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7270018020 con 0x7f7278139150 2023-09-30T22:28:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 Processor -- start 2023-09-30T22:28:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 -- start start 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc0a0a30 0x7f93bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93bc09d630 0x7f93bc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0a2a70 con 0x7f93bc133930 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc0a01d0 con 0x7f93bc09d630 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.382+0000 7f93c2c45700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc09e2b0 con 0x7f93bc0a0a30 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93c09e1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc0a0a30 0x7f93bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93c09e1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc0a0a30 0x7f93bc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34594/0 (socket says 172.21.15.32:34594) 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93c09e1700 1 -- 172.21.15.32:0/963041656 learned_addr learned my addr 172.21.15.32:0/963041656 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bbfff700 1 --2- 172.21.15.32:0/963041656 >> v2:172.21.15.32:3300/0 conn(0x7f93bc09d630 0x7f93bc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bbfff700 1 -- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc0a0a30 msgr2=0x7f93bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bbfff700 1 --2- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc0a0a30 0x7f93bc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bbfff700 1 -- 172.21.15.32:0/963041656 >> v1:172.21.15.32:6789/0 conn(0x7f93bc133930 legacy=0x7f93bc133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bbfff700 1 -- 172.21.15.32:0/963041656 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc09dc50 con 0x7f93bc09d630 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bbfff700 1 --2- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc09d630 0x7f93bc0a1340 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009840 tx=0x7f93a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d462184616ca4093 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.383+0000 7f93bb7fe700 1 -- 172.21.15.32:0/963041656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8011830 con 0x7f93bc09d630 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93bb7fe700 1 -- 172.21.15.32:0/963041656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a8011990 con 0x7f93bc09d630 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93bb7fe700 1 -- 172.21.15.32:0/963041656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8011d20 con 0x7f93bc09d630 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 -- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc09d630 msgr2=0x7f93bc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 --2- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc09d630 0x7f93bc0a1340 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009840 tx=0x7f93a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 -- 172.21.15.32:0/963041656 shutdown_connections 2023-09-30T22:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 --2- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc0a0a30 0x7f93bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 --2- 172.21.15.32:0/963041656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc09d630 0x7f93bc0a1340 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 -- 172.21.15.32:0/963041656 >> 172.21.15.32:0/963041656 conn(0x7f93bc0a3050 msgr2=0x7f93bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 -- 172.21.15.32:0/963041656 shutdown_connections 2023-09-30T22:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.384+0000 7f93c2c45700 1 -- 172.21.15.32:0/963041656 wait complete. 2023-09-30T22:28:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.386+0000 7f93c2c45700 1 Processor -- start 2023-09-30T22:28:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.386+0000 7f93c2c45700 1 -- start start 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c2c45700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93bc09d630 0x7f93bc11f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c2c45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 0x7f93bc11faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c2c45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc133930 0x7f93bc1c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c2c45700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc11f160 con 0x7f93bc0a0a30 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c2c45700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc120030 con 0x7f93bc133930 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c2c45700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc0a0410 con 0x7f93bc09d630 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 0x7f93bc11faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c09e1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93bc09d630 0x7f93bc11f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 0x7f93bc11faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53046/0 (socket says 172.21.15.32:53046) 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93bbfff700 1 -- 172.21.15.32:0/2605037536 learned_addr learned my addr 172.21.15.32:0/2605037536 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93c11e2700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc133930 0x7f93bc1c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93bbfff700 1 -- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93bc09d630 msgr2=0x7f93bc11f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.387+0000 7f93bbfff700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93bc09d630 0x7f93bc11f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93bbfff700 1 -- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc133930 msgr2=0x7f93bc1c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93bbfff700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc133930 0x7f93bc1c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93bbfff700 1 -- 172.21.15.32:0/2605037536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80094a0 con 0x7f93bc0a0a30 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93c09e1700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93bc09d630 0x7f93bc11f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93bbfff700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 0x7f93bc11faf0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f93a80090b0 tx=0x7f93a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80036a0 con 0x7f93bc0a0a30 2023-09-30T22:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a801f070 con 0x7f93bc0a0a30 2023-09-30T22:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8003a60 con 0x7f93bc0a0a30 2023-09-30T22:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1220a0 con 0x7f93bc0a0a30 2023-09-30T22:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.388+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc122410 con 0x7f93bc0a0a30 2023-09-30T22:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.389+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93a8003bc0 con 0x7f93bc0a0a30 2023-09-30T22:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.389+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f93a8021070 con 0x7f93bc0a0a30 2023-09-30T22:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.389+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc0747f0 con 0x7f93bc0a0a30 2023-09-30T22:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.394+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93a8015080 con 0x7f93bc0a0a30 2023-09-30T22:28:46.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.530+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f90e006c360 con 0x7f90e006b080 2023-09-30T22:28:46.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.531+0000 7f90d5ffb700 1 -- 172.21.15.32:0/438150606 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f90d001fb90 con 0x7f90e006b080 2023-09-30T22:28:46.531 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:46.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 msgr2=0x7f90e011c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e011c350 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f90d000f370 tx=0x7f90d0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 shutdown_connections 2023-09-30T22:28:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90e0131b40 0x7f90e011c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90e006a770 0x7f90e011be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 --2- 172.21.15.32:0/438150606 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f90e006b080 0x7f90e011c350 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 >> 172.21.15.32:0/438150606 conn(0x7f90e0066f50 msgr2=0x7f90e0130cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 shutdown_connections 2023-09-30T22:28:46.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.533+0000 7f90e731f700 1 -- 172.21.15.32:0/438150606 wait complete. 2023-09-30T22:28:46.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:46.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.559+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7278074ea0 con 0x7f7278139150 2023-09-30T22:28:46.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.559+0000 7f7266ffd700 1 -- 172.21.15.32:0/3072728993 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f727002b090 con 0x7f7278139150 2023-09-30T22:28:46.559 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 msgr2=0x7f72781199b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 0x7f72781199b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f727000ef90 tx=0x7f727000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 shutdown_connections 2023-09-30T22:28:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7278139150 0x7f72781199b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72781338c0 0x7f7278118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 --2- 172.21.15.32:0/3072728993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7278136830 0x7f7278119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 >> 172.21.15.32:0/3072728993 conn(0x7f72780a2aa0 msgr2=0x7f727812f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 shutdown_connections 2023-09-30T22:28:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.562+0000 7f727caff700 1 -- 172.21.15.32:0/3072728993 wait complete. 2023-09-30T22:28:46.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.602+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f93bc072980 con 0x7f93bc0a0a30 2023-09-30T22:28:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.602+0000 7f93b97fa700 1 -- 172.21.15.32:0/2605037536 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93a8035090 con 0x7f93bc0a0a30 2023-09-30T22:28:46.603 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 msgr2=0x7f93bc11faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 0x7f93bc11faf0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f93a80090b0 tx=0x7f93a8004750 comp rx=0 tx=0).stop 2023-09-30T22:28:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 shutdown_connections 2023-09-30T22:28:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93bc133930 0x7f93bc1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93bc09d630 0x7f93bc11f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 --2- 172.21.15.32:0/2605037536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93bc0a0a30 0x7f93bc11faf0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 >> 172.21.15.32:0/2605037536 conn(0x7f93bc0a3050 msgr2=0x7f93bc1bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 shutdown_connections 2023-09-30T22:28:46.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:46.605+0000 7f93c2c45700 1 -- 172.21.15.32:0/2605037536 wait complete. 2023-09-30T22:28:46.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:46.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:47.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.061+0000 7fc63337d700 1 Processor -- start 2023-09-30T22:28:47.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.061+0000 7fc63337d700 1 -- start start 2023-09-30T22:28:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63337d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09e930 0x7fc62c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63337d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f240 0x7fc62c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63337d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 0x7fc62c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63337d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c070940 con 0x7fc62c09f650 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63337d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06fd60 con 0x7fc62c09e930 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63337d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06fa80 con 0x7fc62c09f240 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63191a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 0x7fc62c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc630918700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f240 0x7fc62c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc631119700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09e930 0x7fc62c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63191a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 0x7fc62c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53054/0 (socket says 172.21.15.32:53054) 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63191a700 1 -- 172.21.15.32:0/156227480 learned_addr learned my addr 172.21.15.32:0/156227480 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc630918700 1 -- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09e930 msgr2=0x7fc62c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:47.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc630918700 1 --2- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09e930 0x7fc62c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc630918700 1 -- 172.21.15.32:0/156227480 >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 msgr2=0x7fc62c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc630918700 1 --2- 172.21.15.32:0/156227480 >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 0x7fc62c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc630918700 1 -- 172.21.15.32:0/156227480 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c09fb60 con 0x7fc62c09f240 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.062+0000 7fc63191a700 1 --2- 172.21.15.32:0/156227480 >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 0x7fc62c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc630918700 1 --2- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f240 0x7fc62c1317f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009840 tx=0x7fc61c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3304e440c037f56c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc623fff700 1 -- 172.21.15.32:0/156227480 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011830 con 0x7fc62c09f240 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc623fff700 1 -- 172.21.15.32:0/156227480 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c011990 con 0x7fc62c09f240 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc623fff700 1 -- 172.21.15.32:0/156227480 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c011d20 con 0x7fc62c09f240 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 -- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f240 msgr2=0x7fc62c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 --2- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f240 0x7fc62c1317f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009840 tx=0x7fc61c009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 -- 172.21.15.32:0/156227480 shutdown_connections 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 --2- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09e930 0x7fc62c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 --2- 172.21.15.32:0/156227480 >> v2:172.21.15.32:3300/0 conn(0x7fc62c09f650 0x7fc62c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 --2- 172.21.15.32:0/156227480 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f240 0x7fc62c1317f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 -- 172.21.15.32:0/156227480 >> 172.21.15.32:0/156227480 conn(0x7fc62c0a4210 msgr2=0x7fc62c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.063+0000 7fc63337d700 1 -- 172.21.15.32:0/156227480 shutdown_connections 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.064+0000 7fc63337d700 1 -- 172.21.15.32:0/156227480 wait complete. 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.064+0000 7fc63337d700 1 Processor -- start 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 -- start start 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc62c09e930 0x7fc62c1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 0x7fc62c1a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f650 0x7fc62c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1a70a0 con 0x7fc62c09e930 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1a66e0 con 0x7fc62c09f240 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63337d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0a3670 con 0x7fc62c09f650 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 0x7fc62c1a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc631119700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc62c09e930 0x7fc62c1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 0x7fc62c1a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34628/0 (socket says 172.21.15.32:34628) 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc63191a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f650 0x7fc62c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 -- 172.21.15.32:0/1850981543 learned_addr learned my addr 172.21.15.32:0/1850981543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 -- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f650 msgr2=0x7fc62c1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f650 0x7fc62c1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 -- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc62c09e930 msgr2=0x7fc62c1a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.065+0000 7fc630918700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc62c09e930 0x7fc62c1a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc630918700 1 -- 172.21.15.32:0/1850981543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc624009b30 con 0x7fc62c09f240 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc630918700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 0x7fc62c1a3100 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc61c000c00 tx=0x7fc61c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004450 con 0x7fc62c09f240 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c01f070 con 0x7fc62c09f240 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0038e0 con 0x7fc62c09f240 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61c0094a0 con 0x7fc62c09f240 2023-09-30T22:28:47.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.066+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c0a0190 con 0x7fc62c09f240 2023-09-30T22:28:47.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.067+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc61c003a40 con 0x7fc62c09f240 2023-09-30T22:28:47.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.067+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc61c021070 con 0x7fc62c09f240 2023-09-30T22:28:47.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.067+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c1a4830 con 0x7fc62c09f240 2023-09-30T22:28:47.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.072+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc61c015020 con 0x7fc62c09f240 2023-09-30T22:28:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.278+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc62c074ea0 con 0x7fc62c09f240 2023-09-30T22:28:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.279+0000 7fc621ffb700 1 -- 172.21.15.32:0/1850981543 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc61c034090 con 0x7fc62c09f240 2023-09-30T22:28:47.279 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 msgr2=0x7fc62c1a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 0x7fc62c1a3100 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc61c000c00 tx=0x7fc61c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 shutdown_connections 2023-09-30T22:28:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc62c09f650 0x7fc62c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc62c09e930 0x7fc62c1a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 --2- 172.21.15.32:0/1850981543 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc62c09f240 0x7fc62c1a3100 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 >> 172.21.15.32:0/1850981543 conn(0x7fc62c0a4210 msgr2=0x7fc62c1b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 shutdown_connections 2023-09-30T22:28:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.281+0000 7fc63337d700 1 -- 172.21.15.32:0/1850981543 wait complete. 2023-09-30T22:28:47.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:47.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:47.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:47.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 Processor -- start 2023-09-30T22:28:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 -- start start 2023-09-30T22:28:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809d6f0 0x7f1d3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 0x7f1d38130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d38130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3809f520 con 0x7f1d3809e000 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3809f130 con 0x7f1d3809d6f0 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d3d421700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3809e930 con 0x7f1d3809e410 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d367fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 0x7f1d38130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.701+0000 7f1d36ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809d6f0 0x7f1d3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d367fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 0x7f1d38130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53058/0 (socket says 172.21.15.32:53058) 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d367fc700 1 -- 172.21.15.32:0/3790238450 learned_addr learned my addr 172.21.15.32:0/3790238450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d377fe700 1 --2- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d38130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d36ffd700 1 -- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 msgr2=0x7f1d38130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d36ffd700 1 --2- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d38130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d36ffd700 1 -- 172.21.15.32:0/3790238450 >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 msgr2=0x7f1d38130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d36ffd700 1 --2- 172.21.15.32:0/3790238450 >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 0x7f1d38130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d36ffd700 1 -- 172.21.15.32:0/3790238450 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3809b2d0 con 0x7f1d3809d6f0 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d367fc700 1 --2- 172.21.15.32:0/3790238450 >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 0x7f1d38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d36ffd700 1 --2- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809d6f0 0x7f1d3809dac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009840 tx=0x7f1d28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6f5c0d7c4bb615d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d35ffb700 1 -- 172.21.15.32:0/3790238450 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28011830 con 0x7f1d3809d6f0 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d35ffb700 1 -- 172.21.15.32:0/3790238450 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d28011990 con 0x7f1d3809d6f0 2023-09-30T22:28:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.702+0000 7f1d35ffb700 1 -- 172.21.15.32:0/3790238450 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28011d20 con 0x7f1d3809d6f0 2023-09-30T22:28:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 -- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809d6f0 msgr2=0x7f1d3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 --2- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809d6f0 0x7f1d3809dac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009840 tx=0x7f1d28009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 -- 172.21.15.32:0/3790238450 shutdown_connections 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 --2- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809d6f0 0x7f1d3809dac0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 --2- 172.21.15.32:0/3790238450 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d38130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 --2- 172.21.15.32:0/3790238450 >> v2:172.21.15.32:3300/0 conn(0x7f1d3809e000 0x7f1d38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 -- 172.21.15.32:0/3790238450 >> 172.21.15.32:0/3790238450 conn(0x7f1d38066f50 msgr2=0x7f1d3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 -- 172.21.15.32:0/3790238450 shutdown_connections 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.703+0000 7f1d3d421700 1 -- 172.21.15.32:0/3790238450 wait complete. 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.705+0000 7f1d3d421700 1 Processor -- start 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.705+0000 7f1d3d421700 1 -- start start 2023-09-30T22:28:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d3d421700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 0x7f1d3811ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d3d421700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809e000 0x7f1d3811e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d3d421700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d3811e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d3d421700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381219f0 con 0x7f1d3809d6f0 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d3d421700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380a16b0 con 0x7f1d3809e000 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d3d421700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d38121180 con 0x7f1d3809e410 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 0x7f1d3811ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 0x7f1d3811ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53062/0 (socket says 172.21.15.32:53062) 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 -- 172.21.15.32:0/4032425362 learned_addr learned my addr 172.21.15.32:0/4032425362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d377fe700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d3811e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d367fc700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809e000 0x7f1d3811e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 -- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 msgr2=0x7f1d3811e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d3811e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 -- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809e000 msgr2=0x7f1d3811e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809e000 0x7f1d3811e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d36ffd700 1 -- 172.21.15.32:0/4032425362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c009550 con 0x7f1d3809d6f0 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.706+0000 7f1d377fe700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d3811e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.707+0000 7f1d36ffd700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 0x7f1d3811ded0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009fd0 tx=0x7f1d28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.707+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28004450 con 0x7f1d3809d6f0 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.707+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2801f070 con 0x7f1d3809d6f0 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.707+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280038e0 con 0x7f1d3809d6f0 2023-09-30T22:28:47.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.707+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d280094a0 con 0x7f1d3809d6f0 2023-09-30T22:28:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.707+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3809c750 con 0x7f1d3809d6f0 2023-09-30T22:28:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.708+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d28003a40 con 0x7f1d3809d6f0 2023-09-30T22:28:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.708+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d28021070 con 0x7f1d3809d6f0 2023-09-30T22:28:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.708+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d38074ea0 con 0x7f1d3809d6f0 2023-09-30T22:28:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.718+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d2801a430 con 0x7f1d3809d6f0 2023-09-30T22:28:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.739+0000 7fcae316a700 1 Processor -- start 2023-09-30T22:28:47.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.739+0000 7fcae316a700 1 -- start start 2023-09-30T22:28:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcae316a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcae316a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcadc139250 0x7fcadc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcae316a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc09edd0 con 0x7fcadc0a0a30 2023-09-30T22:28:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcae316a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc0a18e0 con 0x7fcadc139250 2023-09-30T22:28:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcae316a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc09e1f0 con 0x7fcadc0a1510 2023-09-30T22:28:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34638/0 (socket says 172.21.15.32:34638) 2023-09-30T22:28:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcadbfff700 1 -- 172.21.15.32:0/1650828196 learned_addr learned my addr 172.21.15.32:0/1650828196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.740+0000 7fcae1707700 1 --2- 172.21.15.32:0/1650828196 >> v2:172.21.15.32:3300/0 conn(0x7fcadc139250 0x7fcadc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadbfff700 1 -- 172.21.15.32:0/1650828196 >> v2:172.21.15.32:3300/0 conn(0x7fcadc139250 msgr2=0x7fcadc13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadbfff700 1 --2- 172.21.15.32:0/1650828196 >> v2:172.21.15.32:3300/0 conn(0x7fcadc139250 0x7fcadc13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadbfff700 1 -- 172.21.15.32:0/1650828196 >> v1:172.21.15.32:6789/0 conn(0x7fcadc0a0a30 legacy=0x7fcadc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadbfff700 1 -- 172.21.15.32:0/1650828196 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc09dc50 con 0x7fcadc0a1510 2023-09-30T22:28:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadbfff700 1 --2- 172.21.15.32:0/1650828196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc138d10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009920 tx=0x7fcac8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d073b44d7892d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadb7fe700 1 -- 172.21.15.32:0/1650828196 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac8011830 con 0x7fcadc0a1510 2023-09-30T22:28:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadb7fe700 1 -- 172.21.15.32:0/1650828196 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac8011990 con 0x7fcadc0a1510 2023-09-30T22:28:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.741+0000 7fcadb7fe700 1 -- 172.21.15.32:0/1650828196 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac8011d20 con 0x7fcadc0a1510 2023-09-30T22:28:47.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 -- 172.21.15.32:0/1650828196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 msgr2=0x7fcadc138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 --2- 172.21.15.32:0/1650828196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc138d10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009920 tx=0x7fcac8009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 -- 172.21.15.32:0/1650828196 shutdown_connections 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 --2- 172.21.15.32:0/1650828196 >> v2:172.21.15.32:3300/0 conn(0x7fcadc139250 0x7fcadc13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 --2- 172.21.15.32:0/1650828196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc138d10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 -- 172.21.15.32:0/1650828196 >> 172.21.15.32:0/1650828196 conn(0x7fcadc0a6550 msgr2=0x7fcadc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 -- 172.21.15.32:0/1650828196 shutdown_connections 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.742+0000 7fcae316a700 1 -- 172.21.15.32:0/1650828196 wait complete. 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.743+0000 7fcae316a700 1 Processor -- start 2023-09-30T22:28:47.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.743+0000 7fcae316a700 1 -- start start 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae316a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0a0a30 0x7fcadc1218d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae316a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc121e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae316a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcadc139250 0x7fcadc122350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae316a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1236d0 con 0x7fcadc0a0a30 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae316a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1234e0 con 0x7fcadc0a1510 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae316a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1253f0 con 0x7fcadc139250 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc121e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae0f06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0a0a30 0x7fcadc1218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc121e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34642/0 (socket says 172.21.15.32:34642) 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcadbfff700 1 -- 172.21.15.32:0/2944569979 learned_addr learned my addr 172.21.15.32:0/2944569979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae1707700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcadc139250 0x7fcadc122350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae1707700 1 -- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 msgr2=0x7fcadc121e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae1707700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc121e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae1707700 1 -- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0a0a30 msgr2=0x7fcadc1218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae1707700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0a0a30 0x7fcadc1218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae1707700 1 -- 172.21.15.32:0/2944569979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8009550 con 0x7fcadc139250 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcae0f06700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0a0a30 0x7fcadc1218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.744+0000 7fcadbfff700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc121e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.745+0000 7fcae1707700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcadc139250 0x7fcadc122350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcad400ef90 tx=0x7fcad400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.745+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad400cc90 con 0x7fcadc139250 2023-09-30T22:28:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.745+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad400f040 con 0x7fcadc139250 2023-09-30T22:28:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.745+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc138c30 con 0x7fcadc139250 2023-09-30T22:28:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.745+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40146c0 con 0x7fcadc139250 2023-09-30T22:28:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.745+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc123cd0 con 0x7fcadc139250 2023-09-30T22:28:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.746+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcad4014820 con 0x7fcadc139250 2023-09-30T22:28:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.746+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcad4040290 con 0x7fcadc139250 2023-09-30T22:28:47.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.746+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc074ea0 con 0x7fcadc139250 2023-09-30T22:28:47.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.751+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcad401d400 con 0x7fcadc139250 2023-09-30T22:28:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.783+0000 7fe805dbd700 1 Processor -- start 2023-09-30T22:28:47.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.783+0000 7fe805dbd700 1 -- start start 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe805dbd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009d6f0 0x7fe80009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe805dbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009e410 0x7fe800134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe805dbd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80009b120 con 0x7fe80009e000 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe805dbd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80006b460 con 0x7fe80009e410 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe805dbd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe80009a7e0 con 0x7fe80009d6f0 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7feffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe80009e000 0x7fe8001339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43838/0 (socket says 172.21.15.32:43838) 2023-09-30T22:28:47.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7feffd700 1 -- 172.21.15.32:0/3190157203 learned_addr learned my addr 172.21.15.32:0/3190157203 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fffff700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009e410 0x7fe800134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7ff7fe700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009d6f0 0x7fe80009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fe7fc700 1 -- 172.21.15.32:0/3190157203 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952939789 0 0) 0x7fe80009b120 con 0x7fe80009e000 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fe7fc700 1 -- 172.21.15.32:0/3190157203 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e80035b0 con 0x7fe80009e000 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fffff700 1 -- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009d6f0 msgr2=0x7fe80009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fffff700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009d6f0 0x7fe80009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fffff700 1 -- 172.21.15.32:0/3190157203 >> v1:172.21.15.32:6789/0 conn(0x7fe80009e000 legacy=0x7fe8001339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.784+0000 7fe7fffff700 1 -- 172.21.15.32:0/3190157203 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80009a500 con 0x7fe80009e410 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.785+0000 7fe7fffff700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009e410 0x7fe800134100 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009dc0 tx=0x7fe7f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6cd9841ec61f13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.785+0000 7fe7fe7fc700 1 -- 172.21.15.32:0/3190157203 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800e070 con 0x7fe80009e410 2023-09-30T22:28:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.785+0000 7fe7fe7fc700 1 -- 172.21.15.32:0/3190157203 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f800bb20 con 0x7fe80009e410 2023-09-30T22:28:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.785+0000 7fe7fe7fc700 1 -- 172.21.15.32:0/3190157203 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8012430 con 0x7fe80009e410 2023-09-30T22:28:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 -- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009e410 msgr2=0x7fe800134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009e410 0x7fe800134100 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009dc0 tx=0x7fe7f800b330 comp rx=0 tx=0).stop 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 -- 172.21.15.32:0/3190157203 shutdown_connections 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009e410 0x7fe800134100 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3190157203 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009d6f0 0x7fe80009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 -- 172.21.15.32:0/3190157203 >> 172.21.15.32:0/3190157203 conn(0x7fe800066f50 msgr2=0x7fe80009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 -- 172.21.15.32:0/3190157203 shutdown_connections 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.786+0000 7fe805dbd700 1 -- 172.21.15.32:0/3190157203 wait complete. 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.787+0000 7fe805dbd700 1 Processor -- start 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 -- start start 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 0x7fe800122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009e000 0x7fe800122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80009e410 0x7fe800122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:47.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe800123ea0 con 0x7fe80009e410 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe800123780 con 0x7fe80009d6f0 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe805dbd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe800124d30 con 0x7fe80009e000 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe7ff7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 0x7fe800122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe7ff7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 0x7fe800122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34660/0 (socket says 172.21.15.32:34660) 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.788+0000 7fe7ff7fe700 1 -- 172.21.15.32:0/3633518907 learned_addr learned my addr 172.21.15.32:0/3633518907 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7feffd700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009e000 0x7fe800122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7fffff700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80009e410 0x7fe800122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7ff7fe700 1 -- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009e000 msgr2=0x7fe800122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7ff7fe700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009e000 0x7fe800122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7ff7fe700 1 -- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80009e410 msgr2=0x7fe800122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7ff7fe700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80009e410 0x7fe800122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7ff7fe700 1 -- 172.21.15.32:0/3633518907 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8009a70 con 0x7fe80009d6f0 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7ff7fe700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 0x7fe800122320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e9e0 tx=0x7fe7f000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0009710 con 0x7fe80009d6f0 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f000ee80 con 0x7fe80009d6f0 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe800125810 con 0x7fe80009d6f0 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000ca50 con 0x7fe80009d6f0 2023-09-30T22:28:47.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.789+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80009c710 con 0x7fe80009d6f0 2023-09-30T22:28:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.790+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe7f000cbb0 con 0x7fe80009d6f0 2023-09-30T22:28:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.791+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe7f003fb80 con 0x7fe80009d6f0 2023-09-30T22:28:47.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.791+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80011f030 con 0x7fe80009d6f0 2023-09-30T22:28:47.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.796+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7f0021080 con 0x7fe80009d6f0 2023-09-30T22:28:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.930+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d38121570 con 0x7f1d3809d6f0 2023-09-30T22:28:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.930+0000 7f1d1f7fe700 1 -- 172.21.15.32:0/4032425362 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d28015030 con 0x7f1d3809d6f0 2023-09-30T22:28:47.930 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:47.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.932+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 msgr2=0x7f1d3811ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.932+0000 7f1d3d421700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 0x7f1d3811ded0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009fd0 tx=0x7f1d28004750 comp rx=0 tx=0).stop 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.932+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 shutdown_connections 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.932+0000 7f1d3d421700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d3809e410 0x7f1d3811e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.932+0000 7f1d3d421700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3809d6f0 0x7f1d3811ded0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.932+0000 7f1d3d421700 1 --2- 172.21.15.32:0/4032425362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d3809e000 0x7f1d3811e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.933+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 >> 172.21.15.32:0/4032425362 conn(0x7f1d38066f50 msgr2=0x7f1d3806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.933+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 shutdown_connections 2023-09-30T22:28:47.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.933+0000 7f1d3d421700 1 -- 172.21.15.32:0/4032425362 wait complete. 2023-09-30T22:28:47.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.958+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcadc0a28e0 con 0x7fcadc139250 2023-09-30T22:28:47.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.959+0000 7fcad97fa700 1 -- 172.21.15.32:0/2944569979 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcad4007330 con 0x7fcadc139250 2023-09-30T22:28:47.959 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcadc139250 msgr2=0x7fcadc122350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcadc139250 0x7fcadc122350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcad400ef90 tx=0x7fcad400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 shutdown_connections 2023-09-30T22:28:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcadc139250 0x7fcadc122350 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcadc0a0a30 0x7fcadc1218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 --2- 172.21.15.32:0/2944569979 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcadc0a1510 0x7fcadc121e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 >> 172.21.15.32:0/2944569979 conn(0x7fcadc0a6550 msgr2=0x7fcadc137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 shutdown_connections 2023-09-30T22:28:47.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:47.961+0000 7fcae316a700 1 -- 172.21.15.32:0/2944569979 wait complete. 2023-09-30T22:28:47.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.003+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe8001259a0 con 0x7fe80009d6f0 2023-09-30T22:28:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.003+0000 7fe7e7fff700 1 -- 172.21.15.32:0/3633518907 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe7f001b3b0 con 0x7fe80009d6f0 2023-09-30T22:28:48.003 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 msgr2=0x7fe800122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 0x7fe800122320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e9e0 tx=0x7fe7f000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 shutdown_connections 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe80009e410 0x7fe800122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe80009d6f0 0x7fe800122320 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 --2- 172.21.15.32:0/3633518907 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe80009e000 0x7fe800122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 >> 172.21.15.32:0/3633518907 conn(0x7fe800066f50 msgr2=0x7fe800133f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 shutdown_connections 2023-09-30T22:28:48.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.005+0000 7fe805dbd700 1 -- 172.21.15.32:0/3633518907 wait complete. 2023-09-30T22:28:48.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:48.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.442+0000 7fd0b8a25700 1 Processor -- start 2023-09-30T22:28:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.442+0000 7fd0b8a25700 1 -- start start 2023-09-30T22:28:48.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b8a25700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0b41316c0 0x7fd0b4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b8a25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b406ae80 0x7fd0b4139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b8a25700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b409e130 con 0x7fd0b409d9f0 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b8a25700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b41312b0 con 0x7fd0b41316c0 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b8a25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b409f6e0 con 0x7fd0b406ae80 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0b41316c0 0x7fd0b4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0a9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0b409d9f0 0x7fd0b406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43846/0 (socket says 172.21.15.32:43846) 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0b41316c0 0x7fd0b4131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53086/0 (socket says 172.21.15.32:53086) 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0a9d9b700 1 -- 172.21.15.32:0/46998007 learned_addr learned my addr 172.21.15.32:0/46998007 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b2d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b406ae80 0x7fd0b4139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b259c700 1 -- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b406ae80 msgr2=0x7fd0b4139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:48.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b259c700 1 --2- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b406ae80 0x7fd0b4139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b259c700 1 -- 172.21.15.32:0/46998007 >> v1:172.21.15.32:6789/0 conn(0x7fd0b409d9f0 legacy=0x7fd0b406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b259c700 1 -- 172.21.15.32:0/46998007 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b409e760 con 0x7fd0b41316c0 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b2d9d700 1 --2- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b406ae80 0x7fd0b4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.443+0000 7fd0b1d9b700 1 -- 172.21.15.32:0/46998007 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641321533 0 0) 0x7fd09c001110 con 0x7fd0b409d9f0 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b259c700 1 --2- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b41316c0 0x7fd0b4131a90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009840 tx=0x7fd0a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dadf38e58118c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b1d9b700 1 -- 172.21.15.32:0/46998007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4011830 con 0x7fd0b41316c0 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b1d9b700 1 -- 172.21.15.32:0/46998007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4011990 con 0x7fd0b41316c0 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b1d9b700 1 -- 172.21.15.32:0/46998007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4011d20 con 0x7fd0b41316c0 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 -- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b41316c0 msgr2=0x7fd0b4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:48.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b41316c0 0x7fd0b4131a90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009840 tx=0x7fd0a4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 -- 172.21.15.32:0/46998007 shutdown_connections 2023-09-30T22:28:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b406ae80 0x7fd0b4139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/46998007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b41316c0 0x7fd0b4131a90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 -- 172.21.15.32:0/46998007 >> 172.21.15.32:0/46998007 conn(0x7fd0b4066f50 msgr2=0x7fd0b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 -- 172.21.15.32:0/46998007 shutdown_connections 2023-09-30T22:28:48.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.444+0000 7fd0b8a25700 1 -- 172.21.15.32:0/46998007 wait complete. 2023-09-30T22:28:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 Processor -- start 2023-09-30T22:28:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 -- start start 2023-09-30T22:28:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 0x7fd0b4070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b409d9f0 0x7fd0b4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0b41316c0 0x7fd0b4071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:48.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b411ac10 con 0x7fd0b406ae80 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b411adf0 con 0x7fd0b409d9f0 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b8a25700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b412b900 con 0x7fd0b41316c0 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0a9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b409d9f0 0x7fd0b4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 0x7fd0b4070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0b41316c0 0x7fd0b4071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.446+0000 7fd0b259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 0x7fd0b4070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53102/0 (socket says 172.21.15.32:53102) 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 -- 172.21.15.32:0/1477742542 learned_addr learned my addr 172.21.15.32:0/1477742542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 -- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0b41316c0 msgr2=0x7fd0b4071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0b41316c0 0x7fd0b4071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 -- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b409d9f0 msgr2=0x7fd0b4071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b409d9f0 0x7fd0b4071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 -- 172.21.15.32:0/1477742542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009a70 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0a9d9b700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b409d9f0 0x7fd0b4071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b259c700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 0x7fd0b4070ae0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40090b0 tx=0x7fd0a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40045d0 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a401f070 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a40094a0 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40038c0 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.447+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b40a3760 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.448+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd0a4003a20 con 0x7fd0b406ae80 2023-09-30T22:28:48.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.448+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd0a4021070 con 0x7fd0b406ae80 2023-09-30T22:28:48.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.448+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b4074ea0 con 0x7fd0b406ae80 2023-09-30T22:28:48.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.453+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd0a402b020 con 0x7fd0b406ae80 2023-09-30T22:28:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.658+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd0b411a9c0 con 0x7fd0b406ae80 2023-09-30T22:28:48.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.658+0000 7fd0ab7fe700 1 -- 172.21.15.32:0/1477742542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0a4034070 con 0x7fd0b406ae80 2023-09-30T22:28:48.659 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.660+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 msgr2=0x7fd0b4070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.660+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 0x7fd0b4070ae0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40090b0 tx=0x7fd0a4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 shutdown_connections 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0b41316c0 0x7fd0b4071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0b406ae80 0x7fd0b4070ae0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 --2- 172.21.15.32:0/1477742542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0b409d9f0 0x7fd0b4071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 >> 172.21.15.32:0/1477742542 conn(0x7fd0b4066f50 msgr2=0x7fd0b40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 shutdown_connections 2023-09-30T22:28:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:48.661+0000 7fd0b8a25700 1 -- 172.21.15.32:0/1477742542 wait complete. 2023-09-30T22:28:48.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:48.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:48.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:49.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:49.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 Processor -- start 2023-09-30T22:28:49.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 -- start start 2023-09-30T22:28:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f661406a770 0x7f661406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f6614139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661409ece0 con 0x7f661406b080 2023-09-30T22:28:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661409e760 con 0x7f661406a770 2023-09-30T22:28:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.099+0000 7f661bce1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661409e040 con 0x7f6614131d10 2023-09-30T22:28:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6619a7d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f661406a770 0x7f661406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f661927c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f661406b080 0x7f6614131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43854/0 (socket says 172.21.15.32:43854) 2023-09-30T22:28:49.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f661927c700 1 -- 172.21.15.32:0/775935583 learned_addr learned my addr 172.21.15.32:0/775935583 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f661a27e700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f6614139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6619a7d700 1 -- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 msgr2=0x7f6614139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6619a7d700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f6614139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6619a7d700 1 -- 172.21.15.32:0/775935583 >> v1:172.21.15.32:6789/0 conn(0x7f661406b080 legacy=0x7f6614131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6619a7d700 1 -- 172.21.15.32:0/775935583 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661409f630 con 0x7f661406a770 2023-09-30T22:28:49.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f661a27e700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f6614139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6619a7d700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406a770 0x7f661406ab40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f6610009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c188faee0a03a9ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6618a7b700 1 -- 172.21.15.32:0/775935583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011830 con 0x7f661406a770 2023-09-30T22:28:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6618a7b700 1 -- 172.21.15.32:0/775935583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610011990 con 0x7f661406a770 2023-09-30T22:28:49.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.100+0000 7f6618a7b700 1 -- 172.21.15.32:0/775935583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011d20 con 0x7f661406a770 2023-09-30T22:28:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.101+0000 7f661bce1700 1 -- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406a770 msgr2=0x7f661406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.101+0000 7f661bce1700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406a770 0x7f661406ab40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f6610009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.102+0000 7f661bce1700 1 -- 172.21.15.32:0/775935583 shutdown_connections 2023-09-30T22:28:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.102+0000 7f661bce1700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f6614139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.102+0000 7f661bce1700 1 --2- 172.21.15.32:0/775935583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406a770 0x7f661406ab40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.102+0000 7f661bce1700 1 -- 172.21.15.32:0/775935583 >> 172.21.15.32:0/775935583 conn(0x7f6614066f50 msgr2=0x7f661409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.102+0000 7f661bce1700 1 -- 172.21.15.32:0/775935583 shutdown_connections 2023-09-30T22:28:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.102+0000 7f661bce1700 1 -- 172.21.15.32:0/775935583 wait complete. 2023-09-30T22:28:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.103+0000 7f661bce1700 1 Processor -- start 2023-09-30T22:28:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.103+0000 7f661bce1700 1 -- start start 2023-09-30T22:28:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661bce1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f661406a770 0x7f66141c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661bce1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 0x7f66141c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661bce1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f661411e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661bce1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614121d60 con 0x7f661406b080 2023-09-30T22:28:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661bce1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6614130db0 con 0x7f6614131d10 2023-09-30T22:28:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661bce1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6614120050 con 0x7f661406a770 2023-09-30T22:28:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 0x7f66141c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f6619a7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f661406a770 0x7f66141c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 0x7f66141c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53130/0 (socket says 172.21.15.32:53130) 2023-09-30T22:28:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 -- 172.21.15.32:0/4078450396 learned_addr learned my addr 172.21.15.32:0/4078450396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f6619a7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f661406a770 0x7f66141c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46916/0 (socket says 172.21.15.32:46916) 2023-09-30T22:28:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661a27e700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f661411e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 -- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f661406a770 msgr2=0x7f66141c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f661406a770 0x7f66141c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 -- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 msgr2=0x7f661411e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.104+0000 7f661927c700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f661411e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f661927c700 1 -- 172.21.15.32:0/4078450396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100094a0 con 0x7f661406b080 2023-09-30T22:28:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f661927c700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 0x7f66141c4880 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f660400c550 tx=0x7f660400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604005d70 con 0x7f661406b080 2023-09-30T22:28:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66040070d0 con 0x7f661406b080 2023-09-30T22:28:49.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6614071e60 con 0x7f661406b080 2023-09-30T22:28:49.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604007440 con 0x7f661406b080 2023-09-30T22:28:49.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.105+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661409c260 con 0x7f661406b080 2023-09-30T22:28:49.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.106+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66040075a0 con 0x7f661406b080 2023-09-30T22:28:49.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.106+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f660403ea50 con 0x7f661406b080 2023-09-30T22:28:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.107+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6614074ea0 con 0x7f661406b080 2023-09-30T22:28:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.112+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6604019b40 con 0x7f661406b080 2023-09-30T22:28:49.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 Processor -- start 2023-09-30T22:28:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 -- start start 2023-09-30T22:28:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57cc09d6f0 0x7f57cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09e440 0x7f57cc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc09b120 con 0x7f57cc09e000 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc06b460 con 0x7f57cc09d6f0 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.137+0000 7f57d0886700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc09a7e0 con 0x7f57cc09e440 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57ca59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57cc09d6f0 0x7f57cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57c9d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57cc09e000 0x7f57cc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43864/0 (socket says 172.21.15.32:43864) 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57c9d9b700 1 -- 172.21.15.32:0/714335769 learned_addr learned my addr 172.21.15.32:0/714335769 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:49.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57cad9d700 1 --2- 172.21.15.32:0/714335769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09e440 0x7f57cc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57c959a700 1 -- 172.21.15.32:0/714335769 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202355060 0 0) 0x7f57cc09b120 con 0x7f57cc09e000 2023-09-30T22:28:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57cad9d700 1 -- 172.21.15.32:0/714335769 >> v2:172.21.15.32:3300/0 conn(0x7f57cc09d6f0 msgr2=0x7f57cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57cad9d700 1 --2- 172.21.15.32:0/714335769 >> v2:172.21.15.32:3300/0 conn(0x7f57cc09d6f0 0x7f57cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57cad9d700 1 -- 172.21.15.32:0/714335769 >> v1:172.21.15.32:6789/0 conn(0x7f57cc09e000 legacy=0x7f57cc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57cad9d700 1 -- 172.21.15.32:0/714335769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc09a500 con 0x7f57cc09e440 2023-09-30T22:28:49.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57ca59c700 1 --2- 172.21.15.32:0/714335769 >> v2:172.21.15.32:3300/0 conn(0x7f57cc09d6f0 0x7f57cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57cad9d700 1 --2- 172.21.15.32:0/714335769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09e440 0x7f57cc134100 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009dc0 tx=0x7f57c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13ecf4aa03e7228 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57c959a700 1 -- 172.21.15.32:0/714335769 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400e070 con 0x7f57cc09e440 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57c959a700 1 -- 172.21.15.32:0/714335769 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c400bae0 con 0x7f57cc09e440 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.138+0000 7f57c959a700 1 -- 172.21.15.32:0/714335769 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400be90 con 0x7f57cc09e440 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 -- 172.21.15.32:0/714335769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09e440 msgr2=0x7f57cc134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 --2- 172.21.15.32:0/714335769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09e440 0x7f57cc134100 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009dc0 tx=0x7f57c400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 -- 172.21.15.32:0/714335769 shutdown_connections 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 --2- 172.21.15.32:0/714335769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09e440 0x7f57cc134100 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 --2- 172.21.15.32:0/714335769 >> v2:172.21.15.32:3300/0 conn(0x7f57cc09d6f0 0x7f57cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 -- 172.21.15.32:0/714335769 >> 172.21.15.32:0/714335769 conn(0x7f57cc066f50 msgr2=0x7f57cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 -- 172.21.15.32:0/714335769 shutdown_connections 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.139+0000 7f57d0886700 1 -- 172.21.15.32:0/714335769 wait complete. 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 Processor -- start 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 -- start start 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 0x7f57cc1c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57cc09e000 0x7f57cc1c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57cc09e440 0x7f57cc1c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a65d0 con 0x7f57cc09e440 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a63e0 con 0x7f57cc09e000 2023-09-30T22:28:49.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57d0886700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0a2bb0 con 0x7f57cc09d6f0 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 0x7f57cc1c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 0x7f57cc1c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46932/0 (socket says 172.21.15.32:46932) 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57ca59c700 1 -- 172.21.15.32:0/814908175 learned_addr learned my addr 172.21.15.32:0/814908175 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.141+0000 7f57cad9d700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57cc09e440 0x7f57cc1c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57ca59c700 1 -- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57cc09e000 msgr2=0x7f57cc1c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57ca59c700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57cc09e000 0x7f57cc1c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57ca59c700 1 -- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57cc09e440 msgr2=0x7f57cc1c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57ca59c700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57cc09e440 0x7f57cc1c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57ca59c700 1 -- 172.21.15.32:0/814908175 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0094a0 con 0x7f57cc09d6f0 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57cad9d700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57cc09e440 0x7f57cc1c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57ca59c700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 0x7f57cc1c7fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00ec10 tx=0x7f57bc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00c910 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00ca70 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c4009a70 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00cde0 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.142+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc1a8010 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.143+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc1a8200 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.143+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f57bc013470 con 0x7f57cc09d6f0 2023-09-30T22:28:49.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.144+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f57bc03fc70 con 0x7f57cc09d6f0 2023-09-30T22:28:49.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.153+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57bc01b3b0 con 0x7f57cc09d6f0 2023-09-30T22:28:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.179+0000 7fd753446700 1 Processor -- start 2023-09-30T22:28:49.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.180+0000 7fd753446700 1 -- start start 2023-09-30T22:28:49.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.180+0000 7fd753446700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd74c133420 0x7fd74c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.180+0000 7fd753446700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.180+0000 7fd753446700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74c06b990 con 0x7fd74c133e30 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.180+0000 7fd753446700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74c0a0590 con 0x7fd74c133420 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.180+0000 7fd753446700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c09fb80 con 0x7fd74c134260 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7519e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7519e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c1306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34732/0 (socket says 172.21.15.32:34732) 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7519e3700 1 -- 172.21.15.32:0/1845859767 learned_addr learned my addr 172.21.15.32:0/1845859767 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:49.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7511e2700 1 --2- 172.21.15.32:0/1845859767 >> v2:172.21.15.32:3300/0 conn(0x7fd74c133420 0x7fd74c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7511e2700 1 -- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 msgr2=0x7fd74c1306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7511e2700 1 --2- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c1306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7511e2700 1 -- 172.21.15.32:0/1845859767 >> v1:172.21.15.32:6789/0 conn(0x7fd74c133e30 legacy=0x7fd74c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7511e2700 1 -- 172.21.15.32:0/1845859767 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c0a0030 con 0x7fd74c133420 2023-09-30T22:28:49.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.181+0000 7fd7511e2700 1 --2- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133420 0x7fd74c1337f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd748009840 tx=0x7fd748009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c203b749fc07a83 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd743fff700 1 -- 172.21.15.32:0/1845859767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748011830 con 0x7fd74c133420 2023-09-30T22:28:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd743fff700 1 -- 172.21.15.32:0/1845859767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748011990 con 0x7fd74c133420 2023-09-30T22:28:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd743fff700 1 -- 172.21.15.32:0/1845859767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748011d20 con 0x7fd74c133420 2023-09-30T22:28:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd753446700 1 -- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133420 msgr2=0x7fd74c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd753446700 1 --2- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133420 0x7fd74c1337f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd748009840 tx=0x7fd748009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:49.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd753446700 1 -- 172.21.15.32:0/1845859767 shutdown_connections 2023-09-30T22:28:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd753446700 1 --2- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c1306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd753446700 1 --2- 172.21.15.32:0/1845859767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133420 0x7fd74c1337f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.182+0000 7fd753446700 1 -- 172.21.15.32:0/1845859767 >> 172.21.15.32:0/1845859767 conn(0x7fd74c0a4210 msgr2=0x7fd74c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.183+0000 7fd753446700 1 -- 172.21.15.32:0/1845859767 shutdown_connections 2023-09-30T22:28:49.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.183+0000 7fd753446700 1 -- 172.21.15.32:0/1845859767 wait complete. 2023-09-30T22:28:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.184+0000 7fd753446700 1 Processor -- start 2023-09-30T22:28:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.184+0000 7fd753446700 1 -- start start 2023-09-30T22:28:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd753446700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd74c133420 0x7fd74c11e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd753446700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133e30 0x7fd74c11c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd753446700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c11cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd753446700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c11b8b0 con 0x7fd74c133e30 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd753446700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c11b520 con 0x7fd74c134260 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd753446700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c12d7a0 con 0x7fd74c133420 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c11cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c11cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34748/0 (socket says 172.21.15.32:34748) 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 -- 172.21.15.32:0/890842234 learned_addr learned my addr 172.21.15.32:0/890842234 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7511e2700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd74c133420 0x7fd74c11e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 -- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd74c133420 msgr2=0x7fd74c11e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd74c133420 0x7fd74c11e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 -- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133e30 msgr2=0x7fd74c11c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133e30 0x7fd74c11c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7519e3700 1 -- 172.21.15.32:0/890842234 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480094a0 con 0x7fd74c134260 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.185+0000 7fd7511e2700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd74c133420 0x7fd74c11e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.186+0000 7fd7519e3700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c11cf00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd74400efc0 tx=0x7fd74400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.187+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744003c20 con 0x7fd74c134260 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.187+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74400f040 con 0x7fd74c134260 2023-09-30T22:28:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.187+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744009e30 con 0x7fd74c134260 2023-09-30T22:28:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.188+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c09e310 con 0x7fd74c134260 2023-09-30T22:28:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.188+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c12db30 con 0x7fd74c134260 2023-09-30T22:28:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.189+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd74400c930 con 0x7fd74c134260 2023-09-30T22:28:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.189+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd74403f970 con 0x7fd74c134260 2023-09-30T22:28:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.189+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c11bd40 con 0x7fd74c134260 2023-09-30T22:28:49.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.199+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd744013db0 con 0x7fd74c134260 2023-09-30T22:28:49.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.349+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6614130f60 con 0x7f661406b080 2023-09-30T22:28:49.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.349+0000 7f660a7fc700 1 -- 172.21.15.32:0/4078450396 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6604017020 con 0x7f661406b080 2023-09-30T22:28:49.349 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:49.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 msgr2=0x7f66141c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 0x7f66141c4880 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f660400c550 tx=0x7f660400c860 comp rx=0 tx=0).stop 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 shutdown_connections 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6614131d10 0x7f661411e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f661406a770 0x7f66141c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 --2- 172.21.15.32:0/4078450396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f661406b080 0x7f66141c4880 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 >> 172.21.15.32:0/4078450396 conn(0x7f6614066f50 msgr2=0x7f66140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 shutdown_connections 2023-09-30T22:28:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.351+0000 7f661bce1700 1 -- 172.21.15.32:0/4078450396 wait complete. 2023-09-30T22:28:49.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.363+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f57cc09f420 con 0x7f57cc09d6f0 2023-09-30T22:28:49.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.364+0000 7f57baffd700 1 -- 172.21.15.32:0/814908175 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57bc017030 con 0x7f57cc09d6f0 2023-09-30T22:28:49.364 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:49.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 msgr2=0x7f57cc1c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 0x7f57cc1c7fd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00ec10 tx=0x7f57bc00ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 shutdown_connections 2023-09-30T22:28:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57cc09e440 0x7f57cc1c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57cc09d6f0 0x7f57cc1c7fd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 --2- 172.21.15.32:0/814908175 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57cc09e000 0x7f57cc1c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 >> 172.21.15.32:0/814908175 conn(0x7f57cc066f50 msgr2=0x7f57cc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 shutdown_connections 2023-09-30T22:28:49.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.366+0000 7f57d0886700 1 -- 172.21.15.32:0/814908175 wait complete. 2023-09-30T22:28:49.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.422+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd74c074ea0 con 0x7fd74c134260 2023-09-30T22:28:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.422+0000 7fd741ffb700 1 -- 172.21.15.32:0/890842234 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd74402c090 con 0x7fd74c134260 2023-09-30T22:28:49.422 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 msgr2=0x7fd74c11cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c11cf00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd74400efc0 tx=0x7fd74400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 shutdown_connections 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd74c134260 0x7fd74c11cf00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd74c133420 0x7fd74c11e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 --2- 172.21.15.32:0/890842234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd74c133e30 0x7fd74c11c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 >> 172.21.15.32:0/890842234 conn(0x7fd74c0a4210 msgr2=0x7fd74c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 shutdown_connections 2023-09-30T22:28:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.425+0000 7fd753446700 1 -- 172.21.15.32:0/890842234 wait complete. 2023-09-30T22:28:49.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:49.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 Processor -- start 2023-09-30T22:28:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 -- start start 2023-09-30T22:28:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd00a0a30 0x7f8bd00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bd009d630 0x7f8bd00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 0x7f8bd0133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd00a2a70 con 0x7f8bd009d630 2023-09-30T22:28:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00a01d0 con 0x7f8bd0133930 2023-09-30T22:28:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.822+0000 7f8bd53a5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd009e2b0 con 0x7f8bd00a0a30 2023-09-30T22:28:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bd009d630 0x7f8bd00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bcf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 0x7f8bd0133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bceffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd00a0a30 0x7f8bd00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bd009d630 0x7f8bd00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53174/0 (socket says 172.21.15.32:53174) 2023-09-30T22:28:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bcf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 0x7f8bd0133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:34762/0 (socket says 172.21.15.32:34762) 2023-09-30T22:28:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3919191270 learned_addr learned my addr 172.21.15.32:0/3919191270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd00a0a30 msgr2=0x7f8bd00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:49.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd00a0a30 0x7f8bd00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 msgr2=0x7f8bd0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 0x7f8bd0133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3919191270 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd009dc50 con 0x7f8bd009d630 2023-09-30T22:28:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bceffd700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd00a0a30 0x7f8bd00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bcf7fe700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 0x7f8bd0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:49.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bce7fc700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd009d630 0x7f8bd00a1340 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40098f0 tx=0x7f8bc4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78e68862eaa77fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bcdffb700 1 -- 172.21.15.32:0/3919191270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4011830 con 0x7f8bd009d630 2023-09-30T22:28:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bcdffb700 1 -- 172.21.15.32:0/3919191270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4011990 con 0x7f8bd009d630 2023-09-30T22:28:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.823+0000 7f8bcdffb700 1 -- 172.21.15.32:0/3919191270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4011d20 con 0x7f8bd009d630 2023-09-30T22:28:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 -- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd009d630 msgr2=0x7f8bd00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd009d630 0x7f8bd00a1340 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40098f0 tx=0x7f8bc4009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 -- 172.21.15.32:0/3919191270 shutdown_connections 2023-09-30T22:28:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd0133930 0x7f8bd0133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd00a0a30 0x7f8bd00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/3919191270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd009d630 0x7f8bd00a1340 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 -- 172.21.15.32:0/3919191270 >> 172.21.15.32:0/3919191270 conn(0x7f8bd00a3050 msgr2=0x7f8bd00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:49.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 -- 172.21.15.32:0/3919191270 shutdown_connections 2023-09-30T22:28:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.824+0000 7f8bd53a5700 1 -- 172.21.15.32:0/3919191270 wait complete. 2023-09-30T22:28:49.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.825+0000 7f8bd53a5700 1 Processor -- start 2023-09-30T22:28:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.825+0000 7f8bd53a5700 1 -- start start 2023-09-30T22:28:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bd53a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd009d630 0x7f8bd011f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bd53a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 0x7f8bd011fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bd53a5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd0133930 0x7f8bd01c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bd53a5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd01222e0 con 0x7f8bd00a0a30 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bd53a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0122800 con 0x7f8bd009d630 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bd53a5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0133430 con 0x7f8bd0133930 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 0x7f8bd011fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 0x7f8bd011fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53182/0 (socket says 172.21.15.32:53182) 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bcf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd0133930 0x7f8bd01c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bce7fc700 1 -- 172.21.15.32:0/622534410 learned_addr learned my addr 172.21.15.32:0/622534410 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bceffd700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd009d630 0x7f8bd011f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bcf7fe700 1 -- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd009d630 msgr2=0x7f8bd011f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bcf7fe700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd009d630 0x7f8bd011f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bcf7fe700 1 -- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 msgr2=0x7f8bd011fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bcf7fe700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 0x7f8bd011fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bcf7fe700 1 -- 172.21.15.32:0/622534410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00094a0 con 0x7f8bd0133930 2023-09-30T22:28:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.826+0000 7f8bce7fc700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 0x7f8bd011fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.827+0000 7f8bcf7fe700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd0133930 0x7f8bd01c7ff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009e50 tx=0x7f8bc800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.827+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800ca90 con 0x7f8bd0133930 2023-09-30T22:28:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.827+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc800f040 con 0x7f8bd0133930 2023-09-30T22:28:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.827+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc8009a70 con 0x7f8bd0133930 2023-09-30T22:28:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.827+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800ce50 con 0x7f8bd0133930 2023-09-30T22:28:49.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.827+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc4009550 con 0x7f8bd0133930 2023-09-30T22:28:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.828+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bc8014480 con 0x7f8bd0133930 2023-09-30T22:28:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.828+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd0121f60 con 0x7f8bd0133930 2023-09-30T22:28:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.828+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8bc803fbd0 con 0x7f8bd0133930 2023-09-30T22:28:49.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:49.833+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bd0121f60 con 0x7f8bd0133930 2023-09-30T22:28:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.034+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8bd0121f60 con 0x7f8bd0133930 2023-09-30T22:28:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.035+0000 7f8bb77fe700 1 -- 172.21.15.32:0/622534410 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bd0121f60 con 0x7f8bd0133930 2023-09-30T22:28:50.035 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd0133930 msgr2=0x7f8bd01c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd0133930 0x7f8bd01c7ff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009e50 tx=0x7f8bc800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 shutdown_connections 2023-09-30T22:28:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bd0133930 0x7f8bd01c7ff0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bd009d630 0x7f8bd011f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 --2- 172.21.15.32:0/622534410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd00a0a30 0x7f8bd011fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 >> 172.21.15.32:0/622534410 conn(0x7f8bd00a3050 msgr2=0x7f8bd01bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 shutdown_connections 2023-09-30T22:28:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.037+0000 7f8bd53a5700 1 -- 172.21.15.32:0/622534410 wait complete. 2023-09-30T22:28:50.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:50.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:50.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:50.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:50.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.516+0000 7f396619a700 1 Processor -- start 2023-09-30T22:28:50.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.516+0000 7f396619a700 1 -- start start 2023-09-30T22:28:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f396619a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f396009d6f0 0x7f396009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f396619a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 0x7f3960130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f396619a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f3960130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f396619a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396009f520 con 0x7f396009d6f0 2023-09-30T22:28:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f396619a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396009f130 con 0x7f396009e000 2023-09-30T22:28:50.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f396619a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f396009e930 con 0x7f396009e410 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 0x7f3960130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 0x7f3960130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38010/0 (socket says 172.21.15.32:38010) 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 -- 172.21.15.32:0/288106125 learned_addr learned my addr 172.21.15.32:0/288106125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395f7fe700 1 --2- 172.21.15.32:0/288106125 >> v2:172.21.15.32:3300/0 conn(0x7f396009d6f0 0x7f396009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395ffff700 1 --2- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f3960130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 -- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 msgr2=0x7f3960130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:50.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 --2- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f3960130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 -- 172.21.15.32:0/288106125 >> v2:172.21.15.32:3300/0 conn(0x7f396009d6f0 msgr2=0x7f396009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 --2- 172.21.15.32:0/288106125 >> v2:172.21.15.32:3300/0 conn(0x7f396009d6f0 0x7f396009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395effd700 1 -- 172.21.15.32:0/288106125 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396009b2d0 con 0x7f396009e000 2023-09-30T22:28:50.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.517+0000 7f395f7fe700 1 --2- 172.21.15.32:0/288106125 >> v2:172.21.15.32:3300/0 conn(0x7f396009d6f0 0x7f396009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.518+0000 7f395effd700 1 --2- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 0x7f3960130660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f39540098f0 tx=0x7f3954009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fa74bbd421f45fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.518+0000 7f395e7fc700 1 -- 172.21.15.32:0/288106125 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954011830 con 0x7f396009e000 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.518+0000 7f395e7fc700 1 -- 172.21.15.32:0/288106125 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3954011990 con 0x7f396009e000 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.518+0000 7f395e7fc700 1 -- 172.21.15.32:0/288106125 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954011d20 con 0x7f396009e000 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 -- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 msgr2=0x7f3960130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 --2- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 0x7f3960130660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f39540098f0 tx=0x7f3954009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 -- 172.21.15.32:0/288106125 shutdown_connections 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 --2- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f3960130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 --2- 172.21.15.32:0/288106125 >> v2:172.21.15.32:3300/0 conn(0x7f396009d6f0 0x7f396009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 --2- 172.21.15.32:0/288106125 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009e000 0x7f3960130660 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 -- 172.21.15.32:0/288106125 >> 172.21.15.32:0/288106125 conn(0x7f3960066f50 msgr2=0x7f396009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 -- 172.21.15.32:0/288106125 shutdown_connections 2023-09-30T22:28:50.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.519+0000 7f396619a700 1 -- 172.21.15.32:0/288106125 wait complete. 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.520+0000 7f396619a700 1 Processor -- start 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.520+0000 7f396619a700 1 -- start start 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f396619a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 0x7f39601c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f396619a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f396009e000 0x7f39601c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f396619a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f39601c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f396619a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3960136690 con 0x7f396009e000 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f396619a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3960136870 con 0x7f396009d6f0 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f396619a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f396009c700 con 0x7f396009e410 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 0x7f39601c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f39601c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 0x7f39601c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38014/0 (socket says 172.21.15.32:38014) 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395f7fe700 1 -- 172.21.15.32:0/1923666196 learned_addr learned my addr 172.21.15.32:0/1923666196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395effd700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f396009e000 0x7f39601c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395ffff700 1 -- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 msgr2=0x7f39601c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395ffff700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 0x7f39601c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395ffff700 1 -- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f396009e000 msgr2=0x7f39601c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395ffff700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f396009e000 0x7f39601c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395ffff700 1 -- 172.21.15.32:0/1923666196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500094a0 con 0x7f396009e410 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395f7fe700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 0x7f39601c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:50.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.521+0000 7f395effd700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f396009e000 0x7f39601c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.522+0000 7f395ffff700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f39601c6970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f395800efc0 tx=0x7f395800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.522+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800cc90 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.522+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395800f040 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.522+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3954009550 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.522+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580146c0 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.522+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3960135bb0 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.523+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3958014820 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.523+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f39580402c0 con 0x7f396009e410 2023-09-30T22:28:50.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.523+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39601364f0 con 0x7f396009e410 2023-09-30T22:28:50.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.532+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39601364f0 con 0x7f396009e410 2023-09-30T22:28:50.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.557+0000 7f5d8d2aa700 1 Processor -- start 2023-09-30T22:28:50.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.557+0000 7f5d8d2aa700 1 -- start start 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d8d2aa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d8809e850 0x7f5d8809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d8d2aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d88134290 0x7f5d8813be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d8d2aa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8806bc50 con 0x7f5d8809f160 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d8d2aa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8809ae60 con 0x7f5d8809e850 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d8d2aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8806b990 con 0x7f5d88134290 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d86ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d8809e850 0x7f5d8809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d867fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5d8809f160 0x7f5d88133b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54340/0 (socket says 172.21.15.32:54340) 2023-09-30T22:28:50.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d867fc700 1 -- 172.21.15.32:0/649664579 learned_addr learned my addr 172.21.15.32:0/649664579 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d877fe700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d88134290 0x7f5d8813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d86ffd700 1 -- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d88134290 msgr2=0x7f5d8813be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d86ffd700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d88134290 0x7f5d8813be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d86ffd700 1 -- 172.21.15.32:0/649664579 >> v1:172.21.15.32:6789/0 conn(0x7f5d8809f160 legacy=0x7f5d88133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d85ffb700 1 -- 172.21.15.32:0/649664579 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574686910 0 0) 0x7f5d8806bc50 con 0x7f5d8809f160 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d86ffd700 1 -- 172.21.15.32:0/649664579 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8806cb10 con 0x7f5d8809e850 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.558+0000 7f5d877fe700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d88134290 0x7f5d8813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.559+0000 7f5d86ffd700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d8809ec20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009840 tx=0x7f5d78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbc96708f8117dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.559+0000 7f5d85ffb700 1 -- 172.21.15.32:0/649664579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78011830 con 0x7f5d8809e850 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.559+0000 7f5d85ffb700 1 -- 172.21.15.32:0/649664579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d78011990 con 0x7f5d8809e850 2023-09-30T22:28:50.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.559+0000 7f5d85ffb700 1 -- 172.21.15.32:0/649664579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78011d20 con 0x7f5d8809e850 2023-09-30T22:28:50.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 msgr2=0x7f5d8809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d8809ec20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009840 tx=0x7f5d78009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/649664579 shutdown_connections 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d88134290 0x7f5d8813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/649664579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d8809ec20 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/649664579 >> 172.21.15.32:0/649664579 conn(0x7f5d880a3f70 msgr2=0x7f5d880a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/649664579 shutdown_connections 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.560+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/649664579 wait complete. 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.562+0000 7f5d8d2aa700 1 Processor -- start 2023-09-30T22:28:50.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.562+0000 7f5d8d2aa700 1 -- start start 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.562+0000 7f5d8d2aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d88122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.562+0000 7f5d8d2aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 0x7f5d88122e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.562+0000 7f5d8d2aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d88134290 0x7f5d881233a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.562+0000 7f5d8d2aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88126440 con 0x7f5d8809e850 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d8d2aa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d881245d0 con 0x7f5d8809f160 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d8d2aa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8806fb30 con 0x7f5d88134290 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 0x7f5d88122e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d88122920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d867fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 0x7f5d88122e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38034/0 (socket says 172.21.15.32:38034) 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d88122920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43120/0 (socket says 172.21.15.32:43120) 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d867fc700 1 -- 172.21.15.32:0/3722765386 learned_addr learned my addr 172.21.15.32:0/3722765386 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 -- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d88134290 msgr2=0x7f5d881233a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d88134290 0x7f5d881233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 -- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 msgr2=0x7f5d88122e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 0x7f5d88122e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 -- 172.21.15.32:0/3722765386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d80009a70 con 0x7f5d8809e850 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d867fc700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 0x7f5d88122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d86ffd700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d88122920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009840 tx=0x7f5d78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:50.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.563+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7801a7e0 con 0x7f5d8809e850 2023-09-30T22:28:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.564+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7801a960 con 0x7f5d8809e850 2023-09-30T22:28:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.564+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7801ad10 con 0x7f5d8809e850 2023-09-30T22:28:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.564+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780094a0 con 0x7f5d8809e850 2023-09-30T22:28:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.564+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8806cd80 con 0x7f5d8809e850 2023-09-30T22:28:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.565+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88126080 con 0x7f5d8809e850 2023-09-30T22:28:50.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.565+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d78022070 con 0x7f5d8809e850 2023-09-30T22:28:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.565+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5d780407f0 con 0x7f5d8809e850 2023-09-30T22:28:50.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.570+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d7800bd90 con 0x7f5d8809e850 2023-09-30T22:28:50.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.601+0000 7f2cd72ca700 1 Processor -- start 2023-09-30T22:28:50.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.602+0000 7f2cd72ca700 1 -- start start 2023-09-30T22:28:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.602+0000 7f2cd72ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2cd00a0a30 0x7f2cd00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.602+0000 7f2cd72ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.602+0000 7f2cd72ca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd00a2a70 con 0x7f2cd0133930 2023-09-30T22:28:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.602+0000 7f2cd72ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00a01d0 con 0x7f2cd00a0a30 2023-09-30T22:28:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.602+0000 7f2cd72ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd009e2b0 con 0x7f2cd009d630 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd5867700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2cd0133930 0x7f2cd0133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54342/0 (socket says 172.21.15.32:54342) 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd5066700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2cd00a0a30 0x7f2cd00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd5867700 1 -- 172.21.15.32:0/619807618 learned_addr learned my addr 172.21.15.32:0/619807618 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd4865700 1 --2- 172.21.15.32:0/619807618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd4865700 1 -- 172.21.15.32:0/619807618 >> v2:172.21.15.32:3300/0 conn(0x7f2cd00a0a30 msgr2=0x7f2cd00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd4865700 1 --2- 172.21.15.32:0/619807618 >> v2:172.21.15.32:3300/0 conn(0x7f2cd00a0a30 0x7f2cd00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd4865700 1 -- 172.21.15.32:0/619807618 >> v1:172.21.15.32:6789/0 conn(0x7f2cd0133930 legacy=0x7f2cd0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cc7fff700 1 -- 172.21.15.32:0/619807618 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458700185 0 0) 0x7f2cd00a2a70 con 0x7f2cd0133930 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd4865700 1 -- 172.21.15.32:0/619807618 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd009dc50 con 0x7f2cd009d630 2023-09-30T22:28:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cd4865700 1 --2- 172.21.15.32:0/619807618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd00a1340 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009840 tx=0x7f2cc0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6771ff8d3cf46c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:50.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cc7fff700 1 -- 172.21.15.32:0/619807618 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0011830 con 0x7f2cd009d630 2023-09-30T22:28:50.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cc7fff700 1 -- 172.21.15.32:0/619807618 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc0011990 con 0x7f2cd009d630 2023-09-30T22:28:50.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.603+0000 7f2cc7fff700 1 -- 172.21.15.32:0/619807618 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0011d20 con 0x7f2cd009d630 2023-09-30T22:28:50.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 -- 172.21.15.32:0/619807618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 msgr2=0x7f2cd00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/619807618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd00a1340 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009840 tx=0x7f2cc0009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:50.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 -- 172.21.15.32:0/619807618 shutdown_connections 2023-09-30T22:28:50.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/619807618 >> v2:172.21.15.32:3300/0 conn(0x7f2cd00a0a30 0x7f2cd00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/619807618 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd00a1340 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 -- 172.21.15.32:0/619807618 >> 172.21.15.32:0/619807618 conn(0x7f2cd00a3050 msgr2=0x7f2cd00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 -- 172.21.15.32:0/619807618 shutdown_connections 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.604+0000 7f2cd72ca700 1 -- 172.21.15.32:0/619807618 wait complete. 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.605+0000 7f2cd72ca700 1 Processor -- start 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.605+0000 7f2cd72ca700 1 -- start start 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd72ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd011f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd72ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 0x7f2cd011f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd72ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd0133930 0x7f2cd01c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd72ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0121ea0 con 0x7f2cd0133930 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd72ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd011eed0 con 0x7f2cd009d630 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd72ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0121210 con 0x7f2cd00a0a30 2023-09-30T22:28:50.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 0x7f2cd011f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 0x7f2cd011f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50450/0 (socket says 172.21.15.32:50450) 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 -- 172.21.15.32:0/2929584624 learned_addr learned my addr 172.21.15.32:0/2929584624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd5867700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd0133930 0x7f2cd01c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd5066700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd011f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 -- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 msgr2=0x7f2cd011f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd011f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 -- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd0133930 msgr2=0x7f2cd01c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.606+0000 7f2cd4865700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd0133930 0x7f2cd01c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cd4865700 1 -- 172.21.15.32:0/2929584624 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc00094a0 con 0x7f2cd00a0a30 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cd5066700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd011f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:50.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cd4865700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 0x7f2cd011f940 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009fd0 tx=0x7f2cc0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:50.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004480 con 0x7f2cd00a0a30 2023-09-30T22:28:50.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc001f070 con 0x7f2cd00a0a30 2023-09-30T22:28:50.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd0122160 con 0x7f2cd00a0a30 2023-09-30T22:28:50.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00038e0 con 0x7f2cd00a0a30 2023-09-30T22:28:50.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.607+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd0122440 con 0x7f2cd00a0a30 2023-09-30T22:28:50.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.608+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cc0021030 con 0x7f2cd00a0a30 2023-09-30T22:28:50.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.608+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2cc0040790 con 0x7f2cd00a0a30 2023-09-30T22:28:50.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.608+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd00747f0 con 0x7f2cd00a0a30 2023-09-30T22:28:50.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.613+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cc0015020 con 0x7f2cd00a0a30 2023-09-30T22:28:50.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.739+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39601364f0 con 0x7f396009e410 2023-09-30T22:28:50.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.739+0000 7f3947fff700 1 -- 172.21.15.32:0/1923666196 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39601364f0 con 0x7f396009e410 2023-09-30T22:28:50.739 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:50.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 msgr2=0x7f39601c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f39601c6970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f395800efc0 tx=0x7f395800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 shutdown_connections 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f396009e410 0x7f39601c6970 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f396009d6f0 0x7f39601c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 --2- 172.21.15.32:0/1923666196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f396009e000 0x7f39601c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.742+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 >> 172.21.15.32:0/1923666196 conn(0x7f3960066f50 msgr2=0x7f396006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.743+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 shutdown_connections 2023-09-30T22:28:50.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.743+0000 7f396619a700 1 -- 172.21.15.32:0/1923666196 wait complete. 2023-09-30T22:28:50.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.787+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d880747f0 con 0x7f5d8809e850 2023-09-30T22:28:50.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.788+0000 7f5d6f7fe700 1 -- 172.21.15.32:0/3722765386 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d78029030 con 0x7f5d8809e850 2023-09-30T22:28:50.788 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:50.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 msgr2=0x7f5d88122920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d88122920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009840 tx=0x7f5d78004750 comp rx=0 tx=0).stop 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 shutdown_connections 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5d88134290 0x7f5d881233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d8809e850 0x7f5d88122920 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 --2- 172.21.15.32:0/3722765386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d8809f160 0x7f5d88122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 >> 172.21.15.32:0/3722765386 conn(0x7f5d880a3f70 msgr2=0x7f5d88070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 shutdown_connections 2023-09-30T22:28:50.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.791+0000 7f5d8d2aa700 1 -- 172.21.15.32:0/3722765386 wait complete. 2023-09-30T22:28:50.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.830+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2cd0072980 con 0x7f2cd00a0a30 2023-09-30T22:28:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.831+0000 7f2cc5ffb700 1 -- 172.21.15.32:0/2929584624 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cc00239c0 con 0x7f2cd00a0a30 2023-09-30T22:28:50.831 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 msgr2=0x7f2cd011f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 0x7f2cd011f940 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009fd0 tx=0x7f2cc0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 shutdown_connections 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2cd0133930 0x7f2cd01c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2cd009d630 0x7f2cd011f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 --2- 172.21.15.32:0/2929584624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2cd00a0a30 0x7f2cd011f940 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 >> 172.21.15.32:0/2929584624 conn(0x7f2cd00a3050 msgr2=0x7f2cd01bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 shutdown_connections 2023-09-30T22:28:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:50.833+0000 7f2cd72ca700 1 -- 172.21.15.32:0/2929584624 wait complete. 2023-09-30T22:28:50.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:51.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.200+0000 7f2fe1293700 1 Processor -- start 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.200+0000 7f2fe1293700 1 -- start start 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fe1293700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fe1293700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fdc09d6f0 0x7f2fdc09db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fe1293700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc09c190 con 0x7f2fdc09e080 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fe1293700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc09e520 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fe1293700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc09b090 con 0x7f2fdc130750 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fdc09d6f0 0x7f2fdc09db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2fdc09d6f0 0x7f2fdc09db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43142/0 (socket says 172.21.15.32:43142) 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fdb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2fdc09e080 0x7f2fdc1374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54346/0 (socket says 172.21.15.32:54346) 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 -- 172.21.15.32:0/323391537 learned_addr learned my addr 172.21.15.32:0/323391537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 -- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 msgr2=0x7f2fdc130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 --2- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 -- 172.21.15.32:0/323391537 >> v1:172.21.15.32:6789/0 conn(0x7f2fdc09e080 legacy=0x7f2fdc1374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fda7fc700 1 -- 172.21.15.32:0/323391537 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc09a6c0 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fd9ffb700 1 -- 172.21.15.32:0/323391537 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172427417 0 0) 0x7f2fdc09c190 con 0x7f2fdc09e080 2023-09-30T22:28:51.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.201+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fda7fc700 1 --2- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc09db40 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000b9a0 tx=0x7f2fd000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343eb601314419d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fd9ffb700 1 -- 172.21.15.32:0/323391537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000caa0 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fd9ffb700 1 -- 172.21.15.32:0/323391537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd000cc00 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fd9ffb700 1 -- 172.21.15.32:0/323391537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000f460 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 -- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 msgr2=0x7f2fdc09db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 --2- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc09db40 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000b9a0 tx=0x7f2fd000bcb0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 -- 172.21.15.32:0/323391537 shutdown_connections 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 --2- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 --2- 172.21.15.32:0/323391537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc09db40 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 -- 172.21.15.32:0/323391537 >> 172.21.15.32:0/323391537 conn(0x7f2fdc09f470 msgr2=0x7f2fdc0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.202+0000 7f2fe1293700 1 -- 172.21.15.32:0/323391537 shutdown_connections 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.203+0000 7f2fe1293700 1 -- 172.21.15.32:0/323391537 wait complete. 2023-09-30T22:28:51.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 Processor -- start 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 -- start start 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc1c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fdc09e080 0x7f2fdc1c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc12be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc12d850 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc12f560 con 0x7f2fdc130750 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fe1293700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc098920 con 0x7f2fdc09e080 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc1c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc1c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43146/0 (socket says 172.21.15.32:43146) 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fdaffd700 1 -- 172.21.15.32:0/1620118976 learned_addr learned my addr 172.21.15.32:0/1620118976 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:51.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fda7fc700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fdc09e080 0x7f2fdc1c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.204+0000 7f2fdb7fe700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc12be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fdaffd700 1 -- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fdc09e080 msgr2=0x7f2fdc1c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fdc09e080 0x7f2fdc1c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fdaffd700 1 -- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 msgr2=0x7f2fdc12be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc12be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fdaffd700 1 -- 172.21.15.32:0/1620118976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc00d940 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fdaffd700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc1c44e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc012fc0 tx=0x7f2fcc0103a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc010b00 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc013040 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd000b600 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc010e80 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.205+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc1310e0 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.206+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2fcc018480 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.206+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2fcc043690 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.206+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc12fcd0 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.211+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2fcc016930 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.414+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2fdc09b2b0 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.414+0000 7f2fc37fe700 1 -- 172.21.15.32:0/1620118976 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fcc00dca0 con 0x7f2fdc09d6f0 2023-09-30T22:28:51.414 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.416+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 msgr2=0x7f2fdc1c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.416+0000 7f2fe1293700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc1c44e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc012fc0 tx=0x7f2fcc0103a0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.416+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 shutdown_connections 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.416+0000 7f2fe1293700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2fdc130750 0x7f2fdc12be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.417+0000 7f2fe1293700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc09d6f0 0x7f2fdc1c44e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.417+0000 7f2fe1293700 1 --2- 172.21.15.32:0/1620118976 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2fdc09e080 0x7f2fdc1c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.417+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 >> 172.21.15.32:0/1620118976 conn(0x7f2fdc09f470 msgr2=0x7f2fdc0992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.417+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 shutdown_connections 2023-09-30T22:28:51.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.417+0000 7f2fe1293700 1 -- 172.21.15.32:0/1620118976 wait complete. 2023-09-30T22:28:51.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:51.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:51.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:51.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.920+0000 7f4bf561a700 1 Processor -- start 2023-09-30T22:28:51.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.920+0000 7f4bf561a700 1 -- start start 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bf561a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bf561a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bf561a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf009ece0 con 0x7f4bf006b080 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bf561a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf009e760 con 0x7f4bf0131d10 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bf561a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf009e040 con 0x7f4bf006a770 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bef7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38080/0 (socket says 172.21.15.32:38080) 2023-09-30T22:28:51.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4beeffd700 1 -- 172.21.15.32:0/545275220 learned_addr learned my addr 172.21.15.32:0/545275220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4bee7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4bf006b080 0x7f4bf0131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54354/0 (socket says 172.21.15.32:54354) 2023-09-30T22:28:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4beeffd700 1 -- 172.21.15.32:0/545275220 >> v2:172.21.15.32:3300/0 conn(0x7f4bf0131d10 msgr2=0x7f4bf0139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4beeffd700 1 --2- 172.21.15.32:0/545275220 >> v2:172.21.15.32:3300/0 conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.921+0000 7f4beeffd700 1 -- 172.21.15.32:0/545275220 >> v1:172.21.15.32:6789/0 conn(0x7f4bf006b080 legacy=0x7f4bf0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.922+0000 7f4beeffd700 1 -- 172.21.15.32:0/545275220 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf009f630 con 0x7f4bf006a770 2023-09-30T22:28:51.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.922+0000 7f4beeffd700 1 --2- 172.21.15.32:0/545275220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4be00098f0 tx=0x7f4be0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4a873560bb9b312 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.922+0000 7f4bedffb700 1 -- 172.21.15.32:0/545275220 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0011830 con 0x7f4bf006a770 2023-09-30T22:28:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.922+0000 7f4bedffb700 1 -- 172.21.15.32:0/545275220 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0011990 con 0x7f4bf006a770 2023-09-30T22:28:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.922+0000 7f4bedffb700 1 -- 172.21.15.32:0/545275220 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0011d20 con 0x7f4bf006a770 2023-09-30T22:28:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 -- 172.21.15.32:0/545275220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 msgr2=0x7f4bf006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 --2- 172.21.15.32:0/545275220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4be00098f0 tx=0x7f4be0009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 -- 172.21.15.32:0/545275220 shutdown_connections 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 --2- 172.21.15.32:0/545275220 >> v2:172.21.15.32:3300/0 conn(0x7f4bf0131d10 0x7f4bf0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 --2- 172.21.15.32:0/545275220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf006a770 0x7f4bf006ab40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 -- 172.21.15.32:0/545275220 >> 172.21.15.32:0/545275220 conn(0x7f4bf0066f50 msgr2=0x7f4bf009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 -- 172.21.15.32:0/545275220 shutdown_connections 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.923+0000 7f4bf561a700 1 -- 172.21.15.32:0/545275220 wait complete. 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.924+0000 7f4bf561a700 1 Processor -- start 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.924+0000 7f4bf561a700 1 -- start start 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bf561a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf006a770 0x7f4bf011d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bf561a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf006b080 0x7f4bf011ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bf561a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 0x7f4bf011e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bf561a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf011edf0 con 0x7f4bf006b080 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bf561a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01bccb0 con 0x7f4bf0131d10 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bf561a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0071e60 con 0x7f4bf006a770 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 0x7f4bf011e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bee7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf006b080 0x7f4bf011ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf006a770 0x7f4bf011d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 0x7f4bf011e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38088/0 (socket says 172.21.15.32:38088) 2023-09-30T22:28:51.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bef7fe700 1 -- 172.21.15.32:0/3301675162 learned_addr learned my addr 172.21.15.32:0/3301675162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4beeffd700 1 -- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 msgr2=0x7f4bf011e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4beeffd700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 0x7f4bf011e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4beeffd700 1 -- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf006b080 msgr2=0x7f4bf011ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4beeffd700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf006b080 0x7f4bf011ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4beeffd700 1 -- 172.21.15.32:0/3301675162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0009550 con 0x7f4bf006a770 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bef7fe700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 0x7f4bf011e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.925+0000 7f4bee7fc700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf006b080 0x7f4bf011ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:51.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.926+0000 7f4beeffd700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf006a770 0x7f4bf011d890 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4be0016000 tx=0x7f4be0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:51.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.926+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be00036a0 con 0x7f4bf006a770 2023-09-30T22:28:51.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.926+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0003800 con 0x7f4bf006a770 2023-09-30T22:28:51.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.926+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf0131430 con 0x7f4bf006a770 2023-09-30T22:28:51.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.926+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0003b90 con 0x7f4bf006a770 2023-09-30T22:28:51.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.926+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf01204a0 con 0x7f4bf006a770 2023-09-30T22:28:51.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.927+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4be0021070 con 0x7f4bf006a770 2023-09-30T22:28:51.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.927+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4be0040ab0 con 0x7f4bf006a770 2023-09-30T22:28:51.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.927+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0074ea0 con 0x7f4bf006a770 2023-09-30T22:28:51.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.932+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4be0015080 con 0x7f4bf006a770 2023-09-30T22:28:51.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 Processor -- start 2023-09-30T22:28:51.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 -- start start 2023-09-30T22:28:51.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fef0809e420 0x7fef081388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 0x7fef0813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0809cdc0 con 0x7fef0809d1f0 2023-09-30T22:28:51.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef08099e50 con 0x7fef0809e420 2023-09-30T22:28:51.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef0d3ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0809cff0 con 0x7fef0809dcd0 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.964+0000 7fef077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 0x7fef0813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 0x7fef0813adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38092/0 (socket says 172.21.15.32:38092) 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 -- 172.21.15.32:0/2633214343 learned_addr learned my addr 172.21.15.32:0/2633214343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef067fc700 1 --2- 172.21.15.32:0/2633214343 >> v2:172.21.15.32:3300/0 conn(0x7fef0809e420 0x7fef081388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef05ffb700 1 -- 172.21.15.32:0/2633214343 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069650316 0 0) 0x7fef0809cdc0 con 0x7fef0809d1f0 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 -- 172.21.15.32:0/2633214343 >> v2:172.21.15.32:3300/0 conn(0x7fef0809e420 msgr2=0x7fef081388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 --2- 172.21.15.32:0/2633214343 >> v2:172.21.15.32:3300/0 conn(0x7fef0809e420 0x7fef081388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 -- 172.21.15.32:0/2633214343 >> v1:172.21.15.32:6789/0 conn(0x7fef0809d1f0 legacy=0x7fef0809d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 -- 172.21.15.32:0/2633214343 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0809e250 con 0x7fef0809dcd0 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef067fc700 1 --2- 172.21.15.32:0/2633214343 >> v2:172.21.15.32:3300/0 conn(0x7fef0809e420 0x7fef081388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef077fe700 1 --2- 172.21.15.32:0/2633214343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 0x7fef0813adf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fef00009e50 tx=0x7fef0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71e88a2cec2555 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:51.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef05ffb700 1 -- 172.21.15.32:0/2633214343 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000e070 con 0x7fef0809dcd0 2023-09-30T22:28:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef05ffb700 1 -- 172.21.15.32:0/2633214343 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0000b9c0 con 0x7fef0809dcd0 2023-09-30T22:28:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.965+0000 7fef05ffb700 1 -- 172.21.15.32:0/2633214343 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000bd70 con 0x7fef0809dcd0 2023-09-30T22:28:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.966+0000 7fef0d3ab700 1 -- 172.21.15.32:0/2633214343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 msgr2=0x7fef0813adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.966+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/2633214343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 0x7fef0813adf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fef00009e50 tx=0x7fef0000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:51.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.966+0000 7fef0d3ab700 1 -- 172.21.15.32:0/2633214343 shutdown_connections 2023-09-30T22:28:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.966+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/2633214343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809dcd0 0x7fef0813adf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.966+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/2633214343 >> v2:172.21.15.32:3300/0 conn(0x7fef0809e420 0x7fef081388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.966+0000 7fef0d3ab700 1 -- 172.21.15.32:0/2633214343 >> 172.21.15.32:0/2633214343 conn(0x7fef080a2360 msgr2=0x7fef080a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.967+0000 7fef0d3ab700 1 -- 172.21.15.32:0/2633214343 shutdown_connections 2023-09-30T22:28:51.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.967+0000 7fef0d3ab700 1 -- 172.21.15.32:0/2633214343 wait complete. 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.968+0000 7fef0d3ab700 1 Processor -- start 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.968+0000 7fef0d3ab700 1 -- start start 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef0d3ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 0x7fef0812a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef0d3ab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef0809dcd0 0x7fef0812a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef0d3ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef0809e420 0x7fef0812ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef0d3ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0812be20 con 0x7fef0809e420 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef0d3ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0809c160 con 0x7fef0809d1f0 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef0d3ab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef0812c3e0 con 0x7fef0809dcd0 2023-09-30T22:28:51.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef06ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 0x7fef0812a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef067fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef0809dcd0 0x7fef0812a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef06ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 0x7fef0812a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38106/0 (socket says 172.21.15.32:38106) 2023-09-30T22:28:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef06ffd700 1 -- 172.21.15.32:0/3735417232 learned_addr learned my addr 172.21.15.32:0/3735417232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:51.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef077fe700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef0809e420 0x7fef0812ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.969+0000 7fef077fe700 1 -- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef0809dcd0 msgr2=0x7fef0812a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef077fe700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef0809dcd0 0x7fef0812a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef077fe700 1 -- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 msgr2=0x7fef0812a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef077fe700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 0x7fef0812a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:51.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef077fe700 1 -- 172.21.15.32:0/3735417232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00009b30 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef06ffd700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 0x7fef0812a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef077fe700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef0809e420 0x7fef0812ae60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fef08074360 tx=0x7fef00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00003ea0 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef00004690 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef080a1660 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef000092e0 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.970+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0812db70 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.971+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef0001c440 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.971+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fef00040dd0 con 0x7fef0809e420 2023-09-30T22:28:51.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.971+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0812d760 con 0x7fef0809e420 2023-09-30T22:28:51.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:51.976+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef0812d760 con 0x7fef0809e420 2023-09-30T22:28:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.026+0000 7fbaa770e700 1 Processor -- start 2023-09-30T22:28:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.026+0000 7fbaa770e700 1 -- start start 2023-09-30T22:28:52.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa770e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 0x7fbaa009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa770e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbaa0134300 0x7fbaa013bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa770e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa009f960 con 0x7fbaa009f160 2023-09-30T22:28:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa770e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa009ae60 con 0x7fbaa0134300 2023-09-30T22:28:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa770e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa006cb10 con 0x7fbaa009e850 2023-09-30T22:28:52.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 0x7fbaa009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa5cab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbaa0134300 0x7fbaa013bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 0x7fbaa009ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50492/0 (socket says 172.21.15.32:50492) 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 -- 172.21.15.32:0/2513573953 learned_addr learned my addr 172.21.15.32:0/2513573953 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 -- 172.21.15.32:0/2513573953 >> v2:172.21.15.32:3300/0 conn(0x7fbaa0134300 msgr2=0x7fbaa013bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 --2- 172.21.15.32:0/2513573953 >> v2:172.21.15.32:3300/0 conn(0x7fbaa0134300 0x7fbaa013bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 -- 172.21.15.32:0/2513573953 >> v1:172.21.15.32:6789/0 conn(0x7fbaa009f160 legacy=0x7fbaa0133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.027+0000 7fbaa54aa700 1 -- 172.21.15.32:0/2513573953 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa006b990 con 0x7fbaa009e850 2023-09-30T22:28:52.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.028+0000 7fbaa54aa700 1 --2- 172.21.15.32:0/2513573953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 0x7fbaa009ec20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0098f0 tx=0x7fba9c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb074d754cc25180 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.028+0000 7fba97fff700 1 -- 172.21.15.32:0/2513573953 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c011830 con 0x7fbaa009e850 2023-09-30T22:28:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.028+0000 7fba97fff700 1 -- 172.21.15.32:0/2513573953 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c011990 con 0x7fbaa009e850 2023-09-30T22:28:52.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.028+0000 7fba97fff700 1 -- 172.21.15.32:0/2513573953 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c011d20 con 0x7fbaa009e850 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.028+0000 7fbaa770e700 1 -- 172.21.15.32:0/2513573953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 msgr2=0x7fbaa009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.028+0000 7fbaa770e700 1 --2- 172.21.15.32:0/2513573953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 0x7fbaa009ec20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0098f0 tx=0x7fba9c009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.029+0000 7fbaa770e700 1 -- 172.21.15.32:0/2513573953 shutdown_connections 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.029+0000 7fbaa770e700 1 --2- 172.21.15.32:0/2513573953 >> v2:172.21.15.32:3300/0 conn(0x7fbaa0134300 0x7fbaa013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.029+0000 7fbaa770e700 1 --2- 172.21.15.32:0/2513573953 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa009e850 0x7fbaa009ec20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.029+0000 7fbaa770e700 1 -- 172.21.15.32:0/2513573953 >> 172.21.15.32:0/2513573953 conn(0x7fbaa00a3f70 msgr2=0x7fbaa00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.029+0000 7fbaa770e700 1 -- 172.21.15.32:0/2513573953 shutdown_connections 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.029+0000 7fbaa770e700 1 -- 172.21.15.32:0/2513573953 wait complete. 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 Processor -- start 2023-09-30T22:28:52.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 -- start start 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 0x7fbaa0122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa009f160 0x7fbaa01230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa0134300 0x7fbaa0123630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0124c90 con 0x7fbaa009f160 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa0072550 con 0x7fbaa009e850 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa770e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa006cd30 con 0x7fbaa0134300 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa54aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 0x7fbaa0122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa4ca9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa009f160 0x7fbaa01230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa54aa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 0x7fbaa0122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38122/0 (socket says 172.21.15.32:38122) 2023-09-30T22:28:52.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa54aa700 1 -- 172.21.15.32:0/84039186 learned_addr learned my addr 172.21.15.32:0/84039186 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.030+0000 7fbaa5cab700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa0134300 0x7fbaa0123630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa54aa700 1 -- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa0134300 msgr2=0x7fbaa0123630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa54aa700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa0134300 0x7fbaa0123630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa54aa700 1 -- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa009f160 msgr2=0x7fbaa01230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa54aa700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa009f160 0x7fbaa01230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa54aa700 1 -- 172.21.15.32:0/84039186 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c009550 con 0x7fbaa009e850 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa54aa700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 0x7fbaa0122bb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009c40 tx=0x7fba9c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0036a0 con 0x7fbaa009e850 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c003800 con 0x7fbaa009e850 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c003bb0 con 0x7fbaa009e850 2023-09-30T22:28:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.031+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa0126020 con 0x7fbaa009e850 2023-09-30T22:28:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.032+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa0126d50 con 0x7fbaa009e850 2023-09-30T22:28:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.032+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba9c021070 con 0x7fbaa009e850 2023-09-30T22:28:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.033+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba9c040c90 con 0x7fbaa009e850 2023-09-30T22:28:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.033+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa00747f0 con 0x7fbaa009e850 2023-09-30T22:28:52.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.037+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba9c029070 con 0x7fbaa009e850 2023-09-30T22:28:52.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.147+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4bf006c360 con 0x7f4bf006a770 2023-09-30T22:28:52.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.148+0000 7f4bd77fe700 1 -- 172.21.15.32:0/3301675162 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4be0029030 con 0x7f4bf006a770 2023-09-30T22:28:52.148 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:52.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf006a770 msgr2=0x7f4bf011d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf006a770 0x7f4bf011d890 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4be0016000 tx=0x7f4be0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 shutdown_connections 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4bf0131d10 0x7f4bf011e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4bf006a770 0x7f4bf011d890 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 --2- 172.21.15.32:0/3301675162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf006b080 0x7f4bf011ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 >> 172.21.15.32:0/3301675162 conn(0x7f4bf0066f50 msgr2=0x7f4bf00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:52.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 shutdown_connections 2023-09-30T22:28:52.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.150+0000 7f4bf561a700 1 -- 172.21.15.32:0/3301675162 wait complete. 2023-09-30T22:28:52.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:52.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.200+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef0812d760 con 0x7fef0809e420 2023-09-30T22:28:52.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.200+0000 7feeef7fe700 1 -- 172.21.15.32:0/3735417232 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef0812d760 con 0x7fef0809e420 2023-09-30T22:28:52.201 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef0809e420 msgr2=0x7fef0812ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef0809e420 0x7fef0812ae60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fef08074360 tx=0x7fef00004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 shutdown_connections 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fef0809e420 0x7fef0812ae60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fef0809d1f0 0x7fef0812a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 --2- 172.21.15.32:0/3735417232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fef0809dcd0 0x7fef0812a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 >> 172.21.15.32:0/3735417232 conn(0x7fef080a2360 msgr2=0x7fef080a0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 shutdown_connections 2023-09-30T22:28:52.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.203+0000 7fef0d3ab700 1 -- 172.21.15.32:0/3735417232 wait complete. 2023-09-30T22:28:52.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:52.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.248+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbaa006bbb0 con 0x7fbaa009e850 2023-09-30T22:28:52.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.248+0000 7fba95ffb700 1 -- 172.21.15.32:0/84039186 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba9c015030 con 0x7fbaa009e850 2023-09-30T22:28:52.249 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:52.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 msgr2=0x7fbaa0122bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 0x7fbaa0122bb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009c40 tx=0x7fba9c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:52.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 shutdown_connections 2023-09-30T22:28:52.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbaa0134300 0x7fbaa0123630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbaa009e850 0x7fbaa0122bb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 --2- 172.21.15.32:0/84039186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbaa009f160 0x7fbaa01230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 >> 172.21.15.32:0/84039186 conn(0x7fbaa00a3f70 msgr2=0x7fbaa00a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 shutdown_connections 2023-09-30T22:28:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.251+0000 7fbaa770e700 1 -- 172.21.15.32:0/84039186 wait complete. 2023-09-30T22:28:52.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:52.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.577+0000 7ff8a9fee700 1 Processor -- start 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.577+0000 7ff8a9fee700 1 -- start start 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a9fee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a9fee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a9fee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a409f130 con 0x7ff8a409e410 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a9fee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a409f520 con 0x7ff8a409e000 2023-09-30T22:28:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a9fee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a409e930 con 0x7ff8a409d6f0 2023-09-30T22:28:52.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8a409e410 0x7ff8a4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54384/0 (socket says 172.21.15.32:54384) 2023-09-30T22:28:52.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43204/0 (socket says 172.21.15.32:43204) 2023-09-30T22:28:52.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a3fff700 1 -- 172.21.15.32:0/2814231396 learned_addr learned my addr 172.21.15.32:0/2814231396 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a37fe700 1 -- 172.21.15.32:0/2814231396 >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 msgr2=0x7ff8a4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a37fe700 1 --2- 172.21.15.32:0/2814231396 >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.578+0000 7ff8a37fe700 1 -- 172.21.15.32:0/2814231396 >> v1:172.21.15.32:6789/0 conn(0x7ff8a409e410 legacy=0x7ff8a4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a37fe700 1 -- 172.21.15.32:0/2814231396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a409b2d0 con 0x7ff8a409d6f0 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a27fc700 1 -- 172.21.15.32:0/2814231396 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824769057 0 0) 0x7ff8a409f130 con 0x7ff8a409e410 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a2ffd700 1 --2- 172.21.15.32:0/2814231396 >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a37fe700 1 --2- 172.21.15.32:0/2814231396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff89400bae0 tx=0x7ff89400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906cbb10e32a259a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a27fc700 1 -- 172.21.15.32:0/2814231396 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400dc70 con 0x7ff8a409d6f0 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a27fc700 1 -- 172.21.15.32:0/2814231396 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89400ddd0 con 0x7ff8a409d6f0 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a27fc700 1 -- 172.21.15.32:0/2814231396 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894010690 con 0x7ff8a409d6f0 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a9fee700 1 -- 172.21.15.32:0/2814231396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409d6f0 msgr2=0x7ff8a409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.579+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/2814231396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff89400bae0 tx=0x7ff89400bdf0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.580+0000 7ff8a9fee700 1 -- 172.21.15.32:0/2814231396 shutdown_connections 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.580+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/2814231396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.580+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/2814231396 >> v2:172.21.15.32:3300/0 conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.580+0000 7ff8a9fee700 1 -- 172.21.15.32:0/2814231396 >> 172.21.15.32:0/2814231396 conn(0x7ff8a4066f50 msgr2=0x7ff8a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.580+0000 7ff8a9fee700 1 -- 172.21.15.32:0/2814231396 shutdown_connections 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.580+0000 7ff8a9fee700 1 -- 172.21.15.32:0/2814231396 wait complete. 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 Processor -- start 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 -- start start 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a409e000 0x7ff8a411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409e410 0x7ff8a411ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4121aa0 con 0x7ff8a409e000 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41201b0 con 0x7ff8a409d6f0 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a9fee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a409c700 con 0x7ff8a409e410 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a409e000 0x7ff8a411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38130/0 (socket says 172.21.15.32:38130) 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a37fe700 1 -- 172.21.15.32:0/1419734881 learned_addr learned my addr 172.21.15.32:0/1419734881 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.581+0000 7ff8a3fff700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409e410 0x7ff8a411ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a2ffd700 1 -- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409e410 msgr2=0x7ff8a411ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a2ffd700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409e410 0x7ff8a411ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a2ffd700 1 -- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 msgr2=0x7ff8a411df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a2ffd700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a2ffd700 1 -- 172.21.15.32:0/1419734881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980094a0 con 0x7ff8a409e000 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a37fe700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a2ffd700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a409e000 0x7ff8a411e4c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e980 tx=0x7ff89800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800c690 con 0x7ff8a409e000 2023-09-30T22:28:52.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89800c7f0 con 0x7ff8a409e000 2023-09-30T22:28:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89400b740 con 0x7ff8a409e000 2023-09-30T22:28:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800cb80 con 0x7ff8a409e000 2023-09-30T22:28:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.582+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a4130360 con 0x7ff8a409e000 2023-09-30T22:28:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.583+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff89800cce0 con 0x7ff8a409e000 2023-09-30T22:28:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.583+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff89803fa70 con 0x7ff8a409e000 2023-09-30T22:28:52.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.583+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a4074ea0 con 0x7ff8a409e000 2023-09-30T22:28:52.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.588+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff89801b3b0 con 0x7ff8a409e000 2023-09-30T22:28:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.794+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff8a4120de0 con 0x7ff8a409e000 2023-09-30T22:28:52.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.795+0000 7ff88bfff700 1 -- 172.21.15.32:0/1419734881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff898017020 con 0x7ff8a409e000 2023-09-30T22:28:52.795 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a409e000 msgr2=0x7ff8a411e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a409e000 0x7ff8a411e4c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e980 tx=0x7ff89800ec90 comp rx=0 tx=0).stop 2023-09-30T22:28:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 shutdown_connections 2023-09-30T22:28:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff8a409e410 0x7ff8a411ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 --2- 172.21.15.32:0/1419734881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a409e000 0x7ff8a411e4c0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 >> 172.21.15.32:0/1419734881 conn(0x7ff8a4066f50 msgr2=0x7ff8a4070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 shutdown_connections 2023-09-30T22:28:52.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:52.797+0000 7ff8a9fee700 1 -- 172.21.15.32:0/1419734881 wait complete. 2023-09-30T22:28:52.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:53.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:53.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:53.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.316+0000 7f20bc73a700 1 Processor -- start 2023-09-30T22:28:53.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.316+0000 7f20bc73a700 1 -- start start 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.316+0000 7f20bc73a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41338c0 0x7f20b4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20bc73a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 0x7f20b409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20bc73a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b406f720 con 0x7f20b409e390 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20bc73a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4070c80 con 0x7f20b41338c0 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20bc73a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b406fce0 con 0x7f20b41341d0 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 0x7f20b409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20ba4d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41338c0 0x7f20b4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 0x7f20b409de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50530/0 (socket says 172.21.15.32:50530) 2023-09-30T22:28:53.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 -- 172.21.15.32:0/2116767255 learned_addr learned my addr 172.21.15.32:0/2116767255 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 -- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41338c0 msgr2=0x7f20b4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 --2- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41338c0 0x7f20b4133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 -- 172.21.15.32:0/2116767255 >> v1:172.21.15.32:6789/0 conn(0x7f20b409e390 legacy=0x7f20b413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b94d4700 1 -- 172.21.15.32:0/2116767255 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19684483 0 0) 0x7f20b406f720 con 0x7f20b409e390 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 -- 172.21.15.32:0/2116767255 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b41336a0 con 0x7f20b41341d0 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20ba4d6700 1 --2- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41338c0 0x7f20b4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.317+0000 7f20b9cd5700 1 --2- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 0x7f20b409de50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009920 tx=0x7f20a4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f6ac13715f272d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20b94d4700 1 -- 172.21.15.32:0/2116767255 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4011830 con 0x7f20b41341d0 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20b94d4700 1 -- 172.21.15.32:0/2116767255 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4011990 con 0x7f20b41341d0 2023-09-30T22:28:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20b94d4700 1 -- 172.21.15.32:0/2116767255 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4011d20 con 0x7f20b41341d0 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20bc73a700 1 -- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 msgr2=0x7f20b409de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20bc73a700 1 --2- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 0x7f20b409de50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009920 tx=0x7f20a4009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20bc73a700 1 -- 172.21.15.32:0/2116767255 shutdown_connections 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20bc73a700 1 --2- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41338c0 0x7f20b4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20bc73a700 1 --2- 172.21.15.32:0/2116767255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41341d0 0x7f20b409de50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.318+0000 7f20bc73a700 1 -- 172.21.15.32:0/2116767255 >> 172.21.15.32:0/2116767255 conn(0x7f20b40a2aa0 msgr2=0x7f20b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.319+0000 7f20bc73a700 1 -- 172.21.15.32:0/2116767255 shutdown_connections 2023-09-30T22:28:53.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.319+0000 7f20bc73a700 1 -- 172.21.15.32:0/2116767255 wait complete. 2023-09-30T22:28:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 Processor -- start 2023-09-30T22:28:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 -- start start 2023-09-30T22:28:53.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b409e390 0x7f20b411fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 0x7f20b411ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41341d0 0x7f20b4120510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4121eb0 con 0x7f20b409e390 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b406f960 con 0x7f20b41341d0 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bc73a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4121720 con 0x7f20b41338c0 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20b9cd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 0x7f20b411ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20b9cd5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 0x7f20b411ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50546/0 (socket says 172.21.15.32:50546) 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20b9cd5700 1 -- 172.21.15.32:0/3085791857 learned_addr learned my addr 172.21.15.32:0/3085791857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.320+0000 7f20bacd7700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41341d0 0x7f20b4120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20ba4d6700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b409e390 0x7f20b411fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20b9cd5700 1 -- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41341d0 msgr2=0x7f20b4120510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20b9cd5700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41341d0 0x7f20b4120510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20b9cd5700 1 -- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b409e390 msgr2=0x7f20b411fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20b9cd5700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b409e390 0x7f20b411fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20b9cd5700 1 -- 172.21.15.32:0/3085791857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00094a0 con 0x7f20b41338c0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20ba4d6700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b409e390 0x7f20b411fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20b9cd5700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 0x7f20b411ffd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f20a40098f0 tx=0x7f20a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004510 con 0x7f20b41338c0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a40036a0 con 0x7f20b41338c0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4003a50 con 0x7f20b41338c0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a4009550 con 0x7f20b41338c0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.321+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b4130520 con 0x7f20b41338c0 2023-09-30T22:28:53.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.322+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20a4003bb0 con 0x7f20b41338c0 2023-09-30T22:28:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.322+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f20a40407e0 con 0x7f20b41338c0 2023-09-30T22:28:53.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.322+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b4074ea0 con 0x7f20b41338c0 2023-09-30T22:28:53.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.327+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20a4015070 con 0x7f20b41338c0 2023-09-30T22:28:53.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.362+0000 7f4a69ada700 1 Processor -- start 2023-09-30T22:28:53.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.362+0000 7f4a69ada700 1 -- start start 2023-09-30T22:28:53.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a69ada700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 0x7f4a64130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a69ada700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a64130e50 0x7f4a6406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a69ada700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a640716a0 con 0x7f4a6406fb20 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a69ada700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6409a390 con 0x7f4a64130e50 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a69ada700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a64071290 con 0x7f4a64130540 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 0x7f4a64130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a62ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a64130e50 0x7f4a6406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 0x7f4a64130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50562/0 (socket says 172.21.15.32:50562) 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.363+0000 7f4a637fe700 1 -- 172.21.15.32:0/701459479 learned_addr learned my addr 172.21.15.32:0/701459479 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a637fe700 1 -- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a64130e50 msgr2=0x7f4a6406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a637fe700 1 --2- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a64130e50 0x7f4a6406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a637fe700 1 -- 172.21.15.32:0/701459479 >> v1:172.21.15.32:6789/0 conn(0x7f4a6406fb20 legacy=0x7f4a64137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a637fe700 1 -- 172.21.15.32:0/701459479 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a64130340 con 0x7f4a64130540 2023-09-30T22:28:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a62ffd700 1 --2- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a64130e50 0x7f4a6406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a637fe700 1 --2- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 0x7f4a64130910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009950 tx=0x7f4a54009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc34276db74cc9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a627fc700 1 -- 172.21.15.32:0/701459479 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011830 con 0x7f4a64130540 2023-09-30T22:28:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a627fc700 1 -- 172.21.15.32:0/701459479 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a54011990 con 0x7f4a64130540 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.364+0000 7f4a627fc700 1 -- 172.21.15.32:0/701459479 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011d40 con 0x7f4a64130540 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 -- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 msgr2=0x7f4a64130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 --2- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 0x7f4a64130910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009950 tx=0x7f4a54009c60 comp rx=0 tx=0).stop 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 -- 172.21.15.32:0/701459479 shutdown_connections 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 --2- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130540 0x7f4a64130910 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 --2- 172.21.15.32:0/701459479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a64130e50 0x7f4a6406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 -- 172.21.15.32:0/701459479 >> 172.21.15.32:0/701459479 conn(0x7f4a640a2aa0 msgr2=0x7f4a640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 -- 172.21.15.32:0/701459479 shutdown_connections 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.365+0000 7f4a69ada700 1 -- 172.21.15.32:0/701459479 wait complete. 2023-09-30T22:28:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.366+0000 7f4a69ada700 1 Processor -- start 2023-09-30T22:28:53.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.366+0000 7f4a69ada700 1 -- start start 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a69ada700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 0x7f4a641c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a69ada700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64130540 0x7f4a641c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a69ada700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 0x7f4a64139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a69ada700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a641315c0 con 0x7f4a64130540 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a69ada700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a64132200 con 0x7f4a6406fb20 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a69ada700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640a0680 con 0x7f4a64130e50 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 0x7f4a641c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 0x7f4a64139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 0x7f4a641c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38162/0 (socket says 172.21.15.32:38162) 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 0x7f4a64139c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50576/0 (socket says 172.21.15.32:50576) 2023-09-30T22:28:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a637fe700 1 -- 172.21.15.32:0/2797044439 learned_addr learned my addr 172.21.15.32:0/2797044439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a62ffd700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64130540 0x7f4a641c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 -- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 msgr2=0x7f4a641c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 0x7f4a641c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 -- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64130540 msgr2=0x7f4a641c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64130540 0x7f4a641c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 -- 172.21.15.32:0/2797044439 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580094a0 con 0x7f4a64130e50 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a637fe700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 0x7f4a641c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a62ffd700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64130540 0x7f4a641c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:53.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.367+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 0x7f4a64139c90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00cb90 tx=0x7f4a5c00cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.368+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c003c00 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.368+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c003d60 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.368+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a54009550 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.368+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00aa80 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.368+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6413b040 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.369+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a5c00abe0 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.369+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a5c03f580 con 0x7f4a64130e50 2023-09-30T22:28:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.369+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a640718e0 con 0x7f4a64130e50 2023-09-30T22:28:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.374+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a5c0198f0 con 0x7f4a64130e50 2023-09-30T22:28:53.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.417+0000 7fd383758700 1 Processor -- start 2023-09-30T22:28:53.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd383758700 1 -- start start 2023-09-30T22:28:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd383758700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09e850 0x7fd37c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd383758700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09f5b0 0x7fd37c134300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd383758700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c06bf30 con 0x7fd37c09f160 2023-09-30T22:28:53.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd383758700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c09ffb0 con 0x7fd37c09f5b0 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd383758700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c06cb10 con 0x7fd37c09e850 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd381cf5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09f5b0 0x7fd37c134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd380cf3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd37c09f160 0x7fd37c133bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54426/0 (socket says 172.21.15.32:54426) 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd380cf3700 1 -- 172.21.15.32:0/1015481574 learned_addr learned my addr 172.21.15.32:0/1015481574 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.418+0000 7fd3814f4700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09e850 0x7fd37c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd373fff700 1 -- 172.21.15.32:0/1015481574 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570641933 0 0) 0x7fd37c06bf30 con 0x7fd37c09f160 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd381cf5700 1 -- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09e850 msgr2=0x7fd37c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd381cf5700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09e850 0x7fd37c09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd381cf5700 1 -- 172.21.15.32:0/1015481574 >> v1:172.21.15.32:6789/0 conn(0x7fd37c09f160 legacy=0x7fd37c133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd381cf5700 1 -- 172.21.15.32:0/1015481574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0a0270 con 0x7fd37c09f5b0 2023-09-30T22:28:53.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd3814f4700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09e850 0x7fd37c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:53.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd381cf5700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09f5b0 0x7fd37c134300 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd374009dc0 tx=0x7fd37400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7a6fc838995e4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd373fff700 1 -- 172.21.15.32:0/1015481574 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400e070 con 0x7fd37c09f5b0 2023-09-30T22:28:53.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd373fff700 1 -- 172.21.15.32:0/1015481574 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400bb20 con 0x7fd37c09f5b0 2023-09-30T22:28:53.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.419+0000 7fd373fff700 1 -- 172.21.15.32:0/1015481574 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374012430 con 0x7fd37c09f5b0 2023-09-30T22:28:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 -- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09f5b0 msgr2=0x7fd37c134300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09f5b0 0x7fd37c134300 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd374009dc0 tx=0x7fd37400b330 comp rx=0 tx=0).stop 2023-09-30T22:28:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 -- 172.21.15.32:0/1015481574 shutdown_connections 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09f5b0 0x7fd37c134300 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 --2- 172.21.15.32:0/1015481574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09e850 0x7fd37c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 -- 172.21.15.32:0/1015481574 >> 172.21.15.32:0/1015481574 conn(0x7fd37c0a3f70 msgr2=0x7fd37c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 -- 172.21.15.32:0/1015481574 shutdown_connections 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.420+0000 7fd383758700 1 -- 172.21.15.32:0/1015481574 wait complete. 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.421+0000 7fd383758700 1 Processor -- start 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 -- start start 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 0x7fd37c122520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09f160 0x7fd37c122a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd37c09f5b0 0x7fd37c122fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c126040 con 0x7fd37c09f5b0 2023-09-30T22:28:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c123980 con 0x7fd37c09e850 2023-09-30T22:28:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd383758700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c124fd0 con 0x7fd37c09f160 2023-09-30T22:28:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd3814f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 0x7fd37c122520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd3814f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 0x7fd37c122520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38190/0 (socket says 172.21.15.32:38190) 2023-09-30T22:28:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd3814f4700 1 -- 172.21.15.32:0/2929597128 learned_addr learned my addr 172.21.15.32:0/2929597128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd380cf3700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09f160 0x7fd37c122a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.422+0000 7fd381cf5700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd37c09f5b0 0x7fd37c122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd3814f4700 1 -- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09f160 msgr2=0x7fd37c122a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd3814f4700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09f160 0x7fd37c122a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd3814f4700 1 -- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd37c09f5b0 msgr2=0x7fd37c122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd3814f4700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd37c09f5b0 0x7fd37c122fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd3814f4700 1 -- 172.21.15.32:0/2929597128 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780094a0 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd3814f4700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 0x7fd37c122520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd37800ec10 tx=0x7fd37800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378009760 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800c850 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd374009a70 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.423+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800cbe0 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.424+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c06f860 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.425+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd37800cd40 con 0x7fd37c09e850 2023-09-30T22:28:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.425+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd37803fac0 con 0x7fd37c09e850 2023-09-30T22:28:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.425+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c0747f0 con 0x7fd37c09e850 2023-09-30T22:28:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.430+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd37801b3b0 con 0x7fd37c09e850 2023-09-30T22:28:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.537+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f20b41231a0 con 0x7f20b41338c0 2023-09-30T22:28:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.537+0000 7f20aaffd700 1 -- 172.21.15.32:0/3085791857 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20a4003e60 con 0x7f20b41338c0 2023-09-30T22:28:53.538 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 msgr2=0x7f20b411ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 0x7f20b411ffd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f20a40098f0 tx=0x7f20a4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 shutdown_connections 2023-09-30T22:28:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b41341d0 0x7f20b4120510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b409e390 0x7f20b411fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 --2- 172.21.15.32:0/3085791857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b41338c0 0x7f20b411ffd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 >> 172.21.15.32:0/3085791857 conn(0x7f20b40a2aa0 msgr2=0x7f20b40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 shutdown_connections 2023-09-30T22:28:53.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.540+0000 7f20bc73a700 1 -- 172.21.15.32:0/3085791857 wait complete. 2023-09-30T22:28:53.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:53.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.611+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4a64132060 con 0x7f4a64130e50 2023-09-30T22:28:53.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.611+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2797044439 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a64132060 con 0x7f4a64130e50 2023-09-30T22:28:53.611 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 msgr2=0x7f4a64139c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 0x7f4a64139c90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00cb90 tx=0x7f4a5c00cea0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 shutdown_connections 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a64130e50 0x7f4a64139c90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a6406fb20 0x7f4a641c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 --2- 172.21.15.32:0/2797044439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64130540 0x7f4a641c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 >> 172.21.15.32:0/2797044439 conn(0x7f4a640a2aa0 msgr2=0x7f4a6406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 shutdown_connections 2023-09-30T22:28:53.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.614+0000 7f4a69ada700 1 -- 172.21.15.32:0/2797044439 wait complete. 2023-09-30T22:28:53.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.651+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd37c06bbb0 con 0x7fd37c09e850 2023-09-30T22:28:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.651+0000 7fd371ffb700 1 -- 172.21.15.32:0/2929597128 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd378021020 con 0x7fd37c09e850 2023-09-30T22:28:53.652 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 msgr2=0x7fd37c122520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 0x7fd37c122520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd37800ec10 tx=0x7fd37800ef20 comp rx=0 tx=0).stop 2023-09-30T22:28:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 shutdown_connections 2023-09-30T22:28:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd37c09f5b0 0x7fd37c122fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd37c09e850 0x7fd37c122520 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 --2- 172.21.15.32:0/2929597128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd37c09f160 0x7fd37c122a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 >> 172.21.15.32:0/2929597128 conn(0x7fd37c0a3f70 msgr2=0x7fd37c1340c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 shutdown_connections 2023-09-30T22:28:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.654+0000 7fd383758700 1 -- 172.21.15.32:0/2929597128 wait complete. 2023-09-30T22:28:53.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:53.811 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.954+0000 7f2edc6be700 1 Processor -- start 2023-09-30T22:28:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.954+0000 7f2edc6be700 1 -- start start 2023-09-30T22:28:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.954+0000 7f2edc6be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.954+0000 7f2edc6be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed409d9f0 0x7f2ed406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2edc6be700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed409e130 con 0x7f2ed406acb0 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2edc6be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed41312b0 con 0x7f2ed409d9f0 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2edc6be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed409f6e0 con 0x7f2ed41316c0 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2eda45a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2ed9c59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed409d9f0 0x7f2ed406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2edac5b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ed406acb0 0x7f2ed4138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54430/0 (socket says 172.21.15.32:54430) 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2edac5b700 1 -- 172.21.15.32:0/2557408787 learned_addr learned my addr 172.21.15.32:0/2557408787 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2ed9458700 1 -- 172.21.15.32:0/2557408787 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8002699 0 0) 0x7f2ecc001490 con 0x7f2ed406acb0 2023-09-30T22:28:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2ed9458700 1 -- 172.21.15.32:0/2557408787 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec00035b0 con 0x7f2ed406acb0 2023-09-30T22:28:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2eda45a700 1 -- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed409d9f0 msgr2=0x7f2ed406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2eda45a700 1 --2- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed409d9f0 0x7f2ed406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2eda45a700 1 -- 172.21.15.32:0/2557408787 >> v1:172.21.15.32:6789/0 conn(0x7f2ed406acb0 legacy=0x7f2ed4138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2eda45a700 1 -- 172.21.15.32:0/2557408787 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed409e760 con 0x7f2ed41316c0 2023-09-30T22:28:53.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2ed9c59700 1 --2- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed409d9f0 0x7f2ed406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2eda45a700 1 --2- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4131a90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009870 tx=0x7f2ed0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d3be57b39fdcc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.955+0000 7f2ed9458700 1 -- 172.21.15.32:0/2557408787 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0011830 con 0x7f2ed41316c0 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2ed9458700 1 -- 172.21.15.32:0/2557408787 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed0011990 con 0x7f2ed41316c0 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2ed9458700 1 -- 172.21.15.32:0/2557408787 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0011d20 con 0x7f2ed41316c0 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 -- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 msgr2=0x7f2ed4131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 --2- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4131a90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009870 tx=0x7f2ed0009b80 comp rx=0 tx=0).stop 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 -- 172.21.15.32:0/2557408787 shutdown_connections 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 --2- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4131a90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 --2- 172.21.15.32:0/2557408787 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed409d9f0 0x7f2ed406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 -- 172.21.15.32:0/2557408787 >> 172.21.15.32:0/2557408787 conn(0x7f2ed4066f50 msgr2=0x7f2ed409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:53.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 -- 172.21.15.32:0/2557408787 shutdown_connections 2023-09-30T22:28:53.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.956+0000 7f2edc6be700 1 -- 172.21.15.32:0/2557408787 wait complete. 2023-09-30T22:28:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.957+0000 7f2edc6be700 1 Processor -- start 2023-09-30T22:28:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.957+0000 7f2edc6be700 1 -- start start 2023-09-30T22:28:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edc6be700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 0x7f2ed4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edc6be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed409d9f0 0x7f2ed41199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edc6be700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edc6be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed411bbd0 con 0x7f2ed409d9f0 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edc6be700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed411c740 con 0x7f2ed406acb0 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edc6be700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed411b860 con 0x7f2ed41316c0 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 0x7f2ed4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 0x7f2ed4119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38198/0 (socket says 172.21.15.32:38198) 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 -- 172.21.15.32:0/1404939735 learned_addr learned my addr 172.21.15.32:0/1404939735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2edac5b700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2ed9c59700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed409d9f0 0x7f2ed41199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 -- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 msgr2=0x7f2ed4119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 -- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed409d9f0 msgr2=0x7f2ed41199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:53.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed409d9f0 0x7f2ed41199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 -- 172.21.15.32:0/1404939735 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40094a0 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.958+0000 7f2eda45a700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 0x7f2ed4119480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00090b0 tx=0x7f2ed0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.959+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0004450 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.959+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed00045b0 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.959+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed00094a0 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.959+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00038e0 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.959+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed4130b50 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.960+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2ed0003a40 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.960+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed412b900 con 0x7f2ed406acb0 2023-09-30T22:28:53.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.960+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2ed0040950 con 0x7f2ed406acb0 2023-09-30T22:28:53.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:53.965+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ed0015070 con 0x7f2ed406acb0 2023-09-30T22:28:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.168+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2ed411be00 con 0x7f2ed406acb0 2023-09-30T22:28:54.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.168+0000 7f2ecaffd700 1 -- 172.21.15.32:0/1404939735 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ed001a420 con 0x7f2ed406acb0 2023-09-30T22:28:54.169 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 msgr2=0x7f2ed4119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 0x7f2ed4119480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00090b0 tx=0x7f2ed0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 shutdown_connections 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ed41316c0 0x7f2ed4119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ed406acb0 0x7f2ed4119480 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 --2- 172.21.15.32:0/1404939735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ed409d9f0 0x7f2ed41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.170+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 >> 172.21.15.32:0/1404939735 conn(0x7f2ed4066f50 msgr2=0x7f2ed412bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.171+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 shutdown_connections 2023-09-30T22:28:54.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.171+0000 7f2edc6be700 1 -- 172.21.15.32:0/1404939735 wait complete. 2023-09-30T22:28:54.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:54.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:54.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:54.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:54.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.702+0000 7f315e342700 1 Processor -- start 2023-09-30T22:28:54.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.703+0000 7f315e342700 1 -- start start 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.703+0000 7f315e342700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.703+0000 7f315e342700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315809e000 0x7f3158130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.703+0000 7f315e342700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315809f130 con 0x7f315809e410 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.703+0000 7f315e342700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315809f520 con 0x7f315809e000 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.703+0000 7f315e342700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f315809e930 con 0x7f315809d6f0 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315809e000 0x7f3158130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f315809e000 0x7f3158130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43256/0 (socket says 172.21.15.32:43256) 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 -- 172.21.15.32:0/3143569578 learned_addr learned my addr 172.21.15.32:0/3143569578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f3157fff700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 -- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 msgr2=0x7f315809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 -- 172.21.15.32:0/3143569578 >> v1:172.21.15.32:6789/0 conn(0x7f315809e410 legacy=0x7f3158130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 -- 172.21.15.32:0/3143569578 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315809b2d0 con 0x7f315809e000 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f3156ffd700 1 -- 172.21.15.32:0/3143569578 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361986264 0 0) 0x7f315809f130 con 0x7f315809e410 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f3157fff700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f31577fe700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f3158130660 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f314c0098f0 tx=0x7f314c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5201c1dc9c2c59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f3156ffd700 1 -- 172.21.15.32:0/3143569578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c011830 con 0x7f315809e000 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.704+0000 7f3156ffd700 1 -- 172.21.15.32:0/3143569578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c011990 con 0x7f315809e000 2023-09-30T22:28:54.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.705+0000 7f3156ffd700 1 -- 172.21.15.32:0/3143569578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c011d20 con 0x7f315809e000 2023-09-30T22:28:54.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 -- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 msgr2=0x7f3158130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f3158130660 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f314c0098f0 tx=0x7f314c009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 -- 172.21.15.32:0/3143569578 shutdown_connections 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 --2- 172.21.15.32:0/3143569578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f3158130660 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 -- 172.21.15.32:0/3143569578 >> 172.21.15.32:0/3143569578 conn(0x7f3158066f50 msgr2=0x7f315809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 -- 172.21.15.32:0/3143569578 shutdown_connections 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.706+0000 7f315e342700 1 -- 172.21.15.32:0/3143569578 wait complete. 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.707+0000 7f315e342700 1 Processor -- start 2023-09-30T22:28:54.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.707+0000 7f315e342700 1 -- start start 2023-09-30T22:28:54.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315e342700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315811ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315e342700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f315811f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315e342700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315809e410 0x7f315811f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315e342700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3158122400 con 0x7f315809e000 2023-09-30T22:28:54.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315e342700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3158121920 con 0x7f315809e410 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315e342700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f315809c700 con 0x7f315809d6f0 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f31577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f315811f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f3157fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315811ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f31577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f315811f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43258/0 (socket says 172.21.15.32:43258) 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f31577fe700 1 -- 172.21.15.32:0/1728237616 learned_addr learned my addr 172.21.15.32:0/1728237616 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.708+0000 7f315c8df700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315809e410 0x7f315811f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3157fff700 1 -- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315809e410 msgr2=0x7f315811f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3157fff700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315809e410 0x7f315811f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3157fff700 1 -- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 msgr2=0x7f315811f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3157fff700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f315811f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3157fff700 1 -- 172.21.15.32:0/1728237616 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31480094a0 con 0x7f315809d6f0 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f31577fe700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f315811f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f315c8df700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315809e410 0x7f315811f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:54.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3157fff700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315811ef70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9b0 tx=0x7f314800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314800c850 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314800c9b0 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c009550 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314800cd60 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.709+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3158072670 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.710+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3148011450 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.710+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3148041420 con 0x7f315809d6f0 2023-09-30T22:28:54.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.710+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31581227e0 con 0x7f315809d6f0 2023-09-30T22:28:54.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.715+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f314801e080 con 0x7f315809d6f0 2023-09-30T22:28:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 Processor -- start 2023-09-30T22:28:54.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 -- start start 2023-09-30T22:28:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133420 0x7fe0341337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe034133e30 0x7fe03412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 0x7fe0341304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03406b990 con 0x7fe034133e30 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340a0590 con 0x7fe034134260 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe03a720700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe03409fb80 con 0x7fe034133420 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.771+0000 7fe033fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133420 0x7fe0341337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 0x7fe0341304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 0x7fe0341304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38210/0 (socket says 172.21.15.32:38210) 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe033fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133420 0x7fe0341337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50644/0 (socket says 172.21.15.32:50644) 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 -- 172.21.15.32:0/2443111973 learned_addr learned my addr 172.21.15.32:0/2443111973 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe0337fe700 1 --2- 172.21.15.32:0/2443111973 >> v2:172.21.15.32:3300/0 conn(0x7fe034133e30 0x7fe03412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 -- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133420 msgr2=0x7fe0341337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:54.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 --2- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133420 0x7fe0341337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 -- 172.21.15.32:0/2443111973 >> v2:172.21.15.32:3300/0 conn(0x7fe034133e30 msgr2=0x7fe03412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 --2- 172.21.15.32:0/2443111973 >> v2:172.21.15.32:3300/0 conn(0x7fe034133e30 0x7fe03412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 -- 172.21.15.32:0/2443111973 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340a0030 con 0x7fe034134260 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe0337fe700 1 --2- 172.21.15.32:0/2443111973 >> v2:172.21.15.32:3300/0 conn(0x7fe034133e30 0x7fe03412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe038cbd700 1 --2- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 0x7fe0341304d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009e50 tx=0x7fe02c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ba0dbd86d0acb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe032ffd700 1 -- 172.21.15.32:0/2443111973 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00e070 con 0x7fe034134260 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe032ffd700 1 -- 172.21.15.32:0/2443111973 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c00b910 con 0x7fe034134260 2023-09-30T22:28:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.772+0000 7fe032ffd700 1 -- 172.21.15.32:0/2443111973 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00bcc0 con 0x7fe034134260 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 -- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 msgr2=0x7fe0341304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 --2- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 0x7fe0341304d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009e50 tx=0x7fe02c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 -- 172.21.15.32:0/2443111973 shutdown_connections 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 --2- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034134260 0x7fe0341304d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 --2- 172.21.15.32:0/2443111973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133420 0x7fe0341337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 --2- 172.21.15.32:0/2443111973 >> v2:172.21.15.32:3300/0 conn(0x7fe034133e30 0x7fe03412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 -- 172.21.15.32:0/2443111973 >> 172.21.15.32:0/2443111973 conn(0x7fe0340a4210 msgr2=0x7fe0340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 -- 172.21.15.32:0/2443111973 shutdown_connections 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.773+0000 7fe03a720700 1 -- 172.21.15.32:0/2443111973 wait complete. 2023-09-30T22:28:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.775+0000 7fe03a720700 1 Processor -- start 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.775+0000 7fe03a720700 1 -- start start 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe03a720700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 0x7fe03409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe03a720700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133e30 0x7fe03409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe03a720700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe034134260 0x7fe03409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe03a720700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03409e200 con 0x7fe034134260 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe03a720700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03411fec0 con 0x7fe034133420 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe03a720700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe03412d890 con 0x7fe034133e30 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 0x7fe03409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 0x7fe03409c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38220/0 (socket says 172.21.15.32:38220) 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 -- 172.21.15.32:0/4211548502 learned_addr learned my addr 172.21.15.32:0/4211548502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe038cbd700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe034134260 0x7fe03409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe0337fe700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133e30 0x7fe03409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 -- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133e30 msgr2=0x7fe03409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133e30 0x7fe03409cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 -- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe034134260 msgr2=0x7fe03409d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe034134260 0x7fe03409d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 -- 172.21.15.32:0/4211548502 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0280094a0 con 0x7fe034133420 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe038cbd700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe034134260 0x7fe03409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.776+0000 7fe033fff700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 0x7fe03409c6e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe02400e9b0 tx=0x7fe02400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.777+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024009710 con 0x7fe034133420 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.777+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02400ee50 con 0x7fe034133420 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.777+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c009b30 con 0x7fe034133420 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.777+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02400ca50 con 0x7fe034133420 2023-09-30T22:28:54.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.777+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03411ea90 con 0x7fe034133420 2023-09-30T22:28:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.778+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe02400cbb0 con 0x7fe034133420 2023-09-30T22:28:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.778+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe02403f750 con 0x7fe034133420 2023-09-30T22:28:54.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.778+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0340a1920 con 0x7fe034133420 2023-09-30T22:28:54.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.783+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe024017080 con 0x7fe034133420 2023-09-30T22:28:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.812+0000 7f6ba19c9700 1 Processor -- start 2023-09-30T22:28:54.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.812+0000 7f6ba19c9700 1 -- start start 2023-09-30T22:28:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6ba19c9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b9c0a0a30 0x7f6b9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6ba19c9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c09d630 0x7f6b9c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6ba19c9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9c0a2a70 con 0x7f6b9c133930 2023-09-30T22:28:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6ba19c9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9c0a01d0 con 0x7f6b9c0a0a30 2023-09-30T22:28:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6ba19c9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c09e2b0 con 0x7f6b9c09d630 2023-09-30T22:28:54.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6b9affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b9c0a0a30 0x7f6b9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6b9b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b9c133930 0x7f6b9c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54446/0 (socket says 172.21.15.32:54446) 2023-09-30T22:28:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6b9b7fe700 1 -- 172.21.15.32:0/3090858164 learned_addr learned my addr 172.21.15.32:0/3090858164 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.813+0000 7f6b9a7fc700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c09d630 0x7f6b9c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b9affd700 1 -- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c09d630 msgr2=0x7f6b9c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b9affd700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c09d630 0x7f6b9c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b99ffb700 1 -- 172.21.15.32:0/3090858164 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062818632 0 0) 0x7f6b9c0a2a70 con 0x7f6b9c133930 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b9affd700 1 -- 172.21.15.32:0/3090858164 >> v1:172.21.15.32:6789/0 conn(0x7f6b9c133930 legacy=0x7f6b9c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b9affd700 1 -- 172.21.15.32:0/3090858164 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c09dc50 con 0x7f6b9c0a0a30 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b9a7fc700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c09d630 0x7f6b9c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b9affd700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0a0a30 0x7f6b9c0a0e00 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009d90 tx=0x7f6b8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540c90d7b9c1c340 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b99ffb700 1 -- 172.21.15.32:0/3090858164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c00e070 con 0x7f6b9c0a0a30 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b99ffb700 1 -- 172.21.15.32:0/3090858164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c00bb20 con 0x7f6b9c0a0a30 2023-09-30T22:28:54.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6b99ffb700 1 -- 172.21.15.32:0/3090858164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c012430 con 0x7f6b9c0a0a30 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.814+0000 7f6ba19c9700 1 -- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0a0a30 msgr2=0x7f6b9c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0a0a30 0x7f6b9c0a0e00 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009d90 tx=0x7f6b8c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 -- 172.21.15.32:0/3090858164 shutdown_connections 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c0a0a30 0x7f6b9c0a0e00 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/3090858164 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c09d630 0x7f6b9c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 -- 172.21.15.32:0/3090858164 >> 172.21.15.32:0/3090858164 conn(0x7f6b9c0a3050 msgr2=0x7f6b9c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:54.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 -- 172.21.15.32:0/3090858164 shutdown_connections 2023-09-30T22:28:54.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.815+0000 7f6ba19c9700 1 -- 172.21.15.32:0/3090858164 wait complete. 2023-09-30T22:28:54.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.816+0000 7f6ba19c9700 1 Processor -- start 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.816+0000 7f6ba19c9700 1 -- start start 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6ba19c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 0x7f6b9c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6ba19c9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b9c0a0a30 0x7f6b9c1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6ba19c9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c133930 0x7f6b9c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6ba19c9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c120620 con 0x7f6b9c09d630 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6ba19c9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0a42c0 con 0x7f6b9c0a0a30 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6ba19c9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9c0a0410 con 0x7f6b9c133930 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6b9affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 0x7f6b9c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6b9affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 0x7f6b9c1c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43294/0 (socket says 172.21.15.32:43294) 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6b9affd700 1 -- 172.21.15.32:0/1291824982 learned_addr learned my addr 172.21.15.32:0/1291824982 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6b9b7fe700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c133930 0x7f6b9c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.817+0000 7f6b9a7fc700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b9c0a0a30 0x7f6b9c1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9affd700 1 -- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c133930 msgr2=0x7f6b9c1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9affd700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c133930 0x7f6b9c1c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9affd700 1 -- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b9c0a0a30 msgr2=0x7f6b9c1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9affd700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b9c0a0a30 0x7f6b9c1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9affd700 1 -- 172.21.15.32:0/1291824982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900094a0 con 0x7f6b9c09d630 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9b7fe700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c133930 0x7f6b9c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b9affd700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 0x7f6b9c1c7e80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014040 tx=0x7f6b8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:54.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c012430 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c020070 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004280 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c009a70 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.818+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c1334c0 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.819+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b8c0043e0 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.819+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9c0747f0 con 0x7f6b9c09d630 2023-09-30T22:28:54.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.820+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b8c022070 con 0x7f6b9c09d630 2023-09-30T22:28:54.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.829+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b8c01b400 con 0x7f6b9c09d630 2023-09-30T22:28:54.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.930+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3158074ea0 con 0x7f315809d6f0 2023-09-30T22:28:54.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.930+0000 7f3154ff9700 1 -- 172.21.15.32:0/1728237616 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3148018990 con 0x7f315809d6f0 2023-09-30T22:28:54.931 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 msgr2=0x7f315811ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315811ef70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9b0 tx=0x7f314800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 shutdown_connections 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f315809e410 0x7f315811f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f315809d6f0 0x7f315811ef70 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 --2- 172.21.15.32:0/1728237616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f315809e000 0x7f315811f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.932+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 >> 172.21.15.32:0/1728237616 conn(0x7f3158066f50 msgr2=0x7f31580a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:54.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.933+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 shutdown_connections 2023-09-30T22:28:54.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:54.933+0000 7f315e342700 1 -- 172.21.15.32:0/1728237616 wait complete. 2023-09-30T22:28:54.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.001+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe034074ea0 con 0x7fe034133420 2023-09-30T22:28:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.001+0000 7fe030ff9700 1 -- 172.21.15.32:0/4211548502 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe02401be10 con 0x7fe034133420 2023-09-30T22:28:55.003 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 msgr2=0x7fe03409c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 0x7fe03409c6e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe02400e9b0 tx=0x7fe02400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 shutdown_connections 2023-09-30T22:28:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe034134260 0x7fe03409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe034133420 0x7fe03409c6e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 --2- 172.21.15.32:0/4211548502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe034133e30 0x7fe03409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 >> 172.21.15.32:0/4211548502 conn(0x7fe0340a4210 msgr2=0x7fe0340a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 shutdown_connections 2023-09-30T22:28:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.004+0000 7fe03a720700 1 -- 172.21.15.32:0/4211548502 wait complete. 2023-09-30T22:28:55.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.039+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b9c072980 con 0x7f6b9c09d630 2023-09-30T22:28:55.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.039+0000 7f6b83fff700 1 -- 172.21.15.32:0/1291824982 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b8c01a380 con 0x7f6b9c09d630 2023-09-30T22:28:55.041 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 msgr2=0x7f6b9c1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 0x7f6b9c1c7e80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014040 tx=0x7f6b8c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 shutdown_connections 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b9c133930 0x7f6b9c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b9c09d630 0x7f6b9c1c7e80 secure :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c014040 tx=0x7f6b8c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 --2- 172.21.15.32:0/1291824982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b9c0a0a30 0x7f6b9c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 >> 172.21.15.32:0/1291824982 conn(0x7f6b9c0a3050 msgr2=0x7f6b9c1bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:55.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 shutdown_connections 2023-09-30T22:28:55.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.042+0000 7f6ba19c9700 1 -- 172.21.15.32:0/1291824982 wait complete. 2023-09-30T22:28:55.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:55.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.330+0000 7fb70ef81700 1 Processor -- start 2023-09-30T22:28:55.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.330+0000 7fb70ef81700 1 -- start start 2023-09-30T22:28:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70ef81700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809e930 0x7fb70809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:55.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70ef81700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb70809f240 0x7fb7081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70ef81700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb708070940 con 0x7fb70809f650 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70ef81700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70806fd60 con 0x7fb70809f240 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70ef81700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70806fa80 con 0x7fb70809e930 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb70809f240 0x7fb7081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70cd1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809e930 0x7fb70809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb70809f240 0x7fb7081317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43298/0 (socket says 172.21.15.32:43298) 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 -- 172.21.15.32:0/4118992520 learned_addr learned my addr 172.21.15.32:0/4118992520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 -- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809e930 msgr2=0x7fb70809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 --2- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809e930 0x7fb70809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 -- 172.21.15.32:0/4118992520 >> v1:172.21.15.32:6789/0 conn(0x7fb70809f650 legacy=0x7fb708131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 -- 172.21.15.32:0/4118992520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70809fb60 con 0x7fb70809f240 2023-09-30T22:28:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb7077fe700 1 -- 172.21.15.32:0/4118992520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955637666 0 0) 0x7fb708070940 con 0x7fb70809f650 2023-09-30T22:28:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb70cd1d700 1 --2- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809e930 0x7fb70809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.331+0000 7fb707fff700 1 --2- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809f240 0x7fb7081317f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098f0 tx=0x7fb6f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a303da3017cf91a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb7077fe700 1 -- 172.21.15.32:0/4118992520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4011830 con 0x7fb70809f240 2023-09-30T22:28:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb7077fe700 1 -- 172.21.15.32:0/4118992520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4011990 con 0x7fb70809f240 2023-09-30T22:28:55.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb7077fe700 1 -- 172.21.15.32:0/4118992520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4011d20 con 0x7fb70809f240 2023-09-30T22:28:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 -- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809f240 msgr2=0x7fb7081317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809f240 0x7fb7081317f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098f0 tx=0x7fb6f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 -- 172.21.15.32:0/4118992520 shutdown_connections 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809e930 0x7fb70809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4118992520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809f240 0x7fb7081317f0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 -- 172.21.15.32:0/4118992520 >> 172.21.15.32:0/4118992520 conn(0x7fb7080a4210 msgr2=0x7fb7080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 -- 172.21.15.32:0/4118992520 shutdown_connections 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.332+0000 7fb70ef81700 1 -- 172.21.15.32:0/4118992520 wait complete. 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.333+0000 7fb70ef81700 1 Processor -- start 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.333+0000 7fb70ef81700 1 -- start start 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70ef81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 0x7fb70806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70ef81700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70809f240 0x7fb70806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:55.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70ef81700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809f650 0x7fb70806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70ef81700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70806cc70 con 0x7fb70809e930 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70ef81700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081a97c0 con 0x7fb70809f650 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70ef81700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080a32d0 con 0x7fb70809f240 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70cd1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 0x7fb70806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb707fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70809f240 0x7fb70806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70cd1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 0x7fb70806b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43306/0 (socket says 172.21.15.32:43306) 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70cd1d700 1 -- 172.21.15.32:0/4089441494 learned_addr learned my addr 172.21.15.32:0/4089441494 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.334+0000 7fb70d51e700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809f650 0x7fb70806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70cd1d700 1 -- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70809f240 msgr2=0x7fb70806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70cd1d700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70809f240 0x7fb70806bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70cd1d700 1 -- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809f650 msgr2=0x7fb70806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70cd1d700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809f650 0x7fb70806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70cd1d700 1 -- 172.21.15.32:0/4089441494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0094a0 con 0x7fb70809e930 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb707fff700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70809f240 0x7fb70806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70d51e700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809f650 0x7fb70806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70cd1d700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 0x7fb70806b880 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00ebe0 tx=0x7fb6fc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00c850 con 0x7fb70809e930 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00c9b0 con 0x7fb70809e930 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f4009550 con 0x7fb70809e930 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00cd40 con 0x7fb70809e930 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.335+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7080a0190 con 0x7fb70809e930 2023-09-30T22:28:55.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.336+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6fc013470 con 0x7fb70809e930 2023-09-30T22:28:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.336+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb6fc03fb30 con 0x7fb70809e930 2023-09-30T22:28:55.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.336+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7081a9600 con 0x7fb70809e930 2023-09-30T22:28:55.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.341+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6fc013720 con 0x7fb70809e930 2023-09-30T22:28:55.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.544+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb7081a9440 con 0x7fb70809e930 2023-09-30T22:28:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.545+0000 7fb7057fa700 1 -- 172.21.15.32:0/4089441494 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6fc017020 con 0x7fb70809e930 2023-09-30T22:28:55.546 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 msgr2=0x7fb70806b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 0x7fb70806b880 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00ebe0 tx=0x7fb6fc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 shutdown_connections 2023-09-30T22:28:55.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb70809f650 0x7fb70806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70809e930 0x7fb70806b880 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 --2- 172.21.15.32:0/4089441494 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb70809f240 0x7fb70806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 >> 172.21.15.32:0/4089441494 conn(0x7fb7080a4210 msgr2=0x7fb7081b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 shutdown_connections 2023-09-30T22:28:55.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:55.547+0000 7fb70ef81700 1 -- 172.21.15.32:0/4089441494 wait complete. 2023-09-30T22:28:55.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:55.949 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:56.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:56.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:56.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.102+0000 7fbf18bbd700 1 Processor -- start 2023-09-30T22:28:56.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.102+0000 7fbf18bbd700 1 -- start start 2023-09-30T22:28:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf18bbd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 0x7fbf14131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf18bbd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf14099a40 0x7fbf14099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf18bbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf1409a3b0 0x7fbf14138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf18bbd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf141313c0 con 0x7fbf14131810 2023-09-30T22:28:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf18bbd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1406e8c0 con 0x7fbf1409a3b0 2023-09-30T22:28:56.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf18bbd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14071d20 con 0x7fbf14099a40 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf12d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf1409a3b0 0x7fbf14138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf1259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 0x7fbf14131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf12d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf1409a3b0 0x7fbf14138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38254/0 (socket says 172.21.15.32:38254) 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf1259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 0x7fbf14131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43316/0 (socket says 172.21.15.32:43316) 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf12d9d700 1 -- 172.21.15.32:0/2043804042 learned_addr learned my addr 172.21.15.32:0/2043804042 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.103+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf14099a40 0x7fbf14099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf11d9b700 1 -- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf1409a3b0 msgr2=0x7fbf14138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf1409a3b0 0x7fbf14138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf11d9b700 1 -- 172.21.15.32:0/2043804042 >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 msgr2=0x7fbf14131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/2043804042 >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 0x7fbf14131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf11d9b700 1 -- 172.21.15.32:0/2043804042 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140721a0 con 0x7fbf14099a40 2023-09-30T22:28:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf1259c700 1 --2- 172.21.15.32:0/2043804042 >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 0x7fbf14131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf14099a40 0x7fbf14099e70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009840 tx=0x7fbf08009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab3fec8f36a36cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf1159a700 1 -- 172.21.15.32:0/2043804042 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08011830 con 0x7fbf14099a40 2023-09-30T22:28:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf1159a700 1 -- 172.21.15.32:0/2043804042 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf08011990 con 0x7fbf14099a40 2023-09-30T22:28:56.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.104+0000 7fbf1159a700 1 -- 172.21.15.32:0/2043804042 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08011d20 con 0x7fbf14099a40 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 -- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf14099a40 msgr2=0x7fbf14099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf14099a40 0x7fbf14099e70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009840 tx=0x7fbf08009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 -- 172.21.15.32:0/2043804042 shutdown_connections 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf1409a3b0 0x7fbf14138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/2043804042 >> v2:172.21.15.32:3300/0 conn(0x7fbf14131810 0x7fbf14131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/2043804042 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf14099a40 0x7fbf14099e70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 -- 172.21.15.32:0/2043804042 >> 172.21.15.32:0/2043804042 conn(0x7fbf140a2aa0 msgr2=0x7fbf140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 -- 172.21.15.32:0/2043804042 shutdown_connections 2023-09-30T22:28:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.105+0000 7fbf18bbd700 1 -- 172.21.15.32:0/2043804042 wait complete. 2023-09-30T22:28:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.107+0000 7fbf18bbd700 1 Processor -- start 2023-09-30T22:28:56.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.107+0000 7fbf18bbd700 1 -- start start 2023-09-30T22:28:56.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf18bbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf14099a40 0x7fbf1406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf18bbd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 0x7fbf1406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf18bbd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf14131810 0x7fbf14070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf18bbd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140713a0 con 0x7fbf14099a40 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf18bbd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1412c600 con 0x7fbf14131810 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf18bbd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1411eff0 con 0x7fbf1409a3b0 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 0x7fbf1406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf1259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf14099a40 0x7fbf1406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 0x7fbf1406fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50690/0 (socket says 172.21.15.32:50690) 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 -- 172.21.15.32:0/172758011 learned_addr learned my addr 172.21.15.32:0/172758011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf12d9d700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf14131810 0x7fbf14070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 -- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf14131810 msgr2=0x7fbf14070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf14131810 0x7fbf14070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 -- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf14099a40 msgr2=0x7fbf1406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf14099a40 0x7fbf1406f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf11d9b700 1 -- 172.21.15.32:0/172758011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040094a0 con 0x7fbf1409a3b0 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.108+0000 7fbf12d9d700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf14131810 0x7fbf14070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.109+0000 7fbf11d9b700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 0x7fbf1406fb50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbf08000c00 tx=0x7fbf08004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.109+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08004450 con 0x7fbf1409a3b0 2023-09-30T22:28:56.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.109+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf080094a0 con 0x7fbf1409a3b0 2023-09-30T22:28:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.109+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0801f070 con 0x7fbf1409a3b0 2023-09-30T22:28:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.109+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf080038e0 con 0x7fbf1409a3b0 2023-09-30T22:28:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.109+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1406eb00 con 0x7fbf1409a3b0 2023-09-30T22:28:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.110+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf08003a40 con 0x7fbf1409a3b0 2023-09-30T22:28:56.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.110+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbf08021070 con 0x7fbf1409a3b0 2023-09-30T22:28:56.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.110+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf14074ea0 con 0x7fbf1409a3b0 2023-09-30T22:28:56.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.120+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf0801a430 con 0x7fbf1409a3b0 2023-09-30T22:28:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.168+0000 7f5afee7f700 1 Processor -- start 2023-09-30T22:28:56.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.168+0000 7f5afee7f700 1 -- start start 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afee7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af809f410 0x7f5af8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afee7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5af8136fe0 0x7f5af81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afee7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af8070940 con 0x7f5af809e930 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afee7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af806fd60 con 0x7f5af8136fe0 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afee7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af806fa80 con 0x7f5af809f410 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afcc1b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5af809e930 0x7f5af809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54452/0 (socket says 172.21.15.32:54452) 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afcc1b700 1 -- 172.21.15.32:0/1114286346 learned_addr learned my addr 172.21.15.32:0/1114286346 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1114286346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af809f410 0x7f5af8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afd41c700 1 --2- 172.21.15.32:0/1114286346 >> v2:172.21.15.32:3300/0 conn(0x7f5af8136fe0 0x7f5af81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5af7fff700 1 -- 172.21.15.32:0/1114286346 >> v2:172.21.15.32:3300/0 conn(0x7f5af8136fe0 msgr2=0x7f5af81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1114286346 >> v2:172.21.15.32:3300/0 conn(0x7f5af8136fe0 0x7f5af81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5af7fff700 1 -- 172.21.15.32:0/1114286346 >> v1:172.21.15.32:6789/0 conn(0x7f5af809e930 legacy=0x7f5af809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5af7fff700 1 -- 172.21.15.32:0/1114286346 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af809fb60 con 0x7f5af809f410 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5afd41c700 1 --2- 172.21.15.32:0/1114286346 >> v2:172.21.15.32:3300/0 conn(0x7f5af8136fe0 0x7f5af81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.169+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1114286346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af809f410 0x7f5af8136aa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009840 tx=0x7f5ae4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18fff245b2f63588 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5af77fe700 1 -- 172.21.15.32:0/1114286346 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4011830 con 0x7f5af809f410 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5af77fe700 1 -- 172.21.15.32:0/1114286346 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4011990 con 0x7f5af809f410 2023-09-30T22:28:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5af77fe700 1 -- 172.21.15.32:0/1114286346 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4011d20 con 0x7f5af809f410 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5afee7f700 1 -- 172.21.15.32:0/1114286346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af809f410 msgr2=0x7f5af8136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5afee7f700 1 --2- 172.21.15.32:0/1114286346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af809f410 0x7f5af8136aa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009840 tx=0x7f5ae4009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5afee7f700 1 -- 172.21.15.32:0/1114286346 shutdown_connections 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5afee7f700 1 --2- 172.21.15.32:0/1114286346 >> v2:172.21.15.32:3300/0 conn(0x7f5af8136fe0 0x7f5af81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5afee7f700 1 --2- 172.21.15.32:0/1114286346 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af809f410 0x7f5af8136aa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.170+0000 7f5afee7f700 1 -- 172.21.15.32:0/1114286346 >> 172.21.15.32:0/1114286346 conn(0x7f5af80a4210 msgr2=0x7f5af80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.171+0000 7f5afee7f700 1 -- 172.21.15.32:0/1114286346 shutdown_connections 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.171+0000 7f5afee7f700 1 -- 172.21.15.32:0/1114286346 wait complete. 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.171+0000 7f5afee7f700 1 Processor -- start 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 -- start start 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 0x7f5af806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809f410 0x7f5af806c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8136fe0 0x7f5af81c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af806bd70 con 0x7f5af809f410 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8122080 con 0x7f5af809e930 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afee7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8132420 con 0x7f5af8136fe0 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afcc1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 0x7f5af806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afcc1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 0x7f5af806c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38278/0 (socket says 172.21.15.32:38278) 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5afcc1b700 1 -- 172.21.15.32:0/1425901419 learned_addr learned my addr 172.21.15.32:0/1425901419 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.172+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809f410 0x7f5af806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5afd41c700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8136fe0 0x7f5af81c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af7fff700 1 -- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8136fe0 msgr2=0x7f5af81c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8136fe0 0x7f5af81c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af7fff700 1 -- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 msgr2=0x7f5af806c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 0x7f5af806c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af7fff700 1 -- 172.21.15.32:0/1425901419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af0009a70 con 0x7f5af809f410 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5afcc1b700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 0x7f5af806c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af7fff700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809f410 0x7f5af806c620 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4016000 tx=0x7f5ae4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40036a0 con 0x7f5af809f410 2023-09-30T22:28:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae401f070 con 0x7f5af809f410 2023-09-30T22:28:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4003a60 con 0x7f5af809f410 2023-09-30T22:28:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5afee7f700 1 -- 172.21.15.32:0/1425901419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae40094a0 con 0x7f5af809f410 2023-09-30T22:28:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.173+0000 7f5afee7f700 1 -- 172.21.15.32:0/1425901419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af806d470 con 0x7f5af809f410 2023-09-30T22:28:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.174+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ae4003bc0 con 0x7f5af809f410 2023-09-30T22:28:56.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.175+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ae4021070 con 0x7f5af809f410 2023-09-30T22:28:56.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.175+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af8074ea0 con 0x7f5af809f410 2023-09-30T22:28:56.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.180+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ae4015020 con 0x7f5af809f410 2023-09-30T22:28:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.225+0000 7f357bb11700 1 Processor -- start 2023-09-30T22:28:56.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.226+0000 7f357bb11700 1 -- start start 2023-09-30T22:28:56.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357bb11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f357409d630 0x7f35740a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357bb11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3574138d80 0x7f357413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357bb11700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740a2a70 con 0x7f35740a0a30 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357bb11700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740a01d0 con 0x7f3574138d80 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357bb11700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f357409e2b0 con 0x7f357409d630 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f35798ad700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f35740a0a30 0x7f35740a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54456/0 (socket says 172.21.15.32:54456) 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f35798ad700 1 -- 172.21.15.32:0/1789568411 learned_addr learned my addr 172.21.15.32:0/1789568411 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f35790ac700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f357409d630 0x7f35740a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357a0ae700 1 --2- 172.21.15.32:0/1789568411 >> v2:172.21.15.32:3300/0 conn(0x7f3574138d80 0x7f357413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f35788ab700 1 -- 172.21.15.32:0/1789568411 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948175458 0 0) 0x7f35740a2a70 con 0x7f35740a0a30 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f35788ab700 1 -- 172.21.15.32:0/1789568411 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35600035b0 con 0x7f35740a0a30 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357a0ae700 1 -- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f357409d630 msgr2=0x7f35740a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357a0ae700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f357409d630 0x7f35740a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357a0ae700 1 -- 172.21.15.32:0/1789568411 >> v1:172.21.15.32:6789/0 conn(0x7f35740a0a30 legacy=0x7f35740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f357a0ae700 1 -- 172.21.15.32:0/1789568411 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357409dc50 con 0x7f3574138d80 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.227+0000 7f35790ac700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f357409d630 0x7f35740a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357a0ae700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3574138d80 0x7f357413b160 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f356c009d90 tx=0x7f356c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1a619fab979247 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f35788ab700 1 -- 172.21.15.32:0/1789568411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e070 con 0x7f3574138d80 2023-09-30T22:28:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f35788ab700 1 -- 172.21.15.32:0/1789568411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c00bb20 con 0x7f3574138d80 2023-09-30T22:28:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f35788ab700 1 -- 172.21.15.32:0/1789568411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012430 con 0x7f3574138d80 2023-09-30T22:28:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357bb11700 1 -- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3574138d80 msgr2=0x7f357413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357bb11700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3574138d80 0x7f357413b160 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f356c009d90 tx=0x7f356c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357bb11700 1 -- 172.21.15.32:0/1789568411 shutdown_connections 2023-09-30T22:28:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357bb11700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3574138d80 0x7f357413b160 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357bb11700 1 --2- 172.21.15.32:0/1789568411 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f357409d630 0x7f35740a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.228+0000 7f357bb11700 1 -- 172.21.15.32:0/1789568411 >> 172.21.15.32:0/1789568411 conn(0x7f35740a3050 msgr2=0x7f35740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.229+0000 7f357bb11700 1 -- 172.21.15.32:0/1789568411 shutdown_connections 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.229+0000 7f357bb11700 1 -- 172.21.15.32:0/1789568411 wait complete. 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.230+0000 7f357bb11700 1 Processor -- start 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.230+0000 7f357bb11700 1 -- start start 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357bb11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f357409d630 0x7f3574133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357bb11700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35740a0a30 0x7f3574133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357bb11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 0x7f3574134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357bb11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35741356d0 con 0x7f357409d630 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357bb11700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3574137c20 con 0x7f35740a0a30 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357bb11700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35740a4990 con 0x7f3574138d80 2023-09-30T22:28:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 0x7f3574134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 0x7f3574134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50722/0 (socket says 172.21.15.32:50722) 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 -- 172.21.15.32:0/2211508950 learned_addr learned my addr 172.21.15.32:0/2211508950 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f35798ad700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f357409d630 0x7f3574133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 -- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35740a0a30 msgr2=0x7f3574133c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35740a0a30 0x7f3574133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 -- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f357409d630 msgr2=0x7f3574133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f357409d630 0x7f3574133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 -- 172.21.15.32:0/2211508950 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640094a0 con 0x7f3574138d80 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.231+0000 7f357a0ae700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 0x7f3574134190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f356c014040 tx=0x7f356c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.232+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012a90 con 0x7f3574138d80 2023-09-30T22:28:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.232+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c020070 con 0x7f3574138d80 2023-09-30T22:28:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.232+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c009a70 con 0x7f3574138d80 2023-09-30T22:28:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.232+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012e50 con 0x7f3574138d80 2023-09-30T22:28:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.232+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35740a0410 con 0x7f3574138d80 2023-09-30T22:28:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.233+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f356c004040 con 0x7f3574138d80 2023-09-30T22:28:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.233+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357413b620 con 0x7f3574138d80 2023-09-30T22:28:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.233+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f356c022070 con 0x7f3574138d80 2023-09-30T22:28:56.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.242+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f356c01a970 con 0x7f3574138d80 2023-09-30T22:28:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.350+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbf1411f370 con 0x7fbf1409a3b0 2023-09-30T22:28:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.351+0000 7fbf02ffd700 1 -- 172.21.15.32:0/172758011 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf08015030 con 0x7fbf1409a3b0 2023-09-30T22:28:56.352 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 msgr2=0x7fbf1406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 0x7fbf1406fb50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbf08000c00 tx=0x7fbf08004750 comp rx=0 tx=0).stop 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 shutdown_connections 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf14131810 0x7fbf14070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf14099a40 0x7fbf1406f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 --2- 172.21.15.32:0/172758011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf1409a3b0 0x7fbf1406fb50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 >> 172.21.15.32:0/172758011 conn(0x7fbf140a2aa0 msgr2=0x7fbf1412e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 shutdown_connections 2023-09-30T22:28:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.353+0000 7fbf18bbd700 1 -- 172.21.15.32:0/172758011 wait complete. 2023-09-30T22:28:56.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.402+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5af806b250 con 0x7f5af809f410 2023-09-30T22:28:56.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.403+0000 7f5af57fa700 1 -- 172.21.15.32:0/1425901419 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ae402f3a0 con 0x7f5af809f410 2023-09-30T22:28:56.404 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809f410 msgr2=0x7f5af806c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809f410 0x7f5af806c620 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4016000 tx=0x7f5ae4004750 comp rx=0 tx=0).stop 2023-09-30T22:28:56.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 shutdown_connections 2023-09-30T22:28:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8136fe0 0x7f5af81c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809e930 0x7f5af806c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 --2- 172.21.15.32:0/1425901419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809f410 0x7f5af806c620 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 >> 172.21.15.32:0/1425901419 conn(0x7f5af80a4210 msgr2=0x7f5af80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 shutdown_connections 2023-09-30T22:28:56.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.405+0000 7f5ae2ffd700 1 -- 172.21.15.32:0/1425901419 wait complete. 2023-09-30T22:28:56.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:56.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.453+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f35740747f0 con 0x7f3574138d80 2023-09-30T22:28:56.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.454+0000 7f356a7fc700 1 -- 172.21.15.32:0/2211508950 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f356c02c030 con 0x7f3574138d80 2023-09-30T22:28:56.455 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 msgr2=0x7f3574134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 0x7f3574134190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f356c014040 tx=0x7f356c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 shutdown_connections 2023-09-30T22:28:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3574138d80 0x7f3574134190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f357409d630 0x7f3574133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 --2- 172.21.15.32:0/2211508950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f35740a0a30 0x7f3574133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 >> 172.21.15.32:0/2211508950 conn(0x7f35740a3050 msgr2=0x7f357413add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 shutdown_connections 2023-09-30T22:28:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.456+0000 7f357bb11700 1 -- 172.21.15.32:0/2211508950 wait complete. 2023-09-30T22:28:56.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:56.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:56.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.797+0000 7f95ad075700 1 Processor -- start 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.797+0000 7f95ad075700 1 -- start start 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95ad075700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95a8130540 0x7f95a8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95ad075700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 0x7f95a806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95ad075700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a806fb20 0x7f95a806ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95ad075700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a809a390 con 0x7f95a8130540 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95ad075700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80716a0 con 0x7f95a8130e50 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95ad075700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8071290 con 0x7f95a806fb20 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95a8130540 0x7f95a8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 0x7f95a806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a806fb20 0x7f95a806ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 0x7f95a806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38292/0 (socket says 172.21.15.32:38292) 2023-09-30T22:28:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95a8130540 0x7f95a8130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43374/0 (socket says 172.21.15.32:43374) 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a659c700 1 -- 172.21.15.32:0/3716492177 learned_addr learned my addr 172.21.15.32:0/3716492177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 -- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a806fb20 msgr2=0x7f95a806ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a806fb20 0x7f95a806ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 -- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 msgr2=0x7f95a806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 0x7f95a806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 -- 172.21.15.32:0/3716492177 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8130340 con 0x7f95a8130540 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a659c700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 0x7f95a806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a759e700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a806fb20 0x7f95a806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.798+0000 7f95a6d9d700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a8130540 0x7f95a8130910 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f959800bae0 tx=0x7f959800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127383a9f1c461b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95a5d9b700 1 -- 172.21.15.32:0/3716492177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959800dc70 con 0x7f95a8130540 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95a5d9b700 1 -- 172.21.15.32:0/3716492177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959800ddd0 con 0x7f95a8130540 2023-09-30T22:28:56.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95a5d9b700 1 -- 172.21.15.32:0/3716492177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598010690 con 0x7f95a8130540 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 -- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a8130540 msgr2=0x7f95a8130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a8130540 0x7f95a8130910 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f959800bae0 tx=0x7f959800bdf0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 -- 172.21.15.32:0/3716492177 shutdown_connections 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a806fb20 0x7f95a806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a8130540 0x7f95a8130910 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 --2- 172.21.15.32:0/3716492177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130e50 0x7f95a806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 -- 172.21.15.32:0/3716492177 >> 172.21.15.32:0/3716492177 conn(0x7f95a80a2aa0 msgr2=0x7f95a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 -- 172.21.15.32:0/3716492177 shutdown_connections 2023-09-30T22:28:56.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.799+0000 7f95ad075700 1 -- 172.21.15.32:0/3716492177 wait complete. 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.801+0000 7f95ad075700 1 Processor -- start 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.801+0000 7f95ad075700 1 -- start start 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95ad075700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 0x7f95a811a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95ad075700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130540 0x7f95a811ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95ad075700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a8130e50 0x7f95a811b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95ad075700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a811e280 con 0x7f95a806fb20 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95ad075700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a811c880 con 0x7f95a8130540 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95ad075700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80a0950 con 0x7f95a8130e50 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 0x7f95a811a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a8130e50 0x7f95a811b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130540 0x7f95a811ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:56.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 0x7f95a811a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43384/0 (socket says 172.21.15.32:43384) 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a6d9d700 1 -- 172.21.15.32:0/2847981680 learned_addr learned my addr 172.21.15.32:0/2847981680 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a659c700 1 -- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a8130e50 msgr2=0x7f95a811b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a659c700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a8130e50 0x7f95a811b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a659c700 1 -- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 msgr2=0x7f95a811a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a659c700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 0x7f95a811a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a659c700 1 -- 172.21.15.32:0/2847981680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a0009a70 con 0x7f95a8130540 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a6d9d700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 0x7f95a811a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.802+0000 7f95a759e700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a8130e50 0x7f95a811b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.803+0000 7f95a659c700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130540 0x7f95a811ac70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f959000e9a0 tx=0x7f959000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.803+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590004030 con 0x7f95a8130540 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.803+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590004190 con 0x7f95a8130540 2023-09-30T22:28:56.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.803+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9590009550 con 0x7f95a8130540 2023-09-30T22:28:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.803+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590004520 con 0x7f95a8130540 2023-09-30T22:28:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.803+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959800b740 con 0x7f95a8130540 2023-09-30T22:28:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.804+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95900036a0 con 0x7f95a8130540 2023-09-30T22:28:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.804+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f959003f4c0 con 0x7f95a8130540 2023-09-30T22:28:56.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.804+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a809cd40 con 0x7f95a8130540 2023-09-30T22:28:56.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:56.809+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9590019b80 con 0x7f95a8130540 2023-09-30T22:28:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.011+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f95a8074ea0 con 0x7f95a8130540 2023-09-30T22:28:57.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.011+0000 7f959f7fe700 1 -- 172.21.15.32:0/2847981680 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9590012ab0 con 0x7f95a8130540 2023-09-30T22:28:57.012 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:57.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130540 msgr2=0x7f95a811ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130540 0x7f95a811ac70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f959000e9a0 tx=0x7f959000ecb0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 shutdown_connections 2023-09-30T22:28:57.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95a8130e50 0x7f95a811b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95a806fb20 0x7f95a811a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 --2- 172.21.15.32:0/2847981680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95a8130540 0x7f95a811ac70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 >> 172.21.15.32:0/2847981680 conn(0x7f95a80a2aa0 msgr2=0x7f95a812d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 shutdown_connections 2023-09-30T22:28:57.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.013+0000 7f95ad075700 1 -- 172.21.15.32:0/2847981680 wait complete. 2023-09-30T22:28:57.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:57.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:57.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:57.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.524+0000 7f609f8ac700 1 Processor -- start 2023-09-30T22:28:57.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.524+0000 7f609f8ac700 1 -- start start 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609f8ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f609809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609f8ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609f8ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f609809e410 0x7f6098133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609f8ac700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609806b460 con 0x7f609809e410 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609f8ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609809b120 con 0x7f609809d6f0 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609f8ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f609809a7e0 con 0x7f609809e000 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f609809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609ce47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f609809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38302/0 (socket says 172.21.15.32:38302) 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 -- 172.21.15.32:0/591330379 learned_addr learned my addr 172.21.15.32:0/591330379 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609de49700 1 --2- 172.21.15.32:0/591330379 >> v2:172.21.15.32:3300/0 conn(0x7f609809e410 0x7f6098133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 -- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 msgr2=0x7f60981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 --2- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 -- 172.21.15.32:0/591330379 >> v2:172.21.15.32:3300/0 conn(0x7f609809e410 msgr2=0x7f6098133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 --2- 172.21.15.32:0/591330379 >> v2:172.21.15.32:3300/0 conn(0x7f609809e410 0x7f6098133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609d648700 1 -- 172.21.15.32:0/591330379 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609809a500 con 0x7f609809d6f0 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.525+0000 7f609ce47700 1 --2- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.526+0000 7f609d648700 1 --2- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f609809dac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6094009920 tx=0x7f6094009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5aec2bad25eb3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:57.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.526+0000 7f608ffff700 1 -- 172.21.15.32:0/591330379 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094011830 con 0x7f609809d6f0 2023-09-30T22:28:57.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.526+0000 7f608ffff700 1 -- 172.21.15.32:0/591330379 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6094011990 con 0x7f609809d6f0 2023-09-30T22:28:57.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.526+0000 7f608ffff700 1 -- 172.21.15.32:0/591330379 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094011d20 con 0x7f609809d6f0 2023-09-30T22:28:57.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 -- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 msgr2=0x7f609809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 --2- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f609809dac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6094009920 tx=0x7f6094009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 -- 172.21.15.32:0/591330379 shutdown_connections 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 --2- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f609809dac0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 --2- 172.21.15.32:0/591330379 >> v2:172.21.15.32:3300/0 conn(0x7f609809e410 0x7f6098133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 --2- 172.21.15.32:0/591330379 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 -- 172.21.15.32:0/591330379 >> 172.21.15.32:0/591330379 conn(0x7f6098066f50 msgr2=0x7f609809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 -- 172.21.15.32:0/591330379 shutdown_connections 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.527+0000 7f609f8ac700 1 -- 172.21.15.32:0/591330379 wait complete. 2023-09-30T22:28:57.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 Processor -- start 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 -- start start 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f60981a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f609809e410 0x7f60981a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981a9c30 con 0x7f609809e410 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60981a8340 con 0x7f609809d6f0 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.528+0000 7f609f8ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60981a9000 con 0x7f609809e000 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609ce47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f609809e410 0x7f60981a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609ce47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50742/0 (socket says 172.21.15.32:50742) 2023-09-30T22:28:57.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609ce47700 1 -- 172.21.15.32:0/30868312 learned_addr learned my addr 172.21.15.32:0/30868312 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609d648700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f60981a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 -- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 msgr2=0x7f60981a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 -- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 msgr2=0x7f60981a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f60981a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 -- 172.21.15.32:0/30868312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60880094a0 con 0x7f609809e410 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609ce47700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609d648700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f60981a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f609de49700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f609809e410 0x7f60981a6c70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f609000ef90 tx=0x7f609000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.529+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000cd40 con 0x7f609809e410 2023-09-30T22:28:57.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.530+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000f040 con 0x7f609809e410 2023-09-30T22:28:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.530+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60900146c0 con 0x7f609809e410 2023-09-30T22:28:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.530+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6094009550 con 0x7f609809e410 2023-09-30T22:28:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.530+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609809c710 con 0x7f609809e410 2023-09-30T22:28:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.531+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6090014820 con 0x7f609809e410 2023-09-30T22:28:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.531+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60900402e0 con 0x7f609809e410 2023-09-30T22:28:57.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.531+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60981a95f0 con 0x7f609809e410 2023-09-30T22:28:57.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.540+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60981a95f0 con 0x7f609809e410 2023-09-30T22:28:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.568+0000 7f037d4e1700 1 Processor -- start 2023-09-30T22:28:57.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.568+0000 7f037d4e1700 1 -- start start 2023-09-30T22:28:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f037d4e1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03780a0a30 0x7f03780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f037d4e1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f03780a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f037d4e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 0x7f0378133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f037d4e1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780a2a70 con 0x7f0378133930 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f037d4e1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03780a01d0 con 0x7f03780a0a30 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f037d4e1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f037809e2b0 con 0x7f037809d630 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f036ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f03780a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f0376ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03780a0a30 0x7f03780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f03777fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 0x7f0378133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f036ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f03780a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50752/0 (socket says 172.21.15.32:50752) 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f03777fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 0x7f0378133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43406/0 (socket says 172.21.15.32:43406) 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f0376ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03780a0a30 0x7f03780a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38326/0 (socket says 172.21.15.32:38326) 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.569+0000 7f036ffff700 1 -- 172.21.15.32:0/2873819519 learned_addr learned my addr 172.21.15.32:0/2873819519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f036ffff700 1 -- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03780a0a30 msgr2=0x7f03780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f036ffff700 1 --2- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03780a0a30 0x7f03780a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f036ffff700 1 -- 172.21.15.32:0/2873819519 >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 msgr2=0x7f0378133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:57.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f036ffff700 1 --2- 172.21.15.32:0/2873819519 >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 0x7f0378133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f036ffff700 1 -- 172.21.15.32:0/2873819519 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037809dc50 con 0x7f037809d630 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f03777fe700 1 --2- 172.21.15.32:0/2873819519 >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 0x7f0378133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f036ffff700 1 --2- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f03780a1340 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03600098f0 tx=0x7f0360009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f920eb1fdc1e25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f03767fc700 1 -- 172.21.15.32:0/2873819519 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360011830 con 0x7f037809d630 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f03767fc700 1 -- 172.21.15.32:0/2873819519 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360011990 con 0x7f037809d630 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.570+0000 7f03767fc700 1 -- 172.21.15.32:0/2873819519 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360011d20 con 0x7f037809d630 2023-09-30T22:28:57.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 -- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 msgr2=0x7f03780a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 --2- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f03780a1340 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03600098f0 tx=0x7f0360009c00 comp rx=0 tx=0).stop 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 -- 172.21.15.32:0/2873819519 shutdown_connections 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 --2- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03780a0a30 0x7f03780a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 --2- 172.21.15.32:0/2873819519 >> v2:172.21.15.32:3300/0 conn(0x7f0378133930 0x7f0378133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 --2- 172.21.15.32:0/2873819519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f03780a1340 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 -- 172.21.15.32:0/2873819519 >> 172.21.15.32:0/2873819519 conn(0x7f03780a3050 msgr2=0x7f03780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 -- 172.21.15.32:0/2873819519 shutdown_connections 2023-09-30T22:28:57.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.571+0000 7f037d4e1700 1 -- 172.21.15.32:0/2873819519 wait complete. 2023-09-30T22:28:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 Processor -- start 2023-09-30T22:28:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 -- start start 2023-09-30T22:28:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f037811f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03780a0a30 0x7f037811f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0378133930 0x7f03781c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037811ee60 con 0x7f03780a0a30 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037811fdd0 con 0x7f0378133930 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.573+0000 7f037d4e1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0378121e70 con 0x7f037809d630 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f037811f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f036ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03780a0a30 0x7f037811f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f037811f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50758/0 (socket says 172.21.15.32:50758) 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 -- 172.21.15.32:0/157109420 learned_addr learned my addr 172.21.15.32:0/157109420 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f03777fe700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0378133930 0x7f03781c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 -- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0378133930 msgr2=0x7f03781c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0378133930 0x7f03781c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 -- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03780a0a30 msgr2=0x7f037811f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03780a0a30 0x7f037811f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 -- 172.21.15.32:0/157109420 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370009a70 con 0x7f037809d630 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f0376ffd700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f037811f350 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f036800e9b0 tx=0x7f036800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.574+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368009710 con 0x7f037809d630 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.575+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036800ee40 con 0x7f037809d630 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.575+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0360009550 con 0x7f037809d630 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.575+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036800ca50 con 0x7f037809d630 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.575+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0378122130 con 0x7f037809d630 2023-09-30T22:28:57.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.576+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0378122320 con 0x7f037809d630 2023-09-30T22:28:57.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.576+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f036800cbb0 con 0x7f037809d630 2023-09-30T22:28:57.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.576+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f036803fa90 con 0x7f037809d630 2023-09-30T22:28:57.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.581+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f036801b3b0 con 0x7f037809d630 2023-09-30T22:28:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 Processor -- start 2023-09-30T22:28:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 -- start start 2023-09-30T22:28:57.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64ec09d630 0x7f64ec0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec138d80 0x7f64ec13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec0a2a70 con 0x7f64ec0a0a30 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0a01d0 con 0x7f64ec09d630 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64f2260700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec09e2b0 con 0x7f64ec138d80 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64eaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64ec09d630 0x7f64ec0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64eb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64ec0a0a30 0x7f64ec0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54468/0 (socket says 172.21.15.32:54468) 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64eb7fe700 1 -- 172.21.15.32:0/3186280612 learned_addr learned my addr 172.21.15.32:0/3186280612 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:57.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64eaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64ec09d630 0x7f64ec0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43418/0 (socket says 172.21.15.32:43418) 2023-09-30T22:28:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.616+0000 7f64ebfff700 1 --2- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec138d80 0x7f64ec13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64eaffd700 1 -- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec138d80 msgr2=0x7f64ec13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64eaffd700 1 --2- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec138d80 0x7f64ec13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64eaffd700 1 -- 172.21.15.32:0/3186280612 >> v1:172.21.15.32:6789/0 conn(0x7f64ec0a0a30 legacy=0x7f64ec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64eaffd700 1 -- 172.21.15.32:0/3186280612 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec09dc50 con 0x7f64ec09d630 2023-09-30T22:28:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64eaffd700 1 --2- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec09d630 0x7f64ec0a1510 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009840 tx=0x7f64e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bc442f6753c61f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64ea7fc700 1 -- 172.21.15.32:0/3186280612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0011830 con 0x7f64ec09d630 2023-09-30T22:28:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64ea7fc700 1 -- 172.21.15.32:0/3186280612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0011990 con 0x7f64ec09d630 2023-09-30T22:28:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.617+0000 7f64ea7fc700 1 -- 172.21.15.32:0/3186280612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0011d20 con 0x7f64ec09d630 2023-09-30T22:28:57.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 -- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec09d630 msgr2=0x7f64ec0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 --2- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec09d630 0x7f64ec0a1510 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009840 tx=0x7f64e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:28:57.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 -- 172.21.15.32:0/3186280612 shutdown_connections 2023-09-30T22:28:57.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 --2- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec138d80 0x7f64ec13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 --2- 172.21.15.32:0/3186280612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec09d630 0x7f64ec0a1510 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 -- 172.21.15.32:0/3186280612 >> 172.21.15.32:0/3186280612 conn(0x7f64ec0a3050 msgr2=0x7f64ec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 -- 172.21.15.32:0/3186280612 shutdown_connections 2023-09-30T22:28:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.618+0000 7f64f2260700 1 -- 172.21.15.32:0/3186280612 wait complete. 2023-09-30T22:28:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.619+0000 7f64f2260700 1 Processor -- start 2023-09-30T22:28:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.619+0000 7f64f2260700 1 -- start start 2023-09-30T22:28:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64f2260700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec09d630 0x7f64ec1207a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64f2260700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 0x7f64ec120ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64f2260700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec138d80 0x7f64ec121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64f2260700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec121c70 con 0x7f64ec138d80 2023-09-30T22:28:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64f2260700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec123670 con 0x7f64ec09d630 2023-09-30T22:28:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64f2260700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec122bf0 con 0x7f64ec0a0a30 2023-09-30T22:28:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 0x7f64ec120ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 0x7f64ec120ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50772/0 (socket says 172.21.15.32:50772) 2023-09-30T22:28:57.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 -- 172.21.15.32:0/1933628044 learned_addr learned my addr 172.21.15.32:0/1933628044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:57.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64ebfff700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec138d80 0x7f64ec121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:57.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 -- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec09d630 msgr2=0x7f64ec1207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec09d630 0x7f64ec1207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 -- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec138d80 msgr2=0x7f64ec121220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec138d80 0x7f64ec121220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.620+0000 7f64eaffd700 1 -- 172.21.15.32:0/1933628044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00094a0 con 0x7f64ec0a0a30 2023-09-30T22:28:57.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.621+0000 7f64eaffd700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 0x7f64ec120ce0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f64e00090b0 tx=0x7f64e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.621+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004480 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.621+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e001f070 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.621+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0a4e00 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.621+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00038e0 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.621+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0a0460 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.622+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64e0021030 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.622+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f64e0040bd0 con 0x7f64ec0a0a30 2023-09-30T22:28:57.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.622+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec124b10 con 0x7f64ec0a0a30 2023-09-30T22:28:57.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.632+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64e001a7d0 con 0x7f64ec0a0a30 2023-09-30T22:28:57.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.756+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f60981a95f0 con 0x7f609809e410 2023-09-30T22:28:57.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.757+0000 7f608dffb700 1 -- 172.21.15.32:0/30868312 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6090018030 con 0x7f609809e410 2023-09-30T22:28:57.758 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:57.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f609809e410 msgr2=0x7f60981a6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f609809e410 0x7f60981a6c70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f609000ef90 tx=0x7f609000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 shutdown_connections 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f609809e410 0x7f60981a6c70 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f609809d6f0 0x7f60981a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 --2- 172.21.15.32:0/30868312 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f609809e000 0x7f60981a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 >> 172.21.15.32:0/30868312 conn(0x7f6098066f50 msgr2=0x7f60981baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 shutdown_connections 2023-09-30T22:28:57.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.759+0000 7f609f8ac700 1 -- 172.21.15.32:0/30868312 wait complete. 2023-09-30T22:28:57.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:57.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.808+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f03780747f0 con 0x7f037809d630 2023-09-30T22:28:57.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.808+0000 7f036f7fe700 1 -- 172.21.15.32:0/157109420 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0368017020 con 0x7f037809d630 2023-09-30T22:28:57.810 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 msgr2=0x7f037811f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f037811f350 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f036800e9b0 tx=0x7f036800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 shutdown_connections 2023-09-30T22:28:57.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0378133930 0x7f03781c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f037809d630 0x7f037811f350 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 --2- 172.21.15.32:0/157109420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03780a0a30 0x7f037811f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 >> 172.21.15.32:0/157109420 conn(0x7f03780a3050 msgr2=0x7f037809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 shutdown_connections 2023-09-30T22:28:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.810+0000 7f037d4e1700 1 -- 172.21.15.32:0/157109420 wait complete. 2023-09-30T22:28:57.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:57.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.854+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64ec0747f0 con 0x7f64ec0a0a30 2023-09-30T22:28:57.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.854+0000 7f64d3fff700 1 -- 172.21.15.32:0/1933628044 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64e0015020 con 0x7f64ec0a0a30 2023-09-30T22:28:57.856 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.856+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 msgr2=0x7f64ec120ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.856+0000 7f64f2260700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 0x7f64ec120ce0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f64e00090b0 tx=0x7f64e0004750 comp rx=0 tx=0).stop 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 shutdown_connections 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec138d80 0x7f64ec121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec09d630 0x7f64ec1207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 --2- 172.21.15.32:0/1933628044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64ec0a0a30 0x7f64ec120ce0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 >> 172.21.15.32:0/1933628044 conn(0x7f64ec0a3050 msgr2=0x7f64ec09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 shutdown_connections 2023-09-30T22:28:57.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:57.857+0000 7f64f2260700 1 -- 172.21.15.32:0/1933628044 wait complete. 2023-09-30T22:28:57.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:58.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.174+0000 7fa12d500700 1 Processor -- start 2023-09-30T22:28:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.174+0000 7fa12d500700 1 -- start start 2023-09-30T22:28:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa12d500700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1280a0a30 0x7fa1280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa12d500700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa128138d80 0x7fa12813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa12d500700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1280a2a70 con 0x7fa12809d630 2023-09-30T22:28:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa12d500700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280a01d0 con 0x7fa128138d80 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa12d500700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12809e2b0 con 0x7fa1280a0a30 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa126ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1280a0a30 0x7fa1280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa1277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa128138d80 0x7fa12813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa126ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1280a0a30 0x7fa1280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38350/0 (socket says 172.21.15.32:38350) 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa126ffd700 1 -- 172.21.15.32:0/3145917106 learned_addr learned my addr 172.21.15.32:0/3145917106 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa1277fe700 1 -- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1280a0a30 msgr2=0x7fa1280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa1277fe700 1 --2- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1280a0a30 0x7fa1280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa1277fe700 1 -- 172.21.15.32:0/3145917106 >> v1:172.21.15.32:6789/0 conn(0x7fa12809d630 legacy=0x7fa1280a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa1277fe700 1 -- 172.21.15.32:0/3145917106 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12809dc50 con 0x7fa128138d80 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa125ffb700 1 -- 172.21.15.32:0/3145917106 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309223263 0 0) 0x7fa1280a2a70 con 0x7fa12809d630 2023-09-30T22:28:58.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.175+0000 7fa1277fe700 1 --2- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa128138d80 0x7fa12813b160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa120009d90 tx=0x7fa12000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d445e7cd21e9cfd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa125ffb700 1 -- 172.21.15.32:0/3145917106 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000e070 con 0x7fa128138d80 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa125ffb700 1 -- 172.21.15.32:0/3145917106 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000bb20 con 0x7fa128138d80 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa125ffb700 1 -- 172.21.15.32:0/3145917106 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120012430 con 0x7fa128138d80 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 -- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa128138d80 msgr2=0x7fa12813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 --2- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa128138d80 0x7fa12813b160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa120009d90 tx=0x7fa12000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 -- 172.21.15.32:0/3145917106 shutdown_connections 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 --2- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa128138d80 0x7fa12813b160 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 --2- 172.21.15.32:0/3145917106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1280a0a30 0x7fa1280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 -- 172.21.15.32:0/3145917106 >> 172.21.15.32:0/3145917106 conn(0x7fa1280a3050 msgr2=0x7fa1280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 -- 172.21.15.32:0/3145917106 shutdown_connections 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.176+0000 7fa12d500700 1 -- 172.21.15.32:0/3145917106 wait complete. 2023-09-30T22:28:58.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.177+0000 7fa12d500700 1 Processor -- start 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 -- start start 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 0x7fa128133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 0x7fa128133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 0x7fa1281342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128134d00 con 0x7fa12809d630 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa128136700 con 0x7fa128138d80 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa12d500700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa128137070 con 0x7fa1280a0a30 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa1267fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 0x7fa128133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa1277fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 0x7fa1281342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 0x7fa128133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 0x7fa128133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43462/0 (socket says 172.21.15.32:43462) 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa1267fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 0x7fa128133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50784/0 (socket says 172.21.15.32:50784) 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa1277fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 0x7fa1281342b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38358/0 (socket says 172.21.15.32:38358) 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 -- 172.21.15.32:0/309058689 learned_addr learned my addr 172.21.15.32:0/309058689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 -- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 msgr2=0x7fa128133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 0x7fa128133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 -- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 msgr2=0x7fa1281342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.178+0000 7fa126ffd700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 0x7fa1281342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa126ffd700 1 -- 172.21.15.32:0/309058689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120009a70 con 0x7fa12809d630 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa1267fc700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 0x7fa128133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa1277fe700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 0x7fa1281342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa126ffd700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 0x7fa128133830 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e940 tx=0x7fa11800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180096a0 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118013680 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280a0410 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118013a30 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.179+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa128136e00 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.180+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa118013b90 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.180+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa11803f7b0 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.180+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280747f0 con 0x7fa12809d630 2023-09-30T22:28:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.185+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa118017020 con 0x7fa12809d630 2023-09-30T22:28:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.390+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa128072980 con 0x7fa12809d630 2023-09-30T22:28:58.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.391+0000 7fa11f7fe700 1 -- 172.21.15.32:0/309058689 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa11802a090 con 0x7fa12809d630 2023-09-30T22:28:58.392 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 msgr2=0x7fa128133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 0x7fa128133830 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e940 tx=0x7fa11800ec50 comp rx=0 tx=0).stop 2023-09-30T22:28:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 shutdown_connections 2023-09-30T22:28:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa128138d80 0x7fa1281342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa12809d630 0x7fa128133830 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 --2- 172.21.15.32:0/309058689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa1280a0a30 0x7fa128133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 >> 172.21.15.32:0/309058689 conn(0x7fa1280a3050 msgr2=0x7fa12809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 shutdown_connections 2023-09-30T22:28:58.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.393+0000 7fa12d500700 1 -- 172.21.15.32:0/309058689 wait complete. 2023-09-30T22:28:58.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:28:58.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:28:58.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:28:58.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:28:58.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.926+0000 7f5a4cdea700 1 Processor -- start 2023-09-30T22:28:58.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.926+0000 7f5a4cdea700 1 -- start start 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a4cdea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a4809e1d0 0x7f5a48138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a4cdea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48139170 0x7f5a4813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a4cdea700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4809b120 con 0x7f5a4809d6f0 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a4cdea700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4806b460 con 0x7f5a4809e1d0 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a4cdea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4809a7e0 con 0x7f5a48139170 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a4809e1d0 0x7f5a48138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a46d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48139170 0x7f5a4813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a4809e1d0 0x7f5a48138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43468/0 (socket says 172.21.15.32:43468) 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 -- 172.21.15.32:0/415281555 learned_addr learned my addr 172.21.15.32:0/415281555 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a46d9d700 1 --2- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48139170 0x7f5a4813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38366/0 (socket says 172.21.15.32:38366) 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 -- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48139170 msgr2=0x7f5a4813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 --2- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48139170 0x7f5a4813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 -- 172.21.15.32:0/415281555 >> v1:172.21.15.32:6789/0 conn(0x7f5a4809d6f0 legacy=0x7f5a4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.927+0000 7f5a45d9b700 1 -- 172.21.15.32:0/415281555 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4809a500 con 0x7f5a4809e1d0 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.928+0000 7f5a45d9b700 1 --2- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4809e1d0 0x7f5a48138c30 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009920 tx=0x7f5a3c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5c28151129c691 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.928+0000 7f5a4559a700 1 -- 172.21.15.32:0/415281555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011830 con 0x7f5a4809e1d0 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.928+0000 7f5a4559a700 1 -- 172.21.15.32:0/415281555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c011990 con 0x7f5a4809e1d0 2023-09-30T22:28:58.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.928+0000 7f5a4559a700 1 -- 172.21.15.32:0/415281555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011d20 con 0x7f5a4809e1d0 2023-09-30T22:28:58.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 -- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4809e1d0 msgr2=0x7f5a48138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4809e1d0 0x7f5a48138c30 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009920 tx=0x7f5a3c009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:58.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 -- 172.21.15.32:0/415281555 shutdown_connections 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48139170 0x7f5a4813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/415281555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4809e1d0 0x7f5a48138c30 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 -- 172.21.15.32:0/415281555 >> 172.21.15.32:0/415281555 conn(0x7f5a48066f50 msgr2=0x7f5a4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 -- 172.21.15.32:0/415281555 shutdown_connections 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.929+0000 7f5a4cdea700 1 -- 172.21.15.32:0/415281555 wait complete. 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.930+0000 7f5a4cdea700 1 Processor -- start 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.930+0000 7f5a4cdea700 1 -- start start 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4cdea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a4809d6f0 0x7f5a481c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4cdea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5a4809e1d0 0x7f5a481c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4cdea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a48139170 0x7f5a481c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4cdea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48120800 con 0x7f5a48139170 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4cdea700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48121ff0 con 0x7f5a4809d6f0 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4cdea700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48121d20 con 0x7f5a4809e1d0 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a4809d6f0 0x7f5a481c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5a4809e1d0 0x7f5a481c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a4809d6f0 0x7f5a481c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38376/0 (socket says 172.21.15.32:38376) 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a4659c700 1 -- 172.21.15.32:0/3031869148 learned_addr learned my addr 172.21.15.32:0/3031869148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a46d9d700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a48139170 0x7f5a481c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 -- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a4809d6f0 msgr2=0x7f5a481c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a4809d6f0 0x7f5a481c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 -- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a48139170 msgr2=0x7f5a481c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a48139170 0x7f5a481c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 -- 172.21.15.32:0/3031869148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c009550 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.931+0000 7f5a45d9b700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5a4809e1d0 0x7f5a481c89a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c016000 tx=0x7f5a3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.932+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c004510 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.932+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c0036a0 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.932+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4809c710 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.932+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c003a50 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.932+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48122d20 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.933+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a3c003bb0 con 0x7f5a4809e1d0 2023-09-30T22:28:58.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.933+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5a3c021030 con 0x7f5a4809e1d0 2023-09-30T22:28:58.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.933+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48122210 con 0x7f5a4809e1d0 2023-09-30T22:28:58.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.938+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a48122210 con 0x7f5a4809e1d0 2023-09-30T22:28:58.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 Processor -- start 2023-09-30T22:28:58.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 -- start start 2023-09-30T22:28:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409e850 0x7f4f8409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f84134290 0x7f4f8413be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8406bc50 con 0x7f4f8409f160 2023-09-30T22:28:58.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8409ae60 con 0x7f4f84134290 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.971+0000 7f4f8905f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8406b990 con 0x7f4f8409e850 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f84134290 0x7f4f8413be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f82d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409e850 0x7f4f8409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f8409f160 0x7f4f84133b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54496/0 (socket says 172.21.15.32:54496) 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8259c700 1 -- 172.21.15.32:0/3217023790 learned_addr learned my addr 172.21.15.32:0/3217023790 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8359e700 1 -- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409e850 msgr2=0x7f4f8409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8359e700 1 --2- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409e850 0x7f4f8409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8359e700 1 -- 172.21.15.32:0/3217023790 >> v1:172.21.15.32:6789/0 conn(0x7f4f8409f160 legacy=0x7f4f84133b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f81d9b700 1 -- 172.21.15.32:0/3217023790 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797479837 0 0) 0x7f4f8406bc50 con 0x7f4f8409f160 2023-09-30T22:28:58.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8359e700 1 -- 172.21.15.32:0/3217023790 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8406cb10 con 0x7f4f84134290 2023-09-30T22:28:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f82d9d700 1 --2- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409e850 0x7f4f8409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f8359e700 1 --2- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f84134290 0x7f4f8413be50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009d90 tx=0x7f4f7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de615a449958db8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f81d9b700 1 -- 172.21.15.32:0/3217023790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00e070 con 0x7f4f84134290 2023-09-30T22:28:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f81d9b700 1 -- 172.21.15.32:0/3217023790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c00bb20 con 0x7f4f84134290 2023-09-30T22:28:58.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.972+0000 7f4f81d9b700 1 -- 172.21.15.32:0/3217023790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c012430 con 0x7f4f84134290 2023-09-30T22:28:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 -- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f84134290 msgr2=0x7f4f8413be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 --2- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f84134290 0x7f4f8413be50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009d90 tx=0x7f4f7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:28:58.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 -- 172.21.15.32:0/3217023790 shutdown_connections 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 --2- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f84134290 0x7f4f8413be50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 --2- 172.21.15.32:0/3217023790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409e850 0x7f4f8409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 -- 172.21.15.32:0/3217023790 >> 172.21.15.32:0/3217023790 conn(0x7f4f840a3f70 msgr2=0x7f4f840a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 -- 172.21.15.32:0/3217023790 shutdown_connections 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.973+0000 7f4f8905f700 1 -- 172.21.15.32:0/3217023790 wait complete. 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 Processor -- start 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 -- start start 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8409e850 0x7f4f84122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 0x7f4f84122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f84134290 0x7f4f84123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:58.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84133400 con 0x7f4f8409e850 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8406faa0 con 0x7f4f8409f160 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8905f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8406cd30 con 0x7f4f84134290 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 0x7f4f84122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f82d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8409e850 0x7f4f84122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 0x7f4f84122e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38386/0 (socket says 172.21.15.32:38386) 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.975+0000 7f4f8259c700 1 -- 172.21.15.32:0/2759029120 learned_addr learned my addr 172.21.15.32:0/2759029120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:58.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f8359e700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f84134290 0x7f4f84123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f82d9d700 1 -- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f84134290 msgr2=0x7f4f84123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f82d9d700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f84134290 0x7f4f84123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f82d9d700 1 -- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 msgr2=0x7f4f84122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f82d9d700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 0x7f4f84122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f82d9d700 1 -- 172.21.15.32:0/2759029120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f740094a0 con 0x7f4f8409e850 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f8259c700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 0x7f4f84122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f82d9d700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8409e850 0x7f4f84122900 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400e9b0 tx=0x7f4f7400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400ee50 con 0x7f4f8409e850 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7400c850 con 0x7f4f8409e850 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c009a70 con 0x7f4f8409e850 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400cc00 con 0x7f4f8409e850 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.976+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f84125e80 con 0x7f4f8409e850 2023-09-30T22:28:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.977+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f84126070 con 0x7f4f8409e850 2023-09-30T22:28:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.977+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f74013470 con 0x7f4f8409e850 2023-09-30T22:28:58.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.978+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4f7403f7f0 con 0x7f4f8409e850 2023-09-30T22:28:58.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:58.982+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f74013720 con 0x7f4f8409e850 2023-09-30T22:28:59.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 Processor -- start 2023-09-30T22:28:59.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 -- start start 2023-09-30T22:28:59.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 0x7f64600a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6460138d80 0x7f646013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64600a2a70 con 0x7f64600a0a30 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600a01d0 con 0x7f646009d630 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.014+0000 7f6466aef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f646009e2b0 con 0x7f6460138d80 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 0x7f64600a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 0x7f64600a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38402/0 (socket says 172.21.15.32:38402) 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 -- 172.21.15.32:0/2822241786 learned_addr learned my addr 172.21.15.32:0/2822241786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f646488b700 1 --1- 172.21.15.32:0/2822241786 >> v1:172.21.15.32:6789/0 conn(0x7f64600a0a30 0x7f64600a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54508/0 (socket says 172.21.15.32:54508) 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f646508c700 1 --2- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6460138d80 0x7f646013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 -- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6460138d80 msgr2=0x7f646013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 --2- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6460138d80 0x7f646013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 -- 172.21.15.32:0/2822241786 >> v1:172.21.15.32:6789/0 conn(0x7f64600a0a30 legacy=0x7f64600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645f7fe700 1 -- 172.21.15.32:0/2822241786 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9438902 0 0) 0x7f64600a2a70 con 0x7f64600a0a30 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 -- 172.21.15.32:0/2822241786 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646009dc50 con 0x7f646009d630 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645ffff700 1 --2- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 0x7f64600a1510 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f644c009920 tx=0x7f644c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c06c096ae82660e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.015+0000 7f645f7fe700 1 -- 172.21.15.32:0/2822241786 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c011830 con 0x7f646009d630 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f645f7fe700 1 -- 172.21.15.32:0/2822241786 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c011990 con 0x7f646009d630 2023-09-30T22:28:59.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f645f7fe700 1 -- 172.21.15.32:0/2822241786 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c011d20 con 0x7f646009d630 2023-09-30T22:28:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 -- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 msgr2=0x7f64600a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 --2- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 0x7f64600a1510 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f644c009920 tx=0x7f644c009c30 comp rx=0 tx=0).stop 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 -- 172.21.15.32:0/2822241786 shutdown_connections 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 --2- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6460138d80 0x7f646013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 --2- 172.21.15.32:0/2822241786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646009d630 0x7f64600a1510 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 -- 172.21.15.32:0/2822241786 >> 172.21.15.32:0/2822241786 conn(0x7f64600a3050 msgr2=0x7f64600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 -- 172.21.15.32:0/2822241786 shutdown_connections 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.016+0000 7f6466aef700 1 -- 172.21.15.32:0/2822241786 wait complete. 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.017+0000 7f6466aef700 1 Processor -- start 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.017+0000 7f6466aef700 1 -- start start 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f6466aef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646009d630 0x7f6460133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f6466aef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64600a0a30 0x7f6460133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f6466aef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 0x7f64601341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f6466aef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460137220 con 0x7f646009d630 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f6466aef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6460137d60 con 0x7f6460138d80 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f6466aef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64600a4cb0 con 0x7f64600a0a30 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f646508c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 0x7f64601341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f646488b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646009d630 0x7f6460133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f646508c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 0x7f64601341b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38410/0 (socket says 172.21.15.32:38410) 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f646508c700 1 -- 172.21.15.32:0/2003883265 learned_addr learned my addr 172.21.15.32:0/2003883265 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.018+0000 7f645ffff700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64600a0a30 0x7f6460133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f646508c700 1 -- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64600a0a30 msgr2=0x7f6460133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f646508c700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64600a0a30 0x7f6460133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f646508c700 1 -- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646009d630 msgr2=0x7f6460133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f646508c700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646009d630 0x7f6460133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f646508c700 1 -- 172.21.15.32:0/2003883265 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c009550 con 0x7f6460138d80 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f646508c700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 0x7f64601341b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f645800ef90 tx=0x7f645800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800cc30 con 0x7f6460138d80 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645800f040 con 0x7f6460138d80 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64600a0410 con 0x7f6460138d80 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458014480 con 0x7f6460138d80 2023-09-30T22:28:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.019+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6460137680 con 0x7f6460138d80 2023-09-30T22:28:59.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.020+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64580145e0 con 0x7f6460138d80 2023-09-30T22:28:59.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.020+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6460136530 con 0x7f6460138d80 2023-09-30T22:28:59.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.020+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6458040270 con 0x7f6460138d80 2023-09-30T22:28:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.030+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6458007920 con 0x7f6460138d80 2023-09-30T22:28:59.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.149+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a48122210 con 0x7f5a4809e1d0 2023-09-30T22:28:59.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.149+0000 7f5a36ffd700 1 -- 172.21.15.32:0/3031869148 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a3c015020 con 0x7f5a4809e1d0 2023-09-30T22:28:59.150 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.151+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5a4809e1d0 msgr2=0x7f5a481c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.151+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5a4809e1d0 0x7f5a481c89a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c016000 tx=0x7f5a3c004750 comp rx=0 tx=0).stop 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 shutdown_connections 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a48139170 0x7f5a481c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a4809d6f0 0x7f5a481c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 --2- 172.21.15.32:0/3031869148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5a4809e1d0 0x7f5a481c89a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 >> 172.21.15.32:0/3031869148 conn(0x7f5a48066f50 msgr2=0x7f5a48136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 shutdown_connections 2023-09-30T22:28:59.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.152+0000 7f5a4cdea700 1 -- 172.21.15.32:0/3031869148 wait complete. 2023-09-30T22:28:59.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:28:59.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.198+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4f840747f0 con 0x7f4f8409e850 2023-09-30T22:28:59.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.199+0000 7f4f6b7fe700 1 -- 172.21.15.32:0/2759029120 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f74021070 con 0x7f4f8409e850 2023-09-30T22:28:59.200 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:28:59.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8409e850 msgr2=0x7f4f84122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8409e850 0x7f4f84122900 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400e9b0 tx=0x7f4f7400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 shutdown_connections 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f84134290 0x7f4f84123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f8409e850 0x7f4f84122900 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 --2- 172.21.15.32:0/2759029120 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f8409f160 0x7f4f84122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 >> 172.21.15.32:0/2759029120 conn(0x7f4f840a3f70 msgr2=0x7f4f84070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 shutdown_connections 2023-09-30T22:28:59.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.201+0000 7f4f8905f700 1 -- 172.21.15.32:0/2759029120 wait complete. 2023-09-30T22:28:59.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:28:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.241+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64600747f0 con 0x7f6460138d80 2023-09-30T22:28:59.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.241+0000 7f645d7fa700 1 -- 172.21.15.32:0/2003883265 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6458009ca0 con 0x7f6460138d80 2023-09-30T22:28:59.243 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:28:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 msgr2=0x7f64601341b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 0x7f64601341b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f645800ef90 tx=0x7f645800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 shutdown_connections 2023-09-30T22:28:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6460138d80 0x7f64601341b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f646009d630 0x7f6460133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 --2- 172.21.15.32:0/2003883265 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64600a0a30 0x7f6460133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 >> 172.21.15.32:0/2003883265 conn(0x7f64600a3050 msgr2=0x7f646009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 shutdown_connections 2023-09-30T22:28:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.244+0000 7f6466aef700 1 -- 172.21.15.32:0/2003883265 wait complete. 2023-09-30T22:28:59.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:28:59.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:28:59.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.553+0000 7ff3ce1fb700 1 Processor -- start 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.553+0000 7ff3ce1fb700 1 -- start start 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3ce1fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809e930 0x7ff3c809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3ce1fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f650 0x7ff3c8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3ce1fb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c8070940 con 0x7ff3c809f240 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3ce1fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c806fd60 con 0x7ff3c809e930 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3ce1fb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c806fa80 con 0x7ff3c809f650 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f650 0x7ff3c8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809e930 0x7ff3c809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3c809f240 0x7ff3c81317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54520/0 (socket says 172.21.15.32:54520) 2023-09-30T22:28:59.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c6ffd700 1 -- 172.21.15.32:0/2274832750 learned_addr learned my addr 172.21.15.32:0/2274832750 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c7fff700 1 -- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809e930 msgr2=0x7ff3c809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c7fff700 1 --2- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809e930 0x7ff3c809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c7fff700 1 -- 172.21.15.32:0/2274832750 >> v1:172.21.15.32:6789/0 conn(0x7ff3c809f240 legacy=0x7ff3c81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c67fc700 1 -- 172.21.15.32:0/2274832750 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819249224 0 0) 0x7ff3c8070940 con 0x7ff3c809f240 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c7fff700 1 -- 172.21.15.32:0/2274832750 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c809fb60 con 0x7ff3c809f650 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c77fe700 1 --2- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809e930 0x7ff3c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.554+0000 7ff3c7fff700 1 --2- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f650 0x7ff3c8131f00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009dc0 tx=0x7ff3c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c94810540f5050 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3c67fc700 1 -- 172.21.15.32:0/2274832750 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000e070 con 0x7ff3c809f650 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3c67fc700 1 -- 172.21.15.32:0/2274832750 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c000bb20 con 0x7ff3c809f650 2023-09-30T22:28:59.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3c67fc700 1 -- 172.21.15.32:0/2274832750 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0012430 con 0x7ff3c809f650 2023-09-30T22:28:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f650 msgr2=0x7ff3c8131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f650 0x7ff3c8131f00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009dc0 tx=0x7ff3c000b330 comp rx=0 tx=0).stop 2023-09-30T22:28:59.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/2274832750 shutdown_connections 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f650 0x7ff3c8131f00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/2274832750 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809e930 0x7ff3c809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/2274832750 >> 172.21.15.32:0/2274832750 conn(0x7ff3c80a4210 msgr2=0x7ff3c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/2274832750 shutdown_connections 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.555+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/2274832750 wait complete. 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.556+0000 7ff3ce1fb700 1 Processor -- start 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 -- start start 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c809e930 0x7ff3c81a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 0x7ff3c81a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809f650 0x7ff3c81a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a6cf0 con 0x7ff3c809e930 2023-09-30T22:28:59.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a53b0 con 0x7ff3c809f650 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3ce1fb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81a6f40 con 0x7ff3c809f240 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 0x7ff3c81a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809f650 0x7ff3c81a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 0x7ff3c81a3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50846/0 (socket says 172.21.15.32:50846) 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 -- 172.21.15.32:0/3060355256 learned_addr learned my addr 172.21.15.32:0/3060355256 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c77fe700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c809e930 0x7ff3c81a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 -- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809f650 msgr2=0x7ff3c81a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809f650 0x7ff3c81a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 -- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c809e930 msgr2=0x7ff3c81a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.557+0000 7ff3c6ffd700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c809e930 0x7ff3c81a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3c6ffd700 1 -- 172.21.15.32:0/3060355256 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80094a0 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3c6ffd700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 0x7ff3c81a3ce0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c520 tx=0x7ff3b000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0004c80 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b0004de0 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c0009a70 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0007310 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.558+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c80a0190 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.559+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff3b0007470 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.559+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff3b003f260 con 0x7ff3c809f240 2023-09-30T22:28:59.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.559+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c81a5b60 con 0x7ff3c809f240 2023-09-30T22:28:59.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.564+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3b00123b0 con 0x7ff3c809f240 2023-09-30T22:28:59.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.770+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff3c8074ea0 con 0x7ff3c809f240 2023-09-30T22:28:59.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.770+0000 7ff3bffff700 1 -- 172.21.15.32:0/3060355256 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3b00102c0 con 0x7ff3c809f240 2023-09-30T22:28:59.774 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.772+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 msgr2=0x7ff3c81a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.772+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 0x7ff3c81a3ce0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c520 tx=0x7ff3b000c830 comp rx=0 tx=0).stop 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.772+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 shutdown_connections 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.772+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3c809f650 0x7ff3c81a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.772+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3c809e930 0x7ff3c81a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.772+0000 7ff3ce1fb700 1 --2- 172.21.15.32:0/3060355256 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff3c809f240 0x7ff3c81a3ce0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.773+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 >> 172.21.15.32:0/3060355256 conn(0x7ff3c80a4210 msgr2=0x7ff3c81bbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:28:59.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.773+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 shutdown_connections 2023-09-30T22:28:59.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:28:59.773+0000 7ff3ce1fb700 1 -- 172.21.15.32:0/3060355256 wait complete. 2023-09-30T22:28:59.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:00.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:00.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:00.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.317+0000 7fa0b2e60700 1 Processor -- start 2023-09-30T22:29:00.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.317+0000 7fa0b2e60700 1 -- start start 2023-09-30T22:29:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b2e60700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0ac0a0a30 0x7fa0ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b2e60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 0x7fa0ac133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b2e60700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac09edd0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b2e60700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac09e1f0 con 0x7fa0ac0a0a30 2023-09-30T22:29:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b2e60700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac09dc50 con 0x7fa0ac0a1340 2023-09-30T22:29:00.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 0x7fa0ac133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b13fd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa0ac133fa0 0x7fa0ac13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54534/0 (socket says 172.21.15.32:54534) 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 0x7fa0ac133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38444/0 (socket says 172.21.15.32:38444) 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b13fd700 1 -- 172.21.15.32:0/1146032838 learned_addr learned my addr 172.21.15.32:0/1146032838 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0b0bfc700 1 --2- 172.21.15.32:0/1146032838 >> v2:172.21.15.32:3300/0 conn(0x7fa0ac0a0a30 0x7fa0ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0ab7fe700 1 -- 172.21.15.32:0/1146032838 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364856967 0 0) 0x7fa0ac09edd0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.318+0000 7fa0abfff700 1 -- 172.21.15.32:0/1146032838 >> v2:172.21.15.32:3300/0 conn(0x7fa0ac0a0a30 msgr2=0x7fa0ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0abfff700 1 --2- 172.21.15.32:0/1146032838 >> v2:172.21.15.32:3300/0 conn(0x7fa0ac0a0a30 0x7fa0ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0abfff700 1 -- 172.21.15.32:0/1146032838 >> v1:172.21.15.32:6789/0 conn(0x7fa0ac133fa0 legacy=0x7fa0ac13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0abfff700 1 -- 172.21.15.32:0/1146032838 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0a1fe0 con 0x7fa0ac0a1340 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0b0bfc700 1 --2- 172.21.15.32:0/1146032838 >> v2:172.21.15.32:3300/0 conn(0x7fa0ac0a0a30 0x7fa0ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0abfff700 1 --2- 172.21.15.32:0/1146032838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 0x7fa0ac133a60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa098009840 tx=0x7fa098009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53372a1ac87f58c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0ab7fe700 1 -- 172.21.15.32:0/1146032838 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098011830 con 0x7fa0ac0a1340 2023-09-30T22:29:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0ab7fe700 1 -- 172.21.15.32:0/1146032838 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098011990 con 0x7fa0ac0a1340 2023-09-30T22:29:00.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.319+0000 7fa0ab7fe700 1 -- 172.21.15.32:0/1146032838 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098011d20 con 0x7fa0ac0a1340 2023-09-30T22:29:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 -- 172.21.15.32:0/1146032838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 msgr2=0x7fa0ac133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/1146032838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 0x7fa0ac133a60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa098009840 tx=0x7fa098009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 -- 172.21.15.32:0/1146032838 shutdown_connections 2023-09-30T22:29:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/1146032838 >> v2:172.21.15.32:3300/0 conn(0x7fa0ac0a0a30 0x7fa0ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/1146032838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac0a1340 0x7fa0ac133a60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 -- 172.21.15.32:0/1146032838 >> 172.21.15.32:0/1146032838 conn(0x7fa0ac0a6550 msgr2=0x7fa0ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 -- 172.21.15.32:0/1146032838 shutdown_connections 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.320+0000 7fa0b2e60700 1 -- 172.21.15.32:0/1146032838 wait complete. 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.321+0000 7fa0b2e60700 1 Processor -- start 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.321+0000 7fa0b2e60700 1 -- start start 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b2e60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0ac0a0a30 0x7fa0ac11e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b2e60700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 0x7fa0ac11ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b2e60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac133fa0 0x7fa0ac11f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b2e60700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac120200 con 0x7fa0ac0a1340 2023-09-30T22:29:00.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b2e60700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac0a0220 con 0x7fa0ac133fa0 2023-09-30T22:29:00.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b2e60700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac133580 con 0x7fa0ac0a0a30 2023-09-30T22:29:00.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 0x7fa0ac11ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 0x7fa0ac11ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43532/0 (socket says 172.21.15.32:43532) 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0abfff700 1 -- 172.21.15.32:0/3192419902 learned_addr learned my addr 172.21.15.32:0/3192419902 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b0bfc700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0ac0a0a30 0x7fa0ac11e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.322+0000 7fa0b13fd700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac133fa0 0x7fa0ac11f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b13fd700 1 -- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0ac0a0a30 msgr2=0x7fa0ac11e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b13fd700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0ac0a0a30 0x7fa0ac11e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b13fd700 1 -- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 msgr2=0x7fa0ac11ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b13fd700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 0x7fa0ac11ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b13fd700 1 -- 172.21.15.32:0/3192419902 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00094a0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0abfff700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 0x7fa0ac11ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b0bfc700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0ac0a0a30 0x7fa0ac11e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b13fd700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac133fa0 0x7fa0ac11f110 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400cbe0 tx=0x7fa0a400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400aa40 con 0x7fa0ac133fa0 2023-09-30T22:29:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a400abc0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40070d0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0980094a0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.323+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac0a35d0 con 0x7fa0ac133fa0 2023-09-30T22:29:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.324+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa0a4007230 con 0x7fa0ac133fa0 2023-09-30T22:29:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.324+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac120a90 con 0x7fa0ac133fa0 2023-09-30T22:29:00.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.325+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa0a403f020 con 0x7fa0ac133fa0 2023-09-30T22:29:00.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.329+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0ac120a90 con 0x7fa0ac133fa0 2023-09-30T22:29:00.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 Processor -- start 2023-09-30T22:29:00.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 -- start start 2023-09-30T22:29:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 0x7fc0140a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc014138d80 0x7fc01413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0140a2a70 con 0x7fc0140a0a30 2023-09-30T22:29:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140a01d0 con 0x7fc01409d630 2023-09-30T22:29:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc01993d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc01409e2b0 con 0x7fc014138d80 2023-09-30T22:29:00.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc00a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 0x7fc0140a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc00a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 0x7fc0140a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38450/0 (socket says 172.21.15.32:38450) 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc00a7fc700 1 -- 172.21.15.32:0/1268890752 learned_addr learned my addr 172.21.15.32:0/1268890752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.362+0000 7fc0137fe700 1 --2- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc014138d80 0x7fc01413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc00a7fc700 1 -- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc014138d80 msgr2=0x7fc01413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc00a7fc700 1 --2- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc014138d80 0x7fc01413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc00a7fc700 1 -- 172.21.15.32:0/1268890752 >> v1:172.21.15.32:6789/0 conn(0x7fc0140a0a30 legacy=0x7fc0140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc00a7fc700 1 -- 172.21.15.32:0/1268890752 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01409dc50 con 0x7fc01409d630 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc0127fc700 1 -- 172.21.15.32:0/1268890752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261357999 0 0) 0x7fc0140a2a70 con 0x7fc0140a0a30 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc00a7fc700 1 --2- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 0x7fc0140a1510 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009920 tx=0x7fbffc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5e65b255579427 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc0127fc700 1 -- 172.21.15.32:0/1268890752 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011830 con 0x7fc01409d630 2023-09-30T22:29:00.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc0127fc700 1 -- 172.21.15.32:0/1268890752 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc011990 con 0x7fc01409d630 2023-09-30T22:29:00.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.363+0000 7fc0127fc700 1 -- 172.21.15.32:0/1268890752 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011d20 con 0x7fc01409d630 2023-09-30T22:29:00.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 -- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 msgr2=0x7fc0140a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 --2- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 0x7fc0140a1510 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009920 tx=0x7fbffc009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 -- 172.21.15.32:0/1268890752 shutdown_connections 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 --2- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc014138d80 0x7fc01413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 --2- 172.21.15.32:0/1268890752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc01409d630 0x7fc0140a1510 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 -- 172.21.15.32:0/1268890752 >> 172.21.15.32:0/1268890752 conn(0x7fc0140a3050 msgr2=0x7fc0140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 -- 172.21.15.32:0/1268890752 shutdown_connections 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.364+0000 7fc01993d700 1 -- 172.21.15.32:0/1268890752 wait complete. 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.365+0000 7fc01993d700 1 Processor -- start 2023-09-30T22:29:00.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.366+0000 7fc01993d700 1 -- start start 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.366+0000 7fc01993d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc01409d630 0x7fc0141337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc01993d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 0x7fc014133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc01993d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc014138d80 0x7fc014134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc01993d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc014134c10 con 0x7fc014138d80 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc01993d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141365f0 con 0x7fc0140a0a30 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc01993d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc014136f60 con 0x7fc01409d630 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 0x7fc014133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 0x7fc014133cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:38466/0 (socket says 172.21.15.32:38466) 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 -- 172.21.15.32:0/2493611142 learned_addr learned my addr 172.21.15.32:0/2493611142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc0137fe700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc014138d80 0x7fc014134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc012ffd700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc01409d630 0x7fc0141337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 -- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc01409d630 msgr2=0x7fc0141337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc01409d630 0x7fc0141337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 -- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc014138d80 msgr2=0x7fc014134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc014138d80 0x7fc014134230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 -- 172.21.15.32:0/2493611142 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009550 con 0x7fc0140a0a30 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc00a7fc700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 0x7fc014133cf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00a010 tx=0x7fbffc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.367+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc004240 con 0x7fc0140a0a30 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.368+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc01f070 con 0x7fc0140a0a30 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.368+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0140a4b70 con 0x7fc0140a0a30 2023-09-30T22:29:00.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.368+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0036a0 con 0x7fc0140a0a30 2023-09-30T22:29:00.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.368+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0141379d0 con 0x7fc0140a0a30 2023-09-30T22:29:00.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.369+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbffc021030 con 0x7fc0140a0a30 2023-09-30T22:29:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.369+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0140747f0 con 0x7fc0140a0a30 2023-09-30T22:29:00.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.369+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbffc040730 con 0x7fc0140a0a30 2023-09-30T22:29:00.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.378+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbffc0197c0 con 0x7fc0140a0a30 2023-09-30T22:29:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.408+0000 7fc02682b700 1 Processor -- start 2023-09-30T22:29:00.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 -- start start 2023-09-30T22:29:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc02009d6f0 0x7fc02009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e000 0x7fc020130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e410 0x7fc020130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02009f520 con 0x7fc02009d6f0 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02009f130 con 0x7fc02009e000 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.409+0000 7fc02682b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc02009e930 con 0x7fc02009e410 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc02009d6f0 0x7fc02009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc02009d6f0 0x7fc02009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43560/0 (socket says 172.21.15.32:43560) 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 -- 172.21.15.32:0/3175280886 learned_addr learned my addr 172.21.15.32:0/3175280886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01f7fe700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e000 0x7fc020130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc024dc8700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e410 0x7fc020130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 -- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e410 msgr2=0x7fc020130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e410 0x7fc020130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 -- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e000 msgr2=0x7fc020130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e000 0x7fc020130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 -- 172.21.15.32:0/3175280886 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02009b2d0 con 0x7fc02009d6f0 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01f7fe700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e000 0x7fc020130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01ffff700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02009dac0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc01000bbc0 tx=0x7fc01000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0acdeb1881d5a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.410+0000 7fc01effd700 1 -- 172.21.15.32:0/3175280886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000dcf0 con 0x7fc02009d6f0 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc01effd700 1 -- 172.21.15.32:0/3175280886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01000de50 con 0x7fc02009d6f0 2023-09-30T22:29:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc01effd700 1 -- 172.21.15.32:0/3175280886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010010690 con 0x7fc02009d6f0 2023-09-30T22:29:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 -- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 msgr2=0x7fc02009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02009dac0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fc01000bbc0 tx=0x7fc01000bed0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 -- 172.21.15.32:0/3175280886 shutdown_connections 2023-09-30T22:29:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e410 0x7fc020130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02009dac0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 --2- 172.21.15.32:0/3175280886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e000 0x7fc020130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 -- 172.21.15.32:0/3175280886 >> 172.21.15.32:0/3175280886 conn(0x7fc020066f50 msgr2=0x7fc02009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 -- 172.21.15.32:0/3175280886 shutdown_connections 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.411+0000 7fc02682b700 1 -- 172.21.15.32:0/3175280886 wait complete. 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.412+0000 7fc02682b700 1 Processor -- start 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 -- start start 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02011e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e000 0x7fc02011e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e410 0x7fc02011eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02011ff30 con 0x7fc02009d6f0 2023-09-30T22:29:00.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc020120550 con 0x7fc02009e410 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc02682b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc020130280 con 0x7fc02009e000 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc01ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02011e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc01ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02011e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43562/0 (socket says 172.21.15.32:43562) 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc01ffff700 1 -- 172.21.15.32:0/1415072775 learned_addr learned my addr 172.21.15.32:0/1415072775 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc01f7fe700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e000 0x7fc02011e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.413+0000 7fc024dc8700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e410 0x7fc02011eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01ffff700 1 -- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e000 msgr2=0x7fc02011e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01ffff700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e000 0x7fc02011e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01ffff700 1 -- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e410 msgr2=0x7fc02011eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01ffff700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e410 0x7fc02011eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01ffff700 1 -- 172.21.15.32:0/1415072775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0140094a0 con 0x7fc02009d6f0 2023-09-30T22:29:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc024dc8700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e410 0x7fc02011eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01ffff700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02011e320 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc02009d470 tx=0x7fc010004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01001f070 con 0x7fc02009d6f0 2023-09-30T22:29:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0100038b0 con 0x7fc02009d6f0 2023-09-30T22:29:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01000b7f0 con 0x7fc02009d6f0 2023-09-30T22:29:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010003c40 con 0x7fc02009d6f0 2023-09-30T22:29:00.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.414+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02009c700 con 0x7fc02009d6f0 2023-09-30T22:29:00.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.415+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc010019400 con 0x7fc02009d6f0 2023-09-30T22:29:00.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.415+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc010042f60 con 0x7fc02009d6f0 2023-09-30T22:29:00.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.416+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc020074ea0 con 0x7fc02009d6f0 2023-09-30T22:29:00.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.425+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc01002c3d0 con 0x7fc02009d6f0 2023-09-30T22:29:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.560+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa0ac120a90 con 0x7fa0ac133fa0 2023-09-30T22:29:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.560+0000 7fa0a97fa700 1 -- 172.21.15.32:0/3192419902 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0ac120a90 con 0x7fa0ac133fa0 2023-09-30T22:29:00.562 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.562+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac133fa0 msgr2=0x7fa0ac11f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.562+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac133fa0 0x7fa0ac11f110 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400cbe0 tx=0x7fa0a400cef0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 shutdown_connections 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0ac133fa0 0x7fa0ac11f110 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0ac0a0a30 0x7fa0ac11e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 --2- 172.21.15.32:0/3192419902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0ac0a1340 0x7fa0ac11ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 >> 172.21.15.32:0/3192419902 conn(0x7fa0ac0a6550 msgr2=0x7fa0ac1b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 shutdown_connections 2023-09-30T22:29:00.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.563+0000 7fa0b2e60700 1 -- 172.21.15.32:0/3192419902 wait complete. 2023-09-30T22:29:00.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.601+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc014072980 con 0x7fc0140a0a30 2023-09-30T22:29:00.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.601+0000 7fc010ff9700 1 -- 172.21.15.32:0/2493611142 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbffc015070 con 0x7fc0140a0a30 2023-09-30T22:29:00.603 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 msgr2=0x7fc014133cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 0x7fc014133cf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00a010 tx=0x7fbffc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 shutdown_connections 2023-09-30T22:29:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc014138d80 0x7fc014134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc01409d630 0x7fc0141337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 --2- 172.21.15.32:0/2493611142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0140a0a30 0x7fc014133cf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 >> 172.21.15.32:0/2493611142 conn(0x7fc0140a3050 msgr2=0x7fc01409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 shutdown_connections 2023-09-30T22:29:00.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.604+0000 7fc01993d700 1 -- 172.21.15.32:0/2493611142 wait complete. 2023-09-30T22:29:00.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:00.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.651+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc0200720e0 con 0x7fc02009d6f0 2023-09-30T22:29:00.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.651+0000 7fc01cff9700 1 -- 172.21.15.32:0/1415072775 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc010022370 con 0x7fc02009d6f0 2023-09-30T22:29:00.652 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 msgr2=0x7fc02011e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02011e320 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc02009d470 tx=0x7fc010004750 comp rx=0 tx=0).stop 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 shutdown_connections 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc02009e410 0x7fc02011eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc02009d6f0 0x7fc02011e320 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 --2- 172.21.15.32:0/1415072775 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc02009e000 0x7fc02011e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 >> 172.21.15.32:0/1415072775 conn(0x7fc020066f50 msgr2=0x7fc0200a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 shutdown_connections 2023-09-30T22:29:00.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.653+0000 7fc02682b700 1 -- 172.21.15.32:0/1415072775 wait complete. 2023-09-30T22:29:00.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:00.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.932+0000 7f990935e700 1 Processor -- start 2023-09-30T22:29:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.932+0000 7f990935e700 1 -- start start 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f990935e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 0x7f99040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f990935e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 0x7f99040a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f990935e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 0x7f9904133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f990935e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040a2a70 con 0x7f9904133930 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f990935e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040a01d0 con 0x7f990409d630 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f990935e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f990409e2b0 con 0x7f99040a0a30 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f9902ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 0x7f99040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99037fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 0x7f9904133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 0x7f99040a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 0x7f99040a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39752/0 (socket says 172.21.15.32:39752) 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f9902ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 0x7f99040a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37772/0 (socket says 172.21.15.32:37772) 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99037fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 0x7f9904133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56184/0 (socket says 172.21.15.32:56184) 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 -- 172.21.15.32:0/2296980660 learned_addr learned my addr 172.21.15.32:0/2296980660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 -- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 msgr2=0x7f99040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 --2- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 0x7f99040a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 -- 172.21.15.32:0/2296980660 >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 msgr2=0x7f9904133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 --2- 172.21.15.32:0/2296980660 >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 0x7f9904133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 -- 172.21.15.32:0/2296980660 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990409dc50 con 0x7f990409d630 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f9902ffd700 1 --2- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 0x7f99040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99037fe700 1 --2- 172.21.15.32:0/2296980660 >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 0x7f9904133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.933+0000 7f99027fc700 1 --2- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 0x7f99040a1340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f98f80098f0 tx=0x7f98f8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772c9bb6c6956a63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f9901ffb700 1 -- 172.21.15.32:0/2296980660 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8011830 con 0x7f990409d630 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f9901ffb700 1 -- 172.21.15.32:0/2296980660 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f8011990 con 0x7f990409d630 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f9901ffb700 1 -- 172.21.15.32:0/2296980660 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8011d20 con 0x7f990409d630 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 -- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 msgr2=0x7f99040a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 --2- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 0x7f99040a1340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f98f80098f0 tx=0x7f98f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 -- 172.21.15.32:0/2296980660 shutdown_connections 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 --2- 172.21.15.32:0/2296980660 >> v2:172.21.15.32:3300/0 conn(0x7f9904133930 0x7f9904133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 --2- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99040a0a30 0x7f99040a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 --2- 172.21.15.32:0/2296980660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f990409d630 0x7f99040a1340 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 -- 172.21.15.32:0/2296980660 >> 172.21.15.32:0/2296980660 conn(0x7f99040a3050 msgr2=0x7f99040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:00.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 -- 172.21.15.32:0/2296980660 shutdown_connections 2023-09-30T22:29:00.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.934+0000 7f990935e700 1 -- 172.21.15.32:0/2296980660 wait complete. 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.935+0000 7f990935e700 1 Processor -- start 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 -- start start 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409d630 0x7f990411f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99040a0a30 0x7f990411fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 0x7f99041c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040a0410 con 0x7f990409d630 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9904122640 con 0x7f99040a0a30 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f990935e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f990411ef20 con 0x7f9904133930 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f99037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 0x7f99041c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f99027fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99040a0a30 0x7f990411fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f99037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 0x7f99041c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37782/0 (socket says 172.21.15.32:37782) 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f9902ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409d630 0x7f990411f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f99037fe700 1 -- 172.21.15.32:0/207415925 learned_addr learned my addr 172.21.15.32:0/207415925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f9902ffd700 1 -- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 msgr2=0x7f99041c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f9902ffd700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 0x7f99041c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f9902ffd700 1 -- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99040a0a30 msgr2=0x7f990411fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f9902ffd700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99040a0a30 0x7f990411fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f9902ffd700 1 -- 172.21.15.32:0/207415925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40094a0 con 0x7f990409d630 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.936+0000 7f99037fe700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 0x7f99041c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.937+0000 7f9902ffd700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409d630 0x7f990411f4c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e9e0 tx=0x7f98f400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.937+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400ee80 con 0x7f990409d630 2023-09-30T22:29:00.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.937+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400c850 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.937+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc009a70 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.937+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400cc00 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.937+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f8009550 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.938+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98f400cd60 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.938+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f98f403fa90 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.938+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99041220c0 con 0x7f990409d630 2023-09-30T22:29:00.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:00.943+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99041220c0 con 0x7f990409d630 2023-09-30T22:29:01.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.146+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f99041220c0 con 0x7f990409d630 2023-09-30T22:29:01.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.146+0000 7f98eb7fe700 1 -- 172.21.15.32:0/207415925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99041220c0 con 0x7f990409d630 2023-09-30T22:29:01.147 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409d630 msgr2=0x7f990411f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409d630 0x7f990411f4c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e9e0 tx=0x7f98f400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 shutdown_connections 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9904133930 0x7f99041c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990409d630 0x7f990411f4c0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 --2- 172.21.15.32:0/207415925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99040a0a30 0x7f990411fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 >> 172.21.15.32:0/207415925 conn(0x7f99040a3050 msgr2=0x7f990409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 shutdown_connections 2023-09-30T22:29:01.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.149+0000 7f990935e700 1 -- 172.21.15.32:0/207415925 wait complete. 2023-09-30T22:29:01.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:01.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:01.624 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:01.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:01.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.728+0000 7f39fee77700 1 Processor -- start 2023-09-30T22:29:01.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.728+0000 7f39fee77700 1 -- start start 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.729+0000 7f39fee77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39f81338c0 0x7f39f8133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.729+0000 7f39fee77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 0x7f39f809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.729+0000 7f39fee77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f809e390 0x7f39f809e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.729+0000 7f39fee77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f8070c80 con 0x7f39f81338c0 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.729+0000 7f39fee77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f809ed20 con 0x7f39f809e390 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.729+0000 7f39fee77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f806f720 con 0x7f39f81341d0 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39fcc13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39f81338c0 0x7f39f8133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 0x7f39f809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39fcc13700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f39f81338c0 0x7f39f8133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56210/0 (socket says 172.21.15.32:56210) 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 0x7f39f809de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37788/0 (socket says 172.21.15.32:37788) 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39fcc13700 1 -- 172.21.15.32:0/3628314220 learned_addr learned my addr 172.21.15.32:0/3628314220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39fd414700 1 --2- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f809e390 0x7f39f809e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 -- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f809e390 msgr2=0x7f39f809e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 --2- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f809e390 0x7f39f809e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 -- 172.21.15.32:0/3628314220 >> v2:172.21.15.32:3300/0 conn(0x7f39f81338c0 msgr2=0x7f39f8133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 --2- 172.21.15.32:0/3628314220 >> v2:172.21.15.32:3300/0 conn(0x7f39f81338c0 0x7f39f8133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 -- 172.21.15.32:0/3628314220 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f806fce0 con 0x7f39f81341d0 2023-09-30T22:29:01.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f7fff700 1 --2- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 0x7f39f809de50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009870 tx=0x7f39e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee0799bab1e0a5b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f77fe700 1 -- 172.21.15.32:0/3628314220 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4011830 con 0x7f39f81341d0 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f77fe700 1 -- 172.21.15.32:0/3628314220 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4011990 con 0x7f39f81341d0 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.730+0000 7f39f77fe700 1 -- 172.21.15.32:0/3628314220 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4011d20 con 0x7f39f81341d0 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.731+0000 7f39fee77700 1 -- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 msgr2=0x7f39f809de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.731+0000 7f39fee77700 1 --2- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 0x7f39f809de50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009870 tx=0x7f39e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 -- 172.21.15.32:0/3628314220 shutdown_connections 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 --2- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f809e390 0x7f39f809e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 --2- 172.21.15.32:0/3628314220 >> v2:172.21.15.32:3300/0 conn(0x7f39f81338c0 0x7f39f8133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 --2- 172.21.15.32:0/3628314220 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f81341d0 0x7f39f809de50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 -- 172.21.15.32:0/3628314220 >> 172.21.15.32:0/3628314220 conn(0x7f39f80a2aa0 msgr2=0x7f39f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 -- 172.21.15.32:0/3628314220 shutdown_connections 2023-09-30T22:29:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.732+0000 7f39fee77700 1 -- 172.21.15.32:0/3628314220 wait complete. 2023-09-30T22:29:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 Processor -- start 2023-09-30T22:29:01.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 -- start start 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 0x7f39f8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f81338c0 0x7f39f8119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39f81341d0 0x7f39f8119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f811a510 con 0x7f39f81341d0 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8130d70 con 0x7f39f81338c0 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.733+0000 7f39fee77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f811abf0 con 0x7f39f809e390 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 0x7f39f8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 0x7f39f8118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37804/0 (socket says 172.21.15.32:37804) 2023-09-30T22:29:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 -- 172.21.15.32:0/1919508593 learned_addr learned my addr 172.21.15.32:0/1919508593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39f7fff700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f81338c0 0x7f39f8119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fd414700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39f81341d0 0x7f39f8119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 -- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f81338c0 msgr2=0x7f39f8119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f81338c0 0x7f39f8119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 -- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39f81341d0 msgr2=0x7f39f8119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39f81341d0 0x7f39f8119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 -- 172.21.15.32:0/1919508593 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40094a0 con 0x7f39f809e390 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fcc13700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 0x7f39f8118cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e910 tx=0x7f39ec00ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0096a0 con 0x7f39f809e390 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec013680 con 0x7f39f809e390 2023-09-30T22:29:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f806f960 con 0x7f39f809e390 2023-09-30T22:29:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.734+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec013a30 con 0x7f39f809e390 2023-09-30T22:29:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.735+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f811caa0 con 0x7f39f809e390 2023-09-30T22:29:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.735+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f39ec013b90 con 0x7f39f809e390 2023-09-30T22:29:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.736+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f39ec03f9f0 con 0x7f39f809e390 2023-09-30T22:29:01.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.736+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f806e670 con 0x7f39f809e390 2023-09-30T22:29:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.741+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39ec017080 con 0x7f39f809e390 2023-09-30T22:29:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.768+0000 7f8487df2700 1 Processor -- start 2023-09-30T22:29:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 -- start start 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009d6f0 0x7f848009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 0x7f84801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f848009e410 0x7f8480133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848006b460 con 0x7f848009e410 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848009b120 con 0x7f848009d6f0 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8487df2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f848009a7e0 con 0x7f848009e000 2023-09-30T22:29:01.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f848538d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 0x7f84801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f848638f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f848009e410 0x7f8480133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f848538d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 0x7f84801339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37810/0 (socket says 172.21.15.32:37810) 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f848538d700 1 -- 172.21.15.32:0/2527406062 learned_addr learned my addr 172.21.15.32:0/2527406062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.769+0000 7f8485b8e700 1 --2- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009d6f0 0x7f848009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f848538d700 1 -- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009d6f0 msgr2=0x7f848009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f848538d700 1 --2- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009d6f0 0x7f848009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f848538d700 1 -- 172.21.15.32:0/2527406062 >> v2:172.21.15.32:3300/0 conn(0x7f848009e410 msgr2=0x7f8480133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f848538d700 1 --2- 172.21.15.32:0/2527406062 >> v2:172.21.15.32:3300/0 conn(0x7f848009e410 0x7f8480133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f848538d700 1 -- 172.21.15.32:0/2527406062 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848009a500 con 0x7f848009e000 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f848538d700 1 --2- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 0x7f84801339f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8470009920 tx=0x7f8470009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0259710272ea317 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:01.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f8484b8c700 1 -- 172.21.15.32:0/2527406062 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470011830 con 0x7f848009e000 2023-09-30T22:29:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f8484b8c700 1 -- 172.21.15.32:0/2527406062 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470011990 con 0x7f848009e000 2023-09-30T22:29:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.770+0000 7f8484b8c700 1 -- 172.21.15.32:0/2527406062 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470011d20 con 0x7f848009e000 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 -- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 msgr2=0x7f84801339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 --2- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 0x7f84801339f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8470009920 tx=0x7f8470009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 -- 172.21.15.32:0/2527406062 shutdown_connections 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 --2- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009d6f0 0x7f848009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 --2- 172.21.15.32:0/2527406062 >> v2:172.21.15.32:3300/0 conn(0x7f848009e410 0x7f8480133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 --2- 172.21.15.32:0/2527406062 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009e000 0x7f84801339f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 -- 172.21.15.32:0/2527406062 >> 172.21.15.32:0/2527406062 conn(0x7f8480066f50 msgr2=0x7f848009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 -- 172.21.15.32:0/2527406062 shutdown_connections 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.771+0000 7f8487df2700 1 -- 172.21.15.32:0/2527406062 wait complete. 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.772+0000 7f8487df2700 1 Processor -- start 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.772+0000 7f8487df2700 1 -- start start 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8487df2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 0x7f84801a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8487df2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848009e000 0x7f84801a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8487df2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009e410 0x7f84801a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8487df2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84801a92a0 con 0x7f848009e000 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8487df2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84801aa3f0 con 0x7f848009e410 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8487df2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84801a8d70 con 0x7f848009d6f0 2023-09-30T22:29:01.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 0x7f84801a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f848538d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848009e000 0x7f84801a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 0x7f84801a6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37812/0 (socket says 172.21.15.32:37812) 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 -- 172.21.15.32:0/3017161293 learned_addr learned my addr 172.21.15.32:0/3017161293 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f848638f700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009e410 0x7f84801a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 -- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009e410 msgr2=0x7f84801a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009e410 0x7f84801a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 -- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848009e000 msgr2=0x7f84801a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848009e000 0x7f84801a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f8485b8e700 1 -- 172.21.15.32:0/3017161293 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009550 con 0x7f848009d6f0 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.773+0000 7f848538d700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848009e000 0x7f84801a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.774+0000 7f8485b8e700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 0x7f84801a6060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f847c00e9e0 tx=0x7f847c00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.774+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c009710 con 0x7f848009d6f0 2023-09-30T22:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.774+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c00ee80 con 0x7f848009d6f0 2023-09-30T22:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.774+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84801a9750 con 0x7f848009d6f0 2023-09-30T22:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.774+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c00ca50 con 0x7f848009d6f0 2023-09-30T22:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.774+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848009c710 con 0x7f848009d6f0 2023-09-30T22:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.775+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f847c00cbb0 con 0x7f848009d6f0 2023-09-30T22:29:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.775+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8480074ea0 con 0x7f848009d6f0 2023-09-30T22:29:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.775+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f847c03f6b0 con 0x7f848009d6f0 2023-09-30T22:29:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.780+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f847c0232f0 con 0x7f848009d6f0 2023-09-30T22:29:01.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.812+0000 7f23fbca0700 1 Processor -- start 2023-09-30T22:29:01.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fbca0700 1 -- start start 2023-09-30T22:29:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fbca0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fbca0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f4138d80 0x7f23f413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fbca0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f40a2a70 con 0x7f23f40a0a30 2023-09-30T22:29:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fbca0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f40a01d0 con 0x7f23f4138d80 2023-09-30T22:29:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fbca0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23f409e2b0 con 0x7f23f409d630 2023-09-30T22:29:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23f923b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23f923b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f40a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37816/0 (socket says 172.21.15.32:37816) 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23f923b700 1 -- 172.21.15.32:0/866480664 learned_addr learned my addr 172.21.15.32:0/866480664 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.813+0000 7f23fa23d700 1 --2- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f4138d80 0x7f23f413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f923b700 1 -- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f4138d80 msgr2=0x7f23f413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f923b700 1 --2- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f4138d80 0x7f23f413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f923b700 1 -- 172.21.15.32:0/866480664 >> v1:172.21.15.32:6789/0 conn(0x7f23f40a0a30 legacy=0x7f23f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f923b700 1 -- 172.21.15.32:0/866480664 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f409dc50 con 0x7f23f409d630 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f923b700 1 --2- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f40a1510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009920 tx=0x7f23e4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cf95a13593487e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f8a3a700 1 -- 172.21.15.32:0/866480664 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011830 con 0x7f23f409d630 2023-09-30T22:29:01.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f8a3a700 1 -- 172.21.15.32:0/866480664 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4011990 con 0x7f23f409d630 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.814+0000 7f23f8a3a700 1 -- 172.21.15.32:0/866480664 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011d20 con 0x7f23f409d630 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 -- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 msgr2=0x7f23f40a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 --2- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f40a1510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009920 tx=0x7f23e4009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 -- 172.21.15.32:0/866480664 shutdown_connections 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 --2- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f4138d80 0x7f23f413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 --2- 172.21.15.32:0/866480664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f40a1510 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 -- 172.21.15.32:0/866480664 >> 172.21.15.32:0/866480664 conn(0x7f23f40a3050 msgr2=0x7f23f40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 -- 172.21.15.32:0/866480664 shutdown_connections 2023-09-30T22:29:01.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.815+0000 7f23fbca0700 1 -- 172.21.15.32:0/866480664 wait complete. 2023-09-30T22:29:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.816+0000 7f23fbca0700 1 Processor -- start 2023-09-30T22:29:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.816+0000 7f23fbca0700 1 -- start start 2023-09-30T22:29:01.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fbca0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f41207a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fbca0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f40a0a30 0x7f23f4120ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fbca0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f4138d80 0x7f23f4121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fbca0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f41231b0 con 0x7f23f4138d80 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fbca0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4121c60 con 0x7f23f40a0a30 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fbca0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23f41223b0 con 0x7f23f409d630 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f41207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f923b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f40a0a30 0x7f23f4120ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f41207a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37826/0 (socket says 172.21.15.32:37826) 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 -- 172.21.15.32:0/1067381910 learned_addr learned my addr 172.21.15.32:0/1067381910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:01.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23fa23d700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f4138d80 0x7f23f4121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 -- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f40a0a30 msgr2=0x7f23f4120ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f40a0a30 0x7f23f4120ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 -- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f4138d80 msgr2=0x7f23f4121220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f4138d80 0x7f23f4121220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 -- 172.21.15.32:0/1067381910 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4009550 con 0x7f23f409d630 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23f9a3c700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f41207a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c4e0 tx=0x7f23f000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.817+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f00052d0 con 0x7f23f409d630 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.818+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f0005430 con 0x7f23f409d630 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.818+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f40a4e40 con 0x7f23f409d630 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.818+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f00038c0 con 0x7f23f409d630 2023-09-30T22:29:01.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.818+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f41247c0 con 0x7f23f409d630 2023-09-30T22:29:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.818+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23f0003a20 con 0x7f23f409d630 2023-09-30T22:29:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.819+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f40747f0 con 0x7f23f409d630 2023-09-30T22:29:01.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.819+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f23f003e920 con 0x7f23f409d630 2023-09-30T22:29:01.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.824+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23f0017080 con 0x7f23f409d630 2023-09-30T22:29:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.951+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39f8074ea0 con 0x7f39f809e390 2023-09-30T22:29:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.952+0000 7f39f57fa700 1 -- 172.21.15.32:0/1919508593 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39ec0119a0 con 0x7f39f809e390 2023-09-30T22:29:01.953 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 msgr2=0x7f39f8118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 0x7f39f8118cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e910 tx=0x7f39ec00ec20 comp rx=0 tx=0).stop 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 shutdown_connections 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39f81341d0 0x7f39f8119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f39f809e390 0x7f39f8118cd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 --2- 172.21.15.32:0/1919508593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f39f81338c0 0x7f39f8119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 >> 172.21.15.32:0/1919508593 conn(0x7f39f80a2aa0 msgr2=0x7f39f812e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 shutdown_connections 2023-09-30T22:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.954+0000 7f39fee77700 1 -- 172.21.15.32:0/1919508593 wait complete. 2023-09-30T22:29:01.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.987+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f84801a9a90 con 0x7f848009d6f0 2023-09-30T22:29:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.987+0000 7f84767fc700 1 -- 172.21.15.32:0/3017161293 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f847c021030 con 0x7f848009d6f0 2023-09-30T22:29:01.988 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:01.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 msgr2=0x7f84801a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 0x7f84801a6060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f847c00e9e0 tx=0x7f847c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 shutdown_connections 2023-09-30T22:29:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f848009e410 0x7f84801a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f848009d6f0 0x7f84801a6060 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 --2- 172.21.15.32:0/3017161293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f848009e000 0x7f84801a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 >> 172.21.15.32:0/3017161293 conn(0x7f8480066f50 msgr2=0x7f84801baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 shutdown_connections 2023-09-30T22:29:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:01.989+0000 7f8487df2700 1 -- 172.21.15.32:0/3017161293 wait complete. 2023-09-30T22:29:02.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:02.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.029+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f23f4072980 con 0x7f23f409d630 2023-09-30T22:29:02.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.030+0000 7f23ea7fc700 1 -- 172.21.15.32:0/1067381910 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23f0018260 con 0x7f23f409d630 2023-09-30T22:29:02.031 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 msgr2=0x7f23f41207a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:02.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f41207a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c4e0 tx=0x7f23f000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 shutdown_connections 2023-09-30T22:29:02.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23f4138d80 0x7f23f4121220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23f409d630 0x7f23f41207a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 --2- 172.21.15.32:0/1067381910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23f40a0a30 0x7f23f4120ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 >> 172.21.15.32:0/1067381910 conn(0x7f23f40a3050 msgr2=0x7f23f409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:02.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 shutdown_connections 2023-09-30T22:29:02.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.032+0000 7f23fbca0700 1 -- 172.21.15.32:0/1067381910 wait complete. 2023-09-30T22:29:02.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:02.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:02.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.305+0000 7f6c228ed700 1 Processor -- start 2023-09-30T22:29:02.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.305+0000 7f6c228ed700 1 -- start start 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c228ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 0x7f6c1c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c228ed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 0x7f6c1c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c228ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c070940 con 0x7f6c1c09e930 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c228ed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c06fd60 con 0x7f6c1c136fe0 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c228ed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c06fa80 con 0x7f6c1c09f410 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 0x7f6c1c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c20e8a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 0x7f6c1c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 0x7f6c1c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37828/0 (socket says 172.21.15.32:37828) 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c20e8a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 0x7f6c1c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56254/0 (socket says 172.21.15.32:56254) 2023-09-30T22:29:02.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 -- 172.21.15.32:0/2816523012 learned_addr learned my addr 172.21.15.32:0/2816523012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 -- 172.21.15.32:0/2816523012 >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 msgr2=0x7f6c1c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 --2- 172.21.15.32:0/2816523012 >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 0x7f6c1c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 -- 172.21.15.32:0/2816523012 >> v1:172.21.15.32:6789/0 conn(0x7f6c1c09e930 legacy=0x7f6c1c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 -- 172.21.15.32:0/2816523012 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c09fb60 con 0x7f6c1c09f410 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c20e8a700 1 --2- 172.21.15.32:0/2816523012 >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 0x7f6c1c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.306+0000 7f6c1b7fe700 1 --2- 172.21.15.32:0/2816523012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 0x7f6c1c136aa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c040098f0 tx=0x7f6c04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab893412cb77faf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c1affd700 1 -- 172.21.15.32:0/2816523012 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04011830 con 0x7f6c1c09f410 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c1affd700 1 -- 172.21.15.32:0/2816523012 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c04011990 con 0x7f6c1c09f410 2023-09-30T22:29:02.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c1affd700 1 -- 172.21.15.32:0/2816523012 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04011d20 con 0x7f6c1c09f410 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 -- 172.21.15.32:0/2816523012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 msgr2=0x7f6c1c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 --2- 172.21.15.32:0/2816523012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 0x7f6c1c136aa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c040098f0 tx=0x7f6c04009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 -- 172.21.15.32:0/2816523012 shutdown_connections 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 --2- 172.21.15.32:0/2816523012 >> v2:172.21.15.32:3300/0 conn(0x7f6c1c136fe0 0x7f6c1c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 --2- 172.21.15.32:0/2816523012 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09f410 0x7f6c1c136aa0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 -- 172.21.15.32:0/2816523012 >> 172.21.15.32:0/2816523012 conn(0x7f6c1c0a4210 msgr2=0x7f6c1c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:02.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 -- 172.21.15.32:0/2816523012 shutdown_connections 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.307+0000 7f6c228ed700 1 -- 172.21.15.32:0/2816523012 wait complete. 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.308+0000 7f6c228ed700 1 Processor -- start 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 -- start start 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 0x7f6c1c11c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c1c09f410 0x7f6c1c11cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c1c136fe0 0x7f6c1c11d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c11fe50 con 0x7f6c1c136fe0 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0a2f40 con 0x7f6c1c09f410 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c228ed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c11f8a0 con 0x7f6c1c09e930 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 0x7f6c1c11c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c20e8a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c1c136fe0 0x7f6c1c11d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:02.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 0x7f6c1c11c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37844/0 (socket says 172.21.15.32:37844) 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c1bfff700 1 -- 172.21.15.32:0/3522575184 learned_addr learned my addr 172.21.15.32:0/3522575184 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c1b7fe700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c1c09f410 0x7f6c1c11cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c20e8a700 1 -- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 msgr2=0x7f6c1c11c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.309+0000 7f6c20e8a700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 0x7f6c1c11c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c20e8a700 1 -- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c1c09f410 msgr2=0x7f6c1c11cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c20e8a700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c1c09f410 0x7f6c1c11cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c20e8a700 1 -- 172.21.15.32:0/3522575184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c14009a70 con 0x7f6c1c136fe0 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c1bfff700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 0x7f6c1c11c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c20e8a700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c1c136fe0 0x7f6c1c11d0e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400ef90 tx=0x7f6c1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400cc90 con 0x7f6c1c136fe0 2023-09-30T22:29:02.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400f040 con 0x7f6c1c136fe0 2023-09-30T22:29:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c04009550 con 0x7f6c1c136fe0 2023-09-30T22:29:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c140146c0 con 0x7f6c1c136fe0 2023-09-30T22:29:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.310+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c11e330 con 0x7f6c1c136fe0 2023-09-30T22:29:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.311+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c14014820 con 0x7f6c1c136fe0 2023-09-30T22:29:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.311+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6c14040220 con 0x7f6c1c136fe0 2023-09-30T22:29:02.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.311+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c074ea0 con 0x7f6c1c136fe0 2023-09-30T22:29:02.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.316+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c14018080 con 0x7f6c1c136fe0 2023-09-30T22:29:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.521+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c1c11ea30 con 0x7f6c1c136fe0 2023-09-30T22:29:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.521+0000 7f6c18ff9700 1 -- 172.21.15.32:0/3522575184 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c140076f0 con 0x7f6c1c136fe0 2023-09-30T22:29:02.523 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c1c136fe0 msgr2=0x7f6c1c11d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c1c136fe0 0x7f6c1c11d0e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400ef90 tx=0x7f6c1400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 shutdown_connections 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c1c136fe0 0x7f6c1c11d0e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c1c09e930 0x7f6c1c11c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 --2- 172.21.15.32:0/3522575184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c1c09f410 0x7f6c1c11cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 >> 172.21.15.32:0/3522575184 conn(0x7f6c1c0a4210 msgr2=0x7f6c1c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 shutdown_connections 2023-09-30T22:29:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:02.524+0000 7f6c228ed700 1 -- 172.21.15.32:0/3522575184 wait complete. 2023-09-30T22:29:02.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:02.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:03.005 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:03.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.129+0000 7fafc12b1700 1 Processor -- start 2023-09-30T22:29:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.129+0000 7fafc12b1700 1 -- start start 2023-09-30T22:29:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafc12b1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc1358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafc12b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafc12b1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc1310d0 con 0x7fafbc1305f0 2023-09-30T22:29:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafc12b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1301c0 con 0x7fafbc135e30 2023-09-30T22:29:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafc12b1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc071ff0 con 0x7fafbc133510 2023-09-30T22:29:03.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc1358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc1358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37860/0 (socket says 172.21.15.32:37860) 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 -- 172.21.15.32:0/3900250283 learned_addr learned my addr 172.21.15.32:0/3900250283 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafbaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fafbc1305f0 0x7fafbc1309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54178/0 (socket says 172.21.15.32:54178) 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafbb7fe700 1 --2- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafb9ffb700 1 -- 172.21.15.32:0/3900250283 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841108686 0 0) 0x7fafbc1310d0 con 0x7fafbc1305f0 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 -- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 msgr2=0x7fafbc138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 --2- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 -- 172.21.15.32:0/3900250283 >> v1:172.21.15.32:6789/0 conn(0x7fafbc1305f0 legacy=0x7fafbc1309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 -- 172.21.15.32:0/3900250283 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc070e70 con 0x7fafbc133510 2023-09-30T22:29:03.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.130+0000 7fafba7fc700 1 --2- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc1358f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fafb000dd50 tx=0x7fafb000f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952852343fbb0538 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.131+0000 7fafb9ffb700 1 -- 172.21.15.32:0/3900250283 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0012070 con 0x7fafbc133510 2023-09-30T22:29:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.131+0000 7fafb9ffb700 1 -- 172.21.15.32:0/3900250283 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb000fb20 con 0x7fafbc133510 2023-09-30T22:29:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.131+0000 7fafb9ffb700 1 -- 172.21.15.32:0/3900250283 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000dea0 con 0x7fafbc133510 2023-09-30T22:29:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.131+0000 7fafc12b1700 1 -- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 msgr2=0x7fafbc1358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.131+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc1358f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fafb000dd50 tx=0x7fafb000f330 comp rx=0 tx=0).stop 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.132+0000 7fafc12b1700 1 -- 172.21.15.32:0/3900250283 shutdown_connections 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.132+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.132+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3900250283 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc1358f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.132+0000 7fafc12b1700 1 -- 172.21.15.32:0/3900250283 >> 172.21.15.32:0/3900250283 conn(0x7fafbc0a2aa0 msgr2=0x7fafbc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.132+0000 7fafc12b1700 1 -- 172.21.15.32:0/3900250283 shutdown_connections 2023-09-30T22:29:03.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.132+0000 7fafc12b1700 1 -- 172.21.15.32:0/3900250283 wait complete. 2023-09-30T22:29:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.133+0000 7fafc12b1700 1 Processor -- start 2023-09-30T22:29:03.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 -- start start 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafbc1305f0 0x7fafbc09c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc09ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc09cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc118f10 con 0x7fafbc1305f0 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc0718b0 con 0x7fafbc135e30 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafc12b1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc135900 con 0x7fafbc133510 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafba7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc09ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafbaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafbc1305f0 0x7fafbc09c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.134+0000 7fafba7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc09ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37868/0 (socket says 172.21.15.32:37868) 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 -- 172.21.15.32:0/3088612497 learned_addr learned my addr 172.21.15.32:0/3088612497 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafbb7fe700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc09cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 -- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 msgr2=0x7fafbc09cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc09cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 -- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafbc1305f0 msgr2=0x7fafbc09c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafbc1305f0 0x7fafbc09c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 -- 172.21.15.32:0/3088612497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb000da00 con 0x7fafbc133510 2023-09-30T22:29:03.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafbaffd700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafbc1305f0 0x7fafbc09c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.135+0000 7fafba7fc700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc09ca40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.136+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003b30 con 0x7fafbc133510 2023-09-30T22:29:03.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.136+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0003c90 con 0x7fafbc133510 2023-09-30T22:29:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.136+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0016430 con 0x7fafbc133510 2023-09-30T22:29:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.136+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc119f50 con 0x7fafbc133510 2023-09-30T22:29:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.136+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc1190a0 con 0x7fafbc133510 2023-09-30T22:29:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.137+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc11a320 con 0x7fafbc133510 2023-09-30T22:29:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.137+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fafb0016590 con 0x7fafbc133510 2023-09-30T22:29:03.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.137+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fafb0044f30 con 0x7fafbc133510 2023-09-30T22:29:03.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.142+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafb0032430 con 0x7fafbc133510 2023-09-30T22:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.164+0000 7f3f58ac3700 1 Processor -- start 2023-09-30T22:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.164+0000 7f3f58ac3700 1 -- start start 2023-09-30T22:29:03.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f58ac3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 0x7f3f54138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f58ac3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f54139250 0x7f3f5413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f58ac3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5409edd0 con 0x7f3f540a0a30 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f58ac3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540a18e0 con 0x7f3f540a1510 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f58ac3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5409e1f0 con 0x7f3f54139250 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f51d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 0x7f3f54138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f51d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 0x7f3f54138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39834/0 (socket says 172.21.15.32:39834) 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f52d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f54139250 0x7f3f5413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f51d9b700 1 -- 172.21.15.32:0/269129552 learned_addr learned my addr 172.21.15.32:0/269129552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.165+0000 7f3f5259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f540a0a30 0x7f3f540a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54194/0 (socket says 172.21.15.32:54194) 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f51d9b700 1 -- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f54139250 msgr2=0x7f3f5413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f51d9b700 1 --2- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f54139250 0x7f3f5413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f51d9b700 1 -- 172.21.15.32:0/269129552 >> v1:172.21.15.32:6789/0 conn(0x7f3f540a0a30 legacy=0x7f3f540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f51d9b700 1 -- 172.21.15.32:0/269129552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5409dc50 con 0x7f3f540a1510 2023-09-30T22:29:03.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f5159a700 1 -- 172.21.15.32:0/269129552 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190571116 0 0) 0x7f3f5409edd0 con 0x7f3f540a0a30 2023-09-30T22:29:03.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f52d9d700 1 --2- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f54139250 0x7f3f5413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f51d9b700 1 --2- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 0x7f3f54138d10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009920 tx=0x7f3f48009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a400010368a8dfa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f5159a700 1 -- 172.21.15.32:0/269129552 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48011830 con 0x7f3f540a1510 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f5159a700 1 -- 172.21.15.32:0/269129552 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48011990 con 0x7f3f540a1510 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.166+0000 7f3f5159a700 1 -- 172.21.15.32:0/269129552 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48011d20 con 0x7f3f540a1510 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 -- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 msgr2=0x7f3f54138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 0x7f3f54138d10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009920 tx=0x7f3f48009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:03.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 -- 172.21.15.32:0/269129552 shutdown_connections 2023-09-30T22:29:03.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f54139250 0x7f3f5413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/269129552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f540a1510 0x7f3f54138d10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 -- 172.21.15.32:0/269129552 >> 172.21.15.32:0/269129552 conn(0x7f3f540a6550 msgr2=0x7f3f540a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 -- 172.21.15.32:0/269129552 shutdown_connections 2023-09-30T22:29:03.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.167+0000 7f3f58ac3700 1 -- 172.21.15.32:0/269129552 wait complete. 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 Processor -- start 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 -- start start 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 0x7f3f541217c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540a1510 0x7f3f54121d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f54139250 0x7f3f54122240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54125200 con 0x7f3f540a1510 2023-09-30T22:29:03.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54138a40 con 0x7f3f54139250 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f58ac3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540a0220 con 0x7f3f540a0a30 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f5259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 0x7f3f541217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f51d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540a1510 0x7f3f54121d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f5259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 0x7f3f541217c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37894/0 (socket says 172.21.15.32:37894) 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f5259c700 1 -- 172.21.15.32:0/2352696387 learned_addr learned my addr 172.21.15.32:0/2352696387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.169+0000 7f3f52d9d700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f54139250 0x7f3f54122240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f5259c700 1 -- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f54139250 msgr2=0x7f3f54122240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f5259c700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f54139250 0x7f3f54122240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f5259c700 1 -- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540a1510 msgr2=0x7f3f54121d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f5259c700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540a1510 0x7f3f54121d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f5259c700 1 -- 172.21.15.32:0/2352696387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009a70 con 0x7f3f540a0a30 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f51d9b700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540a1510 0x7f3f54121d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f5259c700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 0x7f3f541217c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c520 tx=0x7f3f4400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44004c80 con 0x7f3f540a0a30 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44004de0 con 0x7f3f540a0a30 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44007310 con 0x7f3f540a0a30 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48009550 con 0x7f3f540a0a30 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.170+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54124c60 con 0x7f3f540a0a30 2023-09-30T22:29:03.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.171+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f44007470 con 0x7f3f540a0a30 2023-09-30T22:29:03.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.172+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f540a1d00 con 0x7f3f540a0a30 2023-09-30T22:29:03.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.172+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3f4403f3a0 con 0x7f3f540a0a30 2023-09-30T22:29:03.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.176+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f4402c070 con 0x7f3f540a0a30 2023-09-30T22:29:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.207+0000 7fae77970700 1 Processor -- start 2023-09-30T22:29:03.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.207+0000 7fae77970700 1 -- start start 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae77970700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 0x7fae701337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae77970700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 0x7fae7012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae77970700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7006b990 con 0x7fae70134260 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae77970700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae700a0590 con 0x7fae70133e30 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae77970700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7009fb80 con 0x7fae70133420 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 0x7fae701337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae74f0b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 0x7fae7012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 0x7fae701337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39858/0 (socket says 172.21.15.32:39858) 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae74f0b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 0x7fae7012ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56294/0 (socket says 172.21.15.32:56294) 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 -- 172.21.15.32:0/3862264163 learned_addr learned my addr 172.21.15.32:0/3862264163 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae75f0d700 1 --1- 172.21.15.32:0/3862264163 >> v1:172.21.15.32:6789/0 conn(0x7fae70134260 0x7fae701304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54200/0 (socket says 172.21.15.32:54200) 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 -- 172.21.15.32:0/3862264163 >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 msgr2=0x7fae7012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 --2- 172.21.15.32:0/3862264163 >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 0x7fae7012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 -- 172.21.15.32:0/3862264163 >> v1:172.21.15.32:6789/0 conn(0x7fae70134260 legacy=0x7fae701304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae7570c700 1 -- 172.21.15.32:0/3862264163 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700a0030 con 0x7fae70133420 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.208+0000 7fae74f0b700 1 --2- 172.21.15.32:0/3862264163 >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 0x7fae7012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.209+0000 7fae7570c700 1 --2- 172.21.15.32:0/3862264163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 0x7fae701337f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0098f0 tx=0x7fae6c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563270afba4001e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.209+0000 7fae67fff700 1 -- 172.21.15.32:0/3862264163 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c011830 con 0x7fae70133420 2023-09-30T22:29:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.209+0000 7fae67fff700 1 -- 172.21.15.32:0/3862264163 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c011990 con 0x7fae70133420 2023-09-30T22:29:03.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.209+0000 7fae67fff700 1 -- 172.21.15.32:0/3862264163 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c011d20 con 0x7fae70133420 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 -- 172.21.15.32:0/3862264163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 msgr2=0x7fae701337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 --2- 172.21.15.32:0/3862264163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 0x7fae701337f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0098f0 tx=0x7fae6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 -- 172.21.15.32:0/3862264163 shutdown_connections 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 --2- 172.21.15.32:0/3862264163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133420 0x7fae701337f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 --2- 172.21.15.32:0/3862264163 >> v2:172.21.15.32:3300/0 conn(0x7fae70133e30 0x7fae7012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 -- 172.21.15.32:0/3862264163 >> 172.21.15.32:0/3862264163 conn(0x7fae700a4210 msgr2=0x7fae700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 -- 172.21.15.32:0/3862264163 shutdown_connections 2023-09-30T22:29:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.210+0000 7fae77970700 1 -- 172.21.15.32:0/3862264163 wait complete. 2023-09-30T22:29:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.212+0000 7fae77970700 1 Processor -- start 2023-09-30T22:29:03.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.212+0000 7fae77970700 1 -- start start 2023-09-30T22:29:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.212+0000 7fae77970700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 0x7fae7009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.212+0000 7fae77970700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133e30 0x7fae7009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae77970700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae70134260 0x7fae7009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae77970700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7009dcd0 con 0x7fae70133420 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae77970700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7011fe90 con 0x7fae70133e30 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae77970700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae7009df30 con 0x7fae70134260 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 0x7fae7009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 0x7fae7009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56304/0 (socket says 172.21.15.32:56304) 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae74f0b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133e30 0x7fae7009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 -- 172.21.15.32:0/4239748602 learned_addr learned my addr 172.21.15.32:0/4239748602 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae75f0d700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae70134260 0x7fae7009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 -- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae70134260 msgr2=0x7fae7009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae70134260 0x7fae7009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 -- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133e30 msgr2=0x7fae7009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133e30 0x7fae7009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 -- 172.21.15.32:0/4239748602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae600094a0 con 0x7fae70133420 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae7570c700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 0x7fae7009c480 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009c40 tx=0x7fae6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.213+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c004400 con 0x7fae70133420 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.214+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c01f070 con 0x7fae70133420 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.214+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c009550 con 0x7fae70133420 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.214+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0038e0 con 0x7fae70133420 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.214+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7011fb60 con 0x7fae70133420 2023-09-30T22:29:03.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.215+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae70131990 con 0x7fae70133420 2023-09-30T22:29:03.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.215+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae6c021030 con 0x7fae70133420 2023-09-30T22:29:03.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.215+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fae6c041120 con 0x7fae70133420 2023-09-30T22:29:03.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.220+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae6c00b9d0 con 0x7fae70133420 2023-09-30T22:29:03.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.357+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fafbc072340 con 0x7fafbc133510 2023-09-30T22:29:03.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.357+0000 7fafa37fe700 1 -- 172.21.15.32:0/3088612497 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafb0028b80 con 0x7fafbc133510 2023-09-30T22:29:03.359 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.360+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 msgr2=0x7fafbc09ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.360+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc09ca40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.360+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 shutdown_connections 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.360+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fafbc135e30 0x7fafbc09cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.360+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafbc1305f0 0x7fafbc09c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.360+0000 7fafc12b1700 1 --2- 172.21.15.32:0/3088612497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fafbc133510 0x7fafbc09ca40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.361+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 >> 172.21.15.32:0/3088612497 conn(0x7fafbc0a2aa0 msgr2=0x7fafbc1312f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.361+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 shutdown_connections 2023-09-30T22:29:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.361+0000 7fafc12b1700 1 -- 172.21.15.32:0/3088612497 wait complete. 2023-09-30T22:29:03.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:03.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.386+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3f54074ea0 con 0x7f3f540a0a30 2023-09-30T22:29:03.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.386+0000 7f3f42ffd700 1 -- 172.21.15.32:0/2352696387 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f440123b0 con 0x7f3f540a0a30 2023-09-30T22:29:03.387 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 msgr2=0x7f3f541217c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 0x7f3f541217c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c520 tx=0x7f3f4400c830 comp rx=0 tx=0).stop 2023-09-30T22:29:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 shutdown_connections 2023-09-30T22:29:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f54139250 0x7f3f54122240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3f540a0a30 0x7f3f541217c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 --2- 172.21.15.32:0/2352696387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540a1510 0x7f3f54121d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 >> 172.21.15.32:0/2352696387 conn(0x7f3f540a6550 msgr2=0x7f3f54137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 shutdown_connections 2023-09-30T22:29:03.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.389+0000 7f3f58ac3700 1 -- 172.21.15.32:0/2352696387 wait complete. 2023-09-30T22:29:03.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.438+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae70074ea0 con 0x7fae70133420 2023-09-30T22:29:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.438+0000 7fae65ffb700 1 -- 172.21.15.32:0/4239748602 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae6c015070 con 0x7fae70133420 2023-09-30T22:29:03.439 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 msgr2=0x7fae7009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 0x7fae7009c480 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009c40 tx=0x7fae6c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 shutdown_connections 2023-09-30T22:29:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fae70134260 0x7fae7009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae70133420 0x7fae7009c480 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 --2- 172.21.15.32:0/4239748602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fae70133e30 0x7fae7009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 >> 172.21.15.32:0/4239748602 conn(0x7fae700a4210 msgr2=0x7fae700a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 shutdown_connections 2023-09-30T22:29:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.441+0000 7fae77970700 1 -- 172.21.15.32:0/4239748602 wait complete. 2023-09-30T22:29:03.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:03.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:03.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.679+0000 7ff1f5ab5700 1 Processor -- start 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.679+0000 7ff1f5ab5700 1 -- start start 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1f5ab5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 0x7ff1f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1f5ab5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1f5ab5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 0x7ff1f0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1f5ab5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f006b460 con 0x7ff1f009e410 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1f5ab5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f009b120 con 0x7ff1f009d6f0 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1f5ab5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f009a7e0 con 0x7ff1f009e000 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 0x7ff1f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1effff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 0x7ff1f0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1eeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1effff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 0x7ff1f0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56320/0 (socket says 172.21.15.32:56320) 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 0x7ff1f009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39862/0 (socket says 172.21.15.32:39862) 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1effff700 1 -- 172.21.15.32:0/1976162217 learned_addr learned my addr 172.21.15.32:0/1976162217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1eeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37916/0 (socket says 172.21.15.32:37916) 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 msgr2=0x7ff1f01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 --2- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/1976162217 >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 msgr2=0x7ff1f0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 --2- 172.21.15.32:0/1976162217 >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 0x7ff1f0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/1976162217 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f009a500 con 0x7ff1f009d6f0 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1eeffd700 1 --2- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:03.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1effff700 1 --2- 172.21.15.32:0/1976162217 >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 0x7ff1f0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.680+0000 7ff1ef7fe700 1 --2- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 0x7ff1f009dac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7017344246a2836 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1ee7fc700 1 -- 172.21.15.32:0/1976162217 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0011830 con 0x7ff1f009d6f0 2023-09-30T22:29:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1ee7fc700 1 -- 172.21.15.32:0/1976162217 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e0011990 con 0x7ff1f009d6f0 2023-09-30T22:29:03.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1ee7fc700 1 -- 172.21.15.32:0/1976162217 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0011d20 con 0x7ff1f009d6f0 2023-09-30T22:29:03.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 msgr2=0x7ff1f009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 0x7ff1f009dac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/1976162217 shutdown_connections 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009d6f0 0x7ff1f009dac0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/1976162217 >> v2:172.21.15.32:3300/0 conn(0x7ff1f009e410 0x7ff1f0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/1976162217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/1976162217 >> 172.21.15.32:0/1976162217 conn(0x7ff1f0066f50 msgr2=0x7ff1f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/1976162217 shutdown_connections 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.681+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/1976162217 wait complete. 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.682+0000 7ff1f5ab5700 1 Processor -- start 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 -- start start 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009e410 0x7ff1f01a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01aa260 con 0x7ff1f009d6f0 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01a7990 con 0x7ff1f009e410 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1f5ab5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01a7fb0 con 0x7ff1f009e000 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56332/0 (socket says 172.21.15.32:56332) 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/375220219 learned_addr learned my addr 172.21.15.32:0/375220219 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1effff700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009e410 0x7ff1f01a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.683+0000 7ff1eeffd700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 msgr2=0x7ff1f01a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1ef7fe700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009e410 msgr2=0x7ff1f01a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1ef7fe700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009e410 0x7ff1f01a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1ef7fe700 1 -- 172.21.15.32:0/375220219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40094a0 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1ef7fe700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01a5d80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00090b0 tx=0x7ff1e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004450 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e001f070 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e8009b30 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00038a0 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.684+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e00094a0 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.685+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1e0003a00 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.685+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff1e0021070 con 0x7ff1f009d6f0 2023-09-30T22:29:03.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.685+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f009c710 con 0x7ff1f009d6f0 2023-09-30T22:29:03.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.690+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1e0015080 con 0x7ff1f009d6f0 2023-09-30T22:29:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.894+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1f0074ea0 con 0x7ff1f009d6f0 2023-09-30T22:29:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.894+0000 7ff1d7fff700 1 -- 172.21.15.32:0/375220219 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1e0035090 con 0x7ff1f009d6f0 2023-09-30T22:29:03.895 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 msgr2=0x7ff1f01a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01a5d80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00090b0 tx=0x7ff1e0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 shutdown_connections 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff1f009e410 0x7ff1f01a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f009d6f0 0x7ff1f01a5d80 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 --2- 172.21.15.32:0/375220219 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff1f009e000 0x7ff1f01a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 >> 172.21.15.32:0/375220219 conn(0x7ff1f0066f50 msgr2=0x7ff1f01bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 shutdown_connections 2023-09-30T22:29:03.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:03.897+0000 7ff1f5ab5700 1 -- 172.21.15.32:0/375220219 wait complete. 2023-09-30T22:29:03.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:04.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:04.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:04.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.529+0000 7f6f8679f700 1 Processor -- start 2023-09-30T22:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.529+0000 7f6f8679f700 1 -- start start 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8679f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 0x7f6f8009dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8679f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009e110 0x7f6f80133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8679f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6f80133da0 0x7f6f80134210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8679f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8009f560 con 0x7f6f80133da0 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8679f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80099e10 con 0x7f6f8009d800 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8679f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8006b4d0 con 0x7f6f8009e110 2023-09-30T22:29:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 0x7f6f8009dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 0x7f6f8009dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39880/0 (socket says 172.21.15.32:39880) 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 -- 172.21.15.32:0/1910860077 learned_addr learned my addr 172.21.15.32:0/1910860077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f85f9e700 1 --2- 172.21.15.32:0/1910860077 >> v2:172.21.15.32:3300/0 conn(0x7f6f80133da0 0x7f6f80134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f84f9c700 1 --2- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009e110 0x7f6f80133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 -- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009e110 msgr2=0x7f6f80133860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 --2- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009e110 0x7f6f80133860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 -- 172.21.15.32:0/1910860077 >> v2:172.21.15.32:3300/0 conn(0x7f6f80133da0 msgr2=0x7f6f80134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 --2- 172.21.15.32:0/1910860077 >> v2:172.21.15.32:3300/0 conn(0x7f6f80133da0 0x7f6f80134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.530+0000 7f6f8579d700 1 -- 172.21.15.32:0/1910860077 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8009a060 con 0x7f6f8009d800 2023-09-30T22:29:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.531+0000 7f6f85f9e700 1 --2- 172.21.15.32:0/1910860077 >> v2:172.21.15.32:3300/0 conn(0x7f6f80133da0 0x7f6f80134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.531+0000 7f6f8579d700 1 --2- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 0x7f6f8009dbd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0098f0 tx=0x7f6f7c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e567982352cfa090 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.531+0000 7f6f77fff700 1 -- 172.21.15.32:0/1910860077 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c011830 con 0x7f6f8009d800 2023-09-30T22:29:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.531+0000 7f6f77fff700 1 -- 172.21.15.32:0/1910860077 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c011990 con 0x7f6f8009d800 2023-09-30T22:29:04.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.531+0000 7f6f77fff700 1 -- 172.21.15.32:0/1910860077 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c011d20 con 0x7f6f8009d800 2023-09-30T22:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 -- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 msgr2=0x7f6f8009dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 --2- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 0x7f6f8009dbd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0098f0 tx=0x7f6f7c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 -- 172.21.15.32:0/1910860077 shutdown_connections 2023-09-30T22:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 --2- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f8009d800 0x7f6f8009dbd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 --2- 172.21.15.32:0/1910860077 >> v2:172.21.15.32:3300/0 conn(0x7f6f80133da0 0x7f6f80134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 --2- 172.21.15.32:0/1910860077 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009e110 0x7f6f80133860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 -- 172.21.15.32:0/1910860077 >> 172.21.15.32:0/1910860077 conn(0x7f6f800a2e20 msgr2=0x7f6f800a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 -- 172.21.15.32:0/1910860077 shutdown_connections 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.532+0000 7f6f8679f700 1 -- 172.21.15.32:0/1910860077 wait complete. 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.533+0000 7f6f8679f700 1 Processor -- start 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.533+0000 7f6f8679f700 1 -- start start 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8679f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009d800 0x7f6f801211c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8679f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 0x7f6f80121700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8679f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f80133da0 0x7f6f80121c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8679f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80122670 con 0x7f6f8009e110 2023-09-30T22:29:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8679f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80123fd0 con 0x7f6f80133da0 2023-09-30T22:29:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8679f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80123c10 con 0x7f6f8009d800 2023-09-30T22:29:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 0x7f6f80121700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 0x7f6f80121700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56358/0 (socket says 172.21.15.32:56358) 2023-09-30T22:29:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 -- 172.21.15.32:0/3950408824 learned_addr learned my addr 172.21.15.32:0/3950408824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f8579d700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009d800 0x7f6f801211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f85f9e700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f80133da0 0x7f6f80121c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 -- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009d800 msgr2=0x7f6f801211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009d800 0x7f6f801211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 -- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f80133da0 msgr2=0x7f6f80121c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f80133da0 0x7f6f80121c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.534+0000 7f6f84f9c700 1 -- 172.21.15.32:0/3950408824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78009a70 con 0x7f6f8009e110 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.535+0000 7f6f84f9c700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 0x7f6f80121700 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000e9b0 tx=0x7f6f7000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.535+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70005a20 con 0x7f6f8009e110 2023-09-30T22:29:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.535+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70005b80 con 0x7f6f8009e110 2023-09-30T22:29:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.535+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c009550 con 0x7f6f8009e110 2023-09-30T22:29:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.535+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f80124740 con 0x7f6f8009e110 2023-09-30T22:29:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.535+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000c640 con 0x7f6f8009e110 2023-09-30T22:29:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.536+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f7000c7a0 con 0x7f6f8009e110 2023-09-30T22:29:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.536+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f7003fb90 con 0x7f6f8009e110 2023-09-30T22:29:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.536+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f600052e0 con 0x7f6f8009e110 2023-09-30T22:29:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.545+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f70020080 con 0x7f6f8009e110 2023-09-30T22:29:04.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.570+0000 7ff31a24c700 1 Processor -- start 2023-09-30T22:29:04.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.570+0000 7ff31a24c700 1 -- start start 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff31a24c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3140a0a30 0x7ff3140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff31a24c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 0x7ff314133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff31a24c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31409edd0 con 0x7ff314133fa0 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff31a24c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31409e1f0 con 0x7ff3140a0a30 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff31a24c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31409dc50 con 0x7ff3140a1340 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 0x7ff314133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 0x7ff314133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39894/0 (socket says 172.21.15.32:39894) 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff313fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff314133fa0 0x7ff31413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54214/0 (socket says 172.21.15.32:54214) 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff30bfff700 1 -- 172.21.15.32:0/1420911844 learned_addr learned my addr 172.21.15.32:0/1420911844 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.571+0000 7ff3137fe700 1 --2- 172.21.15.32:0/1420911844 >> v2:172.21.15.32:3300/0 conn(0x7ff3140a0a30 0x7ff3140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff30bfff700 1 -- 172.21.15.32:0/1420911844 >> v2:172.21.15.32:3300/0 conn(0x7ff3140a0a30 msgr2=0x7ff3140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff30bfff700 1 --2- 172.21.15.32:0/1420911844 >> v2:172.21.15.32:3300/0 conn(0x7ff3140a0a30 0x7ff3140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff30bfff700 1 -- 172.21.15.32:0/1420911844 >> v1:172.21.15.32:6789/0 conn(0x7ff314133fa0 legacy=0x7ff31413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff312ffd700 1 -- 172.21.15.32:0/1420911844 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067208276 0 0) 0x7ff31409edd0 con 0x7ff314133fa0 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff30bfff700 1 -- 172.21.15.32:0/1420911844 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140a1fe0 con 0x7ff3140a1340 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff3137fe700 1 --2- 172.21.15.32:0/1420911844 >> v2:172.21.15.32:3300/0 conn(0x7ff3140a0a30 0x7ff3140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff30bfff700 1 --2- 172.21.15.32:0/1420911844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 0x7ff314133a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0098f0 tx=0x7ff2fc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb29a902ebb1f09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:04.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff312ffd700 1 -- 172.21.15.32:0/1420911844 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc011830 con 0x7ff3140a1340 2023-09-30T22:29:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff312ffd700 1 -- 172.21.15.32:0/1420911844 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc011990 con 0x7ff3140a1340 2023-09-30T22:29:04.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.572+0000 7ff312ffd700 1 -- 172.21.15.32:0/1420911844 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc011d20 con 0x7ff3140a1340 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 -- 172.21.15.32:0/1420911844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 msgr2=0x7ff314133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 --2- 172.21.15.32:0/1420911844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 0x7ff314133a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0098f0 tx=0x7ff2fc009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 -- 172.21.15.32:0/1420911844 shutdown_connections 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 --2- 172.21.15.32:0/1420911844 >> v2:172.21.15.32:3300/0 conn(0x7ff3140a0a30 0x7ff3140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 --2- 172.21.15.32:0/1420911844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a1340 0x7ff314133a60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 -- 172.21.15.32:0/1420911844 >> 172.21.15.32:0/1420911844 conn(0x7ff3140a6550 msgr2=0x7ff3140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 -- 172.21.15.32:0/1420911844 shutdown_connections 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.573+0000 7ff31a24c700 1 -- 172.21.15.32:0/1420911844 wait complete. 2023-09-30T22:29:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.574+0000 7ff31a24c700 1 Processor -- start 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 -- start start 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a0a30 0x7ff31411e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 0x7ff31411ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff314133fa0 0x7ff31411f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314121810 con 0x7ff3140a1340 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff314120a80 con 0x7ff3140a0a30 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff31a24c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff314122130 con 0x7ff314133fa0 2023-09-30T22:29:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 0x7ff31411ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 0x7ff31411ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56376/0 (socket says 172.21.15.32:56376) 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff30bfff700 1 -- 172.21.15.32:0/301303920 learned_addr learned my addr 172.21.15.32:0/301303920 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.575+0000 7ff313fff700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff314133fa0 0x7ff31411f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff3137fe700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a0a30 0x7ff31411e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff30bfff700 1 -- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff314133fa0 msgr2=0x7ff31411f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff30bfff700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff314133fa0 0x7ff31411f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff30bfff700 1 -- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a0a30 msgr2=0x7ff31411e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff30bfff700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a0a30 0x7ff31411e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff30bfff700 1 -- 172.21.15.32:0/301303920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040094a0 con 0x7ff3140a1340 2023-09-30T22:29:04.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff30bfff700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 0x7ff31411ec60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff31409dd90 tx=0x7ff2fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:04.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc004300 con 0x7ff3140a1340 2023-09-30T22:29:04.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc01f070 con 0x7ff3140a1340 2023-09-30T22:29:04.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc009550 con 0x7ff3140a1340 2023-09-30T22:29:04.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0036a0 con 0x7ff3140a1340 2023-09-30T22:29:04.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.576+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3140a3650 con 0x7ff3140a1340 2023-09-30T22:29:04.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.577+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2fc021030 con 0x7ff3140a1340 2023-09-30T22:29:04.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.578+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff2fc040930 con 0x7ff3140a1340 2023-09-30T22:29:04.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.578+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff314121ab0 con 0x7ff3140a1340 2023-09-30T22:29:04.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.583+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2fc015080 con 0x7ff3140a1340 2023-09-30T22:29:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 Processor -- start 2023-09-30T22:29:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 -- start start 2023-09-30T22:29:04.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77540a0a30 0x7f77540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 0x7f7754133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775409edd0 con 0x7f7754133fa0 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775409e1f0 con 0x7f77540a0a30 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.612+0000 7f775c1d1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f775409dc50 con 0x7f77540a1340 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 0x7f7754133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 0x7f7754133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37956/0 (socket says 172.21.15.32:37956) 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 -- 172.21.15.32:0/2004628334 learned_addr learned my addr 172.21.15.32:0/2004628334 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f7759f6d700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77540a0a30 0x7f77540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 -- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77540a0a30 msgr2=0x7f77540a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:04.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77540a0a30 0x7f77540a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 -- 172.21.15.32:0/2004628334 >> v1:172.21.15.32:6789/0 conn(0x7f7754133fa0 legacy=0x7f775413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 -- 172.21.15.32:0/2004628334 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540a1fe0 con 0x7f77540a1340 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f7759f6d700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77540a0a30 0x7f77540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f775976c700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 0x7f7754133a60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7744009920 tx=0x7f7744009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d24d77f26ec92bce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.613+0000 7f7758f6b700 1 -- 172.21.15.32:0/2004628334 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744011830 con 0x7f77540a1340 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f7758f6b700 1 -- 172.21.15.32:0/2004628334 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7744011990 con 0x7f77540a1340 2023-09-30T22:29:04.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f7758f6b700 1 -- 172.21.15.32:0/2004628334 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744011d20 con 0x7f77540a1340 2023-09-30T22:29:04.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 -- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 msgr2=0x7f7754133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 0x7f7754133a60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7744009920 tx=0x7f7744009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 -- 172.21.15.32:0/2004628334 shutdown_connections 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77540a0a30 0x7f77540a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 --2- 172.21.15.32:0/2004628334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a1340 0x7f7754133a60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 -- 172.21.15.32:0/2004628334 >> 172.21.15.32:0/2004628334 conn(0x7f77540a6550 msgr2=0x7f77540a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.614+0000 7f775c1d1700 1 -- 172.21.15.32:0/2004628334 shutdown_connections 2023-09-30T22:29:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.615+0000 7f775c1d1700 1 -- 172.21.15.32:0/2004628334 wait complete. 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.615+0000 7f775c1d1700 1 Processor -- start 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 -- start start 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a0a30 0x7f775411e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 0x7f775411ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7754133fa0 0x7f775411f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7754120740 con 0x7f77540a1340 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775411fe10 con 0x7f7754133fa0 2023-09-30T22:29:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775c1d1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7754120550 con 0x7f77540a0a30 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.616+0000 7f775976c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 0x7f775411ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 0x7f775411ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56388/0 (socket says 172.21.15.32:56388) 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 -- 172.21.15.32:0/3992779527 learned_addr learned my addr 172.21.15.32:0/3992779527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775a76e700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7754133fa0 0x7f775411f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f7759f6d700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a0a30 0x7f775411e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 -- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a0a30 msgr2=0x7f775411e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a0a30 0x7f775411e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 -- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7754133fa0 msgr2=0x7f775411f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7754133fa0 0x7f775411f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 -- 172.21.15.32:0/3992779527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500094a0 con 0x7f77540a1340 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775976c700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 0x7f775411ee80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7744000c00 tx=0x7f7744004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744004450 con 0x7f77540a1340 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77440045b0 con 0x7f77540a1340 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440038c0 con 0x7f77540a1340 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.617+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7744009550 con 0x7f77540a1340 2023-09-30T22:29:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.618+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77540a25e0 con 0x7f77540a1340 2023-09-30T22:29:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.618+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7744003a20 con 0x7f77540a1340 2023-09-30T22:29:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.619+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7754074ea0 con 0x7f77540a1340 2023-09-30T22:29:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.619+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7744021030 con 0x7f77540a1340 2023-09-30T22:29:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.624+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f774402b080 con 0x7f77540a1340 2023-09-30T22:29:04.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.755+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f60005590 con 0x7f6f8009e110 2023-09-30T22:29:04.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.755+0000 7f6f75ffb700 1 -- 172.21.15.32:0/3950408824 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f7002a090 con 0x7f6f8009e110 2023-09-30T22:29:04.757 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 msgr2=0x7f6f80121700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 0x7f6f80121700 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000e9b0 tx=0x7f6f7000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 shutdown_connections 2023-09-30T22:29:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f80133da0 0x7f6f80121c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f8009d800 0x7f6f801211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 --2- 172.21.15.32:0/3950408824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f8009e110 0x7f6f80121700 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 >> 172.21.15.32:0/3950408824 conn(0x7f6f800a2e20 msgr2=0x7f6f801330d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 shutdown_connections 2023-09-30T22:29:04.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.758+0000 7f6f8679f700 1 -- 172.21.15.32:0/3950408824 wait complete. 2023-09-30T22:29:04.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:04.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.794+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff314074ea0 con 0x7ff3140a1340 2023-09-30T22:29:04.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.794+0000 7ff310ff9700 1 -- 172.21.15.32:0/301303920 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2fc019770 con 0x7ff3140a1340 2023-09-30T22:29:04.795 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 msgr2=0x7ff31411ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 0x7ff31411ec60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff31409dd90 tx=0x7ff2fc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 shutdown_connections 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff314133fa0 0x7ff31411f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff3140a0a30 0x7ff31411e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 --2- 172.21.15.32:0/301303920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3140a1340 0x7ff31411ec60 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 >> 172.21.15.32:0/301303920 conn(0x7ff3140a6550 msgr2=0x7ff3140a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.796+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 shutdown_connections 2023-09-30T22:29:04.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.797+0000 7ff31a24c700 1 -- 172.21.15.32:0/301303920 wait complete. 2023-09-30T22:29:04.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:04.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.832+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f77540a28e0 con 0x7f77540a1340 2023-09-30T22:29:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.832+0000 7f774a7fc700 1 -- 172.21.15.32:0/3992779527 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7744015020 con 0x7f77540a1340 2023-09-30T22:29:04.833 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 msgr2=0x7f775411ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 0x7f775411ee80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7744000c00 tx=0x7f7744004750 comp rx=0 tx=0).stop 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 shutdown_connections 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7754133fa0 0x7f775411f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77540a0a30 0x7f775411e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 --2- 172.21.15.32:0/3992779527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77540a1340 0x7f775411ee80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 >> 172.21.15.32:0/3992779527 conn(0x7f77540a6550 msgr2=0x7f77541b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 shutdown_connections 2023-09-30T22:29:04.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:04.834+0000 7f775c1d1700 1 -- 172.21.15.32:0/3992779527 wait complete. 2023-09-30T22:29:04.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:04.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.053+0000 7f08b2c1a700 1 Processor -- start 2023-09-30T22:29:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.053+0000 7f08b2c1a700 1 -- start start 2023-09-30T22:29:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b2c1a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 0x7f08ac131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b2c1a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac09d9f0 0x7f08ac06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b2c1a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08ac06acb0 0x7f08ac06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b2c1a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac09e130 con 0x7f08ac06acb0 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b2c1a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1312b0 con 0x7f08ac1316c0 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b2c1a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac09f6e0 con 0x7f08ac09d9f0 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b09b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 0x7f08ac131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08ac06acb0 0x7f08ac06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac09d9f0 0x7f08ac06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08ac06acb0 0x7f08ac06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56398/0 (socket says 172.21.15.32:56398) 2023-09-30T22:29:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b09b6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 0x7f08ac131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39954/0 (socket says 172.21.15.32:39954) 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 -- 172.21.15.32:0/2740046544 learned_addr learned my addr 172.21.15.32:0/2740046544 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 -- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac09d9f0 msgr2=0x7f08ac06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac09d9f0 0x7f08ac06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 -- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 msgr2=0x7f08ac131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 0x7f08ac131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 -- 172.21.15.32:0/2740046544 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac09e760 con 0x7f08ac06acb0 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b09b6700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 0x7f08ac131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.054+0000 7f08b11b7700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac06acb0 0x7f08ac06b110 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b18f14ad8b8156 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08ab7fe700 1 -- 172.21.15.32:0/2740046544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400e070 con 0x7f08ac06acb0 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08ab7fe700 1 -- 172.21.15.32:0/2740046544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400b910 con 0x7f08ac06acb0 2023-09-30T22:29:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08ab7fe700 1 -- 172.21.15.32:0/2740046544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400bcc0 con 0x7f08ac06acb0 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 -- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac06acb0 msgr2=0x7f08ac06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac06acb0 0x7f08ac06b110 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 -- 172.21.15.32:0/2740046544 shutdown_connections 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac1316c0 0x7f08ac131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac06acb0 0x7f08ac06b110 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/2740046544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac09d9f0 0x7f08ac06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 -- 172.21.15.32:0/2740046544 >> 172.21.15.32:0/2740046544 conn(0x7f08ac066f50 msgr2=0x7f08ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 -- 172.21.15.32:0/2740046544 shutdown_connections 2023-09-30T22:29:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.055+0000 7f08b2c1a700 1 -- 172.21.15.32:0/2740046544 wait complete. 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.056+0000 7f08b2c1a700 1 Processor -- start 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.056+0000 7f08b2c1a700 1 -- start start 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b2c1a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac06acb0 0x7f08ac0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b2c1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 0x7f08ac070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b2c1a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 0x7f08ac071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b2c1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac119740 con 0x7f08ac09d9f0 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b2c1a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac11adc0 con 0x7f08ac06acb0 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b2c1a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0a2680 con 0x7f08ac1316c0 2023-09-30T22:29:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 0x7f08ac070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b11b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 0x7f08ac071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 0x7f08ac070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56402/0 (socket says 172.21.15.32:56402) 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 -- 172.21.15.32:0/804076848 learned_addr learned my addr 172.21.15.32:0/804076848 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b11b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 0x7f08ac071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37996/0 (socket says 172.21.15.32:37996) 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b09b6700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac06acb0 0x7f08ac0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 -- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 msgr2=0x7f08ac071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 0x7f08ac071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 -- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac06acb0 msgr2=0x7f08ac0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac06acb0 0x7f08ac0709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08abfff700 1 -- 172.21.15.32:0/804076848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0094a0 con 0x7f08ac09d9f0 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b09b6700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac06acb0 0x7f08ac0709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.057+0000 7f08b11b7700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 0x7f08ac071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.058+0000 7f08abfff700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 0x7f08ac070f20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f08a000e980 tx=0x7f08a000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.058+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0013680 con 0x7f08ac09d9f0 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.058+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a00137e0 con 0x7f08ac09d9f0 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.058+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a4009b30 con 0x7f08ac09d9f0 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.058+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0013b70 con 0x7f08ac09d9f0 2023-09-30T22:29:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.058+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac11a820 con 0x7f08ac09d9f0 2023-09-30T22:29:05.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.059+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08a0013cd0 con 0x7f08ac09d9f0 2023-09-30T22:29:05.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.059+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f08a003f660 con 0x7f08ac09d9f0 2023-09-30T22:29:05.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.059+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac074ea0 con 0x7f08ac09d9f0 2023-09-30T22:29:05.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.064+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08a000c970 con 0x7f08ac09d9f0 2023-09-30T22:29:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.270+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08ac09e980 con 0x7f08ac09d9f0 2023-09-30T22:29:05.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.271+0000 7f08a97fa700 1 -- 172.21.15.32:0/804076848 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08a0021030 con 0x7f08ac09d9f0 2023-09-30T22:29:05.272 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 msgr2=0x7f08ac070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 0x7f08ac070f20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f08a000e980 tx=0x7f08a000ec90 comp rx=0 tx=0).stop 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 shutdown_connections 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08ac1316c0 0x7f08ac071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08ac06acb0 0x7f08ac0709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 --2- 172.21.15.32:0/804076848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac09d9f0 0x7f08ac070f20 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 >> 172.21.15.32:0/804076848 conn(0x7f08ac066f50 msgr2=0x7f08ac0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 shutdown_connections 2023-09-30T22:29:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.273+0000 7f08b2c1a700 1 -- 172.21.15.32:0/804076848 wait complete. 2023-09-30T22:29:05.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:05.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:05.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:05.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:05.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.927+0000 7f9283acd700 1 Processor -- start 2023-09-30T22:29:05.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.927+0000 7f9283acd700 1 -- start start 2023-09-30T22:29:05.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f9283acd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c1345a0 0x7f927c136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f9283acd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 0x7f927c1392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f9283acd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c099620 con 0x7f927c0984f0 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f9283acd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c09a9e0 con 0x7f927c136ec0 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f9283acd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f927c09bce0 con 0x7f927c1345a0 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 0x7f927c1392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f9281068700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c1345a0 0x7f927c136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 0x7f927c1392a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39978/0 (socket says 172.21.15.32:39978) 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 -- 172.21.15.32:0/716773617 learned_addr learned my addr 172.21.15.32:0/716773617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 -- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c1345a0 msgr2=0x7f927c136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 --2- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c1345a0 0x7f927c136980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 -- 172.21.15.32:0/716773617 >> v1:172.21.15.32:6789/0 conn(0x7f927c0984f0 legacy=0x7f927c0988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 -- 172.21.15.32:0/716773617 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c09aca0 con 0x7f927c136ec0 2023-09-30T22:29:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.928+0000 7f928206a700 1 --2- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 0x7f927c1392a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9274009e80 tx=0x7f927400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17668188b7eae642 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9280867700 1 -- 172.21.15.32:0/716773617 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927400e070 con 0x7f927c136ec0 2023-09-30T22:29:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9280867700 1 -- 172.21.15.32:0/716773617 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927400b910 con 0x7f927c136ec0 2023-09-30T22:29:05.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9280867700 1 -- 172.21.15.32:0/716773617 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927400bcc0 con 0x7f927c136ec0 2023-09-30T22:29:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 -- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 msgr2=0x7f927c1392a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 --2- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 0x7f927c1392a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9274009e80 tx=0x7f927400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 -- 172.21.15.32:0/716773617 shutdown_connections 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 --2- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c136ec0 0x7f927c1392a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 --2- 172.21.15.32:0/716773617 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c1345a0 0x7f927c136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 -- 172.21.15.32:0/716773617 >> 172.21.15.32:0/716773617 conn(0x7f927c09f470 msgr2=0x7f927c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.929+0000 7f9283acd700 1 -- 172.21.15.32:0/716773617 shutdown_connections 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.930+0000 7f9283acd700 1 -- 172.21.15.32:0/716773617 wait complete. 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.930+0000 7f9283acd700 1 Processor -- start 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 -- start start 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c0984f0 0x7f927c097090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 0x7f927c0975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927c136ec0 0x7f927c097b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0a1d70 con 0x7f927c136ec0 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c12c6a0 con 0x7f927c1345a0 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9283acd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f927c072660 con 0x7f927c0984f0 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9281068700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 0x7f927c0975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9281068700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 0x7f927c0975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39990/0 (socket says 172.21.15.32:39990) 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9281068700 1 -- 172.21.15.32:0/506183030 learned_addr learned my addr 172.21.15.32:0/506183030 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f928206a700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927c136ec0 0x7f927c097b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.931+0000 7f9281869700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c0984f0 0x7f927c097090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9281068700 1 -- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c0984f0 msgr2=0x7f927c097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9281068700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c0984f0 0x7f927c097090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9281068700 1 -- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927c136ec0 msgr2=0x7f927c097b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9281068700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927c136ec0 0x7f927c097b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9281068700 1 -- 172.21.15.32:0/506183030 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009b30 con 0x7f927c1345a0 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9281068700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 0x7f927c0975d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f926c00e9b0 tx=0x7f926c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c009710 con 0x7f927c1345a0 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c00ee50 con 0x7f927c1345a0 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c09a2b0 con 0x7f927c1345a0 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c12ebc0 con 0x7f927c1345a0 2023-09-30T22:29:05.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.932+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00ca50 con 0x7f927c1345a0 2023-09-30T22:29:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.934+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92640052e0 con 0x7f927c1345a0 2023-09-30T22:29:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.934+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f926c00cbb0 con 0x7f927c1345a0 2023-09-30T22:29:05.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.934+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f926c03fea0 con 0x7f927c1345a0 2023-09-30T22:29:05.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.939+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f926c013470 con 0x7f927c1345a0 2023-09-30T22:29:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 Processor -- start 2023-09-30T22:29:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 -- start start 2023-09-30T22:29:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff43c0a0a30 0x7ff43c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 0x7ff43c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c09edd0 con 0x7ff43c133fa0 2023-09-30T22:29:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c09e1f0 con 0x7ff43c0a0a30 2023-09-30T22:29:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff441e52700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c09dc50 con 0x7ff43c0a1340 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff43affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 0x7ff43c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff43b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff43c0a0a30 0x7ff43c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.965+0000 7ff43affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 0x7ff43c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39996/0 (socket says 172.21.15.32:39996) 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43affd700 1 -- 172.21.15.32:0/1415386348 learned_addr learned my addr 172.21.15.32:0/1415386348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43affd700 1 -- 172.21.15.32:0/1415386348 >> v2:172.21.15.32:3300/0 conn(0x7ff43c0a0a30 msgr2=0x7ff43c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43affd700 1 --2- 172.21.15.32:0/1415386348 >> v2:172.21.15.32:3300/0 conn(0x7ff43c0a0a30 0x7ff43c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43affd700 1 -- 172.21.15.32:0/1415386348 >> v1:172.21.15.32:6789/0 conn(0x7ff43c133fa0 legacy=0x7ff43c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43affd700 1 -- 172.21.15.32:0/1415386348 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0a1fe0 con 0x7ff43c0a1340 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43affd700 1 --2- 172.21.15.32:0/1415386348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 0x7ff43c133a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff4300098f0 tx=0x7ff430009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f2479a8aaf12ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43a7fc700 1 -- 172.21.15.32:0/1415386348 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430011830 con 0x7ff43c0a1340 2023-09-30T22:29:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43a7fc700 1 -- 172.21.15.32:0/1415386348 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff430011990 con 0x7ff43c0a1340 2023-09-30T22:29:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.966+0000 7ff43a7fc700 1 -- 172.21.15.32:0/1415386348 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430011d20 con 0x7ff43c0a1340 2023-09-30T22:29:05.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 -- 172.21.15.32:0/1415386348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 msgr2=0x7ff43c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 --2- 172.21.15.32:0/1415386348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 0x7ff43c133a60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff4300098f0 tx=0x7ff430009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 -- 172.21.15.32:0/1415386348 shutdown_connections 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 --2- 172.21.15.32:0/1415386348 >> v2:172.21.15.32:3300/0 conn(0x7ff43c0a0a30 0x7ff43c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 --2- 172.21.15.32:0/1415386348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a1340 0x7ff43c133a60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 -- 172.21.15.32:0/1415386348 >> 172.21.15.32:0/1415386348 conn(0x7ff43c0a6550 msgr2=0x7ff43c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 -- 172.21.15.32:0/1415386348 shutdown_connections 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.967+0000 7ff441e52700 1 -- 172.21.15.32:0/1415386348 wait complete. 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.968+0000 7ff441e52700 1 Processor -- start 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.968+0000 7ff441e52700 1 -- start start 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff441e52700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 0x7ff43c11e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff441e52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff43c0a1340 0x7ff43c11ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff441e52700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff43c133fa0 0x7ff43c11f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff441e52700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c1208f0 con 0x7ff43c0a1340 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff441e52700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0a0220 con 0x7ff43c0a0a30 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff441e52700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c1203b0 con 0x7ff43c133fa0 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 0x7ff43c11e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff43c0a1340 0x7ff43c11ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 0x7ff43c11e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40004/0 (socket says 172.21.15.32:40004) 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43b7fe700 1 -- 172.21.15.32:0/476213459 learned_addr learned my addr 172.21.15.32:0/476213459 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43bfff700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff43c133fa0 0x7ff43c11f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43affd700 1 -- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff43c133fa0 msgr2=0x7ff43c11f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.969+0000 7ff43affd700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff43c133fa0 0x7ff43c11f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff43affd700 1 -- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 msgr2=0x7ff43c11e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff43affd700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 0x7ff43c11e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff43affd700 1 -- 172.21.15.32:0/476213459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff430009550 con 0x7ff43c0a1340 2023-09-30T22:29:05.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff43b7fe700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 0x7ff43c11e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:05.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff43affd700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff43c0a1340 0x7ff43c11ec60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff430016000 tx=0x7ff430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430004400 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43001f070 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4300038e0 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c121ed0 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.970+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c1221b0 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.971+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff430021030 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.971+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff430040840 con 0x7ff43c0a1340 2023-09-30T22:29:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.971+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c0a1d40 con 0x7ff43c0a1340 2023-09-30T22:29:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:05.976+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff430019980 con 0x7ff43c0a1340 2023-09-30T22:29:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.006+0000 7f0e49560700 1 Processor -- start 2023-09-30T22:29:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.006+0000 7f0e49560700 1 -- start start 2023-09-30T22:29:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e49560700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e49560700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e4409e000 0x7f0e44130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e49560700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4409f130 con 0x7f0e4409e410 2023-09-30T22:29:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e49560700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4409f520 con 0x7f0e4409e000 2023-09-30T22:29:06.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e49560700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4409e930 con 0x7f0e4409d6f0 2023-09-30T22:29:06.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e4409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38024/0 (socket says 172.21.15.32:38024) 2023-09-30T22:29:06.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1054659274 learned_addr learned my addr 172.21.15.32:0/1054659274 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:06.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e427fc700 1 --2- 172.21.15.32:0/1054659274 >> v2:172.21.15.32:3300/0 conn(0x7f0e4409e000 0x7f0e44130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1054659274 >> v2:172.21.15.32:3300/0 conn(0x7f0e4409e000 msgr2=0x7f0e44130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:06.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e42ffd700 1 --2- 172.21.15.32:0/1054659274 >> v2:172.21.15.32:3300/0 conn(0x7f0e4409e000 0x7f0e44130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.007+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1054659274 >> v1:172.21.15.32:6789/0 conn(0x7f0e4409e410 legacy=0x7f0e44130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e42ffd700 1 -- 172.21.15.32:0/1054659274 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4409b2d0 con 0x7f0e4409d6f0 2023-09-30T22:29:06.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e41ffb700 1 -- 172.21.15.32:0/1054659274 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68602058 0 0) 0x7f0e4409f130 con 0x7f0e4409e410 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e427fc700 1 --2- 172.21.15.32:0/1054659274 >> v2:172.21.15.32:3300/0 conn(0x7f0e4409e000 0x7f0e44130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e42ffd700 1 --2- 172.21.15.32:0/1054659274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e4409dac0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098f0 tx=0x7f0e34009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb50779db27bef89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e41ffb700 1 -- 172.21.15.32:0/1054659274 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34011830 con 0x7f0e4409d6f0 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e41ffb700 1 -- 172.21.15.32:0/1054659274 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e34011990 con 0x7f0e4409d6f0 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.008+0000 7f0e41ffb700 1 -- 172.21.15.32:0/1054659274 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34011d20 con 0x7f0e4409d6f0 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 -- 172.21.15.32:0/1054659274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 msgr2=0x7f0e4409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 --2- 172.21.15.32:0/1054659274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e4409dac0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0e340098f0 tx=0x7f0e34009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 -- 172.21.15.32:0/1054659274 shutdown_connections 2023-09-30T22:29:06.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 --2- 172.21.15.32:0/1054659274 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e4409dac0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 --2- 172.21.15.32:0/1054659274 >> v2:172.21.15.32:3300/0 conn(0x7f0e4409e000 0x7f0e44130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 -- 172.21.15.32:0/1054659274 >> 172.21.15.32:0/1054659274 conn(0x7f0e44066f50 msgr2=0x7f0e4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:06.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 -- 172.21.15.32:0/1054659274 shutdown_connections 2023-09-30T22:29:06.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.009+0000 7f0e49560700 1 -- 172.21.15.32:0/1054659274 wait complete. 2023-09-30T22:29:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 Processor -- start 2023-09-30T22:29:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 -- start start 2023-09-30T22:29:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e44122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 0x7f0e44122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e4409e410 0x7f0e44123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441245e0 con 0x7f0e4409e000 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441260d0 con 0x7f0e4409e410 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.010+0000 7f0e49560700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440a2120 con 0x7f0e4409d6f0 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 0x7f0e44122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 0x7f0e44122b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56470/0 (socket says 172.21.15.32:56470) 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 -- 172.21.15.32:0/54844993 learned_addr learned my addr 172.21.15.32:0/54844993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e42ffd700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e44122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e437fe700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e4409e410 0x7f0e44123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 -- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 msgr2=0x7f0e44122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e44122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 -- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e4409e410 msgr2=0x7f0e44123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e4409e410 0x7f0e44123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 -- 172.21.15.32:0/54844993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3800b740 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e437fe700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e4409e410 0x7f0e44123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e427fc700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 0x7f0e44122b40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800ecf0 tx=0x7f0e3800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800de20 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38010440 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e34009550 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380107b0 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.011+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e441249b0 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.012+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44074ea0 con 0x7f0e4409e000 2023-09-30T22:29:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.013+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e38010910 con 0x7f0e4409e000 2023-09-30T22:29:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.014+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e38040010 con 0x7f0e4409e000 2023-09-30T22:29:06.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.018+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e38016b80 con 0x7f0e4409e000 2023-09-30T22:29:06.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.149+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9264005860 con 0x7f927c1345a0 2023-09-30T22:29:06.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.149+0000 7f92727fc700 1 -- 172.21.15.32:0/506183030 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f926c021020 con 0x7f927c1345a0 2023-09-30T22:29:06.150 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.151+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 msgr2=0x7f927c0975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.151+0000 7f9283acd700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 0x7f927c0975d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f926c00e9b0 tx=0x7f926c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 shutdown_connections 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f927c136ec0 0x7f927c097b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f927c0984f0 0x7f927c097090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 --2- 172.21.15.32:0/506183030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f927c1345a0 0x7f927c0975d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 >> 172.21.15.32:0/506183030 conn(0x7f927c09f470 msgr2=0x7f927c09c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 shutdown_connections 2023-09-30T22:29:06.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.152+0000 7f9283acd700 1 -- 172.21.15.32:0/506183030 wait complete. 2023-09-30T22:29:06.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:06.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.184+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff43c074ea0 con 0x7ff43c0a1340 2023-09-30T22:29:06.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.184+0000 7ff423fff700 1 -- 172.21.15.32:0/476213459 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff430015020 con 0x7ff43c0a1340 2023-09-30T22:29:06.185 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff43c0a1340 msgr2=0x7ff43c11ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff43c0a1340 0x7ff43c11ec60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff430016000 tx=0x7ff430004750 comp rx=0 tx=0).stop 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 shutdown_connections 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff43c133fa0 0x7ff43c11f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff43c0a0a30 0x7ff43c11e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 --2- 172.21.15.32:0/476213459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff43c0a1340 0x7ff43c11ec60 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 >> 172.21.15.32:0/476213459 conn(0x7ff43c0a6550 msgr2=0x7ff43c1bc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:06.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 shutdown_connections 2023-09-30T22:29:06.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.187+0000 7ff441e52700 1 -- 172.21.15.32:0/476213459 wait complete. 2023-09-30T22:29:06.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:06.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.225+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e44125c50 con 0x7f0e4409e000 2023-09-30T22:29:06.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.225+0000 7f0e2b7fe700 1 -- 172.21.15.32:0/54844993 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e44125c50 con 0x7f0e4409e000 2023-09-30T22:29:06.226 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 msgr2=0x7f0e44122b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 0x7f0e44122b40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800ecf0 tx=0x7f0e3800bf70 comp rx=0 tx=0).stop 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 shutdown_connections 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e4409e410 0x7f0e44123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e4409d6f0 0x7f0e44122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 --2- 172.21.15.32:0/54844993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e4409e000 0x7f0e44122b40 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 >> 172.21.15.32:0/54844993 conn(0x7f0e44066f50 msgr2=0x7f0e44070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 shutdown_connections 2023-09-30T22:29:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.227+0000 7f0e49560700 1 -- 172.21.15.32:0/54844993 wait complete. 2023-09-30T22:29:06.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:06.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.428+0000 7f777336a700 1 Processor -- start 2023-09-30T22:29:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.428+0000 7f777336a700 1 -- start start 2023-09-30T22:29:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.429+0000 7f777336a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.429+0000 7f777336a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.429+0000 7f777336a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c09b120 con 0x7f776c09e000 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.429+0000 7f777336a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c06b460 con 0x7f776c09d6f0 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.429+0000 7f777336a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f776c09a7e0 con 0x7f776c09e410 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.429+0000 7f7771106700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771907700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771106700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40022/0 (socket says 172.21.15.32:40022) 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7770905700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f776c09e000 0x7f776c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54252/0 (socket says 172.21.15.32:54252) 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771106700 1 -- 172.21.15.32:0/2227677848 learned_addr learned my addr 172.21.15.32:0/2227677848 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771907700 1 -- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 msgr2=0x7f776c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771907700 1 --2- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771907700 1 -- 172.21.15.32:0/2227677848 >> v1:172.21.15.32:6789/0 conn(0x7f776c09e000 legacy=0x7f776c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7763fff700 1 -- 172.21.15.32:0/2227677848 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032233525 0 0) 0x7f776c09b120 con 0x7f776c09e000 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771907700 1 -- 172.21.15.32:0/2227677848 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c09a500 con 0x7f776c09e410 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771106700 1 --2- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7771907700 1 --2- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c134100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7764009df0 tx=0x7f776400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68bec74be75b5cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7763fff700 1 -- 172.21.15.32:0/2227677848 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776400e070 con 0x7f776c09e410 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7763fff700 1 -- 172.21.15.32:0/2227677848 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776400bae0 con 0x7f776c09e410 2023-09-30T22:29:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.430+0000 7f7763fff700 1 -- 172.21.15.32:0/2227677848 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776400be90 con 0x7f776c09e410 2023-09-30T22:29:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 -- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 msgr2=0x7f776c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 --2- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c134100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7764009df0 tx=0x7f776400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 -- 172.21.15.32:0/2227677848 shutdown_connections 2023-09-30T22:29:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 --2- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c134100 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 --2- 172.21.15.32:0/2227677848 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 -- 172.21.15.32:0/2227677848 >> 172.21.15.32:0/2227677848 conn(0x7f776c066f50 msgr2=0x7f776c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 -- 172.21.15.32:0/2227677848 shutdown_connections 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.431+0000 7f777336a700 1 -- 172.21.15.32:0/2227677848 wait complete. 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.432+0000 7f777336a700 1 Processor -- start 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.432+0000 7f777336a700 1 -- start start 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.432+0000 7f777336a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.432+0000 7f777336a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c09e000 0x7f776c122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.432+0000 7f777336a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f777336a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c1237f0 con 0x7f776c09e000 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f777336a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c124e00 con 0x7f776c09d6f0 2023-09-30T22:29:06.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f777336a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f776c125e40 con 0x7f776c09e410 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771907700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7770905700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c09e000 0x7f776c122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40024/0 (socket says 172.21.15.32:40024) 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 -- 172.21.15.32:0/3707937172 learned_addr learned my addr 172.21.15.32:0/3707937172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 -- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 msgr2=0x7f776c122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 -- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c09e000 msgr2=0x7f776c122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c09e000 0x7f776c122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 -- 172.21.15.32:0/3707937172 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768009550 con 0x7f776c09d6f0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7770905700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c09e000 0x7f776c122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7771106700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c122320 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f776800ec10 tx=0x7f776800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776800c850 con 0x7f776c09d6f0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776800c9b0 con 0x7f776c09d6f0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776800cd40 con 0x7f776c09d6f0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.433+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7764009a70 con 0x7f776c09d6f0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.434+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c0a2020 con 0x7f776c09d6f0 2023-09-30T22:29:06.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.435+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7768013470 con 0x7f776c09d6f0 2023-09-30T22:29:06.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.435+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c11f030 con 0x7f776c09d6f0 2023-09-30T22:29:06.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.435+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f776803f890 con 0x7f776c09d6f0 2023-09-30T22:29:06.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.440+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7768013720 con 0x7f776c09d6f0 2023-09-30T22:29:06.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.643+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f776c074ea0 con 0x7f776c09d6f0 2023-09-30T22:29:06.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.644+0000 7f7761ffb700 1 -- 172.21.15.32:0/3707937172 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f776801bb60 con 0x7f776c09d6f0 2023-09-30T22:29:06.645 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:06.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 msgr2=0x7f776c122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:06.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c122320 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f776800ec10 tx=0x7f776800ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:06.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 shutdown_connections 2023-09-30T22:29:06.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f776c09e410 0x7f776c122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f776c09d6f0 0x7f776c122320 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 --2- 172.21.15.32:0/3707937172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f776c09e000 0x7f776c122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:06.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 >> 172.21.15.32:0/3707937172 conn(0x7f776c066f50 msgr2=0x7f776c133de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:06.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 shutdown_connections 2023-09-30T22:29:06.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:06.646+0000 7f777336a700 1 -- 172.21.15.32:0/3707937172 wait complete. 2023-09-30T22:29:06.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:07.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:07.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:07.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:07.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.318+0000 7fb15f5ab700 1 Processor -- start 2023-09-30T22:29:07.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.318+0000 7fb15f5ab700 1 -- start start 2023-09-30T22:29:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15f5ab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 0x7fb158131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15f5ab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb15809d9f0 0x7fb15806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15f5ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb15806acb0 0x7fb15806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15f5ab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15806b630 con 0x7fb15809d9f0 2023-09-30T22:29:07.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15f5ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15809e130 con 0x7fb15806acb0 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15f5ab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1581312b0 con 0x7fb1581316c0 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15d347700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 0x7fb158131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15cb46700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb15809d9f0 0x7fb15806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15db48700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb15806acb0 0x7fb15806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15d347700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 0x7fb158131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38054/0 (socket says 172.21.15.32:38054) 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.319+0000 7fb15d347700 1 -- 172.21.15.32:0/1962100990 learned_addr learned my addr 172.21.15.32:0/1962100990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15cb46700 1 -- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 msgr2=0x7fb158131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15cb46700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 0x7fb158131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15cb46700 1 -- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb15806acb0 msgr2=0x7fb15806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:07.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15cb46700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb15806acb0 0x7fb15806b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15cb46700 1 -- 172.21.15.32:0/1962100990 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15809f6e0 con 0x7fb15809d9f0 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15d347700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 0x7fb158131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15db48700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb15806acb0 0x7fb15806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb15cb46700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb15806a770 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb148009840 tx=0x7fb148009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6c7600c3bf19a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb14ffff700 1 -- 172.21.15.32:0/1962100990 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148011830 con 0x7fb15809d9f0 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb14ffff700 1 -- 172.21.15.32:0/1962100990 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb148011990 con 0x7fb15809d9f0 2023-09-30T22:29:07.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.320+0000 7fb14ffff700 1 -- 172.21.15.32:0/1962100990 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148011d20 con 0x7fb15809d9f0 2023-09-30T22:29:07.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 -- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 msgr2=0x7fb15806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb15806a770 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb148009840 tx=0x7fb148009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 -- 172.21.15.32:0/1962100990 shutdown_connections 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb15806acb0 0x7fb15806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1581316c0 0x7fb158131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/1962100990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb15806a770 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 -- 172.21.15.32:0/1962100990 >> 172.21.15.32:0/1962100990 conn(0x7fb158066f50 msgr2=0x7fb15809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 -- 172.21.15.32:0/1962100990 shutdown_connections 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.321+0000 7fb15f5ab700 1 -- 172.21.15.32:0/1962100990 wait complete. 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 Processor -- start 2023-09-30T22:29:07.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 -- start start 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 0x7fb158119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb1581199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1581316c0 0x7fb158119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15811b650 con 0x7fb15809d9f0 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb158072740 con 0x7fb1581316c0 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.322+0000 7fb15f5ab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1580722a0 con 0x7fb15806acb0 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 0x7fb158119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15cb46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb1581199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 0x7fb158119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38062/0 (socket says 172.21.15.32:38062) 2023-09-30T22:29:07.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 -- 172.21.15.32:0/2613343770 learned_addr learned my addr 172.21.15.32:0/2613343770 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15db48700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1581316c0 0x7fb158119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 -- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1581316c0 msgr2=0x7fb158119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1581316c0 0x7fb158119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 -- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 msgr2=0x7fb1581199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb1581199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 -- 172.21.15.32:0/2613343770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150014660 con 0x7fb15806acb0 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb15d347700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 0x7fb158119480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb15400bef0 tx=0x7fb15400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.323+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540089b0 con 0x7fb15806acb0 2023-09-30T22:29:07.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.324+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154008b10 con 0x7fb15806acb0 2023-09-30T22:29:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.324+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15400d9f0 con 0x7fb15806acb0 2023-09-30T22:29:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.324+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154008e80 con 0x7fb15806acb0 2023-09-30T22:29:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.324+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1480094a0 con 0x7fb15806acb0 2023-09-30T22:29:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.325+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb158074ea0 con 0x7fb15806acb0 2023-09-30T22:29:07.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.325+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb154015400 con 0x7fb15806acb0 2023-09-30T22:29:07.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.325+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb15403f000 con 0x7fb15806acb0 2023-09-30T22:29:07.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.330+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb15401c2c0 con 0x7fb15806acb0 2023-09-30T22:29:07.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 Processor -- start 2023-09-30T22:29:07.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 -- start start 2023-09-30T22:29:07.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 0x7f1da00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a1340 0x7f1da0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da009edd0 con 0x7f1da00a0a30 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da009e1f0 con 0x7f1da0133fa0 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.358+0000 7f1da5c99700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1da009dc50 con 0x7f1da00a1340 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 0x7f1da00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 0x7f1da00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56508/0 (socket says 172.21.15.32:56508) 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40058/0 (socket says 172.21.15.32:40058) 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9f7fe700 1 -- 172.21.15.32:0/2639525143 learned_addr learned my addr 172.21.15.32:0/2639525143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d97fff700 1 --2- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a1340 0x7f1da0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 -- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a1340 msgr2=0x7f1da0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a1340 0x7f1da0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 -- 172.21.15.32:0/2639525143 >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 msgr2=0x7f1da00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/2639525143 >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 0x7f1da00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 -- 172.21.15.32:0/2639525143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00a1fe0 con 0x7f1da0133fa0 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9f7fe700 1 --2- 172.21.15.32:0/2639525143 >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 0x7f1da00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0134410 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009dc0 tx=0x7f1d9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c09085db9e5ea2d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9effd700 1 -- 172.21.15.32:0/2639525143 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800e070 con 0x7f1da0133fa0 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9effd700 1 -- 172.21.15.32:0/2639525143 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9800bae0 con 0x7f1da0133fa0 2023-09-30T22:29:07.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.359+0000 7f1d9effd700 1 -- 172.21.15.32:0/2639525143 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800be90 con 0x7f1da0133fa0 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 -- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 msgr2=0x7f1da0134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 --2- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0134410 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009dc0 tx=0x7f1d9800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 -- 172.21.15.32:0/2639525143 shutdown_connections 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 --2- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0134410 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 --2- 172.21.15.32:0/2639525143 >> v2:172.21.15.32:3300/0 conn(0x7f1da00a0a30 0x7f1da00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 --2- 172.21.15.32:0/2639525143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a1340 0x7f1da0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 -- 172.21.15.32:0/2639525143 >> 172.21.15.32:0/2639525143 conn(0x7f1da00a6550 msgr2=0x7f1da00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 -- 172.21.15.32:0/2639525143 shutdown_connections 2023-09-30T22:29:07.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.360+0000 7f1da5c99700 1 -- 172.21.15.32:0/2639525143 wait complete. 2023-09-30T22:29:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.361+0000 7f1da5c99700 1 Processor -- start 2023-09-30T22:29:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.361+0000 7f1da5c99700 1 -- start start 2023-09-30T22:29:07.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1da5c99700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a0a30 0x7f1da0120230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1da5c99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 0x7f1da0120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1da5c99700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1da5c99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0121e40 con 0x7f1da00a1340 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1da5c99700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0123090 con 0x7f1da0133fa0 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1da5c99700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1da0123b10 con 0x7f1da00a0a30 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 0x7f1da0120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d9f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a0a30 0x7f1da0120230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 0x7f1da0120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56516/0 (socket says 172.21.15.32:56516) 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 -- 172.21.15.32:0/3451426084 learned_addr learned my addr 172.21.15.32:0/3451426084 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d9f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a0a30 0x7f1da0120230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38070/0 (socket says 172.21.15.32:38070) 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d9ffff700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 -- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a0a30 msgr2=0x7f1da0120230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a0a30 0x7f1da0120230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 -- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 msgr2=0x7f1da0120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.362+0000 7f1d97fff700 1 -- 172.21.15.32:0/3451426084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90009550 con 0x7f1da00a1340 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.363+0000 7f1d97fff700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 0x7f1da0120770 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800e9b0 tx=0x7f1d8800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.363+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88009710 con 0x7f1da00a1340 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.363+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8800ee50 con 0x7f1da00a1340 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.363+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d98009a70 con 0x7f1da00a1340 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.363+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800ca50 con 0x7f1da00a1340 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.363+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da00a3480 con 0x7f1da00a1340 2023-09-30T22:29:07.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.364+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00a0280 con 0x7f1da00a1340 2023-09-30T22:29:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.364+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d8800cbb0 con 0x7f1da00a1340 2023-09-30T22:29:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.365+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d8803f9f0 con 0x7f1da00a1340 2023-09-30T22:29:07.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.369+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d8801b3b0 con 0x7f1da00a1340 2023-09-30T22:29:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.398+0000 7f16e01c4700 1 Processor -- start 2023-09-30T22:29:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.398+0000 7f16e01c4700 1 -- start start 2023-09-30T22:29:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16e01c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16e01c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16d809d630 0x7f16d80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16e01c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80a2a70 con 0x7f16d8133930 2023-09-30T22:29:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16e01c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80a01d0 con 0x7f16d809d630 2023-09-30T22:29:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16e01c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d809e2b0 con 0x7f16d80a0a30 2023-09-30T22:29:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16dd75f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f16d809d630 0x7f16d80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16ddf60700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16de761700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16d8133930 0x7f16d8133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54264/0 (socket says 172.21.15.32:54264) 2023-09-30T22:29:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.399+0000 7f16de761700 1 -- 172.21.15.32:0/2747674269 learned_addr learned my addr 172.21.15.32:0/2747674269 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16ddf60700 1 -- 172.21.15.32:0/2747674269 >> v2:172.21.15.32:3300/0 conn(0x7f16d809d630 msgr2=0x7f16d80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16ddf60700 1 --2- 172.21.15.32:0/2747674269 >> v2:172.21.15.32:3300/0 conn(0x7f16d809d630 0x7f16d80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16ddf60700 1 -- 172.21.15.32:0/2747674269 >> v1:172.21.15.32:6789/0 conn(0x7f16d8133930 legacy=0x7f16d8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16ddf60700 1 -- 172.21.15.32:0/2747674269 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d809dc50 con 0x7f16d80a0a30 2023-09-30T22:29:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16dd75f700 1 --2- 172.21.15.32:0/2747674269 >> v2:172.21.15.32:3300/0 conn(0x7f16d809d630 0x7f16d80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16ddf60700 1 --2- 172.21.15.32:0/2747674269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d80a0e00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009840 tx=0x7f16d4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ddc678feab60680 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16dcf5e700 1 -- 172.21.15.32:0/2747674269 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4011830 con 0x7f16d80a0a30 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16dcf5e700 1 -- 172.21.15.32:0/2747674269 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d4011990 con 0x7f16d80a0a30 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.400+0000 7f16dcf5e700 1 -- 172.21.15.32:0/2747674269 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4011d20 con 0x7f16d80a0a30 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 -- 172.21.15.32:0/2747674269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 msgr2=0x7f16d80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 --2- 172.21.15.32:0/2747674269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d80a0e00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009840 tx=0x7f16d4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 -- 172.21.15.32:0/2747674269 shutdown_connections 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 --2- 172.21.15.32:0/2747674269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d80a0e00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 --2- 172.21.15.32:0/2747674269 >> v2:172.21.15.32:3300/0 conn(0x7f16d809d630 0x7f16d80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 -- 172.21.15.32:0/2747674269 >> 172.21.15.32:0/2747674269 conn(0x7f16d80a3050 msgr2=0x7f16d80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 -- 172.21.15.32:0/2747674269 shutdown_connections 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.401+0000 7f16e01c4700 1 -- 172.21.15.32:0/2747674269 wait complete. 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 Processor -- start 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 -- start start 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16d809d630 0x7f16d81c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d81c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16d8133930 0x7f16d81c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8120670 con 0x7f16d8133930 2023-09-30T22:29:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8121e60 con 0x7f16d80a0a30 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.402+0000 7f16e01c4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80a0410 con 0x7f16d809d630 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16d809d630 0x7f16d81c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16dd75f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d81c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16dd75f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d81c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40084/0 (socket says 172.21.15.32:40084) 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16dd75f700 1 -- 172.21.15.32:0/579616194 learned_addr learned my addr 172.21.15.32:0/579616194 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16de761700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16d8133930 0x7f16d81c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 -- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 msgr2=0x7f16d81c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d81c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 -- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16d8133930 msgr2=0x7f16d81c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16d8133930 0x7f16d81c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 -- 172.21.15.32:0/579616194 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c80094a0 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ddf60700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16d809d630 0x7f16d81c8170 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f16d4016000 tx=0x7f16d4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4004400 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d401f070 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d40094a0 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d40038e0 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.403+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8121160 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.404+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16d4021030 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.404+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f16d4040b30 con 0x7f16d809d630 2023-09-30T22:29:07.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.404+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80747f0 con 0x7f16d809d630 2023-09-30T22:29:07.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.409+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16d4015070 con 0x7f16d809d630 2023-09-30T22:29:07.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.544+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb15809e980 con 0x7fb15806acb0 2023-09-30T22:29:07.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.545+0000 7fb14dffb700 1 -- 172.21.15.32:0/2613343770 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb15401a020 con 0x7fb15806acb0 2023-09-30T22:29:07.546 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:07.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 msgr2=0x7fb158119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 0x7fb158119480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb15400bef0 tx=0x7fb15400bf20 comp rx=0 tx=0).stop 2023-09-30T22:29:07.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 shutdown_connections 2023-09-30T22:29:07.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1581316c0 0x7fb158119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb15806acb0 0x7fb158119480 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 --2- 172.21.15.32:0/2613343770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15809d9f0 0x7fb1581199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 >> 172.21.15.32:0/2613343770 conn(0x7fb158066f50 msgr2=0x7fb15809be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 shutdown_connections 2023-09-30T22:29:07.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.547+0000 7fb15f5ab700 1 -- 172.21.15.32:0/2613343770 wait complete. 2023-09-30T22:29:07.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.577+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1da0074ea0 con 0x7f1da00a1340 2023-09-30T22:29:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.578+0000 7f1d9cff9700 1 -- 172.21.15.32:0/3451426084 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d88021070 con 0x7f1da00a1340 2023-09-30T22:29:07.579 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 msgr2=0x7f1da0120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 0x7f1da0120770 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800e9b0 tx=0x7f1d8800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 shutdown_connections 2023-09-30T22:29:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1da0133fa0 0x7f1da0120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1da00a0a30 0x7f1da0120230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 --2- 172.21.15.32:0/3451426084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1da00a1340 0x7f1da0120770 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 >> 172.21.15.32:0/3451426084 conn(0x7f1da00a6550 msgr2=0x7f1da009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 shutdown_connections 2023-09-30T22:29:07.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.580+0000 7f1da5c99700 1 -- 172.21.15.32:0/3451426084 wait complete. 2023-09-30T22:29:07.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.615+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f16d8072980 con 0x7f16d809d630 2023-09-30T22:29:07.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.616+0000 7f16ce7fc700 1 -- 172.21.15.32:0/579616194 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16d4023790 con 0x7f16d809d630 2023-09-30T22:29:07.617 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16d809d630 msgr2=0x7f16d81c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16d809d630 0x7f16d81c8170 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f16d4016000 tx=0x7f16d4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 shutdown_connections 2023-09-30T22:29:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16d8133930 0x7f16d81c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16d809d630 0x7f16d81c8170 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 --2- 172.21.15.32:0/579616194 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16d80a0a30 0x7f16d81c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 >> 172.21.15.32:0/579616194 conn(0x7f16d80a3050 msgr2=0x7f16d80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 shutdown_connections 2023-09-30T22:29:07.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.618+0000 7f16e01c4700 1 -- 172.21.15.32:0/579616194 wait complete. 2023-09-30T22:29:07.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:07.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.812+0000 7fa61256d700 1 Processor -- start 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.812+0000 7fa61256d700 1 -- start start 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa61256d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09d6f0 0x7fa60c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa61256d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa60c09e000 0x7fa60c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa61256d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c09b120 con 0x7fa60c09e410 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa61256d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c06b460 con 0x7fa60c09e000 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa61256d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c09a7e0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09d6f0 0x7fa60c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa60c09e000 0x7fa60c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa610b0a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa60c09e410 0x7fa60c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54268/0 (socket says 172.21.15.32:54268) 2023-09-30T22:29:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa610b0a700 1 -- 172.21.15.32:0/2955437065 learned_addr learned my addr 172.21.15.32:0/2955437065 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60b7fe700 1 -- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09d6f0 msgr2=0x7fa60c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60b7fe700 1 --2- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09d6f0 0x7fa60c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60b7fe700 1 -- 172.21.15.32:0/2955437065 >> v1:172.21.15.32:6789/0 conn(0x7fa60c09e410 legacy=0x7fa60c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60b7fe700 1 -- 172.21.15.32:0/2955437065 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c09a500 con 0x7fa60c09e000 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60bfff700 1 --2- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09d6f0 0x7fa60c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.813+0000 7fa60b7fe700 1 --2- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09e000 0x7fa60c1339f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009840 tx=0x7fa5f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd5044e8cc7c864 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa60affd700 1 -- 172.21.15.32:0/2955437065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4011830 con 0x7fa60c09e000 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa60affd700 1 -- 172.21.15.32:0/2955437065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f4011990 con 0x7fa60c09e000 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa60affd700 1 -- 172.21.15.32:0/2955437065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4011d20 con 0x7fa60c09e000 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 -- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09e000 msgr2=0x7fa60c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 --2- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09e000 0x7fa60c1339f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009840 tx=0x7fa5f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 -- 172.21.15.32:0/2955437065 shutdown_connections 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 --2- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09d6f0 0x7fa60c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 --2- 172.21.15.32:0/2955437065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09e000 0x7fa60c1339f0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 -- 172.21.15.32:0/2955437065 >> 172.21.15.32:0/2955437065 conn(0x7fa60c066f50 msgr2=0x7fa60c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:07.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 -- 172.21.15.32:0/2955437065 shutdown_connections 2023-09-30T22:29:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.814+0000 7fa61256d700 1 -- 172.21.15.32:0/2955437065 wait complete. 2023-09-30T22:29:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.815+0000 7fa61256d700 1 Processor -- start 2023-09-30T22:29:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 -- start start 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09d6f0 0x7fa60c1a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 0x7fa60c1a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa60c09e410 0x7fa60c1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c1a91f0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c09c710 con 0x7fa60c09e410 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa61256d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0a3160 con 0x7fa60c09e000 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 0x7fa60c1a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09d6f0 0x7fa60c1a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 0x7fa60c1a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38092/0 (socket says 172.21.15.32:38092) 2023-09-30T22:29:07.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60b7fe700 1 -- 172.21.15.32:0/842647947 learned_addr learned my addr 172.21.15.32:0/842647947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa610b0a700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa60c09e410 0x7fa60c1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60bfff700 1 -- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 msgr2=0x7fa60c1a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60bfff700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 0x7fa60c1a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60bfff700 1 -- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa60c09e410 msgr2=0x7fa60c1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60bfff700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa60c09e410 0x7fa60c1a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60bfff700 1 -- 172.21.15.32:0/842647947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc0094a0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.816+0000 7fa60b7fe700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 0x7fa60c1a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.817+0000 7fa60bfff700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09d6f0 0x7fa60c1a5fd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e9b0 tx=0x7fa5fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.817+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00ee50 con 0x7fa60c09d6f0 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.817+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00c850 con 0x7fa60c09d6f0 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.817+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f40094a0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.817+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00cc00 con 0x7fa60c09d6f0 2023-09-30T22:29:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.817+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c1a8da0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.818+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5fc013470 con 0x7fa60c09d6f0 2023-09-30T22:29:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.818+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa5fc03f6c0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.818+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60c074ea0 con 0x7fa60c09d6f0 2023-09-30T22:29:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:07.823+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5fc021080 con 0x7fa60c09d6f0 2023-09-30T22:29:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.029+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa60c06c360 con 0x7fa60c09d6f0 2023-09-30T22:29:08.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.030+0000 7fa608ff9700 1 -- 172.21.15.32:0/842647947 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5fc017020 con 0x7fa60c09d6f0 2023-09-30T22:29:08.031 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09d6f0 msgr2=0x7fa60c1a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09d6f0 0x7fa60c1a5fd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e9b0 tx=0x7fa5fc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 shutdown_connections 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa60c09e410 0x7fa60c1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa60c09d6f0 0x7fa60c1a5fd0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 --2- 172.21.15.32:0/842647947 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa60c09e000 0x7fa60c1a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 >> 172.21.15.32:0/842647947 conn(0x7fa60c066f50 msgr2=0x7fa60c1baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 shutdown_connections 2023-09-30T22:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.032+0000 7fa61256d700 1 -- 172.21.15.32:0/842647947 wait complete. 2023-09-30T22:29:08.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:08.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:08.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:08.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:08.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.720+0000 7fd5f6dfa700 1 Processor -- start 2023-09-30T22:29:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.720+0000 7fd5f6dfa700 1 -- start start 2023-09-30T22:29:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f6dfa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5f0133420 0x7fd5f01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f6dfa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0133e30 0x7fd5f012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f6dfa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f006b990 con 0x7fd5f0134260 2023-09-30T22:29:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f6dfa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f00a0590 con 0x7fd5f0133420 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f6dfa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f009fb80 con 0x7fd5f0133e30 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f5397700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd5f0134260 0x7fd5f01304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54284/0 (socket says 172.21.15.32:54284) 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f4b96700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5f0133420 0x7fd5f01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5f5397700 1 -- 172.21.15.32:0/524049851 learned_addr learned my addr 172.21.15.32:0/524049851 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.721+0000 7fd5effff700 1 --2- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0133e30 0x7fd5f012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5f4b96700 1 -- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0133e30 msgr2=0x7fd5f012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5f4b96700 1 --2- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0133e30 0x7fd5f012ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5f4b96700 1 -- 172.21.15.32:0/524049851 >> v1:172.21.15.32:6789/0 conn(0x7fd5f0134260 legacy=0x7fd5f01304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5f4b96700 1 -- 172.21.15.32:0/524049851 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f00a0030 con 0x7fd5f0133420 2023-09-30T22:29:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5ef7fe700 1 -- 172.21.15.32:0/524049851 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081904784 0 0) 0x7fd5f006b990 con 0x7fd5f0134260 2023-09-30T22:29:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5f4b96700 1 --2- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133420 0x7fd5f01337f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009840 tx=0x7fd5e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07710700fd76649 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5ef7fe700 1 -- 172.21.15.32:0/524049851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4011830 con 0x7fd5f0133420 2023-09-30T22:29:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5ef7fe700 1 -- 172.21.15.32:0/524049851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4011990 con 0x7fd5f0133420 2023-09-30T22:29:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.722+0000 7fd5ef7fe700 1 -- 172.21.15.32:0/524049851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4011d20 con 0x7fd5f0133420 2023-09-30T22:29:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133420 msgr2=0x7fd5f01337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133420 0x7fd5f01337f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009840 tx=0x7fd5e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/524049851 shutdown_connections 2023-09-30T22:29:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133420 0x7fd5f01337f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/524049851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0133e30 0x7fd5f012ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/524049851 >> 172.21.15.32:0/524049851 conn(0x7fd5f00a4210 msgr2=0x7fd5f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/524049851 shutdown_connections 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.723+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/524049851 wait complete. 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.724+0000 7fd5f6dfa700 1 Processor -- start 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.724+0000 7fd5f6dfa700 1 -- start start 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f6dfa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 0x7fd5f009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f6dfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133e30 0x7fd5f009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f6dfa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0134260 0x7fd5f009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f6dfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f011f7e0 con 0x7fd5f0133e30 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f6dfa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f009dd30 con 0x7fd5f0134260 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f6dfa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f011fd00 con 0x7fd5f0133420 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f4b96700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 0x7fd5f009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f4b96700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 0x7fd5f009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38104/0 (socket says 172.21.15.32:38104) 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f4b96700 1 -- 172.21.15.32:0/4198461396 learned_addr learned my addr 172.21.15.32:0/4198461396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5effff700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133e30 0x7fd5f009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.725+0000 7fd5f4b96700 1 -- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0134260 msgr2=0x7fd5f009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f4b96700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0134260 0x7fd5f009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f4b96700 1 -- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133e30 msgr2=0x7fd5f009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f4b96700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133e30 0x7fd5f009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f4b96700 1 -- 172.21.15.32:0/4198461396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40094a0 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f4b96700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 0x7fd5f009c480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009fd0 tx=0x7fd5e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40036a0 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e401f070 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4003a20 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f012db00 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.726+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0120600 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.727+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5e4003b80 con 0x7fd5f0133420 2023-09-30T22:29:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.727+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd5e4021070 con 0x7fd5f0133420 2023-09-30T22:29:08.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.727+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f0074ea0 con 0x7fd5f0133420 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.732+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5e401a430 con 0x7fd5f0133420 2023-09-30T22:29:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.762+0000 7efcca99c700 1 Processor -- start 2023-09-30T22:29:08.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.762+0000 7efcca99c700 1 -- start start 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.762+0000 7efcca99c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.762+0000 7efcca99c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 0x7efcc41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.762+0000 7efcca99c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcc409e410 0x7efcc4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcca99c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc406b460 con 0x7efcc409e410 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcca99c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc409b120 con 0x7efcc409e000 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcca99c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcc409a7e0 con 0x7efcc409d6f0 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 0x7efcc41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 0x7efcc41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40126/0 (socket says 172.21.15.32:40126) 2023-09-30T22:29:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38116/0 (socket says 172.21.15.32:38116) 2023-09-30T22:29:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 -- 172.21.15.32:0/3117429435 learned_addr learned my addr 172.21.15.32:0/3117429435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc8f39700 1 --2- 172.21.15.32:0/3117429435 >> v2:172.21.15.32:3300/0 conn(0x7efcc409e410 0x7efcc4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 -- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 msgr2=0x7efcc409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 --2- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 -- 172.21.15.32:0/3117429435 >> v2:172.21.15.32:3300/0 conn(0x7efcc409e410 msgr2=0x7efcc4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 --2- 172.21.15.32:0/3117429435 >> v2:172.21.15.32:3300/0 conn(0x7efcc409e410 0x7efcc4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 -- 172.21.15.32:0/3117429435 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc409a500 con 0x7efcc409e000 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc3fff700 1 --2- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.763+0000 7efcc37fe700 1 --2- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 0x7efcc41339f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009920 tx=0x7efcb8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703f5bed5e5cd689 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.764+0000 7efcc2ffd700 1 -- 172.21.15.32:0/3117429435 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011830 con 0x7efcc409e000 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.764+0000 7efcc2ffd700 1 -- 172.21.15.32:0/3117429435 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb8011990 con 0x7efcc409e000 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.764+0000 7efcc2ffd700 1 -- 172.21.15.32:0/3117429435 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011d20 con 0x7efcc409e000 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 -- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 msgr2=0x7efcc41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 --2- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 0x7efcc41339f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009920 tx=0x7efcb8009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 -- 172.21.15.32:0/3117429435 shutdown_connections 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 --2- 172.21.15.32:0/3117429435 >> v2:172.21.15.32:3300/0 conn(0x7efcc409e410 0x7efcc4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 --2- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 --2- 172.21.15.32:0/3117429435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e000 0x7efcc41339f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 -- 172.21.15.32:0/3117429435 >> 172.21.15.32:0/3117429435 conn(0x7efcc4066f50 msgr2=0x7efcc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 -- 172.21.15.32:0/3117429435 shutdown_connections 2023-09-30T22:29:08.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.765+0000 7efcca99c700 1 -- 172.21.15.32:0/3117429435 wait complete. 2023-09-30T22:29:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.766+0000 7efcca99c700 1 Processor -- start 2023-09-30T22:29:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 -- start start 2023-09-30T22:29:08.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc41a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 0x7efcc41a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e410 0x7efcc41a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc41a8090 con 0x7efcc409e000 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc41a9150 con 0x7efcc409e410 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcca99c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcc409c710 con 0x7efcc409d6f0 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcc3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc41a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcc37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 0x7efcc41a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcc37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 0x7efcc41a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56592/0 (socket says 172.21.15.32:56592) 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcc3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc41a5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38124/0 (socket says 172.21.15.32:38124) 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.767+0000 7efcc37fe700 1 -- 172.21.15.32:0/3289979195 learned_addr learned my addr 172.21.15.32:0/3289979195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:08.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc8f39700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e410 0x7efcc41a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc37fe700 1 -- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 msgr2=0x7efcc41a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc37fe700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc41a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc37fe700 1 -- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e410 msgr2=0x7efcc41a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc37fe700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e410 0x7efcc41a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc37fe700 1 -- 172.21.15.32:0/3289979195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40094a0 con 0x7efcc409e000 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc37fe700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 0x7efcc41a6470 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7efcb8000c00 tx=0x7efcb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8004240 con 0x7efcc409e000 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb801f070 con 0x7efcc409e000 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb8009550 con 0x7efcc409e000 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80036a0 con 0x7efcc409e000 2023-09-30T22:29:08.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.768+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc41a7ab0 con 0x7efcc409e000 2023-09-30T22:29:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.769+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efcb8021030 con 0x7efcc409e000 2023-09-30T22:29:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.769+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efcb8040680 con 0x7efcc409e000 2023-09-30T22:29:08.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.770+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc41a9380 con 0x7efcc409e000 2023-09-30T22:29:08.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.775+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efcc41a9380 con 0x7efcc409e000 2023-09-30T22:29:08.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.805+0000 7fc68b979700 1 Processor -- start 2023-09-30T22:29:08.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.805+0000 7fc68b979700 1 -- start start 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc68b979700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6841338a0 0x7fc684133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc68b979700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc68b979700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 0x7fc68413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc68b979700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68406fef0 con 0x7fc6841338a0 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc68b979700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406ef30 con 0x7fc68409de50 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc68b979700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841334b0 con 0x7fc68409e760 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6841338a0 0x7fc684133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689f16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 0x7fc68413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689f16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 0x7fc68413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38128/0 (socket says 172.21.15.32:38128) 2023-09-30T22:29:08.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6841338a0 0x7fc684133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56608/0 (socket says 172.21.15.32:56608) 2023-09-30T22:29:08.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689f16700 1 -- 172.21.15.32:0/1555371475 learned_addr learned my addr 172.21.15.32:0/1555371475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:08.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc688f14700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 -- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 msgr2=0x7fc68413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 0x7fc68413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 -- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 msgr2=0x7fc68409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689715700 1 -- 172.21.15.32:0/1555371475 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68406fc30 con 0x7fc6841338a0 2023-09-30T22:29:08.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.806+0000 7fc689f16700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 0x7fc68413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.807+0000 7fc689715700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6841338a0 0x7fc684133c70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc680009840 tx=0x7fc680009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad3aac7b1942365 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.807+0000 7fc67bfff700 1 -- 172.21.15.32:0/1555371475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680011830 con 0x7fc6841338a0 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.807+0000 7fc67bfff700 1 -- 172.21.15.32:0/1555371475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680011990 con 0x7fc6841338a0 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.807+0000 7fc67bfff700 1 -- 172.21.15.32:0/1555371475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680011d20 con 0x7fc6841338a0 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.807+0000 7fc68b979700 1 -- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6841338a0 msgr2=0x7fc684133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.807+0000 7fc68b979700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6841338a0 0x7fc684133c70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc680009840 tx=0x7fc680009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 -- 172.21.15.32:0/1555371475 shutdown_connections 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc68409e760 0x7fc68413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6841338a0 0x7fc684133c70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 --2- 172.21.15.32:0/1555371475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 -- 172.21.15.32:0/1555371475 >> 172.21.15.32:0/1555371475 conn(0x7fc6840a2aa0 msgr2=0x7fc6840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 -- 172.21.15.32:0/1555371475 shutdown_connections 2023-09-30T22:29:08.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.808+0000 7fc68b979700 1 -- 172.21.15.32:0/1555371475 wait complete. 2023-09-30T22:29:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.809+0000 7fc68b979700 1 Processor -- start 2023-09-30T22:29:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.809+0000 7fc68b979700 1 -- start start 2023-09-30T22:29:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc68b979700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68411d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc68b979700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68409e760 0x7fc68411dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc68b979700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6841338a0 0x7fc68411e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc68b979700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841208d0 con 0x7fc68409e760 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc68b979700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406f640 con 0x7fc68409de50 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc68b979700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840a0f90 con 0x7fc6841338a0 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc689715700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68411d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc688f14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68409e760 0x7fc68411dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc689715700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68411d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40146/0 (socket says 172.21.15.32:40146) 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc689715700 1 -- 172.21.15.32:0/3063813847 learned_addr learned my addr 172.21.15.32:0/3063813847 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.810+0000 7fc689f16700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6841338a0 0x7fc68411e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc689715700 1 -- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6841338a0 msgr2=0x7fc68411e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc689715700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6841338a0 0x7fc68411e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc689715700 1 -- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68409e760 msgr2=0x7fc68411dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc689715700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68409e760 0x7fc68411dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc689715700 1 -- 172.21.15.32:0/3063813847 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c009b30 con 0x7fc68409de50 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc689715700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68411d6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc6800090b0 tx=0x7fc680004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680004450 con 0x7fc68409de50 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68001f070 con 0x7fc68409de50 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6800094a0 con 0x7fc68409de50 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6800038e0 con 0x7fc68409de50 2023-09-30T22:29:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.811+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc684120480 con 0x7fc68409de50 2023-09-30T22:29:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.812+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc680003a40 con 0x7fc68409de50 2023-09-30T22:29:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.812+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc680021070 con 0x7fc68409de50 2023-09-30T22:29:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.813+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68411a470 con 0x7fc68409de50 2023-09-30T22:29:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.822+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc68001a430 con 0x7fc68409de50 2023-09-30T22:29:08.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.957+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd5f011ea90 con 0x7fd5f0133420 2023-09-30T22:29:08.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.957+0000 7fd5ed7fa700 1 -- 172.21.15.32:0/4198461396 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5e402b070 con 0x7fd5f0133420 2023-09-30T22:29:08.959 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 msgr2=0x7fd5f009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 0x7fd5f009c480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009fd0 tx=0x7fd5e4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 shutdown_connections 2023-09-30T22:29:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5f0134260 0x7fd5f009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd5f0133420 0x7fd5f009c480 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 --2- 172.21.15.32:0/4198461396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5f0133e30 0x7fd5f009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 >> 172.21.15.32:0/4198461396 conn(0x7fd5f00a4210 msgr2=0x7fd5f01303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 shutdown_connections 2023-09-30T22:29:08.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.960+0000 7fd5f6dfa700 1 -- 172.21.15.32:0/4198461396 wait complete. 2023-09-30T22:29:08.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.989+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efcc41a9540 con 0x7efcc409e000 2023-09-30T22:29:08.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.989+0000 7efcc0ff9700 1 -- 172.21.15.32:0/3289979195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efcc41a9540 con 0x7efcc409e000 2023-09-30T22:29:08.990 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 msgr2=0x7efcc41a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 0x7efcc41a6470 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7efcb8000c00 tx=0x7efcb8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 shutdown_connections 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcc409e410 0x7efcc41a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcc409d6f0 0x7efcc41a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 --2- 172.21.15.32:0/3289979195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcc409e000 0x7efcc41a6470 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.991+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 >> 172.21.15.32:0/3289979195 conn(0x7efcc4066f50 msgr2=0x7efcc41baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.992+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 shutdown_connections 2023-09-30T22:29:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:08.992+0000 7efcca99c700 1 -- 172.21.15.32:0/3289979195 wait complete. 2023-09-30T22:29:09.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.027+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc684120ce0 con 0x7fc68409de50 2023-09-30T22:29:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.027+0000 7fc679ffb700 1 -- 172.21.15.32:0/3063813847 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc680015030 con 0x7fc68409de50 2023-09-30T22:29:09.028 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.029+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 msgr2=0x7fc68411d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.029+0000 7fc68b979700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68411d6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc6800090b0 tx=0x7fc680004750 comp rx=0 tx=0).stop 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 shutdown_connections 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6841338a0 0x7fc68411e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc68409de50 0x7fc68411d6b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 --2- 172.21.15.32:0/3063813847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc68409e760 0x7fc68411dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 >> 172.21.15.32:0/3063813847 conn(0x7fc6840a2aa0 msgr2=0x7fc6841bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 shutdown_connections 2023-09-30T22:29:09.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.030+0000 7fc68b979700 1 -- 172.21.15.32:0/3063813847 wait complete. 2023-09-30T22:29:09.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:09.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 Processor -- start 2023-09-30T22:29:09.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 -- start start 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 0x7fc9501337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133e30 0x7fc95012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95006b990 con 0x7fc950134260 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500a0590 con 0x7fc950133420 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc95724e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc95009fb80 con 0x7fc950133e30 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.226+0000 7fc954fea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 0x7fc9501337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc9557eb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc950134260 0x7fc9501304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54288/0 (socket says 172.21.15.32:54288) 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc954fea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 0x7fc9501337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40150/0 (socket says 172.21.15.32:40150) 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc9557eb700 1 -- 172.21.15.32:0/2286924733 learned_addr learned my addr 172.21.15.32:0/2286924733 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94ffff700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133e30 0x7fc95012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94f7fe700 1 -- 172.21.15.32:0/2286924733 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631300184 0 0) 0x7fc95006b990 con 0x7fc950134260 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94ffff700 1 -- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 msgr2=0x7fc9501337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94ffff700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 0x7fc9501337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94ffff700 1 -- 172.21.15.32:0/2286924733 >> v1:172.21.15.32:6789/0 conn(0x7fc950134260 legacy=0x7fc9501304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94ffff700 1 -- 172.21.15.32:0/2286924733 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500a0030 con 0x7fc950133e30 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc954fea700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 0x7fc9501337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94ffff700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133e30 0x7fc95012ff90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc944009870 tx=0x7fc944009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30131019a6c0687b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94f7fe700 1 -- 172.21.15.32:0/2286924733 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944011830 con 0x7fc950133e30 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94f7fe700 1 -- 172.21.15.32:0/2286924733 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944011990 con 0x7fc950133e30 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.227+0000 7fc94f7fe700 1 -- 172.21.15.32:0/2286924733 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944011d20 con 0x7fc950133e30 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 -- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133e30 msgr2=0x7fc95012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133e30 0x7fc95012ff90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc944009870 tx=0x7fc944009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 -- 172.21.15.32:0/2286924733 shutdown_connections 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133420 0x7fc9501337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 --2- 172.21.15.32:0/2286924733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133e30 0x7fc95012ff90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 -- 172.21.15.32:0/2286924733 >> 172.21.15.32:0/2286924733 conn(0x7fc9500a4210 msgr2=0x7fc9500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 -- 172.21.15.32:0/2286924733 shutdown_connections 2023-09-30T22:29:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.228+0000 7fc95724e700 1 -- 172.21.15.32:0/2286924733 wait complete. 2023-09-30T22:29:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.229+0000 7fc95724e700 1 Processor -- start 2023-09-30T22:29:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.229+0000 7fc95724e700 1 -- start start 2023-09-30T22:29:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc95724e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133420 0x7fc95009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc95724e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133e30 0x7fc95009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc95724e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc950134260 0x7fc95009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc95724e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95011f050 con 0x7fc950134260 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc95724e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95009de50 con 0x7fc950133e30 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc95724e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc95011fc50 con 0x7fc950133420 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc954fea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133420 0x7fc95009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc9557eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc950134260 0x7fc95009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133e30 0x7fc95009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc954fea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133420 0x7fc95009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38152/0 (socket says 172.21.15.32:38152) 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc954fea700 1 -- 172.21.15.32:0/436035170 learned_addr learned my addr 172.21.15.32:0/436035170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:09.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 -- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133420 msgr2=0x7fc95009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133420 0x7fc95009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 -- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc950134260 msgr2=0x7fc95009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc950134260 0x7fc95009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 -- 172.21.15.32:0/436035170 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400094a0 con 0x7fc950133e30 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.230+0000 7fc94ffff700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133e30 0x7fc95009c9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc944016000 tx=0x7fc944004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.231+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944004510 con 0x7fc950133e30 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.231+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94401f070 con 0x7fc950133e30 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.231+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440094a0 con 0x7fc950133e30 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.231+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9440038e0 con 0x7fc950133e30 2023-09-30T22:29:09.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.231+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95009e3e0 con 0x7fc950133e30 2023-09-30T22:29:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.232+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc944003a40 con 0x7fc950133e30 2023-09-30T22:29:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.232+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950074ea0 con 0x7fc950133e30 2023-09-30T22:29:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.232+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc944021070 con 0x7fc950133e30 2023-09-30T22:29:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.237+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc94402f3a0 con 0x7fc950133e30 2023-09-30T22:29:09.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.441+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc950120100 con 0x7fc950133e30 2023-09-30T22:29:09.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.441+0000 7fc94d7fa700 1 -- 172.21.15.32:0/436035170 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc94402b070 con 0x7fc950133e30 2023-09-30T22:29:09.442 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133e30 msgr2=0x7fc95009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133e30 0x7fc95009c9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc944016000 tx=0x7fc944004750 comp rx=0 tx=0).stop 2023-09-30T22:29:09.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 shutdown_connections 2023-09-30T22:29:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc950134260 0x7fc95009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc950133420 0x7fc95009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 --2- 172.21.15.32:0/436035170 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc950133e30 0x7fc95009c9c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 >> 172.21.15.32:0/436035170 conn(0x7fc9500a4210 msgr2=0x7fc9500a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 shutdown_connections 2023-09-30T22:29:09.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:09.443+0000 7fc95724e700 1 -- 172.21.15.32:0/436035170 wait complete. 2023-09-30T22:29:09.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:09.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:10.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:10.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:10.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.127+0000 7f23bebed700 1 Processor -- start 2023-09-30T22:29:10.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.127+0000 7f23bebed700 1 -- start start 2023-09-30T22:29:10.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bebed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b8130670 0x7f23b8130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bebed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23b806f680 0x7f23b806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bebed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b8130260 con 0x7f23b806ffd0 2023-09-30T22:29:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bebed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b8071ff0 con 0x7f23b806f680 2023-09-30T22:29:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bebed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23b809ecf0 con 0x7f23b8130670 2023-09-30T22:29:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bc989700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b8130670 0x7f23b8130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23b7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23b806f680 0x7f23b806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bd18a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23b806ffd0 0x7f23b8137530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54304/0 (socket says 172.21.15.32:54304) 2023-09-30T22:29:10.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.128+0000 7f23bd18a700 1 -- 172.21.15.32:0/166301453 learned_addr learned my addr 172.21.15.32:0/166301453 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:10.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23bc989700 1 -- 172.21.15.32:0/166301453 >> v2:172.21.15.32:3300/0 conn(0x7f23b806f680 msgr2=0x7f23b806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23bc989700 1 --2- 172.21.15.32:0/166301453 >> v2:172.21.15.32:3300/0 conn(0x7f23b806f680 0x7f23b806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23bc989700 1 -- 172.21.15.32:0/166301453 >> v1:172.21.15.32:6789/0 conn(0x7f23b806ffd0 legacy=0x7f23b8137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23bc989700 1 -- 172.21.15.32:0/166301453 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8070e70 con 0x7f23b8130670 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23bc989700 1 --2- 172.21.15.32:0/166301453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b8130670 0x7f23b8130a40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009840 tx=0x7f23ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d04a5968415a97c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23b77fe700 1 -- 172.21.15.32:0/166301453 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac011830 con 0x7f23b8130670 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23b77fe700 1 -- 172.21.15.32:0/166301453 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac011990 con 0x7f23b8130670 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.129+0000 7f23b77fe700 1 -- 172.21.15.32:0/166301453 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac011d20 con 0x7f23b8130670 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 -- 172.21.15.32:0/166301453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b8130670 msgr2=0x7f23b8130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 --2- 172.21.15.32:0/166301453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b8130670 0x7f23b8130a40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009840 tx=0x7f23ac009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 -- 172.21.15.32:0/166301453 shutdown_connections 2023-09-30T22:29:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 --2- 172.21.15.32:0/166301453 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b8130670 0x7f23b8130a40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 --2- 172.21.15.32:0/166301453 >> v2:172.21.15.32:3300/0 conn(0x7f23b806f680 0x7f23b806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 -- 172.21.15.32:0/166301453 >> 172.21.15.32:0/166301453 conn(0x7f23b80a2aa0 msgr2=0x7f23b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 -- 172.21.15.32:0/166301453 shutdown_connections 2023-09-30T22:29:10.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.130+0000 7f23bebed700 1 -- 172.21.15.32:0/166301453 wait complete. 2023-09-30T22:29:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.132+0000 7f23bebed700 1 Processor -- start 2023-09-30T22:29:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.132+0000 7f23bebed700 1 -- start start 2023-09-30T22:29:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bebed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 0x7f23b8138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bebed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23b806ffd0 0x7f23b8138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bebed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23b8130670 0x7f23b81c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bebed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81355c0 con 0x7f23b806ffd0 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bebed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8135c10 con 0x7f23b8130670 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bebed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8134f90 con 0x7f23b806f680 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bc989700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 0x7f23b8138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bc989700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 0x7f23b8138600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38166/0 (socket says 172.21.15.32:38166) 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bc989700 1 -- 172.21.15.32:0/949048715 learned_addr learned my addr 172.21.15.32:0/949048715 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23bd18a700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23b8130670 0x7f23b81c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.133+0000 7f23b7fff700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23b806ffd0 0x7f23b8138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bc989700 1 -- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23b8130670 msgr2=0x7f23b81c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bc989700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23b8130670 0x7f23b81c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bc989700 1 -- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23b806ffd0 msgr2=0x7f23b8138b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bc989700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23b806ffd0 0x7f23b8138b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bc989700 1 -- 172.21.15.32:0/949048715 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0094a0 con 0x7f23b806f680 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bc989700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 0x7f23b8138600 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009b90 tx=0x7f23ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac004590 con 0x7f23b806f680 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac0036a0 con 0x7f23b806f680 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b81363c0 con 0x7f23b806f680 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac003a10 con 0x7f23b806f680 2023-09-30T22:29:10.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.134+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8136fa0 con 0x7f23b806f680 2023-09-30T22:29:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.135+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23ac003b70 con 0x7f23b806f680 2023-09-30T22:29:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.135+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f23ac0406c0 con 0x7f23b806f680 2023-09-30T22:29:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.135+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b81372f0 con 0x7f23b806f680 2023-09-30T22:29:10.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.140+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f23ac02b950 con 0x7f23b806f680 2023-09-30T22:29:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.178+0000 7f09077e5700 1 Processor -- start 2023-09-30T22:29:10.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.178+0000 7f09077e5700 1 -- start start 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f09077e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 0x7f0900133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f09077e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 0x7f090009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f09077e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f09077e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090006fef0 con 0x7f090009de50 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f09077e5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090006ef30 con 0x7f090009e760 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f09077e5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09001334b0 con 0x7f09001338a0 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 0x7f0900133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0904d80700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 0x7f090009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905d82700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 0x7f0900133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38178/0 (socket says 172.21.15.32:38178) 2023-09-30T22:29:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0904d80700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 0x7f090009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56652/0 (socket says 172.21.15.32:56652) 2023-09-30T22:29:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 -- 172.21.15.32:0/1911769796 learned_addr learned my addr 172.21.15.32:0/1911769796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905d82700 1 --2- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090013a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40172/0 (socket says 172.21.15.32:40172) 2023-09-30T22:29:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 -- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 msgr2=0x7f090013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 --2- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 -- 172.21.15.32:0/1911769796 >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 msgr2=0x7f090009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 --2- 172.21.15.32:0/1911769796 >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 0x7f090009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 -- 172.21.15.32:0/1911769796 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090006fc30 con 0x7f09001338a0 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0904d80700 1 --2- 172.21.15.32:0/1911769796 >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 0x7f090009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.179+0000 7f0905581700 1 --2- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 0x7f0900133c70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0098f0 tx=0x7f08fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52be72eadfdcd467 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f08f7fff700 1 -- 172.21.15.32:0/1911769796 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc011830 con 0x7f09001338a0 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f08f7fff700 1 -- 172.21.15.32:0/1911769796 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc011990 con 0x7f09001338a0 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f08f7fff700 1 -- 172.21.15.32:0/1911769796 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc011d20 con 0x7f09001338a0 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 -- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 msgr2=0x7f0900133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 --2- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 0x7f0900133c70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0098f0 tx=0x7f08fc009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 -- 172.21.15.32:0/1911769796 shutdown_connections 2023-09-30T22:29:10.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 --2- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 --2- 172.21.15.32:0/1911769796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09001338a0 0x7f0900133c70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 --2- 172.21.15.32:0/1911769796 >> v2:172.21.15.32:3300/0 conn(0x7f090009de50 0x7f090009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.180+0000 7f09077e5700 1 -- 172.21.15.32:0/1911769796 >> 172.21.15.32:0/1911769796 conn(0x7f09000a2aa0 msgr2=0x7f09000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.181+0000 7f09077e5700 1 -- 172.21.15.32:0/1911769796 shutdown_connections 2023-09-30T22:29:10.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.181+0000 7f09077e5700 1 -- 172.21.15.32:0/1911769796 wait complete. 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.182+0000 7f09077e5700 1 Processor -- start 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.182+0000 7f09077e5700 1 -- start start 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f09077e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f090009de50 0x7f090011d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f09077e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090011dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f09077e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09001338a0 0x7f090011e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f09077e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0900121370 con 0x7f09001338a0 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f09077e5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090011fa40 con 0x7f090009e760 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f09077e5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0900120b00 con 0x7f090009de50 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090011dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090011dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40174/0 (socket says 172.21.15.32:40174) 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 -- 172.21.15.32:0/2487005767 learned_addr learned my addr 172.21.15.32:0/2487005767 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0905d82700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09001338a0 0x7f090011e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0905581700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f090009de50 0x7f090011d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 -- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f090009de50 msgr2=0x7f090011d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f090009de50 0x7f090011d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 -- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09001338a0 msgr2=0x7f090011e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.183+0000 7f0904d80700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09001338a0 0x7f090011e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f0904d80700 1 -- 172.21.15.32:0/2487005767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00094a0 con 0x7f090009e760 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f0905d82700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09001338a0 0x7f090011e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f0904d80700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090011dd90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ebb0 tx=0x7f08f000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000c850 con 0x7f090009e760 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f000c9b0 con 0x7f090009e760 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc009550 con 0x7f090009e760 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000cd40 con 0x7f090009e760 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.184+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09001206b0 con 0x7f090009e760 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.185+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08f0013470 con 0x7f090009e760 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.185+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f08f003f7f0 con 0x7f090009e760 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.187+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09000a0780 con 0x7f090009e760 2023-09-30T22:29:10.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.191+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08f0021020 con 0x7f090009e760 2023-09-30T22:29:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.207+0000 7fa9f0858700 1 Processor -- start 2023-09-30T22:29:10.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.207+0000 7fa9f0858700 1 -- start start 2023-09-30T22:29:10.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9f0858700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9ec09e1d0 0x7fa9ec138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9f0858700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec139170 0x7fa9ec13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9f0858700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec09b120 con 0x7fa9ec09d6f0 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9f0858700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ec06b460 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9f0858700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec09a7e0 con 0x7fa9ec139170 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9e9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9ec09e1d0 0x7fa9ec138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9ea59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa9ec09d6f0 0x7fa9ec09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54306/0 (socket says 172.21.15.32:54306) 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9ea59c700 1 -- 172.21.15.32:0/2657183197 learned_addr learned my addr 172.21.15.32:0/2657183197 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.208+0000 7fa9e9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9ec09e1d0 0x7fa9ec138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56666/0 (socket says 172.21.15.32:56666) 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9ead9d700 1 --2- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec139170 0x7fa9ec13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec139170 msgr2=0x7fa9ec13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e9d9b700 1 --2- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec139170 0x7fa9ec13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/2657183197 >> v1:172.21.15.32:6789/0 conn(0x7fa9ec09d6f0 legacy=0x7fa9ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/2657183197 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec09a500 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e9d9b700 1 --2- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec09e1d0 0x7fa9ec138c30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009870 tx=0x7fa9dc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d458c5f6cfa86a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e959a700 1 -- 172.21.15.32:0/2657183197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc011830 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e959a700 1 -- 172.21.15.32:0/2657183197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc011990 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.209+0000 7fa9e959a700 1 -- 172.21.15.32:0/2657183197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc011d20 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 -- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec09e1d0 msgr2=0x7fa9ec138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 --2- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec09e1d0 0x7fa9ec138c30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009870 tx=0x7fa9dc009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 -- 172.21.15.32:0/2657183197 shutdown_connections 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 --2- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec139170 0x7fa9ec13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 --2- 172.21.15.32:0/2657183197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec09e1d0 0x7fa9ec138c30 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 -- 172.21.15.32:0/2657183197 >> 172.21.15.32:0/2657183197 conn(0x7fa9ec066f50 msgr2=0x7fa9ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 -- 172.21.15.32:0/2657183197 shutdown_connections 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.210+0000 7fa9f0858700 1 -- 172.21.15.32:0/2657183197 wait complete. 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.211+0000 7fa9f0858700 1 Processor -- start 2023-09-30T22:29:10.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.211+0000 7fa9f0858700 1 -- start start 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9f0858700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 0x7fa9ec1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9f0858700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 0x7fa9ec1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9f0858700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec139170 0x7fa9ec1c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9f0858700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec1244c0 con 0x7fa9ec139170 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9f0858700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec122930 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9f0858700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec0a2ab0 con 0x7fa9ec09d6f0 2023-09-30T22:29:10.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9ea59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 0x7fa9ec1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 0x7fa9ec1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 0x7fa9ec1c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40184/0 (socket says 172.21.15.32:40184) 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/932770593 learned_addr learned my addr 172.21.15.32:0/932770593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9ea59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 0x7fa9ec1c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38188/0 (socket says 172.21.15.32:38188) 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9ead9d700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec139170 0x7fa9ec1c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 msgr2=0x7fa9ec1c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 0x7fa9ec1c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec139170 msgr2=0x7fa9ec1c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec139170 0x7fa9ec1c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9e9d9b700 1 -- 172.21.15.32:0/932770593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0094a0 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.212+0000 7fa9ea59c700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 0x7fa9ec1c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.213+0000 7fa9e9d9b700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 0x7fa9ec1c89a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009840 tx=0x7fa9dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.213+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0036a0 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.213+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc01f070 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.213+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec122fa0 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.213+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc003a60 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.213+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec123420 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.214+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9dc003bc0 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.214+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa9dc021070 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.214+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec0a3b10 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.219+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9dc015070 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.397+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f23b8137130 con 0x7f23b806f680 2023-09-30T22:29:10.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.397+0000 7f23b57fa700 1 -- 172.21.15.32:0/949048715 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23b8137130 con 0x7f23b806f680 2023-09-30T22:29:10.399 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 msgr2=0x7f23b8138600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 0x7f23b8138600 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009b90 tx=0x7f23ac004750 comp rx=0 tx=0).stop 2023-09-30T22:29:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 shutdown_connections 2023-09-30T22:29:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23b8130670 0x7f23b81c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23b806f680 0x7f23b8138600 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 --2- 172.21.15.32:0/949048715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23b806ffd0 0x7f23b8138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 >> 172.21.15.32:0/949048715 conn(0x7f23b80a2aa0 msgr2=0x7f23b80a06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 shutdown_connections 2023-09-30T22:29:10.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.400+0000 7f23bebed700 1 -- 172.21.15.32:0/949048715 wait complete. 2023-09-30T22:29:10.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:10.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.423+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa9ec09f420 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.423+0000 7fa9daffd700 1 -- 172.21.15.32:0/932770593 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9dc015030 con 0x7fa9ec09e1d0 2023-09-30T22:29:10.424 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 msgr2=0x7fa9ec1c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 0x7fa9ec1c89a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009840 tx=0x7fa9dc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 shutdown_connections 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9ec139170 0x7fa9ec1c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa9ec09d6f0 0x7fa9ec1c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 --2- 172.21.15.32:0/932770593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa9ec09e1d0 0x7fa9ec1c89a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 >> 172.21.15.32:0/932770593 conn(0x7fa9ec066f50 msgr2=0x7fa9ec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 shutdown_connections 2023-09-30T22:29:10.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.426+0000 7fa9f0858700 1 -- 172.21.15.32:0/932770593 wait complete. 2023-09-30T22:29:10.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.430+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0900120ef0 con 0x7f090009e760 2023-09-30T22:29:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.431+0000 7f08f5ffb700 1 -- 172.21.15.32:0/2487005767 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0900120ef0 con 0x7f090009e760 2023-09-30T22:29:10.432 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 msgr2=0x7f090011dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090011dd90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ebb0 tx=0x7f08f000eec0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 shutdown_connections 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09001338a0 0x7f090011e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f090009de50 0x7f090011d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 --2- 172.21.15.32:0/2487005767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f090009e760 0x7f090011dd90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 >> 172.21.15.32:0/2487005767 conn(0x7f09000a2aa0 msgr2=0x7f090009ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 shutdown_connections 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.433+0000 7f09077e5700 1 -- 172.21.15.32:0/2487005767 wait complete. 2023-09-30T22:29:10.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:10.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:10.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:10.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 Processor -- start 2023-09-30T22:29:10.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 -- start start 2023-09-30T22:29:10.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809d6f0 0x7fa20809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 0x7fa2081339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20809d2f0 con 0x7fa208133f30 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20809a810 con 0x7fa20809e000 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20e3eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa20809a470 con 0x7fa20809d6f0 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa207fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809d6f0 0x7fa20809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa2077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 0x7fa2081339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa2077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 0x7fa2081339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50034/0 (socket says 172.21.15.32:50034) 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa2077fe700 1 -- 172.21.15.32:0/1976109562 learned_addr learned my addr 172.21.15.32:0/1976109562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa207fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809d6f0 0x7fa20809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50164/0 (socket says 172.21.15.32:50164) 2023-09-30T22:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.629+0000 7fa20c988700 1 --1- 172.21.15.32:0/1976109562 >> v1:172.21.15.32:6789/0 conn(0x7fa208133f30 0x7fa20813ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59738/0 (socket says 172.21.15.32:59738) 2023-09-30T22:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa2077fe700 1 -- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809d6f0 msgr2=0x7fa20809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa2077fe700 1 --2- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809d6f0 0x7fa20809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa2077fe700 1 -- 172.21.15.32:0/1976109562 >> v1:172.21.15.32:6789/0 conn(0x7fa208133f30 legacy=0x7fa20813ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa2077fe700 1 -- 172.21.15.32:0/1976109562 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20809ec40 con 0x7fa20809e000 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa206ffd700 1 -- 172.21.15.32:0/1976109562 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417963187 0 0) 0x7fa20809d2f0 con 0x7fa208133f30 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa2077fe700 1 --2- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 0x7fa2081339f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009920 tx=0x7fa1f0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725cf9cba2b514e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa206ffd700 1 -- 172.21.15.32:0/1976109562 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0011830 con 0x7fa20809e000 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa206ffd700 1 -- 172.21.15.32:0/1976109562 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0011990 con 0x7fa20809e000 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.630+0000 7fa206ffd700 1 -- 172.21.15.32:0/1976109562 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0011d20 con 0x7fa20809e000 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 -- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 msgr2=0x7fa2081339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 0x7fa2081339f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009920 tx=0x7fa1f0009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 -- 172.21.15.32:0/1976109562 shutdown_connections 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809d6f0 0x7fa20809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/1976109562 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809e000 0x7fa2081339f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 -- 172.21.15.32:0/1976109562 >> 172.21.15.32:0/1976109562 conn(0x7fa208066f50 msgr2=0x7fa20809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 -- 172.21.15.32:0/1976109562 shutdown_connections 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.631+0000 7fa20e3eb700 1 -- 172.21.15.32:0/1976109562 wait complete. 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 Processor -- start 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 -- start start 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809d6f0 0x7fa2081c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 0x7fa2081c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa208133f30 0x7fa208122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081339d0 con 0x7fa208133f30 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa208124cb0 con 0x7fa20809d6f0 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20e3eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa208124580 con 0x7fa20809e000 2023-09-30T22:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa2077fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 0x7fa2081c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa20c988700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa208133f30 0x7fa208122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa2077fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 0x7fa2081c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50172/0 (socket says 172.21.15.32:50172) 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa2077fe700 1 -- 172.21.15.32:0/665874813 learned_addr learned my addr 172.21.15.32:0/665874813 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.632+0000 7fa207fff700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809d6f0 0x7fa2081c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa2077fe700 1 -- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809d6f0 msgr2=0x7fa2081c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa2077fe700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809d6f0 0x7fa2081c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa2077fe700 1 -- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa208133f30 msgr2=0x7fa208122900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa2077fe700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa208133f30 0x7fa208122900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa2077fe700 1 -- 172.21.15.32:0/665874813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009550 con 0x7fa20809e000 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa207fff700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809d6f0 0x7fa2081c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa20c988700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa208133f30 0x7fa208122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa2077fe700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 0x7fa2081c8a40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000c00 tx=0x7fa1f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0004450 con 0x7fa20809e000 2023-09-30T22:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f00045b0 con 0x7fa20809e000 2023-09-30T22:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00038a0 con 0x7fa20809e000 2023-09-30T22:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20809c640 con 0x7fa20809e000 2023-09-30T22:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.633+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa208125e40 con 0x7fa20809e000 2023-09-30T22:29:10.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.634+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa1f0021070 con 0x7fa20809e000 2023-09-30T22:29:10.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.634+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa1f0040630 con 0x7fa20809e000 2023-09-30T22:29:10.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.634+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa208074ea0 con 0x7fa20809e000 2023-09-30T22:29:10.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.639+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa1f0015080 con 0x7fa20809e000 2023-09-30T22:29:10.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.840+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa20806c360 con 0x7fa20809e000 2023-09-30T22:29:10.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.840+0000 7fa204ff9700 1 -- 172.21.15.32:0/665874813 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1f0019770 con 0x7fa20809e000 2023-09-30T22:29:10.841 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:10.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 msgr2=0x7fa2081c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 0x7fa2081c8a40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000c00 tx=0x7fa1f0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:10.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 shutdown_connections 2023-09-30T22:29:10.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa208133f30 0x7fa208122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa20809d6f0 0x7fa2081c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 --2- 172.21.15.32:0/665874813 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa20809e000 0x7fa2081c8a40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 >> 172.21.15.32:0/665874813 conn(0x7fa208066f50 msgr2=0x7fa2080a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 shutdown_connections 2023-09-30T22:29:10.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:10.842+0000 7fa20e3eb700 1 -- 172.21.15.32:0/665874813 wait complete. 2023-09-30T22:29:10.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:11.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:11.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:11.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:11.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.565+0000 7fd3da56e700 1 Processor -- start 2023-09-30T22:29:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.566+0000 7fd3da56e700 1 -- start start 2023-09-30T22:29:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.566+0000 7fd3da56e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.566+0000 7fd3da56e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3d409f240 0x7fd3d41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.566+0000 7fd3da56e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 0x7fd3d4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3da56e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d4070940 con 0x7fd3d409f240 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3da56e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d406fd60 con 0x7fd3d409f650 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3da56e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d406fa80 con 0x7fd3d409e930 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d8b0b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 0x7fd3d4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d8b0b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 0x7fd3d4131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50056/0 (socket says 172.21.15.32:50056) 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d8b0b700 1 -- 172.21.15.32:0/3853466603 learned_addr learned my addr 172.21.15.32:0/3853466603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d37fe700 1 --2- 172.21.15.32:0/3853466603 >> v2:172.21.15.32:3300/0 conn(0x7fd3d409f240 0x7fd3d41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 msgr2=0x7fd3d4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 --2- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 0x7fd3d4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3853466603 >> v2:172.21.15.32:3300/0 conn(0x7fd3d409f240 msgr2=0x7fd3d41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 --2- 172.21.15.32:0/3853466603 >> v2:172.21.15.32:3300/0 conn(0x7fd3d409f240 0x7fd3d41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3853466603 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d409fb60 con 0x7fd3d409e930 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d8b0b700 1 --2- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 0x7fd3d4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d37fe700 1 --2- 172.21.15.32:0/3853466603 >> v2:172.21.15.32:3300/0 conn(0x7fd3d409f240 0x7fd3d41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.567+0000 7fd3d3fff700 1 --2- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d409ed00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009840 tx=0x7fd3c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f27ce674022bd28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.568+0000 7fd3d2ffd700 1 -- 172.21.15.32:0/3853466603 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4011830 con 0x7fd3d409e930 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.568+0000 7fd3d2ffd700 1 -- 172.21.15.32:0/3853466603 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c4011990 con 0x7fd3d409e930 2023-09-30T22:29:11.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.568+0000 7fd3d2ffd700 1 -- 172.21.15.32:0/3853466603 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4011d20 con 0x7fd3d409e930 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 -- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 msgr2=0x7fd3d409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d409ed00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009840 tx=0x7fd3c4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 -- 172.21.15.32:0/3853466603 shutdown_connections 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f650 0x7fd3d4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3853466603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d409ed00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3853466603 >> v2:172.21.15.32:3300/0 conn(0x7fd3d409f240 0x7fd3d41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 -- 172.21.15.32:0/3853466603 >> 172.21.15.32:0/3853466603 conn(0x7fd3d40a4210 msgr2=0x7fd3d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 -- 172.21.15.32:0/3853466603 shutdown_connections 2023-09-30T22:29:11.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.569+0000 7fd3da56e700 1 -- 172.21.15.32:0/3853466603 wait complete. 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.570+0000 7fd3da56e700 1 Processor -- start 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.570+0000 7fd3da56e700 1 -- start start 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3da56e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d4120410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3da56e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f240 0x7fd3d4120950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3da56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d409f650 0x7fd3d4120e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3da56e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4123c50 con 0x7fd3d409f650 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3da56e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4122130 con 0x7fd3d409f240 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3da56e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4071d20 con 0x7fd3d409e930 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d4120410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d4120410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50202/0 (socket says 172.21.15.32:50202) 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3026442894 learned_addr learned my addr 172.21.15.32:0/3026442894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d37fe700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f240 0x7fd3d4120950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d8b0b700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d409f650 0x7fd3d4120e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f240 msgr2=0x7fd3d4120950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f240 0x7fd3d4120950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d409f650 msgr2=0x7fd3d4120e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d409f650 0x7fd3d4120e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d3fff700 1 -- 172.21.15.32:0/3026442894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c80094a0 con 0x7fd3d409e930 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d8b0b700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d409f650 0x7fd3d4120e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.571+0000 7fd3d37fe700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f240 0x7fd3d4120950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.572+0000 7fd3d3fff700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d4120410 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40090b0 tx=0x7fd3c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.572+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4004450 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.572+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c401f070 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.572+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40038e0 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.572+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc009b30 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.572+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c40094a0 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.573+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d4124780 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.573+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd3c4003a40 con 0x7fd3d409e930 2023-09-30T22:29:11.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.573+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd3c4021070 con 0x7fd3d409e930 2023-09-30T22:29:11.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.578+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3c40193b0 con 0x7fd3d409e930 2023-09-30T22:29:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.609+0000 7ffb07f61700 1 Processor -- start 2023-09-30T22:29:11.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7f04a2b01700 1 Processor -- start 2023-09-30T22:29:11.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7f04a2b01700 1 -- start start 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 -- start start 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 0x7ffb0009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb0009e000 0x7ffb00130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e410 0x7ffb00130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0009f520 con 0x7ffb0009e000 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0009f130 con 0x7ffb0009e410 2023-09-30T22:29:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7ffb07f61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0009e930 con 0x7ffb0009d6f0 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7f04a2b01700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09d6f0 0x7f049c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.610+0000 7f04a2b01700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 0x7f049c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a2b01700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f049c09e410 0x7f049c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a2b01700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c06b460 con 0x7f049c09e410 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a2b01700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c09b120 con 0x7f049c09e000 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a2b01700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f049c09a7e0 con 0x7f049c09d6f0 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a109e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f049c09e410 0x7f049c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 0x7f049c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a109e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f049c09e410 0x7f049c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46468/0 (socket says 172.21.15.32:46468) 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f04a109e700 1 -- 172.21.15.32:0/732919498 learned_addr learned my addr 172.21.15.32:0/732919498 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 0x7f049c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50084/0 (socket says 172.21.15.32:50084) 2023-09-30T22:29:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb0009e000 0x7ffb00130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb064fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e410 0x7ffb00130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb0009e000 0x7ffb00130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46454/0 (socket says 172.21.15.32:46454) 2023-09-30T22:29:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 -- 172.21.15.32:0/692124308 learned_addr learned my addr 172.21.15.32:0/692124308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 -- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 msgr2=0x7ffb0009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 0x7ffb0009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 -- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e410 msgr2=0x7ffb00130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e410 0x7ffb00130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 -- 172.21.15.32:0/692124308 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0009b2d0 con 0x7ffb0009e000 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb064fe700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e410 0x7ffb00130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 -- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09d6f0 msgr2=0x7f049c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 --2- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09d6f0 0x7f049c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 -- 172.21.15.32:0/732919498 >> v2:172.21.15.32:3300/0 conn(0x7f049c09e410 msgr2=0x7f049c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 --2- 172.21.15.32:0/732919498 >> v2:172.21.15.32:3300/0 conn(0x7f049c09e410 0x7f049c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 -- 172.21.15.32:0/732919498 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c09a500 con 0x7f049c09e000 2023-09-30T22:29:11.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7f049bfff700 1 --2- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 0x7f049c1339f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0488009870 tx=0x7f0488009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24954124737b2ae0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.611+0000 7ffb054fc700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e000 0x7ffb00130660 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009920 tx=0x7ffaf0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72deb6c92bedbe1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f049b7fe700 1 -- 172.21.15.32:0/732919498 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488011830 con 0x7f049c09e000 2023-09-30T22:29:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f049b7fe700 1 -- 172.21.15.32:0/732919498 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488011990 con 0x7f049c09e000 2023-09-30T22:29:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f049b7fe700 1 -- 172.21.15.32:0/732919498 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488011d20 con 0x7f049c09e000 2023-09-30T22:29:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7ffb04cfb700 1 -- 172.21.15.32:0/692124308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000ba20 con 0x7ffb0009e000 2023-09-30T22:29:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7ffb04cfb700 1 -- 172.21.15.32:0/692124308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf000bb80 con 0x7ffb0009e000 2023-09-30T22:29:11.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7ffb04cfb700 1 -- 172.21.15.32:0/692124308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0009e70 con 0x7ffb0009e000 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 -- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 msgr2=0x7f049c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 --2- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 0x7f049c1339f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0488009870 tx=0x7f0488009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 -- 172.21.15.32:0/732919498 shutdown_connections 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 --2- 172.21.15.32:0/732919498 >> v2:172.21.15.32:3300/0 conn(0x7f049c09e410 0x7f049c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 --2- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09d6f0 0x7f049c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 --2- 172.21.15.32:0/732919498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e000 0x7f049c1339f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7f04a2b01700 1 -- 172.21.15.32:0/732919498 >> 172.21.15.32:0/732919498 conn(0x7f049c066f50 msgr2=0x7f049c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7ffb07f61700 1 -- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e000 msgr2=0x7ffb00130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.612+0000 7ffb07f61700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e000 0x7ffb00130660 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009920 tx=0x7ffaf0009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 -- 172.21.15.32:0/692124308 shutdown_connections 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e410 0x7ffb00130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 0x7ffb0009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 --2- 172.21.15.32:0/692124308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e000 0x7ffb00130660 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 -- 172.21.15.32:0/692124308 >> 172.21.15.32:0/692124308 conn(0x7ffb00066f50 msgr2=0x7ffb0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7f04a2b01700 1 -- 172.21.15.32:0/732919498 shutdown_connections 2023-09-30T22:29:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7f04a2b01700 1 -- 172.21.15.32:0/732919498 wait complete. 2023-09-30T22:29:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 -- 172.21.15.32:0/692124308 shutdown_connections 2023-09-30T22:29:11.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.613+0000 7ffb07f61700 1 -- 172.21.15.32:0/692124308 wait complete. 2023-09-30T22:29:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7f04a2b01700 1 Processor -- start 2023-09-30T22:29:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 Processor -- start 2023-09-30T22:29:11.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7f04a2b01700 1 -- start start 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 -- start start 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 0x7ffb0011df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 0x7ffb0011e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e410 0x7ffb0011e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0011f3f0 con 0x7ffb0009e410 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00130300 con 0x7ffb0009e000 2023-09-30T22:29:11.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb07f61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb00120d80 con 0x7ffb0009d6f0 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb054fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 0x7ffb0011e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7f04a2b01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f049c09d6f0 0x7f049c119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7f04a2b01700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09e000 0x7f049c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a2b01700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 0x7f049c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a2b01700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c11cfe0 con 0x7f049c09d6f0 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a2b01700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c1b9310 con 0x7f049c09e410 2023-09-30T22:29:11.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a2b01700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f049c11b5e0 con 0x7f049c09e000 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb054fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 0x7ffb0011e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50090/0 (socket says 172.21.15.32:50090) 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.614+0000 7ffb054fc700 1 -- 172.21.15.32:0/3121227604 learned_addr learned my addr 172.21.15.32:0/3121227604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb064fe700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e410 0x7ffb0011e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 0x7f049c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 0x7f049c11a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50106/0 (socket says 172.21.15.32:50106) 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 -- 172.21.15.32:0/2351993206 learned_addr learned my addr 172.21.15.32:0/2351993206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a089d700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f049c09d6f0 0x7f049c119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb054fc700 1 -- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 msgr2=0x7ffb0011df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb054fc700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 0x7ffb0011df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb054fc700 1 -- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e410 msgr2=0x7ffb0011e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb054fc700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e410 0x7ffb0011e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb054fc700 1 -- 172.21.15.32:0/3121227604 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0009550 con 0x7ffb0009e000 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 -- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09e000 msgr2=0x7f049c119c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09e000 0x7f049c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 -- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f049c09d6f0 msgr2=0x7f049c119720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f049c09d6f0 0x7f049c119720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 -- 172.21.15.32:0/2351993206 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880094a0 con 0x7f049c09e410 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a089d700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f049c09d6f0 0x7f049c119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:11.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04a109e700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 0x7f049c11a1a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0494009d60 tx=0x7f049400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7ffb054fc700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 0x7ffb0011e460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009c70 tx=0x7ffaf0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494003c20 con 0x7f049c09e410 2023-09-30T22:29:11.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.615+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049400f040 con 0x7f049c09e410 2023-09-30T22:29:11.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c0a3060 con 0x7f049c09e410 2023-09-30T22:29:11.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0004510 con 0x7ffb0009e000 2023-09-30T22:29:11.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf00036a0 con 0x7ffb0009e000 2023-09-30T22:29:11.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0003a50 con 0x7ffb0009e000 2023-09-30T22:29:11.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb00121760 con 0x7ffb0009e000 2023-09-30T22:29:11.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb000a2020 con 0x7ffb0009e000 2023-09-30T22:29:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400c930 con 0x7f049c09e410 2023-09-30T22:29:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.616+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c11cc60 con 0x7f049c09e410 2023-09-30T22:29:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.617+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f049400ca90 con 0x7f049c09e410 2023-09-30T22:29:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.617+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c074ea0 con 0x7f049c09e410 2023-09-30T22:29:11.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.617+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f049403fa70 con 0x7f049c09e410 2023-09-30T22:29:11.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.617+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb00074ea0 con 0x7ffb0009e000 2023-09-30T22:29:11.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.618+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffaf0003bb0 con 0x7ffb0009e000 2023-09-30T22:29:11.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.622+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffaf0021030 con 0x7ffb0009e000 2023-09-30T22:29:11.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.622+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0494041e60 con 0x7f049c09e410 2023-09-30T22:29:11.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.626+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffaf0030480 con 0x7ffb0009e000 2023-09-30T22:29:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.788+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd3d4074ea0 con 0x7fd3d409e930 2023-09-30T22:29:11.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.788+0000 7fd3d0ff9700 1 -- 172.21.15.32:0/3026442894 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3c4015070 con 0x7fd3d409e930 2023-09-30T22:29:11.790 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 msgr2=0x7fd3d4120410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d4120410 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40090b0 tx=0x7fd3c4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 shutdown_connections 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3d409f650 0x7fd3d4120e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd3d409e930 0x7fd3d4120410 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 --2- 172.21.15.32:0/3026442894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3d409f240 0x7fd3d4120950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 >> 172.21.15.32:0/3026442894 conn(0x7fd3d40a4210 msgr2=0x7fd3d409bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 shutdown_connections 2023-09-30T22:29:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.791+0000 7fd3da56e700 1 -- 172.21.15.32:0/3026442894 wait complete. 2023-09-30T22:29:11.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.847+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f049c09f420 con 0x7f049c09e410 2023-09-30T22:29:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.848+0000 7f04997fa700 1 -- 172.21.15.32:0/2351993206 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f049400cd40 con 0x7f049c09e410 2023-09-30T22:29:11.849 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 msgr2=0x7f049c11a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 0x7f049c11a1a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0494009d60 tx=0x7f049400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 shutdown_connections 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f049c09e410 0x7f049c11a1a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f049c09d6f0 0x7f049c119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 --2- 172.21.15.32:0/2351993206 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f049c09e000 0x7f049c119c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 >> 172.21.15.32:0/2351993206 conn(0x7f049c066f50 msgr2=0x7f049c1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 shutdown_connections 2023-09-30T22:29:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.850+0000 7f04a2b01700 1 -- 172.21.15.32:0/2351993206 wait complete. 2023-09-30T22:29:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.852+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffb0009c700 con 0x7ffb0009e000 2023-09-30T22:29:11.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.852+0000 7ffaf67fc700 1 -- 172.21.15.32:0/3121227604 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffaf002b020 con 0x7ffb0009e000 2023-09-30T22:29:11.853 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:11.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 msgr2=0x7ffb0011e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 0x7ffb0011e460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009c70 tx=0x7ffaf0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 shutdown_connections 2023-09-30T22:29:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb0009e410 0x7ffb0011e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb0009d6f0 0x7ffb0011df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 --2- 172.21.15.32:0/3121227604 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb0009e000 0x7ffb0011e460 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 >> 172.21.15.32:0/3121227604 conn(0x7ffb00066f50 msgr2=0x7ffb00070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 shutdown_connections 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.854+0000 7ffb07f61700 1 -- 172.21.15.32:0/3121227604 wait complete. 2023-09-30T22:29:11.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:11.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:11.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:11.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.994+0000 7f6c3c7e9700 1 Processor -- start 2023-09-30T22:29:11.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3c7e9700 1 -- start start 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3c7e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c3409d6f0 0x7f6c3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3c7e9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 0x7f6c34134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3c7e9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3409b120 con 0x7f6c3409e000 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3c7e9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3406b460 con 0x7f6c3409d6f0 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3c7e9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3409a7e0 con 0x7f6c3409e440 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3ad86700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 0x7f6c34134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3ad86700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 0x7f6c34134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50120/0 (socket says 172.21.15.32:50120) 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3ad86700 1 -- 172.21.15.32:0/1505085410 learned_addr learned my addr 172.21.15.32:0/1505085410 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:11.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.995+0000 7f6c3a585700 1 --2- 172.21.15.32:0/1505085410 >> v2:172.21.15.32:3300/0 conn(0x7f6c3409d6f0 0x7f6c3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c3ad86700 1 -- 172.21.15.32:0/1505085410 >> v2:172.21.15.32:3300/0 conn(0x7f6c3409d6f0 msgr2=0x7f6c3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c3ad86700 1 --2- 172.21.15.32:0/1505085410 >> v2:172.21.15.32:3300/0 conn(0x7f6c3409d6f0 0x7f6c3409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c3ad86700 1 -- 172.21.15.32:0/1505085410 >> v1:172.21.15.32:6789/0 conn(0x7f6c3409e000 legacy=0x7f6c341339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c3ad86700 1 -- 172.21.15.32:0/1505085410 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3409a500 con 0x7f6c3409e440 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c3a585700 1 --2- 172.21.15.32:0/1505085410 >> v2:172.21.15.32:3300/0 conn(0x7f6c3409d6f0 0x7f6c3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c3ad86700 1 --2- 172.21.15.32:0/1505085410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 0x7f6c34134100 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e80 tx=0x7f6c2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b29e014c7fed09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c39583700 1 -- 172.21.15.32:0/1505085410 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00e070 con 0x7f6c3409e440 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c39583700 1 -- 172.21.15.32:0/1505085410 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c00b8d0 con 0x7f6c3409e440 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.996+0000 7f6c39583700 1 -- 172.21.15.32:0/1505085410 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00bc80 con 0x7f6c3409e440 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/1505085410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 msgr2=0x7f6c34134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/1505085410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 0x7f6c34134100 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009e80 tx=0x7f6c2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/1505085410 shutdown_connections 2023-09-30T22:29:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/1505085410 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409e440 0x7f6c34134100 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/1505085410 >> v2:172.21.15.32:3300/0 conn(0x7f6c3409d6f0 0x7f6c3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/1505085410 >> 172.21.15.32:0/1505085410 conn(0x7f6c34066f50 msgr2=0x7f6c3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/1505085410 shutdown_connections 2023-09-30T22:29:12.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.997+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/1505085410 wait complete. 2023-09-30T22:29:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 Processor -- start 2023-09-30T22:29:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 -- start start 2023-09-30T22:29:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 0x7f6c341a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3409e000 0x7f6c341a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c3409e440 0x7f6c341a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341aa360 con 0x7f6c3409e000 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a99a0 con 0x7f6c3409d6f0 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3c7e9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a8ce0 con 0x7f6c3409e440 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3a585700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 0x7f6c341a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c39d84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3409e000 0x7f6c341a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3a585700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 0x7f6c341a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50134/0 (socket says 172.21.15.32:50134) 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3a585700 1 -- 172.21.15.32:0/3956599351 learned_addr learned my addr 172.21.15.32:0/3956599351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.998+0000 7f6c3ad86700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c3409e440 0x7f6c341a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c39d84700 1 -- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c3409e440 msgr2=0x7f6c341a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c39d84700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c3409e440 0x7f6c341a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c39d84700 1 -- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 msgr2=0x7f6c341a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c39d84700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 0x7f6c341a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c39d84700 1 -- 172.21.15.32:0/3956599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c300094a0 con 0x7f6c3409e000 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c3a585700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 0x7f6c341a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c3ad86700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c3409e440 0x7f6c341a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c39d84700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3409e000 0x7f6c341a6410 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000f60 tx=0x7f6c2400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240052d0 con 0x7f6c3409e000 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24005430 con 0x7f6c3409e000 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c009b30 con 0x7f6c3409e000 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240038c0 con 0x7f6c3409e000 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:11.999+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c341a7a90 con 0x7f6c3409e000 2023-09-30T22:29:12.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.000+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c24003a20 con 0x7f6c3409e000 2023-09-30T22:29:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.000+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c341a9540 con 0x7f6c3409e000 2023-09-30T22:29:12.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.000+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6c2403e8a0 con 0x7f6c3409e000 2023-09-30T22:29:12.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.005+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c24011590 con 0x7f6c3409e000 2023-09-30T22:29:12.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.209+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c34074ea0 con 0x7f6c3409e000 2023-09-30T22:29:12.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.209+0000 7f6c2affd700 1 -- 172.21.15.32:0/3956599351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c24017090 con 0x7f6c3409e000 2023-09-30T22:29:12.211 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3409e000 msgr2=0x7f6c341a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3409e000 0x7f6c341a6410 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000f60 tx=0x7f6c2400c730 comp rx=0 tx=0).stop 2023-09-30T22:29:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 shutdown_connections 2023-09-30T22:29:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6c3409e440 0x7f6c341a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c3409d6f0 0x7f6c341a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 --2- 172.21.15.32:0/3956599351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c3409e000 0x7f6c341a6410 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 >> 172.21.15.32:0/3956599351 conn(0x7f6c34066f50 msgr2=0x7f6c341baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 shutdown_connections 2023-09-30T22:29:12.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.212+0000 7f6c3c7e9700 1 -- 172.21.15.32:0/3956599351 wait complete. 2023-09-30T22:29:12.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:12.807 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:12.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:12.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.966+0000 7fc03e958700 1 Processor -- start 2023-09-30T22:29:12.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.966+0000 7fc03e958700 1 -- start start 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03e958700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc03812f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03e958700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03812cff0 0x7fc038137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03e958700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03809a510 con 0x7fc03812c2f0 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03e958700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03812c7d0 con 0x7fc03812cff0 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03e958700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03809c190 con 0x7fc03812cc20 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03cef5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc03812cff0 0x7fc038137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc0377fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc03812c2f0 0x7fc03812fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59752/0 (socket says 172.21.15.32:59752) 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc0377fe700 1 -- 172.21.15.32:0/3261237461 learned_addr learned my addr 172.21.15.32:0/3261237461 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc037fff700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc03812f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03cef5700 1 -- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 msgr2=0x7fc03812f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03cef5700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc03812f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03cef5700 1 -- 172.21.15.32:0/3261237461 >> v1:172.21.15.32:6789/0 conn(0x7fc03812c2f0 legacy=0x7fc03812fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc03cef5700 1 -- 172.21.15.32:0/3261237461 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03809b540 con 0x7fc03812cff0 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.967+0000 7fc037fff700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc03812f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.968+0000 7fc03cef5700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812cff0 0x7fc038137cd0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8085ce579f81b65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.968+0000 7fc036ffd700 1 -- 172.21.15.32:0/3261237461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03000e070 con 0x7fc03812cff0 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.968+0000 7fc036ffd700 1 -- 172.21.15.32:0/3261237461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03000bb20 con 0x7fc03812cff0 2023-09-30T22:29:12.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.968+0000 7fc036ffd700 1 -- 172.21.15.32:0/3261237461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030012430 con 0x7fc03812cff0 2023-09-30T22:29:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 -- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812cff0 msgr2=0x7fc038137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812cff0 0x7fc038137cd0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 -- 172.21.15.32:0/3261237461 shutdown_connections 2023-09-30T22:29:12.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812cff0 0x7fc038137cd0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 --2- 172.21.15.32:0/3261237461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc03812f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 -- 172.21.15.32:0/3261237461 >> 172.21.15.32:0/3261237461 conn(0x7fc03809f470 msgr2=0x7fc0380a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 -- 172.21.15.32:0/3261237461 shutdown_connections 2023-09-30T22:29:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.969+0000 7fc03e958700 1 -- 172.21.15.32:0/3261237461 wait complete. 2023-09-30T22:29:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.970+0000 7fc03e958700 1 Processor -- start 2023-09-30T22:29:12.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.970+0000 7fc03e958700 1 -- start start 2023-09-30T22:29:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03e958700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 0x7fc0381c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03e958700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc0381c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03e958700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc03812cff0 0x7fc0381c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03e958700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038071700 con 0x7fc03812c2f0 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03e958700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc038070870 con 0x7fc03812cc20 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03e958700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc038129f10 con 0x7fc03812cff0 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc037fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 0x7fc0381c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03cef5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc03812cff0 0x7fc0381c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc037fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 0x7fc0381c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46504/0 (socket says 172.21.15.32:46504) 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc03cef5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc03812cff0 0x7fc0381c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50264/0 (socket says 172.21.15.32:50264) 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc037fff700 1 -- 172.21.15.32:0/1238051147 learned_addr learned my addr 172.21.15.32:0/1238051147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.971+0000 7fc0377fe700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc0381c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:12.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc037fff700 1 -- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc03812cff0 msgr2=0x7fc0381c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc037fff700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc03812cff0 0x7fc0381c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc037fff700 1 -- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 msgr2=0x7fc0381c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc037fff700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc0381c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc037fff700 1 -- 172.21.15.32:0/1238051147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02800a640 con 0x7fc03812c2f0 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc037fff700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 0x7fc0381c3e70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc028010be0 tx=0x7fc028010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800ca10 con 0x7fc03812c2f0 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800cb70 con 0x7fc03812c2f0 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc030009a70 con 0x7fc03812c2f0 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028018410 con 0x7fc03812c2f0 2023-09-30T22:29:12.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.972+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038072320 con 0x7fc03812c2f0 2023-09-30T22:29:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.973+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc028018570 con 0x7fc03812c2f0 2023-09-30T22:29:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.973+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc028041140 con 0x7fc03812c2f0 2023-09-30T22:29:12.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.974+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0380747f0 con 0x7fc03812c2f0 2023-09-30T22:29:12.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:12.979+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc028018820 con 0x7fc03812c2f0 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.032+0000 7f60545e0700 1 Processor -- start 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.032+0000 7f60545e0700 1 -- start start 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f60545e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f60545e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f60545e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f604c09e410 0x7f604c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f60545e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c06b460 con 0x7f604c09e410 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f60545e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c09b120 con 0x7f604c09d6f0 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f60545e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f604c09a7e0 con 0x7f604c09e000 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f6051b7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f6051b7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50268/0 (socket says 172.21.15.32:50268) 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50164/0 (socket says 172.21.15.32:50164) 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f6051b7b700 1 -- 172.21.15.32:0/4293690309 learned_addr learned my addr 172.21.15.32:0/4293690309 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f6052b7d700 1 --2- 172.21.15.32:0/4293690309 >> v2:172.21.15.32:3300/0 conn(0x7f604c09e410 0x7f604c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 -- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 msgr2=0x7f604c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 --2- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 -- 172.21.15.32:0/4293690309 >> v2:172.21.15.32:3300/0 conn(0x7f604c09e410 msgr2=0x7f604c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 --2- 172.21.15.32:0/4293690309 >> v2:172.21.15.32:3300/0 conn(0x7f604c09e410 0x7f604c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 -- 172.21.15.32:0/4293690309 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c09a500 con 0x7f604c09d6f0 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.033+0000 7f605237c700 1 --2- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c09dac0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6048009870 tx=0x7f6048009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc12c4cf384a29e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.034+0000 7f605137a700 1 -- 172.21.15.32:0/4293690309 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048011830 con 0x7f604c09d6f0 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.034+0000 7f605137a700 1 -- 172.21.15.32:0/4293690309 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048011990 con 0x7f604c09d6f0 2023-09-30T22:29:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.034+0000 7f605137a700 1 -- 172.21.15.32:0/4293690309 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048011d20 con 0x7f604c09d6f0 2023-09-30T22:29:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.034+0000 7f60545e0700 1 -- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 msgr2=0x7f604c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.034+0000 7f60545e0700 1 --2- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c09dac0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6048009870 tx=0x7f6048009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.034+0000 7f60545e0700 1 -- 172.21.15.32:0/4293690309 shutdown_connections 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.035+0000 7f60545e0700 1 --2- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c09dac0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.035+0000 7f60545e0700 1 --2- 172.21.15.32:0/4293690309 >> v2:172.21.15.32:3300/0 conn(0x7f604c09e410 0x7f604c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.035+0000 7f60545e0700 1 --2- 172.21.15.32:0/4293690309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.035+0000 7f60545e0700 1 -- 172.21.15.32:0/4293690309 >> 172.21.15.32:0/4293690309 conn(0x7f604c066f50 msgr2=0x7f604c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.035+0000 7f60545e0700 1 -- 172.21.15.32:0/4293690309 shutdown_connections 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.035+0000 7f60545e0700 1 -- 172.21.15.32:0/4293690309 wait complete. 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 Processor -- start 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 -- start start 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c1a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604c09e410 0x7f604c1a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c1a9850 con 0x7f604c09e410 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c1a8c20 con 0x7f604c09d6f0 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f60545e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0a2020 con 0x7f604c09e000 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f605237c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c1a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f6051b7b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f605237c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c1a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50168/0 (socket says 172.21.15.32:50168) 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f605237c700 1 -- 172.21.15.32:0/2776254414 learned_addr learned my addr 172.21.15.32:0/2776254414 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:13.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.036+0000 7f6052b7d700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604c09e410 0x7f604c1a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6051b7b700 1 -- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 msgr2=0x7f604c1a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6051b7b700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c1a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6051b7b700 1 -- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604c09e410 msgr2=0x7f604c1a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6051b7b700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604c09e410 0x7f604c1a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6051b7b700 1 -- 172.21.15.32:0/2776254414 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60480094a0 con 0x7f604c09e000 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6052b7d700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604c09e410 0x7f604c1a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6051b7b700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1a6300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e8b0 tx=0x7f603c00ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c013680 con 0x7f604c09e000 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0137e0 con 0x7f604c09e000 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c1a7f00 con 0x7f604c09e000 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c013b50 con 0x7f604c09e000 2023-09-30T22:29:13.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.037+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c09c710 con 0x7f604c09e000 2023-09-30T22:29:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.038+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c074ea0 con 0x7f604c09e000 2023-09-30T22:29:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.039+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f603c013cb0 con 0x7f604c09e000 2023-09-30T22:29:13.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.039+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f603c03ff10 con 0x7f604c09e000 2023-09-30T22:29:13.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.043+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f603c011350 con 0x7f604c09e000 2023-09-30T22:29:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.053+0000 7f137cd14700 1 Processor -- start 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.053+0000 7f137cd14700 1 -- start start 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137cd14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137809de00 0x7f137809e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137cd14700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f137813d1f0 0x7f137813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137cd14700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137809a810 con 0x7f137806a770 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137cd14700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137806ab40 con 0x7f137813d1f0 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137cd14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137809ebc0 con 0x7f137809de00 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1376d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f137813d1f0 0x7f137813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f137806a770 0x7f137809d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59760/0 (socket says 172.21.15.32:59760) 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f137659c700 1 -- 172.21.15.32:0/3022615154 learned_addr learned my addr 172.21.15.32:0/3022615154 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1375d9b700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137809de00 0x7f137809e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1376d9d700 1 -- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137809de00 msgr2=0x7f137809e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1376d9d700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137809de00 0x7f137809e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1376d9d700 1 -- 172.21.15.32:0/3022615154 >> v1:172.21.15.32:6789/0 conn(0x7f137806a770 legacy=0x7f137809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1376d9d700 1 -- 172.21.15.32:0/3022615154 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137809e920 con 0x7f137813d1f0 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.054+0000 7f1375d9b700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137809de00 0x7f137809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.055+0000 7f1376d9d700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813f5d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1370009d90 tx=0x7f137000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71199dd4cd2d5ad4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.055+0000 7f137559a700 1 -- 172.21.15.32:0/3022615154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137000e070 con 0x7f137813d1f0 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.055+0000 7f137559a700 1 -- 172.21.15.32:0/3022615154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137000bb20 con 0x7f137813d1f0 2023-09-30T22:29:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.055+0000 7f137559a700 1 -- 172.21.15.32:0/3022615154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370012430 con 0x7f137813d1f0 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 -- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 msgr2=0x7f137813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813f5d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1370009d90 tx=0x7f137000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 -- 172.21.15.32:0/3022615154 shutdown_connections 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813f5d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 --2- 172.21.15.32:0/3022615154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137809de00 0x7f137809e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 -- 172.21.15.32:0/3022615154 >> 172.21.15.32:0/3022615154 conn(0x7f1378066f50 msgr2=0x7f137809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 -- 172.21.15.32:0/3022615154 shutdown_connections 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.056+0000 7f137cd14700 1 -- 172.21.15.32:0/3022615154 wait complete. 2023-09-30T22:29:13.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.057+0000 7f137cd14700 1 Processor -- start 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.057+0000 7f137cd14700 1 -- start start 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.057+0000 7f137cd14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137806a770 0x7f1378071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.057+0000 7f137cd14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f137809de00 0x7f137813b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.057+0000 7f137cd14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f137cd14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137811ede0 con 0x7f137813d1f0 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f137cd14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137811efc0 con 0x7f137806a770 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f137cd14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f137809f1a0 con 0x7f137809de00 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1375d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f137809de00 0x7f137813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46528/0 (socket says 172.21.15.32:46528) 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1375d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f137809de00 0x7f137813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50292/0 (socket says 172.21.15.32:50292) 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 -- 172.21.15.32:0/1618572931 learned_addr learned my addr 172.21.15.32:0/1618572931 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f137659c700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137806a770 0x7f1378071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 -- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f137809de00 msgr2=0x7f137813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f137809de00 0x7f137813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 -- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137806a770 msgr2=0x7f1378071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137806a770 0x7f1378071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 -- 172.21.15.32:0/1618572931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0094a0 con 0x7f137813d1f0 2023-09-30T22:29:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.058+0000 7f1376d9d700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813bb70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1370014040 tx=0x7f1370004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.059+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370012430 con 0x7f137813d1f0 2023-09-30T22:29:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.059+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1370020070 con 0x7f137813d1f0 2023-09-30T22:29:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.059+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1370009a70 con 0x7f137813d1f0 2023-09-30T22:29:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.059+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370003ea0 con 0x7f137813d1f0 2023-09-30T22:29:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.059+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137811e350 con 0x7f137813d1f0 2023-09-30T22:29:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.060+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1370004040 con 0x7f137813d1f0 2023-09-30T22:29:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.060+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1378074ea0 con 0x7f137813d1f0 2023-09-30T22:29:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.060+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1370022070 con 0x7f137813d1f0 2023-09-30T22:29:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.069+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f137001c400 con 0x7f137813d1f0 2023-09-30T22:29:13.228 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:13.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.230+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc038071d00 con 0x7fc03812c2f0 2023-09-30T22:29:13.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.230+0000 7fc034ff9700 1 -- 172.21.15.32:0/1238051147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc028017b20 con 0x7fc03812c2f0 2023-09-30T22:29:13.231 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.234+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 msgr2=0x7fc0381c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.234+0000 7fc03e958700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 0x7fc0381c3e70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc028010be0 tx=0x7fc028010ef0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 shutdown_connections 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc03812cff0 0x7fc0381c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc03812c2f0 0x7fc0381c3e70 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 --2- 172.21.15.32:0/1238051147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc03812cc20 0x7fc0381c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 >> 172.21.15.32:0/1238051147 conn(0x7fc03809f470 msgr2=0x7fc03812f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 shutdown_connections 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.235+0000 7fc03e958700 1 -- 172.21.15.32:0/1238051147 wait complete. 2023-09-30T22:29:13.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:13.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.263+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f604c1a9210 con 0x7f604c09e000 2023-09-30T22:29:13.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.264+0000 7f6042ffd700 1 -- 172.21.15.32:0/2776254414 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f604c1a9210 con 0x7f604c09e000 2023-09-30T22:29:13.265 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 msgr2=0x7f604c1a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1a6300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e8b0 tx=0x7f603c00ebc0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 shutdown_connections 2023-09-30T22:29:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f604c09e410 0x7f604c1a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f604c09d6f0 0x7f604c1a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 --2- 172.21.15.32:0/2776254414 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f604c09e000 0x7f604c1a6300 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 >> 172.21.15.32:0/2776254414 conn(0x7f604c066f50 msgr2=0x7f604c1bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 shutdown_connections 2023-09-30T22:29:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.266+0000 7f60545e0700 1 -- 172.21.15.32:0/2776254414 wait complete. 2023-09-30T22:29:13.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.308+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f137809a3d0 con 0x7f137813d1f0 2023-09-30T22:29:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.309+0000 7f1366ffd700 1 -- 172.21.15.32:0/1618572931 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1370025380 con 0x7f137813d1f0 2023-09-30T22:29:13.310 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 msgr2=0x7f137813bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813bb70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1370014040 tx=0x7f1370004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 shutdown_connections 2023-09-30T22:29:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f137813d1f0 0x7f137813bb70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f137806a770 0x7f1378071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 --2- 172.21.15.32:0/1618572931 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f137809de00 0x7f137813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 >> 172.21.15.32:0/1618572931 conn(0x7f1378066f50 msgr2=0x7f137809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 shutdown_connections 2023-09-30T22:29:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.311+0000 7f137cd14700 1 -- 172.21.15.32:0/1618572931 wait complete. 2023-09-30T22:29:13.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 Processor -- start 2023-09-30T22:29:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 -- start start 2023-09-30T22:29:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc09d630 0x7fbebc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc0a2a70 con 0x7fbebc133930 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0a01d0 con 0x7fbebc0a0a30 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbec2969700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc09e2b0 con 0x7fbebc09d630 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.369+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc09d630 0x7fbebc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50200/0 (socket says 172.21.15.32:50200) 2023-09-30T22:29:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 -- 172.21.15.32:0/230428006 learned_addr learned my addr 172.21.15.32:0/230428006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbec0f06700 1 --1- 172.21.15.32:0/230428006 >> v1:172.21.15.32:6789/0 conn(0x7fbebc133930 0x7fbebc133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59772/0 (socket says 172.21.15.32:59772) 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebaffd700 1 -- 172.21.15.32:0/230428006 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984248952 0 0) 0x7fbebc0a2a70 con 0x7fbebc133930 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 -- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc09d630 msgr2=0x7fbebc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 --2- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc09d630 0x7fbebc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 -- 172.21.15.32:0/230428006 >> v1:172.21.15.32:6789/0 conn(0x7fbebc133930 legacy=0x7fbebc133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 -- 172.21.15.32:0/230428006 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc09dc50 con 0x7fbebc0a0a30 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebb7fe700 1 --2- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc09d630 0x7fbebc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebbfff700 1 --2- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc0a0e00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009920 tx=0x7fbeac009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f459f8b0ace0d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebaffd700 1 -- 172.21.15.32:0/230428006 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac011830 con 0x7fbebc0a0a30 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebaffd700 1 -- 172.21.15.32:0/230428006 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac011990 con 0x7fbebc0a0a30 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.370+0000 7fbebaffd700 1 -- 172.21.15.32:0/230428006 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac011d20 con 0x7fbebc0a0a30 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 -- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 msgr2=0x7fbebc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 --2- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc0a0e00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009920 tx=0x7fbeac009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 -- 172.21.15.32:0/230428006 shutdown_connections 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 --2- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc0a0e00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 --2- 172.21.15.32:0/230428006 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc09d630 0x7fbebc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 -- 172.21.15.32:0/230428006 >> 172.21.15.32:0/230428006 conn(0x7fbebc0a3050 msgr2=0x7fbebc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 -- 172.21.15.32:0/230428006 shutdown_connections 2023-09-30T22:29:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.371+0000 7fbec2969700 1 -- 172.21.15.32:0/230428006 wait complete. 2023-09-30T22:29:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.372+0000 7fbec2969700 1 Processor -- start 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.372+0000 7fbec2969700 1 -- start start 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec2969700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbebc09d630 0x7fbebc1c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec2969700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec2969700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 0x7fbebc1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec2969700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1208e0 con 0x7fbebc09d630 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec2969700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1223c0 con 0x7fbebc0a0a30 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec2969700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1206b0 con 0x7fbebc133930 2023-09-30T22:29:13.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 0x7fbebc1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbebbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbebc09d630 0x7fbebc1c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 0x7fbebc1c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50298/0 (socket says 172.21.15.32:50298) 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 -- 172.21.15.32:0/2391180705 learned_addr learned my addr 172.21.15.32:0/2391180705 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbebb7fe700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 -- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 msgr2=0x7fbebc1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc1c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 -- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbebc09d630 msgr2=0x7fbebc1c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbebc09d630 0x7fbebc1c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 -- 172.21.15.32:0/2391180705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00094a0 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbec0f06700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 0x7fbebc1c8aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400cc70 tx=0x7fbeb400cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400aac0 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.373+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb400ac40 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.374+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac009550 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.374+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40070d0 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.374+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc120ea0 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.374+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbeb4007250 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.375+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbeb403f110 con 0x7fbebc133930 2023-09-30T22:29:13.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.375+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc133610 con 0x7fbebc133930 2023-09-30T22:29:13.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.380+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbebc133610 con 0x7fbebc133930 2023-09-30T22:29:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.584+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbebc133610 con 0x7fbebc133930 2023-09-30T22:29:13.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.584+0000 7fbeb8ff9700 1 -- 172.21.15.32:0/2391180705 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbebc133610 con 0x7fbebc133930 2023-09-30T22:29:13.585 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 msgr2=0x7fbebc1c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 0x7fbebc1c8aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400cc70 tx=0x7fbeb400cf80 comp rx=0 tx=0).stop 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 shutdown_connections 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbebc133930 0x7fbebc1c8aa0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbebc09d630 0x7fbebc1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 --2- 172.21.15.32:0/2391180705 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbebc0a0a30 0x7fbebc1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 >> 172.21.15.32:0/2391180705 conn(0x7fbebc0a3050 msgr2=0x7fbebc1bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 shutdown_connections 2023-09-30T22:29:13.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:13.587+0000 7fbec2969700 1 -- 172.21.15.32:0/2391180705 wait complete. 2023-09-30T22:29:13.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:14.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:14.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:14.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.409+0000 7f8be61e3700 1 Processor -- start 2023-09-30T22:29:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.409+0000 7f8be61e3700 1 -- start start 2023-09-30T22:29:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8be61e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8be61e3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8be0130a40 0x7f8be013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8be61e3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be006bc00 con 0x7f8be012ff20 2023-09-30T22:29:14.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8be61e3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be006c570 con 0x7f8be0130a40 2023-09-30T22:29:14.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8be61e3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00a0040 con 0x7f8be0133a80 2023-09-30T22:29:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8bdf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8bdf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be0133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50312/0 (socket says 172.21.15.32:50312) 2023-09-30T22:29:14.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8bdf7fe700 1 -- 172.21.15.32:0/1525926166 learned_addr learned my addr 172.21.15.32:0/1525926166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8bdeffd700 1 --1- 172.21.15.32:0/1525926166 >> v1:172.21.15.32:6789/0 conn(0x7f8be012ff20 0x7f8be0130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59788/0 (socket says 172.21.15.32:59788) 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.410+0000 7f8bdffff700 1 --2- 172.21.15.32:0/1525926166 >> v2:172.21.15.32:3300/0 conn(0x7f8be0130a40 0x7f8be013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bdf7fe700 1 -- 172.21.15.32:0/1525926166 >> v2:172.21.15.32:3300/0 conn(0x7f8be0130a40 msgr2=0x7f8be013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bdf7fe700 1 --2- 172.21.15.32:0/1525926166 >> v2:172.21.15.32:3300/0 conn(0x7f8be0130a40 0x7f8be013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bdf7fe700 1 -- 172.21.15.32:0/1525926166 >> v1:172.21.15.32:6789/0 conn(0x7f8be012ff20 legacy=0x7f8be0130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bdf7fe700 1 -- 172.21.15.32:0/1525926166 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0133860 con 0x7f8be0133a80 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bde7fc700 1 -- 172.21.15.32:0/1525926166 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680437173 0 0) 0x7f8be006bc00 con 0x7f8be012ff20 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bdf7fe700 1 --2- 172.21.15.32:0/1525926166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be0133e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00098f0 tx=0x7f8bd0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd04fae22fdb220 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bde7fc700 1 -- 172.21.15.32:0/1525926166 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011830 con 0x7f8be0133a80 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bde7fc700 1 -- 172.21.15.32:0/1525926166 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0011990 con 0x7f8be0133a80 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.411+0000 7f8bde7fc700 1 -- 172.21.15.32:0/1525926166 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011d20 con 0x7f8be0133a80 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 -- 172.21.15.32:0/1525926166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 msgr2=0x7f8be0133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1525926166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be0133e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00098f0 tx=0x7f8bd0009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 -- 172.21.15.32:0/1525926166 shutdown_connections 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1525926166 >> v2:172.21.15.32:3300/0 conn(0x7f8be0130a40 0x7f8be013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1525926166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be0133e50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 -- 172.21.15.32:0/1525926166 >> 172.21.15.32:0/1525926166 conn(0x7f8be00a4210 msgr2=0x7f8be00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 -- 172.21.15.32:0/1525926166 shutdown_connections 2023-09-30T22:29:14.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.412+0000 7f8be61e3700 1 -- 172.21.15.32:0/1525926166 wait complete. 2023-09-30T22:29:14.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.413+0000 7f8be61e3700 1 Processor -- start 2023-09-30T22:29:14.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.414+0000 7f8be61e3700 1 -- start start 2023-09-30T22:29:14.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.414+0000 7f8be61e3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 0x7f8be009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.414+0000 7f8be61e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 0x7f8be009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.414+0000 7f8be61e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.414+0000 7f8be61e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be011ed80 con 0x7f8be0130a40 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.414+0000 7f8be61e3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be011faf0 con 0x7f8be012ff20 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8be61e3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8be0120190 con 0x7f8be0133a80 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 0x7f8be009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 0x7f8be009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 0x7f8be009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46568/0 (socket says 172.21.15.32:46568) 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 0x7f8be009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50220/0 (socket says 172.21.15.32:50220) 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 -- 172.21.15.32:0/1659209917 learned_addr learned my addr 172.21.15.32:0/1659209917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdffff700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 -- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 msgr2=0x7f8be009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 -- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 msgr2=0x7f8be009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 0x7f8be009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 -- 172.21.15.32:0/1659209917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0009550 con 0x7f8be0130a40 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdffff700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdf7fe700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 0x7f8be009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bdeffd700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 0x7f8be009c9c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4001eb0 tx=0x7f8bd400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.415+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4004c80 con 0x7f8be0130a40 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.416+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd4004de0 con 0x7f8be0130a40 2023-09-30T22:29:14.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.416+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4007310 con 0x7f8be0130a40 2023-09-30T22:29:14.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.416+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be01206c0 con 0x7f8be0130a40 2023-09-30T22:29:14.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.416+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be011f440 con 0x7f8be0130a40 2023-09-30T22:29:14.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.416+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bd4007470 con 0x7f8be0130a40 2023-09-30T22:29:14.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.417+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be0074ea0 con 0x7f8be0130a40 2023-09-30T22:29:14.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.417+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8bd403f270 con 0x7f8be0130a40 2023-09-30T22:29:14.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.422+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bd40123b0 con 0x7f8be0130a40 2023-09-30T22:29:14.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.440+0000 7f8ce1b61700 1 Processor -- start 2023-09-30T22:29:14.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.440+0000 7f8ce1b61700 1 -- start start 2023-09-30T22:29:14.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8ce1b61700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8cdc09d630 0x7f8cdc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8ce1b61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8ce1b61700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc0a2a70 con 0x7f8cdc0a0a30 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8ce1b61700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cdc0a01d0 con 0x7f8cdc09d630 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8ce1b61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc09e2b0 con 0x7f8cdc138d80 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50334/0 (socket says 172.21.15.32:50334) 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8cdc0a0a30 0x7f8cdc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59794/0 (socket says 172.21.15.32:59794) 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdbfff700 1 -- 172.21.15.32:0/43101911 learned_addr learned my addr 172.21.15.32:0/43101911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdaffd700 1 --2- 172.21.15.32:0/43101911 >> v2:172.21.15.32:3300/0 conn(0x7f8cdc09d630 0x7f8cdc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdaffd700 1 -- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 msgr2=0x7f8cdc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdaffd700 1 --2- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdaffd700 1 -- 172.21.15.32:0/43101911 >> v1:172.21.15.32:6789/0 conn(0x7f8cdc0a0a30 legacy=0x7f8cdc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdaffd700 1 -- 172.21.15.32:0/43101911 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc09dc50 con 0x7f8cdc09d630 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdbfff700 1 --2- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.441+0000 7f8cdaffd700 1 --2- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc09d630 0x7f8cdc0a1510 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009840 tx=0x7f8cc4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb254c44ceed674 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8cda7fc700 1 -- 172.21.15.32:0/43101911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011830 con 0x7f8cdc09d630 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8cda7fc700 1 -- 172.21.15.32:0/43101911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4011990 con 0x7f8cdc09d630 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8cda7fc700 1 -- 172.21.15.32:0/43101911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011d20 con 0x7f8cdc09d630 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 -- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc09d630 msgr2=0x7f8cdc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc09d630 0x7f8cdc0a1510 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009840 tx=0x7f8cc4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 -- 172.21.15.32:0/43101911 shutdown_connections 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/43101911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc09d630 0x7f8cdc0a1510 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 -- 172.21.15.32:0/43101911 >> 172.21.15.32:0/43101911 conn(0x7f8cdc0a3050 msgr2=0x7f8cdc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 -- 172.21.15.32:0/43101911 shutdown_connections 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.442+0000 7f8ce1b61700 1 -- 172.21.15.32:0/43101911 wait complete. 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.444+0000 7f8ce1b61700 1 Processor -- start 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.444+0000 7f8ce1b61700 1 -- start start 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8ce1b61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cdc09d630 0x7f8cdc120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8ce1b61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc0a0a30 0x7f8cdc120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8ce1b61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8ce1b61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc123fb0 con 0x7f8cdc0a0a30 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8ce1b61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0a4e00 con 0x7f8cdc09d630 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8ce1b61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0a0410 con 0x7f8cdc138d80 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc0a0a30 0x7f8cdc120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc121280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50346/0 (socket says 172.21.15.32:50346) 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 -- 172.21.15.32:0/1653352254 learned_addr learned my addr 172.21.15.32:0/1653352254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdb7fe700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cdc09d630 0x7f8cdc120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 -- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cdc09d630 msgr2=0x7f8cdc120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cdc09d630 0x7f8cdc120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 -- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc0a0a30 msgr2=0x7f8cdc120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc0a0a30 0x7f8cdc120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdbfff700 1 -- 172.21.15.32:0/1653352254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4009550 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.445+0000 7f8cdaffd700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc0a0a30 0x7f8cdc120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.446+0000 7f8cdbfff700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc121280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400eb30 tx=0x7f8cd400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.446+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4009e50 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.446+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd400c850 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.446+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc40094a0 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.446+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400cc00 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.446+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc124a20 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.447+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8cd4013470 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.447+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8cd403fac0 con 0x7f8cdc138d80 2023-09-30T22:29:14.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.447+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc0747f0 con 0x7f8cdc138d80 2023-09-30T22:29:14.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.452+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8cd4013720 con 0x7f8cdc138d80 2023-09-30T22:29:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 Processor -- start 2023-09-30T22:29:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 -- start start 2023-09-30T22:29:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c0a2a70 con 0x7f9f0c133930 2023-09-30T22:29:14.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0a01d0 con 0x7f9f0c09d630 2023-09-30T22:29:14.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.527+0000 7f9f11218700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c09e2b0 con 0x7f9f0c0a0a30 2023-09-30T22:29:14.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0b59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9f0c133930 0x7f9f0c133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59798/0 (socket says 172.21.15.32:59798) 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0b59e700 1 -- 172.21.15.32:0/451043041 learned_addr learned my addr 172.21.15.32:0/451043041 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0ad9d700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0a59c700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f09d9b700 1 -- 172.21.15.32:0/451043041 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797037316 0 0) 0x7f9f0c0a2a70 con 0x7f9f0c133930 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f09d9b700 1 -- 172.21.15.32:0/451043041 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef40035b0 con 0x7f9f0c133930 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 msgr2=0x7f9f0c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0ad9d700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/451043041 >> v1:172.21.15.32:6789/0 conn(0x7f9f0c133930 legacy=0x7f9f0c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/451043041 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c09dc50 con 0x7f9f0c0a0a30 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.528+0000 7f9f0ad9d700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c0a0e00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009870 tx=0x7f9efc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20dfbbef7dd886c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f09d9b700 1 -- 172.21.15.32:0/451043041 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc011830 con 0x7f9f0c0a0a30 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f09d9b700 1 -- 172.21.15.32:0/451043041 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc011990 con 0x7f9f0c0a0a30 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f09d9b700 1 -- 172.21.15.32:0/451043041 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc011d20 con 0x7f9f0c0a0a30 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 -- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 msgr2=0x7f9f0c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c0a0e00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009870 tx=0x7f9efc009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 -- 172.21.15.32:0/451043041 shutdown_connections 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c0a0e00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 --2- 172.21.15.32:0/451043041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 -- 172.21.15.32:0/451043041 >> 172.21.15.32:0/451043041 conn(0x7f9f0c0a3050 msgr2=0x7f9f0c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 -- 172.21.15.32:0/451043041 shutdown_connections 2023-09-30T22:29:14.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.529+0000 7f9f11218700 1 -- 172.21.15.32:0/451043041 wait complete. 2023-09-30T22:29:14.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 Processor -- start 2023-09-30T22:29:14.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 -- start start 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c1c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c1c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f0c133930 0x7f9f0c1c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1208e0 con 0x7f9f0c133930 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1223c0 con 0x7f9f0c09d630 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f11218700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1206b0 con 0x7f9f0c0a0a30 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f0ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c1c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f0ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c1c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50254/0 (socket says 172.21.15.32:50254) 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/2210609641 learned_addr learned my addr 172.21.15.32:0/2210609641 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.531+0000 7f9f0b59e700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f0c133930 0x7f9f0c1c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0a59c700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c1c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 msgr2=0x7f9f0c1c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0ad9d700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c1c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f0c133930 msgr2=0x7f9f0c1c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0ad9d700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f0c133930 0x7f9f0c1c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0ad9d700 1 -- 172.21.15.32:0/2210609641 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0094a0 con 0x7f9f0c09d630 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f0ad9d700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c1c80c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0090b0 tx=0x7f9efc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc004450 con 0x7f9f0c09d630 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc01f070 con 0x7f9f0c09d630 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c0a0410 con 0x7f9f0c09d630 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0038e0 con 0x7f9f0c09d630 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.532+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c133550 con 0x7f9f0c09d630 2023-09-30T22:29:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.533+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9efc003a40 con 0x7f9f0c09d630 2023-09-30T22:29:14.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.534+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9efc021070 con 0x7f9f0c09d630 2023-09-30T22:29:14.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.534+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c0747f0 con 0x7f9f0c09d630 2023-09-30T22:29:14.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.539+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9efc024380 con 0x7f9f0c09d630 2023-09-30T22:29:14.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:14.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.665+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8cdc072980 con 0x7f8cdc138d80 2023-09-30T22:29:14.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.666+0000 7f8cd3fff700 1 -- 172.21.15.32:0/1653352254 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8cd4017020 con 0x7f8cdc138d80 2023-09-30T22:29:14.667 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 msgr2=0x7f8cdc121280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc121280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400eb30 tx=0x7f8cd400ee40 comp rx=0 tx=0).stop 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 shutdown_connections 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8cdc138d80 0x7f8cdc121280 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8cdc09d630 0x7f8cdc120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 --2- 172.21.15.32:0/1653352254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cdc0a0a30 0x7f8cdc120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.668+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 >> 172.21.15.32:0/1653352254 conn(0x7f8cdc0a3050 msgr2=0x7f8cdc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.669+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 shutdown_connections 2023-09-30T22:29:14.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.669+0000 7f8ce1b61700 1 -- 172.21.15.32:0/1653352254 wait complete. 2023-09-30T22:29:14.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:14.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.685+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8be006b250 con 0x7f8be0130a40 2023-09-30T22:29:14.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.685+0000 7f8bc7fff700 1 -- 172.21.15.32:0/1659209917 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bd4011360 con 0x7f8be0130a40 2023-09-30T22:29:14.686 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.687+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 msgr2=0x7f8be009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.687+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 0x7f8be009c9c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4001eb0 tx=0x7f8bd400c800 comp rx=0 tx=0).stop 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 shutdown_connections 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8be0133a80 0x7f8be009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8be012ff20 0x7f8be009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 --2- 172.21.15.32:0/1659209917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8be0130a40 0x7f8be009c9c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 >> 172.21.15.32:0/1659209917 conn(0x7f8be00a4210 msgr2=0x7f8be009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 shutdown_connections 2023-09-30T22:29:14.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.688+0000 7f8be61e3700 1 -- 172.21.15.32:0/1659209917 wait complete. 2023-09-30T22:29:14.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.743+0000 7f30eaa87700 1 Processor -- start 2023-09-30T22:29:14.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.743+0000 7f30eaa87700 1 -- start start 2023-09-30T22:29:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30eaa87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30eaa87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30eaa87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30e409a3b0 0x7f30e4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30eaa87700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e409b580 con 0x7f30e409a3b0 2023-09-30T22:29:14.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30eaa87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41313c0 con 0x7f30e4131810 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30eaa87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e406e8c0 con 0x7f30e4099a40 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e8823700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30dbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e9024700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30e409a3b0 0x7f30e4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e8823700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50260/0 (socket says 172.21.15.32:50260) 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30dbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e4099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50374/0 (socket says 172.21.15.32:50374) 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e8823700 1 -- 172.21.15.32:0/2780920218 learned_addr learned my addr 172.21.15.32:0/2780920218 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e9024700 1 -- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 msgr2=0x7f30e4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e9024700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e9024700 1 -- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 msgr2=0x7f30e4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e9024700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e9024700 1 -- 172.21.15.32:0/2780920218 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e409a820 con 0x7f30e409a3b0 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.744+0000 7f30e8823700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30e9024700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e4138650 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009d90 tx=0x7f30dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48717c2783576ad3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30db7fe700 1 -- 172.21.15.32:0/2780920218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00e070 con 0x7f30e409a3b0 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30db7fe700 1 -- 172.21.15.32:0/2780920218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc00bb20 con 0x7f30e409a3b0 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30db7fe700 1 -- 172.21.15.32:0/2780920218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc012430 con 0x7f30e409a3b0 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 -- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 msgr2=0x7f30e4138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e4138650 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009d90 tx=0x7f30dc00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:14.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 -- 172.21.15.32:0/2780920218 shutdown_connections 2023-09-30T22:29:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e4138650 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 --2- 172.21.15.32:0/2780920218 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 -- 172.21.15.32:0/2780920218 >> 172.21.15.32:0/2780920218 conn(0x7f30e40a2aa0 msgr2=0x7f30e40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 -- 172.21.15.32:0/2780920218 shutdown_connections 2023-09-30T22:29:14.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.745+0000 7f30eaa87700 1 -- 172.21.15.32:0/2780920218 wait complete. 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.746+0000 7f30eaa87700 1 Processor -- start 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.746+0000 7f30eaa87700 1 -- start start 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30eaa87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30eaa87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30eaa87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30eaa87700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4070c60 con 0x7f30e409a3b0 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30eaa87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e411e210 con 0x7f30e4131810 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30eaa87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4130770 con 0x7f30e4099a40 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30e9024700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30e8823700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30e9024700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50272/0 (socket says 172.21.15.32:50272) 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30e9024700 1 -- 172.21.15.32:0/34357221 learned_addr learned my addr 172.21.15.32:0/34357221 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.747+0000 7f30dbfff700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30e9024700 1 -- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 msgr2=0x7f30e406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30e9024700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30e9024700 1 -- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 msgr2=0x7f30e406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30e9024700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e406fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30e9024700 1 -- 172.21.15.32:0/34357221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e0009550 con 0x7f30e4131810 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30e9024700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4070060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f30dc014040 tx=0x7f30dc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc012a40 con 0x7f30e4131810 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc020070 con 0x7f30e4131810 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30dc009a70 con 0x7f30e4131810 2023-09-30T22:29:14.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc012e00 con 0x7f30e4131810 2023-09-30T22:29:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.748+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e411d0e0 con 0x7f30e4131810 2023-09-30T22:29:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.749+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f30dc004040 con 0x7f30e4131810 2023-09-30T22:29:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.749+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f30dc022070 con 0x7f30e4131810 2023-09-30T22:29:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.749+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e406eb60 con 0x7f30e4131810 2023-09-30T22:29:14.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.749+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9f0c072980 con 0x7f9f0c09d630 2023-09-30T22:29:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.754+0000 7f9ef37fe700 1 -- 172.21.15.32:0/2210609641 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9efc02eb90 con 0x7f9f0c09d630 2023-09-30T22:29:14.755 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:14.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.754+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30dc01a970 con 0x7f30e4131810 2023-09-30T22:29:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 msgr2=0x7f9f0c1c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c1c80c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0090b0 tx=0x7f9efc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 shutdown_connections 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f0c133930 0x7f9f0c1c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9f0c09d630 0x7f9f0c1c80c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 --2- 172.21.15.32:0/2210609641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9f0c0a0a30 0x7f9f0c1c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 >> 172.21.15.32:0/2210609641 conn(0x7f9f0c0a3050 msgr2=0x7f9f0c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.756+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 shutdown_connections 2023-09-30T22:29:14.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.757+0000 7f9f11218700 1 -- 172.21.15.32:0/2210609641 wait complete. 2023-09-30T22:29:14.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:14.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.957+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f30e4071d80 con 0x7f30e4131810 2023-09-30T22:29:14.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.958+0000 7f30d97fa700 1 -- 172.21.15.32:0/34357221 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f30dc035090 con 0x7f30e4131810 2023-09-30T22:29:14.959 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 msgr2=0x7f30e4070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4070060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f30dc014040 tx=0x7f30dc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 shutdown_connections 2023-09-30T22:29:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f30e4131810 0x7f30e4070060 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f30e4099a40 0x7f30e406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 --2- 172.21.15.32:0/34357221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30e409a3b0 0x7f30e406fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 >> 172.21.15.32:0/34357221 conn(0x7f30e40a2aa0 msgr2=0x7f30e4067820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 shutdown_connections 2023-09-30T22:29:14.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:14.960+0000 7f30eaa87700 1 -- 172.21.15.32:0/34357221 wait complete. 2023-09-30T22:29:14.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:15.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:15.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:15.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.834+0000 7f3beb186700 1 Processor -- start 2023-09-30T22:29:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.834+0000 7f3beb186700 1 -- start start 2023-09-30T22:29:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3beb186700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3be409e1d0 0x7f3be4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3beb186700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be4139170 0x7f3be413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3beb186700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be409b120 con 0x7f3be409d6f0 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3beb186700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be406b460 con 0x7f3be409e1d0 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3beb186700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be409a7e0 con 0x7f3be4139170 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be8f22700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3be409d6f0 0x7f3be409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59812/0 (socket says 172.21.15.32:59812) 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be8f22700 1 -- 172.21.15.32:0/2077499429 learned_addr learned my addr 172.21.15.32:0/2077499429 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be3fff700 1 --2- 172.21.15.32:0/2077499429 >> v2:172.21.15.32:3300/0 conn(0x7f3be409e1d0 0x7f3be4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be9723700 1 --2- 172.21.15.32:0/2077499429 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be4139170 0x7f3be413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be37fe700 1 -- 172.21.15.32:0/2077499429 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562180867 0 0) 0x7f3be409b120 con 0x7f3be409d6f0 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be37fe700 1 -- 172.21.15.32:0/2077499429 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd40035b0 con 0x7f3be409d6f0 2023-09-30T22:29:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be9723700 1 -- 172.21.15.32:0/2077499429 >> v2:172.21.15.32:3300/0 conn(0x7f3be409e1d0 msgr2=0x7f3be4138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be9723700 1 --2- 172.21.15.32:0/2077499429 >> v2:172.21.15.32:3300/0 conn(0x7f3be409e1d0 0x7f3be4138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be9723700 1 -- 172.21.15.32:0/2077499429 >> v1:172.21.15.32:6789/0 conn(0x7f3be409d6f0 legacy=0x7f3be409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be9723700 1 -- 172.21.15.32:0/2077499429 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be409a500 con 0x7f3be4139170 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be3fff700 1 --2- 172.21.15.32:0/2077499429 >> v2:172.21.15.32:3300/0 conn(0x7f3be409e1d0 0x7f3be4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.835+0000 7f3be9723700 1 --2- 172.21.15.32:0/2077499429 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be4139170 0x7f3be413b550 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009dc0 tx=0x7f3bdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3411f66a3e852c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3be37fe700 1 -- 172.21.15.32:0/2077499429 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00e070 con 0x7f3be4139170 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3be37fe700 1 -- 172.21.15.32:0/2077499429 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc00bae0 con 0x7f3be4139170 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3be37fe700 1 -- 172.21.15.32:0/2077499429 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00be90 con 0x7f3be4139170 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3beb186700 1 -- 172.21.15.32:0/2077499429 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be4139170 msgr2=0x7f3be413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3beb186700 1 --2- 172.21.15.32:0/2077499429 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be4139170 0x7f3be413b550 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009dc0 tx=0x7f3bdc00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:15.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3beb186700 1 -- 172.21.15.32:0/2077499429 shutdown_connections 2023-09-30T22:29:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3beb186700 1 --2- 172.21.15.32:0/2077499429 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be4139170 0x7f3be413b550 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3beb186700 1 --2- 172.21.15.32:0/2077499429 >> v2:172.21.15.32:3300/0 conn(0x7f3be409e1d0 0x7f3be4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.836+0000 7f3beb186700 1 -- 172.21.15.32:0/2077499429 >> 172.21.15.32:0/2077499429 conn(0x7f3be4066f50 msgr2=0x7f3be409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.837+0000 7f3beb186700 1 -- 172.21.15.32:0/2077499429 shutdown_connections 2023-09-30T22:29:15.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.837+0000 7f3beb186700 1 -- 172.21.15.32:0/2077499429 wait complete. 2023-09-30T22:29:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 Processor -- start 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 -- start start 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be409d6f0 0x7f3be4120700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 0x7f3be4120c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be4139170 0x7f3be4121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be409c710 con 0x7f3be409e1d0 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4123f40 con 0x7f3be409d6f0 2023-09-30T22:29:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3beb186700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40a17a0 con 0x7f3be4139170 2023-09-30T22:29:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.838+0000 7f3be8f22700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be409d6f0 0x7f3be4120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 0x7f3be4120c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 0x7f3be4120c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46622/0 (socket says 172.21.15.32:46622) 2023-09-30T22:29:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 -- 172.21.15.32:0/1929153101 learned_addr learned my addr 172.21.15.32:0/1929153101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be9723700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be4139170 0x7f3be4121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 -- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be4139170 msgr2=0x7f3be4121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be4139170 0x7f3be4121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 -- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be409d6f0 msgr2=0x7f3be4120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be409d6f0 0x7f3be4120700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 -- 172.21.15.32:0/1929153101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00094a0 con 0x7f3be409e1d0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be8f22700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be409d6f0 0x7f3be4120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be9723700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be4139170 0x7f3be4121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be3fff700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 0x7f3be4120c40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000ebe0 tx=0x7f3bd000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000c850 con 0x7f3be409e1d0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd000c9b0 con 0x7f3be409e1d0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc009a70 con 0x7f3be409e1d0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.839+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000cd40 con 0x7f3be409e1d0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.840+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be41249b0 con 0x7f3be409e1d0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.841+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be4136670 con 0x7f3be409e1d0 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.841+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bd0013470 con 0x7f3be409e1d0 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.841+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3bd003fe50 con 0x7f3be409e1d0 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.846+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bd000cea0 con 0x7f3be409e1d0 2023-09-30T22:29:15.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 Processor -- start 2023-09-30T22:29:15.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 -- start start 2023-09-30T22:29:15.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed28133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed2812ff20 0x7fed28130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2806bc00 con 0x7fed28130870 2023-09-30T22:29:15.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2806c570 con 0x7fed2812ff20 2023-09-30T22:29:15.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.872+0000 7fed2d648700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed280a0040 con 0x7fed28133a80 2023-09-30T22:29:15.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed267fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed2812ff20 0x7fed28130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed26ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed28133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed277fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed28130870 0x7fed2813a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59820/0 (socket says 172.21.15.32:59820) 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed277fe700 1 -- 172.21.15.32:0/4193297577 learned_addr learned my addr 172.21.15.32:0/4193297577 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed25ffb700 1 -- 172.21.15.32:0/4193297577 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580198640 0 0) 0x7fed2806bc00 con 0x7fed28130870 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed25ffb700 1 -- 172.21.15.32:0/4193297577 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed100035b0 con 0x7fed28130870 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed26ffd700 1 -- 172.21.15.32:0/4193297577 >> v2:172.21.15.32:3300/0 conn(0x7fed2812ff20 msgr2=0x7fed28130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed26ffd700 1 --2- 172.21.15.32:0/4193297577 >> v2:172.21.15.32:3300/0 conn(0x7fed2812ff20 0x7fed28130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed26ffd700 1 -- 172.21.15.32:0/4193297577 >> v1:172.21.15.32:6789/0 conn(0x7fed28130870 legacy=0x7fed2813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed26ffd700 1 -- 172.21.15.32:0/4193297577 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28133860 con 0x7fed28133a80 2023-09-30T22:29:15.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed267fc700 1 --2- 172.21.15.32:0/4193297577 >> v2:172.21.15.32:3300/0 conn(0x7fed2812ff20 0x7fed28130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.873+0000 7fed26ffd700 1 --2- 172.21.15.32:0/4193297577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed28133e50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fed18009840 tx=0x7fed18009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ee9b4d4672fb08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed25ffb700 1 -- 172.21.15.32:0/4193297577 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18011830 con 0x7fed28133a80 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed25ffb700 1 -- 172.21.15.32:0/4193297577 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed18011990 con 0x7fed28133a80 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed25ffb700 1 -- 172.21.15.32:0/4193297577 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18011d20 con 0x7fed28133a80 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed2d648700 1 -- 172.21.15.32:0/4193297577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 msgr2=0x7fed28133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed2d648700 1 --2- 172.21.15.32:0/4193297577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed28133e50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fed18009840 tx=0x7fed18009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed2d648700 1 -- 172.21.15.32:0/4193297577 shutdown_connections 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed2d648700 1 --2- 172.21.15.32:0/4193297577 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed28133e50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.874+0000 7fed2d648700 1 --2- 172.21.15.32:0/4193297577 >> v2:172.21.15.32:3300/0 conn(0x7fed2812ff20 0x7fed28130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.875+0000 7fed2d648700 1 -- 172.21.15.32:0/4193297577 >> 172.21.15.32:0/4193297577 conn(0x7fed280a4210 msgr2=0x7fed280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:15.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.875+0000 7fed2d648700 1 -- 172.21.15.32:0/4193297577 shutdown_connections 2023-09-30T22:29:15.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.875+0000 7fed2d648700 1 -- 172.21.15.32:0/4193297577 wait complete. 2023-09-30T22:29:15.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 Processor -- start 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 -- start start 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 0x7fed2809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed28130870 0x7fed2809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed2809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2811edb0 con 0x7fed28130870 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2811f420 con 0x7fed2812ff20 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.876+0000 7fed2d648700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed2811fb20 con 0x7fed28133a80 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 0x7fed2809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 0x7fed2809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50292/0 (socket says 172.21.15.32:50292) 2023-09-30T22:29:15.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed267fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed28130870 0x7fed2809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/3139451853 learned_addr learned my addr 172.21.15.32:0/3139451853 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed277fe700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed2809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 msgr2=0x7fed2809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed2809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed28130870 msgr2=0x7fed2809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed28130870 0x7fed2809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 -- 172.21.15.32:0/3139451853 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c0094a0 con 0x7fed2812ff20 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed267fc700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed28130870 0x7fed2809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed26ffd700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 0x7fed2809c480 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fed180090b0 tx=0x7fed18004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18004590 con 0x7fed2812ff20 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1801f070 con 0x7fed2812ff20 2023-09-30T22:29:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180094a0 con 0x7fed2812ff20 2023-09-30T22:29:15.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.877+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed180038e0 con 0x7fed2812ff20 2023-09-30T22:29:15.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.878+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2811ea90 con 0x7fed2812ff20 2023-09-30T22:29:15.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.878+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fed18003a40 con 0x7fed2812ff20 2023-09-30T22:29:15.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.879+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fed18021070 con 0x7fed2812ff20 2023-09-30T22:29:15.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.879+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed28074ea0 con 0x7fed2812ff20 2023-09-30T22:29:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.884+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed18015080 con 0x7fed2812ff20 2023-09-30T22:29:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 Processor -- start 2023-09-30T22:29:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 -- start start 2023-09-30T22:29:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29440984f0 0x7f29440988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409e210 0x7f29441395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2944099620 con 0x7f294409d6f0 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294409a9e0 con 0x7f294409e210 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.925+0000 7f294927b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f294409bce0 con 0x7f29440984f0 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29437fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409e210 0x7f29441395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f293a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f294409d6f0 0x7f294409db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59822/0 (socket says 172.21.15.32:59822) 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f293a7fc700 1 -- 172.21.15.32:0/431570896 learned_addr learned my addr 172.21.15.32:0/431570896 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f2942ffd700 1 --2- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29440984f0 0x7f29440988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29437fe700 1 -- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29440984f0 msgr2=0x7f29440988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29437fe700 1 --2- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29440984f0 0x7f29440988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29437fe700 1 -- 172.21.15.32:0/431570896 >> v1:172.21.15.32:6789/0 conn(0x7f294409d6f0 legacy=0x7f294409db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29437fe700 1 -- 172.21.15.32:0/431570896 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294409aca0 con 0x7f294409e210 2023-09-30T22:29:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29427fc700 1 -- 172.21.15.32:0/431570896 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202146147 0 0) 0x7f2944099620 con 0x7f294409d6f0 2023-09-30T22:29:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29437fe700 1 --2- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409e210 0x7f29441395d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f293c009dc0 tx=0x7f293c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e90d0b4d0a943ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.926+0000 7f29427fc700 1 -- 172.21.15.32:0/431570896 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00e070 con 0x7f294409e210 2023-09-30T22:29:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f29427fc700 1 -- 172.21.15.32:0/431570896 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c00bb20 con 0x7f294409e210 2023-09-30T22:29:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f29427fc700 1 -- 172.21.15.32:0/431570896 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c012430 con 0x7f294409e210 2023-09-30T22:29:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 -- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409e210 msgr2=0x7f29441395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 --2- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409e210 0x7f29441395d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f293c009dc0 tx=0x7f293c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 -- 172.21.15.32:0/431570896 shutdown_connections 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 --2- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409e210 0x7f29441395d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 --2- 172.21.15.32:0/431570896 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29440984f0 0x7f29440988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 -- 172.21.15.32:0/431570896 >> 172.21.15.32:0/431570896 conn(0x7f294409f470 msgr2=0x7f29440a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 -- 172.21.15.32:0/431570896 shutdown_connections 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.927+0000 7f294927b700 1 -- 172.21.15.32:0/431570896 wait complete. 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.928+0000 7f294927b700 1 Processor -- start 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.928+0000 7f294927b700 1 -- start start 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f294927b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 0x7f294412e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f294927b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409d6f0 0x7f294412e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f294927b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f294409e210 0x7f294406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f294927b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294412c280 con 0x7f29440984f0 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f294927b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29440a1cf0 con 0x7f294409d6f0 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f294927b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f294412af20 con 0x7f294409e210 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 0x7f294412e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f293a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409d6f0 0x7f294412e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 0x7f294412e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46656/0 (socket says 172.21.15.32:46656) 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 -- 172.21.15.32:0/2504983529 learned_addr learned my addr 172.21.15.32:0/2504983529 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f29437fe700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f294409e210 0x7f294406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 -- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f294409e210 msgr2=0x7f294406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f294409e210 0x7f294406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 -- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409d6f0 msgr2=0x7f294412e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409d6f0 0x7f294412e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f2942ffd700 1 -- 172.21.15.32:0/2504983529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c009a70 con 0x7f29440984f0 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.929+0000 7f293a7fc700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409d6f0 0x7f294412e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.930+0000 7f2942ffd700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 0x7f294412e380 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f293400e9b0 tx=0x7f293400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.930+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934009710 con 0x7f29440984f0 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.930+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293400c850 con 0x7f29440984f0 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.930+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293400cbc0 con 0x7f29440984f0 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.930+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294412b2c0 con 0x7f29440984f0 2023-09-30T22:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.930+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294412b930 con 0x7f29440984f0 2023-09-30T22:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.931+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f293400cd20 con 0x7f29440984f0 2023-09-30T22:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.931+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f293403f730 con 0x7f29440984f0 2023-09-30T22:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.931+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29440747f0 con 0x7f29440984f0 2023-09-30T22:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:15.936+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29340232f0 con 0x7f29440984f0 2023-09-30T22:29:15.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:16.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.080+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3be4074ea0 con 0x7f3be409e1d0 2023-09-30T22:29:16.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.081+0000 7f3be17fa700 1 -- 172.21.15.32:0/1929153101 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bd0017030 con 0x7f3be409e1d0 2023-09-30T22:29:16.084 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 msgr2=0x7f3be4120c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 0x7f3be4120c40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000ebe0 tx=0x7f3bd000eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 shutdown_connections 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3be4139170 0x7f3be4121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3be409d6f0 0x7f3be4120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 --2- 172.21.15.32:0/1929153101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3be409e1d0 0x7f3be4120c40 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 >> 172.21.15.32:0/1929153101 conn(0x7f3be4066f50 msgr2=0x7f3be406ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 shutdown_connections 2023-09-30T22:29:16.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.083+0000 7f3beb186700 1 -- 172.21.15.32:0/1929153101 wait complete. 2023-09-30T22:29:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.095+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fed2806b250 con 0x7fed2812ff20 2023-09-30T22:29:16.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.095+0000 7fed0f7fe700 1 -- 172.21.15.32:0/3139451853 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed1802b020 con 0x7fed2812ff20 2023-09-30T22:29:16.096 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:16.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 msgr2=0x7fed2809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 0x7fed2809c480 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fed180090b0 tx=0x7fed18004750 comp rx=0 tx=0).stop 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 shutdown_connections 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed28133a80 0x7fed2809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed2812ff20 0x7fed2809c480 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 --2- 172.21.15.32:0/3139451853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed28130870 0x7fed2809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 >> 172.21.15.32:0/3139451853 conn(0x7fed280a4210 msgr2=0x7fed2809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 shutdown_connections 2023-09-30T22:29:16.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.098+0000 7fed2d648700 1 -- 172.21.15.32:0/3139451853 wait complete. 2023-09-30T22:29:16.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:16.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 Processor -- start 2023-09-30T22:29:16.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 -- start start 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1628130750 0x7f1628130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 0x7f16281384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162809c190 con 0x7f162809d6f0 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162809b090 con 0x7f162809e250 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f162d49b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f162809a6c0 con 0x7f1628130750 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.119+0000 7f16277fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 0x7f16281384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f16277fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 0x7f16281384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50314/0 (socket says 172.21.15.32:50314) 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1626ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1628130750 0x7f1628130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f16267fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f162809d6f0 0x7f162809db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59826/0 (socket says 172.21.15.32:59826) 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f16277fe700 1 -- 172.21.15.32:0/2727158044 learned_addr learned my addr 172.21.15.32:0/2727158044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1626ffd700 1 -- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 msgr2=0x7f16281384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1626ffd700 1 --2- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 0x7f16281384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1625ffb700 1 -- 172.21.15.32:0/2727158044 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036567618 0 0) 0x7f162809c190 con 0x7f162809d6f0 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1626ffd700 1 -- 172.21.15.32:0/2727158044 >> v1:172.21.15.32:6789/0 conn(0x7f162809d6f0 legacy=0x7f162809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1626ffd700 1 -- 172.21.15.32:0/2727158044 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1628131060 con 0x7f1628130750 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f16277fe700 1 --2- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 0x7f16281384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:16.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1626ffd700 1 --2- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1628130750 0x7f1628130b20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f161800dc90 tx=0x7f161800f050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd43aaa255e92fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1625ffb700 1 -- 172.21.15.32:0/2727158044 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618012070 con 0x7f1628130750 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1625ffb700 1 -- 172.21.15.32:0/2727158044 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161800d2e0 con 0x7f1628130750 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.120+0000 7f1625ffb700 1 -- 172.21.15.32:0/2727158044 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800f8f0 con 0x7f1628130750 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 -- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1628130750 msgr2=0x7f1628130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 --2- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1628130750 0x7f1628130b20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f161800dc90 tx=0x7f161800f050 comp rx=0 tx=0).stop 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 -- 172.21.15.32:0/2727158044 shutdown_connections 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 --2- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809e250 0x7f16281384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 --2- 172.21.15.32:0/2727158044 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1628130750 0x7f1628130b20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 -- 172.21.15.32:0/2727158044 >> 172.21.15.32:0/2727158044 conn(0x7f162809f470 msgr2=0x7f16280a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 -- 172.21.15.32:0/2727158044 shutdown_connections 2023-09-30T22:29:16.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.121+0000 7f162d49b700 1 -- 172.21.15.32:0/2727158044 wait complete. 2023-09-30T22:29:16.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.122+0000 7f162d49b700 1 Processor -- start 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.122+0000 7f162d49b700 1 -- start start 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f162d49b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 0x7f162812ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f162d49b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162809e250 0x7f162812b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f162d49b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628130750 0x7f162812b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f162d49b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162812efa0 con 0x7f1628130750 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f162d49b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162809b340 con 0x7f162809d6f0 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f162d49b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f162812ff70 con 0x7f162809e250 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 0x7f162812ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 0x7f162812ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50322/0 (socket says 172.21.15.32:50322) 2023-09-30T22:29:16.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 -- 172.21.15.32:0/2580712839 learned_addr learned my addr 172.21.15.32:0/2580712839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f16277fe700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628130750 0x7f162812b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f16267fc700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162809e250 0x7f162812b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 -- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162809e250 msgr2=0x7f162812b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162809e250 0x7f162812b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 -- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628130750 msgr2=0x7f162812b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628130750 0x7f162812b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 -- 172.21.15.32:0/2580712839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162000bb50 con 0x7f162809d6f0 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f16277fe700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628130750 0x7f162812b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.123+0000 7f1626ffd700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 0x7f162812ad30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f162809a800 tx=0x7f1618004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:16.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.124+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16180036a0 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.124+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1618003800 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.124+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161800d940 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.124+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618003bb0 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.124+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162812e040 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.125+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1618003d10 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.125+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1618044590 con 0x7f162809d6f0 2023-09-30T22:29:16.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.125+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16280747f0 con 0x7f162809d6f0 2023-09-30T22:29:16.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.130+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f161802d020 con 0x7f162809d6f0 2023-09-30T22:29:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.147+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2944098e90 con 0x7f29440984f0 2023-09-30T22:29:16.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.148+0000 7f293bfff700 1 -- 172.21.15.32:0/2504983529 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2934021030 con 0x7f29440984f0 2023-09-30T22:29:16.149 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 msgr2=0x7f294412e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 0x7f294412e380 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f293400e9b0 tx=0x7f293400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 shutdown_connections 2023-09-30T22:29:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f294409e210 0x7f294406d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29440984f0 0x7f294412e380 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 --2- 172.21.15.32:0/2504983529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f294409d6f0 0x7f294412e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 >> 172.21.15.32:0/2504983529 conn(0x7f294409f470 msgr2=0x7f29440a0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 shutdown_connections 2023-09-30T22:29:16.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.150+0000 7f294927b700 1 -- 172.21.15.32:0/2504983529 wait complete. 2023-09-30T22:29:16.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.336+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f162806b250 con 0x7f162809d6f0 2023-09-30T22:29:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.336+0000 7f160f7fe700 1 -- 172.21.15.32:0/2580712839 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f161800b970 con 0x7f162809d6f0 2023-09-30T22:29:16.338 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 msgr2=0x7f162812ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 0x7f162812ad30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f162809a800 tx=0x7f1618004750 comp rx=0 tx=0).stop 2023-09-30T22:29:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 shutdown_connections 2023-09-30T22:29:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1628130750 0x7f162812b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f162809d6f0 0x7f162812ad30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 --2- 172.21.15.32:0/2580712839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162809e250 0x7f162812b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.338+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 >> 172.21.15.32:0/2580712839 conn(0x7f162809f470 msgr2=0x7f1628071820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.339+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 shutdown_connections 2023-09-30T22:29:16.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:16.339+0000 7f162d49b700 1 -- 172.21.15.32:0/2580712839 wait complete. 2023-09-30T22:29:16.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:17.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:17.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:17.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.253+0000 7ff47857a700 1 Processor -- start 2023-09-30T22:29:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff47857a700 1 -- start start 2023-09-30T22:29:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff47857a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 0x7ff470138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff47857a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff47857a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47006ef30 con 0x7ff4701338a0 2023-09-30T22:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff47857a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47006fef0 con 0x7ff470136710 2023-09-30T22:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff47857a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4701334b0 con 0x7ff470139030 2023-09-30T22:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff475b15700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 0x7ff470138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff475b15700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 0x7ff470138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50334/0 (socket says 172.21.15.32:50334) 2023-09-30T22:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff475b15700 1 -- 172.21.15.32:0/1298976426 learned_addr learned my addr 172.21.15.32:0/1298976426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.254+0000 7ff476b17700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff475314700 1 -- 172.21.15.32:0/1298976426 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510953719 0 0) 0x7ff47006ef30 con 0x7ff4701338a0 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff476b17700 1 -- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 msgr2=0x7ff470138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff476b17700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 0x7ff470138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff476b17700 1 -- 172.21.15.32:0/1298976426 >> v1:172.21.15.32:6789/0 conn(0x7ff4701338a0 legacy=0x7ff470133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff476b17700 1 -- 172.21.15.32:0/1298976426 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47006fc30 con 0x7ff470139030 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff475b15700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 0x7ff470138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff476b17700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47013b410 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff468009dc0 tx=0x7ff46800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13149cbf902ba314 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff475314700 1 -- 172.21.15.32:0/1298976426 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800e070 con 0x7ff470139030 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff475314700 1 -- 172.21.15.32:0/1298976426 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46800bb20 con 0x7ff470139030 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.255+0000 7ff475314700 1 -- 172.21.15.32:0/1298976426 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468012430 con 0x7ff470139030 2023-09-30T22:29:17.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.256+0000 7ff47857a700 1 -- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 msgr2=0x7ff47013b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.256+0000 7ff47857a700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47013b410 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff468009dc0 tx=0x7ff46800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.257+0000 7ff47857a700 1 -- 172.21.15.32:0/1298976426 shutdown_connections 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.257+0000 7ff47857a700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47013b410 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.257+0000 7ff47857a700 1 --2- 172.21.15.32:0/1298976426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff470136710 0x7ff470138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.257+0000 7ff47857a700 1 -- 172.21.15.32:0/1298976426 >> 172.21.15.32:0/1298976426 conn(0x7ff4700a2aa0 msgr2=0x7ff4700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.257+0000 7ff47857a700 1 -- 172.21.15.32:0/1298976426 shutdown_connections 2023-09-30T22:29:17.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.257+0000 7ff47857a700 1 -- 172.21.15.32:0/1298976426 wait complete. 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 Processor -- start 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 -- start start 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 0x7ff4701c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff470136710 0x7ff4701c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47011ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47011e080 con 0x7ff470136710 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff470138d40 con 0x7ff4701338a0 2023-09-30T22:29:17.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff47857a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47011eb30 con 0x7ff470139030 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff476316700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 0x7ff4701c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff475b15700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff470136710 0x7ff4701c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff476316700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 0x7ff4701c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50342/0 (socket says 172.21.15.32:50342) 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff476316700 1 -- 172.21.15.32:0/672180052 learned_addr learned my addr 172.21.15.32:0/672180052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.258+0000 7ff476b17700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47011ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff476316700 1 -- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 msgr2=0x7ff47011ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff476316700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47011ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff476316700 1 -- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff470136710 msgr2=0x7ff4701c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff476316700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff470136710 0x7ff4701c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff476316700 1 -- 172.21.15.32:0/672180052 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009550 con 0x7ff4701338a0 2023-09-30T22:29:17.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff475b15700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff470136710 0x7ff4701c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff476316700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 0x7ff4701c85e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c520 tx=0x7ff46c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0054e0 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00ce90 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff468009a70 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c003880 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.259+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47011e350 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.260+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff46c0039e0 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.260+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff46c03ea10 con 0x7ff4701338a0 2023-09-30T22:29:17.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.261+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff470074ea0 con 0x7ff4701338a0 2023-09-30T22:29:17.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.266+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff46c0194a0 con 0x7ff4701338a0 2023-09-30T22:29:17.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.291+0000 7f2087a0d700 1 Processor -- start 2023-09-30T22:29:17.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.291+0000 7f2087a0d700 1 -- start start 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2087a0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20801338a0 0x7f2080133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2087a0d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f208009de50 0x7f208009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2087a0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2087a0d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208006fef0 con 0x7f20801338a0 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2087a0d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208006ef30 con 0x7f208009de50 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2087a0d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20801334b0 con 0x7f208009e760 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20801338a0 0x7f2080133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20801338a0 0x7f2080133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46690/0 (socket says 172.21.15.32:46690) 2023-09-30T22:29:17.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 -- 172.21.15.32:0/1903173009 learned_addr learned my addr 172.21.15.32:0/1903173009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2085faa700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f2084fa8700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f208009de50 0x7f208009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 -- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 msgr2=0x7f208013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 -- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f208009de50 msgr2=0x7f208009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.292+0000 7f20857a9700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f208009de50 0x7f208009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.293+0000 7f20857a9700 1 -- 172.21.15.32:0/1903173009 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208006fc30 con 0x7f20801338a0 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.293+0000 7f20857a9700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20801338a0 0x7f2080133c70 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f207c0098f0 tx=0x7f207c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f387afc2d65a1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.293+0000 7f2077fff700 1 -- 172.21.15.32:0/1903173009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c011830 con 0x7f20801338a0 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.293+0000 7f2077fff700 1 -- 172.21.15.32:0/1903173009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c011990 con 0x7f20801338a0 2023-09-30T22:29:17.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.293+0000 7f2077fff700 1 -- 172.21.15.32:0/1903173009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c011d20 con 0x7f20801338a0 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 -- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20801338a0 msgr2=0x7f2080133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20801338a0 0x7f2080133c70 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f207c0098f0 tx=0x7f207c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 -- 172.21.15.32:0/1903173009 shutdown_connections 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20801338a0 0x7f2080133c70 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 --2- 172.21.15.32:0/1903173009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f208009de50 0x7f208009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 -- 172.21.15.32:0/1903173009 >> 172.21.15.32:0/1903173009 conn(0x7f20800a2aa0 msgr2=0x7f20800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 -- 172.21.15.32:0/1903173009 shutdown_connections 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.294+0000 7f2087a0d700 1 -- 172.21.15.32:0/1903173009 wait complete. 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.295+0000 7f2087a0d700 1 Processor -- start 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.295+0000 7f2087a0d700 1 -- start start 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.295+0000 7f2087a0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 0x7f208011e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.295+0000 7f2087a0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208011eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.295+0000 7f2087a0d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20801338a0 0x7f208011f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.295+0000 7f2087a0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208006f6b0 con 0x7f208009de50 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f2087a0d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208011fdd0 con 0x7f20801338a0 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f2087a0d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20801221d0 con 0x7f208009e760 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 0x7f208011e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 0x7f208011e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46702/0 (socket says 172.21.15.32:46702) 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 -- 172.21.15.32:0/2940116897 learned_addr learned my addr 172.21.15.32:0/2940116897 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f2084fa8700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208011eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f2085faa700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20801338a0 0x7f208011f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 -- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 msgr2=0x7f208011eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208011eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 -- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20801338a0 msgr2=0x7f208011f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20801338a0 0x7f208011f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 -- 172.21.15.32:0/2940116897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c009550 con 0x7f208009de50 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f20857a9700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 0x7f208011e990 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f207c000c00 tx=0x7f207c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c004240 con 0x7f208009de50 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.296+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c01f070 con 0x7f208009de50 2023-09-30T22:29:17.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.297+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080120dc0 con 0x7f208009de50 2023-09-30T22:29:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.297+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0045e0 con 0x7f208009de50 2023-09-30T22:29:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.297+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208012f8e0 con 0x7f208009de50 2023-09-30T22:29:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.298+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f207c021030 con 0x7f208009de50 2023-09-30T22:29:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.298+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f207c040910 con 0x7f208009de50 2023-09-30T22:29:17.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.298+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2080074ea0 con 0x7f208009de50 2023-09-30T22:29:17.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.303+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f207c015030 con 0x7f208009de50 2023-09-30T22:29:17.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.332+0000 7f7f2e8e5700 1 Processor -- start 2023-09-30T22:29:17.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2e8e5700 1 -- start start 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2e8e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f28130750 0x7f7f28130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2e8e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f2809d6f0 0x7f7f2809db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2e8e5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2809c190 con 0x7f7f2809e080 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2e8e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2809e520 con 0x7f7f2809d6f0 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2e8e5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2809b090 con 0x7f7f28130750 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f2809d6f0 0x7f7f2809db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2ce82700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f2809e080 0x7f7f281374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59856/0 (socket says 172.21.15.32:59856) 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f2ce82700 1 -- 172.21.15.32:0/3844177235 learned_addr learned my addr 172.21.15.32:0/3844177235 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.333+0000 7f7f27fff700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f28130750 0x7f7f28130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f26ffd700 1 -- 172.21.15.32:0/3844177235 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970540155 0 0) 0x7f7f2809c190 con 0x7f7f2809e080 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f26ffd700 1 -- 172.21.15.32:0/3844177235 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f100057c0 con 0x7f7f2809e080 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f277fe700 1 -- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f28130750 msgr2=0x7f7f28130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f277fe700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f28130750 0x7f7f28130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f277fe700 1 -- 172.21.15.32:0/3844177235 >> v1:172.21.15.32:6789/0 conn(0x7f7f2809e080 legacy=0x7f7f281374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f277fe700 1 -- 172.21.15.32:0/3844177235 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2809a6c0 con 0x7f7f2809d6f0 2023-09-30T22:29:17.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.334+0000 7f7f27fff700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f28130750 0x7f7f28130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.335+0000 7f7f277fe700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2809d6f0 0x7f7f2809db40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009840 tx=0x7f7f1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3715013c2bb50cca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.335+0000 7f7f26ffd700 1 -- 172.21.15.32:0/3844177235 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c011830 con 0x7f7f2809d6f0 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.335+0000 7f7f26ffd700 1 -- 172.21.15.32:0/3844177235 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c011990 con 0x7f7f2809d6f0 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.335+0000 7f7f26ffd700 1 -- 172.21.15.32:0/3844177235 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c011d00 con 0x7f7f2809d6f0 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2809d6f0 msgr2=0x7f7f2809db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2809d6f0 0x7f7f2809db40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009840 tx=0x7f7f1c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/3844177235 shutdown_connections 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f28130750 0x7f7f28130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/3844177235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f2809d6f0 0x7f7f2809db40 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/3844177235 >> 172.21.15.32:0/3844177235 conn(0x7f7f2809f470 msgr2=0x7f7f280a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/3844177235 shutdown_connections 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.336+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/3844177235 wait complete. 2023-09-30T22:29:17.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.337+0000 7f7f2e8e5700 1 Processor -- start 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.337+0000 7f7f2e8e5700 1 -- start start 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.337+0000 7f7f2e8e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f2809d6f0 0x7f7f28127bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.337+0000 7f7f2e8e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 0x7f7f28128100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f2e8e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f28130750 0x7f7f28128640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f2e8e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2812a2f0 con 0x7f7f28130750 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f2e8e5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2812b490 con 0x7f7f2809d6f0 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f2e8e5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2812baa0 con 0x7f7f2809e080 2023-09-30T22:29:17.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 0x7f7f28128100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f2ce82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f28130750 0x7f7f28128640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 0x7f7f28128100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50524/0 (socket says 172.21.15.32:50524) 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 -- 172.21.15.32:0/4064987112 learned_addr learned my addr 172.21.15.32:0/4064987112 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 -- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f2809d6f0 msgr2=0x7f7f28127bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f2809d6f0 0x7f7f28127bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 -- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f28130750 msgr2=0x7f7f28128640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f28130750 0x7f7f28128640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 -- 172.21.15.32:0/4064987112 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180094a0 con 0x7f7f2809e080 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f2ce82700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f28130750 0x7f7f28128640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f277fe700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 0x7f7f28128100 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c000c00 tx=0x7f7f1c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c004550 con 0x7f7f2809e080 2023-09-30T22:29:17.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.338+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c01e070 con 0x7f7f2809e080 2023-09-30T22:29:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.339+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0038e0 con 0x7f7f2809e080 2023-09-30T22:29:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.339+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c0094a0 con 0x7f7f2809e080 2023-09-30T22:29:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.339+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2812a690 con 0x7f7f2809e080 2023-09-30T22:29:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.339+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f1c003a40 con 0x7f7f2809e080 2023-09-30T22:29:17.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.340+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7f1c020070 con 0x7f7f2809e080 2023-09-30T22:29:17.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.340+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f280747f0 con 0x7f7f2809e080 2023-09-30T22:29:17.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.345+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f1c01a420 con 0x7f7f2809e080 2023-09-30T22:29:17.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:17.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.480+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff470136320 con 0x7ff4701338a0 2023-09-30T22:29:17.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.481+0000 7ff466ffd700 1 -- 172.21.15.32:0/672180052 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff46c017020 con 0x7ff4701338a0 2023-09-30T22:29:17.482 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 msgr2=0x7ff4701c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 0x7ff4701c85e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c520 tx=0x7ff46c00c830 comp rx=0 tx=0).stop 2023-09-30T22:29:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 shutdown_connections 2023-09-30T22:29:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff470139030 0x7ff47011ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4701338a0 0x7ff4701c85e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 --2- 172.21.15.32:0/672180052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff470136710 0x7ff4701c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.483+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 >> 172.21.15.32:0/672180052 conn(0x7ff4700a2aa0 msgr2=0x7ff470137170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.485+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 shutdown_connections 2023-09-30T22:29:17.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.485+0000 7ff47857a700 1 -- 172.21.15.32:0/672180052 wait complete. 2023-09-30T22:29:17.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.494+0000 7f5c129f4700 1 Processor -- start 2023-09-30T22:29:17.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.494+0000 7f5c129f4700 1 -- start start 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c129f4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c129f4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c0c09f650 0x7f5c0c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c129f4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c070940 con 0x7f5c0c09f240 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c129f4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c06fd60 con 0x7f5c0c09f650 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c129f4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c06fa80 con 0x7f5c0c09e930 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c03fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c0c09f240 0x7f5c0c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59870/0 (socket says 172.21.15.32:59870) 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c03fff700 1 -- 172.21.15.32:0/1373635973 learned_addr learned my addr 172.21.15.32:0/1373635973 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:17.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0bfff700 1 --2- 172.21.15.32:0/1373635973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0b7fe700 1 -- 172.21.15.32:0/1373635973 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530815216 0 0) 0x7f5c0c070940 con 0x7f5c0c09f240 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0b7fe700 1 -- 172.21.15.32:0/1373635973 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf80035b0 con 0x7f5c0c09f240 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0bfff700 1 -- 172.21.15.32:0/1373635973 >> v2:172.21.15.32:3300/0 conn(0x7f5c0c09f650 msgr2=0x7f5c0c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0bfff700 1 --2- 172.21.15.32:0/1373635973 >> v2:172.21.15.32:3300/0 conn(0x7f5c0c09f650 0x7f5c0c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0bfff700 1 -- 172.21.15.32:0/1373635973 >> v1:172.21.15.32:6789/0 conn(0x7f5c0c09f240 legacy=0x7f5c0c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.495+0000 7f5c0bfff700 1 -- 172.21.15.32:0/1373635973 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c09fb60 con 0x7f5c0c09e930 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.496+0000 7f5c0bfff700 1 --2- 172.21.15.32:0/1373635973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c09ed00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009840 tx=0x7f5bfc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bea438e724bde2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.496+0000 7f5c0b7fe700 1 -- 172.21.15.32:0/1373635973 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc004230 con 0x7f5c0c09e930 2023-09-30T22:29:17.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.496+0000 7f5c0b7fe700 1 -- 172.21.15.32:0/1373635973 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc004390 con 0x7f5c0c09e930 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.496+0000 7f5c0b7fe700 1 -- 172.21.15.32:0/1373635973 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0036a0 con 0x7f5c0c09e930 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 -- 172.21.15.32:0/1373635973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 msgr2=0x7f5c0c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1373635973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c09ed00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009840 tx=0x7f5bfc009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 -- 172.21.15.32:0/1373635973 shutdown_connections 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1373635973 >> v2:172.21.15.32:3300/0 conn(0x7f5c0c09f650 0x7f5c0c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1373635973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c09ed00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 -- 172.21.15.32:0/1373635973 >> 172.21.15.32:0/1373635973 conn(0x7f5c0c0a4210 msgr2=0x7f5c0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 -- 172.21.15.32:0/1373635973 shutdown_connections 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.497+0000 7f5c129f4700 1 -- 172.21.15.32:0/1373635973 wait complete. 2023-09-30T22:29:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 Processor -- start 2023-09-30T22:29:17.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 -- start start 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c120350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0c09f240 0x7f5c0c120890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c0c09f650 0x7f5c0c120dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c122710 con 0x7f5c0c09f240 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c123de0 con 0x7f5c0c09f650 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.499+0000 7f5c129f4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0a2f40 con 0x7f5c0c09e930 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c0bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c120350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c0bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c120350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50534/0 (socket says 172.21.15.32:50534) 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c0bfff700 1 -- 172.21.15.32:0/1338468864 learned_addr learned my addr 172.21.15.32:0/1338468864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0c09f240 0x7f5c0c120890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c10f91700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c0c09f650 0x7f5c0c120dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 -- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 msgr2=0x7f5c0c120350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c120350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 -- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c0c09f650 msgr2=0x7f5c0c120dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c0c09f650 0x7f5c0c120dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 -- 172.21.15.32:0/1338468864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc0094a0 con 0x7f5c0c09f240 2023-09-30T22:29:17.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c10f91700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c0c09f650 0x7f5c0c120dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.500+0000 7f5c03fff700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0c09f240 0x7f5c0c120890 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06e6c0 tx=0x7f5bf4001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.501+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf400c070 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.501+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf4004ca0 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.501+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c0a0190 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.501+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4007310 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.501+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c122cf0 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.502+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5bf4007470 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.502+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c074ea0 con 0x7f5c0c09f240 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.502+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5bf403f0b0 con 0x7f5c0c09f240 2023-09-30T22:29:17.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:17.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.511+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5bf4020b20 con 0x7f5c0c09f240 2023-09-30T22:29:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.569+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f208006b9e0 con 0x7f208009de50 2023-09-30T22:29:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.570+0000 7f2075ffb700 1 -- 172.21.15.32:0/2940116897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f207c01a450 con 0x7f208009de50 2023-09-30T22:29:17.571 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 msgr2=0x7f208011e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 0x7f208011e990 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f207c000c00 tx=0x7f207c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 shutdown_connections 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20801338a0 0x7f208011f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f208009de50 0x7f208011e990 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 --2- 172.21.15.32:0/2940116897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f208009e760 0x7f208011eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 >> 172.21.15.32:0/2940116897 conn(0x7f20800a2aa0 msgr2=0x7f208009ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 shutdown_connections 2023-09-30T22:29:17.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.572+0000 7f2087a0d700 1 -- 172.21.15.32:0/2940116897 wait complete. 2023-09-30T22:29:17.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.610+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f2812ff20 con 0x7f7f2809e080 2023-09-30T22:29:17.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.610+0000 7f7f24ff9700 1 -- 172.21.15.32:0/4064987112 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f1c015030 con 0x7f7f2809e080 2023-09-30T22:29:17.611 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 msgr2=0x7f7f28128100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 0x7f7f28128100 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c000c00 tx=0x7f7f1c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 shutdown_connections 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f28130750 0x7f7f28128640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7f2809d6f0 0x7f7f28127bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 --2- 172.21.15.32:0/4064987112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7f2809e080 0x7f7f28128100 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.612+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 >> 172.21.15.32:0/4064987112 conn(0x7f7f2809f470 msgr2=0x7f7f280992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.613+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 shutdown_connections 2023-09-30T22:29:17.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.613+0000 7f7f2e8e5700 1 -- 172.21.15.32:0/4064987112 wait complete. 2023-09-30T22:29:17.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:17.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.719+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5c0c06b250 con 0x7f5c0c09f240 2023-09-30T22:29:17.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.719+0000 7f5c097fa700 1 -- 172.21.15.32:0/1338468864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bf4017e80 con 0x7f5c0c09f240 2023-09-30T22:29:17.721 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:17.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.721+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0c09f240 msgr2=0x7f5c0c120890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.721+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0c09f240 0x7f5c0c120890 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c06e6c0 tx=0x7f5bf4001b80 comp rx=0 tx=0).stop 2023-09-30T22:29:17.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 shutdown_connections 2023-09-30T22:29:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c0c09f650 0x7f5c0c120dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5c0c09e930 0x7f5c0c120350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 --2- 172.21.15.32:0/1338468864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c0c09f240 0x7f5c0c120890 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 >> 172.21.15.32:0/1338468864 conn(0x7f5c0c0a4210 msgr2=0x7f5c0c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 shutdown_connections 2023-09-30T22:29:17.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:17.722+0000 7f5c129f4700 1 -- 172.21.15.32:0/1338468864 wait complete. 2023-09-30T22:29:17.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:18.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:18.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:18.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:18.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 Processor -- start 2023-09-30T22:29:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 -- start start 2023-09-30T22:29:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e34130540 0x7f1e34130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 0x7f1e3406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e3406fb20 0x7f1e3406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3409a390 con 0x7f1e34130540 2023-09-30T22:29:18.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340716a0 con 0x7f1e34130e50 2023-09-30T22:29:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.655+0000 7f1e3afeb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34071290 con 0x7f1e3406fb20 2023-09-30T22:29:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 0x7f1e3406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e38d87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e34130540 0x7f1e34130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 0x7f1e3406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50394/0 (socket says 172.21.15.32:50394) 2023-09-30T22:29:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 -- 172.21.15.32:0/3548429782 learned_addr learned my addr 172.21.15.32:0/3548429782 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e39588700 1 --2- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e3406fb20 0x7f1e3406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 -- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e3406fb20 msgr2=0x7f1e3406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 --2- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e3406fb20 0x7f1e3406ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 -- 172.21.15.32:0/3548429782 >> v2:172.21.15.32:3300/0 conn(0x7f1e34130540 msgr2=0x7f1e34130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 --2- 172.21.15.32:0/3548429782 >> v2:172.21.15.32:3300/0 conn(0x7f1e34130540 0x7f1e34130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 -- 172.21.15.32:0/3548429782 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34130340 con 0x7f1e34130e50 2023-09-30T22:29:18.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e38d87700 1 --2- 172.21.15.32:0/3548429782 >> v2:172.21.15.32:3300/0 conn(0x7f1e34130540 0x7f1e34130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e33fff700 1 --2- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 0x7f1e3406f5e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1e200098f0 tx=0x7f1e20009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef94081300efd660 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e337fe700 1 -- 172.21.15.32:0/3548429782 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000f8a0 con 0x7f1e34130e50 2023-09-30T22:29:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.656+0000 7f1e337fe700 1 -- 172.21.15.32:0/3548429782 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2000fa00 con 0x7f1e34130e50 2023-09-30T22:29:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.657+0000 7f1e337fe700 1 -- 172.21.15.32:0/3548429782 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000fd90 con 0x7f1e34130e50 2023-09-30T22:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.657+0000 7f1e3afeb700 1 -- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 msgr2=0x7f1e3406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.657+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 0x7f1e3406f5e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1e200098f0 tx=0x7f1e20009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 -- 172.21.15.32:0/3548429782 shutdown_connections 2023-09-30T22:29:18.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e3406fb20 0x7f1e3406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/3548429782 >> v2:172.21.15.32:3300/0 conn(0x7f1e34130540 0x7f1e34130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/3548429782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e34130e50 0x7f1e3406f5e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 -- 172.21.15.32:0/3548429782 >> 172.21.15.32:0/3548429782 conn(0x7f1e340a2aa0 msgr2=0x7f1e340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 -- 172.21.15.32:0/3548429782 shutdown_connections 2023-09-30T22:29:18.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.658+0000 7f1e3afeb700 1 -- 172.21.15.32:0/3548429782 wait complete. 2023-09-30T22:29:18.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.659+0000 7f1e3afeb700 1 Processor -- start 2023-09-30T22:29:18.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.659+0000 7f1e3afeb700 1 -- start start 2023-09-30T22:29:18.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e3afeb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3406fb20 0x7f1e3411dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e3afeb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 0x7f1e3411e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e3afeb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e34130e50 0x7f1e3411e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e3afeb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3411f760 con 0x7f1e34130540 2023-09-30T22:29:18.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e3afeb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34121400 con 0x7f1e3406fb20 2023-09-30T22:29:18.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e3afeb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34121670 con 0x7f1e34130e50 2023-09-30T22:29:18.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 0x7f1e3411e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e33fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 0x7f1e3411e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46744/0 (socket says 172.21.15.32:46744) 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e33fff700 1 -- 172.21.15.32:0/248458087 learned_addr learned my addr 172.21.15.32:0/248458087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.660+0000 7f1e39588700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e34130e50 0x7f1e3411e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e33fff700 1 -- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e34130e50 msgr2=0x7f1e3411e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e33fff700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e34130e50 0x7f1e3411e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e33fff700 1 -- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3406fb20 msgr2=0x7f1e3411dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e33fff700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3406fb20 0x7f1e3411dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e33fff700 1 -- 172.21.15.32:0/248458087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280094a0 con 0x7f1e34130540 2023-09-30T22:29:18.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e33fff700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 0x7f1e3411e100 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009c40 tx=0x7f1e20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200038a0 con 0x7f1e34130540 2023-09-30T22:29:18.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e20009550 con 0x7f1e34130540 2023-09-30T22:29:18.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e340a0950 con 0x7f1e34130540 2023-09-30T22:29:18.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20003a00 con 0x7f1e34130540 2023-09-30T22:29:18.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.661+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20003d90 con 0x7f1e34130540 2023-09-30T22:29:18.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.662+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e20019400 con 0x7f1e34130540 2023-09-30T22:29:18.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.662+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e340718e0 con 0x7f1e34130540 2023-09-30T22:29:18.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.663+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e20040c20 con 0x7f1e34130540 2023-09-30T22:29:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.672+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e20021750 con 0x7f1e34130540 2023-09-30T22:29:18.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 Processor -- start 2023-09-30T22:29:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 -- start start 2023-09-30T22:29:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 0x7f9b500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b500a1340 0x7f9b50133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50133fa0 0x7f9b50134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5009edd0 con 0x7f9b500a1340 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5009e1f0 con 0x7f9b50133fa0 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b5625a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5009dc50 con 0x7f9b500a0a30 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b4f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 0x7f9b500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b4f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 0x7f9b500a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50564/0 (socket says 172.21.15.32:50564) 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b4f7fe700 1 -- 172.21.15.32:0/3301570374 learned_addr learned my addr 172.21.15.32:0/3301570374 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.734+0000 7f9b4ffff700 1 --2- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50133fa0 0x7f9b50134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4effd700 1 --2- 172.21.15.32:0/3301570374 >> v2:172.21.15.32:3300/0 conn(0x7f9b500a1340 0x7f9b50133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4f7fe700 1 -- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50133fa0 msgr2=0x7f9b50134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50133fa0 0x7f9b50134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4f7fe700 1 -- 172.21.15.32:0/3301570374 >> v2:172.21.15.32:3300/0 conn(0x7f9b500a1340 msgr2=0x7f9b50133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/3301570374 >> v2:172.21.15.32:3300/0 conn(0x7f9b500a1340 0x7f9b50133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4f7fe700 1 -- 172.21.15.32:0/3301570374 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b500a1fe0 con 0x7f9b500a0a30 2023-09-30T22:29:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 0x7f9b500a0e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009920 tx=0x7f9b40009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aaba3390a24bd14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4e7fc700 1 -- 172.21.15.32:0/3301570374 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40011830 con 0x7f9b500a0a30 2023-09-30T22:29:18.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4e7fc700 1 -- 172.21.15.32:0/3301570374 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40011990 con 0x7f9b500a0a30 2023-09-30T22:29:18.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.735+0000 7f9b4e7fc700 1 -- 172.21.15.32:0/3301570374 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40011d20 con 0x7f9b500a0a30 2023-09-30T22:29:18.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.736+0000 7f9b5625a700 1 -- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 msgr2=0x7f9b500a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.736+0000 7f9b5625a700 1 --2- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 0x7f9b500a0e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009920 tx=0x7f9b40009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.736+0000 7f9b5625a700 1 -- 172.21.15.32:0/3301570374 shutdown_connections 2023-09-30T22:29:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.736+0000 7f9b5625a700 1 --2- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50133fa0 0x7f9b50134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.737+0000 7f9b5625a700 1 --2- 172.21.15.32:0/3301570374 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b500a0a30 0x7f9b500a0e00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.737+0000 7f9b5625a700 1 --2- 172.21.15.32:0/3301570374 >> v2:172.21.15.32:3300/0 conn(0x7f9b500a1340 0x7f9b50133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.737+0000 7f9b5625a700 1 -- 172.21.15.32:0/3301570374 >> 172.21.15.32:0/3301570374 conn(0x7f9b500a6550 msgr2=0x7f9b500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.737+0000 7f9b5625a700 1 -- 172.21.15.32:0/3301570374 shutdown_connections 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.737+0000 7f9b5625a700 1 -- 172.21.15.32:0/3301570374 wait complete. 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.738+0000 7f9b5625a700 1 Processor -- start 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.738+0000 7f9b5625a700 1 -- start start 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b5625a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500a0a30 0x7f9b5011e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b5625a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 0x7f9b5011e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b5625a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50133fa0 0x7f9b5011ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b5625a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50120280 con 0x7f9b500a0a30 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b5625a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50120f40 con 0x7f9b500a1340 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b5625a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500a35a0 con 0x7f9b50133fa0 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b4effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 0x7f9b5011e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b4effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 0x7f9b5011e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50416/0 (socket says 172.21.15.32:50416) 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b4effd700 1 -- 172.21.15.32:0/4094411862 learned_addr learned my addr 172.21.15.32:0/4094411862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b4ffff700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50133fa0 0x7f9b5011ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.739+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500a0a30 0x7f9b5011e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4effd700 1 -- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50133fa0 msgr2=0x7f9b5011ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4effd700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50133fa0 0x7f9b5011ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4effd700 1 -- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500a0a30 msgr2=0x7f9b5011e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4effd700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500a0a30 0x7f9b5011e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4effd700 1 -- 172.21.15.32:0/4094411862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40009550 con 0x7f9b500a1340 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500a0a30 0x7f9b5011e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b4effd700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 0x7f9b5011e590 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9b0 tx=0x7f9b4400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b44009710 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4400ee50 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b50121300 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4400ca50 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.740+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5011fc60 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.741+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b4400cbb0 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.741+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b4403fa60 con 0x7f9b500a1340 2023-09-30T22:29:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.741+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5009b1e0 con 0x7f9b500a1340 2023-09-30T22:29:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.747+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b44017080 con 0x7f9b500a1340 2023-09-30T22:29:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.777+0000 7f776ccab700 1 Processor -- start 2023-09-30T22:29:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.777+0000 7f776ccab700 1 -- start start 2023-09-30T22:29:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776ccab700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7768136710 0x7f7768138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776ccab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776ccab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776806ef30 con 0x7f77681338a0 2023-09-30T22:29:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776ccab700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776806fef0 con 0x7f7768136710 2023-09-30T22:29:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776ccab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77681334b0 con 0x7f7768139030 2023-09-30T22:29:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f7765d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7768136710 0x7f7768138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f77681338a0 0x7f7768133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59878/0 (socket says 172.21.15.32:59878) 2023-09-30T22:29:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776659c700 1 -- 172.21.15.32:0/1812117479 learned_addr learned my addr 172.21.15.32:0/1812117479 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:18.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f7766d9d700 1 --2- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f7765d9b700 1 -- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 msgr2=0x7f776813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f7765d9b700 1 --2- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f7765d9b700 1 -- 172.21.15.32:0/1812117479 >> v1:172.21.15.32:6789/0 conn(0x7f77681338a0 legacy=0x7f7768133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f7765d9b700 1 -- 172.21.15.32:0/1812117479 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776806fc30 con 0x7f7768136710 2023-09-30T22:29:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.778+0000 7f776559a700 1 -- 172.21.15.32:0/1812117479 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504341822 0 0) 0x7f776806ef30 con 0x7f77681338a0 2023-09-30T22:29:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.779+0000 7f7765d9b700 1 --2- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768136710 0x7f7768138af0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f775c009840 tx=0x7f775c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09790849aed0f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.779+0000 7f776559a700 1 -- 172.21.15.32:0/1812117479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c011830 con 0x7f7768136710 2023-09-30T22:29:18.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.779+0000 7f776559a700 1 -- 172.21.15.32:0/1812117479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c011990 con 0x7f7768136710 2023-09-30T22:29:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.779+0000 7f776559a700 1 -- 172.21.15.32:0/1812117479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c011d20 con 0x7f7768136710 2023-09-30T22:29:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.779+0000 7f776ccab700 1 -- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768136710 msgr2=0x7f7768138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.779+0000 7f776ccab700 1 --2- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768136710 0x7f7768138af0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f775c009840 tx=0x7f775c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.780+0000 7f776ccab700 1 -- 172.21.15.32:0/1812117479 shutdown_connections 2023-09-30T22:29:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.780+0000 7f776ccab700 1 --2- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.780+0000 7f776ccab700 1 --2- 172.21.15.32:0/1812117479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7768136710 0x7f7768138af0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.780+0000 7f776ccab700 1 -- 172.21.15.32:0/1812117479 >> 172.21.15.32:0/1812117479 conn(0x7f77680a2aa0 msgr2=0x7f77680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.780+0000 7f776ccab700 1 -- 172.21.15.32:0/1812117479 shutdown_connections 2023-09-30T22:29:18.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.780+0000 7f776ccab700 1 -- 172.21.15.32:0/1812117479 wait complete. 2023-09-30T22:29:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.781+0000 7f776ccab700 1 Processor -- start 2023-09-30T22:29:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 -- start start 2023-09-30T22:29:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 0x7f77681c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7768136710 0x7f77681c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776811ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776811e780 con 0x7f77681338a0 2023-09-30T22:29:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776806f6d0 con 0x7f7768136710 2023-09-30T22:29:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776ccab700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f776811e960 con 0x7f7768139030 2023-09-30T22:29:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.782+0000 7f776659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 0x7f77681c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776811ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f776659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 0x7f77681c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46774/0 (socket says 172.21.15.32:46774) 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f776659c700 1 -- 172.21.15.32:0/2665087113 learned_addr learned my addr 172.21.15.32:0/2665087113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 -- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7768136710 msgr2=0x7f77681c8ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7768136710 0x7f77681c8ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 -- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 msgr2=0x7f77681c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 0x7f77681c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 -- 172.21.15.32:0/2665087113 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0094a0 con 0x7f7768139030 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f776659c700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 0x7f77681c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7766d9d700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776811ae40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f776000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776000cd40 con 0x7f7768139030 2023-09-30T22:29:18.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776000f040 con 0x7f7768139030 2023-09-30T22:29:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77681be110 con 0x7f7768139030 2023-09-30T22:29:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.783+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600146c0 con 0x7f7768139030 2023-09-30T22:29:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.784+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776811d210 con 0x7f7768139030 2023-09-30T22:29:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.784+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7760014820 con 0x7f7768139030 2023-09-30T22:29:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.785+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7760040270 con 0x7f7768139030 2023-09-30T22:29:18.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.787+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776806ea40 con 0x7f7768139030 2023-09-30T22:29:18.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.792+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f776001d400 con 0x7f7768139030 2023-09-30T22:29:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.886+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e34074ea0 con 0x7f1e34130540 2023-09-30T22:29:18.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.887+0000 7f1e317fa700 1 -- 172.21.15.32:0/248458087 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e20021390 con 0x7f1e34130540 2023-09-30T22:29:18.888 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 msgr2=0x7f1e3411e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 0x7f1e3411e100 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009c40 tx=0x7f1e20004750 comp rx=0 tx=0).stop 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 shutdown_connections 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e34130e50 0x7f1e3411e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3406fb20 0x7f1e3411dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 --2- 172.21.15.32:0/248458087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e34130540 0x7f1e3411e100 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 >> 172.21.15.32:0/248458087 conn(0x7f1e340a2aa0 msgr2=0x7f1e3412bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 shutdown_connections 2023-09-30T22:29:18.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.889+0000 7f1e3afeb700 1 -- 172.21.15.32:0/248458087 wait complete. 2023-09-30T22:29:18.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.910+0000 7ff274b5c700 1 Processor -- start 2023-09-30T22:29:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.910+0000 7ff274b5c700 1 -- start start 2023-09-30T22:29:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.910+0000 7ff274b5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 0x7ff270136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff274b5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff270136fe0 0x7ff2701393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff274b5c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff270070940 con 0x7ff27009e930 2023-09-30T22:29:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff274b5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27006fd60 con 0x7ff270136fe0 2023-09-30T22:29:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff274b5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff27006fa80 con 0x7ff27009f410 2023-09-30T22:29:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 0x7ff270136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 0x7ff270136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50606/0 (socket says 172.21.15.32:50606) 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 -- 172.21.15.32:0/2395493316 learned_addr learned my addr 172.21.15.32:0/2395493316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26ed9d700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff270136fe0 0x7ff2701393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 -- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff270136fe0 msgr2=0x7ff2701393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff270136fe0 0x7ff2701393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 -- 172.21.15.32:0/2395493316 >> v1:172.21.15.32:6789/0 conn(0x7ff27009e930 legacy=0x7ff27009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 -- 172.21.15.32:0/2395493316 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27009fb60 con 0x7ff27009f410 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26d59a700 1 -- 172.21.15.32:0/2395493316 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978015680 0 0) 0x7ff270070940 con 0x7ff27009e930 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26ed9d700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff270136fe0 0x7ff2701393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.911+0000 7ff26dd9b700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 0x7ff270136aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff264009920 tx=0x7ff264009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a48446693fbaa2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff26d59a700 1 -- 172.21.15.32:0/2395493316 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264011830 con 0x7ff27009f410 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff26d59a700 1 -- 172.21.15.32:0/2395493316 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264011990 con 0x7ff27009f410 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff26d59a700 1 -- 172.21.15.32:0/2395493316 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264011d20 con 0x7ff27009f410 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 -- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 msgr2=0x7ff270136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 0x7ff270136aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff264009920 tx=0x7ff264009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 -- 172.21.15.32:0/2395493316 shutdown_connections 2023-09-30T22:29:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff270136fe0 0x7ff2701393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 --2- 172.21.15.32:0/2395493316 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff27009f410 0x7ff270136aa0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 -- 172.21.15.32:0/2395493316 >> 172.21.15.32:0/2395493316 conn(0x7ff2700a4210 msgr2=0x7ff2700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 -- 172.21.15.32:0/2395493316 shutdown_connections 2023-09-30T22:29:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.912+0000 7ff274b5c700 1 -- 172.21.15.32:0/2395493316 wait complete. 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 Processor -- start 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 -- start start 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff27009e930 0x7ff27011b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 0x7ff27011b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff270136fe0 0x7ff27011be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27011e9f0 con 0x7ff27009e930 2023-09-30T22:29:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27011c7f0 con 0x7ff27009f410 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.914+0000 7ff274b5c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff27006d880 con 0x7ff270136fe0 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 0x7ff27011b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 0x7ff27011b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50436/0 (socket says 172.21.15.32:50436) 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 -- 172.21.15.32:0/636031672 learned_addr learned my addr 172.21.15.32:0/636031672 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26e59c700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff27009e930 0x7ff27011b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26ed9d700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff270136fe0 0x7ff27011be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 -- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff270136fe0 msgr2=0x7ff27011be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff270136fe0 0x7ff27011be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 -- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff27009e930 msgr2=0x7ff27011b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff27009e930 0x7ff27011b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 -- 172.21.15.32:0/636031672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26800f050 con 0x7ff27009f410 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff26dd9b700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 0x7ff27011b8d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff264000c00 tx=0x7ff264004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.915+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264004450 con 0x7ff27009f410 2023-09-30T22:29:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.916+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2640045b0 con 0x7ff27009f410 2023-09-30T22:29:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.916+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff264009550 con 0x7ff27009f410 2023-09-30T22:29:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.916+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640038c0 con 0x7ff27009f410 2023-09-30T22:29:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.916+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27011ebd0 con 0x7ff27009f410 2023-09-30T22:29:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.916+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff264003a20 con 0x7ff27009f410 2023-09-30T22:29:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.917+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff264021030 con 0x7ff27009f410 2023-09-30T22:29:18.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.917+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27011e820 con 0x7ff27009f410 2023-09-30T22:29:18.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.922+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff26401a430 con 0x7ff27009f410 2023-09-30T22:29:18.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.961+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9b50074ea0 con 0x7f9b500a1340 2023-09-30T22:29:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.961+0000 7f9b37fff700 1 -- 172.21.15.32:0/4094411862 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b44021020 con 0x7f9b500a1340 2023-09-30T22:29:19.176 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 msgr2=0x7f9b5011e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 0x7f9b5011e590 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9b0 tx=0x7f9b4400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 shutdown_connections 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50133fa0 0x7f9b5011ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500a0a30 0x7f9b5011e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 --2- 172.21.15.32:0/4094411862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b500a1340 0x7f9b5011e590 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 >> 172.21.15.32:0/4094411862 conn(0x7f9b500a6550 msgr2=0x7f9b50133060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 shutdown_connections 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:18.964+0000 7f9b5625a700 1 -- 172.21.15.32:0/4094411862 wait complete. 2023-09-30T22:29:19.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.029+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7768074ea0 con 0x7f7768139030 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.029+0000 7f7756ffd700 1 -- 172.21.15.32:0/2665087113 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f776002c090 con 0x7f7768139030 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.031+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 msgr2=0x7f776811ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.031+0000 7f776ccab700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776811ae40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f776000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 shutdown_connections 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7768139030 0x7f776811ae40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77681338a0 0x7f77681c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 --2- 172.21.15.32:0/2665087113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7768136710 0x7f77681c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 >> 172.21.15.32:0/2665087113 conn(0x7f77680a2aa0 msgr2=0x7f77680a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 shutdown_connections 2023-09-30T22:29:19.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.032+0000 7f776ccab700 1 -- 172.21.15.32:0/2665087113 wait complete. 2023-09-30T22:29:19.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.130+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff27011e1f0 con 0x7ff27009f410 2023-09-30T22:29:19.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.130+0000 7ff25effd700 1 -- 172.21.15.32:0/636031672 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff26402b030 con 0x7ff27009f410 2023-09-30T22:29:19.339 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 msgr2=0x7ff27011b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 0x7ff27011b8d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff264000c00 tx=0x7ff264004750 comp rx=0 tx=0).stop 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 shutdown_connections 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff270136fe0 0x7ff27011be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff27009e930 0x7ff27011b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 --2- 172.21.15.32:0/636031672 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff27009f410 0x7ff27011b8d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 >> 172.21.15.32:0/636031672 conn(0x7ff2700a4210 msgr2=0x7ff2700a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 shutdown_connections 2023-09-30T22:29:19.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:19.133+0000 7ff274b5c700 1 -- 172.21.15.32:0/636031672 wait complete. 2023-09-30T22:29:19.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:19.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.052+0000 7f8314c67700 1 Processor -- start 2023-09-30T22:29:20.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.052+0000 7f8314c67700 1 -- start start 2023-09-30T22:29:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f8314c67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8310136710 0x7f8310138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f8314c67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8310139030 0x7f831013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f8314c67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831006ef30 con 0x7f83101338a0 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f8314c67700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831006fef0 con 0x7f8310139030 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f8314c67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83101334b0 con 0x7f8310136710 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8310139030 0x7f831013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8310136710 0x7f8310138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83101338a0 0x7f8310133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59888/0 (socket says 172.21.15.32:59888) 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830e59c700 1 -- 172.21.15.32:0/551441014 learned_addr learned my addr 172.21.15.32:0/551441014 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830dd9b700 1 -- 172.21.15.32:0/551441014 >> v2:172.21.15.32:3300/0 conn(0x7f8310139030 msgr2=0x7f831013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830dd9b700 1 --2- 172.21.15.32:0/551441014 >> v2:172.21.15.32:3300/0 conn(0x7f8310139030 0x7f831013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830dd9b700 1 -- 172.21.15.32:0/551441014 >> v1:172.21.15.32:6789/0 conn(0x7f83101338a0 legacy=0x7f8310133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830dd9b700 1 -- 172.21.15.32:0/551441014 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831006fc30 con 0x7f8310136710 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830d59a700 1 -- 172.21.15.32:0/551441014 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453474140 0 0) 0x7f831006ef30 con 0x7f83101338a0 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.053+0000 7f830ed9d700 1 --2- 172.21.15.32:0/551441014 >> v2:172.21.15.32:3300/0 conn(0x7f8310139030 0x7f831013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f830dd9b700 1 --2- 172.21.15.32:0/551441014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8310136710 0x7f8310138af0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009840 tx=0x7f82f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f294fdd85f73da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f830d59a700 1 -- 172.21.15.32:0/551441014 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8011830 con 0x7f8310136710 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f830d59a700 1 -- 172.21.15.32:0/551441014 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f8011990 con 0x7f8310136710 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f830d59a700 1 -- 172.21.15.32:0/551441014 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8011d20 con 0x7f8310136710 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f8314c67700 1 -- 172.21.15.32:0/551441014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8310136710 msgr2=0x7f8310138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f8314c67700 1 --2- 172.21.15.32:0/551441014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8310136710 0x7f8310138af0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009840 tx=0x7f82f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f8314c67700 1 -- 172.21.15.32:0/551441014 shutdown_connections 2023-09-30T22:29:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f8314c67700 1 --2- 172.21.15.32:0/551441014 >> v2:172.21.15.32:3300/0 conn(0x7f8310139030 0x7f831013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f8314c67700 1 --2- 172.21.15.32:0/551441014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8310136710 0x7f8310138af0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.054+0000 7f8314c67700 1 -- 172.21.15.32:0/551441014 >> 172.21.15.32:0/551441014 conn(0x7f83100a2aa0 msgr2=0x7f83100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.055+0000 7f8314c67700 1 -- 172.21.15.32:0/551441014 shutdown_connections 2023-09-30T22:29:20.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.055+0000 7f8314c67700 1 -- 172.21.15.32:0/551441014 wait complete. 2023-09-30T22:29:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 Processor -- start 2023-09-30T22:29:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 -- start start 2023-09-30T22:29:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 0x7f831011ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 0x7f831011b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8310139030 0x7f831011b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310130590 con 0x7f8310136710 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831011cea0 con 0x7f83101338a0 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f8314c67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f831011c980 con 0x7f8310139030 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f830e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 0x7f831011ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f830dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 0x7f831011b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f830dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 0x7f831011b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46782/0 (socket says 172.21.15.32:46782) 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f830e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 0x7f831011ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:50456/0 (socket says 172.21.15.32:50456) 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f830dd9b700 1 -- 172.21.15.32:0/2694185593 learned_addr learned my addr 172.21.15.32:0/2694185593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.056+0000 7f830ed9d700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8310139030 0x7f831011b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830dd9b700 1 -- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8310139030 msgr2=0x7f831011b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830dd9b700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8310139030 0x7f831011b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830dd9b700 1 -- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 msgr2=0x7f831011ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830dd9b700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 0x7f831011ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830dd9b700 1 -- 172.21.15.32:0/2694185593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308009a70 con 0x7f8310136710 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830e59c700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 0x7f831011ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f830dd9b700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 0x7f831011b1a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f82f80090b0 tx=0x7f82f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80045d0 con 0x7f8310136710 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f801f070 con 0x7f8310136710 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f80094a0 con 0x7f8310136710 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80038c0 con 0x7f8310136710 2023-09-30T22:29:20.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.057+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831011d060 con 0x7f8310136710 2023-09-30T22:29:20.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.058+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8310074ea0 con 0x7f8310136710 2023-09-30T22:29:20.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.059+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82f8003a20 con 0x7f8310136710 2023-09-30T22:29:20.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.059+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f82f8021070 con 0x7f8310136710 2023-09-30T22:29:20.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.063+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82f8024380 con 0x7f8310136710 2023-09-30T22:29:20.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.269+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f831011ef70 con 0x7f8310136710 2023-09-30T22:29:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.269+0000 7f8306ffd700 1 -- 172.21.15.32:0/2694185593 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82f802b020 con 0x7f8310136710 2023-09-30T22:29:20.270 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 msgr2=0x7f831011b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 0x7f831011b1a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f82f80090b0 tx=0x7f82f8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 shutdown_connections 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8310139030 0x7f831011b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83101338a0 0x7f831011ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 --2- 172.21.15.32:0/2694185593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8310136710 0x7f831011b1a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 >> 172.21.15.32:0/2694185593 conn(0x7f83100a2aa0 msgr2=0x7f83101344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 shutdown_connections 2023-09-30T22:29:20.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.271+0000 7f8314c67700 1 -- 172.21.15.32:0/2694185593 wait complete. 2023-09-30T22:29:20.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.315+0000 7f9a10241700 1 Processor -- start 2023-09-30T22:29:20.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.315+0000 7f9a10241700 1 -- start start 2023-09-30T22:29:20.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a10241700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a080a0a30 0x7f9a080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a10241700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0809d630 0x7f9a080a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a10241700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a080a2a70 con 0x7f9a08133930 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a10241700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080a01d0 con 0x7f9a0809d630 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a10241700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0809e2b0 con 0x7f9a080a0a30 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0dfdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a080a0a30 0x7f9a080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0d7dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a0809d630 0x7f9a080a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0e7de700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9a08133930 0x7f9a08133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59894/0 (socket says 172.21.15.32:59894) 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0e7de700 1 -- 172.21.15.32:0/3192674556 learned_addr learned my addr 172.21.15.32:0/3192674556 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:20.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0d7dc700 1 -- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a080a0a30 msgr2=0x7f9a080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0d7dc700 1 --2- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a080a0a30 0x7f9a080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0d7dc700 1 -- 172.21.15.32:0/3192674556 >> v1:172.21.15.32:6789/0 conn(0x7f9a08133930 legacy=0x7f9a08133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0cfdb700 1 -- 172.21.15.32:0/3192674556 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859057607 0 0) 0x7f9a080a2a70 con 0x7f9a08133930 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0d7dc700 1 -- 172.21.15.32:0/3192674556 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0809dc50 con 0x7f9a0809d630 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0dfdd700 1 --2- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a080a0a30 0x7f9a080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.316+0000 7f9a0d7dc700 1 --2- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0809d630 0x7f9a080a1340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009840 tx=0x7f99f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a78850dd31f400 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a0cfdb700 1 -- 172.21.15.32:0/3192674556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8011830 con 0x7f9a0809d630 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a0cfdb700 1 -- 172.21.15.32:0/3192674556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f8011990 con 0x7f9a0809d630 2023-09-30T22:29:20.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a0cfdb700 1 -- 172.21.15.32:0/3192674556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8011d20 con 0x7f9a0809d630 2023-09-30T22:29:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 -- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0809d630 msgr2=0x7f9a080a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 --2- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0809d630 0x7f9a080a1340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009840 tx=0x7f99f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 -- 172.21.15.32:0/3192674556 shutdown_connections 2023-09-30T22:29:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 --2- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a080a0a30 0x7f9a080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 --2- 172.21.15.32:0/3192674556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a0809d630 0x7f9a080a1340 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 -- 172.21.15.32:0/3192674556 >> 172.21.15.32:0/3192674556 conn(0x7f9a080a3050 msgr2=0x7f9a080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 -- 172.21.15.32:0/3192674556 shutdown_connections 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.317+0000 7f9a10241700 1 -- 172.21.15.32:0/3192674556 wait complete. 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.318+0000 7f9a10241700 1 Processor -- start 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.318+0000 7f9a10241700 1 -- start start 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a10241700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 0x7f9a081c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a10241700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a080a0a30 0x7f9a081c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a10241700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a08133930 0x7f9a081c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a10241700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081210e0 con 0x7f9a080a0a30 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a10241700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081207f0 con 0x7f9a08133930 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a10241700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081209e0 con 0x7f9a0809d630 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 0x7f9a081c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 0x7f9a081c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50628/0 (socket says 172.21.15.32:50628) 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 -- 172.21.15.32:0/3136570599 learned_addr learned my addr 172.21.15.32:0/3136570599 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0e7de700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a08133930 0x7f9a081c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0d7dc700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a080a0a30 0x7f9a081c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 -- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a08133930 msgr2=0x7f9a081c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a08133930 0x7f9a081c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 -- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a080a0a30 msgr2=0x7f9a081c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a080a0a30 0x7f9a081c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.319+0000 7f9a0dfdd700 1 -- 172.21.15.32:0/3136570599 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040094a0 con 0x7f9a0809d630 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.320+0000 7f9a0dfdd700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 0x7f9a081c8190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400ec10 tx=0x7f9a0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.320+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400c850 con 0x7f9a0809d630 2023-09-30T22:29:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.320+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0400c9b0 con 0x7f9a0809d630 2023-09-30T22:29:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.320+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400cd20 con 0x7f9a0809d630 2023-09-30T22:29:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.320+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f80094a0 con 0x7f9a0809d630 2023-09-30T22:29:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.320+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a080a0410 con 0x7f9a0809d630 2023-09-30T22:29:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.321+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a04013470 con 0x7f9a0809d630 2023-09-30T22:29:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.321+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a080a5780 con 0x7f9a0809d630 2023-09-30T22:29:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.321+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9a0403f7f0 con 0x7f9a0809d630 2023-09-30T22:29:20.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.327+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a04013720 con 0x7f9a0809d630 2023-09-30T22:29:20.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:20.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:20.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.477+0000 7f6f178b7700 1 Processor -- start 2023-09-30T22:29:20.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.478+0000 7f6f178b7700 1 -- start start 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f178b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 0x7f6f100a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f178b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f1013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f178b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f100a2a70 con 0x7f6f1009d630 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f178b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100a01d0 con 0x7f6f10138d80 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f178b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1009e2b0 con 0x7f6f100a0a30 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15653700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 0x7f6f100a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15653700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 0x7f6f100a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50072/0 (socket says 172.21.15.32:50072) 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15653700 1 -- 172.21.15.32:0/153584758 learned_addr learned my addr 172.21.15.32:0/153584758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:20.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f14e52700 1 --1- 172.21.15.32:0/153584758 >> v1:172.21.15.32:6789/0 conn(0x7f6f1009d630 0x7f6f100a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54924/0 (socket says 172.21.15.32:54924) 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15e54700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f1013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15e54700 1 -- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 msgr2=0x7f6f100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15e54700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 0x7f6f100a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15e54700 1 -- 172.21.15.32:0/153584758 >> v1:172.21.15.32:6789/0 conn(0x7f6f1009d630 legacy=0x7f6f100a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f07fff700 1 -- 172.21.15.32:0/153584758 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272788019 0 0) 0x7f6f100a2a70 con 0x7f6f1009d630 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15e54700 1 -- 172.21.15.32:0/153584758 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1009dc50 con 0x7f6f10138d80 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15653700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 0x7f6f100a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.479+0000 7f6f15e54700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f1013b160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009dc0 tx=0x7f6f0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f538365aa6243ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f07fff700 1 -- 172.21.15.32:0/153584758 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800e070 con 0x7f6f10138d80 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f07fff700 1 -- 172.21.15.32:0/153584758 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0800bb20 con 0x7f6f10138d80 2023-09-30T22:29:20.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f07fff700 1 -- 172.21.15.32:0/153584758 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08012430 con 0x7f6f10138d80 2023-09-30T22:29:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 -- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 msgr2=0x7f6f1013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f1013b160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009dc0 tx=0x7f6f0800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 -- 172.21.15.32:0/153584758 shutdown_connections 2023-09-30T22:29:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f1013b160 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 --2- 172.21.15.32:0/153584758 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f100a0a30 0x7f6f100a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 -- 172.21.15.32:0/153584758 >> 172.21.15.32:0/153584758 conn(0x7f6f100a3050 msgr2=0x7f6f100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.480+0000 7f6f178b7700 1 -- 172.21.15.32:0/153584758 shutdown_connections 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.481+0000 7f6f178b7700 1 -- 172.21.15.32:0/153584758 wait complete. 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.482+0000 7f6f178b7700 1 Processor -- start 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.482+0000 7f6f178b7700 1 -- start start 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f178b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f1009d630 0x7f6f10133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f178b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 0x7f6f10133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f178b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f10134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f178b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10134bd0 con 0x7f6f100a0a30 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f178b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10136140 con 0x7f6f10138d80 2023-09-30T22:29:20.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f178b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10136e60 con 0x7f6f1009d630 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 0x7f6f10133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 0x7f6f10133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56630/0 (socket says 172.21.15.32:56630) 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 -- 172.21.15.32:0/1512266242 learned_addr learned my addr 172.21.15.32:0/1512266242 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f15e54700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f10134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f15653700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f1009d630 0x7f6f10133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 -- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f1009d630 msgr2=0x7f6f10133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f1009d630 0x7f6f10133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 -- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 msgr2=0x7f6f10134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f10134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.483+0000 7f6f14e52700 1 -- 172.21.15.32:0/1512266242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c009550 con 0x7f6f100a0a30 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.484+0000 7f6f14e52700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 0x7f6f10133c40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c520 tx=0x7f6f0000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.484+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004c80 con 0x7f6f100a0a30 2023-09-30T22:29:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.484+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00004de0 con 0x7f6f100a0a30 2023-09-30T22:29:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.484+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00007310 con 0x7f6f100a0a30 2023-09-30T22:29:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.484+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f08009a70 con 0x7f6f100a0a30 2023-09-30T22:29:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.484+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1013ac00 con 0x7f6f100a0a30 2023-09-30T22:29:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.485+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f00007470 con 0x7f6f100a0a30 2023-09-30T22:29:20.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.485+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f00040d40 con 0x7f6f100a0a30 2023-09-30T22:29:20.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.486+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f100747f0 con 0x7f6f100a0a30 2023-09-30T22:29:20.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.490+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f00044060 con 0x7f6f100a0a30 2023-09-30T22:29:20.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 Processor -- start 2023-09-30T22:29:20.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 -- start start 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f29d80a1510 0x7f29d8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 0x7f29d813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d809edd0 con 0x7f29d80a0a30 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d80a18e0 con 0x7f29d80a1510 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29de31e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d809e1f0 con 0x7f29d8139250 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29dc8bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 0x7f29d813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29dc8bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 0x7f29d813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:36990/0 (socket says 172.21.15.32:36990) 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29dc8bb700 1 -- 172.21.15.32:0/639498682 learned_addr learned my addr 172.21.15.32:0/639498682 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.521+0000 7f29d77fe700 1 --2- 172.21.15.32:0/639498682 >> v2:172.21.15.32:3300/0 conn(0x7f29d80a1510 0x7f29d8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29dc8bb700 1 -- 172.21.15.32:0/639498682 >> v2:172.21.15.32:3300/0 conn(0x7f29d80a1510 msgr2=0x7f29d8138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29dc8bb700 1 --2- 172.21.15.32:0/639498682 >> v2:172.21.15.32:3300/0 conn(0x7f29d80a1510 0x7f29d8138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29dc8bb700 1 -- 172.21.15.32:0/639498682 >> v1:172.21.15.32:6789/0 conn(0x7f29d80a0a30 legacy=0x7f29d80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29dc8bb700 1 -- 172.21.15.32:0/639498682 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d809dc50 con 0x7f29d8139250 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29dc8bb700 1 --2- 172.21.15.32:0/639498682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 0x7f29d813b630 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009e50 tx=0x7f29d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf5329a63c9734e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29d6ffd700 1 -- 172.21.15.32:0/639498682 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000e070 con 0x7f29d8139250 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29d6ffd700 1 -- 172.21.15.32:0/639498682 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d000b910 con 0x7f29d8139250 2023-09-30T22:29:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.522+0000 7f29d6ffd700 1 -- 172.21.15.32:0/639498682 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000bcc0 con 0x7f29d8139250 2023-09-30T22:29:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 -- 172.21.15.32:0/639498682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 msgr2=0x7f29d813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 --2- 172.21.15.32:0/639498682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 0x7f29d813b630 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009e50 tx=0x7f29d000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:20.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 -- 172.21.15.32:0/639498682 shutdown_connections 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 --2- 172.21.15.32:0/639498682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d8139250 0x7f29d813b630 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 --2- 172.21.15.32:0/639498682 >> v2:172.21.15.32:3300/0 conn(0x7f29d80a1510 0x7f29d8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 -- 172.21.15.32:0/639498682 >> 172.21.15.32:0/639498682 conn(0x7f29d80a6550 msgr2=0x7f29d80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 -- 172.21.15.32:0/639498682 shutdown_connections 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.523+0000 7f29de31e700 1 -- 172.21.15.32:0/639498682 wait complete. 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.524+0000 7f29de31e700 1 Processor -- start 2023-09-30T22:29:20.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 -- start start 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29d80a0a30 0x7f29d81228e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 0x7f29d8122e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29d8139250 0x7f29d8123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d8126170 con 0x7f29d8139250 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d8125ae0 con 0x7f29d80a1510 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.525+0000 7f29de31e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29d8138980 con 0x7f29d80a0a30 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 0x7f29d8122e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29d80a0a30 0x7f29d81228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 0x7f29d8122e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37004/0 (socket says 172.21.15.32:37004) 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 -- 172.21.15.32:0/3165449143 learned_addr learned my addr 172.21.15.32:0/3165449143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29dc8bb700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29d8139250 0x7f29d8123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 -- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29d80a0a30 msgr2=0x7f29d81228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29d80a0a30 0x7f29d81228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 -- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29d8139250 msgr2=0x7f29d8123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29d8139250 0x7f29d8123360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 -- 172.21.15.32:0/3165449143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0009b30 con 0x7f29d80a1510 2023-09-30T22:29:20.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29dc8bb700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29d8139250 0x7f29d8123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d77fe700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 0x7f29d8122e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00e9b0 tx=0x7f29cc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc009710 con 0x7f29d80a1510 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc00ee50 con 0x7f29d80a1510 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d8124460 con 0x7f29d80a1510 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.526+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00ca50 con 0x7f29d80a1510 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.527+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d8124c10 con 0x7f29d80a1510 2023-09-30T22:29:20.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.528+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29cc00cbb0 con 0x7f29d80a1510 2023-09-30T22:29:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.528+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d8074ea0 con 0x7f29d80a1510 2023-09-30T22:29:20.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.528+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f29cc03faa0 con 0x7f29d80a1510 2023-09-30T22:29:20.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.533+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29cc01b3b0 con 0x7f29d80a1510 2023-09-30T22:29:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.539+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9a08122040 con 0x7f9a0809d630 2023-09-30T22:29:20.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.540+0000 7f99fe7fc700 1 -- 172.21.15.32:0/3136570599 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a0401bb60 con 0x7f9a0809d630 2023-09-30T22:29:20.541 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 msgr2=0x7f9a081c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 0x7f9a081c8190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400ec10 tx=0x7f9a0400ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 shutdown_connections 2023-09-30T22:29:20.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9a08133930 0x7f9a081c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9a0809d630 0x7f9a081c8190 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 --2- 172.21.15.32:0/3136570599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a080a0a30 0x7f9a081c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 >> 172.21.15.32:0/3136570599 conn(0x7f9a080a3050 msgr2=0x7f9a081bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 shutdown_connections 2023-09-30T22:29:20.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.542+0000 7f9a10241700 1 -- 172.21.15.32:0/3136570599 wait complete. 2023-09-30T22:29:20.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.701+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6f10072980 con 0x7f6f100a0a30 2023-09-30T22:29:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.702+0000 7f6f05ffb700 1 -- 172.21.15.32:0/1512266242 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f000123b0 con 0x7f6f100a0a30 2023-09-30T22:29:20.703 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 msgr2=0x7f6f10133c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 0x7f6f10133c40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c520 tx=0x7f6f0000c830 comp rx=0 tx=0).stop 2023-09-30T22:29:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 shutdown_connections 2023-09-30T22:29:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6f10138d80 0x7f6f10134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6f1009d630 0x7f6f10133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 --2- 172.21.15.32:0/1512266242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f100a0a30 0x7f6f10133c40 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 >> 172.21.15.32:0/1512266242 conn(0x7f6f100a3050 msgr2=0x7f6f1009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 shutdown_connections 2023-09-30T22:29:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.704+0000 7f6f178b7700 1 -- 172.21.15.32:0/1512266242 wait complete. 2023-09-30T22:29:20.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.738+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29d80a28e0 con 0x7f29d80a1510 2023-09-30T22:29:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.739+0000 7f29d4ff9700 1 -- 172.21.15.32:0/3165449143 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29cc017020 con 0x7f29d80a1510 2023-09-30T22:29:20.742 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 msgr2=0x7f29d8122e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 0x7f29d8122e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00e9b0 tx=0x7f29cc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 shutdown_connections 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29d8139250 0x7f29d8123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f29d80a0a30 0x7f29d81228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 --2- 172.21.15.32:0/3165449143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f29d80a1510 0x7f29d8122e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 >> 172.21.15.32:0/3165449143 conn(0x7f29d80a6550 msgr2=0x7f29d80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 shutdown_connections 2023-09-30T22:29:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:20.741+0000 7f29de31e700 1 -- 172.21.15.32:0/3165449143 wait complete. 2023-09-30T22:29:20.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:21.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 Processor -- start 2023-09-30T22:29:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 -- start start 2023-09-30T22:29:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 0x7fac880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac8813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8809edd0 con 0x7fac880a1340 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8809e1f0 con 0x7fac88134170 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac8e257700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac8809dc50 con 0x7fac880a0a30 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.434+0000 7fac877fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 0x7fac880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac8813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac877fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 0x7fac880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50114/0 (socket says 172.21.15.32:50114) 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac8813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37008/0 (socket says 172.21.15.32:37008) 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac877fe700 1 -- 172.21.15.32:0/1595640205 learned_addr learned my addr 172.21.15.32:0/1595640205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac86ffd700 1 --1- 172.21.15.32:0/1595640205 >> v1:172.21.15.32:6789/0 conn(0x7fac880a1340 0x7fac88133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54938/0 (socket says 172.21.15.32:54938) 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac867fc700 1 -- 172.21.15.32:0/1595640205 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599094396 0 0) 0x7fac8809edd0 con 0x7fac880a1340 2023-09-30T22:29:21.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 -- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 msgr2=0x7fac880a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 --2- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 0x7fac880a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 -- 172.21.15.32:0/1595640205 >> v1:172.21.15.32:6789/0 conn(0x7fac880a1340 legacy=0x7fac88133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 -- 172.21.15.32:0/1595640205 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac880a1fe0 con 0x7fac88134170 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac877fe700 1 --2- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 0x7fac880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac87fff700 1 --2- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac8813bd30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fac80009dc0 tx=0x7fac8000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4003a8fc608a1bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac867fc700 1 -- 172.21.15.32:0/1595640205 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8000e070 con 0x7fac88134170 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac867fc700 1 -- 172.21.15.32:0/1595640205 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8000bb20 con 0x7fac88134170 2023-09-30T22:29:21.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.435+0000 7fac867fc700 1 -- 172.21.15.32:0/1595640205 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80012430 con 0x7fac88134170 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 -- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 msgr2=0x7fac8813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 --2- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac8813bd30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fac80009dc0 tx=0x7fac8000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 -- 172.21.15.32:0/1595640205 shutdown_connections 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 --2- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac8813bd30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 --2- 172.21.15.32:0/1595640205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a0a30 0x7fac880a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 -- 172.21.15.32:0/1595640205 >> 172.21.15.32:0/1595640205 conn(0x7fac880a6550 msgr2=0x7fac880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 -- 172.21.15.32:0/1595640205 shutdown_connections 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.436+0000 7fac8e257700 1 -- 172.21.15.32:0/1595640205 wait complete. 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.437+0000 7fac8e257700 1 Processor -- start 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.437+0000 7fac8e257700 1 -- start start 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac8e257700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 0x7fac88122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac8e257700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a1340 0x7fac88122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac8e257700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac88123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac8e257700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac880a4ec0 con 0x7fac880a0a30 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac8e257700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac881336b0 con 0x7fac88134170 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac8e257700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac88124410 con 0x7fac880a1340 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 0x7fac88122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 0x7fac88122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56656/0 (socket says 172.21.15.32:56656) 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 -- 172.21.15.32:0/4020720827 learned_addr learned my addr 172.21.15.32:0/4020720827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac87fff700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac88123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac86ffd700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a1340 0x7fac88122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 -- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a1340 msgr2=0x7fac88122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a1340 0x7fac88122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 -- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 msgr2=0x7fac88123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac88123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac877fe700 1 -- 172.21.15.32:0/4020720827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78009550 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac87fff700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac88123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.438+0000 7fac86ffd700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a1340 0x7fac88122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac877fe700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 0x7fac88122780 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fac7800ec10 tx=0x7fac7800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7800c850 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7800c9b0 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7800cd40 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac80009a70 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac880a25e0 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.439+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fac78013470 con 0x7fac880a0a30 2023-09-30T22:29:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.440+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fac7803f920 con 0x7fac880a0a30 2023-09-30T22:29:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.440+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac881332b0 con 0x7fac880a0a30 2023-09-30T22:29:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.445+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fac881332b0 con 0x7fac880a0a30 2023-09-30T22:29:21.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:21.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.652+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fac881332b0 con 0x7fac880a0a30 2023-09-30T22:29:21.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.653+0000 7fac6ffff700 1 -- 172.21.15.32:0/4020720827 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac881332b0 con 0x7fac880a0a30 2023-09-30T22:29:21.654 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 msgr2=0x7fac88122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 0x7fac88122780 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fac7800ec10 tx=0x7fac7800ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 shutdown_connections 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac88134170 0x7fac88123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac880a0a30 0x7fac88122780 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 --2- 172.21.15.32:0/4020720827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac880a1340 0x7fac88122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 >> 172.21.15.32:0/4020720827 conn(0x7fac880a6550 msgr2=0x7fac8809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 shutdown_connections 2023-09-30T22:29:21.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.655+0000 7fac8e257700 1 -- 172.21.15.32:0/4020720827 wait complete. 2023-09-30T22:29:21.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:21.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.694+0000 7fbe3069a700 1 Processor -- start 2023-09-30T22:29:21.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.694+0000 7fbe3069a700 1 -- start start 2023-09-30T22:29:21.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe3069a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 0x7fbe280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe3069a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 0x7fbe2813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe3069a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe280a2a70 con 0x7fbe2809d630 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe3069a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe280a01d0 con 0x7fbe280a0a30 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe3069a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2809e2b0 con 0x7fbe28138d80 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2e436700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 0x7fbe280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 0x7fbe2813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2e436700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 0x7fbe280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56668/0 (socket says 172.21.15.32:56668) 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 0x7fbe2813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37024/0 (socket says 172.21.15.32:37024) 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2dc35700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbe2809d630 0x7fbe280a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54942/0 (socket says 172.21.15.32:54942) 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2e436700 1 -- 172.21.15.32:0/464022487 learned_addr learned my addr 172.21.15.32:0/464022487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 -- 172.21.15.32:0/464022487 >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 msgr2=0x7fbe280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 --2- 172.21.15.32:0/464022487 >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 0x7fbe280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 -- 172.21.15.32:0/464022487 >> v1:172.21.15.32:6789/0 conn(0x7fbe2809d630 legacy=0x7fbe280a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2d434700 1 -- 172.21.15.32:0/464022487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545444230 0 0) 0x7fbe280a2a70 con 0x7fbe2809d630 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 -- 172.21.15.32:0/464022487 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2809dc50 con 0x7fbe28138d80 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2e436700 1 --2- 172.21.15.32:0/464022487 >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 0x7fbe280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2ec37700 1 --2- 172.21.15.32:0/464022487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 0x7fbe2813b160 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009d90 tx=0x7fbe2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffa5bdff362682b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.695+0000 7fbe2d434700 1 -- 172.21.15.32:0/464022487 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000e070 con 0x7fbe28138d80 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe2d434700 1 -- 172.21.15.32:0/464022487 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2000bb20 con 0x7fbe28138d80 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe2d434700 1 -- 172.21.15.32:0/464022487 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20012430 con 0x7fbe28138d80 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 -- 172.21.15.32:0/464022487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 msgr2=0x7fbe2813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 --2- 172.21.15.32:0/464022487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 0x7fbe2813b160 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009d90 tx=0x7fbe2000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 -- 172.21.15.32:0/464022487 shutdown_connections 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 --2- 172.21.15.32:0/464022487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe28138d80 0x7fbe2813b160 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 --2- 172.21.15.32:0/464022487 >> v2:172.21.15.32:3300/0 conn(0x7fbe280a0a30 0x7fbe280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 -- 172.21.15.32:0/464022487 >> 172.21.15.32:0/464022487 conn(0x7fbe280a3050 msgr2=0x7fbe280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 -- 172.21.15.32:0/464022487 shutdown_connections 2023-09-30T22:29:21.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.696+0000 7fbe3069a700 1 -- 172.21.15.32:0/464022487 wait complete. 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.697+0000 7fbe3069a700 1 Processor -- start 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.697+0000 7fbe3069a700 1 -- start start 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe3069a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 0x7fbe2812d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe3069a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe280a0a30 0x7fbe2812df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe3069a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe28138d80 0x7fbe2812e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe3069a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2809b320 con 0x7fbe2809d630 2023-09-30T22:29:21.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe3069a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2809c460 con 0x7fbe280a0a30 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe3069a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280a0480 con 0x7fbe28138d80 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 0x7fbe2812d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 0x7fbe2812d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56674/0 (socket says 172.21.15.32:56674) 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 -- 172.21.15.32:0/2790678167 learned_addr learned my addr 172.21.15.32:0/2790678167 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2dc35700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe280a0a30 0x7fbe2812df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2ec37700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe28138d80 0x7fbe2812e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 -- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe28138d80 msgr2=0x7fbe2812e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe28138d80 0x7fbe2812e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 -- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe280a0a30 msgr2=0x7fbe2812df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe280a0a30 0x7fbe2812df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 -- 172.21.15.32:0/2790678167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c009550 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.698+0000 7fbe2e436700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 0x7fbe2812d9f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00ec10 tx=0x7fbe1c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.699+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00c7f0 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.699+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c00c950 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.699+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00cce0 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.699+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe20009a70 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.699+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2809b870 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.699+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe1c013470 con 0x7fbe2809d630 2023-09-30T22:29:21.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.700+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe280747f0 con 0x7fbe2809d630 2023-09-30T22:29:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.700+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbe1c03fc40 con 0x7fbe2809d630 2023-09-30T22:29:21.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.705+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe1c021080 con 0x7fbe2809d630 2023-09-30T22:29:21.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:21.756 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.857+0000 7f2dc3f2d700 1 Processor -- start 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.857+0000 7f2dc3f2d700 1 -- start start 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc3f2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc3f2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc3f2d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc070940 con 0x7f2dbc09f650 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc3f2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc06fd60 con 0x7f2dbc09e930 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc3f2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc06fa80 con 0x7f2dbc09f240 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37038/0 (socket says 172.21.15.32:37038) 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 -- 172.21.15.32:0/1283942498 learned_addr learned my addr 172.21.15.32:0/1283942498 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc24ca700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2dbc09f650 0x7f2dbc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54944/0 (socket says 172.21.15.32:54944) 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc14c8700 1 --2- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 -- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 msgr2=0x7f2dbc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 --2- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 -- 172.21.15.32:0/1283942498 >> v1:172.21.15.32:6789/0 conn(0x7f2dbc09f650 legacy=0x7f2dbc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.858+0000 7f2dc1cc9700 1 -- 172.21.15.32:0/1283942498 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc09fb60 con 0x7f2dbc09e930 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.859+0000 7f2dc1cc9700 1 --2- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc09ed00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009920 tx=0x7f2db8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe13abba0293a885 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.859+0000 7f2dc0cc7700 1 -- 172.21.15.32:0/1283942498 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8011830 con 0x7f2dbc09e930 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.859+0000 7f2dc0cc7700 1 -- 172.21.15.32:0/1283942498 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db8011990 con 0x7f2dbc09e930 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.859+0000 7f2dc0cc7700 1 -- 172.21.15.32:0/1283942498 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8011d20 con 0x7f2dbc09e930 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.859+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 msgr2=0x7f2dbc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.859+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc09ed00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009920 tx=0x7f2db8009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.860+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1283942498 shutdown_connections 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.860+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc09ed00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.860+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1283942498 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.860+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1283942498 >> 172.21.15.32:0/1283942498 conn(0x7f2dbc0a4210 msgr2=0x7f2dbc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.860+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1283942498 shutdown_connections 2023-09-30T22:29:21.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.860+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1283942498 wait complete. 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 Processor -- start 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 -- start start 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc1a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc09f650 0x7f2dbc1a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1a6080 con 0x7f2dbc09f650 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1a71b0 con 0x7f2dbc09e930 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc3f2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1a49d0 con 0x7f2dbc09f240 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc14c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc14c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50154/0 (socket says 172.21.15.32:50154) 2023-09-30T22:29:21.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.861+0000 7f2dc14c8700 1 -- 172.21.15.32:0/1271199845 learned_addr learned my addr 172.21.15.32:0/1271199845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc1cc9700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc1a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc24ca700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc09f650 0x7f2dbc1a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc14c8700 1 -- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 msgr2=0x7f2dbc1a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc14c8700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc1a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc14c8700 1 -- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc09f650 msgr2=0x7f2dbc1a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc14c8700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc09f650 0x7f2dbc1a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc14c8700 1 -- 172.21.15.32:0/1271199845 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009550 con 0x7f2dbc09f240 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc14c8700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1a3280 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00e9b0 tx=0x7f2dac00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac009710 con 0x7f2dbc09f240 2023-09-30T22:29:21.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac00ee50 con 0x7f2dbc09f240 2023-09-30T22:29:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00ca10 con 0x7f2dbc09f240 2023-09-30T22:29:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc0a0190 con 0x7f2dbc09f240 2023-09-30T22:29:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.862+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc1a5bb0 con 0x7f2dbc09f240 2023-09-30T22:29:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.863+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc074ea0 con 0x7f2dbc09f240 2023-09-30T22:29:21.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.864+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2dac00cb70 con 0x7f2dbc09f240 2023-09-30T22:29:21.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.864+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2dac03f530 con 0x7f2dbc09f240 2023-09-30T22:29:21.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.872+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2dac013470 con 0x7f2dbc09f240 2023-09-30T22:29:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.906+0000 7ff96cae1700 1 Processor -- start 2023-09-30T22:29:21.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.906+0000 7ff96cae1700 1 -- start start 2023-09-30T22:29:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96cae1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968136820 0x7ff968138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96cae1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff968139140 0x7ff96813b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96cae1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96806bc00 con 0x7ff968133a80 2023-09-30T22:29:21.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96cae1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96806c570 con 0x7ff968139140 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96cae1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680a0040 con 0x7ff968136820 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff968133a80 0x7ff968133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54954/0 (socket says 172.21.15.32:54954) 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96659c700 1 -- 172.21.15.32:0/3479165643 learned_addr learned my addr 172.21.15.32:0/3479165643 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff965d9b700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968136820 0x7ff968138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff966d9d700 1 --2- 172.21.15.32:0/3479165643 >> v2:172.21.15.32:3300/0 conn(0x7ff968139140 0x7ff96813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff96559a700 1 -- 172.21.15.32:0/3479165643 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472962606 0 0) 0x7ff96806bc00 con 0x7ff968133a80 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff966d9d700 1 -- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968136820 msgr2=0x7ff968138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff966d9d700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968136820 0x7ff968138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff966d9d700 1 -- 172.21.15.32:0/3479165643 >> v1:172.21.15.32:6789/0 conn(0x7ff968133a80 legacy=0x7ff968133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff966d9d700 1 -- 172.21.15.32:0/3479165643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968133860 con 0x7ff968139140 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff965d9b700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968136820 0x7ff968138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.907+0000 7ff966d9d700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968139140 0x7ff96813b520 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff960009d90 tx=0x7ff96000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baf0dfdf4c459a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96559a700 1 -- 172.21.15.32:0/3479165643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000e070 con 0x7ff968139140 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96559a700 1 -- 172.21.15.32:0/3479165643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000bb20 con 0x7ff968139140 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96559a700 1 -- 172.21.15.32:0/3479165643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960012430 con 0x7ff968139140 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 -- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968139140 msgr2=0x7ff96813b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968139140 0x7ff96813b520 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff960009d90 tx=0x7ff96000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 -- 172.21.15.32:0/3479165643 shutdown_connections 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968139140 0x7ff96813b520 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 --2- 172.21.15.32:0/3479165643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968136820 0x7ff968138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 -- 172.21.15.32:0/3479165643 >> 172.21.15.32:0/3479165643 conn(0x7ff9680a4210 msgr2=0x7ff9680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.908+0000 7ff96cae1700 1 -- 172.21.15.32:0/3479165643 shutdown_connections 2023-09-30T22:29:21.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.909+0000 7ff96cae1700 1 -- 172.21.15.32:0/3479165643 wait complete. 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 Processor -- start 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 -- start start 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968133a80 0x7ff96806f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 0x7ff96806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968139140 0x7ff9680703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96812dc10 con 0x7ff968133a80 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96806d050 con 0x7ff968136820 2023-09-30T22:29:21.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96cae1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff96809bc90 con 0x7ff968139140 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff965d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 0x7ff96806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff966d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968139140 0x7ff9680703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff966d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968139140 0x7ff9680703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50178/0 (socket says 172.21.15.32:50178) 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff965d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 0x7ff96806fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37052/0 (socket says 172.21.15.32:37052) 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff966d9d700 1 -- 172.21.15.32:0/724231146 learned_addr learned my addr 172.21.15.32:0/724231146 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.910+0000 7ff96659c700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968133a80 0x7ff96806f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff965d9b700 1 -- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968139140 msgr2=0x7ff9680703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff965d9b700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968139140 0x7ff9680703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff965d9b700 1 -- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968133a80 msgr2=0x7ff96806f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff965d9b700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968133a80 0x7ff96806f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff965d9b700 1 -- 172.21.15.32:0/724231146 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0094a0 con 0x7ff968136820 2023-09-30T22:29:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff965d9b700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 0x7ff96806fe90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e9e0 tx=0x7ff95c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00ee80 con 0x7ff968136820 2023-09-30T22:29:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c00c850 con 0x7ff968136820 2023-09-30T22:29:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff960009a70 con 0x7ff968136820 2023-09-30T22:29:21.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00cc00 con 0x7ff968136820 2023-09-30T22:29:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.911+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968072280 con 0x7ff968136820 2023-09-30T22:29:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.912+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff95c00cd60 con 0x7ff968136820 2023-09-30T22:29:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.912+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff95c03fa90 con 0x7ff968136820 2023-09-30T22:29:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.913+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9680724a0 con 0x7ff968136820 2023-09-30T22:29:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.916+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbe2812cbf0 con 0x7fbe2809d630 2023-09-30T22:29:21.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.916+0000 7fbe1affd700 1 -- 172.21.15.32:0/2790678167 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe1c013720 con 0x7fbe2809d630 2023-09-30T22:29:21.918 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 msgr2=0x7fbe2812d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 0x7fbe2812d9f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00ec10 tx=0x7fbe1c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 shutdown_connections 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbe28138d80 0x7fbe2812e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe2809d630 0x7fbe2812d9f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 --2- 172.21.15.32:0/2790678167 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe280a0a30 0x7fbe2812df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 >> 172.21.15.32:0/2790678167 conn(0x7fbe280a3050 msgr2=0x7fbe280a4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 shutdown_connections 2023-09-30T22:29:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.919+0000 7fbe3069a700 1 -- 172.21.15.32:0/2790678167 wait complete. 2023-09-30T22:29:21.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:21.921+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9680724a0 con 0x7ff968136820 2023-09-30T22:29:21.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:22.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.082+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2dbc1a5bb0 con 0x7f2dbc09f240 2023-09-30T22:29:22.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.082+0000 7f2db27fc700 1 -- 172.21.15.32:0/1271199845 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2dbc1a5bb0 con 0x7f2dbc09f240 2023-09-30T22:29:22.083 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.084+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 msgr2=0x7f2dbc1a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.084+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1a3280 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00e9b0 tx=0x7f2dac00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 shutdown_connections 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2dbc09f650 0x7f2dbc1a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2dbc09e930 0x7f2dbc1a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 --2- 172.21.15.32:0/1271199845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2dbc09f240 0x7f2dbc1a3280 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 >> 172.21.15.32:0/1271199845 conn(0x7f2dbc0a4210 msgr2=0x7f2dbc1b8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 shutdown_connections 2023-09-30T22:29:22.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.085+0000 7f2dc3f2d700 1 -- 172.21.15.32:0/1271199845 wait complete. 2023-09-30T22:29:22.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.128+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff968074ea0 con 0x7ff968136820 2023-09-30T22:29:22.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.128+0000 7ff956ffd700 1 -- 172.21.15.32:0/724231146 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff95c021080 con 0x7ff968136820 2023-09-30T22:29:22.129 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 msgr2=0x7ff96806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 0x7ff96806fe90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e9e0 tx=0x7ff95c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 shutdown_connections 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff968139140 0x7ff9680703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff968133a80 0x7ff96806f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 --2- 172.21.15.32:0/724231146 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff968136820 0x7ff96806fe90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 >> 172.21.15.32:0/724231146 conn(0x7ff9680a4210 msgr2=0x7ff968137410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 shutdown_connections 2023-09-30T22:29:22.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.131+0000 7ff96cae1700 1 -- 172.21.15.32:0/724231146 wait complete. 2023-09-30T22:29:22.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:22.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.816+0000 7fbadd673700 1 Processor -- start 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.816+0000 7fbadd673700 1 -- start start 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbadd673700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 0x7fbad8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbadd673700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad8139170 0x7fbad813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbadd673700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad809b120 con 0x7fbad809d6f0 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbadd673700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad806b460 con 0x7fbad809e1d0 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbadd673700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbad809a7e0 con 0x7fbad8139170 2023-09-30T22:29:22.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 0x7fbad8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad8139170 0x7fbad813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbad809d6f0 0x7fbad809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54958/0 (socket says 172.21.15.32:54958) 2023-09-30T22:29:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad6ffd700 1 -- 172.21.15.32:0/2851087330 learned_addr learned my addr 172.21.15.32:0/2851087330 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:22.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 0x7fbad8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37066/0 (socket says 172.21.15.32:37066) 2023-09-30T22:29:22.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad67fc700 1 -- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad8139170 msgr2=0x7fbad813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad67fc700 1 --2- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad8139170 0x7fbad813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad5ffb700 1 -- 172.21.15.32:0/2851087330 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698103152 0 0) 0x7fbad809b120 con 0x7fbad809d6f0 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad67fc700 1 -- 172.21.15.32:0/2851087330 >> v1:172.21.15.32:6789/0 conn(0x7fbad809d6f0 legacy=0x7fbad809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad67fc700 1 -- 172.21.15.32:0/2851087330 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad809a500 con 0x7fbad809e1d0 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.817+0000 7fbad77fe700 1 --2- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad8139170 0x7fbad813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbad67fc700 1 --2- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 0x7fbad8138c30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009870 tx=0x7fbacc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d446ffd7b451ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbad5ffb700 1 -- 172.21.15.32:0/2851087330 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc011830 con 0x7fbad809e1d0 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbad5ffb700 1 -- 172.21.15.32:0/2851087330 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc011990 con 0x7fbad809e1d0 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbad5ffb700 1 -- 172.21.15.32:0/2851087330 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc011d20 con 0x7fbad809e1d0 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 -- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 msgr2=0x7fbad8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 --2- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 0x7fbad8138c30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009870 tx=0x7fbacc009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 -- 172.21.15.32:0/2851087330 shutdown_connections 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 --2- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad8139170 0x7fbad813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 --2- 172.21.15.32:0/2851087330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad809e1d0 0x7fbad8138c30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 -- 172.21.15.32:0/2851087330 >> 172.21.15.32:0/2851087330 conn(0x7fbad8066f50 msgr2=0x7fbad809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 -- 172.21.15.32:0/2851087330 shutdown_connections 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.818+0000 7fbadd673700 1 -- 172.21.15.32:0/2851087330 wait complete. 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 Processor -- start 2023-09-30T22:29:22.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 -- start start 2023-09-30T22:29:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 0x7fbad8120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad809e1d0 0x7fbad8120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad8139170 0x7fbad81211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8122ba0 con 0x7fbad809d6f0 2023-09-30T22:29:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8136610 con 0x7fbad8139170 2023-09-30T22:29:22.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbadd673700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbad809c710 con 0x7fbad809e1d0 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbad6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 0x7fbad8120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbad77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad8139170 0x7fbad81211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbad6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 0x7fbad8120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56702/0 (socket says 172.21.15.32:56702) 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbad6ffd700 1 -- 172.21.15.32:0/1493718752 learned_addr learned my addr 172.21.15.32:0/1493718752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.820+0000 7fbad67fc700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad809e1d0 0x7fbad8120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad77fe700 1 -- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad809e1d0 msgr2=0x7fbad8120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad77fe700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad809e1d0 0x7fbad8120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad77fe700 1 -- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 msgr2=0x7fbad8120770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad77fe700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 0x7fbad8120770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad77fe700 1 -- 172.21.15.32:0/1493718752 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009a70 con 0x7fbad8139170 2023-09-30T22:29:22.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad6ffd700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 0x7fbad8120770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbad77fe700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad8139170 0x7fbad81211f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbad000efc0 tx=0x7fbad000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000ca90 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad000f040 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000ce30 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc0094a0 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.821+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad8072150 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.822+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbad0014480 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.822+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbad003fcb0 con 0x7fbad8139170 2023-09-30T22:29:22.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.822+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad8123e80 con 0x7fbad8139170 2023-09-30T22:29:22.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:22.828+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbad0012910 con 0x7fbad8139170 2023-09-30T22:29:22.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:23.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.035+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbad8074ea0 con 0x7fbad8139170 2023-09-30T22:29:23.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.035+0000 7fbabf7fe700 1 -- 172.21.15.32:0/1493718752 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbad0012340 con 0x7fbad8139170 2023-09-30T22:29:23.036 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad8139170 msgr2=0x7fbad81211f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad8139170 0x7fbad81211f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbad000efc0 tx=0x7fbad000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 shutdown_connections 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbad8139170 0x7fbad81211f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad809d6f0 0x7fbad8120770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 --2- 172.21.15.32:0/1493718752 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbad809e1d0 0x7fbad8120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 >> 172.21.15.32:0/1493718752 conn(0x7fbad8066f50 msgr2=0x7fbad8136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 shutdown_connections 2023-09-30T22:29:23.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.038+0000 7fbadd673700 1 -- 172.21.15.32:0/1493718752 wait complete. 2023-09-30T22:29:23.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:23.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.076+0000 7f41fc47e700 1 Processor -- start 2023-09-30T22:29:23.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.076+0000 7f41fc47e700 1 -- start start 2023-09-30T22:29:23.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.076+0000 7f41fc47e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409f410 0x7f41f4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41fc47e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41f4136fe0 0x7f41f41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41fc47e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f4070940 con 0x7f41f409e930 2023-09-30T22:29:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41fc47e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f406fd60 con 0x7f41f4136fe0 2023-09-30T22:29:23.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41fc47e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41f406fa80 con 0x7f41f409f410 2023-09-30T22:29:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9a19700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409f410 0x7f41f4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41faa1b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41f4136fe0 0x7f41f41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41fa21a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41f409e930 0x7f41f409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54966/0 (socket says 172.21.15.32:54966) 2023-09-30T22:29:23.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41fa21a700 1 -- 172.21.15.32:0/697346684 learned_addr learned my addr 172.21.15.32:0/697346684 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9a19700 1 -- 172.21.15.32:0/697346684 >> v2:172.21.15.32:3300/0 conn(0x7f41f4136fe0 msgr2=0x7f41f41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9a19700 1 --2- 172.21.15.32:0/697346684 >> v2:172.21.15.32:3300/0 conn(0x7f41f4136fe0 0x7f41f41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9a19700 1 -- 172.21.15.32:0/697346684 >> v1:172.21.15.32:6789/0 conn(0x7f41f409e930 legacy=0x7f41f409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9a19700 1 -- 172.21.15.32:0/697346684 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f409fb60 con 0x7f41f409f410 2023-09-30T22:29:23.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9218700 1 -- 172.21.15.32:0/697346684 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956465355 0 0) 0x7f41f4070940 con 0x7f41f409e930 2023-09-30T22:29:23.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41faa1b700 1 --2- 172.21.15.32:0/697346684 >> v2:172.21.15.32:3300/0 conn(0x7f41f4136fe0 0x7f41f41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9a19700 1 --2- 172.21.15.32:0/697346684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409f410 0x7f41f4136aa0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009840 tx=0x7f41f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=508d12df79712c54 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9218700 1 -- 172.21.15.32:0/697346684 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0011830 con 0x7f41f409f410 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.077+0000 7f41f9218700 1 -- 172.21.15.32:0/697346684 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0011990 con 0x7f41f409f410 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41f9218700 1 -- 172.21.15.32:0/697346684 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0011d20 con 0x7f41f409f410 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 -- 172.21.15.32:0/697346684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409f410 msgr2=0x7f41f4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 --2- 172.21.15.32:0/697346684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409f410 0x7f41f4136aa0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009840 tx=0x7f41f0009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 -- 172.21.15.32:0/697346684 shutdown_connections 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 --2- 172.21.15.32:0/697346684 >> v2:172.21.15.32:3300/0 conn(0x7f41f4136fe0 0x7f41f41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 --2- 172.21.15.32:0/697346684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409f410 0x7f41f4136aa0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 -- 172.21.15.32:0/697346684 >> 172.21.15.32:0/697346684 conn(0x7f41f40a4210 msgr2=0x7f41f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 -- 172.21.15.32:0/697346684 shutdown_connections 2023-09-30T22:29:23.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.078+0000 7f41fc47e700 1 -- 172.21.15.32:0/697346684 wait complete. 2023-09-30T22:29:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.079+0000 7f41fc47e700 1 Processor -- start 2023-09-30T22:29:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.079+0000 7f41fc47e700 1 -- start start 2023-09-30T22:29:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fc47e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 0x7f41f41c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fc47e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 0x7f41f41c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fc47e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f4136fe0 0x7f41f411c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fc47e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41c7080 con 0x7f41f4136fe0 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fc47e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f411e2f0 con 0x7f41f409f410 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fc47e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4120320 con 0x7f41f409e930 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fa21a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 0x7f41f41c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f4136fe0 0x7f41f411c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41f9a19700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 0x7f41f41c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fa21a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 0x7f41f41c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50210/0 (socket says 172.21.15.32:50210) 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fa21a700 1 -- 172.21.15.32:0/2768975733 learned_addr learned my addr 172.21.15.32:0/2768975733 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41f9a19700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 0x7f41f41c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37088/0 (socket says 172.21.15.32:37088) 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 -- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 msgr2=0x7f41f41c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 0x7f41f41c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 -- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 msgr2=0x7f41f41c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 0x7f41f41c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 -- 172.21.15.32:0/2768975733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009a70 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41fa21a700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 0x7f41f41c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41f9a19700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 0x7f41f41c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.080+0000 7f41faa1b700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f4136fe0 0x7f41f411c6e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00ef90 tx=0x7f41ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.081+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00cc90 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.081+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec00f040 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.081+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f00094a0 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.081+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0146c0 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.081+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f41325d0 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.081+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f41ec014820 con 0x7f41f4136fe0 2023-09-30T22:29:23.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.082+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f41ec040200 con 0x7f41f4136fe0 2023-09-30T22:29:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.082+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f4071d20 con 0x7f41f4136fe0 2023-09-30T22:29:23.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.087+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41ec018080 con 0x7f41f4136fe0 2023-09-30T22:29:23.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:23.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.242+0000 7fed6aa45700 1 Processor -- start 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.242+0000 7fed6aa45700 1 -- start start 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed6aa45700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed6409d6f0 0x7fed6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed6aa45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 0x7fed64134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed6aa45700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6409b120 con 0x7fed6409e000 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed6aa45700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6406b460 con 0x7fed6409d6f0 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed6aa45700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6409a7e0 con 0x7fed6409e440 2023-09-30T22:29:23.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 0x7fed64134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed63fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed6409d6f0 0x7fed6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 0x7fed64134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37098/0 (socket says 172.21.15.32:37098) 2023-09-30T22:29:23.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 -- 172.21.15.32:0/4221518875 learned_addr learned my addr 172.21.15.32:0/4221518875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 -- 172.21.15.32:0/4221518875 >> v2:172.21.15.32:3300/0 conn(0x7fed6409d6f0 msgr2=0x7fed6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 --2- 172.21.15.32:0/4221518875 >> v2:172.21.15.32:3300/0 conn(0x7fed6409d6f0 0x7fed6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 -- 172.21.15.32:0/4221518875 >> v1:172.21.15.32:6789/0 conn(0x7fed6409e000 legacy=0x7fed641339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.243+0000 7fed68fe2700 1 -- 172.21.15.32:0/4221518875 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6409a500 con 0x7fed6409e440 2023-09-30T22:29:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.244+0000 7fed68fe2700 1 --2- 172.21.15.32:0/4221518875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 0x7fed64134100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009e80 tx=0x7fed5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ebfecb7f99ef328 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.244+0000 7fed62ffd700 1 -- 172.21.15.32:0/4221518875 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00e070 con 0x7fed6409e440 2023-09-30T22:29:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.244+0000 7fed62ffd700 1 -- 172.21.15.32:0/4221518875 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00b8d0 con 0x7fed6409e440 2023-09-30T22:29:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.244+0000 7fed62ffd700 1 -- 172.21.15.32:0/4221518875 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00bc80 con 0x7fed6409e440 2023-09-30T22:29:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.244+0000 7fed6aa45700 1 -- 172.21.15.32:0/4221518875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 msgr2=0x7fed64134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.244+0000 7fed6aa45700 1 --2- 172.21.15.32:0/4221518875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 0x7fed64134100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009e80 tx=0x7fed5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.245+0000 7fed6aa45700 1 -- 172.21.15.32:0/4221518875 shutdown_connections 2023-09-30T22:29:23.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.245+0000 7fed6aa45700 1 --2- 172.21.15.32:0/4221518875 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e440 0x7fed64134100 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.245+0000 7fed6aa45700 1 --2- 172.21.15.32:0/4221518875 >> v2:172.21.15.32:3300/0 conn(0x7fed6409d6f0 0x7fed6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.245+0000 7fed6aa45700 1 -- 172.21.15.32:0/4221518875 >> 172.21.15.32:0/4221518875 conn(0x7fed64066f50 msgr2=0x7fed6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.245+0000 7fed6aa45700 1 -- 172.21.15.32:0/4221518875 shutdown_connections 2023-09-30T22:29:23.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.245+0000 7fed6aa45700 1 -- 172.21.15.32:0/4221518875 wait complete. 2023-09-30T22:29:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 Processor -- start 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 -- start start 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 0x7fed641a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e000 0x7fed641a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6409e440 0x7fed641a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed641a99d0 con 0x7fed6409d6f0 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed641a7b60 con 0x7fed6409e000 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed6aa45700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed640a1f90 con 0x7fed6409e440 2023-09-30T22:29:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.246+0000 7fed63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 0x7fed641a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 0x7fed641a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56744/0 (socket says 172.21.15.32:56744) 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 -- 172.21.15.32:0/923360488 learned_addr learned my addr 172.21.15.32:0/923360488 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed637fe700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e000 0x7fed641a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed68fe2700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6409e440 0x7fed641a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 -- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6409e440 msgr2=0x7fed641a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6409e440 0x7fed641a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 -- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e000 msgr2=0x7fed641a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e000 0x7fed641a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 -- 172.21.15.32:0/923360488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c009b30 con 0x7fed6409d6f0 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed68fe2700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6409e440 0x7fed641a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed637fe700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e000 0x7fed641a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed63fff700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 0x7fed641a5eb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e9b0 tx=0x7fed5400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54009710 con 0x7fed6409d6f0 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5400ee50 con 0x7fed6409d6f0 2023-09-30T22:29:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6409c710 con 0x7fed6409d6f0 2023-09-30T22:29:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5400ca50 con 0x7fed6409d6f0 2023-09-30T22:29:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.247+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed641a8dd0 con 0x7fed6409d6f0 2023-09-30T22:29:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.248+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fed5400cbb0 con 0x7fed6409d6f0 2023-09-30T22:29:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.249+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed64074ea0 con 0x7fed6409d6f0 2023-09-30T22:29:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.249+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fed5403fa90 con 0x7fed6409d6f0 2023-09-30T22:29:23.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.254+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed54017020 con 0x7fed6409d6f0 2023-09-30T22:29:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.289+0000 7fd821c7a700 1 Processor -- start 2023-09-30T22:29:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 -- start start 2023-09-30T22:29:23.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c133a80 0x7fd81c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd81c130870 0x7fd81c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c06c570 con 0x7fd81c130870 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c06bc00 con 0x7fd81c133a80 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd821c7a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0a0040 con 0x7fd81c12ff20 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd81affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd81affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50214/0 (socket says 172.21.15.32:50214) 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd81affd700 1 -- 172.21.15.32:0/3580888798 learned_addr learned my addr 172.21.15.32:0/3580888798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd81bfff700 1 --2- 172.21.15.32:0/3580888798 >> v2:172.21.15.32:3300/0 conn(0x7fd81c130870 0x7fd81c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.290+0000 7fd81b7fe700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c133a80 0x7fd81c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81affd700 1 -- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c133a80 msgr2=0x7fd81c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81affd700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c133a80 0x7fd81c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81affd700 1 -- 172.21.15.32:0/3580888798 >> v2:172.21.15.32:3300/0 conn(0x7fd81c130870 msgr2=0x7fd81c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:23.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81affd700 1 --2- 172.21.15.32:0/3580888798 >> v2:172.21.15.32:3300/0 conn(0x7fd81c130870 0x7fd81c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81affd700 1 -- 172.21.15.32:0/3580888798 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c133860 con 0x7fd81c12ff20 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81bfff700 1 --2- 172.21.15.32:0/3580888798 >> v2:172.21.15.32:3300/0 conn(0x7fd81c130870 0x7fd81c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81b7fe700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c133a80 0x7fd81c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81affd700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c130330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb514da96075a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81a7fc700 1 -- 172.21.15.32:0/3580888798 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011830 con 0x7fd81c12ff20 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81a7fc700 1 -- 172.21.15.32:0/3580888798 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810011990 con 0x7fd81c12ff20 2023-09-30T22:29:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.291+0000 7fd81a7fc700 1 -- 172.21.15.32:0/3580888798 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011d20 con 0x7fd81c12ff20 2023-09-30T22:29:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 -- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 msgr2=0x7fd81c130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c130330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 -- 172.21.15.32:0/3580888798 shutdown_connections 2023-09-30T22:29:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c133a80 0x7fd81c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 --2- 172.21.15.32:0/3580888798 >> v2:172.21.15.32:3300/0 conn(0x7fd81c130870 0x7fd81c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 --2- 172.21.15.32:0/3580888798 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c130330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 -- 172.21.15.32:0/3580888798 >> 172.21.15.32:0/3580888798 conn(0x7fd81c0a4210 msgr2=0x7fd81c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 -- 172.21.15.32:0/3580888798 shutdown_connections 2023-09-30T22:29:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.292+0000 7fd821c7a700 1 -- 172.21.15.32:0/3580888798 wait complete. 2023-09-30T22:29:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.293+0000 7fd821c7a700 1 Processor -- start 2023-09-30T22:29:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.293+0000 7fd821c7a700 1 -- start start 2023-09-30T22:29:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd821c7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd821c7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 0x7fd81c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd821c7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd81c133a80 0x7fd81c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd821c7a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c12db30 con 0x7fd81c133a80 2023-09-30T22:29:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd821c7a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c09df50 con 0x7fd81c130870 2023-09-30T22:29:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd821c7a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c12d5c0 con 0x7fd81c12ff20 2023-09-30T22:29:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 0x7fd81c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd81c133a80 0x7fd81c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 0x7fd81c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37128/0 (socket says 172.21.15.32:37128) 2023-09-30T22:29:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 -- 172.21.15.32:0/1386666089 learned_addr learned my addr 172.21.15.32:0/1386666089 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81b7fe700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 -- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 msgr2=0x7fd81c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 -- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd81c133a80 msgr2=0x7fd81c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd81c133a80 0x7fd81c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 -- 172.21.15.32:0/1386666089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009a70 con 0x7fd81c130870 2023-09-30T22:29:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81bfff700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd81c133a80 0x7fd81c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.294+0000 7fd81affd700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 0x7fd81c09c9c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd810000c00 tx=0x7fd810004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.295+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810004240 con 0x7fd81c130870 2023-09-30T22:29:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.295+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81001f070 con 0x7fd81c130870 2023-09-30T22:29:23.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.295+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c0094a0 con 0x7fd81c130870 2023-09-30T22:29:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.295+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100045c0 con 0x7fd81c130870 2023-09-30T22:29:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.295+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd810009550 con 0x7fd81c130870 2023-09-30T22:29:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.296+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd810021030 con 0x7fd81c130870 2023-09-30T22:29:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.296+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c074ea0 con 0x7fd81c130870 2023-09-30T22:29:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.296+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd810040750 con 0x7fd81c130870 2023-09-30T22:29:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.301+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd810015070 con 0x7fd81c130870 2023-09-30T22:29:23.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.308+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f41f4074ea0 con 0x7f41f4136fe0 2023-09-30T22:29:23.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.308+0000 7f41eaffd700 1 -- 172.21.15.32:0/2768975733 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41ec0076f0 con 0x7f41f4136fe0 2023-09-30T22:29:23.309 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f4136fe0 msgr2=0x7f41f411c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f4136fe0 0x7f41f411c6e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00ef90 tx=0x7f41ec00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 shutdown_connections 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f4136fe0 0x7f41f411c6e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f41f409e930 0x7f41f41c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 --2- 172.21.15.32:0/2768975733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41f409f410 0x7f41f41c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 >> 172.21.15.32:0/2768975733 conn(0x7f41f40a4210 msgr2=0x7f41f4134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 shutdown_connections 2023-09-30T22:29:23.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.311+0000 7f41fc47e700 1 -- 172.21.15.32:0/2768975733 wait complete. 2023-09-30T22:29:23.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:23.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.464+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fed6409f420 con 0x7fed6409d6f0 2023-09-30T22:29:23.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.464+0000 7fed60ff9700 1 -- 172.21.15.32:0/923360488 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed5401b3b0 con 0x7fed6409d6f0 2023-09-30T22:29:23.464 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 msgr2=0x7fed641a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 0x7fed641a5eb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e9b0 tx=0x7fed5400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 shutdown_connections 2023-09-30T22:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed6409e440 0x7fed641a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed6409d6f0 0x7fed641a5eb0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 --2- 172.21.15.32:0/923360488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed6409e000 0x7fed641a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 >> 172.21.15.32:0/923360488 conn(0x7fed64066f50 msgr2=0x7fed641badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 shutdown_connections 2023-09-30T22:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.467+0000 7fed6aa45700 1 -- 172.21.15.32:0/923360488 wait complete. 2023-09-30T22:29:23.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:23.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.512+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd81c1201c0 con 0x7fd81c130870 2023-09-30T22:29:23.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.513+0000 7fd803fff700 1 -- 172.21.15.32:0/1386666089 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd81001a450 con 0x7fd81c130870 2023-09-30T22:29:23.513 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:23.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 msgr2=0x7fd81c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 0x7fd81c09c9c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd810000c00 tx=0x7fd810004750 comp rx=0 tx=0).stop 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 shutdown_connections 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd81c133a80 0x7fd81c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd81c12ff20 0x7fd81c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 --2- 172.21.15.32:0/1386666089 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd81c130870 0x7fd81c09c9c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 >> 172.21.15.32:0/1386666089 conn(0x7fd81c0a4210 msgr2=0x7fd81c0a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 shutdown_connections 2023-09-30T22:29:23.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:23.515+0000 7fd821c7a700 1 -- 172.21.15.32:0/1386666089 wait complete. 2023-09-30T22:29:23.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:24.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 Processor -- start 2023-09-30T22:29:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 -- start start 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfac09e930 0x7fbfac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 0x7fbfac131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac070940 con 0x7fbfac09e930 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac06fd60 con 0x7fbfac09f650 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.200+0000 7fbfb440c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac06fa80 con 0x7fbfac09f240 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb19a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 0x7fbfac131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb21a8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbfac09e930 0x7fbfac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 0x7fbfac131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37130/0 (socket says 172.21.15.32:37130) 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb19a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50230/0 (socket says 172.21.15.32:50230) 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 -- 172.21.15.32:0/4245798024 learned_addr learned my addr 172.21.15.32:0/4245798024 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 -- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 msgr2=0x7fbfac1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 --2- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 -- 172.21.15.32:0/4245798024 >> v2:172.21.15.32:3300/0 conn(0x7fbfac09e930 msgr2=0x7fbfac09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 --2- 172.21.15.32:0/4245798024 >> v2:172.21.15.32:3300/0 conn(0x7fbfac09e930 0x7fbfac09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 -- 172.21.15.32:0/4245798024 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac09fb60 con 0x7fbfac09f650 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb19a7700 1 --2- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb29a9700 1 --2- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 0x7fbfac131d30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009e50 tx=0x7fbfa400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b902a4ee552c8368 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb11a6700 1 -- 172.21.15.32:0/4245798024 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa400e070 con 0x7fbfac09f650 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb11a6700 1 -- 172.21.15.32:0/4245798024 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa400b910 con 0x7fbfac09f650 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.201+0000 7fbfb11a6700 1 -- 172.21.15.32:0/4245798024 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa400bcc0 con 0x7fbfac09f650 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 -- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 msgr2=0x7fbfac131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 --2- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 0x7fbfac131d30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009e50 tx=0x7fbfa400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 -- 172.21.15.32:0/4245798024 shutdown_connections 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 --2- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09f650 0x7fbfac131d30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 --2- 172.21.15.32:0/4245798024 >> v2:172.21.15.32:3300/0 conn(0x7fbfac09e930 0x7fbfac09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 --2- 172.21.15.32:0/4245798024 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 -- 172.21.15.32:0/4245798024 >> 172.21.15.32:0/4245798024 conn(0x7fbfac0a4210 msgr2=0x7fbfac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 -- 172.21.15.32:0/4245798024 shutdown_connections 2023-09-30T22:29:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.202+0000 7fbfb440c700 1 -- 172.21.15.32:0/4245798024 wait complete. 2023-09-30T22:29:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.203+0000 7fbfb440c700 1 Processor -- start 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.203+0000 7fbfb440c700 1 -- start start 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 0x7fbfac06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 0x7fbfac06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac06d4c0 con 0x7fbfac09f650 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac1a92b0 con 0x7fbfac09e930 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac06cd10 con 0x7fbfac09f240 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 0x7fbfac06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb29a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 0x7fbfac06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 0x7fbfac06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37138/0 (socket says 172.21.15.32:37138) 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb29a9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 0x7fbfac06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56772/0 (socket says 172.21.15.32:56772) 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 -- 172.21.15.32:0/713503924 learned_addr learned my addr 172.21.15.32:0/713503924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb19a7700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 -- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 msgr2=0x7fbfac06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 -- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 msgr2=0x7fbfac06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 0x7fbfac06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 -- 172.21.15.32:0/713503924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0094a0 con 0x7fbfac09e930 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb19a7700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb29a9700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 0x7fbfac06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb21a8700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 0x7fbfac06b8b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e960 tx=0x7fbfa800ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8013680 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa80137e0 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa4009b30 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.204+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8013b70 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.205+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfac0a0190 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.205+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbfa8013cd0 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.205+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbfa803fa10 con 0x7fbfac09e930 2023-09-30T22:29:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.205+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfac1a0b30 con 0x7fbfac09e930 2023-09-30T22:29:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.210+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfa8021020 con 0x7fbfac09e930 2023-09-30T22:29:24.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:24.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.417+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbfac1a8fe0 con 0x7fbfac09e930 2023-09-30T22:29:24.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.418+0000 7fbfa2ffd700 1 -- 172.21.15.32:0/713503924 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfa8017070 con 0x7fbfac09e930 2023-09-30T22:29:24.418 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 msgr2=0x7fbfac06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 0x7fbfac06b8b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e960 tx=0x7fbfa800ec70 comp rx=0 tx=0).stop 2023-09-30T22:29:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 shutdown_connections 2023-09-30T22:29:24.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfac09f650 0x7fbfac06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbfac09e930 0x7fbfac06b8b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 --2- 172.21.15.32:0/713503924 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbfac09f240 0x7fbfac06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 >> 172.21.15.32:0/713503924 conn(0x7fbfac0a4210 msgr2=0x7fbfac1b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 shutdown_connections 2023-09-30T22:29:24.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.420+0000 7fbfb440c700 1 -- 172.21.15.32:0/713503924 wait complete. 2023-09-30T22:29:24.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:24.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 Processor -- start 2023-09-30T22:29:24.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 -- start start 2023-09-30T22:29:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b9c136820 0x7f9b9c138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c139140 0x7f9b9c13b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c06bc00 con 0x7f9b9c133a80 2023-09-30T22:29:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c06c570 con 0x7f9b9c136820 2023-09-30T22:29:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.466+0000 7f9ba0f67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0a0040 con 0x7f9b9c139140 2023-09-30T22:29:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b99d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b9c136820 0x7f9b9c138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b9c133a80 0x7f9b9c133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54978/0 (socket says 172.21.15.32:54978) 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c139140 0x7f9b9c13b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9a59c700 1 -- 172.21.15.32:0/3453650280 learned_addr learned my addr 172.21.15.32:0/3453650280 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b99d9b700 1 -- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c139140 msgr2=0x7f9b9c13b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b99d9b700 1 --2- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c139140 0x7f9b9c13b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b99d9b700 1 -- 172.21.15.32:0/3453650280 >> v1:172.21.15.32:6789/0 conn(0x7f9b9c133a80 legacy=0x7f9b9c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b99d9b700 1 -- 172.21.15.32:0/3453650280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c133860 con 0x7f9b9c136820 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9959a700 1 -- 172.21.15.32:0/3453650280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569678127 0 0) 0x7f9b9c06bc00 con 0x7f9b9c133a80 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9ad9d700 1 --2- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c139140 0x7f9b9c13b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b99d9b700 1 --2- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c136820 0x7f9b9c138c00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009840 tx=0x7f9b90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3608ffb3b973c0c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9959a700 1 -- 172.21.15.32:0/3453650280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90011830 con 0x7f9b9c136820 2023-09-30T22:29:24.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9959a700 1 -- 172.21.15.32:0/3453650280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90011990 con 0x7f9b9c136820 2023-09-30T22:29:24.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.467+0000 7f9b9959a700 1 -- 172.21.15.32:0/3453650280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90011d20 con 0x7f9b9c136820 2023-09-30T22:29:24.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 -- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c136820 msgr2=0x7f9b9c138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c136820 0x7f9b9c138c00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009840 tx=0x7f9b90009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:24.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 -- 172.21.15.32:0/3453650280 shutdown_connections 2023-09-30T22:29:24.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c139140 0x7f9b9c13b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/3453650280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c136820 0x7f9b9c138c00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 -- 172.21.15.32:0/3453650280 >> 172.21.15.32:0/3453650280 conn(0x7f9b9c0a4210 msgr2=0x7f9b9c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 -- 172.21.15.32:0/3453650280 shutdown_connections 2023-09-30T22:29:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.468+0000 7f9ba0f67700 1 -- 172.21.15.32:0/3453650280 wait complete. 2023-09-30T22:29:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.469+0000 7f9ba0f67700 1 Processor -- start 2023-09-30T22:29:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.469+0000 7f9ba0f67700 1 -- start start 2023-09-30T22:29:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9ba0f67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9c133a80 0x7f9b9c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9ba0f67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c136820 0x7f9b9c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9ba0f67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 0x7f9b9c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9ba0f67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c070bd0 con 0x7f9b9c139140 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9ba0f67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c136530 con 0x7f9b9c136820 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9ba0f67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c071f50 con 0x7f9b9c133a80 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b9ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 0x7f9b9c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c136820 0x7f9b9c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b9ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 0x7f9b9c0703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56794/0 (socket says 172.21.15.32:56794) 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b9ad9d700 1 -- 172.21.15.32:0/2746909614 learned_addr learned my addr 172.21.15.32:0/2746909614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b9a59c700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9c133a80 0x7f9b9c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 -- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9c133a80 msgr2=0x7f9b9c06f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9c133a80 0x7f9b9c06f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 -- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 msgr2=0x7f9b9c0703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 0x7f9b9c0703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 -- 172.21.15.32:0/2746909614 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94009a70 con 0x7f9b9c136820 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b9ad9d700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 0x7f9b9c0703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.470+0000 7f9b99d9b700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c136820 0x7f9b9c06fe90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9b90016000 tx=0x7f9b90004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.471+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900036a0 con 0x7f9b9c136820 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.471+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9001f070 con 0x7f9b9c136820 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.471+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b900094a0 con 0x7f9b9c136820 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.471+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90003a40 con 0x7f9b9c136820 2023-09-30T22:29:24.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.471+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c1300a0 con 0x7f9b9c136820 2023-09-30T22:29:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.472+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b90003ba0 con 0x7f9b9c136820 2023-09-30T22:29:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.472+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b90021070 con 0x7f9b9c136820 2023-09-30T22:29:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.472+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c072450 con 0x7f9b9c136820 2023-09-30T22:29:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.477+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b9002b080 con 0x7f9b9c136820 2023-09-30T22:29:24.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:24.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.629+0000 7f0b40c4b700 1 Processor -- start 2023-09-30T22:29:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.629+0000 7f0b40c4b700 1 -- start start 2023-09-30T22:29:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b40c4b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 0x7f0b3c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b40c4b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 0x7f0b3c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b40c4b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 0x7f0b3c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b40c4b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c06b990 con 0x7f0b3c133e30 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b40c4b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0a0590 con 0x7f0b3c134260 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b40c4b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c09fb80 con 0x7f0b3c133420 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b3a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 0x7f0b3c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b39d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 0x7f0b3c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b3ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 0x7f0b3c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b3a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 0x7f0b3c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50266/0 (socket says 172.21.15.32:50266) 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b39d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 0x7f0b3c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56810/0 (socket says 172.21.15.32:56810) 2023-09-30T22:29:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b3ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 0x7f0b3c1304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37172/0 (socket says 172.21.15.32:37172) 2023-09-30T22:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.630+0000 7f0b3a59c700 1 -- 172.21.15.32:0/3135779033 learned_addr learned my addr 172.21.15.32:0/3135779033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3ad9d700 1 -- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 msgr2=0x7f0b3c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3ad9d700 1 --2- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 0x7f0b3c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3ad9d700 1 -- 172.21.15.32:0/3135779033 >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 msgr2=0x7f0b3c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3ad9d700 1 --2- 172.21.15.32:0/3135779033 >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 0x7f0b3c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3ad9d700 1 -- 172.21.15.32:0/3135779033 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0a0030 con 0x7f0b3c134260 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3a59c700 1 --2- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 0x7f0b3c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b39d9b700 1 --2- 172.21.15.32:0/3135779033 >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 0x7f0b3c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3ad9d700 1 --2- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 0x7f0b3c1304d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009d90 tx=0x7f0b3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e483a4d16bcfad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3959a700 1 -- 172.21.15.32:0/3135779033 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400e070 con 0x7f0b3c134260 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3959a700 1 -- 172.21.15.32:0/3135779033 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400bb20 con 0x7f0b3c134260 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.631+0000 7f0b3959a700 1 -- 172.21.15.32:0/3135779033 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34012430 con 0x7f0b3c134260 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 -- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 msgr2=0x7f0b3c1304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 0x7f0b3c1304d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009d90 tx=0x7f0b3400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 -- 172.21.15.32:0/3135779033 shutdown_connections 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c134260 0x7f0b3c1304d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/3135779033 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133420 0x7f0b3c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/3135779033 >> v2:172.21.15.32:3300/0 conn(0x7f0b3c133e30 0x7f0b3c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 -- 172.21.15.32:0/3135779033 >> 172.21.15.32:0/3135779033 conn(0x7f0b3c0a4210 msgr2=0x7f0b3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 -- 172.21.15.32:0/3135779033 shutdown_connections 2023-09-30T22:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.632+0000 7f0b40c4b700 1 -- 172.21.15.32:0/3135779033 wait complete. 2023-09-30T22:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.633+0000 7f0b40c4b700 1 Processor -- start 2023-09-30T22:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.633+0000 7f0b40c4b700 1 -- start start 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b40c4b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 0x7f0b3c0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b40c4b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133e30 0x7f0b3c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b40c4b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 0x7f0b3c071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b40c4b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c11fe90 con 0x7f0b3c134260 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b40c4b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c120a50 con 0x7f0b3c133420 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b40c4b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c072160 con 0x7f0b3c133e30 2023-09-30T22:29:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 0x7f0b3c071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 0x7f0b3c0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 0x7f0b3c071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56818/0 (socket says 172.21.15.32:56818) 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 0x7f0b3c0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37178/0 (socket says 172.21.15.32:37178) 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3ad9d700 1 -- 172.21.15.32:0/1191649117 learned_addr learned my addr 172.21.15.32:0/1191649117 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b39d9b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133e30 0x7f0b3c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b39d9b700 1 -- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 msgr2=0x7f0b3c0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b39d9b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 0x7f0b3c0709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b39d9b700 1 -- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 msgr2=0x7f0b3c071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b39d9b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 0x7f0b3c071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b39d9b700 1 -- 172.21.15.32:0/1191649117 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c009550 con 0x7f0b3c133e30 2023-09-30T22:29:24.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3ad9d700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 0x7f0b3c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.634+0000 7f0b3a59c700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 0x7f0b3c0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.635+0000 7f0b39d9b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133e30 0x7f0b3c070ee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000e9b0 tx=0x7f0b3000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.635+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000ee50 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.635+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000c850 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.635+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b300094a0 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.635+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000cbe0 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.635+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b34009a70 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.636+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b3000cd40 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.636+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b3003f5c0 con 0x7f0b3c133e30 2023-09-30T22:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.636+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c131460 con 0x7f0b3c133e30 2023-09-30T22:29:24.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.641+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b300232f0 con 0x7f0b3c133e30 2023-09-30T22:29:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.671+0000 7f411ce1c700 1 Processor -- start 2023-09-30T22:29:24.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.671+0000 7f411ce1c700 1 -- start start 2023-09-30T22:29:24.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411ce1c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f411809e930 0x7f411809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411ce1c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411ce1c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4118070940 con 0x7f411809f650 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411ce1c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411806fd60 con 0x7f411809e930 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411ce1c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f411806fa80 con 0x7f411809f240 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f411809e930 0x7f411809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f4116d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f411809f650 0x7f4118131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54994/0 (socket says 172.21.15.32:54994) 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f4116d9d700 1 -- 172.21.15.32:0/239735249 learned_addr learned my addr 172.21.15.32:0/239735249 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f4115d9b700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411659c700 1 -- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 msgr2=0x7f41181317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411659c700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411659c700 1 -- 172.21.15.32:0/239735249 >> v1:172.21.15.32:6789/0 conn(0x7f411809f650 legacy=0x7f4118131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411659c700 1 -- 172.21.15.32:0/239735249 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411809fb60 con 0x7f411809e930 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f4115d9b700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.672+0000 7f411659c700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809e930 0x7f411809ed00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f4108009840 tx=0x7f4108009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf1c9f4823f3dca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411559a700 1 -- 172.21.15.32:0/239735249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108011830 con 0x7f411809e930 2023-09-30T22:29:24.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411559a700 1 -- 172.21.15.32:0/239735249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108011990 con 0x7f411809e930 2023-09-30T22:29:24.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411559a700 1 -- 172.21.15.32:0/239735249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108011d20 con 0x7f411809e930 2023-09-30T22:29:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 -- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809e930 msgr2=0x7f411809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809e930 0x7f411809ed00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f4108009840 tx=0x7f4108009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 -- 172.21.15.32:0/239735249 shutdown_connections 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809e930 0x7f411809ed00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 --2- 172.21.15.32:0/239735249 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 -- 172.21.15.32:0/239735249 >> 172.21.15.32:0/239735249 conn(0x7f41180a4210 msgr2=0x7f41180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 -- 172.21.15.32:0/239735249 shutdown_connections 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.673+0000 7f411ce1c700 1 -- 172.21.15.32:0/239735249 wait complete. 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 Processor -- start 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 -- start start 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f411809e930 0x7f41181a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 0x7f41181c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181a7f50 con 0x7f411809f650 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41181b92d0 con 0x7f411809e930 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f411ce1c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41181a4e00 con 0x7f411809f240 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f4116d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 0x7f41181c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f4116d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 0x7f41181c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56830/0 (socket says 172.21.15.32:56830) 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.675+0000 7f4116d9d700 1 -- 172.21.15.32:0/3282470825 learned_addr learned my addr 172.21.15.32:0/3282470825 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4115d9b700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f411659c700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f411809e930 0x7f41181a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4116d9d700 1 -- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 msgr2=0x7f41181a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4116d9d700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4116d9d700 1 -- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f411809e930 msgr2=0x7f41181a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4116d9d700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f411809e930 0x7f41181a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4116d9d700 1 -- 172.21.15.32:0/3282470825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0094a0 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4116d9d700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 0x7f41181c81a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f411000cbf0 tx=0x7f411000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000ab10 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000ac70 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110012400 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41080094a0 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.676+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41180a0190 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.677+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4110012560 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.677+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f411003f2e0 con 0x7f411809f650 2023-09-30T22:29:24.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.678+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180a3380 con 0x7f411809f650 2023-09-30T22:29:24.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.687+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f411001b3b0 con 0x7f411809f650 2023-09-30T22:29:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.702+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9b9c074ea0 con 0x7f9b9c136820 2023-09-30T22:29:24.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.703+0000 7f9b8affd700 1 -- 172.21.15.32:0/2746909614 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b90015020 con 0x7f9b9c136820 2023-09-30T22:29:24.703 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:24.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c136820 msgr2=0x7f9b9c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c136820 0x7f9b9c06fe90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9b90016000 tx=0x7f9b90004750 comp rx=0 tx=0).stop 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 shutdown_connections 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b9c139140 0x7f9b9c0703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b9c133a80 0x7f9b9c06f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 --2- 172.21.15.32:0/2746909614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b9c136820 0x7f9b9c06fe90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 >> 172.21.15.32:0/2746909614 conn(0x7f9b9c0a4210 msgr2=0x7f9b9c0a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 shutdown_connections 2023-09-30T22:29:24.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.705+0000 7f9ba0f67700 1 -- 172.21.15.32:0/2746909614 wait complete. 2023-09-30T22:29:24.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.855+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0b3c11fcf0 con 0x7f0b3c133e30 2023-09-30T22:29:24.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.856+0000 7f0b2affd700 1 -- 172.21.15.32:0/1191649117 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b3c11fcf0 con 0x7f0b3c133e30 2023-09-30T22:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:24.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133e30 msgr2=0x7f0b3c070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133e30 0x7f0b3c070ee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000e9b0 tx=0x7f0b3000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 shutdown_connections 2023-09-30T22:29:24.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b3c134260 0x7f0b3c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b3c133420 0x7f0b3c0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 --2- 172.21.15.32:0/1191649117 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b3c133e30 0x7f0b3c070ee0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 >> 172.21.15.32:0/1191649117 conn(0x7f0b3c0a4210 msgr2=0x7f0b3c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 shutdown_connections 2023-09-30T22:29:24.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.858+0000 7f0b40c4b700 1 -- 172.21.15.32:0/1191649117 wait complete. 2023-09-30T22:29:24.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.898+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f41181a5e20 con 0x7f411809f650 2023-09-30T22:29:24.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.898+0000 7f4106ffd700 1 -- 172.21.15.32:0/3282470825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4110010300 con 0x7f411809f650 2023-09-30T22:29:24.900 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 msgr2=0x7f41181c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 0x7f41181c81a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f411000cbf0 tx=0x7f411000cf00 comp rx=0 tx=0).stop 2023-09-30T22:29:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 shutdown_connections 2023-09-30T22:29:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f411809f650 0x7f41181c81a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f411809e930 0x7f41181a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 --2- 172.21.15.32:0/3282470825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f411809f240 0x7f41181a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 >> 172.21.15.32:0/3282470825 conn(0x7f41180a4210 msgr2=0x7f411806c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 shutdown_connections 2023-09-30T22:29:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:24.901+0000 7f411ce1c700 1 -- 172.21.15.32:0/3282470825 wait complete. 2023-09-30T22:29:24.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:25.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:25.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.583+0000 7f1a8bc06700 1 Processor -- start 2023-09-30T22:29:25.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.583+0000 7f1a8bc06700 1 -- start start 2023-09-30T22:29:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8bc06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409d6f0 0x7f1a8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8bc06700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a8409e000 0x7f1a84130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8bc06700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8409f130 con 0x7f1a8409e410 2023-09-30T22:29:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8bc06700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8409f520 con 0x7f1a8409e000 2023-09-30T22:29:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8bc06700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8409e930 con 0x7f1a8409d6f0 2023-09-30T22:29:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a891a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1a8409e000 0x7f1a84130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a899a2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409d6f0 0x7f1a8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8a1a3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1a8409e410 0x7f1a84130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54998/0 (socket says 172.21.15.32:54998) 2023-09-30T22:29:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a8a1a3700 1 -- 172.21.15.32:0/2856567278 learned_addr learned my addr 172.21.15.32:0/2856567278 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a899a2700 1 -- 172.21.15.32:0/2856567278 >> v2:172.21.15.32:3300/0 conn(0x7f1a8409e000 msgr2=0x7f1a84130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a899a2700 1 --2- 172.21.15.32:0/2856567278 >> v2:172.21.15.32:3300/0 conn(0x7f1a8409e000 0x7f1a84130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a899a2700 1 -- 172.21.15.32:0/2856567278 >> v1:172.21.15.32:6789/0 conn(0x7f1a8409e410 legacy=0x7f1a84130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a899a2700 1 -- 172.21.15.32:0/2856567278 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8409b2d0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a889a0700 1 -- 172.21.15.32:0/2856567278 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947333971 0 0) 0x7f1a8409f130 con 0x7f1a8409e410 2023-09-30T22:29:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.584+0000 7f1a891a1700 1 --2- 172.21.15.32:0/2856567278 >> v2:172.21.15.32:3300/0 conn(0x7f1a8409e000 0x7f1a84130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a899a2700 1 --2- 172.21.15.32:0/2856567278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409d6f0 0x7f1a8409dac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009840 tx=0x7f1a80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcae76e051460d20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a889a0700 1 -- 172.21.15.32:0/2856567278 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80011830 con 0x7f1a8409d6f0 2023-09-30T22:29:25.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a889a0700 1 -- 172.21.15.32:0/2856567278 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a80011990 con 0x7f1a8409d6f0 2023-09-30T22:29:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a889a0700 1 -- 172.21.15.32:0/2856567278 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80011d20 con 0x7f1a8409d6f0 2023-09-30T22:29:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2856567278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409d6f0 msgr2=0x7f1a8409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2856567278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409d6f0 0x7f1a8409dac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009840 tx=0x7f1a80009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2856567278 shutdown_connections 2023-09-30T22:29:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2856567278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409d6f0 0x7f1a8409dac0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2856567278 >> v2:172.21.15.32:3300/0 conn(0x7f1a8409e000 0x7f1a84130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2856567278 >> 172.21.15.32:0/2856567278 conn(0x7f1a84066f50 msgr2=0x7f1a8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2856567278 shutdown_connections 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.585+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2856567278 wait complete. 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.587+0000 7f1a8bc06700 1 Processor -- start 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.587+0000 7f1a8bc06700 1 -- start start 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.587+0000 7f1a8bc06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 0x7f1a8411de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8bc06700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a8409e000 0x7f1a8411e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8bc06700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409e410 0x7f1a8411e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8bc06700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8411f270 con 0x7f1a8409d6f0 2023-09-30T22:29:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8bc06700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84120000 con 0x7f1a8409e410 2023-09-30T22:29:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8bc06700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841210c0 con 0x7f1a8409e000 2023-09-30T22:29:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 0x7f1a8411de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 0x7f1a8411de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56856/0 (socket says 172.21.15.32:56856) 2023-09-30T22:29:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 -- 172.21.15.32:0/2048202063 learned_addr learned my addr 172.21.15.32:0/2048202063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a891a1700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a8409e000 0x7f1a8411e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8a1a3700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409e410 0x7f1a8411e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 -- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a8409e000 msgr2=0x7f1a8411e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a8409e000 0x7f1a8411e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 -- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409e410 msgr2=0x7f1a8411e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409e410 0x7f1a8411e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 -- 172.21.15.32:0/2048202063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780094a0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8a1a3700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409e410 0x7f1a8411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a891a1700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a8409e000 0x7f1a8411e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a899a2700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 0x7f1a8411de10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009fd0 tx=0x7f1a80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800036a0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8001e070 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.588+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a800094a0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.589+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80003a60 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.589+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a840a2120 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.589+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a80003bc0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.589+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1a80040470 con 0x7f1a8409d6f0 2023-09-30T22:29:25.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.590+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a841214b0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.594+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a841214b0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:25.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.804+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1a841214b0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.805+0000 7f1a767fc700 1 -- 172.21.15.32:0/2048202063 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a841214b0 con 0x7f1a8409d6f0 2023-09-30T22:29:25.805 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:25.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 msgr2=0x7f1a8411de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 0x7f1a8411de10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009fd0 tx=0x7f1a80004750 comp rx=0 tx=0).stop 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 shutdown_connections 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1a8409e410 0x7f1a8411e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a8409d6f0 0x7f1a8411de10 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 --2- 172.21.15.32:0/2048202063 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1a8409e000 0x7f1a8411e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 >> 172.21.15.32:0/2048202063 conn(0x7f1a84066f50 msgr2=0x7f1a84130a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 shutdown_connections 2023-09-30T22:29:25.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.807+0000 7f1a8bc06700 1 -- 172.21.15.32:0/2048202063 wait complete. 2023-09-30T22:29:25.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:25.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.861+0000 7f71ffa7e700 1 Processor -- start 2023-09-30T22:29:25.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.861+0000 7f71ffa7e700 1 -- start start 2023-09-30T22:29:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71ffa7e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 0x7f71f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71ffa7e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f809e000 0x7f71f8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71ffa7e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f809f130 con 0x7f71f809e410 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71ffa7e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f809f520 con 0x7f71f809e000 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71ffa7e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71f809e930 con 0x7f71f809d6f0 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 0x7f71f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd019700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f809e000 0x7f71f8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 0x7f71f809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50314/0 (socket says 172.21.15.32:50314) 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fe01b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71f809e410 0x7f71f8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55014/0 (socket says 172.21.15.32:55014) 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 -- 172.21.15.32:0/3891362284 learned_addr learned my addr 172.21.15.32:0/3891362284 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 -- 172.21.15.32:0/3891362284 >> v2:172.21.15.32:3300/0 conn(0x7f71f809e000 msgr2=0x7f71f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 --2- 172.21.15.32:0/3891362284 >> v2:172.21.15.32:3300/0 conn(0x7f71f809e000 0x7f71f8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 -- 172.21.15.32:0/3891362284 >> v1:172.21.15.32:6789/0 conn(0x7f71f809e410 legacy=0x7f71f8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 -- 172.21.15.32:0/3891362284 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f809b2d0 con 0x7f71f809d6f0 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd019700 1 --2- 172.21.15.32:0/3891362284 >> v2:172.21.15.32:3300/0 conn(0x7f71f809e000 0x7f71f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fc818700 1 -- 172.21.15.32:0/3891362284 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119199507 0 0) 0x7f71f809f130 con 0x7f71f809e410 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fd81a700 1 --2- 172.21.15.32:0/3891362284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 0x7f71f809dac0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f71f40098f0 tx=0x7f71f4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ec583e157af7ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.862+0000 7f71fc818700 1 -- 172.21.15.32:0/3891362284 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4011830 con 0x7f71f809d6f0 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71fc818700 1 -- 172.21.15.32:0/3891362284 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4011990 con 0x7f71f809d6f0 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71fc818700 1 -- 172.21.15.32:0/3891362284 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4011d20 con 0x7f71f809d6f0 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 -- 172.21.15.32:0/3891362284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 msgr2=0x7f71f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/3891362284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 0x7f71f809dac0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f71f40098f0 tx=0x7f71f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 -- 172.21.15.32:0/3891362284 shutdown_connections 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/3891362284 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809d6f0 0x7f71f809dac0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/3891362284 >> v2:172.21.15.32:3300/0 conn(0x7f71f809e000 0x7f71f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 -- 172.21.15.32:0/3891362284 >> 172.21.15.32:0/3891362284 conn(0x7f71f8066f50 msgr2=0x7f71f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 -- 172.21.15.32:0/3891362284 shutdown_connections 2023-09-30T22:29:25.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.863+0000 7f71ffa7e700 1 -- 172.21.15.32:0/3891362284 wait complete. 2023-09-30T22:29:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.864+0000 7f71ffa7e700 1 Processor -- start 2023-09-30T22:29:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.864+0000 7f71ffa7e700 1 -- start start 2023-09-30T22:29:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ffa7e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f809d6f0 0x7f71f811df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ffa7e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809e000 0x7f71f811e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ffa7e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71f809e410 0x7f71f811ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ffa7e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8121aa0 con 0x7f71f809d6f0 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ffa7e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81201b0 con 0x7f71f809e410 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ffa7e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71f809c700 con 0x7f71f809e000 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd019700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809e000 0x7f71f811e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f809d6f0 0x7f71f811df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd019700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809e000 0x7f71f811e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50320/0 (socket says 172.21.15.32:50320) 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd019700 1 -- 172.21.15.32:0/145517475 learned_addr learned my addr 172.21.15.32:0/145517475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fe01b700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71f809e410 0x7f71f811ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 -- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809e000 msgr2=0x7f71f811e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809e000 0x7f71f811e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 -- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71f809e410 msgr2=0x7f71f811ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71f809e410 0x7f71f811ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 -- 172.21.15.32:0/145517475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e800b740 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71fd81a700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f809d6f0 0x7f71f811df80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f71f4016000 tx=0x7f71f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.865+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4004550 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.866+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f401f070 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.866+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f4009550 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.866+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40038e0 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.866+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f8130360 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.866+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71f4003a40 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.866+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f71f4021070 con 0x7f71f809d6f0 2023-09-30T22:29:25.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.867+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f8074ea0 con 0x7f71f809d6f0 2023-09-30T22:29:25.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:25.871+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71f402b080 con 0x7f71f809d6f0 2023-09-30T22:29:25.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:25.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.015+0000 7f54efc31700 1 Processor -- start 2023-09-30T22:29:26.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54efc31700 1 -- start start 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54efc31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f54e80a1510 0x7f54e8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54efc31700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54efc31700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e809edd0 con 0x7f54e80a0a30 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54efc31700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e80a18e0 con 0x7f54e80a1510 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54efc31700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e809e1f0 con 0x7f54e8139250 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54ee1ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54ee1ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37210/0 (socket says 172.21.15.32:37210) 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54ee1ce700 1 -- 172.21.15.32:0/3164265712 learned_addr learned my addr 172.21.15.32:0/3164265712 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.016+0000 7f54ed1cc700 1 --2- 172.21.15.32:0/3164265712 >> v2:172.21.15.32:3300/0 conn(0x7f54e80a1510 0x7f54e8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ee1ce700 1 -- 172.21.15.32:0/3164265712 >> v2:172.21.15.32:3300/0 conn(0x7f54e80a1510 msgr2=0x7f54e8138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ee1ce700 1 --2- 172.21.15.32:0/3164265712 >> v2:172.21.15.32:3300/0 conn(0x7f54e80a1510 0x7f54e8138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ee1ce700 1 -- 172.21.15.32:0/3164265712 >> v1:172.21.15.32:6789/0 conn(0x7f54e80a0a30 legacy=0x7f54e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ee1ce700 1 -- 172.21.15.32:0/3164265712 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e809dc50 con 0x7f54e8139250 2023-09-30T22:29:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ed1cc700 1 --2- 172.21.15.32:0/3164265712 >> v2:172.21.15.32:3300/0 conn(0x7f54e80a1510 0x7f54e8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:26.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ee1ce700 1 --2- 172.21.15.32:0/3164265712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e813b630 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f54e00098f0 tx=0x7f54e0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5b85d451f1d349 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ec9cb700 1 -- 172.21.15.32:0/3164265712 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0011830 con 0x7f54e8139250 2023-09-30T22:29:26.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ec9cb700 1 -- 172.21.15.32:0/3164265712 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e0011990 con 0x7f54e8139250 2023-09-30T22:29:26.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.017+0000 7f54ec9cb700 1 -- 172.21.15.32:0/3164265712 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0011d20 con 0x7f54e8139250 2023-09-30T22:29:26.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 -- 172.21.15.32:0/3164265712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 msgr2=0x7f54e813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 --2- 172.21.15.32:0/3164265712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e813b630 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f54e00098f0 tx=0x7f54e0009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 -- 172.21.15.32:0/3164265712 shutdown_connections 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 --2- 172.21.15.32:0/3164265712 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e813b630 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 --2- 172.21.15.32:0/3164265712 >> v2:172.21.15.32:3300/0 conn(0x7f54e80a1510 0x7f54e8138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 -- 172.21.15.32:0/3164265712 >> 172.21.15.32:0/3164265712 conn(0x7f54e80a6550 msgr2=0x7f54e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 -- 172.21.15.32:0/3164265712 shutdown_connections 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.018+0000 7f54efc31700 1 -- 172.21.15.32:0/3164265712 wait complete. 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.019+0000 7f54efc31700 1 Processor -- start 2023-09-30T22:29:26.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.019+0000 7f54efc31700 1 -- start start 2023-09-30T22:29:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54efc31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54e80a0a30 0x7f54e81c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54efc31700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 0x7f54e81c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54efc31700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e81c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54efc31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e8124820 con 0x7f54e80a0a30 2023-09-30T22:29:26.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54efc31700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e8125d40 con 0x7f54e8139250 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54efc31700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54e81253a0 con 0x7f54e80a1510 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 0x7f54e81c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 0x7f54e81c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50326/0 (socket says 172.21.15.32:50326) 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 -- 172.21.15.32:0/3349301940 learned_addr learned my addr 172.21.15.32:0/3349301940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ee1ce700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e81c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed9cd700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54e80a0a30 0x7f54e81c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 -- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 msgr2=0x7f54e81c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e81c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 -- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54e80a0a30 msgr2=0x7f54e81c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54e80a0a30 0x7f54e81c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.020+0000 7f54ed1cc700 1 -- 172.21.15.32:0/3349301940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d80094a0 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.021+0000 7f54ed1cc700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 0x7f54e81c8a80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f54d800ebe0 tx=0x7f54d800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.021+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8009800 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.021+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d800c910 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.021+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e0009550 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.021+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d800cc80 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.021+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e81262d0 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.022+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f54d8013470 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.022+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e81264c0 con 0x7f54e80a1510 2023-09-30T22:29:26.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.022+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f54d803f970 con 0x7f54e80a1510 2023-09-30T22:29:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.027+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54d8013720 con 0x7f54e80a1510 2023-09-30T22:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 Processor -- start 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 -- start start 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 0x7f5af80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5af809d630 0x7f5af80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 0x7f5af8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80a2a70 con 0x7f5af809d630 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80a01d0 con 0x7f5af80a0a30 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.058+0000 7f5afe9e9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af809e2b0 con 0x7f5af8133930 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5afcf86700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 0x7f5af8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 0x7f5af80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5af809d630 0x7f5af80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5afcf86700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 0x7f5af8133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50332/0 (socket says 172.21.15.32:50332) 2023-09-30T22:29:26.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 0x7f5af80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37236/0 (socket says 172.21.15.32:37236) 2023-09-30T22:29:26.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5afcf86700 1 -- 172.21.15.32:0/3081361722 learned_addr learned my addr 172.21.15.32:0/3081361722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:26.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 -- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 msgr2=0x7f5af8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 0x7f5af8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 -- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 msgr2=0x7f5af80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 0x7f5af80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 -- 172.21.15.32:0/3081361722 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af809dc50 con 0x7f5af809d630 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af7fff700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 0x7f5af80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5afcf86700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 0x7f5af8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af77fe700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809d630 0x7f5af80a1340 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009840 tx=0x7f5aec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e22e15b8f689fc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.059+0000 7f5af6ffd700 1 -- 172.21.15.32:0/3081361722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec011830 con 0x7f5af809d630 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5af6ffd700 1 -- 172.21.15.32:0/3081361722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec011990 con 0x7f5af809d630 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5af6ffd700 1 -- 172.21.15.32:0/3081361722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec011d20 con 0x7f5af809d630 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 -- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809d630 msgr2=0x7f5af80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809d630 0x7f5af80a1340 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009840 tx=0x7f5aec009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 -- 172.21.15.32:0/3081361722 shutdown_connections 2023-09-30T22:29:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af80a0a30 0x7f5af80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af8133930 0x7f5af8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/3081361722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af809d630 0x7f5af80a1340 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 -- 172.21.15.32:0/3081361722 >> 172.21.15.32:0/3081361722 conn(0x7f5af80a3050 msgr2=0x7f5af80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 -- 172.21.15.32:0/3081361722 shutdown_connections 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.060+0000 7f5afe9e9700 1 -- 172.21.15.32:0/3081361722 wait complete. 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.061+0000 7f5afe9e9700 1 Processor -- start 2023-09-30T22:29:26.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.061+0000 7f5afe9e9700 1 -- start start 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afe9e9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809d630 0x7f5af81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afe9e9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 0x7f5af81c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afe9e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af8133930 0x7f5af81c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afe9e9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8120820 con 0x7f5af8133930 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afe9e9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8122010 con 0x7f5af809d630 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afe9e9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80a40c0 con 0x7f5af80a0a30 2023-09-30T22:29:26.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809d630 0x7f5af81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 0x7f5af81c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 0x7f5af81c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50344/0 (socket says 172.21.15.32:50344) 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809d630 0x7f5af81c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37242/0 (socket says 172.21.15.32:37242) 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af77fe700 1 -- 172.21.15.32:0/515485769 learned_addr learned my addr 172.21.15.32:0/515485769 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5afcf86700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af8133930 0x7f5af81c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af77fe700 1 -- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809d630 msgr2=0x7f5af81c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.062+0000 7f5af77fe700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809d630 0x7f5af81c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af77fe700 1 -- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af8133930 msgr2=0x7f5af81c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af77fe700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af8133930 0x7f5af81c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af77fe700 1 -- 172.21.15.32:0/515485769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80094a0 con 0x7f5af80a0a30 2023-09-30T22:29:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af77fe700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 0x7f5af81c85d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0090b0 tx=0x7f5aec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec004450 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec01f070 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af0009b30 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0038e0 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.063+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec0094a0 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.064+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5aec003a40 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.064+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5aec021070 con 0x7f5af80a0a30 2023-09-30T22:29:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.065+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad80052e0 con 0x7f5af80a0a30 2023-09-30T22:29:26.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.074+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5aec026480 con 0x7f5af80a0a30 2023-09-30T22:29:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.089+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f71f8120de0 con 0x7f71f809d6f0 2023-09-30T22:29:26.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.089+0000 7f71ee7fc700 1 -- 172.21.15.32:0/145517475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71f4036090 con 0x7f71f809d6f0 2023-09-30T22:29:26.089 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f809d6f0 msgr2=0x7f71f811df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f809d6f0 0x7f71f811df80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f71f4016000 tx=0x7f71f4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 shutdown_connections 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71f809e410 0x7f71f811ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f809d6f0 0x7f71f811df80 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 --2- 172.21.15.32:0/145517475 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f71f809e000 0x7f71f811e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 >> 172.21.15.32:0/145517475 conn(0x7f71f8066f50 msgr2=0x7f71f8070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 shutdown_connections 2023-09-30T22:29:26.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.093+0000 7f71ffa7e700 1 -- 172.21.15.32:0/145517475 wait complete. 2023-09-30T22:29:26.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.244+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f54e80a1d10 con 0x7f54e80a1510 2023-09-30T22:29:26.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.245+0000 7f54de7fc700 1 -- 172.21.15.32:0/3349301940 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f54d801bb60 con 0x7f54e80a1510 2023-09-30T22:29:26.245 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:26.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 msgr2=0x7f54e81c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 0x7f54e81c8a80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f54d800ebe0 tx=0x7f54d800eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 shutdown_connections 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54e8139250 0x7f54e81c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54e80a0a30 0x7f54e81c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 --2- 172.21.15.32:0/3349301940 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f54e80a1510 0x7f54e81c8a80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 >> 172.21.15.32:0/3349301940 conn(0x7f54e80a6550 msgr2=0x7f54e80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 shutdown_connections 2023-09-30T22:29:26.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.247+0000 7f54efc31700 1 -- 172.21.15.32:0/3349301940 wait complete. 2023-09-30T22:29:26.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.283+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5ad8005590 con 0x7f5af80a0a30 2023-09-30T22:29:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.283+0000 7f5af4ff9700 1 -- 172.21.15.32:0/515485769 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5aec019ba0 con 0x7f5af80a0a30 2023-09-30T22:29:26.284 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 msgr2=0x7f5af81c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 0x7f5af81c85d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0090b0 tx=0x7f5aec004750 comp rx=0 tx=0).stop 2023-09-30T22:29:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 shutdown_connections 2023-09-30T22:29:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5af8133930 0x7f5af81c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5af809d630 0x7f5af81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 --2- 172.21.15.32:0/515485769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5af80a0a30 0x7f5af81c85d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 >> 172.21.15.32:0/515485769 conn(0x7f5af80a3050 msgr2=0x7f5af809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 shutdown_connections 2023-09-30T22:29:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.286+0000 7f5afe9e9700 1 -- 172.21.15.32:0/515485769 wait complete. 2023-09-30T22:29:26.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:26.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:26.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.970+0000 7fd7094b4700 1 Processor -- start 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.970+0000 7fd7094b4700 1 -- start start 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.970+0000 7fd7094b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 0x7fd7040a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.970+0000 7fd7094b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 0x7fd7040a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.970+0000 7fd7094b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd704133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7094b4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7040a1810 con 0x7fd7040a09c0 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7094b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70409ea10 con 0x7fd70409d5c0 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7094b4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd70409e210 con 0x7fd704133920 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 0x7fd7040a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd702ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 0x7fd7040a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd704133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd704133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50356/0 (socket says 172.21.15.32:50356) 2023-09-30T22:29:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd702ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 0x7fd7040a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56904/0 (socket says 172.21.15.32:56904) 2023-09-30T22:29:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 0x7fd7040a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37254/0 (socket says 172.21.15.32:37254) 2023-09-30T22:29:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7037fe700 1 -- 172.21.15.32:0/1296608968 learned_addr learned my addr 172.21.15.32:0/1296608968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 -- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 msgr2=0x7fd704133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 --2- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd704133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 -- 172.21.15.32:0/1296608968 >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 msgr2=0x7fd7040a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 --2- 172.21.15.32:0/1296608968 >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 0x7fd7040a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 -- 172.21.15.32:0/1296608968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7040a26f0 con 0x7fd70409d5c0 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd702ffd700 1 --2- 172.21.15.32:0/1296608968 >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 0x7fd7040a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd7027fc700 1 --2- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 0x7fd7040a12d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009840 tx=0x7fd6f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=413f93f46517ea0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd701ffb700 1 -- 172.21.15.32:0/1296608968 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011830 con 0x7fd70409d5c0 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.971+0000 7fd701ffb700 1 -- 172.21.15.32:0/1296608968 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8011990 con 0x7fd70409d5c0 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd701ffb700 1 -- 172.21.15.32:0/1296608968 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011d20 con 0x7fd70409d5c0 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 -- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 msgr2=0x7fd7040a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 --2- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 0x7fd7040a12d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009840 tx=0x7fd6f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 -- 172.21.15.32:0/1296608968 shutdown_connections 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 --2- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd704133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 --2- 172.21.15.32:0/1296608968 >> v2:172.21.15.32:3300/0 conn(0x7fd7040a09c0 0x7fd7040a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 --2- 172.21.15.32:0/1296608968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd70409d5c0 0x7fd7040a12d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 -- 172.21.15.32:0/1296608968 >> 172.21.15.32:0/1296608968 conn(0x7fd7040a2fe0 msgr2=0x7fd7040a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 -- 172.21.15.32:0/1296608968 shutdown_connections 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.972+0000 7fd7094b4700 1 -- 172.21.15.32:0/1296608968 wait complete. 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.973+0000 7fd7094b4700 1 Processor -- start 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.973+0000 7fd7094b4700 1 -- start start 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7094b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 0x7fd7041c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7094b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7040a09c0 0x7fd7041c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7094b4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd7041c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7094b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040a59c0 con 0x7fd70409d5c0 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7094b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70409cec0 con 0x7fd7040a09c0 2023-09-30T22:29:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7094b4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd704120d70 con 0x7fd704133920 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 0x7fd7041c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7027fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7040a09c0 0x7fd7041c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 0x7fd7041c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56908/0 (socket says 172.21.15.32:56908) 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 -- 172.21.15.32:0/3626241545 learned_addr learned my addr 172.21.15.32:0/3626241545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd7037fe700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd7041c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 -- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 msgr2=0x7fd7041c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd7041c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 -- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7040a09c0 msgr2=0x7fd7041c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7040a09c0 0x7fd7041c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 -- 172.21.15.32:0/3626241545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40094a0 con 0x7fd70409d5c0 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.974+0000 7fd702ffd700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 0x7fd7041c8290 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400ebe0 tx=0x7fd6f400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.975+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400c850 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.975+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400c9b0 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.975+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80094a0 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.975+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400cd40 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.975+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7041207d0 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.975+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6f4013470 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.976+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd6f403f770 con 0x7fd70409d5c0 2023-09-30T22:29:26.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.976+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd704122110 con 0x7fd70409d5c0 2023-09-30T22:29:26.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:26.981+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6f4013720 con 0x7fd70409d5c0 2023-09-30T22:29:27.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:27.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.191+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd704122310 con 0x7fd70409d5c0 2023-09-30T22:29:27.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.192+0000 7fd6eb7fe700 1 -- 172.21.15.32:0/3626241545 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6f4021030 con 0x7fd70409d5c0 2023-09-30T22:29:27.192 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 msgr2=0x7fd7041c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 0x7fd7041c8290 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400ebe0 tx=0x7fd6f400eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 shutdown_connections 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd704133920 0x7fd7041c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd70409d5c0 0x7fd7041c8290 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 --2- 172.21.15.32:0/3626241545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7040a09c0 0x7fd7041c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 >> 172.21.15.32:0/3626241545 conn(0x7fd7040a2fe0 msgr2=0x7fd7041bbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 shutdown_connections 2023-09-30T22:29:27.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.194+0000 7fd7094b4700 1 -- 172.21.15.32:0/3626241545 wait complete. 2023-09-30T22:29:27.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:27.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 Processor -- start 2023-09-30T22:29:27.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 -- start start 2023-09-30T22:29:27.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe89809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe8981339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89809b120 con 0x7fe89809e410 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89806b460 con 0x7fe89809d6f0 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.248+0000 7fe89c8a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe89809a7e0 con 0x7fe89809e000 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe89809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe895d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe8981339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe89809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37274/0 (socket says 172.21.15.32:37274) 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe895d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe8981339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50366/0 (socket says 172.21.15.32:50366) 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 -- 172.21.15.32:0/2255279070 learned_addr learned my addr 172.21.15.32:0/2255279070 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:27.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 -- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 msgr2=0x7fe8981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 --2- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe8981339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89559a700 1 -- 172.21.15.32:0/2255279070 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834676091 0 0) 0x7fe89809b120 con 0x7fe89809e410 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 -- 172.21.15.32:0/2255279070 >> v1:172.21.15.32:6789/0 conn(0x7fe89809e410 legacy=0x7fe898133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 -- 172.21.15.32:0/2255279070 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89809a500 con 0x7fe89809d6f0 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe895d9b700 1 --2- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe8981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89659c700 1 --2- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe89809dac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe888009920 tx=0x7fe888009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9118b8d597f3ef6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89559a700 1 -- 172.21.15.32:0/2255279070 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888011830 con 0x7fe89809d6f0 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89559a700 1 -- 172.21.15.32:0/2255279070 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe888011990 con 0x7fe89809d6f0 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.249+0000 7fe89559a700 1 -- 172.21.15.32:0/2255279070 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888011d20 con 0x7fe89809d6f0 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 -- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 msgr2=0x7fe89809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe89809dac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe888009920 tx=0x7fe888009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:27.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 -- 172.21.15.32:0/2255279070 shutdown_connections 2023-09-30T22:29:27.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe89809dac0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/2255279070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe8981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 -- 172.21.15.32:0/2255279070 >> 172.21.15.32:0/2255279070 conn(0x7fe898066f50 msgr2=0x7fe89809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 -- 172.21.15.32:0/2255279070 shutdown_connections 2023-09-30T22:29:27.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.250+0000 7fe89c8a1700 1 -- 172.21.15.32:0/2255279070 wait complete. 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.251+0000 7fe89c8a1700 1 Processor -- start 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.251+0000 7fe89c8a1700 1 -- start start 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89c8a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe8981223c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89c8a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe898122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89c8a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89809e410 0x7fe898122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89c8a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981336f0 con 0x7fe89809e410 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89c8a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe898123820 con 0x7fe89809d6f0 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89c8a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe898124dc0 con 0x7fe89809e000 2023-09-30T22:29:27.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe898122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe89659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe8981223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe898122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50380/0 (socket says 172.21.15.32:50380) 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe896d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89809e410 0x7fe898122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 -- 172.21.15.32:0/1752033671 learned_addr learned my addr 172.21.15.32:0/1752033671 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 -- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 msgr2=0x7fe8981223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe8981223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 -- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89809e410 msgr2=0x7fe898122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89809e410 0x7fe898122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 -- 172.21.15.32:0/1752033671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0094a0 con 0x7fe89809e000 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.252+0000 7fe895d9b700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe898122900 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00ebe0 tx=0x7fe88c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.253+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00c850 con 0x7fe89809e000 2023-09-30T22:29:27.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.253+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c00c9b0 con 0x7fe89809e000 2023-09-30T22:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.253+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe888009550 con 0x7fe89809e000 2023-09-30T22:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.253+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00cd40 con 0x7fe89809e000 2023-09-30T22:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.253+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8980a2a10 con 0x7fe89809e000 2023-09-30T22:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.253+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe88c013470 con 0x7fe89809e000 2023-09-30T22:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.254+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe88c03f7a0 con 0x7fe89809e000 2023-09-30T22:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.254+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe898074ea0 con 0x7fe89809e000 2023-09-30T22:29:27.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.258+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe88c013720 con 0x7fe89809e000 2023-09-30T22:29:27.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:27.300 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:27.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 Processor -- start 2023-09-30T22:29:27.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 -- start start 2023-09-30T22:29:27.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ddc12ff20 0x7f3ddc130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 0x7f3ddc13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc06c570 con 0x7f3ddc12ff20 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc06bc00 con 0x7f3ddc133a80 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de247a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0a0040 con 0x7f3ddc130870 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.405+0000 7f3de0a17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 0x7f3ddc13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 0x7f3ddc13a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50386/0 (socket says 172.21.15.32:50386) 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 -- 172.21.15.32:0/891663331 learned_addr learned my addr 172.21.15.32:0/891663331 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddb7fe700 1 --2- 172.21.15.32:0/891663331 >> v2:172.21.15.32:3300/0 conn(0x7f3ddc12ff20 0x7f3ddc130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddbfff700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 -- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 msgr2=0x7f3ddc133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 -- 172.21.15.32:0/891663331 >> v2:172.21.15.32:3300/0 conn(0x7f3ddc12ff20 msgr2=0x7f3ddc130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 --2- 172.21.15.32:0/891663331 >> v2:172.21.15.32:3300/0 conn(0x7f3ddc12ff20 0x7f3ddc130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 -- 172.21.15.32:0/891663331 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc133860 con 0x7f3ddc130870 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddbfff700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddb7fe700 1 --2- 172.21.15.32:0/891663331 >> v2:172.21.15.32:3300/0 conn(0x7f3ddc12ff20 0x7f3ddc130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3de0a17700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 0x7f3ddc13a840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009e50 tx=0x7f3dd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26517e88474ed5ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddaffd700 1 -- 172.21.15.32:0/891663331 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400e070 con 0x7f3ddc130870 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddaffd700 1 -- 172.21.15.32:0/891663331 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400b910 con 0x7f3ddc130870 2023-09-30T22:29:27.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.406+0000 7f3ddaffd700 1 -- 172.21.15.32:0/891663331 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bcc0 con 0x7f3ddc130870 2023-09-30T22:29:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 -- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 msgr2=0x7f3ddc13a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 0x7f3ddc13a840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009e50 tx=0x7f3dd400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 -- 172.21.15.32:0/891663331 shutdown_connections 2023-09-30T22:29:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 --2- 172.21.15.32:0/891663331 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc130870 0x7f3ddc13a840 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 --2- 172.21.15.32:0/891663331 >> v2:172.21.15.32:3300/0 conn(0x7f3ddc12ff20 0x7f3ddc130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 -- 172.21.15.32:0/891663331 >> 172.21.15.32:0/891663331 conn(0x7f3ddc0a4210 msgr2=0x7f3ddc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 -- 172.21.15.32:0/891663331 shutdown_connections 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.407+0000 7f3de247a700 1 -- 172.21.15.32:0/891663331 wait complete. 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.408+0000 7f3de247a700 1 Processor -- start 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.408+0000 7f3de247a700 1 -- start start 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de247a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 0x7f3ddc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de247a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc130870 0x7f3ddc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de247a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de247a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc12d6e0 con 0x7f3ddc130870 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de247a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc09e0a0 con 0x7f3ddc133a80 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de247a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0a20a0 con 0x7f3ddc12ff20 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 0x7f3ddc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3de0a17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 0x7f3ddc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50388/0 (socket says 172.21.15.32:50388) 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3ddb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc130870 0x7f3ddc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.409+0000 7f3ddbfff700 1 -- 172.21.15.32:0/777387378 learned_addr learned my addr 172.21.15.32:0/777387378 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3ddbfff700 1 -- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 msgr2=0x7f3ddc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3ddbfff700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3ddbfff700 1 -- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc130870 msgr2=0x7f3ddc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3ddbfff700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc130870 0x7f3ddc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3ddbfff700 1 -- 172.21.15.32:0/777387378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00094a0 con 0x7f3ddc12ff20 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3de0a17700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3ddbfff700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 0x7f3ddc09c480 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00ebe0 tx=0x7f3dcc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00c850 con 0x7f3ddc12ff20 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc00c9b0 con 0x7f3ddc12ff20 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc0094a0 con 0x7f3ddc12ff20 2023-09-30T22:29:27.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00cd40 con 0x7f3ddc12ff20 2023-09-30T22:29:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.410+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd4009b30 con 0x7f3ddc12ff20 2023-09-30T22:29:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.411+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc1203d0 con 0x7f3ddc12ff20 2023-09-30T22:29:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.412+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3dcc013470 con 0x7f3ddc12ff20 2023-09-30T22:29:27.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.412+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3dcc03fed0 con 0x7f3ddc12ff20 2023-09-30T22:29:27.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.421+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3dcc013720 con 0x7f3ddc12ff20 2023-09-30T22:29:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.452+0000 7f40fd1d4700 1 Processor -- start 2023-09-30T22:29:27.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.452+0000 7f40fd1d4700 1 -- start start 2023-09-30T22:29:27.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40fd1d4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40fd1d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f8138d80 0x7f40f813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40fd1d4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f80a2a70 con 0x7f40f809d630 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40fd1d4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80a01d0 con 0x7f40f8138d80 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40fd1d4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40f809e2b0 con 0x7f40f80a0a30 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50390/0 (socket says 172.21.15.32:50390) 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 -- 172.21.15.32:0/2279369886 learned_addr learned my addr 172.21.15.32:0/2279369886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:27.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f759e700 1 --2- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f8138d80 0x7f40f813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 -- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f8138d80 msgr2=0x7f40f813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 --2- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f8138d80 0x7f40f813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 -- 172.21.15.32:0/2279369886 >> v1:172.21.15.32:6789/0 conn(0x7f40f809d630 legacy=0x7f40f80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 -- 172.21.15.32:0/2279369886 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f809dc50 con 0x7f40f80a0a30 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f5d9b700 1 -- 172.21.15.32:0/2279369886 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309303443 0 0) 0x7f40f80a2a70 con 0x7f40f809d630 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.453+0000 7f40f6d9d700 1 --2- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f80a0e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009920 tx=0x7f40e8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=792707a2fe3262a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40f5d9b700 1 -- 172.21.15.32:0/2279369886 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8011830 con 0x7f40f80a0a30 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40f5d9b700 1 -- 172.21.15.32:0/2279369886 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e8011990 con 0x7f40f80a0a30 2023-09-30T22:29:27.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40f5d9b700 1 -- 172.21.15.32:0/2279369886 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8011d20 con 0x7f40f80a0a30 2023-09-30T22:29:27.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 -- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 msgr2=0x7f40f80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f80a0e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009920 tx=0x7f40e8009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 -- 172.21.15.32:0/2279369886 shutdown_connections 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f8138d80 0x7f40f813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/2279369886 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f80a0e00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 -- 172.21.15.32:0/2279369886 >> 172.21.15.32:0/2279369886 conn(0x7f40f80a3050 msgr2=0x7f40f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 -- 172.21.15.32:0/2279369886 shutdown_connections 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.454+0000 7f40fd1d4700 1 -- 172.21.15.32:0/2279369886 wait complete. 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.456+0000 7f40fd1d4700 1 Processor -- start 2023-09-30T22:29:27.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.456+0000 7f40fd1d4700 1 -- start start 2023-09-30T22:29:27.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40fd1d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 0x7f40f81337d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40fd1d4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f8133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40fd1d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8138d80 0x7f40f8134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40fd1d4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80a0410 con 0x7f40f8138d80 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40fd1d4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f8136cc0 con 0x7f40f809d630 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40fd1d4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80a4b00 con 0x7f40f80a0a30 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40f6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 0x7f40f81337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40f6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 0x7f40f81337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37320/0 (socket says 172.21.15.32:37320) 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40f759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8138d80 0x7f40f8134250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40f6d9d700 1 -- 172.21.15.32:0/1601957879 learned_addr learned my addr 172.21.15.32:0/1601957879 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:27.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.457+0000 7f40f659c700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f8133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f6d9d700 1 -- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 msgr2=0x7f40f8133d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f6d9d700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f8133d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f6d9d700 1 -- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8138d80 msgr2=0x7f40f8134250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f6d9d700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8138d80 0x7f40f8134250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f6d9d700 1 -- 172.21.15.32:0/1601957879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8009550 con 0x7f40f809d630 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f759e700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8138d80 0x7f40f8134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40f6d9d700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 0x7f40f81337d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f40e800a010 tx=0x7f40e8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004450 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e80045b0 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e80038e0 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f81369f0 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.458+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f8137960 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.459+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f40e8021070 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.460+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f80747f0 con 0x7f40f809d630 2023-09-30T22:29:27.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.460+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f40e8040ad0 con 0x7f40f809d630 2023-09-30T22:29:27.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.465+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f40e8024470 con 0x7f40f809d630 2023-09-30T22:29:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.471+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe89806c360 con 0x7fe89809e000 2023-09-30T22:29:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.472+0000 7fe886ffd700 1 -- 172.21.15.32:0/1752033671 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe88c02a090 con 0x7fe89809e000 2023-09-30T22:29:27.472 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 msgr2=0x7fe898122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe898122900 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00ebe0 tx=0x7fe88c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 shutdown_connections 2023-09-30T22:29:27.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe89809e410 0x7fe898122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe89809d6f0 0x7fe8981223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 --2- 172.21.15.32:0/1752033671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe89809e000 0x7fe898122900 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 >> 172.21.15.32:0/1752033671 conn(0x7fe898066f50 msgr2=0x7fe8980a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 shutdown_connections 2023-09-30T22:29:27.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.474+0000 7fe89c8a1700 1 -- 172.21.15.32:0/1752033671 wait complete. 2023-09-30T22:29:27.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.631+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ddc134360 con 0x7f3ddc12ff20 2023-09-30T22:29:27.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.631+0000 7f3dd8ff9700 1 -- 172.21.15.32:0/777387378 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3dcc017070 con 0x7f3ddc12ff20 2023-09-30T22:29:27.632 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 msgr2=0x7f3ddc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 0x7f3ddc09c480 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00ebe0 tx=0x7f3dcc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 shutdown_connections 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ddc133a80 0x7f3ddc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ddc12ff20 0x7f3ddc09c480 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 --2- 172.21.15.32:0/777387378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ddc130870 0x7f3ddc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 >> 172.21.15.32:0/777387378 conn(0x7f3ddc0a4210 msgr2=0x7f3ddc0a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 shutdown_connections 2023-09-30T22:29:27.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.634+0000 7f3de247a700 1 -- 172.21.15.32:0/777387378 wait complete. 2023-09-30T22:29:27.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:27.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.672+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f40f8137090 con 0x7f40f809d630 2023-09-30T22:29:27.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.673+0000 7f40df7fe700 1 -- 172.21.15.32:0/1601957879 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40e8029030 con 0x7f40f809d630 2023-09-30T22:29:27.673 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:27.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 msgr2=0x7f40f81337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 0x7f40f81337d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f40e800a010 tx=0x7f40e8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 shutdown_connections 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40f8138d80 0x7f40f8134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40f809d630 0x7f40f81337d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 --2- 172.21.15.32:0/1601957879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f40f80a0a30 0x7f40f8133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 >> 172.21.15.32:0/1601957879 conn(0x7f40f80a3050 msgr2=0x7f40f809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 shutdown_connections 2023-09-30T22:29:27.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:27.675+0000 7f40fd1d4700 1 -- 172.21.15.32:0/1601957879 wait complete. 2023-09-30T22:29:27.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:28.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.358+0000 7f0f58480700 1 Processor -- start 2023-09-30T22:29:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.358+0000 7f0f58480700 1 -- start start 2023-09-30T22:29:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.358+0000 7f0f58480700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f58480700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f50139170 0x7f0f5013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f58480700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5009b120 con 0x7f0f5009d6f0 2023-09-30T22:29:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f58480700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5006b460 con 0x7f0f5009e1d0 2023-09-30T22:29:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f58480700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5009a7e0 con 0x7f0f50139170 2023-09-30T22:29:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f55a1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f5621c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0f5009d6f0 0x7f0f5009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55042/0 (socket says 172.21.15.32:55042) 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f5621c700 1 -- 172.21.15.32:0/1656085185 learned_addr learned my addr 172.21.15.32:0/1656085185 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f56a1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f50139170 0x7f0f5013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f5521a700 1 -- 172.21.15.32:0/1656085185 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791399412 0 0) 0x7f0f5009b120 con 0x7f0f5009d6f0 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f55a1b700 1 -- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f50139170 msgr2=0x7f0f5013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f55a1b700 1 --2- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f50139170 0x7f0f5013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f55a1b700 1 -- 172.21.15.32:0/1656085185 >> v1:172.21.15.32:6789/0 conn(0x7f0f5009d6f0 legacy=0x7f0f5009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f55a1b700 1 -- 172.21.15.32:0/1656085185 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5009a500 con 0x7f0f5009e1d0 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f56a1d700 1 --2- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f50139170 0x7f0f5013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f55a1b700 1 --2- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50138c30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009870 tx=0x7f0f40009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06f833c70665519 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f5521a700 1 -- 172.21.15.32:0/1656085185 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40011830 con 0x7f0f5009e1d0 2023-09-30T22:29:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f5521a700 1 -- 172.21.15.32:0/1656085185 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40011990 con 0x7f0f5009e1d0 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.359+0000 7f0f5521a700 1 -- 172.21.15.32:0/1656085185 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40011d20 con 0x7f0f5009e1d0 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 -- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 msgr2=0x7f0f50138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 --2- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50138c30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009870 tx=0x7f0f40009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 -- 172.21.15.32:0/1656085185 shutdown_connections 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 --2- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f50139170 0x7f0f5013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 --2- 172.21.15.32:0/1656085185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50138c30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 -- 172.21.15.32:0/1656085185 >> 172.21.15.32:0/1656085185 conn(0x7f0f50066f50 msgr2=0x7f0f5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 -- 172.21.15.32:0/1656085185 shutdown_connections 2023-09-30T22:29:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.360+0000 7f0f58480700 1 -- 172.21.15.32:0/1656085185 wait complete. 2023-09-30T22:29:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.361+0000 7f0f58480700 1 Processor -- start 2023-09-30T22:29:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.361+0000 7f0f58480700 1 -- start start 2023-09-30T22:29:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f58480700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 0x7f0f50120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f58480700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f58480700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f50139170 0x7f0f501211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f58480700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50122ba0 con 0x7f0f50139170 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f58480700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50136610 con 0x7f0f5009e1d0 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f58480700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5009c710 con 0x7f0f5009d6f0 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 0x7f0f50120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f55a1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f56a1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f50139170 0x7f0f501211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 0x7f0f50120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50408/0 (socket says 172.21.15.32:50408) 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 -- 172.21.15.32:0/1326090369 learned_addr learned my addr 172.21.15.32:0/1326090369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 -- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 msgr2=0x7f0f50120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 -- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f50139170 msgr2=0x7f0f501211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f50139170 0x7f0f501211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 -- 172.21.15.32:0/1326090369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009a70 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f56a1d700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f50139170 0x7f0f501211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.362+0000 7f0f5621c700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 0x7f0f50120770 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c520 tx=0x7f0f4c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.363+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c004c80 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.363+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c004e00 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.363+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f400094a0 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.363+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c007310 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.363+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f50072150 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.363+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f4c007470 con 0x7f0f5009d6f0 2023-09-30T22:29:28.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.364+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0f4c03f300 con 0x7f0f5009d6f0 2023-09-30T22:29:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.364+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50123e80 con 0x7f0f5009d6f0 2023-09-30T22:29:28.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.369+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f4c0123b0 con 0x7f0f5009d6f0 2023-09-30T22:29:28.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:28.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.573+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0f50074ea0 con 0x7f0f5009d6f0 2023-09-30T22:29:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.573+0000 7f0f46ffd700 1 -- 172.21.15.32:0/1326090369 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f4c00ce60 con 0x7f0f5009d6f0 2023-09-30T22:29:28.573 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 msgr2=0x7f0f50120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 0x7f0f50120770 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c520 tx=0x7f0f4c00c830 comp rx=0 tx=0).stop 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 shutdown_connections 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f50139170 0x7f0f501211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0f5009d6f0 0x7f0f50120770 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 --2- 172.21.15.32:0/1326090369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f5009e1d0 0x7f0f50120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 >> 172.21.15.32:0/1326090369 conn(0x7f0f50066f50 msgr2=0x7f0f50136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 shutdown_connections 2023-09-30T22:29:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.575+0000 7f0f58480700 1 -- 172.21.15.32:0/1326090369 wait complete. 2023-09-30T22:29:28.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.627+0000 7fc0d406f700 1 Processor -- start 2023-09-30T22:29:28.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.627+0000 7fc0d406f700 1 -- start start 2023-09-30T22:29:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d406f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d406f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 0x7fc0cc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d406f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 0x7fc0cc1343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d406f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc09b0f0 con 0x7fc0cc133f30 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d406f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc09d2f0 con 0x7fc0cc09e000 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d406f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc09a810 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 0x7fc0cc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d1e0b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d260c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 0x7fc0cc1343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 0x7fc0cc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37346/0 (socket says 172.21.15.32:37346) 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d260c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 0x7fc0cc1343d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56966/0 (socket says 172.21.15.32:56966) 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 -- 172.21.15.32:0/3032950845 learned_addr learned my addr 172.21.15.32:0/3032950845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d1e0b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50422/0 (socket says 172.21.15.32:50422) 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 -- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 msgr2=0x7fc0cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 --2- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 -- 172.21.15.32:0/3032950845 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 msgr2=0x7fc0cc1343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 --2- 172.21.15.32:0/3032950845 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 0x7fc0cc1343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 -- 172.21.15.32:0/3032950845 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc09a470 con 0x7fc0cc09e000 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d260c700 1 --2- 172.21.15.32:0/3032950845 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 0x7fc0cc1343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.628+0000 7fc0d160a700 1 --2- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 0x7fc0cc1339f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009920 tx=0x7fc0bc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2d6ee624e3c9c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d0e09700 1 -- 172.21.15.32:0/3032950845 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc011830 con 0x7fc0cc09e000 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d0e09700 1 -- 172.21.15.32:0/3032950845 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc011990 con 0x7fc0cc09e000 2023-09-30T22:29:28.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d0e09700 1 -- 172.21.15.32:0/3032950845 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc011d20 con 0x7fc0cc09e000 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 -- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 msgr2=0x7fc0cc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 --2- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 0x7fc0cc1339f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009920 tx=0x7fc0bc009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 -- 172.21.15.32:0/3032950845 shutdown_connections 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 --2- 172.21.15.32:0/3032950845 >> v2:172.21.15.32:3300/0 conn(0x7fc0cc133f30 0x7fc0cc1343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 --2- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 --2- 172.21.15.32:0/3032950845 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc09e000 0x7fc0cc1339f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 -- 172.21.15.32:0/3032950845 >> 172.21.15.32:0/3032950845 conn(0x7fc0cc066f50 msgr2=0x7fc0cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 -- 172.21.15.32:0/3032950845 shutdown_connections 2023-09-30T22:29:28.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.629+0000 7fc0d406f700 1 -- 172.21.15.32:0/3032950845 wait complete. 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.630+0000 7fc0d406f700 1 Processor -- start 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.630+0000 7fc0d406f700 1 -- start start 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d406f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d406f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc09e000 0x7fc0cc120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d406f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 0x7fc0cc121090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d406f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1229f0 con 0x7fc0cc09e000 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d406f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1240a0 con 0x7fc0cc133f30 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d406f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc0a18d0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d260c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 0x7fc0cc121090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d160a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc09e000 0x7fc0cc120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d1e0b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d260c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 0x7fc0cc121090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37350/0 (socket says 172.21.15.32:37350) 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d260c700 1 -- 172.21.15.32:0/4093201960 learned_addr learned my addr 172.21.15.32:0/4093201960 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d1e0b700 1 -- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 msgr2=0x7fc0cc121090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d1e0b700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 0x7fc0cc121090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d1e0b700 1 -- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc09e000 msgr2=0x7fc0cc120b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d1e0b700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc09e000 0x7fc0cc120b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d1e0b700 1 -- 172.21.15.32:0/4093201960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4009a70 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d260c700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 0x7fc0cc121090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.631+0000 7fc0d160a700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc09e000 0x7fc0cc120b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.632+0000 7fc0d1e0b700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc120610 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e900 tx=0x7fc0c800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.632+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8013680 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.632+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c80137e0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.632+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc009550 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.632+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8013b50 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.632+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc123dc0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.633+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc071ff0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.633+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc0c8013cb0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.633+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc0c803f8d0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.638+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0c801b7c0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:28.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:28.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.787+0000 7f1d838df700 1 Processor -- start 2023-09-30T22:29:28.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.787+0000 7f1d838df700 1 -- start start 2023-09-30T22:29:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d838df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d838df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d7c138d80 0x7f1d7c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d838df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0a2a70 con 0x7f1d7c09d630 2023-09-30T22:29:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d838df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0a01d0 con 0x7f1d7c138d80 2023-09-30T22:29:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d838df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c09e2b0 con 0x7f1d7c0a0a30 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d8167b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d8167b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50448/0 (socket says 172.21.15.32:50448) 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d8167b700 1 -- 172.21.15.32:0/3878849744 learned_addr learned my addr 172.21.15.32:0/3878849744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.788+0000 7f1d81e7c700 1 --2- 172.21.15.32:0/3878849744 >> v2:172.21.15.32:3300/0 conn(0x7f1d7c138d80 0x7f1d7c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d8167b700 1 -- 172.21.15.32:0/3878849744 >> v2:172.21.15.32:3300/0 conn(0x7f1d7c138d80 msgr2=0x7f1d7c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d8167b700 1 --2- 172.21.15.32:0/3878849744 >> v2:172.21.15.32:3300/0 conn(0x7f1d7c138d80 0x7f1d7c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d8167b700 1 -- 172.21.15.32:0/3878849744 >> v1:172.21.15.32:6789/0 conn(0x7f1d7c09d630 legacy=0x7f1d7c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d8167b700 1 -- 172.21.15.32:0/3878849744 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c09dc50 con 0x7f1d7c0a0a30 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d81e7c700 1 --2- 172.21.15.32:0/3878849744 >> v2:172.21.15.32:3300/0 conn(0x7f1d7c138d80 0x7f1d7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d8167b700 1 --2- 172.21.15.32:0/3878849744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c0a0e00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1d780098f0 tx=0x7f1d78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d61712edb162460 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d73fff700 1 -- 172.21.15.32:0/3878849744 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011830 con 0x7f1d7c0a0a30 2023-09-30T22:29:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d73fff700 1 -- 172.21.15.32:0/3878849744 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78011990 con 0x7f1d7c0a0a30 2023-09-30T22:29:28.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.789+0000 7f1d73fff700 1 -- 172.21.15.32:0/3878849744 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78011d20 con 0x7f1d7c0a0a30 2023-09-30T22:29:28.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 -- 172.21.15.32:0/3878849744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 msgr2=0x7f1d7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 --2- 172.21.15.32:0/3878849744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c0a0e00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1d780098f0 tx=0x7f1d78009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:28.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 -- 172.21.15.32:0/3878849744 shutdown_connections 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 --2- 172.21.15.32:0/3878849744 >> v2:172.21.15.32:3300/0 conn(0x7f1d7c138d80 0x7f1d7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 --2- 172.21.15.32:0/3878849744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c0a0e00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 -- 172.21.15.32:0/3878849744 >> 172.21.15.32:0/3878849744 conn(0x7f1d7c0a3050 msgr2=0x7f1d7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 -- 172.21.15.32:0/3878849744 shutdown_connections 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.790+0000 7f1d838df700 1 -- 172.21.15.32:0/3878849744 wait complete. 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.791+0000 7f1d838df700 1 Processor -- start 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.791+0000 7f1d838df700 1 -- start start 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d838df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 0x7f1d7c1208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d838df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d838df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7c138d80 0x7f1d7c121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d838df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c122470 con 0x7f1d7c138d80 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d838df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1237c0 con 0x7f1d7c09d630 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d838df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c121d50 con 0x7f1d7c0a0a30 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d8167b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 0x7f1d7c1208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d81e7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7c138d80 0x7f1d7c121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d8167b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 0x7f1d7c1208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37358/0 (socket says 172.21.15.32:37358) 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d8167b700 1 -- 172.21.15.32:0/2643177524 learned_addr learned my addr 172.21.15.32:0/2643177524 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.792+0000 7f1d80e7a700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d8167b700 1 -- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 msgr2=0x7f1d7c120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d8167b700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d8167b700 1 -- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7c138d80 msgr2=0x7f1d7c121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d8167b700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7c138d80 0x7f1d7c121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d8167b700 1 -- 172.21.15.32:0/2643177524 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009a70 con 0x7f1d7c09d630 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d81e7c700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7c138d80 0x7f1d7c121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d8167b700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 0x7f1d7c1208f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1d78000c00 tx=0x7f1d78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780036a0 con 0x7f1d7c09d630 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78003800 con 0x7f1d7c09d630 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d78009550 con 0x7f1d7c09d630 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.793+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78003bb0 con 0x7f1d7c09d630 2023-09-30T22:29:28.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.794+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c0a0410 con 0x7f1d7c09d630 2023-09-30T22:29:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.794+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d78021070 con 0x7f1d7c09d630 2023-09-30T22:29:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.795+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1d78040d30 con 0x7f1d7c09d630 2023-09-30T22:29:28.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.795+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c123cc0 con 0x7f1d7c09d630 2023-09-30T22:29:28.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.800+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d78015070 con 0x7f1d7c09d630 2023-09-30T22:29:28.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.830+0000 7f1812dbe700 1 Processor -- start 2023-09-30T22:29:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1812dbe700 1 -- start start 2023-09-30T22:29:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1812dbe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f180c09f410 0x7f180c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1812dbe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c136fe0 0x7f180c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1812dbe700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c070940 con 0x7f180c09e930 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1812dbe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c06fd60 con 0x7f180c09f410 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1812dbe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c06fa80 con 0x7f180c136fe0 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f180bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f180c09f410 0x7f180c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f180bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f180c09f410 0x7f180c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57002/0 (socket says 172.21.15.32:57002) 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f1810b5a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f180c09e930 0x7f180c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55058/0 (socket says 172.21.15.32:55058) 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f181135b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c136fe0 0x7f180c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.831+0000 7f180bfff700 1 -- 172.21.15.32:0/3215639291 learned_addr learned my addr 172.21.15.32:0/3215639291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180bfff700 1 -- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c136fe0 msgr2=0x7f180c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180bfff700 1 --2- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c136fe0 0x7f180c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180bfff700 1 -- 172.21.15.32:0/3215639291 >> v1:172.21.15.32:6789/0 conn(0x7f180c09e930 legacy=0x7f180c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180bfff700 1 -- 172.21.15.32:0/3215639291 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c09fb60 con 0x7f180c09f410 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f181135b700 1 --2- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c136fe0 0x7f180c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180bfff700 1 --2- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c09f410 0x7f180c136aa0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f18000098f0 tx=0x7f1800009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c8f83c411f7489 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180b7fe700 1 -- 172.21.15.32:0/3215639291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800011830 con 0x7f180c09f410 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180b7fe700 1 -- 172.21.15.32:0/3215639291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1800011990 con 0x7f180c09f410 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.832+0000 7f180b7fe700 1 -- 172.21.15.32:0/3215639291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800011d20 con 0x7f180c09f410 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 -- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c09f410 msgr2=0x7f180c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 --2- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c09f410 0x7f180c136aa0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f18000098f0 tx=0x7f1800009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:28.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 -- 172.21.15.32:0/3215639291 shutdown_connections 2023-09-30T22:29:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 --2- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c136fe0 0x7f180c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 --2- 172.21.15.32:0/3215639291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c09f410 0x7f180c136aa0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 -- 172.21.15.32:0/3215639291 >> 172.21.15.32:0/3215639291 conn(0x7f180c0a4210 msgr2=0x7f180c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 -- 172.21.15.32:0/3215639291 shutdown_connections 2023-09-30T22:29:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.833+0000 7f1812dbe700 1 -- 172.21.15.32:0/3215639291 wait complete. 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.834+0000 7f1812dbe700 1 Processor -- start 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.834+0000 7f1812dbe700 1 -- start start 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.834+0000 7f1812dbe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 0x7f180c1319f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.834+0000 7f1812dbe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c09f410 0x7f180c131f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1812dbe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c136fe0 0x7f180c1c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1812dbe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c120520 con 0x7f180c136fe0 2023-09-30T22:29:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1812dbe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c11e7a0 con 0x7f180c09f410 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1812dbe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f180c11f840 con 0x7f180c09e930 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 0x7f180c1319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 0x7f180c1319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50464/0 (socket says 172.21.15.32:50464) 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 -- 172.21.15.32:0/2683980814 learned_addr learned my addr 172.21.15.32:0/2683980814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f181135b700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c136fe0 0x7f180c1c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f180bfff700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c09f410 0x7f180c131f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 -- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c09f410 msgr2=0x7f180c131f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c09f410 0x7f180c131f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 -- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c136fe0 msgr2=0x7f180c1c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c136fe0 0x7f180c1c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 -- 172.21.15.32:0/2683980814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040094a0 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f180bfff700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c09f410 0x7f180c131f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f181135b700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c136fe0 0x7f180c1c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.835+0000 7f1810b5a700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 0x7f180c1319f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00cc50 tx=0x7f17fc00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.836+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00aac0 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.836+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc00ac20 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.836+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1800009550 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.836+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007070 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.836+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c11f410 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.837+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c1324d0 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.837+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17fc0071d0 con 0x7f180c09e930 2023-09-30T22:29:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.837+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f17fc03f990 con 0x7f180c09e930 2023-09-30T22:29:28.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.846+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17fc01b3b0 con 0x7f180c09e930 2023-09-30T22:29:28.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.858+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc0cc09e8c0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.858+0000 7fc0c27fc700 1 -- 172.21.15.32:0/4093201960 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0c801b3b0 con 0x7fc0cc09d6f0 2023-09-30T22:29:28.858 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:28.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 msgr2=0x7fc0cc120610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc120610 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e900 tx=0x7fc0c800ec10 comp rx=0 tx=0).stop 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 shutdown_connections 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0cc133f30 0x7fc0cc121090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0cc09d6f0 0x7fc0cc120610 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 --2- 172.21.15.32:0/4093201960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0cc09e000 0x7fc0cc120b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.860+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 >> 172.21.15.32:0/4093201960 conn(0x7fc0cc066f50 msgr2=0x7fc0cc132230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.861+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 shutdown_connections 2023-09-30T22:29:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:28.861+0000 7fc0d406f700 1 -- 172.21.15.32:0/4093201960 wait complete. 2023-09-30T22:29:28.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:29.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.009+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1d7c0747f0 con 0x7f1d7c09d630 2023-09-30T22:29:29.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.009+0000 7f1d71ffb700 1 -- 172.21.15.32:0/2643177524 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d78029030 con 0x7f1d7c09d630 2023-09-30T22:29:29.010 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 msgr2=0x7f1d7c1208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 0x7f1d7c1208f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1d78000c00 tx=0x7f1d78004750 comp rx=0 tx=0).stop 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 shutdown_connections 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7c138d80 0x7f1d7c121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7c09d630 0x7f1d7c1208f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 --2- 172.21.15.32:0/2643177524 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1d7c0a0a30 0x7f1d7c120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 >> 172.21.15.32:0/2643177524 conn(0x7f1d7c0a3050 msgr2=0x7f1d7c139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:29.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 shutdown_connections 2023-09-30T22:29:29.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.012+0000 7f1d838df700 1 -- 172.21.15.32:0/2643177524 wait complete. 2023-09-30T22:29:29.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.053+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f180c09e190 con 0x7f180c09e930 2023-09-30T22:29:29.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.053+0000 7f18097fa700 1 -- 172.21.15.32:0/2683980814 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17fc017020 con 0x7f180c09e930 2023-09-30T22:29:29.053 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.055+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 msgr2=0x7f180c1319f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.055+0000 7f1812dbe700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 0x7f180c1319f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00cc50 tx=0x7f17fc00cf60 comp rx=0 tx=0).stop 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 shutdown_connections 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f180c136fe0 0x7f180c1c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f180c09e930 0x7f180c1319f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 --2- 172.21.15.32:0/2683980814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f180c09f410 0x7f180c131f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 >> 172.21.15.32:0/2683980814 conn(0x7f180c0a4210 msgr2=0x7f180c134b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 shutdown_connections 2023-09-30T22:29:29.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.056+0000 7f1812dbe700 1 -- 172.21.15.32:0/2683980814 wait complete. 2023-09-30T22:29:29.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:29.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:29.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.736+0000 7ff5ef03c700 1 Processor -- start 2023-09-30T22:29:29.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 -- start start 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809d6f0 0x7ff5e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5e809e000 0x7ff5e8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 0x7ff5e8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e809f520 con 0x7ff5e809e000 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e809f130 con 0x7ff5e809e410 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ef03c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e809e930 con 0x7ff5e809d6f0 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ed5d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 0x7ff5e8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5e809e000 0x7ff5e8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ecdd8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809d6f0 0x7ff5e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:29.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ed5d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 0x7ff5e8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37380/0 (socket says 172.21.15.32:37380) 2023-09-30T22:29:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5e7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5e809e000 0x7ff5e8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57026/0 (socket says 172.21.15.32:57026) 2023-09-30T22:29:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.737+0000 7ff5ed5d9700 1 -- 172.21.15.32:0/950813384 learned_addr learned my addr 172.21.15.32:0/950813384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:29.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e7fff700 1 -- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809d6f0 msgr2=0x7ff5e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e7fff700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809d6f0 0x7ff5e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e7fff700 1 -- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 msgr2=0x7ff5e8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e7fff700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 0x7ff5e8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e7fff700 1 -- 172.21.15.32:0/950813384 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e809b2d0 con 0x7ff5e809e000 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5ed5d9700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 0x7ff5e8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e7fff700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e000 0x7ff5e8130660 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000bb10 tx=0x7ff5d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaab9ffda8b8af3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e77fe700 1 -- 172.21.15.32:0/950813384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000dc30 con 0x7ff5e809e000 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e77fe700 1 -- 172.21.15.32:0/950813384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d000dd90 con 0x7ff5e809e000 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.738+0000 7ff5e77fe700 1 -- 172.21.15.32:0/950813384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0010690 con 0x7ff5e809e000 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 -- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e000 msgr2=0x7ff5e8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e000 0x7ff5e8130660 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000bb10 tx=0x7ff5d000be20 comp rx=0 tx=0).stop 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 -- 172.21.15.32:0/950813384 shutdown_connections 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809e410 0x7ff5e8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809d6f0 0x7ff5e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/950813384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e000 0x7ff5e8130660 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 -- 172.21.15.32:0/950813384 >> 172.21.15.32:0/950813384 conn(0x7ff5e8066f50 msgr2=0x7ff5e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 -- 172.21.15.32:0/950813384 shutdown_connections 2023-09-30T22:29:29.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.739+0000 7ff5ef03c700 1 -- 172.21.15.32:0/950813384 wait complete. 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 Processor -- start 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 -- start start 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809d6f0 0x7ff5e811e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 0x7ff5e811e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e410 0x7ff5e811ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8121d30 con 0x7ff5e809e410 2023-09-30T22:29:29.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e809c700 con 0x7ff5e809d6f0 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.740+0000 7ff5ef03c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8120660 con 0x7ff5e809e000 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 0x7ff5e811e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5ecdd8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809d6f0 0x7ff5e811e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 0x7ff5e811e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50484/0 (socket says 172.21.15.32:50484) 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 -- 172.21.15.32:0/3632667451 learned_addr learned my addr 172.21.15.32:0/3632667451 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5ed5d9700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e410 0x7ff5e811ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 -- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809d6f0 msgr2=0x7ff5e811e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809d6f0 0x7ff5e811e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 -- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e410 msgr2=0x7ff5e811ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e410 0x7ff5e811ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 -- 172.21.15.32:0/3632667451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009b30 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5ecdd8700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809d6f0 0x7ff5e811e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e7fff700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 0x7ff5e811e750 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000c00 tx=0x7ff5d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0003ab0 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d0003c10 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d000b740 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0010470 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.741+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80a1b40 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.742+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff5d0019400 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.742+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff5d0040a30 con 0x7ff5e809e000 2023-09-30T22:29:29.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.742+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8074ea0 con 0x7ff5e809e000 2023-09-30T22:29:29.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.747+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff5d000f3d0 con 0x7ff5e809e000 2023-09-30T22:29:29.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.955+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff5e8130170 con 0x7ff5e809e000 2023-09-30T22:29:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.956+0000 7ff5e57fa700 1 -- 172.21.15.32:0/3632667451 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff5d0028030 con 0x7ff5e809e000 2023-09-30T22:29:29.956 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 msgr2=0x7ff5e811e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 0x7ff5e811e750 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000c00 tx=0x7ff5d0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 shutdown_connections 2023-09-30T22:29:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5e809e410 0x7ff5e811ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5e809d6f0 0x7ff5e811e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 --2- 172.21.15.32:0/3632667451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5e809e000 0x7ff5e811e750 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 >> 172.21.15.32:0/3632667451 conn(0x7ff5e8066f50 msgr2=0x7ff5e806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 shutdown_connections 2023-09-30T22:29:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:29.958+0000 7ff5ef03c700 1 -- 172.21.15.32:0/3632667451 wait complete. 2023-09-30T22:29:29.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.015+0000 7fc0a6eb0700 1 Processor -- start 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.015+0000 7fc0a6eb0700 1 -- start start 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a6eb0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a009e1d0 0x7fc0a0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a6eb0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0a0135e50 0x7fc0a0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a6eb0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a009f130 con 0x7fc0a009d6f0 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a6eb0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a009f520 con 0x7fc0a0135e50 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a6eb0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a009e930 con 0x7fc0a009e1d0 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a544d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0a0135e50 0x7fc0a0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a544d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0a0135e50 0x7fc0a0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57042/0 (socket says 172.21.15.32:57042) 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a009e1d0 0x7fc0a0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a544d700 1 -- 172.21.15.32:0/1487587210 learned_addr learned my addr 172.21.15.32:0/1487587210 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.016+0000 7fc0a544d700 1 -- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a009e1d0 msgr2=0x7fc0a0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a544d700 1 --2- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a009e1d0 0x7fc0a0135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a544d700 1 -- 172.21.15.32:0/1487587210 >> v1:172.21.15.32:6789/0 conn(0x7fc0a009d6f0 legacy=0x7fc0a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a544d700 1 -- 172.21.15.32:0/1487587210 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a009b2d0 con 0x7fc0a0135e50 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc09f7fe700 1 -- 172.21.15.32:0/1487587210 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002559460 0 0) 0x7fc0a009f130 con 0x7fc0a009d6f0 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc09ffff700 1 --2- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a009e1d0 0x7fc0a0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a544d700 1 --2- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a0135e50 0x7fc0a0138230 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc09800d180 tx=0x7fc09800d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f961ec8c325e180e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc09f7fe700 1 -- 172.21.15.32:0/1487587210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098010070 con 0x7fc0a0135e50 2023-09-30T22:29:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc09f7fe700 1 -- 172.21.15.32:0/1487587210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09800dd00 con 0x7fc0a0135e50 2023-09-30T22:29:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc09f7fe700 1 -- 172.21.15.32:0/1487587210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800b310 con 0x7fc0a0135e50 2023-09-30T22:29:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a0135e50 msgr2=0x7fc0a0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a0135e50 0x7fc0a0138230 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fc09800d180 tx=0x7fc09800d490 comp rx=0 tx=0).stop 2023-09-30T22:29:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.017+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/1487587210 shutdown_connections 2023-09-30T22:29:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.018+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a0135e50 0x7fc0a0138230 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.018+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/1487587210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a009e1d0 0x7fc0a0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.018+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/1487587210 >> 172.21.15.32:0/1487587210 conn(0x7fc0a0066f50 msgr2=0x7fc0a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.018+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/1487587210 shutdown_connections 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.018+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/1487587210 wait complete. 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 Processor -- start 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 -- start start 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a009d6f0 0x7fc0a01c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 0x7fc0a01c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a0135e50 0x7fc0a01c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a009c770 con 0x7fc0a009e1d0 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0124e20 con 0x7fc0a009d6f0 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a6eb0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01264d0 con 0x7fc0a0135e50 2023-09-30T22:29:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 0x7fc0a01c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc0a544d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a0135e50 0x7fc0a01c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 0x7fc0a01c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57044/0 (socket says 172.21.15.32:57044) 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.019+0000 7fc09ffff700 1 -- 172.21.15.32:0/3899880353 learned_addr learned my addr 172.21.15.32:0/3899880353 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc0a4c4c700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a009d6f0 0x7fc0a01c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09ffff700 1 -- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a0135e50 msgr2=0x7fc0a01c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09ffff700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a0135e50 0x7fc0a01c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09ffff700 1 -- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a009d6f0 msgr2=0x7fc0a01c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09ffff700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a009d6f0 0x7fc0a01c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09ffff700 1 -- 172.21.15.32:0/3899880353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0880094a0 con 0x7fc0a009e1d0 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc0a544d700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a0135e50 0x7fc0a01c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09ffff700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 0x7fc0a01c85b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc08800ebe0 tx=0x7fc08800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08800c850 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08800c9b0 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09800bd40 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08800cd40 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.020+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a00720b0 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.021+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc088013470 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.021+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc08803fb30 con 0x7fc0a009e1d0 2023-09-30T22:29:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.021+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a0074ea0 con 0x7fc0a009e1d0 2023-09-30T22:29:30.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.026+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc088021080 con 0x7fc0a009e1d0 2023-09-30T22:29:30.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.168+0000 7fcf5bedb700 1 Processor -- start 2023-09-30T22:29:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.168+0000 7fcf5bedb700 1 -- start start 2023-09-30T22:29:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5bedb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 0x7fcf54133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5bedb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf54130a40 0x7fcf5413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5bedb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5406bc00 con 0x7fcf5412ff20 2023-09-30T22:29:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5bedb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5406c570 con 0x7fcf54130a40 2023-09-30T22:29:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5bedb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540a0040 con 0x7fcf54133a80 2023-09-30T22:29:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf59c77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 0x7fcf54133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5a478700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf54130a40 0x7fcf5413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf59c77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 0x7fcf54133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37406/0 (socket says 172.21.15.32:37406) 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf59c77700 1 -- 172.21.15.32:0/3038422286 learned_addr learned my addr 172.21.15.32:0/3038422286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5a478700 1 -- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 msgr2=0x7fcf54133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5a478700 1 --2- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 0x7fcf54133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5a478700 1 -- 172.21.15.32:0/3038422286 >> v1:172.21.15.32:6789/0 conn(0x7fcf5412ff20 legacy=0x7fcf54130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf5a478700 1 -- 172.21.15.32:0/3038422286 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54133860 con 0x7fcf54130a40 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.169+0000 7fcf58c75700 1 -- 172.21.15.32:0/3038422286 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91709820 0 0) 0x7fcf5406bc00 con 0x7fcf5412ff20 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.170+0000 7fcf59c77700 1 --2- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 0x7fcf54133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.170+0000 7fcf5a478700 1 --2- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf54130a40 0x7fcf5413b850 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009d90 tx=0x7fcf4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b354b16a2d474bc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.170+0000 7fcf58c75700 1 -- 172.21.15.32:0/3038422286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00e070 con 0x7fcf54130a40 2023-09-30T22:29:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.170+0000 7fcf58c75700 1 -- 172.21.15.32:0/3038422286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c00bb20 con 0x7fcf54130a40 2023-09-30T22:29:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.170+0000 7fcf58c75700 1 -- 172.21.15.32:0/3038422286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c012430 con 0x7fcf54130a40 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf54130a40 msgr2=0x7fcf5413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf54130a40 0x7fcf5413b850 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009d90 tx=0x7fcf4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3038422286 shutdown_connections 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf54130a40 0x7fcf5413b850 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3038422286 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54133a80 0x7fcf54133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3038422286 >> 172.21.15.32:0/3038422286 conn(0x7fcf540a4210 msgr2=0x7fcf540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3038422286 shutdown_connections 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.171+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3038422286 wait complete. 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.172+0000 7fcf5bedb700 1 Processor -- start 2023-09-30T22:29:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.172+0000 7fcf5bedb700 1 -- start start 2023-09-30T22:29:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5bedb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 0x7fcf5409c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5bedb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54130a40 0x7fcf5409cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5bedb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf54133a80 0x7fcf5409d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5bedb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5411fe10 con 0x7fcf5412ff20 2023-09-30T22:29:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5bedb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5409dc40 con 0x7fcf54130a40 2023-09-30T22:29:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5bedb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54120840 con 0x7fcf54133a80 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf59c77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 0x7fcf5409c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf59476700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54130a40 0x7fcf5409cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf59c77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 0x7fcf5409c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57062/0 (socket says 172.21.15.32:57062) 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf59476700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54130a40 0x7fcf5409cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:37422/0 (socket says 172.21.15.32:37422) 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf59c77700 1 -- 172.21.15.32:0/3959889987 learned_addr learned my addr 172.21.15.32:0/3959889987 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.173+0000 7fcf5a478700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf54133a80 0x7fcf5409d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf59c77700 1 -- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf54133a80 msgr2=0x7fcf5409d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf59c77700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf54133a80 0x7fcf5409d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf59c77700 1 -- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54130a40 msgr2=0x7fcf5409cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf59c77700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54130a40 0x7fcf5409cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf59c77700 1 -- 172.21.15.32:0/3959889987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009550 con 0x7fcf5412ff20 2023-09-30T22:29:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf59c77700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 0x7fcf5409c6e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000ec10 tx=0x7fcf5000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000c850 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5000c9b0 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c009a70 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000cd40 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.174+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5412d770 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.175+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf50013470 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.175+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54074ea0 con 0x7fcf5412ff20 2023-09-30T22:29:30.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.176+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcf5003f950 con 0x7fcf5412ff20 2023-09-30T22:29:30.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.181+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf50013720 con 0x7fcf5412ff20 2023-09-30T22:29:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.210+0000 7fb1467bc700 1 Processor -- start 2023-09-30T22:29:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.210+0000 7fb1467bc700 1 -- start start 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb1467bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 0x7fb140138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb1467bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb140139250 0x7fb14013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb1467bc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14009edd0 con 0x7fb1400a0a30 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb1467bc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400a18e0 con 0x7fb1400a1510 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb1467bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb14009e1f0 con 0x7fb140139250 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb13f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 0x7fb140138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb13f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 0x7fb140138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57074/0 (socket says 172.21.15.32:57074) 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb13f7fe700 1 -- 172.21.15.32:0/2775309743 learned_addr learned my addr 172.21.15.32:0/2775309743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb144d59700 1 --2- 172.21.15.32:0/2775309743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb140139250 0x7fb14013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb144d59700 1 -- 172.21.15.32:0/2775309743 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 msgr2=0x7fb140138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb144d59700 1 --2- 172.21.15.32:0/2775309743 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 0x7fb140138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb144d59700 1 -- 172.21.15.32:0/2775309743 >> v1:172.21.15.32:6789/0 conn(0x7fb1400a0a30 legacy=0x7fb1400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb144d59700 1 -- 172.21.15.32:0/2775309743 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14009dc50 con 0x7fb140139250 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.211+0000 7fb13f7fe700 1 --2- 172.21.15.32:0/2775309743 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 0x7fb140138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb144d59700 1 --2- 172.21.15.32:0/2775309743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb140139250 0x7fb14013b630 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb138009d90 tx=0x7fb13800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=228898e1e1b83c67 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb13effd700 1 -- 172.21.15.32:0/2775309743 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800e070 con 0x7fb140139250 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb13effd700 1 -- 172.21.15.32:0/2775309743 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800bb20 con 0x7fb140139250 2023-09-30T22:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb13effd700 1 -- 172.21.15.32:0/2775309743 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138012430 con 0x7fb140139250 2023-09-30T22:29:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb1467bc700 1 -- 172.21.15.32:0/2775309743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb140139250 msgr2=0x7fb14013b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb1467bc700 1 --2- 172.21.15.32:0/2775309743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb140139250 0x7fb14013b630 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb138009d90 tx=0x7fb13800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb1467bc700 1 -- 172.21.15.32:0/2775309743 shutdown_connections 2023-09-30T22:29:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb1467bc700 1 --2- 172.21.15.32:0/2775309743 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb140139250 0x7fb14013b630 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.212+0000 7fb1467bc700 1 --2- 172.21.15.32:0/2775309743 >> v2:172.21.15.32:3300/0 conn(0x7fb1400a1510 0x7fb140138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.213+0000 7fb1467bc700 1 -- 172.21.15.32:0/2775309743 >> 172.21.15.32:0/2775309743 conn(0x7fb1400a6550 msgr2=0x7fb1400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.213+0000 7fb1467bc700 1 -- 172.21.15.32:0/2775309743 shutdown_connections 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.213+0000 7fb1467bc700 1 -- 172.21.15.32:0/2775309743 wait complete. 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 Processor -- start 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 -- start start 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1400a0a30 0x7fb140122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 0x7fb140122e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb140139250 0x7fb1401233a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb140126260 con 0x7fb140139250 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb140126440 con 0x7fb1400a0a30 2023-09-30T22:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb1467bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400a0220 con 0x7fb1400a1510 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb13f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 0x7fb140122e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.214+0000 7fb13f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 0x7fb140122e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:50538/0 (socket says 172.21.15.32:50538) 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 -- 172.21.15.32:0/25376138 learned_addr learned my addr 172.21.15.32:0/25376138 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb144d59700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb140139250 0x7fb1401233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13ffff700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1400a0a30 0x7fb140122920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 -- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1400a0a30 msgr2=0x7fb140122920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1400a0a30 0x7fb140122920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 -- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb140139250 msgr2=0x7fb1401233a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb140139250 0x7fb1401233a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 -- 172.21.15.32:0/25376138 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134009550 con 0x7fb1400a1510 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13f7fe700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 0x7fb140122e60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb13400ec10 tx=0x7fb13400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400c850 con 0x7fb1400a1510 2023-09-30T22:29:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13400c9b0 con 0x7fb1400a1510 2023-09-30T22:29:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.215+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400cd40 con 0x7fb1400a1510 2023-09-30T22:29:30.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.216+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb138009a70 con 0x7fb1400a1510 2023-09-30T22:29:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.216+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb140138a50 con 0x7fb1400a1510 2023-09-30T22:29:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.217+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb134013470 con 0x7fb1400a1510 2023-09-30T22:29:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.217+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb13403fc00 con 0x7fb1400a1510 2023-09-30T22:29:30.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.217+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb140074ea0 con 0x7fb1400a1510 2023-09-30T22:29:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.222+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb134021080 con 0x7fb1400a1510 2023-09-30T22:29:30.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.239+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc0a0126250 con 0x7fc0a009e1d0 2023-09-30T22:29:30.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.239+0000 7fc09d7fa700 1 -- 172.21.15.32:0/3899880353 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc088017020 con 0x7fc0a009e1d0 2023-09-30T22:29:30.239 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 msgr2=0x7fc0a01c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 0x7fc0a01c85b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc08800ebe0 tx=0x7fc08800eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 shutdown_connections 2023-09-30T22:29:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc0a0135e50 0x7fc0a01c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0a009d6f0 0x7fc0a01c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 --2- 172.21.15.32:0/3899880353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0a009e1d0 0x7fc0a01c85b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 >> 172.21.15.32:0/3899880353 conn(0x7fc0a0066f50 msgr2=0x7fc0a009fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:30.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 shutdown_connections 2023-09-30T22:29:30.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.242+0000 7fc0a6eb0700 1 -- 172.21.15.32:0/3899880353 wait complete. 2023-09-30T22:29:30.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.390+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcf5406d740 con 0x7fcf5412ff20 2023-09-30T22:29:30.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.391+0000 7fcf4a7fc700 1 -- 172.21.15.32:0/3959889987 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf50021020 con 0x7fcf5412ff20 2023-09-30T22:29:30.391 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 msgr2=0x7fcf5409c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 0x7fcf5409c6e0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000ec10 tx=0x7fcf5000ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 shutdown_connections 2023-09-30T22:29:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf54133a80 0x7fcf5409d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf5412ff20 0x7fcf5409c6e0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 --2- 172.21.15.32:0/3959889987 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf54130a40 0x7fcf5409cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 >> 172.21.15.32:0/3959889987 conn(0x7fcf540a4210 msgr2=0x7fcf5409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 shutdown_connections 2023-09-30T22:29:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.393+0000 7fcf5bedb700 1 -- 172.21.15.32:0/3959889987 wait complete. 2023-09-30T22:29:30.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:30.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.433+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb1401260a0 con 0x7fb1400a1510 2023-09-30T22:29:30.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.434+0000 7fb13cff9700 1 -- 172.21.15.32:0/25376138 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1401260a0 con 0x7fb1400a1510 2023-09-30T22:29:30.434 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:30.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 msgr2=0x7fb140122e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:30.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 0x7fb140122e60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb13400ec10 tx=0x7fb13400ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 shutdown_connections 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb140139250 0x7fb1401233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb1400a0a30 0x7fb140122920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 --2- 172.21.15.32:0/25376138 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb1400a1510 0x7fb140122e60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 >> 172.21.15.32:0/25376138 conn(0x7fb1400a6550 msgr2=0x7fb1400a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 shutdown_connections 2023-09-30T22:29:30.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:30.436+0000 7fb1467bc700 1 -- 172.21.15.32:0/25376138 wait complete. 2023-09-30T22:29:30.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:30.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:31.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 Processor -- start 2023-09-30T22:29:31.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 -- start start 2023-09-30T22:29:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac640a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac6413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac640a2a70 con 0x7fac640a0a30 2023-09-30T22:29:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640a01d0 con 0x7fac6409d630 2023-09-30T22:29:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac699b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac6409e2b0 con 0x7fac64138d80 2023-09-30T22:29:31.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.120+0000 7fac637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac6413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac62ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac640a0a30 0x7fac640a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47592/0 (socket says 172.21.15.32:47592) 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac62ffd700 1 -- 172.21.15.32:0/703532342 learned_addr learned my addr 172.21.15.32:0/703532342 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac627fc700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac640a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac61ffb700 1 -- 172.21.15.32:0/703532342 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789517725 0 0) 0x7fac640a2a70 con 0x7fac640a0a30 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac637fe700 1 -- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 msgr2=0x7fac640a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac637fe700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac640a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac637fe700 1 -- 172.21.15.32:0/703532342 >> v1:172.21.15.32:6789/0 conn(0x7fac640a0a30 legacy=0x7fac640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac637fe700 1 -- 172.21.15.32:0/703532342 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6409dc50 con 0x7fac64138d80 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac627fc700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac640a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac637fe700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac6413b160 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009dc0 tx=0x7fac5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23f7fff84de88fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac61ffb700 1 -- 172.21.15.32:0/703532342 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00e070 con 0x7fac64138d80 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac61ffb700 1 -- 172.21.15.32:0/703532342 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c00bb20 con 0x7fac64138d80 2023-09-30T22:29:31.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.121+0000 7fac61ffb700 1 -- 172.21.15.32:0/703532342 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c012430 con 0x7fac64138d80 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 -- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 msgr2=0x7fac6413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac6413b160 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009dc0 tx=0x7fac5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 -- 172.21.15.32:0/703532342 shutdown_connections 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac6413b160 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 --2- 172.21.15.32:0/703532342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac640a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 -- 172.21.15.32:0/703532342 >> 172.21.15.32:0/703532342 conn(0x7fac640a3050 msgr2=0x7fac640a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 -- 172.21.15.32:0/703532342 shutdown_connections 2023-09-30T22:29:31.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.122+0000 7fac699b7700 1 -- 172.21.15.32:0/703532342 wait complete. 2023-09-30T22:29:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.123+0000 7fac699b7700 1 Processor -- start 2023-09-30T22:29:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 -- start start 2023-09-30T22:29:31.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac64133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac640a0a30 0x7fac64133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac641341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac64134c40 con 0x7fac640a0a30 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac641361b0 con 0x7fac6409d630 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac699b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac64136ed0 con 0x7fac64138d80 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac62ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac64133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac641341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac62ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac64133770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58880/0 (socket says 172.21.15.32:58880) 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac62ffd700 1 -- 172.21.15.32:0/3746685073 learned_addr learned my addr 172.21.15.32:0/3746685073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac641341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57288/0 (socket says 172.21.15.32:57288) 2023-09-30T22:29:31.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.124+0000 7fac627fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac640a0a30 0x7fac64133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac62ffd700 1 -- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 msgr2=0x7fac641341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac62ffd700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac641341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac62ffd700 1 -- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac640a0a30 msgr2=0x7fac64133cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac62ffd700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac640a0a30 0x7fac64133cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac62ffd700 1 -- 172.21.15.32:0/3746685073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580094a0 con 0x7fac6409d630 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac627fc700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac640a0a30 0x7fac64133cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac62ffd700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac64133770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c520 tx=0x7fac5400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54004c80 con 0x7fac6409d630 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac54004e00 con 0x7fac6409d630 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c009a70 con 0x7fac6409d630 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54007310 con 0x7fac6409d630 2023-09-30T22:29:31.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.125+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac640a0410 con 0x7fac6409d630 2023-09-30T22:29:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.126+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fac54007470 con 0x7fac6409d630 2023-09-30T22:29:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.126+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fac5403f3c0 con 0x7fac6409d630 2023-09-30T22:29:31.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.126+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac64136d10 con 0x7fac6409d630 2023-09-30T22:29:31.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.131+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fac540123b0 con 0x7fac6409d630 2023-09-30T22:29:31.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:31.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.336+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fac640747f0 con 0x7fac6409d630 2023-09-30T22:29:31.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.336+0000 7fac4bfff700 1 -- 172.21.15.32:0/3746685073 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac54011360 con 0x7fac6409d630 2023-09-30T22:29:31.337 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 msgr2=0x7fac64133770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac64133770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c520 tx=0x7fac5400c830 comp rx=0 tx=0).stop 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 shutdown_connections 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac64138d80 0x7fac641341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac6409d630 0x7fac64133770 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 --2- 172.21.15.32:0/3746685073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac640a0a30 0x7fac64133cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 >> 172.21.15.32:0/3746685073 conn(0x7fac640a3050 msgr2=0x7fac640a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 shutdown_connections 2023-09-30T22:29:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.339+0000 7fac699b7700 1 -- 172.21.15.32:0/3746685073 wait complete. 2023-09-30T22:29:31.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 Processor -- start 2023-09-30T22:29:31.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 -- start start 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d300a09c0 0x7f4d300a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 0x7f4d300a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d300a1810 con 0x7f4d30133920 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3009ea10 con 0x7f4d300a09c0 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d35595700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3009e210 con 0x7f4d3009d5c0 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d2e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 0x7f4d300a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d2effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d300a09c0 0x7f4d300a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d2e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 0x7f4d300a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57296/0 (socket says 172.21.15.32:57296) 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d2e7fc700 1 -- 172.21.15.32:0/3819765147 learned_addr learned my addr 172.21.15.32:0/3819765147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.396+0000 7f4d2f7fe700 1 --1- 172.21.15.32:0/3819765147 >> v1:172.21.15.32:6789/0 conn(0x7f4d30133920 0x7f4d30133d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47598/0 (socket says 172.21.15.32:47598) 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2effd700 1 -- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 msgr2=0x7f4d300a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2effd700 1 --2- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 0x7f4d300a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2effd700 1 -- 172.21.15.32:0/3819765147 >> v1:172.21.15.32:6789/0 conn(0x7f4d30133920 legacy=0x7f4d30133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2effd700 1 -- 172.21.15.32:0/3819765147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300a26f0 con 0x7f4d300a09c0 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2dffb700 1 -- 172.21.15.32:0/3819765147 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835179217 0 0) 0x7f4d300a1810 con 0x7f4d30133920 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2e7fc700 1 --2- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 0x7f4d300a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2effd700 1 --2- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d300a09c0 0x7f4d300a0d90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009840 tx=0x7f4d20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f538e2687e50b961 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2dffb700 1 -- 172.21.15.32:0/3819765147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20011830 con 0x7f4d300a09c0 2023-09-30T22:29:31.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2dffb700 1 -- 172.21.15.32:0/3819765147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20011990 con 0x7f4d300a09c0 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.397+0000 7f4d2dffb700 1 -- 172.21.15.32:0/3819765147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20011d20 con 0x7f4d300a09c0 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 -- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d300a09c0 msgr2=0x7f4d300a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 --2- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d300a09c0 0x7f4d300a0d90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009840 tx=0x7f4d20009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 -- 172.21.15.32:0/3819765147 shutdown_connections 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 --2- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d300a09c0 0x7f4d300a0d90 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 --2- 172.21.15.32:0/3819765147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d3009d5c0 0x7f4d300a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 -- 172.21.15.32:0/3819765147 >> 172.21.15.32:0/3819765147 conn(0x7f4d300a2fe0 msgr2=0x7f4d300a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 -- 172.21.15.32:0/3819765147 shutdown_connections 2023-09-30T22:29:31.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.398+0000 7f4d35595700 1 -- 172.21.15.32:0/3819765147 wait complete. 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 Processor -- start 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 -- start start 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 0x7f4d301326b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 0x7f4d30132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d30133920 0x7f4d301c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3011ee10 con 0x7f4d30133920 2023-09-30T22:29:31.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30121250 con 0x7f4d3009d5c0 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.399+0000 7f4d35595700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301204e0 con 0x7f4d300a09c0 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 0x7f4d301326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 0x7f4d30132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d30133920 0x7f4d301c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 0x7f4d301326b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58884/0 (socket says 172.21.15.32:58884) 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 0x7f4d30132bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57308/0 (socket says 172.21.15.32:57308) 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2effd700 1 -- 172.21.15.32:0/1210001553 learned_addr learned my addr 172.21.15.32:0/1210001553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 -- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 msgr2=0x7f4d301326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 0x7f4d301326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 -- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d30133920 msgr2=0x7f4d301c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d30133920 0x7f4d301c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 -- 172.21.15.32:0/1210001553 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d24009550 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2f7fe700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d30133920 0x7f4d301c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2effd700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 0x7f4d301326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d2e7fc700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 0x7f4d30132bf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400e980 tx=0x7f4d2400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004030 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d24004190 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d200094a0 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004520 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.400+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d30133130 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.401+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d240036a0 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.401+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4d2403f970 con 0x7f4d300a09c0 2023-09-30T22:29:31.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.401+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d300747f0 con 0x7f4d300a09c0 2023-09-30T22:29:31.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.406+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d24011bd0 con 0x7f4d300a09c0 2023-09-30T22:29:31.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:31.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.550+0000 7f301b574700 1 Processor -- start 2023-09-30T22:29:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.550+0000 7f301b574700 1 -- start start 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f301b574700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30140a0a30 0x7f30140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f301b574700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f30140a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f301b574700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140a2a70 con 0x7f3014133930 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f301b574700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30140a01d0 con 0x7f30140a0a30 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f301b574700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301409e2b0 con 0x7f301409d630 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3019310700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30140a0a30 0x7f30140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3018b0f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f30140a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3019b11700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3014133930 0x7f3014133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47616/0 (socket says 172.21.15.32:47616) 2023-09-30T22:29:31.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3019b11700 1 -- 172.21.15.32:0/965562117 learned_addr learned my addr 172.21.15.32:0/965562117 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:31.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3018b0f700 1 -- 172.21.15.32:0/965562117 >> v2:172.21.15.32:3300/0 conn(0x7f30140a0a30 msgr2=0x7f30140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3018b0f700 1 --2- 172.21.15.32:0/965562117 >> v2:172.21.15.32:3300/0 conn(0x7f30140a0a30 0x7f30140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3018b0f700 1 -- 172.21.15.32:0/965562117 >> v1:172.21.15.32:6789/0 conn(0x7f3014133930 legacy=0x7f3014133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3018b0f700 1 -- 172.21.15.32:0/965562117 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301409dc50 con 0x7f301409d630 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.551+0000 7f3019310700 1 --2- 172.21.15.32:0/965562117 >> v2:172.21.15.32:3300/0 conn(0x7f30140a0a30 0x7f30140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.552+0000 7f3018b0f700 1 --2- 172.21.15.32:0/965562117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f30140a1340 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3004009840 tx=0x7f3004009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe858b2ce8d61938 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.552+0000 7f300bfff700 1 -- 172.21.15.32:0/965562117 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004011830 con 0x7f301409d630 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.552+0000 7f300bfff700 1 -- 172.21.15.32:0/965562117 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3004011990 con 0x7f301409d630 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.552+0000 7f300bfff700 1 -- 172.21.15.32:0/965562117 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004011d20 con 0x7f301409d630 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 -- 172.21.15.32:0/965562117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 msgr2=0x7f30140a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 --2- 172.21.15.32:0/965562117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f30140a1340 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3004009840 tx=0x7f3004009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 -- 172.21.15.32:0/965562117 shutdown_connections 2023-09-30T22:29:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 --2- 172.21.15.32:0/965562117 >> v2:172.21.15.32:3300/0 conn(0x7f30140a0a30 0x7f30140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 --2- 172.21.15.32:0/965562117 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f30140a1340 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 -- 172.21.15.32:0/965562117 >> 172.21.15.32:0/965562117 conn(0x7f30140a3050 msgr2=0x7f30140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 -- 172.21.15.32:0/965562117 shutdown_connections 2023-09-30T22:29:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.553+0000 7f301b574700 1 -- 172.21.15.32:0/965562117 wait complete. 2023-09-30T22:29:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.554+0000 7f301b574700 1 Processor -- start 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.554+0000 7f301b574700 1 -- start start 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f301b574700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f3014121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f301b574700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 0x7f3014121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f301b574700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3014133930 0x7f3014121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f301b574700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30141225b0 con 0x7f30140a0a30 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f301b574700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3014124bd0 con 0x7f301409d630 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f301b574700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3014124630 con 0x7f3014133930 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f3019310700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f3014121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f3018b0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 0x7f3014121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f3018b0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 0x7f3014121640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48178/0 (socket says 172.21.15.32:48178) 2023-09-30T22:29:31.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f3018b0f700 1 -- 172.21.15.32:0/4207028163 learned_addr learned my addr 172.21.15.32:0/4207028163 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f3019310700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f3014121100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58902/0 (socket says 172.21.15.32:58902) 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.555+0000 7f3019b11700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3014133930 0x7f3014121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3018b0f700 1 -- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3014133930 msgr2=0x7f3014121b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3018b0f700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3014133930 0x7f3014121b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3018b0f700 1 -- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 msgr2=0x7f3014121100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3018b0f700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f3014121100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3018b0f700 1 -- 172.21.15.32:0/4207028163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100094a0 con 0x7f30140a0a30 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3019310700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f3014121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3018b0f700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 0x7f3014121640 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f3004009fd0 tx=0x7f3004004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040036a0 con 0x7f30140a0a30 2023-09-30T22:29:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300401f070 con 0x7f30140a0a30 2023-09-30T22:29:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004003a00 con 0x7f30140a0a30 2023-09-30T22:29:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30040094a0 con 0x7f30140a0a30 2023-09-30T22:29:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.556+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30140a0410 con 0x7f30140a0a30 2023-09-30T22:29:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.557+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3004003b60 con 0x7f30140a0a30 2023-09-30T22:29:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.557+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3004021070 con 0x7f30140a0a30 2023-09-30T22:29:31.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.558+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3014124850 con 0x7f30140a0a30 2023-09-30T22:29:31.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.567+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3014124850 con 0x7f30140a0a30 2023-09-30T22:29:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 Processor -- start 2023-09-30T22:29:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 -- start start 2023-09-30T22:29:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a1510 0x7fa2ac138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2ac139250 0x7fa2ac13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac09edd0 con 0x7fa2ac0a0a30 2023-09-30T22:29:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac0a18e0 con 0x7fa2ac139250 2023-09-30T22:29:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2b117b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac09e1f0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2aad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa2ac0a0a30 0x7fa2ac0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47628/0 (socket says 172.21.15.32:47628) 2023-09-30T22:29:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2aad9d700 1 -- 172.21.15.32:0/3783317221 learned_addr learned my addr 172.21.15.32:0/3783317221 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.595+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3783317221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a1510 0x7fa2ac138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2ab59e700 1 --2- 172.21.15.32:0/3783317221 >> v2:172.21.15.32:3300/0 conn(0x7fa2ac139250 0x7fa2ac13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3783317221 >> v2:172.21.15.32:3300/0 conn(0x7fa2ac139250 msgr2=0x7fa2ac13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3783317221 >> v2:172.21.15.32:3300/0 conn(0x7fa2ac139250 0x7fa2ac13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3783317221 >> v1:172.21.15.32:6789/0 conn(0x7fa2ac0a0a30 legacy=0x7fa2ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2aa59c700 1 -- 172.21.15.32:0/3783317221 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611585523 0 0) 0x7fa2ac09edd0 con 0x7fa2ac0a0a30 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3783317221 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac09dc50 con 0x7fa2ac0a1510 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2ab59e700 1 --2- 172.21.15.32:0/3783317221 >> v2:172.21.15.32:3300/0 conn(0x7fa2ac139250 0x7fa2ac13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3783317221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a1510 0x7fa2ac138d10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa294009870 tx=0x7fa294009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9f86f5c82df8a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2aa59c700 1 -- 172.21.15.32:0/3783317221 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294011830 con 0x7fa2ac0a1510 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2aa59c700 1 -- 172.21.15.32:0/3783317221 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa294011990 con 0x7fa2ac0a1510 2023-09-30T22:29:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.596+0000 7fa2aa59c700 1 -- 172.21.15.32:0/3783317221 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294011d20 con 0x7fa2ac0a1510 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 -- 172.21.15.32:0/3783317221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a1510 msgr2=0x7fa2ac138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3783317221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a1510 0x7fa2ac138d10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa294009870 tx=0x7fa294009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 -- 172.21.15.32:0/3783317221 shutdown_connections 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3783317221 >> v2:172.21.15.32:3300/0 conn(0x7fa2ac139250 0x7fa2ac13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3783317221 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a1510 0x7fa2ac138d10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 -- 172.21.15.32:0/3783317221 >> 172.21.15.32:0/3783317221 conn(0x7fa2ac0a6550 msgr2=0x7fa2ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 -- 172.21.15.32:0/3783317221 shutdown_connections 2023-09-30T22:29:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.597+0000 7fa2b117b700 1 -- 172.21.15.32:0/3783317221 wait complete. 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.599+0000 7fa2b117b700 1 Processor -- start 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.599+0000 7fa2b117b700 1 -- start start 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2b117b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a0a30 0x7fa2ac122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2b117b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 0x7fa2ac122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2b117b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2ac139250 0x7fa2ac123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2b117b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1389b0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2b117b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac124390 con 0x7fa2ac139250 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2b117b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac124c60 con 0x7fa2ac0a0a30 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 0x7fa2ac122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2aad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a0a30 0x7fa2ac122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 0x7fa2ac122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48198/0 (socket says 172.21.15.32:48198) 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3163043200 learned_addr learned my addr 172.21.15.32:0/3163043200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2ab59e700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2ac139250 0x7fa2ac123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a0a30 msgr2=0x7fa2ac122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a0a30 0x7fa2ac122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2ac139250 msgr2=0x7fa2ac123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2ac139250 0x7fa2ac123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 -- 172.21.15.32:0/3163043200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4009a70 con 0x7fa2ac0a1510 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.600+0000 7fa2a3fff700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 0x7fa2ac122cc0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa294000c00 tx=0x7fa294004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.601+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940045d0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.601+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29401f070 con 0x7fa2ac0a1510 2023-09-30T22:29:31.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.601+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2940094a0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.601+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940038a0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.601+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac125fc0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.601+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa294003a00 con 0x7fa2ac0a1510 2023-09-30T22:29:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.602+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa294021070 con 0x7fa2ac0a1510 2023-09-30T22:29:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.602+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac074ea0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.607+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa294015020 con 0x7fa2ac0a1510 2023-09-30T22:29:31.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.622+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d30133380 con 0x7f4d300a09c0 2023-09-30T22:29:31.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.623+0000 7f4d177fe700 1 -- 172.21.15.32:0/1210001553 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d2402a090 con 0x7f4d300a09c0 2023-09-30T22:29:31.623 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 msgr2=0x7f4d30132bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 0x7f4d30132bf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400e980 tx=0x7f4d2400ec90 comp rx=0 tx=0).stop 2023-09-30T22:29:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 shutdown_connections 2023-09-30T22:29:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d30133920 0x7f4d301c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d3009d5c0 0x7f4d301326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 --2- 172.21.15.32:0/1210001553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4d300a09c0 0x7f4d30132bf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 >> 172.21.15.32:0/1210001553 conn(0x7f4d300a2fe0 msgr2=0x7f4d301bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 shutdown_connections 2023-09-30T22:29:31.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.625+0000 7f4d35595700 1 -- 172.21.15.32:0/1210001553 wait complete. 2023-09-30T22:29:31.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:31.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.809+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3014124850 con 0x7f30140a0a30 2023-09-30T22:29:31.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.810+0000 7f3009ffb700 1 -- 172.21.15.32:0/4207028163 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3014124850 con 0x7f30140a0a30 2023-09-30T22:29:31.810 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 msgr2=0x7f3014121640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 0x7f3014121640 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f3004009fd0 tx=0x7f3004004750 comp rx=0 tx=0).stop 2023-09-30T22:29:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 shutdown_connections 2023-09-30T22:29:31.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3014133930 0x7f3014121b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f301409d630 0x7f3014121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 --2- 172.21.15.32:0/4207028163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30140a0a30 0x7f3014121640 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 >> 172.21.15.32:0/4207028163 conn(0x7f30140a3050 msgr2=0x7f3014133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 shutdown_connections 2023-09-30T22:29:31.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.812+0000 7f301b574700 1 -- 172.21.15.32:0/4207028163 wait complete. 2023-09-30T22:29:31.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.837+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa2ac0a28e0 con 0x7fa2ac0a1510 2023-09-30T22:29:31.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.838+0000 7fa2a37fe700 1 -- 172.21.15.32:0/3163043200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa29402b020 con 0x7fa2ac0a1510 2023-09-30T22:29:31.838 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 msgr2=0x7fa2ac122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 0x7fa2ac122cc0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa294000c00 tx=0x7fa294004750 comp rx=0 tx=0).stop 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 shutdown_connections 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa2ac139250 0x7fa2ac123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa2ac0a0a30 0x7fa2ac122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 --2- 172.21.15.32:0/3163043200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2ac0a1510 0x7fa2ac122cc0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:31.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 >> 172.21.15.32:0/3163043200 conn(0x7fa2ac0a6550 msgr2=0x7fa2ac0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 shutdown_connections 2023-09-30T22:29:31.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:31.840+0000 7fa2b117b700 1 -- 172.21.15.32:0/3163043200 wait complete. 2023-09-30T22:29:31.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:32.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 Processor -- start 2023-09-30T22:29:32.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 -- start start 2023-09-30T22:29:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 0x7ff5ec136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ec09ece0 con 0x7ff5ec06a770 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec09e760 con 0x7ff5ec06b250 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.502+0000 7ff5f464f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec09e040 con 0x7ff5ec136d80 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f1bea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 0x7ff5ec136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f23eb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff5ec06a770 0x7ff5ec06ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47644/0 (socket says 172.21.15.32:47644) 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f1bea700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 0x7ff5ec136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48212/0 (socket says 172.21.15.32:48212) 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57328/0 (socket says 172.21.15.32:57328) 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f23eb700 1 -- 172.21.15.32:0/77350287 learned_addr learned my addr 172.21.15.32:0/77350287 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 -- 172.21.15.32:0/77350287 >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 msgr2=0x7ff5ec136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 --2- 172.21.15.32:0/77350287 >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 0x7ff5ec136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 -- 172.21.15.32:0/77350287 >> v1:172.21.15.32:6789/0 conn(0x7ff5ec06a770 legacy=0x7ff5ec06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 -- 172.21.15.32:0/77350287 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec09f630 con 0x7ff5ec136d80 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f1bea700 1 --2- 172.21.15.32:0/77350287 >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 0x7ff5ec136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f13e9700 1 -- 172.21.15.32:0/77350287 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289197398 0 0) 0x7ff5ec09ece0 con 0x7ff5ec06a770 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f2bec700 1 --2- 172.21.15.32:0/77350287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec139160 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009d90 tx=0x7ff5e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=836b3fd6dead89d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f13e9700 1 -- 172.21.15.32:0/77350287 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400e070 con 0x7ff5ec136d80 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f13e9700 1 -- 172.21.15.32:0/77350287 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400bb20 con 0x7ff5ec136d80 2023-09-30T22:29:32.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.503+0000 7ff5f13e9700 1 -- 172.21.15.32:0/77350287 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4012430 con 0x7ff5ec136d80 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 -- 172.21.15.32:0/77350287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 msgr2=0x7ff5ec139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 --2- 172.21.15.32:0/77350287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec139160 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009d90 tx=0x7ff5e400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 -- 172.21.15.32:0/77350287 shutdown_connections 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 --2- 172.21.15.32:0/77350287 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec139160 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 --2- 172.21.15.32:0/77350287 >> v2:172.21.15.32:3300/0 conn(0x7ff5ec06b250 0x7ff5ec136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 -- 172.21.15.32:0/77350287 >> 172.21.15.32:0/77350287 conn(0x7ff5ec066f50 msgr2=0x7ff5ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 -- 172.21.15.32:0/77350287 shutdown_connections 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.504+0000 7ff5f464f700 1 -- 172.21.15.32:0/77350287 wait complete. 2023-09-30T22:29:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.505+0000 7ff5f464f700 1 Processor -- start 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.505+0000 7ff5f464f700 1 -- start start 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ec06a770 0x7ff5ec09ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 0x7ff5ec09b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec1c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1321d0 con 0x7ff5ec06a770 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec09a8d0 con 0x7ff5ec06b250 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec09b7e0 con 0x7ff5ec136d80 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 0x7ff5ec09b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f2bec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec1c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 0x7ff5ec09b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58922/0 (socket says 172.21.15.32:58922) 2023-09-30T22:29:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 -- 172.21.15.32:0/3715287145 learned_addr learned my addr 172.21.15.32:0/3715287145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f2bec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec1c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57338/0 (socket says 172.21.15.32:57338) 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f23eb700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ec06a770 0x7ff5ec09ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 -- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 msgr2=0x7ff5ec1c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec1c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 -- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ec06a770 msgr2=0x7ff5ec09ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ec06a770 0x7ff5ec09ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 -- 172.21.15.32:0/3715287145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0094a0 con 0x7ff5ec06b250 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f23eb700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ec06a770 0x7ff5ec09ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f2bec700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec1c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f1bea700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 0x7ff5ec09b2a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00ebe0 tx=0x7ff5dc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00c850 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc00c9b0 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e4009a70 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.506+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00cd40 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.507+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec071eb0 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.507+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff5dc013470 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.507+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff5dc03f7e0 con 0x7ff5ec06b250 2023-09-30T22:29:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.507+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec120f60 con 0x7ff5ec06b250 2023-09-30T22:29:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.512+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff5dc013720 con 0x7ff5ec06b250 2023-09-30T22:29:32.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.720+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff5ec074ea0 con 0x7ff5ec06b250 2023-09-30T22:29:32.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.721+0000 7ff5e2ffd700 1 -- 172.21.15.32:0/3715287145 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff5dc01b9c0 con 0x7ff5ec06b250 2023-09-30T22:29:32.721 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:32.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 msgr2=0x7ff5ec09b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 0x7ff5ec09b2a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00ebe0 tx=0x7ff5dc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 shutdown_connections 2023-09-30T22:29:32.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff5ec136d80 0x7ff5ec1c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5ec06a770 0x7ff5ec09ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 --2- 172.21.15.32:0/3715287145 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff5ec06b250 0x7ff5ec09b2a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 >> 172.21.15.32:0/3715287145 conn(0x7ff5ec066f50 msgr2=0x7ff5ec06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:32.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 shutdown_connections 2023-09-30T22:29:32.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.723+0000 7ff5f464f700 1 -- 172.21.15.32:0/3715287145 wait complete. 2023-09-30T22:29:32.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 Processor -- start 2023-09-30T22:29:32.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 -- start start 2023-09-30T22:29:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8690139170 0x7f869013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869009b120 con 0x7f869009d6f0 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869006b460 con 0x7f8690139170 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.777+0000 7f8695bed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f869009a7e0 con 0x7f869009e1d0 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f869009d6f0 0x7f869009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47660/0 (socket says 172.21.15.32:47660) 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8690139170 0x7f869013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868f7fe700 1 -- 172.21.15.32:0/3673216156 learned_addr learned my addr 172.21.15.32:0/3673216156 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868ffff700 1 -- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 msgr2=0x7f8690138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868ffff700 1 --2- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868ffff700 1 -- 172.21.15.32:0/3673216156 >> v1:172.21.15.32:6789/0 conn(0x7f869009d6f0 legacy=0x7f869009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868e7fc700 1 -- 172.21.15.32:0/3673216156 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746575818 0 0) 0x7f869009b120 con 0x7f869009d6f0 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868ffff700 1 -- 172.21.15.32:0/3673216156 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869009a500 con 0x7f8690139170 2023-09-30T22:29:32.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868effd700 1 --2- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868ffff700 1 --2- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8690139170 0x7f869013b550 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8688009dc0 tx=0x7f868800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5c12683c08e2a51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868e7fc700 1 -- 172.21.15.32:0/3673216156 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800e070 con 0x7f8690139170 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868e7fc700 1 -- 172.21.15.32:0/3673216156 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800bb20 con 0x7f8690139170 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.778+0000 7f868e7fc700 1 -- 172.21.15.32:0/3673216156 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688012430 con 0x7f8690139170 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 -- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8690139170 msgr2=0x7f869013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 --2- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8690139170 0x7f869013b550 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8688009dc0 tx=0x7f868800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 -- 172.21.15.32:0/3673216156 shutdown_connections 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 --2- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8690139170 0x7f869013b550 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 --2- 172.21.15.32:0/3673216156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 -- 172.21.15.32:0/3673216156 >> 172.21.15.32:0/3673216156 conn(0x7f8690066f50 msgr2=0x7f869009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:32.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 -- 172.21.15.32:0/3673216156 shutdown_connections 2023-09-30T22:29:32.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.779+0000 7f8695bed700 1 -- 172.21.15.32:0/3673216156 wait complete. 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.780+0000 7f8695bed700 1 Processor -- start 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.780+0000 7f8695bed700 1 -- start start 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f8695bed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f869009d6f0 0x7f86901205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f8695bed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f8695bed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8690139170 0x7f8690121070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f8695bed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8690123f30 con 0x7f8690139170 2023-09-30T22:29:32.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f8695bed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8690124110 con 0x7f869009d6f0 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f8695bed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86901230a0 con 0x7f869009e1d0 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f869009d6f0 0x7f86901205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f869009d6f0 0x7f86901205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58942/0 (socket says 172.21.15.32:58942) 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868f7fe700 1 -- 172.21.15.32:0/3773438391 learned_addr learned my addr 172.21.15.32:0/3773438391 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868ffff700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8690139170 0x7f8690121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 -- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f869009d6f0 msgr2=0x7f86901205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f869009d6f0 0x7f86901205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 -- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8690139170 msgr2=0x7f8690121070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8690139170 0x7f8690121070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 -- 172.21.15.32:0/3773438391 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780094a0 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868ffff700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8690139170 0x7f8690121070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.781+0000 7f868effd700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690120b30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9e0 tx=0x7f867800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.782+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800ee80 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.782+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867800c850 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.782+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8688009a70 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.782+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800cc00 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.782+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86900721b0 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.782+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f867800cd60 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.783+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f867803f730 con 0x7f869009e1d0 2023-09-30T22:29:32.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.783+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8690074ea0 con 0x7f869009e1d0 2023-09-30T22:29:32.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.788+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f867801bde0 con 0x7f869009e1d0 2023-09-30T22:29:32.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:32.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:32.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.965+0000 7fda4305d700 1 Processor -- start 2023-09-30T22:29:32.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.965+0000 7fda4305d700 1 -- start start 2023-09-30T22:29:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda4305d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c0a1510 0x7fda3c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda4305d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda3c139250 0x7fda3c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda4305d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c09edd0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda4305d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c0a18e0 con 0x7fda3c139250 2023-09-30T22:29:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda4305d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c09e1f0 con 0x7fda3c0a1510 2023-09-30T22:29:32.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda40df9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fda3c0a0a30 0x7fda3c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47674/0 (socket says 172.21.15.32:47674) 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda33fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c0a1510 0x7fda3c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda40df9700 1 -- 172.21.15.32:0/175884799 learned_addr learned my addr 172.21.15.32:0/175884799 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.966+0000 7fda415fa700 1 --2- 172.21.15.32:0/175884799 >> v2:172.21.15.32:3300/0 conn(0x7fda3c139250 0x7fda3c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda337fe700 1 -- 172.21.15.32:0/175884799 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517843446 0 0) 0x7fda3c09edd0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda33fff700 1 -- 172.21.15.32:0/175884799 >> v2:172.21.15.32:3300/0 conn(0x7fda3c139250 msgr2=0x7fda3c13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda33fff700 1 --2- 172.21.15.32:0/175884799 >> v2:172.21.15.32:3300/0 conn(0x7fda3c139250 0x7fda3c13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda33fff700 1 -- 172.21.15.32:0/175884799 >> v1:172.21.15.32:6789/0 conn(0x7fda3c0a0a30 legacy=0x7fda3c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda33fff700 1 -- 172.21.15.32:0/175884799 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c09dc50 con 0x7fda3c0a1510 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda415fa700 1 --2- 172.21.15.32:0/175884799 >> v2:172.21.15.32:3300/0 conn(0x7fda3c139250 0x7fda3c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda33fff700 1 --2- 172.21.15.32:0/175884799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c0a1510 0x7fda3c138d10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009870 tx=0x7fda2c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f77b5b1da970a92 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda337fe700 1 -- 172.21.15.32:0/175884799 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c011830 con 0x7fda3c0a1510 2023-09-30T22:29:32.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda337fe700 1 -- 172.21.15.32:0/175884799 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c011990 con 0x7fda3c0a1510 2023-09-30T22:29:32.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.967+0000 7fda337fe700 1 -- 172.21.15.32:0/175884799 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c011d20 con 0x7fda3c0a1510 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 -- 172.21.15.32:0/175884799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c0a1510 msgr2=0x7fda3c138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 --2- 172.21.15.32:0/175884799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c0a1510 0x7fda3c138d10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009870 tx=0x7fda2c009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 -- 172.21.15.32:0/175884799 shutdown_connections 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 --2- 172.21.15.32:0/175884799 >> v2:172.21.15.32:3300/0 conn(0x7fda3c139250 0x7fda3c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 --2- 172.21.15.32:0/175884799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c0a1510 0x7fda3c138d10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 -- 172.21.15.32:0/175884799 >> 172.21.15.32:0/175884799 conn(0x7fda3c0a6550 msgr2=0x7fda3c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 -- 172.21.15.32:0/175884799 shutdown_connections 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.968+0000 7fda4305d700 1 -- 172.21.15.32:0/175884799 wait complete. 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.969+0000 7fda4305d700 1 Processor -- start 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.969+0000 7fda4305d700 1 -- start start 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda4305d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 0x7fda3c122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda4305d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda3c0a1510 0x7fda3c122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda4305d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c139250 0x7fda3c123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda4305d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c1389b0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda4305d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c124390 con 0x7fda3c139250 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda4305d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c124c60 con 0x7fda3c0a1510 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 0x7fda3c122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda33fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda3c0a1510 0x7fda3c122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 0x7fda3c122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48248/0 (socket says 172.21.15.32:48248) 2023-09-30T22:29:32.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 -- 172.21.15.32:0/297109292 learned_addr learned my addr 172.21.15.32:0/297109292 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:32.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda415fa700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c139250 0x7fda3c123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 -- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda3c0a1510 msgr2=0x7fda3c122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda3c0a1510 0x7fda3c122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 -- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c139250 msgr2=0x7fda3c123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c139250 0x7fda3c123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda40df9700 1 -- 172.21.15.32:0/297109292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34009a70 con 0x7fda3c0a0a30 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.970+0000 7fda415fa700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c139250 0x7fda3c123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.971+0000 7fda40df9700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 0x7fda3c122780 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c520 tx=0x7fda3800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.971+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38004c80 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.971+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38004e00 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.971+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c0094a0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.971+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38007310 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.971+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c125fc0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.972+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c074ea0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.973+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda38007470 con 0x7fda3c0a0a30 2023-09-30T22:29:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.973+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fda3803f8b0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.977+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda380133c0 con 0x7fda3c0a0a30 2023-09-30T22:29:32.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.997+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f869006c360 con 0x7f869009e1d0 2023-09-30T22:29:32.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:32.998+0000 7f8687fff700 1 -- 172.21.15.32:0/3773438391 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8678017020 con 0x7f869009e1d0 2023-09-30T22:29:32.998 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 msgr2=0x7f8690120b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690120b30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f867800e9e0 tx=0x7f867800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 shutdown_connections 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8690139170 0x7f8690121070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f869009d6f0 0x7f86901205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 --2- 172.21.15.32:0/3773438391 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f869009e1d0 0x7f8690120b30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 >> 172.21.15.32:0/3773438391 conn(0x7f8690066f50 msgr2=0x7f8690070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 shutdown_connections 2023-09-30T22:29:33.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.000+0000 7f8695bed700 1 -- 172.21.15.32:0/3773438391 wait complete. 2023-09-30T22:29:33.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.024+0000 7f72ed56a700 1 Processor -- start 2023-09-30T22:29:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.024+0000 7f72ed56a700 1 -- start start 2023-09-30T22:29:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72ed56a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e809d5c0 0x7f72e8138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72ed56a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72ed56a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e809ea10 con 0x7f72e80a09c0 2023-09-30T22:29:33.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72ed56a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e809e210 con 0x7f72e80a14a0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72ed56a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80a26f0 con 0x7f72e809d5c0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e809d5c0 0x7f72e8138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72e80a09c0 0x7f72e80a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47690/0 (socket says 172.21.15.32:47690) 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e6ffd700 1 -- 172.21.15.32:0/1727691395 learned_addr learned my addr 172.21.15.32:0/1727691395 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e77fe700 1 -- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e809d5c0 msgr2=0x7f72e8138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e77fe700 1 --2- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e809d5c0 0x7f72e8138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e77fe700 1 -- 172.21.15.32:0/1727691395 >> v1:172.21.15.32:6789/0 conn(0x7f72e80a09c0 legacy=0x7f72e80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e5ffb700 1 -- 172.21.15.32:0/1727691395 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622334278 0 0) 0x7f72e809ea10 con 0x7f72e80a09c0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.025+0000 7f72e77fe700 1 -- 172.21.15.32:0/1727691395 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e809db70 con 0x7f72e80a14a0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72e77fe700 1 --2- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e813b2c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009870 tx=0x7f72e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4adbac050c879bd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:33.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72e5ffb700 1 -- 172.21.15.32:0/1727691395 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0011830 con 0x7f72e80a14a0 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72e5ffb700 1 -- 172.21.15.32:0/1727691395 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0011990 con 0x7f72e80a14a0 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72e5ffb700 1 -- 172.21.15.32:0/1727691395 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0011d20 con 0x7f72e80a14a0 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 -- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 msgr2=0x7f72e813b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e813b2c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009870 tx=0x7f72e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 -- 172.21.15.32:0/1727691395 shutdown_connections 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e813b2c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1727691395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e809d5c0 0x7f72e8138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 -- 172.21.15.32:0/1727691395 >> 172.21.15.32:0/1727691395 conn(0x7f72e80a2fe0 msgr2=0x7f72e80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 -- 172.21.15.32:0/1727691395 shutdown_connections 2023-09-30T22:29:33.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.026+0000 7f72ed56a700 1 -- 172.21.15.32:0/1727691395 wait complete. 2023-09-30T22:29:33.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.027+0000 7f72ed56a700 1 Processor -- start 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 -- start start 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 0x7f72e811c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 0x7f72e811ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e811d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e811f9a0 con 0x7f72e809d5c0 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e811fba0 con 0x7f72e80a14a0 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72ed56a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72e811ea90 con 0x7f72e80a09c0 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 0x7f72e811c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72e67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 0x7f72e811ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72e6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 0x7f72e811c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48258/0 (socket says 172.21.15.32:48258) 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72e67fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 0x7f72e811ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57384/0 (socket says 172.21.15.32:57384) 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72e6ffd700 1 -- 172.21.15.32:0/1945073548 learned_addr learned my addr 172.21.15.32:0/1945073548 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.028+0000 7f72e77fe700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e811d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e6ffd700 1 -- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 msgr2=0x7f72e811ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e6ffd700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 0x7f72e811ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e6ffd700 1 -- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 msgr2=0x7f72e811d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e6ffd700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e811d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e6ffd700 1 -- 172.21.15.32:0/1945073548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00094a0 con 0x7f72e809d5c0 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e67fc700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 0x7f72e811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e77fe700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e811d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:33.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72e6ffd700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 0x7f72e811c7b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f72d800cc80 tx=0x7f72d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ab00 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d800ac60 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e811dd50 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80070d0 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.029+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e809cec0 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.030+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72d8007230 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.030+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e80747f0 con 0x7f72e809d5c0 2023-09-30T22:29:33.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.030+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f72d803f630 con 0x7f72e809d5c0 2023-09-30T22:29:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.035+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72d801a8b0 con 0x7f72e809d5c0 2023-09-30T22:29:33.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.207+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fda3c0a28e0 con 0x7fda3c0a0a30 2023-09-30T22:29:33.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.207+0000 7fda317fa700 1 -- 172.21.15.32:0/297109292 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda38011360 con 0x7fda3c0a0a30 2023-09-30T22:29:33.208 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 msgr2=0x7fda3c122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 0x7fda3c122780 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fda3800c520 tx=0x7fda3800c830 comp rx=0 tx=0).stop 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 shutdown_connections 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fda3c139250 0x7fda3c123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda3c0a0a30 0x7fda3c122780 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 --2- 172.21.15.32:0/297109292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fda3c0a1510 0x7fda3c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 >> 172.21.15.32:0/297109292 conn(0x7fda3c0a6550 msgr2=0x7fda3c0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:33.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 shutdown_connections 2023-09-30T22:29:33.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.210+0000 7fda4305d700 1 -- 172.21.15.32:0/297109292 wait complete. 2023-09-30T22:29:33.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.240+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f72e8072980 con 0x7f72e809d5c0 2023-09-30T22:29:33.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.241+0000 7f72df7fe700 1 -- 172.21.15.32:0/1945073548 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72d801a2c0 con 0x7f72e809d5c0 2023-09-30T22:29:33.241 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 msgr2=0x7f72e811c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 0x7f72e811c7b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f72d800cc80 tx=0x7f72d800cf90 comp rx=0 tx=0).stop 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 shutdown_connections 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72e80a14a0 0x7f72e811d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72e809d5c0 0x7f72e811c7b0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 --2- 172.21.15.32:0/1945073548 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72e80a09c0 0x7f72e811ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 >> 172.21.15.32:0/1945073548 conn(0x7f72e80a2fe0 msgr2=0x7f72e8136d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 shutdown_connections 2023-09-30T22:29:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.243+0000 7f72ed56a700 1 -- 172.21.15.32:0/1945073548 wait complete. 2023-09-30T22:29:33.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:33.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:33.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.884+0000 7fe125ecc700 1 Processor -- start 2023-09-30T22:29:33.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.885+0000 7fe125ecc700 1 -- start start 2023-09-30T22:29:33.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.885+0000 7fe125ecc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe12009e930 0x7fe12009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.885+0000 7fe125ecc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009f240 0x7fe1201317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.885+0000 7fe125ecc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe120070940 con 0x7fe12009f650 2023-09-30T22:29:33.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.885+0000 7fe125ecc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12006fd60 con 0x7fe12009e930 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.885+0000 7fe125ecc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12006fa80 con 0x7fe12009f240 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe12009f650 0x7fe120131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47698/0 (socket says 172.21.15.32:47698) 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11ffff700 1 -- 172.21.15.32:0/948593740 learned_addr learned my addr 172.21.15.32:0/948593740 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11effd700 1 --2- 172.21.15.32:0/948593740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009f240 0x7fe1201317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11f7fe700 1 --2- 172.21.15.32:0/948593740 >> v2:172.21.15.32:3300/0 conn(0x7fe12009e930 0x7fe12009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11effd700 1 -- 172.21.15.32:0/948593740 >> v2:172.21.15.32:3300/0 conn(0x7fe12009e930 msgr2=0x7fe12009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11effd700 1 --2- 172.21.15.32:0/948593740 >> v2:172.21.15.32:3300/0 conn(0x7fe12009e930 0x7fe12009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11effd700 1 -- 172.21.15.32:0/948593740 >> v1:172.21.15.32:6789/0 conn(0x7fe12009f650 legacy=0x7fe120131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11effd700 1 -- 172.21.15.32:0/948593740 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12009fb60 con 0x7fe12009f240 2023-09-30T22:29:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11f7fe700 1 --2- 172.21.15.32:0/948593740 >> v2:172.21.15.32:3300/0 conn(0x7fe12009e930 0x7fe12009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11effd700 1 --2- 172.21.15.32:0/948593740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009f240 0x7fe1201317f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe108009840 tx=0x7fe108009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8905743e6611eda9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11e7fc700 1 -- 172.21.15.32:0/948593740 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108011830 con 0x7fe12009f240 2023-09-30T22:29:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11e7fc700 1 -- 172.21.15.32:0/948593740 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108011990 con 0x7fe12009f240 2023-09-30T22:29:33.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.886+0000 7fe11e7fc700 1 -- 172.21.15.32:0/948593740 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108011d20 con 0x7fe12009f240 2023-09-30T22:29:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 -- 172.21.15.32:0/948593740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009f240 msgr2=0x7fe1201317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 --2- 172.21.15.32:0/948593740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009f240 0x7fe1201317f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe108009840 tx=0x7fe108009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 -- 172.21.15.32:0/948593740 shutdown_connections 2023-09-30T22:29:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 --2- 172.21.15.32:0/948593740 >> v2:172.21.15.32:3300/0 conn(0x7fe12009e930 0x7fe12009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 --2- 172.21.15.32:0/948593740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009f240 0x7fe1201317f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 -- 172.21.15.32:0/948593740 >> 172.21.15.32:0/948593740 conn(0x7fe1200a4210 msgr2=0x7fe1200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:33.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 -- 172.21.15.32:0/948593740 shutdown_connections 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.887+0000 7fe125ecc700 1 -- 172.21.15.32:0/948593740 wait complete. 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.888+0000 7fe125ecc700 1 Processor -- start 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.888+0000 7fe125ecc700 1 -- start start 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe125ecc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 0x7fe1201a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe125ecc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe12009f240 0x7fe1201a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe125ecc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe12009f650 0x7fe1201c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe125ecc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201b5370 con 0x7fe12009f650 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe125ecc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201a5750 con 0x7fe12009e930 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe125ecc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201a1ff0 con 0x7fe12009f240 2023-09-30T22:29:33.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 0x7fe1201a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe12009f240 0x7fe1201a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe12009f650 0x7fe1201c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 0x7fe1201a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:58998/0 (socket says 172.21.15.32:58998) 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11f7fe700 1 -- 172.21.15.32:0/1694979335 learned_addr learned my addr 172.21.15.32:0/1694979335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 -- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe12009f240 msgr2=0x7fe1201a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe12009f240 0x7fe1201a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 -- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 msgr2=0x7fe1201a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:33.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 0x7fe1201a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 -- 172.21.15.32:0/1694979335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11000d940 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11effd700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe12009f240 0x7fe1201a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11f7fe700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 0x7fe1201a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe11ffff700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe12009f650 0x7fe1201c3cd0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe11800cad0 tx=0x7fe11800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.889+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11800a900 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.890+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11800aa60 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.890+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1080094a0 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.890+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11800add0 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.890+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1200a0190 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.890+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe11801b3b0 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.890+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe11803f170 con 0x7fe12009f650 2023-09-30T22:29:33.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.891+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120074ea0 con 0x7fe12009f650 2023-09-30T22:29:33.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:33.896+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe118017020 con 0x7fe12009f650 2023-09-30T22:29:34.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:34.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.102+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe1201a2e50 con 0x7fe12009f650 2023-09-30T22:29:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.103+0000 7fe117fff700 1 -- 172.21.15.32:0/1694979335 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1180192c0 con 0x7fe12009f650 2023-09-30T22:29:34.103 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe12009f650 msgr2=0x7fe1201c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe12009f650 0x7fe1201c3cd0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe11800cad0 tx=0x7fe11800cde0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 shutdown_connections 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe12009f650 0x7fe1201c3cd0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe12009e930 0x7fe1201a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 --2- 172.21.15.32:0/1694979335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe12009f240 0x7fe1201a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 >> 172.21.15.32:0/1694979335 conn(0x7fe1200a4210 msgr2=0x7fe1201b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 shutdown_connections 2023-09-30T22:29:34.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.105+0000 7fe125ecc700 1 -- 172.21.15.32:0/1694979335 wait complete. 2023-09-30T22:29:34.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:34.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.154+0000 7f203c292700 1 Processor -- start 2023-09-30T22:29:34.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.154+0000 7f203c292700 1 -- start start 2023-09-30T22:29:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203c292700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 0x7f2034138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203c292700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2034139170 0x7f203413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203c292700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203409b120 con 0x7f203409d6f0 2023-09-30T22:29:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203c292700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203406b460 con 0x7f2034139170 2023-09-30T22:29:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203c292700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f203409a7e0 con 0x7f203409e1d0 2023-09-30T22:29:34.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203982d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 0x7f2034138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a02e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f203409d6f0 0x7f203409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47714/0 (socket says 172.21.15.32:47714) 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203982d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 0x7f2034138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57398/0 (socket says 172.21.15.32:57398) 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a82f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2034139170 0x7f203413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a02e700 1 -- 172.21.15.32:0/72072380 learned_addr learned my addr 172.21.15.32:0/72072380 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a82f700 1 -- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 msgr2=0x7f2034138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a82f700 1 --2- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 0x7f2034138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203902c700 1 -- 172.21.15.32:0/72072380 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465198046 0 0) 0x7f203409b120 con 0x7f203409d6f0 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a82f700 1 -- 172.21.15.32:0/72072380 >> v1:172.21.15.32:6789/0 conn(0x7f203409d6f0 legacy=0x7f203409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203a82f700 1 -- 172.21.15.32:0/72072380 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203409a500 con 0x7f2034139170 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.155+0000 7f203982d700 1 --2- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 0x7f2034138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:34.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.156+0000 7f203a82f700 1 --2- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2034139170 0x7f203413b550 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f202c009dc0 tx=0x7f202c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5dc9796f38ced1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.156+0000 7f203902c700 1 -- 172.21.15.32:0/72072380 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00e070 con 0x7f2034139170 2023-09-30T22:29:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.156+0000 7f203902c700 1 -- 172.21.15.32:0/72072380 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00bb20 con 0x7f2034139170 2023-09-30T22:29:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.156+0000 7f203902c700 1 -- 172.21.15.32:0/72072380 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c012430 con 0x7f2034139170 2023-09-30T22:29:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.156+0000 7f203c292700 1 -- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2034139170 msgr2=0x7f203413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.156+0000 7f203c292700 1 --2- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2034139170 0x7f203413b550 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f202c009dc0 tx=0x7f202c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:34.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.157+0000 7f203c292700 1 -- 172.21.15.32:0/72072380 shutdown_connections 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.157+0000 7f203c292700 1 --2- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2034139170 0x7f203413b550 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.157+0000 7f203c292700 1 --2- 172.21.15.32:0/72072380 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409e1d0 0x7f2034138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.157+0000 7f203c292700 1 -- 172.21.15.32:0/72072380 >> 172.21.15.32:0/72072380 conn(0x7f2034066f50 msgr2=0x7f203409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.157+0000 7f203c292700 1 -- 172.21.15.32:0/72072380 shutdown_connections 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.157+0000 7f203c292700 1 -- 172.21.15.32:0/72072380 wait complete. 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 Processor -- start 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 -- start start 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409d6f0 0x7f20341205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 0x7f2034120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2034139170 0x7f2034121060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20341221f0 con 0x7f2034139170 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2034122810 con 0x7f203409e1d0 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203c292700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f203409c780 con 0x7f203409d6f0 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203982d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 0x7f2034120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203a02e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409d6f0 0x7f20341205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203a82f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2034139170 0x7f2034121060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203982d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 0x7f2034120b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59012/0 (socket says 172.21.15.32:59012) 2023-09-30T22:29:34.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203982d700 1 -- 172.21.15.32:0/3995062599 learned_addr learned my addr 172.21.15.32:0/3995062599 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.158+0000 7f203a02e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409d6f0 0x7f20341205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57402/0 (socket says 172.21.15.32:57402) 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203982d700 1 -- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409d6f0 msgr2=0x7f20341205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203982d700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409d6f0 0x7f20341205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203982d700 1 -- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2034139170 msgr2=0x7f2034121060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203982d700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2034139170 0x7f2034121060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203982d700 1 -- 172.21.15.32:0/3995062599 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280094a0 con 0x7f203409e1d0 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203982d700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 0x7f2034120b20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f202800ec10 tx=0x7f202800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800c7f0 con 0x7f203409e1d0 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800c950 con 0x7f203409e1d0 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c009a70 con 0x7f203409e1d0 2023-09-30T22:29:34.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800ccc0 con 0x7f203409e1d0 2023-09-30T22:29:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.159+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2034123e20 con 0x7f203409e1d0 2023-09-30T22:29:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.160+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2028013470 con 0x7f203409e1d0 2023-09-30T22:29:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.160+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2034074ea0 con 0x7f203409e1d0 2023-09-30T22:29:34.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.160+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f202803fc50 con 0x7f203409e1d0 2023-09-30T22:29:34.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.165+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f202803fe60 con 0x7f203409e1d0 2023-09-30T22:29:34.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:34.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.365+0000 7f03f4b9f700 1 Processor -- start 2023-09-30T22:29:34.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.365+0000 7f03f4b9f700 1 -- start start 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03f4b9f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f01338a0 0x7f03f0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03f4b9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03f4b9f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f006ef30 con 0x7f03f009de50 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03f4b9f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f006fef0 con 0x7f03f009e930 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03f4b9f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03f01334b0 con 0x7f03f01338a0 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03edd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03f009de50 0x7f03f009e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47722/0 (socket says 172.21.15.32:47722) 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03edd9b700 1 -- 172.21.15.32:0/455463229 learned_addr learned my addr 172.21.15.32:0/455463229 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03ee59c700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f01338a0 0x7f03f0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03eed9d700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03ee59c700 1 -- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 msgr2=0x7f03f013b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03ee59c700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f013b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03ee59c700 1 -- 172.21.15.32:0/455463229 >> v1:172.21.15.32:6789/0 conn(0x7f03f009de50 legacy=0x7f03f009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03ed59a700 1 -- 172.21.15.32:0/455463229 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242039209 0 0) 0x7f03f006ef30 con 0x7f03f009de50 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.366+0000 7f03ee59c700 1 -- 172.21.15.32:0/455463229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f006fc30 con 0x7f03f01338a0 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.367+0000 7f03ee59c700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f01338a0 0x7f03f0133c70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009870 tx=0x7f03e0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89b62bde0748f79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.367+0000 7f03ed59a700 1 -- 172.21.15.32:0/455463229 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0011830 con 0x7f03f01338a0 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.367+0000 7f03ed59a700 1 -- 172.21.15.32:0/455463229 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0011990 con 0x7f03f01338a0 2023-09-30T22:29:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.367+0000 7f03ed59a700 1 -- 172.21.15.32:0/455463229 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0011d20 con 0x7f03f01338a0 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 -- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f01338a0 msgr2=0x7f03f0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f01338a0 0x7f03f0133c70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009870 tx=0x7f03e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 -- 172.21.15.32:0/455463229 shutdown_connections 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f013b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/455463229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f01338a0 0x7f03f0133c70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 -- 172.21.15.32:0/455463229 >> 172.21.15.32:0/455463229 conn(0x7f03f00a2aa0 msgr2=0x7f03f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 -- 172.21.15.32:0/455463229 shutdown_connections 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.368+0000 7f03f4b9f700 1 -- 172.21.15.32:0/455463229 wait complete. 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.369+0000 7f03f4b9f700 1 Processor -- start 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.369+0000 7f03f4b9f700 1 -- start start 2023-09-30T22:29:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03f4b9f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 0x7f03f011d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03f4b9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f011db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03f4b9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03f01338a0 0x7f03f011e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03f4b9f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f011f200 con 0x7f03f01338a0 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03f4b9f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f011f820 con 0x7f03f009e930 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03f4b9f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0120450 con 0x7f03f009de50 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 0x7f03f011d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 0x7f03f011d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57418/0 (socket says 172.21.15.32:57418) 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03edd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f011db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 -- 172.21.15.32:0/731179497 learned_addr learned my addr 172.21.15.32:0/731179497 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03eed9d700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03f01338a0 0x7f03f011e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 -- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 msgr2=0x7f03f011db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f011db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 -- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03f01338a0 msgr2=0x7f03f011e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03f01338a0 0x7f03f011e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 -- 172.21.15.32:0/731179497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00094a0 con 0x7f03f009de50 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03edd9b700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f011db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.370+0000 7f03ee59c700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 0x7f03f011d5f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009840 tx=0x7f03e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.371+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0004510 con 0x7f03f009de50 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.371+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00036a0 con 0x7f03f009de50 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.371+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f011ea70 con 0x7f03f009de50 2023-09-30T22:29:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.371+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0003a50 con 0x7f03f009de50 2023-09-30T22:29:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.371+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f00a0d50 con 0x7f03f009de50 2023-09-30T22:29:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.372+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03e0003bb0 con 0x7f03f009de50 2023-09-30T22:29:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.372+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f03e00409e0 con 0x7f03f009de50 2023-09-30T22:29:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.372+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f011a470 con 0x7f03f009de50 2023-09-30T22:29:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.377+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03e001a420 con 0x7f03f009de50 2023-09-30T22:29:34.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.384+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f20341229e0 con 0x7f203409e1d0 2023-09-30T22:29:34.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.384+0000 7f2026ffd700 1 -- 172.21.15.32:0/3995062599 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2028013720 con 0x7f203409e1d0 2023-09-30T22:29:34.384 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 msgr2=0x7f2034120b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 0x7f2034120b20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f202800ec10 tx=0x7f202800ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 shutdown_connections 2023-09-30T22:29:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2034139170 0x7f2034121060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f203409d6f0 0x7f20341205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 --2- 172.21.15.32:0/3995062599 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f203409e1d0 0x7f2034120b20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 >> 172.21.15.32:0/3995062599 conn(0x7f2034066f50 msgr2=0x7f2034136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 shutdown_connections 2023-09-30T22:29:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.388+0000 7f203c292700 1 -- 172.21.15.32:0/3995062599 wait complete. 2023-09-30T22:29:34.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 Processor -- start 2023-09-30T22:29:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 -- start start 2023-09-30T22:29:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83dc0a0a30 0x7f83dc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc09edd0 con 0x7f83dc0a1340 2023-09-30T22:29:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc09e1f0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.411+0000 7f83e29f3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc09dc50 con 0x7f83dc134170 2023-09-30T22:29:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83e0f90700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83dc0a0a30 0x7f83dc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83dc0a0a30 0x7f83dc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48308/0 (socket says 172.21.15.32:48308) 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83e0f90700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57428/0 (socket says 172.21.15.32:57428) 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 -- 172.21.15.32:0/3599309142 learned_addr learned my addr 172.21.15.32:0/3599309142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83db7fe700 1 --1- 172.21.15.32:0/3599309142 >> v1:172.21.15.32:6789/0 conn(0x7f83dc0a1340 0x7f83dc133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47726/0 (socket says 172.21.15.32:47726) 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 -- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 msgr2=0x7f83dc13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 --2- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 -- 172.21.15.32:0/3599309142 >> v1:172.21.15.32:6789/0 conn(0x7f83dc0a1340 legacy=0x7f83dc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 -- 172.21.15.32:0/3599309142 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0a1fe0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83daffd700 1 -- 172.21.15.32:0/3599309142 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449244043 0 0) 0x7f83dc09edd0 con 0x7f83dc0a1340 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83e0f90700 1 --2- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83dbfff700 1 --2- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc0a0e00 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0098f0 tx=0x7f83cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c1a25bae108372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83daffd700 1 -- 172.21.15.32:0/3599309142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc011830 con 0x7f83dc0a0a30 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83daffd700 1 -- 172.21.15.32:0/3599309142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc011990 con 0x7f83dc0a0a30 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.412+0000 7f83daffd700 1 -- 172.21.15.32:0/3599309142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc011d20 con 0x7f83dc0a0a30 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 -- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 msgr2=0x7f83dc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 --2- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc0a0e00 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0098f0 tx=0x7f83cc009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 -- 172.21.15.32:0/3599309142 shutdown_connections 2023-09-30T22:29:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 --2- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 --2- 172.21.15.32:0/3599309142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc0a0e00 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 -- 172.21.15.32:0/3599309142 >> 172.21.15.32:0/3599309142 conn(0x7f83dc0a6550 msgr2=0x7f83dc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 -- 172.21.15.32:0/3599309142 shutdown_connections 2023-09-30T22:29:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.413+0000 7f83e29f3700 1 -- 172.21.15.32:0/3599309142 wait complete. 2023-09-30T22:29:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.414+0000 7f83e29f3700 1 Processor -- start 2023-09-30T22:29:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.414+0000 7f83e29f3700 1 -- start start 2023-09-30T22:29:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e29f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc1c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e29f3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83dc0a1340 0x7f83dc1c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e29f3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc1c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e29f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc125260 con 0x7f83dc0a0a30 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e29f3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc125e60 con 0x7f83dc0a1340 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e29f3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc1244f0 con 0x7f83dc134170 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc1c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc1c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48316/0 (socket says 172.21.15.32:48316) 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83e0f90700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc1c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 -- 172.21.15.32:0/1403643922 learned_addr learned my addr 172.21.15.32:0/1403643922 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83db7fe700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83dc0a1340 0x7f83dc1c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 -- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 msgr2=0x7f83dc1c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc1c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 -- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83dc0a1340 msgr2=0x7f83dc1c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83dc0a1340 0x7f83dc1c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.415+0000 7f83dbfff700 1 -- 172.21.15.32:0/1403643922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4009a70 con 0x7f83dc0a0a30 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.416+0000 7f83dbfff700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc1c84f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f83dc09e330 tx=0x7f83cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.416+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0043c0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.416+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc01f070 con 0x7f83dc0a0a30 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.416+0000 7f83e29f3700 1 -- 172.21.15.32:0/1403643922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc009550 con 0x7f83dc0a0a30 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.416+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0036a0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.416+0000 7f83e29f3700 1 -- 172.21.15.32:0/1403643922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc0a25e0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.417+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83cc021030 con 0x7f83dc0a0a30 2023-09-30T22:29:34.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.417+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b40052e0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.417+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f83cc040b60 con 0x7f83dc0a0a30 2023-09-30T22:29:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.422+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f83cc0199a0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.590+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f03f0074ea0 con 0x7f03f009de50 2023-09-30T22:29:34.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.591+0000 7f03deffd700 1 -- 172.21.15.32:0/731179497 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03e0015020 con 0x7f03f009de50 2023-09-30T22:29:34.591 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 msgr2=0x7f03f011d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 0x7f03f011d5f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009840 tx=0x7f03e0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 shutdown_connections 2023-09-30T22:29:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03f01338a0 0x7f03f011e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03f009de50 0x7f03f011d5f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 --2- 172.21.15.32:0/731179497 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03f009e930 0x7f03f011db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 >> 172.21.15.32:0/731179497 conn(0x7f03f00a2aa0 msgr2=0x7f03f0132b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 shutdown_connections 2023-09-30T22:29:34.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.593+0000 7f03f4b9f700 1 -- 172.21.15.32:0/731179497 wait complete. 2023-09-30T22:29:34.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.627+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f83b4005590 con 0x7f83dc0a0a30 2023-09-30T22:29:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.628+0000 7f83d8ff9700 1 -- 172.21.15.32:0/1403643922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83cc0193b0 con 0x7f83dc0a0a30 2023-09-30T22:29:34.628 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 msgr2=0x7f83dc1c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc1c84f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f83dc09e330 tx=0x7f83cc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 shutdown_connections 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f83dc134170 0x7f83dc1c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83dc0a0a30 0x7f83dc1c84f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 --2- 172.21.15.32:0/1403643922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83dc0a1340 0x7f83dc1c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 >> 172.21.15.32:0/1403643922 conn(0x7f83dc0a6550 msgr2=0x7f83dc09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 shutdown_connections 2023-09-30T22:29:34.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:34.630+0000 7f83c27fc700 1 -- 172.21.15.32:0/1403643922 wait complete. 2023-09-30T22:29:34.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:35.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:35.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.268+0000 7f4f59a90700 1 Processor -- start 2023-09-30T22:29:35.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.268+0000 7f4f59a90700 1 -- start start 2023-09-30T22:29:35.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.268+0000 7f4f59a90700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409e1d0 0x7f4f54135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.268+0000 7f4f59a90700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f54135e50 0x7f4f54138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f59a90700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5409f130 con 0x7f4f5409d6f0 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f59a90700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5409f520 con 0x7f4f54135e50 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f59a90700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5409e930 con 0x7f4f5409e1d0 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f537fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f5409d6f0 0x7f4f5409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47732/0 (socket says 172.21.15.32:47732) 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409e1d0 0x7f4f54135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f537fe700 1 -- 172.21.15.32:0/1167342217 learned_addr learned my addr 172.21.15.32:0/1167342217 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f53fff700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f54135e50 0x7f4f54138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f527fc700 1 -- 172.21.15.32:0/1167342217 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579882550 0 0) 0x7f4f5409f130 con 0x7f4f5409d6f0 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f52ffd700 1 -- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f54135e50 msgr2=0x7f4f54138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f52ffd700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f54135e50 0x7f4f54138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f52ffd700 1 -- 172.21.15.32:0/1167342217 >> v1:172.21.15.32:6789/0 conn(0x7f4f5409d6f0 legacy=0x7f4f5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f52ffd700 1 -- 172.21.15.32:0/1167342217 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5409b2d0 con 0x7f4f5409e1d0 2023-09-30T22:29:35.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f53fff700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f54135e50 0x7f4f54138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f52ffd700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409e1d0 0x7f4f54135910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009870 tx=0x7f4f3c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a3c320142ff50a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f527fc700 1 -- 172.21.15.32:0/1167342217 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c011830 con 0x7f4f5409e1d0 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f527fc700 1 -- 172.21.15.32:0/1167342217 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c011990 con 0x7f4f5409e1d0 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.269+0000 7f4f527fc700 1 -- 172.21.15.32:0/1167342217 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c011d20 con 0x7f4f5409e1d0 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 -- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409e1d0 msgr2=0x7f4f54135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409e1d0 0x7f4f54135910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009870 tx=0x7f4f3c009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 -- 172.21.15.32:0/1167342217 shutdown_connections 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f54135e50 0x7f4f54138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 --2- 172.21.15.32:0/1167342217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409e1d0 0x7f4f54135910 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 -- 172.21.15.32:0/1167342217 >> 172.21.15.32:0/1167342217 conn(0x7f4f54066f50 msgr2=0x7f4f5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 -- 172.21.15.32:0/1167342217 shutdown_connections 2023-09-30T22:29:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.270+0000 7f4f59a90700 1 -- 172.21.15.32:0/1167342217 wait complete. 2023-09-30T22:29:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.271+0000 7f4f59a90700 1 Processor -- start 2023-09-30T22:29:35.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.271+0000 7f4f59a90700 1 -- start start 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f59a90700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 0x7f4f54131280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f59a90700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 0x7f4f541317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f59a90700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f54135e50 0x7f4f5411d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f59a90700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54120a70 con 0x7f4f54135e50 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f59a90700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54120c50 con 0x7f4f5409e1d0 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f59a90700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5411f130 con 0x7f4f5409d6f0 2023-09-30T22:29:35.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 0x7f4f54131280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 0x7f4f541317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 0x7f4f54131280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57458/0 (socket says 172.21.15.32:57458) 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 0x7f4f541317c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59060/0 (socket says 172.21.15.32:59060) 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 -- 172.21.15.32:0/3600236696 learned_addr learned my addr 172.21.15.32:0/3600236696 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f53fff700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f54135e50 0x7f4f5411d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 -- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 msgr2=0x7f4f541317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 0x7f4f541317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 -- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f54135e50 msgr2=0x7f4f5411d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f54135e50 0x7f4f5411d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 -- 172.21.15.32:0/3600236696 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c009a70 con 0x7f4f5409d6f0 2023-09-30T22:29:35.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f53fff700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f54135e50 0x7f4f5411d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f52ffd700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 0x7f4f541317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.272+0000 7f4f537fe700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 0x7f4f54131280 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c550 tx=0x7f4f4400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.273+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44004c80 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.273+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f44004e00 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.273+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c0094a0 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.273+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44007310 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.273+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f54131d00 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.273+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f44007470 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.274+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4f4403f2d0 con 0x7f4f5409d6f0 2023-09-30T22:29:35.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.274+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f54074ea0 con 0x7f4f5409d6f0 2023-09-30T22:29:35.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.279+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f440123b0 con 0x7f4f5409d6f0 2023-09-30T22:29:35.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:35.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.486+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4f5411f7d0 con 0x7f4f5409d6f0 2023-09-30T22:29:35.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.486+0000 7f4f4bfff700 1 -- 172.21.15.32:0/3600236696 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f44011360 con 0x7f4f5409d6f0 2023-09-30T22:29:35.487 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 msgr2=0x7f4f54131280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 0x7f4f54131280 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400c550 tx=0x7f4f4400c860 comp rx=0 tx=0).stop 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 shutdown_connections 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f54135e50 0x7f4f5411d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f5409d6f0 0x7f4f54131280 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 --2- 172.21.15.32:0/3600236696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f5409e1d0 0x7f4f541317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 >> 172.21.15.32:0/3600236696 conn(0x7f4f54066f50 msgr2=0x7f4f54071070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 shutdown_connections 2023-09-30T22:29:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.489+0000 7f4f59a90700 1 -- 172.21.15.32:0/3600236696 wait complete. 2023-09-30T22:29:35.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.540+0000 7f234e46b700 1 Processor -- start 2023-09-30T22:29:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.540+0000 7f234e46b700 1 -- start start 2023-09-30T22:29:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234e46b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f234809d630 0x7f23480a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234e46b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2348138d80 0x7f234813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234e46b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23480a2a70 con 0x7f23480a0a30 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234e46b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23480a01d0 con 0x7f234809d630 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234e46b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f234809e2b0 con 0x7f2348138d80 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f23477fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f234809d630 0x7f23480a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234ca08700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2348138d80 0x7f234813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f2347fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23480a0a30 0x7f23480a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47734/0 (socket says 172.21.15.32:47734) 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f2347fff700 1 -- 172.21.15.32:0/1317925705 learned_addr learned my addr 172.21.15.32:0/1317925705 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234ca08700 1 -- 172.21.15.32:0/1317925705 >> v2:172.21.15.32:3300/0 conn(0x7f234809d630 msgr2=0x7f23480a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:35.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234ca08700 1 --2- 172.21.15.32:0/1317925705 >> v2:172.21.15.32:3300/0 conn(0x7f234809d630 0x7f23480a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234ca08700 1 -- 172.21.15.32:0/1317925705 >> v1:172.21.15.32:6789/0 conn(0x7f23480a0a30 legacy=0x7f23480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f234ca08700 1 -- 172.21.15.32:0/1317925705 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234809dc50 con 0x7f2348138d80 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f23477fe700 1 --2- 172.21.15.32:0/1317925705 >> v2:172.21.15.32:3300/0 conn(0x7f234809d630 0x7f23480a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.541+0000 7f2346ffd700 1 -- 172.21.15.32:0/1317925705 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401100818 0 0) 0x7f23480a2a70 con 0x7f23480a0a30 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234ca08700 1 --2- 172.21.15.32:0/1317925705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2348138d80 0x7f234813b160 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2340009d90 tx=0x7f234000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb9e90c8bff03747 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f2346ffd700 1 -- 172.21.15.32:0/1317925705 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000e070 con 0x7f2348138d80 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f2346ffd700 1 -- 172.21.15.32:0/1317925705 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000bb20 con 0x7f2348138d80 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f2346ffd700 1 -- 172.21.15.32:0/1317925705 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340012430 con 0x7f2348138d80 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234e46b700 1 -- 172.21.15.32:0/1317925705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2348138d80 msgr2=0x7f234813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234e46b700 1 --2- 172.21.15.32:0/1317925705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2348138d80 0x7f234813b160 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2340009d90 tx=0x7f234000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234e46b700 1 -- 172.21.15.32:0/1317925705 shutdown_connections 2023-09-30T22:29:35.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234e46b700 1 --2- 172.21.15.32:0/1317925705 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2348138d80 0x7f234813b160 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234e46b700 1 --2- 172.21.15.32:0/1317925705 >> v2:172.21.15.32:3300/0 conn(0x7f234809d630 0x7f23480a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.542+0000 7f234e46b700 1 -- 172.21.15.32:0/1317925705 >> 172.21.15.32:0/1317925705 conn(0x7f23480a3050 msgr2=0x7f23480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.543+0000 7f234e46b700 1 -- 172.21.15.32:0/1317925705 shutdown_connections 2023-09-30T22:29:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.543+0000 7f234e46b700 1 -- 172.21.15.32:0/1317925705 wait complete. 2023-09-30T22:29:35.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.543+0000 7f234e46b700 1 Processor -- start 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 -- start start 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 0x7f2348133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 0x7f2348133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2348138d80 0x7f2348134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2348137180 con 0x7f234809d630 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2348136070 con 0x7f2348138d80 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234e46b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2348136530 con 0x7f23480a0a30 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f2347fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 0x7f2348133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f23477fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 0x7f2348133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f23477fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 0x7f2348133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57466/0 (socket says 172.21.15.32:57466) 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f23477fe700 1 -- 172.21.15.32:0/954407608 learned_addr learned my addr 172.21.15.32:0/954407608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f2347fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 0x7f2348133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48360/0 (socket says 172.21.15.32:48360) 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.544+0000 7f234ca08700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2348138d80 0x7f2348134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2347fff700 1 -- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 msgr2=0x7f2348133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2347fff700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 0x7f2348133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2347fff700 1 -- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2348138d80 msgr2=0x7f2348134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2347fff700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2348138d80 0x7f2348134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2347fff700 1 -- 172.21.15.32:0/954407608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0094a0 con 0x7f234809d630 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f23477fe700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 0x7f2348133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f234ca08700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2348138d80 0x7f2348134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2347fff700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 0x7f2348133700 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f233800c470 tx=0x7f233800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338004c80 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338004de0 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2340009a70 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338007310 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.545+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2348136be0 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.546+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2338007470 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.546+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f233803f370 con 0x7f234809d630 2023-09-30T22:29:35.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.546+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2348137d60 con 0x7f234809d630 2023-09-30T22:29:35.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.551+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2338007720 con 0x7f234809d630 2023-09-30T22:29:35.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:35.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.747+0000 7f44a76f8700 1 Processor -- start 2023-09-30T22:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.747+0000 7f44a76f8700 1 -- start start 2023-09-30T22:29:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a76f8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44a009d300 0x7f44a009d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a76f8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a01335f0 0x7f44a0133a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a76f8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a0099c30 con 0x7f44a0133f40 2023-09-30T22:29:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a76f8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a009ab40 con 0x7f44a009d300 2023-09-30T22:29:35.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a76f8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44a009ced0 con 0x7f44a01335f0 2023-09-30T22:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a5494700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f44a009d300 0x7f44a009d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a5c95700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f44a0133f40 0x7f44a013a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47744/0 (socket says 172.21.15.32:47744) 2023-09-30T22:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a5c95700 1 -- 172.21.15.32:0/2897430421 learned_addr learned my addr 172.21.15.32:0/2897430421 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.748+0000 7f44a4c93700 1 --2- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a01335f0 0x7f44a0133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f44a5494700 1 -- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a01335f0 msgr2=0x7f44a0133a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f44a5494700 1 --2- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a01335f0 0x7f44a0133a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f44a5494700 1 -- 172.21.15.32:0/2897430421 >> v1:172.21.15.32:6789/0 conn(0x7f44a0133f40 legacy=0x7f44a013a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f4497fff700 1 -- 172.21.15.32:0/2897430421 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644823056 0 0) 0x7f44a0099c30 con 0x7f44a0133f40 2023-09-30T22:29:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f44a5494700 1 -- 172.21.15.32:0/2897430421 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0099680 con 0x7f44a009d300 2023-09-30T22:29:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f44a5494700 1 --2- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a009d300 0x7f44a009d6d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f449c009840 tx=0x7f449c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eccc88531d0b586 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f4497fff700 1 -- 172.21.15.32:0/2897430421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c011830 con 0x7f44a009d300 2023-09-30T22:29:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f4497fff700 1 -- 172.21.15.32:0/2897430421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c011990 con 0x7f44a009d300 2023-09-30T22:29:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.749+0000 7f4497fff700 1 -- 172.21.15.32:0/2897430421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c011d20 con 0x7f44a009d300 2023-09-30T22:29:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 -- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a009d300 msgr2=0x7f44a009d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 --2- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a009d300 0x7f44a009d6d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f449c009840 tx=0x7f449c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 -- 172.21.15.32:0/2897430421 shutdown_connections 2023-09-30T22:29:35.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 --2- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a009d300 0x7f44a009d6d0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 --2- 172.21.15.32:0/2897430421 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a01335f0 0x7f44a0133a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 -- 172.21.15.32:0/2897430421 >> 172.21.15.32:0/2897430421 conn(0x7f44a00a2360 msgr2=0x7f44a00a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 -- 172.21.15.32:0/2897430421 shutdown_connections 2023-09-30T22:29:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.750+0000 7f44a76f8700 1 -- 172.21.15.32:0/2897430421 wait complete. 2023-09-30T22:29:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.751+0000 7f44a76f8700 1 Processor -- start 2023-09-30T22:29:35.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.751+0000 7f44a76f8700 1 -- start start 2023-09-30T22:29:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a76f8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a009d300 0x7f44a012eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a76f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 0x7f44a012f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a76f8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44a0133f40 0x7f44a012f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a76f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a01308b0 con 0x7f44a01335f0 2023-09-30T22:29:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a76f8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00a0770 con 0x7f44a0133f40 2023-09-30T22:29:35.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a76f8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44a009c200 con 0x7f44a009d300 2023-09-30T22:29:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 0x7f44a012f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a5c95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44a0133f40 0x7f44a012f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 0x7f44a012f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48364/0 (socket says 172.21.15.32:48364) 2023-09-30T22:29:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 -- 172.21.15.32:0/3811728670 learned_addr learned my addr 172.21.15.32:0/3811728670 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 -- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a009d300 msgr2=0x7f44a012eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a009d300 0x7f44a012eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 -- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44a0133f40 msgr2=0x7f44a012f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44a0133f40 0x7f44a012f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 -- 172.21.15.32:0/3811728670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c0094a0 con 0x7f44a01335f0 2023-09-30T22:29:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a5c95700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44a0133f40 0x7f44a012f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.752+0000 7f44a4c93700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 0x7f44a012f430 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f449000e9b0 tx=0x7f449000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.753+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490009710 con 0x7f44a01335f0 2023-09-30T22:29:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.753+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449000c850 con 0x7f44a01335f0 2023-09-30T22:29:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.753+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449000cbc0 con 0x7f44a01335f0 2023-09-30T22:29:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.753+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a0133280 con 0x7f44a01335f0 2023-09-30T22:29:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.753+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a01328a0 con 0x7f44a01335f0 2023-09-30T22:29:35.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.754+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a012ba00 con 0x7f44a01335f0 2023-09-30T22:29:35.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.754+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f449000cd20 con 0x7f44a01335f0 2023-09-30T22:29:35.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.754+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f449001a360 con 0x7f44a01335f0 2023-09-30T22:29:35.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.763+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f23480747f0 con 0x7f234809d630 2023-09-30T22:29:35.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.763+0000 7f2344ff9700 1 -- 172.21.15.32:0/954407608 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2338011360 con 0x7f234809d630 2023-09-30T22:29:35.765 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:35.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.763+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44900102b0 con 0x7f44a01335f0 2023-09-30T22:29:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 msgr2=0x7f2348133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 0x7f2348133700 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f233800c470 tx=0x7f233800c780 comp rx=0 tx=0).stop 2023-09-30T22:29:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 shutdown_connections 2023-09-30T22:29:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2348138d80 0x7f2348134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f234809d630 0x7f2348133700 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 --2- 172.21.15.32:0/954407608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f23480a0a30 0x7f2348133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 >> 172.21.15.32:0/954407608 conn(0x7f23480a3050 msgr2=0x7f234809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 shutdown_connections 2023-09-30T22:29:35.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.766+0000 7f234e46b700 1 -- 172.21.15.32:0/954407608 wait complete. 2023-09-30T22:29:35.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:35.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.792+0000 7f8010f17700 1 Processor -- start 2023-09-30T22:29:35.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.792+0000 7f8010f17700 1 -- start start 2023-09-30T22:29:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f8010f17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f8010f17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f8010f17700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c09b120 con 0x7f800c09e410 2023-09-30T22:29:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f8010f17700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c06b460 con 0x7f800c09e000 2023-09-30T22:29:35.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f8010f17700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f800c09a7e0 con 0x7f800c09d6f0 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f800a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f800ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f800c09e410 0x7f800c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47758/0 (socket says 172.21.15.32:47758) 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f800ad9d700 1 -- 172.21.15.32:0/2448957641 learned_addr learned my addr 172.21.15.32:0/2448957641 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.793+0000 7f8009d9b700 1 --2- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800959a700 1 -- 172.21.15.32:0/2448957641 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602851505 0 0) 0x7f800c09b120 con 0x7f800c09e410 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800959a700 1 -- 172.21.15.32:0/2448957641 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff40035b0 con 0x7f800c09e410 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800a59c700 1 -- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09e000 msgr2=0x7f800c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800a59c700 1 --2- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800a59c700 1 -- 172.21.15.32:0/2448957641 >> v1:172.21.15.32:6789/0 conn(0x7f800c09e410 legacy=0x7f800c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800a59c700 1 -- 172.21.15.32:0/2448957641 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c09a500 con 0x7f800c09d6f0 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800a59c700 1 --2- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09d6f0 0x7f800c09dac0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098a0 tx=0x7f7ffc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4a7a53c8210c17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800959a700 1 -- 172.21.15.32:0/2448957641 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011830 con 0x7f800c09d6f0 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800959a700 1 -- 172.21.15.32:0/2448957641 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc011990 con 0x7f800c09d6f0 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.794+0000 7f800959a700 1 -- 172.21.15.32:0/2448957641 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc011d00 con 0x7f800c09d6f0 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 -- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09d6f0 msgr2=0x7f800c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 --2- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09d6f0 0x7f800c09dac0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098a0 tx=0x7f7ffc009bb0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 -- 172.21.15.32:0/2448957641 shutdown_connections 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 --2- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09d6f0 0x7f800c09dac0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 --2- 172.21.15.32:0/2448957641 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09e000 0x7f800c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 -- 172.21.15.32:0/2448957641 >> 172.21.15.32:0/2448957641 conn(0x7f800c066f50 msgr2=0x7f800c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 -- 172.21.15.32:0/2448957641 shutdown_connections 2023-09-30T22:29:35.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.795+0000 7f8010f17700 1 -- 172.21.15.32:0/2448957641 wait complete. 2023-09-30T22:29:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.796+0000 7f8010f17700 1 Processor -- start 2023-09-30T22:29:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.796+0000 7f8010f17700 1 -- start start 2023-09-30T22:29:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8010f17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09d6f0 0x7f800c1c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8010f17700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 0x7f800c1c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8010f17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f800c09e410 0x7f800c1c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8010f17700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a63f0 con 0x7f800c09e410 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8010f17700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a7be0 con 0x7f800c09d6f0 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8010f17700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0a2d80 con 0x7f800c09e000 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f800c09e410 0x7f800c1c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8009d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 0x7f800c1c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09d6f0 0x7f800c1c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800ad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f800c09e410 0x7f800c1c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48380/0 (socket says 172.21.15.32:48380) 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8009d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 0x7f800c1c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57512/0 (socket says 172.21.15.32:57512) 2023-09-30T22:29:35.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800ad9d700 1 -- 172.21.15.32:0/745976182 learned_addr learned my addr 172.21.15.32:0/745976182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 -- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 msgr2=0x7f800c1c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 0x7f800c1c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 -- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f800c09e410 msgr2=0x7f800c1c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f800c09e410 0x7f800c1c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 -- 172.21.15.32:0/745976182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc0094a0 con 0x7f800c09d6f0 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f8009d9b700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 0x7f800c1c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.797+0000 7f800a59c700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09d6f0 0x7f800c1c7f40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc016000 tx=0x7f7ffc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.798+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc004550 con 0x7f800c09d6f0 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.798+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc01f070 con 0x7f800c09d6f0 2023-09-30T22:29:35.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.798+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c09c710 con 0x7f800c09d6f0 2023-09-30T22:29:35.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.798+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0038a0 con 0x7f800c09d6f0 2023-09-30T22:29:35.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.798+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1a8910 con 0x7f800c09d6f0 2023-09-30T22:29:35.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.798+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ffc003a00 con 0x7f800c09d6f0 2023-09-30T22:29:35.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.799+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7ffc021070 con 0x7f800c09d6f0 2023-09-30T22:29:35.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.799+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c074ea0 con 0x7f800c09d6f0 2023-09-30T22:29:35.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.804+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ffc02b020 con 0x7f800c09d6f0 2023-09-30T22:29:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.978+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f44a0074ea0 con 0x7f44a01335f0 2023-09-30T22:29:35.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.979+0000 7f4495ffb700 1 -- 172.21.15.32:0/3811728670 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4490017030 con 0x7f44a01335f0 2023-09-30T22:29:35.979 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 msgr2=0x7f44a012f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 0x7f44a012f430 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f449000e9b0 tx=0x7f449000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 shutdown_connections 2023-09-30T22:29:35.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44a0133f40 0x7f44a012f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44a009d300 0x7f44a012eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 --2- 172.21.15.32:0/3811728670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44a01335f0 0x7f44a012f430 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 >> 172.21.15.32:0/3811728670 conn(0x7f44a00a2360 msgr2=0x7f44a0098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 shutdown_connections 2023-09-30T22:29:35.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:35.981+0000 7f44a76f8700 1 -- 172.21.15.32:0/3811728670 wait complete. 2023-09-30T22:29:35.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:36.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.007+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f800c06c360 con 0x7f800c09d6f0 2023-09-30T22:29:36.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.007+0000 7f7ffaffd700 1 -- 172.21.15.32:0/745976182 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ffc015070 con 0x7f800c09d6f0 2023-09-30T22:29:36.008 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09d6f0 msgr2=0x7f800c1c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09d6f0 0x7f800c1c7f40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc016000 tx=0x7f7ffc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 shutdown_connections 2023-09-30T22:29:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f800c09e410 0x7f800c1c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f800c09d6f0 0x7f800c1c7f40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 --2- 172.21.15.32:0/745976182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f800c09e000 0x7f800c1c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.009+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 >> 172.21.15.32:0/745976182 conn(0x7f800c066f50 msgr2=0x7f800c1bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:36.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.010+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 shutdown_connections 2023-09-30T22:29:36.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.010+0000 7f8010f17700 1 -- 172.21.15.32:0/745976182 wait complete. 2023-09-30T22:29:36.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:36.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:36.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.649+0000 7f78caf0d700 1 Processor -- start 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.649+0000 7f78caf0d700 1 -- start start 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.649+0000 7f78caf0d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78c406b250 0x7f78c4136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.649+0000 7f78caf0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78caf0d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c409ece0 con 0x7f78c406a770 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78caf0d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c409e760 con 0x7f78c406b250 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78caf0d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78c409e040 con 0x7f78c4136d80 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c3fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78c406b250 0x7f78c4136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c8ca9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f78c406a770 0x7f78c406ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47768/0 (socket says 172.21.15.32:47768) 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c8ca9700 1 -- 172.21.15.32:0/2568770322 learned_addr learned my addr 172.21.15.32:0/2568770322 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:36.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c94aa700 1 --2- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c3fff700 1 -- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 msgr2=0x7f78c4139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:36.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c3fff700 1 --2- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c3fff700 1 -- 172.21.15.32:0/2568770322 >> v1:172.21.15.32:6789/0 conn(0x7f78c406a770 legacy=0x7f78c406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c3fff700 1 -- 172.21.15.32:0/2568770322 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c409f630 con 0x7f78c406b250 2023-09-30T22:29:36.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c37fe700 1 -- 172.21.15.32:0/2568770322 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541037155 0 0) 0x7f78c409ece0 con 0x7f78c406a770 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c3fff700 1 --2- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406b250 0x7f78c4136840 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009840 tx=0x7f78b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97f9227e6ad8307 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c37fe700 1 -- 172.21.15.32:0/2568770322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0011830 con 0x7f78c406b250 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c37fe700 1 -- 172.21.15.32:0/2568770322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b0011990 con 0x7f78c406b250 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.650+0000 7f78c37fe700 1 -- 172.21.15.32:0/2568770322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0011d20 con 0x7f78c406b250 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 -- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406b250 msgr2=0x7f78c4136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 --2- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406b250 0x7f78c4136840 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009840 tx=0x7f78b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 -- 172.21.15.32:0/2568770322 shutdown_connections 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 --2- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 --2- 172.21.15.32:0/2568770322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406b250 0x7f78c4136840 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 -- 172.21.15.32:0/2568770322 >> 172.21.15.32:0/2568770322 conn(0x7f78c4066f50 msgr2=0x7f78c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:36.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 -- 172.21.15.32:0/2568770322 shutdown_connections 2023-09-30T22:29:36.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.651+0000 7f78caf0d700 1 -- 172.21.15.32:0/2568770322 wait complete. 2023-09-30T22:29:36.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.652+0000 7f78caf0d700 1 Processor -- start 2023-09-30T22:29:36.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.652+0000 7f78caf0d700 1 -- start start 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78caf0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 0x7f78c41356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78caf0d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78c406b250 0x7f78c4135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78caf0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4121970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78caf0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4123d90 con 0x7f78c406a770 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78caf0d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41368b0 con 0x7f78c406b250 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78caf0d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41234e0 con 0x7f78c4136d80 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 0x7f78c41356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78c406b250 0x7f78c4135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c94aa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4121970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 0x7f78c41356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48404/0 (socket says 172.21.15.32:48404) 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 -- 172.21.15.32:0/1048381563 learned_addr learned my addr 172.21.15.32:0/1048381563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 -- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 msgr2=0x7f78c4121970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4121970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 -- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78c406b250 msgr2=0x7f78c4135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78c406b250 0x7f78c4135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 -- 172.21.15.32:0/1048381563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00094a0 con 0x7f78c406a770 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c94aa700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4121970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c3fff700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78c406b250 0x7f78c4135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.653+0000 7f78c8ca9700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 0x7f78c41356d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c550 tx=0x7f78b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.654+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8004c80 con 0x7f78c406a770 2023-09-30T22:29:36.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.654+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8004e00 con 0x7f78c406a770 2023-09-30T22:29:36.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.654+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c409a5f0 con 0x7f78c406a770 2023-09-30T22:29:36.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.654+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8007310 con 0x7f78c406a770 2023-09-30T22:29:36.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.654+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c4136150 con 0x7f78c406a770 2023-09-30T22:29:36.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.654+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f78b8007470 con 0x7f78c406a770 2023-09-30T22:29:36.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.655+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f78b803f290 con 0x7f78c406a770 2023-09-30T22:29:36.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.655+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c4074ea0 con 0x7f78c406a770 2023-09-30T22:29:36.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.660+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f78b80123b0 con 0x7f78c406a770 2023-09-30T22:29:36.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.867+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78c4136380 con 0x7f78c406a770 2023-09-30T22:29:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.868+0000 7f78c17fa700 1 -- 172.21.15.32:0/1048381563 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f78b80102c0 con 0x7f78c406a770 2023-09-30T22:29:36.868 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 msgr2=0x7f78c41356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 0x7f78c41356d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c550 tx=0x7f78b800c860 comp rx=0 tx=0).stop 2023-09-30T22:29:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 shutdown_connections 2023-09-30T22:29:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f78c4136d80 0x7f78c4121970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78c406a770 0x7f78c41356d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 --2- 172.21.15.32:0/1048381563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78c406b250 0x7f78c4135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 >> 172.21.15.32:0/1048381563 conn(0x7f78c4066f50 msgr2=0x7f78c40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 shutdown_connections 2023-09-30T22:29:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.870+0000 7f78caf0d700 1 -- 172.21.15.32:0/1048381563 wait complete. 2023-09-30T22:29:36.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.926+0000 7f4f8dc47700 1 Processor -- start 2023-09-30T22:29:36.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.926+0000 7f4f8dc47700 1 -- start start 2023-09-30T22:29:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f8dc47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f881337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f8dc47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 0x7f4f881306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f8dc47700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8806b990 con 0x7f4f88133e30 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f8dc47700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880a0590 con 0x7f4f88133420 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f8dc47700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8809fb80 con 0x7f4f88134260 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 0x7f4f881306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f881337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f881337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59116/0 (socket says 172.21.15.32:59116) 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f86ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f88133e30 0x7f4f8812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47770/0 (socket says 172.21.15.32:47770) 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 0x7f4f881306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57524/0 (socket says 172.21.15.32:57524) 2023-09-30T22:29:36.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f877fe700 1 -- 172.21.15.32:0/3097911744 learned_addr learned my addr 172.21.15.32:0/3097911744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 -- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 msgr2=0x7f4f881337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 --2- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f881337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 -- 172.21.15.32:0/3097911744 >> v1:172.21.15.32:6789/0 conn(0x7f4f88133e30 legacy=0x7f4f8812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 -- 172.21.15.32:0/3097911744 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880a0030 con 0x7f4f88134260 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f877fe700 1 --2- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f881337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f867fc700 1 -- 172.21.15.32:0/3097911744 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823835355 0 0) 0x7f4f8806b990 con 0x7f4f88133e30 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.927+0000 7f4f87fff700 1 --2- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 0x7f4f881306a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009dc0 tx=0x7f4f8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbc5664b6a6b95d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f867fc700 1 -- 172.21.15.32:0/3097911744 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000e070 con 0x7f4f88134260 2023-09-30T22:29:36.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f867fc700 1 -- 172.21.15.32:0/3097911744 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8000bb20 con 0x7f4f88134260 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f867fc700 1 -- 172.21.15.32:0/3097911744 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80012430 con 0x7f4f88134260 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 -- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 msgr2=0x7f4f881306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 0x7f4f881306a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009dc0 tx=0x7f4f8000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 -- 172.21.15.32:0/3097911744 shutdown_connections 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88134260 0x7f4f881306a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/3097911744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f881337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 -- 172.21.15.32:0/3097911744 >> 172.21.15.32:0/3097911744 conn(0x7f4f880a4210 msgr2=0x7f4f880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:36.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 -- 172.21.15.32:0/3097911744 shutdown_connections 2023-09-30T22:29:36.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.928+0000 7f4f8dc47700 1 -- 172.21.15.32:0/3097911744 wait complete. 2023-09-30T22:29:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.930+0000 7f4f8dc47700 1 Processor -- start 2023-09-30T22:29:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.930+0000 7f4f8dc47700 1 -- start start 2023-09-30T22:29:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f8dc47700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f8809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f8dc47700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88133e30 0x7f4f8809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f8dc47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f88134260 0x7f4f8809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f8dc47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8809de30 con 0x7f4f88134260 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f8dc47700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8811edf0 con 0x7f4f88133420 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f8dc47700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8811faf0 con 0x7f4f88133e30 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f8809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f86ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88133e30 0x7f4f8809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f8809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59118/0 (socket says 172.21.15.32:59118) 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 -- 172.21.15.32:0/263748490 learned_addr learned my addr 172.21.15.32:0/263748490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f87fff700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f88134260 0x7f4f8809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 -- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88133e30 msgr2=0x7f4f8809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88133e30 0x7f4f8809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 -- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f88134260 msgr2=0x7f4f8809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f88134260 0x7f4f8809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 -- 172.21.15.32:0/263748490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f78009550 con 0x7f4f88133420 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f86ffd700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88133e30 0x7f4f8809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:36.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f877fe700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f8809c480 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800ebe0 tx=0x7f4f7800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800c850 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7800c9b0 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.931+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800cd40 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.932+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f80009a70 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.932+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8811fdb0 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.932+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f78013470 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.932+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4f7803fbe0 con 0x7f4f88133420 2023-09-30T22:29:36.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.932+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f88074ea0 con 0x7f4f88133420 2023-09-30T22:29:36.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:36.937+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f7801b3b0 con 0x7f4f88133420 2023-09-30T22:29:36.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:37.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.138+0000 7f446df2d700 1 Processor -- start 2023-09-30T22:29:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.138+0000 7f446df2d700 1 -- start start 2023-09-30T22:29:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f446df2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 0x7f44680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f446df2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f446809d630 0x7f44680a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f446df2d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4468133930 0x7f4468133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f446df2d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44680a2a70 con 0x7f4468133930 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f446df2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44680a01d0 con 0x7f44680a0a30 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f446df2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f446809e2b0 con 0x7f446809d630 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f44677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 0x7f44680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f4467fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4468133930 0x7f4468133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f44677fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 0x7f44680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59122/0 (socket says 172.21.15.32:59122) 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f44677fe700 1 -- 172.21.15.32:0/110981529 learned_addr learned my addr 172.21.15.32:0/110981529 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:37.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.139+0000 7f4466ffd700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f446809d630 0x7f44680a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44677fe700 1 -- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f446809d630 msgr2=0x7f44680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44677fe700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f446809d630 0x7f44680a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44677fe700 1 -- 172.21.15.32:0/110981529 >> v2:172.21.15.32:3300/0 conn(0x7f4468133930 msgr2=0x7f4468133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44677fe700 1 --2- 172.21.15.32:0/110981529 >> v2:172.21.15.32:3300/0 conn(0x7f4468133930 0x7f4468133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44677fe700 1 -- 172.21.15.32:0/110981529 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446809dc50 con 0x7f44680a0a30 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f4466ffd700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f446809d630 0x7f44680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44677fe700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 0x7f44680a0e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f44580098f0 tx=0x7f4458009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde1e012f8f74e74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44667fc700 1 -- 172.21.15.32:0/110981529 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458011830 con 0x7f44680a0a30 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44667fc700 1 -- 172.21.15.32:0/110981529 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458011990 con 0x7f44680a0a30 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.140+0000 7f44667fc700 1 -- 172.21.15.32:0/110981529 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458011d20 con 0x7f44680a0a30 2023-09-30T22:29:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 -- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 msgr2=0x7f44680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 0x7f44680a0e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f44580098f0 tx=0x7f4458009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 -- 172.21.15.32:0/110981529 shutdown_connections 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44680a0a30 0x7f44680a0e00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 --2- 172.21.15.32:0/110981529 >> v2:172.21.15.32:3300/0 conn(0x7f4468133930 0x7f4468133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 --2- 172.21.15.32:0/110981529 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f446809d630 0x7f44680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 -- 172.21.15.32:0/110981529 >> 172.21.15.32:0/110981529 conn(0x7f44680a3050 msgr2=0x7f44680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 -- 172.21.15.32:0/110981529 shutdown_connections 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.141+0000 7f446df2d700 1 -- 172.21.15.32:0/110981529 wait complete. 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.142+0000 7f446df2d700 1 Processor -- start 2023-09-30T22:29:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.143+0000 7f446df2d700 1 -- start start 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.143+0000 7f446df2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f446809d630 0x7f446811f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.143+0000 7f446df2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 0x7f446811f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f446df2d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4468133930 0x7f44681c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f446df2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44680a4150 con 0x7f44680a0a30 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f446df2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44680a0410 con 0x7f446809d630 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f446df2d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f446811edd0 con 0x7f4468133930 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 0x7f446811f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 0x7f446811f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48434/0 (socket says 172.21.15.32:48434) 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 -- 172.21.15.32:0/3053054323 learned_addr learned my addr 172.21.15.32:0/3053054323 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4467fff700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4468133930 0x7f44681c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f44677fe700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f446809d630 0x7f446811f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 -- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4468133930 msgr2=0x7f44681c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4468133930 0x7f44681c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 -- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f446809d630 msgr2=0x7f446811f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f446809d630 0x7f446811f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 -- 172.21.15.32:0/3053054323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0094a0 con 0x7f44680a0a30 2023-09-30T22:29:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4f8806b250 con 0x7f4f88133420 2023-09-30T22:29:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.144+0000 7f4466ffd700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 0x7f446811f800 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f445c00ebb0 tx=0x7f445c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.145+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00c850 con 0x7f44680a0a30 2023-09-30T22:29:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.145+0000 7f4f7ffff700 1 -- 172.21.15.32:0/263748490 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f78011b00 con 0x7f4f88133420 2023-09-30T22:29:37.148 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.145+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c00c9b0 con 0x7f44680a0a30 2023-09-30T22:29:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.145+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4458009550 con 0x7f44680a0a30 2023-09-30T22:29:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.145+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4468121de0 con 0x7f44680a0a30 2023-09-30T22:29:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.145+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00cd40 con 0x7f44680a0a30 2023-09-30T22:29:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.146+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44480052e0 con 0x7f44680a0a30 2023-09-30T22:29:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.146+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f445c013470 con 0x7f44680a0a30 2023-09-30T22:29:37.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.146+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f445c03f7e0 con 0x7f44680a0a30 2023-09-30T22:29:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 msgr2=0x7f4f8809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f8809c480 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800ebe0 tx=0x7f4f7800eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 shutdown_connections 2023-09-30T22:29:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f88134260 0x7f4f8809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f88133420 0x7f4f8809c480 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 --2- 172.21.15.32:0/263748490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f88133e30 0x7f4f8809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 >> 172.21.15.32:0/263748490 conn(0x7f4f880a4210 msgr2=0x7f4f880a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 shutdown_connections 2023-09-30T22:29:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.147+0000 7f4f8dc47700 1 -- 172.21.15.32:0/263748490 wait complete. 2023-09-30T22:29:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.151+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f445c01b3b0 con 0x7f44680a0a30 2023-09-30T22:29:37.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.182+0000 7f57fa3d9700 1 Processor -- start 2023-09-30T22:29:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57fa3d9700 1 -- start start 2023-09-30T22:29:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57fa3d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57f4136820 0x7f57f4138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57fa3d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 0x7f57f413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57fa3d9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f406bc00 con 0x7f57f4133a80 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57fa3d9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f406c570 con 0x7f57f4136820 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57fa3d9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40a0040 con 0x7f57f4139140 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57f8976700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 0x7f57f413b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57f37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57f4136820 0x7f57f4138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57f3fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57f4133a80 0x7f57f4133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47776/0 (socket says 172.21.15.32:47776) 2023-09-30T22:29:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57f8976700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 0x7f57f413b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57580/0 (socket says 172.21.15.32:57580) 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.183+0000 7f57f3fff700 1 -- 172.21.15.32:0/446881691 learned_addr learned my addr 172.21.15.32:0/446881691 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f8976700 1 -- 172.21.15.32:0/446881691 >> v2:172.21.15.32:3300/0 conn(0x7f57f4136820 msgr2=0x7f57f4138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f8976700 1 --2- 172.21.15.32:0/446881691 >> v2:172.21.15.32:3300/0 conn(0x7f57f4136820 0x7f57f4138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f8976700 1 -- 172.21.15.32:0/446881691 >> v1:172.21.15.32:6789/0 conn(0x7f57f4133a80 legacy=0x7f57f4133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f8976700 1 -- 172.21.15.32:0/446881691 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4133860 con 0x7f57f4139140 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f8976700 1 --2- 172.21.15.32:0/446881691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 0x7f57f413b520 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009d90 tx=0x7f57ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc2adedb74721fdd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:37.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f2ffd700 1 -- 172.21.15.32:0/446881691 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00e070 con 0x7f57f4139140 2023-09-30T22:29:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f2ffd700 1 -- 172.21.15.32:0/446881691 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec00bb20 con 0x7f57f4139140 2023-09-30T22:29:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.184+0000 7f57f2ffd700 1 -- 172.21.15.32:0/446881691 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec012430 con 0x7f57f4139140 2023-09-30T22:29:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 -- 172.21.15.32:0/446881691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 msgr2=0x7f57f413b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/446881691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 0x7f57f413b520 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009d90 tx=0x7f57ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 -- 172.21.15.32:0/446881691 shutdown_connections 2023-09-30T22:29:37.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/446881691 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4139140 0x7f57f413b520 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/446881691 >> v2:172.21.15.32:3300/0 conn(0x7f57f4136820 0x7f57f4138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 -- 172.21.15.32:0/446881691 >> 172.21.15.32:0/446881691 conn(0x7f57f40a4210 msgr2=0x7f57f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 -- 172.21.15.32:0/446881691 shutdown_connections 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.185+0000 7f57fa3d9700 1 -- 172.21.15.32:0/446881691 wait complete. 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 Processor -- start 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 -- start start 2023-09-30T22:29:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 0x7f57f4070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4136820 0x7f57f4070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f4139140 0x7f57f41c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4072690 con 0x7f57f4133a80 2023-09-30T22:29:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4070cc0 con 0x7f57f4139140 2023-09-30T22:29:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.186+0000 7f57fa3d9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57f409b530 con 0x7f57f4136820 2023-09-30T22:29:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 0x7f57f4070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4136820 0x7f57f4070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 0x7f57f4070240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48450/0 (socket says 172.21.15.32:48450) 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f3fff700 1 -- 172.21.15.32:0/1085353960 learned_addr learned my addr 172.21.15.32:0/1085353960 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f8976700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f4139140 0x7f57f41c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 -- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f4139140 msgr2=0x7f57f41c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f4139140 0x7f57f41c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 -- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 msgr2=0x7f57f4070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 0x7f57f4070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 -- 172.21.15.32:0/1085353960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009a70 con 0x7f57f4136820 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f3fff700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 0x7f57f4070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f37fe700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4136820 0x7f57f4070780 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f57e800e8d0 tx=0x7f57e800ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8013680 con 0x7f57f4136820 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e80137e0 con 0x7f57f4136820 2023-09-30T22:29:37.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f4070ea0 con 0x7f57f4136820 2023-09-30T22:29:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8013b50 con 0x7f57f4136820 2023-09-30T22:29:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.187+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f4138c60 con 0x7f57f4136820 2023-09-30T22:29:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.188+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f57e8013cb0 con 0x7f57f4136820 2023-09-30T22:29:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.188+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f57e803fa40 con 0x7f57f4136820 2023-09-30T22:29:37.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.189+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f4074ea0 con 0x7f57f4136820 2023-09-30T22:29:37.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.193+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57e8021080 con 0x7f57f4136820 2023-09-30T22:29:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.361+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4448005590 con 0x7f44680a0a30 2023-09-30T22:29:37.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.361+0000 7f444ffff700 1 -- 172.21.15.32:0/3053054323 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f445c017030 con 0x7f44680a0a30 2023-09-30T22:29:37.362 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 msgr2=0x7f446811f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 0x7f446811f800 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f445c00ebb0 tx=0x7f445c00eec0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 shutdown_connections 2023-09-30T22:29:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4468133930 0x7f44681c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f446809d630 0x7f446811f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 --2- 172.21.15.32:0/3053054323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44680a0a30 0x7f446811f800 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 >> 172.21.15.32:0/3053054323 conn(0x7f44680a3050 msgr2=0x7f44681bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 shutdown_connections 2023-09-30T22:29:37.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.364+0000 7f446df2d700 1 -- 172.21.15.32:0/3053054323 wait complete. 2023-09-30T22:29:37.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:37.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.400+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f57f4130dc0 con 0x7f57f4136820 2023-09-30T22:29:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.400+0000 7f57f0ff9700 1 -- 172.21.15.32:0/1085353960 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f57e8017020 con 0x7f57f4136820 2023-09-30T22:29:37.400 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4136820 msgr2=0x7f57f4070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4136820 0x7f57f4070780 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f57e800e8d0 tx=0x7f57e800ebe0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 shutdown_connections 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f57f4139140 0x7f57f41c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57f4133a80 0x7f57f4070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 --2- 172.21.15.32:0/1085353960 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f57f4136820 0x7f57f4070780 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 >> 172.21.15.32:0/1085353960 conn(0x7f57f40a4210 msgr2=0x7f57f40a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 shutdown_connections 2023-09-30T22:29:37.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:37.403+0000 7f57fa3d9700 1 -- 172.21.15.32:0/1085353960 wait complete. 2023-09-30T22:29:37.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:37.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.033+0000 7fa035ab9700 1 Processor -- start 2023-09-30T22:29:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.033+0000 7fa035ab9700 1 -- start start 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa035ab9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 0x7fa030131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa035ab9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03006ae80 0x7fa030139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa035ab9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03009e130 con 0x7fa03009d9f0 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa035ab9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301312b0 con 0x7fa03006ae80 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa035ab9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa03009f6e0 con 0x7fa0301316c0 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 0x7fa030131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03006ae80 0x7fa030139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa03009d9f0 0x7fa03006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47790/0 (socket says 172.21.15.32:47790) 2023-09-30T22:29:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 0x7fa030131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57594/0 (socket says 172.21.15.32:57594) 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02effd700 1 -- 172.21.15.32:0/435152678 learned_addr learned my addr 172.21.15.32:0/435152678 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02ffff700 1 -- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 msgr2=0x7fa030131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02ffff700 1 --2- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 0x7fa030131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02e7fc700 1 -- 172.21.15.32:0/435152678 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668496099 0 0) 0x7fa024001110 con 0x7fa03009d9f0 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02ffff700 1 -- 172.21.15.32:0/435152678 >> v1:172.21.15.32:6789/0 conn(0x7fa03009d9f0 legacy=0x7fa03006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02ffff700 1 -- 172.21.15.32:0/435152678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03009e760 con 0x7fa03006ae80 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.034+0000 7fa02f7fe700 1 --2- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 0x7fa030131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa02ffff700 1 --2- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03006ae80 0x7fa030139550 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa028009dc0 tx=0x7fa02800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bc433080cd48881 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa02e7fc700 1 -- 172.21.15.32:0/435152678 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800e070 con 0x7fa03006ae80 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa02e7fc700 1 -- 172.21.15.32:0/435152678 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02800bb20 con 0x7fa03006ae80 2023-09-30T22:29:38.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa02e7fc700 1 -- 172.21.15.32:0/435152678 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028012430 con 0x7fa03006ae80 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 -- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03006ae80 msgr2=0x7fa030139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 --2- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03006ae80 0x7fa030139550 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa028009dc0 tx=0x7fa02800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 -- 172.21.15.32:0/435152678 shutdown_connections 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 --2- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03006ae80 0x7fa030139550 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 --2- 172.21.15.32:0/435152678 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0301316c0 0x7fa030131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 -- 172.21.15.32:0/435152678 >> 172.21.15.32:0/435152678 conn(0x7fa030066f50 msgr2=0x7fa03009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 -- 172.21.15.32:0/435152678 shutdown_connections 2023-09-30T22:29:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.035+0000 7fa035ab9700 1 -- 172.21.15.32:0/435152678 wait complete. 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.036+0000 7fa035ab9700 1 Processor -- start 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 -- start start 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa03006ae80 0x7fa0300709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 0x7fa030070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0301316c0 0x7fa030071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03011b0f0 con 0x7fa0301316c0 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03011a6c0 con 0x7fa03009d9f0 2023-09-30T22:29:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa035ab9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300a36d0 con 0x7fa03006ae80 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa02effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 0x7fa030070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa02effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 0x7fa030070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:59156/0 (socket says 172.21.15.32:59156) 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa02effd700 1 -- 172.21.15.32:0/1377293708 learned_addr learned my addr 172.21.15.32:0/1377293708 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa02f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa03006ae80 0x7fa0300709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.037+0000 7fa02ffff700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0301316c0 0x7fa030071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa02effd700 1 -- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa03006ae80 msgr2=0x7fa0300709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa02effd700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa03006ae80 0x7fa0300709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa02effd700 1 -- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0301316c0 msgr2=0x7fa030071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa02effd700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0301316c0 0x7fa030071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa02effd700 1 -- 172.21.15.32:0/1377293708 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200094a0 con 0x7fa03009d9f0 2023-09-30T22:29:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa02effd700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 0x7fa030070f10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c750 tx=0x7fa02400ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024004c80 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa024004de0 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa028009a70 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024007310 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.038+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03012b900 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.039+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa024007470 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.039+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa02403f0b0 con 0x7fa03009d9f0 2023-09-30T22:29:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.039+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03009e130 con 0x7fa03009d9f0 2023-09-30T22:29:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.044+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa03009e130 con 0x7fa03009d9f0 2023-09-30T22:29:38.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:38.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.252+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa03011ae20 con 0x7fa03009d9f0 2023-09-30T22:29:38.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.253+0000 7fa017fff700 1 -- 172.21.15.32:0/1377293708 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa03011ae20 con 0x7fa03009d9f0 2023-09-30T22:29:38.253 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:38.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 msgr2=0x7fa030070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 0x7fa030070f10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c750 tx=0x7fa02400ca60 comp rx=0 tx=0).stop 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 shutdown_connections 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0301316c0 0x7fa030071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa03006ae80 0x7fa0300709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 --2- 172.21.15.32:0/1377293708 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa03009d9f0 0x7fa030070f10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 >> 172.21.15.32:0/1377293708 conn(0x7fa030066f50 msgr2=0x7fa0300a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 shutdown_connections 2023-09-30T22:29:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.255+0000 7fa035ab9700 1 -- 172.21.15.32:0/1377293708 wait complete. 2023-09-30T22:29:38.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 Processor -- start 2023-09-30T22:29:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 -- start start 2023-09-30T22:29:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce3c130540 0x7fce3c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130e50 0x7fce3c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c09a390 con 0x7fce3c130540 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0716a0 con 0x7fce3c06fb20 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce4312e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c071290 con 0x7fce3c130e50 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130e50 0x7fce3c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce416cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130e50 0x7fce3c06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57618/0 (socket says 172.21.15.32:57618) 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce3bfff700 1 -- 172.21.15.32:0/2131982997 learned_addr learned my addr 172.21.15.32:0/2131982997 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.304+0000 7fce40eca700 1 --2- 172.21.15.32:0/2131982997 >> v2:172.21.15.32:3300/0 conn(0x7fce3c130540 0x7fce3c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce416cb700 1 -- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130e50 msgr2=0x7fce3c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce416cb700 1 --2- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130e50 0x7fce3c06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce416cb700 1 -- 172.21.15.32:0/2131982997 >> v2:172.21.15.32:3300/0 conn(0x7fce3c130540 msgr2=0x7fce3c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:38.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce416cb700 1 --2- 172.21.15.32:0/2131982997 >> v2:172.21.15.32:3300/0 conn(0x7fce3c130540 0x7fce3c130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce416cb700 1 -- 172.21.15.32:0/2131982997 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c130340 con 0x7fce3c06fb20 2023-09-30T22:29:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce40eca700 1 --2- 172.21.15.32:0/2131982997 >> v2:172.21.15.32:3300/0 conn(0x7fce3c130540 0x7fce3c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce416cb700 1 --2- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c06ffa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce34009d90 tx=0x7fce3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf67d82af7a54ef4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce3b7fe700 1 -- 172.21.15.32:0/2131982997 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400e070 con 0x7fce3c06fb20 2023-09-30T22:29:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce3b7fe700 1 -- 172.21.15.32:0/2131982997 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400bb20 con 0x7fce3c06fb20 2023-09-30T22:29:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.305+0000 7fce3b7fe700 1 -- 172.21.15.32:0/2131982997 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34012430 con 0x7fce3c06fb20 2023-09-30T22:29:38.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 -- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 msgr2=0x7fce3c06ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 --2- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c06ffa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fce34009d90 tx=0x7fce3400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 -- 172.21.15.32:0/2131982997 shutdown_connections 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 --2- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c06ffa0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 --2- 172.21.15.32:0/2131982997 >> v2:172.21.15.32:3300/0 conn(0x7fce3c130540 0x7fce3c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 --2- 172.21.15.32:0/2131982997 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130e50 0x7fce3c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 -- 172.21.15.32:0/2131982997 >> 172.21.15.32:0/2131982997 conn(0x7fce3c0a2aa0 msgr2=0x7fce3c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 -- 172.21.15.32:0/2131982997 shutdown_connections 2023-09-30T22:29:38.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.306+0000 7fce4312e700 1 -- 172.21.15.32:0/2131982997 wait complete. 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.307+0000 7fce4312e700 1 Processor -- start 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.307+0000 7fce4312e700 1 -- start start 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c11d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 0x7fce3c11d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 0x7fce3c11db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c120000 con 0x7fce3c130e50 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c11f0f0 con 0x7fce3c06fb20 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c12bb50 con 0x7fce3c130540 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 0x7fce3c11db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 0x7fce3c11d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce40eca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c11d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 0x7fce3c11db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48474/0 (socket says 172.21.15.32:48474) 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 0x7fce3c11d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57624/0 (socket says 172.21.15.32:57624) 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 -- 172.21.15.32:0/2353001260 learned_addr learned my addr 172.21.15.32:0/2353001260 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 -- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 msgr2=0x7fce3c11d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 0x7fce3c11d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 -- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 msgr2=0x7fce3c11d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c11d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 -- 172.21.15.32:0/2353001260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0094a0 con 0x7fce3c130e50 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce40eca700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c11d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce3bfff700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 0x7fce3c11d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce416cb700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 0x7fce3c11db60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fce34014040 tx=0x7fce34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34012a40 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce34020070 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce34009a70 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.308+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34012e20 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.309+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c0718e0 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.309+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce34004040 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.309+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fce34022070 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.309+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c074ea0 con 0x7fce3c130e50 2023-09-30T22:29:38.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.315+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce3401a970 con 0x7fce3c130e50 2023-09-30T22:29:38.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:38.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.518+0000 7f6eb13dd700 1 Processor -- start 2023-09-30T22:29:38.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.518+0000 7f6eb13dd700 1 -- start start 2023-09-30T22:29:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eb13dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eb13dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eb13dd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac09edd0 con 0x7f6eac0a0a30 2023-09-30T22:29:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eb13dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0a18e0 con 0x7f6eac0a1510 2023-09-30T22:29:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eb13dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac09e1f0 con 0x7f6eac139250 2023-09-30T22:29:38.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eaaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6eac0a0a30 0x7f6eac0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47794/0 (socket says 172.21.15.32:47794) 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eaaffd700 1 -- 172.21.15.32:0/129991334 learned_addr learned my addr 172.21.15.32:0/129991334 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6eaa7fc700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.519+0000 7f6ea9ffb700 1 -- 172.21.15.32:0/129991334 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607787563 0 0) 0x7f6eac09edd0 con 0x7f6eac0a0a30 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6ea9ffb700 1 -- 172.21.15.32:0/129991334 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e940035b0 con 0x7f6eac0a0a30 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6eab7fe700 1 -- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 msgr2=0x7f6eac138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6eab7fe700 1 -- 172.21.15.32:0/129991334 >> v1:172.21.15.32:6789/0 conn(0x7f6eac0a0a30 legacy=0x7f6eac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6eab7fe700 1 -- 172.21.15.32:0/129991334 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac09dc50 con 0x7f6eac139250 2023-09-30T22:29:38.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac13b630 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009870 tx=0x7f6ea4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df8fa2391fa2166 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6ea9ffb700 1 -- 172.21.15.32:0/129991334 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011830 con 0x7f6eac139250 2023-09-30T22:29:38.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6ea9ffb700 1 -- 172.21.15.32:0/129991334 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011990 con 0x7f6eac139250 2023-09-30T22:29:38.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.520+0000 7f6ea9ffb700 1 -- 172.21.15.32:0/129991334 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011d00 con 0x7f6eac139250 2023-09-30T22:29:38.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 -- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 msgr2=0x7f6eac13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac13b630 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009870 tx=0x7f6ea4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 -- 172.21.15.32:0/129991334 shutdown_connections 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac13b630 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/129991334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 -- 172.21.15.32:0/129991334 >> 172.21.15.32:0/129991334 conn(0x7f6eac0a6550 msgr2=0x7f6eac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 -- 172.21.15.32:0/129991334 shutdown_connections 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.521+0000 7f6eb13dd700 1 -- 172.21.15.32:0/129991334 wait complete. 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.522+0000 7f6eb13dd700 1 Processor -- start 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 -- start start 2023-09-30T22:29:38.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 0x7f6eac122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac126020 con 0x7f6eac0a0a30 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0a0220 con 0x7f6eac0a1510 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eb13dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1242a0 con 0x7f6eac139250 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eaaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 0x7f6eac122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eaaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 0x7f6eac122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48488/0 (socket says 172.21.15.32:48488) 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eaaffd700 1 -- 172.21.15.32:0/496407310 learned_addr learned my addr 172.21.15.32:0/496407310 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.523+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eaa7fc700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eab7fe700 1 -- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 msgr2=0x7f6eac122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eab7fe700 1 -- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 msgr2=0x7f6eac122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 0x7f6eac122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eab7fe700 1 -- 172.21.15.32:0/496407310 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40094a0 con 0x7f6eac139250 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eaaffd700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 0x7f6eac122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eaa7fc700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eab7fe700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac123200 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4016000 tx=0x7f6ea4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40044d0 con 0x7f6eac139250 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea40036a0 con 0x7f6eac139250 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003a10 con 0x7f6eac139250 2023-09-30T22:29:38.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac0a5510 con 0x7f6eac139250 2023-09-30T22:29:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.524+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac1269b0 con 0x7f6eac139250 2023-09-30T22:29:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.525+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac074ea0 con 0x7f6eac139250 2023-09-30T22:29:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.525+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ea4003b70 con 0x7f6eac139250 2023-09-30T22:29:38.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.526+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6ea40409c0 con 0x7f6eac139250 2023-09-30T22:29:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.527+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce3c11f5b0 con 0x7fce3c130e50 2023-09-30T22:29:38.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.528+0000 7fce397fa700 1 -- 172.21.15.32:0/2353001260 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce34036090 con 0x7fce3c130e50 2023-09-30T22:29:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:38.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 msgr2=0x7fce3c11db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 0x7fce3c11db60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fce34014040 tx=0x7fce34004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 shutdown_connections 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce3c130e50 0x7fce3c11db60 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce3c06fb20 0x7fce3c11d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 --2- 172.21.15.32:0/2353001260 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce3c130540 0x7fce3c11d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 >> 172.21.15.32:0/2353001260 conn(0x7fce3c0a2aa0 msgr2=0x7fce3c12d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 shutdown_connections 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.530+0000 7fce4312e700 1 -- 172.21.15.32:0/2353001260 wait complete. 2023-09-30T22:29:38.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.535+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ea402d3b0 con 0x7f6eac139250 2023-09-30T22:29:38.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.565+0000 7fc6c383f700 1 Processor -- start 2023-09-30T22:29:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c383f700 1 -- start start 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c383f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc0a1510 0x7fc6bc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c383f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6bc139250 0x7fc6bc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c383f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc09edd0 con 0x7fc6bc0a0a30 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c383f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0a18e0 con 0x7fc6bc139250 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c383f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc09e1f0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c1ddc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6bc139250 0x7fc6bc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c0dda700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc0a1510 0x7fc6bc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c15db700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc6bc0a0a30 0x7fc6bc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47808/0 (socket says 172.21.15.32:47808) 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.566+0000 7fc6c15db700 1 -- 172.21.15.32:0/1502632586 learned_addr learned my addr 172.21.15.32:0/1502632586 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6b3fff700 1 -- 172.21.15.32:0/1502632586 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843682310 0 0) 0x7fc6bc09edd0 con 0x7fc6bc0a0a30 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6c0dda700 1 -- 172.21.15.32:0/1502632586 >> v2:172.21.15.32:3300/0 conn(0x7fc6bc139250 msgr2=0x7fc6bc13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6c0dda700 1 --2- 172.21.15.32:0/1502632586 >> v2:172.21.15.32:3300/0 conn(0x7fc6bc139250 0x7fc6bc13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6c0dda700 1 -- 172.21.15.32:0/1502632586 >> v1:172.21.15.32:6789/0 conn(0x7fc6bc0a0a30 legacy=0x7fc6bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6c0dda700 1 -- 172.21.15.32:0/1502632586 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc09dc50 con 0x7fc6bc0a1510 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6c0dda700 1 --2- 172.21.15.32:0/1502632586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc0a1510 0x7fc6bc138d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009870 tx=0x7fc6ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96cc171706d1d7df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6b3fff700 1 -- 172.21.15.32:0/1502632586 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac011830 con 0x7fc6bc0a1510 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6b3fff700 1 -- 172.21.15.32:0/1502632586 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac011990 con 0x7fc6bc0a1510 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.567+0000 7fc6b3fff700 1 -- 172.21.15.32:0/1502632586 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac011d20 con 0x7fc6bc0a1510 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 -- 172.21.15.32:0/1502632586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc0a1510 msgr2=0x7fc6bc138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 --2- 172.21.15.32:0/1502632586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc0a1510 0x7fc6bc138d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009870 tx=0x7fc6ac009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 -- 172.21.15.32:0/1502632586 shutdown_connections 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 --2- 172.21.15.32:0/1502632586 >> v2:172.21.15.32:3300/0 conn(0x7fc6bc139250 0x7fc6bc13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 --2- 172.21.15.32:0/1502632586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc0a1510 0x7fc6bc138d10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 -- 172.21.15.32:0/1502632586 >> 172.21.15.32:0/1502632586 conn(0x7fc6bc0a6550 msgr2=0x7fc6bc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 -- 172.21.15.32:0/1502632586 shutdown_connections 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.568+0000 7fc6c383f700 1 -- 172.21.15.32:0/1502632586 wait complete. 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.569+0000 7fc6c383f700 1 Processor -- start 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.569+0000 7fc6c383f700 1 -- start start 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c383f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6bc0a0a30 0x7fc6bc122750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c383f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 0x7fc6bc122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c383f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 0x7fc6bc1231d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c383f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1381f0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c383f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc125c40 con 0x7fc6bc0a0a30 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c383f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1243e0 con 0x7fc6bc139250 2023-09-30T22:29:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 0x7fc6bc122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c1ddc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 0x7fc6bc1231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c15db700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6bc0a0a30 0x7fc6bc122750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 0x7fc6bc122c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48508/0 (socket says 172.21.15.32:48508) 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c1ddc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 0x7fc6bc1231d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57666/0 (socket says 172.21.15.32:57666) 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 -- 172.21.15.32:0/775351999 learned_addr learned my addr 172.21.15.32:0/775351999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 -- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 msgr2=0x7fc6bc1231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 0x7fc6bc1231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 -- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6bc0a0a30 msgr2=0x7fc6bc122750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6bc0a0a30 0x7fc6bc122750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 -- 172.21.15.32:0/775351999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0094a0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c1ddc700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 0x7fc6bc1231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6c0dda700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 0x7fc6bc122c90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac000c00 tx=0x7fc6ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.570+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0036a0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.571+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac01f070 con 0x7fc6bc0a1510 2023-09-30T22:29:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.571+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc138590 con 0x7fc6bc0a1510 2023-09-30T22:29:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.571+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac003a20 con 0x7fc6bc0a1510 2023-09-30T22:29:38.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.571+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1258d0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.572+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc074ea0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.572+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6ac003b80 con 0x7fc6bc0a1510 2023-09-30T22:29:38.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.572+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc6ac021070 con 0x7fc6bc0a1510 2023-09-30T22:29:38.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.577+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6ac003e30 con 0x7fc6bc0a1510 2023-09-30T22:29:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.742+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6eac0a28e0 con 0x7f6eac139250 2023-09-30T22:29:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.742+0000 7f6e937fe700 1 -- 172.21.15.32:0/496407310 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ea4015030 con 0x7f6eac139250 2023-09-30T22:29:38.742 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:38.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 msgr2=0x7f6eac123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac123200 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4016000 tx=0x7f6ea4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 shutdown_connections 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6eac139250 0x7f6eac123200 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6eac0a0a30 0x7f6eac122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 --2- 172.21.15.32:0/496407310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6eac0a1510 0x7f6eac122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 >> 172.21.15.32:0/496407310 conn(0x7f6eac0a6550 msgr2=0x7f6eac11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 shutdown_connections 2023-09-30T22:29:38.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.744+0000 7f6eb13dd700 1 -- 172.21.15.32:0/496407310 wait complete. 2023-09-30T22:29:38.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.780+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc6bc0a28e0 con 0x7fc6bc0a1510 2023-09-30T22:29:38.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.780+0000 7fc6b1ffb700 1 -- 172.21.15.32:0/775351999 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6ac015030 con 0x7fc6bc0a1510 2023-09-30T22:29:38.780 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 msgr2=0x7fc6bc122c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 0x7fc6bc122c90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac000c00 tx=0x7fc6ac004750 comp rx=0 tx=0).stop 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 shutdown_connections 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6bc139250 0x7fc6bc1231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6bc0a0a30 0x7fc6bc122750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 --2- 172.21.15.32:0/775351999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6bc0a1510 0x7fc6bc122c90 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:38.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 >> 172.21.15.32:0/775351999 conn(0x7fc6bc0a6550 msgr2=0x7fc6bc0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 shutdown_connections 2023-09-30T22:29:38.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:38.783+0000 7fc6c383f700 1 -- 172.21.15.32:0/775351999 wait complete. 2023-09-30T22:29:38.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:39.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:39.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 Processor -- start 2023-09-30T22:29:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 -- start start 2023-09-30T22:29:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a0a30 0x7fce700a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce70134170 0x7fce7013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7009edd0 con 0x7fce700a1340 2023-09-30T22:29:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7009e1f0 con 0x7fce70134170 2023-09-30T22:29:39.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.418+0000 7fce77256700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce7009dc50 con 0x7fce700a0a30 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce757f3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce70134170 0x7fce7013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce6ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce700a1340 0x7fce70133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47820/0 (socket says 172.21.15.32:47820) 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce6ffff700 1 -- 172.21.15.32:0/231032831 learned_addr learned my addr 172.21.15.32:0/231032831 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce74ff2700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a0a30 0x7fce700a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce757f3700 1 -- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a0a30 msgr2=0x7fce700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce757f3700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a0a30 0x7fce700a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce757f3700 1 -- 172.21.15.32:0/231032831 >> v1:172.21.15.32:6789/0 conn(0x7fce700a1340 legacy=0x7fce70133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce757f3700 1 -- 172.21.15.32:0/231032831 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700a1fe0 con 0x7fce70134170 2023-09-30T22:29:39.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce74ff2700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a0a30 0x7fce700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce757f3700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7013bd30 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fce68009d90 tx=0x7fce6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9ed1278af00aed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce6f7fe700 1 -- 172.21.15.32:0/231032831 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800e070 con 0x7fce70134170 2023-09-30T22:29:39.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce6f7fe700 1 -- 172.21.15.32:0/231032831 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800bb20 con 0x7fce70134170 2023-09-30T22:29:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.419+0000 7fce6f7fe700 1 -- 172.21.15.32:0/231032831 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68012430 con 0x7fce70134170 2023-09-30T22:29:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 -- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 msgr2=0x7fce7013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7013bd30 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fce68009d90 tx=0x7fce6800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 -- 172.21.15.32:0/231032831 shutdown_connections 2023-09-30T22:29:39.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7013bd30 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 --2- 172.21.15.32:0/231032831 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a0a30 0x7fce700a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 -- 172.21.15.32:0/231032831 >> 172.21.15.32:0/231032831 conn(0x7fce700a6550 msgr2=0x7fce700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 -- 172.21.15.32:0/231032831 shutdown_connections 2023-09-30T22:29:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.420+0000 7fce77256700 1 -- 172.21.15.32:0/231032831 wait complete. 2023-09-30T22:29:39.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.421+0000 7fce77256700 1 Processor -- start 2023-09-30T22:29:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 -- start start 2023-09-30T22:29:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce700a0a30 0x7fce7011e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 0x7fce7011ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7011f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70122190 con 0x7fce70134170 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce70121bf0 con 0x7fce700a0a30 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.422+0000 7fce77256700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce70120280 con 0x7fce700a1340 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 0x7fce7011ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce74ff2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce700a0a30 0x7fce7011e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 0x7fce7011ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57676/0 (socket says 172.21.15.32:57676) 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 -- 172.21.15.32:0/1427306207 learned_addr learned my addr 172.21.15.32:0/1427306207 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:39.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce757f3700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7011f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 -- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce700a0a30 msgr2=0x7fce7011e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce700a0a30 0x7fce7011e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 -- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 msgr2=0x7fce7011f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7011f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 -- 172.21.15.32:0/1427306207 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640094a0 con 0x7fce700a1340 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce74ff2700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce700a0a30 0x7fce7011e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce757f3700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7011f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6ffff700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 0x7fce7011ec40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c520 tx=0x7fce5c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c005d70 con 0x7fce700a1340 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c0070d0 con 0x7fce700a1340 2023-09-30T22:29:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce68009a70 con 0x7fce700a1340 2023-09-30T22:29:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c007440 con 0x7fce700a1340 2023-09-30T22:29:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.423+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce70120a30 con 0x7fce700a1340 2023-09-30T22:29:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.424+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce5c0075a0 con 0x7fce700a1340 2023-09-30T22:29:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.424+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce70074ea0 con 0x7fce700a1340 2023-09-30T22:29:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.425+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fce5c03eae0 con 0x7fce700a1340 2023-09-30T22:29:39.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.429+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce5c019b40 con 0x7fce700a1340 2023-09-30T22:29:39.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:39.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.638+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce700a28e0 con 0x7fce700a1340 2023-09-30T22:29:39.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.638+0000 7fce6d7fa700 1 -- 172.21.15.32:0/1427306207 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce5c017070 con 0x7fce700a1340 2023-09-30T22:29:39.639 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 msgr2=0x7fce7011ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 0x7fce7011ec40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c520 tx=0x7fce5c00c830 comp rx=0 tx=0).stop 2023-09-30T22:29:39.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 shutdown_connections 2023-09-30T22:29:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce70134170 0x7fce7011f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce700a0a30 0x7fce7011e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 --2- 172.21.15.32:0/1427306207 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce700a1340 0x7fce7011ec40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 >> 172.21.15.32:0/1427306207 conn(0x7fce700a6550 msgr2=0x7fce7009c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 shutdown_connections 2023-09-30T22:29:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.641+0000 7fce77256700 1 -- 172.21.15.32:0/1427306207 wait complete. 2023-09-30T22:29:39.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:39.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 Processor -- start 2023-09-30T22:29:39.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 -- start start 2023-09-30T22:29:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea4c09e930 0x7fea4c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09f650 0x7fea4c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c070940 con 0x7fea4c09f240 2023-09-30T22:29:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c06fd60 con 0x7fea4c09e930 2023-09-30T22:29:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.687+0000 7fea50d83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c06fa80 con 0x7fea4c09f650 2023-09-30T22:29:39.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea4c09e930 0x7fea4c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09f650 0x7fea4c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea49d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea4c09f240 0x7fea4c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47822/0 (socket says 172.21.15.32:47822) 2023-09-30T22:29:39.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea49d9b700 1 -- 172.21.15.32:0/3665801952 learned_addr learned my addr 172.21.15.32:0/3665801952 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:39.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4a59c700 1 -- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09f650 msgr2=0x7fea4c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:39.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4a59c700 1 --2- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09f650 0x7fea4c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4a59c700 1 -- 172.21.15.32:0/3665801952 >> v1:172.21.15.32:6789/0 conn(0x7fea4c09f240 legacy=0x7fea4c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4a59c700 1 -- 172.21.15.32:0/3665801952 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c09fb60 con 0x7fea4c09e930 2023-09-30T22:29:39.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4ad9d700 1 --2- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09f650 0x7fea4c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4959a700 1 -- 172.21.15.32:0/3665801952 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977523705 0 0) 0x7fea4c070940 con 0x7fea4c09f240 2023-09-30T22:29:39.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4a59c700 1 --2- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09e930 0x7fea4c09ed00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa3a510c2c8cf64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4959a700 1 -- 172.21.15.32:0/3665801952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c011830 con 0x7fea4c09e930 2023-09-30T22:29:39.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4959a700 1 -- 172.21.15.32:0/3665801952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c011990 con 0x7fea4c09e930 2023-09-30T22:29:39.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.688+0000 7fea4959a700 1 -- 172.21.15.32:0/3665801952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c011d20 con 0x7fea4c09e930 2023-09-30T22:29:39.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 -- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09e930 msgr2=0x7fea4c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 --2- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09e930 0x7fea4c09ed00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 -- 172.21.15.32:0/3665801952 shutdown_connections 2023-09-30T22:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 --2- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09f650 0x7fea4c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 --2- 172.21.15.32:0/3665801952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09e930 0x7fea4c09ed00 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 -- 172.21.15.32:0/3665801952 >> 172.21.15.32:0/3665801952 conn(0x7fea4c0a4210 msgr2=0x7fea4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 -- 172.21.15.32:0/3665801952 shutdown_connections 2023-09-30T22:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.689+0000 7fea50d83700 1 -- 172.21.15.32:0/3665801952 wait complete. 2023-09-30T22:29:39.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.690+0000 7fea50d83700 1 Processor -- start 2023-09-30T22:29:39.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.690+0000 7fea50d83700 1 -- start start 2023-09-30T22:29:39.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea50d83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 0x7fea4c1c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea50d83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 0x7fea4c1c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea50d83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea4c09f650 0x7fea4c1c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea50d83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a8270 con 0x7fea4c09f240 2023-09-30T22:29:39.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea50d83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c1a6560 con 0x7fea4c09f650 2023-09-30T22:29:39.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea50d83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0a3060 con 0x7fea4c09e930 2023-09-30T22:29:39.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 0x7fea4c1c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea4a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 0x7fea4c1c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea4a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 0x7fea4c1c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57692/0 (socket says 172.21.15.32:57692) 2023-09-30T22:29:39.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 0x7fea4c1c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48548/0 (socket says 172.21.15.32:48548) 2023-09-30T22:29:39.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea4a59c700 1 -- 172.21.15.32:0/4197163545 learned_addr learned my addr 172.21.15.32:0/4197163545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea4ad9d700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea4c09f650 0x7fea4c1c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 -- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 msgr2=0x7fea4c1c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 0x7fea4c1c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 -- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea4c09f650 msgr2=0x7fea4c1c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea4c09f650 0x7fea4c1c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 -- 172.21.15.32:0/4197163545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009a70 con 0x7fea4c09f240 2023-09-30T22:29:39.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea4a59c700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 0x7fea4c1c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea4ad9d700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea4c09f650 0x7fea4c1c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.691+0000 7fea49d9b700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 0x7fea4c1c8570 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c470 tx=0x7fea4000c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.692+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40004c80 con 0x7fea4c09f240 2023-09-30T22:29:39.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.692+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40004de0 con 0x7fea4c09f240 2023-09-30T22:29:39.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.692+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c0094a0 con 0x7fea4c09f240 2023-09-30T22:29:39.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.692+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40007310 con 0x7fea4c09f240 2023-09-30T22:29:39.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.692+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c0a0190 con 0x7fea4c09f240 2023-09-30T22:29:39.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.692+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea40007470 con 0x7fea4c09f240 2023-09-30T22:29:39.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.693+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea4003f1c0 con 0x7fea4c09f240 2023-09-30T22:29:39.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.693+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c1a6db0 con 0x7fea4c09f240 2023-09-30T22:29:39.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.698+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea40007720 con 0x7fea4c09f240 2023-09-30T22:29:39.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:39.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:39.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.899+0000 7f26282bf700 1 Processor -- start 2023-09-30T22:29:39.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.899+0000 7f26282bf700 1 -- start start 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f26282bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f262009d630 0x7f26200a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f26282bf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2620138d80 0x7f262013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f26282bf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26200a2a70 con 0x7f26200a0a30 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f26282bf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26200a01d0 con 0x7f2620138d80 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f26282bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f262009e2b0 con 0x7f262009d630 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262605b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f26200a0a30 0x7f26200a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47834/0 (socket says 172.21.15.32:47834) 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262605b700 1 -- 172.21.15.32:0/100144930 learned_addr learned my addr 172.21.15.32:0/100144930 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:39.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262585a700 1 --2- 172.21.15.32:0/100144930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f262009d630 0x7f26200a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262685c700 1 --2- 172.21.15.32:0/100144930 >> v2:172.21.15.32:3300/0 conn(0x7f2620138d80 0x7f262013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f2625059700 1 -- 172.21.15.32:0/100144930 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688343487 0 0) 0x7f26200a2a70 con 0x7f26200a0a30 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f2625059700 1 -- 172.21.15.32:0/100144930 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f260c0035b0 con 0x7f26200a0a30 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262585a700 1 -- 172.21.15.32:0/100144930 >> v2:172.21.15.32:3300/0 conn(0x7f2620138d80 msgr2=0x7f262013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262585a700 1 --2- 172.21.15.32:0/100144930 >> v2:172.21.15.32:3300/0 conn(0x7f2620138d80 0x7f262013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262585a700 1 -- 172.21.15.32:0/100144930 >> v1:172.21.15.32:6789/0 conn(0x7f26200a0a30 legacy=0x7f26200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.900+0000 7f262585a700 1 -- 172.21.15.32:0/100144930 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262009dc50 con 0x7f262009d630 2023-09-30T22:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.901+0000 7f262585a700 1 --2- 172.21.15.32:0/100144930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f262009d630 0x7f26200a1510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2610009840 tx=0x7f2610009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8918d1eed49c129c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.901+0000 7f2625059700 1 -- 172.21.15.32:0/100144930 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610011830 con 0x7f262009d630 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.901+0000 7f2625059700 1 -- 172.21.15.32:0/100144930 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2610011990 con 0x7f262009d630 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.901+0000 7f2625059700 1 -- 172.21.15.32:0/100144930 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610011d00 con 0x7f262009d630 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 -- 172.21.15.32:0/100144930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f262009d630 msgr2=0x7f26200a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 --2- 172.21.15.32:0/100144930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f262009d630 0x7f26200a1510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2610009840 tx=0x7f2610009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 -- 172.21.15.32:0/100144930 shutdown_connections 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 --2- 172.21.15.32:0/100144930 >> v2:172.21.15.32:3300/0 conn(0x7f2620138d80 0x7f262013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 --2- 172.21.15.32:0/100144930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f262009d630 0x7f26200a1510 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 -- 172.21.15.32:0/100144930 >> 172.21.15.32:0/100144930 conn(0x7f26200a3050 msgr2=0x7f26200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 -- 172.21.15.32:0/100144930 shutdown_connections 2023-09-30T22:29:39.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.902+0000 7f26282bf700 1 -- 172.21.15.32:0/100144930 wait complete. 2023-09-30T22:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 Processor -- start 2023-09-30T22:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 -- start start 2023-09-30T22:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 0x7f26201336a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26200a0a30 0x7f2620133be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2620138d80 0x7f2620134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2620137130 con 0x7f262009d630 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2620136470 con 0x7f2620138d80 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f26282bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f262013abd0 con 0x7f26200a0a30 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f262605b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 0x7f26201336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f262605b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 0x7f26201336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48564/0 (socket says 172.21.15.32:48564) 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f262605b700 1 -- 172.21.15.32:0/592662243 learned_addr learned my addr 172.21.15.32:0/592662243 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f262585a700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26200a0a30 0x7f2620133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.904+0000 7f262685c700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2620138d80 0x7f2620134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262605b700 1 -- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26200a0a30 msgr2=0x7f2620133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262605b700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26200a0a30 0x7f2620133be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262605b700 1 -- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2620138d80 msgr2=0x7f2620134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262605b700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2620138d80 0x7f2620134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262605b700 1 -- 172.21.15.32:0/592662243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26100094a0 con 0x7f262009d630 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262685c700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2620138d80 0x7f2620134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f262605b700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 0x7f26201336a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c700 tx=0x7f261c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c005d70 con 0x7f262009d630 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c0070d0 con 0x7f262009d630 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c007460 con 0x7f262009d630 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2620135a80 con 0x7f262009d630 2023-09-30T22:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.905+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2620136b90 con 0x7f262009d630 2023-09-30T22:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.907+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26200747f0 con 0x7f262009d630 2023-09-30T22:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.907+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f261c0075c0 con 0x7f262009d630 2023-09-30T22:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.907+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f261c03eff0 con 0x7f262009d630 2023-09-30T22:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.907+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fea4c074ea0 con 0x7fea4c09f240 2023-09-30T22:29:39.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.912+0000 7fea3affd700 1 -- 172.21.15.32:0/4197163545 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea40011360 con 0x7fea4c09f240 2023-09-30T22:29:39.912 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.912+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f261c01bb10 con 0x7f262009d630 2023-09-30T22:29:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 msgr2=0x7fea4c1c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 0x7fea4c1c8570 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c470 tx=0x7fea4000c780 comp rx=0 tx=0).stop 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 shutdown_connections 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea4c09f650 0x7fea4c1c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea4c09e930 0x7fea4c1c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 --2- 172.21.15.32:0/4197163545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea4c09f240 0x7fea4c1c8570 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 >> 172.21.15.32:0/4197163545 conn(0x7fea4c0a4210 msgr2=0x7fea4c06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 shutdown_connections 2023-09-30T22:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.914+0000 7fea50d83700 1 -- 172.21.15.32:0/4197163545 wait complete. 2023-09-30T22:29:39.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.945+0000 7f4a1a317700 1 Processor -- start 2023-09-30T22:29:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.945+0000 7f4a1a317700 1 -- start start 2023-09-30T22:29:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a1a317700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 0x7f4a140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a1a317700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a1340 0x7f4a14133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a1a317700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1409edd0 con 0x7f4a14133fa0 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a1a317700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1409e1f0 con 0x7f4a140a0a30 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a1a317700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1409dc50 con 0x7f4a140a1340 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a13fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 0x7f4a140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a13fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 0x7f4a140a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48580/0 (socket says 172.21.15.32:48580) 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a188b4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4a14133fa0 0x7f4a1413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47848/0 (socket says 172.21.15.32:47848) 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a13fff700 1 -- 172.21.15.32:0/4242720679 learned_addr learned my addr 172.21.15.32:0/4242720679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:39.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.946+0000 7f4a137fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a1340 0x7f4a14133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a137fe700 1 -- 172.21.15.32:0/4242720679 >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 msgr2=0x7f4a140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a137fe700 1 --2- 172.21.15.32:0/4242720679 >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 0x7f4a140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a137fe700 1 -- 172.21.15.32:0/4242720679 >> v1:172.21.15.32:6789/0 conn(0x7f4a14133fa0 legacy=0x7f4a1413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a137fe700 1 -- 172.21.15.32:0/4242720679 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140a1fe0 con 0x7f4a140a1340 2023-09-30T22:29:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a13fff700 1 --2- 172.21.15.32:0/4242720679 >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 0x7f4a140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a137fe700 1 --2- 172.21.15.32:0/4242720679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a1340 0x7f4a14133a60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009840 tx=0x7f4a08009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6cb656cebb4e133 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a12ffd700 1 -- 172.21.15.32:0/4242720679 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08011830 con 0x7f4a140a1340 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a12ffd700 1 -- 172.21.15.32:0/4242720679 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a08011990 con 0x7f4a140a1340 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.947+0000 7f4a12ffd700 1 -- 172.21.15.32:0/4242720679 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08011d20 con 0x7f4a140a1340 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 -- 172.21.15.32:0/4242720679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a1340 msgr2=0x7f4a14133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 --2- 172.21.15.32:0/4242720679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a1340 0x7f4a14133a60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009840 tx=0x7f4a08009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 -- 172.21.15.32:0/4242720679 shutdown_connections 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 --2- 172.21.15.32:0/4242720679 >> v2:172.21.15.32:3300/0 conn(0x7f4a140a0a30 0x7f4a140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 --2- 172.21.15.32:0/4242720679 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a1340 0x7f4a14133a60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 -- 172.21.15.32:0/4242720679 >> 172.21.15.32:0/4242720679 conn(0x7f4a140a6550 msgr2=0x7f4a140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 -- 172.21.15.32:0/4242720679 shutdown_connections 2023-09-30T22:29:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.948+0000 7f4a1a317700 1 -- 172.21.15.32:0/4242720679 wait complete. 2023-09-30T22:29:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.949+0000 7f4a1a317700 1 Processor -- start 2023-09-30T22:29:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.949+0000 7f4a1a317700 1 -- start start 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a1a317700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a0a30 0x7f4a141c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a1a317700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 0x7f4a141c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a1a317700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a14133fa0 0x7f4a141c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a1a317700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141245d0 con 0x7f4a140a1340 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a1a317700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14125f40 con 0x7f4a14133fa0 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a1a317700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140a3710 con 0x7f4a140a0a30 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 0x7f4a141c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 0x7f4a141c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48588/0 (socket says 172.21.15.32:48588) 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 -- 172.21.15.32:0/1584922394 learned_addr learned my addr 172.21.15.32:0/1584922394 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a13fff700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a0a30 0x7f4a141c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a188b4700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a14133fa0 0x7f4a141c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 -- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a0a30 msgr2=0x7f4a141c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a0a30 0x7f4a141c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 -- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a14133fa0 msgr2=0x7f4a141c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a14133fa0 0x7f4a141c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 -- 172.21.15.32:0/1584922394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04009550 con 0x7f4a140a1340 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.950+0000 7f4a137fe700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 0x7f4a141c8860 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009fd0 tx=0x7f4a08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.951+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08004550 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.951+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0801f070 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.951+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a080094a0 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.951+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a080038e0 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.951+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a14126c70 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.951+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a08003a40 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.952+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a08021070 con 0x7f4a140a1340 2023-09-30T22:29:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.952+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a14126440 con 0x7f4a140a1340 2023-09-30T22:29:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:39.957+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a14126440 con 0x7f4a140a1340 2023-09-30T22:29:40.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.122+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2620072980 con 0x7f262009d630 2023-09-30T22:29:40.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.122+0000 7f2616ffd700 1 -- 172.21.15.32:0/592662243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f261c0192c0 con 0x7f262009d630 2023-09-30T22:29:40.122 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:40.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 msgr2=0x7f26201336a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 0x7f26201336a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c700 tx=0x7f261c00ca10 comp rx=0 tx=0).stop 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 shutdown_connections 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2620138d80 0x7f2620134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f262009d630 0x7f26201336a0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 --2- 172.21.15.32:0/592662243 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f26200a0a30 0x7f2620133be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 >> 172.21.15.32:0/592662243 conn(0x7f26200a3050 msgr2=0x7f262009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 shutdown_connections 2023-09-30T22:29:40.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.124+0000 7f26282bf700 1 -- 172.21.15.32:0/592662243 wait complete. 2023-09-30T22:29:40.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:40.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.161+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4a14126440 con 0x7f4a140a1340 2023-09-30T22:29:40.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.161+0000 7f4a10ff9700 1 -- 172.21.15.32:0/1584922394 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a14126440 con 0x7f4a140a1340 2023-09-30T22:29:40.162 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 msgr2=0x7f4a141c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 0x7f4a141c8860 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009fd0 tx=0x7f4a08004750 comp rx=0 tx=0).stop 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 shutdown_connections 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a14133fa0 0x7f4a141c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a140a0a30 0x7f4a141c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 --2- 172.21.15.32:0/1584922394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a140a1340 0x7f4a141c8860 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 >> 172.21.15.32:0/1584922394 conn(0x7f4a140a6550 msgr2=0x7f4a1411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 shutdown_connections 2023-09-30T22:29:40.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.164+0000 7f4a1a317700 1 -- 172.21.15.32:0/1584922394 wait complete. 2023-09-30T22:29:40.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:40.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:40.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.802+0000 7f18732b9700 1 Processor -- start 2023-09-30T22:29:40.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 -- start start 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 0x7f186c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c134260 0x7f186c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c06b990 con 0x7f186c133e30 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0a0590 con 0x7f186c133420 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f18732b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186c09fb80 con 0x7f186c134260 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f1870854700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 0x7f186c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f1871055700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f1871856700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c134260 0x7f186c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f1870854700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 0x7f186c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46390/0 (socket says 172.21.15.32:46390) 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.803+0000 7f1870854700 1 -- 172.21.15.32:0/5854700 learned_addr learned my addr 172.21.15.32:0/5854700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871055700 1 -- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c134260 msgr2=0x7f186c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871055700 1 --2- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c134260 0x7f186c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871055700 1 -- 172.21.15.32:0/5854700 >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 msgr2=0x7f186c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871055700 1 --2- 172.21.15.32:0/5854700 >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 0x7f186c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871055700 1 -- 172.21.15.32:0/5854700 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0a0030 con 0x7f186c133420 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1870854700 1 --2- 172.21.15.32:0/5854700 >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 0x7f186c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871856700 1 --2- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c134260 0x7f186c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1871055700 1 --2- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c1337f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1868009840 tx=0x7f1868009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a36e00648b2423 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1863fff700 1 -- 172.21.15.32:0/5854700 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868011830 con 0x7f186c133420 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1863fff700 1 -- 172.21.15.32:0/5854700 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1868011990 con 0x7f186c133420 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.804+0000 7f1863fff700 1 -- 172.21.15.32:0/5854700 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868011d20 con 0x7f186c133420 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 -- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 msgr2=0x7f186c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 --2- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c1337f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1868009840 tx=0x7f1868009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 -- 172.21.15.32:0/5854700 shutdown_connections 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 --2- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c1337f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 --2- 172.21.15.32:0/5854700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c134260 0x7f186c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 --2- 172.21.15.32:0/5854700 >> v2:172.21.15.32:3300/0 conn(0x7f186c133e30 0x7f186c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 -- 172.21.15.32:0/5854700 >> 172.21.15.32:0/5854700 conn(0x7f186c0a4210 msgr2=0x7f186c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 -- 172.21.15.32:0/5854700 shutdown_connections 2023-09-30T22:29:40.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.805+0000 7f18732b9700 1 -- 172.21.15.32:0/5854700 wait complete. 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 Processor -- start 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 -- start start 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c133e30 0x7f186c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186c134260 0x7f186c071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c11ff90 con 0x7f186c134260 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c120830 con 0x7f186c133420 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.806+0000 7f18732b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186c11fbf0 con 0x7f186c133e30 2023-09-30T22:29:40.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1870854700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c133e30 0x7f186c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871856700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186c134260 0x7f186c071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1870854700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c133e30 0x7f186c070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48698/0 (socket says 172.21.15.32:48698) 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:39948/0 (socket says 172.21.15.32:39948) 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1870854700 1 -- 172.21.15.32:0/2080148476 learned_addr learned my addr 172.21.15.32:0/2080148476 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 -- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c133e30 msgr2=0x7f186c070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c133e30 0x7f186c070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 -- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186c134260 msgr2=0x7f186c071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186c134260 0x7f186c071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 -- 172.21.15.32:0/2080148476 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009550 con 0x7f186c133420 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1871055700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c0709a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1868016000 tx=0x7f1868004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:40.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868004240 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186801f070 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1864009a70 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.807+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18680036a0 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.808+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18680094a0 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.808+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1868021030 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.808+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f18680407c0 con 0x7f186c133420 2023-09-30T22:29:40.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.809+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c11fdb0 con 0x7f186c133420 2023-09-30T22:29:40.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:40.814+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18680239c0 con 0x7f186c133420 2023-09-30T22:29:40.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.021+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f186c074ea0 con 0x7f186c133420 2023-09-30T22:29:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.021+0000 7f1861ffb700 1 -- 172.21.15.32:0/2080148476 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f186801b430 con 0x7f186c133420 2023-09-30T22:29:41.022 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 msgr2=0x7f186c0709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c0709a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1868016000 tx=0x7f1868004750 comp rx=0 tx=0).stop 2023-09-30T22:29:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 shutdown_connections 2023-09-30T22:29:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f186c134260 0x7f186c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f186c133420 0x7f186c0709a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 --2- 172.21.15.32:0/2080148476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f186c133e30 0x7f186c070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 >> 172.21.15.32:0/2080148476 conn(0x7f186c0a4210 msgr2=0x7f186c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 shutdown_connections 2023-09-30T22:29:41.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.024+0000 7f18732b9700 1 -- 172.21.15.32:0/2080148476 wait complete. 2023-09-30T22:29:41.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 Processor -- start 2023-09-30T22:29:41.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 -- start start 2023-09-30T22:29:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba0009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba0009f240 0x7fba001317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba00070940 con 0x7fba0009f650 2023-09-30T22:29:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0006fd60 con 0x7fba0009f240 2023-09-30T22:29:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.069+0000 7fba04f31700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba0006fa80 con 0x7fba0009e930 2023-09-30T22:29:41.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fe59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba0009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fdd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba0009f240 0x7fba001317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba0009f650 0x7fba00131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38086/0 (socket says 172.21.15.32:38086) 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fed9d700 1 -- 172.21.15.32:0/2744299704 learned_addr learned my addr 172.21.15.32:0/2744299704 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fd59a700 1 -- 172.21.15.32:0/2744299704 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677381777 0 0) 0x7fba00070940 con 0x7fba0009f650 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fd59a700 1 -- 172.21.15.32:0/2744299704 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e80035b0 con 0x7fba0009f650 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fdd9b700 1 -- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 msgr2=0x7fba0009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fdd9b700 1 --2- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba0009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fdd9b700 1 -- 172.21.15.32:0/2744299704 >> v1:172.21.15.32:6789/0 conn(0x7fba0009f650 legacy=0x7fba00131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fdd9b700 1 -- 172.21.15.32:0/2744299704 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0009fb60 con 0x7fba0009f240 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fe59c700 1 --2- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba0009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fdd9b700 1 --2- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001317f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009840 tx=0x7fb9f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ff6b547dc28d32 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fd59a700 1 -- 172.21.15.32:0/2744299704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4011830 con 0x7fba0009f240 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fd59a700 1 -- 172.21.15.32:0/2744299704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f4011990 con 0x7fba0009f240 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.070+0000 7fb9fd59a700 1 -- 172.21.15.32:0/2744299704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4011d20 con 0x7fba0009f240 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 -- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 msgr2=0x7fba001317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 --2- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001317f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009840 tx=0x7fb9f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 -- 172.21.15.32:0/2744299704 shutdown_connections 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 --2- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba0009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 --2- 172.21.15.32:0/2744299704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001317f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 -- 172.21.15.32:0/2744299704 >> 172.21.15.32:0/2744299704 conn(0x7fba000a4210 msgr2=0x7fba000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 -- 172.21.15.32:0/2744299704 shutdown_connections 2023-09-30T22:29:41.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.071+0000 7fba04f31700 1 -- 172.21.15.32:0/2744299704 wait complete. 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.072+0000 7fba04f31700 1 Processor -- start 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.072+0000 7fba04f31700 1 -- start start 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fba04f31700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba001a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fba04f31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fba04f31700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba0009f650 0x7fba001a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fba04f31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba001a5f00 con 0x7fba0009f240 2023-09-30T22:29:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fba04f31700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba001a6bc0 con 0x7fba0009f650 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fba04f31700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba000a32f0 con 0x7fba0009e930 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fe59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba001a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fe59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba001a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48720/0 (socket says 172.21.15.32:48720) 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fe59c700 1 -- 172.21.15.32:0/992523583 learned_addr learned my addr 172.21.15.32:0/992523583 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fed9d700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba0009f650 0x7fba001a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 -- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 msgr2=0x7fba001a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba001a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 -- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba0009f650 msgr2=0x7fba001a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba0009f650 0x7fba001a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 -- 172.21.15.32:0/992523583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00094a0 con 0x7fba0009f240 2023-09-30T22:29:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fe59c700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba001a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fed9d700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba0009f650 0x7fba001a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.073+0000 7fb9fdd9b700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001a4210 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4016000 tx=0x7fb9f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.074+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4004380 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.074+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f401f070 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.074+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f40094a0 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.074+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40036a0 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.074+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba000a0190 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.074+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9f4021030 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.075+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb9f40407c0 con 0x7fba0009f240 2023-09-30T22:29:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.075+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba001a0b30 con 0x7fba0009f240 2023-09-30T22:29:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.080+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9f4015080 con 0x7fba0009f240 2023-09-30T22:29:41.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:41.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.278+0000 7f05d98b9700 1 Processor -- start 2023-09-30T22:29:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.278+0000 7f05d98b9700 1 -- start start 2023-09-30T22:29:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d98b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d98b9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05d409f160 0x7f05d4133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d98b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f5b0 0x7f05d4134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d98b9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d406bf30 con 0x7f05d409f160 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d98b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d409ffb0 con 0x7f05d409e850 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d98b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05d406cb10 con 0x7f05d409f5b0 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05d409f160 0x7f05d4133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05d409f160 0x7f05d4133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46412/0 (socket says 172.21.15.32:46412) 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d27fc700 1 -- 172.21.15.32:0/4242671619 learned_addr learned my addr 172.21.15.32:0/4242671619 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:41.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d2ffd700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.279+0000 7f05d37fe700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f5b0 0x7f05d4134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d27fc700 1 -- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f5b0 msgr2=0x7f05d4134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d27fc700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f5b0 0x7f05d4134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d27fc700 1 -- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 msgr2=0x7f05d409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d27fc700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d27fc700 1 -- 172.21.15.32:0/4242671619 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40a0270 con 0x7f05d409f160 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d2ffd700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d27fc700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f160 0x7f05d4133bf0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f05c80098f0 tx=0x7f05c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbd9aa58af2c90a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d1ffb700 1 -- 172.21.15.32:0/4242671619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8011830 con 0x7f05d409f160 2023-09-30T22:29:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d1ffb700 1 -- 172.21.15.32:0/4242671619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8011990 con 0x7f05d409f160 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.280+0000 7f05d1ffb700 1 -- 172.21.15.32:0/4242671619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8011d20 con 0x7f05d409f160 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 -- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f160 msgr2=0x7f05d4133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f160 0x7f05d4133bf0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f05c80098f0 tx=0x7f05c8009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 -- 172.21.15.32:0/4242671619 shutdown_connections 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f5b0 0x7f05d4134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 --2- 172.21.15.32:0/4242671619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f160 0x7f05d4133bf0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 -- 172.21.15.32:0/4242671619 >> 172.21.15.32:0/4242671619 conn(0x7f05d40a3f70 msgr2=0x7f05d40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 -- 172.21.15.32:0/4242671619 shutdown_connections 2023-09-30T22:29:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.281+0000 7f05d98b9700 1 -- 172.21.15.32:0/4242671619 wait complete. 2023-09-30T22:29:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.282+0000 7f05d98b9700 1 Processor -- start 2023-09-30T22:29:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.282+0000 7f05d98b9700 1 -- start start 2023-09-30T22:29:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d98b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d4122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d98b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 0x7f05d4122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d98b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f5b0 0x7f05d4123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d98b9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4123bf0 con 0x7f05d409f5b0 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d98b9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4124440 con 0x7f05d409e850 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d98b9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05d406cd30 con 0x7f05d409f160 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 0x7f05d4122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d4122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 0x7f05d4122cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48734/0 (socket says 172.21.15.32:48734) 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 -- 172.21.15.32:0/3976643739 learned_addr learned my addr 172.21.15.32:0/3976643739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d37fe700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f5b0 0x7f05d4123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 -- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 msgr2=0x7f05d4122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d4122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 -- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f5b0 msgr2=0x7f05d4123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f5b0 0x7f05d4123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d27fc700 1 -- 172.21.15.32:0/3976643739 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40094a0 con 0x7f05d409f160 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.283+0000 7f05d2ffd700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d4122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.284+0000 7f05d27fc700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 0x7f05d4122cd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009c40 tx=0x7f05c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.284+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8004240 con 0x7f05d409f160 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.284+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c801f070 con 0x7f05d409f160 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.284+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c8009550 con 0x7f05d409f160 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.284+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c80045e0 con 0x7f05d409f160 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.284+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d4126a40 con 0x7f05d409f160 2023-09-30T22:29:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.285+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05c8021030 con 0x7f05d409f160 2023-09-30T22:29:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.285+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d40747f0 con 0x7f05d409f160 2023-09-30T22:29:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.285+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f05c80409c0 con 0x7f05d409f160 2023-09-30T22:29:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.290+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05c8003e00 con 0x7f05d409f160 2023-09-30T22:29:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.290+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fba00074ea0 con 0x7fba0009f240 2023-09-30T22:29:41.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.291+0000 7fb9eeffd700 1 -- 172.21.15.32:0/992523583 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9f4023790 con 0x7fba0009f240 2023-09-30T22:29:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:41.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 msgr2=0x7fba001a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001a4210 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4016000 tx=0x7fb9f4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:41.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 shutdown_connections 2023-09-30T22:29:41.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba0009f650 0x7fba001a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba0009e930 0x7fba001a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 --2- 172.21.15.32:0/992523583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba0009f240 0x7fba001a4210 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 >> 172.21.15.32:0/992523583 conn(0x7fba000a4210 msgr2=0x7fba001b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 shutdown_connections 2023-09-30T22:29:41.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.293+0000 7fba04f31700 1 -- 172.21.15.32:0/992523583 wait complete. 2023-09-30T22:29:41.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.324+0000 7f034dc6c700 1 Processor -- start 2023-09-30T22:29:41.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.324+0000 7f034dc6c700 1 -- start start 2023-09-30T22:29:41.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f034dc6c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 0x7f03480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f034dc6c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03480a1340 0x7f0348133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f034dc6c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 0x7f0348134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f034dc6c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034809edd0 con 0x7f03480a0a30 2023-09-30T22:29:41.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f034dc6c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034809e1f0 con 0x7f03480a1340 2023-09-30T22:29:41.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f034dc6c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f034809dc50 con 0x7f0348133fa0 2023-09-30T22:29:41.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f03477fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 0x7f03480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0347fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 0x7f0348134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03480a1340 0x7f0348133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0347fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 0x7f0348134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48744/0 (socket says 172.21.15.32:48744) 2023-09-30T22:29:41.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f03477fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 0x7f03480a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46422/0 (socket says 172.21.15.32:46422) 2023-09-30T22:29:41.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0347fff700 1 -- 172.21.15.32:0/1232772733 learned_addr learned my addr 172.21.15.32:0/1232772733 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:41.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0346ffd700 1 -- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 msgr2=0x7f0348134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0346ffd700 1 --2- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 0x7f0348134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0346ffd700 1 -- 172.21.15.32:0/1232772733 >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 msgr2=0x7f03480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0346ffd700 1 --2- 172.21.15.32:0/1232772733 >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 0x7f03480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0346ffd700 1 -- 172.21.15.32:0/1232772733 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03480a1fe0 con 0x7f03480a1340 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f0347fff700 1 --2- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 0x7f0348134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.325+0000 7f03477fe700 1 --2- 172.21.15.32:0/1232772733 >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 0x7f03480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.326+0000 7f0346ffd700 1 --2- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03480a1340 0x7f0348133a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f033c009870 tx=0x7f033c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f135b7ddb6a14815 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.326+0000 7f03467fc700 1 -- 172.21.15.32:0/1232772733 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c011830 con 0x7f03480a1340 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.326+0000 7f03467fc700 1 -- 172.21.15.32:0/1232772733 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c011990 con 0x7f03480a1340 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.326+0000 7f03467fc700 1 -- 172.21.15.32:0/1232772733 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c011d20 con 0x7f03480a1340 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 -- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03480a1340 msgr2=0x7f0348133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03480a1340 0x7f0348133a60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f033c009870 tx=0x7f033c009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 -- 172.21.15.32:0/1232772733 shutdown_connections 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0348133fa0 0x7f0348134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1232772733 >> v2:172.21.15.32:3300/0 conn(0x7f03480a0a30 0x7f03480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1232772733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f03480a1340 0x7f0348133a60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 -- 172.21.15.32:0/1232772733 >> 172.21.15.32:0/1232772733 conn(0x7f03480a6550 msgr2=0x7f03480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 -- 172.21.15.32:0/1232772733 shutdown_connections 2023-09-30T22:29:41.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.327+0000 7f034dc6c700 1 -- 172.21.15.32:0/1232772733 wait complete. 2023-09-30T22:29:41.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.328+0000 7f034dc6c700 1 Processor -- start 2023-09-30T22:29:41.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.328+0000 7f034dc6c700 1 -- start start 2023-09-30T22:29:41.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f034dc6c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03480a0a30 0x7f03481206a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f034dc6c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 0x7f0348120be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f034dc6c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0348133fa0 0x7f0348121120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f034dc6c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0348121ae0 con 0x7f03480a1340 2023-09-30T22:29:41.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f034dc6c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0348122890 con 0x7f0348133fa0 2023-09-30T22:29:41.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f034dc6c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0348123950 con 0x7f03480a0a30 2023-09-30T22:29:41.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 0x7f0348120be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 0x7f0348120be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46438/0 (socket says 172.21.15.32:46438) 2023-09-30T22:29:41.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 -- 172.21.15.32:0/1241427531 learned_addr learned my addr 172.21.15.32:0/1241427531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0347fff700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0348133fa0 0x7f0348121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f03477fe700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03480a0a30 0x7f03481206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 -- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03480a0a30 msgr2=0x7f03481206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03480a0a30 0x7f03481206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 -- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0348133fa0 msgr2=0x7f0348121120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0348133fa0 0x7f0348121120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.329+0000 7f0346ffd700 1 -- 172.21.15.32:0/1241427531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380094a0 con 0x7f03480a1340 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f0347fff700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0348133fa0 0x7f0348121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f0346ffd700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 0x7f0348120be0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f033c009840 tx=0x7f033c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c004450 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c01f070 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0038e0 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0340009b30 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.330+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c0094a0 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.331+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f033c003a40 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.331+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f033c021070 con 0x7f03480a1340 2023-09-30T22:29:41.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.331+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0348123b80 con 0x7f03480a1340 2023-09-30T22:29:41.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.336+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0348123b80 con 0x7f03480a1340 2023-09-30T22:29:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.504+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05d406bbb0 con 0x7f05d409f160 2023-09-30T22:29:41.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.504+0000 7f05bbfff700 1 -- 172.21.15.32:0/3976643739 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05c80233d0 con 0x7f05d409f160 2023-09-30T22:29:41.504 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 msgr2=0x7f05d4122cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 0x7f05d4122cd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009c40 tx=0x7f05c8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 shutdown_connections 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d409f5b0 0x7f05d4123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d409e850 0x7f05d4122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 --2- 172.21.15.32:0/3976643739 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05d409f160 0x7f05d4122cd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.506+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 >> 172.21.15.32:0/3976643739 conn(0x7f05d40a3f70 msgr2=0x7f05d4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.507+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 shutdown_connections 2023-09-30T22:29:41.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.507+0000 7f05d98b9700 1 -- 172.21.15.32:0/3976643739 wait complete. 2023-09-30T22:29:41.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.542+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0348123b80 con 0x7f03480a1340 2023-09-30T22:29:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.542+0000 7f032ffff700 1 -- 172.21.15.32:0/1241427531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0348123b80 con 0x7f03480a1340 2023-09-30T22:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 msgr2=0x7f0348120be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 0x7f0348120be0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f033c009840 tx=0x7f033c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 shutdown_connections 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0348133fa0 0x7f0348121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f03480a0a30 0x7f03481206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 --2- 172.21.15.32:0/1241427531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03480a1340 0x7f0348120be0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 >> 172.21.15.32:0/1241427531 conn(0x7f03480a6550 msgr2=0x7f034809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 shutdown_connections 2023-09-30T22:29:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:41.545+0000 7f034dc6c700 1 -- 172.21.15.32:0/1241427531 wait complete. 2023-09-30T22:29:41.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:42.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.187+0000 7f5ec8e94700 1 Processor -- start 2023-09-30T22:29:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.187+0000 7f5ec8e94700 1 -- start start 2023-09-30T22:29:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec8e94700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec409e930 0x7f5ec409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec8e94700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec8e94700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec4070940 con 0x7f5ec409f650 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec8e94700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec406fd60 con 0x7f5ec409e930 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec8e94700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec406fa80 con 0x7f5ec409f240 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec259c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec409e930 0x7f5ec409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec2d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ec409f650 0x7f5ec4131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38094/0 (socket says 172.21.15.32:38094) 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec2d9d700 1 -- 172.21.15.32:0/1120090297 learned_addr learned my addr 172.21.15.32:0/1120090297 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec259c700 1 -- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 msgr2=0x7f5ec41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec259c700 1 --2- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec259c700 1 -- 172.21.15.32:0/1120090297 >> v1:172.21.15.32:6789/0 conn(0x7f5ec409f650 legacy=0x7f5ec4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec259c700 1 -- 172.21.15.32:0/1120090297 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec409fb60 con 0x7f5ec409e930 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.188+0000 7f5ec1d9b700 1 --2- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec259c700 1 --2- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409e930 0x7f5ec409ed00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009840 tx=0x7f5eb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c84a0217c51fea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec159a700 1 -- 172.21.15.32:0/1120090297 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4011830 con 0x7f5ec409e930 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec159a700 1 -- 172.21.15.32:0/1120090297 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb4011990 con 0x7f5ec409e930 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec159a700 1 -- 172.21.15.32:0/1120090297 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4011d20 con 0x7f5ec409e930 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 -- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409e930 msgr2=0x7f5ec409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409e930 0x7f5ec409ed00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009840 tx=0x7f5eb4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 -- 172.21.15.32:0/1120090297 shutdown_connections 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409e930 0x7f5ec409ed00 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/1120090297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 -- 172.21.15.32:0/1120090297 >> 172.21.15.32:0/1120090297 conn(0x7f5ec40a4210 msgr2=0x7f5ec40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 -- 172.21.15.32:0/1120090297 shutdown_connections 2023-09-30T22:29:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.189+0000 7f5ec8e94700 1 -- 172.21.15.32:0/1120090297 wait complete. 2023-09-30T22:29:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 Processor -- start 2023-09-30T22:29:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 -- start start 2023-09-30T22:29:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ec409e930 0x7f5ec406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 0x7f5ec406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41b7350 con 0x7f5ec409f650 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41a9500 con 0x7f5ec409e930 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec8e94700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41a8a20 con 0x7f5ec409f240 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 0x7f5ec406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ec409e930 0x7f5ec406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 0x7f5ec406c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46458/0 (socket says 172.21.15.32:46458) 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec2d9d700 1 -- 172.21.15.32:0/2092054624 learned_addr learned my addr 172.21.15.32:0/2092054624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.191+0000 7f5ec1d9b700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec2d9d700 1 -- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 msgr2=0x7f5ec406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec2d9d700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec2d9d700 1 -- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ec409e930 msgr2=0x7f5ec406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec2d9d700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ec409e930 0x7f5ec406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec2d9d700 1 -- 172.21.15.32:0/2092054624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80094a0 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec259c700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ec409e930 0x7f5ec406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec2d9d700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 0x7f5ec406c330 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00cbf0 tx=0x7f5ebc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00aa60 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc00abc0 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb40094a0 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc012400 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.192+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec40a0190 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.193+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ebc01b3b0 con 0x7f5ec409f650 2023-09-30T22:29:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.193+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ebc03f3c0 con 0x7f5ec409f650 2023-09-30T22:29:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.193+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec40a3530 con 0x7f5ec409f650 2023-09-30T22:29:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.198+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ebc017080 con 0x7f5ec409f650 2023-09-30T22:29:42.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:42.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.407+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5ec4071d20 con 0x7f5ec409f650 2023-09-30T22:29:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.408+0000 7f5eb2ffd700 1 -- 172.21.15.32:0/2092054624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ebc0192c0 con 0x7f5ec409f650 2023-09-30T22:29:42.408 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 msgr2=0x7f5ec406c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 0x7f5ec406c330 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00cbf0 tx=0x7f5ebc00cf00 comp rx=0 tx=0).stop 2023-09-30T22:29:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 shutdown_connections 2023-09-30T22:29:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec409f650 0x7f5ec406c330 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ec409e930 0x7f5ec406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 --2- 172.21.15.32:0/2092054624 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5ec409f240 0x7f5ec406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 >> 172.21.15.32:0/2092054624 conn(0x7f5ec40a4210 msgr2=0x7f5ec41b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 shutdown_connections 2023-09-30T22:29:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.410+0000 7f5ec8e94700 1 -- 172.21.15.32:0/2092054624 wait complete. 2023-09-30T22:29:42.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.450+0000 7f5fe52fa700 1 Processor -- start 2023-09-30T22:29:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fe52fa700 1 -- start start 2023-09-30T22:29:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fe52fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe0130750 0x7f5fe0130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fe52fa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fe009d6f0 0x7f5fe009db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fe52fa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe009c190 con 0x7f5fe009e080 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fe52fa700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe009e520 con 0x7f5fe009d6f0 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fe52fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe009b090 con 0x7f5fe0130750 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fdeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe0130750 0x7f5fe0130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.451+0000 7f5fde7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fe009d6f0 0x7f5fe009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fdf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5fe009e080 0x7f5fe01374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38100/0 (socket says 172.21.15.32:38100) 2023-09-30T22:29:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fdf7fe700 1 -- 172.21.15.32:0/1406817149 learned_addr learned my addr 172.21.15.32:0/1406817149 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fde7fc700 1 -- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe0130750 msgr2=0x7f5fe0130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fde7fc700 1 --2- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe0130750 0x7f5fe0130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fde7fc700 1 -- 172.21.15.32:0/1406817149 >> v1:172.21.15.32:6789/0 conn(0x7f5fe009e080 legacy=0x7f5fe01374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fddffb700 1 -- 172.21.15.32:0/1406817149 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371343324 0 0) 0x7f5fe009c190 con 0x7f5fe009e080 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fde7fc700 1 -- 172.21.15.32:0/1406817149 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe009a6c0 con 0x7f5fe009d6f0 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fdeffd700 1 --2- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe0130750 0x7f5fe0130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fde7fc700 1 --2- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe009d6f0 0x7f5fe009db40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009840 tx=0x7f5fd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5fbc46ef0ab74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fddffb700 1 -- 172.21.15.32:0/1406817149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011830 con 0x7f5fe009d6f0 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fddffb700 1 -- 172.21.15.32:0/1406817149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd4011990 con 0x7f5fe009d6f0 2023-09-30T22:29:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.452+0000 7f5fddffb700 1 -- 172.21.15.32:0/1406817149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011d20 con 0x7f5fe009d6f0 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 -- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe009d6f0 msgr2=0x7f5fe009db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe009d6f0 0x7f5fe009db40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009840 tx=0x7f5fd4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 -- 172.21.15.32:0/1406817149 shutdown_connections 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe0130750 0x7f5fe0130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/1406817149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe009d6f0 0x7f5fe009db40 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 -- 172.21.15.32:0/1406817149 >> 172.21.15.32:0/1406817149 conn(0x7f5fe009f470 msgr2=0x7f5fe00a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 -- 172.21.15.32:0/1406817149 shutdown_connections 2023-09-30T22:29:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.453+0000 7f5fe52fa700 1 -- 172.21.15.32:0/1406817149 wait complete. 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 Processor -- start 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 -- start start 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe009d6f0 0x7f5fe012bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fe009e080 0x7f5fe012c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 0x7f5fe012c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe012e690 con 0x7f5fe0130750 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe012ea40 con 0x7f5fe009e080 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fe52fa700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe012f3b0 con 0x7f5fe009d6f0 2023-09-30T22:29:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fdf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 0x7f5fe012c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fde7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fe009e080 0x7f5fe012c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fdf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 0x7f5fe012c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46472/0 (socket says 172.21.15.32:46472) 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.455+0000 7f5fdeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe009d6f0 0x7f5fe012bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 -- 172.21.15.32:0/3046189095 learned_addr learned my addr 172.21.15.32:0/3046189095 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 -- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe009d6f0 msgr2=0x7f5fe012bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe009d6f0 0x7f5fe012bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 -- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fe009e080 msgr2=0x7f5fe012c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fe009e080 0x7f5fe012c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 -- 172.21.15.32:0/3046189095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd000b740 con 0x7f5fe0130750 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fde7fc700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fe009e080 0x7f5fe012c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fdf7fe700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 0x7f5fe012c960 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800cba0 tx=0x7f5fd800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800aaa0 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd800ac20 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd40094a0 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80070d0 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.456+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe01310e0 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.457+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fd8007230 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.457+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5fd803f340 con 0x7f5fe0130750 2023-09-30T22:29:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.457+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe012f050 con 0x7f5fe0130750 2023-09-30T22:29:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.462+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fd8018080 con 0x7f5fe0130750 2023-09-30T22:29:42.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:42.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.661+0000 7f05c3546700 1 Processor -- start 2023-09-30T22:29:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.661+0000 7f05c3546700 1 -- start start 2023-09-30T22:29:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.661+0000 7f05c3546700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 0x7f05bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.661+0000 7f05c3546700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05bc0a1340 0x7f05bc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.661+0000 7f05c3546700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.661+0000 7f05c3546700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc09edd0 con 0x7f05bc0a1340 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c3546700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc09e1f0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c3546700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc09dc50 con 0x7f05bc133fa0 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 0x7f05bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 0x7f05bc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40036/0 (socket says 172.21.15.32:40036) 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 -- 172.21.15.32:0/1424465830 learned_addr learned my addr 172.21.15.32:0/1424465830 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c1ae3700 1 --2- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c0ae1700 1 --2- 172.21.15.32:0/1424465830 >> v2:172.21.15.32:3300/0 conn(0x7f05bc0a1340 0x7f05bc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 -- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 msgr2=0x7f05bc134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 --2- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 -- 172.21.15.32:0/1424465830 >> v2:172.21.15.32:3300/0 conn(0x7f05bc0a1340 msgr2=0x7f05bc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 --2- 172.21.15.32:0/1424465830 >> v2:172.21.15.32:3300/0 conn(0x7f05bc0a1340 0x7f05bc133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 -- 172.21.15.32:0/1424465830 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0a1fe0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.662+0000 7f05c12e2700 1 --2- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 0x7f05bc0a0e00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009920 tx=0x7f05b8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f388685bb2d90a89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05b3fff700 1 -- 172.21.15.32:0/1424465830 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8011830 con 0x7f05bc0a0a30 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05b3fff700 1 -- 172.21.15.32:0/1424465830 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b8011990 con 0x7f05bc0a0a30 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05b3fff700 1 -- 172.21.15.32:0/1424465830 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8011d20 con 0x7f05bc0a0a30 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 -- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 msgr2=0x7f05bc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 --2- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 0x7f05bc0a0e00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009920 tx=0x7f05b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 -- 172.21.15.32:0/1424465830 shutdown_connections 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 --2- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a0a30 0x7f05bc0a0e00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 --2- 172.21.15.32:0/1424465830 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 --2- 172.21.15.32:0/1424465830 >> v2:172.21.15.32:3300/0 conn(0x7f05bc0a1340 0x7f05bc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.663+0000 7f05c3546700 1 -- 172.21.15.32:0/1424465830 >> 172.21.15.32:0/1424465830 conn(0x7f05bc0a6550 msgr2=0x7f05bc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.664+0000 7f05c3546700 1 -- 172.21.15.32:0/1424465830 shutdown_connections 2023-09-30T22:29:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.664+0000 7f05c3546700 1 -- 172.21.15.32:0/1424465830 wait complete. 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.665+0000 7f05c3546700 1 Processor -- start 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.665+0000 7f05c3546700 1 -- start start 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c3546700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 0x7f05bc121410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c3546700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a1340 0x7f05bc121950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c3546700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc121e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c3546700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0a37c0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c3546700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc1235c0 con 0x7f05bc0a1340 2023-09-30T22:29:42.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c3546700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0a0220 con 0x7f05bc133fa0 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 0x7f05bc121410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 0x7f05bc121410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46492/0 (socket says 172.21.15.32:46492) 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 -- 172.21.15.32:0/2407988211 learned_addr learned my addr 172.21.15.32:0/2407988211 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c1ae3700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc121e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c0ae1700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a1340 0x7f05bc121950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 -- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 msgr2=0x7f05bc121e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc121e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 -- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a1340 msgr2=0x7f05bc121950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a1340 0x7f05bc121950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c12e2700 1 -- 172.21.15.32:0/2407988211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009550 con 0x7f05bc0a0a30 2023-09-30T22:29:42.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.666+0000 7f05c1ae3700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc121e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.667+0000 7f05c12e2700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 0x7f05bc121410 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009130 tx=0x7f05b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.667+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8004450 con 0x7f05bc0a0a30 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.667+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b801e070 con 0x7f05bc0a0a30 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.667+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc123e40 con 0x7f05bc0a0a30 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.667+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80038e0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.667+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc124fd0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.668+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05b8003a40 con 0x7f05bc0a0a30 2023-09-30T22:29:42.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.668+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f05b8040430 con 0x7f05bc0a0a30 2023-09-30T22:29:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.669+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc074ea0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.672+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5fe00a04f0 con 0x7f5fe0130750 2023-09-30T22:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.678+0000 7f5fc77fe700 1 -- 172.21.15.32:0/3046189095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fd801a8b0 con 0x7f5fe0130750 2023-09-30T22:29:42.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.678+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05b8015020 con 0x7f05bc0a0a30 2023-09-30T22:29:42.678 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:42.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 msgr2=0x7f5fe012c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 0x7f5fe012c960 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800cba0 tx=0x7f5fd800ceb0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 shutdown_connections 2023-09-30T22:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe0130750 0x7f5fe012c960 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5fe009d6f0 0x7f5fe012bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 --2- 172.21.15.32:0/3046189095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fe009e080 0x7f5fe012c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 >> 172.21.15.32:0/3046189095 conn(0x7f5fe009f470 msgr2=0x7f5fe00992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 shutdown_connections 2023-09-30T22:29:42.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.680+0000 7f5fe52fa700 1 -- 172.21.15.32:0/3046189095 wait complete. 2023-09-30T22:29:42.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:42.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.708+0000 7f11c81ba700 1 Processor -- start 2023-09-30T22:29:42.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 -- start start 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11c012ff20 0x7f11c0130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c013a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c006c570 con 0x7f11c012ff20 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c006bc00 con 0x7f11c0133a80 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c81ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00a0040 con 0x7f11c0130870 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c5755700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11c012ff20 0x7f11c0130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c6757700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c5f56700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c5f56700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c0133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40050/0 (socket says 172.21.15.32:40050) 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c5755700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11c012ff20 0x7f11c0130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46494/0 (socket says 172.21.15.32:46494) 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c5f56700 1 -- 172.21.15.32:0/2468911250 learned_addr learned my addr 172.21.15.32:0/2468911250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.709+0000 7f11c6757700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c013a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48818/0 (socket says 172.21.15.32:48818) 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c6757700 1 -- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 msgr2=0x7f11c0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c6757700 1 --2- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c0133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c6757700 1 -- 172.21.15.32:0/2468911250 >> v2:172.21.15.32:3300/0 conn(0x7f11c012ff20 msgr2=0x7f11c0130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c6757700 1 --2- 172.21.15.32:0/2468911250 >> v2:172.21.15.32:3300/0 conn(0x7f11c012ff20 0x7f11c0130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c6757700 1 -- 172.21.15.32:0/2468911250 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0133860 con 0x7f11c0130870 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c5f56700 1 --2- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c0133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c6757700 1 --2- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c013a840 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009840 tx=0x7f11b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df3c3a4e5945d12 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c4f54700 1 -- 172.21.15.32:0/2468911250 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0011830 con 0x7f11c0130870 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c4f54700 1 -- 172.21.15.32:0/2468911250 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b0011990 con 0x7f11c0130870 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.710+0000 7f11c4f54700 1 -- 172.21.15.32:0/2468911250 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0011d20 con 0x7f11c0130870 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 -- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 msgr2=0x7f11c013a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c013a840 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009840 tx=0x7f11b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 -- 172.21.15.32:0/2468911250 shutdown_connections 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c0133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2468911250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c013a840 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2468911250 >> v2:172.21.15.32:3300/0 conn(0x7f11c012ff20 0x7f11c0130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 -- 172.21.15.32:0/2468911250 >> 172.21.15.32:0/2468911250 conn(0x7f11c00a4210 msgr2=0x7f11c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 -- 172.21.15.32:0/2468911250 shutdown_connections 2023-09-30T22:29:42.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.711+0000 7f11c81ba700 1 -- 172.21.15.32:0/2468911250 wait complete. 2023-09-30T22:29:42.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 Processor -- start 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 -- start start 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c012ff20 0x7f11c009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0120340 con 0x7f11c012ff20 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c011fe30 con 0x7f11c0133a80 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.712+0000 7f11c81ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0120840 con 0x7f11c0130870 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c6757700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5f56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c012ff20 0x7f11c009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c6757700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40062/0 (socket says 172.21.15.32:40062) 2023-09-30T22:29:42.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c6757700 1 -- 172.21.15.32:0/2658488068 learned_addr learned my addr 172.21.15.32:0/2658488068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 -- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 msgr2=0x7f11c009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 -- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c012ff20 msgr2=0x7f11c009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c012ff20 0x7f11c009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 -- 172.21.15.32:0/2658488068 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8009b30 con 0x7f11c0130870 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c6757700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c5755700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c009c9c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e900 tx=0x7f11b400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4013680 con 0x7f11c0130870 2023-09-30T22:29:42.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b40137e0 con 0x7f11c0130870 2023-09-30T22:29:42.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b00094a0 con 0x7f11c0130870 2023-09-30T22:29:42.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4013b50 con 0x7f11c0130870 2023-09-30T22:29:42.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.713+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c0120b10 con 0x7f11c0130870 2023-09-30T22:29:42.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.714+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11b4013cb0 con 0x7f11c0130870 2023-09-30T22:29:42.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.714+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f11b403f970 con 0x7f11c0130870 2023-09-30T22:29:42.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.714+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c0120180 con 0x7f11c0130870 2023-09-30T22:29:42.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.719+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11b4017080 con 0x7f11c0130870 2023-09-30T22:29:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.888+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05bc0a28e0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.888+0000 7f05b1ffb700 1 -- 172.21.15.32:0/2407988211 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05b80045b0 con 0x7f05bc0a0a30 2023-09-30T22:29:42.889 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 msgr2=0x7f05bc121410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 0x7f05bc121410 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009130 tx=0x7f05b8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 shutdown_connections 2023-09-30T22:29:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05bc133fa0 0x7f05bc121e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05bc0a0a30 0x7f05bc121410 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 --2- 172.21.15.32:0/2407988211 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05bc0a1340 0x7f05bc121950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 >> 172.21.15.32:0/2407988211 conn(0x7f05bc0a6550 msgr2=0x7f05bc133630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 shutdown_connections 2023-09-30T22:29:42.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.891+0000 7f05c3546700 1 -- 172.21.15.32:0/2407988211 wait complete. 2023-09-30T22:29:42.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:42.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.922+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11c006d740 con 0x7f11c0130870 2023-09-30T22:29:42.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.922+0000 7f11be7fc700 1 -- 172.21.15.32:0/2658488068 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11b401ada0 con 0x7f11c0130870 2023-09-30T22:29:42.922 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 msgr2=0x7f11c009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c009c9c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e900 tx=0x7f11b400ec10 comp rx=0 tx=0).stop 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 shutdown_connections 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11c0133a80 0x7f11c009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11c012ff20 0x7f11c009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 --2- 172.21.15.32:0/2658488068 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11c0130870 0x7f11c009c9c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 >> 172.21.15.32:0/2658488068 conn(0x7f11c00a4210 msgr2=0x7f11c00a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 shutdown_connections 2023-09-30T22:29:42.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:42.924+0000 7f11c81ba700 1 -- 172.21.15.32:0/2658488068 wait complete. 2023-09-30T22:29:42.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:43.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.571+0000 7fc6c6ca9700 1 Processor -- start 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.571+0000 7fc6c6ca9700 1 -- start start 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c6ca9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6c0133a80 0x7fc6c0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c6ca9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c012ff20 0x7fc6c0130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c6ca9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c006bc00 con 0x7fc6c0130870 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c6ca9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c006c570 con 0x7fc6c0133a80 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c6ca9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00a0040 con 0x7fc6c012ff20 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6bffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c012ff20 0x7fc6c0130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c4a45700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6c0133a80 0x7fc6c0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c5246700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc6c0130870 0x7fc6c013a840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38104/0 (socket says 172.21.15.32:38104) 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6c5246700 1 -- 172.21.15.32:0/2264848266 learned_addr learned my addr 172.21.15.32:0/2264848266 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6bffff700 1 -- 172.21.15.32:0/2264848266 >> v2:172.21.15.32:3300/0 conn(0x7fc6c0133a80 msgr2=0x7fc6c0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6bffff700 1 --2- 172.21.15.32:0/2264848266 >> v2:172.21.15.32:3300/0 conn(0x7fc6c0133a80 0x7fc6c0133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6bffff700 1 -- 172.21.15.32:0/2264848266 >> v1:172.21.15.32:6789/0 conn(0x7fc6c0130870 legacy=0x7fc6c013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6bffff700 1 -- 172.21.15.32:0/2264848266 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c0133860 con 0x7fc6c012ff20 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.572+0000 7fc6bffff700 1 --2- 172.21.15.32:0/2264848266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c012ff20 0x7fc6c0130330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009840 tx=0x7fc6a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a0a02a34667e25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6bf7fe700 1 -- 172.21.15.32:0/2264848266 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8011830 con 0x7fc6c012ff20 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6bf7fe700 1 -- 172.21.15.32:0/2264848266 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a8011990 con 0x7fc6c012ff20 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6bf7fe700 1 -- 172.21.15.32:0/2264848266 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8011d20 con 0x7fc6c012ff20 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/2264848266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c012ff20 msgr2=0x7fc6c0130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/2264848266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c012ff20 0x7fc6c0130330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009840 tx=0x7fc6a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/2264848266 shutdown_connections 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/2264848266 >> v2:172.21.15.32:3300/0 conn(0x7fc6c0133a80 0x7fc6c0133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/2264848266 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c012ff20 0x7fc6c0130330 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/2264848266 >> 172.21.15.32:0/2264848266 conn(0x7fc6c00a4210 msgr2=0x7fc6c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/2264848266 shutdown_connections 2023-09-30T22:29:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.573+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/2264848266 wait complete. 2023-09-30T22:29:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.574+0000 7fc6c6ca9700 1 Processor -- start 2023-09-30T22:29:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.574+0000 7fc6c6ca9700 1 -- start start 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c6ca9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6c012ff20 0x7fc6c009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c6ca9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 0x7fc6c009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c6ca9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c0133a80 0x7fc6c009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c6ca9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c009e4a0 con 0x7fc6c0130870 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c6ca9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0120060 con 0x7fc6c012ff20 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c6ca9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c012db00 con 0x7fc6c0133a80 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c4a45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6c012ff20 0x7fc6c009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 0x7fc6c009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 0x7fc6c009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46534/0 (socket says 172.21.15.32:46534) 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 -- 172.21.15.32:0/424641478 learned_addr learned my addr 172.21.15.32:0/424641478 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c4a45700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6c012ff20 0x7fc6c009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40066/0 (socket says 172.21.15.32:40066) 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6c5246700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c0133a80 0x7fc6c009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 -- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c0133a80 msgr2=0x7fc6c009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c0133a80 0x7fc6c009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 -- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6c012ff20 msgr2=0x7fc6c009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6c012ff20 0x7fc6c009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.575+0000 7fc6bffff700 1 -- 172.21.15.32:0/424641478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a80094a0 con 0x7fc6c0130870 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.576+0000 7fc6bffff700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 0x7fc6c009c9c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009fd0 tx=0x7fc6a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.576+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80036a0 con 0x7fc6c0130870 2023-09-30T22:29:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.576+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a801f070 con 0x7fc6c0130870 2023-09-30T22:29:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.576+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0120ad0 con 0x7fc6c0130870 2023-09-30T22:29:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.576+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8003a60 con 0x7fc6c0130870 2023-09-30T22:29:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.576+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c009def0 con 0x7fc6c0130870 2023-09-30T22:29:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.577+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6a8003bc0 con 0x7fc6c0130870 2023-09-30T22:29:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.577+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c009e0e0 con 0x7fc6c0130870 2023-09-30T22:29:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.577+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc6a8021070 con 0x7fc6c0130870 2023-09-30T22:29:43.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.582+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6a8015070 con 0x7fc6c0130870 2023-09-30T22:29:43.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:43.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.791+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6c0074ea0 con 0x7fc6c0130870 2023-09-30T22:29:43.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.792+0000 7fc6bd7fa700 1 -- 172.21.15.32:0/424641478 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6a801a430 con 0x7fc6c0130870 2023-09-30T22:29:43.792 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:43.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 msgr2=0x7fc6c009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 0x7fc6c009c9c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009fd0 tx=0x7fc6a8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 shutdown_connections 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc6c0133a80 0x7fc6c009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6c012ff20 0x7fc6c009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 --2- 172.21.15.32:0/424641478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0130870 0x7fc6c009c9c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 >> 172.21.15.32:0/424641478 conn(0x7fc6c00a4210 msgr2=0x7fc6c009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 shutdown_connections 2023-09-30T22:29:43.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.794+0000 7fc6c6ca9700 1 -- 172.21.15.32:0/424641478 wait complete. 2023-09-30T22:29:43.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.834+0000 7fc85e388700 1 Processor -- start 2023-09-30T22:29:43.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.834+0000 7fc85e388700 1 -- start start 2023-09-30T22:29:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85e388700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 0x7fc85806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85e388700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 0x7fc858131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85e388700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 0x7fc858131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85e388700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85809ece0 con 0x7fc85806a770 2023-09-30T22:29:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85e388700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85809e760 con 0x7fc858131b40 2023-09-30T22:29:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85e388700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85809e040 con 0x7fc85806b080 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc857fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 0x7fc85806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 0x7fc858131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85c925700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 0x7fc858131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc857fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 0x7fc85806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46536/0 (socket says 172.21.15.32:46536) 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 0x7fc858131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48858/0 (socket says 172.21.15.32:48858) 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85c925700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 0x7fc858131fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40074/0 (socket says 172.21.15.32:40074) 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc857fff700 1 -- 172.21.15.32:0/3631736129 learned_addr learned my addr 172.21.15.32:0/3631736129 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 -- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 msgr2=0x7fc858131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 --2- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 0x7fc858131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 -- 172.21.15.32:0/3631736129 >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 msgr2=0x7fc85806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:43.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 --2- 172.21.15.32:0/3631736129 >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 0x7fc85806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 -- 172.21.15.32:0/3631736129 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85809f630 con 0x7fc85806b080 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc857fff700 1 --2- 172.21.15.32:0/3631736129 >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 0x7fc85806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc85c925700 1 --2- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 0x7fc858131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc8577fe700 1 --2- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 0x7fc858131600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009840 tx=0x7fc84c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70130fa32684a952 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.835+0000 7fc856ffd700 1 -- 172.21.15.32:0/3631736129 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011830 con 0x7fc85806b080 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc856ffd700 1 -- 172.21.15.32:0/3631736129 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c011990 con 0x7fc85806b080 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc856ffd700 1 -- 172.21.15.32:0/3631736129 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011d20 con 0x7fc85806b080 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 -- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 msgr2=0x7fc858131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 --2- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 0x7fc858131600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009840 tx=0x7fc84c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 -- 172.21.15.32:0/3631736129 shutdown_connections 2023-09-30T22:29:43.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 --2- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc858131b40 0x7fc858131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 --2- 172.21.15.32:0/3631736129 >> v2:172.21.15.32:3300/0 conn(0x7fc85806a770 0x7fc85806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 --2- 172.21.15.32:0/3631736129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc85806b080 0x7fc858131600 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 -- 172.21.15.32:0/3631736129 >> 172.21.15.32:0/3631736129 conn(0x7fc858066f50 msgr2=0x7fc85809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 -- 172.21.15.32:0/3631736129 shutdown_connections 2023-09-30T22:29:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.836+0000 7fc85e388700 1 -- 172.21.15.32:0/3631736129 wait complete. 2023-09-30T22:29:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.837+0000 7fc85e388700 1 Processor -- start 2023-09-30T22:29:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.837+0000 7fc85e388700 1 -- start start 2023-09-30T22:29:43.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85e388700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc85806a770 0x7fc8581c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85e388700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 0x7fc8581c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85e388700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc858131b40 0x7fc8581c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85e388700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858122310 con 0x7fc85806b080 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85e388700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc858121f80 con 0x7fc85806a770 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85e388700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85809c010 con 0x7fc858131b40 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 0x7fc8581c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85c925700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc858131b40 0x7fc8581c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 0x7fc8581c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46542/0 (socket says 172.21.15.32:46542) 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 -- 172.21.15.32:0/3416969683 learned_addr learned my addr 172.21.15.32:0/3416969683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc857fff700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc85806a770 0x7fc8581c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 -- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc858131b40 msgr2=0x7fc8581c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc858131b40 0x7fc8581c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 -- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc85806a770 msgr2=0x7fc8581c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc85806a770 0x7fc8581c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc8577fe700 1 -- 172.21.15.32:0/3416969683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848009550 con 0x7fc85806b080 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc857fff700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc85806a770 0x7fc8581c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.838+0000 7fc85c925700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc858131b40 0x7fc8581c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.839+0000 7fc8577fe700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 0x7fc8581c85f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.839+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004450 con 0x7fc85806b080 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.839+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c01f070 con 0x7fc85806b080 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.839+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc850009a70 con 0x7fc85806b080 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.839+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0038e0 con 0x7fc85806b080 2023-09-30T22:29:43.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.839+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c0094a0 con 0x7fc85806b080 2023-09-30T22:29:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.840+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc858074ea0 con 0x7fc85806b080 2023-09-30T22:29:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.840+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc84c003a40 con 0x7fc85806b080 2023-09-30T22:29:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.840+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc84c021070 con 0x7fc85806b080 2023-09-30T22:29:43.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:43.845+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc84c010380 con 0x7fc85806b080 2023-09-30T22:29:43.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:43.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:44.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.045+0000 7f58cee49700 1 Processor -- start 2023-09-30T22:29:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.045+0000 7f58cee49700 1 -- start start 2023-09-30T22:29:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cee49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cee49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cee49700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58c809e410 0x7f58c8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cee49700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c806b460 con 0x7f58c809e410 2023-09-30T22:29:44.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cee49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c809b120 con 0x7f58c809d6f0 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cee49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c809a7e0 con 0x7f58c809e000 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40092/0 (socket says 172.21.15.32:40092) 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 -- 172.21.15.32:0/3159775625 learned_addr learned my addr 172.21.15.32:0/3159775625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cd3e6700 1 --2- 172.21.15.32:0/3159775625 >> v2:172.21.15.32:3300/0 conn(0x7f58c809e410 0x7f58c8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58c7fff700 1 --2- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 -- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 msgr2=0x7f58c81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 --2- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 -- 172.21.15.32:0/3159775625 >> v2:172.21.15.32:3300/0 conn(0x7f58c809e410 msgr2=0x7f58c8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 --2- 172.21.15.32:0/3159775625 >> v2:172.21.15.32:3300/0 conn(0x7f58c809e410 0x7f58c8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58ccbe5700 1 -- 172.21.15.32:0/3159775625 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c809a500 con 0x7f58c809d6f0 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.046+0000 7f58cd3e6700 1 --2- 172.21.15.32:0/3159775625 >> v2:172.21.15.32:3300/0 conn(0x7f58c809e410 0x7f58c8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.047+0000 7f58ccbe5700 1 --2- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c809dac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009920 tx=0x7f58bc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a6d50f1f40f055 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.047+0000 7f58c77fe700 1 -- 172.21.15.32:0/3159775625 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc011830 con 0x7f58c809d6f0 2023-09-30T22:29:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.047+0000 7f58c77fe700 1 -- 172.21.15.32:0/3159775625 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc011990 con 0x7f58c809d6f0 2023-09-30T22:29:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.047+0000 7f58c77fe700 1 -- 172.21.15.32:0/3159775625 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc011d20 con 0x7f58c809d6f0 2023-09-30T22:29:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 -- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 msgr2=0x7f58c809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 --2- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c809dac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009920 tx=0x7f58bc009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 -- 172.21.15.32:0/3159775625 shutdown_connections 2023-09-30T22:29:44.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 --2- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c809dac0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 --2- 172.21.15.32:0/3159775625 >> v2:172.21.15.32:3300/0 conn(0x7f58c809e410 0x7f58c8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 --2- 172.21.15.32:0/3159775625 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 -- 172.21.15.32:0/3159775625 >> 172.21.15.32:0/3159775625 conn(0x7f58c8066f50 msgr2=0x7f58c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 -- 172.21.15.32:0/3159775625 shutdown_connections 2023-09-30T22:29:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.048+0000 7f58cee49700 1 -- 172.21.15.32:0/3159775625 wait complete. 2023-09-30T22:29:44.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.049+0000 7f58cee49700 1 Processor -- start 2023-09-30T22:29:44.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.049+0000 7f58cee49700 1 -- start start 2023-09-30T22:29:44.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cee49700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c81a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cee49700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cee49700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c809e410 0x7f58c81a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cee49700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c81a7ac0 con 0x7f58c809e410 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cee49700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c81a80e0 con 0x7f58c809d6f0 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cee49700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c81a8d10 con 0x7f58c809e000 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58ccbe5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c81a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58ccbe5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c81a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40100/0 (socket says 172.21.15.32:40100) 2023-09-30T22:29:44.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58ccbe5700 1 -- 172.21.15.32:0/2367498361 learned_addr learned my addr 172.21.15.32:0/2367498361 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cd3e6700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c809e410 0x7f58c81a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58c7fff700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cd3e6700 1 -- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 msgr2=0x7f58c81a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cd3e6700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cd3e6700 1 -- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 msgr2=0x7f58c81a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cd3e6700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c81a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58cd3e6700 1 -- 172.21.15.32:0/2367498361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c0009a70 con 0x7f58c809e410 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.050+0000 7f58ccbe5700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c81a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.051+0000 7f58cd3e6700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c809e410 0x7f58c81a69c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f58c000efc0 tx=0x7f58c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.051+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c000cc90 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.051+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c000f040 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.051+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0014480 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.051+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc009550 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.051+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c81a91c0 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.052+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c809c710 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.053+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58c0014600 con 0x7f58c809e410 2023-09-30T22:29:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.053+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f58c0040990 con 0x7f58c809e410 2023-09-30T22:29:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.054+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc858120a60 con 0x7fc85806b080 2023-09-30T22:29:44.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.058+0000 7fc854ff9700 1 -- 172.21.15.32:0/3416969683 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc84c015070 con 0x7fc85806b080 2023-09-30T22:29:44.061 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.058+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58c0012750 con 0x7f58c809e410 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 msgr2=0x7fc8581c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 0x7fc8581c85f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 shutdown_connections 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc858131b40 0x7fc8581c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc85806a770 0x7fc8581c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 --2- 172.21.15.32:0/3416969683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc85806b080 0x7fc8581c85f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.060+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 >> 172.21.15.32:0/3416969683 conn(0x7fc858066f50 msgr2=0x7fc858070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.061+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 shutdown_connections 2023-09-30T22:29:44.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.061+0000 7fc85e388700 1 -- 172.21.15.32:0/3416969683 wait complete. 2023-09-30T22:29:44.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.090+0000 7f38472df700 1 Processor -- start 2023-09-30T22:29:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.090+0000 7f38472df700 1 -- start start 2023-09-30T22:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f38472df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 0x7f3840131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f38472df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384009d9f0 0x7f384006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f38472df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f384006acb0 0x7f384006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f38472df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384009e130 con 0x7f38401316c0 2023-09-30T22:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f38472df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38401312b0 con 0x7f384009d9f0 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f38472df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f384009f6e0 con 0x7f384006acb0 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384507b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 0x7f3840131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384587c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f384006acb0 0x7f384006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384487a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384009d9f0 0x7f384006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384507b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 0x7f3840131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46572/0 (socket says 172.21.15.32:46572) 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384507b700 1 -- 172.21.15.32:0/2647646200 learned_addr learned my addr 172.21.15.32:0/2647646200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384487a700 1 -- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f384006acb0 msgr2=0x7f384006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384487a700 1 --2- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f384006acb0 0x7f384006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384487a700 1 -- 172.21.15.32:0/2647646200 >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 msgr2=0x7f3840131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384487a700 1 --2- 172.21.15.32:0/2647646200 >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 0x7f3840131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384487a700 1 -- 172.21.15.32:0/2647646200 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384009e760 con 0x7f384009d9f0 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384507b700 1 --2- 172.21.15.32:0/2647646200 >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 0x7f3840131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.091+0000 7f384587c700 1 --2- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f384006acb0 0x7f384006b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.092+0000 7f384487a700 1 --2- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384009d9f0 0x7f384006a770 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3834009840 tx=0x7f3834009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6514f38b70b078c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.092+0000 7f383ffff700 1 -- 172.21.15.32:0/2647646200 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834011830 con 0x7f384009d9f0 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.092+0000 7f383ffff700 1 -- 172.21.15.32:0/2647646200 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834011990 con 0x7f384009d9f0 2023-09-30T22:29:44.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.092+0000 7f383ffff700 1 -- 172.21.15.32:0/2647646200 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834011d20 con 0x7f384009d9f0 2023-09-30T22:29:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.092+0000 7f38472df700 1 -- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384009d9f0 msgr2=0x7f384006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.092+0000 7f38472df700 1 --2- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384009d9f0 0x7f384006a770 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3834009840 tx=0x7f3834009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 -- 172.21.15.32:0/2647646200 shutdown_connections 2023-09-30T22:29:44.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 --2- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f384006acb0 0x7f384006b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 --2- 172.21.15.32:0/2647646200 >> v2:172.21.15.32:3300/0 conn(0x7f38401316c0 0x7f3840131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 --2- 172.21.15.32:0/2647646200 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384009d9f0 0x7f384006a770 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 -- 172.21.15.32:0/2647646200 >> 172.21.15.32:0/2647646200 conn(0x7f3840066f50 msgr2=0x7f384009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 -- 172.21.15.32:0/2647646200 shutdown_connections 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.093+0000 7f38472df700 1 -- 172.21.15.32:0/2647646200 wait complete. 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 Processor -- start 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 -- start start 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 0x7f3840070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384009d9f0 0x7f3840071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38401316c0 0x7f38400716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38401198a0 con 0x7f384009d9f0 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3840128360 con 0x7f384006acb0 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f38472df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3840072000 con 0x7f38401316c0 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f384507b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 0x7f3840070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f384587c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38401316c0 0x7f38400716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f384507b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 0x7f3840070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40114/0 (socket says 172.21.15.32:40114) 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f384507b700 1 -- 172.21.15.32:0/1546656658 learned_addr learned my addr 172.21.15.32:0/1546656658 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.094+0000 7f384487a700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384009d9f0 0x7f3840071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384507b700 1 -- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38401316c0 msgr2=0x7f38400716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384507b700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38401316c0 0x7f38400716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384507b700 1 -- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384009d9f0 msgr2=0x7f3840071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384507b700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384009d9f0 0x7f3840071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384507b700 1 -- 172.21.15.32:0/1546656658 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380094a0 con 0x7f384006acb0 2023-09-30T22:29:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384587c700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38401316c0 0x7f38400716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f384507b700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 0x7f3840070c30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f383000efc0 tx=0x7f383000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000cbb0 con 0x7f384006acb0 2023-09-30T22:29:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383000f040 con 0x7f384006acb0 2023-09-30T22:29:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830014480 con 0x7f384006acb0 2023-09-30T22:29:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3830009b30 con 0x7f384006acb0 2023-09-30T22:29:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.095+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38340094a0 con 0x7f384006acb0 2023-09-30T22:29:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.096+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38300145e0 con 0x7f384006acb0 2023-09-30T22:29:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.096+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f383003fcb0 con 0x7f384006acb0 2023-09-30T22:29:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.096+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3840131f50 con 0x7f384006acb0 2023-09-30T22:29:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.101+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3830012930 con 0x7f384006acb0 2023-09-30T22:29:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.266+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f58c81a91c0 con 0x7f58c809e410 2023-09-30T22:29:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.267+0000 7f58c57fa700 1 -- 172.21.15.32:0/2367498361 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58c81a91c0 con 0x7f58c809e410 2023-09-30T22:29:44.267 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c809e410 msgr2=0x7f58c81a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c809e410 0x7f58c81a69c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f58c000efc0 tx=0x7f58c000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 shutdown_connections 2023-09-30T22:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58c809e410 0x7f58c81a69c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58c809d6f0 0x7f58c81a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 --2- 172.21.15.32:0/2367498361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f58c809e000 0x7f58c81a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 >> 172.21.15.32:0/2367498361 conn(0x7f58c8066f50 msgr2=0x7f58c81bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 shutdown_connections 2023-09-30T22:29:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.269+0000 7f58cee49700 1 -- 172.21.15.32:0/2367498361 wait complete. 2023-09-30T22:29:44.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.306+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f384012b900 con 0x7f384006acb0 2023-09-30T22:29:44.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.306+0000 7f383dffb700 1 -- 172.21.15.32:0/1546656658 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3830012340 con 0x7f384006acb0 2023-09-30T22:29:44.306 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 msgr2=0x7f3840070c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 0x7f3840070c30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f383000efc0 tx=0x7f383000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 shutdown_connections 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38401316c0 0x7f38400716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f384006acb0 0x7f3840070c30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 --2- 172.21.15.32:0/1546656658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f384009d9f0 0x7f3840071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.308+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 >> 172.21.15.32:0/1546656658 conn(0x7f3840066f50 msgr2=0x7f38400a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.309+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 shutdown_connections 2023-09-30T22:29:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.309+0000 7f38472df700 1 -- 172.21.15.32:0/1546656658 wait complete. 2023-09-30T22:29:44.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:44.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.955+0000 7fcd63569700 1 Processor -- start 2023-09-30T22:29:44.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.955+0000 7fcd63569700 1 -- start start 2023-09-30T22:29:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd63569700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 0x7fcd5c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd63569700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 0x7fcd5c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd63569700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd5c09f650 0x7fcd5c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd63569700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5c070940 con 0x7fcd5c09f650 2023-09-30T22:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd63569700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c06fd60 con 0x7fcd5c09f240 2023-09-30T22:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd63569700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c06fa80 con 0x7fcd5c09e930 2023-09-30T22:29:44.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd5c09f650 0x7fcd5c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61305700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 0x7fcd5c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd60b04700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 0x7fcd5c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61305700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 0x7fcd5c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48900/0 (socket says 172.21.15.32:48900) 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcd5c09f650 0x7fcd5c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46590/0 (socket says 172.21.15.32:46590) 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd60b04700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 0x7fcd5c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40120/0 (socket says 172.21.15.32:40120) 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61305700 1 -- 172.21.15.32:0/394544614 learned_addr learned my addr 172.21.15.32:0/394544614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 -- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 msgr2=0x7fcd5c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 0x7fcd5c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 -- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 msgr2=0x7fcd5c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 0x7fcd5c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 -- 172.21.15.32:0/394544614 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c09fb60 con 0x7fcd5c09f650 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61305700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 0x7fcd5c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd60b04700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 0x7fcd5c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.956+0000 7fcd61b06700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f650 0x7fcd5c131d30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009d90 tx=0x7fcd5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ae4946bfe2947e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd4ffff700 1 -- 172.21.15.32:0/394544614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400e070 con 0x7fcd5c09f650 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd4ffff700 1 -- 172.21.15.32:0/394544614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5400bb20 con 0x7fcd5c09f650 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd4ffff700 1 -- 172.21.15.32:0/394544614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54012430 con 0x7fcd5c09f650 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 -- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f650 msgr2=0x7fcd5c131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f650 0x7fcd5c131d30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009d90 tx=0x7fcd5400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 -- 172.21.15.32:0/394544614 shutdown_connections 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f650 0x7fcd5c131d30 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09e930 0x7fcd5c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 --2- 172.21.15.32:0/394544614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09f240 0x7fcd5c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 -- 172.21.15.32:0/394544614 >> 172.21.15.32:0/394544614 conn(0x7fcd5c0a4210 msgr2=0x7fcd5c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 -- 172.21.15.32:0/394544614 shutdown_connections 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.957+0000 7fcd63569700 1 -- 172.21.15.32:0/394544614 wait complete. 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.958+0000 7fcd63569700 1 Processor -- start 2023-09-30T22:29:44.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.958+0000 7fcd63569700 1 -- start start 2023-09-30T22:29:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd63569700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 0x7fcd5c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd63569700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f240 0x7fcd5c1a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd63569700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09f650 0x7fcd5c1a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd63569700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1a7190 con 0x7fcd5c09f240 2023-09-30T22:29:44.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd63569700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1a7850 con 0x7fcd5c09e930 2023-09-30T22:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd63569700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1a8aa0 con 0x7fcd5c09f650 2023-09-30T22:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 0x7fcd5c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 0x7fcd5c1a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40122/0 (socket says 172.21.15.32:40122) 2023-09-30T22:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 -- 172.21.15.32:0/1343748803 learned_addr learned my addr 172.21.15.32:0/1343748803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:44.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd60b04700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f240 0x7fcd5c1a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61b06700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09f650 0x7fcd5c1a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 -- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09f650 msgr2=0x7fcd5c1a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09f650 0x7fcd5c1a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 -- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f240 msgr2=0x7fcd5c1a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f240 0x7fcd5c1a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.959+0000 7fcd61305700 1 -- 172.21.15.32:0/1343748803 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58009550 con 0x7fcd5c09e930 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd61305700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 0x7fcd5c1a5ce0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800ec10 tx=0x7fcd5800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004030 con 0x7fcd5c09e930 2023-09-30T22:29:44.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58004190 con 0x7fcd5c09e930 2023-09-30T22:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500094a0 con 0x7fcd5c09e930 2023-09-30T22:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004520 con 0x7fcd5c09e930 2023-09-30T22:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54009a70 con 0x7fcd5c09e930 2023-09-30T22:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.960+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd5800c850 con 0x7fcd5c09e930 2023-09-30T22:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.961+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcd5803f4f0 con 0x7fcd5c09e930 2023-09-30T22:29:44.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.961+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c1a2b40 con 0x7fcd5c09e930 2023-09-30T22:29:44.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:44.966+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd5800ccf0 con 0x7fcd5c09e930 2023-09-30T22:29:45.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.176+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcd5c0a3510 con 0x7fcd5c09e930 2023-09-30T22:29:45.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.177+0000 7fcd4dffb700 1 -- 172.21.15.32:0/1343748803 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd58019300 con 0x7fcd5c09e930 2023-09-30T22:29:45.177 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:45.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 msgr2=0x7fcd5c1a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 0x7fcd5c1a5ce0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800ec10 tx=0x7fcd5800ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 shutdown_connections 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcd5c09f650 0x7fcd5c1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcd5c09e930 0x7fcd5c1a5ce0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 --2- 172.21.15.32:0/1343748803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd5c09f240 0x7fcd5c1a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 >> 172.21.15.32:0/1343748803 conn(0x7fcd5c0a4210 msgr2=0x7fcd5c09dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 shutdown_connections 2023-09-30T22:29:45.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.179+0000 7fcd63569700 1 -- 172.21.15.32:0/1343748803 wait complete. 2023-09-30T22:29:45.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.215+0000 7fa894ce2700 1 Processor -- start 2023-09-30T22:29:45.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.216+0000 7fa894ce2700 1 -- start start 2023-09-30T22:29:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.216+0000 7fa894ce2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009d630 0x7fa8900a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa894ce2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa890138d80 0x7fa89013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa894ce2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8900a2a70 con 0x7fa8900a0a30 2023-09-30T22:29:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa894ce2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8900a01d0 con 0x7fa890138d80 2023-09-30T22:29:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa894ce2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89009e2b0 con 0x7fa89009d630 2023-09-30T22:29:45.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa8900a0a30 0x7fa8900a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38110/0 (socket says 172.21.15.32:38110) 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88e59c700 1 -- 172.21.15.32:0/647522384 learned_addr learned my addr 172.21.15.32:0/647522384 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa890138d80 0x7fa89013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88dd9b700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009d630 0x7fa8900a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88d59a700 1 -- 172.21.15.32:0/647522384 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825100721 0 0) 0x7fa8900a2a70 con 0x7fa8900a0a30 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88d59a700 1 -- 172.21.15.32:0/647522384 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c0035b0 con 0x7fa8900a0a30 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88ed9d700 1 -- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009d630 msgr2=0x7fa8900a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009d630 0x7fa8900a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88ed9d700 1 -- 172.21.15.32:0/647522384 >> v1:172.21.15.32:6789/0 conn(0x7fa8900a0a30 legacy=0x7fa8900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88ed9d700 1 -- 172.21.15.32:0/647522384 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89009dc50 con 0x7fa890138d80 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88dd9b700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009d630 0x7fa8900a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa890138d80 0x7fa89013b160 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d90 tx=0x7fa88800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3732cb1af2c74a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88d59a700 1 -- 172.21.15.32:0/647522384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800e070 con 0x7fa890138d80 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88d59a700 1 -- 172.21.15.32:0/647522384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88800bb20 con 0x7fa890138d80 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.217+0000 7fa88d59a700 1 -- 172.21.15.32:0/647522384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888012430 con 0x7fa890138d80 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 -- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa890138d80 msgr2=0x7fa89013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa890138d80 0x7fa89013b160 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d90 tx=0x7fa88800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 -- 172.21.15.32:0/647522384 shutdown_connections 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa890138d80 0x7fa89013b160 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 --2- 172.21.15.32:0/647522384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89009d630 0x7fa8900a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 -- 172.21.15.32:0/647522384 >> 172.21.15.32:0/647522384 conn(0x7fa8900a3050 msgr2=0x7fa8900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 -- 172.21.15.32:0/647522384 shutdown_connections 2023-09-30T22:29:45.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.218+0000 7fa894ce2700 1 -- 172.21.15.32:0/647522384 wait complete. 2023-09-30T22:29:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.220+0000 7fa894ce2700 1 Processor -- start 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.220+0000 7fa894ce2700 1 -- start start 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.220+0000 7fa894ce2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009d630 0x7fa890133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.220+0000 7fa894ce2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8900a0a30 0x7fa890133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 0x7fa890134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901356d0 con 0x7fa89009d630 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa890137c20 con 0x7fa890138d80 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900a4990 con 0x7fa8900a0a30 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 0x7fa890134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8900a0a30 0x7fa890133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009d630 0x7fa890133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 0x7fa890134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40132/0 (socket says 172.21.15.32:40132) 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 -- 172.21.15.32:0/3941084990 learned_addr learned my addr 172.21.15.32:0/3941084990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 -- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8900a0a30 msgr2=0x7fa890133c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8900a0a30 0x7fa890133c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 -- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009d630 msgr2=0x7fa890133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009d630 0x7fa890133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 -- 172.21.15.32:0/3941084990 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780094a0 con 0x7fa890138d80 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88e59c700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009d630 0x7fa890133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa88ed9d700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 0x7fa890134190 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa888013000 tx=0x7fa888004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888012a90 con 0x7fa890138d80 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888020070 con 0x7fa890138d80 2023-09-30T22:29:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa888009a70 con 0x7fa890138d80 2023-09-30T22:29:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888012e70 con 0x7fa890138d80 2023-09-30T22:29:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8900a0410 con 0x7fa890138d80 2023-09-30T22:29:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa888004040 con 0x7fa890138d80 2023-09-30T22:29:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa888022070 con 0x7fa890138d80 2023-09-30T22:29:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.221+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89013b620 con 0x7fa890138d80 2023-09-30T22:29:45.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.226+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa888025740 con 0x7fa890138d80 2023-09-30T22:29:45.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:45.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.423+0000 7f619827f700 1 Processor -- start 2023-09-30T22:29:45.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.423+0000 7f619827f700 1 -- start start 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619827f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 0x7f61900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619827f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619827f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6190133930 0x7f6190133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619827f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61900a2a70 con 0x7f6190133930 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619827f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900a01d0 con 0x7f619009d630 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619827f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f619009e2b0 con 0x7f61900a0a30 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 0x7f61900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619581a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 0x7f61900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48922/0 (socket says 172.21.15.32:48922) 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 -- 172.21.15.32:0/3681628443 learned_addr learned my addr 172.21.15.32:0/3681628443 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619681c700 1 --2- 172.21.15.32:0/3681628443 >> v2:172.21.15.32:3300/0 conn(0x7f6190133930 0x7f6190133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 -- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 msgr2=0x7f61900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 --2- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61900a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 -- 172.21.15.32:0/3681628443 >> v2:172.21.15.32:3300/0 conn(0x7f6190133930 msgr2=0x7f6190133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 --2- 172.21.15.32:0/3681628443 >> v2:172.21.15.32:3300/0 conn(0x7f6190133930 0x7f6190133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619601b700 1 -- 172.21.15.32:0/3681628443 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619009dc50 con 0x7f61900a0a30 2023-09-30T22:29:45.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619581a700 1 --2- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.424+0000 7f619681c700 1 --2- 172.21.15.32:0/3681628443 >> v2:172.21.15.32:3300/0 conn(0x7f6190133930 0x7f6190133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.425+0000 7f619601b700 1 --2- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 0x7f61900a0e00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f618c009e50 tx=0x7f618c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f84606cd9c9ef9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.425+0000 7f6195019700 1 -- 172.21.15.32:0/3681628443 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00e070 con 0x7f61900a0a30 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.425+0000 7f6195019700 1 -- 172.21.15.32:0/3681628443 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c00b910 con 0x7f61900a0a30 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.425+0000 7f6195019700 1 -- 172.21.15.32:0/3681628443 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00bcc0 con 0x7f61900a0a30 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.425+0000 7f619827f700 1 -- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 msgr2=0x7f61900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.425+0000 7f619827f700 1 --2- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 0x7f61900a0e00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f618c009e50 tx=0x7f618c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 -- 172.21.15.32:0/3681628443 shutdown_connections 2023-09-30T22:29:45.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 --2- 172.21.15.32:0/3681628443 >> v2:172.21.15.32:3300/0 conn(0x7f6190133930 0x7f6190133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 --2- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f61900a0a30 0x7f61900a0e00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 --2- 172.21.15.32:0/3681628443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 -- 172.21.15.32:0/3681628443 >> 172.21.15.32:0/3681628443 conn(0x7f61900a3050 msgr2=0x7f61900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 -- 172.21.15.32:0/3681628443 shutdown_connections 2023-09-30T22:29:45.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.426+0000 7f619827f700 1 -- 172.21.15.32:0/3681628443 wait complete. 2023-09-30T22:29:45.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 Processor -- start 2023-09-30T22:29:45.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 -- start start 2023-09-30T22:29:45.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61901c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900a0a30 0x7f61901c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6190133930 0x7f61901c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6190122430 con 0x7f61900a0a30 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6190120720 con 0x7f619009d630 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619827f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61900a41c0 con 0x7f6190133930 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619601b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61901c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619601b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61901c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40148/0 (socket says 172.21.15.32:40148) 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619601b700 1 -- 172.21.15.32:0/1735488177 learned_addr learned my addr 172.21.15.32:0/1735488177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.427+0000 7f619681c700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6190133930 0x7f61901c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619581a700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900a0a30 0x7f61901c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619601b700 1 -- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6190133930 msgr2=0x7f61901c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619601b700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6190133930 0x7f61901c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619601b700 1 -- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900a0a30 msgr2=0x7f61901c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619601b700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900a0a30 0x7f61901c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619601b700 1 -- 172.21.15.32:0/1735488177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800094a0 con 0x7f619009d630 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619601b700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61901c81a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b370 tx=0x7f618c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c00ce80 con 0x7f619009d630 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c004250 con 0x7f619009d630 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c004600 con 0x7f619009d630 2023-09-30T22:29:45.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61880094a0 con 0x7f619009d630 2023-09-30T22:29:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.428+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c009b30 con 0x7f619009d630 2023-09-30T22:29:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.429+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f618c004760 con 0x7f619009d630 2023-09-30T22:29:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.430+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61900a0410 con 0x7f619009d630 2023-09-30T22:29:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.430+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f618c040c40 con 0x7f619009d630 2023-09-30T22:29:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.434+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f618c0117a0 con 0x7f619009d630 2023-09-30T22:29:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.440+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8900747f0 con 0x7fa890138d80 2023-09-30T22:29:45.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.440+0000 7fa886ffd700 1 -- 172.21.15.32:0/3941084990 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa888036090 con 0x7fa890138d80 2023-09-30T22:29:45.441 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 msgr2=0x7fa890134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 0x7fa890134190 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa888013000 tx=0x7fa888004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 shutdown_connections 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa890138d80 0x7fa890134190 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89009d630 0x7fa890133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 --2- 172.21.15.32:0/3941084990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa8900a0a30 0x7fa890133c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 >> 172.21.15.32:0/3941084990 conn(0x7fa8900a3050 msgr2=0x7fa89013add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 shutdown_connections 2023-09-30T22:29:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.443+0000 7fa894ce2700 1 -- 172.21.15.32:0/3941084990 wait complete. 2023-09-30T22:29:45.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.467+0000 7f026c506700 1 Processor -- start 2023-09-30T22:29:45.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.467+0000 7f026c506700 1 -- start start 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026c506700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 0x7f026409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026c506700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409e410 0x7f0264134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026c506700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026409b120 con 0x7f026409e000 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026c506700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026406b460 con 0x7f026409d6f0 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026c506700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f026409a7e0 con 0x7f026409e410 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026a2a2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 0x7f026409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026a2a2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 0x7f026409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40162/0 (socket says 172.21.15.32:40162) 2023-09-30T22:29:45.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026aaa3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409e410 0x7f0264134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f026a2a2700 1 -- 172.21.15.32:0/1393635574 learned_addr learned my addr 172.21.15.32:0/1393635574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.468+0000 7f0269aa1700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f026409e000 0x7f02641339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38124/0 (socket says 172.21.15.32:38124) 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f026a2a2700 1 -- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409e410 msgr2=0x7f0264134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f026a2a2700 1 --2- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409e410 0x7f0264134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f026a2a2700 1 -- 172.21.15.32:0/1393635574 >> v1:172.21.15.32:6789/0 conn(0x7f026409e000 legacy=0x7f02641339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f026a2a2700 1 -- 172.21.15.32:0/1393635574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026409a500 con 0x7f026409d6f0 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f02692a0700 1 -- 172.21.15.32:0/1393635574 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468884320 0 0) 0x7f026409b120 con 0x7f026409e000 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f026aaa3700 1 --2- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409e410 0x7f0264134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f026a2a2700 1 --2- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 0x7f026409dac0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0254009e80 tx=0x7f025400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240ea39f69f6513e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f02692a0700 1 -- 172.21.15.32:0/1393635574 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025400e070 con 0x7f026409d6f0 2023-09-30T22:29:45.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f02692a0700 1 -- 172.21.15.32:0/1393635574 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025400b910 con 0x7f026409d6f0 2023-09-30T22:29:45.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.469+0000 7f02692a0700 1 -- 172.21.15.32:0/1393635574 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025400bcc0 con 0x7f026409d6f0 2023-09-30T22:29:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 -- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 msgr2=0x7f026409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 --2- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 0x7f026409dac0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0254009e80 tx=0x7f025400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 -- 172.21.15.32:0/1393635574 shutdown_connections 2023-09-30T22:29:45.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 --2- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409e410 0x7f0264134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 --2- 172.21.15.32:0/1393635574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409d6f0 0x7f026409dac0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 -- 172.21.15.32:0/1393635574 >> 172.21.15.32:0/1393635574 conn(0x7f0264066f50 msgr2=0x7f026409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 -- 172.21.15.32:0/1393635574 shutdown_connections 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.470+0000 7f026c506700 1 -- 172.21.15.32:0/1393635574 wait complete. 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.471+0000 7f026c506700 1 Processor -- start 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.471+0000 7f026c506700 1 -- start start 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026c506700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 0x7f02641223b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026c506700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026409e000 0x7f02641228f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026c506700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409e410 0x7f0264122e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026c506700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02641336f0 con 0x7f026409e000 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026c506700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0264123810 con 0x7f026409e410 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026c506700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0264124dc0 con 0x7f026409d6f0 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026a2a2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 0x7f02641223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026aaa3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409e410 0x7f0264122e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026a2a2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 0x7f02641223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48944/0 (socket says 172.21.15.32:48944) 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026a2a2700 1 -- 172.21.15.32:0/2282904939 learned_addr learned my addr 172.21.15.32:0/2282904939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f0269aa1700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026409e000 0x7f02641228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026aaa3700 1 -- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 msgr2=0x7f02641223b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026aaa3700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 0x7f02641223b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026aaa3700 1 -- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026409e000 msgr2=0x7f02641228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026aaa3700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026409e000 0x7f02641228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026aaa3700 1 -- 172.21.15.32:0/2282904939 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0094a0 con 0x7f026409e410 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f026a2a2700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 0x7f02641223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.472+0000 7f0269aa1700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026409e000 0x7f02641228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.473+0000 7f026aaa3700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409e410 0x7f0264122e30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f025c00e970 tx=0x7f025c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.473+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c004020 con 0x7f026409e410 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.473+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c0041a0 con 0x7f026409e410 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.473+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c004550 con 0x7f026409e410 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.473+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254009b30 con 0x7f026409e410 2023-09-30T22:29:45.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.473+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02640a1aa0 con 0x7f026409e410 2023-09-30T22:29:45.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.474+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026411f030 con 0x7f026409e410 2023-09-30T22:29:45.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.474+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f025c0036a0 con 0x7f026409e410 2023-09-30T22:29:45.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.474+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f025c03f360 con 0x7f026409e410 2023-09-30T22:29:45.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.483+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f025c013680 con 0x7f026409e410 2023-09-30T22:29:45.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.648+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f61900747f0 con 0x7f619009d630 2023-09-30T22:29:45.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.648+0000 7f6186ffd700 1 -- 172.21.15.32:0/1735488177 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f618c034090 con 0x7f619009d630 2023-09-30T22:29:45.650 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 msgr2=0x7f61901c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61901c81a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b370 tx=0x7f618c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 shutdown_connections 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6190133930 0x7f61901c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f619009d630 0x7f61901c81a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 --2- 172.21.15.32:0/1735488177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61900a0a30 0x7f61901c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 >> 172.21.15.32:0/1735488177 conn(0x7f61900a3050 msgr2=0x7f61901bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 shutdown_connections 2023-09-30T22:29:45.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.651+0000 7f619827f700 1 -- 172.21.15.32:0/1735488177 wait complete. 2023-09-30T22:29:45.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.690+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f026409c710 con 0x7f026409e410 2023-09-30T22:29:45.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.691+0000 7f0262ffd700 1 -- 172.21.15.32:0/2282904939 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f025c017030 con 0x7f026409e410 2023-09-30T22:29:45.692 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409e410 msgr2=0x7f0264122e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409e410 0x7f0264122e30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f025c00e970 tx=0x7f025c00ec80 comp rx=0 tx=0).stop 2023-09-30T22:29:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 shutdown_connections 2023-09-30T22:29:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f026409e410 0x7f0264122e30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f026409d6f0 0x7f02641223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 --2- 172.21.15.32:0/2282904939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f026409e000 0x7f02641228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 >> 172.21.15.32:0/2282904939 conn(0x7f0264066f50 msgr2=0x7f0264133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 shutdown_connections 2023-09-30T22:29:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:45.693+0000 7f026c506700 1 -- 172.21.15.32:0/2282904939 wait complete. 2023-09-30T22:29:45.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:46.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:46.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 Processor -- start 2023-09-30T22:29:46.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 -- start start 2023-09-30T22:29:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff090070940 con 0x7ff09009f650 2023-09-30T22:29:46.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09006fd60 con 0x7ff09009e930 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.339+0000 7ff097383700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff09006fa80 con 0x7ff09009f240 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09491e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff095920700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40178/0 (socket says 172.21.15.32:40178) 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09491e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48948/0 (socket says 172.21.15.32:48948) 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff095920700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46638/0 (socket says 172.21.15.32:46638) 2023-09-30T22:29:46.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 -- 172.21.15.32:0/728181717 learned_addr learned my addr 172.21.15.32:0/728181717 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 -- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 msgr2=0x7ff0901317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 --2- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 -- 172.21.15.32:0/728181717 >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 msgr2=0x7ff090131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 --2- 172.21.15.32:0/728181717 >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 -- 172.21.15.32:0/728181717 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09009fb60 con 0x7ff09009e930 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09491e700 1 --2- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff095920700 1 --2- 172.21.15.32:0/728181717 >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff09511f700 1 --2- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09009ed00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0098f0 tx=0x7ff08c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba70960d815714d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff087fff700 1 -- 172.21.15.32:0/728181717 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011830 con 0x7ff09009e930 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff087fff700 1 -- 172.21.15.32:0/728181717 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c011990 con 0x7ff09009e930 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.340+0000 7ff087fff700 1 -- 172.21.15.32:0/728181717 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011d20 con 0x7ff09009e930 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 -- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 msgr2=0x7ff09009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 --2- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09009ed00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0098f0 tx=0x7ff08c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 -- 172.21.15.32:0/728181717 shutdown_connections 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 --2- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09009ed00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 --2- 172.21.15.32:0/728181717 >> v2:172.21.15.32:3300/0 conn(0x7ff09009f650 0x7ff090131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 --2- 172.21.15.32:0/728181717 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f240 0x7ff0901317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 -- 172.21.15.32:0/728181717 >> 172.21.15.32:0/728181717 conn(0x7ff0900a4210 msgr2=0x7ff0900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 -- 172.21.15.32:0/728181717 shutdown_connections 2023-09-30T22:29:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.341+0000 7ff097383700 1 -- 172.21.15.32:0/728181717 wait complete. 2023-09-30T22:29:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.342+0000 7ff097383700 1 Processor -- start 2023-09-30T22:29:46.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.342+0000 7ff097383700 1 -- start start 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09009f240 0x7ff09006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f650 0x7ff09006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09006cd80 con 0x7ff09009f240 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a89f0 con 0x7ff09009e930 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901b72e0 con 0x7ff09009f650 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09491e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09009f240 0x7ff09006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09491e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09009f240 0x7ff09006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46646/0 (socket says 172.21.15.32:46646) 2023-09-30T22:29:46.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09491e700 1 -- 172.21.15.32:0/2961609439 learned_addr learned my addr 172.21.15.32:0/2961609439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff095920700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f650 0x7ff09006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 -- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f650 msgr2=0x7ff09006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f650 0x7ff09006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 -- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09009f240 msgr2=0x7ff09006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09009f240 0x7ff09006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 -- 172.21.15.32:0/2961609439 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800094a0 con 0x7ff09009e930 2023-09-30T22:29:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff095920700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f650 0x7ff09006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff09511f700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09006b8b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff08c016000 tx=0x7ff08c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004450 con 0x7ff09009e930 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c01f070 con 0x7ff09009e930 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0045f0 con 0x7ff09009e930 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.343+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff088009a70 con 0x7ff09009e930 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.344+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c009550 con 0x7ff09009e930 2023-09-30T22:29:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.344+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff08c0038e0 con 0x7ff09009e930 2023-09-30T22:29:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.345+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff08c021070 con 0x7ff09009e930 2023-09-30T22:29:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.345+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0901a92e0 con 0x7ff09009e930 2023-09-30T22:29:46.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.350+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff08c015080 con 0x7ff09009e930 2023-09-30T22:29:46.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:46.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.557+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff0900a35b0 con 0x7ff09009e930 2023-09-30T22:29:46.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.557+0000 7ff085ffb700 1 -- 172.21.15.32:0/2961609439 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff08c02b020 con 0x7ff09009e930 2023-09-30T22:29:46.559 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 msgr2=0x7ff09006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09006b8b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff08c016000 tx=0x7ff08c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:46.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 shutdown_connections 2023-09-30T22:29:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff09009f650 0x7ff09006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09009e930 0x7ff09006b8b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 --2- 172.21.15.32:0/2961609439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09009f240 0x7ff09006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 >> 172.21.15.32:0/2961609439 conn(0x7ff0900a4210 msgr2=0x7ff0901b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 shutdown_connections 2023-09-30T22:29:46.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.560+0000 7ff097383700 1 -- 172.21.15.32:0/2961609439 wait complete. 2023-09-30T22:29:46.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 Processor -- start 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 -- start start 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cfc09d6f0 0x7f4cfc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 0x7f4cfc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc09f520 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc09f130 con 0x7f4cfc09e410 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.605+0000 7f4d02889700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc09e930 con 0x7f4cfc09e000 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cfc09d6f0 0x7f4cfc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 0x7f4cfc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48968/0 (socket says 172.21.15.32:48968) 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cfc09d6f0 0x7f4cfc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46648/0 (socket says 172.21.15.32:46648) 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfb7fe700 1 -- 172.21.15.32:0/2182974359 learned_addr learned my addr 172.21.15.32:0/2182974359 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 0x7f4cfc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40192/0 (socket says 172.21.15.32:40192) 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 -- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 msgr2=0x7f4cfc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 -- 172.21.15.32:0/2182974359 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc09d6f0 msgr2=0x7f4cfc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 --2- 172.21.15.32:0/2182974359 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc09d6f0 0x7f4cfc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 -- 172.21.15.32:0/2182974359 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc09b2d0 con 0x7f4cfc09e410 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfb7fe700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4d00e26700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 0x7f4cfc130ba0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009d90 tx=0x7f4cf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0fe05d72681788 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfaffd700 1 -- 172.21.15.32:0/2182974359 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400e070 con 0x7f4cfc09e410 2023-09-30T22:29:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfaffd700 1 -- 172.21.15.32:0/2182974359 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf400bb20 con 0x7f4cfc09e410 2023-09-30T22:29:46.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.606+0000 7f4cfaffd700 1 -- 172.21.15.32:0/2182974359 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4012430 con 0x7f4cfc09e410 2023-09-30T22:29:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 -- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 msgr2=0x7f4cfc130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 0x7f4cfc130ba0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009d90 tx=0x7f4cf400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 -- 172.21.15.32:0/2182974359 shutdown_connections 2023-09-30T22:29:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09e410 0x7f4cfc130ba0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 --2- 172.21.15.32:0/2182974359 >> v2:172.21.15.32:3300/0 conn(0x7f4cfc09d6f0 0x7f4cfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 --2- 172.21.15.32:0/2182974359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 -- 172.21.15.32:0/2182974359 >> 172.21.15.32:0/2182974359 conn(0x7f4cfc066f50 msgr2=0x7f4cfc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 -- 172.21.15.32:0/2182974359 shutdown_connections 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.607+0000 7f4d02889700 1 -- 172.21.15.32:0/2182974359 wait complete. 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.608+0000 7f4d02889700 1 Processor -- start 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 -- start start 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 0x7f4cfc11de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc11e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc09e410 0x7f4cfc11e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc11f2d0 con 0x7f4cfc09e410 2023-09-30T22:29:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc11fa50 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4d02889700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc120070 con 0x7f4cfc09e000 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4cfbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 0x7f4cfc11de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4cfbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 0x7f4cfc11de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40206/0 (socket says 172.21.15.32:40206) 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.609+0000 7f4cfbfff700 1 -- 172.21.15.32:0/1736084348 learned_addr learned my addr 172.21.15.32:0/1736084348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4d00e26700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc09e410 0x7f4cfc11e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfb7fe700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc11e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfbfff700 1 -- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 msgr2=0x7f4cfc11e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfbfff700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc11e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfbfff700 1 -- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc09e410 msgr2=0x7f4cfc11e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfbfff700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc09e410 0x7f4cfc11e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfbfff700 1 -- 172.21.15.32:0/1736084348 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec009550 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cfbfff700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 0x7f4cfc11de40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4010e50 tx=0x7f4ce400e640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011450 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce40115b0 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf4009a70 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011940 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.610+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc0a16b0 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.611+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ce4011aa0 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.611+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ce40421f0 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.611+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc09c760 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.616+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ce4017da0 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:46.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.807+0000 7efc9c9dc700 1 Processor -- start 2023-09-30T22:29:46.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.807+0000 7efc9c9dc700 1 -- start start 2023-09-30T22:29:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9c9dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 0x7efc980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9c9dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc98138d80 0x7efc9813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9c9dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc980a2a70 con 0x7efc9809d630 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9c9dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc980a01d0 con 0x7efc980a0a30 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9c9dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc9809e2b0 con 0x7efc98138d80 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 0x7efc980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 0x7efc980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40216/0 (socket says 172.21.15.32:40216) 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9659c700 1 -- 172.21.15.32:0/2643720383 learned_addr learned my addr 172.21.15.32:0/2643720383 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc96d9d700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc98138d80 0x7efc9813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9559a700 1 -- 172.21.15.32:0/2643720383 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073222166 0 0) 0x7efc980a2a70 con 0x7efc9809d630 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc96d9d700 1 -- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 msgr2=0x7efc980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc96d9d700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 0x7efc980a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc96d9d700 1 -- 172.21.15.32:0/2643720383 >> v1:172.21.15.32:6789/0 conn(0x7efc9809d630 legacy=0x7efc980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc96d9d700 1 -- 172.21.15.32:0/2643720383 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9809dc50 con 0x7efc98138d80 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc9659c700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 0x7efc980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.808+0000 7efc96d9d700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc98138d80 0x7efc9813b160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efc90009dc0 tx=0x7efc9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=264d188805d759e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9559a700 1 -- 172.21.15.32:0/2643720383 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000e070 con 0x7efc98138d80 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9559a700 1 -- 172.21.15.32:0/2643720383 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9000bb20 con 0x7efc98138d80 2023-09-30T22:29:46.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9559a700 1 -- 172.21.15.32:0/2643720383 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90012430 con 0x7efc98138d80 2023-09-30T22:29:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9c9dc700 1 -- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc98138d80 msgr2=0x7efc9813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc98138d80 0x7efc9813b160 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efc90009dc0 tx=0x7efc9000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9c9dc700 1 -- 172.21.15.32:0/2643720383 shutdown_connections 2023-09-30T22:29:46.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc98138d80 0x7efc9813b160 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/2643720383 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc980a0a30 0x7efc980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.809+0000 7efc9c9dc700 1 -- 172.21.15.32:0/2643720383 >> 172.21.15.32:0/2643720383 conn(0x7efc980a3050 msgr2=0x7efc980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.810+0000 7efc9c9dc700 1 -- 172.21.15.32:0/2643720383 shutdown_connections 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.810+0000 7efc9c9dc700 1 -- 172.21.15.32:0/2643720383 wait complete. 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 Processor -- start 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 -- start start 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc9809d630 0x7efc9812c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc980a0a30 0x7efc9812ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc98138d80 0x7efc9812cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc980a49e0 con 0x7efc980a0a30 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9812fb00 con 0x7efc98138d80 2023-09-30T22:29:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc9c9dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc9812e090 con 0x7efc9809d630 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc96d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc98138d80 0x7efc9812cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc95d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc980a0a30 0x7efc9812ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.811+0000 7efc96d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc98138d80 0x7efc9812cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40230/0 (socket says 172.21.15.32:40230) 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc96d9d700 1 -- 172.21.15.32:0/1076006121 learned_addr learned my addr 172.21.15.32:0/1076006121 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc9659c700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc9809d630 0x7efc9812c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc95d9b700 1 -- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc9809d630 msgr2=0x7efc9812c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc95d9b700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc9809d630 0x7efc9812c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc95d9b700 1 -- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc98138d80 msgr2=0x7efc9812cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc95d9b700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc98138d80 0x7efc9812cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc95d9b700 1 -- 172.21.15.32:0/1076006121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88009550 con 0x7efc980a0a30 2023-09-30T22:29:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc9659c700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc9809d630 0x7efc9812c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc95d9b700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc980a0a30 0x7efc9812ca00 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7efc9806d980 tx=0x7efc8c00c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0054e0 con 0x7efc980a0a30 2023-09-30T22:29:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c00ce10 con 0x7efc980a0a30 2023-09-30T22:29:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc90009a70 con 0x7efc980a0a30 2023-09-30T22:29:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.812+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0038c0 con 0x7efc980a0a30 2023-09-30T22:29:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.813+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9812ed80 con 0x7efc980a0a30 2023-09-30T22:29:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.814+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc980747f0 con 0x7efc980a0a30 2023-09-30T22:29:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.814+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc8c003a20 con 0x7efc980a0a30 2023-09-30T22:29:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.815+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efc8c03ede0 con 0x7efc980a0a30 2023-09-30T22:29:46.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.823+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc8c003cd0 con 0x7efc980a0a30 2023-09-30T22:29:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.838+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4cfc11ac20 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.839+0000 7f4cf8ff9700 1 -- 172.21.15.32:0/1736084348 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ce4017340 con 0x7f4cfc09d6f0 2023-09-30T22:29:46.840 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 msgr2=0x7f4cfc11de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 0x7f4cfc11de40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4010e50 tx=0x7f4ce400e640 comp rx=0 tx=0).stop 2023-09-30T22:29:46.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 shutdown_connections 2023-09-30T22:29:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cfc09e410 0x7f4cfc11e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cfc09d6f0 0x7f4cfc11de40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 --2- 172.21.15.32:0/1736084348 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cfc09e000 0x7f4cfc11e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 >> 172.21.15.32:0/1736084348 conn(0x7f4cfc066f50 msgr2=0x7f4cfc130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 shutdown_connections 2023-09-30T22:29:46.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.841+0000 7f4d02889700 1 -- 172.21.15.32:0/1736084348 wait complete. 2023-09-30T22:29:46.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 Processor -- start 2023-09-30T22:29:46.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 -- start start 2023-09-30T22:29:46.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:46.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3e80a0a30 0x7fe3e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e8134170 0x7fe3e813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e809edd0 con 0x7fe3e80a1340 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e809e1f0 con 0x7fe3e80a0a30 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.855+0000 7fe3ef658700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e809dc50 con 0x7fe3e8134170 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ed3f4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3e80a0a30 0x7fe3e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3edbf5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e8134170 0x7fe3e813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ecbf3700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe3e80a1340 0x7fe3e8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38144/0 (socket says 172.21.15.32:38144) 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ecbf3700 1 -- 172.21.15.32:0/790808514 learned_addr learned my addr 172.21.15.32:0/790808514 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ed3f4700 1 -- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e8134170 msgr2=0x7fe3e813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ed3f4700 1 --2- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e8134170 0x7fe3e813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ed3f4700 1 -- 172.21.15.32:0/790808514 >> v1:172.21.15.32:6789/0 conn(0x7fe3e80a1340 legacy=0x7fe3e8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ed3f4700 1 -- 172.21.15.32:0/790808514 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80a1fe0 con 0x7fe3e80a0a30 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3edbf5700 1 --2- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e8134170 0x7fe3e813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3ed3f4700 1 --2- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a0a30 0x7fe3e80a0e00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009840 tx=0x7fe3e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88432f6044ac2910 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.856+0000 7fe3dffff700 1 -- 172.21.15.32:0/790808514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011830 con 0x7fe3e80a0a30 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3dffff700 1 -- 172.21.15.32:0/790808514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4011990 con 0x7fe3e80a0a30 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3dffff700 1 -- 172.21.15.32:0/790808514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011d20 con 0x7fe3e80a0a30 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 -- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a0a30 msgr2=0x7fe3e80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 --2- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a0a30 0x7fe3e80a0e00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009840 tx=0x7fe3e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 -- 172.21.15.32:0/790808514 shutdown_connections 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 --2- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e8134170 0x7fe3e813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 --2- 172.21.15.32:0/790808514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a0a30 0x7fe3e80a0e00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 -- 172.21.15.32:0/790808514 >> 172.21.15.32:0/790808514 conn(0x7fe3e80a6550 msgr2=0x7fe3e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 -- 172.21.15.32:0/790808514 shutdown_connections 2023-09-30T22:29:46.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.857+0000 7fe3ef658700 1 -- 172.21.15.32:0/790808514 wait complete. 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.859+0000 7fe3ef658700 1 Processor -- start 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.859+0000 7fe3ef658700 1 -- start start 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ef658700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e80a0a30 0x7fe3e81c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ef658700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 0x7fe3e81c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ef658700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3e8134170 0x7fe3e81c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ef658700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80a0290 con 0x7fe3e80a1340 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ef658700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8124660 con 0x7fe3e80a0a30 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ef658700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8124d40 con 0x7fe3e8134170 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 0x7fe3e81c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3edbf5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3e8134170 0x7fe3e81c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 0x7fe3e81c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46692/0 (socket says 172.21.15.32:46692) 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 -- 172.21.15.32:0/3287577586 learned_addr learned my addr 172.21.15.32:0/3287577586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ed3f4700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e80a0a30 0x7fe3e81c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 -- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3e8134170 msgr2=0x7fe3e81c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3e8134170 0x7fe3e81c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 -- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e80a0a30 msgr2=0x7fe3e81c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e80a0a30 0x7fe3e81c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 -- 172.21.15.32:0/3287577586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0009a70 con 0x7fe3e80a1340 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3edbf5700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3e8134170 0x7fe3e81c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.860+0000 7fe3ecbf3700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 0x7fe3e81c89c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c550 tx=0x7fe3d800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.861+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8005d70 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.861+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d80070d0 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.861+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e40094a0 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.861+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8007440 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.861+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e80a25e0 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.861+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3d80075a0 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.862+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe3d803ea50 con 0x7fe3e80a1340 2023-09-30T22:29:46.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.862+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e80a4f20 con 0x7fe3e80a1340 2023-09-30T22:29:46.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:46.867+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3d8019b40 con 0x7fe3e80a1340 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.036+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efc980a0410 con 0x7efc980a0a30 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.036+0000 7efc86ffd700 1 -- 172.21.15.32:0/1076006121 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc8c0192c0 con 0x7efc980a0a30 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc980a0a30 msgr2=0x7efc9812ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc980a0a30 0x7efc9812ca00 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7efc9806d980 tx=0x7efc8c00c710 comp rx=0 tx=0).stop 2023-09-30T22:29:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 shutdown_connections 2023-09-30T22:29:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc98138d80 0x7efc9812cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efc9809d630 0x7efc9812c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 --2- 172.21.15.32:0/1076006121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc980a0a30 0x7efc9812ca00 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 >> 172.21.15.32:0/1076006121 conn(0x7efc980a3050 msgr2=0x7efc9809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 shutdown_connections 2023-09-30T22:29:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.039+0000 7efc9c9dc700 1 -- 172.21.15.32:0/1076006121 wait complete. 2023-09-30T22:29:47.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.073+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe3e8074ea0 con 0x7fe3e80a1340 2023-09-30T22:29:47.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.073+0000 7fe3ddffb700 1 -- 172.21.15.32:0/3287577586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3d8017020 con 0x7fe3e80a1340 2023-09-30T22:29:47.074 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 msgr2=0x7fe3e81c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 0x7fe3e81c89c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c550 tx=0x7fe3d800c860 comp rx=0 tx=0).stop 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 shutdown_connections 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3e8134170 0x7fe3e81c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3e80a0a30 0x7fe3e81c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 --2- 172.21.15.32:0/3287577586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3e80a1340 0x7fe3e81c89c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 >> 172.21.15.32:0/3287577586 conn(0x7fe3e80a6550 msgr2=0x7fe3e809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 shutdown_connections 2023-09-30T22:29:47.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.076+0000 7fe3ef658700 1 -- 172.21.15.32:0/3287577586 wait complete. 2023-09-30T22:29:47.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:47.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.725+0000 7fc671875700 1 Processor -- start 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.725+0000 7fc671875700 1 -- start start 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc671875700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc66c09e930 0x7fc66c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc671875700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09f650 0x7fc66c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc671875700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c070940 con 0x7fc66c09f240 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc671875700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c06fd60 con 0x7fc66c09e930 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc671875700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c06fa80 con 0x7fc66c09f650 2023-09-30T22:29:47.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66affd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc66c09e930 0x7fc66c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc66c09f240 0x7fc66c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38154/0 (socket says 172.21.15.32:38154) 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66a7fc700 1 -- 172.21.15.32:0/3201706457 learned_addr learned my addr 172.21.15.32:0/3201706457 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66b7fe700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09f650 0x7fc66c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc669ffb700 1 -- 172.21.15.32:0/3201706457 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032092685 0 0) 0x7fc66c070940 con 0x7fc66c09f240 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc669ffb700 1 -- 172.21.15.32:0/3201706457 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6540035b0 con 0x7fc66c09f240 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66affd700 1 -- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09f650 msgr2=0x7fc66c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66affd700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09f650 0x7fc66c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66affd700 1 -- 172.21.15.32:0/3201706457 >> v1:172.21.15.32:6789/0 conn(0x7fc66c09f240 legacy=0x7fc66c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66affd700 1 -- 172.21.15.32:0/3201706457 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c09fb60 con 0x7fc66c09e930 2023-09-30T22:29:47.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.726+0000 7fc66b7fe700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09f650 0x7fc66c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc66affd700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09e930 0x7fc66c09ed00 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009840 tx=0x7fc65c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b8fbe43a6123c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc669ffb700 1 -- 172.21.15.32:0/3201706457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011830 con 0x7fc66c09e930 2023-09-30T22:29:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc669ffb700 1 -- 172.21.15.32:0/3201706457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c011990 con 0x7fc66c09e930 2023-09-30T22:29:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc669ffb700 1 -- 172.21.15.32:0/3201706457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011d20 con 0x7fc66c09e930 2023-09-30T22:29:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 -- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09e930 msgr2=0x7fc66c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09e930 0x7fc66c09ed00 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009840 tx=0x7fc65c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 -- 172.21.15.32:0/3201706457 shutdown_connections 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09f650 0x7fc66c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 --2- 172.21.15.32:0/3201706457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09e930 0x7fc66c09ed00 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 -- 172.21.15.32:0/3201706457 >> 172.21.15.32:0/3201706457 conn(0x7fc66c0a4210 msgr2=0x7fc66c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 -- 172.21.15.32:0/3201706457 shutdown_connections 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.727+0000 7fc671875700 1 -- 172.21.15.32:0/3201706457 wait complete. 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.729+0000 7fc671875700 1 Processor -- start 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.729+0000 7fc671875700 1 -- start start 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc671875700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09e930 0x7fc66c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc671875700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc66c09f240 0x7fc66c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc671875700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 0x7fc66c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc671875700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c06d4c0 con 0x7fc66c09f650 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc671875700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1a91e0 con 0x7fc66c09f240 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc671875700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1a8700 con 0x7fc66c09e930 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 0x7fc66c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc66c09f240 0x7fc66c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 0x7fc66c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46708/0 (socket says 172.21.15.32:46708) 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66affd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09e930 0x7fc66c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc66c09f240 0x7fc66c06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40260/0 (socket says 172.21.15.32:40260) 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 -- 172.21.15.32:0/1607579060 learned_addr learned my addr 172.21.15.32:0/1607579060 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 -- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09e930 msgr2=0x7fc66c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09e930 0x7fc66c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 -- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc66c09f240 msgr2=0x7fc66c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc66c09f240 0x7fc66c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 -- 172.21.15.32:0/1607579060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc664009a70 con 0x7fc66c09f650 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.730+0000 7fc66b7fe700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 0x7fc66c06c330 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc664009f00 tx=0x7fc66400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.731+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664009d70 con 0x7fc66c09f650 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.731+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66400f040 con 0x7fc66c09f650 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.731+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c0094a0 con 0x7fc66c09f650 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.731+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66400cd70 con 0x7fc66c09f650 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.731+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c0a0190 con 0x7fc66c09f650 2023-09-30T22:29:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.731+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc664014480 con 0x7fc66c09f650 2023-09-30T22:29:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.732+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc66403fce0 con 0x7fc66c09f650 2023-09-30T22:29:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.732+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c1a0b30 con 0x7fc66c09f650 2023-09-30T22:29:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.737+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc664012930 con 0x7fc66c09f650 2023-09-30T22:29:47.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.940+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc66c0a3260 con 0x7fc66c09f650 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.940+0000 7fc653fff700 1 -- 172.21.15.32:0/1607579060 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc664012340 con 0x7fc66c09f650 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 msgr2=0x7fc66c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 0x7fc66c06c330 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc664009f00 tx=0x7fc66400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 shutdown_connections 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc66c09f650 0x7fc66c06c330 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc66c09e930 0x7fc66c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 --2- 172.21.15.32:0/1607579060 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc66c09f240 0x7fc66c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 >> 172.21.15.32:0/1607579060 conn(0x7fc66c0a4210 msgr2=0x7fc66c1b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 shutdown_connections 2023-09-30T22:29:47.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.943+0000 7fc671875700 1 -- 172.21.15.32:0/1607579060 wait complete. 2023-09-30T22:29:47.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:47.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.995+0000 7ff6aa301700 1 Processor -- start 2023-09-30T22:29:47.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.995+0000 7ff6aa301700 1 -- start start 2023-09-30T22:29:47.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6aa301700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409e930 0x7ff6a409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6aa301700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6a409f650 0x7ff6a4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6aa301700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a4070940 con 0x7ff6a409f240 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6aa301700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a406fd60 con 0x7ff6a409f650 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6aa301700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a406fa80 con 0x7ff6a409e930 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409e930 0x7ff6a409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a889e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6a409f650 0x7ff6a4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff6a409f240 0x7ff6a41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38162/0 (socket says 172.21.15.32:38162) 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a37fe700 1 -- 172.21.15.32:0/1307021623 learned_addr learned my addr 172.21.15.32:0/1307021623 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1307021623 >> v2:172.21.15.32:3300/0 conn(0x7ff6a409f650 msgr2=0x7ff6a4131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a3fff700 1 --2- 172.21.15.32:0/1307021623 >> v2:172.21.15.32:3300/0 conn(0x7ff6a409f650 0x7ff6a4131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1307021623 >> v1:172.21.15.32:6789/0 conn(0x7ff6a409f240 legacy=0x7ff6a41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1307021623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a409fb60 con 0x7ff6a409e930 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a889e700 1 --2- 172.21.15.32:0/1307021623 >> v2:172.21.15.32:3300/0 conn(0x7ff6a409f650 0x7ff6a4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.996+0000 7ff6a3fff700 1 --2- 172.21.15.32:0/1307021623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409e930 0x7ff6a409ed00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff694009840 tx=0x7ff694009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea85820c1c3201c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6a2ffd700 1 -- 172.21.15.32:0/1307021623 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694011830 con 0x7ff6a409e930 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6a2ffd700 1 -- 172.21.15.32:0/1307021623 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff694011990 con 0x7ff6a409e930 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6a2ffd700 1 -- 172.21.15.32:0/1307021623 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694011d20 con 0x7ff6a409e930 2023-09-30T22:29:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 -- 172.21.15.32:0/1307021623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409e930 msgr2=0x7ff6a409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 --2- 172.21.15.32:0/1307021623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409e930 0x7ff6a409ed00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff694009840 tx=0x7ff694009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 -- 172.21.15.32:0/1307021623 shutdown_connections 2023-09-30T22:29:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 --2- 172.21.15.32:0/1307021623 >> v2:172.21.15.32:3300/0 conn(0x7ff6a409f650 0x7ff6a4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 --2- 172.21.15.32:0/1307021623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409e930 0x7ff6a409ed00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 -- 172.21.15.32:0/1307021623 >> 172.21.15.32:0/1307021623 conn(0x7ff6a40a4210 msgr2=0x7ff6a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 -- 172.21.15.32:0/1307021623 shutdown_connections 2023-09-30T22:29:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.997+0000 7ff6aa301700 1 -- 172.21.15.32:0/1307021623 wait complete. 2023-09-30T22:29:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.999+0000 7ff6aa301700 1 Processor -- start 2023-09-30T22:29:48.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:47.999+0000 7ff6aa301700 1 -- start start 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6aa301700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 0x7ff6a41a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6aa301700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6a409f240 0x7ff6a41a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6aa301700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409f650 0x7ff6a41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6aa301700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a5120 con 0x7ff6a409e930 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6aa301700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a5880 con 0x7ff6a409f240 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6aa301700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a5ea0 con 0x7ff6a409f650 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 0x7ff6a41a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 0x7ff6a41a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46726/0 (socket says 172.21.15.32:46726) 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a3fff700 1 -- 172.21.15.32:0/2113801971 learned_addr learned my addr 172.21.15.32:0/2113801971 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6a409f240 0x7ff6a41a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a889e700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409f650 0x7ff6a41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 -- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409f650 msgr2=0x7ff6a41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409f650 0x7ff6a41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 -- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 msgr2=0x7ff6a41a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 0x7ff6a41a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 -- 172.21.15.32:0/2113801971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c009a70 con 0x7ff6a409f240 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a889e700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409f650 0x7ff6a41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a3fff700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 0x7ff6a41a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.000+0000 7ff6a37fe700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6a409f240 0x7ff6a41a41b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00c4d0 tx=0x7ff68c00c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.001+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c005d70 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.001+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c00ce70 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.001+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0072d0 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.001+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6940094a0 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.001+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a40a0190 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.001+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff68c007430 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.002+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff68c03e910 con 0x7ff6a409f240 2023-09-30T22:29:48.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.002+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a41a0b30 con 0x7ff6a409f240 2023-09-30T22:29:48.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.006+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff68c017030 con 0x7ff6a409f240 2023-09-30T22:29:48.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:48.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.194+0000 7f2c0ef6a700 1 Processor -- start 2023-09-30T22:29:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.194+0000 7f2c0ef6a700 1 -- start start 2023-09-30T22:29:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0ef6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0ef6a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c08139170 0x7f2c0813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0ef6a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0809b120 con 0x7f2c0809d6f0 2023-09-30T22:29:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0ef6a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0806b460 con 0x7f2c08139170 2023-09-30T22:29:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0ef6a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0809a7e0 con 0x7f2c0809e1d0 2023-09-30T22:29:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0d507700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c08139170 0x7f2c0813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0cd06700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2c0809d6f0 0x7f2c0809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38164/0 (socket says 172.21.15.32:38164) 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0cd06700 1 -- 172.21.15.32:0/1858877571 learned_addr learned my addr 172.21.15.32:0/1858877571 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0d507700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2c08139170 0x7f2c0813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46732/0 (socket says 172.21.15.32:46732) 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c07fff700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0d507700 1 -- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 msgr2=0x7f2c08138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0d507700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0d507700 1 -- 172.21.15.32:0/1858877571 >> v1:172.21.15.32:6789/0 conn(0x7f2c0809d6f0 legacy=0x7f2c0809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c0d507700 1 -- 172.21.15.32:0/1858877571 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0809a500 con 0x7f2c08139170 2023-09-30T22:29:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c077fe700 1 -- 172.21.15.32:0/1858877571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023130003 0 0) 0x7f2c0809b120 con 0x7f2c0809d6f0 2023-09-30T22:29:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.195+0000 7f2c07fff700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.196+0000 7f2c0d507700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c08139170 0x7f2c0813b550 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009d90 tx=0x7f2c0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37044cfd1ae1ff8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.196+0000 7f2c077fe700 1 -- 172.21.15.32:0/1858877571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000e070 con 0x7f2c08139170 2023-09-30T22:29:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.196+0000 7f2c077fe700 1 -- 172.21.15.32:0/1858877571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0000bb20 con 0x7f2c08139170 2023-09-30T22:29:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.196+0000 7f2c077fe700 1 -- 172.21.15.32:0/1858877571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00012430 con 0x7f2c08139170 2023-09-30T22:29:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c08139170 msgr2=0x7f2c0813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c08139170 0x7f2c0813b550 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009d90 tx=0x7f2c0000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1858877571 shutdown_connections 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c08139170 0x7f2c0813b550 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1858877571 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1858877571 >> 172.21.15.32:0/1858877571 conn(0x7f2c08066f50 msgr2=0x7f2c0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1858877571 shutdown_connections 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.197+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1858877571 wait complete. 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.198+0000 7f2c0ef6a700 1 Processor -- start 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.198+0000 7f2c0ef6a700 1 -- start start 2023-09-30T22:29:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0ef6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0809d6f0 0x7f2c081216d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0ef6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08121c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0ef6a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2c08139170 0x7f2c08122150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0ef6a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0809c7a0 con 0x7f2c0809d6f0 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0ef6a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c081233a0 con 0x7f2c08139170 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0ef6a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c08124ec0 con 0x7f2c0809e1d0 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c07fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08121c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0cd06700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0809d6f0 0x7f2c081216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c07fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08121c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49066/0 (socket says 172.21.15.32:49066) 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c07fff700 1 -- 172.21.15.32:0/1433098519 learned_addr learned my addr 172.21.15.32:0/1433098519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.199+0000 7f2c0d507700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2c08139170 0x7f2c08122150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0cd06700 1 -- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 msgr2=0x7f2c08121c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0cd06700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08121c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0cd06700 1 -- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2c08139170 msgr2=0x7f2c08122150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0cd06700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2c08139170 0x7f2c08122150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0cd06700 1 -- 172.21.15.32:0/1433098519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf40094a0 con 0x7f2c0809d6f0 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0cd06700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0809d6f0 0x7f2c081216d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00c4a0 tx=0x7f2bfc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc004c80 con 0x7f2c0809d6f0 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc004de0 con 0x7f2c0809d6f0 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc007310 con 0x7f2c0809d6f0 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c00009a70 con 0x7f2c0809d6f0 2023-09-30T22:29:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.200+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c08123a80 con 0x7f2c0809d6f0 2023-09-30T22:29:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.201+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2bfc007470 con 0x7f2c0809d6f0 2023-09-30T22:29:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.201+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c08124b60 con 0x7f2c0809d6f0 2023-09-30T22:29:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.202+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2bfc03f310 con 0x7f2c0809d6f0 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.211+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c08124b60 con 0x7f2c0809d6f0 2023-09-30T22:29:48.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.219+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff6a40a3570 con 0x7ff6a409f240 2023-09-30T22:29:48.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.220+0000 7ff6a0ff9700 1 -- 172.21.15.32:0/2113801971 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff68c01b3b0 con 0x7ff6a409f240 2023-09-30T22:29:48.221 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6a409f240 msgr2=0x7ff6a41a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6a409f240 0x7ff6a41a41b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00c4d0 tx=0x7ff68c00c7e0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 shutdown_connections 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6a409f650 0x7ff6a41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a409e930 0x7ff6a41a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 --2- 172.21.15.32:0/2113801971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6a409f240 0x7ff6a41a41b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 >> 172.21.15.32:0/2113801971 conn(0x7ff6a40a4210 msgr2=0x7ff6a41b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 shutdown_connections 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.223+0000 7ff6aa301700 1 -- 172.21.15.32:0/2113801971 wait complete. 2023-09-30T22:29:48.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:48.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.242+0000 7f291a12f700 1 Processor -- start 2023-09-30T22:29:48.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.242+0000 7f291a12f700 1 -- start start 2023-09-30T22:29:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f291a12f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 0x7f291412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f291a12f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2914135c20 0x7f2914138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f291a12f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291406fa40 con 0x7f291412cf60 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f291a12f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291412cd90 con 0x7f2914135c20 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f291a12f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2914071780 con 0x7f291406f5d0 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 0x7f291412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 0x7f291412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49068/0 (socket says 172.21.15.32:49068) 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 -- 172.21.15.32:0/18723990 learned_addr learned my addr 172.21.15.32:0/18723990 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2913fff700 1 --2- 172.21.15.32:0/18723990 >> v2:172.21.15.32:3300/0 conn(0x7f2914135c20 0x7f2914138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 -- 172.21.15.32:0/18723990 >> v2:172.21.15.32:3300/0 conn(0x7f2914135c20 msgr2=0x7f2914138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 --2- 172.21.15.32:0/18723990 >> v2:172.21.15.32:3300/0 conn(0x7f2914135c20 0x7f2914138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 -- 172.21.15.32:0/18723990 >> v1:172.21.15.32:6789/0 conn(0x7f291412cf60 legacy=0x7f291412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2912ffd700 1 -- 172.21.15.32:0/18723990 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914071a40 con 0x7f291406f5d0 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.243+0000 7f2913fff700 1 --2- 172.21.15.32:0/18723990 >> v2:172.21.15.32:3300/0 conn(0x7f2914135c20 0x7f2914138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f2912ffd700 1 --2- 172.21.15.32:0/18723990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 0x7f291412ffc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f290800bb90 tx=0x7f290800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e28e46964dfdb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f29127fc700 1 -- 172.21.15.32:0/18723990 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290800dd30 con 0x7f291406f5d0 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f29127fc700 1 -- 172.21.15.32:0/18723990 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290800de90 con 0x7f291406f5d0 2023-09-30T22:29:48.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f29127fc700 1 -- 172.21.15.32:0/18723990 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908010690 con 0x7f291406f5d0 2023-09-30T22:29:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 -- 172.21.15.32:0/18723990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 msgr2=0x7f291412ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 --2- 172.21.15.32:0/18723990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 0x7f291412ffc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f290800bb90 tx=0x7f290800bea0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 -- 172.21.15.32:0/18723990 shutdown_connections 2023-09-30T22:29:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 --2- 172.21.15.32:0/18723990 >> v2:172.21.15.32:3300/0 conn(0x7f2914135c20 0x7f2914138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 --2- 172.21.15.32:0/18723990 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f291406f5d0 0x7f291412ffc0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 -- 172.21.15.32:0/18723990 >> 172.21.15.32:0/18723990 conn(0x7f29140a2aa0 msgr2=0x7f29140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 -- 172.21.15.32:0/18723990 shutdown_connections 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.244+0000 7f291a12f700 1 -- 172.21.15.32:0/18723990 wait complete. 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.245+0000 7f291a12f700 1 Processor -- start 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 -- start start 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f291406f5d0 0x7f29141c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f291412cf60 0x7f29141c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 0x7f29141c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291409eb60 con 0x7f291412cf60 2023-09-30T22:29:48.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2914131af0 con 0x7f291406f5d0 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f291a12f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f291412c110 con 0x7f2914135c20 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f2913fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 0x7f29141c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f29137fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f291406f5d0 0x7f29141c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f2913fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 0x7f29141c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49070/0 (socket says 172.21.15.32:49070) 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f2912ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f291412cf60 0x7f29141c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.246+0000 7f2913fff700 1 -- 172.21.15.32:0/2068230011 learned_addr learned my addr 172.21.15.32:0/2068230011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2912ffd700 1 -- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 msgr2=0x7f29141c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2912ffd700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 0x7f29141c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2912ffd700 1 -- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f291406f5d0 msgr2=0x7f29141c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2912ffd700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f291406f5d0 0x7f29141c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2912ffd700 1 -- 172.21.15.32:0/2068230011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c009a70 con 0x7f291412cf60 2023-09-30T22:29:48.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2913fff700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 0x7f29141c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f29137fe700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f291406f5d0 0x7f29141c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f2912ffd700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f291412cf60 0x7f29141c4650 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2908018000 tx=0x7f2908004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290801f070 con 0x7f291412cf60 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908003970 con 0x7f291412cf60 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290800b7f0 con 0x7f291412cf60 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908003d20 con 0x7f291412cf60 2023-09-30T22:29:48.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.247+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291409e0a0 con 0x7f291412cf60 2023-09-30T22:29:48.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.248+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f290802d440 con 0x7f291412cf60 2023-09-30T22:29:48.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.248+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2908042e90 con 0x7f291412cf60 2023-09-30T22:29:48.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.248+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2914074ea0 con 0x7f291412cf60 2023-09-30T22:29:48.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.253+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f290806e750 con 0x7f291412cf60 2023-09-30T22:29:48.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.420+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2c08124b60 con 0x7f2c0809d6f0 2023-09-30T22:29:48.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.421+0000 7f2c057fa700 1 -- 172.21.15.32:0/1433098519 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c08124b60 con 0x7f2c0809d6f0 2023-09-30T22:29:48.422 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0809d6f0 msgr2=0x7f2c081216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0809d6f0 0x7f2c081216d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00c4a0 tx=0x7f2bfc00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 shutdown_connections 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2c08139170 0x7f2c08122150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2c0809d6f0 0x7f2c081216d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 --2- 172.21.15.32:0/1433098519 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2c0809e1d0 0x7f2c08121c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 >> 172.21.15.32:0/1433098519 conn(0x7f2c08066f50 msgr2=0x7f2c080a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 shutdown_connections 2023-09-30T22:29:48.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.423+0000 7f2c0ef6a700 1 -- 172.21.15.32:0/1433098519 wait complete. 2023-09-30T22:29:48.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.461+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29140713a0 con 0x7f291412cf60 2023-09-30T22:29:48.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.462+0000 7f28fbfff700 1 -- 172.21.15.32:0/2068230011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2908031070 con 0x7f291412cf60 2023-09-30T22:29:48.463 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f291412cf60 msgr2=0x7f29141c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f291412cf60 0x7f29141c4650 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2908018000 tx=0x7f2908004750 comp rx=0 tx=0).stop 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 shutdown_connections 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2914135c20 0x7f29141c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f291406f5d0 0x7f29141c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 --2- 172.21.15.32:0/2068230011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f291412cf60 0x7f29141c4650 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 >> 172.21.15.32:0/2068230011 conn(0x7f29140a2aa0 msgr2=0x7f291406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 shutdown_connections 2023-09-30T22:29:48.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:48.464+0000 7f291a12f700 1 -- 172.21.15.32:0/2068230011 wait complete. 2023-09-30T22:29:48.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:48.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.104+0000 7ffb982da700 1 Processor -- start 2023-09-30T22:29:49.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.104+0000 7ffb982da700 1 -- start start 2023-09-30T22:29:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb982da700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb9006b250 0x7ffb90136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb982da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb90136d80 0x7ffb90139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb982da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9009ece0 con 0x7ffb9006a770 2023-09-30T22:29:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb982da700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9009e760 con 0x7ffb9006b250 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb982da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9009e040 con 0x7ffb90136d80 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb95875700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb9006b250 0x7ffb90136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96877700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb90136d80 0x7ffb90139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96076700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb9006a770 0x7ffb9006ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38178/0 (socket says 172.21.15.32:38178) 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96076700 1 -- 172.21.15.32:0/740184127 learned_addr learned my addr 172.21.15.32:0/740184127 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96877700 1 -- 172.21.15.32:0/740184127 >> v2:172.21.15.32:3300/0 conn(0x7ffb9006b250 msgr2=0x7ffb90136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96877700 1 --2- 172.21.15.32:0/740184127 >> v2:172.21.15.32:3300/0 conn(0x7ffb9006b250 0x7ffb90136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96877700 1 -- 172.21.15.32:0/740184127 >> v1:172.21.15.32:6789/0 conn(0x7ffb9006a770 legacy=0x7ffb9006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96877700 1 -- 172.21.15.32:0/740184127 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9009f630 con 0x7ffb90136d80 2023-09-30T22:29:49.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb96877700 1 --2- 172.21.15.32:0/740184127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb90136d80 0x7ffb90139160 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009d90 tx=0x7ffb8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8721972940120301 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.105+0000 7ffb95074700 1 -- 172.21.15.32:0/740184127 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800e070 con 0x7ffb90136d80 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb95074700 1 -- 172.21.15.32:0/740184127 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8800bb20 con 0x7ffb90136d80 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb95074700 1 -- 172.21.15.32:0/740184127 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88012430 con 0x7ffb90136d80 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 -- 172.21.15.32:0/740184127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb90136d80 msgr2=0x7ffb90139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 --2- 172.21.15.32:0/740184127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb90136d80 0x7ffb90139160 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009d90 tx=0x7ffb8800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 -- 172.21.15.32:0/740184127 shutdown_connections 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 --2- 172.21.15.32:0/740184127 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb90136d80 0x7ffb90139160 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 --2- 172.21.15.32:0/740184127 >> v2:172.21.15.32:3300/0 conn(0x7ffb9006b250 0x7ffb90136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 -- 172.21.15.32:0/740184127 >> 172.21.15.32:0/740184127 conn(0x7ffb90066f50 msgr2=0x7ffb9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 -- 172.21.15.32:0/740184127 shutdown_connections 2023-09-30T22:29:49.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.106+0000 7ffb982da700 1 -- 172.21.15.32:0/740184127 wait complete. 2023-09-30T22:29:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.107+0000 7ffb982da700 1 Processor -- start 2023-09-30T22:29:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.107+0000 7ffb982da700 1 -- start start 2023-09-30T22:29:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb982da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 0x7ffb9011b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb982da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb9006b250 0x7ffb9011b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb982da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb90136d80 0x7ffb9011be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb982da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9011de20 con 0x7ffb90136d80 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb982da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9011d020 con 0x7ffb9006b250 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb982da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9011e370 con 0x7ffb9006a770 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 0x7ffb9011b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 0x7ffb9011b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49096/0 (socket says 172.21.15.32:49096) 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 -- 172.21.15.32:0/21215148 learned_addr learned my addr 172.21.15.32:0/21215148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96877700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb90136d80 0x7ffb9011be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb95875700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb9006b250 0x7ffb9011b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 -- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb9006b250 msgr2=0x7ffb9011b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb9006b250 0x7ffb9011b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 -- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb90136d80 msgr2=0x7ffb9011be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb90136d80 0x7ffb9011be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 -- 172.21.15.32:0/21215148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88009a70 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96877700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb90136d80 0x7ffb9011be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb95875700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb9006b250 0x7ffb9011b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb96076700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 0x7ffb9011b410 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c550 tx=0x7ffb8c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.108+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c005d70 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.109+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c0070d0 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.109+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9011ef30 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.109+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c007440 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.109+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9011ec50 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.109+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb8c0075a0 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.109+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffb8c03ead0 con 0x7ffb9006a770 2023-09-30T22:29:49.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.110+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb90074ea0 con 0x7ffb9006a770 2023-09-30T22:29:49.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.115+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb8c017080 con 0x7ffb9006a770 2023-09-30T22:29:49.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:49.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.319+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffb9006c360 con 0x7ffb9006a770 2023-09-30T22:29:49.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.320+0000 7ffb82ffd700 1 -- 172.21.15.32:0/21215148 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb8c019b40 con 0x7ffb9006a770 2023-09-30T22:29:49.321 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:49.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 msgr2=0x7ffb9011b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 0x7ffb9011b410 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c550 tx=0x7ffb8c00c860 comp rx=0 tx=0).stop 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 shutdown_connections 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb90136d80 0x7ffb9011be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb9006a770 0x7ffb9011b410 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 --2- 172.21.15.32:0/21215148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb9006b250 0x7ffb9011b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 >> 172.21.15.32:0/21215148 conn(0x7ffb90066f50 msgr2=0x7ffb900a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 shutdown_connections 2023-09-30T22:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.322+0000 7ffb982da700 1 -- 172.21.15.32:0/21215148 wait complete. 2023-09-30T22:29:49.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.378+0000 7f704f228700 1 Processor -- start 2023-09-30T22:29:49.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.378+0000 7f704f228700 1 -- start start 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704f228700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 0x7f704809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704f228700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 0x7f7048131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704f228700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7048070940 con 0x7f704809f240 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704f228700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704806fd60 con 0x7f704809f650 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704f228700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704806fa80 con 0x7f704809e930 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704d7c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 0x7f7048131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 0x7f704809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 0x7f704809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40300/0 (socket says 172.21.15.32:40300) 2023-09-30T22:29:49.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f7047fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f704809f240 0x7f70481317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38194/0 (socket says 172.21.15.32:38194) 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704d7c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 0x7f7048131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46762/0 (socket says 172.21.15.32:46762) 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 -- 172.21.15.32:0/3233204048 learned_addr learned my addr 172.21.15.32:0/3233204048 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 -- 172.21.15.32:0/3233204048 >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 msgr2=0x7f7048131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 --2- 172.21.15.32:0/3233204048 >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 0x7f7048131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 -- 172.21.15.32:0/3233204048 >> v1:172.21.15.32:6789/0 conn(0x7f704809f240 legacy=0x7f70481317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704cfc4700 1 -- 172.21.15.32:0/3233204048 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704809fb60 con 0x7f704809e930 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f70477fe700 1 -- 172.21.15.32:0/3233204048 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758403706 0 0) 0x7f7048070940 con 0x7f704809f240 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.379+0000 7f704d7c5700 1 --2- 172.21.15.32:0/3233204048 >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 0x7f7048131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704cfc4700 1 --2- 172.21.15.32:0/3233204048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 0x7f704809ed00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f70380098f0 tx=0x7f7038009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc37b9d18c72970 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f70477fe700 1 -- 172.21.15.32:0/3233204048 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038011830 con 0x7f704809e930 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f70477fe700 1 -- 172.21.15.32:0/3233204048 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038011990 con 0x7f704809e930 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f70477fe700 1 -- 172.21.15.32:0/3233204048 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038011d20 con 0x7f704809e930 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 -- 172.21.15.32:0/3233204048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 msgr2=0x7f704809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 --2- 172.21.15.32:0/3233204048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 0x7f704809ed00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f70380098f0 tx=0x7f7038009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 -- 172.21.15.32:0/3233204048 shutdown_connections 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 --2- 172.21.15.32:0/3233204048 >> v2:172.21.15.32:3300/0 conn(0x7f704809f650 0x7f7048131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 --2- 172.21.15.32:0/3233204048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809e930 0x7f704809ed00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 -- 172.21.15.32:0/3233204048 >> 172.21.15.32:0/3233204048 conn(0x7f70480a4210 msgr2=0x7f70480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 -- 172.21.15.32:0/3233204048 shutdown_connections 2023-09-30T22:29:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.380+0000 7f704f228700 1 -- 172.21.15.32:0/3233204048 wait complete. 2023-09-30T22:29:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.381+0000 7f704f228700 1 Processor -- start 2023-09-30T22:29:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 -- start start 2023-09-30T22:29:49.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 0x7f70481a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704809f240 0x7f70481a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809f650 0x7f70481a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481a50e0 con 0x7f704809f240 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70481a5840 con 0x7f704809f650 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704f228700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70481a6a90 con 0x7f704809e930 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704cfc4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 0x7f70481a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704d7c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809f650 0x7f70481a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704cfc4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 0x7f70481a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49108/0 (socket says 172.21.15.32:49108) 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f704cfc4700 1 -- 172.21.15.32:0/3984167270 learned_addr learned my addr 172.21.15.32:0/3984167270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.382+0000 7f7047fff700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704809f240 0x7f70481a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704cfc4700 1 -- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809f650 msgr2=0x7f70481a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704cfc4700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809f650 0x7f70481a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704cfc4700 1 -- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704809f240 msgr2=0x7f70481a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704cfc4700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704809f240 0x7f70481a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704cfc4700 1 -- 172.21.15.32:0/3984167270 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009a70 con 0x7f704809e930 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f7047fff700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704809f240 0x7f70481a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704d7c5700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809f650 0x7f70481a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704cfc4700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 0x7f70481a3c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f704806fbc0 tx=0x7f7038004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70380044d0 con 0x7f704809e930 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70380036a0 con 0x7f704809e930 2023-09-30T22:29:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038003a30 con 0x7f704809e930 2023-09-30T22:29:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7038009550 con 0x7f704809e930 2023-09-30T22:29:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.383+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480a0190 con 0x7f704809e930 2023-09-30T22:29:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.384+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7038021070 con 0x7f704809e930 2023-09-30T22:29:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.384+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7038040b80 con 0x7f704809e930 2023-09-30T22:29:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.384+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70480a3220 con 0x7f704809e930 2023-09-30T22:29:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.389+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7038040e60 con 0x7f704809e930 2023-09-30T22:29:49.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:49.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.580+0000 7f4762b7a700 1 Processor -- start 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 -- start start 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09d6f0 0x7f475c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09e000 0x7f475c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f475c09e410 0x7f475c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c06b460 con 0x7f475c09e410 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c09b120 con 0x7f475c09e000 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4762b7a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f475c09a7e0 con 0x7f475c09d6f0 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4761117700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f475c09e410 0x7f475c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4761117700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f475c09e410 0x7f475c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46774/0 (socket says 172.21.15.32:46774) 2023-09-30T22:29:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f475bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09e000 0x7f475c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4761117700 1 -- 172.21.15.32:0/227754780 learned_addr learned my addr 172.21.15.32:0/227754780 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.581+0000 7f4760916700 1 --2- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09d6f0 0x7f475c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475bfff700 1 -- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09d6f0 msgr2=0x7f475c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475bfff700 1 --2- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09d6f0 0x7f475c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475bfff700 1 -- 172.21.15.32:0/227754780 >> v2:172.21.15.32:3300/0 conn(0x7f475c09e410 msgr2=0x7f475c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475bfff700 1 --2- 172.21.15.32:0/227754780 >> v2:172.21.15.32:3300/0 conn(0x7f475c09e410 0x7f475c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475bfff700 1 -- 172.21.15.32:0/227754780 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c09a500 con 0x7f475c09e000 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475bfff700 1 --2- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09e000 0x7f475c1339f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4748009870 tx=0x7f4748009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b6eebc570e93b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475b7fe700 1 -- 172.21.15.32:0/227754780 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011830 con 0x7f475c09e000 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475b7fe700 1 -- 172.21.15.32:0/227754780 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748011990 con 0x7f475c09e000 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.582+0000 7f475b7fe700 1 -- 172.21.15.32:0/227754780 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011d20 con 0x7f475c09e000 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 -- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09e000 msgr2=0x7f475c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 --2- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09e000 0x7f475c1339f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4748009870 tx=0x7f4748009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 -- 172.21.15.32:0/227754780 shutdown_connections 2023-09-30T22:29:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 --2- 172.21.15.32:0/227754780 >> v2:172.21.15.32:3300/0 conn(0x7f475c09e410 0x7f475c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 --2- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09d6f0 0x7f475c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 --2- 172.21.15.32:0/227754780 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09e000 0x7f475c1339f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 -- 172.21.15.32:0/227754780 >> 172.21.15.32:0/227754780 conn(0x7f475c066f50 msgr2=0x7f475c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 -- 172.21.15.32:0/227754780 shutdown_connections 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.583+0000 7f4762b7a700 1 -- 172.21.15.32:0/227754780 wait complete. 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.584+0000 7f4762b7a700 1 Processor -- start 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.584+0000 7f4762b7a700 1 -- start start 2023-09-30T22:29:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4762b7a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 0x7f475c1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4762b7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c09e000 0x7f475c1a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4762b7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09e410 0x7f475c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4762b7a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1a91f0 con 0x7f475c09e000 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4762b7a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1aa340 con 0x7f475c09d6f0 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4762b7a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1a8cc0 con 0x7f475c09e410 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 0x7f475c1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 0x7f475c1a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40316/0 (socket says 172.21.15.32:40316) 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f475bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c09e000 0x7f475c1a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 -- 172.21.15.32:0/1313616542 learned_addr learned my addr 172.21.15.32:0/1313616542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4761117700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09e410 0x7f475c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 -- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09e410 msgr2=0x7f475c1a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09e410 0x7f475c1a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 -- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c09e000 msgr2=0x7f475c1a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c09e000 0x7f475c1a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 -- 172.21.15.32:0/1313616542 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47480094a0 con 0x7f475c09d6f0 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.585+0000 7f4760916700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 0x7f475c1a5eb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f475000e9e0 tx=0x7f475000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.586+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750009710 con 0x7f475c09d6f0 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.586+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475000c850 con 0x7f475c09d6f0 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.586+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c1a96a0 con 0x7f475c09d6f0 2023-09-30T22:29:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.586+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475000cbc0 con 0x7f475c09d6f0 2023-09-30T22:29:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.586+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c09c710 con 0x7f475c09d6f0 2023-09-30T22:29:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.587+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c074ea0 con 0x7f475c09d6f0 2023-09-30T22:29:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.587+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f475000cd20 con 0x7f475c09d6f0 2023-09-30T22:29:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.588+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f475003ff40 con 0x7f475c09d6f0 2023-09-30T22:29:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.592+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4750011b40 con 0x7f475c09d6f0 2023-09-30T22:29:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.612+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f704809ffe0 con 0x7f704809e930 2023-09-30T22:29:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.613+0000 7f70457fa700 1 -- 172.21.15.32:0/3984167270 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7038029020 con 0x7f704809e930 2023-09-30T22:29:49.614 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 msgr2=0x7f70481a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 0x7f70481a3c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f704806fbc0 tx=0x7f7038004750 comp rx=0 tx=0).stop 2023-09-30T22:29:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 shutdown_connections 2023-09-30T22:29:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f704809f650 0x7f70481a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f704809e930 0x7f70481a3c30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 --2- 172.21.15.32:0/3984167270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f704809f240 0x7f70481a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 >> 172.21.15.32:0/3984167270 conn(0x7f70480a4210 msgr2=0x7f70481b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 shutdown_connections 2023-09-30T22:29:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.616+0000 7f704f228700 1 -- 172.21.15.32:0/3984167270 wait complete. 2023-09-30T22:29:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.622+0000 7f7440833700 1 Processor -- start 2023-09-30T22:29:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.622+0000 7f7440833700 1 -- start start 2023-09-30T22:29:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f7440833700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c1338a0 0x7f743c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f7440833700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f743c09de50 0x7f743c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f7440833700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c06ef30 con 0x7f743c09e760 2023-09-30T22:29:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f7440833700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c06fef0 con 0x7f743c09de50 2023-09-30T22:29:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f7440833700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f743c1334b0 con 0x7f743c1338a0 2023-09-30T22:29:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f7439d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f743c09de50 0x7f743c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f743a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c1338a0 0x7f743c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f743ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f743c09e760 0x7f743c13a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38196/0 (socket says 172.21.15.32:38196) 2023-09-30T22:29:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.623+0000 7f743ad9d700 1 -- 172.21.15.32:0/2569651423 learned_addr learned my addr 172.21.15.32:0/2569651423 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743a59c700 1 -- 172.21.15.32:0/2569651423 >> v2:172.21.15.32:3300/0 conn(0x7f743c09de50 msgr2=0x7f743c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743a59c700 1 --2- 172.21.15.32:0/2569651423 >> v2:172.21.15.32:3300/0 conn(0x7f743c09de50 0x7f743c09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743a59c700 1 -- 172.21.15.32:0/2569651423 >> v1:172.21.15.32:6789/0 conn(0x7f743c09e760 legacy=0x7f743c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743a59c700 1 -- 172.21.15.32:0/2569651423 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c06fc30 con 0x7f743c1338a0 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743a59c700 1 --2- 172.21.15.32:0/2569651423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c1338a0 0x7f743c133c70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f742c009840 tx=0x7f742c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966e8a710cce3d83 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743959a700 1 -- 172.21.15.32:0/2569651423 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c011830 con 0x7f743c1338a0 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743959a700 1 -- 172.21.15.32:0/2569651423 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c011990 con 0x7f743c1338a0 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.624+0000 7f743959a700 1 -- 172.21.15.32:0/2569651423 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c011d20 con 0x7f743c1338a0 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 -- 172.21.15.32:0/2569651423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c1338a0 msgr2=0x7f743c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 --2- 172.21.15.32:0/2569651423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c1338a0 0x7f743c133c70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f742c009840 tx=0x7f742c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 -- 172.21.15.32:0/2569651423 shutdown_connections 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 --2- 172.21.15.32:0/2569651423 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c1338a0 0x7f743c133c70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 --2- 172.21.15.32:0/2569651423 >> v2:172.21.15.32:3300/0 conn(0x7f743c09de50 0x7f743c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 -- 172.21.15.32:0/2569651423 >> 172.21.15.32:0/2569651423 conn(0x7f743c0a2aa0 msgr2=0x7f743c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.625+0000 7f7440833700 1 -- 172.21.15.32:0/2569651423 shutdown_connections 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.626+0000 7f7440833700 1 -- 172.21.15.32:0/2569651423 wait complete. 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.627+0000 7f7440833700 1 Processor -- start 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.627+0000 7f7440833700 1 -- start start 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7440833700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 0x7f743c11ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7440833700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f743c09e760 0x7f743c11f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7440833700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f743c1338a0 0x7f743c11f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7440833700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c120830 con 0x7f743c09e760 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7440833700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0a1000 con 0x7f743c1338a0 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7440833700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f743c06f640 con 0x7f743c09de50 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f743a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 0x7f743c11ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7439d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f743c09e760 0x7f743c11f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f743a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 0x7f743c11ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:49134/0 (socket says 172.21.15.32:49134) 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f743a59c700 1 -- 172.21.15.32:0/3183756978 learned_addr learned my addr 172.21.15.32:0/3183756978 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f743ad9d700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f743c1338a0 0x7f743c11f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7439d9b700 1 -- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 msgr2=0x7f743c11ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7439d9b700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 0x7f743c11ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7439d9b700 1 -- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f743c1338a0 msgr2=0x7f743c11f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7439d9b700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f743c1338a0 0x7f743c11f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f7439d9b700 1 -- 172.21.15.32:0/3183756978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c0094a0 con 0x7f743c09e760 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f743a59c700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 0x7f743c11ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.628+0000 7f743ad9d700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f743c1338a0 0x7f743c11f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.629+0000 7f7439d9b700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f743c09e760 0x7f743c11f160 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f743000e840 tx=0x7f743000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.629+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430013680 con 0x7f743c09e760 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.629+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74300137e0 con 0x7f743c09e760 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.629+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c12fb70 con 0x7f743c09e760 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.629+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430013b50 con 0x7f743c09e760 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.629+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c122c40 con 0x7f743c09e760 2023-09-30T22:29:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.630+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c130730 con 0x7f743c09e760 2023-09-30T22:29:49.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.631+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7430013cb0 con 0x7f743c09e760 2023-09-30T22:29:49.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.631+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f743003fc80 con 0x7f743c09e760 2023-09-30T22:29:49.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.635+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7430011db0 con 0x7f743c09e760 2023-09-30T22:29:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.811+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f475c1a99e0 con 0x7f475c09d6f0 2023-09-30T22:29:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.812+0000 7f47597fa700 1 -- 172.21.15.32:0/1313616542 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4750021070 con 0x7f475c09d6f0 2023-09-30T22:29:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 msgr2=0x7f475c1a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 0x7f475c1a5eb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f475000e9e0 tx=0x7f475000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 shutdown_connections 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f475c09e410 0x7f475c1a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c09d6f0 0x7f475c1a5eb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 --2- 172.21.15.32:0/1313616542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c09e000 0x7f475c1a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 >> 172.21.15.32:0/1313616542 conn(0x7f475c066f50 msgr2=0x7f475c1baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 shutdown_connections 2023-09-30T22:29:49.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.814+0000 7f4762b7a700 1 -- 172.21.15.32:0/1313616542 wait complete. 2023-09-30T22:29:49.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.842+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f743c074ea0 con 0x7f743c09e760 2023-09-30T22:29:49.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.842+0000 7f742affd700 1 -- 172.21.15.32:0/3183756978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7430021070 con 0x7f743c09e760 2023-09-30T22:29:49.843 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f743c09e760 msgr2=0x7f743c11f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f743c09e760 0x7f743c11f160 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f743000e840 tx=0x7f743000eb50 comp rx=0 tx=0).stop 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 shutdown_connections 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f743c1338a0 0x7f743c11f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f743c09de50 0x7f743c11ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 --2- 172.21.15.32:0/3183756978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f743c09e760 0x7f743c11f160 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 >> 172.21.15.32:0/3183756978 conn(0x7f743c0a2aa0 msgr2=0x7f743c09edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 shutdown_connections 2023-09-30T22:29:49.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:49.845+0000 7f7440833700 1 -- 172.21.15.32:0/3183756978 wait complete. 2023-09-30T22:29:49.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:50.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 Processor -- start 2023-09-30T22:29:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 -- start start 2023-09-30T22:29:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 0x7f682009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f6820131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6820070940 con 0x7f682009e930 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682006fd60 con 0x7f682009f650 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6826f5d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f682006fa80 con 0x7f682009f240 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f6824cf9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 0x7f682009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.484+0000 7f681ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57248/0 (socket says 172.21.15.32:57248) 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f6824cf9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 0x7f682009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39594/0 (socket says 172.21.15.32:39594) 2023-09-30T22:29:50.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 -- 172.21.15.32:0/422901966 learned_addr learned my addr 172.21.15.32:0/422901966 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f68254fa700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f6820131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 -- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 msgr2=0x7f6820131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f6820131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 -- 172.21.15.32:0/422901966 >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 msgr2=0x7f682009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 --2- 172.21.15.32:0/422901966 >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 0x7f682009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 -- 172.21.15.32:0/422901966 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682009fb60 con 0x7f682009f240 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f6824cf9700 1 --2- 172.21.15.32:0/422901966 >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 0x7f682009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f68254fa700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f6820131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:50.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681ffff700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201317f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f68140098f0 tx=0x7f6814009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0c353edac6ffa2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681f7fe700 1 -- 172.21.15.32:0/422901966 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011830 con 0x7f682009f240 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681f7fe700 1 -- 172.21.15.32:0/422901966 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814011990 con 0x7f682009f240 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.485+0000 7f681f7fe700 1 -- 172.21.15.32:0/422901966 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011d20 con 0x7f682009f240 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 -- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 msgr2=0x7f68201317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201317f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f68140098f0 tx=0x7f6814009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 -- 172.21.15.32:0/422901966 shutdown_connections 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f6820131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 --2- 172.21.15.32:0/422901966 >> v2:172.21.15.32:3300/0 conn(0x7f682009e930 0x7f682009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 --2- 172.21.15.32:0/422901966 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201317f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 -- 172.21.15.32:0/422901966 >> 172.21.15.32:0/422901966 conn(0x7f68200a4210 msgr2=0x7f68200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 -- 172.21.15.32:0/422901966 shutdown_connections 2023-09-30T22:29:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.486+0000 7f6826f5d700 1 -- 172.21.15.32:0/422901966 wait complete. 2023-09-30T22:29:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.487+0000 7f6826f5d700 1 Processor -- start 2023-09-30T22:29:50.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.487+0000 7f6826f5d700 1 -- start start 2023-09-30T22:29:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6826f5d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f682009e930 0x7f68201a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6826f5d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6826f5d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f68201a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6826f5d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201a5ee0 con 0x7f682009e930 2023-09-30T22:29:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6826f5d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68201a8160 con 0x7f682009f650 2023-09-30T22:29:50.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6826f5d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68201a77a0 con 0x7f682009f240 2023-09-30T22:29:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f68254fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f68201a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f6824cf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f682009e930 0x7f68201a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57258/0 (socket says 172.21.15.32:57258) 2023-09-30T22:29:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 -- 172.21.15.32:0/1643109205 learned_addr learned my addr 172.21.15.32:0/1643109205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:50.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 -- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 msgr2=0x7f68201a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f68201a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 -- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f682009e930 msgr2=0x7f68201a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f682009e930 0x7f68201a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 -- 172.21.15.32:0/1643109205 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100094a0 con 0x7f682009f240 2023-09-30T22:29:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.488+0000 7f681ffff700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201a4250 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6814009c40 tx=0x7f6814004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:50.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.489+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814004240 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.489+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681401f070 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.489+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68140045e0 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.489+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6818009a70 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.489+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6814009550 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.490+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6820071d20 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.490+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6814021030 con 0x7f682009f240 2023-09-30T22:29:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.490+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6814040aa0 con 0x7f682009f240 2023-09-30T22:29:50.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.494+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f681400b7f0 con 0x7f682009f240 2023-09-30T22:29:50.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:50.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.698+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6820074ea0 con 0x7f682009f240 2023-09-30T22:29:50.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.699+0000 7f681d7fa700 1 -- 172.21.15.32:0/1643109205 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6814015070 con 0x7f682009f240 2023-09-30T22:29:50.700 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 msgr2=0x7f68201a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201a4250 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6814009c40 tx=0x7f6814004750 comp rx=0 tx=0).stop 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 shutdown_connections 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f682009f650 0x7f68201a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f682009e930 0x7f68201a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 --2- 172.21.15.32:0/1643109205 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f682009f240 0x7f68201a4250 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 >> 172.21.15.32:0/1643109205 conn(0x7f68200a4210 msgr2=0x7f68201b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:50.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 shutdown_connections 2023-09-30T22:29:50.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.701+0000 7f6826f5d700 1 -- 172.21.15.32:0/1643109205 wait complete. 2023-09-30T22:29:50.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 Processor -- start 2023-09-30T22:29:50.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 -- start start 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1009b120 con 0x7f3c1009e410 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1006b460 con 0x7f3c1009e000 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.775+0000 7f3c1670c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1009a7e0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c14ca9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c1009e410 0x7f3c10133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55722/0 (socket says 172.21.15.32:55722) 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c14ca9700 1 -- 172.21.15.32:0/759767342 learned_addr learned my addr 172.21.15.32:0/759767342 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0ffff700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0f7fe700 1 -- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 msgr2=0x7f3c1009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0effd700 1 -- 172.21.15.32:0/759767342 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661533081 0 0) 0x7f3c1009b120 con 0x7f3c1009e410 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c1009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0f7fe700 1 -- 172.21.15.32:0/759767342 >> v1:172.21.15.32:6789/0 conn(0x7f3c1009e410 legacy=0x7f3c10133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0f7fe700 1 -- 172.21.15.32:0/759767342 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1009a500 con 0x7f3c1009e000 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0ffff700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101339f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009870 tx=0x7f3c00009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9626f4f7569fdb9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0effd700 1 -- 172.21.15.32:0/759767342 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00011830 con 0x7f3c1009e000 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0effd700 1 -- 172.21.15.32:0/759767342 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c00011990 con 0x7f3c1009e000 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.776+0000 7f3c0effd700 1 -- 172.21.15.32:0/759767342 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00011d20 con 0x7f3c1009e000 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 -- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 msgr2=0x7f3c101339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101339f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009870 tx=0x7f3c00009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 -- 172.21.15.32:0/759767342 shutdown_connections 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 --2- 172.21.15.32:0/759767342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101339f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 -- 172.21.15.32:0/759767342 >> 172.21.15.32:0/759767342 conn(0x7f3c10066f50 msgr2=0x7f3c1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 -- 172.21.15.32:0/759767342 shutdown_connections 2023-09-30T22:29:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.777+0000 7f3c1670c700 1 -- 172.21.15.32:0/759767342 wait complete. 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.778+0000 7f3c1670c700 1 Processor -- start 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.778+0000 7f3c1670c700 1 -- start start 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c1670c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c10122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c1670c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c1670c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1009e410 0x7f3c10122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c1670c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10125cb0 con 0x7f3c1009e410 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c1670c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10125f90 con 0x7f3c1009e000 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c1670c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100a30d0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c10122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c10122470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57282/0 (socket says 172.21.15.32:57282) 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 -- 172.21.15.32:0/1956559841 learned_addr learned my addr 172.21.15.32:0/1956559841 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c14ca9700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1009e410 0x7f3c10122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0f7fe700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 -- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 msgr2=0x7f3c101229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 -- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1009e410 msgr2=0x7f3c10122ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1009e410 0x7f3c10122ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 -- 172.21.15.32:0/1956559841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf80094a0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.779+0000 7f3c0ffff700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c10122470 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800ebe0 tx=0x7f3bf800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.780+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800c850 con 0x7f3c1009d6f0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.780+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf800c9b0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.780+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c000094a0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.780+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800cd40 con 0x7f3c1009d6f0 2023-09-30T22:29:50.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.780+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c10124ea0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.780+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bf8013470 con 0x7f3c1009d6f0 2023-09-30T22:29:50.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.781+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3bf803fb30 con 0x7f3c1009d6f0 2023-09-30T22:29:50.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.781+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c10074ea0 con 0x7f3c1009d6f0 2023-09-30T22:29:50.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.786+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bf8013720 con 0x7f3c1009d6f0 2023-09-30T22:29:50.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:50.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.968+0000 7fa4be3ed700 1 Processor -- start 2023-09-30T22:29:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.968+0000 7fa4be3ed700 1 -- start start 2023-09-30T22:29:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4be3ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809e930 0x7fa4b809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4be3ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f650 0x7fa4b8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4be3ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b8070940 con 0x7fa4b809f240 2023-09-30T22:29:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4be3ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b806fd60 con 0x7fa4b809e930 2023-09-30T22:29:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4be3ed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b806fa80 con 0x7fa4b809f650 2023-09-30T22:29:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa4b809f240 0x7fa4b81317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55726/0 (socket says 172.21.15.32:55726) 2023-09-30T22:29:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4bc98a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f650 0x7fa4b8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b77fe700 1 -- 172.21.15.32:0/3905639575 learned_addr learned my addr 172.21.15.32:0/3905639575 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b7fff700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809e930 0x7fa4b809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b7fff700 1 -- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f650 msgr2=0x7fa4b8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b6ffd700 1 -- 172.21.15.32:0/3905639575 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125098599 0 0) 0x7fa4b8070940 con 0x7fa4b809f240 2023-09-30T22:29:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b7fff700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f650 0x7fa4b8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b7fff700 1 -- 172.21.15.32:0/3905639575 >> v1:172.21.15.32:6789/0 conn(0x7fa4b809f240 legacy=0x7fa4b81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4b7fff700 1 -- 172.21.15.32:0/3905639575 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b809fb60 con 0x7fa4b809e930 2023-09-30T22:29:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.969+0000 7fa4bc98a700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f650 0x7fa4b8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.970+0000 7fa4b7fff700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809e930 0x7fa4b809ed00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a546cfa682bcf05a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.970+0000 7fa4b6ffd700 1 -- 172.21.15.32:0/3905639575 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8011830 con 0x7fa4b809e930 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.970+0000 7fa4b6ffd700 1 -- 172.21.15.32:0/3905639575 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a8011990 con 0x7fa4b809e930 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.970+0000 7fa4b6ffd700 1 -- 172.21.15.32:0/3905639575 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8011d20 con 0x7fa4b809e930 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.970+0000 7fa4be3ed700 1 -- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809e930 msgr2=0x7fa4b809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.970+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809e930 0x7fa4b809ed00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a8009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.971+0000 7fa4be3ed700 1 -- 172.21.15.32:0/3905639575 shutdown_connections 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.971+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f650 0x7fa4b8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.971+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/3905639575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809e930 0x7fa4b809ed00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.971+0000 7fa4be3ed700 1 -- 172.21.15.32:0/3905639575 >> 172.21.15.32:0/3905639575 conn(0x7fa4b80a4210 msgr2=0x7fa4b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.971+0000 7fa4be3ed700 1 -- 172.21.15.32:0/3905639575 shutdown_connections 2023-09-30T22:29:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.971+0000 7fa4be3ed700 1 -- 172.21.15.32:0/3905639575 wait complete. 2023-09-30T22:29:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.972+0000 7fa4be3ed700 1 Processor -- start 2023-09-30T22:29:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.972+0000 7fa4be3ed700 1 -- start start 2023-09-30T22:29:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4be3ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4b809e930 0x7fa4b81a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4be3ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 0x7fa4b81a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4be3ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 0x7fa4b81a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4be3ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81a8030 con 0x7fa4b809e930 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4be3ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81a5760 con 0x7fa4b809f650 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4be3ed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b81a5d80 con 0x7fa4b809f240 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4b77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 0x7fa4b81a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4bc98a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 0x7fa4b81a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4b7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4b809e930 0x7fa4b81a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4bc98a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 0x7fa4b81a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45218/0 (socket says 172.21.15.32:45218) 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4b77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 0x7fa4b81a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57296/0 (socket says 172.21.15.32:57296) 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.973+0000 7fa4bc98a700 1 -- 172.21.15.32:0/518933783 learned_addr learned my addr 172.21.15.32:0/518933783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4bc98a700 1 -- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 msgr2=0x7fa4b81a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4bc98a700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 0x7fa4b81a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4bc98a700 1 -- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4b809e930 msgr2=0x7fa4b81a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4bc98a700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4b809e930 0x7fa4b81a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:50.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4bc98a700 1 -- 172.21.15.32:0/518933783 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0009a70 con 0x7fa4b809f650 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4b77fe700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 0x7fa4b81a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4b7fff700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4b809e930 0x7fa4b81a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4bc98a700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 0x7fa4b81a45d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000efc0 tx=0x7fa4b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000cc90 con 0x7fa4b809f650 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b000f040 con 0x7fa4b809f650 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0014480 con 0x7fa4b809f650 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a80094a0 con 0x7fa4b809f650 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.974+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b80a0190 con 0x7fa4b809f650 2023-09-30T22:29:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.975+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa4b0014600 con 0x7fa4b809f650 2023-09-30T22:29:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.975+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b80a3550 con 0x7fa4b809f650 2023-09-30T22:29:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.976+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa4b00400d0 con 0x7fa4b809f650 2023-09-30T22:29:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.980+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa4b0018070 con 0x7fa4b809f650 2023-09-30T22:29:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.997+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3c1006c360 con 0x7f3c1009d6f0 2023-09-30T22:29:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:50.997+0000 7f3c0cff9700 1 -- 172.21.15.32:0/1956559841 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bf8021030 con 0x7f3c1009d6f0 2023-09-30T22:29:50.998 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 msgr2=0x7f3c10122470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c10122470 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800ebe0 tx=0x7f3bf800eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 shutdown_connections 2023-09-30T22:29:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c1009e410 0x7f3c10122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3c1009d6f0 0x7f3c10122470 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 --2- 172.21.15.32:0/1956559841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3c1009e000 0x7f3c101229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 >> 172.21.15.32:0/1956559841 conn(0x7f3c10066f50 msgr2=0x7f3c10070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 shutdown_connections 2023-09-30T22:29:51.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.000+0000 7f3c1670c700 1 -- 172.21.15.32:0/1956559841 wait complete. 2023-09-30T22:29:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.013+0000 7fb87dc83700 1 Processor -- start 2023-09-30T22:29:51.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.013+0000 7fb87dc83700 1 -- start start 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.013+0000 7fb87dc83700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8780a1510 0x7fb878138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.013+0000 7fb87dc83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb878139250 0x7fb87813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb87dc83700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87809edd0 con 0x7fb8780a0a30 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb87dc83700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8780a18e0 con 0x7fb8780a1510 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb87dc83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87809e1f0 con 0x7fb878139250 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8780a1510 0x7fb878138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8780a1510 0x7fb878138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39628/0 (socket says 172.21.15.32:39628) 2023-09-30T22:29:51.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb8777fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8780a0a30 0x7fb8780a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55732/0 (socket says 172.21.15.32:55732) 2023-09-30T22:29:51.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 -- 172.21.15.32:0/2828589017 learned_addr learned my addr 172.21.15.32:0/2828589017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:51.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb877fff700 1 --2- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb878139250 0x7fb87813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 -- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb878139250 msgr2=0x7fb87813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 --2- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb878139250 0x7fb87813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 -- 172.21.15.32:0/2828589017 >> v1:172.21.15.32:6789/0 conn(0x7fb8780a0a30 legacy=0x7fb8780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 -- 172.21.15.32:0/2828589017 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87809dc50 con 0x7fb8780a1510 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb876ffd700 1 -- 172.21.15.32:0/2828589017 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544545623 0 0) 0x7fb87809edd0 con 0x7fb8780a0a30 2023-09-30T22:29:51.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.014+0000 7fb86effd700 1 --2- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a1510 0x7fb878138d10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb8600098f0 tx=0x7fb860009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c7c19039cb3c3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb876ffd700 1 -- 172.21.15.32:0/2828589017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860011830 con 0x7fb8780a1510 2023-09-30T22:29:51.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb876ffd700 1 -- 172.21.15.32:0/2828589017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860011990 con 0x7fb8780a1510 2023-09-30T22:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb876ffd700 1 -- 172.21.15.32:0/2828589017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860011d20 con 0x7fb8780a1510 2023-09-30T22:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 -- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a1510 msgr2=0x7fb878138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a1510 0x7fb878138d10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb8600098f0 tx=0x7fb860009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 -- 172.21.15.32:0/2828589017 shutdown_connections 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb878139250 0x7fb87813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2828589017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a1510 0x7fb878138d10 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 -- 172.21.15.32:0/2828589017 >> 172.21.15.32:0/2828589017 conn(0x7fb8780a6550 msgr2=0x7fb8780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 -- 172.21.15.32:0/2828589017 shutdown_connections 2023-09-30T22:29:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.015+0000 7fb87dc83700 1 -- 172.21.15.32:0/2828589017 wait complete. 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.017+0000 7fb87dc83700 1 Processor -- start 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.017+0000 7fb87dc83700 1 -- start start 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb87dc83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a0a30 0x7fb878122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb87dc83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 0x7fb878122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb87dc83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb878139250 0x7fb8781232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb87dc83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781243e0 con 0x7fb8780a0a30 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb87dc83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb878124a40 con 0x7fb8780a1510 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb87dc83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781386c0 con 0x7fb878139250 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 0x7fb878122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 0x7fb878122da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45250/0 (socket says 172.21.15.32:45250) 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 -- 172.21.15.32:0/2606051916 learned_addr learned my addr 172.21.15.32:0/2606051916 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb877fff700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb878139250 0x7fb8781232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb8777fe700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a0a30 0x7fb878122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 -- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb878139250 msgr2=0x7fb8781232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb878139250 0x7fb8781232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 -- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a0a30 msgr2=0x7fb878122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a0a30 0x7fb878122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb86effd700 1 -- 172.21.15.32:0/2606051916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860009550 con 0x7fb8780a1510 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.018+0000 7fb877fff700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb878139250 0x7fb8781232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.019+0000 7fb86effd700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 0x7fb878122da0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb860009c40 tx=0x7fb860004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.019+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860004300 con 0x7fb8780a1510 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.019+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86001f070 con 0x7fb8780a1510 2023-09-30T22:29:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.019+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8781262f0 con 0x7fb8780a1510 2023-09-30T22:29:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.019+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8600036a0 con 0x7fb8780a1510 2023-09-30T22:29:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.019+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb878124bd0 con 0x7fb8780a1510 2023-09-30T22:29:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.020+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb860021030 con 0x7fb8780a1510 2023-09-30T22:29:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.020+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb878074ea0 con 0x7fb8780a1510 2023-09-30T22:29:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.020+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb860040930 con 0x7fb8780a1510 2023-09-30T22:29:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.025+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8600199a0 con 0x7fb8780a1510 2023-09-30T22:29:51.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.193+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa4b81a0b30 con 0x7fa4b809f650 2023-09-30T22:29:51.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.194+0000 7fa4b4ff9700 1 -- 172.21.15.32:0/518933783 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4b0011680 con 0x7fa4b809f650 2023-09-30T22:29:51.195 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 msgr2=0x7fa4b81a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 0x7fa4b81a45d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000efc0 tx=0x7fa4b000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 shutdown_connections 2023-09-30T22:29:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4b809f650 0x7fa4b81a45d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4b809e930 0x7fa4b81a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 --2- 172.21.15.32:0/518933783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4b809f240 0x7fa4b81a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 >> 172.21.15.32:0/518933783 conn(0x7fa4b80a4210 msgr2=0x7fa4b81b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 shutdown_connections 2023-09-30T22:29:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.196+0000 7fa4be3ed700 1 -- 172.21.15.32:0/518933783 wait complete. 2023-09-30T22:29:51.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.235+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb8780a28e0 con 0x7fb8780a1510 2023-09-30T22:29:51.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.235+0000 7fb874ff9700 1 -- 172.21.15.32:0/2606051916 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb860015070 con 0x7fb8780a1510 2023-09-30T22:29:51.236 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:51.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 msgr2=0x7fb878122da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 0x7fb878122da0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb860009c40 tx=0x7fb860004750 comp rx=0 tx=0).stop 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 shutdown_connections 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb878139250 0x7fb8781232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8780a0a30 0x7fb878122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 --2- 172.21.15.32:0/2606051916 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8780a1510 0x7fb878122da0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 >> 172.21.15.32:0/2606051916 conn(0x7fb8780a6550 msgr2=0x7fb8780a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 shutdown_connections 2023-09-30T22:29:51.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.237+0000 7fb87dc83700 1 -- 172.21.15.32:0/2606051916 wait complete. 2023-09-30T22:29:51.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:51.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.863+0000 7fe8b866d700 1 Processor -- start 2023-09-30T22:29:51.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.863+0000 7fe8b866d700 1 -- start start 2023-09-30T22:29:51.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b866d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 0x7fe8b009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b866d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8b009f650 0x7fe8b0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b866d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b0070940 con 0x7fe8b009f240 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b866d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b006fd60 con 0x7fe8b009f650 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b866d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b006fa80 con 0x7fe8b009e930 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6409700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 0x7fe8b009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6409700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 0x7fe8b009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45256/0 (socket says 172.21.15.32:45256) 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6409700 1 -- 172.21.15.32:0/935099719 learned_addr learned my addr 172.21.15.32:0/935099719 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6c0a700 1 --2- 172.21.15.32:0/935099719 >> v2:172.21.15.32:3300/0 conn(0x7fe8b009f650 0x7fe8b0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6c0a700 1 -- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 msgr2=0x7fe8b009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6c0a700 1 --2- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 0x7fe8b009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6c0a700 1 -- 172.21.15.32:0/935099719 >> v1:172.21.15.32:6789/0 conn(0x7fe8b009f240 legacy=0x7fe8b01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b6c0a700 1 -- 172.21.15.32:0/935099719 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b009fb60 con 0x7fe8b009f650 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b6409700 1 --2- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 0x7fe8b009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.864+0000 7fe8b5407700 1 -- 172.21.15.32:0/935099719 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758287603 0 0) 0x7fe8b0070940 con 0x7fe8b009f240 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b6c0a700 1 --2- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009f650 0x7fe8b0131f00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009d90 tx=0x7fe8a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a0ea84d8879a59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b5407700 1 -- 172.21.15.32:0/935099719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800e070 con 0x7fe8b009f650 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b5407700 1 -- 172.21.15.32:0/935099719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a800bb20 con 0x7fe8b009f650 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b5407700 1 -- 172.21.15.32:0/935099719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8012430 con 0x7fe8b009f650 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b866d700 1 -- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009f650 msgr2=0x7fe8b0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.865+0000 7fe8b866d700 1 --2- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009f650 0x7fe8b0131f00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009d90 tx=0x7fe8a800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:51.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.866+0000 7fe8b866d700 1 -- 172.21.15.32:0/935099719 shutdown_connections 2023-09-30T22:29:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.866+0000 7fe8b866d700 1 --2- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009f650 0x7fe8b0131f00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.866+0000 7fe8b866d700 1 --2- 172.21.15.32:0/935099719 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009e930 0x7fe8b009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.866+0000 7fe8b866d700 1 -- 172.21.15.32:0/935099719 >> 172.21.15.32:0/935099719 conn(0x7fe8b00a4210 msgr2=0x7fe8b00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.866+0000 7fe8b866d700 1 -- 172.21.15.32:0/935099719 shutdown_connections 2023-09-30T22:29:51.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.866+0000 7fe8b866d700 1 -- 172.21.15.32:0/935099719 wait complete. 2023-09-30T22:29:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 Processor -- start 2023-09-30T22:29:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 -- start start 2023-09-30T22:29:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 0x7fe8b01c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009f240 0x7fe8b01c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8b009f650 0x7fe8b01c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b01a6cd0 con 0x7fe8b009e930 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b006d6b0 con 0x7fe8b009f240 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b866d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b01a8310 con 0x7fe8b009f650 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b6409700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 0x7fe8b01c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b5c08700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009f240 0x7fe8b01c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b6c0a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8b009f650 0x7fe8b01c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b6409700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 0x7fe8b01c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39656/0 (socket says 172.21.15.32:39656) 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b5c08700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009f240 0x7fe8b01c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45268/0 (socket says 172.21.15.32:45268) 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.867+0000 7fe8b6409700 1 -- 172.21.15.32:0/2036393017 learned_addr learned my addr 172.21.15.32:0/2036393017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b6409700 1 -- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8b009f650 msgr2=0x7fe8b01c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b6409700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8b009f650 0x7fe8b01c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b6409700 1 -- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009f240 msgr2=0x7fe8b01c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b6409700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009f240 0x7fe8b01c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b6409700 1 -- 172.21.15.32:0/2036393017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac009550 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b6409700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 0x7fe8b01c8000 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00ec10 tx=0x7fe8ac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00c850 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac00c9b0 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a8009a70 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00cd40 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.868+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b00a0190 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.869+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8ac013470 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.869+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0071d80 con 0x7fe8b009e930 2023-09-30T22:29:51.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.869+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe8ac03fc20 con 0x7fe8b009e930 2023-09-30T22:29:51.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:51.874+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8ac013aa0 con 0x7fe8b009e930 2023-09-30T22:29:52.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:52.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.077+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe8b0074ea0 con 0x7fe8b009e930 2023-09-30T22:29:52.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.078+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2036393017 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8ac021030 con 0x7fe8b009e930 2023-09-30T22:29:52.079 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 msgr2=0x7fe8b01c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 0x7fe8b01c8000 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00ec10 tx=0x7fe8ac00ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 shutdown_connections 2023-09-30T22:29:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8b009f650 0x7fe8b01c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8b009e930 0x7fe8b01c8000 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 --2- 172.21.15.32:0/2036393017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8b009f240 0x7fe8b01c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 >> 172.21.15.32:0/2036393017 conn(0x7fe8b00a4210 msgr2=0x7fe8b00a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 shutdown_connections 2023-09-30T22:29:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.080+0000 7fe8b866d700 1 -- 172.21.15.32:0/2036393017 wait complete. 2023-09-30T22:29:52.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:52.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.157+0000 7f87e21eb700 1 Processor -- start 2023-09-30T22:29:52.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.157+0000 7f87e21eb700 1 -- start start 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87e21eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87e21eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 0x7f87dc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87e21eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc09b120 con 0x7f87dc09d6f0 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87e21eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc06b460 con 0x7f87dc139170 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87e21eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc09a7e0 con 0x7f87dc09e1d0 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87dbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 0x7f87dc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87daffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87dbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 0x7f87dc13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39672/0 (socket says 172.21.15.32:39672) 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87daffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45278/0 (socket says 172.21.15.32:45278) 2023-09-30T22:29:52.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87dbfff700 1 -- 172.21.15.32:0/2196676168 learned_addr learned my addr 172.21.15.32:0/2196676168 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87db7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f87dc09d6f0 0x7f87dc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55752/0 (socket says 172.21.15.32:55752) 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87daffd700 1 -- 172.21.15.32:0/2196676168 >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 msgr2=0x7f87dc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87daffd700 1 --2- 172.21.15.32:0/2196676168 >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 0x7f87dc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87daffd700 1 -- 172.21.15.32:0/2196676168 >> v1:172.21.15.32:6789/0 conn(0x7f87dc09d6f0 legacy=0x7f87dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87daffd700 1 -- 172.21.15.32:0/2196676168 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc09a500 con 0x7f87dc09e1d0 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87dbfff700 1 --2- 172.21.15.32:0/2196676168 >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 0x7f87dc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.158+0000 7f87da7fc700 1 -- 172.21.15.32:0/2196676168 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230308373 0 0) 0x7f87dc09b120 con 0x7f87dc09d6f0 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87daffd700 1 --2- 172.21.15.32:0/2196676168 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc138c30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009870 tx=0x7f87d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54008b4c0f7184bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87da7fc700 1 -- 172.21.15.32:0/2196676168 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0011830 con 0x7f87dc09e1d0 2023-09-30T22:29:52.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87da7fc700 1 -- 172.21.15.32:0/2196676168 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0011990 con 0x7f87dc09e1d0 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87da7fc700 1 -- 172.21.15.32:0/2196676168 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0011d20 con 0x7f87dc09e1d0 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 -- 172.21.15.32:0/2196676168 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 msgr2=0x7f87dc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2196676168 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc138c30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009870 tx=0x7f87d0009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 -- 172.21.15.32:0/2196676168 shutdown_connections 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2196676168 >> v2:172.21.15.32:3300/0 conn(0x7f87dc139170 0x7f87dc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2196676168 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc138c30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 -- 172.21.15.32:0/2196676168 >> 172.21.15.32:0/2196676168 conn(0x7f87dc066f50 msgr2=0x7f87dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 -- 172.21.15.32:0/2196676168 shutdown_connections 2023-09-30T22:29:52.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.159+0000 7f87e21eb700 1 -- 172.21.15.32:0/2196676168 wait complete. 2023-09-30T22:29:52.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.160+0000 7f87e21eb700 1 Processor -- start 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 -- start start 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 0x7f87dc121730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87dc139170 0x7f87dc1221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc123340 con 0x7f87dc09d6f0 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc137620 con 0x7f87dc09e1d0 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87e21eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1249e0 con 0x7f87dc139170 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 0x7f87dc121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 0x7f87dc121730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39674/0 (socket says 172.21.15.32:39674) 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87db7fe700 1 -- 172.21.15.32:0/2509507086 learned_addr learned my addr 172.21.15.32:0/2509507086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:52.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87daffd700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.161+0000 7f87dbfff700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87dc139170 0x7f87dc1221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87db7fe700 1 -- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87dc139170 msgr2=0x7f87dc1221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87db7fe700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87dc139170 0x7f87dc1221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87db7fe700 1 -- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 msgr2=0x7f87dc121c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87db7fe700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc121c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87db7fe700 1 -- 172.21.15.32:0/2509507086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4009b30 con 0x7f87dc09d6f0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87db7fe700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 0x7f87dc121730 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00c520 tx=0x7f87cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc004c80 con 0x7f87dc09d6f0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc004e00 con 0x7f87dc09d6f0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d00094a0 con 0x7f87dc09d6f0 2023-09-30T22:29:52.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc007310 con 0x7f87dc09d6f0 2023-09-30T22:29:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.162+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc09c710 con 0x7f87dc09d6f0 2023-09-30T22:29:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.163+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87cc007470 con 0x7f87dc09d6f0 2023-09-30T22:29:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.163+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc074ea0 con 0x7f87dc09d6f0 2023-09-30T22:29:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.163+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f87cc03f300 con 0x7f87dc09d6f0 2023-09-30T22:29:52.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.168+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87cc0133c0 con 0x7f87dc09d6f0 2023-09-30T22:29:52.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:52.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.351+0000 7f574b7ac700 1 Processor -- start 2023-09-30T22:29:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.351+0000 7f574b7ac700 1 -- start start 2023-09-30T22:29:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f574b7ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5744130750 0x7f5744130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f574b7ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574409e250 0x7f57441384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f574b7ac700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574409c190 con 0x7f574409d6f0 2023-09-30T22:29:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f574b7ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574409b090 con 0x7f5744130750 2023-09-30T22:29:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f574b7ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f574409a6c0 con 0x7f574409e250 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f5749548700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5744130750 0x7f5744130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f5748d47700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f574409d6f0 0x7f574409db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55758/0 (socket says 172.21.15.32:55758) 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f5748d47700 1 -- 172.21.15.32:0/3605550231 learned_addr learned my addr 172.21.15.32:0/3605550231 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.352+0000 7f5749d49700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574409e250 0x7f57441384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f5749548700 1 -- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574409e250 msgr2=0x7f57441384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f5749548700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574409e250 0x7f57441384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f5749548700 1 -- 172.21.15.32:0/3605550231 >> v1:172.21.15.32:6789/0 conn(0x7f574409d6f0 legacy=0x7f574409db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f573bfff700 1 -- 172.21.15.32:0/3605550231 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152388064 0 0) 0x7f574409c190 con 0x7f574409d6f0 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f5749548700 1 -- 172.21.15.32:0/3605550231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744131060 con 0x7f5744130750 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f5749d49700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574409e250 0x7f57441384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f5749548700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5744130750 0x7f5744130b20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5740009870 tx=0x7f5740009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=803ffb96d03f0c19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f573bfff700 1 -- 172.21.15.32:0/3605550231 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740011830 con 0x7f5744130750 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f573bfff700 1 -- 172.21.15.32:0/3605550231 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5740011990 con 0x7f5744130750 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.353+0000 7f573bfff700 1 -- 172.21.15.32:0/3605550231 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740011d20 con 0x7f5744130750 2023-09-30T22:29:52.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 -- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5744130750 msgr2=0x7f5744130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5744130750 0x7f5744130b20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5740009870 tx=0x7f5740009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 -- 172.21.15.32:0/3605550231 shutdown_connections 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f574409e250 0x7f57441384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 --2- 172.21.15.32:0/3605550231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5744130750 0x7f5744130b20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 -- 172.21.15.32:0/3605550231 >> 172.21.15.32:0/3605550231 conn(0x7f574409f470 msgr2=0x7f57440a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 -- 172.21.15.32:0/3605550231 shutdown_connections 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.354+0000 7f574b7ac700 1 -- 172.21.15.32:0/3605550231 wait complete. 2023-09-30T22:29:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.355+0000 7f574b7ac700 1 Processor -- start 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.355+0000 7f574b7ac700 1 -- start start 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f574b7ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 0x7f57441c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f574b7ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574409e250 0x7f57441c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f574b7ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5744130750 0x7f57441c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f574b7ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441384f0 con 0x7f574409d6f0 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f574b7ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574409b2b0 con 0x7f574409e250 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f574b7ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5744138fd0 con 0x7f5744130750 2023-09-30T22:29:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f5749548700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 0x7f57441c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f5749548700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 0x7f57441c54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39682/0 (socket says 172.21.15.32:39682) 2023-09-30T22:29:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f5749548700 1 -- 172.21.15.32:0/4118589351 learned_addr learned my addr 172.21.15.32:0/4118589351 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f5748d47700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574409e250 0x7f57441c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f5749d49700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5744130750 0x7f57441c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.356+0000 7f5749548700 1 -- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5744130750 msgr2=0x7f57441c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5749548700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5744130750 0x7f57441c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5749548700 1 -- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574409e250 msgr2=0x7f57441c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5749548700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574409e250 0x7f57441c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5749548700 1 -- 172.21.15.32:0/4118589351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c009a70 con 0x7f574409d6f0 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5748d47700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574409e250 0x7f57441c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5749d49700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5744130750 0x7f57441c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.357+0000 7f5749548700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 0x7f57441c54f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5740009840 tx=0x7f5740004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.359+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400044d0 con 0x7f574409d6f0 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.359+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57400094a0 con 0x7f574409d6f0 2023-09-30T22:29:52.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.359+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57400036a0 con 0x7f574409d6f0 2023-09-30T22:29:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.359+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740003a50 con 0x7f574409d6f0 2023-09-30T22:29:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.359+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57441375f0 con 0x7f574409d6f0 2023-09-30T22:29:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.361+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57441377e0 con 0x7f574409d6f0 2023-09-30T22:29:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.361+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5740003bb0 con 0x7f574409d6f0 2023-09-30T22:29:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.361+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5740040650 con 0x7f574409d6f0 2023-09-30T22:29:52.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.366+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f574001a420 con 0x7f574409d6f0 2023-09-30T22:29:52.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.385+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f87dc09f420 con 0x7f87dc09d6f0 2023-09-30T22:29:52.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.386+0000 7f87c3fff700 1 -- 172.21.15.32:0/2509507086 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87cc011360 con 0x7f87dc09d6f0 2023-09-30T22:29:52.387 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 msgr2=0x7f87dc121730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 0x7f87dc121730 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00c520 tx=0x7f87cc00c830 comp rx=0 tx=0).stop 2023-09-30T22:29:52.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 shutdown_connections 2023-09-30T22:29:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f87dc139170 0x7f87dc1221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87dc09d6f0 0x7f87dc121730 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 --2- 172.21.15.32:0/2509507086 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87dc09e1d0 0x7f87dc121c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 >> 172.21.15.32:0/2509507086 conn(0x7f87dc066f50 msgr2=0x7f87dc070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 shutdown_connections 2023-09-30T22:29:52.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.388+0000 7f87e21eb700 1 -- 172.21.15.32:0/2509507086 wait complete. 2023-09-30T22:29:52.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.396+0000 7fd94672e700 1 Processor -- start 2023-09-30T22:29:52.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd94672e700 1 -- start start 2023-09-30T22:29:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd94672e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009d6f0 0x7fd94009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd94672e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009e000 0x7fd940130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd94672e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94009f130 con 0x7fd94009e410 2023-09-30T22:29:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd94672e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94009f520 con 0x7fd94009d6f0 2023-09-30T22:29:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd94672e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd94009e930 con 0x7fd94009e000 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd944ccb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd94009e410 0x7fd940130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55762/0 (socket says 172.21.15.32:55762) 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd944ccb700 1 -- 172.21.15.32:0/2543586152 learned_addr learned my addr 172.21.15.32:0/2543586152 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd93ffff700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009d6f0 0x7fd94009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.397+0000 7fd93f7fe700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009e000 0x7fd940130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93ffff700 1 -- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009e000 msgr2=0x7fd940130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93ffff700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009e000 0x7fd940130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93ffff700 1 -- 172.21.15.32:0/2543586152 >> v1:172.21.15.32:6789/0 conn(0x7fd94009e410 legacy=0x7fd940130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93ffff700 1 -- 172.21.15.32:0/2543586152 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94009b2d0 con 0x7fd94009d6f0 2023-09-30T22:29:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93f7fe700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009e000 0x7fd940130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93ffff700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009d6f0 0x7fd94009dac0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd930009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b573c84f32fb5ccc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93effd700 1 -- 172.21.15.32:0/2543586152 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930011830 con 0x7fd94009d6f0 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93effd700 1 -- 172.21.15.32:0/2543586152 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930011990 con 0x7fd94009d6f0 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.398+0000 7fd93effd700 1 -- 172.21.15.32:0/2543586152 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930011d20 con 0x7fd94009d6f0 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 -- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009d6f0 msgr2=0x7fd94009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009d6f0 0x7fd94009dac0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd930009870 tx=0x7fd930009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 -- 172.21.15.32:0/2543586152 shutdown_connections 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009d6f0 0x7fd94009dac0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 --2- 172.21.15.32:0/2543586152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009e000 0x7fd940130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 -- 172.21.15.32:0/2543586152 >> 172.21.15.32:0/2543586152 conn(0x7fd940066f50 msgr2=0x7fd94009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 -- 172.21.15.32:0/2543586152 shutdown_connections 2023-09-30T22:29:52.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.399+0000 7fd94672e700 1 -- 172.21.15.32:0/2543586152 wait complete. 2023-09-30T22:29:52.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.400+0000 7fd94672e700 1 Processor -- start 2023-09-30T22:29:52.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.400+0000 7fd94672e700 1 -- start start 2023-09-30T22:29:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd94672e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009d6f0 0x7fd94012db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd94672e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 0x7fd94012e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd94672e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009e410 0x7fd94012e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd94672e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94012ff40 con 0x7fd94009e000 2023-09-30T22:29:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd94672e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940131670 con 0x7fd94009e410 2023-09-30T22:29:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd94672e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400a1f20 con 0x7fd94009d6f0 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 0x7fd94012e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd944ccb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009e410 0x7fd94012e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 0x7fd94012e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39696/0 (socket says 172.21.15.32:39696) 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 -- 172.21.15.32:0/862373855 learned_addr learned my addr 172.21.15.32:0/862373855 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93ffff700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009d6f0 0x7fd94012db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 -- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009d6f0 msgr2=0x7fd94012db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009d6f0 0x7fd94012db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 -- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009e410 msgr2=0x7fd94012e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009e410 0x7fd94012e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93f7fe700 1 -- 172.21.15.32:0/862373855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340094a0 con 0x7fd94009e000 2023-09-30T22:29:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd93ffff700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009d6f0 0x7fd94012db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.401+0000 7fd944ccb700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009e410 0x7fd94012e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.402+0000 7fd93f7fe700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 0x7fd94012e080 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd93400ebb0 tx=0x7fd93400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.402+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400c850 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.402+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93400c9b0 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.402+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400cd40 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.402+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9300094a0 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.402+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9401308a0 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.403+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd934013470 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.403+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd93403faf0 con 0x7fd94009e000 2023-09-30T22:29:52.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.403+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd940074ea0 con 0x7fd94009e000 2023-09-30T22:29:52.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.408+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd934017020 con 0x7fd94009e000 2023-09-30T22:29:52.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.573+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5744132a10 con 0x7f574409d6f0 2023-09-30T22:29:52.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.573+0000 7f5739ffb700 1 -- 172.21.15.32:0/4118589351 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5740028030 con 0x7f574409d6f0 2023-09-30T22:29:52.575 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.575+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 msgr2=0x7f57441c54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.575+0000 7f574b7ac700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 0x7f57441c54f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5740009840 tx=0x7f5740004750 comp rx=0 tx=0).stop 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.575+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 shutdown_connections 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.575+0000 7f574b7ac700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5744130750 0x7f57441c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.575+0000 7f574b7ac700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574409d6f0 0x7f57441c54f0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.575+0000 7f574b7ac700 1 --2- 172.21.15.32:0/4118589351 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f574409e250 0x7f57441c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.576+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 >> 172.21.15.32:0/4118589351 conn(0x7f574409f470 msgr2=0x7f57440715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.576+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 shutdown_connections 2023-09-30T22:29:52.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.576+0000 7f574b7ac700 1 -- 172.21.15.32:0/4118589351 wait complete. 2023-09-30T22:29:52.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.612+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd94006c360 con 0x7fd94009e000 2023-09-30T22:29:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.613+0000 7fd93cff9700 1 -- 172.21.15.32:0/862373855 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd93401b3b0 con 0x7fd94009e000 2023-09-30T22:29:52.614 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 msgr2=0x7fd94012e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 0x7fd94012e080 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd93400ebb0 tx=0x7fd93400eec0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 shutdown_connections 2023-09-30T22:29:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd94009e410 0x7fd94012e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd94009d6f0 0x7fd94012db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 --2- 172.21.15.32:0/862373855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd94009e000 0x7fd94012e080 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 >> 172.21.15.32:0/862373855 conn(0x7fd940066f50 msgr2=0x7fd94012a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 shutdown_connections 2023-09-30T22:29:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:52.615+0000 7fd94672e700 1 -- 172.21.15.32:0/862373855 wait complete. 2023-09-30T22:29:52.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:53.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.242+0000 7f4ef40a4700 1 Processor -- start 2023-09-30T22:29:53.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.242+0000 7f4ef40a4700 1 -- start start 2023-09-30T22:29:53.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef40a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef40a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec139170 0x7f4eec13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef40a4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec09d2f0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef40a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec09b0f0 con 0x7f4eec139170 2023-09-30T22:29:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef40a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec09a810 con 0x7f4eec09e1d0 2023-09-30T22:29:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef1e40700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4eec09d6f0 0x7f4eec09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55766/0 (socket says 172.21.15.32:55766) 2023-09-30T22:29:53.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef1e40700 1 -- 172.21.15.32:0/3792855403 learned_addr learned my addr 172.21.15.32:0/3792855403 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef2641700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec139170 0x7f4eec13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.243+0000 7f4ef163f700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef2641700 1 -- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 msgr2=0x7f4eec138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef0e3e700 1 -- 172.21.15.32:0/3792855403 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100990982 0 0) 0x7f4eec09d2f0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef2641700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef2641700 1 -- 172.21.15.32:0/3792855403 >> v1:172.21.15.32:6789/0 conn(0x7f4eec09d6f0 legacy=0x7f4eec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef2641700 1 -- 172.21.15.32:0/3792855403 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec09a470 con 0x7f4eec139170 2023-09-30T22:29:53.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef163f700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef2641700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec139170 0x7f4eec13b550 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009df0 tx=0x7f4ee400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a98619a49fe55221 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef0e3e700 1 -- 172.21.15.32:0/3792855403 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400e070 con 0x7f4eec139170 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef0e3e700 1 -- 172.21.15.32:0/3792855403 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee400bae0 con 0x7f4eec139170 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.244+0000 7f4ef0e3e700 1 -- 172.21.15.32:0/3792855403 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400be90 con 0x7f4eec139170 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 -- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec139170 msgr2=0x7f4eec13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec139170 0x7f4eec13b550 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009df0 tx=0x7f4ee400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 -- 172.21.15.32:0/3792855403 shutdown_connections 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec139170 0x7f4eec13b550 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/3792855403 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 -- 172.21.15.32:0/3792855403 >> 172.21.15.32:0/3792855403 conn(0x7f4eec066f50 msgr2=0x7f4eec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 -- 172.21.15.32:0/3792855403 shutdown_connections 2023-09-30T22:29:53.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.245+0000 7f4ef40a4700 1 -- 172.21.15.32:0/3792855403 wait complete. 2023-09-30T22:29:53.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.246+0000 7f4ef40a4700 1 Processor -- start 2023-09-30T22:29:53.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.246+0000 7f4ef40a4700 1 -- start start 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef40a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 0x7f4eec121620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef40a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec121b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef40a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eec139170 0x7f4eec1220a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef40a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec124140 con 0x7f4eec139170 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef40a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec124e60 con 0x7f4eec09d6f0 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef40a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec09bfb0 con 0x7f4eec09e1d0 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef1e40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 0x7f4eec121620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef1e40700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 0x7f4eec121620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45344/0 (socket says 172.21.15.32:45344) 2023-09-30T22:29:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef1e40700 1 -- 172.21.15.32:0/1049921430 learned_addr learned my addr 172.21.15.32:0/1049921430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef2641700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eec139170 0x7f4eec1220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.247+0000 7f4ef163f700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec121b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef1e40700 1 -- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 msgr2=0x7f4eec121b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef1e40700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec121b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef1e40700 1 -- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eec139170 msgr2=0x7f4eec1220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef1e40700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eec139170 0x7f4eec1220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef1e40700 1 -- 172.21.15.32:0/1049921430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00094a0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef1e40700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 0x7f4eec121620 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800c520 tx=0x7f4ee800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8004c80 con 0x7f4eec09d6f0 2023-09-30T22:29:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee8004de0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee4009a70 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8007310 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.248+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec09c690 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.249+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ee8007470 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.249+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ee803f1f0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.249+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec125990 con 0x7f4eec09d6f0 2023-09-30T22:29:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.254+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ee80123b0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.410 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.460+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4eec074ea0 con 0x7f4eec09d6f0 2023-09-30T22:29:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.460+0000 7f4ede7fc700 1 -- 172.21.15.32:0/1049921430 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ee8011360 con 0x7f4eec09d6f0 2023-09-30T22:29:53.462 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 msgr2=0x7f4eec121620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 0x7f4eec121620 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800c520 tx=0x7f4ee800c830 comp rx=0 tx=0).stop 2023-09-30T22:29:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 shutdown_connections 2023-09-30T22:29:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4eec139170 0x7f4eec1220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4eec09d6f0 0x7f4eec121620 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 --2- 172.21.15.32:0/1049921430 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4eec09e1d0 0x7f4eec121b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 >> 172.21.15.32:0/1049921430 conn(0x7f4eec066f50 msgr2=0x7f4eec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 shutdown_connections 2023-09-30T22:29:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.463+0000 7f4ef40a4700 1 -- 172.21.15.32:0/1049921430 wait complete. 2023-09-30T22:29:53.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 Processor -- start 2023-09-30T22:29:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 -- start start 2023-09-30T22:29:53.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6724133a80 0x7f6724133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672412ff20 0x7f6724130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 0x7f672413a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672406c570 con 0x7f6724133a80 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672406bc00 con 0x7f6724130870 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.548+0000 7f672a513700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67240a0040 con 0x7f672412ff20 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6724133a80 0x7f6724133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f67237fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672412ff20 0x7f6724130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6728ab0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 0x7f672413a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f67237fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672412ff20 0x7f6724130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57392/0 (socket says 172.21.15.32:57392) 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6724133a80 0x7f6724133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39714/0 (socket says 172.21.15.32:39714) 2023-09-30T22:29:53.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6728ab0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 0x7f672413a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45356/0 (socket says 172.21.15.32:45356) 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f67237fe700 1 -- 172.21.15.32:0/1014707078 learned_addr learned my addr 172.21.15.32:0/1014707078 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 -- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672412ff20 msgr2=0x7f6724130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672412ff20 0x7f6724130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 -- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 msgr2=0x7f672413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 0x7f672413a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 -- 172.21.15.32:0/1014707078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724133860 con 0x7f6724133a80 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6728ab0700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 0x7f672413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6723fff700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f6724133e50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f670c009840 tx=0x7f670c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d9a32e316c070f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6722ffd700 1 -- 172.21.15.32:0/1014707078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c011830 con 0x7f6724133a80 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6722ffd700 1 -- 172.21.15.32:0/1014707078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c011990 con 0x7f6724133a80 2023-09-30T22:29:53.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.549+0000 7f6722ffd700 1 -- 172.21.15.32:0/1014707078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c011d20 con 0x7f6724133a80 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 -- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 msgr2=0x7f6724133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f6724133e50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f670c009840 tx=0x7f670c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 -- 172.21.15.32:0/1014707078 shutdown_connections 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6724130870 0x7f672413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f6724133e50 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 --2- 172.21.15.32:0/1014707078 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672412ff20 0x7f6724130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 -- 172.21.15.32:0/1014707078 >> 172.21.15.32:0/1014707078 conn(0x7f67240a4210 msgr2=0x7f67240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 -- 172.21.15.32:0/1014707078 shutdown_connections 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.550+0000 7f672a513700 1 -- 172.21.15.32:0/1014707078 wait complete. 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.551+0000 7f672a513700 1 Processor -- start 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.551+0000 7f672a513700 1 -- start start 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f672a513700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 0x7f672409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f672a513700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6724130870 0x7f672409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f672a513700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f672409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f672a513700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672411f970 con 0x7f6724133a80 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f672a513700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67241208a0 con 0x7f672412ff20 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f672a513700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f672412d6a0 con 0x7f6724130870 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f6723fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 0x7f672409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f67237fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6724130870 0x7f672409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f6723fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 0x7f672409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45372/0 (socket says 172.21.15.32:45372) 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f6723fff700 1 -- 172.21.15.32:0/2254251150 learned_addr learned my addr 172.21.15.32:0/2254251150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f6728ab0700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f672409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f67237fe700 1 -- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 msgr2=0x7f672409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f67237fe700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 0x7f672409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f67237fe700 1 -- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 msgr2=0x7f672409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f67237fe700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f672409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f67237fe700 1 -- 172.21.15.32:0/2254251150 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009a70 con 0x7f6724130870 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.552+0000 7f6723fff700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 0x7f672409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f67237fe700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6724130870 0x7f672409c9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f671400e8e0 tx=0x7f671400ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714013680 con 0x7f6724130870 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67140137e0 con 0x7f6724130870 2023-09-30T22:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c0094a0 con 0x7f6724130870 2023-09-30T22:29:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714013b70 con 0x7f6724130870 2023-09-30T22:29:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672409e0a0 con 0x7f6724130870 2023-09-30T22:29:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.553+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6714013cd0 con 0x7f6724130870 2023-09-30T22:29:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.554+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f671403f660 con 0x7f6724130870 2023-09-30T22:29:53.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.554+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6724074ea0 con 0x7f6724130870 2023-09-30T22:29:53.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.559+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6714017080 con 0x7f6724130870 2023-09-30T22:29:53.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:53.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.731+0000 7fc5adaf6700 1 Processor -- start 2023-09-30T22:29:53.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.731+0000 7fc5adaf6700 1 -- start start 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5adaf6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5adaf6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a80a0d90 0x7fc5a80a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5adaf6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80a2a20 con 0x7fc5a81338a0 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5adaf6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a809e690 con 0x7fc5a80a09c0 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5adaf6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80a1cc0 con 0x7fc5a80a0d90 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a806c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45376/0 (socket says 172.21.15.32:45376) 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5a77fe700 1 -- 172.21.15.32:0/32013199 learned_addr learned my addr 172.21.15.32:0/32013199 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.732+0000 7fc5a6ffd700 1 --2- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a80a0d90 0x7fc5a80a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a77fe700 1 -- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a80a0d90 msgr2=0x7fc5a80a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a77fe700 1 --2- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a80a0d90 0x7fc5a80a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a67fc700 1 -- 172.21.15.32:0/32013199 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598160087 0 0) 0x7fc5a0001110 con 0x7fc5a81338a0 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a77fe700 1 -- 172.21.15.32:0/32013199 >> v1:172.21.15.32:6789/0 conn(0x7fc5a81338a0 legacy=0x7fc5a8133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a77fe700 1 -- 172.21.15.32:0/32013199 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80a1f40 con 0x7fc5a80a09c0 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a77fe700 1 --2- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a806c5d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc598009e80 tx=0x7fc59800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b2261c0501abc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a67fc700 1 -- 172.21.15.32:0/32013199 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800e070 con 0x7fc5a80a09c0 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a67fc700 1 -- 172.21.15.32:0/32013199 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59800b910 con 0x7fc5a80a09c0 2023-09-30T22:29:53.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.733+0000 7fc5a67fc700 1 -- 172.21.15.32:0/32013199 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59800bcc0 con 0x7fc5a80a09c0 2023-09-30T22:29:53.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 -- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 msgr2=0x7fc5a806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a806c5d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc598009e80 tx=0x7fc59800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 -- 172.21.15.32:0/32013199 shutdown_connections 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a806c5d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/32013199 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a80a0d90 0x7fc5a80a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 -- 172.21.15.32:0/32013199 >> 172.21.15.32:0/32013199 conn(0x7fc5a80a60c0 msgr2=0x7fc5a80a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 -- 172.21.15.32:0/32013199 shutdown_connections 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.734+0000 7fc5adaf6700 1 -- 172.21.15.32:0/32013199 wait complete. 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.735+0000 7fc5adaf6700 1 Processor -- start 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.735+0000 7fc5adaf6700 1 -- start start 2023-09-30T22:29:53.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5adaf6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a811f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5adaf6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80a0d90 0x7fc5a811f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5adaf6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a81338a0 0x7fc5a81c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5adaf6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a811ecd0 con 0x7fc5a80a0d90 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5adaf6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8122410 con 0x7fc5a80a09c0 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5adaf6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8120400 con 0x7fc5a81338a0 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a811f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5a77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a811f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45382/0 (socket says 172.21.15.32:45382) 2023-09-30T22:29:53.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5a77fe700 1 -- 172.21.15.32:0/3859137396 learned_addr learned my addr 172.21.15.32:0/3859137396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5a6ffd700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80a0d90 0x7fc5a811f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.736+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a81338a0 0x7fc5a81c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a7fff700 1 -- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 msgr2=0x7fc5a811f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a811f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a7fff700 1 -- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80a0d90 msgr2=0x7fc5a811f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80a0d90 0x7fc5a811f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a7fff700 1 -- 172.21.15.32:0/3859137396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009b30 con 0x7fc5a81338a0 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a77fe700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a811f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a6ffd700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80a0d90 0x7fc5a811f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5a7fff700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a81338a0 0x7fc5a81c8000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c750 tx=0x7fc5a000ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00054e0 con 0x7fc5a81338a0 2023-09-30T22:29:53.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0001d30 con 0x7fc5a81338a0 2023-09-30T22:29:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80a3fc0 con 0x7fc5a81338a0 2023-09-30T22:29:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00038c0 con 0x7fc5a81338a0 2023-09-30T22:29:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.737+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a8121d70 con 0x7fc5a81338a0 2023-09-30T22:29:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.738+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5a0003a20 con 0x7fc5a81338a0 2023-09-30T22:29:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.738+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc5a003e810 con 0x7fc5a81338a0 2023-09-30T22:29:53.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.739+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a811fcd0 con 0x7fc5a81338a0 2023-09-30T22:29:53.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.743+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5a001b3b0 con 0x7fc5a81338a0 2023-09-30T22:29:53.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.778+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f672406b250 con 0x7f6724130870 2023-09-30T22:29:53.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.778+0000 7f6720ff9700 1 -- 172.21.15.32:0/2254251150 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6714021020 con 0x7f6724130870 2023-09-30T22:29:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6724130870 msgr2=0x7f672409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6724130870 0x7f672409c9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f671400e8e0 tx=0x7f671400ebf0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 shutdown_connections 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6724133a80 0x7f672409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672412ff20 0x7f672409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 --2- 172.21.15.32:0/2254251150 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6724130870 0x7f672409c9c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 >> 172.21.15.32:0/2254251150 conn(0x7f67240a4210 msgr2=0x7f672409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 shutdown_connections 2023-09-30T22:29:53.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.781+0000 7f672a513700 1 -- 172.21.15.32:0/2254251150 wait complete. 2023-09-30T22:29:53.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.787+0000 7fc90fc12700 1 Processor -- start 2023-09-30T22:29:53.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.787+0000 7fc90fc12700 1 -- start start 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90fc12700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc90809e930 0x7fc90809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90fc12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90fc12700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc908070940 con 0x7fc90809f650 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90fc12700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90806fd60 con 0x7fc90809e930 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90fc12700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90806fa80 con 0x7fc90809f240 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90d9ae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc90809e930 0x7fc90809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90e1af700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc90809f650 0x7fc908131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55776/0 (socket says 172.21.15.32:55776) 2023-09-30T22:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90e1af700 1 -- 172.21.15.32:0/1301413449 learned_addr learned my addr 172.21.15.32:0/1301413449 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:53.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90d1ad700 1 --2- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90d9ae700 1 -- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 msgr2=0x7fc9081317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.788+0000 7fc90d9ae700 1 --2- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.789+0000 7fc90d9ae700 1 -- 172.21.15.32:0/1301413449 >> v1:172.21.15.32:6789/0 conn(0x7fc90809f650 legacy=0x7fc908131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.789+0000 7fc90d9ae700 1 -- 172.21.15.32:0/1301413449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90809fb60 con 0x7fc90809e930 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.789+0000 7fc90d9ae700 1 --2- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc90809ed00 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc904009840 tx=0x7fc904009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acfe18ec083cde7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.789+0000 7fc90c9ac700 1 -- 172.21.15.32:0/1301413449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904011830 con 0x7fc90809e930 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.789+0000 7fc90c9ac700 1 -- 172.21.15.32:0/1301413449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904011990 con 0x7fc90809e930 2023-09-30T22:29:53.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.789+0000 7fc90c9ac700 1 -- 172.21.15.32:0/1301413449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904011d20 con 0x7fc90809e930 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 -- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 msgr2=0x7fc90809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 --2- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc90809ed00 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc904009840 tx=0x7fc904009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 -- 172.21.15.32:0/1301413449 shutdown_connections 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 --2- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc90809ed00 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 --2- 172.21.15.32:0/1301413449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 -- 172.21.15.32:0/1301413449 >> 172.21.15.32:0/1301413449 conn(0x7fc9080a4210 msgr2=0x7fc9080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 -- 172.21.15.32:0/1301413449 shutdown_connections 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.790+0000 7fc90fc12700 1 -- 172.21.15.32:0/1301413449 wait complete. 2023-09-30T22:29:53.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.792+0000 7fc90fc12700 1 Processor -- start 2023-09-30T22:29:53.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.792+0000 7fc90fc12700 1 -- start start 2023-09-30T22:29:53.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.792+0000 7fc90fc12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc9081a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.792+0000 7fc90fc12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.792+0000 7fc90fc12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809f650 0x7fc9081a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90fc12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9081a7010 con 0x7fc90809e930 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90fc12700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9081a5930 con 0x7fc90809f240 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d9ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc9081a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d9ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc9081a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39756/0 (socket says 172.21.15.32:39756) 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d9ae700 1 -- 172.21.15.32:0/3474038536 learned_addr learned my addr 172.21.15.32:0/3474038536 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d1ad700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90e1af700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809f650 0x7fc9081a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9081a74c0 con 0x7fc90809f650 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d1ad700 1 -- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809f650 msgr2=0x7fc9081a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d1ad700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809f650 0x7fc9081a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d1ad700 1 -- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 msgr2=0x7fc9081a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d1ad700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc9081a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d1ad700 1 -- 172.21.15.32:0/3474038536 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040094a0 con 0x7fc90809f240 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90d9ae700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc9081a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.793+0000 7fc90e1af700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809f650 0x7fc9081a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.794+0000 7fc90d1ad700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081a4260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800e930 tx=0x7fc8f800ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.794+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8013680 con 0x7fc90809f240 2023-09-30T22:29:53.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.794+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f80137e0 con 0x7fc90809f240 2023-09-30T22:29:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.794+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8013b50 con 0x7fc90809f240 2023-09-30T22:29:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.794+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9080a0190 con 0x7fc90809f240 2023-09-30T22:29:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.794+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc908071d20 con 0x7fc90809f240 2023-09-30T22:29:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.795+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9081a0b30 con 0x7fc90809f240 2023-09-30T22:29:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.795+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc8f8013cb0 con 0x7fc90809f240 2023-09-30T22:29:53.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.796+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc8f803fe80 con 0x7fc90809f240 2023-09-30T22:29:53.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:53.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.804+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8f8011350 con 0x7fc90809f240 2023-09-30T22:29:53.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.973+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc5a80a2a20 con 0x7fc5a81338a0 2023-09-30T22:29:53.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.973+0000 7fc58ffff700 1 -- 172.21.15.32:0/3859137396 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5a80a2a20 con 0x7fc5a81338a0 2023-09-30T22:29:53.974 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a81338a0 msgr2=0x7fc5a81c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a81338a0 0x7fc5a81c8000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c750 tx=0x7fc5a000ca60 comp rx=0 tx=0).stop 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 shutdown_connections 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5a81338a0 0x7fc5a81c8000 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5a80a09c0 0x7fc5a811f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 --2- 172.21.15.32:0/3859137396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a80a0d90 0x7fc5a811f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 >> 172.21.15.32:0/3859137396 conn(0x7fc5a80a60c0 msgr2=0x7fc5a81bbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 shutdown_connections 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:53.975+0000 7fc5adaf6700 1 -- 172.21.15.32:0/3859137396 wait complete. 2023-09-30T22:29:53.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.012+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc908074ea0 con 0x7fc90809f240 2023-09-30T22:29:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.013+0000 7fc8fe7fc700 1 -- 172.21.15.32:0/3474038536 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8f8024b40 con 0x7fc90809f240 2023-09-30T22:29:54.014 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 msgr2=0x7fc9081a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081a4260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800e930 tx=0x7fc8f800ec40 comp rx=0 tx=0).stop 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 shutdown_connections 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc90809f650 0x7fc9081a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc90809e930 0x7fc9081a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 --2- 172.21.15.32:0/3474038536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc90809f240 0x7fc9081a4260 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 >> 172.21.15.32:0/3474038536 conn(0x7fc9080a4210 msgr2=0x7fc9081b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 shutdown_connections 2023-09-30T22:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.015+0000 7fc90fc12700 1 -- 172.21.15.32:0/3474038536 wait complete. 2023-09-30T22:29:54.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:54.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.624+0000 7fa9792b7700 1 Processor -- start 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.624+0000 7fa9792b7700 1 -- start start 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9792b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa97409e1d0 0x7fa974135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9792b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa974135e50 0x7fa974138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9792b7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97409f130 con 0x7fa97409d6f0 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9792b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97409f520 con 0x7fa974135e50 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9792b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa97409e930 con 0x7fa97409e1d0 2023-09-30T22:29:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9727fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa97409e1d0 0x7fa974135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa972ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa97409d6f0 0x7fa97409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55786/0 (socket says 172.21.15.32:55786) 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa972ffd700 1 -- 172.21.15.32:0/2553263623 learned_addr learned my addr 172.21.15.32:0/2553263623 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9737fe700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa974135e50 0x7fa974138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa971ffb700 1 -- 172.21.15.32:0/2553263623 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062906818 0 0) 0x7fa97409f130 con 0x7fa97409d6f0 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9727fc700 1 -- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa974135e50 msgr2=0x7fa974138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9727fc700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa974135e50 0x7fa974138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9727fc700 1 -- 172.21.15.32:0/2553263623 >> v1:172.21.15.32:6789/0 conn(0x7fa97409d6f0 legacy=0x7fa97409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9727fc700 1 -- 172.21.15.32:0/2553263623 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97409b2d0 con 0x7fa97409e1d0 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.625+0000 7fa9737fe700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa974135e50 0x7fa974138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9727fc700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa97409e1d0 0x7fa974135910 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa96800bb10 tx=0x7fa96800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea86070c606773c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa971ffb700 1 -- 172.21.15.32:0/2553263623 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800dc70 con 0x7fa97409e1d0 2023-09-30T22:29:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa971ffb700 1 -- 172.21.15.32:0/2553263623 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96800ddd0 con 0x7fa97409e1d0 2023-09-30T22:29:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa971ffb700 1 -- 172.21.15.32:0/2553263623 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968010690 con 0x7fa97409e1d0 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 -- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa97409e1d0 msgr2=0x7fa974135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa97409e1d0 0x7fa974135910 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa96800bb10 tx=0x7fa96800be20 comp rx=0 tx=0).stop 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 -- 172.21.15.32:0/2553263623 shutdown_connections 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa974135e50 0x7fa974138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2553263623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa97409e1d0 0x7fa974135910 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 -- 172.21.15.32:0/2553263623 >> 172.21.15.32:0/2553263623 conn(0x7fa974066f50 msgr2=0x7fa97409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 -- 172.21.15.32:0/2553263623 shutdown_connections 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.626+0000 7fa9792b7700 1 -- 172.21.15.32:0/2553263623 wait complete. 2023-09-30T22:29:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.628+0000 7fa9792b7700 1 Processor -- start 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.628+0000 7fa9792b7700 1 -- start start 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9792b7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 0x7fa974119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9792b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa97409e1d0 0x7fa9741199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9792b7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa974135e50 0x7fa974119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9792b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974130090 con 0x7fa97409e1d0 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9792b7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97411c850 con 0x7fa97409d6f0 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9792b7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa97409c700 con 0x7fa974135e50 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 0x7fa974119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 0x7fa974119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45404/0 (socket says 172.21.15.32:45404) 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 -- 172.21.15.32:0/2516191600 learned_addr learned my addr 172.21.15.32:0/2516191600 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:54.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9737fe700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa974135e50 0x7fa974119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa9727fc700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa97409e1d0 0x7fa9741199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 -- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa974135e50 msgr2=0x7fa974119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa974135e50 0x7fa974119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 -- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa97409e1d0 msgr2=0x7fa9741199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa97409e1d0 0x7fa9741199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 -- 172.21.15.32:0/2516191600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c009a70 con 0x7fa97409d6f0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.629+0000 7fa972ffd700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 0x7fa974119480 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c520 tx=0x7fa96400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.630+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964004c80 con 0x7fa97409d6f0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.630+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964004de0 con 0x7fa97409d6f0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.630+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96800b740 con 0x7fa97409d6f0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.630+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964007310 con 0x7fa97409d6f0 2023-09-30T22:29:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.630+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97411cdf0 con 0x7fa97409d6f0 2023-09-30T22:29:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.630+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa964007470 con 0x7fa97409d6f0 2023-09-30T22:29:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.631+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa96403f2a0 con 0x7fa97409d6f0 2023-09-30T22:29:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.631+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97411cc10 con 0x7fa97409d6f0 2023-09-30T22:29:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.636+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9640123b0 con 0x7fa97409d6f0 2023-09-30T22:29:54.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:54.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.843+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa974074ea0 con 0x7fa97409d6f0 2023-09-30T22:29:54.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.843+0000 7fa95b7fe700 1 -- 172.21.15.32:0/2516191600 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa96400ce60 con 0x7fa97409d6f0 2023-09-30T22:29:54.844 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:54.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 msgr2=0x7fa974119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 0x7fa974119480 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c520 tx=0x7fa96400c830 comp rx=0 tx=0).stop 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 shutdown_connections 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa974135e50 0x7fa974119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa97409d6f0 0x7fa974119480 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 --2- 172.21.15.32:0/2516191600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa97409e1d0 0x7fa9741199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 >> 172.21.15.32:0/2516191600 conn(0x7fa974066f50 msgr2=0x7fa9741329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 shutdown_connections 2023-09-30T22:29:54.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.847+0000 7fa9792b7700 1 -- 172.21.15.32:0/2516191600 wait complete. 2023-09-30T22:29:54.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.943+0000 7f1f53e87700 1 Processor -- start 2023-09-30T22:29:54.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.943+0000 7f1f53e87700 1 -- start start 2023-09-30T22:29:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f53e87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09d6f0 0x7f1f4c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f53e87700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f4c09e000 0x7f1f4c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f53e87700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c09b120 con 0x7f1f4c09e410 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f53e87700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c06b460 con 0x7f1f4c09e000 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f53e87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c09a7e0 con 0x7f1f4c09d6f0 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51422700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f4c09e000 0x7f1f4c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f52424700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f4c09e410 0x7f1f4c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55796/0 (socket says 172.21.15.32:55796) 2023-09-30T22:29:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f52424700 1 -- 172.21.15.32:0/1925639759 learned_addr learned my addr 172.21.15.32:0/1925639759 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51c23700 1 --2- 172.21.15.32:0/1925639759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09d6f0 0x7f1f4c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51c23700 1 -- 172.21.15.32:0/1925639759 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c09e000 msgr2=0x7f1f4c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51c23700 1 --2- 172.21.15.32:0/1925639759 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c09e000 0x7f1f4c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51c23700 1 -- 172.21.15.32:0/1925639759 >> v1:172.21.15.32:6789/0 conn(0x7f1f4c09e410 legacy=0x7f1f4c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51c23700 1 -- 172.21.15.32:0/1925639759 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c09a500 con 0x7f1f4c09d6f0 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f50c21700 1 -- 172.21.15.32:0/1925639759 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075248617 0 0) 0x7f1f4c09b120 con 0x7f1f4c09e410 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51422700 1 --2- 172.21.15.32:0/1925639759 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c09e000 0x7f1f4c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.944+0000 7f1f51c23700 1 --2- 172.21.15.32:0/1925639759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09d6f0 0x7f1f4c09dac0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009870 tx=0x7f1f48009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=519ff16df0c86e1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f50c21700 1 -- 172.21.15.32:0/1925639759 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48011830 con 0x7f1f4c09d6f0 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f50c21700 1 -- 172.21.15.32:0/1925639759 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f48011990 con 0x7f1f4c09d6f0 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f50c21700 1 -- 172.21.15.32:0/1925639759 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48011d20 con 0x7f1f4c09d6f0 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 -- 172.21.15.32:0/1925639759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09d6f0 msgr2=0x7f1f4c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 --2- 172.21.15.32:0/1925639759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09d6f0 0x7f1f4c09dac0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009870 tx=0x7f1f48009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 -- 172.21.15.32:0/1925639759 shutdown_connections 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 --2- 172.21.15.32:0/1925639759 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09d6f0 0x7f1f4c09dac0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 --2- 172.21.15.32:0/1925639759 >> v2:172.21.15.32:3300/0 conn(0x7f1f4c09e000 0x7f1f4c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 -- 172.21.15.32:0/1925639759 >> 172.21.15.32:0/1925639759 conn(0x7f1f4c066f50 msgr2=0x7f1f4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 -- 172.21.15.32:0/1925639759 shutdown_connections 2023-09-30T22:29:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.945+0000 7f1f53e87700 1 -- 172.21.15.32:0/1925639759 wait complete. 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.946+0000 7f1f53e87700 1 Processor -- start 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.946+0000 7f1f53e87700 1 -- start start 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f53e87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 0x7f1f4c1c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f53e87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c09e000 0x7f1f4c1c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f53e87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 0x7f1f4c1c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f53e87700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c1a64d0 con 0x7f1f4c09e000 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f53e87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0a2e40 con 0x7f1f4c09d6f0 2023-09-30T22:29:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f53e87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c09c710 con 0x7f1f4c09e410 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f51c23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 0x7f1f4c1c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 0x7f1f4c1c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 0x7f1f4c1c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57448/0 (socket says 172.21.15.32:57448) 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f51c23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 0x7f1f4c1c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45418/0 (socket says 172.21.15.32:45418) 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 -- 172.21.15.32:0/3410055540 learned_addr learned my addr 172.21.15.32:0/3410055540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f51422700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c09e000 0x7f1f4c1c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 -- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 msgr2=0x7f1f4c1c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 0x7f1f4c1c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 -- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c09e000 msgr2=0x7f1f4c1c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c09e000 0x7f1f4c1c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f52424700 1 -- 172.21.15.32:0/3410055540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400094a0 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f51c23700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 0x7f1f4c1c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.947+0000 7f1f51422700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c09e000 0x7f1f4c1c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.948+0000 7f1f52424700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 0x7f1f4c1c8b60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1f44002230 tx=0x7f1f4400cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.948+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400aac0 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.948+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4400ac40 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.948+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f440070d0 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.948+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f480094a0 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.948+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c1b9370 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.949+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f44007250 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.949+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f4403f160 con 0x7f1f4c09e410 2023-09-30T22:29:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.949+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c1a8020 con 0x7f1f4c09e410 2023-09-30T22:29:54.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:54.954+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f44018070 con 0x7f1f4c09e410 2023-09-30T22:29:54.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:55.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:55.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.130+0000 7fab6d166700 1 Processor -- start 2023-09-30T22:29:55.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6d166700 1 -- start start 2023-09-30T22:29:55.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6d166700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809c650 0x7fab6809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6d166700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809cf60 0x7fab68132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6d166700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6809ba70 con 0x7fab680a0090 2023-09-30T22:29:55.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6d166700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6809e8f0 con 0x7fab6809cf60 2023-09-30T22:29:55.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6d166700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab6809d400 con 0x7fab6809c650 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab66d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809c650 0x7fab6809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab680a0090 0x7fab68131970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55804/0 (socket says 172.21.15.32:55804) 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.131+0000 7fab6659c700 1 -- 172.21.15.32:0/972666511 learned_addr learned my addr 172.21.15.32:0/972666511 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab6759e700 1 --2- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809cf60 0x7fab68132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab66d9d700 1 -- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809cf60 msgr2=0x7fab68132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab65d9b700 1 -- 172.21.15.32:0/972666511 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758966018 0 0) 0x7fab6809ba70 con 0x7fab680a0090 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab66d9d700 1 --2- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809cf60 0x7fab68132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab66d9d700 1 -- 172.21.15.32:0/972666511 >> v1:172.21.15.32:6789/0 conn(0x7fab680a0090 legacy=0x7fab68131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab66d9d700 1 -- 172.21.15.32:0/972666511 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680a0760 con 0x7fab6809c650 2023-09-30T22:29:55.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab66d9d700 1 --2- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809c650 0x7fab6809ca20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab58009870 tx=0x7fab58009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c1fa120b0a2c4f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab65d9b700 1 -- 172.21.15.32:0/972666511 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58011830 con 0x7fab6809c650 2023-09-30T22:29:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab65d9b700 1 -- 172.21.15.32:0/972666511 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58011990 con 0x7fab6809c650 2023-09-30T22:29:55.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.132+0000 7fab65d9b700 1 -- 172.21.15.32:0/972666511 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58011d20 con 0x7fab6809c650 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 -- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809c650 msgr2=0x7fab6809ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 --2- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809c650 0x7fab6809ca20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fab58009870 tx=0x7fab58009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 -- 172.21.15.32:0/972666511 shutdown_connections 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 --2- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809cf60 0x7fab68132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 --2- 172.21.15.32:0/972666511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809c650 0x7fab6809ca20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 -- 172.21.15.32:0/972666511 >> 172.21.15.32:0/972666511 conn(0x7fab680a3f70 msgr2=0x7fab680a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 -- 172.21.15.32:0/972666511 shutdown_connections 2023-09-30T22:29:55.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.133+0000 7fab6d166700 1 -- 172.21.15.32:0/972666511 wait complete. 2023-09-30T22:29:55.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.135+0000 7fab6d166700 1 Processor -- start 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.135+0000 7fab6d166700 1 -- start start 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6d166700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 0x7fab6806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6d166700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809cf60 0x7fab6806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6d166700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 0x7fab6806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6d166700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6809d980 con 0x7fab680a0090 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6d166700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6812fd40 con 0x7fab6809c650 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6d166700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab6811eb70 con 0x7fab6809cf60 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 0x7fab6806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab66d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 0x7fab6806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809cf60 0x7fab6806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 0x7fab6806c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39794/0 (socket says 172.21.15.32:39794) 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab6759e700 1 -- 172.21.15.32:0/355747568 learned_addr learned my addr 172.21.15.32:0/355747568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.136+0000 7fab66d9d700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 0x7fab6806b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45440/0 (socket says 172.21.15.32:45440) 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6659c700 1 -- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 msgr2=0x7fab6806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6659c700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 0x7fab6806b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6659c700 1 -- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 msgr2=0x7fab6806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6659c700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 0x7fab6806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6659c700 1 -- 172.21.15.32:0/355747568 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580094a0 con 0x7fab6809cf60 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6759e700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 0x7fab6806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab66d9d700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 0x7fab6806b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6659c700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809cf60 0x7fab6806bdc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fab680a4ee0 tx=0x7fab5c00c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c004c80 con 0x7fab6809cf60 2023-09-30T22:29:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c004de0 con 0x7fab6809cf60 2023-09-30T22:29:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6811f3f0 con 0x7fab6809cf60 2023-09-30T22:29:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c007310 con 0x7fab6809cf60 2023-09-30T22:29:55.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.137+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab68072240 con 0x7fab6809cf60 2023-09-30T22:29:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.138+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab5c007470 con 0x7fab6809cf60 2023-09-30T22:29:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.138+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab5c03f1a0 con 0x7fab6809cf60 2023-09-30T22:29:55.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.139+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab680a3610 con 0x7fab6809cf60 2023-09-30T22:29:55.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.144+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab5c007720 con 0x7fab6809cf60 2023-09-30T22:29:55.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.164+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f4c1a84b0 con 0x7f1f4c09e410 2023-09-30T22:29:55.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.165+0000 7f1f3e7fc700 1 -- 172.21.15.32:0/3410055540 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f44019630 con 0x7f1f4c09e410 2023-09-30T22:29:55.166 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 msgr2=0x7f1f4c1c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 0x7f1f4c1c8b60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1f44002230 tx=0x7f1f4400cf50 comp rx=0 tx=0).stop 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 shutdown_connections 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f4c09e410 0x7f1f4c1c8b60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f4c09d6f0 0x7f1f4c1c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 --2- 172.21.15.32:0/3410055540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f4c09e000 0x7f1f4c1c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 >> 172.21.15.32:0/3410055540 conn(0x7f1f4c066f50 msgr2=0x7f1f4c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 shutdown_connections 2023-09-30T22:29:55.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.167+0000 7f1f53e87700 1 -- 172.21.15.32:0/3410055540 wait complete. 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.175+0000 7f2781b76700 1 Processor -- start 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.175+0000 7f2781b76700 1 -- start start 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.175+0000 7f2781b76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 0x7f277c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.175+0000 7f2781b76700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f277c133e30 0x7f277c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f2781b76700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277c06b990 con 0x7f277c134260 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f2781b76700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277c0a0590 con 0x7f277c133e30 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f2781b76700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c09fb80 con 0x7f277c133420 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 0x7f277c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 0x7f277c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45450/0 (socket says 172.21.15.32:45450) 2023-09-30T22:29:55.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 -- 172.21.15.32:0/841417080 learned_addr learned my addr 172.21.15.32:0/841417080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:55.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277affd700 1 --2- 172.21.15.32:0/841417080 >> v2:172.21.15.32:3300/0 conn(0x7f277c133e30 0x7f277c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277a7fc700 1 -- 172.21.15.32:0/841417080 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314052825 0 0) 0x7f277c06b990 con 0x7f277c134260 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277a7fc700 1 -- 172.21.15.32:0/841417080 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27640035b0 con 0x7f277c134260 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 -- 172.21.15.32:0/841417080 >> v2:172.21.15.32:3300/0 conn(0x7f277c133e30 msgr2=0x7f277c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 --2- 172.21.15.32:0/841417080 >> v2:172.21.15.32:3300/0 conn(0x7f277c133e30 0x7f277c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 -- 172.21.15.32:0/841417080 >> v1:172.21.15.32:6789/0 conn(0x7f277c134260 legacy=0x7f277c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 -- 172.21.15.32:0/841417080 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0a0030 con 0x7f277c133420 2023-09-30T22:29:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.176+0000 7f277b7fe700 1 --2- 172.21.15.32:0/841417080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 0x7f277c1337f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098f0 tx=0x7f276c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8103fc0dd5241448 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f277a7fc700 1 -- 172.21.15.32:0/841417080 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c011830 con 0x7f277c133420 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f277a7fc700 1 -- 172.21.15.32:0/841417080 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c011990 con 0x7f277c133420 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f277a7fc700 1 -- 172.21.15.32:0/841417080 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c011d00 con 0x7f277c133420 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f2781b76700 1 -- 172.21.15.32:0/841417080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 msgr2=0x7f277c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f2781b76700 1 --2- 172.21.15.32:0/841417080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 0x7f277c1337f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098f0 tx=0x7f276c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f2781b76700 1 -- 172.21.15.32:0/841417080 shutdown_connections 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f2781b76700 1 --2- 172.21.15.32:0/841417080 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133420 0x7f277c1337f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.177+0000 7f2781b76700 1 --2- 172.21.15.32:0/841417080 >> v2:172.21.15.32:3300/0 conn(0x7f277c133e30 0x7f277c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.178+0000 7f2781b76700 1 -- 172.21.15.32:0/841417080 >> 172.21.15.32:0/841417080 conn(0x7f277c0a4210 msgr2=0x7f277c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.178+0000 7f2781b76700 1 -- 172.21.15.32:0/841417080 shutdown_connections 2023-09-30T22:29:55.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.178+0000 7f2781b76700 1 -- 172.21.15.32:0/841417080 wait complete. 2023-09-30T22:29:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.179+0000 7f2781b76700 1 Processor -- start 2023-09-30T22:29:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.179+0000 7f2781b76700 1 -- start start 2023-09-30T22:29:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f2781b76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c133420 0x7f277c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f2781b76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 0x7f277c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f2781b76700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277c134260 0x7f277c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f2781b76700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c11fad0 con 0x7f277c133420 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f2781b76700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c11f440 con 0x7f277c133e30 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f2781b76700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f277c11edd0 con 0x7f277c134260 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f277affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 0x7f277c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f277b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c133420 0x7f277c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f277affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 0x7f277c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45456/0 (socket says 172.21.15.32:45456) 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f277affd700 1 -- 172.21.15.32:0/167930492 learned_addr learned my addr 172.21.15.32:0/167930492 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.180+0000 7f277bfff700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277c134260 0x7f277c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277affd700 1 -- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277c134260 msgr2=0x7f277c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277affd700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277c134260 0x7f277c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277affd700 1 -- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c133420 msgr2=0x7f277c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277affd700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c133420 0x7f277c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277affd700 1 -- 172.21.15.32:0/167930492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c009550 con 0x7f277c133e30 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277b7fe700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c133420 0x7f277c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f277affd700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 0x7f277c09c9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f277000e980 tx=0x7f277000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770009710 con 0x7f277c133e30 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277000c690 con 0x7f277c133e30 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c120570 con 0x7f277c133e30 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000ca20 con 0x7f277c133e30 2023-09-30T22:29:55.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.181+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c09df70 con 0x7f277c133e30 2023-09-30T22:29:55.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.182+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f277000cb80 con 0x7f277c133e30 2023-09-30T22:29:55.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.182+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f277003f6d0 con 0x7f277c133e30 2023-09-30T22:29:55.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.183+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c11fcf0 con 0x7f277c133e30 2023-09-30T22:29:55.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.188+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f27700232f0 con 0x7f277c133e30 2023-09-30T22:29:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.355+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fab680747f0 con 0x7fab6809cf60 2023-09-30T22:29:55.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.355+0000 7fab4f7fe700 1 -- 172.21.15.32:0/355747568 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab5c01b760 con 0x7fab6809cf60 2023-09-30T22:29:55.356 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809cf60 msgr2=0x7fab6806bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809cf60 0x7fab6806bdc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fab680a4ee0 tx=0x7fab5c00c750 comp rx=0 tx=0).stop 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 shutdown_connections 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab680a0090 0x7fab6806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fab6809c650 0x7fab6806b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 --2- 172.21.15.32:0/355747568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fab6809cf60 0x7fab6806bdc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 >> 172.21.15.32:0/355747568 conn(0x7fab680a3f70 msgr2=0x7fab6812ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.357+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 shutdown_connections 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.358+0000 7fab6d166700 1 -- 172.21.15.32:0/355747568 wait complete. 2023-09-30T22:29:55.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:55.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.396+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f277c074ea0 con 0x7f277c133e30 2023-09-30T22:29:55.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.396+0000 7f2763fff700 1 -- 172.21.15.32:0/167930492 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2770021020 con 0x7f277c133e30 2023-09-30T22:29:55.398 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 msgr2=0x7f277c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 0x7f277c09c9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f277000e980 tx=0x7f277000ec90 comp rx=0 tx=0).stop 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 shutdown_connections 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f277c134260 0x7f277c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f277c133420 0x7f277c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 --2- 172.21.15.32:0/167930492 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f277c133e30 0x7f277c09c9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 >> 172.21.15.32:0/167930492 conn(0x7f277c0a4210 msgr2=0x7f277c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 shutdown_connections 2023-09-30T22:29:55.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:55.399+0000 7f2781b76700 1 -- 172.21.15.32:0/167930492 wait complete. 2023-09-30T22:29:55.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:55.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.012+0000 7f8b461bb700 1 Processor -- start 2023-09-30T22:29:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.012+0000 7f8b461bb700 1 -- start start 2023-09-30T22:29:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b461bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 0x7f8b4009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b461bb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 0x7f8b40131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b461bb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b40070940 con 0x7f8b4009f240 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b461bb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4006fd60 con 0x7f8b4009e930 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b461bb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4006fa80 con 0x7f8b4009f650 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b3f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 0x7f8b4009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b3ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 0x7f8b40131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b3f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 0x7f8b4009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45460/0 (socket says 172.21.15.32:45460) 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/719234335 learned_addr learned my addr 172.21.15.32:0/719234335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b3ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 0x7f8b40131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57494/0 (socket says 172.21.15.32:57494) 2023-09-30T22:29:56.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.013+0000 7f8b3effd700 1 --1- 172.21.15.32:0/719234335 >> v1:172.21.15.32:6789/0 conn(0x7f8b4009f240 0x7f8b401317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55830/0 (socket says 172.21.15.32:55830) 2023-09-30T22:29:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 msgr2=0x7f8b40131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3f7fe700 1 --2- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 0x7f8b40131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/719234335 >> v1:172.21.15.32:6789/0 conn(0x7f8b4009f240 legacy=0x7f8b401317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3e7fc700 1 -- 172.21.15.32:0/719234335 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010332949 0 0) 0x7f8b40070940 con 0x7f8b4009f240 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/719234335 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4009fb60 con 0x7f8b4009e930 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3ffff700 1 --2- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 0x7f8b40131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3f7fe700 1 --2- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 0x7f8b4009ed00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009920 tx=0x7f8b30009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f208d6267c30a06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3e7fc700 1 -- 172.21.15.32:0/719234335 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011830 con 0x7f8b4009e930 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3e7fc700 1 -- 172.21.15.32:0/719234335 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30011990 con 0x7f8b4009e930 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.014+0000 7f8b3e7fc700 1 -- 172.21.15.32:0/719234335 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011d20 con 0x7f8b4009e930 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 -- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 msgr2=0x7f8b4009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 --2- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 0x7f8b4009ed00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009920 tx=0x7f8b30009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 -- 172.21.15.32:0/719234335 shutdown_connections 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 --2- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009f650 0x7f8b40131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 --2- 172.21.15.32:0/719234335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009e930 0x7f8b4009ed00 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 -- 172.21.15.32:0/719234335 >> 172.21.15.32:0/719234335 conn(0x7f8b400a4210 msgr2=0x7f8b400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 -- 172.21.15.32:0/719234335 shutdown_connections 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.015+0000 7f8b461bb700 1 -- 172.21.15.32:0/719234335 wait complete. 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 Processor -- start 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 -- start start 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 0x7f8b401a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b4009f240 0x7f8b401a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009f650 0x7f8b401a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401a80f0 con 0x7f8b4009f240 2023-09-30T22:29:56.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401a7730 con 0x7f8b4009f650 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b461bb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b401a6a70 con 0x7f8b4009e930 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b3f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 0x7f8b401a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b3effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b4009f240 0x7f8b401a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b3ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009f650 0x7f8b401a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b3f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 0x7f8b401a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57504/0 (socket says 172.21.15.32:57504) 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.016+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/2686904338 learned_addr learned my addr 172.21.15.32:0/2686904338 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009f650 msgr2=0x7f8b401a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b3f7fe700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009f650 0x7f8b401a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b4009f240 msgr2=0x7f8b401a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b3f7fe700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b4009f240 0x7f8b401a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b3f7fe700 1 -- 172.21.15.32:0/2686904338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009a70 con 0x7f8b4009e930 2023-09-30T22:29:56.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b3f7fe700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 0x7f8b401a3c10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8b300098f0 tx=0x7f8b30004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30004450 con 0x7f8b4009e930 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b300045b0 con 0x7f8b4009e930 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b30009550 con 0x7f8b4009e930 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b300038c0 con 0x7f8b4009e930 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.017+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b400a0190 con 0x7f8b4009e930 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.018+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b30003a20 con 0x7f8b4009e930 2023-09-30T22:29:56.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.018+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b401a0b30 con 0x7f8b4009e930 2023-09-30T22:29:56.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.018+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8b30021030 con 0x7f8b4009e930 2023-09-30T22:29:56.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.023+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b30015020 con 0x7f8b4009e930 2023-09-30T22:29:56.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.228+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b401a5880 con 0x7f8b4009e930 2023-09-30T22:29:56.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.228+0000 7f8b27fff700 1 -- 172.21.15.32:0/2686904338 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b3001a430 con 0x7f8b4009e930 2023-09-30T22:29:56.230 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.230+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 msgr2=0x7f8b401a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 0x7f8b401a3c10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8b300098f0 tx=0x7f8b30004750 comp rx=0 tx=0).stop 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 shutdown_connections 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8b4009f650 0x7f8b401a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8b4009e930 0x7f8b401a3c10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 --2- 172.21.15.32:0/2686904338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b4009f240 0x7f8b401a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 >> 172.21.15.32:0/2686904338 conn(0x7f8b400a4210 msgr2=0x7f8b401b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 shutdown_connections 2023-09-30T22:29:56.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.231+0000 7f8b461bb700 1 -- 172.21.15.32:0/2686904338 wait complete. 2023-09-30T22:29:56.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.324+0000 7efeee740700 1 Processor -- start 2023-09-30T22:29:56.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.324+0000 7efeee740700 1 -- start start 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeee740700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 0x7efee8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeee740700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efee8139250 0x7efee813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeee740700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee809edd0 con 0x7efee80a0a30 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeee740700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80a18e0 con 0x7efee8139250 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeee740700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee809e1f0 con 0x7efee80a1510 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 0x7efee8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeeccdd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efee8139250 0x7efee813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efee80a0a30 0x7efee80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55844/0 (socket says 172.21.15.32:55844) 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee7fff700 1 -- 172.21.15.32:0/1043570794 learned_addr learned my addr 172.21.15.32:0/1043570794 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:56.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 --2- 172.21.15.32:0/1043570794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 0x7efee8138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45476/0 (socket says 172.21.15.32:45476) 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 -- 172.21.15.32:0/1043570794 >> v2:172.21.15.32:3300/0 conn(0x7efee8139250 msgr2=0x7efee813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 --2- 172.21.15.32:0/1043570794 >> v2:172.21.15.32:3300/0 conn(0x7efee8139250 0x7efee813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 -- 172.21.15.32:0/1043570794 >> v1:172.21.15.32:6789/0 conn(0x7efee80a0a30 legacy=0x7efee80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 -- 172.21.15.32:0/1043570794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee809dc50 con 0x7efee80a1510 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee6ffd700 1 -- 172.21.15.32:0/1043570794 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577956130 0 0) 0x7efee809edd0 con 0x7efee80a0a30 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efeeccdd700 1 --2- 172.21.15.32:0/1043570794 >> v2:172.21.15.32:3300/0 conn(0x7efee8139250 0x7efee813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee77fe700 1 --2- 172.21.15.32:0/1043570794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 0x7efee8138d10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7efed0009870 tx=0x7efed0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8a32c7adf06a4f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee6ffd700 1 -- 172.21.15.32:0/1043570794 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0011830 con 0x7efee80a1510 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.325+0000 7efee6ffd700 1 -- 172.21.15.32:0/1043570794 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0011990 con 0x7efee80a1510 2023-09-30T22:29:56.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efee6ffd700 1 -- 172.21.15.32:0/1043570794 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0011d20 con 0x7efee80a1510 2023-09-30T22:29:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 -- 172.21.15.32:0/1043570794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 msgr2=0x7efee8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 --2- 172.21.15.32:0/1043570794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 0x7efee8138d10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7efed0009870 tx=0x7efed0009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 -- 172.21.15.32:0/1043570794 shutdown_connections 2023-09-30T22:29:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 --2- 172.21.15.32:0/1043570794 >> v2:172.21.15.32:3300/0 conn(0x7efee8139250 0x7efee813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 --2- 172.21.15.32:0/1043570794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a1510 0x7efee8138d10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 -- 172.21.15.32:0/1043570794 >> 172.21.15.32:0/1043570794 conn(0x7efee80a6550 msgr2=0x7efee80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 -- 172.21.15.32:0/1043570794 shutdown_connections 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.326+0000 7efeee740700 1 -- 172.21.15.32:0/1043570794 wait complete. 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.327+0000 7efeee740700 1 Processor -- start 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.327+0000 7efeee740700 1 -- start start 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeee740700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 0x7efee8122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeee740700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee80a1510 0x7efee8122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeee740700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efee8139250 0x7efee8123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeee740700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81389b0 con 0x7efee80a1510 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeee740700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee8124390 con 0x7efee80a0a30 2023-09-30T22:29:56.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeee740700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee8124c60 con 0x7efee8139250 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 0x7efee8122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee80a1510 0x7efee8122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 0x7efee8122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45490/0 (socket says 172.21.15.32:45490) 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 -- 172.21.15.32:0/3221364454 learned_addr learned my addr 172.21.15.32:0/3221364454 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efeeccdd700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efee8139250 0x7efee8123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 -- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efee8139250 msgr2=0x7efee8123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efee8139250 0x7efee8123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 -- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee80a1510 msgr2=0x7efee8122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee80a1510 0x7efee8122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee7fff700 1 -- 172.21.15.32:0/3221364454 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee0009a70 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.328+0000 7efee77fe700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee80a1510 0x7efee8122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.329+0000 7efee7fff700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 0x7efee8122780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efed800c550 tx=0x7efed800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.329+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8004c80 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.329+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed8004e00 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.329+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed00094a0 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.329+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8007310 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.329+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee8125fc0 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.330+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee8074ea0 con 0x7efee80a0a30 2023-09-30T22:29:56.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.330+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efed8007470 con 0x7efee80a0a30 2023-09-30T22:29:56.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.330+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efed803f2b0 con 0x7efee80a0a30 2023-09-30T22:29:56.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.335+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efed80133c0 con 0x7efee80a0a30 2023-09-30T22:29:56.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:56.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:56.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.520+0000 7fb2a3e64700 1 Processor -- start 2023-09-30T22:29:56.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.520+0000 7fb2a3e64700 1 -- start start 2023-09-30T22:29:56.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a3e64700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb29c09c650 0x7fb29c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a3e64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09cf60 0x7fb29c132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a3e64700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29c09ba70 con 0x7fb29c0a0090 2023-09-30T22:29:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a3e64700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29c09e8f0 con 0x7fb29c09c650 2023-09-30T22:29:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a3e64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0a0760 con 0x7fb29c09cf60 2023-09-30T22:29:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a1c00700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb29c09c650 0x7fb29c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a1c00700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb29c09c650 0x7fb29c09ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39832/0 (socket says 172.21.15.32:39832) 2023-09-30T22:29:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.521+0000 7fb2a1c00700 1 -- 172.21.15.32:0/2219315825 learned_addr learned my addr 172.21.15.32:0/2219315825 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a2401700 1 --2- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09cf60 0x7fb29c132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a1c00700 1 -- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09cf60 msgr2=0x7fb29c132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a1c00700 1 --2- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09cf60 0x7fb29c132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a1c00700 1 -- 172.21.15.32:0/2219315825 >> v1:172.21.15.32:6789/0 conn(0x7fb29c0a0090 legacy=0x7fb29c131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a1c00700 1 -- 172.21.15.32:0/2219315825 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c09a080 con 0x7fb29c09c650 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a1c00700 1 --2- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c09c650 0x7fb29c09ca20 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098f0 tx=0x7fb298009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9887fce3adba2de9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a0bfe700 1 -- 172.21.15.32:0/2219315825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298011830 con 0x7fb29c09c650 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a0bfe700 1 -- 172.21.15.32:0/2219315825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298011990 con 0x7fb29c09c650 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.522+0000 7fb2a0bfe700 1 -- 172.21.15.32:0/2219315825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298011d20 con 0x7fb29c09c650 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c09c650 msgr2=0x7fb29c09ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c09c650 0x7fb29c09ca20 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb2980098f0 tx=0x7fb298009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2219315825 shutdown_connections 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09cf60 0x7fb29c132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2219315825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c09c650 0x7fb29c09ca20 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2219315825 >> 172.21.15.32:0/2219315825 conn(0x7fb29c0a3f70 msgr2=0x7fb29c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2219315825 shutdown_connections 2023-09-30T22:29:56.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.523+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2219315825 wait complete. 2023-09-30T22:29:56.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.524+0000 7fb2a3e64700 1 Processor -- start 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 -- start start 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 0x7fb29c11a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb29c09cf60 0x7fb29c11ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c0a0090 0x7fb29c11b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c071d90 con 0x7fb29c0a0090 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c11bdd0 con 0x7fb29c09c650 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a3e64700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c11c850 con 0x7fb29c09cf60 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a13ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb29c09cf60 0x7fb29c11ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a1c00700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 0x7fb29c11a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a2401700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c0a0090 0x7fb29c11b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a1c00700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 0x7fb29c11a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45514/0 (socket says 172.21.15.32:45514) 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a13ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb29c09cf60 0x7fb29c11ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57514/0 (socket says 172.21.15.32:57514) 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.525+0000 7fb2a1c00700 1 -- 172.21.15.32:0/2512383196 learned_addr learned my addr 172.21.15.32:0/2512383196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a2401700 1 -- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb29c09cf60 msgr2=0x7fb29c11ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a2401700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb29c09cf60 0x7fb29c11ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a2401700 1 -- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 msgr2=0x7fb29c11a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a2401700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 0x7fb29c11a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a2401700 1 -- 172.21.15.32:0/2512383196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb298009550 con 0x7fb29c0a0090 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a1c00700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 0x7fb29c11a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a2401700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c0a0090 0x7fb29c11b0a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d90 tx=0x7fb29400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294009c00 con 0x7fb29c0a0090 2023-09-30T22:29:56.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29400f040 con 0x7fb29c0a0090 2023-09-30T22:29:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c0725f0 con 0x7fb29c0a0090 2023-09-30T22:29:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400ce90 con 0x7fb29c0a0090 2023-09-30T22:29:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.526+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c12ffd0 con 0x7fb29c0a0090 2023-09-30T22:29:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.527+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb294014480 con 0x7fb29c0a0090 2023-09-30T22:29:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.527+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb2940401f0 con 0x7fb29c0a0090 2023-09-30T22:29:56.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.527+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c11c990 con 0x7fb29c0a0090 2023-09-30T22:29:56.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.533+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb294018020 con 0x7fb29c0a0090 2023-09-30T22:29:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.548+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efee80a28e0 con 0x7efee80a0a30 2023-09-30T22:29:56.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.548+0000 7efee4ff9700 1 -- 172.21.15.32:0/3221364454 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efed8011360 con 0x7efee80a0a30 2023-09-30T22:29:56.549 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 msgr2=0x7efee8122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 0x7efee8122780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efed800c550 tx=0x7efed800c860 comp rx=0 tx=0).stop 2023-09-30T22:29:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 shutdown_connections 2023-09-30T22:29:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efee8139250 0x7efee8123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee80a0a30 0x7efee8122780 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 --2- 172.21.15.32:0/3221364454 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee80a1510 0x7efee8122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 >> 172.21.15.32:0/3221364454 conn(0x7efee80a6550 msgr2=0x7efee80a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 shutdown_connections 2023-09-30T22:29:56.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.551+0000 7efeee740700 1 -- 172.21.15.32:0/3221364454 wait complete. 2023-09-30T22:29:56.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:56.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.568+0000 7f9d477ec700 1 Processor -- start 2023-09-30T22:29:56.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.568+0000 7f9d477ec700 1 -- start start 2023-09-30T22:29:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d477ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d4009d6f0 0x7f9d4009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d477ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 0x7f9d40134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d477ec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4009b120 con 0x7f9d4009e000 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d477ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4006b460 con 0x7f9d4009d6f0 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d477ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4009a7e0 con 0x7f9d4009e440 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 0x7f9d40134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45588700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d4009d6f0 0x7f9d4009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 0x7f9d40134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57526/0 (socket says 172.21.15.32:57526) 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d44d87700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d4009e000 0x7f9d401339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55864/0 (socket says 172.21.15.32:55864) 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 -- 172.21.15.32:0/1256304763 learned_addr learned my addr 172.21.15.32:0/1256304763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 -- 172.21.15.32:0/1256304763 >> v2:172.21.15.32:3300/0 conn(0x7f9d4009d6f0 msgr2=0x7f9d4009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 --2- 172.21.15.32:0/1256304763 >> v2:172.21.15.32:3300/0 conn(0x7f9d4009d6f0 0x7f9d4009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 -- 172.21.15.32:0/1256304763 >> v1:172.21.15.32:6789/0 conn(0x7f9d4009e000 legacy=0x7f9d401339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 -- 172.21.15.32:0/1256304763 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4009a500 con 0x7f9d4009e440 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45588700 1 --2- 172.21.15.32:0/1256304763 >> v2:172.21.15.32:3300/0 conn(0x7f9d4009d6f0 0x7f9d4009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.569+0000 7f9d45d89700 1 --2- 172.21.15.32:0/1256304763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 0x7f9d40134100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009e80 tx=0x7f9d3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9756b9f3209169 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d37fff700 1 -- 172.21.15.32:0/1256304763 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800e070 con 0x7f9d4009e440 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d37fff700 1 -- 172.21.15.32:0/1256304763 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3800b8d0 con 0x7f9d4009e440 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d37fff700 1 -- 172.21.15.32:0/1256304763 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800bc80 con 0x7f9d4009e440 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 -- 172.21.15.32:0/1256304763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 msgr2=0x7f9d40134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1256304763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 0x7f9d40134100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009e80 tx=0x7f9d3800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 -- 172.21.15.32:0/1256304763 shutdown_connections 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1256304763 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e440 0x7f9d40134100 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1256304763 >> v2:172.21.15.32:3300/0 conn(0x7f9d4009d6f0 0x7f9d4009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 -- 172.21.15.32:0/1256304763 >> 172.21.15.32:0/1256304763 conn(0x7f9d40066f50 msgr2=0x7f9d4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 -- 172.21.15.32:0/1256304763 shutdown_connections 2023-09-30T22:29:56.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.570+0000 7f9d477ec700 1 -- 172.21.15.32:0/1256304763 wait complete. 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.571+0000 7f9d477ec700 1 Processor -- start 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 -- start start 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 0x7f9d401a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 0x7f9d401a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d4009e440 0x7f9d401a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d401aa360 con 0x7f9d4009d6f0 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d401a99a0 con 0x7f9d4009e440 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d477ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d401a8ce0 con 0x7f9d4009e000 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d44d87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 0x7f9d401a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d45588700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 0x7f9d401a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d45588700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 0x7f9d401a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39876/0 (socket says 172.21.15.32:39876) 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d44d87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 0x7f9d401a6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57534/0 (socket says 172.21.15.32:57534) 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d45588700 1 -- 172.21.15.32:0/1525914297 learned_addr learned my addr 172.21.15.32:0/1525914297 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.572+0000 7f9d45d89700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d4009e440 0x7f9d401a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d44d87700 1 -- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d4009e440 msgr2=0x7f9d401a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d44d87700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d4009e440 0x7f9d401a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d44d87700 1 -- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 msgr2=0x7f9d401a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d44d87700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 0x7f9d401a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d44d87700 1 -- 172.21.15.32:0/1525914297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c0094a0 con 0x7f9d4009e000 2023-09-30T22:29:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d45588700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 0x7f9d401a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d44d87700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 0x7f9d401a6410 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c480 tx=0x7f9d3000c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30005cb0 con 0x7f9d4009e000 2023-09-30T22:29:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30005e10 con 0x7f9d4009e000 2023-09-30T22:29:56.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d38009b30 con 0x7f9d4009e000 2023-09-30T22:29:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d300072d0 con 0x7f9d4009e000 2023-09-30T22:29:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.573+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d401a7a90 con 0x7f9d4009e000 2023-09-30T22:29:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.574+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d30007430 con 0x7f9d4009e000 2023-09-30T22:29:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.574+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d40074ea0 con 0x7f9d4009e000 2023-09-30T22:29:56.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.574+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d3003eaa0 con 0x7f9d4009e000 2023-09-30T22:29:56.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.579+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d30017070 con 0x7f9d4009e000 2023-09-30T22:29:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.744+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb29c0747f0 con 0x7fb29c0a0090 2023-09-30T22:29:56.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.744+0000 7fb2927fc700 1 -- 172.21.15.32:0/2512383196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb29402b090 con 0x7fb29c0a0090 2023-09-30T22:29:56.745 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c0a0090 msgr2=0x7fb29c11b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c0a0090 0x7fb29c11b0a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d90 tx=0x7fb29400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 shutdown_connections 2023-09-30T22:29:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb29c0a0090 0x7fb29c11b0a0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb29c09c650 0x7fb29c11a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 --2- 172.21.15.32:0/2512383196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb29c09cf60 0x7fb29c11ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 >> 172.21.15.32:0/2512383196 conn(0x7fb29c0a3f70 msgr2=0x7fb29c09eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 shutdown_connections 2023-09-30T22:29:56.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.747+0000 7fb2a3e64700 1 -- 172.21.15.32:0/2512383196 wait complete. 2023-09-30T22:29:56.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:56.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.785+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9d4006c360 con 0x7f9d4009e000 2023-09-30T22:29:56.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.785+0000 7f9d35ffb700 1 -- 172.21.15.32:0/1525914297 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d300198b0 con 0x7f9d4009e000 2023-09-30T22:29:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:56.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 msgr2=0x7f9d401a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 0x7f9d401a6410 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c480 tx=0x7f9d3000c790 comp rx=0 tx=0).stop 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 shutdown_connections 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d4009e440 0x7f9d401a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4009d6f0 0x7f9d401a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 --2- 172.21.15.32:0/1525914297 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d4009e000 0x7f9d401a6410 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 >> 172.21.15.32:0/1525914297 conn(0x7f9d40066f50 msgr2=0x7f9d401baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 shutdown_connections 2023-09-30T22:29:56.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:56.787+0000 7f9d477ec700 1 -- 172.21.15.32:0/1525914297 wait complete. 2023-09-30T22:29:56.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:57.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.394+0000 7f09d334e700 1 Processor -- start 2023-09-30T22:29:57.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.394+0000 7f09d334e700 1 -- start start 2023-09-30T22:29:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d334e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d334e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d334e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc09f130 con 0x7f09cc09e000 2023-09-30T22:29:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d334e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc09f520 con 0x7f09cc09d6f0 2023-09-30T22:29:57.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d334e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc09e930 con 0x7f09cc09e410 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d10ea700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d08e9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09cc09e000 0x7f09cc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55870/0 (socket says 172.21.15.32:55870) 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d08e9700 1 -- 172.21.15.32:0/3795099061 learned_addr learned my addr 172.21.15.32:0/3795099061 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d18eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09c3fff700 1 -- 172.21.15.32:0/3795099061 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578075229 0 0) 0x7f09cc09f130 con 0x7f09cc09e000 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d10ea700 1 -- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 msgr2=0x7f09cc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d10ea700 1 --2- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d10ea700 1 -- 172.21.15.32:0/3795099061 >> v1:172.21.15.32:6789/0 conn(0x7f09cc09e000 legacy=0x7f09cc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d10ea700 1 -- 172.21.15.32:0/3795099061 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc09b2d0 con 0x7f09cc09d6f0 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d18eb700 1 --2- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09d10ea700 1 --2- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc09dac0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009870 tx=0x7f09c8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ff2933dbda99d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.395+0000 7f09c3fff700 1 -- 172.21.15.32:0/3795099061 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8011830 con 0x7f09cc09d6f0 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09c3fff700 1 -- 172.21.15.32:0/3795099061 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c8011990 con 0x7f09cc09d6f0 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09c3fff700 1 -- 172.21.15.32:0/3795099061 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8011d20 con 0x7f09cc09d6f0 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 -- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 msgr2=0x7f09cc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 --2- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc09dac0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009870 tx=0x7f09c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 -- 172.21.15.32:0/3795099061 shutdown_connections 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 --2- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 --2- 172.21.15.32:0/3795099061 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc09dac0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 -- 172.21.15.32:0/3795099061 >> 172.21.15.32:0/3795099061 conn(0x7f09cc066f50 msgr2=0x7f09cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 -- 172.21.15.32:0/3795099061 shutdown_connections 2023-09-30T22:29:57.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.396+0000 7f09d334e700 1 -- 172.21.15.32:0/3795099061 wait complete. 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.397+0000 7f09d334e700 1 Processor -- start 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.397+0000 7f09d334e700 1 -- start start 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d334e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc11e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d334e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 0x7f09cc11e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d334e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc11eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d334e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc121b50 con 0x7f09cc09e000 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d334e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc09c770 con 0x7f09cc09d6f0 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d334e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc072150 con 0x7f09cc09e410 2023-09-30T22:29:57.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 0x7f09cc11e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 0x7f09cc11e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39884/0 (socket says 172.21.15.32:39884) 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 -- 172.21.15.32:0/987275128 learned_addr learned my addr 172.21.15.32:0/987275128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d10ea700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc11e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d18eb700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc11eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 -- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 msgr2=0x7f09cc11eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc11eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 -- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 msgr2=0x7f09cc11e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc11e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 -- 172.21.15.32:0/987275128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4009a70 con 0x7f09cc09e000 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d18eb700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc11eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d10ea700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc11e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09d08e9700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 0x7f09cc11e570 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f09bc001eb0 tx=0x7f09bc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.398+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc004c80 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.399+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc00ce90 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.399+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c80094a0 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.399+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0070d0 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.399+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc1215b0 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.399+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09bc007230 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.400+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f09bc03f220 con 0x7f09cc09e000 2023-09-30T22:29:57.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.400+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc120e90 con 0x7f09cc09e000 2023-09-30T22:29:57.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.405+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09bc01b990 con 0x7f09cc09e000 2023-09-30T22:29:57.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.612+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f09cc11ac20 con 0x7f09cc09e000 2023-09-30T22:29:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.612+0000 7f09c1ffb700 1 -- 172.21.15.32:0/987275128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09bc01b7b0 con 0x7f09cc09e000 2023-09-30T22:29:57.613 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 msgr2=0x7f09cc11e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 0x7f09cc11e570 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f09bc001eb0 tx=0x7f09bc00c800 comp rx=0 tx=0).stop 2023-09-30T22:29:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 shutdown_connections 2023-09-30T22:29:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f09cc09e410 0x7f09cc11eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09cc09d6f0 0x7f09cc11e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 --2- 172.21.15.32:0/987275128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09cc09e000 0x7f09cc11e570 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 >> 172.21.15.32:0/987275128 conn(0x7f09cc066f50 msgr2=0x7f09cc130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 shutdown_connections 2023-09-30T22:29:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.615+0000 7f09d334e700 1 -- 172.21.15.32:0/987275128 wait complete. 2023-09-30T22:29:57.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:57.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.704+0000 7f3684ee3700 1 Processor -- start 2023-09-30T22:29:57.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.704+0000 7f3684ee3700 1 -- start start 2023-09-30T22:29:57.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f3684ee3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a0a30 0x7f36800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f3684ee3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3680134000 0x7f368013bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f3684ee3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368009eaf0 con 0x7f36800a1340 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f3684ee3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800a27c0 con 0x7f3680134000 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f3684ee3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368009dbe0 con 0x7f36800a0a30 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367ed9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3680134000 0x7f368013bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a0a30 0x7f36800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367dd9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f36800a1340 0x7f36801338f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55878/0 (socket says 172.21.15.32:55878) 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367dd9b700 1 -- 172.21.15.32:0/4289431644 learned_addr learned my addr 172.21.15.32:0/4289431644 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367e59c700 1 -- 172.21.15.32:0/4289431644 >> v2:172.21.15.32:3300/0 conn(0x7f3680134000 msgr2=0x7f368013bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:57.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367e59c700 1 --2- 172.21.15.32:0/4289431644 >> v2:172.21.15.32:3300/0 conn(0x7f3680134000 0x7f368013bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367e59c700 1 -- 172.21.15.32:0/4289431644 >> v1:172.21.15.32:6789/0 conn(0x7f36800a1340 legacy=0x7f36801338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367e59c700 1 -- 172.21.15.32:0/4289431644 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800a1f90 con 0x7f36800a0a30 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367d59a700 1 -- 172.21.15.32:0/4289431644 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235545654 0 0) 0x7f3668001110 con 0x7f36800a1340 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367ed9d700 1 --2- 172.21.15.32:0/4289431644 >> v2:172.21.15.32:3300/0 conn(0x7f3680134000 0x7f368013bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.705+0000 7f367e59c700 1 --2- 172.21.15.32:0/4289431644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a0a30 0x7f36800a0e00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3670009840 tx=0x7f3670009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd58506c6cb19446 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f367d59a700 1 -- 172.21.15.32:0/4289431644 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670011830 con 0x7f36800a0a30 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f367d59a700 1 -- 172.21.15.32:0/4289431644 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670011990 con 0x7f36800a0a30 2023-09-30T22:29:57.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f367d59a700 1 -- 172.21.15.32:0/4289431644 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670011d20 con 0x7f36800a0a30 2023-09-30T22:29:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 -- 172.21.15.32:0/4289431644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a0a30 msgr2=0x7f36800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 --2- 172.21.15.32:0/4289431644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a0a30 0x7f36800a0e00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3670009840 tx=0x7f3670009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 -- 172.21.15.32:0/4289431644 shutdown_connections 2023-09-30T22:29:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 --2- 172.21.15.32:0/4289431644 >> v2:172.21.15.32:3300/0 conn(0x7f3680134000 0x7f368013bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 --2- 172.21.15.32:0/4289431644 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a0a30 0x7f36800a0e00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 -- 172.21.15.32:0/4289431644 >> 172.21.15.32:0/4289431644 conn(0x7f36800a6550 msgr2=0x7f36800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 -- 172.21.15.32:0/4289431644 shutdown_connections 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.706+0000 7f3684ee3700 1 -- 172.21.15.32:0/4289431644 wait complete. 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.707+0000 7f3684ee3700 1 Processor -- start 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 -- start start 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36800a0a30 0x7f3680122710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 0x7f3680122c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3680134000 0x7f3680123190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680123b70 con 0x7f3680134000 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800a02e0 con 0x7f36800a1340 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f3684ee3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36801243e0 con 0x7f36800a0a30 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 0x7f3680122c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36800a0a30 0x7f3680122710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3680134000 0x7f3680123190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 0x7f3680122c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45560/0 (socket says 172.21.15.32:45560) 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 -- 172.21.15.32:0/710660753 learned_addr learned my addr 172.21.15.32:0/710660753 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 -- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36800a0a30 msgr2=0x7f3680122710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36800a0a30 0x7f3680122710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 -- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3680134000 msgr2=0x7f3680123190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3680134000 0x7f3680123190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367dd9b700 1 -- 172.21.15.32:0/710660753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678009a70 con 0x7f36800a1340 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.708+0000 7f367ed9d700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3680134000 0x7f3680123190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.709+0000 7f367dd9b700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 0x7f3680122c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3668001eb0 tx=0x7f366800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.709+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668004c80 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.709+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668004de0 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.709+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36700094a0 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.709+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668007310 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.709+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368009d710 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.710+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3668007470 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.710+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f366803f120 con 0x7f36800a1340 2023-09-30T22:29:57.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.710+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368009eaf0 con 0x7f36800a1340 2023-09-30T22:29:57.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.715+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f368009eaf0 con 0x7f36800a1340 2023-09-30T22:29:57.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:57.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.900+0000 7fbf08e8f700 1 Processor -- start 2023-09-30T22:29:57.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.900+0000 7fbf08e8f700 1 -- start start 2023-09-30T22:29:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf08e8f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf040a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf08e8f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 0x7fbf0413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf08e8f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040a2a70 con 0x7fbf040a0a30 2023-09-30T22:29:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf08e8f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040a01d0 con 0x7fbf04138d80 2023-09-30T22:29:57.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf08e8f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0409e2b0 con 0x7fbf0409d630 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf02d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 0x7fbf0413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf02d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 0x7fbf0413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45562/0 (socket says 172.21.15.32:45562) 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf02d9d700 1 -- 172.21.15.32:0/4184474740 learned_addr learned my addr 172.21.15.32:0/4184474740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf01d9b700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf040a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.901+0000 7fbf0259c700 1 --1- 172.21.15.32:0/4184474740 >> v1:172.21.15.32:6789/0 conn(0x7fbf040a0a30 0x7fbf040a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55884/0 (socket says 172.21.15.32:55884) 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf01d9b700 1 -- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 msgr2=0x7fbf0413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf01d9b700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 0x7fbf0413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf01d9b700 1 -- 172.21.15.32:0/4184474740 >> v1:172.21.15.32:6789/0 conn(0x7fbf040a0a30 legacy=0x7fbf040a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf01d9b700 1 -- 172.21.15.32:0/4184474740 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0409dc50 con 0x7fbf0409d630 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf02d9d700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 0x7fbf0413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf01d9b700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf040a1510 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d002b3c102c71ea5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf0159a700 1 -- 172.21.15.32:0/4184474740 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8011830 con 0x7fbf0409d630 2023-09-30T22:29:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf0159a700 1 -- 172.21.15.32:0/4184474740 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8011990 con 0x7fbf0409d630 2023-09-30T22:29:57.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.902+0000 7fbf0159a700 1 -- 172.21.15.32:0/4184474740 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8011d20 con 0x7fbf0409d630 2023-09-30T22:29:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 -- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 msgr2=0x7fbf040a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf040a1510 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef8009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 -- 172.21.15.32:0/4184474740 shutdown_connections 2023-09-30T22:29:57.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf04138d80 0x7fbf0413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/4184474740 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf040a1510 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 -- 172.21.15.32:0/4184474740 >> 172.21.15.32:0/4184474740 conn(0x7fbf040a3050 msgr2=0x7fbf040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 -- 172.21.15.32:0/4184474740 shutdown_connections 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.903+0000 7fbf08e8f700 1 -- 172.21.15.32:0/4184474740 wait complete. 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.904+0000 7fbf08e8f700 1 Processor -- start 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.904+0000 7fbf08e8f700 1 -- start start 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf08e8f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf04120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf08e8f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 0x7fbf04120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf08e8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04138d80 0x7fbf04121410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf08e8f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf04124220 con 0x7fbf04138d80 2023-09-30T22:29:57.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf08e8f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf04121df0 con 0x7fbf040a0a30 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf08e8f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041237d0 con 0x7fbf0409d630 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf01d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 0x7fbf04120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf0259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf04120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf01d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 0x7fbf04120ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45568/0 (socket says 172.21.15.32:45568) 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf0259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf04120990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57592/0 (socket says 172.21.15.32:57592) 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf01d9b700 1 -- 172.21.15.32:0/1863719309 learned_addr learned my addr 172.21.15.32:0/1863719309 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf02d9d700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04138d80 0x7fbf04121410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf02d9d700 1 -- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 msgr2=0x7fbf04120990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf02d9d700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf04120990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf02d9d700 1 -- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 msgr2=0x7fbf04120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf02d9d700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 0x7fbf04120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf02d9d700 1 -- 172.21.15.32:0/1863719309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc009550 con 0x7fbf04138d80 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf01d9b700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 0x7fbf04120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.905+0000 7fbf0259c700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf04120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.906+0000 7fbf02d9d700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04138d80 0x7fbf04121410 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009fd0 tx=0x7fbefc00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.906+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004020 con 0x7fbf04138d80 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.906+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc0041a0 con 0x7fbf04138d80 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.906+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef80094a0 con 0x7fbf04138d80 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.906+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004550 con 0x7fbf04138d80 2023-09-30T22:29:57.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.906+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0413af30 con 0x7fbf04138d80 2023-09-30T22:29:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.907+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbefc0036a0 con 0x7fbf04138d80 2023-09-30T22:29:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.907+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf040a0410 con 0x7fbf04138d80 2023-09-30T22:29:57.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.907+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbefc03f2a0 con 0x7fbf04138d80 2023-09-30T22:29:57.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.916+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbefc020020 con 0x7fbf04138d80 2023-09-30T22:29:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.923+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f368009eaf0 con 0x7f36800a1340 2023-09-30T22:29:57.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.923+0000 7f3676ffd700 1 -- 172.21.15.32:0/710660753 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f368009eaf0 con 0x7f36800a1340 2023-09-30T22:29:57.924 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 msgr2=0x7f3680122c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 0x7f3680122c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3668001eb0 tx=0x7f366800c800 comp rx=0 tx=0).stop 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 shutdown_connections 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3680134000 0x7f3680123190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f36800a0a30 0x7f3680122710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 --2- 172.21.15.32:0/710660753 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f36800a1340 0x7f3680122c50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 >> 172.21.15.32:0/710660753 conn(0x7f36800a6550 msgr2=0x7f36800a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 shutdown_connections 2023-09-30T22:29:57.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.926+0000 7f3684ee3700 1 -- 172.21.15.32:0/710660753 wait complete. 2023-09-30T22:29:57.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:57.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.956+0000 7f4499f67700 1 Processor -- start 2023-09-30T22:29:57.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.956+0000 7f4499f67700 1 -- start start 2023-09-30T22:29:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4499f67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f4494131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4499f67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f4494139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4499f67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449409e130 con 0x7f449409d9f0 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4499f67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44941312b0 con 0x7f449406ae80 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4499f67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f449409f6e0 con 0x7f44941316c0 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44937fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f4494131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4492ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f449409d9f0 0x7f449406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55888/0 (socket says 172.21.15.32:55888) 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44937fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f4494131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57594/0 (socket says 172.21.15.32:57594) 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4492ffd700 1 -- 172.21.15.32:0/260100402 learned_addr learned my addr 172.21.15.32:0/260100402 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4493fff700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f4494139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44927fc700 1 -- 172.21.15.32:0/260100402 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062490565 0 0) 0x7f4488001110 con 0x7f449409d9f0 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44937fe700 1 -- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 msgr2=0x7f4494139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:57.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44937fe700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f4494139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44937fe700 1 -- 172.21.15.32:0/260100402 >> v1:172.21.15.32:6789/0 conn(0x7f449409d9f0 legacy=0x7f449406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f44937fe700 1 -- 172.21.15.32:0/260100402 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449409e760 con 0x7f44941316c0 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.957+0000 7f4493fff700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f4494139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f44937fe700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f4494131a90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4484009dc0 tx=0x7f448400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a675078c00bed2f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f44927fc700 1 -- 172.21.15.32:0/260100402 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400e070 con 0x7f44941316c0 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f44927fc700 1 -- 172.21.15.32:0/260100402 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448400bb20 con 0x7f44941316c0 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f44927fc700 1 -- 172.21.15.32:0/260100402 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484012430 con 0x7f44941316c0 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f4499f67700 1 -- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 msgr2=0x7f4494131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f4499f67700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f4494131a90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4484009dc0 tx=0x7f448400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.958+0000 7f4499f67700 1 -- 172.21.15.32:0/260100402 shutdown_connections 2023-09-30T22:29:57.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.959+0000 7f4499f67700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f4494139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.959+0000 7f4499f67700 1 --2- 172.21.15.32:0/260100402 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f4494131a90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.959+0000 7f4499f67700 1 -- 172.21.15.32:0/260100402 >> 172.21.15.32:0/260100402 conn(0x7f4494066f50 msgr2=0x7f449409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.959+0000 7f4499f67700 1 -- 172.21.15.32:0/260100402 shutdown_connections 2023-09-30T22:29:57.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.959+0000 7f4499f67700 1 -- 172.21.15.32:0/260100402 wait complete. 2023-09-30T22:29:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 Processor -- start 2023-09-30T22:29:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 -- start start 2023-09-30T22:29:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f449406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f449409d9f0 0x7f449406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f449406f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:57.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494071d50 con 0x7f449409d9f0 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449412b410 con 0x7f449406ae80 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4499f67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4494070dd0 con 0x7f44941316c0 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4493fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f449406f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f44937fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f449406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4492ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f449409d9f0 0x7f449406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4493fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f449406f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57598/0 (socket says 172.21.15.32:57598) 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f44937fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f449406e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45584/0 (socket says 172.21.15.32:45584) 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.960+0000 7f4493fff700 1 -- 172.21.15.32:0/3317984701 learned_addr learned my addr 172.21.15.32:0/3317984701 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f44937fe700 1 -- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 msgr2=0x7f449406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f44937fe700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f449406f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f44937fe700 1 -- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f449409d9f0 msgr2=0x7f449406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f44937fe700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f449409d9f0 0x7f449406ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f44937fe700 1 -- 172.21.15.32:0/3317984701 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c0094a0 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f4492ffd700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f449409d9f0 0x7f449406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f44937fe700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f449406e8e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4484016000 tx=0x7f4484004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484004900 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484020070 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4484009a70 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840276c0 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.961+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449412b900 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.962+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4484012430 con 0x7f449406ae80 2023-09-30T22:29:57.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.962+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4484022070 con 0x7f449406ae80 2023-09-30T22:29:57.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.962+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4494074ea0 con 0x7f449406ae80 2023-09-30T22:29:57.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:57.972+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f448401b400 con 0x7f449406ae80 2023-09-30T22:29:58.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.142+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbf040747f0 con 0x7fbf04138d80 2023-09-30T22:29:58.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.143+0000 7fbef2ffd700 1 -- 172.21.15.32:0/1863719309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbefc017070 con 0x7fbf04138d80 2023-09-30T22:29:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04138d80 msgr2=0x7fbf04121410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04138d80 0x7fbf04121410 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009fd0 tx=0x7fbefc00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 shutdown_connections 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04138d80 0x7fbf04121410 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf0409d630 0x7fbf04120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 --2- 172.21.15.32:0/1863719309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf040a0a30 0x7fbf04120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 >> 172.21.15.32:0/1863719309 conn(0x7fbf040a3050 msgr2=0x7fbf04139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 shutdown_connections 2023-09-30T22:29:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.145+0000 7fbf08e8f700 1 -- 172.21.15.32:0/1863719309 wait complete. 2023-09-30T22:29:58.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.181+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4494070ff0 con 0x7f449406ae80 2023-09-30T22:29:58.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.181+0000 7f447bfff700 1 -- 172.21.15.32:0/3317984701 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4484011390 con 0x7f449406ae80 2023-09-30T22:29:58.182 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 msgr2=0x7f449406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f449406e8e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4484016000 tx=0x7f4484004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 shutdown_connections 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44941316c0 0x7f449406f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f449406ae80 0x7f449406e8e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 --2- 172.21.15.32:0/3317984701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f449409d9f0 0x7f449406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 >> 172.21.15.32:0/3317984701 conn(0x7f4494066f50 msgr2=0x7f44940a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 shutdown_connections 2023-09-30T22:29:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.184+0000 7f4499f67700 1 -- 172.21.15.32:0/3317984701 wait complete. 2023-09-30T22:29:58.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:29:58.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:58.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 Processor -- start 2023-09-30T22:29:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 -- start start 2023-09-30T22:29:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 0x7f31f409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 0x7f31f4133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f5b0 0x7f31f41341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f406bec0 con 0x7f31f409e850 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40a0030 con 0x7f31f409f5b0 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.776+0000 7f31f9702700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40a0830 con 0x7f31f409f160 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 0x7f31f409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 0x7f31f4133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f5b0 0x7f31f41341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 0x7f31f4133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57610/0 (socket says 172.21.15.32:57610) 2023-09-30T22:29:58.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 0x7f31f409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39930/0 (socket says 172.21.15.32:39930) 2023-09-30T22:29:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f27fc700 1 -- 172.21.15.32:0/4127807614 learned_addr learned my addr 172.21.15.32:0/4127807614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 -- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 msgr2=0x7f31f4133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 --2- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 0x7f31f4133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 -- 172.21.15.32:0/4127807614 >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 msgr2=0x7f31f409ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 --2- 172.21.15.32:0/4127807614 >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 0x7f31f409ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 -- 172.21.15.32:0/4127807614 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f406caa0 con 0x7f31f409f5b0 2023-09-30T22:29:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f27fc700 1 --2- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 0x7f31f4133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f2ffd700 1 --2- 172.21.15.32:0/4127807614 >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 0x7f31f409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f37fe700 1 --2- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f5b0 0x7f31f41341a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009d90 tx=0x7f31ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7539464a340ff6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f1ffb700 1 -- 172.21.15.32:0/4127807614 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00e070 con 0x7f31f409f5b0 2023-09-30T22:29:58.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f1ffb700 1 -- 172.21.15.32:0/4127807614 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00bb20 con 0x7f31f409f5b0 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.777+0000 7f31f1ffb700 1 -- 172.21.15.32:0/4127807614 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec012430 con 0x7f31f409f5b0 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 -- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f5b0 msgr2=0x7f31f41341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 --2- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f5b0 0x7f31f41341a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009d90 tx=0x7f31ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 -- 172.21.15.32:0/4127807614 shutdown_connections 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 --2- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f5b0 0x7f31f41341a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 --2- 172.21.15.32:0/4127807614 >> v2:172.21.15.32:3300/0 conn(0x7f31f409e850 0x7f31f409ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 --2- 172.21.15.32:0/4127807614 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f160 0x7f31f4133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 -- 172.21.15.32:0/4127807614 >> 172.21.15.32:0/4127807614 conn(0x7f31f40a3f70 msgr2=0x7f31f40a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 -- 172.21.15.32:0/4127807614 shutdown_connections 2023-09-30T22:29:58.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.778+0000 7f31f9702700 1 -- 172.21.15.32:0/4127807614 wait complete. 2023-09-30T22:29:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.779+0000 7f31f9702700 1 Processor -- start 2023-09-30T22:29:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.779+0000 7f31f9702700 1 -- start start 2023-09-30T22:29:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f9702700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 0x7f31f4120370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f9702700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f160 0x7f31f41208b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f9702700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f5b0 0x7f31f4120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f9702700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f4123e90 con 0x7f31f409e850 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f9702700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f4122020 con 0x7f31f409f160 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f9702700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40a3470 con 0x7f31f409f5b0 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 0x7f31f4120370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 0x7f31f4120370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39934/0 (socket says 172.21.15.32:39934) 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f5b0 0x7f31f4120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f160 0x7f31f41208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 -- 172.21.15.32:0/2796441285 learned_addr learned my addr 172.21.15.32:0/2796441285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 -- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f5b0 msgr2=0x7f31f4120df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f5b0 0x7f31f4120df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 -- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f160 msgr2=0x7f31f41208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f160 0x7f31f41208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 -- 172.21.15.32:0/2796441285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40094a0 con 0x7f31f409e850 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f37fe700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f5b0 0x7f31f4120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.780+0000 7f31f2ffd700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 0x7f31f4120370 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f31e400ebe0 tx=0x7f31e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.781+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400c850 con 0x7f31f409e850 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.781+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400c9b0 con 0x7f31f409e850 2023-09-30T22:29:58.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.781+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400cd40 con 0x7f31f409e850 2023-09-30T22:29:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.781+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e8009550 con 0x7f31f409e850 2023-09-30T22:29:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.781+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec009a70 con 0x7f31f409e850 2023-09-30T22:29:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.781+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31e4013470 con 0x7f31f409e850 2023-09-30T22:29:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.782+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f31e403fb30 con 0x7f31f409e850 2023-09-30T22:29:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.782+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f40747f0 con 0x7f31f409e850 2023-09-30T22:29:58.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.787+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31e4017020 con 0x7f31f409e850 2023-09-30T22:29:58.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.994+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31f4072820 con 0x7f31f409e850 2023-09-30T22:29:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.995+0000 7f31db7fe700 1 -- 172.21.15.32:0/2796441285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31e400cea0 con 0x7f31f409e850 2023-09-30T22:29:58.996 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:29:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 msgr2=0x7f31f4120370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 0x7f31f4120370 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f31e400ebe0 tx=0x7f31e400eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 shutdown_connections 2023-09-30T22:29:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f409f5b0 0x7f31f4120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f409e850 0x7f31f4120370 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 --2- 172.21.15.32:0/2796441285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f409f160 0x7f31f41208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 >> 172.21.15.32:0/2796441285 conn(0x7f31f40a3f70 msgr2=0x7f31f41b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 shutdown_connections 2023-09-30T22:29:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:58.997+0000 7f31f9702700 1 -- 172.21.15.32:0/2796441285 wait complete. 2023-09-30T22:29:59.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:29:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.081+0000 7fc86f4f2700 1 Processor -- start 2023-09-30T22:29:59.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.081+0000 7fc86f4f2700 1 -- start start 2023-09-30T22:29:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86f4f2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 0x7fc8680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86f4f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 0x7fc8680a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86f4f2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680a2a70 con 0x7fc868133930 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86f4f2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680a01d0 con 0x7fc86809d630 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86f4f2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc86809e2b0 con 0x7fc8680a0a30 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86d28e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 0x7fc8680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86ca8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 0x7fc8680a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86da8f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc868133930 0x7fc868133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55902/0 (socket says 172.21.15.32:55902) 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86d28e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 0x7fc8680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57624/0 (socket says 172.21.15.32:57624) 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86ca8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 0x7fc8680a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39946/0 (socket says 172.21.15.32:39946) 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86da8f700 1 -- 172.21.15.32:0/1696092021 learned_addr learned my addr 172.21.15.32:0/1696092021 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86d28e700 1 -- 172.21.15.32:0/1696092021 >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 msgr2=0x7fc8680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86d28e700 1 --2- 172.21.15.32:0/1696092021 >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 0x7fc8680a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86d28e700 1 -- 172.21.15.32:0/1696092021 >> v1:172.21.15.32:6789/0 conn(0x7fc868133930 legacy=0x7fc868133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86d28e700 1 -- 172.21.15.32:0/1696092021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86809dc50 con 0x7fc8680a0a30 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc85ffff700 1 -- 172.21.15.32:0/1696092021 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769295037 0 0) 0x7fc8680a2a70 con 0x7fc868133930 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.082+0000 7fc86ca8d700 1 --2- 172.21.15.32:0/1696092021 >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 0x7fc8680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86d28e700 1 --2- 172.21.15.32:0/1696092021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 0x7fc8680a0e00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc864009840 tx=0x7fc864009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e5a86b45930e3b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc85ffff700 1 -- 172.21.15.32:0/1696092021 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864011830 con 0x7fc8680a0a30 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc85ffff700 1 -- 172.21.15.32:0/1696092021 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864011990 con 0x7fc8680a0a30 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc85ffff700 1 -- 172.21.15.32:0/1696092021 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864011d20 con 0x7fc8680a0a30 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 -- 172.21.15.32:0/1696092021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 msgr2=0x7fc8680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/1696092021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 0x7fc8680a0e00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc864009840 tx=0x7fc864009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 -- 172.21.15.32:0/1696092021 shutdown_connections 2023-09-30T22:29:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/1696092021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8680a0a30 0x7fc8680a0e00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/1696092021 >> v2:172.21.15.32:3300/0 conn(0x7fc86809d630 0x7fc8680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 -- 172.21.15.32:0/1696092021 >> 172.21.15.32:0/1696092021 conn(0x7fc8680a3050 msgr2=0x7fc8680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 -- 172.21.15.32:0/1696092021 shutdown_connections 2023-09-30T22:29:59.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.083+0000 7fc86f4f2700 1 -- 172.21.15.32:0/1696092021 wait complete. 2023-09-30T22:29:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.084+0000 7fc86f4f2700 1 Processor -- start 2023-09-30T22:29:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 -- start start 2023-09-30T22:29:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86809d630 0x7fc8681c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8680a0a30 0x7fc8681c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc868133930 0x7fc8681c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868121e90 con 0x7fc8680a0a30 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc868121290 con 0x7fc86809d630 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86f4f2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc868120520 con 0x7fc868133930 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86d28e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86809d630 0x7fc8681c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86ca8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8680a0a30 0x7fc8681c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86da8f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc868133930 0x7fc8681c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86d28e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86809d630 0x7fc8681c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45602/0 (socket says 172.21.15.32:45602) 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.085+0000 7fc86d28e700 1 -- 172.21.15.32:0/905498408 learned_addr learned my addr 172.21.15.32:0/905498408 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86ca8d700 1 -- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc868133930 msgr2=0x7fc8681c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86ca8d700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc868133930 0x7fc8681c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86ca8d700 1 -- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86809d630 msgr2=0x7fc8681c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86ca8d700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86809d630 0x7fc8681c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86ca8d700 1 -- 172.21.15.32:0/905498408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580094a0 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86da8f700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc868133930 0x7fc8681c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86ca8d700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8680a0a30 0x7fc8681c8430 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc85800e980 tx=0x7fc85800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800c690 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85800c7f0 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8640094a0 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800cb80 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.086+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8680a0410 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.087+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc85800cce0 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.087+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc85803f720 con 0x7fc8680a0a30 2023-09-30T22:29:59.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.087+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868133420 con 0x7fc8680a0a30 2023-09-30T22:29:59.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.092+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8580232f0 con 0x7fc8680a0a30 2023-09-30T22:29:59.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:59.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.302+0000 7f2da93dd700 1 Processor -- start 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.302+0000 7f2da93dd700 1 -- start start 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da93dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409d6f0 0x7f2da409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da93dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da93dd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da409f130 con 0x7f2da409e410 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da93dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da409f520 con 0x7f2da409d6f0 2023-09-30T22:29:59.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da93dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2da409e930 con 0x7f2da409e000 2023-09-30T22:29:59.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2da409e410 0x7f2da4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55912/0 (socket says 172.21.15.32:55912) 2023-09-30T22:29:59.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409d6f0 0x7f2da409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da37fe700 1 -- 172.21.15.32:0/1049038811 learned_addr learned my addr 172.21.15.32:0/1049038811 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:29:59.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da27fc700 1 -- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409d6f0 msgr2=0x7f2da409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da27fc700 1 --2- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409d6f0 0x7f2da409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da27fc700 1 -- 172.21.15.32:0/1049038811 >> v1:172.21.15.32:6789/0 conn(0x7f2da409e410 legacy=0x7f2da4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da1ffb700 1 -- 172.21.15.32:0/1049038811 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554464874 0 0) 0x7f2da409f130 con 0x7f2da409e410 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da27fc700 1 -- 172.21.15.32:0/1049038811 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da409b2d0 con 0x7f2da409e000 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.303+0000 7f2da2ffd700 1 --2- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409d6f0 0x7f2da409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.304+0000 7f2da27fc700 1 --2- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4130660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800bb40 tx=0x7f2d9800be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92dabd0e6236f25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.304+0000 7f2da1ffb700 1 -- 172.21.15.32:0/1049038811 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800dc30 con 0x7f2da409e000 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.304+0000 7f2da1ffb700 1 -- 172.21.15.32:0/1049038811 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800dd90 con 0x7f2da409e000 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.304+0000 7f2da1ffb700 1 -- 172.21.15.32:0/1049038811 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98010690 con 0x7f2da409e000 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 -- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 msgr2=0x7f2da4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 --2- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4130660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800bb40 tx=0x7f2d9800be50 comp rx=0 tx=0).stop 2023-09-30T22:29:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 -- 172.21.15.32:0/1049038811 shutdown_connections 2023-09-30T22:29:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 --2- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409d6f0 0x7f2da409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 --2- 172.21.15.32:0/1049038811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4130660 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 -- 172.21.15.32:0/1049038811 >> 172.21.15.32:0/1049038811 conn(0x7f2da4066f50 msgr2=0x7f2da409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 -- 172.21.15.32:0/1049038811 shutdown_connections 2023-09-30T22:29:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.305+0000 7f2da93dd700 1 -- 172.21.15.32:0/1049038811 wait complete. 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.306+0000 7f2da93dd700 1 Processor -- start 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.306+0000 7f2da93dd700 1 -- start start 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da93dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2da409d6f0 0x7f2da4122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da93dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4122ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da93dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409e410 0x7f2da4122ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da93dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4125db0 con 0x7f2da409d6f0 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da93dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da4126090 con 0x7f2da409e410 2023-09-30T22:29:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da93dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40720b0 con 0x7f2da409e000 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409e410 0x7f2da4122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2da409d6f0 0x7f2da4122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409e410 0x7f2da4122ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45618/0 (socket says 172.21.15.32:45618) 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da37fe700 1 -- 172.21.15.32:0/482531728 learned_addr learned my addr 172.21.15.32:0/482531728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da27fc700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4122ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 -- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 msgr2=0x7f2da4122ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4122ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 -- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409e410 msgr2=0x7f2da4122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409e410 0x7f2da4122ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 -- 172.21.15.32:0/482531728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940094a0 con 0x7f2da409d6f0 2023-09-30T22:29:59.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.307+0000 7f2da2ffd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2da409d6f0 0x7f2da4122570 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e9b0 tx=0x7f2d9400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.308+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400ee50 con 0x7f2da409d6f0 2023-09-30T22:29:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.308+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9400c850 con 0x7f2da409d6f0 2023-09-30T22:29:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.308+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9800b740 con 0x7f2da409d6f0 2023-09-30T22:29:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.308+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400cc00 con 0x7f2da409d6f0 2023-09-30T22:29:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.308+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da409c750 con 0x7f2da409d6f0 2023-09-30T22:29:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.309+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da4125a50 con 0x7f2da409d6f0 2023-09-30T22:29:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.309+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d94013470 con 0x7f2da409d6f0 2023-09-30T22:29:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.310+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2d9403fe90 con 0x7f2da409d6f0 2023-09-30T22:29:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.313+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc8680a1800 con 0x7fc8680a0a30 2023-09-30T22:29:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.316+0000 7fc85dffb700 1 -- 172.21.15.32:0/905498408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc858017020 con 0x7fc8680a0a30 2023-09-30T22:29:59.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.316+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d9400cd60 con 0x7f2da409d6f0 2023-09-30T22:29:59.317 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:29:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8680a0a30 msgr2=0x7fc8681c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8680a0a30 0x7fc8681c8430 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc85800e980 tx=0x7fc85800ec90 comp rx=0 tx=0).stop 2023-09-30T22:29:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 shutdown_connections 2023-09-30T22:29:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc868133930 0x7fc8681c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc86809d630 0x7fc8681c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 --2- 172.21.15.32:0/905498408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8680a0a30 0x7fc8681c8430 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 >> 172.21.15.32:0/905498408 conn(0x7fc8680a3050 msgr2=0x7fc8681bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 shutdown_connections 2023-09-30T22:29:59.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.318+0000 7fc86f4f2700 1 -- 172.21.15.32:0/905498408 wait complete. 2023-09-30T22:29:59.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:29:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.355+0000 7f34c27ba700 1 Processor -- start 2023-09-30T22:29:59.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 -- start start 2023-09-30T22:29:59.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34bc09e410 0x7f34bc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34bc06b460 con 0x7f34bc09e410 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc09b120 con 0x7f34bc09e000 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c27ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc09a7e0 con 0x7f34bc09d6f0 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34c0d57700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34bc09e410 0x7f34bc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57662/0 (socket says 172.21.15.32:57662) 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45634/0 (socket says 172.21.15.32:45634) 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.356+0000 7f34bbfff700 1 -- 172.21.15.32:0/127665590 learned_addr learned my addr 172.21.15.32:0/127665590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:59.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bbfff700 1 -- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 msgr2=0x7f34bc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bbfff700 1 --2- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bbfff700 1 -- 172.21.15.32:0/127665590 >> v2:172.21.15.32:3300/0 conn(0x7f34bc09e410 msgr2=0x7f34bc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bbfff700 1 --2- 172.21.15.32:0/127665590 >> v2:172.21.15.32:3300/0 conn(0x7f34bc09e410 0x7f34bc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bbfff700 1 -- 172.21.15.32:0/127665590 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc09a500 con 0x7f34bc09d6f0 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bb7fe700 1 --2- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34c0d57700 1 --2- 172.21.15.32:0/127665590 >> v2:172.21.15.32:3300/0 conn(0x7f34bc09e410 0x7f34bc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34bbfff700 1 --2- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc09dac0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009920 tx=0x7f34ac009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e94e27be6f0430c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34baffd700 1 -- 172.21.15.32:0/127665590 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac011830 con 0x7f34bc09d6f0 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34baffd700 1 -- 172.21.15.32:0/127665590 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac011990 con 0x7f34bc09d6f0 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.357+0000 7f34baffd700 1 -- 172.21.15.32:0/127665590 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac011d20 con 0x7f34bc09d6f0 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 -- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 msgr2=0x7f34bc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 --2- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc09dac0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009920 tx=0x7f34ac009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 -- 172.21.15.32:0/127665590 shutdown_connections 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 --2- 172.21.15.32:0/127665590 >> v2:172.21.15.32:3300/0 conn(0x7f34bc09e410 0x7f34bc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 --2- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc09dac0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 --2- 172.21.15.32:0/127665590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 -- 172.21.15.32:0/127665590 >> 172.21.15.32:0/127665590 conn(0x7f34bc066f50 msgr2=0x7f34bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 -- 172.21.15.32:0/127665590 shutdown_connections 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.358+0000 7f34c27ba700 1 -- 172.21.15.32:0/127665590 wait complete. 2023-09-30T22:29:59.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.359+0000 7f34c27ba700 1 Processor -- start 2023-09-30T22:29:59.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.359+0000 7f34c27ba700 1 -- start start 2023-09-30T22:29:59.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.359+0000 7f34c27ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.359+0000 7f34c27ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.359+0000 7f34c27ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34bc09e410 0x7f34bc1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c27ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1a9170 con 0x7f34bc09e410 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c27ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1a7a90 con 0x7f34bc09e000 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c27ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1a9620 con 0x7f34bc09d6f0 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c0d57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34bc09e410 0x7f34bc1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c0d57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34bc09e410 0x7f34bc1a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39974/0 (socket says 172.21.15.32:39974) 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c0d57700 1 -- 172.21.15.32:0/3193613999 learned_addr learned my addr 172.21.15.32:0/3193613999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45642/0 (socket says 172.21.15.32:45642) 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bbfff700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 -- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 msgr2=0x7f34bc1a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc1a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 -- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34bc09e410 msgr2=0x7f34bc1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34bc09e410 0x7f34bc1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 -- 172.21.15.32:0/3193613999 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00094a0 con 0x7f34bc09e000 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bbfff700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34bb7fe700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1a6370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34b000e9b0 tx=0x7f34b000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000ee50 con 0x7f34bc09e000 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000c850 con 0x7f34bc09e000 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b4009a70 con 0x7f34bc09e000 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.360+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000cbe0 con 0x7f34bc09e000 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.361+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac009550 con 0x7f34bc09e000 2023-09-30T22:29:59.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.361+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34b000cd40 con 0x7f34bc09e000 2023-09-30T22:29:59.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.362+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f34b003fae0 con 0x7f34bc09e000 2023-09-30T22:29:59.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.362+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34bc074ea0 con 0x7f34bc09e000 2023-09-30T22:29:59.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.368+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34b001b3b0 con 0x7f34bc09e000 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.540+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2da4125c10 con 0x7f2da409d6f0 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.540+0000 7f2d8b7fe700 1 -- 172.21.15.32:0/482531728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2da4125c10 con 0x7f2da409d6f0 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:29:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2da409d6f0 msgr2=0x7f2da4122570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2da409d6f0 0x7f2da4122570 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e9b0 tx=0x7f2d9400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 shutdown_connections 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2da409e410 0x7f2da4122ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2da409d6f0 0x7f2da4122570 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 --2- 172.21.15.32:0/482531728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2da409e000 0x7f2da4122ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 >> 172.21.15.32:0/482531728 conn(0x7f2da4066f50 msgr2=0x7f2da40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 shutdown_connections 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.543+0000 7f2da93dd700 1 -- 172.21.15.32:0/482531728 wait complete. 2023-09-30T22:29:59.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:29:59.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.571+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f34bc1a8c40 con 0x7f34bc09e000 2023-09-30T22:29:59.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.572+0000 7f34b8ff9700 1 -- 172.21.15.32:0/3193613999 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34b002a090 con 0x7f34bc09e000 2023-09-30T22:29:59.573 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:29:59.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 msgr2=0x7f34bc1a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1a6370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f34b000e9b0 tx=0x7f34b000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 shutdown_connections 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34bc09e410 0x7f34bc1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34bc09d6f0 0x7f34bc1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 --2- 172.21.15.32:0/3193613999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34bc09e000 0x7f34bc1a6370 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 >> 172.21.15.32:0/3193613999 conn(0x7f34bc066f50 msgr2=0x7f34bc1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 shutdown_connections 2023-09-30T22:29:59.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:29:59.574+0000 7f34c27ba700 1 -- 172.21.15.32:0/3193613999 wait complete. 2023-09-30T22:29:59.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:00.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 Processor -- start 2023-09-30T22:30:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 -- start start 2023-09-30T22:30:00.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 0x7fa70409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa704134100 0x7fa70413bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70409d2f0 con 0x7fa70409e000 2023-09-30T22:30:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70409a810 con 0x7fa704134100 2023-09-30T22:30:00.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.161+0000 7fa709e29700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa70409a470 con 0x7fa70409d6f0 2023-09-30T22:30:00.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa7037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 0x7fa70409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa704134100 0x7fa70413bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa704134100 0x7fa70413bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39988/0 (socket says 172.21.15.32:39988) 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa702ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa70409e000 0x7fa7041339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55928/0 (socket says 172.21.15.32:55928) 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 -- 172.21.15.32:0/2609030780 learned_addr learned my addr 172.21.15.32:0/2609030780 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa7037fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 0x7fa70409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57666/0 (socket says 172.21.15.32:57666) 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 -- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 msgr2=0x7fa70409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 --2- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 0x7fa70409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 -- 172.21.15.32:0/2609030780 >> v1:172.21.15.32:6789/0 conn(0x7fa70409e000 legacy=0x7fa7041339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 -- 172.21.15.32:0/2609030780 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70409ec40 con 0x7fa704134100 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa7027fc700 1 -- 172.21.15.32:0/2609030780 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952608504 0 0) 0x7fa70409d2f0 con 0x7fa70409e000 2023-09-30T22:30:00.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa7037fe700 1 --2- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 0x7fa70409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa703fff700 1 --2- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa704134100 0x7fa70413bc50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009e50 tx=0x7fa6fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d2efb2bda24e45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa7027fc700 1 -- 172.21.15.32:0/2609030780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00e070 con 0x7fa704134100 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.162+0000 7fa7027fc700 1 -- 172.21.15.32:0/2609030780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc00b910 con 0x7fa704134100 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa7027fc700 1 -- 172.21.15.32:0/2609030780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00bcc0 con 0x7fa704134100 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 -- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa704134100 msgr2=0x7fa70413bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 --2- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa704134100 0x7fa70413bc50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009e50 tx=0x7fa6fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 -- 172.21.15.32:0/2609030780 shutdown_connections 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 --2- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa704134100 0x7fa70413bc50 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 --2- 172.21.15.32:0/2609030780 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa70409d6f0 0x7fa70409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 -- 172.21.15.32:0/2609030780 >> 172.21.15.32:0/2609030780 conn(0x7fa704066f50 msgr2=0x7fa70409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 -- 172.21.15.32:0/2609030780 shutdown_connections 2023-09-30T22:30:00.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.163+0000 7fa709e29700 1 -- 172.21.15.32:0/2609030780 wait complete. 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.164+0000 7fa709e29700 1 Processor -- start 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 -- start start 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa70409d6f0 0x7fa7041226c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 0x7fa704122c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa704134100 0x7fa704123140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7041259f0 con 0x7fa70409e000 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa704133610 con 0x7fa70409d6f0 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa709e29700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa70409c640 con 0x7fa704134100 2023-09-30T22:30:00.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa7037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa70409d6f0 0x7fa7041226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa702ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 0x7fa704122c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa7037fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa70409d6f0 0x7fa7041226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45648/0 (socket says 172.21.15.32:45648) 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa702ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 0x7fa704122c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40002/0 (socket says 172.21.15.32:40002) 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa7037fe700 1 -- 172.21.15.32:0/3971602649 learned_addr learned my addr 172.21.15.32:0/3971602649 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.165+0000 7fa703fff700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa704134100 0x7fa704123140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa702ffd700 1 -- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa704134100 msgr2=0x7fa704123140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa702ffd700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa704134100 0x7fa704123140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa702ffd700 1 -- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa70409d6f0 msgr2=0x7fa7041226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa702ffd700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa70409d6f0 0x7fa7041226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa702ffd700 1 -- 172.21.15.32:0/3971602649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40094a0 con 0x7fa70409e000 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa702ffd700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 0x7fa704122c00 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa70406d650 tx=0x7fa6f800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8004c80 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f8004de0 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc009b30 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8007310 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.166+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa704133950 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.167+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6f8007470 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.167+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa6f803f1d0 con 0x7fa70409e000 2023-09-30T22:30:00.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.167+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa704074ea0 con 0x7fa70409e000 2023-09-30T22:30:00.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.172+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6f801bb70 con 0x7fa70409e000 2023-09-30T22:30:00.335 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.382+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa70406c360 con 0x7fa70409e000 2023-09-30T22:30:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.382+0000 7fa6ebfff700 1 -- 172.21.15.32:0/3971602649 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6f801b760 con 0x7fa70409e000 2023-09-30T22:30:00.384 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 msgr2=0x7fa704122c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 0x7fa704122c00 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa70406d650 tx=0x7fa6f800c750 comp rx=0 tx=0).stop 2023-09-30T22:30:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 shutdown_connections 2023-09-30T22:30:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa704134100 0x7fa704123140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa70409d6f0 0x7fa7041226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 --2- 172.21.15.32:0/3971602649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70409e000 0x7fa704122c00 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 >> 172.21.15.32:0/3971602649 conn(0x7fa704066f50 msgr2=0x7fa7040a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 shutdown_connections 2023-09-30T22:30:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.385+0000 7fa709e29700 1 -- 172.21.15.32:0/3971602649 wait complete. 2023-09-30T22:30:00.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 Processor -- start 2023-09-30T22:30:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 -- start start 2023-09-30T22:30:00.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f368009c730 0x7f3680136720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3680136c60 0x7f3680139040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368009d8a0 con 0x7f368009bc50 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368009cb00 con 0x7f3680136c60 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367ffff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f368006b490 con 0x7f368009c730 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f368009c730 0x7f3680136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.476+0000 7f367effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f368009bc50 0x7f368009c020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41664/0 (socket says 172.21.15.32:41664) 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367effd700 1 -- 172.21.15.32:0/365922157 learned_addr learned my addr 172.21.15.32:0/365922157 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367f7fe700 1 --2- 172.21.15.32:0/365922157 >> v2:172.21.15.32:3300/0 conn(0x7f3680136c60 0x7f3680139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367dffb700 1 -- 172.21.15.32:0/365922157 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528637410 0 0) 0x7f368009d8a0 con 0x7f368009bc50 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367e7fc700 1 -- 172.21.15.32:0/365922157 >> v2:172.21.15.32:3300/0 conn(0x7f3680136c60 msgr2=0x7f3680139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367e7fc700 1 --2- 172.21.15.32:0/365922157 >> v2:172.21.15.32:3300/0 conn(0x7f3680136c60 0x7f3680139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367e7fc700 1 -- 172.21.15.32:0/365922157 >> v1:172.21.15.32:6789/0 conn(0x7f368009bc50 legacy=0x7f368009c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367e7fc700 1 -- 172.21.15.32:0/365922157 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368009e5e0 con 0x7f368009c730 2023-09-30T22:30:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367f7fe700 1 --2- 172.21.15.32:0/365922157 >> v2:172.21.15.32:3300/0 conn(0x7f3680136c60 0x7f3680139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367e7fc700 1 --2- 172.21.15.32:0/365922157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f368009c730 0x7f3680136720 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3670009850 tx=0x7f3670009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9b7f2941077c1e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367dffb700 1 -- 172.21.15.32:0/365922157 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670005e30 con 0x7f368009c730 2023-09-30T22:30:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367dffb700 1 -- 172.21.15.32:0/365922157 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670006030 con 0x7f368009c730 2023-09-30T22:30:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.477+0000 7f367dffb700 1 -- 172.21.15.32:0/365922157 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700063c0 con 0x7f368009c730 2023-09-30T22:30:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 -- 172.21.15.32:0/365922157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f368009c730 msgr2=0x7f3680136720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 --2- 172.21.15.32:0/365922157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f368009c730 0x7f3680136720 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3670009850 tx=0x7f3670009b60 comp rx=0 tx=0).stop 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 -- 172.21.15.32:0/365922157 shutdown_connections 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 --2- 172.21.15.32:0/365922157 >> v2:172.21.15.32:3300/0 conn(0x7f3680136c60 0x7f3680139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 --2- 172.21.15.32:0/365922157 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f368009c730 0x7f3680136720 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 -- 172.21.15.32:0/365922157 >> 172.21.15.32:0/365922157 conn(0x7f36800a2e20 msgr2=0x7f36800a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 -- 172.21.15.32:0/365922157 shutdown_connections 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.478+0000 7f367ffff700 1 -- 172.21.15.32:0/365922157 wait complete. 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 Processor -- start 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 -- start start 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 0x7f368011f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f368009c730 0x7f368011f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3680136c60 0x7f368011fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680134ce0 con 0x7f368009bc50 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3680120ed0 con 0x7f368009c730 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.480+0000 7f367ffff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36801229f0 con 0x7f3680136c60 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 0x7f368011f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3680136c60 0x7f368011fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 0x7f368011f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34962/0 (socket says 172.21.15.32:34962) 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367effd700 1 -- 172.21.15.32:0/1679833255 learned_addr learned my addr 172.21.15.32:0/1679833255 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367e7fc700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f368009c730 0x7f368011f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 -- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f368009c730 msgr2=0x7f368011f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f368009c730 0x7f368011f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 -- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 msgr2=0x7f368011f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 0x7f368011f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 -- 172.21.15.32:0/1679833255 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678009a70 con 0x7f3680136c60 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367effd700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 0x7f368011f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367f7fe700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3680136c60 0x7f368011fc30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f367800ef90 tx=0x7f367800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800ca90 con 0x7f3680136c60 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800f040 con 0x7f3680136c60 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3670009430 con 0x7f3680136c60 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800ce30 con 0x7f3680136c60 2023-09-30T22:30:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.481+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680135080 con 0x7f3680136c60 2023-09-30T22:30:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.482+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3678014480 con 0x7f3680136c60 2023-09-30T22:30:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.482+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f367803fd60 con 0x7f3680136c60 2023-09-30T22:30:00.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.482+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368004f2a0 con 0x7f3680136c60 2023-09-30T22:30:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.487+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3678012930 con 0x7f3680136c60 2023-09-30T22:30:00.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:00.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.691+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f368006c0a0 con 0x7f3680136c60 2023-09-30T22:30:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.691+0000 7f36677fe700 1 -- 172.21.15.32:0/1679833255 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3678012340 con 0x7f3680136c60 2023-09-30T22:30:00.693 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:00.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3680136c60 msgr2=0x7f368011fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3680136c60 0x7f368011fc30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f367800ef90 tx=0x7f367800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 shutdown_connections 2023-09-30T22:30:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3680136c60 0x7f368011fc30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f368009bc50 0x7f368011f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 --2- 172.21.15.32:0/1679833255 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f368009c730 0x7f368011f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 >> 172.21.15.32:0/1679833255 conn(0x7f36800a2e20 msgr2=0x7f36800a19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 shutdown_connections 2023-09-30T22:30:00.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.694+0000 7f367ffff700 1 -- 172.21.15.32:0/1679833255 wait complete. 2023-09-30T22:30:00.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:00.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.708+0000 7f6785ff1700 1 Processor -- start 2023-09-30T22:30:00.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.708+0000 7f6785ff1700 1 -- start start 2023-09-30T22:30:00.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f6785ff1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 0x7f678009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f6785ff1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f678009e000 0x7f67801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f6785ff1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009e410 0x7f6780133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f6785ff1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678006b460 con 0x7f678009e000 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f6785ff1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678009b120 con 0x7f678009e410 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f6785ff1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f678009a7e0 con 0x7f678009d6f0 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 0x7f678009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009e410 0x7f6780133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 0x7f678009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33496/0 (socket says 172.21.15.32:33496) 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 -- 172.21.15.32:0/936729689 learned_addr learned my addr 172.21.15.32:0/936729689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677effd700 1 --2- 172.21.15.32:0/936729689 >> v2:172.21.15.32:3300/0 conn(0x7f678009e000 0x7f67801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 -- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009e410 msgr2=0x7f6780133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 --2- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009e410 0x7f6780133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 -- 172.21.15.32:0/936729689 >> v2:172.21.15.32:3300/0 conn(0x7f678009e000 msgr2=0x7f67801339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 --2- 172.21.15.32:0/936729689 >> v2:172.21.15.32:3300/0 conn(0x7f678009e000 0x7f67801339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677f7fe700 1 -- 172.21.15.32:0/936729689 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678009a500 con 0x7f678009d6f0 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677ffff700 1 --2- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009e410 0x7f6780133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.709+0000 7f677effd700 1 --2- 172.21.15.32:0/936729689 >> v2:172.21.15.32:3300/0 conn(0x7f678009e000 0x7f67801339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.710+0000 7f677f7fe700 1 --2- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 0x7f678009dac0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6770009920 tx=0x7f6770009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d808be04ebc6ea88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.710+0000 7f677e7fc700 1 -- 172.21.15.32:0/936729689 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770011830 con 0x7f678009d6f0 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.710+0000 7f677e7fc700 1 -- 172.21.15.32:0/936729689 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6770011990 con 0x7f678009d6f0 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.710+0000 7f677e7fc700 1 -- 172.21.15.32:0/936729689 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770011d20 con 0x7f678009d6f0 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 -- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 msgr2=0x7f678009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 --2- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 0x7f678009dac0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6770009920 tx=0x7f6770009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 -- 172.21.15.32:0/936729689 shutdown_connections 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 --2- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009e410 0x7f6780133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 --2- 172.21.15.32:0/936729689 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009d6f0 0x7f678009dac0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 --2- 172.21.15.32:0/936729689 >> v2:172.21.15.32:3300/0 conn(0x7f678009e000 0x7f67801339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 -- 172.21.15.32:0/936729689 >> 172.21.15.32:0/936729689 conn(0x7f6780066f50 msgr2=0x7f678009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 -- 172.21.15.32:0/936729689 shutdown_connections 2023-09-30T22:30:00.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.711+0000 7f6785ff1700 1 -- 172.21.15.32:0/936729689 wait complete. 2023-09-30T22:30:00.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.712+0000 7f6785ff1700 1 Processor -- start 2023-09-30T22:30:00.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 -- start start 2023-09-30T22:30:00.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 0x7f67801a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 0x7f67801a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678009e410 0x7f67801a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67801aa270 con 0x7f678009e410 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67801a79a0 con 0x7f678009d6f0 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f6785ff1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67801a8c80 con 0x7f678009e000 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f677f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 0x7f67801a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.713+0000 7f677effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 0x7f67801a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678009e410 0x7f67801a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 0x7f67801a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33506/0 (socket says 172.21.15.32:33506) 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 0x7f67801a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60512/0 (socket says 172.21.15.32:60512) 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677effd700 1 -- 172.21.15.32:0/4102649511 learned_addr learned my addr 172.21.15.32:0/4102649511 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:00.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 -- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 msgr2=0x7f67801a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 0x7f67801a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 -- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678009e410 msgr2=0x7f67801a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678009e410 0x7f67801a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 -- 172.21.15.32:0/4102649511 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740094a0 con 0x7f678009d6f0 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677ffff700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678009e410 0x7f67801a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677effd700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 0x7f67801a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f677f7fe700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 0x7f67801a5e30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f678006e470 tx=0x7f6770004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770004450 con 0x7f678009d6f0 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67700036a0 con 0x7f678009d6f0 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6778009a70 con 0x7f678009d6f0 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67700045f0 con 0x7f678009d6f0 2023-09-30T22:30:00.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.714+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6770009550 con 0x7f678009d6f0 2023-09-30T22:30:00.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.715+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6770003a50 con 0x7f678009d6f0 2023-09-30T22:30:00.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.715+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6770021030 con 0x7f678009d6f0 2023-09-30T22:30:00.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.715+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6780074ea0 con 0x7f678009d6f0 2023-09-30T22:30:00.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.720+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f677002b020 con 0x7f678009d6f0 2023-09-30T22:30:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 Processor -- start 2023-09-30T22:30:00.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 -- start start 2023-09-30T22:30:00.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1510 0x7f38fc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc139250 0x7f38fc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc09edd0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0a18e0 con 0x7f38fc0a1510 2023-09-30T22:30:00.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.753+0000 7f390388c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc09e1f0 con 0x7f38fc139250 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3900e27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1510 0x7f38fc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901e29700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc139250 0x7f38fc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901628700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38fc0a0a30 0x7f38fc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41678/0 (socket says 172.21.15.32:41678) 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901628700 1 -- 172.21.15.32:0/553688102 learned_addr learned my addr 172.21.15.32:0/553688102 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901e29700 1 -- 172.21.15.32:0/553688102 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1510 msgr2=0x7f38fc138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901e29700 1 --2- 172.21.15.32:0/553688102 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1510 0x7f38fc138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901e29700 1 -- 172.21.15.32:0/553688102 >> v1:172.21.15.32:6789/0 conn(0x7f38fc0a0a30 legacy=0x7f38fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901e29700 1 -- 172.21.15.32:0/553688102 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc09dc50 con 0x7f38fc139250 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3900e27700 1 --2- 172.21.15.32:0/553688102 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1510 0x7f38fc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f3901e29700 1 --2- 172.21.15.32:0/553688102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc139250 0x7f38fc13b630 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009d90 tx=0x7f38f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551f2d8f8f54f7e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f38f3fff700 1 -- 172.21.15.32:0/553688102 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400e070 con 0x7f38fc139250 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f38f3fff700 1 -- 172.21.15.32:0/553688102 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400bb20 con 0x7f38fc139250 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.754+0000 7f38f3fff700 1 -- 172.21.15.32:0/553688102 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4012430 con 0x7f38fc139250 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 -- 172.21.15.32:0/553688102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc139250 msgr2=0x7f38fc13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 --2- 172.21.15.32:0/553688102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc139250 0x7f38fc13b630 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009d90 tx=0x7f38f400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 -- 172.21.15.32:0/553688102 shutdown_connections 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 --2- 172.21.15.32:0/553688102 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc139250 0x7f38fc13b630 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 --2- 172.21.15.32:0/553688102 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1510 0x7f38fc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 -- 172.21.15.32:0/553688102 >> 172.21.15.32:0/553688102 conn(0x7f38fc0a6550 msgr2=0x7f38fc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 -- 172.21.15.32:0/553688102 shutdown_connections 2023-09-30T22:30:00.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.755+0000 7f390388c700 1 -- 172.21.15.32:0/553688102 wait complete. 2023-09-30T22:30:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.756+0000 7f390388c700 1 Processor -- start 2023-09-30T22:30:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 -- start start 2023-09-30T22:30:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc1c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 0x7f38fc1c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc139250 0x7f38fc1c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc124820 con 0x7f38fc0a0a30 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc125d40 con 0x7f38fc139250 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f390388c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1253a0 con 0x7f38fc0a1510 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f3900e27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 0x7f38fc1c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f3900e27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 0x7f38fc1c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33536/0 (socket says 172.21.15.32:33536) 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f3900e27700 1 -- 172.21.15.32:0/421026584 learned_addr learned my addr 172.21.15.32:0/421026584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f3901628700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc1c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.757+0000 7f3901e29700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc139250 0x7f38fc1c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3901628700 1 -- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 msgr2=0x7f38fc1c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3901628700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 0x7f38fc1c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3901628700 1 -- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc139250 msgr2=0x7f38fc1c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3901628700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc139250 0x7f38fc1c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3901628700 1 -- 172.21.15.32:0/421026584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0094a0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3900e27700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 0x7f38fc1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f3901628700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc1c8540 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c500 tx=0x7f38f800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8005d70 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f80070d0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f4009a70 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8007440 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.758+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc1262d0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.759+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38f80075a0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.759+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f38f803e9c0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.759+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc1264c0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.764+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38f8019d20 con 0x7f38fc0a0a30 2023-09-30T22:30:00.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.930+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f678006c360 con 0x7f678009d6f0 2023-09-30T22:30:00.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.930+0000 7f6767fff700 1 -- 172.21.15.32:0/4102649511 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f677001a430 con 0x7f678009d6f0 2023-09-30T22:30:00.932 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:00.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 msgr2=0x7f67801a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 0x7f67801a5e30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f678006e470 tx=0x7f6770004750 comp rx=0 tx=0).stop 2023-09-30T22:30:00.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 shutdown_connections 2023-09-30T22:30:00.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f678009e410 0x7f67801a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f678009d6f0 0x7f67801a5e30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 --2- 172.21.15.32:0/4102649511 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f678009e000 0x7f67801a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.932+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 >> 172.21.15.32:0/4102649511 conn(0x7f6780066f50 msgr2=0x7f67801badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.933+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 shutdown_connections 2023-09-30T22:30:00.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.933+0000 7f6785ff1700 1 -- 172.21.15.32:0/4102649511 wait complete. 2023-09-30T22:30:00.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.971+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f38fc074ea0 con 0x7f38fc0a0a30 2023-09-30T22:30:00.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.972+0000 7f38f1ffb700 1 -- 172.21.15.32:0/421026584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38f8017020 con 0x7f38fc0a0a30 2023-09-30T22:30:00.973 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:00.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 msgr2=0x7f38fc1c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc1c8540 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c500 tx=0x7f38f800c810 comp rx=0 tx=0).stop 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 shutdown_connections 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc139250 0x7f38fc1c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc1c8540 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 --2- 172.21.15.32:0/421026584 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a1510 0x7f38fc1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 >> 172.21.15.32:0/421026584 conn(0x7f38fc0a6550 msgr2=0x7f38fc0a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 shutdown_connections 2023-09-30T22:30:00.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:00.974+0000 7f390388c700 1 -- 172.21.15.32:0/421026584 wait complete. 2023-09-30T22:30:00.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:01.403 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.548+0000 7fe5eb95e700 1 Processor -- start 2023-09-30T22:30:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5eb95e700 1 -- start start 2023-09-30T22:30:01.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5eb95e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e409e1d0 0x7fe5e4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5eb95e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5eb95e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e409f130 con 0x7fe5e409d6f0 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5eb95e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e409f520 con 0x7fe5e409e1d0 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5eb95e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e409e930 con 0x7fe5e4135e50 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5e8ef9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e409e1d0 0x7fe5e4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5e9efb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5e96fa700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5e409d6f0 0x7fe5e409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41694/0 (socket says 172.21.15.32:41694) 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5e8ef9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5e409e1d0 0x7fe5e4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35010/0 (socket says 172.21.15.32:35010) 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5e9efb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e4138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60534/0 (socket says 172.21.15.32:60534) 2023-09-30T22:30:01.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.549+0000 7fe5e96fa700 1 -- 172.21.15.32:0/4124876099 learned_addr learned my addr 172.21.15.32:0/4124876099 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5e8ef9700 1 -- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 msgr2=0x7fe5e4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5e8ef9700 1 --2- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5e8ef9700 1 -- 172.21.15.32:0/4124876099 >> v1:172.21.15.32:6789/0 conn(0x7fe5e409d6f0 legacy=0x7fe5e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5e8ef9700 1 -- 172.21.15.32:0/4124876099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e409b2d0 con 0x7fe5e409e1d0 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5dbfff700 1 -- 172.21.15.32:0/4124876099 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838859219 0 0) 0x7fe5e409f130 con 0x7fe5e409d6f0 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5e9efb700 1 --2- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5e8ef9700 1 --2- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409e1d0 0x7fe5e4135910 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009840 tx=0x7fe5d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65428d1a53abf74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5dbfff700 1 -- 172.21.15.32:0/4124876099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4011830 con 0x7fe5e409e1d0 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5dbfff700 1 -- 172.21.15.32:0/4124876099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4011990 con 0x7fe5e409e1d0 2023-09-30T22:30:01.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.550+0000 7fe5dbfff700 1 -- 172.21.15.32:0/4124876099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4011d20 con 0x7fe5e409e1d0 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 -- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409e1d0 msgr2=0x7fe5e4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409e1d0 0x7fe5e4135910 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009840 tx=0x7fe5d4009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 -- 172.21.15.32:0/4124876099 shutdown_connections 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/4124876099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409e1d0 0x7fe5e4135910 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 -- 172.21.15.32:0/4124876099 >> 172.21.15.32:0/4124876099 conn(0x7fe5e4066f50 msgr2=0x7fe5e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 -- 172.21.15.32:0/4124876099 shutdown_connections 2023-09-30T22:30:01.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.551+0000 7fe5eb95e700 1 -- 172.21.15.32:0/4124876099 wait complete. 2023-09-30T22:30:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 Processor -- start 2023-09-30T22:30:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 -- start start 2023-09-30T22:30:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409d6f0 0x7fe5e411d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e409e1d0 0x7fe5e411d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e411def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4120f90 con 0x7fe5e409d6f0 2023-09-30T22:30:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e411f080 con 0x7fe5e4135e50 2023-09-30T22:30:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5eb95e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40a1b80 con 0x7fe5e409e1d0 2023-09-30T22:30:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5e96fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409d6f0 0x7fe5e411d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5e9efb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e411def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5e9efb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e411def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60548/0 (socket says 172.21.15.32:60548) 2023-09-30T22:30:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5e96fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409d6f0 0x7fe5e411d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35026/0 (socket says 172.21.15.32:35026) 2023-09-30T22:30:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5e9efb700 1 -- 172.21.15.32:0/3243236153 learned_addr learned my addr 172.21.15.32:0/3243236153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.552+0000 7fe5e8ef9700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e409e1d0 0x7fe5e411d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5e9efb700 1 -- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e409e1d0 msgr2=0x7fe5e411d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5e9efb700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e409e1d0 0x7fe5e411d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5e9efb700 1 -- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409d6f0 msgr2=0x7fe5e411d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5e9efb700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409d6f0 0x7fe5e411d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5e9efb700 1 -- 172.21.15.32:0/3243236153 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009a70 con 0x7fe5e4135e50 2023-09-30T22:30:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5e9efb700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e411def0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00efb0 tx=0x7fe5dc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00cc90 con 0x7fe5e4135e50 2023-09-30T22:30:01.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc00f040 con 0x7fe5e4135e50 2023-09-30T22:30:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40094a0 con 0x7fe5e4135e50 2023-09-30T22:30:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc014480 con 0x7fe5e4135e50 2023-09-30T22:30:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.553+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e41209f0 con 0x7fe5e4135e50 2023-09-30T22:30:01.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.554+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5dc0145e0 con 0x7fe5e4135e50 2023-09-30T22:30:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.554+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe5dc0403d0 con 0x7fe5e4135e50 2023-09-30T22:30:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.554+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4074ea0 con 0x7fe5e4135e50 2023-09-30T22:30:01.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.559+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5dc007920 con 0x7fe5e4135e50 2023-09-30T22:30:01.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.764+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5e411f870 con 0x7fe5e4135e50 2023-09-30T22:30:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.764+0000 7fe5d9ffb700 1 -- 172.21.15.32:0/3243236153 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5dc007330 con 0x7fe5e4135e50 2023-09-30T22:30:01.765 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 msgr2=0x7fe5e411def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e411def0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00efb0 tx=0x7fe5dc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 shutdown_connections 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5e4135e50 0x7fe5e411def0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5e409d6f0 0x7fe5e411d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 --2- 172.21.15.32:0/3243236153 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe5e409e1d0 0x7fe5e411d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 >> 172.21.15.32:0/3243236153 conn(0x7fe5e4066f50 msgr2=0x7fe5e4071070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 shutdown_connections 2023-09-30T22:30:01.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.767+0000 7fe5eb95e700 1 -- 172.21.15.32:0/3243236153 wait complete. 2023-09-30T22:30:01.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 Processor -- start 2023-09-30T22:30:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 -- start start 2023-09-30T22:30:01.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 0x7f91d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d009b120 con 0x7f91d009e000 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d006b460 con 0x7f91d009e410 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d679f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91d009a7e0 con 0x7f91d009d6f0 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91cffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 0x7f91d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91cffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 0x7f91d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33562/0 (socket says 172.21.15.32:33562) 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.849+0000 7f91d4d3c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91cffff700 1 -- 172.21.15.32:0/694102733 learned_addr learned my addr 172.21.15.32:0/694102733 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91cf7fe700 1 --1- 172.21.15.32:0/694102733 >> v1:172.21.15.32:6789/0 conn(0x7f91d009e000 0x7f91d01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41698/0 (socket says 172.21.15.32:41698) 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91d4d3c700 1 -- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 msgr2=0x7f91d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:01.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91ceffd700 1 -- 172.21.15.32:0/694102733 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055452129 0 0) 0x7f91d009b120 con 0x7f91d009e000 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91d4d3c700 1 --2- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 0x7f91d009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91d4d3c700 1 -- 172.21.15.32:0/694102733 >> v1:172.21.15.32:6789/0 conn(0x7f91d009e000 legacy=0x7f91d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91d4d3c700 1 -- 172.21.15.32:0/694102733 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d009a500 con 0x7f91d009e410 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91cffff700 1 --2- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 0x7f91d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91d4d3c700 1 --2- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0134100 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009dc0 tx=0x7f91c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5696b219a4b3e749 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91ceffd700 1 -- 172.21.15.32:0/694102733 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800e070 con 0x7f91d009e410 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91ceffd700 1 -- 172.21.15.32:0/694102733 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800bb20 con 0x7f91d009e410 2023-09-30T22:30:01.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.850+0000 7f91ceffd700 1 -- 172.21.15.32:0/694102733 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8012430 con 0x7f91d009e410 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 -- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 msgr2=0x7f91d0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 --2- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0134100 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009dc0 tx=0x7f91c800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 -- 172.21.15.32:0/694102733 shutdown_connections 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 --2- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0134100 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 --2- 172.21.15.32:0/694102733 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009d6f0 0x7f91d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 -- 172.21.15.32:0/694102733 >> 172.21.15.32:0/694102733 conn(0x7f91d0066f50 msgr2=0x7f91d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 -- 172.21.15.32:0/694102733 shutdown_connections 2023-09-30T22:30:01.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.851+0000 7f91d679f700 1 -- 172.21.15.32:0/694102733 wait complete. 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.852+0000 7f91d679f700 1 Processor -- start 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.852+0000 7f91d679f700 1 -- start start 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d679f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 0x7f91d0122350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d679f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009e000 0x7f91d0122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d679f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d679f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0125ba0 con 0x7f91d009d6f0 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d679f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0133690 con 0x7f91d009e410 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d679f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0124d60 con 0x7f91d009e000 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 0x7f91d0122350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91cf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009e000 0x7f91d0122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91cffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 0x7f91d0122350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35036/0 (socket says 172.21.15.32:35036) 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91cffff700 1 -- 172.21.15.32:0/1345277418 learned_addr learned my addr 172.21.15.32:0/1345277418 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 -- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009e000 msgr2=0x7f91d0122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009e000 0x7f91d0122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 -- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 msgr2=0x7f91d0122350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 0x7f91d0122350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 -- 172.21.15.32:0/1345277418 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0009550 con 0x7f91d009e410 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91cffff700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 0x7f91d0122350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.853+0000 7f91d4d3c700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0122dd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009dc0 tx=0x7f91c8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.854+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c801b810 con 0x7f91d009e410 2023-09-30T22:30:01.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.854+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8020070 con 0x7f91d009e410 2023-09-30T22:30:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.854+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c8009a70 con 0x7f91d009e410 2023-09-30T22:30:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.854+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c801bbd0 con 0x7f91d009e410 2023-09-30T22:30:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.854+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d009c710 con 0x7f91d009e410 2023-09-30T22:30:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.854+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91c801bd30 con 0x7f91d009e410 2023-09-30T22:30:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.855+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f91c8022070 con 0x7f91d009e410 2023-09-30T22:30:01.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.855+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d0074ea0 con 0x7f91d009e410 2023-09-30T22:30:01.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:01.859+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91c8019310 con 0x7f91d009e410 2023-09-30T22:30:01.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:01.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.071+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f91d006c360 con 0x7f91d009e410 2023-09-30T22:30:02.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.072+0000 7f91ccff9700 1 -- 172.21.15.32:0/1345277418 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91c8024b20 con 0x7f91d009e410 2023-09-30T22:30:02.073 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:02.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 msgr2=0x7f91d0122dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0122dd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009dc0 tx=0x7f91c8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 shutdown_connections 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91d009e410 0x7f91d0122dd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91d009d6f0 0x7f91d0122350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 --2- 172.21.15.32:0/1345277418 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f91d009e000 0x7f91d0122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.074+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 >> 172.21.15.32:0/1345277418 conn(0x7f91d0066f50 msgr2=0x7f91d0070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.075+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 shutdown_connections 2023-09-30T22:30:02.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.075+0000 7f91d679f700 1 -- 172.21.15.32:0/1345277418 wait complete. 2023-09-30T22:30:02.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.089+0000 7fc4ccd67700 1 Processor -- start 2023-09-30T22:30:02.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.089+0000 7fc4ccd67700 1 -- start start 2023-09-30T22:30:02.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4ccd67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 0x7fc4c8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4ccd67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 0x7fc4c8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4ccd67700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c809b4f0 con 0x7fc4c8131810 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4ccd67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81313c0 con 0x7fc4c8134630 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4ccd67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c806e8c0 con 0x7fc4c8136f50 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 0x7fc4c8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 0x7fc4c8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 0x7fc4c8136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60570/0 (socket says 172.21.15.32:60570) 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 0x7fc4c8139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33574/0 (socket says 172.21.15.32:33574) 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 -- 172.21.15.32:0/328097329 learned_addr learned my addr 172.21.15.32:0/328097329 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 -- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 msgr2=0x7fc4c8139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c559a700 1 -- 172.21.15.32:0/328097329 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293675166 0 0) 0x7fc4c809b4f0 con 0x7fc4c8131810 2023-09-30T22:30:02.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 --2- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 0x7fc4c8139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 -- 172.21.15.32:0/328097329 >> v1:172.21.15.32:6789/0 conn(0x7fc4c8131810 legacy=0x7fc4c8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 -- 172.21.15.32:0/328097329 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8071d20 con 0x7fc4c8134630 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c6d9d700 1 --2- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 0x7fc4c8139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.090+0000 7fc4c5d9b700 1 --2- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 0x7fc4c8136a10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009920 tx=0x7fc4bc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0db9309dbba1607 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4c559a700 1 -- 172.21.15.32:0/328097329 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc011830 con 0x7fc4c8134630 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4c559a700 1 -- 172.21.15.32:0/328097329 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc011990 con 0x7fc4c8134630 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4c559a700 1 -- 172.21.15.32:0/328097329 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc011d20 con 0x7fc4c8134630 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4ccd67700 1 -- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 msgr2=0x7fc4c8136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 0x7fc4c8136a10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009920 tx=0x7fc4bc009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:02.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4ccd67700 1 -- 172.21.15.32:0/328097329 shutdown_connections 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.091+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8136f50 0x7fc4c8139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.092+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/328097329 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8134630 0x7fc4c8136a10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.092+0000 7fc4ccd67700 1 -- 172.21.15.32:0/328097329 >> 172.21.15.32:0/328097329 conn(0x7fc4c80a2aa0 msgr2=0x7fc4c80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.092+0000 7fc4ccd67700 1 -- 172.21.15.32:0/328097329 shutdown_connections 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.092+0000 7fc4ccd67700 1 -- 172.21.15.32:0/328097329 wait complete. 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 Processor -- start 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 -- start start 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 0x7fc4c8118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8134630 0x7fc4c8119290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8136f50 0x7fc4c81197d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c812e750 con 0x7fc4c8136f50 2023-09-30T22:30:02.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c811a8d0 con 0x7fc4c8131810 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4ccd67700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c811c460 con 0x7fc4c8134630 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4c659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 0x7fc4c8118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4c659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 0x7fc4c8118d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60584/0 (socket says 172.21.15.32:60584) 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8136f50 0x7fc4c81197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.093+0000 7fc4c659c700 1 -- 172.21.15.32:0/1028997235 learned_addr learned my addr 172.21.15.32:0/1028997235 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c5d9b700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8134630 0x7fc4c8119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c659c700 1 -- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8134630 msgr2=0x7fc4c8119290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c659c700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8134630 0x7fc4c8119290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c659c700 1 -- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8136f50 msgr2=0x7fc4c81197d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c659c700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8136f50 0x7fc4c81197d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c659c700 1 -- 172.21.15.32:0/1028997235 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0009a70 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c6d9d700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8136f50 0x7fc4c81197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4c659c700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 0x7fc4c8118d50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8001eb0 tx=0x7fc4b800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80054e0 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8003680 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80039f0 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc009550 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.094+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c812eb10 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.095+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c8074ea0 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.096+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc4b8003b50 con 0x7fc4c8131810 2023-09-30T22:30:02.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.096+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc4b803efc0 con 0x7fc4c8131810 2023-09-30T22:30:02.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.100+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc4b801b3b0 con 0x7fc4c8131810 2023-09-30T22:30:02.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 Processor -- start 2023-09-30T22:30:02.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 -- start start 2023-09-30T22:30:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38801316c0 0x7f3880131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388006ae80 0x7f3880139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388009e130 con 0x7f388009d9f0 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38801312b0 con 0x7f38801316c0 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.145+0000 7f38865cd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f388009f6e0 con 0x7f388006ae80 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f388009d9f0 0x7f388006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41706/0 (socket says 172.21.15.32:41706) 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f3884b6a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388006ae80 0x7f3880139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387f7fe700 1 -- 172.21.15.32:0/2761030713 learned_addr learned my addr 172.21.15.32:0/2761030713 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38801316c0 0x7f3880131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387effd700 1 -- 172.21.15.32:0/2761030713 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309327397 0 0) 0x7f3874001110 con 0x7f388009d9f0 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387effd700 1 -- 172.21.15.32:0/2761030713 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38680035b0 con 0x7f388009d9f0 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387ffff700 1 -- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388006ae80 msgr2=0x7f3880139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387ffff700 1 --2- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388006ae80 0x7f3880139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387ffff700 1 -- 172.21.15.32:0/2761030713 >> v1:172.21.15.32:6789/0 conn(0x7f388009d9f0 legacy=0x7f388006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387ffff700 1 -- 172.21.15.32:0/2761030713 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388009e760 con 0x7f38801316c0 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f3884b6a700 1 --2- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388006ae80 0x7f3880139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387ffff700 1 --2- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38801316c0 0x7f3880131a90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3870009870 tx=0x7f3870009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eccee45252a7fc64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.146+0000 7f387effd700 1 -- 172.21.15.32:0/2761030713 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870011830 con 0x7f38801316c0 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f387effd700 1 -- 172.21.15.32:0/2761030713 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870011990 con 0x7f38801316c0 2023-09-30T22:30:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f387effd700 1 -- 172.21.15.32:0/2761030713 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870011d00 con 0x7f38801316c0 2023-09-30T22:30:02.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 -- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38801316c0 msgr2=0x7f3880131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 --2- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38801316c0 0x7f3880131a90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3870009870 tx=0x7f3870009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:02.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 -- 172.21.15.32:0/2761030713 shutdown_connections 2023-09-30T22:30:02.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 --2- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388006ae80 0x7f3880139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 --2- 172.21.15.32:0/2761030713 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38801316c0 0x7f3880131a90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 -- 172.21.15.32:0/2761030713 >> 172.21.15.32:0/2761030713 conn(0x7f3880066f50 msgr2=0x7f388009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 -- 172.21.15.32:0/2761030713 shutdown_connections 2023-09-30T22:30:02.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.147+0000 7f38865cd700 1 -- 172.21.15.32:0/2761030713 wait complete. 2023-09-30T22:30:02.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.149+0000 7f38865cd700 1 Processor -- start 2023-09-30T22:30:02.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.149+0000 7f38865cd700 1 -- start start 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f38865cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f388006ae80 0x7f38801c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f38865cd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 0x7f38801c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f38865cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38801316c0 0x7f38800709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f38865cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388012b970 con 0x7f38801316c0 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f38865cd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38800727b0 con 0x7f388006ae80 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f38865cd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f388011d930 con 0x7f388009d9f0 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f387f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 0x7f38801c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38801316c0 0x7f38800709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f387f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 0x7f38801c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33598/0 (socket says 172.21.15.32:33598) 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f387f7fe700 1 -- 172.21.15.32:0/2523483925 learned_addr learned my addr 172.21.15.32:0/2523483925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f387ffff700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f388006ae80 0x7f38801c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 -- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 msgr2=0x7f38801c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 0x7f38801c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 -- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f388006ae80 msgr2=0x7f38801c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f388006ae80 0x7f38801c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 -- 172.21.15.32:0/2523483925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878009a70 con 0x7f38801316c0 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f387ffff700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f388006ae80 0x7f38801c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f387f7fe700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 0x7f38801c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.150+0000 7f3884b6a700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38801316c0 0x7f38800709d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f387800efc0 tx=0x7f387800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.151+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387800ca30 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.151+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387800f040 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.151+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38700094a0 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.151+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387800cdd0 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.151+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388009c310 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.152+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3878014480 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.152+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880074ea0 con 0x7f38801316c0 2023-09-30T22:30:02.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.152+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f387803fb30 con 0x7f38801316c0 2023-09-30T22:30:02.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.157+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3878012930 con 0x7f38801316c0 2023-09-30T22:30:02.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.325+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc4c811b020 con 0x7fc4c8131810 2023-09-30T22:30:02.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.326+0000 7fc4b6ffd700 1 -- 172.21.15.32:0/1028997235 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc4b8017030 con 0x7fc4c8131810 2023-09-30T22:30:02.327 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 msgr2=0x7fc4c8118d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 0x7fc4c8118d50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8001eb0 tx=0x7fc4b800c800 comp rx=0 tx=0).stop 2023-09-30T22:30:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 shutdown_connections 2023-09-30T22:30:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c8136f50 0x7fc4c81197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c8131810 0x7fc4c8118d50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 --2- 172.21.15.32:0/1028997235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc4c8134630 0x7fc4c8119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 >> 172.21.15.32:0/1028997235 conn(0x7fc4c80a2aa0 msgr2=0x7fc4c80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 shutdown_connections 2023-09-30T22:30:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.328+0000 7fc4ccd67700 1 -- 172.21.15.32:0/1028997235 wait complete. 2023-09-30T22:30:02.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.363+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f388009a430 con 0x7f38801316c0 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.363+0000 7f387cff9700 1 -- 172.21.15.32:0/2523483925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3878012340 con 0x7f38801316c0 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38801316c0 msgr2=0x7f38800709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38801316c0 0x7f38800709d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f387800efc0 tx=0x7f387800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 shutdown_connections 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38801316c0 0x7f38800709d0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f388006ae80 0x7f38801c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 --2- 172.21.15.32:0/2523483925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f388009d9f0 0x7f38801c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 >> 172.21.15.32:0/2523483925 conn(0x7f3880066f50 msgr2=0x7f38800a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.365+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 shutdown_connections 2023-09-30T22:30:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.366+0000 7f38865cd700 1 -- 172.21.15.32:0/2523483925 wait complete. 2023-09-30T22:30:02.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:02.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.929+0000 7fefb7267700 1 Processor -- start 2023-09-30T22:30:02.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.929+0000 7fefb7267700 1 -- start start 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb7267700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 0x7fefb0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb7267700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb7267700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb009b120 con 0x7fefb009d6f0 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb7267700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb006b460 con 0x7fefb0139170 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb7267700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefb009a7e0 con 0x7fefb009e1d0 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb4802700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 0x7fefb0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb4802700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 0x7fefb0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33610/0 (socket says 172.21.15.32:33610) 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb4802700 1 -- 172.21.15.32:0/722184363 learned_addr learned my addr 172.21.15.32:0/722184363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:02.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.930+0000 7fefb5804700 1 --2- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefb4802700 1 -- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 msgr2=0x7fefb013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefb4802700 1 --2- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefb4802700 1 -- 172.21.15.32:0/722184363 >> v1:172.21.15.32:6789/0 conn(0x7fefb009d6f0 legacy=0x7fefb009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefb4802700 1 -- 172.21.15.32:0/722184363 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb009a500 con 0x7fefb009e1d0 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefb4802700 1 --2- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 0x7fefb0138c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009950 tx=0x7fefa0009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81ee4f24819b3c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefa7fff700 1 -- 172.21.15.32:0/722184363 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011830 con 0x7fefb009e1d0 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefa7fff700 1 -- 172.21.15.32:0/722184363 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0011990 con 0x7fefb009e1d0 2023-09-30T22:30:02.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.931+0000 7fefa7fff700 1 -- 172.21.15.32:0/722184363 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011d40 con 0x7fefb009e1d0 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 -- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 msgr2=0x7fefb0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 --2- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 0x7fefb0138c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009950 tx=0x7fefa0009c60 comp rx=0 tx=0).stop 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 -- 172.21.15.32:0/722184363 shutdown_connections 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 --2- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 --2- 172.21.15.32:0/722184363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e1d0 0x7fefb0138c30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 -- 172.21.15.32:0/722184363 >> 172.21.15.32:0/722184363 conn(0x7fefb0066f50 msgr2=0x7fefb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 -- 172.21.15.32:0/722184363 shutdown_connections 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.932+0000 7fefb7267700 1 -- 172.21.15.32:0/722184363 wait complete. 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 Processor -- start 2023-09-30T22:30:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 -- start start 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e1d0 0x7fefb011ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb011f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0121a80 con 0x7fefb009e1d0 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0120400 con 0x7fefb0139170 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb7267700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefb00a2020 con 0x7fefb009d6f0 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb4802700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e1d0 0x7fefb011ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb5804700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb011f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb5003700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb011f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60624/0 (socket says 172.21.15.32:60624) 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.933+0000 7fefb4802700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e1d0 0x7fefb011ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35070/0 (socket says 172.21.15.32:35070) 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 -- 172.21.15.32:0/3196275866 learned_addr learned my addr 172.21.15.32:0/3196275866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 -- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 msgr2=0x7fefb011e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 -- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e1d0 msgr2=0x7fefb011ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e1d0 0x7fefb011ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 -- 172.21.15.32:0/3196275866 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009550 con 0x7fefb0139170 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb5804700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb011f270 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fefa800efc0 tx=0x7fefa800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8009ea0 con 0x7fefb0139170 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa800f040 con 0x7fefb0139170 2023-09-30T22:30:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb009c710 con 0x7fefb0139170 2023-09-30T22:30:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800ced0 con 0x7fefb0139170 2023-09-30T22:30:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.934+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb0122810 con 0x7fefb0139170 2023-09-30T22:30:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.935+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefa8014480 con 0x7fefb0139170 2023-09-30T22:30:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.935+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb0074ea0 con 0x7fefb0139170 2023-09-30T22:30:02.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.936+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fefa803ff30 con 0x7fefb0139170 2023-09-30T22:30:02.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:02.940+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefa801a8b0 con 0x7fefb0139170 2023-09-30T22:30:03.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:03.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.151+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fefb0122ac0 con 0x7fefb0139170 2023-09-30T22:30:03.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.151+0000 7fefa5ffb700 1 -- 172.21.15.32:0/3196275866 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefa8018020 con 0x7fefb0139170 2023-09-30T22:30:03.152 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 msgr2=0x7fefb011f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb011f270 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fefa800efc0 tx=0x7fefa800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 shutdown_connections 2023-09-30T22:30:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb0139170 0x7fefb011f270 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 --2- 172.21.15.32:0/3196275866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e1d0 0x7fefb011ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 >> 172.21.15.32:0/3196275866 conn(0x7fefb0066f50 msgr2=0x7fefb0136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 shutdown_connections 2023-09-30T22:30:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.154+0000 7fefb7267700 1 -- 172.21.15.32:0/3196275866 wait complete. 2023-09-30T22:30:03.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.240+0000 7f96e19f0700 1 Processor -- start 2023-09-30T22:30:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.240+0000 7f96e19f0700 1 -- start start 2023-09-30T22:30:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96e19f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96e19f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96dc09ffb0 0x7f96dc09f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96e19f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0a06b0 con 0x7f96dc133b80 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96e19f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc06b920 con 0x7f96dc09ffb0 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96e19f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc09aa30 con 0x7f96dc09e850 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96daffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96da7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96dc09ffb0 0x7f96dc09f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96daffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc09ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33636/0 (socket says 172.21.15.32:33636) 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96daffd700 1 -- 172.21.15.32:0/2090409381 learned_addr learned my addr 172.21.15.32:0/2090409381 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.241+0000 7f96db7fe700 1 --1- 172.21.15.32:0/2090409381 >> v1:172.21.15.32:6789/0 conn(0x7f96dc133b80 0x7f96dc133f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41730/0 (socket says 172.21.15.32:41730) 2023-09-30T22:30:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96da7fc700 1 -- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 msgr2=0x7f96dc09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96da7fc700 1 --2- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96da7fc700 1 -- 172.21.15.32:0/2090409381 >> v1:172.21.15.32:6789/0 conn(0x7f96dc133b80 legacy=0x7f96dc133f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96da7fc700 1 -- 172.21.15.32:0/2090409381 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc06caa0 con 0x7f96dc09ffb0 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96d9ffb700 1 -- 172.21.15.32:0/2090409381 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794677644 0 0) 0x7f96dc0a06b0 con 0x7f96dc133b80 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96daffd700 1 --2- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96da7fc700 1 --2- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc09f160 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009840 tx=0x7f96d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80ecc1de6bdb85f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96d9ffb700 1 -- 172.21.15.32:0/2090409381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0011830 con 0x7f96dc09ffb0 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96d9ffb700 1 -- 172.21.15.32:0/2090409381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d0011990 con 0x7f96dc09ffb0 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96d9ffb700 1 -- 172.21.15.32:0/2090409381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0011d20 con 0x7f96dc09ffb0 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96e19f0700 1 -- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 msgr2=0x7f96dc09f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.242+0000 7f96e19f0700 1 --2- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc09f160 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009840 tx=0x7f96d0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.243+0000 7f96e19f0700 1 -- 172.21.15.32:0/2090409381 shutdown_connections 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.243+0000 7f96e19f0700 1 --2- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.243+0000 7f96e19f0700 1 --2- 172.21.15.32:0/2090409381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc09f160 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.243+0000 7f96e19f0700 1 -- 172.21.15.32:0/2090409381 >> 172.21.15.32:0/2090409381 conn(0x7f96dc0a3f70 msgr2=0x7f96dc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.243+0000 7f96e19f0700 1 -- 172.21.15.32:0/2090409381 shutdown_connections 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.243+0000 7f96e19f0700 1 -- 172.21.15.32:0/2090409381 wait complete. 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 Processor -- start 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 -- start start 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc120480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc1209c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc133b80 0x7f96dc120f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc121950 con 0x7f96dc09ffb0 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc133870 con 0x7f96dc133b80 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96e19f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1232e0 con 0x7f96dc09e850 2023-09-30T22:30:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96daffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc120480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96da7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc1209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96db7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc133b80 0x7f96dc120f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96daffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc120480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33644/0 (socket says 172.21.15.32:33644) 2023-09-30T22:30:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.244+0000 7f96daffd700 1 -- 172.21.15.32:0/1204282460 learned_addr learned my addr 172.21.15.32:0/1204282460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96daffd700 1 -- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc133b80 msgr2=0x7f96dc120f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96daffd700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc133b80 0x7f96dc120f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96daffd700 1 -- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 msgr2=0x7f96dc1209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96daffd700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc1209c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96daffd700 1 -- 172.21.15.32:0/1204282460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009b30 con 0x7f96dc09e850 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96db7fe700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc133b80 0x7f96dc120f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96da7fc700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc1209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96daffd700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc120480 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009f20 tx=0x7f96cc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00cbc0 con 0x7f96dc09e850 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00f040 con 0x7f96dc09e850 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d00094a0 con 0x7f96dc09e850 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc014480 con 0x7f96dc09e850 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.245+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc123730 con 0x7f96dc09e850 2023-09-30T22:30:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.246+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f96cc0145e0 con 0x7f96dc09e850 2023-09-30T22:30:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.246+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f96cc03fc40 con 0x7f96dc09e850 2023-09-30T22:30:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.246+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc134610 con 0x7f96dc09e850 2023-09-30T22:30:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.251+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96cc0128f0 con 0x7f96dc09e850 2023-09-30T22:30:03.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:03.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:03.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.465+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f96dc0a3520 con 0x7f96dc09e850 2023-09-30T22:30:03.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.466+0000 7f96c3fff700 1 -- 172.21.15.32:0/1204282460 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96cc0126c0 con 0x7f96dc09e850 2023-09-30T22:30:03.467 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:03.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 msgr2=0x7f96dc120480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc120480 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009f20 tx=0x7f96cc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 shutdown_connections 2023-09-30T22:30:03.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f96dc133b80 0x7f96dc120f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f96dc09e850 0x7f96dc120480 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 --2- 172.21.15.32:0/1204282460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96dc09ffb0 0x7f96dc1209c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 >> 172.21.15.32:0/1204282460 conn(0x7f96dc0a3f70 msgr2=0x7f96dc1b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 shutdown_connections 2023-09-30T22:30:03.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.468+0000 7f96e19f0700 1 -- 172.21.15.32:0/1204282460 wait complete. 2023-09-30T22:30:03.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:03.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.483+0000 7fb2f9b6b700 1 Processor -- start 2023-09-30T22:30:03.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.483+0000 7fb2f9b6b700 1 -- start start 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f9b6b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2f4136710 0x7fb2f4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f9b6b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 0x7fb2f413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f9b6b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f406ef30 con 0x7fb2f41338a0 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f9b6b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f406fef0 con 0x7fb2f4136710 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f9b6b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41334b0 con 0x7fb2f4139030 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 0x7fb2f413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2f4136710 0x7fb2f4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 0x7fb2f413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60636/0 (socket says 172.21.15.32:60636) 2023-09-30T22:30:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f3fff700 1 -- 172.21.15.32:0/4136813412 learned_addr learned my addr 172.21.15.32:0/4136813412 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f2ffd700 1 --2- 172.21.15.32:0/4136813412 >> v2:172.21.15.32:3300/0 conn(0x7fb2f4136710 0x7fb2f4138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35082/0 (socket says 172.21.15.32:35082) 2023-09-30T22:30:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f2ffd700 1 -- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 msgr2=0x7fb2f413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f2ffd700 1 --2- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 0x7fb2f413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.484+0000 7fb2f2ffd700 1 -- 172.21.15.32:0/4136813412 >> v1:172.21.15.32:6789/0 conn(0x7fb2f41338a0 legacy=0x7fb2f4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f2ffd700 1 -- 172.21.15.32:0/4136813412 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f406fc30 con 0x7fb2f4136710 2023-09-30T22:30:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f27fc700 1 -- 172.21.15.32:0/4136813412 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385915108 0 0) 0x7fb2f406ef30 con 0x7fb2f41338a0 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f3fff700 1 --2- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 0x7fb2f413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f2ffd700 1 --2- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4136710 0x7fb2f4138af0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009840 tx=0x7fb2e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e75a532418eb0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f27fc700 1 -- 172.21.15.32:0/4136813412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8011830 con 0x7fb2f4136710 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f27fc700 1 -- 172.21.15.32:0/4136813412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8011990 con 0x7fb2f4136710 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.485+0000 7fb2f27fc700 1 -- 172.21.15.32:0/4136813412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8011d20 con 0x7fb2f4136710 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4136710 msgr2=0x7fb2f4138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4136710 0x7fb2f4138af0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009840 tx=0x7fb2e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/4136813412 shutdown_connections 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f4139030 0x7fb2f413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/4136813412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4136710 0x7fb2f4138af0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/4136813412 >> 172.21.15.32:0/4136813412 conn(0x7fb2f40a2aa0 msgr2=0x7fb2f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/4136813412 shutdown_connections 2023-09-30T22:30:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.486+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/4136813412 wait complete. 2023-09-30T22:30:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.487+0000 7fb2f9b6b700 1 Processor -- start 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.487+0000 7fb2f9b6b700 1 -- start start 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f9b6b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f41338a0 0x7fb2f41c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f9b6b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb2f4136710 0x7fb2f41c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f9b6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 0x7fb2f411adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f9b6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f411e930 con 0x7fb2f4139030 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f9b6b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f411e4e0 con 0x7fb2f41338a0 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f9b6b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f41be1b0 con 0x7fb2f4136710 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 0x7fb2f411adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 0x7fb2f411adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35090/0 (socket says 172.21.15.32:35090) 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f3fff700 1 -- 172.21.15.32:0/3652686938 learned_addr learned my addr 172.21.15.32:0/3652686938 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f37fe700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f41338a0 0x7fb2f41c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.488+0000 7fb2f2ffd700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb2f4136710 0x7fb2f41c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f37fe700 1 -- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb2f4136710 msgr2=0x7fb2f41c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f37fe700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb2f4136710 0x7fb2f41c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f37fe700 1 -- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 msgr2=0x7fb2f411adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f37fe700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 0x7fb2f411adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f37fe700 1 -- 172.21.15.32:0/3652686938 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec009550 con 0x7fb2f41338a0 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f3fff700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 0x7fb2f411adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f2ffd700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb2f4136710 0x7fb2f41c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f37fe700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f41338a0 0x7fb2f41c8400 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c7e0 tx=0x7fb2e400caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4003e30 con 0x7fb2f41338a0 2023-09-30T22:30:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4005070 con 0x7fb2f41338a0 2023-09-30T22:30:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40053e0 con 0x7fb2f41338a0 2023-09-30T22:30:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e80094a0 con 0x7fb2f41338a0 2023-09-30T22:30:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.489+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f406f640 con 0x7fb2f41338a0 2023-09-30T22:30:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.490+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb2e4005540 con 0x7fb2f41338a0 2023-09-30T22:30:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.490+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb2e403f1e0 con 0x7fb2f41338a0 2023-09-30T22:30:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.490+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f411e720 con 0x7fb2f41338a0 2023-09-30T22:30:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.495+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2e4010710 con 0x7fb2f41338a0 2023-09-30T22:30:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 Processor -- start 2023-09-30T22:30:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 -- start start 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25080984f0 0x7f25080988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250809db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2508099620 con 0x7f250809e040 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250809a9e0 con 0x7f25080984f0 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f250d79c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250809bce0 con 0x7f250809d6f0 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f2506ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25080984f0 0x7f25080988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f25067fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250809db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.527+0000 7f25077fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f250809e040 0x7f25081385c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41736/0 (socket says 172.21.15.32:41736) 2023-09-30T22:30:03.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f25077fe700 1 -- 172.21.15.32:0/2103926756 learned_addr learned my addr 172.21.15.32:0/2103926756 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2506ffd700 1 -- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 msgr2=0x7f250809db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2506ffd700 1 --2- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250809db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2506ffd700 1 -- 172.21.15.32:0/2103926756 >> v1:172.21.15.32:6789/0 conn(0x7f250809e040 legacy=0x7f25081385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2506ffd700 1 -- 172.21.15.32:0/2103926756 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250809aca0 con 0x7f25080984f0 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f25067fc700 1 --2- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250809db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2506ffd700 1 --2- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25080984f0 0x7f25080988c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009840 tx=0x7f24f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4563ea3a2fe7b85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2505ffb700 1 -- 172.21.15.32:0/2103926756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011830 con 0x7f25080984f0 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2505ffb700 1 -- 172.21.15.32:0/2103926756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f8011990 con 0x7f25080984f0 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.528+0000 7f2505ffb700 1 -- 172.21.15.32:0/2103926756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011d20 con 0x7f25080984f0 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 -- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25080984f0 msgr2=0x7f25080988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 --2- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25080984f0 0x7f25080988c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009840 tx=0x7f24f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 -- 172.21.15.32:0/2103926756 shutdown_connections 2023-09-30T22:30:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 --2- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25080984f0 0x7f25080988c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 --2- 172.21.15.32:0/2103926756 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250809db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 -- 172.21.15.32:0/2103926756 >> 172.21.15.32:0/2103926756 conn(0x7f250809f470 msgr2=0x7f25080a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 -- 172.21.15.32:0/2103926756 shutdown_connections 2023-09-30T22:30:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.529+0000 7f250d79c700 1 -- 172.21.15.32:0/2103926756 wait complete. 2023-09-30T22:30:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 Processor -- start 2023-09-30T22:30:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 -- start start 2023-09-30T22:30:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25080984f0 0x7f250812ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250812bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f250809e040 0x7f25081c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250812b640 con 0x7f250809e040 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250812ecc0 con 0x7f250809d6f0 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f250d79c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f250812d920 con 0x7f25080984f0 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f25067fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250812bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f25067fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250812bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60656/0 (socket says 172.21.15.32:60656) 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f25067fc700 1 -- 172.21.15.32:0/2164773530 learned_addr learned my addr 172.21.15.32:0/2164773530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f2506ffd700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25080984f0 0x7f250812ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.531+0000 7f25077fe700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f250809e040 0x7f25081c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f25067fc700 1 -- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25080984f0 msgr2=0x7f250812ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f25067fc700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25080984f0 0x7f250812ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f25067fc700 1 -- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f250809e040 msgr2=0x7f25081c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f25067fc700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f250809e040 0x7f25081c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f25067fc700 1 -- 172.21.15.32:0/2164773530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc0094a0 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f2506ffd700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25080984f0 0x7f250812ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f25067fc700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250812bf80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00ebe0 tx=0x7f24fc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00c850 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc00c9b0 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f80094a0 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00cd40 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.532+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250812cd10 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.533+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24fc013470 con 0x7f250809d6f0 2023-09-30T22:30:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.533+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250812eaf0 con 0x7f250809d6f0 2023-09-30T22:30:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.533+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f24fc03f9d0 con 0x7f250809d6f0 2023-09-30T22:30:03.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.538+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24fc00cea0 con 0x7f250809d6f0 2023-09-30T22:30:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.704+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb2f4074ea0 con 0x7fb2f41338a0 2023-09-30T22:30:03.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.705+0000 7fb2dbfff700 1 -- 172.21.15.32:0/3652686938 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2e401a360 con 0x7fb2f41338a0 2023-09-30T22:30:03.706 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f41338a0 msgr2=0x7fb2f41c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f41338a0 0x7fb2f41c8400 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c7e0 tx=0x7fb2e400caf0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 shutdown_connections 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2f4139030 0x7fb2f411adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb2f41338a0 0x7fb2f41c8400 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 --2- 172.21.15.32:0/3652686938 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb2f4136710 0x7fb2f41c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.707+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 >> 172.21.15.32:0/3652686938 conn(0x7fb2f40a2aa0 msgr2=0x7fb2f40a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.708+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 shutdown_connections 2023-09-30T22:30:03.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.708+0000 7fb2f9b6b700 1 -- 172.21.15.32:0/3652686938 wait complete. 2023-09-30T22:30:03.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:03.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.743+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f25080a0e20 con 0x7f250809d6f0 2023-09-30T22:30:03.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.743+0000 7f24ef7fe700 1 -- 172.21.15.32:0/2164773530 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24fc017030 con 0x7f250809d6f0 2023-09-30T22:30:03.745 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:03.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 msgr2=0x7f250812bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250812bf80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00ebe0 tx=0x7f24fc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 shutdown_connections 2023-09-30T22:30:03.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f250809e040 0x7f25081c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25080984f0 0x7f250812ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 --2- 172.21.15.32:0/2164773530 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f250809d6f0 0x7f250812bf80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 >> 172.21.15.32:0/2164773530 conn(0x7f250809f470 msgr2=0x7f250809fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 shutdown_connections 2023-09-30T22:30:03.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:03.746+0000 7f250d79c700 1 -- 172.21.15.32:0/2164773530 wait complete. 2023-09-30T22:30:03.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:04.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.315+0000 7ff134f4c700 1 Processor -- start 2023-09-30T22:30:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.315+0000 7ff134f4c700 1 -- start start 2023-09-30T22:30:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff134f4c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 0x7ff13009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff134f4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff134f4c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff130131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff134f4c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff130070940 con 0x7ff13009e930 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff134f4c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13006fd60 con 0x7ff13009f650 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff134f4c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff13006fa80 con 0x7ff13009f240 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 0x7ff13009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff130131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff130131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60662/0 (socket says 172.21.15.32:60662) 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 0x7ff13009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35112/0 (socket says 172.21.15.32:35112) 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33672/0 (socket says 172.21.15.32:33672) 2023-09-30T22:30:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 -- 172.21.15.32:0/2758918431 learned_addr learned my addr 172.21.15.32:0/2758918431 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 -- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 msgr2=0x7ff1301317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 --2- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 -- 172.21.15.32:0/2758918431 >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 msgr2=0x7ff13009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 --2- 172.21.15.32:0/2758918431 >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 0x7ff13009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12ed9d700 1 -- 172.21.15.32:0/2758918431 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13009fb60 con 0x7ff13009f650 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12e59c700 1 --2- 172.21.15.32:0/2758918431 >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 0x7ff13009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.316+0000 7ff12dd9b700 1 --2- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.317+0000 7ff12ed9d700 1 --2- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff130131d30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff128009e50 tx=0x7ff12800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e9b656a1dfb1ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.317+0000 7ff12d59a700 1 -- 172.21.15.32:0/2758918431 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12800e070 con 0x7ff13009f650 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.317+0000 7ff12d59a700 1 -- 172.21.15.32:0/2758918431 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12800b910 con 0x7ff13009f650 2023-09-30T22:30:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.317+0000 7ff12d59a700 1 -- 172.21.15.32:0/2758918431 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12800bcc0 con 0x7ff13009f650 2023-09-30T22:30:04.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.317+0000 7ff134f4c700 1 -- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 msgr2=0x7ff130131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.317+0000 7ff134f4c700 1 --2- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff130131d30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff128009e50 tx=0x7ff12800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 -- 172.21.15.32:0/2758918431 shutdown_connections 2023-09-30T22:30:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 --2- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff130131d30 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 --2- 172.21.15.32:0/2758918431 >> v2:172.21.15.32:3300/0 conn(0x7ff13009e930 0x7ff13009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 --2- 172.21.15.32:0/2758918431 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 -- 172.21.15.32:0/2758918431 >> 172.21.15.32:0/2758918431 conn(0x7ff1300a4210 msgr2=0x7ff1300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 -- 172.21.15.32:0/2758918431 shutdown_connections 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.318+0000 7ff134f4c700 1 -- 172.21.15.32:0/2758918431 wait complete. 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.319+0000 7ff134f4c700 1 Processor -- start 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.319+0000 7ff134f4c700 1 -- start start 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff134f4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 0x7ff1301a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff134f4c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff134f4c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff1301a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff134f4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301a81a0 con 0x7ff13009e930 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff134f4c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1301a77e0 con 0x7ff13009f650 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff134f4c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff130071d20 con 0x7ff13009f240 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff12e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 0x7ff1301a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff12e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 0x7ff1301a3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35126/0 (socket says 172.21.15.32:35126) 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff12e59c700 1 -- 172.21.15.32:0/927581112 learned_addr learned my addr 172.21.15.32:0/927581112 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff12dd9b700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.320+0000 7ff12ed9d700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff1301a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12e59c700 1 -- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 msgr2=0x7ff1301a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12e59c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12e59c700 1 -- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 msgr2=0x7ff1301a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12e59c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff1301a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12e59c700 1 -- 172.21.15.32:0/927581112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200094a0 con 0x7ff13009e930 2023-09-30T22:30:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12dd9b700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12ed9d700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff1301a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff12e59c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 0x7ff1301a3d50 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff12000ec10 tx=0x7ff12000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120009780 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12000c850 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1240094a0 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000cbe0 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.321+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff128009b30 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.322+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff12000cd40 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.322+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff12003f750 con 0x7ff13009e930 2023-09-30T22:30:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.322+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130074ea0 con 0x7ff13009e930 2023-09-30T22:30:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.327+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1200232f0 con 0x7ff13009e930 2023-09-30T22:30:04.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.531+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff1301a7360 con 0x7ff13009e930 2023-09-30T22:30:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.532+0000 7ff11effd700 1 -- 172.21.15.32:0/927581112 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1301a7360 con 0x7ff13009e930 2023-09-30T22:30:04.533 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:04.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 msgr2=0x7ff1301a3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 0x7ff1301a3d50 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff12000ec10 tx=0x7ff12000ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 shutdown_connections 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff13009f650 0x7ff1301a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff13009e930 0x7ff1301a3d50 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 --2- 172.21.15.32:0/927581112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff13009f240 0x7ff1301a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 >> 172.21.15.32:0/927581112 conn(0x7ff1300a4210 msgr2=0x7ff1301b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.534+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 shutdown_connections 2023-09-30T22:30:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.535+0000 7ff134f4c700 1 -- 172.21.15.32:0/927581112 wait complete. 2023-09-30T22:30:04.549 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.622+0000 7fbb0292f700 1 Processor -- start 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.622+0000 7fbb0292f700 1 -- start start 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.622+0000 7fbb0292f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb0292f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 0x7fbafc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb0292f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb0292f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc06b460 con 0x7fbafc09e000 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb0292f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09b120 con 0x7fbafc09d6f0 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb0292f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09a7e0 con 0x7fbafc09e410 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbafb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 0x7fbafc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbafbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbafb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 0x7fbafc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35142/0 (socket says 172.21.15.32:35142) 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbafb7fe700 1 -- 172.21.15.32:0/3445659793 learned_addr learned my addr 172.21.15.32:0/3445659793 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33694/0 (socket says 172.21.15.32:33694) 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 -- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 msgr2=0x7fbafc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 --2- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 -- 172.21.15.32:0/3445659793 >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 msgr2=0x7fbafc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 --2- 172.21.15.32:0/3445659793 >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 0x7fbafc1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 -- 172.21.15.32:0/3445659793 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc09a500 con 0x7fbafc09e410 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbafb7fe700 1 --2- 172.21.15.32:0/3445659793 >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 0x7fbafc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbb00ecc700 1 --2- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc133f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009dc0 tx=0x7fbaf400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72287ae43697ef3f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.623+0000 7fbafaffd700 1 -- 172.21.15.32:0/3445659793 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400e070 con 0x7fbafc09e410 2023-09-30T22:30:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbafaffd700 1 -- 172.21.15.32:0/3445659793 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf400bae0 con 0x7fbafc09e410 2023-09-30T22:30:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbafaffd700 1 -- 172.21.15.32:0/3445659793 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400be90 con 0x7fbafc09e410 2023-09-30T22:30:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 -- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 msgr2=0x7fbafc133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 --2- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc133f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009dc0 tx=0x7fbaf400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 -- 172.21.15.32:0/3445659793 shutdown_connections 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 --2- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 --2- 172.21.15.32:0/3445659793 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc133f30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 --2- 172.21.15.32:0/3445659793 >> v2:172.21.15.32:3300/0 conn(0x7fbafc09e000 0x7fbafc1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 -- 172.21.15.32:0/3445659793 >> 172.21.15.32:0/3445659793 conn(0x7fbafc066f50 msgr2=0x7fbafc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 -- 172.21.15.32:0/3445659793 shutdown_connections 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.624+0000 7fbb0292f700 1 -- 172.21.15.32:0/3445659793 wait complete. 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.625+0000 7fbb0292f700 1 Processor -- start 2023-09-30T22:30:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.625+0000 7fbb0292f700 1 -- start start 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb0292f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc1c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb0292f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc09e000 0x7fbafc1c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb0292f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc1c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb0292f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc1a8170 con 0x7fbafc09e000 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb0292f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc1a6460 con 0x7fbafc09d6f0 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb0292f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0a2000 con 0x7fbafc09e410 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc1c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb00ecc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc1c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc1c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60692/0 (socket says 172.21.15.32:60692) 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 -- 172.21.15.32:0/1116810745 learned_addr learned my addr 172.21.15.32:0/1116810745 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafb7fe700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc09e000 0x7fbafc1c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 -- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 msgr2=0x7fbafc1c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc1c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 -- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc09e000 msgr2=0x7fbafc1c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc09e000 0x7fbafc1c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbafbfff700 1 -- 172.21.15.32:0/1116810745 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae4009550 con 0x7fbafc09d6f0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.626+0000 7fbb00ecc700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.627+0000 7fbafbfff700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc1c80b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00e9b0 tx=0x7fbaec00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.627+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec009710 con 0x7fbafc09d6f0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.627+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec00ee50 con 0x7fbafc09d6f0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.627+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf4009a70 con 0x7fbafc09d6f0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.627+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00ca50 con 0x7fbafc09d6f0 2023-09-30T22:30:04.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.627+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc09c760 con 0x7fbafc09d6f0 2023-09-30T22:30:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.628+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbaec00cbb0 con 0x7fbafc09d6f0 2023-09-30T22:30:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.628+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbaec03faa0 con 0x7fbafc09d6f0 2023-09-30T22:30:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.628+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc1a8bc0 con 0x7fbafc09d6f0 2023-09-30T22:30:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.633+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbaec017080 con 0x7fbafc09d6f0 2023-09-30T22:30:04.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:04.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.837+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbafc074ea0 con 0x7fbafc09d6f0 2023-09-30T22:30:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.838+0000 7fbaf8ff9700 1 -- 172.21.15.32:0/1116810745 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbaec021020 con 0x7fbafc09d6f0 2023-09-30T22:30:04.839 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 msgr2=0x7fbafc1c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc1c80b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00e9b0 tx=0x7fbaec00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 shutdown_connections 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbafc09e410 0x7fbafc1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbafc09d6f0 0x7fbafc1c80b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 --2- 172.21.15.32:0/1116810745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbafc09e000 0x7fbafc1c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 >> 172.21.15.32:0/1116810745 conn(0x7fbafc066f50 msgr2=0x7fbafc06f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 shutdown_connections 2023-09-30T22:30:04.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.840+0000 7fbb0292f700 1 -- 172.21.15.32:0/1116810745 wait complete. 2023-09-30T22:30:04.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.863+0000 7f7a5d99b700 1 Processor -- start 2023-09-30T22:30:04.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.863+0000 7f7a5d99b700 1 -- start start 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a5d99b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a5d99b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a5809d630 0x7f7a580a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a5d99b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a58133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a5d99b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580a2a70 con 0x7f7a5809d630 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a5d99b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580a01d0 con 0x7f7a58133930 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a5d99b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5809e2b0 con 0x7f7a580a0a30 2023-09-30T22:30:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a58133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a56ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a56ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a580a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33710/0 (socket says 172.21.15.32:33710) 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a56ffd700 1 -- 172.21.15.32:0/51901795 learned_addr learned my addr 172.21.15.32:0/51901795 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a58133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60700/0 (socket says 172.21.15.32:60700) 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a567fc700 1 --2- 172.21.15.32:0/51901795 >> v2:172.21.15.32:3300/0 conn(0x7f7a5809d630 0x7f7a580a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 -- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 msgr2=0x7f7a580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 --2- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 -- 172.21.15.32:0/51901795 >> v2:172.21.15.32:3300/0 conn(0x7f7a5809d630 msgr2=0x7f7a580a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 --2- 172.21.15.32:0/51901795 >> v2:172.21.15.32:3300/0 conn(0x7f7a5809d630 0x7f7a580a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 -- 172.21.15.32:0/51901795 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5809dc50 con 0x7f7a58133930 2023-09-30T22:30:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a56ffd700 1 --2- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.864+0000 7f7a577fe700 1 --2- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a58133dd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009d90 tx=0x7f7a5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9025bc30fd2a8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a55ffb700 1 -- 172.21.15.32:0/51901795 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000e070 con 0x7f7a58133930 2023-09-30T22:30:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a55ffb700 1 -- 172.21.15.32:0/51901795 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000bb20 con 0x7f7a58133930 2023-09-30T22:30:04.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a55ffb700 1 -- 172.21.15.32:0/51901795 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50012430 con 0x7f7a58133930 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 -- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 msgr2=0x7f7a58133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a58133dd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009d90 tx=0x7f7a5000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 -- 172.21.15.32:0/51901795 shutdown_connections 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a58133dd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/51901795 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/51901795 >> v2:172.21.15.32:3300/0 conn(0x7f7a5809d630 0x7f7a580a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 -- 172.21.15.32:0/51901795 >> 172.21.15.32:0/51901795 conn(0x7f7a580a3050 msgr2=0x7f7a580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 -- 172.21.15.32:0/51901795 shutdown_connections 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.865+0000 7f7a5d99b700 1 -- 172.21.15.32:0/51901795 wait complete. 2023-09-30T22:30:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.867+0000 7f7a5d99b700 1 Processor -- start 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.867+0000 7f7a5d99b700 1 -- start start 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a5d99b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5809d630 0x7f7a5811f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a5d99b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a5811f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a5d99b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a581c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a5d99b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58122390 con 0x7f7a5809d630 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a5d99b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58122570 con 0x7f7a58133930 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a5d99b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581333e0 con 0x7f7a580a0a30 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a5811f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a5811f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33722/0 (socket says 172.21.15.32:33722) 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a577fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a581c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 -- 172.21.15.32:0/309189557 learned_addr learned my addr 172.21.15.32:0/309189557 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a56ffd700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5809d630 0x7f7a5811f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 -- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 msgr2=0x7f7a581c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a581c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 -- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5809d630 msgr2=0x7f7a5811f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5809d630 0x7f7a5811f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a567fc700 1 -- 172.21.15.32:0/309189557 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48009550 con 0x7f7a580a0a30 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a577fe700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a581c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.868+0000 7f7a56ffd700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5809d630 0x7f7a5811f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.869+0000 7f7a567fc700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a5811f9f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00e980 tx=0x7f7a4c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.869+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c009710 con 0x7f7a580a0a30 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.869+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00c690 con 0x7f7a580a0a30 2023-09-30T22:30:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.869+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a50009a70 con 0x7f7a580a0a30 2023-09-30T22:30:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.869+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00ca20 con 0x7f7a580a0a30 2023-09-30T22:30:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.869+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a580a0410 con 0x7f7a580a0a30 2023-09-30T22:30:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.870+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a58120e50 con 0x7f7a580a0a30 2023-09-30T22:30:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.870+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a4c00cb80 con 0x7f7a580a0a30 2023-09-30T22:30:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.870+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7a4c03ffb0 con 0x7f7a580a0a30 2023-09-30T22:30:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.875+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a4c06b750 con 0x7f7a580a0a30 2023-09-30T22:30:04.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.912+0000 7fea085bf700 1 Processor -- start 2023-09-30T22:30:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea085bf700 1 -- start start 2023-09-30T22:30:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea085bf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea000a1510 0x7fea00138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea085bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea00139250 0x7fea0013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea085bf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0009edd0 con 0x7fea000a0a30 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea085bf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea000a18e0 con 0x7fea000a1510 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea085bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea0009e1f0 con 0x7fea00139250 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea05b5a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea000a1510 0x7fea00138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea0635b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea000a0a30 0x7fea000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41746/0 (socket says 172.21.15.32:41746) 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.913+0000 7fea0635b700 1 -- 172.21.15.32:0/2533006091 learned_addr learned my addr 172.21.15.32:0/2533006091 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea06b5c700 1 --2- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea00139250 0x7fea0013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05b5a700 1 -- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea00139250 msgr2=0x7fea0013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05b5a700 1 --2- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea00139250 0x7fea0013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05b5a700 1 -- 172.21.15.32:0/2533006091 >> v1:172.21.15.32:6789/0 conn(0x7fea000a0a30 legacy=0x7fea000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05b5a700 1 -- 172.21.15.32:0/2533006091 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0009dc50 con 0x7fea000a1510 2023-09-30T22:30:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05359700 1 -- 172.21.15.32:0/2533006091 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052949908 0 0) 0x7fea0009edd0 con 0x7fea000a0a30 2023-09-30T22:30:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05b5a700 1 --2- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea000a1510 0x7fea00138d10 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009840 tx=0x7fe9f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaef890b7c613c5a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05359700 1 -- 172.21.15.32:0/2533006091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4011830 con 0x7fea000a1510 2023-09-30T22:30:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05359700 1 -- 172.21.15.32:0/2533006091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4011990 con 0x7fea000a1510 2023-09-30T22:30:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.914+0000 7fea05359700 1 -- 172.21.15.32:0/2533006091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4011d20 con 0x7fea000a1510 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 -- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea000a1510 msgr2=0x7fea00138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 --2- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea000a1510 0x7fea00138d10 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009840 tx=0x7fe9f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 -- 172.21.15.32:0/2533006091 shutdown_connections 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 --2- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea00139250 0x7fea0013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 --2- 172.21.15.32:0/2533006091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea000a1510 0x7fea00138d10 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 -- 172.21.15.32:0/2533006091 >> 172.21.15.32:0/2533006091 conn(0x7fea000a6550 msgr2=0x7fea000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 -- 172.21.15.32:0/2533006091 shutdown_connections 2023-09-30T22:30:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.915+0000 7fea085bf700 1 -- 172.21.15.32:0/2533006091 wait complete. 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 Processor -- start 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 -- start start 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea000a0a30 0x7fea00122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea000a1510 0x7fea00122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 0x7fea001232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea001243e0 con 0x7fea00139250 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea00124a40 con 0x7fea000a1510 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.916+0000 7fea085bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea001386c0 con 0x7fea000a0a30 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 0x7fea001232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea05b5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea000a1510 0x7fea00122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 0x7fea001232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35198/0 (socket says 172.21.15.32:35198) 2023-09-30T22:30:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea05b5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea000a1510 0x7fea00122da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60718/0 (socket says 172.21.15.32:60718) 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 -- 172.21.15.32:0/2577792365 learned_addr learned my addr 172.21.15.32:0/2577792365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea0635b700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea000a0a30 0x7fea00122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 -- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea000a0a30 msgr2=0x7fea00122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea000a0a30 0x7fea00122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 -- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea000a1510 msgr2=0x7fea00122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea000a1510 0x7fea00122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 -- 172.21.15.32:0/2577792365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40094a0 con 0x7fea00139250 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea06b5c700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 0x7fea001232e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009e30 tx=0x7fe9f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8009ca0 con 0x7fea00139250 2023-09-30T22:30:04.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800f040 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea001262f0 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8014480 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.917+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea000a0270 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.918+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea00074ea0 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.919+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe9f80145e0 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.919+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe9f80402e0 con 0x7fea00139250 2023-09-30T22:30:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:04.924+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9f8007900 con 0x7fea00139250 2023-09-30T22:30:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.082+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7a58074770 con 0x7f7a580a0a30 2023-09-30T22:30:05.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.082+0000 7f7a3ffff700 1 -- 172.21.15.32:0/309189557 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a4c021090 con 0x7f7a580a0a30 2023-09-30T22:30:05.083 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:05.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 msgr2=0x7f7a5811f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a5811f9f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00e980 tx=0x7f7a4c00ec90 comp rx=0 tx=0).stop 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 shutdown_connections 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a58133930 0x7f7a581c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5809d630 0x7f7a5811f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 --2- 172.21.15.32:0/309189557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a580a0a30 0x7f7a5811f9f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.084+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 >> 172.21.15.32:0/309189557 conn(0x7f7a580a3050 msgr2=0x7f7a5809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.085+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 shutdown_connections 2023-09-30T22:30:05.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.085+0000 7f7a5d99b700 1 -- 172.21.15.32:0/309189557 wait complete. 2023-09-30T22:30:05.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.133+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea000a28e0 con 0x7fea00139250 2023-09-30T22:30:05.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.133+0000 7fe9feffd700 1 -- 172.21.15.32:0/2577792365 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9f8018020 con 0x7fea00139250 2023-09-30T22:30:05.134 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 msgr2=0x7fea001232e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 0x7fea001232e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009e30 tx=0x7fe9f800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 shutdown_connections 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea00139250 0x7fea001232e0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea000a0a30 0x7fea00122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 --2- 172.21.15.32:0/2577792365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea000a1510 0x7fea00122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 >> 172.21.15.32:0/2577792365 conn(0x7fea000a6550 msgr2=0x7fea000a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 shutdown_connections 2023-09-30T22:30:05.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.136+0000 7fea085bf700 1 -- 172.21.15.32:0/2577792365 wait complete. 2023-09-30T22:30:05.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:05.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:05.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.693+0000 7f67332c5700 1 Processor -- start 2023-09-30T22:30:05.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f67332c5700 1 -- start start 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f67332c5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f672c09e930 0x7f672c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f67332c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 0x7f672c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f67332c5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c070940 con 0x7f672c09f650 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f67332c5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c06fd60 con 0x7f672c09e930 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f67332c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c06fa80 con 0x7f672c09f240 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f6730860700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 0x7f672c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f6731061700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f672c09e930 0x7f672c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f6730860700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 0x7f672c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60728/0 (socket says 172.21.15.32:60728) 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.694+0000 7f6730860700 1 -- 172.21.15.32:0/4217165773 learned_addr learned my addr 172.21.15.32:0/4217165773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:05.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6731061700 1 -- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 msgr2=0x7f672c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6731061700 1 --2- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 0x7f672c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6731061700 1 -- 172.21.15.32:0/4217165773 >> v1:172.21.15.32:6789/0 conn(0x7f672c09f650 legacy=0x7f672c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6731061700 1 -- 172.21.15.32:0/4217165773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c09fb60 con 0x7f672c09e930 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6723fff700 1 -- 172.21.15.32:0/4217165773 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655202235 0 0) 0x7f672c070940 con 0x7f672c09f650 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6730860700 1 --2- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 0x7f672c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6731061700 1 --2- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09e930 0x7f672c09ed00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6728009840 tx=0x7f6728009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfa88117e0e029d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6723fff700 1 -- 172.21.15.32:0/4217165773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728011830 con 0x7f672c09e930 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6723fff700 1 -- 172.21.15.32:0/4217165773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728011990 con 0x7f672c09e930 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f6723fff700 1 -- 172.21.15.32:0/4217165773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728011d20 con 0x7f672c09e930 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f67332c5700 1 -- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09e930 msgr2=0x7f672c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.695+0000 7f67332c5700 1 --2- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09e930 0x7f672c09ed00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6728009840 tx=0x7f6728009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.696+0000 7f67332c5700 1 -- 172.21.15.32:0/4217165773 shutdown_connections 2023-09-30T22:30:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.696+0000 7f67332c5700 1 --2- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09e930 0x7f672c09ed00 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.696+0000 7f67332c5700 1 --2- 172.21.15.32:0/4217165773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09f240 0x7f672c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.696+0000 7f67332c5700 1 -- 172.21.15.32:0/4217165773 >> 172.21.15.32:0/4217165773 conn(0x7f672c0a4210 msgr2=0x7f672c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.696+0000 7f67332c5700 1 -- 172.21.15.32:0/4217165773 shutdown_connections 2023-09-30T22:30:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.696+0000 7f67332c5700 1 -- 172.21.15.32:0/4217165773 wait complete. 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 Processor -- start 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 -- start start 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 0x7f672c06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09f240 0x7f672c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672c09f650 0x7f672c06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1a87b0 con 0x7f672c09f240 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1a97c0 con 0x7f672c09e930 2023-09-30T22:30:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f67332c5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0a31d0 con 0x7f672c09f650 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f6731061700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 0x7f672c06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f6731061700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 0x7f672c06b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60740/0 (socket says 172.21.15.32:60740) 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f6731061700 1 -- 172.21.15.32:0/1219440445 learned_addr learned my addr 172.21.15.32:0/1219440445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f6730860700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09f240 0x7f672c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.697+0000 7f6731862700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672c09f650 0x7f672c06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6731061700 1 -- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672c09f650 msgr2=0x7f672c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6731061700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672c09f650 0x7f672c06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6731061700 1 -- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09f240 msgr2=0x7f672c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6731061700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09f240 0x7f672c06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6731061700 1 -- 172.21.15.32:0/1219440445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009550 con 0x7f672c09e930 2023-09-30T22:30:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6731061700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 0x7f672c06b880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f67280090b0 tx=0x7f6728004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728004400 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672801f070 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67280094a0 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67280038e0 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.698+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c0a0190 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.699+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6728021030 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.699+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f67280407d0 con 0x7f672c09e930 2023-09-30T22:30:05.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.699+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c074ea0 con 0x7f672c09e930 2023-09-30T22:30:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.704+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6728015020 con 0x7f672c09e930 2023-09-30T22:30:05.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.909+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f672c06d3d0 con 0x7f672c09e930 2023-09-30T22:30:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.910+0000 7f6721ffb700 1 -- 172.21.15.32:0/1219440445 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67280199a0 con 0x7f672c09e930 2023-09-30T22:30:05.911 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 msgr2=0x7f672c06b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 0x7f672c06b880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f67280090b0 tx=0x7f6728004750 comp rx=0 tx=0).stop 2023-09-30T22:30:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 shutdown_connections 2023-09-30T22:30:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f672c09f650 0x7f672c06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f672c09e930 0x7f672c06b880 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 --2- 172.21.15.32:0/1219440445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f672c09f240 0x7f672c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 >> 172.21.15.32:0/1219440445 conn(0x7f672c0a4210 msgr2=0x7f672c0a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 shutdown_connections 2023-09-30T22:30:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.912+0000 7f67332c5700 1 -- 172.21.15.32:0/1219440445 wait complete. 2023-09-30T22:30:05.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.997+0000 7f8def711700 1 Processor -- start 2023-09-30T22:30:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8def711700 1 -- start start 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8def711700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809e930 0x7f8de809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8def711700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f650 0x7f8de8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8def711700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de8070940 con 0x7f8de809f240 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8def711700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de806fd60 con 0x7f8de809f650 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8def711700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8de806fa80 con 0x7f8de809e930 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8deccac700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8de809f240 0x7f8de81317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41758/0 (socket says 172.21.15.32:41758) 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8deccac700 1 -- 172.21.15.32:0/3772908079 learned_addr learned my addr 172.21.15.32:0/3772908079 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:06.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8ded4ad700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809e930 0x7f8de809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.998+0000 7f8dedcae700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f650 0x7f8de8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ddffff700 1 -- 172.21.15.32:0/3772908079 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813382964 0 0) 0x7f8de8070940 con 0x7f8de809f240 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ddffff700 1 -- 172.21.15.32:0/3772908079 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dd40035b0 con 0x7f8de809f240 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f650 msgr2=0x7f8de8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ded4ad700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f650 0x7f8de8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3772908079 >> v1:172.21.15.32:6789/0 conn(0x7f8de809f240 legacy=0x7f8de81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3772908079 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de809fb60 con 0x7f8de809e930 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8dedcae700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f650 0x7f8de8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ded4ad700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809e930 0x7f8de809ed00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009870 tx=0x7f8de4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2f08da8322b7f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ddffff700 1 -- 172.21.15.32:0/3772908079 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4011830 con 0x7f8de809e930 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ddffff700 1 -- 172.21.15.32:0/3772908079 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de4011990 con 0x7f8de809e930 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:05.999+0000 7f8ddffff700 1 -- 172.21.15.32:0/3772908079 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4011d20 con 0x7f8de809e930 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 -- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809e930 msgr2=0x7f8de809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809e930 0x7f8de809ed00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009870 tx=0x7f8de4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 -- 172.21.15.32:0/3772908079 shutdown_connections 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f650 0x7f8de8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 --2- 172.21.15.32:0/3772908079 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809e930 0x7f8de809ed00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 -- 172.21.15.32:0/3772908079 >> 172.21.15.32:0/3772908079 conn(0x7f8de80a4210 msgr2=0x7f8de80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 -- 172.21.15.32:0/3772908079 shutdown_connections 2023-09-30T22:30:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.000+0000 7f8def711700 1 -- 172.21.15.32:0/3772908079 wait complete. 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.001+0000 7f8def711700 1 Processor -- start 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.001+0000 7f8def711700 1 -- start start 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.001+0000 7f8def711700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 0x7f8de81a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.001+0000 7f8def711700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f240 0x7f8de81a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.001+0000 7f8def711700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809f650 0x7f8de81a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.001+0000 7f8def711700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de81a7670 con 0x7f8de809e930 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8def711700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de81a8010 con 0x7f8de809f240 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8def711700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8de81a6e00 con 0x7f8de809f650 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 0x7f8de81a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 0x7f8de81a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35224/0 (socket says 172.21.15.32:35224) 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3436395390 learned_addr learned my addr 172.21.15.32:0/3436395390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8deccac700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f240 0x7f8de81a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8dedcae700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809f650 0x7f8de81a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809f650 msgr2=0x7f8de81a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809f650 0x7f8de81a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f240 msgr2=0x7f8de81a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f240 0x7f8de81a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 -- 172.21.15.32:0/3436395390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de0009a70 con 0x7f8de809e930 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8ded4ad700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 0x7f8de81a3b50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8de40090b0 tx=0x7f8de4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4004450 con 0x7f8de809e930 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de401f070 con 0x7f8de809e930 2023-09-30T22:30:06.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de40094a0 con 0x7f8de809e930 2023-09-30T22:30:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.002+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40038e0 con 0x7f8de809e930 2023-09-30T22:30:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.003+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de80a0190 con 0x7f8de809e930 2023-09-30T22:30:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.003+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8de4003a40 con 0x7f8de809e930 2023-09-30T22:30:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.004+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8de4021070 con 0x7f8de809e930 2023-09-30T22:30:06.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.004+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de81a0b30 con 0x7f8de809e930 2023-09-30T22:30:06.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.009+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8de4015030 con 0x7f8de809e930 2023-09-30T22:30:06.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:06.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:06.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.218+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8de81a71f0 con 0x7f8de809e930 2023-09-30T22:30:06.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.218+0000 7f8dddffb700 1 -- 172.21.15.32:0/3436395390 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8de4026480 con 0x7f8de809e930 2023-09-30T22:30:06.219 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 msgr2=0x7f8de81a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 0x7f8de81a3b50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8de40090b0 tx=0x7f8de4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 shutdown_connections 2023-09-30T22:30:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8de809f650 0x7f8de81a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8de809e930 0x7f8de81a3b50 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 --2- 172.21.15.32:0/3436395390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8de809f240 0x7f8de81a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 >> 172.21.15.32:0/3436395390 conn(0x7f8de80a4210 msgr2=0x7f8de81b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 shutdown_connections 2023-09-30T22:30:06.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.221+0000 7f8def711700 1 -- 172.21.15.32:0/3436395390 wait complete. 2023-09-30T22:30:06.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.241+0000 7f8dc2977700 1 Processor -- start 2023-09-30T22:30:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.241+0000 7f8dc2977700 1 -- start start 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc2977700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09d6f0 0x7f8dbc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc2977700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09e410 0x7f8dbc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc2977700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc09f130 con 0x7f8dbc09e000 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc2977700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc09f520 con 0x7f8dbc09e410 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc2977700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc09e930 con 0x7f8dbc09d6f0 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09d6f0 0x7f8dbc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc0f14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09e410 0x7f8dbc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dbb7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8dbc09e000 0x7f8dbc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41768/0 (socket says 172.21.15.32:41768) 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dbb7fe700 1 -- 172.21.15.32:0/4100839968 learned_addr learned my addr 172.21.15.32:0/4100839968 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dbaffd700 1 -- 172.21.15.32:0/4100839968 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988121919 0 0) 0x7f8dbc09f130 con 0x7f8dbc09e000 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dbaffd700 1 -- 172.21.15.32:0/4100839968 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da40035b0 con 0x7f8dbc09e000 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc0f14700 1 -- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09d6f0 msgr2=0x7f8dbc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc0f14700 1 --2- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09d6f0 0x7f8dbc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc0f14700 1 -- 172.21.15.32:0/4100839968 >> v1:172.21.15.32:6789/0 conn(0x7f8dbc09e000 legacy=0x7f8dbc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.242+0000 7f8dc0f14700 1 -- 172.21.15.32:0/4100839968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc09b2d0 con 0x7f8dbc09e410 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.243+0000 7f8dbbfff700 1 --2- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09d6f0 0x7f8dbc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.243+0000 7f8dc0f14700 1 --2- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09e410 0x7f8dbc130d70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009dc0 tx=0x7f8db400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2296045343945fc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.243+0000 7f8dbaffd700 1 -- 172.21.15.32:0/4100839968 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db400e070 con 0x7f8dbc09e410 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.243+0000 7f8dbaffd700 1 -- 172.21.15.32:0/4100839968 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db400bb20 con 0x7f8dbc09e410 2023-09-30T22:30:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.243+0000 7f8dbaffd700 1 -- 172.21.15.32:0/4100839968 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4012430 con 0x7f8dbc09e410 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 -- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09e410 msgr2=0x7f8dbc130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 --2- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09e410 0x7f8dbc130d70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009dc0 tx=0x7f8db400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 -- 172.21.15.32:0/4100839968 shutdown_connections 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 --2- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09e410 0x7f8dbc130d70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 --2- 172.21.15.32:0/4100839968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09d6f0 0x7f8dbc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 -- 172.21.15.32:0/4100839968 >> 172.21.15.32:0/4100839968 conn(0x7f8dbc066f50 msgr2=0x7f8dbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 -- 172.21.15.32:0/4100839968 shutdown_connections 2023-09-30T22:30:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.244+0000 7f8dc2977700 1 -- 172.21.15.32:0/4100839968 wait complete. 2023-09-30T22:30:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.245+0000 7f8dc2977700 1 Processor -- start 2023-09-30T22:30:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.245+0000 7f8dc2977700 1 -- start start 2023-09-30T22:30:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.245+0000 7f8dc2977700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09d6f0 0x7f8dbc11dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc2977700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 0x7f8dbc11e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc2977700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 0x7f8dbc11e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc2977700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc1221b0 con 0x7f8dbc09e410 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc2977700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc1217f0 con 0x7f8dbc09d6f0 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc2977700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc120f80 con 0x7f8dbc09e000 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 0x7f8dbc11e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc0f14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 0x7f8dbc11e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 0x7f8dbc11e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33786/0 (socket says 172.21.15.32:33786) 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc0f14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 0x7f8dbc11e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35238/0 (socket says 172.21.15.32:35238) 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 -- 172.21.15.32:0/51741014 learned_addr learned my addr 172.21.15.32:0/51741014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbbfff700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09d6f0 0x7f8dbc11dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 -- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09d6f0 msgr2=0x7f8dbc11dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09d6f0 0x7f8dbc11dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 -- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 msgr2=0x7f8dbc11e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 0x7f8dbc11e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 -- 172.21.15.32:0/51741014 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac00b740 con 0x7f8dbc09e000 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbbfff700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09d6f0 0x7f8dbc11dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dc0f14700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 0x7f8dbc11e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8dbb7fe700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 0x7f8dbc11e210 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c700 tx=0x7f8db000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.246+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0005d70 con 0x7f8dbc09e000 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.247+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00070d0 con 0x7f8dbc09e000 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.247+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db4009a70 con 0x7f8dbc09e000 2023-09-30T22:30:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.247+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007460 con 0x7f8dbc09e000 2023-09-30T22:30:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.247+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc120b80 con 0x7f8dbc09e000 2023-09-30T22:30:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.248+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc11ac20 con 0x7f8dbc09e000 2023-09-30T22:30:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.248+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8db00075c0 con 0x7f8dbc09e000 2023-09-30T22:30:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.248+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8db003efa0 con 0x7f8dbc09e000 2023-09-30T22:30:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.257+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8db001bb10 con 0x7f8dbc09e000 2023-09-30T22:30:06.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.295+0000 7feb833a0700 1 Processor -- start 2023-09-30T22:30:06.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.295+0000 7feb833a0700 1 -- start start 2023-09-30T22:30:06.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb833a0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 0x7feb7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb833a0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb7c0a1340 0x7feb7c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb833a0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c09edd0 con 0x7feb7c133fa0 2023-09-30T22:30:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb833a0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c09e1f0 con 0x7feb7c0a1340 2023-09-30T22:30:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb833a0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c09dc50 con 0x7feb7c0a0a30 2023-09-30T22:30:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8113c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 0x7feb7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8113c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 0x7feb7c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33798/0 (socket says 172.21.15.32:33798) 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8113c700 1 -- 172.21.15.32:0/1683130703 learned_addr learned my addr 172.21.15.32:0/1683130703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8093b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb7c0a1340 0x7feb7c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8113c700 1 -- 172.21.15.32:0/1683130703 >> v2:172.21.15.32:3300/0 conn(0x7feb7c0a1340 msgr2=0x7feb7c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8113c700 1 --2- 172.21.15.32:0/1683130703 >> v2:172.21.15.32:3300/0 conn(0x7feb7c0a1340 0x7feb7c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.296+0000 7feb8113c700 1 -- 172.21.15.32:0/1683130703 >> v1:172.21.15.32:6789/0 conn(0x7feb7c133fa0 legacy=0x7feb7c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb7bfff700 1 -- 172.21.15.32:0/1683130703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210353343 0 0) 0x7feb7c09edd0 con 0x7feb7c133fa0 2023-09-30T22:30:06.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb8113c700 1 -- 172.21.15.32:0/1683130703 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c0a1fe0 con 0x7feb7c0a0a30 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb8093b700 1 --2- 172.21.15.32:0/1683130703 >> v2:172.21.15.32:3300/0 conn(0x7feb7c0a1340 0x7feb7c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb8113c700 1 --2- 172.21.15.32:0/1683130703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 0x7feb7c0a0e00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feb740098f0 tx=0x7feb74009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e115959f1be38b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb7bfff700 1 -- 172.21.15.32:0/1683130703 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74011830 con 0x7feb7c0a0a30 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb7bfff700 1 -- 172.21.15.32:0/1683130703 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb74011990 con 0x7feb7c0a0a30 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.297+0000 7feb7bfff700 1 -- 172.21.15.32:0/1683130703 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74011d20 con 0x7feb7c0a0a30 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 -- 172.21.15.32:0/1683130703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 msgr2=0x7feb7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 --2- 172.21.15.32:0/1683130703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 0x7feb7c0a0e00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feb740098f0 tx=0x7feb74009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 -- 172.21.15.32:0/1683130703 shutdown_connections 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 --2- 172.21.15.32:0/1683130703 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a0a30 0x7feb7c0a0e00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 --2- 172.21.15.32:0/1683130703 >> v2:172.21.15.32:3300/0 conn(0x7feb7c0a1340 0x7feb7c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 -- 172.21.15.32:0/1683130703 >> 172.21.15.32:0/1683130703 conn(0x7feb7c0a6550 msgr2=0x7feb7c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 -- 172.21.15.32:0/1683130703 shutdown_connections 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.298+0000 7feb833a0700 1 -- 172.21.15.32:0/1683130703 wait complete. 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 Processor -- start 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 -- start start 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7c0a0a30 0x7feb7c1c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 0x7feb7c1c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7c133fa0 0x7feb7c122850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c125d40 con 0x7feb7c133fa0 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1263e0 con 0x7feb7c0a0a30 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb833a0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0a35a0 con 0x7feb7c0a1340 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb8093b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 0x7feb7c1c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb8193d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7c133fa0 0x7feb7c122850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb8093b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 0x7feb7c1c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33802/0 (socket says 172.21.15.32:33802) 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.299+0000 7feb8093b700 1 -- 172.21.15.32:0/3442493279 learned_addr learned my addr 172.21.15.32:0/3442493279 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8093b700 1 -- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7c0a0a30 msgr2=0x7feb7c1c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8093b700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7c0a0a30 0x7feb7c1c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8093b700 1 -- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7c133fa0 msgr2=0x7feb7c122850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8093b700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7c133fa0 0x7feb7c122850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8093b700 1 -- 172.21.15.32:0/3442493279 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700094a0 con 0x7feb7c0a1340 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8193d700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7c133fa0 0x7feb7c122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb8093b700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 0x7feb7c1c8ad0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feb7000ebe0 tx=0x7feb7000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70009800 con 0x7feb7c0a1340 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7000c910 con 0x7feb7c0a1340 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb74009550 con 0x7feb7c0a1340 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000cc80 con 0x7feb7c0a1340 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.300+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c1243d0 con 0x7feb7c0a1340 2023-09-30T22:30:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.301+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb70013470 con 0x7feb7c0a1340 2023-09-30T22:30:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.301+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb7003fc00 con 0x7feb7c0a1340 2023-09-30T22:30:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.301+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c074ea0 con 0x7feb7c0a1340 2023-09-30T22:30:06.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.306+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb70017070 con 0x7feb7c0a1340 2023-09-30T22:30:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.465+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8dbc074ea0 con 0x7f8dbc09e000 2023-09-30T22:30:06.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.466+0000 7f8db8ff9700 1 -- 172.21.15.32:0/51741014 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8db00192c0 con 0x7f8dbc09e000 2023-09-30T22:30:06.467 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 msgr2=0x7f8dbc11e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 0x7f8dbc11e210 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c700 tx=0x7f8db000ca10 comp rx=0 tx=0).stop 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 shutdown_connections 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dbc09e410 0x7f8dbc11e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8dbc09d6f0 0x7f8dbc11dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 --2- 172.21.15.32:0/51741014 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8dbc09e000 0x7f8dbc11e210 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 >> 172.21.15.32:0/51741014 conn(0x7f8dbc066f50 msgr2=0x7f8dbc130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.468+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 shutdown_connections 2023-09-30T22:30:06.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.469+0000 7f8dc2977700 1 -- 172.21.15.32:0/51741014 wait complete. 2023-09-30T22:30:06.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.516+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feb7c0a28e0 con 0x7feb7c0a1340 2023-09-30T22:30:06.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.517+0000 7feb79ffb700 1 -- 172.21.15.32:0/3442493279 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb7002a090 con 0x7feb7c0a1340 2023-09-30T22:30:06.518 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 msgr2=0x7feb7c1c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 0x7feb7c1c8ad0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feb7000ebe0 tx=0x7feb7000eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 shutdown_connections 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7c133fa0 0x7feb7c122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7c0a0a30 0x7feb7c1c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 --2- 172.21.15.32:0/3442493279 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7c0a1340 0x7feb7c1c8ad0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 >> 172.21.15.32:0/3442493279 conn(0x7feb7c0a6550 msgr2=0x7feb7c11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 shutdown_connections 2023-09-30T22:30:06.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:06.519+0000 7feb833a0700 1 -- 172.21.15.32:0/3442493279 wait complete. 2023-09-30T22:30:06.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:06.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.074+0000 7f24451a7700 1 Processor -- start 2023-09-30T22:30:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.074+0000 7f24451a7700 1 -- start start 2023-09-30T22:30:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.074+0000 7f24451a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f24451a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f24451a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f244009e410 0x7f2440130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f24451a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244009f520 con 0x7f244009e410 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f24451a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244009f130 con 0x7f244009d6f0 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f24451a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f244009e930 con 0x7f244009e000 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f244009e410 0x7f2440130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f244009e410 0x7f2440130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35274/0 (socket says 172.21.15.32:35274) 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 -- 172.21.15.32:0/1603982490 learned_addr learned my addr 172.21.15.32:0/1603982490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243e59c700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 -- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 msgr2=0x7f2440130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 -- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 msgr2=0x7f244009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 -- 172.21.15.32:0/1603982490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244009b2d0 con 0x7f244009e410 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243ed9d700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243f59e700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440130ba0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e50 tx=0x7f243800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d81de4d1c64349 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.075+0000 7f243dd9b700 1 -- 172.21.15.32:0/1603982490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e070 con 0x7f244009e410 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f243dd9b700 1 -- 172.21.15.32:0/1603982490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800b910 con 0x7f244009e410 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f243dd9b700 1 -- 172.21.15.32:0/1603982490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800bcc0 con 0x7f244009e410 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 -- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 msgr2=0x7f2440130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440130ba0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e50 tx=0x7f243800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 -- 172.21.15.32:0/1603982490 shutdown_connections 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440130ba0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 --2- 172.21.15.32:0/1603982490 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 -- 172.21.15.32:0/1603982490 >> 172.21.15.32:0/1603982490 conn(0x7f2440066f50 msgr2=0x7f244009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 -- 172.21.15.32:0/1603982490 shutdown_connections 2023-09-30T22:30:07.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.076+0000 7f24451a7700 1 -- 172.21.15.32:0/1603982490 wait complete. 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 Processor -- start 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 -- start start 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244011ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440120510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440120a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440123af0 con 0x7f244009e410 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440123280 con 0x7f244009d6f0 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f24451a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2440122410 con 0x7f244009e000 2023-09-30T22:30:07.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f243e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f243f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440120a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f243e59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440120510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33814/0 (socket says 172.21.15.32:33814) 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f243e59c700 1 -- 172.21.15.32:0/4047632369 learned_addr learned my addr 172.21.15.32:0/4047632369 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f243f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440120a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35286/0 (socket says 172.21.15.32:35286) 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.078+0000 7f243ed9d700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244011ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243e59c700 1 -- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 msgr2=0x7f244011ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243e59c700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244011ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243e59c700 1 -- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 msgr2=0x7f2440120a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243e59c700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440120a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243e59c700 1 -- 172.21.15.32:0/4047632369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300094a0 con 0x7f244009e000 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243f59e700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440120a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243ed9d700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244011ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f243e59c700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440120510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f243400e980 tx=0x7f243400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434009710 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243400c690 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2438009b30 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400ca20 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.079+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440130390 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.080+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f243400cb80 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.080+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f243403f700 con 0x7f244009e000 2023-09-30T22:30:07.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.080+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440074ea0 con 0x7f244009e000 2023-09-30T22:30:07.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.085+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24340232f0 con 0x7f244009e000 2023-09-30T22:30:07.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.296+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24401235b0 con 0x7f244009e000 2023-09-30T22:30:07.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.296+0000 7f24277fe700 1 -- 172.21.15.32:0/4047632369 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f243401bd00 con 0x7f244009e000 2023-09-30T22:30:07.334 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.298+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 msgr2=0x7f2440120510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.298+0000 7f24451a7700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440120510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f243400e980 tx=0x7f243400ec90 comp rx=0 tx=0).stop 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 shutdown_connections 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f244009e410 0x7f2440120a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f244009d6f0 0x7f244011ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 --2- 172.21.15.32:0/4047632369 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f244009e000 0x7f2440120510 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 >> 172.21.15.32:0/4047632369 conn(0x7f2440066f50 msgr2=0x7f244006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 shutdown_connections 2023-09-30T22:30:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.299+0000 7f24451a7700 1 -- 172.21.15.32:0/4047632369 wait complete. 2023-09-30T22:30:07.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:07.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:07.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.518+0000 7f646c336700 1 Processor -- start 2023-09-30T22:30:07.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.518+0000 7f646c336700 1 -- start start 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646c336700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f646409d630 0x7f64640a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646c336700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6464138d80 0x7f646413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646c336700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64640a2a70 con 0x7f64640a0a30 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646c336700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64640a01d0 con 0x7f6464138d80 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646c336700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f646409e2b0 con 0x7f646409d630 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646a0d2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64640a0a30 0x7f64640a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41794/0 (socket says 172.21.15.32:41794) 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646a0d2700 1 -- 172.21.15.32:0/1288616913 learned_addr learned my addr 172.21.15.32:0/1288616913 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f64698d1700 1 --2- 172.21.15.32:0/1288616913 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f646409d630 0x7f64640a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f646a8d3700 1 --2- 172.21.15.32:0/1288616913 >> v2:172.21.15.32:3300/0 conn(0x7f6464138d80 0x7f646413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f64698d1700 1 -- 172.21.15.32:0/1288616913 >> v2:172.21.15.32:3300/0 conn(0x7f6464138d80 msgr2=0x7f646413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f64698d1700 1 --2- 172.21.15.32:0/1288616913 >> v2:172.21.15.32:3300/0 conn(0x7f6464138d80 0x7f646413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f64698d1700 1 -- 172.21.15.32:0/1288616913 >> v1:172.21.15.32:6789/0 conn(0x7f64640a0a30 legacy=0x7f64640a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f64698d1700 1 -- 172.21.15.32:0/1288616913 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646409dc50 con 0x7f646409d630 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.519+0000 7f64690d0700 1 -- 172.21.15.32:0/1288616913 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910397648 0 0) 0x7f64640a2a70 con 0x7f64640a0a30 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.520+0000 7f64698d1700 1 --2- 172.21.15.32:0/1288616913 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f646409d630 0x7f64640a1510 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6454009840 tx=0x7f6454009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b789ecf2a227450 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.520+0000 7f64690d0700 1 -- 172.21.15.32:0/1288616913 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011830 con 0x7f646409d630 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.520+0000 7f64690d0700 1 -- 172.21.15.32:0/1288616913 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454011990 con 0x7f646409d630 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.520+0000 7f64690d0700 1 -- 172.21.15.32:0/1288616913 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011d20 con 0x7f646409d630 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.520+0000 7f646c336700 1 -- 172.21.15.32:0/1288616913 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f646409d630 msgr2=0x7f64640a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.520+0000 7f646c336700 1 --2- 172.21.15.32:0/1288616913 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f646409d630 0x7f64640a1510 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6454009840 tx=0x7f6454009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.521+0000 7f646c336700 1 -- 172.21.15.32:0/1288616913 shutdown_connections 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.521+0000 7f646c336700 1 --2- 172.21.15.32:0/1288616913 >> v2:172.21.15.32:3300/0 conn(0x7f6464138d80 0x7f646413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.521+0000 7f646c336700 1 --2- 172.21.15.32:0/1288616913 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f646409d630 0x7f64640a1510 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.521+0000 7f646c336700 1 -- 172.21.15.32:0/1288616913 >> 172.21.15.32:0/1288616913 conn(0x7f64640a3050 msgr2=0x7f64640a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.521+0000 7f646c336700 1 -- 172.21.15.32:0/1288616913 shutdown_connections 2023-09-30T22:30:07.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.521+0000 7f646c336700 1 -- 172.21.15.32:0/1288616913 wait complete. 2023-09-30T22:30:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.522+0000 7f646c336700 1 Processor -- start 2023-09-30T22:30:07.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.522+0000 7f646c336700 1 -- start start 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646c336700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646409d630 0x7f64641336a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646c336700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 0x7f6464133be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646c336700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6464138d80 0x7f6464134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646c336700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464137130 con 0x7f6464138d80 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646c336700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6464136020 con 0x7f646409d630 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646c336700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f646413ac80 con 0x7f64640a0a30 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 0x7f6464133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 0x7f6464133be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33838/0 (socket says 172.21.15.32:33838) 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 -- 172.21.15.32:0/332558451 learned_addr learned my addr 172.21.15.32:0/332558451 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646a8d3700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6464138d80 0x7f6464134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646a0d2700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646409d630 0x7f64641336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 -- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646409d630 msgr2=0x7f64641336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646409d630 0x7f64641336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 -- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6464138d80 msgr2=0x7f6464134120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6464138d80 0x7f6464134120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f64698d1700 1 -- 172.21.15.32:0/332558451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540094a0 con 0x7f64640a0a30 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.523+0000 7f646a8d3700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6464138d80 0x7f6464134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.524+0000 7f64698d1700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 0x7f6464133be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6454009fd0 tx=0x7f6454004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.524+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64540036a0 con 0x7f64640a0a30 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.524+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645401f070 con 0x7f64640a0a30 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.524+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640a0410 con 0x7f64640a0a30 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.524+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454003a60 con 0x7f64640a0a30 2023-09-30T22:30:07.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.524+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6464134a70 con 0x7f64640a0a30 2023-09-30T22:30:07.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.525+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6454003bc0 con 0x7f64640a0a30 2023-09-30T22:30:07.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.525+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6454021070 con 0x7f64640a0a30 2023-09-30T22:30:07.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.525+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64640747f0 con 0x7f64640a0a30 2023-09-30T22:30:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.530+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64540253e0 con 0x7f64640a0a30 2023-09-30T22:30:07.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:07.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.631+0000 7fb466277700 1 Processor -- start 2023-09-30T22:30:07.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.631+0000 7fb466277700 1 -- start start 2023-09-30T22:30:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb466277700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb460130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb466277700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb46006f680 0x7fb46006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb466277700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 0x7fb460137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb466277700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46009a350 con 0x7fb46006f680 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb466277700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb460130260 con 0x7fb46006ffd0 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb466277700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb460071ff0 con 0x7fb460130670 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb45f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb46006f680 0x7fb46006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb464814700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 0x7fb460137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb45f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb46006f680 0x7fb46006fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35316/0 (socket says 172.21.15.32:35316) 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb45f7fe700 1 -- 172.21.15.32:0/2707373152 learned_addr learned my addr 172.21.15.32:0/2707373152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb464814700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 0x7fb460137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60818/0 (socket says 172.21.15.32:60818) 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.632+0000 7fb45ffff700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb460130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45f7fe700 1 -- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 msgr2=0x7fb460130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45f7fe700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb460130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45f7fe700 1 -- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 msgr2=0x7fb460137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45f7fe700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 0x7fb460137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45f7fe700 1 -- 172.21.15.32:0/2707373152 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460070470 con 0x7fb46006f680 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45ffff700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb460130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb464814700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 0x7fb460137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45f7fe700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006f680 0x7fb46006fa90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098f0 tx=0x7fb454009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f05dc2a375c27bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45effd700 1 -- 172.21.15.32:0/2707373152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454011830 con 0x7fb46006f680 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45effd700 1 -- 172.21.15.32:0/2707373152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454011990 con 0x7fb46006f680 2023-09-30T22:30:07.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.633+0000 7fb45effd700 1 -- 172.21.15.32:0/2707373152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454011d20 con 0x7fb46006f680 2023-09-30T22:30:07.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 -- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006f680 msgr2=0x7fb46006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006f680 0x7fb46006fa90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098f0 tx=0x7fb454009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 -- 172.21.15.32:0/2707373152 shutdown_connections 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006ffd0 0x7fb460137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb460130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 --2- 172.21.15.32:0/2707373152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006f680 0x7fb46006fa90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 -- 172.21.15.32:0/2707373152 >> 172.21.15.32:0/2707373152 conn(0x7fb4600a2aa0 msgr2=0x7fb4600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 -- 172.21.15.32:0/2707373152 shutdown_connections 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.634+0000 7fb466277700 1 -- 172.21.15.32:0/2707373152 wait complete. 2023-09-30T22:30:07.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.635+0000 7fb466277700 1 Processor -- start 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 -- start start 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006f680 0x7fb4601c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006ffd0 0x7fb4601c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb4601c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46009e480 con 0x7fb46006ffd0 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb460071890 con 0x7fb46006f680 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb466277700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb46012b110 con 0x7fb460130670 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb464814700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb4601c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb464814700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb4601c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33864/0 (socket says 172.21.15.32:33864) 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.636+0000 7fb464814700 1 -- 172.21.15.32:0/2712463939 learned_addr learned my addr 172.21.15.32:0/2712463939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb45f7fe700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006ffd0 0x7fb4601c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb45ffff700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006f680 0x7fb4601c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb464814700 1 -- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006f680 msgr2=0x7fb4601c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb464814700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006f680 0x7fb4601c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb464814700 1 -- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006ffd0 msgr2=0x7fb4601c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb464814700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006ffd0 0x7fb4601c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb464814700 1 -- 172.21.15.32:0/2712463939 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009a70 con 0x7fb460130670 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb464814700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb4601c8bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb458009f00 tx=0x7fb45800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458009d70 con 0x7fb460130670 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800f040 con 0x7fb460130670 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4500094a0 con 0x7fb460130670 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800cd10 con 0x7fb460130670 2023-09-30T22:30:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.637+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb454009550 con 0x7fb460130670 2023-09-30T22:30:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.639+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb458014480 con 0x7fb460130670 2023-09-30T22:30:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.639+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb460129750 con 0x7fb460130670 2023-09-30T22:30:07.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.639+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb45803fce0 con 0x7fb460130670 2023-09-30T22:30:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.644+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb458012930 con 0x7fb460130670 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.677+0000 7fe8a98b8700 1 Processor -- start 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.677+0000 7fe8a98b8700 1 -- start start 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a98b8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 0x7fe8a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a98b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409e000 0x7fe8a41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a98b8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8a409e410 0x7fe8a4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a98b8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a406b460 con 0x7fe8a409e410 2023-09-30T22:30:07.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a98b8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a409b120 con 0x7fe8a409e000 2023-09-30T22:30:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a98b8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a409a7e0 con 0x7fe8a409d6f0 2023-09-30T22:30:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 0x7fe8a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 0x7fe8a409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33872/0 (socket says 172.21.15.32:33872) 2023-09-30T22:30:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2461525415 learned_addr learned my addr 172.21.15.32:0/2461525415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409e000 0x7fe8a41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409e000 msgr2=0x7fe8a41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 --2- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409e000 0x7fe8a41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2461525415 >> v2:172.21.15.32:3300/0 conn(0x7fe8a409e410 msgr2=0x7fe8a4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 --2- 172.21.15.32:0/2461525415 >> v2:172.21.15.32:3300/0 conn(0x7fe8a409e410 0x7fe8a4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a2ffd700 1 -- 172.21.15.32:0/2461525415 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a409a500 con 0x7fe8a409d6f0 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.678+0000 7fe8a27fc700 1 --2- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409e000 0x7fe8a41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a2ffd700 1 --2- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 0x7fe8a409dac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe894009920 tx=0x7fe894009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bccb98f8c48c6a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a1ffb700 1 -- 172.21.15.32:0/2461525415 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011830 con 0x7fe8a409d6f0 2023-09-30T22:30:07.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a1ffb700 1 -- 172.21.15.32:0/2461525415 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894011990 con 0x7fe8a409d6f0 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a1ffb700 1 -- 172.21.15.32:0/2461525415 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011d20 con 0x7fe8a409d6f0 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 -- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 msgr2=0x7fe8a409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 0x7fe8a409dac0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe894009920 tx=0x7fe894009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 -- 172.21.15.32:0/2461525415 shutdown_connections 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/2461525415 >> v2:172.21.15.32:3300/0 conn(0x7fe8a409e410 0x7fe8a4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409d6f0 0x7fe8a409dac0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/2461525415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409e000 0x7fe8a41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.679+0000 7fe8a98b8700 1 -- 172.21.15.32:0/2461525415 >> 172.21.15.32:0/2461525415 conn(0x7fe8a4066f50 msgr2=0x7fe8a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.680+0000 7fe8a98b8700 1 -- 172.21.15.32:0/2461525415 shutdown_connections 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.680+0000 7fe8a98b8700 1 -- 172.21.15.32:0/2461525415 wait complete. 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 Processor -- start 2023-09-30T22:30:07.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 -- start start 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409d6f0 0x7fe8a41a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 0x7fe8a41a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409e410 0x7fe8a41a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a41a7440 con 0x7fe8a409e000 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a41a7a70 con 0x7fe8a409d6f0 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a98b8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a41a8cc0 con 0x7fe8a409e410 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 0x7fe8a41a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409d6f0 0x7fe8a41a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 0x7fe8a41a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35346/0 (socket says 172.21.15.32:35346) 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.681+0000 7fe8a27fc700 1 -- 172.21.15.32:0/3728642911 learned_addr learned my addr 172.21.15.32:0/3728642911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:07.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a37fe700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409e410 0x7fe8a41a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a27fc700 1 -- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409e410 msgr2=0x7fe8a41a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a27fc700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409e410 0x7fe8a41a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a27fc700 1 -- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409d6f0 msgr2=0x7fe8a41a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a27fc700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409d6f0 0x7fe8a41a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a27fc700 1 -- 172.21.15.32:0/3728642911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980094a0 con 0x7fe8a409e000 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a2ffd700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409d6f0 0x7fe8a41a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a27fc700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 0x7fe8a41a64d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fe89800ebe0 tx=0x7fe89800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89800c850 con 0x7fe8a409e000 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89800c9b0 con 0x7fe8a409e000 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe894009550 con 0x7fe8a409e000 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89800cd40 con 0x7fe8a409e000 2023-09-30T22:30:07.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.682+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a40a3180 con 0x7fe8a409e000 2023-09-30T22:30:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.683+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a409c710 con 0x7fe8a409e000 2023-09-30T22:30:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.683+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe898013470 con 0x7fe8a409e000 2023-09-30T22:30:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.684+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe89803f7c0 con 0x7fe8a409e000 2023-09-30T22:30:07.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.688+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe898013720 con 0x7fe8a409e000 2023-09-30T22:30:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.740+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6464072980 con 0x7f64640a0a30 2023-09-30T22:30:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.740+0000 7f645affd700 1 -- 172.21.15.32:0/332558451 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6454010b70 con 0x7f64640a0a30 2023-09-30T22:30:07.741 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.742+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 msgr2=0x7f6464133be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.742+0000 7f646c336700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 0x7f6464133be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6454009fd0 tx=0x7f6454004750 comp rx=0 tx=0).stop 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.742+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 shutdown_connections 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.742+0000 7f646c336700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6464138d80 0x7f6464134120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.742+0000 7f646c336700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f646409d630 0x7f64641336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.742+0000 7f646c336700 1 --2- 172.21.15.32:0/332558451 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f64640a0a30 0x7f6464133be0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.743+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 >> 172.21.15.32:0/332558451 conn(0x7f64640a3050 msgr2=0x7f64640a69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.743+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 shutdown_connections 2023-09-30T22:30:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.743+0000 7f646c336700 1 -- 172.21.15.32:0/332558451 wait complete. 2023-09-30T22:30:07.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.852+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb460074ea0 con 0x7fb460130670 2023-09-30T22:30:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.853+0000 7fb45cff9700 1 -- 172.21.15.32:0/2712463939 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb45801c3b0 con 0x7fb460130670 2023-09-30T22:30:07.854 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 msgr2=0x7fb4601c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb4601c8bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb458009f00 tx=0x7fb45800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 shutdown_connections 2023-09-30T22:30:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb460130670 0x7fb4601c8bf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46006f680 0x7fb4601c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 --2- 172.21.15.32:0/2712463939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb46006ffd0 0x7fb4601c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 >> 172.21.15.32:0/2712463939 conn(0x7fb4600a2aa0 msgr2=0x7fb46006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 shutdown_connections 2023-09-30T22:30:07.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.855+0000 7fb466277700 1 -- 172.21.15.32:0/2712463939 wait complete. 2023-09-30T22:30:07.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:07.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.895+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe8a41a8050 con 0x7fe8a409e000 2023-09-30T22:30:07.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.895+0000 7fe88bfff700 1 -- 172.21.15.32:0/3728642911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8a41a8050 con 0x7fe8a409e000 2023-09-30T22:30:07.897 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 msgr2=0x7fe8a41a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 0x7fe8a41a64d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fe89800ebe0 tx=0x7fe89800eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 shutdown_connections 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8a409e410 0x7fe8a41a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8a409d6f0 0x7fe8a41a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 --2- 172.21.15.32:0/3728642911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8a409e000 0x7fe8a41a64d0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 >> 172.21.15.32:0/3728642911 conn(0x7fe8a4066f50 msgr2=0x7fe8a41bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 shutdown_connections 2023-09-30T22:30:07.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:07.898+0000 7fe8a98b8700 1 -- 172.21.15.32:0/3728642911 wait complete. 2023-09-30T22:30:07.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:08.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:08.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.614+0000 7f639451c700 1 Processor -- start 2023-09-30T22:30:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.614+0000 7f639451c700 1 -- start start 2023-09-30T22:30:08.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f639451c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 0x7f638c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f639451c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 0x7f638c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f639451c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 0x7f638c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f639451c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c070940 con 0x7f638c09f650 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f639451c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c06fd60 con 0x7f638c09f240 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f639451c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f638c06fa80 con 0x7f638c09e930 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6392ab9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 0x7f638c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 0x7f638c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f63922b8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 0x7f638c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 0x7f638c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60850/0 (socket says 172.21.15.32:60850) 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f63922b8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 0x7f638c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33890/0 (socket says 172.21.15.32:33890) 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 -- 172.21.15.32:0/2523894305 learned_addr learned my addr 172.21.15.32:0/2523894305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6392ab9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 0x7f638c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35358/0 (socket says 172.21.15.32:35358) 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 -- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 msgr2=0x7f638c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 --2- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 0x7f638c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 -- 172.21.15.32:0/2523894305 >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 msgr2=0x7f638c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 --2- 172.21.15.32:0/2523894305 >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 0x7f638c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 -- 172.21.15.32:0/2523894305 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c09fb60 con 0x7f638c09f240 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6392ab9700 1 --2- 172.21.15.32:0/2523894305 >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 0x7f638c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f63922b8700 1 --2- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 0x7f638c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.615+0000 7f6391ab7700 1 --2- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 0x7f638c1317f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f637c0098f0 tx=0x7f637c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84d988d1b819db5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f63912b6700 1 -- 172.21.15.32:0/2523894305 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c011830 con 0x7f638c09f240 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f63912b6700 1 -- 172.21.15.32:0/2523894305 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c011990 con 0x7f638c09f240 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f63912b6700 1 -- 172.21.15.32:0/2523894305 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c011d20 con 0x7f638c09f240 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 -- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 msgr2=0x7f638c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 --2- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 0x7f638c1317f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f637c0098f0 tx=0x7f637c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 -- 172.21.15.32:0/2523894305 shutdown_connections 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 --2- 172.21.15.32:0/2523894305 >> v2:172.21.15.32:3300/0 conn(0x7f638c09f650 0x7f638c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 --2- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09e930 0x7f638c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 --2- 172.21.15.32:0/2523894305 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f240 0x7f638c1317f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 -- 172.21.15.32:0/2523894305 >> 172.21.15.32:0/2523894305 conn(0x7f638c0a4210 msgr2=0x7f638c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 -- 172.21.15.32:0/2523894305 shutdown_connections 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.616+0000 7f639451c700 1 -- 172.21.15.32:0/2523894305 wait complete. 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.617+0000 7f639451c700 1 Processor -- start 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.617+0000 7f639451c700 1 -- start start 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f639451c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e930 0x7f638c1a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f639451c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 0x7f638c1a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f639451c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f650 0x7f638c1a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f639451c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1a5740 con 0x7f638c09e930 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f639451c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1a6280 con 0x7f638c09f650 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f639451c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0a3580 con 0x7f638c09f240 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 0x7f638c1a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f63922b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e930 0x7f638c1a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6392ab9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f650 0x7f638c1a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 0x7f638c1a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33892/0 (socket says 172.21.15.32:33892) 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f63922b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e930 0x7f638c1a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35364/0 (socket says 172.21.15.32:35364) 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 -- 172.21.15.32:0/438249700 learned_addr learned my addr 172.21.15.32:0/438249700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 -- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f650 msgr2=0x7f638c1a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f650 0x7f638c1a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 -- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e930 msgr2=0x7f638c1a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e930 0x7f638c1a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.618+0000 7f6391ab7700 1 -- 172.21.15.32:0/438249700 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880094a0 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f6391ab7700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 0x7f638c1a3350 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f637c009c40 tx=0x7f637c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c004240 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c01f070 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6384009a70 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c0045e0 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c009550 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.619+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f637c021030 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.620+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f637c040850 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.620+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c071d80 con 0x7f638c09f240 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.625+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f637c015030 con 0x7f638c09f240 2023-09-30T22:30:08.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.833+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f638c074ea0 con 0x7f638c09f240 2023-09-30T22:30:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.833+0000 7f6382ffd700 1 -- 172.21.15.32:0/438249700 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f637c01a450 con 0x7f638c09f240 2023-09-30T22:30:08.834 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.835+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 msgr2=0x7f638c1a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.835+0000 7f639451c700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 0x7f638c1a3350 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f637c009c40 tx=0x7f637c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.835+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 shutdown_connections 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.835+0000 7f639451c700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f638c09f650 0x7f638c1a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.835+0000 7f639451c700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f638c09e930 0x7f638c1a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.835+0000 7f639451c700 1 --2- 172.21.15.32:0/438249700 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f638c09f240 0x7f638c1a3350 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.836+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 >> 172.21.15.32:0/438249700 conn(0x7f638c0a4210 msgr2=0x7f638c1b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.836+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 shutdown_connections 2023-09-30T22:30:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.836+0000 7f639451c700 1 -- 172.21.15.32:0/438249700 wait complete. 2023-09-30T22:30:08.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:08.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.900+0000 7fd05869c700 1 Processor -- start 2023-09-30T22:30:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.900+0000 7fd05869c700 1 -- start start 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd05869c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0500a0a30 0x7fd0500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd05869c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 0x7fd050133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd05869c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 0x7fd050134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd05869c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05009edd0 con 0x7fd0500a0a30 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd05869c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05009e1f0 con 0x7fd050133fa0 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd05869c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd05009dc50 con 0x7fd0500a1340 2023-09-30T22:30:08.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd055c37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 0x7fd050133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0500a0a30 0x7fd0500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056c39700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 0x7fd050134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd055c37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 0x7fd050133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33906/0 (socket says 172.21.15.32:33906) 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056c39700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 0x7fd050134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60874/0 (socket says 172.21.15.32:60874) 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd0500a0a30 0x7fd0500a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35374/0 (socket says 172.21.15.32:35374) 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056c39700 1 -- 172.21.15.32:0/510930564 learned_addr learned my addr 172.21.15.32:0/510930564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 -- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 msgr2=0x7fd050133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 0x7fd050133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 -- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 msgr2=0x7fd050134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 0x7fd050134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056438700 1 -- 172.21.15.32:0/510930564 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0500a1fe0 con 0x7fd0500a0a30 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd055c37700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 0x7fd050133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.901+0000 7fd056c39700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 0x7fd050134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.902+0000 7fd056438700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500a0a30 0x7fd0500a0e00 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009840 tx=0x7fd04c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8797d2e762d1c40c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.902+0000 7fd055436700 1 -- 172.21.15.32:0/510930564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011830 con 0x7fd0500a0a30 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.902+0000 7fd055436700 1 -- 172.21.15.32:0/510930564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c011990 con 0x7fd0500a0a30 2023-09-30T22:30:08.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.902+0000 7fd055436700 1 -- 172.21.15.32:0/510930564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011d20 con 0x7fd0500a0a30 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.902+0000 7fd05869c700 1 -- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500a0a30 msgr2=0x7fd0500a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.902+0000 7fd05869c700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500a0a30 0x7fd0500a0e00 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009840 tx=0x7fd04c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 -- 172.21.15.32:0/510930564 shutdown_connections 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd050133fa0 0x7fd050134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0500a0a30 0x7fd0500a0e00 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 --2- 172.21.15.32:0/510930564 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a1340 0x7fd050133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 -- 172.21.15.32:0/510930564 >> 172.21.15.32:0/510930564 conn(0x7fd0500a6550 msgr2=0x7fd0500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 -- 172.21.15.32:0/510930564 shutdown_connections 2023-09-30T22:30:08.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.903+0000 7fd05869c700 1 -- 172.21.15.32:0/510930564 wait complete. 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 Processor -- start 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 -- start start 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 0x7fd0501212d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 0x7fd050121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050133fa0 0x7fd050121d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0501326a0 con 0x7fd050133fa0 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd050123d70 con 0x7fd0500a1340 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd05869c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd050124850 con 0x7fd0500a0a30 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd055c37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 0x7fd050121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd056438700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 0x7fd0501212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd055c37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 0x7fd050121810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60886/0 (socket says 172.21.15.32:60886) 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd056438700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 0x7fd0501212d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33914/0 (socket says 172.21.15.32:33914) 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd055c37700 1 -- 172.21.15.32:0/207082317 learned_addr learned my addr 172.21.15.32:0/207082317 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.904+0000 7fd056c39700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050133fa0 0x7fd050121d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd055c37700 1 -- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 msgr2=0x7fd0501212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd055c37700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 0x7fd0501212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd055c37700 1 -- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050133fa0 msgr2=0x7fd050121d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd055c37700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050133fa0 0x7fd050121d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd055c37700 1 -- 172.21.15.32:0/207082317 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400094a0 con 0x7fd0500a1340 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd056438700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 0x7fd0501212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd056c39700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050133fa0 0x7fd050121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd055c37700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 0x7fd050121810 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd04000ec10 tx=0x7fd04000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04000c850 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04000c9b0 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd048009b30 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04000cd20 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.905+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c0094a0 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.906+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd040013470 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.906+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd04003fc70 con 0x7fd0500a1340 2023-09-30T22:30:08.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.906+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd050124990 con 0x7fd0500a1340 2023-09-30T22:30:08.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:08.911+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd040021020 con 0x7fd0500a1340 2023-09-30T22:30:08.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:09.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.012+0000 7f3ffe688700 1 Processor -- start 2023-09-30T22:30:09.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.012+0000 7f3ffe688700 1 -- start start 2023-09-30T22:30:09.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffe688700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ff809e930 0x7f3ff809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffe688700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 0x7f3ff81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffe688700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffe688700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff8070940 con 0x7f3ff809e930 2023-09-30T22:30:09.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffe688700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff806fd60 con 0x7f3ff809f240 2023-09-30T22:30:09.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffe688700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff806fa80 con 0x7f3ff809f650 2023-09-30T22:30:09.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 0x7f3ff81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 0x7f3ff81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60900/0 (socket says 172.21.15.32:60900) 2023-09-30T22:30:09.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 -- 172.21.15.32:0/3092803989 learned_addr learned my addr 172.21.15.32:0/3092803989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:09.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffcc25700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff7fff700 1 --2- 172.21.15.32:0/3092803989 >> v2:172.21.15.32:3300/0 conn(0x7f3ff809e930 0x7f3ff809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 -- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 msgr2=0x7f3ff8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff8131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 -- 172.21.15.32:0/3092803989 >> v2:172.21.15.32:3300/0 conn(0x7f3ff809e930 msgr2=0x7f3ff809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 --2- 172.21.15.32:0/3092803989 >> v2:172.21.15.32:3300/0 conn(0x7f3ff809e930 0x7f3ff809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff77fe700 1 -- 172.21.15.32:0/3092803989 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff809fb60 con 0x7f3ff809f240 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ffcc25700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:09.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.013+0000 7f3ff7fff700 1 --2- 172.21.15.32:0/3092803989 >> v2:172.21.15.32:3300/0 conn(0x7f3ff809e930 0x7f3ff809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ff77fe700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 0x7f3ff81317f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098f0 tx=0x7f3fec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82cab1280740ff44 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ff6ffd700 1 -- 172.21.15.32:0/3092803989 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec011830 con 0x7f3ff809f240 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ff6ffd700 1 -- 172.21.15.32:0/3092803989 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec011990 con 0x7f3ff809f240 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ff6ffd700 1 -- 172.21.15.32:0/3092803989 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec011d20 con 0x7f3ff809f240 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ffe688700 1 -- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 msgr2=0x7f3ff81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ffe688700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 0x7f3ff81317f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098f0 tx=0x7f3fec009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ffe688700 1 -- 172.21.15.32:0/3092803989 shutdown_connections 2023-09-30T22:30:09.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.014+0000 7f3ffe688700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff8131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.015+0000 7f3ffe688700 1 --2- 172.21.15.32:0/3092803989 >> v2:172.21.15.32:3300/0 conn(0x7f3ff809e930 0x7f3ff809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.015+0000 7f3ffe688700 1 --2- 172.21.15.32:0/3092803989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809f240 0x7f3ff81317f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.015+0000 7f3ffe688700 1 -- 172.21.15.32:0/3092803989 >> 172.21.15.32:0/3092803989 conn(0x7f3ff80a4210 msgr2=0x7f3ff80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.015+0000 7f3ffe688700 1 -- 172.21.15.32:0/3092803989 shutdown_connections 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.015+0000 7f3ffe688700 1 -- 172.21.15.32:0/3092803989 wait complete. 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 Processor -- start 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 -- start start 2023-09-30T22:30:09.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809e930 0x7f3ff806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 0x7f3ff806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff81a86e0 con 0x7f3ff809f240 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff806cd10 con 0x7f3ff809e930 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffe688700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff81a9b10 con 0x7f3ff809f650 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 0x7f3ff806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ffcc25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 0x7f3ff806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35394/0 (socket says 172.21.15.32:35394) 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.016+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809e930 0x7f3ff806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ff77fe700 1 -- 172.21.15.32:0/2326193722 learned_addr learned my addr 172.21.15.32:0/2326193722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffcc25700 1 -- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809e930 msgr2=0x7f3ff806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffcc25700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809e930 0x7f3ff806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffcc25700 1 -- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 msgr2=0x7f3ff806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffcc25700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 0x7f3ff806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffcc25700 1 -- 172.21.15.32:0/2326193722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80094a0 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ff7fff700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809e930 0x7f3ff806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ff77fe700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 0x7f3ff806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffcc25700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff806c330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009e50 tx=0x7f3ff000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000ca90 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000f040 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff0009a70 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000ce50 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.017+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec009550 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.018+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ff0014480 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.018+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3ff003fc60 con 0x7f3ff809f650 2023-09-30T22:30:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.019+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff8074ea0 con 0x7f3ff809f650 2023-09-30T22:30:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.024+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ff0012930 con 0x7f3ff809f650 2023-09-30T22:30:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 Processor -- start 2023-09-30T22:30:09.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 -- start start 2023-09-30T22:30:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67100984f0 0x7f67100988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 0x7f671009db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710099620 con 0x7f671009e040 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671009a9e0 con 0x7f67100984f0 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.052+0000 7f67159d7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f671009bce0 con 0x7f671009d6f0 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 0x7f671009db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 0x7f671009db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33944/0 (socket says 172.21.15.32:33944) 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 -- 172.21.15.32:0/2602317957 learned_addr learned my addr 172.21.15.32:0/2602317957 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670effd700 1 --2- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67100984f0 0x7f67100988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 -- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67100984f0 msgr2=0x7f67100988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 --2- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67100984f0 0x7f67100988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 -- 172.21.15.32:0/2602317957 >> v1:172.21.15.32:6789/0 conn(0x7f671009e040 legacy=0x7f67101385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 -- 172.21.15.32:0/2602317957 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671009aca0 con 0x7f671009d6f0 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670e7fc700 1 --2- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 0x7f671009db00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6704009920 tx=0x7f6704009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862fbf1a3899e0b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670dffb700 1 -- 172.21.15.32:0/2602317957 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704011830 con 0x7f671009d6f0 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670dffb700 1 -- 172.21.15.32:0/2602317957 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704011990 con 0x7f671009d6f0 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.053+0000 7f670dffb700 1 -- 172.21.15.32:0/2602317957 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704011d20 con 0x7f671009d6f0 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 -- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 msgr2=0x7f671009db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 --2- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 0x7f671009db00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6704009920 tx=0x7f6704009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 -- 172.21.15.32:0/2602317957 shutdown_connections 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 --2- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67100984f0 0x7f67100988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 --2- 172.21.15.32:0/2602317957 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f671009d6f0 0x7f671009db00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 -- 172.21.15.32:0/2602317957 >> 172.21.15.32:0/2602317957 conn(0x7f671009f470 msgr2=0x7f67100a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 -- 172.21.15.32:0/2602317957 shutdown_connections 2023-09-30T22:30:09.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.054+0000 7f67159d7700 1 -- 172.21.15.32:0/2602317957 wait complete. 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.055+0000 7f67159d7700 1 Processor -- start 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.055+0000 7f67159d7700 1 -- start start 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f67159d7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67100984f0 0x7f671012e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f67159d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 0x7f671012e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f67159d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f671009e040 0x7f671006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f67159d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671006f360 con 0x7f671009d6f0 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f67159d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671012bf30 con 0x7f671009e040 2023-09-30T22:30:09.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f67159d7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f671012b3e0 con 0x7f67100984f0 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f670e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 0x7f671012e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f670e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 0x7f671012e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35406/0 (socket says 172.21.15.32:35406) 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f670e7fc700 1 -- 172.21.15.32:0/1617990851 learned_addr learned my addr 172.21.15.32:0/1617990851 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.056+0000 7f670effd700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67100984f0 0x7f671012e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670f7fe700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f671009e040 0x7f671006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670e7fc700 1 -- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67100984f0 msgr2=0x7f671012e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670e7fc700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67100984f0 0x7f671012e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670e7fc700 1 -- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f671009e040 msgr2=0x7f671006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670e7fc700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f671009e040 0x7f671006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670e7fc700 1 -- 172.21.15.32:0/1617990851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009550 con 0x7f671009d6f0 2023-09-30T22:30:09.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f670e7fc700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 0x7f671012e830 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f6704000c00 tx=0x7f6704004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704004450 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040045b0 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6710097e50 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040038a0 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.057+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671012cc60 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.058+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6704003a00 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.058+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6704021030 con 0x7f671009d6f0 2023-09-30T22:30:09.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.059+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67100747f0 con 0x7f671009d6f0 2023-09-30T22:30:09.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.064+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6704015080 con 0x7f671009d6f0 2023-09-30T22:30:09.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.124+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd0500a0220 con 0x7fd0500a1340 2023-09-30T22:30:09.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.124+0000 7fd046ffd700 1 -- 172.21.15.32:0/207082317 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd04002b090 con 0x7fd0500a1340 2023-09-30T22:30:09.126 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 msgr2=0x7fd050121810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 0x7fd050121810 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd04000ec10 tx=0x7fd04000ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 shutdown_connections 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd050133fa0 0x7fd050121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0500a0a30 0x7fd0501212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 --2- 172.21.15.32:0/207082317 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0500a1340 0x7fd050121810 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 >> 172.21.15.32:0/207082317 conn(0x7fd0500a6550 msgr2=0x7fd0500a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 shutdown_connections 2023-09-30T22:30:09.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.127+0000 7fd05869c700 1 -- 172.21.15.32:0/207082317 wait complete. 2023-09-30T22:30:09.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:09.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.235+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ff81a8e50 con 0x7f3ff809f650 2023-09-30T22:30:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.236+0000 7f3ff4ff9700 1 -- 172.21.15.32:0/2326193722 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ff0012340 con 0x7f3ff809f650 2023-09-30T22:30:09.237 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 msgr2=0x7f3ff806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff806c330 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009e50 tx=0x7f3ff000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 shutdown_connections 2023-09-30T22:30:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ff809f650 0x7f3ff806c330 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ff809e930 0x7f3ff806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 --2- 172.21.15.32:0/2326193722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ff809f240 0x7f3ff806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 >> 172.21.15.32:0/2326193722 conn(0x7f3ff80a4210 msgr2=0x7f3ff81b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 shutdown_connections 2023-09-30T22:30:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.238+0000 7f3ffe688700 1 -- 172.21.15.32:0/2326193722 wait complete. 2023-09-30T22:30:09.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:09.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.270+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f671012c560 con 0x7f671009d6f0 2023-09-30T22:30:09.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.271+0000 7f66f7fff700 1 -- 172.21.15.32:0/1617990851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f670402b020 con 0x7f671009d6f0 2023-09-30T22:30:09.272 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 msgr2=0x7f671012e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 0x7f671012e830 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f6704000c00 tx=0x7f6704004750 comp rx=0 tx=0).stop 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 shutdown_connections 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f671009e040 0x7f671006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67100984f0 0x7f671012e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 --2- 172.21.15.32:0/1617990851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f671009d6f0 0x7f671012e830 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 >> 172.21.15.32:0/1617990851 conn(0x7f671009f470 msgr2=0x7f67101311a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 shutdown_connections 2023-09-30T22:30:09.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.273+0000 7f67159d7700 1 -- 172.21.15.32:0/1617990851 wait complete. 2023-09-30T22:30:09.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:09.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.998+0000 7fd790f12700 1 Processor -- start 2023-09-30T22:30:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.998+0000 7fd790f12700 1 -- start start 2023-09-30T22:30:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd790f12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c09e1d0 0x7fd78c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd790f12700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd78c139170 0x7fd78c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd790f12700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c09b120 con 0x7fd78c09d6f0 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd790f12700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c06b460 con 0x7fd78c139170 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd790f12700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c09a7e0 con 0x7fd78c09e1d0 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd789d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c09e1d0 0x7fd78c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd78c139170 0x7fd78c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd78c09d6f0 0x7fd78c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41814/0 (socket says 172.21.15.32:41814) 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78a59c700 1 -- 172.21.15.32:0/187921504 learned_addr learned my addr 172.21.15.32:0/187921504 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78ad9d700 1 -- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c09e1d0 msgr2=0x7fd78c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78ad9d700 1 --2- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c09e1d0 0x7fd78c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78ad9d700 1 -- 172.21.15.32:0/187921504 >> v1:172.21.15.32:6789/0 conn(0x7fd78c09d6f0 legacy=0x7fd78c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78ad9d700 1 -- 172.21.15.32:0/187921504 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c09a500 con 0x7fd78c139170 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78959a700 1 -- 172.21.15.32:0/187921504 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428745322 0 0) 0x7fd78c09b120 con 0x7fd78c09d6f0 2023-09-30T22:30:10.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd789d9b700 1 --2- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c09e1d0 0x7fd78c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:09.999+0000 7fd78ad9d700 1 --2- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c139170 0x7fd78c13b550 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd784009d90 tx=0x7fd78400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3970c3d24ece1091 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd78959a700 1 -- 172.21.15.32:0/187921504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400e070 con 0x7fd78c139170 2023-09-30T22:30:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd78959a700 1 -- 172.21.15.32:0/187921504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400bb20 con 0x7fd78c139170 2023-09-30T22:30:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd78959a700 1 -- 172.21.15.32:0/187921504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784012430 con 0x7fd78c139170 2023-09-30T22:30:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 -- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c139170 msgr2=0x7fd78c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 --2- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c139170 0x7fd78c13b550 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd784009d90 tx=0x7fd78400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 -- 172.21.15.32:0/187921504 shutdown_connections 2023-09-30T22:30:10.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 --2- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c139170 0x7fd78c13b550 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 --2- 172.21.15.32:0/187921504 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c09e1d0 0x7fd78c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 -- 172.21.15.32:0/187921504 >> 172.21.15.32:0/187921504 conn(0x7fd78c066f50 msgr2=0x7fd78c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 -- 172.21.15.32:0/187921504 shutdown_connections 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.000+0000 7fd790f12700 1 -- 172.21.15.32:0/187921504 wait complete. 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.001+0000 7fd790f12700 1 Processor -- start 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 -- start start 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd78c09d6f0 0x7fd78c11e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 0x7fd78c11edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c139170 0x7fd78c11f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0720b0 con 0x7fd78c09e1d0 2023-09-30T22:30:10.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1205d0 con 0x7fd78c09d6f0 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd790f12700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c121e30 con 0x7fd78c139170 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 0x7fd78c11edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd78ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c139170 0x7fd78c11f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd78a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd78c09d6f0 0x7fd78c11e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 0x7fd78c11edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35422/0 (socket says 172.21.15.32:35422) 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 -- 172.21.15.32:0/1210499728 learned_addr learned my addr 172.21.15.32:0/1210499728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 -- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c139170 msgr2=0x7fd78c11f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c139170 0x7fd78c11f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 -- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd78c09d6f0 msgr2=0x7fd78c11e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd78c09d6f0 0x7fd78c11e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.002+0000 7fd789d9b700 1 -- 172.21.15.32:0/1210499728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740094a0 con 0x7fd78c09e1d0 2023-09-30T22:30:10.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.003+0000 7fd789d9b700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 0x7fd78c11edf0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd77400ebe0 tx=0x7fd77400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.003+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400c850 con 0x7fd78c09e1d0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.003+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77400c9b0 con 0x7fd78c09e1d0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.003+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd784009a70 con 0x7fd78c09e1d0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.003+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400cd40 con 0x7fd78c09e1d0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.003+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c1220f0 con 0x7fd78c09e1d0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.004+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd774013470 con 0x7fd78c09e1d0 2023-09-30T22:30:10.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.004+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd77403fb10 con 0x7fd78c09e1d0 2023-09-30T22:30:10.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.004+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c074ea0 con 0x7fd78c09e1d0 2023-09-30T22:30:10.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.009+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd774013720 con 0x7fd78c09e1d0 2023-09-30T22:30:10.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.223+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd78c071e90 con 0x7fd78c09e1d0 2023-09-30T22:30:10.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.223+0000 7fd782ffd700 1 -- 172.21.15.32:0/1210499728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd77402a090 con 0x7fd78c09e1d0 2023-09-30T22:30:10.224 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 msgr2=0x7fd78c11edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 0x7fd78c11edf0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd77400ebe0 tx=0x7fd77400eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 shutdown_connections 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd78c139170 0x7fd78c11f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd78c09d6f0 0x7fd78c11e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 --2- 172.21.15.32:0/1210499728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c09e1d0 0x7fd78c11edf0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 >> 172.21.15.32:0/1210499728 conn(0x7fd78c066f50 msgr2=0x7fd78c136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 shutdown_connections 2023-09-30T22:30:10.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.226+0000 7fd790f12700 1 -- 172.21.15.32:0/1210499728 wait complete. 2023-09-30T22:30:10.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:10.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.283+0000 7f423e7a4700 1 Processor -- start 2023-09-30T22:30:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.283+0000 7f423e7a4700 1 -- start start 2023-09-30T22:30:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423e7a4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f423809d6f0 0x7f423809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423e7a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f4238134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423e7a4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423809b120 con 0x7f423809e000 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423e7a4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423806b460 con 0x7f423809d6f0 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423e7a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f423809a7e0 con 0x7f423809e440 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423cd41700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f4238134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4237fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f423809d6f0 0x7f423809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423cd41700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f4238134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33982/0 (socket says 172.21.15.32:33982) 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423cd41700 1 -- 172.21.15.32:0/2581604200 learned_addr learned my addr 172.21.15.32:0/2581604200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4237fff700 1 -- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 msgr2=0x7f4238134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4237fff700 1 --2- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f4238134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4237fff700 1 -- 172.21.15.32:0/2581604200 >> v1:172.21.15.32:6789/0 conn(0x7f423809e000 legacy=0x7f42381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4237fff700 1 -- 172.21.15.32:0/2581604200 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423809a500 con 0x7f423809d6f0 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f423cd41700 1 --2- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f4238134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4237fff700 1 --2- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f423809dac0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f4228009870 tx=0x7f4228009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca4ba3012e95ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.284+0000 7f4236ffd700 1 -- 172.21.15.32:0/2581604200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228011830 con 0x7f423809d6f0 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f4236ffd700 1 -- 172.21.15.32:0/2581604200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4228011990 con 0x7f423809d6f0 2023-09-30T22:30:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f4236ffd700 1 -- 172.21.15.32:0/2581604200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228011d20 con 0x7f423809d6f0 2023-09-30T22:30:10.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 -- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 msgr2=0x7f423809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 --2- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f423809dac0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f4228009870 tx=0x7f4228009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 -- 172.21.15.32:0/2581604200 shutdown_connections 2023-09-30T22:30:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 --2- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f4238134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 --2- 172.21.15.32:0/2581604200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f423809dac0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 -- 172.21.15.32:0/2581604200 >> 172.21.15.32:0/2581604200 conn(0x7f4238066f50 msgr2=0x7f423809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 -- 172.21.15.32:0/2581604200 shutdown_connections 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.285+0000 7f423e7a4700 1 -- 172.21.15.32:0/2581604200 wait complete. 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.286+0000 7f423e7a4700 1 Processor -- start 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 -- start start 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f42381a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f423809e000 0x7f42381a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f42381a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42381a7a60 con 0x7f423809d6f0 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423809c710 con 0x7f423809e000 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423e7a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42381a9690 con 0x7f423809e440 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f4237fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f42381a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f423cd41700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f42381a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f4237fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f42381a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35428/0 (socket says 172.21.15.32:35428) 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.287+0000 7f4237fff700 1 -- 172.21.15.32:0/3886602566 learned_addr learned my addr 172.21.15.32:0/3886602566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f42377fe700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f423809e000 0x7f42381a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4237fff700 1 -- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 msgr2=0x7f42381a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4237fff700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f42381a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4237fff700 1 -- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f423809e000 msgr2=0x7f42381a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4237fff700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f423809e000 0x7f42381a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4237fff700 1 -- 172.21.15.32:0/3886602566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4230009b30 con 0x7f423809d6f0 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f42377fe700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f423809e000 0x7f42381a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4237fff700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f42381a5e90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f4228000c00 tx=0x7f4228004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42280036a0 con 0x7f423809d6f0 2023-09-30T22:30:10.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422801f070 con 0x7f423809d6f0 2023-09-30T22:30:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280094a0 con 0x7f423809d6f0 2023-09-30T22:30:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228003a40 con 0x7f423809d6f0 2023-09-30T22:30:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.288+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42381a8cb0 con 0x7f423809d6f0 2023-09-30T22:30:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.289+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4228003ba0 con 0x7f423809d6f0 2023-09-30T22:30:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.289+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4228021070 con 0x7f423809d6f0 2023-09-30T22:30:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.289+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4238074ea0 con 0x7f423809d6f0 2023-09-30T22:30:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.294+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f422802b020 con 0x7f423809d6f0 2023-09-30T22:30:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.396+0000 7f8280b3a700 1 Processor -- start 2023-09-30T22:30:10.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.396+0000 7f8280b3a700 1 -- start start 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.396+0000 7f8280b3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.396+0000 7f8280b3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f8280b3a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f827c133930 0x7f827c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f8280b3a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0a2a70 con 0x7f827c133930 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f8280b3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0a01d0 con 0x7f827c0a0a30 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f8280b3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f827c09e2b0 con 0x7f827c09d630 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:60958/0 (socket says 172.21.15.32:60958) 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 -- 172.21.15.32:0/3258478704 learned_addr learned my addr 172.21.15.32:0/3258478704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f8279d9b700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827ad9d700 1 --2- 172.21.15.32:0/3258478704 >> v2:172.21.15.32:3300/0 conn(0x7f827c133930 0x7f827c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 -- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 msgr2=0x7f827c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 -- 172.21.15.32:0/3258478704 >> v2:172.21.15.32:3300/0 conn(0x7f827c133930 msgr2=0x7f827c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 --2- 172.21.15.32:0/3258478704 >> v2:172.21.15.32:3300/0 conn(0x7f827c133930 0x7f827c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 -- 172.21.15.32:0/3258478704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c09dc50 con 0x7f827c0a0a30 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827ad9d700 1 --2- 172.21.15.32:0/3258478704 >> v2:172.21.15.32:3300/0 conn(0x7f827c133930 0x7f827c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f8279d9b700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.397+0000 7f827a59c700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c0a0e00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098f0 tx=0x7f826c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2784f9e2787a1ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f827959a700 1 -- 172.21.15.32:0/3258478704 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c011830 con 0x7f827c0a0a30 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f827959a700 1 -- 172.21.15.32:0/3258478704 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c011990 con 0x7f827c0a0a30 2023-09-30T22:30:10.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f827959a700 1 -- 172.21.15.32:0/3258478704 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c011d20 con 0x7f827c0a0a30 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 -- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 msgr2=0x7f827c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c0a0e00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f826c0098f0 tx=0x7f826c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 -- 172.21.15.32:0/3258478704 shutdown_connections 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c0a0e00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 --2- 172.21.15.32:0/3258478704 >> v2:172.21.15.32:3300/0 conn(0x7f827c133930 0x7f827c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 --2- 172.21.15.32:0/3258478704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 -- 172.21.15.32:0/3258478704 >> 172.21.15.32:0/3258478704 conn(0x7f827c0a3050 msgr2=0x7f827c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.398+0000 7f8280b3a700 1 -- 172.21.15.32:0/3258478704 shutdown_connections 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.399+0000 7f8280b3a700 1 -- 172.21.15.32:0/3258478704 wait complete. 2023-09-30T22:30:10.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 Processor -- start 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 -- start start 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c132630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c133930 0x7f827c11e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c120440 con 0x7f827c133930 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c122430 con 0x7f827c0a0a30 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8280b3a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0a4150 con 0x7f827c09d630 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f827a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c132630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f827a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c132630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:33988/0 (socket says 172.21.15.32:33988) 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f8279d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f827a59c700 1 -- 172.21.15.32:0/1433433479 learned_addr learned my addr 172.21.15.32:0/1433433479 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.400+0000 7f827ad9d700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c133930 0x7f827c11e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827a59c700 1 -- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 msgr2=0x7f827c132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827a59c700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c132b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827a59c700 1 -- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c133930 msgr2=0x7f827c11e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827a59c700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c133930 0x7f827c11e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827a59c700 1 -- 172.21.15.32:0/1433433479 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82700094a0 con 0x7f827c09d630 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827ad9d700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c133930 0x7f827c11e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f827a59c700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c132630 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c004510 con 0x7f827c09d630 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c0036a0 con 0x7f827c09d630 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c009550 con 0x7f827c09d630 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c003a50 con 0x7f827c09d630 2023-09-30T22:30:10.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.401+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c1330b0 con 0x7f827c09d630 2023-09-30T22:30:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.402+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c0747f0 con 0x7f827c09d630 2023-09-30T22:30:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.403+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f826c021070 con 0x7f827c09d630 2023-09-30T22:30:10.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.403+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f826c040bc0 con 0x7f827c09d630 2023-09-30T22:30:10.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.411+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f826c019770 con 0x7f827c09d630 2023-09-30T22:30:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.431+0000 7f9642eb5700 1 Processor -- start 2023-09-30T22:30:10.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.431+0000 7f9642eb5700 1 -- start start 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9642eb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c0a0a30 0x7f963c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9642eb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963c09d630 0x7f963c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9642eb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c133930 0x7f963c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9642eb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c0a2a70 con 0x7f963c09d630 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9642eb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0a01d0 con 0x7f963c133930 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9642eb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f963c09e2b0 con 0x7f963c0a0a30 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963c09d630 0x7f963c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963c09d630 0x7f963c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54814/0 (socket says 172.21.15.32:54814) 2023-09-30T22:30:10.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 -- 172.21.15.32:0/626523425 learned_addr learned my addr 172.21.15.32:0/626523425 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9640c51700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c0a0a30 0x7f963c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9641452700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c133930 0x7f963c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 -- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c0a0a30 msgr2=0x7f963c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c0a0a30 0x7f963c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 -- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c133930 msgr2=0x7f963c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c133930 0x7f963c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f963bfff700 1 -- 172.21.15.32:0/626523425 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c09dc50 con 0x7f963c09d630 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.432+0000 7f9640c51700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c0a0a30 0x7f963c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.433+0000 7f963bfff700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c0a1340 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f96280098f0 tx=0x7f9628009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee6c45ec5332ae0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.433+0000 7f963b7fe700 1 -- 172.21.15.32:0/626523425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628011830 con 0x7f963c09d630 2023-09-30T22:30:10.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.433+0000 7f963b7fe700 1 -- 172.21.15.32:0/626523425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9628011990 con 0x7f963c09d630 2023-09-30T22:30:10.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.433+0000 7f963b7fe700 1 -- 172.21.15.32:0/626523425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628011d20 con 0x7f963c09d630 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 -- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 msgr2=0x7f963c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c0a1340 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f96280098f0 tx=0x7f9628009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 -- 172.21.15.32:0/626523425 shutdown_connections 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c133930 0x7f963c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c0a0a30 0x7f963c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 --2- 172.21.15.32:0/626523425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c0a1340 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 -- 172.21.15.32:0/626523425 >> 172.21.15.32:0/626523425 conn(0x7f963c0a3050 msgr2=0x7f963c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 -- 172.21.15.32:0/626523425 shutdown_connections 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.434+0000 7f9642eb5700 1 -- 172.21.15.32:0/626523425 wait complete. 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.435+0000 7f9642eb5700 1 Processor -- start 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.435+0000 7f9642eb5700 1 -- start start 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9642eb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c121120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9642eb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c0a0a30 0x7f963c121660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9642eb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c133930 0x7f963c121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9642eb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c123c00 con 0x7f963c09d630 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9642eb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c124c40 con 0x7f963c0a0a30 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9642eb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0a51d0 con 0x7f963c133930 2023-09-30T22:30:10.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c121120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54824/0 (socket says 172.21.15.32:54824) 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 -- 172.21.15.32:0/894107908 learned_addr learned my addr 172.21.15.32:0/894107908 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f963bfff700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c0a0a30 0x7f963c121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9641452700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c133930 0x7f963c121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 -- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c133930 msgr2=0x7f963c121ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c133930 0x7f963c121ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 -- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c0a0a30 msgr2=0x7f963c121660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c0a0a30 0x7f963c121660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.436+0000 7f9640c51700 1 -- 172.21.15.32:0/894107908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300094a0 con 0x7f963c09d630 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.437+0000 7f9640c51700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c121120 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f963000ebe0 tx=0x7f963000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.437+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630009800 con 0x7f963c09d630 2023-09-30T22:30:10.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.437+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963000c910 con 0x7f963c09d630 2023-09-30T22:30:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.437+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9628009550 con 0x7f963c09d630 2023-09-30T22:30:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.437+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963000cc80 con 0x7f963c09d630 2023-09-30T22:30:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.437+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c122d30 con 0x7f963c09d630 2023-09-30T22:30:10.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.438+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c11df60 con 0x7f963c09d630 2023-09-30T22:30:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.439+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9630013470 con 0x7f963c09d630 2023-09-30T22:30:10.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.439+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9630040040 con 0x7f963c09d630 2023-09-30T22:30:10.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.443+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9630013720 con 0x7f963c09d630 2023-09-30T22:30:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.507+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f423806c360 con 0x7f423809d6f0 2023-09-30T22:30:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.507+0000 7f4234ff9700 1 -- 172.21.15.32:0/3886602566 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4228015070 con 0x7f423809d6f0 2023-09-30T22:30:10.508 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 msgr2=0x7f42381a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f42381a5e90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f4228000c00 tx=0x7f4228004750 comp rx=0 tx=0).stop 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 shutdown_connections 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f423809e440 0x7f42381a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f423809d6f0 0x7f42381a5e90 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 --2- 172.21.15.32:0/3886602566 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f423809e000 0x7f42381a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 >> 172.21.15.32:0/3886602566 conn(0x7f4238066f50 msgr2=0x7f42381bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 shutdown_connections 2023-09-30T22:30:10.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.510+0000 7f423e7a4700 1 -- 172.21.15.32:0/3886602566 wait complete. 2023-09-30T22:30:10.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:10.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.620+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f827c122290 con 0x7f827c09d630 2023-09-30T22:30:10.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.621+0000 7f826affd700 1 -- 172.21.15.32:0/1433433479 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f827c122290 con 0x7f827c09d630 2023-09-30T22:30:10.622 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 msgr2=0x7f827c132630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c132630 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 shutdown_connections 2023-09-30T22:30:10.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f827c133930 0x7f827c11e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f827c09d630 0x7f827c132630 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 --2- 172.21.15.32:0/1433433479 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f827c0a0a30 0x7f827c132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 >> 172.21.15.32:0/1433433479 conn(0x7f827c0a3050 msgr2=0x7f827c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 shutdown_connections 2023-09-30T22:30:10.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.623+0000 7f8280b3a700 1 -- 172.21.15.32:0/1433433479 wait complete. 2023-09-30T22:30:10.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:10.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.648+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f963c0747f0 con 0x7f963c09d630 2023-09-30T22:30:10.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.648+0000 7f96397fa700 1 -- 172.21.15.32:0/894107908 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f963002a090 con 0x7f963c09d630 2023-09-30T22:30:10.649 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 msgr2=0x7f963c121120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c121120 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f963000ebe0 tx=0x7f963000eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 shutdown_connections 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f963c133930 0x7f963c121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963c09d630 0x7f963c121120 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 --2- 172.21.15.32:0/894107908 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f963c0a0a30 0x7f963c121660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 >> 172.21.15.32:0/894107908 conn(0x7f963c0a3050 msgr2=0x7f963c1331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 shutdown_connections 2023-09-30T22:30:10.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:10.650+0000 7f9642eb5700 1 -- 172.21.15.32:0/894107908 wait complete. 2023-09-30T22:30:10.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:11.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:11.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 Processor -- start 2023-09-30T22:30:11.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 -- start start 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409f410 0x7f2d64136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d64136fe0 0x7f2d641393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d64070940 con 0x7f2d6409e930 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6406fd60 con 0x7f2d64136fe0 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d691b5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6406fa80 con 0x7f2d6409f410 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d6259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409f410 0x7f2d64136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d62d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d6409e930 0x7f2d6409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59214/0 (socket says 172.21.15.32:59214) 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d62d9d700 1 -- 172.21.15.32:0/1279739853 learned_addr learned my addr 172.21.15.32:0/1279739853 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.387+0000 7f2d6359e700 1 --2- 172.21.15.32:0/1279739853 >> v2:172.21.15.32:3300/0 conn(0x7f2d64136fe0 0x7f2d641393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d61d9b700 1 -- 172.21.15.32:0/1279739853 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854060082 0 0) 0x7f2d64070940 con 0x7f2d6409e930 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d61d9b700 1 -- 172.21.15.32:0/1279739853 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c0035b0 con 0x7f2d6409e930 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d6259c700 1 -- 172.21.15.32:0/1279739853 >> v2:172.21.15.32:3300/0 conn(0x7f2d64136fe0 msgr2=0x7f2d641393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d6259c700 1 --2- 172.21.15.32:0/1279739853 >> v2:172.21.15.32:3300/0 conn(0x7f2d64136fe0 0x7f2d641393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d6259c700 1 -- 172.21.15.32:0/1279739853 >> v1:172.21.15.32:6789/0 conn(0x7f2d6409e930 legacy=0x7f2d6409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d6259c700 1 -- 172.21.15.32:0/1279739853 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6409fb60 con 0x7f2d6409f410 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d6359e700 1 --2- 172.21.15.32:0/1279739853 >> v2:172.21.15.32:3300/0 conn(0x7f2d64136fe0 0x7f2d641393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d6259c700 1 --2- 172.21.15.32:0/1279739853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409f410 0x7f2d64136aa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009840 tx=0x7f2d58009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44515c1b61dfa4e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d61d9b700 1 -- 172.21.15.32:0/1279739853 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011830 con 0x7f2d6409f410 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d61d9b700 1 -- 172.21.15.32:0/1279739853 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58011990 con 0x7f2d6409f410 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.388+0000 7f2d61d9b700 1 -- 172.21.15.32:0/1279739853 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011d20 con 0x7f2d6409f410 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 -- 172.21.15.32:0/1279739853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409f410 msgr2=0x7f2d64136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 --2- 172.21.15.32:0/1279739853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409f410 0x7f2d64136aa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009840 tx=0x7f2d58009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 -- 172.21.15.32:0/1279739853 shutdown_connections 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 --2- 172.21.15.32:0/1279739853 >> v2:172.21.15.32:3300/0 conn(0x7f2d64136fe0 0x7f2d641393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 --2- 172.21.15.32:0/1279739853 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409f410 0x7f2d64136aa0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 -- 172.21.15.32:0/1279739853 >> 172.21.15.32:0/1279739853 conn(0x7f2d640a4210 msgr2=0x7f2d640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 -- 172.21.15.32:0/1279739853 shutdown_connections 2023-09-30T22:30:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.389+0000 7f2d691b5700 1 -- 172.21.15.32:0/1279739853 wait complete. 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.390+0000 7f2d691b5700 1 Processor -- start 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.390+0000 7f2d691b5700 1 -- start start 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d691b5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 0x7f2d641c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d691b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d6409f410 0x7f2d641c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d691b5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d64136fe0 0x7f2d6406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d691b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6406d6b0 con 0x7f2d6409f410 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d691b5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d641c90c0 con 0x7f2d64136fe0 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d691b5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6406d4c0 con 0x7f2d6409e930 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d62d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 0x7f2d641c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d62d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 0x7f2d641c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57636/0 (socket says 172.21.15.32:57636) 2023-09-30T22:30:11.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d62d9d700 1 -- 172.21.15.32:0/2458178716 learned_addr learned my addr 172.21.15.32:0/2458178716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d6259c700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d6409f410 0x7f2d641c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.391+0000 7f2d6359e700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d64136fe0 0x7f2d6406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d62d9d700 1 -- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d64136fe0 msgr2=0x7f2d6406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d62d9d700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d64136fe0 0x7f2d6406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d62d9d700 1 -- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d6409f410 msgr2=0x7f2d641c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d62d9d700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d6409f410 0x7f2d641c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d62d9d700 1 -- 172.21.15.32:0/2458178716 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c009a70 con 0x7f2d6409e930 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d6259c700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d6409f410 0x7f2d641c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d62d9d700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 0x7f2d641c8640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400c730 tx=0x7f2d5400ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54005d70 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d540070d0 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d580094a0 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54007440 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.392+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d641219c0 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.393+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d540075a0 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.393+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2d5403eaa0 con 0x7f2d6409e930 2023-09-30T22:30:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.393+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64074ea0 con 0x7f2d6409e930 2023-09-30T22:30:11.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.398+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d54019b40 con 0x7f2d6409e930 2023-09-30T22:30:11.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:11.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.607+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d6406b250 con 0x7f2d6409e930 2023-09-30T22:30:11.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.608+0000 7f2d4b7fe700 1 -- 172.21.15.32:0/2458178716 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d54017020 con 0x7f2d6409e930 2023-09-30T22:30:11.609 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:11.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 msgr2=0x7f2d641c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 0x7f2d641c8640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400c730 tx=0x7f2d5400ca40 comp rx=0 tx=0).stop 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 shutdown_connections 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d64136fe0 0x7f2d6406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2d6409e930 0x7f2d641c8640 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 --2- 172.21.15.32:0/2458178716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d6409f410 0x7f2d641c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 >> 172.21.15.32:0/2458178716 conn(0x7f2d640a4210 msgr2=0x7f2d64134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 shutdown_connections 2023-09-30T22:30:11.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.610+0000 7f2d691b5700 1 -- 172.21.15.32:0/2458178716 wait complete. 2023-09-30T22:30:11.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:11.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:11.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.670+0000 7efd68ed4700 1 Processor -- start 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.670+0000 7efd68ed4700 1 -- start start 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd68ed4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 0x7efd6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd68ed4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd6409e000 0x7efd64130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd68ed4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6409f130 con 0x7efd6409e410 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd68ed4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6409f520 con 0x7efd6409e000 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd68ed4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd6409e930 con 0x7efd6409d6f0 2023-09-30T22:30:11.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd61d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd6409e000 0x7efd64130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd6259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 0x7efd6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd62d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efd6409e410 0x7efd64130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59218/0 (socket says 172.21.15.32:59218) 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd6259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 0x7efd6409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57650/0 (socket says 172.21.15.32:57650) 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd61d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd6409e000 0x7efd64130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54840/0 (socket says 172.21.15.32:54840) 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd62d9d700 1 -- 172.21.15.32:0/2542725959 learned_addr learned my addr 172.21.15.32:0/2542725959 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd61d9b700 1 -- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 msgr2=0x7efd6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd61d9b700 1 --2- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 0x7efd6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd61d9b700 1 -- 172.21.15.32:0/2542725959 >> v1:172.21.15.32:6789/0 conn(0x7efd6409e410 legacy=0x7efd64130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd6159a700 1 -- 172.21.15.32:0/2542725959 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928182170 0 0) 0x7efd6409f130 con 0x7efd6409e410 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd61d9b700 1 -- 172.21.15.32:0/2542725959 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6409b2d0 con 0x7efd6409e000 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.671+0000 7efd6259c700 1 --2- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 0x7efd6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:11.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.672+0000 7efd61d9b700 1 --2- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e000 0x7efd64130660 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efd58009840 tx=0x7efd58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f95a2a8938168a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.672+0000 7efd6159a700 1 -- 172.21.15.32:0/2542725959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58011830 con 0x7efd6409e000 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.672+0000 7efd6159a700 1 -- 172.21.15.32:0/2542725959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58011990 con 0x7efd6409e000 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.672+0000 7efd6159a700 1 -- 172.21.15.32:0/2542725959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58011d20 con 0x7efd6409e000 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.672+0000 7efd68ed4700 1 -- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e000 msgr2=0x7efd64130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.672+0000 7efd68ed4700 1 --2- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e000 0x7efd64130660 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efd58009840 tx=0x7efd58009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.673+0000 7efd68ed4700 1 -- 172.21.15.32:0/2542725959 shutdown_connections 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.673+0000 7efd68ed4700 1 --2- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409d6f0 0x7efd6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.673+0000 7efd68ed4700 1 --2- 172.21.15.32:0/2542725959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e000 0x7efd64130660 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.673+0000 7efd68ed4700 1 -- 172.21.15.32:0/2542725959 >> 172.21.15.32:0/2542725959 conn(0x7efd64066f50 msgr2=0x7efd6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.673+0000 7efd68ed4700 1 -- 172.21.15.32:0/2542725959 shutdown_connections 2023-09-30T22:30:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.673+0000 7efd68ed4700 1 -- 172.21.15.32:0/2542725959 wait complete. 2023-09-30T22:30:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.674+0000 7efd68ed4700 1 Processor -- start 2023-09-30T22:30:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.674+0000 7efd68ed4700 1 -- start start 2023-09-30T22:30:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.674+0000 7efd68ed4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd6409d6f0 0x7efd6411f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.674+0000 7efd68ed4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 0x7efd6411f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.674+0000 7efd68ed4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e410 0x7efd6411fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.674+0000 7efd68ed4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64122c60 con 0x7efd6409e410 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd68ed4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6412f4d0 con 0x7efd6409d6f0 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd68ed4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd6409c700 con 0x7efd6409e000 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 0x7efd6411f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 0x7efd6411f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57656/0 (socket says 172.21.15.32:57656) 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 -- 172.21.15.32:0/4182904362 learned_addr learned my addr 172.21.15.32:0/4182904362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd62d9d700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e410 0x7efd6411fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd6259c700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd6409d6f0 0x7efd6411f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 -- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd6409d6f0 msgr2=0x7efd6411f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd6409d6f0 0x7efd6411f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 -- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e410 msgr2=0x7efd6411fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e410 0x7efd6411fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 -- 172.21.15.32:0/4182904362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540094a0 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.675+0000 7efd61d9b700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 0x7efd6411f710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7efd58009fd0 tx=0x7efd58004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.676+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd580036a0 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.676+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5801f070 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.676+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd580094a0 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.676+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58003a60 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.676+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6412f080 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.677+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd64074ea0 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.677+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efd58003bc0 con 0x7efd6409e000 2023-09-30T22:30:11.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.677+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efd58021070 con 0x7efd6409e000 2023-09-30T22:30:11.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.683+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efd580253e0 con 0x7efd6409e000 2023-09-30T22:30:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.787+0000 7fa4e5a66700 1 Processor -- start 2023-09-30T22:30:11.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.787+0000 7fa4e5a66700 1 -- start start 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4e5a66700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0133420 0x7fa4e01337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4e5a66700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4e0134260 0x7fa4e01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4e5a66700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e006b990 con 0x7fa4e0133e30 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4e5a66700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e00a0590 con 0x7fa4e0134260 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4e5a66700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e009fb80 con 0x7fa4e0133420 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4dffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4e0134260 0x7fa4e01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4deffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa4e0133e30 0x7fa4e012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59232/0 (socket says 172.21.15.32:59232) 2023-09-30T22:30:11.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4deffd700 1 -- 172.21.15.32:0/3053570862 learned_addr learned my addr 172.21.15.32:0/3053570862 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4dffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4e0134260 0x7fa4e01306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54866/0 (socket says 172.21.15.32:54866) 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4df7fe700 1 --2- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0133420 0x7fa4e01337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4dffff700 1 -- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0133420 msgr2=0x7fa4e01337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4dffff700 1 --2- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0133420 0x7fa4e01337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4dffff700 1 -- 172.21.15.32:0/3053570862 >> v1:172.21.15.32:6789/0 conn(0x7fa4e0133e30 legacy=0x7fa4e012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.788+0000 7fa4dffff700 1 -- 172.21.15.32:0/3053570862 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e00a0030 con 0x7fa4e0134260 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.789+0000 7fa4dffff700 1 --2- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0134260 0x7fa4e01306a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009d90 tx=0x7fa4d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43f9ad686a241c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.789+0000 7fa4de7fc700 1 -- 172.21.15.32:0/3053570862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800e070 con 0x7fa4e0134260 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.789+0000 7fa4de7fc700 1 -- 172.21.15.32:0/3053570862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800bbd0 con 0x7fa4e0134260 2023-09-30T22:30:11.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.789+0000 7fa4de7fc700 1 -- 172.21.15.32:0/3053570862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8012430 con 0x7fa4e0134260 2023-09-30T22:30:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.789+0000 7fa4e5a66700 1 -- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0134260 msgr2=0x7fa4e01306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.789+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0134260 0x7fa4e01306a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009d90 tx=0x7fa4d800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:11.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.790+0000 7fa4e5a66700 1 -- 172.21.15.32:0/3053570862 shutdown_connections 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.790+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0134260 0x7fa4e01306a0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.790+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/3053570862 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0133420 0x7fa4e01337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.790+0000 7fa4e5a66700 1 -- 172.21.15.32:0/3053570862 >> 172.21.15.32:0/3053570862 conn(0x7fa4e00a4210 msgr2=0x7fa4e00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.790+0000 7fa4e5a66700 1 -- 172.21.15.32:0/3053570862 shutdown_connections 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.790+0000 7fa4e5a66700 1 -- 172.21.15.32:0/3053570862 wait complete. 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.791+0000 7fa4e5a66700 1 Processor -- start 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.791+0000 7fa4e5a66700 1 -- start start 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.791+0000 7fa4e5a66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 0x7fa4e009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.791+0000 7fa4e5a66700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4e0133e30 0x7fa4e009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.791+0000 7fa4e5a66700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0134260 0x7fa4e009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4e5a66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e009d850 con 0x7fa4e0133420 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4e5a66700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e009ddf0 con 0x7fa4e0134260 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4e5a66700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e01202d0 con 0x7fa4e0133e30 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 0x7fa4e009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 0x7fa4e009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54876/0 (socket says 172.21.15.32:54876) 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4dffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0134260 0x7fa4e009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 -- 172.21.15.32:0/2422143201 learned_addr learned my addr 172.21.15.32:0/2422143201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4deffd700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4e0133e30 0x7fa4e009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 -- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4e0133e30 msgr2=0x7fa4e009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4e0133e30 0x7fa4e009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 -- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0134260 msgr2=0x7fa4e009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0134260 0x7fa4e009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 -- 172.21.15.32:0/2422143201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009a70 con 0x7fa4e0133420 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4dffff700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0134260 0x7fa4e009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4df7fe700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 0x7fa4e009c480 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e810 tx=0x7fa4d000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000c070 con 0x7fa4e0133420 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.792+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d0013610 con 0x7fa4e0133420 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.793+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e012d140 con 0x7fa4e0133420 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.793+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d00139c0 con 0x7fa4e0133420 2023-09-30T22:30:11.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.793+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e009e360 con 0x7fa4e0133420 2023-09-30T22:30:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.794+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa4d0013b20 con 0x7fa4e0133420 2023-09-30T22:30:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.794+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e0074ea0 con 0x7fa4e0133420 2023-09-30T22:30:11.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.794+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa4d003f3a0 con 0x7fa4e0133420 2023-09-30T22:30:11.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.799+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa4d000f040 con 0x7fa4e0133420 2023-09-30T22:30:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 Processor -- start 2023-09-30T22:30:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 -- start start 2023-09-30T22:30:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff69809c650 0x7ff69809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 0x7ff69809cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69809ba70 con 0x7ff698131970 2023-09-30T22:30:11.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69809d4a0 con 0x7ff69809c650 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.813+0000 7ff69e48f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff69809e8f0 con 0x7ff6980a0090 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 0x7ff69809cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 0x7ff69809cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57672/0 (socket says 172.21.15.32:57672) 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 -- 172.21.15.32:0/3243219019 learned_addr learned my addr 172.21.15.32:0/3243219019 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff697fff700 1 --2- 172.21.15.32:0/3243219019 >> v2:172.21.15.32:3300/0 conn(0x7ff69809c650 0x7ff69809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 -- 172.21.15.32:0/3243219019 >> v2:172.21.15.32:3300/0 conn(0x7ff69809c650 msgr2=0x7ff69809ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 --2- 172.21.15.32:0/3243219019 >> v2:172.21.15.32:3300/0 conn(0x7ff69809c650 0x7ff69809ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 -- 172.21.15.32:0/3243219019 >> v1:172.21.15.32:6789/0 conn(0x7ff698131970 legacy=0x7ff698131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 -- 172.21.15.32:0/3243219019 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6980a0760 con 0x7ff6980a0090 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff6977fe700 1 --2- 172.21.15.32:0/3243219019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 0x7ff69809cf60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0098f0 tx=0x7ff68c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5b8d69e9d68da5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff696ffd700 1 -- 172.21.15.32:0/3243219019 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c011830 con 0x7ff6980a0090 2023-09-30T22:30:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff696ffd700 1 -- 172.21.15.32:0/3243219019 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c011990 con 0x7ff6980a0090 2023-09-30T22:30:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.814+0000 7ff696ffd700 1 -- 172.21.15.32:0/3243219019 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c011d20 con 0x7ff6980a0090 2023-09-30T22:30:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 -- 172.21.15.32:0/3243219019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 msgr2=0x7ff69809cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3243219019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 0x7ff69809cf60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0098f0 tx=0x7ff68c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 -- 172.21.15.32:0/3243219019 shutdown_connections 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3243219019 >> v2:172.21.15.32:3300/0 conn(0x7ff69809c650 0x7ff69809ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3243219019 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff6980a0090 0x7ff69809cf60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 -- 172.21.15.32:0/3243219019 >> 172.21.15.32:0/3243219019 conn(0x7ff6980a3f70 msgr2=0x7ff6980a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 -- 172.21.15.32:0/3243219019 shutdown_connections 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.815+0000 7ff69e48f700 1 -- 172.21.15.32:0/3243219019 wait complete. 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.817+0000 7ff69e48f700 1 Processor -- start 2023-09-30T22:30:11.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.817+0000 7ff69e48f700 1 -- start start 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69e48f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff69809c650 0x7ff69811a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69e48f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6980a0090 0x7ff69811ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69e48f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff698131970 0x7ff69811b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69e48f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69811d5a0 con 0x7ff6980a0090 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69e48f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69811bc40 con 0x7ff698131970 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69e48f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff69811d7d0 con 0x7ff69809c650 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff6977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6980a0090 0x7ff69811ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff6977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6980a0090 0x7ff69811ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54898/0 (socket says 172.21.15.32:54898) 2023-09-30T22:30:11.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff6977fe700 1 -- 172.21.15.32:0/3149230574 learned_addr learned my addr 172.21.15.32:0/3149230574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff697fff700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff69809c650 0x7ff69811a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff698131970 0x7ff69811b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 -- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff69809c650 msgr2=0x7ff69811a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff69809c650 0x7ff69811a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 -- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6980a0090 msgr2=0x7ff69811ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6980a0090 0x7ff69811ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 -- 172.21.15.32:0/3149230574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009550 con 0x7ff698131970 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.818+0000 7ff69ca2c700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff698131970 0x7ff69811b0a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cb90 tx=0x7ff69000cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.819+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690003c00 con 0x7ff698131970 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.819+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff690003d60 con 0x7ff698131970 2023-09-30T22:30:11.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.819+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69811c150 con 0x7ff698131970 2023-09-30T22:30:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.819+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69811dab0 con 0x7ff698131970 2023-09-30T22:30:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.820+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69000aa80 con 0x7ff698131970 2023-09-30T22:30:11.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.820+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff69000ac20 con 0x7ff698131970 2023-09-30T22:30:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.820+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6981b8e10 con 0x7ff698131970 2023-09-30T22:30:11.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.820+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff69003f770 con 0x7ff698131970 2023-09-30T22:30:11.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.829+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6900050d0 con 0x7ff698131970 2023-09-30T22:30:11.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.908+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efd641227e0 con 0x7efd6409e000 2023-09-30T22:30:11.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.909+0000 7efd52ffd700 1 -- 172.21.15.32:0/4182904362 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd641227e0 con 0x7efd6409e000 2023-09-30T22:30:11.910 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 msgr2=0x7efd6411f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 0x7efd6411f710 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7efd58009fd0 tx=0x7efd58004750 comp rx=0 tx=0).stop 2023-09-30T22:30:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 shutdown_connections 2023-09-30T22:30:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd6409e410 0x7efd6411fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd6409d6f0 0x7efd6411f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 --2- 172.21.15.32:0/4182904362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efd6409e000 0x7efd6411f710 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 >> 172.21.15.32:0/4182904362 conn(0x7efd64066f50 msgr2=0x7efd6412f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 shutdown_connections 2023-09-30T22:30:11.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:11.911+0000 7efd68ed4700 1 -- 172.21.15.32:0/4182904362 wait complete. 2023-09-30T22:30:11.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.009+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa4e011f860 con 0x7fa4e0133420 2023-09-30T22:30:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.009+0000 7fa4c7fff700 1 -- 172.21.15.32:0/2422143201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4d0010540 con 0x7fa4e0133420 2023-09-30T22:30:12.010 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 msgr2=0x7fa4e009c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 0x7fa4e009c480 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e810 tx=0x7fa4d000eb20 comp rx=0 tx=0).stop 2023-09-30T22:30:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 shutdown_connections 2023-09-30T22:30:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa4e0134260 0x7fa4e009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4e0133420 0x7fa4e009c480 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 --2- 172.21.15.32:0/2422143201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa4e0133e30 0x7fa4e009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 >> 172.21.15.32:0/2422143201 conn(0x7fa4e00a4210 msgr2=0x7fa4e00a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 shutdown_connections 2023-09-30T22:30:12.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.012+0000 7fa4e5a66700 1 -- 172.21.15.32:0/2422143201 wait complete. 2023-09-30T22:30:12.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:12.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.036+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff6980747f0 con 0x7ff698131970 2023-09-30T22:30:12.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.036+0000 7ff694ff9700 1 -- 172.21.15.32:0/3149230574 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff690011380 con 0x7ff698131970 2023-09-30T22:30:12.037 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff698131970 msgr2=0x7ff69811b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff698131970 0x7ff69811b0a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cb90 tx=0x7ff69000cea0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 shutdown_connections 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff698131970 0x7ff69811b0a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff69809c650 0x7ff69811a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 --2- 172.21.15.32:0/3149230574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6980a0090 0x7ff69811ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 >> 172.21.15.32:0/3149230574 conn(0x7ff6980a3f70 msgr2=0x7ff69812ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 shutdown_connections 2023-09-30T22:30:12.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.039+0000 7ff69e48f700 1 -- 172.21.15.32:0/3149230574 wait complete. 2023-09-30T22:30:12.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:12.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.776+0000 7efcbb745700 1 Processor -- start 2023-09-30T22:30:12.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 -- start start 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcb409d6f0 0x7efcb409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 0x7efcb41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 0x7efcb4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb406b460 con 0x7efcb409d6f0 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb409b120 con 0x7efcb409e410 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcbb745700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcb409a7e0 con 0x7efcb409e000 2023-09-30T22:30:12.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb94e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcb409d6f0 0x7efcb409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb8ce0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 0x7efcb41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb9ce2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 0x7efcb4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb94e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcb409d6f0 0x7efcb409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54904/0 (socket says 172.21.15.32:54904) 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb8ce0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 0x7efcb41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57684/0 (socket says 172.21.15.32:57684) 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb94e1700 1 -- 172.21.15.32:0/1729981807 learned_addr learned my addr 172.21.15.32:0/1729981807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.777+0000 7efcb9ce2700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 0x7efcb4133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42772/0 (socket says 172.21.15.32:42772) 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb94e1700 1 -- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 msgr2=0x7efcb41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb94e1700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 0x7efcb41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb94e1700 1 -- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 msgr2=0x7efcb4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb94e1700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 0x7efcb4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb94e1700 1 -- 172.21.15.32:0/1729981807 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb409a500 con 0x7efcb409d6f0 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb9ce2700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 0x7efcb4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb8ce0700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 0x7efcb41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcb94e1700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb409dac0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7efcb00098f0 tx=0x7efcb0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624827a155248498 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcabfff700 1 -- 172.21.15.32:0/1729981807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0011830 con 0x7efcb409d6f0 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcabfff700 1 -- 172.21.15.32:0/1729981807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb0011990 con 0x7efcb409d6f0 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.778+0000 7efcabfff700 1 -- 172.21.15.32:0/1729981807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0011d20 con 0x7efcb409d6f0 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 -- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 msgr2=0x7efcb409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb409dac0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7efcb00098f0 tx=0x7efcb0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 -- 172.21.15.32:0/1729981807 shutdown_connections 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e410 0x7efcb4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb409dac0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 --2- 172.21.15.32:0/1729981807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e000 0x7efcb41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 -- 172.21.15.32:0/1729981807 >> 172.21.15.32:0/1729981807 conn(0x7efcb4066f50 msgr2=0x7efcb409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 -- 172.21.15.32:0/1729981807 shutdown_connections 2023-09-30T22:30:12.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.779+0000 7efcbb745700 1 -- 172.21.15.32:0/1729981807 wait complete. 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 Processor -- start 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 -- start start 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb41a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e000 0x7efcb41a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e410 0x7efcb41c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb41a84b0 con 0x7efcb409d6f0 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb41b9320 con 0x7efcb409e000 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.780+0000 7efcbb745700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcb41a4ec0 con 0x7efcb409e410 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb41a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb9ce2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e410 0x7efcb41c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb8ce0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e000 0x7efcb41a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb41a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54914/0 (socket says 172.21.15.32:54914) 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 -- 172.21.15.32:0/1352831353 learned_addr learned my addr 172.21.15.32:0/1352831353 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 -- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e410 msgr2=0x7efcb41c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e410 0x7efcb41c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 -- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e000 msgr2=0x7efcb41a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e000 0x7efcb41a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 -- 172.21.15.32:0/1352831353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40094a0 con 0x7efcb409d6f0 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb8ce0700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e000 0x7efcb41a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcb94e1700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb41a5340 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009f50 tx=0x7efcb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0004240 con 0x7efcb409d6f0 2023-09-30T22:30:12.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb001f070 con 0x7efcb409d6f0 2023-09-30T22:30:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb00045c0 con 0x7efcb409d6f0 2023-09-30T22:30:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.781+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac009a70 con 0x7efcb409d6f0 2023-09-30T22:30:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.782+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb0009550 con 0x7efcb409d6f0 2023-09-30T22:30:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.782+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efcb0021030 con 0x7efcb409d6f0 2023-09-30T22:30:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.783+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efcb0040850 con 0x7efcb409d6f0 2023-09-30T22:30:12.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.783+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb409c980 con 0x7efcb409d6f0 2023-09-30T22:30:12.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.788+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efcb0015080 con 0x7efcb409d6f0 2023-09-30T22:30:12.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.994+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efcb409c710 con 0x7efcb409d6f0 2023-09-30T22:30:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.994+0000 7efca9ffb700 1 -- 172.21.15.32:0/1352831353 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efcb001a450 con 0x7efcb409d6f0 2023-09-30T22:30:12.995 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 msgr2=0x7efcb41a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb41a5340 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009f50 tx=0x7efcb0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 shutdown_connections 2023-09-30T22:30:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efcb409e410 0x7efcb41c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb409d6f0 0x7efcb41a5340 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 --2- 172.21.15.32:0/1352831353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcb409e000 0x7efcb41a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 >> 172.21.15.32:0/1352831353 conn(0x7efcb4066f50 msgr2=0x7efcb41bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 shutdown_connections 2023-09-30T22:30:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:12.997+0000 7efcbb745700 1 -- 172.21.15.32:0/1352831353 wait complete. 2023-09-30T22:30:13.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:13.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:13.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.064+0000 7feebc619700 1 Processor -- start 2023-09-30T22:30:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.064+0000 7feebc619700 1 -- start start 2023-09-30T22:30:13.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebc619700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb40a0a30 0x7feeb40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebc619700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb409d630 0x7feeb40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebc619700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb40a2a70 con 0x7feeb4133930 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebc619700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40a01d0 con 0x7feeb40a0a30 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebc619700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb409e2b0 con 0x7feeb409d630 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeb9bb4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb409d630 0x7feeb40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebabb6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feeb4133930 0x7feeb4133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59260/0 (socket says 172.21.15.32:59260) 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feebabb6700 1 -- 172.21.15.32:0/1769118656 learned_addr learned my addr 172.21.15.32:0/1769118656 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:13.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeba3b5700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb40a0a30 0x7feeb40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeb93b3700 1 -- 172.21.15.32:0/1769118656 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975317824 0 0) 0x7feeb40a2a70 con 0x7feeb4133930 2023-09-30T22:30:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeba3b5700 1 -- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb409d630 msgr2=0x7feeb40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeba3b5700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb409d630 0x7feeb40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeba3b5700 1 -- 172.21.15.32:0/1769118656 >> v1:172.21.15.32:6789/0 conn(0x7feeb4133930 legacy=0x7feeb4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeba3b5700 1 -- 172.21.15.32:0/1769118656 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb409dc50 con 0x7feeb40a0a30 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.065+0000 7feeb9bb4700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb409d630 0x7feeb40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.066+0000 7feeba3b5700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb40a0a30 0x7feeb40a0e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7feea4009dc0 tx=0x7feea400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a86eeec975cbef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.066+0000 7feeb93b3700 1 -- 172.21.15.32:0/1769118656 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400e070 con 0x7feeb40a0a30 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.066+0000 7feeb93b3700 1 -- 172.21.15.32:0/1769118656 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400bb20 con 0x7feeb40a0a30 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.066+0000 7feeb93b3700 1 -- 172.21.15.32:0/1769118656 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4012430 con 0x7feeb40a0a30 2023-09-30T22:30:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 -- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb40a0a30 msgr2=0x7feeb40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb40a0a30 0x7feeb40a0e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7feea4009dc0 tx=0x7feea400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:13.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 -- 172.21.15.32:0/1769118656 shutdown_connections 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb40a0a30 0x7feeb40a0e00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 --2- 172.21.15.32:0/1769118656 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb409d630 0x7feeb40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 -- 172.21.15.32:0/1769118656 >> 172.21.15.32:0/1769118656 conn(0x7feeb40a3050 msgr2=0x7feeb40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 -- 172.21.15.32:0/1769118656 shutdown_connections 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.067+0000 7feebc619700 1 -- 172.21.15.32:0/1769118656 wait complete. 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.068+0000 7feebc619700 1 Processor -- start 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.068+0000 7feebc619700 1 -- start start 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebc619700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb409d630 0x7feeb41c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebc619700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 0x7feeb41c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebc619700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb4133930 0x7feeb41c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebc619700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4120e20 con 0x7feeb409d630 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebc619700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb41206d0 con 0x7feeb4133930 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebc619700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb40a5360 con 0x7feeb40a0a30 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feeb9bb4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 0x7feeb41c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebabb6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb4133930 0x7feeb41c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feeb9bb4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 0x7feeb41c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57708/0 (socket says 172.21.15.32:57708) 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feeba3b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb409d630 0x7feeb41c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feeb9bb4700 1 -- 172.21.15.32:0/3343446310 learned_addr learned my addr 172.21.15.32:0/3343446310 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebabb6700 1 -- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 msgr2=0x7feeb41c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebabb6700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 0x7feeb41c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebabb6700 1 -- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb409d630 msgr2=0x7feeb41c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebabb6700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb409d630 0x7feeb41c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feebabb6700 1 -- 172.21.15.32:0/3343446310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea80094a0 con 0x7feeb4133930 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.069+0000 7feeb9bb4700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 0x7feeb41c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:13.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.070+0000 7feebabb6700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb4133930 0x7feeb41c8b10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c430 tx=0x7feeac00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.070+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac004c80 con 0x7feeb4133930 2023-09-30T22:30:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.070+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac004e00 con 0x7feeb4133930 2023-09-30T22:30:13.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.070+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea4009a70 con 0x7feeb4133930 2023-09-30T22:30:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.070+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac007310 con 0x7feeb4133930 2023-09-30T22:30:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.070+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb4121f10 con 0x7feeb4133930 2023-09-30T22:30:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.071+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feeac007470 con 0x7feeb4133930 2023-09-30T22:30:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.071+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb41220e0 con 0x7feeb4133930 2023-09-30T22:30:13.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.071+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feeac01a2b0 con 0x7feeb4133930 2023-09-30T22:30:13.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.080+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feeb41220e0 con 0x7feeb4133930 2023-09-30T22:30:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.174+0000 7f4a69fb6700 1 Processor -- start 2023-09-30T22:30:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.174+0000 7f4a69fb6700 1 -- start start 2023-09-30T22:30:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.174+0000 7f4a69fb6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 0x7f4a640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a69fb6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a64134170 0x7f4a6413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a69fb6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6409edd0 con 0x7f4a640a1340 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a69fb6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6409e1f0 con 0x7f4a64134170 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a69fb6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6409dc50 con 0x7f4a640a0a30 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 0x7f4a640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 0x7f4a640a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57724/0 (socket says 172.21.15.32:57724) 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 -- 172.21.15.32:0/2287729096 learned_addr learned my addr 172.21.15.32:0/2287729096 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2287729096 >> v2:172.21.15.32:3300/0 conn(0x7f4a64134170 0x7f4a6413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 -- 172.21.15.32:0/2287729096 >> v2:172.21.15.32:3300/0 conn(0x7f4a64134170 msgr2=0x7f4a6413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 --2- 172.21.15.32:0/2287729096 >> v2:172.21.15.32:3300/0 conn(0x7f4a64134170 0x7f4a6413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 -- 172.21.15.32:0/2287729096 >> v1:172.21.15.32:6789/0 conn(0x7f4a640a1340 legacy=0x7f4a64133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 -- 172.21.15.32:0/2287729096 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a640a1fe0 con 0x7f4a640a0a30 2023-09-30T22:30:13.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a627fc700 1 -- 172.21.15.32:0/2287729096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388509752 0 0) 0x7f4a6409edd0 con 0x7f4a640a1340 2023-09-30T22:30:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.175+0000 7f4a637fe700 1 --2- 172.21.15.32:0/2287729096 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 0x7f4a640a0e00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e50 tx=0x7f4a5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4264eb39d58bb5d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:13.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a627fc700 1 -- 172.21.15.32:0/2287729096 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400e070 con 0x7f4a640a0a30 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a627fc700 1 -- 172.21.15.32:0/2287729096 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5400b910 con 0x7f4a640a0a30 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a627fc700 1 -- 172.21.15.32:0/2287729096 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400bcc0 con 0x7f4a640a0a30 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2287729096 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 msgr2=0x7f4a640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2287729096 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 0x7f4a640a0e00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e50 tx=0x7f4a5400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2287729096 shutdown_connections 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2287729096 >> v2:172.21.15.32:3300/0 conn(0x7f4a64134170 0x7f4a6413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2287729096 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a0a30 0x7f4a640a0e00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2287729096 >> 172.21.15.32:0/2287729096 conn(0x7f4a640a6550 msgr2=0x7f4a640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.176+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2287729096 shutdown_connections 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.177+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2287729096 wait complete. 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.177+0000 7f4a69fb6700 1 Processor -- start 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 -- start start 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a640a0a30 0x7f4a64122830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a1340 0x7f4a64122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 0x7f4a641232b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a641332a0 con 0x7f4a64134170 2023-09-30T22:30:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a64124450 con 0x7f4a640a0a30 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a69fb6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640a4cd0 con 0x7f4a640a1340 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 0x7f4a641232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a62ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a1340 0x7f4a64122d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 0x7f4a641232b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54940/0 (socket says 172.21.15.32:54940) 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.178+0000 7f4a63fff700 1 -- 172.21.15.32:0/2071501273 learned_addr learned my addr 172.21.15.32:0/2071501273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a637fe700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a640a0a30 0x7f4a64122830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a63fff700 1 -- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a1340 msgr2=0x7f4a64122d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a1340 0x7f4a64122d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a63fff700 1 -- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a640a0a30 msgr2=0x7f4a64122830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a640a0a30 0x7f4a64122830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a63fff700 1 -- 172.21.15.32:0/2071501273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a54009b30 con 0x7f4a64134170 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a62ffd700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a1340 0x7f4a64122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a63fff700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 0x7f4a641232b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00e950 tx=0x7f4a5c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c009730 con 0x7f4a64134170 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c013680 con 0x7f4a64134170 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c013a10 con 0x7f4a64134170 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a640a0220 con 0x7f4a64134170 2023-09-30T22:30:13.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.179+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a641262c0 con 0x7f4a64134170 2023-09-30T22:30:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.181+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a5c013b70 con 0x7f4a64134170 2023-09-30T22:30:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.181+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a64125f00 con 0x7f4a64134170 2023-09-30T22:30:13.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.181+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4a5c03fa10 con 0x7f4a64134170 2023-09-30T22:30:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.186+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a5c01ada0 con 0x7f4a64134170 2023-09-30T22:30:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.208+0000 7fba5588d700 1 Processor -- start 2023-09-30T22:30:13.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.208+0000 7fba5588d700 1 -- start start 2023-09-30T22:30:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba5588d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba5588d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 0x7fba50133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba5588d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba50133fa0 0x7fba50134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba5588d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5009edd0 con 0x7fba50133fa0 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba5588d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5009e1f0 con 0x7fba500a1340 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba5588d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba5009dc50 con 0x7fba500a0a30 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 0x7fba50133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 0x7fba50133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42824/0 (socket says 172.21.15.32:42824) 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 -- 172.21.15.32:0/1681213714 learned_addr learned my addr 172.21.15.32:0/1681213714 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:13.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4f7fe700 1 --2- 172.21.15.32:0/1681213714 >> v2:172.21.15.32:3300/0 conn(0x7fba50133fa0 0x7fba50134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 -- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 msgr2=0x7fba500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 --2- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba500a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 -- 172.21.15.32:0/1681213714 >> v2:172.21.15.32:3300/0 conn(0x7fba50133fa0 msgr2=0x7fba50134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 --2- 172.21.15.32:0/1681213714 >> v2:172.21.15.32:3300/0 conn(0x7fba50133fa0 0x7fba50134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.209+0000 7fba4e7fc700 1 -- 172.21.15.32:0/1681213714 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500a1fe0 con 0x7fba500a1340 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.210+0000 7fba4e7fc700 1 --2- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 0x7fba50133a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fba440098f0 tx=0x7fba44009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3023d74c4189164b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.210+0000 7fba4dffb700 1 -- 172.21.15.32:0/1681213714 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44011830 con 0x7fba500a1340 2023-09-30T22:30:13.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.210+0000 7fba4dffb700 1 -- 172.21.15.32:0/1681213714 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba44011990 con 0x7fba500a1340 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.210+0000 7fba4dffb700 1 -- 172.21.15.32:0/1681213714 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44011d20 con 0x7fba500a1340 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.210+0000 7fba5588d700 1 -- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 msgr2=0x7fba50133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.210+0000 7fba5588d700 1 --2- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 0x7fba50133a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fba440098f0 tx=0x7fba44009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 -- 172.21.15.32:0/1681213714 shutdown_connections 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 --2- 172.21.15.32:0/1681213714 >> v2:172.21.15.32:3300/0 conn(0x7fba50133fa0 0x7fba50134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 --2- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 --2- 172.21.15.32:0/1681213714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba500a1340 0x7fba50133a60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 -- 172.21.15.32:0/1681213714 >> 172.21.15.32:0/1681213714 conn(0x7fba500a6550 msgr2=0x7fba500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 -- 172.21.15.32:0/1681213714 shutdown_connections 2023-09-30T22:30:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.211+0000 7fba5588d700 1 -- 172.21.15.32:0/1681213714 wait complete. 2023-09-30T22:30:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.212+0000 7fba5588d700 1 Processor -- start 2023-09-30T22:30:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.212+0000 7fba5588d700 1 -- start start 2023-09-30T22:30:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba5588d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba5011df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba5588d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba500a1340 0x7fba5011e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba5588d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba50133fa0 0x7fba5011e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba5588d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba501223e0 con 0x7fba500a1340 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba5588d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba50121a20 con 0x7fba50133fa0 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba5588d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba50120df0 con 0x7fba500a0a30 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba5011df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba5011df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57748/0 (socket says 172.21.15.32:57748) 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 -- 172.21.15.32:0/574777945 learned_addr learned my addr 172.21.15.32:0/574777945 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4e7fc700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba500a1340 0x7fba5011e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4f7fe700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba50133fa0 0x7fba5011e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 -- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba50133fa0 msgr2=0x7fba5011e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba50133fa0 0x7fba5011e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 -- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba500a1340 msgr2=0x7fba5011e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba500a1340 0x7fba5011e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4effd700 1 -- 172.21.15.32:0/574777945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009550 con 0x7fba500a0a30 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4f7fe700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba50133fa0 0x7fba5011e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.213+0000 7fba4e7fc700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba500a1340 0x7fba5011e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.214+0000 7fba4effd700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba5011df00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fba40009db0 tx=0x7fba4000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.214+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40009c20 con 0x7fba500a0a30 2023-09-30T22:30:13.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.214+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4000f040 con 0x7fba500a0a30 2023-09-30T22:30:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.214+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40014480 con 0x7fba500a0a30 2023-09-30T22:30:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.214+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba500a0220 con 0x7fba500a0a30 2023-09-30T22:30:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.214+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba50120340 con 0x7fba500a0a30 2023-09-30T22:30:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.215+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba400145e0 con 0x7fba500a0a30 2023-09-30T22:30:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.215+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba50074ea0 con 0x7fba500a0a30 2023-09-30T22:30:13.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.215+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fba400402f0 con 0x7fba500a0a30 2023-09-30T22:30:13.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.224+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba4001d400 con 0x7fba500a0a30 2023-09-30T22:30:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.305+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feeb4122cb0 con 0x7feeb4133930 2023-09-30T22:30:13.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.305+0000 7feeb2ffd700 1 -- 172.21.15.32:0/3343446310 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feeac044060 con 0x7feeb4133930 2023-09-30T22:30:13.306 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:13.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.307+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb4133930 msgr2=0x7feeb41c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.307+0000 7feebc619700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb4133930 0x7feeb41c8b10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c430 tx=0x7feeac00c740 comp rx=0 tx=0).stop 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.307+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 shutdown_connections 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.307+0000 7feebc619700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb4133930 0x7feeb41c8b10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.307+0000 7feebc619700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb409d630 0x7feeb41c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.308+0000 7feebc619700 1 --2- 172.21.15.32:0/3343446310 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb40a0a30 0x7feeb41c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.308+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 >> 172.21.15.32:0/3343446310 conn(0x7feeb40a3050 msgr2=0x7feeb41bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.308+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 shutdown_connections 2023-09-30T22:30:13.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.308+0000 7feebc619700 1 -- 172.21.15.32:0/3343446310 wait complete. 2023-09-30T22:30:13.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.393+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4a64074ea0 con 0x7f4a64134170 2023-09-30T22:30:13.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.393+0000 7f4a4bfff700 1 -- 172.21.15.32:0/2071501273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a5c01a360 con 0x7f4a64134170 2023-09-30T22:30:13.394 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 msgr2=0x7f4a641232b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 0x7f4a641232b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00e950 tx=0x7f4a5c00ec60 comp rx=0 tx=0).stop 2023-09-30T22:30:13.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 shutdown_connections 2023-09-30T22:30:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a64134170 0x7f4a641232b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4a640a0a30 0x7f4a64122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 --2- 172.21.15.32:0/2071501273 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4a640a1340 0x7f4a64122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 >> 172.21.15.32:0/2071501273 conn(0x7f4a640a6550 msgr2=0x7f4a6409c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 shutdown_connections 2023-09-30T22:30:13.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.395+0000 7f4a69fb6700 1 -- 172.21.15.32:0/2071501273 wait complete. 2023-09-30T22:30:13.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:13.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.432+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fba5011fb10 con 0x7fba500a0a30 2023-09-30T22:30:13.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.432+0000 7fba37fff700 1 -- 172.21.15.32:0/574777945 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba40007330 con 0x7fba500a0a30 2023-09-30T22:30:13.433 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:13.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 msgr2=0x7fba5011df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba5011df00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fba40009db0 tx=0x7fba4000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 shutdown_connections 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fba50133fa0 0x7fba5011e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fba500a0a30 0x7fba5011df00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 --2- 172.21.15.32:0/574777945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba500a1340 0x7fba5011e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.434+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 >> 172.21.15.32:0/574777945 conn(0x7fba500a6550 msgr2=0x7fba5009c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.435+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 shutdown_connections 2023-09-30T22:30:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:13.435+0000 7fba5588d700 1 -- 172.21.15.32:0/574777945 wait complete. 2023-09-30T22:30:13.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:14.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.160+0000 7fb7a0800700 1 Processor -- start 2023-09-30T22:30:14.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.160+0000 7fb7a0800700 1 -- start start 2023-09-30T22:30:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb7a0800700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809e1d0 0x7fb798135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb7a0800700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb798135e50 0x7fb798138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb7a0800700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79809f130 con 0x7fb79809d6f0 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb7a0800700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79809f520 con 0x7fb79809e1d0 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb7a0800700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb79809e930 con 0x7fb798135e50 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb79809d6f0 0x7fb79809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59272/0 (socket says 172.21.15.32:59272) 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb798135e50 0x7fb798138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79e59c700 1 -- 172.21.15.32:0/1626862982 learned_addr learned my addr 172.21.15.32:0/1626862982 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79dd9b700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809e1d0 0x7fb798135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79dd9b700 1 -- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb798135e50 msgr2=0x7fb798138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79dd9b700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb798135e50 0x7fb798138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79dd9b700 1 -- 172.21.15.32:0/1626862982 >> v1:172.21.15.32:6789/0 conn(0x7fb79809d6f0 legacy=0x7fb79809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79d59a700 1 -- 172.21.15.32:0/1626862982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621567077 0 0) 0x7fb79809f130 con 0x7fb79809d6f0 2023-09-30T22:30:14.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79dd9b700 1 -- 172.21.15.32:0/1626862982 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79809b2d0 con 0x7fb79809e1d0 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79ed9d700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb798135e50 0x7fb798138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.161+0000 7fb79dd9b700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809e1d0 0x7fb798135910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb784009870 tx=0x7fb784009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95e3543aa80b186 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb79d59a700 1 -- 172.21.15.32:0/1626862982 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784011830 con 0x7fb79809e1d0 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb79d59a700 1 -- 172.21.15.32:0/1626862982 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb784011990 con 0x7fb79809e1d0 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb79d59a700 1 -- 172.21.15.32:0/1626862982 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784011d20 con 0x7fb79809e1d0 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 -- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809e1d0 msgr2=0x7fb798135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809e1d0 0x7fb798135910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb784009870 tx=0x7fb784009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 -- 172.21.15.32:0/1626862982 shutdown_connections 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb798135e50 0x7fb798138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 --2- 172.21.15.32:0/1626862982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809e1d0 0x7fb798135910 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 -- 172.21.15.32:0/1626862982 >> 172.21.15.32:0/1626862982 conn(0x7fb798066f50 msgr2=0x7fb79809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 -- 172.21.15.32:0/1626862982 shutdown_connections 2023-09-30T22:30:14.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.162+0000 7fb7a0800700 1 -- 172.21.15.32:0/1626862982 wait complete. 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.164+0000 7fb7a0800700 1 Processor -- start 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.164+0000 7fb7a0800700 1 -- start start 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb7a0800700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 0x7fb79811b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb7a0800700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb79809e1d0 0x7fb79811b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb7a0800700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798135e50 0x7fb79811bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb7a0800700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798072090 con 0x7fb798135e50 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb7a0800700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79809c700 con 0x7fb79809d6f0 2023-09-30T22:30:14.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb7a0800700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb79811e4c0 con 0x7fb79809e1d0 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 0x7fb79811b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb79809e1d0 0x7fb79811b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 0x7fb79811b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42850/0 (socket says 172.21.15.32:42850) 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 -- 172.21.15.32:0/3948992485 learned_addr learned my addr 172.21.15.32:0/3948992485 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb79809e1d0 0x7fb79811b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57764/0 (socket says 172.21.15.32:57764) 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79ed9d700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798135e50 0x7fb79811bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 -- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb79809e1d0 msgr2=0x7fb79811b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb79809e1d0 0x7fb79811b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 -- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798135e50 msgr2=0x7fb79811bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798135e50 0x7fb79811bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 -- 172.21.15.32:0/3948992485 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794009a70 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb79e59c700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 0x7fb79811b260 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c520 tx=0x7fb78c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.165+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c004c80 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.166+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c004e00 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.166+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7840094a0 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.166+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c007310 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.166+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79811d5d0 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.166+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb78c007470 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.166+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb78c03f270 con 0x7fb79809d6f0 2023-09-30T22:30:14.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.167+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb798074ea0 con 0x7fb79809d6f0 2023-09-30T22:30:14.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.172+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb78c0123b0 con 0x7fb79809d6f0 2023-09-30T22:30:14.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:14.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.379+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb79806c360 con 0x7fb79809d6f0 2023-09-30T22:30:14.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.379+0000 7fb792ffd700 1 -- 172.21.15.32:0/3948992485 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb78c00ce60 con 0x7fb79809d6f0 2023-09-30T22:30:14.379 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 msgr2=0x7fb79811b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 0x7fb79811b260 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00c520 tx=0x7fb78c00c830 comp rx=0 tx=0).stop 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 shutdown_connections 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb798135e50 0x7fb79811bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb79809d6f0 0x7fb79811b260 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 --2- 172.21.15.32:0/3948992485 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb79809e1d0 0x7fb79811b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 >> 172.21.15.32:0/3948992485 conn(0x7fb798066f50 msgr2=0x7fb7981326f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 shutdown_connections 2023-09-30T22:30:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.381+0000 7fb7a0800700 1 -- 172.21.15.32:0/3948992485 wait complete. 2023-09-30T22:30:14.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:14.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:14.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:14.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.465+0000 7fdf870ba700 1 Processor -- start 2023-09-30T22:30:14.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.465+0000 7fdf870ba700 1 -- start start 2023-09-30T22:30:14.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf870ba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf8009d6f0 0x7fdf8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf870ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e000 0x7fdf80130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf870ba700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8009f130 con 0x7fdf8009e410 2023-09-30T22:30:14.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf870ba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8009f520 con 0x7fdf8009d6f0 2023-09-30T22:30:14.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf870ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8009e930 con 0x7fdf8009e000 2023-09-30T22:30:14.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf84e56700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf8009d6f0 0x7fdf8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf85657700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdf8009e410 0x7fdf80130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59276/0 (socket says 172.21.15.32:59276) 2023-09-30T22:30:14.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf85657700 1 -- 172.21.15.32:0/4018105696 learned_addr learned my addr 172.21.15.32:0/4018105696 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:14.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.466+0000 7fdf7ffff700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e000 0x7fdf80130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf84e56700 1 -- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e000 msgr2=0x7fdf80130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf84e56700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e000 0x7fdf80130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf7f7fe700 1 -- 172.21.15.32:0/4018105696 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518372148 0 0) 0x7fdf8009f130 con 0x7fdf8009e410 2023-09-30T22:30:14.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf84e56700 1 -- 172.21.15.32:0/4018105696 >> v1:172.21.15.32:6789/0 conn(0x7fdf8009e410 legacy=0x7fdf80130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf84e56700 1 -- 172.21.15.32:0/4018105696 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8009b2d0 con 0x7fdf8009d6f0 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf7ffff700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e000 0x7fdf80130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf84e56700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009d6f0 0x7fdf8009dac0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400bae0 tx=0x7fdf7400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60940814aaee7ebc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf7f7fe700 1 -- 172.21.15.32:0/4018105696 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400dc70 con 0x7fdf8009d6f0 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf7f7fe700 1 -- 172.21.15.32:0/4018105696 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7400ddd0 con 0x7fdf8009d6f0 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.467+0000 7fdf7f7fe700 1 -- 172.21.15.32:0/4018105696 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf74010690 con 0x7fdf8009d6f0 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 -- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009d6f0 msgr2=0x7fdf8009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009d6f0 0x7fdf8009dac0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400bae0 tx=0x7fdf7400bdf0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 -- 172.21.15.32:0/4018105696 shutdown_connections 2023-09-30T22:30:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009d6f0 0x7fdf8009dac0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 --2- 172.21.15.32:0/4018105696 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e000 0x7fdf80130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 -- 172.21.15.32:0/4018105696 >> 172.21.15.32:0/4018105696 conn(0x7fdf80066f50 msgr2=0x7fdf8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 -- 172.21.15.32:0/4018105696 shutdown_connections 2023-09-30T22:30:14.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.468+0000 7fdf870ba700 1 -- 172.21.15.32:0/4018105696 wait complete. 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.469+0000 7fdf870ba700 1 Processor -- start 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.469+0000 7fdf870ba700 1 -- start start 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf870ba700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 0x7fdf8011e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf870ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009e000 0x7fdf8011e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf870ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e410 0x7fdf8011ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf870ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80121840 con 0x7fdf8009e000 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf870ba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801303e0 con 0x7fdf8009e410 2023-09-30T22:30:14.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf870ba700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800a2070 con 0x7fdf8009d6f0 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf84e56700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 0x7fdf8011e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf84e56700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 0x7fdf8011e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57766/0 (socket says 172.21.15.32:57766) 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf84e56700 1 -- 172.21.15.32:0/1659923842 learned_addr learned my addr 172.21.15.32:0/1659923842 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf7ffff700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009e000 0x7fdf8011e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf84e56700 1 -- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e410 msgr2=0x7fdf8011ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.470+0000 7fdf84e56700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e410 0x7fdf8011ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf84e56700 1 -- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009e000 msgr2=0x7fdf8011e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf84e56700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009e000 0x7fdf8011e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf84e56700 1 -- 172.21.15.32:0/1659923842 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c0094a0 con 0x7fdf8009d6f0 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf7ffff700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009e000 0x7fdf8011e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf84e56700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 0x7fdf8011e000 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400b2c0 tx=0x7fdf74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7401f070 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf74003970 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf74003d20 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7400b740 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.471+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80121580 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.472+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf74019400 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.472+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf74042af0 con 0x7fdf8009d6f0 2023-09-30T22:30:14.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.472+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf80120ef0 con 0x7fdf8009d6f0 2023-09-30T22:30:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.477+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf7400f3d0 con 0x7fdf8009d6f0 2023-09-30T22:30:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.555+0000 7faed0747700 1 Processor -- start 2023-09-30T22:30:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.555+0000 7faed0747700 1 -- start start 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faed0747700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec80a0a30 0x7faec80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faed0747700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faec809d630 0x7faec80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faed0747700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec80a2a70 con 0x7faec8133930 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faed0747700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec80a01d0 con 0x7faec809d630 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faed0747700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faec809e2b0 con 0x7faec80a0a30 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faec809d630 0x7faec80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faec809d630 0x7faec80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54988/0 (socket says 172.21.15.32:54988) 2023-09-30T22:30:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 -- 172.21.15.32:0/3818488184 learned_addr learned my addr 172.21.15.32:0/3818488184 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faece4e3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec80a0a30 0x7faec80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 -- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec80a0a30 msgr2=0x7faec80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 --2- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec80a0a30 0x7faec80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 -- 172.21.15.32:0/3818488184 >> v1:172.21.15.32:6789/0 conn(0x7faec8133930 legacy=0x7faec8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 -- 172.21.15.32:0/3818488184 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec809dc50 con 0x7faec809d630 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faece4e3700 1 --2- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec80a0a30 0x7faec80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.556+0000 7faecdce2700 1 --2- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec809d630 0x7faec80a1340 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7faeb80098f0 tx=0x7faeb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e27aa8a03ebbb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faecd4e1700 1 -- 172.21.15.32:0/3818488184 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8011830 con 0x7faec809d630 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faecd4e1700 1 -- 172.21.15.32:0/3818488184 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8011990 con 0x7faec809d630 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faecd4e1700 1 -- 172.21.15.32:0/3818488184 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8011d20 con 0x7faec809d630 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faed0747700 1 -- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec809d630 msgr2=0x7faec80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faed0747700 1 --2- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec809d630 0x7faec80a1340 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7faeb80098f0 tx=0x7faeb8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faed0747700 1 -- 172.21.15.32:0/3818488184 shutdown_connections 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faed0747700 1 --2- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec80a0a30 0x7faec80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faed0747700 1 --2- 172.21.15.32:0/3818488184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec809d630 0x7faec80a1340 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.557+0000 7faed0747700 1 -- 172.21.15.32:0/3818488184 >> 172.21.15.32:0/3818488184 conn(0x7faec80a3050 msgr2=0x7faec80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.558+0000 7faed0747700 1 -- 172.21.15.32:0/3818488184 shutdown_connections 2023-09-30T22:30:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.558+0000 7faed0747700 1 -- 172.21.15.32:0/3818488184 wait complete. 2023-09-30T22:30:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.559+0000 7faed0747700 1 Processor -- start 2023-09-30T22:30:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.559+0000 7faed0747700 1 -- start start 2023-09-30T22:30:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faed0747700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec809d630 0x7faec81c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faed0747700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faec80a0a30 0x7faec81c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faed0747700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 0x7faec81c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faed0747700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec8120a70 con 0x7faec8133930 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faed0747700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec8120880 con 0x7faec80a0a30 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faed0747700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faec8121920 con 0x7faec809d630 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faecece4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 0x7faec81c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faecece4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 0x7faec81c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54990/0 (socket says 172.21.15.32:54990) 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faecece4700 1 -- 172.21.15.32:0/3475816131 learned_addr learned my addr 172.21.15.32:0/3475816131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faece4e3700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec809d630 0x7faec81c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faece4e3700 1 -- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faec80a0a30 msgr2=0x7faec81c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faece4e3700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faec80a0a30 0x7faec81c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faece4e3700 1 -- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 msgr2=0x7faec81c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faece4e3700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 0x7faec81c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faece4e3700 1 -- 172.21.15.32:0/3475816131 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009a70 con 0x7faec809d630 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.560+0000 7faecece4700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 0x7faec81c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.561+0000 7faece4e3700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec809d630 0x7faec81c8310 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faec400efc0 tx=0x7faec400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.561+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400cdc0 con 0x7faec809d630 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.561+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec400f040 con 0x7faec809d630 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.561+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec40146c0 con 0x7faec809d630 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.561+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb8009550 con 0x7faec809d630 2023-09-30T22:30:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.561+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec81224b0 con 0x7faec809d630 2023-09-30T22:30:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.562+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faec4014820 con 0x7faec809d630 2023-09-30T22:30:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.562+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec8121e90 con 0x7faec809d630 2023-09-30T22:30:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.562+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faec40403d0 con 0x7faec809d630 2023-09-30T22:30:14.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.567+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faec401d400 con 0x7faec809d630 2023-09-30T22:30:14.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.598+0000 7fa6170b0700 1 Processor -- start 2023-09-30T22:30:14.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.598+0000 7fa6170b0700 1 -- start start 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa6170b0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa61009d630 0x7fa6100a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa6170b0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa610138d80 0x7fa61013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa6170b0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6100a2a70 con 0x7fa6100a0a30 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa6170b0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100a01d0 con 0x7fa610138d80 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa6170b0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa61009e2b0 con 0x7fa61009d630 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa614e4c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6100a0a30 0x7fa6100a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59294/0 (socket says 172.21.15.32:59294) 2023-09-30T22:30:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa614e4c700 1 -- 172.21.15.32:0/1097841239 learned_addr learned my addr 172.21.15.32:0/1097841239 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa61564d700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa610138d80 0x7fa61013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.599+0000 7fa60ffff700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa61009d630 0x7fa6100a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa60f7fe700 1 -- 172.21.15.32:0/1097841239 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830249228 0 0) 0x7fa6100a2a70 con 0x7fa6100a0a30 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa60f7fe700 1 -- 172.21.15.32:0/1097841239 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f80035b0 con 0x7fa6100a0a30 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa61564d700 1 -- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa61009d630 msgr2=0x7fa6100a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa61564d700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa61009d630 0x7fa6100a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa61564d700 1 -- 172.21.15.32:0/1097841239 >> v1:172.21.15.32:6789/0 conn(0x7fa6100a0a30 legacy=0x7fa6100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa61564d700 1 -- 172.21.15.32:0/1097841239 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61009dc50 con 0x7fa610138d80 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa61564d700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa610138d80 0x7fa61013b160 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa608009dc0 tx=0x7fa60800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=736ae5bc90779c9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa60f7fe700 1 -- 172.21.15.32:0/1097841239 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60800e070 con 0x7fa610138d80 2023-09-30T22:30:14.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa60f7fe700 1 -- 172.21.15.32:0/1097841239 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60800bb20 con 0x7fa610138d80 2023-09-30T22:30:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.600+0000 7fa60f7fe700 1 -- 172.21.15.32:0/1097841239 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608012430 con 0x7fa610138d80 2023-09-30T22:30:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 -- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa610138d80 msgr2=0x7fa61013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa610138d80 0x7fa61013b160 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa608009dc0 tx=0x7fa60800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 -- 172.21.15.32:0/1097841239 shutdown_connections 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa610138d80 0x7fa61013b160 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1097841239 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa61009d630 0x7fa6100a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 -- 172.21.15.32:0/1097841239 >> 172.21.15.32:0/1097841239 conn(0x7fa6100a3050 msgr2=0x7fa6100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 -- 172.21.15.32:0/1097841239 shutdown_connections 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.601+0000 7fa6170b0700 1 -- 172.21.15.32:0/1097841239 wait complete. 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.603+0000 7fa6170b0700 1 Processor -- start 2023-09-30T22:30:14.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.603+0000 7fa6170b0700 1 -- start start 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa6170b0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa61009d630 0x7fa6101336b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa6170b0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 0x7fa610133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa6170b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa610138d80 0x7fa610134130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa6170b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6101364a0 con 0x7fa610138d80 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa6170b0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa610136e10 con 0x7fa61009d630 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa6170b0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa61013ab60 con 0x7fa6100a0a30 2023-09-30T22:30:14.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 0x7fa610133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 0x7fa610133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57806/0 (socket says 172.21.15.32:57806) 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 -- 172.21.15.32:0/1293598704 learned_addr learned my addr 172.21.15.32:0/1293598704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa614e4c700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa61009d630 0x7fa6101336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa61564d700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa610138d80 0x7fa610134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 -- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa61009d630 msgr2=0x7fa6101336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa61009d630 0x7fa6101336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 -- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa610138d80 msgr2=0x7fa610134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa610138d80 0x7fa610134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.604+0000 7fa60ffff700 1 -- 172.21.15.32:0/1293598704 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa608009a70 con 0x7fa6100a0a30 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.605+0000 7fa60ffff700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 0x7fa610133bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa60400e9b0 tx=0x7fa60400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.605+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604009710 con 0x7fa6100a0a30 2023-09-30T22:30:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.605+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60400ee50 con 0x7fa6100a0a30 2023-09-30T22:30:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.605+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100a0410 con 0x7fa6100a0a30 2023-09-30T22:30:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.605+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60400ca50 con 0x7fa6100a0a30 2023-09-30T22:30:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.605+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa610137880 con 0x7fa6100a0a30 2023-09-30T22:30:14.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.606+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa60400cbb0 con 0x7fa6100a0a30 2023-09-30T22:30:14.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.606+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa60403f750 con 0x7fa6100a0a30 2023-09-30T22:30:14.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.606+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6100747f0 con 0x7fa6100a0a30 2023-09-30T22:30:14.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.616+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa604021020 con 0x7fa6100a0a30 2023-09-30T22:30:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.696+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdf8011ac20 con 0x7fdf8009d6f0 2023-09-30T22:30:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.697+0000 7fdf7d7fa700 1 -- 172.21.15.32:0/1659923842 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf74019e60 con 0x7fdf8009d6f0 2023-09-30T22:30:14.697 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 msgr2=0x7fdf8011e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 0x7fdf8011e000 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400b2c0 tx=0x7fdf74004750 comp rx=0 tx=0).stop 2023-09-30T22:30:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 shutdown_connections 2023-09-30T22:30:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf8009e410 0x7fdf8011ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8009d6f0 0x7fdf8011e000 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 --2- 172.21.15.32:0/1659923842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8009e000 0x7fdf8011e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 >> 172.21.15.32:0/1659923842 conn(0x7fdf80066f50 msgr2=0x7fdf80130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 shutdown_connections 2023-09-30T22:30:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.699+0000 7fdf870ba700 1 -- 172.21.15.32:0/1659923842 wait complete. 2023-09-30T22:30:14.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.776+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faec8074770 con 0x7faec809d630 2023-09-30T22:30:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.776+0000 7faebeffd700 1 -- 172.21.15.32:0/3475816131 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faec4007330 con 0x7faec809d630 2023-09-30T22:30:14.776 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec809d630 msgr2=0x7faec81c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec809d630 0x7faec81c8310 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faec400efc0 tx=0x7faec400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 shutdown_connections 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faec8133930 0x7faec81c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faec809d630 0x7faec81c8310 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 --2- 172.21.15.32:0/3475816131 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faec80a0a30 0x7faec81c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.778+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 >> 172.21.15.32:0/3475816131 conn(0x7faec80a3050 msgr2=0x7faec81bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.779+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 shutdown_connections 2023-09-30T22:30:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.779+0000 7faed0747700 1 -- 172.21.15.32:0/3475816131 wait complete. 2023-09-30T22:30:14.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:14.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.822+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa610072980 con 0x7fa6100a0a30 2023-09-30T22:30:14.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.823+0000 7fa60d7fa700 1 -- 172.21.15.32:0/1293598704 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa60401be10 con 0x7fa6100a0a30 2023-09-30T22:30:14.823 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 msgr2=0x7fa610133bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 0x7fa610133bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa60400e9b0 tx=0x7fa60400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 shutdown_connections 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa610138d80 0x7fa610134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa61009d630 0x7fa6101336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 --2- 172.21.15.32:0/1293598704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa6100a0a30 0x7fa610133bf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 >> 172.21.15.32:0/1293598704 conn(0x7fa6100a3050 msgr2=0x7fa61009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 shutdown_connections 2023-09-30T22:30:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:14.825+0000 7fa6170b0700 1 -- 172.21.15.32:0/1293598704 wait complete. 2023-09-30T22:30:14.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:15.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.543+0000 7f3ddc8dd700 1 Processor -- start 2023-09-30T22:30:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.543+0000 7f3ddc8dd700 1 -- start start 2023-09-30T22:30:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3ddc8dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 0x7f3dd81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3ddc8dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133e30 0x7f3dd812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3ddc8dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3dd8134260 0x7f3dd81304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3ddc8dd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd806b990 con 0x7f3dd8134260 2023-09-30T22:30:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3ddc8dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80a0590 con 0x7f3dd8133420 2023-09-30T22:30:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3ddc8dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd809fb80 con 0x7f3dd8133e30 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 0x7f3dd81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133e30 0x7f3dd812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3dd8134260 0x7f3dd81304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 0x7f3dd81337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42920/0 (socket says 172.21.15.32:42920) 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3dd8134260 0x7f3dd81304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55012/0 (socket says 172.21.15.32:55012) 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd5d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133e30 0x7f3dd812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57814/0 (socket says 172.21.15.32:57814) 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.544+0000 7f3dd659c700 1 -- 172.21.15.32:0/876045038 learned_addr learned my addr 172.21.15.32:0/876045038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd6d9d700 1 -- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133e30 msgr2=0x7f3dd812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd6d9d700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133e30 0x7f3dd812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd6d9d700 1 -- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 msgr2=0x7f3dd81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd6d9d700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 0x7f3dd81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd6d9d700 1 -- 172.21.15.32:0/876045038 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd80a0030 con 0x7f3dd8134260 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd659c700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 0x7f3dd81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd6d9d700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8134260 0x7f3dd81304d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009d90 tx=0x7f3dd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7910c9a6960c420e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd559a700 1 -- 172.21.15.32:0/876045038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000e070 con 0x7f3dd8134260 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd559a700 1 -- 172.21.15.32:0/876045038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd000bb20 con 0x7f3dd8134260 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.545+0000 7f3dd559a700 1 -- 172.21.15.32:0/876045038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0012430 con 0x7f3dd8134260 2023-09-30T22:30:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8134260 msgr2=0x7f3dd81304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8134260 0x7f3dd81304d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009d90 tx=0x7f3dd000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/876045038 shutdown_connections 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8133420 0x7f3dd81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8134260 0x7f3dd81304d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/876045038 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133e30 0x7f3dd812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/876045038 >> 172.21.15.32:0/876045038 conn(0x7f3dd80a4210 msgr2=0x7f3dd80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/876045038 shutdown_connections 2023-09-30T22:30:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.546+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/876045038 wait complete. 2023-09-30T22:30:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 Processor -- start 2023-09-30T22:30:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 -- start start 2023-09-30T22:30:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133420 0x7f3dd812d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8133e30 0x7f3dd809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8134260 0x7f3dd809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd811ff00 con 0x7f3dd8133e30 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd8120270 con 0x7f3dd8134260 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.547+0000 7f3ddc8dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd81206c0 con 0x7f3dd8133420 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8134260 0x7f3dd809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133420 0x7f3dd812d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd6d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8134260 0x7f3dd809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42922/0 (socket says 172.21.15.32:42922) 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd6d9d700 1 -- 172.21.15.32:0/3616662002 learned_addr learned my addr 172.21.15.32:0/3616662002 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd5d9b700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8133e30 0x7f3dd809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 -- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8134260 msgr2=0x7f3dd809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8134260 0x7f3dd809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 -- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8133e30 msgr2=0x7f3dd809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8133e30 0x7f3dd809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 -- 172.21.15.32:0/3616662002 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8009550 con 0x7f3dd8133420 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd5d9b700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8133e30 0x7f3dd809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dd659c700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133420 0x7f3dd812d8b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800e9a0 tx=0x7f3dc800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004030 con 0x7f3dd8133420 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8004190 con 0x7f3dd8133420 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd0009a70 con 0x7f3dd8133420 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004520 con 0x7f3dd8133420 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.548+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd811efa0 con 0x7f3dd8133420 2023-09-30T22:30:15.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.549+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3dc80036a0 con 0x7f3dd8133420 2023-09-30T22:30:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.550+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3dc803f870 con 0x7f3dd8133420 2023-09-30T22:30:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.550+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd811fd40 con 0x7f3dd8133420 2023-09-30T22:30:15.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.554+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3dc8020020 con 0x7f3dd8133420 2023-09-30T22:30:15.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:15.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.762+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3dd8074ea0 con 0x7f3dd8133420 2023-09-30T22:30:15.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.762+0000 7f3dceffd700 1 -- 172.21.15.32:0/3616662002 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3dc8011b80 con 0x7f3dd8133420 2023-09-30T22:30:15.763 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133420 msgr2=0x7f3dd812d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133420 0x7f3dd812d8b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800e9a0 tx=0x7f3dc800ecb0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 shutdown_connections 2023-09-30T22:30:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3dd8134260 0x7f3dd809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3dd8133420 0x7f3dd812d8b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 --2- 172.21.15.32:0/3616662002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dd8133e30 0x7f3dd809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 >> 172.21.15.32:0/3616662002 conn(0x7f3dd80a4210 msgr2=0x7f3dd80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 shutdown_connections 2023-09-30T22:30:15.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.765+0000 7f3ddc8dd700 1 -- 172.21.15.32:0/3616662002 wait complete. 2023-09-30T22:30:15.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:15.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:15.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:15.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.854+0000 7fe798062700 1 Processor -- start 2023-09-30T22:30:15.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe798062700 1 -- start start 2023-09-30T22:30:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe798062700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009d6f0 0x7fe79009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe798062700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe79009e000 0x7fe7901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe798062700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79009b120 con 0x7fe79009e410 2023-09-30T22:30:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe798062700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79006b460 con 0x7fe79009e000 2023-09-30T22:30:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe798062700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79009a7e0 con 0x7fe79009d6f0 2023-09-30T22:30:15.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe7955fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe79009e000 0x7fe7901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe7965ff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe79009e410 0x7fe790133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59296/0 (socket says 172.21.15.32:59296) 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe7965ff700 1 -- 172.21.15.32:0/3614313773 learned_addr learned my addr 172.21.15.32:0/3614313773 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe7955fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe79009e000 0x7fe7901339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55040/0 (socket says 172.21.15.32:55040) 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.855+0000 7fe795dfe700 1 --2- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009d6f0 0x7fe79009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe7955fd700 1 -- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009d6f0 msgr2=0x7fe79009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe7955fd700 1 --2- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009d6f0 0x7fe79009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe7955fd700 1 -- 172.21.15.32:0/3614313773 >> v1:172.21.15.32:6789/0 conn(0x7fe79009e410 legacy=0x7fe790133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe7955fd700 1 -- 172.21.15.32:0/3614313773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79009a500 con 0x7fe79009e000 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe7955fd700 1 --2- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009e000 0x7fe7901339f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fe780009840 tx=0x7fe780009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bb6fe4fe71d345 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe794dfc700 1 -- 172.21.15.32:0/3614313773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780011830 con 0x7fe79009e000 2023-09-30T22:30:15.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe794dfc700 1 -- 172.21.15.32:0/3614313773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe780011990 con 0x7fe79009e000 2023-09-30T22:30:15.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.856+0000 7fe794dfc700 1 -- 172.21.15.32:0/3614313773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780011d20 con 0x7fe79009e000 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 -- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009e000 msgr2=0x7fe7901339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 --2- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009e000 0x7fe7901339f0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fe780009840 tx=0x7fe780009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 -- 172.21.15.32:0/3614313773 shutdown_connections 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 --2- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009d6f0 0x7fe79009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 --2- 172.21.15.32:0/3614313773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009e000 0x7fe7901339f0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 -- 172.21.15.32:0/3614313773 >> 172.21.15.32:0/3614313773 conn(0x7fe790066f50 msgr2=0x7fe79009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 -- 172.21.15.32:0/3614313773 shutdown_connections 2023-09-30T22:30:15.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.857+0000 7fe798062700 1 -- 172.21.15.32:0/3614313773 wait complete. 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.858+0000 7fe798062700 1 Processor -- start 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.858+0000 7fe798062700 1 -- start start 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe798062700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009d6f0 0x7fe7901a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe798062700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 0x7fe7901a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe798062700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe79009e410 0x7fe7901a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe798062700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7901a7b30 con 0x7fe79009d6f0 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe798062700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7901a8150 con 0x7fe79009e000 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe798062700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7901a8d80 con 0x7fe79009e410 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe7955fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 0x7fe7901a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe7965ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe79009e410 0x7fe7901a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe7955fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 0x7fe7901a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42936/0 (socket says 172.21.15.32:42936) 2023-09-30T22:30:15.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe7955fd700 1 -- 172.21.15.32:0/4164378603 learned_addr learned my addr 172.21.15.32:0/4164378603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:15.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe795dfe700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009d6f0 0x7fe7901a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe795dfe700 1 -- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe79009e410 msgr2=0x7fe7901a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe795dfe700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe79009e410 0x7fe7901a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe795dfe700 1 -- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 msgr2=0x7fe7901a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe795dfe700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 0x7fe7901a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe795dfe700 1 -- 172.21.15.32:0/4164378603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800094a0 con 0x7fe79009d6f0 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.859+0000 7fe7955fd700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 0x7fe7901a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.860+0000 7fe795dfe700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009d6f0 0x7fe7901a5fb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00e930 tx=0x7fe78c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.860+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c013680 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.860+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c0137e0 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.860+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c013b50 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.860+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7901a91d0 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.860+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7900a2fd0 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.861+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe790074ea0 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.861+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe78c013cb0 con 0x7fe79009d6f0 2023-09-30T22:30:15.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.861+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe78c03fa00 con 0x7fe79009d6f0 2023-09-30T22:30:15.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.871+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe78c01abe0 con 0x7fe79009d6f0 2023-09-30T22:30:15.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.938+0000 7fad80942700 1 Processor -- start 2023-09-30T22:30:15.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.938+0000 7fad80942700 1 -- start start 2023-09-30T22:30:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad80942700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad80942700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad7c138d80 0x7fad7c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad80942700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c0a2a70 con 0x7fad7c09d630 2023-09-30T22:30:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad80942700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7c0a01d0 con 0x7fad7c138d80 2023-09-30T22:30:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad80942700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c09e2b0 con 0x7fad7c0a0a30 2023-09-30T22:30:15.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57842/0 (socket says 172.21.15.32:57842) 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad7a59c700 1 -- 172.21.15.32:0/1061311991 learned_addr learned my addr 172.21.15.32:0/1061311991 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.939+0000 7fad7ad9d700 1 --2- 172.21.15.32:0/1061311991 >> v2:172.21.15.32:3300/0 conn(0x7fad7c138d80 0x7fad7c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7a59c700 1 -- 172.21.15.32:0/1061311991 >> v2:172.21.15.32:3300/0 conn(0x7fad7c138d80 msgr2=0x7fad7c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7a59c700 1 --2- 172.21.15.32:0/1061311991 >> v2:172.21.15.32:3300/0 conn(0x7fad7c138d80 0x7fad7c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7959a700 1 -- 172.21.15.32:0/1061311991 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276519422 0 0) 0x7fad7c0a2a70 con 0x7fad7c09d630 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7a59c700 1 -- 172.21.15.32:0/1061311991 >> v1:172.21.15.32:6789/0 conn(0x7fad7c09d630 legacy=0x7fad7c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7a59c700 1 -- 172.21.15.32:0/1061311991 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c09dc50 con 0x7fad7c0a0a30 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7ad9d700 1 --2- 172.21.15.32:0/1061311991 >> v2:172.21.15.32:3300/0 conn(0x7fad7c138d80 0x7fad7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7a59c700 1 --2- 172.21.15.32:0/1061311991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c0a0e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0098f0 tx=0x7fad6c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5a98eeea72a138f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7959a700 1 -- 172.21.15.32:0/1061311991 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00b8a0 con 0x7fad7c0a0a30 2023-09-30T22:30:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7959a700 1 -- 172.21.15.32:0/1061311991 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c00ba00 con 0x7fad7c0a0a30 2023-09-30T22:30:15.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.940+0000 7fad7959a700 1 -- 172.21.15.32:0/1061311991 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00bd90 con 0x7fad7c0a0a30 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 -- 172.21.15.32:0/1061311991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 msgr2=0x7fad7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 --2- 172.21.15.32:0/1061311991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c0a0e00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0098f0 tx=0x7fad6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 -- 172.21.15.32:0/1061311991 shutdown_connections 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 --2- 172.21.15.32:0/1061311991 >> v2:172.21.15.32:3300/0 conn(0x7fad7c138d80 0x7fad7c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 --2- 172.21.15.32:0/1061311991 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c0a0e00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 -- 172.21.15.32:0/1061311991 >> 172.21.15.32:0/1061311991 conn(0x7fad7c0a3050 msgr2=0x7fad7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 -- 172.21.15.32:0/1061311991 shutdown_connections 2023-09-30T22:30:15.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.941+0000 7fad80942700 1 -- 172.21.15.32:0/1061311991 wait complete. 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.942+0000 7fad80942700 1 Processor -- start 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.942+0000 7fad80942700 1 -- start start 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad80942700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 0x7fad7c133720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad80942700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c133c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad80942700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad7c138d80 0x7fad7c1341a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad80942700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c137240 con 0x7fad7c09d630 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad80942700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c136580 con 0x7fad7c138d80 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad80942700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c13ae00 con 0x7fad7c0a0a30 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 0x7fad7c133720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 0x7fad7c133720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55064/0 (socket says 172.21.15.32:55064) 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 -- 172.21.15.32:0/1184347804 learned_addr learned my addr 172.21.15.32:0/1184347804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad79d9b700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 -- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 msgr2=0x7fad7c133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 -- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad7c138d80 msgr2=0x7fad7c1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad7c138d80 0x7fad7c1341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 -- 172.21.15.32:0/1184347804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c009550 con 0x7fad7c09d630 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad79d9b700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.943+0000 7fad7a59c700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 0x7fad7c133720 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fad7c09e3f0 tx=0x7fad6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.944+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004300 con 0x7fad7c09d630 2023-09-30T22:30:15.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.944+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c01f070 con 0x7fad7c09d630 2023-09-30T22:30:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.944+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0036a0 con 0x7fad7c09d630 2023-09-30T22:30:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.944+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c135b90 con 0x7fad7c09d630 2023-09-30T22:30:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.944+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c136ca0 con 0x7fad7c09d630 2023-09-30T22:30:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.945+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c136a90 con 0x7fad7c09d630 2023-09-30T22:30:15.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.945+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad6c021030 con 0x7fad7c09d630 2023-09-30T22:30:15.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.945+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fad6c0409e0 con 0x7fad7c09d630 2023-09-30T22:30:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.954+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad6c023990 con 0x7fad7c09d630 2023-09-30T22:30:15.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 Processor -- start 2023-09-30T22:30:15.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 -- start start 2023-09-30T22:30:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 0x7f12540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f125409d630 0x7f12540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1254133930 0x7f1254133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540a2a70 con 0x7f1254133930 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540a01d0 con 0x7f125409d630 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.995+0000 7f125b944700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f125409e2b0 con 0x7f12540a0a30 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 0x7f12540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 0x7f12540a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57868/0 (socket says 172.21.15.32:57868) 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 -- 172.21.15.32:0/3995781473 learned_addr learned my addr 172.21.15.32:0/3995781473 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f1258edf700 1 --2- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f125409d630 0x7f12540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f1259ee1700 1 --2- 172.21.15.32:0/3995781473 >> v2:172.21.15.32:3300/0 conn(0x7f1254133930 0x7f1254133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 -- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f125409d630 msgr2=0x7f12540a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 --2- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f125409d630 0x7f12540a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 -- 172.21.15.32:0/3995781473 >> v2:172.21.15.32:3300/0 conn(0x7f1254133930 msgr2=0x7f1254133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 --2- 172.21.15.32:0/3995781473 >> v2:172.21.15.32:3300/0 conn(0x7f1254133930 0x7f1254133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 -- 172.21.15.32:0/3995781473 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125409dc50 con 0x7f12540a0a30 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f1259ee1700 1 --2- 172.21.15.32:0/3995781473 >> v2:172.21.15.32:3300/0 conn(0x7f1254133930 0x7f1254133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f12596e0700 1 --2- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 0x7f12540a0e00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1250009e50 tx=0x7f125000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da24e583f025f8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f124bfff700 1 -- 172.21.15.32:0/3995781473 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000e070 con 0x7f12540a0a30 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f124bfff700 1 -- 172.21.15.32:0/3995781473 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125000b910 con 0x7f12540a0a30 2023-09-30T22:30:15.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.996+0000 7f124bfff700 1 -- 172.21.15.32:0/3995781473 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000bcc0 con 0x7f12540a0a30 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 -- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 msgr2=0x7f12540a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 --2- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 0x7f12540a0e00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1250009e50 tx=0x7f125000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 -- 172.21.15.32:0/3995781473 shutdown_connections 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 --2- 172.21.15.32:0/3995781473 >> v2:172.21.15.32:3300/0 conn(0x7f1254133930 0x7f1254133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 --2- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f12540a0a30 0x7f12540a0e00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 --2- 172.21.15.32:0/3995781473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f125409d630 0x7f12540a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 -- 172.21.15.32:0/3995781473 >> 172.21.15.32:0/3995781473 conn(0x7f12540a3050 msgr2=0x7f12540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 -- 172.21.15.32:0/3995781473 shutdown_connections 2023-09-30T22:30:16.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.997+0000 7f125b944700 1 -- 172.21.15.32:0/3995781473 wait complete. 2023-09-30T22:30:16.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.998+0000 7f125b944700 1 Processor -- start 2023-09-30T22:30:16.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.998+0000 7f125b944700 1 -- start start 2023-09-30T22:30:16.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f125b944700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 0x7f12541c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f125b944700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540a0a30 0x7f12541c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f125b944700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1254133930 0x7f12541c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f125b944700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1254122430 con 0x7f12540a0a30 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f125b944700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1254120720 con 0x7f1254133930 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f125b944700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12540a0410 con 0x7f125409d630 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 0x7f12541c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 0x7f12541c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57870/0 (socket says 172.21.15.32:57870) 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 -- 172.21.15.32:0/558014169 learned_addr learned my addr 172.21.15.32:0/558014169 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f1258edf700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540a0a30 0x7f12541c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f1259ee1700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1254133930 0x7f12541c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 -- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1254133930 msgr2=0x7f12541c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1254133930 0x7f12541c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 -- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540a0a30 msgr2=0x7f12541c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540a0a30 0x7f12541c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 -- 172.21.15.32:0/558014169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250009b30 con 0x7f125409d630 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:15.999+0000 7f12596e0700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 0x7f12541c81a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f125409e3f0 tx=0x7f1250004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.000+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500092e0 con 0x7f125409d630 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.000+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1250004250 con 0x7f125409d630 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.000+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12540a41c0 con 0x7f125409d630 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.000+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250004600 con 0x7f125409d630 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.000+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12541335c0 con 0x7f125409d630 2023-09-30T22:30:16.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.001+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1250004760 con 0x7f125409d630 2023-09-30T22:30:16.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.001+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12540747f0 con 0x7f125409d630 2023-09-30T22:30:16.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.001+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1250040e60 con 0x7f125409d630 2023-09-30T22:30:16.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.006+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12500117a0 con 0x7f125409d630 2023-09-30T22:30:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.084+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe79006c360 con 0x7fe79009d6f0 2023-09-30T22:30:16.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.084+0000 7fe7867fc700 1 -- 172.21.15.32:0/4164378603 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe78c017070 con 0x7fe79009d6f0 2023-09-30T22:30:16.084 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:16.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009d6f0 msgr2=0x7fe7901a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009d6f0 0x7fe7901a5fb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00e930 tx=0x7fe78c00ec40 comp rx=0 tx=0).stop 2023-09-30T22:30:16.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 shutdown_connections 2023-09-30T22:30:16.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe79009e410 0x7fe7901a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe79009d6f0 0x7fe7901a5fb0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 --2- 172.21.15.32:0/4164378603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe79009e000 0x7fe7901a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 >> 172.21.15.32:0/4164378603 conn(0x7fe790066f50 msgr2=0x7fe7901baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.086+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 shutdown_connections 2023-09-30T22:30:16.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.087+0000 7fe798062700 1 -- 172.21.15.32:0/4164378603 wait complete. 2023-09-30T22:30:16.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.178+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fad7c0747f0 con 0x7fad7c09d630 2023-09-30T22:30:16.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.178+0000 7fad6affd700 1 -- 172.21.15.32:0/1184347804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad6c015070 con 0x7fad7c09d630 2023-09-30T22:30:16.178 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:16.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 msgr2=0x7fad7c133720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 0x7fad7c133720 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fad7c09e3f0 tx=0x7fad6c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 shutdown_connections 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fad7c138d80 0x7fad7c1341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad7c09d630 0x7fad7c133720 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 --2- 172.21.15.32:0/1184347804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fad7c0a0a30 0x7fad7c133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.180+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 >> 172.21.15.32:0/1184347804 conn(0x7fad7c0a3050 msgr2=0x7fad7c0a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.181+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 shutdown_connections 2023-09-30T22:30:16.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.181+0000 7fad80942700 1 -- 172.21.15.32:0/1184347804 wait complete. 2023-09-30T22:30:16.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:16.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.214+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12540a4f80 con 0x7f125409d630 2023-09-30T22:30:16.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.214+0000 7f1249ffb700 1 -- 172.21.15.32:0/558014169 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12500243c0 con 0x7f125409d630 2023-09-30T22:30:16.214 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.216+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 msgr2=0x7f12541c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.216+0000 7f125b944700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 0x7f12541c81a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f125409e3f0 tx=0x7f1250004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 shutdown_connections 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1254133930 0x7f12541c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f125409d630 0x7f12541c81a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 --2- 172.21.15.32:0/558014169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12540a0a30 0x7f12541c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 >> 172.21.15.32:0/558014169 conn(0x7f12540a3050 msgr2=0x7f12541bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 shutdown_connections 2023-09-30T22:30:16.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.217+0000 7f125b944700 1 -- 172.21.15.32:0/558014169 wait complete. 2023-09-30T22:30:16.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:16.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 Processor -- start 2023-09-30T22:30:16.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 -- start start 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac09f130 con 0x7fc5ac09e000 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac09f520 con 0x7fc5ac09d6f0 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.927+0000 7fc5b21ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac09e930 con 0x7fc5ac09e410 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5ab7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5aaffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5ac09e000 0x7fc5ac130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59318/0 (socket says 172.21.15.32:59318) 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5aaffd700 1 -- 172.21.15.32:0/1952375557 learned_addr learned my addr 172.21.15.32:0/1952375557 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:16.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5aa7fc700 1 -- 172.21.15.32:0/1952375557 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184020644 0 0) 0x7fc5ac09f130 con 0x7fc5ac09e000 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5abfff700 1 -- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 msgr2=0x7fc5ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5abfff700 1 --2- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5abfff700 1 -- 172.21.15.32:0/1952375557 >> v1:172.21.15.32:6789/0 conn(0x7fc5ac09e000 legacy=0x7fc5ac130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5abfff700 1 -- 172.21.15.32:0/1952375557 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac09b2d0 con 0x7fc5ac09e410 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5ab7fe700 1 --2- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5abfff700 1 --2- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac130d70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009dc0 tx=0x7fc5a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c2b5b96548e597 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5aa7fc700 1 -- 172.21.15.32:0/1952375557 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400e070 con 0x7fc5ac09e410 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5aa7fc700 1 -- 172.21.15.32:0/1952375557 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a400bb20 con 0x7fc5ac09e410 2023-09-30T22:30:16.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.928+0000 7fc5aa7fc700 1 -- 172.21.15.32:0/1952375557 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4012430 con 0x7fc5ac09e410 2023-09-30T22:30:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 msgr2=0x7fc5ac130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac130d70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009dc0 tx=0x7fc5a400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1952375557 shutdown_connections 2023-09-30T22:30:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac130d70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1952375557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1952375557 >> 172.21.15.32:0/1952375557 conn(0x7fc5ac066f50 msgr2=0x7fc5ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1952375557 shutdown_connections 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.929+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1952375557 wait complete. 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.930+0000 7fc5b21ee700 1 Processor -- start 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.930+0000 7fc5b21ee700 1 -- start start 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5b21ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac137370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5b21ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 0x7fc5ac1378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5b21ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac137df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5b21ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac138840 con 0x7fc5ac09e000 2023-09-30T22:30:16.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5b21ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac119600 con 0x7fc5ac09d6f0 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5b21ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac11a1c0 con 0x7fc5ac09e410 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 0x7fc5ac1378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 0x7fc5ac1378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55094/0 (socket says 172.21.15.32:55094) 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5ab7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5aaffd700 1 -- 172.21.15.32:0/1497508416 learned_addr learned my addr 172.21.15.32:0/1497508416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 -- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 msgr2=0x7fc5ac137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 -- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 msgr2=0x7fc5ac1378b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 0x7fc5ac1378b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 -- 172.21.15.32:0/1497508416 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0094a0 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5ab7fe700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5aaffd700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 0x7fc5ac1378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.931+0000 7fc5abfff700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac137df0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4016000 tx=0x7fc5a4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.932+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4004880 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.932+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a4020070 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.932+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a4009a70 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.932+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40276c0 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.932+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac0a2070 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.932+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5a4012430 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.933+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc5a4022070 con 0x7fc5ac09e410 2023-09-30T22:30:16.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.933+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac070590 con 0x7fc5ac09e410 2023-09-30T22:30:16.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:16.938+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5a401b400 con 0x7fc5ac09e410 2023-09-30T22:30:17.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.145+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc5ac13a920 con 0x7fc5ac09e410 2023-09-30T22:30:17.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.145+0000 7fc593fff700 1 -- 172.21.15.32:0/1497508416 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5a402c020 con 0x7fc5ac09e410 2023-09-30T22:30:17.146 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 msgr2=0x7fc5ac137df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac137df0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4016000 tx=0x7fc5a4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 shutdown_connections 2023-09-30T22:30:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc5ac09e410 0x7fc5ac137df0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc5ac09d6f0 0x7fc5ac137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 --2- 172.21.15.32:0/1497508416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5ac09e000 0x7fc5ac1378b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 >> 172.21.15.32:0/1497508416 conn(0x7fc5ac066f50 msgr2=0x7fc5ac130980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 shutdown_connections 2023-09-30T22:30:17.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.148+0000 7fc5b21ee700 1 -- 172.21.15.32:0/1497508416 wait complete. 2023-09-30T22:30:17.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:17.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:17.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:17.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.241+0000 7f0bf8103700 1 Processor -- start 2023-09-30T22:30:17.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.241+0000 7f0bf8103700 1 -- start start 2023-09-30T22:30:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf8103700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf8103700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0bf009f240 0x7f0bf01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf8103700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf0070940 con 0x7f0bf009f650 2023-09-30T22:30:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf8103700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf006fd60 con 0x7f0bf009f240 2023-09-30T22:30:17.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf8103700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf006fa80 con 0x7f0bf009e930 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42996/0 (socket says 172.21.15.32:42996) 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 -- 172.21.15.32:0/1711629271 learned_addr learned my addr 172.21.15.32:0/1711629271 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf66a0700 1 --1- 172.21.15.32:0/1711629271 >> v1:172.21.15.32:6789/0 conn(0x7f0bf009f650 0x7f0bf0131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59332/0 (socket says 172.21.15.32:59332) 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf569e700 1 --2- 172.21.15.32:0/1711629271 >> v2:172.21.15.32:3300/0 conn(0x7f0bf009f240 0x7f0bf01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 -- 172.21.15.32:0/1711629271 >> v2:172.21.15.32:3300/0 conn(0x7f0bf009f240 msgr2=0x7f0bf01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 --2- 172.21.15.32:0/1711629271 >> v2:172.21.15.32:3300/0 conn(0x7f0bf009f240 0x7f0bf01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 -- 172.21.15.32:0/1711629271 >> v1:172.21.15.32:6789/0 conn(0x7f0bf009f650 legacy=0x7f0bf0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf5e9f700 1 -- 172.21.15.32:0/1711629271 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf009fb60 con 0x7f0bf009e930 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.242+0000 7f0bf569e700 1 --2- 172.21.15.32:0/1711629271 >> v2:172.21.15.32:3300/0 conn(0x7f0bf009f240 0x7f0bf01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.243+0000 7f0bf5e9f700 1 --2- 172.21.15.32:0/1711629271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf009ed00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0098f0 tx=0x7f0bec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb4183df2797d57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.243+0000 7f0bf4e9d700 1 -- 172.21.15.32:0/1711629271 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec011830 con 0x7f0bf009e930 2023-09-30T22:30:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.243+0000 7f0bf4e9d700 1 -- 172.21.15.32:0/1711629271 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec011990 con 0x7f0bf009e930 2023-09-30T22:30:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.243+0000 7f0bf4e9d700 1 -- 172.21.15.32:0/1711629271 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec011d20 con 0x7f0bf009e930 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 -- 172.21.15.32:0/1711629271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 msgr2=0x7f0bf009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 --2- 172.21.15.32:0/1711629271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf009ed00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0098f0 tx=0x7f0bec009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 -- 172.21.15.32:0/1711629271 shutdown_connections 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 --2- 172.21.15.32:0/1711629271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf009ed00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 --2- 172.21.15.32:0/1711629271 >> v2:172.21.15.32:3300/0 conn(0x7f0bf009f240 0x7f0bf01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 -- 172.21.15.32:0/1711629271 >> 172.21.15.32:0/1711629271 conn(0x7f0bf00a4210 msgr2=0x7f0bf00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 -- 172.21.15.32:0/1711629271 shutdown_connections 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.244+0000 7f0bf8103700 1 -- 172.21.15.32:0/1711629271 wait complete. 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.245+0000 7f0bf8103700 1 Processor -- start 2023-09-30T22:30:17.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 -- start start 2023-09-30T22:30:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf01a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 0x7f0bf01a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf009f650 0x7f0bf01a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf01a51b0 con 0x7f0bf009f650 2023-09-30T22:30:17.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf01a5910 con 0x7f0bf009e930 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.246+0000 7f0bf8103700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf01a6b60 con 0x7f0bf009f240 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 0x7f0bf01a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf5e9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf01a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf66a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf009f650 0x7f0bf01a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 0x7f0bf01a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57900/0 (socket says 172.21.15.32:57900) 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf5e9f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf01a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43002/0 (socket says 172.21.15.32:43002) 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 -- 172.21.15.32:0/2606895542 learned_addr learned my addr 172.21.15.32:0/2606895542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 -- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 msgr2=0x7f0bf01a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf01a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 -- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf009f650 msgr2=0x7f0bf01a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf009f650 0x7f0bf01a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 -- 172.21.15.32:0/2606895542 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be00094a0 con 0x7f0bf009f240 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf569e700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 0x7f0bf01a4240 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0be000db70 tx=0x7f0be000de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000b840 con 0x7f0bf009f240 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be000b9a0 con 0x7f0bf009f240 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.247+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec009550 con 0x7f0bf009f240 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.248+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000bd30 con 0x7f0bf009f240 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.248+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf00a0190 con 0x7f0bf009f240 2023-09-30T22:30:17.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.249+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0be0012400 con 0x7f0bf009f240 2023-09-30T22:30:17.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.249+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0be003f3e0 con 0x7f0bf009f240 2023-09-30T22:30:17.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.249+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf01a0b30 con 0x7f0bf009f240 2023-09-30T22:30:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.254+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0be0020070 con 0x7f0bf009f240 2023-09-30T22:30:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.347+0000 7f24e32e4700 1 Processor -- start 2023-09-30T22:30:17.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.347+0000 7f24e32e4700 1 -- start start 2023-09-30T22:30:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.347+0000 7f24e32e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc0a0090 0x7f24dc136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.347+0000 7f24e32e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24dc09d130 0x7f24dc139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.347+0000 7f24e32e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24dc09ba70 con 0x7f24dc09c650 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e32e4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24dc09e8f0 con 0x7f24dc09d130 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e32e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc0a0760 con 0x7f24dc0a0090 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24dc09d130 0x7f24dc139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e087f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc0a0090 0x7f24dc136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24dc09d130 0x7f24dc139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55112/0 (socket says 172.21.15.32:55112) 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 -- 172.21.15.32:0/1601217575 learned_addr learned my addr 172.21.15.32:0/1601217575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:17.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 -- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc0a0090 msgr2=0x7f24dc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 --2- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc0a0090 0x7f24dc136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 -- 172.21.15.32:0/1601217575 >> v1:172.21.15.32:6789/0 conn(0x7f24dc09c650 legacy=0x7f24dc09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 -- 172.21.15.32:0/1601217575 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc09a080 con 0x7f24dc09d130 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e087f700 1 --2- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc0a0090 0x7f24dc136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.348+0000 7f24e1881700 1 --2- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc09d130 0x7f24dc139170 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009e50 tx=0x7f24d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b818c16be224f73f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24d3fff700 1 -- 172.21.15.32:0/1601217575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400e070 con 0x7f24dc09d130 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24d3fff700 1 -- 172.21.15.32:0/1601217575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d400b910 con 0x7f24dc09d130 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24d3fff700 1 -- 172.21.15.32:0/1601217575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400bcc0 con 0x7f24dc09d130 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 -- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc09d130 msgr2=0x7f24dc139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 --2- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc09d130 0x7f24dc139170 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009e50 tx=0x7f24d400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 -- 172.21.15.32:0/1601217575 shutdown_connections 2023-09-30T22:30:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 --2- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc09d130 0x7f24dc139170 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 --2- 172.21.15.32:0/1601217575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc0a0090 0x7f24dc136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 -- 172.21.15.32:0/1601217575 >> 172.21.15.32:0/1601217575 conn(0x7f24dc0a3f70 msgr2=0x7f24dc0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 -- 172.21.15.32:0/1601217575 shutdown_connections 2023-09-30T22:30:17.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.349+0000 7f24e32e4700 1 -- 172.21.15.32:0/1601217575 wait complete. 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 Processor -- start 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 -- start start 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24dc09c650 0x7f24dc12db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 0x7f24dc12e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc0a0090 0x7f24dc12e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc11ce30 con 0x7f24dc0a0090 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc11d7a0 con 0x7f24dc09c650 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.351+0000 7f24e32e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc130b30 con 0x7f24dc09d130 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 0x7f24dc12e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e1080700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24dc09c650 0x7f24dc12db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 0x7f24dc12e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57918/0 (socket says 172.21.15.32:57918) 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 -- 172.21.15.32:0/427762025 learned_addr learned my addr 172.21.15.32:0/427762025 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e1881700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc0a0090 0x7f24dc12e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 -- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24dc09c650 msgr2=0x7f24dc12db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24dc09c650 0x7f24dc12db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 -- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc0a0090 msgr2=0x7f24dc12e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc0a0090 0x7f24dc12e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 -- 172.21.15.32:0/427762025 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0094a0 con 0x7f24dc09d130 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e1881700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc0a0090 0x7f24dc12e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24e087f700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 0x7f24dc12e050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ebe0 tx=0x7f24cc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00c7f0 con 0x7f24dc09d130 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.352+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00c950 con 0x7f24dc09d130 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.353+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00cce0 con 0x7f24dc09d130 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.353+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d4009b30 con 0x7f24dc09d130 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.353+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc11c890 con 0x7f24dc09d130 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.354+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24dc0747f0 con 0x7f24dc09d130 2023-09-30T22:30:17.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.354+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24cc013470 con 0x7f24dc09d130 2023-09-30T22:30:17.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.354+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f24cc03fae0 con 0x7f24dc09d130 2023-09-30T22:30:17.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.359+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24cc013720 con 0x7f24dc09d130 2023-09-30T22:30:17.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.383+0000 7fa3687df700 1 Processor -- start 2023-09-30T22:30:17.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.383+0000 7fa3687df700 1 -- start start 2023-09-30T22:30:17.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.384+0000 7fa3687df700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3601337f0 0x7fa360138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.384+0000 7fa3687df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa360138dc0 0x7fa36013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.384+0000 7fa3687df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36006b990 con 0x7fa360133420 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.384+0000 7fa3687df700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36009b0e0 con 0x7fa3601337f0 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.384+0000 7fa3687df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa360133c20 con 0x7fa360138dc0 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365d7a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3601337f0 0x7fa360138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa36657b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa360133420 0x7fa360133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59338/0 (socket says 172.21.15.32:59338) 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa36657b700 1 -- 172.21.15.32:0/2155810235 learned_addr learned my addr 172.21.15.32:0/2155810235 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:17.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa366d7c700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa360138dc0 0x7fa36013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365d7a700 1 -- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa360138dc0 msgr2=0x7fa36013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365d7a700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa360138dc0 0x7fa36013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365d7a700 1 -- 172.21.15.32:0/2155810235 >> v1:172.21.15.32:6789/0 conn(0x7fa360133420 legacy=0x7fa360133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365d7a700 1 -- 172.21.15.32:0/2155810235 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600a0590 con 0x7fa3601337f0 2023-09-30T22:30:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365579700 1 -- 172.21.15.32:0/2155810235 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133925750 0 0) 0x7fa36006b990 con 0x7fa360133420 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa366d7c700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa360138dc0 0x7fa36013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365d7a700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3601337f0 0x7fa360138880 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa350009840 tx=0x7fa350009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2367586d1e1ffe1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365579700 1 -- 172.21.15.32:0/2155810235 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350011830 con 0x7fa3601337f0 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.385+0000 7fa365579700 1 -- 172.21.15.32:0/2155810235 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa350011990 con 0x7fa3601337f0 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa365579700 1 -- 172.21.15.32:0/2155810235 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350011d20 con 0x7fa3601337f0 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 -- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3601337f0 msgr2=0x7fa360138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3601337f0 0x7fa360138880 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa350009840 tx=0x7fa350009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 -- 172.21.15.32:0/2155810235 shutdown_connections 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa360138dc0 0x7fa36013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 --2- 172.21.15.32:0/2155810235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3601337f0 0x7fa360138880 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 -- 172.21.15.32:0/2155810235 >> 172.21.15.32:0/2155810235 conn(0x7fa3600a4210 msgr2=0x7fa3600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 -- 172.21.15.32:0/2155810235 shutdown_connections 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.386+0000 7fa3687df700 1 -- 172.21.15.32:0/2155810235 wait complete. 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.387+0000 7fa3687df700 1 Processor -- start 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.387+0000 7fa3687df700 1 -- start start 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa3687df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa360133420 0x7fa360137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa3687df700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3601337f0 0x7fa360137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa3687df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa360138dc0 0x7fa36009c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa3687df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360134650 con 0x7fa360133420 2023-09-30T22:30:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa3687df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36009e020 con 0x7fa360138dc0 2023-09-30T22:30:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa3687df700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa360130ab0 con 0x7fa3601337f0 2023-09-30T22:30:17.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa36657b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa360133420 0x7fa360137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa365d7a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3601337f0 0x7fa360137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa36657b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa360133420 0x7fa360137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55128/0 (socket says 172.21.15.32:55128) 2023-09-30T22:30:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa36657b700 1 -- 172.21.15.32:0/1401173115 learned_addr learned my addr 172.21.15.32:0/1401173115 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:17.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa366d7c700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa360138dc0 0x7fa36009c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa365d7a700 1 -- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa360138dc0 msgr2=0x7fa36009c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa365d7a700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa360138dc0 0x7fa36009c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa365d7a700 1 -- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa360133420 msgr2=0x7fa360137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa365d7a700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa360133420 0x7fa360137810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.388+0000 7fa365d7a700 1 -- 172.21.15.32:0/1401173115 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009a70 con 0x7fa3601337f0 2023-09-30T22:30:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.389+0000 7fa365d7a700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3601337f0 0x7fa360137d50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa350016000 tx=0x7fa350004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.389+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500043c0 con 0x7fa3601337f0 2023-09-30T22:30:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.389+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35001f070 con 0x7fa3601337f0 2023-09-30T22:30:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.389+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3500094a0 con 0x7fa3601337f0 2023-09-30T22:30:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.389+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500036a0 con 0x7fa3601337f0 2023-09-30T22:30:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.389+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36012f280 con 0x7fa3601337f0 2023-09-30T22:30:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.390+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3600a47a0 con 0x7fa3601337f0 2023-09-30T22:30:17.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.390+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa350021030 con 0x7fa3601337f0 2023-09-30T22:30:17.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.391+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa3500407d0 con 0x7fa3601337f0 2023-09-30T22:30:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.400+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3500199a0 con 0x7fa3601337f0 2023-09-30T22:30:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.466+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0bf00a34c0 con 0x7f0bf009f240 2023-09-30T22:30:17.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.467+0000 7f0be67fc700 1 -- 172.21.15.32:0/2606895542 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0be00102e0 con 0x7f0bf009f240 2023-09-30T22:30:17.467 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:17.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 msgr2=0x7f0bf01a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 0x7f0bf01a4240 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0be000db70 tx=0x7f0be000de80 comp rx=0 tx=0).stop 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 shutdown_connections 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf009f650 0x7f0bf01a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0bf009e930 0x7f0bf01a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 --2- 172.21.15.32:0/2606895542 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0bf009f240 0x7f0bf01a4240 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 >> 172.21.15.32:0/2606895542 conn(0x7f0bf00a4210 msgr2=0x7f0bf01b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 shutdown_connections 2023-09-30T22:30:17.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.470+0000 7f0bf8103700 1 -- 172.21.15.32:0/2606895542 wait complete. 2023-09-30T22:30:17.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.585+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24dc072840 con 0x7f24dc09d130 2023-09-30T22:30:17.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.585+0000 7f24d1ffb700 1 -- 172.21.15.32:0/427762025 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24cc021070 con 0x7f24dc09d130 2023-09-30T22:30:17.586 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:17.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 msgr2=0x7f24dc12e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 0x7f24dc12e050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ebe0 tx=0x7f24cc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 shutdown_connections 2023-09-30T22:30:17.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24dc0a0090 0x7f24dc12e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24dc09c650 0x7f24dc12db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 --2- 172.21.15.32:0/427762025 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f24dc09d130 0x7f24dc12e050 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 >> 172.21.15.32:0/427762025 conn(0x7f24dc0a3f70 msgr2=0x7f24dc0a25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 shutdown_connections 2023-09-30T22:30:17.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.588+0000 7f24e32e4700 1 -- 172.21.15.32:0/427762025 wait complete. 2023-09-30T22:30:17.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:17.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.606+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa360074ea0 con 0x7fa3601337f0 2023-09-30T22:30:17.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.607+0000 7fa356ffd700 1 -- 172.21.15.32:0/1401173115 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa350015070 con 0x7fa3601337f0 2023-09-30T22:30:17.607 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3601337f0 msgr2=0x7fa360137d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3601337f0 0x7fa360137d50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa350016000 tx=0x7fa350004750 comp rx=0 tx=0).stop 2023-09-30T22:30:17.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 shutdown_connections 2023-09-30T22:30:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa360138dc0 0x7fa36009c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa360133420 0x7fa360137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 --2- 172.21.15.32:0/1401173115 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3601337f0 0x7fa360137d50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 >> 172.21.15.32:0/1401173115 conn(0x7fa3600a4210 msgr2=0x7fa3600a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 shutdown_connections 2023-09-30T22:30:17.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:17.609+0000 7fa3687df700 1 -- 172.21.15.32:0/1401173115 wait complete. 2023-09-30T22:30:17.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:18.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 Processor -- start 2023-09-30T22:30:18.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 -- start start 2023-09-30T22:30:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ff409d6f0 0x7f0ff409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 0x7f0ff41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff406b460 con 0x7f0ff409d6f0 2023-09-30T22:30:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff409b120 con 0x7f0ff409e410 2023-09-30T22:30:18.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ffae0c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff409a7e0 con 0x7f0ff409e000 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ff3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 0x7f0ff41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ff93a9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.309+0000 7f0ff8ba8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ff409d6f0 0x7f0ff409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 0x7f0ff41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57954/0 (socket says 172.21.15.32:57954) 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 -- 172.21.15.32:0/1718017499 learned_addr learned my addr 172.21.15.32:0/1718017499 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff93a9700 1 --2- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff4133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43018/0 (socket says 172.21.15.32:43018) 2023-09-30T22:30:18.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 -- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 msgr2=0x7f0ff4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 --2- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 -- 172.21.15.32:0/1718017499 >> v2:172.21.15.32:3300/0 conn(0x7f0ff409d6f0 msgr2=0x7f0ff409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 --2- 172.21.15.32:0/1718017499 >> v2:172.21.15.32:3300/0 conn(0x7f0ff409d6f0 0x7f0ff409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 -- 172.21.15.32:0/1718017499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff409a500 con 0x7f0ff409e000 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff8ba8700 1 --2- 172.21.15.32:0/1718017499 >> v2:172.21.15.32:3300/0 conn(0x7f0ff409d6f0 0x7f0ff409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff3fff700 1 --2- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 0x7f0ff41339f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009920 tx=0x7f0fdc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3251a68253cfa023 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1718017499 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc011830 con 0x7f0ff409e000 2023-09-30T22:30:18.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1718017499 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc011990 con 0x7f0ff409e000 2023-09-30T22:30:18.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.310+0000 7f0ff37fe700 1 -- 172.21.15.32:0/1718017499 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc011d20 con 0x7f0ff409e000 2023-09-30T22:30:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 -- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 msgr2=0x7f0ff41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 0x7f0ff41339f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009920 tx=0x7f0fdc009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 -- 172.21.15.32:0/1718017499 shutdown_connections 2023-09-30T22:30:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/1718017499 >> v2:172.21.15.32:3300/0 conn(0x7f0ff409d6f0 0x7f0ff409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/1718017499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409e000 0x7f0ff41339f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 -- 172.21.15.32:0/1718017499 >> 172.21.15.32:0/1718017499 conn(0x7f0ff4066f50 msgr2=0x7f0ff409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 -- 172.21.15.32:0/1718017499 shutdown_connections 2023-09-30T22:30:18.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.311+0000 7f0ffae0c700 1 -- 172.21.15.32:0/1718017499 wait complete. 2023-09-30T22:30:18.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.312+0000 7f0ffae0c700 1 Processor -- start 2023-09-30T22:30:18.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.312+0000 7f0ffae0c700 1 -- start start 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ffae0c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 0x7f0ff41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ffae0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff409e000 0x7f0ff41a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ffae0c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff41a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ffae0c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41a8c00 con 0x7f0ff409e000 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ffae0c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41a9650 con 0x7f0ff409e410 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ffae0c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41aa1f0 con 0x7f0ff409d6f0 2023-09-30T22:30:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.313+0000 7f0ff8ba8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 0x7f0ff41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 0x7f0ff41a5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57964/0 (socket says 172.21.15.32:57964) 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 -- 172.21.15.32:0/3814973863 learned_addr learned my addr 172.21.15.32:0/3814973863 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff93a9700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff41a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff3fff700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff409e000 0x7f0ff41a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 -- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 msgr2=0x7f0ff41a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff41a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 -- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff409e000 msgr2=0x7f0ff41a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff409e000 0x7f0ff41a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 -- 172.21.15.32:0/3814973863 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40094a0 con 0x7f0ff409d6f0 2023-09-30T22:30:18.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff8ba8700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 0x7f0ff41a5d30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400ec10 tx=0x7f0fe400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400c850 con 0x7f0ff409d6f0 2023-09-30T22:30:18.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe400c9b0 con 0x7f0ff409d6f0 2023-09-30T22:30:18.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc009550 con 0x7f0ff409d6f0 2023-09-30T22:30:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400cd40 con 0x7f0ff409d6f0 2023-09-30T22:30:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.314+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41a79e0 con 0x7f0ff409d6f0 2023-09-30T22:30:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.315+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0fe4013470 con 0x7f0ff409d6f0 2023-09-30T22:30:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.315+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0fe403fb60 con 0x7f0ff409d6f0 2023-09-30T22:30:18.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.315+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff40a1f80 con 0x7f0ff409d6f0 2023-09-30T22:30:18.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.320+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0fe4017080 con 0x7f0ff409d6f0 2023-09-30T22:30:18.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:18.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.528+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0ff409c710 con 0x7f0ff409d6f0 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.528+0000 7f0ff17fa700 1 -- 172.21.15.32:0/3814973863 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0fe400cea0 con 0x7f0ff409d6f0 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:18.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 msgr2=0x7f0ff41a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 0x7f0ff41a5d30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400ec10 tx=0x7f0fe400ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 shutdown_connections 2023-09-30T22:30:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ff409e410 0x7f0ff41a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ff409d6f0 0x7f0ff41a5d30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 --2- 172.21.15.32:0/3814973863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff409e000 0x7f0ff41a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 >> 172.21.15.32:0/3814973863 conn(0x7f0ff4066f50 msgr2=0x7f0ff41bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 shutdown_connections 2023-09-30T22:30:18.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.531+0000 7f0ffae0c700 1 -- 172.21.15.32:0/3814973863 wait complete. 2023-09-30T22:30:18.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:18.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:18.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 Processor -- start 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 -- start start 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69a40a0a30 0x7f69a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a40a2a70 con 0x7f69a409d630 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a40a01d0 con 0x7f69a40a0a30 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a9a27700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69a409e2b0 con 0x7f69a4138d80 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.629+0000 7f69a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69a40a0a30 0x7f69a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57976/0 (socket says 172.21.15.32:57976) 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 -- 172.21.15.32:0/3870060247 learned_addr learned my addr 172.21.15.32:0/3870060247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:18.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a27fc700 1 --1- 172.21.15.32:0/3870060247 >> v1:172.21.15.32:6789/0 conn(0x7f69a409d630 0x7f69a40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59354/0 (socket says 172.21.15.32:59354) 2023-09-30T22:30:18.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 -- 172.21.15.32:0/3870060247 >> v2:172.21.15.32:3300/0 conn(0x7f69a40a0a30 msgr2=0x7f69a40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 --2- 172.21.15.32:0/3870060247 >> v2:172.21.15.32:3300/0 conn(0x7f69a40a0a30 0x7f69a40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 -- 172.21.15.32:0/3870060247 >> v1:172.21.15.32:6789/0 conn(0x7f69a409d630 legacy=0x7f69a40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 -- 172.21.15.32:0/3870060247 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a409dc50 con 0x7f69a4138d80 2023-09-30T22:30:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a37fe700 1 --2- 172.21.15.32:0/3870060247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a413b160 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f699c009e50 tx=0x7f699c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfd246c00d11fc3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a1ffb700 1 -- 172.21.15.32:0/3870060247 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c00e070 con 0x7f69a4138d80 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a1ffb700 1 -- 172.21.15.32:0/3870060247 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c003a20 con 0x7f69a4138d80 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.630+0000 7f69a1ffb700 1 -- 172.21.15.32:0/3870060247 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c003dd0 con 0x7f69a4138d80 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 -- 172.21.15.32:0/3870060247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 msgr2=0x7f69a413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3870060247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a413b160 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f699c009e50 tx=0x7f699c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 -- 172.21.15.32:0/3870060247 shutdown_connections 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3870060247 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a413b160 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3870060247 >> v2:172.21.15.32:3300/0 conn(0x7f69a40a0a30 0x7f69a40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 -- 172.21.15.32:0/3870060247 >> 172.21.15.32:0/3870060247 conn(0x7f69a40a3050 msgr2=0x7f69a40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 -- 172.21.15.32:0/3870060247 shutdown_connections 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.631+0000 7f69a9a27700 1 -- 172.21.15.32:0/3870060247 wait complete. 2023-09-30T22:30:18.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.633+0000 7f69a9a27700 1 Processor -- start 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.633+0000 7f69a9a27700 1 -- start start 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.633+0000 7f69a9a27700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69a409d630 0x7f69a4133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a9a27700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 0x7f69a4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a9a27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a41341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a9a27700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a40a4f60 con 0x7f69a409d630 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a9a27700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4136520 con 0x7f69a40a0a30 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a9a27700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69a4135740 con 0x7f69a4138d80 2023-09-30T22:30:18.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 0x7f69a4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 0x7f69a4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43042/0 (socket says 172.21.15.32:43042) 2023-09-30T22:30:18.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69a409d630 0x7f69a4133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 -- 172.21.15.32:0/3252460721 learned_addr learned my addr 172.21.15.32:0/3252460721 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:18.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a37fe700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a41341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 -- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 msgr2=0x7f69a41341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a41341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 -- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69a409d630 msgr2=0x7f69a4133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69a409d630 0x7f69a4133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 -- 172.21.15.32:0/3252460721 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c009b30 con 0x7f69a40a0a30 2023-09-30T22:30:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a2ffd700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69a409d630 0x7f69a4133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.634+0000 7f69a27fc700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 0x7f69a4133c70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f699800c7b0 tx=0x7f699800cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.635+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998004c80 con 0x7f69a40a0a30 2023-09-30T22:30:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.635+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6998004de0 con 0x7f69a40a0a30 2023-09-30T22:30:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.635+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998007310 con 0x7f69a40a0a30 2023-09-30T22:30:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.635+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a40a0410 con 0x7f69a40a0a30 2023-09-30T22:30:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.635+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a4136f20 con 0x7f69a40a0a30 2023-09-30T22:30:18.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.636+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6998007470 con 0x7f69a40a0a30 2023-09-30T22:30:18.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.636+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f699803f480 con 0x7f69a40a0a30 2023-09-30T22:30:18.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.636+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a40747f0 con 0x7f69a40a0a30 2023-09-30T22:30:18.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.641+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6998007720 con 0x7f69a40a0a30 2023-09-30T22:30:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.745+0000 7f162a3c2700 1 Processor -- start 2023-09-30T22:30:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162a3c2700 1 -- start start 2023-09-30T22:30:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162a3c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16240a0a30 0x7f16240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162a3c2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f162413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162a3c2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16240a2a70 con 0x7f162409d630 2023-09-30T22:30:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162a3c2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240a01d0 con 0x7f1624138d80 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162a3c2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f162409e2b0 con 0x7f16240a0a30 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f16237fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f162409d630 0x7f16240a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59366/0 (socket says 172.21.15.32:59366) 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f162895f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f162413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f16237fe700 1 -- 172.21.15.32:0/2951988137 learned_addr learned my addr 172.21.15.32:0/2951988137 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.746+0000 7f1623fff700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16240a0a30 0x7f16240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f1622ffd700 1 -- 172.21.15.32:0/2951988137 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232130744 0 0) 0x7f16240a2a70 con 0x7f162409d630 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f1622ffd700 1 -- 172.21.15.32:0/2951988137 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f160c0035b0 con 0x7f162409d630 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f162895f700 1 -- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16240a0a30 msgr2=0x7f16240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f162895f700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16240a0a30 0x7f16240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f162895f700 1 -- 172.21.15.32:0/2951988137 >> v1:172.21.15.32:6789/0 conn(0x7f162409d630 legacy=0x7f16240a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f162895f700 1 -- 172.21.15.32:0/2951988137 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162409dc50 con 0x7f1624138d80 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f1623fff700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16240a0a30 0x7f16240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f162895f700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f162413b160 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f161c009dc0 tx=0x7f161c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59ed0b7a4b8f29e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f1622ffd700 1 -- 172.21.15.32:0/2951988137 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00e070 con 0x7f1624138d80 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f1622ffd700 1 -- 172.21.15.32:0/2951988137 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c00bae0 con 0x7f1624138d80 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.747+0000 7f1622ffd700 1 -- 172.21.15.32:0/2951988137 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00be50 con 0x7f1624138d80 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 -- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 msgr2=0x7f162413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f162413b160 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f161c009dc0 tx=0x7f161c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 -- 172.21.15.32:0/2951988137 shutdown_connections 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f162413b160 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 --2- 172.21.15.32:0/2951988137 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f16240a0a30 0x7f16240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 -- 172.21.15.32:0/2951988137 >> 172.21.15.32:0/2951988137 conn(0x7f16240a3050 msgr2=0x7f16240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 -- 172.21.15.32:0/2951988137 shutdown_connections 2023-09-30T22:30:18.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.748+0000 7f162a3c2700 1 -- 172.21.15.32:0/2951988137 wait complete. 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.749+0000 7f162a3c2700 1 Processor -- start 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.749+0000 7f162a3c2700 1 -- start start 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162a3c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162409d630 0x7f1624133720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162a3c2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 0x7f1624133c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162a3c2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f16241341a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162a3c2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16241371a0 con 0x7f16240a0a30 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162a3c2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240a0410 con 0x7f1624138d80 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162a3c2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f162413ac50 con 0x7f162409d630 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f16237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 0x7f1624133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f16237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 0x7f1624133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55170/0 (socket says 172.21.15.32:55170) 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f16237fe700 1 -- 172.21.15.32:0/3903583400 learned_addr learned my addr 172.21.15.32:0/3903583400 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f1623fff700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162409d630 0x7f1624133720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.750+0000 7f162895f700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f16241341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1623fff700 1 -- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 msgr2=0x7f16241341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1623fff700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f16241341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1623fff700 1 -- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 msgr2=0x7f1624133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1623fff700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 0x7f1624133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1623fff700 1 -- 172.21.15.32:0/3903583400 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140094a0 con 0x7f162409d630 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f162895f700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f16241341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f16237fe700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 0x7f1624133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1623fff700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162409d630 0x7f1624133720 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9b0 tx=0x7f161400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400ee50 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400c850 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c009a70 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400cc00 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.751+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624136c50 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.752+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1614013470 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.752+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f161403f740 con 0x7f162409d630 2023-09-30T22:30:18.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.753+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16240747f0 con 0x7f162409d630 2023-09-30T22:30:18.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.758+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1614013720 con 0x7f162409d630 2023-09-30T22:30:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.773+0000 7fb468b61700 1 Processor -- start 2023-09-30T22:30:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.773+0000 7fb468b61700 1 -- start start 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb468b61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409f410 0x7fb464136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb468b61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb464136fe0 0x7fb4641393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb468b61700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb464070940 con 0x7fb46409e930 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb468b61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46406fd60 con 0x7fb464136fe0 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb468b61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb46406fa80 con 0x7fb46409f410 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb46259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb46409e930 0x7fb46409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59370/0 (socket says 172.21.15.32:59370) 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb462d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb464136fe0 0x7fb4641393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb46259c700 1 -- 172.21.15.32:0/1442038623 learned_addr learned my addr 172.21.15.32:0/1442038623 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.774+0000 7fb461d9b700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409f410 0x7fb464136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb46159a700 1 -- 172.21.15.32:0/1442038623 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771172510 0 0) 0x7fb464070940 con 0x7fb46409e930 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb46159a700 1 -- 172.21.15.32:0/1442038623 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb44c0035b0 con 0x7fb46409e930 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb462d9d700 1 -- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409f410 msgr2=0x7fb464136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb462d9d700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409f410 0x7fb464136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb462d9d700 1 -- 172.21.15.32:0/1442038623 >> v1:172.21.15.32:6789/0 conn(0x7fb46409e930 legacy=0x7fb46409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb462d9d700 1 -- 172.21.15.32:0/1442038623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46409fb60 con 0x7fb464136fe0 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb461d9b700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409f410 0x7fb464136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb462d9d700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb464136fe0 0x7fb4641393c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009dc0 tx=0x7fb45c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42ac96799718820 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb46159a700 1 -- 172.21.15.32:0/1442038623 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00e070 con 0x7fb464136fe0 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb46159a700 1 -- 172.21.15.32:0/1442038623 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00bb20 con 0x7fb464136fe0 2023-09-30T22:30:18.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.775+0000 7fb46159a700 1 -- 172.21.15.32:0/1442038623 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c012430 con 0x7fb464136fe0 2023-09-30T22:30:18.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 -- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb464136fe0 msgr2=0x7fb4641393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb464136fe0 0x7fb4641393c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009dc0 tx=0x7fb45c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 -- 172.21.15.32:0/1442038623 shutdown_connections 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb464136fe0 0x7fb4641393c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 --2- 172.21.15.32:0/1442038623 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409f410 0x7fb464136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 -- 172.21.15.32:0/1442038623 >> 172.21.15.32:0/1442038623 conn(0x7fb4640a4210 msgr2=0x7fb4640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 -- 172.21.15.32:0/1442038623 shutdown_connections 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.776+0000 7fb468b61700 1 -- 172.21.15.32:0/1442038623 wait complete. 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 Processor -- start 2023-09-30T22:30:18.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 -- start start 2023-09-30T22:30:18.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409e930 0x7fb46406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 0x7fb46406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb464136fe0 0x7fb46406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464123e30 con 0x7fb464136fe0 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46406d510 con 0x7fb46409f410 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.778+0000 7fb468b61700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641341d0 con 0x7fb46409e930 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 0x7fb46406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 0x7fb46406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43076/0 (socket says 172.21.15.32:43076) 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 -- 172.21.15.32:0/1131139488 learned_addr learned my addr 172.21.15.32:0/1131139488 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb46259c700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409e930 0x7fb46406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb462d9d700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb464136fe0 0x7fb46406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 -- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409e930 msgr2=0x7fb46406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409e930 0x7fb46406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 -- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb464136fe0 msgr2=0x7fb46406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb464136fe0 0x7fb46406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 -- 172.21.15.32:0/1131139488 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580094a0 con 0x7fb46409f410 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb461d9b700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 0x7fb46406bdf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb45800ebe0 tx=0x7fb45800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.779+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800c850 con 0x7fb46409f410 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.780+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800c9b0 con 0x7fb46409f410 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.780+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c009a70 con 0x7fb46409f410 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.780+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800cd40 con 0x7fb46409f410 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.780+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4641229f0 con 0x7fb46409f410 2023-09-30T22:30:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.781+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464074ea0 con 0x7fb46409f410 2023-09-30T22:30:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.781+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb458013470 con 0x7fb46409f410 2023-09-30T22:30:18.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.781+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb45803ff90 con 0x7fb46409f410 2023-09-30T22:30:18.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.786+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb458013720 con 0x7fb46409f410 2023-09-30T22:30:18.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.855+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f69a4136bc0 con 0x7f69a40a0a30 2023-09-30T22:30:18.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.856+0000 7f698bfff700 1 -- 172.21.15.32:0/3252460721 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6998011e80 con 0x7f69a40a0a30 2023-09-30T22:30:18.856 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:18.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 msgr2=0x7f69a4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 0x7f69a4133c70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f699800c7b0 tx=0x7f699800cac0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 shutdown_connections 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69a4138d80 0x7f69a41341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69a409d630 0x7f69a4133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 --2- 172.21.15.32:0/3252460721 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69a40a0a30 0x7f69a4133c70 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 >> 172.21.15.32:0/3252460721 conn(0x7f69a40a3050 msgr2=0x7f69a409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 shutdown_connections 2023-09-30T22:30:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.858+0000 7f69a9a27700 1 -- 172.21.15.32:0/3252460721 wait complete. 2023-09-30T22:30:18.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:18.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.964+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1624072980 con 0x7f162409d630 2023-09-30T22:30:18.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.965+0000 7f1620ff9700 1 -- 172.21.15.32:0/3903583400 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1614021030 con 0x7f162409d630 2023-09-30T22:30:18.965 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162409d630 msgr2=0x7f1624133720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162409d630 0x7f1624133720 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9b0 tx=0x7f161400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 shutdown_connections 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1624138d80 0x7f16241341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f162409d630 0x7f1624133720 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 --2- 172.21.15.32:0/3903583400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16240a0a30 0x7f1624133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 >> 172.21.15.32:0/3903583400 conn(0x7f16240a3050 msgr2=0x7f16240a6890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 shutdown_connections 2023-09-30T22:30:18.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.967+0000 7f162a3c2700 1 -- 172.21.15.32:0/3903583400 wait complete. 2023-09-30T22:30:18.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:18.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.996+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb464123ad0 con 0x7fb46409f410 2023-09-30T22:30:18.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.996+0000 7fb452ffd700 1 -- 172.21.15.32:0/1131139488 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb464123ad0 con 0x7fb46409f410 2023-09-30T22:30:18.996 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.998+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 msgr2=0x7fb46406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 0x7fb46406bdf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb45800ebe0 tx=0x7fb45800eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 shutdown_connections 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb464136fe0 0x7fb46406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb46409e930 0x7fb46406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 --2- 172.21.15.32:0/1131139488 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb46409f410 0x7fb46406bdf0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 >> 172.21.15.32:0/1131139488 conn(0x7fb4640a4210 msgr2=0x7fb464134890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 shutdown_connections 2023-09-30T22:30:18.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:18.999+0000 7fb468b61700 1 -- 172.21.15.32:0/1131139488 wait complete. 2023-09-30T22:30:19.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:19.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.693+0000 7fed9e378700 1 Processor -- start 2023-09-30T22:30:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.693+0000 7fed9e378700 1 -- start start 2023-09-30T22:30:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9e378700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed9809e1d0 0x7fed98138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9e378700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed98139170 0x7fed9813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9e378700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9809b120 con 0x7fed9809d6f0 2023-09-30T22:30:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9e378700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9806b460 con 0x7fed98139170 2023-09-30T22:30:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9e378700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9809a7e0 con 0x7fed9809e1d0 2023-09-30T22:30:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9c915700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fed98139170 0x7fed9813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed9809e1d0 0x7fed98138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed97fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed9809d6f0 0x7fed9809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59374/0 (socket says 172.21.15.32:59374) 2023-09-30T22:30:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed97fff700 1 -- 172.21.15.32:0/3284213831 learned_addr learned my addr 172.21.15.32:0/3284213831 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed977fe700 1 -- 172.21.15.32:0/3284213831 >> v2:172.21.15.32:3300/0 conn(0x7fed98139170 msgr2=0x7fed9813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed977fe700 1 --2- 172.21.15.32:0/3284213831 >> v2:172.21.15.32:3300/0 conn(0x7fed98139170 0x7fed9813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed977fe700 1 -- 172.21.15.32:0/3284213831 >> v1:172.21.15.32:6789/0 conn(0x7fed9809d6f0 legacy=0x7fed9809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed977fe700 1 -- 172.21.15.32:0/3284213831 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9809a500 con 0x7fed9809e1d0 2023-09-30T22:30:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed96ffd700 1 -- 172.21.15.32:0/3284213831 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700442150 0 0) 0x7fed9809b120 con 0x7fed9809d6f0 2023-09-30T22:30:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed9c915700 1 --2- 172.21.15.32:0/3284213831 >> v2:172.21.15.32:3300/0 conn(0x7fed98139170 0x7fed9813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.694+0000 7fed977fe700 1 --2- 172.21.15.32:0/3284213831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed9809e1d0 0x7fed98138c30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fed80009870 tx=0x7fed80009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bba4fe19abf0e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed96ffd700 1 -- 172.21.15.32:0/3284213831 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80011830 con 0x7fed9809e1d0 2023-09-30T22:30:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed96ffd700 1 -- 172.21.15.32:0/3284213831 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80011990 con 0x7fed9809e1d0 2023-09-30T22:30:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed96ffd700 1 -- 172.21.15.32:0/3284213831 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80011d20 con 0x7fed9809e1d0 2023-09-30T22:30:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 -- 172.21.15.32:0/3284213831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed9809e1d0 msgr2=0x7fed98138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 --2- 172.21.15.32:0/3284213831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed9809e1d0 0x7fed98138c30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fed80009870 tx=0x7fed80009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 -- 172.21.15.32:0/3284213831 shutdown_connections 2023-09-30T22:30:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 --2- 172.21.15.32:0/3284213831 >> v2:172.21.15.32:3300/0 conn(0x7fed98139170 0x7fed9813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 --2- 172.21.15.32:0/3284213831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed9809e1d0 0x7fed98138c30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 -- 172.21.15.32:0/3284213831 >> 172.21.15.32:0/3284213831 conn(0x7fed98066f50 msgr2=0x7fed9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 -- 172.21.15.32:0/3284213831 shutdown_connections 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.695+0000 7fed9e378700 1 -- 172.21.15.32:0/3284213831 wait complete. 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.696+0000 7fed9e378700 1 Processor -- start 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 -- start start 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 0x7fed98121730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed9809e1d0 0x7fed98121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 0x7fed981221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed98123340 con 0x7fed9809d6f0 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed98137620 con 0x7fed98139170 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9e378700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed981249e0 con 0x7fed9809e1d0 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 0x7fed98121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9c915700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 0x7fed981221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9c915700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 0x7fed981221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43102/0 (socket says 172.21.15.32:43102) 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 0x7fed98121730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55204/0 (socket says 172.21.15.32:55204) 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed9c915700 1 -- 172.21.15.32:0/2998976967 learned_addr learned my addr 172.21.15.32:0/2998976967 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.697+0000 7fed977fe700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed9809e1d0 0x7fed98121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9c915700 1 -- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed9809e1d0 msgr2=0x7fed98121c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9c915700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed9809e1d0 0x7fed98121c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9c915700 1 -- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 msgr2=0x7fed98121730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9c915700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 0x7fed98121730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9c915700 1 -- 172.21.15.32:0/2998976967 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90009a70 con 0x7fed98139170 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed97fff700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 0x7fed98121730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9c915700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 0x7fed981221b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fed9000efc0 tx=0x7fed9000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000ca90 con 0x7fed98139170 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9000f040 con 0x7fed98139170 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed800094a0 con 0x7fed98139170 2023-09-30T22:30:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000ce10 con 0x7fed98139170 2023-09-30T22:30:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.698+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed98123bb0 con 0x7fed98139170 2023-09-30T22:30:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.699+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fed90014480 con 0x7fed98139170 2023-09-30T22:30:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.699+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed98074ea0 con 0x7fed98139170 2023-09-30T22:30:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.699+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fed9003fe70 con 0x7fed98139170 2023-09-30T22:30:19.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.704+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fed90012930 con 0x7fed98139170 2023-09-30T22:30:19.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.909+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fed9806c360 con 0x7fed98139170 2023-09-30T22:30:19.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.910+0000 7fed94ff9700 1 -- 172.21.15.32:0/2998976967 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fed9001c3b0 con 0x7fed98139170 2023-09-30T22:30:19.910 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 msgr2=0x7fed981221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 0x7fed981221b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fed9000efc0 tx=0x7fed9000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 shutdown_connections 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fed98139170 0x7fed981221b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed9809d6f0 0x7fed98121730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 --2- 172.21.15.32:0/2998976967 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fed9809e1d0 0x7fed98121c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 >> 172.21.15.32:0/2998976967 conn(0x7fed98066f50 msgr2=0x7fed98070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 shutdown_connections 2023-09-30T22:30:19.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:19.912+0000 7fed9e378700 1 -- 172.21.15.32:0/2998976967 wait complete. 2023-09-30T22:30:19.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:19.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.011+0000 7f93ab938700 1 Processor -- start 2023-09-30T22:30:20.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.011+0000 7f93ab938700 1 -- start start 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93ab938700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 0x7f93a409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93ab938700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93ab938700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a4070940 con 0x7f93a409f650 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93ab938700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a406fd60 con 0x7f93a409e930 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93ab938700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93a406fa80 con 0x7f93a409f240 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93a8ed3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93a96d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 0x7f93a409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93a9ed5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93a409f650 0x7f93a4131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59384/0 (socket says 172.21.15.32:59384) 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93a96d4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 0x7f93a409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43104/0 (socket says 172.21.15.32:43104) 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93a9ed5700 1 -- 172.21.15.32:0/1945026730 learned_addr learned my addr 172.21.15.32:0/1945026730 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.012+0000 7f93a8ed3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:58034/0 (socket says 172.21.15.32:58034) 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f93a96d4700 1 -- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 msgr2=0x7f93a41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f93a96d4700 1 --2- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f93a96d4700 1 -- 172.21.15.32:0/1945026730 >> v1:172.21.15.32:6789/0 conn(0x7f93a409f650 legacy=0x7f93a4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f939bfff700 1 -- 172.21.15.32:0/1945026730 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111511654 0 0) 0x7f93a4070940 con 0x7f93a409f650 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f93a96d4700 1 -- 172.21.15.32:0/1945026730 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a409fb60 con 0x7f93a409e930 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f93a8ed3700 1 --2- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:20.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f93a96d4700 1 --2- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 0x7f93a409ed00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009870 tx=0x7f93a0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a656dff889cedad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f939bfff700 1 -- 172.21.15.32:0/1945026730 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0011830 con 0x7f93a409e930 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f939bfff700 1 -- 172.21.15.32:0/1945026730 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0011990 con 0x7f93a409e930 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.013+0000 7f939bfff700 1 -- 172.21.15.32:0/1945026730 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0011d20 con 0x7f93a409e930 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 -- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 msgr2=0x7f93a409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 --2- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 0x7f93a409ed00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009870 tx=0x7f93a0009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 -- 172.21.15.32:0/1945026730 shutdown_connections 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 --2- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409e930 0x7f93a409ed00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 --2- 172.21.15.32:0/1945026730 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 -- 172.21.15.32:0/1945026730 >> 172.21.15.32:0/1945026730 conn(0x7f93a40a4210 msgr2=0x7f93a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 -- 172.21.15.32:0/1945026730 shutdown_connections 2023-09-30T22:30:20.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.014+0000 7f93ab938700 1 -- 172.21.15.32:0/1945026730 wait complete. 2023-09-30T22:30:20.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 Processor -- start 2023-09-30T22:30:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 -- start start 2023-09-30T22:30:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a409e930 0x7f93a41a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 0x7f93a41c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41a3af0 con 0x7f93a409e930 2023-09-30T22:30:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41b4f00 con 0x7f93a409f650 2023-09-30T22:30:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.015+0000 7f93ab938700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41a29e0 con 0x7f93a409f240 2023-09-30T22:30:20.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 0x7f93a41c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 0x7f93a41c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43110/0 (socket says 172.21.15.32:43110) 2023-09-30T22:30:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 -- 172.21.15.32:0/1429197770 learned_addr learned my addr 172.21.15.32:0/1429197770 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a96d4700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a409e930 0x7f93a41a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a8ed3700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 -- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 msgr2=0x7f93a41a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 -- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a409e930 msgr2=0x7f93a41a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a409e930 0x7f93a41a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 -- 172.21.15.32:0/1429197770 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940094a0 con 0x7f93a409f650 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a96d4700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a409e930 0x7f93a41a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93a9ed5700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 0x7f93a41c3c20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f939c00cca0 tx=0x7f939c00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00aad0 con 0x7f93a409f650 2023-09-30T22:30:20.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c00ac50 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a00094a0 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.016+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0070d0 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.017+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a40a0190 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.017+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f939c007230 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.018+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f939c03f470 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.018+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a41a2340 con 0x7f93a409f650 2023-09-30T22:30:20.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.023+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f939c018080 con 0x7f93a409f650 2023-09-30T22:30:20.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 Processor -- start 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 -- start start 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67a01316c0 0x7f67a0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a009d9f0 0x7f67a006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a009e130 con 0x7f67a006acb0 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a01312b0 con 0x7f67a01316c0 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f67a58a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67a009f6e0 con 0x7f67a009d9f0 2023-09-30T22:30:20.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.127+0000 7f679f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67a006acb0 0x7f67a0138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59388/0 (socket says 172.21.15.32:59388) 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a009d9f0 0x7f67a006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679f7fe700 1 -- 172.21.15.32:0/2403630538 learned_addr learned my addr 172.21.15.32:0/2403630538 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679effd700 1 --2- 172.21.15.32:0/2403630538 >> v2:172.21.15.32:3300/0 conn(0x7f67a01316c0 0x7f67a0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679dffb700 1 -- 172.21.15.32:0/2403630538 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213496580 0 0) 0x7f6798001490 con 0x7f67a006acb0 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679dffb700 1 -- 172.21.15.32:0/2403630538 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67880035b0 con 0x7f67a006acb0 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679e7fc700 1 -- 172.21.15.32:0/2403630538 >> v2:172.21.15.32:3300/0 conn(0x7f67a01316c0 msgr2=0x7f67a0131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:20.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679e7fc700 1 --2- 172.21.15.32:0/2403630538 >> v2:172.21.15.32:3300/0 conn(0x7f67a01316c0 0x7f67a0131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679e7fc700 1 -- 172.21.15.32:0/2403630538 >> v1:172.21.15.32:6789/0 conn(0x7f67a006acb0 legacy=0x7f67a0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679e7fc700 1 -- 172.21.15.32:0/2403630538 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a009e760 con 0x7f67a009d9f0 2023-09-30T22:30:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679e7fc700 1 --2- 172.21.15.32:0/2403630538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a009d9f0 0x7f67a006a770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6794009840 tx=0x7f6794009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd997d88d9d65d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679dffb700 1 -- 172.21.15.32:0/2403630538 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794011830 con 0x7f67a009d9f0 2023-09-30T22:30:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.128+0000 7f679dffb700 1 -- 172.21.15.32:0/2403630538 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6794011990 con 0x7f67a009d9f0 2023-09-30T22:30:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f679dffb700 1 -- 172.21.15.32:0/2403630538 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794011d00 con 0x7f67a009d9f0 2023-09-30T22:30:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 -- 172.21.15.32:0/2403630538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a009d9f0 msgr2=0x7f67a006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 --2- 172.21.15.32:0/2403630538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a009d9f0 0x7f67a006a770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6794009840 tx=0x7f6794009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 -- 172.21.15.32:0/2403630538 shutdown_connections 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 --2- 172.21.15.32:0/2403630538 >> v2:172.21.15.32:3300/0 conn(0x7f67a01316c0 0x7f67a0131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 --2- 172.21.15.32:0/2403630538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a009d9f0 0x7f67a006a770 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 -- 172.21.15.32:0/2403630538 >> 172.21.15.32:0/2403630538 conn(0x7f67a0066f50 msgr2=0x7f67a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 -- 172.21.15.32:0/2403630538 shutdown_connections 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.129+0000 7f67a58a1700 1 -- 172.21.15.32:0/2403630538 wait complete. 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.130+0000 7f67a58a1700 1 Processor -- start 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 -- start start 2023-09-30T22:30:20.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 0x7f67a0070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a009d9f0 0x7f67a0070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67a01316c0 0x7f67a0071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0072260 con 0x7f67a009d9f0 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a011a6e0 con 0x7f67a01316c0 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f67a58a1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0119fe0 con 0x7f67a006acb0 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f679effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 0x7f67a0070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f679effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 0x7f67a0070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:58072/0 (socket says 172.21.15.32:58072) 2023-09-30T22:30:20.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f679effd700 1 -- 172.21.15.32:0/1127077911 learned_addr learned my addr 172.21.15.32:0/1127077911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f679f7fe700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67a01316c0 0x7f67a0071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.131+0000 7f679e7fc700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a009d9f0 0x7f67a0070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f679effd700 1 -- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67a01316c0 msgr2=0x7f67a0071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f679effd700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67a01316c0 0x7f67a0071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f679effd700 1 -- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a009d9f0 msgr2=0x7f67a0070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f679effd700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a009d9f0 0x7f67a0070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f679effd700 1 -- 172.21.15.32:0/1127077911 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940094a0 con 0x7f67a006acb0 2023-09-30T22:30:20.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f679effd700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 0x7f67a0070890 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f679000e810 tx=0x7f679000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000c070 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790013610 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900139a0 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a0128530 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.132+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a011b410 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.133+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6790013b00 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.133+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a0071cf0 con 0x7f67a006acb0 2023-09-30T22:30:20.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.133+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f679003f390 con 0x7f67a006acb0 2023-09-30T22:30:20.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.138+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6790010900 con 0x7f67a006acb0 2023-09-30T22:30:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.166+0000 7fac50fc5700 1 Processor -- start 2023-09-30T22:30:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.166+0000 7fac50fc5700 1 -- start start 2023-09-30T22:30:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac50fc5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 0x7fac4c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac50fc5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac50fc5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c06b990 con 0x7fac4c133e30 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac50fc5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0a0590 con 0x7fac4c133420 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac50fc5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c09fb80 con 0x7fac4c134260 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac4a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 0x7fac4c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac4ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac4a59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 0x7fac4c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43140/0 (socket says 172.21.15.32:43140) 2023-09-30T22:30:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac43fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac4c133e30 0x7fac4c12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59398/0 (socket says 172.21.15.32:59398) 2023-09-30T22:30:20.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.167+0000 7fac4a59c700 1 -- 172.21.15.32:0/1372713328 learned_addr learned my addr 172.21.15.32:0/1372713328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac4a59c700 1 -- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 msgr2=0x7fac4c1306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac4a59c700 1 --2- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c1306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac4a59c700 1 -- 172.21.15.32:0/1372713328 >> v1:172.21.15.32:6789/0 conn(0x7fac4c133e30 legacy=0x7fac4c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac4a59c700 1 -- 172.21.15.32:0/1372713328 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0a0030 con 0x7fac4c133420 2023-09-30T22:30:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac4a59c700 1 --2- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 0x7fac4c1337f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009920 tx=0x7fac3c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6727f8d7461a373 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac49d9b700 1 -- 172.21.15.32:0/1372713328 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c011830 con 0x7fac4c133420 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac49d9b700 1 -- 172.21.15.32:0/1372713328 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c011990 con 0x7fac4c133420 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.168+0000 7fac49d9b700 1 -- 172.21.15.32:0/1372713328 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c011d20 con 0x7fac4c133420 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 -- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 msgr2=0x7fac4c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 --2- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 0x7fac4c1337f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009920 tx=0x7fac3c009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 -- 172.21.15.32:0/1372713328 shutdown_connections 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 --2- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c1306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 --2- 172.21.15.32:0/1372713328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133420 0x7fac4c1337f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 -- 172.21.15.32:0/1372713328 >> 172.21.15.32:0/1372713328 conn(0x7fac4c0a4210 msgr2=0x7fac4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 -- 172.21.15.32:0/1372713328 shutdown_connections 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.169+0000 7fac50fc5700 1 -- 172.21.15.32:0/1372713328 wait complete. 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.170+0000 7fac50fc5700 1 Processor -- start 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.170+0000 7fac50fc5700 1 -- start start 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.170+0000 7fac50fc5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac4c133420 0x7fac4c09c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.170+0000 7fac50fc5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 0x7fac4c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.170+0000 7fac50fc5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c09d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.170+0000 7fac50fc5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c121140 con 0x7fac4c133420 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac50fc5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1206b0 con 0x7fac4c133e30 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac50fc5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c09e240 con 0x7fac4c134260 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 0x7fac4c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 0x7fac4c09ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:43150/0 (socket says 172.21.15.32:43150) 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 -- 172.21.15.32:0/2836442561 learned_addr learned my addr 172.21.15.32:0/2836442561 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac4ad9d700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c09d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac4a59c700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac4c133420 0x7fac4c09c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 -- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 msgr2=0x7fac4c09d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c09d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 -- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac4c133420 msgr2=0x7fac4c09c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac4c133420 0x7fac4c09c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 -- 172.21.15.32:0/2836442561 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c009550 con 0x7fac4c133e30 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac4ad9d700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c09d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac43fff700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 0x7fac4c09ce00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fac3400c4f0 tx=0x7fac3400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.171+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34005cb0 con 0x7fac4c133e30 2023-09-30T22:30:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.172+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac34005e10 con 0x7fac4c133e30 2023-09-30T22:30:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.172+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c11ffc0 con 0x7fac4c133e30 2023-09-30T22:30:20.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.172+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340072d0 con 0x7fac4c133e30 2023-09-30T22:30:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.172+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c120cd0 con 0x7fac4c133e30 2023-09-30T22:30:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.173+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fac34007430 con 0x7fac4c133e30 2023-09-30T22:30:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.173+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fac3403ea50 con 0x7fac4c133e30 2023-09-30T22:30:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.173+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c1204c0 con 0x7fac4c133e30 2023-09-30T22:30:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.178+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fac34017080 con 0x7fac4c133e30 2023-09-30T22:30:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.237+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f93a4074ea0 con 0x7f93a409f650 2023-09-30T22:30:20.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.237+0000 7f9399ffb700 1 -- 172.21.15.32:0/1429197770 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f939c0074e0 con 0x7f93a409f650 2023-09-30T22:30:20.237 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.239+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 msgr2=0x7f93a41c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.239+0000 7f93ab938700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 0x7f93a41c3c20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f939c00cca0 tx=0x7f939c00a060 comp rx=0 tx=0).stop 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 shutdown_connections 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93a409f650 0x7f93a41c3c20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93a409e930 0x7f93a41a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 --2- 172.21.15.32:0/1429197770 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93a409f240 0x7f93a41a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 >> 172.21.15.32:0/1429197770 conn(0x7f93a40a4210 msgr2=0x7f93a41b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 shutdown_connections 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.240+0000 7f93ab938700 1 -- 172.21.15.32:0/1429197770 wait complete. 2023-09-30T22:30:20.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:20.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.353+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f67a011abe0 con 0x7f67a006acb0 2023-09-30T22:30:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.354+0000 7f6787fff700 1 -- 172.21.15.32:0/1127077911 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6790010310 con 0x7f67a006acb0 2023-09-30T22:30:20.354 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:20.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 msgr2=0x7f67a0070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 0x7f67a0070890 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f679000e810 tx=0x7f679000eb20 comp rx=0 tx=0).stop 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 shutdown_connections 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67a01316c0 0x7f67a0071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f67a006acb0 0x7f67a0070890 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 --2- 172.21.15.32:0/1127077911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67a009d9f0 0x7f67a0070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 >> 172.21.15.32:0/1127077911 conn(0x7f67a0066f50 msgr2=0x7f67a00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 shutdown_connections 2023-09-30T22:30:20.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.356+0000 7f67a58a1700 1 -- 172.21.15.32:0/1127077911 wait complete. 2023-09-30T22:30:20.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:20.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.386+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fac4c074ea0 con 0x7fac4c133e30 2023-09-30T22:30:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.386+0000 7fac42ffd700 1 -- 172.21.15.32:0/2836442561 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac3401b3b0 con 0x7fac4c133e30 2023-09-30T22:30:20.387 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 msgr2=0x7fac4c09ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 0x7fac4c09ce00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fac3400c4f0 tx=0x7fac3400c800 comp rx=0 tx=0).stop 2023-09-30T22:30:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 shutdown_connections 2023-09-30T22:30:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fac4c134260 0x7fac4c09d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac4c133420 0x7fac4c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 --2- 172.21.15.32:0/2836442561 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac4c133e30 0x7fac4c09ce00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 >> 172.21.15.32:0/2836442561 conn(0x7fac4c0a4210 msgr2=0x7fac4c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 shutdown_connections 2023-09-30T22:30:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:20.389+0000 7fac50fc5700 1 -- 172.21.15.32:0/2836442561 wait complete. 2023-09-30T22:30:20.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:20.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:21.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.072+0000 7f019b3ce700 1 Processor -- start 2023-09-30T22:30:21.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.072+0000 7f019b3ce700 1 -- start start 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019b3ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f019409d6f0 0x7f019409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019b3ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e000 0x7f0194130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019b3ce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019409f130 con 0x7f019409e410 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019b3ce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019409f520 con 0x7f019409d6f0 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019b3ce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f019409e930 con 0x7f019409e000 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f0198969700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e000 0x7f0194130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019996b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f019409e410 0x7f0194130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53100/0 (socket says 172.21.15.32:53100) 2023-09-30T22:30:21.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019916a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f019409d6f0 0x7f019409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019996b700 1 -- 172.21.15.32:0/1711697688 learned_addr learned my addr 172.21.15.32:0/1711697688 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f0198969700 1 -- 172.21.15.32:0/1711697688 >> v2:172.21.15.32:3300/0 conn(0x7f019409d6f0 msgr2=0x7f019409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f0198969700 1 --2- 172.21.15.32:0/1711697688 >> v2:172.21.15.32:3300/0 conn(0x7f019409d6f0 0x7f019409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f0198969700 1 -- 172.21.15.32:0/1711697688 >> v1:172.21.15.32:6789/0 conn(0x7f019409e410 legacy=0x7f0194130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f0198969700 1 -- 172.21.15.32:0/1711697688 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019409b2d0 con 0x7f019409e000 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f019916a700 1 --2- 172.21.15.32:0/1711697688 >> v2:172.21.15.32:3300/0 conn(0x7f019409d6f0 0x7f019409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.073+0000 7f0198969700 1 --2- 172.21.15.32:0/1711697688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e000 0x7f0194130660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0184009840 tx=0x7f0184009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea753723c14875a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f018bfff700 1 -- 172.21.15.32:0/1711697688 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184011830 con 0x7f019409e000 2023-09-30T22:30:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f018bfff700 1 -- 172.21.15.32:0/1711697688 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0184011990 con 0x7f019409e000 2023-09-30T22:30:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f018bfff700 1 -- 172.21.15.32:0/1711697688 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184011d20 con 0x7f019409e000 2023-09-30T22:30:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 -- 172.21.15.32:0/1711697688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e000 msgr2=0x7f0194130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 --2- 172.21.15.32:0/1711697688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e000 0x7f0194130660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0184009840 tx=0x7f0184009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:21.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 -- 172.21.15.32:0/1711697688 shutdown_connections 2023-09-30T22:30:21.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 --2- 172.21.15.32:0/1711697688 >> v2:172.21.15.32:3300/0 conn(0x7f019409d6f0 0x7f019409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 --2- 172.21.15.32:0/1711697688 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e000 0x7f0194130660 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 -- 172.21.15.32:0/1711697688 >> 172.21.15.32:0/1711697688 conn(0x7f0194066f50 msgr2=0x7f019409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 -- 172.21.15.32:0/1711697688 shutdown_connections 2023-09-30T22:30:21.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.074+0000 7f019b3ce700 1 -- 172.21.15.32:0/1711697688 wait complete. 2023-09-30T22:30:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.075+0000 7f019b3ce700 1 Processor -- start 2023-09-30T22:30:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 -- start start 2023-09-30T22:30:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 0x7f019406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f019409e000 0x7f019406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 0x7f019406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194125ee0 con 0x7f019409e000 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01941302e0 con 0x7f019409d6f0 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019b3ce700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f01940703f0 con 0x7f019409e410 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019916a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 0x7f019406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019996b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 0x7f019406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019916a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 0x7f019406e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47650/0 (socket says 172.21.15.32:47650) 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019916a700 1 -- 172.21.15.32:0/3190474030 learned_addr learned my addr 172.21.15.32:0/3190474030 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f019996b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 0x7f019406f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:45938/0 (socket says 172.21.15.32:45938) 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.076+0000 7f0198969700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f019409e000 0x7f019406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019996b700 1 -- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 msgr2=0x7f019406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019996b700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 0x7f019406e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019996b700 1 -- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f019409e000 msgr2=0x7f019406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019996b700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f019409e000 0x7f019406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019996b700 1 -- 172.21.15.32:0/3190474030 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900094a0 con 0x7f019409e410 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f0198969700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f019409e000 0x7f019406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019916a700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 0x7f019406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019996b700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 0x7f019406f2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f018c00caa0 tx=0x7f018c00cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00a9a0 con 0x7f019409e410 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c00ab00 con 0x7f019409e410 2023-09-30T22:30:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01840094a0 con 0x7f019409e410 2023-09-30T22:30:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00ae70 con 0x7f019409e410 2023-09-30T22:30:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.077+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01941249c0 con 0x7f019409e410 2023-09-30T22:30:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.078+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f018c01b3b0 con 0x7f019409e410 2023-09-30T22:30:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.078+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f018c03f2b0 con 0x7f019409e410 2023-09-30T22:30:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.078+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0194125880 con 0x7f019409e410 2023-09-30T22:30:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.083+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0194125880 con 0x7f019409e410 2023-09-30T22:30:21.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.288+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0194125880 con 0x7f019409e410 2023-09-30T22:30:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.288+0000 7f0189ffb700 1 -- 172.21.15.32:0/3190474030 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0194125880 con 0x7f019409e410 2023-09-30T22:30:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 msgr2=0x7f019406f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 0x7f019406f2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f018c00caa0 tx=0x7f018c00cdb0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 shutdown_connections 2023-09-30T22:30:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f019409e410 0x7f019406f2f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f019409d6f0 0x7f019406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 --2- 172.21.15.32:0/3190474030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f019409e000 0x7f019406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 >> 172.21.15.32:0/3190474030 conn(0x7f0194066f50 msgr2=0x7f019406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 shutdown_connections 2023-09-30T22:30:21.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.291+0000 7f019b3ce700 1 -- 172.21.15.32:0/3190474030 wait complete. 2023-09-30T22:30:21.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:21.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:21.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.399+0000 7fedd100e700 1 Processor -- start 2023-09-30T22:30:21.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.399+0000 7fedd100e700 1 -- start start 2023-09-30T22:30:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedd100e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedcc09d6f0 0x7fedcc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedd100e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 0x7fedcc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedd100e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedd100e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc09f520 con 0x7fedcc09d6f0 2023-09-30T22:30:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedd100e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc09f130 con 0x7fedcc09e000 2023-09-30T22:30:21.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedd100e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc09e930 con 0x7fedcc09e410 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcb59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedcc09d6f0 0x7fedcc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedca59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 0x7fedcc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedcc09d6f0 0x7fedcc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58516/0 (socket says 172.21.15.32:58516) 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedca59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 0x7fedcc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47666/0 (socket says 172.21.15.32:47666) 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcb59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:45948/0 (socket says 172.21.15.32:45948) 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 -- 172.21.15.32:0/3186910210 learned_addr learned my addr 172.21.15.32:0/3186910210 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:21.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 -- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 msgr2=0x7fedcc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 -- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 msgr2=0x7fedcc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 0x7fedcc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.400+0000 7fedcad9d700 1 -- 172.21.15.32:0/3186910210 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc09b2d0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.401+0000 7fedca59c700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 0x7fedcc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.401+0000 7fedcad9d700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09d6f0 0x7fedcc09dac0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00bbc0 tx=0x7fedbc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e996ac2ba35af8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.401+0000 7fedc9d9b700 1 -- 172.21.15.32:0/3186910210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00dcf0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.401+0000 7fedc9d9b700 1 -- 172.21.15.32:0/3186910210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc00de50 con 0x7fedcc09d6f0 2023-09-30T22:30:21.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.401+0000 7fedc9d9b700 1 -- 172.21.15.32:0/3186910210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc010690 con 0x7fedcc09d6f0 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 -- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09d6f0 msgr2=0x7fedcc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09d6f0 0x7fedcc09dac0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00bbc0 tx=0x7fedbc00bed0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 -- 172.21.15.32:0/3186910210 shutdown_connections 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09d6f0 0x7fedcc09dac0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 --2- 172.21.15.32:0/3186910210 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09e000 0x7fedcc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 -- 172.21.15.32:0/3186910210 >> 172.21.15.32:0/3186910210 conn(0x7fedcc066f50 msgr2=0x7fedcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 -- 172.21.15.32:0/3186910210 shutdown_connections 2023-09-30T22:30:21.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.402+0000 7fedd100e700 1 -- 172.21.15.32:0/3186910210 wait complete. 2023-09-30T22:30:21.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.403+0000 7fedd100e700 1 Processor -- start 2023-09-30T22:30:21.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.403+0000 7fedd100e700 1 -- start start 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedd100e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 0x7fedcc11e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedd100e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09e000 0x7fedcc11e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedd100e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc11eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedd100e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc11ff30 con 0x7fedcc09e000 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedd100e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc120550 con 0x7fedcc09d6f0 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedd100e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc130280 con 0x7fedcc09e410 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 0x7fedcc11e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 0x7fedcc11e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47668/0 (socket says 172.21.15.32:47668) 2023-09-30T22:30:21.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 -- 172.21.15.32:0/3794335251 learned_addr learned my addr 172.21.15.32:0/3794335251 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcb59e700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc11eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedca59c700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09e000 0x7fedcc11e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 -- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 msgr2=0x7fedcc11eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc11eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 -- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09e000 msgr2=0x7fedcc11e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09e000 0x7fedcc11e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.404+0000 7fedcad9d700 1 -- 172.21.15.32:0/3794335251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00094a0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.405+0000 7fedcad9d700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 0x7fedcc11e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fedcc09d470 tx=0x7fedbc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.405+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc01f070 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.405+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0038b0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.405+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc00b7f0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.405+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc003c40 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.405+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc09c700 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.406+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fedbc019400 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.406+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc074ea0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.406+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fedbc042f60 con 0x7fedcc09d6f0 2023-09-30T22:30:21.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.411+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fedbc022370 con 0x7fedcc09d6f0 2023-09-30T22:30:21.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 Processor -- start 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 -- start start 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a8134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a809ea10 con 0x7f50a809d5c0 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a809e210 con 0x7f50a80a09c0 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.516+0000 7f50ad893700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80a26f0 con 0x7f50a80a12d0 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a67fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f50a809d5c0 0x7f50a8133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53114/0 (socket says 172.21.15.32:53114) 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a67fc700 1 -- 172.21.15.32:0/1096690596 learned_addr learned my addr 172.21.15.32:0/1096690596 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:21.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a77fe700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a8134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a5ffb700 1 -- 172.21.15.32:0/1096690596 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850976236 0 0) 0x7f50a809ea10 con 0x7f50a809d5c0 2023-09-30T22:30:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a5ffb700 1 -- 172.21.15.32:0/1096690596 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50900035b0 con 0x7f50a809d5c0 2023-09-30T22:30:21.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a77fe700 1 -- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 msgr2=0x7f50a80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a77fe700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a80a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a77fe700 1 -- 172.21.15.32:0/1096690596 >> v1:172.21.15.32:6789/0 conn(0x7f50a809d5c0 legacy=0x7f50a8133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a77fe700 1 -- 172.21.15.32:0/1096690596 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a809db70 con 0x7f50a80a12d0 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a6ffd700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.517+0000 7f50a77fe700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a8134030 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009dc0 tx=0x7f50a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a970973997f0de1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50a5ffb700 1 -- 172.21.15.32:0/1096690596 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a000e070 con 0x7f50a80a12d0 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50a5ffb700 1 -- 172.21.15.32:0/1096690596 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a000bb20 con 0x7f50a80a12d0 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50a5ffb700 1 -- 172.21.15.32:0/1096690596 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0012430 con 0x7f50a80a12d0 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 -- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 msgr2=0x7f50a8134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a8134030 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009dc0 tx=0x7f50a000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:21.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 -- 172.21.15.32:0/1096690596 shutdown_connections 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a8134030 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 --2- 172.21.15.32:0/1096690596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 -- 172.21.15.32:0/1096690596 >> 172.21.15.32:0/1096690596 conn(0x7f50a80a2fe0 msgr2=0x7f50a80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 -- 172.21.15.32:0/1096690596 shutdown_connections 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.518+0000 7f50ad893700 1 -- 172.21.15.32:0/1096690596 wait complete. 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.519+0000 7f50ad893700 1 Processor -- start 2023-09-30T22:30:21.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 -- start start 2023-09-30T22:30:21.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809d5c0 0x7f50a8120410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a8120950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a81c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8120e90 con 0x7f50a809d5c0 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a809cec0 con 0x7f50a80a09c0 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50ad893700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8123230 con 0x7f50a80a12d0 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50a67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a8120950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50a67fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a8120950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47680/0 (socket says 172.21.15.32:47680) 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50a67fc700 1 -- 172.21.15.32:0/3875699435 learned_addr learned my addr 172.21.15.32:0/3875699435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50a77fe700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a81c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.520+0000 7f50a6ffd700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809d5c0 0x7f50a8120410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a67fc700 1 -- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 msgr2=0x7f50a81c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a67fc700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a81c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a67fc700 1 -- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809d5c0 msgr2=0x7f50a8120410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a67fc700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809d5c0 0x7f50a8120410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a67fc700 1 -- 172.21.15.32:0/3875699435 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980094a0 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a77fe700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a81c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50a67fc700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a8120950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f509c005910 tx=0x7f509c00c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c005d70 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c0070d0 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0009a70 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c007460 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.521+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a811ffa0 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.522+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a8123030 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.522+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f509c0075c0 con 0x7f50a80a09c0 2023-09-30T22:30:21.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.522+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f509c040b90 con 0x7f50a80a09c0 2023-09-30T22:30:21.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.527+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f509c044060 con 0x7f50a80a09c0 2023-09-30T22:30:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.554+0000 7f925be54700 1 Processor -- start 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.554+0000 7f925be54700 1 -- start start 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925be54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925be54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925be54700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925409ba70 con 0x7f925409c650 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925be54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925409e8f0 con 0x7f925409d130 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925be54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92540a0760 con 0x7f92540a0090 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f9259bf0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f925409c650 0x7f925409ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53118/0 (socket says 172.21.15.32:53118) 2023-09-30T22:30:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f9259bf0700 1 -- 172.21.15.32:0/4204977572 learned_addr learned my addr 172.21.15.32:0/4204977572 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f92593ef700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925a3f1700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925a3f1700 1 -- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 msgr2=0x7f9254136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925a3f1700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925a3f1700 1 -- 172.21.15.32:0/4204977572 >> v1:172.21.15.32:6789/0 conn(0x7f925409c650 legacy=0x7f925409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f9258bee700 1 -- 172.21.15.32:0/4204977572 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185149697 0 0) 0x7f925409ba70 con 0x7f925409c650 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f925a3f1700 1 -- 172.21.15.32:0/4204977572 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925409a080 con 0x7f925409d130 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.555+0000 7f92593ef700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.556+0000 7f925a3f1700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254139170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f924c009dc0 tx=0x7f924c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa633636059817f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.556+0000 7f9258bee700 1 -- 172.21.15.32:0/4204977572 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00e070 con 0x7f925409d130 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.556+0000 7f9258bee700 1 -- 172.21.15.32:0/4204977572 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c00bb20 con 0x7f925409d130 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.556+0000 7f9258bee700 1 -- 172.21.15.32:0/4204977572 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c012430 con 0x7f925409d130 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.556+0000 7f925be54700 1 -- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 msgr2=0x7f9254139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.556+0000 7f925be54700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254139170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f924c009dc0 tx=0x7f924c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.557+0000 7f925be54700 1 -- 172.21.15.32:0/4204977572 shutdown_connections 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.557+0000 7f925be54700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254139170 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.557+0000 7f925be54700 1 --2- 172.21.15.32:0/4204977572 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.557+0000 7f925be54700 1 -- 172.21.15.32:0/4204977572 >> 172.21.15.32:0/4204977572 conn(0x7f92540a3f70 msgr2=0x7f92540a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.557+0000 7f925be54700 1 -- 172.21.15.32:0/4204977572 shutdown_connections 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.557+0000 7f925be54700 1 -- 172.21.15.32:0/4204977572 wait complete. 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 Processor -- start 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 -- start start 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 0x7f92541350f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925411b2f0 con 0x7f925409c650 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9254138670 con 0x7f925409d130 2023-09-30T22:30:21.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925be54700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92540a18a0 con 0x7f92540a0090 2023-09-30T22:30:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f9259bf0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 0x7f92541350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f9259bf0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 0x7f92541350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58548/0 (socket says 172.21.15.32:58548) 2023-09-30T22:30:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f9259bf0700 1 -- 172.21.15.32:0/3822238936 learned_addr learned my addr 172.21.15.32:0/3822238936 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f92593ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.558+0000 7f925a3f1700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f9259bf0700 1 -- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 msgr2=0x7f9254135b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f9259bf0700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254135b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f9259bf0700 1 -- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 msgr2=0x7f9254135630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f9259bf0700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254135630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f9259bf0700 1 -- 172.21.15.32:0/3822238936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440094a0 con 0x7f925409c650 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f92593ef700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f9259bf0700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 0x7f92541350f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f925000c520 tx=0x7f925000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250004c80 con 0x7f925409c650 2023-09-30T22:30:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9250004e00 con 0x7f925409c650 2023-09-30T22:30:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250007310 con 0x7f925409c650 2023-09-30T22:30:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c009a70 con 0x7f925409c650 2023-09-30T22:30:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.559+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92541374b0 con 0x7f925409c650 2023-09-30T22:30:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.560+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9250007470 con 0x7f925409c650 2023-09-30T22:30:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.560+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f925003f2c0 con 0x7f925409c650 2023-09-30T22:30:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.561+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92540747f0 con 0x7f925409c650 2023-09-30T22:30:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.566+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f92500123b0 con 0x7f925409c650 2023-09-30T22:30:21.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.619+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fedcc0720e0 con 0x7fedcc09d6f0 2023-09-30T22:30:21.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.620+0000 7fedb37fe700 1 -- 172.21.15.32:0/3794335251 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fedbc022370 con 0x7fedcc09d6f0 2023-09-30T22:30:21.621 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 msgr2=0x7fedcc11e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 0x7fedcc11e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fedcc09d470 tx=0x7fedbc004750 comp rx=0 tx=0).stop 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 shutdown_connections 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fedcc09e410 0x7fedcc11eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fedcc09d6f0 0x7fedcc11e320 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 --2- 172.21.15.32:0/3794335251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedcc09e000 0x7fedcc11e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 >> 172.21.15.32:0/3794335251 conn(0x7fedcc066f50 msgr2=0x7fedcc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 shutdown_connections 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.622+0000 7fedd100e700 1 -- 172.21.15.32:0/3794335251 wait complete. 2023-09-30T22:30:21.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.737+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50a80747f0 con 0x7f50a80a09c0 2023-09-30T22:30:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.737+0000 7f508ffff700 1 -- 172.21.15.32:0/3875699435 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f509c017020 con 0x7f50a80a09c0 2023-09-30T22:30:21.737 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.739+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 msgr2=0x7f50a8120950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.739+0000 7f50ad893700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a8120950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f509c005910 tx=0x7f509c00c9e0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 shutdown_connections 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f50a80a12d0 0x7f50a81c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a809d5c0 0x7f50a8120410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 --2- 172.21.15.32:0/3875699435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50a80a09c0 0x7f50a8120950 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 >> 172.21.15.32:0/3875699435 conn(0x7f50a80a2fe0 msgr2=0x7f50a81bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 shutdown_connections 2023-09-30T22:30:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.740+0000 7f50ad893700 1 -- 172.21.15.32:0/3875699435 wait complete. 2023-09-30T22:30:21.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.773+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9254072840 con 0x7f925409c650 2023-09-30T22:30:21.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.773+0000 7f924a7fc700 1 -- 172.21.15.32:0/3822238936 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9250011360 con 0x7f925409c650 2023-09-30T22:30:21.774 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 msgr2=0x7f92541350f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 0x7f92541350f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f925000c520 tx=0x7f925000c830 comp rx=0 tx=0).stop 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 shutdown_connections 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f92540a0090 0x7f9254135b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f925409c650 0x7f92541350f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 --2- 172.21.15.32:0/3822238936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f925409d130 0x7f9254135630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 >> 172.21.15.32:0/3822238936 conn(0x7f92540a3f70 msgr2=0x7f9254138cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 shutdown_connections 2023-09-30T22:30:21.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:21.776+0000 7f925be54700 1 -- 172.21.15.32:0/3822238936 wait complete. 2023-09-30T22:30:21.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:22.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:22.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.452+0000 7f807897e700 1 Processor -- start 2023-09-30T22:30:22.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.452+0000 7f807897e700 1 -- start start 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807897e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f8074135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807897e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 0x7f8074138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807897e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807409f130 con 0x7f807409d6f0 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807897e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807409f520 con 0x7f8074135e50 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807897e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f807409e930 con 0x7f807409e1d0 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f8072d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 0x7f8074138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807259c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f807409d6f0 0x7f807409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53122/0 (socket says 172.21.15.32:53122) 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f8072d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 0x7f8074138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58552/0 (socket says 172.21.15.32:58552) 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f807259c700 1 -- 172.21.15.32:0/3106895749 learned_addr learned my addr 172.21.15.32:0/3106895749 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:22.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.453+0000 7f8071d9b700 1 --2- 172.21.15.32:0/3106895749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f8074135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f807159a700 1 -- 172.21.15.32:0/3106895749 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911674338 0 0) 0x7f807409f130 con 0x7f807409d6f0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f8071d9b700 1 -- 172.21.15.32:0/3106895749 >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 msgr2=0x7f8074138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f8071d9b700 1 --2- 172.21.15.32:0/3106895749 >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 0x7f8074138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f8071d9b700 1 -- 172.21.15.32:0/3106895749 >> v1:172.21.15.32:6789/0 conn(0x7f807409d6f0 legacy=0x7f807409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f8071d9b700 1 -- 172.21.15.32:0/3106895749 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807409b2d0 con 0x7f807409e1d0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f8072d9d700 1 --2- 172.21.15.32:0/3106895749 >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 0x7f8074138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f8071d9b700 1 --2- 172.21.15.32:0/3106895749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f8074135910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8068009870 tx=0x7f8068009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb799e794d80c7a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f807159a700 1 -- 172.21.15.32:0/3106895749 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068011830 con 0x7f807409e1d0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f807159a700 1 -- 172.21.15.32:0/3106895749 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068011990 con 0x7f807409e1d0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.454+0000 7f807159a700 1 -- 172.21.15.32:0/3106895749 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068011d20 con 0x7f807409e1d0 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 -- 172.21.15.32:0/3106895749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 msgr2=0x7f8074135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 --2- 172.21.15.32:0/3106895749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f8074135910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8068009870 tx=0x7f8068009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:22.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 -- 172.21.15.32:0/3106895749 shutdown_connections 2023-09-30T22:30:22.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 --2- 172.21.15.32:0/3106895749 >> v2:172.21.15.32:3300/0 conn(0x7f8074135e50 0x7f8074138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 --2- 172.21.15.32:0/3106895749 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f8074135910 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 -- 172.21.15.32:0/3106895749 >> 172.21.15.32:0/3106895749 conn(0x7f8074066f50 msgr2=0x7f807409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 -- 172.21.15.32:0/3106895749 shutdown_connections 2023-09-30T22:30:22.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.455+0000 7f807897e700 1 -- 172.21.15.32:0/3106895749 wait complete. 2023-09-30T22:30:22.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.456+0000 7f807897e700 1 Processor -- start 2023-09-30T22:30:22.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.456+0000 7f807897e700 1 -- start start 2023-09-30T22:30:22.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f807409d6f0 0x7f807412f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f807412f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 0x7f80741c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807411f7a0 con 0x7f8074135e50 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807411f370 con 0x7f807409d6f0 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f807411fe60 con 0x7f807409e1d0 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 0x7f80741c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f807409d6f0 0x7f807412f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8071d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f807412f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 0x7f80741c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58568/0 (socket says 172.21.15.32:58568) 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f807409d6f0 0x7f807412f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47702/0 (socket says 172.21.15.32:47702) 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 -- 172.21.15.32:0/2991383852 learned_addr learned my addr 172.21.15.32:0/2991383852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 -- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 msgr2=0x7f807412f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f807412f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 -- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f807409d6f0 msgr2=0x7f807412f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f807409d6f0 0x7f807412f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 -- 172.21.15.32:0/2991383852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c00bc80 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8072d9d700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 0x7f80741c8070 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f806c0093d0 tx=0x7f806c00e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c010040 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c00e940 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c00ecf0 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.457+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80680094a0 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.458+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8074120e20 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.458+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f806c009540 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.458+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f806c03fa70 con 0x7f8074135e50 2023-09-30T22:30:22.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.459+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8074074ea0 con 0x7f8074135e50 2023-09-30T22:30:22.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.464+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f806c0143e0 con 0x7f8074135e50 2023-09-30T22:30:22.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:22.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.669+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f807411f9e0 con 0x7f8074135e50 2023-09-30T22:30:22.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.669+0000 7f8062ffd700 1 -- 172.21.15.32:0/2991383852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f806c01a020 con 0x7f8074135e50 2023-09-30T22:30:22.670 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 msgr2=0x7f80741c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 0x7f80741c8070 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f806c0093d0 tx=0x7f806c00e3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 shutdown_connections 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8074135e50 0x7f80741c8070 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f807409d6f0 0x7f807412f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 --2- 172.21.15.32:0/2991383852 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f807409e1d0 0x7f807412f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 >> 172.21.15.32:0/2991383852 conn(0x7f8074066f50 msgr2=0x7f80741327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 shutdown_connections 2023-09-30T22:30:22.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.672+0000 7f807897e700 1 -- 172.21.15.32:0/2991383852 wait complete. 2023-09-30T22:30:22.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:22.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:22.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.777+0000 7f0ce7b87700 1 Processor -- start 2023-09-30T22:30:22.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.777+0000 7f0ce7b87700 1 -- start start 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce7b87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 0x7f0ce0131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce7b87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce009d9f0 0x7f0ce006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce7b87700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce009e130 con 0x7f0ce006acb0 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce7b87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01312b0 con 0x7f0ce009d9f0 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce7b87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce009f6e0 con 0x7f0ce01316c0 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 0x7f0ce0131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5122700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce009d9f0 0x7f0ce006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 0x7f0ce0131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46020/0 (socket says 172.21.15.32:46020) 2023-09-30T22:30:22.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 -- 172.21.15.32:0/3111773477 learned_addr learned my addr 172.21.15.32:0/3111773477 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 -- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce009d9f0 msgr2=0x7f0ce006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 --2- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce009d9f0 0x7f0ce006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce4921700 1 -- 172.21.15.32:0/3111773477 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130957145 0 0) 0x7f0cd8001490 con 0x7f0ce006acb0 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 -- 172.21.15.32:0/3111773477 >> v1:172.21.15.32:6789/0 conn(0x7f0ce006acb0 legacy=0x7f0ce0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5923700 1 -- 172.21.15.32:0/3111773477 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce009e760 con 0x7f0ce01316c0 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.778+0000 7f0ce5122700 1 --2- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce009d9f0 0x7f0ce006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.779+0000 7f0ce5923700 1 --2- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 0x7f0ce0131a90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009920 tx=0x7f0cdc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d5b05f91060d393 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.779+0000 7f0ce4921700 1 -- 172.21.15.32:0/3111773477 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc011830 con 0x7f0ce01316c0 2023-09-30T22:30:22.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.779+0000 7f0ce4921700 1 -- 172.21.15.32:0/3111773477 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc011990 con 0x7f0ce01316c0 2023-09-30T22:30:22.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.779+0000 7f0ce4921700 1 -- 172.21.15.32:0/3111773477 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc011d20 con 0x7f0ce01316c0 2023-09-30T22:30:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.779+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 msgr2=0x7f0ce0131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.779+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 0x7f0ce0131a90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009920 tx=0x7f0cdc009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.780+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3111773477 shutdown_connections 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.780+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce01316c0 0x7f0ce0131a90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.780+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3111773477 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce009d9f0 0x7f0ce006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.780+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3111773477 >> 172.21.15.32:0/3111773477 conn(0x7f0ce0066f50 msgr2=0x7f0ce009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.780+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3111773477 shutdown_connections 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.780+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3111773477 wait complete. 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.781+0000 7f0ce7b87700 1 Processor -- start 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.781+0000 7f0ce7b87700 1 -- start start 2023-09-30T22:30:22.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce7b87700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce006acb0 0x7f0ce0070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce7b87700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce009d9f0 0x7f0ce0070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce7b87700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce01316c0 0x7f0ce0071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce7b87700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce011ae20 con 0x7f0ce006acb0 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce7b87700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce012b900 con 0x7f0ce01316c0 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce7b87700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0130ac0 con 0x7f0ce009d9f0 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce5923700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce006acb0 0x7f0ce0070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce6124700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce01316c0 0x7f0ce0071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce5923700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce006acb0 0x7f0ce0070790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58580/0 (socket says 172.21.15.32:58580) 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce5923700 1 -- 172.21.15.32:0/3573675921 learned_addr learned my addr 172.21.15.32:0/3573675921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:22.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce5122700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce009d9f0 0x7f0ce0070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce6124700 1 -- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce009d9f0 msgr2=0x7f0ce0070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce6124700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce009d9f0 0x7f0ce0070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce6124700 1 -- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce006acb0 msgr2=0x7f0ce0070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce6124700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce006acb0 0x7f0ce0070790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.782+0000 7f0ce6124700 1 -- 172.21.15.32:0/3573675921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd00094a0 con 0x7f0ce01316c0 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.783+0000 7f0ce6124700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce01316c0 0x7f0ce0071210 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c7c0 tx=0x7f0cd800cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.783+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8003e30 con 0x7f0ce01316c0 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.783+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8005070 con 0x7f0ce01316c0 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.783+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc009550 con 0x7f0ce01316c0 2023-09-30T22:30:22.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.783+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8005420 con 0x7f0ce01316c0 2023-09-30T22:30:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.783+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0119d80 con 0x7f0ce01316c0 2023-09-30T22:30:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.784+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cd8005580 con 0x7f0ce01316c0 2023-09-30T22:30:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.784+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0cd803ea70 con 0x7f0ce01316c0 2023-09-30T22:30:22.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.784+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce009e130 con 0x7f0ce01316c0 2023-09-30T22:30:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.789+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ce009e130 con 0x7f0ce01316c0 2023-09-30T22:30:22.790 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.916+0000 7f8f8b214700 1 Processor -- start 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.916+0000 7f8f8b214700 1 -- start start 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f8b214700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f8b214700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f8b214700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f8409e410 0x7f8f84133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f8b214700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8406b460 con 0x7f8f8409e410 2023-09-30T22:30:22.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f8b214700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8409b120 con 0x7f8f8409d6f0 2023-09-30T22:30:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f8b214700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8409a7e0 con 0x7f8f8409e000 2023-09-30T22:30:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f88fb0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f88fb0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f8409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47726/0 (socket says 172.21.15.32:47726) 2023-09-30T22:30:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f88fb0700 1 -- 172.21.15.32:0/744694927 learned_addr learned my addr 172.21.15.32:0/744694927 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f83fff700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f897b1700 1 --2- 172.21.15.32:0/744694927 >> v2:172.21.15.32:3300/0 conn(0x7f8f8409e410 0x7f8f84133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f83fff700 1 -- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 msgr2=0x7f8f8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f83fff700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f8409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f83fff700 1 -- 172.21.15.32:0/744694927 >> v2:172.21.15.32:3300/0 conn(0x7f8f8409e410 msgr2=0x7f8f84133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f83fff700 1 --2- 172.21.15.32:0/744694927 >> v2:172.21.15.32:3300/0 conn(0x7f8f8409e410 0x7f8f84133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f83fff700 1 -- 172.21.15.32:0/744694927 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8409a500 con 0x7f8f8409e000 2023-09-30T22:30:22.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.917+0000 7f8f88fb0700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f83fff700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841339f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009870 tx=0x7f8f70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ef238fee5506cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f837fe700 1 -- 172.21.15.32:0/744694927 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70011830 con 0x7f8f8409e000 2023-09-30T22:30:22.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f837fe700 1 -- 172.21.15.32:0/744694927 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70011990 con 0x7f8f8409e000 2023-09-30T22:30:22.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f837fe700 1 -- 172.21.15.32:0/744694927 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70011d20 con 0x7f8f8409e000 2023-09-30T22:30:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 -- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 msgr2=0x7f8f841339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841339f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009870 tx=0x7f8f70009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 -- 172.21.15.32:0/744694927 shutdown_connections 2023-09-30T22:30:22.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 --2- 172.21.15.32:0/744694927 >> v2:172.21.15.32:3300/0 conn(0x7f8f8409e410 0x7f8f84133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 --2- 172.21.15.32:0/744694927 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841339f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.918+0000 7f8f8b214700 1 -- 172.21.15.32:0/744694927 >> 172.21.15.32:0/744694927 conn(0x7f8f84066f50 msgr2=0x7f8f8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.919+0000 7f8f8b214700 1 -- 172.21.15.32:0/744694927 shutdown_connections 2023-09-30T22:30:22.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.919+0000 7f8f8b214700 1 -- 172.21.15.32:0/744694927 wait complete. 2023-09-30T22:30:22.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.920+0000 7f8f8b214700 1 Processor -- start 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.920+0000 7f8f8b214700 1 -- start start 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f8b214700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f841a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f8b214700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f8b214700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f8409e410 0x7f8f841a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f8b214700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841aa280 con 0x7f8f8409e410 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f8b214700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f841a98c0 con 0x7f8f8409d6f0 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f8b214700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840a1f90 con 0x7f8f8409e000 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46050/0 (socket says 172.21.15.32:46050) 2023-09-30T22:30:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f83fff700 1 -- 172.21.15.32:0/1544491321 learned_addr learned my addr 172.21.15.32:0/1544491321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f88fb0700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f841a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.921+0000 7f8f897b1700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f8409e410 0x7f8f841a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f83fff700 1 -- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 msgr2=0x7f8f841a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f83fff700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f841a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f83fff700 1 -- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f8409e410 msgr2=0x7f8f841a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f83fff700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f8409e410 0x7f8f841a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f83fff700 1 -- 172.21.15.32:0/1544491321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009550 con 0x7f8f8409e000 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f83fff700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841a6330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8f700090b0 tx=0x7f8f70004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70004450 con 0x7f8f8409e000 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7001f070 con 0x7f8f8409e000 2023-09-30T22:30:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f700094a0 con 0x7f8f8409e000 2023-09-30T22:30:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700038a0 con 0x7f8f8409e000 2023-09-30T22:30:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.922+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f841a79b0 con 0x7f8f8409e000 2023-09-30T22:30:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.923+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f70003a00 con 0x7f8f8409e000 2023-09-30T22:30:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.924+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8f70021070 con 0x7f8f8409e000 2023-09-30T22:30:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.924+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f841a9280 con 0x7f8f8409e000 2023-09-30T22:30:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.930+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f841a9280 con 0x7f8f8409e000 2023-09-30T22:30:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.931+0000 7f05fef36700 1 Processor -- start 2023-09-30T22:30:22.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.931+0000 7f05fef36700 1 -- start start 2023-09-30T22:30:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fef36700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 0x7f05f8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fef36700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05f8139170 0x7f05f813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fef36700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f809b120 con 0x7f05f809d6f0 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fef36700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f806b460 con 0x7f05f8139170 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fef36700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05f809a7e0 con 0x7f05f809e1d0 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 0x7f05f8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05f8139170 0x7f05f813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fccd2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05f809d6f0 0x7f05f809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53148/0 (socket says 172.21.15.32:53148) 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05f8139170 0x7f05f813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58604/0 (socket says 172.21.15.32:58604) 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05f7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 0x7f05f8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46052/0 (socket says 172.21.15.32:46052) 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fccd2700 1 -- 172.21.15.32:0/3585325026 learned_addr learned my addr 172.21.15.32:0/3585325026 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 -- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 msgr2=0x7f05f8138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 --2- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 0x7f05f8138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 -- 172.21.15.32:0/3585325026 >> v1:172.21.15.32:6789/0 conn(0x7f05f809d6f0 legacy=0x7f05f809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 -- 172.21.15.32:0/3585325026 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f809a500 con 0x7f05f8139170 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05f77fe700 1 -- 172.21.15.32:0/3585325026 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140592551 0 0) 0x7f05f809b120 con 0x7f05f809d6f0 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05f7fff700 1 --2- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 0x7f05f8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.932+0000 7f05fd4d3700 1 --2- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f8139170 0x7f05f813b550 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d90 tx=0x7f05f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780d510c045454db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05f77fe700 1 -- 172.21.15.32:0/3585325026 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000e070 con 0x7f05f8139170 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05f77fe700 1 -- 172.21.15.32:0/3585325026 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000bb20 con 0x7f05f8139170 2023-09-30T22:30:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05f77fe700 1 -- 172.21.15.32:0/3585325026 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0012430 con 0x7f05f8139170 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 -- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f8139170 msgr2=0x7f05f813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 --2- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f8139170 0x7f05f813b550 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d90 tx=0x7f05f000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 -- 172.21.15.32:0/3585325026 shutdown_connections 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 --2- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f8139170 0x7f05f813b550 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 --2- 172.21.15.32:0/3585325026 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809e1d0 0x7f05f8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 -- 172.21.15.32:0/3585325026 >> 172.21.15.32:0/3585325026 conn(0x7f05f8066f50 msgr2=0x7f05f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 -- 172.21.15.32:0/3585325026 shutdown_connections 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.933+0000 7f05fef36700 1 -- 172.21.15.32:0/3585325026 wait complete. 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.934+0000 7f05fef36700 1 Processor -- start 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.934+0000 7f05fef36700 1 -- start start 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fef36700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 0x7f05f81c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fef36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f809e1d0 0x7f05f81c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fef36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05f8139170 0x7f05f81217b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fef36700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8137680 con 0x7f05f809e1d0 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fef36700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f81233c0 con 0x7f05f8139170 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fef36700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80a20d0 con 0x7f05f809d6f0 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fccd2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 0x7f05f81c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f809e1d0 0x7f05f81c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fccd2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 0x7f05f81c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46056/0 (socket says 172.21.15.32:46056) 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fccd2700 1 -- 172.21.15.32:0/2433071788 learned_addr learned my addr 172.21.15.32:0/2433071788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.935+0000 7f05fd4d3700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05f8139170 0x7f05f81217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fccd2700 1 -- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05f8139170 msgr2=0x7f05f81217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fccd2700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05f8139170 0x7f05f81217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fccd2700 1 -- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f809e1d0 msgr2=0x7f05f81c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fccd2700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f809e1d0 0x7f05f81c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fccd2700 1 -- 172.21.15.32:0/2433071788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40094a0 con 0x7f05f809d6f0 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05f7fff700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f809e1d0 0x7f05f81c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fccd2700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 0x7f05f81c8460 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c520 tx=0x7f05ec00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec004c80 con 0x7f05f809d6f0 2023-09-30T22:30:22.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec004e00 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f0009a70 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec007310 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.936+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f8125a60 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.937+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05ec007470 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.937+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f8074ea0 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.937+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f05ec03f230 con 0x7f05f809d6f0 2023-09-30T22:30:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:22.942+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05ec0133c0 con 0x7f05f809d6f0 2023-09-30T22:30:23.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.051+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0ce009e130 con 0x7f0ce01316c0 2023-09-30T22:30:23.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.051+0000 7f0cd67fc700 1 -- 172.21.15.32:0/3573675921 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ce009e130 con 0x7f0ce01316c0 2023-09-30T22:30:23.051 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.053+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce01316c0 msgr2=0x7f0ce0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.053+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce01316c0 0x7f0ce0071210 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800c7c0 tx=0x7f0cd800cad0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 shutdown_connections 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ce01316c0 0x7f0ce0071210 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ce006acb0 0x7f0ce0070790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 --2- 172.21.15.32:0/3573675921 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0ce009d9f0 0x7f0ce0070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 >> 172.21.15.32:0/3573675921 conn(0x7f0ce0066f50 msgr2=0x7f0ce00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 shutdown_connections 2023-09-30T22:30:23.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.054+0000 7f0ce7b87700 1 -- 172.21.15.32:0/3573675921 wait complete. 2023-09-30T22:30:23.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:23.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.152+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f05f8124c90 con 0x7f05f809d6f0 2023-09-30T22:30:23.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.152+0000 7f05f57fa700 1 -- 172.21.15.32:0/2433071788 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05ec011360 con 0x7f05f809d6f0 2023-09-30T22:30:23.153 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 msgr2=0x7f05f81c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 0x7f05f81c8460 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c520 tx=0x7f05ec00c830 comp rx=0 tx=0).stop 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 shutdown_connections 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05f8139170 0x7f05f81217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f05f809d6f0 0x7f05f81c8460 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 --2- 172.21.15.32:0/2433071788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05f809e1d0 0x7f05f81c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 >> 172.21.15.32:0/2433071788 conn(0x7f05f8066f50 msgr2=0x7f05f8070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 shutdown_connections 2023-09-30T22:30:23.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.155+0000 7f05fef36700 1 -- 172.21.15.32:0/2433071788 wait complete. 2023-09-30T22:30:23.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.164+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8f841a9280 con 0x7f8f8409e000 2023-09-30T22:30:23.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.165+0000 7f8f817fa700 1 -- 172.21.15.32:0/1544491321 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f841a9280 con 0x7f8f8409e000 2023-09-30T22:30:23.165 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 msgr2=0x7f8f841a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841a6330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8f700090b0 tx=0x7f8f70004750 comp rx=0 tx=0).stop 2023-09-30T22:30:23.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 shutdown_connections 2023-09-30T22:30:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f8409e410 0x7f8f841a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f8409d6f0 0x7f8f841a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 --2- 172.21.15.32:0/1544491321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f8409e000 0x7f8f841a6330 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 >> 172.21.15.32:0/1544491321 conn(0x7f8f84066f50 msgr2=0x7f8f841bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 shutdown_connections 2023-09-30T22:30:23.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.167+0000 7f8f8b214700 1 -- 172.21.15.32:0/1544491321 wait complete. 2023-09-30T22:30:23.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:23.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:23.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:23.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.835+0000 7f2eea8c3700 1 Processor -- start 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.835+0000 7f2eea8c3700 1 -- start start 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2eea8c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2eea8c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee40a1340 0x7f2ee4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2eea8c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2eea8c3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee409edd0 con 0x7f2ee40a1340 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2eea8c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee409e1f0 con 0x7f2ee40a0a30 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2eea8c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee409dc50 con 0x7f2ee4133fa0 2023-09-30T22:30:23.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee8e60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee40a1340 0x7f2ee4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee40a1340 0x7f2ee4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58624/0 (socket says 172.21.15.32:58624) 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee8e60700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46072/0 (socket says 172.21.15.32:46072) 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 -- 172.21.15.32:0/1856780307 learned_addr learned my addr 172.21.15.32:0/1856780307 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 -- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 msgr2=0x7f2ee4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 -- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 msgr2=0x7f2ee40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 -- 172.21.15.32:0/1856780307 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40a1fe0 con 0x7f2ee40a1340 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee3fff700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee8e60700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.836+0000 7f2ee37fe700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee4133a60 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0098f0 tx=0x7f2ecc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ccc8b4f5ea1231 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/1856780307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc011830 con 0x7f2ee40a1340 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/1856780307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc011990 con 0x7f2ee40a1340 2023-09-30T22:30:23.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/1856780307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc011d20 con 0x7f2ee40a1340 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 -- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 msgr2=0x7f2ee4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee4133a60 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0098f0 tx=0x7f2ecc009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 -- 172.21.15.32:0/1856780307 shutdown_connections 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/1856780307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee4133a60 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 -- 172.21.15.32:0/1856780307 >> 172.21.15.32:0/1856780307 conn(0x7f2ee40a6550 msgr2=0x7f2ee40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 -- 172.21.15.32:0/1856780307 shutdown_connections 2023-09-30T22:30:23.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.837+0000 7f2eea8c3700 1 -- 172.21.15.32:0/1856780307 wait complete. 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.838+0000 7f2eea8c3700 1 Processor -- start 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 -- start start 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee41213a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee41218e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4121e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40a0220 con 0x7f2ee40a1340 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee41237d0 con 0x7f2ee40a0a30 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2eea8c3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40a36e0 con 0x7f2ee4133fa0 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2ee3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee41213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:23.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2ee3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee41213a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47770/0 (socket says 172.21.15.32:47770) 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2ee37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee41218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2ee3fff700 1 -- 172.21.15.32:0/2540404723 learned_addr learned my addr 172.21.15.32:0/2540404723 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2ee8e60700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4121e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.839+0000 7f2ee37fe700 1 -- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 msgr2=0x7f2ee4121e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee37fe700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4121e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee37fe700 1 -- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 msgr2=0x7f2ee41213a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee37fe700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee41213a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee37fe700 1 -- 172.21.15.32:0/2540404723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40094a0 con 0x7f2ee40a1340 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee8e60700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee37fe700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee41218e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0098f0 tx=0x7f2ecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:23.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc004240 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc01f070 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0045c0 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc009a70 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.840+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc009550 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.841+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2ecc021030 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.841+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2ecc0406e0 con 0x7f2ee40a1340 2023-09-30T22:30:23.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.841+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4074ea0 con 0x7f2ee40a1340 2023-09-30T22:30:23.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:23.846+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ecc01a450 con 0x7f2ee40a1340 2023-09-30T22:30:24.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.049+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2ee40a28e0 con 0x7f2ee40a1340 2023-09-30T22:30:24.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.049+0000 7f2ee0ff9700 1 -- 172.21.15.32:0/2540404723 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ecc0233d0 con 0x7f2ee40a1340 2023-09-30T22:30:24.049 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 msgr2=0x7f2ee41218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee41218e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0098f0 tx=0x7f2ecc004750 comp rx=0 tx=0).stop 2023-09-30T22:30:24.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 shutdown_connections 2023-09-30T22:30:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2ee4133fa0 0x7f2ee4121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2ee40a0a30 0x7f2ee41213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 --2- 172.21.15.32:0/2540404723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40a1340 0x7f2ee41218e0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 >> 172.21.15.32:0/2540404723 conn(0x7f2ee40a6550 msgr2=0x7f2ee40a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 shutdown_connections 2023-09-30T22:30:24.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.051+0000 7f2eea8c3700 1 -- 172.21.15.32:0/2540404723 wait complete. 2023-09-30T22:30:24.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:24.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:24.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:24.182 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 Processor -- start 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 -- start start 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009f410 0x7f2a40136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a40136fe0 0x7f2a401393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a40070940 con 0x7f2a4009e930 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4006fd60 con 0x7f2a40136fe0 2023-09-30T22:30:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.209+0000 7f2a47c21700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4006fa80 con 0x7f2a4009f410 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a40136fe0 0x7f2a401393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a459bd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a4009e930 0x7f2a4009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53160/0 (socket says 172.21.15.32:53160) 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a459bd700 1 -- 172.21.15.32:0/2153424748 learned_addr learned my addr 172.21.15.32:0/2153424748 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a40136fe0 0x7f2a401393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58638/0 (socket says 172.21.15.32:58638) 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a451bc700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009f410 0x7f2a40136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 -- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009f410 msgr2=0x7f2a40136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009f410 0x7f2a40136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 -- 172.21.15.32:0/2153424748 >> v1:172.21.15.32:6789/0 conn(0x7f2a4009e930 legacy=0x7f2a4009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 -- 172.21.15.32:0/2153424748 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4009fb60 con 0x7f2a40136fe0 2023-09-30T22:30:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a451bc700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009f410 0x7f2a40136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a461be700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40136fe0 0x7f2a401393c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009d90 tx=0x7f2a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06c51550cf15770 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a449bb700 1 -- 172.21.15.32:0/2153424748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800e070 con 0x7f2a40136fe0 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a449bb700 1 -- 172.21.15.32:0/2153424748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3800bb20 con 0x7f2a40136fe0 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.210+0000 7f2a449bb700 1 -- 172.21.15.32:0/2153424748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38012430 con 0x7f2a40136fe0 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 -- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40136fe0 msgr2=0x7f2a401393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40136fe0 0x7f2a401393c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009d90 tx=0x7f2a3800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 -- 172.21.15.32:0/2153424748 shutdown_connections 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a40136fe0 0x7f2a401393c0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 --2- 172.21.15.32:0/2153424748 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009f410 0x7f2a40136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 -- 172.21.15.32:0/2153424748 >> 172.21.15.32:0/2153424748 conn(0x7f2a400a4210 msgr2=0x7f2a400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 -- 172.21.15.32:0/2153424748 shutdown_connections 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.211+0000 7f2a47c21700 1 -- 172.21.15.32:0/2153424748 wait complete. 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 Processor -- start 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 -- start start 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009e930 0x7f2a401c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 0x7f2a401c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a40136fe0 0x7f2a4006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40124000 con 0x7f2a4009f410 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401bc0c0 con 0x7f2a40136fe0 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a47c21700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40134370 con 0x7f2a4009e930 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a451bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 0x7f2a401c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a451bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 0x7f2a401c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58646/0 (socket says 172.21.15.32:58646) 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a451bc700 1 -- 172.21.15.32:0/510212090 learned_addr learned my addr 172.21.15.32:0/510212090 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.213+0000 7f2a461be700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a40136fe0 0x7f2a4006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a459bd700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009e930 0x7f2a401c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a461be700 1 -- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009e930 msgr2=0x7f2a401c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a461be700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009e930 0x7f2a401c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a461be700 1 -- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 msgr2=0x7f2a401c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a461be700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 0x7f2a401c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a461be700 1 -- 172.21.15.32:0/510212090 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300094a0 con 0x7f2a40136fe0 2023-09-30T22:30:24.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a451bc700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 0x7f2a401c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a461be700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a40136fe0 0x7f2a4006b8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2a38016000 tx=0x7f2a38004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38012430 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a38020070 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38004040 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a38009a70 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.214+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a40122bc0 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.215+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2a380041a0 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.216+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2a38022070 con 0x7f2a40136fe0 2023-09-30T22:30:24.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.216+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40074ea0 con 0x7f2a40136fe0 2023-09-30T22:30:24.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.222+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2a3802c020 con 0x7f2a40136fe0 2023-09-30T22:30:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.316+0000 7f9d551ff700 1 Processor -- start 2023-09-30T22:30:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.316+0000 7f9d551ff700 1 -- start start 2023-09-30T22:30:24.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.316+0000 7f9d551ff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d5009e930 0x7f9d5009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d551ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f650 0x7f9d50131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d551ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d50070940 con 0x7f9d5009f240 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d551ff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5006fd60 con 0x7f9d5009e930 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d551ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5006fa80 con 0x7f9d5009f650 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4f59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f650 0x7f9d50131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d5009f240 0x7f9d501317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53168/0 (socket says 172.21.15.32:53168) 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4659c700 1 -- 172.21.15.32:0/3482655039 learned_addr learned my addr 172.21.15.32:0/3482655039 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:24.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4ed9d700 1 --2- 172.21.15.32:0/3482655039 >> v2:172.21.15.32:3300/0 conn(0x7f9d5009e930 0x7f9d5009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4f59e700 1 -- 172.21.15.32:0/3482655039 >> v2:172.21.15.32:3300/0 conn(0x7f9d5009e930 msgr2=0x7f9d5009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4f59e700 1 --2- 172.21.15.32:0/3482655039 >> v2:172.21.15.32:3300/0 conn(0x7f9d5009e930 0x7f9d5009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4f59e700 1 -- 172.21.15.32:0/3482655039 >> v1:172.21.15.32:6789/0 conn(0x7f9d5009f240 legacy=0x7f9d501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4f59e700 1 -- 172.21.15.32:0/3482655039 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5009fb60 con 0x7f9d5009f650 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4e59c700 1 -- 172.21.15.32:0/3482655039 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036672899 0 0) 0x7f9d50070940 con 0x7f9d5009f240 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.317+0000 7f9d4f59e700 1 --2- 172.21.15.32:0/3482655039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f650 0x7f9d50131f00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009d90 tx=0x7f9d4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a980d416ebc7ceb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d4e59c700 1 -- 172.21.15.32:0/3482655039 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800e070 con 0x7f9d5009f650 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d4e59c700 1 -- 172.21.15.32:0/3482655039 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800bb20 con 0x7f9d5009f650 2023-09-30T22:30:24.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d4e59c700 1 -- 172.21.15.32:0/3482655039 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48012430 con 0x7f9d5009f650 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 -- 172.21.15.32:0/3482655039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f650 msgr2=0x7f9d50131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 --2- 172.21.15.32:0/3482655039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f650 0x7f9d50131f00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009d90 tx=0x7f9d4800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 -- 172.21.15.32:0/3482655039 shutdown_connections 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 --2- 172.21.15.32:0/3482655039 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f650 0x7f9d50131f00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 --2- 172.21.15.32:0/3482655039 >> v2:172.21.15.32:3300/0 conn(0x7f9d5009e930 0x7f9d5009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 -- 172.21.15.32:0/3482655039 >> 172.21.15.32:0/3482655039 conn(0x7f9d500a4210 msgr2=0x7f9d500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 -- 172.21.15.32:0/3482655039 shutdown_connections 2023-09-30T22:30:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.318+0000 7f9d551ff700 1 -- 172.21.15.32:0/3482655039 wait complete. 2023-09-30T22:30:24.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.320+0000 7f9d551ff700 1 Processor -- start 2023-09-30T22:30:24.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.320+0000 7f9d551ff700 1 -- start start 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d551ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 0x7f9d501a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d551ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 0x7f9d501a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d551ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5009f650 0x7f9d501c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d551ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d501a7250 con 0x7f9d5009e930 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d551ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d501a6730 con 0x7f9d5009f650 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d551ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d501a6f90 con 0x7f9d5009f240 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d4ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 0x7f9d501a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d4659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 0x7f9d501a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d4ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 0x7f9d501a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58662/0 (socket says 172.21.15.32:58662) 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d4659c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 0x7f9d501a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46104/0 (socket says 172.21.15.32:46104) 2023-09-30T22:30:24.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.321+0000 7f9d4ed9d700 1 -- 172.21.15.32:0/1177924330 learned_addr learned my addr 172.21.15.32:0/1177924330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4f59e700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5009f650 0x7f9d501c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4659c700 1 -- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5009f650 msgr2=0x7f9d501c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4659c700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5009f650 0x7f9d501c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4659c700 1 -- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 msgr2=0x7f9d501a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4659c700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 0x7f9d501a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4659c700 1 -- 172.21.15.32:0/1177924330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48009a70 con 0x7f9d5009f240 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4ed9d700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 0x7f9d501a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4f59e700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5009f650 0x7f9d501c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d4659c700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 0x7f9d501a4910 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c470 tx=0x7f9d3800c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38004c80 con 0x7f9d5009f240 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d38004de0 con 0x7f9d5009f240 2023-09-30T22:30:24.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d500a0190 con 0x7f9d5009f240 2023-09-30T22:30:24.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38007310 con 0x7f9d5009f240 2023-09-30T22:30:24.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.322+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d501a6290 con 0x7f9d5009f240 2023-09-30T22:30:24.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.323+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9d38007470 con 0x7f9d5009f240 2023-09-30T22:30:24.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.323+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9d3803f280 con 0x7f9d5009f240 2023-09-30T22:30:24.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.324+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d501a6ca0 con 0x7f9d5009f240 2023-09-30T22:30:24.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.333+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d38007720 con 0x7f9d5009f240 2023-09-30T22:30:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.357+0000 7fdd4ac37700 1 Processor -- start 2023-09-30T22:30:24.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.357+0000 7fdd4ac37700 1 -- start start 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd4ac37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412cc20 0x7fdd4412f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd4ac37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 0x7fdd4412fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd4ac37700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd4412cff0 0x7fdd441303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd4ac37700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4409a510 con 0x7fdd4412cff0 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd4ac37700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4412c7d0 con 0x7fdd4412cc20 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd4ac37700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4409c190 con 0x7fdd4412c2f0 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 0x7fdd4412fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 0x7fdd4412fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46106/0 (socket says 172.21.15.32:46106) 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 -- 172.21.15.32:0/1751022104 learned_addr learned my addr 172.21.15.32:0/1751022104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd491d4700 1 --2- 172.21.15.32:0/1751022104 >> v2:172.21.15.32:3300/0 conn(0x7fdd4412cff0 0x7fdd441303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd489d3700 1 --2- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412cc20 0x7fdd4412f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 -- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412cc20 msgr2=0x7fdd4412f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 --2- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412cc20 0x7fdd4412f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 -- 172.21.15.32:0/1751022104 >> v2:172.21.15.32:3300/0 conn(0x7fdd4412cff0 msgr2=0x7fdd441303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 --2- 172.21.15.32:0/1751022104 >> v2:172.21.15.32:3300/0 conn(0x7fdd4412cff0 0x7fdd441303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd43fff700 1 -- 172.21.15.32:0/1751022104 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4409b540 con 0x7fdd4412c2f0 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.358+0000 7fdd491d4700 1 --2- 172.21.15.32:0/1751022104 >> v2:172.21.15.32:3300/0 conn(0x7fdd4412cff0 0x7fdd441303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd43fff700 1 --2- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 0x7fdd4412fe70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000bb90 tx=0x7fdd3000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5059979d127c8366 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd437fe700 1 -- 172.21.15.32:0/1751022104 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3000dc70 con 0x7fdd4412c2f0 2023-09-30T22:30:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd437fe700 1 -- 172.21.15.32:0/1751022104 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3000ddd0 con 0x7fdd4412c2f0 2023-09-30T22:30:24.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd437fe700 1 -- 172.21.15.32:0/1751022104 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30012990 con 0x7fdd4412c2f0 2023-09-30T22:30:24.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 msgr2=0x7fdd4412fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 0x7fdd4412fe70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000bb90 tx=0x7fdd3000bea0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1751022104 shutdown_connections 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412cc20 0x7fdd4412f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1751022104 >> v2:172.21.15.32:3300/0 conn(0x7fdd4412cff0 0x7fdd441303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1751022104 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412c2f0 0x7fdd4412fe70 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.359+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1751022104 >> 172.21.15.32:0/1751022104 conn(0x7fdd4409f470 msgr2=0x7fdd440a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.360+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1751022104 shutdown_connections 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.360+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1751022104 wait complete. 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 Processor -- start 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 -- start start 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 0x7fdd4412eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412cc20 0x7fdd4412f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd4412cff0 0x7fdd4412f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4406fea0 con 0x7fdd4412cff0 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4406ed20 con 0x7fdd4412c2f0 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.361+0000 7fdd4ac37700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4412ff60 con 0x7fdd4412cc20 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 0x7fdd4412eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 0x7fdd4412eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47792/0 (socket says 172.21.15.32:47792) 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 -- 172.21.15.32:0/1186128955 learned_addr learned my addr 172.21.15.32:0/1186128955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd43fff700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412cc20 0x7fdd4412f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd491d4700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd4412cff0 0x7fdd4412f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 -- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412cc20 msgr2=0x7fdd4412f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412cc20 0x7fdd4412f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 -- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd4412cff0 msgr2=0x7fdd4412f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd4412cff0 0x7fdd4412f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 -- 172.21.15.32:0/1186128955 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009a70 con 0x7fdd4412c2f0 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd489d3700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 0x7fdd4412eb10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800e9b0 tx=0x7fdd3800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38009710 con 0x7fdd4412c2f0 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3800ee40 con 0x7fdd4412c2f0 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.362+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3000b7f0 con 0x7fdd4412c2f0 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.363+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800ca50 con 0x7fdd4412c2f0 2023-09-30T22:30:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.363+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4406ea00 con 0x7fdd4412c2f0 2023-09-30T22:30:24.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.363+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdd3800cbb0 con 0x7fdd4412c2f0 2023-09-30T22:30:24.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.364+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdd3803f6d0 con 0x7fdd4412c2f0 2023-09-30T22:30:24.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.364+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd440747f0 con 0x7fdd4412c2f0 2023-09-30T22:30:24.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.368+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdd380232f0 con 0x7fdd4412c2f0 2023-09-30T22:30:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.469+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2a40124140 con 0x7f2a40136fe0 2023-09-30T22:30:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.469+0000 7f2a367fc700 1 -- 172.21.15.32:0/510212090 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a40124140 con 0x7f2a40136fe0 2023-09-30T22:30:24.469 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a40136fe0 msgr2=0x7f2a4006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a40136fe0 0x7f2a4006b8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2a38016000 tx=0x7f2a38004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 shutdown_connections 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2a40136fe0 0x7f2a4006b8b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2a4009e930 0x7f2a401c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 --2- 172.21.15.32:0/510212090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a4009f410 0x7f2a401c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.471+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 >> 172.21.15.32:0/510212090 conn(0x7f2a400a4210 msgr2=0x7f2a400a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.472+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 shutdown_connections 2023-09-30T22:30:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.472+0000 7f2a47c21700 1 -- 172.21.15.32:0/510212090 wait complete. 2023-09-30T22:30:24.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.572+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9d50074ea0 con 0x7f9d5009f240 2023-09-30T22:30:24.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.572+0000 7f9d47fff700 1 -- 172.21.15.32:0/1177924330 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d38011e80 con 0x7f9d5009f240 2023-09-30T22:30:24.573 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.574+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 msgr2=0x7f9d501a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.574+0000 7f9d551ff700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 0x7f9d501a4910 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c470 tx=0x7f9d3800c780 comp rx=0 tx=0).stop 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 shutdown_connections 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d5009f650 0x7f9d501c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5009e930 0x7f9d501a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 --2- 172.21.15.32:0/1177924330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9d5009f240 0x7f9d501a4910 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 >> 172.21.15.32:0/1177924330 conn(0x7f9d500a4210 msgr2=0x7f9d501b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 shutdown_connections 2023-09-30T22:30:24.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.575+0000 7f9d551ff700 1 -- 172.21.15.32:0/1177924330 wait complete. 2023-09-30T22:30:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.577+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdd4406ef10 con 0x7fdd4412c2f0 2023-09-30T22:30:24.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.577+0000 7fdd417fa700 1 -- 172.21.15.32:0/1186128955 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd38021020 con 0x7fdd4412c2f0 2023-09-30T22:30:24.577 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.579+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 msgr2=0x7fdd4412eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.579+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 0x7fdd4412eb10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800e9b0 tx=0x7fdd3800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 shutdown_connections 2023-09-30T22:30:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd4412cff0 0x7fdd4412f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdd4412c2f0 0x7fdd4412eb10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 --2- 172.21.15.32:0/1186128955 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdd4412cc20 0x7fdd4412f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 >> 172.21.15.32:0/1186128955 conn(0x7fdd4409f470 msgr2=0x7fdd4412a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 shutdown_connections 2023-09-30T22:30:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:24.580+0000 7fdd4ac37700 1 -- 172.21.15.32:0/1186128955 wait complete. 2023-09-30T22:30:24.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:24.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:25.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:25.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.213+0000 7f7b3f980700 1 Processor -- start 2023-09-30T22:30:25.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 -- start start 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809d6f0 0x7f7b3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b3809e000 0x7f7b381339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 0x7f7b38133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3806b460 con 0x7f7b3809e000 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3809b120 con 0x7f7b3809e410 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3f980700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3809a7e0 con 0x7f7b3809d6f0 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3df1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 0x7f7b38133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3cf1b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b3809e000 0x7f7b381339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3d71c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809d6f0 0x7f7b3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3cf1b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7b3809e000 0x7f7b381339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58696/0 (socket says 172.21.15.32:58696) 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3df1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 0x7f7b38133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47804/0 (socket says 172.21.15.32:47804) 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.214+0000 7f7b3cf1b700 1 -- 172.21.15.32:0/2388523114 learned_addr learned my addr 172.21.15.32:0/2388523114 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3d71c700 1 -- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 msgr2=0x7f7b38133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3d71c700 1 --2- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 0x7f7b38133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3d71c700 1 -- 172.21.15.32:0/2388523114 >> v2:172.21.15.32:3300/0 conn(0x7f7b3809e000 msgr2=0x7f7b381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3d71c700 1 --2- 172.21.15.32:0/2388523114 >> v2:172.21.15.32:3300/0 conn(0x7f7b3809e000 0x7f7b381339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3d71c700 1 -- 172.21.15.32:0/2388523114 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3809a500 con 0x7f7b3809d6f0 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3df1d700 1 --2- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 0x7f7b38133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b3d71c700 1 --2- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809d6f0 0x7f7b3809dac0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009870 tx=0x7f7b34009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2ba3478d2d789a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b2ffff700 1 -- 172.21.15.32:0/2388523114 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34011830 con 0x7f7b3809d6f0 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b2ffff700 1 -- 172.21.15.32:0/2388523114 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b34011990 con 0x7f7b3809d6f0 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.215+0000 7f7b2ffff700 1 -- 172.21.15.32:0/2388523114 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34011d20 con 0x7f7b3809d6f0 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 -- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809d6f0 msgr2=0x7f7b3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 --2- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809d6f0 0x7f7b3809dac0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009870 tx=0x7f7b34009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 -- 172.21.15.32:0/2388523114 shutdown_connections 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 --2- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809e410 0x7f7b38133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 --2- 172.21.15.32:0/2388523114 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809d6f0 0x7f7b3809dac0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 --2- 172.21.15.32:0/2388523114 >> v2:172.21.15.32:3300/0 conn(0x7f7b3809e000 0x7f7b381339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 -- 172.21.15.32:0/2388523114 >> 172.21.15.32:0/2388523114 conn(0x7f7b38066f50 msgr2=0x7f7b3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 -- 172.21.15.32:0/2388523114 shutdown_connections 2023-09-30T22:30:25.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.216+0000 7f7b3f980700 1 -- 172.21.15.32:0/2388523114 wait complete. 2023-09-30T22:30:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.217+0000 7f7b3f980700 1 Processor -- start 2023-09-30T22:30:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.217+0000 7f7b3f980700 1 -- start start 2023-09-30T22:30:25.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3f980700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 0x7f7b381a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3f980700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809e000 0x7f7b381a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3f980700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3809e410 0x7f7b381a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3f980700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381b93d0 con 0x7f7b3809e410 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3f980700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a8e40 con 0x7f7b3809d6f0 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3f980700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a81d0 con 0x7f7b3809e000 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3d71c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 0x7f7b381a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3cf1b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809e000 0x7f7b381a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3df1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3809e410 0x7f7b381a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3d71c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 0x7f7b381a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47808/0 (socket says 172.21.15.32:47808) 2023-09-30T22:30:25.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3d71c700 1 -- 172.21.15.32:0/1841334327 learned_addr learned my addr 172.21.15.32:0/1841334327 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3cf1b700 1 -- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 msgr2=0x7f7b381a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3cf1b700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 0x7f7b381a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3cf1b700 1 -- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3809e410 msgr2=0x7f7b381a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.218+0000 7f7b3cf1b700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3809e410 0x7f7b381a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b3cf1b700 1 -- 172.21.15.32:0/1841334327 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30009a70 con 0x7f7b3809e000 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b3d71c700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 0x7f7b381a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b3cf1b700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809e000 0x7f7b381a65b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e8b0 tx=0x7f7b2800ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28013680 con 0x7f7b3809e000 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b280137e0 con 0x7f7b3809e000 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28013b50 con 0x7f7b3809e000 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b340094a0 con 0x7f7b3809e000 2023-09-30T22:30:25.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.219+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b380a21d0 con 0x7f7b3809e000 2023-09-30T22:30:25.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.220+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b28013cb0 con 0x7f7b3809e000 2023-09-30T22:30:25.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.220+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7b2803f620 con 0x7f7b3809e000 2023-09-30T22:30:25.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.220+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b38074ea0 con 0x7f7b3809e000 2023-09-30T22:30:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.225+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b2801be10 con 0x7f7b3809e000 2023-09-30T22:30:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.429+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7b381a94c0 con 0x7f7b3809e000 2023-09-30T22:30:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.430+0000 7f7b2dffb700 1 -- 172.21.15.32:0/1841334327 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b28017020 con 0x7f7b3809e000 2023-09-30T22:30:25.430 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809e000 msgr2=0x7f7b381a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809e000 0x7f7b381a65b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e8b0 tx=0x7f7b2800ebc0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 shutdown_connections 2023-09-30T22:30:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3809e410 0x7f7b381a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7b3809d6f0 0x7f7b381a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 --2- 172.21.15.32:0/1841334327 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7b3809e000 0x7f7b381a65b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 >> 172.21.15.32:0/1841334327 conn(0x7f7b38066f50 msgr2=0x7f7b381bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 shutdown_connections 2023-09-30T22:30:25.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.432+0000 7f7b3f980700 1 -- 172.21.15.32:0/1841334327 wait complete. 2023-09-30T22:30:25.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:25.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:25.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:25.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:25.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.627+0000 7f25c5b9a700 1 Processor -- start 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25c5b9a700 1 -- start start 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25c5b9a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25c5b9a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25c0135e50 0x7f25c0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25c5b9a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c009f130 con 0x7f25c009d6f0 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25c5b9a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c009f520 con 0x7f25c0135e50 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25c5b9a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c009e930 con 0x7f25c009e1d0 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25beffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25bffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25c0135e50 0x7f25c0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25beffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c0135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46150/0 (socket says 172.21.15.32:46150) 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25bffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25c0135e50 0x7f25c0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58708/0 (socket says 172.21.15.32:58708) 2023-09-30T22:30:25.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.628+0000 7f25beffd700 1 -- 172.21.15.32:0/2977069664 learned_addr learned my addr 172.21.15.32:0/2977069664 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25beffd700 1 -- 172.21.15.32:0/2977069664 >> v2:172.21.15.32:3300/0 conn(0x7f25c0135e50 msgr2=0x7f25c0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25beffd700 1 --2- 172.21.15.32:0/2977069664 >> v2:172.21.15.32:3300/0 conn(0x7f25c0135e50 0x7f25c0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25beffd700 1 -- 172.21.15.32:0/2977069664 >> v1:172.21.15.32:6789/0 conn(0x7f25c009d6f0 legacy=0x7f25c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25beffd700 1 -- 172.21.15.32:0/2977069664 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c009b2d0 con 0x7f25c009e1d0 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25beffd700 1 --2- 172.21.15.32:0/2977069664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c0135910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009920 tx=0x7f25b4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f44a18ce883e64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25be7fc700 1 -- 172.21.15.32:0/2977069664 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4011830 con 0x7f25c009e1d0 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25be7fc700 1 -- 172.21.15.32:0/2977069664 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4011990 con 0x7f25c009e1d0 2023-09-30T22:30:25.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.629+0000 7f25be7fc700 1 -- 172.21.15.32:0/2977069664 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4011d20 con 0x7f25c009e1d0 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 -- 172.21.15.32:0/2977069664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 msgr2=0x7f25c0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/2977069664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c0135910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009920 tx=0x7f25b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 -- 172.21.15.32:0/2977069664 shutdown_connections 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/2977069664 >> v2:172.21.15.32:3300/0 conn(0x7f25c0135e50 0x7f25c0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/2977069664 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c0135910 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 -- 172.21.15.32:0/2977069664 >> 172.21.15.32:0/2977069664 conn(0x7f25c0066f50 msgr2=0x7f25c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 -- 172.21.15.32:0/2977069664 shutdown_connections 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.630+0000 7f25c5b9a700 1 -- 172.21.15.32:0/2977069664 wait complete. 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.631+0000 7f25c5b9a700 1 Processor -- start 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.631+0000 7f25c5b9a700 1 -- start start 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25c5b9a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 0x7f25c01c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25c5b9a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c01c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25c5b9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c0135e50 0x7f25c01c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25c5b9a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0122780 con 0x7f25c0135e50 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25c5b9a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0122590 con 0x7f25c009d6f0 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25c5b9a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01244a0 con 0x7f25c009e1d0 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 0x7f25c01c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 0x7f25c01c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47810/0 (socket says 172.21.15.32:47810) 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 -- 172.21.15.32:0/1329156522 learned_addr learned my addr 172.21.15.32:0/1329156522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bffff700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c0135e50 0x7f25c01c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 -- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 msgr2=0x7f25c01c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c01c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 -- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c0135e50 msgr2=0x7f25c01c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c0135e50 0x7f25c01c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bf7fe700 1 -- 172.21.15.32:0/1329156522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4009550 con 0x7f25c009d6f0 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.632+0000 7f25bffff700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c0135e50 0x7f25c01c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.633+0000 7f25bf7fe700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 0x7f25c01c82f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f25b0001ed0 tx=0x7f25b00021e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.633+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0003c00 con 0x7f25c009d6f0 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.633+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b0003d60 con 0x7f25c009d6f0 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.633+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c00a1d00 con 0x7f25c009d6f0 2023-09-30T22:30:25.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.633+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b000abf0 con 0x7f25c009d6f0 2023-09-30T22:30:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.633+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c0122d80 con 0x7f25c009d6f0 2023-09-30T22:30:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.634+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25b000ad50 con 0x7f25c009d6f0 2023-09-30T22:30:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.634+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f25b003f780 con 0x7f25c009d6f0 2023-09-30T22:30:25.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.634+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c0123e60 con 0x7f25c009d6f0 2023-09-30T22:30:25.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.639+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25c0123e60 con 0x7f25c009d6f0 2023-09-30T22:30:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.742+0000 7f0d089d0700 1 Processor -- start 2023-09-30T22:30:25.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.742+0000 7f0d089d0700 1 -- start start 2023-09-30T22:30:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d089d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d040a0a30 0x7f0d040a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d089d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d0413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d089d0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d040a2a70 con 0x7f0d0409d630 2023-09-30T22:30:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d089d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040a01d0 con 0x7f0d04138d80 2023-09-30T22:30:25.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d089d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0409e2b0 con 0x7f0d040a0a30 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d01d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d0409d630 0x7f0d040a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53178/0 (socket says 172.21.15.32:53178) 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d01d9b700 1 -- 172.21.15.32:0/830378870 learned_addr learned my addr 172.21.15.32:0/830378870 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d0259c700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d040a0a30 0x7f0d040a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.743+0000 7f0d02d9d700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d0413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0159a700 1 -- 172.21.15.32:0/830378870 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232205807 0 0) 0x7f0d040a2a70 con 0x7f0d0409d630 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0159a700 1 -- 172.21.15.32:0/830378870 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cec0035b0 con 0x7f0d0409d630 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0259c700 1 -- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 msgr2=0x7f0d0413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0259c700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d0413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0259c700 1 -- 172.21.15.32:0/830378870 >> v1:172.21.15.32:6789/0 conn(0x7f0d0409d630 legacy=0x7f0d040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0259c700 1 -- 172.21.15.32:0/830378870 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0409dc50 con 0x7f0d040a0a30 2023-09-30T22:30:25.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d02d9d700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d0413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0259c700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d040a0a30 0x7f0d040a0e00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009870 tx=0x7f0cf4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4cea29f60f51533 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0159a700 1 -- 172.21.15.32:0/830378870 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4011830 con 0x7f0d040a0a30 2023-09-30T22:30:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0159a700 1 -- 172.21.15.32:0/830378870 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4011990 con 0x7f0d040a0a30 2023-09-30T22:30:25.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.744+0000 7f0d0159a700 1 -- 172.21.15.32:0/830378870 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4011d20 con 0x7f0d040a0a30 2023-09-30T22:30:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 -- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d040a0a30 msgr2=0x7f0d040a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d040a0a30 0x7f0d040a0e00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009870 tx=0x7f0cf4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:25.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 -- 172.21.15.32:0/830378870 shutdown_connections 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d0413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 --2- 172.21.15.32:0/830378870 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d040a0a30 0x7f0d040a0e00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 -- 172.21.15.32:0/830378870 >> 172.21.15.32:0/830378870 conn(0x7f0d040a3050 msgr2=0x7f0d040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 -- 172.21.15.32:0/830378870 shutdown_connections 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.745+0000 7f0d089d0700 1 -- 172.21.15.32:0/830378870 wait complete. 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.746+0000 7f0d089d0700 1 Processor -- start 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.746+0000 7f0d089d0700 1 -- start start 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d089d0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d0409d630 0x7f0d041337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d089d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d040a0a30 0x7f0d04133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d089d0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d04134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d089d0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04134c80 con 0x7f0d040a0a30 2023-09-30T22:30:25.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d089d0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04136680 con 0x7f0d04138d80 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d089d0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04136ff0 con 0x7f0d0409d630 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d04134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d04134230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47842/0 (socket says 172.21.15.32:47842) 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d0409d630 0x7f0d041337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 -- 172.21.15.32:0/338614008 learned_addr learned my addr 172.21.15.32:0/338614008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d01d9b700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d040a0a30 0x7f0d04133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 -- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d0409d630 msgr2=0x7f0d041337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d0409d630 0x7f0d041337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 -- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d040a0a30 msgr2=0x7f0d04133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d040a0a30 0x7f0d04133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d02d9d700 1 -- 172.21.15.32:0/338614008 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009a70 con 0x7f0d04138d80 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d0259c700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d0409d630 0x7f0d041337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.747+0000 7f0d01d9b700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d040a0a30 0x7f0d04133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.748+0000 7f0d02d9d700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d04134230 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00ef90 tx=0x7f0cfc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.748+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00cc90 con 0x7f0d04138d80 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.748+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc00f040 con 0x7f0d04138d80 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.748+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf40094a0 con 0x7f0d04138d80 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.748+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0146c0 con 0x7f0d04138d80 2023-09-30T22:30:25.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.748+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d04136a60 con 0x7f0d04138d80 2023-09-30T22:30:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.749+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cfc014820 con 0x7f0d04138d80 2023-09-30T22:30:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.749+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0cfc040380 con 0x7f0d04138d80 2023-09-30T22:30:25.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.749+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d040747f0 con 0x7f0d04138d80 2023-09-30T22:30:25.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.754+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cfc018020 con 0x7f0d04138d80 2023-09-30T22:30:25.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.770+0000 7f0d75b6b700 1 Processor -- start 2023-09-30T22:30:25.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.770+0000 7f0d75b6b700 1 -- start start 2023-09-30T22:30:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d75b6b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d70133a80 0x7f0d70133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d75b6b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d70130a70 0x7f0d7013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d75b6b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7006bc00 con 0x7f0d7012ff20 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d75b6b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7006c570 con 0x7f0d70133a80 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d75b6b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d700a0040 con 0x7f0d70130a70 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d70133a80 0x7f0d70133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d70133a80 0x7f0d70133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58722/0 (socket says 172.21.15.32:58722) 2023-09-30T22:30:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 -- 172.21.15.32:0/658346394 learned_addr learned my addr 172.21.15.32:0/658346394 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6ffff700 1 --2- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d70130a70 0x7f0d7013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 -- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d70130a70 msgr2=0x7f0d7013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 --2- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d70130a70 0x7f0d7013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 -- 172.21.15.32:0/658346394 >> v1:172.21.15.32:6789/0 conn(0x7f0d7012ff20 legacy=0x7f0d70130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.771+0000 7f0d6f7fe700 1 -- 172.21.15.32:0/658346394 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70133860 con 0x7f0d70133a80 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.772+0000 7f0d6f7fe700 1 --2- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d70133e50 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0d600098f0 tx=0x7f0d60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d47971cb93ea1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.772+0000 7f0d6e7fc700 1 -- 172.21.15.32:0/658346394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60011830 con 0x7f0d70133a80 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.772+0000 7f0d6e7fc700 1 -- 172.21.15.32:0/658346394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d60011990 con 0x7f0d70133a80 2023-09-30T22:30:25.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.772+0000 7f0d6e7fc700 1 -- 172.21.15.32:0/658346394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60011d20 con 0x7f0d70133a80 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.772+0000 7f0d75b6b700 1 -- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 msgr2=0x7f0d70133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.772+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d70133e50 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0d600098f0 tx=0x7f0d60009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.773+0000 7f0d75b6b700 1 -- 172.21.15.32:0/658346394 shutdown_connections 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.773+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d70130a70 0x7f0d7013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.773+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/658346394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d70133e50 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.773+0000 7f0d75b6b700 1 -- 172.21.15.32:0/658346394 >> 172.21.15.32:0/658346394 conn(0x7f0d700a4210 msgr2=0x7f0d700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.773+0000 7f0d75b6b700 1 -- 172.21.15.32:0/658346394 shutdown_connections 2023-09-30T22:30:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.773+0000 7f0d75b6b700 1 -- 172.21.15.32:0/658346394 wait complete. 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 Processor -- start 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 -- start start 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d7012ff20 0x7f0d7009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d70130a70 0x7f0d7009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d7009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7012d260 con 0x7f0d70133a80 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d700a1fa0 con 0x7f0d70130a70 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d75b6b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7009e070 con 0x7f0d7012ff20 2023-09-30T22:30:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.774+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d7009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d70130a70 0x7f0d7009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d7012ff20 0x7f0d7009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d7009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58738/0 (socket says 172.21.15.32:58738) 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 -- 172.21.15.32:0/3528525339 learned_addr learned my addr 172.21.15.32:0/3528525339 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 -- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d7012ff20 msgr2=0x7f0d7009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d7012ff20 0x7f0d7009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 -- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d70130a70 msgr2=0x7f0d7009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d70130a70 0x7f0d7009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 -- 172.21.15.32:0/3528525339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009550 con 0x7f0d70133a80 2023-09-30T22:30:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6f7fe700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d7012ff20 0x7f0d7009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6effd700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d70130a70 0x7f0d7009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d6ffff700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d7009cf00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800efc0 tx=0x7f0d6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68009ea0 con 0x7f0d70133a80 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6800f040 con 0x7f0d70133a80 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7011fcc0 con 0x7f0d70133a80 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.775+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800ced0 con 0x7f0d70133a80 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.776+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7012dac0 con 0x7f0d70133a80 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.776+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d68014480 con 0x7f0d70133a80 2023-09-30T22:30:25.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.777+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0d6803ff20 con 0x7f0d70133a80 2023-09-30T22:30:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.777+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d70074ea0 con 0x7f0d70133a80 2023-09-30T22:30:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.781+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d68018070 con 0x7f0d70133a80 2023-09-30T22:30:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.860+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f25c0123e60 con 0x7f25c009d6f0 2023-09-30T22:30:25.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.861+0000 7f25a7fff700 1 -- 172.21.15.32:0/1329156522 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25b0029090 con 0x7f25c009d6f0 2023-09-30T22:30:25.861 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 msgr2=0x7f25c01c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 0x7f25c01c82f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f25b0001ed0 tx=0x7f25b00021e0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 shutdown_connections 2023-09-30T22:30:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25c0135e50 0x7f25c01c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f25c009d6f0 0x7f25c01c82f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 --2- 172.21.15.32:0/1329156522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f25c009e1d0 0x7f25c01c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 >> 172.21.15.32:0/1329156522 conn(0x7f25c0066f50 msgr2=0x7f25c00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 shutdown_connections 2023-09-30T22:30:25.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.863+0000 7f25c5b9a700 1 -- 172.21.15.32:0/1329156522 wait complete. 2023-09-30T22:30:25.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:25.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.961+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0d04135ba0 con 0x7f0d04138d80 2023-09-30T22:30:25.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.961+0000 7f0cf2ffd700 1 -- 172.21.15.32:0/338614008 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cfc02b090 con 0x7f0d04138d80 2023-09-30T22:30:25.961 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 msgr2=0x7f0d04134230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d04134230 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00ef90 tx=0x7f0cfc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 shutdown_connections 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d04138d80 0x7f0d04134230 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d0409d630 0x7f0d041337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 --2- 172.21.15.32:0/338614008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d040a0a30 0x7f0d04133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.963+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 >> 172.21.15.32:0/338614008 conn(0x7f0d040a3050 msgr2=0x7f0d0409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.964+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 shutdown_connections 2023-09-30T22:30:25.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.964+0000 7f0d089d0700 1 -- 172.21.15.32:0/338614008 wait complete. 2023-09-30T22:30:25.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:25.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.990+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0d7006b250 con 0x7f0d70133a80 2023-09-30T22:30:25.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.990+0000 7f0d57fff700 1 -- 172.21.15.32:0/3528525339 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d6801a4a0 con 0x7f0d70133a80 2023-09-30T22:30:25.991 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:25.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 msgr2=0x7f0d7009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d7009cf00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800efc0 tx=0x7f0d6800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 shutdown_connections 2023-09-30T22:30:25.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d70133a80 0x7f0d7009cf00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d7012ff20 0x7f0d7009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 --2- 172.21.15.32:0/3528525339 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d70130a70 0x7f0d7009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 >> 172.21.15.32:0/3528525339 conn(0x7f0d700a4210 msgr2=0x7f0d7009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 shutdown_connections 2023-09-30T22:30:25.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:25.993+0000 7f0d75b6b700 1 -- 172.21.15.32:0/3528525339 wait complete. 2023-09-30T22:30:26.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:26.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 Processor -- start 2023-09-30T22:30:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 -- start start 2023-09-30T22:30:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f5214138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521409edd0 con 0x7f52140a0a30 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140a18e0 con 0x7f5214139250 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f521af73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f521409e1f0 con 0x7f52140a1510 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f5219510700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f5213fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f5214138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f5219510700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521413b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47852/0 (socket says 172.21.15.32:47852) 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.593+0000 7f5219510700 1 -- 172.21.15.32:0/3336528553 learned_addr learned my addr 172.21.15.32:0/3336528553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f5219510700 1 -- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 msgr2=0x7f5214138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f5219510700 1 --2- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f5214138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f5219510700 1 -- 172.21.15.32:0/3336528553 >> v1:172.21.15.32:6789/0 conn(0x7f52140a0a30 legacy=0x7f52140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f5219510700 1 -- 172.21.15.32:0/3336528553 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521409dc50 con 0x7f5214139250 2023-09-30T22:30:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f52137fe700 1 -- 172.21.15.32:0/3336528553 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089090846 0 0) 0x7f521409edd0 con 0x7f52140a0a30 2023-09-30T22:30:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f5213fff700 1 --2- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f5214138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f5219510700 1 --2- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521413b630 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f520c009e80 tx=0x7f520c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7dda416110b7a26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f52137fe700 1 -- 172.21.15.32:0/3336528553 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c00e070 con 0x7f5214139250 2023-09-30T22:30:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f52137fe700 1 -- 172.21.15.32:0/3336528553 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c00b910 con 0x7f5214139250 2023-09-30T22:30:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.594+0000 7f52137fe700 1 -- 172.21.15.32:0/3336528553 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c00bcc0 con 0x7f5214139250 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 -- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 msgr2=0x7f521413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 --2- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521413b630 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f520c009e80 tx=0x7f520c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 -- 172.21.15.32:0/3336528553 shutdown_connections 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 --2- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521413b630 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 --2- 172.21.15.32:0/3336528553 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f5214138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 -- 172.21.15.32:0/3336528553 >> 172.21.15.32:0/3336528553 conn(0x7f52140a6550 msgr2=0x7f52140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 -- 172.21.15.32:0/3336528553 shutdown_connections 2023-09-30T22:30:26.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.595+0000 7f521af73700 1 -- 172.21.15.32:0/3336528553 wait complete. 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.596+0000 7f521af73700 1 Processor -- start 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.596+0000 7f521af73700 1 -- start start 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f521af73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 0x7f521411e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f521af73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f521411ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f521af73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521411f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f521af73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140a0220 con 0x7f52140a0a30 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f521af73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52141204b0 con 0x7f5214139250 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f521af73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f521409c9e0 con 0x7f52140a1510 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f5218d0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 0x7f521411e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f5219510700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521411f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f5218d0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 0x7f521411e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58746/0 (socket says 172.21.15.32:58746) 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f5218d0f700 1 -- 172.21.15.32:0/2175950143 learned_addr learned my addr 172.21.15.32:0/2175950143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f5219510700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521411f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47858/0 (socket says 172.21.15.32:47858) 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.597+0000 7f5213fff700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f521411ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5218d0f700 1 -- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 msgr2=0x7f521411ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5218d0f700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f521411ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5218d0f700 1 -- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 msgr2=0x7f521411f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5218d0f700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521411f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5218d0f700 1 -- 172.21.15.32:0/2175950143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080094a0 con 0x7f52140a0a30 2023-09-30T22:30:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5219510700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521411f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f5218d0f700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 0x7f521411e790 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f5204001eb0 tx=0x7f520400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040054e0 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204003680 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c009b30 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040039f0 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.598+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5214122a40 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.599+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5204003b50 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.599+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f520403eab0 con 0x7f52140a0a30 2023-09-30T22:30:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.599+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214074ea0 con 0x7f52140a0a30 2023-09-30T22:30:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.604+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5204017020 con 0x7f52140a0a30 2023-09-30T22:30:26.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.807+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f52140a28e0 con 0x7f52140a0a30 2023-09-30T22:30:26.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.807+0000 7f52117fa700 1 -- 172.21.15.32:0/2175950143 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f520401b3b0 con 0x7f52140a0a30 2023-09-30T22:30:26.807 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 msgr2=0x7f521411e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 0x7f521411e790 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f5204001eb0 tx=0x7f520400c800 comp rx=0 tx=0).stop 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 shutdown_connections 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5214139250 0x7f521411f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52140a0a30 0x7f521411e790 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 --2- 172.21.15.32:0/2175950143 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f52140a1510 0x7f521411ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 >> 172.21.15.32:0/2175950143 conn(0x7f52140a6550 msgr2=0x7f52140a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 shutdown_connections 2023-09-30T22:30:26.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:26.809+0000 7f521af73700 1 -- 172.21.15.32:0/2175950143 wait complete. 2023-09-30T22:30:26.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:26.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:26.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:27.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.022+0000 7efeac1bb700 1 Processor -- start 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.022+0000 7efeac1bb700 1 -- start start 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efeac1bb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 0x7efea409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efeac1bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efea409f650 0x7efea4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efeac1bb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea4070940 con 0x7efea409f240 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efeac1bb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea406fd60 con 0x7efea409f650 2023-09-30T22:30:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efeac1bb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efea406fa80 con 0x7efea409e930 2023-09-30T22:30:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efea9f57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 0x7efea409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efea9f57700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 0x7efea409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46236/0 (socket says 172.21.15.32:46236) 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efeaa758700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efea409f650 0x7efea4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efea9f57700 1 -- 172.21.15.32:0/3126831774 learned_addr learned my addr 172.21.15.32:0/3126831774 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.023+0000 7efea9756700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efea409f240 0x7efea41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53194/0 (socket says 172.21.15.32:53194) 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efeaa758700 1 -- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 msgr2=0x7efea409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efeaa758700 1 --2- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 0x7efea409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efeaa758700 1 -- 172.21.15.32:0/3126831774 >> v1:172.21.15.32:6789/0 conn(0x7efea409f240 legacy=0x7efea41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efeaa758700 1 -- 172.21.15.32:0/3126831774 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea409fb60 con 0x7efea409f650 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efea9f57700 1 --2- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 0x7efea409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efeaa758700 1 --2- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409f650 0x7efea4131f00 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009d90 tx=0x7efe9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759d187862f07d38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efea8f55700 1 -- 172.21.15.32:0/3126831774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00e070 con 0x7efea409f650 2023-09-30T22:30:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efea8f55700 1 -- 172.21.15.32:0/3126831774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c00bb20 con 0x7efea409f650 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.024+0000 7efea8f55700 1 -- 172.21.15.32:0/3126831774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c012430 con 0x7efea409f650 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 -- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409f650 msgr2=0x7efea4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409f650 0x7efea4131f00 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009d90 tx=0x7efe9c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 -- 172.21.15.32:0/3126831774 shutdown_connections 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409f650 0x7efea4131f00 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3126831774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409e930 0x7efea409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 -- 172.21.15.32:0/3126831774 >> 172.21.15.32:0/3126831774 conn(0x7efea40a4210 msgr2=0x7efea40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 -- 172.21.15.32:0/3126831774 shutdown_connections 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.025+0000 7efeac1bb700 1 -- 172.21.15.32:0/3126831774 wait complete. 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.026+0000 7efeac1bb700 1 Processor -- start 2023-09-30T22:30:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 -- start start 2023-09-30T22:30:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 0x7efea41a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efea409f240 0x7efea41a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409f650 0x7efea406b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41a3f80 con 0x7efea409e930 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea41a5510 con 0x7efea409f240 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efeac1bb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efea41a4270 con 0x7efea409f650 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efea9f57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 0x7efea41a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.027+0000 7efea9f57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 0x7efea41a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58772/0 (socket says 172.21.15.32:58772) 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 -- 172.21.15.32:0/3241208643 learned_addr learned my addr 172.21.15.32:0/3241208643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efeaa758700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409f650 0x7efea406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9756700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efea409f240 0x7efea41a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 -- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409f650 msgr2=0x7efea406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409f650 0x7efea406b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 -- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efea409f240 msgr2=0x7efea41a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efea409f240 0x7efea41a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 -- 172.21.15.32:0/3241208643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0009550 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efea9f57700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 0x7efea41a7330 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7efea000ec10 tx=0x7efea000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000c850 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea000c9b0 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c009a70 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000cd40 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.028+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea40a0190 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.030+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efea0013470 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.029+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea4071d20 con 0x7efea409e930 2023-09-30T22:30:27.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.030+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efea003f830 con 0x7efea409e930 2023-09-30T22:30:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.035+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efea0013720 con 0x7efea409e930 2023-09-30T22:30:27.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.133+0000 7f280dab0700 1 Processor -- start 2023-09-30T22:30:27.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.133+0000 7f280dab0700 1 -- start start 2023-09-30T22:30:27.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f280dab0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809d6f0 0x7f280809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f280dab0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f280809e410 0x7f2808134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f280dab0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280809b120 con 0x7f280809e000 2023-09-30T22:30:27.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f280dab0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280806b460 con 0x7f280809e410 2023-09-30T22:30:27.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f280dab0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f280809a7e0 con 0x7f280809d6f0 2023-09-30T22:30:27.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2806ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f280809e000 0x7f28081339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53196/0 (socket says 172.21.15.32:53196) 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2806ffd700 1 -- 172.21.15.32:0/1273776477 learned_addr learned my addr 172.21.15.32:0/1273776477 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2807fff700 1 --2- 172.21.15.32:0/1273776477 >> v2:172.21.15.32:3300/0 conn(0x7f280809e410 0x7f2808134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f28077fe700 1 --2- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809d6f0 0x7f280809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f28067fc700 1 -- 172.21.15.32:0/1273776477 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893207586 0 0) 0x7f280809b120 con 0x7f280809e000 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f28067fc700 1 -- 172.21.15.32:0/1273776477 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f00035b0 con 0x7f280809e000 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2807fff700 1 -- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809d6f0 msgr2=0x7f280809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2807fff700 1 --2- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809d6f0 0x7f280809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2807fff700 1 -- 172.21.15.32:0/1273776477 >> v1:172.21.15.32:6789/0 conn(0x7f280809e000 legacy=0x7f28081339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.134+0000 7f2807fff700 1 -- 172.21.15.32:0/1273776477 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280809a500 con 0x7f280809e410 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.135+0000 7f2807fff700 1 --2- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f2808134100 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2800009d90 tx=0x7f280000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c40a50e7f58610 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.135+0000 7f28067fc700 1 -- 172.21.15.32:0/1273776477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000e070 con 0x7f280809e410 2023-09-30T22:30:27.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.135+0000 7f28067fc700 1 -- 172.21.15.32:0/1273776477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280000bb20 con 0x7f280809e410 2023-09-30T22:30:27.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.135+0000 7f28067fc700 1 -- 172.21.15.32:0/1273776477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800012430 con 0x7f280809e410 2023-09-30T22:30:27.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.135+0000 7f280dab0700 1 -- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 msgr2=0x7f2808134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.135+0000 7f280dab0700 1 --2- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f2808134100 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2800009d90 tx=0x7f280000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.136+0000 7f280dab0700 1 -- 172.21.15.32:0/1273776477 shutdown_connections 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.136+0000 7f280dab0700 1 --2- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f2808134100 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.136+0000 7f280dab0700 1 --2- 172.21.15.32:0/1273776477 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809d6f0 0x7f280809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.136+0000 7f280dab0700 1 -- 172.21.15.32:0/1273776477 >> 172.21.15.32:0/1273776477 conn(0x7f2808066f50 msgr2=0x7f280809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.136+0000 7f280dab0700 1 -- 172.21.15.32:0/1273776477 shutdown_connections 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.136+0000 7f280dab0700 1 -- 172.21.15.32:0/1273776477 wait complete. 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.137+0000 7f280dab0700 1 Processor -- start 2023-09-30T22:30:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.137+0000 7f280dab0700 1 -- start start 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f280dab0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f280809d6f0 0x7f28081a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f280dab0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809e000 0x7f28081a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f280dab0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f28081c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f280dab0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a7d20 con 0x7f280809e410 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f280dab0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28081b90a0 con 0x7f280809d6f0 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f280dab0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a4bd0 con 0x7f280809e000 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f2806ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809e000 0x7f28081a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f28077fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f280809d6f0 0x7f28081a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f2806ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809e000 0x7f28081a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46264/0 (socket says 172.21.15.32:46264) 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f2806ffd700 1 -- 172.21.15.32:0/1566728527 learned_addr learned my addr 172.21.15.32:0/1566728527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.138+0000 7f2807fff700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f28081c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f28077fe700 1 -- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809e000 msgr2=0x7f28081a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f28077fe700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809e000 0x7f28081a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f28077fe700 1 -- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 msgr2=0x7f28081c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f28077fe700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f28081c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f28077fe700 1 -- 172.21.15.32:0/1566728527 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800009a70 con 0x7f280809d6f0 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f28077fe700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f280809d6f0 0x7f28081a5070 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e930 tx=0x7f27f800ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8013680 con 0x7f280809d6f0 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f80137e0 con 0x7f280809d6f0 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8013b50 con 0x7f280809d6f0 2023-09-30T22:30:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28081a7fb0 con 0x7f280809d6f0 2023-09-30T22:30:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.139+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28081a5af0 con 0x7f280809d6f0 2023-09-30T22:30:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.140+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f27f8013cb0 con 0x7f280809d6f0 2023-09-30T22:30:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.140+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f27f803fa90 con 0x7f280809d6f0 2023-09-30T22:30:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.141+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2808074ea0 con 0x7f280809d6f0 2023-09-30T22:30:27.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.147+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f27f8017080 con 0x7f280809d6f0 2023-09-30T22:30:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 Processor -- start 2023-09-30T22:30:27.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 -- start start 2023-09-30T22:30:27.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c133a80 0x7f193c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f193c130a40 0x7f193c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c06bc00 con 0x7f193c12ff20 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c06c570 con 0x7f193c130a40 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.167+0000 7f1943810700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0a0040 con 0x7f193c133a80 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1941dad700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f193c130a40 0x7f193c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1940dab700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f193c12ff20 0x7f193c130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53202/0 (socket says 172.21.15.32:53202) 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1940dab700 1 -- 172.21.15.32:0/2399444567 learned_addr learned my addr 172.21.15.32:0/2399444567 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f19415ac700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c133a80 0x7f193c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1941dad700 1 -- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c133a80 msgr2=0x7f193c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1941dad700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c133a80 0x7f193c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1941dad700 1 -- 172.21.15.32:0/2399444567 >> v1:172.21.15.32:6789/0 conn(0x7f193c12ff20 legacy=0x7f193c130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1941dad700 1 -- 172.21.15.32:0/2399444567 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c133860 con 0x7f193c130a40 2023-09-30T22:30:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1933fff700 1 -- 172.21.15.32:0/2399444567 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368412154 0 0) 0x7f193c06bc00 con 0x7f193c12ff20 2023-09-30T22:30:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f19415ac700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c133a80 0x7f193c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1941dad700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c130a40 0x7f193c13b850 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f1934009d90 tx=0x7f193400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826c809ab92e8d7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.168+0000 7f1933fff700 1 -- 172.21.15.32:0/2399444567 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400e070 con 0x7f193c130a40 2023-09-30T22:30:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1933fff700 1 -- 172.21.15.32:0/2399444567 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193400bb20 con 0x7f193c130a40 2023-09-30T22:30:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1933fff700 1 -- 172.21.15.32:0/2399444567 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934012430 con 0x7f193c130a40 2023-09-30T22:30:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 -- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c130a40 msgr2=0x7f193c13b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c130a40 0x7f193c13b850 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f1934009d90 tx=0x7f193400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 -- 172.21.15.32:0/2399444567 shutdown_connections 2023-09-30T22:30:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c130a40 0x7f193c13b850 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 --2- 172.21.15.32:0/2399444567 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c133a80 0x7f193c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 -- 172.21.15.32:0/2399444567 >> 172.21.15.32:0/2399444567 conn(0x7f193c0a4210 msgr2=0x7f193c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 -- 172.21.15.32:0/2399444567 shutdown_connections 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.169+0000 7f1943810700 1 -- 172.21.15.32:0/2399444567 wait complete. 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.170+0000 7f1943810700 1 Processor -- start 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 -- start start 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c12ff20 0x7f193c09c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 0x7f193c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 0x7f193c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c11fe10 con 0x7f193c133a80 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09dc40 con 0x7f193c130a40 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1943810700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c120840 con 0x7f193c12ff20 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1940dab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 0x7f193c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1941dad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 0x7f193c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1940dab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 0x7f193c09cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47866/0 (socket says 172.21.15.32:47866) 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1941dad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 0x7f193c09d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58802/0 (socket says 172.21.15.32:58802) 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f19415ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c12ff20 0x7f193c09c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.171+0000 7f1940dab700 1 -- 172.21.15.32:0/4016526772 learned_addr learned my addr 172.21.15.32:0/4016526772 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1941dad700 1 -- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c12ff20 msgr2=0x7f193c09c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1941dad700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c12ff20 0x7f193c09c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1941dad700 1 -- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 msgr2=0x7f193c09cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1941dad700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 0x7f193c09cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1941dad700 1 -- 172.21.15.32:0/4016526772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380094a0 con 0x7f193c133a80 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1940dab700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 0x7f193c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1941dad700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 0x7f193c09d160 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1934016000 tx=0x7f1934004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934012430 con 0x7f193c133a80 2023-09-30T22:30:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1934020070 con 0x7f193c133a80 2023-09-30T22:30:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1943810700 1 -- 172.21.15.32:0/4016526772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1934009a70 con 0x7f193c133a80 2023-09-30T22:30:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934004280 con 0x7f193c133a80 2023-09-30T22:30:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.172+0000 7f1943810700 1 -- 172.21.15.32:0/4016526772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c12d770 con 0x7f193c133a80 2023-09-30T22:30:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.173+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19340043e0 con 0x7f193c133a80 2023-09-30T22:30:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.173+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19140052e0 con 0x7f193c133a80 2023-09-30T22:30:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.173+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1934022070 con 0x7f193c133a80 2023-09-30T22:30:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.178+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f193401b400 con 0x7f193c133a80 2023-09-30T22:30:27.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.260+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efea41a4520 con 0x7efea409e930 2023-09-30T22:30:27.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.260+0000 7efe9a7fc700 1 -- 172.21.15.32:0/3241208643 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efea0017090 con 0x7efea409e930 2023-09-30T22:30:27.260 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 msgr2=0x7efea41a7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 0x7efea41a7330 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7efea000ec10 tx=0x7efea000ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 shutdown_connections 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efea409f650 0x7efea406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efea409e930 0x7efea41a7330 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 --2- 172.21.15.32:0/3241208643 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efea409f240 0x7efea41a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 >> 172.21.15.32:0/3241208643 conn(0x7efea40a4210 msgr2=0x7efea41b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 shutdown_connections 2023-09-30T22:30:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.263+0000 7efeac1bb700 1 -- 172.21.15.32:0/3241208643 wait complete. 2023-09-30T22:30:27.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.355+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28081a6890 con 0x7f280809d6f0 2023-09-30T22:30:27.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.355+0000 7f27effff700 1 -- 172.21.15.32:0/1566728527 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f27f801a9b0 con 0x7f280809d6f0 2023-09-30T22:30:27.355 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:27.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f280809d6f0 msgr2=0x7f28081a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f280809d6f0 0x7f28081a5070 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e930 tx=0x7f27f800ec40 comp rx=0 tx=0).stop 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 shutdown_connections 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f280809e410 0x7f28081c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f280809d6f0 0x7f28081a5070 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 --2- 172.21.15.32:0/1566728527 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f280809e000 0x7f28081a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.358+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 >> 172.21.15.32:0/1566728527 conn(0x7f2808066f50 msgr2=0x7f28081bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.359+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 shutdown_connections 2023-09-30T22:30:27.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.359+0000 7f280dab0700 1 -- 172.21.15.32:0/1566728527 wait complete. 2023-09-30T22:30:27.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:27.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.395+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1914005590 con 0x7f193c133a80 2023-09-30T22:30:27.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.396+0000 7f1931ffb700 1 -- 172.21.15.32:0/4016526772 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19340263e0 con 0x7f193c133a80 2023-09-30T22:30:27.396 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 msgr2=0x7f193c09d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 0x7f193c09d160 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1934016000 tx=0x7f1934004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 shutdown_connections 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f193c133a80 0x7f193c09d160 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f193c12ff20 0x7f193c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 --2- 172.21.15.32:0/4016526772 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f193c130a40 0x7f193c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 >> 172.21.15.32:0/4016526772 conn(0x7f193c0a4210 msgr2=0x7f193c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 shutdown_connections 2023-09-30T22:30:27.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.398+0000 7f19277fe700 1 -- 172.21.15.32:0/4016526772 wait complete. 2023-09-30T22:30:27.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:27.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:27.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.970+0000 7f05023af700 1 Processor -- start 2023-09-30T22:30:27.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.970+0000 7f05023af700 1 -- start start 2023-09-30T22:30:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f05023af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc09e1d0 0x7f04fc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f05023af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc139170 0x7f04fc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f05023af700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc09b120 con 0x7f04fc09d6f0 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f05023af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc06b460 con 0x7f04fc09e1d0 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f05023af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc09a7e0 con 0x7f04fc139170 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04fc09d6f0 0x7f04fc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53212/0 (socket says 172.21.15.32:53212) 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f050094c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc139170 0x7f04fc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fbfff700 1 -- 172.21.15.32:0/2489251879 learned_addr learned my addr 172.21.15.32:0/2489251879 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fb7fe700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc09e1d0 0x7f04fc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04faffd700 1 -- 172.21.15.32:0/2489251879 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277470232 0 0) 0x7f04fc09b120 con 0x7f04fc09d6f0 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fb7fe700 1 -- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc139170 msgr2=0x7f04fc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fb7fe700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc139170 0x7f04fc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fb7fe700 1 -- 172.21.15.32:0/2489251879 >> v1:172.21.15.32:6789/0 conn(0x7f04fc09d6f0 legacy=0x7f04fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f04fb7fe700 1 -- 172.21.15.32:0/2489251879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc09a500 con 0x7f04fc09e1d0 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.971+0000 7f050094c700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc139170 0x7f04fc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f04fb7fe700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc09e1d0 0x7f04fc138c30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009870 tx=0x7f04e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3facd947db276b6d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f04faffd700 1 -- 172.21.15.32:0/2489251879 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4011830 con 0x7f04fc09e1d0 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f04faffd700 1 -- 172.21.15.32:0/2489251879 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4011990 con 0x7f04fc09e1d0 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f04faffd700 1 -- 172.21.15.32:0/2489251879 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4011d20 con 0x7f04fc09e1d0 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 -- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc09e1d0 msgr2=0x7f04fc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc09e1d0 0x7f04fc138c30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009870 tx=0x7f04e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 -- 172.21.15.32:0/2489251879 shutdown_connections 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc139170 0x7f04fc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 --2- 172.21.15.32:0/2489251879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc09e1d0 0x7f04fc138c30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 -- 172.21.15.32:0/2489251879 >> 172.21.15.32:0/2489251879 conn(0x7f04fc066f50 msgr2=0x7f04fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 -- 172.21.15.32:0/2489251879 shutdown_connections 2023-09-30T22:30:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.972+0000 7f05023af700 1 -- 172.21.15.32:0/2489251879 wait complete. 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.973+0000 7f05023af700 1 Processor -- start 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 -- start start 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 0x7f04fc11e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc09e1d0 0x7f04fc11ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc139170 0x7f04fc11f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc133fa0 con 0x7f04fc09d6f0 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc134580 con 0x7f04fc139170 2023-09-30T22:30:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f05023af700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc120b00 con 0x7f04fc09e1d0 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f04fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 0x7f04fc11e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f04fb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc09e1d0 0x7f04fc11ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f050094c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc139170 0x7f04fc11f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f04fb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc09e1d0 0x7f04fc11ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46284/0 (socket says 172.21.15.32:46284) 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f04fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 0x7f04fc11e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58812/0 (socket says 172.21.15.32:58812) 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.974+0000 7f04fb7fe700 1 -- 172.21.15.32:0/3173792616 learned_addr learned my addr 172.21.15.32:0/3173792616 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f050094c700 1 -- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc09e1d0 msgr2=0x7f04fc11ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f050094c700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc09e1d0 0x7f04fc11ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f050094c700 1 -- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 msgr2=0x7f04fc11e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f050094c700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 0x7f04fc11e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f050094c700 1 -- 172.21.15.32:0/3173792616 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4009a70 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f04fbfff700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 0x7f04fc11e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f050094c700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc139170 0x7f04fc11f110 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009e50 tx=0x7f04f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400ca90 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f400f040 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e40094a0 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f400ce50 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.975+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc072110 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.976+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f04f4014480 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.976+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f04f403fc60 con 0x7f04fc139170 2023-09-30T22:30:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.976+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc074ea0 con 0x7f04fc139170 2023-09-30T22:30:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:27.981+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f04f4012930 con 0x7f04fc139170 2023-09-30T22:30:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.188+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f04fc06c360 con 0x7f04fc139170 2023-09-30T22:30:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.189+0000 7f04f8ff9700 1 -- 172.21.15.32:0/3173792616 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f04f4012340 con 0x7f04fc139170 2023-09-30T22:30:28.189 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc139170 msgr2=0x7f04fc11f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc139170 0x7f04fc11f110 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009e50 tx=0x7f04f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 shutdown_connections 2023-09-30T22:30:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04fc139170 0x7f04fc11f110 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04fc09d6f0 0x7f04fc11e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 --2- 172.21.15.32:0/3173792616 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f04fc09e1d0 0x7f04fc11ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 >> 172.21.15.32:0/3173792616 conn(0x7f04fc066f50 msgr2=0x7f04fc136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 shutdown_connections 2023-09-30T22:30:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.191+0000 7f05023af700 1 -- 172.21.15.32:0/3173792616 wait complete. 2023-09-30T22:30:28.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:28.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:28.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:28.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:28.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 Processor -- start 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 -- start start 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 0x7f0fc81307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 0x7f0fc809e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc8070700 con 0x7f0fc809e7c0 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc8071f20 con 0x7f0fc809de50 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.421+0000 7f0fcd2ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc812ffe0 con 0x7f0fc8130410 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 0x7f0fc81307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 0x7f0fc809e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc67fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 0x7f0fc809e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58826/0 (socket says 172.21.15.32:58826) 2023-09-30T22:30:28.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc67fc700 1 -- 172.21.15.32:0/2255655665 learned_addr learned my addr 172.21.15.32:0/2255655665 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc77fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0fc809e7c0 0x7f0fc813a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53220/0 (socket says 172.21.15.32:53220) 2023-09-30T22:30:28.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 --2- 172.21.15.32:0/2255655665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 0x7f0fc81307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47898/0 (socket says 172.21.15.32:47898) 2023-09-30T22:30:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 -- 172.21.15.32:0/2255655665 >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 msgr2=0x7f0fc809e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 --2- 172.21.15.32:0/2255655665 >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 0x7f0fc809e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 -- 172.21.15.32:0/2255655665 >> v1:172.21.15.32:6789/0 conn(0x7f0fc809e7c0 legacy=0x7f0fc813a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 -- 172.21.15.32:0/2255655665 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc806f7f0 con 0x7f0fc8130410 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc67fc700 1 --2- 172.21.15.32:0/2255655665 >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 0x7f0fc809e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc6ffd700 1 --2- 172.21.15.32:0/2255655665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 0x7f0fc81307e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009d90 tx=0x7f0fb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ce49abcb565dbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc5ffb700 1 -- 172.21.15.32:0/2255655665 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb800e070 con 0x7f0fc8130410 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc5ffb700 1 -- 172.21.15.32:0/2255655665 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb800bb20 con 0x7f0fc8130410 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.422+0000 7f0fc5ffb700 1 -- 172.21.15.32:0/2255655665 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8012430 con 0x7f0fc8130410 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2255655665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 msgr2=0x7f0fc81307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2255655665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 0x7f0fc81307e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009d90 tx=0x7f0fb800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2255655665 shutdown_connections 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2255655665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc8130410 0x7f0fc81307e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2255655665 >> v2:172.21.15.32:3300/0 conn(0x7f0fc809de50 0x7f0fc809e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2255655665 >> 172.21.15.32:0/2255655665 conn(0x7f0fc80a2610 msgr2=0x7f0fc80a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2255655665 shutdown_connections 2023-09-30T22:30:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.423+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2255655665 wait complete. 2023-09-30T22:30:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.424+0000 7f0fcd2ca700 1 Processor -- start 2023-09-30T22:30:28.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 -- start start 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc809de50 0x7f0fc8119660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 0x7f0fc8119ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0fc8130410 0x7f0fc811a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc811d180 con 0x7f0fc809e7c0 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80715a0 con 0x7f0fc809de50 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fcd2ca700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc811aaa0 con 0x7f0fc8130410 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 0x7f0fc8119ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fc6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc809de50 0x7f0fc8119660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fc67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 0x7f0fc8119ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58842/0 (socket says 172.21.15.32:58842) 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.425+0000 7f0fc67fc700 1 -- 172.21.15.32:0/2580670593 learned_addr learned my addr 172.21.15.32:0/2580670593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc6ffd700 1 -- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0fc8130410 msgr2=0x7f0fc811a0e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc6ffd700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0fc8130410 0x7f0fc811a0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc6ffd700 1 -- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 msgr2=0x7f0fc8119ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc6ffd700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 0x7f0fc8119ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc6ffd700 1 -- 172.21.15.32:0/2580670593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc009550 con 0x7f0fc809de50 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc67fc700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 0x7f0fc8119ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fc6ffd700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc809de50 0x7f0fc8119660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8016000 tx=0x7f0fb8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8012430 con 0x7f0fc809de50 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb8020070 con 0x7f0fc809de50 2023-09-30T22:30:28.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8004280 con 0x7f0fc809de50 2023-09-30T22:30:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb8009a70 con 0x7f0fc809de50 2023-09-30T22:30:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.426+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc806f390 con 0x7f0fc809de50 2023-09-30T22:30:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.427+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0fb80043e0 con 0x7f0fc809de50 2023-09-30T22:30:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.427+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0fb8022070 con 0x7f0fc809de50 2023-09-30T22:30:28.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.428+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc811b2d0 con 0x7f0fc809de50 2023-09-30T22:30:28.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.433+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0fb801c400 con 0x7f0fc809de50 2023-09-30T22:30:28.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.520+0000 7fc9f07da700 1 Processor -- start 2023-09-30T22:30:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9f07da700 1 -- start start 2023-09-30T22:30:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9f07da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9f07da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e8139170 0x7fc9e813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9f07da700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e809b120 con 0x7fc9e809d6f0 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9f07da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e806b460 con 0x7fc9e809e1d0 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9f07da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e809a7e0 con 0x7fc9e8139170 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.521+0000 7fc9ee576700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9e809d6f0 0x7fc9e809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53232/0 (socket says 172.21.15.32:53232) 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9ee576700 1 -- 172.21.15.32:0/3499087560 learned_addr learned my addr 172.21.15.32:0/3499087560 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9eed77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e8139170 0x7fc9e813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9edd75700 1 --2- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9ed574700 1 -- 172.21.15.32:0/3499087560 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370962991 0 0) 0x7fc9e809b120 con 0x7fc9e809d6f0 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9ed574700 1 -- 172.21.15.32:0/3499087560 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d40035b0 con 0x7fc9e809d6f0 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9edd75700 1 -- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e8139170 msgr2=0x7fc9e813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9edd75700 1 --2- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e8139170 0x7fc9e813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9edd75700 1 -- 172.21.15.32:0/3499087560 >> v1:172.21.15.32:6789/0 conn(0x7fc9e809d6f0 legacy=0x7fc9e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9edd75700 1 -- 172.21.15.32:0/3499087560 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e809a500 con 0x7fc9e809e1d0 2023-09-30T22:30:28.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9edd75700 1 --2- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8138c30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009870 tx=0x7fc9d8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74052fbeb0f04977 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.522+0000 7fc9ed574700 1 -- 172.21.15.32:0/3499087560 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8011830 con 0x7fc9e809e1d0 2023-09-30T22:30:28.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9ed574700 1 -- 172.21.15.32:0/3499087560 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8011990 con 0x7fc9e809e1d0 2023-09-30T22:30:28.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9ed574700 1 -- 172.21.15.32:0/3499087560 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8011d00 con 0x7fc9e809e1d0 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 -- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 msgr2=0x7fc9e8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 --2- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8138c30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009870 tx=0x7fc9d8009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 -- 172.21.15.32:0/3499087560 shutdown_connections 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 --2- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e8139170 0x7fc9e813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 --2- 172.21.15.32:0/3499087560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8138c30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 -- 172.21.15.32:0/3499087560 >> 172.21.15.32:0/3499087560 conn(0x7fc9e8066f50 msgr2=0x7fc9e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 -- 172.21.15.32:0/3499087560 shutdown_connections 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.523+0000 7fc9f07da700 1 -- 172.21.15.32:0/3499087560 wait complete. 2023-09-30T22:30:28.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.524+0000 7fc9f07da700 1 Processor -- start 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.524+0000 7fc9f07da700 1 -- start start 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9f07da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e809d6f0 0x7fc9e8120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9f07da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9f07da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8139170 0x7fc9e8121090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9f07da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8122410 con 0x7fc9e8139170 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9f07da700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8122220 con 0x7fc9e809e1d0 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9f07da700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8136590 con 0x7fc9e809d6f0 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9edd75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9edd75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8120b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47936/0 (socket says 172.21.15.32:47936) 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9edd75700 1 -- 172.21.15.32:0/1827023502 learned_addr learned my addr 172.21.15.32:0/1827023502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9eed77700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8139170 0x7fc9e8121090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9eed77700 1 -- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e809d6f0 msgr2=0x7fc9e8120610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9eed77700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e809d6f0 0x7fc9e8120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9eed77700 1 -- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 msgr2=0x7fc9e8120b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9eed77700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8120b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.525+0000 7fc9eed77700 1 -- 172.21.15.32:0/1827023502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80094a0 con 0x7fc9e8139170 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9edd75700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8120b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9eed77700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8139170 0x7fc9e8121090 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000efc0 tx=0x7fc9e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000cce0 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000f040 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00146c0 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e809c710 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.526+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8122a10 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.527+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9e0014820 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.527+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc9e0040720 con 0x7fc9e8139170 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.527+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e8074ea0 con 0x7fc9e8139170 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.537+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9e0014c10 con 0x7fc9e8139170 2023-09-30T22:30:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.567+0000 7fb09e7bc700 1 Processor -- start 2023-09-30T22:30:28.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.567+0000 7fb09e7bc700 1 -- start start 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09e7bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809d6f0 0x7fb09809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09e7bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb09809e410 0x7fb098130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09e7bc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09809f130 con 0x7fb09809e000 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09e7bc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09809f520 con 0x7fb09809e410 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09e7bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb09809e930 con 0x7fb09809d6f0 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb0977fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb09809e000 0x7fb098130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53244/0 (socket says 172.21.15.32:53244) 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb0977fe700 1 -- 172.21.15.32:0/3571877082 learned_addr learned my addr 172.21.15.32:0/3571877082 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09cd59700 1 --2- 172.21.15.32:0/3571877082 >> v2:172.21.15.32:3300/0 conn(0x7fb09809e410 0x7fb098130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb097fff700 1 --2- 172.21.15.32:0/3571877082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809d6f0 0x7fb09809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb096ffd700 1 -- 172.21.15.32:0/3571877082 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168546348 0 0) 0x7fb09809f130 con 0x7fb09809e000 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb096ffd700 1 -- 172.21.15.32:0/3571877082 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0800035b0 con 0x7fb09809e000 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb097fff700 1 -- 172.21.15.32:0/3571877082 >> v2:172.21.15.32:3300/0 conn(0x7fb09809e410 msgr2=0x7fb098130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb097fff700 1 --2- 172.21.15.32:0/3571877082 >> v2:172.21.15.32:3300/0 conn(0x7fb09809e410 0x7fb098130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb097fff700 1 -- 172.21.15.32:0/3571877082 >> v1:172.21.15.32:6789/0 conn(0x7fb09809e000 legacy=0x7fb098130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb097fff700 1 -- 172.21.15.32:0/3571877082 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09809b2d0 con 0x7fb09809d6f0 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.568+0000 7fb09cd59700 1 --2- 172.21.15.32:0/3571877082 >> v2:172.21.15.32:3300/0 conn(0x7fb09809e410 0x7fb098130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.569+0000 7fb097fff700 1 --2- 172.21.15.32:0/3571877082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809d6f0 0x7fb09809dac0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb088009840 tx=0x7fb088009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ce938e0f39ffab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.569+0000 7fb096ffd700 1 -- 172.21.15.32:0/3571877082 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088011830 con 0x7fb09809d6f0 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.569+0000 7fb096ffd700 1 -- 172.21.15.32:0/3571877082 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088011990 con 0x7fb09809d6f0 2023-09-30T22:30:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.569+0000 7fb096ffd700 1 -- 172.21.15.32:0/3571877082 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088011d20 con 0x7fb09809d6f0 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.569+0000 7fb09e7bc700 1 -- 172.21.15.32:0/3571877082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809d6f0 msgr2=0x7fb09809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.569+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/3571877082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809d6f0 0x7fb09809dac0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb088009840 tx=0x7fb088009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.570+0000 7fb09e7bc700 1 -- 172.21.15.32:0/3571877082 shutdown_connections 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.570+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/3571877082 >> v2:172.21.15.32:3300/0 conn(0x7fb09809e410 0x7fb098130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.570+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/3571877082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809d6f0 0x7fb09809dac0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.570+0000 7fb09e7bc700 1 -- 172.21.15.32:0/3571877082 >> 172.21.15.32:0/3571877082 conn(0x7fb098066f50 msgr2=0x7fb09809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.570+0000 7fb09e7bc700 1 -- 172.21.15.32:0/3571877082 shutdown_connections 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.570+0000 7fb09e7bc700 1 -- 172.21.15.32:0/3571877082 wait complete. 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 Processor -- start 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 -- start start 2023-09-30T22:30:28.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 0x7fb0981c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 0x7fb0981c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809e410 0x7fb098131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981351e0 con 0x7fb09809d6f0 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098134520 con 0x7fb09809e000 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb09e7bc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981c2880 con 0x7fb09809e410 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 0x7fb0981c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb097fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 0x7fb0981c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 0x7fb0981c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47952/0 (socket says 172.21.15.32:47952) 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb0977fe700 1 -- 172.21.15.32:0/4025360337 learned_addr learned my addr 172.21.15.32:0/4025360337 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.571+0000 7fb097fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 0x7fb0981c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58864/0 (socket says 172.21.15.32:58864) 2023-09-30T22:30:28.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb09cd59700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809e410 0x7fb098131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb0977fe700 1 -- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809e410 msgr2=0x7fb098131630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb0977fe700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809e410 0x7fb098131630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb0977fe700 1 -- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 msgr2=0x7fb0981c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb0977fe700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 0x7fb0981c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb0977fe700 1 -- 172.21.15.32:0/4025360337 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009a70 con 0x7fb09809e000 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb09cd59700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809e410 0x7fb098131630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb097fff700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 0x7fb0981c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb0977fe700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 0x7fb0981c31f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005910 tx=0x7fb08c00c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c005d70 con 0x7fb09809e000 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0070d0 con 0x7fb09809e000 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880094a0 con 0x7fb09809e000 2023-09-30T22:30:28.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.572+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c007460 con 0x7fb09809e000 2023-09-30T22:30:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.573+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981330e0 con 0x7fb09809e000 2023-09-30T22:30:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.573+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb098074ea0 con 0x7fb09809e000 2023-09-30T22:30:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.574+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb08c0075c0 con 0x7fb09809e000 2023-09-30T22:30:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.574+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb08c03eb00 con 0x7fb09809e000 2023-09-30T22:30:28.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.578+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb08c00f260 con 0x7fb09809e000 2023-09-30T22:30:28.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.648+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0fc80747f0 con 0x7f0fc809de50 2023-09-30T22:30:28.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.648+0000 7f0faf7fe700 1 -- 172.21.15.32:0/2580670593 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0fb802c030 con 0x7f0fc809de50 2023-09-30T22:30:28.648 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc809de50 msgr2=0x7f0fc8119660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc809de50 0x7f0fc8119660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8016000 tx=0x7f0fb8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 shutdown_connections 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0fc8130410 0x7f0fc811a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0fc809de50 0x7f0fc8119660 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 --2- 172.21.15.32:0/2580670593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fc809e7c0 0x7f0fc8119ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 >> 172.21.15.32:0/2580670593 conn(0x7f0fc80a2610 msgr2=0x7f0fc812e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 shutdown_connections 2023-09-30T22:30:28.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.651+0000 7f0fcd2ca700 1 -- 172.21.15.32:0/2580670593 wait complete. 2023-09-30T22:30:28.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:28.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.754+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc9e81237b0 con 0x7fc9e8139170 2023-09-30T22:30:28.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.755+0000 7fc9deffd700 1 -- 172.21.15.32:0/1827023502 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9e0012340 con 0x7fc9e8139170 2023-09-30T22:30:28.755 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8139170 msgr2=0x7fc9e8121090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8139170 0x7fc9e8121090 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000efc0 tx=0x7fc9e000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 shutdown_connections 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8139170 0x7fc9e8121090 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc9e809d6f0 0x7fc9e8120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 --2- 172.21.15.32:0/1827023502 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc9e809e1d0 0x7fc9e8120b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 >> 172.21.15.32:0/1827023502 conn(0x7fc9e8066f50 msgr2=0x7fc9e809b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 shutdown_connections 2023-09-30T22:30:28.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.757+0000 7fc9f07da700 1 -- 172.21.15.32:0/1827023502 wait complete. 2023-09-30T22:30:28.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:28.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.786+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb0981328f0 con 0x7fb09809e000 2023-09-30T22:30:28.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.786+0000 7fb094ff9700 1 -- 172.21.15.32:0/4025360337 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb08c017030 con 0x7fb09809e000 2023-09-30T22:30:28.786 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:28.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 msgr2=0x7fb0981c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 0x7fb0981c31f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005910 tx=0x7fb08c00c9e0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 shutdown_connections 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb09809e410 0x7fb098131630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb09809d6f0 0x7fb0981c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 --2- 172.21.15.32:0/4025360337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb09809e000 0x7fb0981c31f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 >> 172.21.15.32:0/4025360337 conn(0x7fb098066f50 msgr2=0x7fb098070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 shutdown_connections 2023-09-30T22:30:28.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:28.788+0000 7fb09e7bc700 1 -- 172.21.15.32:0/4025360337 wait complete. 2023-09-30T22:30:28.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:29.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.353+0000 7fe1b7f44700 1 Processor -- start 2023-09-30T22:30:29.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.353+0000 7fe1b7f44700 1 -- start start 2023-09-30T22:30:29.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b7f44700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009d6f0 0x7fe1b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b7f44700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1b009e410 0x7fe1b0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b7f44700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b009f130 con 0x7fe1b009e000 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b7f44700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b009f520 con 0x7fe1b009e410 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b7f44700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b009e930 con 0x7fe1b009d6f0 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b64e1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1b009e410 0x7fe1b0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b54df700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1b009e000 0x7fe1b0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53248/0 (socket says 172.21.15.32:53248) 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b54df700 1 -- 172.21.15.32:0/2531279099 learned_addr learned my addr 172.21.15.32:0/2531279099 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b5ce0700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009d6f0 0x7fe1b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b64e1700 1 -- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009d6f0 msgr2=0x7fe1b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b64e1700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009d6f0 0x7fe1b009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b64e1700 1 -- 172.21.15.32:0/2531279099 >> v1:172.21.15.32:6789/0 conn(0x7fe1b009e000 legacy=0x7fe1b0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b64e1700 1 -- 172.21.15.32:0/2531279099 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b009b2d0 con 0x7fe1b009e410 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b4cde700 1 -- 172.21.15.32:0/2531279099 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607562725 0 0) 0x7fe1b009f130 con 0x7fe1b009e000 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.354+0000 7fe1b5ce0700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009d6f0 0x7fe1b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b64e1700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e410 0x7fe1b0130d70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009840 tx=0x7fe1a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b370b6f10c34f9ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b4cde700 1 -- 172.21.15.32:0/2531279099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8011830 con 0x7fe1b009e410 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b4cde700 1 -- 172.21.15.32:0/2531279099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8011990 con 0x7fe1b009e410 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b4cde700 1 -- 172.21.15.32:0/2531279099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8011d20 con 0x7fe1b009e410 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b7f44700 1 -- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e410 msgr2=0x7fe1b0130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e410 0x7fe1b0130d70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009840 tx=0x7fe1a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b7f44700 1 -- 172.21.15.32:0/2531279099 shutdown_connections 2023-09-30T22:30:29.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e410 0x7fe1b0130d70 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/2531279099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009d6f0 0x7fe1b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.355+0000 7fe1b7f44700 1 -- 172.21.15.32:0/2531279099 >> 172.21.15.32:0/2531279099 conn(0x7fe1b0066f50 msgr2=0x7fe1b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.356+0000 7fe1b7f44700 1 -- 172.21.15.32:0/2531279099 shutdown_connections 2023-09-30T22:30:29.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.356+0000 7fe1b7f44700 1 -- 172.21.15.32:0/2531279099 wait complete. 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 Processor -- start 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 -- start start 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 0x7fe1b01194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 0x7fe1b01199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009e410 0x7fe1b0119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b011a960 con 0x7fe1b009e000 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00a1b80 con 0x7fe1b009e410 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b7f44700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b011ccf0 con 0x7fe1b009d6f0 2023-09-30T22:30:29.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b54df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 0x7fe1b01199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b5ce0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 0x7fe1b01194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b54df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 0x7fe1b01199f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58892/0 (socket says 172.21.15.32:58892) 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b5ce0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 0x7fe1b01194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46330/0 (socket says 172.21.15.32:46330) 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b54df700 1 -- 172.21.15.32:0/1356815560 learned_addr learned my addr 172.21.15.32:0/1356815560 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.357+0000 7fe1b64e1700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009e410 0x7fe1b0119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b54df700 1 -- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 msgr2=0x7fe1b01194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b54df700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 0x7fe1b01194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b54df700 1 -- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009e410 msgr2=0x7fe1b0119f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b54df700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009e410 0x7fe1b0119f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b54df700 1 -- 172.21.15.32:0/1356815560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0009a70 con 0x7fe1b009e000 2023-09-30T22:30:29.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b64e1700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009e410 0x7fe1b0119f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b5ce0700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 0x7fe1b01194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b54df700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 0x7fe1b01199f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4001eb0 tx=0x7fe1a400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4004c80 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a4004de0 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4007310 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a80094a0 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.358+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b011bab0 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.359+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe1a4007470 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.359+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe1a403f270 con 0x7fe1b009e000 2023-09-30T22:30:29.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.359+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b011cb30 con 0x7fe1b009e000 2023-09-30T22:30:29.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.364+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe1a40123b0 con 0x7fe1b009e000 2023-09-30T22:30:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.570+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe1b0074ea0 con 0x7fe1b009e000 2023-09-30T22:30:29.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.570+0000 7fe1ae7fc700 1 -- 172.21.15.32:0/1356815560 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe1a4011360 con 0x7fe1b009e000 2023-09-30T22:30:29.571 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 msgr2=0x7fe1b01199f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 0x7fe1b01199f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4001eb0 tx=0x7fe1a400c800 comp rx=0 tx=0).stop 2023-09-30T22:30:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 shutdown_connections 2023-09-30T22:30:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b009e410 0x7fe1b0119f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe1b009d6f0 0x7fe1b01194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 --2- 172.21.15.32:0/1356815560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b009e000 0x7fe1b01199f0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 >> 172.21.15.32:0/1356815560 conn(0x7fe1b0066f50 msgr2=0x7fe1b006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 shutdown_connections 2023-09-30T22:30:29.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.573+0000 7fe1b7f44700 1 -- 172.21.15.32:0/1356815560 wait complete. 2023-09-30T22:30:29.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:29.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:29.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:29.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:29.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 Processor -- start 2023-09-30T22:30:29.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 -- start start 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85241337f0 0x7f8524138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8524138dc0 0x7f852413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852406b990 con 0x7f8524133420 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852409b0e0 con 0x7f8524138dc0 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f852a394700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8524133c20 con 0x7f85241337f0 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f8528931700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8524138dc0 0x7f852413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f8523fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8524133420 0x7f8524133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53262/0 (socket says 172.21.15.32:53262) 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f8523fff700 1 -- 172.21.15.32:0/1143303930 learned_addr learned my addr 172.21.15.32:0/1143303930 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.812+0000 7f85237fe700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85241337f0 0x7f8524138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8528931700 1 -- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85241337f0 msgr2=0x7f8524138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8528931700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85241337f0 0x7f8524138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8528931700 1 -- 172.21.15.32:0/1143303930 >> v1:172.21.15.32:6789/0 conn(0x7f8524133420 legacy=0x7f8524133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8522ffd700 1 -- 172.21.15.32:0/1143303930 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871413463 0 0) 0x7f852406b990 con 0x7f8524133420 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8528931700 1 -- 172.21.15.32:0/1143303930 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240a0590 con 0x7f8524138dc0 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f85237fe700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85241337f0 0x7f8524138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8528931700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 0x7f852413b1a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f851c009d90 tx=0x7f851c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd1e13304ea9f2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8522ffd700 1 -- 172.21.15.32:0/1143303930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00e070 con 0x7f8524138dc0 2023-09-30T22:30:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8522ffd700 1 -- 172.21.15.32:0/1143303930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00bb20 con 0x7f8524138dc0 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.813+0000 7f8522ffd700 1 -- 172.21.15.32:0/1143303930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c012430 con 0x7f8524138dc0 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 -- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 msgr2=0x7f852413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 0x7f852413b1a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f851c009d90 tx=0x7f851c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 -- 172.21.15.32:0/1143303930 shutdown_connections 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 0x7f852413b1a0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 --2- 172.21.15.32:0/1143303930 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85241337f0 0x7f8524138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 -- 172.21.15.32:0/1143303930 >> 172.21.15.32:0/1143303930 conn(0x7f85240a4210 msgr2=0x7f85240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 -- 172.21.15.32:0/1143303930 shutdown_connections 2023-09-30T22:30:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.814+0000 7f852a394700 1 -- 172.21.15.32:0/1143303930 wait complete. 2023-09-30T22:30:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.816+0000 7f852a394700 1 Processor -- start 2023-09-30T22:30:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.816+0000 7f852a394700 1 -- start start 2023-09-30T22:30:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f852a394700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 0x7f8524070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f852a394700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85241337f0 0x7f8524071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f852a394700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 0x7f8524071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f852a394700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852409ca90 con 0x7f8524138dc0 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f852a394700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8524072720 con 0x7f85241337f0 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f852a394700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8524138930 con 0x7f8524133420 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 0x7f8524070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f85237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85241337f0 0x7f8524071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 0x7f8524070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46346/0 (socket says 172.21.15.32:46346) 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 -- 172.21.15.32:0/1227562923 learned_addr learned my addr 172.21.15.32:0/1227562923 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 -- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85241337f0 msgr2=0x7f8524071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85241337f0 0x7f8524071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 -- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 msgr2=0x7f8524071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 0x7f8524071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f8523fff700 1 -- 172.21.15.32:0/1227562923 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180094a0 con 0x7f8524133420 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.817+0000 7f85237fe700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85241337f0 0x7f8524071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.818+0000 7f8523fff700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 0x7f8524070c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f851400c520 tx=0x7f851400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.818+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514004c80 con 0x7f8524133420 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.818+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8514004de0 con 0x7f8524133420 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.818+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c009a70 con 0x7f8524133420 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.818+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514007310 con 0x7f8524133420 2023-09-30T22:30:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.818+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852409d7c0 con 0x7f8524133420 2023-09-30T22:30:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.819+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8524134040 con 0x7f8524133420 2023-09-30T22:30:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.820+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8514007470 con 0x7f8524133420 2023-09-30T22:30:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.820+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f851403f8f0 con 0x7f8524133420 2023-09-30T22:30:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.825+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85140133c0 con 0x7f8524133420 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.916+0000 7f2207464700 1 Processor -- start 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.916+0000 7f2207464700 1 -- start start 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2207464700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f220009d6f0 0x7f220009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2207464700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 0x7f22001339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2207464700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009e410 0x7f2200133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2207464700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006b460 con 0x7f220009d6f0 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2207464700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220009b120 con 0x7f220009e000 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2207464700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f220009a7e0 con 0x7f220009e410 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 0x7f22001339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 0x7f22001339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47988/0 (socket says 172.21.15.32:47988) 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 -- 172.21.15.32:0/2265118404 learned_addr learned my addr 172.21.15.32:0/2265118404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2205200700 1 --2- 172.21.15.32:0/2265118404 >> v2:172.21.15.32:3300/0 conn(0x7f220009d6f0 0x7f220009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f2205a01700 1 --2- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009e410 0x7f2200133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 -- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009e410 msgr2=0x7f2200133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 --2- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009e410 0x7f2200133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 -- 172.21.15.32:0/2265118404 >> v2:172.21.15.32:3300/0 conn(0x7f220009d6f0 msgr2=0x7f220009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 --2- 172.21.15.32:0/2265118404 >> v2:172.21.15.32:3300/0 conn(0x7f220009d6f0 0x7f220009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 -- 172.21.15.32:0/2265118404 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220009a500 con 0x7f220009e000 2023-09-30T22:30:29.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.917+0000 7f22049ff700 1 --2- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 0x7f22001339f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009920 tx=0x7f21f0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dca7f0d3b8f0fcd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f21f7fff700 1 -- 172.21.15.32:0/2265118404 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0011830 con 0x7f220009e000 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f21f7fff700 1 -- 172.21.15.32:0/2265118404 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0011990 con 0x7f220009e000 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f21f7fff700 1 -- 172.21.15.32:0/2265118404 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0011d20 con 0x7f220009e000 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 -- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 msgr2=0x7f22001339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 --2- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 0x7f22001339f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009920 tx=0x7f21f0009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 -- 172.21.15.32:0/2265118404 shutdown_connections 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 --2- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009e410 0x7f2200133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 --2- 172.21.15.32:0/2265118404 >> v2:172.21.15.32:3300/0 conn(0x7f220009d6f0 0x7f220009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 --2- 172.21.15.32:0/2265118404 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e000 0x7f22001339f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.918+0000 7f2207464700 1 -- 172.21.15.32:0/2265118404 >> 172.21.15.32:0/2265118404 conn(0x7f2200066f50 msgr2=0x7f220009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.919+0000 7f2207464700 1 -- 172.21.15.32:0/2265118404 shutdown_connections 2023-09-30T22:30:29.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.919+0000 7f2207464700 1 -- 172.21.15.32:0/2265118404 wait complete. 2023-09-30T22:30:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 Processor -- start 2023-09-30T22:30:29.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 -- start start 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 0x7f22001a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220009e000 0x7f22001a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e410 0x7f22001a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22001a79e0 con 0x7f220009e000 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22001a7370 con 0x7f220009e410 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2207464700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22001a9450 con 0x7f220009d6f0 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2205200700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 0x7f22001a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2205200700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 0x7f22001a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:46370/0 (socket says 172.21.15.32:46370) 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2205200700 1 -- 172.21.15.32:0/1998636204 learned_addr learned my addr 172.21.15.32:0/1998636204 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:29.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f2205a01700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e410 0x7f22001a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.920+0000 7f22049ff700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220009e000 0x7f22001a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205200700 1 -- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e410 msgr2=0x7f22001a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205200700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e410 0x7f22001a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205200700 1 -- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220009e000 msgr2=0x7f22001a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205200700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220009e000 0x7f22001a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205200700 1 -- 172.21.15.32:0/1998636204 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f0009550 con 0x7f220009d6f0 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f22049ff700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220009e000 0x7f22001a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205a01700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e410 0x7f22001a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2205200700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 0x7f22001a5ec0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00e9e0 tx=0x7f21fc00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc009710 con 0x7f220009d6f0 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc00c850 con 0x7f220009d6f0 2023-09-30T22:30:29.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22001a87e0 con 0x7f220009d6f0 2023-09-30T22:30:29.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.921+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00cbc0 con 0x7f220009d6f0 2023-09-30T22:30:29.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.922+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22001a9960 con 0x7f220009d6f0 2023-09-30T22:30:29.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.922+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21fc00cd20 con 0x7f220009d6f0 2023-09-30T22:30:29.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.923+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f21fc03f7a0 con 0x7f220009d6f0 2023-09-30T22:30:29.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.923+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2200074ea0 con 0x7f220009d6f0 2023-09-30T22:30:29.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.927+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21fc021020 con 0x7f220009d6f0 2023-09-30T22:30:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.954+0000 7f1c324dc700 1 Processor -- start 2023-09-30T22:30:29.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.954+0000 7f1c324dc700 1 -- start start 2023-09-30T22:30:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.955+0000 7f1c324dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c2c0a0090 0x7f1c2c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.955+0000 7f1c324dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.955+0000 7f1c324dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c09ba70 con 0x7f1c2c09c650 2023-09-30T22:30:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.955+0000 7f1c324dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c09e8f0 con 0x7f1c2c0a0090 2023-09-30T22:30:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.955+0000 7f1c324dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0a0760 con 0x7f1c2c09d130 2023-09-30T22:30:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.955+0000 7f1c2b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c2c0a0090 0x7f1c2c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c2c09c650 0x7f1c2c09ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53274/0 (socket says 172.21.15.32:53274) 2023-09-30T22:30:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2bfff700 1 -- 172.21.15.32:0/1956033015 learned_addr learned my addr 172.21.15.32:0/1956033015 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c30a79700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 msgr2=0x7f1c2c139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2b7fe700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/1956033015 >> v1:172.21.15.32:6789/0 conn(0x7f1c2c09c650 legacy=0x7f1c2c09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/1956033015 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c09a080 con 0x7f1c2c0a0090 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2affd700 1 -- 172.21.15.32:0/1956033015 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513899596 0 0) 0x7f1c2c09ba70 con 0x7f1c2c09c650 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c30a79700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2b7fe700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c0a0090 0x7f1c2c136c20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009840 tx=0x7f1c20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f0e7c7cc3ebb41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2affd700 1 -- 172.21.15.32:0/1956033015 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20011830 con 0x7f1c2c0a0090 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2affd700 1 -- 172.21.15.32:0/1956033015 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20011990 con 0x7f1c2c0a0090 2023-09-30T22:30:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.956+0000 7f1c2affd700 1 -- 172.21.15.32:0/1956033015 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20011d20 con 0x7f1c2c0a0090 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 -- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c0a0090 msgr2=0x7f1c2c136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c0a0090 0x7f1c2c136c20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009840 tx=0x7f1c20009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 -- 172.21.15.32:0/1956033015 shutdown_connections 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 --2- 172.21.15.32:0/1956033015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c0a0090 0x7f1c2c136c20 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 -- 172.21.15.32:0/1956033015 >> 172.21.15.32:0/1956033015 conn(0x7f1c2c0a3f70 msgr2=0x7f1c2c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 -- 172.21.15.32:0/1956033015 shutdown_connections 2023-09-30T22:30:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.957+0000 7f1c324dc700 1 -- 172.21.15.32:0/1956033015 wait complete. 2023-09-30T22:30:29.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.958+0000 7f1c324dc700 1 Processor -- start 2023-09-30T22:30:29.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.958+0000 7f1c324dc700 1 -- start start 2023-09-30T22:30:29.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c324dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c09c650 0x7f1c2c11a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c324dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c11ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c324dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2c0a0090 0x7f1c2c11b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c324dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c11cca0 con 0x7f1c2c09c650 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c324dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c09d9f0 con 0x7f1c2c09d130 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c324dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c11d610 con 0x7f1c2c0a0090 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c11ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c11ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48020/0 (socket says 172.21.15.32:48020) 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/4176260321 learned_addr learned my addr 172.21.15.32:0/4176260321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c30a79700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2c0a0090 0x7f1c2c11b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2bfff700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c09c650 0x7f1c2c11a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2c0a0090 msgr2=0x7f1c2c11b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2c0a0090 0x7f1c2c11b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c09c650 msgr2=0x7f1c2c11a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.959+0000 7f1c2b7fe700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c09c650 0x7f1c2c11a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c2b7fe700 1 -- 172.21.15.32:0/4176260321 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c24009a70 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c2b7fe700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c11ab60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1c200090b0 tx=0x7f1c20004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200045d0 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2001f070 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200038c0 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c200094a0 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.960+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c1389a0 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.961+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c20003a20 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.961+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c11c7c0 con 0x7f1c2c09d130 2023-09-30T22:30:29.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.961+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1c20021070 con 0x7f1c2c09d130 2023-09-30T22:30:29.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:29.966+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c2002f3a0 con 0x7f1c2c09d130 2023-09-30T22:30:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.035+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f852409ccb0 con 0x7f8524133420 2023-09-30T22:30:30.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.035+0000 7f8520ff9700 1 -- 172.21.15.32:0/1227562923 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f851401b340 con 0x7f8524133420 2023-09-30T22:30:30.035 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 msgr2=0x7f8524070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 0x7f8524070c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f851400c520 tx=0x7f851400c830 comp rx=0 tx=0).stop 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 shutdown_connections 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8524138dc0 0x7f8524071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8524133420 0x7f8524070c00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 --2- 172.21.15.32:0/1227562923 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85241337f0 0x7f8524071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 >> 172.21.15.32:0/1227562923 conn(0x7f85240a4210 msgr2=0x7f85240a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 shutdown_connections 2023-09-30T22:30:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.038+0000 7f852a394700 1 -- 172.21.15.32:0/1227562923 wait complete. 2023-09-30T22:30:30.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.138+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f220006c360 con 0x7f220009d6f0 2023-09-30T22:30:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.138+0000 7f21f5ffb700 1 -- 172.21.15.32:0/1998636204 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21fc0232f0 con 0x7f220009d6f0 2023-09-30T22:30:30.139 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 msgr2=0x7f22001a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 0x7f22001a5ec0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00e9e0 tx=0x7f21fc00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 shutdown_connections 2023-09-30T22:30:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f220009e410 0x7f22001a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f220009d6f0 0x7f22001a5ec0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 --2- 172.21.15.32:0/1998636204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220009e000 0x7f22001a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 >> 172.21.15.32:0/1998636204 conn(0x7f2200066f50 msgr2=0x7f22001badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 shutdown_connections 2023-09-30T22:30:30.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.141+0000 7f2207464700 1 -- 172.21.15.32:0/1998636204 wait complete. 2023-09-30T22:30:30.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.175+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1c2c0747f0 con 0x7f1c2c09d130 2023-09-30T22:30:30.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.176+0000 7f1c28ff9700 1 -- 172.21.15.32:0/4176260321 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c20015030 con 0x7f1c2c09d130 2023-09-30T22:30:30.176 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 msgr2=0x7f1c2c11ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c11ab60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1c200090b0 tx=0x7f1c20004750 comp rx=0 tx=0).stop 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 shutdown_connections 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1c2c0a0090 0x7f1c2c11b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2c09c650 0x7f1c2c11a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 --2- 172.21.15.32:0/4176260321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c2c09d130 0x7f1c2c11ab60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 >> 172.21.15.32:0/4176260321 conn(0x7f1c2c0a3f70 msgr2=0x7f1c2c09eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 shutdown_connections 2023-09-30T22:30:30.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.178+0000 7f1c324dc700 1 -- 172.21.15.32:0/4176260321 wait complete. 2023-09-30T22:30:30.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:30.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.733+0000 7f68430e0700 1 Processor -- start 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.733+0000 7f68430e0700 1 -- start start 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f68430e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c133b10 0x7f683c133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f68430e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f683c09de50 0x7f683c09e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f68430e0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c1334b0 con 0x7f683c09e7e0 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f68430e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c134420 con 0x7f683c09de50 2023-09-30T22:30:30.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f68430e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f683c070bc0 con 0x7f683c133b10 2023-09-30T22:30:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f683c09de50 0x7f683c09e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f683c09de50 0x7f683c09e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59594/0 (socket says 172.21.15.32:59594) 2023-09-30T22:30:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f684167d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f683c09e7e0 0x7f683c13a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34312/0 (socket says 172.21.15.32:34312) 2023-09-30T22:30:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 -- 172.21.15.32:0/119574975 learned_addr learned my addr 172.21.15.32:0/119574975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f6840e7c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c133b10 0x7f683c133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 -- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c133b10 msgr2=0x7f683c133ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 --2- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c133b10 0x7f683c133ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 -- 172.21.15.32:0/119574975 >> v1:172.21.15.32:6789/0 conn(0x7f683c09e7e0 legacy=0x7f683c13a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683bfff700 1 -- 172.21.15.32:0/119574975 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c070350 con 0x7f683c09de50 2023-09-30T22:30:30.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.734+0000 7f683b7fe700 1 -- 172.21.15.32:0/119574975 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002290358 0 0) 0x7f683c1334b0 con 0x7f683c09e7e0 2023-09-30T22:30:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f683bfff700 1 --2- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c09de50 0x7f683c09e2a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f68240098f0 tx=0x7f6824009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffc888a38e129a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f683b7fe700 1 -- 172.21.15.32:0/119574975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824011830 con 0x7f683c09de50 2023-09-30T22:30:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f683b7fe700 1 -- 172.21.15.32:0/119574975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6824011990 con 0x7f683c09de50 2023-09-30T22:30:30.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f683b7fe700 1 -- 172.21.15.32:0/119574975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824011d20 con 0x7f683c09de50 2023-09-30T22:30:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 -- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c09de50 msgr2=0x7f683c09e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 --2- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c09de50 0x7f683c09e2a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f68240098f0 tx=0x7f6824009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 -- 172.21.15.32:0/119574975 shutdown_connections 2023-09-30T22:30:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 --2- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c133b10 0x7f683c133ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 --2- 172.21.15.32:0/119574975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c09de50 0x7f683c09e2a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 -- 172.21.15.32:0/119574975 >> 172.21.15.32:0/119574975 conn(0x7f683c0a2aa0 msgr2=0x7f683c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 -- 172.21.15.32:0/119574975 shutdown_connections 2023-09-30T22:30:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.735+0000 7f68430e0700 1 -- 172.21.15.32:0/119574975 wait complete. 2023-09-30T22:30:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.736+0000 7f68430e0700 1 Processor -- start 2023-09-30T22:30:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.736+0000 7f68430e0700 1 -- start start 2023-09-30T22:30:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f68430e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c09de50 0x7f683c11e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f68430e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c09e7e0 0x7f683c11ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f68430e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 0x7f683c11f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f68430e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c121770 con 0x7f683c133b10 2023-09-30T22:30:30.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f68430e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c12fe60 con 0x7f683c09e7e0 2023-09-30T22:30:30.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f68430e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f683c1213b0 con 0x7f683c09de50 2023-09-30T22:30:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f684167d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 0x7f683c11f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f683bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c09e7e0 0x7f683c11ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f684167d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 0x7f683c11f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59598/0 (socket says 172.21.15.32:59598) 2023-09-30T22:30:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f684167d700 1 -- 172.21.15.32:0/4198942182 learned_addr learned my addr 172.21.15.32:0/4198942182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.737+0000 7f6840e7c700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c09de50 0x7f683c11e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f684167d700 1 -- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c09de50 msgr2=0x7f683c11e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f684167d700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c09de50 0x7f683c11e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f684167d700 1 -- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c09e7e0 msgr2=0x7f683c11ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f684167d700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c09e7e0 0x7f683c11ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f684167d700 1 -- 172.21.15.32:0/4198942182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824009550 con 0x7f683c133b10 2023-09-30T22:30:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f684167d700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 0x7f683c11f390 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f683400cbd0 tx=0x7f683400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683400aa60 con 0x7f683c133b10 2023-09-30T22:30:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683400abe0 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68340070d0 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c06f910 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.738+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c122150 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.739+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6834007230 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.739+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f683403f070 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.739+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c074ea0 con 0x7f683c133b10 2023-09-30T22:30:30.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.744+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6834018020 con 0x7f683c133b10 2023-09-30T22:30:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.947+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f683c121c70 con 0x7f683c133b10 2023-09-30T22:30:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.948+0000 7f68397fa700 1 -- 172.21.15.32:0/4198942182 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68340074e0 con 0x7f683c133b10 2023-09-30T22:30:30.948 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 msgr2=0x7f683c11f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 0x7f683c11f390 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f683400cbd0 tx=0x7f683400cee0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 shutdown_connections 2023-09-30T22:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c133b10 0x7f683c11f390 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f683c09de50 0x7f683c11e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 --2- 172.21.15.32:0/4198942182 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c09e7e0 0x7f683c11ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 >> 172.21.15.32:0/4198942182 conn(0x7f683c0a2aa0 msgr2=0x7f683c09f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 shutdown_connections 2023-09-30T22:30:30.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:30.950+0000 7f68430e0700 1 -- 172.21.15.32:0/4198942182 wait complete. 2023-09-30T22:30:30.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:31.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:31.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:31.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.199+0000 7fbbc17e8700 1 Processor -- start 2023-09-30T22:30:31.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.199+0000 7fbbc17e8700 1 -- start start 2023-09-30T22:30:31.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbc17e8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 0x7fbbbc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbc17e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f240 0x7fbbbc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbc17e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09f650 0x7fbbbc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbc17e8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc070940 con 0x7fbbbc09e930 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbc17e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc06fd60 con 0x7fbbbc09f650 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbc17e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc06fa80 con 0x7fbbbc09f240 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbbaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 0x7fbbbc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbbb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09f650 0x7fbbbc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbbaffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 0x7fbbbc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59612/0 (socket says 172.21.15.32:59612) 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbb27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f240 0x7fbbbc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbbaffd700 1 -- 172.21.15.32:0/2989208525 learned_addr learned my addr 172.21.15.32:0/2989208525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbb27fc700 1 -- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09f650 msgr2=0x7fbbbc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbb27fc700 1 --2- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09f650 0x7fbbbc131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbb27fc700 1 -- 172.21.15.32:0/2989208525 >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 msgr2=0x7fbbbc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbb27fc700 1 --2- 172.21.15.32:0/2989208525 >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 0x7fbbbc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbb27fc700 1 -- 172.21.15.32:0/2989208525 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc09fb60 con 0x7fbbbc09f240 2023-09-30T22:30:31.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbbaffd700 1 --2- 172.21.15.32:0/2989208525 >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 0x7fbbbc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.200+0000 7fbbbb7fe700 1 --2- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09f650 0x7fbbbc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.201+0000 7fbbb27fc700 1 --2- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f240 0x7fbbbc1317f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009840 tx=0x7fbba4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ca8955fed40e54 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.201+0000 7fbbba7fc700 1 -- 172.21.15.32:0/2989208525 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4011830 con 0x7fbbbc09f240 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.201+0000 7fbbba7fc700 1 -- 172.21.15.32:0/2989208525 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba4011990 con 0x7fbbbc09f240 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.201+0000 7fbbba7fc700 1 -- 172.21.15.32:0/2989208525 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4011d20 con 0x7fbbbc09f240 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 -- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f240 msgr2=0x7fbbbc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f240 0x7fbbbc1317f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009840 tx=0x7fbba4009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 -- 172.21.15.32:0/2989208525 shutdown_connections 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09f650 0x7fbbbc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/2989208525 >> v2:172.21.15.32:3300/0 conn(0x7fbbbc09e930 0x7fbbbc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/2989208525 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f240 0x7fbbbc1317f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 -- 172.21.15.32:0/2989208525 >> 172.21.15.32:0/2989208525 conn(0x7fbbbc0a4210 msgr2=0x7fbbbc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 -- 172.21.15.32:0/2989208525 shutdown_connections 2023-09-30T22:30:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.202+0000 7fbbc17e8700 1 -- 172.21.15.32:0/2989208525 wait complete. 2023-09-30T22:30:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.203+0000 7fbbc17e8700 1 Processor -- start 2023-09-30T22:30:31.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.203+0000 7fbbc17e8700 1 -- start start 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbc17e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 0x7fbbbc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbc17e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbbc09f240 0x7fbbbc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbc17e8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f650 0x7fbbbc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbc17e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc1b7250 con 0x7fbbbc09f240 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbc17e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc1a9400 con 0x7fbbbc09e930 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbc17e8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc1a9760 con 0x7fbbbc09f650 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 0x7fbbbc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 0x7fbbbc06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44658/0 (socket says 172.21.15.32:44658) 2023-09-30T22:30:31.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbb27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbbc09f240 0x7fbbbc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 -- 172.21.15.32:0/803049520 learned_addr learned my addr 172.21.15.32:0/803049520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbb7fe700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f650 0x7fbbbc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 -- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f650 msgr2=0x7fbbbc06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f650 0x7fbbbc06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 -- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbbc09f240 msgr2=0x7fbbbc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbbc09f240 0x7fbbbc06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbaffd700 1 -- 172.21.15.32:0/803049520 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009550 con 0x7fbbbc09e930 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbb27fc700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbbc09f240 0x7fbbbc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.204+0000 7fbbbb7fe700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f650 0x7fbbbc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.205+0000 7fbbbaffd700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 0x7fbbbc06b8b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00ebe0 tx=0x7fbbac00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.205+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004030 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.205+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac004190 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.205+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004520 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.205+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb4009a70 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.205+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba40094a0 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.206+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc071d20 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.207+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbac00c7f0 con 0x7fbbbc09e930 2023-09-30T22:30:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.207+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbbac03fd10 con 0x7fbbbc09e930 2023-09-30T22:30:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.212+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbac0036a0 con 0x7fbbbc09e930 2023-09-30T22:30:31.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 Processor -- start 2023-09-30T22:30:31.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 -- start start 2023-09-30T22:30:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 0x7fb0ec09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0ec09f160 0x7fb0ec133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec06bf30 con 0x7fb0ec09f160 2023-09-30T22:30:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec09ffb0 con 0x7fb0ec09f5b0 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0f2799700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec06cb10 con 0x7fb0ec09e850 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.307+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 0x7fb0ec09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0eb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0ec09f160 0x7fb0ec133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 0x7fb0ec09ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53166/0 (socket says 172.21.15.32:53166) 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0eb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb0ec09f160 0x7fb0ec133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59630/0 (socket says 172.21.15.32:59630) 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 -- 172.21.15.32:0/3916480363 learned_addr learned my addr 172.21.15.32:0/3916480363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0f0d36700 1 --2- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 -- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 msgr2=0x7fb0ec134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 --2- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 -- 172.21.15.32:0/3916480363 >> v2:172.21.15.32:3300/0 conn(0x7fb0ec09f160 msgr2=0x7fb0ec133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 --2- 172.21.15.32:0/3916480363 >> v2:172.21.15.32:3300/0 conn(0x7fb0ec09f160 0x7fb0ec133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 -- 172.21.15.32:0/3916480363 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0a0270 con 0x7fb0ec09e850 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0ebfff700 1 --2- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 0x7fb0ec09ec20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0098f0 tx=0x7fb0dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690799a43fd30962 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0eaffd700 1 -- 172.21.15.32:0/3916480363 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011830 con 0x7fb0ec09e850 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0eaffd700 1 -- 172.21.15.32:0/3916480363 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc011990 con 0x7fb0ec09e850 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.308+0000 7fb0eaffd700 1 -- 172.21.15.32:0/3916480363 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011d20 con 0x7fb0ec09e850 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 -- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 msgr2=0x7fb0ec09ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 --2- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 0x7fb0ec09ec20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0098f0 tx=0x7fb0dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 -- 172.21.15.32:0/3916480363 shutdown_connections 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 --2- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 --2- 172.21.15.32:0/3916480363 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09e850 0x7fb0ec09ec20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 --2- 172.21.15.32:0/3916480363 >> v2:172.21.15.32:3300/0 conn(0x7fb0ec09f160 0x7fb0ec133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 -- 172.21.15.32:0/3916480363 >> 172.21.15.32:0/3916480363 conn(0x7fb0ec0a3f70 msgr2=0x7fb0ec0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 -- 172.21.15.32:0/3916480363 shutdown_connections 2023-09-30T22:30:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.309+0000 7fb0f2799700 1 -- 172.21.15.32:0/3916480363 wait complete. 2023-09-30T22:30:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.310+0000 7fb0f2799700 1 Processor -- start 2023-09-30T22:30:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.310+0000 7fb0f2799700 1 -- start start 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f2799700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 0x7fb0ec122880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f2799700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09f160 0x7fb0ec122dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f2799700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec123300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f2799700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec125aa0 con 0x7fb0ec09e850 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f2799700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec124490 con 0x7fb0ec09f5b0 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f2799700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec06f8a0 con 0x7fb0ec09f160 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 0x7fb0ec122880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f0d36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec123300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 0x7fb0ec122880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59636/0 (socket says 172.21.15.32:59636) 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0ebfff700 1 -- 172.21.15.32:0/522863201 learned_addr learned my addr 172.21.15.32:0/522863201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0f0d36700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec123300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44674/0 (socket says 172.21.15.32:44674) 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09f160 0x7fb0ec122dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0eb7fe700 1 -- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 msgr2=0x7fb0ec123300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0eb7fe700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec123300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0eb7fe700 1 -- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 msgr2=0x7fb0ec122880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0eb7fe700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 0x7fb0ec122880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0eb7fe700 1 -- 172.21.15.32:0/522863201 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc009550 con 0x7fb0ec09f160 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.311+0000 7fb0ebfff700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 0x7fb0ec122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.312+0000 7fb0eb7fe700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09f160 0x7fb0ec122dc0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e8d0 tx=0x7fb0e000ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.312+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0013680 con 0x7fb0ec09f160 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.312+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e00137e0 con 0x7fb0ec09f160 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.312+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec1263a0 con 0x7fb0ec09f160 2023-09-30T22:30:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.312+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0013b50 con 0x7fb0ec09f160 2023-09-30T22:30:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.312+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec1260c0 con 0x7fb0ec09f160 2023-09-30T22:30:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.313+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0e0013cb0 con 0x7fb0ec09f160 2023-09-30T22:30:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.313+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec0747f0 con 0x7fb0ec09f160 2023-09-30T22:30:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.313+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb0e003f5a0 con 0x7fb0ec09f160 2023-09-30T22:30:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.318+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0e001bde0 con 0x7fb0ec09f160 2023-09-30T22:30:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.341+0000 7f81f71f0700 1 Processor -- start 2023-09-30T22:30:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f71f0700 1 -- start start 2023-09-30T22:30:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f71f0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81f006a770 0x7f81f009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f71f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f013d1f0 0x7f81f013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f71f0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f009a810 con 0x7f81f009dc30 2023-09-30T22:30:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f71f0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f006ab40 con 0x7f81f006a770 2023-09-30T22:30:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f71f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81f009ebc0 con 0x7f81f013d1f0 2023-09-30T22:30:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81effff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81f009dc30 0x7f81f009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34320/0 (socket says 172.21.15.32:34320) 2023-09-30T22:30:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81effff700 1 -- 172.21.15.32:0/3354247070 learned_addr learned my addr 172.21.15.32:0/3354247070 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f578d700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f013d1f0 0x7f81f013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.342+0000 7f81f4f8c700 1 --2- 172.21.15.32:0/3354247070 >> v2:172.21.15.32:3300/0 conn(0x7f81f006a770 0x7f81f009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81f4f8c700 1 -- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f013d1f0 msgr2=0x7f81f013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81f4f8c700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f013d1f0 0x7f81f013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81f4f8c700 1 -- 172.21.15.32:0/3354247070 >> v1:172.21.15.32:6789/0 conn(0x7f81f009dc30 legacy=0x7f81f009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81f4f8c700 1 -- 172.21.15.32:0/3354247070 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f009e920 con 0x7f81f006a770 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81f578d700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f013d1f0 0x7f81f013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81f4f8c700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f006a770 0x7f81f009d6f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009840 tx=0x7f81e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738014ffedc6d0b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81ef7fe700 1 -- 172.21.15.32:0/3354247070 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011830 con 0x7f81f006a770 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81ef7fe700 1 -- 172.21.15.32:0/3354247070 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4011990 con 0x7f81f006a770 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.343+0000 7f81ef7fe700 1 -- 172.21.15.32:0/3354247070 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4011d20 con 0x7f81f006a770 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 -- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f006a770 msgr2=0x7f81f009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f006a770 0x7f81f009d6f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009840 tx=0x7f81e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 -- 172.21.15.32:0/3354247070 shutdown_connections 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f013d1f0 0x7f81f013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3354247070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f006a770 0x7f81f009d6f0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 -- 172.21.15.32:0/3354247070 >> 172.21.15.32:0/3354247070 conn(0x7f81f0066f50 msgr2=0x7f81f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 -- 172.21.15.32:0/3354247070 shutdown_connections 2023-09-30T22:30:31.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.344+0000 7f81f71f0700 1 -- 172.21.15.32:0/3354247070 wait complete. 2023-09-30T22:30:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.345+0000 7f81f71f0700 1 Processor -- start 2023-09-30T22:30:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.345+0000 7f81f71f0700 1 -- start start 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f71f0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f006a770 0x7f81f013bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f71f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 0x7f81f013c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f71f0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81f013d1f0 0x7f81f013c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f71f0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0136310 con 0x7f81f009dc30 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f71f0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f009f1a0 con 0x7f81f013d1f0 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f71f0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81f013f670 con 0x7f81f006a770 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 0x7f81f013c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 0x7f81f013c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59656/0 (socket says 172.21.15.32:59656) 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81effff700 1 -- 172.21.15.32:0/3827417603 learned_addr learned my addr 172.21.15.32:0/3827417603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f4f8c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f006a770 0x7f81f013bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.346+0000 7f81f578d700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81f013d1f0 0x7f81f013c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f4f8c700 1 -- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81f013d1f0 msgr2=0x7f81f013c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f4f8c700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81f013d1f0 0x7f81f013c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f4f8c700 1 -- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 msgr2=0x7f81f013c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f4f8c700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 0x7f81f013c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f4f8c700 1 -- 172.21.15.32:0/3827417603 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009a70 con 0x7f81f006a770 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81effff700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 0x7f81f013c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f4f8c700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f006a770 0x7f81f013bbb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f81e4016000 tx=0x7f81e4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40045d0 con 0x7f81f006a770 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e401f070 con 0x7f81f006a770 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40094a0 con 0x7f81f006a770 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40038e0 con 0x7f81f006a770 2023-09-30T22:30:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.347+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f011f5e0 con 0x7f81f006a770 2023-09-30T22:30:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.348+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81e4003a40 con 0x7f81f006a770 2023-09-30T22:30:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.348+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f009da50 con 0x7f81f006a770 2023-09-30T22:30:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.349+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f81e4021070 con 0x7f81f006a770 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.353+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81e4025c60 con 0x7f81f006a770 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.434+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbbbc074ea0 con 0x7fbbbc09e930 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.435+0000 7fbbb3fff700 1 -- 172.21.15.32:0/803049520 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbac012b80 con 0x7fbbbc09e930 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 msgr2=0x7fbbbc06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 0x7fbbbc06b8b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00ebe0 tx=0x7fbbac00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 shutdown_connections 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbbc09f650 0x7fbbbc06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbbc09e930 0x7fbbbc06b8b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 --2- 172.21.15.32:0/803049520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbbc09f240 0x7fbbbc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.437+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 >> 172.21.15.32:0/803049520 conn(0x7fbbbc0a4210 msgr2=0x7fbbbc1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.438+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 shutdown_connections 2023-09-30T22:30:31.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.438+0000 7fbbc17e8700 1 -- 172.21.15.32:0/803049520 wait complete. 2023-09-30T22:30:31.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.527+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb0ec06bbb0 con 0x7fb0ec09f160 2023-09-30T22:30:31.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.527+0000 7fb0e8ff9700 1 -- 172.21.15.32:0/522863201 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0e0021030 con 0x7fb0ec09f160 2023-09-30T22:30:31.527 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09f160 msgr2=0x7fb0ec122dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09f160 0x7fb0ec122dc0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e8d0 tx=0x7fb0e000ebe0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 shutdown_connections 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb0ec09f5b0 0x7fb0ec123300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0ec09e850 0x7fb0ec122880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 --2- 172.21.15.32:0/522863201 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb0ec09f160 0x7fb0ec122dc0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 >> 172.21.15.32:0/522863201 conn(0x7fb0ec0a3f70 msgr2=0x7fb0ec071130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 shutdown_connections 2023-09-30T22:30:31.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.529+0000 7fb0f2799700 1 -- 172.21.15.32:0/522863201 wait complete. 2023-09-30T22:30:31.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:31.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.561+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f81f0074ea0 con 0x7f81f006a770 2023-09-30T22:30:31.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.561+0000 7f81ed7fa700 1 -- 172.21.15.32:0/3827417603 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81e4015030 con 0x7f81f006a770 2023-09-30T22:30:31.561 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f006a770 msgr2=0x7f81f013bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f006a770 0x7f81f013bbb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f81e4016000 tx=0x7f81e4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 shutdown_connections 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f81f013d1f0 0x7f81f013c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f81f006a770 0x7f81f013bbb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 --2- 172.21.15.32:0/3827417603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f009dc30 0x7f81f013c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 >> 172.21.15.32:0/3827417603 conn(0x7f81f0066f50 msgr2=0x7f81f009b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 shutdown_connections 2023-09-30T22:30:31.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:31.563+0000 7f81f71f0700 1 -- 172.21.15.32:0/3827417603 wait complete. 2023-09-30T22:30:31.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:31.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.109+0000 7f2b8bbb5700 1 Processor -- start 2023-09-30T22:30:32.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8bbb5700 1 -- start start 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8bbb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409d6f0 0x7f2b8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8bbb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b8409e410 0x7f2b84134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8bbb5700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8409b120 con 0x7f2b8409e000 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8bbb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8406b460 con 0x7f2b8409e410 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8bbb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8409a7e0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b8a152700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2b8409e410 0x7f2b84134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b89951700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409d6f0 0x7f2b8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b89150700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b8409e000 0x7f2b841339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34324/0 (socket says 172.21.15.32:34324) 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.110+0000 7f2b89150700 1 -- 172.21.15.32:0/1628073180 learned_addr learned my addr 172.21.15.32:0/1628073180 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8a152700 1 -- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409d6f0 msgr2=0x7f2b8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8a152700 1 --2- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409d6f0 0x7f2b8409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8a152700 1 -- 172.21.15.32:0/1628073180 >> v1:172.21.15.32:6789/0 conn(0x7f2b8409e000 legacy=0x7f2b841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8a152700 1 -- 172.21.15.32:0/1628073180 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8409a500 con 0x7f2b8409e410 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b89951700 1 --2- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409d6f0 0x7f2b8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8a152700 1 --2- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409e410 0x7f2b84134100 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009d90 tx=0x7f2b7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a29d99efa7c0ab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8894f700 1 -- 172.21.15.32:0/1628073180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00e070 con 0x7f2b8409e410 2023-09-30T22:30:32.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8894f700 1 -- 172.21.15.32:0/1628073180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c00bb20 con 0x7f2b8409e410 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8894f700 1 -- 172.21.15.32:0/1628073180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c012430 con 0x7f2b8409e410 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409e410 msgr2=0x7f2b84134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.111+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409e410 0x7f2b84134100 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009d90 tx=0x7f2b7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.112+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/1628073180 shutdown_connections 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.112+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409e410 0x7f2b84134100 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.112+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/1628073180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409d6f0 0x7f2b8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.112+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/1628073180 >> 172.21.15.32:0/1628073180 conn(0x7f2b84066f50 msgr2=0x7f2b8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.112+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/1628073180 shutdown_connections 2023-09-30T22:30:32.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.112+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/1628073180 wait complete. 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 Processor -- start 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 -- start start 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 0x7f2b841a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b8409e000 0x7f2b841a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409e410 0x7f2b841a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841a9870 con 0x7f2b8409d6f0 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841a7b90 con 0x7f2b8409e410 2023-09-30T22:30:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8bbb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b841a9a70 con 0x7f2b8409e000 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b89951700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 0x7f2b841a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b89150700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b8409e000 0x7f2b841a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b8a152700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409e410 0x7f2b841a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b89951700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 0x7f2b841a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59662/0 (socket says 172.21.15.32:59662) 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.113+0000 7f2b89951700 1 -- 172.21.15.32:0/996811148 learned_addr learned my addr 172.21.15.32:0/996811148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89951700 1 -- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b8409e000 msgr2=0x7f2b841a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89951700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b8409e000 0x7f2b841a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89951700 1 -- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409e410 msgr2=0x7f2b841a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89951700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409e410 0x7f2b841a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89951700 1 -- 172.21.15.32:0/996811148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800094a0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89150700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b8409e000 0x7f2b841a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b8a152700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409e410 0x7f2b841a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b89951700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 0x7f2b841a60a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000ebe0 tx=0x7f2b8000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000c850 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8000c9b0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c009a70 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000cd40 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.114+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b841a88f0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.115+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b80013470 con 0x7f2b8409d6f0 2023-09-30T22:30:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.115+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2b8003f7a0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.115+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b841aa400 con 0x7f2b8409d6f0 2023-09-30T22:30:32.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.120+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b80013720 con 0x7f2b8409d6f0 2023-09-30T22:30:32.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.325+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b84074ea0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.325+0000 7f2b7a7fc700 1 -- 172.21.15.32:0/996811148 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b8001b3b0 con 0x7f2b8409d6f0 2023-09-30T22:30:32.325 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:32.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 msgr2=0x7f2b841a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 0x7f2b841a60a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000ebe0 tx=0x7f2b8000eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 shutdown_connections 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2b8409e410 0x7f2b841a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b8409d6f0 0x7f2b841a60a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 --2- 172.21.15.32:0/996811148 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2b8409e000 0x7f2b841a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 >> 172.21.15.32:0/996811148 conn(0x7f2b84066f50 msgr2=0x7f2b841baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 shutdown_connections 2023-09-30T22:30:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.327+0000 7f2b8bbb5700 1 -- 172.21.15.32:0/996811148 wait complete. 2023-09-30T22:30:32.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:32.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:32.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:32.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:32.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.598+0000 7f1f71853700 1 Processor -- start 2023-09-30T22:30:32.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.598+0000 7f1f71853700 1 -- start start 2023-09-30T22:30:32.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f71853700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f71853700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 0x7f1f6c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f71853700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f6c09e410 0x7f1f6c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f71853700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c09f520 con 0x7f1f6c09e410 2023-09-30T22:30:32.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f71853700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c09f130 con 0x7f1f6c09e000 2023-09-30T22:30:32.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f71853700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c09e930 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 0x7f1f6c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 0x7f1f6c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44716/0 (socket says 172.21.15.32:44716) 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 -- 172.21.15.32:0/4271422467 learned_addr learned my addr 172.21.15.32:0/4271422467 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6b7fe700 1 --2- 172.21.15.32:0/4271422467 >> v2:172.21.15.32:3300/0 conn(0x7f1f6c09e410 0x7f1f6c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6affd700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 -- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09d6f0 msgr2=0x7f1f6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 -- 172.21.15.32:0/4271422467 >> v2:172.21.15.32:3300/0 conn(0x7f1f6c09e410 msgr2=0x7f1f6c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 --2- 172.21.15.32:0/4271422467 >> v2:172.21.15.32:3300/0 conn(0x7f1f6c09e410 0x7f1f6c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6a7fc700 1 -- 172.21.15.32:0/4271422467 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c09b2d0 con 0x7f1f6c09e000 2023-09-30T22:30:32.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.599+0000 7f1f6affd700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:32.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.600+0000 7f1f6a7fc700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 0x7f1f6c130660 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009920 tx=0x7f1f60009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b21cca96380a573 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.600+0000 7f1f69ffb700 1 -- 172.21.15.32:0/4271422467 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60011830 con 0x7f1f6c09e000 2023-09-30T22:30:32.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.600+0000 7f1f69ffb700 1 -- 172.21.15.32:0/4271422467 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60011990 con 0x7f1f6c09e000 2023-09-30T22:30:32.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.600+0000 7f1f69ffb700 1 -- 172.21.15.32:0/4271422467 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60011d20 con 0x7f1f6c09e000 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 -- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 msgr2=0x7f1f6c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 0x7f1f6c130660 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009920 tx=0x7f1f60009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 -- 172.21.15.32:0/4271422467 shutdown_connections 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 --2- 172.21.15.32:0/4271422467 >> v2:172.21.15.32:3300/0 conn(0x7f1f6c09e410 0x7f1f6c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09d6f0 0x7f1f6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 --2- 172.21.15.32:0/4271422467 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e000 0x7f1f6c130660 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 -- 172.21.15.32:0/4271422467 >> 172.21.15.32:0/4271422467 conn(0x7f1f6c066f50 msgr2=0x7f1f6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 -- 172.21.15.32:0/4271422467 shutdown_connections 2023-09-30T22:30:32.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.601+0000 7f1f71853700 1 -- 172.21.15.32:0/4271422467 wait complete. 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.602+0000 7f1f71853700 1 Processor -- start 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.602+0000 7f1f71853700 1 -- start start 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f71853700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f71853700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09e000 0x7f1f6c137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f71853700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e410 0x7f1f6c1383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f71853700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c11a500 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f71853700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c139220 con 0x7f1f6c09e410 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f71853700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c11a730 con 0x7f1f6c09e000 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59682/0 (socket says 172.21.15.32:59682) 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f6a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09e000 0x7f1f6c137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f6affd700 1 -- 172.21.15.32:0/3376604484 learned_addr learned my addr 172.21.15.32:0/3376604484 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.603+0000 7f1f6b7fe700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e410 0x7f1f6c1383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6affd700 1 -- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09e000 msgr2=0x7f1f6c137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6affd700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09e000 0x7f1f6c137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6affd700 1 -- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e410 msgr2=0x7f1f6c1383a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6affd700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e410 0x7f1f6c1383a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6affd700 1 -- 172.21.15.32:0/3376604484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60009550 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6a7fc700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09e000 0x7f1f6c137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6b7fe700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e410 0x7f1f6c1383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f6affd700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c137920 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00d800 tx=0x7f1f5c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c004030 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c072560 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c09c750 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c004190 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.604+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c004500 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.605+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f5c0036a0 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.606+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f5c03f090 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.606+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c074ea0 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.611+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f5c019080 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.688+0000 7f994105f700 1 Processor -- start 2023-09-30T22:30:32.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.688+0000 7f994105f700 1 -- start start 2023-09-30T22:30:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f994105f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f993c09e1d0 0x7f993c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f994105f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c139170 0x7f993c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f994105f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c09b120 con 0x7f993c09d6f0 2023-09-30T22:30:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f994105f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c06b460 con 0x7f993c09e1d0 2023-09-30T22:30:32.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f994105f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f993c09a7e0 con 0x7f993c139170 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f993a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f993c09e1d0 0x7f993c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f993ad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f993c09d6f0 0x7f993c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34338/0 (socket says 172.21.15.32:34338) 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f993a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f993c09e1d0 0x7f993c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59698/0 (socket says 172.21.15.32:59698) 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f993b59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c139170 0x7f993c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.689+0000 7f993ad9d700 1 -- 172.21.15.32:0/751795365 learned_addr learned my addr 172.21.15.32:0/751795365 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f993a59c700 1 -- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c139170 msgr2=0x7f993c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f993a59c700 1 --2- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c139170 0x7f993c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f993a59c700 1 -- 172.21.15.32:0/751795365 >> v1:172.21.15.32:6789/0 conn(0x7f993c09d6f0 legacy=0x7f993c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f993a59c700 1 -- 172.21.15.32:0/751795365 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c09a500 con 0x7f993c09e1d0 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f9939d9b700 1 -- 172.21.15.32:0/751795365 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386133029 0 0) 0x7f993c09b120 con 0x7f993c09d6f0 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f993b59e700 1 --2- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c139170 0x7f993c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f993a59c700 1 --2- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c09e1d0 0x7f993c138c30 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f9930009870 tx=0x7f9930009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a858818df3e6eac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f9939d9b700 1 -- 172.21.15.32:0/751795365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930011830 con 0x7f993c09e1d0 2023-09-30T22:30:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f9939d9b700 1 -- 172.21.15.32:0/751795365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930011990 con 0x7f993c09e1d0 2023-09-30T22:30:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.690+0000 7f9939d9b700 1 -- 172.21.15.32:0/751795365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930011d20 con 0x7f993c09e1d0 2023-09-30T22:30:32.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 -- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c09e1d0 msgr2=0x7f993c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 --2- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c09e1d0 0x7f993c138c30 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f9930009870 tx=0x7f9930009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 -- 172.21.15.32:0/751795365 shutdown_connections 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 --2- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c139170 0x7f993c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 --2- 172.21.15.32:0/751795365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c09e1d0 0x7f993c138c30 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 -- 172.21.15.32:0/751795365 >> 172.21.15.32:0/751795365 conn(0x7f993c066f50 msgr2=0x7f993c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 -- 172.21.15.32:0/751795365 shutdown_connections 2023-09-30T22:30:32.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.691+0000 7f994105f700 1 -- 172.21.15.32:0/751795365 wait complete. 2023-09-30T22:30:32.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.693+0000 7f994105f700 1 Processor -- start 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.693+0000 7f994105f700 1 -- start start 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f994105f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 0x7f993c1c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f994105f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c09e1d0 0x7f993c1c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f994105f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c139170 0x7f993c120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f994105f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1366d0 con 0x7f993c139170 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f994105f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1c9130 con 0x7f993c09d6f0 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f994105f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f993c123ab0 con 0x7f993c09e1d0 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f993ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 0x7f993c1c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f993ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 0x7f993c1c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44722/0 (socket says 172.21.15.32:44722) 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f993ad9d700 1 -- 172.21.15.32:0/895679129 learned_addr learned my addr 172.21.15.32:0/895679129 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:32.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f993a59c700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c09e1d0 0x7f993c1c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.694+0000 7f993b59e700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c139170 0x7f993c120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993ad9d700 1 -- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c09e1d0 msgr2=0x7f993c1c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993ad9d700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c09e1d0 0x7f993c1c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993ad9d700 1 -- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c139170 msgr2=0x7f993c120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993ad9d700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c139170 0x7f993c120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993ad9d700 1 -- 172.21.15.32:0/895679129 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934009a70 con 0x7f993c09d6f0 2023-09-30T22:30:32.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993a59c700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c09e1d0 0x7f993c1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f993ad9d700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 0x7f993c1c86b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c520 tx=0x7f992c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c004c80 con 0x7f993c09d6f0 2023-09-30T22:30:32.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c004e00 con 0x7f993c09d6f0 2023-09-30T22:30:32.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99300094a0 con 0x7f993c09d6f0 2023-09-30T22:30:32.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c007310 con 0x7f993c09d6f0 2023-09-30T22:30:32.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.695+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c1243b0 con 0x7f993c09d6f0 2023-09-30T22:30:32.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.696+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f992c007470 con 0x7f993c09d6f0 2023-09-30T22:30:32.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.696+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f992c03f270 con 0x7f993c09d6f0 2023-09-30T22:30:32.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.697+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c09c530 con 0x7f993c09d6f0 2023-09-30T22:30:32.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.702+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f992c0123b0 con 0x7f993c09d6f0 2023-09-30T22:30:32.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 Processor -- start 2023-09-30T22:30:32.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 -- start start 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0a2a70 con 0x7f9fdc09d630 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0a01d0 con 0x7f9fdc138d80 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe231d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc09e2b0 con 0x7f9fdc0a0a30 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe08ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe08ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44726/0 (socket says 172.21.15.32:44726) 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.739+0000 7f9fe08ba700 1 -- 172.21.15.32:0/166008568 learned_addr learned my addr 172.21.15.32:0/166008568 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fe08ba700 1 -- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 msgr2=0x7f9fdc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fe08ba700 1 --2- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fe08ba700 1 -- 172.21.15.32:0/166008568 >> v1:172.21.15.32:6789/0 conn(0x7f9fdc09d630 legacy=0x7f9fdc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fe08ba700 1 -- 172.21.15.32:0/166008568 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc09dc50 con 0x7f9fdc138d80 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fe08ba700 1 --2- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc13b160 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009e80 tx=0x7f9fd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae4bb769211313 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fdaffd700 1 -- 172.21.15.32:0/166008568 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400e070 con 0x7f9fdc138d80 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fdaffd700 1 -- 172.21.15.32:0/166008568 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd400b910 con 0x7f9fdc138d80 2023-09-30T22:30:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.740+0000 7f9fdaffd700 1 -- 172.21.15.32:0/166008568 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400bcc0 con 0x7f9fdc138d80 2023-09-30T22:30:32.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 -- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 msgr2=0x7f9fdc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 --2- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc13b160 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009e80 tx=0x7f9fd400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:32.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 -- 172.21.15.32:0/166008568 shutdown_connections 2023-09-30T22:30:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 --2- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc13b160 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 --2- 172.21.15.32:0/166008568 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 -- 172.21.15.32:0/166008568 >> 172.21.15.32:0/166008568 conn(0x7f9fdc0a3050 msgr2=0x7f9fdc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 -- 172.21.15.32:0/166008568 shutdown_connections 2023-09-30T22:30:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.741+0000 7f9fe231d700 1 -- 172.21.15.32:0/166008568 wait complete. 2023-09-30T22:30:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.743+0000 7f9fe231d700 1 Processor -- start 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.743+0000 7f9fe231d700 1 -- start start 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe231d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 0x7f9fdc133850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe231d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc133d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe231d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc1342d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe231d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc136fb0 con 0x7f9fdc09d630 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe231d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0a0410 con 0x7f9fdc138d80 2023-09-30T22:30:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe231d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc134c40 con 0x7f9fdc0a0a30 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 0x7f9fdc133850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 0x7f9fdc133850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59708/0 (socket says 172.21.15.32:59708) 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 -- 172.21.15.32:0/3672901819 learned_addr learned my addr 172.21.15.32:0/3672901819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fe08ba700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc1342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdb7fe700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 -- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 msgr2=0x7f9fdc133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc133d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 -- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 msgr2=0x7f9fdc1342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc1342d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.744+0000 7f9fdbfff700 1 -- 172.21.15.32:0/3672901819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4009b30 con 0x7f9fdc09d630 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fdbfff700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 0x7f9fdc133850 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e9b0 tx=0x7f9fcc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fe08ba700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc1342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00ee50 con 0x7f9fdc09d630 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc00c7f0 con 0x7f9fdc09d630 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc137290 con 0x7f9fdc09d630 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00cb60 con 0x7f9fdc09d630 2023-09-30T22:30:32.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.745+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc1365d0 con 0x7f9fdc09d630 2023-09-30T22:30:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.746+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc0747f0 con 0x7f9fdc09d630 2023-09-30T22:30:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.746+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9fcc00ccc0 con 0x7f9fdc09d630 2023-09-30T22:30:32.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.746+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9fcc03fba0 con 0x7f9fdc09d630 2023-09-30T22:30:32.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.751+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9fcc01b3b0 con 0x7f9fdc09d630 2023-09-30T22:30:32.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.829+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f6c06c360 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.830+0000 7f1f53fff700 1 -- 172.21.15.32:0/3376604484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f5c013980 con 0x7f1f6c09d6f0 2023-09-30T22:30:32.830 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 msgr2=0x7f1f6c137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c137920 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00d800 tx=0x7f1f5c00db10 comp rx=0 tx=0).stop 2023-09-30T22:30:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 shutdown_connections 2023-09-30T22:30:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f6c09e410 0x7f1f6c1383a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f6c09d6f0 0x7f1f6c137920 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 --2- 172.21.15.32:0/3376604484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f6c09e000 0x7f1f6c137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 >> 172.21.15.32:0/3376604484 conn(0x7f1f6c066f50 msgr2=0x7f1f6c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 shutdown_connections 2023-09-30T22:30:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.832+0000 7f1f71853700 1 -- 172.21.15.32:0/3376604484 wait complete. 2023-09-30T22:30:32.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.915+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f993c09f420 con 0x7f993c09d6f0 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.915+0000 7f99237fe700 1 -- 172.21.15.32:0/895679129 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f992c0102c0 con 0x7f993c09d6f0 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 msgr2=0x7f993c1c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 0x7f993c1c86b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c520 tx=0x7f992c00c830 comp rx=0 tx=0).stop 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 shutdown_connections 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993c139170 0x7f993c120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f993c09d6f0 0x7f993c1c86b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 --2- 172.21.15.32:0/895679129 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f993c09e1d0 0x7f993c1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 >> 172.21.15.32:0/895679129 conn(0x7f993c066f50 msgr2=0x7f993c136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 shutdown_connections 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.917+0000 7f994105f700 1 -- 172.21.15.32:0/895679129 wait complete. 2023-09-30T22:30:32.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.961+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9fdc136a80 con 0x7f9fdc09d630 2023-09-30T22:30:32.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.961+0000 7f9fd8ff9700 1 -- 172.21.15.32:0/3672901819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fcc017070 con 0x7f9fdc09d630 2023-09-30T22:30:32.961 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 msgr2=0x7f9fdc133850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 0x7f9fdc133850 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e9b0 tx=0x7f9fcc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 shutdown_connections 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fdc138d80 0x7f9fdc1342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc09d630 0x7f9fdc133850 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 --2- 172.21.15.32:0/3672901819 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9fdc0a0a30 0x7f9fdc133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 >> 172.21.15.32:0/3672901819 conn(0x7f9fdc0a3050 msgr2=0x7f9fdc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 shutdown_connections 2023-09-30T22:30:32.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:32.964+0000 7f9fe231d700 1 -- 172.21.15.32:0/3672901819 wait complete. 2023-09-30T22:30:32.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:33.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 Processor -- start 2023-09-30T22:30:33.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 -- start start 2023-09-30T22:30:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8136710 0x7f11b8138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11b8139030 0x7f11b813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b806ef30 con 0x7f11b81338a0 2023-09-30T22:30:33.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b806fef0 con 0x7f11b8139030 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.489+0000 7f11be547700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81334b0 con 0x7f11b8136710 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11bcae4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11b8139030 0x7f11b813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8136710 0x7f11b8138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f11b81338a0 0x7f11b8133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34362/0 (socket says 172.21.15.32:34362) 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b7fff700 1 -- 172.21.15.32:0/1183722947 learned_addr learned my addr 172.21.15.32:0/1183722947 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b77fe700 1 -- 172.21.15.32:0/1183722947 >> v2:172.21.15.32:3300/0 conn(0x7f11b8139030 msgr2=0x7f11b813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b77fe700 1 --2- 172.21.15.32:0/1183722947 >> v2:172.21.15.32:3300/0 conn(0x7f11b8139030 0x7f11b813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b77fe700 1 -- 172.21.15.32:0/1183722947 >> v1:172.21.15.32:6789/0 conn(0x7f11b81338a0 legacy=0x7f11b8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b77fe700 1 -- 172.21.15.32:0/1183722947 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b806fc30 con 0x7f11b8136710 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11bcae4700 1 --2- 172.21.15.32:0/1183722947 >> v2:172.21.15.32:3300/0 conn(0x7f11b8139030 0x7f11b813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:33.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b77fe700 1 --2- 172.21.15.32:0/1183722947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8136710 0x7f11b8138af0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009840 tx=0x7f11ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224fd71546249190 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b6ffd700 1 -- 172.21.15.32:0/1183722947 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac011830 con 0x7f11b8136710 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b6ffd700 1 -- 172.21.15.32:0/1183722947 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac011990 con 0x7f11b8136710 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.490+0000 7f11b6ffd700 1 -- 172.21.15.32:0/1183722947 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac011d20 con 0x7f11b8136710 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 -- 172.21.15.32:0/1183722947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8136710 msgr2=0x7f11b8138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 --2- 172.21.15.32:0/1183722947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8136710 0x7f11b8138af0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009840 tx=0x7f11ac009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 -- 172.21.15.32:0/1183722947 shutdown_connections 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 --2- 172.21.15.32:0/1183722947 >> v2:172.21.15.32:3300/0 conn(0x7f11b8139030 0x7f11b813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 --2- 172.21.15.32:0/1183722947 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8136710 0x7f11b8138af0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 -- 172.21.15.32:0/1183722947 >> 172.21.15.32:0/1183722947 conn(0x7f11b80a2aa0 msgr2=0x7f11b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 -- 172.21.15.32:0/1183722947 shutdown_connections 2023-09-30T22:30:33.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.491+0000 7f11be547700 1 -- 172.21.15.32:0/1183722947 wait complete. 2023-09-30T22:30:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.492+0000 7f11be547700 1 Processor -- start 2023-09-30T22:30:33.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.492+0000 7f11be547700 1 -- start start 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11be547700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11b81338a0 0x7f11b81c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11be547700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b8136710 0x7f11b81c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11be547700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 0x7f11b811ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11be547700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b806f640 con 0x7f11b8136710 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11be547700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8130c00 con 0x7f11b8139030 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11be547700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11b811e2b0 con 0x7f11b81338a0 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11b77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b8136710 0x7f11b81c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 0x7f11b811ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11b7fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11b81338a0 0x7f11b81c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11b77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b8136710 0x7f11b81c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59724/0 (socket says 172.21.15.32:59724) 2023-09-30T22:30:33.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 0x7f11b811ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44756/0 (socket says 172.21.15.32:44756) 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11b77fe700 1 -- 172.21.15.32:0/894605596 learned_addr learned my addr 172.21.15.32:0/894605596 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 -- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11b81338a0 msgr2=0x7f11b81c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11b81338a0 0x7f11b81c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 -- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b8136710 msgr2=0x7f11b81c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b8136710 0x7f11b81c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 -- 172.21.15.32:0/894605596 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b0009a70 con 0x7f11b8139030 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11bcae4700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 0x7f11b811ad70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f11b000ef90 tx=0x7f11b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.493+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b000ca90 con 0x7f11b8139030 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.494+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b000f040 con 0x7f11b8139030 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.494+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac0094a0 con 0x7f11b8139030 2023-09-30T22:30:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.494+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b000ce30 con 0x7f11b8139030 2023-09-30T22:30:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.494+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b811efe0 con 0x7f11b8139030 2023-09-30T22:30:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.494+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11b0014480 con 0x7f11b8139030 2023-09-30T22:30:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.494+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f11b003fbd0 con 0x7f11b8139030 2023-09-30T22:30:33.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.495+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b8074ea0 con 0x7f11b8139030 2023-09-30T22:30:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.500+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11b0012930 con 0x7f11b8139030 2023-09-30T22:30:33.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.705+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f11b811e470 con 0x7f11b8139030 2023-09-30T22:30:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.705+0000 7f11b4ff9700 1 -- 172.21.15.32:0/894605596 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11b0012340 con 0x7f11b8139030 2023-09-30T22:30:33.705 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 msgr2=0x7f11b811ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 0x7f11b811ad70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f11b000ef90 tx=0x7f11b000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 shutdown_connections 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f11b8139030 0x7f11b811ad70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f11b81338a0 0x7f11b81c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 --2- 172.21.15.32:0/894605596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11b8136710 0x7f11b81c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.707+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 >> 172.21.15.32:0/894605596 conn(0x7f11b80a2aa0 msgr2=0x7f11b8137200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.708+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 shutdown_connections 2023-09-30T22:30:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.708+0000 7f11be547700 1 -- 172.21.15.32:0/894605596 wait complete. 2023-09-30T22:30:33.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:33.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:33.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:33.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.989+0000 7f854be8d700 1 Processor -- start 2023-09-30T22:30:33.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.989+0000 7f854be8d700 1 -- start start 2023-09-30T22:30:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f854be8d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 0x7f854409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f854be8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f854409e000 0x7f8544130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f854be8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f8544130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f854be8d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854409f520 con 0x7f854409e000 2023-09-30T22:30:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f854be8d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854409f130 con 0x7f854409d6f0 2023-09-30T22:30:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f854be8d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f854409e930 con 0x7f854409e410 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f8549c29700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 0x7f854409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f8549428700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f854409e000 0x7f8544130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f8549c29700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 0x7f854409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44768/0 (socket says 172.21.15.32:44768) 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.990+0000 7f8549c29700 1 -- 172.21.15.32:0/3869445248 learned_addr learned my addr 172.21.15.32:0/3869445248 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549c29700 1 -- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 msgr2=0x7f8544130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f854a42a700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f8544130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549c29700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f8544130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549c29700 1 -- 172.21.15.32:0/3869445248 >> v2:172.21.15.32:3300/0 conn(0x7f854409e000 msgr2=0x7f8544130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549c29700 1 --2- 172.21.15.32:0/3869445248 >> v2:172.21.15.32:3300/0 conn(0x7f854409e000 0x7f8544130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549c29700 1 -- 172.21.15.32:0/3869445248 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854409b2d0 con 0x7f854409d6f0 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549428700 1 --2- 172.21.15.32:0/3869445248 >> v2:172.21.15.32:3300/0 conn(0x7f854409e000 0x7f8544130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f854a42a700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f8544130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8549c29700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 0x7f854409dac0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f854000b940 tx=0x7f854000bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a836375e6171e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8548c27700 1 -- 172.21.15.32:0/3869445248 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540016d20 con 0x7f854409d6f0 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8548c27700 1 -- 172.21.15.32:0/3869445248 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540016e80 con 0x7f854409d6f0 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.991+0000 7f8548c27700 1 -- 172.21.15.32:0/3869445248 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854001e640 con 0x7f854409d6f0 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 -- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 msgr2=0x7f854409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 0x7f854409dac0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f854000b940 tx=0x7f854000bc50 comp rx=0 tx=0).stop 2023-09-30T22:30:33.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 -- 172.21.15.32:0/3869445248 shutdown_connections 2023-09-30T22:30:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409d6f0 0x7f854409dac0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 --2- 172.21.15.32:0/3869445248 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f8544130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 --2- 172.21.15.32:0/3869445248 >> v2:172.21.15.32:3300/0 conn(0x7f854409e000 0x7f8544130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 -- 172.21.15.32:0/3869445248 >> 172.21.15.32:0/3869445248 conn(0x7f8544066f50 msgr2=0x7f854409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 -- 172.21.15.32:0/3869445248 shutdown_connections 2023-09-30T22:30:33.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.992+0000 7f854be8d700 1 -- 172.21.15.32:0/3869445248 wait complete. 2023-09-30T22:30:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.994+0000 7f854be8d700 1 Processor -- start 2023-09-30T22:30:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.994+0000 7f854be8d700 1 -- start start 2023-09-30T22:30:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.994+0000 7f854be8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 0x7f854411e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.994+0000 7f854be8d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409e000 0x7f854411e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f854be8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f854411ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:33.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f854be8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544121540 con 0x7f854409d6f0 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f854be8d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854409c700 con 0x7f854409e000 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f854be8d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f854411fe00 con 0x7f854409e410 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549c29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 0x7f854411e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409e000 0x7f854411e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549c29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 0x7f854411e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59742/0 (socket says 172.21.15.32:59742) 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549c29700 1 -- 172.21.15.32:0/2189191457 learned_addr learned my addr 172.21.15.32:0/2189191457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 -- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 msgr2=0x7f854411ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f854411ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 -- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 msgr2=0x7f854411e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 0x7f854411e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 -- 172.21.15.32:0/2189191457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340094a0 con 0x7f854409e000 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549c29700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 0x7f854411e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.995+0000 7f8549428700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409e000 0x7f854411e730 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f85440a3e20 tx=0x7f853400ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.996+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400eea0 con 0x7f854409e000 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.996+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8534004240 con 0x7f854409e000 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.996+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854000b5a0 con 0x7f854409e000 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.996+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85340036a0 con 0x7f854409e000 2023-09-30T22:30:33.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.996+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85440a19b0 con 0x7f854409e000 2023-09-30T22:30:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.997+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8534003800 con 0x7f854409e000 2023-09-30T22:30:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.997+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f853403f8b0 con 0x7f854409e000 2023-09-30T22:30:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:33.997+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8544074ea0 con 0x7f854409e000 2023-09-30T22:30:34.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.002+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f853400c5e0 con 0x7f854409e000 2023-09-30T22:30:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.077+0000 7f1e4fafe700 1 Processor -- start 2023-09-30T22:30:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.077+0000 7f1e4fafe700 1 -- start start 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.077+0000 7f1e4fafe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e480a0a30 0x7f1e480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.077+0000 7f1e4fafe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 0x7f1e480a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4fafe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e48133930 0x7f1e48133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4fafe700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e480a2a70 con 0x7f1e480a0a30 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4fafe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480a01d0 con 0x7f1e4809d630 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4fafe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4809e2b0 con 0x7f1e48133930 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d099700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 0x7f1e480a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d099700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 0x7f1e480a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44786/0 (socket says 172.21.15.32:44786) 2023-09-30T22:30:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e480a0a30 0x7f1e480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d099700 1 -- 172.21.15.32:0/3778262166 learned_addr learned my addr 172.21.15.32:0/3778262166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4e09b700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e48133930 0x7f1e48133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 -- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e48133930 msgr2=0x7f1e48133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e48133930 0x7f1e48133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 -- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 msgr2=0x7f1e480a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 0x7f1e480a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 -- 172.21.15.32:0/3778262166 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4809dc50 con 0x7f1e480a0a30 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d099700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 0x7f1e480a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.078+0000 7f1e4d89a700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e480a0e00 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009840 tx=0x7f1e44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00194f68fb2eaaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4c898700 1 -- 172.21.15.32:0/3778262166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011830 con 0x7f1e480a0a30 2023-09-30T22:30:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4c898700 1 -- 172.21.15.32:0/3778262166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44011990 con 0x7f1e480a0a30 2023-09-30T22:30:34.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4c898700 1 -- 172.21.15.32:0/3778262166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011d20 con 0x7f1e480a0a30 2023-09-30T22:30:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 -- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 msgr2=0x7f1e480a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e480a0e00 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009840 tx=0x7f1e44009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:34.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 -- 172.21.15.32:0/3778262166 shutdown_connections 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e48133930 0x7f1e48133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e480a0e00 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/3778262166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e4809d630 0x7f1e480a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.079+0000 7f1e4fafe700 1 -- 172.21.15.32:0/3778262166 >> 172.21.15.32:0/3778262166 conn(0x7f1e480a3050 msgr2=0x7f1e480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.080+0000 7f1e4fafe700 1 -- 172.21.15.32:0/3778262166 shutdown_connections 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.080+0000 7f1e4fafe700 1 -- 172.21.15.32:0/3778262166 wait complete. 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.081+0000 7f1e4fafe700 1 Processor -- start 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.081+0000 7f1e4fafe700 1 -- start start 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4fafe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e4809d630 0x7f1e4811f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4fafe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e4811f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4fafe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e48133930 0x7f1e481c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4fafe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4811edd0 con 0x7f1e480a0a30 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4fafe700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4811fd40 con 0x7f1e48133930 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4fafe700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e48121de0 con 0x7f1e4809d630 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4d89a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e4809d630 0x7f1e4811f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4d099700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e4811f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4d89a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e4809d630 0x7f1e4811f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53274/0 (socket says 172.21.15.32:53274) 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4d89a700 1 -- 172.21.15.32:0/715550665 learned_addr learned my addr 172.21.15.32:0/715550665 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4e09b700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e48133930 0x7f1e481c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.082+0000 7f1e4d099700 1 -- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e4809d630 msgr2=0x7f1e4811f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4d099700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e4809d630 0x7f1e4811f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4d099700 1 -- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e48133930 msgr2=0x7f1e481c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4d099700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e48133930 0x7f1e481c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4d099700 1 -- 172.21.15.32:0/715550665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38009550 con 0x7f1e480a0a30 2023-09-30T22:30:34.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4d099700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e4811f800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e9a0 tx=0x7f1e3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38004030 con 0x7f1e480a0a30 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e38004190 con 0x7f1e480a0a30 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e440094a0 con 0x7f1e480a0a30 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38004520 con 0x7f1e480a0a30 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.083+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e480a0410 con 0x7f1e480a0a30 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.084+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e481222c0 con 0x7f1e480a0a30 2023-09-30T22:30:34.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.085+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e380036a0 con 0x7f1e480a0a30 2023-09-30T22:30:34.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.085+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1e3803fb30 con 0x7f1e480a0a30 2023-09-30T22:30:34.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.089+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e38011350 con 0x7f1e480a0a30 2023-09-30T22:30:34.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.121+0000 7f6bb8373700 1 Processor -- start 2023-09-30T22:30:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb8373700 1 -- start start 2023-09-30T22:30:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb8373700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb00a0a30 0x7f6bb00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb8373700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb009d630 0x7f6bb00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb8373700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb00a2a70 con 0x7f6bb0133930 2023-09-30T22:30:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb8373700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00a01d0 con 0x7f6bb009d630 2023-09-30T22:30:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb8373700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb009e2b0 con 0x7f6bb00a0a30 2023-09-30T22:30:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb6910700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6bb0133930 0x7f6bb0133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34370/0 (socket says 172.21.15.32:34370) 2023-09-30T22:30:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb590e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb009d630 0x7f6bb00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.122+0000 7f6bb6910700 1 -- 172.21.15.32:0/143115595 learned_addr learned my addr 172.21.15.32:0/143115595 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb610f700 1 --2- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb00a0a30 0x7f6bb00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb590e700 1 -- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb00a0a30 msgr2=0x7f6bb00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb590e700 1 --2- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb00a0a30 0x7f6bb00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb590e700 1 -- 172.21.15.32:0/143115595 >> v1:172.21.15.32:6789/0 conn(0x7f6bb0133930 legacy=0x7f6bb0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb590e700 1 -- 172.21.15.32:0/143115595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb009dc50 con 0x7f6bb009d630 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb510d700 1 -- 172.21.15.32:0/143115595 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509004918 0 0) 0x7f6bb00a2a70 con 0x7f6bb0133930 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb590e700 1 --2- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb009d630 0x7f6bb00a1340 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009870 tx=0x7f6ba0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb6b64bf819ce179 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb510d700 1 -- 172.21.15.32:0/143115595 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0011830 con 0x7f6bb009d630 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb510d700 1 -- 172.21.15.32:0/143115595 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0011990 con 0x7f6bb009d630 2023-09-30T22:30:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.123+0000 7f6bb510d700 1 -- 172.21.15.32:0/143115595 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0011d20 con 0x7f6bb009d630 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 -- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb009d630 msgr2=0x7f6bb00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 --2- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb009d630 0x7f6bb00a1340 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009870 tx=0x7f6ba0009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 -- 172.21.15.32:0/143115595 shutdown_connections 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 --2- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb00a0a30 0x7f6bb00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 --2- 172.21.15.32:0/143115595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb009d630 0x7f6bb00a1340 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 -- 172.21.15.32:0/143115595 >> 172.21.15.32:0/143115595 conn(0x7f6bb00a3050 msgr2=0x7f6bb00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.124+0000 7f6bb8373700 1 -- 172.21.15.32:0/143115595 shutdown_connections 2023-09-30T22:30:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.125+0000 7f6bb8373700 1 -- 172.21.15.32:0/143115595 wait complete. 2023-09-30T22:30:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.125+0000 7f6bb8373700 1 Processor -- start 2023-09-30T22:30:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 -- start start 2023-09-30T22:30:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb009d630 0x7f6bb0132650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 0x7f6bb0132b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb0133930 0x7f6bb01c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb011ed20 con 0x7f6bb009d630 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb01206b0 con 0x7f6bb00a0a30 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb8373700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb0120480 con 0x7f6bb0133930 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb590e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 0x7f6bb0132b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb590e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 0x7f6bb0132b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44808/0 (socket says 172.21.15.32:44808) 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb590e700 1 -- 172.21.15.32:0/1535570361 learned_addr learned my addr 172.21.15.32:0/1535570361 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb610f700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb009d630 0x7f6bb0132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.126+0000 7f6bb6910700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb0133930 0x7f6bb01c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb590e700 1 -- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb0133930 msgr2=0x7f6bb01c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb590e700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb0133930 0x7f6bb01c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb590e700 1 -- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb009d630 msgr2=0x7f6bb0132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb590e700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb009d630 0x7f6bb0132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb590e700 1 -- 172.21.15.32:0/1535570361 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00094a0 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb590e700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 0x7f6bb0132b90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00090b0 tx=0x7f6ba0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0004340 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba001f070 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb00a0410 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00036a0 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.127+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb0121e40 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.128+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6ba0021030 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.128+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb0122260 con 0x7f6bb00a0a30 2023-09-30T22:30:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.128+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6ba00408f0 con 0x7f6bb00a0a30 2023-09-30T22:30:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.133+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bb0122260 con 0x7f6bb00a0a30 2023-09-30T22:30:34.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.213+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8544121830 con 0x7f854409e000 2023-09-30T22:30:34.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.214+0000 7f853a7fc700 1 -- 172.21.15.32:0/2189191457 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f853402a090 con 0x7f854409e000 2023-09-30T22:30:34.214 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409e000 msgr2=0x7f854411e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409e000 0x7f854411e730 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f85440a3e20 tx=0x7f853400ec60 comp rx=0 tx=0).stop 2023-09-30T22:30:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 shutdown_connections 2023-09-30T22:30:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f854409e410 0x7f854411ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f854409d6f0 0x7f854411e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 --2- 172.21.15.32:0/2189191457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f854409e000 0x7f854411e730 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 >> 172.21.15.32:0/2189191457 conn(0x7f8544066f50 msgr2=0x7f854406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 shutdown_connections 2023-09-30T22:30:34.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.216+0000 7f854be8d700 1 -- 172.21.15.32:0/2189191457 wait complete. 2023-09-30T22:30:34.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.299+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1e481222c0 con 0x7f1e480a0a30 2023-09-30T22:30:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.299+0000 7f1e3e7fc700 1 -- 172.21.15.32:0/715550665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e481222c0 con 0x7f1e480a0a30 2023-09-30T22:30:34.299 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:34.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 msgr2=0x7f1e4811f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e4811f800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e9a0 tx=0x7f1e3800ecb0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 shutdown_connections 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e48133930 0x7f1e481c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1e4809d630 0x7f1e4811f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 --2- 172.21.15.32:0/715550665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e480a0a30 0x7f1e4811f800 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.301+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 >> 172.21.15.32:0/715550665 conn(0x7f1e480a3050 msgr2=0x7f1e481bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.302+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 shutdown_connections 2023-09-30T22:30:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.302+0000 7f1e4fafe700 1 -- 172.21.15.32:0/715550665 wait complete. 2023-09-30T22:30:34.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.340+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6bb0122260 con 0x7f6bb00a0a30 2023-09-30T22:30:34.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.341+0000 7f6ba6ffd700 1 -- 172.21.15.32:0/1535570361 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bb0122260 con 0x7f6bb00a0a30 2023-09-30T22:30:34.341 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 msgr2=0x7f6bb0132b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 0x7f6bb0132b90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00090b0 tx=0x7f6ba0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 shutdown_connections 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6bb0133930 0x7f6bb01c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bb009d630 0x7f6bb0132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 --2- 172.21.15.32:0/1535570361 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6bb00a0a30 0x7f6bb0132b90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 >> 172.21.15.32:0/1535570361 conn(0x7f6bb00a3050 msgr2=0x7f6bb01bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 shutdown_connections 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.343+0000 7f6bb8373700 1 -- 172.21.15.32:0/1535570361 wait complete. 2023-09-30T22:30:34.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:34.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.868+0000 7f8c0344c700 1 Processor -- start 2023-09-30T22:30:34.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.868+0000 7f8c0344c700 1 -- start start 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c0344c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09d6f0 0x7f8bfc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c0344c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bfc09e000 0x7f8bfc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c0344c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc09f130 con 0x7f8bfc09e410 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c0344c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc09f520 con 0x7f8bfc09e000 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c0344c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc09e930 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c009e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bfc09e000 0x7f8bfc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c019e9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8bfc09e410 0x7f8bfc130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34374/0 (socket says 172.21.15.32:34374) 2023-09-30T22:30:34.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c019e9700 1 -- 172.21.15.32:0/3495698084 learned_addr learned my addr 172.21.15.32:0/3495698084 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c011e8700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09d6f0 0x7f8bfc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c009e7700 1 -- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09d6f0 msgr2=0x7f8bfc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c009e7700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09d6f0 0x7f8bfc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c009e7700 1 -- 172.21.15.32:0/3495698084 >> v1:172.21.15.32:6789/0 conn(0x7f8bfc09e410 legacy=0x7f8bfc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c009e7700 1 -- 172.21.15.32:0/3495698084 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc09b2d0 con 0x7f8bfc09e000 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8bf3fff700 1 -- 172.21.15.32:0/3495698084 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415631193 0 0) 0x7f8bfc09f130 con 0x7f8bfc09e410 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.869+0000 7f8c011e8700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09d6f0 0x7f8bfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c009e7700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09e000 0x7f8bfc130660 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009840 tx=0x7f8bf8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d200e31b287bdec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8bf3fff700 1 -- 172.21.15.32:0/3495698084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8011830 con 0x7f8bfc09e000 2023-09-30T22:30:34.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8bf3fff700 1 -- 172.21.15.32:0/3495698084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8011990 con 0x7f8bfc09e000 2023-09-30T22:30:34.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8bf3fff700 1 -- 172.21.15.32:0/3495698084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8011d20 con 0x7f8bfc09e000 2023-09-30T22:30:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c0344c700 1 -- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09e000 msgr2=0x7f8bfc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c0344c700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09e000 0x7f8bfc130660 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009840 tx=0x7f8bf8009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:34.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c0344c700 1 -- 172.21.15.32:0/3495698084 shutdown_connections 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c0344c700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09d6f0 0x7f8bfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c0344c700 1 --2- 172.21.15.32:0/3495698084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09e000 0x7f8bfc130660 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.870+0000 7f8c0344c700 1 -- 172.21.15.32:0/3495698084 >> 172.21.15.32:0/3495698084 conn(0x7f8bfc066f50 msgr2=0x7f8bfc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.871+0000 7f8c0344c700 1 -- 172.21.15.32:0/3495698084 shutdown_connections 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.871+0000 7f8c0344c700 1 -- 172.21.15.32:0/3495698084 wait complete. 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 Processor -- start 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 -- start start 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 0x7f8bfc1c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bfc09e000 0x7f8bfc1c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09e410 0x7f8bfc1c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc125f00 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc09c700 con 0x7f8bfc09e000 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c0344c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc06b1f0 con 0x7f8bfc09e410 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c011e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 0x7f8bfc1c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c011e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 0x7f8bfc1c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59780/0 (socket says 172.21.15.32:59780) 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c011e8700 1 -- 172.21.15.32:0/1601425343 learned_addr learned my addr 172.21.15.32:0/1601425343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c019e9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09e410 0x7f8bfc1c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.872+0000 7f8c009e7700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bfc09e000 0x7f8bfc1c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c011e8700 1 -- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09e410 msgr2=0x7f8bfc1c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c011e8700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09e410 0x7f8bfc1c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c011e8700 1 -- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bfc09e000 msgr2=0x7f8bfc1c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c011e8700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bfc09e000 0x7f8bfc1c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c011e8700 1 -- 172.21.15.32:0/1601425343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0094a0 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c019e9700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09e410 0x7f8bfc1c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c011e8700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 0x7f8bfc1c84a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ebe0 tx=0x7f8bec00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec009760 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec00c850 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf80094a0 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00cbe0 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.873+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc06a950 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.874+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bec00cd40 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.874+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8bec03f710 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.874+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc074ea0 con 0x7f8bfc09d6f0 2023-09-30T22:30:34.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:34.879+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bec0232f0 con 0x7f8bfc09d6f0 2023-09-30T22:30:35.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.084+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8bfc125a80 con 0x7f8bfc09d6f0 2023-09-30T22:30:35.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.085+0000 7f8bf1ffb700 1 -- 172.21.15.32:0/1601425343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bfc125a80 con 0x7f8bfc09d6f0 2023-09-30T22:30:35.085 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 msgr2=0x7f8bfc1c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 0x7f8bfc1c84a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00ebe0 tx=0x7f8bec00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 shutdown_connections 2023-09-30T22:30:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8bfc09e410 0x7f8bfc1c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bfc09d6f0 0x7f8bfc1c84a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 --2- 172.21.15.32:0/1601425343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8bfc09e000 0x7f8bfc1c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 >> 172.21.15.32:0/1601425343 conn(0x7f8bfc066f50 msgr2=0x7f8bfc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 shutdown_connections 2023-09-30T22:30:35.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.087+0000 7f8c0344c700 1 -- 172.21.15.32:0/1601425343 wait complete. 2023-09-30T22:30:35.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:35.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:35.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:35.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 Processor -- start 2023-09-30T22:30:35.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 -- start start 2023-09-30T22:30:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006a770 0x7fc20006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006b080 0x7fc200131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20009ece0 con 0x7fc200131b40 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20009e760 con 0x7fc20006a770 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.381+0000 7fc2055cd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc20009e040 con 0x7fc20006b080 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1feffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006a770 0x7fc20006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fe7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006b080 0x7fc200131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1ff7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc200131b40 0x7fc200138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34388/0 (socket says 172.21.15.32:34388) 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1ff7fe700 1 -- 172.21.15.32:0/606026769 learned_addr learned my addr 172.21.15.32:0/606026769 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fdffb700 1 -- 172.21.15.32:0/606026769 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020729225 0 0) 0x7fc20009ece0 con 0x7fc200131b40 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fdffb700 1 -- 172.21.15.32:0/606026769 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e80035b0 con 0x7fc200131b40 2023-09-30T22:30:35.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fe7fc700 1 -- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006a770 msgr2=0x7fc20006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fe7fc700 1 --2- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006a770 0x7fc20006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fe7fc700 1 -- 172.21.15.32:0/606026769 >> v1:172.21.15.32:6789/0 conn(0x7fc200131b40 legacy=0x7fc200138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fe7fc700 1 -- 172.21.15.32:0/606026769 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20009f630 con 0x7fc20006b080 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1feffd700 1 --2- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006a770 0x7fc20006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.382+0000 7fc1fe7fc700 1 --2- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006b080 0x7fc200131600 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400dc90 tx=0x7fc1f400f050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d300e1a8750117 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc1fdffb700 1 -- 172.21.15.32:0/606026769 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4012070 con 0x7fc20006b080 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc1fdffb700 1 -- 172.21.15.32:0/606026769 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f400d2e0 con 0x7fc20006b080 2023-09-30T22:30:35.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc1fdffb700 1 -- 172.21.15.32:0/606026769 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400f8f0 con 0x7fc20006b080 2023-09-30T22:30:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc2055cd700 1 -- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006b080 msgr2=0x7fc200131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc2055cd700 1 --2- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006b080 0x7fc200131600 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400dc90 tx=0x7fc1f400f050 comp rx=0 tx=0).stop 2023-09-30T22:30:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc2055cd700 1 -- 172.21.15.32:0/606026769 shutdown_connections 2023-09-30T22:30:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc2055cd700 1 --2- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006a770 0x7fc20006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc2055cd700 1 --2- 172.21.15.32:0/606026769 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006b080 0x7fc200131600 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.383+0000 7fc2055cd700 1 -- 172.21.15.32:0/606026769 >> 172.21.15.32:0/606026769 conn(0x7fc200066f50 msgr2=0x7fc20009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.384+0000 7fc2055cd700 1 -- 172.21.15.32:0/606026769 shutdown_connections 2023-09-30T22:30:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.384+0000 7fc2055cd700 1 -- 172.21.15.32:0/606026769 wait complete. 2023-09-30T22:30:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.385+0000 7fc2055cd700 1 Processor -- start 2023-09-30T22:30:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.385+0000 7fc2055cd700 1 -- start start 2023-09-30T22:30:35.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc2055cd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 0x7fc20011e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc2055cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006b080 0x7fc20011e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc2055cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc200131b40 0x7fc20011ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc2055cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2001200e0 con 0x7fc200131b40 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc2055cd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc200071e60 con 0x7fc20006b080 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc2055cd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2001313a0 con 0x7fc20006a770 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1feffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 0x7fc20011e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1feffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 0x7fc20011e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53308/0 (socket says 172.21.15.32:53308) 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1feffd700 1 -- 172.21.15.32:0/3119044682 learned_addr learned my addr 172.21.15.32:0/3119044682 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1fe7fc700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006b080 0x7fc20011e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1ff7fe700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc200131b40 0x7fc20011ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1fe7fc700 1 -- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 msgr2=0x7fc20011e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1fe7fc700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 0x7fc20011e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1fe7fc700 1 -- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc200131b40 msgr2=0x7fc20011ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.386+0000 7fc1fe7fc700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc200131b40 0x7fc20011ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc1fe7fc700 1 -- 172.21.15.32:0/3119044682 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00094a0 con 0x7fc20006b080 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc1feffd700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 0x7fc20011e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc1fe7fc700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006b080 0x7fc20011e900 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4019000 tx=0x7fc1f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4010d40 con 0x7fc20006b080 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40036a0 con 0x7fc20006b080 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f400d940 con 0x7fc20006b080 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4010ee0 con 0x7fc20006b080 2023-09-30T22:30:35.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.387+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc200121ee0 con 0x7fc20006b080 2023-09-30T22:30:35.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.388+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc200074ea0 con 0x7fc20006b080 2023-09-30T22:30:35.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.389+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc1f4003a50 con 0x7fc20006b080 2023-09-30T22:30:35.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.389+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc1f40447c0 con 0x7fc20006b080 2023-09-30T22:30:35.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.394+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1f4015390 con 0x7fc20006b080 2023-09-30T22:30:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.460+0000 7f6eafaa1700 1 Processor -- start 2023-09-30T22:30:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.460+0000 7f6eafaa1700 1 -- start start 2023-09-30T22:30:35.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eafaa1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ea80a0a30 0x7f6ea80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eafaa1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea809d630 0x7f6ea80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eafaa1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea8133930 0x7f6ea8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eafaa1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea80a2a70 con 0x7f6ea80a0a30 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eafaa1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80a01d0 con 0x7f6ea809d630 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eafaa1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea809e2b0 con 0x7f6ea8133930 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ea80a0a30 0x7f6ea80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eae03e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea8133930 0x7f6ea8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead03c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea809d630 0x7f6ea80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ea80a0a30 0x7f6ea80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59798/0 (socket says 172.21.15.32:59798) 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 -- 172.21.15.32:0/1375941999 learned_addr learned my addr 172.21.15.32:0/1375941999 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 -- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea8133930 msgr2=0x7f6ea8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea8133930 0x7f6ea8133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 -- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea809d630 msgr2=0x7f6ea80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea809d630 0x7f6ea80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6ead83d700 1 -- 172.21.15.32:0/1375941999 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea809dc50 con 0x7f6ea80a0a30 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.461+0000 7f6eae03e700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea8133930 0x7f6ea8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.462+0000 7f6ead83d700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea80a0a30 0x7f6ea80a0e00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40098f0 tx=0x7f6ea4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c86c377150fbb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.462+0000 7f6eac83b700 1 -- 172.21.15.32:0/1375941999 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011830 con 0x7f6ea80a0a30 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.462+0000 7f6eac83b700 1 -- 172.21.15.32:0/1375941999 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011990 con 0x7f6ea80a0a30 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.462+0000 7f6eac83b700 1 -- 172.21.15.32:0/1375941999 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011d20 con 0x7f6ea80a0a30 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 -- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea80a0a30 msgr2=0x7f6ea80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea80a0a30 0x7f6ea80a0e00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40098f0 tx=0x7f6ea4009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 -- 172.21.15.32:0/1375941999 shutdown_connections 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea8133930 0x7f6ea8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea80a0a30 0x7f6ea80a0e00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/1375941999 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea809d630 0x7f6ea80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 -- 172.21.15.32:0/1375941999 >> 172.21.15.32:0/1375941999 conn(0x7f6ea80a3050 msgr2=0x7f6ea80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 -- 172.21.15.32:0/1375941999 shutdown_connections 2023-09-30T22:30:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.463+0000 7f6eafaa1700 1 -- 172.21.15.32:0/1375941999 wait complete. 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 Processor -- start 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 -- start start 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 0x7f6ea811f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea80a0a30 0x7f6ea811f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea8133930 0x7f6ea81c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80a0410 con 0x7f6ea809d630 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea811f090 con 0x7f6ea8133930 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.464+0000 7f6eafaa1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8133560 con 0x7f6ea80a0a30 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead83d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 0x7f6ea811f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea80a0a30 0x7f6ea811f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead83d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 0x7f6ea811f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59814/0 (socket says 172.21.15.32:59814) 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead83d700 1 -- 172.21.15.32:0/353077554 learned_addr learned my addr 172.21.15.32:0/353077554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6eae03e700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea8133930 0x7f6ea81c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 -- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea8133930 msgr2=0x7f6ea81c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea8133930 0x7f6ea81c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 -- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 msgr2=0x7f6ea811f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 0x7f6ea811f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 -- 172.21.15.32:0/353077554 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0009a70 con 0x7f6ea80a0a30 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6eae03e700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea8133930 0x7f6ea81c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead83d700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 0x7f6ea811f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6ead03c700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea80a0a30 0x7f6ea811f910 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800e900 tx=0x7f6e9800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.465+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800c640 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.466+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9800c7a0 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.466+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea4009550 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.466+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800cb10 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.466+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea81211b0 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.467+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e9800cc70 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.467+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6e9803f980 con 0x7f6ea80a0a30 2023-09-30T22:30:35.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.467+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea80747f0 con 0x7f6ea80a0a30 2023-09-30T22:30:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.476+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e98017080 con 0x7f6ea80a0a30 2023-09-30T22:30:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.507+0000 7ff2c6d73700 1 Processor -- start 2023-09-30T22:30:35.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.507+0000 7ff2c6d73700 1 -- start start 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c6d73700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c00a0a30 0x7ff2c00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c6d73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c6d73700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c00a2a70 con 0x7ff2c0133930 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c6d73700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c00a01d0 con 0x7ff2c00a0a30 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c6d73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c009e2b0 con 0x7ff2c009d630 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c4b0f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c00a0a30 0x7ff2c00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44852/0 (socket says 172.21.15.32:44852) 2023-09-30T22:30:35.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 -- 172.21.15.32:0/3492855273 learned_addr learned my addr 172.21.15.32:0/3492855273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 -- 172.21.15.32:0/3492855273 >> v2:172.21.15.32:3300/0 conn(0x7ff2c00a0a30 msgr2=0x7ff2c00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 --2- 172.21.15.32:0/3492855273 >> v2:172.21.15.32:3300/0 conn(0x7ff2c00a0a30 0x7ff2c00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 -- 172.21.15.32:0/3492855273 >> v1:172.21.15.32:6789/0 conn(0x7ff2c0133930 legacy=0x7ff2c0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bffff700 1 -- 172.21.15.32:0/3492855273 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c009dc50 con 0x7ff2c009d630 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2bf7fe700 1 -- 172.21.15.32:0/3492855273 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126407038 0 0) 0x7ff2c00a2a70 con 0x7ff2c0133930 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.508+0000 7ff2c4b0f700 1 --2- 172.21.15.32:0/3492855273 >> v2:172.21.15.32:3300/0 conn(0x7ff2c00a0a30 0x7ff2c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2bffff700 1 --2- 172.21.15.32:0/3492855273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c00a1340 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0098f0 tx=0x7ff2ac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b03805a69b189793 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2bf7fe700 1 -- 172.21.15.32:0/3492855273 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac011830 con 0x7ff2c009d630 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2bf7fe700 1 -- 172.21.15.32:0/3492855273 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac011990 con 0x7ff2c009d630 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2bf7fe700 1 -- 172.21.15.32:0/3492855273 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac011d20 con 0x7ff2c009d630 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2c6d73700 1 -- 172.21.15.32:0/3492855273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 msgr2=0x7ff2c00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/3492855273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c00a1340 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0098f0 tx=0x7ff2ac009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2c6d73700 1 -- 172.21.15.32:0/3492855273 shutdown_connections 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/3492855273 >> v2:172.21.15.32:3300/0 conn(0x7ff2c00a0a30 0x7ff2c00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/3492855273 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c00a1340 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.509+0000 7ff2c6d73700 1 -- 172.21.15.32:0/3492855273 >> 172.21.15.32:0/3492855273 conn(0x7ff2c00a3050 msgr2=0x7ff2c00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.510+0000 7ff2c6d73700 1 -- 172.21.15.32:0/3492855273 shutdown_connections 2023-09-30T22:30:35.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.510+0000 7ff2c6d73700 1 -- 172.21.15.32:0/3492855273 wait complete. 2023-09-30T22:30:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.511+0000 7ff2c6d73700 1 Processor -- start 2023-09-30T22:30:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.511+0000 7ff2c6d73700 1 -- start start 2023-09-30T22:30:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c6d73700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c01c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c6d73700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00a0a30 0x7ff2c01c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c6d73700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2c0133930 0x7ff2c01c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c6d73700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c0120620 con 0x7ff2c00a0a30 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c6d73700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c00a5360 con 0x7ff2c009d630 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c6d73700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c00a0410 con 0x7ff2c0133930 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c01c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c01c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44868/0 (socket says 172.21.15.32:44868) 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 -- 172.21.15.32:0/1961435237 learned_addr learned my addr 172.21.15.32:0/1961435237 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2bffff700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00a0a30 0x7ff2c01c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 -- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2c0133930 msgr2=0x7ff2c01c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2c0133930 0x7ff2c01c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 -- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00a0a30 msgr2=0x7ff2c01c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00a0a30 0x7ff2c01c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 -- 172.21.15.32:0/1961435237 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40094a0 con 0x7ff2c009d630 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2bffff700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00a0a30 0x7ff2c01c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.512+0000 7ff2c4b0f700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c01c7ef0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400ebe0 tx=0x7ff2b400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.513+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4009800 con 0x7ff2c009d630 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.513+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b400c910 con 0x7ff2c009d630 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.513+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400cc80 con 0x7ff2c009d630 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.513+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac009550 con 0x7ff2c009d630 2023-09-30T22:30:35.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.513+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c0120c90 con 0x7ff2c009d630 2023-09-30T22:30:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.514+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2b4013470 con 0x7ff2c009d630 2023-09-30T22:30:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.514+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c00747f0 con 0x7ff2c009d630 2023-09-30T22:30:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.515+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff2b403fc00 con 0x7ff2c009d630 2023-09-30T22:30:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.519+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2b4013720 con 0x7ff2c009d630 2023-09-30T22:30:35.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.605+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc200121b60 con 0x7fc20006b080 2023-09-30T22:30:35.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.605+0000 7fc1e77fe700 1 -- 172.21.15.32:0/3119044682 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc1f4024070 con 0x7fc20006b080 2023-09-30T22:30:35.606 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006b080 msgr2=0x7fc20011e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006b080 0x7fc20011e900 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4019000 tx=0x7fc1f4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 shutdown_connections 2023-09-30T22:30:35.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc200131b40 0x7fc20011ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc20006a770 0x7fc20011e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 --2- 172.21.15.32:0/3119044682 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc20006b080 0x7fc20011e900 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 >> 172.21.15.32:0/3119044682 conn(0x7fc200066f50 msgr2=0x7fc2000a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 shutdown_connections 2023-09-30T22:30:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.609+0000 7fc2055cd700 1 -- 172.21.15.32:0/3119044682 wait complete. 2023-09-30T22:30:35.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.696+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6ea8072980 con 0x7f6ea80a0a30 2023-09-30T22:30:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.696+0000 7f6e9e7fc700 1 -- 172.21.15.32:0/353077554 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e98021020 con 0x7f6ea80a0a30 2023-09-30T22:30:35.697 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea80a0a30 msgr2=0x7f6ea811f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea80a0a30 0x7f6ea811f910 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800e900 tx=0x7f6e9800ec10 comp rx=0 tx=0).stop 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 shutdown_connections 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ea8133930 0x7f6ea81c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ea809d630 0x7f6ea811f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 --2- 172.21.15.32:0/353077554 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ea80a0a30 0x7f6ea811f910 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 >> 172.21.15.32:0/353077554 conn(0x7f6ea80a3050 msgr2=0x7f6ea809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 shutdown_connections 2023-09-30T22:30:35.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.699+0000 7f6eafaa1700 1 -- 172.21.15.32:0/353077554 wait complete. 2023-09-30T22:30:35.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.726+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff2c0133340 con 0x7ff2c009d630 2023-09-30T22:30:35.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.726+0000 7ff2bd7fa700 1 -- 172.21.15.32:0/1961435237 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2b4017020 con 0x7ff2c009d630 2023-09-30T22:30:35.726 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.728+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 msgr2=0x7ff2c01c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.728+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c01c7ef0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400ebe0 tx=0x7ff2b400eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 shutdown_connections 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff2c0133930 0x7ff2c01c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff2c009d630 0x7ff2c01c7ef0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 --2- 172.21.15.32:0/1961435237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00a0a30 0x7ff2c01c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 >> 172.21.15.32:0/1961435237 conn(0x7ff2c00a3050 msgr2=0x7ff2c01bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 shutdown_connections 2023-09-30T22:30:35.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:35.729+0000 7ff2c6d73700 1 -- 172.21.15.32:0/1961435237 wait complete. 2023-09-30T22:30:35.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:36.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:36.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.248+0000 7fa23fda1700 1 Processor -- start 2023-09-30T22:30:36.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23fda1700 1 -- start start 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23fda1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa238135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23fda1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 0x7fa238138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23fda1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23809f130 con 0x7fa23809d6f0 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23fda1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23809f520 con 0x7fa238135e50 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23fda1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23809e930 con 0x7fa23809e1d0 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23e33e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 0x7fa238138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23d33c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa238135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23e33e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 0x7fa238138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59836/0 (socket says 172.21.15.32:59836) 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23d33c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa238135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44870/0 (socket says 172.21.15.32:44870) 2023-09-30T22:30:36.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23db3d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa23809d6f0 0x7fa23809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34408/0 (socket says 172.21.15.32:34408) 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.249+0000 7fa23e33e700 1 -- 172.21.15.32:0/1658415877 learned_addr learned my addr 172.21.15.32:0/1658415877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23d33c700 1 -- 172.21.15.32:0/1658415877 >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 msgr2=0x7fa238138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23d33c700 1 --2- 172.21.15.32:0/1658415877 >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 0x7fa238138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23d33c700 1 -- 172.21.15.32:0/1658415877 >> v1:172.21.15.32:6789/0 conn(0x7fa23809d6f0 legacy=0x7fa23809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23d33c700 1 -- 172.21.15.32:0/1658415877 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23809b2d0 con 0x7fa23809e1d0 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23cb3b700 1 -- 172.21.15.32:0/1658415877 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712765941 0 0) 0x7fa23809f130 con 0x7fa23809d6f0 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23e33e700 1 --2- 172.21.15.32:0/1658415877 >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 0x7fa238138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23d33c700 1 --2- 172.21.15.32:0/1658415877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa238135910 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa228009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3154ed62b8310be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23cb3b700 1 -- 172.21.15.32:0/1658415877 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228011830 con 0x7fa23809e1d0 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23cb3b700 1 -- 172.21.15.32:0/1658415877 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228011990 con 0x7fa23809e1d0 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.250+0000 7fa23cb3b700 1 -- 172.21.15.32:0/1658415877 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228011d20 con 0x7fa23809e1d0 2023-09-30T22:30:36.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 -- 172.21.15.32:0/1658415877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 msgr2=0x7fa238135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 --2- 172.21.15.32:0/1658415877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa238135910 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa228009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 -- 172.21.15.32:0/1658415877 shutdown_connections 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 --2- 172.21.15.32:0/1658415877 >> v2:172.21.15.32:3300/0 conn(0x7fa238135e50 0x7fa238138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 --2- 172.21.15.32:0/1658415877 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa238135910 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 -- 172.21.15.32:0/1658415877 >> 172.21.15.32:0/1658415877 conn(0x7fa238066f50 msgr2=0x7fa23809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 -- 172.21.15.32:0/1658415877 shutdown_connections 2023-09-30T22:30:36.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.251+0000 7fa23fda1700 1 -- 172.21.15.32:0/1658415877 wait complete. 2023-09-30T22:30:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 Processor -- start 2023-09-30T22:30:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 -- start start 2023-09-30T22:30:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa23809d6f0 0x7fa2381c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa2381c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa238135e50 0x7fa238119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381301f0 con 0x7fa238135e50 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23811b180 con 0x7fa23809e1d0 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23fda1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa23809a0c0 con 0x7fa23809d6f0 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23d33c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa2381c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23d33c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa2381c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44880/0 (socket says 172.21.15.32:44880) 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23d33c700 1 -- 172.21.15.32:0/3259644768 learned_addr learned my addr 172.21.15.32:0/3259644768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.252+0000 7fa23e33e700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa238135e50 0x7fa238119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23db3d700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa23809d6f0 0x7fa2381c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23d33c700 1 -- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa23809d6f0 msgr2=0x7fa2381c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23d33c700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa23809d6f0 0x7fa2381c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23d33c700 1 -- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa238135e50 msgr2=0x7fa238119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23d33c700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa238135e50 0x7fa238119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23d33c700 1 -- 172.21.15.32:0/3259644768 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23000bd40 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23e33e700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa238135e50 0x7fa238119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23d33c700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa2381c4830 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa228000c00 tx=0x7fa228004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22801a8b0 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22801aa10 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2280094a0 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22801ada0 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.253+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23809c700 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.254+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2280243f0 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.254+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa228040c10 con 0x7fa23809e1d0 2023-09-30T22:30:36.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.254+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa238074ea0 con 0x7fa23809e1d0 2023-09-30T22:30:36.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.259+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa228015020 con 0x7fa23809e1d0 2023-09-30T22:30:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.463+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa2380a20d0 con 0x7fa23809e1d0 2023-09-30T22:30:36.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.464+0000 7fa22e7fc700 1 -- 172.21.15.32:0/3259644768 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa22800b360 con 0x7fa23809e1d0 2023-09-30T22:30:36.465 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 msgr2=0x7fa2381c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa2381c4830 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa228000c00 tx=0x7fa228004750 comp rx=0 tx=0).stop 2023-09-30T22:30:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 shutdown_connections 2023-09-30T22:30:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa238135e50 0x7fa238119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa23809d6f0 0x7fa2381c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 --2- 172.21.15.32:0/3259644768 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa23809e1d0 0x7fa2381c4830 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 >> 172.21.15.32:0/3259644768 conn(0x7fa238066f50 msgr2=0x7fa2381329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 shutdown_connections 2023-09-30T22:30:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.466+0000 7fa23fda1700 1 -- 172.21.15.32:0/3259644768 wait complete. 2023-09-30T22:30:36.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:36.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:36.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:36.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:36.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.772+0000 7f69876dd700 1 Processor -- start 2023-09-30T22:30:36.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f69876dd700 1 -- start start 2023-09-30T22:30:36.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f69876dd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f698009f410 0x7f6980136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f69876dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6980136fe0 0x7f69801393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f69876dd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6980070940 con 0x7f698009e930 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f69876dd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698006fd60 con 0x7f698009f410 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f69876dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f698006fa80 con 0x7f6980136fe0 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f6985479700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f698009e930 0x7f698009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34414/0 (socket says 172.21.15.32:34414) 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f6985479700 1 -- 172.21.15.32:0/3695609682 learned_addr learned my addr 172.21.15.32:0/3695609682 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.773+0000 7f6985c7a700 1 --2- 172.21.15.32:0/3695609682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6980136fe0 0x7f69801393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6984c78700 1 --2- 172.21.15.32:0/3695609682 >> v2:172.21.15.32:3300/0 conn(0x7f698009f410 0x7f6980136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6985c7a700 1 -- 172.21.15.32:0/3695609682 >> v2:172.21.15.32:3300/0 conn(0x7f698009f410 msgr2=0x7f6980136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6985c7a700 1 --2- 172.21.15.32:0/3695609682 >> v2:172.21.15.32:3300/0 conn(0x7f698009f410 0x7f6980136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6985c7a700 1 -- 172.21.15.32:0/3695609682 >> v1:172.21.15.32:6789/0 conn(0x7f698009e930 legacy=0x7f698009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6985c7a700 1 -- 172.21.15.32:0/3695609682 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698009fb60 con 0x7f6980136fe0 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6984c78700 1 --2- 172.21.15.32:0/3695609682 >> v2:172.21.15.32:3300/0 conn(0x7f698009f410 0x7f6980136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6985c7a700 1 --2- 172.21.15.32:0/3695609682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6980136fe0 0x7f69801393c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6978009d90 tx=0x7f697800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a70d2b3209261e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6977fff700 1 -- 172.21.15.32:0/3695609682 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800e070 con 0x7f6980136fe0 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6977fff700 1 -- 172.21.15.32:0/3695609682 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697800bb20 con 0x7f6980136fe0 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.774+0000 7f6977fff700 1 -- 172.21.15.32:0/3695609682 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978012430 con 0x7f6980136fe0 2023-09-30T22:30:36.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 -- 172.21.15.32:0/3695609682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6980136fe0 msgr2=0x7f69801393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 --2- 172.21.15.32:0/3695609682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6980136fe0 0x7f69801393c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6978009d90 tx=0x7f697800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 -- 172.21.15.32:0/3695609682 shutdown_connections 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 --2- 172.21.15.32:0/3695609682 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6980136fe0 0x7f69801393c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 --2- 172.21.15.32:0/3695609682 >> v2:172.21.15.32:3300/0 conn(0x7f698009f410 0x7f6980136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 -- 172.21.15.32:0/3695609682 >> 172.21.15.32:0/3695609682 conn(0x7f69800a4210 msgr2=0x7f69800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 -- 172.21.15.32:0/3695609682 shutdown_connections 2023-09-30T22:30:36.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.775+0000 7f69876dd700 1 -- 172.21.15.32:0/3695609682 wait complete. 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.776+0000 7f69876dd700 1 Processor -- start 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.776+0000 7f69876dd700 1 -- start start 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f69876dd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f698009e930 0x7f698011c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f69876dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f698009f410 0x7f698011ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f69876dd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6980136fe0 0x7f698011cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f69876dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980071db0 con 0x7f698009f410 2023-09-30T22:30:36.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f69876dd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69800a33f0 con 0x7f6980136fe0 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f69876dd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f698011f820 con 0x7f698009e930 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f6985479700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f698009e930 0x7f698011c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f6985479700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f698009e930 0x7f698011c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53364/0 (socket says 172.21.15.32:53364) 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f6985479700 1 -- 172.21.15.32:0/923099531 learned_addr learned my addr 172.21.15.32:0/923099531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f6985c7a700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6980136fe0 0x7f698011cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.777+0000 7f6984c78700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f698009f410 0x7f698011ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6985c7a700 1 -- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f698009e930 msgr2=0x7f698011c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6985c7a700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f698009e930 0x7f698011c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6985c7a700 1 -- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f698009f410 msgr2=0x7f698011ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6985c7a700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f698009f410 0x7f698011ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6985c7a700 1 -- 172.21.15.32:0/923099531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700094a0 con 0x7f6980136fe0 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6984c78700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f698009f410 0x7f698011ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6985c7a700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6980136fe0 0x7f698011cfd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f698006f570 tx=0x7f6978004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978022070 con 0x7f6980136fe0 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697801dc60 con 0x7f6980136fe0 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6978009a70 con 0x7f6980136fe0 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978012430 con 0x7f6980136fe0 2023-09-30T22:30:36.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.778+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698011e9c0 con 0x7f6980136fe0 2023-09-30T22:30:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.779+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6980074ea0 con 0x7f6980136fe0 2023-09-30T22:30:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.779+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69780125e0 con 0x7f6980136fe0 2023-09-30T22:30:36.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.780+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6978044ac0 con 0x7f6980136fe0 2023-09-30T22:30:36.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.789+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6978025df0 con 0x7f6980136fe0 2023-09-30T22:30:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.861+0000 7f08c03d2700 1 Processor -- start 2023-09-30T22:30:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.861+0000 7f08c03d2700 1 -- start start 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.861+0000 7f08c03d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08c03d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08b809e410 0x7f08b8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08c03d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b809b120 con 0x7f08b809e000 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08c03d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b806b460 con 0x7f08b809e410 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08c03d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b809a7e0 con 0x7f08b809d6f0 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44886/0 (socket says 172.21.15.32:44886) 2023-09-30T22:30:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 -- 172.21.15.32:0/550391285 learned_addr learned my addr 172.21.15.32:0/550391285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be96f700 1 --2- 172.21.15.32:0/550391285 >> v2:172.21.15.32:3300/0 conn(0x7f08b809e410 0x7f08b8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 -- 172.21.15.32:0/550391285 >> v2:172.21.15.32:3300/0 conn(0x7f08b809e410 msgr2=0x7f08b8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 --2- 172.21.15.32:0/550391285 >> v2:172.21.15.32:3300/0 conn(0x7f08b809e410 0x7f08b8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 -- 172.21.15.32:0/550391285 >> v1:172.21.15.32:6789/0 conn(0x7f08b809e000 legacy=0x7f08b81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 -- 172.21.15.32:0/550391285 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b809a500 con 0x7f08b809d6f0 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.862+0000 7f08be16e700 1 --2- 172.21.15.32:0/550391285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b809dac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009920 tx=0x7f08b4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c0c73451a96516d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.863+0000 7f08bd16c700 1 -- 172.21.15.32:0/550391285 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4011830 con 0x7f08b809d6f0 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.863+0000 7f08bd16c700 1 -- 172.21.15.32:0/550391285 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4011990 con 0x7f08b809d6f0 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.863+0000 7f08bd16c700 1 -- 172.21.15.32:0/550391285 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4011d20 con 0x7f08b809d6f0 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.863+0000 7f08c03d2700 1 -- 172.21.15.32:0/550391285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 msgr2=0x7f08b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.863+0000 7f08c03d2700 1 --2- 172.21.15.32:0/550391285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b809dac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009920 tx=0x7f08b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.864+0000 7f08c03d2700 1 -- 172.21.15.32:0/550391285 shutdown_connections 2023-09-30T22:30:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.864+0000 7f08c03d2700 1 --2- 172.21.15.32:0/550391285 >> v2:172.21.15.32:3300/0 conn(0x7f08b809e410 0x7f08b8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.864+0000 7f08c03d2700 1 --2- 172.21.15.32:0/550391285 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b809dac0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.864+0000 7f08c03d2700 1 -- 172.21.15.32:0/550391285 >> 172.21.15.32:0/550391285 conn(0x7f08b8066f50 msgr2=0x7f08b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.864+0000 7f08c03d2700 1 -- 172.21.15.32:0/550391285 shutdown_connections 2023-09-30T22:30:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.864+0000 7f08c03d2700 1 -- 172.21.15.32:0/550391285 wait complete. 2023-09-30T22:30:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 Processor -- start 2023-09-30T22:30:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 -- start start 2023-09-30T22:30:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b81a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08b809e000 0x7f08b81a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b809e410 0x7f08b81a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b81ab880 con 0x7f08b809e410 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b809c710 con 0x7f08b809d6f0 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08c03d2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08b81a9ba0 con 0x7f08b809e000 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08be16e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b81a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08be16e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b81a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44892/0 (socket says 172.21.15.32:44892) 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08be16e700 1 -- 172.21.15.32:0/3655299405 learned_addr learned my addr 172.21.15.32:0/3655299405 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08bd96d700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08b809e000 0x7f08b81a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.865+0000 7f08be96f700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b809e410 0x7f08b81a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08be16e700 1 -- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08b809e000 msgr2=0x7f08b81a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08be16e700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08b809e000 0x7f08b81a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08be16e700 1 -- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b809e410 msgr2=0x7f08b81a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08be16e700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b809e410 0x7f08b81a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08be16e700 1 -- 172.21.15.32:0/3655299405 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009550 con 0x7f08b809d6f0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08be16e700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b81a8070 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009c70 tx=0x7f08b4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4004240 con 0x7f08b809d6f0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b401f070 con 0x7f08b809d6f0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40036a0 con 0x7f08b809d6f0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b81abae0 con 0x7f08b809d6f0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.866+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b81aa290 con 0x7f08b809d6f0 2023-09-30T22:30:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.867+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08b4021030 con 0x7f08b809d6f0 2023-09-30T22:30:36.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.867+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f08b4040980 con 0x7f08b809d6f0 2023-09-30T22:30:36.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.867+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8074ea0 con 0x7f08b809d6f0 2023-09-30T22:30:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.872+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08b4015070 con 0x7f08b809d6f0 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.899+0000 7f4f9e758700 1 Processor -- start 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.899+0000 7f4f9e758700 1 -- start start 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9e758700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809e930 0x7f4f9809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9e758700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809f240 0x7f4f981317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9e758700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f98070940 con 0x7f4f9809f650 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9e758700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9806fd60 con 0x7f4f9809e930 2023-09-30T22:30:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9e758700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9806fa80 con 0x7f4f9809f240 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f97fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809e930 0x7f4f9809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9ccf5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f9809f650 0x7f4f98131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34444/0 (socket says 172.21.15.32:34444) 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f977fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809f240 0x7f4f981317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f9ccf5700 1 -- 172.21.15.32:0/844808528 learned_addr learned my addr 172.21.15.32:0/844808528 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f977fe700 1 -- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809e930 msgr2=0x7f4f9809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f977fe700 1 --2- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809e930 0x7f4f9809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f977fe700 1 -- 172.21.15.32:0/844808528 >> v1:172.21.15.32:6789/0 conn(0x7f4f9809f650 legacy=0x7f4f98131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f977fe700 1 -- 172.21.15.32:0/844808528 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9809fb60 con 0x7f4f9809f240 2023-09-30T22:30:36.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.900+0000 7f4f97fff700 1 --2- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809e930 0x7f4f9809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.901+0000 7f4f977fe700 1 --2- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809f240 0x7f4f981317f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009870 tx=0x7f4f8c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26d4f53433410f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.901+0000 7f4f96ffd700 1 -- 172.21.15.32:0/844808528 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011830 con 0x7f4f9809f240 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.901+0000 7f4f96ffd700 1 -- 172.21.15.32:0/844808528 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c011990 con 0x7f4f9809f240 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.901+0000 7f4f96ffd700 1 -- 172.21.15.32:0/844808528 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011d20 con 0x7f4f9809f240 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.901+0000 7f4f9e758700 1 -- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809f240 msgr2=0x7f4f981317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.901+0000 7f4f9e758700 1 --2- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809f240 0x7f4f981317f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009870 tx=0x7f4f8c009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.902+0000 7f4f9e758700 1 -- 172.21.15.32:0/844808528 shutdown_connections 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.902+0000 7f4f9e758700 1 --2- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809e930 0x7f4f9809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.902+0000 7f4f9e758700 1 --2- 172.21.15.32:0/844808528 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809f240 0x7f4f981317f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.902+0000 7f4f9e758700 1 -- 172.21.15.32:0/844808528 >> 172.21.15.32:0/844808528 conn(0x7f4f980a4210 msgr2=0x7f4f980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.902+0000 7f4f9e758700 1 -- 172.21.15.32:0/844808528 shutdown_connections 2023-09-30T22:30:36.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.902+0000 7f4f9e758700 1 -- 172.21.15.32:0/844808528 wait complete. 2023-09-30T22:30:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 Processor -- start 2023-09-30T22:30:36.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 -- start start 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809e930 0x7f4f981a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 0x7f4f981a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809f650 0x7f4f981a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f981a53a0 con 0x7f4f9809f240 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f981b72c0 con 0x7f4f9809f650 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f9e758700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f981a6d30 con 0x7f4f9809e930 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 0x7f4f981a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.903+0000 7f4f977fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 0x7f4f981a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59886/0 (socket says 172.21.15.32:59886) 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f977fe700 1 -- 172.21.15.32:0/2608172890 learned_addr learned my addr 172.21.15.32:0/2608172890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:36.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809e930 0x7f4f981a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f9ccf5700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809f650 0x7f4f981a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 -- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809f650 msgr2=0x7f4f981a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809f650 0x7f4f981a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 -- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 msgr2=0x7f4f981a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 0x7f4f981a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 -- 172.21.15.32:0/2608172890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880094a0 con 0x7f4f9809e930 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f977fe700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 0x7f4f981a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f9ccf5700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809f650 0x7f4f981a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f97fff700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809e930 0x7f4f981a3ed0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e9b0 tx=0x7f4f8800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800c850 con 0x7f4f9809e930 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8800c9b0 con 0x7f4f9809e930 2023-09-30T22:30:36.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c0094a0 con 0x7f4f9809e930 2023-09-30T22:30:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800cd60 con 0x7f4f9809e930 2023-09-30T22:30:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.904+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f980a0190 con 0x7f4f9809e930 2023-09-30T22:30:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.906+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f980a3770 con 0x7f4f9809e930 2023-09-30T22:30:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.906+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f88013470 con 0x7f4f9809e930 2023-09-30T22:30:36.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.906+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4f8803f780 con 0x7f4f9809e930 2023-09-30T22:30:36.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:36.911+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f880232f0 con 0x7f4f9809e930 2023-09-30T22:30:37.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.002+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f698006b250 con 0x7f6980136fe0 2023-09-30T22:30:37.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.003+0000 7f6975ffb700 1 -- 172.21.15.32:0/923099531 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6978025390 con 0x7f6980136fe0 2023-09-30T22:30:37.004 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6980136fe0 msgr2=0x7f698011cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6980136fe0 0x7f698011cfd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f698006f570 tx=0x7f6978004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 shutdown_connections 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6980136fe0 0x7f698011cfd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f698009e930 0x7f698011c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 --2- 172.21.15.32:0/923099531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f698009f410 0x7f698011ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.005+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 >> 172.21.15.32:0/923099531 conn(0x7f69800a4210 msgr2=0x7f6980135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.006+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 shutdown_connections 2023-09-30T22:30:37.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.006+0000 7f69876dd700 1 -- 172.21.15.32:0/923099531 wait complete. 2023-09-30T22:30:37.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.092+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08b806c360 con 0x7f08b809d6f0 2023-09-30T22:30:37.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.092+0000 7f08aeffd700 1 -- 172.21.15.32:0/3655299405 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08b40199a0 con 0x7f08b809d6f0 2023-09-30T22:30:37.094 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 msgr2=0x7f08b81a8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b81a8070 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009c70 tx=0x7f08b4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 shutdown_connections 2023-09-30T22:30:37.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b809e410 0x7f08b81a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b809d6f0 0x7f08b81a8070 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 --2- 172.21.15.32:0/3655299405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08b809e000 0x7f08b81a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 >> 172.21.15.32:0/3655299405 conn(0x7f08b8066f50 msgr2=0x7f08b806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 shutdown_connections 2023-09-30T22:30:37.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.094+0000 7f08c03d2700 1 -- 172.21.15.32:0/3655299405 wait complete. 2023-09-30T22:30:37.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:37.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.126+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4f98071d20 con 0x7f4f9809e930 2023-09-30T22:30:37.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.127+0000 7f4f94ff9700 1 -- 172.21.15.32:0/2608172890 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f88021020 con 0x7f4f9809e930 2023-09-30T22:30:37.128 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:37.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809e930 msgr2=0x7f4f981a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809e930 0x7f4f981a3ed0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e9b0 tx=0x7f4f8800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 shutdown_connections 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4f9809f650 0x7f4f981a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4f9809e930 0x7f4f981a3ed0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 --2- 172.21.15.32:0/2608172890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f9809f240 0x7f4f981a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 >> 172.21.15.32:0/2608172890 conn(0x7f4f980a4210 msgr2=0x7f4f981b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 shutdown_connections 2023-09-30T22:30:37.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.129+0000 7f4f9e758700 1 -- 172.21.15.32:0/2608172890 wait complete. 2023-09-30T22:30:37.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:37.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.627+0000 7f348fc1e700 1 Processor -- start 2023-09-30T22:30:37.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.627+0000 7f348fc1e700 1 -- start start 2023-09-30T22:30:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348fc1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 0x7f3488136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348fc1e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 0x7f34881393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348fc1e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3488070940 con 0x7f348809e930 2023-09-30T22:30:37.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348fc1e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348806fd60 con 0x7f3488136fe0 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348fc1e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348806fa80 con 0x7f348809f410 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 0x7f3488136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348e1bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 0x7f34881393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d9ba700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f348809e930 0x7f348809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34450/0 (socket says 172.21.15.32:34450) 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 0x7f3488136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44924/0 (socket says 172.21.15.32:44924) 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348e1bb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 0x7f34881393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59898/0 (socket says 172.21.15.32:59898) 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d9ba700 1 -- 172.21.15.32:0/715607808 learned_addr learned my addr 172.21.15.32:0/715607808 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 -- 172.21.15.32:0/715607808 >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 msgr2=0x7f34881393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 --2- 172.21.15.32:0/715607808 >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 0x7f34881393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 -- 172.21.15.32:0/715607808 >> v1:172.21.15.32:6789/0 conn(0x7f348809e930 legacy=0x7f348809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 -- 172.21.15.32:0/715607808 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348809fb60 con 0x7f348809f410 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348e1bb700 1 --2- 172.21.15.32:0/715607808 >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 0x7f34881393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348d1b9700 1 --2- 172.21.15.32:0/715607808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 0x7f3488136aa0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3478009840 tx=0x7f3478009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9fad4497d25b54b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.628+0000 7f348c9b8700 1 -- 172.21.15.32:0/715607808 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478011830 con 0x7f348809f410 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348c9b8700 1 -- 172.21.15.32:0/715607808 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478011990 con 0x7f348809f410 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348c9b8700 1 -- 172.21.15.32:0/715607808 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478011d20 con 0x7f348809f410 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 -- 172.21.15.32:0/715607808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 msgr2=0x7f3488136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 --2- 172.21.15.32:0/715607808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 0x7f3488136aa0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3478009840 tx=0x7f3478009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 -- 172.21.15.32:0/715607808 shutdown_connections 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 --2- 172.21.15.32:0/715607808 >> v2:172.21.15.32:3300/0 conn(0x7f3488136fe0 0x7f34881393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 --2- 172.21.15.32:0/715607808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f348809f410 0x7f3488136aa0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 -- 172.21.15.32:0/715607808 >> 172.21.15.32:0/715607808 conn(0x7f34880a4210 msgr2=0x7f34880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 -- 172.21.15.32:0/715607808 shutdown_connections 2023-09-30T22:30:37.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.629+0000 7f348fc1e700 1 -- 172.21.15.32:0/715607808 wait complete. 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 Processor -- start 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 -- start start 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 0x7f348811c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f348809f410 0x7f348811cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3488136fe0 0x7f348811d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348811fe50 con 0x7f348809e930 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880a2f40 con 0x7f3488136fe0 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348fc1e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f348811f8a0 con 0x7f348809f410 2023-09-30T22:30:37.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.631+0000 7f348d9ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 0x7f348811c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 0x7f348811c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59912/0 (socket says 172.21.15.32:59912) 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 -- 172.21.15.32:0/2281283724 learned_addr learned my addr 172.21.15.32:0/2281283724 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348e1bb700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3488136fe0 0x7f348811d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d1b9700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f348809f410 0x7f348811cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 -- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f348809f410 msgr2=0x7f348811cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f348809f410 0x7f348811cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 -- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3488136fe0 msgr2=0x7f348811d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3488136fe0 0x7f348811d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 -- 172.21.15.32:0/2281283724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480009a70 con 0x7f348809e930 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348e1bb700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3488136fe0 0x7f348811d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:37.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348d9ba700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 0x7f348811c660 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f348400c520 tx=0x7f348400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484005d70 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34840070d0 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34780094a0 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484007440 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.632+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348811e330 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.633+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34840075a0 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.633+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f348403ea00 con 0x7f348809e930 2023-09-30T22:30:37.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.633+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3488074ea0 con 0x7f348809e930 2023-09-30T22:30:37.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.638+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3484019b40 con 0x7f348809e930 2023-09-30T22:30:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.844+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3488132500 con 0x7f348809e930 2023-09-30T22:30:37.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.844+0000 7f347e7fc700 1 -- 172.21.15.32:0/2281283724 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3484017030 con 0x7f348809e930 2023-09-30T22:30:37.845 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:37.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 msgr2=0x7f348811c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 0x7f348811c660 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f348400c520 tx=0x7f348400c830 comp rx=0 tx=0).stop 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 shutdown_connections 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3488136fe0 0x7f348811d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f348809e930 0x7f348811c660 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 --2- 172.21.15.32:0/2281283724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f348809f410 0x7f348811cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 >> 172.21.15.32:0/2281283724 conn(0x7f34880a4210 msgr2=0x7f34880a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 shutdown_connections 2023-09-30T22:30:37.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:37.846+0000 7f348fc1e700 1 -- 172.21.15.32:0/2281283724 wait complete. 2023-09-30T22:30:37.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:38.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:38.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:38.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 Processor -- start 2023-09-30T22:30:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 -- start start 2023-09-30T22:30:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77cc09e1d0 0x7f77cc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc09b120 con 0x7f77cc09d6f0 2023-09-30T22:30:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc06b460 con 0x7f77cc09e1d0 2023-09-30T22:30:38.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.168+0000 7f77d455f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc09a7e0 con 0x7f77cc139170 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d1afa700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f77cc09e1d0 0x7f77cc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d22fb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f77cc09d6f0 0x7f77cc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34454/0 (socket says 172.21.15.32:34454) 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d22fb700 1 -- 172.21.15.32:0/3315290065 learned_addr learned my addr 172.21.15.32:0/3315290065 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d2afc700 1 --2- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d1afa700 1 -- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 msgr2=0x7f77cc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d1afa700 1 --2- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d1afa700 1 -- 172.21.15.32:0/3315290065 >> v1:172.21.15.32:6789/0 conn(0x7f77cc09d6f0 legacy=0x7f77cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d1afa700 1 -- 172.21.15.32:0/3315290065 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc09a500 con 0x7f77cc09e1d0 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d1afa700 1 --2- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc138c30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009870 tx=0x7f77bc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f867f0f7fe5e60a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d12f9700 1 -- 172.21.15.32:0/3315290065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc011830 con 0x7f77cc09e1d0 2023-09-30T22:30:38.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d12f9700 1 -- 172.21.15.32:0/3315290065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc011990 con 0x7f77cc09e1d0 2023-09-30T22:30:38.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.169+0000 7f77d12f9700 1 -- 172.21.15.32:0/3315290065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc011d20 con 0x7f77cc09e1d0 2023-09-30T22:30:38.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 -- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 msgr2=0x7f77cc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 --2- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc138c30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009870 tx=0x7f77bc009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 -- 172.21.15.32:0/3315290065 shutdown_connections 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 --2- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 --2- 172.21.15.32:0/3315290065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc138c30 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 -- 172.21.15.32:0/3315290065 >> 172.21.15.32:0/3315290065 conn(0x7f77cc066f50 msgr2=0x7f77cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 -- 172.21.15.32:0/3315290065 shutdown_connections 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.170+0000 7f77d455f700 1 -- 172.21.15.32:0/3315290065 wait complete. 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.172+0000 7f77d455f700 1 Processor -- start 2023-09-30T22:30:38.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.172+0000 7f77d455f700 1 -- start start 2023-09-30T22:30:38.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d455f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77cc09d6f0 0x7f77cc11f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d455f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc11f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d455f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc1c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d455f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc11ef50 con 0x7f77cc09e1d0 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d455f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0a3080 con 0x7f77cc139170 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d455f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1221e0 con 0x7f77cc09d6f0 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d22fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77cc09d6f0 0x7f77cc11f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d22fb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77cc09d6f0 0x7f77cc11f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53406/0 (socket says 172.21.15.32:53406) 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc11f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d22fb700 1 -- 172.21.15.32:0/334670608 learned_addr learned my addr 172.21.15.32:0/334670608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d2afc700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc1c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 -- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77cc09d6f0 msgr2=0x7f77cc11f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77cc09d6f0 0x7f77cc11f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 -- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 msgr2=0x7f77cc1c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc1c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 -- 172.21.15.32:0/334670608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc0094a0 con 0x7f77cc09e1d0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.173+0000 7f77d1afa700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc11f800 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f77bc016000 tx=0x7f77bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.174+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc004590 con 0x7f77cc09e1d0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.174+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc01f070 con 0x7f77cc09e1d0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.174+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0038e0 con 0x7f77cc09e1d0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.174+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc120da0 con 0x7f77cc09e1d0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.174+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc122c50 con 0x7f77cc09e1d0 2023-09-30T22:30:38.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.175+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f77bc003a40 con 0x7f77cc09e1d0 2023-09-30T22:30:38.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.175+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f77bc021070 con 0x7f77cc09e1d0 2023-09-30T22:30:38.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.175+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc122320 con 0x7f77cc09e1d0 2023-09-30T22:30:38.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.180+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f77cc122320 con 0x7f77cc09e1d0 2023-09-30T22:30:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 Processor -- start 2023-09-30T22:30:38.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 -- start start 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f145c133a80 0x7f145c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145c06bc00 con 0x7f145c130870 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145c06c570 con 0x7f145c133a80 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.256+0000 7f1460fb9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0a0040 con 0x7f145c12ff20 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f1459d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f1459d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44958/0 (socket says 172.21.15.32:44958) 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f1459d9b700 1 -- 172.21.15.32:0/2627863828 learned_addr learned my addr 172.21.15.32:0/2627863828 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f145c133a80 0x7f145c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145a59c700 1 -- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 msgr2=0x7f145c130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145a59c700 1 --2- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145a59c700 1 -- 172.21.15.32:0/2627863828 >> v1:172.21.15.32:6789/0 conn(0x7f145c130870 legacy=0x7f145c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145a59c700 1 -- 172.21.15.32:0/2627863828 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c133860 con 0x7f145c133a80 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145959a700 1 -- 172.21.15.32:0/2627863828 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162717110 0 0) 0x7f145c06bc00 con 0x7f145c130870 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f1459d9b700 1 --2- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.257+0000 7f145a59c700 1 --2- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c133a80 0x7f145c133e50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f144c009840 tx=0x7f144c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7955602ec2e79f4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.258+0000 7f145959a700 1 -- 172.21.15.32:0/2627863828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c011830 con 0x7f145c133a80 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.258+0000 7f145959a700 1 -- 172.21.15.32:0/2627863828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c011990 con 0x7f145c133a80 2023-09-30T22:30:38.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.258+0000 7f145959a700 1 -- 172.21.15.32:0/2627863828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c011d20 con 0x7f145c133a80 2023-09-30T22:30:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 -- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c133a80 msgr2=0x7f145c133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 --2- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c133a80 0x7f145c133e50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f144c009840 tx=0x7f144c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:38.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 -- 172.21.15.32:0/2627863828 shutdown_connections 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 --2- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c133a80 0x7f145c133e50 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 --2- 172.21.15.32:0/2627863828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 -- 172.21.15.32:0/2627863828 >> 172.21.15.32:0/2627863828 conn(0x7f145c0a4210 msgr2=0x7f145c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 -- 172.21.15.32:0/2627863828 shutdown_connections 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.259+0000 7f1460fb9700 1 -- 172.21.15.32:0/2627863828 wait complete. 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.260+0000 7f1460fb9700 1 Processor -- start 2023-09-30T22:30:38.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.260+0000 7f1460fb9700 1 -- start start 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1460fb9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c09c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1460fb9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 0x7f145c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1460fb9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f145c133a80 0x7f145c09d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1460fb9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c09e240 con 0x7f145c130870 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1460fb9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c120280 con 0x7f145c12ff20 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1460fb9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f145c120980 con 0x7f145c133a80 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1459d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 0x7f145c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1459d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 0x7f145c09ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59946/0 (socket says 172.21.15.32:59946) 2023-09-30T22:30:38.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f1459d9b700 1 -- 172.21.15.32:0/4143360837 learned_addr learned my addr 172.21.15.32:0/4143360837 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f145ad9d700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f145c133a80 0x7f145c09d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.261+0000 7f145a59c700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c09c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145ad9d700 1 -- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 msgr2=0x7f145c09c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145ad9d700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c09c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145ad9d700 1 -- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 msgr2=0x7f145c09ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145ad9d700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 0x7f145c09ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145ad9d700 1 -- 172.21.15.32:0/4143360837 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1450009550 con 0x7f145c133a80 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f1459d9b700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 0x7f145c09ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145a59c700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f145ad9d700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f145c133a80 0x7f145c09d340 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f145400c7f0 tx=0x7f145400cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454003e30 con 0x7f145c133a80 2023-09-30T22:30:38.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454005070 con 0x7f145c133a80 2023-09-30T22:30:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c0094a0 con 0x7f145c133a80 2023-09-30T22:30:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454005420 con 0x7f145c133a80 2023-09-30T22:30:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.262+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c120480 con 0x7f145c133a80 2023-09-30T22:30:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.263+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1454005580 con 0x7f145c133a80 2023-09-30T22:30:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.263+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f145403ec60 con 0x7f145c133a80 2023-09-30T22:30:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.263+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c0a2610 con 0x7f145c133a80 2023-09-30T22:30:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.268+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1454017070 con 0x7f145c133a80 2023-09-30T22:30:38.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.299+0000 7f9155e77700 1 Processor -- start 2023-09-30T22:30:38.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.299+0000 7f9155e77700 1 -- start start 2023-09-30T22:30:38.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f9155e77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f9155e77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f915009e000 0x7f9150130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f9155e77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f9150130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f9155e77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915009f520 con 0x7f915009e000 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f9155e77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915009f130 con 0x7f915009e410 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f9155e77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f915009e930 con 0x7f915009d6f0 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f914effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f915009e000 0x7f9150130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f914f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.300+0000 7f914ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f9150130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f915009e000 0x7f9150130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59950/0 (socket says 172.21.15.32:59950) 2023-09-30T22:30:38.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 -- 172.21.15.32:0/4030160781 learned_addr learned my addr 172.21.15.32:0/4030160781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:38.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 -- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 msgr2=0x7f915009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 -- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 msgr2=0x7f9150130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f9150130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 -- 172.21.15.32:0/4030160781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915009b2d0 con 0x7f915009e000 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914ffff700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f9150130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914effd700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f9150130660 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9144009920 tx=0x7f9144009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a07d85149f8017a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914e7fc700 1 -- 172.21.15.32:0/4030160781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144011830 con 0x7f915009e000 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914e7fc700 1 -- 172.21.15.32:0/4030160781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9144011990 con 0x7f915009e000 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.301+0000 7f914e7fc700 1 -- 172.21.15.32:0/4030160781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144011d20 con 0x7f915009e000 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 -- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 msgr2=0x7f9150130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f9150130660 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9144009920 tx=0x7f9144009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 -- 172.21.15.32:0/4030160781 shutdown_connections 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f9150130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 --2- 172.21.15.32:0/4030160781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f9150130660 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 -- 172.21.15.32:0/4030160781 >> 172.21.15.32:0/4030160781 conn(0x7f9150066f50 msgr2=0x7f915009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 -- 172.21.15.32:0/4030160781 shutdown_connections 2023-09-30T22:30:38.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.302+0000 7f9155e77700 1 -- 172.21.15.32:0/4030160781 wait complete. 2023-09-30T22:30:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.303+0000 7f9155e77700 1 Processor -- start 2023-09-30T22:30:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.303+0000 7f9155e77700 1 -- start start 2023-09-30T22:30:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f9155e77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915011e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f9155e77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f915011e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f9155e77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f915011ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f9155e77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150121dc0 con 0x7f915009e000 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f9155e77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9150120350 con 0x7f915009e410 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f9155e77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91500a1840 con 0x7f915009d6f0 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f915011e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915011e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f915011e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59962/0 (socket says 172.21.15.32:59962) 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 -- 172.21.15.32:0/720709677 learned_addr learned my addr 172.21.15.32:0/720709677 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914ffff700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f915011ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 -- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 msgr2=0x7f915011e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915011e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 -- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 msgr2=0x7f915011ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f915011ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914effd700 1 -- 172.21.15.32:0/720709677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148009a70 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.304+0000 7f914f7fe700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915011e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.305+0000 7f914effd700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f915011e6d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9144000c00 tx=0x7f9144004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.305+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144004240 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.305+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914401f070 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.305+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9144009550 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.305+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91440036a0 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.305+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9150121440 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.306+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9144021030 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.306+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9150121630 con 0x7f915009e000 2023-09-30T22:30:38.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.306+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9144040600 con 0x7f915009e000 2023-09-30T22:30:38.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.315+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91440239c0 con 0x7f915009e000 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.393+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f77cc122320 con 0x7f77cc09e1d0 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.393+0000 7f77c2ffd700 1 -- 172.21.15.32:0/334670608 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f77cc122320 con 0x7f77cc09e1d0 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 msgr2=0x7f77cc11f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc11f800 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f77bc016000 tx=0x7f77bc004750 comp rx=0 tx=0).stop 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 shutdown_connections 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f77cc139170 0x7f77cc1c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f77cc09d6f0 0x7f77cc11f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 --2- 172.21.15.32:0/334670608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77cc09e1d0 0x7f77cc11f800 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 >> 172.21.15.32:0/334670608 conn(0x7f77cc066f50 msgr2=0x7f77cc136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 shutdown_connections 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.395+0000 7f77d455f700 1 -- 172.21.15.32:0/334670608 wait complete. 2023-09-30T22:30:38.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.483+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f145c120c00 con 0x7f145c133a80 2023-09-30T22:30:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.483+0000 7f144affd700 1 -- 172.21.15.32:0/4143360837 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1454017020 con 0x7f145c133a80 2023-09-30T22:30:38.485 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.486+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f145c133a80 msgr2=0x7f145c09d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f145c133a80 0x7f145c09d340 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f145400c7f0 tx=0x7f145400cb00 comp rx=0 tx=0).stop 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 shutdown_connections 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f145c133a80 0x7f145c09d340 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f145c12ff20 0x7f145c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 --2- 172.21.15.32:0/4143360837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f145c130870 0x7f145c09ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 >> 172.21.15.32:0/4143360837 conn(0x7f145c0a4210 msgr2=0x7f145c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 shutdown_connections 2023-09-30T22:30:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.487+0000 7f1460fb9700 1 -- 172.21.15.32:0/4143360837 wait complete. 2023-09-30T22:30:38.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.542+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9150074ea0 con 0x7f915009e000 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.542+0000 7f9137fff700 1 -- 172.21.15.32:0/720709677 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9144015030 con 0x7f915009e000 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 msgr2=0x7f915011e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f915011e6d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9144000c00 tx=0x7f9144004750 comp rx=0 tx=0).stop 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 shutdown_connections 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f915009e410 0x7f915011ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f915009d6f0 0x7f915011e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 --2- 172.21.15.32:0/720709677 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f915009e000 0x7f915011e6d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 >> 172.21.15.32:0/720709677 conn(0x7f9150066f50 msgr2=0x7f915006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 shutdown_connections 2023-09-30T22:30:38.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:38.545+0000 7f9155e77700 1 -- 172.21.15.32:0/720709677 wait complete. 2023-09-30T22:30:38.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:38.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:39.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.005+0000 7f8e8f311700 1 Processor -- start 2023-09-30T22:30:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8f311700 1 -- start start 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8f311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 0x7f8e88138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8f311700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e88139170 0x7f8e8813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8f311700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8809b120 con 0x7f8e8809d6f0 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8f311700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8806b460 con 0x7f8e88139170 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8f311700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8809a7e0 con 0x7f8e8809e1d0 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8c8ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 0x7f8e88138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8c8ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 0x7f8e88138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44986/0 (socket says 172.21.15.32:44986) 2023-09-30T22:30:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8c8ac700 1 -- 172.21.15.32:0/1423293459 learned_addr learned my addr 172.21.15.32:0/1423293459 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.006+0000 7f8e8d8ae700 1 --2- 172.21.15.32:0/1423293459 >> v2:172.21.15.32:3300/0 conn(0x7f8e88139170 0x7f8e8813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e8d8ae700 1 -- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 msgr2=0x7f8e88138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e8d8ae700 1 --2- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 0x7f8e88138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e8d8ae700 1 -- 172.21.15.32:0/1423293459 >> v1:172.21.15.32:6789/0 conn(0x7f8e8809d6f0 legacy=0x7f8e8809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e8d8ae700 1 -- 172.21.15.32:0/1423293459 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8809a500 con 0x7f8e88139170 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e7bfff700 1 -- 172.21.15.32:0/1423293459 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050612148 0 0) 0x7f8e8809b120 con 0x7f8e8809d6f0 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e8c8ac700 1 --2- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 0x7f8e88138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e8d8ae700 1 --2- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e88139170 0x7f8e8813b550 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009d90 tx=0x7f8e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b10aa0d39f357b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e7bfff700 1 -- 172.21.15.32:0/1423293459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000e070 con 0x7f8e88139170 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e7bfff700 1 -- 172.21.15.32:0/1423293459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000bb20 con 0x7f8e88139170 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.007+0000 7f8e7bfff700 1 -- 172.21.15.32:0/1423293459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80012430 con 0x7f8e88139170 2023-09-30T22:30:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 -- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e88139170 msgr2=0x7f8e8813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 --2- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e88139170 0x7f8e8813b550 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009d90 tx=0x7f8e8000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 -- 172.21.15.32:0/1423293459 shutdown_connections 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 --2- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e88139170 0x7f8e8813b550 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 --2- 172.21.15.32:0/1423293459 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e8809e1d0 0x7f8e88138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 -- 172.21.15.32:0/1423293459 >> 172.21.15.32:0/1423293459 conn(0x7f8e88066f50 msgr2=0x7f8e8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 -- 172.21.15.32:0/1423293459 shutdown_connections 2023-09-30T22:30:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.008+0000 7f8e8f311700 1 -- 172.21.15.32:0/1423293459 wait complete. 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 Processor -- start 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 -- start start 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e8809d6f0 0x7f8e881218a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e8809e1d0 0x7f8e88121de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 0x7f8e88122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88124f00 con 0x7f8e8809d6f0 2023-09-30T22:30:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881250e0 con 0x7f8e88139170 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8f311700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8809c710 con 0x7f8e8809e1d0 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8d8ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 0x7f8e88122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.009+0000 7f8e8d0ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e8809d6f0 0x7f8e881218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d8ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 0x7f8e88122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:44996/0 (socket says 172.21.15.32:44996) 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d8ae700 1 -- 172.21.15.32:0/235632843 learned_addr learned my addr 172.21.15.32:0/235632843 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8c8ac700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e8809e1d0 0x7f8e88121de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d0ad700 1 -- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e8809e1d0 msgr2=0x7f8e88121de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d0ad700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e8809e1d0 0x7f8e88121de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d0ad700 1 -- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 msgr2=0x7f8e88122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d0ad700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 0x7f8e88122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d0ad700 1 -- 172.21.15.32:0/235632843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009550 con 0x7f8e8809d6f0 2023-09-30T22:30:39.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d8ae700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 0x7f8e88122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8d0ad700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e8809d6f0 0x7f8e881218a0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c460 tx=0x7f8e8400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840054e0 con 0x7f8e8809d6f0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8400ce30 con 0x7f8e8809d6f0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e80009a70 con 0x7f8e8809d6f0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.010+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840038c0 con 0x7f8e8809d6f0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.011+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e88072150 con 0x7f8e8809d6f0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.011+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e84003a20 con 0x7f8e8809d6f0 2023-09-30T22:30:39.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.011+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e8403e9b0 con 0x7f8e8809d6f0 2023-09-30T22:30:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.012+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e88125220 con 0x7f8e8809d6f0 2023-09-30T22:30:39.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.017+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e88125220 con 0x7f8e8809d6f0 2023-09-30T22:30:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.226+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8e88125220 con 0x7f8e8809d6f0 2023-09-30T22:30:39.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.226+0000 7f8e79ffb700 1 -- 172.21.15.32:0/235632843 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e88125220 con 0x7f8e8809d6f0 2023-09-30T22:30:39.227 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e8809d6f0 msgr2=0x7f8e881218a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e8809d6f0 0x7f8e881218a0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c460 tx=0x7f8e8400c770 comp rx=0 tx=0).stop 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 shutdown_connections 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e88139170 0x7f8e88122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e8809d6f0 0x7f8e881218a0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 --2- 172.21.15.32:0/235632843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e8809e1d0 0x7f8e88121de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.228+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 >> 172.21.15.32:0/235632843 conn(0x7f8e88066f50 msgr2=0x7f8e88070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.229+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 shutdown_connections 2023-09-30T22:30:39.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.229+0000 7f8e8f311700 1 -- 172.21.15.32:0/235632843 wait complete. 2023-09-30T22:30:39.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:39.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:39.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:39.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.563+0000 7fd31b1c6700 1 Processor -- start 2023-09-30T22:30:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.563+0000 7fd31b1c6700 1 -- start start 2023-09-30T22:30:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd31b1c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409d6f0 0x7fd31409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd31b1c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd31409e000 0x7fd314130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd31b1c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409e410 0x7fd314130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd31b1c6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd31409f520 con 0x7fd31409e000 2023-09-30T22:30:39.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd31b1c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31409f130 con 0x7fd31409e410 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd31b1c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd31409e930 con 0x7fd31409d6f0 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd313fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd31409e000 0x7fd314130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd313fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd31409e000 0x7fd314130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59980/0 (socket says 172.21.15.32:59980) 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd313fff700 1 -- 172.21.15.32:0/1103679330 learned_addr learned my addr 172.21.15.32:0/1103679330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd319763700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409e410 0x7fd314130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.564+0000 7fd318f62700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409d6f0 0x7fd31409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd313fff700 1 -- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409d6f0 msgr2=0x7fd31409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd313fff700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409d6f0 0x7fd31409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd313fff700 1 -- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409e410 msgr2=0x7fd314130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd313fff700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409e410 0x7fd314130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd313fff700 1 -- 172.21.15.32:0/1103679330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31409b2d0 con 0x7fd31409e000 2023-09-30T22:30:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd319763700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409e410 0x7fd314130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd313fff700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e000 0x7fd314130660 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd300009920 tx=0x7fd300009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a447206de234a17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd3137fe700 1 -- 172.21.15.32:0/1103679330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300011830 con 0x7fd31409e000 2023-09-30T22:30:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd3137fe700 1 -- 172.21.15.32:0/1103679330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300011990 con 0x7fd31409e000 2023-09-30T22:30:39.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.565+0000 7fd3137fe700 1 -- 172.21.15.32:0/1103679330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300011d20 con 0x7fd31409e000 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 -- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e000 msgr2=0x7fd314130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e000 0x7fd314130660 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd300009920 tx=0x7fd300009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 -- 172.21.15.32:0/1103679330 shutdown_connections 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409e410 0x7fd314130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409d6f0 0x7fd31409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/1103679330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e000 0x7fd314130660 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 -- 172.21.15.32:0/1103679330 >> 172.21.15.32:0/1103679330 conn(0x7fd314066f50 msgr2=0x7fd31409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 -- 172.21.15.32:0/1103679330 shutdown_connections 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.566+0000 7fd31b1c6700 1 -- 172.21.15.32:0/1103679330 wait complete. 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.567+0000 7fd31b1c6700 1 Processor -- start 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.568+0000 7fd31b1c6700 1 -- start start 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd31b1c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 0x7fd3141c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd31b1c6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409e000 0x7fd3141c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd31b1c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e410 0x7fd3141225a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd31b1c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141241b0 con 0x7fd31409e410 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd31b1c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141247d0 con 0x7fd31409d6f0 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd31b1c6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd314125de0 con 0x7fd31409e000 2023-09-30T22:30:39.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 0x7fd3141c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 0x7fd3141c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45014/0 (socket says 172.21.15.32:45014) 2023-09-30T22:30:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd319763700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e410 0x7fd3141225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 -- 172.21.15.32:0/4126989436 learned_addr learned my addr 172.21.15.32:0/4126989436 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd313fff700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409e000 0x7fd3141c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 -- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409e000 msgr2=0x7fd3141c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409e000 0x7fd3141c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 -- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e410 msgr2=0x7fd3141225a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e410 0x7fd3141225a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 -- 172.21.15.32:0/4126989436 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c00bc80 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd319763700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e410 0x7fd3141225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.569+0000 7fd318f62700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 0x7fd3141c8840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd30800e930 tx=0x7fd30800ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.570+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308013680 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.570+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3080137e0 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.570+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd300009550 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.570+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308013b70 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.570+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31409c700 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.571+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd308013cd0 con 0x7fd31409d6f0 2023-09-30T22:30:39.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.571+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd30803f6c0 con 0x7fd31409d6f0 2023-09-30T22:30:39.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.571+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3140a18a0 con 0x7fd31409d6f0 2023-09-30T22:30:39.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.577+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd30801bc00 con 0x7fd31409d6f0 2023-09-30T22:30:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 Processor -- start 2023-09-30T22:30:39.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 -- start start 2023-09-30T22:30:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95ac0a1340 0x7f95ac133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac09edd0 con 0x7f95ac133fa0 2023-09-30T22:30:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac09e1f0 con 0x7f95ac0a1340 2023-09-30T22:30:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b29c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac09dc50 con 0x7f95ac0a0a30 2023-09-30T22:30:39.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95ab7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95ac0a1340 0x7f95ac133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95abfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53484/0 (socket says 172.21.15.32:53484) 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95ab7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95ac0a1340 0x7f95ac133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59994/0 (socket says 172.21.15.32:59994) 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95abfff700 1 -- 172.21.15.32:0/2777263797 learned_addr learned my addr 172.21.15.32:0/2777263797 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.666+0000 7f95b0f65700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f95ac133fa0 0x7f95ac13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34464/0 (socket says 172.21.15.32:34464) 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac0a0a30 msgr2=0x7f95ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95ab7fe700 1 --2- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2777263797 >> v1:172.21.15.32:6789/0 conn(0x7f95ac133fa0 legacy=0x7f95ac13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2777263797 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0a1fe0 con 0x7f95ac0a1340 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95ab7fe700 1 --2- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a1340 0x7f95ac133a60 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009840 tx=0x7f95a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff221eba35afb3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95aaffd700 1 -- 172.21.15.32:0/2777263797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0011830 con 0x7f95ac0a1340 2023-09-30T22:30:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95aaffd700 1 -- 172.21.15.32:0/2777263797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0011990 con 0x7f95ac0a1340 2023-09-30T22:30:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.667+0000 7f95aaffd700 1 -- 172.21.15.32:0/2777263797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0011d20 con 0x7f95ac0a1340 2023-09-30T22:30:39.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 -- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a1340 msgr2=0x7f95ac133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 --2- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a1340 0x7f95ac133a60 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009840 tx=0x7f95a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 -- 172.21.15.32:0/2777263797 shutdown_connections 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 --2- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac0a0a30 0x7f95ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 --2- 172.21.15.32:0/2777263797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a1340 0x7f95ac133a60 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 -- 172.21.15.32:0/2777263797 >> 172.21.15.32:0/2777263797 conn(0x7f95ac0a6550 msgr2=0x7f95ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 -- 172.21.15.32:0/2777263797 shutdown_connections 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.668+0000 7f95b29c8700 1 -- 172.21.15.32:0/2777263797 wait complete. 2023-09-30T22:30:39.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.670+0000 7f95b29c8700 1 Processor -- start 2023-09-30T22:30:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.670+0000 7f95b29c8700 1 -- start start 2023-09-30T22:30:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b29c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a0a30 0x7f95ac1c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b29c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 0x7f95ac1c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b29c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac133fa0 0x7f95ac1c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b29c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1207e0 con 0x7f95ac0a0a30 2023-09-30T22:30:39.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b29c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1205f0 con 0x7f95ac0a1340 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b29c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac120ec0 con 0x7f95ac133fa0 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 0x7f95ac1c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95ab7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 0x7f95ac1c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45016/0 (socket says 172.21.15.32:45016) 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95ab7fe700 1 -- 172.21.15.32:0/1377773353 learned_addr learned my addr 172.21.15.32:0/1377773353 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95abfff700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a0a30 0x7f95ac1c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.671+0000 7f95b0f65700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac133fa0 0x7f95ac1c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95ab7fe700 1 -- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac133fa0 msgr2=0x7f95ac1c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95ab7fe700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac133fa0 0x7f95ac1c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95ab7fe700 1 -- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a0a30 msgr2=0x7f95ac1c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95ab7fe700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a0a30 0x7f95ac1c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95ab7fe700 1 -- 172.21.15.32:0/1377773353 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00094a0 con 0x7f95ac0a1340 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95abfff700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a0a30 0x7f95ac1c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95ab7fe700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 0x7f95ac1c8b00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009fd0 tx=0x7f95a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00036a0 con 0x7f95ac0a1340 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a001f070 con 0x7f95ac0a1340 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac121cb0 con 0x7f95ac0a1340 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0003a60 con 0x7f95ac0a1340 2023-09-30T22:30:39.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.672+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac122220 con 0x7f95ac0a1340 2023-09-30T22:30:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.673+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95a0003bc0 con 0x7f95ac0a1340 2023-09-30T22:30:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.673+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f95a0021070 con 0x7f95ac0a1340 2023-09-30T22:30:39.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.674+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac122530 con 0x7f95ac0a1340 2023-09-30T22:30:39.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.678+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95a002b080 con 0x7f95ac0a1340 2023-09-30T22:30:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.714+0000 7efec0a97700 1 Processor -- start 2023-09-30T22:30:39.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.714+0000 7efec0a97700 1 -- start start 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efec0a97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 0x7efebc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efec0a97700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 0x7efebc131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efec0a97700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efebc070940 con 0x7efebc09f240 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efec0a97700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efebc06fd60 con 0x7efebc09f650 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efec0a97700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efebc06fa80 con 0x7efebc09e930 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efeba59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 0x7efebc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efebad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 0x7efebc131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efeba59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 0x7efebc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53496/0 (socket says 172.21.15.32:53496) 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efeba59c700 1 -- 172.21.15.32:0/3835298440 learned_addr learned my addr 172.21.15.32:0/3835298440 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efebad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 0x7efebc131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60018/0 (socket says 172.21.15.32:60018) 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.715+0000 7efeb9d9b700 1 -- 172.21.15.32:0/3835298440 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592085486 0 0) 0x7efebc070940 con 0x7efebc09f240 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeba59c700 1 -- 172.21.15.32:0/3835298440 >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 msgr2=0x7efebc131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeba59c700 1 --2- 172.21.15.32:0/3835298440 >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 0x7efebc131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeba59c700 1 -- 172.21.15.32:0/3835298440 >> v1:172.21.15.32:6789/0 conn(0x7efebc09f240 legacy=0x7efebc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeba59c700 1 -- 172.21.15.32:0/3835298440 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc09fb60 con 0x7efebc09e930 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efebad9d700 1 --2- 172.21.15.32:0/3835298440 >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 0x7efebc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeba59c700 1 --2- 172.21.15.32:0/3835298440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 0x7efebc09ed00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7efeac00dd10 tx=0x7efeac00f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15877c1a0dab8540 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeb9d9b700 1 -- 172.21.15.32:0/3835298440 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac012070 con 0x7efebc09e930 2023-09-30T22:30:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeb9d9b700 1 -- 172.21.15.32:0/3835298440 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac00d2e0 con 0x7efebc09e930 2023-09-30T22:30:39.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.716+0000 7efeb9d9b700 1 -- 172.21.15.32:0/3835298440 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac00fb10 con 0x7efebc09e930 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 -- 172.21.15.32:0/3835298440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 msgr2=0x7efebc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 --2- 172.21.15.32:0/3835298440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 0x7efebc09ed00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7efeac00dd10 tx=0x7efeac00f330 comp rx=0 tx=0).stop 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 -- 172.21.15.32:0/3835298440 shutdown_connections 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 --2- 172.21.15.32:0/3835298440 >> v2:172.21.15.32:3300/0 conn(0x7efebc09f650 0x7efebc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 --2- 172.21.15.32:0/3835298440 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09e930 0x7efebc09ed00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 -- 172.21.15.32:0/3835298440 >> 172.21.15.32:0/3835298440 conn(0x7efebc0a4210 msgr2=0x7efebc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 -- 172.21.15.32:0/3835298440 shutdown_connections 2023-09-30T22:30:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.717+0000 7efec0a97700 1 -- 172.21.15.32:0/3835298440 wait complete. 2023-09-30T22:30:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.719+0000 7efec0a97700 1 Processor -- start 2023-09-30T22:30:39.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.719+0000 7efec0a97700 1 -- start start 2023-09-30T22:30:39.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efec0a97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efebc09e930 0x7efebc1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efec0a97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 0x7efebc1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efec0a97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc09f650 0x7efebc1c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efec0a97700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc1a1b50 con 0x7efebc09f650 2023-09-30T22:30:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efec0a97700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efebc1a0b70 con 0x7efebc09e930 2023-09-30T22:30:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efec0a97700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efebc1b4ff0 con 0x7efebc09f240 2023-09-30T22:30:39.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 0x7efebc1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 0x7efebc1a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:53498/0 (socket says 172.21.15.32:53498) 2023-09-30T22:30:39.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 -- 172.21.15.32:0/865531362 learned_addr learned my addr 172.21.15.32:0/865531362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:39.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efebad9d700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc09f650 0x7efebc1c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeba59c700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efebc09e930 0x7efebc1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 -- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efebc09e930 msgr2=0x7efebc1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efebc09e930 0x7efebc1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 -- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc09f650 msgr2=0x7efebc1c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc09f650 0x7efebc1c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeb1d9b700 1 -- 172.21.15.32:0/865531362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4009a70 con 0x7efebc09f240 2023-09-30T22:30:39.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efebad9d700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc09f650 0x7efebc1c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.720+0000 7efeba59c700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efebc09e930 0x7efebc1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.721+0000 7efeb1d9b700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 0x7efebc1a1610 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7efea400c580 tx=0x7efea400c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.721+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40054e0 con 0x7efebc09f240 2023-09-30T22:30:39.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.721+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea400ce80 con 0x7efebc09f240 2023-09-30T22:30:39.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.721+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac00d9f0 con 0x7efebc09f240 2023-09-30T22:30:39.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.721+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40038c0 con 0x7efebc09f240 2023-09-30T22:30:39.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.721+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc0a0190 con 0x7efebc09f240 2023-09-30T22:30:39.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.722+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efea4003a20 con 0x7efebc09f240 2023-09-30T22:30:39.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.722+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efea403ea90 con 0x7efebc09f240 2023-09-30T22:30:39.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.722+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc09a3c0 con 0x7efebc09f240 2023-09-30T22:30:39.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.731+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efea4017080 con 0x7efebc09f240 2023-09-30T22:30:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.815+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd314074ea0 con 0x7fd31409d6f0 2023-09-30T22:30:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.815+0000 7fd3117fa700 1 -- 172.21.15.32:0/4126989436 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd308021030 con 0x7fd31409d6f0 2023-09-30T22:30:39.817 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 msgr2=0x7fd3141c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 0x7fd3141c8840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd30800e930 tx=0x7fd30800ec40 comp rx=0 tx=0).stop 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 shutdown_connections 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd31409e410 0x7fd3141225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd31409d6f0 0x7fd3141c8840 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 --2- 172.21.15.32:0/4126989436 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd31409e000 0x7fd3141c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 >> 172.21.15.32:0/4126989436 conn(0x7fd314066f50 msgr2=0x7fd31406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 shutdown_connections 2023-09-30T22:30:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.818+0000 7fd31b1c6700 1 -- 172.21.15.32:0/4126989436 wait complete. 2023-09-30T22:30:39.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.915+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f95ac0a1d40 con 0x7f95ac0a1340 2023-09-30T22:30:39.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.915+0000 7f95a8ff9700 1 -- 172.21.15.32:0/1377773353 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95a0015020 con 0x7f95ac0a1340 2023-09-30T22:30:39.916 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 msgr2=0x7f95ac1c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 0x7f95ac1c8b00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009fd0 tx=0x7f95a0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 shutdown_connections 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac133fa0 0x7f95ac1c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0a0a30 0x7f95ac1c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 --2- 172.21.15.32:0/1377773353 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a1340 0x7f95ac1c8b00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 >> 172.21.15.32:0/1377773353 conn(0x7f95ac0a6550 msgr2=0x7f95ac1b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 shutdown_connections 2023-09-30T22:30:39.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.918+0000 7f95b29c8700 1 -- 172.21.15.32:0/1377773353 wait complete. 2023-09-30T22:30:39.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.953+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efebc074ea0 con 0x7efebc09f240 2023-09-30T22:30:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.953+0000 7efeb37fe700 1 -- 172.21.15.32:0/865531362 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efea4019b40 con 0x7efebc09f240 2023-09-30T22:30:39.954 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 msgr2=0x7efebc1a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 0x7efebc1a1610 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7efea400c580 tx=0x7efea400c890 comp rx=0 tx=0).stop 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 shutdown_connections 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efebc09f650 0x7efebc1c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efebc09e930 0x7efebc1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 --2- 172.21.15.32:0/865531362 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efebc09f240 0x7efebc1a1610 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 >> 172.21.15.32:0/865531362 conn(0x7efebc0a4210 msgr2=0x7efebc06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 shutdown_connections 2023-09-30T22:30:39.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:39.955+0000 7efec0a97700 1 -- 172.21.15.32:0/865531362 wait complete. 2023-09-30T22:30:39.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:40.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 Processor -- start 2023-09-30T22:30:40.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 -- start start 2023-09-30T22:30:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f323009e1d0 0x7f3230138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f323013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323009b120 con 0x7f323009d6f0 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323006b460 con 0x7f323009e1d0 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3235c2e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f323009a7e0 con 0x7f3230139170 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f3226ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f323009e1d0 0x7f3230138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f322f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f323009d6f0 0x7f323009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34490/0 (socket says 172.21.15.32:34490) 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f322f7fe700 1 -- 172.21.15.32:0/3886587356 learned_addr learned my addr 172.21.15.32:0/3886587356 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.389+0000 7f322ffff700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f323013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f3226ffd700 1 -- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 msgr2=0x7f323013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:40.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f3226ffd700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f323013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f3226ffd700 1 -- 172.21.15.32:0/3886587356 >> v1:172.21.15.32:6789/0 conn(0x7f323009d6f0 legacy=0x7f323009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f3226ffd700 1 -- 172.21.15.32:0/3886587356 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323009a500 con 0x7f323009e1d0 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f322effd700 1 -- 172.21.15.32:0/3886587356 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084912507 0 0) 0x7f323009b120 con 0x7f323009d6f0 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f322ffff700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f323013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f3226ffd700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230138c30 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3218009870 tx=0x7f3218009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4199deaa336ca507 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f322effd700 1 -- 172.21.15.32:0/3886587356 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011830 con 0x7f323009e1d0 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f322effd700 1 -- 172.21.15.32:0/3886587356 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218011990 con 0x7f323009e1d0 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.390+0000 7f322effd700 1 -- 172.21.15.32:0/3886587356 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011d20 con 0x7f323009e1d0 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 -- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 msgr2=0x7f3230138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230138c30 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3218009870 tx=0x7f3218009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:40.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 -- 172.21.15.32:0/3886587356 shutdown_connections 2023-09-30T22:30:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f323013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 --2- 172.21.15.32:0/3886587356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230138c30 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 -- 172.21.15.32:0/3886587356 >> 172.21.15.32:0/3886587356 conn(0x7f3230066f50 msgr2=0x7f323009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 -- 172.21.15.32:0/3886587356 shutdown_connections 2023-09-30T22:30:40.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.391+0000 7f3235c2e700 1 -- 172.21.15.32:0/3886587356 wait complete. 2023-09-30T22:30:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.392+0000 7f3235c2e700 1 Processor -- start 2023-09-30T22:30:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 -- start start 2023-09-30T22:30:40.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323009d6f0 0x7f3230121750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230121c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f32301221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323009c710 con 0x7f323009e1d0 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3230123bc0 con 0x7f323009d6f0 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3235c2e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3230137640 con 0x7f3230139170 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323009d6f0 0x7f3230121750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230121c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230121c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60048/0 (socket says 172.21.15.32:60048) 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f322f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323009d6f0 0x7f3230121750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:45034/0 (socket says 172.21.15.32:45034) 2023-09-30T22:30:40.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f3226ffd700 1 -- 172.21.15.32:0/2368043844 learned_addr learned my addr 172.21.15.32:0/2368043844 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.393+0000 7f322ffff700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f32301221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3226ffd700 1 -- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 msgr2=0x7f32301221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3226ffd700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f32301221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3226ffd700 1 -- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323009d6f0 msgr2=0x7f3230121750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3226ffd700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323009d6f0 0x7f3230121750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3226ffd700 1 -- 172.21.15.32:0/2368043844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228009a70 con 0x7f323009e1d0 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3226ffd700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230121c90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3218000c00 tx=0x7f3218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180036a0 con 0x7f323009e1d0 2023-09-30T22:30:40.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321801f070 con 0x7f323009e1d0 2023-09-30T22:30:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32180094a0 con 0x7f323009e1d0 2023-09-30T22:30:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218003a20 con 0x7f323009e1d0 2023-09-30T22:30:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.394+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230123450 con 0x7f323009e1d0 2023-09-30T22:30:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.395+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3218003b80 con 0x7f323009e1d0 2023-09-30T22:30:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.395+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3218021070 con 0x7f323009e1d0 2023-09-30T22:30:40.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.395+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230074ea0 con 0x7f323009e1d0 2023-09-30T22:30:40.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.400+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f321802b080 con 0x7f323009e1d0 2023-09-30T22:30:40.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.604+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f323009f420 con 0x7f323009e1d0 2023-09-30T22:30:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.605+0000 7f322cff9700 1 -- 172.21.15.32:0/2368043844 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3218035090 con 0x7f323009e1d0 2023-09-30T22:30:40.606 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 msgr2=0x7f3230121c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230121c90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3218000c00 tx=0x7f3218004750 comp rx=0 tx=0).stop 2023-09-30T22:30:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 shutdown_connections 2023-09-30T22:30:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3230139170 0x7f32301221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f323009d6f0 0x7f3230121750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 --2- 172.21.15.32:0/2368043844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f323009e1d0 0x7f3230121c90 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 >> 172.21.15.32:0/2368043844 conn(0x7f3230066f50 msgr2=0x7f3230070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 shutdown_connections 2023-09-30T22:30:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.607+0000 7f3235c2e700 1 -- 172.21.15.32:0/2368043844 wait complete. 2023-09-30T22:30:40.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:40.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:40.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:40.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:40.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.982+0000 7f6211689700 1 Processor -- start 2023-09-30T22:30:40.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.982+0000 7f6211689700 1 -- start start 2023-09-30T22:30:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f6211689700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f6211689700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f620c09e410 0x7f620c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f6211689700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c09b120 con 0x7f620c09e000 2023-09-30T22:30:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f6211689700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c06b460 con 0x7f620c09e410 2023-09-30T22:30:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f6211689700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c09a7e0 con 0x7f620c09d6f0 2023-09-30T22:30:40.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f620b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f620c09e410 0x7f620c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f620b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f620c09e410 0x7f620c134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45784/0 (socket says 172.21.15.32:45784) 2023-09-30T22:30:40.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f620affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.983+0000 7f620b7fe700 1 -- 172.21.15.32:0/439471005 learned_addr learned my addr 172.21.15.32:0/439471005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:40.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f620b7fe700 1 -- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 msgr2=0x7f620c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f620b7fe700 1 --2- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f620b7fe700 1 -- 172.21.15.32:0/439471005 >> v1:172.21.15.32:6789/0 conn(0x7f620c09e000 legacy=0x7f620c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f620b7fe700 1 -- 172.21.15.32:0/439471005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c09a500 con 0x7f620c09e410 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f620affd700 1 --2- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f620b7fe700 1 --2- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c134100 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6204009e50 tx=0x7f620400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acf749e28a590f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f6209ffb700 1 -- 172.21.15.32:0/439471005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400e070 con 0x7f620c09e410 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f6209ffb700 1 -- 172.21.15.32:0/439471005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620400b910 con 0x7f620c09e410 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.984+0000 7f6209ffb700 1 -- 172.21.15.32:0/439471005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400bcc0 con 0x7f620c09e410 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 -- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 msgr2=0x7f620c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 --2- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c134100 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6204009e50 tx=0x7f620400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 -- 172.21.15.32:0/439471005 shutdown_connections 2023-09-30T22:30:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 --2- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c134100 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 --2- 172.21.15.32:0/439471005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 -- 172.21.15.32:0/439471005 >> 172.21.15.32:0/439471005 conn(0x7f620c066f50 msgr2=0x7f620c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 -- 172.21.15.32:0/439471005 shutdown_connections 2023-09-30T22:30:40.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.985+0000 7f6211689700 1 -- 172.21.15.32:0/439471005 wait complete. 2023-09-30T22:30:40.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 Processor -- start 2023-09-30T22:30:40.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 -- start start 2023-09-30T22:30:40.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c1a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f620c09e000 0x7f620c1a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c1a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1a8940 con 0x7f620c09e410 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1a7470 con 0x7f620c09d6f0 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f6211689700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1a7b40 con 0x7f620c09e000 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f620a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f620c09e000 0x7f620c1a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f620affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c1a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f620a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f620c09e000 0x7f620c1a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48046/0 (socket says 172.21.15.32:48046) 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f620affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c1a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47788/0 (socket says 172.21.15.32:47788) 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f620a7fc700 1 -- 172.21.15.32:0/4282770082 learned_addr learned my addr 172.21.15.32:0/4282770082 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.987+0000 7f620b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c1a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620affd700 1 -- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f620c09e000 msgr2=0x7f620c1a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620affd700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f620c09e000 0x7f620c1a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620affd700 1 -- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 msgr2=0x7f620c1a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620affd700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c1a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620affd700 1 -- 172.21.15.32:0/4282770082 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0094a0 con 0x7f620c09d6f0 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620b7fe700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c1a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f620affd700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c1a5fc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e9b0 tx=0x7f61fc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00ee50 con 0x7f620c09d6f0 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc00c850 con 0x7f620c09d6f0 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6204009b30 con 0x7f620c09d6f0 2023-09-30T22:30:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.988+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c09c710 con 0x7f620c09d6f0 2023-09-30T22:30:40.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.989+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00cc00 con 0x7f620c09d6f0 2023-09-30T22:30:40.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.989+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f61fc013470 con 0x7f620c09d6f0 2023-09-30T22:30:40.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.990+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f61fc03f8d0 con 0x7f620c09d6f0 2023-09-30T22:30:40.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.990+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c074ea0 con 0x7f620c09d6f0 2023-09-30T22:30:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:40.995+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f61fc017080 con 0x7f620c09d6f0 2023-09-30T22:30:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.078+0000 7f543e674700 1 Processor -- start 2023-09-30T22:30:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.078+0000 7f543e674700 1 -- start start 2023-09-30T22:30:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f543e674700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f5438136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f543e674700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5438136a70 0x7f5438138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f543e674700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543809e130 con 0x7f54381316c0 2023-09-30T22:30:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f543e674700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54381321a0 con 0x7f5438136a70 2023-09-30T22:30:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f543e674700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f54381312b0 con 0x7f543809d9f0 2023-09-30T22:30:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f54377fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f5438136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f5437fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f54381316c0 0x7f5438131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48128/0 (socket says 172.21.15.32:48128) 2023-09-30T22:30:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f543cc11700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5438136a70 0x7f5438138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f5437fff700 1 -- 172.21.15.32:0/900661788 learned_addr learned my addr 172.21.15.32:0/900661788 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.079+0000 7f54377fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f5438136530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48052/0 (socket says 172.21.15.32:48052) 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f54377fe700 1 -- 172.21.15.32:0/900661788 >> v2:172.21.15.32:3300/0 conn(0x7f5438136a70 msgr2=0x7f5438138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f54377fe700 1 --2- 172.21.15.32:0/900661788 >> v2:172.21.15.32:3300/0 conn(0x7f5438136a70 0x7f5438138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f54377fe700 1 -- 172.21.15.32:0/900661788 >> v1:172.21.15.32:6789/0 conn(0x7f54381316c0 legacy=0x7f5438131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f54377fe700 1 -- 172.21.15.32:0/900661788 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543809f6e0 con 0x7f543809d9f0 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f54377fe700 1 --2- 172.21.15.32:0/900661788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f5438136530 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f542c00dc60 tx=0x7f542c00df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a39357c3b26869 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f5436ffd700 1 -- 172.21.15.32:0/900661788 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c012070 con 0x7f543809d9f0 2023-09-30T22:30:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f5436ffd700 1 -- 172.21.15.32:0/900661788 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c00d2e0 con 0x7f543809d9f0 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.080+0000 7f5436ffd700 1 -- 172.21.15.32:0/900661788 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00f900 con 0x7f543809d9f0 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 -- 172.21.15.32:0/900661788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 msgr2=0x7f5438136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 --2- 172.21.15.32:0/900661788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f5438136530 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f542c00dc60 tx=0x7f542c00df70 comp rx=0 tx=0).stop 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 -- 172.21.15.32:0/900661788 shutdown_connections 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 --2- 172.21.15.32:0/900661788 >> v2:172.21.15.32:3300/0 conn(0x7f5438136a70 0x7f5438138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 --2- 172.21.15.32:0/900661788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f5438136530 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 -- 172.21.15.32:0/900661788 >> 172.21.15.32:0/900661788 conn(0x7f5438066f50 msgr2=0x7f543809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 -- 172.21.15.32:0/900661788 shutdown_connections 2023-09-30T22:30:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.081+0000 7f543e674700 1 -- 172.21.15.32:0/900661788 wait complete. 2023-09-30T22:30:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.082+0000 7f543e674700 1 Processor -- start 2023-09-30T22:30:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 -- start start 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f543812a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 0x7f543812af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5438136a70 0x7f54380712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543812b900 con 0x7f5438136a70 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380704a0 con 0x7f54381316c0 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.083+0000 7f543e674700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5438136550 con 0x7f543809d9f0 2023-09-30T22:30:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 0x7f543812af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 0x7f543812af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47794/0 (socket says 172.21.15.32:47794) 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 -- 172.21.15.32:0/2808462905 learned_addr learned my addr 172.21.15.32:0/2808462905 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f5437fff700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f543812a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f543cc11700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5438136a70 0x7f54380712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 -- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 msgr2=0x7f543812a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f543812a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 -- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5438136a70 msgr2=0x7f54380712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5438136a70 0x7f54380712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 -- 172.21.15.32:0/2808462905 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c00d940 con 0x7f54381316c0 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f5437fff700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f543812a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f54377fe700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 0x7f543812af20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f54381313f0 tx=0x7f542c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.084+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0036a0 con 0x7f54381316c0 2023-09-30T22:30:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.085+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c003820 con 0x7f54381316c0 2023-09-30T22:30:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.085+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543812b460 con 0x7f54381316c0 2023-09-30T22:30:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.085+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c003bd0 con 0x7f54381316c0 2023-09-30T22:30:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.085+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5438072620 con 0x7f54381316c0 2023-09-30T22:30:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.086+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f542c003d30 con 0x7f54381316c0 2023-09-30T22:30:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.086+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54380702e0 con 0x7f54381316c0 2023-09-30T22:30:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.086+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f542c044cf0 con 0x7f54381316c0 2023-09-30T22:30:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.095+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54380702e0 con 0x7f54381316c0 2023-09-30T22:30:41.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.127+0000 7f7c5cf4a700 1 Processor -- start 2023-09-30T22:30:41.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.127+0000 7f7c5cf4a700 1 -- start start 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5cf4a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809e930 0x7f7c5809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5cf4a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c5809f650 0x7f7c58131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5cf4a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c58070940 con 0x7f7c5809f240 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5cf4a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5806fd60 con 0x7f7c5809f650 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5cf4a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5806fa80 con 0x7f7c5809e930 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c56d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c5809f650 0x7f7c58131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c55d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c5809f240 0x7f7c581317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48138/0 (socket says 172.21.15.32:48138) 2023-09-30T22:30:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c55d9b700 1 -- 172.21.15.32:0/1504628782 learned_addr learned my addr 172.21.15.32:0/1504628782 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:41.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5659c700 1 --2- 172.21.15.32:0/1504628782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809e930 0x7f7c5809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5559a700 1 -- 172.21.15.32:0/1504628782 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376213773 0 0) 0x7f7c58070940 con 0x7f7c5809f240 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.128+0000 7f7c5559a700 1 -- 172.21.15.32:0/1504628782 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c400035b0 con 0x7f7c5809f240 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5659c700 1 -- 172.21.15.32:0/1504628782 >> v2:172.21.15.32:3300/0 conn(0x7f7c5809f650 msgr2=0x7f7c58131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5659c700 1 --2- 172.21.15.32:0/1504628782 >> v2:172.21.15.32:3300/0 conn(0x7f7c5809f650 0x7f7c58131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5659c700 1 -- 172.21.15.32:0/1504628782 >> v1:172.21.15.32:6789/0 conn(0x7f7c5809f240 legacy=0x7f7c581317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5659c700 1 -- 172.21.15.32:0/1504628782 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5809fb60 con 0x7f7c5809e930 2023-09-30T22:30:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c56d9d700 1 --2- 172.21.15.32:0/1504628782 >> v2:172.21.15.32:3300/0 conn(0x7f7c5809f650 0x7f7c58131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5659c700 1 --2- 172.21.15.32:0/1504628782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809e930 0x7f7c5809ed00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009840 tx=0x7f7c48009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530952b84607ae9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5559a700 1 -- 172.21.15.32:0/1504628782 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48011830 con 0x7f7c5809e930 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5559a700 1 -- 172.21.15.32:0/1504628782 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c48011990 con 0x7f7c5809e930 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.129+0000 7f7c5559a700 1 -- 172.21.15.32:0/1504628782 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48011d20 con 0x7f7c5809e930 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/1504628782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809e930 msgr2=0x7f7c5809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/1504628782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809e930 0x7f7c5809ed00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009840 tx=0x7f7c48009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/1504628782 shutdown_connections 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/1504628782 >> v2:172.21.15.32:3300/0 conn(0x7f7c5809f650 0x7f7c58131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/1504628782 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809e930 0x7f7c5809ed00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/1504628782 >> 172.21.15.32:0/1504628782 conn(0x7f7c580a4210 msgr2=0x7f7c580a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/1504628782 shutdown_connections 2023-09-30T22:30:41.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.130+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/1504628782 wait complete. 2023-09-30T22:30:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.131+0000 7f7c5cf4a700 1 Processor -- start 2023-09-30T22:30:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.131+0000 7f7c5cf4a700 1 -- start start 2023-09-30T22:30:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5cf4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c5809e930 0x7f7c581a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5cf4a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809f240 0x7f7c581a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5cf4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 0x7f7c581c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5cf4a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581a23a0 con 0x7f7c5809f650 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5cf4a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581a21b0 con 0x7f7c5809e930 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5cf4a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581a27d0 con 0x7f7c5809f240 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c56d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 0x7f7c581c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c55d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809f240 0x7f7c581a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c56d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 0x7f7c581c3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45836/0 (socket says 172.21.15.32:45836) 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c56d9d700 1 -- 172.21.15.32:0/2272543837 learned_addr learned my addr 172.21.15.32:0/2272543837 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c5659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c5809e930 0x7f7c581a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c56d9d700 1 -- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809f240 msgr2=0x7f7c581a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c56d9d700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809f240 0x7f7c581a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.132+0000 7f7c56d9d700 1 -- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c5809e930 msgr2=0x7f7c581a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c56d9d700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c5809e930 0x7f7c581a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c56d9d700 1 -- 172.21.15.32:0/2272543837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009a70 con 0x7f7c5809f650 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c55d9b700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809f240 0x7f7c581a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c5659c700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c5809e930 0x7f7c581a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c56d9d700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 0x7f7c581c3c40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b330 tx=0x7f7c5000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50015070 con 0x7f7c5809f650 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c480094a0 con 0x7f7c5809f650 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5000bd70 con 0x7f7c5809f650 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50009d70 con 0x7f7c5809f650 2023-09-30T22:30:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.133+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c580a0190 con 0x7f7c5809f650 2023-09-30T22:30:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.134+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c50019410 con 0x7f7c5809f650 2023-09-30T22:30:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.134+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c50043560 con 0x7f7c5809f650 2023-09-30T22:30:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.135+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c580a30d0 con 0x7f7c5809f650 2023-09-30T22:30:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.140+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c50017900 con 0x7f7c5809f650 2023-09-30T22:30:41.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.250+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f620c1a9410 con 0x7f620c09d6f0 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.251+0000 7f61f37fe700 1 -- 172.21.15.32:0/4282770082 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f620c1a9410 con 0x7f620c09d6f0 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 msgr2=0x7f620c1a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c1a5fc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e9b0 tx=0x7f61fc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 shutdown_connections 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f620c09e410 0x7f620c1a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f620c09d6f0 0x7f620c1a5fc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 --2- 172.21.15.32:0/4282770082 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f620c09e000 0x7f620c1a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 >> 172.21.15.32:0/4282770082 conn(0x7f620c066f50 msgr2=0x7f620c1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 shutdown_connections 2023-09-30T22:30:41.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.253+0000 7f6211689700 1 -- 172.21.15.32:0/4282770082 wait complete. 2023-09-30T22:30:41.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.307+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f54380702e0 con 0x7f54381316c0 2023-09-30T22:30:41.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.308+0000 7f5434ff9700 1 -- 172.21.15.32:0/2808462905 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f54380702e0 con 0x7f54381316c0 2023-09-30T22:30:41.309 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 msgr2=0x7f543812af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 0x7f543812af20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f54381313f0 tx=0x7f542c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 shutdown_connections 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5438136a70 0x7f54380712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f543809d9f0 0x7f543812a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 --2- 172.21.15.32:0/2808462905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f54381316c0 0x7f543812af20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 >> 172.21.15.32:0/2808462905 conn(0x7f5438066f50 msgr2=0x7f543809a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 shutdown_connections 2023-09-30T22:30:41.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.310+0000 7f543e674700 1 -- 172.21.15.32:0/2808462905 wait complete. 2023-09-30T22:30:41.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:41.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.372+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c58071d20 con 0x7f7c5809f650 2023-09-30T22:30:41.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.372+0000 7f7c46ffd700 1 -- 172.21.15.32:0/2272543837 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c50017310 con 0x7f7c5809f650 2023-09-30T22:30:41.373 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 msgr2=0x7f7c581c3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 0x7f7c581c3c40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b330 tx=0x7f7c5000b640 comp rx=0 tx=0).stop 2023-09-30T22:30:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 shutdown_connections 2023-09-30T22:30:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c5809f650 0x7f7c581c3c40 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c5809e930 0x7f7c581a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 --2- 172.21.15.32:0/2272543837 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c5809f240 0x7f7c581a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 >> 172.21.15.32:0/2272543837 conn(0x7f7c580a4210 msgr2=0x7f7c5806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 shutdown_connections 2023-09-30T22:30:41.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.374+0000 7f7c5cf4a700 1 -- 172.21.15.32:0/2272543837 wait complete. 2023-09-30T22:30:41.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:41.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.767+0000 7fabb0bb8700 1 Processor -- start 2023-09-30T22:30:41.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.767+0000 7fabb0bb8700 1 -- start start 2023-09-30T22:30:41.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabb0bb8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 0x7fabac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabb0bb8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 0x7fabac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabb0bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09e410 0x7fabac133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabb0bb8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac06b460 con 0x7fabac09e000 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabb0bb8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac09b120 con 0x7fabac09d6f0 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabb0bb8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabac09a7e0 con 0x7fabac09e410 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 0x7fabac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7faba9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 0x7fabac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09e410 0x7fabac133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 0x7fabac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47812/0 (socket says 172.21.15.32:47812) 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7faba9d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 0x7fabac1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45850/0 (socket says 172.21.15.32:45850) 2023-09-30T22:30:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 -- 172.21.15.32:0/2325535858 learned_addr learned my addr 172.21.15.32:0/2325535858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 -- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09e410 msgr2=0x7fabac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 --2- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09e410 0x7fabac133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 -- 172.21.15.32:0/2325535858 >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 msgr2=0x7fabac1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 --2- 172.21.15.32:0/2325535858 >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 0x7fabac1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 -- 172.21.15.32:0/2325535858 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac09a500 con 0x7fabac09d6f0 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7faba9d9b700 1 --2- 172.21.15.32:0/2325535858 >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 0x7fabac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaad9d700 1 --2- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09e410 0x7fabac133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.768+0000 7fabaa59c700 1 --2- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 0x7fabac09dac0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009920 tx=0x7fab9c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf0e76eda14252e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7faba959a700 1 -- 172.21.15.32:0/2325535858 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011830 con 0x7fabac09d6f0 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7faba959a700 1 -- 172.21.15.32:0/2325535858 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c011990 con 0x7fabac09d6f0 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7faba959a700 1 -- 172.21.15.32:0/2325535858 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011d20 con 0x7fabac09d6f0 2023-09-30T22:30:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 -- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 msgr2=0x7fabac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 0x7fabac09dac0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009920 tx=0x7fab9c009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 -- 172.21.15.32:0/2325535858 shutdown_connections 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09d6f0 0x7fabac09dac0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/2325535858 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09e410 0x7fabac133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/2325535858 >> v2:172.21.15.32:3300/0 conn(0x7fabac09e000 0x7fabac1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 -- 172.21.15.32:0/2325535858 >> 172.21.15.32:0/2325535858 conn(0x7fabac066f50 msgr2=0x7fabac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 -- 172.21.15.32:0/2325535858 shutdown_connections 2023-09-30T22:30:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.769+0000 7fabb0bb8700 1 -- 172.21.15.32:0/2325535858 wait complete. 2023-09-30T22:30:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.771+0000 7fabb0bb8700 1 Processor -- start 2023-09-30T22:30:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.771+0000 7fabb0bb8700 1 -- start start 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabb0bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 0x7fabac1225b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabb0bb8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 0x7fabac122af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabb0bb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabac09e410 0x7fabac123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabb0bb8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac123a10 con 0x7fabac09e410 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabb0bb8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac09c710 con 0x7fabac09e000 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabb0bb8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0a1d00 con 0x7fabac09d6f0 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabaa59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 0x7fabac1225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 0x7fabac122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabaa59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 0x7fabac1225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48122/0 (socket says 172.21.15.32:48122) 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabaa59c700 1 -- 172.21.15.32:0/1988812968 learned_addr learned my addr 172.21.15.32:0/1988812968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 0x7fabac122af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47820/0 (socket says 172.21.15.32:47820) 2023-09-30T22:30:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabaad9d700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabac09e410 0x7fabac123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 -- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 msgr2=0x7fabac1225b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 0x7fabac1225b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 -- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabac09e410 msgr2=0x7fabac123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabac09e410 0x7fabac123030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 -- 172.21.15.32:0/1988812968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab940094a0 con 0x7fabac09e000 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7fabaa59c700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 0x7fabac1225b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.772+0000 7faba9d9b700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 0x7fabac122af0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fab9400e980 tx=0x7fab9400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.773+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400c640 con 0x7fabac09e000 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.773+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9400c7a0 con 0x7fabac09e000 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.773+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba4009a70 con 0x7fabac09e000 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.773+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400cb10 con 0x7fabac09e000 2023-09-30T22:30:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.773+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c009550 con 0x7fabac09e000 2023-09-30T22:30:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.773+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fab9400cc70 con 0x7fabac09e000 2023-09-30T22:30:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.774+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fab9403fa20 con 0x7fabac09e000 2023-09-30T22:30:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.774+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac074ea0 con 0x7fabac09e000 2023-09-30T22:30:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.779+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fab94017020 con 0x7fabac09e000 2023-09-30T22:30:41.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.986+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fabac125c00 con 0x7fabac09e000 2023-09-30T22:30:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.987+0000 7faba2ffd700 1 -- 172.21.15.32:0/1988812968 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab9401a9b0 con 0x7fabac09e000 2023-09-30T22:30:41.988 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 msgr2=0x7fabac122af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 0x7fabac122af0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fab9400e980 tx=0x7fab9400ec90 comp rx=0 tx=0).stop 2023-09-30T22:30:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 shutdown_connections 2023-09-30T22:30:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabac09e410 0x7fabac123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabac09d6f0 0x7fabac1225b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 --2- 172.21.15.32:0/1988812968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabac09e000 0x7fabac122af0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 >> 172.21.15.32:0/1988812968 conn(0x7fabac066f50 msgr2=0x7fabac0a23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 shutdown_connections 2023-09-30T22:30:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:41.989+0000 7fabb0bb8700 1 -- 172.21.15.32:0/1988812968 wait complete. 2023-09-30T22:30:42.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:42.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:42.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:42.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 Processor -- start 2023-09-30T22:30:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 -- start start 2023-09-30T22:30:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb7009e930 0x7feb7009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 0x7feb701317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009f650 0x7feb70131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb70070940 con 0x7feb7009e930 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7006fd60 con 0x7feb7009f650 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.423+0000 7feb76f1a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb7006fa80 con 0x7feb7009f240 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 0x7feb701317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb74cb6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb7009e930 0x7feb7009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb74cb6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb7009e930 0x7feb7009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45874/0 (socket says 172.21.15.32:45874) 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 0x7feb701317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48134/0 (socket says 172.21.15.32:48134) 2023-09-30T22:30:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb74cb6700 1 -- 172.21.15.32:0/3283143181 learned_addr learned my addr 172.21.15.32:0/3283143181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb754b7700 1 --2- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009f650 0x7feb70131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 -- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009f650 msgr2=0x7feb70131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 --2- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009f650 0x7feb70131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 -- 172.21.15.32:0/3283143181 >> v2:172.21.15.32:3300/0 conn(0x7feb7009e930 msgr2=0x7feb7009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 --2- 172.21.15.32:0/3283143181 >> v2:172.21.15.32:3300/0 conn(0x7feb7009e930 0x7feb7009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 -- 172.21.15.32:0/3283143181 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7009fb60 con 0x7feb7009f240 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.424+0000 7feb6ffff700 1 --2- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 0x7feb701317f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009840 tx=0x7feb5c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7d06220c48e254 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.425+0000 7feb6f7fe700 1 -- 172.21.15.32:0/3283143181 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c011830 con 0x7feb7009f240 2023-09-30T22:30:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.425+0000 7feb6f7fe700 1 -- 172.21.15.32:0/3283143181 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c011990 con 0x7feb7009f240 2023-09-30T22:30:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.425+0000 7feb6f7fe700 1 -- 172.21.15.32:0/3283143181 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c011d20 con 0x7feb7009f240 2023-09-30T22:30:42.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 -- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 msgr2=0x7feb701317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 0x7feb701317f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009840 tx=0x7feb5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 -- 172.21.15.32:0/3283143181 shutdown_connections 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009f650 0x7feb70131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3283143181 >> v2:172.21.15.32:3300/0 conn(0x7feb7009e930 0x7feb7009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3283143181 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f240 0x7feb701317f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 -- 172.21.15.32:0/3283143181 >> 172.21.15.32:0/3283143181 conn(0x7feb700a4210 msgr2=0x7feb700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 -- 172.21.15.32:0/3283143181 shutdown_connections 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.426+0000 7feb76f1a700 1 -- 172.21.15.32:0/3283143181 wait complete. 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.427+0000 7feb76f1a700 1 Processor -- start 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.427+0000 7feb76f1a700 1 -- start start 2023-09-30T22:30:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.427+0000 7feb76f1a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009e930 0x7feb7006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.427+0000 7feb76f1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 0x7feb7006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb76f1a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f650 0x7feb7006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb76f1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb701a9500 con 0x7feb7009f240 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb76f1a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7006cce0 con 0x7feb7009e930 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb76f1a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb7006d340 con 0x7feb7009f650 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 0x7feb7006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 0x7feb7006bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45880/0 (socket says 172.21.15.32:45880) 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 -- 172.21.15.32:0/3346422405 learned_addr learned my addr 172.21.15.32:0/3346422405 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb74cb6700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009e930 0x7feb7006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb754b7700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f650 0x7feb7006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 -- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f650 msgr2=0x7feb7006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f650 0x7feb7006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 -- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009e930 msgr2=0x7feb7006b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009e930 0x7feb7006b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 -- 172.21.15.32:0/3346422405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0094a0 con 0x7feb7009f240 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6ffff700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 0x7feb7006bdc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000c00 tx=0x7feb5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.428+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c004450 con 0x7feb7009f240 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.429+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c01f070 con 0x7feb7009f240 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.429+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb700a0190 con 0x7feb7009f240 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.429+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0038e0 con 0x7feb7009f240 2023-09-30T22:30:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.429+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb70071d20 con 0x7feb7009f240 2023-09-30T22:30:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.430+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feb5c003a40 con 0x7feb7009f240 2023-09-30T22:30:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.430+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feb5c021070 con 0x7feb7009f240 2023-09-30T22:30:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.430+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7006d530 con 0x7feb7009f240 2023-09-30T22:30:42.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.439+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feb5c01a430 con 0x7feb7009f240 2023-09-30T22:30:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.474+0000 7f0b4c08e700 1 Processor -- start 2023-09-30T22:30:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4c08e700 1 -- start start 2023-09-30T22:30:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4c08e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b440a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4c08e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b4413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4c08e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b440a2a70 con 0x7f0b440a0a30 2023-09-30T22:30:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4c08e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440a01d0 con 0x7f0b4409d630 2023-09-30T22:30:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4c08e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4409e2b0 con 0x7f0b44138d80 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b49e2a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b440a0a30 0x7f0b440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48154/0 (socket says 172.21.15.32:48154) 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b49e2a700 1 -- 172.21.15.32:0/2738482685 learned_addr learned my addr 172.21.15.32:0/2738482685 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.475+0000 7f0b4a62b700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b4413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b49629700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b440a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b4a62b700 1 -- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 msgr2=0x7f0b440a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b48e28700 1 -- 172.21.15.32:0/2738482685 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598350512 0 0) 0x7f0b440a2a70 con 0x7f0b440a0a30 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b4a62b700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b440a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b4a62b700 1 -- 172.21.15.32:0/2738482685 >> v1:172.21.15.32:6789/0 conn(0x7f0b440a0a30 legacy=0x7f0b440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b4a62b700 1 -- 172.21.15.32:0/2738482685 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4409dc50 con 0x7f0b44138d80 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b4a62b700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b4413b160 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009dc0 tx=0x7f0b3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474ec87bed5eb796 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b48e28700 1 -- 172.21.15.32:0/2738482685 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00e070 con 0x7f0b44138d80 2023-09-30T22:30:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b48e28700 1 -- 172.21.15.32:0/2738482685 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c00bb20 con 0x7f0b44138d80 2023-09-30T22:30:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.476+0000 7f0b48e28700 1 -- 172.21.15.32:0/2738482685 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c012430 con 0x7f0b44138d80 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 msgr2=0x7f0b4413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b4413b160 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009dc0 tx=0x7f0b3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2738482685 shutdown_connections 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b4413b160 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2738482685 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b440a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2738482685 >> 172.21.15.32:0/2738482685 conn(0x7f0b440a3050 msgr2=0x7f0b440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2738482685 shutdown_connections 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.477+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2738482685 wait complete. 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 Processor -- start 2023-09-30T22:30:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 -- start start 2023-09-30T22:30:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b44133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 0x7f0b44133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b441341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441371f0 con 0x7f0b440a0a30 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441360e0 con 0x7f0b4409d630 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b4c08e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441365a0 con 0x7f0b44138d80 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.479+0000 7f0b49629700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 0x7f0b44133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 0x7f0b44133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45892/0 (socket says 172.21.15.32:45892) 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 -- 172.21.15.32:0/2274483593 learned_addr learned my addr 172.21.15.32:0/2274483593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49e2a700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b44133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b4a62b700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b441341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 -- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 msgr2=0x7f0b441341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b441341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 -- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 msgr2=0x7f0b44133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b44133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 -- 172.21.15.32:0/2274483593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c009a70 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b49629700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 0x7f0b44133ca0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400e9b0 tx=0x7f0b3400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34009710 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400ee50 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400ca50 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.480+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b440a4b20 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.481+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b44137980 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.482+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b440747f0 con 0x7f0b440a0a30 2023-09-30T22:30:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.482+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0b3400cbb0 con 0x7f0b440a0a30 2023-09-30T22:30:42.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.482+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0b3403fee0 con 0x7f0b440a0a30 2023-09-30T22:30:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.487+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0b34011350 con 0x7f0b440a0a30 2023-09-30T22:30:42.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 Processor -- start 2023-09-30T22:30:42.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 -- start start 2023-09-30T22:30:42.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56340a0a30 0x7f56340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 0x7f5634133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563409edd0 con 0x7f5634133fa0 2023-09-30T22:30:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563409e1f0 con 0x7f56340a0a30 2023-09-30T22:30:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.562+0000 7f563ab35700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563409dc50 con 0x7f56340a1340 2023-09-30T22:30:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 0x7f5634133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f56388d1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56340a0a30 0x7f56340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 0x7f5634133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47872/0 (socket says 172.21.15.32:47872) 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 -- 172.21.15.32:0/3284056437 learned_addr learned my addr 172.21.15.32:0/3284056437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 -- 172.21.15.32:0/3284056437 >> v2:172.21.15.32:3300/0 conn(0x7f56340a0a30 msgr2=0x7f56340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 --2- 172.21.15.32:0/3284056437 >> v2:172.21.15.32:3300/0 conn(0x7f56340a0a30 0x7f56340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 -- 172.21.15.32:0/3284056437 >> v1:172.21.15.32:6789/0 conn(0x7f5634133fa0 legacy=0x7f563413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 -- 172.21.15.32:0/3284056437 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340a1fe0 con 0x7f56340a1340 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f56388d1700 1 --2- 172.21.15.32:0/3284056437 >> v2:172.21.15.32:3300/0 conn(0x7f56340a0a30 0x7f56340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f5633fff700 1 --2- 172.21.15.32:0/3284056437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 0x7f5634133a60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f56200098f0 tx=0x7f5620009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6674540e24a52762 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.563+0000 7f56337fe700 1 -- 172.21.15.32:0/3284056437 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620011830 con 0x7f56340a1340 2023-09-30T22:30:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f56337fe700 1 -- 172.21.15.32:0/3284056437 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620011990 con 0x7f56340a1340 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f56337fe700 1 -- 172.21.15.32:0/3284056437 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620011d20 con 0x7f56340a1340 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 -- 172.21.15.32:0/3284056437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 msgr2=0x7f5634133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 --2- 172.21.15.32:0/3284056437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 0x7f5634133a60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f56200098f0 tx=0x7f5620009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 -- 172.21.15.32:0/3284056437 shutdown_connections 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 --2- 172.21.15.32:0/3284056437 >> v2:172.21.15.32:3300/0 conn(0x7f56340a0a30 0x7f56340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 --2- 172.21.15.32:0/3284056437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f56340a1340 0x7f5634133a60 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 -- 172.21.15.32:0/3284056437 >> 172.21.15.32:0/3284056437 conn(0x7f56340a6550 msgr2=0x7f56340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 -- 172.21.15.32:0/3284056437 shutdown_connections 2023-09-30T22:30:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.564+0000 7f563ab35700 1 -- 172.21.15.32:0/3284056437 wait complete. 2023-09-30T22:30:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.565+0000 7f563ab35700 1 Processor -- start 2023-09-30T22:30:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.565+0000 7f563ab35700 1 -- start start 2023-09-30T22:30:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f563ab35700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 0x7f563411e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f563ab35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56340a1340 0x7f563411ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f563ab35700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5634133fa0 0x7f563411f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f563ab35700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634120bc0 con 0x7f56340a1340 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f563ab35700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340a3760 con 0x7f5634133fa0 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f563ab35700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56340a0220 con 0x7f56340a0a30 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 0x7f563411e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56390d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5634133fa0 0x7f563411f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 0x7f563411e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48162/0 (socket says 172.21.15.32:48162) 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 -- 172.21.15.32:0/351406818 learned_addr learned my addr 172.21.15.32:0/351406818 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f5633fff700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56340a1340 0x7f563411ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 -- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5634133fa0 msgr2=0x7f563411f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5634133fa0 0x7f563411f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 -- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56340a1340 msgr2=0x7f563411ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.566+0000 7f56388d1700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56340a1340 0x7f563411ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f56388d1700 1 -- 172.21.15.32:0/351406818 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56280094a0 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f56390d2700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5634133fa0 0x7f563411f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f56388d1700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 0x7f563411e750 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f562800ebe0 tx=0x7f562800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562800c850 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562800c9b0 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5620009550 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562800cd40 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.567+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5634122a00 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.568+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5628013470 con 0x7f56340a0a30 2023-09-30T22:30:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.568+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f562803f770 con 0x7f56340a0a30 2023-09-30T22:30:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.568+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5634074ea0 con 0x7f56340a0a30 2023-09-30T22:30:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.573+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5628013720 con 0x7f56340a0a30 2023-09-30T22:30:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.695+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0b44072980 con 0x7f0b440a0a30 2023-09-30T22:30:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.695+0000 7f0b3a7fc700 1 -- 172.21.15.32:0/2274483593 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0b34017070 con 0x7f0b440a0a30 2023-09-30T22:30:42.696 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:42.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.697+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 msgr2=0x7f0b44133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.697+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 0x7f0b44133ca0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400e9b0 tx=0x7f0b3400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.697+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 shutdown_connections 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.697+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0b44138d80 0x7f0b441341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.697+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0b4409d630 0x7f0b44133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.697+0000 7f0b4c08e700 1 --2- 172.21.15.32:0/2274483593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b440a0a30 0x7f0b44133ca0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.698+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 >> 172.21.15.32:0/2274483593 conn(0x7f0b440a3050 msgr2=0x7f0b4409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.698+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 shutdown_connections 2023-09-30T22:30:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.698+0000 7f0b4c08e700 1 -- 172.21.15.32:0/2274483593 wait complete. 2023-09-30T22:30:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.699+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feb701a0b30 con 0x7feb7009f240 2023-09-30T22:30:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.699+0000 7feb6d7fa700 1 -- 172.21.15.32:0/3346422405 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb5c015030 con 0x7feb7009f240 2023-09-30T22:30:42.700 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 msgr2=0x7feb7006bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 0x7feb7006bdc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000c00 tx=0x7feb5c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 shutdown_connections 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feb7009f650 0x7feb7006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb7009e930 0x7feb7006b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 --2- 172.21.15.32:0/3346422405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb7009f240 0x7feb7006bdc0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 >> 172.21.15.32:0/3346422405 conn(0x7feb700a4210 msgr2=0x7feb701b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.701+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 shutdown_connections 2023-09-30T22:30:42.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.702+0000 7feb76f1a700 1 -- 172.21.15.32:0/3346422405 wait complete. 2023-09-30T22:30:42.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:42.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:42.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.795+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5634121e40 con 0x7f56340a0a30 2023-09-30T22:30:42.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.795+0000 7f56317fa700 1 -- 172.21.15.32:0/351406818 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5628021030 con 0x7f56340a0a30 2023-09-30T22:30:42.796 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 msgr2=0x7f563411e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 0x7f563411e750 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f562800ebe0 tx=0x7f562800eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 shutdown_connections 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5634133fa0 0x7f563411f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f56340a0a30 0x7f563411e750 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 --2- 172.21.15.32:0/351406818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56340a1340 0x7f563411ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 >> 172.21.15.32:0/351406818 conn(0x7f56340a6550 msgr2=0x7f56340a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 shutdown_connections 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:42.798+0000 7f563ab35700 1 -- 172.21.15.32:0/351406818 wait complete. 2023-09-30T22:30:42.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:43.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.151+0000 7ffb37653700 1 Processor -- start 2023-09-30T22:30:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb37653700 1 -- start start 2023-09-30T22:30:43.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb37653700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30133420 0x7ffb301337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb37653700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb37653700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3006b990 con 0x7ffb30134260 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb37653700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300a0590 con 0x7ffb30133420 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb37653700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3009fb80 con 0x7ffb30133e30 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb34bee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb353ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30133420 0x7ffb301337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb35bf0700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb30134260 0x7ffb301304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48166/0 (socket says 172.21.15.32:48166) 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.152+0000 7ffb35bf0700 1 -- 172.21.15.32:0/4224857621 learned_addr learned my addr 172.21.15.32:0/4224857621 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb27fff700 1 -- 172.21.15.32:0/4224857621 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952086387 0 0) 0x7ffb3006b990 con 0x7ffb30134260 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb27fff700 1 -- 172.21.15.32:0/4224857621 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1c0035b0 con 0x7ffb30134260 2023-09-30T22:30:43.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb34bee700 1 -- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30133420 msgr2=0x7ffb301337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb34bee700 1 --2- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30133420 0x7ffb301337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb34bee700 1 -- 172.21.15.32:0/4224857621 >> v1:172.21.15.32:6789/0 conn(0x7ffb30134260 legacy=0x7ffb301304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb34bee700 1 -- 172.21.15.32:0/4224857621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb300a0030 con 0x7ffb30133e30 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb353ef700 1 --2- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30133420 0x7ffb301337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb34bee700 1 --2- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3012ff90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009870 tx=0x7ffb20009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e07c5aa08677e5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb27fff700 1 -- 172.21.15.32:0/4224857621 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20011830 con 0x7ffb30133e30 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb27fff700 1 -- 172.21.15.32:0/4224857621 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20011990 con 0x7ffb30133e30 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.153+0000 7ffb27fff700 1 -- 172.21.15.32:0/4224857621 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20011d20 con 0x7ffb30133e30 2023-09-30T22:30:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 -- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 msgr2=0x7ffb3012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 --2- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3012ff90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009870 tx=0x7ffb20009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 -- 172.21.15.32:0/4224857621 shutdown_connections 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 --2- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30133420 0x7ffb301337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 --2- 172.21.15.32:0/4224857621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3012ff90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 -- 172.21.15.32:0/4224857621 >> 172.21.15.32:0/4224857621 conn(0x7ffb300a4210 msgr2=0x7ffb300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 -- 172.21.15.32:0/4224857621 shutdown_connections 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.154+0000 7ffb37653700 1 -- 172.21.15.32:0/4224857621 wait complete. 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.155+0000 7ffb37653700 1 Processor -- start 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.155+0000 7ffb37653700 1 -- start start 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb37653700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb30133420 0x7ffb3009c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb37653700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3009c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb37653700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30134260 0x7ffb3009ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb37653700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3009d690 con 0x7ffb30133420 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb37653700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3012da80 con 0x7ffb30134260 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb37653700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3011fc10 con 0x7ffb30133e30 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3009c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb353ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb30133420 0x7ffb3009c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3009c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48190/0 (socket says 172.21.15.32:48190) 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 -- 172.21.15.32:0/3505097122 learned_addr learned my addr 172.21.15.32:0/3505097122 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb35bf0700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30134260 0x7ffb3009ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 -- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30134260 msgr2=0x7ffb3009ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30134260 0x7ffb3009ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 -- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb30133420 msgr2=0x7ffb3009c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb30133420 0x7ffb3009c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 -- 172.21.15.32:0/3505097122 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0094a0 con 0x7ffb30133e30 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb35bf0700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30134260 0x7ffb3009ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb353ef700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb30133420 0x7ffb3009c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb34bee700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3009c770 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009bc0 tx=0x7ffb20004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20004510 con 0x7ffb30133e30 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.156+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2001f070 con 0x7ffb30133e30 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.157+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb200094a0 con 0x7ffb30133e30 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.157+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200038e0 con 0x7ffb30133e30 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.157+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3011f560 con 0x7ffb30133e30 2023-09-30T22:30:43.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.157+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb20003a40 con 0x7ffb30133e30 2023-09-30T22:30:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.158+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ffb20021070 con 0x7ffb30133e30 2023-09-30T22:30:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.158+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb30132f90 con 0x7ffb30133e30 2023-09-30T22:30:43.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.163+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb20015020 con 0x7ffb30133e30 2023-09-30T22:30:43.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.367+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffb30074ea0 con 0x7ffb30133e30 2023-09-30T22:30:43.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.368+0000 7ffb25ffb700 1 -- 172.21.15.32:0/3505097122 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb20010380 con 0x7ffb30133e30 2023-09-30T22:30:43.369 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 msgr2=0x7ffb3009c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3009c770 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009bc0 tx=0x7ffb20004750 comp rx=0 tx=0).stop 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 shutdown_connections 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffb30134260 0x7ffb3009ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb30133420 0x7ffb3009c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 --2- 172.21.15.32:0/3505097122 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ffb30133e30 0x7ffb3009c770 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 >> 172.21.15.32:0/3505097122 conn(0x7ffb300a4210 msgr2=0x7ffb30130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 shutdown_connections 2023-09-30T22:30:43.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.370+0000 7ffb37653700 1 -- 172.21.15.32:0/3505097122 wait complete. 2023-09-30T22:30:43.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:43.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:43.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:43.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.864+0000 7fefb62cb700 1 Processor -- start 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.864+0000 7fefb62cb700 1 -- start start 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb62cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 0x7fefb009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb62cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 0x7fefb0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb62cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e410 0x7fefb0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb62cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb009f520 con 0x7fefb009e000 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb62cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb009f130 con 0x7fefb009d6f0 2023-09-30T22:30:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb62cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefb009e930 con 0x7fefb009e410 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 0x7fefb0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 0x7fefb009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 0x7fefb009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47904/0 (socket says 172.21.15.32:47904) 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 0x7fefb0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45950/0 (socket says 172.21.15.32:45950) 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 -- 172.21.15.32:0/1478725910 learned_addr learned my addr 172.21.15.32:0/1478725910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefb4868700 1 --2- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e410 0x7fefb0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 -- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e410 msgr2=0x7fefb0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 --2- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e410 0x7fefb0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 -- 172.21.15.32:0/1478725910 >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 msgr2=0x7fefb0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 --2- 172.21.15.32:0/1478725910 >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 0x7fefb0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaffff700 1 -- 172.21.15.32:0/1478725910 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb009b2d0 con 0x7fefb009d6f0 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.865+0000 7fefaf7fe700 1 --2- 172.21.15.32:0/1478725910 >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 0x7fefb0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.866+0000 7fefaffff700 1 --2- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 0x7fefb009dac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fefa00098f0 tx=0x7fefa0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455b720857ca5c0d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.866+0000 7fefaeffd700 1 -- 172.21.15.32:0/1478725910 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011830 con 0x7fefb009d6f0 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.866+0000 7fefaeffd700 1 -- 172.21.15.32:0/1478725910 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0011990 con 0x7fefb009d6f0 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.866+0000 7fefaeffd700 1 -- 172.21.15.32:0/1478725910 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011d20 con 0x7fefb009d6f0 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.866+0000 7fefb62cb700 1 -- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 msgr2=0x7fefb009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.866+0000 7fefb62cb700 1 --2- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 0x7fefb009dac0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fefa00098f0 tx=0x7fefa0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 -- 172.21.15.32:0/1478725910 shutdown_connections 2023-09-30T22:30:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 --2- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009d6f0 0x7fefb009dac0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 --2- 172.21.15.32:0/1478725910 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009e410 0x7fefb0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 --2- 172.21.15.32:0/1478725910 >> v2:172.21.15.32:3300/0 conn(0x7fefb009e000 0x7fefb0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 -- 172.21.15.32:0/1478725910 >> 172.21.15.32:0/1478725910 conn(0x7fefb0066f50 msgr2=0x7fefb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 -- 172.21.15.32:0/1478725910 shutdown_connections 2023-09-30T22:30:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.867+0000 7fefb62cb700 1 -- 172.21.15.32:0/1478725910 wait complete. 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.868+0000 7fefb62cb700 1 Processor -- start 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.868+0000 7fefb62cb700 1 -- start start 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb62cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb62cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e000 0x7fefb011e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb62cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009e410 0x7fefb011e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb62cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb011fad0 con 0x7fefb009e000 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb62cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0120d20 con 0x7fefb009e410 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb62cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01200b0 con 0x7fefb009d6f0 2023-09-30T22:30:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefaffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefaffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48206/0 (socket says 172.21.15.32:48206) 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefaffff700 1 -- 172.21.15.32:0/804139108 learned_addr learned my addr 172.21.15.32:0/804139108 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefaf7fe700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e000 0x7fefb011e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.869+0000 7fefb4868700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009e410 0x7fefb011e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaffff700 1 -- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009e410 msgr2=0x7fefb011e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaffff700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009e410 0x7fefb011e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaffff700 1 -- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e000 msgr2=0x7fefb011e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaffff700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e000 0x7fefb011e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaffff700 1 -- 172.21.15.32:0/804139108 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa400d940 con 0x7fefb009d6f0 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaf7fe700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e000 0x7fefb011e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefaffff700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011dec0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009f50 tx=0x7fefa0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0004240 con 0x7fefb009d6f0 2023-09-30T22:30:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa001f070 con 0x7fefb009d6f0 2023-09-30T22:30:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00045a0 con 0x7fefb009d6f0 2023-09-30T22:30:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa0009550 con 0x7fefb009d6f0 2023-09-30T22:30:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.870+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb00a1600 con 0x7fefb009d6f0 2023-09-30T22:30:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.871+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefa0021030 con 0x7fefb009d6f0 2023-09-30T22:30:43.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.871+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb009c760 con 0x7fefb009d6f0 2023-09-30T22:30:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.872+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fefa0040790 con 0x7fefb009d6f0 2023-09-30T22:30:43.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.877+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefa00237e0 con 0x7fefb009d6f0 2023-09-30T22:30:43.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.908+0000 7f271a4d6700 1 Processor -- start 2023-09-30T22:30:43.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.908+0000 7f271a4d6700 1 -- start start 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f271a4d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409d6f0 0x7f271409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f271a4d6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f271409e410 0x7f2714130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f271a4d6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271409f130 con 0x7f271409e000 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f271a4d6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271409f520 con 0x7f271409e410 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f271a4d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f271409e930 con 0x7f271409d6f0 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2718a73700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f271409e410 0x7f2714130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f27137fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f271409e000 0x7f2714130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48172/0 (socket says 172.21.15.32:48172) 2023-09-30T22:30:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f27137fe700 1 -- 172.21.15.32:0/3183642760 learned_addr learned my addr 172.21.15.32:0/3183642760 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2713fff700 1 --2- 172.21.15.32:0/3183642760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409d6f0 0x7f271409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2712ffd700 1 -- 172.21.15.32:0/3183642760 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905895370 0 0) 0x7f271409f130 con 0x7f271409e000 2023-09-30T22:30:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2713fff700 1 -- 172.21.15.32:0/3183642760 >> v2:172.21.15.32:3300/0 conn(0x7f271409e410 msgr2=0x7f2714130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2713fff700 1 --2- 172.21.15.32:0/3183642760 >> v2:172.21.15.32:3300/0 conn(0x7f271409e410 0x7f2714130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2713fff700 1 -- 172.21.15.32:0/3183642760 >> v1:172.21.15.32:6789/0 conn(0x7f271409e000 legacy=0x7f2714130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.909+0000 7f2713fff700 1 -- 172.21.15.32:0/3183642760 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271409b2d0 con 0x7f271409d6f0 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f2713fff700 1 --2- 172.21.15.32:0/3183642760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409d6f0 0x7f271409dac0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2704009840 tx=0x7f2704009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7bbd4461a08d49 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f2712ffd700 1 -- 172.21.15.32:0/3183642760 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704011830 con 0x7f271409d6f0 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f2712ffd700 1 -- 172.21.15.32:0/3183642760 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704011990 con 0x7f271409d6f0 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f2712ffd700 1 -- 172.21.15.32:0/3183642760 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704011d20 con 0x7f271409d6f0 2023-09-30T22:30:43.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f271a4d6700 1 -- 172.21.15.32:0/3183642760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409d6f0 msgr2=0x7f271409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f271a4d6700 1 --2- 172.21.15.32:0/3183642760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409d6f0 0x7f271409dac0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2704009840 tx=0x7f2704009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.910+0000 7f271a4d6700 1 -- 172.21.15.32:0/3183642760 shutdown_connections 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.911+0000 7f271a4d6700 1 --2- 172.21.15.32:0/3183642760 >> v2:172.21.15.32:3300/0 conn(0x7f271409e410 0x7f2714130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.911+0000 7f271a4d6700 1 --2- 172.21.15.32:0/3183642760 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409d6f0 0x7f271409dac0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.911+0000 7f271a4d6700 1 -- 172.21.15.32:0/3183642760 >> 172.21.15.32:0/3183642760 conn(0x7f2714066f50 msgr2=0x7f271409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.911+0000 7f271a4d6700 1 -- 172.21.15.32:0/3183642760 shutdown_connections 2023-09-30T22:30:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.911+0000 7f271a4d6700 1 -- 172.21.15.32:0/3183642760 wait complete. 2023-09-30T22:30:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.912+0000 7f271a4d6700 1 Processor -- start 2023-09-30T22:30:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.912+0000 7f271a4d6700 1 -- start start 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f271a4d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 0x7f2714121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f271a4d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409e000 0x7f2714121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f271a4d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f271409e410 0x7f2714121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f271a4d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2714122310 con 0x7f271409d6f0 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f271a4d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271406e840 con 0x7f271409e410 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f271a4d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2714123bd0 con 0x7f271409e000 2023-09-30T22:30:43.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f2713fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 0x7f2714121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f2713fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 0x7f2714121220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45994/0 (socket says 172.21.15.32:45994) 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f27137fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409e000 0x7f2714121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f2713fff700 1 -- 172.21.15.32:0/4058857724 learned_addr learned my addr 172.21.15.32:0/4058857724 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.913+0000 7f2718a73700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f271409e410 0x7f2714121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2713fff700 1 -- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409e000 msgr2=0x7f2714121760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2713fff700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409e000 0x7f2714121760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2713fff700 1 -- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f271409e410 msgr2=0x7f2714121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2713fff700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f271409e410 0x7f2714121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2713fff700 1 -- 172.21.15.32:0/4058857724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040094a0 con 0x7f271409d6f0 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2713fff700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 0x7f2714121220 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2704009b90 tx=0x7f2704004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27040036a0 con 0x7f271409d6f0 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270401e070 con 0x7f271409d6f0 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271409c700 con 0x7f271409d6f0 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704003a60 con 0x7f271409d6f0 2023-09-30T22:30:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.914+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27141224b0 con 0x7f271409d6f0 2023-09-30T22:30:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.915+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2704003bc0 con 0x7f271409d6f0 2023-09-30T22:30:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.915+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2714123750 con 0x7f271409d6f0 2023-09-30T22:30:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.915+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2704040550 con 0x7f271409d6f0 2023-09-30T22:30:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.920+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2714123750 con 0x7f271409d6f0 2023-09-30T22:30:43.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.961+0000 7f17f4982700 1 Processor -- start 2023-09-30T22:30:43.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.962+0000 7f17f4982700 1 -- start start 2023-09-30T22:30:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17f4982700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17f009d630 0x7f17f00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17f4982700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f0138d80 0x7f17f013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17f4982700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f00a2a70 con 0x7f17f00a0a30 2023-09-30T22:30:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17f4982700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f00a01d0 con 0x7f17f009d630 2023-09-30T22:30:43.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17f4982700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17f009e2b0 con 0x7f17f0138d80 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17f009d630 0x7f17f00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17f009d630 0x7f17f00a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46008/0 (socket says 172.21.15.32:46008) 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 -- 172.21.15.32:0/893552230 learned_addr learned my addr 172.21.15.32:0/893552230 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17ee59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17f00a0a30 0x7f17f00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48174/0 (socket says 172.21.15.32:48174) 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17eed9d700 1 --2- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f0138d80 0x7f17f013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 -- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f0138d80 msgr2=0x7f17f013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 --2- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f0138d80 0x7f17f013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 -- 172.21.15.32:0/893552230 >> v1:172.21.15.32:6789/0 conn(0x7f17f00a0a30 legacy=0x7f17f00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 -- 172.21.15.32:0/893552230 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f009dc50 con 0x7f17f009d630 2023-09-30T22:30:43.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.963+0000 7f17edd9b700 1 --2- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f009d630 0x7f17f00a1510 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f17e40098f0 tx=0x7f17e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb87ffcf2d3aa291 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17ed59a700 1 -- 172.21.15.32:0/893552230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4011830 con 0x7f17f009d630 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17ed59a700 1 -- 172.21.15.32:0/893552230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e4011990 con 0x7f17f009d630 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17ed59a700 1 -- 172.21.15.32:0/893552230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4011d20 con 0x7f17f009d630 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 -- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f009d630 msgr2=0x7f17f00a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 --2- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f009d630 0x7f17f00a1510 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f17e40098f0 tx=0x7f17e4009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 -- 172.21.15.32:0/893552230 shutdown_connections 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 --2- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f0138d80 0x7f17f013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 --2- 172.21.15.32:0/893552230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f009d630 0x7f17f00a1510 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 -- 172.21.15.32:0/893552230 >> 172.21.15.32:0/893552230 conn(0x7f17f00a3050 msgr2=0x7f17f00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.964+0000 7f17f4982700 1 -- 172.21.15.32:0/893552230 shutdown_connections 2023-09-30T22:30:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.965+0000 7f17f4982700 1 -- 172.21.15.32:0/893552230 wait complete. 2023-09-30T22:30:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.966+0000 7f17f4982700 1 Processor -- start 2023-09-30T22:30:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.966+0000 7f17f4982700 1 -- start start 2023-09-30T22:30:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17f4982700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 0x7f17f0133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17f4982700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f00a0a30 0x7f17f0133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17f4982700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f0138d80 0x7f17f01342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17f4982700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f0134d10 con 0x7f17f0138d80 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17f4982700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f0138020 con 0x7f17f009d630 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17f4982700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17f0137470 con 0x7f17f00a0a30 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 0x7f17f0133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 0x7f17f0133840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47948/0 (socket says 172.21.15.32:47948) 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 -- 172.21.15.32:0/720656636 learned_addr learned my addr 172.21.15.32:0/720656636 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17eed9d700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f0138d80 0x7f17f01342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17edd9b700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f00a0a30 0x7f17f0133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 -- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f00a0a30 msgr2=0x7f17f0133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f00a0a30 0x7f17f0133d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 -- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f0138d80 msgr2=0x7f17f01342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f0138d80 0x7f17f01342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.967+0000 7f17ee59c700 1 -- 172.21.15.32:0/720656636 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4009550 con 0x7f17f009d630 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.968+0000 7f17ee59c700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 0x7f17f0133840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f17e000c580 tx=0x7f17e000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.968+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0005cb0 con 0x7f17f009d630 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.968+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e0005e10 con 0x7f17f009d630 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.968+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f00a0410 con 0x7f17f009d630 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.968+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e00072d0 con 0x7f17f009d630 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.968+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f0137080 con 0x7f17f009d630 2023-09-30T22:30:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.969+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17e0007430 con 0x7f17f009d630 2023-09-30T22:30:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.969+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f17e003ead0 con 0x7f17f009d630 2023-09-30T22:30:43.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.969+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f00747f0 con 0x7f17f009d630 2023-09-30T22:30:43.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:43.974+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17e0017030 con 0x7f17f009d630 2023-09-30T22:30:44.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.086+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fefb011ac20 con 0x7fefb009d6f0 2023-09-30T22:30:44.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.086+0000 7fefacff9700 1 -- 172.21.15.32:0/804139108 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefa0015070 con 0x7fefb009d6f0 2023-09-30T22:30:44.088 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 msgr2=0x7fefb011dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011dec0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009f50 tx=0x7fefa0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 shutdown_connections 2023-09-30T22:30:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefb009e410 0x7fefb011e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefb009d6f0 0x7fefb011dec0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 --2- 172.21.15.32:0/804139108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefb009e000 0x7fefb011e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 >> 172.21.15.32:0/804139108 conn(0x7fefb0066f50 msgr2=0x7fefb0130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 shutdown_connections 2023-09-30T22:30:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.090+0000 7fefb62cb700 1 -- 172.21.15.32:0/804139108 wait complete. 2023-09-30T22:30:44.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.126+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2714123750 con 0x7f271409d6f0 2023-09-30T22:30:44.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.127+0000 7f2710ff9700 1 -- 172.21.15.32:0/4058857724 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2714123750 con 0x7f271409d6f0 2023-09-30T22:30:44.128 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:44.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 msgr2=0x7f2714121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 0x7f2714121220 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2704009b90 tx=0x7f2704004750 comp rx=0 tx=0).stop 2023-09-30T22:30:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 shutdown_connections 2023-09-30T22:30:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f271409e410 0x7f2714121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f271409d6f0 0x7f2714121220 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 --2- 172.21.15.32:0/4058857724 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f271409e000 0x7f2714121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 >> 172.21.15.32:0/4058857724 conn(0x7f2714066f50 msgr2=0x7f2714130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 shutdown_connections 2023-09-30T22:30:44.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.129+0000 7f271a4d6700 1 -- 172.21.15.32:0/4058857724 wait complete. 2023-09-30T22:30:44.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.186+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f17f0135c40 con 0x7f17f009d630 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.187+0000 7f17deffd700 1 -- 172.21.15.32:0/720656636 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17e003ce60 con 0x7f17f009d630 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 msgr2=0x7f17f0133840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 0x7f17f0133840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f17e000c580 tx=0x7f17e000c890 comp rx=0 tx=0).stop 2023-09-30T22:30:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 shutdown_connections 2023-09-30T22:30:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17f0138d80 0x7f17f01342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f17f009d630 0x7f17f0133840 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 --2- 172.21.15.32:0/720656636 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f17f00a0a30 0x7f17f0133d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 >> 172.21.15.32:0/720656636 conn(0x7f17f00a3050 msgr2=0x7f17f009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 shutdown_connections 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.189+0000 7f17f4982700 1 -- 172.21.15.32:0/720656636 wait complete. 2023-09-30T22:30:44.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:44.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:44.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.531+0000 7fdf0333d700 1 Processor -- start 2023-09-30T22:30:44.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.531+0000 7fdf0333d700 1 -- start start 2023-09-30T22:30:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf0333d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdefc09e930 0x7fdefc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf0333d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 0x7fdefc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf0333d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f650 0x7fdefc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf0333d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc070940 con 0x7fdefc09e930 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf0333d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc06fd60 con 0x7fdefc09f240 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf0333d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc06fa80 con 0x7fdefc09f650 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdefc09e930 0x7fdefc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf008d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 0x7fdefc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf018da700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f650 0x7fdefc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdefc09e930 0x7fdefc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46022/0 (socket says 172.21.15.32:46022) 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf008d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 0x7fdefc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:47962/0 (socket says 172.21.15.32:47962) 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 -- 172.21.15.32:0/3283479833 learned_addr learned my addr 172.21.15.32:0/3283479833 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 -- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f650 msgr2=0x7fdefc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f650 0x7fdefc131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 -- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 msgr2=0x7fdefc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 0x7fdefc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 -- 172.21.15.32:0/3283479833 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc09fb60 con 0x7fdefc09e930 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf018da700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f650 0x7fdefc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf008d8700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 0x7fdefc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.532+0000 7fdf010d9700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc09ed00 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fdef80098f0 tx=0x7fdef8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1190bcae2463e7f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdef3fff700 1 -- 172.21.15.32:0/3283479833 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8011830 con 0x7fdefc09e930 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdef3fff700 1 -- 172.21.15.32:0/3283479833 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8011990 con 0x7fdefc09e930 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdef3fff700 1 -- 172.21.15.32:0/3283479833 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8011d20 con 0x7fdefc09e930 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 -- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 msgr2=0x7fdefc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc09ed00 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fdef80098f0 tx=0x7fdef8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 -- 172.21.15.32:0/3283479833 shutdown_connections 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f650 0x7fdefc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc09ed00 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 --2- 172.21.15.32:0/3283479833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f240 0x7fdefc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 -- 172.21.15.32:0/3283479833 >> 172.21.15.32:0/3283479833 conn(0x7fdefc0a4210 msgr2=0x7fdefc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 -- 172.21.15.32:0/3283479833 shutdown_connections 2023-09-30T22:30:44.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.533+0000 7fdf0333d700 1 -- 172.21.15.32:0/3283479833 wait complete. 2023-09-30T22:30:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.534+0000 7fdf0333d700 1 Processor -- start 2023-09-30T22:30:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.534+0000 7fdf0333d700 1 -- start start 2023-09-30T22:30:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf0333d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc1a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf0333d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 0x7fdefc1a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf0333d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f650 0x7fdefc1a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf0333d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1a4920 con 0x7fdefc09e930 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf0333d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1a64b0 con 0x7fdefc09f650 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf0333d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0a3580 con 0x7fdefc09f240 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 0x7fdefc1a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf010d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc1a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf018da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f650 0x7fdefc1a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 0x7fdefc1a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48248/0 (socket says 172.21.15.32:48248) 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 -- 172.21.15.32:0/2388941573 learned_addr learned my addr 172.21.15.32:0/2388941573 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 -- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f650 msgr2=0x7fdefc1a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f650 0x7fdefc1a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 -- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 msgr2=0x7fdefc1a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc1a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.535+0000 7fdf008d8700 1 -- 172.21.15.32:0/2388941573 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0094a0 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.536+0000 7fdf008d8700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 0x7fdefc1a31b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00ebe0 tx=0x7fdeec00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.536+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec009780 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.536+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00c850 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.536+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef4009a70 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.536+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00cbe0 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.536+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef8009550 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.537+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdeec00cd40 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.537+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdeec03fac0 con 0x7fdefc09f240 2023-09-30T22:30:44.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.537+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc074ea0 con 0x7fdefc09f240 2023-09-30T22:30:44.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.542+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdeec011b00 con 0x7fdefc09f240 2023-09-30T22:30:44.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.744+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdefc1a6120 con 0x7fdefc09f240 2023-09-30T22:30:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.745+0000 7fdef1ffb700 1 -- 172.21.15.32:0/2388941573 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdeec021020 con 0x7fdefc09f240 2023-09-30T22:30:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 msgr2=0x7fdefc1a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 0x7fdefc1a31b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00ebe0 tx=0x7fdeec00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 shutdown_connections 2023-09-30T22:30:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdefc09f650 0x7fdefc1a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdefc09e930 0x7fdefc1a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 --2- 172.21.15.32:0/2388941573 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdefc09f240 0x7fdefc1a31b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 >> 172.21.15.32:0/2388941573 conn(0x7fdefc0a4210 msgr2=0x7fdefc1b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 shutdown_connections 2023-09-30T22:30:44.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:44.747+0000 7fdf0333d700 1 -- 172.21.15.32:0/2388941573 wait complete. 2023-09-30T22:30:44.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:45.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:45.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:45.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.257+0000 7f68d6c9b700 1 Processor -- start 2023-09-30T22:30:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.257+0000 7f68d6c9b700 1 -- start start 2023-09-30T22:30:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d6c9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d009e1d0 0x7f68d0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d6c9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68d0139170 0x7f68d013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d6c9b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d009b120 con 0x7f68d009d6f0 2023-09-30T22:30:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d6c9b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d006b460 con 0x7f68d0139170 2023-09-30T22:30:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d6c9b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d009a7e0 con 0x7f68d009e1d0 2023-09-30T22:30:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d5238700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68d0139170 0x7f68d013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d4a37700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68d009d6f0 0x7f68d009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48178/0 (socket says 172.21.15.32:48178) 2023-09-30T22:30:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d4a37700 1 -- 172.21.15.32:0/1909751574 learned_addr learned my addr 172.21.15.32:0/1909751574 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68cffff700 1 --2- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d009e1d0 0x7f68d0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d5238700 1 -- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d009e1d0 msgr2=0x7f68d0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d5238700 1 --2- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d009e1d0 0x7f68d0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d5238700 1 -- 172.21.15.32:0/1909751574 >> v1:172.21.15.32:6789/0 conn(0x7f68d009d6f0 legacy=0x7f68d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68cf7fe700 1 -- 172.21.15.32:0/1909751574 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771155570 0 0) 0x7f68d009b120 con 0x7f68d009d6f0 2023-09-30T22:30:45.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.258+0000 7f68d5238700 1 -- 172.21.15.32:0/1909751574 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d009a500 con 0x7f68d0139170 2023-09-30T22:30:45.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.259+0000 7f68d5238700 1 --2- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d0139170 0x7f68d013b550 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009d90 tx=0x7f68c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd5ddcca4526691 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.259+0000 7f68cf7fe700 1 -- 172.21.15.32:0/1909751574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800e070 con 0x7f68d0139170 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.259+0000 7f68cf7fe700 1 -- 172.21.15.32:0/1909751574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800bb20 con 0x7f68d0139170 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.259+0000 7f68cf7fe700 1 -- 172.21.15.32:0/1909751574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8012430 con 0x7f68d0139170 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.259+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d0139170 msgr2=0x7f68d013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.259+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d0139170 0x7f68d013b550 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009d90 tx=0x7f68c800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.260+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1909751574 shutdown_connections 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.260+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d0139170 0x7f68d013b550 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.260+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1909751574 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d009e1d0 0x7f68d0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.260+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1909751574 >> 172.21.15.32:0/1909751574 conn(0x7f68d0066f50 msgr2=0x7f68d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.260+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1909751574 shutdown_connections 2023-09-30T22:30:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.260+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1909751574 wait complete. 2023-09-30T22:30:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 Processor -- start 2023-09-30T22:30:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 -- start start 2023-09-30T22:30:45.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 0x7f68d0120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d009e1d0 0x7f68d0120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d0139170 0x7f68d0121410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0124130 con 0x7f68d009d6f0 2023-09-30T22:30:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0123b80 con 0x7f68d009e1d0 2023-09-30T22:30:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.261+0000 7f68d6c9b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0071f20 con 0x7f68d0139170 2023-09-30T22:30:45.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 0x7f68d0120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 0x7f68d0120990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46060/0 (socket says 172.21.15.32:46060) 2023-09-30T22:30:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 -- 172.21.15.32:0/1779595268 learned_addr learned my addr 172.21.15.32:0/1779595268 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d5238700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d0139170 0x7f68d0121410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68cffff700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d009e1d0 0x7f68d0120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 -- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d0139170 msgr2=0x7f68d0121410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d0139170 0x7f68d0121410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 -- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d009e1d0 msgr2=0x7f68d0120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d009e1d0 0x7f68d0120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 -- 172.21.15.32:0/1779595268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8009a70 con 0x7f68d009d6f0 2023-09-30T22:30:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68cffff700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d009e1d0 0x7f68d0120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d5238700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d0139170 0x7f68d0121410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d4a37700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 0x7f68d0120990 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f68c400c520 tx=0x7f68c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4004c80 con 0x7f68d009d6f0 2023-09-30T22:30:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c4004de0 con 0x7f68d009d6f0 2023-09-30T22:30:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d009c710 con 0x7f68d009d6f0 2023-09-30T22:30:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.262+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4007310 con 0x7f68d009d6f0 2023-09-30T22:30:45.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.263+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d0122cf0 con 0x7f68d009d6f0 2023-09-30T22:30:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.264+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d0074ea0 con 0x7f68d009d6f0 2023-09-30T22:30:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.264+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68c4007470 con 0x7f68d009d6f0 2023-09-30T22:30:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.265+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f68c403f9f0 con 0x7f68d009d6f0 2023-09-30T22:30:45.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.269+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68c40106d0 con 0x7f68d009d6f0 2023-09-30T22:30:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.295+0000 7f3aed0ac700 1 Processor -- start 2023-09-30T22:30:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.295+0000 7f3aed0ac700 1 -- start start 2023-09-30T22:30:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3aed0ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae812cf60 0x7f3ae806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3aed0ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 0x7f3ae812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3aed0ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 0x7f3ae8136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3aed0ac700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae812d330 con 0x7f3ae812f8b0 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3aed0ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae812cd90 con 0x7f3ae812cf60 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3aed0ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae81306d0 con 0x7f3ae8130260 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3ae659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 0x7f3ae812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3ae759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 0x7f3ae8136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3ae659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 0x7f3ae812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46068/0 (socket says 172.21.15.32:46068) 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3ae759e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 0x7f3ae8136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48262/0 (socket says 172.21.15.32:48262) 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3ae659c700 1 -- 172.21.15.32:0/557636517 learned_addr learned my addr 172.21.15.32:0/557636517 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.296+0000 7f3ae6d9d700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae812cf60 0x7f3ae806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae759e700 1 -- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae812cf60 msgr2=0x7f3ae806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae759e700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae812cf60 0x7f3ae806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae759e700 1 -- 172.21.15.32:0/557636517 >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 msgr2=0x7f3ae812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae759e700 1 --2- 172.21.15.32:0/557636517 >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 0x7f3ae812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae759e700 1 -- 172.21.15.32:0/557636517 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae8071780 con 0x7f3ae8130260 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae6d9d700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae812cf60 0x7f3ae806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae659c700 1 --2- 172.21.15.32:0/557636517 >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 0x7f3ae812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae759e700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 0x7f3ae8136c20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009dc0 tx=0x7f3ae000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7f89ea96743e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae5d9b700 1 -- 172.21.15.32:0/557636517 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000e070 con 0x7f3ae8130260 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae5d9b700 1 -- 172.21.15.32:0/557636517 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae000bae0 con 0x7f3ae8130260 2023-09-30T22:30:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.297+0000 7f3ae5d9b700 1 -- 172.21.15.32:0/557636517 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000be90 con 0x7f3ae8130260 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 -- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 msgr2=0x7f3ae8136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 0x7f3ae8136c20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009dc0 tx=0x7f3ae000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 -- 172.21.15.32:0/557636517 shutdown_connections 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae812cf60 0x7f3ae806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/557636517 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae8130260 0x7f3ae8136c20 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/557636517 >> v2:172.21.15.32:3300/0 conn(0x7f3ae812f8b0 0x7f3ae812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 -- 172.21.15.32:0/557636517 >> 172.21.15.32:0/557636517 conn(0x7f3ae80a2aa0 msgr2=0x7f3ae80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 -- 172.21.15.32:0/557636517 shutdown_connections 2023-09-30T22:30:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.298+0000 7f3aed0ac700 1 -- 172.21.15.32:0/557636517 wait complete. 2023-09-30T22:30:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.299+0000 7f3aed0ac700 1 Processor -- start 2023-09-30T22:30:45.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.299+0000 7f3aed0ac700 1 -- start start 2023-09-30T22:30:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3aed0ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 0x7f3ae81c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3aed0ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae812f8b0 0x7f3ae81c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3aed0ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae8130260 0x7f3ae809b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3aed0ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae809e500 con 0x7f3ae812f8b0 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3aed0ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8129970 con 0x7f3ae8130260 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3aed0ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae809eaa0 con 0x7f3ae812cf60 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 0x7f3ae81c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 0x7f3ae81c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48264/0 (socket says 172.21.15.32:48264) 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 -- 172.21.15.32:0/585472631 learned_addr learned my addr 172.21.15.32:0/585472631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:45.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae759e700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae8130260 0x7f3ae809b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae659c700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae812f8b0 0x7f3ae81c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 -- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae8130260 msgr2=0x7f3ae809b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae8130260 0x7f3ae809b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 -- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae812f8b0 msgr2=0x7f3ae81c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae812f8b0 0x7f3ae81c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.300+0000 7f3ae6d9d700 1 -- 172.21.15.32:0/585472631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80094a0 con 0x7f3ae812cf60 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.301+0000 7f3ae6d9d700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 0x7f3ae81c41e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009f70 tx=0x7f3ad800ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.301+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8017c60 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.301+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8017dc0 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.301+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc00f880 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.301+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800c800 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.301+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae0009a70 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.302+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3ad800c960 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.302+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae812a000 con 0x7f3ae812cf60 2023-09-30T22:30:45.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.302+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3ad8047b40 con 0x7f3ae812cf60 2023-09-30T22:30:45.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.307+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ad801e990 con 0x7f3ae812cf60 2023-09-30T22:30:45.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 Processor -- start 2023-09-30T22:30:45.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 -- start start 2023-09-30T22:30:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f84400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84400a1340 0x7f8440133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844009edd0 con 0x7f8440133fa0 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844009e1f0 con 0x7f84400a1340 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.346+0000 7f844553b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f844009dc50 con 0x7f84400a0a30 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84400a1340 0x7f8440133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8440133fa0 0x7f844013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48192/0 (socket says 172.21.15.32:48192) 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f84400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843f7fe700 1 -- 172.21.15.32:0/2681454052 learned_addr learned my addr 172.21.15.32:0/2681454052 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843dffb700 1 -- 172.21.15.32:0/2681454052 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640261533 0 0) 0x7f844009edd0 con 0x7f8440133fa0 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843dffb700 1 -- 172.21.15.32:0/2681454052 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84280035b0 con 0x7f8440133fa0 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843effd700 1 -- 172.21.15.32:0/2681454052 >> v2:172.21.15.32:3300/0 conn(0x7f84400a1340 msgr2=0x7f8440133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843effd700 1 --2- 172.21.15.32:0/2681454052 >> v2:172.21.15.32:3300/0 conn(0x7f84400a1340 0x7f8440133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843effd700 1 -- 172.21.15.32:0/2681454052 >> v1:172.21.15.32:6789/0 conn(0x7f8440133fa0 legacy=0x7f844013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843effd700 1 -- 172.21.15.32:0/2681454052 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400a1fe0 con 0x7f84400a0a30 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843e7fc700 1 --2- 172.21.15.32:0/2681454052 >> v2:172.21.15.32:3300/0 conn(0x7f84400a1340 0x7f8440133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.347+0000 7f843effd700 1 --2- 172.21.15.32:0/2681454052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f84400a0e00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f8430009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e05e51cf20f8cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f843dffb700 1 -- 172.21.15.32:0/2681454052 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430004230 con 0x7f84400a0a30 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f843dffb700 1 -- 172.21.15.32:0/2681454052 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430004390 con 0x7f84400a0a30 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f843dffb700 1 -- 172.21.15.32:0/2681454052 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300036a0 con 0x7f84400a0a30 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 -- 172.21.15.32:0/2681454052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 msgr2=0x7f84400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 --2- 172.21.15.32:0/2681454052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f84400a0e00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f8430009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 -- 172.21.15.32:0/2681454052 shutdown_connections 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 --2- 172.21.15.32:0/2681454052 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f84400a0e00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 --2- 172.21.15.32:0/2681454052 >> v2:172.21.15.32:3300/0 conn(0x7f84400a1340 0x7f8440133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 -- 172.21.15.32:0/2681454052 >> 172.21.15.32:0/2681454052 conn(0x7f84400a6550 msgr2=0x7f84400a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 -- 172.21.15.32:0/2681454052 shutdown_connections 2023-09-30T22:30:45.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.348+0000 7f844553b700 1 -- 172.21.15.32:0/2681454052 wait complete. 2023-09-30T22:30:45.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 Processor -- start 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 -- start start 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f844011e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 0x7f844011eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8440133fa0 0x7f844011f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440121f50 con 0x7f84400a1340 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8440122150 con 0x7f8440133fa0 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f844553b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84401218c0 con 0x7f84400a0a30 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f843effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f844011e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f843e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 0x7f844011eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f843effd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f844011e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48276/0 (socket says 172.21.15.32:48276) 2023-09-30T22:30:45.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8440133fa0 0x7f844011f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f843effd700 1 -- 172.21.15.32:0/1815425802 learned_addr learned my addr 172.21.15.32:0/1815425802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.350+0000 7f843e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 0x7f844011eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46100/0 (socket says 172.21.15.32:46100) 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843e7fc700 1 -- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 msgr2=0x7f844011e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843e7fc700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f844011e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843e7fc700 1 -- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8440133fa0 msgr2=0x7f844011f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843e7fc700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8440133fa0 0x7f844011f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843e7fc700 1 -- 172.21.15.32:0/1815425802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340094a0 con 0x7f84400a1340 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843f7fe700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8440133fa0 0x7f844011f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f843e7fc700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 0x7f844011eb70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f843400e9b0 tx=0x7f843400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400ee50 con 0x7f84400a1340 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400c850 con 0x7f84400a1340 2023-09-30T22:30:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84300094a0 con 0x7f84400a1340 2023-09-30T22:30:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400cc00 con 0x7f84400a1340 2023-09-30T22:30:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.351+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84400a0220 con 0x7f84400a1340 2023-09-30T22:30:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.352+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8434013470 con 0x7f84400a1340 2023-09-30T22:30:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.352+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f843403f750 con 0x7f84400a1340 2023-09-30T22:30:45.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.353+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84400a1d40 con 0x7f84400a1340 2023-09-30T22:30:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.358+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8434013720 con 0x7f84400a1340 2023-09-30T22:30:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.480+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f68d0071d00 con 0x7f68d009d6f0 2023-09-30T22:30:45.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.480+0000 7f68cd7fa700 1 -- 172.21.15.32:0/1779595268 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68c4011360 con 0x7f68d009d6f0 2023-09-30T22:30:45.482 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.483+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 msgr2=0x7f68d0120990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.483+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 0x7f68d0120990 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f68c400c520 tx=0x7f68c400c830 comp rx=0 tx=0).stop 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 shutdown_connections 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68d0139170 0x7f68d0121410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68d009d6f0 0x7f68d0120990 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 --2- 172.21.15.32:0/1779595268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68d009e1d0 0x7f68d0120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 >> 172.21.15.32:0/1779595268 conn(0x7f68d0066f50 msgr2=0x7f68d0070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 shutdown_connections 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.484+0000 7f68d6c9b700 1 -- 172.21.15.32:0/1779595268 wait complete. 2023-09-30T22:30:45.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:45.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.524+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ae8074ea0 con 0x7f3ae812cf60 2023-09-30T22:30:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.525+0000 7f3acf7fe700 1 -- 172.21.15.32:0/585472631 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ad8024030 con 0x7f3ae812cf60 2023-09-30T22:30:45.526 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 msgr2=0x7f3ae81c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 0x7f3ae81c41e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009f70 tx=0x7f3ad800ee10 comp rx=0 tx=0).stop 2023-09-30T22:30:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 shutdown_connections 2023-09-30T22:30:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ae8130260 0x7f3ae809b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ae812cf60 0x7f3ae81c41e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 --2- 172.21.15.32:0/585472631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ae812f8b0 0x7f3ae81c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 >> 172.21.15.32:0/585472631 conn(0x7f3ae80a2aa0 msgr2=0x7f3ae81b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 shutdown_connections 2023-09-30T22:30:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.527+0000 7f3aed0ac700 1 -- 172.21.15.32:0/585472631 wait complete. 2023-09-30T22:30:45.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:45.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.584+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8440074ea0 con 0x7f84400a1340 2023-09-30T22:30:45.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.584+0000 7f84277fe700 1 -- 172.21.15.32:0/1815425802 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8434017020 con 0x7f84400a1340 2023-09-30T22:30:45.585 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 msgr2=0x7f844011eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 0x7f844011eb70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f843400e9b0 tx=0x7f843400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 shutdown_connections 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8440133fa0 0x7f844011f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f84400a0a30 0x7f844011e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 --2- 172.21.15.32:0/1815425802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84400a1340 0x7f844011eb70 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 >> 172.21.15.32:0/1815425802 conn(0x7f84400a6550 msgr2=0x7f84401b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 shutdown_connections 2023-09-30T22:30:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.586+0000 7f844553b700 1 -- 172.21.15.32:0/1815425802 wait complete. 2023-09-30T22:30:45.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:45.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.906+0000 7fcb80f3e700 1 Processor -- start 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.906+0000 7fcb80f3e700 1 -- start start 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb80f3e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 0x7fcb7c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb80f3e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb7c139170 0x7fcb7c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb80f3e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7c09b120 con 0x7fcb7c09d6f0 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb80f3e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7c06b460 con 0x7fcb7c139170 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb80f3e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c09a7e0 con 0x7fcb7c09e1d0 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb7c139170 0x7fcb7c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb71d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 0x7fcb7c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb71d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 0x7fcb7c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48280/0 (socket says 172.21.15.32:48280) 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb7c09d6f0 0x7fcb7c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48204/0 (socket says 172.21.15.32:48204) 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb71d9b700 1 -- 172.21.15.32:0/3009439001 learned_addr learned my addr 172.21.15.32:0/3009439001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7ad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb7c139170 0x7fcb7c13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46104/0 (socket says 172.21.15.32:46104) 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 msgr2=0x7fcb7c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7ad9d700 1 --2- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 0x7fcb7c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3009439001 >> v1:172.21.15.32:6789/0 conn(0x7fcb7c09d6f0 legacy=0x7fcb7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3009439001 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c09a500 con 0x7fcb7c139170 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb71d9b700 1 --2- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 0x7fcb7c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.907+0000 7fcb79d9b700 1 -- 172.21.15.32:0/3009439001 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126941250 0 0) 0x7fcb7c09b120 con 0x7fcb7c09d6f0 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb7ad9d700 1 --2- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c139170 0x7fcb7c13b550 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009d90 tx=0x7fcb7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a010c3b75901e946 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb79d9b700 1 -- 172.21.15.32:0/3009439001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400e070 con 0x7fcb7c139170 2023-09-30T22:30:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb79d9b700 1 -- 172.21.15.32:0/3009439001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7400bb20 con 0x7fcb7c139170 2023-09-30T22:30:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb79d9b700 1 -- 172.21.15.32:0/3009439001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74012430 con 0x7fcb7c139170 2023-09-30T22:30:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c139170 msgr2=0x7fcb7c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c139170 0x7fcb7c13b550 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009d90 tx=0x7fcb7400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3009439001 shutdown_connections 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c139170 0x7fcb7c13b550 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3009439001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c09e1d0 0x7fcb7c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3009439001 >> 172.21.15.32:0/3009439001 conn(0x7fcb7c066f50 msgr2=0x7fcb7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3009439001 shutdown_connections 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.908+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3009439001 wait complete. 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.910+0000 7fcb80f3e700 1 Processor -- start 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.910+0000 7fcb80f3e700 1 -- start start 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb80f3e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c09d6f0 0x7fcb7c11e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb80f3e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7c09e1d0 0x7fcb7c11edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb80f3e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 0x7fcb7c11f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb80f3e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c0720b0 con 0x7fcb7c09d6f0 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb80f3e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1205d0 con 0x7fcb7c09e1d0 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb80f3e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c121e30 con 0x7fcb7c139170 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 0x7fcb7c11f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c09d6f0 0x7fcb7c11e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 0x7fcb7c11f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48282/0 (socket says 172.21.15.32:48282) 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3718797021 learned_addr learned my addr 172.21.15.32:0/3718797021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb71d9b700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7c09e1d0 0x7fcb7c11edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7c09e1d0 msgr2=0x7fcb7c11edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7c09e1d0 0x7fcb7c11edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c09d6f0 msgr2=0x7fcb7c11e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c09d6f0 0x7fcb7c11e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7ad9d700 1 -- 172.21.15.32:0/3718797021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb64009550 con 0x7fcb7c139170 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb71d9b700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7c09e1d0 0x7fcb7c11edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.911+0000 7fcb7a59c700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c09d6f0 0x7fcb7c11e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.912+0000 7fcb7ad9d700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 0x7fcb7c11f330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcb74014040 tx=0x7fcb74004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.912+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74012430 con 0x7fcb7c139170 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.912+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb74020070 con 0x7fcb7c139170 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.912+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb74009a70 con 0x7fcb7c139170 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.912+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7401ba30 con 0x7fcb7c139170 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.912+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c1220f0 con 0x7fcb7c139170 2023-09-30T22:30:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.913+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb7401bb90 con 0x7fcb7c139170 2023-09-30T22:30:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.913+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcb74022070 con 0x7fcb7c139170 2023-09-30T22:30:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.913+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c09c710 con 0x7fcb7c139170 2023-09-30T22:30:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:45.918+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb7401be40 con 0x7fcb7c139170 2023-09-30T22:30:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.121+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb7c074ea0 con 0x7fcb7c139170 2023-09-30T22:30:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.121+0000 7fcb737fe700 1 -- 172.21.15.32:0/3718797021 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb7402c020 con 0x7fcb7c139170 2023-09-30T22:30:46.123 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 msgr2=0x7fcb7c11f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 0x7fcb7c11f330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcb74014040 tx=0x7fcb74004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 shutdown_connections 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcb7c139170 0x7fcb7c11f330 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb7c09d6f0 0x7fcb7c11e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 --2- 172.21.15.32:0/3718797021 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcb7c09e1d0 0x7fcb7c11edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 >> 172.21.15.32:0/3718797021 conn(0x7fcb7c066f50 msgr2=0x7fcb7c136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 shutdown_connections 2023-09-30T22:30:46.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.124+0000 7fcb80f3e700 1 -- 172.21.15.32:0/3718797021 wait complete. 2023-09-30T22:30:46.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:46.505 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:46.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:46.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.652+0000 7ff4aa1f9700 1 Processor -- start 2023-09-30T22:30:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4aa1f9700 1 -- start start 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4aa1f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4131810 0x7ff4a4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4aa1f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 0x7ff4a4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4aa1f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a41313c0 con 0x7ff4a409a3b0 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4aa1f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a409b580 con 0x7ff4a4131810 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4aa1f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a406e8c0 con 0x7ff4a4099a40 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4a2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 0x7ff4a4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4a37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4131810 0x7ff4a4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4a2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 0x7ff4a4099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48298/0 (socket says 172.21.15.32:48298) 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.653+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/4099943723 learned_addr learned my addr 172.21.15.32:0/4099943723 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4131810 msgr2=0x7ff4a4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a2ffd700 1 --2- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4131810 0x7ff4a4131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/4099943723 >> v1:172.21.15.32:6789/0 conn(0x7ff4a409a3b0 legacy=0x7ff4a4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/4099943723 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a409a820 con 0x7ff4a4099a40 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a2ffd700 1 --2- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 0x7ff4a4099e70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff498009920 tx=0x7ff498009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b155d2a42d9b927f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a27fc700 1 -- 172.21.15.32:0/4099943723 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498011830 con 0x7ff4a4099a40 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a27fc700 1 -- 172.21.15.32:0/4099943723 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498011990 con 0x7ff4a4099a40 2023-09-30T22:30:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.654+0000 7ff4a27fc700 1 -- 172.21.15.32:0/4099943723 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498011d20 con 0x7ff4a4099a40 2023-09-30T22:30:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 msgr2=0x7ff4a4099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 0x7ff4a4099e70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff498009920 tx=0x7ff498009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/4099943723 shutdown_connections 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4131810 0x7ff4a4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/4099943723 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4099a40 0x7ff4a4099e70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/4099943723 >> 172.21.15.32:0/4099943723 conn(0x7ff4a40a2aa0 msgr2=0x7ff4a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/4099943723 shutdown_connections 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.655+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/4099943723 wait complete. 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.656+0000 7ff4aa1f9700 1 Processor -- start 2023-09-30T22:30:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.656+0000 7ff4aa1f9700 1 -- start start 2023-09-30T22:30:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4aa1f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 0x7ff4a411b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4aa1f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a409a3b0 0x7ff4a411be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4aa1f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4131810 0x7ff4a411c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4aa1f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a411f2c0 con 0x7ff4a409a3b0 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4aa1f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a411e690 con 0x7ff4a4099a40 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4aa1f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a411d990 con 0x7ff4a4131810 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4a37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 0x7ff4a411b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4a37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 0x7ff4a411b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48032/0 (socket says 172.21.15.32:48032) 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4a37fe700 1 -- 172.21.15.32:0/1839460197 learned_addr learned my addr 172.21.15.32:0/1839460197 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.657+0000 7ff4a2ffd700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a409a3b0 0x7ff4a411be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a3fff700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4131810 0x7ff4a411c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4131810 msgr2=0x7ff4a411c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a2ffd700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4131810 0x7ff4a411c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 msgr2=0x7ff4a411b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a2ffd700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 0x7ff4a411b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a2ffd700 1 -- 172.21.15.32:0/1839460197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498009550 con 0x7ff4a409a3b0 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a37fe700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 0x7ff4a411b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a3fff700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4131810 0x7ff4a411c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4a2ffd700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a409a3b0 0x7ff4a411be10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff498016000 tx=0x7ff498004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498004510 con 0x7ff4a409a3b0 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49801f070 con 0x7ff4a409a3b0 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4980038e0 con 0x7ff4a409a3b0 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a411ea50 con 0x7ff4a409a3b0 2023-09-30T22:30:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.658+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a4130a90 con 0x7ff4a409a3b0 2023-09-30T22:30:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.659+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff498003a40 con 0x7ff4a409a3b0 2023-09-30T22:30:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.659+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a411ec80 con 0x7ff4a409a3b0 2023-09-30T22:30:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.660+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff498021070 con 0x7ff4a409a3b0 2023-09-30T22:30:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.669+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4a411ec80 con 0x7ff4a409a3b0 2023-09-30T22:30:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.694+0000 7f3903bb5700 1 Processor -- start 2023-09-30T22:30:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.694+0000 7f3903bb5700 1 -- start start 2023-09-30T22:30:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3903bb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 0x7f38fc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3903bb5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1340 0x7f38fc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3903bb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc133fa0 0x7f38fc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3903bb5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc09edd0 con 0x7f38fc0a1340 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3903bb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc09e1f0 con 0x7f38fc133fa0 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3903bb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc09dc50 con 0x7f38fc0a0a30 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 0x7f38fc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901150700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1340 0x7f38fc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 0x7f38fc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48322/0 (socket says 172.21.15.32:48322) 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 -- 172.21.15.32:0/3210887821 learned_addr learned my addr 172.21.15.32:0/3210887821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3902152700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc133fa0 0x7f38fc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 -- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc133fa0 msgr2=0x7f38fc134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc133fa0 0x7f38fc134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 -- 172.21.15.32:0/3210887821 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1340 msgr2=0x7f38fc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 --2- 172.21.15.32:0/3210887821 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1340 0x7f38fc133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901951700 1 -- 172.21.15.32:0/3210887821 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0a1fe0 con 0x7f38fc0a0a30 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3901150700 1 --2- 172.21.15.32:0/3210887821 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1340 0x7f38fc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.695+0000 7f3902152700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc133fa0 0x7f38fc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3901951700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 0x7f38fc0a0e00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009920 tx=0x7f38f8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c94bb3fbed6150 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f390094f700 1 -- 172.21.15.32:0/3210887821 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8011830 con 0x7f38fc0a0a30 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f390094f700 1 -- 172.21.15.32:0/3210887821 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f8011990 con 0x7f38fc0a0a30 2023-09-30T22:30:46.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f390094f700 1 -- 172.21.15.32:0/3210887821 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8011d20 con 0x7f38fc0a0a30 2023-09-30T22:30:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 -- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 msgr2=0x7f38fc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 0x7f38fc0a0e00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009920 tx=0x7f38f8009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 -- 172.21.15.32:0/3210887821 shutdown_connections 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc133fa0 0x7f38fc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 --2- 172.21.15.32:0/3210887821 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc0a0a30 0x7f38fc0a0e00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 --2- 172.21.15.32:0/3210887821 >> v2:172.21.15.32:3300/0 conn(0x7f38fc0a1340 0x7f38fc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.696+0000 7f3903bb5700 1 -- 172.21.15.32:0/3210887821 >> 172.21.15.32:0/3210887821 conn(0x7f38fc0a6550 msgr2=0x7f38fc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.697+0000 7f3903bb5700 1 -- 172.21.15.32:0/3210887821 shutdown_connections 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.697+0000 7f3903bb5700 1 -- 172.21.15.32:0/3210887821 wait complete. 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.698+0000 7f3903bb5700 1 Processor -- start 2023-09-30T22:30:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.698+0000 7f3903bb5700 1 -- start start 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3903bb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc120760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3903bb5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 0x7f38fc120ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3903bb5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc133fa0 0x7f38fc1211e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3903bb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc122370 con 0x7f38fc0a0a30 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3903bb5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc122990 con 0x7f38fc0a1340 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3903bb5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1235c0 con 0x7f38fc133fa0 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3901150700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 0x7f38fc120ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3901150700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 0x7f38fc120ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48038/0 (socket says 172.21.15.32:48038) 2023-09-30T22:30:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3901150700 1 -- 172.21.15.32:0/2067511263 learned_addr learned my addr 172.21.15.32:0/2067511263 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3901951700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc120760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3902152700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc133fa0 0x7f38fc1211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.699+0000 7f3901150700 1 -- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc133fa0 msgr2=0x7f38fc1211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3901150700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc133fa0 0x7f38fc1211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3901150700 1 -- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 msgr2=0x7f38fc120760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3901150700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc120760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3901150700 1 -- 172.21.15.32:0/2067511263 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0094a0 con 0x7f38fc0a1340 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3902152700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc133fa0 0x7f38fc1211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3901150700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 0x7f38fc120ca0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00ec10 tx=0x7f38ec00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00c850 con 0x7f38fc0a1340 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec00c9b0 con 0x7f38fc0a1340 2023-09-30T22:30:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f4009a70 con 0x7f38fc0a1340 2023-09-30T22:30:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00cd20 con 0x7f38fc0a1340 2023-09-30T22:30:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.700+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f8009550 con 0x7f38fc0a1340 2023-09-30T22:30:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.701+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38ec013470 con 0x7f38fc0a1340 2023-09-30T22:30:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.701+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f38ec03f7a0 con 0x7f38fc0a1340 2023-09-30T22:30:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.701+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc0a0220 con 0x7f38fc0a1340 2023-09-30T22:30:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.706+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38ec013720 con 0x7f38fc0a1340 2023-09-30T22:30:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.751+0000 7fa4592b0700 1 Processor -- start 2023-09-30T22:30:46.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.751+0000 7fa4592b0700 1 -- start start 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4592b0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa4541337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4592b0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa454134260 0x7fa4541306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4592b0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45406b990 con 0x7fa454133e30 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4592b0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540a0590 con 0x7fa454134260 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4592b0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa45409fb80 con 0x7fa454133420 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4537fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa454134260 0x7fa4541306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4527fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa454133e30 0x7fa45412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48218/0 (socket says 172.21.15.32:48218) 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa4527fc700 1 -- 172.21.15.32:0/2101935422 learned_addr learned my addr 172.21.15.32:0/2101935422 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa452ffd700 1 --2- 172.21.15.32:0/2101935422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa4541337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa452ffd700 1 -- 172.21.15.32:0/2101935422 >> v2:172.21.15.32:3300/0 conn(0x7fa454134260 msgr2=0x7fa4541306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa452ffd700 1 --2- 172.21.15.32:0/2101935422 >> v2:172.21.15.32:3300/0 conn(0x7fa454134260 0x7fa4541306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa452ffd700 1 -- 172.21.15.32:0/2101935422 >> v1:172.21.15.32:6789/0 conn(0x7fa454133e30 legacy=0x7fa45412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa452ffd700 1 -- 172.21.15.32:0/2101935422 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540a0030 con 0x7fa454133420 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.752+0000 7fa451ffb700 1 -- 172.21.15.32:0/2101935422 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352398069 0 0) 0x7fa45406b990 con 0x7fa454133e30 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa4537fe700 1 --2- 172.21.15.32:0/2101935422 >> v2:172.21.15.32:3300/0 conn(0x7fa454134260 0x7fa4541306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa452ffd700 1 --2- 172.21.15.32:0/2101935422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa4541337f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa444009840 tx=0x7fa444009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb9aab9b0099b6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa451ffb700 1 -- 172.21.15.32:0/2101935422 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011830 con 0x7fa454133420 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa451ffb700 1 -- 172.21.15.32:0/2101935422 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444011990 con 0x7fa454133420 2023-09-30T22:30:46.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa451ffb700 1 -- 172.21.15.32:0/2101935422 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011d20 con 0x7fa454133420 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa4592b0700 1 -- 172.21.15.32:0/2101935422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 msgr2=0x7fa4541337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.753+0000 7fa4592b0700 1 --2- 172.21.15.32:0/2101935422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa4541337f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa444009840 tx=0x7fa444009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.754+0000 7fa4592b0700 1 -- 172.21.15.32:0/2101935422 shutdown_connections 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.754+0000 7fa4592b0700 1 --2- 172.21.15.32:0/2101935422 >> v2:172.21.15.32:3300/0 conn(0x7fa454134260 0x7fa4541306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.754+0000 7fa4592b0700 1 --2- 172.21.15.32:0/2101935422 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa4541337f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.754+0000 7fa4592b0700 1 -- 172.21.15.32:0/2101935422 >> 172.21.15.32:0/2101935422 conn(0x7fa4540a4210 msgr2=0x7fa4540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.754+0000 7fa4592b0700 1 -- 172.21.15.32:0/2101935422 shutdown_connections 2023-09-30T22:30:46.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.754+0000 7fa4592b0700 1 -- 172.21.15.32:0/2101935422 wait complete. 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.755+0000 7fa4592b0700 1 Processor -- start 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.755+0000 7fa4592b0700 1 -- start start 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4592b0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa45409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4592b0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 0x7fa45409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4592b0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa454134260 0x7fa45409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4592b0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45411ff20 con 0x7fa454134260 2023-09-30T22:30:46.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4592b0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa454120100 con 0x7fa454133e30 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4592b0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa45412d9b0 con 0x7fa454133420 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 0x7fa45409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 0x7fa45409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48046/0 (socket says 172.21.15.32:48046) 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 -- 172.21.15.32:0/515033586 learned_addr learned my addr 172.21.15.32:0/515033586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa452ffd700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa45409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4537fe700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa454134260 0x7fa45409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 -- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 msgr2=0x7fa45409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa45409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 -- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa454134260 msgr2=0x7fa45409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa454134260 0x7fa45409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 -- 172.21.15.32:0/515033586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c009a70 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa452ffd700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa45409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.756+0000 7fa4527fc700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 0x7fa45409c9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa448001eb0 tx=0x7fa44800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.757+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448004c80 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.757+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448004de0 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.757+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4440094a0 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.757+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448007310 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.757+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa454131750 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.758+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa448007470 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.758+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa44803f320 con 0x7fa454133e30 2023-09-30T22:30:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.758+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45411fd40 con 0x7fa454133e30 2023-09-30T22:30:46.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.763+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa4480123b0 con 0x7fa454133e30 2023-09-30T22:30:46.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.879+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4a411ec80 con 0x7ff4a409a3b0 2023-09-30T22:30:46.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.879+0000 7ff48bfff700 1 -- 172.21.15.32:0/1839460197 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4a411ec80 con 0x7ff4a409a3b0 2023-09-30T22:30:46.880 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a409a3b0 msgr2=0x7ff4a411be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a409a3b0 0x7ff4a411be10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff498016000 tx=0x7ff498004750 comp rx=0 tx=0).stop 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 shutdown_connections 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff4a4131810 0x7ff4a411c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff4a4099a40 0x7ff4a411b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 --2- 172.21.15.32:0/1839460197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff4a409a3b0 0x7ff4a411be10 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.881+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 >> 172.21.15.32:0/1839460197 conn(0x7ff4a40a2aa0 msgr2=0x7ff4a409d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.882+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 shutdown_connections 2023-09-30T22:30:46.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.882+0000 7ff4aa1f9700 1 -- 172.21.15.32:0/1839460197 wait complete. 2023-09-30T22:30:46.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.913+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38fc074ea0 con 0x7f38fc0a1340 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.914+0000 7f38f27fc700 1 -- 172.21.15.32:0/2067511263 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38ec021020 con 0x7f38fc0a1340 2023-09-30T22:30:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 msgr2=0x7f38fc120ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 0x7f38fc120ca0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00ec10 tx=0x7f38ec00ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 shutdown_connections 2023-09-30T22:30:46.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f38fc133fa0 0x7f38fc1211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38fc0a0a30 0x7f38fc120760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 --2- 172.21.15.32:0/2067511263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38fc0a1340 0x7f38fc120ca0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 >> 172.21.15.32:0/2067511263 conn(0x7f38fc0a6550 msgr2=0x7f38fc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 shutdown_connections 2023-09-30T22:30:46.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.916+0000 7f3903bb5700 1 -- 172.21.15.32:0/2067511263 wait complete. 2023-09-30T22:30:46.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:46.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.969+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa454074ea0 con 0x7fa454133e30 2023-09-30T22:30:46.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.969+0000 7fa43b7fe700 1 -- 172.21.15.32:0/515033586 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa448011360 con 0x7fa454133e30 2023-09-30T22:30:46.970 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:46.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 msgr2=0x7fa45409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 0x7fa45409c9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa448001eb0 tx=0x7fa44800c800 comp rx=0 tx=0).stop 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 shutdown_connections 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa454134260 0x7fa45409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa454133420 0x7fa45409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 --2- 172.21.15.32:0/515033586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa454133e30 0x7fa45409c9c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 >> 172.21.15.32:0/515033586 conn(0x7fa4540a4210 msgr2=0x7fa4540a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 shutdown_connections 2023-09-30T22:30:46.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:46.971+0000 7fa4592b0700 1 -- 172.21.15.32:0/515033586 wait complete. 2023-09-30T22:30:46.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:47.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.283+0000 7fc3e5d98700 1 Processor -- start 2023-09-30T22:30:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 -- start start 2023-09-30T22:30:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 0x7fc3e0131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e009ece0 con 0x7fc3e0131b40 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e009e760 con 0x7fc3e006a770 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3e5d98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e009e040 con 0x7fc3e006b080 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3df7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3dffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 0x7fc3e0131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3dffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 0x7fc3e0131fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46178/0 (socket says 172.21.15.32:46178) 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3df7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48048/0 (socket says 172.21.15.32:48048) 2023-09-30T22:30:47.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3dffff700 1 -- 172.21.15.32:0/3586340356 learned_addr learned my addr 172.21.15.32:0/3586340356 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.284+0000 7fc3d6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3df7fe700 1 -- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 msgr2=0x7fc3e0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3df7fe700 1 --2- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3df7fe700 1 -- 172.21.15.32:0/3586340356 >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 msgr2=0x7fc3e0131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3df7fe700 1 --2- 172.21.15.32:0/3586340356 >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 0x7fc3e0131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3df7fe700 1 -- 172.21.15.32:0/3586340356 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e009f630 con 0x7fc3e006a770 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3dffff700 1 --2- 172.21.15.32:0/3586340356 >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 0x7fc3e0131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3d6ffd700 1 --2- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3df7fe700 1 --2- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e006ab40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009840 tx=0x7fc3d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadbb1660f4bda0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3deffd700 1 -- 172.21.15.32:0/3586340356 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0011830 con 0x7fc3e006a770 2023-09-30T22:30:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3deffd700 1 -- 172.21.15.32:0/3586340356 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0011990 con 0x7fc3e006a770 2023-09-30T22:30:47.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.285+0000 7fc3deffd700 1 -- 172.21.15.32:0/3586340356 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0011d20 con 0x7fc3e006a770 2023-09-30T22:30:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 -- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 msgr2=0x7fc3e006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e006ab40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009840 tx=0x7fc3d0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 -- 172.21.15.32:0/3586340356 shutdown_connections 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e006ab40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/3586340356 >> v2:172.21.15.32:3300/0 conn(0x7fc3e0131b40 0x7fc3e0131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/3586340356 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 -- 172.21.15.32:0/3586340356 >> 172.21.15.32:0/3586340356 conn(0x7fc3e0066f50 msgr2=0x7fc3e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 -- 172.21.15.32:0/3586340356 shutdown_connections 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.286+0000 7fc3e5d98700 1 -- 172.21.15.32:0/3586340356 wait complete. 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 Processor -- start 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 -- start start 2023-09-30T22:30:47.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e01227b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0122cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3e0131b40 0x7fc3e0123230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e01243c0 con 0x7fc3e0131b40 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e0124c50 con 0x7fc3e006a770 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.287+0000 7fc3e5d98700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e0123c10 con 0x7fc3e006b080 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e01227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e01227b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48050/0 (socket says 172.21.15.32:48050) 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 -- 172.21.15.32:0/325531867 learned_addr learned my addr 172.21.15.32:0/325531867 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:47.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3dffff700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3e0131b40 0x7fc3e0123230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3d6ffd700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0122cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 -- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 msgr2=0x7fc3e0122cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0122cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 -- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3e0131b40 msgr2=0x7fc3e0123230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3e0131b40 0x7fc3e0123230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 -- 172.21.15.32:0/325531867 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80094a0 con 0x7fc3e006a770 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3df7fe700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e01227b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0004450 con 0x7fc3e006a770 2023-09-30T22:30:47.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d001f070 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d8009b30 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00038e0 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.288+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d00094a0 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.289+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e0071e60 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.290+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3d0003a40 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.290+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc3d0021070 con 0x7fc3e006a770 2023-09-30T22:30:47.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.294+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3d0040e60 con 0x7fc3e006a770 2023-09-30T22:30:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.499+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc3e0074ea0 con 0x7fc3e006a770 2023-09-30T22:30:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.499+0000 7fc3dcff9700 1 -- 172.21.15.32:0/325531867 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3d0034090 con 0x7fc3e006a770 2023-09-30T22:30:47.501 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 msgr2=0x7fc3e01227b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e01227b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 shutdown_connections 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3e0131b40 0x7fc3e0123230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3e006a770 0x7fc3e01227b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 --2- 172.21.15.32:0/325531867 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc3e006b080 0x7fc3e0122cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 >> 172.21.15.32:0/325531867 conn(0x7fc3e0066f50 msgr2=0x7fc3e0071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 shutdown_connections 2023-09-30T22:30:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:47.502+0000 7fc3e5d98700 1 -- 172.21.15.32:0/325531867 wait complete. 2023-09-30T22:30:47.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:47.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:47.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:47.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.051+0000 7f2af83a4700 1 Processor -- start 2023-09-30T22:30:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.051+0000 7f2af83a4700 1 -- start start 2023-09-30T22:30:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af83a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af83a4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 0x7f2af0133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af83a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af83a4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af006bf30 con 0x7f2af009f160 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af83a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af009ffb0 con 0x7f2af009e850 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af83a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2af006cb10 con 0x7f2af009f5b0 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af593f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 0x7f2af0133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af593f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 0x7f2af0133bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46204/0 (socket says 172.21.15.32:46204) 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af593f700 1 -- 172.21.15.32:0/951156005 learned_addr learned my addr 172.21.15.32:0/951156005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af6140700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.052+0000 7f2af6941700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6140700 1 -- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 msgr2=0x7f2af0134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6140700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0134130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6140700 1 -- 172.21.15.32:0/951156005 >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 msgr2=0x7f2af0133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6140700 1 --2- 172.21.15.32:0/951156005 >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 0x7f2af0133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6140700 1 -- 172.21.15.32:0/951156005 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00a0270 con 0x7f2af009e850 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af593f700 1 --2- 172.21.15.32:0/951156005 >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 0x7f2af0133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6941700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af6140700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af009ec20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009840 tx=0x7f2aec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4d26d1132cea5e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af513e700 1 -- 172.21.15.32:0/951156005 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec011830 con 0x7f2af009e850 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af513e700 1 -- 172.21.15.32:0/951156005 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec011990 con 0x7f2af009e850 2023-09-30T22:30:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.053+0000 7f2af513e700 1 -- 172.21.15.32:0/951156005 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec011d20 con 0x7f2af009e850 2023-09-30T22:30:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 -- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 msgr2=0x7f2af009ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af009ec20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009840 tx=0x7f2aec009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 -- 172.21.15.32:0/951156005 shutdown_connections 2023-09-30T22:30:48.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af009ec20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 --2- 172.21.15.32:0/951156005 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0134130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 --2- 172.21.15.32:0/951156005 >> v2:172.21.15.32:3300/0 conn(0x7f2af009f160 0x7f2af0133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 -- 172.21.15.32:0/951156005 >> 172.21.15.32:0/951156005 conn(0x7f2af00a3f70 msgr2=0x7f2af00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 -- 172.21.15.32:0/951156005 shutdown_connections 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.054+0000 7f2af83a4700 1 -- 172.21.15.32:0/951156005 wait complete. 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.055+0000 7f2af83a4700 1 Processor -- start 2023-09-30T22:30:48.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.055+0000 7f2af83a4700 1 -- start start 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.055+0000 7f2af83a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af0122890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af83a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 0x7f2af0122dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af83a4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af83a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01244a0 con 0x7f2af009f160 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af83a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0125e10 con 0x7f2af009e850 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af83a4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0133420 con 0x7f2af009f5b0 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 0x7f2af0122dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af6140700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af0122890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af6140700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af0122890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48074/0 (socket says 172.21.15.32:48074) 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 0x7f2af0122dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46210/0 (socket says 172.21.15.32:46210) 2023-09-30T22:30:48.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af6140700 1 -- 172.21.15.32:0/176380796 learned_addr learned my addr 172.21.15.32:0/176380796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 -- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 msgr2=0x7f2af0123310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 -- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 msgr2=0x7f2af0122890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af0122890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 -- 172.21.15.32:0/176380796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae0009550 con 0x7f2af009f160 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af6140700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af0122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.056+0000 7f2af593f700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 0x7f2af0122dd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000e9a0 tx=0x7f2ae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.057+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0004240 con 0x7f2af009f160 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.057+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae00043a0 con 0x7f2af009f160 2023-09-30T22:30:48.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.057+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00036a0 con 0x7f2af009f160 2023-09-30T22:30:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.057+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae8009a70 con 0x7f2af009f160 2023-09-30T22:30:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.057+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec0094a0 con 0x7f2af009f160 2023-09-30T22:30:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.058+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2ae0003800 con 0x7f2af009f160 2023-09-30T22:30:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.058+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2ae003f860 con 0x7f2af009f160 2023-09-30T22:30:48.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.058+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af006cd90 con 0x7f2af009f160 2023-09-30T22:30:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.063+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ae000c640 con 0x7f2af009f160 2023-09-30T22:30:48.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 Processor -- start 2023-09-30T22:30:48.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 -- start start 2023-09-30T22:30:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fd0133a80 0x7f7fd0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd0130a70 0x7f7fd013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd006bc00 con 0x7f7fd012ff20 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd006c570 con 0x7f7fd0133a80 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fd5da4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00a0040 con 0x7f7fd0130a70 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fceffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7fd012ff20 0x7f7fd0130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48230/0 (socket says 172.21.15.32:48230) 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.087+0000 7f7fcf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7fd0133a80 0x7f7fd0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fceffd700 1 -- 172.21.15.32:0/34452001 learned_addr learned my addr 172.21.15.32:0/34452001 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcffff700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd0130a70 0x7f7fd013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcf7fe700 1 -- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd0130a70 msgr2=0x7f7fd013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcf7fe700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd0130a70 0x7f7fd013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcf7fe700 1 -- 172.21.15.32:0/34452001 >> v1:172.21.15.32:6789/0 conn(0x7f7fd012ff20 legacy=0x7f7fd0130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fce7fc700 1 -- 172.21.15.32:0/34452001 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433223370 0 0) 0x7f7fd006bc00 con 0x7f7fd012ff20 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcf7fe700 1 -- 172.21.15.32:0/34452001 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0133860 con 0x7f7fd0133a80 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcffff700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd0130a70 0x7f7fd013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fcf7fe700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd0133e50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009840 tx=0x7f7fc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257f128bdd396895 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fce7fc700 1 -- 172.21.15.32:0/34452001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0011830 con 0x7f7fd0133a80 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fce7fc700 1 -- 172.21.15.32:0/34452001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0011990 con 0x7f7fd0133a80 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.088+0000 7f7fce7fc700 1 -- 172.21.15.32:0/34452001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0011d20 con 0x7f7fd0133a80 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 -- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 msgr2=0x7f7fd0133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd0133e50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009840 tx=0x7f7fc0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 -- 172.21.15.32:0/34452001 shutdown_connections 2023-09-30T22:30:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd0130a70 0x7f7fd013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/34452001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd0133e50 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 -- 172.21.15.32:0/34452001 >> 172.21.15.32:0/34452001 conn(0x7f7fd00a4210 msgr2=0x7f7fd00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 -- 172.21.15.32:0/34452001 shutdown_connections 2023-09-30T22:30:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.089+0000 7f7fd5da4700 1 -- 172.21.15.32:0/34452001 wait complete. 2023-09-30T22:30:48.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.091+0000 7f7fd5da4700 1 Processor -- start 2023-09-30T22:30:48.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.091+0000 7f7fd5da4700 1 -- start start 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fd5da4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 0x7f7fd009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fd5da4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7fd0130a70 0x7f7fd009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fd5da4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fd5da4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd011f180 con 0x7f7fd0133a80 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fd5da4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd012da40 con 0x7f7fd0130a70 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fd5da4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd012dc30 con 0x7f7fd012ff20 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fcf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 0x7f7fd009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fceffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7fd0130a70 0x7f7fd009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fcf7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 0x7f7fd009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48406/0 (socket says 172.21.15.32:48406) 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fcf7fe700 1 -- 172.21.15.32:0/3827324116 learned_addr learned my addr 172.21.15.32:0/3827324116 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:48.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fcffff700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fceffd700 1 -- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 msgr2=0x7f7fd009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fceffd700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 0x7f7fd009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fceffd700 1 -- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 msgr2=0x7f7fd009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fceffd700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fceffd700 1 -- 172.21.15.32:0/3827324116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8009a70 con 0x7f7fd0130a70 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.092+0000 7f7fcf7fe700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 0x7f7fd009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.093+0000 7f7fceffd700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7fd0130a70 0x7f7fd009c9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c470 tx=0x7f7fc400c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.093+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4004c80 con 0x7f7fd0130a70 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.093+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc4004de0 con 0x7f7fd0130a70 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.093+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc00094a0 con 0x7f7fd0130a70 2023-09-30T22:30:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.093+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4007310 con 0x7f7fd0130a70 2023-09-30T22:30:48.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.093+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd00a21a0 con 0x7f7fd0130a70 2023-09-30T22:30:48.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.094+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7fc4007470 con 0x7f7fd0130a70 2023-09-30T22:30:48.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.094+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7fc403f220 con 0x7f7fd0130a70 2023-09-30T22:30:48.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.094+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd0074ea0 con 0x7f7fd0130a70 2023-09-30T22:30:48.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.103+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7fc4007720 con 0x7f7fd0130a70 2023-09-30T22:30:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.134+0000 7f98ce89a700 1 Processor -- start 2023-09-30T22:30:48.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.134+0000 7f98ce89a700 1 -- start start 2023-09-30T22:30:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98ce89a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a1510 0x7f98c8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98ce89a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98ce89a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c809edd0 con 0x7f98c80a0a30 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98ce89a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80a18e0 con 0x7f98c8139250 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98ce89a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c809e1f0 con 0x7f98c80a1510 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98c7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f98c80a0a30 0x7f98c80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48238/0 (socket says 172.21.15.32:48238) 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98c7fff700 1 -- 172.21.15.32:0/251167808 learned_addr learned my addr 172.21.15.32:0/251167808 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98c77fe700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a1510 0x7f98c8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98cce37700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98c6ffd700 1 -- 172.21.15.32:0/251167808 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146722003 0 0) 0x7f98c809edd0 con 0x7f98c80a0a30 2023-09-30T22:30:48.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.135+0000 7f98c6ffd700 1 -- 172.21.15.32:0/251167808 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98b00035b0 con 0x7f98c80a0a30 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c77fe700 1 -- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 msgr2=0x7f98c813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c77fe700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c77fe700 1 -- 172.21.15.32:0/251167808 >> v1:172.21.15.32:6789/0 conn(0x7f98c80a0a30 legacy=0x7f98c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c77fe700 1 -- 172.21.15.32:0/251167808 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c809dc50 con 0x7f98c80a1510 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98cce37700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c77fe700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a1510 0x7f98c8138d10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009870 tx=0x7f98bc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6164f8a0d5ba70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c6ffd700 1 -- 172.21.15.32:0/251167808 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc011830 con 0x7f98c80a1510 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c6ffd700 1 -- 172.21.15.32:0/251167808 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc011990 con 0x7f98c80a1510 2023-09-30T22:30:48.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.136+0000 7f98c6ffd700 1 -- 172.21.15.32:0/251167808 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc011d20 con 0x7f98c80a1510 2023-09-30T22:30:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 -- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a1510 msgr2=0x7f98c8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a1510 0x7f98c8138d10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009870 tx=0x7f98bc009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:48.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 -- 172.21.15.32:0/251167808 shutdown_connections 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 --2- 172.21.15.32:0/251167808 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a1510 0x7f98c8138d10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 -- 172.21.15.32:0/251167808 >> 172.21.15.32:0/251167808 conn(0x7f98c80a6550 msgr2=0x7f98c80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 -- 172.21.15.32:0/251167808 shutdown_connections 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.137+0000 7f98ce89a700 1 -- 172.21.15.32:0/251167808 wait complete. 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.138+0000 7f98ce89a700 1 Processor -- start 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 -- start start 2023-09-30T22:30:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a0a30 0x7f98c81216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c80a1510 0x7f98c8121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c8122120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8123260 con 0x7f98c80a1510 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8137540 con 0x7f98c8139250 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.139+0000 7f98ce89a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8123aa0 con 0x7f98c80a0a30 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c80a1510 0x7f98c8121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c8122120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c80a1510 0x7f98c8121be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46234/0 (socket says 172.21.15.32:46234) 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c8122120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48098/0 (socket says 172.21.15.32:48098) 2023-09-30T22:30:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c77fe700 1 -- 172.21.15.32:0/3673987261 learned_addr learned my addr 172.21.15.32:0/3673987261 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c7fff700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a0a30 0x7f98c81216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 -- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a0a30 msgr2=0x7f98c81216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a0a30 0x7f98c81216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 -- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c80a1510 msgr2=0x7f98c8121be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c80a1510 0x7f98c8121be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 -- 172.21.15.32:0/3673987261 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc0094a0 con 0x7f98c8139250 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c7fff700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a0a30 0x7f98c81216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98cce37700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c8122120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f98c0007f20 tx=0x7f98c000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000dde0 con 0x7f98c8139250 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000f040 con 0x7f98c8139250 2023-09-30T22:30:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c8124900 con 0x7f98c8139250 2023-09-30T22:30:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0003c60 con 0x7f98c8139250 2023-09-30T22:30:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.140+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c80a0220 con 0x7f98c8139250 2023-09-30T22:30:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.141+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98c0004030 con 0x7f98c8139250 2023-09-30T22:30:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.141+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f98c003f740 con 0x7f98c8139250 2023-09-30T22:30:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.142+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c8074ea0 con 0x7f98c8139250 2023-09-30T22:30:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.151+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98c001a080 con 0x7f98c8139250 2023-09-30T22:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.279+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2af00747f0 con 0x7f2af009f160 2023-09-30T22:30:48.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.279+0000 7f2ae6ffd700 1 -- 172.21.15.32:0/176380796 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ae00123d0 con 0x7f2af009f160 2023-09-30T22:30:48.280 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 msgr2=0x7f2af0122dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 0x7f2af0122dd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000e9a0 tx=0x7f2ae000ecb0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 shutdown_connections 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2af009f5b0 0x7f2af0123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af009e850 0x7f2af0122890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 --2- 172.21.15.32:0/176380796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af009f160 0x7f2af0122dd0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 >> 172.21.15.32:0/176380796 conn(0x7f2af00a3f70 msgr2=0x7f2af00a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 shutdown_connections 2023-09-30T22:30:48.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.282+0000 7f2af83a4700 1 -- 172.21.15.32:0/176380796 wait complete. 2023-09-30T22:30:48.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.321+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7fd011ff70 con 0x7f7fd0130a70 2023-09-30T22:30:48.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.321+0000 7f7fb7fff700 1 -- 172.21.15.32:0/3827324116 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fc4011e80 con 0x7f7fd0130a70 2023-09-30T22:30:48.322 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.323+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7fd0130a70 msgr2=0x7f7fd009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.323+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7fd0130a70 0x7f7fd009c9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c470 tx=0x7f7fc400c780 comp rx=0 tx=0).stop 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 shutdown_connections 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7fd0133a80 0x7f7fd009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7fd012ff20 0x7f7fd009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 --2- 172.21.15.32:0/3827324116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7fd0130a70 0x7f7fd009c9c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 >> 172.21.15.32:0/3827324116 conn(0x7f7fd00a4210 msgr2=0x7f7fd009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 shutdown_connections 2023-09-30T22:30:48.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.324+0000 7f7fd5da4700 1 -- 172.21.15.32:0/3827324116 wait complete. 2023-09-30T22:30:48.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.358+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98c80a28c0 con 0x7f98c8139250 2023-09-30T22:30:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.358+0000 7f98c4ff9700 1 -- 172.21.15.32:0/3673987261 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98c001e3b0 con 0x7f98c8139250 2023-09-30T22:30:48.360 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 msgr2=0x7f98c8122120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c8122120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f98c0007f20 tx=0x7f98c000d3b0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 shutdown_connections 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f98c8139250 0x7f98c8122120 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f98c80a0a30 0x7f98c81216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 --2- 172.21.15.32:0/3673987261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98c80a1510 0x7f98c8121be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.360+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 >> 172.21.15.32:0/3673987261 conn(0x7f98c80a6550 msgr2=0x7f98c8137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.361+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 shutdown_connections 2023-09-30T22:30:48.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.361+0000 7f98ce89a700 1 -- 172.21.15.32:0/3673987261 wait complete. 2023-09-30T22:30:48.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:48.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.659+0000 7f20b602f700 1 Processor -- start 2023-09-30T22:30:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 -- start start 2023-09-30T22:30:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 0x7f20b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 0x7f20b0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 0x7f20b0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b009f520 con 0x7f20b009d6f0 2023-09-30T22:30:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b009f130 con 0x7f20b009e000 2023-09-30T22:30:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20b602f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b009e930 con 0x7f20b009e410 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20affff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 0x7f20b0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20aeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 0x7f20b0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20af7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 0x7f20b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20affff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 0x7f20b0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48428/0 (socket says 172.21.15.32:48428) 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20aeffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 0x7f20b0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48114/0 (socket says 172.21.15.32:48114) 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20af7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 0x7f20b009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46244/0 (socket says 172.21.15.32:46244) 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.660+0000 7f20affff700 1 -- 172.21.15.32:0/3636772674 learned_addr learned my addr 172.21.15.32:0/3636772674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20affff700 1 -- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 msgr2=0x7f20b0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20affff700 1 --2- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 0x7f20b0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20affff700 1 -- 172.21.15.32:0/3636772674 >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 msgr2=0x7f20b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20affff700 1 --2- 172.21.15.32:0/3636772674 >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 0x7f20b009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20affff700 1 -- 172.21.15.32:0/3636772674 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b009b2d0 con 0x7f20b009e410 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20aeffd700 1 --2- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 0x7f20b0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20af7fe700 1 --2- 172.21.15.32:0/3636772674 >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 0x7f20b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20affff700 1 --2- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 0x7f20b0130ba0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f20a800bf20 tx=0x7f20a80093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0937b41c2bd71d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20ae7fc700 1 -- 172.21.15.32:0/3636772674 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a800b4e0 con 0x7f20b009e410 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20ae7fc700 1 -- 172.21.15.32:0/3636772674 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a800c910 con 0x7f20b009e410 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.661+0000 7f20ae7fc700 1 -- 172.21.15.32:0/3636772674 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a800ccc0 con 0x7f20b009e410 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 -- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 msgr2=0x7f20b0130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 --2- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 0x7f20b0130ba0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f20a800bf20 tx=0x7f20a80093b0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 -- 172.21.15.32:0/3636772674 shutdown_connections 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 --2- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e410 0x7f20b0130ba0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 --2- 172.21.15.32:0/3636772674 >> v2:172.21.15.32:3300/0 conn(0x7f20b009d6f0 0x7f20b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 --2- 172.21.15.32:0/3636772674 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e000 0x7f20b0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 -- 172.21.15.32:0/3636772674 >> 172.21.15.32:0/3636772674 conn(0x7f20b0066f50 msgr2=0x7f20b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 -- 172.21.15.32:0/3636772674 shutdown_connections 2023-09-30T22:30:48.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.662+0000 7f20b602f700 1 -- 172.21.15.32:0/3636772674 wait complete. 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.663+0000 7f20b602f700 1 Processor -- start 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.663+0000 7f20b602f700 1 -- start start 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.663+0000 7f20b602f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b009d6f0 0x7f20b011e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20b602f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 0x7f20b011e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20b602f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e410 0x7f20b011eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20b602f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0130410 con 0x7f20b009d6f0 2023-09-30T22:30:48.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20b602f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0120260 con 0x7f20b009e410 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20b602f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0120e90 con 0x7f20b009e000 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20aeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 0x7f20b011e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b009d6f0 0x7f20b011e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20aeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 0x7f20b011e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48442/0 (socket says 172.21.15.32:48442) 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20aeffd700 1 -- 172.21.15.32:0/2143569612 learned_addr learned my addr 172.21.15.32:0/2143569612 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20affff700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e410 0x7f20b011eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 -- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 msgr2=0x7f20b011e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 0x7f20b011e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 -- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e410 msgr2=0x7f20b011eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e410 0x7f20b011eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 -- 172.21.15.32:0/2143569612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00094a0 con 0x7f20b009d6f0 2023-09-30T22:30:48.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20aeffd700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 0x7f20b011e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20af7fe700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b009d6f0 0x7f20b011e030 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e9b0 tx=0x7f20a000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a000ee50 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a0004030 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a400d940 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.664+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00043c0 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.665+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a800bb50 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.665+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20a000c7f0 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.665+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f20a003f830 con 0x7f20b009d6f0 2023-09-30T22:30:48.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.666+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b009c700 con 0x7f20b009d6f0 2023-09-30T22:30:48.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.671+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20a0020080 con 0x7f20b009d6f0 2023-09-30T22:30:48.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.875+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f20b0074ea0 con 0x7f20b009d6f0 2023-09-30T22:30:49.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.876+0000 7f2097fff700 1 -- 172.21.15.32:0/2143569612 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20a0011db0 con 0x7f20b009d6f0 2023-09-30T22:30:49.053 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b009d6f0 msgr2=0x7f20b011e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b009d6f0 0x7f20b011e030 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e9b0 tx=0x7f20a000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 shutdown_connections 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f20b009e410 0x7f20b011eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20b009d6f0 0x7f20b011e030 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 --2- 172.21.15.32:0/2143569612 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f20b009e000 0x7f20b011e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 >> 172.21.15.32:0/2143569612 conn(0x7f20b0066f50 msgr2=0x7f20b006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 shutdown_connections 2023-09-30T22:30:49.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:48.878+0000 7f20b602f700 1 -- 172.21.15.32:0/2143569612 wait complete. 2023-09-30T22:30:49.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:49.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:49.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:49.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:49.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.526+0000 7f656c540700 1 Processor -- start 2023-09-30T22:30:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.526+0000 7f601030a700 1 Processor -- start 2023-09-30T22:30:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.526+0000 7f601030a700 1 -- start start 2023-09-30T22:30:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.526+0000 7f656c540700 1 -- start start 2023-09-30T22:30:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656c540700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f656409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656c540700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e000 0x7f6564130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656c540700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656409f130 con 0x7f656409e410 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656c540700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656409f520 con 0x7f656409d6f0 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656c540700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f656409e930 con 0x7f656409e000 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f601030a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60081338a0 0x7f6008133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f601030a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809e930 0x7f600813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f601030a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600806ef30 con 0x7f600809de50 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f601030a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600806fef0 con 0x7f60081338a0 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f601030a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60081334b0 con 0x7f600809e930 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656aadd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f656409e410 0x7f6564130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48250/0 (socket says 172.21.15.32:48250) 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f656aadd700 1 -- 172.21.15.32:0/705894686 learned_addr learned my addr 172.21.15.32:0/705894686 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f600d8a5700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f600809de50 0x7f600809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48262/0 (socket says 172.21.15.32:48262) 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f600d8a5700 1 -- 172.21.15.32:0/1845135156 learned_addr learned my addr 172.21.15.32:0/1845135156 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f600e0a6700 1 --2- 172.21.15.32:0/1845135156 >> v2:172.21.15.32:3300/0 conn(0x7f60081338a0 0x7f6008133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f6569adb700 1 --2- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e000 0x7f6564130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f600d0a4700 1 -- 172.21.15.32:0/1845135156 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335918696 0 0) 0x7f600806ef30 con 0x7f600809de50 2023-09-30T22:30:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f65692da700 1 -- 172.21.15.32:0/705894686 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578932262 0 0) 0x7f656409f130 con 0x7f656409e410 2023-09-30T22:30:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f65692da700 1 -- 172.21.15.32:0/705894686 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65500057c0 con 0x7f656409e410 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.527+0000 7f600d0a4700 1 -- 172.21.15.32:0/1845135156 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff40035b0 con 0x7f600809de50 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600e0a6700 1 -- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809e930 msgr2=0x7f600813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600e0a6700 1 --2- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809e930 0x7f600813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600e0a6700 1 -- 172.21.15.32:0/1845135156 >> v1:172.21.15.32:6789/0 conn(0x7f600809de50 legacy=0x7f600809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600e0a6700 1 -- 172.21.15.32:0/1845135156 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600806fc30 con 0x7f60081338a0 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f6569adb700 1 -- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 msgr2=0x7f656409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:49.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f6569adb700 1 --2- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f656409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f6569adb700 1 -- 172.21.15.32:0/705894686 >> v1:172.21.15.32:6789/0 conn(0x7f656409e410 legacy=0x7f6564130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f65692da700 1 -- 172.21.15.32:0/705894686 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 461801105 0 0) 0x7f65500057c0 con 0x7f656409e410 2023-09-30T22:30:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f6569adb700 1 -- 172.21.15.32:0/705894686 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656409b2d0 con 0x7f656409e000 2023-09-30T22:30:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600e0a6700 1 --2- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60081338a0 0x7f6008133c70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6004009840 tx=0x7f6004009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939f56eab58bc3e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f6569adb700 1 --2- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e000 0x7f6564130660 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f65540098a0 tx=0x7f6554009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f27e9f80e443d5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600d0a4700 1 -- 172.21.15.32:0/1845135156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004011830 con 0x7f60081338a0 2023-09-30T22:30:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600d0a4700 1 -- 172.21.15.32:0/1845135156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004011990 con 0x7f60081338a0 2023-09-30T22:30:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f600d0a4700 1 -- 172.21.15.32:0/1845135156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004011d20 con 0x7f60081338a0 2023-09-30T22:30:49.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f65692da700 1 -- 172.21.15.32:0/705894686 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0019f0 con 0x7f656409e000 2023-09-30T22:30:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f65692da700 1 -- 172.21.15.32:0/705894686 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554004030 con 0x7f656409e000 2023-09-30T22:30:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.528+0000 7f65692da700 1 -- 172.21.15.32:0/705894686 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65540043c0 con 0x7f656409e000 2023-09-30T22:30:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 -- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60081338a0 msgr2=0x7f6008133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 --2- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60081338a0 0x7f6008133c70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6004009840 tx=0x7f6004009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:49.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 -- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e000 msgr2=0x7f6564130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 --2- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e000 0x7f6564130660 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f65540098a0 tx=0x7f6554009bb0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 -- 172.21.15.32:0/1845135156 shutdown_connections 2023-09-30T22:30:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 --2- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809e930 0x7f600813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 --2- 172.21.15.32:0/1845135156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60081338a0 0x7f6008133c70 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 -- 172.21.15.32:0/1845135156 >> 172.21.15.32:0/1845135156 conn(0x7f60080a2aa0 msgr2=0x7f60080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 -- 172.21.15.32:0/705894686 shutdown_connections 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 --2- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f656409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 --2- 172.21.15.32:0/705894686 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e000 0x7f6564130660 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 -- 172.21.15.32:0/705894686 >> 172.21.15.32:0/705894686 conn(0x7f6564066f50 msgr2=0x7f656409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 -- 172.21.15.32:0/705894686 shutdown_connections 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 -- 172.21.15.32:0/1845135156 shutdown_connections 2023-09-30T22:30:49.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f601030a700 1 -- 172.21.15.32:0/1845135156 wait complete. 2023-09-30T22:30:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.529+0000 7f656c540700 1 -- 172.21.15.32:0/705894686 wait complete. 2023-09-30T22:30:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.531+0000 7f656c540700 1 Processor -- start 2023-09-30T22:30:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.531+0000 7f601030a700 1 Processor -- start 2023-09-30T22:30:49.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.531+0000 7f601030a700 1 -- start start 2023-09-30T22:30:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.531+0000 7f656c540700 1 -- start start 2023-09-30T22:30:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656c540700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f6564122330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656c540700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656409e000 0x7f6564122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f601030a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809de50 0x7f600811a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f601030a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 0x7f600811aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656c540700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e410 0x7f6564122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656c540700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6564125940 con 0x7f656409e000 2023-09-30T22:30:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656c540700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6564123790 con 0x7f656409d6f0 2023-09-30T22:30:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656c540700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f656409c700 con 0x7f656409e410 2023-09-30T22:30:49.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f6569adb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656409e000 0x7f6564122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656a2dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f6564122330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f6569adb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656409e000 0x7f6564122870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46266/0 (socket says 172.21.15.32:46266) 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f6569adb700 1 -- 172.21.15.32:0/3647750462 learned_addr learned my addr 172.21.15.32:0/3647750462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656aadd700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e410 0x7f6564122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f601030a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60081338a0 0x7f600811b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f601030a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600811ded0 con 0x7f600809e930 2023-09-30T22:30:49.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f601030a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600811e0d0 con 0x7f600809de50 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f601030a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f600811cfc0 con 0x7f60081338a0 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 0x7f600811aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600e8a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60081338a0 0x7f600811b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 0x7f600811aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46254/0 (socket says 172.21.15.32:46254) 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 -- 172.21.15.32:0/3240834078 learned_addr learned my addr 172.21.15.32:0/3240834078 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600e0a6700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809de50 0x7f600811a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 -- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60081338a0 msgr2=0x7f600811b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60081338a0 0x7f600811b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 -- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809de50 msgr2=0x7f600811a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809de50 0x7f600811a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f600d8a5700 1 -- 172.21.15.32:0/3240834078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040094a0 con 0x7f600809e930 2023-09-30T22:30:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656a2dc700 1 -- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e410 msgr2=0x7f6564122db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656a2dc700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e410 0x7f6564122db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656a2dc700 1 -- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656409e000 msgr2=0x7f6564122870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656a2dc700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656409e000 0x7f6564122870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.532+0000 7f656a2dc700 1 -- 172.21.15.32:0/3647750462 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540094a0 con 0x7f656409d6f0 2023-09-30T22:30:49.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f600d8a5700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 0x7f600811aaf0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8001b20 tx=0x7f5ff8014050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f656a2dc700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f6564122330 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f656000b480 tx=0x7f656000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8017070 con 0x7f600809e930 2023-09-30T22:30:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff8005b70 con 0x7f600809e930 2023-09-30T22:30:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80120d0 con 0x7f600809e930 2023-09-30T22:30:49.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560011830 con 0x7f656409d6f0 2023-09-30T22:30:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600811c260 con 0x7f600809e930 2023-09-30T22:30:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600811d4e0 con 0x7f600809e930 2023-09-30T22:30:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560011990 con 0x7f656409d6f0 2023-09-30T22:30:49.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.533+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560011d40 con 0x7f656409d6f0 2023-09-30T22:30:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.534+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ff8012230 con 0x7f600809e930 2023-09-30T22:30:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.534+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65641247a0 con 0x7f656409d6f0 2023-09-30T22:30:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.534+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6564123f40 con 0x7f656409d6f0 2023-09-30T22:30:49.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.534+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ff8047020 con 0x7f600809e930 2023-09-30T22:30:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.534+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600811d480 con 0x7f600809e930 2023-09-30T22:30:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.535+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f656001a400 con 0x7f656409d6f0 2023-09-30T22:30:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.535+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f656003f510 con 0x7f656409d6f0 2023-09-30T22:30:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.535+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6564074ea0 con 0x7f656409d6f0 2023-09-30T22:30:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.540+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ff8014800 con 0x7f600809e930 2023-09-30T22:30:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.540+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f656000f2e0 con 0x7f656409d6f0 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.567+0000 7f72c56bf700 1 Processor -- start 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.567+0000 7f72c56bf700 1 -- start start 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72c56bf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72c56bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72c56bf700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c009e410 0x7f72c0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72c56bf700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c009f520 con 0x7f72c009e410 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72c56bf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c009f130 con 0x7f72c009d6f0 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72c56bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72c009e930 con 0x7f72c009e000 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72bf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c009e410 0x7f72c0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72be7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72bf7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c009e410 0x7f72c0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46276/0 (socket says 172.21.15.32:46276) 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72bf7fe700 1 -- 172.21.15.32:0/4009815390 learned_addr learned my addr 172.21.15.32:0/4009815390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72beffd700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72be7fc700 1 -- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 msgr2=0x7f72c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72be7fc700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72be7fc700 1 -- 172.21.15.32:0/4009815390 >> v2:172.21.15.32:3300/0 conn(0x7f72c009e410 msgr2=0x7f72c0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72be7fc700 1 --2- 172.21.15.32:0/4009815390 >> v2:172.21.15.32:3300/0 conn(0x7f72c009e410 0x7f72c0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.568+0000 7f72be7fc700 1 -- 172.21.15.32:0/4009815390 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c009b2d0 con 0x7f72c009e000 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.569+0000 7f72beffd700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.569+0000 7f72be7fc700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c0130660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c249651fb7f9b162 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.569+0000 7f72bdffb700 1 -- 172.21.15.32:0/4009815390 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4011830 con 0x7f72c009e000 2023-09-30T22:30:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.569+0000 7f72bdffb700 1 -- 172.21.15.32:0/4009815390 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b4011990 con 0x7f72c009e000 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.569+0000 7f72bdffb700 1 -- 172.21.15.32:0/4009815390 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4011d20 con 0x7f72c009e000 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 -- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 msgr2=0x7f72c0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c0130660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 -- 172.21.15.32:0/4009815390 shutdown_connections 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 --2- 172.21.15.32:0/4009815390 >> v2:172.21.15.32:3300/0 conn(0x7f72c009e410 0x7f72c0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 --2- 172.21.15.32:0/4009815390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c0130660 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 -- 172.21.15.32:0/4009815390 >> 172.21.15.32:0/4009815390 conn(0x7f72c0066f50 msgr2=0x7f72c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 -- 172.21.15.32:0/4009815390 shutdown_connections 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.570+0000 7f72c56bf700 1 -- 172.21.15.32:0/4009815390 wait complete. 2023-09-30T22:30:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.571+0000 7f72c56bf700 1 Processor -- start 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.571+0000 7f72c56bf700 1 -- start start 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72c56bf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c011dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72c56bf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c011e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72c56bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 0x7f72c011ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72c56bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0121810 con 0x7f72c009e410 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72c56bf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c011fb40 con 0x7f72c009d6f0 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72c56bf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0120160 con 0x7f72c009e000 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72bf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 0x7f72c011ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72beffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c011dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72bf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 0x7f72c011ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46284/0 (socket says 172.21.15.32:46284) 2023-09-30T22:30:49.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72bf7fe700 1 -- 172.21.15.32:0/2989070191 learned_addr learned my addr 172.21.15.32:0/2989070191 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.572+0000 7f72be7fc700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c011e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72bf7fe700 1 -- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 msgr2=0x7f72c011e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c011e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72bf7fe700 1 -- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 msgr2=0x7f72c011dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c011dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72bf7fe700 1 -- 172.21.15.32:0/2989070191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00094a0 con 0x7f72c009e410 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72beffd700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c011dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72be7fc700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c011e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72bf7fe700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 0x7f72c011ea50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e140 tx=0x7f72b800e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8003a20 con 0x7f72c009e410 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b8010050 con 0x7f72c009e410 2023-09-30T22:30:49.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8003da0 con 0x7f72c009e410 2023-09-30T22:30:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b40094a0 con 0x7f72c009e410 2023-09-30T22:30:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.573+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c00a16b0 con 0x7f72c009e410 2023-09-30T22:30:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.575+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c011ac20 con 0x7f72c009e410 2023-09-30T22:30:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.575+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72b800ea50 con 0x7f72c009e410 2023-09-30T22:30:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.576+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f72b8041ca0 con 0x7f72c009e410 2023-09-30T22:30:49.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.584+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72b801e770 con 0x7f72c009e410 2023-09-30T22:30:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.754+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f656406c360 con 0x7f656409d6f0 2023-09-30T22:30:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.754+0000 7f655affd700 1 -- 172.21.15.32:0/3647750462 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6560028070 con 0x7f656409d6f0 2023-09-30T22:30:49.756 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:49.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.755+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6008074ea0 con 0x7f600809e930 2023-09-30T22:30:49.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.756+0000 7f5ffeffd700 1 -- 172.21.15.32:0/3240834078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ff801ab20 con 0x7f600809e930 2023-09-30T22:30:49.757 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 msgr2=0x7f6564122330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f6564122330 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f656000b480 tx=0x7f656000b790 comp rx=0 tx=0).stop 2023-09-30T22:30:49.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 shutdown_connections 2023-09-30T22:30:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f656409e410 0x7f6564122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f656409d6f0 0x7f6564122330 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 --2- 172.21.15.32:0/3647750462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f656409e000 0x7f6564122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 >> 172.21.15.32:0/3647750462 conn(0x7f6564066f50 msgr2=0x7f6564130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 shutdown_connections 2023-09-30T22:30:49.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.757+0000 7f656c540700 1 -- 172.21.15.32:0/3647750462 wait complete. 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 msgr2=0x7f600811aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 0x7f600811aaf0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8001b20 tx=0x7f5ff8014050 comp rx=0 tx=0).stop 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 shutdown_connections 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60081338a0 0x7f600811b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f600809de50 0x7f600811a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 --2- 172.21.15.32:0/3240834078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f600809e930 0x7f600811aaf0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 >> 172.21.15.32:0/3240834078 conn(0x7f60080a2aa0 msgr2=0x7f6008130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.758+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 shutdown_connections 2023-09-30T22:30:49.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.759+0000 7f601030a700 1 -- 172.21.15.32:0/3240834078 wait complete. 2023-09-30T22:30:49.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:49.774 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:49.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.792+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f72c0074ea0 con 0x7f72c009e410 2023-09-30T22:30:49.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.793+0000 7f72a77fe700 1 -- 172.21.15.32:0/2989070191 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72b801b020 con 0x7f72c009e410 2023-09-30T22:30:49.794 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:49.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 msgr2=0x7f72c011ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 0x7f72c011ea50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e140 tx=0x7f72b800e450 comp rx=0 tx=0).stop 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 shutdown_connections 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c009e410 0x7f72c011ea50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72c009d6f0 0x7f72c011dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 --2- 172.21.15.32:0/2989070191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72c009e000 0x7f72c011e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 >> 172.21.15.32:0/2989070191 conn(0x7f72c0066f50 msgr2=0x7f72c0130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 shutdown_connections 2023-09-30T22:30:49.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:49.795+0000 7f72c56bf700 1 -- 172.21.15.32:0/2989070191 wait complete. 2023-09-30T22:30:49.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:50.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:50.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.370+0000 7fb8f8f35700 1 Processor -- start 2023-09-30T22:30:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.370+0000 7fb8f8f35700 1 -- start start 2023-09-30T22:30:50.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:51.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:51.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f8f35700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 0x7fb8f406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f8f35700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f8f35700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8f4130260 0x7fb8f4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f8f35700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f412d330 con 0x7fb8f4130260 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f8f35700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f412cd90 con 0x7fb8f412f8b0 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f8f35700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41306d0 con 0x7fb8f412cf60 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 0x7fb8f406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f2d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8f4130260 0x7fb8f4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:48198/0 (socket says 172.21.15.32:48198) 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 0x7fb8f406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:48508/0 (socket says 172.21.15.32:48508) 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f1d9b700 1 -- 172.21.15.32:0/4078456932 learned_addr learned my addr 172.21.15.32:0/4078456932 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f2d9d700 1 -- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 msgr2=0x7fb8f406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f2d9d700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 0x7fb8f406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f2d9d700 1 -- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 msgr2=0x7fb8f412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f2d9d700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f2d9d700 1 -- 172.21.15.32:0/4078456932 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4071780 con 0x7fb8f4130260 2023-09-30T22:30:51.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f1d9b700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.371+0000 7fb8f259c700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 0x7fb8f406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.372+0000 7fb8f2d9d700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f4130260 0x7fb8f4136c20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec014a30 tx=0x7fb8ec014d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16a3f0426f80654 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.372+0000 7fb8f159a700 1 -- 172.21.15.32:0/4078456932 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00fd50 con 0x7fb8f4130260 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.372+0000 7fb8f159a700 1 -- 172.21.15.32:0/4078456932 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec0138a0 con 0x7fb8f4130260 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.372+0000 7fb8f159a700 1 -- 172.21.15.32:0/4078456932 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec013c50 con 0x7fb8f4130260 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.372+0000 7fb8f8f35700 1 -- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f4130260 msgr2=0x7fb8f4136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.372+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f4130260 0x7fb8f4136c20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec014a30 tx=0x7fb8ec014d40 comp rx=0 tx=0).stop 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 -- 172.21.15.32:0/4078456932 shutdown_connections 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f4130260 0x7fb8f4136c20 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f412cf60 0x7fb8f406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/4078456932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 -- 172.21.15.32:0/4078456932 >> 172.21.15.32:0/4078456932 conn(0x7fb8f40a2aa0 msgr2=0x7fb8f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 -- 172.21.15.32:0/4078456932 shutdown_connections 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.373+0000 7fb8f8f35700 1 -- 172.21.15.32:0/4078456932 wait complete. 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 Processor -- start 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 -- start start 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 0x7fb8f41c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f41c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f4130260 0x7fb8f41c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f409e440 con 0x7fb8f412cf60 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f4129d00 con 0x7fb8f412f8b0 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f8f35700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f409eb30 con 0x7fb8f4130260 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 0x7fb8f41c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 0x7fb8f41c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46298/0 (socket says 172.21.15.32:46298) 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.374+0000 7fb8f259c700 1 -- 172.21.15.32:0/204360684 learned_addr learned my addr 172.21.15.32:0/204360684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f2d9d700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f4130260 0x7fb8f41c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f1d9b700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f41c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f259c700 1 -- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f4130260 msgr2=0x7fb8f41c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f259c700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f4130260 0x7fb8f41c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f259c700 1 -- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 msgr2=0x7fb8f41c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f259c700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f41c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f259c700 1 -- 172.21.15.32:0/204360684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40094a0 con 0x7fb8f412cf60 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f259c700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 0x7fb8f41c46e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ebe0 tx=0x7fb8e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400c850 con 0x7fb8f412cf60 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400c9b0 con 0x7fb8f412cf60 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400cd40 con 0x7fb8f412cf60 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc00d9f0 con 0x7fb8f412cf60 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.375+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec014660 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.376+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8e4013470 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.376+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb8e403f780 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.376+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f4074ea0 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.381+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8e4013720 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.586+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb8f406d530 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.586+0000 7fb8eaffd700 1 -- 172.21.15.32:0/204360684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8e4021030 con 0x7fb8f412cf60 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 msgr2=0x7fb8f41c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 0x7fb8f41c46e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ebe0 tx=0x7fb8e400eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 shutdown_connections 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb8f4130260 0x7fb8f41c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8f412cf60 0x7fb8f41c46e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 --2- 172.21.15.32:0/204360684 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb8f412f8b0 0x7fb8f41c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.588+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 >> 172.21.15.32:0/204360684 conn(0x7fb8f40a2aa0 msgr2=0x7fb8f40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.589+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 shutdown_connections 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:50.589+0000 7fb8f8f35700 1 -- 172.21.15.32:0/204360684 wait complete. 2023-09-30T22:30:51.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.244+0000 7f8fb86b2700 1 Processor -- start 2023-09-30T22:30:51.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.244+0000 7f8fb86b2700 1 -- start start 2023-09-30T22:30:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb86b2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 0x7f8fb006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb86b2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 0x7f8fb0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb86b2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb009ece0 con 0x7f8fb0131b40 2023-09-30T22:30:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb86b2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb009e760 con 0x7f8fb006b080 2023-09-30T22:30:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb86b2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb009e040 con 0x7f8fb006a770 2023-09-30T22:30:51.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 0x7f8fb006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb5c4d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 0x7f8fb0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 0x7f8fb006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55804/0 (socket says 172.21.15.32:55804) 2023-09-30T22:30:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb5c4d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 0x7f8fb0131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49372/0 (socket says 172.21.15.32:49372) 2023-09-30T22:30:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 -- 172.21.15.32:0/786379801 learned_addr learned my addr 172.21.15.32:0/786379801 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb6c4f700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8fb0131b40 0x7f8fb0138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39458/0 (socket says 172.21.15.32:39458) 2023-09-30T22:30:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb544c700 1 -- 172.21.15.32:0/786379801 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570931949 0 0) 0x7f8fb009ece0 con 0x7f8fb0131b40 2023-09-30T22:30:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb544c700 1 -- 172.21.15.32:0/786379801 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f9c0035b0 con 0x7f8fb0131b40 2023-09-30T22:30:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 -- 172.21.15.32:0/786379801 >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 msgr2=0x7f8fb0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 --2- 172.21.15.32:0/786379801 >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 0x7f8fb0131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 -- 172.21.15.32:0/786379801 >> v1:172.21.15.32:6789/0 conn(0x7f8fb0131b40 legacy=0x7f8fb0138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb644e700 1 -- 172.21.15.32:0/786379801 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb009f630 con 0x7f8fb006a770 2023-09-30T22:30:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.245+0000 7f8fb5c4d700 1 --2- 172.21.15.32:0/786379801 >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 0x7f8fb0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb644e700 1 --2- 172.21.15.32:0/786379801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 0x7f8fb006ab40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80098f0 tx=0x7f8fa8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83beaceea7d9d99 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb544c700 1 -- 172.21.15.32:0/786379801 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8011830 con 0x7f8fb006a770 2023-09-30T22:30:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb544c700 1 -- 172.21.15.32:0/786379801 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8011990 con 0x7f8fb006a770 2023-09-30T22:30:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb544c700 1 -- 172.21.15.32:0/786379801 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8011d20 con 0x7f8fb006a770 2023-09-30T22:30:51.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb86b2700 1 -- 172.21.15.32:0/786379801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 msgr2=0x7f8fb006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/786379801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 0x7f8fb006ab40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80098f0 tx=0x7f8fa8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb86b2700 1 -- 172.21.15.32:0/786379801 shutdown_connections 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/786379801 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb006a770 0x7f8fb006ab40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/786379801 >> v2:172.21.15.32:3300/0 conn(0x7f8fb006b080 0x7f8fb0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.246+0000 7f8fb86b2700 1 -- 172.21.15.32:0/786379801 >> 172.21.15.32:0/786379801 conn(0x7f8fb0066f50 msgr2=0x7f8fb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.247+0000 7f8fb86b2700 1 -- 172.21.15.32:0/786379801 shutdown_connections 2023-09-30T22:30:51.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.247+0000 7f8fb86b2700 1 -- 172.21.15.32:0/786379801 wait complete. 2023-09-30T22:30:51.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.247+0000 7f763fc96700 1 Processor -- start 2023-09-30T22:30:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.247+0000 7f1241bde700 1 Processor -- start 2023-09-30T22:30:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f1241bde700 1 -- start start 2023-09-30T22:30:51.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f763fc96700 1 -- start start 2023-09-30T22:30:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f8fb86b2700 1 Processor -- start 2023-09-30T22:30:51.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f8fb86b2700 1 -- start start 2023-09-30T22:30:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f1241bde700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c12f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f1241bde700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f123c12c2f0 0x7f123c12fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f1241bde700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123c09a510 con 0x7f123c12cff0 2023-09-30T22:30:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f1241bde700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c12c7d0 con 0x7f123c12c2f0 2023-09-30T22:30:51.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f1241bde700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f123c09c190 con 0x7f123c12cc20 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c12f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb86b2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fb006a770 0x7f8fb01c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb86b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 0x7f8fb01c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb86b2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb0131b40 0x7f8fb01c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb86b2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01311a0 con 0x7f8fb006b080 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb86b2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01221b0 con 0x7f8fb006a770 2023-09-30T22:30:51.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb86b2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01209d0 con 0x7f8fb0131b40 2023-09-30T22:30:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f123b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c12f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55814/0 (socket says 172.21.15.32:55814) 2023-09-30T22:30:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f123b7fe700 1 -- 172.21.15.32:0/1999468790 learned_addr learned my addr 172.21.15.32:0/1999468790 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f763fc96700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809d6f0 0x7f763809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f763fc96700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e000 0x7f7638130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.248+0000 7f763fc96700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763809f130 con 0x7f763809e410 2023-09-30T22:30:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763fc96700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763809f520 con 0x7f763809d6f0 2023-09-30T22:30:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763fc96700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f763809e930 con 0x7f763809e000 2023-09-30T22:30:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763e233700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f763809e410 0x7f7638130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39476/0 (socket says 172.21.15.32:39476) 2023-09-30T22:30:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763e233700 1 -- 172.21.15.32:0/863265448 learned_addr learned my addr 172.21.15.32:0/863265448 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:51.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f123affd700 1 --2- 172.21.15.32:0/1999468790 >> v2:172.21.15.32:3300/0 conn(0x7f123c12c2f0 0x7f123c12fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb5c4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 0x7f8fb01c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb5c4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 0x7f8fb01c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49396/0 (socket says 172.21.15.32:49396) 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb6c4f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb0131b40 0x7f8fb01c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb5c4d700 1 -- 172.21.15.32:0/2939130271 learned_addr learned my addr 172.21.15.32:0/2939130271 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb644e700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fb006a770 0x7f8fb01c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763da32700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809d6f0 0x7f763809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763d231700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e000 0x7f7638130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb6c4f700 1 -- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fb006a770 msgr2=0x7f8fb01c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f8fb6c4f700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fb006a770 0x7f8fb01c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb6c4f700 1 -- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 msgr2=0x7f8fb01c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb6c4f700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 0x7f8fb01c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb6c4f700 1 -- 172.21.15.32:0/2939130271 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00094a0 con 0x7f8fb0131b40 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb5c4d700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 0x7f8fb01c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb644e700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fb006a770 0x7f8fb01c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123affd700 1 -- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 msgr2=0x7f123c12f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123affd700 1 --2- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c12f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.249+0000 7f763da32700 1 -- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e000 msgr2=0x7f7638130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f763da32700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e000 0x7f7638130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f763da32700 1 -- 172.21.15.32:0/863265448 >> v1:172.21.15.32:6789/0 conn(0x7f763809e410 legacy=0x7f7638130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123affd700 1 -- 172.21.15.32:0/1999468790 >> v1:172.21.15.32:6789/0 conn(0x7f123c12cff0 legacy=0x7f123c136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123affd700 1 -- 172.21.15.32:0/1999468790 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c09b540 con 0x7f123c12c2f0 2023-09-30T22:30:51.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f763da32700 1 -- 172.21.15.32:0/863265448 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763809b2d0 con 0x7f763809d6f0 2023-09-30T22:30:51.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123affd700 1 --2- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12c2f0 0x7f123c12fe70 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1224009870 tx=0x7f1224009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480e7777bb51ff8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f763da32700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809d6f0 0x7f763809dac0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7634009870 tx=0x7f7634009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374f6b0fa0be9d03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb6c4f700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb0131b40 0x7f8fb01c6860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00cb70 tx=0x7f8fac00ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123a7fc700 1 -- 172.21.15.32:0/1999468790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224011830 con 0x7f123c12c2f0 2023-09-30T22:30:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00aa60 con 0x7f8fb0131b40 2023-09-30T22:30:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac00abc0 con 0x7f8fb0131b40 2023-09-30T22:30:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123a7fc700 1 -- 172.21.15.32:0/1999468790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224011990 con 0x7f123c12c2f0 2023-09-30T22:30:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa8009550 con 0x7f8fb0131b40 2023-09-30T22:30:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f123a7fc700 1 -- 172.21.15.32:0/1999468790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224011d20 con 0x7f123c12c2f0 2023-09-30T22:30:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac007070 con 0x7f8fb0131b40 2023-09-30T22:30:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.250+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb0123b20 con 0x7f8fb0131b40 2023-09-30T22:30:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f763ca30700 1 -- 172.21.15.32:0/863265448 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011830 con 0x7f763809d6f0 2023-09-30T22:30:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f763ca30700 1 -- 172.21.15.32:0/863265448 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7634011990 con 0x7f763809d6f0 2023-09-30T22:30:51.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f763ca30700 1 -- 172.21.15.32:0/863265448 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634011d20 con 0x7f763809d6f0 2023-09-30T22:30:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 -- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12c2f0 msgr2=0x7f123c12fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 --2- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12c2f0 0x7f123c12fe70 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1224009870 tx=0x7f1224009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 -- 172.21.15.32:0/1999468790 shutdown_connections 2023-09-30T22:30:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 --2- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c12f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 --2- 172.21.15.32:0/1999468790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12c2f0 0x7f123c12fe70 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 -- 172.21.15.32:0/1999468790 >> 172.21.15.32:0/1999468790 conn(0x7f123c09f470 msgr2=0x7f123c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 -- 172.21.15.32:0/1999468790 shutdown_connections 2023-09-30T22:30:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f1241bde700 1 -- 172.21.15.32:0/1999468790 wait complete. 2023-09-30T22:30:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.251+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8fac0071d0 con 0x7f8fb0131b40 2023-09-30T22:30:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.252+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8fac03f440 con 0x7f8fb0131b40 2023-09-30T22:30:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.252+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb0074ea0 con 0x7f8fb0131b40 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.252+0000 7f763fc96700 1 -- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809d6f0 msgr2=0x7f763809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.252+0000 7f763fc96700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809d6f0 0x7f763809dac0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7634009870 tx=0x7f7634009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f763fc96700 1 -- 172.21.15.32:0/863265448 shutdown_connections 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f763fc96700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809d6f0 0x7f763809dac0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f763fc96700 1 --2- 172.21.15.32:0/863265448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e000 0x7f7638130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f763fc96700 1 -- 172.21.15.32:0/863265448 >> 172.21.15.32:0/863265448 conn(0x7f7638066f50 msgr2=0x7f763809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f763fc96700 1 -- 172.21.15.32:0/863265448 shutdown_connections 2023-09-30T22:30:51.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f763fc96700 1 -- 172.21.15.32:0/863265448 wait complete. 2023-09-30T22:30:51.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f1241bde700 1 Processor -- start 2023-09-30T22:30:51.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.253+0000 7f1241bde700 1 -- start start 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f1241bde700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f123c12c2f0 0x7f123c1b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f1241bde700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c1b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f1241bde700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 0x7f123c1b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f1241bde700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c129f10 con 0x7f123c12cff0 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f1241bde700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c08b5e0 con 0x7f123c12c2f0 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f1241bde700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f123c08b150 con 0x7f123c12cc20 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f123bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 0x7f123c1b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f123bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 0x7f123c1b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49410/0 (socket says 172.21.15.32:49410) 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f123bfff700 1 -- 172.21.15.32:0/4241719776 learned_addr learned my addr 172.21.15.32:0/4241719776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f123affd700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c1b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f763fc96700 1 Processor -- start 2023-09-30T22:30:51.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f763fc96700 1 -- start start 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.254+0000 7f123b7fe700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f123c12c2f0 0x7f123c1b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f123bfff700 1 -- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 msgr2=0x7f123c1b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f123bfff700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c1b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f123bfff700 1 -- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f123c12c2f0 msgr2=0x7f123c1b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f123bfff700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f123c12c2f0 0x7f123c1b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f123bfff700 1 -- 172.21.15.32:0/4241719776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240094a0 con 0x7f123c12cff0 2023-09-30T22:30:51.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f123bfff700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 0x7f123c1b7a10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f123400cc80 tx=0x7f123400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763fc96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 0x7f76381c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763fc96700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809e000 0x7f76381c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763fc96700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e410 0x7f76381c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763fc96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763809c700 con 0x7f763809d6f0 2023-09-30T22:30:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763fc96700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7638132540 con 0x7f763809e000 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763fc96700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7638132970 con 0x7f763809e410 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 0x7f76381c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763e233700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e410 0x7f76381c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763d231700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809e000 0x7f76381c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 0x7f76381c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49424/0 (socket says 172.21.15.32:49424) 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 -- 172.21.15.32:0/997869213 learned_addr learned my addr 172.21.15.32:0/997869213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 -- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e410 msgr2=0x7f76381c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e410 0x7f76381c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 -- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809e000 msgr2=0x7f76381c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234003c00 con 0x7f123c12cff0 2023-09-30T22:30:51.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809e000 0x7f76381c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.255+0000 7f763da32700 1 -- 172.21.15.32:0/997869213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340094a0 con 0x7f763809d6f0 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f763d231700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809e000 0x7f76381c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f763da32700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 0x7f76381c8170 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f76340090b0 tx=0x7f7634004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634004510 con 0x7f763809d6f0 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763401f070 con 0x7f763809d6f0 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76381331d0 con 0x7f763809d6f0 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340038c0 con 0x7f763809d6f0 2023-09-30T22:30:51.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7638134010 con 0x7f763809d6f0 2023-09-30T22:30:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1234003d60 con 0x7f123c12cff0 2023-09-30T22:30:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400aaa0 con 0x7f123c12cff0 2023-09-30T22:30:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c12b670 con 0x7f123c12cff0 2023-09-30T22:30:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.256+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c1b7f50 con 0x7f123c12cff0 2023-09-30T22:30:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.257+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7634003a20 con 0x7f763809d6f0 2023-09-30T22:30:51.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.257+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638074ea0 con 0x7f763809d6f0 2023-09-30T22:30:51.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.257+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7634021070 con 0x7f763809d6f0 2023-09-30T22:30:51.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.257+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f123400ac00 con 0x7f123c12cff0 2023-09-30T22:30:51.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.258+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f123403f600 con 0x7f123c12cff0 2023-09-30T22:30:51.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.258+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c099e70 con 0x7f123c12cff0 2023-09-30T22:30:51.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.258+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fac017080 con 0x7f8fb0131b40 2023-09-30T22:30:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.262+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f763401a430 con 0x7f763809d6f0 2023-09-30T22:30:51.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.267+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12340052b0 con 0x7f123c12cff0 2023-09-30T22:30:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.466+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8fb006c360 con 0x7f8fb0131b40 2023-09-30T22:30:51.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.466+0000 7f8fa6ffd700 1 -- 172.21.15.32:0/2939130271 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fac0198b0 con 0x7f8fb0131b40 2023-09-30T22:30:51.468 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb0131b40 msgr2=0x7f8fb01c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb0131b40 0x7f8fb01c6860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00cb70 tx=0x7f8fac00ce80 comp rx=0 tx=0).stop 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 shutdown_connections 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8fb0131b40 0x7f8fb01c6860 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fb006a770 0x7f8fb01c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 --2- 172.21.15.32:0/2939130271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fb006b080 0x7f8fb01c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 >> 172.21.15.32:0/2939130271 conn(0x7f8fb0066f50 msgr2=0x7f8fb006ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 shutdown_connections 2023-09-30T22:30:51.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.469+0000 7f8fb86b2700 1 -- 172.21.15.32:0/2939130271 wait complete. 2023-09-30T22:30:51.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.471+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f763806c360 con 0x7f763809d6f0 2023-09-30T22:30:51.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.472+0000 7f762e7fc700 1 -- 172.21.15.32:0/997869213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7634015030 con 0x7f763809d6f0 2023-09-30T22:30:51.473 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 msgr2=0x7f76381c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 0x7f76381c8170 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f76340090b0 tx=0x7f7634004750 comp rx=0 tx=0).stop 2023-09-30T22:30:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 shutdown_connections 2023-09-30T22:30:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f763809e410 0x7f76381c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f763809d6f0 0x7f76381c8170 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 --2- 172.21.15.32:0/997869213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f763809e000 0x7f76381c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 >> 172.21.15.32:0/997869213 conn(0x7f7638066f50 msgr2=0x7f76380a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 shutdown_connections 2023-09-30T22:30:51.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.475+0000 7f763fc96700 1 -- 172.21.15.32:0/997869213 wait complete. 2023-09-30T22:30:51.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.479+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f123c130330 con 0x7f123c12cff0 2023-09-30T22:30:51.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.480+0000 7f1233fff700 1 -- 172.21.15.32:0/4241719776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1234019300 con 0x7f123c12cff0 2023-09-30T22:30:51.481 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:51.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 msgr2=0x7f123c1b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 0x7f123c1b7a10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f123400cc80 tx=0x7f123400cf90 comp rx=0 tx=0).stop 2023-09-30T22:30:51.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 shutdown_connections 2023-09-30T22:30:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f123c12cff0 0x7f123c1b7a10 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f123c12c2f0 0x7f123c1b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 --2- 172.21.15.32:0/4241719776 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f123c12cc20 0x7f123c1b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.484+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 >> 172.21.15.32:0/4241719776 conn(0x7f123c09f470 msgr2=0x7f123c0a03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.485+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 shutdown_connections 2023-09-30T22:30:51.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:51.485+0000 7f1241bde700 1 -- 172.21.15.32:0/4241719776 wait complete. 2023-09-30T22:30:51.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:51.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:52.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:52.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 Processor -- start 2023-09-30T22:30:52.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 -- start start 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c130a40 0x7f297c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c06bc00 con 0x7f297c12ff20 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c06c570 con 0x7f297c133a80 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f2981259700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0a0040 con 0x7f297c130a40 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f297b59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c130a40 0x7f297c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f297a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f297c12ff20 0x7f297c130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39478/0 (socket says 172.21.15.32:39478) 2023-09-30T22:30:52.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f297ad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.201+0000 7f297a59c700 1 -- 172.21.15.32:0/73493184 learned_addr learned my addr 172.21.15.32:0/73493184 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f297b59e700 1 -- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 msgr2=0x7f297c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f297b59e700 1 --2- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f297b59e700 1 -- 172.21.15.32:0/73493184 >> v1:172.21.15.32:6789/0 conn(0x7f297c12ff20 legacy=0x7f297c130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f2979d9b700 1 -- 172.21.15.32:0/73493184 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434098550 0 0) 0x7f297c06bc00 con 0x7f297c12ff20 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f297b59e700 1 -- 172.21.15.32:0/73493184 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c133860 con 0x7f297c130a40 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f297ad9d700 1 --2- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f297b59e700 1 --2- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c130a40 0x7f297c13b850 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2974009dc0 tx=0x7f297400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d393cf194bb75363 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f2979d9b700 1 -- 172.21.15.32:0/73493184 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400e070 con 0x7f297c130a40 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f2979d9b700 1 -- 172.21.15.32:0/73493184 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400bb20 con 0x7f297c130a40 2023-09-30T22:30:52.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.202+0000 7f2979d9b700 1 -- 172.21.15.32:0/73493184 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974012430 con 0x7f297c130a40 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 -- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c130a40 msgr2=0x7f297c13b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 --2- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c130a40 0x7f297c13b850 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2974009dc0 tx=0x7f297400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 -- 172.21.15.32:0/73493184 shutdown_connections 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 --2- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c130a40 0x7f297c13b850 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 --2- 172.21.15.32:0/73493184 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 -- 172.21.15.32:0/73493184 >> 172.21.15.32:0/73493184 conn(0x7f297c0a4210 msgr2=0x7f297c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 -- 172.21.15.32:0/73493184 shutdown_connections 2023-09-30T22:30:52.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.203+0000 7f2981259700 1 -- 172.21.15.32:0/73493184 wait complete. 2023-09-30T22:30:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.204+0000 7f2981259700 1 Processor -- start 2023-09-30T22:30:52.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.204+0000 7f2981259700 1 -- start start 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f2981259700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 0x7f297c09c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f2981259700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c130a40 0x7f297c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f2981259700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f2981259700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c120cb0 con 0x7f297c130a40 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f2981259700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c120090 con 0x7f297c133a80 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f2981259700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f297c11eb90 con 0x7f297c12ff20 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 0x7f297c09c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c130a40 0x7f297c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 0x7f297c09c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55860/0 (socket says 172.21.15.32:55860) 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 -- 172.21.15.32:0/327683839 learned_addr learned my addr 172.21.15.32:0/327683839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:52.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297b59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 -- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 msgr2=0x7f297c09d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c09d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 -- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c130a40 msgr2=0x7f297c09cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c130a40 0x7f297c09cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 -- 172.21.15.32:0/327683839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0094a0 con 0x7f297c12ff20 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297b59e700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c09d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297a59c700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c130a40 0x7f297c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f297ad9d700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 0x7f297c09c6e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f296c00ebe0 tx=0x7f296c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.205+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00c850 con 0x7f297c12ff20 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.206+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c00c9b0 con 0x7f297c12ff20 2023-09-30T22:30:52.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.206+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2974009a70 con 0x7f297c12ff20 2023-09-30T22:30:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.206+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00cd40 con 0x7f297c12ff20 2023-09-30T22:30:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.206+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c120370 con 0x7f297c12ff20 2023-09-30T22:30:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.207+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f296c013470 con 0x7f297c12ff20 2023-09-30T22:30:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.207+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c06d4d0 con 0x7f297c12ff20 2023-09-30T22:30:52.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.207+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f296c03fb30 con 0x7f297c12ff20 2023-09-30T22:30:52.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.212+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f296c017020 con 0x7f297c12ff20 2023-09-30T22:30:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.417+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f297c074ea0 con 0x7f297c12ff20 2023-09-30T22:30:52.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.417+0000 7f29637fe700 1 -- 172.21.15.32:0/327683839 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f296c00cea0 con 0x7f297c12ff20 2023-09-30T22:30:52.418 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 msgr2=0x7f297c09c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 0x7f297c09c6e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f296c00ebe0 tx=0x7f296c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 shutdown_connections 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f297c133a80 0x7f297c09d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f297c12ff20 0x7f297c09c6e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 --2- 172.21.15.32:0/327683839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c130a40 0x7f297c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 >> 172.21.15.32:0/327683839 conn(0x7f297c0a4210 msgr2=0x7f297c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 shutdown_connections 2023-09-30T22:30:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.420+0000 7f2981259700 1 -- 172.21.15.32:0/327683839 wait complete. 2023-09-30T22:30:52.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:52.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:52.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:52.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.629+0000 7f5083b59700 1 Processor -- start 2023-09-30T22:30:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.629+0000 7f5083b59700 1 -- start start 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f5083b59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133420 0x7f507c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f5083b59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 0x7f507c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f5083b59700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c06b990 con 0x7f507c134260 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f5083b59700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0a0590 con 0x7f507c133420 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f5083b59700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507c09fb80 con 0x7f507c133e30 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50818f5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133420 0x7f507c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50810f4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 0x7f507c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50810f4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 0x7f507c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55866/0 (socket says 172.21.15.32:55866) 2023-09-30T22:30:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50810f4700 1 -- 172.21.15.32:0/766016890 learned_addr learned my addr 172.21.15.32:0/766016890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50818f5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133420 0x7f507c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41708/0 (socket says 172.21.15.32:41708) 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50810f4700 1 -- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133420 msgr2=0x7f507c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50810f4700 1 --2- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133420 0x7f507c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50810f4700 1 -- 172.21.15.32:0/766016890 >> v1:172.21.15.32:6789/0 conn(0x7f507c134260 legacy=0x7f507c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.630+0000 7f50808f3700 1 -- 172.21.15.32:0/766016890 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855750508 0 0) 0x7f507c06b990 con 0x7f507c134260 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.631+0000 7f50810f4700 1 -- 172.21.15.32:0/766016890 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c0a0030 con 0x7f507c133e30 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.631+0000 7f50810f4700 1 --2- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 0x7f507c12ff90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f506c009920 tx=0x7f506c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02e72c6322c47bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.631+0000 7f50808f3700 1 -- 172.21.15.32:0/766016890 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c011830 con 0x7f507c133e30 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.631+0000 7f50808f3700 1 -- 172.21.15.32:0/766016890 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c011990 con 0x7f507c133e30 2023-09-30T22:30:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.631+0000 7f50808f3700 1 -- 172.21.15.32:0/766016890 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c011d20 con 0x7f507c133e30 2023-09-30T22:30:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 -- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 msgr2=0x7f507c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 --2- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 0x7f507c12ff90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f506c009920 tx=0x7f506c009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 -- 172.21.15.32:0/766016890 shutdown_connections 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 --2- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133420 0x7f507c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 --2- 172.21.15.32:0/766016890 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133e30 0x7f507c12ff90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 -- 172.21.15.32:0/766016890 >> 172.21.15.32:0/766016890 conn(0x7f507c0a4210 msgr2=0x7f507c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 -- 172.21.15.32:0/766016890 shutdown_connections 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.632+0000 7f5083b59700 1 -- 172.21.15.32:0/766016890 wait complete. 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.633+0000 7f5083b59700 1 Processor -- start 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.633+0000 7f5083b59700 1 -- start start 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.633+0000 7f5083b59700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133420 0x7f507c09c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.633+0000 7f5083b59700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133e30 0x7f507c09c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.633+0000 7f5083b59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c134260 0x7f507c09cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f5083b59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c131590 con 0x7f507c134260 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f5083b59700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1205e0 con 0x7f507c133e30 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f5083b59700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507c09d4f0 con 0x7f507c133420 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50810f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133e30 0x7f507c09c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50810f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133e30 0x7f507c09c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41714/0 (socket says 172.21.15.32:41714) 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50810f4700 1 -- 172.21.15.32:0/2619247173 learned_addr learned my addr 172.21.15.32:0/2619247173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133420 0x7f507c09c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50820f6700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c134260 0x7f507c09cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 -- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133e30 msgr2=0x7f507c09c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133e30 0x7f507c09c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 -- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c134260 msgr2=0x7f507c09cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c134260 0x7f507c09cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 -- 172.21.15.32:0/2619247173 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c009550 con 0x7f507c133420 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50820f6700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c134260 0x7f507c09cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.634+0000 7f50818f5700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133420 0x7f507c09c120 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f507800e900 tx=0x7f507800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.635+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078013680 con 0x7f507c133420 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.635+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50780137e0 con 0x7f507c133420 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.635+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078013b50 con 0x7f507c133420 2023-09-30T22:30:52.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.635+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c11fa80 con 0x7f507c133420 2023-09-30T22:30:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.635+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c12da60 con 0x7f507c133420 2023-09-30T22:30:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.636+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5078013cb0 con 0x7f507c133420 2023-09-30T22:30:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.636+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c074ea0 con 0x7f507c133420 2023-09-30T22:30:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.636+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f507803f650 con 0x7f507c133420 2023-09-30T22:30:52.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.641+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f507801bc10 con 0x7f507c133420 2023-09-30T22:30:52.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 Processor -- start 2023-09-30T22:30:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f6828481700 1 Processor -- start 2023-09-30T22:30:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f6828481700 1 -- start start 2023-09-30T22:30:52.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 -- start start 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d4070940 con 0x7f93d409e930 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d406fd60 con 0x7f93d4136fe0 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f93dbe91700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93d406fa80 con 0x7f93d409f410 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f6828481700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a0a30 0x7f68200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f6828481700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6820134170 0x7f682013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f6828481700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682009edd0 con 0x7f68200a1340 2023-09-30T22:30:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.672+0000 7f6828481700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682009e1f0 con 0x7f6820134170 2023-09-30T22:30:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6828481700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f682009dc50 con 0x7f68200a0a30 2023-09-30T22:30:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f93d9c2d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f93d409e930 0x7f93d409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39494/0 (socket says 172.21.15.32:39494) 2023-09-30T22:30:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f93d9c2d700 1 -- 172.21.15.32:0/2078795541 learned_addr learned my addr 172.21.15.32:0/2078795541 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:52.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6820134170 0x7f682013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6825a1c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68200a1340 0x7f6820133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39506/0 (socket says 172.21.15.32:39506) 2023-09-30T22:30:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6825a1c700 1 -- 172.21.15.32:0/1467967926 learned_addr learned my addr 172.21.15.32:0/1467967926 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6820134170 0x7f682013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49444/0 (socket says 172.21.15.32:49444) 2023-09-30T22:30:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f93da42e700 1 --2- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f682621d700 1 --2- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a0a30 0x7f68200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f93d8c2b700 1 -- 172.21.15.32:0/2078795541 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448651506 0 0) 0x7f93d4070940 con 0x7f93d409e930 2023-09-30T22:30:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 -- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a0a30 msgr2=0x7f68200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 --2- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a0a30 0x7f68200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 -- 172.21.15.32:0/1467967926 >> v1:172.21.15.32:6789/0 conn(0x7f68200a1340 legacy=0x7f6820133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f682521b700 1 -- 172.21.15.32:0/1467967926 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69027322 0 0) 0x7f682009edd0 con 0x7f68200a1340 2023-09-30T22:30:52.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 -- 172.21.15.32:0/1467967926 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200a1fe0 con 0x7f6820134170 2023-09-30T22:30:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f93d8c2b700 1 -- 172.21.15.32:0/2078795541 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c00035b0 con 0x7f93d409e930 2023-09-30T22:30:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.673+0000 7f6826a1e700 1 --2- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f682013bd30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6818009d90 tx=0x7f681800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ca27feb6e97e36 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93da42e700 1 -- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 msgr2=0x7f93d4136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93da42e700 1 --2- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93d8c2b700 1 -- 172.21.15.32:0/2078795541 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350285263 0 0) 0x7f93c00035b0 con 0x7f93d409e930 2023-09-30T22:30:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93da42e700 1 -- 172.21.15.32:0/2078795541 >> v1:172.21.15.32:6789/0 conn(0x7f93d409e930 legacy=0x7f93d409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93da42e700 1 -- 172.21.15.32:0/2078795541 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d409fb60 con 0x7f93d4136fe0 2023-09-30T22:30:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f682521b700 1 -- 172.21.15.32:0/1467967926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800e070 con 0x7f6820134170 2023-09-30T22:30:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f682521b700 1 -- 172.21.15.32:0/1467967926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681800bbd0 con 0x7f6820134170 2023-09-30T22:30:52.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f682521b700 1 -- 172.21.15.32:0/1467967926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818012430 con 0x7f6820134170 2023-09-30T22:30:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93da42e700 1 --2- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d41393c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009dc0 tx=0x7f93cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da084d65461e233 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93d8c2b700 1 -- 172.21.15.32:0/2078795541 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00e070 con 0x7f93d4136fe0 2023-09-30T22:30:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93d8c2b700 1 -- 172.21.15.32:0/2078795541 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc003a20 con 0x7f93d4136fe0 2023-09-30T22:30:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.674+0000 7f93d8c2b700 1 -- 172.21.15.32:0/2078795541 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0001670 con 0x7f93d4136fe0 2023-09-30T22:30:52.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 -- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 msgr2=0x7f682013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 --2- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f682013bd30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6818009d90 tx=0x7f681800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 -- 172.21.15.32:0/1467967926 shutdown_connections 2023-09-30T22:30:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 --2- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f682013bd30 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 --2- 172.21.15.32:0/1467967926 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a0a30 0x7f68200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 -- 172.21.15.32:0/1467967926 >> 172.21.15.32:0/1467967926 conn(0x7f68200a6550 msgr2=0x7f68200a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 -- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 msgr2=0x7f93d41393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 --2- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d41393c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009dc0 tx=0x7f93cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 -- 172.21.15.32:0/1467967926 shutdown_connections 2023-09-30T22:30:52.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 -- 172.21.15.32:0/2078795541 shutdown_connections 2023-09-30T22:30:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 --2- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d41393c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 --2- 172.21.15.32:0/2078795541 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d4136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 -- 172.21.15.32:0/2078795541 >> 172.21.15.32:0/2078795541 conn(0x7f93d40a4210 msgr2=0x7f93d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f6828481700 1 -- 172.21.15.32:0/1467967926 wait complete. 2023-09-30T22:30:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 -- 172.21.15.32:0/2078795541 shutdown_connections 2023-09-30T22:30:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.675+0000 7f93dbe91700 1 -- 172.21.15.32:0/2078795541 wait complete. 2023-09-30T22:30:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.676+0000 7f93dbe91700 1 Processor -- start 2023-09-30T22:30:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.676+0000 7f6828481700 1 Processor -- start 2023-09-30T22:30:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 -- start start 2023-09-30T22:30:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 -- start start 2023-09-30T22:30:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d409e930 0x7f93d411c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68200a0a30 0x7f6820122590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 0x7f6820122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f6820123010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201239f0 con 0x7f6820134170 2023-09-30T22:30:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6820133550 con 0x7f68200a0a30 2023-09-30T22:30:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f6828481700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68201257b0 con 0x7f68200a1340 2023-09-30T22:30:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d411cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d411d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d4071d20 con 0x7f93d409e930 2023-09-30T22:30:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d4120460 con 0x7f93d4136fe0 2023-09-30T22:30:52.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.677+0000 7f93dbe91700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93d411ff40 con 0x7f93d409f410 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6825a1c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 0x7f6820122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6825a1c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 0x7f6820122ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55906/0 (socket says 172.21.15.32:55906) 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6825a1c700 1 -- 172.21.15.32:0/876103405 learned_addr learned my addr 172.21.15.32:0/876103405 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f682621d700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68200a0a30 0x7f6820122590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d942c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d411cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d942c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d411cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55912/0 (socket says 172.21.15.32:55912) 2023-09-30T22:30:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d942c700 1 -- 172.21.15.32:0/50879825 learned_addr learned my addr 172.21.15.32:0/50879825 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d409e930 0x7f93d411c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6826a1e700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f6820123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93da42e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d411d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 -- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 msgr2=0x7f93d411cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d411cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 -- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 msgr2=0x7f93d411d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d411d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 -- 172.21.15.32:0/50879825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc009a70 con 0x7f93d409e930 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93d9c2d700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d409e930 0x7f93d411c700 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c520 tx=0x7f93d000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6826a1e700 1 -- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 msgr2=0x7f6820122ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6826a1e700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 0x7f6820122ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6826a1e700 1 -- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68200a0a30 msgr2=0x7f6820122590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6826a1e700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68200a0a30 0x7f6820122590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6826a1e700 1 -- 172.21.15.32:0/876103405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6818009a70 con 0x7f6820134170 2023-09-30T22:30:52.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f6825a1c700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 0x7f6820122ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0004c80 con 0x7f93d409e930 2023-09-30T22:30:52.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0004de0 con 0x7f93d409e930 2023-09-30T22:30:52.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f682621d700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68200a0a30 0x7f6820122590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d411f9b0 con 0x7f93d409e930 2023-09-30T22:30:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.678+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0007310 con 0x7f93d409e930 2023-09-30T22:30:52.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d41325f0 con 0x7f93d409e930 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f6826a1e700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f6820123010 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6818016000 tx=0x7f6818004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818012430 con 0x7f6820134170 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6818020070 con 0x7f6820134170 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818004280 con 0x7f6820134170 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68201241a0 con 0x7f6820134170 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.679+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6820125b60 con 0x7f6820134170 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.680+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68180043e0 con 0x7f6820134170 2023-09-30T22:30:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.680+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d4074ea0 con 0x7f93d409e930 2023-09-30T22:30:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.680+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6818022070 con 0x7f6820134170 2023-09-30T22:30:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.680+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93d0007470 con 0x7f93d409e930 2023-09-30T22:30:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.680+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6820074ea0 con 0x7f6820134170 2023-09-30T22:30:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.680+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f93d003f3c0 con 0x7f93d409e930 2023-09-30T22:30:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.685+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93d00133c0 con 0x7f93d409e930 2023-09-30T22:30:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.690+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f681802c020 con 0x7f6820134170 2023-09-30T22:30:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.879+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f507c11fc70 con 0x7f507c133420 2023-09-30T22:30:52.880 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:52.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.879+0000 7f50727fc700 1 -- 172.21.15.32:0/2619247173 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5078021020 con 0x7f507c133420 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.881+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133420 msgr2=0x7f507c09c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.881+0000 7f5083b59700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133420 0x7f507c09c120 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f507800e900 tx=0x7f507800ec10 comp rx=0 tx=0).stop 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.881+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 shutdown_connections 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.882+0000 7f5083b59700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f507c134260 0x7f507c09cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.882+0000 7f5083b59700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f507c133420 0x7f507c09c120 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.882+0000 7f5083b59700 1 --2- 172.21.15.32:0/2619247173 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f507c133e30 0x7f507c09c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.882+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 >> 172.21.15.32:0/2619247173 conn(0x7f507c0a4210 msgr2=0x7f507c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.882+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 shutdown_connections 2023-09-30T22:30:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.882+0000 7f5083b59700 1 -- 172.21.15.32:0/2619247173 wait complete. 2023-09-30T22:30:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.894+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f68200a28e0 con 0x7f6820134170 2023-09-30T22:30:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.895+0000 7f6816ffd700 1 -- 172.21.15.32:0/876103405 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6818027480 con 0x7f6820134170 2023-09-30T22:30:52.896 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:52.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:52.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 msgr2=0x7f6820123010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f6820123010 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6818016000 tx=0x7f6818004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 shutdown_connections 2023-09-30T22:30:52.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6820134170 0x7f6820123010 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f68200a0a30 0x7f6820122590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 --2- 172.21.15.32:0/876103405 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f68200a1340 0x7f6820122ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 >> 172.21.15.32:0/876103405 conn(0x7f68200a6550 msgr2=0x7f682009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 shutdown_connections 2023-09-30T22:30:52.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.897+0000 7f6828481700 1 -- 172.21.15.32:0/876103405 wait complete. 2023-09-30T22:30:52.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:52.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.919+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93d40a33f0 con 0x7f93d409e930 2023-09-30T22:30:52.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.920+0000 7f93ca7fc700 1 -- 172.21.15.32:0/50879825 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93d001a2b0 con 0x7f93d409e930 2023-09-30T22:30:52.921 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:52.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d409e930 msgr2=0x7f93d411c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d409e930 0x7f93d411c700 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c520 tx=0x7f93d000c830 comp rx=0 tx=0).stop 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 shutdown_connections 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f93d4136fe0 0x7f93d411d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93d409e930 0x7f93d411c700 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 --2- 172.21.15.32:0/50879825 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f93d409f410 0x7f93d411cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 >> 172.21.15.32:0/50879825 conn(0x7f93d40a4210 msgr2=0x7f93d4134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 shutdown_connections 2023-09-30T22:30:52.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:52.922+0000 7f93dbe91700 1 -- 172.21.15.32:0/50879825 wait complete. 2023-09-30T22:30:52.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:53.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:53.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 Processor -- start 2023-09-30T22:30:53.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 -- start start 2023-09-30T22:30:53.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c136fe0 0x7fa84c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c070940 con 0x7fa84c09e930 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06fd60 con 0x7fa84c09f410 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.581+0000 7fa853d75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06fa80 con 0x7fa84c136fe0 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa852312700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c136fe0 0x7fa84c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851b11700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa84c09e930 0x7fa84c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39518/0 (socket says 172.21.15.32:39518) 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851b11700 1 -- 172.21.15.32:0/77309354 learned_addr learned my addr 172.21.15.32:0/77309354 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851310700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851310700 1 -- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c136fe0 msgr2=0x7fa84c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851310700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c136fe0 0x7fa84c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851310700 1 -- 172.21.15.32:0/77309354 >> v1:172.21.15.32:6789/0 conn(0x7fa84c09e930 legacy=0x7fa84c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa850b0f700 1 -- 172.21.15.32:0/77309354 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404884961 0 0) 0x7fa84c070940 con 0x7fa84c09e930 2023-09-30T22:30:53.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851310700 1 -- 172.21.15.32:0/77309354 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c09fb60 con 0x7fa84c09f410 2023-09-30T22:30:53.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa852312700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c136fe0 0x7fa84c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa851310700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c136aa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009870 tx=0x7fa83c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2856f7dc2c7976 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa850b0f700 1 -- 172.21.15.32:0/77309354 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c011830 con 0x7fa84c09f410 2023-09-30T22:30:53.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa850b0f700 1 -- 172.21.15.32:0/77309354 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c011990 con 0x7fa84c09f410 2023-09-30T22:30:53.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.582+0000 7fa850b0f700 1 -- 172.21.15.32:0/77309354 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c011d20 con 0x7fa84c09f410 2023-09-30T22:30:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 -- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 msgr2=0x7fa84c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c136aa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009870 tx=0x7fa83c009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 -- 172.21.15.32:0/77309354 shutdown_connections 2023-09-30T22:30:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c136fe0 0x7fa84c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 --2- 172.21.15.32:0/77309354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c136aa0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 -- 172.21.15.32:0/77309354 >> 172.21.15.32:0/77309354 conn(0x7fa84c0a4210 msgr2=0x7fa84c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 -- 172.21.15.32:0/77309354 shutdown_connections 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.583+0000 7fa853d75700 1 -- 172.21.15.32:0/77309354 wait complete. 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.584+0000 7fa853d75700 1 Processor -- start 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.584+0000 7fa853d75700 1 -- start start 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.584+0000 7fa853d75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 0x7fa84c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa853d75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa853d75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c136fe0 0x7fa84c1c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa853d75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c121160 con 0x7fa84c136fe0 2023-09-30T22:30:53.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa853d75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06d5c0 con 0x7fa84c09f410 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa853d75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c06cc40 con 0x7fa84c09e930 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 0x7fa84c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851310700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 0x7fa84c06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55922/0 (socket says 172.21.15.32:55922) 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851310700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41762/0 (socket says 172.21.15.32:41762) 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 -- 172.21.15.32:0/1678917017 learned_addr learned my addr 172.21.15.32:0/1678917017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa852312700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c136fe0 0x7fa84c1c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 -- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 msgr2=0x7fa84c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 -- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c136fe0 msgr2=0x7fa84c1c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c136fe0 0x7fa84c1c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 -- 172.21.15.32:0/1678917017 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844009a70 con 0x7fa84c09e930 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851310700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa852312700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c136fe0 0x7fa84c1c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa851b11700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 0x7fa84c06c1c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c550 tx=0x7fa84800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848004c80 con 0x7fa84c09e930 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848004e00 con 0x7fa84c09e930 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c0094a0 con 0x7fa84c09e930 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.585+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848007310 con 0x7fa84c09e930 2023-09-30T22:30:53.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.586+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c121590 con 0x7fa84c09e930 2023-09-30T22:30:53.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.586+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa848007470 con 0x7fa84c09e930 2023-09-30T22:30:53.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.586+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa84803f300 con 0x7fa84c09e930 2023-09-30T22:30:53.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.586+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c09dfd0 con 0x7fa84c09e930 2023-09-30T22:30:53.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.591+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa8480123b0 con 0x7fa84c09e930 2023-09-30T22:30:53.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.797+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa84c09fdb0 con 0x7fa84c09e930 2023-09-30T22:30:53.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.797+0000 7fa8427fc700 1 -- 172.21.15.32:0/1678917017 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa848011360 con 0x7fa84c09e930 2023-09-30T22:30:53.799 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:53.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 msgr2=0x7fa84c06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 0x7fa84c06c1c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa84800c550 tx=0x7fa84800c860 comp rx=0 tx=0).stop 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 shutdown_connections 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa84c136fe0 0x7fa84c1c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa84c09e930 0x7fa84c06c1c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 --2- 172.21.15.32:0/1678917017 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa84c09f410 0x7fa84c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 >> 172.21.15.32:0/1678917017 conn(0x7fa84c0a4210 msgr2=0x7fa84c135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 shutdown_connections 2023-09-30T22:30:53.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:53.800+0000 7fa853d75700 1 -- 172.21.15.32:0/1678917017 wait complete. 2023-09-30T22:30:53.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:53.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:53.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:53.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:54.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.041+0000 7f166040a700 1 Processor -- start 2023-09-30T22:30:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f166040a700 1 -- start start 2023-09-30T22:30:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f166040a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16580a0a30 0x7f16580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f166040a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f16580a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f166040a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16580a2a70 con 0x7f1658133930 2023-09-30T22:30:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f166040a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16580a01d0 con 0x7f16580a0a30 2023-09-30T22:30:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f166040a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f165809e2b0 con 0x7f165809d630 2023-09-30T22:30:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.042+0000 7f165e9a7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1658133930 0x7f1658133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39534/0 (socket says 172.21.15.32:39534) 2023-09-30T22:30:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165e9a7700 1 -- 172.21.15.32:0/3202255196 learned_addr learned my addr 172.21.15.32:0/3202255196 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165e1a6700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16580a0a30 0x7f16580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d9a5700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f16580a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d9a5700 1 -- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16580a0a30 msgr2=0x7f16580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d1a4700 1 -- 172.21.15.32:0/3202255196 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953518125 0 0) 0x7f16580a2a70 con 0x7f1658133930 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d9a5700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16580a0a30 0x7f16580a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d9a5700 1 -- 172.21.15.32:0/3202255196 >> v1:172.21.15.32:6789/0 conn(0x7f1658133930 legacy=0x7f1658133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d9a5700 1 -- 172.21.15.32:0/3202255196 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165809dc50 con 0x7f165809d630 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165e1a6700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16580a0a30 0x7f16580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d9a5700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f16580a1340 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1648009870 tx=0x7f1648009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87bb49e012f35185 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d1a4700 1 -- 172.21.15.32:0/3202255196 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648011830 con 0x7f165809d630 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d1a4700 1 -- 172.21.15.32:0/3202255196 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1648011990 con 0x7f165809d630 2023-09-30T22:30:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.043+0000 7f165d1a4700 1 -- 172.21.15.32:0/3202255196 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648011d20 con 0x7f165809d630 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 -- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 msgr2=0x7f16580a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f16580a1340 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1648009870 tx=0x7f1648009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 -- 172.21.15.32:0/3202255196 shutdown_connections 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f16580a0a30 0x7f16580a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 --2- 172.21.15.32:0/3202255196 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f16580a1340 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 -- 172.21.15.32:0/3202255196 >> 172.21.15.32:0/3202255196 conn(0x7f16580a3050 msgr2=0x7f16580a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 -- 172.21.15.32:0/3202255196 shutdown_connections 2023-09-30T22:30:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.044+0000 7f166040a700 1 -- 172.21.15.32:0/3202255196 wait complete. 2023-09-30T22:30:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.045+0000 7f166040a700 1 Processor -- start 2023-09-30T22:30:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.045+0000 7f166040a700 1 -- start start 2023-09-30T22:30:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f166040a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f165811f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f166040a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16580a0a30 0x7f165811f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f166040a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 0x7f16581c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f166040a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165811ed60 con 0x7f16580a0a30 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f166040a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16581224a0 con 0x7f1658133930 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f166040a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1658120490 con 0x7f165809d630 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f165e9a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 0x7f16581c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f165e9a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 0x7f16581c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41786/0 (socket says 172.21.15.32:41786) 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f165e9a7700 1 -- 172.21.15.32:0/1583573099 learned_addr learned my addr 172.21.15.32:0/1583573099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f165d9a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16580a0a30 0x7f165811f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.046+0000 7f165e1a6700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f165811f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165e9a7700 1 -- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 msgr2=0x7f165811f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165e9a7700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f165811f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165e9a7700 1 -- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16580a0a30 msgr2=0x7f165811f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165e9a7700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16580a0a30 0x7f165811f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165e9a7700 1 -- 172.21.15.32:0/1583573099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540094a0 con 0x7f1658133930 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165d9a5700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16580a0a30 0x7f165811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f165e9a7700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 0x7f16581c7f90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1650002230 tx=0x7f165000cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165000aac0 con 0x7f1658133930 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165000ac40 con 0x7f1658133930 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16480094a0 con 0x7f1658133930 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500070d0 con 0x7f1658133930 2023-09-30T22:30:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.047+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16580a0410 con 0x7f1658133930 2023-09-30T22:30:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.048+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1650007250 con 0x7f1658133930 2023-09-30T22:30:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.048+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16581222e0 con 0x7f1658133930 2023-09-30T22:30:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.049+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f165003f4f0 con 0x7f1658133930 2023-09-30T22:30:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.053+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16581222e0 con 0x7f1658133930 2023-09-30T22:30:54.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 Processor -- start 2023-09-30T22:30:54.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 Processor -- start 2023-09-30T22:30:54.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 -- start start 2023-09-30T22:30:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 -- start start 2023-09-30T22:30:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 0x7f55bc09db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cf0134630 0x7f5cf0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0136f50 0x7f5cf0139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf009b4f0 con 0x7f5cf0131810 2023-09-30T22:30:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf01313c0 con 0x7f5cf0134630 2023-09-30T22:30:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f5cf6208700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf006e8c0 con 0x7f5cf0136f50 2023-09-30T22:30:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc13bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc06b0a0 con 0x7f55bc09e070 2023-09-30T22:30:54.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc06b6d0 con 0x7f55bc134130 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.085+0000 7f55c144c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc09b1c0 con 0x7f55bc09d760 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 0x7f55bc09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 0x7f55bc09db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55952/0 (socket says 172.21.15.32:55952) 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 -- 172.21.15.32:0/3555593328 learned_addr learned my addr 172.21.15.32:0/3555593328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0136f50 0x7f5cf0139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cf0134630 0x7f5cf0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55bb7fe700 1 --2- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc13bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cf0134630 0x7f5cf0136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49476/0 (socket says 172.21.15.32:49476) 2023-09-30T22:30:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 -- 172.21.15.32:0/3161136745 learned_addr learned my addr 172.21.15.32:0/3161136745 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0136f50 0x7f5cf0139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55954/0 (socket says 172.21.15.32:55954) 2023-09-30T22:30:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 -- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 msgr2=0x7f55bc13bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 --2- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc13bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 -- 172.21.15.32:0/3555593328 >> v1:172.21.15.32:6789/0 conn(0x7f55bc09e070 legacy=0x7f55bc133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 -- 172.21.15.32:0/3555593328 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc06b490 con 0x7f55bc09d760 2023-09-30T22:30:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 -- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0136f50 msgr2=0x7f5cf0139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 --2- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0136f50 0x7f5cf0139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 -- 172.21.15.32:0/3161136745 >> v1:172.21.15.32:6789/0 conn(0x7f5cf0131810 legacy=0x7f5cf0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:54.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f5ceeffd700 1 -- 172.21.15.32:0/3161136745 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0071d20 con 0x7f5cf0134630 2023-09-30T22:30:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.086+0000 7f55baffd700 1 --2- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 0x7f55bc09db30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009920 tx=0x7f55ac009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7610259163e611e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f5ceeffd700 1 --2- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0134630 0x7f5cf0136a10 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40098f0 tx=0x7f5ce4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69b74d975be6f95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f55b9ffb700 1 -- 172.21.15.32:0/3555593328 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac011830 con 0x7f55bc09d760 2023-09-30T22:30:54.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f55b9ffb700 1 -- 172.21.15.32:0/3555593328 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac011990 con 0x7f55bc09d760 2023-09-30T22:30:54.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f55b9ffb700 1 -- 172.21.15.32:0/3555593328 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac011d20 con 0x7f55bc09d760 2023-09-30T22:30:54.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f5cee7fc700 1 -- 172.21.15.32:0/3161136745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4004230 con 0x7f5cf0134630 2023-09-30T22:30:54.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f5cee7fc700 1 -- 172.21.15.32:0/3161136745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4004390 con 0x7f5cf0134630 2023-09-30T22:30:54.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f5cee7fc700 1 -- 172.21.15.32:0/3161136745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40036a0 con 0x7f5cf0134630 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f55c144c700 1 -- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 msgr2=0x7f55bc09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.087+0000 7f55c144c700 1 --2- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 0x7f55bc09db30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009920 tx=0x7f55ac009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f55c144c700 1 -- 172.21.15.32:0/3555593328 shutdown_connections 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f55c144c700 1 --2- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc13bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f55c144c700 1 --2- 172.21.15.32:0/3555593328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09d760 0x7f55bc09db30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f55c144c700 1 -- 172.21.15.32:0/3555593328 >> 172.21.15.32:0/3555593328 conn(0x7f55bc066f50 msgr2=0x7f55bc0a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 -- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0134630 msgr2=0x7f5cf0136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 --2- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0134630 0x7f5cf0136a10 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40098f0 tx=0x7f5ce4009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f55c144c700 1 -- 172.21.15.32:0/3555593328 shutdown_connections 2023-09-30T22:30:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 -- 172.21.15.32:0/3161136745 shutdown_connections 2023-09-30T22:30:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 --2- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0136f50 0x7f5cf0139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 --2- 172.21.15.32:0/3161136745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0134630 0x7f5cf0136a10 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 -- 172.21.15.32:0/3161136745 >> 172.21.15.32:0/3161136745 conn(0x7f5cf00a2aa0 msgr2=0x7f5cf00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f55c144c700 1 -- 172.21.15.32:0/3555593328 wait complete. 2023-09-30T22:30:54.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 -- 172.21.15.32:0/3161136745 shutdown_connections 2023-09-30T22:30:54.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.088+0000 7f5cf6208700 1 -- 172.21.15.32:0/3161136745 wait complete. 2023-09-30T22:30:54.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f5cf6208700 1 Processor -- start 2023-09-30T22:30:54.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f55c144c700 1 Processor -- start 2023-09-30T22:30:54.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f55c144c700 1 -- start start 2023-09-30T22:30:54.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f55c144c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc09d760 0x7f55bc1c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f55c144c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 0x7f55bc1c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55c144c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc11e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55c144c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc133420 con 0x7f55bc09d760 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55c144c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1205d0 con 0x7f55bc134130 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55c144c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1c93e0 con 0x7f55bc09e070 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f5cf6208700 1 -- start start 2023-09-30T22:30:54.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.089+0000 7f5cf6208700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0131810 0x7f5cf011b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5cf6208700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0134630 0x7f5cf011b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5cf6208700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cf0136f50 0x7f5cf011bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5cf6208700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf011c3a0 con 0x7f5cf0131810 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5cf6208700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf011a3f0 con 0x7f5cf0136f50 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5cf6208700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf0070a90 con 0x7f5cf0134630 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 0x7f55bc1c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 0x7f55bc1c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55958/0 (socket says 172.21.15.32:55958) 2023-09-30T22:30:54.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 -- 172.21.15.32:0/4050781053 learned_addr learned my addr 172.21.15.32:0/4050781053 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:54.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0134630 0x7f5cf011b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceeffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0134630 0x7f5cf011b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55972/0 (socket says 172.21.15.32:55972) 2023-09-30T22:30:54.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceeffd700 1 -- 172.21.15.32:0/2570694592 learned_addr learned my addr 172.21.15.32:0/2570694592 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:54.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceffff700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cf0136f50 0x7f5cf011bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55bb7fe700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc11e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55baffd700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc09d760 0x7f55bc1c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5cef7fe700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0131810 0x7f5cf011b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 -- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 msgr2=0x7f55bc11e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc11e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 -- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc09d760 msgr2=0x7f55bc1c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc09d760 0x7f55bc1c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55ba7fc700 1 -- 172.21.15.32:0/4050781053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac009550 con 0x7f55bc09e070 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f55baffd700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc09d760 0x7f55bc1c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceffff700 1 -- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0134630 msgr2=0x7f5cf011b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceffff700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0134630 0x7f5cf011b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceffff700 1 -- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0131810 msgr2=0x7f5cf011b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceffff700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0131810 0x7f5cf011b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.090+0000 7f5ceffff700 1 -- 172.21.15.32:0/2570694592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4009550 con 0x7f5cf0136f50 2023-09-30T22:30:54.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f55ba7fc700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 0x7f55bc1c8ea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c530 tx=0x7f55b000c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f5ceffff700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cf0136f50 0x7f5cf011bd30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00052d0 con 0x7f55bc09e070 2023-09-30T22:30:54.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0005430 con 0x7f55bc09e070 2023-09-30T22:30:54.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc133ab0 con 0x7f55bc09e070 2023-09-30T22:30:54.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00038c0 con 0x7f55bc09e070 2023-09-30T22:30:54.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc09c740 con 0x7f55bc09e070 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800cd40 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce800f040 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf009a7a0 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80146c0 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.091+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf01191b0 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.092+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf011a530 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.092+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ce8014820 con 0x7f5cf0136f50 2023-09-30T22:30:54.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.093+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5ce80402c0 con 0x7f5cf0136f50 2023-09-30T22:30:54.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.092+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55b0003a20 con 0x7f55bc09e070 2023-09-30T22:30:54.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.093+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f55b003eb20 con 0x7f55bc09e070 2023-09-30T22:30:54.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.093+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc0729a0 con 0x7f55bc09e070 2023-09-30T22:30:54.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.098+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5cf011a530 con 0x7f5cf0136f50 2023-09-30T22:30:54.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.098+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55b0019e60 con 0x7f55bc09e070 2023-09-30T22:30:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.264+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f16581222e0 con 0x7f1658133930 2023-09-30T22:30:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.265+0000 7f164effd700 1 -- 172.21.15.32:0/1583573099 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16581222e0 con 0x7f1658133930 2023-09-30T22:30:54.266 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 msgr2=0x7f16581c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 0x7f16581c7f90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1650002230 tx=0x7f165000cf50 comp rx=0 tx=0).stop 2023-09-30T22:30:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 shutdown_connections 2023-09-30T22:30:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1658133930 0x7f16581c7f90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f165809d630 0x7f165811f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 --2- 172.21.15.32:0/1583573099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16580a0a30 0x7f165811f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 >> 172.21.15.32:0/1583573099 conn(0x7f16580a3050 msgr2=0x7f16581bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 shutdown_connections 2023-09-30T22:30:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.267+0000 7f166040a700 1 -- 172.21.15.32:0/1583573099 wait complete. 2023-09-30T22:30:54.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.305+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5cf011a530 con 0x7f5cf0136f50 2023-09-30T22:30:54.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.305+0000 7f5cd7fff700 1 -- 172.21.15.32:0/2570694592 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5cf011a530 con 0x7f5cf0136f50 2023-09-30T22:30:54.307 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cf0136f50 msgr2=0x7f5cf011bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cf0136f50 0x7f5cf011bd30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 shutdown_connections 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cf0136f50 0x7f5cf011bd30 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf0131810 0x7f5cf011b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 --2- 172.21.15.32:0/2570694592 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5cf0134630 0x7f5cf011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 >> 172.21.15.32:0/2570694592 conn(0x7f5cf00a2aa0 msgr2=0x7f5cf0099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 shutdown_connections 2023-09-30T22:30:54.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.308+0000 7f5cf6208700 1 -- 172.21.15.32:0/2570694592 wait complete. 2023-09-30T22:30:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.313+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f55bc0747f0 con 0x7f55bc09e070 2023-09-30T22:30:54.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.313+0000 7f55a37fe700 1 -- 172.21.15.32:0/4050781053 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55b0019a70 con 0x7f55bc09e070 2023-09-30T22:30:54.314 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 msgr2=0x7f55bc1c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 0x7f55bc1c8ea0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c530 tx=0x7f55b000c840 comp rx=0 tx=0).stop 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 shutdown_connections 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f55bc134130 0x7f55bc11e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc09d760 0x7f55bc1c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 --2- 172.21.15.32:0/4050781053 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f55bc09e070 0x7f55bc1c8ea0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 >> 172.21.15.32:0/4050781053 conn(0x7f55bc066f50 msgr2=0x7f55bc0a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 shutdown_connections 2023-09-30T22:30:54.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.316+0000 7f55c144c700 1 -- 172.21.15.32:0/4050781053 wait complete. 2023-09-30T22:30:54.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:54.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:54.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.960+0000 7f4cb77fd700 1 Processor -- start 2023-09-30T22:30:54.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb77fd700 1 -- start start 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb77fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006a770 0x7f4cb006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb77fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006b080 0x7f4cb0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb77fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb009ece0 con 0x7f4cb0131b40 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb77fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb009e760 con 0x7f4cb006a770 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb77fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb009e040 con 0x7f4cb006b080 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb4d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006b080 0x7f4cb0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb5d9a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4cb0131b40 0x7f4cb0138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39578/0 (socket says 172.21.15.32:39578) 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb5599700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006a770 0x7f4cb006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.961+0000 7f4cb5d9a700 1 -- 172.21.15.32:0/2239815718 learned_addr learned my addr 172.21.15.32:0/2239815718 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb4d98700 1 -- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006a770 msgr2=0x7f4cb006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb4d98700 1 --2- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006a770 0x7f4cb006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4ca3fff700 1 -- 172.21.15.32:0/2239815718 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164651316 0 0) 0x7f4cb009ece0 con 0x7f4cb0131b40 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb4d98700 1 -- 172.21.15.32:0/2239815718 >> v1:172.21.15.32:6789/0 conn(0x7f4cb0131b40 legacy=0x7f4cb0138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb4d98700 1 -- 172.21.15.32:0/2239815718 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb009f630 con 0x7f4cb006b080 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb4d98700 1 --2- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006b080 0x7f4cb0131600 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009870 tx=0x7f4ca4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18a74a7abbcc2c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4ca3fff700 1 -- 172.21.15.32:0/2239815718 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011830 con 0x7f4cb006b080 2023-09-30T22:30:54.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4ca3fff700 1 -- 172.21.15.32:0/2239815718 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4011990 con 0x7f4cb006b080 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4ca3fff700 1 -- 172.21.15.32:0/2239815718 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011d20 con 0x7f4cb006b080 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb77fd700 1 -- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006b080 msgr2=0x7f4cb0131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.962+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006b080 0x7f4cb0131600 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009870 tx=0x7f4ca4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.963+0000 7f4cb77fd700 1 -- 172.21.15.32:0/2239815718 shutdown_connections 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.963+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006a770 0x7f4cb006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.963+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/2239815718 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006b080 0x7f4cb0131600 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.963+0000 7f4cb77fd700 1 -- 172.21.15.32:0/2239815718 >> 172.21.15.32:0/2239815718 conn(0x7f4cb0066f50 msgr2=0x7f4cb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.963+0000 7f4cb77fd700 1 -- 172.21.15.32:0/2239815718 shutdown_connections 2023-09-30T22:30:54.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.963+0000 7f4cb77fd700 1 -- 172.21.15.32:0/2239815718 wait complete. 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 Processor -- start 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 -- start start 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006a770 0x7f4cb009ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 0x7f4cb009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0131b40 0x7f4cb01c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb006ece0 con 0x7f4cb0131b40 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0125f50 con 0x7f4cb006b080 2023-09-30T22:30:54.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb77fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb01311f0 con 0x7f4cb006a770 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb5599700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006a770 0x7f4cb009ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb4d98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 0x7f4cb009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb4d98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 0x7f4cb009b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41828/0 (socket says 172.21.15.32:41828) 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb5599700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006a770 0x7f4cb009ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:55976/0 (socket says 172.21.15.32:55976) 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb4d98700 1 -- 172.21.15.32:0/3700667846 learned_addr learned my addr 172.21.15.32:0/3700667846 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.964+0000 7f4cb5d9a700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0131b40 0x7f4cb01c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb4d98700 1 -- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006a770 msgr2=0x7f4cb009ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb4d98700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006a770 0x7f4cb009ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb4d98700 1 -- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0131b40 msgr2=0x7f4cb01c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb4d98700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0131b40 0x7f4cb01c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb4d98700 1 -- 172.21.15.32:0/3700667846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40094a0 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb5d9a700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0131b40 0x7f4cb01c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb4d98700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 0x7f4cb009b2b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40090b0 tx=0x7f4ca4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4004510 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca401f070 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb0125c90 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40038c0 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.965+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb01269c0 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.966+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ca4003a20 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.966+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb0074ea0 con 0x7f4cb006b080 2023-09-30T22:30:54.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.966+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4ca4021070 con 0x7f4cb006b080 2023-09-30T22:30:54.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:54.971+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ca401a430 con 0x7f4cb006b080 2023-09-30T22:30:55.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.178+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4cb006c360 con 0x7f4cb006b080 2023-09-30T22:30:55.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.178+0000 7f4ca1ffb700 1 -- 172.21.15.32:0/3700667846 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ca402b070 con 0x7f4cb006b080 2023-09-30T22:30:55.180 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:55.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 msgr2=0x7f4cb009b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 0x7f4cb009b2b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40090b0 tx=0x7f4ca4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 shutdown_connections 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cb0131b40 0x7f4cb01c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4cb006a770 0x7f4cb009ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 --2- 172.21.15.32:0/3700667846 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cb006b080 0x7f4cb009b2b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 >> 172.21.15.32:0/3700667846 conn(0x7f4cb0066f50 msgr2=0x7f4cb006ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 shutdown_connections 2023-09-30T22:30:55.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.180+0000 7f4cb77fd700 1 -- 172.21.15.32:0/3700667846 wait complete. 2023-09-30T22:30:55.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:55.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:55.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:55.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.427+0000 7fa52c36d700 1 Processor -- start 2023-09-30T22:30:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.427+0000 7fa52c36d700 1 -- start start 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.427+0000 7fa52c36d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa524136820 0x7fa524138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.427+0000 7fa52c36d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa52413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa52c36d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52406bc00 con 0x7fa524133a80 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa52c36d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52406c570 con 0x7fa524136820 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa52c36d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240a0040 con 0x7fa524139140 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa524136820 0x7fa524138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa524136820 0x7fa524138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49508/0 (socket says 172.21.15.32:49508) 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 -- 172.21.15.32:0/3696995982 learned_addr learned my addr 172.21.15.32:0/3696995982 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa52a90a700 1 --2- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa52413b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 -- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 msgr2=0x7fa52413b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 --2- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa52413b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 -- 172.21.15.32:0/3696995982 >> v1:172.21.15.32:6789/0 conn(0x7fa524133a80 legacy=0x7fa524133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 -- 172.21.15.32:0/3696995982 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa524133860 con 0x7fa524136820 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529107700 1 -- 172.21.15.32:0/3696995982 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045609507 0 0) 0x7fa52406bc00 con 0x7fa524133a80 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529908700 1 --2- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524136820 0x7fa524138c00 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa5140098f0 tx=0x7fa514009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c52b7d243fba21e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529107700 1 -- 172.21.15.32:0/3696995982 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514011830 con 0x7fa524136820 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.428+0000 7fa529107700 1 -- 172.21.15.32:0/3696995982 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa514011990 con 0x7fa524136820 2023-09-30T22:30:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa529107700 1 -- 172.21.15.32:0/3696995982 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514011d20 con 0x7fa524136820 2023-09-30T22:30:55.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 -- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524136820 msgr2=0x7fa524138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 --2- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524136820 0x7fa524138c00 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa5140098f0 tx=0x7fa514009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 -- 172.21.15.32:0/3696995982 shutdown_connections 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 --2- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa52413b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 --2- 172.21.15.32:0/3696995982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524136820 0x7fa524138c00 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 -- 172.21.15.32:0/3696995982 >> 172.21.15.32:0/3696995982 conn(0x7fa5240a4210 msgr2=0x7fa5240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 -- 172.21.15.32:0/3696995982 shutdown_connections 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.429+0000 7fa52c36d700 1 -- 172.21.15.32:0/3696995982 wait complete. 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.430+0000 7fa52c36d700 1 Processor -- start 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.430+0000 7fa52c36d700 1 -- start start 2023-09-30T22:30:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52c36d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 0x7fa52409d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52c36d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa524136820 0x7fa52409d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52c36d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa5241c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52c36d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5241309b0 con 0x7fa524133a80 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52c36d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa524138ca0 con 0x7fa524139140 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52c36d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa52409df20 con 0x7fa524136820 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52a109700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 0x7fa52409d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52a109700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 0x7fa52409d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49516/0 (socket says 172.21.15.32:49516) 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52a90a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa5241c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa52a109700 1 -- 172.21.15.32:0/2288335804 learned_addr learned my addr 172.21.15.32:0/2288335804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.431+0000 7fa529908700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa524136820 0x7fa52409d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a90a700 1 -- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa524136820 msgr2=0x7fa52409d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a90a700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa524136820 0x7fa52409d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a90a700 1 -- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 msgr2=0x7fa52409d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a90a700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 0x7fa52409d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a90a700 1 -- 172.21.15.32:0/2288335804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514009550 con 0x7fa524139140 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a109700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 0x7fa52409d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52a90a700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa5241c8330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00ef90 tx=0x7fa51c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00cd40 con 0x7fa524139140 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c00f040 con 0x7fa524139140 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0146c0 con 0x7fa524139140 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52409c040 con 0x7fa524139140 2023-09-30T22:30:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.432+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52409bd30 con 0x7fa524139140 2023-09-30T22:30:55.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.433+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa51c014820 con 0x7fa524139140 2023-09-30T22:30:55.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.433+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5240a12d0 con 0x7fa524139140 2023-09-30T22:30:55.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.433+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa51c0401f0 con 0x7fa524139140 2023-09-30T22:30:55.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.438+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa51c007900 con 0x7fa524139140 2023-09-30T22:30:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.479+0000 7fbf28626700 1 Processor -- start 2023-09-30T22:30:55.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.479+0000 7fbf28626700 1 -- start start 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf28626700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf2006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf28626700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf28626700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf20131b40 0x7fbf20131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf28626700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2009ece0 con 0x7fbf20131b40 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf28626700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2009e760 con 0x7fbf2006b080 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf28626700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2009e040 con 0x7fbf2006a770 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41854/0 (socket says 172.21.15.32:41854) 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 -- 172.21.15.32:0/3699657147 learned_addr learned my addr 172.21.15.32:0/3699657147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:55.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf263c2700 1 --2- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf2006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf26bc3700 1 --2- 172.21.15.32:0/3699657147 >> v2:172.21.15.32:3300/0 conn(0x7fbf20131b40 0x7fbf20131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 -- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 msgr2=0x7fbf2006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 --2- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf2006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 -- 172.21.15.32:0/3699657147 >> v2:172.21.15.32:3300/0 conn(0x7fbf20131b40 msgr2=0x7fbf20131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 --2- 172.21.15.32:0/3699657147 >> v2:172.21.15.32:3300/0 conn(0x7fbf20131b40 0x7fbf20131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.480+0000 7fbf25bc1700 1 -- 172.21.15.32:0/3699657147 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2009f630 con 0x7fbf2006b080 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.481+0000 7fbf25bc1700 1 --2- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20131600 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbf100098f0 tx=0x7fbf10009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c53b58126142b0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.481+0000 7fbf253c0700 1 -- 172.21.15.32:0/3699657147 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10011830 con 0x7fbf2006b080 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.481+0000 7fbf253c0700 1 -- 172.21.15.32:0/3699657147 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf10011990 con 0x7fbf2006b080 2023-09-30T22:30:55.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.481+0000 7fbf253c0700 1 -- 172.21.15.32:0/3699657147 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10011d20 con 0x7fbf2006b080 2023-09-30T22:30:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 -- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 msgr2=0x7fbf20131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 --2- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20131600 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbf100098f0 tx=0x7fbf10009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 -- 172.21.15.32:0/3699657147 shutdown_connections 2023-09-30T22:30:55.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 --2- 172.21.15.32:0/3699657147 >> v2:172.21.15.32:3300/0 conn(0x7fbf20131b40 0x7fbf20131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 --2- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf2006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 --2- 172.21.15.32:0/3699657147 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20131600 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 -- 172.21.15.32:0/3699657147 >> 172.21.15.32:0/3699657147 conn(0x7fbf20066f50 msgr2=0x7fbf2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 -- 172.21.15.32:0/3699657147 shutdown_connections 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.482+0000 7fbf28626700 1 -- 172.21.15.32:0/3699657147 wait complete. 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 Processor -- start 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 -- start start 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf20120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf20131b40 0x7fbf20120ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201313e0 con 0x7fbf20131b40 2023-09-30T22:30:55.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf20123830 con 0x7fbf2006b080 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf28626700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200a38f0 con 0x7fbf2006a770 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf25bc1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf263c2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf20120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf25bc1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20120580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41860/0 (socket says 172.21.15.32:41860) 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.483+0000 7fbf25bc1700 1 -- 172.21.15.32:0/2750615499 learned_addr learned my addr 172.21.15.32:0/2750615499 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf26bc3700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf20131b40 0x7fbf20120ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf263c2700 1 -- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 msgr2=0x7fbf20120580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf263c2700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20120580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf263c2700 1 -- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf20131b40 msgr2=0x7fbf20120ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf263c2700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf20131b40 0x7fbf20120ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf263c2700 1 -- 172.21.15.32:0/2750615499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009550 con 0x7fbf2006a770 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf263c2700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf20120040 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009ca0 tx=0x7fbf1c010f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c015070 con 0x7fbf2006a770 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00b930 con 0x7fbf2006a770 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf20071e60 con 0x7fbf2006a770 2023-09-30T22:30:55.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00bca0 con 0x7fbf2006a770 2023-09-30T22:30:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.484+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf20123b10 con 0x7fbf2006a770 2023-09-30T22:30:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.485+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf1c019410 con 0x7fbf2006a770 2023-09-30T22:30:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.485+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbf1c042eb0 con 0x7fbf2006a770 2023-09-30T22:30:55.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.486+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20074ea0 con 0x7fbf2006a770 2023-09-30T22:30:55.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.495+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf1c01e080 con 0x7fbf2006a770 2023-09-30T22:30:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 Processor -- start 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 -- start start 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 0x7fbf2c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 0x7fbf2c13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2c09d2f0 con 0x7fbf2c09e000 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2c09a810 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.530+0000 7fbf308ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c09a470 con 0x7fbf2c134100 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 0x7fbf2c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 0x7fbf2c13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 0x7fbf2c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49540/0 (socket says 172.21.15.32:49540) 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 0x7fbf2c13bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56014/0 (socket says 172.21.15.32:56014) 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2a59c700 1 -- 172.21.15.32:0/3046832358 learned_addr learned my addr 172.21.15.32:0/3046832358 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 -- 172.21.15.32:0/3046832358 >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 msgr2=0x7fbf2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 --2- 172.21.15.32:0/3046832358 >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 0x7fbf2c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 -- 172.21.15.32:0/3046832358 >> v1:172.21.15.32:6789/0 conn(0x7fbf2c09e000 legacy=0x7fbf2c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 -- 172.21.15.32:0/3046832358 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c09ec40 con 0x7fbf2c134100 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2959a700 1 -- 172.21.15.32:0/3046832358 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865475346 0 0) 0x7fbf2c09d2f0 con 0x7fbf2c09e000 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2a59c700 1 --2- 172.21.15.32:0/3046832358 >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 0x7fbf2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2ad9d700 1 --2- 172.21.15.32:0/3046832358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 0x7fbf2c13bc50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009d90 tx=0x7fbf2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e56a29a3a497b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.531+0000 7fbf2959a700 1 -- 172.21.15.32:0/3046832358 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400e070 con 0x7fbf2c134100 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf2959a700 1 -- 172.21.15.32:0/3046832358 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2400bb20 con 0x7fbf2c134100 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf2959a700 1 -- 172.21.15.32:0/3046832358 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24012430 con 0x7fbf2c134100 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 -- 172.21.15.32:0/3046832358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 msgr2=0x7fbf2c13bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 --2- 172.21.15.32:0/3046832358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 0x7fbf2c13bc50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009d90 tx=0x7fbf2400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 -- 172.21.15.32:0/3046832358 shutdown_connections 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 --2- 172.21.15.32:0/3046832358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c134100 0x7fbf2c13bc50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 --2- 172.21.15.32:0/3046832358 >> v2:172.21.15.32:3300/0 conn(0x7fbf2c09d6f0 0x7fbf2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 -- 172.21.15.32:0/3046832358 >> 172.21.15.32:0/3046832358 conn(0x7fbf2c066f50 msgr2=0x7fbf2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 -- 172.21.15.32:0/3046832358 shutdown_connections 2023-09-30T22:30:55.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.532+0000 7fbf308ec700 1 -- 172.21.15.32:0/3046832358 wait complete. 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.533+0000 7fbf308ec700 1 Processor -- start 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 -- start start 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 0x7fbf2c122760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c09e000 0x7fbf2c122ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2c134100 0x7fbf2c1231e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c0a1e20 con 0x7fbf2c09e000 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c133720 con 0x7fbf2c134100 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf308ec700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c124480 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf2a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 0x7fbf2c122760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf2a59c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 0x7fbf2c122760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56020/0 (socket says 172.21.15.32:56020) 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf2a59c700 1 -- 172.21.15.32:0/540318857 learned_addr learned my addr 172.21.15.32:0/540318857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf29d9b700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c09e000 0x7fbf2c122ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.534+0000 7fbf2ad9d700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2c134100 0x7fbf2c1231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf2a59c700 1 -- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2c134100 msgr2=0x7fbf2c1231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf2a59c700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2c134100 0x7fbf2c1231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf2a59c700 1 -- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c09e000 msgr2=0x7fbf2c122ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf2a59c700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c09e000 0x7fbf2c122ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf2a59c700 1 -- 172.21.15.32:0/540318857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c009550 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf29d9b700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c09e000 0x7fbf2c122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf2a59c700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 0x7fbf2c122760 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00ec10 tx=0x7fbf1c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00c850 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00c9b0 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00cd40 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf24009a70 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.535+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c133260 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.536+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf1c013470 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.536+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c074ea0 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.536+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbf1c03fbe0 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.546+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf1c013720 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.651+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa524130bf0 con 0x7fa524139140 2023-09-30T22:30:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.651+0000 7fa51affd700 1 -- 172.21.15.32:0/2288335804 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa51c018020 con 0x7fa524139140 2023-09-30T22:30:55.652 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.654+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 msgr2=0x7fa5241c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.654+0000 7fa52c36d700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa5241c8330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00ef90 tx=0x7fa51c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.654+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 shutdown_connections 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.654+0000 7fa52c36d700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa524139140 0x7fa5241c8330 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.654+0000 7fa52c36d700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa524133a80 0x7fa52409d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.654+0000 7fa52c36d700 1 --2- 172.21.15.32:0/2288335804 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa524136820 0x7fa52409d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.655+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 >> 172.21.15.32:0/2288335804 conn(0x7fa5240a4210 msgr2=0x7fa524134600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.655+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 shutdown_connections 2023-09-30T22:30:55.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.655+0000 7fa52c36d700 1 -- 172.21.15.32:0/2288335804 wait complete. 2023-09-30T22:30:55.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:55.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.727+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbf20123650 con 0x7fbf2006a770 2023-09-30T22:30:55.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.727+0000 7fbf16ffd700 1 -- 172.21.15.32:0/2750615499 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf1c0196c0 con 0x7fbf2006a770 2023-09-30T22:30:55.728 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.729+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 msgr2=0x7fbf20120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.729+0000 7fbf28626700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf20120040 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009ca0 tx=0x7fbf1c010f60 comp rx=0 tx=0).stop 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.729+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 shutdown_connections 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.729+0000 7fbf28626700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf20131b40 0x7fbf20120ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.729+0000 7fbf28626700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2006a770 0x7fbf20120040 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.729+0000 7fbf28626700 1 --2- 172.21.15.32:0/2750615499 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2006b080 0x7fbf20120580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.730+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 >> 172.21.15.32:0/2750615499 conn(0x7fbf20066f50 msgr2=0x7fbf2006fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.730+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 shutdown_connections 2023-09-30T22:30:55.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.730+0000 7fbf28626700 1 -- 172.21.15.32:0/2750615499 wait complete. 2023-09-30T22:30:55.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:55.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.754+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbf2c09cb80 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.754+0000 7fbf1affd700 1 -- 172.21.15.32:0/540318857 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf1c011b40 con 0x7fbf2c09d6f0 2023-09-30T22:30:55.755 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 msgr2=0x7fbf2c122760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 0x7fbf2c122760 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00ec10 tx=0x7fbf1c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 shutdown_connections 2023-09-30T22:30:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbf2c134100 0x7fbf2c1231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbf2c09d6f0 0x7fbf2c122760 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 --2- 172.21.15.32:0/540318857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2c09e000 0x7fbf2c122ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 >> 172.21.15.32:0/540318857 conn(0x7fbf2c066f50 msgr2=0x7fbf2c0a3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 shutdown_connections 2023-09-30T22:30:55.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:55.757+0000 7fbf308ec700 1 -- 172.21.15.32:0/540318857 wait complete. 2023-09-30T22:30:55.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:56.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.340+0000 7fc8dcfa2700 1 Processor -- start 2023-09-30T22:30:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.340+0000 7fc8dcfa2700 1 -- start start 2023-09-30T22:30:56.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8dcfa2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8dcfa2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8d809f650 0x7fc8d8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8dcfa2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d8070940 con 0x7fc8d809f240 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8dcfa2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d806fd60 con 0x7fc8d809f650 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8dcfa2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d806fa80 con 0x7fc8d809e930 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d6d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8d809f650 0x7fc8d8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41872/0 (socket says 172.21.15.32:41872) 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 -- 172.21.15.32:0/1784184832 learned_addr learned my addr 172.21.15.32:0/1784184832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d5d9b700 1 --1- 172.21.15.32:0/1784184832 >> v1:172.21.15.32:6789/0 conn(0x7fc8d809f240 0x7fc8d81317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39610/0 (socket says 172.21.15.32:39610) 2023-09-30T22:30:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 -- 172.21.15.32:0/1784184832 >> v2:172.21.15.32:3300/0 conn(0x7fc8d809f650 msgr2=0x7fc8d8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 --2- 172.21.15.32:0/1784184832 >> v2:172.21.15.32:3300/0 conn(0x7fc8d809f650 0x7fc8d8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 -- 172.21.15.32:0/1784184832 >> v1:172.21.15.32:6789/0 conn(0x7fc8d809f240 legacy=0x7fc8d81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d659c700 1 -- 172.21.15.32:0/1784184832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d809fb60 con 0x7fc8d809e930 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.341+0000 7fc8d6d9d700 1 --2- 172.21.15.32:0/1784184832 >> v2:172.21.15.32:3300/0 conn(0x7fc8d809f650 0x7fc8d8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8d659c700 1 --2- 172.21.15.32:0/1784184832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d809ed00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80098f0 tx=0x7fc8c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e171c0c5e08a82f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8d559a700 1 -- 172.21.15.32:0/1784184832 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011830 con 0x7fc8d809e930 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8d559a700 1 -- 172.21.15.32:0/1784184832 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8011990 con 0x7fc8d809e930 2023-09-30T22:30:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8d559a700 1 -- 172.21.15.32:0/1784184832 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011d20 con 0x7fc8d809e930 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1784184832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 msgr2=0x7fc8d809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1784184832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d809ed00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80098f0 tx=0x7fc8c8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1784184832 shutdown_connections 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1784184832 >> v2:172.21.15.32:3300/0 conn(0x7fc8d809f650 0x7fc8d8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1784184832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d809ed00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.342+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1784184832 >> 172.21.15.32:0/1784184832 conn(0x7fc8d80a4210 msgr2=0x7fc8d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.343+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1784184832 shutdown_connections 2023-09-30T22:30:56.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.343+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1784184832 wait complete. 2023-09-30T22:30:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.344+0000 7fc8dcfa2700 1 Processor -- start 2023-09-30T22:30:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.344+0000 7fc8dcfa2700 1 -- start start 2023-09-30T22:30:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8dcfa2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8dcfa2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d809f240 0x7fc8d806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8dcfa2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8d809f650 0x7fc8d806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8dcfa2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d81b7240 con 0x7fc8d809f240 2023-09-30T22:30:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8dcfa2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d81a93f0 con 0x7fc8d809e930 2023-09-30T22:30:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8dcfa2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d81a8910 con 0x7fc8d809f650 2023-09-30T22:30:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41884/0 (socket says 172.21.15.32:41884) 2023-09-30T22:30:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 -- 172.21.15.32:0/1132137680 learned_addr learned my addr 172.21.15.32:0/1132137680 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d809f240 0x7fc8d806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d6d9d700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8d809f650 0x7fc8d806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 -- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8d809f650 msgr2=0x7fc8d806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8d809f650 0x7fc8d806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 -- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d809f240 msgr2=0x7fc8d806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d809f240 0x7fc8d806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d659c700 1 -- 172.21.15.32:0/1132137680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d0009a70 con 0x7fc8d809e930 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.345+0000 7fc8d5d9b700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d809f240 0x7fc8d806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.346+0000 7fc8d659c700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d806b8b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000c00 tx=0x7fc8c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.346+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c80043c0 con 0x7fc8d809e930 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.346+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c801f070 con 0x7fc8d809e930 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.346+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c80038c0 con 0x7fc8d809e930 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.346+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c8009550 con 0x7fc8d809e930 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.346+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d80a0190 con 0x7fc8d809e930 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.347+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc8c8021030 con 0x7fc8d809e930 2023-09-30T22:30:56.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.347+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc8c8040990 con 0x7fc8d809e930 2023-09-30T22:30:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.347+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d81a9180 con 0x7fc8d809e930 2023-09-30T22:30:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.352+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8d81a9180 con 0x7fc8d809e930 2023-09-30T22:30:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.558+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc8d81a9180 con 0x7fc8d809e930 2023-09-30T22:30:56.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.559+0000 7fc8c6ffd700 1 -- 172.21.15.32:0/1132137680 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8d81a9180 con 0x7fc8d809e930 2023-09-30T22:30:56.560 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 msgr2=0x7fc8d806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d806b8b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000c00 tx=0x7fc8c8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 shutdown_connections 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc8d809f650 0x7fc8d806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc8d809e930 0x7fc8d806b8b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 --2- 172.21.15.32:0/1132137680 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d809f240 0x7fc8d806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 >> 172.21.15.32:0/1132137680 conn(0x7fc8d80a4210 msgr2=0x7fc8d81b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 shutdown_connections 2023-09-30T22:30:56.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.561+0000 7fc8dcfa2700 1 -- 172.21.15.32:0/1132137680 wait complete. 2023-09-30T22:30:56.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:56.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:56.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:56.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:56.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.817+0000 7f37fdc8e700 1 Processor -- start 2023-09-30T22:30:56.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.817+0000 7f37fdc8e700 1 -- start start 2023-09-30T22:30:56.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37fdc8e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37f8131810 0x7f37f8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37fdc8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f8139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37fdc8e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f81313c0 con 0x7f37f8099a40 2023-09-30T22:30:56.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37fdc8e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f806e8c0 con 0x7f37f8131810 2023-09-30T22:30:56.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37fdc8e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8071d20 con 0x7f37f809a580 2023-09-30T22:30:56.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37f6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37f8099a40 0x7f37f8099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39614/0 (socket says 172.21.15.32:39614) 2023-09-30T22:30:56.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37f6ffd700 1 -- 172.21.15.32:0/3917502973 learned_addr learned my addr 172.21.15.32:0/3917502973 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:56.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37f7fff700 1 --2- 172.21.15.32:0/3917502973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f8139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37f77fe700 1 --2- 172.21.15.32:0/3917502973 >> v2:172.21.15.32:3300/0 conn(0x7f37f8131810 0x7f37f8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.818+0000 7f37f7fff700 1 -- 172.21.15.32:0/3917502973 >> v2:172.21.15.32:3300/0 conn(0x7f37f8131810 msgr2=0x7f37f8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f7fff700 1 --2- 172.21.15.32:0/3917502973 >> v2:172.21.15.32:3300/0 conn(0x7f37f8131810 0x7f37f8131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f7fff700 1 -- 172.21.15.32:0/3917502973 >> v1:172.21.15.32:6789/0 conn(0x7f37f8099a40 legacy=0x7f37f8099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f7fff700 1 -- 172.21.15.32:0/3917502973 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f80721a0 con 0x7f37f809a580 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f77fe700 1 --2- 172.21.15.32:0/3917502973 >> v2:172.21.15.32:3300/0 conn(0x7f37f8131810 0x7f37f8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f7fff700 1 --2- 172.21.15.32:0/3917502973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f8139660 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009d90 tx=0x7f37f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700c4747ac07ec58 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f67fc700 1 -- 172.21.15.32:0/3917502973 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000e070 con 0x7f37f809a580 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f67fc700 1 -- 172.21.15.32:0/3917502973 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000bb20 con 0x7f37f809a580 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.819+0000 7f37f67fc700 1 -- 172.21.15.32:0/3917502973 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0012430 con 0x7f37f809a580 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 -- 172.21.15.32:0/3917502973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 msgr2=0x7f37f8139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/3917502973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f8139660 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009d90 tx=0x7f37f000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 -- 172.21.15.32:0/3917502973 shutdown_connections 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/3917502973 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f8139660 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/3917502973 >> v2:172.21.15.32:3300/0 conn(0x7f37f8131810 0x7f37f8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 -- 172.21.15.32:0/3917502973 >> 172.21.15.32:0/3917502973 conn(0x7f37f80a2aa0 msgr2=0x7f37f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 -- 172.21.15.32:0/3917502973 shutdown_connections 2023-09-30T22:30:56.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.820+0000 7f37fdc8e700 1 -- 172.21.15.32:0/3917502973 wait complete. 2023-09-30T22:30:56.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.821+0000 7f37fdc8e700 1 Processor -- start 2023-09-30T22:30:56.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.821+0000 7f37fdc8e700 1 -- start start 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37fdc8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f8099a40 0x7f37f806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37fdc8e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37fdc8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f8131810 0x7f37f8070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37fdc8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f811e900 con 0x7f37f8131810 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37fdc8e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f80a1b00 con 0x7f37f8099a40 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37fdc8e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37f806eb00 con 0x7f37f809a580 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f806fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56044/0 (socket says 172.21.15.32:56044) 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 -- 172.21.15.32:0/2613556009 learned_addr learned my addr 172.21.15.32:0/2613556009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:56.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f77fe700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f8099a40 0x7f37f806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f7fff700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f8131810 0x7f37f8070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 -- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f8099a40 msgr2=0x7f37f806f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f8099a40 0x7f37f806f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 -- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f8131810 msgr2=0x7f37f8070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f8131810 0x7f37f8070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.822+0000 7f37f6ffd700 1 -- 172.21.15.32:0/2613556009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80094a0 con 0x7f37f809a580 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37f7fff700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f8131810 0x7f37f8070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37f77fe700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f8099a40 0x7f37f806f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37f6ffd700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f806fb50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c500 tx=0x7f37ec00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec005d70 con 0x7f37f809a580 2023-09-30T22:30:56.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec0070d0 con 0x7f37f809a580 2023-09-30T22:30:56.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f0009a70 con 0x7f37f809a580 2023-09-30T22:30:56.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec007440 con 0x7f37f809a580 2023-09-30T22:30:56.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.823+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f811ec70 con 0x7f37f809a580 2023-09-30T22:30:56.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.824+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37ec0075a0 con 0x7f37f809a580 2023-09-30T22:30:56.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.824+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f37ec03eaf0 con 0x7f37f809a580 2023-09-30T22:30:56.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.824+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f8074ea0 con 0x7f37f809a580 2023-09-30T22:30:56.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.829+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37ec017080 con 0x7f37f809a580 2023-09-30T22:30:56.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.908+0000 7fb6af4e2700 1 Processor -- start 2023-09-30T22:30:56.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.908+0000 7fb6af4e2700 1 -- start start 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6af4e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a81337f0 0x7fb6a8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6af4e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6af4e2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a806b990 con 0x7fb6a8133420 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6af4e2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a809b0e0 con 0x7fb6a81337f0 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6af4e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8133c20 con 0x7fb6a8138dc0 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ad27e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb6a8133420 0x7fb6a8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39630/0 (socket says 172.21.15.32:39630) 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ad27e700 1 -- 172.21.15.32:0/3235018984 learned_addr learned my addr 172.21.15.32:0/3235018984 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6aca7d700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a81337f0 0x7fb6a8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ada7f700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb69ffff700 1 -- 172.21.15.32:0/3235018984 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283487213 0 0) 0x7fb6a806b990 con 0x7fb6a8133420 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb69ffff700 1 -- 172.21.15.32:0/3235018984 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6940035b0 con 0x7fb6a8133420 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ada7f700 1 -- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a81337f0 msgr2=0x7fb6a8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ada7f700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a81337f0 0x7fb6a8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ada7f700 1 -- 172.21.15.32:0/3235018984 >> v1:172.21.15.32:6789/0 conn(0x7fb6a8133420 legacy=0x7fb6a8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6ada7f700 1 -- 172.21.15.32:0/3235018984 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80a0590 con 0x7fb6a8138dc0 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.909+0000 7fb6aca7d700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a81337f0 0x7fb6a8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6ada7f700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a813b1a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009dc0 tx=0x7fb6a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc47b857a02d064 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb69ffff700 1 -- 172.21.15.32:0/3235018984 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000e070 con 0x7fb6a8138dc0 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb69ffff700 1 -- 172.21.15.32:0/3235018984 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a000bb20 con 0x7fb6a8138dc0 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb69ffff700 1 -- 172.21.15.32:0/3235018984 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0012430 con 0x7fb6a8138dc0 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6af4e2700 1 -- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 msgr2=0x7fb6a813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a813b1a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009dc0 tx=0x7fb6a000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6af4e2700 1 -- 172.21.15.32:0/3235018984 shutdown_connections 2023-09-30T22:30:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a813b1a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/3235018984 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a81337f0 0x7fb6a8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.910+0000 7fb6af4e2700 1 -- 172.21.15.32:0/3235018984 >> 172.21.15.32:0/3235018984 conn(0x7fb6a80a4210 msgr2=0x7fb6a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.911+0000 7fb6af4e2700 1 -- 172.21.15.32:0/3235018984 shutdown_connections 2023-09-30T22:30:56.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.911+0000 7fb6af4e2700 1 -- 172.21.15.32:0/3235018984 wait complete. 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.912+0000 7fb6af4e2700 1 Processor -- start 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.912+0000 7fb6af4e2700 1 -- start start 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.912+0000 7fb6af4e2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 0x7fb6a812f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.912+0000 7fb6af4e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6a81337f0 0x7fb6a812f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.912+0000 7fb6af4e2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a812fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.912+0000 7fb6af4e2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8130c10 con 0x7fb6a81337f0 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6af4e2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a809e1c0 con 0x7fb6a8133420 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6af4e2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a809c8d0 con 0x7fb6a8138dc0 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 0x7fb6a812f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6aca7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6a81337f0 0x7fb6a812f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 0x7fb6a812f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41906/0 (socket says 172.21.15.32:41906) 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 -- 172.21.15.32:0/1221124388 learned_addr learned my addr 172.21.15.32:0/1221124388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ada7f700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a812fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 -- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 msgr2=0x7fb6a812fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a812fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 -- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6a81337f0 msgr2=0x7fb6a812f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6a81337f0 0x7fb6a812f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 -- 172.21.15.32:0/1221124388 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6980094a0 con 0x7fb6a8133420 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6ad27e700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 0x7fb6a812f290 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400c700 tx=0x7fb6a400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb6aca7d700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6a81337f0 0x7fb6a812f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.913+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4005d70 con 0x7fb6a8133420 2023-09-30T22:30:56.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.914+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a40070d0 con 0x7fb6a8133420 2023-09-30T22:30:56.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.914+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4007440 con 0x7fb6a8133420 2023-09-30T22:30:56.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.914+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a0009a70 con 0x7fb6a8133420 2023-09-30T22:30:56.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.914+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a809da50 con 0x7fb6a8133420 2023-09-30T22:30:56.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.915+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a8134040 con 0x7fb6a8133420 2023-09-30T22:30:56.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.916+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6a40075a0 con 0x7fb6a8133420 2023-09-30T22:30:56.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.916+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb6a403f2e0 con 0x7fb6a8133420 2023-09-30T22:30:56.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.920+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6a4023b70 con 0x7fb6a8133420 2023-09-30T22:30:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.929+0000 7f34bd51f700 1 Processor -- start 2023-09-30T22:30:56.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.929+0000 7f34bd51f700 1 -- start start 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.929+0000 7f34bd51f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b809d6f0 0x7f34b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.929+0000 7f34bd51f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.929+0000 7f34bd51f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b809b120 con 0x7f34b809e000 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.929+0000 7f34bd51f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b806b460 con 0x7f34b809d6f0 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34bd51f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b809a7e0 con 0x7f34b809e440 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b809d6f0 0x7f34b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b77fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b6ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b809d6f0 0x7f34b809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49580/0 (socket says 172.21.15.32:49580) 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b6ffd700 1 -- 172.21.15.32:0/3811312045 learned_addr learned my addr 172.21.15.32:0/3811312045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b77fe700 1 -- 172.21.15.32:0/3811312045 >> v2:172.21.15.32:3300/0 conn(0x7f34b809d6f0 msgr2=0x7f34b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b77fe700 1 --2- 172.21.15.32:0/3811312045 >> v2:172.21.15.32:3300/0 conn(0x7f34b809d6f0 0x7f34b809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b77fe700 1 -- 172.21.15.32:0/3811312045 >> v1:172.21.15.32:6789/0 conn(0x7f34b809e000 legacy=0x7f34b81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b77fe700 1 -- 172.21.15.32:0/3811312045 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b809a500 con 0x7f34b809e440 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b77fe700 1 --2- 172.21.15.32:0/3811312045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b8134100 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009dc0 tx=0x7f34b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6681db0614d6359 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.930+0000 7f34b5ffb700 1 -- 172.21.15.32:0/3811312045 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000e070 con 0x7f34b809e440 2023-09-30T22:30:56.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34b5ffb700 1 -- 172.21.15.32:0/3811312045 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000bae0 con 0x7f34b809e440 2023-09-30T22:30:56.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34b5ffb700 1 -- 172.21.15.32:0/3811312045 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000be90 con 0x7f34b809e440 2023-09-30T22:30:56.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 -- 172.21.15.32:0/3811312045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 msgr2=0x7f34b8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 --2- 172.21.15.32:0/3811312045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b8134100 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009dc0 tx=0x7f34b000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 -- 172.21.15.32:0/3811312045 shutdown_connections 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 --2- 172.21.15.32:0/3811312045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b8134100 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 --2- 172.21.15.32:0/3811312045 >> v2:172.21.15.32:3300/0 conn(0x7f34b809d6f0 0x7f34b809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 -- 172.21.15.32:0/3811312045 >> 172.21.15.32:0/3811312045 conn(0x7f34b8066f50 msgr2=0x7f34b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 -- 172.21.15.32:0/3811312045 shutdown_connections 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.931+0000 7f34bd51f700 1 -- 172.21.15.32:0/3811312045 wait complete. 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.932+0000 7f34bd51f700 1 Processor -- start 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 -- start start 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 0x7f34b81a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34b809e000 0x7f34b81a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b81a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81a81f0 con 0x7f34b809d6f0 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81a6b10 con 0x7f34b809e440 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34bd51f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81a86a0 con 0x7f34b809e000 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 0x7f34b81a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 0x7f34b81a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49594/0 (socket says 172.21.15.32:49594) 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34b6ffd700 1 -- 172.21.15.32:0/985262764 learned_addr learned my addr 172.21.15.32:0/985262764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.933+0000 7f34b67fc700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34b809e000 0x7f34b81a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b77fe700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b81a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b6ffd700 1 -- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34b809e000 msgr2=0x7f34b81a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b6ffd700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34b809e000 0x7f34b81a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b6ffd700 1 -- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 msgr2=0x7f34b81a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b6ffd700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b81a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b6ffd700 1 -- 172.21.15.32:0/985262764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b0009a70 con 0x7f34b809d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b67fc700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34b809e000 0x7f34b81a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b77fe700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b81a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34b6ffd700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 0x7f34b81a4e60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e910 tx=0x7f34a800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a80096a0 con 0x7f34b809d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a800c850 con 0x7f34b809d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b80a1fb0 con 0x7f34b809d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a800cc00 con 0x7f34b809d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.934+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b809c760 con 0x7f34b809d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.935+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34a800cd60 con 0x7f34b809d6f0 2023-09-30T22:30:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.935+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f34a803fb60 con 0x7f34b809d6f0 2023-09-30T22:30:56.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.936+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b81a8aa0 con 0x7f34b809d6f0 2023-09-30T22:30:56.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:56.946+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34a8017080 con 0x7f34b809d6f0 2023-09-30T22:30:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.075+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f37f8070a40 con 0x7f37f809a580 2023-09-30T22:30:57.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.076+0000 7f37dffff700 1 -- 172.21.15.32:0/2613556009 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37ec019b40 con 0x7f37f809a580 2023-09-30T22:30:57.077 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:57.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 msgr2=0x7f37f806fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f806fb50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c500 tx=0x7f37ec00c810 comp rx=0 tx=0).stop 2023-09-30T22:30:57.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 shutdown_connections 2023-09-30T22:30:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f8131810 0x7f37f8070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f8099a40 0x7f37f806f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 --2- 172.21.15.32:0/2613556009 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f37f809a580 0x7f37f806fb50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 >> 172.21.15.32:0/2613556009 conn(0x7f37f80a2aa0 msgr2=0x7f37f812e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 shutdown_connections 2023-09-30T22:30:57.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.078+0000 7f37fdc8e700 1 -- 172.21.15.32:0/2613556009 wait complete. 2023-09-30T22:30:57.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.140+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb6a8074ea0 con 0x7fb6a8133420 2023-09-30T22:30:57.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.140+0000 7fb69dffb700 1 -- 172.21.15.32:0/1221124388 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6a4017030 con 0x7fb6a8133420 2023-09-30T22:30:57.141 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 msgr2=0x7fb6a812f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 0x7fb6a812f290 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400c700 tx=0x7fb6a400ca10 comp rx=0 tx=0).stop 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 shutdown_connections 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6a8138dc0 0x7fb6a812fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb6a8133420 0x7fb6a812f290 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 --2- 172.21.15.32:0/1221124388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6a81337f0 0x7fb6a812f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 >> 172.21.15.32:0/1221124388 conn(0x7fb6a80a4210 msgr2=0x7fb6a8099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 shutdown_connections 2023-09-30T22:30:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.142+0000 7fb6af4e2700 1 -- 172.21.15.32:0/1221124388 wait complete. 2023-09-30T22:30:57.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:57.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.159+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f34b809f420 con 0x7f34b809d6f0 2023-09-30T22:30:57.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.160+0000 7f349f7fe700 1 -- 172.21.15.32:0/985262764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34a801b3b0 con 0x7f34b809d6f0 2023-09-30T22:30:57.161 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 msgr2=0x7f34b81a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 0x7f34b81a4e60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e910 tx=0x7f34a800ecd0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 shutdown_connections 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34b809e440 0x7f34b81a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b809d6f0 0x7f34b81a4e60 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 --2- 172.21.15.32:0/985262764 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34b809e000 0x7f34b81a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 >> 172.21.15.32:0/985262764 conn(0x7f34b8066f50 msgr2=0x7f34b81b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 shutdown_connections 2023-09-30T22:30:57.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.162+0000 7f34bd51f700 1 -- 172.21.15.32:0/985262764 wait complete. 2023-09-30T22:30:57.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:57.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:57.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.722+0000 7faa903eb700 1 Processor -- start 2023-09-30T22:30:57.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.722+0000 7faa903eb700 1 -- start start 2023-09-30T22:30:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa903eb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa881316c0 0x7faa88131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa903eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8806ae80 0x7faa88139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa903eb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8809e130 con 0x7faa8809d9f0 2023-09-30T22:30:57.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa903eb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa881312b0 con 0x7faa881316c0 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa903eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8809f6e0 con 0x7faa8806ae80 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e988700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8806ae80 0x7faa88139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e187700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faa881316c0 0x7faa88131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8d986700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa8809d9f0 0x7faa8806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39654/0 (socket says 172.21.15.32:39654) 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8d986700 1 -- 172.21.15.32:0/236146866 learned_addr learned my addr 172.21.15.32:0/236146866 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8d185700 1 -- 172.21.15.32:0/236146866 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697430886 0 0) 0x7faa78001110 con 0x7faa8809d9f0 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e988700 1 -- 172.21.15.32:0/236146866 >> v2:172.21.15.32:3300/0 conn(0x7faa881316c0 msgr2=0x7faa88131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e988700 1 --2- 172.21.15.32:0/236146866 >> v2:172.21.15.32:3300/0 conn(0x7faa881316c0 0x7faa88131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e988700 1 -- 172.21.15.32:0/236146866 >> v1:172.21.15.32:6789/0 conn(0x7faa8809d9f0 legacy=0x7faa8806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e988700 1 -- 172.21.15.32:0/236146866 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8809e760 con 0x7faa8806ae80 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e187700 1 --2- 172.21.15.32:0/236146866 >> v2:172.21.15.32:3300/0 conn(0x7faa881316c0 0x7faa88131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8e988700 1 --2- 172.21.15.32:0/236146866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8806ae80 0x7faa88139550 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632e518c5f1fe472 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.723+0000 7faa8d185700 1 -- 172.21.15.32:0/236146866 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000e070 con 0x7faa8806ae80 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa8d185700 1 -- 172.21.15.32:0/236146866 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000bb20 con 0x7faa8806ae80 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa8d185700 1 -- 172.21.15.32:0/236146866 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80012430 con 0x7faa8806ae80 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 -- 172.21.15.32:0/236146866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8806ae80 msgr2=0x7faa88139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 --2- 172.21.15.32:0/236146866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8806ae80 0x7faa88139550 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 -- 172.21.15.32:0/236146866 shutdown_connections 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 --2- 172.21.15.32:0/236146866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8806ae80 0x7faa88139550 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 --2- 172.21.15.32:0/236146866 >> v2:172.21.15.32:3300/0 conn(0x7faa881316c0 0x7faa88131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 -- 172.21.15.32:0/236146866 >> 172.21.15.32:0/236146866 conn(0x7faa88066f50 msgr2=0x7faa8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 -- 172.21.15.32:0/236146866 shutdown_connections 2023-09-30T22:30:57.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.724+0000 7faa903eb700 1 -- 172.21.15.32:0/236146866 wait complete. 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 Processor -- start 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 -- start start 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa8806ae80 0x7faa88070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8809d9f0 0x7faa88070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 0x7faa880711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8811a640 con 0x7faa881316c0 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8811abe0 con 0x7faa8809d9f0 2023-09-30T22:30:57.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.726+0000 7faa903eb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa88119ad0 con 0x7faa8806ae80 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 0x7faa880711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e187700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa8806ae80 0x7faa88070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8d986700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8809d9f0 0x7faa88070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 0x7faa880711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49610/0 (socket says 172.21.15.32:49610) 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 -- 172.21.15.32:0/203343809 learned_addr learned my addr 172.21.15.32:0/203343809 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e187700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa8806ae80 0x7faa88070750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56084/0 (socket says 172.21.15.32:56084) 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 -- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa8806ae80 msgr2=0x7faa88070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa8806ae80 0x7faa88070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 -- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8809d9f0 msgr2=0x7faa88070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8809d9f0 0x7faa88070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 -- 172.21.15.32:0/203343809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840094a0 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa8e988700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 0x7faa880711d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7faa8809f820 tx=0x7faa80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80012430 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80020070 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa80009a70 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80004280 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.727+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa880a3540 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.728+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa800043e0 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.729+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa80022070 con 0x7faa881316c0 2023-09-30T22:30:57.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.729+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa88119fe0 con 0x7faa881316c0 2023-09-30T22:30:57.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.734+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa8001b400 con 0x7faa881316c0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.935+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa8811a840 con 0x7faa881316c0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.936+0000 7faa7effd700 1 -- 172.21.15.32:0/203343809 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa80034090 con 0x7faa881316c0 2023-09-30T22:30:57.937 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 msgr2=0x7faa880711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 0x7faa880711d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7faa8809f820 tx=0x7faa80004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 shutdown_connections 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa881316c0 0x7faa880711d0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa8806ae80 0x7faa88070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 --2- 172.21.15.32:0/203343809 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa8809d9f0 0x7faa88070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 >> 172.21.15.32:0/203343809 conn(0x7faa88066f50 msgr2=0x7faa880a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 shutdown_connections 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:57.938+0000 7faa903eb700 1 -- 172.21.15.32:0/203343809 wait complete. 2023-09-30T22:30:57.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:58.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:58.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:58.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.240+0000 7feebcdcf700 1 Processor -- start 2023-09-30T22:30:58.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.240+0000 7feebcdcf700 1 -- start start 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feebcdcf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feebcdcf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 0x7feeb81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feebcdcf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb809b120 con 0x7feeb809e410 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feebcdcf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb806b460 con 0x7feeb809d6f0 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feebcdcf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb809a7e0 con 0x7feeb809e000 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeaffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 0x7feeb81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeb659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeb6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feeb809e410 0x7feeb8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39662/0 (socket says 172.21.15.32:39662) 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeb659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41968/0 (socket says 172.21.15.32:41968) 2023-09-30T22:30:58.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeaffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 0x7feeb81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56088/0 (socket says 172.21.15.32:56088) 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeb6d9d700 1 -- 172.21.15.32:0/3149636575 learned_addr learned my addr 172.21.15.32:0/3149636575 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeaffff700 1 -- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 msgr2=0x7feeb809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeaffff700 1 --2- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeaffff700 1 -- 172.21.15.32:0/3149636575 >> v1:172.21.15.32:6789/0 conn(0x7feeb809e410 legacy=0x7feeb8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.242+0000 7feeaffff700 1 -- 172.21.15.32:0/3149636575 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb809a500 con 0x7feeb809e000 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.241+0000 7feeb5d9b700 1 -- 172.21.15.32:0/3149636575 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988279244 0 0) 0x7feeb809b120 con 0x7feeb809e410 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.242+0000 7feeaffff700 1 --2- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 0x7feeb81339f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feea00098a0 tx=0x7feea0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad360a42b8fe349 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.242+0000 7feeb5d9b700 1 -- 172.21.15.32:0/3149636575 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0011830 con 0x7feeb809e000 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.242+0000 7feeb5d9b700 1 -- 172.21.15.32:0/3149636575 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea0011990 con 0x7feeb809e000 2023-09-30T22:30:58.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.242+0000 7feeb5d9b700 1 -- 172.21.15.32:0/3149636575 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0011d20 con 0x7feeb809e000 2023-09-30T22:30:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 -- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 msgr2=0x7feeb81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 --2- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 0x7feeb81339f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feea00098a0 tx=0x7feea0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 -- 172.21.15.32:0/3149636575 shutdown_connections 2023-09-30T22:30:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 --2- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 --2- 172.21.15.32:0/3149636575 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e000 0x7feeb81339f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 -- 172.21.15.32:0/3149636575 >> 172.21.15.32:0/3149636575 conn(0x7feeb8066f50 msgr2=0x7feeb809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 -- 172.21.15.32:0/3149636575 shutdown_connections 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.243+0000 7feebcdcf700 1 -- 172.21.15.32:0/3149636575 wait complete. 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.244+0000 7feebcdcf700 1 Processor -- start 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.244+0000 7feebcdcf700 1 -- start start 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feebcdcf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb81a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feebcdcf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb809e000 0x7feeb81a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feebcdcf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e410 0x7feeb81c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feebcdcf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb81a4f70 con 0x7feeb809e000 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feebcdcf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb81b93f0 con 0x7feeb809d6f0 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feebcdcf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb809c710 con 0x7feeb809e410 2023-09-30T22:30:58.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb81a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb81a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41970/0 (socket says 172.21.15.32:41970) 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb659c700 1 -- 172.21.15.32:0/2986721827 learned_addr learned my addr 172.21.15.32:0/2986721827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e410 0x7feeb81c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeaffff700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb809e000 0x7feeb81a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 -- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 msgr2=0x7feeb81a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb81a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 -- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb809e000 msgr2=0x7feeb81a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb809e000 0x7feeb81a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 -- 172.21.15.32:0/2986721827 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00094a0 con 0x7feeb809e410 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeaffff700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb809e000 0x7feeb81a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.245+0000 7feeb6d9d700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e410 0x7feeb81c82c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feeb809e8b0 tx=0x7feeb000cec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.246+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb000aa40 con 0x7feeb809e410 2023-09-30T22:30:58.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.246+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb000abc0 con 0x7feeb809e410 2023-09-30T22:30:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.246+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb00070d0 con 0x7feeb809e410 2023-09-30T22:30:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.246+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb80a2e80 con 0x7feeb809e410 2023-09-30T22:30:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.246+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb81a8d10 con 0x7feeb809e410 2023-09-30T22:30:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.247+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feeb0007230 con 0x7feeb809e410 2023-09-30T22:30:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.247+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb81a76e0 con 0x7feeb809e410 2023-09-30T22:30:58.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.247+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7feeb003efd0 con 0x7feeb809e410 2023-09-30T22:30:58.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.252+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feeb0018070 con 0x7feeb809e410 2023-09-30T22:30:58.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.305+0000 7eff1b5d6700 1 Processor -- start 2023-09-30T22:30:58.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.305+0000 7eff1b5d6700 1 -- start start 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff1b5d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 0x7eff14138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff1b5d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff1b5d6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1409b120 con 0x7eff1409d6f0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff1b5d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1406b460 con 0x7eff14139170 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff1b5d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff1409a7e0 con 0x7eff1409e1d0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 0x7eff14138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 0x7eff14138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56112/0 (socket says 172.21.15.32:56112) 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 -- 172.21.15.32:0/4172050036 learned_addr learned my addr 172.21.15.32:0/4172050036 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff19372700 1 --1- 172.21.15.32:0/4172050036 >> v1:172.21.15.32:6789/0 conn(0x7eff1409d6f0 0x7eff1409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39678/0 (socket says 172.21.15.32:39678) 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff19b73700 1 --2- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff0bfff700 1 -- 172.21.15.32:0/4172050036 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529975614 0 0) 0x7eff1409b120 con 0x7eff1409d6f0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 -- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 msgr2=0x7eff1413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 --2- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 -- 172.21.15.32:0/4172050036 >> v1:172.21.15.32:6789/0 conn(0x7eff1409d6f0 legacy=0x7eff1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.306+0000 7eff18b71700 1 -- 172.21.15.32:0/4172050036 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1409a500 con 0x7eff1409e1d0 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.307+0000 7eff18b71700 1 --2- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 0x7eff14138c30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7eff04009950 tx=0x7eff04009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0bb12063072e0a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.307+0000 7eff0bfff700 1 -- 172.21.15.32:0/4172050036 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04011830 con 0x7eff1409e1d0 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.307+0000 7eff0bfff700 1 -- 172.21.15.32:0/4172050036 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff04011990 con 0x7eff1409e1d0 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.307+0000 7eff0bfff700 1 -- 172.21.15.32:0/4172050036 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04011d40 con 0x7eff1409e1d0 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 -- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 msgr2=0x7eff14138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 0x7eff14138c30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7eff04009950 tx=0x7eff04009c60 comp rx=0 tx=0).stop 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 -- 172.21.15.32:0/4172050036 shutdown_connections 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/4172050036 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409e1d0 0x7eff14138c30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 -- 172.21.15.32:0/4172050036 >> 172.21.15.32:0/4172050036 conn(0x7eff14066f50 msgr2=0x7eff1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 -- 172.21.15.32:0/4172050036 shutdown_connections 2023-09-30T22:30:58.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.308+0000 7eff1b5d6700 1 -- 172.21.15.32:0/4172050036 wait complete. 2023-09-30T22:30:58.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.309+0000 7eff1b5d6700 1 Processor -- start 2023-09-30T22:30:58.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.309+0000 7eff1b5d6700 1 -- start start 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff1b5d6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 0x7eff1411e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff1b5d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1409e1d0 0x7eff1411ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff1b5d6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1411f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff1b5d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff14134590 con 0x7eff1409e1d0 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff1b5d6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff141221c0 con 0x7eff14139170 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff1b5d6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff14120ad0 con 0x7eff1409d6f0 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff19372700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 0x7eff1411e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff19372700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 0x7eff1411e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56116/0 (socket says 172.21.15.32:56116) 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff19372700 1 -- 172.21.15.32:0/2501005710 learned_addr learned my addr 172.21.15.32:0/2501005710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff18b71700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1409e1d0 0x7eff1411ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.310+0000 7eff19b73700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1411f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff18b71700 1 -- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 msgr2=0x7eff1411e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff18b71700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 0x7eff1411e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff18b71700 1 -- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 msgr2=0x7eff1411f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff18b71700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1411f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff18b71700 1 -- 172.21.15.32:0/2501005710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04009550 con 0x7eff1409e1d0 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff19372700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 0x7eff1411e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff19b73700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1411f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff18b71700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1409e1d0 0x7eff1411ebe0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7eff04016000 tx=0x7eff04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0401a840 con 0x7eff1409e1d0 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0401f070 con 0x7eff1409e1d0 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff140a2120 con 0x7eff1409e1d0 2023-09-30T22:30:58.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0401ac00 con 0x7eff1409e1d0 2023-09-30T22:30:58.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.311+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff14121ee0 con 0x7eff1409e1d0 2023-09-30T22:30:58.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.312+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff04026480 con 0x7eff1409e1d0 2023-09-30T22:30:58.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.312+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7eff04021070 con 0x7eff1409e1d0 2023-09-30T22:30:58.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.313+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff14074ea0 con 0x7eff1409e1d0 2023-09-30T22:30:58.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.318+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff04015020 con 0x7eff1409e1d0 2023-09-30T22:30:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.347+0000 7fa38641c700 1 Processor -- start 2023-09-30T22:30:58.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 -- start start 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3801338a0 0x7fa380133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 0x7fa38009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa38009e760 0x7fa38013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38006fef0 con 0x7fa38009de50 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38006ef30 con 0x7fa38009e760 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa38641c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801334b0 con 0x7fa3801338a0 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa37f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 0x7fa38009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa37f7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 0x7fa38009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49652/0 (socket says 172.21.15.32:49652) 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa3849b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa38009e760 0x7fa38013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3801338a0 0x7fa380133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.348+0000 7fa37f7fe700 1 -- 172.21.15.32:0/3866557925 learned_addr learned my addr 172.21.15.32:0/3866557925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa3849b9700 1 -- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3801338a0 msgr2=0x7fa380133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa3849b9700 1 --2- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3801338a0 0x7fa380133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa3849b9700 1 -- 172.21.15.32:0/3866557925 >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 msgr2=0x7fa38009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa3849b9700 1 --2- 172.21.15.32:0/3866557925 >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 0x7fa38009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa3849b9700 1 -- 172.21.15.32:0/3866557925 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38006fc30 con 0x7fa38009e760 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa37ffff700 1 --2- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3801338a0 0x7fa380133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa37f7fe700 1 --2- 172.21.15.32:0/3866557925 >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 0x7fa38009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa3849b9700 1 --2- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa38009e760 0x7fa38013a730 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa378009d90 tx=0x7fa37800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6fcb39f14f90170 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa37effd700 1 -- 172.21.15.32:0/3866557925 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800e070 con 0x7fa38009e760 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa37effd700 1 -- 172.21.15.32:0/3866557925 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37800bb20 con 0x7fa38009e760 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.349+0000 7fa37effd700 1 -- 172.21.15.32:0/3866557925 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378012430 con 0x7fa38009e760 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 -- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa38009e760 msgr2=0x7fa38013a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 --2- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa38009e760 0x7fa38013a730 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa378009d90 tx=0x7fa37800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 -- 172.21.15.32:0/3866557925 shutdown_connections 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 --2- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa38009e760 0x7fa38013a730 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 --2- 172.21.15.32:0/3866557925 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa3801338a0 0x7fa380133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 --2- 172.21.15.32:0/3866557925 >> v2:172.21.15.32:3300/0 conn(0x7fa38009de50 0x7fa38009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 -- 172.21.15.32:0/3866557925 >> 172.21.15.32:0/3866557925 conn(0x7fa3800a2aa0 msgr2=0x7fa3800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 -- 172.21.15.32:0/3866557925 shutdown_connections 2023-09-30T22:30:58.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.350+0000 7fa38641c700 1 -- 172.21.15.32:0/3866557925 wait complete. 2023-09-30T22:30:58.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.352+0000 7fa38641c700 1 Processor -- start 2023-09-30T22:30:58.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.352+0000 7fa38641c700 1 -- start start 2023-09-30T22:30:58.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa38641c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 0x7fa38011b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa38641c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa38009e760 0x7fa38011bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa38641c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3801338a0 0x7fa38011c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa38641c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38011ca80 con 0x7fa38009de50 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa38641c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38011d1e0 con 0x7fa3801338a0 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa38641c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa38011e430 con 0x7fa38009e760 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 0x7fa38011b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa38009e760 0x7fa38011bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 0x7fa38011b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49658/0 (socket says 172.21.15.32:49658) 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 -- 172.21.15.32:0/854865320 learned_addr learned my addr 172.21.15.32:0/854865320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:58.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa38009e760 0x7fa38011bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56134/0 (socket says 172.21.15.32:56134) 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa3849b9700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3801338a0 0x7fa38011c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 -- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa38009e760 msgr2=0x7fa38011bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa38009e760 0x7fa38011bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 -- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3801338a0 msgr2=0x7fa38011c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3801338a0 0x7fa38011c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 -- 172.21.15.32:0/854865320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374009550 con 0x7fa38009de50 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.353+0000 7fa37ffff700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 0x7fa38011b5d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa37000ec10 tx=0x7fa37000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.354+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000c850 con 0x7fa38009de50 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.354+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3700094a0 con 0x7fa38009de50 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.354+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37000c9b0 con 0x7fa38009de50 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.354+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000cd20 con 0x7fa38009de50 2023-09-30T22:30:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.354+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa378009a70 con 0x7fa38009de50 2023-09-30T22:30:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.355+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa370013470 con 0x7fa38009de50 2023-09-30T22:30:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.355+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa37003f710 con 0x7fa38009de50 2023-09-30T22:30:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.355+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3800a0ea0 con 0x7fa38009de50 2023-09-30T22:30:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.364+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa370021080 con 0x7fa38009de50 2023-09-30T22:30:58.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.483+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feeb8074ea0 con 0x7feeb809e410 2023-09-30T22:30:58.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.483+0000 7feeaeffd700 1 -- 172.21.15.32:0/2986721827 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feeb0018020 con 0x7feeb809e410 2023-09-30T22:30:58.484 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e410 msgr2=0x7feeb81c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e410 0x7feeb81c82c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feeb809e8b0 tx=0x7feeb000cec0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 shutdown_connections 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7feeb809e410 0x7feeb81c82c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feeb809d6f0 0x7feeb81a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 --2- 172.21.15.32:0/2986721827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feeb809e000 0x7feeb81a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 >> 172.21.15.32:0/2986721827 conn(0x7feeb8066f50 msgr2=0x7feeb806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 shutdown_connections 2023-09-30T22:30:58.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.485+0000 7feebcdcf700 1 -- 172.21.15.32:0/2986721827 wait complete. 2023-09-30T22:30:58.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.543+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7eff1406c360 con 0x7eff1409e1d0 2023-09-30T22:30:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.544+0000 7eff09ffb700 1 -- 172.21.15.32:0/2501005710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff04010560 con 0x7eff1409e1d0 2023-09-30T22:30:58.545 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1409e1d0 msgr2=0x7eff1411ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1409e1d0 0x7eff1411ebe0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7eff04016000 tx=0x7eff04004750 comp rx=0 tx=0).stop 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 shutdown_connections 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7eff14139170 0x7eff1411f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7eff1409d6f0 0x7eff1411e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 --2- 172.21.15.32:0/2501005710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff1409e1d0 0x7eff1411ebe0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 >> 172.21.15.32:0/2501005710 conn(0x7eff14066f50 msgr2=0x7eff14136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 shutdown_connections 2023-09-30T22:30:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.546+0000 7eff1b5d6700 1 -- 172.21.15.32:0/2501005710 wait complete. 2023-09-30T22:30:58.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.572+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa38006f640 con 0x7fa38009de50 2023-09-30T22:30:58.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.572+0000 7fa37cff9700 1 -- 172.21.15.32:0/854865320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa370013720 con 0x7fa38009de50 2023-09-30T22:30:58.573 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 msgr2=0x7fa38011b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 0x7fa38011b5d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa37000ec10 tx=0x7fa37000ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 shutdown_connections 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3801338a0 0x7fa38011c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa38009de50 0x7fa38011b5d0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 --2- 172.21.15.32:0/854865320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa38009e760 0x7fa38011bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 >> 172.21.15.32:0/854865320 conn(0x7fa3800a2aa0 msgr2=0x7fa380130610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 shutdown_connections 2023-09-30T22:30:58.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:58.575+0000 7fa38641c700 1 -- 172.21.15.32:0/854865320 wait complete. 2023-09-30T22:30:58.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:30:58.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:59.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.097+0000 7f581d733700 1 Processor -- start 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.097+0000 7f581d733700 1 -- start start 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f581d733700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 0x7f5818133c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f581d733700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f581809e560 0x7f581813b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f581d733700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58181334b0 con 0x7f5818134180 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f581d733700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581809fa40 con 0x7f581809e560 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f581d733700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581806f250 con 0x7f5818133870 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f581809e560 0x7f581813b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 0x7f5818133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58167fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5818134180 0x7f581809de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39688/0 (socket says 172.21.15.32:39688) 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58177fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f581809e560 0x7f581813b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49662/0 (socket says 172.21.15.32:49662) 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58167fc700 1 -- 172.21.15.32:0/1819600498 learned_addr learned my addr 172.21.15.32:0/1819600498 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 0x7f5818133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42026/0 (socket says 172.21.15.32:42026) 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58177fe700 1 -- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 msgr2=0x7f5818133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58177fe700 1 --2- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 0x7f5818133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58177fe700 1 -- 172.21.15.32:0/1819600498 >> v1:172.21.15.32:6789/0 conn(0x7f5818134180 legacy=0x7f581809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f58177fe700 1 -- 172.21.15.32:0/1819600498 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581806fb20 con 0x7f581809e560 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f5815ffb700 1 -- 172.21.15.32:0/1819600498 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670406614 0 0) 0x7f58181334b0 con 0x7f5818134180 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.098+0000 7f5816ffd700 1 --2- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 0x7f5818133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f58177fe700 1 --2- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581809e560 0x7f581813b7f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5810009d90 tx=0x7f581000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4262aa3eada47c42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f5815ffb700 1 -- 172.21.15.32:0/1819600498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000e070 con 0x7f581809e560 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f5815ffb700 1 -- 172.21.15.32:0/1819600498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581000bb20 con 0x7f581809e560 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f5815ffb700 1 -- 172.21.15.32:0/1819600498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810012430 con 0x7f581809e560 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f581d733700 1 -- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581809e560 msgr2=0x7f581813b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f581d733700 1 --2- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581809e560 0x7f581813b7f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5810009d90 tx=0x7f581000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f581d733700 1 -- 172.21.15.32:0/1819600498 shutdown_connections 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f581d733700 1 --2- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f581809e560 0x7f581813b7f0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f581d733700 1 --2- 172.21.15.32:0/1819600498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5818133870 0x7f5818133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.099+0000 7f581d733700 1 -- 172.21.15.32:0/1819600498 >> 172.21.15.32:0/1819600498 conn(0x7f58180a2610 msgr2=0x7f58180a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.100+0000 7f581d733700 1 -- 172.21.15.32:0/1819600498 shutdown_connections 2023-09-30T22:30:59.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.100+0000 7f581d733700 1 -- 172.21.15.32:0/1819600498 wait complete. 2023-09-30T22:30:59.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 Processor -- start 2023-09-30T22:30:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 -- start start 2023-09-30T22:30:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 0x7f581811a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5818133870 0x7f581811aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5818134180 0x7f581811b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581811e080 con 0x7f5818133870 2023-09-30T22:30:59.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581811e280 con 0x7f581809e560 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f581d733700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f581811d170 con 0x7f5818134180 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 0x7f581811a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f58167fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5818133870 0x7f581811aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 0x7f581811a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42032/0 (socket says 172.21.15.32:42032) 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f5816ffd700 1 -- 172.21.15.32:0/51326974 learned_addr learned my addr 172.21.15.32:0/51326974 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.101+0000 7f58177fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5818134180 0x7f581811b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58167fc700 1 -- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5818134180 msgr2=0x7f581811b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58167fc700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5818134180 0x7f581811b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58167fc700 1 -- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 msgr2=0x7f581811a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58167fc700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 0x7f581811a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58167fc700 1 -- 172.21.15.32:0/51326974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080094a0 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f5816ffd700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 0x7f581811a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58177fe700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5818134180 0x7f581811b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f58167fc700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5818133870 0x7f581811aca0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f581809fde0 tx=0x7f580c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c004a80 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c004be0 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5810009a70 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0070d0 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.102+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581806f490 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.103+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58180747f0 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.103+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f580c007230 con 0x7f5818133870 2023-09-30T22:30:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.104+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f580c03f780 con 0x7f5818133870 2023-09-30T22:30:59.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.108+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f580c0074e0 con 0x7f5818133870 2023-09-30T22:30:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.315+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f581812e3e0 con 0x7f5818133870 2023-09-30T22:30:59.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.315+0000 7f57ff7fe700 1 -- 172.21.15.32:0/51326974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f580c01b340 con 0x7f5818133870 2023-09-30T22:30:59.316 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.317+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5818133870 msgr2=0x7f581811aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.317+0000 7f581d733700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5818133870 0x7f581811aca0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f581809fde0 tx=0x7f580c00c730 comp rx=0 tx=0).stop 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 shutdown_connections 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5818134180 0x7f581811b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f581809e560 0x7f581811a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 --2- 172.21.15.32:0/51326974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5818133870 0x7f581811aca0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 >> 172.21.15.32:0/51326974 conn(0x7f58180a2610 msgr2=0x7f58181306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 shutdown_connections 2023-09-30T22:30:59.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.318+0000 7f581d733700 1 -- 172.21.15.32:0/51326974 wait complete. 2023-09-30T22:30:59.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:30:59.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:59.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:59.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.649+0000 7f475ba75700 1 Processor -- start 2023-09-30T22:30:59.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.649+0000 7f475ba75700 1 -- start start 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f475ba75700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47541337f0 0x7f4754138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f475ba75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4754138dc0 0x7f475413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f475ba75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475406b990 con 0x7f4754133420 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f475ba75700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475409b0e0 con 0x7f47541337f0 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f475ba75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4754133c20 con 0x7f4754138dc0 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759010700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47541337f0 0x7f4754138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759811700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4754133420 0x7f4754133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39690/0 (socket says 172.21.15.32:39690) 2023-09-30T22:30:59.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f475a012700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4754138dc0 0x7f475413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759811700 1 -- 172.21.15.32:0/2546929894 learned_addr learned my addr 172.21.15.32:0/2546929894 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:59.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759010700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47541337f0 0x7f4754138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49688/0 (socket says 172.21.15.32:49688) 2023-09-30T22:30:59.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759010700 1 -- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4754138dc0 msgr2=0x7f475413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759010700 1 --2- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4754138dc0 0x7f475413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759010700 1 -- 172.21.15.32:0/2546929894 >> v1:172.21.15.32:6789/0 conn(0x7f4754133420 legacy=0x7f4754133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.650+0000 7f4759010700 1 -- 172.21.15.32:0/2546929894 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540a0590 con 0x7f47541337f0 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.651+0000 7f4759010700 1 --2- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47541337f0 0x7f4754138880 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f4744009840 tx=0x7f4744009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34eb0fd4150cc8dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.651+0000 7f475880f700 1 -- 172.21.15.32:0/2546929894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744011830 con 0x7f47541337f0 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.651+0000 7f475880f700 1 -- 172.21.15.32:0/2546929894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744011990 con 0x7f47541337f0 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.651+0000 7f475880f700 1 -- 172.21.15.32:0/2546929894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744011d20 con 0x7f47541337f0 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 -- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47541337f0 msgr2=0x7f4754138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 --2- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47541337f0 0x7f4754138880 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f4744009840 tx=0x7f4744009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 -- 172.21.15.32:0/2546929894 shutdown_connections 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 --2- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4754138dc0 0x7f475413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 --2- 172.21.15.32:0/2546929894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47541337f0 0x7f4754138880 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 -- 172.21.15.32:0/2546929894 >> 172.21.15.32:0/2546929894 conn(0x7f47540a4210 msgr2=0x7f47540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 -- 172.21.15.32:0/2546929894 shutdown_connections 2023-09-30T22:30:59.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.652+0000 7f475ba75700 1 -- 172.21.15.32:0/2546929894 wait complete. 2023-09-30T22:30:59.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.653+0000 7f475ba75700 1 Processor -- start 2023-09-30T22:30:59.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.653+0000 7f475ba75700 1 -- start start 2023-09-30T22:30:59.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475ba75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4754133420 0x7f475412ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475ba75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 0x7f475412f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475ba75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754138dc0 0x7f475412f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475ba75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475409d9b0 con 0x7f4754138dc0 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475ba75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47541388a0 con 0x7f4754133420 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475ba75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f475409c590 con 0x7f47541337f0 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f4759010700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 0x7f475412f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f4759010700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 0x7f475412f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56158/0 (socket says 172.21.15.32:56158) 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f4759010700 1 -- 172.21.15.32:0/1830457683 learned_addr learned my addr 172.21.15.32:0/1830457683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.654+0000 7f475a012700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754138dc0 0x7f475412f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759811700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4754133420 0x7f475412ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759010700 1 -- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4754133420 msgr2=0x7f475412ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759010700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4754133420 0x7f475412ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759010700 1 -- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754138dc0 msgr2=0x7f475412f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759010700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754138dc0 0x7f475412f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759010700 1 -- 172.21.15.32:0/1830457683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47440094a0 con 0x7f47541337f0 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f475a012700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754138dc0 0x7f475412f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759811700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4754133420 0x7f475412ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f4759010700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 0x7f475412f390 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4744009fd0 tx=0x7f4744004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744004480 con 0x7f47541337f0 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474401f070 con 0x7f47541337f0 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47440038e0 con 0x7f47541337f0 2023-09-30T22:30:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475409b6b0 con 0x7f47541337f0 2023-09-30T22:30:59.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.655+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475409d2d0 con 0x7f47541337f0 2023-09-30T22:30:59.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.656+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4744021030 con 0x7f47541337f0 2023-09-30T22:30:59.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.656+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4744040c50 con 0x7f47541337f0 2023-09-30T22:30:59.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.656+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475409ffb0 con 0x7f47541337f0 2023-09-30T22:30:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.661+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f474401a7d0 con 0x7f47541337f0 2023-09-30T22:30:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.707+0000 7f3cafc95700 1 Processor -- start 2023-09-30T22:30:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.707+0000 7f3cafc95700 1 -- start start 2023-09-30T22:30:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cafc95700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ca81337f0 0x7f3ca8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cafc95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8138dc0 0x7f3ca813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cafc95700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca806b990 con 0x7f3ca8133420 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cafc95700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca809b0e0 con 0x7f3ca81337f0 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cafc95700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8133c20 con 0x7f3ca8138dc0 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cad230700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ca81337f0 0x7f3ca8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cad230700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3ca81337f0 0x7f3ca8138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49706/0 (socket says 172.21.15.32:49706) 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cad230700 1 -- 172.21.15.32:0/270589009 learned_addr learned my addr 172.21.15.32:0/270589009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cada31700 1 --1- 172.21.15.32:0/270589009 >> v1:172.21.15.32:6789/0 conn(0x7f3ca8133420 0x7f3ca8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39694/0 (socket says 172.21.15.32:39694) 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.708+0000 7f3cae232700 1 --2- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8138dc0 0x7f3ca813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3cad230700 1 -- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8138dc0 msgr2=0x7f3ca813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3cad230700 1 --2- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8138dc0 0x7f3ca813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3cad230700 1 -- 172.21.15.32:0/270589009 >> v1:172.21.15.32:6789/0 conn(0x7f3ca8133420 legacy=0x7f3ca8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3cad230700 1 -- 172.21.15.32:0/270589009 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca80a0590 con 0x7f3ca81337f0 2023-09-30T22:30:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3cad230700 1 --2- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca8138880 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098f0 tx=0x7f3c98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b657845cfe0aa21c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3caca2f700 1 -- 172.21.15.32:0/270589009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98011830 con 0x7f3ca81337f0 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3caca2f700 1 -- 172.21.15.32:0/270589009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98011990 con 0x7f3ca81337f0 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.709+0000 7f3caca2f700 1 -- 172.21.15.32:0/270589009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98011d20 con 0x7f3ca81337f0 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 -- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 msgr2=0x7f3ca8138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 --2- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca8138880 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098f0 tx=0x7f3c98009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 -- 172.21.15.32:0/270589009 shutdown_connections 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 --2- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8138dc0 0x7f3ca813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 --2- 172.21.15.32:0/270589009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca8138880 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 -- 172.21.15.32:0/270589009 >> 172.21.15.32:0/270589009 conn(0x7f3ca80a4210 msgr2=0x7f3ca80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 -- 172.21.15.32:0/270589009 shutdown_connections 2023-09-30T22:30:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.710+0000 7f3cafc95700 1 -- 172.21.15.32:0/270589009 wait complete. 2023-09-30T22:30:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.711+0000 7f3cafc95700 1 Processor -- start 2023-09-30T22:30:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 -- start start 2023-09-30T22:30:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8133420 0x7f3ca812ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca812f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ca8138dc0 0x7f3ca812f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca809d9b0 con 0x7f3ca81337f0 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81388a0 con 0x7f3ca8133420 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cafc95700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca809c590 con 0x7f3ca8138dc0 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cad230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca812f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cad230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca812f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49710/0 (socket says 172.21.15.32:49710) 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cad230700 1 -- 172.21.15.32:0/1609335501 learned_addr learned my addr 172.21.15.32:0/1609335501 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cada31700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8133420 0x7f3ca812ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.712+0000 7f3cae232700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ca8138dc0 0x7f3ca812f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cad230700 1 -- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ca8138dc0 msgr2=0x7f3ca812f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cad230700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ca8138dc0 0x7f3ca812f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cad230700 1 -- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8133420 msgr2=0x7f3ca812ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cad230700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8133420 0x7f3ca812ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cad230700 1 -- 172.21.15.32:0/1609335501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98009550 con 0x7f3ca81337f0 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cad230700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca812f390 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c980036a0 con 0x7f3ca81337f0 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98003800 con 0x7f3ca81337f0 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca809b6b0 con 0x7f3ca81337f0 2023-09-30T22:30:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98003bb0 con 0x7f3ca81337f0 2023-09-30T22:30:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.713+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca809d2d0 con 0x7f3ca81337f0 2023-09-30T22:30:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.714+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca809ffb0 con 0x7f3ca81337f0 2023-09-30T22:30:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.715+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c98021070 con 0x7f3ca81337f0 2023-09-30T22:30:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.715+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3c98040de0 con 0x7f3ca81337f0 2023-09-30T22:30:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.724+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c9802d3a0 con 0x7f3ca81337f0 2023-09-30T22:30:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.751+0000 7f69fb7ff700 1 Processor -- start 2023-09-30T22:30:59.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.752+0000 7f69fb7ff700 1 -- start start 2023-09-30T22:30:59.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.752+0000 7f69fb7ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69fb7ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69fb7ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40a2a70 con 0x7f69f4133930 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69fb7ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40a01d0 con 0x7f69f40a0a30 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69fb7ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69f409e2b0 con 0x7f69f409d630 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f959b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f9d9c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f69f4133930 0x7f69f4133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39710/0 (socket says 172.21.15.32:39710) 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f9d9c700 1 -- 172.21.15.32:0/4135563972 learned_addr learned my addr 172.21.15.32:0/4135563972 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:30:59.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f8d9a700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f959b700 1 -- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 msgr2=0x7f69f40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f959b700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f959b700 1 -- 172.21.15.32:0/4135563972 >> v1:172.21.15.32:6789/0 conn(0x7f69f4133930 legacy=0x7f69f4133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69ebfff700 1 -- 172.21.15.32:0/4135563972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343436027 0 0) 0x7f69f40a2a70 con 0x7f69f4133930 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f959b700 1 -- 172.21.15.32:0/4135563972 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f409dc50 con 0x7f69f40a0a30 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f8d9a700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.753+0000 7f69f959b700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f40a0e00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009870 tx=0x7f69f0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b14d5a87d5f741 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69ebfff700 1 -- 172.21.15.32:0/4135563972 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0011830 con 0x7f69f40a0a30 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69ebfff700 1 -- 172.21.15.32:0/4135563972 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0011990 con 0x7f69f40a0a30 2023-09-30T22:30:59.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69ebfff700 1 -- 172.21.15.32:0/4135563972 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0011d20 con 0x7f69f40a0a30 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 -- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 msgr2=0x7f69f40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f40a0e00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009870 tx=0x7f69f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 -- 172.21.15.32:0/4135563972 shutdown_connections 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f40a0e00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/4135563972 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 -- 172.21.15.32:0/4135563972 >> 172.21.15.32:0/4135563972 conn(0x7f69f40a3050 msgr2=0x7f69f40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 -- 172.21.15.32:0/4135563972 shutdown_connections 2023-09-30T22:30:59.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.754+0000 7f69fb7ff700 1 -- 172.21.15.32:0/4135563972 wait complete. 2023-09-30T22:30:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.756+0000 7f69fb7ff700 1 Processor -- start 2023-09-30T22:30:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.756+0000 7f69fb7ff700 1 -- start start 2023-09-30T22:30:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69fb7ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f411f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69fb7ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f411f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69fb7ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69f4133930 0x7f69f41c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69fb7ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f4121ed0 con 0x7f69f4133930 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69fb7ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f4133250 con 0x7f69f40a0a30 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69fb7ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69f411ee20 con 0x7f69f409d630 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f411f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f411f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:56192/0 (socket says 172.21.15.32:56192) 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 -- 172.21.15.32:0/1023037160 learned_addr learned my addr 172.21.15.32:0/1023037160 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f9d9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69f4133930 0x7f69f41c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f8d9a700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f411f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 -- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 msgr2=0x7f69f411f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f411f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 -- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69f4133930 msgr2=0x7f69f41c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69f4133930 0x7f69f41c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.757+0000 7f69f959b700 1 -- 172.21.15.32:0/1023037160 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40094a0 con 0x7f69f409d630 2023-09-30T22:30:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.758+0000 7f69f959b700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f411f2c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f69f00090b0 tx=0x7f69f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.758+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0004300 con 0x7f69f409d630 2023-09-30T22:30:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.758+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f001f070 con 0x7f69f409d630 2023-09-30T22:30:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.758+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f00036a0 con 0x7f69f409d630 2023-09-30T22:30:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.758+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f00094a0 con 0x7f69f409d630 2023-09-30T22:30:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.758+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f40a4250 con 0x7f69f409d630 2023-09-30T22:30:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.759+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f69f0021030 con 0x7f69f409d630 2023-09-30T22:30:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.759+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f69f00406a0 con 0x7f69f409d630 2023-09-30T22:30:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.759+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f411fe00 con 0x7f69f409d630 2023-09-30T22:30:59.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.764+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69f0015020 con 0x7f69f409d630 2023-09-30T22:30:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.870+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4754074ea0 con 0x7f47541337f0 2023-09-30T22:30:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.871+0000 7f474a7fc700 1 -- 172.21.15.32:0/1830457683 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4744015020 con 0x7f47541337f0 2023-09-30T22:30:59.872 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:30:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 msgr2=0x7f475412f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 0x7f475412f390 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4744009fd0 tx=0x7f4744004750 comp rx=0 tx=0).stop 2023-09-30T22:30:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 shutdown_connections 2023-09-30T22:30:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4754138dc0 0x7f475412f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4754133420 0x7f475412ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 --2- 172.21.15.32:0/1830457683 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f47541337f0 0x7f475412f390 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 >> 172.21.15.32:0/1830457683 conn(0x7f47540a4210 msgr2=0x7f4754131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 shutdown_connections 2023-09-30T22:30:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.873+0000 7f475ba75700 1 -- 172.21.15.32:0/1830457683 wait complete. 2023-09-30T22:30:59.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:30:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.935+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ca8074ea0 con 0x7f3ca81337f0 2023-09-30T22:30:59.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.936+0000 7f3c9e7fc700 1 -- 172.21.15.32:0/1609335501 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c98015020 con 0x7f3ca81337f0 2023-09-30T22:30:59.937 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:30:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 msgr2=0x7f3ca812f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca812f390 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3c98000c00 tx=0x7f3c98004750 comp rx=0 tx=0).stop 2023-09-30T22:30:59.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 shutdown_connections 2023-09-30T22:30:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3ca8138dc0 0x7f3ca812f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3ca8133420 0x7f3ca812ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 --2- 172.21.15.32:0/1609335501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ca81337f0 0x7f3ca812f390 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 >> 172.21.15.32:0/1609335501 conn(0x7f3ca80a4210 msgr2=0x7f3ca8131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 shutdown_connections 2023-09-30T22:30:59.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.938+0000 7f3cafc95700 1 -- 172.21.15.32:0/1609335501 wait complete. 2023-09-30T22:30:59.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:30:59.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.971+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f69f4120a40 con 0x7f69f409d630 2023-09-30T22:30:59.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.971+0000 7f69e9ffb700 1 -- 172.21.15.32:0/1023037160 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f69f00199a0 con 0x7f69f409d630 2023-09-30T22:30:59.972 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:30:59.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 msgr2=0x7f69f411f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f411f2c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f69f00090b0 tx=0x7f69f0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:59.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 shutdown_connections 2023-09-30T22:30:59.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69f4133930 0x7f69f41c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f69f409d630 0x7f69f411f2c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 --2- 172.21.15.32:0/1023037160 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69f40a0a30 0x7f69f411f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 >> 172.21.15.32:0/1023037160 conn(0x7f69f40a3050 msgr2=0x7f69f41bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 shutdown_connections 2023-09-30T22:30:59.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:30:59.974+0000 7f69fb7ff700 1 -- 172.21.15.32:0/1023037160 wait complete. 2023-09-30T22:30:59.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:00.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.479+0000 7f399d16c700 1 Processor -- start 2023-09-30T22:31:00.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.479+0000 7f399d16c700 1 -- start start 2023-09-30T22:31:00.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399d16c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f399809d6f0 0x7f399809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399d16c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 0x7f3998130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399d16c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399809f130 con 0x7f399809e000 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399d16c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399809f520 con 0x7f399809d6f0 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399d16c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399809e930 con 0x7f399809e410 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 0x7f3998130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3996d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f399809d6f0 0x7f399809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 0x7f3998130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54480/0 (socket says 172.21.15.32:54480) 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399759e700 1 -- 172.21.15.32:0/3314560866 learned_addr learned my addr 172.21.15.32:0/3314560866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399659c700 1 --1- 172.21.15.32:0/3314560866 >> v1:172.21.15.32:6789/0 conn(0x7f399809e000 0x7f3998130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57424/0 (socket says 172.21.15.32:57424) 2023-09-30T22:31:00.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3996d9d700 1 -- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 msgr2=0x7f3998130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3996d9d700 1 --2- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 0x7f3998130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3996d9d700 1 -- 172.21.15.32:0/3314560866 >> v1:172.21.15.32:6789/0 conn(0x7f399809e000 legacy=0x7f3998130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3995d9b700 1 -- 172.21.15.32:0/3314560866 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154745703 0 0) 0x7f399809f130 con 0x7f399809e000 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3996d9d700 1 -- 172.21.15.32:0/3314560866 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399809b2d0 con 0x7f399809d6f0 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f399759e700 1 --2- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 0x7f3998130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.480+0000 7f3996d9d700 1 --2- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f399809dac0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3988009840 tx=0x7f3988009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dc55b07d8d8053 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f3995d9b700 1 -- 172.21.15.32:0/3314560866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988011830 con 0x7f399809d6f0 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f3995d9b700 1 -- 172.21.15.32:0/3314560866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3988011990 con 0x7f399809d6f0 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f3995d9b700 1 -- 172.21.15.32:0/3314560866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988011d20 con 0x7f399809d6f0 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 -- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 msgr2=0x7f399809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 --2- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f399809dac0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f3988009840 tx=0x7f3988009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:00.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 -- 172.21.15.32:0/3314560866 shutdown_connections 2023-09-30T22:31:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 --2- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e410 0x7f3998130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 --2- 172.21.15.32:0/3314560866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f399809dac0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 -- 172.21.15.32:0/3314560866 >> 172.21.15.32:0/3314560866 conn(0x7f3998066f50 msgr2=0x7f399809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 -- 172.21.15.32:0/3314560866 shutdown_connections 2023-09-30T22:31:00.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.481+0000 7f399d16c700 1 -- 172.21.15.32:0/3314560866 wait complete. 2023-09-30T22:31:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.483+0000 7f399d16c700 1 Processor -- start 2023-09-30T22:31:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.483+0000 7f399d16c700 1 -- start start 2023-09-30T22:31:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399d16c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f3998122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399d16c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e000 0x7f3998122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399d16c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f399809e410 0x7f3998122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399d16c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3998125d50 con 0x7f399809d6f0 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399d16c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3998123670 con 0x7f399809e000 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399d16c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3998123e40 con 0x7f399809e410 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f3998122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e000 0x7f3998122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f3996d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f3998122230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43020/0 (socket says 172.21.15.32:43020) 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f3996d9d700 1 -- 172.21.15.32:0/3816765232 learned_addr learned my addr 172.21.15.32:0/3816765232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399759e700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f399809e410 0x7f3998122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 -- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f399809e410 msgr2=0x7f3998122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f399809e410 0x7f3998122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 -- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 msgr2=0x7f3998122230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f3998122230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 -- 172.21.15.32:0/3816765232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009b30 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f3996d9d700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f3998122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.484+0000 7f399659c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e000 0x7f3998122770 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c470 tx=0x7f398c00c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.485+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c004c80 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.485+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c004de0 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.485+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39880094a0 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.485+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c007310 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.485+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3998130150 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.485+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f398c007470 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.486+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f398c03f240 con 0x7f399809e000 2023-09-30T22:31:00.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.486+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39981246b0 con 0x7f399809e000 2023-09-30T22:31:00.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.491+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f398c007720 con 0x7f399809e000 2023-09-30T22:31:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.696+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3998074ea0 con 0x7f399809e000 2023-09-30T22:31:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.696+0000 7f397f7fe700 1 -- 172.21.15.32:0/3816765232 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f398c011360 con 0x7f399809e000 2023-09-30T22:31:00.697 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e000 msgr2=0x7f3998122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e000 0x7f3998122770 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c470 tx=0x7f398c00c780 comp rx=0 tx=0).stop 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 shutdown_connections 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f399809e410 0x7f3998122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f399809d6f0 0x7f3998122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 --2- 172.21.15.32:0/3816765232 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f399809e000 0x7f3998122770 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 >> 172.21.15.32:0/3816765232 conn(0x7f3998066f50 msgr2=0x7f3998130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 shutdown_connections 2023-09-30T22:31:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:00.698+0000 7f399d16c700 1 -- 172.21.15.32:0/3816765232 wait complete. 2023-09-30T22:31:00.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:00.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:00.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:00.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:01.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.036+0000 7fca27532700 1 Processor -- start 2023-09-30T22:31:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.036+0000 7fca27532700 1 -- start start 2023-09-30T22:31:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca27532700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca201338c0 0x7fca20133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca27532700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca2009e560 0x7fca2013b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca27532700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2006f720 con 0x7fca201341d0 2023-09-30T22:31:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca27532700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca20070c80 con 0x7fca201338c0 2023-09-30T22:31:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca27532700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca2006fce0 con 0x7fca2009e560 2023-09-30T22:31:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca252ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca201338c0 0x7fca20133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca252ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca201338c0 0x7fca20133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43024/0 (socket says 172.21.15.32:43024) 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca252ce700 1 -- 172.21.15.32:0/4104390866 learned_addr learned my addr 172.21.15.32:0/4104390866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.037+0000 7fca25acf700 1 --2- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca2009e560 0x7fca2013b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca252ce700 1 -- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca2009e560 msgr2=0x7fca2013b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca252ce700 1 --2- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca2009e560 0x7fca2013b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca252ce700 1 -- 172.21.15.32:0/4104390866 >> v1:172.21.15.32:6789/0 conn(0x7fca201341d0 legacy=0x7fca2009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca252ce700 1 -- 172.21.15.32:0/4104390866 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca201336a0 con 0x7fca201338c0 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca252ce700 1 --2- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca20133c90 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0098f0 tx=0x7fca1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcce120a86f7669 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca17fff700 1 -- 172.21.15.32:0/4104390866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c011830 con 0x7fca201338c0 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca17fff700 1 -- 172.21.15.32:0/4104390866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c011990 con 0x7fca201338c0 2023-09-30T22:31:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.038+0000 7fca17fff700 1 -- 172.21.15.32:0/4104390866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c011d20 con 0x7fca201338c0 2023-09-30T22:31:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 -- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 msgr2=0x7fca20133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 --2- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca20133c90 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0098f0 tx=0x7fca1c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 -- 172.21.15.32:0/4104390866 shutdown_connections 2023-09-30T22:31:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 --2- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca2009e560 0x7fca2013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 --2- 172.21.15.32:0/4104390866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca20133c90 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 -- 172.21.15.32:0/4104390866 >> 172.21.15.32:0/4104390866 conn(0x7fca200a2aa0 msgr2=0x7fca200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 -- 172.21.15.32:0/4104390866 shutdown_connections 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.039+0000 7fca27532700 1 -- 172.21.15.32:0/4104390866 wait complete. 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.040+0000 7fca27532700 1 Processor -- start 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.040+0000 7fca27532700 1 -- start start 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca27532700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca2009e560 0x7fca2011fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca27532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca201200c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca27532700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca201341d0 0x7fca20120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca27532700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20130780 con 0x7fca201338c0 2023-09-30T22:31:01.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca27532700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca20122e10 con 0x7fca2009e560 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca27532700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca20123190 con 0x7fca201341d0 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca201200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca252ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca2009e560 0x7fca2011fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca201200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43034/0 (socket says 172.21.15.32:43034) 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 -- 172.21.15.32:0/3116998390 learned_addr learned my addr 172.21.15.32:0/3116998390 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca25acf700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca201341d0 0x7fca20120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 -- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca201341d0 msgr2=0x7fca20120600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca201341d0 0x7fca20120600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 -- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca2009e560 msgr2=0x7fca2011fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca2009e560 0x7fca2011fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca24acd700 1 -- 172.21.15.32:0/3116998390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c009550 con 0x7fca201338c0 2023-09-30T22:31:01.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca25acf700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca201341d0 0x7fca20120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.041+0000 7fca252ce700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca2009e560 0x7fca2011fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.042+0000 7fca24acd700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca201200c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c510 tx=0x7fca1000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.042+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca100052d0 con 0x7fca201338c0 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.042+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca10005430 con 0x7fca201338c0 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.042+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20072660 con 0x7fca201338c0 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.042+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca100038c0 con 0x7fca201338c0 2023-09-30T22:31:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.042+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca20121790 con 0x7fca201338c0 2023-09-30T22:31:01.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.043+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20074ea0 con 0x7fca201338c0 2023-09-30T22:31:01.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.044+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca10003a20 con 0x7fca201338c0 2023-09-30T22:31:01.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.044+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca1003f070 con 0x7fca201338c0 2023-09-30T22:31:01.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.048+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca10003cd0 con 0x7fca201338c0 2023-09-30T22:31:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 Processor -- start 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 -- start start 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94bc0a0a30 0x7f94bc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc0a1340 0x7f94bc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc09edd0 con 0x7f94bc133fa0 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc09e1f0 con 0x7f94bc0a0a30 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.099+0000 7f94c10ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc09dc50 con 0x7f94bc0a1340 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94bb59e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f94bc133fa0 0x7f94bc13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57444/0 (socket says 172.21.15.32:57444) 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94bad9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94bc0a0a30 0x7f94bc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94bb59e700 1 -- 172.21.15.32:0/2278972032 learned_addr learned my addr 172.21.15.32:0/2278972032 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94ba59c700 1 --2- 172.21.15.32:0/2278972032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc0a1340 0x7f94bc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94b9d9b700 1 -- 172.21.15.32:0/2278972032 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242212288 0 0) 0x7f94bc09edd0 con 0x7f94bc133fa0 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94b9d9b700 1 -- 172.21.15.32:0/2278972032 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a80035b0 con 0x7f94bc133fa0 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94ba59c700 1 -- 172.21.15.32:0/2278972032 >> v2:172.21.15.32:3300/0 conn(0x7f94bc0a0a30 msgr2=0x7f94bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94ba59c700 1 --2- 172.21.15.32:0/2278972032 >> v2:172.21.15.32:3300/0 conn(0x7f94bc0a0a30 0x7f94bc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94ba59c700 1 -- 172.21.15.32:0/2278972032 >> v1:172.21.15.32:6789/0 conn(0x7f94bc133fa0 legacy=0x7f94bc13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94ba59c700 1 -- 172.21.15.32:0/2278972032 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0a1fe0 con 0x7f94bc0a1340 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94bad9d700 1 --2- 172.21.15.32:0/2278972032 >> v2:172.21.15.32:3300/0 conn(0x7f94bc0a0a30 0x7f94bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.100+0000 7f94ba59c700 1 --2- 172.21.15.32:0/2278972032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc0a1340 0x7f94bc133a60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009840 tx=0x7f94a4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a21a9dfe7beb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94b9d9b700 1 -- 172.21.15.32:0/2278972032 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4011830 con 0x7f94bc0a1340 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94b9d9b700 1 -- 172.21.15.32:0/2278972032 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a4011990 con 0x7f94bc0a1340 2023-09-30T22:31:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94b9d9b700 1 -- 172.21.15.32:0/2278972032 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4011d20 con 0x7f94bc0a1340 2023-09-30T22:31:01.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 -- 172.21.15.32:0/2278972032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc0a1340 msgr2=0x7f94bc133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 --2- 172.21.15.32:0/2278972032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc0a1340 0x7f94bc133a60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009840 tx=0x7f94a4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 -- 172.21.15.32:0/2278972032 shutdown_connections 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 --2- 172.21.15.32:0/2278972032 >> v2:172.21.15.32:3300/0 conn(0x7f94bc0a0a30 0x7f94bc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 --2- 172.21.15.32:0/2278972032 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc0a1340 0x7f94bc133a60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 -- 172.21.15.32:0/2278972032 >> 172.21.15.32:0/2278972032 conn(0x7f94bc0a6550 msgr2=0x7f94bc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 -- 172.21.15.32:0/2278972032 shutdown_connections 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.101+0000 7f94c10ac700 1 -- 172.21.15.32:0/2278972032 wait complete. 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 Processor -- start 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 -- start start 2023-09-30T22:31:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 0x7f94bc122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc0a1340 0x7f94bc122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc133fa0 0x7f94bc1232f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1244d0 con 0x7f94bc0a0a30 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc133790 con 0x7f94bc0a1340 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94c10ac700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc125cc0 con 0x7f94bc133fa0 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 0x7f94bc122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94bad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 0x7f94bc122870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43046/0 (socket says 172.21.15.32:43046) 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.103+0000 7f94bad9d700 1 -- 172.21.15.32:0/383862156 learned_addr learned my addr 172.21.15.32:0/383862156 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94ba59c700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc0a1340 0x7f94bc122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc133fa0 0x7f94bc1232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 -- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc0a1340 msgr2=0x7f94bc122db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc0a1340 0x7f94bc122db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 -- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 msgr2=0x7f94bc122870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 0x7f94bc122870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 -- 172.21.15.32:0/383862156 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac009a70 con 0x7f94bc133fa0 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bad9d700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 0x7f94bc122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94bb59e700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc133fa0 0x7f94bc1232f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c400 tx=0x7f94b400c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4005d70 con 0x7f94bc133fa0 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b400ce10 con 0x7f94bc133fa0 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40072d0 con 0x7f94bc133fa0 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.104+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a40094a0 con 0x7f94bc133fa0 2023-09-30T22:31:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.105+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc0a0220 con 0x7f94bc133fa0 2023-09-30T22:31:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.105+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94b4007430 con 0x7f94bc133fa0 2023-09-30T22:31:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.106+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f94b403e990 con 0x7f94bc133fa0 2023-09-30T22:31:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.106+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc125ee0 con 0x7f94bc133fa0 2023-09-30T22:31:01.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.115+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94b4017020 con 0x7f94bc133fa0 2023-09-30T22:31:01.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.163+0000 7f60c52f2700 1 Processor -- start 2023-09-30T22:31:01.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.163+0000 7f60c52f2700 1 -- start start 2023-09-30T22:31:01.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60c52f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60c009d5c0 0x7f60c0138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60c52f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c013b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60c52f2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c009ea10 con 0x7f60c00a09c0 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60c52f2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c009e210 con 0x7f60c009d5c0 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60c52f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00a26f0 con 0x7f60c00a14a0 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60beffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f60c00a09c0 0x7f60c00a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57456/0 (socket says 172.21.15.32:57456) 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60beffd700 1 -- 172.21.15.32:0/48811693 learned_addr learned my addr 172.21.15.32:0/48811693 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60b7fff700 1 --2- 172.21.15.32:0/48811693 >> v2:172.21.15.32:3300/0 conn(0x7f60c009d5c0 0x7f60c0138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.164+0000 7f60bf7fe700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c013b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60b7fff700 1 -- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 msgr2=0x7f60c013b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:01.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60b7fff700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c013b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60b7fff700 1 -- 172.21.15.32:0/48811693 >> v1:172.21.15.32:6789/0 conn(0x7f60c00a09c0 legacy=0x7f60c00a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60b7fff700 1 -- 172.21.15.32:0/48811693 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c009db70 con 0x7f60c009d5c0 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60be7fc700 1 -- 172.21.15.32:0/48811693 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762810929 0 0) 0x7f60c009ea10 con 0x7f60c00a09c0 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60bf7fe700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c013b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60b7fff700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c009d5c0 0x7f60c0138d70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009840 tx=0x7f60a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edc8b8be72ab46e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60be7fc700 1 -- 172.21.15.32:0/48811693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8011830 con 0x7f60c009d5c0 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60be7fc700 1 -- 172.21.15.32:0/48811693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a8011990 con 0x7f60c009d5c0 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.165+0000 7f60be7fc700 1 -- 172.21.15.32:0/48811693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8011d20 con 0x7f60c009d5c0 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 -- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c009d5c0 msgr2=0x7f60c0138d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c009d5c0 0x7f60c0138d70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009840 tx=0x7f60a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:01.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 -- 172.21.15.32:0/48811693 shutdown_connections 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c013b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 --2- 172.21.15.32:0/48811693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c009d5c0 0x7f60c0138d70 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 -- 172.21.15.32:0/48811693 >> 172.21.15.32:0/48811693 conn(0x7f60c00a2fe0 msgr2=0x7f60c00a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 -- 172.21.15.32:0/48811693 shutdown_connections 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.166+0000 7f60c52f2700 1 -- 172.21.15.32:0/48811693 wait complete. 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.167+0000 7f60c52f2700 1 Processor -- start 2023-09-30T22:31:01.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 -- start start 2023-09-30T22:31:01.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c009d5c0 0x7f60c011c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c00a09c0 0x7f60c011cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c011d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c009d2b0 con 0x7f60c00a09c0 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c009cec0 con 0x7f60c00a14a0 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.169+0000 7f60c52f2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60c011e4f0 con 0x7f60c009d5c0 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60bf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c011d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60beffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c009d5c0 0x7f60c011c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c00a09c0 0x7f60c011cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60bf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c011d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54526/0 (socket says 172.21.15.32:54526) 2023-09-30T22:31:01.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60bf7fe700 1 -- 172.21.15.32:0/2229336675 learned_addr learned my addr 172.21.15.32:0/2229336675 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 -- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c009d5c0 msgr2=0x7f60c011c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c009d5c0 0x7f60c011c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 -- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 msgr2=0x7f60c011d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c011d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 -- 172.21.15.32:0/2229336675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009a70 con 0x7f60c00a09c0 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60bf7fe700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c011d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60beffd700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c009d5c0 0x7f60c011c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b7fff700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c00a09c0 0x7f60c011cda0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f60a8016000 tx=0x7f60a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8004480 con 0x7f60c00a09c0 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a801f070 con 0x7f60c00a09c0 2023-09-30T22:31:01.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a80094a0 con 0x7f60c00a09c0 2023-09-30T22:31:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.170+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80038a0 con 0x7f60c00a09c0 2023-09-30T22:31:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.171+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c011ebd0 con 0x7f60c00a09c0 2023-09-30T22:31:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.171+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60a8021030 con 0x7f60c00a09c0 2023-09-30T22:31:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.171+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60a8040770 con 0x7f60c00a09c0 2023-09-30T22:31:01.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.172+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c00a0fa0 con 0x7f60c00a09c0 2023-09-30T22:31:01.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.177+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60a8003a00 con 0x7f60c00a09c0 2023-09-30T22:31:01.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.290+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fca20131100 con 0x7fca201338c0 2023-09-30T22:31:01.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.291+0000 7fca15ffb700 1 -- 172.21.15.32:0/3116998390 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca10017070 con 0x7fca201338c0 2023-09-30T22:31:01.292 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 msgr2=0x7fca201200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca201200c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c510 tx=0x7fca1000c820 comp rx=0 tx=0).stop 2023-09-30T22:31:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 shutdown_connections 2023-09-30T22:31:01.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca201341d0 0x7fca20120600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca2009e560 0x7fca2011fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 --2- 172.21.15.32:0/3116998390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca201338c0 0x7fca201200c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 >> 172.21.15.32:0/3116998390 conn(0x7fca200a2aa0 msgr2=0x7fca201314c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 shutdown_connections 2023-09-30T22:31:01.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.293+0000 7fca27532700 1 -- 172.21.15.32:0/3116998390 wait complete. 2023-09-30T22:31:01.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:01.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.335+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f94bc074ea0 con 0x7f94bc133fa0 2023-09-30T22:31:01.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.336+0000 7f94b37fe700 1 -- 172.21.15.32:0/383862156 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94b401b3b0 con 0x7f94bc133fa0 2023-09-30T22:31:01.337 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc133fa0 msgr2=0x7f94bc1232f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc133fa0 0x7f94bc1232f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c400 tx=0x7f94b400c710 comp rx=0 tx=0).stop 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 shutdown_connections 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f94bc133fa0 0x7f94bc1232f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0a0a30 0x7f94bc122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 --2- 172.21.15.32:0/383862156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc0a1340 0x7f94bc122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 >> 172.21.15.32:0/383862156 conn(0x7f94bc0a6550 msgr2=0x7f94bc0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 shutdown_connections 2023-09-30T22:31:01.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.338+0000 7f94c10ac700 1 -- 172.21.15.32:0/383862156 wait complete. 2023-09-30T22:31:01.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:01.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.386+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f60c00747f0 con 0x7f60c00a09c0 2023-09-30T22:31:01.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.387+0000 7f60b77fe700 1 -- 172.21.15.32:0/2229336675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60a8015020 con 0x7f60c00a09c0 2023-09-30T22:31:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c00a09c0 msgr2=0x7f60c011cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c00a09c0 0x7f60c011cda0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f60a8016000 tx=0x7f60a8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:01.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 shutdown_connections 2023-09-30T22:31:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c00a14a0 0x7f60c011d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c009d5c0 0x7f60c011c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 --2- 172.21.15.32:0/2229336675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c00a09c0 0x7f60c011cda0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 >> 172.21.15.32:0/2229336675 conn(0x7f60c00a2fe0 msgr2=0x7f60c0134b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 shutdown_connections 2023-09-30T22:31:01.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.389+0000 7f60c52f2700 1 -- 172.21.15.32:0/2229336675 wait complete. 2023-09-30T22:31:01.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:01.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 Processor -- start 2023-09-30T22:31:01.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 -- start start 2023-09-30T22:31:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f4c09e410 0x7f8f4c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c09f130 con 0x7f8f4c09e000 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c09f520 con 0x7f8f4c09e410 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.860+0000 7f8f53cd6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c09e930 con 0x7f8f4c09d6f0 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37974/0 (socket says 172.21.15.32:37974) 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 -- 172.21.15.32:0/594582217 learned_addr learned my addr 172.21.15.32:0/594582217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f52273700 1 --2- 172.21.15.32:0/594582217 >> v2:172.21.15.32:3300/0 conn(0x7f8f4c09e410 0x7f8f4c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f50a70700 1 -- 172.21.15.32:0/594582217 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141660343 0 0) 0x7f8f4c09f130 con 0x7f8f4c09e000 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f50a70700 1 -- 172.21.15.32:0/594582217 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f380035b0 con 0x7f8f4c09e000 2023-09-30T22:31:01.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 -- 172.21.15.32:0/594582217 >> v2:172.21.15.32:3300/0 conn(0x7f8f4c09e410 msgr2=0x7f8f4c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 --2- 172.21.15.32:0/594582217 >> v2:172.21.15.32:3300/0 conn(0x7f8f4c09e410 0x7f8f4c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 -- 172.21.15.32:0/594582217 >> v1:172.21.15.32:6789/0 conn(0x7f8f4c09e000 legacy=0x7f8f4c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 -- 172.21.15.32:0/594582217 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c09b2d0 con 0x7f8f4c09d6f0 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f52273700 1 --2- 172.21.15.32:0/594582217 >> v2:172.21.15.32:3300/0 conn(0x7f8f4c09e410 0x7f8f4c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f51a72700 1 --2- 172.21.15.32:0/594582217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c09dac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8f480098f0 tx=0x7f8f48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0d4da57496f173 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f50a70700 1 -- 172.21.15.32:0/594582217 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011830 con 0x7f8f4c09d6f0 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f50a70700 1 -- 172.21.15.32:0/594582217 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48011990 con 0x7f8f4c09d6f0 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.861+0000 7f8f50a70700 1 -- 172.21.15.32:0/594582217 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011d00 con 0x7f8f4c09d6f0 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 -- 172.21.15.32:0/594582217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 msgr2=0x7f8f4c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/594582217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c09dac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8f480098f0 tx=0x7f8f48009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:01.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 -- 172.21.15.32:0/594582217 shutdown_connections 2023-09-30T22:31:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/594582217 >> v2:172.21.15.32:3300/0 conn(0x7f8f4c09e410 0x7f8f4c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/594582217 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c09dac0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 -- 172.21.15.32:0/594582217 >> 172.21.15.32:0/594582217 conn(0x7f8f4c066f50 msgr2=0x7f8f4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 -- 172.21.15.32:0/594582217 shutdown_connections 2023-09-30T22:31:01.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.862+0000 7f8f53cd6700 1 -- 172.21.15.32:0/594582217 wait complete. 2023-09-30T22:31:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.863+0000 7f8f53cd6700 1 Processor -- start 2023-09-30T22:31:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.863+0000 7f8f53cd6700 1 -- start start 2023-09-30T22:31:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c1200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 0x7f8f4c120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f4c09e410 0x7f8f4c120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c123b60 con 0x7f8f4c09e000 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c122270 con 0x7f8f4c09e410 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c09c770 con 0x7f8f4c09d6f0 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51a72700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c1200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 0x7f8f4c120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 0x7f8f4c120610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43096/0 (socket says 172.21.15.32:43096) 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51a72700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c1200d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:37984/0 (socket says 172.21.15.32:37984) 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 -- 172.21.15.32:0/2869174783 learned_addr learned my addr 172.21.15.32:0/2869174783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f52273700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f4c09e410 0x7f8f4c120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 -- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 msgr2=0x7f8f4c1200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c1200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 -- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f4c09e410 msgr2=0x7f8f4c120b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f4c09e410 0x7f8f4c120b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 -- 172.21.15.32:0/2869174783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009a70 con 0x7f8f4c09e000 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f51271700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 0x7f8f4c120610 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0018d0 tx=0x7f8f3c001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c013070 con 0x7f8f4c09e000 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c00e0d0 con 0x7f8f4c09e000 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f48009550 con 0x7f8f4c09e000 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.864+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0050d0 con 0x7f8f4c09e000 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.865+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c1232f0 con 0x7f8f4c09e000 2023-09-30T22:31:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.865+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f3c005250 con 0x7f8f4c09e000 2023-09-30T22:31:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.865+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8f3c042d70 con 0x7f8f4c09e000 2023-09-30T22:31:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.866+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c074ea0 con 0x7f8f4c09e000 2023-09-30T22:31:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:01.870+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f3c005500 con 0x7f8f4c09e000 2023-09-30T22:31:02.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.074+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8f4c122440 con 0x7f8f4c09e000 2023-09-30T22:31:02.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.074+0000 7f8f427fc700 1 -- 172.21.15.32:0/2869174783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f3c016390 con 0x7f8f4c09e000 2023-09-30T22:31:02.076 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.076+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 msgr2=0x7f8f4c120610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.076+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 0x7f8f4c120610 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0018d0 tx=0x7f8f3c001be0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 shutdown_connections 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8f4c09e410 0x7f8f4c120b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8f4c09d6f0 0x7f8f4c1200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 --2- 172.21.15.32:0/2869174783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f4c09e000 0x7f8f4c120610 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 >> 172.21.15.32:0/2869174783 conn(0x7f8f4c066f50 msgr2=0x7f8f4c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 shutdown_connections 2023-09-30T22:31:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.077+0000 7f8f53cd6700 1 -- 172.21.15.32:0/2869174783 wait complete. 2023-09-30T22:31:02.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:02.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:02.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:02.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.463+0000 7f952a6e0700 1 Processor -- start 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.463+0000 7f952a6e0700 1 -- start start 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f952a6e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95240a0a30 0x7f95240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f952a6e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f952a6e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9524133fa0 0x7f9524134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f952a6e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952409edd0 con 0x7f95240a0a30 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f952a6e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952409e1f0 con 0x7f95240a1340 2023-09-30T22:31:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f952a6e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f952409dc50 con 0x7f9524133fa0 2023-09-30T22:31:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95240a0a30 0x7f95240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9528c7d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9524133fa0 0x7f9524134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95240a0a30 0x7f95240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43108/0 (socket says 172.21.15.32:43108) 2023-09-30T22:31:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 -- 172.21.15.32:0/2180750437 learned_addr learned my addr 172.21.15.32:0/2180750437 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f95237fe700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 -- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9524133fa0 msgr2=0x7f9524134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9524133fa0 0x7f9524134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 -- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 msgr2=0x7f9524133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 -- 172.21.15.32:0/2180750437 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240a1fe0 con 0x7f95240a0a30 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9528c7d700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9524133fa0 0x7f9524134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.464+0000 7f9523fff700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240a0a30 0x7f95240a0e00 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f95140098f0 tx=0x7f9514009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bdde9efdd172b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.465+0000 7f9522ffd700 1 -- 172.21.15.32:0/2180750437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514011830 con 0x7f95240a0a30 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.465+0000 7f9522ffd700 1 -- 172.21.15.32:0/2180750437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514011990 con 0x7f95240a0a30 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.465+0000 7f9522ffd700 1 -- 172.21.15.32:0/2180750437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514011d20 con 0x7f95240a0a30 2023-09-30T22:31:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 -- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240a0a30 msgr2=0x7f95240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240a0a30 0x7f95240a0e00 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f95140098f0 tx=0x7f9514009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 -- 172.21.15.32:0/2180750437 shutdown_connections 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9524133fa0 0x7f9524134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240a0a30 0x7f95240a0e00 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2180750437 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 -- 172.21.15.32:0/2180750437 >> 172.21.15.32:0/2180750437 conn(0x7f95240a6550 msgr2=0x7f95240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 -- 172.21.15.32:0/2180750437 shutdown_connections 2023-09-30T22:31:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.466+0000 7f952a6e0700 1 -- 172.21.15.32:0/2180750437 wait complete. 2023-09-30T22:31:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.467+0000 7f952a6e0700 1 Processor -- start 2023-09-30T22:31:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 -- start start 2023-09-30T22:31:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95240a0a30 0x7f95241200c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524120600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524133fa0 0x7f9524120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524123be0 con 0x7f9524133fa0 2023-09-30T22:31:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9524122fb0 con 0x7f95240a1340 2023-09-30T22:31:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.468+0000 7f952a6e0700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95240a34c0 con 0x7f95240a0a30 2023-09-30T22:31:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524120600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f9528c7d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524133fa0 0x7f9524120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f9523fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95240a0a30 0x7f95241200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524120600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54544/0 (socket says 172.21.15.32:54544) 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 -- 172.21.15.32:0/2287357560 learned_addr learned my addr 172.21.15.32:0/2287357560 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 -- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95240a0a30 msgr2=0x7f95241200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95240a0a30 0x7f95241200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 -- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524133fa0 msgr2=0x7f9524120b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524133fa0 0x7f9524120b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 -- 172.21.15.32:0/2287357560 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009a70 con 0x7f95240a1340 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f9528c7d700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524133fa0 0x7f9524120b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f95237fe700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524120600 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f951800e9b0 tx=0x7f951800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518009710 con 0x7f95240a1340 2023-09-30T22:31:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951800ee40 con 0x7f95240a1340 2023-09-30T22:31:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951800ca50 con 0x7f95240a1340 2023-09-30T22:31:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.469+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9514009550 con 0x7f95240a1340 2023-09-30T22:31:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.470+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9524121cd0 con 0x7f95240a1340 2023-09-30T22:31:02.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.471+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f951800cbb0 con 0x7f95240a1340 2023-09-30T22:31:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.471+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9524074ea0 con 0x7f95240a1340 2023-09-30T22:31:02.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.471+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f951803fa90 con 0x7f95240a1340 2023-09-30T22:31:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.480+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f951801b3b0 con 0x7f95240a1340 2023-09-30T22:31:02.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 Processor -- start 2023-09-30T22:31:02.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 -- start start 2023-09-30T22:31:02.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f80a0a30 0x7f72f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f8138d80 0x7f72f813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80a2a70 con 0x7f72f809d630 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80a01d0 con 0x7f72f8138d80 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.504+0000 7f7300071700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f809e2b0 con 0x7f72f80a0a30 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fde0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f80a0a30 0x7f72f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fd60c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72f809d630 0x7f72f80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57472/0 (socket says 172.21.15.32:57472) 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fd60c700 1 -- 172.21.15.32:0/801731322 learned_addr learned my addr 172.21.15.32:0/801731322 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:02.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fe60e700 1 --2- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f8138d80 0x7f72f813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fde0d700 1 -- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f8138d80 msgr2=0x7f72f813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fde0d700 1 --2- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f8138d80 0x7f72f813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fde0d700 1 -- 172.21.15.32:0/801731322 >> v1:172.21.15.32:6789/0 conn(0x7f72f809d630 legacy=0x7f72f80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fde0d700 1 -- 172.21.15.32:0/801731322 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f809dc50 con 0x7f72f80a0a30 2023-09-30T22:31:02.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fde0d700 1 --2- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f80a0a30 0x7f72f80a0e00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009870 tx=0x7f72f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d85a496abdb9b0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.505+0000 7f72fce0b700 1 -- 172.21.15.32:0/801731322 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4011830 con 0x7f72f80a0a30 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f72fce0b700 1 -- 172.21.15.32:0/801731322 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f4011990 con 0x7f72f80a0a30 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f72fce0b700 1 -- 172.21.15.32:0/801731322 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4011d20 con 0x7f72f80a0a30 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 -- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f80a0a30 msgr2=0x7f72f80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 --2- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f80a0a30 0x7f72f80a0e00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009870 tx=0x7f72f4009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 -- 172.21.15.32:0/801731322 shutdown_connections 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 --2- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f8138d80 0x7f72f813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 --2- 172.21.15.32:0/801731322 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f80a0a30 0x7f72f80a0e00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 -- 172.21.15.32:0/801731322 >> 172.21.15.32:0/801731322 conn(0x7f72f80a3050 msgr2=0x7f72f80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 -- 172.21.15.32:0/801731322 shutdown_connections 2023-09-30T22:31:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.506+0000 7f7300071700 1 -- 172.21.15.32:0/801731322 wait complete. 2023-09-30T22:31:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.508+0000 7f7300071700 1 Processor -- start 2023-09-30T22:31:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.508+0000 7f7300071700 1 -- start start 2023-09-30T22:31:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f7300071700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 0x7f72f811d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f7300071700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f80a0a30 0x7f72f811da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f7300071700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8138d80 0x7f72f811df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f7300071700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f811f5b0 con 0x7f72f8138d80 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f7300071700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f811fdd0 con 0x7f72f80a0a30 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f7300071700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f811efd0 con 0x7f72f809d630 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 0x7f72f811d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 0x7f72f811d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38020/0 (socket says 172.21.15.32:38020) 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 -- 172.21.15.32:0/2781389521 learned_addr learned my addr 172.21.15.32:0/2781389521 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fd60c700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f80a0a30 0x7f72f811da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fe60e700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8138d80 0x7f72f811df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 -- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f80a0a30 msgr2=0x7f72f811da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f80a0a30 0x7f72f811da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 -- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8138d80 msgr2=0x7f72f811df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8138d80 0x7f72f811df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 -- 172.21.15.32:0/2781389521 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40094a0 con 0x7f72f809d630 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.509+0000 7f72fde0d700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 0x7f72f811d4f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009840 tx=0x7f72f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.510+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4004450 con 0x7f72f809d630 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.510+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f401f070 con 0x7f72f809d630 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.510+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f813aec0 con 0x7f72f809d630 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.510+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f40038e0 con 0x7f72f809d630 2023-09-30T22:31:02.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.510+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f80a0470 con 0x7f72f809d630 2023-09-30T22:31:02.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.511+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72f4003a40 con 0x7f72f809d630 2023-09-30T22:31:02.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.511+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f72f4021070 con 0x7f72f809d630 2023-09-30T22:31:02.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.511+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f80747f0 con 0x7f72f809d630 2023-09-30T22:31:02.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.516+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72f402b080 con 0x7f72f809d630 2023-09-30T22:31:02.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.552+0000 7faa4ae50700 1 Processor -- start 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.552+0000 7faa4ae50700 1 -- start start 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa4ae50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa4409d630 0x7faa440a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa4ae50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa44138d80 0x7faa4413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa4ae50700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440a2a70 con 0x7faa440a0a30 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa4ae50700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440a01d0 con 0x7faa4409d630 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa4ae50700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa4409e2b0 con 0x7faa44138d80 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa48bec700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faa440a0a30 0x7faa440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57486/0 (socket says 172.21.15.32:57486) 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa48bec700 1 -- 172.21.15.32:0/771116476 learned_addr learned my addr 172.21.15.32:0/771116476 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa43fff700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa4409d630 0x7faa440a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa493ed700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa44138d80 0x7faa4413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa43fff700 1 -- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa44138d80 msgr2=0x7faa4413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa43fff700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa44138d80 0x7faa4413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa43fff700 1 -- 172.21.15.32:0/771116476 >> v1:172.21.15.32:6789/0 conn(0x7faa440a0a30 legacy=0x7faa440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa437fe700 1 -- 172.21.15.32:0/771116476 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955433921 0 0) 0x7faa440a2a70 con 0x7faa440a0a30 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa43fff700 1 -- 172.21.15.32:0/771116476 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4409dc50 con 0x7faa4409d630 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.553+0000 7faa493ed700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa44138d80 0x7faa4413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.554+0000 7faa43fff700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa4409d630 0x7faa440a1510 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faa30009870 tx=0x7faa30009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110812860dd03fdb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.554+0000 7faa437fe700 1 -- 172.21.15.32:0/771116476 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30011830 con 0x7faa4409d630 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.554+0000 7faa437fe700 1 -- 172.21.15.32:0/771116476 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30011990 con 0x7faa4409d630 2023-09-30T22:31:02.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.554+0000 7faa437fe700 1 -- 172.21.15.32:0/771116476 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30011d20 con 0x7faa4409d630 2023-09-30T22:31:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.554+0000 7faa4ae50700 1 -- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa4409d630 msgr2=0x7faa440a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.554+0000 7faa4ae50700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa4409d630 0x7faa440a1510 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faa30009870 tx=0x7faa30009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.555+0000 7faa4ae50700 1 -- 172.21.15.32:0/771116476 shutdown_connections 2023-09-30T22:31:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.555+0000 7faa4ae50700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa44138d80 0x7faa4413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.555+0000 7faa4ae50700 1 --2- 172.21.15.32:0/771116476 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa4409d630 0x7faa440a1510 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.555+0000 7faa4ae50700 1 -- 172.21.15.32:0/771116476 >> 172.21.15.32:0/771116476 conn(0x7faa440a3050 msgr2=0x7faa440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.555+0000 7faa4ae50700 1 -- 172.21.15.32:0/771116476 shutdown_connections 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.555+0000 7faa4ae50700 1 -- 172.21.15.32:0/771116476 wait complete. 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 Processor -- start 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 -- start start 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa4409d630 0x7faa44133910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 0x7faa44133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa44138d80 0x7faa44134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44134de0 con 0x7faa4409d630 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa441374b0 con 0x7faa44138d80 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa4ae50700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa44136e00 con 0x7faa440a0a30 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa43fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 0x7faa44133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa43fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 0x7faa44133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38028/0 (socket says 172.21.15.32:38028) 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa43fff700 1 -- 172.21.15.32:0/1116373666 learned_addr learned my addr 172.21.15.32:0/1116373666 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa48bec700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa4409d630 0x7faa44133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.556+0000 7faa493ed700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa44138d80 0x7faa44134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa43fff700 1 -- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa44138d80 msgr2=0x7faa44134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa43fff700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa44138d80 0x7faa44134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa43fff700 1 -- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa4409d630 msgr2=0x7faa44133910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa43fff700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa4409d630 0x7faa44133910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa43fff700 1 -- 172.21.15.32:0/1116373666 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c009a70 con 0x7faa440a0a30 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa43fff700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 0x7faa44133e50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7faa30009840 tx=0x7faa30004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300042c0 con 0x7faa440a0a30 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3001f070 con 0x7faa440a0a30 2023-09-30T22:31:02.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa300094a0 con 0x7faa440a0a30 2023-09-30T22:31:02.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300036a0 con 0x7faa440a0a30 2023-09-30T22:31:02.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.557+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa441370c0 con 0x7faa440a0a30 2023-09-30T22:31:02.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.558+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa30021030 con 0x7faa440a0a30 2023-09-30T22:31:02.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.559+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa44135ce0 con 0x7faa440a0a30 2023-09-30T22:31:02.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.559+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faa300406c0 con 0x7faa440a0a30 2023-09-30T22:31:02.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.568+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa300199a0 con 0x7faa440a0a30 2023-09-30T22:31:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.689+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f95240a28e0 con 0x7f95240a1340 2023-09-30T22:31:02.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.689+0000 7f9520ff9700 1 -- 172.21.15.32:0/2287357560 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9518017030 con 0x7f95240a1340 2023-09-30T22:31:02.691 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 msgr2=0x7f9524120600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524120600 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f951800e9b0 tx=0x7f951800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 shutdown_connections 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9524133fa0 0x7f9524120b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95240a0a30 0x7f95241200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 --2- 172.21.15.32:0/2287357560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95240a1340 0x7f9524120600 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 >> 172.21.15.32:0/2287357560 conn(0x7f95240a6550 msgr2=0x7f952409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 shutdown_connections 2023-09-30T22:31:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.692+0000 7f952a6e0700 1 -- 172.21.15.32:0/2287357560 wait complete. 2023-09-30T22:31:02.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:02.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.729+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f72f81209d0 con 0x7f72f809d630 2023-09-30T22:31:02.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.729+0000 7f72ee7fc700 1 -- 172.21.15.32:0/2781389521 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72f4015020 con 0x7f72f809d630 2023-09-30T22:31:02.730 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 msgr2=0x7f72f811d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 0x7f72f811d4f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009840 tx=0x7f72f4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 shutdown_connections 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8138d80 0x7f72f811df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f72f809d630 0x7f72f811d4f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 --2- 172.21.15.32:0/2781389521 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f72f80a0a30 0x7f72f811da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 >> 172.21.15.32:0/2781389521 conn(0x7f72f80a3050 msgr2=0x7f72f8139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 shutdown_connections 2023-09-30T22:31:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.732+0000 7f7300071700 1 -- 172.21.15.32:0/2781389521 wait complete. 2023-09-30T22:31:02.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.779+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faa440747f0 con 0x7faa440a0a30 2023-09-30T22:31:02.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.779+0000 7faa417fa700 1 -- 172.21.15.32:0/1116373666 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa30015070 con 0x7faa440a0a30 2023-09-30T22:31:02.781 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 msgr2=0x7faa44133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 0x7faa44133e50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7faa30009840 tx=0x7faa30004750 comp rx=0 tx=0).stop 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 shutdown_connections 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faa44138d80 0x7faa44134390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa4409d630 0x7faa44133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 --2- 172.21.15.32:0/1116373666 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7faa440a0a30 0x7faa44133e50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 >> 172.21.15.32:0/1116373666 conn(0x7faa440a3050 msgr2=0x7faa4409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 shutdown_connections 2023-09-30T22:31:02.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:02.782+0000 7faa4ae50700 1 -- 172.21.15.32:0/1116373666 wait complete. 2023-09-30T22:31:02.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:03.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:03.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 Processor -- start 2023-09-30T22:31:03.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 -- start start 2023-09-30T22:31:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009d6f0 0x7f0e3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e410 0x7f0e30130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3009f130 con 0x7f0e3009e000 2023-09-30T22:31:03.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3009f520 con 0x7f0e3009e410 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.239+0000 7f0e36725700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3009e930 con 0x7f0e3009d6f0 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2ffff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009d6f0 0x7f0e3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e34cc2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e410 0x7f0e30130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e3009e000 0x7f0e30130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57494/0 (socket says 172.21.15.32:57494) 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2f7fe700 1 -- 172.21.15.32:0/4041783933 learned_addr learned my addr 172.21.15.32:0/4041783933 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2effd700 1 -- 172.21.15.32:0/4041783933 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920389743 0 0) 0x7f0e3009f130 con 0x7f0e3009e000 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2effd700 1 -- 172.21.15.32:0/4041783933 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e180035b0 con 0x7f0e3009e000 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e34cc2700 1 -- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009d6f0 msgr2=0x7f0e3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e34cc2700 1 --2- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009d6f0 0x7f0e3009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e34cc2700 1 -- 172.21.15.32:0/4041783933 >> v1:172.21.15.32:6789/0 conn(0x7f0e3009e000 legacy=0x7f0e30130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e34cc2700 1 -- 172.21.15.32:0/4041783933 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3009b2d0 con 0x7f0e3009e410 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2ffff700 1 --2- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009d6f0 0x7f0e3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e34cc2700 1 --2- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e410 0x7f0e30130d70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009dc0 tx=0x7f0e2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1432fd1fef34a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2effd700 1 -- 172.21.15.32:0/4041783933 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800e070 con 0x7f0e3009e410 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2effd700 1 -- 172.21.15.32:0/4041783933 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2800bb20 con 0x7f0e3009e410 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.240+0000 7f0e2effd700 1 -- 172.21.15.32:0/4041783933 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28012430 con 0x7f0e3009e410 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 -- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e410 msgr2=0x7f0e30130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 --2- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e410 0x7f0e30130d70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009dc0 tx=0x7f0e2800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 -- 172.21.15.32:0/4041783933 shutdown_connections 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 --2- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e410 0x7f0e30130d70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 --2- 172.21.15.32:0/4041783933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009d6f0 0x7f0e3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 -- 172.21.15.32:0/4041783933 >> 172.21.15.32:0/4041783933 conn(0x7f0e30066f50 msgr2=0x7f0e3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 -- 172.21.15.32:0/4041783933 shutdown_connections 2023-09-30T22:31:03.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.241+0000 7f0e36725700 1 -- 172.21.15.32:0/4041783933 wait complete. 2023-09-30T22:31:03.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.242+0000 7f0e36725700 1 Processor -- start 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.242+0000 7f0e36725700 1 -- start start 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 0x7f0e30137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e000 0x7f0e301377c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009e410 0x7f0e30137d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3011a120 con 0x7f0e3009d6f0 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30138e90 con 0x7f0e3009e000 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3013a0e0 con 0x7f0e3009e410 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 0x7f0e30137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e000 0x7f0e301377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 0x7f0e30137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43156/0 (socket says 172.21.15.32:43156) 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2ffff700 1 -- 172.21.15.32:0/1735371250 learned_addr learned my addr 172.21.15.32:0/1735371250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e34cc2700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009e410 0x7f0e30137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 -- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009e410 msgr2=0x7f0e30137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009e410 0x7f0e30137d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 -- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 msgr2=0x7f0e30137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 0x7f0e30137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 -- 172.21.15.32:0/1735371250 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200094a0 con 0x7f0e3009e000 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2ffff700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 0x7f0e30137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2f7fe700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e000 0x7f0e301377c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c4c0 tx=0x7f0e2400c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24005d70 con 0x7f0e3009e000 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e240070d0 con 0x7f0e3009e000 2023-09-30T22:31:03.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.243+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e28009a70 con 0x7f0e3009e000 2023-09-30T22:31:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.244+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007440 con 0x7f0e3009e000 2023-09-30T22:31:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.244+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3009c700 con 0x7f0e3009e000 2023-09-30T22:31:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.244+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e240075a0 con 0x7f0e3009e000 2023-09-30T22:31:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.244+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e2403ea80 con 0x7f0e3009e000 2023-09-30T22:31:03.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.245+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e30070590 con 0x7f0e3009e000 2023-09-30T22:31:03.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.250+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e24017070 con 0x7f0e3009e000 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.452+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e30074ea0 con 0x7f0e3009e000 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.452+0000 7f0e2cff9700 1 -- 172.21.15.32:0/1735371250 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e2401bb10 con 0x7f0e3009e000 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:03.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e000 msgr2=0x7f0e301377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e000 0x7f0e301377c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c4c0 tx=0x7f0e2400c7d0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 shutdown_connections 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e3009e410 0x7f0e30137d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e3009d6f0 0x7f0e30137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 --2- 172.21.15.32:0/1735371250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e3009e000 0x7f0e301377c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 >> 172.21.15.32:0/1735371250 conn(0x7f0e30066f50 msgr2=0x7f0e30130980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 shutdown_connections 2023-09-30T22:31:03.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.454+0000 7f0e36725700 1 -- 172.21.15.32:0/1735371250 wait complete. 2023-09-30T22:31:03.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:03.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:03.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:03.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:03.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 Processor -- start 2023-09-30T22:31:03.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 -- start start 2023-09-30T22:31:03.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e480a0a30 0x7f8e480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e4809d630 0x7f8e480a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e48133930 0x7f8e48133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e480a2a70 con 0x7f8e4809d630 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480a01d0 con 0x7f8e480a0a30 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e4d7d8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4809e2b0 con 0x7f8e48133930 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e467fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e4809d630 0x7f8e480a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.859+0000 7f8e46ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e480a0a30 0x7f8e480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e477fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e48133930 0x7f8e48133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e4809d630 0x7f8e480a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43158/0 (socket says 172.21.15.32:43158) 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 -- 172.21.15.32:0/1582176985 learned_addr learned my addr 172.21.15.32:0/1582176985 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 -- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e48133930 msgr2=0x7f8e48133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e48133930 0x7f8e48133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 -- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e480a0a30 msgr2=0x7f8e480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e480a0a30 0x7f8e480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 -- 172.21.15.32:0/1582176985 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4809dc50 con 0x7f8e4809d630 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e477fe700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e48133930 0x7f8e48133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e46ffd700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e480a0a30 0x7f8e480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e467fc700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e480a1340 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0098f0 tx=0x7f8e3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9805feaf0314654 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e45ffb700 1 -- 172.21.15.32:0/1582176985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c011830 con 0x7f8e4809d630 2023-09-30T22:31:03.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e45ffb700 1 -- 172.21.15.32:0/1582176985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c011990 con 0x7f8e4809d630 2023-09-30T22:31:03.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.860+0000 7f8e45ffb700 1 -- 172.21.15.32:0/1582176985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c011d20 con 0x7f8e4809d630 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 msgr2=0x7f8e480a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e480a1340 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0098f0 tx=0x7f8e3c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/1582176985 shutdown_connections 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e480a0a30 0x7f8e480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e48133930 0x7f8e48133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/1582176985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e480a1340 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/1582176985 >> 172.21.15.32:0/1582176985 conn(0x7f8e480a3050 msgr2=0x7f8e480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/1582176985 shutdown_connections 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.861+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/1582176985 wait complete. 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.862+0000 7f8e4d7d8700 1 Processor -- start 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 -- start start 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e48121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e480a0a30 0x7f8e48121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e48133930 0x7f8e48121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e481226d0 con 0x7f8e4809d630 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48124cb0 con 0x7f8e48133930 2023-09-30T22:31:03.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e4d7d8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48124710 con 0x7f8e480a0a30 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e46ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e48121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e467fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e480a0a30 0x7f8e48121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e46ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e48121220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43172/0 (socket says 172.21.15.32:43172) 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e46ffd700 1 -- 172.21.15.32:0/2022965552 learned_addr learned my addr 172.21.15.32:0/2022965552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.863+0000 7f8e477fe700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e48133930 0x7f8e48121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e46ffd700 1 -- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e480a0a30 msgr2=0x7f8e48121760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e46ffd700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e480a0a30 0x7f8e48121760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e46ffd700 1 -- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e48133930 msgr2=0x7f8e48121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e46ffd700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e48133930 0x7f8e48121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e46ffd700 1 -- 172.21.15.32:0/2022965552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380094a0 con 0x7f8e4809d630 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e467fc700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e480a0a30 0x7f8e48121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e46ffd700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e48121220 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800ebe0 tx=0x7f8e3800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38009800 con 0x7f8e4809d630 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40009a70 con 0x7f8e4809d630 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3800c850 con 0x7f8e4809d630 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3800cbc0 con 0x7f8e4809d630 2023-09-30T22:31:03.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.864+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c009550 con 0x7f8e4809d630 2023-09-30T22:31:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.865+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e480a0410 con 0x7f8e4809d630 2023-09-30T22:31:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.866+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e3800cd20 con 0x7f8e4809d630 2023-09-30T22:31:03.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.866+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8e3803f7e0 con 0x7f8e4809d630 2023-09-30T22:31:03.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.871+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e3801bde0 con 0x7f8e4809d630 2023-09-30T22:31:03.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.898+0000 7f251c3e4700 1 Processor -- start 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.898+0000 7f251c3e4700 1 -- start start 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251c3e4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f25140a0a30 0x7f25140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251c3e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 0x7f25140a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251c3e4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140a2a70 con 0x7f2514133930 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251c3e4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140a01d0 con 0x7f25140a0a30 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251c3e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f251409e2b0 con 0x7f251409d630 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251997f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 0x7f25140a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251997f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 0x7f25140a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38068/0 (socket says 172.21.15.32:38068) 2023-09-30T22:31:03.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251997f700 1 -- 172.21.15.32:0/3190068919 learned_addr learned my addr 172.21.15.32:0/3190068919 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.899+0000 7f251a180700 1 --2- 172.21.15.32:0/3190068919 >> v2:172.21.15.32:3300/0 conn(0x7f25140a0a30 0x7f25140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251997f700 1 -- 172.21.15.32:0/3190068919 >> v2:172.21.15.32:3300/0 conn(0x7f25140a0a30 msgr2=0x7f25140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251997f700 1 --2- 172.21.15.32:0/3190068919 >> v2:172.21.15.32:3300/0 conn(0x7f25140a0a30 0x7f25140a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251997f700 1 -- 172.21.15.32:0/3190068919 >> v1:172.21.15.32:6789/0 conn(0x7f2514133930 legacy=0x7f2514133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251997f700 1 -- 172.21.15.32:0/3190068919 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251409dc50 con 0x7f251409d630 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251a180700 1 --2- 172.21.15.32:0/3190068919 >> v2:172.21.15.32:3300/0 conn(0x7f25140a0a30 0x7f25140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251997f700 1 --2- 172.21.15.32:0/3190068919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 0x7f25140a1340 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f25040098f0 tx=0x7f2504009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511721eec70e75dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251917e700 1 -- 172.21.15.32:0/3190068919 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011830 con 0x7f251409d630 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251917e700 1 -- 172.21.15.32:0/3190068919 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504011990 con 0x7f251409d630 2023-09-30T22:31:03.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.900+0000 7f251917e700 1 -- 172.21.15.32:0/3190068919 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011d20 con 0x7f251409d630 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 -- 172.21.15.32:0/3190068919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 msgr2=0x7f25140a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 --2- 172.21.15.32:0/3190068919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 0x7f25140a1340 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f25040098f0 tx=0x7f2504009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 -- 172.21.15.32:0/3190068919 shutdown_connections 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 --2- 172.21.15.32:0/3190068919 >> v2:172.21.15.32:3300/0 conn(0x7f25140a0a30 0x7f25140a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 --2- 172.21.15.32:0/3190068919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f251409d630 0x7f25140a1340 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 -- 172.21.15.32:0/3190068919 >> 172.21.15.32:0/3190068919 conn(0x7f25140a3050 msgr2=0x7f25140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 -- 172.21.15.32:0/3190068919 shutdown_connections 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.901+0000 7f251c3e4700 1 -- 172.21.15.32:0/3190068919 wait complete. 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.902+0000 7f251c3e4700 1 Processor -- start 2023-09-30T22:31:03.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.902+0000 7f251c3e4700 1 -- start start 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251c3e4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 0x7f251411f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251c3e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 0x7f251411fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251c3e4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2514133930 0x7f25141c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251c3e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514122320 con 0x7f25140a0a30 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251c3e4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2514120c40 con 0x7f251409d630 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251c3e4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2514122500 con 0x7f2514133930 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a180700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 0x7f251411f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251997f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 0x7f251411fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a180700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 0x7f251411f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54630/0 (socket says 172.21.15.32:54630) 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251997f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 0x7f251411fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43184/0 (socket says 172.21.15.32:43184) 2023-09-30T22:31:03.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a180700 1 -- 172.21.15.32:0/163464522 learned_addr learned my addr 172.21.15.32:0/163464522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2514133930 0x7f25141c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 -- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 msgr2=0x7f251411f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 0x7f251411f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 -- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 msgr2=0x7f251411fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 0x7f251411fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 -- 172.21.15.32:0/163464522 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100094a0 con 0x7f2514133930 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a180700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 0x7f251411f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251997f700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 0x7f251411fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.903+0000 7f251a981700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2514133930 0x7f25141c8120 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f250c00cbc0 tx=0x7f250c00ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.904+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c003c00 con 0x7f2514133930 2023-09-30T22:31:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.904+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c003d60 con 0x7f2514133930 2023-09-30T22:31:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.904+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2504009550 con 0x7f2514133930 2023-09-30T22:31:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.904+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00aa40 con 0x7f2514133930 2023-09-30T22:31:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.904+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251411ee90 con 0x7f2514133930 2023-09-30T22:31:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.905+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f250c00aba0 con 0x7f2514133930 2023-09-30T22:31:03.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.905+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f250c03f570 con 0x7f2514133930 2023-09-30T22:31:03.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.905+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514122640 con 0x7f2514133930 2023-09-30T22:31:03.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.915+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2514122640 con 0x7f2514133930 2023-09-30T22:31:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.940+0000 7f351fe32700 1 Processor -- start 2023-09-30T22:31:03.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.940+0000 7f351fe32700 1 -- start start 2023-09-30T22:31:03.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351fe32700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35180a0a30 0x7f35180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351fe32700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f35180a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351fe32700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35180a2a70 con 0x7f3518133930 2023-09-30T22:31:03.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351fe32700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180a01d0 con 0x7f35180a0a30 2023-09-30T22:31:03.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351fe32700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351809e2b0 con 0x7f351809d630 2023-09-30T22:31:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351d3cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f35180a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351d3cd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f35180a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54638/0 (socket says 172.21.15.32:54638) 2023-09-30T22:31:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351d3cd700 1 -- 172.21.15.32:0/3323896439 learned_addr learned my addr 172.21.15.32:0/3323896439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.941+0000 7f351dbce700 1 --2- 172.21.15.32:0/3323896439 >> v2:172.21.15.32:3300/0 conn(0x7f35180a0a30 0x7f35180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351d3cd700 1 -- 172.21.15.32:0/3323896439 >> v2:172.21.15.32:3300/0 conn(0x7f35180a0a30 msgr2=0x7f35180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351d3cd700 1 --2- 172.21.15.32:0/3323896439 >> v2:172.21.15.32:3300/0 conn(0x7f35180a0a30 0x7f35180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351d3cd700 1 -- 172.21.15.32:0/3323896439 >> v1:172.21.15.32:6789/0 conn(0x7f3518133930 legacy=0x7f3518133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351cbcc700 1 -- 172.21.15.32:0/3323896439 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939700418 0 0) 0x7f35180a2a70 con 0x7f3518133930 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351d3cd700 1 -- 172.21.15.32:0/3323896439 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351809dc50 con 0x7f351809d630 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351d3cd700 1 --2- 172.21.15.32:0/3323896439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f35180a1340 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f35080098f0 tx=0x7f3508009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b6858b8b1c6b9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351cbcc700 1 -- 172.21.15.32:0/3323896439 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508011830 con 0x7f351809d630 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351cbcc700 1 -- 172.21.15.32:0/3323896439 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508011990 con 0x7f351809d630 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.942+0000 7f351cbcc700 1 -- 172.21.15.32:0/3323896439 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508011d20 con 0x7f351809d630 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 -- 172.21.15.32:0/3323896439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 msgr2=0x7f35180a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 --2- 172.21.15.32:0/3323896439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f35180a1340 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f35080098f0 tx=0x7f3508009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:03.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 -- 172.21.15.32:0/3323896439 shutdown_connections 2023-09-30T22:31:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 --2- 172.21.15.32:0/3323896439 >> v2:172.21.15.32:3300/0 conn(0x7f35180a0a30 0x7f35180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 --2- 172.21.15.32:0/3323896439 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f35180a1340 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 -- 172.21.15.32:0/3323896439 >> 172.21.15.32:0/3323896439 conn(0x7f35180a3050 msgr2=0x7f35180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 -- 172.21.15.32:0/3323896439 shutdown_connections 2023-09-30T22:31:03.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.943+0000 7f351fe32700 1 -- 172.21.15.32:0/3323896439 wait complete. 2023-09-30T22:31:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 Processor -- start 2023-09-30T22:31:03.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 -- start start 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f351811e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35180a0a30 0x7f351811ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3518133930 0x7f351811f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518120430 con 0x7f3518133930 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3518121da0 con 0x7f351809d630 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.944+0000 7f351fe32700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3518122420 con 0x7f35180a0a30 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f351811e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f351811e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54644/0 (socket says 172.21.15.32:54644) 2023-09-30T22:31:03.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 -- 172.21.15.32:0/3005271899 learned_addr learned my addr 172.21.15.32:0/3005271899 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351d3cd700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35180a0a30 0x7f351811ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351e3cf700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3518133930 0x7f351811f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 -- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35180a0a30 msgr2=0x7f351811ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35180a0a30 0x7f351811ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 -- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3518133930 msgr2=0x7f351811f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3518133930 0x7f351811f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 -- 172.21.15.32:0/3005271899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508009550 con 0x7f351809d630 2023-09-30T22:31:03.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351e3cf700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3518133930 0x7f351811f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351dbce700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f351811e8c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f351400e9b0 tx=0x7f351400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514009710 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351400c850 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400cbc0 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.945+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35180a5360 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.946+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35181332d0 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.946+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35180747f0 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.947+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f351400cd20 con 0x7f351809d630 2023-09-30T22:31:03.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.947+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f351403fb40 con 0x7f351809d630 2023-09-30T22:31:03.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:03.952+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f351401b3b0 con 0x7f351809d630 2023-09-30T22:31:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.085+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e48124930 con 0x7f8e4809d630 2023-09-30T22:31:04.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.086+0000 7f8e2f7fe700 1 -- 172.21.15.32:0/2022965552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e48124930 con 0x7f8e4809d630 2023-09-30T22:31:04.087 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 msgr2=0x7f8e48121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e48121220 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800ebe0 tx=0x7f8e3800eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 shutdown_connections 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8e48133930 0x7f8e48121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e4809d630 0x7f8e48121220 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 --2- 172.21.15.32:0/2022965552 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8e480a0a30 0x7f8e48121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 >> 172.21.15.32:0/2022965552 conn(0x7f8e480a3050 msgr2=0x7f8e48099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 shutdown_connections 2023-09-30T22:31:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.088+0000 7f8e4d7d8700 1 -- 172.21.15.32:0/2022965552 wait complete. 2023-09-30T22:31:04.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.125+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2514122640 con 0x7f2514133930 2023-09-30T22:31:04.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.126+0000 7f250affd700 1 -- 172.21.15.32:0/163464522 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2514122640 con 0x7f2514133930 2023-09-30T22:31:04.127 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:04.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2514133930 msgr2=0x7f25141c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2514133930 0x7f25141c8120 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f250c00cbc0 tx=0x7f250c00ced0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 shutdown_connections 2023-09-30T22:31:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2514133930 0x7f25141c8120 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f251409d630 0x7f251411f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 --2- 172.21.15.32:0/163464522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25140a0a30 0x7f251411fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 >> 172.21.15.32:0/163464522 conn(0x7f25140a3050 msgr2=0x7f25140a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 shutdown_connections 2023-09-30T22:31:04.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.128+0000 7f251c3e4700 1 -- 172.21.15.32:0/163464522 wait complete. 2023-09-30T22:31:04.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.165+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3518072980 con 0x7f351809d630 2023-09-30T22:31:04.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.166+0000 7f350e7fc700 1 -- 172.21.15.32:0/3005271899 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3514021070 con 0x7f351809d630 2023-09-30T22:31:04.167 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:04.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 msgr2=0x7f351811e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f351811e8c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f351400e9b0 tx=0x7f351400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 shutdown_connections 2023-09-30T22:31:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3518133930 0x7f351811f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f351809d630 0x7f351811e8c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 --2- 172.21.15.32:0/3005271899 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f35180a0a30 0x7f351811ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 >> 172.21.15.32:0/3005271899 conn(0x7f35180a3050 msgr2=0x7f35181bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 shutdown_connections 2023-09-30T22:31:04.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.168+0000 7f351fe32700 1 -- 172.21.15.32:0/3005271899 wait complete. 2023-09-30T22:31:04.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:04.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 Processor -- start 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 -- start start 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa89809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89806bec0 con 0x7fa89809f160 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980a0030 con 0x7fa89809f5b0 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa89d564700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980a0830 con 0x7fa89809e850 2023-09-30T22:31:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa896ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa89809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa8977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa896ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa89809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38090/0 (socket says 172.21.15.32:38090) 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa896ffd700 1 -- 172.21.15.32:0/2122867681 learned_addr learned my addr 172.21.15.32:0/2122867681 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.616+0000 7fa8967fc700 1 --1- 172.21.15.32:0/2122867681 >> v1:172.21.15.32:6789/0 conn(0x7fa89809f160 0x7fa898133c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57520/0 (socket says 172.21.15.32:57520) 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa895ffb700 1 -- 172.21.15.32:0/2122867681 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312108995 0 0) 0x7fa89806bec0 con 0x7fa89809f160 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa896ffd700 1 -- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 msgr2=0x7fa898134370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa896ffd700 1 --2- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898134370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa896ffd700 1 -- 172.21.15.32:0/2122867681 >> v1:172.21.15.32:6789/0 conn(0x7fa89809f160 legacy=0x7fa898133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa896ffd700 1 -- 172.21.15.32:0/2122867681 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89806caa0 con 0x7fa89809e850 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa8977fe700 1 --2- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa896ffd700 1 --2- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa89809ec20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa888009920 tx=0x7fa888009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d6b3ac4e03ab04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa895ffb700 1 -- 172.21.15.32:0/2122867681 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888011830 con 0x7fa89809e850 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa895ffb700 1 -- 172.21.15.32:0/2122867681 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888011990 con 0x7fa89809e850 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.617+0000 7fa895ffb700 1 -- 172.21.15.32:0/2122867681 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888011d20 con 0x7fa89809e850 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 -- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 msgr2=0x7fa89809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 --2- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa89809ec20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa888009920 tx=0x7fa888009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 -- 172.21.15.32:0/2122867681 shutdown_connections 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 --2- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 --2- 172.21.15.32:0/2122867681 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa89809ec20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 -- 172.21.15.32:0/2122867681 >> 172.21.15.32:0/2122867681 conn(0x7fa8980a3f70 msgr2=0x7fa8980a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 -- 172.21.15.32:0/2122867681 shutdown_connections 2023-09-30T22:31:04.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.618+0000 7fa89d564700 1 -- 172.21.15.32:0/2122867681 wait complete. 2023-09-30T22:31:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 Processor -- start 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 -- start start 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa898120450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 0x7fa898120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898121920 con 0x7fa89809f160 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa898123f70 con 0x7fa89809f5b0 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa89d564700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa898122680 con 0x7fa89809e850 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa8977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.619+0000 7fa8967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 0x7fa898120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa898120450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa8977fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898120ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54668/0 (socket says 172.21.15.32:54668) 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa8977fe700 1 -- 172.21.15.32:0/1572846321 learned_addr learned my addr 172.21.15.32:0/1572846321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa8967fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 0x7fa898120990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43222/0 (socket says 172.21.15.32:43222) 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 -- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 msgr2=0x7fa898120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 -- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 msgr2=0x7fa898120990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 0x7fa898120990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 -- 172.21.15.32:0/1572846321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009a70 con 0x7fa89809e850 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa8967fc700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 0x7fa898120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa896ffd700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa898120450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa888016000 tx=0x7fa888004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888004450 con 0x7fa89809e850 2023-09-30T22:31:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88801e070 con 0x7fa89809e850 2023-09-30T22:31:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa888009550 con 0x7fa89809e850 2023-09-30T22:31:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880038e0 con 0x7fa89809e850 2023-09-30T22:31:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.620+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980a3500 con 0x7fa89809e850 2023-09-30T22:31:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.621+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa888003a40 con 0x7fa89809e850 2023-09-30T22:31:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.621+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa8880403d0 con 0x7fa89809e850 2023-09-30T22:31:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.621+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89806cd20 con 0x7fa89809e850 2023-09-30T22:31:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.626+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa88802d7d0 con 0x7fa89809e850 2023-09-30T22:31:04.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.830+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa89809caf0 con 0x7fa89809e850 2023-09-30T22:31:04.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.831+0000 7fa88f7fe700 1 -- 172.21.15.32:0/1572846321 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa888023ba0 con 0x7fa89809e850 2023-09-30T22:31:04.832 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 msgr2=0x7fa898120450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa898120450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa888016000 tx=0x7fa888004750 comp rx=0 tx=0).stop 2023-09-30T22:31:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 shutdown_connections 2023-09-30T22:31:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa89809f5b0 0x7fa898120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa89809e850 0x7fa898120450 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 --2- 172.21.15.32:0/1572846321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa89809f160 0x7fa898120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 >> 172.21.15.32:0/1572846321 conn(0x7fa8980a3f70 msgr2=0x7fa8981bb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 shutdown_connections 2023-09-30T22:31:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:04.833+0000 7fa89d564700 1 -- 172.21.15.32:0/1572846321 wait complete. 2023-09-30T22:31:04.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:05.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:05.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:05.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:05.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.271+0000 7f44f8877700 1 Processor -- start 2023-09-30T22:31:05.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.271+0000 7f44f8877700 1 -- start start 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f8877700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f406a770 0x7f44f406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f8877700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 0x7f44f4139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f8877700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f409ece0 con 0x7f44f406b080 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f8877700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f409e760 con 0x7f44f4131d10 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f8877700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44f409e040 con 0x7f44f406a770 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 0x7f44f4139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 0x7f44f4139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54674/0 (socket says 172.21.15.32:54674) 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 -- 172.21.15.32:0/3100315278 learned_addr learned my addr 172.21.15.32:0/3100315278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:05.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f1d9b700 1 --1- 172.21.15.32:0/3100315278 >> v1:172.21.15.32:6789/0 conn(0x7f44f406b080 0x7f44f4131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57528/0 (socket says 172.21.15.32:57528) 2023-09-30T22:31:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f259c700 1 --2- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f406a770 0x7f44f406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 -- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f406a770 msgr2=0x7f44f406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 --2- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f406a770 0x7f44f406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 -- 172.21.15.32:0/3100315278 >> v1:172.21.15.32:6789/0 conn(0x7f44f406b080 legacy=0x7f44f4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f2d9d700 1 -- 172.21.15.32:0/3100315278 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f409f630 con 0x7f44f4131d10 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.272+0000 7f44f159a700 1 -- 172.21.15.32:0/3100315278 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117160637 0 0) 0x7f44f409ece0 con 0x7f44f406b080 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.273+0000 7f44f2d9d700 1 --2- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 0x7f44f4139860 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009e80 tx=0x7f44ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e7088364263534 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.273+0000 7f44f159a700 1 -- 172.21.15.32:0/3100315278 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00e070 con 0x7f44f4131d10 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.273+0000 7f44f159a700 1 -- 172.21.15.32:0/3100315278 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec00b910 con 0x7f44f4131d10 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.273+0000 7f44f159a700 1 -- 172.21.15.32:0/3100315278 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00bcc0 con 0x7f44f4131d10 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.273+0000 7f44f8877700 1 -- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 msgr2=0x7f44f4139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.273+0000 7f44f8877700 1 --2- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 0x7f44f4139860 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009e80 tx=0x7f44ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.274+0000 7f44f8877700 1 -- 172.21.15.32:0/3100315278 shutdown_connections 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.274+0000 7f44f8877700 1 --2- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f4131d10 0x7f44f4139860 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.274+0000 7f44f8877700 1 --2- 172.21.15.32:0/3100315278 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f406a770 0x7f44f406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.274+0000 7f44f8877700 1 -- 172.21.15.32:0/3100315278 >> 172.21.15.32:0/3100315278 conn(0x7f44f4066f50 msgr2=0x7f44f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.274+0000 7f44f8877700 1 -- 172.21.15.32:0/3100315278 shutdown_connections 2023-09-30T22:31:05.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.274+0000 7f44f8877700 1 -- 172.21.15.32:0/3100315278 wait complete. 2023-09-30T22:31:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.275+0000 7f44f8877700 1 Processor -- start 2023-09-30T22:31:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.275+0000 7f44f8877700 1 -- start start 2023-09-30T22:31:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f8877700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 0x7f44f4120460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f8877700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f406b080 0x7f44f41209a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f8877700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f4131d10 0x7f44f4120ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f8877700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4131420 con 0x7f44f406a770 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f8877700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4131650 con 0x7f44f406b080 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f8877700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4071e60 con 0x7f44f4131d10 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 0x7f44f4120460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f406b080 0x7f44f41209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 0x7f44f4120460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43236/0 (socket says 172.21.15.32:43236) 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f259c700 1 -- 172.21.15.32:0/2588349538 learned_addr learned my addr 172.21.15.32:0/2588349538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.276+0000 7f44f2d9d700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f4131d10 0x7f44f4120ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f259c700 1 -- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f4131d10 msgr2=0x7f44f4120ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f259c700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f4131d10 0x7f44f4120ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f259c700 1 -- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f406b080 msgr2=0x7f44f41209a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f259c700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f406b080 0x7f44f41209a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f259c700 1 -- 172.21.15.32:0/2588349538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009b30 con 0x7f44f406a770 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f2d9d700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f4131d10 0x7f44f4120ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f1d9b700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f406b080 0x7f44f41209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f259c700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 0x7f44f4120460 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f44e400e9b0 tx=0x7f44e400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4009710 con 0x7f44f406a770 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400ee50 con 0x7f44f406a770 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400ca50 con 0x7f44f406a770 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f4123ca0 con 0x7f44f406a770 2023-09-30T22:31:05.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.277+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f4124710 con 0x7f44f406a770 2023-09-30T22:31:05.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.278+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44e400cbb0 con 0x7f44f406a770 2023-09-30T22:31:05.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.278+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f44e403f6e0 con 0x7f44f406a770 2023-09-30T22:31:05.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.279+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f4074ea0 con 0x7f44f406a770 2023-09-30T22:31:05.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.284+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44e40232f0 con 0x7f44f406a770 2023-09-30T22:31:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 Processor -- start 2023-09-30T22:31:05.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 -- start start 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf84133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf84130a40 0x7fdf8413b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8406bc00 con 0x7fdf8412ff20 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8406c570 con 0x7fdf84130a40 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.311+0000 7fdf89fd1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840a0040 con 0x7fdf84133a80 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf82ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdf8412ff20 0x7fdf84130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57530/0 (socket says 172.21.15.32:57530) 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf82ffd700 1 -- 172.21.15.32:0/700127571 learned_addr learned my addr 172.21.15.32:0/700127571 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf837fe700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf84133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf83fff700 1 --2- 172.21.15.32:0/700127571 >> v2:172.21.15.32:3300/0 conn(0x7fdf84130a40 0x7fdf8413b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf83fff700 1 -- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 msgr2=0x7fdf84133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf83fff700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf84133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf83fff700 1 -- 172.21.15.32:0/700127571 >> v1:172.21.15.32:6789/0 conn(0x7fdf8412ff20 legacy=0x7fdf84130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf83fff700 1 -- 172.21.15.32:0/700127571 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84133860 con 0x7fdf84130a40 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf827fc700 1 -- 172.21.15.32:0/700127571 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692625014 0 0) 0x7fdf8406bc00 con 0x7fdf8412ff20 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf837fe700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf84133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.312+0000 7fdf83fff700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8413b850 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009d90 tx=0x7fdf7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b52747ed7747074 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf827fc700 1 -- 172.21.15.32:0/700127571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00e070 con 0x7fdf84130a40 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf827fc700 1 -- 172.21.15.32:0/700127571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c00bb20 con 0x7fdf84130a40 2023-09-30T22:31:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf827fc700 1 -- 172.21.15.32:0/700127571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c012430 con 0x7fdf84130a40 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 -- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 msgr2=0x7fdf8413b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8413b850 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009d90 tx=0x7fdf7c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 -- 172.21.15.32:0/700127571 shutdown_connections 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8413b850 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/700127571 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf84133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 -- 172.21.15.32:0/700127571 >> 172.21.15.32:0/700127571 conn(0x7fdf840a4210 msgr2=0x7fdf840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 -- 172.21.15.32:0/700127571 shutdown_connections 2023-09-30T22:31:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.313+0000 7fdf89fd1700 1 -- 172.21.15.32:0/700127571 wait complete. 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.315+0000 7fdf89fd1700 1 Processor -- start 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.315+0000 7fdf89fd1700 1 -- start start 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf89fd1700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8412ff20 0x7fdf8409c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf89fd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8409c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf89fd1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf8409cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf89fd1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8411ee60 con 0x7fdf84130a40 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf89fd1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8411fff0 con 0x7fdf84133a80 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf89fd1700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf84120f50 con 0x7fdf8412ff20 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8409c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf837fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8412ff20 0x7fdf8409c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8409c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43258/0 (socket says 172.21.15.32:43258) 2023-09-30T22:31:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 -- 172.21.15.32:0/2854970142 learned_addr learned my addr 172.21.15.32:0/2854970142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf83fff700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf8409cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 -- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8412ff20 msgr2=0x7fdf8409c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8412ff20 0x7fdf8409c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 -- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 msgr2=0x7fdf8409cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf8409cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf82ffd700 1 -- 172.21.15.32:0/2854970142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf740094a0 con 0x7fdf84130a40 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.316+0000 7fdf837fe700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8412ff20 0x7fdf8409c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.317+0000 7fdf82ffd700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8409c960 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c570 tx=0x7fdf7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.317+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78004c80 con 0x7fdf84130a40 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.317+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78004de0 con 0x7fdf84130a40 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.317+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c009a70 con 0x7fdf84130a40 2023-09-30T22:31:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.317+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780072f0 con 0x7fdf84130a40 2023-09-30T22:31:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.317+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8412db30 con 0x7fdf84130a40 2023-09-30T22:31:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.318+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf78007450 con 0x7fdf84130a40 2023-09-30T22:31:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.318+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf840a45e0 con 0x7fdf84130a40 2023-09-30T22:31:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.318+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fdf7803f3e0 con 0x7fdf84130a40 2023-09-30T22:31:05.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.327+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf78007700 con 0x7fdf84130a40 2023-09-30T22:31:05.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.345+0000 7fea1084c700 1 Processor -- start 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.345+0000 7fea1084c700 1 -- start start 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea1084c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea0c09e1d0 0x7fea0c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea1084c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c135e50 0x7fea0c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea1084c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0c09f130 con 0x7fea0c09d6f0 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea1084c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c09f520 con 0x7fea0c09e1d0 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea1084c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c09e930 con 0x7fea0c135e50 2023-09-30T22:31:05.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea09d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea0c09e1d0 0x7fea0c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea0a59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea0c09d6f0 0x7fea0c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57546/0 (socket says 172.21.15.32:57546) 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea0a59c700 1 -- 172.21.15.32:0/638354049 learned_addr learned my addr 172.21.15.32:0/638354049 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea0ad9d700 1 --2- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c135e50 0x7fea0c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea09d9b700 1 -- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c135e50 msgr2=0x7fea0c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea09d9b700 1 --2- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c135e50 0x7fea0c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea09d9b700 1 -- 172.21.15.32:0/638354049 >> v1:172.21.15.32:6789/0 conn(0x7fea0c09d6f0 legacy=0x7fea0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.346+0000 7fea0959a700 1 -- 172.21.15.32:0/638354049 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865021713 0 0) 0x7fea0c09f130 con 0x7fea0c09d6f0 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea09d9b700 1 -- 172.21.15.32:0/638354049 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c09b2d0 con 0x7fea0c09e1d0 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea09d9b700 1 --2- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c135910 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fea0000bae0 tx=0x7fea0000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc58866a5e6e469 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea0959a700 1 -- 172.21.15.32:0/638354049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000dc70 con 0x7fea0c09e1d0 2023-09-30T22:31:05.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea0959a700 1 -- 172.21.15.32:0/638354049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0000ddd0 con 0x7fea0c09e1d0 2023-09-30T22:31:05.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea0959a700 1 -- 172.21.15.32:0/638354049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00010690 con 0x7fea0c09e1d0 2023-09-30T22:31:05.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 -- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 msgr2=0x7fea0c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 --2- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c135910 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fea0000bae0 tx=0x7fea0000bdf0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 -- 172.21.15.32:0/638354049 shutdown_connections 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 --2- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c135e50 0x7fea0c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 --2- 172.21.15.32:0/638354049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c135910 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 -- 172.21.15.32:0/638354049 >> 172.21.15.32:0/638354049 conn(0x7fea0c066f50 msgr2=0x7fea0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 -- 172.21.15.32:0/638354049 shutdown_connections 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.348+0000 7fea1084c700 1 -- 172.21.15.32:0/638354049 wait complete. 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.349+0000 7fea1084c700 1 Processor -- start 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.349+0000 7fea1084c700 1 -- start start 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea1084c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c09d6f0 0x7fea0c119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea1084c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea1084c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0c135e50 0x7fea0c119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea1084c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c0721b0 con 0x7fea0c09e1d0 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea1084c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c11b9c0 con 0x7fea0c09d6f0 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea1084c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c09c700 con 0x7fea0c135e50 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c1199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43278/0 (socket says 172.21.15.32:43278) 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 -- 172.21.15.32:0/2819851774 learned_addr learned my addr 172.21.15.32:0/2819851774 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea0ad9d700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0c135e50 0x7fea0c119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea0a59c700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c09d6f0 0x7fea0c119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 -- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0c135e50 msgr2=0x7fea0c119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0c135e50 0x7fea0c119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 -- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c09d6f0 msgr2=0x7fea0c119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c09d6f0 0x7fea0c119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea09d9b700 1 -- 172.21.15.32:0/2819851774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0000b740 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.350+0000 7fea0a59c700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c09d6f0 0x7fea0c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.351+0000 7fea09d9b700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c1199c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fea0000b2c0 tx=0x7fea00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.351+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0001f070 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.351+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00003970 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.351+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c12f220 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.351+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00003d20 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.351+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c071800 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.352+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea00019400 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.352+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fea00042a00 con 0x7fea0c09e1d0 2023-09-30T22:31:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.352+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c074ea0 con 0x7fea0c09e1d0 2023-09-30T22:31:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.357+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea0000f3d0 con 0x7fea0c09e1d0 2023-09-30T22:31:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.498+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f44f406c360 con 0x7f44f406a770 2023-09-30T22:31:05.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.498+0000 7f44e2ffd700 1 -- 172.21.15.32:0/2588349538 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44e401ae80 con 0x7f44f406a770 2023-09-30T22:31:05.498 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.500+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 msgr2=0x7f44f4120460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.500+0000 7f44f8877700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 0x7f44f4120460 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f44e400e9b0 tx=0x7f44e400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.500+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 shutdown_connections 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.500+0000 7f44f8877700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f44f4131d10 0x7f44f4120ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.500+0000 7f44f8877700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f406a770 0x7f44f4120460 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.500+0000 7f44f8877700 1 --2- 172.21.15.32:0/2588349538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f406b080 0x7f44f41209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.501+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 >> 172.21.15.32:0/2588349538 conn(0x7f44f4066f50 msgr2=0x7f44f406ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.501+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 shutdown_connections 2023-09-30T22:31:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.501+0000 7f44f8877700 1 -- 172.21.15.32:0/2588349538 wait complete. 2023-09-30T22:31:05.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.537+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf84074ea0 con 0x7fdf84130a40 2023-09-30T22:31:05.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.537+0000 7fdf6bfff700 1 -- 172.21.15.32:0/2854970142 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf78011360 con 0x7fdf84130a40 2023-09-30T22:31:05.538 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:05.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.539+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 msgr2=0x7fdf8409c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.539+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8409c960 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c570 tx=0x7fdf7800c880 comp rx=0 tx=0).stop 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 shutdown_connections 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf84133a80 0x7fdf8409cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdf8412ff20 0x7fdf8409c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 --2- 172.21.15.32:0/2854970142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf84130a40 0x7fdf8409c960 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 >> 172.21.15.32:0/2854970142 conn(0x7fdf840a4210 msgr2=0x7fdf8409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 shutdown_connections 2023-09-30T22:31:05.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.540+0000 7fdf89fd1700 1 -- 172.21.15.32:0/2854970142 wait complete. 2023-09-30T22:31:05.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.563+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fea0c06c360 con 0x7fea0c09e1d0 2023-09-30T22:31:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.564+0000 7fe9faffd700 1 -- 172.21.15.32:0/2819851774 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea00029020 con 0x7fea0c09e1d0 2023-09-30T22:31:05.564 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 msgr2=0x7fea0c1199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c1199c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fea0000b2c0 tx=0x7fea00004750 comp rx=0 tx=0).stop 2023-09-30T22:31:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 shutdown_connections 2023-09-30T22:31:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fea0c135e50 0x7fea0c119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fea0c09d6f0 0x7fea0c119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 --2- 172.21.15.32:0/2819851774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea0c09e1d0 0x7fea0c1199c0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 >> 172.21.15.32:0/2819851774 conn(0x7fea0c066f50 msgr2=0x7fea0c06f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 shutdown_connections 2023-09-30T22:31:05.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.566+0000 7fea1084c700 1 -- 172.21.15.32:0/2819851774 wait complete. 2023-09-30T22:31:05.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:05.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:05.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.995+0000 7f0e0eabc700 1 Processor -- start 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.995+0000 7f0e0eabc700 1 -- start start 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0eabc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08133a80 0x7f0e08133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0eabc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e08130a40 0x7f0e0813b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0eabc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0806bc00 con 0x7f0e0812ff20 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0eabc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0806c570 con 0x7f0e08130a40 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0eabc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080a0040 con 0x7f0e08133a80 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0d059700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e08130a40 0x7f0e0813b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0c858700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08133a80 0x7f0e08133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e07fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0e0812ff20 0x7f0e08130330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57558/0 (socket says 172.21.15.32:57558) 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e07fff700 1 -- 172.21.15.32:0/3645374156 learned_addr learned my addr 172.21.15.32:0/3645374156 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0d059700 1 -- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08133a80 msgr2=0x7f0e08133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0d059700 1 --2- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08133a80 0x7f0e08133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0d059700 1 -- 172.21.15.32:0/3645374156 >> v1:172.21.15.32:6789/0 conn(0x7f0e0812ff20 legacy=0x7f0e08130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0d059700 1 -- 172.21.15.32:0/3645374156 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08133860 con 0x7f0e08130a40 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.996+0000 7f0e0c858700 1 --2- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08133a80 0x7f0e08133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0d059700 1 --2- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08130a40 0x7f0e0813b850 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009d90 tx=0x7f0e0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fdde97784ee6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e077fe700 1 -- 172.21.15.32:0/3645374156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000e070 con 0x7f0e08130a40 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e077fe700 1 -- 172.21.15.32:0/3645374156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000bb20 con 0x7f0e08130a40 2023-09-30T22:31:05.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e077fe700 1 -- 172.21.15.32:0/3645374156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00012430 con 0x7f0e08130a40 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 -- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08130a40 msgr2=0x7f0e0813b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08130a40 0x7f0e0813b850 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009d90 tx=0x7f0e0000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 -- 172.21.15.32:0/3645374156 shutdown_connections 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08130a40 0x7f0e0813b850 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/3645374156 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08133a80 0x7f0e08133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 -- 172.21.15.32:0/3645374156 >> 172.21.15.32:0/3645374156 conn(0x7f0e080a4210 msgr2=0x7f0e080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 -- 172.21.15.32:0/3645374156 shutdown_connections 2023-09-30T22:31:06.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.997+0000 7f0e0eabc700 1 -- 172.21.15.32:0/3645374156 wait complete. 2023-09-30T22:31:06.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.998+0000 7f0e0eabc700 1 Processor -- start 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.998+0000 7f0e0eabc700 1 -- start start 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0eabc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 0x7f0e0809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0eabc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08130a40 0x7f0e0809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0eabc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08133a80 0x7f0e0809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0eabc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0811ee60 con 0x7f0e08133a80 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0eabc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0811fff0 con 0x7f0e08130a40 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0eabc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08120420 con 0x7f0e0812ff20 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0c858700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 0x7f0e0809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0d059700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08133a80 0x7f0e0809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0c858700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 0x7f0e0809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38146/0 (socket says 172.21.15.32:38146) 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e0c858700 1 -- 172.21.15.32:0/2859905961 learned_addr learned my addr 172.21.15.32:0/2859905961 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:05.999+0000 7f0e07fff700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08130a40 0x7f0e0809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0d059700 1 -- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 msgr2=0x7f0e0809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0d059700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 0x7f0e0809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0d059700 1 -- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08130a40 msgr2=0x7f0e0809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0d059700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08130a40 0x7f0e0809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0d059700 1 -- 172.21.15.32:0/2859905961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0094a0 con 0x7f0e08133a80 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0c858700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 0x7f0e0809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0d059700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08133a80 0x7f0e0809cf00 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f0e00016000 tx=0x7f0e00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00012430 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e00020070 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e00009a70 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000098f0 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.000+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e080a20c0 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.001+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e00004040 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.001+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0e00022070 con 0x7f0e08133a80 2023-09-30T22:31:06.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.001+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e08120230 con 0x7f0e08133a80 2023-09-30T22:31:06.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.006+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e000269d0 con 0x7f0e08133a80 2023-09-30T22:31:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.214+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e08074ea0 con 0x7f0e08133a80 2023-09-30T22:31:06.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.214+0000 7f0e057fa700 1 -- 172.21.15.32:0/2859905961 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e00041080 con 0x7f0e08133a80 2023-09-30T22:31:06.214 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:06.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08133a80 msgr2=0x7f0e0809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08133a80 0x7f0e0809cf00 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f0e00016000 tx=0x7f0e00004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 shutdown_connections 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08133a80 0x7f0e0809cf00 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0e0812ff20 0x7f0e0809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 --2- 172.21.15.32:0/2859905961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e08130a40 0x7f0e0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.216+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 >> 172.21.15.32:0/2859905961 conn(0x7f0e080a4210 msgr2=0x7f0e0809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.217+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 shutdown_connections 2023-09-30T22:31:06.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.217+0000 7f0e0eabc700 1 -- 172.21.15.32:0/2859905961 wait complete. 2023-09-30T22:31:06.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:06.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:06.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:06.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.674+0000 7f4be978f700 1 Processor -- start 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.674+0000 7f4be978f700 1 -- start start 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be978f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4131810 0x7f4be4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be978f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 0x7f4be4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be978f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be409a850 con 0x7f4be409a3b0 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be978f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be409b580 con 0x7f4be4099a40 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be978f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41313c0 con 0x7f4be4131810 2023-09-30T22:31:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 0x7f4be4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4131810 0x7f4be4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be27fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 0x7f4be4099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43304/0 (socket says 172.21.15.32:43304) 2023-09-30T22:31:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be27fc700 1 -- 172.21.15.32:0/3297971010 learned_addr learned my addr 172.21.15.32:0/3297971010 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be37fe700 1 --1- 172.21.15.32:0/3297971010 >> v1:172.21.15.32:6789/0 conn(0x7f4be409a3b0 0x7f4be4138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57562/0 (socket says 172.21.15.32:57562) 2023-09-30T22:31:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be2ffd700 1 -- 172.21.15.32:0/3297971010 >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 msgr2=0x7f4be4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be2ffd700 1 --2- 172.21.15.32:0/3297971010 >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 0x7f4be4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be2ffd700 1 -- 172.21.15.32:0/3297971010 >> v1:172.21.15.32:6789/0 conn(0x7f4be409a3b0 legacy=0x7f4be4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be2ffd700 1 -- 172.21.15.32:0/3297971010 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be406e8c0 con 0x7f4be4131810 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be27fc700 1 --2- 172.21.15.32:0/3297971010 >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 0x7f4be4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.675+0000 7f4be2ffd700 1 --2- 172.21.15.32:0/3297971010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4131810 0x7f4be4131be0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009870 tx=0x7f4bd4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e427d95e57c3ab56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.676+0000 7f4be1ffb700 1 -- 172.21.15.32:0/3297971010 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4010a90 con 0x7f4be4131810 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.676+0000 7f4be1ffb700 1 -- 172.21.15.32:0/3297971010 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd4010bf0 con 0x7f4be4131810 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.676+0000 7f4be1ffb700 1 -- 172.21.15.32:0/3297971010 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4018440 con 0x7f4be4131810 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 -- 172.21.15.32:0/3297971010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4131810 msgr2=0x7f4be4131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 --2- 172.21.15.32:0/3297971010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4131810 0x7f4be4131be0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009870 tx=0x7f4bd4009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 -- 172.21.15.32:0/3297971010 shutdown_connections 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 --2- 172.21.15.32:0/3297971010 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4131810 0x7f4be4131be0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 --2- 172.21.15.32:0/3297971010 >> v2:172.21.15.32:3300/0 conn(0x7f4be4099a40 0x7f4be4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 -- 172.21.15.32:0/3297971010 >> 172.21.15.32:0/3297971010 conn(0x7f4be40a2aa0 msgr2=0x7f4be40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 -- 172.21.15.32:0/3297971010 shutdown_connections 2023-09-30T22:31:06.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.677+0000 7f4be978f700 1 -- 172.21.15.32:0/3297971010 wait complete. 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 Processor -- start 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 -- start start 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4099a40 0x7f4be4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 0x7f4be4119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4be4131810 0x7f4be4119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40a1c60 con 0x7f4be409a3b0 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be411a3b0 con 0x7f4be4099a40 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.679+0000 7f4be978f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be411bf40 con 0x7f4be4131810 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 0x7f4be4119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 0x7f4be4119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43318/0 (socket says 172.21.15.32:43318) 2023-09-30T22:31:06.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 -- 172.21.15.32:0/2243853981 learned_addr learned my addr 172.21.15.32:0/2243853981 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be37fe700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4be4131810 0x7f4be4119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be2ffd700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4099a40 0x7f4be4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 -- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4be4131810 msgr2=0x7f4be4119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4be4131810 0x7f4be4119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 -- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4099a40 msgr2=0x7f4be4118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4099a40 0x7f4be4118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 -- 172.21.15.32:0/2243853981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8009550 con 0x7f4be409a3b0 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be27fc700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 0x7f4be4119210 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800ebe0 tx=0x7f4bd800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800c850 con 0x7f4be409a3b0 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800c9b0 con 0x7f4be409a3b0 2023-09-30T22:31:06.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.680+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd40094a0 con 0x7f4be409a3b0 2023-09-30T22:31:06.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.681+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800cd60 con 0x7f4be409a3b0 2023-09-30T22:31:06.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.681+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be411b5b0 con 0x7f4be409a3b0 2023-09-30T22:31:06.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.681+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bd8013470 con 0x7f4be409a3b0 2023-09-30T22:31:06.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.682+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be411c340 con 0x7f4be409a3b0 2023-09-30T22:31:06.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.682+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4bd803fcb0 con 0x7f4be409a3b0 2023-09-30T22:31:06.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.687+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bd801b3b0 con 0x7f4be409a3b0 2023-09-30T22:31:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.704+0000 7f22cb6cb700 1 Processor -- start 2023-09-30T22:31:06.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.705+0000 7f22cb6cb700 1 -- start start 2023-09-30T22:31:06.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.705+0000 7f22cb6cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c41305f0 0x7f22c41309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.705+0000 7f22cb6cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c4070160 0x7f22c4138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.705+0000 7f22cb6cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c41301c0 con 0x7f22c406f660 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.705+0000 7f22cb6cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4071ff0 con 0x7f22c4070160 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.705+0000 7f22cb6cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4070e70 con 0x7f22c41305f0 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c8c66700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f22c406f660 0x7f22c406fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57578/0 (socket says 172.21.15.32:57578) 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c8c66700 1 -- 172.21.15.32:0/2230816142 learned_addr learned my addr 172.21.15.32:0/2230816142 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9c68700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c4070160 0x7f22c4138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9467700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c41305f0 0x7f22c41309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22bbfff700 1 -- 172.21.15.32:0/2230816142 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057416133 0 0) 0x7f22c41301c0 con 0x7f22c406f660 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22bbfff700 1 -- 172.21.15.32:0/2230816142 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b00035b0 con 0x7f22c406f660 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9467700 1 -- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c4070160 msgr2=0x7f22c4138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9467700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c4070160 0x7f22c4138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9467700 1 -- 172.21.15.32:0/2230816142 >> v1:172.21.15.32:6789/0 conn(0x7f22c406f660 legacy=0x7f22c406fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9467700 1 -- 172.21.15.32:0/2230816142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c40722b0 con 0x7f22c41305f0 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22c9467700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c41305f0 0x7f22c41309c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009870 tx=0x7f22c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8ec407ef957351 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.706+0000 7f22bbfff700 1 -- 172.21.15.32:0/2230816142 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0004230 con 0x7f22c41305f0 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22bbfff700 1 -- 172.21.15.32:0/2230816142 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c0004390 con 0x7f22c41305f0 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22bbfff700 1 -- 172.21.15.32:0/2230816142 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00036a0 con 0x7f22c41305f0 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 -- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c41305f0 msgr2=0x7f22c41309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c41305f0 0x7f22c41309c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009870 tx=0x7f22c0009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 -- 172.21.15.32:0/2230816142 shutdown_connections 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c4070160 0x7f22c4138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/2230816142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c41305f0 0x7f22c41309c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 -- 172.21.15.32:0/2230816142 >> 172.21.15.32:0/2230816142 conn(0x7f22c40a2aa0 msgr2=0x7f22c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 -- 172.21.15.32:0/2230816142 shutdown_connections 2023-09-30T22:31:06.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.707+0000 7f22cb6cb700 1 -- 172.21.15.32:0/2230816142 wait complete. 2023-09-30T22:31:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.708+0000 7f22cb6cb700 1 Processor -- start 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 -- start start 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 0x7f22c4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c4070160 0x7f22c4119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c41305f0 0x7f22c4119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c411a930 con 0x7f22c406f660 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4071820 con 0x7f22c41305f0 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22cb6cb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22c411a140 con 0x7f22c4070160 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 0x7f22c4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c8c66700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c4070160 0x7f22c4119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 0x7f22c4118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43320/0 (socket says 172.21.15.32:43320) 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 -- 172.21.15.32:0/3102155537 learned_addr learned my addr 172.21.15.32:0/3102155537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9c68700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c41305f0 0x7f22c4119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 -- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c4070160 msgr2=0x7f22c4119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c4070160 0x7f22c4119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 -- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c41305f0 msgr2=0x7f22c4119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c41305f0 0x7f22c4119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.709+0000 7f22c9467700 1 -- 172.21.15.32:0/3102155537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c00094a0 con 0x7f22c406f660 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.710+0000 7f22c9467700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 0x7f22c4118cd0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f22c00090b0 tx=0x7f22c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.710+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0003ae0 con 0x7f22c406f660 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.710+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c001f070 con 0x7f22c406f660 2023-09-30T22:31:06.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.710+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c411ba80 con 0x7f22c406f660 2023-09-30T22:31:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.710+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0003ea0 con 0x7f22c406f660 2023-09-30T22:31:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.710+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c411aea0 con 0x7f22c406f660 2023-09-30T22:31:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.711+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c4074ea0 con 0x7f22c406f660 2023-09-30T22:31:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.712+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22c0021030 con 0x7f22c406f660 2023-09-30T22:31:06.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.712+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f22c00407f0 con 0x7f22c406f660 2023-09-30T22:31:06.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.717+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22c0023790 con 0x7f22c406f660 2023-09-30T22:31:06.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.756+0000 7fefdf19e700 1 Processor -- start 2023-09-30T22:31:06.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.756+0000 7fefdf19e700 1 -- start start 2023-09-30T22:31:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefdf19e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 0x7fefd80a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefdf19e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefd80a1110 0x7fefd8133780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefdf19e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd809e770 con 0x7fefd80a1520 2023-09-30T22:31:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefdf19e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd80a1aa0 con 0x7fefd80a1110 2023-09-30T22:31:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefdf19e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd809cc80 con 0x7fefd80a0800 2023-09-30T22:31:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 0x7fefd80a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 0x7fefd80a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54778/0 (socket says 172.21.15.32:54778) 2023-09-30T22:31:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 -- 172.21.15.32:0/1029389507 learned_addr learned my addr 172.21.15.32:0/1029389507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefdd99b700 1 --2- 172.21.15.32:0/1029389507 >> v2:172.21.15.32:3300/0 conn(0x7fefd80a1110 0x7fefd8133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 -- 172.21.15.32:0/1029389507 >> v2:172.21.15.32:3300/0 conn(0x7fefd80a1110 msgr2=0x7fefd8133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 --2- 172.21.15.32:0/1029389507 >> v2:172.21.15.32:3300/0 conn(0x7fefd80a1110 0x7fefd8133780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 -- 172.21.15.32:0/1029389507 >> v1:172.21.15.32:6789/0 conn(0x7fefd80a1520 legacy=0x7fefd8133cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.757+0000 7fefde19c700 1 -- 172.21.15.32:0/1029389507 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80a1c70 con 0x7fefd80a0800 2023-09-30T22:31:06.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.758+0000 7fefde19c700 1 --2- 172.21.15.32:0/1029389507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 0x7fefd80a0bd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009920 tx=0x7fefd4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b08d6e576863e48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.758+0000 7fefdd19a700 1 -- 172.21.15.32:0/1029389507 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011830 con 0x7fefd80a0800 2023-09-30T22:31:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.758+0000 7fefdd19a700 1 -- 172.21.15.32:0/1029389507 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4011990 con 0x7fefd80a0800 2023-09-30T22:31:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.758+0000 7fefdd19a700 1 -- 172.21.15.32:0/1029389507 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011d20 con 0x7fefd80a0800 2023-09-30T22:31:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 -- 172.21.15.32:0/1029389507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 msgr2=0x7fefd80a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 --2- 172.21.15.32:0/1029389507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 0x7fefd80a0bd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009920 tx=0x7fefd4009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:06.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 -- 172.21.15.32:0/1029389507 shutdown_connections 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 --2- 172.21.15.32:0/1029389507 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a0800 0x7fefd80a0bd0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 --2- 172.21.15.32:0/1029389507 >> v2:172.21.15.32:3300/0 conn(0x7fefd80a1110 0x7fefd8133780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 -- 172.21.15.32:0/1029389507 >> 172.21.15.32:0/1029389507 conn(0x7fefd80a6140 msgr2=0x7fefd80a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 -- 172.21.15.32:0/1029389507 shutdown_connections 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.759+0000 7fefdf19e700 1 -- 172.21.15.32:0/1029389507 wait complete. 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.760+0000 7fefdf19e700 1 Processor -- start 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.760+0000 7fefdf19e700 1 -- start start 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdf19e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 0x7fefd8120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdf19e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd80a1110 0x7fefd8121410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdf19e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a1520 0x7fefd8121950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdf19e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8124610 con 0x7fefd80a0800 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdf19e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80a5100 con 0x7fefd80a1520 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdf19e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8122af0 con 0x7fefd80a1110 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefde19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 0x7fefd8120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd80a1110 0x7fefd8121410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefde19c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 0x7fefd8120ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43346/0 (socket says 172.21.15.32:43346) 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefde19c700 1 -- 172.21.15.32:0/3269429838 learned_addr learned my addr 172.21.15.32:0/3269429838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefde99d700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a1520 0x7fefd8121950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 -- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a1520 msgr2=0x7fefd8121950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a1520 0x7fefd8121950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 -- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 msgr2=0x7fefd8120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 0x7fefd8120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 -- 172.21.15.32:0/3269429838 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009550 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefde19c700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 0x7fefd8120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefde99d700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a1520 0x7fefd8121950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefdd99b700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd80a1110 0x7fefd8121410 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fefc800e900 tx=0x7fefc800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.761+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc800ee50 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.762+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc800c850 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.762+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd809cec0 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.762+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc800cbc0 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.762+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd81248f0 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.762+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefc800cd20 con 0x7fefd80a1110 2023-09-30T22:31:06.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.763+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fefc803fb20 con 0x7fefd80a1110 2023-09-30T22:31:06.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.763+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd8124750 con 0x7fefd80a1110 2023-09-30T22:31:06.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.772+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefd8124750 con 0x7fefd80a1110 2023-09-30T22:31:06.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.904+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4be4071db0 con 0x7f4be409a3b0 2023-09-30T22:31:06.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.904+0000 7f4bcb7fe700 1 -- 172.21.15.32:0/2243853981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bd8021030 con 0x7f4be409a3b0 2023-09-30T22:31:06.904 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 msgr2=0x7f4be4119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 0x7f4be4119210 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800ebe0 tx=0x7f4bd800eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 shutdown_connections 2023-09-30T22:31:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4be4131810 0x7f4be4119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4be4099a40 0x7f4be4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 --2- 172.21.15.32:0/2243853981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4be409a3b0 0x7f4be4119210 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 >> 172.21.15.32:0/2243853981 conn(0x7f4be40a2aa0 msgr2=0x7f4be412e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 shutdown_connections 2023-09-30T22:31:06.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.907+0000 7f4be978f700 1 -- 172.21.15.32:0/2243853981 wait complete. 2023-09-30T22:31:06.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:06.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.936+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f22c4074ea0 con 0x7f22c406f660 2023-09-30T22:31:06.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.937+0000 7f22b9ffb700 1 -- 172.21.15.32:0/3102155537 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22c00233d0 con 0x7f22c406f660 2023-09-30T22:31:06.937 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 msgr2=0x7f22c4118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 0x7f22c4118cd0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f22c00090b0 tx=0x7f22c0004750 comp rx=0 tx=0).stop 2023-09-30T22:31:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 shutdown_connections 2023-09-30T22:31:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f22c41305f0 0x7f22c4119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22c406f660 0x7f22c4118cd0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 --2- 172.21.15.32:0/3102155537 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f22c4070160 0x7f22c4119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 >> 172.21.15.32:0/3102155537 conn(0x7f22c40a2aa0 msgr2=0x7f22c412d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 shutdown_connections 2023-09-30T22:31:06.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.939+0000 7f22cb6cb700 1 -- 172.21.15.32:0/3102155537 wait complete. 2023-09-30T22:31:06.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.990+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fefd8124750 con 0x7fefd80a1110 2023-09-30T22:31:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.991+0000 7fefceffd700 1 -- 172.21.15.32:0/3269429838 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefd8124750 con 0x7fefd80a1110 2023-09-30T22:31:06.991 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd80a1110 msgr2=0x7fefd8121410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd80a1110 0x7fefd8121410 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fefc800e900 tx=0x7fefc800ec10 comp rx=0 tx=0).stop 2023-09-30T22:31:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 shutdown_connections 2023-09-30T22:31:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fefd80a1520 0x7fefd8121950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd80a0800 0x7fefd8120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 --2- 172.21.15.32:0/3269429838 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fefd80a1110 0x7fefd8121410 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 >> 172.21.15.32:0/3269429838 conn(0x7fefd80a6140 msgr2=0x7fefd81bad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 shutdown_connections 2023-09-30T22:31:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:06.993+0000 7fefdf19e700 1 -- 172.21.15.32:0/3269429838 wait complete. 2023-09-30T22:31:07.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:07.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:07.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.378+0000 7fcc52821700 1 Processor -- start 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.378+0000 7fcc52821700 1 -- start start 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.378+0000 7fcc52821700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c133420 0x7fcc4c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc52821700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc4c134260 0x7fcc4c1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc52821700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c06b990 con 0x7fcc4c133e30 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc52821700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0a0590 con 0x7fcc4c134260 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc52821700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c09fb80 con 0x7fcc4c133420 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc50dbe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc4c134260 0x7fcc4c1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c133420 0x7fcc4c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc4c133e30 0x7fcc4c12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57590/0 (socket says 172.21.15.32:57590) 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4b7fe700 1 -- 172.21.15.32:0/1134058395 learned_addr learned my addr 172.21.15.32:0/1134058395 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc50dbe700 1 -- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c133420 msgr2=0x7fcc4c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc50dbe700 1 --2- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c133420 0x7fcc4c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc50dbe700 1 -- 172.21.15.32:0/1134058395 >> v1:172.21.15.32:6789/0 conn(0x7fcc4c133e30 legacy=0x7fcc4c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc50dbe700 1 -- 172.21.15.32:0/1134058395 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0a0030 con 0x7fcc4c134260 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4affd700 1 -- 172.21.15.32:0/1134058395 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685431680 0 0) 0x7fcc4c06b990 con 0x7fcc4c133e30 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4bfff700 1 --2- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c133420 0x7fcc4c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc50dbe700 1 --2- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c134260 0x7fcc4c1306a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009d90 tx=0x7fcc4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b803c9d3243ae7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4affd700 1 -- 172.21.15.32:0/1134058395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4400e070 con 0x7fcc4c134260 2023-09-30T22:31:07.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.379+0000 7fcc4affd700 1 -- 172.21.15.32:0/1134058395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4400bb20 con 0x7fcc4c134260 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc4affd700 1 -- 172.21.15.32:0/1134058395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44012430 con 0x7fcc4c134260 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 -- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c134260 msgr2=0x7fcc4c1306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 --2- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c134260 0x7fcc4c1306a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009d90 tx=0x7fcc4400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 -- 172.21.15.32:0/1134058395 shutdown_connections 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 --2- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c134260 0x7fcc4c1306a0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 --2- 172.21.15.32:0/1134058395 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c133420 0x7fcc4c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 -- 172.21.15.32:0/1134058395 >> 172.21.15.32:0/1134058395 conn(0x7fcc4c0a4210 msgr2=0x7fcc4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 -- 172.21.15.32:0/1134058395 shutdown_connections 2023-09-30T22:31:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.380+0000 7fcc52821700 1 -- 172.21.15.32:0/1134058395 wait complete. 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.381+0000 7fcc52821700 1 Processor -- start 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.381+0000 7fcc52821700 1 -- start start 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc52821700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 0x7fcc4c0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc52821700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc4c133e30 0x7fcc4c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc52821700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c134260 0x7fcc4c071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc52821700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c11faf0 con 0x7fcc4c133420 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc52821700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c11ed80 con 0x7fcc4c133e30 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc52821700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c120090 con 0x7fcc4c134260 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 0x7fcc4c0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc4c133e30 0x7fcc4c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 0x7fcc4c0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43360/0 (socket says 172.21.15.32:43360) 2023-09-30T22:31:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 -- 172.21.15.32:0/2928056919 learned_addr learned my addr 172.21.15.32:0/2928056919 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc50dbe700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c134260 0x7fcc4c071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 -- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c134260 msgr2=0x7fcc4c071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c134260 0x7fcc4c071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 -- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc4c133e30 msgr2=0x7fcc4c070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc4c133e30 0x7fcc4c070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4bfff700 1 -- 172.21.15.32:0/2928056919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0094a0 con 0x7fcc4c133420 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.382+0000 7fcc4b7fe700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc4c133e30 0x7fcc4c070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.383+0000 7fcc4bfff700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 0x7fcc4c0709a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00ebe0 tx=0x7fcc3c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.383+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00c850 con 0x7fcc4c133420 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.383+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c00c9b0 con 0x7fcc4c133420 2023-09-30T22:31:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.383+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc44009a70 con 0x7fcc4c133420 2023-09-30T22:31:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.383+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00cd40 con 0x7fcc4c133420 2023-09-30T22:31:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.383+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c072350 con 0x7fcc4c133420 2023-09-30T22:31:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.384+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc3c013470 con 0x7fcc4c133420 2023-09-30T22:31:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.384+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcc3c03f730 con 0x7fcc4c133420 2023-09-30T22:31:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.384+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c11f3f0 con 0x7fcc4c133420 2023-09-30T22:31:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.389+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc3c013720 con 0x7fcc4c133420 2023-09-30T22:31:07.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.592+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc4c074ea0 con 0x7fcc4c133420 2023-09-30T22:31:07.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.593+0000 7fcc48ff9700 1 -- 172.21.15.32:0/2928056919 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc3c021020 con 0x7fcc4c133420 2023-09-30T22:31:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 msgr2=0x7fcc4c0709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 0x7fcc4c0709a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00ebe0 tx=0x7fcc3c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 shutdown_connections 2023-09-30T22:31:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcc4c134260 0x7fcc4c071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc4c133420 0x7fcc4c0709a0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 --2- 172.21.15.32:0/2928056919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc4c133e30 0x7fcc4c070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 >> 172.21.15.32:0/2928056919 conn(0x7fcc4c0a4210 msgr2=0x7fcc4c130330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 shutdown_connections 2023-09-30T22:31:07.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:07.595+0000 7fcc52821700 1 -- 172.21.15.32:0/2928056919 wait complete. 2023-09-30T22:31:07.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:07.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:07.954 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:08.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:08.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.073+0000 7f083f764700 1 Processor -- start 2023-09-30T22:31:08.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.073+0000 7f083f764700 1 -- start start 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083f764700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806b250 0x7f0838136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083f764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0838136d80 0x7f0838139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083f764700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083809ece0 con 0x7f083806a770 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083f764700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083806b640 con 0x7f0838136d80 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083f764700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f083809e760 con 0x7f083806b250 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083dd01700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0838136d80 0x7f0838139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083d500700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f083806a770 0x7f083806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57606/0 (socket says 172.21.15.32:57606) 2023-09-30T22:31:08.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083d500700 1 -- 172.21.15.32:0/3961876241 learned_addr learned my addr 172.21.15.32:0/3961876241 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083ccff700 1 --2- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806b250 0x7f0838136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083dd01700 1 -- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806b250 msgr2=0x7f0838136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083dd01700 1 --2- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806b250 0x7f0838136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083dd01700 1 -- 172.21.15.32:0/3961876241 >> v1:172.21.15.32:6789/0 conn(0x7f083806a770 legacy=0x7f083806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083dd01700 1 -- 172.21.15.32:0/3961876241 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083809e040 con 0x7f0838136d80 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.074+0000 7f083dd01700 1 --2- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0838136d80 0x7f0838139160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0830009dc0 tx=0x7f083000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=971dc854ec21ce1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f082ffff700 1 -- 172.21.15.32:0/3961876241 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083000e070 con 0x7f0838136d80 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f082ffff700 1 -- 172.21.15.32:0/3961876241 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083000bb20 con 0x7f0838136d80 2023-09-30T22:31:08.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f082ffff700 1 -- 172.21.15.32:0/3961876241 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830012430 con 0x7f0838136d80 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f083f764700 1 -- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0838136d80 msgr2=0x7f0838139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f083f764700 1 --2- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0838136d80 0x7f0838139160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0830009dc0 tx=0x7f083000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f083f764700 1 -- 172.21.15.32:0/3961876241 shutdown_connections 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f083f764700 1 --2- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0838136d80 0x7f0838139160 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f083f764700 1 --2- 172.21.15.32:0/3961876241 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806b250 0x7f0838136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.075+0000 7f083f764700 1 -- 172.21.15.32:0/3961876241 >> 172.21.15.32:0/3961876241 conn(0x7f0838066f50 msgr2=0x7f083809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.076+0000 7f083f764700 1 -- 172.21.15.32:0/3961876241 shutdown_connections 2023-09-30T22:31:08.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.076+0000 7f083f764700 1 -- 172.21.15.32:0/3961876241 wait complete. 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 Processor -- start 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 -- start start 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806a770 0x7f083809ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 0x7f083809b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0838136d80 0x7f08381c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083809a9b0 con 0x7f0838136d80 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0838122070 con 0x7f083806b250 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083f764700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f083809b7f0 con 0x7f083806a770 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083ccff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 0x7f083809b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083ccff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 0x7f083809b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54826/0 (socket says 172.21.15.32:54826) 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083ccff700 1 -- 172.21.15.32:0/2766340864 learned_addr learned my addr 172.21.15.32:0/2766340864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:08.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.077+0000 7f083dd01700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0838136d80 0x7f08381c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083ccff700 1 -- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806a770 msgr2=0x7f083809ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083ccff700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806a770 0x7f083809ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083ccff700 1 -- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0838136d80 msgr2=0x7f08381c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083ccff700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0838136d80 0x7f08381c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083ccff700 1 -- 172.21.15.32:0/2766340864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0830009a70 con 0x7f083806b250 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083ccff700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 0x7f083809b2b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f082800ea10 tx=0x7f082800ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828009710 con 0x7f083806b250 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082800c850 con 0x7f083806b250 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0838132360 con 0x7f083806b250 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800cbc0 con 0x7f083806b250 2023-09-30T22:31:08.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.078+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0838121330 con 0x7f083806b250 2023-09-30T22:31:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.079+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f082800cd20 con 0x7f083806b250 2023-09-30T22:31:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.080+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0838074ea0 con 0x7f083806b250 2023-09-30T22:31:08.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.080+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f082803fc20 con 0x7f083806b250 2023-09-30T22:31:08.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.085+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0828017070 con 0x7f083806b250 2023-09-30T22:31:08.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.110+0000 7f21b5d79700 1 Processor -- start 2023-09-30T22:31:08.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.110+0000 7f21b5d79700 1 -- start start 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21b5d79700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21b00a0a30 0x7f21b00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21b5d79700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21b5d79700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b009edd0 con 0x7f21b00a1340 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21b5d79700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b009e1f0 con 0x7f21b00a0a30 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21b5d79700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21b009dc50 con 0x7f21b0134170 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21affff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21a6ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21b00a1340 0x7f21b0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57612/0 (socket says 172.21.15.32:57612) 2023-09-30T22:31:08.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21a6ffd700 1 -- 172.21.15.32:0/793739465 learned_addr learned my addr 172.21.15.32:0/793739465 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21af7fe700 1 --2- 172.21.15.32:0/793739465 >> v2:172.21.15.32:3300/0 conn(0x7f21b00a0a30 0x7f21b00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21affff700 1 -- 172.21.15.32:0/793739465 >> v2:172.21.15.32:3300/0 conn(0x7f21b00a0a30 msgr2=0x7f21b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21affff700 1 --2- 172.21.15.32:0/793739465 >> v2:172.21.15.32:3300/0 conn(0x7f21b00a0a30 0x7f21b00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21affff700 1 -- 172.21.15.32:0/793739465 >> v1:172.21.15.32:6789/0 conn(0x7f21b00a1340 legacy=0x7f21b0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21affff700 1 -- 172.21.15.32:0/793739465 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b00a1fe0 con 0x7f21b0134170 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.111+0000 7f21af7fe700 1 --2- 172.21.15.32:0/793739465 >> v2:172.21.15.32:3300/0 conn(0x7f21b00a0a30 0x7f21b00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.112+0000 7f21affff700 1 --2- 172.21.15.32:0/793739465 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b013bd30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009d90 tx=0x7f21a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac50b12b99ff3704 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.112+0000 7f21aeffd700 1 -- 172.21.15.32:0/793739465 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800e070 con 0x7f21b0134170 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.112+0000 7f21aeffd700 1 -- 172.21.15.32:0/793739465 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a800bb20 con 0x7f21b0134170 2023-09-30T22:31:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.112+0000 7f21aeffd700 1 -- 172.21.15.32:0/793739465 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8012430 con 0x7f21b0134170 2023-09-30T22:31:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 -- 172.21.15.32:0/793739465 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 msgr2=0x7f21b013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 --2- 172.21.15.32:0/793739465 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b013bd30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009d90 tx=0x7f21a800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 -- 172.21.15.32:0/793739465 shutdown_connections 2023-09-30T22:31:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 --2- 172.21.15.32:0/793739465 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b013bd30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 --2- 172.21.15.32:0/793739465 >> v2:172.21.15.32:3300/0 conn(0x7f21b00a0a30 0x7f21b00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 -- 172.21.15.32:0/793739465 >> 172.21.15.32:0/793739465 conn(0x7f21b00a6550 msgr2=0x7f21b00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 -- 172.21.15.32:0/793739465 shutdown_connections 2023-09-30T22:31:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.113+0000 7f21b5d79700 1 -- 172.21.15.32:0/793739465 wait complete. 2023-09-30T22:31:08.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.114+0000 7f21b5d79700 1 Processor -- start 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.114+0000 7f21b5d79700 1 -- start start 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21b5d79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21b00a0a30 0x7f21b011e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21b5d79700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 0x7f21b011ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21b5d79700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b011f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21b5d79700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b011fbe0 con 0x7f21b00a0a30 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21b5d79700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b01221a0 con 0x7f21b00a1340 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21b5d79700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0120290 con 0x7f21b0134170 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 0x7f21b011ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 0x7f21b011ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54836/0 (socket says 172.21.15.32:54836) 2023-09-30T22:31:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 -- 172.21.15.32:0/1361902093 learned_addr learned my addr 172.21.15.32:0/1361902093 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21affff700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b011f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 -- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 msgr2=0x7f21b011f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b011f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 -- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21b00a0a30 msgr2=0x7f21b011e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21b00a0a30 0x7f21b011e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21a6ffd700 1 -- 172.21.15.32:0/1361902093 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a00094a0 con 0x7f21b00a1340 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.115+0000 7f21affff700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b011f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.116+0000 7f21a6ffd700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 0x7f21b011ec50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f219800c520 tx=0x7f219800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.116+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198005d70 con 0x7f21b00a1340 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.116+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21980070d0 con 0x7f21b00a1340 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.116+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198007440 con 0x7f21b00a1340 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.116+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a8009a70 con 0x7f21b00a1340 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.116+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b00a25e0 con 0x7f21b00a1340 2023-09-30T22:31:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.117+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b0074ea0 con 0x7f21b00a1340 2023-09-30T22:31:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.118+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21980075a0 con 0x7f21b00a1340 2023-09-30T22:31:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.118+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f219803f060 con 0x7f21b00a1340 2023-09-30T22:31:08.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.122+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2198019b40 con 0x7f21b00a1340 2023-09-30T22:31:08.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.155+0000 7f0a548fd700 1 Processor -- start 2023-09-30T22:31:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.155+0000 7f0a548fd700 1 -- start start 2023-09-30T22:31:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a548fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a50133420 0x7f0a501337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a548fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50133e30 0x7f0a5012ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a548fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5006b990 con 0x7f0a50134260 2023-09-30T22:31:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a548fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a500a0590 con 0x7f0a50133420 2023-09-30T22:31:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a548fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5009fb80 con 0x7f0a50133e30 2023-09-30T22:31:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a4ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a50134260 0x7f0a501304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57614/0 (socket says 172.21.15.32:57614) 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a4ed9d700 1 -- 172.21.15.32:0/2106146360 learned_addr learned my addr 172.21.15.32:0/2106146360 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a4dd9b700 1 --2- 172.21.15.32:0/2106146360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50133e30 0x7f0a5012ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.156+0000 7f0a4e59c700 1 --2- 172.21.15.32:0/2106146360 >> v2:172.21.15.32:3300/0 conn(0x7f0a50133420 0x7f0a501337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4d59a700 1 -- 172.21.15.32:0/2106146360 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880277235 0 0) 0x7f0a5006b990 con 0x7f0a50134260 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/2106146360 >> v2:172.21.15.32:3300/0 conn(0x7f0a50133420 msgr2=0x7f0a501337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4dd9b700 1 --2- 172.21.15.32:0/2106146360 >> v2:172.21.15.32:3300/0 conn(0x7f0a50133420 0x7f0a501337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/2106146360 >> v1:172.21.15.32:6789/0 conn(0x7f0a50134260 legacy=0x7f0a501304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/2106146360 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500a0030 con 0x7f0a50133e30 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4dd9b700 1 --2- 172.21.15.32:0/2106146360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50133e30 0x7f0a5012ff90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009840 tx=0x7f0a44009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da87b7af9e3d9986 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4d59a700 1 -- 172.21.15.32:0/2106146360 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011830 con 0x7f0a50133e30 2023-09-30T22:31:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4d59a700 1 -- 172.21.15.32:0/2106146360 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a44011990 con 0x7f0a50133e30 2023-09-30T22:31:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.157+0000 7f0a4d59a700 1 -- 172.21.15.32:0/2106146360 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011d20 con 0x7f0a50133e30 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 -- 172.21.15.32:0/2106146360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50133e30 msgr2=0x7f0a5012ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 --2- 172.21.15.32:0/2106146360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50133e30 0x7f0a5012ff90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009840 tx=0x7f0a44009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 -- 172.21.15.32:0/2106146360 shutdown_connections 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 --2- 172.21.15.32:0/2106146360 >> v2:172.21.15.32:3300/0 conn(0x7f0a50133420 0x7f0a501337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 --2- 172.21.15.32:0/2106146360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50133e30 0x7f0a5012ff90 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 -- 172.21.15.32:0/2106146360 >> 172.21.15.32:0/2106146360 conn(0x7f0a500a4210 msgr2=0x7f0a500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 -- 172.21.15.32:0/2106146360 shutdown_connections 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.158+0000 7f0a548fd700 1 -- 172.21.15.32:0/2106146360 wait complete. 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 Processor -- start 2023-09-30T22:31:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 -- start start 2023-09-30T22:31:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a50133420 0x7f0a5009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 0x7f0a5009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50134260 0x7f0a5009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5011fde0 con 0x7f0a50133420 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5012d900 con 0x7f0a50134260 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a548fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a50120300 con 0x7f0a50133e30 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.160+0000 7f0a4dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 0x7f0a5009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 0x7f0a5009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38252/0 (socket says 172.21.15.32:38252) 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/1296256807 learned_addr learned my addr 172.21.15.32:0/1296256807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4ed9d700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50134260 0x7f0a5009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4e59c700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a50133420 0x7f0a5009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50134260 msgr2=0x7f0a5009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50134260 0x7f0a5009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a50133420 msgr2=0x7f0a5009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a50133420 0x7f0a5009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 -- 172.21.15.32:0/1296256807 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440094a0 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4ed9d700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50134260 0x7f0a5009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a4dd9b700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 0x7f0a5009c9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0a440090b0 tx=0x7f0a44004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44004400 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4401f070 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a440038e0 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5011ea90 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.161+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5011fb70 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.162+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a44021030 con 0x7f0a50133e30 2023-09-30T22:31:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.162+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a44040890 con 0x7f0a50133e30 2023-09-30T22:31:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.163+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5009dcf0 con 0x7f0a50133e30 2023-09-30T22:31:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.168+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a44015080 con 0x7f0a50133e30 2023-09-30T22:31:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.297+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0838122b40 con 0x7f083806b250 2023-09-30T22:31:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.297+0000 7f082dffb700 1 -- 172.21.15.32:0/2766340864 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f082801b3b0 con 0x7f083806b250 2023-09-30T22:31:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 msgr2=0x7f083809b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 0x7f083809b2b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f082800ea10 tx=0x7f082800ed20 comp rx=0 tx=0).stop 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 shutdown_connections 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0838136d80 0x7f08381c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f083806a770 0x7f083809ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 --2- 172.21.15.32:0/2766340864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f083806b250 0x7f083809b2b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.299+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 >> 172.21.15.32:0/2766340864 conn(0x7f0838066f50 msgr2=0x7f0838070280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.300+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 shutdown_connections 2023-09-30T22:31:08.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.300+0000 7f083f764700 1 -- 172.21.15.32:0/2766340864 wait complete. 2023-09-30T22:31:08.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.332+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21b00a28e0 con 0x7f21b00a1340 2023-09-30T22:31:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.332+0000 7f21acff9700 1 -- 172.21.15.32:0/1361902093 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2198019730 con 0x7f21b00a1340 2023-09-30T22:31:08.332 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 msgr2=0x7f21b011ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 0x7f21b011ec50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f219800c520 tx=0x7f219800c830 comp rx=0 tx=0).stop 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 shutdown_connections 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f21b0134170 0x7f21b011f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21b00a0a30 0x7f21b011e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 --2- 172.21.15.32:0/1361902093 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21b00a1340 0x7f21b011ec50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.334+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 >> 172.21.15.32:0/1361902093 conn(0x7f21b00a6550 msgr2=0x7f21b009ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.335+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 shutdown_connections 2023-09-30T22:31:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.335+0000 7f21b5d79700 1 -- 172.21.15.32:0/1361902093 wait complete. 2023-09-30T22:31:08.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:08.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.377+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0a50074ea0 con 0x7f0a50133e30 2023-09-30T22:31:08.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.377+0000 7f0a3effd700 1 -- 172.21.15.32:0/1296256807 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a44004560 con 0x7f0a50133e30 2023-09-30T22:31:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 msgr2=0x7f0a5009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 0x7f0a5009c9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0a440090b0 tx=0x7f0a44004750 comp rx=0 tx=0).stop 2023-09-30T22:31:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 shutdown_connections 2023-09-30T22:31:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a50134260 0x7f0a5009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a50133420 0x7f0a5009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 --2- 172.21.15.32:0/1296256807 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a50133e30 0x7f0a5009c9c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 >> 172.21.15.32:0/1296256807 conn(0x7f0a500a4210 msgr2=0x7f0a500a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 shutdown_connections 2023-09-30T22:31:08.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.380+0000 7f0a548fd700 1 -- 172.21.15.32:0/1296256807 wait complete. 2023-09-30T22:31:08.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:08.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.757+0000 7f0a988ee700 1 Processor -- start 2023-09-30T22:31:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.757+0000 7f0a988ee700 1 -- start start 2023-09-30T22:31:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a988ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a988ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a988ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9409edd0 con 0x7f0a940a1340 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a988ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9409e1f0 con 0x7f0a94134170 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a988ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9409dc50 con 0x7f0a940a0a30 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a9259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a91d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a940a1340 0x7f0a94133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57622/0 (socket says 172.21.15.32:57622) 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a91d9b700 1 -- 172.21.15.32:0/508424064 learned_addr learned my addr 172.21.15.32:0/508424064 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.758+0000 7f0a92d9d700 1 --2- 172.21.15.32:0/508424064 >> v2:172.21.15.32:3300/0 conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9159a700 1 -- 172.21.15.32:0/508424064 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539738661 0 0) 0x7f0a9409edd0 con 0x7f0a940a1340 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9159a700 1 -- 172.21.15.32:0/508424064 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a800035b0 con 0x7f0a940a1340 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9259c700 1 -- 172.21.15.32:0/508424064 >> v2:172.21.15.32:3300/0 conn(0x7f0a94134170 msgr2=0x7f0a9413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9259c700 1 --2- 172.21.15.32:0/508424064 >> v2:172.21.15.32:3300/0 conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9259c700 1 -- 172.21.15.32:0/508424064 >> v1:172.21.15.32:6789/0 conn(0x7f0a940a1340 legacy=0x7f0a94133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9259c700 1 -- 172.21.15.32:0/508424064 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940a1fe0 con 0x7f0a940a0a30 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a92d9d700 1 --2- 172.21.15.32:0/508424064 >> v2:172.21.15.32:3300/0 conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9259c700 1 --2- 172.21.15.32:0/508424064 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009840 tx=0x7f0a7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19725b8edfe90017 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9159a700 1 -- 172.21.15.32:0/508424064 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c011830 con 0x7f0a940a0a30 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9159a700 1 -- 172.21.15.32:0/508424064 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c011990 con 0x7f0a940a0a30 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.759+0000 7f0a9159a700 1 -- 172.21.15.32:0/508424064 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c011d20 con 0x7f0a940a0a30 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 -- 172.21.15.32:0/508424064 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 msgr2=0x7f0a940a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 --2- 172.21.15.32:0/508424064 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009840 tx=0x7f0a7c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:08.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 -- 172.21.15.32:0/508424064 shutdown_connections 2023-09-30T22:31:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 --2- 172.21.15.32:0/508424064 >> v2:172.21.15.32:3300/0 conn(0x7f0a94134170 0x7f0a9413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 --2- 172.21.15.32:0/508424064 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a940a0a30 0x7f0a940a0e00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 -- 172.21.15.32:0/508424064 >> 172.21.15.32:0/508424064 conn(0x7f0a940a6550 msgr2=0x7f0a940a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 -- 172.21.15.32:0/508424064 shutdown_connections 2023-09-30T22:31:08.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.760+0000 7f0a988ee700 1 -- 172.21.15.32:0/508424064 wait complete. 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.761+0000 7f0a988ee700 1 Processor -- start 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.761+0000 7f0a988ee700 1 -- start start 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940a0220 con 0x7f0a940a1340 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94124470 con 0x7f0a940a0a30 2023-09-30T22:31:08.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a94125f90 con 0x7f0a94134170 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a9259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43434/0 (socket says 172.21.15.32:43434) 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 -- 172.21.15.32:0/1959367489 learned_addr learned my addr 172.21.15.32:0/1959367489 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a92d9d700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 -- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a94134170 msgr2=0x7f0a94123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 -- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 msgr2=0x7f0a941227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 -- 172.21.15.32:0/1959367489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c009a70 con 0x7f0a940a1340 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a9259c700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a91d9b700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 0x7f0a94122d20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c700 tx=0x7f0a8400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:08.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84005d70 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840070d0 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c0094a0 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.762+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84007460 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.763+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a94133270 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.763+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a840075c0 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.764+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0a8403eae0 con 0x7f0a940a1340 2023-09-30T22:31:08.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.764+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94074ea0 con 0x7f0a940a1340 2023-09-30T22:31:08.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.769+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a84017070 con 0x7f0a940a1340 2023-09-30T22:31:08.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.972+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a940a28e0 con 0x7f0a940a1340 2023-09-30T22:31:08.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.973+0000 7f0a8affd700 1 -- 172.21.15.32:0/1959367489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a8400f260 con 0x7f0a940a1340 2023-09-30T22:31:08.973 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:08.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 msgr2=0x7f0a94122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 0x7f0a94122d20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c700 tx=0x7f0a8400ca10 comp rx=0 tx=0).stop 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 shutdown_connections 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0a94134170 0x7f0a94123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0a940a0a30 0x7f0a941227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 --2- 172.21.15.32:0/1959367489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a940a1340 0x7f0a94122d20 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 >> 172.21.15.32:0/1959367489 conn(0x7f0a940a6550 msgr2=0x7f0a9409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 shutdown_connections 2023-09-30T22:31:08.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:08.975+0000 7f0a988ee700 1 -- 172.21.15.32:0/1959367489 wait complete. 2023-09-30T22:31:08.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:09.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:09.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:09.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 Processor -- start 2023-09-30T22:31:09.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 -- start start 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f304006b080 0x7f3040131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3040131b40 0x7f3040131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304009ece0 con 0x7f304006b080 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304009e760 con 0x7f3040131b40 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.466+0000 7f30450e5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f304009e040 con 0x7f304006a770 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3040131b40 0x7f3040131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38274/0 (socket says 172.21.15.32:38274) 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 -- 172.21.15.32:0/208344030 learned_addr learned my addr 172.21.15.32:0/208344030 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303e59c700 1 --2- 172.21.15.32:0/208344030 >> v2:172.21.15.32:3300/0 conn(0x7f304006b080 0x7f3040131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 -- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3040131b40 msgr2=0x7f3040131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 --2- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3040131b40 0x7f3040131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 -- 172.21.15.32:0/208344030 >> v2:172.21.15.32:3300/0 conn(0x7f304006b080 msgr2=0x7f3040131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 --2- 172.21.15.32:0/208344030 >> v2:172.21.15.32:3300/0 conn(0x7f304006b080 0x7f3040131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 -- 172.21.15.32:0/208344030 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304009f630 con 0x7f304006a770 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303ed9d700 1 --2- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304006ab40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f30300098f0 tx=0x7f3030009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a1055cda4a1f668 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.467+0000 7f303dd9b700 1 -- 172.21.15.32:0/208344030 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030010a90 con 0x7f304006a770 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f303dd9b700 1 -- 172.21.15.32:0/208344030 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030010bf0 con 0x7f304006a770 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f303dd9b700 1 -- 172.21.15.32:0/208344030 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030018440 con 0x7f304006a770 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 -- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 msgr2=0x7f304006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 --2- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304006ab40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f30300098f0 tx=0x7f3030009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 -- 172.21.15.32:0/208344030 shutdown_connections 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 --2- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3040131b40 0x7f3040131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 --2- 172.21.15.32:0/208344030 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304006ab40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 --2- 172.21.15.32:0/208344030 >> v2:172.21.15.32:3300/0 conn(0x7f304006b080 0x7f3040131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 -- 172.21.15.32:0/208344030 >> 172.21.15.32:0/208344030 conn(0x7f3040066f50 msgr2=0x7f304009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 -- 172.21.15.32:0/208344030 shutdown_connections 2023-09-30T22:31:09.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.468+0000 7f30450e5700 1 -- 172.21.15.32:0/208344030 wait complete. 2023-09-30T22:31:09.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.470+0000 7f30450e5700 1 Processor -- start 2023-09-30T22:31:09.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.470+0000 7f30450e5700 1 -- start start 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f30450e5700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304011ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f30450e5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304006b080 0x7f304011b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f30450e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3040131b40 0x7f304011b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f30450e5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304011e9a0 con 0x7f3040131b40 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f30450e5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30400a3640 con 0x7f304006b080 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f30450e5700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f304011c2c0 con 0x7f304006a770 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f303ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304011ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f303f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3040131b40 0x7f304011b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f303ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304011ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38276/0 (socket says 172.21.15.32:38276) 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f303ed9d700 1 -- 172.21.15.32:0/2967671510 learned_addr learned my addr 172.21.15.32:0/2967671510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.471+0000 7f303e59c700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304006b080 0x7f304011b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303ed9d700 1 -- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304006b080 msgr2=0x7f304011b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303ed9d700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304006b080 0x7f304011b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303ed9d700 1 -- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3040131b40 msgr2=0x7f304011b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303ed9d700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3040131b40 0x7f304011b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303ed9d700 1 -- 172.21.15.32:0/2967671510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3030009550 con 0x7f304006a770 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303f59e700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3040131b40 0x7f304011b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f303ed9d700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304011ae80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3030009c40 tx=0x7f3030004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030003ab0 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030003c10 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030018440 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f30450e5700 1 -- 172.21.15.32:0/2967671510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304011e130 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.472+0000 7f30450e5700 1 -- 172.21.15.32:0/2967671510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304011ca90 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.473+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f303001f040 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.473+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30180052e0 con 0x7f304006a770 2023-09-30T22:31:09.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.474+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3030044b90 con 0x7f304006a770 2023-09-30T22:31:09.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.478+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f30300185a0 con 0x7f304006a770 2023-09-30T22:31:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.508+0000 7fe67d7ed700 1 Processor -- start 2023-09-30T22:31:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe67d7ed700 1 -- start start 2023-09-30T22:31:09.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe67d7ed700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe67809d630 0x7fe6780a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe67d7ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe678138d80 0x7fe67813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe67d7ed700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6780a2a70 con 0x7fe6780a0a30 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe67d7ed700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6780a01d0 con 0x7fe67809d630 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe67d7ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67809e2b0 con 0x7fe678138d80 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe66e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe67809d630 0x7fe6780a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe676ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6780a0a30 0x7fe6780a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57638/0 (socket says 172.21.15.32:57638) 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe676ffd700 1 -- 172.21.15.32:0/3620346627 learned_addr learned my addr 172.21.15.32:0/3620346627 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.509+0000 7fe6777fe700 1 --2- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe678138d80 0x7fe67813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe66e7fc700 1 -- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe678138d80 msgr2=0x7fe67813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe66e7fc700 1 --2- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe678138d80 0x7fe67813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe66e7fc700 1 -- 172.21.15.32:0/3620346627 >> v1:172.21.15.32:6789/0 conn(0x7fe6780a0a30 legacy=0x7fe6780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe6767fc700 1 -- 172.21.15.32:0/3620346627 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41503860 0 0) 0x7fe6780a2a70 con 0x7fe6780a0a30 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe66e7fc700 1 -- 172.21.15.32:0/3620346627 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67809dc50 con 0x7fe67809d630 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe66e7fc700 1 --2- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe67809d630 0x7fe6780a1510 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe660009840 tx=0x7fe660009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa2103fb329e503 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe6767fc700 1 -- 172.21.15.32:0/3620346627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660011830 con 0x7fe67809d630 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe6767fc700 1 -- 172.21.15.32:0/3620346627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660011990 con 0x7fe67809d630 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.510+0000 7fe6767fc700 1 -- 172.21.15.32:0/3620346627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660011d20 con 0x7fe67809d630 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 -- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe67809d630 msgr2=0x7fe6780a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe67809d630 0x7fe6780a1510 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe660009840 tx=0x7fe660009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 -- 172.21.15.32:0/3620346627 shutdown_connections 2023-09-30T22:31:09.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe678138d80 0x7fe67813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/3620346627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe67809d630 0x7fe6780a1510 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 -- 172.21.15.32:0/3620346627 >> 172.21.15.32:0/3620346627 conn(0x7fe6780a3050 msgr2=0x7fe6780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 -- 172.21.15.32:0/3620346627 shutdown_connections 2023-09-30T22:31:09.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.511+0000 7fe67d7ed700 1 -- 172.21.15.32:0/3620346627 wait complete. 2023-09-30T22:31:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.512+0000 7fe67d7ed700 1 Processor -- start 2023-09-30T22:31:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.512+0000 7fe67d7ed700 1 -- start start 2023-09-30T22:31:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe67d7ed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 0x7fe6781337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe67d7ed700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 0x7fe678133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe67d7ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe678138d80 0x7fe678134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe67d7ed700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678136270 con 0x7fe6780a0a30 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe67d7ed700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe678136f90 con 0x7fe67809d630 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe67d7ed700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780a4b10 con 0x7fe678138d80 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 0x7fe6781337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe66e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 0x7fe678133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe66e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 0x7fe678133ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43468/0 (socket says 172.21.15.32:43468) 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 0x7fe6781337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54914/0 (socket says 172.21.15.32:54914) 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe66e7fc700 1 -- 172.21.15.32:0/612507047 learned_addr learned my addr 172.21.15.32:0/612507047 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:09.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 -- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe678138d80 msgr2=0x7fe678134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe678138d80 0x7fe678134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 -- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 msgr2=0x7fe678133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 0x7fe678133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 -- 172.21.15.32:0/612507047 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600094a0 con 0x7fe67809d630 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe66e7fc700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 0x7fe678133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe676ffd700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 0x7fe6781337a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe66800c470 tx=0x7fe66800c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.513+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668004c80 con 0x7fe67809d630 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.514+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668004de0 con 0x7fe67809d630 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.514+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668007310 con 0x7fe67809d630 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.514+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6780a0410 con 0x7fe67809d630 2023-09-30T22:31:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.514+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe678137270 con 0x7fe67809d630 2023-09-30T22:31:09.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.514+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe668007470 con 0x7fe67809d630 2023-09-30T22:31:09.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.514+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe66803f210 con 0x7fe67809d630 2023-09-30T22:31:09.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.515+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe678136dd0 con 0x7fe67809d630 2023-09-30T22:31:09.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.520+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe668007720 con 0x7fe67809d630 2023-09-30T22:31:09.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 Processor -- start 2023-09-30T22:31:09.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 -- start start 2023-09-30T22:31:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 0x7fe3900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3900a1340 0x7fe390133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39009edd0 con 0x7fe390133fa0 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39009e1f0 con 0x7fe3900a0a30 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.569+0000 7fe397b97700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe39009dc50 con 0x7fe3900a1340 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395933700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 0x7fe3900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395933700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 0x7fe3900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43478/0 (socket says 172.21.15.32:43478) 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395933700 1 -- 172.21.15.32:0/3350729747 learned_addr learned my addr 172.21.15.32:0/3350729747 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395132700 1 --2- 172.21.15.32:0/3350729747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3900a1340 0x7fe390133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395132700 1 -- 172.21.15.32:0/3350729747 >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 msgr2=0x7fe3900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395132700 1 --2- 172.21.15.32:0/3350729747 >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 0x7fe3900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395132700 1 -- 172.21.15.32:0/3350729747 >> v1:172.21.15.32:6789/0 conn(0x7fe390133fa0 legacy=0x7fe39013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe394931700 1 -- 172.21.15.32:0/3350729747 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250664127 0 0) 0x7fe39009edd0 con 0x7fe390133fa0 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395132700 1 -- 172.21.15.32:0/3350729747 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3900a1fe0 con 0x7fe3900a1340 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395933700 1 --2- 172.21.15.32:0/3350729747 >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 0x7fe3900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe395132700 1 --2- 172.21.15.32:0/3350729747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3900a1340 0x7fe390133a60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe380009840 tx=0x7fe380009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba9bd0cf2ce75716 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.570+0000 7fe394931700 1 -- 172.21.15.32:0/3350729747 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380011830 con 0x7fe3900a1340 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe394931700 1 -- 172.21.15.32:0/3350729747 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe380011990 con 0x7fe3900a1340 2023-09-30T22:31:09.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe394931700 1 -- 172.21.15.32:0/3350729747 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380011d20 con 0x7fe3900a1340 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 -- 172.21.15.32:0/3350729747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3900a1340 msgr2=0x7fe390133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 --2- 172.21.15.32:0/3350729747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3900a1340 0x7fe390133a60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe380009840 tx=0x7fe380009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 -- 172.21.15.32:0/3350729747 shutdown_connections 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 --2- 172.21.15.32:0/3350729747 >> v2:172.21.15.32:3300/0 conn(0x7fe3900a0a30 0x7fe3900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 --2- 172.21.15.32:0/3350729747 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe3900a1340 0x7fe390133a60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 -- 172.21.15.32:0/3350729747 >> 172.21.15.32:0/3350729747 conn(0x7fe3900a6550 msgr2=0x7fe3900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.571+0000 7fe397b97700 1 -- 172.21.15.32:0/3350729747 shutdown_connections 2023-09-30T22:31:09.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.572+0000 7fe397b97700 1 -- 172.21.15.32:0/3350729747 wait complete. 2023-09-30T22:31:09.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.573+0000 7fe397b97700 1 Processor -- start 2023-09-30T22:31:09.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.573+0000 7fe397b97700 1 -- start start 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe397b97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3900a0a30 0x7fe390123470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe397b97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 0x7fe3901239b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe397b97700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe390133fa0 0x7fe3901c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe397b97700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe390126370 con 0x7fe3900a1340 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe397b97700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe390126550 con 0x7fe3900a0a30 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe397b97700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900a3580 con 0x7fe390133fa0 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 0x7fe3901239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 0x7fe3901239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43490/0 (socket says 172.21.15.32:43490) 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 -- 172.21.15.32:0/1107309224 learned_addr learned my addr 172.21.15.32:0/1107309224 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe396134700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe390133fa0 0x7fe3901c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395933700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3900a0a30 0x7fe390123470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 -- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe390133fa0 msgr2=0x7fe3901c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe390133fa0 0x7fe3901c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 -- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3900a0a30 msgr2=0x7fe390123470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3900a0a30 0x7fe390123470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.574+0000 7fe395132700 1 -- 172.21.15.32:0/1107309224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c009550 con 0x7fe3900a1340 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.575+0000 7fe395132700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 0x7fe3901239b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe3800090b0 tx=0x7fe380004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.575+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380004550 con 0x7fe3900a1340 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.575+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38001f070 con 0x7fe3900a1340 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.575+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3800094a0 con 0x7fe3900a1340 2023-09-30T22:31:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.575+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3800038c0 con 0x7fe3900a1340 2023-09-30T22:31:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.575+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe390133750 con 0x7fe3900a1340 2023-09-30T22:31:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.576+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe390074ea0 con 0x7fe3900a1340 2023-09-30T22:31:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.576+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe380003a20 con 0x7fe3900a1340 2023-09-30T22:31:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.576+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe380021070 con 0x7fe3900a1340 2023-09-30T22:31:09.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.586+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3800253e0 con 0x7fe3900a1340 2023-09-30T22:31:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.723+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3018005590 con 0x7f304006a770 2023-09-30T22:31:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.723+0000 7f30277fe700 1 -- 172.21.15.32:0/2967671510 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f303002f070 con 0x7f304006a770 2023-09-30T22:31:09.723 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:09.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 msgr2=0x7f304011ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304011ae80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3030009c40 tx=0x7f3030004750 comp rx=0 tx=0).stop 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 shutdown_connections 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3040131b40 0x7f304011b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f304006a770 0x7f304011ae80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 --2- 172.21.15.32:0/2967671510 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f304006b080 0x7f304011b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.725+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 >> 172.21.15.32:0/2967671510 conn(0x7f3040066f50 msgr2=0x7f3040130ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.726+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 shutdown_connections 2023-09-30T22:31:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.726+0000 7f30257fa700 1 -- 172.21.15.32:0/2967671510 wait complete. 2023-09-30T22:31:09.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.730+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe678136c30 con 0x7fe67809d630 2023-09-30T22:31:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.730+0000 7fe66ffff700 1 -- 172.21.15.32:0/612507047 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe678136c30 con 0x7fe67809d630 2023-09-30T22:31:09.730 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:09.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 msgr2=0x7fe6781337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 0x7fe6781337a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe66800c470 tx=0x7fe66800c780 comp rx=0 tx=0).stop 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 shutdown_connections 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe678138d80 0x7fe678134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe67809d630 0x7fe6781337a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 --2- 172.21.15.32:0/612507047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6780a0a30 0x7fe678133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.732+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 >> 172.21.15.32:0/612507047 conn(0x7fe6780a3050 msgr2=0x7fe67809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.733+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 shutdown_connections 2023-09-30T22:31:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.733+0000 7fe67d7ed700 1 -- 172.21.15.32:0/612507047 wait complete. 2023-09-30T22:31:09.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:09.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:09.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.794+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe39009bc20 con 0x7fe3900a1340 2023-09-30T22:31:09.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.795+0000 7fe3867fc700 1 -- 172.21.15.32:0/1107309224 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe380015030 con 0x7fe3900a1340 2023-09-30T22:31:09.795 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 msgr2=0x7fe3901239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 0x7fe3901239b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe3800090b0 tx=0x7fe380004750 comp rx=0 tx=0).stop 2023-09-30T22:31:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 shutdown_connections 2023-09-30T22:31:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe390133fa0 0x7fe3901c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe3900a0a30 0x7fe390123470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 --2- 172.21.15.32:0/1107309224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3900a1340 0x7fe3901239b0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 >> 172.21.15.32:0/1107309224 conn(0x7fe3900a6550 msgr2=0x7fe39011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 shutdown_connections 2023-09-30T22:31:09.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:09.797+0000 7fe397b97700 1 -- 172.21.15.32:0/1107309224 wait complete. 2023-09-30T22:31:09.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:09.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 Processor -- start 2023-09-30T22:31:10.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 -- start start 2023-09-30T22:31:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8fc0a0a30 0x7fe8fc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8fc0a2a70 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc0a01d0 con 0x7fe8fc09d630 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.136+0000 7fe903be9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc09e2b0 con 0x7fe8fc133930 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe902186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901985700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8fc0a0a30 0x7fe8fc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe902186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:38290/0 (socket says 172.21.15.32:38290) 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54938/0 (socket says 172.21.15.32:54938) 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe902186700 1 -- 172.21.15.32:0/3389654276 learned_addr learned my addr 172.21.15.32:0/3389654276 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 -- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 msgr2=0x7fe8fc133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 --2- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 -- 172.21.15.32:0/3389654276 >> v2:172.21.15.32:3300/0 conn(0x7fe8fc0a0a30 msgr2=0x7fe8fc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 --2- 172.21.15.32:0/3389654276 >> v2:172.21.15.32:3300/0 conn(0x7fe8fc0a0a30 0x7fe8fc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 -- 172.21.15.32:0/3389654276 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc09dc50 con 0x7fe8fc09d630 2023-09-30T22:31:10.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe902186700 1 --2- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe901184700 1 --2- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc0a1340 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009840 tx=0x7fe8ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e919afea86759a86 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe900983700 1 -- 172.21.15.32:0/3389654276 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec011830 con 0x7fe8fc09d630 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe900983700 1 -- 172.21.15.32:0/3389654276 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec011990 con 0x7fe8fc09d630 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.137+0000 7fe900983700 1 -- 172.21.15.32:0/3389654276 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec011d20 con 0x7fe8fc09d630 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 -- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 msgr2=0x7fe8fc0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 --2- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc0a1340 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009840 tx=0x7fe8ec009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 -- 172.21.15.32:0/3389654276 shutdown_connections 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 --2- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 --2- 172.21.15.32:0/3389654276 >> v2:172.21.15.32:3300/0 conn(0x7fe8fc0a0a30 0x7fe8fc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 --2- 172.21.15.32:0/3389654276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc0a1340 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 -- 172.21.15.32:0/3389654276 >> 172.21.15.32:0/3389654276 conn(0x7fe8fc0a3050 msgr2=0x7fe8fc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 -- 172.21.15.32:0/3389654276 shutdown_connections 2023-09-30T22:31:10.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.138+0000 7fe903be9700 1 -- 172.21.15.32:0/3389654276 wait complete. 2023-09-30T22:31:10.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.139+0000 7fe903be9700 1 Processor -- start 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.139+0000 7fe903be9700 1 -- start start 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc11f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 0x7fe8fc11fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc1c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc122330 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1333a0 con 0x7fe8fc09d630 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc122530 con 0x7fe8fc133930 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901985700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc11f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 0x7fe8fc11fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe902186700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc1c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901985700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc11f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54952/0 (socket says 172.21.15.32:54952) 2023-09-30T22:31:10.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901985700 1 -- 172.21.15.32:0/1794311585 learned_addr learned my addr 172.21.15.32:0/1794311585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 0x7fe8fc11fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43510/0 (socket says 172.21.15.32:43510) 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 -- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 msgr2=0x7fe8fc1c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc1c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 -- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 msgr2=0x7fe8fc11f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc11f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 -- 172.21.15.32:0/1794311585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f4009b30 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901985700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc11f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe902186700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe901184700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 0x7fe8fc11fae0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec016000 tx=0x7fe8ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec004240 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec01f070 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0094a0 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.140+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0036a0 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.141+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc11efb0 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.141+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8fc0a0410 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.142+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8ec021030 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.142+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe8ec0409b0 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.147+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8ec010970 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.348+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe8fc0747f0 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.349+0000 7fe8f27fc700 1 -- 172.21.15.32:0/1794311585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8ec015070 con 0x7fe8fc0a0a30 2023-09-30T22:31:10.349 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:10.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 msgr2=0x7fe8fc11fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 0x7fe8fc11fae0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec016000 tx=0x7fe8ec004750 comp rx=0 tx=0).stop 2023-09-30T22:31:10.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 shutdown_connections 2023-09-30T22:31:10.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8fc133930 0x7fe8fc1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8fc09d630 0x7fe8fc11f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 --2- 172.21.15.32:0/1794311585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8fc0a0a30 0x7fe8fc11fae0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 >> 172.21.15.32:0/1794311585 conn(0x7fe8fc0a3050 msgr2=0x7fe8fc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 shutdown_connections 2023-09-30T22:31:10.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.351+0000 7fe903be9700 1 -- 172.21.15.32:0/1794311585 wait complete. 2023-09-30T22:31:10.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:10.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:10.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:10.811 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:10.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.891+0000 7f34e1ffc700 1 Processor -- start 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.891+0000 7f34e1ffc700 1 -- start start 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.891+0000 7f34e1ffc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.891+0000 7f34e1ffc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34e1ffc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc09f130 con 0x7f34dc09e000 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34e1ffc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc09f520 con 0x7f34dc09d6f0 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34e1ffc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc09e930 con 0x7f34dc09e410 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34daffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34dc09e000 0x7f34dc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44108/0 (socket says 172.21.15.32:44108) 2023-09-30T22:31:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34daffd700 1 -- 172.21.15.32:0/1463067839 learned_addr learned my addr 172.21.15.32:0/1463067839 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:10.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34db7fe700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34dbfff700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34db7fe700 1 -- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 msgr2=0x7f34dc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34db7fe700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34db7fe700 1 -- 172.21.15.32:0/1463067839 >> v1:172.21.15.32:6789/0 conn(0x7f34dc09e000 legacy=0x7f34dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34da7fc700 1 -- 172.21.15.32:0/1463067839 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908179147 0 0) 0x7f34dc09f130 con 0x7f34dc09e000 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34db7fe700 1 -- 172.21.15.32:0/1463067839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc09b2d0 con 0x7f34dc09d6f0 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.892+0000 7f34db7fe700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc09dac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16adf994150139b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34da7fc700 1 -- 172.21.15.32:0/1463067839 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc011830 con 0x7f34dc09d6f0 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34da7fc700 1 -- 172.21.15.32:0/1463067839 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc011990 con 0x7f34dc09d6f0 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34da7fc700 1 -- 172.21.15.32:0/1463067839 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc011d20 con 0x7f34dc09d6f0 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 -- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 msgr2=0x7f34dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc09dac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009870 tx=0x7f34cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 -- 172.21.15.32:0/1463067839 shutdown_connections 2023-09-30T22:31:10.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/1463067839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc09dac0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 -- 172.21.15.32:0/1463067839 >> 172.21.15.32:0/1463067839 conn(0x7f34dc066f50 msgr2=0x7f34dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.893+0000 7f34e1ffc700 1 -- 172.21.15.32:0/1463067839 shutdown_connections 2023-09-30T22:31:10.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.894+0000 7f34e1ffc700 1 -- 172.21.15.32:0/1463067839 wait complete. 2023-09-30T22:31:10.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.894+0000 7f34e1ffc700 1 Processor -- start 2023-09-30T22:31:10.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 -- start start 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc09e000 0x7f34dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc125a20 con 0x7f34dc09e000 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc09c700 con 0x7f34dc09d6f0 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34e1ffc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc070590 con 0x7f34dc09e410 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34db7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34db7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41680/0 (socket says 172.21.15.32:41680) 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34db7fe700 1 -- 172.21.15.32:0/3465974257 learned_addr learned my addr 172.21.15.32:0/3465974257 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.895+0000 7f34dbfff700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34daffd700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc09e000 0x7f34dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34db7fe700 1 -- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 msgr2=0x7f34dc06f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34db7fe700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc06f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34db7fe700 1 -- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc09e000 msgr2=0x7f34dc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34db7fe700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc09e000 0x7f34dc06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34db7fe700 1 -- 172.21.15.32:0/3465974257 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0094a0 con 0x7f34dc09d6f0 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34dbfff700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34daffd700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc09e000 0x7f34dc06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34db7fe700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc06e870 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0090b0 tx=0x7f34cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc004590 con 0x7f34dc09d6f0 2023-09-30T22:31:10.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc01f070 con 0x7f34dc09d6f0 2023-09-30T22:31:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc0a1c10 con 0x7f34dc09d6f0 2023-09-30T22:31:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0038e0 con 0x7f34dc09d6f0 2023-09-30T22:31:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.896+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc125d00 con 0x7f34dc09d6f0 2023-09-30T22:31:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.897+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34cc003a40 con 0x7f34dc09d6f0 2023-09-30T22:31:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.897+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f34cc021070 con 0x7f34dc09d6f0 2023-09-30T22:31:10.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.898+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc125b60 con 0x7f34dc09d6f0 2023-09-30T22:31:10.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.907+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34dc125b60 con 0x7f34dc09d6f0 2023-09-30T22:31:10.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 Processor -- start 2023-09-30T22:31:10.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 -- start start 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc09d630 0x7f60bc0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60bc138d80 0x7f60bc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60bc0a2a70 con 0x7f60bc0a0a30 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60bc0a01d0 con 0x7f60bc138d80 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c309f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc09e2b0 con 0x7f60bc09d630 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60b3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc09d630 0x7f60bc0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c0e3b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f60bc0a0a30 0x7f60bc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44118/0 (socket says 172.21.15.32:44118) 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c163c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60bc138d80 0x7f60bc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.931+0000 7f60c0e3b700 1 -- 172.21.15.32:0/198957004 learned_addr learned my addr 172.21.15.32:0/198957004 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:10.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b3fff700 1 -- 172.21.15.32:0/198957004 >> v2:172.21.15.32:3300/0 conn(0x7f60bc138d80 msgr2=0x7f60bc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b3fff700 1 --2- 172.21.15.32:0/198957004 >> v2:172.21.15.32:3300/0 conn(0x7f60bc138d80 0x7f60bc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b3fff700 1 -- 172.21.15.32:0/198957004 >> v1:172.21.15.32:6789/0 conn(0x7f60bc0a0a30 legacy=0x7f60bc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b3fff700 1 -- 172.21.15.32:0/198957004 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc09dc50 con 0x7f60bc09d630 2023-09-30T22:31:10.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b3fff700 1 --2- 172.21.15.32:0/198957004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc09d630 0x7f60bc0a1510 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009840 tx=0x7f60ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91440d3fd075f50d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b37fe700 1 -- 172.21.15.32:0/198957004 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac011830 con 0x7f60bc09d630 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b37fe700 1 -- 172.21.15.32:0/198957004 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac011990 con 0x7f60bc09d630 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.932+0000 7f60b37fe700 1 -- 172.21.15.32:0/198957004 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac011d20 con 0x7f60bc09d630 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 -- 172.21.15.32:0/198957004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc09d630 msgr2=0x7f60bc0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 --2- 172.21.15.32:0/198957004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc09d630 0x7f60bc0a1510 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009840 tx=0x7f60ac009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 -- 172.21.15.32:0/198957004 shutdown_connections 2023-09-30T22:31:10.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 --2- 172.21.15.32:0/198957004 >> v2:172.21.15.32:3300/0 conn(0x7f60bc138d80 0x7f60bc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 --2- 172.21.15.32:0/198957004 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc09d630 0x7f60bc0a1510 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 -- 172.21.15.32:0/198957004 >> 172.21.15.32:0/198957004 conn(0x7f60bc0a3050 msgr2=0x7f60bc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 -- 172.21.15.32:0/198957004 shutdown_connections 2023-09-30T22:31:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.933+0000 7f60c309f700 1 -- 172.21.15.32:0/198957004 wait complete. 2023-09-30T22:31:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.934+0000 7f60c309f700 1 Processor -- start 2023-09-30T22:31:10.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.934+0000 7f60c309f700 1 -- start start 2023-09-30T22:31:10.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c309f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 0x7f60bc1c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c309f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc0a0a30 0x7f60bc1c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c309f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc138d80 0x7f60bc1c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c309f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1381b0 con 0x7f60bc0a0a30 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c309f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1361d0 con 0x7f60bc138d80 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c309f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc13b030 con 0x7f60bc09d630 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 0x7f60bc1c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 0x7f60bc1c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54066/0 (socket says 172.21.15.32:54066) 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 -- 172.21.15.32:0/2764905229 learned_addr learned my addr 172.21.15.32:0/2764905229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60b3fff700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc0a0a30 0x7f60bc1c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c163c700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc138d80 0x7f60bc1c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 -- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc138d80 msgr2=0x7f60bc1c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc138d80 0x7f60bc1c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 -- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc0a0a30 msgr2=0x7f60bc1c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc0a0a30 0x7f60bc1c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.935+0000 7f60c0e3b700 1 -- 172.21.15.32:0/2764905229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0094a0 con 0x7f60bc09d630 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60b3fff700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc0a0a30 0x7f60bc1c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60c163c700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc138d80 0x7f60bc1c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60c0e3b700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 0x7f60bc1c8600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f60b800cba0 tx=0x7f60b800ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800aaa0 con 0x7f60bc09d630 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b800ac00 con 0x7f60bc09d630 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8012400 con 0x7f60bc09d630 2023-09-30T22:31:10.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc0a0410 con 0x7f60bc09d630 2023-09-30T22:31:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.936+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc136c40 con 0x7f60bc09d630 2023-09-30T22:31:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.937+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc136010 con 0x7f60bc09d630 2023-09-30T22:31:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.938+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60b801b3b0 con 0x7f60bc09d630 2023-09-30T22:31:10.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.938+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60b803f7c0 con 0x7f60bc09d630 2023-09-30T22:31:10.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.947+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60b80192c0 con 0x7f60bc09d630 2023-09-30T22:31:10.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 Processor -- start 2023-09-30T22:31:10.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 -- start start 2023-09-30T22:31:10.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 0x7f99bc130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc130870 0x7f99bc13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99bc06c570 con 0x7f99bc12ff20 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc06bc00 con 0x7f99bc130870 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.976+0000 7f99c456c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc0a0040 con 0x7f99bc133a80 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1b07700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 0x7f99bc130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1b07700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 0x7f99bc130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58872/0 (socket says 172.21.15.32:58872) 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1b07700 1 -- 172.21.15.32:0/2219182269 learned_addr learned my addr 172.21.15.32:0/2219182269 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:10.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2b09700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc130870 0x7f99bc13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 -- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc130870 msgr2=0x7f99bc13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc130870 0x7f99bc13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 -- 172.21.15.32:0/2219182269 >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 msgr2=0x7f99bc130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 --2- 172.21.15.32:0/2219182269 >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 0x7f99bc130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 -- 172.21.15.32:0/2219182269 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc133860 con 0x7f99bc133a80 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1b07700 1 --2- 172.21.15.32:0/2219182269 >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 0x7f99bc130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c2308700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc133e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009840 tx=0x7f99b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1efc763225a0f53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1306700 1 -- 172.21.15.32:0/2219182269 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8011830 con 0x7f99bc133a80 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1306700 1 -- 172.21.15.32:0/2219182269 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8011990 con 0x7f99bc133a80 2023-09-30T22:31:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.977+0000 7f99c1306700 1 -- 172.21.15.32:0/2219182269 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8011d20 con 0x7f99bc133a80 2023-09-30T22:31:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 -- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 msgr2=0x7f99bc133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc133e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009840 tx=0x7f99b8009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 -- 172.21.15.32:0/2219182269 shutdown_connections 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc130870 0x7f99bc13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 --2- 172.21.15.32:0/2219182269 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc133e50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 --2- 172.21.15.32:0/2219182269 >> v2:172.21.15.32:3300/0 conn(0x7f99bc12ff20 0x7f99bc130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 -- 172.21.15.32:0/2219182269 >> 172.21.15.32:0/2219182269 conn(0x7f99bc0a4210 msgr2=0x7f99bc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 -- 172.21.15.32:0/2219182269 shutdown_connections 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.978+0000 7f99c456c700 1 -- 172.21.15.32:0/2219182269 wait complete. 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.979+0000 7f99c456c700 1 Processor -- start 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.979+0000 7f99c456c700 1 -- start start 2023-09-30T22:31:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c456c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc12ff20 0x7f99bc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c456c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 0x7f99bc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c456c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c456c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc11ff10 con 0x7f99bc130870 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c456c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc120dd0 con 0x7f99bc12ff20 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c456c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc12db80 con 0x7f99bc133a80 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c1b07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 0x7f99bc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c1b07700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 0x7f99bc09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58880/0 (socket says 172.21.15.32:58880) 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c1b07700 1 -- 172.21.15.32:0/189005716 learned_addr learned my addr 172.21.15.32:0/189005716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2308700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc12ff20 0x7f99bc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2b09700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2308700 1 -- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 msgr2=0x7f99bc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2308700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2308700 1 -- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 msgr2=0x7f99bc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2308700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 0x7f99bc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.980+0000 7f99c2308700 1 -- 172.21.15.32:0/189005716 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac009550 con 0x7f99bc12ff20 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99c1b07700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 0x7f99bc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99c2308700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc12ff20 0x7f99bc09c480 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f99b8016000 tx=0x7f99b8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004450 con 0x7f99bc12ff20 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80036a0 con 0x7f99bc12ff20 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b80094a0 con 0x7f99bc12ff20 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80045f0 con 0x7f99bc12ff20 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.981+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc11f940 con 0x7f99bc12ff20 2023-09-30T22:31:10.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.982+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99b8003a50 con 0x7f99bc12ff20 2023-09-30T22:31:10.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.982+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99bc074ea0 con 0x7f99bc12ff20 2023-09-30T22:31:10.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.982+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f99b80407b0 con 0x7f99bc12ff20 2023-09-30T22:31:10.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:10.992+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99b8003d00 con 0x7f99bc12ff20 2023-09-30T22:31:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.138+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f34dc125b60 con 0x7f34dc09d6f0 2023-09-30T22:31:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.138+0000 7f34c3fff700 1 -- 172.21.15.32:0/3465974257 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34dc125b60 con 0x7f34dc09d6f0 2023-09-30T22:31:11.138 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 msgr2=0x7f34dc06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc06e870 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0090b0 tx=0x7f34cc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 shutdown_connections 2023-09-30T22:31:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f34dc09e410 0x7f34dc06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f34dc09d6f0 0x7f34dc06e870 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 --2- 172.21.15.32:0/3465974257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34dc09e000 0x7f34dc06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 >> 172.21.15.32:0/3465974257 conn(0x7f34dc066f50 msgr2=0x7f34dc071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 shutdown_connections 2023-09-30T22:31:11.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.141+0000 7f34e1ffc700 1 -- 172.21.15.32:0/3465974257 wait complete. 2023-09-30T22:31:11.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:11.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.160+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f60bc0747f0 con 0x7f60bc09d630 2023-09-30T22:31:11.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.160+0000 7f60b17fa700 1 -- 172.21.15.32:0/2764905229 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60b8017030 con 0x7f60bc09d630 2023-09-30T22:31:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:11.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 msgr2=0x7f60bc1c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 0x7f60bc1c8600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f60b800cba0 tx=0x7f60b800ceb0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 shutdown_connections 2023-09-30T22:31:11.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc138d80 0x7f60bc1c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60bc09d630 0x7f60bc1c8600 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 --2- 172.21.15.32:0/2764905229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc0a0a30 0x7f60bc1c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 >> 172.21.15.32:0/2764905229 conn(0x7f60bc0a3050 msgr2=0x7f60bc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 shutdown_connections 2023-09-30T22:31:11.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.163+0000 7f60c309f700 1 -- 172.21.15.32:0/2764905229 wait complete. 2023-09-30T22:31:11.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:11.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.203+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f99bc120130 con 0x7f99bc12ff20 2023-09-30T22:31:11.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.203+0000 7f99b2ffd700 1 -- 172.21.15.32:0/189005716 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99b8015030 con 0x7f99bc12ff20 2023-09-30T22:31:11.203 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc12ff20 msgr2=0x7f99bc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc12ff20 0x7f99bc09c480 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f99b8016000 tx=0x7f99b8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:11.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 shutdown_connections 2023-09-30T22:31:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f99bc133a80 0x7f99bc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f99bc12ff20 0x7f99bc09c480 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 --2- 172.21.15.32:0/189005716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99bc130870 0x7f99bc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 >> 172.21.15.32:0/189005716 conn(0x7f99bc0a4210 msgr2=0x7f99bc09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 shutdown_connections 2023-09-30T22:31:11.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.205+0000 7f99c456c700 1 -- 172.21.15.32:0/189005716 wait complete. 2023-09-30T22:31:11.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:11.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 Processor -- start 2023-09-30T22:31:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 -- start start 2023-09-30T22:31:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf00136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf00136fe0 0x7fcf001393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf00070940 con 0x7fcf0009e930 2023-09-30T22:31:11.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0006fd60 con 0x7fcf0009f410 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcf0525f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0006fa80 con 0x7fcf00136fe0 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf00136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fceff59e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf00136fe0 0x7fcf001393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf00136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41712/0 (socket says 172.21.15.32:41712) 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.511+0000 7fcefe59c700 1 -- 172.21.15.32:0/4143450621 learned_addr learned my addr 172.21.15.32:0/4143450621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fceff59e700 1 -- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 msgr2=0x7fcf00136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fceff59e700 1 --2- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf00136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fceff59e700 1 -- 172.21.15.32:0/4143450621 >> v1:172.21.15.32:6789/0 conn(0x7fcf0009e930 legacy=0x7fcf0009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fceff59e700 1 -- 172.21.15.32:0/4143450621 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0009fb60 con 0x7fcf00136fe0 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fcefdd9b700 1 -- 172.21.15.32:0/4143450621 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190562763 0 0) 0x7fcf00070940 con 0x7fcf0009e930 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fcefe59c700 1 --2- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf00136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fceff59e700 1 --2- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf00136fe0 0x7fcf001393c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009dc0 tx=0x7fcef800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ec1a413fb1df21 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fcefdd9b700 1 -- 172.21.15.32:0/4143450621 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef800e070 con 0x7fcf00136fe0 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fcefdd9b700 1 -- 172.21.15.32:0/4143450621 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef800bb20 con 0x7fcf00136fe0 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.512+0000 7fcefdd9b700 1 -- 172.21.15.32:0/4143450621 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012430 con 0x7fcf00136fe0 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 -- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf00136fe0 msgr2=0x7fcf001393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 --2- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf00136fe0 0x7fcf001393c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009dc0 tx=0x7fcef800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:11.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 -- 172.21.15.32:0/4143450621 shutdown_connections 2023-09-30T22:31:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 --2- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf00136fe0 0x7fcf001393c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 --2- 172.21.15.32:0/4143450621 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf00136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 -- 172.21.15.32:0/4143450621 >> 172.21.15.32:0/4143450621 conn(0x7fcf000a4210 msgr2=0x7fcf000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 -- 172.21.15.32:0/4143450621 shutdown_connections 2023-09-30T22:31:11.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.513+0000 7fcf0525f700 1 -- 172.21.15.32:0/4143450621 wait complete. 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.514+0000 7fcf0525f700 1 Processor -- start 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.514+0000 7fcf0525f700 1 -- start start 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcf0525f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 0x7fcf001c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcf0525f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf001c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcf0525f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00136fe0 0x7fcf0006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcf0525f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf001345f0 con 0x7fcf00136fe0 2023-09-30T22:31:11.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcf0525f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0006d640 con 0x7fcf0009f410 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcf0525f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00122ae0 con 0x7fcf0009e930 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 0x7fcf001c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fceff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00136fe0 0x7fcf0006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf001c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 0x7fcf001c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54106/0 (socket says 172.21.15.32:54106) 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 -- 172.21.15.32:0/2556962084 learned_addr learned my addr 172.21.15.32:0/2556962084 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 -- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 msgr2=0x7fcf001c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf001c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 -- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00136fe0 msgr2=0x7fcf0006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00136fe0 0x7fcf0006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 -- 172.21.15.32:0/2556962084 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4009550 con 0x7fcf0009e930 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefe59c700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf001c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fceff59e700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00136fe0 0x7fcf0006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.515+0000 7fcefed9d700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 0x7fcf001c8520 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcef0001eb0 tx=0x7fcef000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.516+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00054e0 con 0x7fcf0009e930 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.516+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0003680 con 0x7fcf0009e930 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.516+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00039f0 con 0x7fcf0009e930 2023-09-30T22:31:11.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.516+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8009a70 con 0x7fcf0009e930 2023-09-30T22:31:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.516+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf001c8fa0 con 0x7fcf0009e930 2023-09-30T22:31:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.516+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcef0003b50 con 0x7fcf0009e930 2023-09-30T22:31:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.517+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fcef003e9e0 con 0x7fcf0009e930 2023-09-30T22:31:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.517+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf00071d20 con 0x7fcf0009e930 2023-09-30T22:31:11.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.522+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcef001b3b0 con 0x7fcf0009e930 2023-09-30T22:31:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.726+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcf00074ea0 con 0x7fcf0009e930 2023-09-30T22:31:11.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.726+0000 7fcee77fe700 1 -- 172.21.15.32:0/2556962084 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcef0017020 con 0x7fcf0009e930 2023-09-30T22:31:11.727 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.728+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 msgr2=0x7fcf001c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.728+0000 7fcf0525f700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 0x7fcf001c8520 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcef0001eb0 tx=0x7fcef000c800 comp rx=0 tx=0).stop 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 shutdown_connections 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00136fe0 0x7fcf0006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcf0009e930 0x7fcf001c8520 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 --2- 172.21.15.32:0/2556962084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcf0009f410 0x7fcf001c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 >> 172.21.15.32:0/2556962084 conn(0x7fcf000a4210 msgr2=0x7fcf000a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 shutdown_connections 2023-09-30T22:31:11.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:11.729+0000 7fcf0525f700 1 -- 172.21.15.32:0/2556962084 wait complete. 2023-09-30T22:31:11.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:12.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:12.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:12.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 Processor -- start 2023-09-30T22:31:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 -- start start 2023-09-30T22:31:12.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c0a0a30 0x7f7a9c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a9c0a1340 0x7f7a9c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c09edd0 con 0x7f7a9c133fa0 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c09e1f0 con 0x7f7a9c0a1340 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa2d10700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c09dc50 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7a9bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a9c0a1340 0x7f7a9c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa12ad700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7a9c133fa0 0x7f7a9c13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44138/0 (socket says 172.21.15.32:44138) 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7aa12ad700 1 -- 172.21.15.32:0/956178413 learned_addr learned my addr 172.21.15.32:0/956178413 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.311+0000 7f7a9bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a9c0a1340 0x7f7a9c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58910/0 (socket says 172.21.15.32:58910) 2023-09-30T22:31:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7aa0aac700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c0a0a30 0x7f7a9c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9bfff700 1 -- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c0a0a30 msgr2=0x7f7a9c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/956178413 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457887836 0 0) 0x7f7a9c09edd0 con 0x7f7a9c133fa0 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9bfff700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c0a0a30 0x7f7a9c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9bfff700 1 -- 172.21.15.32:0/956178413 >> v1:172.21.15.32:6789/0 conn(0x7f7a9c133fa0 legacy=0x7f7a9c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9bfff700 1 -- 172.21.15.32:0/956178413 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0a1fe0 con 0x7f7a9c0a1340 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7aa0aac700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c0a0a30 0x7f7a9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9bfff700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a1340 0x7f7a9c133a60 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009840 tx=0x7f7a88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4067b86db199678 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/956178413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011830 con 0x7f7a9c0a1340 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/956178413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88011990 con 0x7f7a9c0a1340 2023-09-30T22:31:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.312+0000 7f7a9b7fe700 1 -- 172.21.15.32:0/956178413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011d20 con 0x7f7a9c0a1340 2023-09-30T22:31:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 -- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a1340 msgr2=0x7f7a9c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a1340 0x7f7a9c133a60 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009840 tx=0x7f7a88009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 -- 172.21.15.32:0/956178413 shutdown_connections 2023-09-30T22:31:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c0a0a30 0x7f7a9c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/956178413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a1340 0x7f7a9c133a60 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 -- 172.21.15.32:0/956178413 >> 172.21.15.32:0/956178413 conn(0x7f7a9c0a6550 msgr2=0x7f7a9c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 -- 172.21.15.32:0/956178413 shutdown_connections 2023-09-30T22:31:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.313+0000 7f7aa2d10700 1 -- 172.21.15.32:0/956178413 wait complete. 2023-09-30T22:31:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.314+0000 7f7aa2d10700 1 Processor -- start 2023-09-30T22:31:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.314+0000 7f7aa2d10700 1 -- start start 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa2d10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 0x7f7a9c122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa2d10700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9c0a1340 0x7f7a9c122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa2d10700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c133fa0 0x7f7a9c1232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa2d10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c126f80 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa2d10700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c125d60 con 0x7f7a9c133fa0 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa2d10700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0a3580 con 0x7f7a9c0a1340 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 0x7f7a9c122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 0x7f7a9c122820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58924/0 (socket says 172.21.15.32:58924) 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 -- 172.21.15.32:0/590015396 learned_addr learned my addr 172.21.15.32:0/590015396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa12ad700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c133fa0 0x7f7a9c1232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 -- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9c0a1340 msgr2=0x7f7a9c122d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9c0a1340 0x7f7a9c122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 -- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c133fa0 msgr2=0x7f7a9c1232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c133fa0 0x7f7a9c1232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.315+0000 7f7aa0aac700 1 -- 172.21.15.32:0/590015396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a900094a0 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.316+0000 7f7aa0aac700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 0x7f7a9c122820 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000ebe0 tx=0x7f7a9000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.316+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90009800 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.316+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9000c910 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.316+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880094a0 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.316+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000cc80 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.316+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c124bf0 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.317+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a90013470 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.317+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7a9003fbc0 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.317+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c126b50 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.322+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a90017080 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.349+0000 7fec13abb700 1 Processor -- start 2023-09-30T22:31:12.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.349+0000 7fec13abb700 1 -- start start 2023-09-30T22:31:12.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec13abb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c0a0a30 0x7fec0c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec13abb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec13abb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c09edd0 con 0x7fec0c133fa0 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec13abb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c09e1f0 con 0x7fec0c0a1340 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec13abb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c09dc50 con 0x7fec0c0a0a30 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec12058700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec0c133fa0 0x7fec0c13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44142/0 (socket says 172.21.15.32:44142) 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec11056700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec12058700 1 -- 172.21.15.32:0/2135067050 learned_addr learned my addr 172.21.15.32:0/2135067050 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec11857700 1 --2- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c0a0a30 0x7fec0c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec11056700 1 -- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c0a0a30 msgr2=0x7fec0c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec11056700 1 --2- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c0a0a30 0x7fec0c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec11056700 1 -- 172.21.15.32:0/2135067050 >> v1:172.21.15.32:6789/0 conn(0x7fec0c133fa0 legacy=0x7fec0c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec10855700 1 -- 172.21.15.32:0/2135067050 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129239492 0 0) 0x7fec0c09edd0 con 0x7fec0c133fa0 2023-09-30T22:31:12.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.350+0000 7fec11056700 1 -- 172.21.15.32:0/2135067050 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c0a1fe0 con 0x7fec0c0a1340 2023-09-30T22:31:12.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.351+0000 7fec11056700 1 --2- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c133a60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e59506531f540a9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.351+0000 7fec10855700 1 -- 172.21.15.32:0/2135067050 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc011830 con 0x7fec0c0a1340 2023-09-30T22:31:12.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.351+0000 7fec10855700 1 -- 172.21.15.32:0/2135067050 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc011990 con 0x7fec0c0a1340 2023-09-30T22:31:12.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.351+0000 7fec10855700 1 -- 172.21.15.32:0/2135067050 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc011d20 con 0x7fec0c0a1340 2023-09-30T22:31:12.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.351+0000 7fec13abb700 1 -- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 msgr2=0x7fec0c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.351+0000 7fec13abb700 1 --2- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c133a60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:12.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.352+0000 7fec13abb700 1 -- 172.21.15.32:0/2135067050 shutdown_connections 2023-09-30T22:31:12.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.352+0000 7fec13abb700 1 --2- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c0a0a30 0x7fec0c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.352+0000 7fec13abb700 1 --2- 172.21.15.32:0/2135067050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c133a60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.352+0000 7fec13abb700 1 -- 172.21.15.32:0/2135067050 >> 172.21.15.32:0/2135067050 conn(0x7fec0c0a6550 msgr2=0x7fec0c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.352+0000 7fec13abb700 1 -- 172.21.15.32:0/2135067050 shutdown_connections 2023-09-30T22:31:12.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.352+0000 7fec13abb700 1 -- 172.21.15.32:0/2135067050 wait complete. 2023-09-30T22:31:12.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.353+0000 7fec13abb700 1 Processor -- start 2023-09-30T22:31:12.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.353+0000 7fec13abb700 1 -- start start 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec13abb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c0a0a30 0x7fec0c11e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec13abb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c11edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec13abb700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c133fa0 0x7fec0c11f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec13abb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c11fd60 con 0x7fec0c0a0a30 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec13abb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c120410 con 0x7fec0c0a1340 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec13abb700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c122320 con 0x7fec0c133fa0 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec11056700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c11edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec11056700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c11edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41764/0 (socket says 172.21.15.32:41764) 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec11056700 1 -- 172.21.15.32:0/1384642936 learned_addr learned my addr 172.21.15.32:0/1384642936 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec12058700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c133fa0 0x7fec0c11f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.354+0000 7fec11857700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c0a0a30 0x7fec0c11e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec11056700 1 -- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c133fa0 msgr2=0x7fec0c11f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec11056700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c133fa0 0x7fec0c11f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec11056700 1 -- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c0a0a30 msgr2=0x7fec0c11e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec11056700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c0a0a30 0x7fec0c11e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec11056700 1 -- 172.21.15.32:0/1384642936 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc0094a0 con 0x7fec0c0a1340 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec12058700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c133fa0 0x7fec0c11f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec11056700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c11edd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7febfc0090b0 tx=0x7febfc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc004590 con 0x7fec0c0a1340 2023-09-30T22:31:12.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc01f070 con 0x7fec0c0a1340 2023-09-30T22:31:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c0a25e0 con 0x7fec0c0a1340 2023-09-30T22:31:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0038c0 con 0x7fec0c0a1340 2023-09-30T22:31:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.355+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c122040 con 0x7fec0c0a1340 2023-09-30T22:31:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.356+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febfc003a20 con 0x7fec0c0a1340 2023-09-30T22:31:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.356+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c121ce0 con 0x7fec0c0a1340 2023-09-30T22:31:12.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.356+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7febfc021070 con 0x7fec0c0a1340 2023-09-30T22:31:12.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.361+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec0c121ce0 con 0x7fec0c0a1340 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.391+0000 7f6b7ab5b700 1 Processor -- start 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.391+0000 7f6b7ab5b700 1 -- start start 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b7ab5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 0x7f6b74133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b7ab5b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409e930 0x7f6b7413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b7ab5b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7406ef30 con 0x7f6b7409de50 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b7ab5b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7406fef0 con 0x7f6b741338a0 2023-09-30T22:31:12.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b7ab5b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741334b0 con 0x7f6b7409e930 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b788f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 0x7f6b74133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b788f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 0x7f6b74133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41768/0 (socket says 172.21.15.32:41768) 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b788f7700 1 -- 172.21.15.32:0/257366355 learned_addr learned my addr 172.21.15.32:0/257366355 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b73fff700 1 --1- 172.21.15.32:0/257366355 >> v1:172.21.15.32:6789/0 conn(0x7f6b7409de50 0x7f6b7409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44150/0 (socket says 172.21.15.32:44150) 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b790f8700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409e930 0x7f6b7413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b788f7700 1 -- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409e930 msgr2=0x7f6b7413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.392+0000 7f6b788f7700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409e930 0x7f6b7413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b788f7700 1 -- 172.21.15.32:0/257366355 >> v1:172.21.15.32:6789/0 conn(0x7f6b7409de50 legacy=0x7f6b7409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b788f7700 1 -- 172.21.15.32:0/257366355 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7406fc30 con 0x7f6b741338a0 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b737fe700 1 -- 172.21.15.32:0/257366355 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184973785 0 0) 0x7f6b7406ef30 con 0x7f6b7409de50 2023-09-30T22:31:12.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b790f8700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409e930 0x7f6b7413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b788f7700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 0x7f6b74133c70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009920 tx=0x7f6b68009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a317f2b46620f89 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b737fe700 1 -- 172.21.15.32:0/257366355 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68011830 con 0x7f6b741338a0 2023-09-30T22:31:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b737fe700 1 -- 172.21.15.32:0/257366355 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b68011990 con 0x7f6b741338a0 2023-09-30T22:31:12.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.393+0000 7f6b737fe700 1 -- 172.21.15.32:0/257366355 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68011d20 con 0x7f6b741338a0 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 msgr2=0x7f6b74133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 0x7f6b74133c70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009920 tx=0x7f6b68009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/257366355 shutdown_connections 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409e930 0x7f6b7413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/257366355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b741338a0 0x7f6b74133c70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/257366355 >> 172.21.15.32:0/257366355 conn(0x7f6b740a2aa0 msgr2=0x7f6b740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/257366355 shutdown_connections 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.394+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/257366355 wait complete. 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 Processor -- start 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 -- start start 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 0x7f6b7411d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b7409e930 0x7f6b7411dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b741338a0 0x7f6b7411e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74130d30 con 0x7f6b741338a0 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741211b0 con 0x7f6b7409e930 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.395+0000 7f6b7ab5b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7411f230 con 0x7f6b7409de50 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b73fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b7409e930 0x7f6b7411dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 0x7f6b7411d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b73fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b7409e930 0x7f6b7411dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41772/0 (socket says 172.21.15.32:41772) 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b73fff700 1 -- 172.21.15.32:0/173357040 learned_addr learned my addr 172.21.15.32:0/173357040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 0x7f6b7411d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54138/0 (socket says 172.21.15.32:54138) 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b790f8700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b741338a0 0x7f6b7411e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 -- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b7409e930 msgr2=0x7f6b7411dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b7409e930 0x7f6b7411dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 -- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b741338a0 msgr2=0x7f6b7411e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b741338a0 0x7f6b7411e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 -- 172.21.15.32:0/173357040 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009a70 con 0x7f6b7409de50 2023-09-30T22:31:12.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b788f7700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 0x7f6b7411d710 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6b68016000 tx=0x7f6b68004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68004450 con 0x7f6b7409de50 2023-09-30T22:31:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b680045d0 con 0x7f6b7409de50 2023-09-30T22:31:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68009550 con 0x7f6b7409de50 2023-09-30T22:31:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.396+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680038e0 con 0x7f6b7409de50 2023-09-30T22:31:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.397+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7406f690 con 0x7f6b7409de50 2023-09-30T22:31:12.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.397+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7411a470 con 0x7f6b7409de50 2023-09-30T22:31:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.398+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b68003a40 con 0x7f6b7409de50 2023-09-30T22:31:12.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.398+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6b68040a90 con 0x7f6b7409de50 2023-09-30T22:31:12.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.403+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b6802d350 con 0x7f6b7409de50 2023-09-30T22:31:12.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.535+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7a9c074ea0 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.536+0000 7f7a997fa700 1 -- 172.21.15.32:0/590015396 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a90021020 con 0x7f7a9c0a0a30 2023-09-30T22:31:12.536 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:12.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 msgr2=0x7f7a9c122820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 0x7f7a9c122820 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000ebe0 tx=0x7f7a9000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 shutdown_connections 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a9c133fa0 0x7f7a9c1232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0a0a30 0x7f7a9c122820 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 --2- 172.21.15.32:0/590015396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a9c0a1340 0x7f7a9c122d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 >> 172.21.15.32:0/590015396 conn(0x7f7a9c0a6550 msgr2=0x7f7a9c11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 shutdown_connections 2023-09-30T22:31:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.538+0000 7f7aa2d10700 1 -- 172.21.15.32:0/590015396 wait complete. 2023-09-30T22:31:12.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.594+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fec0c121ce0 con 0x7fec0c0a1340 2023-09-30T22:31:12.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.594+0000 7fec027fc700 1 -- 172.21.15.32:0/1384642936 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec0c121ce0 con 0x7fec0c0a1340 2023-09-30T22:31:12.594 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.596+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 msgr2=0x7fec0c11edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.596+0000 7fec13abb700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c11edd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7febfc0090b0 tx=0x7febfc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 shutdown_connections 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fec0c133fa0 0x7fec0c11f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec0c0a0a30 0x7fec0c11e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 --2- 172.21.15.32:0/1384642936 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fec0c0a1340 0x7fec0c11edd0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 >> 172.21.15.32:0/1384642936 conn(0x7fec0c0a6550 msgr2=0x7fec0c1b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 shutdown_connections 2023-09-30T22:31:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.597+0000 7fec13abb700 1 -- 172.21.15.32:0/1384642936 wait complete. 2023-09-30T22:31:12.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:12.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.632+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6b74074ea0 con 0x7f6b7409de50 2023-09-30T22:31:12.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.633+0000 7f6b717fa700 1 -- 172.21.15.32:0/173357040 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b68034020 con 0x7f6b7409de50 2023-09-30T22:31:12.633 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 msgr2=0x7f6b7411d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 0x7f6b7411d710 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6b68016000 tx=0x7f6b68004750 comp rx=0 tx=0).stop 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 shutdown_connections 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b741338a0 0x7f6b7411e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6b7409de50 0x7f6b7411d710 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 --2- 172.21.15.32:0/173357040 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6b7409e930 0x7f6b7411dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 >> 172.21.15.32:0/173357040 conn(0x7f6b740a2aa0 msgr2=0x7f6b741bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 shutdown_connections 2023-09-30T22:31:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.635+0000 7f6b7ab5b700 1 -- 172.21.15.32:0/173357040 wait complete. 2023-09-30T22:31:12.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:12.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:12.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.884+0000 7f167f687700 1 Processor -- start 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167f687700 1 -- start start 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167f687700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1678131810 0x7f1678131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167f687700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f1678099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167f687700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16781313c0 con 0x7f167809a3b0 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167f687700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167809b580 con 0x7f1678131810 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167f687700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167806e8c0 con 0x7f1678099a40 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167d423700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1678131810 0x7f1678131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167dc24700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f167809a3b0 0x7f1678138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44166/0 (socket says 172.21.15.32:44166) 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167dc24700 1 -- 172.21.15.32:0/2896914620 learned_addr learned my addr 172.21.15.32:0/2896914620 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:12.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.885+0000 7f167cc22700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f1678099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f167d423700 1 -- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 msgr2=0x7f1678099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f167d423700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f1678099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f167d423700 1 -- 172.21.15.32:0/2896914620 >> v1:172.21.15.32:6789/0 conn(0x7f167809a3b0 legacy=0x7f1678138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f167d423700 1 -- 172.21.15.32:0/2896914620 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167809a830 con 0x7f1678131810 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f166ffff700 1 -- 172.21.15.32:0/2896914620 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128873435 0 0) 0x7f16781313c0 con 0x7f167809a3b0 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f167cc22700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f1678099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f167d423700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f1678131be0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f1674009840 tx=0x7f1674009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded1a6b730ed8c97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f166ffff700 1 -- 172.21.15.32:0/2896914620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674011830 con 0x7f1678131810 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f166ffff700 1 -- 172.21.15.32:0/2896914620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674011990 con 0x7f1678131810 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.886+0000 7f166ffff700 1 -- 172.21.15.32:0/2896914620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674011d20 con 0x7f1678131810 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 -- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 msgr2=0x7f1678131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f1678131be0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f1674009840 tx=0x7f1674009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 -- 172.21.15.32:0/2896914620 shutdown_connections 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f1678131be0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 --2- 172.21.15.32:0/2896914620 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f1678099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 -- 172.21.15.32:0/2896914620 >> 172.21.15.32:0/2896914620 conn(0x7f16780a2aa0 msgr2=0x7f16780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 -- 172.21.15.32:0/2896914620 shutdown_connections 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.887+0000 7f167f687700 1 -- 172.21.15.32:0/2896914620 wait complete. 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 Processor -- start 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 -- start start 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f167811a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f167809a3b0 0x7f167811ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f167811b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167811e200 con 0x7f1678131810 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167811d190 con 0x7f1678099a40 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167f687700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f167811bb90 con 0x7f167809a3b0 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167d423700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f167811a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167dc24700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f167811b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167d423700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f167811a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41792/0 (socket says 172.21.15.32:41792) 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167d423700 1 -- 172.21.15.32:0/3981834292 learned_addr learned my addr 172.21.15.32:0/3981834292 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.888+0000 7f167cc22700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f167809a3b0 0x7f167811ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167d423700 1 -- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f167809a3b0 msgr2=0x7f167811ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167d423700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f167809a3b0 0x7f167811ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167d423700 1 -- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 msgr2=0x7f167811b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167d423700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f167811b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167d423700 1 -- 172.21.15.32:0/3981834292 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680094a0 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167dc24700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f167811b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167d423700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f167811a730 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f16740090b0 tx=0x7f1674004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167401a7e0 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167401a940 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16740094a0 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167401acf0 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.889+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167806eb00 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.890+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1674022070 con 0x7f1678099a40 2023-09-30T22:31:12.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.890+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1674040790 con 0x7f1678099a40 2023-09-30T22:31:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.890+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167811c350 con 0x7f1678099a40 2023-09-30T22:31:12.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:12.895+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1674029080 con 0x7f1678099a40 2023-09-30T22:31:13.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.099+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1678074ea0 con 0x7f1678099a40 2023-09-30T22:31:13.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.099+0000 7f166dffb700 1 -- 172.21.15.32:0/3981834292 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1674015020 con 0x7f1678099a40 2023-09-30T22:31:13.099 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 msgr2=0x7f167811a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f167811a730 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f16740090b0 tx=0x7f1674004750 comp rx=0 tx=0).stop 2023-09-30T22:31:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 shutdown_connections 2023-09-30T22:31:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1678131810 0x7f167811b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1678099a40 0x7f167811a730 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 --2- 172.21.15.32:0/3981834292 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f167809a3b0 0x7f167811ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 >> 172.21.15.32:0/3981834292 conn(0x7f16780a2aa0 msgr2=0x7f167812fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 shutdown_connections 2023-09-30T22:31:13.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.102+0000 7f167f687700 1 -- 172.21.15.32:0/3981834292 wait complete. 2023-09-30T22:31:13.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:13.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:13.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:13.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 Processor -- start 2023-09-30T22:31:13.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 -- start start 2023-09-30T22:31:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f413009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 0x7f41301339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413009b120 con 0x7f413009e410 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413006b460 con 0x7f413009d6f0 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.704+0000 7f4135938700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f413009a7e0 con 0x7f413009e000 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 0x7f41301339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 0x7f41301339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54150/0 (socket says 172.21.15.32:54150) 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 -- 172.21.15.32:0/903297337 learned_addr learned my addr 172.21.15.32:0/903297337 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412effd700 1 --2- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f413009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 -- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 msgr2=0x7f413009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 --2- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f413009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 -- 172.21.15.32:0/903297337 >> v1:172.21.15.32:6789/0 conn(0x7f413009e410 legacy=0x7f4130133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 -- 172.21.15.32:0/903297337 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413009a500 con 0x7f413009e000 2023-09-30T22:31:13.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412e7fc700 1 --2- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 0x7f41301339f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4124009950 tx=0x7f4124009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68df26d50161ec63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412dffb700 1 -- 172.21.15.32:0/903297337 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124011830 con 0x7f413009e000 2023-09-30T22:31:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.705+0000 7f412dffb700 1 -- 172.21.15.32:0/903297337 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4124011990 con 0x7f413009e000 2023-09-30T22:31:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.706+0000 7f412dffb700 1 -- 172.21.15.32:0/903297337 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124011d40 con 0x7f413009e000 2023-09-30T22:31:13.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.706+0000 7f4135938700 1 -- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 msgr2=0x7f41301339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.706+0000 7f4135938700 1 --2- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 0x7f41301339f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4124009950 tx=0x7f4124009c60 comp rx=0 tx=0).stop 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.707+0000 7f4135938700 1 -- 172.21.15.32:0/903297337 shutdown_connections 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.707+0000 7f4135938700 1 --2- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f413009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.707+0000 7f4135938700 1 --2- 172.21.15.32:0/903297337 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e000 0x7f41301339f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.707+0000 7f4135938700 1 -- 172.21.15.32:0/903297337 >> 172.21.15.32:0/903297337 conn(0x7f4130066f50 msgr2=0x7f413009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.707+0000 7f4135938700 1 -- 172.21.15.32:0/903297337 shutdown_connections 2023-09-30T22:31:13.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.707+0000 7f4135938700 1 -- 172.21.15.32:0/903297337 wait complete. 2023-09-30T22:31:13.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.708+0000 7f4135938700 1 Processor -- start 2023-09-30T22:31:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.708+0000 7f4135938700 1 -- start start 2023-09-30T22:31:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f4135938700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f4130122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f4135938700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 0x7f4130122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f4135938700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e410 0x7f41301230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f4135938700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130126290 con 0x7f413009e000 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f4135938700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413009c710 con 0x7f413009d6f0 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f4135938700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4130124320 con 0x7f413009e410 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f412e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 0x7f4130122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f412e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 0x7f4130122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58956/0 (socket says 172.21.15.32:58956) 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f412effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f4130122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f412e7fc700 1 -- 172.21.15.32:0/3953652897 learned_addr learned my addr 172.21.15.32:0/3953652897 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.709+0000 7f412f7fe700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e410 0x7f41301230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412effd700 1 -- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e410 msgr2=0x7f41301230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412effd700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e410 0x7f41301230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412effd700 1 -- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 msgr2=0x7f4130122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412effd700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 0x7f4130122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412effd700 1 -- 172.21.15.32:0/3953652897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4124009550 con 0x7f413009d6f0 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412e7fc700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 0x7f4130122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f412effd700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f4130122670 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f412000e900 tx=0x7f412000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000ee50 con 0x7f413009d6f0 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412000c850 con 0x7f413009d6f0 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4130133630 con 0x7f413009d6f0 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000cbc0 con 0x7f413009d6f0 2023-09-30T22:31:13.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.710+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4130126920 con 0x7f413009d6f0 2023-09-30T22:31:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.711+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41300723c0 con 0x7f413009d6f0 2023-09-30T22:31:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.711+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f412000cd20 con 0x7f413009d6f0 2023-09-30T22:31:13.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.712+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f412001a360 con 0x7f413009d6f0 2023-09-30T22:31:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.720+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f412003fe60 con 0x7f413009d6f0 2023-09-30T22:31:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.764+0000 7f4459b6e700 1 Processor -- start 2023-09-30T22:31:13.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.764+0000 7f4459b6e700 1 -- start start 2023-09-30T22:31:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4459b6e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f445409f410 0x7f4454136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4459b6e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f44541393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4459b6e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4454070940 con 0x7f445409e930 2023-09-30T22:31:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4459b6e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445406fd60 con 0x7f445409f410 2023-09-30T22:31:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4459b6e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445406fa80 con 0x7f4454136fe0 2023-09-30T22:31:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f444bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f445409f410 0x7f4454136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f44537fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f445409e930 0x7f445409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44174/0 (socket says 172.21.15.32:44174) 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f444bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f445409f410 0x7f4454136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58968/0 (socket says 172.21.15.32:58968) 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f44537fe700 1 -- 172.21.15.32:0/622160569 learned_addr learned my addr 172.21.15.32:0/622160569 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4453fff700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f44541393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f444bfff700 1 -- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 msgr2=0x7f44541393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f444bfff700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f44541393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f444bfff700 1 -- 172.21.15.32:0/622160569 >> v1:172.21.15.32:6789/0 conn(0x7f445409e930 legacy=0x7f445409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4452ffd700 1 -- 172.21.15.32:0/622160569 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296757060 0 0) 0x7f4454070940 con 0x7f445409e930 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f444bfff700 1 -- 172.21.15.32:0/622160569 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445409fb60 con 0x7f445409f410 2023-09-30T22:31:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.765+0000 7f4453fff700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f44541393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.766+0000 7f444bfff700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f4454136aa0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f443c009840 tx=0x7f443c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d34fe4fef75c8eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.766+0000 7f4452ffd700 1 -- 172.21.15.32:0/622160569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c011830 con 0x7f445409f410 2023-09-30T22:31:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.766+0000 7f4452ffd700 1 -- 172.21.15.32:0/622160569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c011990 con 0x7f445409f410 2023-09-30T22:31:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.766+0000 7f4452ffd700 1 -- 172.21.15.32:0/622160569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c011d20 con 0x7f445409f410 2023-09-30T22:31:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.766+0000 7f4459b6e700 1 -- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 msgr2=0x7f4454136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.766+0000 7f4459b6e700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f4454136aa0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f443c009840 tx=0x7f443c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.767+0000 7f4459b6e700 1 -- 172.21.15.32:0/622160569 shutdown_connections 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.767+0000 7f4459b6e700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f44541393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.767+0000 7f4459b6e700 1 --2- 172.21.15.32:0/622160569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f4454136aa0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.767+0000 7f4459b6e700 1 -- 172.21.15.32:0/622160569 >> 172.21.15.32:0/622160569 conn(0x7f44540a4210 msgr2=0x7f44540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.767+0000 7f4459b6e700 1 -- 172.21.15.32:0/622160569 shutdown_connections 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.767+0000 7f4459b6e700 1 -- 172.21.15.32:0/622160569 wait complete. 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 Processor -- start 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 -- start start 2023-09-30T22:31:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f445409e930 0x7f44541c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f44541c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f445406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454132270 con 0x7f445409f410 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445409de80 con 0x7f4454136fe0 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4459b6e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44541bc160 con 0x7f445409e930 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f44537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f445409e930 0x7f44541c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f444bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f44541c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f4453fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f445406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f44537fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f445409e930 0x7f44541c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54172/0 (socket says 172.21.15.32:54172) 2023-09-30T22:31:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f444bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f44541c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58972/0 (socket says 172.21.15.32:58972) 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.768+0000 7f44537fe700 1 -- 172.21.15.32:0/1326749095 learned_addr learned my addr 172.21.15.32:0/1326749095 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4453fff700 1 -- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f445409e930 msgr2=0x7f44541c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4453fff700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f445409e930 0x7f44541c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4453fff700 1 -- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 msgr2=0x7f44541c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4453fff700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f44541c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4453fff700 1 -- 172.21.15.32:0/1326749095 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009a70 con 0x7f4454136fe0 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4453fff700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f445406b8b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f444c00efc0 tx=0x7f444c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00ca90 con 0x7f4454136fe0 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c00f040 con 0x7f4454136fe0 2023-09-30T22:31:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c0094a0 con 0x7f4454136fe0 2023-09-30T22:31:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00ce30 con 0x7f4454136fe0 2023-09-30T22:31:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.769+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44541c8f50 con 0x7f4454136fe0 2023-09-30T22:31:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.770+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f444c014480 con 0x7f4454136fe0 2023-09-30T22:31:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.770+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f444c03fcb0 con 0x7f4454136fe0 2023-09-30T22:31:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.770+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4454074ea0 con 0x7f4454136fe0 2023-09-30T22:31:13.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.779+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f444c012910 con 0x7f4454136fe0 2023-09-30T22:31:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.805+0000 7f572abdd700 1 Processor -- start 2023-09-30T22:31:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.805+0000 7f572abdd700 1 -- start start 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572abdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f572409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572abdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572abdd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572409b120 con 0x7f572409e410 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572abdd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572406b460 con 0x7f572409d6f0 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572abdd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f572409a7e0 con 0x7f572409e000 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572917a700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f572409e410 0x7f5724133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44186/0 (socket says 172.21.15.32:44186) 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f572917a700 1 -- 172.21.15.32:0/1212874444 learned_addr learned my addr 172.21.15.32:0/1212874444 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5723fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5728979700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f572409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5728979700 1 -- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 msgr2=0x7f57241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5728979700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5728979700 1 -- 172.21.15.32:0/1212874444 >> v1:172.21.15.32:6789/0 conn(0x7f572409e410 legacy=0x7f5724133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5728979700 1 -- 172.21.15.32:0/1212874444 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572409a500 con 0x7f572409d6f0 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.806+0000 7f5723fff700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f5728979700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f572409dac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5718009870 tx=0x7f5718009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c68b31329e8d80f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f57237fe700 1 -- 172.21.15.32:0/1212874444 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718011830 con 0x7f572409d6f0 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f57237fe700 1 -- 172.21.15.32:0/1212874444 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5718011990 con 0x7f572409d6f0 2023-09-30T22:31:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f57237fe700 1 -- 172.21.15.32:0/1212874444 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718011d20 con 0x7f572409d6f0 2023-09-30T22:31:13.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 -- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 msgr2=0x7f572409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f572409dac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5718009870 tx=0x7f5718009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 -- 172.21.15.32:0/1212874444 shutdown_connections 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f572409dac0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 --2- 172.21.15.32:0/1212874444 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 -- 172.21.15.32:0/1212874444 >> 172.21.15.32:0/1212874444 conn(0x7f5724066f50 msgr2=0x7f572409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 -- 172.21.15.32:0/1212874444 shutdown_connections 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.807+0000 7f572abdd700 1 -- 172.21.15.32:0/1212874444 wait complete. 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.808+0000 7f572abdd700 1 Processor -- start 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 -- start start 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f5724122490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241229d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f572409e410 0x7f5724122f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5724125a10 con 0x7f572409e410 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57241238f0 con 0x7f572409d6f0 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572abdd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5724124900 con 0x7f572409e000 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f5728979700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f5724122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f572917a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f572409e410 0x7f5724122f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f5723fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f5728979700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f5724122490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41874/0 (socket says 172.21.15.32:41874) 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f5723fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241229d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54184/0 (socket says 172.21.15.32:54184) 2023-09-30T22:31:13.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.809+0000 7f5728979700 1 -- 172.21.15.32:0/3140195074 learned_addr learned my addr 172.21.15.32:0/3140195074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572917a700 1 -- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 msgr2=0x7f57241229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572917a700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572917a700 1 -- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 msgr2=0x7f5724122490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572917a700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f5724122490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572917a700 1 -- 172.21.15.32:0/3140195074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100094a0 con 0x7f572409e410 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f5723fff700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f5728979700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f5724122490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572917a700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f572409e410 0x7f5724122f10 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f571c00caa0 tx=0x7f571c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00a960 con 0x7f572409e410 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c00aac0 con 0x7f572409e410 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00ae30 con 0x7f572409e410 2023-09-30T22:31:13.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57180094a0 con 0x7f572409e410 2023-09-30T22:31:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.810+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5724125cd0 con 0x7f572409e410 2023-09-30T22:31:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.811+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f571c012400 con 0x7f572409e410 2023-09-30T22:31:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.811+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f571c03f140 con 0x7f572409e410 2023-09-30T22:31:13.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.812+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5724074ea0 con 0x7f572409e410 2023-09-30T22:31:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.817+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f571c017080 con 0x7f572409e410 2023-09-30T22:31:13.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.930+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4130074ea0 con 0x7f413009d6f0 2023-09-30T22:31:13.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.931+0000 7f4117fff700 1 -- 172.21.15.32:0/3953652897 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4120017070 con 0x7f413009d6f0 2023-09-30T22:31:13.931 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 msgr2=0x7f4130122670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f4130122670 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f412000e900 tx=0x7f412000ec10 comp rx=0 tx=0).stop 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 shutdown_connections 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f413009e410 0x7f41301230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f413009d6f0 0x7f4130122670 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 --2- 172.21.15.32:0/3953652897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f413009e000 0x7f4130122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.933+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 >> 172.21.15.32:0/3953652897 conn(0x7f4130066f50 msgr2=0x7f4130070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.934+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 shutdown_connections 2023-09-30T22:31:13.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.934+0000 7f4135938700 1 -- 172.21.15.32:0/3953652897 wait complete. 2023-09-30T22:31:13.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.991+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f445406b250 con 0x7f4454136fe0 2023-09-30T22:31:13.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.991+0000 7f4450ff9700 1 -- 172.21.15.32:0/1326749095 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f444c012340 con 0x7f4454136fe0 2023-09-30T22:31:13.991 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:13.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 msgr2=0x7f445406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f445406b8b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f444c00efc0 tx=0x7f444c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 shutdown_connections 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4454136fe0 0x7f445406b8b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f445409e930 0x7f44541c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 --2- 172.21.15.32:0/1326749095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f445409f410 0x7f44541c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 >> 172.21.15.32:0/1326749095 conn(0x7f44540a4210 msgr2=0x7f4454134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 shutdown_connections 2023-09-30T22:31:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:13.994+0000 7f4459b6e700 1 -- 172.21.15.32:0/1326749095 wait complete. 2023-09-30T22:31:14.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.025+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f572406c360 con 0x7f572409e410 2023-09-30T22:31:14.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.025+0000 7f57217fa700 1 -- 172.21.15.32:0/3140195074 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f571c010710 con 0x7f572409e410 2023-09-30T22:31:14.025 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f572409e410 msgr2=0x7f5724122f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f572409e410 0x7f5724122f10 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f571c00caa0 tx=0x7f571c00cdb0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 shutdown_connections 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f572409e410 0x7f5724122f10 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f572409d6f0 0x7f5724122490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 --2- 172.21.15.32:0/3140195074 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f572409e000 0x7f57241229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 >> 172.21.15.32:0/3140195074 conn(0x7f5724066f50 msgr2=0x7f57240a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 shutdown_connections 2023-09-30T22:31:14.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.027+0000 7f572abdd700 1 -- 172.21.15.32:0/3140195074 wait complete. 2023-09-30T22:31:14.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:14.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.259+0000 7f31fe37b700 1 Processor -- start 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.259+0000 7f31fe37b700 1 -- start start 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.259+0000 7f31fe37b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.259+0000 7f31fe37b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.259+0000 7f31fe37b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f809f130 con 0x7f31f809d6f0 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.259+0000 7f31fe37b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f809f520 con 0x7f31f8135e50 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fe37b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31f809e930 con 0x7f31f809e1d0 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f31f809d6f0 0x7f31f809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44188/0 (socket says 172.21.15.32:44188) 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fc918700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f7fff700 1 -- 172.21.15.32:0/3897467384 learned_addr learned my addr 172.21.15.32:0/3897467384 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f6ffd700 1 -- 172.21.15.32:0/3897467384 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414547659 0 0) 0x7f31f809f130 con 0x7f31f809d6f0 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f6ffd700 1 -- 172.21.15.32:0/3897467384 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e00035b0 con 0x7f31f809d6f0 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fc918700 1 -- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 msgr2=0x7f31f8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fc918700 1 --2- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fc918700 1 -- 172.21.15.32:0/3897467384 >> v1:172.21.15.32:6789/0 conn(0x7f31f809d6f0 legacy=0x7f31f809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fc918700 1 -- 172.21.15.32:0/3897467384 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f809b2d0 con 0x7f31f8135e50 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f77fe700 1 --2- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31fc918700 1 --2- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f8138230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009dc0 tx=0x7f31f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4698ffe6332df97f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f6ffd700 1 -- 172.21.15.32:0/3897467384 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000e070 con 0x7f31f8135e50 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f6ffd700 1 -- 172.21.15.32:0/3897467384 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f000bb20 con 0x7f31f8135e50 2023-09-30T22:31:14.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.260+0000 7f31f6ffd700 1 -- 172.21.15.32:0/3897467384 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0012430 con 0x7f31f8135e50 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 -- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 msgr2=0x7f31f8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f8138230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009dc0 tx=0x7f31f000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 -- 172.21.15.32:0/3897467384 shutdown_connections 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f8138230 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3897467384 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 -- 172.21.15.32:0/3897467384 >> 172.21.15.32:0/3897467384 conn(0x7f31f8066f50 msgr2=0x7f31f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 -- 172.21.15.32:0/3897467384 shutdown_connections 2023-09-30T22:31:14.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.261+0000 7f31fe37b700 1 -- 172.21.15.32:0/3897467384 wait complete. 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.262+0000 7f31fe37b700 1 Processor -- start 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.262+0000 7f31fe37b700 1 -- start start 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fe37b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 0x7f31f811d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fe37b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f811d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fe37b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f811de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fe37b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f8131d50 con 0x7f31f809d6f0 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fe37b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f811f0c0 con 0x7f31f8135e50 2023-09-30T22:31:14.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fe37b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31f8120be0 con 0x7f31f809e1d0 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 0x7f31f811d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f811d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 0x7f31f811d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58994/0 (socket says 172.21.15.32:58994) 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 -- 172.21.15.32:0/3026946933 learned_addr learned my addr 172.21.15.32:0/3026946933 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f77fe700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f811d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54214/0 (socket says 172.21.15.32:54214) 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fc918700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f811de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 -- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 msgr2=0x7f31f811d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f811d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 -- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 msgr2=0x7f31f811de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f811de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f7fff700 1 -- 172.21.15.32:0/3026946933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0094a0 con 0x7f31f809d6f0 2023-09-30T22:31:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31f77fe700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f811d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.263+0000 7f31fc918700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f811de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.264+0000 7f31f7fff700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 0x7f31f811d3a0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f31e8001eb0 tx=0x7f31e800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.264+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8005d70 con 0x7f31f809d6f0 2023-09-30T22:31:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.264+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80070d0 con 0x7f31f809d6f0 2023-09-30T22:31:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.264+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0009a70 con 0x7f31f809d6f0 2023-09-30T22:31:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.264+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8007460 con 0x7f31f809d6f0 2023-09-30T22:31:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.264+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f811f7a0 con 0x7f31f809d6f0 2023-09-30T22:31:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.265+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31e80075c0 con 0x7f31f809d6f0 2023-09-30T22:31:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.265+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f31e803eaa0 con 0x7f31f809d6f0 2023-09-30T22:31:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.265+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f8074ea0 con 0x7f31f809d6f0 2023-09-30T22:31:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.270+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31e8017070 con 0x7f31f809d6f0 2023-09-30T22:31:14.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.476+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31f806c360 con 0x7f31f809d6f0 2023-09-30T22:31:14.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.476+0000 7f31f4ff9700 1 -- 172.21.15.32:0/3026946933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31e8017070 con 0x7f31f809d6f0 2023-09-30T22:31:14.476 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 msgr2=0x7f31f811d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 0x7f31f811d3a0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f31e8001eb0 tx=0x7f31e800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 shutdown_connections 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31f8135e50 0x7f31f811de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31f809d6f0 0x7f31f811d3a0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 --2- 172.21.15.32:0/3026946933 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f31f809e1d0 0x7f31f811d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 >> 172.21.15.32:0/3026946933 conn(0x7f31f8066f50 msgr2=0x7f31f8071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 shutdown_connections 2023-09-30T22:31:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:14.478+0000 7f31fe37b700 1 -- 172.21.15.32:0/3026946933 wait complete. 2023-09-30T22:31:14.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:14.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:15.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:15.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:15.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.097+0000 7fbbfbc58700 1 Processor -- start 2023-09-30T22:31:15.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.097+0000 7fbbfbc58700 1 -- start start 2023-09-30T22:31:15.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbfbc58700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a1510 0x7fbbf4138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbfbc58700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbf4139250 0x7fbbf413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbfbc58700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf409edd0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbfbc58700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf40a18e0 con 0x7fbbf4139250 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbfbc58700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf409e1f0 con 0x7fbbf40a1510 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbfa1f5700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbf4139250 0x7fbbf413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf99f4700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbbf40a0a30 0x7fbbf40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44196/0 (socket says 172.21.15.32:44196) 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf99f4700 1 -- 172.21.15.32:0/144668939 learned_addr learned my addr 172.21.15.32:0/144668939 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf91f3700 1 --2- 172.21.15.32:0/144668939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a1510 0x7fbbf4138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf89f2700 1 -- 172.21.15.32:0/144668939 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077621660 0 0) 0x7fbbf409edd0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf89f2700 1 -- 172.21.15.32:0/144668939 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe00035b0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf91f3700 1 -- 172.21.15.32:0/144668939 >> v2:172.21.15.32:3300/0 conn(0x7fbbf4139250 msgr2=0x7fbbf413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf91f3700 1 --2- 172.21.15.32:0/144668939 >> v2:172.21.15.32:3300/0 conn(0x7fbbf4139250 0x7fbbf413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf91f3700 1 -- 172.21.15.32:0/144668939 >> v1:172.21.15.32:6789/0 conn(0x7fbbf40a0a30 legacy=0x7fbbf40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.098+0000 7fbbf91f3700 1 -- 172.21.15.32:0/144668939 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf409dc50 con 0x7fbbf40a1510 2023-09-30T22:31:15.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.099+0000 7fbbf91f3700 1 --2- 172.21.15.32:0/144668939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a1510 0x7fbbf4138d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009870 tx=0x7fbbe4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67985115e38764de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.099+0000 7fbbf89f2700 1 -- 172.21.15.32:0/144668939 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4011830 con 0x7fbbf40a1510 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.099+0000 7fbbf89f2700 1 -- 172.21.15.32:0/144668939 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4011990 con 0x7fbbf40a1510 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.099+0000 7fbbf89f2700 1 -- 172.21.15.32:0/144668939 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4011d20 con 0x7fbbf40a1510 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 -- 172.21.15.32:0/144668939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a1510 msgr2=0x7fbbf4138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/144668939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a1510 0x7fbbf4138d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009870 tx=0x7fbbe4009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 -- 172.21.15.32:0/144668939 shutdown_connections 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/144668939 >> v2:172.21.15.32:3300/0 conn(0x7fbbf4139250 0x7fbbf413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/144668939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a1510 0x7fbbf4138d10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 -- 172.21.15.32:0/144668939 >> 172.21.15.32:0/144668939 conn(0x7fbbf40a6550 msgr2=0x7fbbf40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 -- 172.21.15.32:0/144668939 shutdown_connections 2023-09-30T22:31:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.100+0000 7fbbfbc58700 1 -- 172.21.15.32:0/144668939 wait complete. 2023-09-30T22:31:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 Processor -- start 2023-09-30T22:31:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 -- start start 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 0x7fbbf41227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbf40a1510 0x7fbbf4122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf4139250 0x7fbbf4123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf41386f0 con 0x7fbbf4139250 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf4126350 con 0x7fbbf40a0a30 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.101+0000 7fbbfbc58700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf4124bc0 con 0x7fbbf40a1510 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 0x7fbbf41227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 0x7fbbf41227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41914/0 (socket says 172.21.15.32:41914) 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 -- 172.21.15.32:0/3621160152 learned_addr learned my addr 172.21.15.32:0/3621160152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:15.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbfa1f5700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf4139250 0x7fbbf4123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf91f3700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbf40a1510 0x7fbbf4122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 -- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbf40a1510 msgr2=0x7fbbf4122d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbf40a1510 0x7fbbf4122d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 -- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf4139250 msgr2=0x7fbbf4123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf4139250 0x7fbbf4123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 -- 172.21.15.32:0/3621160152 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40094a0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbfa1f5700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf4139250 0x7fbbf4123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbf99f4700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 0x7fbbf41227e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c700 tx=0x7fbbf000ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0005d70 con 0x7fbbf40a0a30 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf00070d0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf4125a60 con 0x7fbbf40a0a30 2023-09-30T22:31:15.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0007460 con 0x7fbbf40a0a30 2023-09-30T22:31:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.102+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf40a0280 con 0x7fbbf40a0a30 2023-09-30T22:31:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.103+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbf00075c0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.104+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fbbf003ebd0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.104+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf4074ea0 con 0x7fbbf40a0a30 2023-09-30T22:31:15.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.109+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbf0017020 con 0x7fbbf40a0a30 2023-09-30T22:31:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.153+0000 7f7c94a25700 1 Processor -- start 2023-09-30T22:31:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.153+0000 7f7c94a25700 1 -- start start 2023-09-30T22:31:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c94a25700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c9009e930 0x7f7c9009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c94a25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009f650 0x7f7c90131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c94a25700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c90070940 con 0x7f7c9009f240 2023-09-30T22:31:15.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c94a25700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9006fd60 con 0x7f7c9009e930 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c94a25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9006fa80 con 0x7f7c9009f650 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c8e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c9009e930 0x7f7c9009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c8e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7c9009e930 0x7f7c9009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59026/0 (socket says 172.21.15.32:59026) 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c8ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009f650 0x7f7c90131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.154+0000 7f7c8e59c700 1 -- 172.21.15.32:0/3954943081 learned_addr learned my addr 172.21.15.32:0/3954943081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8ed9d700 1 -- 172.21.15.32:0/3954943081 >> v2:172.21.15.32:3300/0 conn(0x7f7c9009e930 msgr2=0x7f7c9009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8ed9d700 1 --2- 172.21.15.32:0/3954943081 >> v2:172.21.15.32:3300/0 conn(0x7f7c9009e930 0x7f7c9009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8ed9d700 1 -- 172.21.15.32:0/3954943081 >> v1:172.21.15.32:6789/0 conn(0x7f7c9009f240 legacy=0x7f7c901317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8d59a700 1 -- 172.21.15.32:0/3954943081 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274840691 0 0) 0x7f7c90070940 con 0x7f7c9009f240 2023-09-30T22:31:15.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8ed9d700 1 -- 172.21.15.32:0/3954943081 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9009fb60 con 0x7f7c9009f650 2023-09-30T22:31:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8ed9d700 1 --2- 172.21.15.32:0/3954943081 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009f650 0x7f7c90131f00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009840 tx=0x7f7c88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed81357245b68eb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8d59a700 1 -- 172.21.15.32:0/3954943081 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88011830 con 0x7f7c9009f650 2023-09-30T22:31:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8d59a700 1 -- 172.21.15.32:0/3954943081 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c88011990 con 0x7f7c9009f650 2023-09-30T22:31:15.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.155+0000 7f7c8d59a700 1 -- 172.21.15.32:0/3954943081 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88011d20 con 0x7f7c9009f650 2023-09-30T22:31:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 -- 172.21.15.32:0/3954943081 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009f650 msgr2=0x7f7c90131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 --2- 172.21.15.32:0/3954943081 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009f650 0x7f7c90131f00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009840 tx=0x7f7c88009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:15.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 -- 172.21.15.32:0/3954943081 shutdown_connections 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 --2- 172.21.15.32:0/3954943081 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009f650 0x7f7c90131f00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 --2- 172.21.15.32:0/3954943081 >> v2:172.21.15.32:3300/0 conn(0x7f7c9009e930 0x7f7c9009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 -- 172.21.15.32:0/3954943081 >> 172.21.15.32:0/3954943081 conn(0x7f7c900a4210 msgr2=0x7f7c900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 -- 172.21.15.32:0/3954943081 shutdown_connections 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.156+0000 7f7c94a25700 1 -- 172.21.15.32:0/3954943081 wait complete. 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.157+0000 7f7c94a25700 1 Processor -- start 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.157+0000 7f7c94a25700 1 -- start start 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c94a25700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009e930 0x7f7c901a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c94a25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 0x7f7c901a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c94a25700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c9009f650 0x7f7c901c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c94a25700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901a7250 con 0x7f7c9009f240 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c94a25700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901a6730 con 0x7f7c9009e930 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c94a25700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901a6f90 con 0x7f7c9009f650 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c8dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 0x7f7c901a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c8dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 0x7f7c901a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59032/0 (socket says 172.21.15.32:59032) 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c8ed9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c9009f650 0x7f7c901c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c8dd9b700 1 -- 172.21.15.32:0/1391446056 learned_addr learned my addr 172.21.15.32:0/1391446056 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:15.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.158+0000 7f7c8e59c700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009e930 0x7f7c901a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c8dd9b700 1 -- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c9009f650 msgr2=0x7f7c901c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c8dd9b700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c9009f650 0x7f7c901c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c8dd9b700 1 -- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009e930 msgr2=0x7f7c901a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c8dd9b700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009e930 0x7f7c901a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c8dd9b700 1 -- 172.21.15.32:0/1391446056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880094a0 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c8dd9b700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 0x7f7c901a4910 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f7c84001eb0 tx=0x7f7c8400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c840054e0 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c84003680 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c900a0190 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c840039f0 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.159+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901a6290 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.160+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c901a6ca0 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.161+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c84003b50 con 0x7f7c9009f240 2023-09-30T22:31:15.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.161+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f7c8403ea30 con 0x7f7c9009f240 2023-09-30T22:31:15.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.170+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c8401b3b0 con 0x7f7c9009f240 2023-09-30T22:31:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.197+0000 7f85f9a16700 1 Processor -- start 2023-09-30T22:31:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.197+0000 7f85f9a16700 1 -- start start 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f9a16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f409e1d0 0x7f85f4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f9a16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f4139170 0x7f85f413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f9a16700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f409b120 con 0x7f85f409d6f0 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f9a16700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406b460 con 0x7f85f409e1d0 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f9a16700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f409a7e0 con 0x7f85f4139170 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f2ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f85f409d6f0 0x7f85f409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44216/0 (socket says 172.21.15.32:44216) 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f2ffd700 1 -- 172.21.15.32:0/3347329054 learned_addr learned my addr 172.21.15.32:0/3347329054 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f27fc700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f409e1d0 0x7f85f4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f37fe700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f4139170 0x7f85f413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f1ffb700 1 -- 172.21.15.32:0/3347329054 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120917720 0 0) 0x7f85f409b120 con 0x7f85f409d6f0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f1ffb700 1 -- 172.21.15.32:0/3347329054 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85dc0035b0 con 0x7f85f409d6f0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f27fc700 1 -- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f4139170 msgr2=0x7f85f413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f27fc700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f4139170 0x7f85f413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f27fc700 1 -- 172.21.15.32:0/3347329054 >> v1:172.21.15.32:6789/0 conn(0x7f85f409d6f0 legacy=0x7f85f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f27fc700 1 -- 172.21.15.32:0/3347329054 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f409a500 con 0x7f85f409e1d0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.198+0000 7f85f37fe700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f4139170 0x7f85f413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.199+0000 7f85f27fc700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f409e1d0 0x7f85f4138c30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009870 tx=0x7f85e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d08d27dc8446664 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.199+0000 7f85f1ffb700 1 -- 172.21.15.32:0/3347329054 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8011830 con 0x7f85f409e1d0 2023-09-30T22:31:15.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.199+0000 7f85f1ffb700 1 -- 172.21.15.32:0/3347329054 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8011990 con 0x7f85f409e1d0 2023-09-30T22:31:15.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.199+0000 7f85f1ffb700 1 -- 172.21.15.32:0/3347329054 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8011d00 con 0x7f85f409e1d0 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 -- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f409e1d0 msgr2=0x7f85f4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f409e1d0 0x7f85f4138c30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009870 tx=0x7f85e8009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 -- 172.21.15.32:0/3347329054 shutdown_connections 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f4139170 0x7f85f413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 --2- 172.21.15.32:0/3347329054 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f409e1d0 0x7f85f4138c30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 -- 172.21.15.32:0/3347329054 >> 172.21.15.32:0/3347329054 conn(0x7f85f4066f50 msgr2=0x7f85f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 -- 172.21.15.32:0/3347329054 shutdown_connections 2023-09-30T22:31:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.200+0000 7f85f9a16700 1 -- 172.21.15.32:0/3347329054 wait complete. 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.201+0000 7f85f9a16700 1 Processor -- start 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.201+0000 7f85f9a16700 1 -- start start 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.201+0000 7f85f9a16700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f409d6f0 0x7f85f41218f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f9a16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 0x7f85f4121e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f9a16700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 0x7f85f4122370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f9a16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4125130 con 0x7f85f409e1d0 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f9a16700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4123610 con 0x7f85f4139170 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f9a16700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40a1760 con 0x7f85f409d6f0 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 0x7f85f4121e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 0x7f85f4122370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 0x7f85f4121e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59048/0 (socket says 172.21.15.32:59048) 2023-09-30T22:31:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f37fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 0x7f85f4122370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41936/0 (socket says 172.21.15.32:41936) 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f409d6f0 0x7f85f41218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 -- 172.21.15.32:0/873513043 learned_addr learned my addr 172.21.15.32:0/873513043 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 -- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f409d6f0 msgr2=0x7f85f41218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f409d6f0 0x7f85f41218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 -- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 msgr2=0x7f85f4122370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 0x7f85f4122370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 -- 172.21.15.32:0/873513043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80094a0 con 0x7f85f409e1d0 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f2ffd700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f409d6f0 0x7f85f41218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f37fe700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 0x7f85f4122370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.202+0000 7f85f27fc700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 0x7f85f4121e30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009bc0 tx=0x7f85e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.203+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80045d0 con 0x7f85f409e1d0 2023-09-30T22:31:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.203+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e801f070 con 0x7f85f409e1d0 2023-09-30T22:31:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.203+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80038c0 con 0x7f85f409e1d0 2023-09-30T22:31:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.203+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f409c710 con 0x7f85f409e1d0 2023-09-30T22:31:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.203+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f4123da0 con 0x7f85f409e1d0 2023-09-30T22:31:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.204+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f4074ea0 con 0x7f85f409e1d0 2023-09-30T22:31:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.204+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85e8003a20 con 0x7f85f409e1d0 2023-09-30T22:31:15.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.204+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f85e8021070 con 0x7f85f409e1d0 2023-09-30T22:31:15.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.213+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85e8024380 con 0x7f85f409e1d0 2023-09-30T22:31:15.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.333+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbbf40a1d10 con 0x7fbbf40a0a30 2023-09-30T22:31:15.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.334+0000 7fbbea7fc700 1 -- 172.21.15.32:0/3621160152 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbf0018260 con 0x7fbbf40a0a30 2023-09-30T22:31:15.334 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:15.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 msgr2=0x7fbbf41227e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 0x7fbbf41227e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c700 tx=0x7fbbf000ca10 comp rx=0 tx=0).stop 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 shutdown_connections 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbf4139250 0x7fbbf4123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbf40a0a30 0x7fbbf41227e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 --2- 172.21.15.32:0/3621160152 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fbbf40a1510 0x7fbbf4122d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 >> 172.21.15.32:0/3621160152 conn(0x7fbbf40a6550 msgr2=0x7fbbf40a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 shutdown_connections 2023-09-30T22:31:15.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.337+0000 7fbbfbc58700 1 -- 172.21.15.32:0/3621160152 wait complete. 2023-09-30T22:31:15.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:15.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.382+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c90074ea0 con 0x7f7c9009f240 2023-09-30T22:31:15.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.382+0000 7f7c7effd700 1 -- 172.21.15.32:0/1391446056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c84017030 con 0x7f7c9009f240 2023-09-30T22:31:15.382 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:15.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 msgr2=0x7f7c901a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 0x7f7c901a4910 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f7c84001eb0 tx=0x7f7c8400c800 comp rx=0 tx=0).stop 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 shutdown_connections 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7c9009f650 0x7f7c901c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c9009e930 0x7f7c901a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 --2- 172.21.15.32:0/1391446056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c9009f240 0x7f7c901a4910 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 >> 172.21.15.32:0/1391446056 conn(0x7f7c900a4210 msgr2=0x7f7c901b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 shutdown_connections 2023-09-30T22:31:15.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.385+0000 7f7c94a25700 1 -- 172.21.15.32:0/1391446056 wait complete. 2023-09-30T22:31:15.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.433+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f85f406acb0 con 0x7f85f409e1d0 2023-09-30T22:31:15.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.433+0000 7f85dbfff700 1 -- 172.21.15.32:0/873513043 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85e8015020 con 0x7f85f409e1d0 2023-09-30T22:31:15.433 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:15.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 msgr2=0x7f85f4121e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 0x7f85f4121e30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009bc0 tx=0x7f85e8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 shutdown_connections 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f85f4139170 0x7f85f4122370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f85f409d6f0 0x7f85f41218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 --2- 172.21.15.32:0/873513043 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85f409e1d0 0x7f85f4121e30 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.435+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 >> 172.21.15.32:0/873513043 conn(0x7f85f4066f50 msgr2=0x7f85f409b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.436+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 shutdown_connections 2023-09-30T22:31:15.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.436+0000 7f85f9a16700 1 -- 172.21.15.32:0/873513043 wait complete. 2023-09-30T22:31:15.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:15.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.633+0000 7f512b7fd700 1 Processor -- start 2023-09-30T22:31:15.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 -- start start 2023-09-30T22:31:15.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f512406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 0x7f5124131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5124131b40 0x7f5124131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512409ece0 con 0x7f5124131b40 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512409e760 con 0x7f512406a770 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f512b7fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f512409e040 con 0x7f512406b080 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f5129599700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f512406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f5128d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 0x7f5124131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f5129d9a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5124131b40 0x7f5124131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f5129599700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f512406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41946/0 (socket says 172.21.15.32:41946) 2023-09-30T22:31:15.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f5128d98700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 0x7f5124131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54234/0 (socket says 172.21.15.32:54234) 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.634+0000 7f5129599700 1 -- 172.21.15.32:0/3497293359 learned_addr learned my addr 172.21.15.32:0/3497293359 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129599700 1 -- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 msgr2=0x7f5124131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129599700 1 --2- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 0x7f5124131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129599700 1 -- 172.21.15.32:0/3497293359 >> v2:172.21.15.32:3300/0 conn(0x7f5124131b40 msgr2=0x7f5124131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129599700 1 --2- 172.21.15.32:0/3497293359 >> v2:172.21.15.32:3300/0 conn(0x7f5124131b40 0x7f5124131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129599700 1 -- 172.21.15.32:0/3497293359 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512409f630 con 0x7f512406a770 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5128d98700 1 --2- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 0x7f5124131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129d9a700 1 --2- 172.21.15.32:0/3497293359 >> v2:172.21.15.32:3300/0 conn(0x7f5124131b40 0x7f5124131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5129599700 1 --2- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f512406ab40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f51200098f0 tx=0x7f5120009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25057386fdb2234c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5117fff700 1 -- 172.21.15.32:0/3497293359 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120011830 con 0x7f512406a770 2023-09-30T22:31:15.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5117fff700 1 -- 172.21.15.32:0/3497293359 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120011990 con 0x7f512406a770 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f5117fff700 1 -- 172.21.15.32:0/3497293359 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120011d20 con 0x7f512406a770 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f512b7fd700 1 -- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 msgr2=0x7f512406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.635+0000 7f512b7fd700 1 --2- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f512406ab40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f51200098f0 tx=0x7f5120009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 -- 172.21.15.32:0/3497293359 shutdown_connections 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 --2- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f512406ab40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 --2- 172.21.15.32:0/3497293359 >> v2:172.21.15.32:3300/0 conn(0x7f5124131b40 0x7f5124131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 --2- 172.21.15.32:0/3497293359 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f512406b080 0x7f5124131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 -- 172.21.15.32:0/3497293359 >> 172.21.15.32:0/3497293359 conn(0x7f5124066f50 msgr2=0x7f512409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 -- 172.21.15.32:0/3497293359 shutdown_connections 2023-09-30T22:31:15.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.636+0000 7f512b7fd700 1 -- 172.21.15.32:0/3497293359 wait complete. 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 Processor -- start 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 -- start start 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f5124120170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512406b080 0x7f51241206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5124131b40 0x7f5124120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5124122aa0 con 0x7f512406b080 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5124122570 con 0x7f512406a770 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f512b7fd700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5124121ca0 con 0x7f5124131b40 2023-09-30T22:31:15.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f5129599700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f5124120170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f5128d98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512406b080 0x7f51241206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f5129599700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f5124120170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41962/0 (socket says 172.21.15.32:41962) 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f5129599700 1 -- 172.21.15.32:0/1678670343 learned_addr learned my addr 172.21.15.32:0/1678670343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.637+0000 7f5129d9a700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5124131b40 0x7f5124120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5129599700 1 -- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5124131b40 msgr2=0x7f5124120bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5129599700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5124131b40 0x7f5124120bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5129599700 1 -- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512406b080 msgr2=0x7f51241206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5129599700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512406b080 0x7f51241206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5129599700 1 -- 172.21.15.32:0/1678670343 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180094a0 con 0x7f512406a770 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5128d98700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512406b080 0x7f51241206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:15.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5129599700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f5124120170 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5120000c00 tx=0x7f5120004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120004240 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512001f070 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c009a70 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51200045c0 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.638+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120009550 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.639+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5120021030 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.639+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f5120040820 con 0x7f512406a770 2023-09-30T22:31:15.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.639+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5124074ea0 con 0x7f512406a770 2023-09-30T22:31:15.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.644+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f512001a450 con 0x7f512406a770 2023-09-30T22:31:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.847+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f512406c360 con 0x7f512406a770 2023-09-30T22:31:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.847+0000 7f5115ffb700 1 -- 172.21.15.32:0/1678670343 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51200193b0 con 0x7f512406a770 2023-09-30T22:31:15.848 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 msgr2=0x7f5124120170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f5124120170 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5120000c00 tx=0x7f5120004750 comp rx=0 tx=0).stop 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 shutdown_connections 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5124131b40 0x7f5124120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f512406a770 0x7f5124120170 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 --2- 172.21.15.32:0/1678670343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f512406b080 0x7f51241206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 >> 172.21.15.32:0/1678670343 conn(0x7f5124066f50 msgr2=0x7f5124071550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 shutdown_connections 2023-09-30T22:31:15.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:15.850+0000 7f512b7fd700 1 -- 172.21.15.32:0/1678670343 wait complete. 2023-09-30T22:31:15.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:16.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:16.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:16.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:16.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.507+0000 7f60cdc77700 1 Processor -- start 2023-09-30T22:31:16.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.507+0000 7f60cdc77700 1 -- start start 2023-09-30T22:31:16.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60cdc77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60cdc77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60c8133e30 0x7f60c812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60cdc77700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c806b990 con 0x7f60c8134260 2023-09-30T22:31:16.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60cdc77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c80a0590 con 0x7f60c8133e30 2023-09-30T22:31:16.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60cdc77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60c809fb80 con 0x7f60c8133420 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60c7fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f60c8134260 0x7f60c81304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44222/0 (socket says 172.21.15.32:44222) 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60c7fff700 1 -- 172.21.15.32:0/1027635116 learned_addr learned my addr 172.21.15.32:0/1027635116 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60c6ffd700 1 --2- 172.21.15.32:0/1027635116 >> v2:172.21.15.32:3300/0 conn(0x7f60c8133e30 0x7f60c812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.508+0000 7f60c77fe700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c67fc700 1 -- 172.21.15.32:0/1027635116 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249133140 0 0) 0x7f60c806b990 con 0x7f60c8134260 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c67fc700 1 -- 172.21.15.32:0/1027635116 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b00035b0 con 0x7f60c8134260 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c6ffd700 1 -- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 msgr2=0x7f60c81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c6ffd700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c6ffd700 1 -- 172.21.15.32:0/1027635116 >> v1:172.21.15.32:6789/0 conn(0x7f60c8134260 legacy=0x7f60c81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c6ffd700 1 -- 172.21.15.32:0/1027635116 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80a0030 con 0x7f60c8133e30 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c77fe700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c6ffd700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8133e30 0x7f60c812ff90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009840 tx=0x7f60bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e472e61e54fb9b23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c67fc700 1 -- 172.21.15.32:0/1027635116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc011830 con 0x7f60c8133e30 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c67fc700 1 -- 172.21.15.32:0/1027635116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc011990 con 0x7f60c8133e30 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.509+0000 7f60c67fc700 1 -- 172.21.15.32:0/1027635116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc011d20 con 0x7f60c8133e30 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 -- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8133e30 msgr2=0x7f60c812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8133e30 0x7f60c812ff90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009840 tx=0x7f60bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 -- 172.21.15.32:0/1027635116 shutdown_connections 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 --2- 172.21.15.32:0/1027635116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8133e30 0x7f60c812ff90 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 -- 172.21.15.32:0/1027635116 >> 172.21.15.32:0/1027635116 conn(0x7f60c80a4210 msgr2=0x7f60c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 -- 172.21.15.32:0/1027635116 shutdown_connections 2023-09-30T22:31:16.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.510+0000 7f60cdc77700 1 -- 172.21.15.32:0/1027635116 wait complete. 2023-09-30T22:31:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.511+0000 7f60cdc77700 1 Processor -- start 2023-09-30T22:31:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.511+0000 7f60cdc77700 1 -- start start 2023-09-30T22:31:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60cdc77700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60cdc77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c8133e30 0x7f60c809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60cdc77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8134260 0x7f60c809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60cdc77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c809df70 con 0x7f60c8134260 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60cdc77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c812da90 con 0x7f60c8133e30 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60cdc77700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60c812d750 con 0x7f60c8133420 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c6ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c8133e30 0x7f60c809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8134260 0x7f60c809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c77fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54270/0 (socket says 172.21.15.32:54270) 2023-09-30T22:31:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c77fe700 1 -- 172.21.15.32:0/313712435 learned_addr learned my addr 172.21.15.32:0/313712435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c7fff700 1 -- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 msgr2=0x7f60c809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c7fff700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c7fff700 1 -- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c8133e30 msgr2=0x7f60c809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.512+0000 7f60c7fff700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c8133e30 0x7f60c809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60c7fff700 1 -- 172.21.15.32:0/313712435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b80094a0 con 0x7f60c8134260 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60c6ffd700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c8133e30 0x7f60c809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60c77fe700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60c7fff700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8134260 0x7f60c809cf00 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f60c000ca50 tx=0x7f60c000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c000a9e0 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c000ab40 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc0094a0 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c0012400 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.513+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c8120060 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.514+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60c001b3b0 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.514+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f60c003f1d0 con 0x7f60c8134260 2023-09-30T22:31:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.514+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c80a4a00 con 0x7f60c8134260 2023-09-30T22:31:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.524+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60c00192c0 con 0x7f60c8134260 2023-09-30T22:31:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 Processor -- start 2023-09-30T22:31:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 -- start start 2023-09-30T22:31:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10480a0a30 0x7f10480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1048134170 0x7f104813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104809edd0 con 0x7f10480a1340 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104809e1f0 con 0x7f10480a0a30 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f104ddbf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f104809dc50 con 0x7f1048134170 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.549+0000 7f1046ffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10480a1340 0x7f1048133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44228/0 (socket says 172.21.15.32:44228) 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f1046ffd700 1 -- 172.21.15.32:0/3997254847 learned_addr learned my addr 172.21.15.32:0/3997254847 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10477fe700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10480a0a30 0x7f10480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f1047fff700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1048134170 0x7f104813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10467fc700 1 -- 172.21.15.32:0/3997254847 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218024829 0 0) 0x7f104809edd0 con 0x7f10480a1340 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10477fe700 1 -- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1048134170 msgr2=0x7f104813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10477fe700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1048134170 0x7f104813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10477fe700 1 -- 172.21.15.32:0/3997254847 >> v1:172.21.15.32:6789/0 conn(0x7f10480a1340 legacy=0x7f1048133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10477fe700 1 -- 172.21.15.32:0/3997254847 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480a1fe0 con 0x7f10480a0a30 2023-09-30T22:31:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10477fe700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10480a0a30 0x7f10480a0e00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1038009870 tx=0x7f1038009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f03a107d2a91b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10467fc700 1 -- 172.21.15.32:0/3997254847 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038011830 con 0x7f10480a0a30 2023-09-30T22:31:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10467fc700 1 -- 172.21.15.32:0/3997254847 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1038011990 con 0x7f10480a0a30 2023-09-30T22:31:16.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.550+0000 7f10467fc700 1 -- 172.21.15.32:0/3997254847 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038011d20 con 0x7f10480a0a30 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 -- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10480a0a30 msgr2=0x7f10480a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10480a0a30 0x7f10480a0e00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1038009870 tx=0x7f1038009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 -- 172.21.15.32:0/3997254847 shutdown_connections 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1048134170 0x7f104813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3997254847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f10480a0a30 0x7f10480a0e00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 -- 172.21.15.32:0/3997254847 >> 172.21.15.32:0/3997254847 conn(0x7f10480a6550 msgr2=0x7f10480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 -- 172.21.15.32:0/3997254847 shutdown_connections 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.551+0000 7f104ddbf700 1 -- 172.21.15.32:0/3997254847 wait complete. 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.552+0000 7f104ddbf700 1 Processor -- start 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 -- start start 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10480a0a30 0x7f1048122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 0x7f1048122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1048134170 0x7f1048123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048124350 con 0x7f10480a1340 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1048125cc0 con 0x7f1048134170 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f104ddbf700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1048126340 con 0x7f10480a0a30 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 0x7f1048122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 0x7f1048122cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59094/0 (socket says 172.21.15.32:59094) 2023-09-30T22:31:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f1046ffd700 1 -- 172.21.15.32:0/3157426843 learned_addr learned my addr 172.21.15.32:0/3157426843 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f10477fe700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10480a0a30 0x7f1048122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.553+0000 7f1047fff700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1048134170 0x7f1048123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f10477fe700 1 -- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1048134170 msgr2=0x7f1048123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f10477fe700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1048134170 0x7f1048123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f10477fe700 1 -- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 msgr2=0x7f1048122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f10477fe700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 0x7f1048122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f10477fe700 1 -- 172.21.15.32:0/3157426843 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380094a0 con 0x7f10480a0a30 2023-09-30T22:31:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f1046ffd700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 0x7f1048122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f10477fe700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10480a0a30 0x7f1048122790 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1038016000 tx=0x7f1038004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1038004510 con 0x7f10480a0a30 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103801f070 con 0x7f10480a0a30 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380038c0 con 0x7f10480a0a30 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10480a25e0 con 0x7f10480a0a30 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.554+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1048133690 con 0x7f10480a0a30 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.555+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1038003a20 con 0x7f10480a0a30 2023-09-30T22:31:16.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.555+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1038021070 con 0x7f10480a0a30 2023-09-30T22:31:16.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.555+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048074ea0 con 0x7f10480a0a30 2023-09-30T22:31:16.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.560+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1038015080 con 0x7f10480a0a30 2023-09-30T22:31:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.595+0000 7fe95f64d700 1 Processor -- start 2023-09-30T22:31:16.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.595+0000 7fe95f64d700 1 -- start start 2023-09-30T22:31:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95f64d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe958133420 0x7fe9581337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95f64d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 0x7fe95812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95f64d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958134260 0x7fe9581304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95f64d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95806b990 con 0x7fe958133420 2023-09-30T22:31:16.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95f64d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580a0590 con 0x7fe958134260 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95f64d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe95809fb80 con 0x7fe958133e30 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95cbe8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 0x7fe95812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95d3e9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe958133420 0x7fe9581337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95cbe8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 0x7fe95812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54298/0 (socket says 172.21.15.32:54298) 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95cbe8700 1 -- 172.21.15.32:0/282682448 learned_addr learned my addr 172.21.15.32:0/282682448 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.596+0000 7fe95dbea700 1 --2- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958134260 0x7fe9581304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95cbe8700 1 -- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958134260 msgr2=0x7fe9581304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95cbe8700 1 --2- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958134260 0x7fe9581304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95cbe8700 1 -- 172.21.15.32:0/282682448 >> v2:172.21.15.32:3300/0 conn(0x7fe958133420 msgr2=0x7fe9581337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95cbe8700 1 --2- 172.21.15.32:0/282682448 >> v2:172.21.15.32:3300/0 conn(0x7fe958133420 0x7fe9581337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95cbe8700 1 -- 172.21.15.32:0/282682448 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580a0030 con 0x7fe958133e30 2023-09-30T22:31:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95d3e9700 1 --2- 172.21.15.32:0/282682448 >> v2:172.21.15.32:3300/0 conn(0x7fe958133420 0x7fe9581337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe95cbe8700 1 --2- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 0x7fe95812ff90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe9480098f0 tx=0x7fe948009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=304fffb3e6f9bae3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe94ffff700 1 -- 172.21.15.32:0/282682448 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011830 con 0x7fe958133e30 2023-09-30T22:31:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe94ffff700 1 -- 172.21.15.32:0/282682448 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948011990 con 0x7fe958133e30 2023-09-30T22:31:16.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.597+0000 7fe94ffff700 1 -- 172.21.15.32:0/282682448 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011d20 con 0x7fe958133e30 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 -- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 msgr2=0x7fe95812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 --2- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 0x7fe95812ff90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe9480098f0 tx=0x7fe948009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 -- 172.21.15.32:0/282682448 shutdown_connections 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 --2- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958134260 0x7fe9581304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 --2- 172.21.15.32:0/282682448 >> v2:172.21.15.32:3300/0 conn(0x7fe958133420 0x7fe9581337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 --2- 172.21.15.32:0/282682448 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958133e30 0x7fe95812ff90 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 -- 172.21.15.32:0/282682448 >> 172.21.15.32:0/282682448 conn(0x7fe9580a4210 msgr2=0x7fe9580a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 -- 172.21.15.32:0/282682448 shutdown_connections 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.598+0000 7fe95f64d700 1 -- 172.21.15.32:0/282682448 wait complete. 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.599+0000 7fe95f64d700 1 Processor -- start 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.599+0000 7fe95f64d700 1 -- start start 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95f64d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe958133420 0x7fe95809c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95f64d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 0x7fe95809cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95f64d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 0x7fe95809d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95f64d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe958120720 con 0x7fe958133420 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95f64d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95811f0f0 con 0x7fe958133e30 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95f64d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe958120910 con 0x7fe958134260 2023-09-30T22:31:16.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 0x7fe95809d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95cbe8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 0x7fe95809cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95d3e9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe958133420 0x7fe95809c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 0x7fe95809d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54312/0 (socket says 172.21.15.32:54312) 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 -- 172.21.15.32:0/3005345011 learned_addr learned my addr 172.21.15.32:0/3005345011 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95cbe8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 0x7fe95809cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:41998/0 (socket says 172.21.15.32:41998) 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 -- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 msgr2=0x7fe95809cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 0x7fe95809cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 -- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe958133420 msgr2=0x7fe95809c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe958133420 0x7fe95809c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95dbea700 1 -- 172.21.15.32:0/3005345011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540094a0 con 0x7fe958134260 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.600+0000 7fe95cbe8700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 0x7fe95809cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.601+0000 7fe95dbea700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 0x7fe95809d1d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe95000efc0 tx=0x7fe95000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.601+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000cc90 con 0x7fe958134260 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.601+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95000f040 con 0x7fe958134260 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.601+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe948009550 con 0x7fe958134260 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.601+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500146c0 con 0x7fe958134260 2023-09-30T22:31:16.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.601+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95809e070 con 0x7fe958134260 2023-09-30T22:31:16.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.602+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe950014820 con 0x7fe958134260 2023-09-30T22:31:16.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.602+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe958120050 con 0x7fe958134260 2023-09-30T22:31:16.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.602+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe950040140 con 0x7fe958134260 2023-09-30T22:31:16.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.611+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe950007920 con 0x7fe958134260 2023-09-30T22:31:16.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.736+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f60c811fe10 con 0x7f60c8134260 2023-09-30T22:31:16.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.737+0000 7f60affff700 1 -- 172.21.15.32:0/313712435 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60c0017020 con 0x7f60c8134260 2023-09-30T22:31:16.737 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.740+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8134260 msgr2=0x7f60c809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.740+0000 7f60cdc77700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8134260 0x7f60c809cf00 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f60c000ca50 tx=0x7f60c000cd60 comp rx=0 tx=0).stop 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 shutdown_connections 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60c8134260 0x7f60c809cf00 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f60c8133420 0x7f60c809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 --2- 172.21.15.32:0/313712435 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60c8133e30 0x7f60c809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 >> 172.21.15.32:0/313712435 conn(0x7f60c80a4210 msgr2=0x7f60c812de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 shutdown_connections 2023-09-30T22:31:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.741+0000 7f60cdc77700 1 -- 172.21.15.32:0/313712435 wait complete. 2023-09-30T22:31:16.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.783+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f10480a28e0 con 0x7f10480a0a30 2023-09-30T22:31:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.784+0000 7f102ffff700 1 -- 172.21.15.32:0/3157426843 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1038035090 con 0x7f10480a0a30 2023-09-30T22:31:16.784 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10480a0a30 msgr2=0x7f1048122790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10480a0a30 0x7f1048122790 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1038016000 tx=0x7f1038004750 comp rx=0 tx=0).stop 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 shutdown_connections 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1048134170 0x7f1048123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f10480a0a30 0x7f1048122790 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 --2- 172.21.15.32:0/3157426843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10480a1340 0x7f1048122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 >> 172.21.15.32:0/3157426843 conn(0x7f10480a6550 msgr2=0x7f104809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 shutdown_connections 2023-09-30T22:31:16.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.787+0000 7f104ddbf700 1 -- 172.21.15.32:0/3157426843 wait complete. 2023-09-30T22:31:16.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:16.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.828+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe958074ea0 con 0x7fe958134260 2023-09-30T22:31:16.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.829+0000 7fe94dffb700 1 -- 172.21.15.32:0/3005345011 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe950009ca0 con 0x7fe958134260 2023-09-30T22:31:16.829 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 msgr2=0x7fe95809d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 0x7fe95809d1d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe95000efc0 tx=0x7fe95000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 shutdown_connections 2023-09-30T22:31:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe958134260 0x7fe95809d1d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe958133420 0x7fe95809c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 --2- 172.21.15.32:0/3005345011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe958133e30 0x7fe95809cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 >> 172.21.15.32:0/3005345011 conn(0x7fe9580a4210 msgr2=0x7fe9580a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 shutdown_connections 2023-09-30T22:31:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:16.831+0000 7fe95f64d700 1 -- 172.21.15.32:0/3005345011 wait complete. 2023-09-30T22:31:16.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:16.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 Processor -- start 2023-09-30T22:31:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 -- start start 2023-09-30T22:31:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c1316c0 0x7f962c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c06ae80 0x7f962c139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c09e130 con 0x7f962c09d9f0 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c1312b0 con 0x7f962c06ae80 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.025+0000 7f963233d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f962c09f6e0 con 0x7f962c1316c0 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f96308da700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c06ae80 0x7f962c139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f962c09d9f0 0x7f962c06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44236/0 (socket says 172.21.15.32:44236) 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c1316c0 0x7f962c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962b7fe700 1 -- 172.21.15.32:0/2138372349 learned_addr learned my addr 172.21.15.32:0/2138372349 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962affd700 1 -- 172.21.15.32:0/2138372349 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191135120 0 0) 0x7f961c001110 con 0x7f962c09d9f0 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f96308da700 1 -- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c1316c0 msgr2=0x7f962c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f96308da700 1 --2- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c1316c0 0x7f962c131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f96308da700 1 -- 172.21.15.32:0/2138372349 >> v1:172.21.15.32:6789/0 conn(0x7f962c09d9f0 legacy=0x7f962c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f96308da700 1 -- 172.21.15.32:0/2138372349 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c09e760 con 0x7f962c06ae80 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962bfff700 1 --2- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c1316c0 0x7f962c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f96308da700 1 --2- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c06ae80 0x7f962c139550 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9624009dc0 tx=0x7f962400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd287c02652cb47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962affd700 1 -- 172.21.15.32:0/2138372349 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400e070 con 0x7f962c06ae80 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962affd700 1 -- 172.21.15.32:0/2138372349 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962400bb20 con 0x7f962c06ae80 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.026+0000 7f962affd700 1 -- 172.21.15.32:0/2138372349 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624012430 con 0x7f962c06ae80 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 -- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c06ae80 msgr2=0x7f962c139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 --2- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c06ae80 0x7f962c139550 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9624009dc0 tx=0x7f962400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:17.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 -- 172.21.15.32:0/2138372349 shutdown_connections 2023-09-30T22:31:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 --2- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c06ae80 0x7f962c139550 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 --2- 172.21.15.32:0/2138372349 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c1316c0 0x7f962c131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 -- 172.21.15.32:0/2138372349 >> 172.21.15.32:0/2138372349 conn(0x7f962c066f50 msgr2=0x7f962c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 -- 172.21.15.32:0/2138372349 shutdown_connections 2023-09-30T22:31:17.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.027+0000 7f963233d700 1 -- 172.21.15.32:0/2138372349 wait complete. 2023-09-30T22:31:17.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.028+0000 7f963233d700 1 Processor -- start 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.028+0000 7f963233d700 1 -- start start 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f963233d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 0x7f962c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f963233d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c09d9f0 0x7f962c070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f963233d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c1316c0 0x7f962c071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f963233d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c11b0f0 con 0x7f962c06ae80 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f963233d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c11a6c0 con 0x7f962c1316c0 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f963233d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0a36d0 con 0x7f962c09d9f0 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 0x7f962c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 0x7f962c0709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59108/0 (socket says 172.21.15.32:59108) 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 -- 172.21.15.32:0/2540562969 learned_addr learned my addr 172.21.15.32:0/2540562969 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962b7fe700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c09d9f0 0x7f962c070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f96308da700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c1316c0 0x7f962c071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 -- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c09d9f0 msgr2=0x7f962c070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c09d9f0 0x7f962c070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 -- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c1316c0 msgr2=0x7f962c071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c1316c0 0x7f962c071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 -- 172.21.15.32:0/2540562969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140094a0 con 0x7f962c06ae80 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962b7fe700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c09d9f0 0x7f962c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f962bfff700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 0x7f962c0709d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f961400ebe0 tx=0x7f961400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.029+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400c850 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.030+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961400c9b0 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.030+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624009a70 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.030+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400cd40 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.030+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c12b900 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.030+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9614013470 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.031+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c127f70 con 0x7f962c06ae80 2023-09-30T22:31:17.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.031+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f961403fc40 con 0x7f962c06ae80 2023-09-30T22:31:17.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.035+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9614017080 con 0x7f962c06ae80 2023-09-30T22:31:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.242+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f962c11ae20 con 0x7f962c06ae80 2023-09-30T22:31:17.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.242+0000 7f9628ff9700 1 -- 172.21.15.32:0/2540562969 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f962c11ae20 con 0x7f962c06ae80 2023-09-30T22:31:17.242 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.244+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 msgr2=0x7f962c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.244+0000 7f963233d700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 0x7f962c0709d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f961400ebe0 tx=0x7f961400eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.244+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 shutdown_connections 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.244+0000 7f963233d700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f962c1316c0 0x7f962c071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.244+0000 7f963233d700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f962c06ae80 0x7f962c0709d0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.244+0000 7f963233d700 1 --2- 172.21.15.32:0/2540562969 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f962c09d9f0 0x7f962c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.245+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 >> 172.21.15.32:0/2540562969 conn(0x7f962c066f50 msgr2=0x7f962c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.245+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 shutdown_connections 2023-09-30T22:31:17.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.245+0000 7f963233d700 1 -- 172.21.15.32:0/2540562969 wait complete. 2023-09-30T22:31:17.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:17.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:17.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:17.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:17.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.911+0000 7fc43f8c8700 1 Processor -- start 2023-09-30T22:31:17.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.911+0000 7fc43f8c8700 1 -- start start 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43f8c8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc43809d6f0 0x7fc43809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43f8c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809e000 0x7fc4381339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43f8c8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43809b120 con 0x7fc43809e410 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43f8c8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43806b460 con 0x7fc43809d6f0 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43f8c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43809a7e0 con 0x7fc43809e000 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43ce63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809e000 0x7fc4381339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43ce63700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809e000 0x7fc4381339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42026/0 (socket says 172.21.15.32:42026) 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43d664700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc43809d6f0 0x7fc43809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.912+0000 7fc43ce63700 1 -- 172.21.15.32:0/3936694334 learned_addr learned my addr 172.21.15.32:0/3936694334 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc43d664700 1 -- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809e000 msgr2=0x7fc4381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc43d664700 1 --2- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809e000 0x7fc4381339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc43d664700 1 -- 172.21.15.32:0/3936694334 >> v1:172.21.15.32:6789/0 conn(0x7fc43809e410 legacy=0x7fc438133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc43d664700 1 -- 172.21.15.32:0/3936694334 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43809a500 con 0x7fc43809d6f0 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc42ffff700 1 -- 172.21.15.32:0/3936694334 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930976829 0 0) 0x7fc43809b120 con 0x7fc43809e410 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc43d664700 1 --2- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809d6f0 0x7fc43809dac0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fc434009840 tx=0x7fc434009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc17f9a4a3b066d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc42ffff700 1 -- 172.21.15.32:0/3936694334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434011830 con 0x7fc43809d6f0 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc42ffff700 1 -- 172.21.15.32:0/3936694334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434011990 con 0x7fc43809d6f0 2023-09-30T22:31:17.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.913+0000 7fc42ffff700 1 -- 172.21.15.32:0/3936694334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434011d20 con 0x7fc43809d6f0 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 -- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809d6f0 msgr2=0x7fc43809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809d6f0 0x7fc43809dac0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fc434009840 tx=0x7fc434009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 -- 172.21.15.32:0/3936694334 shutdown_connections 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809d6f0 0x7fc43809dac0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/3936694334 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809e000 0x7fc4381339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 -- 172.21.15.32:0/3936694334 >> 172.21.15.32:0/3936694334 conn(0x7fc438066f50 msgr2=0x7fc43809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 -- 172.21.15.32:0/3936694334 shutdown_connections 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.914+0000 7fc43f8c8700 1 -- 172.21.15.32:0/3936694334 wait complete. 2023-09-30T22:31:17.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.915+0000 7fc43f8c8700 1 Processor -- start 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.915+0000 7fc43f8c8700 1 -- start start 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43f8c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809d6f0 0x7fc4381b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43f8c8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 0x7fc4381b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43f8c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809e410 0x7fc4381c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43f8c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4381a4cc0 con 0x7fc43809e410 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43f8c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4381a7d90 con 0x7fc43809d6f0 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43f8c8700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4381a6420 con 0x7fc43809e000 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43ce63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 0x7fc4381b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809d6f0 0x7fc4381b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43ce63700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 0x7fc4381b8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54332/0 (socket says 172.21.15.32:54332) 2023-09-30T22:31:17.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43ce63700 1 -- 172.21.15.32:0/673822839 learned_addr learned my addr 172.21.15.32:0/673822839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43de65700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809e410 0x7fc4381c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 -- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 msgr2=0x7fc4381b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 0x7fc4381b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 -- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809e410 msgr2=0x7fc4381c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809e410 0x7fc4381c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 -- 172.21.15.32:0/673822839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4340094a0 con 0x7fc43809d6f0 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43ce63700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 0x7fc4381b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.916+0000 7fc43d664700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809d6f0 0x7fc4381b85f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc434009840 tx=0x7fc434004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.917+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43401a860 con 0x7fc43809d6f0 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.917+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43401a9e0 con 0x7fc43809d6f0 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.917+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4381a8410 con 0x7fc43809d6f0 2023-09-30T22:31:17.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.917+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43401ad90 con 0x7fc43809d6f0 2023-09-30T22:31:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.917+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4381b9070 con 0x7fc43809d6f0 2023-09-30T22:31:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.918+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc434022070 con 0x7fc43809d6f0 2023-09-30T22:31:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.918+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc434040500 con 0x7fc43809d6f0 2023-09-30T22:31:17.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.918+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc438074ea0 con 0x7fc43809d6f0 2023-09-30T22:31:17.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.923+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc43400b360 con 0x7fc43809d6f0 2023-09-30T22:31:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.951+0000 7fce2f783700 1 Processor -- start 2023-09-30T22:31:17.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.951+0000 7fce2f783700 1 -- start start 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2f783700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2f783700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce28138d80 0x7fce2813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2f783700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce280a2a70 con 0x7fce2809d630 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2f783700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce280a01d0 con 0x7fce28138d80 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2f783700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce2809e2b0 con 0x7fce280a0a30 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2d51f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2cd1e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce2809d630 0x7fce280a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44258/0 (socket says 172.21.15.32:44258) 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2cd1e700 1 -- 172.21.15.32:0/1228661671 learned_addr learned my addr 172.21.15.32:0/1228661671 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:17.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2dd20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce28138d80 0x7fce2813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2d51f700 1 -- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce28138d80 msgr2=0x7fce2813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2d51f700 1 --2- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce28138d80 0x7fce2813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2d51f700 1 -- 172.21.15.32:0/1228661671 >> v1:172.21.15.32:6789/0 conn(0x7fce2809d630 legacy=0x7fce280a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2d51f700 1 -- 172.21.15.32:0/1228661671 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2809dc50 con 0x7fce280a0a30 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.952+0000 7fce2dd20700 1 --2- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce28138d80 0x7fce2813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.953+0000 7fce2d51f700 1 --2- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce280a0e00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce24009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f584ed47feffd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.953+0000 7fce1ffff700 1 -- 172.21.15.32:0/1228661671 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24011830 con 0x7fce280a0a30 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.953+0000 7fce1ffff700 1 -- 172.21.15.32:0/1228661671 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24011990 con 0x7fce280a0a30 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.953+0000 7fce1ffff700 1 -- 172.21.15.32:0/1228661671 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24011d20 con 0x7fce280a0a30 2023-09-30T22:31:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 -- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 msgr2=0x7fce280a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 --2- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce280a0e00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce24009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 -- 172.21.15.32:0/1228661671 shutdown_connections 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 --2- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce28138d80 0x7fce2813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 --2- 172.21.15.32:0/1228661671 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce280a0e00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 -- 172.21.15.32:0/1228661671 >> 172.21.15.32:0/1228661671 conn(0x7fce280a3050 msgr2=0x7fce280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 -- 172.21.15.32:0/1228661671 shutdown_connections 2023-09-30T22:31:17.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.954+0000 7fce2f783700 1 -- 172.21.15.32:0/1228661671 wait complete. 2023-09-30T22:31:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.955+0000 7fce2f783700 1 Processor -- start 2023-09-30T22:31:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.955+0000 7fce2f783700 1 -- start start 2023-09-30T22:31:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2f783700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce2809d630 0x7fce2811d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2f783700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce2811dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2f783700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce28138d80 0x7fce2811e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2f783700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28134e50 con 0x7fce28138d80 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2f783700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce28120dd0 con 0x7fce2809d630 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2f783700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce281200b0 con 0x7fce280a0a30 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2cd1e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce2811dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2cd1e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce2811dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54342/0 (socket says 172.21.15.32:54342) 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2cd1e700 1 -- 172.21.15.32:0/2113542706 learned_addr learned my addr 172.21.15.32:0/2113542706 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2d51f700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce2809d630 0x7fce2811d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.956+0000 7fce2dd20700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce28138d80 0x7fce2811e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2cd1e700 1 -- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce2809d630 msgr2=0x7fce2811d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2cd1e700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce2809d630 0x7fce2811d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2cd1e700 1 -- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce28138d80 msgr2=0x7fce2811e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2cd1e700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce28138d80 0x7fce2811e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2cd1e700 1 -- 172.21.15.32:0/2113542706 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20009a70 con 0x7fce280a0a30 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2dd20700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce28138d80 0x7fce2811e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2cd1e700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce2811dc20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c530 tx=0x7fce1800c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18005d70 con 0x7fce280a0a30 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce180070d0 con 0x7fce280a0a30 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce240094a0 con 0x7fce280a0a30 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18007440 con 0x7fce280a0a30 2023-09-30T22:31:17.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.957+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2811f990 con 0x7fce280a0a30 2023-09-30T22:31:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.958+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce180075a0 con 0x7fce280a0a30 2023-09-30T22:31:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.958+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fce1803ea70 con 0x7fce280a0a30 2023-09-30T22:31:17.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.958+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce280747f0 con 0x7fce280a0a30 2023-09-30T22:31:17.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.963+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce18019b40 con 0x7fce280a0a30 2023-09-30T22:31:17.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.994+0000 7f4c7799f700 1 Processor -- start 2023-09-30T22:31:17.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.994+0000 7f4c7799f700 1 -- start start 2023-09-30T22:31:17.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7799f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 0x7f4c7006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7799f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c7006b080 0x7f4c70131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7799f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7009ece0 con 0x7f4c70131b40 2023-09-30T22:31:17.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7799f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7009e760 con 0x7f4c7006b080 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7799f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7009e040 con 0x7f4c7006a770 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 0x7f4c7006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c74f3a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c7006b080 0x7f4c70131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 0x7f4c7006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42076/0 (socket says 172.21.15.32:42076) 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 -- 172.21.15.32:0/3872794023 learned_addr learned my addr 172.21.15.32:0/3872794023 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c75f3c700 1 --1- 172.21.15.32:0/3872794023 >> v1:172.21.15.32:6789/0 conn(0x7f4c70131b40 0x7f4c70138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44270/0 (socket says 172.21.15.32:44270) 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 -- 172.21.15.32:0/3872794023 >> v2:172.21.15.32:3300/0 conn(0x7f4c7006b080 msgr2=0x7f4c70131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 --2- 172.21.15.32:0/3872794023 >> v2:172.21.15.32:3300/0 conn(0x7f4c7006b080 0x7f4c70131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 -- 172.21.15.32:0/3872794023 >> v1:172.21.15.32:6789/0 conn(0x7f4c70131b40 legacy=0x7f4c70138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c67fff700 1 -- 172.21.15.32:0/3872794023 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650796208 0 0) 0x7f4c7009ece0 con 0x7f4c70131b40 2023-09-30T22:31:17.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c7573b700 1 -- 172.21.15.32:0/3872794023 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7009f630 con 0x7f4c7006a770 2023-09-30T22:31:17.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.995+0000 7f4c74f3a700 1 --2- 172.21.15.32:0/3872794023 >> v2:172.21.15.32:3300/0 conn(0x7f4c7006b080 0x7f4c70131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.996+0000 7f4c7573b700 1 --2- 172.21.15.32:0/3872794023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 0x7f4c7006ab40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00dd10 tx=0x7f4c6c00f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1756f0f83c712aed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.996+0000 7f4c67fff700 1 -- 172.21.15.32:0/3872794023 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c012070 con 0x7f4c7006a770 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.996+0000 7f4c67fff700 1 -- 172.21.15.32:0/3872794023 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00d2e0 con 0x7f4c7006a770 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.996+0000 7f4c67fff700 1 -- 172.21.15.32:0/3872794023 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00fb10 con 0x7f4c7006a770 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.996+0000 7f4c7799f700 1 -- 172.21.15.32:0/3872794023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 msgr2=0x7f4c7006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.996+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3872794023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 0x7f4c7006ab40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00dd10 tx=0x7f4c6c00f330 comp rx=0 tx=0).stop 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.997+0000 7f4c7799f700 1 -- 172.21.15.32:0/3872794023 shutdown_connections 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.997+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3872794023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c7006a770 0x7f4c7006ab40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.997+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3872794023 >> v2:172.21.15.32:3300/0 conn(0x7f4c7006b080 0x7f4c70131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.997+0000 7f4c7799f700 1 -- 172.21.15.32:0/3872794023 >> 172.21.15.32:0/3872794023 conn(0x7f4c70066f50 msgr2=0x7f4c7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.997+0000 7f4c7799f700 1 -- 172.21.15.32:0/3872794023 shutdown_connections 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.997+0000 7f4c7799f700 1 -- 172.21.15.32:0/3872794023 wait complete. 2023-09-30T22:31:18.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 Processor -- start 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 -- start start 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c7006a770 0x7f4c7011e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c7006b080 0x7f4c7011e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c70131b40 0x7f4c7011ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70121f30 con 0x7f4c7006b080 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c701200c0 con 0x7f4c70131b40 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7799f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c701313f0 con 0x7f4c7006a770 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7573b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c7006a770 0x7f4c7011e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7573b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c7006a770 0x7f4c7011e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54358/0 (socket says 172.21.15.32:54358) 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c75f3c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c70131b40 0x7f4c7011ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c7573b700 1 -- 172.21.15.32:0/3122932508 learned_addr learned my addr 172.21.15.32:0/3122932508 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.998+0000 7f4c74f3a700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c7006b080 0x7f4c7011e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c74f3a700 1 -- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c7006a770 msgr2=0x7f4c7011e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c74f3a700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c7006a770 0x7f4c7011e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c74f3a700 1 -- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c70131b40 msgr2=0x7f4c7011ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c74f3a700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c70131b40 0x7f4c7011ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c74f3a700 1 -- 172.21.15.32:0/3122932508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600094a0 con 0x7f4c7006b080 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c74f3a700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c7006b080 0x7f4c7011e950 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000e980 tx=0x7f4c6000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000c690 con 0x7f4c7006b080 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6000c7f0 con 0x7f4c7006b080 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c00d9f0 con 0x7f4c7006b080 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000cb80 con 0x7f4c7006b080 2023-09-30T22:31:18.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:17.999+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c70071e60 con 0x7f4c7006b080 2023-09-30T22:31:18.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.001+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4c6000cce0 con 0x7f4c7006b080 2023-09-30T22:31:18.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.001+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c70074ea0 con 0x7f4c7006b080 2023-09-30T22:31:18.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.001+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f4c6003f660 con 0x7f4c7006b080 2023-09-30T22:31:18.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.006+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c6001bb50 con 0x7f4c7006b080 2023-09-30T22:31:18.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.137+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc4381b92c0 con 0x7fc43809d6f0 2023-09-30T22:31:18.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.138+0000 7fc42dffb700 1 -- 172.21.15.32:0/673822839 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc434029070 con 0x7fc43809d6f0 2023-09-30T22:31:18.138 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809d6f0 msgr2=0x7fc4381b85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809d6f0 0x7fc4381b85f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc434009840 tx=0x7fc434004750 comp rx=0 tx=0).stop 2023-09-30T22:31:18.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 shutdown_connections 2023-09-30T22:31:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc43809e410 0x7fc4381c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc43809d6f0 0x7fc4381b85f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 --2- 172.21.15.32:0/673822839 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc43809e000 0x7fc4381b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 >> 172.21.15.32:0/673822839 conn(0x7fc438066f50 msgr2=0x7fc4380a3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 shutdown_connections 2023-09-30T22:31:18.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.140+0000 7fc43f8c8700 1 -- 172.21.15.32:0/673822839 wait complete. 2023-09-30T22:31:18.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:18.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.175+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fce2809fc90 con 0x7fce280a0a30 2023-09-30T22:31:18.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.176+0000 7fce1dffb700 1 -- 172.21.15.32:0/2113542706 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce18017020 con 0x7fce280a0a30 2023-09-30T22:31:18.176 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:18.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 msgr2=0x7fce2811dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce2811dc20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c530 tx=0x7fce1800c840 comp rx=0 tx=0).stop 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 shutdown_connections 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce28138d80 0x7fce2811e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fce2809d630 0x7fce2811d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 --2- 172.21.15.32:0/2113542706 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fce280a0a30 0x7fce2811dc20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 >> 172.21.15.32:0/2113542706 conn(0x7fce280a3050 msgr2=0x7fce28139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 shutdown_connections 2023-09-30T22:31:18.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.178+0000 7fce2f783700 1 -- 172.21.15.32:0/2113542706 wait complete. 2023-09-30T22:31:18.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:18.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.215+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4c70130f30 con 0x7f4c7006b080 2023-09-30T22:31:18.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.215+0000 7f4c65ffb700 1 -- 172.21.15.32:0/3122932508 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c6001bb50 con 0x7f4c7006b080 2023-09-30T22:31:18.216 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c7006b080 msgr2=0x7f4c7011e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c7006b080 0x7f4c7011e950 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000e980 tx=0x7f4c6000ec90 comp rx=0 tx=0).stop 2023-09-30T22:31:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 shutdown_connections 2023-09-30T22:31:18.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c70131b40 0x7f4c7011ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f4c7006a770 0x7f4c7011e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 --2- 172.21.15.32:0/3122932508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c7006b080 0x7f4c7011e950 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 >> 172.21.15.32:0/3122932508 conn(0x7f4c70066f50 msgr2=0x7f4c700a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 shutdown_connections 2023-09-30T22:31:18.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.218+0000 7f4c7799f700 1 -- 172.21.15.32:0/3122932508 wait complete. 2023-09-30T22:31:18.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:18.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 Processor -- start 2023-09-30T22:31:18.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 -- start start 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc19c138d80 0x7fc19c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c0a2a70 con 0x7fc19c09d630 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c0a01d0 con 0x7fc19c138d80 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.411+0000 7fc1a13f7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c09e2b0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19affd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19a7fc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc19c09d630 0x7fc19c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44276/0 (socket says 172.21.15.32:44276) 2023-09-30T22:31:18.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19a7fc700 1 -- 172.21.15.32:0/663487402 learned_addr learned my addr 172.21.15.32:0/663487402 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19b7fe700 1 --2- 172.21.15.32:0/663487402 >> v2:172.21.15.32:3300/0 conn(0x7fc19c138d80 0x7fc19c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc199ffb700 1 -- 172.21.15.32:0/663487402 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053990998 0 0) 0x7fc19c0a2a70 con 0x7fc19c09d630 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19b7fe700 1 -- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 msgr2=0x7fc19c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19b7fe700 1 --2- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19b7fe700 1 -- 172.21.15.32:0/663487402 >> v1:172.21.15.32:6789/0 conn(0x7fc19c09d630 legacy=0x7fc19c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19b7fe700 1 -- 172.21.15.32:0/663487402 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c09dc50 con 0x7fc19c138d80 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19affd700 1 --2- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc19b7fe700 1 --2- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c13b160 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc194009d90 tx=0x7fc19400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a25b19cd8c90adb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc199ffb700 1 -- 172.21.15.32:0/663487402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400e070 con 0x7fc19c138d80 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc199ffb700 1 -- 172.21.15.32:0/663487402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19400bb20 con 0x7fc19c138d80 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.412+0000 7fc199ffb700 1 -- 172.21.15.32:0/663487402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194012430 con 0x7fc19c138d80 2023-09-30T22:31:18.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 -- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 msgr2=0x7fc19c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c13b160 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc194009d90 tx=0x7fc19400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 -- 172.21.15.32:0/663487402 shutdown_connections 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c13b160 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/663487402 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 -- 172.21.15.32:0/663487402 >> 172.21.15.32:0/663487402 conn(0x7fc19c0a3050 msgr2=0x7fc19c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 -- 172.21.15.32:0/663487402 shutdown_connections 2023-09-30T22:31:18.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.413+0000 7fc1a13f7700 1 -- 172.21.15.32:0/663487402 wait complete. 2023-09-30T22:31:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.414+0000 7fc1a13f7700 1 Processor -- start 2023-09-30T22:31:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.414+0000 7fc1a13f7700 1 -- start start 2023-09-30T22:31:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.414+0000 7fc1a13f7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc19c09d630 0x7fc19c133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.414+0000 7fc1a13f7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1a13f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1a13f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c1361b0 con 0x7fc19c138d80 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1a13f7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c136ed0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1a13f7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0a4b60 con 0x7fc19c09d630 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42096/0 (socket says 172.21.15.32:42096) 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c134110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59190/0 (socket says 172.21.15.32:59190) 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 -- 172.21.15.32:0/1346308190 learned_addr learned my addr 172.21.15.32:0/1346308190 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19affd700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc19c09d630 0x7fc19c133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 -- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc19c09d630 msgr2=0x7fc19c133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc19c09d630 0x7fc19c133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 -- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 msgr2=0x7fc19c134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 -- 172.21.15.32:0/1346308190 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1900094a0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19affd700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc19c09d630 0x7fc19c133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19b7fe700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc19a7fc700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c133bd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc18c001eb0 tx=0x7fc18c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004c80 con 0x7fc19c0a0a30 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c004de0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007310 con 0x7fc19c0a0a30 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.415+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc194009a70 con 0x7fc19c0a0a30 2023-09-30T22:31:18.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.416+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c136560 con 0x7fc19c0a0a30 2023-09-30T22:31:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.416+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc18c007470 con 0x7fc19c0a0a30 2023-09-30T22:31:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.417+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fc18c03f2b0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.417+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c0747f0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.421+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc18c0123b0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.625+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc19c1379a0 con 0x7fc19c0a0a30 2023-09-30T22:31:18.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.625+0000 7fc1837fe700 1 -- 172.21.15.32:0/1346308190 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc18c011360 con 0x7fc19c0a0a30 2023-09-30T22:31:18.626 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 msgr2=0x7fc19c133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c133bd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc18c001eb0 tx=0x7fc18c00c800 comp rx=0 tx=0).stop 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 shutdown_connections 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc19c138d80 0x7fc19c134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fc19c09d630 0x7fc19c133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 --2- 172.21.15.32:0/1346308190 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc19c0a0a30 0x7fc19c133bd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 >> 172.21.15.32:0/1346308190 conn(0x7fc19c0a3050 msgr2=0x7fc19c13af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 shutdown_connections 2023-09-30T22:31:18.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:18.628+0000 7fc1a13f7700 1 -- 172.21.15.32:0/1346308190 wait complete. 2023-09-30T22:31:18.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:19.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:19.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:19.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.306+0000 7fa7c826b700 1 Processor -- start 2023-09-30T22:31:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.306+0000 7fa7c826b700 1 -- start start 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c826b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7c01338a0 0x7fa7c0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c826b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 0x7fa7c009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c826b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c826b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c006fef0 con 0x7fa7c01338a0 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c826b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c006ef30 con 0x7fa7c009e760 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c826b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01334b0 con 0x7fa7c009de50 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c5806700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 0x7fa7c009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c5806700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 0x7fa7c009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54372/0 (socket says 172.21.15.32:54372) 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c5806700 1 -- 172.21.15.32:0/803107445 learned_addr learned my addr 172.21.15.32:0/803107445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6007700 1 --2- 172.21.15.32:0/803107445 >> v2:172.21.15.32:3300/0 conn(0x7fa7c01338a0 0x7fa7c0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6808700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6007700 1 -- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 msgr2=0x7fa7c009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6007700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 0x7fa7c009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6007700 1 -- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 msgr2=0x7fa7c013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6007700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c6007700 1 -- 172.21.15.32:0/803107445 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c006fc30 con 0x7fa7c01338a0 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.307+0000 7fa7c5806700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 0x7fa7c009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.308+0000 7fa7c6007700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c01338a0 0x7fa7c0133c70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009840 tx=0x7fa7bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fc380f09989f16 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.308+0000 7fa7c5005700 1 -- 172.21.15.32:0/803107445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011830 con 0x7fa7c01338a0 2023-09-30T22:31:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.308+0000 7fa7c5005700 1 -- 172.21.15.32:0/803107445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc011990 con 0x7fa7c01338a0 2023-09-30T22:31:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.308+0000 7fa7c5005700 1 -- 172.21.15.32:0/803107445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011d20 con 0x7fa7c01338a0 2023-09-30T22:31:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.308+0000 7fa7c826b700 1 -- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c01338a0 msgr2=0x7fa7c0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.308+0000 7fa7c826b700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c01338a0 0x7fa7c0133c70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009840 tx=0x7fa7bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 -- 172.21.15.32:0/803107445 shutdown_connections 2023-09-30T22:31:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c01338a0 0x7fa7c0133c70 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 --2- 172.21.15.32:0/803107445 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c009de50 0x7fa7c009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 -- 172.21.15.32:0/803107445 >> 172.21.15.32:0/803107445 conn(0x7fa7c00a2aa0 msgr2=0x7fa7c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 -- 172.21.15.32:0/803107445 shutdown_connections 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.309+0000 7fa7c826b700 1 -- 172.21.15.32:0/803107445 wait complete. 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 Processor -- start 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 -- start start 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c009de50 0x7fa7c011ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c011efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 0x7fa7c011f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01224e0 con 0x7fa7c009de50 2023-09-30T22:31:19.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c006f640 con 0x7fa7c009e760 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c826b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01205d0 con 0x7fa7c01338a0 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c6808700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 0x7fa7c011f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c5806700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c011efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c6808700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 0x7fa7c011f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54374/0 (socket says 172.21.15.32:54374) 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c6808700 1 -- 172.21.15.32:0/1334000124 learned_addr learned my addr 172.21.15.32:0/1334000124 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.310+0000 7fa7c6007700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c009de50 0x7fa7c011ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c5806700 1 -- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 msgr2=0x7fa7c011f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c5806700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 0x7fa7c011f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c5806700 1 -- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c009de50 msgr2=0x7fa7c011ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c5806700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c009de50 0x7fa7c011ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c5806700 1 -- 172.21.15.32:0/1334000124 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b0009550 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c6007700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c009de50 0x7fa7c011ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c6808700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 0x7fa7c011f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c5806700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c011efa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e980 tx=0x7fa7b000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004030 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0004190 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0094a0 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004520 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.311+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c012f910 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.312+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa7b00036a0 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.312+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c0074ea0 con 0x7fa7c009e760 2023-09-30T22:31:19.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.313+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa7b003f860 con 0x7fa7c009e760 2023-09-30T22:31:19.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.317+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa7b0011bd0 con 0x7fa7c009e760 2023-09-30T22:31:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.345+0000 7f6fafc1d700 1 Processor -- start 2023-09-30T22:31:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.345+0000 7f6fafc1d700 1 -- start start 2023-09-30T22:31:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.345+0000 7f6fafc1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.345+0000 7f6fafc1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 0x7f6fa813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fafc1d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa809edd0 con 0x7f6fa80a1340 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fafc1d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa809e1f0 con 0x7f6fa8134170 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fafc1d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa809dc50 con 0x7f6fa80a0a30 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fae1ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 0x7f6fa813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fad9b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fae1ba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 0x7f6fa813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42118/0 (socket says 172.21.15.32:42118) 2023-09-30T22:31:19.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fae1ba700 1 -- 172.21.15.32:0/419217320 learned_addr learned my addr 172.21.15.32:0/419217320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fac9b7700 1 -- 172.21.15.32:0/419217320 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888579595 0 0) 0x7f6fa809edd0 con 0x7f6fa80a1340 2023-09-30T22:31:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fac9b7700 1 -- 172.21.15.32:0/419217320 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f940035b0 con 0x7f6fa80a1340 2023-09-30T22:31:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fad9b9700 1 -- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 msgr2=0x7f6fa813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fad9b9700 1 --2- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 0x7f6fa813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fad9b9700 1 -- 172.21.15.32:0/419217320 >> v1:172.21.15.32:6789/0 conn(0x7f6fa80a1340 legacy=0x7f6fa8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fad9b9700 1 -- 172.21.15.32:0/419217320 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80a1fe0 con 0x7f6fa80a0a30 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fae1ba700 1 --2- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 0x7f6fa813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.346+0000 7f6fad9b9700 1 --2- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa80a0e00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009870 tx=0x7f6fa4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12918c3458d1b960 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.347+0000 7f6fac9b7700 1 -- 172.21.15.32:0/419217320 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4011830 con 0x7f6fa80a0a30 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.347+0000 7f6fac9b7700 1 -- 172.21.15.32:0/419217320 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4011990 con 0x7f6fa80a0a30 2023-09-30T22:31:19.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.347+0000 7f6fac9b7700 1 -- 172.21.15.32:0/419217320 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4011d20 con 0x7f6fa80a0a30 2023-09-30T22:31:19.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 -- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 msgr2=0x7f6fa80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa80a0e00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009870 tx=0x7f6fa4009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 -- 172.21.15.32:0/419217320 shutdown_connections 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa8134170 0x7f6fa813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/419217320 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa80a0e00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 -- 172.21.15.32:0/419217320 >> 172.21.15.32:0/419217320 conn(0x7f6fa80a6550 msgr2=0x7f6fa80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 -- 172.21.15.32:0/419217320 shutdown_connections 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.348+0000 7f6fafc1d700 1 -- 172.21.15.32:0/419217320 wait complete. 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.349+0000 7f6fafc1d700 1 Processor -- start 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.349+0000 7f6fafc1d700 1 -- start start 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.349+0000 7f6fafc1d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa81233f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.349+0000 7f6fafc1d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa80a1340 0x7f6fa8123930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.349+0000 7f6fafc1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa8134170 0x7f6fa81c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fafc1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8122fa0 con 0x7f6fa8134170 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fafc1d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8125ee0 con 0x7f6fa80a1340 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fafc1d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa81261a0 con 0x7f6fa80a0a30 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad9b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa81233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa80a1340 0x7f6fa8123930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad9b9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa81233f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:54382/0 (socket says 172.21.15.32:54382) 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad9b9700 1 -- 172.21.15.32:0/1410527788 learned_addr learned my addr 172.21.15.32:0/1410527788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:19.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fae1ba700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa8134170 0x7f6fa81c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 -- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 msgr2=0x7f6fa81233f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa81233f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 -- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa8134170 msgr2=0x7f6fa81c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa8134170 0x7f6fa81c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 -- 172.21.15.32:0/1410527788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0009b30 con 0x7f6fa80a1340 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad9b9700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa81233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fae1ba700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa8134170 0x7f6fa81c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.350+0000 7f6fad1b8700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa80a1340 0x7f6fa8123930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c450 tx=0x7f6f9800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.351+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980052d0 con 0x7f6fa80a1340 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.351+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98005430 con 0x7f6fa80a1340 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.351+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980038c0 con 0x7f6fa80a1340 2023-09-30T22:31:19.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.351+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa40094a0 con 0x7f6fa80a1340 2023-09-30T22:31:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.351+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa80a25e0 con 0x7f6fa80a1340 2023-09-30T22:31:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.352+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f98003a20 con 0x7f6fa80a1340 2023-09-30T22:31:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.352+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa80a41a0 con 0x7f6fa80a1340 2023-09-30T22:31:19.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.352+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6f9803eb00 con 0x7f6fa80a1340 2023-09-30T22:31:19.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.361+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f98011590 con 0x7f6fa80a1340 2023-09-30T22:31:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.387+0000 7f0d0fba4700 1 Processor -- start 2023-09-30T22:31:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.388+0000 7f0d0fba4700 1 -- start start 2023-09-30T22:31:19.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.388+0000 7f0d0fba4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d080a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.388+0000 7f0d0fba4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d0813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.388+0000 7f0d0fba4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d080a2a70 con 0x7f0d080a0a30 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.388+0000 7f0d0fba4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080a01d0 con 0x7f0d0809d630 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.388+0000 7f0d0fba4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0809e2b0 con 0x7f0d08138d80 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d13f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d080a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d940700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d080a0a30 0x7f0d080a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44294/0 (socket says 172.21.15.32:44294) 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d940700 1 -- 172.21.15.32:0/4217225538 learned_addr learned my addr 172.21.15.32:0/4217225538 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:19.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0e141700 1 --2- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d0813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d13f700 1 -- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 msgr2=0x7f0d0813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d13f700 1 --2- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d0813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d13f700 1 -- 172.21.15.32:0/4217225538 >> v1:172.21.15.32:6789/0 conn(0x7f0d080a0a30 legacy=0x7f0d080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d13f700 1 -- 172.21.15.32:0/4217225538 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0809dc50 con 0x7f0d0809d630 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0d13f700 1 --2- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d080a1510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009870 tx=0x7f0cfc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa671ef886aad950 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0c93e700 1 -- 172.21.15.32:0/4217225538 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011830 con 0x7f0d0809d630 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0c93e700 1 -- 172.21.15.32:0/4217225538 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc011990 con 0x7f0d0809d630 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.389+0000 7f0d0c93e700 1 -- 172.21.15.32:0/4217225538 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011d20 con 0x7f0d0809d630 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 -- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 msgr2=0x7f0d080a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d080a1510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009870 tx=0x7f0cfc009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:19.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 -- 172.21.15.32:0/4217225538 shutdown_connections 2023-09-30T22:31:19.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d0813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/4217225538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d080a1510 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 -- 172.21.15.32:0/4217225538 >> 172.21.15.32:0/4217225538 conn(0x7f0d080a3050 msgr2=0x7f0d080a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 -- 172.21.15.32:0/4217225538 shutdown_connections 2023-09-30T22:31:19.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.390+0000 7f0d0fba4700 1 -- 172.21.15.32:0/4217225538 wait complete. 2023-09-30T22:31:19.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.391+0000 7f0d0fba4700 1 Processor -- start 2023-09-30T22:31:19.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.391+0000 7f0d0fba4700 1 -- start start 2023-09-30T22:31:19.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0fba4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d081208b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0fba4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 0x7f0d08120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0fba4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d08121330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0fba4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d081236f0 con 0x7f0d080a0a30 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0fba4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08124340 con 0x7f0d0809d630 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0fba4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08123230 con 0x7f0d08138d80 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 0x7f0d08120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 0x7f0d08120df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59218/0 (socket says 172.21.15.32:59218) 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 -- 172.21.15.32:0/210354191 learned_addr learned my addr 172.21.15.32:0/210354191 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0e141700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d08121330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d940700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d081208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 -- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 msgr2=0x7f0d08121330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d08121330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 -- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 msgr2=0x7f0d081208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d081208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.392+0000 7f0d0d13f700 1 -- 172.21.15.32:0/210354191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0094a0 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.393+0000 7f0d0d13f700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 0x7f0d08120df0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009840 tx=0x7f0cfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.393+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc004450 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.393+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc01f070 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.393+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d081224d0 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.393+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0038e0 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.393+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d080a0410 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.394+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0cfc003a40 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.394+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d080a1430 con 0x7f0d080a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.394+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f0cfc021070 con 0x7f0d080a0a30 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.399+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0cfc010380 con 0x7f0d080a0a30 2023-09-30T22:31:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.532+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa7c012f4e0 con 0x7fa7c009e760 2023-09-30T22:31:19.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.533+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/1334000124 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7b0011e00 con 0x7fa7c009e760 2023-09-30T22:31:19.533 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 msgr2=0x7fa7c011efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c011efa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e980 tx=0x7fa7b000ec90 comp rx=0 tx=0).stop 2023-09-30T22:31:19.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 shutdown_connections 2023-09-30T22:31:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa7c01338a0 0x7fa7c011f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c009de50 0x7fa7c011ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 --2- 172.21.15.32:0/1334000124 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7c009e760 0x7fa7c011efa0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 >> 172.21.15.32:0/1334000124 conn(0x7fa7c00a2aa0 msgr2=0x7fa7c009ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 shutdown_connections 2023-09-30T22:31:19.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.535+0000 7fa7c826b700 1 -- 172.21.15.32:0/1334000124 wait complete. 2023-09-30T22:31:19.549 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:19.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.574+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6fa8074ea0 con 0x7f6fa80a1340 2023-09-30T22:31:19.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.575+0000 7f6f9e7fc700 1 -- 172.21.15.32:0/1410527788 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f98017020 con 0x7f6fa80a1340 2023-09-30T22:31:19.575 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa80a1340 msgr2=0x7f6fa8123930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa80a1340 0x7f6fa8123930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c450 tx=0x7f6f9800c760 comp rx=0 tx=0).stop 2023-09-30T22:31:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 shutdown_connections 2023-09-30T22:31:19.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fa8134170 0x7f6fa81c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6fa80a0a30 0x7f6fa81233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 --2- 172.21.15.32:0/1410527788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6fa80a1340 0x7f6fa8123930 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 >> 172.21.15.32:0/1410527788 conn(0x7f6fa80a6550 msgr2=0x7f6fa809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 shutdown_connections 2023-09-30T22:31:19.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.577+0000 7f6fafc1d700 1 -- 172.21.15.32:0/1410527788 wait complete. 2023-09-30T22:31:19.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:19.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.606+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0d080747f0 con 0x7f0d080a0a30 2023-09-30T22:31:19.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.607+0000 7f0d067fc700 1 -- 172.21.15.32:0/210354191 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0cfc015070 con 0x7f0d080a0a30 2023-09-30T22:31:19.607 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 msgr2=0x7f0d08120df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 0x7f0d08120df0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009840 tx=0x7f0cfc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 shutdown_connections 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f0d08138d80 0x7f0d08121330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0d0809d630 0x7f0d081208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 --2- 172.21.15.32:0/210354191 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d080a0a30 0x7f0d08120df0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 >> 172.21.15.32:0/210354191 conn(0x7f0d080a3050 msgr2=0x7f0d0809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 shutdown_connections 2023-09-30T22:31:19.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.609+0000 7f0d0fba4700 1 -- 172.21.15.32:0/210354191 wait complete. 2023-09-30T22:31:19.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:19.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.802+0000 7f882b6a7700 1 Processor -- start 2023-09-30T22:31:19.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.802+0000 7f882b6a7700 1 -- start start 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f882b6a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409e1d0 0x7f8824135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f882b6a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8824135e50 0x7f8824138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f882b6a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882409f130 con 0x7f882409d6f0 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f882b6a7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882409f520 con 0x7f8824135e50 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f882b6a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882409e930 con 0x7f882409e1d0 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8828c42700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409e1d0 0x7f8824135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8829443700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f882409d6f0 0x7f882409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44296/0 (socket says 172.21.15.32:44296) 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8829443700 1 -- 172.21.15.32:0/4014113106 learned_addr learned my addr 172.21.15.32:0/4014113106 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8829c44700 1 --2- 172.21.15.32:0/4014113106 >> v2:172.21.15.32:3300/0 conn(0x7f8824135e50 0x7f8824138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f881bfff700 1 -- 172.21.15.32:0/4014113106 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916319114 0 0) 0x7f882409f130 con 0x7f882409d6f0 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8828c42700 1 -- 172.21.15.32:0/4014113106 >> v2:172.21.15.32:3300/0 conn(0x7f8824135e50 msgr2=0x7f8824138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8828c42700 1 --2- 172.21.15.32:0/4014113106 >> v2:172.21.15.32:3300/0 conn(0x7f8824135e50 0x7f8824138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8828c42700 1 -- 172.21.15.32:0/4014113106 >> v1:172.21.15.32:6789/0 conn(0x7f882409d6f0 legacy=0x7f882409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.803+0000 7f8828c42700 1 -- 172.21.15.32:0/4014113106 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882409b2d0 con 0x7f882409e1d0 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f8828c42700 1 --2- 172.21.15.32:0/4014113106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409e1d0 0x7f8824135910 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8814009870 tx=0x7f8814009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b780bf9ded4c218 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f881bfff700 1 -- 172.21.15.32:0/4014113106 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814011830 con 0x7f882409e1d0 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f881bfff700 1 -- 172.21.15.32:0/4014113106 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8814011990 con 0x7f882409e1d0 2023-09-30T22:31:19.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f881bfff700 1 -- 172.21.15.32:0/4014113106 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814011d20 con 0x7f882409e1d0 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 -- 172.21.15.32:0/4014113106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409e1d0 msgr2=0x7f8824135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 --2- 172.21.15.32:0/4014113106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409e1d0 0x7f8824135910 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8814009870 tx=0x7f8814009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 -- 172.21.15.32:0/4014113106 shutdown_connections 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 --2- 172.21.15.32:0/4014113106 >> v2:172.21.15.32:3300/0 conn(0x7f8824135e50 0x7f8824138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 --2- 172.21.15.32:0/4014113106 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409e1d0 0x7f8824135910 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 -- 172.21.15.32:0/4014113106 >> 172.21.15.32:0/4014113106 conn(0x7f8824066f50 msgr2=0x7f882409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 -- 172.21.15.32:0/4014113106 shutdown_connections 2023-09-30T22:31:19.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.804+0000 7f882b6a7700 1 -- 172.21.15.32:0/4014113106 wait complete. 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.806+0000 7f882b6a7700 1 Processor -- start 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.806+0000 7f882b6a7700 1 -- start start 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.806+0000 7f882b6a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 0x7f882411c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.806+0000 7f882b6a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 0x7f882411ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f882b6a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8824135e50 0x7f882411cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f882b6a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882411d9c0 con 0x7f882409e1d0 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f882b6a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882411e120 con 0x7f882409d6f0 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f882b6a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f882411f370 con 0x7f8824135e50 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 0x7f882411ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8829443700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 0x7f882411c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 0x7f882411ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59242/0 (socket says 172.21.15.32:59242) 2023-09-30T22:31:19.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8829443700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 0x7f882411c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:42182/0 (socket says 172.21.15.32:42182) 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8829c44700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8824135e50 0x7f882411cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 -- 172.21.15.32:0/2642225977 learned_addr learned my addr 172.21.15.32:0/2642225977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 -- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8824135e50 msgr2=0x7f882411cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8824135e50 0x7f882411cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 -- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 msgr2=0x7f882411c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 0x7f882411c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 -- 172.21.15.32:0/2642225977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88140094a0 con 0x7f882409e1d0 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8829443700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 0x7f882411c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8829c44700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8824135e50 0x7f882411cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8828c42700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 0x7f882411ca50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8814000c00 tx=0x7f8814004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881401a8b0 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881401aa10 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881401ada0 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.807+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882411fd50 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.808+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88240a2020 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.809+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882409c700 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.809+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88140243f0 con 0x7f882409e1d0 2023-09-30T22:31:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.809+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8814040af0 con 0x7f882409e1d0 2023-09-30T22:31:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:19.814+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f881400b360 con 0x7f882409e1d0 2023-09-30T22:31:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.020+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8824074ea0 con 0x7f882409e1d0 2023-09-30T22:31:20.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.020+0000 7f8819ffb700 1 -- 172.21.15.32:0/2642225977 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8814040e60 con 0x7f882409e1d0 2023-09-30T22:31:20.020 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 msgr2=0x7f882411ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 0x7f882411ca50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8814000c00 tx=0x7f8814004750 comp rx=0 tx=0).stop 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 shutdown_connections 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8824135e50 0x7f882411cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f882409d6f0 0x7f882411c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 --2- 172.21.15.32:0/2642225977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f882409e1d0 0x7f882411ca50 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.022+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 >> 172.21.15.32:0/2642225977 conn(0x7f8824066f50 msgr2=0x7f88241368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.023+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 shutdown_connections 2023-09-30T22:31:20.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.023+0000 7f882b6a7700 1 -- 172.21.15.32:0/2642225977 wait complete. 2023-09-30T22:31:20.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:20.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:20.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:20.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.701+0000 7fb1240f2700 1 Processor -- start 2023-09-30T22:31:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.701+0000 7fb1240f2700 1 -- start start 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb1240f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb11c131810 0x7fb11c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb1240f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb1240f2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 0x7fb11c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb1240f2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c1313c0 con 0x7fb11c131810 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb1240f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c06e8c0 con 0x7fb11c099a40 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb1240f2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c071d20 con 0x7fb11c09a3b0 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb12168d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb12268f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 0x7fb11c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb12168d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55856/0 (socket says 172.21.15.32:55856) 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb12168d700 1 -- 172.21.15.32:0/1482188076 learned_addr learned my addr 172.21.15.32:0/1482188076 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb12268f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 0x7fb11c138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57218/0 (socket says 172.21.15.32:57218) 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.702+0000 7fb121e8e700 1 --2- 172.21.15.32:0/1482188076 >> v2:172.21.15.32:3300/0 conn(0x7fb11c131810 0x7fb11c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12268f700 1 -- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 msgr2=0x7fb11c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12268f700 1 --2- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12268f700 1 -- 172.21.15.32:0/1482188076 >> v2:172.21.15.32:3300/0 conn(0x7fb11c131810 msgr2=0x7fb11c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12268f700 1 --2- 172.21.15.32:0/1482188076 >> v2:172.21.15.32:3300/0 conn(0x7fb11c131810 0x7fb11c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12268f700 1 -- 172.21.15.32:0/1482188076 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0721a0 con 0x7fb11c09a3b0 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12168d700 1 --2- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb12268f700 1 --2- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 0x7fb11c138650 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb114009d90 tx=0x7fb11400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127521acc6179798 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb120e8c700 1 -- 172.21.15.32:0/1482188076 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11400e070 con 0x7fb11c09a3b0 2023-09-30T22:31:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb120e8c700 1 -- 172.21.15.32:0/1482188076 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11400bb20 con 0x7fb11c09a3b0 2023-09-30T22:31:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.703+0000 7fb120e8c700 1 -- 172.21.15.32:0/1482188076 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114012430 con 0x7fb11c09a3b0 2023-09-30T22:31:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 -- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 msgr2=0x7fb11c138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 --2- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 0x7fb11c138650 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb114009d90 tx=0x7fb11400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 -- 172.21.15.32:0/1482188076 shutdown_connections 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 --2- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c09a3b0 0x7fb11c138650 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 --2- 172.21.15.32:0/1482188076 >> v2:172.21.15.32:3300/0 conn(0x7fb11c131810 0x7fb11c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 --2- 172.21.15.32:0/1482188076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 -- 172.21.15.32:0/1482188076 >> 172.21.15.32:0/1482188076 conn(0x7fb11c0a2aa0 msgr2=0x7fb11c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 -- 172.21.15.32:0/1482188076 shutdown_connections 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.704+0000 7fb1240f2700 1 -- 172.21.15.32:0/1482188076 wait complete. 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.705+0000 7fb1240f2700 1 Processor -- start 2023-09-30T22:31:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.705+0000 7fb1240f2700 1 -- start start 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb1240f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c11b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb1240f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb11c09a3b0 0x7fb11c11bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb1240f2700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c131810 0x7fb11c11c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb1240f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c11fa10 con 0x7fb11c09a3b0 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb1240f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c11df40 con 0x7fb11c099a40 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb1240f2700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c11d1e0 con 0x7fb11c131810 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c11b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c11b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55870/0 (socket says 172.21.15.32:55870) 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 -- 172.21.15.32:0/351249251 learned_addr learned my addr 172.21.15.32:0/351249251 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb12268f700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c131810 0x7fb11c11c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 -- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c131810 msgr2=0x7fb11c11c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c131810 0x7fb11c11c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb12168d700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb11c09a3b0 0x7fb11c11bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 -- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb11c09a3b0 msgr2=0x7fb11c11bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb11c09a3b0 0x7fb11c11bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 -- 172.21.15.32:0/351249251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c009550 con 0x7fb11c099a40 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb12168d700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb11c09a3b0 0x7fb11c11bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.706+0000 7fb121e8e700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c11b5c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9e0 tx=0x7fb11800ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.707+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118009710 con 0x7fb11c099a40 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.707+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11800c850 con 0x7fb11c099a40 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.707+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11800cbc0 con 0x7fb11c099a40 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.707+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb114009a70 con 0x7fb11c099a40 2023-09-30T22:31:20.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.707+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c11e400 con 0x7fb11c099a40 2023-09-30T22:31:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.708+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c074ea0 con 0x7fb11c099a40 2023-09-30T22:31:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.708+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb11800cd20 con 0x7fb11c099a40 2023-09-30T22:31:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.708+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb11803f730 con 0x7fb11c099a40 2023-09-30T22:31:20.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.717+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1180232f0 con 0x7fb11c099a40 2023-09-30T22:31:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.751+0000 7f95b2ac4700 1 Processor -- start 2023-09-30T22:31:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.751+0000 7f95b2ac4700 1 -- start start 2023-09-30T22:31:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b2ac4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95ac09c650 0x7f95ac09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b2ac4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac09cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b2ac4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac09ba70 con 0x7f95ac131970 2023-09-30T22:31:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b2ac4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac09d4a0 con 0x7f95ac09c650 2023-09-30T22:31:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b2ac4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac09e8f0 con 0x7f95ac0a0090 2023-09-30T22:31:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac09cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac09cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55882/0 (socket says 172.21.15.32:55882) 2023-09-30T22:31:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 -- 172.21.15.32:0/2753790331 learned_addr learned my addr 172.21.15.32:0/2753790331 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b0860700 1 --2- 172.21.15.32:0/2753790331 >> v2:172.21.15.32:3300/0 conn(0x7f95ac09c650 0x7f95ac09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 -- 172.21.15.32:0/2753790331 >> v2:172.21.15.32:3300/0 conn(0x7f95ac09c650 msgr2=0x7f95ac09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 --2- 172.21.15.32:0/2753790331 >> v2:172.21.15.32:3300/0 conn(0x7f95ac09c650 0x7f95ac09ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 -- 172.21.15.32:0/2753790331 >> v1:172.21.15.32:6789/0 conn(0x7f95ac131970 legacy=0x7f95ac131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2753790331 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387507938 0 0) 0x7f95ac09ba70 con 0x7f95ac131970 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95abfff700 1 -- 172.21.15.32:0/2753790331 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0a0760 con 0x7f95ac0a0090 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.752+0000 7f95b0860700 1 --2- 172.21.15.32:0/2753790331 >> v2:172.21.15.32:3300/0 conn(0x7f95ac09c650 0x7f95ac09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.753+0000 7f95abfff700 1 --2- 172.21.15.32:0/2753790331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac09cf60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f95980098f0 tx=0x7f9598009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb892a6406803ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.753+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2753790331 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598011830 con 0x7f95ac0a0090 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.753+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2753790331 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598011990 con 0x7f95ac0a0090 2023-09-30T22:31:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.753+0000 7f95ab7fe700 1 -- 172.21.15.32:0/2753790331 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598011d20 con 0x7f95ac0a0090 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.753+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2753790331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 msgr2=0x7f95ac09cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.753+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2753790331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac09cf60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f95980098f0 tx=0x7f9598009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.754+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2753790331 shutdown_connections 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.754+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2753790331 >> v2:172.21.15.32:3300/0 conn(0x7f95ac09c650 0x7f95ac09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.754+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2753790331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac09cf60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.754+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2753790331 >> 172.21.15.32:0/2753790331 conn(0x7f95ac0a3f70 msgr2=0x7f95ac0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.754+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2753790331 shutdown_connections 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.754+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2753790331 wait complete. 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 Processor -- start 2023-09-30T22:31:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 -- start start 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 0x7f95ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac131970 0x7f95ac06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac11efa0 con 0x7f95ac09c650 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac11eb70 con 0x7f95ac0a0090 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b2ac4700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac11f8f0 con 0x7f95ac131970 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.755+0000 7f95b0860700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 0x7f95ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95abfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 0x7f95ac06d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55194/0 (socket says 172.21.15.32:55194) 2023-09-30T22:31:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 -- 172.21.15.32:0/2626666811 learned_addr learned my addr 172.21.15.32:0/2626666811 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 -- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac131970 msgr2=0x7f95ac06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac131970 0x7f95ac06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 -- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 msgr2=0x7f95ac06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac06b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 -- 172.21.15.32:0/2626666811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00094a0 con 0x7f95ac09c650 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95abfff700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac06b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b0860700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 0x7f95ac06d190 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f95a000ebe0 tx=0x7f95a000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0009800 con 0x7f95ac09c650 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a000c910 con 0x7f95ac09c650 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9598009550 con 0x7f95ac09c650 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a000cc80 con 0x7f95ac09c650 2023-09-30T22:31:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.756+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac06c300 con 0x7f95ac09c650 2023-09-30T22:31:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.758+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac0a3610 con 0x7f95ac09c650 2023-09-30T22:31:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.758+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f95a0013470 con 0x7f95ac09c650 2023-09-30T22:31:20.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.758+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f95a003f920 con 0x7f95ac09c650 2023-09-30T22:31:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.763+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95a0013720 con 0x7f95ac09c650 2023-09-30T22:31:20.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.790+0000 7fd5114dc700 1 Processor -- start 2023-09-30T22:31:20.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.790+0000 7fd5114dc700 1 -- start start 2023-09-30T22:31:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd5114dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd50c09e850 0x7fd50c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd5114dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd5114dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c1341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd5114dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c06bec0 con 0x7fd50c09e850 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd5114dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0a0030 con 0x7fd50c09f5b0 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd5114dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0a0830 con 0x7fd50c09f160 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd50a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd50b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c1341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd50a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c133c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57246/0 (socket says 172.21.15.32:57246) 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd50a7fc700 1 -- 172.21.15.32:0/3288117208 learned_addr learned my addr 172.21.15.32:0/3288117208 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.791+0000 7fd50affd700 1 --2- 172.21.15.32:0/3288117208 >> v2:172.21.15.32:3300/0 conn(0x7fd50c09e850 0x7fd50c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd50b7fe700 1 -- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 msgr2=0x7fd50c133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd50b7fe700 1 --2- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c133c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd50b7fe700 1 -- 172.21.15.32:0/3288117208 >> v2:172.21.15.32:3300/0 conn(0x7fd50c09e850 msgr2=0x7fd50c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd50b7fe700 1 --2- 172.21.15.32:0/3288117208 >> v2:172.21.15.32:3300/0 conn(0x7fd50c09e850 0x7fd50c09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd50b7fe700 1 -- 172.21.15.32:0/3288117208 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c06caa0 con 0x7fd50c09f5b0 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd50b7fe700 1 --2- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c1341a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd504009840 tx=0x7fd504009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed5d8187765b3ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd509ffb700 1 -- 172.21.15.32:0/3288117208 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504011830 con 0x7fd50c09f5b0 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd509ffb700 1 -- 172.21.15.32:0/3288117208 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd504011990 con 0x7fd50c09f5b0 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.792+0000 7fd509ffb700 1 -- 172.21.15.32:0/3288117208 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504011d20 con 0x7fd50c09f5b0 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 -- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 msgr2=0x7fd50c1341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 --2- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c1341a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd504009840 tx=0x7fd504009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 -- 172.21.15.32:0/3288117208 shutdown_connections 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 --2- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c1341a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 --2- 172.21.15.32:0/3288117208 >> v2:172.21.15.32:3300/0 conn(0x7fd50c09e850 0x7fd50c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 --2- 172.21.15.32:0/3288117208 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 -- 172.21.15.32:0/3288117208 >> 172.21.15.32:0/3288117208 conn(0x7fd50c0a3f70 msgr2=0x7fd50c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 -- 172.21.15.32:0/3288117208 shutdown_connections 2023-09-30T22:31:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.793+0000 7fd5114dc700 1 -- 172.21.15.32:0/3288117208 wait complete. 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 Processor -- start 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 -- start start 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd50c09e850 0x7fd50c120370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c1208b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c123e90 con 0x7fd50c09e850 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c122020 con 0x7fd50c09f5b0 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.795+0000 7fd5114dc700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0a3470 con 0x7fd50c09f160 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c1208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd50c09e850 0x7fd50c120370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c1208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57256/0 (socket says 172.21.15.32:57256) 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 -- 172.21.15.32:0/2140980163 learned_addr learned my addr 172.21.15.32:0/2140980163 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50b7fe700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 -- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 msgr2=0x7fd50c120df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c120df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 -- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd50c09e850 msgr2=0x7fd50c120370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd50c09e850 0x7fd50c120370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 -- 172.21.15.32:0/2140980163 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040094a0 con 0x7fd50c09f160 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50affd700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd50c09e850 0x7fd50c120370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50b7fe700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd50a7fc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c1208b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e910 tx=0x7fd50000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000096a0 con 0x7fd50c09f160 2023-09-30T22:31:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000ee60 con 0x7fd50c09f160 2023-09-30T22:31:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c06ccc0 con 0x7fd50c09f160 2023-09-30T22:31:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000ca90 con 0x7fd50c09f160 2023-09-30T22:31:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.796+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c1232a0 con 0x7fd50c09f160 2023-09-30T22:31:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.797+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd50000cbf0 con 0x7fd50c09f160 2023-09-30T22:31:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.797+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd50003fa10 con 0x7fd50c09f160 2023-09-30T22:31:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.798+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c0747f0 con 0x7fd50c09f160 2023-09-30T22:31:20.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.803+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd50001b3b0 con 0x7fd50c09f160 2023-09-30T22:31:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.925+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb11c06b9e0 con 0x7fb11c099a40 2023-09-30T22:31:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.926+0000 7fb1127fc700 1 -- 172.21.15.32:0/351249251 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb118017030 con 0x7fb11c099a40 2023-09-30T22:31:20.926 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 msgr2=0x7fb11c11b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c11b5c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9e0 tx=0x7fb11800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 shutdown_connections 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb11c131810 0x7fb11c11c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb11c099a40 0x7fb11c11b5c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 --2- 172.21.15.32:0/351249251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb11c09a3b0 0x7fb11c11bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 >> 172.21.15.32:0/351249251 conn(0x7fb11c0a2aa0 msgr2=0x7fb11c1b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 shutdown_connections 2023-09-30T22:31:20.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.928+0000 7fb1240f2700 1 -- 172.21.15.32:0/351249251 wait complete. 2023-09-30T22:31:20.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:20.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.987+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f95ac0747f0 con 0x7f95ac09c650 2023-09-30T22:31:20.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.988+0000 7f95a97fa700 1 -- 172.21.15.32:0/2626666811 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95a0017020 con 0x7f95ac09c650 2023-09-30T22:31:20.988 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:20.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 msgr2=0x7f95ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 0x7f95ac06d190 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f95a000ebe0 tx=0x7f95a000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 shutdown_connections 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f95ac131970 0x7f95ac06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac09c650 0x7f95ac06d190 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 --2- 172.21.15.32:0/2626666811 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95ac0a0090 0x7f95ac06b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 >> 172.21.15.32:0/2626666811 conn(0x7f95ac0a3f70 msgr2=0x7f95ac12eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 shutdown_connections 2023-09-30T22:31:20.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:20.991+0000 7f95b2ac4700 1 -- 172.21.15.32:0/2626666811 wait complete. 2023-09-30T22:31:21.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.025+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd50c072820 con 0x7fd50c09f160 2023-09-30T22:31:21.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.026+0000 7fd4f37fe700 1 -- 172.21.15.32:0/2140980163 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd500011eb0 con 0x7fd50c09f160 2023-09-30T22:31:21.026 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 msgr2=0x7fd50c1208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c1208b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e910 tx=0x7fd50000ec20 comp rx=0 tx=0).stop 2023-09-30T22:31:21.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 shutdown_connections 2023-09-30T22:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd50c09f5b0 0x7fd50c120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd50c09e850 0x7fd50c120370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 --2- 172.21.15.32:0/2140980163 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd50c09f160 0x7fd50c1208b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 >> 172.21.15.32:0/2140980163 conn(0x7fd50c0a3f70 msgr2=0x7fd50c1b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 shutdown_connections 2023-09-30T22:31:21.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.028+0000 7fd5114dc700 1 -- 172.21.15.32:0/2140980163 wait complete. 2023-09-30T22:31:21.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:21.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.174+0000 7f1f3870c700 1 Processor -- start 2023-09-30T22:31:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.174+0000 7f1f3870c700 1 -- start start 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f3870c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 0x7f1f3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f3870c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 0x7f1f30130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f3870c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3009f130 con 0x7f1f3009e410 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f3870c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3009f520 con 0x7f1f3009d6f0 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f3870c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3009e930 con 0x7f1f3009e000 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f364a8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 0x7f1f3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f35ca7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 0x7f1f30130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f364a8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 0x7f1f3009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55220/0 (socket says 172.21.15.32:55220) 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f36ca9700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f3009e410 0x7f1f30130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43842/0 (socket says 172.21.15.32:43842) 2023-09-30T22:31:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f364a8700 1 -- 172.21.15.32:0/3941325744 learned_addr learned my addr 172.21.15.32:0/3941325744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.175+0000 7f1f35ca7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 0x7f1f30130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55914/0 (socket says 172.21.15.32:55914) 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f35ca7700 1 -- 172.21.15.32:0/3941325744 >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 msgr2=0x7f1f3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f35ca7700 1 --2- 172.21.15.32:0/3941325744 >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 0x7f1f3009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f35ca7700 1 -- 172.21.15.32:0/3941325744 >> v1:172.21.15.32:6789/0 conn(0x7f1f3009e410 legacy=0x7f1f30130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f35ca7700 1 -- 172.21.15.32:0/3941325744 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3009b2d0 con 0x7f1f3009e000 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f364a8700 1 --2- 172.21.15.32:0/3941325744 >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 0x7f1f3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f35ca7700 1 --2- 172.21.15.32:0/3941325744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 0x7f1f30130660 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009840 tx=0x7f1f20009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e3488f15d783dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f354a6700 1 -- 172.21.15.32:0/3941325744 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20011830 con 0x7f1f3009e000 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f354a6700 1 -- 172.21.15.32:0/3941325744 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f20011990 con 0x7f1f3009e000 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.176+0000 7f1f354a6700 1 -- 172.21.15.32:0/3941325744 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20011d20 con 0x7f1f3009e000 2023-09-30T22:31:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 -- 172.21.15.32:0/3941325744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 msgr2=0x7f1f30130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 --2- 172.21.15.32:0/3941325744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 0x7f1f30130660 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009840 tx=0x7f1f20009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 -- 172.21.15.32:0/3941325744 shutdown_connections 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 --2- 172.21.15.32:0/3941325744 >> v2:172.21.15.32:3300/0 conn(0x7f1f3009d6f0 0x7f1f3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 --2- 172.21.15.32:0/3941325744 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e000 0x7f1f30130660 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 -- 172.21.15.32:0/3941325744 >> 172.21.15.32:0/3941325744 conn(0x7f1f30066f50 msgr2=0x7f1f3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 -- 172.21.15.32:0/3941325744 shutdown_connections 2023-09-30T22:31:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.177+0000 7f1f3870c700 1 -- 172.21.15.32:0/3941325744 wait complete. 2023-09-30T22:31:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 Processor -- start 2023-09-30T22:31:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 -- start start 2023-09-30T22:31:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f3009d6f0 0x7f1f30122360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f3009e000 0x7f1f301228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e410 0x7f1f30122de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f301254d0 con 0x7f1f3009d6f0 2023-09-30T22:31:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30124780 con 0x7f1f3009e410 2023-09-30T22:31:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f3870c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30123ee0 con 0x7f1f3009e000 2023-09-30T22:31:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f364a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f3009d6f0 0x7f1f30122360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f36ca9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e410 0x7f1f30122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.178+0000 7f1f364a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f3009d6f0 0x7f1f30122360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55234/0 (socket says 172.21.15.32:55234) 2023-09-30T22:31:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f364a8700 1 -- 172.21.15.32:0/1821209358 learned_addr learned my addr 172.21.15.32:0/1821209358 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f35ca7700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f3009e000 0x7f1f301228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f36ca9700 1 -- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f3009e000 msgr2=0x7f1f301228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f36ca9700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f3009e000 0x7f1f301228a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f36ca9700 1 -- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f3009d6f0 msgr2=0x7f1f30122360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f36ca9700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f3009d6f0 0x7f1f30122360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f36ca9700 1 -- 172.21.15.32:0/1821209358 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c00b7f0 con 0x7f1f3009e410 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f36ca9700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e410 0x7f1f30122de0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800cb70 tx=0x7f1f2800ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800aa20 con 0x7f1f3009e410 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800ab80 con 0x7f1f3009e410 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f200094a0 con 0x7f1f3009e410 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28007070 con 0x7f1f3009e410 2023-09-30T22:31:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.179+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3009c700 con 0x7f1f3009e410 2023-09-30T22:31:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.180+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f280071d0 con 0x7f1f3009e410 2023-09-30T22:31:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.180+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1f2803f270 con 0x7f1f3009e410 2023-09-30T22:31:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.180+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f30074ea0 con 0x7f1f3009e410 2023-09-30T22:31:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.185+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f28017070 con 0x7f1f3009e410 2023-09-30T22:31:21.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.390+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1f3006c360 con 0x7f1f3009e410 2023-09-30T22:31:21.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.390+0000 7f1f26ffd700 1 -- 172.21.15.32:0/1821209358 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f2801b3b0 con 0x7f1f3009e410 2023-09-30T22:31:21.390 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:21.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e410 msgr2=0x7f1f30122de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e410 0x7f1f30122de0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800cb70 tx=0x7f1f2800ce80 comp rx=0 tx=0).stop 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 shutdown_connections 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f3009e410 0x7f1f30122de0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f3009d6f0 0x7f1f30122360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 --2- 172.21.15.32:0/1821209358 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f1f3009e000 0x7f1f301228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 >> 172.21.15.32:0/1821209358 conn(0x7f1f30066f50 msgr2=0x7f1f30130ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.392+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 shutdown_connections 2023-09-30T22:31:21.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:21.393+0000 7f1f3870c700 1 -- 172.21.15.32:0/1821209358 wait complete. 2023-09-30T22:31:21.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:21.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:22.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:22.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.095+0000 7f8d52cf3700 1 Processor -- start 2023-09-30T22:31:22.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.095+0000 7f8d52cf3700 1 -- start start 2023-09-30T22:31:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d52cf3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09e930 0x7f8d4c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d52cf3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d4c09f240 0x7f8d4c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d52cf3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c070940 con 0x7f8d4c09f650 2023-09-30T22:31:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d52cf3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4c06fd60 con 0x7f8d4c09f240 2023-09-30T22:31:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d52cf3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c06fa80 con 0x7f8d4c09e930 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d4bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d4c09f240 0x7f8d4c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d4bfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d4c09f240 0x7f8d4c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55238/0 (socket says 172.21.15.32:55238) 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.096+0000 7f8d4bfff700 1 -- 172.21.15.32:0/1787833233 learned_addr learned my addr 172.21.15.32:0/1787833233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d50a8f700 1 --2- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09e930 0x7f8d4c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4bfff700 1 -- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09e930 msgr2=0x7f8d4c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4bfff700 1 --2- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09e930 0x7f8d4c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4bfff700 1 -- 172.21.15.32:0/1787833233 >> v1:172.21.15.32:6789/0 conn(0x7f8d4c09f650 legacy=0x7f8d4c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4bfff700 1 -- 172.21.15.32:0/1787833233 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c09fb60 con 0x7f8d4c09f240 2023-09-30T22:31:22.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4bfff700 1 --2- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f240 0x7f8d4c1317f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098f0 tx=0x7f8d38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996945c6372527c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4b7fe700 1 -- 172.21.15.32:0/1787833233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800b8a0 con 0x7f8d4c09f240 2023-09-30T22:31:22.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4b7fe700 1 -- 172.21.15.32:0/1787833233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800ba00 con 0x7f8d4c09f240 2023-09-30T22:31:22.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.097+0000 7f8d4b7fe700 1 -- 172.21.15.32:0/1787833233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800bd90 con 0x7f8d4c09f240 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 -- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f240 msgr2=0x7f8d4c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f240 0x7f8d4c1317f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098f0 tx=0x7f8d38009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 -- 172.21.15.32:0/1787833233 shutdown_connections 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09e930 0x7f8d4c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/1787833233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f240 0x7f8d4c1317f0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 -- 172.21.15.32:0/1787833233 >> 172.21.15.32:0/1787833233 conn(0x7f8d4c0a4210 msgr2=0x7f8d4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 -- 172.21.15.32:0/1787833233 shutdown_connections 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.098+0000 7f8d52cf3700 1 -- 172.21.15.32:0/1787833233 wait complete. 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.099+0000 7f8d52cf3700 1 Processor -- start 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.099+0000 7f8d52cf3700 1 -- start start 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d52cf3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 0x7f8d4c1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d52cf3700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09f240 0x7f8d4c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d52cf3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 0x7f8d4c1c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d52cf3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1a82c0 con 0x7f8d4c09f650 2023-09-30T22:31:22.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d52cf3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1a84a0 con 0x7f8d4c09e930 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d52cf3700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1b9310 con 0x7f8d4c09f240 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d51290700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 0x7f8d4c1c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d50a8f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 0x7f8d4c1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d50a8f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 0x7f8d4c1a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55926/0 (socket says 172.21.15.32:55926) 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d51290700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 0x7f8d4c1c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55248/0 (socket says 172.21.15.32:55248) 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d50a8f700 1 -- 172.21.15.32:0/3785403056 learned_addr learned my addr 172.21.15.32:0/3785403056 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d4bfff700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09f240 0x7f8d4c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d51290700 1 -- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09f240 msgr2=0x7f8d4c1a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d51290700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09f240 0x7f8d4c1a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.100+0000 7f8d51290700 1 -- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 msgr2=0x7f8d4c1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d51290700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 0x7f8d4c1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d51290700 1 -- 172.21.15.32:0/3785403056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d38009550 con 0x7f8d4c09f650 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d50a8f700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 0x7f8d4c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d4bfff700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09f240 0x7f8d4c1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d51290700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 0x7f8d4c1c8210 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400cbc0 tx=0x7f8d4400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44003c00 con 0x7f8d4c09f650 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d44003d60 con 0x7f8d4c09f650 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c0a0190 con 0x7f8d4c09f650 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400aa40 con 0x7f8d4c09f650 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.101+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c0a33d0 con 0x7f8d4c09f650 2023-09-30T22:31:22.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.102+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d4400aba0 con 0x7f8d4c09f650 2023-09-30T22:31:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.102+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8d4403f570 con 0x7f8d4c09f650 2023-09-30T22:31:22.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.103+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c074ea0 con 0x7f8d4c09f650 2023-09-30T22:31:22.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.108+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d44012420 con 0x7f8d4c09f650 2023-09-30T22:31:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.150+0000 7f3aaef8d700 1 Processor -- start 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.150+0000 7f3aaef8d700 1 -- start start 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aaef8d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa806c0e0 0x7f3aa809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aaef8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa813fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aaef8d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa8134980 con 0x7f3aa8134ec0 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aaef8d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa806c4b0 con 0x7f3aa806c0e0 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aaef8d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a09c0 con 0x7f3aa81380f0 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa806c0e0 0x7f3aa809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3aa806c0e0 0x7f3aa809ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55262/0 (socket says 172.21.15.32:55262) 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 -- 172.21.15.32:0/245569090 learned_addr learned my addr 172.21.15.32:0/245569090 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aad52a700 1 --2- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa813fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 -- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 msgr2=0x7f3aa813fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 --2- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa813fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 -- 172.21.15.32:0/245569090 >> v1:172.21.15.32:6789/0 conn(0x7f3aa8134ec0 legacy=0x7f3aa81379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aacd29700 1 -- 172.21.15.32:0/245569090 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80a0600 con 0x7f3aa806c0e0 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.151+0000 7f3aa77fe700 1 -- 172.21.15.32:0/245569090 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280646350 0 0) 0x7f3aa8134980 con 0x7f3aa8134ec0 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.152+0000 7f3aacd29700 1 --2- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa809ac90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0098f0 tx=0x7f3a9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a970ad162327055a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.152+0000 7f3aa77fe700 1 -- 172.21.15.32:0/245569090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c011830 con 0x7f3aa806c0e0 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.152+0000 7f3aa77fe700 1 -- 172.21.15.32:0/245569090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c011990 con 0x7f3aa806c0e0 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.152+0000 7f3aa77fe700 1 -- 172.21.15.32:0/245569090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c011d20 con 0x7f3aa806c0e0 2023-09-30T22:31:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.152+0000 7f3aaef8d700 1 -- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 msgr2=0x7f3aa809ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.152+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa809ac90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0098f0 tx=0x7f3a9c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.153+0000 7f3aaef8d700 1 -- 172.21.15.32:0/245569090 shutdown_connections 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.153+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa813fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.153+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/245569090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa809ac90 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.153+0000 7f3aaef8d700 1 -- 172.21.15.32:0/245569090 >> 172.21.15.32:0/245569090 conn(0x7f3aa80a4210 msgr2=0x7f3aa80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.153+0000 7f3aaef8d700 1 -- 172.21.15.32:0/245569090 shutdown_connections 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.153+0000 7f3aaef8d700 1 -- 172.21.15.32:0/245569090 wait complete. 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.154+0000 7f3aaef8d700 1 Processor -- start 2023-09-30T22:31:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.154+0000 7f3aaef8d700 1 -- start start 2023-09-30T22:31:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aaef8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa81ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aaef8d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 0x7f3aa81cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aaef8d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa81cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aaef8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa811c7f0 con 0x7f3aa806c0e0 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aaef8d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa811c0e0 con 0x7f3aa8134ec0 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aaef8d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a03f0 con 0x7f3aa81380f0 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aa7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 0x7f3aa81cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aa7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 0x7f3aa81cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55942/0 (socket says 172.21.15.32:55942) 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aa7fff700 1 -- 172.21.15.32:0/1701057458 learned_addr learned my addr 172.21.15.32:0/1701057458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aacd29700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa81ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.155+0000 7f3aad52a700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa81cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa7fff700 1 -- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 msgr2=0x7f3aa81cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa7fff700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa81cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa7fff700 1 -- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 msgr2=0x7f3aa81ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa7fff700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa81ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa7fff700 1 -- 172.21.15.32:0/1701057458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c009550 con 0x7f3aa8134ec0 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aad52a700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa81cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa7fff700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 0x7f3aa81cd020 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3a94001eb0 tx=0x7f3a9400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940054e0 con 0x7f3aa8134ec0 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a94003680 con 0x7f3aa8134ec0 2023-09-30T22:31:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa811d940 con 0x7f3aa8134ec0 2023-09-30T22:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940039f0 con 0x7f3aa8134ec0 2023-09-30T22:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.156+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa811dee0 con 0x7f3aa8134ec0 2023-09-30T22:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.157+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a94003b50 con 0x7f3aa8134ec0 2023-09-30T22:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.157+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3a9403ebc0 con 0x7f3aa8134ec0 2023-09-30T22:31:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.157+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa811db30 con 0x7f3aa8134ec0 2023-09-30T22:31:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.162+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a94017020 con 0x7f3aa8134ec0 2023-09-30T22:31:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.194+0000 7ff052230700 1 Processor -- start 2023-09-30T22:31:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.194+0000 7ff052230700 1 -- start start 2023-09-30T22:31:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.194+0000 7ff052230700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.194+0000 7ff052230700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c09d630 0x7ff04c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff052230700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 0x7ff04c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff052230700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0a2a70 con 0x7ff04c0a0a30 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff052230700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0a01d0 con 0x7ff04c133930 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff052230700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c09e2b0 con 0x7ff04c09d630 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 0x7ff04c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 0x7ff04c133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55956/0 (socket says 172.21.15.32:55956) 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 -- 172.21.15.32:0/3462475637 learned_addr learned my addr 172.21.15.32:0/3462475637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55294/0 (socket says 172.21.15.32:55294) 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff042ffd700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c09d630 0x7ff04c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 -- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c09d630 msgr2=0x7ff04c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:22.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c09d630 0x7ff04c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 -- 172.21.15.32:0/3462475637 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 msgr2=0x7ff04c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 --2- 172.21.15.32:0/3462475637 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 -- 172.21.15.32:0/3462475637 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c09dc50 con 0x7ff04c133930 2023-09-30T22:31:22.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff042ffd700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c09d630 0x7ff04c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.195+0000 7ff04bfff700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 0x7ff04c133db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff044009e50 tx=0x7ff04400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da776e74441357d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff04affd700 1 -- 172.21.15.32:0/3462475637 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400e070 con 0x7ff04c133930 2023-09-30T22:31:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff04affd700 1 -- 172.21.15.32:0/3462475637 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400b910 con 0x7ff04c133930 2023-09-30T22:31:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff04affd700 1 -- 172.21.15.32:0/3462475637 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400bcc0 con 0x7ff04c133930 2023-09-30T22:31:22.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 -- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 msgr2=0x7ff04c133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 0x7ff04c133db0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff044009e50 tx=0x7ff04400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 -- 172.21.15.32:0/3462475637 shutdown_connections 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c133930 0x7ff04c133db0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 --2- 172.21.15.32:0/3462475637 >> v2:172.21.15.32:3300/0 conn(0x7ff04c0a0a30 0x7ff04c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 --2- 172.21.15.32:0/3462475637 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c09d630 0x7ff04c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 -- 172.21.15.32:0/3462475637 >> 172.21.15.32:0/3462475637 conn(0x7ff04c0a3050 msgr2=0x7ff04c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 -- 172.21.15.32:0/3462475637 shutdown_connections 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.196+0000 7ff052230700 1 -- 172.21.15.32:0/3462475637 wait complete. 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.198+0000 7ff052230700 1 Processor -- start 2023-09-30T22:31:22.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.198+0000 7ff052230700 1 -- start start 2023-09-30T22:31:22.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff052230700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 0x7ff04c1c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff052230700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c0a0a30 0x7ff04c1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff052230700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c133930 0x7ff04c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff052230700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c120630 con 0x7ff04c133930 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff052230700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0a4030 con 0x7ff04c09d630 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff052230700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0a0410 con 0x7ff04c0a0a30 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff04b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 0x7ff04c1c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff04b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 0x7ff04c1c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:55960/0 (socket says 172.21.15.32:55960) 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff04b7fe700 1 -- 172.21.15.32:0/569977864 learned_addr learned my addr 172.21.15.32:0/569977864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff042ffd700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c0a0a30 0x7ff04c1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.199+0000 7ff04bfff700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c133930 0x7ff04c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04b7fe700 1 -- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c0a0a30 msgr2=0x7ff04c1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04b7fe700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c0a0a30 0x7ff04c1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04b7fe700 1 -- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c133930 msgr2=0x7ff04c1c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04b7fe700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c133930 0x7ff04c1c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04b7fe700 1 -- 172.21.15.32:0/569977864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009b30 con 0x7ff04c09d630 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04bfff700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c133930 0x7ff04c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff042ffd700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c0a0a30 0x7ff04c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff04b7fe700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 0x7ff04c1c7e80 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e9b0 tx=0x7ff03c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c009710 con 0x7ff04c09d630 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00c850 con 0x7ff04c09d630 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c1220e0 con 0x7ff04c09d630 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00cbc0 con 0x7ff04c09d630 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.200+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c133350 con 0x7ff04c09d630 2023-09-30T22:31:22.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.201+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff03c00cd20 con 0x7ff04c09d630 2023-09-30T22:31:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.201+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0747f0 con 0x7ff04c09d630 2023-09-30T22:31:22.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.201+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff03c03f830 con 0x7ff04c09d630 2023-09-30T22:31:22.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.206+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff03c01be10 con 0x7ff04c09d630 2023-09-30T22:31:22.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.322+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d4c1a8120 con 0x7f8d4c09f650 2023-09-30T22:31:22.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.323+0000 7f8d497fa700 1 -- 172.21.15.32:0/3785403056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d4c1a8120 con 0x7f8d4c09f650 2023-09-30T22:31:22.323 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 msgr2=0x7f8d4c1c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 0x7f8d4c1c8210 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400cbc0 tx=0x7f8d4400ced0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 shutdown_connections 2023-09-30T22:31:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d4c09f650 0x7f8d4c1c8210 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8d4c09e930 0x7f8d4c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 --2- 172.21.15.32:0/3785403056 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8d4c09f240 0x7f8d4c1a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 >> 172.21.15.32:0/3785403056 conn(0x7f8d4c0a4210 msgr2=0x7f8d4c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 shutdown_connections 2023-09-30T22:31:22.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.326+0000 7f8d52cf3700 1 -- 172.21.15.32:0/3785403056 wait complete. 2023-09-30T22:31:22.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.379+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3aa8074ea0 con 0x7f3aa8134ec0 2023-09-30T22:31:22.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.379+0000 7f3aa57fa700 1 -- 172.21.15.32:0/1701057458 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a94024070 con 0x7f3aa8134ec0 2023-09-30T22:31:22.379 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 msgr2=0x7f3aa81cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 0x7f3aa81cd020 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3a94001eb0 tx=0x7f3a9400c800 comp rx=0 tx=0).stop 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 shutdown_connections 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3aa81380f0 0x7f3aa81cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3aa806c0e0 0x7f3aa81ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 --2- 172.21.15.32:0/1701057458 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3aa8134ec0 0x7f3aa81cd020 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.381+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 >> 172.21.15.32:0/1701057458 conn(0x7f3aa80a4210 msgr2=0x7f3aa8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.382+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 shutdown_connections 2023-09-30T22:31:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.382+0000 7f3aaef8d700 1 -- 172.21.15.32:0/1701057458 wait complete. 2023-09-30T22:31:22.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:22.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:22.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.414+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff04c121eb0 con 0x7ff04c09d630 2023-09-30T22:31:22.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.414+0000 7ff048ff9700 1 -- 172.21.15.32:0/569977864 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff03c017090 con 0x7ff04c09d630 2023-09-30T22:31:22.415 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 msgr2=0x7ff04c1c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 0x7ff04c1c7e80 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e9b0 tx=0x7ff03c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 shutdown_connections 2023-09-30T22:31:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c133930 0x7ff04c1c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff04c09d630 0x7ff04c1c7e80 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 --2- 172.21.15.32:0/569977864 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff04c0a0a30 0x7ff04c1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 >> 172.21.15.32:0/569977864 conn(0x7ff04c0a3050 msgr2=0x7ff04c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 shutdown_connections 2023-09-30T22:31:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.417+0000 7ff052230700 1 -- 172.21.15.32:0/569977864 wait complete. 2023-09-30T22:31:22.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:22.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.551+0000 7fd0c89f6700 1 Processor -- start 2023-09-30T22:31:22.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.551+0000 7fd0c89f6700 1 -- start start 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c89f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133420 0x7fd0c41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c89f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4134260 0x7fd0c41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c89f6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c406b990 con 0x7fd0c4133e30 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c89f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40a0590 con 0x7fd0c4133420 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c89f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c409fb80 con 0x7fd0c4134260 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c1d9b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd0c4133e30 0x7fd0c412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43874/0 (socket says 172.21.15.32:43874) 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133420 0x7fd0c41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c2d9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4134260 0x7fd0c41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c1d9b700 1 -- 172.21.15.32:0/2398887277 learned_addr learned my addr 172.21.15.32:0/2398887277 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:22.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c159a700 1 -- 172.21.15.32:0/2398887277 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453641150 0 0) 0x7fd0c406b990 con 0x7fd0c4133e30 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c259c700 1 -- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4134260 msgr2=0x7fd0c41306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c259c700 1 --2- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4134260 0x7fd0c41306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c259c700 1 -- 172.21.15.32:0/2398887277 >> v1:172.21.15.32:6789/0 conn(0x7fd0c4133e30 legacy=0x7fd0c412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c259c700 1 -- 172.21.15.32:0/2398887277 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40a0030 con 0x7fd0c4133420 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c2d9d700 1 --2- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4134260 0x7fd0c41306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.552+0000 7fd0c259c700 1 --2- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133420 0x7fd0c41337f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009dc0 tx=0x7fd0bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257840fe5e056d27 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c159a700 1 -- 172.21.15.32:0/2398887277 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00e070 con 0x7fd0c4133420 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c159a700 1 -- 172.21.15.32:0/2398887277 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc00bb20 con 0x7fd0c4133420 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c159a700 1 -- 172.21.15.32:0/2398887277 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc012430 con 0x7fd0c4133420 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 -- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133420 msgr2=0x7fd0c41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133420 0x7fd0c41337f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009dc0 tx=0x7fd0bc00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:22.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 -- 172.21.15.32:0/2398887277 shutdown_connections 2023-09-30T22:31:22.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4134260 0x7fd0c41306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/2398887277 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133420 0x7fd0c41337f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 -- 172.21.15.32:0/2398887277 >> 172.21.15.32:0/2398887277 conn(0x7fd0c40a4210 msgr2=0x7fd0c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 -- 172.21.15.32:0/2398887277 shutdown_connections 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.553+0000 7fd0c89f6700 1 -- 172.21.15.32:0/2398887277 wait complete. 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.554+0000 7fd0c89f6700 1 Processor -- start 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.554+0000 7fd0c89f6700 1 -- start start 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c89f6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 0x7fd0c40709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c89f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133e30 0x7fd0c4070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c89f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 0x7fd0c4071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c89f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4072480 con 0x7fd0c4134260 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c89f6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4072290 con 0x7fd0c4133e30 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c89f6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4120190 con 0x7fd0c4133420 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 0x7fd0c4071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c1d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133e30 0x7fd0c4070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c259c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 0x7fd0c40709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c2d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 0x7fd0c4071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55306/0 (socket says 172.21.15.32:55306) 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c2d9d700 1 -- 172.21.15.32:0/1198110484 learned_addr learned my addr 172.21.15.32:0/1198110484 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c259c700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 0x7fd0c40709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57352/0 (socket says 172.21.15.32:57352) 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c1d9b700 1 -- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 msgr2=0x7fd0c40709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c1d9b700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 0x7fd0c40709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c1d9b700 1 -- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 msgr2=0x7fd0c4071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c1d9b700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 0x7fd0c4071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c1d9b700 1 -- 172.21.15.32:0/1198110484 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40094a0 con 0x7fd0c4133e30 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c2d9d700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 0x7fd0c4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.555+0000 7fd0c259c700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 0x7fd0c40709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:22.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.556+0000 7fd0c1d9b700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133e30 0x7fd0c4070ee0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409f2c0 tx=0x7fd0ac00c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.556+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004c80 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.556+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac004de0 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.556+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc009a70 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.556+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac007310 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.556+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c411f530 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.557+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd0ac007470 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.557+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c41209e0 con 0x7fd0c4133e30 2023-09-30T22:31:22.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.557+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd0ac03f300 con 0x7fd0c4133e30 2023-09-30T22:31:22.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.562+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd0ac01bda0 con 0x7fd0c4133e30 2023-09-30T22:31:22.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.765+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd0c4074ea0 con 0x7fd0c4133e30 2023-09-30T22:31:22.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.765+0000 7fd0baffd700 1 -- 172.21.15.32:0/1198110484 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0ac01b340 con 0x7fd0c4133e30 2023-09-30T22:31:22.795 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133e30 msgr2=0x7fd0c4070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133e30 0x7fd0c4070ee0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409f2c0 tx=0x7fd0ac00c750 comp rx=0 tx=0).stop 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 shutdown_connections 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0c4134260 0x7fd0c4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd0c4133420 0x7fd0c40709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 --2- 172.21.15.32:0/1198110484 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0c4133e30 0x7fd0c4070ee0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 >> 172.21.15.32:0/1198110484 conn(0x7fd0c40a4210 msgr2=0x7fd0c40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 shutdown_connections 2023-09-30T22:31:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:22.768+0000 7fd0c89f6700 1 -- 172.21.15.32:0/1198110484 wait complete. 2023-09-30T22:31:22.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:23.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:23.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:23.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.497+0000 7fa0a965f700 1 Processor -- start 2023-09-30T22:31:23.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.497+0000 7fa0a965f700 1 -- start start 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a965f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a965f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a965f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a965f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a409edd0 con 0x7fa0a40a0a30 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a965f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a409e1f0 con 0x7fa0a4133fa0 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a965f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a409dc50 con 0x7fa0a40a1340 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a27fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55312/0 (socket says 172.21.15.32:55312) 2023-09-30T22:31:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/3199146244 learned_addr learned my addr 172.21.15.32:0/3199146244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a37fe700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 msgr2=0x7fa0a4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 msgr2=0x7fa0a4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/3199146244 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40a1fe0 con 0x7fa0a40a0a30 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a27fc700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.498+0000 7fa0a37fe700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.499+0000 7fa0a2ffd700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098f0 tx=0x7fa094009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cb3c2db48ec1a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.499+0000 7fa0a1ffb700 1 -- 172.21.15.32:0/3199146244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094011830 con 0x7fa0a40a0a30 2023-09-30T22:31:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.499+0000 7fa0a1ffb700 1 -- 172.21.15.32:0/3199146244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094011990 con 0x7fa0a40a0a30 2023-09-30T22:31:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.499+0000 7fa0a1ffb700 1 -- 172.21.15.32:0/3199146244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094011d20 con 0x7fa0a40a0a30 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 -- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 msgr2=0x7fa0a40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098f0 tx=0x7fa094009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 -- 172.21.15.32:0/3199146244 shutdown_connections 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a40a0e00 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 --2- 172.21.15.32:0/3199146244 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 -- 172.21.15.32:0/3199146244 >> 172.21.15.32:0/3199146244 conn(0x7fa0a40a6550 msgr2=0x7fa0a40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 -- 172.21.15.32:0/3199146244 shutdown_connections 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.500+0000 7fa0a965f700 1 -- 172.21.15.32:0/3199146244 wait complete. 2023-09-30T22:31:23.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.501+0000 7fa0a965f700 1 Processor -- start 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.501+0000 7fa0a965f700 1 -- start start 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a965f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a41212d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a965f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a965f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4121d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a965f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41326a0 con 0x7fa0a40a0a30 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a965f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4123d70 con 0x7fa0a4133fa0 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a965f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4124850 con 0x7fa0a40a1340 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a41212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a41212d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55328/0 (socket says 172.21.15.32:55328) 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/241743424 learned_addr learned my addr 172.21.15.32:0/241743424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a27fc700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a37fe700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4121d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 msgr2=0x7fa0a4121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 msgr2=0x7fa0a4121d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4121d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 -- 172.21.15.32:0/241743424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0980094a0 con 0x7fa0a40a0a30 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a27fc700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a37fe700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.502+0000 7fa0a2ffd700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a41212d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fa094011db0 tx=0x7fa094004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.503+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094004240 con 0x7fa0a40a0a30 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.503+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09401f070 con 0x7fa0a40a0a30 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.503+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c009a70 con 0x7fa0a40a0a30 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.503+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0940045a0 con 0x7fa0a40a0a30 2023-09-30T22:31:23.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.503+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa094009550 con 0x7fa0a40a0a30 2023-09-30T22:31:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.504+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa094021030 con 0x7fa0a40a0a30 2023-09-30T22:31:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.504+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a4124990 con 0x7fa0a40a0a30 2023-09-30T22:31:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.504+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa094040730 con 0x7fa0a40a0a30 2023-09-30T22:31:23.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.509+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0940036a0 con 0x7fa0a40a0a30 2023-09-30T22:31:23.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.545+0000 7f2e88f2a700 1 Processor -- start 2023-09-30T22:31:23.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 -- start start 2023-09-30T22:31:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a0a30 0x7f2e840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e840a1340 0x7f2e84133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e84133fa0 0x7f2e84134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8409edd0 con 0x7f2e840a1340 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8409e1f0 con 0x7f2e840a0a30 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e88f2a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8409dc50 con 0x7f2e84133fa0 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.546+0000 7f2e81d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e840a1340 0x7f2e84133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2e840a1340 0x7f2e84133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55342/0 (socket says 172.21.15.32:55342) 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3862242646 learned_addr learned my addr 172.21.15.32:0/3862242646 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e8259c700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a0a30 0x7f2e840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e82d9d700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e84133fa0 0x7f2e84134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e84133fa0 msgr2=0x7f2e84134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e84133fa0 0x7f2e84134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a0a30 msgr2=0x7f2e840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a0a30 0x7f2e840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3862242646 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840a1fe0 con 0x7f2e840a1340 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e8259c700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a0a30 0x7f2e840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e81d9b700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840a1340 0x7f2e84133a60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098f0 tx=0x7f2e78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887276e503efe4b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e8159a700 1 -- 172.21.15.32:0/3862242646 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78011830 con 0x7f2e840a1340 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e8159a700 1 -- 172.21.15.32:0/3862242646 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78011990 con 0x7f2e840a1340 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.547+0000 7f2e8159a700 1 -- 172.21.15.32:0/3862242646 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78011d20 con 0x7f2e840a1340 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840a1340 msgr2=0x7f2e84133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840a1340 0x7f2e84133a60 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098f0 tx=0x7f2e78009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3862242646 shutdown_connections 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a0a30 0x7f2e840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e84133fa0 0x7f2e84134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3862242646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e840a1340 0x7f2e84133a60 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3862242646 >> 172.21.15.32:0/3862242646 conn(0x7f2e840a6550 msgr2=0x7f2e840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3862242646 shutdown_connections 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.548+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3862242646 wait complete. 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 Processor -- start 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 -- start start 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e840a0a30 0x7f2e84120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 0x7f2e84120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84133fa0 0x7f2e84120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84121d90 con 0x7f2e84133fa0 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84123920 con 0x7f2e840a1340 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e88f2a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840a3510 con 0x7f2e840a0a30 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e81d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 0x7f2e84120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e81d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 0x7f2e84120670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56016/0 (socket says 172.21.15.32:56016) 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3515410133 learned_addr learned my addr 172.21.15.32:0/3515410133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.550+0000 7f2e82d9d700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84133fa0 0x7f2e84120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e8259c700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e840a0a30 0x7f2e84120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e840a0a30 msgr2=0x7f2e84120130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e81d9b700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e840a0a30 0x7f2e84120130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84133fa0 msgr2=0x7f2e84120bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e81d9b700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84133fa0 0x7f2e84120bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e81d9b700 1 -- 172.21.15.32:0/3515410133 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e74009a70 con 0x7f2e840a1340 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e8259c700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e840a0a30 0x7f2e84120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e82d9d700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84133fa0 0x7f2e84120bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e81d9b700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 0x7f2e84120670 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009c40 tx=0x7f2e78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004300 con 0x7f2e840a1340 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7801f070 con 0x7f2e840a1340 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e78009550 con 0x7f2e840a1340 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780036a0 con 0x7f2e840a1340 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.551+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e84122f40 con 0x7f2e840a1340 2023-09-30T22:31:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.552+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e78021030 con 0x7f2e840a1340 2023-09-30T22:31:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.552+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e84074ea0 con 0x7f2e840a1340 2023-09-30T22:31:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.552+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f2e78040820 con 0x7f2e840a1340 2023-09-30T22:31:23.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.557+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e780199a0 con 0x7f2e840a1340 2023-09-30T22:31:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 Processor -- start 2023-09-30T22:31:23.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 -- start start 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8280a0a30 0x7fe8280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8280a1340 0x7fe828133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82809edd0 con 0x7fe828133fa0 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82809e1f0 con 0x7fe8280a0a30 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe82dd4a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82809dc50 con 0x7fe8280a1340 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe826ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8280a1340 0x7fe828133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe827fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe828133fa0 0x7fe82813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43882/0 (socket says 172.21.15.32:43882) 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.588+0000 7fe827fff700 1 -- 172.21.15.32:0/1157406243 learned_addr learned my addr 172.21.15.32:0/1157406243 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:23.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe8277fe700 1 --2- 172.21.15.32:0/1157406243 >> v2:172.21.15.32:3300/0 conn(0x7fe8280a0a30 0x7fe8280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe826ffd700 1 -- 172.21.15.32:0/1157406243 >> v2:172.21.15.32:3300/0 conn(0x7fe8280a0a30 msgr2=0x7fe8280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe826ffd700 1 --2- 172.21.15.32:0/1157406243 >> v2:172.21.15.32:3300/0 conn(0x7fe8280a0a30 0x7fe8280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe826ffd700 1 -- 172.21.15.32:0/1157406243 >> v1:172.21.15.32:6789/0 conn(0x7fe828133fa0 legacy=0x7fe82813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe826ffd700 1 -- 172.21.15.32:0/1157406243 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8280a1fe0 con 0x7fe8280a1340 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe8267fc700 1 -- 172.21.15.32:0/1157406243 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212601253 0 0) 0x7fe82809edd0 con 0x7fe828133fa0 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe826ffd700 1 --2- 172.21.15.32:0/1157406243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8280a1340 0x7fe828133a60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009840 tx=0x7fe81c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9d1bb4149a1f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe8267fc700 1 -- 172.21.15.32:0/1157406243 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c011830 con 0x7fe8280a1340 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe8267fc700 1 -- 172.21.15.32:0/1157406243 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c011990 con 0x7fe8280a1340 2023-09-30T22:31:23.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.589+0000 7fe8267fc700 1 -- 172.21.15.32:0/1157406243 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c011d20 con 0x7fe8280a1340 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 -- 172.21.15.32:0/1157406243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8280a1340 msgr2=0x7fe828133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/1157406243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8280a1340 0x7fe828133a60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009840 tx=0x7fe81c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 -- 172.21.15.32:0/1157406243 shutdown_connections 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/1157406243 >> v2:172.21.15.32:3300/0 conn(0x7fe8280a0a30 0x7fe8280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/1157406243 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8280a1340 0x7fe828133a60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 -- 172.21.15.32:0/1157406243 >> 172.21.15.32:0/1157406243 conn(0x7fe8280a6550 msgr2=0x7fe8280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 -- 172.21.15.32:0/1157406243 shutdown_connections 2023-09-30T22:31:23.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.590+0000 7fe82dd4a700 1 -- 172.21.15.32:0/1157406243 wait complete. 2023-09-30T22:31:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.591+0000 7fe82dd4a700 1 Processor -- start 2023-09-30T22:31:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.592+0000 7fe82dd4a700 1 -- start start 2023-09-30T22:31:23.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.592+0000 7fe82dd4a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8280a0a30 0x7fe8281227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.592+0000 7fe82dd4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8280a1340 0x7fe828122d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.592+0000 7fe82dd4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe828133fa0 0x7fe828123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.592+0000 7fe82dd4a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe828123c90 con 0x7fe8280a1340 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.592+0000 7fe82dd4a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8281332e0 con 0x7fe828133fa0 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe82dd4a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8281244e0 con 0x7fe8280a0a30 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe826ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8280a1340 0x7fe828122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe826ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8280a1340 0x7fe828122d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55366/0 (socket says 172.21.15.32:55366) 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe826ffd700 1 -- 172.21.15.32:0/708001171 learned_addr learned my addr 172.21.15.32:0/708001171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe827fff700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe828133fa0 0x7fe828123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8280a0a30 0x7fe8281227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 -- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe828133fa0 msgr2=0x7fe828123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe828133fa0 0x7fe828123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 -- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8280a1340 msgr2=0x7fe828122d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8280a1340 0x7fe828122d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 -- 172.21.15.32:0/708001171 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0094a0 con 0x7fe8280a0a30 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe827fff700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe828133fa0 0x7fe828123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.593+0000 7fe8277fe700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8280a0a30 0x7fe8281227c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe81800e930 tx=0x7fe81800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.594+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818013680 con 0x7fe8280a0a30 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.594+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8180137e0 con 0x7fe8280a0a30 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.594+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe828133710 con 0x7fe8280a0a30 2023-09-30T22:31:23.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.594+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818013b50 con 0x7fe8280a0a30 2023-09-30T22:31:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.594+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe828124bc0 con 0x7fe8280a0a30 2023-09-30T22:31:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.595+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe828125e50 con 0x7fe8280a0a30 2023-09-30T22:31:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.595+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe818013cb0 con 0x7fe8280a0a30 2023-09-30T22:31:23.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.595+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe81803f9b0 con 0x7fe8280a0a30 2023-09-30T22:31:23.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.604+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe81801abe0 con 0x7fe8280a0a30 2023-09-30T22:31:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.723+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa0a40a0220 con 0x7fa0a40a0a30 2023-09-30T22:31:23.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.723+0000 7fa08b7fe700 1 -- 172.21.15.32:0/241743424 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa09401b430 con 0x7fa0a40a0a30 2023-09-30T22:31:23.723 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.725+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 msgr2=0x7fa0a41212d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.725+0000 7fa0a965f700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a41212d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fa094011db0 tx=0x7fa094004750 comp rx=0 tx=0).stop 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 shutdown_connections 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa0a4133fa0 0x7fa0a4121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0a40a0a30 0x7fa0a41212d0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 --2- 172.21.15.32:0/241743424 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa0a40a1340 0x7fa0a4121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 >> 172.21.15.32:0/241743424 conn(0x7fa0a40a6550 msgr2=0x7fa0a40a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 shutdown_connections 2023-09-30T22:31:23.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.726+0000 7fa0a965f700 1 -- 172.21.15.32:0/241743424 wait complete. 2023-09-30T22:31:23.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.770+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e840a28e0 con 0x7f2e840a1340 2023-09-30T22:31:23.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.770+0000 7f2e72ffd700 1 -- 172.21.15.32:0/3515410133 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e78015030 con 0x7f2e840a1340 2023-09-30T22:31:23.770 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.772+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 msgr2=0x7f2e84120670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.772+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 0x7f2e84120670 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009c40 tx=0x7f2e78004750 comp rx=0 tx=0).stop 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 shutdown_connections 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2e84133fa0 0x7f2e84120bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f2e840a0a30 0x7f2e84120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 --2- 172.21.15.32:0/3515410133 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2e840a1340 0x7f2e84120670 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 >> 172.21.15.32:0/3515410133 conn(0x7f2e840a6550 msgr2=0x7f2e841bb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 shutdown_connections 2023-09-30T22:31:23.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.773+0000 7f2e88f2a700 1 -- 172.21.15.32:0/3515410133 wait complete. 2023-09-30T22:31:23.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:23.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.811+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe8280a1d40 con 0x7fe8280a0a30 2023-09-30T22:31:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.811+0000 7fe80ffff700 1 -- 172.21.15.32:0/708001171 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe818021020 con 0x7fe8280a0a30 2023-09-30T22:31:23.811 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8280a0a30 msgr2=0x7fe8281227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8280a0a30 0x7fe8281227c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe81800e930 tx=0x7fe81800ec40 comp rx=0 tx=0).stop 2023-09-30T22:31:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 shutdown_connections 2023-09-30T22:31:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe828133fa0 0x7fe828123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe8280a0a30 0x7fe8281227c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 --2- 172.21.15.32:0/708001171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8280a1340 0x7fe828122d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 >> 172.21.15.32:0/708001171 conn(0x7fe8280a6550 msgr2=0x7fe82811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 shutdown_connections 2023-09-30T22:31:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.814+0000 7fe82dd4a700 1 -- 172.21.15.32:0/708001171 wait complete. 2023-09-30T22:31:23.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.947+0000 7f1373b3d700 1 Processor -- start 2023-09-30T22:31:23.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.947+0000 7f1373b3d700 1 -- start start 2023-09-30T22:31:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f1373b3d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f136c09e930 0x7f136c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f1373b3d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f650 0x7f136c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f1373b3d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c070940 con 0x7f136c09f240 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f1373b3d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136c06fd60 con 0x7f136c09e930 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f1373b3d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f136c06fa80 con 0x7f136c09f650 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13718d9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f136c09e930 0x7f136c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13710d8700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f136c09f240 0x7f136c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43884/0 (socket says 172.21.15.32:43884) 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13710d8700 1 -- 172.21.15.32:0/630357725 learned_addr learned my addr 172.21.15.32:0/630357725 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13720da700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f650 0x7f136c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13718d9700 1 -- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f650 msgr2=0x7f136c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13718d9700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f650 0x7f136c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13718d9700 1 -- 172.21.15.32:0/630357725 >> v1:172.21.15.32:6789/0 conn(0x7f136c09f240 legacy=0x7f136c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13718d9700 1 -- 172.21.15.32:0/630357725 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c09fb60 con 0x7f136c09e930 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13708d7700 1 -- 172.21.15.32:0/630357725 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353778970 0 0) 0x7f136c070940 con 0x7f136c09f240 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13720da700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f650 0x7f136c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.948+0000 7f13718d9700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09e930 0x7f136c09ed00 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1360009840 tx=0x7f1360009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af49970553903915 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f13708d7700 1 -- 172.21.15.32:0/630357725 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360011830 con 0x7f136c09e930 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f13708d7700 1 -- 172.21.15.32:0/630357725 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1360011990 con 0x7f136c09e930 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f13708d7700 1 -- 172.21.15.32:0/630357725 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360011d20 con 0x7f136c09e930 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 -- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09e930 msgr2=0x7f136c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09e930 0x7f136c09ed00 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1360009840 tx=0x7f1360009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 -- 172.21.15.32:0/630357725 shutdown_connections 2023-09-30T22:31:23.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f650 0x7f136c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 --2- 172.21.15.32:0/630357725 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09e930 0x7f136c09ed00 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 -- 172.21.15.32:0/630357725 >> 172.21.15.32:0/630357725 conn(0x7f136c0a4210 msgr2=0x7f136c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 -- 172.21.15.32:0/630357725 shutdown_connections 2023-09-30T22:31:23.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.949+0000 7f1373b3d700 1 -- 172.21.15.32:0/630357725 wait complete. 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.950+0000 7f1373b3d700 1 Processor -- start 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.950+0000 7f1373b3d700 1 -- start start 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f1373b3d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f136c09e930 0x7f136c1a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f1373b3d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 0x7f136c1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f1373b3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09f650 0x7f136c1a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f1373b3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1a5940 con 0x7f136c09f650 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f1373b3d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1a6b90 con 0x7f136c09e930 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f1373b3d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f136c1a5f20 con 0x7f136c09f240 2023-09-30T22:31:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13710d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 0x7f136c1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f136c09e930 0x7f136c1a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13710d8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 0x7f136c1a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57430/0 (socket says 172.21.15.32:57430) 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13720da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09f650 0x7f136c1a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13710d8700 1 -- 172.21.15.32:0/2354297602 learned_addr learned my addr 172.21.15.32:0/2354297602 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 -- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 msgr2=0x7f136c1a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 0x7f136c1a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 -- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09f650 msgr2=0x7f136c1a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09f650 0x7f136c1a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 -- 172.21.15.32:0/2354297602 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364009a70 con 0x7f136c09e930 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13710d8700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 0x7f136c1a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.951+0000 7f13718d9700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f136c09e930 0x7f136c1a3d30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1360016000 tx=0x7f1360004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13600045d0 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136001f070 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13600094a0 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13600038e0 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c0a0190 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1360003a40 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.952+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f1360021070 con 0x7f136c09e930 2023-09-30T22:31:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.953+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c1a0b30 con 0x7f136c09e930 2023-09-30T22:31:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:23.958+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f136002b080 con 0x7f136c09e930 2023-09-30T22:31:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.164+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f136c071d20 con 0x7f136c09e930 2023-09-30T22:31:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.164+0000 7f135e7fc700 1 -- 172.21.15.32:0/2354297602 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1360015020 con 0x7f136c09e930 2023-09-30T22:31:24.164 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:24.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f136c09e930 msgr2=0x7f136c1a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f136c09e930 0x7f136c1a3d30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1360016000 tx=0x7f1360004750 comp rx=0 tx=0).stop 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 shutdown_connections 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f136c09f650 0x7f136c1a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f136c09e930 0x7f136c1a3d30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 --2- 172.21.15.32:0/2354297602 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f136c09f240 0x7f136c1a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.166+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 >> 172.21.15.32:0/2354297602 conn(0x7f136c0a4210 msgr2=0x7f136c1b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.167+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 shutdown_connections 2023-09-30T22:31:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.167+0000 7f1373b3d700 1 -- 172.21.15.32:0/2354297602 wait complete. 2023-09-30T22:31:24.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:24.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:24.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:24.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.892+0000 7f9b5857a700 1 Processor -- start 2023-09-30T22:31:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.892+0000 7f9b5857a700 1 -- start start 2023-09-30T22:31:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b5857a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130540 0x7f9b50130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b5857a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130e50 0x7f9b5006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b5857a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b500716a0 con 0x7f9b5006fb20 2023-09-30T22:31:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b5857a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5009a390 con 0x7f9b50130540 2023-09-30T22:31:24.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b5857a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50071290 con 0x7f9b50130e50 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56b17700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b5006fb20 0x7f9b50137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43888/0 (socket says 172.21.15.32:43888) 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56b17700 1 -- 172.21.15.32:0/648807832 learned_addr learned my addr 172.21.15.32:0/648807832 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56316700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130540 0x7f9b50130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b55b15700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130e50 0x7f9b5006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b55314700 1 -- 172.21.15.32:0/648807832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507960645 0 0) 0x7f9b500716a0 con 0x7f9b5006fb20 2023-09-30T22:31:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56316700 1 -- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130e50 msgr2=0x7f9b5006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56316700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130e50 0x7f9b5006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56316700 1 -- 172.21.15.32:0/648807832 >> v1:172.21.15.32:6789/0 conn(0x7f9b5006fb20 legacy=0x7f9b50137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b56316700 1 -- 172.21.15.32:0/648807832 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b50130340 con 0x7f9b50130540 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.893+0000 7f9b55b15700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130e50 0x7f9b5006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.894+0000 7f9b56316700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130540 0x7f9b50130910 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098a0 tx=0x7f9b4c009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3538011b8ac21e56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.894+0000 7f9b55314700 1 -- 172.21.15.32:0/648807832 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c011830 con 0x7f9b50130540 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.894+0000 7f9b55314700 1 -- 172.21.15.32:0/648807832 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c011990 con 0x7f9b50130540 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.894+0000 7f9b55314700 1 -- 172.21.15.32:0/648807832 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c011d20 con 0x7f9b50130540 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 -- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130540 msgr2=0x7f9b50130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130540 0x7f9b50130910 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098a0 tx=0x7f9b4c009bb0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 -- 172.21.15.32:0/648807832 shutdown_connections 2023-09-30T22:31:24.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130540 0x7f9b50130910 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 --2- 172.21.15.32:0/648807832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130e50 0x7f9b5006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 -- 172.21.15.32:0/648807832 >> 172.21.15.32:0/648807832 conn(0x7f9b500a2aa0 msgr2=0x7f9b500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 -- 172.21.15.32:0/648807832 shutdown_connections 2023-09-30T22:31:24.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.895+0000 7f9b5857a700 1 -- 172.21.15.32:0/648807832 wait complete. 2023-09-30T22:31:24.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 Processor -- start 2023-09-30T22:31:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 -- start start 2023-09-30T22:31:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5006fb20 0x7f9b50138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130540 0x7f9b50138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 0x7f9b501c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50134a80 con 0x7f9b5006fb20 2023-09-30T22:31:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50139170 con 0x7f9b50130e50 2023-09-30T22:31:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.897+0000 7f9b5857a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500718e0 con 0x7f9b50130540 2023-09-30T22:31:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 0x7f9b501c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 0x7f9b501c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56076/0 (socket says 172.21.15.32:56076) 2023-09-30T22:31:24.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 -- 172.21.15.32:0/923010698 learned_addr learned my addr 172.21.15.32:0/923010698 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56316700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5006fb20 0x7f9b50138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b55b15700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130540 0x7f9b50138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 -- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130540 msgr2=0x7f9b50138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130540 0x7f9b50138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 -- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5006fb20 msgr2=0x7f9b50138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5006fb20 0x7f9b50138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 -- 172.21.15.32:0/923010698 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400094a0 con 0x7f9b50130e50 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b55b15700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130540 0x7f9b50138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b56b17700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 0x7f9b501c5c70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9b48002230 tx=0x7f9b4800cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.898+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800aac0 con 0x7f9b50130e50 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.899+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4800ac40 con 0x7f9b50130e50 2023-09-30T22:31:24.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.899+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c0094a0 con 0x7f9b50130e50 2023-09-30T22:31:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.899+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480070d0 con 0x7f9b50130e50 2023-09-30T22:31:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.899+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b50138dc0 con 0x7f9b50130e50 2023-09-30T22:31:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.900+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b48007250 con 0x7f9b50130e50 2023-09-30T22:31:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.900+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b50074ea0 con 0x7f9b50130e50 2023-09-30T22:31:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.900+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b4803f190 con 0x7f9b50130e50 2023-09-30T22:31:24.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.905+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b48007500 con 0x7f9b50130e50 2023-09-30T22:31:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.933+0000 7fdea4a4f700 1 Processor -- start 2023-09-30T22:31:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.933+0000 7fdea4a4f700 1 -- start start 2023-09-30T22:31:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fdea4a4f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdea01338a0 0x7fdea0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fdea4a4f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fdea4a4f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea006ef30 con 0x7fdea009e760 2023-09-30T22:31:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fdea4a4f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea006fef0 con 0x7fdea01338a0 2023-09-30T22:31:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fdea4a4f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdea01334b0 con 0x7fdea009de50 2023-09-30T22:31:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdea009e760 0x7fdea013a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43904/0 (socket says 172.21.15.32:43904) 2023-09-30T22:31:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9ed9d700 1 -- 172.21.15.32:0/3402679950 learned_addr learned my addr 172.21.15.32:0/3402679950 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9e59c700 1 --2- 172.21.15.32:0/3402679950 >> v2:172.21.15.32:3300/0 conn(0x7fdea01338a0 0x7fdea0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9d59a700 1 -- 172.21.15.32:0/3402679950 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675054880 0 0) 0x7fdea006ef30 con 0x7fdea009e760 2023-09-30T22:31:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9d59a700 1 -- 172.21.15.32:0/3402679950 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde880035b0 con 0x7fdea009e760 2023-09-30T22:31:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9e59c700 1 -- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 msgr2=0x7fdea009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9e59c700 1 --2- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea009e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9e59c700 1 -- 172.21.15.32:0/3402679950 >> v1:172.21.15.32:6789/0 conn(0x7fdea009e760 legacy=0x7fdea013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9e59c700 1 -- 172.21.15.32:0/3402679950 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea006fc30 con 0x7fdea01338a0 2023-09-30T22:31:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.934+0000 7fde9dd9b700 1 --2- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.935+0000 7fde9e59c700 1 --2- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea01338a0 0x7fdea0133c70 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fde90009840 tx=0x7fde90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456b11235ab92aeb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.935+0000 7fde9d59a700 1 -- 172.21.15.32:0/3402679950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90011830 con 0x7fdea01338a0 2023-09-30T22:31:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.935+0000 7fde9d59a700 1 -- 172.21.15.32:0/3402679950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90011990 con 0x7fdea01338a0 2023-09-30T22:31:24.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.935+0000 7fde9d59a700 1 -- 172.21.15.32:0/3402679950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90011d20 con 0x7fdea01338a0 2023-09-30T22:31:24.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.935+0000 7fdea4a4f700 1 -- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea01338a0 msgr2=0x7fdea0133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.935+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea01338a0 0x7fdea0133c70 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fde90009840 tx=0x7fde90009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.936+0000 7fdea4a4f700 1 -- 172.21.15.32:0/3402679950 shutdown_connections 2023-09-30T22:31:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.936+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea01338a0 0x7fdea0133c70 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.936+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/3402679950 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.936+0000 7fdea4a4f700 1 -- 172.21.15.32:0/3402679950 >> 172.21.15.32:0/3402679950 conn(0x7fdea00a2aa0 msgr2=0x7fdea00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.936+0000 7fdea4a4f700 1 -- 172.21.15.32:0/3402679950 shutdown_connections 2023-09-30T22:31:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.936+0000 7fdea4a4f700 1 -- 172.21.15.32:0/3402679950 wait complete. 2023-09-30T22:31:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.937+0000 7fdea4a4f700 1 Processor -- start 2023-09-30T22:31:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.937+0000 7fdea4a4f700 1 -- start start 2023-09-30T22:31:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fdea4a4f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea011b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fdea4a4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 0x7fdea011bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fdea4a4f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdea01338a0 0x7fdea011c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fdea4a4f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea011faa0 con 0x7fdea009e760 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fdea4a4f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea011d200 con 0x7fdea01338a0 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fdea4a4f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdea011f0e0 con 0x7fdea009de50 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fde9dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 0x7fdea011bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fde9dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 0x7fdea011bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55412/0 (socket says 172.21.15.32:55412) 2023-09-30T22:31:24.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fde9dd9b700 1 -- 172.21.15.32:0/1930240954 learned_addr learned my addr 172.21.15.32:0/1930240954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:24.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fde9e59c700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea011b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.938+0000 7fde9ed9d700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdea01338a0 0x7fdea011c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9dd9b700 1 -- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 msgr2=0x7fdea011b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9dd9b700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea011b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9dd9b700 1 -- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdea01338a0 msgr2=0x7fdea011c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9dd9b700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdea01338a0 0x7fdea011c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9dd9b700 1 -- 172.21.15.32:0/1930240954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde940094a0 con 0x7fdea009e760 2023-09-30T22:31:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9e59c700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea011b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde9dd9b700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 0x7fdea011bb80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fde9400ebe0 tx=0x7fde9400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400c850 con 0x7fdea009e760 2023-09-30T22:31:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400c9b0 con 0x7fdea009e760 2023-09-30T22:31:24.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400cd40 con 0x7fdea009e760 2023-09-30T22:31:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900094a0 con 0x7fdea009e760 2023-09-30T22:31:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.939+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea011df60 con 0x7fdea009e760 2023-09-30T22:31:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.941+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea0074ea0 con 0x7fdea009e760 2023-09-30T22:31:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.941+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde94013470 con 0x7fdea009e760 2023-09-30T22:31:24.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.941+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fde9403ff80 con 0x7fdea009e760 2023-09-30T22:31:24.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.946+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde94013720 con 0x7fdea009e760 2023-09-30T22:31:24.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.974+0000 7fcaa4e9e700 1 Processor -- start 2023-09-30T22:31:24.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.974+0000 7fcaa4e9e700 1 -- start start 2023-09-30T22:31:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fcaa4e9e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcaa009c650 0x7fcaa009ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fcaa4e9e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa00a0090 0x7fcaa009cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fcaa4e9e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa009ba70 con 0x7fcaa0131970 2023-09-30T22:31:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fcaa4e9e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa009d4a0 con 0x7fcaa009c650 2023-09-30T22:31:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fcaa4e9e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa009e8f0 con 0x7fcaa00a0090 2023-09-30T22:31:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fca9e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcaa009c650 0x7fcaa009ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fca9ed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcaa0131970 0x7fcaa0131db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43906/0 (socket says 172.21.15.32:43906) 2023-09-30T22:31:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fca9ed9d700 1 -- 172.21.15.32:0/4181674112 learned_addr learned my addr 172.21.15.32:0/4181674112 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.975+0000 7fca9dd9b700 1 --2- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa00a0090 0x7fcaa009cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9e59c700 1 -- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa00a0090 msgr2=0x7fcaa009cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9e59c700 1 --2- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa00a0090 0x7fcaa009cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9e59c700 1 -- 172.21.15.32:0/4181674112 >> v1:172.21.15.32:6789/0 conn(0x7fcaa0131970 legacy=0x7fcaa0131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9d59a700 1 -- 172.21.15.32:0/4181674112 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688243368 0 0) 0x7fcaa009ba70 con 0x7fcaa0131970 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9e59c700 1 -- 172.21.15.32:0/4181674112 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00a0760 con 0x7fcaa009c650 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9e59c700 1 --2- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa009ca20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fca90009840 tx=0x7fca90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d42b278569c0278 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9d59a700 1 -- 172.21.15.32:0/4181674112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011830 con 0x7fcaa009c650 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9d59a700 1 -- 172.21.15.32:0/4181674112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90011990 con 0x7fcaa009c650 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.976+0000 7fca9d59a700 1 -- 172.21.15.32:0/4181674112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011d20 con 0x7fcaa009c650 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 msgr2=0x7fcaa009ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa009ca20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fca90009840 tx=0x7fca90009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/4181674112 shutdown_connections 2023-09-30T22:31:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa009ca20 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/4181674112 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa00a0090 0x7fcaa009cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/4181674112 >> 172.21.15.32:0/4181674112 conn(0x7fcaa00a3f70 msgr2=0x7fcaa00a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/4181674112 shutdown_connections 2023-09-30T22:31:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.977+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/4181674112 wait complete. 2023-09-30T22:31:24.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.979+0000 7fcaa4e9e700 1 Processor -- start 2023-09-30T22:31:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.979+0000 7fcaa4e9e700 1 -- start start 2023-09-30T22:31:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fcaa4e9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fcaa4e9e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcaa00a0090 0x7fcaa006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fcaa4e9e700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa0131970 0x7fcaa006c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fcaa4e9e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa011f860 con 0x7fcaa009c650 2023-09-30T22:31:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fcaa4e9e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa01204b0 con 0x7fcaa00a0090 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fcaa4e9e700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa006cdc0 con 0x7fcaa0131970 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55422/0 (socket says 172.21.15.32:55422) 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 -- 172.21.15.32:0/1337247328 learned_addr learned my addr 172.21.15.32:0/1337247328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9dd9b700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcaa00a0090 0x7fcaa006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9ed9d700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa0131970 0x7fcaa006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 -- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa0131970 msgr2=0x7fcaa006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa0131970 0x7fcaa006c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 -- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcaa00a0090 msgr2=0x7fcaa006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcaa00a0090 0x7fcaa006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.980+0000 7fca9e59c700 1 -- 172.21.15.32:0/1337247328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900094a0 con 0x7fcaa009c650 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.981+0000 7fca9e59c700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa006b880 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fca900090b0 tx=0x7fca90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.981+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90004550 con 0x7fcaa009c650 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.981+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9001f070 con 0x7fcaa009c650 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.981+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900038e0 con 0x7fcaa009c650 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.981+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa009d980 con 0x7fcaa009c650 2023-09-30T22:31:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.981+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa0072340 con 0x7fcaa009c650 2023-09-30T22:31:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.982+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa00747f0 con 0x7fcaa009c650 2023-09-30T22:31:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.983+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca90003a40 con 0x7fcaa009c650 2023-09-30T22:31:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.983+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca90021070 con 0x7fcaa009c650 2023-09-30T22:31:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:24.987+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca900253e0 con 0x7fcaa009c650 2023-09-30T22:31:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.116+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b501370d0 con 0x7f9b50130e50 2023-09-30T22:31:25.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.116+0000 7f9b46ffd700 1 -- 172.21.15.32:0/923010698 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b48018020 con 0x7f9b50130e50 2023-09-30T22:31:25.117 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 msgr2=0x7f9b501c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 0x7f9b501c5c70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9b48002230 tx=0x7f9b4800cf50 comp rx=0 tx=0).stop 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 shutdown_connections 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b50130e50 0x7f9b501c5c70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5006fb20 0x7f9b50138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 --2- 172.21.15.32:0/923010698 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b50130540 0x7f9b50138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 >> 172.21.15.32:0/923010698 conn(0x7f9b500a2aa0 msgr2=0x7f9b5006e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 shutdown_connections 2023-09-30T22:31:25.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.119+0000 7f9b5857a700 1 -- 172.21.15.32:0/923010698 wait complete. 2023-09-30T22:31:25.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:25.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.174+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdea011eaa0 con 0x7fdea009e760 2023-09-30T22:31:25.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.174+0000 7fde8effd700 1 -- 172.21.15.32:0/1930240954 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde94021020 con 0x7fdea009e760 2023-09-30T22:31:25.174 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 msgr2=0x7fdea011bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 0x7fdea011bb80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fde9400ebe0 tx=0x7fde9400eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 shutdown_connections 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdea01338a0 0x7fdea011c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fdea009de50 0x7fdea011b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 --2- 172.21.15.32:0/1930240954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdea009e760 0x7fdea011bb80 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 >> 172.21.15.32:0/1930240954 conn(0x7fdea00a2aa0 msgr2=0x7fdea01305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 shutdown_connections 2023-09-30T22:31:25.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.176+0000 7fdea4a4f700 1 -- 172.21.15.32:0/1930240954 wait complete. 2023-09-30T22:31:25.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:25.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.215+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcaa0072840 con 0x7fcaa009c650 2023-09-30T22:31:25.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.215+0000 7fca8effd700 1 -- 172.21.15.32:0/1337247328 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca90015070 con 0x7fcaa009c650 2023-09-30T22:31:25.215 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.217+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 msgr2=0x7fcaa006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.217+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa006b880 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fca900090b0 tx=0x7fca90004750 comp rx=0 tx=0).stop 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 shutdown_connections 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fcaa0131970 0x7fcaa006c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa009c650 0x7fcaa006b880 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 --2- 172.21.15.32:0/1337247328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcaa00a0090 0x7fcaa006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 >> 172.21.15.32:0/1337247328 conn(0x7fcaa00a3f70 msgr2=0x7fcaa012ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 shutdown_connections 2023-09-30T22:31:25.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.218+0000 7fcaa4e9e700 1 -- 172.21.15.32:0/1337247328 wait complete. 2023-09-30T22:31:25.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.320+0000 7f8936f7f700 1 Processor -- start 2023-09-30T22:31:25.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.320+0000 7f8936f7f700 1 -- start start 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8936f7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f893009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8936f7f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 0x7f8930134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8936f7f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893009b120 con 0x7f893009e000 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8936f7f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893006b460 con 0x7f893009e410 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8936f7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893009a7e0 con 0x7f893009d6f0 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8934d1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f893009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f893551c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 0x7f8930134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8934d1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f893009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56096/0 (socket says 172.21.15.32:56096) 2023-09-30T22:31:25.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f8934d1b700 1 -- 172.21.15.32:0/2583114386 learned_addr learned my addr 172.21.15.32:0/2583114386 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f893551c700 1 --2- 172.21.15.32:0/2583114386 >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 0x7f8930134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55434/0 (socket says 172.21.15.32:55434) 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.321+0000 7f892f7fe700 1 -- 172.21.15.32:0/2583114386 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538770478 0 0) 0x7f893009b120 con 0x7f893009e000 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f8934d1b700 1 -- 172.21.15.32:0/2583114386 >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 msgr2=0x7f8930134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f8934d1b700 1 --2- 172.21.15.32:0/2583114386 >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 0x7f8930134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f8934d1b700 1 -- 172.21.15.32:0/2583114386 >> v1:172.21.15.32:6789/0 conn(0x7f893009e000 legacy=0x7f89301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f8934d1b700 1 -- 172.21.15.32:0/2583114386 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893009a500 con 0x7f893009d6f0 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f893551c700 1 --2- 172.21.15.32:0/2583114386 >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 0x7f8930134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f8934d1b700 1 --2- 172.21.15.32:0/2583114386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f893009dac0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8920009920 tx=0x7f8920009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7be2016df6117b1d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f892f7fe700 1 -- 172.21.15.32:0/2583114386 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920011830 con 0x7f893009d6f0 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f892f7fe700 1 -- 172.21.15.32:0/2583114386 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8920011990 con 0x7f893009d6f0 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.322+0000 7f892f7fe700 1 -- 172.21.15.32:0/2583114386 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920011d20 con 0x7f893009d6f0 2023-09-30T22:31:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 -- 172.21.15.32:0/2583114386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 msgr2=0x7f893009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 --2- 172.21.15.32:0/2583114386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f893009dac0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8920009920 tx=0x7f8920009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 -- 172.21.15.32:0/2583114386 shutdown_connections 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 --2- 172.21.15.32:0/2583114386 >> v2:172.21.15.32:3300/0 conn(0x7f893009e410 0x7f8930134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 --2- 172.21.15.32:0/2583114386 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f893009dac0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 -- 172.21.15.32:0/2583114386 >> 172.21.15.32:0/2583114386 conn(0x7f8930066f50 msgr2=0x7f893009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 -- 172.21.15.32:0/2583114386 shutdown_connections 2023-09-30T22:31:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.323+0000 7f8936f7f700 1 -- 172.21.15.32:0/2583114386 wait complete. 2023-09-30T22:31:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.324+0000 7f8936f7f700 1 Processor -- start 2023-09-30T22:31:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.324+0000 7f8936f7f700 1 -- start start 2023-09-30T22:31:25.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8936f7f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f89301a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8936f7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893009e000 0x7f89301a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8936f7f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893009e410 0x7f89301a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8936f7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89301aa220 con 0x7f893009e000 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8936f7f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89301a7950 con 0x7f893009d6f0 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8936f7f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89301a7f70 con 0x7f893009e410 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8934d1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f89301a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f892ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893009e000 0x7f89301a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f893551c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893009e410 0x7f89301a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8934d1b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f89301a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56108/0 (socket says 172.21.15.32:56108) 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f8934d1b700 1 -- 172.21.15.32:0/1311936794 learned_addr learned my addr 172.21.15.32:0/1311936794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:25.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f892ffff700 1 -- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893009e410 msgr2=0x7f89301a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f892ffff700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893009e410 0x7f89301a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.325+0000 7f892ffff700 1 -- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 msgr2=0x7f89301a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f892ffff700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f89301a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f892ffff700 1 -- 172.21.15.32:0/1311936794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928009a70 con 0x7f893009e000 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f8934d1b700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f89301a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f892ffff700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893009e000 0x7f89301a6320 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f891800c430 tx=0x7f891800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180052d0 con 0x7f893009e000 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8918005450 con 0x7f893009e000 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8920009550 con 0x7f893009e000 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180038c0 con 0x7f893009e000 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.326+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89301a8ff0 con 0x7f893009e000 2023-09-30T22:31:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.327+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8918003a20 con 0x7f893009e000 2023-09-30T22:31:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.327+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89300a30f0 con 0x7f893009e000 2023-09-30T22:31:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.327+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f891803e980 con 0x7f893009e000 2023-09-30T22:31:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.332+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f891801b8d0 con 0x7f893009e000 2023-09-30T22:31:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.540+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f893009c710 con 0x7f893009e000 2023-09-30T22:31:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.540+0000 7f892d7fa700 1 -- 172.21.15.32:0/1311936794 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8918017030 con 0x7f893009e000 2023-09-30T22:31:25.540 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893009e000 msgr2=0x7f89301a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893009e000 0x7f89301a6320 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f891800c430 tx=0x7f891800c740 comp rx=0 tx=0).stop 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 shutdown_connections 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f893009e410 0x7f89301a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f893009d6f0 0x7f89301a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 --2- 172.21.15.32:0/1311936794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f893009e000 0x7f89301a6320 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.542+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 >> 172.21.15.32:0/1311936794 conn(0x7f8930066f50 msgr2=0x7f89301bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.543+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 shutdown_connections 2023-09-30T22:31:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:25.543+0000 7f8936f7f700 1 -- 172.21.15.32:0/1311936794 wait complete. 2023-09-30T22:31:25.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:26.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:26.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:26.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.286+0000 7f28cb21a700 1 Processor -- start 2023-09-30T22:31:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.286+0000 7f28cb21a700 1 -- start start 2023-09-30T22:31:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28cb21a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c41338a0 0x7f28c4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28cb21a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 0x7f28c409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28cb21a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28c409e760 0x7f28c413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28cb21a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c406fef0 con 0x7f28c409e760 2023-09-30T22:31:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28cb21a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c406ef30 con 0x7f28c41338a0 2023-09-30T22:31:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28cb21a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41334b0 con 0x7f28c409de50 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 0x7f28c409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 0x7f28c409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57512/0 (socket says 172.21.15.32:57512) 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 -- 172.21.15.32:0/3422018728 learned_addr learned my addr 172.21.15.32:0/3422018728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c8fb6700 1 --2- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c41338a0 0x7f28c4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c97b7700 1 --2- 172.21.15.32:0/3422018728 >> v2:172.21.15.32:3300/0 conn(0x7f28c409e760 0x7f28c413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 -- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c41338a0 msgr2=0x7f28c4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 --2- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c41338a0 0x7f28c4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 -- 172.21.15.32:0/3422018728 >> v2:172.21.15.32:3300/0 conn(0x7f28c409e760 msgr2=0x7f28c413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 --2- 172.21.15.32:0/3422018728 >> v2:172.21.15.32:3300/0 conn(0x7f28c409e760 0x7f28c413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.287+0000 7f28c3fff700 1 -- 172.21.15.32:0/3422018728 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c406fc30 con 0x7f28c409de50 2023-09-30T22:31:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.288+0000 7f28c3fff700 1 --2- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 0x7f28c409e220 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009920 tx=0x7f28b0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411527a6a368f2fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.288+0000 7f28c37fe700 1 -- 172.21.15.32:0/3422018728 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0011830 con 0x7f28c409de50 2023-09-30T22:31:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.288+0000 7f28c37fe700 1 -- 172.21.15.32:0/3422018728 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0011990 con 0x7f28c409de50 2023-09-30T22:31:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.288+0000 7f28c37fe700 1 -- 172.21.15.32:0/3422018728 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0011d20 con 0x7f28c409de50 2023-09-30T22:31:26.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 -- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 msgr2=0x7f28c409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 --2- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 0x7f28c409e220 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009920 tx=0x7f28b0009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:26.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 -- 172.21.15.32:0/3422018728 shutdown_connections 2023-09-30T22:31:26.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 --2- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c41338a0 0x7f28c4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 --2- 172.21.15.32:0/3422018728 >> v2:172.21.15.32:3300/0 conn(0x7f28c409e760 0x7f28c413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 --2- 172.21.15.32:0/3422018728 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c409de50 0x7f28c409e220 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 -- 172.21.15.32:0/3422018728 >> 172.21.15.32:0/3422018728 conn(0x7f28c40a2aa0 msgr2=0x7f28c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 -- 172.21.15.32:0/3422018728 shutdown_connections 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.289+0000 7f28cb21a700 1 -- 172.21.15.32:0/3422018728 wait complete. 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.290+0000 7f28cb21a700 1 Processor -- start 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.290+0000 7f28cb21a700 1 -- start start 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.290+0000 7f28cb21a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c409de50 0x7f28c411ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.290+0000 7f28cb21a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c409e760 0x7f28c411ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.290+0000 7f28cb21a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c41338a0 0x7f28c411f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.290+0000 7f28cb21a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4122280 con 0x7f28c409e760 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28cb21a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4120760 con 0x7f28c409de50 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28cb21a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4120e90 con 0x7f28c41338a0 2023-09-30T22:31:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c409e760 0x7f28c411ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c409e760 0x7f28c411ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55470/0 (socket says 172.21.15.32:55470) 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c3fff700 1 -- 172.21.15.32:0/2563661555 learned_addr learned my addr 172.21.15.32:0/2563661555 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c409de50 0x7f28c411ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c97b7700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c41338a0 0x7f28c411f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 -- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c41338a0 msgr2=0x7f28c411f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c41338a0 0x7f28c411f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 -- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c409e760 msgr2=0x7f28c411ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c409e760 0x7f28c411ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 -- 172.21.15.32:0/2563661555 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b0009550 con 0x7f28c409de50 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.291+0000 7f28c8fb6700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c409de50 0x7f28c411ea40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f28b800e930 tx=0x7f28b800ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.292+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8013680 con 0x7f28c409de50 2023-09-30T22:31:26.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.292+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b80137e0 con 0x7f28c409de50 2023-09-30T22:31:26.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.292+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c40a1160 con 0x7f28c409de50 2023-09-30T22:31:26.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.292+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8013b50 con 0x7f28c409de50 2023-09-30T22:31:26.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.292+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c4122cf0 con 0x7f28c409de50 2023-09-30T22:31:26.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.293+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28b8013cb0 con 0x7f28c409de50 2023-09-30T22:31:26.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.293+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f28b803f600 con 0x7f28c409de50 2023-09-30T22:31:26.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.293+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c4074ea0 con 0x7f28c409de50 2023-09-30T22:31:26.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.298+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28b801bbf0 con 0x7f28c409de50 2023-09-30T22:31:26.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.340+0000 7f6ff69ef700 1 Processor -- start 2023-09-30T22:31:26.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.340+0000 7f6ff69ef700 1 -- start start 2023-09-30T22:31:26.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6ff69ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff01358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6ff69ef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ff0135e30 0x7f6ff0138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6ff69ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff01310d0 con 0x7f6ff01305f0 2023-09-30T22:31:26.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6ff69ef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff01301c0 con 0x7f6ff0135e30 2023-09-30T22:31:26.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6ff69ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0071ff0 con 0x7f6ff0133510 2023-09-30T22:31:26.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6feffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ff01305f0 0x7f6ff01309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43928/0 (socket says 172.21.15.32:43928) 2023-09-30T22:31:26.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6feffff700 1 -- 172.21.15.32:0/3492212009 learned_addr learned my addr 172.21.15.32:0/3492212009 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:26.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6fef7fe700 1 --2- 172.21.15.32:0/3492212009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff01358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.341+0000 7f6ff4f8c700 1 --2- 172.21.15.32:0/3492212009 >> v2:172.21.15.32:3300/0 conn(0x7f6ff0135e30 0x7f6ff0138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6fef7fe700 1 -- 172.21.15.32:0/3492212009 >> v2:172.21.15.32:3300/0 conn(0x7f6ff0135e30 msgr2=0x7f6ff0138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6fef7fe700 1 --2- 172.21.15.32:0/3492212009 >> v2:172.21.15.32:3300/0 conn(0x7f6ff0135e30 0x7f6ff0138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6fef7fe700 1 -- 172.21.15.32:0/3492212009 >> v1:172.21.15.32:6789/0 conn(0x7f6ff01305f0 legacy=0x7f6ff01309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6feeffd700 1 -- 172.21.15.32:0/3492212009 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809620659 0 0) 0x7f6ff01310d0 con 0x7f6ff01305f0 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6fef7fe700 1 -- 172.21.15.32:0/3492212009 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0070e70 con 0x7f6ff0133510 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6fef7fe700 1 --2- 172.21.15.32:0/3492212009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff01358f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009840 tx=0x7f6fe4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1aa73ff94a409c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6feeffd700 1 -- 172.21.15.32:0/3492212009 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4011830 con 0x7f6ff0133510 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6feeffd700 1 -- 172.21.15.32:0/3492212009 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe4011990 con 0x7f6ff0133510 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.342+0000 7f6feeffd700 1 -- 172.21.15.32:0/3492212009 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4011d20 con 0x7f6ff0133510 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3492212009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 msgr2=0x7f6ff01358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3492212009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff01358f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009840 tx=0x7f6fe4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3492212009 shutdown_connections 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3492212009 >> v2:172.21.15.32:3300/0 conn(0x7f6ff0135e30 0x7f6ff0138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3492212009 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff01358f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3492212009 >> 172.21.15.32:0/3492212009 conn(0x7f6ff00a2aa0 msgr2=0x7f6ff00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3492212009 shutdown_connections 2023-09-30T22:31:26.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.343+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3492212009 wait complete. 2023-09-30T22:31:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.344+0000 7f6ff69ef700 1 Processor -- start 2023-09-30T22:31:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.345+0000 7f6ff69ef700 1 -- start start 2023-09-30T22:31:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.345+0000 7f6ff69ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 0x7f6ff012c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6ff69ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff012ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6ff69ef700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ff0135e30 0x7f6ff012cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6ff69ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff011b8d0 con 0x7f6ff01305f0 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6ff69ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff011afc0 con 0x7f6ff0133510 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6ff69ef700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0071820 con 0x7f6ff0135e30 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 0x7f6ff012c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 0x7f6ff012c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55484/0 (socket says 172.21.15.32:55484) 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 -- 172.21.15.32:0/3786358939 learned_addr learned my addr 172.21.15.32:0/3786358939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6fef7fe700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff012ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6ff4f8c700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ff0135e30 0x7f6ff012cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 -- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ff0135e30 msgr2=0x7f6ff012cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ff0135e30 0x7f6ff012cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 -- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 msgr2=0x7f6ff012ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff012ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 -- 172.21.15.32:0/3786358939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40094a0 con 0x7f6ff01305f0 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6fef7fe700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff012ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.346+0000 7f6feffff700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 0x7f6ff012c510 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e9c0 tx=0x7f6fe000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.347+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0004870 con 0x7f6ff01305f0 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.347+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe00049d0 con 0x7f6ff01305f0 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.347+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff012fe40 con 0x7f6ff01305f0 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.347+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0004d80 con 0x7f6ff01305f0 2023-09-30T22:31:26.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.347+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff011a0d0 con 0x7f6ff01305f0 2023-09-30T22:31:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.348+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fe00070d0 con 0x7f6ff01305f0 2023-09-30T22:31:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.348+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff011b390 con 0x7f6ff01305f0 2023-09-30T22:31:26.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.348+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f6fe0041840 con 0x7f6ff01305f0 2023-09-30T22:31:26.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.357+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fe00076b0 con 0x7f6ff01305f0 2023-09-30T22:31:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.384+0000 7f08e3150700 1 Processor -- start 2023-09-30T22:31:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.384+0000 7f08e3150700 1 -- start start 2023-09-30T22:31:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.384+0000 7f08e3150700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.384+0000 7f08e3150700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08dc09d630 0x7f08dc0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.384+0000 7f08e3150700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e3150700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc0a2a70 con 0x7f08dc09d630 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e3150700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0a01d0 con 0x7f08dc0a0a30 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e3150700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc09e2b0 con 0x7f08dc133930 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08dbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08dc09d630 0x7f08dc0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08dbfff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08dc09d630 0x7f08dc0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55500/0 (socket says 172.21.15.32:55500) 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08dbfff700 1 -- 172.21.15.32:0/1963944653 learned_addr learned my addr 172.21.15.32:0/1963944653 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e0eec700 1 --2- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 -- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 msgr2=0x7f08dc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 --2- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 -- 172.21.15.32:0/1963944653 >> v2:172.21.15.32:3300/0 conn(0x7f08dc09d630 msgr2=0x7f08dc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 --2- 172.21.15.32:0/1963944653 >> v2:172.21.15.32:3300/0 conn(0x7f08dc09d630 0x7f08dc0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 -- 172.21.15.32:0/1963944653 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc09dc50 con 0x7f08dc133930 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.385+0000 7f08e16ed700 1 --2- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc133dd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009d90 tx=0x7f08d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ee3ef50f18547f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08db7fe700 1 -- 172.21.15.32:0/1963944653 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400e070 con 0x7f08dc133930 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08db7fe700 1 -- 172.21.15.32:0/1963944653 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d400bb20 con 0x7f08dc133930 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08db7fe700 1 -- 172.21.15.32:0/1963944653 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4012430 con 0x7f08dc133930 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 -- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 msgr2=0x7f08dc133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 --2- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc133dd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009d90 tx=0x7f08d400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 -- 172.21.15.32:0/1963944653 shutdown_connections 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 --2- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 --2- 172.21.15.32:0/1963944653 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc133dd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 --2- 172.21.15.32:0/1963944653 >> v2:172.21.15.32:3300/0 conn(0x7f08dc09d630 0x7f08dc0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 -- 172.21.15.32:0/1963944653 >> 172.21.15.32:0/1963944653 conn(0x7f08dc0a3050 msgr2=0x7f08dc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 -- 172.21.15.32:0/1963944653 shutdown_connections 2023-09-30T22:31:26.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.386+0000 7f08e3150700 1 -- 172.21.15.32:0/1963944653 wait complete. 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.388+0000 7f08e3150700 1 Processor -- start 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.388+0000 7f08e3150700 1 -- start start 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e3150700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 0x7f08dc11f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e3150700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc11fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e3150700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc1c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e3150700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc121fc0 con 0x7f08dc09d630 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e3150700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc11ef90 con 0x7f08dc0a0a30 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e3150700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc120650 con 0x7f08dc133930 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 0x7f08dc11f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 0x7f08dc11f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55506/0 (socket says 172.21.15.32:55506) 2023-09-30T22:31:26.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 -- 172.21.15.32:0/1893090426 learned_addr learned my addr 172.21.15.32:0/1893090426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e16ed700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc1c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08dbfff700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc11fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 -- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 msgr2=0x7f08dc1c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc1c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 -- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 msgr2=0x7f08dc11fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc11fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.389+0000 7f08e0eec700 1 -- 172.21.15.32:0/1893090426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4009a70 con 0x7f08dc09d630 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.390+0000 7f08e0eec700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 0x7f08dc11f530 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f08d000e9b0 tx=0x7f08d000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.390+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d000ee50 con 0x7f08dc09d630 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.390+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d000c850 con 0x7f08dc09d630 2023-09-30T22:31:26.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.390+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc0a5d70 con 0x7f08dc09d630 2023-09-30T22:31:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.390+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d000cbc0 con 0x7f08dc09d630 2023-09-30T22:31:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.390+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc1334f0 con 0x7f08dc09d630 2023-09-30T22:31:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.391+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc11ffb0 con 0x7f08dc09d630 2023-09-30T22:31:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.392+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08d000cd20 con 0x7f08dc09d630 2023-09-30T22:31:26.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.392+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f08d003fe40 con 0x7f08dc09d630 2023-09-30T22:31:26.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.401+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08d0011350 con 0x7f08dc09d630 2023-09-30T22:31:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.536+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28c406b9e0 con 0x7f28c409de50 2023-09-30T22:31:26.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.536+0000 7f28c17fa700 1 -- 172.21.15.32:0/2563661555 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28b8017020 con 0x7f28c409de50 2023-09-30T22:31:26.536 INFO:teuthology.orchestra.run.smithi032.stdout:128849018883 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c409de50 msgr2=0x7f28c411ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c409de50 0x7f28c411ea40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f28b800e930 tx=0x7f28b800ec40 comp rx=0 tx=0).stop 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 shutdown_connections 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f28c41338a0 0x7f28c411f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28c409de50 0x7f28c411ea40 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 --2- 172.21.15.32:0/2563661555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28c409e760 0x7f28c411ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 >> 172.21.15.32:0/2563661555 conn(0x7f28c40a2aa0 msgr2=0x7f28c409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 shutdown_connections 2023-09-30T22:31:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.539+0000 7f28cb21a700 1 -- 172.21.15.32:0/2563661555 wait complete. 2023-09-30T22:31:26.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 128849018884 got 128849018883 for osd.1 2023-09-30T22:31:26.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:26.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.575+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6ff0074ea0 con 0x7f6ff01305f0 2023-09-30T22:31:26.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.576+0000 7f6fecff9700 1 -- 172.21.15.32:0/3786358939 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fe0015020 con 0x7f6ff01305f0 2023-09-30T22:31:26.576 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 msgr2=0x7f6ff012c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 0x7f6ff012c510 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e9c0 tx=0x7f6fe000ecd0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 shutdown_connections 2023-09-30T22:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f6ff0135e30 0x7f6ff012cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ff01305f0 0x7f6ff012c510 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 --2- 172.21.15.32:0/3786358939 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6ff0133510 0x7f6ff012ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 >> 172.21.15.32:0/3786358939 conn(0x7f6ff00a2aa0 msgr2=0x7f6ff006f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 shutdown_connections 2023-09-30T22:31:26.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.580+0000 7f6ff69ef700 1 -- 172.21.15.32:0/3786358939 wait complete. 2023-09-30T22:31:26.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.631+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f08dc0747f0 con 0x7f08dc09d630 2023-09-30T22:31:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.632+0000 7f08d97fa700 1 -- 172.21.15.32:0/1893090426 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08d0024b00 con 0x7f08dc09d630 2023-09-30T22:31:26.632 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 msgr2=0x7f08dc11f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 0x7f08dc11f530 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f08d000e9b0 tx=0x7f08d000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 shutdown_connections 2023-09-30T22:31:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f08dc133930 0x7f08dc1c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08dc09d630 0x7f08dc11f530 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 --2- 172.21.15.32:0/1893090426 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08dc0a0a30 0x7f08dc11fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 >> 172.21.15.32:0/1893090426 conn(0x7f08dc0a3050 msgr2=0x7f08dc09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 shutdown_connections 2023-09-30T22:31:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.634+0000 7f08e3150700 1 -- 172.21.15.32:0/1893090426 wait complete. 2023-09-30T22:31:26.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:31:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.699+0000 7f75fc61d700 1 Processor -- start 2023-09-30T22:31:26.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.699+0000 7f75fc61d700 1 -- start start 2023-09-30T22:31:26.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fc61d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fc61d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fc61d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75f409e410 0x7f75f4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fc61d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f406b460 con 0x7f75f409e410 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fc61d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f409b120 con 0x7f75f409d6f0 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fc61d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75f409a7e0 con 0x7f75f409e000 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fa3b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fabba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75f409e410 0x7f75f4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57548/0 (socket says 172.21.15.32:57548) 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 -- 172.21.15.32:0/1232863110 learned_addr learned my addr 172.21.15.32:0/1232863110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 -- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 msgr2=0x7f75f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 --2- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 -- 172.21.15.32:0/1232863110 >> v2:172.21.15.32:3300/0 conn(0x7f75f409e410 msgr2=0x7f75f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 --2- 172.21.15.32:0/1232863110 >> v2:172.21.15.32:3300/0 conn(0x7f75f409e410 0x7f75f4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75f9bb8700 1 -- 172.21.15.32:0/1232863110 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f409a500 con 0x7f75f409e000 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fabba700 1 --2- 172.21.15.32:0/1232863110 >> v2:172.21.15.32:3300/0 conn(0x7f75f409e410 0x7f75f4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:26.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.700+0000 7f75fa3b9700 1 --2- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75f9bb8700 1 --2- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41339f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009920 tx=0x7f75e8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638ad6e0d218acbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75f93b7700 1 -- 172.21.15.32:0/1232863110 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8011830 con 0x7f75f409e000 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75f93b7700 1 -- 172.21.15.32:0/1232863110 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8011990 con 0x7f75f409e000 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75f93b7700 1 -- 172.21.15.32:0/1232863110 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8011d20 con 0x7f75f409e000 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 -- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 msgr2=0x7f75f41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 --2- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41339f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009920 tx=0x7f75e8009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 -- 172.21.15.32:0/1232863110 shutdown_connections 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 --2- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 --2- 172.21.15.32:0/1232863110 >> v2:172.21.15.32:3300/0 conn(0x7f75f409e410 0x7f75f4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 --2- 172.21.15.32:0/1232863110 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41339f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 -- 172.21.15.32:0/1232863110 >> 172.21.15.32:0/1232863110 conn(0x7f75f4066f50 msgr2=0x7f75f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.701+0000 7f75fc61d700 1 -- 172.21.15.32:0/1232863110 shutdown_connections 2023-09-30T22:31:26.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.702+0000 7f75fc61d700 1 -- 172.21.15.32:0/1232863110 wait complete. 2023-09-30T22:31:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.702+0000 7f75fc61d700 1 Processor -- start 2023-09-30T22:31:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.702+0000 7f75fc61d700 1 -- start start 2023-09-30T22:31:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fc61d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f41a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fc61d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fc61d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f409e410 0x7f75f41a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fc61d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41a92b0 con 0x7f75f409e410 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fc61d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41a88f0 con 0x7f75f409d6f0 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fc61d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41a8080 con 0x7f75f409e000 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75f9bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f41a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fabba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f409e410 0x7f75f41a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75f9bb8700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:57552/0 (socket says 172.21.15.32:57552) 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f41a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56168/0 (socket says 172.21.15.32:56168) 2023-09-30T22:31:26.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75f9bb8700 1 -- 172.21.15.32:0/3550805791 learned_addr learned my addr 172.21.15.32:0/3550805791 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 -- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 msgr2=0x7f75f41a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 -- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f409e410 msgr2=0x7f75f41a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f409e410 0x7f75f41a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75fa3b9700 1 -- 172.21.15.32:0/3550805791 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f00094a0 con 0x7f75f409d6f0 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.703+0000 7f75f9bb8700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.704+0000 7f75fa3b9700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f41a4dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f75f000e9e0 tx=0x7f75f000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.704+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000ee80 con 0x7f75f409d6f0 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.704+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f000c850 con 0x7f75f409d6f0 2023-09-30T22:31:26.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.704+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec009a70 con 0x7f75f409d6f0 2023-09-30T22:31:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.704+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000cc00 con 0x7f75f409d6f0 2023-09-30T22:31:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.704+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e8009550 con 0x7f75f409d6f0 2023-09-30T22:31:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.705+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75f000cd60 con 0x7f75f409d6f0 2023-09-30T22:31:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.705+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f75f003fa90 con 0x7f75f409d6f0 2023-09-30T22:31:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.705+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f409c710 con 0x7f75f409d6f0 2023-09-30T22:31:26.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.710+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f75f0021070 con 0x7f75f409d6f0 2023-09-30T22:31:26.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.912+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f75f4074ea0 con 0x7f75f409d6f0 2023-09-30T22:31:26.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.912+0000 7f75e6ffd700 1 -- 172.21.15.32:0/3550805791 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75f002a090 con 0x7f75f409d6f0 2023-09-30T22:31:26.912 INFO:teuthology.orchestra.run.smithi032.stdout:137438953474 2023-09-30T22:31:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 msgr2=0x7f75f41a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f41a4dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f75f000e9e0 tx=0x7f75f000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 shutdown_connections 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75f409e410 0x7f75f41a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75f409d6f0 0x7f75f41a4dd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 --2- 172.21.15.32:0/3550805791 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f75f409e000 0x7f75f41a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 >> 172.21.15.32:0/3550805791 conn(0x7f75f4066f50 msgr2=0x7f75f41b9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 shutdown_connections 2023-09-30T22:31:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:26.914+0000 7f75fc61d700 1 -- 172.21.15.32:0/3550805791 wait complete. 2023-09-30T22:31:26.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953476 got 137438953474 for osd.3 2023-09-30T22:31:27.554 INFO:tasks.ceph:ignoring flush pg stats error, probably testing upgrade: timed out waiting for mon to be updated with osd.1: 128849018883 < 128849018884 2023-09-30T22:31:27.555 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:31:27.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:31:27.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:27.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:27.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 Processor -- start 2023-09-30T22:31:27.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 -- start start 2023-09-30T22:31:27.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3d009e000 0x7fb3d0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d009f130 con 0x7fb3d009e410 2023-09-30T22:31:27.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d009f520 con 0x7fb3d009e000 2023-09-30T22:31:27.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3d5f14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d009e930 con 0x7fb3d009d6f0 2023-09-30T22:31:27.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3ceffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3d009e000 0x7fb3d0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3cf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3ceffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3d009e000 0x7fb3d0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55530/0 (socket says 172.21.15.32:55530) 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.706+0000 7fb3ceffd700 1 -- 172.21.15.32:0/685687376 learned_addr learned my addr 172.21.15.32:0/685687376 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/685687376 >> v2:172.21.15.32:3300/0 conn(0x7fb3d009e000 msgr2=0x7fb3d0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3cf7fe700 1 --2- 172.21.15.32:0/685687376 >> v2:172.21.15.32:3300/0 conn(0x7fb3d009e000 0x7fb3d0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/685687376 >> v1:172.21.15.32:6789/0 conn(0x7fb3d009e410 legacy=0x7fb3d0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/685687376 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d009b2d0 con 0x7fb3d009d6f0 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3cf7fe700 1 --2- 172.21.15.32:0/685687376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d009dac0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009840 tx=0x7fb3c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6b3a4375d0b6ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3ce7fc700 1 -- 172.21.15.32:0/685687376 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0011830 con 0x7fb3d009d6f0 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3ce7fc700 1 -- 172.21.15.32:0/685687376 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0011990 con 0x7fb3d009d6f0 2023-09-30T22:31:27.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.707+0000 7fb3ce7fc700 1 -- 172.21.15.32:0/685687376 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0011d20 con 0x7fb3d009d6f0 2023-09-30T22:31:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 -- 172.21.15.32:0/685687376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 msgr2=0x7fb3d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 --2- 172.21.15.32:0/685687376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d009dac0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009840 tx=0x7fb3c0009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 -- 172.21.15.32:0/685687376 shutdown_connections 2023-09-30T22:31:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 --2- 172.21.15.32:0/685687376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d009dac0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 --2- 172.21.15.32:0/685687376 >> v2:172.21.15.32:3300/0 conn(0x7fb3d009e000 0x7fb3d0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 -- 172.21.15.32:0/685687376 >> 172.21.15.32:0/685687376 conn(0x7fb3d0066f50 msgr2=0x7fb3d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 -- 172.21.15.32:0/685687376 shutdown_connections 2023-09-30T22:31:27.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.708+0000 7fb3d5f14700 1 -- 172.21.15.32:0/685687376 wait complete. 2023-09-30T22:31:27.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.709+0000 7fb3d5f14700 1 Processor -- start 2023-09-30T22:31:27.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 -- start start 2023-09-30T22:31:27.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d011e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3d009e000 0x7fb3d011e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d009e410 0x7fb3d011ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d011f610 con 0x7fb3d009e410 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d01303f0 con 0x7fb3d009d6f0 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3d5f14700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0120240 con 0x7fb3d009e000 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3cf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d011e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3ceffd700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3d009e000 0x7fb3d011e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3cf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d011e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56184/0 (socket says 172.21.15.32:56184) 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/191281487 learned_addr learned my addr 172.21.15.32:0/191281487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.710+0000 7fb3cffff700 1 --2- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d009e410 0x7fb3d011ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3d009e000 msgr2=0x7fb3d011e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cf7fe700 1 --2- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3d009e000 0x7fb3d011e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d009e410 msgr2=0x7fb3d011ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cf7fe700 1 --2- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d009e410 0x7fb3d011ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cf7fe700 1 -- 172.21.15.32:0/191281487 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c00094a0 con 0x7fb3d009d6f0 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3ceffd700 1 --2- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb3d009e000 0x7fb3d011e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cffff700 1 --2- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3d009e410 0x7fb3d011ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3cf7fe700 1 --2- 172.21.15.32:0/191281487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb3d009d6f0 0x7fb3d011e140 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009fd0 tx=0x7fb3c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3b7fff700 1 -- 172.21.15.32:0/191281487 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00036a0 con 0x7fb3d009d6f0 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3b7fff700 1 -- 172.21.15.32:0/191281487 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c001e070 con 0x7fb3d009d6f0 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3b7fff700 1 -- 172.21.15.32:0/191281487 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0003a40 con 0x7fb3d009d6f0 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3d5f14700 1 -- 172.21.15.32:0/191281487 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0120e70 con 0x7fb3d009d6f0 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.711+0000 7fb3d5f14700 1 -- 172.21.15.32:0/191281487 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d00a1d00 con 0x7fb3d009d6f0 2023-09-30T22:31:27.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.712+0000 7fb3b7fff700 1 -- 172.21.15.32:0/191281487 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb3c0003ba0 con 0x7fb3d009d6f0 2023-09-30T22:31:27.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.712+0000 7fb3b7fff700 1 -- 172.21.15.32:0/191281487 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb3c0040550 con 0x7fb3d009d6f0 2023-09-30T22:31:27.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.713+0000 7fb3d5f14700 1 -- 172.21.15.32:0/191281487 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d009c700 con 0x7fb3d009d6f0 2023-09-30T22:31:27.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.722+0000 7fb3b7fff700 1 -- 172.21.15.32:0/191281487 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb3c0015030 con 0x7fb3d009d6f0 2023-09-30T22:31:27.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.747+0000 7f9b7146a700 1 Processor -- start 2023-09-30T22:31:27.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.748+0000 7f9b7146a700 1 -- start start 2023-09-30T22:31:27.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.748+0000 7f9b7146a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.748+0000 7f9b7146a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b6c139170 0x7f9b6c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.748+0000 7f9b7146a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c09b120 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.748+0000 7f9b7146a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c06b460 con 0x7f9b6c139170 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.748+0000 7f9b7146a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c09a7e0 con 0x7f9b6c09e1d0 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6b7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b6c139170 0x7f9b6c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6a7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56192/0 (socket says 172.21.15.32:56192) 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6affd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b6c09d6f0 0x7f9b6c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43944/0 (socket says 172.21.15.32:43944) 2023-09-30T22:31:27.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6a7fc700 1 -- 172.21.15.32:0/2559850444 learned_addr learned my addr 172.21.15.32:0/2559850444 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6b7fe700 1 -- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 msgr2=0x7f9b6c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6b7fe700 1 --2- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6b7fe700 1 -- 172.21.15.32:0/2559850444 >> v1:172.21.15.32:6789/0 conn(0x7f9b6c09d6f0 legacy=0x7f9b6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b69ffb700 1 -- 172.21.15.32:0/2559850444 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365704230 0 0) 0x7f9b6c09b120 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6b7fe700 1 -- 172.21.15.32:0/2559850444 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c09a500 con 0x7f9b6c139170 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6a7fc700 1 --2- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b6b7fe700 1 --2- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c139170 0x7f9b6c13b550 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009d90 tx=0x7f9b6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65636b84aec3920b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b69ffb700 1 -- 172.21.15.32:0/2559850444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6400e070 con 0x7f9b6c139170 2023-09-30T22:31:27.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b69ffb700 1 -- 172.21.15.32:0/2559850444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6400bb20 con 0x7f9b6c139170 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.749+0000 7f9b69ffb700 1 -- 172.21.15.32:0/2559850444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64012430 con 0x7f9b6c139170 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 -- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c139170 msgr2=0x7f9b6c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 --2- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c139170 0x7f9b6c13b550 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009d90 tx=0x7f9b6400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 -- 172.21.15.32:0/2559850444 shutdown_connections 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 --2- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c139170 0x7f9b6c13b550 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 --2- 172.21.15.32:0/2559850444 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 -- 172.21.15.32:0/2559850444 >> 172.21.15.32:0/2559850444 conn(0x7f9b6c066f50 msgr2=0x7f9b6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 -- 172.21.15.32:0/2559850444 shutdown_connections 2023-09-30T22:31:27.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.750+0000 7f9b7146a700 1 -- 172.21.15.32:0/2559850444 wait complete. 2023-09-30T22:31:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.752+0000 7f9b7146a700 1 Processor -- start 2023-09-30T22:31:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.752+0000 7f9b7146a700 1 -- start start 2023-09-30T22:31:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b7146a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 0x7f9b6c1216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b7146a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b7146a700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b6c139170 0x7f9b6c122120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b7146a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1234a0 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b7146a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1232b0 con 0x7f9b6c09e1d0 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b7146a700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c137620 con 0x7f9b6c139170 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 0x7f9b6c1216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 0x7f9b6c1216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55544/0 (socket says 172.21.15.32:55544) 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 -- 172.21.15.32:0/521630007 learned_addr learned my addr 172.21.15.32:0/521630007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6a7fc700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6b7fe700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b6c139170 0x7f9b6c122120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 -- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b6c139170 msgr2=0x7f9b6c122120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b6c139170 0x7f9b6c122120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 -- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 msgr2=0x7f9b6c121be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c121be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.753+0000 7f9b6affd700 1 -- 172.21.15.32:0/521630007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009550 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.754+0000 7f9b6affd700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 0x7f9b6c1216a0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c520 tx=0x7f9b5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.754+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c004c80 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.754+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c004de0 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.754+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b64009a70 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.754+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0072f0 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.754+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c124ee0 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.755+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c123b00 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.755+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9b5c007450 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.756+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f9b5c03f9c0 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.764+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9b5c007700 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 Processor -- start 2023-09-30T22:31:27.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 -- start start 2023-09-30T22:31:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009f410 0x7f5940136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5940070940 con 0x7f594009e930 2023-09-30T22:31:27.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594006fd60 con 0x7f594009f410 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f5946532700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594006fa80 con 0x7f5940136fe0 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f593ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f594009e930 0x7f594009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43954/0 (socket says 172.21.15.32:43954) 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f593ffff700 1 -- 172.21.15.32:0/1779915396 learned_addr learned my addr 172.21.15.32:0/1779915396 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.802+0000 7f593f7fe700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009f410 0x7f5940136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f5944acf700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593effd700 1 -- 172.21.15.32:0/1779915396 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575975500 0 0) 0x7f5940070940 con 0x7f594009e930 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593effd700 1 -- 172.21.15.32:0/1779915396 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59280035b0 con 0x7f594009e930 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593f7fe700 1 -- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 msgr2=0x7f59401393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593f7fe700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593f7fe700 1 -- 172.21.15.32:0/1779915396 >> v1:172.21.15.32:6789/0 conn(0x7f594009e930 legacy=0x7f594009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593f7fe700 1 -- 172.21.15.32:0/1779915396 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594009fb60 con 0x7f594009f410 2023-09-30T22:31:27.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593f7fe700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009f410 0x7f5940136aa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5934009870 tx=0x7f5934009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59550f2ed95cd2d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593effd700 1 -- 172.21.15.32:0/1779915396 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011830 con 0x7f594009f410 2023-09-30T22:31:27.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593effd700 1 -- 172.21.15.32:0/1779915396 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934011990 con 0x7f594009f410 2023-09-30T22:31:27.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.803+0000 7f593effd700 1 -- 172.21.15.32:0/1779915396 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934011d20 con 0x7f594009f410 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 -- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009f410 msgr2=0x7f5940136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009f410 0x7f5940136aa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5934009870 tx=0x7f5934009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 -- 172.21.15.32:0/1779915396 shutdown_connections 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 --2- 172.21.15.32:0/1779915396 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009f410 0x7f5940136aa0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 -- 172.21.15.32:0/1779915396 >> 172.21.15.32:0/1779915396 conn(0x7f59400a4210 msgr2=0x7f59400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 -- 172.21.15.32:0/1779915396 shutdown_connections 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.804+0000 7f5946532700 1 -- 172.21.15.32:0/1779915396 wait complete. 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.806+0000 7f5946532700 1 Processor -- start 2023-09-30T22:31:27.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.806+0000 7f5946532700 1 -- start start 2023-09-30T22:31:27.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5946532700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 0x7f594011f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5946532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594009f410 0x7f594011fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5946532700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5946532700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594011ef40 con 0x7f594009f410 2023-09-30T22:31:27.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5946532700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5940071db0 con 0x7f594009e930 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5946532700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f594011f120 con 0x7f5940136fe0 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 0x7f594011f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594009f410 0x7f594011fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 0x7f594011f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56212/0 (socket says 172.21.15.32:56212) 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 -- 172.21.15.32:0/4215127325 learned_addr learned my addr 172.21.15.32:0/4215127325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f5944acf700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 -- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 msgr2=0x7f59401c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 -- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594009f410 msgr2=0x7f594011fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594009f410 0x7f594011fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593ffff700 1 -- 172.21.15.32:0/4215127325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340094a0 con 0x7f594009e930 2023-09-30T22:31:27.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.807+0000 7f593f7fe700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594009f410 0x7f594011fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.808+0000 7f593ffff700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 0x7f594011f570 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f593000c730 tx=0x7f593000ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.808+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930005d70 con 0x7f594009e930 2023-09-30T22:31:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.808+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59300070d0 con 0x7f594009e930 2023-09-30T22:31:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.808+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5940121f40 con 0x7f594009e930 2023-09-30T22:31:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.808+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930007440 con 0x7f594009e930 2023-09-30T22:31:27.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.808+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5940122c70 con 0x7f594009e930 2023-09-30T22:31:27.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.809+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59300075a0 con 0x7f594009e930 2023-09-30T22:31:27.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.809+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f593003ea40 con 0x7f594009e930 2023-09-30T22:31:27.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.809+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594009fdb0 con 0x7f594009e930 2023-09-30T22:31:27.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.814+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5930017070 con 0x7f594009e930 2023-09-30T22:31:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.973+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b6c074ea0 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.974+0000 7f9b537fe700 1 -- 172.21.15.32:0/521630007 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b5c011e80 con 0x7f9b6c09d6f0 2023-09-30T22:31:27.975 INFO:teuthology.orchestra.run.smithi032.stdout:133143986178 2023-09-30T22:31:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 msgr2=0x7f9b6c1216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 0x7f9b6c1216a0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c520 tx=0x7f9b5c00c830 comp rx=0 tx=0).stop 2023-09-30T22:31:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 shutdown_connections 2023-09-30T22:31:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f9b6c139170 0x7f9b6c122120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b6c09d6f0 0x7f9b6c1216a0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 --2- 172.21.15.32:0/521630007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b6c09e1d0 0x7f9b6c121be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 >> 172.21.15.32:0/521630007 conn(0x7f9b6c066f50 msgr2=0x7f9b6c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 shutdown_connections 2023-09-30T22:31:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:27.976+0000 7f9b7146a700 1 -- 172.21.15.32:0/521630007 wait complete. 2023-09-30T22:31:27.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986180 got 133143986178 for osd.2 2023-09-30T22:31:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.021+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5940074ea0 con 0x7f594009e930 2023-09-30T22:31:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.022+0000 7f593cff9700 1 -- 172.21.15.32:0/4215127325 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5930011be0 con 0x7f594009e930 2023-09-30T22:31:28.023 INFO:teuthology.orchestra.run.smithi032.stdout:124554051587 2023-09-30T22:31:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 msgr2=0x7f594011f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 0x7f594011f570 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f593000c730 tx=0x7f593000ca40 comp rx=0 tx=0).stop 2023-09-30T22:31:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 shutdown_connections 2023-09-30T22:31:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f5940136fe0 0x7f59401c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f594009e930 0x7f594011f570 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 --2- 172.21.15.32:0/4215127325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f594009f410 0x7f594011fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 >> 172.21.15.32:0/4215127325 conn(0x7f59400a4210 msgr2=0x7f5940134920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 shutdown_connections 2023-09-30T22:31:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:31:28.024+0000 7f5946532700 1 -- 172.21.15.32:0/4215127325 wait complete. 2023-09-30T22:31:28.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051587 for osd.0 2023-09-30T22:33:27.597 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-30T22:33:27.599 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:33:27.674 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c1bc4302dbac483884ac48e094f5ffae Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:33:27.678 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:33:27.688 DEBUG:teuthology.run_tasks:Unwinding manager ceph.stop 2023-09-30T22:33:27.698 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:33:27.708 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:33:27.717 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2023-09-30T22:33:27.727 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-30T22:33:27.737 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-30T22:33:27.737 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:33:27.737 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:33:27.764 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:33:27.765 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:33:27.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:33:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.971+0000 7f3d53ea6700 1 Processor -- start 2023-09-30T22:33:27.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.971+0000 7f3d53ea6700 1 -- start start 2023-09-30T22:33:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.971+0000 7f3d53ea6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c09e1d0 0x7f3d4c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.971+0000 7f3d53ea6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d4c135e50 0x7f3d4c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.971+0000 7f3d53ea6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c09f130 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.971+0000 7f3d53ea6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4c09f520 con 0x7f3d4c135e50 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d53ea6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c09e930 con 0x7f3d4c09e1d0 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d51c42700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3d4c09d6f0 0x7f3d4c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37888/0 (socket says 172.21.15.32:37888) 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d52443700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d4c135e50 0x7f3d4c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d51441700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c09e1d0 0x7f3d4c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d51c42700 1 -- 172.21.15.32:0/2681735635 learned_addr learned my addr 172.21.15.32:0/2681735635 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d52443700 1 -- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c09e1d0 msgr2=0x7f3d4c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:27.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d52443700 1 --2- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c09e1d0 0x7f3d4c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d52443700 1 -- 172.21.15.32:0/2681735635 >> v1:172.21.15.32:6789/0 conn(0x7f3d4c09d6f0 legacy=0x7f3d4c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d52443700 1 -- 172.21.15.32:0/2681735635 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c09b2d0 con 0x7f3d4c135e50 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d50c40700 1 -- 172.21.15.32:0/2681735635 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646697687 0 0) 0x7f3d4c09f130 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d51441700 1 --2- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c09e1d0 0x7f3d4c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d52443700 1 --2- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c135e50 0x7f3d4c138230 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009dc0 tx=0x7f3d4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc266aa52bb0a497 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d50c40700 1 -- 172.21.15.32:0/2681735635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400e070 con 0x7f3d4c135e50 2023-09-30T22:33:27.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d50c40700 1 -- 172.21.15.32:0/2681735635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4400bae0 con 0x7f3d4c135e50 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.972+0000 7f3d50c40700 1 -- 172.21.15.32:0/2681735635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400be90 con 0x7f3d4c135e50 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 -- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c135e50 msgr2=0x7f3d4c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c135e50 0x7f3d4c138230 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009dc0 tx=0x7f3d4400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 -- 172.21.15.32:0/2681735635 shutdown_connections 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c135e50 0x7f3d4c138230 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/2681735635 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c09e1d0 0x7f3d4c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 -- 172.21.15.32:0/2681735635 >> 172.21.15.32:0/2681735635 conn(0x7f3d4c066f50 msgr2=0x7f3d4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 -- 172.21.15.32:0/2681735635 shutdown_connections 2023-09-30T22:33:27.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.973+0000 7f3d53ea6700 1 -- 172.21.15.32:0/2681735635 wait complete. 2023-09-30T22:33:27.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.974+0000 7f3d53ea6700 1 Processor -- start 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.974+0000 7f3d53ea6700 1 -- start start 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c09d6f0 0x7f3d4c11b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d4c09e1d0 0x7f3d4c11b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 0x7f3d4c11be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c11ecd0 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c11eed0 con 0x7f3d4c09e1d0 2023-09-30T22:33:27.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c072070 con 0x7f3d4c135e50 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d52443700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 0x7f3d4c11be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51441700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d4c09e1d0 0x7f3d4c11b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d52443700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 0x7f3d4c11be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60952/0 (socket says 172.21.15.32:60952) 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d52443700 1 -- 172.21.15.32:0/4278743264 learned_addr learned my addr 172.21.15.32:0/4278743264 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c09d6f0 0x7f3d4c11b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 -- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 msgr2=0x7f3d4c11be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 0x7f3d4c11be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 -- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d4c09e1d0 msgr2=0x7f3d4c11b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d4c09e1d0 0x7f3d4c11b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 -- 172.21.15.32:0/4278743264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c0094a0 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d52443700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 0x7f3d4c11be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51441700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d4c09e1d0 0x7f3d4c11b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d51c42700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c09d6f0 0x7f3d4c11b3f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c470 tx=0x7f3d4800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48004c80 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d48004de0 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48007310 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.975+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d44009a70 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.976+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c132190 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.976+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3d48007470 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.976+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f3d4803f240 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.977+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c09a2d0 con 0x7f3d4c09d6f0 2023-09-30T22:33:27.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:27.982+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3d48007720 con 0x7f3d4c09d6f0 2023-09-30T22:33:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.187+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d4c074ea0 con 0x7f3d4c09d6f0 2023-09-30T22:33:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.188+0000 7f3d427fc700 1 -- 172.21.15.32:0/4278743264 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13396 (secure 0 0 0) 0x7f3d48011360 con 0x7f3d4c09d6f0 2023-09-30T22:33:28.189 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:33:28.189 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:53.053728+0000","last_up_change":"2023-09-30T22:24:14.089689+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":30,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":129367}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":129367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":129367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":129367}]},"public_addr":"172.21.15.32:6804/129367","cluster_addr":"172.21.15.32:6805/129367","heartbeat_back_addr":"172.21.15.32:6807/129367","heartbeat_front_addr":"172.21.15.32:6806/129367","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":129477}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":129477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":129477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":129477}]},"public_addr":"172.21.15.32:6808/129477","cluster_addr":"172.21.15.32:6809/129477","heartbeat_back_addr":"172.21.15.32:6811/129477","heartbeat_front_addr":"172.21.15.32:6810/129477","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":129817}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":129817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":129817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":129817}]},"public_addr":"172.21.15.32:6812/129817","cluster_addr":"172.21.15.32:6813/129817","heartbeat_back_addr":"172.21.15.32:6815/129817","heartbeat_front_addr":"172.21.15.32:6814/129817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/338848985":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/3143218070":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/2265333326":"2023-10-01T22:24:53.053679+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/1488807751":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/2199662186":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6817/3095020018":"2023-10-01T22:24:21.192854+0000","172.21.15.32:6816/126553":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/1998798597":"2023-10-01T22:24:53.053679+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:33:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c09d6f0 msgr2=0x7f3d4c11b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c09d6f0 0x7f3d4c11b3f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c470 tx=0x7f3d4800c780 comp rx=0 tx=0).stop 2023-09-30T22:33:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 shutdown_connections 2023-09-30T22:33:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f3d4c135e50 0x7f3d4c11be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d4c09d6f0 0x7f3d4c11b3f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 --2- 172.21.15.32:0/4278743264 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d4c09e1d0 0x7f3d4c11b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 >> 172.21.15.32:0/4278743264 conn(0x7f3d4c066f50 msgr2=0x7f3d4c09fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 shutdown_connections 2023-09-30T22:33:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.190+0000 7f3d53ea6700 1 -- 172.21.15.32:0/4278743264 wait complete. 2023-09-30T22:33:28.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:33:28.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:33:28.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.344+0000 7effe418b700 1 Processor -- start 2023-09-30T22:33:28.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.344+0000 7effe418b700 1 -- start start 2023-09-30T22:33:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe418b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc138f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe418b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effdc1394a0 0x7effdc13b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe418b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc09f860 con 0x7effdc071cd0 2023-09-30T22:33:28.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe418b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc0a0680 con 0x7effdc1394a0 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe418b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc06cb80 con 0x7effdc136b80 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe2728700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7effdc1394a0 0x7effdc13b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe1726700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe1f27700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7effdc071cd0 0x7effdc0720a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37898/0 (socket says 172.21.15.32:37898) 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe1f27700 1 -- 172.21.15.32:0/1260759307 learned_addr learned my addr 172.21.15.32:0/1260759307 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe2728700 1 -- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 msgr2=0x7effdc138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe2728700 1 --2- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc138f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe2728700 1 -- 172.21.15.32:0/1260759307 >> v1:172.21.15.32:6789/0 conn(0x7effdc071cd0 legacy=0x7effdc0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe0f25700 1 -- 172.21.15.32:0/1260759307 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572601825 0 0) 0x7effdc09f860 con 0x7effdc071cd0 2023-09-30T22:33:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe2728700 1 -- 172.21.15.32:0/1260759307 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc06c0a0 con 0x7effdc1394a0 2023-09-30T22:33:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.345+0000 7effe1726700 1 --2- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe2728700 1 --2- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc13b880 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7effd4009d90 tx=0x7effd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02eb984767381c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe0f25700 1 -- 172.21.15.32:0/1260759307 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400e070 con 0x7effdc1394a0 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe0f25700 1 -- 172.21.15.32:0/1260759307 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400bb20 con 0x7effdc1394a0 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe0f25700 1 -- 172.21.15.32:0/1260759307 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4012430 con 0x7effdc1394a0 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe418b700 1 -- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 msgr2=0x7effdc13b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe418b700 1 --2- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc13b880 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7effd4009d90 tx=0x7effd400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe418b700 1 -- 172.21.15.32:0/1260759307 shutdown_connections 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe418b700 1 --2- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc13b880 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe418b700 1 --2- 172.21.15.32:0/1260759307 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.346+0000 7effe418b700 1 -- 172.21.15.32:0/1260759307 >> 172.21.15.32:0/1260759307 conn(0x7effdc0a4130 msgr2=0x7effdc0a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.347+0000 7effe418b700 1 -- 172.21.15.32:0/1260759307 shutdown_connections 2023-09-30T22:33:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.347+0000 7effe418b700 1 -- 172.21.15.32:0/1260759307 wait complete. 2023-09-30T22:33:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.347+0000 7effe418b700 1 Processor -- start 2023-09-30T22:33:28.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 -- start start 2023-09-30T22:33:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc071cd0 0x7effdc120910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc120e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc121390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc124430 con 0x7effdc1394a0 2023-09-30T22:33:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc06cda0 con 0x7effdc136b80 2023-09-30T22:33:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe418b700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0a17a0 con 0x7effdc071cd0 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe1726700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc120e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe1f27700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc071cd0 0x7effdc120910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe2728700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc121390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe1726700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc120e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54536/0 (socket says 172.21.15.32:54536) 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.348+0000 7effe1726700 1 -- 172.21.15.32:0/90284464 learned_addr learned my addr 172.21.15.32:0/90284464 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1f27700 1 -- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 msgr2=0x7effdc120e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1f27700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc120e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1f27700 1 -- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 msgr2=0x7effdc121390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1f27700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc121390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1f27700 1 -- 172.21.15.32:0/90284464 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80094a0 con 0x7effdc071cd0 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe2728700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc121390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1726700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc120e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe1f27700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc071cd0 0x7effdc120910 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7effd000c3c0 tx=0x7effd000c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0004a70 con 0x7effdc071cd0 2023-09-30T22:33:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0004bf0 con 0x7effdc071cd0 2023-09-30T22:33:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd4009a70 con 0x7effdc071cd0 2023-09-30T22:33:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd00070d0 con 0x7effdc071cd0 2023-09-30T22:33:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.349+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc123e90 con 0x7effdc071cd0 2023-09-30T22:33:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.350+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc04f2a0 con 0x7effdc071cd0 2023-09-30T22:33:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.350+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7effd0007230 con 0x7effdc071cd0 2023-09-30T22:33:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.351+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7effd003f8a0 con 0x7effdc071cd0 2023-09-30T22:33:28.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.355+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7effd00074e0 con 0x7effdc071cd0 2023-09-30T22:33:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.554+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effdc0728b0 con 0x7effdc071cd0 2023-09-30T22:33:28.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.555+0000 7effce7fc700 1 -- 172.21.15.32:0/90284464 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7effd001b340 con 0x7effdc071cd0 2023-09-30T22:33:28.556 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:33:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc071cd0 msgr2=0x7effdc120910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc071cd0 0x7effdc120910 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7effd000c3c0 tx=0x7effd000c6d0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 shutdown_connections 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effdc1394a0 0x7effdc121390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7effdc071cd0 0x7effdc120910 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 --2- 172.21.15.32:0/90284464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7effdc136b80 0x7effdc120e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 >> 172.21.15.32:0/90284464 conn(0x7effdc0a4130 msgr2=0x7effdc0a1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 shutdown_connections 2023-09-30T22:33:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.557+0000 7effe418b700 1 -- 172.21.15.32:0/90284464 wait complete. 2023-09-30T22:33:28.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:33:28.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 Processor -- start 2023-09-30T22:33:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 -- start start 2023-09-30T22:33:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c131ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd52c09bc50 0x7fd52c09c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09c5a0 0x7fd52c138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c09ec00 con 0x7fd52c09bc50 2023-09-30T22:33:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c06ee30 con 0x7fd52c09c5a0 2023-09-30T22:33:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd53124c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c131780 con 0x7fd52c131c20 2023-09-30T22:33:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd52ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c131ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd52a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd52c09bc50 0x7fd52c09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd52b59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09c5a0 0x7fd52c138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd52a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd52c09bc50 0x7fd52c09c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40622/0 (socket says 172.21.15.32:40622) 2023-09-30T22:33:28.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd52a59c700 1 -- 172.21.15.32:0/2462480368 learned_addr learned my addr 172.21.15.32:0/2462480368 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.710+0000 7fd52ad9d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c131ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60970/0 (socket says 172.21.15.32:60970) 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52a59c700 1 -- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 msgr2=0x7fd52c131ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52a59c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c131ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52a59c700 1 -- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09c5a0 msgr2=0x7fd52c138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52a59c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09c5a0 0x7fd52c138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52a59c700 1 -- 172.21.15.32:0/2462480368 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c131a20 con 0x7fd52c09bc50 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52b59e700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09c5a0 0x7fd52c138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd52a59c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09bc50 0x7fd52c09c060 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fd51400dd10 tx=0x7fd51400f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9389ce762d4174 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd529d9b700 1 -- 172.21.15.32:0/2462480368 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514012070 con 0x7fd52c09bc50 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd529d9b700 1 -- 172.21.15.32:0/2462480368 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400d2e0 con 0x7fd52c09bc50 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.711+0000 7fd529d9b700 1 -- 172.21.15.32:0/2462480368 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400fb10 con 0x7fd52c09bc50 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 -- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09bc50 msgr2=0x7fd52c09c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09bc50 0x7fd52c09c060 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fd51400dd10 tx=0x7fd51400f330 comp rx=0 tx=0).stop 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 -- 172.21.15.32:0/2462480368 shutdown_connections 2023-09-30T22:33:28.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09c5a0 0x7fd52c138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c131ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 --2- 172.21.15.32:0/2462480368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09bc50 0x7fd52c09c060 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 -- 172.21.15.32:0/2462480368 >> 172.21.15.32:0/2462480368 conn(0x7fd52c0a2c10 msgr2=0x7fd52c0a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 -- 172.21.15.32:0/2462480368 shutdown_connections 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.712+0000 7fd53124c700 1 -- 172.21.15.32:0/2462480368 wait complete. 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.713+0000 7fd53124c700 1 Processor -- start 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.713+0000 7fd53124c700 1 -- start start 2023-09-30T22:33:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09bc50 0x7fd52c118e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 0x7fd52c119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c1198c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c11a1f0 con 0x7fd52c09c5a0 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c11b4b0 con 0x7fd52c09bc50 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c12bf70 con 0x7fd52c131c20 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 0x7fd52c119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 0x7fd52c119380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40632/0 (socket says 172.21.15.32:40632) 2023-09-30T22:33:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 -- 172.21.15.32:0/1824401766 learned_addr learned my addr 172.21.15.32:0/1824401766 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52b59e700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c1198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52ad9d700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09bc50 0x7fd52c118e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 -- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 msgr2=0x7fd52c1198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c1198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 -- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09bc50 msgr2=0x7fd52c118e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09bc50 0x7fd52c118e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 -- 172.21.15.32:0/1824401766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd524009a70 con 0x7fd52c09c5a0 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52b59e700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c1198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd52a59c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 0x7fd52c119380 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fd51400f370 tx=0x7fd514004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400d870 con 0x7fd52c09c5a0 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51400de60 con 0x7fd52c09c5a0 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.714+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51400d9f0 con 0x7fd52c09c5a0 2023-09-30T22:33:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.715+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140038e0 con 0x7fd52c09c5a0 2023-09-30T22:33:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.715+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c11c190 con 0x7fd52c09c5a0 2023-09-30T22:33:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.715+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd514003a40 con 0x7fd52c09c5a0 2023-09-30T22:33:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.715+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fd514044760 con 0x7fd52c09c5a0 2023-09-30T22:33:28.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.716+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c09b1e0 con 0x7fd52c09c5a0 2023-09-30T22:33:28.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.721+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd51400b7d0 con 0x7fd52c09c5a0 2023-09-30T22:33:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.923+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd52c04f2a0 con 0x7fd52c09c5a0 2023-09-30T22:33:28.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.923+0000 7fd5237fe700 1 -- 172.21.15.32:0/1824401766 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd51401fb90 con 0x7fd52c09c5a0 2023-09-30T22:33:28.925 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:33:28.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 msgr2=0x7fd52c119380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 0x7fd52c119380 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fd51400f370 tx=0x7fd514004750 comp rx=0 tx=0).stop 2023-09-30T22:33:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 shutdown_connections 2023-09-30T22:33:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fd52c131c20 0x7fd52c1198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd52c09bc50 0x7fd52c118e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 --2- 172.21.15.32:0/1824401766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd52c09c5a0 0x7fd52c119380 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 >> 172.21.15.32:0/1824401766 conn(0x7fd52c0a2c10 msgr2=0x7fd52c0a1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 shutdown_connections 2023-09-30T22:33:28.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:28.926+0000 7fd53124c700 1 -- 172.21.15.32:0/1824401766 wait complete. 2023-09-30T22:33:28.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:33:29.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.079+0000 7f7a15a1f700 1 Processor -- start 2023-09-30T22:33:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.079+0000 7f7a15a1f700 1 -- start start 2023-09-30T22:33:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a15a1f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10131760 0x7f7a10131b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a15a1f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006a770 0x7f7a1006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a15a1f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a10131350 con 0x7f7a1006b100 2023-09-30T22:33:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a15a1f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1009ea40 con 0x7f7a10131760 2023-09-30T22:33:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a15a1f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1009dfc0 con 0x7f7a1006a770 2023-09-30T22:33:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10131760 0x7f7a10131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7a1006b100 0x7f7a101385e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37900/0 (socket says 172.21.15.32:37900) 2023-09-30T22:33:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0f7fe700 1 -- 172.21.15.32:0/2195640234 learned_addr learned my addr 172.21.15.32:0/2195640234 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0e7fc700 1 --2- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006a770 0x7f7a1006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0effd700 1 -- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006a770 msgr2=0x7f7a1006abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0effd700 1 --2- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006a770 0x7f7a1006abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0dffb700 1 -- 172.21.15.32:0/2195640234 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684742500 0 0) 0x7f7a10131350 con 0x7f7a1006b100 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0effd700 1 -- 172.21.15.32:0/2195640234 >> v1:172.21.15.32:6789/0 conn(0x7f7a1006b100 legacy=0x7f7a101385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.080+0000 7f7a0effd700 1 -- 172.21.15.32:0/2195640234 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1009e210 con 0x7f7a10131760 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a0effd700 1 --2- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10131760 0x7f7a10131b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009870 tx=0x7f7a00009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c1f587e8b59a4bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a0dffb700 1 -- 172.21.15.32:0/2195640234 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00011830 con 0x7f7a10131760 2023-09-30T22:33:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a0dffb700 1 -- 172.21.15.32:0/2195640234 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00011990 con 0x7f7a10131760 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a0dffb700 1 -- 172.21.15.32:0/2195640234 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00011d20 con 0x7f7a10131760 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 -- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10131760 msgr2=0x7f7a10131b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10131760 0x7f7a10131b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009870 tx=0x7f7a00009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 -- 172.21.15.32:0/2195640234 shutdown_connections 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a10131760 0x7f7a10131b30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/2195640234 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006a770 0x7f7a1006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 -- 172.21.15.32:0/2195640234 >> 172.21.15.32:0/2195640234 conn(0x7f7a10066d70 msgr2=0x7f7a100a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 -- 172.21.15.32:0/2195640234 shutdown_connections 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.081+0000 7f7a15a1f700 1 -- 172.21.15.32:0/2195640234 wait complete. 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.082+0000 7f7a15a1f700 1 Processor -- start 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 -- start start 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a1006a770 0x7f7a1009a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 0x7f7a1009a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10131760 0x7f7a1009ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1011f950 con 0x7f7a10131760 2023-09-30T22:33:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1011fef0 con 0x7f7a1006a770 2023-09-30T22:33:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a15a1f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1011e820 con 0x7f7a1006b100 2023-09-30T22:33:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 0x7f7a1009a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10131760 0x7f7a1009ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0e7fc700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 0x7f7a1009a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:60998/0 (socket says 172.21.15.32:60998) 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0e7fc700 1 -- 172.21.15.32:0/3593801142 learned_addr learned my addr 172.21.15.32:0/3593801142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0effd700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a1006a770 0x7f7a1009a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0e7fc700 1 -- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a1006a770 msgr2=0x7f7a1009a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.083+0000 7f7a0e7fc700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a1006a770 0x7f7a1009a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a0e7fc700 1 -- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10131760 msgr2=0x7f7a1009ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a0e7fc700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10131760 0x7f7a1009ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a0e7fc700 1 -- 172.21.15.32:0/3593801142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000094a0 con 0x7f7a1006b100 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a0effd700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a1006a770 0x7f7a1009a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a0f7fe700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10131760 0x7f7a1009ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a0e7fc700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 0x7f7a1009a940 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f79f800e840 tx=0x7f79f800eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8009590 con 0x7f7a1006b100 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8013680 con 0x7f7a1006b100 2023-09-30T22:33:29.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1012d270 con 0x7f7a1006b100 2023-09-30T22:33:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8013a10 con 0x7f7a1006b100 2023-09-30T22:33:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.084+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1011fc10 con 0x7f7a1006b100 2023-09-30T22:33:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.085+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1004f2a0 con 0x7f7a1006b100 2023-09-30T22:33:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.085+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79f8013b70 con 0x7f7a1006b100 2023-09-30T22:33:29.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.085+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f79f803f630 con 0x7f7a1006b100 2023-09-30T22:33:29.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.090+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79f801be10 con 0x7f7a1006b100 2023-09-30T22:33:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.292+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a1006c380 con 0x7f7a1006b100 2023-09-30T22:33:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.292+0000 7f7a07fff700 1 -- 172.21.15.32:0/3593801142 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79f801b3b0 con 0x7f7a1006b100 2023-09-30T22:33:29.294 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:33:29.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 msgr2=0x7f7a1009a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 0x7f7a1009a940 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f79f800e840 tx=0x7f79f800eb50 comp rx=0 tx=0).stop 2023-09-30T22:33:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 shutdown_connections 2023-09-30T22:33:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a10131760 0x7f7a1009ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7a1006a770 0x7f7a1009a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 --2- 172.21.15.32:0/3593801142 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f7a1006b100 0x7f7a1009a940 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.294+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 >> 172.21.15.32:0/3593801142 conn(0x7f7a10066d70 msgr2=0x7f7a10130690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.295+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 shutdown_connections 2023-09-30T22:33:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.295+0000 7f7a15a1f700 1 -- 172.21.15.32:0/3593801142 wait complete. 2023-09-30T22:33:29.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:33:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.448+0000 7efe6a68f700 1 Processor -- start 2023-09-30T22:33:29.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.448+0000 7efe6a68f700 1 -- start start 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe6a68f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe64131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe6a68f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409e850 0x7efe6409ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe6a68f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe640a01c0 con 0x7efe6409f1c0 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe6a68f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6409b560 con 0x7efe6409e850 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe6a68f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe6409a260 con 0x7efe64131b30 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe63fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe64131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe68c2c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe6409f1c0 0x7efe641389c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37908/0 (socket says 172.21.15.32:37908) 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe68c2c700 1 -- 172.21.15.32:0/3359300832 learned_addr learned my addr 172.21.15.32:0/3359300832 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe637fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409e850 0x7efe6409ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe63fff700 1 -- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409e850 msgr2=0x7efe6409ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe63fff700 1 --2- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409e850 0x7efe6409ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe63fff700 1 -- 172.21.15.32:0/3359300832 >> v1:172.21.15.32:6789/0 conn(0x7efe6409f1c0 legacy=0x7efe641389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.449+0000 7efe62ffd700 1 -- 172.21.15.32:0/3359300832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034299467 0 0) 0x7efe640a01c0 con 0x7efe6409f1c0 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe63fff700 1 -- 172.21.15.32:0/3359300832 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe640a07a0 con 0x7efe64131b30 2023-09-30T22:33:29.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe63fff700 1 --2- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe64131f00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efe54009870 tx=0x7efe54009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e134f09d31071f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe62ffd700 1 -- 172.21.15.32:0/3359300832 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54011830 con 0x7efe64131b30 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe62ffd700 1 -- 172.21.15.32:0/3359300832 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe54011990 con 0x7efe64131b30 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe62ffd700 1 -- 172.21.15.32:0/3359300832 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54011d20 con 0x7efe64131b30 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe6a68f700 1 -- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 msgr2=0x7efe64131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.450+0000 7efe6a68f700 1 --2- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe64131f00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efe54009870 tx=0x7efe54009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.451+0000 7efe6a68f700 1 -- 172.21.15.32:0/3359300832 shutdown_connections 2023-09-30T22:33:29.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.451+0000 7efe6a68f700 1 --2- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe64131f00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.451+0000 7efe6a68f700 1 --2- 172.21.15.32:0/3359300832 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409e850 0x7efe6409ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.451+0000 7efe6a68f700 1 -- 172.21.15.32:0/3359300832 >> 172.21.15.32:0/3359300832 conn(0x7efe640a4130 msgr2=0x7efe640a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:29.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.451+0000 7efe6a68f700 1 -- 172.21.15.32:0/3359300832 shutdown_connections 2023-09-30T22:33:29.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.451+0000 7efe6a68f700 1 -- 172.21.15.32:0/3359300832 wait complete. 2023-09-30T22:33:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 Processor -- start 2023-09-30T22:33:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 -- start start 2023-09-30T22:33:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 0x7efe6411b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409f1c0 0x7efe6411b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe6411baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6411d280 con 0x7efe6409e850 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6411eb40 con 0x7efe6409f1c0 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe6a68f700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe6411df10 con 0x7efe64131b30 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 0x7efe6411b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 0x7efe6411b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40654/0 (socket says 172.21.15.32:40654) 2023-09-30T22:33:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe68c2c700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe6411baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe637fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409f1c0 0x7efe6411b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.452+0000 7efe63fff700 1 -- 172.21.15.32:0/909927544 learned_addr learned my addr 172.21.15.32:0/909927544 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe637fe700 1 -- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 msgr2=0x7efe6411baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe637fe700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe6411baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe637fe700 1 -- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 msgr2=0x7efe6411b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe637fe700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 0x7efe6411b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe637fe700 1 -- 172.21.15.32:0/909927544 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540094a0 con 0x7efe6409f1c0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe63fff700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 0x7efe6411b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe637fe700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409f1c0 0x7efe6411b5b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e8d0 tx=0x7efe4c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c013680 con 0x7efe6409f1c0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c0137e0 con 0x7efe6409f1c0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe640a11a0 con 0x7efe6409f1c0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c013b50 con 0x7efe6409f1c0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.453+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6409da60 con 0x7efe6409f1c0 2023-09-30T22:33:29.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.454+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe4c013cb0 con 0x7efe6409f1c0 2023-09-30T22:33:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.454+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7efe4c03f630 con 0x7efe6409f1c0 2023-09-30T22:33:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.454+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6404f2a0 con 0x7efe6409f1c0 2023-09-30T22:33:29.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.459+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe4c01be10 con 0x7efe6409f1c0 2023-09-30T22:33:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.659+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe640728b0 con 0x7efe6409f1c0 2023-09-30T22:33:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.660+0000 7efe60ff9700 1 -- 172.21.15.32:0/909927544 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe4c01b3b0 con 0x7efe6409f1c0 2023-09-30T22:33:29.661 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:33:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409f1c0 msgr2=0x7efe6411b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409f1c0 0x7efe6411b5b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e8d0 tx=0x7efe4c00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 shutdown_connections 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7efe64131b30 0x7efe6411baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6409e850 0x7efe6411b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 --2- 172.21.15.32:0/909927544 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6409f1c0 0x7efe6411b5b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 >> 172.21.15.32:0/909927544 conn(0x7efe640a4130 msgr2=0x7efe64131050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:29.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 shutdown_connections 2023-09-30T22:33:29.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.662+0000 7efe6a68f700 1 -- 172.21.15.32:0/909927544 wait complete. 2023-09-30T22:33:29.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T22:33:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.813+0000 7fa602359700 1 Processor -- start 2023-09-30T22:33:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.813+0000 7fa602359700 1 -- start start 2023-09-30T22:33:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa602359700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5fc133510 0x7fa5fc1358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa602359700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc135e30 0x7fa5fc138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa602359700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc130260 con 0x7fa5fc130670 2023-09-30T22:33:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa602359700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5fc09a350 con 0x7fa5fc133510 2023-09-30T22:33:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa602359700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc071ff0 con 0x7fa5fc135e30 2023-09-30T22:33:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fb7fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5fc133510 0x7fa5fc1358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fbfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5fc130670 0x7fa5fc130a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37924/0 (socket says 172.21.15.32:37924) 2023-09-30T22:33:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fbfff700 1 -- 172.21.15.32:0/2973592007 learned_addr learned my addr 172.21.15.32:0/2973592007 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:29.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa6008f6700 1 --2- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc135e30 0x7fa5fc138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fb7fe700 1 -- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc135e30 msgr2=0x7fa5fc138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fb7fe700 1 --2- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc135e30 0x7fa5fc138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fb7fe700 1 -- 172.21.15.32:0/2973592007 >> v1:172.21.15.32:6789/0 conn(0x7fa5fc130670 legacy=0x7fa5fc130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5faffd700 1 -- 172.21.15.32:0/2973592007 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162752028 0 0) 0x7fa5fc130260 con 0x7fa5fc130670 2023-09-30T22:33:29.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.814+0000 7fa5fb7fe700 1 -- 172.21.15.32:0/2973592007 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc09ecf0 con 0x7fa5fc133510 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa5fb7fe700 1 --2- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc133510 0x7fa5fc1358f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009840 tx=0x7fa5f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e560816db0242f92 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa5faffd700 1 -- 172.21.15.32:0/2973592007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0011830 con 0x7fa5fc133510 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa5faffd700 1 -- 172.21.15.32:0/2973592007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0011990 con 0x7fa5fc133510 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa5faffd700 1 -- 172.21.15.32:0/2973592007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0011d20 con 0x7fa5fc133510 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 -- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc133510 msgr2=0x7fa5fc1358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 --2- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc133510 0x7fa5fc1358f0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009840 tx=0x7fa5f0009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 -- 172.21.15.32:0/2973592007 shutdown_connections 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 --2- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc135e30 0x7fa5fc138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 --2- 172.21.15.32:0/2973592007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc133510 0x7fa5fc1358f0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 -- 172.21.15.32:0/2973592007 >> 172.21.15.32:0/2973592007 conn(0x7fa5fc0a2aa0 msgr2=0x7fa5fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 -- 172.21.15.32:0/2973592007 shutdown_connections 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.815+0000 7fa602359700 1 -- 172.21.15.32:0/2973592007 wait complete. 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.817+0000 7fa602359700 1 Processor -- start 2023-09-30T22:33:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.817+0000 7fa602359700 1 -- start start 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa602359700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 0x7fa5fc09a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa602359700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 0x7fa5fc09aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa602359700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc135e30 0x7fa5fc09afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa602359700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc09cb20 con 0x7fa5fc135e30 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa602359700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc09bd20 con 0x7fa5fc133510 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa602359700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc09d070 con 0x7fa5fc130670 2023-09-30T22:33:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 0x7fa5fc09a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 0x7fa5fc09aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fb7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 0x7fa5fc09aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54588/0 (socket says 172.21.15.32:54588) 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fbfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 0x7fa5fc09a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:32808/0 (socket says 172.21.15.32:32808) 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fb7fe700 1 -- 172.21.15.32:0/556186538 learned_addr learned my addr 172.21.15.32:0/556186538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc135e30 0x7fa5fc09afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 -- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 msgr2=0x7fa5fc09a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 0x7fa5fc09a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 -- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 msgr2=0x7fa5fc09aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 0x7fa5fc09aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 -- 172.21.15.32:0/556186538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00094a0 con 0x7fa5fc135e30 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fb7fe700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 0x7fa5fc09aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa5fbfff700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 0x7fa5fc09a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.818+0000 7fa6008f6700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc135e30 0x7fa5fc09afd0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400efc0 tx=0x7fa5f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.819+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f400cc90 con 0x7fa5fc135e30 2023-09-30T22:33:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.819+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f400f040 con 0x7fa5fc135e30 2023-09-30T22:33:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.819+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc09c5b0 con 0x7fa5fc135e30 2023-09-30T22:33:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.819+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4009e60 con 0x7fa5fc135e30 2023-09-30T22:33:29.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.819+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc071820 con 0x7fa5fc135e30 2023-09-30T22:33:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.820+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5f4014480 con 0x7fa5fc135e30 2023-09-30T22:33:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.820+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fa5f403fd00 con 0x7fa5fc135e30 2023-09-30T22:33:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.820+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc09c5b0 con 0x7fa5fc135e30 2023-09-30T22:33:29.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:29.825+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5fc09c5b0 con 0x7fa5fc135e30 2023-09-30T22:33:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.042+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5fc09d5f0 con 0x7fa5fc135e30 2023-09-30T22:33:30.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.042+0000 7fa5f8ff9700 1 -- 172.21.15.32:0/556186538 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+637 (secure 0 0 0) 0x7fa5fc09d5f0 con 0x7fa5fc135e30 2023-09-30T22:33:30.043 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/338848985 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.044 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3143218070 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.044 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2265333326 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.044 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6816/118188 2023-10-01T22:23:54.019278+0000 2023-09-30T22:33:30.044 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/708160155 2023-10-01T22:23:54.019278+0000 2023-09-30T22:33:30.044 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1488807751 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.045 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2199662186 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.045 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2862279367 2023-10-01T22:23:54.019278+0000 2023-09-30T22:33:30.045 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6817/3095020018 2023-10-01T22:24:21.192854+0000 2023-09-30T22:33:30.045 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6816/126553 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.045 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1998798597 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc135e30 msgr2=0x7fa5fc09afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc135e30 0x7fa5fc09afd0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400efc0 tx=0x7fa5f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 shutdown_connections 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5fc135e30 0x7fa5fc09afd0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fa5fc130670 0x7fa5fc09a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 --2- 172.21.15.32:0/556186538 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa5fc133510 0x7fa5fc09aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 >> 172.21.15.32:0/556186538 conn(0x7fa5fc0a2aa0 msgr2=0x7fa5fc137ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 shutdown_connections 2023-09-30T22:33:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.044+0000 7fa602359700 1 -- 172.21.15.32:0/556186538 wait complete. 2023-09-30T22:33:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:listed 11 entries 2023-09-30T22:33:30.059 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-09-30T22:33:30.059 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:33:30.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:33:30.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 Processor -- start 2023-09-30T22:33:30.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 -- start start 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89200a2a70 con 0x7f8920133930 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89200a01d0 con 0x7f892009d630 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.221+0000 7f892585d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892009e2b0 con 0x7f89200a0a30 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891effd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891f7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8920133930 0x7f8920133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37930/0 (socket says 172.21.15.32:37930) 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891f7fe700 1 -- 172.21.15.32:0/1081149139 learned_addr learned my addr 172.21.15.32:0/1081149139 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891effd700 1 -- 172.21.15.32:0/1081149139 >> v2:172.21.15.32:3300/0 conn(0x7f892009d630 msgr2=0x7f89200a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891effd700 1 --2- 172.21.15.32:0/1081149139 >> v2:172.21.15.32:3300/0 conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891effd700 1 -- 172.21.15.32:0/1081149139 >> v1:172.21.15.32:6789/0 conn(0x7f8920133930 legacy=0x7f8920133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891effd700 1 -- 172.21.15.32:0/1081149139 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892009dc50 con 0x7f89200a0a30 2023-09-30T22:33:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891effd700 1 --2- 172.21.15.32:0/1081149139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8910009840 tx=0x7f8910009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb436b02e17425d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891dffb700 1 -- 172.21.15.32:0/1081149139 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910011830 con 0x7f89200a0a30 2023-09-30T22:33:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891dffb700 1 -- 172.21.15.32:0/1081149139 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8910011990 con 0x7f89200a0a30 2023-09-30T22:33:30.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.222+0000 7f891dffb700 1 -- 172.21.15.32:0/1081149139 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910011d20 con 0x7f89200a0a30 2023-09-30T22:33:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 -- 172.21.15.32:0/1081149139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 msgr2=0x7f89200a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 --2- 172.21.15.32:0/1081149139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8910009840 tx=0x7f8910009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 -- 172.21.15.32:0/1081149139 shutdown_connections 2023-09-30T22:33:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 --2- 172.21.15.32:0/1081149139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89200a0e00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 --2- 172.21.15.32:0/1081149139 >> v2:172.21.15.32:3300/0 conn(0x7f892009d630 0x7f89200a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 -- 172.21.15.32:0/1081149139 >> 172.21.15.32:0/1081149139 conn(0x7f89200a3050 msgr2=0x7f89200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 -- 172.21.15.32:0/1081149139 shutdown_connections 2023-09-30T22:33:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.223+0000 7f892585d700 1 -- 172.21.15.32:0/1081149139 wait complete. 2023-09-30T22:33:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.224+0000 7f892585d700 1 Processor -- start 2023-09-30T22:33:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 -- start start 2023-09-30T22:33:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f89201c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89201c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8920120670 con 0x7f892009d630 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8920121e60 con 0x7f89200a0a30 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f892585d700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89200a0410 con 0x7f8920133930 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f891effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f89201c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f891e7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89201c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f891effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f89201c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40668/0 (socket says 172.21.15.32:40668) 2023-09-30T22:33:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f891effd700 1 -- 172.21.15.32:0/2011125427 learned_addr learned my addr 172.21.15.32:0/2011125427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.225+0000 7f891e7fc700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89201c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:54614/0 (socket says 172.21.15.32:54614) 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891f7fe700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891effd700 1 -- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 msgr2=0x7f89201c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891effd700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891effd700 1 -- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 msgr2=0x7f89201c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891effd700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89201c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891effd700 1 -- 172.21.15.32:0/2011125427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100094a0 con 0x7f892009d630 2023-09-30T22:33:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891e7fc700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89201c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f891effd700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f89201c8170 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f8910009fd0 tx=0x7f8910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910004400 con 0x7f892009d630 2023-09-30T22:33:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891001f070 con 0x7f892009d630 2023-09-30T22:33:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8920133390 con 0x7f892009d630 2023-09-30T22:33:30.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100038e0 con 0x7f892009d630 2023-09-30T22:33:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.226+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8920122b90 con 0x7f892009d630 2023-09-30T22:33:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.227+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89200747f0 con 0x7f892009d630 2023-09-30T22:33:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.228+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8910021030 con 0x7f892009d630 2023-09-30T22:33:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.228+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f8910040d60 con 0x7f892009d630 2023-09-30T22:33:30.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.232+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89100239c0 con 0x7f892009d630 2023-09-30T22:33:30.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.438+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8920072980 con 0x7f892009d630 2023-09-30T22:33:30.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.439+0000 7f8917fff700 1 -- 172.21.15.32:0/2011125427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13396 (secure 0 0 0) 0x7f8910015070 con 0x7f892009d630 2023-09-30T22:33:30.440 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-09-30T22:33:30.441 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"9e20f926-ec7d-45ec-8e27-e5e820815322","created":"2023-09-30T22:21:19.836852+0000","modified":"2023-09-30T22:24:53.053728+0000","last_up_change":"2023-09-30T22:24:14.089689+0000","last_in_change":"2023-09-30T22:21:22.213069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:21:28.838521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:21:30.226037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:21:34.900746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:21:36.177318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.059999942779541,"score_stable":1.059999942779541,"optimal_score":1,"raw_score_acting":1.059999942779541,"raw_score_stable":1.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"762e2f2b-1ff4-4d81-840e-a100cc5ed4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6800","nonce":129269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6801","nonce":129269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6803","nonce":129269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6802","nonce":129269}]},"public_addr":"172.21.15.32:6800/129269","cluster_addr":"172.21.15.32:6801/129269","heartbeat_back_addr":"172.21.15.32:6803/129269","heartbeat_front_addr":"172.21.15.32:6802/129269","state":["exists","up"]},{"osd":1,"uuid":"3ff901b4-5d78-49f6-b425-d90361753515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":23,"up_from":30,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6804","nonce":129367}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6805","nonce":129367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6807","nonce":129367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6806","nonce":129367}]},"public_addr":"172.21.15.32:6804/129367","cluster_addr":"172.21.15.32:6805/129367","heartbeat_back_addr":"172.21.15.32:6807/129367","heartbeat_front_addr":"172.21.15.32:6806/129367","state":["exists","up"]},{"osd":2,"uuid":"bfe588b1-e7fe-46ec-a3f2-fef1646f54c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":31,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6808","nonce":129477}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6809","nonce":129477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6811","nonce":129477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6810","nonce":129477}]},"public_addr":"172.21.15.32:6808/129477","cluster_addr":"172.21.15.32:6809/129477","heartbeat_back_addr":"172.21.15.32:6811/129477","heartbeat_front_addr":"172.21.15.32:6810/129477","state":["exists","up"]},{"osd":3,"uuid":"5b3ee2e9-f923-4976-ad87-11ba143ec874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":32,"up_thru":32,"down_at":27,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6812","nonce":129817}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6813","nonce":129817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6815","nonce":129817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.32:6814","nonce":129817}]},"public_addr":"172.21.15.32:6812/129817","cluster_addr":"172.21.15.32:6813/129817","heartbeat_back_addr":"172.21.15.32:6815/129817","heartbeat_front_addr":"172.21.15.32:6814/129817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.629625+0000","dead_epoch":24},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.321860+0000","dead_epoch":25},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.365072+0000","dead_epoch":26},{"osd":3,"down_stamp":"2023-09-30T22:24:08.073447+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:21:27.445985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":3,"to":0}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":0}]},{"pgid":"4.31","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/338848985":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/3143218070":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/2265333326":"2023-10-01T22:24:53.053679+0000","172.21.15.32:6816/118188":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/708160155":"2023-10-01T22:23:54.019278+0000","172.21.15.32:0/1488807751":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/2199662186":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/2862279367":"2023-10-01T22:23:54.019278+0000","172.21.15.32:6817/3095020018":"2023-10-01T22:24:21.192854+0000","172.21.15.32:6816/126553":"2023-10-01T22:24:53.053679+0000","172.21.15.32:0/1998798597":"2023-10-01T22:24:53.053679+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:33:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 msgr2=0x7f89201c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f89201c8170 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f8910009fd0 tx=0x7f8910004750 comp rx=0 tx=0).stop 2023-09-30T22:33:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 shutdown_connections 2023-09-30T22:33:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f8920133930 0x7f89201c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892009d630 0x7f89201c8170 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 --2- 172.21.15.32:0/2011125427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f89200a0a30 0x7f89201c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 >> 172.21.15.32:0/2011125427 conn(0x7f89200a3050 msgr2=0x7f89200a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 shutdown_connections 2023-09-30T22:33:30.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.442+0000 7f892585d700 1 -- 172.21.15.32:0/2011125427 wait complete. 2023-09-30T22:33:30.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:21:28.838521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:21:30.226037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:21:34.900746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:21:36.177318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.059999942779541, 'score_stable': 1.059999942779541, 'optimal_score': 1, 'raw_score_acting': 1.059999942779541, 'raw_score_stable': 1.059999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:33:30.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:33:30.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.597+0000 7fe22e836700 1 Processor -- start 2023-09-30T22:33:30.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.597+0000 7fe22e836700 1 -- start start 2023-09-30T22:33:30.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.597+0000 7fe22e836700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228066410 0x7fe228130b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.597+0000 7fe22e836700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 0x7fe228099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.597+0000 7fe22e836700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22809a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.597+0000 7fe22e836700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280667e0 con 0x7fe228131070 2023-09-30T22:33:30.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22e836700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22809f060 con 0x7fe228099f80 2023-09-30T22:33:30.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22e836700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe22809b7e0 con 0x7fe228066410 2023-09-30T22:33:30.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22809a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe2277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 0x7fe228099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe227fff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228066410 0x7fe228130b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22809a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56072/0 (socket says 172.21.15.32:56072) 2023-09-30T22:33:30.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe2277fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 0x7fe228099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56550/0 (socket says 172.21.15.32:56550) 2023-09-30T22:33:30.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 -- 172.21.15.32:0/3746106970 learned_addr learned my addr 172.21.15.32:0/3746106970 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:30.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 -- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228066410 msgr2=0x7fe228130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 --2- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228066410 0x7fe228130b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 -- 172.21.15.32:0/3746106970 >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 msgr2=0x7fe228099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 --2- 172.21.15.32:0/3746106970 >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 0x7fe228099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 -- 172.21.15.32:0/3746106970 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2281314c0 con 0x7fe228099f80 2023-09-30T22:33:30.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe227fff700 1 --2- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228066410 0x7fe228130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe2277fe700 1 --2- 172.21.15.32:0/3746106970 >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 0x7fe228099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe22cdd3700 1 --2- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22809a420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe22000d150 tx=0x7fe22000d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39c1e268e91d42d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe226ffd700 1 -- 172.21.15.32:0/3746106970 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220010070 con 0x7fe228099f80 2023-09-30T22:33:30.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.598+0000 7fe226ffd700 1 -- 172.21.15.32:0/3746106970 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000dd10 con 0x7fe228099f80 2023-09-30T22:33:30.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe226ffd700 1 -- 172.21.15.32:0/3746106970 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000b310 con 0x7fe228099f80 2023-09-30T22:33:30.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 -- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 msgr2=0x7fe22809a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 --2- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22809a420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe22000d150 tx=0x7fe22000d460 comp rx=0 tx=0).stop 2023-09-30T22:33:30.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 -- 172.21.15.32:0/3746106970 shutdown_connections 2023-09-30T22:33:30.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 --2- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22809a420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 --2- 172.21.15.32:0/3746106970 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228066410 0x7fe228130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 --2- 172.21.15.32:0/3746106970 >> v2:172.21.15.32:3300/0 conn(0x7fe228131070 0x7fe228099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 -- 172.21.15.32:0/3746106970 >> 172.21.15.32:0/3746106970 conn(0x7fe2280a2c10 msgr2=0x7fe2280a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 -- 172.21.15.32:0/3746106970 shutdown_connections 2023-09-30T22:33:30.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.599+0000 7fe22e836700 1 -- 172.21.15.32:0/3746106970 wait complete. 2023-09-30T22:33:30.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 Processor -- start 2023-09-30T22:33:30.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 -- start start 2023-09-30T22:33:30.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe228066410 0x7fe22811ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22811bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228131070 0x7fe22811c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22811e510 con 0x7fe228066410 2023-09-30T22:33:30.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22809b970 con 0x7fe228099f80 2023-09-30T22:33:30.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.601+0000 7fe22e836700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe22811f230 con 0x7fe228131070 2023-09-30T22:33:30.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22811bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22811bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56084/0 (socket says 172.21.15.32:56084) 2023-09-30T22:33:30.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe227fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe228066410 0x7fe22811ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 -- 172.21.15.32:0/3671831633 learned_addr learned my addr 172.21.15.32:0/3671831633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:30.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe22cdd3700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228131070 0x7fe22811c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 -- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228131070 msgr2=0x7fe22811c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228131070 0x7fe22811c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 -- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe228066410 msgr2=0x7fe22811ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe228066410 0x7fe22811ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 -- 172.21.15.32:0/3671831633 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100094a0 con 0x7fe228099f80 2023-09-30T22:33:30.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe227fff700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe228066410 0x7fe22811ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe2277fe700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22811bf80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe21800ec10 tx=0x7fe21800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800c850 con 0x7fe228099f80 2023-09-30T22:33:30.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21800c9b0 con 0x7fe228099f80 2023-09-30T22:33:30.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2180094a0 con 0x7fe228099f80 2023-09-30T22:33:30.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800cd20 con 0x7fe228099f80 2023-09-30T22:33:30.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.602+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22000bd40 con 0x7fe228099f80 2023-09-30T22:33:30.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.603+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe218013470 con 0x7fe228099f80 2023-09-30T22:33:30.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.603+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fe21803fac0 con 0x7fe228099f80 2023-09-30T22:33:30.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.604+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22804f2a0 con 0x7fe228099f80 2023-09-30T22:33:30.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.608+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe218013720 con 0x7fe228099f80 2023-09-30T22:33:30.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.809+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe22809c050 con 0x7fe228099f80 2023-09-30T22:33:30.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.810+0000 7fe224ff9700 1 -- 172.21.15.32:0/3671831633 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe218017020 con 0x7fe228099f80 2023-09-30T22:33:30.811 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-09-30T22:33:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 msgr2=0x7fe22811bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22811bf80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe21800ec10 tx=0x7fe21800ef20 comp rx=0 tx=0).stop 2023-09-30T22:33:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 shutdown_connections 2023-09-30T22:33:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fe228131070 0x7fe22811c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe228066410 0x7fe22811ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 --2- 172.21.15.32:0/3671831633 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe228099f80 0x7fe22811bf80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 >> 172.21.15.32:0/3671831633 conn(0x7fe2280a2c10 msgr2=0x7fe22812e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 shutdown_connections 2023-09-30T22:33:30.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.812+0000 7fe22e836700 1 -- 172.21.15.32:0/3671831633 wait complete. 2023-09-30T22:33:30.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:33:30.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.965+0000 7fb64c9a7700 1 Processor -- start 2023-09-30T22:33:30.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.965+0000 7fb64c9a7700 1 -- start start 2023-09-30T22:33:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64c9a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb6481369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64c9a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64c9a7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb648131350 con 0x7fb648131760 2023-09-30T22:33:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64c9a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64809ea40 con 0x7fb648136ee0 2023-09-30T22:33:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64c9a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb648132240 con 0x7fb6481345c0 2023-09-30T22:33:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb645d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb6481369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb646d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb648131760 0x7fb648131b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44364/0 (socket says 172.21.15.32:44364) 2023-09-30T22:33:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.966+0000 7fb64659c700 1 -- 172.21.15.32:0/455068398 learned_addr learned my addr 172.21.15.32:0/455068398 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb64559a700 1 -- 172.21.15.32:0/455068398 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415091644 0 0) 0x7fb648131350 con 0x7fb648131760 2023-09-30T22:33:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb64559a700 1 -- 172.21.15.32:0/455068398 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6340035b0 con 0x7fb648131760 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb646d9d700 1 -- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 msgr2=0x7fb6481369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb646d9d700 1 --2- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb6481369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb646d9d700 1 -- 172.21.15.32:0/455068398 >> v1:172.21.15.32:6789/0 conn(0x7fb648131760 legacy=0x7fb648131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb646d9d700 1 -- 172.21.15.32:0/455068398 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64809dfc0 con 0x7fb648136ee0 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb645d9b700 1 --2- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb6481369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb646d9d700 1 --2- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481392c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb640009df0 tx=0x7fb64000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f6dd038074ce4ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb64559a700 1 -- 172.21.15.32:0/455068398 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64000e070 con 0x7fb648136ee0 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb64559a700 1 -- 172.21.15.32:0/455068398 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64000bae0 con 0x7fb648136ee0 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.967+0000 7fb64559a700 1 -- 172.21.15.32:0/455068398 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64000be90 con 0x7fb648136ee0 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 -- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 msgr2=0x7fb6481392c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481392c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb640009df0 tx=0x7fb64000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 -- 172.21.15.32:0/455068398 shutdown_connections 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481392c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/455068398 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb6481369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 -- 172.21.15.32:0/455068398 >> 172.21.15.32:0/455068398 conn(0x7fb648066d70 msgr2=0x7fb6480a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 -- 172.21.15.32:0/455068398 shutdown_connections 2023-09-30T22:33:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.968+0000 7fb64c9a7700 1 -- 172.21.15.32:0/455068398 wait complete. 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 Processor -- start 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 -- start start 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 0x7fb648127a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb648127fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481284f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64812ad40 con 0x7fb648131760 2023-09-30T22:33:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.969+0000 7fb64c9a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb648129e70 con 0x7fb648136ee0 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb64c9a7700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481295b0 con 0x7fb6481345c0 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb64659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 0x7fb648127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb648127fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb646d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb648127fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:58598/0 (socket says 172.21.15.32:58598) 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 -- 172.21.15.32:0/1960852523 learned_addr learned my addr 172.21.15.32:0/1960852523 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb646d9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481284f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56094/0 (socket says 172.21.15.32:56094) 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb64659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 0x7fb648127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56562/0 (socket says 172.21.15.32:56562) 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 -- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 msgr2=0x7fb6481284f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481284f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 -- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 msgr2=0x7fb648127a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 0x7fb648127a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 -- 172.21.15.32:0/1960852523 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300094a0 con 0x7fb6481345c0 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb64659c700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 0x7fb648127a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb645d9b700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb648127fb0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb63000ec10 tx=0x7fb63000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63000c850 con 0x7fb6481345c0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63000c9b0 con 0x7fb6481345c0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63000cd20 con 0x7fb6481345c0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.970+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb640009a70 con 0x7fb6481345c0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.971+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64812b000 con 0x7fb6481345c0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.971+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb630013470 con 0x7fb6481345c0 2023-09-30T22:33:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.971+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64809e210 con 0x7fb6481345c0 2023-09-30T22:33:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.972+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fb63003f780 con 0x7fb6481345c0 2023-09-30T22:33:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:30.976+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb630021020 con 0x7fb6481345c0 2023-09-30T22:33:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.181+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb64804f2a0 con 0x7fb6481345c0 2023-09-30T22:33:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.181+0000 7fb63effd700 1 -- 172.21.15.32:0/1960852523 <== mon.2 v2:172.21.15.32:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb630013720 con 0x7fb6481345c0 2023-09-30T22:33:31.182 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 8 2023-09-30T22:33:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 msgr2=0x7fb648127fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb648127fb0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb63000ec10 tx=0x7fb63000ef20 comp rx=0 tx=0).stop 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 shutdown_connections 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb648136ee0 0x7fb6481284f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb648131760 0x7fb648127a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 --2- 172.21.15.32:0/1960852523 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fb6481345c0 0x7fb648127fb0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 >> 172.21.15.32:0/1960852523 conn(0x7fb648066d70 msgr2=0x7fb64812c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 shutdown_connections 2023-09-30T22:33:31.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.183+0000 7fb64c9a7700 1 -- 172.21.15.32:0/1960852523 wait complete. 2023-09-30T22:33:31.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:33:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.337+0000 7ff9978ff700 1 Processor -- start 2023-09-30T22:33:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.337+0000 7ff9978ff700 1 -- start start 2023-09-30T22:33:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff9978ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009e2b0 0x7ff990135a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff9978ff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff990135f60 0x7ff990138340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff9978ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99009b800 con 0x7ff99009d7d0 2023-09-30T22:33:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff9978ff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99009b620 con 0x7ff990135f60 2023-09-30T22:33:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff9978ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99009f370 con 0x7ff99009e2b0 2023-09-30T22:33:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff995e9c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff990135f60 0x7ff990138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff994e9a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009e2b0 0x7ff990135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff99569b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff99009d7d0 0x7ff99009dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44378/0 (socket says 172.21.15.32:44378) 2023-09-30T22:33:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff99569b700 1 -- 172.21.15.32:0/2769309554 learned_addr learned my addr 172.21.15.32:0/2769309554 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-09-30T22:33:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff995e9c700 1 -- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009e2b0 msgr2=0x7ff990135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff995e9c700 1 --2- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009e2b0 0x7ff990135a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff995e9c700 1 -- 172.21.15.32:0/2769309554 >> v1:172.21.15.32:6789/0 conn(0x7ff99009d7d0 legacy=0x7ff99009dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff995e9c700 1 -- 172.21.15.32:0/2769309554 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99009ef80 con 0x7ff990135f60 2023-09-30T22:33:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff987fff700 1 -- 172.21.15.32:0/2769309554 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936127029 0 0) 0x7ff99009b800 con 0x7ff99009d7d0 2023-09-30T22:33:31.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff994e9a700 1 --2- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009e2b0 0x7ff990135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff995e9c700 1 --2- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990135f60 0x7ff990138340 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff988009d90 tx=0x7ff98800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa30f1295d97699 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff987fff700 1 -- 172.21.15.32:0/2769309554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800e070 con 0x7ff990135f60 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff987fff700 1 -- 172.21.15.32:0/2769309554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98800bb20 con 0x7ff990135f60 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.338+0000 7ff987fff700 1 -- 172.21.15.32:0/2769309554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988012430 con 0x7ff990135f60 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 -- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990135f60 msgr2=0x7ff990138340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990135f60 0x7ff990138340 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff988009d90 tx=0x7ff98800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:31.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 -- 172.21.15.32:0/2769309554 shutdown_connections 2023-09-30T22:33:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff990135f60 0x7ff990138340 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2769309554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009e2b0 0x7ff990135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 -- 172.21.15.32:0/2769309554 >> 172.21.15.32:0/2769309554 conn(0x7ff990066d70 msgr2=0x7ff9900a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 -- 172.21.15.32:0/2769309554 shutdown_connections 2023-09-30T22:33:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.339+0000 7ff9978ff700 1 -- 172.21.15.32:0/2769309554 wait complete. 2023-09-30T22:33:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.340+0000 7ff9978ff700 1 Processor -- start 2023-09-30T22:33:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.340+0000 7ff9978ff700 1 -- start start 2023-09-30T22:33:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 0x7ff99011e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff99009e2b0 0x7ff99011e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff990135f60 0x7ff99011ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901357e0 con 0x7ff99009e2b0 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901216b0 con 0x7ff99009d7d0 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff990121fc0 con 0x7ff990135f60 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff99569b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 0x7ff99011e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff99009e2b0 0x7ff99011e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff99569b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 0x7ff99011e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56118/0 (socket says 172.21.15.32:56118) 2023-09-30T22:33:31.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff99569b700 1 -- 172.21.15.32:0/2784734654 learned_addr learned my addr 172.21.15.32:0/2784734654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff995e9c700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff990135f60 0x7ff99011ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 -- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff990135f60 msgr2=0x7ff99011ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff990135f60 0x7ff99011ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 -- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 msgr2=0x7ff99011e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 0x7ff99011e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 -- 172.21.15.32:0/2784734654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800094a0 con 0x7ff99009e2b0 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff99569b700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 0x7ff99011e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff994e9a700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff99009e2b0 0x7ff99011e940 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff98000e900 tx=0x7ff98000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980013680 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9800137e0 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980013b70 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.341+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff988009a70 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.342+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901226b0 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.343+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff980013cd0 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.343+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7ff98003f9d0 con 0x7ff99009e2b0 2023-09-30T22:33:31.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.343+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99011f8e0 con 0x7ff99009e2b0 2023-09-30T22:33:31.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.348+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff980011db0 con 0x7ff99009e2b0 2023-09-30T22:33:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.553+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff99004f2a0 con 0x7ff99009e2b0 2023-09-30T22:33:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.553+0000 7ff985ffb700 1 -- 172.21.15.32:0/2784734654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff980017070 con 0x7ff99009e2b0 2023-09-30T22:33:31.554 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.555+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff99009e2b0 msgr2=0x7ff99011e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.555+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff99009e2b0 0x7ff99011e940 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff98000e900 tx=0x7ff98000ec10 comp rx=0 tx=0).stop 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.555+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 shutdown_connections 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.555+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7ff990135f60 0x7ff99011ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.555+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff99009d7d0 0x7ff99011e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.555+0000 7ff9978ff700 1 --2- 172.21.15.32:0/2784734654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff99009e2b0 0x7ff99011e940 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.556+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 >> 172.21.15.32:0/2784734654 conn(0x7ff990066d70 msgr2=0x7ff9900a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.556+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 shutdown_connections 2023-09-30T22:33:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.556+0000 7ff9978ff700 1 -- 172.21.15.32:0/2784734654 wait complete. 2023-09-30T22:33:31.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:33:31.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.708+0000 7fabb97f9700 1 Processor -- start 2023-09-30T22:33:31.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.708+0000 7fabb97f9700 1 -- start start 2023-09-30T22:33:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb97f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb41303c0 0x7fabb4130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb97f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb406ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb97f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb409f620 con 0x7fabb406b0c0 2023-09-30T22:33:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb97f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb4130250 con 0x7fabb41303c0 2023-09-30T22:33:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb97f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb409e1b0 con 0x7fabb406a770 2023-09-30T22:33:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb406ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb2ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb41303c0 0x7fabb4130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb406ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56126/0 (socket says 172.21.15.32:56126) 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb27fc700 1 -- 172.21.15.32:0/2371842137 learned_addr learned my addr 172.21.15.32:0/2371842137 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb2ffd700 1 -- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 msgr2=0x7fabb406ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb2ffd700 1 --2- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb406ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb2ffd700 1 -- 172.21.15.32:0/2371842137 >> v1:172.21.15.32:6789/0 conn(0x7fabb406b0c0 legacy=0x7fabb413a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb1ffb700 1 -- 172.21.15.32:0/2371842137 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493787787 0 0) 0x7fabb409f620 con 0x7fabb406b0c0 2023-09-30T22:33:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb2ffd700 1 -- 172.21.15.32:0/2371842137 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb406b520 con 0x7fabb41303c0 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.709+0000 7fabb27fc700 1 --2- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb406ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb2ffd700 1 --2- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb4130790 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7faba4009840 tx=0x7faba4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273bff588b02c4a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb1ffb700 1 -- 172.21.15.32:0/2371842137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4011830 con 0x7fabb41303c0 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb1ffb700 1 -- 172.21.15.32:0/2371842137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4011990 con 0x7fabb41303c0 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb1ffb700 1 -- 172.21.15.32:0/2371842137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4011d20 con 0x7fabb41303c0 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 -- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 msgr2=0x7fabb4130790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 --2- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb4130790 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7faba4009840 tx=0x7faba4009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 -- 172.21.15.32:0/2371842137 shutdown_connections 2023-09-30T22:33:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 --2- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb4130790 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 --2- 172.21.15.32:0/2371842137 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb406ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 -- 172.21.15.32:0/2371842137 >> 172.21.15.32:0/2371842137 conn(0x7fabb4066d70 msgr2=0x7fabb40a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 -- 172.21.15.32:0/2371842137 shutdown_connections 2023-09-30T22:33:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.710+0000 7fabb97f9700 1 -- 172.21.15.32:0/2371842137 wait complete. 2023-09-30T22:33:31.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.711+0000 7fabb97f9700 1 Processor -- start 2023-09-30T22:33:31.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 -- start start 2023-09-30T22:33:31.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb411d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb406b0c0 0x7fabb411de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4072470 con 0x7fabb41303c0 2023-09-30T22:33:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4120280 con 0x7fabb406a770 2023-09-30T22:33:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb97f9700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40a3380 con 0x7fabb406b0c0 2023-09-30T22:33:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb411d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb411d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56140/0 (socket says 172.21.15.32:56140) 2023-09-30T22:33:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 -- 172.21.15.32:0/1791868704 learned_addr learned my addr 172.21.15.32:0/1791868704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb27fc700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb406b0c0 0x7fabb411de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 -- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb406b0c0 msgr2=0x7fabb411de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb406b0c0 0x7fabb411de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 -- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 msgr2=0x7fabb411e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb411e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb2ffd700 1 -- 172.21.15.32:0/1791868704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009550 con 0x7fabb406a770 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.712+0000 7fabb37fe700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb411e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.713+0000 7fabb2ffd700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb411d930 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faba4009fd0 tx=0x7faba4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.713+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba40036a0 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.713+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba401f070 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.713+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba40094a0 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.713+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4003a60 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.713+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb4130dc0 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.714+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faba4003bc0 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.714+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7faba4021070 con 0x7fabb406a770 2023-09-30T22:33:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.714+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb404f2a0 con 0x7fabb406a770 2023-09-30T22:33:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.719+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faba4015080 con 0x7fabb406a770 2023-09-30T22:33:31.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.923+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabb4071c30 con 0x7fabb406a770 2023-09-30T22:33:31.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.924+0000 7fabab7fe700 1 -- 172.21.15.32:0/1791868704 <== mon.1 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7faba40253e0 con 0x7fabb406a770 2023-09-30T22:33:31.926 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-09-30T22:33:31.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 msgr2=0x7fabb411d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb411d930 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faba4009fd0 tx=0x7faba4004750 comp rx=0 tx=0).stop 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 shutdown_connections 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb41303c0 0x7fabb411e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fabb406a770 0x7fabb411d930 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 --2- 172.21.15.32:0/1791868704 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fabb406b0c0 0x7fabb411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 >> 172.21.15.32:0/1791868704 conn(0x7fabb4066d70 msgr2=0x7fabb409b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 shutdown_connections 2023-09-30T22:33:31.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:31.926+0000 7fabb97f9700 1 -- 172.21.15.32:0/1791868704 wait complete. 2023-09-30T22:33:31.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T22:33:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.077+0000 7f0031f75700 1 Processor -- start 2023-09-30T22:33:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.077+0000 7f0031f75700 1 -- start start 2023-09-30T22:33:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f0031f75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 0x7f002c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f0031f75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 0x7f002c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f0031f75700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c070940 con 0x7f002c09f240 2023-09-30T22:33:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f0031f75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c06fd60 con 0x7f002c09e930 2023-09-30T22:33:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f0031f75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f002c06fa80 con 0x7f002c09f650 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 0x7f002c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 0x7f002c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 0x7f002c131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:58616/0 (socket says 172.21.15.32:58616) 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002b7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 0x7f002c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56152/0 (socket says 172.21.15.32:56152) 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 -- 172.21.15.32:0/2613434888 learned_addr learned my addr 172.21.15.32:0/2613434888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002affd700 1 --1- 172.21.15.32:0/2613434888 >> v1:172.21.15.32:6789/0 conn(0x7f002c09f240 0x7f002c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:44400/0 (socket says 172.21.15.32:44400) 2023-09-30T22:33:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 -- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 msgr2=0x7f002c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 --2- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 0x7f002c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002a7fc700 1 -- 172.21.15.32:0/2613434888 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852159114 0 0) 0x7f002c070940 con 0x7f002c09f240 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 -- 172.21.15.32:0/2613434888 >> v1:172.21.15.32:6789/0 conn(0x7f002c09f240 legacy=0x7f002c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002bfff700 1 -- 172.21.15.32:0/2613434888 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c09fb60 con 0x7f002c09f650 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.078+0000 7f002b7fe700 1 --2- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 0x7f002c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f002bfff700 1 --2- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 0x7f002c131f00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0024009e80 tx=0x7f002400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5673c05638e33bc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f002a7fc700 1 -- 172.21.15.32:0/2613434888 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400e070 con 0x7f002c09f650 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f002a7fc700 1 -- 172.21.15.32:0/2613434888 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002400b910 con 0x7f002c09f650 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f002a7fc700 1 -- 172.21.15.32:0/2613434888 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400bcc0 con 0x7f002c09f650 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 -- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 msgr2=0x7f002c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 --2- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 0x7f002c131f00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0024009e80 tx=0x7f002400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 -- 172.21.15.32:0/2613434888 shutdown_connections 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 --2- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f650 0x7f002c131f00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 --2- 172.21.15.32:0/2613434888 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09e930 0x7f002c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 -- 172.21.15.32:0/2613434888 >> 172.21.15.32:0/2613434888 conn(0x7f002c0a4210 msgr2=0x7f002c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 -- 172.21.15.32:0/2613434888 shutdown_connections 2023-09-30T22:33:32.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.079+0000 7f0031f75700 1 -- 172.21.15.32:0/2613434888 wait complete. 2023-09-30T22:33:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.081+0000 7f0031f75700 1 Processor -- start 2023-09-30T22:33:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.081+0000 7f0031f75700 1 -- start start 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f0031f75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 0x7f002c1a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f0031f75700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f240 0x7f002c1a4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f0031f75700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09f650 0x7f002c1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f0031f75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1a7680 con 0x7f002c09e930 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f0031f75700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0a32d0 con 0x7f002c09f650 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f0031f75700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1a6e10 con 0x7f002c09f240 2023-09-30T22:33:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 0x7f002c1a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 0x7f002c1a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56620/0 (socket says 172.21.15.32:56620) 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 -- 172.21.15.32:0/154124761 learned_addr learned my addr 172.21.15.32:0/154124761 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002affd700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f240 0x7f002c1a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002bfff700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09f650 0x7f002c1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 -- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f240 msgr2=0x7f002c1a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f240 0x7f002c1a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 -- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09f650 msgr2=0x7f002c1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09f650 0x7f002c1a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 -- 172.21.15.32:0/154124761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0094a0 con 0x7f002c09e930 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.082+0000 7f002b7fe700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 0x7f002c1a3bf0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f001c00ebe0 tx=0x7f001c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.083+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00c850 con 0x7f002c09e930 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.083+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c00c9b0 con 0x7f002c09e930 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.083+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00cd40 con 0x7f002c09e930 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.083+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0024009b30 con 0x7f002c09e930 2023-09-30T22:33:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.083+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c0a0190 con 0x7f002c09e930 2023-09-30T22:33:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.083+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f001c013470 con 0x7f002c09e930 2023-09-30T22:33:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.084+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7f001c03f770 con 0x7f002c09e930 2023-09-30T22:33:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.084+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c074ea0 con 0x7f002c09e930 2023-09-30T22:33:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.089+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f001c013720 con 0x7f002c09e930 2023-09-30T22:33:32.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.314+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f002c1a69c0 con 0x7f002c09e930 2023-09-30T22:33:32.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.314+0000 7f0013fff700 1 -- 172.21.15.32:0/154124761 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+637 (secure 0 0 0) 0x7f001c021030 con 0x7f002c09e930 2023-09-30T22:33:32.316 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/338848985 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.316 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3143218070 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.316 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2265333326 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.317 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6816/118188 2023-10-01T22:23:54.019278+0000 2023-09-30T22:33:32.317 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/708160155 2023-10-01T22:23:54.019278+0000 2023-09-30T22:33:32.317 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1488807751 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.317 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2199662186 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.318 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2862279367 2023-10-01T22:23:54.019278+0000 2023-09-30T22:33:32.318 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6817/3095020018 2023-10-01T22:24:21.192854+0000 2023-09-30T22:33:32.318 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6816/126553 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.318 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1998798597 2023-10-01T22:24:53.053679+0000 2023-09-30T22:33:32.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 msgr2=0x7f002c1a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 0x7f002c1a3bf0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f001c00ebe0 tx=0x7f001c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 shutdown_connections 2023-09-30T22:33:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f002c09f650 0x7f002c1a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f002c09e930 0x7f002c1a3bf0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 --2- 172.21.15.32:0/154124761 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7f002c09f240 0x7f002c1a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 >> 172.21.15.32:0/154124761 conn(0x7f002c0a4210 msgr2=0x7f002c1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:32.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 shutdown_connections 2023-09-30T22:33:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:32.317+0000 7f0031f75700 1 -- 172.21.15.32:0/154124761 wait complete. 2023-09-30T22:33:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:listed 11 entries 2023-09-30T22:33:32.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi121.front.sepia.ceph.com... 2023-09-30T22:33:32.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:33:32.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-30T22:33:32.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[112242]: fuse finished with error 0 and tester_r 0 2023-09-30T22:33:32.381 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:33:38.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2023-09-30T22:33:38.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:33:38.484 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-30T22:33:38.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:33:38.530 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:33:38.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:33:38.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2023-09-30T22:33:38.739 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-30T22:33:38.751 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:33:38.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:33:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.895+0000 7fca64f83700 1 Processor -- start 2023-09-30T22:33:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 -- start start 2023-09-30T22:33:38.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 0x7fca60133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 0x7fca60130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca60130870 0x7fca6013a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6006c570 con 0x7fca60133a80 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6006bc00 con 0x7fca60130870 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca64f83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca600a0040 con 0x7fca6012ff20 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca5dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 0x7fca60130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca5e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 0x7fca60133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca5dd9b700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 0x7fca60130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3302/0 says I am v2:172.21.15.32:58636/0 (socket says 172.21.15.32:58636) 2023-09-30T22:33:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca5e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 0x7fca60133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56630/0 (socket says 172.21.15.32:56630) 2023-09-30T22:33:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca5dd9b700 1 -- 172.21.15.32:0/3101140055 learned_addr learned my addr 172.21.15.32:0/3101140055 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.896+0000 7fca5ed9d700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca60130870 0x7fca6013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5dd9b700 1 -- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca60130870 msgr2=0x7fca6013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca60130870 0x7fca6013a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5dd9b700 1 -- 172.21.15.32:0/3101140055 >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 msgr2=0x7fca60133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/3101140055 >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 0x7fca60133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5dd9b700 1 -- 172.21.15.32:0/3101140055 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60133860 con 0x7fca6012ff20 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5e59c700 1 --2- 172.21.15.32:0/3101140055 >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 0x7fca60133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5ed9d700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca60130870 0x7fca6013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 0x7fca60130330 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fca540098f0 tx=0x7fca54009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf7b9ab799f1e79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5d59a700 1 -- 172.21.15.32:0/3101140055 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54011830 con 0x7fca6012ff20 2023-09-30T22:33:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5d59a700 1 -- 172.21.15.32:0/3101140055 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca54011990 con 0x7fca6012ff20 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.897+0000 7fca5d59a700 1 -- 172.21.15.32:0/3101140055 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54011d20 con 0x7fca6012ff20 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 -- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 msgr2=0x7fca60130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 0x7fca60130330 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fca540098f0 tx=0x7fca54009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 -- 172.21.15.32:0/3101140055 shutdown_connections 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca60130870 0x7fca6013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 --2- 172.21.15.32:0/3101140055 >> v2:172.21.15.32:3300/0 conn(0x7fca60133a80 0x7fca60133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 --2- 172.21.15.32:0/3101140055 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca6012ff20 0x7fca60130330 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 -- 172.21.15.32:0/3101140055 >> 172.21.15.32:0/3101140055 conn(0x7fca600a4210 msgr2=0x7fca600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 -- 172.21.15.32:0/3101140055 shutdown_connections 2023-09-30T22:33:38.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.898+0000 7fca64f83700 1 -- 172.21.15.32:0/3101140055 wait complete. 2023-09-30T22:33:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.899+0000 7fca64f83700 1 Processor -- start 2023-09-30T22:33:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.899+0000 7fca64f83700 1 -- start start 2023-09-30T22:33:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca6012ff20 0x7fca6009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 --2- >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca60130870 0x7fca6009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca60133a80 0x7fca6009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6009d7e0 con 0x7fca60133a80 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca600a20e0 con 0x7fca6012ff20 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 -- --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca6009df00 con 0x7fca60130870 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca6012ff20 0x7fca6009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca60133a80 0x7fca6009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca6012ff20 0x7fca6009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:56166/0 (socket says 172.21.15.32:56166) 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5e59c700 1 -- 172.21.15.32:0/1829062415 learned_addr learned my addr 172.21.15.32:0/1829062415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-09-30T22:33:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca60130870 0x7fca6009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 -- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca6012ff20 msgr2=0x7fca6009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca6012ff20 0x7fca6009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 -- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca60133a80 msgr2=0x7fca6009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca60133a80 0x7fca6009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 -- 172.21.15.32:0/1829062415 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca58009a70 con 0x7fca60130870 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5e59c700 1 --2- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fca6012ff20 0x7fca6009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca5dd9b700 1 --2- 172.21.15.32:0/1829062415 >> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] conn(0x7fca60130870 0x7fca6009c9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fca54016000 tx=0x7fca54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca4effd700 1 -- 172.21.15.32:0/1829062415 <== mon.2 v2:172.21.15.32:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54004240 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca4effd700 1 -- 172.21.15.32:0/1829062415 <== mon.2 v2:172.21.15.32:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5401f070 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca64f83700 1 -- 172.21.15.32:0/1829062415 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca500094a0 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.900+0000 7fca4effd700 1 -- 172.21.15.32:0/1829062415 <== mon.2 v2:172.21.15.32:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540036a0 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.901+0000 7fca64f83700 1 -- 172.21.15.32:0/1829062415 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca54009550 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.901+0000 7fca4effd700 1 -- 172.21.15.32:0/1829062415 <== mon.2 v2:172.21.15.32:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca54021030 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.901+0000 7fca4effd700 1 -- 172.21.15.32:0/1829062415 <== mon.2 v2:172.21.15.32:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4364+0+0 (secure 0 0 0) 0x7fca54040840 con 0x7fca60130870 2023-09-30T22:33:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.901+0000 7fca64f83700 1 -- 172.21.15.32:0/1829062415 --> [v2:172.21.15.32:3302/0,v1:172.21.15.32:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca60074ea0 con 0x7fca60130870 2023-09-30T22:33:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-09-30T22:33:38.906+0000 7fca4effd700 1 -- 172.21.15.32:0/1829062415 <== mon.2 v2:172.21.15.32:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca540239c0 con 0x7fca60130870 2023-09-30T22:35:38.794 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-30T22:35:38.796 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:35:38.808 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-30T22:35:38.808 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-30T22:35:38.808 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:38.824 INFO:tasks.ceph.mds.a:Stopped 2023-09-30T22:35:38.824 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-30T22:35:38.824 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T22:35:38.824 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:38.892 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T22:35:38.893 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T22:35:38.893 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:38.963 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T22:35:38.963 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T22:35:38.963 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:39.030 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T22:35:39.030 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T22:35:39.030 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:39.099 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T22:35:39.100 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-30T22:35:39.100 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T22:35:39.100 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:39.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:35:39.100 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-09-30T22:35:39.113 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T22:35:39.113 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-30T22:35:39.113 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T22:35:39.113 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:39.153 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T22:35:39.154 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T22:35:39.154 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:39.208 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T22:35:39.208 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T22:35:39.208 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:35:39.261 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T22:35:39.261 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-30T22:35:39.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2023-09-30T22:35:39.350 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:35:39.350 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-30T22:35:39.479 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:35:39.480 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-30T22:35:39.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:35:39.653 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-30T22:35:39.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:35:39.848 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-30T22:35:39.948 INFO:tasks.ceph:Archiving mon data... 2023-09-30T22:35:39.950 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/data/mon.a.tgz 2023-09-30T22:35:39.950 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-09-30T22:35:39.973 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.ZZbanptqh6 2023-09-30T22:35:39.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ZZbanptqh6 2023-09-30T22:35:40.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.ZZbanptqh6 2023-09-30T22:35:40.224 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.ZZbanptqh6 is 313KB 2023-09-30T22:35:40.329 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.ZZbanptqh6 2023-09-30T22:35:40.591 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/data/mon.b.tgz 2023-09-30T22:35:40.591 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-09-30T22:35:40.656 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.2nG4ZsR9eU 2023-09-30T22:35:40.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2nG4ZsR9eU 2023-09-30T22:35:40.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.2nG4ZsR9eU 2023-09-30T22:35:40.894 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.2nG4ZsR9eU is 316KB 2023-09-30T22:35:40.969 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.2nG4ZsR9eU 2023-09-30T22:35:40.992 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/data/mon.c.tgz 2023-09-30T22:35:40.992 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-09-30T22:35:41.054 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.SrsFHP0R1f 2023-09-30T22:35:41.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SrsFHP0R1f 2023-09-30T22:35:41.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.SrsFHP0R1f 2023-09-30T22:35:41.288 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.SrsFHP0R1f is 315KB 2023-09-30T22:35:41.384 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.SrsFHP0R1f 2023-09-30T22:35:41.407 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-30T22:35:41.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T22:35:41.449 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T22:35:41.487 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-30T22:35:41.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T22:35:41.525 DEBUG:teuthology.orchestra.run.smithi121:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T22:35:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T22:35:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T22:35:41.567 INFO:tasks.ceph:Archiving crash dumps... 2023-09-30T22:35:41.569 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/remote/smithi032/crash 2023-09-30T22:35:41.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T22:35:41.642 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/remote/smithi121/crash 2023-09-30T22:35:41.643 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T22:35:41.702 INFO:tasks.ceph:Compressing logs... 2023-09-30T22:35:41.702 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T22:35:41.705 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T22:35:41.733 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112242.log 2023-09-30T22:35:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125428.log 2023-09-30T22:35:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160219.log 2023-09-30T22:35:41.739 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173289.log 2023-09-30T22:35:41.741 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125428.log.gz 2023-09-30T22:35:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149108.log 2023-09-30T22:35:41.741 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160219.log.gz 2023-09-30T22:35:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135208.log 2023-09-30T22:35:41.741 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173289.log: /var/log/ceph/ceph-client.admin.149108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135493.log 2023-09-30T22:35:41.741 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173289.log.gz 2023-09-30T22:35:41.742 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149108.log.gz 2023-09-30T22:35:41.742 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176137.log 2023-09-30T22:35:41.743 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.0.112242.log: 96.6% -- replaced with /var/log/ceph/ceph-client.0.112242.log.gz 2023-09-30T22:35:41.743 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-09-30T22:35:41.744 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m0.019s 2023-09-30T22:35:41.744 INFO:teuthology.orchestra.run.smithi121.stderr:user 0m0.015s 2023-09-30T22:35:41.744 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m0.013s 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135208.log.gz 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177976.log 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135493.log.gz 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123495.log 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.176137.log.gz 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171842.log 2023-09-30T22:35:41.750 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177976.log.gz 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.123495.log: -- /var/log/ceph/ceph-client.admin.182096.log 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190976.log 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123495.log.gz 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171842.log.gz 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163824.log 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.182096.log.gz 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164629.log 2023-09-30T22:35:41.751 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.190976.log.gz 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149082.log 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163824.log.gz 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158827.log 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164629.log.gz 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158800.log 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149082.log.gz 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172701.log 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158827.log.gz 2023-09-30T22:35:41.752 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153304.log 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158800.log.gz 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153722.log 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153304.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.172701.log.gz 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133859.log 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153304.log.gz 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151566.log 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.153722.log.gz 2023-09-30T22:35:41.753 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188183.log 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151566.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133859.log.gz 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135744.log 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151566.log.gz 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118609.log 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.188183.log.gz 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191719.log 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135744.log.gz 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143914.log 2023-09-30T22:35:41.754 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118609.log.gz 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149580.log 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191719.log.gz 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161003.log 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143914.log.gz 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122239.log 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149580.log.gz 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177680.log 2023-09-30T22:35:41.755 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161003.log.gz 2023-09-30T22:35:41.756 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137902.log 2023-09-30T22:35:41.770 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122239.log.gz 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171695.log 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.177680.log.gz 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177706.log 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137902.log.gz 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181212.log 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177706.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171695.log.gz 2023-09-30T22:35:41.771 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167922.log 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177706.log.gz 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175035.log 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181212.log.gz 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156832.log 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175035.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.167922.log.gz 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189947.log 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175035.log.gz 2023-09-30T22:35:41.772 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156662.log 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156832.log.gz 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175941.log 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156662.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.189947.log.gz 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179348.log 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156662.log.gz 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154186.log 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175941.log.gz 2023-09-30T22:35:41.773 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125379.log 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179348.log.gz/var/log/ceph/ceph-client.admin.154186.log: 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167726.log 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154186.log.gz 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148690.log 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125379.log.gz 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175427.log 2023-09-30T22:35:41.774 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.167726.log.gz 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179591.log 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148690.log.gz 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179248.log 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175427.log.gz 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189042.log 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.179591.log.gz 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149802.log 2023-09-30T22:35:41.775 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179248.log.gz 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192111.log 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189042.log.gz 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191002.log 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149802.log.gz 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139372.log 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191002.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192111.log.gz 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162648.log 2023-09-30T22:35:41.776 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.191002.log.gz 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138490.log 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139372.log.gz 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156244.log 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162648.log.gz 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143718.log 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138490.log.gz 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127711.log 2023-09-30T22:35:41.777 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156244.log.gz 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165094.log 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127711.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143718.log.gz 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165021.log 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127711.log.gz 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166027.log 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165094.log.gz 2023-09-30T22:35:41.778 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180058.log 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165021.log.gz 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151834.log 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.166027.log.gz 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163722.log 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180058.log.gz 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168194.log 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163722.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151834.log.gz 2023-09-30T22:35:41.779 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163210.log 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163722.log.gz 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192605.log 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168194.log.gz 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173459.log 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192605.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163210.log.gz 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161276.log 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.192605.log.gz 2023-09-30T22:35:41.780 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182880.log 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173459.log.gz 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141928.log 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161276.log.gz 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128243.log 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.182880.log.gz 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187866.log 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.141928.log.gz 2023-09-30T22:35:41.781 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180328.log 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128243.log.gz 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139300.log 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.187866.log.gz 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179787.log 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180328.log.gz 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170274.log 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139300.log.gz 2023-09-30T22:35:41.782 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185635.log 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179787.log.gz 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192503.log 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170274.log.gz 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139398.log 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192503.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185635.log.gz 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134620.log 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.192503.log.gz 2023-09-30T22:35:41.783 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154578.log 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.139398.log.gz 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141758.log 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154578.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134620.log.gz 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156342.log 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154578.log.gz 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142810.log 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141758.log.gz 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161736.log 2023-09-30T22:35:41.784 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156342.log.gz 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182339.log 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142810.log.gz 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128114.log 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161736.log.gz 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182183.log 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.182339.log.gz 2023-09-30T22:35:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156368.log 2023-09-30T22:35:41.786 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128114.log.gz 2023-09-30T22:35:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190731.log 2023-09-30T22:35:41.786 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.182183.log.gz 2023-09-30T22:35:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144796.log 2023-09-30T22:35:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156368.log.gz 2023-09-30T22:35:41.786 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168390.log 2023-09-30T22:35:41.838 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.190731.log.gz 2023-09-30T22:35:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166847.log 2023-09-30T22:35:41.838 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144796.log.gz 2023-09-30T22:35:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135352.log 2023-09-30T22:35:41.838 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168390.log.gz 2023-09-30T22:35:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193792.log 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166847.log.gz 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155264.log 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135352.log.gz 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182927.log 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.193792.log.gz 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171159.log 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155264.log.gz 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146854.log 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.182927.log.gz 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163236.log 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146854.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171159.log.gz 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144182.log 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146854.log.gz 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177288.log 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163236.log.gz 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158728.log 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177288.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144182.log.gz 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162939.log 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177288.log.gz 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165217.log 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158728.log.gz 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145848.log 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162939.log.gz/var/log/ceph/ceph-client.admin.165217.log: 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162128.log 2023-09-30T22:35:41.841 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.165217.log.gz 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184774.log 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145848.log.gz 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144992.log 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184774.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162128.log.gz 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122746.log 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184774.log.gz 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142908.log 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144992.log.gz 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163330.log 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142908.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122746.log.gz 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166550.log 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142908.log.gz 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186664.log 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163330.log.gz 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189408.log 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.166550.log.gz 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124377.log 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.186664.log.gz 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189359.log 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.189408.log.gz 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150168.log 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124377.log.gz 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153696.log 2023-09-30T22:35:41.844 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150168.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189359.log.gz 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171646.log 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150168.log.gz 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182414.log 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153696.log.gz 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149508.log 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182414.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171646.log.gz 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177827.log 2023-09-30T22:35:41.845 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.182414.log.gz 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136724.log 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149508.log.gz 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166793.log 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.177827.log.gz 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126938.log 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136724.log.gz 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166450.log 2023-09-30T22:35:41.846 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166793.log.gz 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150364.log 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166450.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126938.log.gz 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185044.log 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166450.log.gz 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138882.log 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.150364.log.gz 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142516.log 2023-09-30T22:35:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138882.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.185044.log.gz 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181508.log 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138882.log.gz 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129147.log 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142516.log.gz 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123961.log 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129147.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.181508.log.gz 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166829.log 2023-09-30T22:35:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158924.log 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.129147.log.gz 88.2% 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123961.log.gz 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167043.log 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.166829.log.gz 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191990.log 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158924.log: /var/log/ceph/ceph-client.admin.167043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134667.log 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158924.log.gz 2023-09-30T22:35:41.849 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167043.log.gz 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168167.log 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191990.log.gz 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172283.log 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134667.log.gz 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158016.log 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168167.log.gz 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161668.log 2023-09-30T22:35:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158016.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172283.log.gz 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171552.log 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158016.log.gz 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161394.log 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161668.log.gz 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186076.log 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161394.log: gzip -5 --verbose -- 87.8%/var/log/ceph/ceph-client.admin.172140.log 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.171552.log.gz 2023-09-30T22:35:41.851 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.161394.log.gz 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174839.log 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.186076.log.gz 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145776.log 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172140.log.gz/var/log/ceph/ceph-client.admin.174839.log: 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144894.log 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174839.log.gz 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147612.log 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145776.log.gz 2023-09-30T22:35:41.852 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172038.log 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147612.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144894.log.gz 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142934.log 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147612.log.gz 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159094.log 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.172038.log.gz 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149206.log 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159094.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142934.log.gz 2023-09-30T22:35:41.853 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145384.log 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159094.log.gz 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160611.log 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149206.log.gz 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139006.log 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160611.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145384.log.gz 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146534.log 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.160611.log.gz 2023-09-30T22:35:41.854 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190022.log 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139006.log.gz 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124573.log 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190022.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146534.log.gz 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165635.log 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190022.log.gz 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193166.log 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124573.log.gz 2023-09-30T22:35:41.855 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150266.log 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193166.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.165635.log.gz 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137779.log 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193166.log.gz 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143620.log 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.150266.log.gz 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170323.log 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143620.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137779.log.gz 2023-09-30T22:35:41.856 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165243.log 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143620.log.gz 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147906.log 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170323.log.gz 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127015.log 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147906.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165243.log.gz 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176454.log 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147906.log.gz 2023-09-30T22:35:41.857 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171721.log 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127015.log.gz 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140770.log 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171721.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.176454.log.gz 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163089.log 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.171721.log.gz 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176725.log 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140770.log.gz 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149010.log 2023-09-30T22:35:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176725.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163089.log.gz 2023-09-30T22:35:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185436.log 2023-09-30T22:35:41.861 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176725.log.gz 2023-09-30T22:35:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175327.log 2023-09-30T22:35:41.861 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149010.log.gz 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181359.log 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175327.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.185436.log.gz 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192894.log 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175327.log.gz 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158310.log 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181359.log.gz 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183990.log 2023-09-30T22:35:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158310.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.192894.log.gz 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158994.log 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.158310.log.gz 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125477.log 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183990.log.gz 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180842.log 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158994.log.gz 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153500.log 2023-09-30T22:35:41.863 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180842.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125477.log.gz 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151468.log 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.180842.log.gz 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136063.log 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153500.log.gz 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146658.log 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136063.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151468.log.gz 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174643.log 2023-09-30T22:35:41.864 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136063.log.gz 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138588.log 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146658.log.gz 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124013.log 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138588.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174643.log.gz 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129407.log 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138588.log.gz 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167409.log 2023-09-30T22:35:41.865 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124013.log.gz 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184260.log 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129407.log.gz 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192552.log 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184260.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167409.log.gz 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174865.log 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.184260.log.gz/var/log/ceph/ceph-client.admin.192552.log: 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164310.log 2023-09-30T22:35:41.866 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.192552.log.gz 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125085.log 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174865.log.gz 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123348.log 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125085.log: gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.164310.log.gz--verbose 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:-- /var/log/ceph/ceph-client.admin.192409.log 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125085.log.gz 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127854.log 2023-09-30T22:35:41.867 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123348.log.gz 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181434.log 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.192409.log.gz 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134594.log 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127854.log.gz 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129220.log 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134594.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.181434.log.gz 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141340.log 2023-09-30T22:35:41.868 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.134594.log.gz 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188088.log 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129220.log.gz 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156564.log 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188088.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141340.log.gz 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188088.log.gz 2023-09-30T22:35:41.869 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143888.log 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.156564.log.gz 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126814.log 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141660.log 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126814.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143888.log.gz 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149606.log 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.126814.log.gz 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151736.log 2023-09-30T22:35:41.870 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141660.log.gz 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134960.log 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149606.log.gz 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134863.log 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151736.log: /var/log/ceph/ceph-client.admin.134960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128657.log 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.151736.log.gz 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134960.log.gz 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157846.log 2023-09-30T22:35:41.871 INFO:teuthology.orchestra.run.smithi032.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.134863.log.gz 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168559.log 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128657.log.gz 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152912.log 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157846.log.gz 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159512.log 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168559.log: /var/log/ceph/ceph-client.admin.152912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166354.log 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168559.log.gz 2023-09-30T22:35:41.872 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152912.log.gz 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153232.log 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166354.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.159512.log.gz 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166054.log 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166354.log.gz 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188650.log 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153232.log.gz 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166648.log 2023-09-30T22:35:41.873 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166054.log.gz 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190610.log 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166648.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.188650.log.gz 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137434.log 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166648.log.gz 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141634.log 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190610.log.gz 2023-09-30T22:35:41.874 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157348.log 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167138.log 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137434.log.gz 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141634.log.gz 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153892.log 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157348.log.gz 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185093.log 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167138.log: /var/log/ceph/ceph-client.admin.153892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170519.log 2023-09-30T22:35:41.875 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167138.log.gz 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153892.log.gz 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136920.log 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170519.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.185093.log.gz 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154114.log 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.170519.log.gz 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173655.log 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136920.log.gz 2023-09-30T22:35:41.876 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167235.log 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154114.log: /var/log/ceph/ceph-client.admin.173655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127401.log 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154114.log.gz 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173655.log.gz 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147932.log 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167235.log.gz 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185880.log 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127401.log.gz 2023-09-30T22:35:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139176.log 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185880.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.147932.log.gz 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188428.log 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185880.log.gz 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186468.log 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139176.log.gz 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169147.log 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186468.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.188428.log.gz 2023-09-30T22:35:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178490.log 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.186468.log.gz 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159190.log 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169147.log.gz 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153526.log 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159190.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.178490.log.gz 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147736.log 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159190.log.gz 2023-09-30T22:35:41.879 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164878.log 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153526.log.gz 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140876.log 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147736.log.gz 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191429.log 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164878.log: /var/log/ceph/ceph-client.admin.140876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181800.log 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164878.log.gz 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191429.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140876.log.gz 2023-09-30T22:35:41.880 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173067.log 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191429.log.gz 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140378.log 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181800.log: /var/log/ceph/ceph-client.admin.173067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174465.log 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181800.log.gz 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140378.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173067.log.gz 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182535.log 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140378.log.gz 2023-09-30T22:35:41.881 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161932.log 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174465.log.gz 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138124.log 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182535.log.gz/var/log/ceph/ceph-client.admin.161932.log: 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191598.log 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161932.log.gz 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154800.log 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.138124.log.gz 2023-09-30T22:35:41.882 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190244.log 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191598.log.gz 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138686.log 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190244.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154800.log.gz 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163994.log 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.190244.log.gz 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181826.log 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138686.log.gz 2023-09-30T22:35:41.883 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143815.log 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181826.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163994.log.gz 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171525.log 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.181826.log.gz 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166158.log 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143815.log: /var/log/ceph/ceph-client.admin.171525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179274.log 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143815.log.gz 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171525.log.gz 2023-09-30T22:35:41.884 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147808.log 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140156.log 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166158.log.gz 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179274.log: /var/log/ceph/ceph-client.admin.147808.log: gzip -5 --verbose 88.5%-- /var/log/ceph/ceph-client.admin.134450.log -- replaced with /var/log/ceph/ceph-client.admin.179274.log.gz 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147808.log.gz 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164582.log 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140156.log: /var/log/ceph/ceph-client.admin.134450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178294.log 2023-09-30T22:35:41.885 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140156.log.gz 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134450.log.gz 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190388.log 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.164582.log.gz 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142542.log 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190388.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.178294.log.gz 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190414.log 2023-09-30T22:35:41.886 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190388.log.gz 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144404.log 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142542.log.gz 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184309.log 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190414.log.gz 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151370.log 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144404.log.gz 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128706.log 2023-09-30T22:35:41.887 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.184309.log.gz 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164851.log 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151370.log: /var/log/ceph/ceph-client.admin.128706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150390.log 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151370.log.gz 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128706.log.gz 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156858.log 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164851.log.gz 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158604.log 2023-09-30T22:35:41.888 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150390.log: /var/log/ceph/ceph-client.admin.156858.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150390.log.gz 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124965.log 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156858.log.gz 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184456.log 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158604.log.gz 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163526.log 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138221.log 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184456.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.124965.log.gz 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133909.log 2023-09-30T22:35:41.889 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.184456.log.gz 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163526.log.gz 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168363.log 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133909.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138221.log.gz 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180571.log 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.133909.log.gz 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160492.log 2023-09-30T22:35:41.890 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168363.log.gz 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188087.log 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180571.log.gz 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178611.log 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160492.log: /var/log/ceph/ceph-client.admin.188087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171329.log 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160492.log.gz 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178611.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188087.log.gz 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148127.log 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.178611.log.gz 2023-09-30T22:35:41.891 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.171329.log: -- /var/log/ceph/ceph-client.admin.149776.log 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163304.log 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171329.log.gz 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149776.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148127.log.gz 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142150.log 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163304.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149776.log.gz 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166001.log 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163304.log.gz 2023-09-30T22:35:41.892 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169370.log 2023-09-30T22:35:41.893 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142150.log.gz 2023-09-30T22:35:41.893 INFO:teuthology.orchestra.run.smithi032.stderr:gzip /var/log/ceph/ceph-client.admin.166001.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.180620.log 2023-09-30T22:35:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163134.log 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166001.log.gz 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189826.log 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169370.log.gz 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163134.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.180620.log.gz 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122946.log 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163134.log.gz 2023-09-30T22:35:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176921.log 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.189826.log.gz 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125183.log 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.122946.log.gz 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186923.log 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125183.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176921.log.gz 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145750.log 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125183.log.gz 2023-09-30T22:35:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160952.log 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.186923.log.gz 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185558.log 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145750.log.gz 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191198.log 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160952.log.gz 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162844.log 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.185558.log.gz/var/log/ceph/ceph-client.admin.191198.log: 2023-09-30T22:35:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186298.log 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191198.log.gz 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177314.log 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162844.log.gz 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162912.log 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186298.log.gz 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188624.log 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177314.log.gz 2023-09-30T22:35:41.992 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122308.log 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162912.log.gz 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177902.log 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.188624.log.gz 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136648.log 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177902.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.122308.log.gz 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.173877.log 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177902.log.gz 2023-09-30T22:35:41.993 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140254.log 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136648.log.gz 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141438.log 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173877.log.gz 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125624.log 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140254.log.gz 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158898.log 2023-09-30T22:35:41.994 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125624.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141438.log.gz 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147246.log 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142836.log 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.125624.log.gz 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158898.log.gz 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141144.log 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147246.log.gz 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133588.log 2023-09-30T22:35:41.995 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142836.log.gz 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175167.log 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.141144.log.gz 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191915.log 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175167.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133588.log.gz 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133835.log 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175167.log.gz 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:gzip /var/log/ceph/ceph-client.admin.191915.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.186272.log 2023-09-30T22:35:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133835.log: gzip -5 88.1%--verbose -- /var/log/ceph/ceph-client.admin.147514.log -- replaced with /var/log/ceph/ceph-client.admin.191915.log.gz 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133835.log.gz 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165609.log 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186272.log.gz 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125281.log 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147514.log.gz 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191402.log 2023-09-30T22:35:41.997 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125281.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165609.log.gz 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124867.log 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125281.log.gz 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165465.log 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.191402.log.gz 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133883.log 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165465.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124867.log.gz 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180646.log 2023-09-30T22:35:41.998 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165465.log.gz 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187109.log 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133883.log.gz 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183206.log 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187109.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180646.log.gz 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182218.log 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187109.log.gz 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189016.log 2023-09-30T22:35:41.999 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.183206.log.gz 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137809.log 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189016.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.182218.log.gz 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159220.log 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189016.log.gz 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157518.log 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137809.log.gz 2023-09-30T22:35:42.000 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163918.log 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157518.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159220.log.gz 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160688.log 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157518.log.gz 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141562.log 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163918.log.gz 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149180.log 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141562.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160688.log.gz 2023-09-30T22:35:42.001 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162159.log 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141562.log.gz 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142026.log 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149180.log.gz 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144280.log 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142026.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.162159.log.gz 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133786.log 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142026.log.gz 2023-09-30T22:35:42.002 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134254.log 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144280.log.gz 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173802.log 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134254.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.133786.log.gz 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164190.log 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134254.log.gz 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150880.log 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173802.log.gz 2023-09-30T22:35:42.003 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160560.log 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150880.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164190.log.gz 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173606.log 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150880.log.gz 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175866.log 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160560.log.gz 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159708.log 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175866.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173606.log.gz 2023-09-30T22:35:42.004 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139274.log 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175866.log.gz 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154996.log 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159708.log.gz 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186811.log 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154996.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139274.log.gz 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170153.log 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154996.log.gz 2023-09-30T22:35:42.005 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137510.log 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.186811.log.gz 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186691.log 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137510.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170153.log.gz 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123765.log 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137510.log.gz 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138320.log 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186691.log.gz 2023-09-30T22:35:42.006 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179666.log 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138320.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123765.log.gz 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.138320.log.gz 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150756.log 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179666.log.gz 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171499.log 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.150756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190143.log 2023-09-30T22:35:42.007 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150756.log.gz 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178219.log 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171499.log.gz 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188846.log 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.190143.log.gz 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158506.log 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178219.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178219.log.gz 2023-09-30T22:35:42.008 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138516.log 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180424.log 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8%/var/log/ceph/ceph-client.admin.158506.log: -- replaced with /var/log/ceph/ceph-client.admin.188846.log.gz 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138516.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158506.log.gz 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145188.log 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138516.log.gz 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188284.log 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185831.log 2023-09-30T22:35:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145188.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180424.log.gz 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187865.log 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145188.log.gz 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.188284.log.gz-5 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:--verbose -- /var/log/ceph/ceph-client.admin.185831.log: /var/log/ceph/ceph-client.admin.161472.log 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185831.log.gz 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174047.log 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.187865.log.gz 2023-09-30T22:35:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181163.log 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161472.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161472.log.gz 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137732.log 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139790.log 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181163.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.174047.log.gz 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181163.log.gz 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174194.log 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150096.log 2023-09-30T22:35:42.011 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139790.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137732.log.gz 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139790.log.gz 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152226.log 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187791.log 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150096.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174194.log.gz 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152226.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150096.log.gz 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187278.log 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152226.log.gz 2023-09-30T22:35:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193193.log 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187791.log: /var/log/ceph/ceph-client.admin.187278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173093.log 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.187791.log.gz 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193193.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.187278.log.gz 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187007.log 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193193.log.gz 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153598.log 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173093.log.gz 2023-09-30T22:35:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141732.log 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr: 95.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2%/var/log/ceph/ceph-client.admin.153598.log: -- replaced with /var/log/ceph/ceph-client.admin.187007.log.gz 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155290.log 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153598.log.gz 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176895.log 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141732.log.gz 2023-09-30T22:35:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172626.log 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155290.log.gz 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123446.log 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176895.log.gz 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142640.log 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.172626.log.gz 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168632.log 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123446.log.gz 2023-09-30T22:35:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135070.log 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142640.log.gz 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144672.log 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135070.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168632.log.gz 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179740.log 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135070.log.gz 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124524.log 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.144672.log.gz 2023-09-30T22:35:42.016 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187203.log 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179740.log: /var/log/ceph/ceph-client.admin.124524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169098.log 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179740.log.gz 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187203.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124524.log.gz 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192970.log 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.187203.log.gz 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153036.log 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.169098.log.gz 2023-09-30T22:35:42.017 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146168.log 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.192970.log.gz 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191028.log 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146168.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153036.log.gz 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157944.log 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146168.log.gz 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171450.log 2023-09-30T22:35:42.018 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191028.log.gz 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159120.log 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171450.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.157944.log.gz 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128608.log 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171450.log.gz 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156760.log 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159120.log.gz 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174980.log 2023-09-30T22:35:42.019 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128608.log.gz 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142738.log 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156760.log.gz 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148618.log 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.174980.log.gz 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177092.log 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142738.log.gz 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156956.log 2023-09-30T22:35:42.020 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148618.log.gz 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139692.log 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177092.log.gz 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157820.log 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139692.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156956.log.gz 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188479.log 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139692.log.gz 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136136.log 2023-09-30T22:35:42.021 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127880.log 89.4% 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.157820.log.gz 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136136.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188479.log.gz 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152324.log 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136136.log.gz 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150070.log 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127880.log.gz 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163014.log 2023-09-30T22:35:42.022 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145456.log 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152324.log.gz 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150070.log.gz 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145554.log 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163014.log.gz 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188283.log 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145456.log.gz 2023-09-30T22:35:42.023 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169176.log 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145554.log.gz 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165562.log 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188283.log: /var/log/ceph/ceph-client.admin.169176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153330.log 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.188283.log.gz 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169176.log.gz 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128559.log 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136358.log 2023-09-30T22:35:42.024 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165562.log.gz 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128559.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153330.log.gz 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167632.log 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155878.log 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128559.log.gz 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136358.log.gz 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192997.log 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155878.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.167632.log.gz 2023-09-30T22:35:42.025 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154676.log 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155878.log.gz 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193089.log 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.192997.log.gz 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148519.log 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193089.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154676.log.gz 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134252.log 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.193089.log.gz 2023-09-30T22:35:42.026 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175915.log 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148519.log.gz/var/log/ceph/ceph-client.admin.134252.log: 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193400.log 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134252.log.gz 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188967.log 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.175915.log.gz 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167381.log 2023-09-30T22:35:42.027 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.193400.log.gz 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183917.log 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188967.log.gz 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181985.log 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167381.log.gz 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179003.log 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181985.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.183917.log.gz 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127981.log 2023-09-30T22:35:42.028 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181985.log.gz 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180918.log 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179003.log.gz 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154310.log 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127981.log.gz 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140084.log 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180918.log.gz 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172113.log 2023-09-30T22:35:42.029 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154310.log.gz 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172531.log 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140084.log.gz 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146436.log 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172531.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.172113.log.gz 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174695.log 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.172531.log.gz 2023-09-30T22:35:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181312.log 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174695.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146436.log.gz 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134764.log 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174695.log.gz 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192212.log 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181312.log.gz 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146560.log 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192212.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134764.log.gz 2023-09-30T22:35:42.031 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168706.log 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.192212.log.gz 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180032.log 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146560.log.gz 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189212.log 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168706.log.gz 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128755.log 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.180032.log.gz 2023-09-30T22:35:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135297.log 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189212.log.gz 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145482.log 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135297.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128755.log.gz 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168755.log 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135297.log.gz 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159024.log 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145482.log.gz/var/log/ceph/ceph-client.admin.168755.log: 2023-09-30T22:35:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144306.log 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159024.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168755.log.gz 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178660.log 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159024.log.gz 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187501.log 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144306.log.gz 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126636.log 2023-09-30T22:35:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187501.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178660.log.gz 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162452.log 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.187501.log.gz 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191319.log 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126636.log.gz 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123818.log 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191319.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162452.log.gz 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159776.log 2023-09-30T22:35:42.035 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191319.log.gz 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180816.log 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123818.log.gz 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166989.log 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180816.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.159776.log.gz-5 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:--verbose -- /var/log/ceph/ceph-client.admin.150488.log 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166989.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180816.log.gz 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163628.log 2023-09-30T22:35:42.036 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166989.log.gz 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168314.log 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163628.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150488.log.gz 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157224.log 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163628.log.gz 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122500.log 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.168314.log.gz 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163602.log 2023-09-30T22:35:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122500.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157224.log.gz 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179199.log 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122500.log.gz 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170375.log 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163602.log.gz 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123621.log 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170375.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179199.log.gz 2023-09-30T22:35:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170375.log.gz 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177118.log 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.123621.log.gz 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151540.log 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151860.log 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177118.log.gz 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171917.log 2023-09-30T22:35:42.039 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151540.log.gz 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156930.log 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151860.log.gz 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128853.log 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171917.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171917.log.gz 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118457.log 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161054.log 2023-09-30T22:35:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128853.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156930.log.gz 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118457.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128853.log.gz 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154284.log 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118457.log.gz 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144011.log 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169808.log 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154284.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161054.log.gz 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128902.log 2023-09-30T22:35:42.041 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154284.log.gz 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144011.log.gz 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153624.log 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181555.log 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169808.log.gz 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128902.log.gz 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143692.log 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181555.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153624.log.gz 2023-09-30T22:35:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165413.log 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143692.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.181555.log.gz 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127907.log 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.143692.log.gz 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158406.log 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165413.log: /var/log/ceph/ceph-client.admin.127907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169763.log 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165413.log.gz 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158406.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127907.log.gz 2023-09-30T22:35:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135574.log 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158406.log.gz 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139594.log 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135574.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169763.log.gz 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192016.log 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161148.log 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135574.log.gz 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139594.log: /var/log/ceph/ceph-client.admin.192016.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139594.log.gz 2023-09-30T22:35:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139202.log 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192016.log.gz 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145260.log 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139960.log 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161148.log.gz 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139202.log: /var/log/ceph/ceph-client.admin.145260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123299.log 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139202.log.gz 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145260.log.gz 2023-09-30T22:35:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155460.log 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155068.log 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123299.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139960.log.gz 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148226.log 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123299.log.gz 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155460.log.gz 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175474.log 2023-09-30T22:35:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147540.log 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148226.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155068.log.gz 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175474.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148226.log.gz 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141536.log 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175474.log.gz 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165439.log 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147540.log.gz 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182610.log 2023-09-30T22:35:42.047 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141536.log: /var/log/ceph/ceph-client.admin.165439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168828.log 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141536.log.gz 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152840.log 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165439.log.gz 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182610.log.gz 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137340.log 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170571.log 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152840.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.168828.log.gz 2023-09-30T22:35:42.048 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137340.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.152840.log.gz 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165047.log 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137340.log.gz 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177876.log 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170571.log.gz 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165047.log: /var/log/ceph/ceph-client.admin.177876.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165047.log.gz 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167999.log 2023-09-30T22:35:42.049 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177876.log.gz 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150658.log 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190440.log 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167999.log.gz 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159580.log 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150194.log 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190440.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.150658.log.gz 2023-09-30T22:35:42.050 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159580.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190440.log.gz 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174784.log 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 88.1%--verbose -- /var/log/ceph/ceph-client.admin.176333.log -- replaced with /var/log/ceph/ceph-client.admin.159580.log.gz 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150194.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.150194.log.gz 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148912.log 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174784.log.gz 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151174.log 2023-09-30T22:35:42.051 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176333.log.gz 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134568.log 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151174.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148912.log.gz 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134449.log 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151174.log.gz 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134568.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134568.log.gz 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134449.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134449.log.gz 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135866.log 2023-09-30T22:35:42.052 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139986.log 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189163.log 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135866.log.gz 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127374.log 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189163.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139986.log.gz 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154604.log 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149286.log 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189163.log.gz 2023-09-30T22:35:42.053 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127374.log.gz 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123152.log 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154604.log.gz 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172897.log 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123152.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.149286.log.gz 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123152.log.gz 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172897.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172897.log.gz 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180179.log 2023-09-30T22:35:42.054 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176650.log 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178956.log 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180179.log.gz 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134251.log 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178956.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176650.log.gz 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149312.log 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.178956.log.gz 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134251.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134251.log.gz 2023-09-30T22:35:42.055 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149312.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149312.log.gz 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146462.log 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128804.log 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118154.log 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146462.log.gz 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141072.log 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128804.log.gz 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189996.log 2023-09-30T22:35:42.056 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141072.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.118154.log.gz 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157322.log 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141072.log.gz 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191123.log 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189996.log.gz 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138022.log 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162230.log 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191123.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157322.log.gz 2023-09-30T22:35:42.057 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138022.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.191123.log.gz 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162230.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138022.log.gz 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162230.log.gz 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175061.log 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136750.log 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133980.log 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175061.log.gz 2023-09-30T22:35:42.058 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187595.log 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127635.log 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133980.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136750.log.gz 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187595.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133980.log.gz 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138614.log 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.187595.log.gz 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127635.log: /var/log/ceph/ceph-client.admin.138614.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127635.log.gz 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138614.log.gz 2023-09-30T22:35:42.059 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140744.log 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127799.log 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172087.log 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140744.log.gz 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159904.log 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192356.log 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127799.log.gz 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159904.log: gzip -5 89.3%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.172087.log.gz/var/log/ceph/ceph-client.admin.127220.log 2023-09-30T22:35:42.060 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159904.log.gz 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192356.log: /var/log/ceph/ceph-client.admin.127220.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.192356.log.gz 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127220.log.gz 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180524.log 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151344.log 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138222.log 2023-09-30T22:35:42.061 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180524.log.gz 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151344.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151344.log.gz 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187696.log 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138222.log.gz 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139496.log 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169735.log 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139496.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187696.log.gz 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.139496.log.gz 2023-09-30T22:35:42.062 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169735.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.169735.log.gz 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125036.log 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128145.log 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183576.log 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125036.log.gz 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151050.log 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190927.log 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128145.log.gz 2023-09-30T22:35:42.063 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183576.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.183576.log.gz 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151050.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151050.log.gz 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157642.log 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.190927.log.gz 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142320.log 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181900.log 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142320.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157642.log.gz 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142320.log.gz 2023-09-30T22:35:42.064 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181900.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.181900.log.gz 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193645.log 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161446.log 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118870.log 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.193645.log.gz 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161446.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161446.log.gz 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118870.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118870.log.gz 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169294.log 2023-09-30T22:35:42.065 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178760.log 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164825.log 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169294.log.gz 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152030.log 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125330.log 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164825.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178760.log.gz 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164825.log.gz 2023-09-30T22:35:42.066 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152030.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152030.log.gz 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125330.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125330.log.gz 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192160.log 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151148.log 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181604.log 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151148.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.192160.log.gz 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3%/var/log/ceph/ceph-client.admin.181604.log: -- replaced with /var/log/ceph/ceph-client.admin.151148.log.gz 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150684.log 2023-09-30T22:35:42.067 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181604.log.gz 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160074.log 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135156.log 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150684.log.gz 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174099.log 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160074.log.gz 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174098.log 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149410.log 2023-09-30T22:35:42.068 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174099.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135156.log.gz 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174098.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174099.log.gz 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166258.log 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174098.log.gz 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138418.log 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149410.log.gz 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166258.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166258.log.gz 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138418.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138418.log.gz 2023-09-30T22:35:42.069 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189751.log 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134790.log 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125526.log 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134790.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189751.log.gz 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189555.log 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134790.log.gz 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166942.log 2023-09-30T22:35:42.070 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149900.log 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125526.log.gz 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189555.log: /var/log/ceph/ceph-client.admin.166942.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189555.log.gz 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166942.log.gz 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149900.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149900.log.gz 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134253.log 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160296.log 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:gzip /var/log/ceph/ceph-client.admin.134253.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.179152.log 2023-09-30T22:35:42.071 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124426.log 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134253.log.gz 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5%/var/log/ceph/ceph-client.admin.179152.log: -- replaced with /var/log/ceph/ceph-client.admin.160296.log.gz 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186223.log 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179152.log.gz 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159827.log 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124426.log.gz 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123250.log 2023-09-30T22:35:42.072 INFO:teuthology.orchestra.run.smithi032.stderr:gzip 88.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.186223.log.gz-- 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186541.log 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159827.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159827.log.gz 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175623.log 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171107.log 88.9% 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123250.log.gz 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186541.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.186541.log.gz 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154970.log 2023-09-30T22:35:42.073 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158240.log 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175623.log.gz 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171107.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.171107.log.gz 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154970.log: 88.1%/var/log/ceph/ceph-client.admin.158240.log: -- replaced with /var/log/ceph/ceph-client.admin.154970.log.gz 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158240.log.gz 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164459.log 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191523.log 2023-09-30T22:35:42.074 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163798.log 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164459.log.gz 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141046.log 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161642.log 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.191523.log.gz 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163798.log: /var/log/ceph/ceph-client.admin.141046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158336.log 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163798.log.gz 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141046.log.gz 2023-09-30T22:35:42.075 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135689.log 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8%gzip /var/log/ceph/ceph-client.admin.158336.log: -- replaced with /var/log/ceph/ceph-client.admin.161642.log.gz-5 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:--verbose -- /var/log/ceph/ceph-client.admin.152742.log 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158336.log.gz 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135689.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135689.log.gz 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152742.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.152742.log.gz 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157152.log 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182292.log 2023-09-30T22:35:42.076 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135548.log 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157152.log.gz 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168588.log 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185512.log 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135548.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.182292.log.gz 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135548.log.gz 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168588.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168588.log.gz 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185512.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185512.log.gz 2023-09-30T22:35:42.077 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171355.log 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144110.log 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133739.log 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171355.log.gz 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144110.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144110.log.gz 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133739.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133739.log.gz 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145358.log 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187474.log 2023-09-30T22:35:42.078 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152546.log 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145358.log.gz 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155166.log 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.187474.log.gz 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161864.log 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184382.log 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155166.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152546.log.gz 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161864.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.155166.log.gz 2023-09-30T22:35:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190806.log 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- 87.8%/var/log/ceph/ceph-client.admin.147122.log 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.161864.log.gz 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184382.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.184382.log.gz 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190806.log: 88.1%/var/log/ceph/ceph-client.admin.147122.log: -- replaced with /var/log/ceph/ceph-client.admin.190806.log.gz 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147122.log.gz 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183402.log 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179395.log 2023-09-30T22:35:42.080 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169416.log 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179395.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.183402.log.gz 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167239.log 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179395.log.gz 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142346.log 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169416.log.gz 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151664.log 2023-09-30T22:35:42.081 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.167239.log.gz 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136209.log 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142346.log.gz 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137191.log 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:gzip 89.3%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.151664.log.gz-- 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181238.log 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136209.log: /var/log/ceph/ceph-client.admin.137191.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136209.log.gz 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137191.log.gz 2023-09-30T22:35:42.082 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158633.log 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174580.log 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.181238.log.gz 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157446.log 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158633.log.gz 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168044.log 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- 89.2%/var/log/ceph/ceph-client.admin.156734.log 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.174580.log.gz 2023-09-30T22:35:42.083 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157446.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157446.log.gz 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157748.log 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168044.log.gz 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156734.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156734.log.gz 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157748.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.157748.log.gz 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185906.log 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187892.log 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185906.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.185906.log.gz 2023-09-30T22:35:42.084 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187892.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.187892.log.gz 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162256.log 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145162.log 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162256.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162256.log.gz 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145162.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145162.log.gz 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148324.log 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184848.log 2023-09-30T22:35:42.085 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180254.log 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148324.log.gz 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126813.log 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152716.log 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184848.log.gz 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180254.log: /var/log/ceph/ceph-client.admin.126813.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180254.log.gz 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.126813.log.gz 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123962.log 2023-09-30T22:35:42.086 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152716.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152716.log.gz 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177239.log 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138294.log 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123962.log.gz 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142248.log 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173315.log 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177239.log.gz 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138294.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138294.log.gz 2023-09-30T22:35:42.087 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142124.log 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173315.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.142248.log.gz 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173315.log.gz 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142124.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.142124.log.gz 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179078.log 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137706.log 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185316.log 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179078.log.gz 2023-09-30T22:35:42.088 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123052.log 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157126.log 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185316.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137706.log.gz 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.185316.log.gz 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123052.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123052.log.gz 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157126.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157126.log.gz 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179983.log 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142712.log 2023-09-30T22:35:42.089 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140450.log 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179983.log.gz 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173998.log 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179444.log 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142712.log.gz 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156048.log 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140450.log.gz 2023-09-30T22:35:42.090 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173998.log.gz 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154872.log 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156074.log 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.179444.log.gz 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156048.log.gz 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151932.log 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 87.8%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.154872.log.gz/var/log/ceph/ceph-client.admin.185289.log 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T22:35:42.091 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156074.log: /var/log/ceph/ceph-client.admin.151932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184534.log 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156074.log.gz 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151932.log.gz 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185289.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.185289.log.gz 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160168.log 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184534.log.gz 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123101.log 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179836.log 2023-09-30T22:35:42.092 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123101.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.160168.log.gz 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159878.log 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179836.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123101.log.gz 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175819.log 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179836.log.gz 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174669.log 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152448.log 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159878.log.gz 2023-09-30T22:35:42.093 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175819.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.175819.log.gz 2023-09-30T22:35:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174669.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174669.log.gz 2023-09-30T22:35:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152448.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.152448.log.gz 2023-09-30T22:35:42.094 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129098.log 2023-09-30T22:35:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182566.log 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175523.log 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182566.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129098.log.gz 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175523.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.182566.log.gz 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175523.log.gz 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146828.log 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122697.log 2023-09-30T22:35:42.151 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160807.log 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146828.log.gz 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183357.log 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152128.log 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122697.log.gz 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160807.log: /var/log/ceph/ceph-client.admin.183357.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160807.log.gz 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152128.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183357.log.gz 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152128.log.gz 2023-09-30T22:35:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191768.log 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154774.log 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146952.log 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191768.log.gz 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157616.log 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147148.log 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154774.log.gz 2023-09-30T22:35:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146952.log: /var/log/ceph/ceph-client.admin.157616.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146952.log.gz 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157616.log.gz 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147148.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147148.log.gz 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172430.log 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159386.log 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137314.log 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159386.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172430.log.gz 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162716.log 2023-09-30T22:35:42.154 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159386.log.gz 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136332.log 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162716.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137314.log.gz 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162716.log.gz 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167436.log 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136332.log.gz 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144698.log 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135966.log 2023-09-30T22:35:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167436.log: /var/log/ceph/ceph-client.admin.144698.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167436.log.gz 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144698.log.gz 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135966.log.gz 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169882.log 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162426.log 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177780.log 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169882.log.gz 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152520.log 2023-09-30T22:35:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185953.log 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162426.log.gz 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177780.log: /var/log/ceph/ceph-client.admin.152520.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177780.log.gz 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179862.log 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152520.log.gz 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185953.log: /var/log/ceph/ceph-client.admin.179862.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.185953.log.gz 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.135182.log 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179862.log.gz 2023-09-30T22:35:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193449.log 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124328.log 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135182.log: /var/log/ceph/ceph-client.admin.193449.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135182.log.gz 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124328.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193449.log.gz 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124328.log.gz 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169931.log 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154212.log 2023-09-30T22:35:42.158 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183672.log 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169931.log.gz 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140850.log 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162818.log 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154212.log.gz 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183672.log: /var/log/ceph/ceph-client.admin.140850.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183672.log.gz 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178098.log 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140850.log.gz 2023-09-30T22:35:42.159 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165270.log 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162818.log.gz 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175353.log 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178098.log.gz 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165270.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165270.log.gz 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175353.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175353.log.gz 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149704.log 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182761.log 2023-09-30T22:35:42.160 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148592.log 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149704.log.gz 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140476.log 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148592.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.182761.log.gz 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.148592.log.gz 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140476.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140476.log.gz 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190584.log 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139568.log 2023-09-30T22:35:42.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139764.log 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190584.log.gz 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141830.log 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139764.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139568.log.gz 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179052.log 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139764.log.gz 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161344.log 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141830.log.gz 2023-09-30T22:35:42.162 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179052.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179052.log.gz 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161344.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161344.log.gz 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143130.log 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175231.log 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172675.log 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143130.log.gz 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188820.log 2023-09-30T22:35:42.163 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172675.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175231.log.gz 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183476.log 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.172675.log.gz 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188820.log: /var/log/ceph/ceph-client.admin.183476.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188820.log.gz 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.183476.log.gz 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123817.log 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174296.log 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129000.log 2023-09-30T22:35:42.164 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174296.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123817.log.gz 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135940.log 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174296.log.gz 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129000.log: /var/log/ceph/ceph-client.admin.135940.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.129000.log.gz 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143326.log 89.1% 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr: -- replaced with /var/log/ceph/ceph-client.admin.135940.log.gz 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184160.log 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159972.log 2023-09-30T22:35:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143326.log: /var/log/ceph/ceph-client.admin.184160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186737.log 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143326.log.gz 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184160.log.gz 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159972.log: /var/log/ceph/ceph-client.admin.186737.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159972.log.gz 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.186737.log.gz 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163040.log 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166597.log 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185684.log 2023-09-30T22:35:42.166 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163040.log.gz 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192307.log 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185684.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.166597.log.gz 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151638.log 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.185684.log.gz 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192307.log.gz 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151442.log 2023-09-30T22:35:42.167 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161838.log 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151442.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151638.log.gz 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151442.log.gz 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161838.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161838.log.gz 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148520.log 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146142.log 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160884.log 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148520.log.gz 2023-09-30T22:35:42.168 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146142.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146142.log.gz 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160884.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160884.log.gz 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173903.log 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156440.log 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162520.log 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.173903.log.gz 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156440.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156440.log.gz 2023-09-30T22:35:42.169 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162520.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162520.log.gz 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172727.log 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193351.log 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172727.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172727.log.gz 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167848.log 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193351.log.gz 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147318.log 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174491.log 2023-09-30T22:35:42.170 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167848.log.gz 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147318.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147318.log.gz 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174491.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174491.log.gz 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147442.log 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183327.log 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180132.log 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183327.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147442.log.gz 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183327.log.gz 2023-09-30T22:35:42.171 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180132.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180132.log.gz 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136406.log 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157420.log 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182488.log 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157420.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136406.log.gz 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.157420.log.gz 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182488.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.182488.log.gz 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189656.log 2023-09-30T22:35:42.172 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125575.log 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189656.log: /var/log/ceph/ceph-client.admin.125575.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189656.log.gz 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125575.log.gz 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178415.log 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.111871.log 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162060.log 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.178415.log.gz 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.tmp-client.admin.111871.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.111871.log.gz 2023-09-30T22:35:42.173 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144476.log 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162060.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162060.log.gz 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164655.log 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176699.log 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144476.log: /var/log/ceph/ceph-client.admin.164655.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144476.log.gz 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164655.log.gz 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176699.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176699.log.gz 2023-09-30T22:35:42.174 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187670.log 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167577.log 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187670.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.187670.log.gz 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167577.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167577.log.gz 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150586.log 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157918.log 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182684.log 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157918.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150586.log.gz 2023-09-30T22:35:42.175 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157918.log.gz 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182684.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.182684.log.gz 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192774.log 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128461.log 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189434.log 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128461.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192774.log.gz 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128461.log.gz 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189434.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.189434.log.gz 2023-09-30T22:35:42.176 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148200.log 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186494.log 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170545.log 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148200.log.gz 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186494.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.186494.log.gz 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170545.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170545.log.gz 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126687.log 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191964.log 2023-09-30T22:35:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126687.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126687.log.gz 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191964.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.191964.log.gz 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185485.log 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139104.log 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185485.log: /var/log/ceph/ceph-client.admin.139104.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.185485.log.gz 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139104.log.gz 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172309.log 2023-09-30T22:35:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149384.log 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162034.log 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172309.log.gz 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149384.log: /var/log/ceph/ceph-client.admin.162034.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.149384.log.gz 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.162034.log.gz 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126537.log 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177435.log 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163696.log 2023-09-30T22:35:42.179 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177435.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.126537.log.gz 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177435.log.gz 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163696.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163696.log.gz 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178268.log 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155584.log 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123397.log 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.178268.log.gz 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155584.log: /var/log/ceph/ceph-client.admin.123397.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155584.log.gz 2023-09-30T22:35:42.180 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123397.log.gz 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140352.log 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135993.log 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154898.log 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140352.log.gz 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135993.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135993.log.gz 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154898.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154898.log.gz 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187252.log 2023-09-30T22:35:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162324.log 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187252.log: /var/log/ceph/ceph-client.admin.162324.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.187252.log.gz 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162324.log.gz 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168510.log 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141856.log 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183131.log 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168510.log.gz 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141856.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141856.log.gz 2023-09-30T22:35:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183131.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183131.log.gz 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141170.log 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173707.log 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188086.log 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4%/var/log/ceph/ceph-client.admin.173707.log: -- replaced with /var/log/ceph/ceph-client.admin.141170.log.gz 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173707.log.gz 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188086.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188086.log.gz 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136162.log 2023-09-30T22:35:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193694.log 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136162.log: /var/log/ceph/ceph-client.admin.193694.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136162.log.gz 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193694.log.gz 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155852.log 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137630.log 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187305.log 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155852.log.gz 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137630.log: /var/log/ceph/ceph-client.admin.187305.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137630.log.gz 2023-09-30T22:35:42.184 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.187305.log.gz 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142052.log 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144012.log 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152644.log 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.142052.log.gz 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144012.log: /var/log/ceph/ceph-client.admin.152644.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144012.log.gz 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152644.log.gz 2023-09-30T22:35:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143398.log 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165805.log 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179640.log 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143398.log.gz 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167605.log 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158702.log 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179640.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165805.log.gz 2023-09-30T22:35:42.186 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167605.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179640.log.gz 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189265.log 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163406.log 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8%/var/log/ceph/ceph-client.admin.158702.log: -- replaced with /var/log/ceph/ceph-client.admin.167605.log.gz 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158702.log.gz 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189265.log: /var/log/ceph/ceph-client.admin.163406.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189265.log.gz 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163406.log.gz 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167773.log 2023-09-30T22:35:42.187 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163500.log 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.167773.log: -- /var/log/ceph/ceph-client.admin.138098.log 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163500.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.167773.log.gz-5 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:--verbose -- /var/log/ceph/ceph-client.admin.183964.log 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163500.log.gz 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148030.log 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138098.log.gz 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148984.log 2023-09-30T22:35:42.188 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152422.log 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148030.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183964.log.gz 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148984.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148030.log.gz 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.148984.log.gz 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152422.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152422.log.gz 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173214.log 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140058.log 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152350.log 2023-09-30T22:35:42.189 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.173214.log.gz 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173410.log 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167971.log 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152350.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.140058.log.gz 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173410.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152350.log.gz 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167971.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173410.log.gz 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167971.log.gz 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189852.log 2023-09-30T22:35:42.190 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156146.log 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176062.log 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189852.log.gz 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188883.log 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184924.log 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176062.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.156146.log.gz 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188883.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176062.log.gz 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150952.log 2023-09-30T22:35:42.191 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188883.log.gz 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184924.log: /var/log/ceph/ceph-client.admin.150952.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.184924.log.gz 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150952.log.gz 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134127.log 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143006.log 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156270.log 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134127.log.gz 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138980.log 2023-09-30T22:35:42.192 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191172.log 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156270.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143006.log.gz 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138980.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156270.log.gz 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128951.log 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138980.log.gz 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191172.log: /var/log/ceph/ceph-client.admin.128951.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191172.log.gz 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128951.log.gz 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165858.log 2023-09-30T22:35:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129049.log 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140948.log 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165858.log.gz 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168240.log 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159316.log 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140948.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129049.log.gz 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168240.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140948.log.gz 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168240.log.gz 2023-09-30T22:35:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178368.log 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159316.log.gz 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169612.log 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188676.log 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178368.log: /var/log/ceph/ceph-client.admin.169612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181949.log 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178368.log.gz 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.169612.log.gz 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189070.log 2023-09-30T22:35:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188676.log.gz 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.181949.log.gz 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189070.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.189070.log.gz 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134986.log 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126986.log 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175670.log 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134986.log.gz 2023-09-30T22:35:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161977.log 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126986.log.gz/var/log/ceph/ceph-client.admin.175670.log: 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176111.log 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170078.log 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175670.log.gz 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176111.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161977.log.gz 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176111.log.gz 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170078.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170078.log.gz 2023-09-30T22:35:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136972.log 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139862.log 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136972.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136972.log.gz 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139862.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139862.log.gz 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149678.log 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146070.log 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124279.log 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149678.log.gz 2023-09-30T22:35:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179936.log 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.146070.log.gz 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160662.log 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose 88.9%-- /var/log/ceph/ceph-client.admin.186860.log -- replaced with /var/log/ceph/ceph-client.admin.124279.log.gz 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179936.log: 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179936.log.gz 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165954.log 2023-09-30T22:35:42.199 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124720.log 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160662.log.gz 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186860.log: /var/log/ceph/ceph-client.admin.165954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190832.log 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.186860.log.gz 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124720.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165954.log.gz 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124769.log 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124720.log.gz 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143522.log 2023-09-30T22:35:42.200 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.190832.log.gz 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124769.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124769.log.gz 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143522.log.gz 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186345.log 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176211.log 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169958.log 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.186345.log.gz 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176211.log: /var/log/ceph/ceph-client.admin.169958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182806.log 2023-09-30T22:35:42.201 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176211.log.gz 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.169958.log.gz 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171755.log 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182806.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162743.log 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.182806.log.gz 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180450.log 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171755.log: /var/log/ceph/ceph-client.admin.162743.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171755.log.gz 2023-09-30T22:35:42.202 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180450.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162743.log.gz 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180450.log.gz 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157054.log 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186027.log 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177192.log 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.157054.log.gz 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186027.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186027.log.gz 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177192.log.gz 2023-09-30T22:35:42.203 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137584.log 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169220.log 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149972.log 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169220.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137584.log.gz 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169220.log.gz 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149972.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.149972.log.gz 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129517.log 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153990.log 2023-09-30T22:35:42.204 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129517.log: /var/log/ceph/ceph-client.admin.153990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129517.log.gz 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153990.log.gz 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169539.log 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125134.log 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170715.log 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169539.log.gz 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.125134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162547.log 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125134.log.gz 2023-09-30T22:35:42.205 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170715.log: /var/log/ceph/ceph-client.admin.162547.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170715.log.gz 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162547.log.gz 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135796.log 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188232.log 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155754.log 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135796.log.gz 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188232.log: /var/log/ceph/ceph-client.admin.155754.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188232.log.gz 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155754.log.gz 2023-09-30T22:35:42.206 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183280.log 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183280.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183280.log.gz 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145972.log 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.145972.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145972.log.gz 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165758.log 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165758.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165758.log.gz 2023-09-30T22:35:42.207 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190218.log 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190218.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190218.log.gz 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175745.log 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175745.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.175745.log.gz 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188379.log 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188379.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188379.log.gz 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170937.log 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170937.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.170937.log.gz 2023-09-30T22:35:42.208 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147024.log 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147024.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147024.log.gz 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126816.log 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126816.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126816.log.gz 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122599.log 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122599.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122599.log.gz 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173851.log 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173851.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173851.log.gz 2023-09-30T22:35:42.209 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145064.log 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145064.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145064.log.gz 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145286.log 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145286.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145286.log.gz 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128314.log 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128314.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128314.log.gz 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158042.log 2023-09-30T22:35:42.210 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158042.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158042.log.gz 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148788.log 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148788.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148788.log.gz 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173681.log 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173681.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173681.log.gz 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191624.log 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191624.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.191624.log.gz 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172336.log 2023-09-30T22:35:42.211 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172336.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172336.log.gz 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176529.log 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143104.log 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176529.log: /var/log/ceph/ceph-client.admin.143104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176529.log.gz 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143104.log.gz 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140280.log 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140280.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140280.log.gz 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164114.log 2023-09-30T22:35:42.212 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164114.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164114.log.gz 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160756.log 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160756.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160756.log.gz 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155094.log 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155094.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155094.log.gz 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162622.log 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162622.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162622.log.gz 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124671.log 2023-09-30T22:35:42.213 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124671.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124671.log.gz 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192382.log 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192382.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192382.log.gz 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160100.log 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160100.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160100.log.gz 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153402.log 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153402.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153402.log.gz 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159682.log 2023-09-30T22:35:42.214 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159682.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159682.log.gz 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179544.log 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179544.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.179544.log.gz 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164070.log 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164070.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164070.log.gz 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135014.log 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135014.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135014.log.gz 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186615.log 2023-09-30T22:35:42.215 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186615.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186615.log.gz 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170862.log 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170862.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170862.log.gz 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184897.log 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184897.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184897.log.gz 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156466.log 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156466.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156466.log.gz 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145090.log 2023-09-30T22:35:42.216 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145090.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145090.log.gz 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170180.log 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170180.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170180.log.gz 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150462.log 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150462.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150462.log.gz 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140974.log 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140974.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140974.log.gz 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144208.log 2023-09-30T22:35:42.217 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144208.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144208.log.gz 2023-09-30T22:35:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165170.log 2023-09-30T22:35:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165170.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165170.log.gz 2023-09-30T22:35:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183868.log 2023-09-30T22:35:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183868.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183868.log.gz 2023-09-30T22:35:42.218 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168951.log 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168951.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168951.log.gz 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134816.log 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134816.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134816.log.gz 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144378.log 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144378.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.144378.log.gz 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142614.log 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142614.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142614.log.gz 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181704.log 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181704.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181704.log.gz 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166205.log 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166205.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.166205.log.gz 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161540.log 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161540.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.161540.log.gz 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177510.log 2023-09-30T22:35:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177510.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177510.log.gz 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189630.log 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189630.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189630.log.gz 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188480.log 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188480.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.188480.log.gz 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163432.log 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158140.log 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163432.log: /var/log/ceph/ceph-client.admin.158140.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163432.log.gz 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158140.log.gz 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190535.log 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190535.log: /var/log/ceph/ceph-mon.c.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190535.log.gz 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147638.log 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147638.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147638.log.gz 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127451.log 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127451.log.gz 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136798.log 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136798.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136798.log.gz 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134394.log 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134394.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134394.log.gz 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173511.log 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173511.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173511.log.gz 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164778.log 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164778.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.164778.log.gz 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159486.log 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159486.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159486.log.gz 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164506.log 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164506.log.gz 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147344.log 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147344.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147344.log.gz 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138908.log 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138908.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138908.log.gz 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148396.log 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148396.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148396.log.gz 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143424.log 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143424.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143424.log.gz 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144600.log 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144600.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.144600.log.gz 2023-09-30T22:35:42.241 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118506.log 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118506.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118506.log.gz 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150292.log 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150292.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150292.log.gz 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143032.log 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143032.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143032.log.gz 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182956.log 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182956.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182956.log.gz 2023-09-30T22:35:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143202.log 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143202.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143202.log.gz 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147050.log 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147050.log.gz 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176407.log 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176407.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176407.log.gz 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168118.log 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168118.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168118.log.gz 2023-09-30T22:35:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123618.log 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123618.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.123618.log.gz 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148814.log 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148814.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148814.log.gz 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192944.log 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192944.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.192944.log.gz 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170741.log 2023-09-30T22:35:42.244 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170741.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170741.log.gz 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173263.log 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173263.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173263.log.gz 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141954.log 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141954.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141954.log.gz 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184578.log 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184578.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184578.log.gz 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192748.log 2023-09-30T22:35:42.245 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192748.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192748.log.gz 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155558.log 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155558.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.155558.log.gz 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164216.log 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164216.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164216.log.gz 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182143.log 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182143.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.182143.log.gz 2023-09-30T22:35:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123620.log 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123620.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123620.log.gz 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134029.log 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134029.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134029.log.gz 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140548.log 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140548.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140548.log.gz 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175108.log 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175108.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.175108.log.gz 2023-09-30T22:35:42.247 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145652.log 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145652.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.145652.log.gz 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146338.log 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146338.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146338.log.gz 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141366.log 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141366.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141366.log.gz 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151958.log 2023-09-30T22:35:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151958.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151958.log.gz 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154088.log 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154088.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154088.log.gz 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153134.log 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153134.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153134.log.gz 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185240.log 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185240.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185240.log.gz 2023-09-30T22:35:42.249 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136452.log 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124230.log 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136452.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136452.log.gz 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124230.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124230.log.gz 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135770.log 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122357.log 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135770.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135770.log.gz 2023-09-30T22:35:42.250 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122357.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122357.log.gz 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154480.log 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122795.log 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154480.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.154480.log.gz 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122795.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122795.log.gz 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157722.log 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153428.log 2023-09-30T22:35:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157722.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157722.log.gz 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153428.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153428.log.gz 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187082.log 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158453.log 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187082.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.187082.log.gz 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158453.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.158453.log.gz 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160023.log 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146756.log 2023-09-30T22:35:42.252 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160023.log: /var/log/ceph/ceph-client.admin.146756.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160023.log.gz 2023-09-30T22:35:42.262 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.146756.log.gz 2023-09-30T22:35:42.262 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151246.log 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140672.log 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151246.log: /var/log/ceph/ceph-client.admin.140672.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151246.log.gz 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140672.log.gz 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193302.log 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178564.log 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193302.log: /var/log/ceph/ceph-client.admin.178564.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193302.log.gz 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178564.log.gz 2023-09-30T22:35:42.263 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154702.log 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171945.log 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154702.log: /var/log/ceph/ceph-client.admin.171945.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154702.log.gz 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.171945.log.gz 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136554.log 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146730.log 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136554.log: /var/log/ceph/ceph-client.admin.146730.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136554.log.gz 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146730.log.gz 2023-09-30T22:35:42.264 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193743.log 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185710.log 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193743.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193743.log.gz 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185710.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185710.log.gz 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147220.log 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166401.log 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147220.log: /var/log/ceph/ceph-client.admin.166401.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147220.log.gz 2023-09-30T22:35:42.265 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166401.log.gz 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171058.log 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.171058.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.171058.log.gz 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133933.log 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr: 94.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.133933.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133933.log.gz 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127348.log 2023-09-30T22:35:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144084.log 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127348.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127348.log.gz 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144084.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144084.log.gz 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184728.log 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174891.log 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184728.log: /var/log/ceph/ceph-client.admin.174891.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.184728.log.gz 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174891.log.gz 2023-09-30T22:35:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164020.log 2023-09-30T22:35:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170911.log 2023-09-30T22:35:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164020.log: /var/log/ceph/ceph-client.admin.170911.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164020.log.gz 2023-09-30T22:35:42.269 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170911.log.gz 2023-09-30T22:35:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137928.log 2023-09-30T22:35:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178856.log 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137928.log: /var/log/ceph/ceph-client.admin.178856.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137928.log.gz 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.178856.log.gz 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183719.log 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169024.log 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183719.log: /var/log/ceph/ceph-client.admin.169024.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183719.log.gz 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.169024.log.gz 2023-09-30T22:35:42.270 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184652.log 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124132.log 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184652.log: /var/log/ceph/ceph-client.admin.124132.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184652.log.gz 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124132.log.gz 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165831.log 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165831.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165831.log.gz 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186419.log 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167530.log 2023-09-30T22:35:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.186419.log: 88.3%/var/log/ceph/ceph-client.admin.167530.log: -- replaced with /var/log/ceph/ceph-client.admin.186419.log.gz 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178807.log 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167530.log.gz 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193140.log 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178807.log: /var/log/ceph/ceph-client.admin.193140.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178807.log.gz 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.193140.log.gz 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152814.log 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146926.log 2023-09-30T22:35:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152814.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152814.log.gz 2023-09-30T22:35:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146926.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146926.log.gz 2023-09-30T22:35:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134078.log 2023-09-30T22:35:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153820.log 2023-09-30T22:35:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134078.log: /var/log/ceph/ceph-client.admin.153820.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134078.log.gz 2023-09-30T22:35:42.273 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.153820.log.gz 2023-09-30T22:35:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140574.log 2023-09-30T22:35:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181116.log 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140574.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140574.log.gz 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181116.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181116.log.gz 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192186.log 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167185.log 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192186.log.gz 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167185.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167185.log.gz 2023-09-30T22:35:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170004.log 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177584.log 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170004.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170004.log.gz 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177584.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177584.log.gz 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161199.log 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173119.log 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161199.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161199.log.gz 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173119.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173119.log.gz 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176603.log 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143228.log 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176603.log: /var/log/ceph/ceph-client.admin.143228.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176603.log.gz 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143228.log.gz 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129632.log 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186886.log 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129632.log: /var/log/ceph/ceph-client.admin.186886.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129632.log.gz 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.186886.log.gz 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160858.log 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160270.log 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160858.log: /var/log/ceph/ceph-client.admin.160270.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160858.log.gz 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.160270.log.gz 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127221.log 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169490.log 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127221.log: /var/log/ceph/ceph-client.admin.169490.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127221.log.gz 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169490.log.gz 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156636.log 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122431.log 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156636.log: /var/log/ceph/ceph-client.admin.122431.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156636.log.gz 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122431.log.gz 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168902.log 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137117.log 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168902.log: /var/log/ceph/ceph-client.admin.137117.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168902.log.gz 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137117.log.gz 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137536.log 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155656.log 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137536.log: /var/log/ceph/ceph-client.admin.155656.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137536.log.gz 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155656.log.gz 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123201.log 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161589.log 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123201.log: /var/log/ceph/ceph-client.admin.161589.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123201.log.gz 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161589.log.gz 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153918.log 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123003.log 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153918.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153918.log.gz 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123003.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123003.log.gz 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141268.log 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136602.log 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141268.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141268.log.gz/var/log/ceph/ceph-client.admin.136602.log: 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136602.log.gz 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155950.log 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146266.log 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155950.log: /var/log/ceph/ceph-client.admin.146266.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155950.log.gz 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.146266.log.gz 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152618.log 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122648.log 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152618.log: /var/log/ceph/ceph-client.admin.122648.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152618.log.gz 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122648.log.gz 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157250.log 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150978.log 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157250.log: /var/log/ceph/ceph-client.admin.150978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157250.log.gz 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.150978.log.gz 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191794.log 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148422.log 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191794.log: /var/log/ceph/ceph-client.admin.148422.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191794.log.gz 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148422.log.gz 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118553.log 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147710.log 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118553.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118553.log.gz/var/log/ceph/ceph-client.admin.147710.log: 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147710.log.gz 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151076.log 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160415.log 2023-09-30T22:35:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151076.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151076.log.gz 2023-09-30T22:35:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.160415.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160415.log.gz 2023-09-30T22:35:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159417.log 2023-09-30T22:35:42.291 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170963.log 2023-09-30T22:35:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159417.log: /var/log/ceph/ceph-client.admin.170963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159417.log.gz 2023-09-30T22:35:42.292 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.170963.log.gz 2023-09-30T22:35:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180720.log 2023-09-30T22:35:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184701.log 2023-09-30T22:35:42.292 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193841.log 2023-09-30T22:35:42.292 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180720.log.gz 2023-09-30T22:35:42.293 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184701.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184701.log.gz 2023-09-30T22:35:42.293 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193841.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193841.log.gz 2023-09-30T22:35:42.293 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151762.log 2023-09-30T22:35:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144868.log 2023-09-30T22:35:42.294 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.151762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189238.log 2023-09-30T22:35:42.294 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151762.log.gz 2023-09-30T22:35:42.295 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144868.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144868.log.gz 2023-09-30T22:35:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185362.log 2023-09-30T22:35:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178882.log 2023-09-30T22:35:42.297 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189238.log.gz 2023-09-30T22:35:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185362.log: /var/log/ceph/ceph-client.admin.178882.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.185362.log.gz 2023-09-30T22:35:42.297 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178882.log.gz 2023-09-30T22:35:42.297 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185757.log 2023-09-30T22:35:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175719.log 2023-09-30T22:35:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.185757.log: /var/log/ceph/ceph-client.admin.175719.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.185757.log.gz 2023-09-30T22:35:42.298 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175719.log.gz 2023-09-30T22:35:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188575.log 2023-09-30T22:35:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189800.log 2023-09-30T22:35:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188575.log: /var/log/ceph/ceph-client.admin.189800.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.188575.log.gz 2023-09-30T22:35:42.299 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.189800.log.gz 2023-09-30T22:35:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178072.log 2023-09-30T22:35:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172822.log 2023-09-30T22:35:42.300 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178072.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178072.log.gz 2023-09-30T22:35:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172822.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172822.log.gz 2023-09-30T22:35:42.301 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128191.log 2023-09-30T22:35:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156538.log 2023-09-30T22:35:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128191.log: /var/log/ceph/ceph-client.admin.156538.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128191.log.gz 2023-09-30T22:35:42.302 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156538.log.gz 2023-09-30T22:35:42.302 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155362.log 2023-09-30T22:35:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165366.log 2023-09-30T22:35:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155362.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155362.log.gz 2023-09-30T22:35:42.303 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165366.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165366.log.gz 2023-09-30T22:35:42.304 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167334.log 2023-09-30T22:35:42.304 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118761.log 2023-09-30T22:35:42.305 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.167334.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.167334.log.gz 2023-09-30T22:35:42.305 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.118761.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118761.log.gz 2023-09-30T22:35:42.305 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122549.log 2023-09-30T22:35:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176015.log 2023-09-30T22:35:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122549.log: /var/log/ceph/ceph-client.admin.176015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122549.log.gz 2023-09-30T22:35:42.306 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176015.log.gz 2023-09-30T22:35:42.306 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176307.log 2023-09-30T22:35:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151272.log 2023-09-30T22:35:42.307 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176307.log: /var/log/ceph/ceph-client.admin.151272.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.176307.log.gz 2023-09-30T22:35:42.307 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151272.log.gz 2023-09-30T22:35:42.308 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148716.log 2023-09-30T22:35:42.308 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139888.log 2023-09-30T22:35:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148716.log: /var/log/ceph/ceph-client.admin.139888.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148716.log.gz 2023-09-30T22:35:42.309 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139888.log.gz 2023-09-30T22:35:42.309 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187399.log 2023-09-30T22:35:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145678.log 2023-09-30T22:35:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187399.log: /var/log/ceph/ceph-client.admin.145678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.187399.log.gz 2023-09-30T22:35:42.310 INFO:teuthology.orchestra.run.smithi032.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.145678.log.gz 2023-09-30T22:35:42.310 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138392.log 2023-09-30T22:35:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142222.log 2023-09-30T22:35:42.311 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138392.log: /var/log/ceph/ceph-client.admin.142222.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.138392.log.gz 2023-09-30T22:35:42.311 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.142222.log.gz 2023-09-30T22:35:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183523.log 2023-09-30T22:35:42.312 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172505.log 2023-09-30T22:35:42.313 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183523.log: /var/log/ceph/ceph-client.admin.172505.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183523.log.gz 2023-09-30T22:35:42.313 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172505.log.gz 2023-09-30T22:35:42.313 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187987.log 2023-09-30T22:35:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176503.log 2023-09-30T22:35:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187987.log: /var/log/ceph/ceph-client.admin.176503.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.187987.log.gz 2023-09-30T22:35:42.314 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176503.log.gz 2023-09-30T22:35:42.314 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192578.log 2023-09-30T22:35:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185166.log 2023-09-30T22:35:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192578.log: /var/log/ceph/ceph-client.admin.185166.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.192578.log.gz 2023-09-30T22:35:42.315 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.185166.log.gz 2023-09-30T22:35:42.315 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122844.log 2023-09-30T22:35:42.316 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164386.log 2023-09-30T22:35:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122844.log: /var/log/ceph/ceph-client.admin.164386.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122844.log.gz 2023-09-30T22:35:42.317 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.164386.log.gz 2023-09-30T22:35:42.317 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181408.log 2023-09-30T22:35:42.318 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177484.log 2023-09-30T22:35:42.318 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181408.log: /var/log/ceph/ceph-client.admin.177484.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.181408.log.gz 2023-09-30T22:35:42.318 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177484.log.gz 2023-09-30T22:35:42.318 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170767.log 2023-09-30T22:35:42.319 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170470.log 2023-09-30T22:35:42.319 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170767.log: /var/log/ceph/ceph-client.admin.170470.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170767.log.gz 2023-09-30T22:35:42.319 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.170470.log.gz 2023-09-30T22:35:42.319 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136258.log 2023-09-30T22:35:42.320 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161785.log 2023-09-30T22:35:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136258.log: /var/log/ceph/ceph-client.admin.161785.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136258.log.gz 2023-09-30T22:35:42.321 INFO:teuthology.orchestra.run.smithi032.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161785.log.gz 2023-09-30T22:35:42.321 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157544.log 2023-09-30T22:35:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177388.log 2023-09-30T22:35:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.157544.log: /var/log/ceph/ceph-client.admin.177388.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.157544.log.gz 2023-09-30T22:35:42.322 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177388.log.gz 2023-09-30T22:35:42.322 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192800.log 2023-09-30T22:35:42.323 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145580.log 2023-09-30T22:35:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.192800.log: /var/log/ceph/ceph-client.admin.145580.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192800.log.gz 2023-09-30T22:35:42.324 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145580.log.gz 2023-09-30T22:35:42.324 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190192.log 2023-09-30T22:35:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183084.log 2023-09-30T22:35:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190192.log: /var/log/ceph/ceph-client.admin.183084.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.190192.log.gz 2023-09-30T22:35:42.325 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.183084.log.gz 2023-09-30T22:35:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128510.log 2023-09-30T22:35:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192699.log 2023-09-30T22:35:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128510.log: /var/log/ceph/ceph-client.admin.192699.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128510.log.gz 2023-09-30T22:35:42.326 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192699.log.gz 2023-09-30T22:35:42.326 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176799.log 2023-09-30T22:35:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183159.log 2023-09-30T22:35:42.327 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176799.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176799.log.gz 2023-09-30T22:35:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183159.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183159.log.gz 2023-09-30T22:35:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183002.log 2023-09-30T22:35:42.328 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142444.log 2023-09-30T22:35:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183002.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183002.log.gz 2023-09-30T22:35:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139666.log 2023-09-30T22:35:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142444.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142444.log.gz 2023-09-30T22:35:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169565.log 2023-09-30T22:35:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139666.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139666.log.gz 2023-09-30T22:35:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127143.log 2023-09-30T22:35:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169565.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169565.log.gz 2023-09-30T22:35:42.331 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135600.log 2023-09-30T22:35:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127143.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127143.log.gz 2023-09-30T22:35:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193890.log 2023-09-30T22:35:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180767.log 2023-09-30T22:35:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135600.log: /var/log/ceph/ceph-client.admin.193890.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135600.log.gz 2023-09-30T22:35:42.333 INFO:teuthology.orchestra.run.smithi032.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.193890.log.gz 2023-09-30T22:35:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143300.log 2023-09-30T22:35:42.333 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180767.log.gz 2023-09-30T22:35:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125232.log 2023-09-30T22:35:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143300.log: /var/log/ceph/ceph-client.admin.125232.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143300.log.gz 2023-09-30T22:35:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154016.log 2023-09-30T22:35:42.334 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125232.log.gz 2023-09-30T22:35:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136844.log 2023-09-30T22:35:42.335 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154016.log: /var/log/ceph/ceph-client.admin.136844.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.154016.log.gz 2023-09-30T22:35:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144966.log 2023-09-30T22:35:42.336 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136844.log.gz 2023-09-30T22:35:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144966.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144966.log.gz 2023-09-30T22:35:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177631.log 2023-09-30T22:35:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124014.log 2023-09-30T22:35:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.177631.log: /var/log/ceph/ceph-client.admin.124014.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177631.log.gz 2023-09-30T22:35:42.338 INFO:teuthology.orchestra.run.smithi032.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124014.log.gz 2023-09-30T22:35:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158114.log 2023-09-30T22:35:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158210.log 2023-09-30T22:35:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158114.log: /var/log/ceph/ceph-client.admin.158210.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.158114.log.gz 2023-09-30T22:35:42.339 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.158210.log.gz 2023-09-30T22:35:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154506.log 2023-09-30T22:35:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155976.log 2023-09-30T22:35:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154506.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154506.log.gz 2023-09-30T22:35:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155976.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155976.log.gz 2023-09-30T22:35:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181042.log 2023-09-30T22:35:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185121.log 2023-09-30T22:35:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181042.log: /var/log/ceph/ceph-client.admin.185121.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181042.log.gz 2023-09-30T22:35:42.342 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185121.log.gz 2023-09-30T22:35:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148886.log 2023-09-30T22:35:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137041.log 2023-09-30T22:35:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148886.log: /var/log/ceph/ceph-client.admin.137041.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148886.log.gz 2023-09-30T22:35:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138712.log 2023-09-30T22:35:42.343 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137041.log.gz 2023-09-30T22:35:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166453.log 2023-09-30T22:35:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.138712.log: /var/log/ceph/ceph-client.admin.166453.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138712.log.gz 2023-09-30T22:35:42.345 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166453.log.gz 2023-09-30T22:35:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136946.log 2023-09-30T22:35:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191376.log 2023-09-30T22:35:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136946.log: /var/log/ceph/ceph-client.admin.191376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136946.log.gz 2023-09-30T22:35:42.345 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.191376.log.gz 2023-09-30T22:35:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178023.log 2023-09-30T22:35:42.346 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172871.log 2023-09-30T22:35:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178023.log: /var/log/ceph/ceph-client.admin.172871.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178023.log.gz 2023-09-30T22:35:42.347 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172871.log.gz 2023-09-30T22:35:42.347 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158532.log 2023-09-30T22:35:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173485.log 2023-09-30T22:35:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.158532.log: /var/log/ceph/ceph-client.admin.173485.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158532.log.gz 2023-09-30T22:35:42.348 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173485.log.gz 2023-09-30T22:35:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174439.log 2023-09-30T22:35:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147416.log 2023-09-30T22:35:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174439.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174439.log.gz 2023-09-30T22:35:42.349 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147416.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147416.log.gz 2023-09-30T22:35:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163890.log 2023-09-30T22:35:42.350 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186149.log 2023-09-30T22:35:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.163890.log: /var/log/ceph/ceph-client.admin.186149.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163890.log.gz 2023-09-30T22:35:42.351 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186149.log.gz 2023-09-30T22:35:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155682.log 2023-09-30T22:35:42.351 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135404.log 2023-09-30T22:35:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155682.log: /var/log/ceph/ceph-client.admin.135404.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155682.log.gz 2023-09-30T22:35:42.352 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191225.log 2023-09-30T22:35:42.352 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135404.log.gz 2023-09-30T22:35:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193498.log 2023-09-30T22:35:42.353 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191225.log: /var/log/ceph/ceph-client.admin.193498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.191225.log.gz 2023-09-30T22:35:42.354 INFO:teuthology.orchestra.run.smithi032.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.193498.log.gz 2023-09-30T22:35:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174269.log 2023-09-30T22:35:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167039.log 2023-09-30T22:35:42.354 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174269.log: /var/log/ceph/ceph-client.admin.167039.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174269.log.gz 2023-09-30T22:35:42.354 INFO:teuthology.orchestra.run.smithi032.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167039.log.gz 2023-09-30T22:35:42.355 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181016.log 2023-09-30T22:35:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164263.log 2023-09-30T22:35:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181016.log: /var/log/ceph/ceph-client.admin.164263.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.181016.log.gz 2023-09-30T22:35:42.356 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164263.log.gz 2023-09-30T22:35:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159290.log 2023-09-30T22:35:42.356 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167802.log 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159290.log: /var/log/ceph/ceph-client.admin.167802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159290.log.gz 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.167802.log.gz 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170349.log 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180375.log 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170349.log: /var/log/ceph/ceph-client.admin.180375.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170349.log.gz 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180375.log.gz 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140646.log 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144574.log 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193547.log 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140646.log.gz 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144574.log: /var/log/ceph/ceph-client.admin.193547.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144574.log.gz 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132924.log 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193547.log.gz 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138784.log 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.132924.log: /var/log/ceph/ceph-client.admin.138784.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132924.log.gz 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138784.log.gz 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169343.log 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170127.log 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169343.log: /var/log/ceph/ceph-client.admin.170127.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169343.log.gz 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170127.log.gz 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147834.log 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171303.log 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.147834.log: /var/log/ceph/ceph-client.admin.171303.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147834.log.gz 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145946.log 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171303.log.gz 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145946.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145946.log.gz 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142418.log 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148128.log 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.142418.log: /var/log/ceph/ceph-client.admin.148128.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.142418.log.gz 2023-09-30T22:35:42.369 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148128.log.gz 2023-09-30T22:35:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141464.log 2023-09-30T22:35:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160364.log 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141464.log: /var/log/ceph/ceph-client.admin.160364.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141464.log.gz 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160364.log.gz 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171133.log 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181751.log 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171133.log: /var/log/ceph/ceph-client.admin.181751.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171133.log.gz 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181751.log.gz 2023-09-30T22:35:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144770.log 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149998.log 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.144770.log: /var/log/ceph/ceph-client.admin.149998.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144770.log.gz 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149998.log.gz 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184970.log 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133662.log 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184970.log: /var/log/ceph/ceph-client.admin.133662.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184970.log.gz 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189604.log 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133662.log.gz 2023-09-30T22:35:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153010.log 2023-09-30T22:35:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189604.log: /var/log/ceph/ceph-client.admin.153010.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.189604.log.gz 2023-09-30T22:35:42.374 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153010.log.gz 2023-09-30T22:35:42.374 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150854.log 2023-09-30T22:35:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153794.log 2023-09-30T22:35:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150854.log: /var/log/ceph/ceph-client.admin.153794.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.150854.log.gz 2023-09-30T22:35:42.376 INFO:teuthology.orchestra.run.smithi032.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153794.log.gz 2023-09-30T22:35:42.376 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128363.log 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160466.log 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128363.log: /var/log/ceph/ceph-client.admin.160466.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128363.log.gz 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160466.log.gz 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172479.log 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143594.log 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172479.log: /var/log/ceph/ceph-client.admin.143594.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172479.log.gz 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143594.log.gz 2023-09-30T22:35:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180967.log 2023-09-30T22:35:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146364.log 2023-09-30T22:35:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180967.log: /var/log/ceph/ceph-client.admin.146364.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180967.log.gz 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.146364.log.gz 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128412.log 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127169.log 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.128412.log: /var/log/ceph/ceph-client.admin.127169.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128412.log.gz 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127169.log.gz 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161250.log 2023-09-30T22:35:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150782.log 2023-09-30T22:35:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161250.log: /var/log/ceph/ceph-client.admin.150782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161250.log.gz 2023-09-30T22:35:42.381 INFO:teuthology.orchestra.run.smithi032.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.150782.log.gz 2023-09-30T22:35:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169686.log 2023-09-30T22:35:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183598.log 2023-09-30T22:35:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.169686.log: /var/log/ceph/ceph-client.admin.183598.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169686.log.gz 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.183598.log.gz 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155388.log 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144502.log 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155388.log: /var/log/ceph/ceph-client.admin.144502.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155388.log.gz 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144502.log.gz 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123766.log 2023-09-30T22:35:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155486.log 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123766.log: 88.6%/var/log/ceph/ceph-client.admin.155486.log: -- replaced with /var/log/ceph/ceph-client.admin.123766.log.gz 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.155486.log.gz 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152252.log 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126817.log 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152252.log: /var/log/ceph/ceph-client.admin.126817.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.152252.log.gz 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.126817.log.gz 2023-09-30T22:35:42.385 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141242.log 2023-09-30T22:35:42.386 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178464.log 2023-09-30T22:35:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.141242.log: /var/log/ceph/ceph-client.admin.178464.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141242.log.gz 2023-09-30T22:35:42.387 INFO:teuthology.orchestra.run.smithi032.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.178464.log.gz 2023-09-30T22:35:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171254.log 2023-09-30T22:35:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178172.log 2023-09-30T22:35:42.387 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148323.log 2023-09-30T22:35:42.388 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171254.log.gz 2023-09-30T22:35:42.388 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137238.log 2023-09-30T22:35:42.388 INFO:teuthology.orchestra.run.smithi032.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.178172.log.gz 2023-09-30T22:35:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148323.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148323.log.gz 2023-09-30T22:35:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137238.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137238.log.gz 2023-09-30T22:35:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168988.log 2023-09-30T22:35:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177043.log 2023-09-30T22:35:42.389 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168988.log: /var/log/ceph/ceph-client.admin.177043.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168988.log.gz 2023-09-30T22:35:42.390 INFO:teuthology.orchestra.run.smithi032.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177043.log.gz 2023-09-30T22:35:42.390 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184505.log 2023-09-30T22:35:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186102.log 2023-09-30T22:35:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184505.log: /var/log/ceph/ceph-client.admin.186102.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184505.log.gz 2023-09-30T22:35:42.391 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.186102.log.gz 2023-09-30T22:35:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122895.log 2023-09-30T22:35:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124181.log 2023-09-30T22:35:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.122895.log: /var/log/ceph/ceph-client.admin.124181.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122895.log.gz 2023-09-30T22:35:42.392 INFO:teuthology.orchestra.run.smithi032.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124181.log.gz 2023-09-30T22:35:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171891.log 2023-09-30T22:35:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157028.log 2023-09-30T22:35:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.171891.log: /var/log/ceph/ceph-client.admin.157028.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.171891.log.gz 2023-09-30T22:35:42.393 INFO:teuthology.orchestra.run.smithi032.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.157028.log.gz 2023-09-30T22:35:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188771.log 2023-09-30T22:35:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-09-30T22:35:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.188771.log: /var/log/ceph/ceph-osd.1.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188771.log.gz 2023-09-30T22:35:42.395 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152154.log 2023-09-30T22:35:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152154.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152154.log.gz 2023-09-30T22:35:42.396 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124622.log 2023-09-30T22:35:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124622.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124622.log.gz 2023-09-30T22:35:42.397 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124818.log 2023-09-30T22:35:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124818.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124818.log.gz 2023-09-30T22:35:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154382.log 2023-09-30T22:35:42.400 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154382.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.154382.log.gz 2023-09-30T22:35:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156172.log 2023-09-30T22:35:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.156172.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156172.log.gz 2023-09-30T22:35:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161080.log 2023-09-30T22:35:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.161080.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161080.log.gz 2023-09-30T22:35:42.403 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154408.log 2023-09-30T22:35:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.154408.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154408.log.gz 2023-09-30T22:35:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193596.log 2023-09-30T22:35:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.193596.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193596.log.gz 2023-09-30T22:35:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140182.log 2023-09-30T22:35:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.140182.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140182.log.gz 2023-09-30T22:35:42.406 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164412.log 2023-09-30T22:35:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164412.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164412.log.gz 2023-09-30T22:35:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166649.log 2023-09-30T22:35:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166649.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166649.log.gz 2023-09-30T22:35:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129309.log 2023-09-30T22:35:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.129309.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129309.log.gz 2023-09-30T22:35:42.410 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146240.log 2023-09-30T22:35:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146240.log.gz 2023-09-30T22:35:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176258.log 2023-09-30T22:35:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176258.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176258.log.gz 2023-09-30T22:35:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174386.log 2023-09-30T22:35:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174386.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174386.log.gz 2023-09-30T22:35:42.413 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190048.log 2023-09-30T22:35:42.416 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190048.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.190048.log.gz 2023-09-30T22:35:42.416 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146044.log 2023-09-30T22:35:42.417 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.146044.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146044.log.gz 2023-09-30T22:35:42.417 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176846.log 2023-09-30T22:35:42.418 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176846.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176846.log.gz 2023-09-30T22:35:42.418 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189462.log 2023-09-30T22:35:42.419 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.189462.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189462.log.gz 2023-09-30T22:35:42.420 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149874.log 2023-09-30T22:35:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149874.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149874.log.gz 2023-09-30T22:35:42.421 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150560.log 2023-09-30T22:35:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.150560.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.150560.log.gz 2023-09-30T22:35:42.422 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184186.log 2023-09-30T22:35:42.423 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184186.log.gz 2023-09-30T22:35:42.423 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148004.log 2023-09-30T22:35:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.148004.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148004.log.gz 2023-09-30T22:35:42.424 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124475.log 2023-09-30T22:35:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124475.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124475.log.gz 2023-09-30T22:35:42.425 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179470.log 2023-09-30T22:35:42.426 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.179470.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179470.log.gz 2023-09-30T22:35:42.427 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135378.log 2023-09-30T22:35:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.135378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135378.log.gz 2023-09-30T22:35:42.428 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191572.log 2023-09-30T22:35:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191572.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.191572.log.gz 2023-09-30T22:35:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182731.log 2023-09-30T22:35:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182731.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.182731.log.gz 2023-09-30T22:35:42.430 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126987.log 2023-09-30T22:35:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.126987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126987.log.gz 2023-09-30T22:35:42.432 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123622.log 2023-09-30T22:35:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.123622.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123622.log.gz 2023-09-30T22:35:42.433 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153206.log 2023-09-30T22:35:42.434 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153206.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.153206.log.gz 2023-09-30T22:35:42.434 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175278.log 2023-09-30T22:35:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175278.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175278.log.gz 2023-09-30T22:35:42.435 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168436.log 2023-09-30T22:35:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.168436.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168436.log.gz 2023-09-30T22:35:42.436 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152938.log 2023-09-30T22:35:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152938.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152938.log.gz 2023-09-30T22:35:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152056.log 2023-09-30T22:35:42.437 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182022.log 2023-09-30T22:35:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.152056.log: /var/log/ceph/ceph-client.admin.182022.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.152056.log.gz 2023-09-30T22:35:42.438 INFO:teuthology.orchestra.run.smithi032.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.182022.log.gz 2023-09-30T22:35:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187448.log 2023-09-30T22:35:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187448.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.187448.log.gz 2023-09-30T22:35:42.439 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134398.log 2023-09-30T22:35:42.440 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.134398.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.134398.log.gz 2023-09-30T22:35:42.440 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165662.log 2023-09-30T22:35:42.441 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.165662.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165662.log.gz 2023-09-30T22:35:42.441 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137388.log 2023-09-30T22:35:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137388.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137388.log.gz 2023-09-30T22:35:42.442 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137143.log 2023-09-30T22:35:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137143.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137143.log.gz 2023-09-30T22:35:42.443 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127553.log 2023-09-30T22:35:42.444 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146632.log 2023-09-30T22:35:42.444 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.127553.log: /var/log/ceph/ceph-client.admin.146632.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127553.log.gz 2023-09-30T22:35:42.445 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146632.log.gz 2023-09-30T22:35:42.445 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159631.log 2023-09-30T22:35:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.159631.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159631.log.gz 2023-09-30T22:35:42.446 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190636.log 2023-09-30T22:35:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190636.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190636.log.gz 2023-09-30T22:35:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187644.log 2023-09-30T22:35:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187644.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.187644.log.gz 2023-09-30T22:35:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172923.log 2023-09-30T22:35:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172923.log.gz 2023-09-30T22:35:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155780.log 2023-09-30T22:35:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155780.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155780.log.gz 2023-09-30T22:35:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145874.log 2023-09-30T22:35:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.145874.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145874.log.gz 2023-09-30T22:35:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187056.log 2023-09-30T22:35:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.187056.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.187056.log.gz 2023-09-30T22:35:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191820.log 2023-09-30T22:35:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.191820.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191820.log.gz 2023-09-30T22:35:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162353.log 2023-09-30T22:35:42.454 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.162353.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162353.log.gz 2023-09-30T22:35:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183762.log 2023-09-30T22:35:42.456 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183762.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.183762.log.gz 2023-09-30T22:35:42.456 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180228.log 2023-09-30T22:35:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.180228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180228.log.gz 2023-09-30T22:35:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124916.log 2023-09-30T22:35:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.124916.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124916.log.gz 2023-09-30T22:35:42.458 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190339.log 2023-09-30T22:35:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190339.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190339.log.gz 2023-09-30T22:35:42.459 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176995.log 2023-09-30T22:35:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.176995.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176995.log.gz 2023-09-30T22:35:42.460 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164683.log 2023-09-30T22:35:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164683.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164683.log.gz 2023-09-30T22:35:42.461 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184338.log 2023-09-30T22:35:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184338.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184338.log.gz 2023-09-30T22:35:42.462 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190780.log 2023-09-30T22:35:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.190780.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.190780.log.gz 2023-09-30T22:35:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183794.log 2023-09-30T22:35:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168784.log 2023-09-30T22:35:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.183794.log: /var/log/ceph/ceph-client.admin.168784.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183794.log.gz 2023-09-30T22:35:42.465 INFO:teuthology.orchestra.run.smithi032.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168784.log.gz 2023-09-30T22:35:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184112.log 2023-09-30T22:35:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184112.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.184112.log.gz 2023-09-30T22:35:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153108.log 2023-09-30T22:35:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.153108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153108.log.gz 2023-09-30T22:35:42.467 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178686.log 2023-09-30T22:35:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.178686.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178686.log.gz 2023-09-30T22:35:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174243.log 2023-09-30T22:35:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.174243.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174243.log.gz 2023-09-30T22:35:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184064.log 2023-09-30T22:35:42.474 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.184064.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.184064.log.gz 2023-09-30T22:35:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166746.log 2023-09-30T22:35:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166746.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166746.log.gz 2023-09-30T22:35:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175549.log 2023-09-30T22:35:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.175549.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175549.log.gz 2023-09-30T22:35:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181630.log 2023-09-30T22:35:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.181630.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181630.log.gz 2023-09-30T22:35:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137975.log 2023-09-30T22:35:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.137975.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137975.log.gz 2023-09-30T22:35:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155192.log 2023-09-30T22:35:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.155192.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155192.log.gz 2023-09-30T22:35:42.478 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136528.log 2023-09-30T22:35:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.136528.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136528.log.gz 2023-09-30T22:35:42.479 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172234.log 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.172234.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172234.log.gz 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149482.log 2023-09-30T22:35:42.481 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.149482.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149482.log.gz 2023-09-30T22:35:42.481 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139078.log 2023-09-30T22:35:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139078.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139078.log.gz 2023-09-30T22:35:42.482 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182386.log 2023-09-30T22:35:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.182386.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.182386.log.gz 2023-09-30T22:35:42.483 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170666.log 2023-09-30T22:35:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.170666.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170666.log.gz 2023-09-30T22:35:42.484 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139470.log 2023-09-30T22:35:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.139470.log: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-09-30T22:35:42.486 INFO:teuthology.orchestra.run.smithi032.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.139470.log.gz 2023-09-30T22:35:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164974.log 2023-09-30T22:35:42.486 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138810.log 2023-09-30T22:35:42.487 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.164974.log: /var/log/ceph/ceph-client.admin.138810.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.164974.log.gz 2023-09-30T22:35:42.487 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138810.log.gz 2023-09-30T22:35:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166231.log 2023-09-30T22:35:42.488 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143496.log 2023-09-30T22:35:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.166231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143816.log 2023-09-30T22:35:42.489 INFO:teuthology.orchestra.run.smithi032.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166231.log.gz 2023-09-30T22:35:42.489 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143496.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143496.log.gz 2023-09-30T22:35:42.490 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.143816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173018.log 2023-09-30T22:35:42.491 INFO:teuthology.orchestra.run.smithi032.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143816.log.gz 2023-09-30T22:35:42.491 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/ceph-client.admin.173018.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.173018.log.gz 2023-09-30T22:35:42.512 INFO:teuthology.orchestra.run.smithi032.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-09-30T22:35:42.610 INFO:teuthology.orchestra.run.smithi032.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-09-30T22:35:42.618 INFO:teuthology.orchestra.run.smithi032.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-09-30T22:35:42.780 INFO:teuthology.orchestra.run.smithi032.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-09-30T22:35:42.858 INFO:teuthology.orchestra.run.smithi032.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-09-30T22:35:42.860 INFO:teuthology.orchestra.run.smithi032.stderr: 2023-09-30T22:35:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m1.135s 2023-09-30T22:35:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m4.278s 2023-09-30T22:35:42.860 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.978s 2023-09-30T22:35:42.860 INFO:tasks.ceph:Archiving logs... 2023-09-30T22:35:42.861 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/remote/smithi032/log 2023-09-30T22:35:42.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T22:35:44.525 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/remote/smithi121/log 2023-09-30T22:35:44.526 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T22:35:44.561 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:35:44.571 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-30T22:35:44.589 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:35:44.593 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-30T22:35:44.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T22:35:44.599 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T22:35:44.668 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-30T22:35:44.679 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-30T22:35:44.680 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:35:44.682 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:35:44.701 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2023-09-30T22:35:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2023-09-30T22:35:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:35:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2023-09-30T22:35:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 -20us[ -22us] +/- 75ms 2023-09-30T22:35:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -420us[ -422us] +/- 56ms 2023-09-30T22:35:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +785us[ +783us] +/- 90ms 2023-09-30T22:35:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:35:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:35:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2023-09-30T22:35:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 -46us[ -34us] +/- 75ms 2023-09-30T22:35:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -445us[ -434us] +/- 56ms 2023-09-30T22:35:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +760us[ +760us] +/- 90ms 2023-09-30T22:35:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:35:44.709 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-30T22:35:44.721 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-30T22:35:44.721 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-30T22:35:44.735 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T22:35:44.807 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T22:35:44.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:35:44.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:35:44.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:35:44.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:35:44.970 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-30T22:35:44.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-30T22:35:45.005 INFO:teuthology.task.internal:Duration was 1517.022245 seconds 2023-09-30T22:35:45.006 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-30T22:35:45.017 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-30T22:35:45.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T22:35:45.020 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:35:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:35:45.476 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-30T22:35:45.476 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-09-30T22:35:45.476 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T22:35:45.516 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2023-09-30T22:35:45.517 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T22:35:45.549 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-30T22:35:45.549 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T22:35:45.571 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T22:35:45.642 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-30T22:35:45.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:35:45.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:35:45.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-30T22:35:45.787 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-30T22:35:45.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T22:35:45.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T22:35:45.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-30T22:35:45.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:35:45.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:35:45.924 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2023-09-30T22:35:45.928 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-09-30T22:35:45.970 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:35:46.036 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi032.front.sepia.ceph.com, flagging run as failed 2023-09-30T22:35:46.037 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:35:46.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:35:46.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-30T22:35:46.086 INFO:teuthology.task.internal:Transferring archived files... 2023-09-30T22:35:46.087 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/remote/smithi032 2023-09-30T22:35:46.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T22:37:15.400 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-09-30T22:37:15.410 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042/remote/smithi032/coredump/1696112663.126553.core.gz: gzip compressed data, was "1696112663.126553.core", last modified: Sat Sep 30 22:35:46 2023, max compression, original size modulo 2^32 1365299200 2023-09-30T22:37:15.411 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-09-30T22:37:21.168 INFO:teuthology.task.internal: core looks like: /tmp/tmp7d5yxp_x: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mgr -f --cluster ceph -i x', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mgr', platform: 'x86_64' 2023-09-30T22:37:21.312 INFO:teuthology.task.internal: dump_program: ceph-mgr -f --cluster ceph -i x 2023-09-30T22:37:21.312 DEBUG:teuthology.orchestra.run.smithi032:> which 'ceph-mgr -f --cluster ceph -i x' 2023-09-30T22:37:21.337 INFO:teuthology.orchestra.run.smithi032.stderr:/usr/bin/which: invalid option -- 'f' 2023-09-30T22:37:21.337 INFO:teuthology.orchestra.run.smithi032.stderr:/usr/bin/which: unrecognized option '--cluster' 2023-09-30T22:37:21.337 INFO:teuthology.orchestra.run.smithi032.stderr:/usr/bin/which: no x in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2023-09-30T22:37:21.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-mgr 2023-09-30T22:37:21.339 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph 2023-09-30T22:37:21.339 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 1: "which 'ceph-mgr -f --cluster ceph -i x'" 2023-09-30T22:37:21.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-30T22:37:21.373 INFO:teuthology.task.internal:Not uploading archives. 2023-09-30T22:37:21.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-30T22:37:21.393 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-30T22:37:21.393 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T22:37:21.418 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T22:37:21.431 INFO:teuthology.orchestra.run.smithi032.stdout: 525357 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest 2023-09-30T22:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: 525358 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest/archive 2023-09-30T22:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: 525359 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:24 /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: 525598 362168 -rw------- 1 root root 1365299200 Sep 30 22:24 /home/ubuntu/cephtest/archive/coredump/1696112663.126553.core 2023-09-30T22:37:21.432 INFO:teuthology.orchestra.run.smithi032.stdout: 655784 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest/archive/audit 2023-09-30T22:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: 655822 28 -rw------- 1 ubuntu ubuntu 25470 Sep 30 22:35 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-09-30T22:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: 525360 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: 525482 4 -rw-rw-rw- 1 ubuntu ubuntu 765 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-09-30T22:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: 525362 112 -rw-rw-r-- 1 ubuntu ubuntu 113608 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:37:21.433 INFO:teuthology.orchestra.run.smithi032.stdout: 525361 132 -rw-rw-rw- 1 ubuntu ubuntu 133702 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-09-30T22:37:21.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:37:21.436 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-30T22:37:21.436 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-09-30T22:37:21.449 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-30T22:37:21.464 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-30T22:37:21.478 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:37:21.478 INFO:teuthology.nuke:Checking targets against current locks 2023-09-30T22:37:21.486 INFO:teuthology.orchestra.run.smithi121.stdout: 525357 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest 2023-09-30T22:37:21.487 INFO:teuthology.orchestra.run.smithi121.stdout: 525358 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest/archive 2023-09-30T22:37:21.487 INFO:teuthology.orchestra.run.smithi121.stdout: 655784 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest/archive/audit 2023-09-30T22:37:21.488 INFO:teuthology.orchestra.run.smithi121.stdout: 655786 220 -rw------- 1 ubuntu ubuntu 223800 Sep 30 22:35 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-09-30T22:37:21.488 INFO:teuthology.orchestra.run.smithi121.stdout: 525360 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:37:21.488 INFO:teuthology.orchestra.run.smithi121.stdout: 525482 4 -rw-rw-rw- 1 ubuntu ubuntu 892 Sep 30 22:33 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-09-30T22:37:21.488 INFO:teuthology.orchestra.run.smithi121.stdout: 525362 92 -rw-rw-r-- 1 ubuntu ubuntu 92107 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:37:21.489 INFO:teuthology.orchestra.run.smithi121.stdout: 525361 88 -rw-rw-rw- 1 ubuntu ubuntu 86330 Sep 30 22:35 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-09-30T22:37:21.490 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-30T22:37:21.516 DEBUG:teuthology.nuke:shortname: smithi032 2023-09-30T22:37:21.516 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:37:21.540 DEBUG:teuthology.nuke:shortname: smithi121 2023-09-30T22:37:21.541 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:37:21.544 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:02:23.993557', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGWsMKYuwsdGXjp6h34gZzBBh7PMayUXXTdEL6+J6Wa8ik43W5zzZGraVOj6EMsemGb/p2oTMkpaEPWxgLu8C6s='} 2023-09-30T22:37:21.570 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:02:23.991688', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWwuWg1cgtHrdcSkqy8HMifSeAJRJaas2j9YYMDpI0uJcMS/XGWzDLabQwiCS8c2GHvWxSIgIcSkmi4OXt3cFU='} 2023-09-30T22:37:21.596 INFO:teuthology.orchestra.console.smithi032:Power off 2023-09-30T22:37:21.597 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T22:37:21.621 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-09-30T22:37:21.621 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:21.641 INFO:teuthology.orchestra.console.smithi121:Power off 2023-09-30T22:37:21.641 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T22:37:21.651 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-09-30T22:37:21.662 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2023-09-30T22:37:21.663 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:21.688 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2023-09-30T22:37:25.652 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:25.690 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:25.763 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-09-30T22:37:25.800 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2023-09-30T22:37:29.766 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:29.801 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:29.877 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-09-30T22:37:29.912 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2023-09-30T22:37:29.979 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-09-30T22:37:33.914 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:37:34.025 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2023-09-30T22:37:34.126 INFO:teuthology.orchestra.console.smithi121:Power off completed 2023-09-30T22:37:34.375 INFO:teuthology.run:Summary data: description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}} duration: 1517.022245168686 failure_reason: 'Command failed on smithi032 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json''' flavor: default owner: scheduled_dparmar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c1bc4302dbac483884ac48e094f5ffae status: fail success: false 2023-09-30T22:37:34.376 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T22:37:34.443 INFO:teuthology.run:FAIL